Quantitative Argumentation Debates with Votes for Opinion Polling

Abstract

Opinion polls are used in a variety of settings to assess the opinions of a population, but they mostly conceal the reasoning behind these opinions. Argumentation, as understood in AI, can be used to evaluate opinions in dialectical exchanges, transparently articulating the reasoning behind the opinions. We give a method integrating argumentation within opinion polling to empower voters to add new statements that render their opinions in the polls individually rational while at the same time justifying them. We then show how these poll results can be amalgamated to give a collectively rational set of voters in an argumentation framework. Our method relies upon Quantitative Argumentation Debate for Voting (QuAD-V) frameworks, which extend QuAD frameworks (a form of bipolar argumentation frameworks in which arguments have an intrinsic strength) with votes expressing individuals’ opinions on arguments.

1 Introduction

Two of the main aims of e-Democracy are to move from a representative to a direct democracy, shifting power to citizens, and to facilitate the necessary deliberations for direct democracy to function effectively [13]. These aims are shared by existing concepts of democracy, such as Agonistic Pluralism [19], which accepts and encourages conflicts on policy, and Deliberative Democracy [2], which allows the resolution of conflicts using voting if a rational consensus is not reached.

Voting is also core in opinion polling, a method for both obtaining information on people’s sentiment and engaging them in the political process in a bottom up manner. In conventional opinion poll systems, a prominent example of which is YouGov ¹, questions are put to users in a flat list format. More engaging user interfaces, as shown on the WhichIt platform², can be used, as well as the reverse wording of questions to ensure that responses are valid. Some systems, e.g. [18, 23, 24], integrate opinion polling with other techniques or systems, e.g. Twitter [23] or machine learning algorithms [18]. Moreover, Deliberative Polling [10] is a fully-fledged system for decision-making based on deliberation, incorporating aspects of deliberative democracy, e.g. samples of the users in the poll are given balanced information and are invited to deliberate with one another to improve the quality of the responses. To the best of our knowledge, no existing opinion polling system/method takes into account evaluation of the dialectical strength of the opinions based on voters’ responses.

Argumentation, as understood in AI [22], can be used to evaluate the strength of opinions in dialectical exchanges, transparently articulating the reasoning behind them, when these exchanges are represented as argumentation frameworks. The simplest among such frameworks are Abstract Argumentation frameworks (AAFs), defined in terms of arguments and an attack relation between them [8], whereas Bipolar Argumentation frameworks (BAFs) [6] also include a support relation between arguments, and Quantitative Argumentation Debate (QuAD) frameworks [1], based on the IBIS methodology [15], distinguish answer, pro and con arguments and ascribe intrinsic strengths to arguments prior to debates. All frameworks are equipped with methods for evaluating the dialectical acceptability or strength of arguments.

Several argumentation frameworks have already been used to support collaborative debates and deliberation within e-Democracy or otherwise (e.g. see [3, 5, 7, 12, 14, 16, 17, 20]). We propose QuAD for Voting (QuAD-V) frameworks and use them to support a novel, arguably more informative form of opinion polling in the spirit of deliberative democracy. Our QuAD-V opinion polling allows voters to provide information about the reasoning behind their opinions, while dynamically expanding the originally specified polls by eliciting information from users. The elicitation is driven by the semantic evaluation of voters’ opinions, using a suitable notion of strength of arguments for QuAD-V frameworks that we define, instantiating the notion in [21]. The elicitation aims at rendering the opinions of the voters (i.e. their arguments and votes) individually and collectively rational.

This paper is organised as follows. In Sect. 2 we give a motivating example for our approach. In Sect. 3 we give necessary background on QuAD frameworks, the starting point for our approach. In Sect. 4 we define QuAD-V frameworks, and in Sect. 5 we study their properties. In Sect. 6 we discuss the convention we use to class voters as individually/collectively rational. In Sect. 7 we describe our opinion poll method, based on QuAD-V frameworks, and in Sect. 8 we conclude.

2 Motivation

To illustrate the motivation for this paper we look at two recent examples of political debate, “Brexit”, the recent referendum on the UK exit from the European Union, and the US 2016 Presidential Election. In both examples, opinion polling failed to accurately predict the results of the voting^{3 , 4} and the voters on the winning side felt that their voices were not being heard⁵. Many of the fundamental issues with the polling were related to statistical and sampling errors, but one of significance was voters being disingenuous or not fully expressing their opinions in the polls⁶. Improved sharing of this information could be achieved by more informed debates, rather than false promises, negative campaigning and scaremongering^{7 , 8}, which led to many voters expressing regret after voting under what they felt were false pretences⁹. We aim to address both disingenuous behaviour and disengagement of the public by developing a novel argumentation-based methodology supporting debating and voting in opinion polling to help ensure that information is shared and voters are more engaged.

In conventional opinion polls, users are asked to state (or grade) their agreement on statements by votes, e.g. users’ votes may amount to agree, neutral or disagree. The aggregation of users’ votes allows pollsters to obtain statistics on public agreement on issues the statements refer to. However, these methods ignore the relationships between statements and users’ votes on statements have no bearing on their votes on related statements. Thus, the reasoning that may result from analysing users’ votes given these relationships is neglected and opinion polls may disregard “irrationalities” in the voter’s opinions.

For example, consider the following statements relating to the Brexit debate:

• $S1$ - The UK should leave the EU.

• $S2$ - The UK staying in the EU is good for its economy.

• $S3$ - The EU’s immigration policies are bad for the UK.

• $S4$ - EU membership fees are too high.

Here, $S2$ may be deemed to attack $S1$, while $S3$ and $S4$ may be deemed to support it (where attack and support are dialectical relationships). So, if a user’s votes indicate disagreement with $S1$, $S2$ and $S4$ but agreement with $S3$, the user may be disingenuous (hiding that she/he actually agrees with $S1$, but giving it away by agreeing with one of its supporters) or the poll may not provide sufficiently many statements to fully reflect the voter’s opinions, e.g. the user may agree with some other argument (statement) attacking $S1$, such as:

• $S5$ - The UK staying in the EU is good for world peace.

In both cases, we may deem the voter’s opinions to be irrational.

Our opinion polling method interprets statements in opinion polls as arguments in a type of argumentation framework that we define. Moreover, it uses a measure of strength of arguments, based on both the direct votes on the statements/arguments and the indirect votes on their (dialectically) related statements/arguments. It then uses this measure to highlight voting that may be deemed as irrational and then gives voters the opportunity to become “rational” by dynamic transformations of the underlying argumentation framework. This information elicitation obtains additional data for the opinion poll while at the same time increasing engagement of its voters.

3 Background

As introduced in [1], a Quantitative Argumentation Debate (QuAD) framework is a 5-tuple $\langle \mathcal {A}, \mathcal {C}, \mathcal {P}, \mathcal {R}, \tau \rangle$ such that $\mathcal {A}$ is a finite set of answer arguments; $\mathcal {C}$ is a finite set of con arguments; $\mathcal {P}$ is a finite set of pro arguments; the sets $\mathcal {A}$, $\mathcal {C}$ and $\mathcal {P}$ are pairwise disjoint¹⁰; $\mathcal {R}\subseteq (\mathcal {C}\cup \mathcal {P}) \times (\mathcal {A}\cup \mathcal {C}\cup \mathcal {P})$ is an acyclic binary relation; for $\mathbb {I}\,=\,[0,1]$, $\tau : (\mathcal {A}\cup \mathcal {C}\cup \mathcal {P}) \rightarrow \mathbb {I}$ is a total function: $\tau (a)$ is the base score of a, representing its intrinsic strength, prior to considering other arguments dialectically related to it. The Brexit debate from Sect. 2 can be represented as a QuAD framework $\langle \{S1\}, {\{S2,S5\}, \{S3,S4\},} \{(S2,S1),(S3,S1),(S4,S1), (S5,S1)\}, \tau \rangle$, for any suitable $\tau$. The relation component can be visualised as in Fig. 1.

Fig. 1.

Example QuAD framework

Pro and con arguments determine the attackers and supporters of arguments they are in relation with. Formally, for any argument $a \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$, the set of attackers of a is $\mathcal {R}^{-}(a)=\lbrace b \in \mathcal {C}| (b,a) \in \mathcal {R}\rbrace$ and the set of supporters of a is $\mathcal {R}^{+}(a)=\lbrace b \in \mathcal {P}| (b,a) \in \mathcal {R}\rbrace$.

Due to the acyclicity requirement, QuAD frameworks amount to sets of trees and each argument is the root of a (sub-)tree. For any argument $a \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$, we will use $T_a$ to denote the tree with root a such that, for any node b in $T_a$, the children of b are the arguments in $\mathcal {R}^{-}(b) \cup \mathcal {R}^{+}(b)$.

The Discontinuity-Free QuAD (DF-QuAD) algorithm [21] aggregates the strengths of attackers and supporters of an argument in a QuAD framework using the strength aggregation function, which is defined as $\mathcal {F}: \mathbb {I}^{*}\rightarrow \mathbb {I}$, where for $S = (v_1,\ldots ,v_n) \in \mathbb {I}^{*}$:

$$\begin{aligned} \text{ if } n=0&: {\mathcal {F}}(S) = 0 \nonumber \\ \text{ if } n=1&: {\mathcal {F}}(S) = v_1 \nonumber \\ \text{ if } n=2&: {\mathcal {F}}(S) = {f}(v_{1},v_{2}) \nonumber \\ \text{ if } n>2&: {\mathcal {F}}(S) = {f}({\mathcal {F}}(v_1,\ldots ,v_{n\,-\,1}),v_n) \nonumber \end{aligned}$$

with the base function $f\!:\! \mathbb {I}\!\times \!\mathbb {I}\!\rightarrow \!\mathbb {I}$ defined, for $v_1, \!v_2 \!\in \!\mathbb {I}$, as:

$$\begin{aligned} f(v_1,v_2)&= {v_1} + (1-v_1) \cdot v_2 = v_1 + v_2 - v_1 \cdot v_2 \nonumber \end{aligned}$$

Once the strengths of an argument’s attackers and supporters have been aggregated separately using $\mathcal {F}$, the combination function, defined as $c:\,\mathbb {I}\,\times \,\mathbb {I}\,\times \,\mathbb {I}\,\rightarrow \,\mathbb {I}$, is used to combine the two ($v^{-}$ and $v^{+}$) with the base score of the argument ($v^{0}$), in different ways depending on which of $v^{-}$ and $v^{+}$ is larger, as follows:

$$\begin{aligned} c(v^{0},v^{-},v^{+})&=v^{0}-v^{0}\cdot |v^{+}-v^{-}|&\text {if} \;\; v^{-}\ge v^{+}\nonumber \\ c(v^{0},v^{-},v^{+})&=v^{0}+ (1-v^{0}) \cdot |v^{+}-v^{-}|&\text {if} \;\; v^{-}<v^{+}\nonumber \end{aligned}$$

The score function, $\sigma :\mathcal {A}\,\cup \,\mathcal {C}\,\cup \,\mathcal {P}\,\rightarrow \,\mathbb {I}$, determines the inputs for the combination function, giving the arguments’ strength, as follows, for any $a \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$:

$$\begin{aligned} {\sigma (a) = c(\tau (a), \mathcal {F}(\sigma (\mathcal {R}^{-}(a))),\mathcal {F}(\sigma (\mathcal {R}^{+}(a))))} \nonumber \end{aligned}$$

where if $(a_{1},\ldots ,a_{n})$ is an arbitrary permutation of the $(n \, \ge \, 0)$ attackers in $\mathcal {R}^{-}(a)$, $\sigma (\mathcal {R}^{-}(a))=(\sigma (a_{1}),\ldots ,\sigma (a_{n}))$ (similarly for supporters).

For the framework in Fig. 1, if all arguments have a base score of 0.5, each of the arguments’ resulting strength is 0.5, due to the framework’s symmetry.

4 The QuAD-V Framework

We extend the QuAD framework defined in [1] to incorporate a set of users and their votes on arguments, while dropping the base score as given.

Definition 1

A QuAD for Voting (QuAD-V) framework is a 6-tuple $\langle \mathcal {A}, \mathcal {C}, \mathcal {P}, \mathcal {R}, \mathcal {U}, \mathcal {V}\rangle$ such that:

• $\mathcal {A}$ is a finite set of answer arguments;

• $\mathcal {C}$ is a finite set of con arguments;

• $\mathcal {P}$ is a finite set of pro arguments;

• the sets $\mathcal {A}$, $\mathcal {C}$ and $\mathcal {P}$ are pairwise disjoint;

• $\mathcal {R}\subseteq (\mathcal {C}\cup \mathcal {P}) \times (\mathcal {A}\cup \mathcal {C}\cup \mathcal {P})$ is an acyclic binary relation;

• $\mathcal {U}$ is a finite set of users;

• $\mathcal {V}: \mathcal {U}\times (\mathcal {A}\cup \mathcal {C}\cup \mathcal {P}) \rightarrow \lbrace - , ? , + \rbrace$ is a total function; $\mathcal {V}(u,a)$ is the vote of user $u \in \mathcal {U}$ on argument $a \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$.

Note that we impose that $\mathcal {V}$ is total and users explicitly specify ? as a vote. Alternatively, we could have allowed $\mathcal {V}$ to be partial, interpreting the absence of a vote by a user as ?.

In the remainder of the paper, unless otherwise indicated, we assume as given a QuAD-V framework $\mathcal {Q}=\langle \mathcal {A}, \mathcal {C}, \mathcal {P}, \mathcal {R}, \mathcal {U}, \mathcal {V}\rangle$.

Definition 2

For any argument $a \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$, the set of users voting for a is $\mathcal {V}^{+}(a)=\lbrace u \, \in \, \mathcal {U}: \mathcal {V}(u,a) \, = \, + \rbrace$ and the set of users voting against a is $\mathcal {V}^{-}(a)=\lbrace u \, \in \, \mathcal {U}: \mathcal {V}(u,a) = - \rbrace$.

The number of positive or negative votes on an argument are summated using the following functions:

Definition 3

The positive vote count for an argument is $\mathcal {N}^{+}: (\mathcal {A}\cup \mathcal {C}\cup \mathcal {P}) \rightarrow \mathbb {N}$, such that, for any argument $a\in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$, $\mathcal {N}^{+}(a) = | \mathcal {V}^{+}(a) |$. The negative vote count for an argument is $\mathcal {N}^{-}: (\mathcal {A}\cup \mathcal {C}\cup \mathcal {P}) \rightarrow \mathbb {N}$, such that, for any argument $a\in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$, $\mathcal {N}^{-}(a) = | \mathcal {V}^{-}(a) |$.

We use both vote counts to calculate base scores of arguments, providing a measure of the direct votes on the arguments. It should be noted that this differs from the method of treating the positive counts as supporters and the negative count as attackers, as in [20].

Definition 4

The vote base score (wrt $\mathcal {Q}$) is defined as ${\tau _v}:\mathcal {A}\,\cup \,\mathcal {C}\,\cup \,\mathcal {P}\,\rightarrow \,\mathbb {I}$ where, for any $a \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$:

This definition implies that the neutral or starting point for a vote base score is 0.5. A positive (negative, resp.) vote from a user will then add (subtract, resp.) 0.5 divided by the number of users in $\mathcal {U}$ to (from, resp.) this starting point of 0.5. A neutral vote will not have any effect on the vote base score. For example, for the framework in Fig. 1, if $\mathcal {N}^{+}(S1)= 3$ and $\mathcal {N}^{-}(S1)=3$ and there are 10 users, then $\tau _v(S1)=0.5$. For the same framework and number of users, if $\mathcal {N}^{+}(S2)= 8$ and $\mathcal {N}^{-}(S2)=0$, then $\tau _v(S2)=0.9$.

The score function from the DF-QuAD algorithm can then be used to calculate the strength of each argument using the vote base score as the base score. We refer to this instantiation of the DF-QuAD algorithm as the QuAD-V Algorithm. This strength provides a combined measure of the direct votes on the argument and its indirect votes. For an argument a, the indirect votes are those on any other argument in the tree $T_a$. These votes affect a through the attacking and supporting relations, with the underlying assumption that votes justified by “reasoning” (e.g. supporting arguments in the case of positive votes) are stronger than votes which are not. For example, for the framework in Fig. 1, the strength of argument $S1$ is increased if users agree with its supporter $S3$.

5 Properties of the QuAD-V Algorithm

Since the QuAD-V algorithm is an instantiation of the DF-QuAD algorithm, equivalent properties to those given in [21] for the latter hold for the former. We omit them here for lack of space to focus on new properties, specific to QuAD-V.

Firstly, in QuAD-V, an argument with more positive (negative, resp.) votes has a higher (lower, resp.) vote base score than an argument with fewer positive (negative, resp.) votes:

Property 1

For any $a, b \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$:

Note that the “only-if” direction of the three statements in Property 1 does not hold in general. For example, if we have two arguments a and b such that $\mathcal {N}^{+}(a) = \mathcal {N}^{-}(a) = 2$ and $\mathcal {N}^{+}(b) = \mathcal {N}^{-}(b) = 3$, then $\tau _v(a)=\tau _v(b)$.

In the following properties, the attacking and supporting strengths of an argument $a \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$, i.e. $\mathcal {F}(\sigma (\mathcal {R}^{-}(a)))$ and $\mathcal {F}(\sigma (\mathcal {R}^{+}(a)))$, are represented, resp., as $v^{-}_a$ and $v^{+}_a$.

An argument with more positive (negative, resp.) votes does not have a lower (higher, resp.) strength than an argument with fewer positive (negative, resp.) votes, equal negative (positive, resp.) votes, equal attacking strength and equal supporting strength:

Property 2

For any $a,b \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$, if $v^{-}_a = v^{-}_b$, $v^{+}_a = v^{+}_b$, then:

An argument with a higher attacking (supporting, resp.) strength does not have a higher (lower, resp.) strength than an argument with a lower attacking (supporting, resp.) strength, equal supporting (attacking, resp.) strength, equal positive votes and equal negative votes:

Property 3

For any $a,b \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$, if $\tau _v(a)=\tau _v(b)$, then:

An argument with stronger (weaker, resp.) attackers than supporters has a strength lower (higher, resp.) than the argument’s vote base score, provided that this base score is not already minimal (maximal, resp.):

Property 4

For any $a \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$:

If all users vote against (for, resp.) an argument, the vote base score is the minimum (maximum, resp.) value, while if equal numbers of users vote for and against an argument, the vote base score is the neutral value (0.5):

Property 5

For any $a \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$:

$$\begin{aligned} \tau _v(a)&= 0&\qquad \qquad \qquad \text {iff} \;\; \mathcal {N}^{-}(a) = |\mathcal {U}| \end{aligned}$$

(1)

$$\begin{aligned} \tau _v(a)&= 0.5&\qquad \qquad \qquad \text {iff} \;\; \mathcal {N}^{+}(a) = \mathcal {N}^{-}(a) \nonumber \\ \tau _v(a)&= 1&\qquad \text {iff} \;\; \mathcal {N}^{+}(a) = |\mathcal {U}| \end{aligned}$$

(2)

Our final property gives that for an argument to have the minimum (maximum, resp.) strength, either the supporters (attackers, resp.) have the minimum value and the attackers (supporters, resp.) the maximum or 100% of the users vote against (for, resp.) it with its attackers (supporters, resp.) at least as strong as its supporters (attackers, resp.).

Property 6

For any $a \in \mathcal {A}\cup \mathcal {C}\cup \mathcal {P}$:

$$\begin{aligned} \sigma (a) = 0 \;\;\;\; \text { iff }&[v^{-}_a = 1 \wedge v^{+}_a = 0] \vee [\mathcal {N}^{-}(a) = |\mathcal {U}| \wedge v^{-}_a \ge v^{+}_a] \end{aligned}$$

(3)

$$\begin{aligned} \sigma (a) = 1 \;\;\;\; \text { iff }&[v^{-}_a = 0 \wedge v^{+}_a = 1] \vee [\mathcal {N}^{+}(a)= |\mathcal {U}| \wedge v^{-}_a \le v^{+}_a] \end{aligned}$$

(4)

We may deem an argument with a strength of 1 to be accepted, of 0.5 to be neutral and of 0 to be rejected. Then, directly from the properties above, an accepted argument either has universally positive votes from the users and supporters at least as strong as its attackers, or it has an accepted argument amongst its supporters and all of its attackers are rejected. Similarly, a rejected argument either has universally negative votes and attackers at least as strong as its supporters, or it has an accepted argument amongst its attackers and all of its supporters are rejected. This interpretation of arguments as accepted, neutral or rejected, depending on their strength, is a form of bipolar labelling semantics, in the spirit of the labelling semantics of [4] for abstract argumentation frameworks. There, arguments are labelled in, undecided or out, and, for a labelling to be complete, an argument is labelled in iff its attackers are all labelled out and an argument is labelled out iff at least one of its attackers is labelled in.

Overall, these properties show that the QuAD-V algorithm produces a notion of strength which is based on direct as well as indirect votes on arguments. Thus, if an argument has attackers and/or supporters then its strength is generally different from its base score, based exclusively on direct votes. This is only meaningful if the voters are voting rationally and the underlying argumentation frameworks are able to represent these opinions effectively, as discussed in the next section.

6 Rational Voters

QuAD-V frameworks offer the potential for characterising a user as rational. In this section we define rationality in a QuAD-V framework and some requirements which, if held, remove instances of irrationality.

In order to define rationality for individual voters we first reduce frameworks to delegate frameworks for each user, which are to QuAD-V frameworks with a single user.

Definition 5

A delegate framework for a user u is $\mathcal {Q}^u \!=\! \langle \mathcal {A}^u\!,\! \mathcal {C}^u\!,\! \mathcal {P}^u\!,\! \mathcal {R}^u\!,\! \{u\}\!,\! \mathcal {V}^u \rangle$.

In the remainder of the paper, when given a delegate framework, we use $\tau _v(a)$, $\sigma (a)$, $\mathcal {R}^{-}(a)$, $\mathcal {R}^{+}(a)$, $v^{-}_a$ and $v^{+}_a$ to indicate, resp., the vote base score, strength, attackers, supporters, attacking strength and supporting strength of an argument a wrt the delegate framework.

We posit that if a user votes for an argument, the supporters of that argument should be at least as strong as the attackers. This amounts to the user’s reasoning for that argument being at least as strong as that against it, and therefore justifies the vote for the argument. Conversely, if a user votes against an argument, the attackers of that argument should be at least as strong as the supporters, amounting to the user’s reasoning against that argument being at least as strong as that for it, therefore justifying the vote against the argument. If these conditions do not hold, we may therefore infer that either there is something missing from the framework or that the user is voting irrationally.

Definition 6

Given a delegate framework $\mathcal {Q}^u{= \langle \mathcal {A}^{u}, \mathcal {C}^{u}, \mathcal {P}^{u}, \mathcal {R}^{u}, \{ u \}, \mathcal {V}^{u} \rangle }$, u is strictly rational (wrt $\mathcal {Q}^u$) iff $\forall a \in \mathcal {A}^u \cup \mathcal {C}^u \cup \mathcal {P}^u$:

$$\begin{aligned}&\text {if } \tau _v(a) = 0 \text { then } v^{-}_a \ge v^{+}_a; \end{aligned}$$

(5)

$$\begin{aligned}&\text {if } \tau _v(a) = 1 \text { then } v^{-}_a \le v^{+}_a. \end{aligned}$$

(6)

There are a number of ways for a user in a QuAD-V framework to fail to satisfy strict rationality by this definition. Due to a lack of space, in this paper we chose one weaker definition of rationality and show how this instance (and, we predict, others) may be used to give more information about a voter’s reasoning.

Definition 7

Given a delegate framework $\mathcal {Q}^u{= \langle \mathcal {A}^{u}, \mathcal {C}^{u}, \mathcal {P}^{u}, \mathcal {R}^{u}, \{ u \}, \mathcal {V}^{u} \rangle }$, u is individually rational (wrt $\mathcal {Q}^u$) iff:

$$\begin{aligned} \mathbf {R1}: \not \exists a \in \mathcal {A}^u \cup \mathcal {C}^u \cup \mathcal {P}^u \text { such that: }&\mathcal {V}^u(u,a) = +, \nonumber \\&\exists b \in \mathcal {R}^{-}(a): \mathcal {V}^u(u,b)=+, \text {and} \nonumber \\&\forall c \in \mathcal {R}^{+}(a) : \mathcal {V}^u(u,c) = - \nonumber \end{aligned}$$

and:

$$\begin{aligned} \mathbf {R2}: \not \exists d \in \mathcal {A}^u \cup \mathcal {C}^u \cup \mathcal {P}^u \text { such that: }&\mathcal {V}^u(u,d) = -, \nonumber \\&\exists e \in \mathcal {R}^{+}(d): \mathcal {V}^u(u,e)=+, \text {and} \nonumber \\&\forall f \in \mathcal {R}^{-}(d) : \mathcal {V}^u(u,f) = - \nonumber \end{aligned}$$

If R1 is violated for some user u, then the user agrees with some argument a, agrees with one of its attackers b but disagrees with all of its supporters, which we see as being irrational. This violation can be avoided if the user also agrees with an argument (c) supporting a. Likewise, for requirement R2 to be violated, the user disagrees with the argument a, agrees with one of its supporters b but disagrees with all of its attackers, which we also see as being irrational. This violation can be avoided if the user also agrees with an argument (f) attacking d. We can therefore characterise the situations where R1 and R2 are violated as those where either pro or con arguments are missing from the debate or the voter is voting irrationally. In the Brexit debate in Sect. 2, the addition of $S5$ is an example of enforcement of R2.

The following proposition shows that if a user in a QuAD-V framework fails to meet the requirements of being individually rational, then it also fails to meet those of being strictly rational.

Proposition 1

Given a delegate framework $\mathcal {Q}^u = \langle \mathcal {A}^u, \mathcal {C}^u, \mathcal {P}^u, \mathcal {R}^u, \{u\}, \mathcal {V}^u \rangle$, if u is not individually rational (wrt $\mathcal {Q}^u$) then u is not strictly rational (wrt $\mathcal {Q}^u$).

Proof

For any user $u \in \mathcal {U}$ , if u is not individually rational then one (or both) of R1 or R2 fail to hold.

If R1 does not hold then consequently $\exists a \in \mathcal {A}^u \cup \mathcal {C}^u \cup \mathcal {P}^u$ such that $\mathcal {V}^u(u,a) = +$, $\exists b \in \mathcal {R}^{-}(a): \mathcal {V}^u(u,b)=+$ and $\forall c \in \mathcal {R}^{+}(a) : \mathcal {V}^u(u,c) = -$. By Property 5 (1) and (2), $\tau _v(a)=1$, $\tau _v(b)=1$ and $\tau _v(c)=0$ (for any c). For u to be strictly rational, it must hold that $v^{-}_a \, \le \, v^{+}_a$, by (6). By Property 4, $\sigma (b) = 1$ as if $\sigma (b)<\tau _v(b)$, then $v^{-}_b > v^{+}_b$, which itself causes u to fail to be strictly rational. Then, $\sigma (b) = 1$ implies, by the QuAD-V algorithm definition, that $v^{-}_a = 1$. Then, for u to be strictly rational, it must hold that $v^{+}_a = 1$, which, by the QuAD-V algorithm definition, requires some $c \in \mathcal {R}^{+}(a) : \sigma (c) = 1$. Since $\tau _v(c) = 0$, by Property 4, this requires that $v^{-}_c < v^{+}_c$ but this itself causes u to fail to be strictly rational.

If R2 does not hold then consequently $\exists d \in \mathcal {A}^u \cup \mathcal {C}^u \cup \mathcal {P}^u$ such that $\mathcal {V}^u(u,d) = -$, $\exists e \in \mathcal {R}^{+}(d): \mathcal {V}^u(u,e)=+$ and $\forall f \in \mathcal {R}^{-}(d) : \mathcal {V}^u(u,f) = -$. By Property 5 (1) and (2), $\tau _v(d)=0$, $\tau _v(e)=1$ and $\tau _v(f)=0$ (for any f). For u to be strictly rational, it must hold that $v^{-}_d \ge v^{+}_d$, by (5). By Property 4, $\sigma (e) = 1$ as if $\sigma (e)<\tau _v(e)$, then $v^{-}_e > v^{+}_e$, which itself causes u to fail to be strictly rational. Then, $\sigma (b) = 1$ implies, by the QuAD-V algorithm definition, that $v^{+}_d = 1$. Then, for u to be strictly rational, it must hold that $v^{-}_d = 1$, which, by the QuAD-V algorithm definition, requires some $f \in \mathcal {R}^{-}(d) : \sigma (f) = 1$. Since $\tau _v(f) = 0$, by Property 4, this requires that $v^{-}_f < v^{+}_f$ but this itself causes u to fail to be strictly rational.    $\square$

Then, collective rationality amounts to individual rationality for all users.

Definition 8

Given a QuAD-V framework $\mathcal {Q}= \langle \mathcal {A}, \mathcal {C}, \mathcal {P}, \mathcal {R}, \mathcal {U}, \mathcal {V}\rangle$, $\mathcal {U}$ is collectively rational (wrt $\mathcal {Q}$) iff $\forall u \in \mathcal {U}$, u is individually rational (wrt the delegate framework $\mathcal {Q}^u= \langle \mathcal {A}^{u}, \mathcal {C}^{u}, \mathcal {P}^{u}, \mathcal {R}^{u}, \{ u \}, \mathcal {V}^{u} \rangle$).

Note that we assume that the given QuAD-V framework correctly represents dialectical relations between arguments and do not accommodate the possibility that a violation of the requirements may be due to a user actually disagreeing with the attack between two arguments it agrees with or the support between an argument it agrees with and one it disagrees with. We leave accommodating this possibility for future work.

Note also that other definitions of rationality, in addition to those shown here, may be possible but are left for future work.

In the next section we describe how QuAD-V frameworks can be used in opinion polls to highlight irrational voting and give the voters the opportunity to render their votes rational, if required.

7 QuAD-V  Opinion Polls

QuAD-V opinion polls use an initial QuAD framework to specify (and relate) statements for users to vote on. The users may be asked to vote on the statements sequentially (and be unaware of the relations in the underlying QuAD framework) or may be presented with a graphical representation of the QuAD framework. Whichever the case, the result of the voting is a QuAD-V framework (referred to as master framework below). Users in this framework that are not individually rational are then asked dynamic questions. The users’ responses to these dynamic questions transform their delegate frameworks iteratively until all of the individual irrationalities are removed and delegate frameworks become stable. A revised master framework is then created, which may be seen as the amalgamation of the stable delegate frameworks, and its set of users is guaranteed to be collectively rational. Multiple runs of this process may take place to allow voting on new arguments introduced on previous runs. Figure 2 summarises (a run of) this process, which is described in detail in this section.

Fig. 2.

QuAD-V opinion polling process for users $u_1$ to $u_n$

In the remainder of this section, $\mathcal {Q}= \langle \mathcal {A}, \mathcal {C},\! \mathcal {P}\!, \mathcal {R}, \mathcal {U},\! \mathcal {V}\rangle$ is the master framework and $u_i \in \mathcal {U}$ is a generic user, where $1\,\le \,i\,\le \,n$ and $n \!=\! |\mathcal {U}|$. Further, given any QuAD-V framework $\mathcal {Q}^*\,=\,\langle \mathcal {A}^*\!, \mathcal {C}^*\!, \!\mathcal {P}^*\!, \mathcal {R}^*\!, \mathcal {U}^*\!, \!\mathcal {V}^* \rangle$, we denote $\mathcal {A}^* \!\cup \mathcal {C}^* \!\cup \mathcal {P}^*$ as $\mathcal {X}\!(\mathcal {Q}^*\!)$.

7.1 Iteration and Initial Delegate Frameworks

Initial and iteration delegate frameworks are restrictions of the master framework and transformations thereof, resp.:

Definition 9

For $j \ge 0$, $\mathcal {Q}_{j}^{u_{i}}=\langle \mathcal {A}_{j}^{u_{i}}, \mathcal {C}_{j}^{u_{i}}, \mathcal {P}_{j}^{u_{i}}, \mathcal {R}_{j}^{u_{i}}, \{u_{i}\}, \mathcal {V}_{j}^{u_{i}} \rangle$ is the $j^{th}$ iteration delegate framework, defined as follows:

• If $j=0$ then $\mathcal {A}_{0}^{u_{i}} = \mathcal {A}$, $\mathcal {C}_{0}^{u_{i}}=\mathcal {C}$, $\mathcal {P}_{0}^{u_{i}} = \mathcal {P}$, $\mathcal {R}_{0}^{u_{i}}=\mathcal {R}$ and, $\forall a \in \mathcal {X}(\mathcal {Q}_{0}^{u_{i}})$, $\mathcal {V}_{0}^{u_{i}}(u_i,a) = \mathcal {V}(u_i,a)$.

• If $j>0$ then $\mathcal {A}_{j}^{u_{i}}= \mathcal {A}_{j\,-\,1}^{u_{i}},\mathcal {C}_{j}^{u_{i}} \supseteq \mathcal {C}_{j\,-\,1}^{u_{i}},\mathcal {P}_{j}^{u_{i}} \supseteq \mathcal {P}_{j\,-\,1}^{u_{i}},\mathcal {R}_{j}^{u_{i}} \supseteq \mathcal {R}_{j\,-\,1}^{u_{i}}$, and there exists at most one argument $a \in \mathcal {X}(\mathcal {Q}_{j}^{u_{i}})$ such that if $\mathcal {V}_{{j\,-\,1}}^{u_{i}}(u_i,a) = +$ then $\mathcal {V}_{j}^{u_{i}}(u_i,a)=-$, if $\mathcal {V}_{{j\,-\,1}}^{u_{i}}(u_i,a) = -$ then $\mathcal {V}_{j}^{u_{i}}(u_i,a)=+$, and $\forall b \in \mathcal {X}(\mathcal {Q}_{j}^{u_{i}}) \backslash \{ a \}$, $\mathcal {V}_{j}^{u_{i}}(u_i,b) = \mathcal {V}_{{j\,-\,1}}^{u_{i}}(u_i,b)$.

We refer to $\mathcal {Q}_{0}^{u_{i}}$ as the initial delegate framework.

Note that at each iteration users may change their votes and/or add arguments and relations between arguments.

7.2 Dynamic Questions and Responses

In the remainder of this section, where there is no ambiguity, we will assume as given a $j^{th}$ iteration delegate framework $\mathcal {Q}_{j}^{u_{i}}\,=\,\langle \mathcal {A}_{j}, \mathcal {C}_{j}, \mathcal {P}_{j}, \mathcal {R}_{j}, \{u_{i}\}, \mathcal {V}_{j} \rangle$ for $j\ge 0$.

Dynamic questions are put to users that are found to be individually irrational. The allowed responses to these questions indicate how to remove the irrationalities from the delegate frameworks. We define two such questions. The first is produced when requirement R1 is not fulfilled:

Definition 10

A Type 1 Dynamic Question $\varOmega _1(\mathcal {Q}_j^{u_{i}}, u_{i}, a, b)$ with possible responses $\rho _{1}(\alpha ), \rho _{2}, \rho _{3}$ is produced for arguments $a,b\,\in \,\mathcal {X}(\mathcal {Q}_{j}^{u_{i}})$ such that $b \, \in \, \mathcal {R}^{-}_{j}(a)$ when $\mathcal {V}_{j}({u_{i}},a) = +$, $\mathcal {V}_{j}({u_{i}},b) = +$ and $\forall c \in \mathcal {R}^{+}_{j}(a)$, $\mathcal {V}_{j}(u_{i},c) = -$.

Informally, question and responses may be read as follows:

• $\varOmega _1(\mathcal {Q}_j^{u_{i}}, u_{i}, a, b)$ - “Why do you agree with argument a when you agree with its attacker b and none of its supporters?”

• $\rho _{1}(\alpha )$ - [User inputs pro argument $\alpha$ for a]

• $\rho _{2}$ - “I made a mistake, I disagree with a”

• $\rho _{3}$ - “I made a mistake, I disagree with b”

The first response gives insight into the reasons for the user agreeing with a by providing a supporting argument for a, which we envisage not to belong already to the (current) delegate framework. The second and third responses help to rectify mistakes or prevent users from voting randomly.

Responses are used to revise delegate frameworks:

Definition 11

Given a Type 1 Dynamic Question $\varOmega _1(\mathcal {Q}_j^{u_{i}}, u_{i}, a, b)$, let $\rho _{*}$ be its response. Then $\mathcal {Q}_{j\,+\,1}^{u_{i}}$ is the revision of $\mathcal {Q}_{j}^{u_{i}}$ by $\rho _{*}$ to $\varOmega _1(\mathcal {Q}_j^{u_{i}}, u_{i}, a, b)$ where¹¹:

The second question is produced when R2 is not fulfilled:

Definition 12

A Type 2 Dynamic Question $\varOmega _2(\mathcal {Q}_j^{u_{i}}, u_{i}, a, b)$ with possible responses $\rho _{1}(\alpha ), \rho _{2}, \rho _{3}$ is produced for arguments $a,b\,\in \,\mathcal {X}(\mathcal {Q}_{j}^{u_{i}})$ such that $b \, \in \, \mathcal {R}^{+}_{j}(a)$ when $\mathcal {V}_{j}({u_{i}},a) = -$, $\mathcal {V}_{j}({u_{i}},b) = +$ and $\forall c \in \mathcal {R}^{-}_{j}(a)$, $\mathcal {V}_{j}(u_{i},c) = -$.

Informally, question and responses may be read as follows:

• $\varOmega _2(\mathcal {Q}_j^{u_{i}}, u_{i}, a, b)$ - “Why do you disagree with argument a when you agree with its supporter b and none of its attackers?”

• $\rho _{1}(\alpha )$ - [User inputs con argument $\alpha$ against a]

• $\rho _{2}$ - “I made a mistake, I agree with a”

• $\rho _{3}$ - “I made a mistake, I disagree with b”

Definition 13

Given a Type 2 Dynamic Question $\varOmega _2(\mathcal {Q}_j^{u_{i}}, u_{i}, a, b)$, let $\rho _{*}$ be its response. Then $\mathcal {Q}_{j+1}^{u_{i}}$ is the revision of $\mathcal {Q}_{j}^{u_{i}}$ by $\rho _{*}$ to $\varOmega _2(\mathcal {Q}_j^{u_{i}}, u_{i}, a, b)$ where:

Note that users are not allowed to give no response to either type of dynamic question, i.e. users are assumed to be cooperative.

When no more dynamic questions can be produced for arguments in a delegate framework then it is deemed stable:

Definition 14

$\mathcal {Q}_{j}^{u_{i}}$ is stable iff no dynamic questions are produced for any arguments in $\mathcal {X}(\mathcal {Q}_{j}^{u_{i}})$.

Stable delegate frameworks are guaranteed to exist and their users are guaranteed to be individually rational , provided that they change their vote on each argument at most once:

Proposition 2

Let us assume that $u_i$ is such that for every $\mathcal {Q}_0^{u_i},\ldots$, for every $a \, \in \, \mathcal {X}(\mathcal {Q}_{0}^{u_{i}})$, there exists at most one j, for $0 \, \le \, j$, such that . Then $\exists m_i \ge 0$ and $\mathcal {Q}_0^{u_i},\ldots ,\mathcal {Q}_{m_i}^{u_{i}}$ such that $\mathcal {Q}_0^{u_i}$ is the initial delegate framework, each $\mathcal {Q}_j^{u_i}$, for $0\,<\,j\,\le \,m_i$, is the revision of $\mathcal {Q}_{j-1}^{u_i}$ (by some response to some dynamic question), and $\mathcal {Q}_{m_i}^{u_{i}}$ is stable. Further, $u_i$ is individually rational (wrt $\mathcal {Q}_{m_i}^{u_{i}}$).

Proof

(Sketch). Each revision eliminates one violation of R1 or R2, and adds at most one argument (in the case of instances of $\rho _{1}$) which cannot introduce a violation. However, changing votes may do so. Each additional violation will give rise to an additional dynamic question and so votes on arguments would have to be changed back and forth for a delegate framework not to be reached. The order in which these questions and responses are produced is irrelevant, as the conditions for the dynamic questions are mutually exclusive. Thus convergence to $\mathcal {Q}_{m_i}^{u_{i}}$ is guaranteed, under the stated conditions. It is easy to see that users in stable frameworks are individually rational.    $\square$

7.3 Revised Master Framework

Once all delegate frameworks are stable, amalgamating the delegate frameworks gives the revised master frameworks.

Definition 15

Let $\mathcal {Q}_{m_i}^{u_i}$ be the stable delegate frameworks for $u_i \in \mathcal {U}$, where $i \ge 1$. A revised master framework $\dot{\mathcal {Q}}$ is $\langle \mathcal {A}, \dot{\mathcal {C}}, \dot{\mathcal {P}}, \dot{\mathcal {R}}, \mathcal {U}, \dot{\mathcal {V}} \rangle$, where $\dot{\mathcal {C}} = \mathcal {C}\cup \mathcal {C}_+$ such that $\mathcal {C}_+ \subseteq \mathcal {C}_{m_1} \cup \ldots \cup \mathcal {C}_{m_n} \backslash \mathcal {C}$, $\dot{\mathcal {P}} = \mathcal {P}\cup \mathcal {P}_+$ such that $\mathcal {P}_+ \subseteq \mathcal {P}_{m_1} \cup \ldots \cup \mathcal {P}_{m_n} \backslash \mathcal {P}$, $\dot{\mathcal {R}} = \mathcal {R}\cup \mathcal {R}_+$ such that $\mathcal {R}_+ \subseteq \mathcal {R}_{m_1} \cup \ldots \cup \mathcal {R}_{m_n} \backslash \mathcal {R}$, and $\forall a \in \mathcal {X}(\dot{\mathcal {Q}})$ and $\forall u_i \in \mathcal {U}$, if $\exists a \in \mathcal {Q}_{m_i}^{u_i}$ then $\dot{\mathcal {V}}(u_i, a) = \mathcal {V}_{m_i}(u_i, a)$, otherwise $\dot{\mathcal {V}}(u_i, a)\,=\,?$.

Basically, each selection of “new” arguments, in the revised but not in the initial delegate frameworks, gives a revised master framework, with users’ votes on “unseen” arguments (from other users’ revised delegate frameworks) set to the neutral value.¹² The largest possible revised master framework includes all these arguments, whereas the smallest includes none. Our definition allows for human intervention to review the contents of the stable delegate frameworks and disregard, for example, “new” arguments that are not valid or relevant. We leave more sophisticated forms of amalgamation, e.g. taking into account duplications across users and natural language processing, for future work.

Irrespective of the choice of revised master framework, any given user is guaranteed to be at least as individually rational as they were in the master framework the process started with, as this process does not introduce any violations of requirements R1, R2.

Proposition 3

Let $\dot{\mathcal {Q}}$ be a revised master framework. Let x be the number of violations of R1 and R2 in $\mathcal {Q}$ and y be the number of violations of R1 and R2 in $\dot{\mathcal {Q}}$. Then, $y \le x$.

If a user’s new arguments have been integrated into the revised master framework then the user is guaranteed to be individually rational.

Proposition 4

If $\mathcal {C}_{m_i} \!\subseteq \dot{\mathcal {C}}$ and $\mathcal {P}_{m_i}\! \subseteq \dot{\mathcal {P}}$ then $u_{i}$ is individually rational (wrt $\dot{\mathcal {Q}}$).

Finally, the largest possible revised master framework’s set of users is collectively rational.

Proposition 5

If $\dot{\mathcal {C}} = \mathcal {C}\cup \mathcal {C}_{m_1} \cup \ldots \cup \mathcal {C}_{m_n}$ and $\dot{\mathcal {P}} = \mathcal {P}\cup \mathcal {P}_{m_1} \cup \ldots \cup \mathcal {P}_{m_n}$ then $\dot{\mathcal {U}}$ is collectively rational (wrt $\dot{\mathcal {Q}}$).

8 Conclusions

We have presented QuAD-V frameworks, extending QuAD frameworks [1] to incorporate voting, and applied them to support opinion polling.

QuAD-V frameworks can be also seen as extending the Social Argumentation Frameworks (SAFs) of [16] by also allowing support between arguments. Differently from QuAD-V frameworks, SAFs are not restricted to acyclic attack relations: we leave the relaxation of this restriction for QuAD-V frameworks as future work. Also, it would be interesting to study formal relationships between our vote aggregation mechanism and the one in [16] and our notion of strength and the evaluation of arguments in SAFs (determining, in particular, whether SAFs fulfil versions of the properties in Sect. 5). Like QuAD-V frameworks, mDICE frameworks [20] accommodate votes on arguments as well as attack and support relations, but keep votes and dialectical relations somewhat separate. Another approach for determining rationality in users’ labellings of arguments is described in [11]. Differently from QuAD-V frameworks, the approach of [11] has not been applied to opinion polling (using dynamic questions and a revision process such as the one we have defined). Also, the aggregation function in [11] differs from, and exhibits different properties to, the QuAD-V algorithm. However, [11] also consider the relationship between direct and indirect opinions (votes, in our case) and their definition of “coherence” aligns with our definition of strict rationality (within the respective contexts). We plan to study relationships of QuAD-V frameworks with these approaches, along with their relative suitability to support opinion polling, in the future.

Our proposed QuAD-V opinion polling holds two main advantages over the flat, conventional approach which is almost universally adopted. Firstly, as we have shown, the use of an underlying QuAD-V framework to structure (semantically) statements in opinion polls paves the way to empower users to iteratively evolve polls so that they highlight, and potentially eradicate, irrationalities in users’ opinions and, as a consequence, are more informative to the pollster. Secondly, the use of a method for determining the strength of opinions seen as arguments in QuAD-V frameworks can give useful additional measures of public sentiment on statements in a poll. We plan to develop the system further in future work, e.g. allowing users to respond to dynamic questions uncooperatively (e.g. “I don’t know”) or by disagreeing with the relation itself (e.g. “I don’t agree that $S3$ supports $S1$”, in the Brexit debate). The former may indicate irrational voting, while the latter would give an added dimension of dynamicity and self-correction to QuAD-V frameworks and could be implemented without losing the rationality properties by weighting relations (e.g. see [9]).

We have defined a basic notion of a user being strictly rational based on their voting and a weaker notion of a user being individually rational. We have shown how the latter may be beneficial for eliciting reasoning from users, highlighting “illogical” voting and filtering mistakes and random voting using our theoretical evaluation. It could be interesting to utilise strict rationality in this elicitation. An empirical evaluation of an implementation QuAD-V polling is also left as future work, along with comparisons with existing systems, e.g. [10], and verifying our assumption that users are cooperative.

Overall, we hope that our e-polling methodology will help to increase public engagement in a number of settings by letting users take an active part in debates that adapt to user opinions, rather than restricting these to predetermined opinions only.