Two Procedures Based on Ratings

Abstract

While most social choice results pertain to ranking environments where the individuals submit their preference relations (and these only) to the balloting procedure, there are procedures that require a slightly different kind of input from the voters. We discuss two such systems: the majority judgment and the range voting. These are relatively recent entrants in the social choice field. As all procedures they have their advantages and disadvantages, but deserve attention is some decision situations.

8.1 Introduction

The standard assumptions of voting theory include complete and transitive preference relations or rankings. In Hillinger’s (2005) view this assumption is at least partly to blame for the paradoxes of voting as well:

$\ldots$ a new ‘paradox of voting’: It is theorists’ fixation on a context dependent and ordinal preference scale; the most primitive scale imaginable and the mother of all paradoxes.

In any event, one could argue that there are settings in which people are capable of submitting not only rankings but ratings of alternatives in a meaningful way. Accordingly some methods have been devised to aggregate ratings of alternatives into social ratings, rankings and/or choices. In this section we discuss briefly two such methods.

8.2 Majority Judgment

Rating that is familiar to most people is the grading of essays, exams, presentations etc. in institutions of learning. The assessors, judges or instructors are asked to evaluate the participants or their works using predetermined scales, such as numbers from 0 to 5 or letters from A to F or similar ratings that are supposed to reflect the superiority of items with respect to each other. Since we are very familiar with these kinds of performance assessments in schools, it has been suggested that this familiarity should be expected to extend to political contexts as well. If we can assign grades to student papers reflecting their academic quality, what objections are there to our using the same principles in assessing the qualities of candidates or policy alternatives?

Probably none at all, apart from the general idea that academic merit is a more ‘objective’ notion than political merit. Yet, one can envision that also academic assessments are based on some more or less well-defined notion of an ideal – be it of an essay, oral presentation or written exam paper. Similarly our assignment of political merit to alternatives may also be based on some subjective notion of political quality, such as agreement with our own political values. No doubt assessments of political merit of any given candidate are bound to vary a great deal more among the assessors than the assessments of their academic merits. This doesn’t mean that the grades or other ratings could not be used in political elections. Indeed, the method of majority judgment (MJ, for short) introduced and elaborated by Balinski and Laraki is based on aggregating ratings, numerical or non-numerical (Balinski and Laraki 2007, 2010).

To illustrate, consider the profile of Table 8.1. It is consistent with the profile of Table 8.3 in the sense that the preference rankings of the 9 voters are the same in both tables. Thus in Table 8.1 we have assumed that the 4 voters ranking A first give it the grade ‘excellent’, C the grade ‘good’ and B ‘reject’. The 3 voters with $B \succ C \succ A$ ranking give the grades “very good’, ‘good’ and ‘reject’, respectively. Finally, the 2 voters with $C \succ B \succ A$ ranking assign these alternatives the grades ‘very good’, ‘satisfactory’ and ‘reject’, respectively. Now, the MJ method focuses on the median of the grades assigned to alternatives. To determine the median, let the number of voters be n (9 in our example). Denote the grades from the lowest to the highest by $g_{1}, \ldots , g_{h}$ and the number of voters assigning an alternative the grade $g_{j}$ by $n_{j}$. Then the median grade $g_{med}$ is defined by the following two properties:

$$\begin{aligned} \sum _{j=1}^{med} n_{j} {\quad }> & {} {\quad } n/2 \\ \sum _{j=med}^{n} n_{j} {\quad }> & {} {\quad } n/2 \\ \end{aligned}$$

The MJ choice set consists of those alternatives associated with the highest median grade. In Table 8.1 the MJ winner is C as its median grade is the highest. Thus, the Borda and Condorcet winner C of Table 8.3 is chosen. There is, however, also an assignment of grades such that C is not chosen. This is shown in Table 8.2. So, despite the fact the distribution of voter rankings remains the same, there is some variation in MJ outcomes due to the more detailed information on voter opinions utilized in determining the outcomes. It is to be noted, however, that MJ makes a very limited use of the detailed information in focusing on the median grade only. Thus, the median grade of A in Table 8.2 is unaffected if all those voters giving it the grade ‘excellent’ were to assign it the grade ‘reject’. By definition all modifications in the distribution of opinions that leave the median grades of alternatives unaffected result in the same MJ choices.

The fact that several grade assignments can correspond to a given ranking profile – such as in Tables 8.1 and 8.2 – would seem to suggest that the grading methods call for different performance evaluation criteria than the ranking ones. Be that as it may, the grading methods can be – and have been – evaluated in terms of ranking criteria (see Felsenthal and Machover 2008; Felsenthal and Nurmi 2016). For example, as we have just seen in Table 8.2, MJ does not always result in a Condorcet or Borda winner.

Table 8.1 Majority judgment example

Table 8.2 MJ does not elect the Condorcet and Borda winner

Table 8.3 Ambiguous majority principle

8.3 Range Voting

Range voting (RV) is in some respects similar to MJ: each voter gives a grade or value out of a set of grades or values to each alternative. In contradistinction to MJ, the winner in RV is determined on the basis of the grade averages of alternatives: the alternative with the highest average wins. This means that RV is applicable only in those settings where the grades are numerical, while MJ is applicable also in situations where the grades have only ordinal significance. In contradistinction to most other voting systems, the advocates of RV have established a web site – RangeVoting.org – with frequent updates on issues related to RV and its competitors.

RV enables (or requires) the voters not only to express their preferences in terms of an ordinal ranking, but also to indicate for any pair of alternatives how much they prefer one to the other. The grades can, thus, be viewed as values or utilities of alternatives from the voters’ point of view. Hence, RV is sometimes called utilitarian voting. Summing up the grades given by voters to an alternative in a way reflects its collective utility or value. It has been shown that RV satisfies a number of social choice desiderata (e.g. monotonicity and consistency), but fails on Condorcet winner and loser criteria. To see this, consider the profile of Table 8.3 and assign the alternatives grades from the interval 0–100 (the larger the more preferred) as in Table 8.4.

There is a Condorcet winner, C, in this profile, but the Condorcet loser A emerges as the RV winner under the grade assignment of Table 8.4.

In contrast to all ranking based procedures, RV seems to satisfy independence of irrelevant alternatives (IIA) condition that is one of those desiderata that Arrow’s impossibility theorem shows to be mutually incompatible (Arrow 1963). In ranking context this requirement states that if for any pair of n-person profiles, R and $R'$, over a fixed set of alternatives A, the profiles agree on the relative ranking of any two alternatives $x, y \in A$, then so must the relative ranking of x and y also be identical in the collective ranking that ensues from applying the procedure to R and $R'$. Strictly speaking, IIA is not applicable to RV since the latter is a rating, not ranking based system. However, it is obvious that the collective RV rankings of x and y depend only on the individual ratings assigned to them in R and $R'$, not on whatever grades are assigned to other alternatives.

Table 8.4 Range voting fails on Condorcet criteria

8.4 Topics for Further Reflection

• 1. Construct an example where MJ ends up with a Condorcet winner.

• 2. Construct an example where RV elects the Condorcet loser.

• 3. Let us define Nash’s method as follows: each voter assigns each alternative a utility value from the [0.5, 1.0] interval. The Nash score of each alternative is the product of the utility values assigned to this alternative by all voters. The winner is the alternative with the highest score (Riker 1982). Construct an example where Nash’s method does not result in a Condorcet winner.

8.5 Suggestions for Reading

The best available exposition of MJ is (Balinski and Laraki 2010). RV, introduced by Warren D. Smith, is explained, illustrated and compared with other voting rules on the web site maintained by RangeVoting.Org.

Answers to Selected Problems

• 1. Consider the following setting with three voters, three candidates and grades from a (worst) to d (best):

  	Voter 1	Voter 2	Voter 3	Median
  A	b	c	b	b
  B	c	c	c	c
  C	a	b	d	b

  Here B has the highest median and is thus the MJ winner. At the same time it is the Condorcet winner.

• 2. Consider the above MJ example and let $a = 0, b = 1, c = 2, d = 10$. Then C, the Condorcet loser, is the RV winner.

• 3. Consider again the above MJ example and let all voters assign the same values to grades so that $a = 0.5, b = 0.55, c = 0.6, d = 1.0$. Now C, the Condorcet loser, is elected.