Citizens are good forecasters. Ask a sample of voters who they think will win the next election, and the average forecast will likely come close to the true outcome. Evidence shows that this is the case for presidential elections in the United States (Graefe, 2014; Lewis-Beck & Tien, 1999; Miller et al., 2012; Murr & Lewis-Beck, 2021; Rothschild & Wolfers, 2013), and general elections in the United Kingdom (UK) (Lewis-Beck & Stegmaier, 2011; Murr, 2011, 2016; Murr et al., 2021). These studies suggest the existence of a “wisdom of the crowd,” meaning that citizens possess a type of knowledge of everyday politics that is embedded in social networks and can hardly be captured by traditional polling methods. Yet, despite scholarly attention toward citizen forecasts as an alternative forecasting method, our knowledge of how voters form their electoral predictions is limited. What factors shape individual expectations of election results?

Understanding how citizens develop electoral forecasts is relevant because such predictions can exert an important influence on campaign donations (Mutz, 1995), preferences for political candidates (Ansolabehere & Iyengar, 1994), strategic voting (Meffert & Gschwend, 2011) and vote choice. In making voting decisions, voters take into account candidates’ prospects of being elected (Abramowitz, 1987), a process that can lead to “bandwagon” effects, especially when a candidate has clear chances of winning (see, for example, Bartels, 1985; Granberg & Brent, 1983; Nadeau et al., 1993). Perhaps even more importantly, inaccurate electoral predictions can exacerbate disappointment with electoral results and undermine the perceived legitimacy of electoral winners (Krizan et al., 2010). Thus, following Irwin and van Holsteyn, we argue that “[i]f expectations are central to vote decisions, it becomes as important to investigate the source of these expectations as to study the vote decision itself” (Irwin & van Holsteyn, 2002, p. 92).

In this study, we focus on three factors that might influence electoral expectations—i.e., elite cues, social cues, and campaign arguments—in addition to two correlates of forecasts at the individual level—i.e., voting preferences and political sophistication. Apart from a few notable exceptions (Meffert & Gschwend, 2011; Searles et al., 2018), current research provides observational evidence, which leaves us with limited knowledge of the causal determinants of expectations. To understand the influence of these factors, we conducted an original survey experiment during the campaign for the UK referendum on membership of the European Union (EU) (the “Brexit” referendum). We supplement the analysis with representative surveys from the British Election Study (BES) Internet Panel (Fieldhouse et al., 2016).

Although current research has focused on elections for candidates or parties, we argue that citizens’ expectations about how fellow citizens will vote in direct democracy might be equally important, especially when political elites are divided. When party leaders send “conflicting” signals, citizens’ expectations about other citizens’ voting behavior can become a crucial component of decision-making. Furthermore, predicting a close referendum outcome can boost turnout, as citizens realize that a handful of votes can determine the final result, in line with evidence of a positive effect of election closeness on voter turnout (for a review, see Moy & Rinke, 2012). By focusing on the Brexit referendum as a case study, we aim to provide novel evidence of what influences citizen forecasts in the as-yet unexplored context of direct democracy.Footnote 1

Our findings show that citizens’ average forecast fell short of the actual result but was quite close to the referendum vote share. We also find that important individual heterogeneity exists, and in particular, that politically sophisticated voters are more accurate in their predictions and less prone to wishful thinking than non-sophisticated voters. Experimental findings reveal that information signals do affect electoral expectations of partisan voters who adjust their predictions in response to cues provided by their favorite party’s elites and partly in response to campaign arguments. The effects of experimental treatments are larger for low-sophisticated voters, in line with research on the effects of political cues and campaign arguments on voters with low political knowledge (Kam, 2005; Lau & Redlawsk, 2001; Slothuus & de Vreese, 2010). Furthermore, we find that predicting a close result correlates with a higher likelihood of turning out to vote in the referendum. We discuss the implications of these findings for research on citizen forecasts, elite influence and public opinion.

What Influences Electoral Expectations?

A growing number of studies has investigated whether citizens’ perceptions of politics can yield accurate election forecasts in different types of elections (Ganser & Riordan, 2015; Graefe, 2014, 2015; Lewis-Beck & Skalaban, 1989; Lewis-Beck & Tien, 1999; Murr, 2016; Murr & Lewis-Beck, 2021; Murr et al., 2021; Rothschild & Wolfers, 2013). While these studies are primarily concerned with whether citizen forecasts outperform other forecast methods, we focus instead on the relatively unexplored question of what factors influence citizens’ predictions of electoral results. In particular, we consider two individual factors that have been previously investigated in the literature—i.e., voting preferences and political sophistication—in addition to three “exogenous factors” that we manipulate experimentally—i.e., social cues, elite cues and campaign arguments.

Although we focus primarily on the impact that these factors have on the direction of forecasts—such as predicting a higher share of Yes or No votes in a referendum—we also consider the impact of individual characteristics on the accuracy of such forecasts, that is the ability to predict the outcome of a referendum correctly.Footnote 2 Our goal, in this case, is to provide novel evidence of who is more likely to “guess it right” within the as-yet unexplored context of direct democracy. This evidence can inform future studies drawing on citizens’ expectations as a prediction method.

Individual Factors

Several studies have highlighted a strong correlation between voting preferences and electoral expectations. While expectations can influence voting preferences through a “bandwagon effect”—i.e., when voters change their preferences in order to support the candidate that is expected to win (see, for example, Bartels, 1985; Granberg & Brent, 1983; Nadeau et al., 1993)—there is also evidence of the opposite process—namely, wishful thinking. In this case, voters adjust their expectations to predict more favorable results for their preferred candidates (e.g., Babad & Yacobos, 1993; Bartels, 1985; Krizan et al., 2010; Searles et al., 2018). Regardless of the direction of causality, we can expect that voting preferences also correlate with forecasts of a referendum result, with voters being more inclined to predict success for their favorite side. In the context of the Brexit referendum, we should therefore observe that supporters of the Leave option forecast a higher share of Leave votes compared to supporters of the Remain option (H1).

The second characteristic that has received scholarly attention is political sophistication. While sophistication should not influence the direction of forecasts in a systematic manner, it should affect the accuracy of forecasts both directly and indirectly. First, we can expect politically sophisticated citizens to provide accurate predictions since they are likely to have more well-developed prior judgments about public sentiment and better access to formal and informal instruments of public opinion measurement (such as exposure to poll results). Indeed, research shows that high education and political knowledge correlate with forecast accuracy in primary elections (Bartels, 1985) and presidential elections in the United States (Dolan & Holbrook, 2001; Lewis-Beck & Skalaban, 1989) and elections in Germany and Austria (Meffert & Gschwend, 2011; Meffert et al., 2011). If similar dynamics apply to direct-democratic votes, we can expect that political sophistication should be associated with greater accuracy in predicting the result of the Brexit referendum (H2A).

Political sophistication can also impact the accuracy of predictions indirectly by reducing the presence of wishful thinking in the formation of electoral forecasts. Although forming electoral expectations based partly on voting preferences might not necessarily lead to inaccurate predictions, it can nonetheless lead voters astray if their preferences are not aligned with the majority of the electorate. For example, wishful thinking can lead to overestimating the vote share of a minority party in parliamentary elections or a “minority option” in a referendum. In this sense, finding that voting preferences have a strong influence on electoral predictions should be an indication of inaccurate predictions. For example, research on presidential elections in the United States shows that the correlation between voting preferences and electoral expectations is weaker among highly educated voters compared to those with low education (Dolan & Holbrook, 2001; Granberg & Brent, 1983; Miller et al., 2012). Similarly, Meffert et al. (2011, p. 811) find that “knowledge and education reduce the tendency of wishful thinking” in predicting the results of national elections in Germany and Austria. Thus, we also hypothesize that, in the context of the Brexit referendum, sophisticated voters are less prone to wishful thinking—measured as the strength of the relationship between one’s own voting preferences and referendum expectations—than non-sophisticated voters (H2B).

Exogenous Factors

While individual voting preferences and political sophistication might correlate with the ability to predict referendum results, we are still left with the question of what actually influences citizen forecasts during a campaign. What factors lead citizens to predict a higher or lower vote share for a party, a candidate or a referendum option?

Previous studies have focused mainly on opinion polls, which influence electoral forecasts by conveying information on the expected voting behavior of fellow citizens (Blais et al., 2006; Irwin & van Holsteyn, 2002; Meffert & Gschwend, 2011). However, in everyday life, citizens also receive “social cues” from other sources, including interpersonal discussions with peers. Research shows that this type of discussion occurs primarily within networks of peers who share a similar political orientation (Beck et al., 2002; Huckfeldt et al., 2002; Mutz, 2006). Thus, we can expect that social cues are particularly persuasive if they come from people with similar political preferences, such as in-group partisans, in line with research showing that party cues from the in-group are more persuasive than cues from the out-group (for a review, see Merkley & Stecula, 2021). For these reasons, in our experimental design, we decided to test the effect of social cues concerning how fellow partisans would vote in the Brexit referendum. Given that our focus is on pro-Remain parties only (see design section below), we hypothesize that reminding partisans that a majority of in-party voters is expected to vote “Remain” will lead to higher forecasts of a Remain vote (H3).

In addition to social cues, we can expect cues from political elites to also influence forecasts. Besides long-standing research documenting how political parties influence public opinion (for a review, see Leeper & Slothuus, 2014), recent studies have provided robust causal evidence that political elites can influence citizens’ views on different policies (e.g., Bisgaard & Slothuus, 2018; Brader et al., 2020; Bullock, 2011; Colombo & Kriesi, 2017; Pannico, 2020). In the case of a referendum, we assume that if elite cues influence individual preferences, they will also influence individual expectations about the referendum result. As further discussed below, two mechanisms might be at play: a “rational” mechanism—insofar as citizens derive election forecasts based on the expected impact of elite messages in the electorate—and/or a mechanism of wishful thinking—if elite cues influence voters’ preferences, which, in turn, affect their expectations. Either way, this leads to the hypothesis that receiving information about where political elites stand in relation to policy decisions should influence citizen forecasts accordingly. Regarding our experimental design, we expect that reminding partisans that a majority of in-party elites is going to vote “Remain” will lead to higher forecasts of a Remain vote (H4).

If this line of reasoning is correct, we should observe that campaign arguments—even if unrelated to the expected outcome of an election—also affect citizens’ election forecasts. Extensive evidence indicates that political information can influence citizens’ political preferences in general (for a review, see Druckman & Lupia, 2016) and in referendum campaigns (Christin et al., 2002; Colombo & Kriesi, 2017; Hobolt, 2005; Kriesi, 2005; Morisi, 2016). If campaign arguments succeed in persuading a part of the electorate, they might also influence election forecasts through one of the channels underlying the effects of elite cues. Voters who anticipate that persuasive messages will change the preferences of part of the electorate might revise their own electoral expectations accordingly (a “rational” mechanism). For example, they might believe that certain anti-immigration rhetoric used by the Leave campaign will convince several people to vote Leave, thus increasing the forecast share of Leave votes. Alternatively, voters might themselves be persuaded by campaign messages and, in turn, adjust their expectations to accommodate their revised preferences (a mechanism of wishful thinking). Either way, as with elite cues, these mechanisms lead to the hypothesis that campaign messages will influence electoral expectations. Specifically, they imply that pro-Remain arguments will lead to higher forecasts of a Remain vote (H5A), while pro-Leave arguments will lead to higher forecasts of a Leave vote (H5B). These hypotheses find support in recent experimental evidence that partisan media coverage of election campaigns influences electoral predictions (Searles et al., 2018).

Lastly, political sophistication should play a crucial role in moderating not only the relationship between vote preferences and electoral predictions but also the effect of experimental stimuli. Numerous studies have demonstrated that political knowledge is a key moderator of political reasoning (see, for example, Barabas et al., 2014; Carpini et al., 1997). In particular, evidence shows that voters with low levels of political knowledge are more likely to be influenced by political cues (Kam, 2005; Lau & Redlawsk, 2001) and by the content of campaign messages (Slothuus & de Vreese, 2010). Thus, we can hypothesize that the effect of social cues, elite cues, and campaign arguments on electoral expectations will be stronger among non-sophisticated voters compared to sophisticated ones (H6).

Citizen Forecasts in the Brexit Referendum

In the following sections, we evaluate these hypotheses through an original study conducted in the context of the British “Brexit” referendum on EU membership held on June 23, 2016. In addition, we supplement the analysis with data from Wave 7 and Wave 8 of the BES Internet Panel (Fieldhouse et al., 2016). The Brexit referendum asked UK citizens: “Should the United Kingdom remain a member of the European Union or leave the European Union?” While non-binding (as the United Kingdom has no constitutional provision for binding referendums), the resulting vote with 51.9% for “Leave” and 48.1% for “Remain” started the process of Britain exiting the EU. Despite the expressed shock of many commentators, the referendum’s final result was within the range of the possible outcomes predicted by most opinion polls conducted in the two months before the vote (see Figure A2 in Appendix A).

The Brexit vote is a particularly useful case for examining citizens’ election forecasts because all the major parties except the UK Independence Party (“UKIP”) were officially pro-Remain, although the membership, leadership, and voters of both the Labour and Conservative parties were heavily divided. While analyzing referendum voting diverges from much of the citizen forecasting literature, which examines partisan election outcomes, it provides a useful extrapolation beyond these better-studied contexts in a manner that may be even more appropriate for inferring the causal influence of information on expectations. In particular, expectations in the Brexit case should not be strongly determined by long-standing knowledge of support for the “Leave” and “Remain” alternatives nor by heavily crystallized beliefs about partisan positions on the issue.

In the following section, we will present the observational analysis of our survey data and the data from the BES, followed by the results. We will then describe the design of the experiment included in our survey and the related results.

Observational Analysis: Measures

Between May 30 and June 3, 2016, we fielded an original survey via YouGov’s online Omnibus panel. For the Omnibus panel, YouGov recruits a nonrepresentative panel of participants who complete surveys in exchange for compensation and then adopts a quota sampling procedure for each individual survey, yielding a sample that is representative of the UK adult population (England, Scotland, and Wales) with respect to age, sex, education, and geographical region. In total, 3,385 respondents completed our survey.

To measure citizens’ overall expectations about the referendum result, we asked the respondents to freely guess the percentage of the electorate that would have voted “Leave” in the referendum. The question reads as follows:

We would now like to know what you expect the results of the referendum will be, expressed as a percent. A percent can be thought of as the number of votes out of 100. For example, a number like 5 percent means 5 out of every 100 votes will be for “Leave”, 50 percent means 50 out of every 100 votes will be for “Leave,” and 95 percent means 95 out of everyone 100 votes will be for “Leave.”

Regardless of how you yourself intend to vote, what percent of voters do you anticipate will vote for Britain to leave the European Union? (Please enter a number between 0 and 100.)

The responses to this question represent our measure of “Leave forecasts.” In addition, we created a measure of forecast accuracy by first subtracting the actual referendum result (corresponding to 51.9 percentage points) from the recorded value and then taking either the squared result or the absolute value.

We test our first and second hypotheses in a simple cross-sectional regression approach. Concretely, we regress Leave forecasts and forecasts accuracy on our key independent variables (i.e., voting preferences and political sophistication), controlling for standard socio-demographic variables (age, gender, ethnicity, and gross household income), assignment to treatment conditions, party identification, and the day of the interview. We employ the last control because research indicates that respondents provide more accurate forecast estimates as election day approaches (Lewis-Beck & Tien, 1999). We use two proxies for political sophistication: a categorical variable for the respondents’ level of education and an 11-point measure of how much attention the respondent pays to politics, which we rescaled from 0 (minimum) to 1 (maximum).

In Appendix B, we replicate the observational analysis using data from Wave 7 and Wave 8 of the BES Internet Panel (Fieldhouse et al., 2016), in which the respondents were asked to provide the probability that the UK would vote “Leave.” This question does not measure the predicted share of Leave votes but the perceived probability that a majority of voters—regardless of the size of the majority—will vote Leave. In this sense, it does not strictly provide a measure of citizen forecasts. However, if we consider that individuals have difficulties understanding probabilities (Visschers et al., 2009) and often misinterpret probabilities as proportions (Gigerenzer et al., 2005), we suspect that most respondents replied to this question by giving their best estimate of the share of Leave votes. This is particularly plausible since the respondents had to reply using a scale from 0 to 100—which could be easily equated to a scale of vote share—instead of choosing from a series of fully-labeled categories such as “very likely” or “very unlikely.” For these reasons, we cautiously treat this question as a proxy for a forecast estimate and use it to replicate our analysis of the correlates of citizen forecasts in the Brexit referendum.

Observational Analysis: Results

Figure 1 shows a density plot of the forecast share of Leave votes in the Brexit referendum.Footnote 3 Despite a peak of the distribution below the 50% threshold, the average, weighted forecast slightly favors the Leave side (at 50.6%), falling short of the true referendum vote share for Leave (51.9%). Some respondents forecasted the chance of a Leave vote at 100%, while a few others predicted a 100% chance of a Remain vote. If we trim the variable to include only those who reported forecasts above 10% and below 90%, the average forecast shifts only slightly to 50.3% Leave votes (see Table A1 and Figure A1 in Appendix A).

Fig. 1
figure 1

Forecast share of “Leave” votes. Distribution of forecasts of Leave votes. Respondents from control group only. Black line: Epanechnikov Kernel density

Full size image

When we replicate the analysis using data from the BES Internet Panel, we obtain very similar results. The average forecast probability of Britain voting Leave from Wave 7, which was conducted around two months before the referendum, is exactly 50%, while one month later, in Wave 8, the respondents’ average expectation rose to 50.6% (see Figures B1 and B2 in Appendix B), again falling short of the actual result of the referendum.Footnote 4

Moving beyond the overall distribution of forecasts, we investigate whether voting preferences correlate with forecasts of the referendum result and if political sophistication moderates the impact of such preferences. The left-hand plot of Fig. 2 shows that supporters of the Remain and Leave sides differ starkly in their predictions, in line with our first hypothesis (H1). While Remain voters predict that only 45% of voters will choose Leave, Leave voters predict that 54% will do so. However, the gap between Remain and Leave voters changes by level of education, indicating that political sophistication moderates the impact of voting preferences on vote predictions, in line with hypothesis H2B. As the center and right-hand plots of Fig. 2 show, the “prediction gap” between Remain and Leave voters is 3 percentage points smaller among highly educated respondents compared to those with low education. The change occurs mostly among Leave voters with high education who are less optimistic about their side’s prospect of winning than Leave voters with low education.

Fig. 2
figure 2

Forecast share of “Leave” votes by voting preferences and education. Estimates based on OLS regressions (Table A2 in Appendix A), controlling for socio-demographic variables (age, gender, ethnicity, and gross household income), assignment to treatment conditions, party identification, and the day of the interview. Center and right-hand plots: estimates based on interaction between voting preferences and low versus high education (Model 2 in Table A2). Vertical bars are 95% confidence intervals

Full size image

When we interact voting intentions with attention to politics, we obtain a similar negative interaction, indicating that the impact of voting preferences on forecasts is smaller among those who pay close attention to politics compared to those who seldom follow it (see Model 3 in Table A2 in Appendix A). Replication with data from the BES Internet Panel provides a very similar picture: both education and knowledge moderate the impact of voting preferences on predictions (see Tables B2 and B3 in Appendix B). These findings provide clear evidence that political sophistication influences forecasts indirectly (H2B) by reducing the impact of voting preferences on predictions of the referendum result. If we interpret the correlation between preferences and expectations as an indication of wishful thinking, these findings suggest that politically sophisticated voters are less prone to wishful thinking in predicting the result of a referendum.

Lastly, we investigated whether political sophistication also influences the accuracy of prediction directly by increasing the likelihood that individuals guess the outcome of the referendum correctly. Figure 3 displays the result from regression of forecast accuracy on several factors, with positive coefficients indicating higher accuracy (smaller forecast error) and vice versa for negative coefficients. We find that both higher levels of education and paying attention to politics are associated with a higher probability of correctly guessing the result of the referendum, in line with hypothesis H2A. Furthermore, besides political sophistication, we find that older people and wealthier people provide more accurate forecasts than their respective counterparts.Footnote 5

Fig. 3
figure 3

Correlates of Forecast Accuracy. Estimates from OLS regression Model 1 in Table A3 in Appendix A. Forecast accuracy rescaled from 0 (minimum accuracy) to 100 (maximum accuracy). Horizontal bars are 95% confidence intervals

Full size image

Experimental Analysis: Design

We designed an experiment to assess how elite cues, social cues, and campaign arguments influence citizens’ forecasts and included it in our survey. The respondents were randomly assigned to one of six experimental conditions (or an untreated control condition) that varied their exposure to either an “elite cue” or a “social cue.” Those assigned to the control condition simply proceeded to the outcome question. In each experimental condition, we matched the name of either party members or party voters with the respondent’s party identification, as measured before treatment. To avoid deception, we focused only on partisans of the four major parties that took official pro-Remain positions—namely, the Conservatives, Labour, the Liberal Democrats (“Lib Dems”), and the Scottish National Party (“SNP”). Thus, our experimental sample (N = 1125) does not include non-partisans and supporters of pro-Leave parties, such as UKIP supporters.Footnote 6 Despite the exclusion of these respondents, however, our sample includes a balanced distribution of respondents who intend to vote “Remain” (46%) and respondents who intend to vote “Leave” (42%). Furthermore, we find that the main determinants of both forecast accuracy and Leave forecasts also apply within the experimental sample (see Table A4 in Appendix A).

Those assigned to the elite cue received the following information about the positions taken by within-party members of Parliament in the referendum:

When deciding how to vote in the upcoming referendum on Britain’s membership of the European Union, many voters want to know where their preferred party stands. A clear majority of “PARTY” politicians favour Britain remaining in the EU.

Those receiving a social cue, on the other hand, were informed about the expected voting intentions of co-partisans, according to opinion polls. It reads as follows:

When deciding how to vote in the upcoming referendum on Britain’s membership of the European Union, many voters want to know where their fellow citizens stand. Polls show a clear majority of “PARTY” voters favour Britain remaining in the EU.

This factor was crossed by a second factor that manipulated whether respondents additionally received either a pro-Remain or a pro-Leave argument (or no such argument). These arguments were chosen after pretesting 16 arguments using an online convenience sample of 80 participants recruited from Prolific (see Appendix D for complete details). The Remain and Leave arguments that were rated as most effective in the pilot study were used in the main experiment. The Remain argument read:

One argument being made in the debate is that the EU safeguards British jobs because it provides access to a market of 500 million consumers and because EU membership attracts foreign firms keen to be part of that market. The attractiveness of Britain as a place to invest is clearly underpinned by its membership of the EU. It is estimated that over three million jobs in Britain are linked, directly or indirectly, to its exports to the European Union. Walking away from Europe’s single market would be catastrophic for people’s jobs, and would leave households £4,300 worse off, according to estimates. A vote to Remain would safeguard the economic benefits of the EU single market.

The Leave argument read:

One argument being made in the debate is that in the EU, Britain’s borders lay open to criminals and terrorists trying to enter the UK from the continent. This makes the whole of the UK vulnerable to terrorist attacks and crimes committed by those from abroad. At present, more than 100 EU migrants per day are convicted of crimes ranging from theft to rape and murder. These rates have risen as the EU has expanded further into Eastern Europe. Outside the EU, the Westminster parliament will regain its sovereignty and the ability to secure the country’s borders and towns. Failure to leave now significantly decreases public safety and endangers the British people.

While these arguments vary in their particular topic, they were rated as similarly effective by the pilot test respondents. Given that we are not considering this as a framing experiment (but rather a study of responsiveness to substantive arguments), we are unconcerned by the fact that the messages supply slightly different information and prime different aspects of the EU. Together, these manipulations produce a 2 × 3 factorial design plus an untreated control (N = 151), as described in Table 1.

Table 1 Experimental conditions
Full size table

In the regression analysis, the dependent variable is the forecast share of Leave votes, ranging from 0 to 100. To increase the precision of the estimates (see Angrist & Pischke, 2009; Kam & Trussler, 2017), in Appendix C, we estimate average treatment effects controlling for the same set of covariates used in the observational analysis. The covariate-adjusted estimates are substantially the same as the non-adjusted estimates.

Lastly, as a manipulation check, we rely on two questions asking the respondents to estimate the percent of in-party politicians and in-party voters who will vote for leaving the EU. These questions followed our outcome variable and were worded as follows: “What percent of [in-party] voters/[in-party] politicians do you anticipate will vote for Britain to leave the European Union? (Please enter a number between 0 and 100).” The analysis of the responses to these questions confirms that our manipulations “worked” in the expected direction since both pro-Remain social cues and pro-Remain elite cues increased the respondents’ estimates of how many in-party voters and in-party elites would cast a Remain vote (see Table C6 in Appendix C).

Experimental Analysis: Results

According to our hypotheses, both elite and social cues should affect forecasts by providing voters with information about how the majority of like-minded party leaders and fellow partisans are expected to vote. Given that the participants received only pro-Remain cues, we should expect an increase in the estimate of the share of Remain votes (or, conversely, a decrease in the estimate of Leave votes).

Figure 4 illustrates the average treatment effects relative to the control condition. As the estimate about the “social cue” condition shows, receiving information about co-partisans’ voting intentions (as provided by opinion polls) does not alter respondents’ expectations about the general result of the referendum. Thus, we do not find evidence in favor of hypothesis H3.

Fig. 4
figure 4

Treatment effects on Leave forecasts relative to control. Estimates based on OLS regression Model 1 in Table C1. The vertical dotted line corresponds to the control group. Thick/thin horizontal bars correspond to 90%/95% confidence intervals

Full size image

A possible explanation for this “null finding” might be the low credibility of the opinion polls in the UK at the time the study was conducted since major polls missed the final result of the 2015 UK general election by a considerable margin (Mellon & Prosser, 2017). In this sense, the respondents might have simply discounted the information included in the treatment as not credible since it was based on opinion polls. However, in this case, social cues should not affect any vote estimates, while our manipulation check shows that social cues do influence respondents’ estimates of how many in-party voters will cast a Leave vote (see Table C6 in Appendix C).

We further inspected whether social cues have heterogeneous effects depending on party identification since learning that a majority of voters belonging to a small party favors “Remain” should have a different impact than learning the same about voters from a large party.Footnote 7 However, we do not find differences by identification with either small or large parties (see Table C5 and Figure C2 in Appendix C).

Elite cues, on the other hand, substantially affect citizens’ forecasts, as Fig. 4 reveals: receiving information that a majority of within-party politicians support the UK’s membership of the EU significantly reduces participants’ forecasts of Leave votes by around 3.6 percentage points (corresponding to 0.27 standard deviations). While the participants in the control group estimated that, on average, 50.6% of the electorate would have voted Leave, those in the elite cue condition estimated that only 47% would have voted that way. In other words, receiving a consistent signal about the position taken by political elites in a referendum campaign changes the expectations of in-party supporters about the general result of the referendum. This finding supports the hypothesis that citizens change their election forecasts in line with the signals received from political elites (H4).

When we inspect heterogeneous effects, we find that elite cues influence mostly Labour partisans and, to a lesser extent, Conservative partisans, while they do not affect Lib Dem and SNP supporters (see Table C5 and Figure C2 in Appendix C). We can interpret these findings either as evidence that only elite cues from large parties can change electoral forecasts or as evidence that receiving clear information about where party leaders stand is particularly effective if parties are divided. We cannot disentangle the two possibilities since the Conservative and Labour parties were both the largest parties and the parties that were at least partially divided during the Brexit campaign.

Furthermore, the findings in Fig. 4 indicate that campaign arguments also influence referendum forecasts. While the effect of the elite cue combined with a pro-Remain argument is substantially the same as the effect of the elite cue presented in isolation, a social cue combined with a pro-Remain argument significantly and substantially changes participants’ expectations about the outcome of the referendum by reducing Leave forecasts. On the other hand, a pro-Leave argument appears to “cancel” the effect of elite cues.

To estimate the net effect of campaign arguments, we conducted further regression models using either the “elite cue” or the “social cue” conditions as baseline categories. The results show that a pro-Remain argument does not have a significant “additive” effect in the presence of elite cues. However, the same argument reduces participants’ Leave forecasts when social cues are present, although the effect reaches statistical significance only when covariates are included in the model (see Table C2 in Appendix C). On the other hand, receiving a pro-Leave argument significantly and substantially increases forecasts of Leave votes by 4 percentage points when the argument is presented in combination with elite cues (but not when social cues are present). These findings partially support our hypotheses H5A and H5B that exposure to campaign arguments changes forecasts of a referendum result, even if this type of information is unrelated to the possible outcome of the referendum.

In the next step of the analysis, we interacted assignment to treatment with a dummy variable for education to test whether political sophistication moderates treatment effects. A statistically significant interaction between elite cues and education indicates that the effect of elite cues differs by education category (see Table C3 in Appendix C). When we plot the average marginal effects by education categories, we find that the effects previously identified in the entire sample are larger and statistically significant only among those with low education. As Fig. 5 shows, the elite cue alone and both elite and social cues combined with a pro-Remain argument substantially shift the expectations of low-educated voters toward a more favorable outcome for the Remain side. On the other hand, treatment effects among highly educated voters are smaller and not statistically significant. When we replicate the analysis interacting treatment assignment with political attention instead of education, we obtain very similar results (see Table C4 and Figure C1 in Appendix C). These findings are in line with our hypothesis that political sophistication moderates the effects of information on electoral expectations (H6), providing further evidence that low-sophisticated voters are especially influenced by political cues and campaign arguments.

Fig. 5
figure 5

Treatment effects on Leave forecasts by education levels. Estimates based on OLS regression Model 1 in Table C3. Assignment to treatment interacted with a dummy variable for low (below A level) versus high (A level or above) education (for complete regression models, see Table C3 in Appendix C). Thick/thin horizontal bars correspond to 90%/95% confidence intervals

Full size image

Lastly, we inspected whether our treatments affected forecast accuracy. While we do not find effects when considering all the respondents, we find that accuracy increases among those with low education once they receive social cues in isolation or combined with campaign arguments (see Table C9 and Figure C3 in Appendix). These effects are highly contextual and can be explained by the fact that low-sophisticated voters overestimated the Leave vote share to start with, as the data from the control group show (see Panel B in Table C3). Thus, by shifting the forecast toward the Remain side, the treatments also increased the accuracy of predictions among this group of voters. On the other hand, highly-sophisticated voters underestimated the Leave vote share, but their forecast accuracy did not change since they were generally not affected by the treatments.

Mechanisms

What is the mechanism underlying the effect of elite cues and campaign arguments on forecasting direct-democratic decisions? As mentioned above, we can hypothesize that citizens rationally derive electoral forecasts based on the expected impact of elite and campaign messages on the electorate. For example, if political leaders repeatedly declare their voting preferences or if persuasive campaign messages are frequently voiced in the public debate and the media, these appeals might shift the opinion of a certain number of voters, which, in turn, will affect the final voting outcome. Thus, voters exposed to these messages might adjust their electoral expectations if they anticipate the “social” impact of information on the general electorate beyond the impact on their individual preferences.

However, it is also possible that elite cues and campaign arguments affect citizen forecasts through a mechanism of wishful thinking that operates in two steps: first, information influences voters’ attitudes on a referendum proposal, and second, voters adjust their forecasts to accommodate their revised attitudes. Following this logic, for example, a persuasive pro-Remain argument should first convince a voter of the benefits of remaining in the EU. Second, the same voter should then align her expectations about the referendum result by predicting a larger Remain vote share. Although our experimental design does not allow us to test this mechanism directly, we have evidence that both Remain and Leave arguments significantly influence respondents’ attitudes on the United Kingdom’s EU membership as measured after treatment, thus suggesting that wishful thinking might be a possible explanation for the identified information effects (see Table C7 in Appendix C). Indeed, a mediation analysis shows that a substantial share of the effect of pro-Leave arguments on Leave forecasts is mediated by the respondents’ attitudes on EU membership, thus lending further support to the presence of wishful thinking in the formation of referendum forecasts (see Table C8 in Appendix C).

Implications for Voting Behavior

In the last section, we briefly analyze the implications of citizen forecasts for voting behavior in referendums. In line with elections for parties (Meffert & Gschwend, 2011; Meffert et al., 2011), electoral expectations might lead to strategic voting also in referendums if a turnout threshold is present. For example, those who predict a low turnout and are against a referendum proposal might strategically decide to stay home instead of voting “No”, since not casting a vote can be perceived as a more effective way of making the referendum proposal fail. However, these strategic considerations do not apply to the Brexit referendum since there was no formal turnout threshold. In addition, and contrary to elections in multi-party systems, referendums do not offer voters the possibility to choose a “second-best” option with higher chances to win than the first-preferred option since the outcome is binary.Footnote 8

Still, forecasts of referendum results might lead to bandwagon effects if voters shift their voting preferences to support the side that is most likely to win. Regression analysis suggests that the probability of voting Leave (versus Remain) increases as respondents’ forecasts of Leave votes increase as well, after controlling for numerous factors (see Table A5 in Appendix A). Although we cannot establish whether it is voting preferences that influence forecasts, or vice versa, it is conceivable that some voters decided to vote Leave once they expected that that side was going to win, in line with evidence of bandwagon effects (Bartels, 1985; Granberg & Brent, 1983; Meffert et al., 2011; Nadeau et al., 1993).

Furthermore, forecasting the result of a referendum can affect turnout, especially if voters predict a close competition (see Moy & Rinke, 2012 for a review of positive effects of election closeness on turnout). If voters predict a very close result, they might have an incentive to turn out to vote, as every single vote can be crucial for the final outcome. Indeed, we find a positive correlation between the likelihood of turning out and the prediction of a neck-and-neck outcome in the Brexit referendum: The more voters believe that the final result will be close to 50%, the more likely they are to go to vote, after controlling for voting intentions, party identification and other socio-demographic factors (see Table A6 in Appendix A). These results suggest that citizen forecasts in a referendum campaign can have relevant consequences for both voting decisions and voter turnout.

Conclusions

The present research has shown that individuals can use various kinds of information to form expectations about others’ political attitudes and behavior. These results thus contribute to an ongoing theoretical debate about the ways in which individuals’ political beliefs, attitudes, and behaviors influence and are influenced by their social connections (Bolsen, 2013; Mutz, 2002; Ryan, 2011). Focusing on expectations about voting in Britain’s 2016 referendum on membership of the European Union, we confirm what recent research has shown, namely that the average citizen forecast draws near the electoral outcome. Around three weeks before the Brexit referendum, British voters’ predicted outcome fell short of the actual result by only 1.3 percentage points.Footnote 9 However, important heterogeneity in individual predictions exists. In particular, we find that politically sophisticated, in addition to older people and wealthier voters, predict the referendum result more closely than their relative counterparts. These findings contribute to the emergent literature on citizen forecasting (Graefe, 2014; Irwin & van Holsteyn, 2002; Murr, 2011, 2016; Murr & Lewis-Beck, 2021) by showing that some voters are able to forecast election results more accurately than others. Further research should explore additional individual factors that might explain forecasting accuracy, such as interpersonal discussion, political involvement, media consumption, or numeracy, using a number of measures that were not available in our study.

Besides these correlational findings, we provide novel experimental evidence that political elites can influence citizens’ forecasts of a referendum vote. Our results show that partisans adjust their forecasts in response to signals from within-party politicians: Knowing how the majority of within-party members will vote in a referendum changes partisans’ expectations about the general result of the referendum. This finding adds another piece of evidence to research on the effects of political elites (e.g., Bisgaard & Slothuus, 2018; Brader et al., 2020; Bullock, 2011; Colombo & Kriesi, 2017; Pannico, 2020) by showing that elite cues can influence not only public opinion and voting behavior, but also citizens’ expectations about election results, at least in a referendum campaign.

When informed about other co-partisans’ expected voting intentions (as provided by opinion polls), however, our respondents did not change their forecasts of the referendum outcome. It is possible that voters consider a relative shift of vote preferences within their partisan in-group to be not sufficient to influence the final result of a referendum, thus the lack of effect on the general forecast. However, this null finding does not preclude the possibility that social cues related to other types of in-group identities—defined, for example, in terms of social class, age, or national identity—might influence electoral expectations. Further research should address this limitation of our experimental study by testing the effect of social cues beyond partisan in-groups.

Lastly, we find that in forming expectations about a referendum result, voters also rely on policy arguments, even if these arguments do not concern the projected outcome of the referendum. In line with recent studies (Searles et al., 2018), we find partial evidence of a mechanism of wishful thinking, in which, first, information changes respondents’ attitudes, and second, respondents adjust their forecasts to accommodate their attitudes. These findings contribute to research on information effects by indicating that campaign arguments and policy information can shape not only voters’ attitudes but also their expectations about political outcomes.