Upset but (almost) correct: a conceptual replication of Di Tella, Perez-Truglia, Babino and Sigman (2015)

Abstract

This paper provides a close conceptual replication of the study by Di Tella et al. (Am Econ Rev 105: 3416–42, 2015) on self-serving beliefs. The design differs in some aspects from the original study, but maintains its fundamental structure and uses a larger sample size. The main findings of the original study are not replicated. If anything, beliefs seem to be biased in the opposite direction. These results are discussed jointly with two other replication efforts by Ging-Jehli et al. (Games Econ Behav 122–341, 2020) and Ahumada et al. (Well excuse me! replicating and connecting excuse-seeking behaviors, 2022). The main conclusion is that self-serving beliefs about others in strategic settings seem to be quite sensitive and hard to capture.

1 Introduction

In recent years, a large body of literature has researched the conditions under which people display moral behavior and those under which they act in their own self-interest. It has become clear that people often develop biased views about the world around them to resolve the tension between being or appearing moral and pure self-interest (Miller & Ross, 1975; Kahan et al., 2012; Gino et al., 2016). Examples of such self-serving beliefs include self-serving manipulation of fairness arguments (Konow, 2000), of ambiguity (Haisley & Weber, 2010) and of risk (Exley, 2015).

Another way in which people may justify their selfishness is by biasing their beliefs about the behavior of the people they interact with. The idea is simple: if the other is unfair or bad, then I can be unfair or bad in return.¹ Di Tella et al. (2015) investigate the formation of such self-serving beliefs justifying unfair behavior in a modified Dictator Game, the “corruption game”. In the “basic” version of the game, an Allocator decides how to split 20 tokens between herself and a Seller. The Seller decides how to convert the tokens into money. She can decide to convert them at a high conversion rate (1 token = 2 AR$ (Argentinian Pesos)) or a lower one (1 token = 1 AR$), in which case she receives a side-payment or “bribe” (10 AR$). Importantly, the Allocator is asked to state her beliefs about the behavior of the Seller she is paired with and about the behavior of the Sellers in her session. A self-serving bias in beliefs can be identified by comparing two between-subjects treatments. In both treatments, 10 tokens are assigned to the Allocator and 10 to the Seller. In the Able=8 treatment, an Allocator can give or take up to 8 tokens. In the Able=2 treatment, the ability to take is restricted, and an Allocator can give or take only up to 2 tokens.

Crucially, Sellers do not know about these restrictions and believe that Allocators can give or take any amount they want in both treatments. In turn, Allocators know that the Sellers are not aware of any restrictions. An Allocator who cares about her self-image (Bénabou & Tirole, 2006) and wants to allocate more tokens to herself in the Able=8 might self-servingly believe that the Seller chose to take the bribe. The authors find that, indeed, Allocators in the Able=8 treatment think that more Sellers chose the unfair option compared to the Able=2 treatment.

This paper presents a close conceptual replication (Crandall & Sherman, 2016; DellaVigna & Pope, 2019; Derksen & Morawski, 2022) of the basic corruption game of Di Tella et al. (2015). This means that some details of the design were different (as described in Sect. 2), but the main spirit of the experiment remained intact. As such, the present experiment can be seen as a robustness or sensitivity check of the theoretical construct rather than a direct replication.² The experiment was well powered (80%) based on the original effect size and also in comparison to the original study, but fails to reproduce the main findings.³ Allocators’ beliefs, in this experiment, are not self-servingly biased. If anything, Allocators in the Able=8 treatment have less negative beliefs about Sellers, i.e., were less upset, than Allocators in the Able=2 treatment.

2 Experimental design

The basic corruption game of Di Tella et al. (2015) was run with only some changes in the design to maintain the basic structure of the experiment. The changes are described in the following and a summary can be found in Table 1. The main task of the experiment remained substantively unchanged. Allocators and Sellers earned 10 tokens each through a real effort task. Then, each Allocator could choose how to distribute the 20 tokens between herself and the Seller matched with her. Her choice was restricted depending on the treatment. In the Able=8 (Able=2) treatment, she could give or take up to 8 (2) tokens. The Seller simultaneously decided how to convert the tokens into Euros. Each token was worth 0.50 € if she chose the high conversion rate (Option 1). If she chose the low conversion rate, instead, each token was only worth 0.25 €, but she got an additional payment of 5 € just for herself (Option 2). A first difference concerns the fact that this side-payment is higher than in the original study. however, this does not change the strategic nature of the game and, in some respects, it may even increase the potential for holding a negative belief about the Seller.⁴ A second difference regards the way in which the restriction in the choice set of Allocators was communicated to them. In this study, Allocators were told from the beginning that they could give or take either 2 or 8 tokens, depending on the treatment they were in. On the contrary, in the original design of the basic game, Allocators were led to believe they could give or take any amount and learned about the restriction only upon making their allocation decision, without an explanation as to why this was the case. The restriction was communicated in yet a different way in the “modified game” that Di Tella et al. (2015) use as a robustness check, where Allocators were told about the existence of both treatments and that they had been assigned to one of the two. The most important difference between the basic and the modified game in the original study is that, while the corrupt option is a weakly dominant strategy in the basic corruption game (provided Allocators only take from and do not give to the Seller), in its modified version the optimal action of the Seller depends on how much the Allocator takes from her.

After the main task and before knowing the actual outcome of the interaction, the beliefs of the Allocator were elicited in two ways. First, Allocators were asked which option they thought the Seller matched with them had chosen. This question was not incentivized, as in the original study. Second, Allocators made an incentivized guess of the proportion of Sellers who took Option 2 in their session. A third difference between this study and the original one concerns the way in which these beliefs were incentivized. Allocators in the original study were asked to estimate the percentage of Sellers who chose the unfair option in their session and were given 10 predefined brackets going from 0–10% to 90–100%. Subjects were rewarded if the actual percentage of Sellers choosing the unfair option fell in the bracket they had chosen. While easy to understand, this method has one major drawback. Depending on the number of subjects in a session, a single bracket can contain no correct response at all or more than one, making the brackets unequally likely and potential distorting incentives. This introduces noise in the elicitation procedure. To avoid this, in this study, Allocators were told the actual number of Sellers present in their session and were asked to estimate the exact number of Sellers who chose the unfair option. The shift from percentages to natural numbers should not interfere with the formation of self-servingly biased beliefs, which Allocators were likely to have formed already before they reach the beliefs elicitation stage. Correct beliefs in the incentivized elicitation were rewarded with 3 €.

The instructions were translated to German and the use of loaded language was avoided (see Supplementary file), which constitutes a fourth change to the design of the original study.⁵ The Allocator and the Seller, as they were called in the original instructions, were simply referred to as participant A and B. The two options the Seller could choose from were described neutrally. On the contrary, in the original instructions, Allocators were told that the experimenter would have preferred the Seller to take Option 2. Note that this was not the way the options were described to the Seller, to whom they were presented neutrally, and also note that Allocators saw the Seller’s instructions. Option 2 was also not described as the experimenter’s preferred option in the modified game of the original study. A last minor difference compared to the original study is that a different real effort task was used at the beginning of the experiment. In the original study, participants had to find a given hidden sequence of 0s and 1s in a series of 0s and 1s. In this study, participants, instead, had to complete a computerized slider task (see Supplementary file). As in the original experiment the task lasted for around 1 minute and participants earned 10 tokens by completing it.

Table 1 Summary of main changes to original design

2.1 Procedure

The experiment was run at the Cologne Laboratory for Economic Research (CLER) in June and July 2018. The entire experiment was programmed using z-Tree (Fischbacher, 2007). Participants were recruited via ORSEE (Greiner, 2015). A total of 118 subjects (55% female, average age 24.9 years) took part in the experiment across 8 sessions.

Subjects received on-screen instructions. After reading the instructions they went through a series of control questions. Once they answered all questions subjects received personalized, on-screen feedback about their answers. They were told whether their answers were correct or not, and, in the latter case, what the correct answer was. The experiment started only after all participants had finished this phase and lasted between 30 and 40 minutes overall. Subjects earned 9.25 € on average, including a show-up fee of 4 €.

3 Results

Table 2 Comparison of Allocators’ behavior and beliefs

Table 2 offers a comparison between the original study and the present one, reproducing the main analysis presented by the authors in their paper. It compares Allocators’ average behavior and beliefs in the Able=2 and Able=8 treatment. Allocators in the present study took slightly fewer tokens in both treatments compared to the original study. More strikingly, there is a failure to replicate the pattern of results concerning Allocators’ beliefs about Sellers. The average difference between the two treatment conditions for the unincentivized measure (Is Corrupt) goes in the opposite direction and is statistically significant. 86% of Allocators in the Able=2 treatment thought that the Seller they were paired with chose the unfair option, while only 57% of Allocators in the Able=8 treatment believe so. The same pattern is also present in the incentivized measure (%-Corrupt), although the difference between the two averages is not statistically significant. However, a comparison between the distributions of the incentivized beliefs measure (%-Corrupt) in the Able=2 and Able=8 treatment yields a significant difference (see “Appendix A”).

4 Discussion and conclusion

This close conceptual replication was more highly powered than the original experiment, but failed to replicate its findings. If anything, the unincentivized beliefs measure (Is Corrupt) suggests that Allocators who were able to take only little (Able=2 treatment) were more upset than those who could take more (Able=8 treatment). In fact, beliefs in the Able=2 treatment are objectively less biased, since they are closer to the actual share of Sellers who choose the unfair option (80 %). It is also interesting to note that beliefs about Sellers are in general more negative in the present study.

Table 3 Comparison of Allocators’ beliefs (%-Corrupt) for Equalizers and Takers

Digging deeper in the data of the present experiment, the right part of Table 3 shows a comparison of Allocators who decided to split the tokens equally (Equalizers) with those who took at least one token from the Seller (Takers).⁶ As one would expect, Takers hold more negative beliefs compared to Equalizers. If beliefs are used in a self-serving way, they should be more negative in the Able=8 treatment compared to the Able=2 treatment and this difference should come from Takers who use their beliefs to justify taking more. However, Takers hold almost the same beliefs across the two treatments ($p=0.61$). Interestingly, the percentage of Sellers who actually choose the unfair option in the present experiment is 80%, which almost perfectly coincides with the average belief of Takers across the two treatments (81%). The difference between the two treatments comes from Equalizers, who hold much more negative beliefs than their peers in the Able=2 treatment compared to the Able=8 treatment ($p=0.01$). This evidence seems to suggest that Allocators who equalize the tokens are rather “optimistic” in their beliefs about Sellers and become more “realistic” when they take.

The left part of Table 3 reconstructs the same comparison with the original data of the basic game.⁷ As in the present study, there is no significant difference in Takers’ beliefs ($p=0.28$). A similar test for Equalizers cannot be performed, since there is only one observation in the Able=8 treatment. However, it seems that while Equalizers in the present study had more negative beliefs in the Able=2 compared to the Able=8 treatment, the data in Di Tella et al. (2015) go in the opposite direction.⁸

A potential explanation for the pattern observed in this study could be that beliefs are not used self-servingly, but rather that subjects simply trade off the possibility to increase their payoff against the loss (gain) in image that comes from (not) taking. This would lead to a different composition of the groups of Equalizers and Takers in the two treatments. In line with what is reported above, beliefs of Equalizers in the Able=2 treatment would then be more negative compared to those of Equalizers in the Able=8 treatment, in which only Allocators who have very positive beliefs would still split the tokens equally. Beliefs of Takers, instead, would not necessarily change. However, there are two points that speak against such an explanation. First, the proportion of Equalizers in the Able=8 treatment should be lower compared to the Able=2 treatment, which is not the case in the data. Second, average beliefs should not change across the two treatments, which there is also some weak evidence against.

To provide a broader view on the phenomenon of self-serving beliefs, the results of two further studies based on the paper by Di Tella et al. (2015) are discussed. In a paper by Ging-Jehli et al. (2020), the authors study the role of self-serving strategic beliefs in a “pre-emptive taking game” and also replicate the modified corruption game of Di Tella et al. (2015).⁹ While they find no evidence of self-serving beliefs in the former game, they are able to replicate the same results of the original study for the latter. In addition, Ging-Jehli et al. (2020) also elicit beliefs about the Sellers’ behavior from third parties, who are not involved in the game, and find no difference between the beliefs of Allocators in the Able=8 treatment which are supposed to be biased self-servingly and those of third parties. Moreover, these beliefs are closer than those of Allocators in the Able=2 treatment to the true share of corrupt Sellers. The authors interpret this as evidence against the presence of an absolute self-serving bias and in favor of a “positivity” bias. Hence, it is not subjects in the Able=8 treatment who are upset, but rather those in the Able=2 treatment who have too rosy expectations about the Sellers.

Ahumada et al. (2022) also replicate the modified corruption game by Di Tella et al. (2015) alongside two other experiments on excuse-driven behavior (i.e., Exley (2015) and Dana et al. (2007)). The authors only find a marginally significant effect in the same direction as in the original study for the belief about the Seller an Allocator is matched with, which they incentivize. They find no significant effect on the main beliefs measure about the share of Sellers choosing the corrupt option at the session-level. Ahumada et al. (2022) also investigate whether excuse-driven behavior across the three experiments is related and find no positive association. They conclude that this may be driven by the sensitivity of such behavior to the type of measurement used.

In conclusion, while it is not in doubt that self-serving beliefs arise in certain settings (see, Gino et al. (2016)), these phenomena seem to be quite sensitive and not very easy to capture. Future research should investigate the formation of self-serving beliefs to understand the precise mechanisms that cause them and their bounds.

Appendix A: Further analysis

Figure 1 displays the cumulative density function of %-Corrupt. The solid line corresponding to the Able=2 treatment lies always to the right of the dotted line of the Able=8 treatment, meaning that the whole distribution of the incentivized beliefs measure is shifted to the right in the Able=2 treatment compared to the Able=8 treatment (Chi-squared test with simulated p-value, p = 0.01). This confirms the observations made for Table 2 and shows that the shift in beliefs is substantial and goes in the opposite direction compared to the original study.

Fig. 1

Ecdf plot of %-Corrupt