In a widely discussed article, Machery, Mallon, Nichols, and Stich (2004, MMNS) argued that the common practice of appealing to one’s own intuitions about cases as evidence for or against philosophical theories of reference is suspect. Specifically, MMNS targeted a standard justification for this practice. They noted that philosophers of language often assume that the relevant semantic intuitions are widely held and do not systematically differ across theoretically salient groups, taking such widespread agreement to indicate that the intuitions are reliable. Following Sytsma and Livengood (2011), we will refer to this assumption as the uniformity conjecture. MMNS presented empirical evidence suggesting that the uniformity conjecture is false: Intuitions about the semantic reference of a proper name in one prominent case from the literature (Kripke’s Gödel case) differ in a statistically significant way between Western and East Asian cultures.

In response, Sytsma and Livengood (2011) presented experimental evidence indicating that a perspectival ambiguity in the questions that MMNS asked their participants affected the responses of one of their target groups (Westerners). Sytsma and Livengood argued that their results cast doubt on MMNS’s assumption that their survey instrument reliably indicates the semantic intuitions of Westerners. On this basis, Sytsma and Livengood charged that MMNS’s study does not allow us to conclude that semantic intuitions differ across cultures. In other words, they claimed that MMNS’s “results do not provide the basis for a compelling case against the uniformity conjecture” (316). In doing so, however, Sytsma and Livengood did not claim that the uniformity conjecture is true with respect to Kripke’s Gödel case. Their point was rather that MMNS’s study did not show that the uniformity conjecture was false. As such, the status of the uniformity conjecture with respect to intuitions about reference remains an open empirical question.

In this paper, we present new evidence that the uniformity conjecture does not hold for Kripke’s Gödel case. We report the results of a study comparing the responses of one group of Westerners and one group of East Asians for the suite of probes used by Sytsma and Livengood. What we found is that while the Western participants were quite sensitive to the perspectival ambiguity, the East Asian participants were not. Further, while the Western participants tended to give answers consistent with Kripke’s intuition about the Gödel case once the perspectival ambiguity was clarified, a majority of the East Asian participants gave the opposite answer for each probe. Assuming that the clarified Gödel probe tracks semantic intuitions, the semantic intuitions of the Western participants were statistically significantly different from those of the East Asian participants, suggesting that the uniformity conjecture is in fact false for Kripke’s Gödel case.

1 The Gödel Case and the Uniformity Conjecture

Machery, Mallon, Nichols, and Stich (2004) challenge a standard methodology used in recent philosophy of language. They note that “philosophers agree that theories of reference for names have to be consistent with our intuitions regarding who or what the names refer to” (B2). This is drawn out by considering a prominent example from the literature— Kripke’s (1972) Gödel case.

When Kripke developed this case, he was replying to theories of reference that followed Russell (1905, 1919) in taking an associated definite description to express the meaning of a proper name. The Gödel case purports to show that descriptivist accounts of proper names cannot handle cases where ignorance leads a speaker to associate some description with the wrong individual. In order to illustrate this problem, Kripke sets up his case so that the description generally associated with the name “Gödel” best corresponds with someone other than the man given that name at birth (a man named “Schmidt”). And, as MMNS note, this story has elicited widely shared intuitions amongst philosophers that run counter to traditional descriptivist accounts. Further, many philosophers have held that theories of reference need to accommodate these intuitions.

MMNS challenge this standard methodology by raising doubts about a plausible justification for the practice. With regard to the Gödel case, they write:

Philosophers typically share the Kripkean intuitions and expect theories of reference to accommodate them…. [We] suspect that most philosophers exploring the nature of reference assume that the Kripkean intuitions are universal. Suppose that semantic intuitions exhibit systematic differences between groups or individuals. This would raise questions about whose intuitions are going to count, putting in jeopardy philosophers’ methodology. (B4)

Put another way, MMNS’s challenge focuses on the uniformity conjecture—the assumption that intuitions about key cases will be (near) universally shared in the general population. MMNS argue that the truth of the uniformity conjecture with regard to intuitions about Kripke’s Gödel case would support the assumption that theories of reference should be consistent with these intuitions. And, conversely, they note that the falsity of the uniformity conjecture would raise significant doubts about this assumption.

Whether or not the uniformity conjecture holds for the Gödel case, however, is rather clearly an empirical question. If it were to turn out that there is variation in intuitions about the case within or across theoretically interesting populations, then advocates of the standard methodology would be under pressure to provide a compelling alternative justification for the practice. Failure of the uniformity conjecture would seriously undermine common practice in philosophy of language. Hence, the burning question is whether the uniformity conjecture actually holds.

Based on work in cultural psychology indicating that there are systematic cognitive differences between Westerners and East Asians (see Nisbett et al. 2001), MMNS predicted that East Asians would be more likely to have descriptivist intuitions than Westerners, and the results of their empirical studies were in line with their prediction. MMNS presented two groups of English-speaking undergraduates—one group from Rutgers University (Westerners) and one group from the University of Hong Kong (East Asians)—with two probes modeled on Kripke’s Gödel example. The probes were presented in English and differed in whether Western or Chinese names were used. Each participant was given both probes. The Western-name probe reads as follows:

Suppose that John has learned in college that Gödel is the man who proved an important mathematical theorem, called the incompleteness of arithmetic. John is quite good at mathematics and he can give an accurate statement of the incompleteness theorem, which he attributes to Gödel as the discoverer. But this is the only thing that he has heard about Gödel. Now suppose that Gödel was not the author of this theorem. A man called “Schmidt,” whose body was found in Vienna under mysterious circumstances many years ago, actually did the work in question. His friend Gödel somehow got hold of the manuscript and claimed credit for the work, which was thereafter attributed to Gödel. Thus, he has been known as the man who proved the incompleteness of arithmetic. Most people who have heard the name “Gödel” are like John; the claim that Gödel discovered the incompleteness theorem is the only thing they have ever heard about Gödel. When John uses the name “Gödel,” is he talking about:

  1. (A)

    the person who really discovered the incompleteness of arithmetic? or

  2. (B)

    the person who got hold of the manuscript and claimed credit for the work?

MMNS scored answers as either 0 or 1, where 0 corresponds with an (A) answer and 1 corresponds with a (B) answer. They then added the scores for the two probes together, resulting in a scale running from 0 to 2 for each participant. Using this scale, MMNS found a mean of 1.13 for American participants compared to a mean of 0.63 for Chinese participants. Converting the mean scores to percentages of (B) answers, they found that 56.5 % of American participants answered (B) compared to 31.5 % of Chinese participants.

The difference between the responses for the two groups is statistically significant.Footnote 1 From this, MMNS concluded that there is cross-cultural variation in intuitions about the Gödel case. Further, they pointed out that there is significant variation within each sample—neither the Western nor the East Asian participants were uniform in their responses. These results suggest that the uniformity conjecture does not hold, which in turn puts pressure on practitioners of the standard methodology to offer an alternative justification from the uniformity conjecture for the practice of treating their own intuitions about such cases as evidence in constructing theories of reference.

Several objections have been raised against MMNS’s work and subsequently been answered in the literature (for criticisms, see Ludwig 2007; Deutsch 2009; Martí 2009; Cullen 2010; Lam 2010; Ichikawa et al. 2012; Devitt 2011; Devitt 2012a, and Devitt 2012b; for replies, see Machery et al. 2009, 2010, 2012a, b, 2013, Machery, E., J. Sytsma, and M. Deutsch. Speaker’s reference and cross-cultural semantics, In: A. Bianchi (Ed.), On reference. (forthcoming)). One objection that has not yet been addressed in the current literature was put forward by Sytsma and Livengood (2011). Focusing on the Western name probe seen above, Sytsma and Livengood argued that there is a perspectival ambiguity in the question that MMNS asked participants. This perspectival ambiguity concerns the epistemic perspective that a participant adopts in reading the definite descriptions given as the answer choices. Sytsma and Livengood noted that there is an asymmetry between what John knows in the Gödel story and what the narrator knows in telling the story. From the narrator’s perspective, Schmidt discovered the incompleteness of arithmetic, while Gödel is merely the person who got hold of the manuscript and claimed credit for the work, but as far as John knows, Gödel discovered the theorem. While MMNS assumed that participants would adopt the narrator’s perspective in responding to the Gödel probe, participants might have adopted John’s perspective instead. But if participants adopted John’s perspective, then their responses might not correspond with their semantic intuitions: They might give the “descriptivist” response despite having causal-historical intuitions, since John does not know anything about the theft.

To test whether perspectival ambiguity actually affected responses to MMNS’s Gödel probe, Sytsma and Livengood ran a series of studies testing different versions of the probe question. In addition to MMNS’s original question, they designed variations intended to implicate that participants should answer either from John’s perspective (one variation) or from the narrator’s perspective (two variations). The result was a suite of four probes, with each using the Western-name vignette developed by MMNS and only the probe question being varied:

  • Original: When John uses the name “Gödel,” is he talking about: (A) the person who really discovered the incompleteness of arithmetic? Or, (B) the person who got hold of the manuscript and claimed credit for the work?

  • John’s Perspective: When John uses the name “Gödel,” does John think he is talking about: (A) the person who the story says really discovered the incompleteness of arithmetic? Or, (B) the person who the story says got hold of the manuscript and claimed credit for the work?

  • Narrator’s Perspective: When John uses the name “Gödel,” is he actually talking about: (A) the person who the story says really discovered the incompleteness of arithmetic? Or, (B) the person who the story says got hold of the manuscript and claimed credit for the work?

  • Clarified Narrator’s Perspective: Having read the above story and accepting that it is true, when John uses the name “Gödel,” would you take him to actually be talking about: (A) the person who (unbeknownst to John) really discovered the incompleteness of arithmetic? Or, (B) the person who is widely believed to have discovered the incompleteness of arithmetic, but actually got hold of the manuscript and claimed credit for the work?

These four probes were given to Western participants (native English-speakers), using both between-subjects (Studies 1 and 2) and within-subjects designs (Study 3). The overall results were similar in both cases (see Fig. 1). Compared to the original probe, participants were significantly less likely to select the “causal-historical” answer (B) when given the John’s perspective probe, and significantly more likely to select the “causal-historical” answer when given either of the narrator’s perspective probes.

Fig. 1
figure 1

Results of the first three studies from Sytsma and Livengood (2011)

Full size image

On the basis of these studies, Sytsma and Livengood claimed that MMNS’s original Gödel probe is ambiguous for Western participants. They argued that their studies raise significant doubts about whether MMNS’s results actually reflect participants’ semantic intuitions about the Gödel case, as opposed to the epistemic perspective adopted in reading the probe question. Sytsma and Livengood went on to argue that although they only tested Westerners, their results nonetheless challenge MMNS’s claim to have shown significant cross-cultural variation between Westerners and East Asians, as well as significant variation within East Asians. In brief, having evidence that the original Gödel probe does not reliably track semantic intuitions in Westerners, they argued that it should not be assumed to do so for East Asians.

It is important to note that Sytsma and Livengood did not provide positive evidence for the uniformity conjecture, nor did they argue that MMNS’s claim that the uniformity conjecture does not hold for the Gödel case was false. Rather, they questioned the reliability of the evidence that MMNS provided for that claim. In other words, given that the perspectival ambiguity noted had a significant effect on the responses of Western participants, they argued that whether or not the uniformity conjecture holds remains an open question. As such, further experimental work is needed to answer that question, and such work should use probes that control for the perspectival ambiguity identified. In the following section we present a new cross-cultural study that does this. The results suggest that MMNS were right after all: The uniformity conjecture is false.

2 New Evidence Against the Uniformity Conjecture

The cross-cultural study we conducted had three primary goals. First, we wanted to test whether or not the perspectival ambiguity identified by Sytsma and Livengood significantly affects the responses of East Asians to MMNS’s Gödel probe, as it appears to do for Westerners. Second, we wanted to test whether the cross-cultural variation found by MMNS remained after the probe question was rewritten to control for the perspectival ambiguity. Third, we wanted to test Lam’s (2010) charge that the cross-cultural variation found by MMNS reflects that Western participants received the probe in their native language, while East Asian participants received it in a second language. Since both groups received the probes in English, and since it is plausible that native English-speakers have greater competency with the language than those for whom English is a second language, it is possible that MMNS’s results primarily reflect differences in linguistic competency rather than semantic intuitions.

The study we developed consists of three stages. In the first stage we presented Sytsma and Livengood’s suite of Gödel probes to American participants in English to get a baseline response rate for each of the four probes. We then translated these probes into Japanese and, in the second stage of the study, presented the translated probes to participants at five different Japanese universities. Finally, in the third stage of the study, we had the Japanese-language versions reverse-translated back into English, and we presented the reverse-translated probes to a new set of American participants to check for potential problems with the Japanese translation. The full set of probes is given in the appendix.

For each stage we used a between-subjects design, with each participant receiving only one of the four types of probes. Following Sytsma and Livengood, (A) and (B) answer choices were counterbalanced for order, although we will report the results using the ordering given in the previous section: “(A)” for the descriptivist answer choice, “(B)” for the causal-historical answer choice. In addition, participants were asked for demographic information and about any training they had in philosophy. The first and third stages of the study were conducted online, while the second stage was conducted in person in university classrooms.Footnote 2 Participants were excluded if they were under 18 years of age, did not complete the survey, had previously participated, or had more than minimal training in philosophy.Footnote 3 In addition, participants in Stages 1 and 3 were excluded if they were not native speakers of English born in and currently residing in the United States. Participants in Stage 2 were excluded if they were not native speakers of Japanese born in and currently residing in Japan. Responses were collected from 596 participants in Stage 1, from 221 participants in Stage 2, and from 583 participants in Stage 3.Footnote 4 The results are shown in Table 1 below, which gives the proportion of (B) responses for each probe by stage.

Table 1 Proportion of (B) answers by stage
Full size table

The results for American participants in Stage 1 replicate the pattern of variation across probes found by Sytsma and Livengood, although it should be noted that the percentage of (B) answers for the original probe is more in keeping with that originally reported by MMNS. As predicted, the percentage of (B) answers is significantly lower for the John’s perspective probe than for the original probe, and is significantly higher for the clarified narrator’s perspective probe than for the original probe.Footnote 5 Hence, our Stage 1 results provide further support for the claim that the perspectival ambiguity that Sytsma and Livengood describe significantly affects the answers given by Westerners.

In contrast, the variation across the four probes for Japanese participants in Stage 2 was minimal. We found that in each case the percentage of (B) answers was within 10 percentage points of the 31.5 % that MMNS reported for East Asian participants on the original probe, with only a minority of participants giving the (B) answer in each case. In line with the findings of MMNS, this suggests that contra Lam the language of presentation (native language versus second language) does not explain the cross-cultural variation found by MMNS. Further, emphasizing either John’s perspective or the narrator’s perspective had minimal impact on the responses. For example, while the percentage of (B) answers for American participants was 37.2 percentage points higher for the clarified narrator’s perspective probe than it was for the John’s perspective probe, for Japanese participants the percentage of (B) answers was only 10.4 points higher for the clarified narrator’s perspective probe than it was for the John’s perspective probe. This indicates that the perspectival ambiguity identified by Sytsma and Livengood does not have a major impact on the responses of at least one group of East Asians.

Most importantly, the results of the first two stages of our study suggest that significant cross-cultural variation remains even after controlling for the perspectival ambiguity and the language of presentation. Whereas the probe a participant saw made a significant difference to the responses of American participants, the probe a participant saw made no statistically significant difference to the responses of Japanese participants.Footnote 6 Moreover, consistent with MMNS’s findings, Japanese participants were less likely than American participants to choose the (B) answer in response to the Original probe, the Narrator’s Perspective probe, and the Clarified Narrator’s Perspective probe.Footnote 7 In fact, we find that while a majority of the American participants give that response in each of those cases, only a minority of the Japanese participants do. Based on these results, we tentatively conclude that MMNS were correct: The uniformity conjecture does not hold for the Gödel case.Footnote 8

One potential issue with the comparison between the responses of American and Japanese participants in Stages 1 and 2 of our study, however, is that they might reflect the translation used in Stage 2 rather than differences in semantic intuitions. The third stage of our study was designed to control for problems with the translation. If the difference between American and Japanese responses observed in Stages 1 and 2 is due to something in the translation rather than to semantic intuitions, then we would expect the responses of Americans to the reverse-translated probes used in Stage 3 to be different from those received in Stage 1. Specifically, we would expect the responses of Americans to the reverse-translated probes to be more uniform across the four conditions. But that is not at all what we found. The patterns of response in Stages 1 and 3 were statistically very similar to each other, and they were statistically very different from the pattern in Stage 2.Footnote 9 Moreover, with the exception of the Clarified Narrator’s Perspective probe, which we discuss in greater detail below, the variation in American responses to the reverse-translated probes was more pronounced than the variation in American responses to the baseline (un-translated) probes: the exact opposite of the prediction made on the assumption that the differences between Stages 1 and 2 were due to translation.

Earlier, we compared the results of Stages 1 and 2, and we found that Japanese participants were less likely than American participants to choose the (B) answer in response to the Original probe, the Narrator’s Perspective probe, and the Clarified Narrator’s Perspective probe. Comparing the results of Stages 2 and 3, we again found that Japanese participants were less likely than American participants to choose the (B) answer in response to the Original probe, the Narrator’s Perspective probe, and the Clarified Narrator’s Perspective probe.Footnote 10 However, we had expected that in Stage 3, as in Stage 1, the proportion of (B) answers for the Clarified Narrator’s Perspective probe would be greater than the proportion of (B) answers for the Narrator’s Perspective probe—an expectation that was not fulfilled.Footnote 11 The unexpectedly low proportion of (B) answers for the reverse-translated Clarified Narrator’s Perspective probe does not raise any issues for the other probes (for the reasons already given above), but it does call for some explanation. In the remainder of this section, we will give reasons for thinking that the translation problem arose in the reverse-translation step.

Recall that the baseline English version of the question for the Clarified Narrator’s Perspective probe reads as follows:

Having read the above story and accepting that it is true, when John uses the name “Gödel,” would you take him to actually be talking about: (A) the person who (unbeknownst to John) really discovered the incompleteness of arithmetic? Or, (B) the person who is widely believed to have discovered the incompleteness of arithmetic, but actually got hold of the manuscript and claimed credit for the work?

Translated into Japanese, the question reads:

上のストーリーが実話だったとしよう。ジョンが 「ゲーデル」 という名前を使うとき、あなたは彼が語っているのは実際のところ誰についてだと思うか?

  1. (A)

    算術の不完全性を本当に発見した人物 (ジョンはこのことを知らない)

  2. (B)

    算術の不完全性を発見したと広く信じられているが、実際には、手稿を手

に入れ、その業績を自分のものだと主張した人物

When reverse-translated into English, the question reads like this:

Suppose that the above story is true. When he uses the name “Gödel,” whom do you think he is really talking about?

  1. (A)

    The person who really discovered the incompleteness of arithmetic (John does not know this)

  2. (B)

    The person who is widely believed to have discovered the incompleteness of arithmetic but in reality obtained the manuscript and claimed that it is his work

Looking at the reverse-translated Clarified Narrator’s Perspective probe, we guessed that the problem had something to do with how the parenthetical remark in answer (A) was being translated. Specifically, we found that the parenthetical in the (A) answer choice reads awkwardly.

To test whether some infelicity in the reverse-translated (A) answer choice explains the drop in (B) answers that we found for the reverse-translated Clarified Narrator's Perspective probe, we conducted a further study in which we set the (A) answer choice back to its original form: “The person who (unbeknownst to John) really discovered the incompleteness of arithmetic.” The probe was otherwise the same as the version used in Stage 3. We gave the revised probe to 143 American participants online, adopting the same restrictions used in Stages 1 and 3.Footnote 12 We found that with the corrected (A) answer, the percentage of (B) answers rose from 54.1 % to 69.9 %, which is in line with the 68.5 % found for the pre-translation probe and a significantly greater percentage of (B) answers than we observed among Japanese participants.Footnote 13

Identifying the awkward parenthetical as the problem in the reverse-translated version of the Clarified Narrator’s Perspective probe, however, does not tell us whether it was introduced in the original translation, which should reduce the confidence one has in our results, or in the reverse-translation, which should not make any difference to the confidence one has in our results.Footnote 14 Fortunately, there is good reason to think that the problem arises in the reverse-translation.

One issue with the parenthetical in answer choice (A) in the reverse-translation is that it is unclear what it is that John does not know. Specifically, the word “this” at the end of the parenthetical feels like it should be followed by another word that makes clear what it refers to, and several options are available—including “this fact,” “this person,” and “this theorem.” Not all of the plausible referents serve to emphasize the narrator’s perspective, however. In contrast, the Japanese translation of the answer choice is not ambiguous in this way. Thus, “このこと” (kono-koto) in the parenthetical literally means “this fact.” Hence, while there are several possible referents for the “this” in the reverse-translated answer choice, there is only one for the Japanese answer choice. As such, we are convinced that the comparison between the baseline English version and the Japanese translation is sound.Footnote 15 Replacing the result for the Clarified Narrator’s Perspective probe in Stage 3 with the result for the edited version, the pattern of results is especially clear, as seen in the plot in Fig. 2.

Fig. 2
figure 2

Plot of the results of our cross-cultural using the percentage for the edited version of the clarified narrator’s probe in stage 3, with 95 % confidence intervals

Full size image

3 Conclusion

Our results support the claim that the uniformity conjecture does not hold for Kripke’s Gödel case, and hence, our results confirm MMNS’s conclusions, at least in rough outline. Japanese participants, a group of East Asians not previously studied by MMNS or their critics, were significantly less likely to give responses consistent with the causal-historical intuition about the Gödel case than were American participants—even after controlling for the perspectival ambiguity noted by Sytsma and Livengood (2011).

Philosophers of language interested in defending the standard methodology with respect to theories of reference may have taken comfort from Sytsma and Livengood (2011), reasoning as follows: Perhaps East Asians are simply more likely than Westerners to adopt John’s perspective in responding to MMNS’s original probe. If so, then the cross-cultural variation found by MMNS might simply be taken to reflect cultural differences with regard to perspective-taking, not cultural differences with regard to semantic intuitions. And this is consistent with the findings of Nisbett and colleagues that MMNS appeal to.Footnote 16 Thus, it might be thought that if East Asians and Westerners could both be made to read the Gödel probe from the narrator’s perspective, then they would respond to it in the same way. Our cross-cultural study substantially undermines this reply.

As such, our results support MMNS’s challenge to the standard methodology. But we want to be careful on this point in two ways. First, MMNS’s challenge admits various formulations that trade off boldness and security. More securely, our results challenge the use of one’s own intuitions about cases like Kripke’s Gödel example (i.e., cases involving ignorance and error) as evidence for a given account of reference. More boldly, the results could fuel (a) a challenge to the use of one’s own intuitions as evidence in theorizing about reference more generally, (b) a challenge to the use of one’s own intuitions as evidence in theorizing about semantics, or even (c) a challenge to the use of one’s own intuitions as evidence in philosophy as a whole. While we are generally skeptical of the use of one’s own intuitions as evidence in philosophical theorizing—and while our results add to a growing body of data that casts doubt on this practice (see, for example, the discussion in Alexander, Mallon, and Weinberg 2010)—we will restrict ourselves to the more secure claim.

Second, it should be repeated that MMNS’s empirical challenge targets one justification that has been offered for the practice of using one’s own intuitions as evidence in theorizing about reference. Specifically, this practice has been justified by claiming that the uniformity conjecture holds for the intuitions at issue. On the basis of their empirical results, MMNS contend that the uniformity conjecture does not hold for the Gödel case, and our results support this contention. Nonetheless, the standard methodology might be justified in other ways. For example, Devitt (2011) argues that the intuitions of philosophers about the cases at issue are more reliable than those of non-philosophers because philosophers are more expert with regard to these issues. While we are skeptical of so-called “expertise defense,” our results do not bear on this alternative justification.Footnote 17 As such, we again restrict ourselves to the more cautious conclusion in this paper: Responsible philosophers working on reference should not continue to assume without empirical support that the uniformity conjecture holds for cases like Kripke’s Gödel example.