What Is the Basic Unit of Scientific Progress? A Quantitative, Corpus-Based Study

Abstract

This paper presents the results of an empirical study following up on Mizrahi (2021). Using the same methods of text mining and corpus analysis used by Mizrahi (2021), we test empirically a philosophical account of scientific progress that Mizrahi (2021) left out of his empirical study, namely, the so-called functional-internalist account of scientific progress according to which the aim or goal or scientific research is to solve problems. In general, our results do not lend much empirical evidence in support of the problem-solving model of scientific progress over the other philosophical accounts of scientific progress (namely, the epistemic, noetic, and semantic accounts of scientific progress) tested in Mizrahi (2021) and in this follow-up study. Of all the subjects in the JSTOR database we have tested in this study, however, Mathematics is an interesting exception as far as the problem-solving model of scientific progress is concerned. For, in Mathematics alone, we have found that there is significantly more talk of the aims and/or goals of research in terms of solutions than in terms of truth, knowledge, or understanding.

1 Introduction

Mizrahi (2021) sets out to test empirically three philosophical accounts of scientific progress that have been the focus of the debate over the nature of scientific progress in contemporary philosophy of science. The first is the so-called epistemic account of scientific progress, which defines scientific progress in terms of knowledge. Bird (2008, 279) characterizes the epistemic account as follows: “An episode constitutes scientific progress precisely when it shows the accumulation of scientific knowledge” (italics added).¹ The second is the semantic account of scientific progress, which defines scientific progress in terms of truth (or truthlikeness). Bird (2008, 279) characterizes the semantic account as follows: “An episode constitutes scientific progress precisely when it either (a) shows the accumulation of true scientific belief, or (b) shows increasing approximation to true scientific belief” (italics added).² The third is the noetic account of scientific progress, which defines scientific progress in terms of understanding. Dellsén (2018b, 7) characterizes the noetic account as follows: “an episode in science is progressive just in case scientists grasp how to correctly explain or reliably predict more aspects of the world than they did before” (italics added).³

Mizrahi (2021) presents the results of a quantitative, corpus-based empirical study, which suggest that practicing scientists use the terms for the basic units of scientific progress ‘knowledge’ and ‘understanding’ significantly more often than the term for the basic unit of scientific progress ‘truth’ when they talk about the aims and/or goals of scientific research in scientific publications. Accordingly, Mizrahi (2021) argues, these results can be construed as providing some empirical evidence in favor of the epistemic and the noetic accounts over the semantic account of scientific progress. However, Mizrahi (2021) argues, these results “do not favor the epistemic account over the noetic account, or vice versa, for they reveal no significant differences between the frequency with which practicing scientists use the terms ‘knowledge’ and ‘understanding’ when they talk about the aims or goals of scientific research in their published works.”

In this paper, we seek to build on Mizrahi’s (2021) empirical study of philosophical accounts of scientific progress. In particular, using the same methods of text mining and corpus analysis used by Mizrahi (2021), we test a philosophical account of scientific progress that Mizrahi (2021) left out of his empirical study, namely, the so-called functional-internalist account of scientific progress. Bird (2008, 279) characterizes the functional-internalist account as follows: “An episode shows scientific progress precisely when it achieves a specific goal of science, where that goal is such that its achievement can be determined by scientists at that time (e.g. solving scientific puzzles)” (italics added). In terms of “solving scientific puzzles” in particular, it was Laudan (1977) who built on Kuhn’s (1962/1996, 36–38) discussion of “puzzle-solving” in science to develop “a problem-solving model of progress” (Laudan 1977, 66). According to Laudan (1977, 66):

The core assumptions of such a model are simple ones: (1) the solved problem—empirical or conceptual—is the basic unit of scientific progress; and (2) the aim of science is to maximize the scope of solved empirical problems, while minimizing the scope of anomalous and conceptual problems” (italics in original).

Accordingly, on the so-called functional-internalist account of scientific progress, or more specifically, the problem-solving model of scientific progress, the basic unit of scientific progress is a solution to a scientific problem. On the epistemic account of scientific progress, the basic unit of scientific progress is knowledge. On the semantic account of scientific progress, the basic unit of scientific progress is truth. Finally, on the noetic account of scientific progress, the basic unit of scientific progress is understanding.

It is important to point out that there are various versions of the so-called functional-internalist account of scientific progress in the philosophy of science literature.⁴ For example, Shan (2020, 101–102) characterizes the functional-internalist account, as applied to genetics, as follows: “the development of genetics is progressive if it shows the success of the fulfilment of a certain function (for example, problem-solving), where the fulfilment of the function can be judged by scientists at that time” (emphasis added). However, Shan (2020, 102) then goes on to offer “a new functional approach to scientific progress,” according to which “scientific progress is defined in terms of usefulness of problem-defining and problem-solving” (cf. Shan 2019). In Shan’s new functional approach to scientific progress, too, the notion of the solved problem, which is the basic unit of scientific progress according to Laudan’s problem-solving model of progress, is central.⁵ For the purposes of the present study, then, we will focus on this key notion of the solved problem or a solution to a problem. Obviously, to say that a problem is solved is to say that it has a solution. This is also one of the four accounts of scientific progress identified in Dellsén’s (2018b) survey of philosophical accounts of scientific progress. According to Dellsén (2018b, 2), “Each account places its own distinctive type of cognitive achievement at the heart of scientific progress—truthlikeness, problem‐solving, knowledge, or understanding.”

In philosophy of science, then, the scientific progress debate is a debate about the basic unit of scientific progress. As Douglas (2014, 55) puts it, the main question of the debate is this: “How should we understand scientific progress?” According to the epistemic account, we should understand scientific progress in terms of knowledge. According to the semantic account, we should understand scientific progress in terms of truth. According to the noetic account, we should understand scientific progress in terms of understanding. Finally, according to the problem-solving model, we should understand scientific progress in terms of solved problems or solutions.

In what follows, we will describe how we tested these philosophical accounts of scientific progress against what practicing scientists say when they talk about the aims and goals of scientific research in their academic publications. We used the same methods used by Mizrahi (2021) but expanded on his empirical study by testing the problem-solving model of scientific progress empirically as well. After we discuss Mizrahi’s (2021) methodology and how we built on it to test the problem-solving model as well, we discuss the results of our quantitative, corpus-based study. In general, our results do not lend much empirical evidence in support of the problem-solving model of scientific progress over the other philosophical accounts of scientific progress (namely, the epistemic, noetic, and semantic accounts of scientific progress) tested in Mizrahi (2021) and in this follow-up study. Of all the subjects in the JSTOR database we have tested in this study, however, Mathematics is an interesting exception as far as the problem-solving model of scientific progress is concerned. For, in Mathematics alone, we have found that there is significantly more talk of the aims and/or goals of research in terms of solutions than in terms of truth, knowledge, or understanding.

2 Methods

As we mentioned in Sect. 1, we are using the same methodology Mizrahi (2021) uses to test three philosophical accounts of scientific progress empirically, namely, the epistemic, semantic, and noetic accounts of scientific progress, but we add a fourth philosophical account to our empirical study, namely, the problem-solving model of scientific progress. For these accounts of scientific progress have empirical consequences that we would be able to observe in scientific practice. As Mizrahi (2021) puts it:

• On the semantic account, which defines scientific progress in terms of truth, we would expect to find that practicing scientists talk about scientific progress in terms of truth more than knowledge or understanding in scientific publications.

• On the epistemic account, which defines scientific progress in terms of knowledge, we would expect to find that practicing scientists talk about scientific progress in terms of knowledge more than truth or understanding in scientific publications.

• On the noetic account, which defines scientific progress in terms of understanding, we would expect to find that practicing scientists talk about scientific progress in terms of understanding more than knowledge or truth in scientific publications (italics in original).

To this list, we can add the following empirical consequence of the problem-solving model of scientific progress:

• On the problem-solving account, which defines scientific progress in terms of the solved problem, we would expect to find that practicing scientists talk about scientific progress in terms of solutions more than knowledge, truth, or understanding in scientific publications.

As we have also seen in Sect. 1, philosophical accounts of scientific progress are about the “aim of science” (Laudan 1977, 66) or the “goal of science” (Bird 2008, 279). This is because “Progress is a goal-relative concept,” as Niiniluoto (2019) points out, and so the debate over the nature of scientific progress is “concerned with axiological questions about the aims and goals of science” (italics added). Likewise, according to Dellsén (2018b, 7–8), “there seems to be a close conceptual connection between scientific progress and the aim of science” insofar as “it seems plausible to say that X is the aim of science just in case science makes progress when we increase, accumulate, or get closer to achieving X” (emphasis added).⁶ Accordingly, if we look for the terms for the basic units of scientific progress, namely, knowledge, truth, understanding, and solution, in contexts where practicing scientists talk about the aims and/or goals of scientific research, we can get a pretty good idea of how practicing scientists define scientific progress in their scientific publications. Adding to Mizrahi’s (2021) search pairs of terms for the basic units of scientific progress and progress terms, then, we get the search pairs listed in Table 1 by combining the terms for the basic units of scientific progress, namely, ‘knowledge’, ‘truth’, ‘understanding’, and ‘solution’, with the progress terms ‘aim’ and ‘goal’.

Table 1 Search pairs for the epistemic, semantic, noetic, and problem-solving accounts of scientific progress

It is worth noting that, when philosophers of science talk about the aims and/or goals of science, they are not talking about the desires and/or motives of individual scientists. As van Fraassen (1980, 8) puts it, “The aim of science is of course not to be identified with the individual scientist’s motives. […]. What the aim determines is what counts as success in the enterprise as such; and this aim may be pursued for any number of reasons” (emphasis added).⁷ Accordingly, when philosophers of science talk about the aims and/or goals of science, they are talking about criteria for success in science. That is, philosophers of science are interested in the sort of cognitive achievements in science that constitute progress in science (Dellsén, 2018b, 2). On the epistemic account, accumulating knowledge constitutes a cognitive achievement (or success) in science. On the semantic account, getting closer to truth constitutes a cognitive achievement in science. On the noetic account, increasing understanding constitutes a cognitive achievement in science. And on the problem-solving model, solving problems constitutes a cognitive achievement in science. Now, presumably, when they talk about the aims and/or goals of their research in their scientific publications, practicing scientists are unlikely to talk about their desires and/or motives for doing research (such as, a desire to win the Nobel Prize or a motive for becoming famous). Instead, they are likely to talk about the aims and/or goals of the research (such as, knowing about x, finding out the truth about x, understanding x, solving the x problem, etc.).

As Mizrahi (2021) explains, this text-mining methodology is designed to minimize the number of false positives and false negatives. As far as this follow-up study is concerned, a false positive is a result indicating a match for one of the terms for the basic units of scientific progress, namely, ‘knowledge’, ‘truth’, ‘understanding’, and ‘solution’, which does not occur in the context of talk about the aims and/or goals of scientific research. Mizrahi’s (2021) text-mining methodology is designed to minimize the number of such false positives by pairing the terms for the basic units of scientific progress with the progress terms ‘aim’ and/or ‘goal’ as follows: (“aim knowledge” ~ 10), (“goal knowledge” ~ 10), etc. This means that a term for a basic unit of progress, such as ‘knowledge’, cannot be separated from a progress term, such as ‘aim’, by more than ten words. A false negative is a result indicating no match for one of the terms for the basic units of scientific progress, namely, ‘knowledge’, ‘truth’, ‘understanding’, and ‘solution’, even though a synonym for a progress term we have not searched for occurs in the corpus. For example, if we search for (“aim knowledge” ~ 10) and find no positive matches because our corpus does not contain the term ‘aim’ but rather the synonym ‘goal’, then our negative result would count as a false negative. Mizrahi’s (2021) text-mining methodology is designed to minimize the number of such false negatives by searching for both ‘aim’ and ‘goal’ as synonymous progress terms. For, as we have seen, the debate over the nature of scientific progress is “concerned with axiological questions about the aims and goals of science” (Niiniluoto 2019).

Even though Mizrahi’s (2021) text-mining methodology is designed to minimize the number of false positives and false negatives, a few results might still count as false positives or false negatives. In other words, like any other empirical methodology, the text-mining methodology employed in Mizrahi (2021) and in this follow-up study is not immune to having a few false positive or false negative results. This is why the results of this study, like the results of any other empirical study, must not be interpreted as providing conclusive evidence either for or against any philosophical account of scientific progress. Rather, the results of this follow-up study are meant to contribute to our understanding of the nature of scientific progress by testing accounts of scientific progress empirically against what practicing scientists say about scientific progress in their published research. For, as Rouse (2007, 84) puts it, “by attending more closely to what scientists say and do,” we can “avoid unwarranted philosophical impositions upon science” (italics added). If what scientists say and do in their scientific publications is part of scientific activity, and “[a]ny philosophical view of science is to be held accountable to actual scientific practice, scientific activity” (van Fraassen 1994, 184), then philosophical views of science are to be held accountable to what scientists say and do in their scientific publications. For the purposes of this empirical study, then, philosophical accounts of scientific progress are to be held accountable to what scientists say about the aims and/or goals of scientific research in their scientific publications.⁸

The data for this quantitative, corpus-based study were collected from JSTOR Data for Research (www.jstor.org/dfr/). Researchers can use JSTOR DfR to create datasets, including metadata, n-grams, and word counts, for most of the articles and book chapters contained in the JSTOR database. Accordingly, we can search the JSTOR database for occurrences of the terms ‘truth’, ‘knowledge’, ‘understanding’, and ‘solution’ in scientific publications in order to find out whether, and to what extent, practicing scientists use these terms when they talk about the aims and/or goals of scientific research in their published works. In order to have a diverse and large sample of scientific texts, we followed Mizrahi (2021) in gathering data from the following subjects in the JSTOR database: Anthropology, Archeology, Astronomy, Biological Sciences, Economics, Geography, Geology, Linguistics, Mathematics, Paleontology, Physics, Psychology, Sociology, Statistics, and Zoology. This gives us two representative fields from the life sciences (namely, Biological Sciences and Zoology), two representative fields from the physical sciences (namely, Astronomy and Physics), two representative fields from the social sciences (namely, Anthropology and Sociology), and two representative fields from the formal sciences (namely, Mathematics and Statistics).

Now, if the semantic account of scientific progress were true, we would expect to find practicing scientists across the aforementioned fields using the term ‘truth’ much more frequently than the terms ‘knowledge’, ‘understanding’, or ‘solution’ when they talk about the aims and/or goals of scientific research in scientific publications. If the epistemic account of scientific progress were true, we would expect to find practicing scientists across the aforementioned fields using the term ‘knowledge’ much more frequently than the terms ‘truth’, ‘understanding’, or ‘solution’ when they talk about the aims and/or goals of scientific research in scientific publications. If the noetic account of scientific progress were true, we would expect to find practicing scientists across the aforementioned fields using the term ‘understanding’ much more frequently than the terms ‘knowledge’, ‘truth’, or ‘solution’ when they talk about the aims and/or goals of scientific research in scientific publications. Finally, if the problem-solving model of scientific progress were true, we would expect to find practicing scientists across the aforementioned fields using the term ‘solution’ much more frequently than the terms ‘knowledge’, ‘understanding’, or ‘truth’ when they talk about the aims and/or goals of scientific research in scientific publications. To be clear, these are empirical, not logical, consequences of the semantic, epistemic, noetic, and problem-solving accounts of scientific progress that can be derived from these accounts only by assuming that we can gain insights about scientific progress from studying what scientists say and do. This is another reason why the results of this empirical study, like the results of any other empirical study, must not be interpreted as providing conclusive evidence either for or against any philosophical account of scientific progress. Proponents of any of the four accounts of scientific progress can always reject the methodological assumptions of this empirical study for metaphilosophical reasons, such as a metaphilosophical preference for testing philosophical accounts of science against intuitions elicited by hypothetical cases (see, e.g., Bird 2007; Rowbottom 2008), or against episodes from the history of science (see, e.g., Niiniluoto 2014; 2017), rather than against “actual scientific practice, scientific activity” (van Fraassen 1994, 184).⁹

Before we report the results of our quantitative, corpus-based study, we should address the following question raised by an anonymous reviewer:

Imagine one author says “My aim is to solve the problem of induction” and another author says “My aim is to find a true answer to the question ‘What licenses inductive inference?’” Aren’t they trying to do the same thing? It seems so. So does this mean the first case should ideally be counted as a truth aim statement too, and the second as a problem solving aim statement too? If not, why not?¹⁰

We think that proponents of the problem-solving model and proponents of the semantic account of scientific progress would respond to the reviewer’s question by saying that the two hypothetical authors are not trying to do the same thing. That is, proponents of the problem-solving model and proponents of the semantic account of scientific progress would say that the first statement, namely, “My aim is to solve the problem of induction,” should not be counted as an instance of both truth talk and problem-solving talk, and that the second statement, namely, “My aim is to find a true answer to the question ‘What licenses inductive inference?’,” should not be counted as an instance of both problem-solving talk and truth talk. Rather, proponents of the problem-solving model and proponents of the semantic account of scientific progress would say that the first statement, namely, “My aim is to solve the problem of induction,” counts as an instance of problem-solving talk, not truth talk, whereas the second statement, namely, “My aim is to find a true answer to the question ‘What licenses inductive inference?’,” counts as an instance of truth talk, not problem-solving talk. In other words, as far as the scientific progress debate in philosophy of science is concerned, the terms ‘truth’ and ‘solution’ are not intersubstitutable salva veritate, as the reviewer seems to suggest. Here is why.

Proponents of the problem-solving model of scientific progress tend to think that theoretical truth is unattainable in science. As Kuhn writes in the Postscript to the second edition of The Structure of Scientific Revolutions (1962/1996, 206): “There is, I think, no theory-independent way to reconstruct phrases like ‘really there’; the notion of a match between the ontology of a theory and its ‘real’ counterpart in nature now seems to me illusive in principle.”¹¹ Unlike theoretical truth, however, solutions to problems are attainable, according to Kuhn, and so solutions are the proper aim or goal of scientific research; otherwise, science would be aiming at something that is unattainable. As Mizrahi and Buckwalter (2014, 153) put it, “Kuhn is not a skeptic about scientific progress but he is a skeptic about theoretical truth. So he construes progress in terms of an aim that he deems attainable, namely, puzzle solutions, rather than an aim that he deems unattainable, namely, theoretical truth.” Like Kuhn, Laudan is also a skeptic about theoretical truth in science, but he is not a skeptic about scientific progress. Like Kuhn, then, Laudan construes scientific progress in terms of an aim or a goal that he thinks is attainable, which is solutions to problems, rather than an unattainable aim or goal like theoretical truth. In fact, Laudan explicitly says that a solution to a problem need not be true at all, and even the problem need not be a real problem, for science to make progress. As Laudan (1977, 16) puts it, “A problem need not accurately describe a real state of affairs to be a problem: all that is required is that it be thought to be an actual state of affairs by some agent” (original emphasis). Accordingly, on the problem-solving model, the terms ‘truth’ and ‘solution’ cannot be substituted salva veritate because scientists can make progress by proposing solutions that are not true to problems that are not even real. As Bird (2007, 68) puts it, “Laudan does not require that T is true. He does not even require that P really exist.”

Unlike proponents of the problem-solving model, proponents of the semantic account of scientific progress do think that truth is necessary for progress in science. Proponents of the epistemic account of scientific progress also think that truth is necessary for scientific progress. Unlike proponents of the semantic account, however, proponents of the epistemic account do not think that truth is sufficient for scientific progress. In addition to truth, justification is also necessary for scientific progress, according to the epistemic account (Rowbottom, 2008, 278). Consequently, proponents of the semantic account reject the claim that scientists can make progress by proposing solutions that are not true to problems that are not even real. Accordingly, on the semantic account of scientific progress, too, the terms ‘truth’ and ‘solution’ cannot be substituted salva veritate because scientific progress consists in “the accumulation of true scientific belief” or “increasing approximation to true scientific belief” (Bird 2008, 279). In other words, since “it is only truth that is relevant to progress” in science (Bird 2008, 279), and solutions to problems do not have to be true on the problem-solving model of scientific progress, the terms ‘truth’ and ‘solution’ are not intersubstitutable salva veritate. For this reason, the reviewer’s hypothetical authors are not trying to do the same thing when one says “My aim is to solve the problem of induction” and the other says “My aim is to find a true answer to the question ‘What licenses inductive inference?’.”

Now, if we want to test the aforementioned philosophical accounts of scientific progress empirically, which is what we set out to do in this study, we have to test them on their own terms, literally. On the semantic account of scientific progress, truth is both necessary and sufficient for scientific progress to occur. This means that only instances of the term ‘truth’ in the context of talk about the aims and/or goals of science count as truth talk about scientific progress in scientific publications. On the problem-solving model of scientific progress, only a solution, which need not be true, to a problem, which need not be real, is required for scientific progress to occur. This means that only instances of the term ‘solution’ in the context of talk about the aims and/or goals of science count as problem-solving talk about scientific progress in scientific publications.

3 Results

As Mizrahi (2021) did in his original study, we conducted searches for the terms for the basic units of scientific progress ‘truth’, ‘knowledge’, ‘understanding’, and ‘solution’ in the context of (i.e., within ten words of) the progress terms ‘aim’ and ‘goal’ (see Table 1) using the text-mining methodology described in Sect. 2. The search results for the terms for the basic units of scientific progress in the context of (i.e., within ten words of) the progress term ‘aim’ in each subject are listed in Table 2. All searches were conducted on October 14, 2020 and results were verified on October 30, 2020.

Table 2 Total number of publications, and those that contain the terms for the basic units of scientific progress within no more than 10 words of the progress term ‘aim’, by subject (

The JSTOR database does not contain the same number of publications in each subject, which is why we need to compare the proportions of positive matches for each search pair rather than raw counts. Figure 1 depicts the proportions of the terms for the basic units of scientific progress in the context of (i.e., without ten words of) the progress term ‘aim’ by subject.

Fig. 1

Proportions of publications that contain the terms for the basic units of scientific progress within no more than 10 words of the progress term ‘aim’ by subject

Like Mizrahi’s (2021) results, our results also show that the terms for the basic units of scientific progress do not occur all that frequently in the context of talk about the aims of scientific research in scientific publications (less than 0.04% overall). Still, the terms for basic units of scientific progress that appear to occur more frequently than others are ‘understanding’ and ‘knowledge’. These results are also similar to Mizrahi’s (2021) results. A notable exception, however, is Mathematics, where ‘solution’ appears to be the most frequent term for a basic unit of scientific progress. Mizrahi (2021) did not obtain this result, of course, because his study was not designed to test the problem-solving model of scientific progress.

To find out whether the aforementioned differences in proportions are statistically significant, we conducted Welch’s t-tests. First, there was a statistically significant difference between the proportions of (“aim knowledge” ~ 10) publications (M = 0.001, SD = 0.0007, N = 15) and (“aim truth” ~ 10) publications (M = 0.0002, SD = 0.0002, N = 15), t(17) = −3.98, p < 0.001, two-tailed. Second, there was a statistically significant difference between the proportions of (“aim understanding” ~ 10) publications (M = 0.001, SD = 0.0008, N = 15) and (“aim truth” ~ 10) publications (M = 0.0002, SD = 0.0002, N = 15), t(17) = − 4.48, p < 0.001, two-tailed. Third, there was no statistically significant difference between the proportions of (“aim knowledge” ~ 10) publications (M = 0.001, SD = 0.0007, N = 15) and (“aim understanding” ~ 10) publications (M = 0.001, SD = 0.0008, N = 15), t(28) = − 0.58, p = 0.56, two-tailed. These results are similar to the results obtained by Mizrahi (2021). They suggest that there is significantly more talk of the aim of scientific research in terms of knowledge than in terms of truth, and in terms of understanding than in terms of truth, in scientific publications. But the same cannot be said about talk of the aim of scientific research in terms of knowledge or in terms of understanding in scientific publications, since there is no statistically significant difference between the terms for basic units of scientific progress ‘understanding’ and ‘knowledge’ in the context of aim talk.

As mentioned in Sect. 1, beyond attempting to replicate Mizrahi’s (2021) results, we also set out to test the problem-solving model of scientific progress empirically, which Mizrahi (2021) did not test empirically in his study. In that respect, we found a statistically significant difference between the proportions of (“aim knowledge” ~ 10) publications (M = 0.001, SD = 0.0007, N = 15) and (“aim solution” ~ 10) publications (M = 0.0004, SD = 0.0005, N = 15), t(25) = 2.82, p = 0.009, two-tailed. We also found a statistically significant difference between the proportions of (“aim understanding” ~ 10) publications (M = 0.001, SD = 0.0008, N = 15) and (“aim solution” ~ 10) publications (M = 0.0004, SD = 0.0005, N = 15), t(24) = 3.35, p = 0.002, two-tailed. But we did not find a statistically significant difference between the proportions of (“aim truth” ~ 10) publications (M = 0.0002, SD = 0.0002, N = 15) and (“aim solution” ~ 10) publications (M = 0.0004, SD = 0.0005, N = 15), t(21) = – 1.00, p = 0.32, two-tailed. These results suggest that there is significantly more talk of the aim of scientific research in terms of knowledge than in terms of solutions, and in terms of understanding than in terms of solutions, in scientific publications, whereas talk of the aim of scientific research in terms of truth and in terms of solutions are equally infrequent in scientific publications, with no significant differences between proportions.

As mentioned above, Mathematics is a notable exception. It is alone among the subjects tested in this empirical study in having ‘solution’ as the most frequently used term for a basic unit of scientific progress in the context of aim talk. To see if ‘solution’ is used significantly more frequently than the other terms for the basic units of scientific progress in Mathematics, we conducted additional z-tests for proportions. First, the difference between the proportion of Mathematics publications that contain the term for the basic unit of progress ‘solution’ in the context of aim talk (0.001) and the proportion of Mathematics publications that contain the term for the basic unit of progress ‘truth’ in the context of aim talk (0.0002) is statistically significant (z = 21.85, p < 0.001, two-sided). Second, the difference between the proportion of Mathematics publications that contain the term for the basic unit of progress ‘solution’ in the context of aim talk (0.001) and the proportion of Mathematics publications that contain the term for the basic unit of progress ‘knowledge’ in the context of aim talk (0.0008) is statistically significant (z = 12.08, p < 0.001, two-sided). Finally, the difference between the proportion of Mathematics publications that contain the term for the basic unit of progress ‘solution’ in the context of aim talk (0.001) and the proportion of Mathematics publications that contain the term for the basic unit of progress ‘understanding’ in the context of aim talk (0.001) is statistically significant (z = 7.92, p < 0.001, two-sided). These results suggest that there is significantly more talk of the aim of research in terms of solutions than in terms of truth, knowledge, or understanding, in Mathematics publications.

The search results for the terms for the basic units of scientific progress in the context of (i.e., within ten words of) the progress term ‘goal’ in each subject are listed in Table 3. As before, all searches were conducted on October 14, 2020 and results were verified on October 30, 2020.

Table 3 Total number of publications, and those that contain the terms for the basic units of scientific progress within no more than 10 words of the progress term ‘goal’, by subject (

Again, since the JSTOR database does not contain the same number of publications in each subject, we need to compare the proportions of positive matches for each search pair rather than raw counts. Figure 2 depicts the proportions of the terms for the basic units of scientific progress in the context of (i.e., without ten words of) the progress term ‘goal’ by subject.

Fig. 2

Proportions of publications that contain the terms for the basic units of scientific progress within no more than 10 words of the progress term ‘goal’ by subject

As before, and consistent with Mizrahi’s (2021) results, our results show that the terms for the basic units of scientific progress do not occur all that frequently in the context of talk about the goals of scientific research in scientific publications (less than 0.008% overall). Still, the terms for the basic units of scientific progress that appear to occur more frequently than others are ‘understanding’ and ‘knowledge’. These results are also similar to Mizrahi’s (2021) results. Again, the noticeable exception is Mathematics, where ‘solution’ appears to be the most frequent term for a basic unit of scientific progress.

To find out whether the aforementioned differences in proportions are statistically significant, we conducted Welch’s t-tests. First, there was a statistically significant difference between the proportions of (“goal knowledge” ~ 10) publications (M = 0.001, SD = 0.001, N = 15) and (“goal truth” ~ 10) publications (M = 0.0003, SD = 0.0004, N = 15), t(16) = 2.58, p = 0.02, two-tailed. Second, there was a statistically significant difference between the proportions of (“goal understanding” ~ 10) publications (M = 0.002, SD = 0.001, N = 15) and (“goal truth” ~ 10) publications (M = 0.0003, SD = 0.0004, N = 15), t(15) = 3.45, p = 0.003, two-tailed. Third, there was no statistically significant difference between the proportions of (“goal knowledge” ~ 10) publications (M = 0.001, SD = 0.001, N = 15) and (“goal understanding” ~ 10) publications (M = 0.002, SD = 0.001, N = 15), t(28) = − 0.76, p = 0.45, two-tailed. These results are similar to the results obtained by Mizrahi (2021). They suggest that there is significantly more talk of the goal of scientific research in terms of knowledge than in terms of truth, and in terms of understanding than in terms of truth, in scientific publications. But the same cannot be said about talk of the goal of scientific research in terms of knowledge or in terms of understanding in scientific publications, since there is no statistically significant difference between the terms for basic units of scientific progress ‘understanding’ and ‘knowledge’ in the context of goal talk.

As mentioned in Sect. 1, beyond attempting to replicate Mizrahi’s (2021) results, we also set out to test the problem-solving model of scientific progress empirically, which Mizrahi (2021) did not test empirically in his study. In that respect, we found a statistically significant difference between the proportions of (“goal knowledge” ~ 10) publications (M = 0.001, SD = 0.001, N = 15) and (“goal solution” ~ 10) publications (M = 0.0005, SD = 0.0005, N = 15), t(17) = 2.18, p = 0.04, two-tailed. We also found a statistically significant difference between the proportions of (“goal understanding” ~ 10) publications (M = 0.002, SD = 0.001, N = 15) and (“goal solution” ~ 10) publications (M = 0.0005, SD = 0.0005, N = 15), t(17) = 3.06, p = 0.007, two-tailed. But we did not find a statistically significant difference between the proportions of (“goal truth” ~ 10) publications (M = 0.0003, SD = 0.0004, N = 15) and (“goal solution” ~ 10) publications (M = 0.0005, SD = 0.0005, N = 15), t(26) = − 0.87, p = 0.38, two-tailed. These results suggest that there is significantly more talk of the goal of scientific research in terms of knowledge than in terms of solutions, and in terms of understanding than in terms of solutions, in scientific publications, whereas talk of the goal of scientific research in terms of truth and in terms of solutions are equally infrequent in scientific publications, with no significant differences between proportions.

As mentioned above, Mathematics is a notable exception, as it is in the case of the ‘aim’ dataset. It is alone among the subjects tested in this empirical study in having ‘solution’ as the most frequently used term for a basic unit of scientific progress in the context of goal talk. To see if ‘solution’ is used significantly more frequently than the other terms for the basic units of scientific progress in Mathematics, we conducted additional z-tests for proportions. First, the difference between the proportion of Mathematics publications that contain the term for the basic unit of progress ‘solution’ in the context of goal talk (0.001) and the proportion of Mathematics publications that contain the term for the basic unit of progress ‘truth’ in the context of goal talk (0.0002) is statistically significant (z = 22.09, p < 0.001, two-sided). Second, the difference between the proportion of Mathematics publications that contain the term for the basic unit of progress ‘solution’ in the context of goal talk (0.001) and the proportion of Mathematics publications that contain the term for the basic unit of progress ‘knowledge’ in the context of goal talk (0.0009) is statistically significant (z = 10.00, p < 0.001, two-sided). Finally, the difference between the proportion of Mathematics publications that contain the term for the basic unit of progress ‘solution’ in the context of goal talk (0.001) and the proportion of Mathematics publications that contain the term for the basic unit of progress ‘understanding’ in the context of goal talk (0.001) is statistically significant (z = 3.73, p < 0.001, two-sided). As in the case of the ‘aim’ dataset, these results suggest that there is significantly more talk of the goal of research in terms of solutions than in terms of truth, knowledge, or understanding, in Mathematics publications.

4 Discussion

This quantitative, corpus-based study was designed as a follow-up study to Mizrahi (2021) in order to test the following philosophical accounts of scientific progress against scientific activity empirically: the semantic account, which defines scientific progress in terms of truth, the epistemic account, which defines scientific progress in terms of knowledge, and the noetic account, which defines scientific progress in terms of understanding. In this follow-up study, we also tested empirically a fourth philosophical account of scientific progress, which Mizrahi (2021) leaves out of his original study, against scientific activity. This account is the problem-solving model, which defines scientific progress in terms of the solved problem or solutions to scientific problems.

The results of our follow-up study are consistent with the results obtained by Mizrahi (2021). That is, our results suggest that practicing scientists use the terms for the basic units of scientific progress ‘knowledge’ and ‘understanding’ significantly more often than the term for the basic unit of scientific progress ‘truth’ when they talk about the aims and/or goals of scientific research in scientific publications. In that respect, the results of our study can also be construed as providing some empirical support to the epistemic and the noetic accounts over the semantic account of scientific progress. For, as we would expect if the epistemic account of scientific progress were true, we have found that practicing scientists talk about scientific progress in terms of knowledge significantly more often than in terms of truth in scientific publications. And, as we would expect if the noetic account of scientific progress were true, we have found that practicing scientists talk about scientific progress in terms of understanding significantly more often than in terms of truth in scientific publications. Since there are no significant differences between the frequencies with which practicing scientists talk about scientific progress in terms of knowledge or in terms of understanding, however, we cannot say that our results favor the epistemic account over the noetic account, or vice versa.

Our results do not merely replicate the results obtained by Mizrahi (2021). Indeed, they go beyond the results obtained by Mizrahi (2021) insofar as they suggest that the terms for the basic units of scientific progress ‘knowledge’ and ‘understanding’ are used by practicing scientists when they talk about the aims and/or goals of scientific research in scientific publications much more frequently than not only ‘truth’ but also ‘solution’. As we discussed in Sect. 1, Mizrahi (2021) left the problem-solving model of scientific progress out of his empirical study. In this study, by using the methods of text mining and corpus analysis, which are discussed in Sect. 2, we tested the problem-solving model of scientific progress against scientific activity empirically. The results of our study do not lend much empirical evidence in support of the problem-solving model of scientific progress over the other philosophical accounts of scientific progress (namely, the epistemic, noetic, and semantics accounts of scientific progress) tested in this empirical study. For, contrary to what we would expect if the problem-solving model of scientific progress were true, we have found that practicing scientists talk about scientific progress in terms of understanding or in terms of knowledge significantly more often than in terms of solutions in scientific publications. Accordingly, the results of our study can also be construed as providing some empirical support to the epistemic and the noetic accounts over the problem-solving model of scientific progress. Since there are no significant differences between the frequencies with which practicing scientists talk about scientific progress in terms of truth or in terms of solutions, however, we cannot say that our results favor the problem-solving model over the semantic account of scientific progress, or vice versa.

Of all the subjects in the JSTOR database we have tested in this study, however, Mathematics is an interesting exception as far as the problem-solving model of scientific progress is concerned. In Mathematics publications, we have found that there is significantly more talk of the aims and/or goals of research in terms of solutions than in terms of truth, knowledge, or understanding. In other words, both in the context of talk about the aim of research and in the context of talk about the goal of research, we have found that the term for the basic unit of progress ‘solution’ is used significantly more often than the terms for the basic units of progress ‘truth’, ‘knowledge’, and ‘understanding’ in Mathematics publications. For example, “we aim to define a new solution {z`, v`, u} of z₁z₂z₃ = − 1 for which z`, v` ∉ Ω_p” (Sendov & Sendov 2014, 5177). These findings suggest that, although we did not find empirical evidence in support of the problem-solving model as an accurate model of scientific progress in almost all of the subjects in the JSTOR database tested in this empirical study, it may still be an accurate model of progress in Mathematics.¹² For, only in Mathematics, we find that there is significantly more talk of the aims and/or goals of research in terms of solutions than in terms of truth, knowledge, or understanding, which is what we would expect to find if the problem-solving model were an accurate model of progress in Mathematics.¹³

Beyond these empirical findings, Mizrahi (2021) and the present empirical study have additional significance in terms of introducing scalable methods into philosophy of science. As Mizrahi (2021) points out, philosophers of science engaged in the scientific progress debate have mostly appealed to intuitions elicited from hypothetical cases or case studies from the history of science as evidence for or against philosophical accounts of scientific progress. For example, according to Niiniluoto (2014), “Historical case studies, which illustrate progress as increasing truthlikeness, include shifts from Ptolemy to Snell’s law of refraction (Niiniluoto 1984, 172), from Aristotle to Galileo’s law of free fall (Kieseppä 1996), from the phlogiston theory of combustion to Lavoisier’s oxygen theory (Niiniluoto 1999, 191), and from Rutherford to Bohr’s and Sommerfeld’s theories of the atom (Hettema & Kuipers 1995).” However, there are those who do not share Niiniluoto’s interpretation of these episodes from the history of science as evidence for the semantic account of scientific progress (see, e.g., Park 2017). Likewise, Bird (2007) provides several hypothetical cases of scientific progress (or lack thereof) and uses his intuitions about these hypothetical cases as evidence against the semantic account but for the epistemic account of scientific progress. However, there are those who do not share Bird’s intuitions about these hypothetical cases (see, e.g., Cevolani & Tambolo 2013). When there is such disagreement about intuitions elicited from hypothetical cases or interpretations of episodes from the history of science, the debate appears to be at an impasse. As Pitt (2001, 373) puts the point with respect to historical case studies, “if one starts with a case study, it is not clear where to go from there–for it is unreasonable to generalize from one case or even two or three.” A similar point applies to intuitions elicited from hypothetical cases as well. When one starts with one’s intuition about a hypothetical case, it is not clear where to go from there, for it is unreasonable to generalize from one’s intuition, especially if one’s intuition may be idiosyncratic.

By contrast, with the empirical methods of text mining and corpus analysis, it is clear where to go from here. Indeed, in this empirical study, we took the empirical methods of text mining and corpus analysis used by Mizrahi (2021) and scaled them up (vertically) to test a philosophical account of scientific progress that Mizrahi (2021) did not test in his empirical study (namely, the problem-solving model of scientific progress). This is an illustration of the scalability of the text mining and corpus analysis methods used by Mizrahi (2021) and in the present study. These empirical methods, then, can be scaled up (vertically) further to test other philosophical accounts of scientific progress. They can also be scaled out (horizontally) to test philosophical accounts against different corpora. In this study, we used text mined from the JSTOR database, as Mizrahi (2021) did as well. But the methods of text mining and corpus analysis we used are scalable in such a way that one can use other corpora against which to test philosophical accounts of scientific progress empirically.

5 Conclusion

In this paper, we have scaled up the methods of text mining and corpus analysis used by Mizrahi (2021) in order to test empirically a philosophical account of scientific progress that Mizrahi (2021) left out of his empirical study, namely, the problem-solving model, which defines scientific progress in terms of the solved problem or solutions to scientific problems. If the problem-solving model of scientific progress were true, we would expect to find that practicing scientists talk about scientific progress in terms of solutions much more than knowledge, truth, or understanding in scientific publications. In general, our results do not lend much empirical evidence in support of the problem-solving model of scientific progress over the other philosophical accounts of scientific progress (namely, the epistemic, noetic, and semantic accounts of scientific progress) tested in Mizrahi (2021) and in this follow-up study. For, contrary to what we would expect if the problem-solving model of scientific progress were true, we have found that practicing scientists talk about scientific progress in terms of understanding or in terms of knowledge significantly more often than in terms of solutions or in terms of truth in scientific publications.

Of all the subjects in the JSTOR database we have tested in this study, however, Mathematics is an interesting exception as far as the problem-solving model of scientific progress is concerned. For, in Mathematics alone, we have found that there is significantly more talk of the aims and/or goals of research in terms of solutions than in terms of truth, knowledge, or understanding. These findings suggest that, although we did not find empirical evidence in support of the problem-solving model as an accurate model of scientific progress in almost all of the subjects in the JSTOR database tested in this empirical study, it may still be an accurate model of progress in Mathematics.