Keywords

1 Introduction

Research articles (RAs) are the typical means scientists use for communicating their findings to the scientific community (Gross et al. 2002). There is no doubt that reading RAs is a major skill that university science students should learn. In secondary science education, RAs are increasingly used as authentic or adapted sources for teaching scientific knowledge, showing students the ways scientists use to communicate their findings and teaching them about the nature of science (Yarden et al. 2001; Norris et al. 2009). One of the reasons for this trend is that authentic RAs are readily accessible for people outside the academic community through open access journals available on the Internet.

Studies have shown that students struggle with reading scientific texts (e.g., Guilford 2001; Yarden et al. 2001; Van Lacum et al. 2012), but not much research has been done on effective pedagogical strategies to teach students to read RAs. For understanding RAs, both knowledge of the research discipline and knowledge of the genre of RAs are essential. Genre knowledge comprises knowledge of conventions and structural and rhetorical features of RAs.

To improve students’ reading abilities with regard to RAs, we want to make them familiar with one specific aspect of genre knowledge: the structural features of the genre. Several authors have suggested a relation between reading ability and knowledge about the structural characteristics of a text (Hill et al. 1982; Samuels et al. 1988; Blanton 1990; Swales 1990; Du Boulay 1999). In other words, by teaching students about the structural features of RAs, we expect them to become better readers. This is in line with Norris and Phillips (2003), who stated that a scientifically literate person should be able to determine the different types of statements in a scientific text (hypothesis, evidence, conclusion, expressed doubt, etc.).

Therefore, we think that it is important to design pedagogical tools that support science students in reading RAs. To achieve this, we will make use of the structural characteristics of RAs in the natural sciences domain. In this paper, we will describe the development and validation of an argumentation model, called the Scientific Argumentation Model (SAM), which focuses on these characteristics and may serve as a heuristic for science students in secondary or higher education when they learn to read RAs and identify the authors’ argument.

In the next section, we describe the structure of RAs from the perspectives of genre analysis and argumentation theory. Then in Sect. 12.3, we present a synthesis of these two perspectives. In Sect. 12.4, we present the heuristic SAM. In Sect. 12.5, we describe the first results on the validation of the heuristic.

2 Theoretical Perspectives

Genre analysts have intensively studied written genres, especially RAs. The goal of genre analysis is “to describe the communicative purposes of a text by categorizing the various discourse units within the text according to their communicative purposes or rhetorical moves” (Connor et al. 2007, p. 23). A rhetorical move refers to “a section of a text that performs a specific communicative function” (ibid., p. 23). The work in genre analysis has produced rich descriptions of the rhetorical moves that occur in the different sections (Introduction, Method, Results, and Discussion sections) of RAs. However, genre analysts have paid relatively little attention to the relations between rhetorical moves in the different sections of RAs. These relations are of an argumentative nature. As stated by Du Boulay (1999), an argument refers to “authors’ claims (including their degree of strength), his or her theoretical orientation, the quality of the evidence produced or demonstrated and how this is linked to theory” (p. 148).

2.1 Argumentation in Research Articles

Several argumentation frameworks have been used to describe the argumentative structure of scientific papers. For our model, we built on Kelly and Takao’s (2002) epistemic levels of argument and on Toulmin’s argumentation model. Toulmin (1958) devised a “logically candid layout of arguments” (p. 95) that may be used in a variety of cases. The Toulmin scheme consists of the following elements: data, warrant, backing, qualifier, rebuttal, and claim. The data are the facts that form the foundation of the claim. A warrant is a proposition that is used to make the step from the data to claim. These can be rules, principles, and so forth. A warrant is often supported by a backing, “without which the warrants themselves would possess neither authority nor currency” (ibid., p. 103). The qualifier is “the reference to the degree of force which our data confer on our claim in virtue of our warrant” (ibid., p. 101). The rebuttal indicates the circumstances under which the warrant is not applicable. Because the Toulmin scheme is very visual and accessible, it is widely used in educational settings, but it has also been criticized for a number of reasons (Sampson and Clark 2008). The first reason is ambiguity in its categories. Toulmin (1958) himself has already pointed at the confusion that may occur between backings and warrants. In RAs, backings often take the form of rather implicit assumptions held by the scientific community and therefore difficult to identify by novice readers. The second reason is that the model cannot easily be applied to complex arguments frequently used in scientific texts, for example, embedded claims in larger arguments (Kelly and Takao 2002). Some authors have elaborated on the original Toulmin framework for scientific arguments. For instance, Thompson (1993) used a two-step version of the Toulmin scheme in the analysis of RAs. Although more successful in dealing with the complex arguments in RAs, these approaches lack the simplicity of the original Toulmin scheme and require epistemological knowledge that cannot be expected from novice readers. For these reasons, we tried to conceive an approach that is applicable for reading RAs and still easy to use by students.

2.2 Rhetorical Moves in Research Articles

For the development of our heuristic, we built on several genre analysis studies (Swales 1990; Thompson 1993; Dudley-Evans 1994; Nwogu 1997; Williams 1999; Peacock 2002; Kanoksilapatham 2005). In the following, we show how genre analysts analyzed the four different sections in RAs: Introduction, Method, Results, and Discussion (IMRD). This four-part structure is found across a wide variety of disciplines in the natural sciences.

In the Introduction section of RAs, Swales (1990) identified three rhetorical moves. The first move is establishing a territory, in which the significance of the research field is explained. This is done by the so-called steps (or sub-moves): claiming centrality, making topic generalization(s), and/or reviewing items of previous research. The second move is establishing a niche, in which the authors explain the reasons behind their particular research. These reasons can consist of counter-claiming, indicating a gap of knowledge, question raising, or continuing a tradition. Occupying the niche is the third move, in which the authors’ research is introduced. This move can consist of specifying the purpose of the study, followed by an announcement of the principal findings and the structure of the article. Nwogu (1997) and Kanoksilapatham (2005) use more or less the same categories, albeit with some alterations (Table 12.1).

Table 12.1 Moves in the Introduction section of research articles
Full size table

The Method section of RAs describes the procedures and materials the authors used to obtain their results. Procedures are usually only explained and justified if they are new or adapted. Standard procedures derived from previous studies are simply listed with a reference (Penrose and Katz 1998). Nwogu (1997) identified three moves in the Method section of medical RAs: (1) describing data collection procedures, (2) describing experimental procedures, and (3) describing data analysis procedures. Kanoksilapatham (2005) identified four moves in biochemical RAs: (1) describing materials, (2) describing experimental procedures, (3) detailing equipment, and (4) describing statistical procedures. Like Nwogu, Kanoksilapatham does not mention a move that justifies procedures or methodology, possibly because his analysis suggests that in biochemical RAs, this move occurs in the Results section and not in the Method section.

The Results section is used by the authors to present their collected data. Data are reduced (e.g., by calculating averages) and processed into tables and graphs (Penrose and Katz 1998). Authors use the so-called pointers to link tables or graphs with textual statements (Penrose and Katz 1998). Authors not only present their data; they also comment on them, for example, by providing explanations or drawing preliminary conclusions. Genre analysts (Thompson 1993; Nwogu 1997; Williams 1999; Kanoksilapatham 2005) identified a number of rhetorical moves in the Results section (Table 12.2). Nwogu (1997) distinguishes moves that indicate consistent (i.e., expected) observations and moves that indicate non-consistent (i.e., unexpected) observations. Kanoksilapatham (2005) distinguishes four moves (Table 12.2). Move 11 is divided into several steps representing similar descriptions of moves by Thompson (1993) and by Williams (1999). Williams (1999) makes a distinction between presentational and comment moves. Presentational moves contain procedural information (statements about how and why the data has been produced) and what the findings were, while comment moves are – among other things – explanations of findings, interpretations of findings, or comparisons with literature. Interestingly, Thompson (1993) and Kanoksilapatham (2005) identify methodological justifications in the Results section.

Table 12.2 Moves in the Results section of research articles
Full size table

The Discussion section is probably the most variable and complex part of an RA. There is a direct relationship between the Discussion and the Introduction section: “Whereas the introduction introduces the research question and reviews the state of knowledge in the field that motivated the question, the discussion explains how the question has been answered (at least in part) by the new research and shows how the field’s knowledge is changed with the addition of this new knowledge” (Penrose and Katz 1998, p. 57). The Discussion states the main knowledge claim (also called main conclusion or thesis). When expressing knowledge claims, authors generally use hedges: words like “likely,” “probably,” or “may”, expressing the uncertainty of a claim (Hyland 1998).

Table 12.3 lists the Discussion section’s moves mentioned by Dudley-Evans (1994), Nwogu (1997), Peacock (2002), and Kanoksilapatham (2005). Dudley-Evans’ (1994) nine moves often occur in recurring move cycles. For example, a statement of result or finding is regularly followed by a reference to previous research. Peacock (2002) made some small alterations to Dudley-Evans’ model. Kanoksilapatham’s model (2005) is not so much dissimilar, but within one move, several steps are distinguished, which Peacock distinguishes as separate moves. All four studies mention limitations, being remarks about “problems with errors, methods, and validity” (Ioannidis 2007, p. 324). Limitations are also called weaknesses, caveats, or shortcomings. Aspects not explicitly mentioned by genre analysts are alternative explanations or alternative interpretations of results. These doubts, as Suppe (1998) calls them, are included in a selective way: “…confining attention only to those specific doubts the discipline recognizes as legitimate counterpossibilities” (Suppe 1998, p. 384). According to Suppe, doubts are often coupled with rejoinders, which impeach these alternatives as much as possible. The explicitation of these doubts and rejoinders plays an important part in the persuasive process. Genre analysts do mention recommendations for further research, but remarks summarizing the potential significance of the findings (e.g., possible changes in clinical practice) are often not included in their descriptions of the Discussion section (Alexandrov 2004).

Table 12.3 Moves in the Discussion section of research articles
Full size table

3 Synthesis: The Argumentative Structure of Research Articles

We wanted to use the rhetorical moves described above for the development of a heuristic, centered around argumentation that can be applied by novice readers of RAs. With this purpose in mind, we formulated four criteria to which our heuristic should adhere:

  1. 1.

    The heuristic describes the RA’s rhetorical moves that play an important role in the authors’ argumentation.

  2. 2.

    The heuristic describes the relations between these rhetorical moves in a visual way.

  3. 3.

    The heuristic is generic; it is applicable to a broad range of RAs from different sciences, in all variations that different journals exhibit.

  4. 4.

    Because novice readers of RAs should be able to work with it, the descriptions of the rhetorical moves are as simple and as unambiguous as possible.

In this section, we describe how we selected, combined, and supplemented the rhetorical moves identified by various genre analysts in the Introduction, Method, Results, and Discussion sections of RAs. Then we present the design of a schematic representation of the argumentative structure of RAs, stretching from the very reason to undertake the research, through data collection and interpretation, to the outcome of the study. It contains those text elements corresponding with rhetorical moves that, taken from the text and reassembled into a scheme, coherently show the line of reasoning in RAs as a whole.

As stated above, Swales (1990) identified three rhetorical moves in the Introduction section: establishing a territory, establishing a niche, and occupying the niche. Nwogu (1997) and Kanoksilapatham (2005) described similar moves (Table 12.1). The first move describes the research area and is not a part in the RA’s argument. Establishing a niche and occupying the niche are more specific for a study, and the authors use these moves to describe the “gap of knowledge” and the reasons why their research is important. Because the concept of niches is probably difficult to grasp for students, we avoided this term. We called the establishment of a territory and niche the motive of the study (why was the study done?) and the occupation of a niche the objective of the study (what did the authors want do know?). The objective may be formulated as a research question, a research aim, or a hypothesis that needs to be tested. The motive and objective are related to each other, as the objective emerges from the motive.

Nwogu (1997) and Kanoksilapatham (2005) both show that the Method section contains moves that describe experimental procedures and moves that describe data analysis/statistical procedures (or, as we called this move, data processing procedures). Additionally, Nwogu (1997) mentions a move that describes data collection procedures. Kanoksilapatham (2005) also mentions a move that describes the materials and a move that details the equipment. For the sake of simplicity, we share these methodological aspects under experimental procedures. So this leaves us with two moves: (1) description of experimental procedures and (2) description of data processing procedures. The former is directly connected to the objective, because the objective influences the choice of experimental procedures. In turn, the experimental procedures will influence the data processing procedures.

Statement of finding (after Williams 1999) is arguably one of the most essential moves in the Results section. These are statements that provide interpretations of the inscriptions. Roth, Bowen, and McGinn (1999) describe inscriptions (following Latour 1987) as “representations other than text” (p. 977). Inscriptions can be “readings from simple devices, recordings from automated devices, computer screen output, photographs, micrographs, data tables, graphs, and equations” (p. 978). Statements of findings can lead to a preliminary conclusion. This is a generalization/interpretation of a statement of finding (Kanoksilapatham 2005). Preliminary (or sub-) conclusions can be located in the Results or Discussion section. A statement of finding could be “Rats which were given drug X had a lower blood pressure than rats in the control group.” A preliminary conclusion could be “Drug X lowers the blood pressure in rats.” Of course, the difference between statements of findings and preliminary conclusions will not always be clear-cut. It is possible that authors only present statements of findings in the Results section and save their preliminary conclusions for the Discussion section. Inscriptions, statements of findings, and preliminary conclusions are comparable to Kelly and Takao’s (2002) epistemic levels of claims: these levels describe how specific statements (e.g., simple interpretations of data) develop into more generalized statements, moving away from the original observation or measurement, called externalizations (Pinch 1985). The lines of reasoning between these epistemic levels may be described as a chain of interpretative steps.

One of the most important elements in the Discussion section is the claim: a generalization arising from the results (Dudley-Evans 1994). In our selection, we included two types of claims, preliminary conclusions and the main conclusion. The main conclusion is supported by a wide variety of moves that are located in the Results and Discussion sections: statements of findings, preliminary conclusions, and references to previous research. Together, these moves (and inscriptions) form the supports of the main conclusion. A main conclusion may include a hedge or qualifier, expressing the (un)certainty of the conclusion. The main conclusion may lead to remarks about the potential significance of the findings and their possible influence on society or practice (Alexandrov 2004) and recommendations for further research. We call these remarks implications. We grouped comments about unexpected outcomes and limitations as counterarguments, because they shed doubt on the validity or generalizability of the main conclusion. Suppe’s (1998) doubts fall into this category. A counterargument may be “weakened” by refutations (or rejoinders, as Suppe calls them). For instance, explanations for unexpected results can serve as refutations of counterarguments. We did not incorporate Dudley-Evans’ (1994) information move in our selection, because this move (which describes background information about theory, research aim, methodology, or previous research) is a recapitulation of the motive and/or objective.

In summary, we think that the following moves adequately describe the line of reasoning of RAs: motive, objective, experimental procedures, data processing procedures, inscriptions, statements of findings, preliminary conclusions, main conclusion, implication, counterarguments, and refutations. Due to their extratextual nature, inscriptions are not identified as rhetorical moves by genre analysts. However, because of their importance, we will subsequently group them together with rhetorical moves. Thus, we conceived an argumentation scheme that depicts the abovementioned moves and their relations (Fig. 12.1). The circles represent the four different sections of a typical RA (Introduction, Method, Results, and Discussion sections). In these circles, the connected moves are shown. The central arrow of the scheme is the connection between the objective and the main conclusion.

Fig. 12.1
figure 1

The research article’s argumentation scheme. All moves (boxes) and their relations (arrows and bars) are explained in Sect. 12.3. The darker boxes represent the simplified scheme (Sect. 12.4)

Full size image

Below the central arrow, the supports – which form the pillars that justify the main conclusion – are placed: inscriptions, statements of findings, preliminary conclusions, and references to previous research. Supports in these pillars are connected to each other, forming chains. These chains express the increasing abstraction in supports from inscriptions to more general statements (statements of finding and preliminary conclusions), similar to Kelly and Takao’s (2002) description of epistemic levels. Counterarguments are placed above the central arrow indicating that their argumentative function is the opposite of that of supports: they weaken the main conclusion. In turn, counterarguments may be weakened or refuted by refutations. Finally, the stems, connecting the circles to the “body of scientific knowledge,” symbolize how the references in the Introduction, Method, and Discussion sections connect RAs with the body of scientific knowledge (Amsterdamska and Leydesdorff 1989). References may serve to highlight the relevance of the study, justify the used methods, provide further support for claims, or indicate how the findings can solve a certain problem (Gilbert 1977).

4 A Heuristic for Reading Research Articles

With the criteria mentioned at the beginning of Sect. 12.3 in mind, we stripped the schematic representation of the argumentative structure of RAs down to such a level that it contained only those elements that are understandable for non-expert readers: the darker boxes, connected by the black arrows and bars in Fig. 12.1. We omitted the elements from the Method section and their connections, because understanding this section requires a significant amount of prior knowledge due to its technical details. Although this simplified scheme lacks the lowermost connection from the objective to the supports through elements from the Method section, it still features a complete line of reasoning because of the direct link between objective and main conclusion. We kept the chains of supports but decided to simplify the terminology by giving inscriptions, statements of findings, preliminary conclusions, and references all the same name: supports. This makes the terminology more straightforward for students. The resulting heuristic, called SAM (Scientific Argumentation Model), consists of descriptions of seven rhetorical moves (see below) and a simplified version of the RA’s argumentative structure (Fig. 12.1, darker boxes, and Fig. 12.2).

Fig. 12.2
figure 2

Schematic representation of Scientific Argumentation Model (SAM)

Full size image

The moves of SAM are described to the students as follows:

  1. 1.

    Motive: Statement indicating why the research was done (e.g., a gap of knowledge, contradictory findings). The motive leads to the objective.

  2. 2.

    Objective: Statement about what the authors wanted to know (may be formulated as a research question, a research aim, or a hypothesis).

  3. 3.

    Main conclusion: Statement about the main outcome of the research. The main conclusion is closely connected to the objective. It answers the research question, it says to what extent the research aim was achieved, or it states whether the hypothesis was supported by the evidence. The main conclusion will lead to implications.

  4. 4.

    Implication: Statement indicating the consequences of the research. This may be a recommendation, a statement about the applicability of the results (in the scientific community or society), or a suggestion for future research.

  5. 5.

    Support: Statement used by the authors to justify their main conclusion. These statements can be based on the authors’ own data or can be drawn from literature (references). Supports may be presented in the so-called support chains. For example, Table → Interpretation of the table’s data in the Results section (statement of finding) → Further interpretation of the table’s data in the Discussion section (preliminary conclusion).

  6. 6.

    Counterargument: Statement that weaken the main conclusion. For example, possible methodological flaws, anomalous data, results that contradict previous studies, or alternative explanations. Counterarguments are sometimes presented as limitations.

  7. 7.

    Refutation: Statement that weakens or refutes a counterargument.

These descriptions mention the so-called content-based features of the moves (Paltridge 1994), which may help to recognize moves by their function. Further, we added organizational and lexical features of all moves to our descriptions. Organizational features describe the location of the move in RAs. For example, the objective is always found in the Introduction section. Lexical features are words/phrases that may trigger the reader to identify a certain statement as a move. For instance, reporting verbs like “suggest” or “show” may signal a conclusion (Bloch 2010), and “it is still unknown” is a phrase indicating a motive. Thus, our heuristic consists of: a set of seven moves characterizing the argumentative structure of RAs; a description of their content-based, organizational, and lexical features; and the SAM scheme.

5 Validation of the Heuristic

To validate the heuristic, we selected ten empirical RAs from the journal Science from two different domains: astronomy and biomedical science. By using RAs from one journal, it was guaranteed that all have the same format. We randomly drew five RAs per domain, based on the categorization Science uses, from the period between March 2006 and 2011. Within this time frame, each domain contained more than ten reports. This enabled us to take a random sample. Two graduate students, one with a major in astronomy and one with a major in medical biology, read five RAs each and identified the moves. The third author read all five RAs of each domain and identified the moves. Differences in the analysis between the graduate students and the third author were negligible. The results show that there are some noticeable differences between the papers of both domains. The average frequency of motives, main conclusions, implications, and support chains seems somewhat higher in astronomy papers compared to biomedical RAs, although this may depend on the sample of papers (Table 12.4).

Table 12.4 Average frequencies and frequency ranges of moves per research article from different disciplines
Full size table

Most notable is the difference in the number of counterarguments. Furthermore, the number of refutations in astronomy RAs does not match the number of counterarguments. This stems from a particular argumentation pattern within these astronomy RAs, starting with one or two objectives. The authors then present preliminary conclusions, which they subsequently reject as being incorrect using a counterargument, after which they present other data from another observation trying to confirm their model or theory. This is repeated several times before eventually the final main conclusions are drawn. This pattern was not observed in biomedical RAs, where the authors actually refuted all counterarguments. This difference in argumentation may be caused by the nature of research in both disciplines: research in astronomy focuses on validating models and testing theoretical predictions using multiple observations, while biomedical research focuses, for instance, on investigating the effects of a treatment to prevent or cure a disease using a single experiment or a set of closely related experiments measuring several parameters.

6 Discussion

In this paper, we described how genre analysis and argumentation theory were used to design a model representing an RA’s argumentation structure. This model (the Scientific Argumentation Model or SAM) may serve as a heuristic for novice readers of RAs. In this way, we can extend their genre knowledge with respect to RAs and help them with their enculturation into the scientific community.

Furthermore, SAM is expected to give science students more insight into the persuasive nature of RAs. Students are used to textbooks that tend to neglect the processes by which scientific knowledge is produced (Duncan et al. 2011). As a result, students view academic texts as “autonomous, uncontested and unnegotiated, unencumbered by the values and oppositions that they may freely recognize in their out-of-school lives and textual experiences” (Johns 2002, pp. 239–240). By focusing on the argumentative structure, students may become more conscious of the idea that RAs are persuasive texts.

In Sect. 12.3, we formulated four criteria to which our heuristic should adhere. The first criterion is that the moves that play an important role in the authors’ argumentation should be included in the heuristic. By applying ideas from argumentation theory and genre analysis, we made a careful selection of seven moves in such a way that we expect that it will be a valid representation of the RA’s line of reasoning.

The second criterion concerns the relations between moves and their visual representation in the SAM scheme (Fig. 12.2). Relations in SAM are pictured by arrows and bars. Arrows represent a sequence – not necessarily a chronology in the research process, but a sequence in the authors’ argument as presented in RAs – from motive and objective to conclusion and implications. The vertical bars represent the evidence and contra-evidence for the conclusion. Different than Toulmin (1958), we represented the evidence and the contra-evidence on different sides of the central arrow. It is our expectation that this will visualize for students how the strength of a conclusion is determined by the balance between supports, counterarguments, and their refutations.

The third criterion about the general nature of the heuristic implies that it should be applicable to a wide variety of RAs from different disciplines. Our first analysis demonstrated that the heuristic is indeed applicable to a number of astronomy and biomedical RAs. However, the applicability of the heuristic to reading RAs in various domains should be further investigated.

According to the fourth criterion, novice readers should be able to work with the heuristic. We already used SAM for teaching pre-university and first-year undergraduate students how to read authentic RAs (Koeneman et al. 2013; Van Lacum et al. 2014). These studies show that the heuristic is a promising method for supporting students’ reading.

The visual representation of SAM shows some similarities with the Toulmin scheme (Toulmin 1958). However, we believe that our model is more suitable for educational purposes with respect to RAs. For example, Toulmin’s scheme does not include explicitly a motive or objective. Our reason for including motive and objective in the model is that they help novice readers with finding other moves. After readers locate the motive and objective – which is relatively easy for them (Van Lacum et al. 2012) – they can deduce what the RA’s main conclusion is. Since the main conclusion srelates directly to the objective, it is often worded in a similar way. This points to a major difference between our model and the models we found in literature. The latter are devised and used to analyze RAs from a linguistic or philosophical perspective and were not intended to serve as a pedagogical aid.

We believe that SAM has many potential uses in science education. For example, students may be assigned to individually read an RA and construct a SAM scheme. Then they can compare their schemes in small groups and together analyze the RA’s argumentative structure. Since students share the same vocabulary – the model’s framework – they can easily discuss their views on the RA’s argumentation and reach a deeper understanding of the text. Furthermore, SAM may be used by students to compose reports on their own research projects or lab work. For example, after having done experimental work, students may be asked to organize their results following the pattern of the model. As the SAM scheme lacks the Method section’s moves, educators might choose to use the more comprehensive version of the model instead.

It could be argued that a focus on genre knowledge stifles students’ creativity and self-expression. However, genre knowledge allows them to explore the possibilities of a genre. According to Cooper (1998), genre knowledge opens up “many possibilities for students and leaving countless decisions for them to make as they develop and shape their arguments” (p. 48).