1 Introduction

A large body of literature has shown that the use of diagrams can support students in reading of scientific texts (e.g. Slough et al., 2010) especially analytical diagrams which involve the integration of graphical information with corresponding textual information (Roth et al., 2005). It has been found that if a passage is accompanied by a diagram that explains concepts in the passage, it has the ability to support the reading of the text and the learning of scientific concepts (Carney & Levin, 2002).

In recent years, eye-tracking technologies have been used to investigate the cognitive processes that are involved in learners’ engagement and learning with text and diagrams (see Alemdag & Cagiltay, 2018 for a review) as conventional approaches like students’ reflections, interviews, and think-aloud approaches were unable to probe into the students’ cognitive processes while reading.

Paivio (1990) proposed the Dual coding theory which postulated that while processing cognitive information, humans make use of both the verbal system (which represents textual information) and the pictorial system (which represents visual images). This theory has been used as the underlying basis of subsequent empirical studies that investigated the integration of diagram into text. One of the classic studies undertaken in this area was conducted by Hegarty and Just (1993) on students reading a passage with a diagram of a pulley system in their native language. The findings revealed that irrespective of the participants’ background knowledge, they generally performed better in a subsequent comprehension test when reading both the text and diagram compared to when having either one of the representations. However, participants with lesser knowledge of mechanics generally had lower scores in the comprehension test. The amount of time their eyes were fixated on the various parts of the diagrams and their eye movements from text to diagram were also comparatively more. This strongly suggests that these participants had more difficulty in constructing a complete mental model of the pulley system. However, Hegarty and Just’s conclusion was based only on the reading of a particular content, i.e. mechanics. The same principle may not apply to the reading of other scientific texts.

Jian and Wu (2011, 2012) went a step further by attempting to categorize the strategies that readers would adopt when reading a Biology passage in Mandarin (the participants’ native language) containing a diagram. Both studies involved the participation of Taiwanese university students. In the first study, Jian and Wu (2011) discovered other strategies, in addition to the text-diagram referencing strategy proposed by Hegarty and Just (1993). Specifically, they discovered that about 60% of the participants adopted the text-diagram referencing strategy, while 35% looked at the diagram-first before reading the text. In a follow-up study, Jian and Wu (2012) found that 67% of the participants adopted the text-diagram referencing strategy and 21% looked at the diagram-first before reading the text. Interestingly, the remaining 12% completed reading the text before examining the diagram. Based on these findings, they inferred that the reading materials on mechanics used by Hegarty and Just (1993) possessed concepts that could only be mastered through reliance on spatial messages whereas their studies required only preliminary spatial relationship because the text was less dependent on the spatial messages in the diagram.

In a follow-up study, Jian and Wu (2014) undertook another study to investigate whether the various strategies used by the students would lead to different levels of reading efficiency and effectiveness. The study was undertaken on 71 students from the National Taiwan Normal University who majored in education, management, arts, and social science. Students with neuroscience background or knowledge relevant to the passage were excluded. The strategy that was most commonly used was still the text-diagram referencing strategy. They also found another reading strategy, i.e. text-only strategy. The overall findings revealed all readers who had referred to the diagram were equally effective readers, whereas the text-only readers were found to perform more poorly on factual questions.

2 Related Literature

As the number of non-native English speakers learning in English continues to grow, it is imperative to explore and understand their learning processes and strategies in more detail. This paper, as indicated above, particularly focuses on the strategies employed by English Second Language (ESL) learners when reading an illustrated Biology passage. It is necessary to point out here that despite the growing number of people learning through the medium of English as well as the growing number of research on reading of illustrated texts, little has been done to marry these two together.

Studies that have looked at ESL learners reading such materials indeed revealed interesting findings. Yusof et al. (2019), for instance, found that Malaysian primary school students in general paid more attention to the text as opposed to the graphic part of two graphic novels in English when reading. In another study, Pellicer-Sanchez et al. (2018) found similar results. Native Spanish-speaking children and adults alike displayed heavily text-based reading strategies of illustrated narratives in English. Attention on both text and picture parts of the narratives, respectively, was found to correlate with comprehension performance.

The findings that reading of illustrated texts is heavily text-based are not entirely limited to narratives. A large body of research on those reading scientific expository texts in a variety of first languages has been conducted much earlier and a similar reading pattern has been found. In a seminal study, Hannus and Hyönä (1999) observed that Finnish school children spent 94% of their reading time on the text part of illustrated Biology passages. In a more recent study, Makransky et al. (2019) found their university participants to spend approximately 70% of learning time studying text and less than 15% on illustrations regarding the development of lightning storms.

This brief review of literature has painted a worrying picture in the ways learners (both native and non-native users of a particular language) read and process an illustrated text. If the use of diagrams in addition to text can support learners in acquiring scientific concepts as proposed by the multimedia proponents, predominant attention on text at the expense of diagrams can potentially be harmful to learning. In their study, Jian and Wu (2014), as reported earlier, found some evidence to validate this theory. Native Mandarin speakers (NMS) who failed to attend to the diagram scored more poorly on a subsequent comprehension task compared to those who attended to it. Indeed, a number of studies have found that the more learners integrate information in diagrams with information in texts, the more they come to learn from the illustrated passages. More importantly, this finding appears to apply to materials in a variety of first languages as it has been found among native speakers of Mandarin (Jian & Wu, 2014), Finnish (Hannus & Hyönä, 1999), English (Johnson & Mayer, 2012), and Italian (Mason et al., 2015). If this relationship applies to those reading illustrated passages in their L1, the burning question would be, does this relationship similarly apply when students read illustrated passages in their non-native language?

3 The Present Study

This study used the same approach used by Jian and Wu (2014) in their study. However, some modifications were undertaken. The major difference of this study was that the participants were non-native speakers unlike that of Jian and Wu who undertook their studies on native speakers of Mandarin. Earlier research studies in this field were also undertaken on native speakers. This means that the findings of this study will offer new knowledge and insights to this field of study.

The two research questions of the study are as follows:

  1. (1)

    What are the reading strategies used by the Malaysian English as Second Language (ESL) adult readers when reading a Biology passage containing an analytical diagram in English, and is this dependent on English proficiency levels?

  2. (2)

    Is there a relationship between the ESL learners’ reading strategy and their comprehension performance when reading the abovementioned Biology passage?

4 Method

4.1 Participants

45 undergraduate majors of Social Sciences and Humanities at a Malaysian public university took part in this study. They were all native speakers of Bahasa Malaysia (or the Malay language) and English was their second language. In addition, all participants had normal or corrected-to-normal vision.

The present study also examined whether or not English proficiency levels play a role in the learners’ choice of reading strategy. Hence, English proficiency data was collected. This was gauged based on their performance in the Malaysian University English Examination (MUET), an English language proficiency test undertaken for university admissions. MUET has six bands. Students who achieved bands 5 and 6 are classified as high proficiency learners, 3 and 4 as average proficiency learners, and 1 and 2 as low proficiency learners. See Appendix B for the equivalent of these scores according to the CEFR framework.

The study participants all scored either a MUET band 3 or 4. The majority of the students from this faculty have English scores within this range. To differentiate between these groups, band 3 participants were classified as low proficient (LP) whereas band 4 participants as higher proficient (HP) learners. Altogether, there were 32 LP and 13 HP learners. Due to low precision and invalid data, 17 participants were removed, leaving only 28 participants; 20 LP and 8 HP learners.

4.2 Materials

Participants read two Biology texts in English. The first text was on respiration with a labeled diagram of the lungs and arrows showing the processes involved during breathing. The presentation of this text was followed by three comprehension questions. This was an introductory task used to expose and familiarize the participants with the procedures involved.

The second text was the actual stimulus used for the present study, describing the pathways of the fear response and awareness in the human brain. It was a neuroscience passage extracted from the Scientific American magazine (LeDoux, 1994, p. 56) and was the same text used by Jian and Wu (2014). This diagram was annotated with arrows as well as labels of relevant parts of the human brain. The last paragraph of the original text was removed from the present stimulus as it was considered too complicated.

A reading comprehension test comprising five yes/no questions was displayed on the screen, one question at a time, after participants completed their reading. Jian and Wu’s study utilized ten yes/no questions. The present study only used five of their questions which were deemed clear-cut and also in line with the following classification: two were text-based, two were pathway and one was integrative. Each correct answer was awarded one point, hence, the maximum score for the comprehension task was five (see Appendix A for the questions).

4.3 Procedures

Data collection took place in an eye-tracking research laboratory and each participant was tested individually. Participants were first briefed about the study and its purpose, before their informed consent was obtained. Next, they were asked to complete a background questionnaire. Once this stage was completed, the participants were seated before an eye-tracker, their eyes were calibrated on a nine-point grid, and they were then exposed to the introductory task. Upon completion of this stage, the participants’ eyes were again calibrated for the actual reading experiment.

Participants were instructed to read the passage at their own pace. Then they were required to answer the comprehension test. Eye movements while reading and response time for each comprehension question were recorded. The experiment took from 20 to 30 minutes to complete.

4.4 Apparatus

An EyeNTNU-120 eye tracker was used to collect eye movement data. The sampling rate of this eye tracker is 120 Hz and the angle error is 0.3 degree. It has two infrared light-emitting diodes (LEDs) that provide light which would reflect on the cornea of the left eye to detect eye movement. The output energy of the LEDs is 3.5 mW/cm2 at a working distance of 4 cm. The EyeNTNU-120 eye-tracking system includes three software tools: a Hot Zone analyzer, a Region of Interest (ROI) Tool, and a fixation calculator. A participant’s eye movement data can be detected using the Hot Zone analyzer. The ROI Tool is used to define the ROIs whereas the fixation calculator generates matrices such as total fixation duration (TFD), number of fixations (NOF), and duration of first fixations (FFD). The eye movement data were collected at a 120 Hz sampling rate.

5 Findings

5.1 Comparing the Findings of the Present Study with Jian and Wu’s Studies

Four reading strategies were identified in the present study; text-only, text-diagram, text-first, and diagram-first. The distribution of participants into these four strategies is displayed in the second column of Table 1. Most of the participants (50%) displayed a text-only reading strategy and only one displayed the diagram-first reading strategy.

Table 1 Comparing the strategies of the participants in the present study with that of Jian and Wu (2011, 2012, 2014)
Full size table

Table 1 further revealed that the most common strategy used by the students in Jian and Wu’s studies (2011, 2012) was the text-diagram strategies (60, 67% respectively) followed by the diagram-first strategy (35, 21% respectively) and text-first strategy (0, 12% respectively). Interestingly, Jian and Wu (2014) found 11% of their students using the text-only strategy, a strategy not used by students in their previous studies. This new pattern is also evident in the present study whereby 50% of the participants used text-only strategy. In the present study, text-only strategy was followed by the text-diagram strategy (32.1%) and text-first strategy (14.3%). Only one student in the present study used the diagram-first strategy.

Another issue worth looking into is whether or not the reading strategies are dependent on the learner’s English proficiency level. Table 2 reports the distribution of the LP and HP learners by reading strategies. As can be seen from this table, while the majority of the LP learners (i.e. 60%) used the text-only strategy when reading the illustrated text, only 25% of the HP learners used it. The majority of the HP learners (62.5%) appeared to use the text-diagram strategy, while only 20% of the LP learners used it.

Table 2 Reading strategies used by the LP and HP learners
Full size table

To determine whether these strategies were associated with the ESL learners’ English proficiency level, a Fisher’s exact test was performed. It revealed no significant relationship between the two variables, p = .172, suggesting that the distribution of participants across the reading strategies was not associated with whether they had low or higher English proficiency.

5.2 Examining the Relationship Between Reading Strategies and Comprehension Performance

Table 3 displays the mean scores of each of the reading strategy groups. The non-parametric Kruskal-Wallis test was used to examine the differences in comprehension scores and it indicated that the differences were indeed significant, p = .022. Pairwise comparisons indicated that those who displayed the text-diagram strategy significantly outperformed those displaying the text-first strategy, p = .033. No other significant differences were found, all p values > .05.

Table 3 Mean scores of reading comprehension test by reading strategy
Full size table

6 Discussion

The findings are discussed in line with the two research questions.

Research Question 1: What are the reading strategies used by the Malaysian ESL adult readers when reading a Biology passage containing an analytical diagram in English, and is this dependent on English proficiency levels?

Similar to Jian and Wu’s studies, the present study also found that the ESL learners employed the text-diagram, text-first, and diagram-first strategies employed by the native Mandarin speakers in Jian and Wu studies. However, another reading strategy was the most dominant among the ESL learners. This was the text-only strategy which was only sparingly employed by the Mandarin speakers in Jian and Wu’s (2014) study. Although, a heavily text-based reading of illustrated passages has been found in earlier research with native speakers (Hannus & Hyönä, 1999; Makransky et al., 2019) and non-native speakers (Pellicer-Sanchez et al., 2018; Yusof et al., 2019), it was nevertheless surprising (and worrying) that half of the ESL learners completely ignored the diagram in the present study.

Perhaps, this reliance on text was driven by the ESL learners’ difficulty in comprehending the English text. A look at the text’s readability scores substantiated this claim. Its Flesch reading ease was 40.6, and its Flesch-Kincaid grade level was 12.5, indicating that on the whole, the text was relatively difficult. The ESL learners might have felt that they needed to spend all of their reading time on the text part of the passage to both understand it and to be able to answer the subsequent comprehension questions. As this is a Biology passage and all the learners are Social Sciences and Humanities students, unfamiliarity with the text content and the structure of such texts might have also contributed to this text reliance.

The above argument suggests that the ESL learners’ English proficiency might have explained the reading strategies that they employed. Such relationship, however, was not evident from the findings, despite the indication. However, the findings did indicate that more HP than LP learners used the text-diagram strategy, and more LP than HP learners used the text-only strategy. It is postulated that since both groups of learners fell into the category of average proficiency students and the difference in proficiency levels between them might not be sufficiently substantial to bring about a significant result. Hence, a larger sample of participants with more marked differences in English proficiency could be used in future studies to examine this relationship in more detail.

Research Question 2: Is there a relationship between the ESL learners’ reading strategy and their comprehension performance when reading the abovementioned Biology passage?

Research has shown that strategies that learners employed when reading illustrated science passages in their native language do implicate their comprehension and learning of those passages (Hannus & Hyönä, 1999; Jian & Wu, 2014; Johnson & Mayer, 2012; Mason et al., 2015). When it comes to reading in a non-native language, the present study revealed similar findings. More specifically, text-diagram learners appeared to learn fairly well from the passage and this is clearly in line with the idea that diagrams serve to facilitate the reading of the text and the learning of scientific concepts, in general (Carney & Levin, 2002). This idea, therefore, appears to similarly apply to the reading of illustrated passages in the learning of a second language.

Additionally, the present study found a clear difference between how the text-diagram and the text-first learners performed in the comprehension test. To understand why the text-first learners fared poorly in the test, it is necessary to understand how this strategy was computed. A text-first learner was someone who completed reading of the text before inspecting the diagram. It could be the case that these learners have already formed a mental representation of the passage content from their text reading before they inspected the diagram. Upon inspection of the diagram, they might have failed to integrate the diagrammatic information with the mental representation they have already constructed, and worse, this mental representation might have become incoherent. Since the text-only learners did not inspect the diagram at all, their mental representation of the passage remained intact and they could rely on this alone to tackle the comprehension questions.

The text-diagram learners were those who shifted between the text and diagram parts of the passage frequently (Jian & Wu, 2014). It would appear that these learners were able to construct a more coherent mental representation of the passage by combining textual and diagrammatic information than the other types of learners. Since they frequently gazed back and forth between the two, their comprehension would be continuously updated, resulting in a more coherent and integrated mental representation. The superiority of this strategy confirms the necessity for students to learn how to integrate textual and diagrammatic information in order to benefit from reading illustrated materials.

7 Implications and Conclusion

This paper, in general, reaffirms findings on the reading of illustrated science passages in the native language. More specifically, our study found that similar to those reading in their native language, ESL learners also displayed a variety of reading strategies with the text-only reading strategy being the most common. This suggests that reliance on text applies when reading in both native and non-native languages. Unfortunately, as our study and previous native language studies have shown, this strategy is not the most effective for learning with such texts. The text-diagram strategy involving learners shifting their attention between the text and diagram frequently is more effective in enhancing comprehension.

Given these findings, we can therefore see that issues surrounding the reading of illustrated science passages in the native and non-native languages are rather similar. It is worrying to find that what works for comprehension and learning of these materials is not much practiced by the learners themselves. This study therefore has considerable implications for the educational practices. First of all, it would be useful for teachers to remember that a learner’s text reliance could have stemmed from their own inattention to diagrams (McTigue & Flowers, 2011). Teachers are therefore recommended to teach how to use diagrams more seriously. Secondly, learners, unsurprisingly, require adequate instructional support to help them learn and master any given strategy. But what is adequate support when it comes to reading illustrated passages? Existing literature has indicated that learners benefit extensively when supported via direct and explicit instructions (Cromley et al., 2013; McTigue & Flowers, 2011). Modeling the most effective strategy to learners, therefore, can potentially lead to adoption of this strategy. Before asking learners to read an illustrated passage, teachers can, for instance, use an eye movement modeling example (EMM) for their learners to emulate. An EMME shows a replay of an expert’s reading behavior and therefore allows learners to identify and learn the best strategy via a concrete model (Mason et al., 2015).