Introduction

“A picture is worth a thousand words,” but only if one can comprehend it. Otherwise, it is just a page decoration at best and a waste of space at worst. Considering how many graphics are found in textbooks and other non-fiction texts for children and adults and the amount of information they contain, it is important to understand what readers are doing when they see these graphics, how they are processing them, how they are understanding them, and whether these graphics are contributing to their overall comprehension of the text. This study begins to address some of these questions by investigating: What is the relationship, if any, between children’s processes prompted by the graphics in informational text and their overall comprehension of those texts?

Theoretical framework

This study is grounded in semiotic (e.g., Jewitt & Oyama, 2001), multiple literacies (The New London Group, 1996), and dual coding (e.g., Paivio, 1991) theories and in recognition of previous research on the importance of informational text and graphics in today’s society. In the twenty-first century, reading and writing printed words is not enough—one must be able to read and communicate meaning with spoken, written, and visual text (e.g., IRA/NCTE, 1996; The New London Group, 1996). Therefore, the graphics in text—which I define as any photograph or illustration in narrative or informational text including but not limited to diagrams, maps, graphs, and tables—are meaningful signs that need to be understood in order to make meaning (e.g., Jewitt & Oyama, 2001).

These graphics are particularly important in informational text, where they can help to organize ideas (e.g., a flow chart explaining how fossils are made), make abstract ideas more concrete (e.g., an illustration of the circulatory system as plumbing, Levin, 1981), and provide extra information not included in the written text (e.g., photographs of fish fossils, plant fossils, and insect fossils to accompany the written text, “There are many types of fossils”). Research has identified 6 common functions of graphics in informational text (Duke et al., 2009; Bishop & Hickman, 1992; Carney & Levin, 2002; Clark & Lyons, 2004; Levin, 1981; Levin, Anglin, & Carney, 1987):

  • decoration, which appear as an ornament on the page without adding to or relating to the written text;

  • representation, which depict the information presented in informational text;

  • organization, which supply a framework for classifying information from the written text;

  • interpretation, which explain abstract ideas by depicting them in a more concrete fashion;

  • transformation, which represent mnemonics to help readers remember the written text by making it more concrete and meaningful; and

  • extension, which provide extra details not directly stated in the text.

Younger students particularly need to learn to read and understand these graphics because their informational texts contain many graphics (Duke & Kays, 1998; Fingeret, 2010) and because children often have difficulty reading informational texts (e.g., Park, 2008). Perhaps if children better understood how to gain information from the graphics in informational text, these texts would be less of a struggle (Moss, 2008).

In his dual coding theory, Paivio (1991) suggests the brain codes graphics and verbal text into memory differently. Therefore, students who use comprehension processes as they study the graphics, as well as the written text, may be better able to comprehend and remember what they have read. This research study has been designed to study whether and how the processing of the graphics relates to students’ comprehension of the overall text.

Review of the literature

Comprehension of written text

Research over the past half century has provided us with a well-developed picture of what skilled readers are doing and thinking as they read narrative and informational text. Skilled readers use many and varied reading processes (e.g., prediction, inferring, summarizing, visualizing, comprehension monitoring) strategically and flexibly (e.g., Duke & Pearson, 2002; National Institute of Child Health and Human Development, 2000; Pressley & Afflerbach, 1995). Furthermore, they utilize their prior knowledge of the topic and of the genre and consider their purpose for reading the text to assist them in their comprehension (e.g., Duke, 2005; RAND Reading Study Group, 2002).

Not only has it been found that skilled readers use more strategies than less proficient readers, but utilization of these strategies has been found to be correlated with better comprehension (e.g., Braten & Stromso, 2003; Dermitzaki, Andreou, & Paraskeva, 2008; Samuelstuen & Braten, 2005). For example, Dermitzaki et al. (2008) found that third graders’ use of strategies such as planning and monitoring of learning processes were highly correlated with their performance on a comprehension assessment related to the same text. Moreover, Samuelstuen and Braten (2005) found that 10th graders who were poor decoders but used many strategies while reading two expository texts scored better on a comprehension assessment about the text than good decoders who did not use many strategies. While these findings speak to the importance of strategy use to comprehension of written text, the question remains: does this correlation between strategy use and comprehension carry over to the reading of graphics?

Comprehension of graphics

Although research on comprehension of graphics does exist (and will be discussed next), it is not as robust as the research on comprehension of written text. Furthermore, the findings have been inconsistent, with some researchers finding that the inclusion of graphics has no effect on participants’ comprehension, others finding detrimental effects, and still others finding beneficial effects. Representative studies are discussed below.

Graphics have no effect on comprehension of written text

A number of studies (e.g., Brookshire, Scharff, & Moses, 2002; Miller, 1938; Rose & Robinson, 1984) conducted with children found that the presence of graphics had no effect on the participants’ comprehension. For example, Brookshire et al. (2002) randomly assigned first- and third-grade students to one of three conditions for reading a narrative text designed specifically for the experiment: text-only, text plus illustrations, and illustrations-only (for which the students were told the names of the characters and instructed to make up a story). After reading (in third grade) or hearing (in first grade) the text, students were asked 15 comprehension questions. Five of the questions’ answers could be found in the written text, five could be found in the graphics, and five could be found in both the written text and the graphics. They found that the presence of graphics did not significantly increase students’ comprehension scores on the questions whose answers could be found in the written text only (which was not surprising) or in the text and the graphics, and slightly decreased their comprehension scores of illustration-only questions.

Graphics have negative effects on comprehension of written text

Other studies on graphics and comprehension of connected text, however, have concluded that graphics negatively impact the reading process (Harber, 1983; Rose, 1986; Watkins, Miller, & Brubaker, 2004). For instance, Rose (1986) researched the effects of illustrations on the comprehension of students with learning disabilities. Thirty-two learning disabled elementary students (age 9 years, 3 months–12 years, 8 months) read passages at their reading level, in both illustrated and unillustrated conditions. Within-subject analyses indicated that the students’ comprehension of unillustrated passages was significantly higher than their comprehension of the illustrated passages.

Graphics have beneficial effects on comprehension of written text

Other researchers (e.g., Hannus & Hyona, 1999; Rusted & Coltheart, 1979; Small, Lovett, & Scher, 1993) have concluded from their research that graphics actually aid in students’ comprehension. Yet these researchers do not agree for whom or in what ways.

Graphics benefit All readers

For example, in one study, Small et al. (1993) studied 33 first and 33 third graders of varying reading abilities learning about three unfamiliar animals. Students were randomly assigned to one of three conditions: description-only (listened to passages), picture-only (examined graphics), and description-plus-picture condition (listened to passage and examined graphics). After listening to each passage and/or examining each graphic, the students answered 12 comprehension questions about the animal–four of the answers could be found in the description, four in the graphic, and four in both. Students were encouraged to guess or make up answers if they were unsure. Students in the description-plus-picture condition recalled more information presented only in the text and more information presented in the text and the graphics than those students in the description-only condition. Thus, the results of the study suggested that the presence of graphics facilitated recall of information, even if the information had not been presented in the graphics, for all students in the study.

Graphics benefit Good readers more

In contrast to the findings above, based on their study of fourth graders’ comprehension of biology textbook passages, Hannus and Hyona (1999) concluded that the graphics assisted good readers more than others. The participants in this study, 108 fourth graders from two urban elementary schools in Finland, were exposed to three illustrated passages and three unillustrated passages. After reading, they answered ten or eleven comprehension questions for each passage. The questions assessed the students’ recall of specific details as well as their comprehension of biological principles. The results indicated that graphics led to better recall of detail information for all students, but only better comprehension of biological principles for the stronger readers. They asserted that this difference was due to the fact that students needed to integrate information from the written text and graphics in order to benefit from the graphics, which only high-ability readers appeared to be capable of doing. Furthermore, they hypothesized that better readers recalled and comprehended more from the illustrated texts because they were better able to understand four key ideas: (1) when to examine the graphics while reading; (2) which graphics they should examine; (3) what information they should obtain from the graphics; and, (4) how to combine information in the written text and the graphics into one mental representation.

Graphics benefit Poor readers more

Still other studies have found that, while graphics may help all students, they especially improve the reading comprehension of poor readers and readers from disadvantaged populations. For instance, Rusted and Coltheart (1979) found that graphics improved the comprehension of poor readers more than that of good readers. In this study, thirty-six 9- and 10-year olds (half good and half poor readers) were randomly assigned two sets of six informational passages about unusual animals, with one set of passages containing graphics of the animals and the other not containing any graphics. The passages included information about the physical characteristics of the animals, as well as their living and eating habits. Before reading, students were told to remember what they read and to pay attention to graphics if they were present. The students then read each passage aloud two times. Immediately after the second reading of each passage, students told the researcher everything they remembered about the passage. The results of the study suggested that the presence of graphics improved the general recall of all students, but poor readers actually recalled more illustrated features than good readers, thus improving their comprehension as a whole. Furthermore, the researchers reported that the poor readers appeared to study the graphics more often and use them more intentionally than the good readers.

Conclusion

In summary, the research conducted with children thus far has investigated whether graphics improve children’s comprehension of written text, but the results have been inconsistent, with some researchers finding them to have neutral effects, others finding them to have negative effects, and still others finding that they have beneficial effects. Even those researchers who agree that they are beneficial to students’ comprehension do not agree for whom or for what reasons.

Outcome versus concurrent measures

Most of the previous research on graphics and comprehension explored whether the mere presence of graphics had effects on participants’ overall comprehension of the texts presented to them. Few studies used think alouds (e.g., Norman, 2010a; Schnotz, Picard, & Hron, 1993) to examine what they were doing with those graphics, and what relationship that had to participants’ comprehension of the text. In fact, only one study (i.e., Norman, 2010a) has research the specific reading processes that were prompted by the graphics and only one study (i.e., Schnotz et al., 1993) has attempted to investigate how participants used the graphics to assist in their comprehension of informational text through the use of verbal protocols.

Reading processes prompted by the graphics in informational text

In Norman’s (2010a) study, 9 second graders read two informational texts and were prompted to think aloud whenever they studied a graphic. These think-alouds revealed that 17 reading processes (i.e., label; literal description; inferential description; prediction; infer the author’s purpose; confirm/disconfirm text; connection-to-self; irrelevant connection; connection-to-prior knowledge; wonder; knowledge monitoring; affective response; compare-contrast graphics; evaluate; use of running text; use of captions, labels, map key, etc.; and word identification) were prompted by the graphics in these two texts. This study did not utilize any outcome measures, however, so it was not possible to determine whether or not a relationship existed between the reading processes prompted by the graphics and students’ comprehension of those same texts.

Use of graphics in informational text

Schnotz et al. (1993) studied 26 college students as they read a passage, accompanied by a map of the time zones, about time and date changes as one passes through the different time zones in order to determine whether there were differences in how successful and unsuccessful learners used the graphics (i.e., the map). After reading, the students first described what information they could extract from the map; then thought aloud as they answered 11 questions using a map; and finally took a 25 question test for which they had to apply the information in the written text and the map to figure out times in different areas of the world. Schnotz et al. compared how often successful and unsuccessful learners—as determined by the 25 question test about the material—referred to the map during the think-alouds, finding that successful learners referred to the map significantly more often than their less successful counterparts (i.e., an average of 21.3 and 12.5 times respectively). Moreover, successful learners interpreted more sections of the map during their think-alouds and used the map to determine more spatial and temporal differences as indicated by their think-alouds. Schnotz et al. hypothesized that the successful learners were using the graphics to develop mental models, and used the written text to add to these mental models.

In sum, Norman’s (2010a) study provides us with an understanding of what processes are prompted by the graphics as children reading informational texts, but no outcome measure was used so it is unknown how well students comprehended the texts or whether the use of reading processes as they studied the graphics improved or hindered the students’ comprehension of the overall text. Schnotz et al.’s (1993) study indicated that successful learners—those students who comprehended the text more—used the graphic more often, but did not investigate the specific reading processes that were prompted by the graphic. Therefore more research is needed in order to investigate specifically the relationship between the reading processes prompted by the graphics and students’ comprehension of the same texts.

Methods

Research design

I conducted a study using verbal protocols (Afflerbach, 2000; Pressley & Afflerbach, 1995; Pressley & Hilden, 2004). As the students read, they were prompted to think aloud whenever they studied a graphic; students were also encouraged to share their thinking at any other time during the reading. After each reading, the students retold the book and answered eight researcher-designed comprehension questions. I then analyzed the verbal protocol transcripts using modified open coding (Strauss & Corbin, 1998) and scored the retellings and researcher-designed comprehension questions. Finally, correlations were run between students’ scores on the retelling and research-designed comprehension questions and the number of times any process was prompted, the number of different processes, and the number of instances of individual processes to answer the question: What is the relationship, if any, between children’s processes prompted by the graphics in informational text and their comprehension of the same texts?

Participants

The study was conducted with 30 second-grade students (17 males and 13 females) from eight classrooms in five schools in five school districts in two Northeastern states. Second graders were chosen for two reasons: (1) previous research has indicated that second graders are able to produce reliable verbal protocols that provide insight into their thinking as they read narrative (i.e., Alvermann, 1984; Hilden, 2006) and informational texts (i.e., Norman, 2010a; Hilden, 2008); and (2) as indicated above, informational texts written for primary-aged children, which includes second graders, contain a number of graphics (Duke & Kays, 1998; Fingeret, 2010).

In order to provide a diverse sample of students based on ethnicity and socioeconomic status, the school districts were selected purposively. Within each school district, one school agreed to participate; within each of these schools, two classrooms were randomly selected from all consenting classrooms. Two of the selected classrooms (one in each of two schools) had to be eliminated from the study because they lacked a range of readers as indicated by the Gates MacGinitie Reading Test (GMRT) (MacGinitie, MacGinitie, Maria, & Dreyer, 2000) comprehension subtest. In these two schools, all six students were randomly selected from one classroom. Including only two classrooms from each school and three to six students from each classroom helped to decrease the likelihood that the results of the study would be influenced by any one school or teacher placing a greater emphasis on the graphics in text.

The decoding and comprehension sections of the GMRT, Form T, Level 2 were administered to all consenting students from each classroom, and students were identified as below-average readers (i.e., they scored between the 1st and 30th percentile), average readers (i.e., they scored between the 40th and 60th percentile), and above-average readers (i.e., they scored between the 70th and 100th percentile). Those students who did not score between these percentile ranges were eliminated from the study. Additionally, in order to better control for any language or learning disability factors, students who received special education services or were English Language Learners were eliminate from the possible participant pool prior to selection (i.e., below-average, average, and above-average). Once all students had been identified or eliminated, randomized cluster sampling was used to select one reader from each class in each achievement group so that one was a below-average reader, one was an average reader, and one was an above average reader.

The participants were diverse in terms of ethnicity and socio-economic status, which was measured using maternal education level (Entwisle & Astone, 1994). For the sample, per parent report, 6% of the children were Asian–American, 10% African–American, 50% Caucasian, 20% Hispanic/Latino, and 10% other; and the parents of two students did not respond. In terms of socio-economic status, 6% of mothers reported completing eighth grade but not high school, 23% reported completing high school, 6% held associates degrees, 30% reported holding a bachelors or other four-year degree, 20% reported holding a masters degree, and 3% reported holding a doctorate; four declined to respond.

Materials

Students read Dino Dig (Odgers, 2008) and Weather Watching (Ryan, 2008), two informational texts that are part of Weldon Owen Publishings’ stage two Top Readers series, a series of books that are designed with simple sentences, specialized vocabulary, and graphics that provide support (Odgers, 2008, back cover; Ryan, 2008, back cover) and contain a range of graphics. Dino Dig explains the process by which dinosaurs become fossils and how scientists discover and use these fossils to learn about dinosaurs. Weather Watching describes different types of weather and how scientists study weather. Both of the books are written for second graders to read independently in the second half of the school year according to Chall’s (1996) Qualitative Assessment of Text Difficulty.

The graphics in both books represent many of the prototypical graphics found in informational texts, such as photographs (six in Dino Dig and six in Weather Watching) and realistic drawings (11 in Dino Dig and 10 in Weather Watching) with captions, labels (five labeled graphics in Dino Dig and two in Weather Watching), and keys (four graphics with keys in Weather Watching); maps (one in Weather Watching); flow charts (one in Dino Dig and two in Weather Watching); diagrams (one in Dino Dig and one in Weather Watching); cross-sectional graphics (three in Dino Dig); and cross-sectional diagrams (one in Dino Dig) (Fingeret, 2010; Purcell-Gates, Duke, & Martineau, 2007). Also, these graphics represent many of the communicative properties of graphics (i.e., representation [10 in Dino Dig and seven in Weather Watching], extension [nine in Dino Dig and nine in Weather Watching], organization [one in Dino Dig and one in Weather Watching], and decoration [one in Dino Dig and four in Weather Watching]) identified by Levin et al. (Carney & Levin, 2002; Levin, 1981; Levin et al., 1987), Clark and Lyons (Clark & Lyons, 2004), Bishop and Hickman (1992), and Norman et al. (2010). Please note that some graphics were counted more than once (e.g., a map is also a graphic with a key).

Because the length of the books was a concern, three sections (five pages) were removed from each book. All sections removed met the following criteria: (1) the exclusion of those pages would not diminish the range of graphics in the text, and (2) the removal of those sections would not impact the comprehension of later sections. Within sections that met these criteria, I gave priority to removing sections that (1) contained vocabulary or concepts deemed difficult or confusing for second-grade students and (2) that were not authentic to informational text (i.e., comprehension questions at the end of the reading).

Finally, in order to more easily track where the students were looking as they read (i.e., the running text, the captions, or the graphics), the books were modified (as in Norman, 2010a). The original books, which measured 6 inches by 9 inches (15.24 cm by 22.86 cm), were cut apart and glued onto construction paper, which measured 12 inches by 18 inches (30.48 cm by 45.72 cm), so that the distance between the main text and the graphics was increased. Only blank space was added. The labels and titles of graphics were not cut apart from the image, and headings and the accompanying running text remained intact; the relative position of graphics, written text, and captions on the page was kept the same. These modifications were piloted prior to this study and a previous study (Norman, 2010a) to ensure the increased space did not distract students from looking at the graphics.

Data collection and analysis

Sessions one and two

During the first two sessions, which occurred within the classroom, I introduced myself to the students and explained the study to them. Then, all students with parental consent completed the decoding and comprehension subtests of the GMRT, Form T, Level 2 (MacGinitie et al., 2000).

Sessions three and four

During the third and fourth sessions, which were held on a one-to-one basis in a quiet place outside of the classroom, the students read aloud either Dino Dig (Odgers, 2008) or Weather Watching (Ryan, 2008). As the students read, they were not corrected and were provided with little assistance in the decoding of words. When students asked for help, they were encouraged to try their best. In a few instances, students were supplied with words because they refused to continue reading. As students read, they were asked to verbalize what they were thinking (procedures for this are described below). The order of presentation of the two books was counterbalanced within ability group and classroom.

Verbal protocols

Verbal protocols were used to determine the readings processes prompted by the graphics. As students read, they were prompted to think aloud about the text when they looked at a graphic. As is recommended in methodological pieces on verbal protocols (i.e., Pressley & Afflerbach, 1995; Pressley & Hilden, 2004) the directions to students for sharing their thinking were general:

Today, you are going to be reading a book for me. The book is going to look a little funny (show students book), so don’t worry about that. As you read, I cannot help you with any of the words. If you come to a word you don’t know, try your best and keep reading. When you are done reading, I am going to ask you to retell the book to me. I am also going to ask you some questions about the book. As you read, I want to know what you are thinking. Sometimes, I am going to stop you to ask you to tell me what you are thinking. If you have nothing to say, you can tell me that too. You can also talk about the book at other times when I don’t ask if you want to. Is it okay to tell me you have nothing to say? Is it okay to talk about the book whenever you want?

As students read, I watched continuously in order to determine where their eyes were looking, the text or the graphics. When I noticed that they were looking at a graphic, I prompted them to share their thinking with, “What are you thinking?” If students had not looked at any graphics after four pages, they were also prompted with “What are you thinking?”

In order to identify the processes prompted by the graphics, students’ verbal protocols were transcribed verbatim. The transcription of their readings and that of their thinking was done in different colors in order to make it easier to distinguish the two kinds of verbalization.

First, verbalizations were analyzed to determine to what the student was referring: the written text without reference to the graphic (e.g., “D-I-G [underlines word as spells it out]”); the graphic (e.g., “Oh, I see a bones [sic] [student points to bones in picture]”); the graphic and the written text (e.g., “So in here they would have to put a replica [points to spot on the skeleton where a piece is missing]” after reading that scientists make replicas to fill in missing piece of a fossil); or an unrelated comment (e.g., “I need to go to the bathroom”). Only those comments that pertained to the graphics, either because they directly related to the graphic or because the student was looking at the graphic as they spoke, were coded further. Second, modified grounded theory (Strauss & Corbin, 1998) was used to reveal the reading processes prompted by the graphics in each of the texts. Because previous research (i.e., Norman, 2010a) using grounded theory had already identified 18 codes, I began by analyzing for these processes (i.e., label; literal description; inferential description; prediction; infer the author’s purpose; confirm/disconfirm text; connection-to-self; irrelevant connection; connection-to-prior knowledge; wonder; knowledge monitoring; affective response; compare-contrast graphics; evaluate; use of running text; use of captions, labels, map key, etc.; word identification) and no process. Any verbalizations that did not fit one of the 18 previous codes were then described with a short phrase that explained the process prompted by the graphic. These phrases were compared continuously to ensure that none of them could be collapsed. In this way, six new codes were developed (i.e., names, graphic-to-written text connection, intertextual [across texts] connection, creates narrative, repeat-paraphrase written text, and other). Please see results section below for further discussion of these codes.

In some instances, the graphic prompted more than one reading process; therefore, the verbalizations could be coded using multiple processes. For example, while reading Dino Dig (pp. 22–23) and studying a cross-sectional diagram of a dinosaur, one student commented, “That looks gross (points to dinosaur)! I think maybe they’re going to tell us, like (points to muscle), the layers of a dinosaur, maybe.” The first sentence in this verbalization was coded as an affective response because the student was reacting to the graphic; the second sentence was coded a prediction because the student explained what he thought the page would discuss.

A literacy expert familiar with reading processes of written text was trained to code the verbal protocols using six of the transcripts. She then coded a random sample of the transcripts (n = 16 or more than 25%), stratified by book. Inter-rater agreement was 86.7% or 320 out of 369 codes, which is comparable to inter-rater agreement in other verbal protocol studies (e.g., Brown, Pressley, Van Meter, & Schuder, 1996).

Retellings

When students had finished reading the book, they were asked to retell the text. For this study, directions for retelling were adapted from previous research on younger children’s retellings of expository text (e.g., Romero, Paris, & Brem, 2005), but were modified to utilize the fact that the session was videotaped.

Another second grader hasn’t read this book and wants to know about it. She (He) will watch the video to hear the retelling. Can you retell the book using your words and the words in the book as you remember them? Try to include as many details as you can.

Retellings have been used in previous research on the influence of graphics on students’ comprehension of the written text (e.g., Gambrell & Jawitz, 1993). They have also been successfully used to assess young children’s comprehension of narrative (e.g., Baumann & Bergeron, 1993; Morrow, 1985; Roberts, 2010) and expository text (e.g., Moss, 1997; Romero et al., 2005).

Retellings were scored based on protocols developed for each book. The protocol scoring procedure is based on that used by Meyer, Brandt, and Bluth (1978) in their study of the use of top-level text structures and by Taylor (1980) in her study of children’s memory for expository text. To develop retelling protocols, an expert in designing retelling protocols and I analyzed each book independently, identifying superordinate and subordinate ideas from the running text and captions and developing a checklist to be used in scoring the retellings. Next, we compared checklists and found three discrepancies—two for Dino Dig and one for Weather Watching. These disagreements were resolved through discussion and consultation with another literacy expert.

Finally, we assigned points to each idea. Superordinate ideas received two points per page—on some pages the superordinate ideas were two separate, but relate ideas and, therefore, students received one point for each—, and subordinate ideas received one point. Students could also receive half a point for retelling the topic of a superordinate idea (e.g., it talked about rainbows) or for retelling part of a subordinate idea (e.g., the sun makes rain). At times, a superordinate idea for one page was discussed as a subordinate idea on another page. In these instances, students received two points (see “Appendix 1” for retelling protocol for Dino Dig). Finally, in order to equate the two retellings, the students’ scores on the retelling were divided by the total possible points (i.e., 45 for Weather Watching and 49 for Dino Dig) and multiplied by 100 to determine their scores on the retelling.

A literacy expert familiar with scoring retellings was trained to score the retellings using 2 transcripts and then scored 10 transcripts for Weather Watching and 10 transcripts for Dino Dig. First, I looked at whether or not a student mentioned a main idea or supporting detail; interrater agreement was 96.2%, or 885 ideas out of 920. Then, I compared our final retelling scores for each protocol to see if they were within 1 point; 80%, or 16 out of 20, were within 1 point.

Book-specific comprehension questions

Finally, students were asked 8 book-specific comprehension questions about the text. All questions were open-ended and students were asked to respond verbally. Students’ answers were recorded and later transcribed.

Researcher-designed comprehension questions have also been used as comprehension assessments in the study of graphics (e.g., Harber, 1983), as well as in assessing children’s comprehension of narrative (e.g., Paris & Paris, 2003) and expository text (e.g., Purcell-Gates et al., 2007). To construct the questions, I developed a concept map of the written text to illustrate the macro- and microstructures (Kintsch & van Dijk, 1978). I then wrote open-ended questions that tapped both concepts at the macro- and micro-level of the text. These questions were designed to assess the three levels of comprehension recommended by the 2009 National Reading Framework (National Assessment Governing Board, 2008): locate/recall (for which students must identify information explicitly stated in the text), integrate/interpret (for which students must make inferences within and across texts), and critique/evaluate (for which students must assess the quality of the text, decide what is most important in the text, or judge the plausibility of an argument). Eight experts in the field of literacy reviewed and suggested revisions for the questions. After questions were refined, the experts identified whether each question assessed a macro- or micro-level idea, and whether it assessed literal, inferential, or critical comprehension skills to ensure this distribution was met. Based on their expert review, it was met (i.e., 2 literal in Dino Dig and 3 in Weather Watching, 4 inferential in Dino Dig and 5 in Weather Watching, and 1 critical in Dino Dig and 1 in Weather Watching). See “Appendix 2” for questions from Weather Watching.

On the comprehension questions, students could receive up to two points for each correct answer. To develop a scoring protocol, an expert in literacy and I answered each question with what we thought would be considered two-point and one-point answers for second-grade students. These answers were compared to each other and to a random sample of six transcripts. From these transcripts, sample 0-point, 1-point, and 2-point answers were selected to include in the scoring protocol. Once all answers were scored, students’ comprehension question scores for each book were divided by the number of points scored by the total number of points possible (i.e., 16, though for two students, one question was missed so their possible points were out of 14 instead of 16) and multiplied by 100.

A literacy expert familiar with scoring comprehension questions was trained to score the comprehension questions using 2 transcripts. She then scored 10 transcripts for Weather Watching and 10 transcripts for Dino Dig. Interrater agreement was 87.5%, or 140 out of 160 questions.

Videotaping

The 2 one-on-one reading sessions were videotaped in order to record where students were looking and pointing (e.g., running text, caption, label, graphic) as they discussed their thinking. The transcriptions included references to where the students were looking or pointing as they read and verbalized.

Statistical analysis

After all verbalizations pertaining to the graphics, retellings, and comprehension questions were analyzed, a number of statistical analyses were run. First, the number of times each process was prompted for each student was calculated separately for each book. Second, the number of different processes prompted for each student was calculated separately for each book. Third, Spearman’s ρ correlations were run to determine whether and if so, to what degree, there was a statistically significant relationship between (a) retelling scores and (b) comprehension question scores and (1) the range of processes used by children, (2) the number of times any process was used by children, and (3) the number of times each individual process was used by children.

Results

The reading processes

The modified open-coding of the transcripts resulted in 25 codes—23 reading processes (i.e., affective response; confirm-disconfirm text; intertextual connection; graphic-to-graphic connection; graphic-to-written text connection; irrelevant connection; connection-to-prior knowledge; connection-to-self; create narrative; evaluate; infer the author’s purpose; inferential description; knowledge monitoring; label; literal description; name; prediction; repeat-paraphrase written text; reading process-other; use of graphical devices; use of running text; wonder; and word identification), no process, and uninterpretable. Table 1 provides complete descriptions of and student comments that exemplify each process.

Table 1 Definitions and examples of all processes prompted by the graphics
Full size table

The total number of processes prompted by the graphics across the two books ranged from 9 to 62 (M = 28, SD = 12.625). The number of different processes prompted by the graphics across both books ranged from 1 to 16 (M = 9.93, SD = 4.226). Table 2 provides descriptive statistics for the total number of times each process was prompted for each book, and Table 3 provides descriptive statistics for the number of times any process was prompted and the number of different processes prompted by the graphics for each book. For further discussion of these processes see Norman (2010b).

Table 2 Number of times each process was prompted
Full size table
Table 3 Descriptive statistics for number of different processes and number of total processes prompted
Full size table

Correlations between the reading processes and retelling scores

Out of a score of 100, students’ scores on the retellings ranged from 1.02 to 22.45 (M = 9.03, SD = 5.65) for Dino Dig and from 1.11 to 26.67 (M = 8.00, SD = 5.72) for Weather Watching. The correlation between retelling and number of times any process was prompted by the graphics approached significance for Dino Dig (ρ = 0.333, p = 0.072). This correlation indicates that, for Dino Dig, a greater use of any process prompted by the graphics was associated with higher retelling scores. In fact, about 11.09% of the variation in students’ retelling scores for Dino Dig can be explained by the number of times any process was prompted. When studying the three reading-achievement groups separately, only the correlation between retelling scores and above-average students’ use of any process approached significance (ρ = 0.607, p = 0.063), indicating that for above-average students 36.84% of the variance in their retelling scores can be explained by the number of times any process was prompted.

There was no correlation between number of times any process was prompted and retelling scores for Weather Watching, though the correlation for average readers did approach significance (ρ = −0.597, p = 0.068). This negative correlation indicates that, for average readers, the use of fewer processes is associated with higher scores on the retelling for Weather Watching. The effect size was strong (ρ2 = 0.356). Neither for Weather Watching nor for Dino Dig was the correlation between the number of different processes and retelling scores statistically significant. When reading Dino Dig, however, above-average students use of different processes was positively, significantly correlated with their retelling scores (ρ = 0.632, p = 0.05); in fact, 39.94% of the variation in above-average readers’ retelling scores for Dino Dig can be explained by the number of different process that were prompted. See Table 4 for means, standard deviations, and ranges of scores on the retellings for the two books, correlations between retelling scores and total number of processes, and correlations between retellings and number of different processes.

Table 4 Descriptive statistics for the comprehension measures and correlations between comprehension measures and number of different processes and number of total processes prompted
Full size table

Several individual reading processes were also associated with higher retelling scores for each book, while others were associated with lower retelling scores. For Dino Dig, use of graphical devices (ρ = 0.363, p = 0.049) was positively, significantly correlated with retelling scores, while connection-to-prior knowledge (ρ = 0.31, p = 0.096), label (ρ = 0.329, p = 0.076), and evaluate (ρ = −0.356, p = 0.053) approached significance. The effect sizes for these processes were all moderate, ranging from 0.096 (connection-to-prior knowledge) to 0.132 (use of graphical devices).

When studying Dino Dig retelling scores for the three achievement groups separately, different processes are correlated with higher or lower retelling scores for the different groups. For below-average students, knowledge monitoring has a negative, statistically significant correlation (ρ = −0.698, p = 0.025) and repeat-paraphrasing written text (ρ = 0.554, p = 0.096) approached significance. Literal descriptions (ρ = −0.677, p = 0.032) and names (ρ = −0.700, p = 0.024) were statistically significantly correlated with lower retelling scores for average students. Finally, for above-average students, label (ρ = 0.717, p = 0.019) was statistically significantly correlated with higher retelling scores, and both create narrative (ρ = 0.610, p = 0.061) and knowledge monitoring (ρ = 0.591, p = 0.072) approached significance. The effect sizes for these processes were all strong, ranging from 0.307 (repeat-paraphrasing written text for below-average reader) to 0.514 (label for above-average readers). See Table 5 for means, standard deviations, and correlations between retelling scores and individual processes.

Table 5 Descriptive statistics for individual processes and correlations between comprehension measures and individual processes for Dino Dig
Full size table

For Weather Watching, students’ scores on the retelling were statistically significantly correlated with connection-to-prior knowledge (ρ = −0.432, p = 0.017) and word identification (ρ = −0.373, p = 0.042); use of running text (ρ = 0.315, p = 0.091) approached significance. Interestingly, the former two correlations were negative, indicating that more connections-to-prior knowledge and word identifications were associated with lower retelling scores for Weather Watching. All of these effect sizes were moderate (ρ2 = 0.187 for connection-to-prior knowledge, ρ2 = 0.139 for word identification, and ρ2 = 0.099 for use of running text). Please see Table 6.

Table 6 Descriptive statistics for individual processes and correlations between comprehension measures and individual processes for Weather Watching
Full size table

When investigating correlations for the three achievement groups, connection-to-prior knowledge (ρ = −0.640, p = 0.046) was statistically significantly correlated with retelling scores for the above-average readers. Interestingly, students’ connections-to-prior knowledge were correlated with lower retelling scores. The effect sizes for both of this processes was strong (ρ2 = 0.410). There were no significant correlations found for either of the other two sub-groups.

Correlations between the reading processes and researcher-designed comprehension question scores

Students’ scores on the researcher-designed comprehension questions ranged from 15.63 to 81.25 (M = 47.43, SD = 16.83) out of a score of 100 for Dino Dig and from 3.13 to 62.5 (M = 41.38, SD = 18.09) out of a score of 100 for Weather Watching. No correlations between these scores and number of times any process was prompted or number of different processes used were statistically significant when considering all students together. For the three sub-groups, the number of times any process was prompted (ρ = −0.696, p = 0.025) was significantly correlated with lower scores on the comprehension questions for the average readers when reading Weather Watching, and the number of different processes prompted approached significance (ρ = 0.601, p = 0.066) for below-average readers reading Weather Watching. For both of these cases, the effect sizes were strong (ρ2 = 0.484 and 0.361 respectively). Again, see Table 4 for means, standard deviations, and ranges of scores on the comprehension questions for the two books, correlations between comprehension scores and the number of times any process was prompted, and correlations between comprehension scores and the number of different processes.

Correlations run between the number of times individual reading processes were prompted by the graphics and students’ scores on the comprehension questions indicated that only graphic-to-graphic connection was significantly correlated with comprehension scores (ρ = 0.484, p = 0.007) for Dino Dig. The prompting of labeling also approached significance (ρ = 0.353, p = 0.056) for Dino Dig. The effect sizes of these processes on their comprehension question score were moderate (ρ2 = 0.125 for labeling and ρ2 = 0.232 for graphic-to-graphic connection). There were no significant correlations between any specific reading processes and the comprehension questions for Weather Watching, though use of running text approached significance (ρ = 0.354, p = 0.055). See Tables 5 and 6 for means, standard deviations, and correlations.

When considering the three sub-groups individually, different processes were correlated with comprehension question scores for different groups and for different books. When reading Dino Dig, for below-average students, irrelevant connections (ρ = 0.665, p = 0.036) is positively, statistically significantly correlated with comprehension question scores for Dino Dig and prediction (ρ = 0.601, p = 0.066) approached significance; wondering (ρ = 0.574, p = 0.083) approached significance for students who were average readers; and connection-to-self (ρ = −0.690, p = 0.027) was negatively, statistically significantly correlated for above-average students. For Weather Watching, correlations between label (ρ = 0.589, p = 0.072) and comprehension questions scores approached significance for the above-average readers and between repeat-paraphrase written-text and comprehension question scores approached significance for below-average readers (ρ = 0.554, p = 0.096). All effect sizes were strong.

Discussion

This study investigated what relationship, if any, existed between the processes prompted by the graphics as second graders read informational text and their scores on two comprehension measures for that text: retelling and researcher-designed comprehension questions. A discussion of these findings follows.

Sheer number of processes may not always help

As discussed above, research on reading processes and the comprehension of written text has shown that students who employ a greater number of total processes score higher on comprehension assessments (e.g., Braten & Stromso, 2003; Dermitzaki et al., 2008; Samuelstuen & Braten, 2005). In this study, the correlation between the number of times students used any process while studying the graphics in Dino Dig—but not Weather Watching—and the retelling measure approached significance. When considering the three reading-achievement groups separately, this correlation approached significance for above-average readers as they read Dino Dig, but not Weather Watching. Furthermore, for average achieving students reading Weather Watching, the greater the number of times any process was prompted, the lower their scores on the comprehension questions. In this case, using more comprehension processes did not help average readers.

Previous research has also found that good readers use a greater range of processes with written text (e.g., Dermitzaki et al., 2008). With respect to the visual elements of text that were the focus of this study, there were no significant correlations between range of different processes used and scores on either of the comprehension measures for either of the books. Above-average students reading Dino Dig did score higher on the retelling, but not on the comprehension questions, when they used a great number of different processes; below-average students reading Weather Watching, this correlation approached significance.

Why do these differences exist between the relationship of how students process written text and graphical text and subsequent comprehension? I propose three possible reasons: (1) the designs of the studies differed, (2) it is possible that not all processes are created equal—that is to say that the use of some are associated with improved comprehension, while the use of others are not, and (3) students are being taught to strategically use processes when reading written text, but not when reading graphics (if they are taught to read graphics, at all).

The studies’ designs

The design of this study differed from the designs of the studies investigating reading processes prompted by written text and comprehension. In this study, students’ reading processes were revealed through verbal protocols and modified open-coding. Students were not asked to identify specific processes, but instead shared their thinking about the graphics in general. In Samuelstuen and Braten (2005), students were asked to indicate how often they had used specific processes by selecting (1) not at all to (10) very often on a Likert-type scale. Perhaps some students over- or under-represented their use of these processes.

Additionally, previous research on the relationship between reading processes and comprehension has not included as many different reading processes as were included in this study. For example, Samuelstuen and Braten (2005) investigated the use of only three types of reading processes—elaboration, organization, and monitoring. They did not include many of the reading processes included in this study, such as word identification, predicting, or irrelevant connections. However, this does not necessarily mean that fewer processes were present, but could mean that the level of analysis was less refined than in the present study, with several micro-processes mapping onto one macro-process. Limiting the number and type of reading processes investigated in this study or consolidating processes may have led to different results. Please see Norman (2010c) for further discussion of these macro-processes.

Finally, in research, such as that by Samuelstuen and Braten’s (2005) and Dermitzaki et al. (2008), participants have only read one selection. In this study, students read two informational science trade books. Although the books were from the same publisher and the same series, and they were chosen because of their similarities in graphics and writing, they do discuss different topics and areas of science (i.e., life sciences vs. physical sciences). These differences may have contributed to the different findings. After all, McTigue (2009) did find that when sixth graders read a life science text about the life cycle of protozoan, the inclusion of diagrams improved their comprehension, but when they read physical science text about how heat energy is transformed to mechanical energy, the addition of diagrams did not improve their comprehension. For further discussion of how specific books may influence what graphical reading processes are prompted, please see Norman (2010a, b).

Not all processes are created equal

When looking at the correlations between reading processes and comprehension scores, it appears that not all reading processes contribute to a readers’ overall comprehension equally. For example, students who were prompted to respond affectively did not necessarily score better or worse on the comprehension measures—these emotional responses did not hurt the students’ comprehension, but it did not help them recall the information either.

Students’ use of graphical devices (positively statistically significantly correlated with retellings for Dino Dig) and use of the running text (positively correlated with retellings for Weather Watching), on the other hand, appear be associated with better retelling scores (for Dino Dig and Weather Watching respectively). Perhaps this is partially due to the fact that the graphical devices and the running text helped them to comprehend the graphic better. For example, Weather Watching discusses the fact that, long ago, scientists used kites to study the weather. When first looking at the accompanying graphic, a number of students thought it was a picture of the Wright brothers and their airplane. Only after reading the running text did some of them comment, “I just learned that that’s a kite.” Understanding that the graphic showed scientists flying a kite may have cleared up misconceptions and helped students to remember how scientists studied weather long ago so that they could correctly include this in their retellings. Qualitative analysis of students’ answers to the question “How did people study weather long ago?” indicates that students who used the running text to revise their understanding of this graphic were more likely to respond correctly than those students who did not use the running text to revise their thinking. These latter students were more likely to reply that they used airplanes.

Greater use of labeling a graphic was also associated with higher retelling scores, but not with higher comprehension question scores. This could be due in part because to the fact that when children labeled the pictures they turned visual stimuli into verbal stimuli, thus dual coding (Paivio, 1991) the information and making them more memorable to the children, and thus made them so they were more likely to mention them in their retellings. Just restating items did not necessarily assist them in answering comprehension questions, however, because many of the comprehension questions asked them to go beyond simply recalling specific items included in the text. One exception was a question about Dino Dig, which asked students to “Tell me as many things as you can from this book that were turned into fossils.” Qualitative analysis of this question suggests that students who labeled the photographs and illustrations of the different types of fossils were able to include more types of fossils.

Some processes are associated with lower comprehension scores. For example, students’ use of word identification while reading Weather Watching is negatively correlated with their retelling scores for that book. Perhaps students are concentrating on decoding the words, thus interfering with their recall of text. Others depended on the reading-achievement of the students. For example, knowledge monitoring was associated with lower retelling scores for Dino Dig for below-average readers, but with higher retelling scores for above-average readers. Perhaps below-average readers were able to identify whether or not they already knew information presented in the book, but did not integrate this new information with their already existing understanding, while above-average readers did integrate this information, allowing them to better understand and recall what they read.

Reading instruction

Currently, many students (likely including those in this study) have been taught to predict, summarize, make inferences, make connections, and use other processes strategically as they read written text. In fact, this strategy instruction is emphasized in research-based programs, such as those based on transactional strategy instruction (Brown et al., 1996) and reciprocal teaching (Palincsar & Brown, 1994), as well as popular books written for teachers, such as Strategies that Work (Harvey & Goudvis, 2007) and Mosaic of Thought (Keene & Zimmermann, 2007). Importantly, this instruction focuses on the quality of the use of these processes. That is to say, teachers spend time teaching the difference between a helpful and an unhelpful prediction or connection. By comparison, less emphasis appears to be placed on the reading and processing of graphics as no research exists as to whether or not teachers are teaching students to read and process graphics or to suggest effective ways in which to teach students how to read and process these graphics. Therefore, students are not learning how to effectively use these reading processes when studying graphics and it would appear that quality, strategic use of processes while reading written-text may not transfer to quality, strategic use of processes while reading the graphics. Given the increasingly visual nature of the world in which we are living and the fact that according to the semiotic perspective (e.g., Jewitt & Oyama, 2001) these visuals carry meaningful information, visual literacy is one “new” literacy that we cannot afford to ignore. The implications of lack of instruction in processing—and subsequently lack of ability to process and glean information from—visuals could negatively affect students’ abilities to comprehend the texts they encounter in and out of school.

Despite this lack of instruction, just as some readers are naturally strategic at processing written-text while others are not (e.g., Duke & Pearson, 2002; National Institute of Child Health and Human Development, 2000; Pressley & Afflerbach, 1995), some students appeared to be able to understand what the graphics were attempting to convey, while others misinterpreted some graphics. For example, when studying the map of different climates in Weather Watching, many students understood the map key explained the symbols—or in this case colors—on the map and were able to integrate the information in the map key with the information in map; others appeared to believe they were two separate graphics that did not relate. When the caption asked students if they could find the deserts, students who could read these symbols and integrate the information, found the deserts on the map based on the color indicated in the map key. Those students who did not integrate these two graphics pointed to the picture of the desert in the map key. With more—or perhaps any—instruction on how to interpret and process graphics, perhaps all students would be better able to improve their comprehension of the overall text.

Reading achievement as moderating factor

Although not explicitly explored in this study, it would appear that a students’ reading achievement may be one moderating factor in determining which processes are positively and negatively correlated with comprehension scores. For example, as indicated above, knowledge monitoring and retelling scores for Dino Dig were negatively, statistically significantly correlated for below-average readers, while the correlation approached positive significance for above-average readers. These differences are not surprising in light of the extant verbal protocol research on graphical reading (i.e., Norman, 2010a, b), which suggests that students’ reading achievement and the processes prompted by the graphics are related.

The curious case of prior knowledge

In looking at specific processes, previous research has found that prior knowledge of a subject assists in students’ comprehension of the topic (e.g., McNamara & Kintsch, 1996). In this study, the graphics prompted 20 students to make connections-to-prior knowledge for a total of 56 times (33 for Weather Watching and 23 for Dino Dig). Interestingly, these connections-to-prior knowledge were positively correlated to retelling scores for Dino Dig, but negatively correlated to retelling scores for Weather Watching, though the former only approached statistical significance. In some cases, it seems that children’s connections relating prior knowledge to Dino Dig were of greater quality or were more accurate, thus assisting them in their comprehension of the text. For example, one above-average reader was prompted to make connections-to-prior knowledge while reading Dino Dig and Weather Watching. For Dino Dig, one of his connections-to-prior knowledge was about mummies while studying a photograph of Leonardo, a dinosaur fossil that was a mummy. In his retelling, he discussed Leonardo and received three out of the four possible points for that page, which was almost half of his Dino Dig retelling points. While reading Weather Watching, his connection-to-prior knowledge was incorrect—while studying a photograph of lightning, he stated that lightning could come from the sky or the ground. In his retelling, he did not mention lightning at all. Accurate and relevant prior knowledge likely assists students in their comprehension more than inaccurate or less relevant prior knowledge.

A note about correlations and causations

This study investigated correlations between graphical reading processes and comprehension scores on retelling and comprehension questions, without addressing causation. Based on the results, it appears that some graphical reading processes are correlated with some comprehension measures for some books for some students. Perhaps the use of these specific processes as they studied the graphics in the books led to the students’ better comprehension scores. For example, students who were prompted to use the running text to better understand a graphic may have comprehended the overall text better because they coded the information from both the written- and graphical-text. However, we also have to consider the fact that students’ comprehension of the overall text actually led them to use, or not use, specific graphical reading processes. For instance, students who did not comprehend the overall text may have been prompted to make irrelevant connections because of the fact that they did not understand the author’s meaning.

Limitations

Precautions were taken to maximize the validity of this study, but a few limitations remain. First, because verbal protocols rely on participants self-reporting of their thinking and the comprehension measures were dependent upon students verbalizing their answers, students’ verbal abilities may have impacted their reporting of their thinking (Afflerbach & Johnston, 1984) and their ability to retell or answer the comprehension questions. Second, the fact that students were prompted to share their thinking whenever they looked at a graphic may have affected how often students studied the graphics. Some students may have studied them more often because of this prompting, while others may have looked at them less often in order to avoid prompting. In fact, although never specifically told that they would be asked to share their thinking when they looked at a graphic, some students appeared to figure out the pattern and began to report their thinking as they looked at each graphic.

Implications and conclusions

Previous research on comprehension and graphics has been inconsistent with regard to whether or not the presence of graphics is beneficial to students’ overall comprehension of the text. None of these previous studies investigated how students were processing these graphics. The present study indicates that the use of certain processes when reading graphics in informational text is associated with a better overall understanding of that same text. Some possible reasons are discussed above, but more research is needed to investigate how and why these processes are contributing to the students’ comprehension. Moreover, differences in the relationship between the use of specific processes and comprehension scores were found amongst the three reading-achievement groups. Future research utilizing a larger sample size and more sophisticated statistical analyses than correlations is needed to determine whether and to what extent the relationship between the use of processes and comprehension scores is moderated by reading-achievement. Additionally, neither the quality of students’ processes nor the relationship between the quality and students’ comprehension scores was analyzed in this study. Future research should study whether such a relationship exists. Furthermore, research indicates that students are being taught to use processes as they read written text, but no research exists to indicate they are being taught to use these processes as they read graphical text; survey and observational studies are needed to investigate whether and how teachers are instructing students to use processes as they read the graphics in text. Finally, because this research suggests that the use of specific processes is associated with better comprehension, intervention studies that teach students how to strategically use these processes and comprehend graphics are needed. We are developing our understanding of the use and comprehension of the graphics in informational text, but we have a long road before we have a complete picture.