Contemporary texts require readers to comprehend information from an extensive range of images integrated with print material. The increasing use of images in instructional material, particularly in the case of multimedia and web-based texts, marks a shift away from the print medium towards digital media as the means through which resources for learning and teaching are designed and distributed. New computer capabilities have enabled text and graphics to be combined in different ways, entailing not only new ways of representing knowledge but transforming the ways in which this knowledge is produced and communicated, and in many cases, transforming the knowledge itself (Kress 2005). These transformations are both qualitative and quantitative, and it could be argued, with image displacing writing as the central mode of representation in learning resources across curriculum areas including those of Science, Mathematics and English (Bezemer and Kress 2008).

The impact these shifts have had on literacy practices can readily be seen both inside and outside the classroom. Even so, discrepancies exist between the literacy practices of students growing up reading image-saturated multimedia texts, contemporary reading practices in schools, and how these are formally assessed; as yet, a substantive account of the ways in which different kinds of images interact with language in models of reading comprehension which could be applied to assessment frameworks is lacking (Unsworth 2006, 2008a). Furthermore, current assessment instruments such as the National Assessment Program—Literacy and Numeracy (NAPLAN) do not test the reading skills or strategies students require specifically to interpret the meanings made jointly through images and print in contemporary texts. However, some group test materials have increasingly targeted the reading of images in their item sets. This was the case with the NSW state government Basic Skills Tests (BST) and English Language and Literacy Asssessment (ELLA), now replaced by the National Assessment Program in Australia. The Program for International Student Assessment (PISA) in recent years has also incorporated “non-continuous texts” such as charts, graphs, maps and diagrams into its assessment of reading (OECD 2006). Further advances were made in the 2009 PISA, which extended the reading test to the electronic medium, taking into account a range of different skills needed for online reading literacy (OECD 2009). Despite these developments, a comprehensive account of the interaction of meanings across different semiotic modes and the impact this interplay may have on reading comprehension in print and digital environments is still in its infancy.

Related studies

Studies of the cognitive processes involved in learning from pictures, text and multimedia, and comparisons of verbal and visual processing have contributed some insights into what may be needed (e.g. Ainsworth 2006; Mayer 2008); while research on reading in online environments and studies of hypertext media from the perspective of new literacies studies (e.g. Leu et al. 2004, 2008; Unsworth 2008c) are gaining momentum, complemented by the growing body of work on multimodal semiotics (Kress and van Leeuwen 2006; O’Halloran 2004, 2008; Royce and Bowcher 2007) and its applications in education (e.g. Jewitt 2006, 2009; Unsworth 2008b). Building on this research trajectory, this paper examines some of the complexities presented by the interaction of image and language in hyper-linked multisemiotic texts, and the challenges of reading such texts in online environments. The results reported offer a view of the reading processes from the perspective of students’ own comments on their experiences.

Image–language interaction

When knowledge is represented and communicated via multiple semiotic modes, meaning is distributed across these modes. In other words, different aspects of meaning are carried in different ways by each mode (Jewitt 2006; Lemke 2005b), and each carries only part of the meaning in relation to the whole, not only complementing each other in representing various dimensions of meaning in a “division of semiotic labour” (Matthiessen 2007) but also having a multiplying effect on these meanings (Lemke 1998). Furthermore, the meanings represented in one semiotic mode influence our interpretation of the other. For example, when text and graphics are combined in various ways, such as when images are inserted into text or when images are accompanied by titles, captions or commentaries, the juxtaposed meanings interact. As a result, the meaning relations among multisemiotic elements may orient the reader towards a particular sequence of viewing and interpreting the text (Lemke 2005a). Another consequence of the interplay between modes is that there are often tensions in the meanings represented through different semiotic resources, and while these may encourage reflection and critique (Jewitt 2006) and so enhance comprehension, they could also lead to confusion and misconceptions. An understanding of the communicative and representational potential of different modes and how they interact in print and in digital media are thus important considerations in the design of pedagogical material and is critical to understanding the processes of reading in multimodal environments.

In a study of image-text relations in group literacy tests for schools in NSW (Unsworth et al. 2006–2008), we found that different types of relations between words and images vary in the degree of difficulty they pose for students’ reading comprehension, and that negotiating some of these image-language relations were among the most difficult reading tasks in the test. The relationship between relative item difficulty and the type of image-text relation was demonstrated to be statistically significant, with no significant interaction effects from other variables contributing to item difficulty (Unsworth and Chan 2008). In light of the increased difficulty that students may experience in integrating meanings across image and language in the print medium, it could be expected that hypermedia reading environments pose additional challenges where the synthesis of meaning across semiotic modes is required for reading comprehension.

Hypertext and reading comprehension

Studies of online reading comprehension have been consistent in reporting that reading on the internet requires skills and abilities both similar to and additional to those required in the comprehension of conventional print (e.g., Coiro 2003; Dalton and Proctor 2008; Leu et al. 2004; Henry 2006; Leu 2007; Palinscar and Dalton 2005; Walsh 2006; Wyatt-Smith and Elkins 2008). In traditional media, students locate main ideas, summarise, make inferences and evaluate from print and two-dimensional graphics. The meanings in written text unfold in linear sequences and compositional structures in print are fixed. Online reading environments present new text formats characterised by non-linear structures, interactivity, and multimedia hypertext. These characteristics, Coiro (2003) argues, require different thought processes to use these features, including high-level metacognitive skills and new ways of interacting with information. In a qualitative study of highly proficient sixth grade readers, Coiro and Dobler (2007) observed that children who had successful Internet reading experiences required more complex applications of prior knowledge sources, inferential reasoning strategies, and self-regulated reading processes than when reading traditional print. The authors suggest that reading internet text prompts a process of self-directed text construction that may explain the additional complexities of online reading comprehension.

These findings are supported by research focusing on the analysis of hypertext from various perspectives. A defining feature of hypertext is that it allows users to combine and sequence minimal textual units (or lexia) “logically, temporally or experientially in ways not uniquely determined or anticipated by authors/designers” (Lemke 2005a, p. 52). The fluid character of hypertext and its linking devices enables readers to traverse connections across a website and beyond its boundaries, without being constrained by a linear progression imposed by the structure of the information. While having greater freedom of navigation, readers also risk becoming lost in their quest for coherent meaning. In studies of the structure of children’s websites and user navigation paths, Djonov (2007, 2008) demonstrates how the interaction between content, webpage and navigation design has the potential to obscure the structure of information on the web and so disorient readers. The multiple interconnections between elements on websites can be explicit or implicit, with hyperlinks creating relations and continuity between elements, but also discontinuity (Lemke 2002, p. 300).

Success in navigating a coherent reading path through discontinuous elements has been found to be related to students’ prior knowledge of both content and system domains. Studies of the navigational behaviour of older students indicate that prior knowledge of the content domain enables students to navigate more easily through information in hypermedia environments, and to make more efficient and effective selections of content, resulting in better comprehension of material than those with little prior knowledge. This effect was enhanced when students were also familiar with the online hypermedia contexts in their content areas (Lawless et al. 2007; Lawless and Schrader 2008).

One of the major tasks of successful online reading then is the effective integration of meaningful information relevant to the activity or purpose at hand. Leu (2007, p. 27) describes integrating as “connecting various pieces of information in order to identify similarities and differences, make comparisons of degree, and understand cause and effect relationships”. Together with integrating, accessing information (i.e. search and retrieve skills, including searching, navigating and locating), and evaluating information are identified in the 2009 PISA as other key competencies essential for online reading (OECD 2009). Interpreted in the light of multimedia hypertext studies, successful application of these skills involves interpreting the interplay of representations realised through different semiotic resources, such as image and language, and engaging prior knowledge to predict relations of coherence between them where there may be apparent discontinuity. At the same time, threads of meaning need to be held in memory across traversals (some pathways are lost along the way), assessing information for its fit to purpose in the process, while synthesising meanings into a focused understanding of the information accessed.

Background to the study

In the earlier stages of the project (Unsworth et al. 2006–2008), we analysed the interaction in representational meaning between images and written text in paper-based test materials in terms of semantic expansion, organising our description around the relations of “elaboration”, “extension”, and “enhancement” (Halliday 2004). Sub-categories of these relations were applied in the analysis (Chan 2010; Unsworth 2006, 2008a for detailed discussion), with examples of these appearing in a section to follow. Across five reading tests for students in years 3, 5 and 7 (NSW Department of Education and Training, 2005a, b, 2007a, b, c), 40 items (62.5%) involved relations of elaboration, where one mode further specifies or describes the other while no new ideational element is introduced; and 24 items (37.5%) involved extension, where meanings additional to those in one mode are represented in the other mode. There were no test items which targeted relations of enhancement.

A one-way analysis of variance revealed a difference in the mean item difficulty (δ), determined by Rasch analysis, of the image-text relation types, significant at the 0.05 level. Subcategories of extension (“augmentation” and “distribution”) were found to be more difficult than subcategories of elaboration (“exposition” and “equivalence”) for this set of test data. Further ad hoc analyses found no significant interaction effects from other variables such as text genre, image type, the type of reading skill assessed, and response strategy required to answer the questions correctly. (For a more detailed account of the preliminary results, see Unsworth and Chan 2008.)

In relations of equivalence between text and image, where there is a correspondence of meaning across modes, each mode mutually reinforces the meanings of the other. It could be expected that image-language relations of this type are the easiest for students to comprehend because of the redundancy of information. By contrast, in materials where the meanings in the text and image/s are different and complementary in the information they represent, such as when different events or processes are distributed across image and language, it could be expected that greater demands are made on a student’s ability to access and integrate this information across the modes. These expectations were supported by the test results of student cohorts across the state on items targeting these kinds of information (Unsworth and Chan 2008). The findings imply that the greater the difference in the meanings represented in image and language, the greater the level of cognitive demand on the reader in synthesising these meanings into a coherent understanding of the material as a multi-semiotic whole. In the next section, we examine whether these findings may be extrapolated to reading tasks in an online environment.

Research questions

The understandings about image–text interactions and online reading outlined above raise a number of questions of interest for this study:

  1. (1)

    Are image-text relations in online reading materials comparable to those encountered in print materials?

  2. (2)

    How do students read these image-text relations?

  3. (3)

    What evidence is there of students integrating meanings from across semiotic modes in an online environment?

  4. (4)

    What features of an online reading environment may interact with students’ integrative reading of images and text?

Methodology

Thirty-two Year 6 students were surveyed about their internet usage and then interviewed while working through a selection of materials from the Australian Museum website, Wild Kids (Australian Museum Education 2003). The surveys and interviews were conducted face-to-face with individual students using structured protocols and response coding sheets. Interviewers noted student responses on the coding sheets in situ. Each interview was also audio-recorded and then transcribed to support the coding and analysis of results. Active consent for participation was obtained from students and their parents/guardians according to ethical requirements for research involving children.

Participants

The 32 students in this pilot study were a sub-group of the participants in a larger longitudinal research project (n = 125) designed to investigate students’ reading of image-text relations in statewide literacy test materials for schools (Unsworth et al. 2006–2008). In this phase of the study, consenting participants from the Year 3 cohort of students, who sat for the NSW BST in 2005 (NSW Department of Education and Training 2005a) and participated in the first 2 years of the research, were followed up when they were in Year 6.

The participants represent a sample of high, medium, and low performers on the state reading comprehension tests selected from 4 metropolitan primary schools out of the 10 participating schools across metropolitan, provincial and remote regions of New South Wales. Only metropolitan schools were included at this stage of the study as reliable access to the internet could not be guaranteed during the data collection period at the remote and provincial sites included in the earlier stages of the project (Table 1).

Table 1 Participant sample for the pilot study
Full size table

Materials

A selection of three webpages from sites within Australian Museum Education (2003) were included as the stimulus materials for the online reading tasks: “Wildlife of Sydney: Activities” (Life Cycles); “Australia’s Lost Kingdom: Games” (Layers of the Past; The Big Animal Debate); and, “Living Colour”. The material on these pages consisted of short, complete texts covering a range of factual and literary genres accompanied by images of various kinds, with content appropriate for students in the final year of primary schooling (11–12 years of age), making them comparable to the BST and ELLA stimulus texts included in the earlier stages of the research.

These webpages also met our selection criteria in that they included a range of features typical of internet texts—the pages were hyperlinked; had some degree of interactivity and a variety of navigational prompts; included both still and/or moving images or multimedia; and could only be accessed via a live internet connection. An analysis of the materials identified a set of features which could be applied systematically to the description of each webpage.

Instruments and procedures

Participants were withdrawn individually from regular class lessons and seated at a computer with a live internet connection to complete the online reading tasks during a recorded interview. A structured interview protocol was used to elicit open-ended answers to the questions shown in Table 2. Following the response given for each question, students were asked to indicate whether they thought the question was difficult and if so, why. Further prompts were used to elicit information about their awareness of relationships among the words and images and to identify aspects of the language and images they found difficult to understand. The interviewers guided students to the selected webpages through basic navigational instructions. For the purposes of the research, the potential reading paths of the participants were constrained in this manner to focus attention on the selected materials and reading tasks. Participants’ navigational choices were observed and noted on the coding sheets by the researcher/interviewer.

Table 2 Item descriptors
Full size table

Prior to the online reading tasks, the participants were surveyed about their internet access, usage and engagement. The survey, consisting of 10 questions with closed response options, is included in Appendix.

Student responses were coded independently by two researchers then checked against the transcribed audio recordings for accuracy of coding. The image-text relations encountered in the reading tasks were analysed by one interviewer/researcher and a third independent analyst, achieving an inter-coder reliability measure of 0.9 using the percentage agreement method.

Discussion of results

Students’ internet access, usage, and engagement

Thirty-one out of the 32 participants reported having internet connections at home, with the majority (78%) having broadband or wireless connections; only one female student had no access. This compares favourably with the national average of 72% households with internet access, mostly with broadband connections (Australian Bureau of Statistics (ABS) 2009). All but one male student also accessed the internet at other locations, such as “friend’s house”, “school”, “library”, or specified “other”, with more girls reporting access at more locations than boys. In relation to the number of years of experience with the internet, 56% participants reported using the internet for 2–4 years and 38% for more than 4 years. Only two girls had less than 2 years experience. The majority of participants (76%) spent less than 4 h per week online; however, the boys as a group tend to spend more time online than the girls. Similarly, while half of the participants spent less than 2 h per week on a computer offline, more than double the number of boys spent more than 8 h compared to girls (5 boys, 2 girls). Thus, for both time online and offline, boys’ computer usage exceeded the girls’.

Participants were found to use the internet for multiple purposes. In order of response frequency, students used the internet for research (100%), games (88%), email (75%), chat (56%), blogging or social networking (16%), and other purposes (59%), for example, video (mostly You Tube) and downloading music. When boys and girls are compared, differences can be seen in the socially-oriented communicative activities, with greater proportions of girls engaging in email and social networking, and more boys engaging in individually-oriented activities such as games, viewing videos and downloading music (Table 3).

Table 3 Gender differences in internet activities
Full size table

All participants used the internet for research, but of interest is the relatively high frequency of this activity. 85% indicated that they sometimes or mostly use the internet to find and organise information for homework, research or learning, and 38% would communicate this information to friends. Also significant is that 84% chose the internet as their first source of information for research, with the proportion of low performing readers (89%) exceeding both medium (85%) and high (80%) groups. Less than half (41%) chose “books/other print materials” as their preferred source of information.

To capture a sense of students’ affective engagement when researching online, we asked how they felt when using the internet to look for information, with multiple responses permitted. In general, girls reported positive affect more than boys (confident, relaxed, excited). Similar proportions of girls and boys felt challenged or frustrated; and more boys reported confusion than girls.

A sense of competence appeared to accompany students’ feelings of confidence. Half of the students reported feeling confident, citing reasons such as:

  • I’m very assured to find what I’m looking for. (Lm)

  • I can find what I need. (Mm)

  • I’ve used the internet for a long time. (Hm)

  • I’m good at computers. (Lf)

Others reported feeling confident under certain conditions, for example, “when I find a good site” or “when it’s easy to locate information”.

Many of the reasons students gave for feeling relaxed relate to a preference for using computers over other activities:

  • no one annoys me… I prefer to use computers. (Lm)

  • I don’t have to write. (Mm)

  • I don’t have to read as much. (Lf)

  • don’t have to get up and go to the library. (Mf)

  • when playing, emailing. (Mm)

  • it’s normal; fun. (Lm)

Some students experience frustration when they “can’t find information” (Mf, Hf) and others feel challenged because it’s “hard to find information” (Hf), or “when I can’t find right website”, which suggests that the skills of searching for and locating information are important for a satisfying internet experience. One student stated that “it’s hard to learn new stuff” (Mf), suggesting the difficulty of cognitive tasks in an online environment.

Confusion was experienced by those with limited computer skills: “when I have to go on by myself, I don’t know what to do on website”; “I’m not very good on computer”; and information overload, “too much information”, was also a cause of confusion, as was a lack of understanding of web structures: “I don’t know what websites are what” (Mm); “index pages, I don’t understand, and getting out of them” (Hf), and ineffective searching skills:

  • getting right information for answers. (Mm)

  • not the information I want.

  • don’t always find what I’m looking for. (Hm)

The abundance of information on the internet was not always experienced negatively; it was also a reason for excitement: “lots of things you can find” (Lf) and “finding out new things” (Mf).

Compared across reading performance bands, students in all groups reported a range of both positive and negative feelings, with a greater proportion of low performing students reporting relaxed, excited and frustrated than both high and medium groups (see Table 4). For these students who often struggle with reading in the print medium, the internet offers an alternative way to engage with texts of various kinds.

Table 4 Internet research and affect: group comparisons
Full size table

Multimodal text comprehension in interactive, digital environments

As would be predicted by the model of image-text relations developed in the earlier stages of the research, questions involving equivalence of information in words and image were the easiest to answer with all participants obtaining the correct responses for Items 1, 15, and 18 (see Table 2). 91 and 75% correctly answered Items 2 and 16 respectively. All of these questions could be answered correctly using single strategies with alternative representational modes. For example, to answer Item 2 (“How many legs do hatchlings have?”), readers could find the information in the image or the text (see Fig. 1). Thus, in relations of equivalence there is some redundancy in representational meaning where the content corresponds across modes.

Fig. 1
figure 1

Equivalence of meaning in image and text (Item 2)

Full size image

Tasks identified as more difficult according to student performance on the questions involved inter-semiotic relations where image and text provided complementary information. These results are similar to those obtained in the in earlier stages of the study using print-based materials only:

  • Distribution, where complementary meanings (processes or activities) are distributed across image and text (Items 4, 5, 8, 9, see Table 2), on average, 65% students answered such items correctly.

  • Augmentation, where image provides additional ideational elements (participants) to text or vice versa (Items 3, 6, 14, 17, 20), on average, approximately 50% answered correctly.

  • Divergence where the ideational content of the text/image is opposed or at variance (Item 10), approximately 30% answered correctly.

Where complementary meanings are distributed across language and image, such as in “Life cycle—fishes” (see Fig. 2), readers had to locate the larval stage of the life cycle from the image and find the information needed to answer the question in the caption which only appears when the mouse is rolled over the image of the larvae. Thus, for Item 5 (“How do fish larvae move about?”), interpreting the image correctly was essential for accessing the appropriate text required for the answer.

Fig. 2
figure 2

Ideational complementarity—distribution (Item 5)

Full size image

In “Layers from the Past” (see Fig. 3), readers had to locate information stated in words (3.1) and connect this with image following (3.2) to answer Item 8, “Can you explain how the layer of shale was formed?” 59% of participants answered this item correctly with 25% of respondents identifying this as a difficult task.

Fig. 3
figure 3

Ideational complementarity—distribution (Item 8)

Full size image

The non-simultaneous, sequential display of the text window followed by the animated image meant that some students focused only on the text or only on the image, missing some of the more relevant meanings required for answering the question adequately. The distribution of information across the words and images is described in the following excerpt from the interview with a low-performing male participant (Lm):

  1. I:

    Now, if you look at this text here, and then the image that follows it, is the information you get the same or different?

  2. Lm:

    It’s the same. Except there’s a spider.

  3. Lm:

    The writing kind of gives … gives some information on how it happens.

  4. I:

    Mm. Is there anything that’s different? Do you want to look again?

  5. Lm:

    You don’t see it getting fossilised.

  6. I:

    Anything else?

  7. Lm:

    Yeah. You don’t see the mud building up.

  8. I:

    Okay, alright.

  9. Lm:

    And it doesn’t look like a swamp at all. It’s more like a river.

In the image “Why is the Sky Blue?” (see Fig. 4), the line indicating the white light is animated, moving in a single line from the sun in the direction of the earth where it spreads out on reaching the arc depicting the atmosphere. The words to the right of the diagram augment the meanings represented in the diagram, providing additional information about the scattering of light of different wavelengths. To answer Item 17 “What happens when white light is scattered by the earth’s atmosphere?” adequately, participants needed to connect information in the labels and diagram, recognise the cohesive links (synonymy: white light = sunlight; atmosphere = sky) and relate it to the explanation contained in complex sentences in a long paragraph. While the animated diagram depicts the physical phenomenon, the text explains the scientific concept. Many students responded with “it turns blue” or something similar. Only 31% provided some kind of explanation of how visible light is scattered, for example: “It goes through the air and reflects like blue colour and that appears making it blue… I didn’t really understand it at first, but then I got to, like, understand it more and when I was reading through it”.

Fig. 4
figure 4

Ideational complementarity—augmentation (Item 17)

Full size image

Overall performance on the tasks, such as in the example above, did not always correspond with the participants’ perception of task difficulty, that is, the tasks that students identified as difficult. Another example of where there was a discrepancy in performance and perceived task difficulty was Question 10. Which of the dinner guests thought Australia’s megafauna became extinct because of hunting? While a majority of respondents could identify one of the dinner guests, only 31% managed to identify both.

The animated image “The Great Animal Debate” (see Fig. 5) depicts activity (5.1) which is unrelated to the content represented in the text of the speech bubbles (5.2), which is revealed by clicking in turn on each depicted speaker. Here, the meanings constructed in image and language are divergent and there is no redundancy of information in the image which might support the reading of the text. As articulated by a high-performing male student (Hm):

Fig. 5
figure 5

Ideational complementarity—divergence (Item 10)

Full size image

‘Cause here it’s just them having dinner and then… but when they’re actually talking, it’s actually facts. Here it’s just an animated group of people having dinner, which… yeah and if you were just to look at this you wouldn’t know it had anything to do with big animals going extinct, like the dinosaurs.

With this stimulus, the speech bubbles did not display simultaneously and each pop-up text disappeared before many of the students could finish reading. Most respondents clicked to access and re-read the text more than once before answering the question and approximately a quarter of the students identified this item as difficult. Some of the reasons given were:

  • don’t get to finish reading. (Mf20)

  • can’t read text properly; words disappear and can’t find again. (Lf24)

  • re-finding who said what; lot of information. (Hm15)

  • hard to remember which one actually said what. (Hf17)

The difficulty here was not in accessing or integrating related pieces of information but in retrieving and holding the traces of meaning in memory while searching and locating. This difficulty is one related to the design of the material and could be alleviated by changes in how the speech bubble pop-ups may be viewed.

In this phase of the study, results concerning relations of exposition (Items 7, 11, 12, 13, 19, see Table 2), where image and language reinforce each other by reformulating meaning in some way, were not as clear cut as the findings from the earlier stages of the study involving print-based texts only. This could be explained by the small scale of the pilot study—the small participant sample and/or the relatively limited number of items in each subset and how the different features of online texts interacted with the image-text relations in this selection of texts. Another possible reason for this could be the unfamiliarity of some of the scientific concepts encountered in the texts. Further investigations on a larger scale would be necessary to confirm the results of this pilot study, and to examine more closely the interaction effects among possible sources of reading difficulty, the interplay of meaning across modes and the added dimension of hyperlinked text features.

Conclusions

The results above highlight some of the complexities involved in reading multimodal, hyperlinked texts in an online environment. While far from a presenting a comprehensive picture of the many facets of reading online, these illustrations are indicative of the kinds of challenges readers face even when dealing with web-based material specifically designed for educational purposes. In this study, 84% of the students said the internet is the first place they look for information for homework assignments and research. Readers who performed poorly on school reading tests, as a group, especially engaged in internet research as their first preference. Despite the positive affect and engagement reported by these low performing participants, their performance on reading tasks was not enhanced measurably in the online environment. If these figures reflect school populations elsewhere, then the ramifications for effective internet reading skills are substantial. As is evidenced by the results of this pilot investigation, the integration of meanings from complementary semiotic resources is essential for the effective reading of hyperlinked materials and web-based screen displays. Furthermore, the type of inter-semiotic relations that need to be negotiated in integrating such information play a part in the difficulty of reading tasks in both print and electronic media. Additional to the interaction of meaning across different semiotic resources are the challenges presented by how material is organised on the internet.

While learner characteristics such as a lack of familiarity or experience with using the internet may lead to frustration or confusion for young readers, the characteristics inherent in hypertext media present greater challenges for reading online effectively. The factors identified in this study which contribute to these challenges include:

  • Intersemiotic interaction of meaning across different media, and whether students can successfully integrate complementary meanings from image and text;

  • Timing, sequence, and interactivity of electronic multimedia, and how this may influence what meanings readers focus on, and how they connect, process and retain information;

  • Navigational difficulties and the potential for disorientation, while mapping a reading path through complex hypertext structures;

  • Implicitness of cohesive devices in moving between continuous and discontinuous texts, and visual and verbal information, while inferring connections and constructing coherent meaning; and

  • Higher-order skills required to access, evaluate, select, and synthesise information to achieve a task or learning goal.

As seen in this study, online multimedia texts offer the potential for greater engagement and enjoyment of reading for struggling readers, but educators must ensure that these students are well supported in the additional skills required for the successful location, integration and evaluation of information. Supporting students in their research activities on the internet until they become familiar and confident with conventions for navigation and the organisation of material on websites can assist in developing their skills in searching for and locating relevant information. Developing prior knowledge of the content domain and explicit guidance in predicting and making connections between different parts of a webpage or website is a pedagogical imperative for complex reading tasks, particularly where links are implicit and information is distributed across multiple, intersemiotic representations and needs to be integrated to achieve comprehension. The different ways in which image and language in a range of media complement each other in representing the significant meanings of a multisemiotic text need to be brought into focus in assisting students to become competent, critical readers of complex material. Finally, the assessment of reading needs to be considered in the light of students’ online reading practices and the characteristics of the texts they read, not only for the formative evaluation that informs pedagogy, but importantly, also in the design of test constructs of literacy so widely used as indicators of reading competence, instructional effectiveness and school performance.

While this study has afforded these general implications for practice, there are a number of inherent constraints in the nature of a pilot project, such as the small number of participants and the restricted sampling of websites, which limit the applicability of the results to a wider range of contexts. However, in keeping with the scope of a pilot study, the investigation has established that the analytical frameworks and instruments employed, with some refinement depending on the intended application, would also be of potential value for research on a larger scale, and/or in different contexts for negotiating multimodal reading in hypertext environments. Improvements which may benefit further research would include: in the sampling and analysis of websites, a larger range of educational websites used by school students as well as sites most frequently accessed out of school; and a more sophisticated tracking of students’ navigational choices, using video screen capture, for example, to record specific patterns of navigational difficulty.

Importantly, the study has enabled a mapping of features of interactive websites which contribute to the complexity of reading image-language interaction in hypertext environments and the difficulties this may present for readers. A direction for future research would be to examine how students negotiate multimodal meaning in online texts, in both school and non-school settings, in order to strengthen our understanding of the connections between students’ web-mediated learning in classroom and out-of-school contexts. It is envisaged that research of this kind may progress the work to be done in addressing the gap between engagement and educational outcomes that is prevalent in the literature on educational disadvantage.