Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

The Need for Early Interventions

Children will have had many encounters with written language before the start of formal reading and writing instruction in first grade. Most parents read to their children, often on a daily basis. Parents support children’s emerging story comprehension by asking questions or explaining unknown words. Scaffolding can trigger young children’s interest in books as the increased story comprehension may make sessions more enjoyable (Bergin 2001; Bus 2001; Bus et al. 1995, 1997; Bus and Van IJzendoorn 1997).

Children are also socialized in writing. They see their parents writing letters or shopping lists, thus making children aware that written language conveys meaning and can be understood by others. The value of written language for communication is even more compellingly demonstrated to children through modern written communications such as e-mail and texting as replies are often instantaneous. These experiences, with adults engaging in meaningful written communication, inspire young children to write themselves (Levy et al. 2006).

The support provided by parents in understanding the language in books and with early attempts to write helps children to gain an insight into the relation between spoken and written language. Thus, shared reading and writing activities not only provide quality time with the parent but are also cognitive stimulating activities (Bus et al. 1995; Mol et al. 2008, 2009). Also, with the support of the adult, children may come to understand the sophisticated language in books. In contrast to the language children encounter in daily life, the type of language used in books is more lexically diverse and higher in thought content (Chafe and Tannen 1987).

Familiarity with the type of language used in books is essential for learning to read. For instance, reading unknown words is more difficult than reading words that are already stored in the mental lexicon. Compare the words confirm and dinner, words not uncommon in children’s storybooks. If young readers do not know the words, they may find it more difficult to decode these words. The graphemes refer to phonemes, but that the second syllable in confirm and the first in dinner should be emphasized cannot be inferred from the graphemes. Writing does not include features such as intonation, rhythm, phrasing, and pausing.

Later on when the focus in reading instruction changes from decoding words to text comprehension, language skills and background knowledge become the limiting factors (Cunningham and Stanovich 1998). The foundation for these skills is laid early in a child’s life, long before formal schooling starts. Vocabulary gaps that arise in these early years persist or even widen in the years that follow (Biemiller and Slonim 2001). Familiarity with the language in books is essential to participate in education as it is the form of discourse used in schools (Anderson et al. 2003).

Ferreiro and Teberosky’s publication (1978) triggered educators’ and reading researchers’ awareness that lacking these early experiences puts children at a disadvantage. Children may miss the necessary skills and knowledge to profit from initial reading instruction, resulting in increasing delays in the years that follow (Juel 2006; Duursma et al. 2007; National Center for Family Literacy 2008; Raudenbush 2009; Snow et al. 1998; Stanat and Christensen 2006).

In line with this, Heckman (2006) demonstrated that the yield of early interventions is significantly higher than interventions later on. Early interventions can prevent early delays in language or at least diminish them. This in turn may give children a better kickoff when formal schooling begins, thus making reading instruction more effective (Al Otaiba et al. 2010).

Early interventions might reach more children through the use of technology. Internet-connected computers are everywhere nowadays (Anand and Krosnick 2005; Calvert et al. 2005; CBS 2008). Especially devices such as laptop, computers, and tablets allow digital content to be accessed wherever and whenever wanted. Training teachers to deliver effective interventions (Justice et al. 2008) or requiring parents to build in literacy activities that are not part of their every day experiences can be a daunting task (e.g., Brooks-Gunn et al. 2002). The distribution of effective software programs might be easier to accomplish.

Therefore, the aim of this chapter is to explore whether computer programs can effectively intensify early literacy experiences of preliterate children at risk. Story comprehension and vocabulary are not only major determinants of beginning but also of advanced reading comprehension (Cain and Oakhill 2006; Lesaux et al. 2010). In this contribution, we will focus on effects of digital stories or “video storybooks” (also referred to as e-books, CD-ROM storybooks, or living books) as an alternative for shared reading to stimulate story comprehension and vocabulary learning.

Story Understanding

Video Storybooks: A New Generation of Digital Picture Storybooks

From their first appearance, it took more than a decade for digital picture storybooks to attain a well-designed format. The first generation appeared in the 1990s with numerous features that were attractive but did not relate to book reading (De Jong and Bus 2003; Korat and Shamir 2004). Spoken text and pictures were accompanied by hot spots unrelated to the story content, illustrations that could be activated to show a video not related to the events, and a table of contents, icons, and other rather arbitrary extras. These early digital storybooks were hybrid and could be considered as books but also as computer games. Especially for kindergartners, who mainly used computers for playing games, activating these hot spots to evoke funny sounds or animations became a goal in itself. They completely ignored the story line (De Jong and Bus 2002; for older children see also DeJean et al. 1997). An experiment with these first-generation digital books (De Jong and Bus 2002) showed that most kindergarten children scarcely listened to the story text and ignored book-reading conventions such as reading a text from top to bottom or from front to back. They crisscrossed the book in search of games and funny animations.

A new generation of digital picture storybooks emphasizes story meaning. Multimedia contributions in these video storybooks are limited to the spoken text and animated pictures with well-chosen sounds and music to convey the story meaning. The end product is a filmlike presentation of the story that maintains the (literary) story text and the original storybook’s pictures. There are several excellent examples, see, for instance, the scholastic video collection of award-winning picture storybooks (Scholastic 2002) that consists of video adaptations of picture storybooks such as Where the Wild Things Are, Harold and the Purple Crayon, and many others. Unfortunately, there are bad examples as well, which contain many redundant animations that distract attention from the story line, thus obstructing story comprehension (see also De Jong and Bus 2003).

Effects of Video Storybooks on Understanding the Main Story Line

It is evident that video picture storybooks leave less room for guessing the meaning of the story than print storybooks. With the aid of cinematic techniques, scenes that are normally compressed into just one static illustration are cut into a series of shots, thus showing the successive steps that form the action (Gibbons et al. 1986). Compare, for instance, an illustration from the static picture storybook Winnie the Witch by Thomas and Paul (1987)) with the video version of the same book. The illustration in the static picture storybook shows Winnie in the doorway holding her cat Wilbur above the grass. From an early age, children try to infer causal relations from what they see (Gergely et al. 1995), but illustrations often are ambiguous (Greenhoot and Semb 2008; Nikolajeva and Scott 2000; Peek 1993). Does Winnie want to prevent Wilbur from sneaking out of the house? The video storybook shows the opposite to be true: Winnie puts Wilbur outside on the grass.

The additional information provided in video storybooks may offer the extra support needed by children who have difficulty understanding the story text. We tested this in an experiment with 5-year-old Moroccan and Turkish children from low SES immigrant families (Verhallen et al. 2006). Since attending school (the day they reached the age of 4), children had been receiving instruction by means of a second language educational program in immersion classes.

In their retellings, children mentioned about half of the story actions after being read the print version, where they heard the story on the computer with static illustrations. However, after listening to and seeing the video version of the story, they were able to retell significantly more story actions (more than 60%). The video version supported the memory of story events, but this was only true for children who scored among the lowest 25% on a Dutch standardized test for language development (CITO Language Test for Senior Kindergarten Children, Centraal Instituut voor Toets Ontwikkeling 1996). Children scoring within the normal range understood the story’s actions irrespective of the version (Verhallen and Bus 2009). It thus seems plausible that the need for visual support will eventually disappear as a result of increasing language proficiency (see also Silverman and Hines 2009).

Effects of Video Storybooks on Understanding Internal Responses

Understanding internal responses of the main story characters is essential for story comprehension (Van den Broek et al. 2005). These emotions, beliefs, or motives form the reasons for story actions. When Wilbur the cat is no longer black but all the colors of the rainbow, he feels embarrassed and hides in a tree. Winnie the witch worries about Wilbur and undoes the spell. Internal responses from Wilbur (embarrassment) and Winnie (worry) determine the course of events here. In order to ­follow the story line, the reader needs to identify with Winnie’s and Wilbur’s emotions. When kindergarten children have an underdeveloped theory of mind (Wellman 1990), this is too difficult (Thompson and Myers 1985; Van den Broek et al. 1996). They do not pay attention to the internal responses of the main story characters. In their retellings of the stories, children will mention that Winnie turned the cat green (action) or that she put the cat outside (action), but not that Winnie became furious (internal response) when she stumbled over Wilbur for the umpteenth time, which resulted in her taking action. The Winnie the Witch story included six internal responses, but the kindergarten children in our experiments who heard the story accompanied by static pictures only recalled one (Verhallen et al. 2006).

However, in video storybooks, attention is more easily drawn to the internal responses, mental states, or intentions of the main characters such as anger, happiness, and concern. All children – those scoring below as well as within the normal range of second language development – were more aware of internal responses when they “read” the video version of Winnie the Witch (Verhallen and Bus 2009). On average, they retold 50% more internal responses compared to children who only saw static pictures. As the use of mental state language is related to ToM performance (e.g., Nation and Norbury 2005), it may be that children’s theory of mind develops when they focus more on the internal responses of the main story characters and imagine how these emotions have an impact on the story events that follow. This hypothesis awaits further testing in studies that also include false belief tasks as an outcome measure. Apparently, video storybooks not only stimulate story comprehension of children at risk but children who are not at risk also profit.

The Impact of Video Storybooks on Internalizing the Story’s Vocabulary

The language in books is rich and complex. Compared to oral language, the number of rare words in books is much higher (Chafe and Tannen 1987; Cunningham and Stanovich 1998). The use of difficult words, such as orchard and clearing, for example, is common in children’s books. Sentences are also more complex in books than in day-to-day communication. Even books for the youngest contain complex sentences to describe relations in the story. Take, for example, the phrase “He played tag with other dogs and became dirtier still,” from the still very popular picture storybook Harry the Dirty Dog by Gene Zion (1976). Children have to understand that the word “and” refers to a consequence of “playing tag” with other dogs, namely, you become dirty. The last two words infer that he was not clean to begin with. Children’s books also contain many proverbs and turns of phrase such as: he has earned his rest not even one note of a birdsong (from: Frog and the Birdsong, by Max Velthuijs 1991).

It is often claimed that animations have a detrimental effect on internalizing the story’s wording, thus obstructing vocabulary learning (Hayes and Birnbaum 1980; Hayes et al. 1981). Film scenes that almost speak for themselves are supposed to reduce children’s motivation to carefully listen to the story text. Video storybooks are considered an easy medium that induces less effort being put into understanding the story than a story with static pictures that is read to them by an adult (Cennamo 1993). We found no empirical evidence for this assumption. Video storybooks not only stimulate story understanding but vocabulary growth as well (e.g., De Jong and Bus 2002, 2004). Figure 4.1 shows that repeated reading resulted in vocabulary learning (more gains in the video and static storybook groups compared to the control group), but that the video version was an extra strong stimulus for the acquisition of new words (Verhallen and Bus 2009). Apparently, guiding children’s attention to the meanings of unfamiliar words by cinematic techniques may take the place of more interactive features added to video storybooks like a dictionary option as was used in Korat’s study (2010).

Fig. 4.1
figure 1

Vocabulary growth in Moroccan and Turkish kindergarten children as a result of repeated “readings” of video or static storybooks or no storybook reading (control group)

Full size image

Other research among children who learn Dutch as a second language showed similar results (Verhallen et al. 2006). In the United States, Silverman and Hines (2009) found similar positive effects of video additions to storybooks on English vocabulary of English language learners.

Why Are Video Storybooks a Strong Incentive for Understanding and Internalizing the Story Text?

The theory that images linked to language help to give meaning to words and sentences, and in turn help to store the story’s wording in the mental lexicon, has been extensively researched (Nino and Bruner 1978; Sipe 1998; Snow and Goldfield 1983; Weizman and Snow 2001). In both static and video storybooks, illustrations with their numerous details are added to the story text, and there is a strong possibility that what children hear coincides with what they see. Children focus on human characters or, as is often the case in stories for the youngest, animals with human characteristics. This is not surprising as their adventures generally form the core of the story. Using eye-tracking equipment to measure where children look in illustrations, Verhallen and Bus (2010) showed that children usually focus on details in the illustrations that match the spoken text, thus matching visual and spoken information. Similar results were found in studies by Evans and Saint-Aubin (2005) and Justice et al. (2005). Video storybooks increase the chance of a good match. Take, for instance, the illustration from Bear and Piglet (by Max Velthuijs 2003). We see Bear sitting in the sun and in the background Piglet carrying a heavy load on his back. The story text tells that Piglet has no time; he needs to work in order to prepare for wintertime. Besides Piglet with her heavy load, the static illustration shows many other details. There is a risk that children will not focus on Piglet but on less relevant details in the illustration. Cinematographic techniques in a video storybook like motion, zooms, and pans attract children’s attention to that part of the illustration that is mentioned in the story text. Piglet walking will thus attract the child’s attention. It becomes more likely that children will fixate on Piglet when the computer voice says that she “keeps on working.” This raises the chance that a strong association between “keep on working” and the image of hardworking Piglet will be forged.

Video storybooks with cinematographic techniques like animation and zoom seem to help children match verbal and visual information. Preliminary results showed that children who saw and heard the video version indeed spend significantly more time looking at the relevant visual information compared to children who saw static pictures (Verhallen and Bus 2010).

Effects of Interactive Video Storybooks on Understanding and Internalizing the Story Text

Multiple-Choice Questions

Biemiller and Boote (2006) showed that asking questions during shared reading positively influences the numbers of difficult words kindergarten children will learn. Asking questions and requesting answers seem to provide an opportunity to learn unfamiliar words (Karpicke and Roediger 2008; Mol et al. 2008; Whitehurst et al. 1988). Questions can easily be inserted in digital storybooks, but would these digital questions produce similar learning effects as when questions are asked by an adult during shared reading? To test this, Smeets and Bus (2009) used video storybooks with multiple-choice questions built in. In each reading session, a computer buddy appeared on the screen and asked a question about a difficult word in the story text (e.g., “Where do you see a shy bear?”). The children had to choose the correct one from three pictures that show the bear in different moods (shy, broken, furious). In all conditions (with and without questions), the number of readings was kept the same to test whether a simple adaptation can make video storybooks more effective than hearing the story two times while looking at the corresponding video images without inserted questions. Each of the difficult words was presented once as a multiple-choice question.

Words that were the main focus of questions were more often learned. Dutch kindergarten children learned more words expressively when questions were added. Children learned one out of eight difficult words when no questions were asked but two to three words when questions were added. Accordingly, with questions, they learned 15–20% more words than after hearing the story as often but without inserted questions (Smeets and Bus 2009). The questions required an active stance in which children needed to connect the target word to its meaning instead of listening to the target word being repeated. The mechanism described by Karpicke and Roediger (2008), namely, the “repeated retrieval through testing” probably explains the extra effect of the multiple-choice questions. The questions require children to retrieve the meaning of the target word whenever a question pops up on the screen, whereas reading without questions is more similar to what they call “encoding during study.” These results contradict Sénéchal’s hypothesis (Sénéchal and LeFevre 2002) that saying the focus word(s) is necessary for children to expand their expressive vocabulary skills.

Kindergarten children in these experiments were native speakers from Dutch low SES backgrounds with only moderate language skills. Whether results will be the same in groups of at-risk second language learners needs to be tested. However, a recent study by Roberts and colleagues (Roberts et al. 2010), showing that SES may be the more significant factor when explaining growth trajectories in language skills, seems to suggest that similar results may be obtained using this technique with at-risk second language learners from similarly low SES backgrounds.

Digital features, such as the multiple-choice questions, seem to draw children’s attention to depictions of unfamiliar words and their meanings quite naturally. Digital storybooks permit the repetition needed to add new words to children’s lexicon to occur without the aid of adults. Computers thus become a powerful tool for expanding the child’s vocabulary especially in the lives of young at-risk children for whom adult-led shared book reading may not be a common practice (Shamir et al. 2008; Verhallen and Bus 2010; Verhallen et al. 2006).

Level-Up Effects

When at-risk children who score within the lowest 25% on a standardized language test repeatedly hear the same story, they learn 15–20% of the difficult words of the story. After 20 min, the time needed to “read” the story of Winnie the Witch four times, Turkish and Moroccan kindergartners had learned 6 of the 42 difficult words (Verhallen and Bus 2010; Verhallen et al. 2006). The magnitude of this effect becomes clear if we extrapolate this increase to the number of words that children would learn after reading a series of books spread over a whole year. Suppose that children “read” video storybooks for 20 min a week. In that case, their vocabulary would expand by more than 300 words a year (6 per week) (Bus et al. 2009).

Of course, a number of conditions would have to be met: (1) children are attentive, even when no adult is present when books are repeatedly read as was the case in our experiments; (2) each story offers a varied vocabulary of about 40 new words; (3) digital libraries provide enough books to promote reading on a regular basis. Children at risk who learn Dutch as a second language need more reading sessions per week in order to close the gap with their more affluent peers. Reading twice a week for 20 min per session would result in learning about 600 words per year. Unfortunately, there are obstacles. A digital library with at least 60 picture storybooks is necessary, but this number of video storybooks is not yet available. Furthermore, reading routines in classrooms or at home need to be developed. “Reading” video storybooks should be a pleasant event integrated in daily routines (see also Bus et al. 2009).

For native speakers from Dutch low SES backgrounds with only moderate language skills, inserting multiple-choice question into video storybooks appears to be a promising road. With only a few well-chosen questions inserted, children gained on average 15–20% more words.

The Future

Experimental research convincingly shows that video storybooks positively influence story comprehension and vocabulary. The investigation of how effective interventions could find their way to children at risk is essential but too often neglected. Even though all children may profit from reading video storybooks, children at risk have the greatest need of extra learning opportunities. They will profit from the unique features of video storybooks, so the next step is to test how these computer programs can be implemented in the homes and in (pre)schools. Efforts to discuss and model literacy activities in the home with low-educated parents are not very successful (e.g., Raikes et al. 2006). Offering video storybooks via the Internet, however, might provide new opportunities, but it is naive to believe that the availability of educational content on the Internet in itself will change (the lack of) home literacy practices (see, for instance, Neuman and Celano 2006). Similarly, we expect that efforts are needed to help teachers to implement video storybooks in daily classroom practices (see also Selwyn et al. 2009). The use of computers to compensate for the lack of early literacy activities provides teachers with new possibilities. However, finding ways of implementing this in a day-to-day setting in classrooms for at-risk children may be difficult. Teachers need to abandon an egalitarian approach in order to give each child an equal chance to learn. In kindergarten classes, the time allocated to independent seatwork is limited due to other activities like circle time, playing outside, and time for snacks and drinks. Although computers could be used all day, we found teachers to use computers only for limited periods of time, for instance, during work and play (see also La Paro et al. 2009). Further, with few computers available per class, teachers are reluctant to let some children on the computer while excluding others. However, not all children need the same educational content. The evidence provided here suggests a more strategic use of the computer. Although it is currently often used in class management as a way of rewarding more advanced pupils, computers could be more effectively deployed as a learning tool.

The planning problems that may result from attempts to make computers an integral part of the curriculum can, according to Morrison and colleagues (Connor et al. 2007), be solved by applying computer algorithms that facilitate decision making in educational content and educational management. We expect that planning problems can also be reduced when computer programs assist in selecting children at risk, for whom the program is suitable or when the content is adapted to individual differences.

In sum, although there are still some practical obstacles to overcome, video storybooks have the potential to positively affect literacy development, thus empowering young children at risk. The results with video storybooks so far, and meta-analytic evidence with shared book reading by an adult (Bus et al. 1995), justify high expectations for long-term effects of video storybooks, but studies are needed to demonstrate the validity of this assumption.