Since the 1970s, developments in computer technology have led to digital devices becoming increasingly smaller, faster, cheaper and interconnected. In line with these developments, digital book production and consumption have progressively increased (McLean and Kulo, 2013; Yokata and Teale, 2014, p. 577). Against this backdrop, book apps have become a new format for the picture book (Yokata and Teale, 2014). Just as the printing press had a distinct impact on the codex book (Eisenstein, 2011), the computational environment now stands as the latest invention to affect literature in a significant way.

An increasing number of children access their literature via digital devices (McLean and Kulo, 2013), so it seems vital to understand the ways that digitisation has affected and will continue to affect the picture book. In Western societies, printed picture books have been a cornerstone in teaching children social and literary skills (Smeets and Bus, 2013, pp. 176–177; Yokota and Teale, 2014, p. 578). Through these books children can learn to “read” both words and images, and a number of researchers have detailed what this process involves (Nodelman, 1988; Sipe, 1998; Stephens et al., 2003; Nikolajeva and Scott, 2006). Yet there has been very little research that analyses children’s digital picture book literature (Yokata and Teale, 2014, p. 577).

The 2012 O’Reilly’s Tools of Change for Publishing conference focussed on the publication of children’s digital literature. Publishers at this event noted that, while there is a glut of software, hardware and content being developed, as yet there are “no dominant models [and] no standards” within this area (Hedlund, 2012, p. 23). This article therefore seeks to look at the underlying principles and frameworks relating to the design and analysis of digital works for children. In order to make such a foray, a transdisciplinary approach is necessary to explore the links being forged between literature and computer technology (Madej, 2003; Smeets and Bus, 2013; 2014; Yokata and Teale, 2014). This article considers a range of case studies and aims to contribute to the emerging body of literature directed towards discussing the design, analysis and selection of children’s digital books.

An Overview of Research Relating to Digital Picture Books

The educational theorist, Bill Cope, suggests a broad definition of the term “book”:

A book is no longer a physical thing. A book is what a book does…a book in this definition does not have to be printed. It can be rendered in many ways, including electronic-visual and audio (talking books). A book [is] not a thing. It is a textual form, a way of communicating. A book is not a product. It is an information architecture. (Cope, 2001, pp. 6–7; emphasis in original)

According to Cope, books can be handcrafted, printed or digital artefacts. They can also be presented in audio formats. In addition to this, Perry Nodelman reminds us that books can be constructed entirely from visual images, containing no written text (1988, p. vii). Some theorists state that these wide definitional parameters, together with the digitisation of books, may serve to unravel the core nature and value of the book itself (Birkerts, 2006). Yet, regardless of such fears, it is likely that both print and digital book formats will continue to coexist for some time (Eisenstein, 2011, p. 245; Yokata and Teale, 2014, pp. 577–578).

Printed picture books can combine written text and images (Nodelman, 1988; Nikolajeva and Scott, 2006) and they may contain interactive elements: “pop-up, open-the-flap, and peep-through-the-hole books give children the opportunity to play and learn as they make things happen” (Madej, 2003, p. 11). For example, Jan Pieńkowski’s Phone Book (1991) contains pop-up paper sculpting and inbuilt sound. The pop-up images move as readers open and close the covers and an audio sound-bite plays when audiences press a button situated in the top corner of the book. This work contains interactive aspects and inbuilt audio (Sargeant, 2013, p. 31). For decades, picture book authors, illustrators, designers, editors and publishers have combined their skills to present refined combinations of interactive features and media-rich designs. In doing this, they have stretched the boundaries of print literature production. Consequently, it is not surprising that the interactive, media-rich digital environment has provided a fertile field for picture book designers.

In the later part of the twentieth century, authors experimented with incorporating written text, visuals and audio into digital narratives. In 1987 the desktop computer application known as “Hypercard” was released. This software “made hypermedia a reality” (Madej, 2003, p. 9). Media theorist Ted Nelson states that hypermedia technology allows for “films, sound recordings, and video recordings” to be arranged “as non-linear systems” (1965, p. 96). For example, the 1987 children’s digital narrative, Inigo Gets Out (Goodenough, 1987) uses the Hypercard system as a means of storing and delivering a series of hyperlinked words, images and related sound effects. Users navigate this story, and to a certain extent shape the narrative, by “clicking on” specific areas of the screen. By interacting with the material presented, the story is revealed to the audience.

Game designers are often acutely aware of the ways in which narratives operate within interactive digital environments. During the first decade of the twenty-first century, a debate occurred in the discipline of games research. This involved two groups, the ludologists and the narratologists (Frasca, 2001, n.p.). Two of the central voices in this discussion were Jesper Juul and Janet Murray. Juul maintained what became known as a ludologist’s perspective; that is, he advocated that games studies should be informed by an examination of play and game theory (Juul, 2003, n.p.; Jenkins, 2004, p. 118). Murray, in contrast, adopted a narratological stance, discussing the potential for games to be informed by an underlying story (Jenkins, 2004, p. 118). This debate centred on the relationship between play and story in games, and it uncovered many of the challenges involved in designing interactive, narrative works. These discussions offer a rich backdrop for studies relating to the design and analysis of children’s digital storybooks.

Contemporary digital picture books not only feature narratives and interactivity, but can also contain a range of media elements. As children’s digital narrative theorist Krystina Madej states, computers “were originally text-based…but image, sound, and animation quickly took their place alongside text” (2003, p. 2). Digital books can contain written text, illustration, animation, video content, music, audio narration, sound effects and interactivity. According to Madej, when “children use a computer, they have before them, combined, all of the storytelling media of the past rolled into one. This makes for a rich experience. It is part oral tradition, part print tradition, part television tradition, all integrated to create a fascinating whole” (p. 2). The combinations of media found within children’s digital books thus provide a rich sensory experience for audiences.

The education theorists Junko Yokata and William Teale analyse the “quality of digital picture book features” (2014, pp. 580–584). They note that digital books can contain game design and filmic elements (pp. 579–580). They state that some books contain activities that can distract child audiences from the central narrative (p. 581). When reading digital books, children may focus their attention on interactive animated visuals and play-based features, and these may disrupt learning and narrative comprehension (Sargeant, 2013). The convergence of animated film, game design and picture book conventions can lead to a destabilisation of narrative structures within children’s digital books.

Cognitive scientists Daisy Smeets and Adriana Bus (2013; 2014) have studied the ways in which children learn from digital picture books. They have collected data relating to the educational outcomes that occur as a result of four and five year-old children reading interactive “storybook apps”; this research involves user-testing. Their findings show that animated digital books “are more effective in attracting children’s attention than are static e-books” (Smeets and Bus, 2014, p. 17). Within their studies, Smeets and Bus detail the ways in which digital books incorporate interactivity and “multimedia” features (2013, p. 176), these being central aspects of children’s digital book design.

Within the various fields that centre on the study of children’s literature, and the ways in which children learn from literature, the terms “ebook” and “book app” are frequently used to describe digital picture books (Smeets and Bus, 2013; 2014; Yokata and Teale, 2014). Yet we need to be clear about what these terms mean.

What is an ebook?

An “ebook” is an “electronic representation of a book” (Garrish, 2011, p. 1). The term is a contraction of the phrase “electronic book.” From a computer engineering perspective, digital books may be described by noting the software that was used in making the work. The most widely supported ebook platform is known as EPUB, which, the software developer Matt Garrrish (2011, p. 1) informs us, is a contraction of the phrase “electronic publication”. It is a software platform and a digital file type. EPUB is an extension of the Open eBook Publication Structure and as such it is touted as a global standard for ebooks, a format that may be used across different hardware devices (Garrish, 2011, p. v). The EPUB platform differs from software that is designed for use on specific computer hardware (the initial format for Kindle books, for example, could only be read on Kindle devices). Ebooks can be designed using a variety of software platforms and it is not always evident, outside of the development team, what platform has been used.

Commentators can also analyse digital books by assessing the type of content and the features offered. For example, Diary of a Wombat (French and Whatley, 2011) is an ebook that contains written text and static imagery. This work presents clear “skeuomorphic” elements. The term skeuomorph describes “legacy designs” that are used ornamentally within the digital realm. Computer scientist Nicholas Gessler states that skeuomorphs “provide us with familiar cues to an unfamiliar domain” (1998, p. 229). The skeuomorphic features that are often present in ebooks include virtual pages, virtual book gutters and virtual page turning animations. Diary of a Wombat has virtual verso and recto pages and a virtual gutter, as seen in Fig. 1; it also retains the conventional thirty-two page printed picture book structure. In addition to these print-related features, the designers of Diary of a Wombat provide readers with zoom functionality, which enables a close-up perspective of Bruce Whatley’s illustrations. There is no animation or audio included in this work; thus Diary of a Wombat is essentially a digital replica of the printed book of the same title.

Fig. 1
figure 1

Diary of a Wombat (French and Whatley, 2011) is a fixed format ebook with clear skeuomorphic elements including virtual verso and recto pages and a virtual gutter

Full size image

Ebook software originally produced “basic text and image documents” primarily designed for dedicated e-reading devices (Garrish, 2011, p. 6). EPUB, for example, was “always intended to be a general purpose document format” (p. 1). As a result, ebooks were, and to an extent still are, static digitised versions of printed books, or books designed to be read on handheld e-readers. In 2010 there was a sharp increase in the production and sale of tablet computers. This was partly due to the release of touchscreen handheld devices such as Apple’s iPad. According to Garrish, tablet computers offer “visual and aural dimensions” within multifunction devices that are designed for “reading, browsing, gaming, and music” (p. 6). These hardware developments had an impact on digital book design.

Software developers responded to advancements in touchscreen tablet computing by upgrading ebook platforms. As a result of these upgrades, ebook designers could incorporate more subtle combinations of audio and video into their works (Garrish, 2011, p. 6).Footnote 1 Digital books could now contain written text, imagery, animation, video, music, sound effects, recorded audio narration, hyperlinked material, language or dictionary functions, and levels of interactivity. As these changes occurred, multiple media features and interactive functionality became an increasingly important factor in digital book design.

Artefacts can contain different levels of interactivity (Crawford, 2003, p. 6). For example, Red (Sargeant, 2012) is an ebook that includes written text, static imagery, animation, sound effects and low levels of interactivity. When audiences touch or select specific fish (see Fig. 2), the fish move; this movement is accompanied by a brief sound effect. The use of moving imagery, audio and a level of interactivity does differentiate this ebook from fixed format works, such as Diary of a Wombat. It must be said, however, that Red is a book that incorporates a fraction of what is technically and artistically possible within the digital realm.

Fig. 2
figure 2

Red (Sargeant, 2012) is an ebook with sound effects, a minimal amount of moving imagery and low levels of interactivity

Full size image

Ebooks for older readers may contain hyperlinks and hypermedia; these works can collate data, and can connect readers with a wider scope of information than that which is available within printed books. Many ebooks designed for pre- and semi-literate children, however, contain clear skeuomorphic features and low levels of interactivity, as seen in Diary of a Wombat and Red. Traditionally, picture book designers have pushed at the boundaries of what is possible within print technology; in particular they have presented refined combinations of different media and interactive features. Consequently, these book designers may aspire to creating digital works that take advantage of the full range of affordances offered by the digital realm, as opposed to producing ebooks that contain non-remediated content.

What is a Book App?

The term “app” is an abbreviation of the word “application.”Apps are computer software programs that are well suited to delivering high levels of interactive, media-rich content (Kleinfeld and McCoy, cited in Wikert, 2012a, n.p.). Apps can be designed for use on any computational device. They can seamlessly integrate written text, visuals, audio and interaction design.Footnote 2

The term “book app” emerged as a descriptor of a type of application sold within Apple’s App Store. This term operates as a sales category and it is used to describe a type of digital book. Apple has created a range of categories within its App Store, including books, education, entertainment, games and music. The App Store is currently the single largest online market for the sale of apps, it being “where most (though not all) ‘book apps’ are distributed from” (Qadri, 2011a, n.p.). Book apps are primarily designed for touchscreen devices such as mobile tablets and smart phones.

Apple has considerable control over app content. In order for any “app to be distributed anywhere on the App Store, it must first be examined and approved by Apple’s App Review Board” (Qadri, 2011a, n.p.). The review board commonly rejects book apps that have limited interactivity and a lack of media features. Apple states that if an app has “insufficient functionality” or is just a digitised version of a printed book then it will not be approved for App Store sales (Qadri, 2011b, n.p.). Apple has been instrumental in providing not only hardware and software, but also sales categories, terminology and standards relating to app design.

The App Store is a sales steam for iOS apps, the technical term used to describe content that is designed for use on Apple devices. These iOS apps can be constructed from a combination of computer coding languages, the primary one being X-code. These types of apps are not “reflowable”; that is, the content does not automatically resize according to the shape of the screen that is being used. Instead, iOS apps are custom designed to suit the screen dimensions and the resolution of each Apple device. This means that each app may need to be redesigned in order to be compatible with upgraded and newer devices. As a consequence, there are high production costs associated with making iOS apps.

Apps can also be designed using the computer language known as HTML5 (Kleinfeld and McCoy, cited in Wikert, 2012a, n.p.;Wikert, 2012c, n.p.). HTML5 can be combined with other computer languages to create what is known as “web apps.” Web apps are small computer programs that can be compatible with all networked computing devices. Unlike iOS apps, web apps are reflowable (i.e. they do not need to be custom designed to suit the dimensions and resolution of each hardware screen), which considerably reduces the associated production costs. However, current sales streams for web apps do not have the capacity to compete with the App Store; as a result, most apps are produced in iOS format.

Book app designers can take advantage of all the affordances that the digital environment offers; they can create “multimedia, interactive storytelling experiences” (Yokata and Teale, 2014, p. 579). For example, the book app, The Monster at the End of this Book…Starring Grover (Sesame Workshop, 2011), is based on a 1971 printed book of the same name (Stone and Smollin, 1971). This app is not entirely native to the digital realm, yet the designers have remediated the content so that it appears as if it were a digital native. The Monster at the End of this Book does not contain the skeuomorphic features present in many ebooks. It does, however, feature images of a virtual printed book. The protagonist, Grover, functions as a metafictive animated guide, navigating users through the story using a virtual book as the setting. This work does not visually replicate redundant features that have been carried over from print technology; rather, the written text, animation, audio and interaction design work together to reinforce the telling of the story.

Many discussions involving apps and ebooks include the descriptor “enhanced.” Enhancement is a word that has historically been used to describe a specific improvement in software upgrades. Yet this term now appears to characterise a broad variety of detail relating to the digital book. Electronic books have therefore been described as being “non-enhanced text-centric publications,” or as “enhanced interactive publications” (McCoy, cited in Wikert, 2012c, n.p.). The latter reference might point to text-based ebooks that incorporate embedded music or video clips (Kleinfeld, cited in Wikert, 2012a, n.p.) or works that deploy touchscreen and gyroscopic hardware functionality (Yokata and Teale, 2014, p. 579). Given this range of use, the term is not sufficiently denotative to be useful in developing a common understanding of this new landscape.

The lack of clarity surrounding the term ebook has not assisted in establishing a comprehensive understanding of the ways that digitisation has altered the picture book. A book app, for instance, may be considered to be an ebook, in that it is an electronic, or digital book, yet it may not be useful to describe a book app using this contracted term. A clear distinction can be drawn between ebooks and book apps. Ebooks can be digital replicas of printed books; they can contain written text, imagery and low levels of interactivity. Ebooks can also feature hypermedia links that connect readers to a range of related digital data. By contrast, book app designers commonly combine conventions used in printed picture books, game design and animated film. Book apps present a fusion of written text, visuals, audio and interaction design. As books incorporate higher levels of interactivity, the reader, in turn, is likely to become more active. The reader becomes the user: people read ebooks, whereas they use book apps. By differentiating ebooks from book apps in this way, some clarity can be given to the design and analysis of children’s digital books.

Interactivity in Book Apps

Children have the expectation that digital content will be responsive, that it will enable them to actively participate in digital storytelling experiences. Interactivity, according to game design theorists, Katie Salen and Eric Zimmerman, is the “process of action and outcome” (2004, p. 58). The quantity and type of actions required reflect the level of interactivity supplied within a system or artefact. Certainly, with book app content, users can physically interact with it by tapping, touching or swiping the touchscreen, or by tilting or shaking the hardware device itself.

Children’s book apps commonly feature games, puzzles and play activities, which engage users in discrete ways (Sargeant, 2013). For Juul, games are a “rule-based formal system with a variable and quantifiable outcome, where different outcomes are assigned different values, where the player exerts effort in order to influence the outcome, [and] the player feels attached to the outcome” (2003, n.p.). Formal games involve rules and scoring methods, and they provide players with distinct aims and goals. In contrast, puzzles are distinctive in that they “hide a solution, and the player must experiment to divine that solution” (Crawford, 2003, p. 161). Puzzles are seen as less regulated than formal games, yet they are more structured than “play” activities. According to Salen and Zimmerman, play is considered to be “free movement within a more rigid structure” (2004, p. 304); it involves exploratory action. Games therefore, are the most structured, and play is the most flexible, of these activities (Sargeant, 2013, p. 32).

Some book apps, such as Noisy Crow’s Little Red Riding Hood (2013), engage users in nonlinear interactive narratives, which are really puzzles. Users choose, from a variety of paths, which route Red Riding Hood will travel to her grandma’s house. The path selected has an impact on the story’s ending. In making these choices users do not, however, contribute to the app’s content; rather, users reveal a set of pre-designed solutions. In contrast, The Heart and the Bottle (Penguin, 2011) includes a feature that allows users to draw a picture that can then be displayed in a subsequent scene of the book. This is an unstructured play activity that allows users to contribute to the app’s visual content.

The designers of The Fantastic Flying Books of Mr. Morris Lessmore (Moonbot Studios, 2011) have adopted a more expansive approach to incorporating puzzles and play into the narrative environment of a children’s book app. This app contains written text, still images, animation, audio narration, sound effects, a musical score, and a number of puzzles and play activities. Users are invited to play a virtual piano, arrange letters in a virtual bowl of breakfast cereal and solve virtual jigsaw puzzles; in doing this, they engage in actions involving tapping and swiping the screen and “dragging” objects across the screen. This is a highly interactive work and the activities are embedded within the story; yet these activities are only loosely connected to the narrative. Smeets (2012, p. 12) notes that these kinds of interactive book app features are not only likely to entertain users and therefore “increase children’s motivation,” but “may also promote a ‘play-mode’ rather than stimulate children to read the entire story.” These activities may distract users from engaging in the narrative. In this way, the designers of The Fantastic Flying Books of Mr. Morris Lessmore privilege play-based action over story comprehension.

How Far is Up? (Sargent, 2014) is a narrative-based book app that I myself designed. It presents ideas relating to distance and space. It features written text, animation, video, audio narration, music and interactive play activities. The media-rich interactivity in this app seeks to provide users with opportunities to engage more deeply with the thematic content, as the play activities relate to the narrative themes. As Yokata and Teale state, such interactivity should maintain “the integrity of the main story” (2014, p. 580). Thus How Far is Up? allows users to move a toy rocket (see Fig. 3) around a scene by tilting the hardware device or by dragging a finger across the screen. In designing this simple play activity, my aim was to provide users with the experience of moving an object around the space of the fiction. This activity might be engaging but it was not intended to be immersive. It was designed to provide thematically linked, experiential play, allowing users to maintain a connection with narrative content whilst they engage in this simple activity.

Fig. 3
figure 3

How Far is Up? (Sargeant, 2014) is a narrative-based book app featuring written text, animation, audio and interaction design

Full size image

There are many ways in which designers can direct users’ attention to the story itself. Hence the designers of The Monster at the End of this Book have produced some interactive tasks that present users with a restricted number of choices. For example, the protagonist, Grover, binds a virtual book with a rope, as he fears that there is, indeed, a monster at the end of the book. In this way he hopes to stop users from reaching the work’s conclusion. The interaction designers offer just two choices: users may remain on the page, or touch the knots to access the next section of the book. Here narrative flow has been privileged over play, with no distracting puzzles or games. Users can focus on the book’s core theme through engaging with the integrated, compelling interactions presented to them.

Of course, not all book apps contain narratives. Guess Who? At the Zoo (What’s the Fuss, 2011) for example, is a digitised version of an “open-the-flap,” printed picture book. Animals are partially concealed behind visual barriers, which, when the screen is swiped, reveal formerly hidden creatures. Similarly, Animalia (Base Factory, 2013) is an app that invites users to find hidden items within the intricate detail of the illustrations. These activities are also timed, allowing users to accumulate a score.

In summary, the central components of a book app are the written text, visuals, audio and the interaction design. Designers can use animated imagery and interactivity as a means of attracting users’ attention; as research has shown, these facets are effective in engaging young minds (Smeets and Bus 2013; 2014). However, such engagement can sometimes be at the expense of attending to the narrative itself. Interaction designers always need to ask themselves whether their games or puzzles contribute to the story or central theme—or detract from it.

Conclusion: Why Should We Care About the Digitisation of Picture Books?

In contemporary cultures, digital picture books are not only becoming increasingly popular, but also easier to access. A shift has therefore occurred in the way that many children consume literature. It is clear that printed picture books can involve interaction, yet it is apparent that book apps augment the ways that children can interact with their literature. Through animated, interactive digital content, book app designers can expand upon a book’s central theme, engaging users in a more immersive manner. However, as noted, designers can also distract users by including interactivity and animation that is tangential to the narrative or central theme.

While this article has focussed on the general principles involved in children’s digital picture book design, there is a need for more specific research to look at such things as the use of formal games in book apps, showing how the structure of these games may affect children’s engagement with a literary work. In seeking to better comprehend the ways in which designs can shape children’s digital books, we may consolidate and extend the role of the picture book as a valuable cultural tool. Just as print picture book designers stretched the boundaries of print production, digital designers can use the full affordances of the digital realm to present valuable literary works to children.