Keywords

1 Storytelling and Embodied Interaction

Vygotsky and Bandura [5] stated that the physicality of words and gestures should also be considered as types of tools for interaction for children. They believed communicating with other people was one method to help a child develop. A picture book is one example of a tool that helps children to gain verbal ability through communication with adults. It has been confirmed that children are able to learn languages from adults through picture books [1, 2]. Moreover, according to Bruner [6], the phenomenon of joint attention is how children display empathy for others. Bruner’s infant study suggested that from an early age, children actively use their bodies to engage with others. Thus, collaborative interaction plays an essential role in the child’s development for the child to understand themselves and others. Therefore, storytelling events at libraries and kids’ spaces are beneficial to a child’s development as they encourage collaborative interaction. However, the primary way that children explore the world using their bodies is through touch. As a result, educators, such as Frobel [3] and Montessori [4], attempted to demonstrate how interacting with “things” was essential for child development. Frobel [3], for example, founded a kindergarten that used the “Gifts,” or educational play materials, to encourage children to learn while playing freely with objects. Montessori [4], similarly, developed teaching tools that trained a child’s senses. Reggio Emilia [7] took this notion further by introducing the importance of creativity and developed a preschool education method that stressed physical learning through art expression. Iverson et al. [9, 10] and Antle et al. [11] also showed that exploratory activity that allowed a child to use their body influenced the depth of their understanding. By utilizing human physicality, especially one’s tactile sense, the brain is more actively stimulated, which can help comprehension. Therefore, many museums have incorporated hands-on [8] activities to encourage a deep understanding of exhibits, particularly exhibitions related to cultural properties. Already there are many studies in which tactile technology has been adapted to children’s education. Several of which focus on storytelling and reading. Yannier [12] et al. and Zhao et al. [13] verified that children reading with a vibrotactile tablet could achieve a deeper understanding of the content they were reading. However, a limitation to these studies was that they focused on individual or one-on-one readers. Through this research, we hope to explore how to focus children’s interest in picture books in a group reading setting. During storytelling time, at public spaces like the library or kid space, children of various backgrounds listen to a story together. However, there are usually differences in age, the ability of understanding, and the interest level of the children. Also, there is a field problem that it was difficult to grab children’s attention all of them because children’s attention was different when long storytelling. As a result, selecting a picture-book that is suitable for everyone is a difficult task. In this study, we aimed to design experience to grab children’s attention to picture books when multiple children listen to storytelling together. In the next chapter, we did fieldwork to explore the issue of a typical kindergarten setting and to find what interaction needed to make the system. Based on this result we developed a cushion that multiple children can feel picture book world through their body by haptic feedback same time. And then we conducted experiment to verification the effective of this system, and we analyzed children behavior to use behavior psychology method. Finally, we discussed what value the embodied experience of picture book could provide to children.

2 Design

2.1 Fieldwork and Proposed Method

We interviewed stakeholders, such as storytelling staff, to explore the issues of a typical kindergarten setting (The Froebel-Kan staff who worked in kindergarten and public library staff). Also, we observed children playing in a kid’s space operated by Frobel-Kan as well as the storytelling activity provided by them. As a result, we found that staff tended to continually choose the same books because finding books that will interest all the children was difficult. Also, the books selected were one’s that sold many copies as this aligned with the books parents were choosing to read to their children. In other words, there was a lack of diversity in the range of picture book experiences being introduced to the children. Additionally, based on our observations of the kid’s space, fewer children were interested in the picture books than those that played with the play equipment and toys. Furthermore, during the storytelling activity, a large number of children got bored immediately, as opposed to those who retained interest in the picture book. Thus, we proposed the following system to get children interested in the story of picture books as in Fig. 1.

Fig. 1.
figure 1

Concept design of “Kinder BURU BURU Cushion”

Full size image
  1. 1.

    Improvement of Interest Through Embodied Experiences with Picture Books

    As shown in previous studies, children use their bodies to explore objects and phenomena. Such learning methods have proven to be successful for expanding interest and deepening understanding. We thought it is possible to use such methods during storytelling in the kindergarten. To make children think about the story beyond the words on the page, we needed to help them embody the actions of the story.

  2. 2.

    Explore Different Viewpoints Through Embodied Interactions Together

    We thought if children can share their interests with each other, this would help expand their perspective of others. Allowing them to enjoy picture books more collectively. We decided to use a cushion, because if it were a wearable device, the operational costs would be too high. By simply sitting on the cushion it was possible for multiple users to experience the tactile sensations of the picture book all together.

2.2 Hardware Design and Haptic Design

We designed the configuration to satisfying our concept as in Fig. 2. We used a PC or tablet device uploaded with the tactile data for each picture book, and then transmitted the tactile data with a transmitter. The tactile data was received by multiple receivers and multiple Vibro-transducers (Acouve Co., Ltd./Vp6) operating all at same time. We created the transmit module and receive module. These modules could send and receive tactile information without cables by using wireless audio unit PLL synthesizer (Circuit Design, Inc./WA-TX-03, WA-RX-03). We also created the Haptic amp, and equipped it with a battery to eliminate cable. We then created corresponding haptic experiences for five picture books published by Froebel-Kan Co.,Ltd. In creating our haptic experiences, we did haptic design in the following process: First, we made haptic design storyboard to consider listener’s perspective (subjectivity or objectivity). Second, we selected vibration from instrument sample data in Abltone (Ableton Co., Ltd.). For example, in the scene where a bear hit a tree, we used instruments such as drum kicks. We mainly used percussion instruments samples and rhythms to explain the body sensation of character. (Footsteps of characters, feeling of character knocked on the door etc.). Because percussion instruments could feel vibration clearly by our system. Finally, we adjusted the volume of all the contents to balance the vibrations, especially we strengthened data below 20 Hz. We aimed to put at least one sensation on each page to prevent the children from getting bored.

Fig. 2.
figure 2

System design and implementation

Full size image

3 Validation

We conducted experiment to verification the effective of this system. Drew et al. [17] reported haptic feedback can draw children’s attention and conversation about the sensation explain in picture book between parents and children. Therefore, we thought if children get information through the sensation, children’s behavior like attention to the picture book and sitting on the cushion will change. For our study, we compared storytelling using our “BURU BURU” cushion and normal storytelling. And we analyzed children behavior to use behavior psychology method. The test was performed in four groups over two days (am/pm), and we analyzed what changes occurred in the children’s behavior.

3.1 Experimental Method

Participants

Participants were recruited using flyers in a kid space operated by Froebel-Kan Co., Ltd. For the first test, we recruited a total of 10 children and their parents, 6 children in the morning (3 boys and 3 girls, aged 3 years) and 4 children in the afternoon (2 boys and 2 girls, aged 3 years) on July 23th 2019. In the second test, we recruited a total of 21 children and their parents, 10 pairs of children and parents in the morning (4 boys and 6 girls, aged 3–4 years) and 11 pairs of children and parents in the afternoon (6 boys and 5 girls, aged 3–4 years) on October 19th 2019. Participants were paid 3,000 JPY per hour for their participation. The total 31 children (average aged: 43.4 months) were used for the analysis, excluding those who were absence or attendees who joined in midway. Additionally, parents were asked to complete a questionnaire: “How many picture books do you read a week?” to help us gauge the participating children’s existing level of familiarity with picture books. The majority, 21 people, answered that they read picture books every day. On the other hand, two participants stated that they rarely read picture books. The remaining eight people said they read picture books a few days a week. Overall, many of the children who participated were familiar with picture books.

Procedure

Testing was performed in a kid’s space operated by Froebel-Kan Co., Ltd. We had children sit on the cushions as picture books were read aloud. We alternated between normal storytelling and storytelling with the system (see Table 1). We switched the order (with and without) between morning test and afternoon test to prevent preference bias due to the ordering. Also, we told parents to not stop a child should they decide to move off the cushion. Children were free to come and go as they wanted. Between the second and third storytelling we put in a brief exercise play break. In total, four picture books were read per group. For each test we captured each child’s face on video.

Table 1. Summary of procedure
Full size table

Video Coding and Depend Variables

Video Coding began when the reader opened the picture book and started to read then continued until the picture book completed. The video was divided into 15 s increments to elicit the rate of how often a set of variables occurred during that timeframe. We measured five different variables (attention to picture books, looking at mother, pointing at picture books, utterances, sitting on the cushion.): Attention to picture books: It was defined as the child looking at a picture book for 15 s. Looking at mother: It was scored when the child looks at the mother at least once every 15 s. Also, pointing at picture books and utterances was scored when these actions occurred at least once every 15 s. Utterances: it was scored that children talk about a picture book. Sitting on the cushion: It was defined as the child sitting on the cushion for 15 s. Coding was performed with the assistance of people not familiar with the contents of the experiment (hereinafter referred to naïve). The test on July 23, 2019, was analyzed by three people (author and two naïve) while the test on October 19, 2019, was analyzed by seven people (author and six naïve raters). And then we examined the agreement value between authors and naïve observers: attention to picture books of agreement value average is 88% (minimum/maximum values from 55% to 100%), looking at the mother is 96% (min/max values from 61% to 100%), pointing at picture books is 100% (min/max values from 91% to 100%), utterances is 96% (min/max values from 75% to 100%), and sitting on the cushion 95% (min/max values from 28% to 100%), all of the agreement value average maintained at least 85% or higher.

3.2 Result

Figure 3 shows average data of the percentage of 15 s intervals with Attention to picture books, sitting on the cushion, Looking at mother. We analyzed for haptic feedback factor (with vibration and without vibration) and order factor (the first half and the second half) using an ANOVA in each different variable. We found an interaction between haptic feedback factor and order factor in attention to picture books [F (1,30) = 9.272, p = .005]. There is no measurable effect of haptic feedback [F (1, 30) = 0.057, p = .813] and order [F (1,30) = 2.420, p = .130]. We noticed children’s attention to picture books reduced in the second half when there was no haptic feedback. Also, there was significant interaction in looking at mother [F (1, 30) = 4.937, p = .034]. There is no main effect of haptic feedback [F (1, 30) = 0.080, p = .779] and order [F (1, 30) = 2.564, p = .120]. Looking at mother increased in the second half when there was no haptic feedback. Regarding sitting on the cushion, we noticed it decreased in the second half regardless of haptic feedback (first half vs second half), [F (1, 30) = 15.498, p < .001]. Also, utterance had the same result (6% first half, 3% second half with haptic feedback condition and 7% first half, 4% second half, without haptic feedback condition), [F (1,30) = 5.094, p = .031]. Additionally, most children did not point to the picture books. Only four children pointed to picture book during the vibrotactile sessions, and this behavior occurred only once during the normal storytelling. Finally, we got feedback from their parents by using a free-form questionnaire about differences in the child’s behavior from the parent’s perspective. Table 2 is categorized feedback from parents.

Fig. 3.
figure 3

Percentage of 15 s intervals with attention to picture books, sitting on the cushion, looking at the mother. Error bars is Standard Deviation

Full size image
Table 2. Examples of feedback comments from parents
Full size table

4 Discussion

We concluded the embodied experience with picture books using haptic feedback can reduced the declined children’s attention to picture book when long storytelling with multiple children. Because, children’s attention to picture books decreased in second half when normal storytelling condition but there was no change when with haptic feedback condition. Previous studies by Andreson et al. [14] that children watched TV-content a longer time when they were concentrating. Ruff et al. [15] defined this concentration “Sustained attention”, it is an important ability when they learn something in a school environment. Weighrt et al. [16] reported that continuous TV-contents helped to language acquisition, it can say that focus attention to subjects affects the child’s process of knowledge. The value of “seating on the cushion” decreased second half time in both conditions. Because, it is conceivable that the children were listening to the picture book for a longer time than usually storytelling for the experiment. But Fig. 4 shows it recovered slightly with haptic feedback condition in the third book and fourth book. Also, we could observation that children who began to walk around kid space at third book normal storytelling, but they returned to their seat when fourth book by haptic feedback because children noticed that cushion was vibrating. In other words, it was possible to recover children’s attention by haptic feedback. Additionally, this system might be more effective for children who normally have little ability to pay attention to picture books as it showed a significant shift in the child’s behavior. Also, we could get these comments from parents that “she gets bored of long stories, but I confirmed that she was concentrating because she thought it was fun (3 years 8 months old girls and mothers).” And yet another, “She can’t listen to storytelling too long at kids’ spaces, but today she listened to four books and I was surprised (though during the third one she seemed a little tired) (3 years 8 months old girl and mother).” These comments support our results. We predicted the children’s behavior that looking at mother occurred when children wanted to tell their mother about a picture book. It was occurred when first half and second half. However, it increased in the second half with no feedback. It was also observed that children then went to their mother after performing this behavior. Therefore, it seems that the behavior has two meaning: a sharing picture book interest and an expression of boredom. It is need to more detailed to measure these behaviors separately.

Fig. 4.
figure 4

Compared to order with or without haptic feedback

Full size image

5 Conclusion

In this study, we aimed to design experience to grab children’s attention to picture books when multiple children listen to storytelling together. Then we developed the Kinder “BURU BURU” cushion which multiple children can experience the world of picture books using their body. And we investigated the effects of our system using haptic feedback to use behavior analyzed method. As a result of experiment that we confirmed that our system can reduced the declined children’s attention to picture book when long storytelling with multiple children. However, we found that the effect of the group experience was not enough to variables the children looking at mother. During the experiment, there were situations where children looked at each other who participated with friends. It is the possibility that more behavior will lead to joint attention. In the future, we would like to more clarify the effects of a group experience to storytelling, our system will be using continuous such as the environment where children know each other. Also, we believed that this method can helped other group work like elderly activity.