Keywords

1 Introduction

Recent years saw the development of several play environments for children that integrate interactive technology. Grounded in the constructivist paradigm, new approaches focus on interaction design that incorporates body and space. In the human-computer interaction (HCI) context of playful learning environments, bodily interaction has been revitalized and research on evaluating this interaction has highlighted important benefits in enhancing meaning-making, exploration, and collaboration in children’s play [1,2,3]. However, further research is still needed to articulate ways of bringing children back into the places of their daily environments. There is a reciprocal relationship between physical activity and the activity setting [4]. Most digitally-enhanced play environments are based on specific interactive objects and the isolated play to be performed with them [3, 5, 6]. Especially, for indoor play, these environments target a structured form of play that rules and goals are predefined.

To address these shortcomings, our study aims to support children’s bodily experiences by integrating the physical and digital environment through intuitive interaction modalities. In a previous study, we presented the theoretical framework for designing digitally-enhanced environments for children’s play to support their engagement and active exploration in their everyday places through objects [7]. Based on that framework, this paper presents a scenario to interact with digital environments from within the physical world through unique feedback to children and to invite them to intervene in the flow of their play. The crucial question has been how to use the features of children’s surroundings to integrate physical and digital environments.

This paper introduces the design, development, and assessment of the “Monnom” prototype as a digitally-enhanced play environment. The prototype integrates the physical environment and the digital environment into the child's experience in order to enable children to transform a place physically via giving a symbolic meaning to objects around while actively engaging with the daily environments. For this purpose, we present a qualitative study with children playing with Monnom to explore how interaction with digital environments through tangibles affects children’s play-action patterns, the creation of their own play space, and social interactions. Below, we analyze four children's experiences as being representative of variations of children’s play and activities. The analysis highlights the potentials of blended environments, a combination of digital and physical media. To conclude, we derive a set of qualities to think about interactive design for play and future research. Unless otherwise stated, all images and figures in this article are the first author’s own creation.

2 Background

In the context of designing an environment for play, it is important to understand the role of the play environment [8, 9]. Research on designing places for children has shown, there is a direct connection between play and the physical environment. Children’s indoor and outdoor play differ physically and socially related to the play actions that happen there [9, 10]. One of the reasons for this difference is that indoor and outdoor settings have different types of equipment and materials [10, 11]. In outdoor settings, natural landscape environments provide a setting for open-ended play with various conditions such as wind, sun-shadow, rain and provide greater space and freedom of movement for children. Thus, children frequently engage in the exploration, manipulation of natural materials, and locomotion. In indoor settings, objects become more important for children’s play and children frequently engage in conversations with peers through playing with objects.

Frost [12] describes three different forms of play that are particularly relevant with physical activity and the environment during play: functional play, constructive play and symbolic play. Functional play is play that involves full-body activities such as running or climbing a tree. Constructive play involves building things, such as building a shelter by collecting materials or manipulating parts to build. Symbolic play involves pretend activity which allows children to construct alternative worlds by role-playing. Henniger [13] shows that preschool children prefer constructive play in indoor settings and functional play in outdoor settings due to the different toys and equipment that were available in each setting.

With regards to interactive digital environments for outdoor play, most approaches focus on location-based surface design [6]. On the other hand, for indoor play environments, most digital environments are based on specific interactive objects [2, 3]. For the purpose of this paper, we look for a way of using digital technology to support interaction both with the physical and digital environment. We design a blended play environment called Monnom to support (a) different forms of play, (b) spatial interaction with their surrounding, (c) social interaction. Monnom invites children to interact with the digital environment through using features of the child’s surroundings.

3 The Prototype: Monnom

Monnom, a digitally-enhanced physical environment offers body-object interaction where the body frames the scene, controls, and improvises the play. This system does not require the user to wear any physical device. It is able to employ various tracking systems, and thus allows easy integration of nearly any object into the virtual world. Moreover, physical objects are used, not only for haptic interaction, but as controllers, and to drive the interaction with the digital world. Children can use Monnom individually or as a group.

The whole installation consists of colorful objects, a vertical surface where the interactive-digital content is projected, a bounded area where the children can play, a projection, a webcam for collecting data (in front of the screen to perceive children's play with objects), a computer with Monnom software for data transmitting and transforming the interactive digital content (see Fig. 1). Two fixed video cameras record children’s activity.

Fig. 1
figure 1

Technical infrastructure of Monnom

Full size image

We use vertical surfaces for a display to provide children’s own composition in the digital environment synchronously by allowing the reciprocal connection between the digital and physical environments. Children, with their eyes on the two-dimensional display, move around and add objects to the physical space while the system captures size and color information of objects through body movements, analyses and interprets compositions with predefined patterns. The technology inside the system allows children to make their own spatial composition in both physical and digital environments, they can resize patterns in the digital world by moving with objects in the physical world.

Here, we report on children’s interaction with “Monnom”. The present exploration is part of a larger study aimed at evaluating user experience in Monnom. The purpose of the current analysis is to demonstrate how digital augmentation supports children’s play through activating objects.

4 Study Setup

We conducted user studies with children in two different settings, one at a museum and one at a school. In order to optimize the installation conditions for Monnom, we first tested its prototype in a multi-purpose room on our campus, Istanbul Technical University, Architecture Faculty and four children played with Monnom. The main goal was to try and decide how we should install our system. We then assessed the prototype in studies with 67 children (4–12 years old) in two different settings. These studies included unique features such as thematic patterns and background images related to the settings at a museum and a school (Fig. 2).

Fig. 2
figure 2

Research settings and sample outcomes

Full size image

Our study protocol was approved by the Istanbul Technical University Committee On the Use of Humans as Experimental Subjects. For the first study, the museum (Istanbul Modern) made an announcement about the workshop named for two different age groups. Each group was limited to 20 participants. It was an open public event that families could apply to without any precondition. Participants were informed by the museum about the study and permission procedures. 15 children participated in the first group (aged 7–8) whereas 14 children participated in the second group (aged 9–12). For the second study, the school (ITU Vakfi Özel Sedat Üründül Kindergarten) wrote an invitation letter to parents seeking individual permissions. Parents of 38 children (aged 4–5) agreed to have their children participate. They gave written informed consent prior to the start of the study, and on the day of the study, the children assented to participate. All children had the opportunity to play with Monnom for about 10 min.

Children’s activities and the digital content that children created were recorded with video cameras. Recorded documents were stored on computers that were password protected and accessible only to the researchers. Each session was digitally filed with the name and date of birth (month-year) of the participating children. For any publication of the work, any visuals and information that could identify the individuals were omitted, e.g. faces of the children in the images and videos were blurred. In research documents, children were identified by age and a letter code rather than by name. Audio recordings were kept only as transcription and confidentially.

5 Data Collection Method

The primary form of data collection was video recording and notes taken based on the play session observations. With reference to this data, we considered how interacting with the digital environment through physical objects influences children’s play, spatial interaction with their surrounding and peer communication. In post-study viewing of the videos, any interesting interaction, a special play event or anything notable was noted down and labeled with the time it happened, duration, number of participants, age group and gender of the involved children for each play event. We defined behavioral codes to illustrate patterns related to:

  • Play actions

  • Spatial interaction (use of space, engagement with object)

  • Peer communication.

Based on the detected patterns, one case study was analyzed with focus on four children's experiences as being representative of diverse children’s play activities. We then performed an in-depth transcription by focusing on the behavior of each child during the play session. These transcriptions were performed by annotating data from both the video recordings and verbal interaction.

6 Observations

Here we report our study of one of the nine groups in the school case study. The school prepared a group of 4 or 5 children for each classroom. We designed play patterns in two themes according to the school program in order to create a shared and familiar interest among the children: Autumn and Space (see Figs. 3 and 4). For each theme, we created eight patterns and used four colors. We also created an additional theme to provide children a theme out of the school context: Sea (see Fig. 5).

Fig. 3
figure 3

Patterns for the Autumn Theme

Full size image
Fig. 4
figure 4

Patterns for the Space Theme

Full size image
Fig. 5
figure 5

Patterns for the Sea Theme

Full size image

We designed the soft objects, i.e. pillow and blanket, that are familiar to children’s daily environments (see Fig. 6). We intentionally made their shape simple for easy physical manipulation. These objects were in four different colors.

Fig. 6
figure 6

Objects and their dimensions (cm)

Full size image

The session began with the introduction of Monnom. We made a demo to help the children understand the system. We showed how to change, place and resize patterns in the digital world by playing ourselves with different colored objects and by moving around with them in hand. Then the children decided on the theme, selected the play patterns which we then imported into the software. Afterwards play time started.

The group in the study results we present here consisted of four girls. Three were 4 years old and classmates whereas the fourth was 5 years old. Together, they selected autumn as the theme and decided on the patterns (see Fig. 7). They matched the cloud with blue, birds with yellow, the kite with red, and leaves with green.

While the size of the pillow and blanket affected the size of the pattern in the digital environment, children also resized them by coming closer to the camera or stepping back. Each child took on a different role in the session.

Fig. 7
figure 7

Patterns that children chose together

Full size image

Child 1: planner

Once the play session started, the first child C1 quickly picked up objects and built a place on the floor. And then the second and third child participated. They prepared a place together for “sleep” by using pillows and blankets as they should be in the sleeping environment. When their interaction with Monnom started, the first child tried to collect all blue objects. After she picked up the big blue pillow, she made big blue clouds on the screen. With this initial exploration, she began to create patterns with different colored objects. She was not specifically exploring variations in physical movements, but she used her actions to systematically test out cause-and-effect relations until she tried and observed all colors and their patterns. She then went back to her initial location where they made a place on the floor, and she continued to build her place without looking at the screen. After this point, her play pattern mainly focused on collecting all objects, trying to keep them together, and creating her own place via spatial transformation using objects. She also communicated with the second child who participated at some points. She started to give directions to her, e.g. “let’s sleep”, while observing others’ play.

Child 2: player

Child C2 started the play by trying to take the big blue pillow from C1. At this point, the C3 helped them negotiate by bringing the small blue pillow. After trying to create patterns with that object, C2 stopped and observed the screen for the effect of her actions. Later she tried other small pillows. It was difficult to observe the effects of her movement because she always chose small objects. She also observed her peers playing. After trying a couple of times, she participated in C1′s play on the floor, and she helped her build a place for their collective play.

Child 3: problem-solver

At the beginning of the play, C3 picked up the blue blanket. At that moment, she realized that the first and second children had a problem sharing a big blue pillow. In order to solve this problem, she found and gave another blue pillow to the second child. Then she began to create clouds with the blue blanket. After the initial trial, showing a clear understanding of the effects of different colored objects, she began to have a strategic approach. She collected four pillows of the same size but with different colors and created different patterns with them. Instead of focusing on resizing patterns, she focused on creating different patterns on the different places of the screen. Thus, with every new object, she also changed her position.

Child 4: competitor

Child C4 was from a different class and did not speak to the other children during the entire session. She started with the red blanket and physical play activities, such as walking, jumping. When she moved closer to the screen and showed the entire blanket, she created the biggest pattern. She observed that in the front position while jumping, she was able to control her creation on the screen. She also explored the effects of her bodily movements on resizing patterns. After this observation, her activity mainly focused on competition for creating the biggest pattern and exploring variations between different colored objects and their effects on the digital environment.

7 Results

Interacting with the digital environment through tangibles invites children to experience different interaction paths (see Figs. 8 and 9). Children tended towards different actions based on their play patterns and familiarity with one another. For all, the seamless integration of the physical and digital world in Monnom allowed easy moves from a physical environment to a digital environment. With the prototype, the children immediately understood its basic idea—picking up differently colored objects for selecting patterns in the digital environment and resizing and relocating the pattern by moving in the physical space. This simplicity allowed children to make their own rules. Over time, children seemed to collect objects for creating their own palette and learned to control the size and the location of the pattern as they watched the results of their movements.

Fig. 8
figure 8

Children engaging in multiple play patterns during playing with Monnom

Full size image
Fig. 9
figure 9

Childrens’ creation during the play session

Full size image

In Monnom, the body becomes the vantage point and frames the scene, while the eye controls and improvises the play. We have observed that with feedback from a digital environment, children can fictionalize the meaning of that space by moving around and adding objects to the physical space. For example a child picked the blue object and drew clouds, and she said: “I have so many clouds in my sky” while showing physical surroundings. Then she went to the object box to take a yellow object, she said “let’s add birds here” while jumping around pretending to be a bird.

Since the system allows multiple players, children learned different interaction paths observing other actions. They talked about which color produces which pattern on the screen as well as their exploration of different actions with different objects together. In particular, the visibility of actions via the screen in Monnom helped children to acknowledge others’ perspectives and encouraged negotiation. This feedback allowed children to influence and react to the behaviour of the system and also observe others’ action.

Beside simplicity, feedback and multiple-player approach, Monnom invited children to engage diverse play patterns through dispersed quality. Instead of designing a special interactive object, the system involves multiple objects without requiring any digital equipment. During play, children expand their play beyond the digital interface, and the objects serve as both tools of digitally represented narration and props for physical play. For instance, the first child focused on creating patterns on the screen, later she focused on making a physical place for her play by using the same object.

8 Conclusion

In the context of designing a digitally-enhanced environment for play, it is necessary to ask how we can design conditions to facilitate children’s different play patterns while supporting bodily experiences. Our study with Monnom explored different interaction paths children followed us in a blended environment. The broader analysis is ongoing but a few preliminary insights expand the understanding of using digital technologies for play. The current study can provide an initial set of qualities for designing blended play setups: simplicity, feedback, multiple players, dispersed. Furthermore, by documenting the differences in children’s interaction paths, the study highlights the importance of designing for diversity by offering affordances that support children’s active engagement and exploration. The prototype is responsive only to predefined objects’ features. Future work aims to make this blended system accessible directly from children’s daily environments and open for children to use their own daily objects instead of determined play objects.