1 Introduction

Even with increasing demands for computationally savvy workers, there is a dearth of representation of women in science, technology, engineering, and mathematics fields. According to a recent National Center for Women in Information Technology (NCWIT) report, even though 25 % of the current computing workforce is composed of women, Black and Hispanic women make up only 3 and 1 % of this population respectively (Ashcraft and Blithe 2009). Such statistics demand innovative ways to encourage girls to build their computational skills and make a greater impact in these fields.

Virtual environment interactions (VEnvI) is software and curriculum for blending movement and programming, which offers a novel and embodied strategy of engaging 5th and 6th grade girls in computational thinking. In the process of developing this emerging technology, we conduct user-centered design research for creating choreography for a virtual character, promoting motivating and engaging social context, through which girls can be introduced to alternative applications in computing as well as an embodied strategy for learning computational thinking. Computational thinking skills support problem solving in a wide range of fields beyond computer science, including the sciences, technology, engineering and mathematics (Bundy 2007). In this report, we present initial findings of one cycle of the user centered design research in creating VEnvI.

2 Relevance for Learning, Instruction, and Assessment

A growing body of research suggests that cognition is grounded and embodied in the sensorimotor system (Barsalou 2008; Fauconnier and Turner 2008). In other words, the mind and body are inextricably tied. Embodied learning can be an effective instructional strategy in almost any knowledge domain (Johnson-Glenberg et al. 2013) and has been applied to teach mathematics, physics (Lu et al. 2010a), geology (Birchfield and Megowan-Romanowicz 2009), Chinese characters (Lu et al. 2010b), and computational thinking (Fadjo et al. 2009a, b).

In conducting this research, we ask to what extent can embodied pedagogical strategies support computational thinking? Consequently, we adopt the view that computational thinking is a set of concepts (sequences, loops, parallelism), practices (iterating and reusing), and perspectives (seeing computing as a tool for self expression) that draw upon the world of computing, and yet are applicable in many STEM domains (Brennan and Resnick 2012). Further, we utilize movement choreography as both an engaging and a parallel context for introducing computational thinking. For example, sequences (executing one bit of code or movement one after the other) exist in both programming and choreography. Likewise, loops, repeating a set of steps, also occur in both contexts. Compositional strategies in the choreographic process of ordering and reordering movement sequences also mirror computational practices of reusing and remixing. As a result, as students are moving and creating pieces for their virtual characters to perform, explicit connections between what their bodies are doing and computational thinking can be made.

3 Emerging Technology in Practice

Working with students and teachers through iterative analysis, design, development, and implementation cycles (Wang and Hannafin 2005), we are refining and developing guidelines for VEnvI. In line with this approach, rather than immediately constructing a brand new technology, we have relied on existing platforms to conduct pilot work. For our initial iteration, we chose Carnegie Mellon’s Alice, a 3D interactive graphics programming environment (Cooper et al. 2000). In Alice, for a character to lift or turn an arm or other appendage, one must use multiple commands. However, the necessity of excessive commands for a simple move is not ideal. To simplify this process, we created methods within Alice to support basic moves (e.g., arm, foot, and head positions) of an existing character. These positions could be integrated to create more complex movements and then choreography. Yet, the quality of the virtual humans in Alice did not meet the requirements of VEnvI goals in that the virtual human could only be rendered at low quality and appeared unrealistic and incorrect when performing.

To improve upon this shortcoming, in the next iteration we first created a character in Maya, a high-end 3D computer graphics and 3D modeling software package. As we were building the character, an individual piece of geometry had to be modeled at every joint that needed to move. We then moved the location of the pivot point to the location of the joint and changed the way the limbs are controlled. Once this was accomplished, the character was textured (i.e., given skin, hair, clothing and other features) and exported to a format that Alice accepts. Similar to the first Alice character, to actuate this character, various native Alice programming commands (“Loop,” “Do Together,” “Do In Order”) were used to change the orientation of her legs, arms, and head. Users could then choreograph sequences by creating poses and by assembling these poses in an order.

In a five-week pilot during an after-school program, nine students met with instructors and learned a basic curriculum involving the elements of dance, choreography, and Alice programming. At every session, activities in the choreography space were mirrored in the computing space. We began by teaching a popular line dance, the Cha-cha slide, and then had the students recreate it in Alice with the character. Next, the students created a dance based on variations of this initial sequence, and, finally, students choreographed an original sequence, again, replicating it in Alice.

One of our first observations was that students moved their bodies as they figured out how to make the character perform the Cha-Cha slide (see Fig. 1). A typical interaction involved sitting in front of the computer, testing some code, standing to remind themselves of the actions, and sitting back down. Students also wanted to change the avatar’s clothes, ethnicity, body shape, or gender, often replicating their own identifying characteristics. For example, one student reported that her least favorite thing about the software program was that, “You can’t do any physical changes,” while another described the Mii on the Wii™ gaming console wherein players can change clothing, hair, etc. in order to personalize the character. For the new platform design (Fig. 2), we will have a variety of characters for students to choose from that vary in age, gender, body shape, and ethnicity.

Fig. 1
figure 1

Students moving with virtual character on screen

Full size image
Fig. 2
figure 2

In-progress interface mock-up drawing insight from previously developed interfaces

Full size image

Participants also cited the lack of realism (i.e., very mechanical movements). As mentioned previously, the character’s hips, knees, shoulders, elbows, and neck were the only body parts that could be manipulated and each part was an individual, rigid piece of geometry. Moving forward, to create dance motions with the best possible quality, we will enable the user to create choreography based on previously motion-captured dance clips. A 14-camera Vicon optical motion capture system will allow us to accurately record even the small subtleties of a dancer’s movements. Based on the motions of retro-reflective markers attached to the dancer’s body, we will compute the associated joint rotations and transfer the results to a virtual character (Leonard et al., 2013, Daily et al., 2014).

4 Significant Challenges and Conclusions

Our biggest challenges moving forward will be a natural integration of the physical and virtual worlds, keeping rich interactions in a constrained microworld and addressing the limitations of motion capture. As previously mentioned, finding the immediate parallels between computing and dance remains continually vital. Additionally, we have worked with students to develop a sequence as well as to implement their own sequence. In order to sustain engagement with the technology, we will need to find new ways to enhance interaction, including gamification strategies such as unlocking movements based on demonstrated mastery, providing more editing tools as students “level up,” or creating flawed performances that students must debug.

Finally, motion capture’s main limitation is that it does not inherently adapt to new situations. Adapting motion data to specific needs is an important and active research area, and a large body of research has explored how to increase the reusability and flexibility of such motion capture clips. Numerous techniques exist today, such as interpolation, blending, retargeting, move trees, motion graphs, overlays, splicing, displacement maps, and dynamic time warping (Kovar et al. 2002; Witkin and Popović 1995). However, the creation of choreography based on previously recorded clips brings new challenges. For example, if the arms of a character point to one side in the recorded motion and the user wants them to point to the other side, the whole body motion should adapt to the change in balance. In this work, we will design the first control algorithm that links concepts from computational thinking to animation algorithms, thus creating and evaluating new animation algorithms to ensure the quality of the resulting choreography.

The active presentation of concepts and future scalability of the virtual environment will add to the rich landscape of emerging technologies geared towards more inclusive strategies to engage girls in computational thinking. Furthermore, by evaluating different strategies for linking the physical with the virtual world in future iterations that will be conducted in after school programs, we seek to add to the literature on embodied pedagogical strategies in science, technology, engineering, and mathematics fields. In these iterations, we intend to compare embodied with non-embodied strategies to examine the benefits of the approach. In sum, this emerging technology has the potential to widen the scope of current technologies that seek to cultivate computational thinking for diverse designers, users, and audiences.