Keywords

Introduction

Artifacts—cultural objects embedded in social practice—do not cease to fascinate scholars of human cognition and development. As design-based researchers of educational media, the pedagogical artifacts we investigate are historically young technologies. Nevertheless, we view these novel artifacts from the same theoretical perspectives as we would a seemingly humdrum manual tool. Specifically, we ask: What educational gains can such an artifact foster? What can it teach us about human learning?

Yet for the purposes of this particular chapter we are less interested in material artifacts such as a piano or an abacus; neither are we presently concerned with symbolic artifacts such as musical notes or numerals. We focus, instead, on embodied artifacts—the cognitive products of rehearsed performances or trained routines,Footnote 1 such as the capacity to play Für Elise or manipulate an abacus. As we shall argue, novel motion-sensitive cyber-technologies (e.g., Nintendo Wii) are uniquely geared both to craft and leverage embodied artifacts as means of fostering learning and, for researchers, opening a window into how learning occurs.

To illustrate and elaborate the construct of an embodied artifact, which will be central to our thesis, we begin by taking the readers on a guided tour of a few decidedly low-tech instantiations. For the sake of clarity, we initially focus on embodied artifacts within explicitly embodied domains. Later in the text, we will introduce a mathematical, technology-embedded embodied artifact.

To begin, imagine a first surfing lesson in Honolulu, Hawaii. Despite the endless crowds at the sun-drenched Waikiki beach, a neophyte surfer is eager to get in the water. Doing so immediately, however, is likely to invite disappointment. His inability to distinguish the many types of waves, crowding by dozens of other nearby surfers, neuromuscular fatigue from continuous paddling, an uneasy sense of unspoken social hierarchies among more experienced surfers, and a myriad other factors large and small all conspire to quickly dizzy and exhaust the novice. Yet the beachboys (surfing instructors) of Waikiki are famous for claiming they can make anyone ride a wave—at least, that is, for a second or two. How?

Before getting in the water, the beachboy will ask the first-timer to lay down upon the surfboard on the sand. There, the beginner is taught the elementary sequence of Stand Up (SUFootnote 2) on the surfboard, roughly: (1) kneel; (2) one knee up; and (3) stand up. Only once the beachboy determines the neophyte is capable of executing this basic sequence with confidence does the surfer take to the water. There, the instructor will wait for an appropriate wave, a selection process beyond the novice’s capacity, then push his charge into said wave at the appropriate moment. At this point, all the neophyte must do is paddle hard into the wave—and (attempt to) execute SU. A complex activity is thus partitioned into: (1) select a wave; (2) approach a wave; and (3) SU. Hence the beachboys accomplish their claim of getting anyone to surf by performing (1) and (2) on behalf of their charge and having given the neophyte an embodied artifact, the elementary Stand Up sequence (3).

Note that the function of an embodied artifact is modular, in the sense that it can be taught and learned as a standalone sequence of operations, yet later it can be contextualized into a larger system as well as refined via analysis into component parts. For example, the learner becomes more adept at timing, instigating, and performing SU in respect to his distance to the wave (contextualization via integration—recall that the SU sequence was learned on sand); and learns the optimum placement for his knee during the kneeling portion of SU (refinement via analysis). We therefore arrive at disciplinary competence by entering at the level of actions in the form of rehearsed performances. In other words, embodied artifacts serve as entry into disciplinary engagement—as knowledge through practice (cf. Ericsson, 2002) and reflection (cf. Dewey, 1933; Schön, 1983). Importantly, the learner may rehearse operatory elements of this modular action (SU) independently of any larger activity system (surfing).

Because they are modular and thus portable, embodied artifacts tend to be adaptable in their application. Consider the Flying Sidekick (FS; see Fig. 1b), an aerial attack historically used to strike over ground fortifications (e.g., defensive spikes) and dismount fighters off of warhorses and other beasts of war. In modern times, neither mounted warriors nor spike-barricades pose a serious concern, yet FS continues its existence as more than a text-bound technique. The flying sidekick was practiced for centuries in martial arts halls concurrent to, yet independent of, its combative application: due to its modular nature, it survived the disappearance of its original context, mêlée warfare. Nowadays, FS continues its existence primarily as a test of a learner’s discipline and body-mastery.

Fig. 1
figure 1

Embodied artifacts in practice: (a) A novice surfer and his coach (seated); and (b) a Flying Kick demonstration by a Soo Bahk Do Master

Full size image

These two brief examples are meant to illustrate some of the variety of embodied artifacts. While embodied artifacts may work in tandem with other artifacts (as in the case of surfing, operating an abacus, or playing the piano), they may merely require space and gravity (such as dance, see Fig. 2a). So, what does this have to do with learning? The critical common thread is that all embodied artifacts are rehearsed performances, ready-to-hand cultural equipment created by “packaging” procedures for skillfully encountering particular situations in the world (cf. Rosenbaum, Kenny, & Derr, 1983, on motor learning via “chunking”).Footnote 3 Indeed, as we have defined them, embodied artifacts, by mediating one’s encounters with the world, constitute an integral part of cultural and individual development. First, humans embody cultural procedures through participating in social activities. Through observation, demonstration, imitation, and training, these cultural procedures become our resources in the form of embodied artifacts. Therefore, through embodied artifacts we store cultural knowledge in the body, using the body as both the material for and means of encountering the world (cf. Dourish, 2001; Dreyfus & Dreyfus, 1999).

Fig. 2
figure 2

Embodied artifacts take many forms: (a) Traditional Cham dancers. (b) Mathematical Imagery Trainer (MIT) in use by two 10-year-old students, with the tutor (center) prompting and monitoring their problem solving

Full size image

As learning scientists, we are interested in the role of embodied artifacts in the emergence of disciplinary competence, particularly disciplines traditionally viewed as “pure” in the sense of independence from the physical world, such as mathematics.Footnote 4 Our interest is twofold. First, as we elaborate in the next section, current empirically supported theories of mind suggest that embodiment—having and using a physical body in the world—is fundamentally linked to all reasoning, whether involving “pure” thought or getting one’s hands dirty (literally or figuratively). Second, we hold that deliberate use of embodied artifacts in mathematics instruction may render hitherto undetectable learning processes open to both formative assessment in classrooms and empirical scrutiny in laboratories. The idea is simple: if students must perform physically in the service of doing mathematics, then such doing becomes publicly observable rather than hidden away “in their heads.”

So, what does this have to do with technology? In addition to our practice as learning scientists, we are designers of pedagogical artifacts. As designers, we are interested in availing of novel technologies to engineer learning environments in which students appropriate embodied artifacts in pursuit of mathematical competence. We then observe students engaged with our design and, hopefully, we learn more about the process of learning (see Collins, 1992 on design-based research as educational science). So doing, in turn, we also learn more about designing learning environments. And on it goes. This chapter is, then, a design-meets-theory-meets-design piece on embodied artifacts and educational technologies.

We begin with observations about the pedagogical potential of embodied artifacts in light of increasingly ubiquitous motion-sensor technologies; these observations, in turn, form the theme of the following section, where we situate our study in the broader context of research on the role of embodiment in human learning and knowing. From the perspective of educational design, we consider the following question: How, if at all, may novel motion-sensor technologies be pedagogically utilized, particularly in light of recent advances indicating the fundamental role of embodiment?

Taking on this question, we present a proof-of-existence educational intervention that leverages cutting-edge technology, namely the Mathematical Imagery Trainer (hence, “MIT,” see Fig. 2b). Working with the MIT for Proportion (MIT–P), students move their hands in an environment that changes its state in accord with the ratio of the hands’ respective heights, effectively training an embodied artifact of moving the hands in parallel and at different rates, that is, proportionately to each other, with the distance between the hands increasing. Students then reflect on, analyze mathematically, and articulate this spatial–dynamical embodied artifact and then contextualize it as a particular case of proportionality.

Finally, we broaden our discussion to present a particular type of educational design, embodied interaction. This type of design, we argue, is ideally suited to foster embodied artifacts in a powerful way towards normative disciplinary competence and, furthermore, enables researchers a window into conceptual development. We then contextualize our arguments by presenting a case of embodied interaction design that suggests how mathematics education and embodied artifacts may be systemically linked in practice.

Theoretical Framework

The Rise of Embodied-Cognition Theory and Its Application to Mathematics Education

Can conceptual understanding emerge from embodied interaction? One answer is that we are physical beings living in a physical world; hence, attempts to understand the development of conceptual thought need look to physical, sensory interaction. Yet this answer appears naïve and, perhaps due precisely to its apparent simplicity, has been ignored by cognitive science throughout the last century. Traditional cognitivist views partitioned mundane interaction into three mutually exclusive constituent facets: perception, thought, and action (e.g., Fodor, 1975; Tulving, 1983). Thinking, or concepts, thus intervenes between perception and action and is characterized as distinct from those real-time embodied processes by token of being symbolic–propositional. Yet in alternative views discussed below, cognition is not secluded or elevated from perception and action but is rather embedded in, distributed across, and inseparable from these corporeal processes.

Embodiment studies rose fast in prominence towards the end of last centuryFootnote 5 through the converging efforts of numerous pioneers in fields as disparate as robotics, psychology, philosophy, and computer science (Brooks, 1991; Gibson, 1979; Varela, Thompson, & Rosch, 1991; Winograd & Flores, 1987). Though many of these perspectives initially emerged in opposition to then-prevalent symbolic architecture models of the mind, embodiment studies have, over the last few decades, burgeoned into a vast area of investigation in their own right—replete with a spectrum of proponents. Within this spectrum, we can roughly identify conservatives, who cautiously posit that reasoning may be connected with some aspects of non-corporeal cognition (e.g., Dove, 2009); moderates, who argue that physical action underpins or forms the substrate of cognition (Barsalou, 2010; Goldstone, Landy, & Son, 2010; e.g., Sheets-Johnstone, 1990); and radicals, who hold that cognition itself is merely another action (e.g., Melser, 2004). Indeed, the scope of embodiment studies has grownFootnote 6 to the point where scholars concern themselves defining what, exactly, it means to be “embodied” (Kiverstein & Clark, 2009).

In our current work we tend to hold with those who favor the middle ground, and we interpret available empirical evidence as indicating that physical action indeed undergirds thinking, including so-called “abstract” thinking (e.g., thinking about the word antepenultimate, or solving for x). We are therefore not concerned by the controversy over what role corporeality plays in thought: indeed, it gives us something to do. A fortiori, as interaction designers of mathematical learning we find ourselves in a unique position to contribute toward resolving this theoretical controversy.

Particularly relevant to our work, embodiment has been presented as a useful framework for theorizing processes inherent to “abstract” disciplinary mastery, including mathematics learning and reasoning (Abrahamson, 2009a; Campbell, 2003; Namirovsky, 2003; Roth & Thom, 2009). One consequence of this view is that observations, measurements, and analyses of physiological activities associated with brain and body behavior can provide insights into lived subjective experiences pertaining to cognition and learning in general, and mathematical thinking in particular. In a strong form, we conjecture that physical action is neither epiphenomenal nor merely supportive to “pure” mental activity. Rather, conceptual understanding—including reasoning about would-be “abstract” contents such as pure mathematics—emerges through and is phenomenologically situated and embedded in actual and simulated perceptuomotor interactions in the world.

Technology for Using the Body

Even as cognitive scientists recognize this essential role of the body, industry has made dramatic advances in engineering technological affordances for embodied interaction. At the time of this writing, Nintendo Wii and Playstation Move players worldwide are waving hand-held “wands” so as to remote-control virtual tennis rackets; iPhone owners are tilting their devices to navigate a virtual ball through a maze; and Xbox Kinect users are controlling video-game avatars with their bare hands—activities hitherto confined to the realms of futuristic fantasy, like flying cars. Moreover, innovative designers tuned to this progress are constantly devising ways of adapting commercial motion-sensor technology in the service of researchers and practitioners (Antle, Corness, & Droumeva, 2009; Lee, 2008). As such, media that only recently appeared as esoteric instructional equipment will imminently be at the fingertips of billions of potential learners. We are excited about the prospect of using these new media to create learning environments centered on embodied artifacts that may be rehearsed and consequently investigated via mathematics, allowing an embodied entry into this disciplinary domain. In the remainder of the chapter we document our attempts to utilize these capabilities and what we have learned doing so.

Learning as Performance: Appropriating Artifact-Bound Conceptual Systems

Our work in the Embodied Design Research Laboratory involves the design, testing, and refinement of pedagogical artifacts as well as the development of theoretical models of learning via interaction with said artifacts (Abrahamson, 2009b; Abrahamson, Gutiérrez, Lee, Reinholz, & Trninic, 2011). The work we present here is subpart of Action-Before-Concept (ABC), a cluster of cross-disciplinary studies of performance in mathematics, music, climbing, and the martial arts centered around relations between procedural and conceptual knowledge. ABC, writ large, explores the relation between performance and knowledge. It is an inquiry into cultural precedence for pedagogical practice within explicitly embodied domains (e.g., martial arts), wherein procedures are initially learned on trust yet subsequently—only toward perfecting the procedures toward mastery and further dissemination—are interpreted by experts as embodying disciplinary knowledge. The results of these inquiries within explicitly embodied domains are then leveraged in investigations of, arguably, implicitly embodied domains, such as mathematics. In practical terms of design, much of our work consists of creating learning situations where (bi)manual performances culminate in the learner’s guided reinvention of disciplinary knowledge (cf. Freudenthal, 1983). These performances take form as concerted dynamical coordination of embodied, material, and symbolic artifacts.

Thus we espouse a position that learning is the residual effect of engaging artifacts as means of accomplishing one’s goals (cf. Salomon, Perkins, & Globerson, 1991; Vérillon & Rabardel, 1995). Yet against the backdrop view of learning as imitating, internalizing, and appropriating the elders’ artifactual actions (e.g., Vygotsky, 1987), we foreground the pedagogical philosophy of learning-as-discovering these artifacts’ horizons in the course of explorative problem solving and theory building (e.g., Karmiloff-Smith & Inhelder, 1975). The challenge for us as designers lies in taking this position on learning and making it a product, that is, designing a pedagogical artifact that encapsulates our theory of learning and respects current embodiment-informed theories of mind. Our response to this challenge is addressed in the following section.

Instructional and Experimental Design

Embodied-Interaction (EI) is a form of technology-supported training activity. By participating in EI activities, users encounter, discover, rehearse, and ultimately investigate embodied artifacts.

A general objective of EI design is for users to develop or enhance cognitive resources that presumably undergird specialized forms of human practice, such as proportional reasoning. As is true of all simulation-based training, EI is particularly powerful when everyday authentic opportunities to develop the targeted schemes are too infrequent, complex, expensive, or risky. Emblematic of EI activities, and what distinguishes EI from “hands on” educational activities in general, whether involving concrete or virtual objects, is that EI users’ physical actions are intrinsic, and not just logistically instrumental, to obtaining information (Kirsh & Maglio, 1994). That is, the learner is to some degree physically immersed in the microworld, so that the embodied artifact—instantiated in finger, limb, torso, or even whole-body movements—emerges not only in the service of acting upon objects but rather the motions themselves become part of this learned cultural–perceptuomotor structure. EI is “hands in.”

Before describing our design, it is useful to mention two related designs to illustrate the present scope of this emerging field to the reader. Antle et al. (2009) used EI to leverage participants’ embodied metaphors of “Music is a physical body movement” as a means of developing fluency with music creation. Another EI design (Fischer, Moeller, Bientzle, Cress, & Nuerk, 2011) used digital dance mats in design intended to improve kindergarteners’ fluency with relative numerical magnitude. As EI technologies become increasingly ubiquitous, we anticipate an exponential growth in EI designs catering to various educational needs (see also Birchfield & Johnson-Glenberg, 2010).

Embodied Interaction Design: Mathematical Imagery Trainer

Our overarching design conjecture is that some mathematical concepts are difficult to learn because mundane life does not occasion opportunities to embody and rehearse their spatial–dynamical foundations. Specifically, we conjectured that students’ canonically incorrect solutions for rational-number problems—the misapplication of additive reasoning and procedures to multiplicative situations (see Lamon, 2007 for an overview)—indicate students’ lack of appropriate dynamical imagery to ground proportion-related concepts (see also Pirie & Kieren, 1994).

In addition to theories of embodiment, this conjecture is grounded in our previous work. In particular, a pilot study by Fuson and Abrahamson (2005) suggested that children’s cognitive difficulties in understanding proportional reasoning may be related to their difficulty in physically enacting proportion. Namely, when asked to enact scenarios involving proportional growth of plants—e.g., “If a rose grows twice as fast as a tulip, can you show me what that looks like with your hands?”—children manually demonstrated a “fixed difference” misconception; that is, they tended to raise their hands while keeping the distance between them fixed. The similarity between this physical “fixed difference” performance and aforementioned, presumably conceptual, “fixed difference” mathematical solution suggested a possible relation between the two.

Accordingly, we engineered an EI computer-supported inquiry activity for students to discover and rehearse the physical performance of a particular proportional transformation of our design. This activity, we reasoned, should train students’ physical proportional skill via allowing them to experience “fixed difference” as contextually inappropriate. Let us step back and elaborate on this central design principle.

We define conceptual performances as embodied artifacts that are recognized by their enactors as physically inscribing essential semantics and syntax of corresponding mathematical content. For example, ‘adding’ gestures—such as bringing hands together as if amassing stuff into a single location—are well suited for signifying the arithmetic operation ‘addition,’ because the gestures objectify and manipulate imagined quantity sets. As such, the objective of embodied-interaction mathematics learning activities, per our framework, is to foster student development of customized embodied artifacts that subsequently—through symbolic instrumentation and regulating discursive interaction—emerge as conceptual performances. That is, embodied artifacts become conceptual performances once they have served as semiotic resources for discussing, and thus signifying, target curricular content. Our solution to this general design problem is the Mathematical Imagery Trainer (MIT),Footnote 7 which the following section further explains (see Fig. 3).

Fig. 3
figure 3

MIT in action: (a) a student’s “incorrect” performance (hand height locations do not match a 1:2 ratio as measured from the table) turns the screen red; (b) “correct” performance (right hand at approximately twice the height of the left hand, as measured from the table, forming a 1:2 ratio) keeps the screen green

Full size image

We wish to emphasize that we arrived at building the MIT–P technology only after having considered a variety of “low-tech” design solutions. We feel privileged to be designing in an age where we can expand on the vision of luminaries such as Froebel or Montessori by using available media to expand everyday experience.

Technical and Interface Properties

Our instructional design leverages the high-resolution infrared camera available in the inexpensive Nintendo Wii remote to perform motion tracking of students’ hands. In our setup, an array of 84 infrared (940 nm) LEDs aligned with the camera provides the light source, and 3M 3000X high-gain reflective tape attached to tennis balls can be effectively tracked at distances as great as 12 ft. Later iterations used battery-powered, hand-held IR emitters that the students point directly at the Wii camera. The Wii remote is a standard Bluetooth device, with several open-source libraries available to access it through Java or .NET. Our accompanying software, called WiiKinemathics, is Java-based and presents students with a visual representation on a large display in the form of two crosshair symbols. Further details on technical (Howison, Trninic, Reinholz, & Abrahamson, 2011) and interface (Trninic, Gutiérrez, & Abrahamson, 2011) properties can be found elsewhere.

The orientation of the 22″ LED display (rotated 90 degrees and aligned to table height) and the responsiveness of the trackers are carefully calibrated so as to continuously position each tracker at a height that is near to the actual physical height of the students’ hand above the desk. This feature is an attempt to enhance the embodied experience of the virtual, remote manipulation (Clinton, 2006).

In practice, the MIT measures the heights of the users’ hands above the desk. When these heights (e.g., 10″ and 20″) match the unknown ratio set on the interviewer’s console (e.g., 1:2), the screen is green. If the user then raises her hands in front of this “mystery device” by proportionate increments the screen will remain green (e.g., raising by 5″ and 10″ to 15″ and 30″, thus maintaining a 1:2 ratio) but will otherwise turn red (e.g., raising by a equal increments of 5″ to 15″ and 25″). In other words, the embodied artifact of the MIT–P activity is the continuous physical articulation of all the pairs effecting a green screen. From this perspective, the initial purpose of the MIT–P is to train a particular proportion-relevant embodied artifact of Bimanual Proportional Transformation (BPT, see Fig. 4). As SU in surfing, BPT constitutes an activity whose meaning is situational.

Fig. 4
figure 4

The Mathematical Imagery Trainer for Proportion (MIT-P) set at a 1:2 ratio, so that the right hand needs to be twice as high along the monitor as the left hand in order to make the screen green (a “success”). Schema of student paradigmatic interaction sequence—while exploring, student: (a) positions hands “incorrectly” (red feedback); (b) stumbles on a “correct” position (green); (c) raises hands maintaining constant distance between them (red); and (d) corrects position (green). Compare (b) and (d) and note the embodied artifact constitutes different distances between the hands/cursors

Full size image

Participants, Protocol, and Data Analysis

Participants included 22 students from a private K–8 suburban school in the greater San Francisco (33% on financial aid; 10% minority students). Care was taken to include students of both genders from low-, middle-, and high-achieving groups as ranked by their teachers. Students participated either individually or paired.

Interviews took place in a quiet room within the school facility. Students participated either individually or paired with a classmate in semi-structured clinical interviews (duration: mean 70 min.; SD 20 min.). In addition to the interviewer, typically at least one observer was present, whose duty included taking written notes in real-time, crewing the video camera, and assisting in operating the technological system.

Study participants were initially tasked to move their hands so as to find a position that effects a green screen and, once they achieved this objective, to keep moving their hands yet maintain a continuously green screen. That is, the participants needed to discover a means of enacting a green-keeping performance that the technology interprets as a transformation of two values sustaining an invariant ratio, such as 1:2. In a sense, the MIT offers students a pre-numerical “What’s-My-Rule?” mathematical game. The protocol included gradual layering of supplementary mathematical instruments onto this microworld, such as a Cartesian grid (see Fig. 5, below). Hence, once the proportional-transformation dynamical image is embodied, semiotic resources (mathematical instruments) and discursive support (the tutor) are present for it to be mathematically signified, elaborated, and analyzed.

Fig. 5
figure 5

The Mathematical Imagery Trainer: (a) overview of the system featuring an earlier MIT version, in which students held tennis balls with reflective tape. (b) 5b through 5e are schematic representations of different display configurations, beginning with (b) a blank screen, and then featuring a set of symbolical objects that are incrementally overlain onto the display: (c) crosshairs; (d) a grid; and (e) numerals along the y-axis of the grid (in the actual design, the flexible grid and corresponding numerals were initially set by default as ranging from 1 to 10)

Full size image

The interview ended with an informal conversation, in which the interviewer explained the objectives of the study so as to help participants situate the activities within their school curriculum and everyday experiences. Finally, the interviewer answered any questions participants had, with the objective that they achieve closure and depart with a sense of achievement in this challenging task.

Our investigation of the empirical data—field notes and videography—was conducted post hoc in the leisure of the laboratory as collaborative, intensive micro-ethnographic analysis of participants’ conceptual ontogenesis (Schoenfeld, Smith, & Arcavi, 1991). Microgenetic analysis is a research methodology, typically applied to video data, where study participants’ presumed cognitive trajectories are interrogated and modeled via analyzing their moment-to-moment behaviors, essentially actions, interactions, and multimodal utterance. This methodology is emergent and iterative, in the sense that the researchers’ insights from specific events inform successive waves of scrutinizing the entire data corpus. Importantly, microgenetic analysis enables us to maximize the theoretical significance of our work (see Yin, 2009 on analytic generalizability).

Findings

General Findings

We began the chapter by way of introducing the notion of embodied artifacts as well as their function in learning. We also mentioned the increasingly ubiquitous motion-sensor technologies that utilize users’ bodily movements. Next, we explained our work at the intersection of these recent theoretical and technological advances, namely designing educational technologies that leverage embodied artifacts in the service of teaching the chronically challenging mathematical concept of proportion. Finally, we are in the position to summarily present some of our findings so as to provide evidence for the feasibility of this design-based research program. Presently we provide some general findings across all students and then focus on a case indicative of the struggles and insights encountered by them all.

Importantly, all students succeeded in devising and articulating strategies for making the screen green, and these initially qualitative strategies came to be aligned with the mathematical content of proportionality. This particular finding serves as a proof-of-existence supporting the conjecture that embodied artifacts such as Bimanual Proportional Transformation create pedagogical opportunities to support student learning of targeted mathematical concepts. Naturally, there existed minor variations in individual participants’ initial interpretation of the task as well as consequent variation in their subsequent trajectory through the intervention protocol. However, the students progressed through similar problem-solving stages, with the more mathematically competent students generating more strategies and coordinating more among quantitative properties, relations, and patterns they noticed. We now elaborate on the learning trajectory.

Each student began either by working with only one hand at a time, waving both hands up and down in opposite directions, or lifting both hands up at the same pace, occasionally in abrupt gestures. They realized quickly (<1 min. on average) that the simultaneous actions of both hands are necessary to achieve green and, consequently, that the vertical distance between their hands was critical, although at first they viewed the distance between their hands as fixed. We found this default “fixed distance” approach of importance, as it arguably matches an enduring (mis)conception where students see 2/3 as “the same” as 4/5 (for both the numerator and denominator values respectively increased equally). Indeed, our hope was that by uncovering and addressing such conceptions physically, we could elicit and treat students’ pre-numerical conceptual reasoning underlying their arithmetic competence.

The following sequence of insights into problem-solving the MIT-P compiles our observations based on real-time notes and close analysis of the video data from all study participants’ interactions. Each step corresponds to students' “successful” or “correct” physical articulations with the MIT-P (that is, “making green”) and consequent verbal articulations of what it is they are doing. The numerical example case will be a 2:3 ratio.Footnote 8

Student discoveries:

  1. (a)

    The actions of both hands are necessary to achieve green.

  2. (b)

    Green is achieved by positioning the hands at particular stable locations.

  3. (c)

    The critical quality for achieving green is a type of relation between the hands’ relative positions.

  4. (d)

    These positions can and should be reinterpreted as magnitudes—the distance between the objects or their respective heights above a common base line.

  5. (e)

    The distance between the hands in correct (green) pairs is not constant—it will necessarily change between correct pairs.

  6. (f)

    This distance should increase as the pair’s height increases (and vice versa).

  7. (g)

    Moving from one correct position to another can be achieved by increasing the hands’ heights differentially, for example, for every 2 units the left hand rises, the right hand should rise 3 units (or the distance between the hands should grow by 1 unit from move to move)—a recursive rule for iterated transitions.

  8. (h)

    The multiplicative relation within each pair—for example at 4 and 6 units the right hand is 1.5 times higher than the left hand—is also a constant across correct pairs.

  9. (i)

    One and the same number pair (e.g., 2 and 3) expresses three aspects of the interaction: for example 2 and 3 units are the lowest correct integer pair of heights, raising the left hand by 2 units for every 3 raised by the right hand will result in another correct location, and 2/3 or 3/2 is the constant within-pair multiplicative relation.

In brief, students were given the initial opportunity to practice the embodied artifact BPT (Bimanual Proportional Transformation) amathematically. Gradually the protocol encouraged integrating BPT within the broader world of proportional mathematics and providing mathematical tools for analyzing and expressing it in mathematically normative ways (see Fig. 5). As such, BPT gradually instantiated the practice of “proportional reasoning.” Similar to the Waikiki surfer who embodied, utilized, contextualized, and refined SU, the students in our study integrated BPT into the broader world of proportional mathematics as well as analyzed its component parts and, so doing, displayed an emerging mastery of the mathematical concept of proportion. The following excerpt provides supporting evidence of this gradual emergence.

Excerpts from an Empirical Study

Shani was a 5th-grade female student identified by her teachers as “low achieving.” During the exploratory phase of the interview, as Shani attempted to discover a means of making the screen green (refer to earlier Fig. 5b, c), she stumbled upon the embodied artifact.

Shani: [excitedly] Oh! Is it about the distance between these two [pointing to hand-held devices]?

Thus Shani, similar to all our participant students, noticed that an embedded property of the interaction, the distance between her hands, was associated with the desirable feedback. She articulated the “farther-up–more-apart” strategy, that is, the distance between the hands should increase with the hands’ elevation in order to effect green (see Item f. in the list of discoveries, above). Once we overlaid the grid on the screen (see Fig. 5d), Shani discovered the “a-per-b” strategy, by which the hands rise at different yet constant intervals (see Item g.). When we next introduced the numerals (see Fig. 5d), Shani initially availed of them as mere location markers rather than quantitative indices. In particular, she used the numerals to recite the respective locations of her left and right hands, as she iteratively scaled the hands up the screen at 1-per-2 quotas: “One and two, two and four, three and six, and four and eight.” Even though the “doubling” multiplicative relation within each of these number pairs is quite striking, Shani was oblivious to this relation. Indeed, it took a gentle suggestion by the interviewer.

Interviewer: What else can you say about those numbers? One and two …

Shani: [continuing] One and two, then two and four, three and six. Hey wait. Um, oh, it’s … [fidgets, becomes animated] It’s all doubles! The bottom number, like time … times two is the top number. [motions at monitor] We had, like, one and two, then three and six, then, um, then four and eight, then five and ten.

Prior to the introduction of the grid, Shani’s articulation of the embodied artifact was based on the qualitative relation of “farther up” and “more apart,” yet once the grid and numerals were introduced, she instrumentalized them so as to analyze the embodied artifact BPT, rendering the description quantitative. Yet this was not a straightforward process—it is not the case that Shani noticed the green pairs and immediately saw them as proportionally related. Rather, her observation emerged through interaction with numerals, which she initially used merely to mark green locations.

Shani continued to discover new properties of the situation through appropriating symbolic artifacts as means of better enacting her strategy. In the following transcription, she responds to the interviewer’s request to recount her recent findings, and in so doing she notices a relation between recursive (1-per-2) and multiplicative (double) strategies:

Shani: Then … this one [indicates right hand] is always going up by two, and this one [indicates left hand] is going up by one, which would mean that … that, uhm, this one [right-hand side] is always double this [left-hand side].

Shortly after, Shani accomplished what we believe was an important shift from discrete proportion to continuous proportional reasoning.

Shani: Wait a minute. A while ago you asked me, uhm, how many green there are. It could really be infinite. Like, because, if it is really all about the distance between them [the hands]—which is, like, I think it is, because it’s getting darker depending on that—uhm, then it really doesn’t matter where on the screen it is.

We would argue that this level of reasoning is surprisingly sophisticated for a fifth grader—particularly a student labeled by her teacher as “low achieving.”

Eventually, Shani coordinated quantitative reasoning with a qualitative feel of “faster.”

Shani: So this one [indicates right controller] should be … So my right hand should be moving faster. So that it can make … be going up two spaces on the grid … while the other one is only going up one.

Note how Shani’s embodied experience with the green-making artifact supported her coordination between rate and speed, just as the embodied artifact supported her leap from discrete to continuous reasoning in the previous excerpt. Like the novice surfer, Shani used the embodied artifact as a means of gaining entry to a novel activity—in her case, proportion. Her actions, initially amathematical, became mathematically meaningful, a conceptual performance.

Discussion

Epistemic, Cognitive, and Pedagogical Features of Embodied-Interaction Design

One of our design challenges rested on leading students via an embodied artifact towards a conceptual performance without explicit instruction. In the MIT–P activity, this is accomplished via the automated feedback “green,” which is triggered whenever the user’s bimanual action matches the ratio setting on the interviewer’s console interface. The meaning of “green” evolved throughout the activity, and this evolution captures the process of embodying the dynamical artifact as well as integrating and refining it, as follows.

Green: (a) began as the objective of the “Make the screen green” task; (b) soon became feedback on the perceptuomotor activity, as the users attempted to complete this task objective, thus shaping the emerging embodied artifact; and finally (c) came to function as a conceptual placeholder by grouping a set of otherwise unrelated hand-location pairs sharing a common effect of “green.” As such, “green” formed, sculpted, and refined the embodied artifact Bimanual Proportional Transformation (BPT), so that BPT—similar to ancient dance or martial arts forms perpetuated across generations—inherited, instantiated, and preserved a cultural practice. Ultimately, once users determined the activity’s mathematical rule and recognized its power for anticipating, recording, and communicating BPT, this embodied artifact became situated within the larger practice of proportional reasoning.

Learning Is Where the Action Is, Then Down to Operations and Up to Activity

We offer a preliminary account for the emergence of conceptual knowledge from performance as seen in our data. We have found Leontiev’s (1981) account of activity useful, and here we modify it to suit our needs. In brief, Leontiev proposes that social activity has a hierarchical structure with three distinct levels; the activity level, the action level, and the operation level. Activities consist of actions; actions, in turn, consist of operations. A typical example goes: building a house (activity), fixing the roof (action), and using a hammer (operation). While the levels are somewhat flexible, the basic message is that every activity consists of some number of actions; each action, in turn, consists of some number of operations.

Our current conjecture is that that learning from others happens at the middle level of action in the form of embodied artifacts. As an action becomes an embodied artifact via deliberate training, the learner may analyze her activity, moving “down” to the level of operations and refining those. Furthermore, through participation in discourse broadly construed and observing the embodied artifact in various contexts, the learner comes to understand the larger framework and how the activity integrates within it. That is, she moves “up” toward contextualization. For example, the students in our study practiced the embodied artifact BPT and then mathematically analyzed it by articulating its constituent physical operations with semiotic resources of the discipline. Even so doing, learners generate various observations connecting BPT to their existing knowledge (sometimes appropriately and sometimes not), and, in dialogue with the instructor and each other, come to see the activity and the various ways of mathematically treating it as “cases of” proportion. Thus the initially modular action becomes a conceptual performance.

Conclusion

Throughout this chapter we have been threading together two central themes. First is that movement matters. Physically interacting in a physical world is our mode of being and the roots of our thinking. This thread, then, dealt with the relation between performance and knowledge: namely, we interpreted existing embodiment studies as suggesting that conceptual understanding—including reasoning about would-be “abstract” contents such as pure mathematics—emerges through and is embedded in actual and simulated perceptuomotor interactions in the world. We introduced the construct of an embodied artifact as a means of articulating how cultural practices are “packaged” and “given” to learners, enabling their entry into the world of skillful action and, furthermore, disciplinary competence.

The second theme is that recent decades have witnessed advances not only in theoretical models of embodiment but also remote-interaction cyber-technologies, yet critical questions have remained unanswered regarding the interaction of the two. It is in embodied-interaction (EI) design that our two themes meet. We introduced EI as a form of physically immersed instrumented activity geared to augment everyday learning by crafting embodied artifacts targeted towards specific disciplinary practice, such as proportional reasoning. In pursuing these problems, our strategy has been to engage in conjecture-driven cycles of building, testing, and reflecting on these two themes. The current text aimed to share our conviction that EI offers unique affordances for teaching mathematical concepts via cultivating the conceptual performance of embodied artifacts.

To the extent that mathematics-education researchers and practitioners take seriously the grounded-cognition thesis, the community should pay far greater attention to the somatic substrate of subject matter. Students’ perceptuomotor manifestations as they engage in learning activities could be far more than mere support for, or communicative visualization of essentially abstract notions. On the contrary, notions become abstracted only through bodily incorporation. In fact, the grounded-cognition approach suggests that there need not be any tension at all between concrete and abstract ideas, because intrinsically embodied mathematical notions can transcend local contexts.

We anticipate that, when coupled with recent cyber-technological advances, EI stands to become a focus of design for and research on mathematical learning. As our work indicates, EI activities serve as highly useful empirical settings for research on the ontogenesis of mathematical concepts and, more generally, relations between performance and knowledge in mathematics education. These immersive activities create opportunities for design-based researchers to observe and help resolve tension between theoretical conceptualizations of: (a) unreflective orientation in a multimodal instrumented space, such as riding a bicycle or playing pong; and (b) reflective mastery over the symbol-based re-description of this acquired competence, such as in mathematical numerical forms.

We hope this line of investigation will contribute to developing a model of embodied mathematics instruction. Researchers could look to diverse cultural–historical forms of physical performance, such as music, dance, and the martial arts, as ethnographic entries into traditional and indigenous pedagogical acumen. The skills inherent to these cultural practices might, at first blush, be viewed as aconceptual and, as such, hardly bearing on mathematical reasoning and learning. Yet as recent theoretical and empirical work, including our own, suggests, our shared biology implies that even the most abstract of mathematical concepts may first be embodied, then verbally articulated, and finally reified in conventional semiotic forms. Such issues are more than academic, for all too often proverbial lines are drawn in the sand regarding the importance of “conceptual” knowledge versus “procedural” performance (e.g., see Schoenfeld, 2004 on “math wars”). Yet corporeal actions performed in the context of disciplinary activity constitute vital aspects of cognition and knowledge (cf. Alač & Hutchins, 2004; Kirsh, 2009, 2010), so that knowledge is developed, elaborated, and expressed as situated conceptual performance. In our future work, we will continue to investigate the embodiment of mathematical concepts through the reciprocal efforts of developing theories of embodied learning and designing educational technologies.