What’s all the fuss about gestures? A commentary

Abstract

While reading the articles assembled in this volume, one cannot help asking Why gestures? What’s all the fuss about them? In the last few years, the fuss is, indeed, considerable, and not just here, in this special issue, but also in research on learning and teaching at large. What changed? After all, gestures have been around ever since the birth of humanity, if not much longer, but until recently, not many students of human cognition seemed to care. In this commentary, while reporting on what I saw while scrutinizing this volume for an answer, I will share some thoughts on the relationship between gesturing and speaking and about their relative roles in mathematical thinking.

1 On enthusing about gestures and the questions it raises

One immediate, self imposing answer to the “What’s the fuss” question is that the upsurge in attention came in response to the recent technological advances, which made gesturing researchable. Before the camcorder became a standard ingredient of a researcher’s toolkit, one did not have any reliable empirical data to play with while trying to capture the gist of these ephemeral phenomena. Today, helped by the tools that turn the transient into permanent, the investigator may revisit a given body movement as many times as she feels necessary and can look at even the most short-lived of gestures for as long and as inquisitively as she desires.

But the argument “I do because I can” does not seem to be telling the whole story. Far from being a local phenomenon, restricted to mathematics education or even to education at large, the interest in gesturing crosses disciplinary boundaries and brings together researchers from fields of study as different as psychology, brain science, and linguistics. Listening to declarations of those who have plunged into this burgeoning domain, we immediately notice this one thing that unifies the otherwise heterogenous gesture community: the deep conviction that without getting more knowledgeable about gesturing, we have little chance to arrive at a satisfactory understanding of human thinking. And thus, one team of contributors to this volume claims that “gestures … are … genuine constituents of thinking” (Roth and Thom), another describes gestures as “the very texture of thinking” and as “important sources ... of abstract thinking” (Radford), yet another posits that gestures, along with speech and inscriptions “jointly support the thinking processes of students in a unitary way.” (Arzarello, Domingo, Ornella, and Sabena). In making all these claims, the authors join the leading gesturologists of our day, and thus, David McNeil, whose inspiration came from linguistics and psychology, views human thinking as an intricate dialectic between speech and imagery, whereas “[t]he imagery in question is embodied in the gestures that universally and automatically occur with speech” (McNeill 2005, p. 4). Similarly, the renowned expert on sign languages, David Armstrong, argues together with his colleagues that “concept formation is evolutionary and anatomically linked ... to perception which, itself, depends on an organism’s movements and gestures” (Armstrong, Stokoe & Wilcox 1995, p. 53).

In addition, the special issue authors are unanimous in their belief that there is an intimate relationship, indeed symbiosis, between gestures and language. Radford speaks about “the sophisticated linkage of language and the perceptual, auditory, tactile, and kinesthetic sensorial channels,” both of which are claimed to be vitally important for full-fledged mathematical thinking. According to Nemirovsky and Ferrara, utterances, these basically linguistic constructs, include nevertheless “multimodal aspects such as: facial expression, gesture, tone of voice, sound production, eye motion, body poise, gaze, and so forth;” and Arzarello, Domingo, Ornella, and Sabena devote their research to “the complex intertwining among gestures, speech and inscriptions in learning mathematics.” These statements fully resonate with McNeils’ claim that “gesture and language are two parts of one entity” (xii).

The gestures emerge from this collection of articles triumphant, their role in thinking upgraded from that of an auxiliary device for emotional fine-tuning to that of “the very fabric” of human cognition. It is not surprising that the authors are excited about their subject; and yet, their rather intemperate enthusiasm may actually be hurting the case. Fully focused on gestural communication, the articles fail to make a clear distinction between the roles of speech and that of gesturing in constituting and sustaining mathematical discourse.¹ The lack of differentiation between these two rather dramatically different modes of communicating is likely to lead the reader to conclusions the authors themselves would find difficult to accept.

Of course, we have more than one reason to claim that speaking and gesturing are not equal shareholders in the business of thinking and communicating. The most immediate, albeit indirect, argument is that talk, being unique to our kind, is the primary suspect whenever one inquires about the sources of human uniqueness. A more direct evidence against putting speaking and gesturing on a par is the obvious fact that eliminating gestures, as is the case in phone conversations, email exchanges, and in other types of written exchanges, does not reduce the overall complexity of the interaction and leaves intact the majority of the unique features of face-to-face interaction. This said, it is the example of email that helps us realize that there are things one can “say” or think with gestures which have no linguistic counterpart—thus the emoticons, which are supposed to make up, if only partially, for what is missing.

It is time, it seems, for a deeper reflection on the differing ways in which speaking and gesturing contribute to the uniquely human forms of being in the world, including mathematics. Let me immediately clarify that while saying this, I do not mean the mere fact that language and gestures function and contribute in widely differing ways and with differing effect—this is trivial. I am also not trying to put them in a competition. My interest is in the question of what and how each one of these two modalities makes possible, how the unique contributions of each modality draw on the unique properties of the medium, and why combining speech and gesture brings such an obvious synergetic effect.

In the rest of this commentary, my aim is to engage the authors of this special issue in a conversation on the above questions. I will do this by putting forward the following conjecture: The property of self-reference (or recursivity), unique to human languages, plays the decisive role in making us able to transcend the concrete and to proceed to ever more advanced levels of mathematical abstractions. In this process, gestures and other visual mediators constitute the material of which the abstractions (e.g., mathematical objects) are produced, one layer after another. Some terminological clarifications are in order before I can substantiate this claim and discuss the ways in which it supports or contradicts positions of other contributors to this volume.

2 Glossing the conceptual foundation

Clearly, to address the question of relation between language and gestures, one needs to be explicit about the way these two terms are used. To investigate the relative contributions of the two modalities to mathematical thinking, it is also necessary to make explicit what is meant by the term thinking at large, and by abstract thinking in particular. I will now try to answer these lexical queries beginning, in each case, with a brief survey of what the contributors to this volume have to say about their use of the word.

Language

A scrutiny of the papers has shown that none of the authors made explicit what he or she meant by language. In the literature, in general, the use of the word is fraught with ambiguities. In his influential recent book, McNeill (2005) admits to be using the term language in two ways: “First, I use ‘language’ in a technical–linguistic way, to refer to those static structures of language consisting of grammar, words, etc.... Second, I use ‘language’ in a traditional nontechnical way, to indicate what it is we know when we say we ‘know language’ or what we ‘use’ when we speak, listen, read, etc.” (p. 21). Echoing McNeil, Armstrong and his colleagues criticize the common uses of the term² as all too restrictive: “We tend to consider ‘linguistic’ what we can write down, and ‘nonlingusitic’ everything else; but this division is a cultural artifact, an arbitrary limitation derived from historical evolution” (McNeil 1985, p. 352; quoted in Armstrong et al. 1995, pp. 7–8). Without explicitly defining language, they talk about ‘language-cum-syntax’ (p. 18), thus signaling the importance of grammar as the central property of those languages that are of the greatest interest for students of human communication. In this spirit, I take syntax to be the defining property of language, thus eliminating those collections of signs that do not form a well-defined grammar-driven system. I will thus be using the word language as denoting a communication–mediating symbolic system with rules for creating permissible elements (meaningful expressions) from those previously constructed (see also Sfard 2008, p. 101). The advantage of this definition is that it does not rely on the nature of the symbols and thus does not restrict language to what can be said or written.

Gestures

The only definition to be found in this volume describes gestures as “spontaneous ‘movements of the arms and hands… closely synchronized with the flow of speech’” (Edwards, quoted from McNeil 1992, p. 11)’. This description may be deemed as both too restrictive (why hands only?) and too broad (why any hand movement? should uncontrollable nervous trembles also count as gestures?). For the sake of the present discussion, I will thus tighten the conditions of use and say that gesture is a body movement fulfilling communicational function. According to my reading, this definition is in tune with the use of the word made by the contributors to this volume.

When one confronts this last description with the definition of language formulated above, one realizes that the terms gesture and language, being products of differing categorizations, should not be counterpoised to one another. Although this is not the way gestures are used in this volume, we need to remember that they can be organized into a system with syntax, that is, into language. This is best evidenced by the American Sign Language (ASL). Moreover, language is a tool for communication, whereas gesture, as used in this volume, is an actual communicational action. It seems therefore that the proper verbal counterpart of gesture would be utterance ³—an act executed by emitting a series of sounds, the communicational effectiveness of which originates, among others, in how these sounds are related one to another. Communicational act (or discursive act) is the term that will be used in the rest of this commentary to signify the combination of utterance and gesture constituting the basic unit of face-to-face interactions. “”Of course, utterances and gestures do not always come together and each one of them can constitute a communicational act on its own. However, the authors of this volume are interested mainly in those communicational acts in which gesturing accompanies talking.

Table 1 compares utterances and gestures according to two characteristics, type of use and medium. By saying that gesture is used in a nonlinguistic way, I stress the fact that the gestures considered in this volume do not owe their communicational effectiveness to their being members of a symbolic system with syntax, but rather to our spontaneous ability, grounded in our cultural experience, to relate certain body movements to certain familiar things in the world.

Table 1 Comparison between two types of communicational acts, utterances and gestures, as used in this volume

Thinking

It is now time to return to the claim, made unanimously by all the contributors to this special issue, that gestures, although defined by their communicational function, are also a part and parcel of the processes of thinking. This raises the question of how thinking and communicating are related to each other. Possible responses range from the dualist positions, according to which thinking and communicating are separate processes, coming from different sources and often running in parallel, and to the vision of thinking and communication as two different manifestations of basically the same type of phenomenon. According to the uncompromising dualists, thinking is a primary process that unfolds naturally “from inside” the person, whereas communicating is a skill that originates in interpersonal interactions and is acquired for the sake of “conveying one’s thoughts” to interlocutors. The extreme nondualists, on the other hand, view thinking as developing gradually through communicating with others. In this process, one learns to communicate with oneself the way people communicate with each other. It is this self-communicating that they have in mind while talking about thinking. The disappearance of a thinking–communicating dichotomy automatically eliminates several other splits. Indeed, once talking and gesturing stop being but “expressions” of thinking and become the process of thinking in itself, there is no point any longer in trying to separate the form (a sequence of words or body movements) from the content (thoughts, ideas) of communication, and it becomes equally pointless to speak about meanings “conveyed” in words, implying that it is a self-sustained entity of sorts, capable of appearing in many different verbal disguises.⁴

The extreme dualist and radical nondualist positions delineate a whole spectrum of intermediate possibilities. Quite a number of these can be found in recent publications on gestures, including this special issue. Whatever the exact placement of a given writer with respect to the two extremal points, however, his or her language is rarely free of dichotomizing/objectifying expressions, such as “package meaning,” “carry meaning” and “express ideas” (McNeill 2005, p. 22) “signal-producing gestures” (Armstrong et al. 1995, p. 28), “mathematical meanings [that] cannot be noticed through mere observation” (Radford), “semantic content of concepts” (Nemirovsky & Ferrara), “gestures [that] express meaning” (Edwards), “appropriation of the culturally shared meaning of signs” (Arzarello, Domingo, Ornella, and Sabena) or “semiotic systems allow pupils to construct (or to appropriate) in the long run the meaning of mathematical objects” (Maschietto and Bartolini Bussi).

It is the radical nondualist position that seems to provide the firmest grounds for the claim about gestures as an inextricable part of the processes of thinking. Indeed, whoever posits that any act of communicating is already an act of thinking must also agree that thinking can take any communicational form, including gesturing. In the rest of this commentary I will opt for the fully nondualist version. From now on, I will be using the neologism commognition to embrace both (interpersonal) communication and thinking. According to this terminology, utterances and gestures are the building blocks of commognitive process. Having said this, I will now try to specify the way each of these modalities contributes to commognitive processes at large and to mathematical commognition in particular.

3 So, what do utterances and gestures have to offer?

Let me recapitulate: once thinking and communicating have been unified under the umbrella of commognition, the question of the respective roles of utterances and gestures in communicating becomes, automatically, also the question of their roles in thinking. What we shall see while trying to answer this query in the context of mathematical thinking may well be the key to perhaps the most teasing unanswered queries about human kind: What is it that gives humans their unique capability to change their forms of life from one generation to another? How do human children become gradually able to act in ways incomparably more complex than those of all their ancestors?

Utterances

One can hardly exaggerate in stressing the role of speech (utterances) in human commognition. An insightful account of how speech changes a child’s thinking (and I would say, commognition) can already be found in Vygotsky’s writings. Here, I will restrict myself to one property of linguistic commognition which has not been given much attention by Vygotsky, but which may be claimed to be the one that underlies all our uniquely human abilities. In particular, it may be the source of our ability to systematically increase the complexity of our actions. This special property, which in itself is claimed to be unique to humans, is called recursivity.⁵ Its most salient manifestation is our ability to turn commognition on itself—to speak about speaking, to think about thinking, to communicate about communicating.

Detailed substantiation of the claim about the decisive role of the property of self-reference in accumulating complexity is beyond the scope of this commentary (the interested reader is referred, once again, to Sfard 2008). Here, let me say only a few words about the part of recursivity in the development of mathematical discourse.⁶ It is one of the basic commognitive tenets that mathematical discourse grows by annexing its own meta-discourses. Thus, for example, algebra began its existence as a metadiscourse of arithmetic. To see this, consider for instance the brief algebraic statement a × b = b × a, which may be “unpacked” into the meta-arithmetic sentence, “For any two numbers, the product of the first one by the second equals the product of the second by the first.” Also equations, such as $2x + 5 = 11$, which are questions about numbers (“What is the number which, if doubled and increased by 5, would produce 11?”) belong to the metadiscourse of arithmetic. Historically, the mathematization of the metadiscourse of arithmetic, that is, subjecting this metadiscourse to the canons of mathematical discourse, led to the creation of the algebraic discourse, which is now considered to be a part of mathematics itself. The process of mathematizing included introduction of new signifiers in the form of algebraic formulas, such as a × b or x ². These new signifiers served as pointers to previously developed mathematical signifiers or to processes on such previous signifiers. Thus, x ² could be “realized”⁷ as ordered pairs of numbers where the second member of the pair is a square of the first, as a line called parabola, as the operation of squaring numbers, or even as just the phrase “the square.” These and many additional realizations of x ², together with their own realizations and the realizations of these realizations, could now be seen as what constituted the new mathematical object signified by x ². Introduction of these new objects brought an important gain: by tying many diverse mathematical objects with the help of one signifier and thus collapsing them into one, this move made it possible to formulate mathematical statements of unprecedented generality and of equally unprecedented complexity.

The increase in complexity, however, was transparent. In fact, due to the dramatic increase in the effectiveness of mathematical discourse—one could now say much more with a much less—the overall impression was that of a considerable drop in complexity. A whole hierarchy of discourses and their metadiscourses was now hiding in the elaborate “realization trees” of the new signifiers.⁸ The latter functioned as “storages of possibilities”: never present in the discourse in their entirety, the realization tree could provide a specific realization of its principal signifier whenever such realization was needed. One may say that these invisible constructs served as repositories of complexity. Thanks to the “squeezing” effect of turning mathematical metadiscourse into a part of mathematics itself, it now became possible to engage in a conversation about the new discourse. The focus of this conversation would be on this discourse’s characteristic patterns and on the ways in which one could generalize these patterns succinctly, with the help of new signifiers. Eventually, a new layer would be added to mathematics when this new metadiscourse, with its own new objects, undergoes the process of mathematization (abstract algebra may count as one of the results of this meta-algebraic activity—as an outcome of reflecting on the elementary algebra).

This very brief, and some readers would say all too dense, description of the development of mathematical discourse, although presented in historical terms, pertains also to individual learning. It was offered here with the double aim of, first, showing the critical role of recursively interrelated linguistic utterances in creating mathematical discourses, and second, of preparing the grounds for explaining the part played by gestures. The point I was trying to make about the role of the linguistic ingredient of commognition may now be formulated as follows: I proposed that mathematics is a product of linguistic commognition and that recursivity of human languages is the property that made its emergence possible. I will complete my argument by describing the role of gestures in mathematical discourse.

Gestures

The next claim I wish to present and substantiate is that gestures are crucial to the effectiveness of mathematical communication. More specifically, I will posit that gestures are invaluable means for ensuring that all the interlocutors “speak about the same mathematical object.” Let me precede by making a number of summarizing claims about the latter type of objects.

Just to recapitulate, a mathematical object is a signifier together with its realization tree, whereas the realization tree is a hierarchically organized set of all the realizations of the given signifier, together with the realizations of these realizations, as well as the realizations of these latter realizations, and so forth. An immediate conclusion from this definition is that mathematical objects are not any less material than any concrete, tangible thing, except that they are usually inaccessible to us in their entirety, that is, their numerous parts are never simultaneously visible. This view is contrary to the common vision of mathematical objects as being “ideal,” essentially intangible and immaterial. Some of the authors of this special issue seem to be not too distant from the claim about materiality of mathematical objects when they speak about “material” nature of thinking (Radford) or about conceptions as existing “only in, through, and as of the experiences” through which the individual realizes it (Roth and Thom).

One of the basic conditions for effective mathematical communication is that all the interlocutors realize the focal nouns in the same way. An even earlier prerequisite is that they have well-defined realizing procedures for any such signifier. The literature on classroom studies abounds in examples showing that this is not always the case. Here is a representative example from a recent study.⁹ In the following brief excerpt, student J tried to realize the signifier slope:

This student was quite awkward in realizing the signifier slope via any of the two available media: either via algebraic symbols, where he should have pointed to the coefficient of x, −2, or via the graph, where he could have traced the straight line and its projections so as to identify the “rise over run.”

Realizing mathematical signifiers is exactly the activity in which gesturing can play a critical role. It is with gestures that realizing procedures may be implemented either actually, when the medium in which the realization is to take place is present or, virtually, when the medium is only imagined. The articles in this volume provide numerous examples for both these types of gestural realizations. Thus for instance, in a problem not unlike the one J was trying to solve, student A in one of Arzarello and his colleague’s projects used gestures to demonstrate how he did what J was unable to do: realized the signifier slope of function f in point x by performing a series of appropriate hand movements directed at, and guided by, the graph of f.

the slope of the function at a point called x (... A is marking “x”), we do like that (he is tracing a vertical segment reaching the point on the graph corresponding to x); then, we compute a little interval x + h, (writing x + h and tracing a corresponding vertical segment), and at this point, we execute the classic operation !x, uhm..., we find out also the y, of course: the y are ... this is f(x) ..., this is a generic function (then he writes the formula of the incremental ratio....).

Simillarly, a student Bobby in Radford’s study has shown with gesture how the signifier distance should be realized in the drawing he was looking at: “But he, the distance (pointing to segment BC, P4) ... between them does not increase.” Maschietto and Bartolini Bussi tell us about children who realized parallel planes with parallel hand movements. Finally, students in Edwards’s project were reported to have realized the noun fraction with “gestures referring to ‘cutting,’ ‘splitting,’ and ‘slices’.” In this last case, the cutting gestures were made in the air and the parts that were to be obtained as realizations of the fractions were only imagined, not physically present. It was thus an instance of virtual realization.

These examples do not exhaust all the uses of gestures in mathematical discourse, but they bring to the fore one of the most important. Using gestures to make interlocutors’ realizing procedures public is an effective way to help all the participants to interpret mathematical signifiers in the same way and thus to play with the same objects. A combination of gestures with their verbal description would probably be even more effective, in this respect.

The relation between utterances and gestures

As was argued above, gestures realize words; as noted by Roth and Thom, the relationship goes in the opposite direction as well, with words acting as indexes to gestures. This relationship may also be described in terms of embodiment: by using gestures to realize words, we create a bodily counterpart of what is being talked about. The gestural procedures would often be automated; sometimes, they would be remembered by our bodies much better than the words are remembered by our minds. To realize this, it suffices to think about the gestures we perform whenever we have a word “on the tip of our tongue.” Little children displaying a number of fingers rather than uttering a number word in response to the question “How many?” may serve as another pertinent example. Yet another comes from one of my classroom studies: a student who was incessantly confusing the words slope and intercept was nevertheless able to confidently translate a table of linear function into a formula of this function simply because his solutions were mediated by canonical bodily realizations of slope and intercept rather than by the words (Sfard 2008). To sum up, words and gestures remain in a symbiotic relation and, in many situations, act as a “backup” one for the other.

4 Why and how study gestures

Having addressed, if only briefly, the question of relative contributions of verbal utterances and of physical gestures to mathematical commognition, I would now like to complete my answer to the “What’s the fuss” question by making the case for research on gestures. That this kind of investigations is a worthwhile endeavor does not go without saying. Documentation and analyzing gestures are very demanding tasks, requiring considerable investment. As observed by Arzarello, Domingo, Ornella, and Sabena, “The multimodal paradigm and specifically the analysis of gestures (for example how speech, gestures, and inscriptions are related to each other) implies the necessity of using an extremely fine grained analysis, namely, scales of time of few tenths of second.” To engage in this kind of undertaking, one needs to be pretty sure of the prospective gains. In particular, one needs to know what kind of questions the painstakingly collected data can answer.

Taking my earlier analysis of the role of gestures in mathematical communication as a point of departure, I will now formulate a number of such questions.

• In this commentary, I spoke about the importance of gesturing for mathematical communication. But using body movements, as any “concretization,” may also have its pitfalls. What are the possible negative effects of gesturing while learning mathematics? What kinds of uses, if any, may bring undesirable outcomes?

• How may one enhance mathematical commognition by encouraging gestures that demonstrate realizing procedures for mathematical signifiers? What would the practice of creating verbal descriptions of the gestures do to students’ learning of realizing procedures and, in consequence, to their ability to construct mathematical objects?

• How unique is the mathematical discourse of those whose body movements are somehow impaired and who are thus incapable of gesturing the way other people do?

• In a similar vein, is there anything unique in the mathematical discourse developed by those who learn the subject predominantly at distance, through electronic communication, where gestures are unavailable to interlocutors?

I hope that the obvious practical and theoretical importance of the issues raised in these questions would convince even the skeptics about the worthiness of research on gestures.