1 Introduction

In British Columbia, Canada, “In 2011–12, one in four (23.8 %) of public school students spoke a primary language at home other than English. Almost double the number of [English language learners] (135,651) live in families where the primary language spoken at home is other than English […]” (BCTF, 2012, p.11–12). Speaking from my own experience teaching mathematics in Canada, the home languages spoken in a typical mathematics classroom are very diverse, ranging from five to ten in any given classroom. This context is one result of globalisation and rapidly changing student demographics not only locally but worldwide. The National Council of Teaching Mathematics (NCTM) which has strong ties with the Western Canadian mathematics curriculum expressed the needs to address mathematics learning in a non-native language regardless of a lack of proficiency in the language of instruction. In addition, the NCTM (2000) emphasised communication as an essential process in mathematics learning: “communicating to learn mathematics and learning to communicate mathematically” (p.60). In today’s increasingly complex and multilingual mathematics classrooms, there is a growing challenge to utilize communication to access learners of all linguistic backgrounds.

Setati and Moschkovich (2011) use Grosjean’s (1985) analogy from the domain of athletics to explain the unnecessary dichotomy about home language and English language learning. They argue that, like high hurdlers who blend high jumping and sprinting, multilingual leaners blend multiple language competencies. Similarly, Moschkovich (2010) reminds future researchers to exercise caution when comparing monolingual with bilingual learners. One must not assume that monolingual learners have an advantage over bilinguals or that monolinguals are the norm because of their proficiency in the language of instruction: “Any time we use monolingual learners (or classrooms) as the norm, we are imposing a deficit model on bilingual learners. Bilinguals learning mathematics need to be described and understood on their own terms and not only by comparison to monolinguals” (Moschkovich, 2010, p.11). Her study recognises the importance of making sense of bilingual learners’ competence when learning mathematics. This paper shares the same view that bilingual learners blend multiple competencies, in particular, in mathematical communication. Given this position, the goal of this study is not how bilingual learners are different from monolinguals, but how bilingual learners utilise different resources to communicate effectively in mathematical activities. This line of work is much needed for achieving equity in mathematics education, since it aims to understand bilinguals’ mathematics learning on their own terms.

Currently, research focussing on linguistic diversity in mathematics education has provided tremendous insights into the complexities of teaching and learning mathematics in multilingual contexts: the language dilemmas of teaching mathematics (Adler, 2001), the role of code switching in learning mathematics (Clarkson, 2007) as well as associating mathematics learning with socio-economic and epistemological access (Setati, 2005). These studies, however, have not critically examined bilingual learners’ communication patterns, nor in particular, addressed their competences in mathematical activities. As argued by Moschkovich (2010), future studies on bilingual learners must consider broader linguistic frameworks for understanding bilingual learning.

Research on multimodality can shed light on bilingual learners’ communication as a multimodal activity that includes the use of language, gestures and interactions with diagrams. Radford et al. (2009) point out that in our acts of knowing, different sensorial modalities—tactile, perceptual and kinaesthetic—become integral parts of our cognitive processes. Other studies discuss gestures in mathematics teaching and learning, with respect to teacher’s gestures in relation to students’ meaning making (Arzarello, Paola, Robutti, & Sabena, 2009), the cultural dimension of gestures (Radford, 2009) and the role of gestures in mathematical imagination (Nemirovsky & Ferrara, 2009). With regards to diagrams, recent work suggests both that animated diagrams evoke new, mathematically-relevant gestures (Edwards, Ferrara, & Moore-Russo, 2014; Gol Tabaghi, 2012) and that diagramming itself functions as a non-linguistic form of mathematical communication (de Freitas & Sinclair, 2012). Aligned with the idea of multimodality in mathematical thinking, a small number of studies have drawn on bilingual learners’ non-linguistic forms of communication such as gestures and diagrams (Gutierrez, Sengupta-Irving, & Dieckmann, 2010; Moschkovich, 2007, 2009). For example, Moschkovich (2007) analysed the transcript of two bilingual students engaged in a mathematical discussion on the steepness of linear functions. The students did not struggle with the vocabulary, but rather used their home language, Spanish, and English interchangeably to negotiate the meaning of steep and less steep, as well as used gestures and their everyday experience of “x-axis is the ground” as resources in their communications.

Similarly, this paper addresses both linguistic and non-linguistic features of bilingual learners’ communications to uncover their mathematical competence in mathematical activities. Furthermore, studying the interplay between linguistic and non-linguistic communication may provide insights on mathematical thinking and learning. As Arzarello (2006) explains, the notion of semiotic bundle can be used to highlight the different semiotic resources used synchronically in mathematical activities. For example, Chen and Herbst (2012) examine the semiotic bundle consisting of language, gestures and paper diagrams in high school students’ geometrical reasoning. They suggest that “the constraints of diagrams may enable students to use particular gestures and verbal expressions, that rather than using known facts, permit students to make hypothetical claims about diagrams” (p.304). My study differs from Chen and Herbst (2012) in that I chose a setting in which bilingual learners interact with touchscreen, dynamic technology for exploring calculus concepts. The use of digital technologies, and dynamic geometry environments (DGEs) in particular, have been shown to facilitate student communication by providing visual and dynamic modes of interaction (Falcade, Laborde, & Mariotti, 2007; Ferrara, Pratt, & Robutti, 2006). Although numerous studies have discussed the effect of DGE-mediated learning of calculus concepts (Hong & Thomas, 2013; Yerushalmy, & Swidan, 2012; Yoon, Thomas, & Dreyfus, 2011), research on the effect of touchscreen-based DGE is limited. It is hypothesized that a touchscreen-based DGE may offer additional affordances by providing tactile and kinesthetic mode of interaction—hence, further facilitate bilingual learners’ communication in calculus.

The current study examines the communication patterns that arise as bilingual learners interact with touchscreen-based DGEs. It concerns the linguistic and non-linguistic features of bilingual learners’ communication. In particular, I investigate the interplay between language, gestures and touchscreen- dragging on DGEs as part of the emergence of new mathematical ideas. Further, it is the goal of the study that this analysis will uncover bilingual learners’ competence in mathematical communications.

2 Theoretical framework

2.1 Learning as a discursive activity: talking, gesturing and mathematical thinking

The notion that learning mathematics is building mathematical communicative competence is suitable for the current study because it establishes a strong link between mathematics learning and communication. The learning as participation perspective complements this view; it is a broader framework for conceptualising learning in its social dimensions (Lave & Wenger, 1991; Wenger, 1998). This perspective suggests that learning is located neither in the heads nor outside of the individual, but in the relationship between a person and a social world. Sfard’s communicational framework (2008) is based upon the social dimensions of learning and highlights the communicative aspects of thinking and learning. For Sfard, thinking and communicating are two parts of the same entity. This approach disobjectifies thinking as a purely cognitive phenomenon and examines the relationship between talking, gesturing and thinking in non-dualistic terms. Sfard redefines thinking as an “individualised version of (interpersonal) communicat[ion]” (p.81). The term commognition stresses the fact that thinking (individual cognition) and interpersonal communication are manifestations of the same phenomenon. Communications can take the form of written or spoken words and algebraic symbols; it can also take a visual form through gestures.

Sfard (2008) proposes four features of the mathematical discourse, word use, visual mediator, routines, and narratives, which could be used to analyse mathematical thinking and changes in thinking. For the purpose of this paper, the first three features will be used for examining the role of language, gestures and dragging in one’s the mathematical discourse. Word use is a main feature in mathematical discourse; it is “an-all important matter because […] it is what the user is able to say about (and thus to see in) the world” (p. 133). However, as a student engages in a mathematical problem, her mathematical discourse is not limited to the vocabulary she uses. For example, her hand-drawn diagram and gestures can be taken as a form of visual mediator to complement word use. For the present study, the DGE is taken as a dynamic visual mediator, for it enables users to observe and manipulate visual objects that are moving and changing over time. While static visual mediators evoke images of static mathematical objects such as triangles or artifacts such as a number line, dynamic visual mediators evoke mathematical relationships and properties more readily, by displaying mathematical objects with an invariant property continuously.

According to Sfard (2009), utterances and gestures are different modes of communication and take on different roles in the commognitive process. Recursivity is a linguistic property offered by utterances. It provides an unlimited possibility to expand linguistically and allows humans to work in meta-discourses, or thinking about thinking. On the other hand, gestures enable effective communication to ensure all interlocutors “speak about the same mathematical object” (p.197). Gestures are essential for effective mathematical communication: “Using gestures to make interlocutors’ realizing procedures public is an effective way to help all the participants to interpret mathematical signifiers in the same way and thus to play with the same objects” (p.198). Gestures can be realised actually when the signifier is present, or virtually when the signifier is imagined. Sfard (2009) illustrates how a student uses “cutting”, “splitting”, and “slicing” gestures to realise the signifier “fraction”. Since these gestures were performed in the air, where the signifier “fraction” is imagined, they provide an instance of virtual realization. Therefore, the same signifier “fraction” may be realised differently with different kinds of gesture or word use.

Routines are meta-rules defining a discursive pattern that repeats itself in certain types of situations. In learning situations, teachers may use certain words or gestures repeatedly to model a discursive pattern, such as looking for similarities and what it means to be “the same”. For example, if a student repeatedly uses her hand or arm to signify slope when comparing slopes of line segments, she is using gestures as a routine to look for what is “the same”. Sfard (2008) conceptualises learning mathematics as a change in one’s mathematical discourse. Incommensurable discourses arise when inconsistent words, visual mediators or routines are used in communication. Communicating in incommensurable discourses may lead to commognitive conflicts between one or more interlocutors.

2.2 Communicating with touchscreen: dragging as a form of communication

In a previous study working with bilingual learners, I showed that certain dragging actions on touchscreen-based DGEs constitute a form of communication (Ng, 2014). Using a Sfardian approach, my analysis showed that some dragging actions were not merely dragging but also gestural communications—to communicate the dynamic features and properties in the sketch as obtained by dragging. To illustrate why certain dragging actions are also considered gestures, it would be possible to imagine a static environment where the dragging modality is not available. If a speaker moves his/her finger along a graph while referring to the tangent slope as “increasing” or “decreasing”, this action can be considered a kind of gesture for communicating the idea, “as x varies on this graph”. On the other hand, the touchscreen-dragging modality allows the dragging with one finger on the touchscreen and the gesturing with the index finger to blend together as one action. The importance here is that the dragging-gesturing action subsumes both dragging and gesturing characteristics: it allows the point to be moved on the screen (dragging), and it fulfills a communicational function (Sfard’s definition of gesturing).

Hence, dragging-gesturing is taken as a significant form of communication in this study. It is taken as both a routine for defining a discursive pattern that repeats itself in activities with touchscreen-based DGE, and a visual mediator as a multimodal feature of the students’ discourse. A student may use dragging-gesturing repeatedly to compare the variance of the tangent slope, of which it becomes a routine. Or, it can be used to signify the tangent slope, in which it becomes a visual mediator. This communication can be interpersonal when it is directed to another student or intrapersonal when it is directed to oneself. Using this notion, it is possible for students to incorporate dragging-gesturing to respond to each other in communications. Indeed, it was found that as one student suggested that the secant line will get “closer” to the tangent line, another student seemed to have responded by her dragging-gesturing to bring the lines “together”. These gesture-utterance correspondences were noted in other pairs of bilingual learners’ communications as well (Ng, 2014).

In summary, I use Sfard’s communicational theory to analyse bilingual learners’ communication about calculus concepts given a dynamic visual mediator, a touchscreen-based DGE. This allows me to analyse bilingual learners’ thinking––and changes in thinking––as they communicate. I focus on their word use as linguistic features; and visual mediators and routines, in the form of gestures and touchscreen-dragging with a DGE, as non-linguistic features of their mathematical discourse. In addition, I attend to the complementary functions served by these different features to identify their competence in the mathematical activity.

3 The study: participants, data collection and task

The study is part of a larger research project that aims at investigating patterns of bilingual learners’ communication in a touchscreen, dynamic calculus environment. In order to address this aim, the participants needed to have a certain degree of experience working with DGEs before the study. Therefore, the teacher-researcher (also the author) taught a year-long calculus class where the use of touchscreen-based DGEs were consistently incorporated into the lessons for exploring calculus ideas. During these lessons where DGEs were incorporated, the teacher-researcher invited students to explore the pre-designed dynamic sketches in pairs for roughly ten minutes before leading a whole class discussion about the exploratory activities. Upon exploring concepts with the DGEs, she would formally introduce the concepts, provide examples to be solved algebraically on paper, and ask for their solutions to be related graphically on the DGEs. As the researcher took on the dual role of a teacher-researcher, measures were taken to ensure that the two roles would not interfere each other (Ainley, 1999). For example all potential participants were informed: (1) that their participation was entirely voluntary, (2) that regular classroom teaching would not be affected in any way, and (3) that in no way will any form of their regular calculus classroom learning be affected as a result of the study. The study was conducted during non-class time in the participants’ regular calculus classroom. The rationale for choosing this setting was to provide a physical environment that they were used to, at the same time, not to affect their regular classroom teaching and learning.

The participants of the study were four pairs of 12th grade students (aged 17 to 18) enrolled in an Advanced PlacementFootnote 1 calculus class in a culturally diverse high school in Western Canada. The participants were bilingual learners in a class of 25, in which roughly half of the class were also bilingual learners. They were invited to participate voluntarily for their bilingual background––all of them had been studying in Canada and in an English-speaking schooling environment for 2 to 4 years. Amongst the four pairs of participants, two pairs do not share a common home language. As mentioned in the introduction, the setting in which students do not share a common non-English, home language under English instructions was quite typical in the study’s context of Western Canada. Planas (2014) describes this context as “individual bilingualism”, since English is the only verbal language that learners share in common, and phenomenon such as code-switching is insignificant (Planas & Setati-Phakeng, 2014). Furthermore, the context of “individual bilingualism” is highly relevant to the study’s goal because it highlights bilingual learners’ non-linguistic means of communication.

The study took place at the end of the first trimester of the school year. At the time, the participants had just finished learning the differential calculus component of the course and completed one lesson on antiderivative. The task used in the study invited the participants to discuss a sketch presented on an iPad-based DGE that they had not previously seen. For the purpose of comparing students’ routines during the exploratory activity with the touchscreen-based DGE, they were given a sketch containing five pages related to the concept of area-accumulating functions, A(x) = a  ∫x f(t)dt. The participants were asked to “explore the pages, talk about what you see, what concepts may be involved” on each page of the sketch and then to move onto the “Try” page of the sketch where a problem was posed. They were asked to solve the problem on a dry-erase whiteboard. All pairs of participants were regular partners during assigned pair-work activities and were described by the teacher-researcher as motivated and comfortable working with each other. They were told that their teacher would check in with them from time to time to make sure that they were on task and could ask any technical questions related to the sketch. Each student-pair took around 30 to 45 min to complete the task. In total, 135 min of video data were collected in the study.

Following the order of a commonly used single-variable calculus textbook, Stewart (2008), the participants had just completed a lesson on antiderivative functions at the time of study. Specifically, they had just learned the notation ∫f(x)dx = F(x) + C, where f(x) = F ′ (x). However, they had not encountered definite integral as area. The target concept of area-accumulating function was chosen because it was possible for one to grasp the change of A(x) = a  ∫x f(t)dt by interpreting change of “area under curve” geometrically without knowing the corresponding symbols. Secondly, the timing was fitting because the students would have had some experience learning calculus with the use of DGEs at the time of study. In particular, they would have used a similar dynamic sketch for exploring derivative functions by interpreting derivative as tangent slope of a graph geometrically. This experience was similar to exploring area-accumulating function, since both explorations required learners to interpret change (tangent slope and area-accumulation) geometrically and to identify some covariance between x and the function f’(x) or A(x). In the next section, I detail the design of sketch used to evoke the idea of area-accumulating function in the study.

3.1 Design of sketches

In line with the study’s aims and aspects of semiotic mediation (Bartolini-Bussi & Mariotti, 2008), the dynamic sketch used in the study was designed to mediate dynamic aspects of calculus, activate touchscreen dragging and produce signs that evoke algebraic and geometric representations of calculus. Semiotic mediation is based on a Vygotskian perspective which suggests that learning occurs through a process of internalization of tools into signs (Bartolini-Bussi & Mariotti, 2008). Within this lens, the functionalities offered by a DGE produce many signs. An external, goal-oriented activity such as “dragging” and “tracing” an object in a sketch can be internalized to produce personal meanings. The design of the present sketch mainly features three functionalities offered by The Geometer’s Sketchpad (Jackiw, 2001): the Hide/Show button, the Dragging tool, and the Trace tool. The Hide/Show button allows different mathematical objects, texts, and numerical calculations to be shown or hidden when pressed. Other than the last page, the Try page, all pages contain the same Hide/Show buttons to allow the objects, Function f, Bounds, Area under f and the Trace of A to be shown or hidden conveniently. The first four pages display different functions when the Show Function f button is activated: a constant function on Page 1 (Fig. 1a), a linear function of degree-1 on Page 2, a quadratic function on Page 3, and the sine function on the Page 4 (Fig. 1b). Having shown the functions, the student-pairs may explore the “area under the functions” (A = lim n → ∞∑ n i = 1 f(x i )Δx) both numerically and geometrically by dragging the points a or x along the x-axis. For example, Fig. 1a shows that the area under the function f(x) = 1 is A = 4.70 when the bounds are set to a = 0 and x = 4.70.

Fig. 1
figure 1

a: A sketch showing the area under a constant function with all buttons activated and x dragged from zero to its current positions. b: A sketch showing the area under a sine function with two kinds of colours used

Full size image

The Dragging tool can be combined with the Hide/Show button to mediate relationships between mathematical objects. As the user drags the points a or x along the x-axis continuously, the value of A and the shaded area change correspondingly. Performing this kind of dragging actions mediates functional dependency between variables, by producing signs that contrast between what is independent (x) and dependent (A) (see Falcade et al., 2007). Furthermore, the present sketch illustrates how the dynamicity of dragging may connect numerical and geometrical representations of calculus, since dragging simultaneously changes the numerical values of a or x and the geometrical representation of A. In the lens of semiotic mediation, the simultaneous change of all the variables, x, f(x), and A can be visually mediated in this way.

The Trace tool can be used to generate a set of “green traces”, which represent the graph of the area-accumulating function A(x) for the chosen a and f(x). When the button Show Trace of A is pressed, a green point appears on the page at (x, A). This point is not draggable which implies that it is not an independent object. More importantly, the green point leaves behind traces of its previous positions as x is dragged, ultimately producing a set of green traces in the shape of the corresponding area-accumulating function. Since the movement of the green point is dependent upon dragging x (or a which would result in a vertical translation the green point), and dragging x also changes the area under f, the sketch can potentially evoke the mathematical meaning of area as a function of x.

As the students had not encountered the function A(x) =  a ∫x f(t)dt in their regular classroom, the goal of this sketch was to introduce the idea of area as a function, and this can be achieved when the students are able to relate the “green traces” as the graph of “area under f” from a to x. It is anticipated that when f(x) is below the x-axis, the students would find it difficult to interpret the “area” bounded by f(x) and the x-axis as “negative” since they had yet to learn “area accumulation” as meant by the definite integral a ∫b f(t)dt. To facilitate students’ exploration of “area accumulation” when f(x) > 0 and f(x) < 0, different colours were used as signs for mediating “positive” and “negative area” (Fig. 1b).

4 Methods

All data were transcribed and analysed in terms of the student pairs’ utterances, gestures and dragging actions during the aforementioned task. Several methodological choices were made to achieve the analysis. First, according to Arzarello (2006), a synchronic analysis can be applied to examine the inter-relationships between language, gestures and diagrams at a certain point in time (see also Chen & Herbst, 2012). In line with Arzarello (2006), I transcribed and organised the data collected to highlight the interplay between word spoken, gestures and dragging actions within the student pairs’ communications synchronically. Unlike conventional transcripts which informs only “who spoke what”, I introduced two columns, the “gesturer” and “dragger” columns, in the transcript in order to track “who gestured” and “who dragged” simultaneously. Snapshots of certain gesturing and dragging actions were taken and included in the transcript. In addition, I used underlining of the transcript to record what words were spoken while a dragging or gesturing action was performed simultaneously by one of the students. Second, the data was analysed diachronically to enable an investigation of whether certain utterances, gestures and dragging actions remained prevalent or changed over time.

The data analysis process also considers Moschkovich’s examination of how different methodological views may help reveal or undermine bilingual learners’ competence in mathematical activities. Moschkovich’s (2007) sociocultural view focuses on what bilingual learners can do, by identifying the resources that bilingual learners use in mathematical communication. “Resources” are taken broadly by Moschkovich as any oral or written language, images, equations, symbols, sounds, gestures, graphs and artifacts. In the study, “resources” mainly take the form of oral language, gestures and touchscreen-dragging on the DGEs. These resources can be used in the participation of mathematical discourse practices—practices that are shared by members who belong in the mathematics or classroom community. Moschkovich argues that analysing the types of mathematical discourse practices can highlight the mathematical competence of bilingual learners: “even a student who is missing vocabulary may be proficient in describing patterns, using mathematical constructions, or presenting mathematically sound arguments” (p.20). In general, abstracting, generalising, searching for certainty, and being precise are highly valued mathematical discourse practices across different mathematical communities.

Blending Sfard (2008) and Moschkovich (2007), my analysis is guided by the following questions:

  1. 1.

    What are the situated meanings of the words, phrases, visual mediators and routines in the mathematical activity?

  2. 2.

    How do students utilise multiple resources (speaking, gesturing, dragging) to communicate mathematically? What signifiers are realised, and how are they realised?

  3. 3.

    What mathematical discourse practices are demonstrated in the activity, and how are they demonstrated?

One of the goals in presenting the analysis is to show how the sociocultural view, combined with studying the interplay between language, gestures and touchscreen-dragging on the DGEs, may uncover bilingual learners’ mathematical competence in communications. For the purpose of investigating the interplay within the linguistic and non-linguistic resources used in communication, it was necessary to observe the way that these resources were utilised simultaneously and in sequence in communication. This was achieved by attending to turn-taking; instances of simultaneous speaking, gestures and dragging (either by the same person or by different persons); and gestures or dragging without accompanying speech.

5 Analysis of data

In this section, I provide a detailed analysis of one participant pair’s use of speech, gestures and dragging on DGE in their developing discourse around area-accumulating function. In particular, I focus my analysis on the first ten minutes from a total of thirty minutes of data collected on the student pair, Sam and Mario. The chosen data is further divided into three episodes for identifying themes and to illustrate a change of discourse (speech, gestures and dragging) over the course of their exploration. I have chosen to focus on Sam and Mario, whose home languages were Mandarin and Cantonese respectively and had the least experience studying in an English environment. Hence, it could be said that the Sam and Mario had the least linguistic resources available to them, since English was the only language they share in common and code-switching to their home languages was not possible.

5.1 Excerpt 1: Questioning and communicating with utterances, gestures and dragging

Excerpt 1 below revolves around Sam and Mario’s first 2-min interaction with the sketch conveying area-accumulating functions. At the start, all buttons were in the Hide position.

(Sp = Speaker, Dr = Dragger, Ge = Gesturer, S = Sam, M = Mario, Underline transcript = utterance spoken simultaneously with dragging or gesturing)

figure a

Excerpt 1 highlights the way Sam and Mario made use of gestures and dragging to question and communicate mathematically. The students seemed unsure about what to do with the points a, x and the green point initially. They questioned the functions of the DGE with question markers “what” (eight times) and “how” (once) in the first two minutes of the excerpt. Most of these questions were formulated as Sam used the dragging modality to investigate the behaviour of different points. For example, in Turn 5, Sam asked “what” repeatedly as he tried to drag the green point which was not draggable, and the whole page was moved incidentally. Although he had acknowledged that he had previously pressed a button which showed “an area” (Turn 4), he had not realised that the green point had plotted the area in terms of x, evident in his questions, “what’s trace of A” (Turn 6) and “how do we drag this trace” (Turn 7). Then, upon dragging x around, he finally concluded, “oh, oh, it's this one. Ok, make sense,” (Turn 9). At this point, his commognitive conflict seemed to have resolved perhaps because his dragging of x made the green point move; hence he realised the green point was a dependent non-draggable object. However, it appeared that he remained unsure about what the green point meant, stating that he did not yet “understand” in Turn 23.

From Turn 9 to 18, Sam and Mario took turns dragging-gesturing in a conversation-like manner, beginning with Sam’s dragging-gesturing, which spanned 30 s, from Turn 9 to 13 (Figure 2b). During this occurrence, Sam dragged x, then a, and finally x again. Observing the students’ word-use and dragging actions, it seemed that both students made some progress in their learning of area-accumulating functions during this span. For example, in Turn 11, Mario asked, “Is that the area” as Sam dragged a around to leave some green traces that were vertical. This was the second time the word “area” appeared in the transcript, and it was used differently from the first usage in Turn 4. The word “area” was first used when Sam pressed the Show Area under f button and uttered, “It’s an area”. Since Sam simply pressed the button and did not use any dragging to change the area, it is suggested that his realisation of area was static in Turn 4. In contrast, Mario’s utterance seemed to suggest that he might be thinking about area in a dynamic sense. This is indicated by the way Mario referred to the green point as “the area” while the shaded area was changing dynamically upon Sam’s dragging.

In Turn 12, Sam uttered, “As we drag this, the area becomes…” while he dragged x. This utterance-dragging combination suggests that Sam was also thinking about area as having dynamic qualities. It shows how dragging mediated the way Sam thought of area as a becoming. The use of “as…become” implied something was happening, in particular, the area was changing as x was dragged. Furthermore, Sam’s statement structure resembled an “if… then…” statement structure which called upon a causal or functional relationship between x and the area. It was interesting to note that Sam never finished his sentence after uttering “become”. Since Sam mentioned that he did not “understand” in the last part of the excerpt, it is speculated that Sam did not finish his sentence because he had yet to realise, in a Sfardian sense, the simultaneous change in the variables despite noticing the area is changing. Similarly, Mario used a hedge word in his utterance, “it's like the area,” suggesting a degree of uncertainty about whether or not the green traces meant the area.

Different draggers and speakers were observed concurrently in the episode. In Turn 17, as Mario dragged x back and forth, Sam was responding verbally and simultaneously, “You see how this one moves? So it's like the area.” A similar exchange was also noted in Turn 11, where Sam was the dragger, as Mario spoke, “Is that the area?” These two instances where the dragger and speaker were different people seemed effective for creating a mutual and simultaneous communication. Although it may seem impolite and unconventional for one student to “talk over” another student, the presence of “talking over someone else’s dragging” was not an issue here. Indeed, Sam’s utterance did not interfere with Mario’s dragging and vice versa; rather, from the way one talked about area while the other was dragging, they seemed to be making significant progress as a result of this concurrent communications.

Also observed was the consistent use of gestures in mathematical communications. Namely, Sam used three types of gestures, which in Sfard’s terms, functioned quite differently in each usage. In Turn 3, Sam used a pointing gesture as he talked about the bounds to make sure both interlocutors spoke about the same mathematical object (Figure 2a). In Turn 17, he used his hand to signify the linear pattern of the green traces, an instance of actual realisation (Figure 2c). Finally, in Turn 22, he flipped his right index finger left and right (Figure 2d) while uttering, “No you can't. You can only go like”, which was another actual realisation of the possible movement of the green point. Moreover, this gesture was not accompanied by any speech, which suggests that Sam relied on gestures as a visual mediator in his mathematical discourse to communicate in the absence of word use.

5.2 Excerpt 2: Exploring and conjecturing with utterances, gestures and dragging

Immediately following Excerpt 1, the students continued to explore the sketch for another 2 min and 30 s, as seen in the transcript below.

figure b

Like Excerpt 1, Sam and Mario consistently utilised gestures and dragging in different parts of Excerpt 2 to communicate mathematically. The dragger column of the transcript clearly shows that in the first half of the transcript (Turn 27–39), the students’ communication was dominated by dragging, with or without speech. For example, in Turn 28, Sam was dragging while he asked the question, “Is that how the area is changing?” Unlike Excerpt 1, Sam was able to describe exactly that the area is changing with no hedge words in this excerpt. He continued to drag for a span of 9 s without speech before letting Mario also tried dragging for another 6 s without speech (Turn 29–30). The switching of draggers suggest that both students were engaged in some interpersonal or intrapersonal mathematical communications while dragging. This was also interesting because it seemed as though, for Mario, it was not enough to see Sam dragged; he had to do it himself too. If speech was analysed alone, some important analyses about the students’ thinking in between speech would have been missed.

Through word use, gestures and dragging, the students demonstrated valued mathematical discourse practices such as exploring and conjecturing. My analysis of Excerpt 2 shows that the students paid more attention to the green traces compared to Excerpt 1. For example, they used words like “it” and “this” to refer to the green trace and verbs to describe its motions throughout this excerpt. The student pair used the words “move” five times, and “goes” and “changing” once each to talk about the state of the sketch as they dragged either a or x. This shows that the student-pair moved from questioning about the technology to exploring and describing the dynamism shown in the sketch. Thus, it can be said that the students had shifted their discourse from the act of dragging earlier to the meaning of dragging. In addition, the students used the Hide/Show buttons and dragged points a and x purposefully without struggling with their functions as they did previously. This can be observed in Turn 35 (Figure 3a) when Sam used his right index finger to drag x around and then his left index finger to press the Show Trace of A button. It seemed like Sam was checking if x was draggable before he pressed the next button. Sam’s coordination of two fingers from different hands to interact with the Dragging tool and the Hide/Show button rather seamlessly suggest that Sam had begun to use the technology meaningfully for exploring the meaning of the sketch.

A little after this discussion, Sam and Mario performed a series of hand gestures from Turn 40 to 54. Initiated by his own dragging of x, Sam remarked that, “oh, wait a sec. This is actually, the derivative of the graph, function” (Turn 40) while he used a hand gesture to signify the shape of a linear function (Figure 3b). To restate what he had said, he then used his index finger and traced a “U” shape in the air as he continued to conjecture that the line was “probably the derivative of x, x-squared” (Turn 42, Figure 3c). Mario responded with a similar “U” shape gesture as he asked, “is this x-squared” (Turn 45, Figure 3d). These gestures and word use pairings provide evidence that the students were engaging in conjecturing about the shape of the green traces. In addition, they helped identify Sam and Mario’s competence in the mathematical activity.

As the students conjectured about the relationships between the two graphs, their mathematical discourse become more developed in both geometric and algebraic terms. This can be suggested through their word use accompanied by different kinds of gestures. One kind of gestures was a kind of hand gestures mimicking the geometrical shape of the functions, as used by both students on three occasions. In Turn 40, Sam aligned his fingers and palm together and gestured a line in the air, as he uttered, “This is actually, the derivative of the graph, function.” Since his gesture was performed simultaneously with the utterance “this is actually”, it is suggested that Sam was referring to the linear function. In addition, Sam and Mario both used similar hand gestures to trace a “U” in the air to refer to the parabolic green traces on the page. These gestures mimicking the shape of functions revealed the students’ geometrical realisation of derivatives, that the “line” (Turn 49) was the derivative of the parabolic “graph thing” (Turn 52). Besides talking in geometrical terms, the students mentioned some algebraic expressions such as “x-squared” and “y equals to x” as well in the same discussion about the relationships of the two functions. With respect to gestures, Sam used a kind of scribing gesture in which his index finger enacted a pen as if he was writing something on the table (Figure 3e), while he said, “one over two, no, x-squared over two” (Turn 54). This analysis supports the claim that the students were thinking about derivatives algebraically, in the sense that y = x is the derivative of y = x 2 /2.

It was noted that the deictic pointing word “this” was used extensively, appearing five times in the last part of the episode. Using deictic words, the speakers no longer needed to refer to the mathematical objects by describing them verbally, but they could use deictic words along with different gestures to replace the descriptions completely. This was found in Sam’s “this is actually, the derivative” (Turn 40), “no matter how you move, this one always” (Turn 38), and “this is probably x, x-squared” (Turn 42). As Sfard explains, gestures help ensure that the interlocutors speak about the same mathematical objects. Significantly for Sam and Mario, gestures served complementary functions to speech in communication. In this episode, the two students were able to use a combination of utterances and gestures to communicate the relationship of the mathematical objects effectively.

Through word use, gestures and dragging, the students realised dynamic, geometrical and algebraic notions of calculus. Moreover, the Trace tool and shaded area gave feedback about the relationship of the green traces and the area under f, which enabled the students to conjecture the possible relationship between the two graphs as one being the derivative of the other. However, at this point, the students’ language still contained the hedge word “probably” (Turn 42).

5.3 Excerpt 3: Predicting and verifying conjectures with utterances, dragging and gestures

Except 3 was taken forty seconds after the end of Except 2. Sam and Mario continued to use the DGE for exploring calculus ideas.

figure c

The students’ discourse around area-accumulating functions became increasingly developed in Excerpt 3. First, the questions posed in this excerpt were markedly different from the first two excerpts. Recall that during Excerpt 1, the students asked repeatedly, “what”, at times without finishing their questions. The previous analysis showed that these questions reflected a degree of uncertainty about “what” the sketch meant to them. In contrast, Sam asked three questions that began with “why” in Excerpt 3. He asked, “Why is there something to do with area?” in Turns 78 and 84, as well as “why is it?” in Turn 101. By asking these “why” questions, Sam seemed to be looking for the reason as to why the relationship that they found about the two graphs were related to the area under a function. This was a valid question considering that Sam had yet to learn the idea of “definite integral as area” in his class. Regardless, asking “why” implies investigating the reasons of something that is clearly existential. In this case, Sam seemed to be investigating the reason why the area under a function had to do with its antiderivative.

From Turn 73 to 77, a prolonged dragging action was performed by Sam (Figure 4a), while the two students exchanged comments verbally back and forth. In particular, by far the longest spoken sentence was observed in Turn 75, spoken by Sam while he was simultaneously dragging:

“Ya. So the graph we have here is specifically the derivative of the… what we just graphed here, like the function here is basically the derivative of what we just graphed here.”

The sentence was very rich in a multimodal sense because it was spoken while the speaker was dragging, and gestures were used simultaneously as the speaker uttered, “the function here” (Figure 4b). Some interesting word uses were also observed. For instance, the word “here” was used four times, and the words “specifically” and “basically” each once. In line with a previous analysis of the use of deictic words, the use of locative noun “here” accompanied by gestures allowed the speakers to talk about the same mathematical object. Although Sam used the same word “here” four times, he actually meant to refer to two different mathematical objects, the function and its derivative. Perhaps this was why Sam complemented his utterances with gestures to specify the objects he was talking about. Secondly, the contrasting use between “specifically” and “basically” by Sam was also fascinating. Since Sam used the word “basically” quite frequently throughout the task, his word use “specifically” as opposed to “basically” in this sentence drew attention to the analysis. Consistent with his usage of “basically” in other parts of the transcript, it seemed that Sam used the word to suggest a generality or invariance that exists outside of the sketch. In contrast, it is speculated that he used “specifically” to refer to the particular “graph” that was the derivative of another on a specific page of the sketch. According to this speculation, Sam was able to talk about area-accumulating functions both in its generality and particularity, which is a highly valued practice in the mathematics community.

Although up to this point, the analysis seems as though Sam was more engaged in the activity, one can see that Mario was also an active participant through careful observations of his utterances, gestures and dragging. Throughout Excerpts 1 to 3, the two students used the pronoun “we” extensively, with few occasions where “I” or “you” were used to differentiate between the speaker’s own intention from the collective. For example, in Turn 75 which was described above as a significant moment in the students’ discussion, the word “we” appeared three times in a single utterance. In addition, the word “let’s” [let us] was proposed three times by Mario, in “let’s look at this one” (Turn 92), “let’s predict” (Turn 97), and “let’s see, erase trace” (Turn 103). The most significant of the three was, “let’s predict”, since it led the students to predicting and verifying conjectures, from what was exploring and occasionally conjecturing in Excerpts 1 and 2.

“Predicting” the shape of the graph of area under the sine curve marked a significant change in the students’ discourse around area-accumulating functions. At first, Mario suggested, “Let’s look at this one,” as he opened the last page of the sketch and pressed all but the Show Trace of A button. Upon Sam’s acknowledgement, “we have another one” (Turn 96), Mario responded, “Let’s predict. This, the graph is going to be a… negative cosine.” The tone of Mario’s utterance was firm and the use of present continuous tense “is going to be” confirms that he was predicting the shape of the graph of area under the sine curve. Furthermore, the statement contained no hedge words and so the degree of certainty was much higher than previous statements with “like” and “probably”. It can be argued that Mario did not press the Show Trace of A button intentionally when he first opened the page because he wanted to predict the shape of the green traces all along. Upon both students’ agreement that the equation of the green traces should be “negative cosine”, Mario began to drag x back and forth rapidly (Figure 4d) which left behind a set of green traces in the shape of A(x) =  0 ∫x sin(t)dt = − cos(x) + 1. His rapid dragging action had never been done prior to this moment, as all dragging actions performed by the two students had been steady but not rapid. By dragging rapidly, the green traces could be achieved quicker, and this seemed to be Mario’s intention behind the action. More importantly, his rapid dragging seemed to be a case of Mario trying to advance the process of tracing, in so doing, encapsulating the set of all ordered pairs (x,A(x)) into a singular discursive object (Sfard, 2008)—the area-accumulating function.

Besides his dragging and leading the discussion towards predicting the shape of the graph of area under sine, Mario also showed that he was fully engaged in the activity with his word use and gestures. For example, he finished Sam’s sentence, “this is basically…” with the word “derivative” in Turn 74. He also performed another “scribing gesture” (Figure 4c) when Sam questioned “why it has to do with area” (Turn 84). This “scribing gesture” can be taken as his non-verbal response to Sam or his own intrapersonal communication. In either case, it can be shown that Mario was thinking-communicating mathematically and not as disengaged as it might seem, with the present analysis incorporating word use, gestures and dragging.

As mentioned, the two students took 30 min in total to complete the task. A few minutes after Excerpt 3, Sam mentioned that “it is the integral, but with the ‘c’, because…” Further analysis beyond Excerpt 3 shows that the students appeared to have achieved beyond the learning outcome of the activity. However, for the scope of this paper, I have decided to focus my analysis on the interplay between linguistic and non-linguistic features of the students’ discourse with touchscreen-based DGE rather than on students’ learning of calculus in a dynamic environment. As stated, the goal of the study is to uncover bilingual learners’ competence in mathematical communications, which I find the current analysis sufficient to serve its purpose.

6 Discussion and conclusion

The data analysis provides ample evidence that Sam and Mario utilised a variety of resources in communication, with visual mediators in the form of gestures and dragging taking on a prevalent role. These included gestures accompanying deictic words, gestures for communicating geometrical notions of calculus, as well as scribing gestures for communicating derivatives and antiderivatives algebraically. Moreover, the dragging-gesturing action emerged in the students’ interaction with the touchscreen DGE and fulfilled the dual function of dragging and gesturing. These actions were repeatedly demonstrated by both students for questioning and communicating about calculus ideas in Excerpts 1 and 2, as well as for developing their routines of conjecturing and verifying calculus relationships in Excerpt 3. In the presence of a dynamic visual mediator, the students’ mathematical discourse differed from the traditional mathematical discourse that emphasises word use. In particular, gestures-gestures and gestures-utterances sequences were observed repeatedly in the communication. Related to this, I observed one student dragging-gesturing simultaneously as the other spoke; this allowed two students to communicate simultaneously without interfering with each other. Using Sfard’s communicational framework, which defines gestures as communicational acts, is especially useful for understanding the mutual communications involved in these new kinds of communication routines.

The analysis also suggests an important interplay between utterance, gestures and dragging in the mathematical discourse. Initially, the students seemed unsure as to what to make of the sketch; dragging enabled them to formulate their questions about the behaviour of the sketch. Then, they began to explore and conjecture the relationship of the two functions in both geometrical and algebraic terms through dragging and gesturing. Sam and Mario made extensive use of verbs such as “become”, “move” and “go,” which imply change or motion while they used the dragging modality to change the area under a function. Moreover, gestures in the form of actual realisations were accompanied by the use of locative nouns “here” and deictic word “this”. Thus, the results concur with Sfard (2008) in that visual mediators in the form of dragging and gesturing served complementary functions with word use. More significantly in the context of bilingual learners, the study suggests that non-linguistic communication such as gestures and dragging could be used to complement linguistic communication to reduce the language demands on bilingual learners in mathematical communications. Throughout their discussion, Sam and Mario communicated significant calculus ideas without speaking in long sentences. Rather, by using a combination of rapid-dragging and limited word use, Mario demonstrated his competence to encapsulate the set of all (x,A(x)) and predict the shape of the area-accumulating function. Hence, the use of visual mediators could significantly reduce the number of words or even replace the words to be spoken in a sentence, simultaneously reducing the language demands on bilingual learners.

Prior to the study, Mario was described as a student who seldom participated verbally in whole class discussions. However, the analysis showed that Mario was not as “quiet” as he seemed to be; in fact, he was participating actively together with Sam in their development of the mathematical discourse. This analysis was achieved by adopting a sociocultural view (Moschkovich, 2007), together with attending to the interplay between non-linguistic and linguistic communication. Methodologically, the organisation of the transcript was helpful to illuminate this analysis, by informing the “speaker”, “gesturer”, “dragger” and the words that were spoken while a dragging and gesturing was performed. Although they did not share a common home-language, Sam and Mario communicated effectively with each other using a combination of linguistic and non-linguistic resources. Results like these have implications for day-to-day teaching and learning in classrooms where individual bilingualism is manifested (Planas, 2014). In particular, gestures and dragging enabled individual bilinguals to engage in communication by means other than the verbal language alone. This implication is different from the one previously discussed concerning the language demands on bilingual learners. It is argued here that in multilingual classrooms where learners do not share a common home-language, there is a need for widening the view of language, defined by Sfard as tools for communication, to include non-linguistic tools. Above all, the study points to the use of DGEs and pair-work activities for facilitating meaningful discussion of mathematical ideas and development in one’s mathematical discourse in today’s increasingly multilingual classrooms.

More broadly speaking, the results of the study also argue for an expanded view of communication for monolingual and bilingual learners alike. In other words, focusing on speech alone is not sufficient to fully capture one’s competence in mathematical communication. This calls attention on assessment practices that place excessive emphasis on linguistic (written) features of the mathematical discourse, thereby dismissing opportunities for learners to demonstrate their mathematical learning by non-linguistic means (gestures and diagrams). Although this paper could have been written to argue for an expanded view of communication in general featuring these implications, I have chosen to use a sociocultural lens to focus on a bilingual context, through which bilingual learners’ competence in mathematical communication is highlighted. This line of work is much needed in the field of linguistic diversity in mathematics education to challenge a deficit model that focusses on what bilingual learners can’t do and don’t know. It also opens dialogue for cross-disciplinary work towards a “proficiency-based approach” for minority groups of learners, where much research has been already been done with mathematics learners with special needs (Peltenburg, van den Heuvel-Panhuizen, & Robitzsch, 2012; van den Heuvel-Panhuizen, 2015).

On the design of the sketch, it could be said that the sketch took on an important role of facilitating the students’ mathematical communications. The Hide/Show buttons allowed the students to talk about their ideas gradually, one button at a time; the dragging affordance enabled them to attend to dynamic relationships and connect algebraic with geometric representations of calculus, and the Trace tool afforded the encapsulation of the set of ordered pairs (x,A(x)) into the area-accumulating function. The study is in tune with previous studies on DGE-mediated student thinking: as the students saw the green traces “grow” on the screen, the idea of area as a function is visually mediated (Ferrara, Pratt, & Robutti, 2006; Falcade et al., 2007). Although Sfard has not specifically addressed the distinction between dynamic and static visual mediators in general, the distinction is important for this study because of the potential for dynamic visual mediators such as gestures and DGEs to evoke temporal and mathematical relations (Ng & Sinclair, 2013), particular for the study of calculus (Núñez, 2006). Finally, the touchscreen-based DGE seemed to offer a haptic environment for learners to interact with dynamic relationships with their fingers, where the nature of gestures is re-conceptualised (Sinclair & de Freitas, 2015). Future studies should consider extending the notion of visual mediators and routines to include gestures and dragging on touchscreen-based DGEs. In particular, this paper calls for more studies in the area of DGE-mediated learning to investigate the role of dragging-gesturing in other types of mathematical activities and with other branches of mathematics.