Keywords

1 Introduction

In the semiotics literature, the existence of the sign tends to be presupposed. But new signs come to life continually. How signs are born, however, is much less frequently studied by scholars interested in semiotics. That signs do not just exist around us but actually emerge or are generated in the course of communication became salient to me during a research project when a colleague and I met for an entire week to engage in intense video analysis of physics lectures for preservice elementary teachers. Despite having volunteered to teach this course and despite spending a lot of time on preparing the daily lectures, our research showed that the students had considerable difficulties understanding what the course was all about. Among others, the professor taught about a demonstration that he had shown during some preceding lecture while reviewing the differences between Aristotle and Galileo on motion. As part of his talk about having run a cart on an air track, the professor’s arm moves in what appears to be along a curvilinear trajectory (Fig. 61.1). When the students were tested subsequently, they had no idea what the professor talked about in this lecture (Roth and Tobin 1996). While analyzing this lecture, one of us reproduced the hand movement, thereby creating an iconic signifier of the hand movement we had seen (signified).Footnote 1 Soon, however, there was a shift in the manner we used this hand movement: it began, in our use, to refer to all instances where we discovered that the students in these lectures did not understand. Yet another shift occurred even later in our analytic work, when we used the gesture to denote learning difficulties in the sciences more generally.

Fig. 61.1
figure 1

A physics professor enacts a hand movement while talking about the movement of a cart that he had observed together with his students

Full size image

Looking at this episode, we note first that there is a hand gesture. With it, the physics professor iconically signifies an event to which all of those present in the room at the time had been privy. For a sign to exist, there has to be a signifier–signified relation, which, from the perspective of the professor, exists in the relationship between the hand movement (signifier) and the previous event that they had seen of a cart moving on an air track (signified). There are at least two conditions for this sign to exist. First, the original movement has to be seen—which, according to neuroscience research, requires the capacity to produce such movements with their eyes and other parts of the body. Second, the signifier can exist only as an intentional movement when there exists a form of immanent knowledge of the capacity for such movement (Henry 2000; Maine de Biran 1841). That is, the production of the iconic signifier requires fundamental movement capacities—on the part of the eyes and the body.

Second, we note that the two observers of the episode—my colleague and I—also perceive the hand movement; and they, in turn, reproduce the movement to denote, in an iconic manner, the movement they have seen. This, too, requires, as shown in neuroscientific research, that the observers have the capacity to make the movements with their eyes and body (Gallese et al. 2007). Moreover, we see in this episode a second type of shift, whereby the same signifier first signifies another feature of this university course (students’ problems in comprehending the lectures) and then signifies problems in scientific understanding more generally. There is therefore a shift from a motivated relation between the signifier and signified to an arbitrary relation, where the movement (here of the hand) bears no relation with the signified.

In this episode, we therefore observe an emergence of signs that has already been described by the little known French philosopher quoted in the exergue of this chapter, who was concerned with habits long before these were made popular by William James, John Dewey, or Pierre Bourdieu. Thus, signs as we use them arise from originary signs. For Maine de Biran, the originary sign is actually a self-relation, for the movement only refers to the movement, which coincides with the original impressions of the movement. It is not yet a sign in the traditional sense, where the sign relations between two (three) different entities. But with this single entity that in referring to itself is nonself-identical, we actually arrive at a post-modern conception whereby the originary sign is not a bi- (Saussurian) or tri-valued (Peircean) entity but a unitary entity that is not self-same. In post-modern philosophy, this nonself-identity of the originary sign has been expressed in the emphasis on the bar that separates the Saussurian signifier and signified (Nancy and Lacoue-Labarthe 1992). This idea has been foreshadowed in the Marxian analysis of value: it manifests itself as use value and exchange value, but these different manifestations have the nonself-identity of value as their prerequisite (Roth 2006).

In this chapter, I take up from research on the emergence of new communicative forms, which I have studied in a variety of contexts, including school science (Roth 2000, 2014, in press; Roth and Lawless 2002b), natural scientific laboratories (e.g., Roth 2009; Roth et al. 2002), among scientists in think-aloud sessions (e.g., Roth 2008; Roth and Bowen 2001), and in university science lectures (e.g., Roth 2012). I present these empirical materials in support of theorizing the emergence of signs from hand movements that originally had only work (ergotic) purposes.

2 Evolution of Hand Movements

In this section, I provide a case study of the evolution of communicative forms from a tenth-grade physics course on static electricity. The group that appears in this episode is seated around a laboratory table where one of the two teachers of the unit repeatedly passes by to ask the students about what they have done and learned (Fig. 61.2). The students have conducted an investigation concerning the electrostatic influence on a metal-coated pith ball suspended near the end of a steel rod when a charged plastic ruler is brought near the latter. Physicists explain this phenomenon in terms of a temporary charge separation in the steel rod—achieved by means of electrostatic influence—which itself produces a temporary charge separation in the pith ball in such a way that there are opposite charges on the sides of rod and ball that face each other. This causes the pith ball to be attracted to the steel rod, which it touches, and thereby becomes charged. This charge is given back to the rod after the ruler has been removed when the pith ball successively bounces off the end of the steel rod. The purpose of the task is for students to gain familiarity with the phenomenon and then to produce descriptions and explanations, partially through making inferences from known properties of materials and known physics discourse and partially with the help of teachers.

Fig. 61.2
figure 2

A physics student (Philipp, standing up in center) explains what they have done in an experiment and how to understand the observations by moving with the end of a ruler along a horizontal metal rod that was part of the experiment. The teacher stands up on the left just behind Matthias

Full size image

In this section, I describe how students evolve first descriptions and explanations by reenacting parts (or all) of the investigation. Rather than describing what they have done and explaining their observations, students invite the researcher/teacher “to look,” while they do again what they have done before. Later, still in the presence of the materials and equipment, the hand movements replace actual objects and events in the multimodal narrative. Subsequently, students employ a different object or gesture to signify some relevant aspect of the event, and finally their entire account of the investigation and its explanation are produced in verbal form. I distinguish three types of functions that hand movements have in this development: In addition to the evident symbolic function obtaining to gestures during speech, gestures have epistemic and ergotic functions. As to the epistemic function, the hands (as well as other body parts) permit the person to perceive qualitative aspects such as the temperature, form, texture, or movement of objects. The ergotic function relates to the fact that humans change their environment: for turning, displacing, compressing, or pulling objects. Epistemic and ergotic movements constitute forms of practical thinking. The following three subsections, therefore, exemplify a shift in the modes and modalities of the signs from work related to iconic and symbolic (e.g., Roth and Lawless 2002c).

2.1 From Ergotic to Epistemic Movements

The videotapes of the repeated description and explanation of the investigation and phenomenon show that when asked to talk about the phenomena at hand, students initially rely almost exclusively on redoing the investigations as part of their attempts and as a context for evolving atomic-level explanations for their observations (i.e., the bouncing pith ball). As the associated events are too fast, students simulate the events by moving the objects through the different stages of the phenomenon. This allows them to describe the observed objects and unfolding (simulated) events in real time making possible the copresence of expressive means and aspects of the world.

Philipp (standing up, Fig. 61.2), Matthias (left, Fig. 61.2) and their two peers not featured in the conversation have repeatedly done the investigation where they bring a charged object (e.g., a plastic ruler) close to (or in contact with) the end of the metal rod opposite to the covered pith ball. In this first of six episodes recorded by the camera, Philipp and Matthias talk about and explain what they have done and seen.

Rather than just talking about the original investigation, Philipp actually runs it again. He discharges the rod and brings the pith ball to its resting position (Fig. 61.3a.i) before charging a plastic ruler and bringing it to the steel rod (Figs. 61.2, 61.3a.ii). He uses his right hand to point (with ruler) to the part of the steel rod where there should be a surplus of electrons (Fig. 61.3a.iii). His left hand first point to the end of the rod where “subfluous” electrons would move so that “it” (the coated pith ball) would be attracted (61.3a.iv).Footnote 2 While talking about the attraction, he makes a sweeping movement with the hand parallel to the pith ball–rod axis where this attraction is to be expected. In the second part of this episode (Fig. 61.3b), Matthias articulates an explanation in terms of the movement of electrons and protons. While talking about the electrons, his hand moves along a linear trajectory parallel to the steel rod (literally “away” [Fig. 61.3b.i, ii] from the pith ball at the opposite end of the table) and returns along the same trajectory; the hand then moves forward again while talking about the movement of protons toward the pith ball (Fig. 61.3b.iii). In the final part of the episode (Fig. 61.3c), Philipp takes the pith ball in his left hand and, while providing descriptions and explanations (“a part is transferred to here”), moves it. While the pith ball approaches the rod (Fig. 61.3c.i), he describes what is happening, “then this is coming there,” and then provides an explanation of the event: “a part is transferred to here” (Fig. 61.3c.ii). He moves the pith ball away from the rod and describes, “then it swings back” (Fig. 61.3c.iii); and, while moving the pith ball toward the steel rod, he utters the description “and then gets to it here again” (Fig. 61.3c.iv).

Fig. 61.3
figure 3

Excerpt from a conversation between two students, Philipp (a, c) and Matthias (b). Philipp (a) constructs an initial explanation as they watch the pith ball bounce. Matthias (b) and Philipp (c) resort to gestures over the equipment but without actually charging the different bodies. The arrow indicates the timing of the gesture with respect to the speech. (The “up” arrow indicates the point where utterance and image coincide)

Full size image

There are several dimensions typical for the early stages of communicative competence. First, students use equipment and materials, which they describe in observational terms. Second, their explanations are often scientifically inappropriate. Third, in the early stages, students often speak from the point of view of the inanimate entities involved and thereby portray these entities as animate. Fourth, their communication relies heavily on verbal and gestural deixis. In the following, I elaborate on each of these issues.

First, Philipp reenacts the investigation while providing a phenomenal description of his actions and observations (“discharging,” “hold it here,” “it will be attracted”). His hand movements, therefore, refer only to his hand movements even in this reenactment of what he has done before; there is therefore a self-relation, and this makes this movement an originary sign. As the event unfolds, he begins a first theoretical description against the backdrop of the events seen by those present. His movements are ergotic, bringing about changes in the world that can be observed. However, the resulting events, here the bouncing of the pith ball, are fast—too fast to be described simultaneously. By taking the pith ball in his hand and reproducing the observation but in slow motion, Philipp recreates the perceptual aspects of the events in iconic form at a rate that allows him to coordinate the events with his speech. Here, Philipp’s gestures become epistemic as he takes the pith ball into his fingers and moves it through the observed trajectory (Fig. 61.3c). While guiding the pith ball through its trajectory, he stops the motion (or does slow motion) so that his talk about coming, transferring, swinging back, and returning stays in relative synchrony with the actual position of the pith ball.

Second, given that the students in this episode are to learn physics and have to begin this process with the language currently available to them, it is not surprising that they use words inappropriately and described inappropriate physical events—as seen from the current stand of science. In science education, researchers often make a big deal about students’ misconceptions as if it were a deficiency. This, however, does not recognize the fact that the experiences and discourses available to them in everyday life are the very ground and resource for developing into mature science. Historically, this is the process by means of which geometry became an objective science during the time of ancient Greek, who developed their first intuitions in the course of experiences that were not scientific (mathematical) at all (Husserl 1939). The very point of this chapter is to develop a theory that can explain how scientific explanation can emerge of our incarnate experiences in the material world. In this case, Philipp makes up a new word “subfluous”; and Matthias describes protons as moving in the metal rod which, from the physicists’ perspective is impossible for the nuclei, where the protons are located, are fixed in the lattice of a solid. Although these ways of speaking are not appropriate, they can be seen as first attempts in creating a new form of communication about events that the students learn about and become familiar with at the same time. Whether these new forms actually survive cannot be determined at this point in the events. This is so because language and its purpose arise simultaneously so that any creator of a new language

is typically unable to make clear exactly what it is that he wants to do before developing the language in which he succeeds doing it. His new vocabulary makes possible, for the first time, a formulation of its own purpose. It is a tool for doing something which could not have been envisaged prior to the development of a particular set of descriptions, those which it itself helps to provide. (Rorty 1989, p. 13)

Third, in the early stages of learning about new phenomena and theoretical entities, students’ speech and gestures frequently are from the object point of view or portrayed them as animate. In this excerpt, Philipp talks about electrons that “want to go” some place (Fig. 61.3a.iii) and Matthias suggests that the electrons “put themselves away” and protons “want to go” some place (Fig. 61.3b).

Fourth, the videotapes show that in the early stages of these students’ examination of physical phenomena and their explanations, there is a high degree of verbal and gestural deixis. This allows direct reference to the objects in the world and does not require the abstract forms typical of written texts, which in fact constitutes their world as a different one (Ricœur 1986). For example, Philipp not only uses the deictic terms “here,” “there,” “this,” and “it” but the signifieds of these terms shift even in the brief episode displayed here. “It” refers to the ruler (Fig. 61.3a.ii), pith ball (Fig. 61.3a.iv, c.iii), and steel rod (Fig. 61.3c.iv). From Philipp’s perspective, “there” both refers to the right end of the steel rod where there was an electron surplus (Fig. 61.3a.ii), the left end where there is an electron deficit (Fig. 61.3a.iii), and the right extreme of the pith ball’s trajectory. The same entity also is designated with different indexical terms: when the pith ball approaches the end of the steel rod it is both “here” (Fig. 61.3c.iv) and “there” (Fig. 61.3c.i). Despite these variations and apparent inconsistencies at the verbal level, there is no problem apparent in the students’ communication. With the materials and equipment as ground available to all of them, the respective listeners disambiguate what is being communicated.

2.2 From Epistemic to Symbolic Movements

In a second stage, students use some of the materials from their investigations as ground against which they layer their explanations. A transition occurs as some movement takes on different functions by transcending itself in referring to something that is other than and external to itself. As the introductory quotation shows, originary signs need to be remarked, that is, marked and re/marked, to become secondary signs, that is, signs in the way semioticians use them. This is the beginning of a signifier–signified relation in the way that semiotics scholars are familiar with. In this episode, Matthias evolves another phenomenal and explanatory description of the pith ball and steel rod investigation. In the first line of the episode (Fig. 61.4), he describes how holding the charged ruler to the end of the steel rod repels the electrons that then move to the opposite end of the steel rod. His hand movement has two functions: pointing to the place in the rod where the electrons will go (Fig. 61.4a.ii) and iconically imaging the movement of electrons that are repelled (Fig. 61.4a.i, ii, iii, iv).

Fig. 61.4
figure 4

Matthias provides an explanation by using the equipment as indexical ground. (Colon[s] in or at the end of a word indicate that the preceding phoneme is drawn out by about 0.1 s per colon. Numbers [1.68] indicate pauses in seconds)

Full size image

In the second line of the transcript (Fig. 61.4b), Matthias describes and explains the effect of those electrons that are supposed to be at the (from his position) opposite end of the rod. He suggests that the pith ball (“this uncharged body”) makes an attempt to cancel the charge surplus and therefore pulls itself to the rod (Fig. 61.4b.iv, c.i). His deictic gestures that accompanied verbal deixis (“this,” “it”) make the pith ball stand out as figure against everything else as ground (Fig. 61.4b.ii, iv). The performance of the hand movements, having symbolic function, parallel the articulation of the verbs “equilibrate” and “pull to” as the hand moves from the stretched-out position at the end of the rod all the way in front of him. Toward the end of the episode (Fig. 61.4c.iii, iv), Matthias begins to hesitate about how to complete the description. His explanatory (“equilibrates” Fig. 61.4c.iii) and observational descriptions (“repels” Fig. 61.4c.iv) appear to hang in the air without relating to each other. At this point, he does not yet (as it happens later in the lesson) provide a complete explanation for the process in which observation and explanation are coordinated.

In the evolution of students’ communicative forms, this example is further along than the previous episode where students actually observe the event or move the objects around to be able to describe them and their relations to each other in the various configurations. Here, the original work of the investigation and associated observations are enacted by means of hand/arm movements or described with words. Against the objects present and the verbal description of actions and observations, Matthias presents an explanation. Materials and equipment serve as dialectical ground to/against the explanation; in some cases, arbitrary objects replace them. In the present instance, a pencil (Fig. 61.4a.i) replaces and stands in for the original ruler that he had used to induce electrical charges in the steel rod. The presence of these materials or their substitutes allows students to point to particular aspects without the ne ed to generate verbal signifiers. Matthias does not generally name the iron rod (except in Fig. 61.4c.i), although he repeatedly refers to it. Also, he does not name the pith ball; he brings the object into the narrative by means of verbal (“this” [Fig. 61.4b.ii], “it” [Fig. 61.4b.iv, c.iv]) and gestural deixis.

As in the previous episode, the same indexical items have different referents; because of the logic of the underlying event to be described and explained, these indexicals are therefore disambiguated. Thus, “it” refers to the charged ruler (Fig. 61.4a.i), pith ball (Fig. 61.4b.iv, c.iv), iron rod (Fig. 61.4c.iii), and the explanation itself (“how was it again,” Fig. 61.4c.iii). In the same way, both ends of the steel rod are designated by the deictic term “here” and the opposite end of the rod is also “there” (Fig. 61.4b.i, c.ii). In each case, the use of deictic reference allows the things in the world to stand for themselves, without requiring additional signifiers (representation). At this stage in the evolving explanation, the visible objects and invisible are animate entities and engage in intentional action. Thus, the pith ball “attempts to make… equilibrate,” “pulls itself” or “repels…immediately.” Furthermore, the rod “equilibrates” and the electrons “repel each other.”

Toward the end of this episode, Matthias has become more independent of the material ground, though he still makes use of indexical words and gestures to designate the things at hand. As he is co-present with these things, he does not need to fully describe them: They go without saying and it suffices to designate the relevant objects, entities, and events by pointing to them or expressing them by means of an iconic hand movement. Moving entities constitutes a form of thought and eliminates thought forms mediated by signs. That is, as in other workplaces, the perceptual gestalts have their own communicative force and do not require additional verbal articulation or elaboration (Roth 2004). Against the material ground, the emerging explanation takes its hold. In these first episodes, speakers take the point of view of the entities with which they themselves are co-present in the situation. In the evolution of communicative forms, even scientists find it easier to talk about the objects and entities as if they are imbued with agency and to move to a dispassionate and nonanimate perspective at a stage when very familiar with the phenomena (Fox Keller 1983).

2.3 From Symbolic Movements to Speech

The videotapes show that when students become familiar with the objects, equipment, and phenomena produced with them, they no longer require the presence of the materials when they give their explanations. At this point, arbitrary objects serve as signifiers that stand for some object or entity. Thus, toward the end of the second lesson on the steel rod-pith ball investigation, Philipp produces an explanation in which he uses a polyvinyl chloride (PVC) rod in place of the steel rod. He produces observational and theoretical descriptions against the arbitrary PVC rod, which is a placeholder for the materials and equipment of the investigation as a whole. He presents both the phenomenal events and the conceptual entities and their relations (Fig. 61.5). Philipp shows with his right hand where the (charged) ruler is held relative to the rod and, with his left hand, how “everything” is repelled inside the rod toward the other end (Fig. 61.5a.i, ii). As a consequence, the pith ball moves to the end of the rod and then moves away again. At the same time, there are events that his hand movements reenact over the rod and that therefore are attributed to the steel rod in the original investigation. That is, here we have the sign in its full form, where one thing stands for another. Philipp’s right hand moves while he talks about “something” (or “they”) that moves within the rod (Fig. 61.5b.i–iii, c.iv, d.iv), which apparently explains the intimated movement of the pith ball enacted twice with the left hand, following a similar initial presentation with the right hand (Fig. 61.5a.iii, b.i).

Fig. 61.5
figure 5

Philipp is in the process of constructing an explanation for induction on the steel rod and the pith ball so that they can describe it in their laboratory notebook. Philipp picks up a PVC rod from his desk and then enacts his explanation using gestures over and about the rod

Full size image

In this episode, the PVC rod stands in for the steel rod that had been part of the original investigation; it is a signifier for the steel rod (signified). Resembling the steel rod in shape and size, the PVC rod bears an iconic relation to the object it signifies. In this, its right end corresponds to part where the ruler is approached to the metal rod, and the left end corresponds to that part where the pith ball is found. The atomic level events are gestured to occur inside the rod. The articulation of what happens at the atomic level after the pith ball is initially attracted and repelled is crucial and also the most difficult part of the explanation. Here, Philipp does not yet have the means for a verbal description, but his hand movement enacts a process according to which there is a cyclic process of movements (perhaps equilibration) across the rod (Fig. 61.5c.iv, d.iii). Changing to signifying the pith ball with the left hand while depicting the atomic-level movements with the right hand constitutes the current thought, which I understand, following Vygotskij (2002), inseparable from this multimodal, inherently external presentation (e.g., Roth 2010). As in the previous episodes, he attributes agency to the objects and entities.

The entities used together with the hand movements obtain symbolic function. They track objects and entities and therefore do not need to be signified in other form. Thus, once the PVC rod is designated as a signifier of the steel rod, then no further elaboration is necessary. It is perceptually available to all participants in the setting where it is available for future reference and as long as it is needed. A sign function is established in the here and now and on the fly, enabled by the shared background in regard to the phenomena (equipment, observation) to be explained. As such, these objects also serve as indexical ground to symbolic hand movements without requiring further description.

3 Emergence and Evolution of Sign Systems

Previous research has shown that once there are transcendental signs, there is a shift in modes and modality in which signs are produced as students become increasingly familiar with some perceptual field (Roth and Lawless 2002d). My fundamental question is about the requirements for such an emergence that is followed by reproduction and transformation of the signs. Thus, I am concerned with the emergence and evolution of communicative forms that goes beyond accepting the existence of sign systems that children and students appropriate in the course of their participation in collective, cultural life. In the introductory episode featuring two researchers and in the episodes from the high school physics class, new signs that did not yet exist in general culture emerge and are used. These new signs emerge from initial hand movements, which, in the present study, are proposed as the originary, immanent signifiers that only refer to themselves (signified). Or they may emerge, when some object takes the place of another because of an iconic relation, and then may become part of an abstract sign when the signifier points to something that only bears an arbitrary relation. The originary signs, however, exist in and as a self-relation—steel rod standing for steel rod, ergotic/epistemic hand movement standing for the same movement—rather than as the external relation between two material bodies, one constituting the signifier, the other constituting the signified.

3.1 Immanent Nature of Originary Signs

In the episode from the tenth-grade physics course, we observe movements that I characterize by means of the adjective “ergotic”; these constitute the originary signs. These movements are designed to get some work done—though in subsequent stages they may also have semiotic (symbolic) function. Thus, for example, to do the investigation, Philipp has taken the plastic ruler, rubbed it on a piece of cloth, and then approached it to the steel rod on the end opposite to where the pith ball is suspended. When he enacts these movements, it is to produce the investigation so that it affords observation; his movements do not (yet) have symbolic function. Initially, when the teacher asks the students to describe and explain what they had done and seen, Philipp asks the teacher to look and then does what he has done before. That is, Philipp reproduces a movement for the purpose of getting the same work done. That is, he enacts work-related movements that stand for themselves in that they constitute a nonself-same repetition of an earlier movement. In the introductory quotation that opens this chapter, this movement, once it is reproducible, is referred to as a natural or originary sign. It does not involve, however, a signifier that stands for a differently embodied signified (another signifier); rather, it signifies itself (Maine de Biran 1841). Once the flesh immanently knows to produce a movement and even without any form of deliberate consciousness of this movement (e.g., in the walk, ride a bicycle, play a sport), the movement may recall itself (it has become a habit, and, according to Maine de Biran, the basic form of thought). Thus, once Philipp has learned to bring the plastic ruler in the proper way to the end of the metal rod, he can produce this movement over and over again.

A reproducible movement constitutes a force that stands for itself (Maine de Biran 1841). But this self-signification should not be read in terms of a self-identity. The movement recalls itself but is not precisely the same—athletes would not get better at their sport with practice and would play with machine-like precision rather than creatively adapting their movements to the situation. Thus, the “reproduction of the Same is not a motor of bodily movements. We know that even the simplest imitation involves a difference between inside and outside” (Deleuze 1968, p. 35). Movement therefore has the fundamental structure of a signifier–signified relation but within an entity that is not self-same. With repetition, the movement becomes habit, where the “essence of habit” is understood as “an acquired predisposition to ways or modes or response, not to particular acts except as, under special conditions, these express a way of behaving” (Dewey 1983/2008, p. 32). This means that the movement is able to refer to itself as movement intention, an expression that is not meant to introduce transcendent forms of intentionality but leads us to the very predisposition that allows us to move in a particular way.

When the originary sign is defined in this way, the difference between signifier and signified is actually included in an original unity, which, therefore, is a unity of multiplicity. Here, difference comes to be internal to the idea of the sign and therefore “unfolds as pure movement, creative of a dynamic space and time which correspond to the Idea” (Deleuze 1968, p. 24). In this manner, the sign as I define it here “comprehends difference, and comprehends itself in the alterity of the Idea, in the heterogeneity of an a-representation” (p. 24).Footnote 3 It is precisely because of this self-relation within the originary sign that there can be an external relation between its two (transcendent) manifestations (signifier, signified).

3.2 From Immanent to Transcendent Signs

In subsequent explanations, there is a slight change. Philipp brings the plastic ruler close to the steel road for the express purpose of showing what he has done. Here, then, a hand movement that initially has been part of charging the metal rod now is used to symbolize the charging of the road. Rather than just changing the world by means of ergotic movements, he reproduces these same movements for symbolic means. The originary signs, movements pointing to themselves, which come about as auto-donation (Henry 2000), now have expanded to secondary signs that are characterized by their transcendence and by their external relation to the thing that they stand for. This first transition from the immanent body to the transcendent (felt) body constitutes an auto-revelation. This comes about as the individual

transforms in this way the first into artificial or secondary signs, and multiplies his communicative means, either on the outside or with his own thought. He does more, he communicates to the most transient modifications a part of the availability of his movements, forces them to enter the sphere of his memory, and sort of creates the terms or motives of his will, where there existed none of any kind. (Maine de Biran 1841, pp. 56–57)

At this point, therefore a secondary form of sign has emerged, where the hand movement is enacted independently of the work but refers the listener and speaker back to the work situation: The movement transcends itself and no longer is immanent to itself. Whereas in the former situation the movement recalls itself without requiring consciousness, as an immanent form of knowing, here we have a transcendent form of the sign and, therefore, a transcendental form of knowing. The movement now has become symbolic, used for the explicit purpose of referring to a situation other than the present one. It now makes present again a past present: it re-presences or represents. But the material still is the same, the body/flesh moving the hand/arm combination in a particular manner. A fully transcendental form is achieved only when the material making the signifier is different from the material making the signified—in other words, when one part of the material continuum is used to refer to another part of the material continuum (Eco 1984). But importantly, the transcendent form is not possible without the initial capacity of reproducing the movement, which differs from itself in reproduction and therefore also constitutes a transformation. This nonself-identity (nonself-sameness) is a requirement for any form of signification, for any sort of memory produced by a living organism.

The distinction made here has its equivalence in the three ways in which the body appears: in immanent (“original/originary flesh”), transcendent (felt, “constituting body”), and external form (“constituted body”) (Henry 2000). The development from the first to the third constitutes a movement of “auto-exteriorization of exteriority that places everything outside of itself, stripping it of its own reality, everything that reveals itself in it is marked by the principle of the seal of irreality” (p. 120). The secondary signs, however, are not independent but they take their functional structure from the primary signs. Thus, “the artificial signs are but grafted, so to speak, onto the natural signs” (Maine de Biran 1841, p. 57). There is therefore a close association between the secondary signs—which tend to be the objects of interest in semiotics or those of (science, mathematics) educators concerned with the development of abstract forms of thought—and hands-on experiences. In the field of education—as in cognitive science and artificial intelligence—the relationship between hands-on experience and higher order concepts not only is little understood but also creates a problem, because ideas appear to float freely and there are questions about how ideas generally and symbols more specifically are connected to the everyday material world.Footnote 4 As shown here, the primary signs, those existing in and as movements, are foundational to the development of our first forms of thought and for any more advanced forms of thought that are premised by and build upon our first ideas.

Visual perception is an integral part of the episodes. Thus, for example, Philipp refers in different ways to the movement of the pith ball: First to the steel rod and then, following its initial contact, repeatedly moving away and toward it. One way in which Philipp refers to the movement is by reproducing it or an iconic version of it. For example, in one of the explanations (Fig. 61.3c), he grabs the pith ball, moves it toward the steel rod until it makes contact, then moves it away only to return for a second time. In this way, he reproduces the movement but this time moving the pith ball rather than allowing it to react to the electrostatic forces that would act without his interference. The movement also is symbolically reproduced, for example, when Matthias points to the pith ball and then moves his hand parallel along the steel rod while saying that it pulls itself followed by the reverse movement while talking about the ball being repelled after contact with the rod (Fig. 61.4b, c). In this instance, they report a visual-perceptual experience in iconic form. Here, the forces that produce the movements differ: In the investigation the force is a natural one whereas in the explanations it is a life force. But considered as an instant of thought, the symbolic movement coincides with the natural movement. We therefore have the beginning of an answer for the “difficulty constantly facing every theoretician [which] lies in understanding what it is that links knowledge… and its subject matter together” (Il’enkov 1977, p. 16). In these symbolic hand movements, we therefore overcome the Cartesian divide: There is but “one single object, which is the thinking body of living, real man” (p. 31).

Now it may appear that the hand movement and the eye movement have little to do with each other; but this is not the case. Initially described at the beginning of the 1800s, it is now well established in the neurocognitive sciences that without bodily movement in the world, we do not learn to see (Varela et al. 1993). But, “as soon as vision is intimately associated with all the operations of the exercise of motility, it extends itself to the distance” (Maine de Biran 1841, p. 62). This leads to the situation that for the individual

the play of his imagination fuses itself, confounds itself with that of the external sense, without that he could in most instances distinguish their products; he simply believes to see, sense, and he imagines, he compares, he acts even as a consequence of multiple judgments of which he is not actually conscious. (p. 63)

That is, the perceptual mode is directly modeled and dependent upon other movements that displace the organism or move parts of its body (e.g., to reach for something). Thus, whether the students use a metal rod or a PVC rod, the eye movements still are the same, following the elongated object. One rod can take the place of the other because the apperceptive eye movements are the same (but not self-same). The students would not be able to see the movement whether there did not already have the immanent capacity to produce this movement with some part of the body. As neuroscientists showed, “the mere observation of an object-related hand action occasions in the observer an automatic activation of the same neural network active in the person performing the action” (Gallese et al. 2007, p. 133).

3.3 From Motivated to Arbitrary Signifier–Signified Relations

The final step in the development is taken when an arbitrary signifier—different material and form—takes the place of the perceptually motivated signifier. For example, when Philipp withdraws the pith ball from the metal rod and approaches it (Fig. 61.3c.i), the movement depicted by his hand also is described verbally. Whereas the symbolic hand movement already has transcended the original phenomenon, it is still bound to the original movement, if not materially then by means of form. But, being transcendent, it already is outside of the original movement. The arbitrary relation becomes possible because the felt body is already a transcendent one, which makes available the sensible world that transcends the person and becomes independent of it (Henry 2000). As the hand movement has taken on transcendent character and therefore symbolic function, it can now be viewed from the outside, is a movement objectively available to others, who, in their own bodies, may reproduce it. The movement, thereby, has become independent of the immanent and of the felt, constituting body. This has as consequence that the movement can also be referred to by something that no longer bears the same material or form. As There is an interlacement of the constituting (body) and the constituted world, as shown in the phenomenological investigations where one hand explores the world and is itself explored by the other hand (Merleau-Ponty 1964). Both are transcendent, but the one is felt from the inside, the other felt as being on the outside. It is this transition from the inside to the outside transcendent forms that constitutes the difference from the iconic to the arbitrary relation. As my opening example shows, once the hand movement of the physics professor stands out and is reproduced intentionally on the part of the researchers, there is only a small step to use the movement, qua transcendent (secondary) sign to refer to something else or, conversely, to be referred by other semiotic means, for example, as a “curvilinear left-hand gesture.” It is the first step in the metaphorization from bodily movements to the world of ideas that is said to underlie all conceptual systems (Johnson 1987; Roth and Lawless 2002a).

That a transition occurs does not mean that it is an easy one in every instance as shown in research that highlights the difficulties of professors having to produce alternate verbal descriptions for the line graphs that they have produced for a lecture (Roth and Bowen 1999a). That is, precisely because arbitrary signifiers bear no relation other than conventional ones to the signified, even experts may be hard pressed translating between different sign systems to denote the same natural phenomenon (Roth and Bowen 2003); considerable incarnate coordination work is necessary to make and stabilize this connection (Roth and Bowen 1999b).

4 Coda

In this chapter, I provide empirical examples of the emergence and transformation of sign forms from initial ergotic hand movements. I develop an explanation for this development that is grounded in a theory of the incarnate body (flesh), which auto-donates immanent originary signs that are subsequently transformed into transcendent secondary signs that historically have been the focus of semiotic studies. This work is of tremendous importance to those areas of education interested in supporting cognitive development through hands-on experiences.