The Integration of Dual-Interaction Spaces

Abstract

Dual-interaction spaces—that combine text chat with a shared graphical work area—have been developed in recent years as CSCL applications to support the synchronous construction and discussion of shared artifacts by distributed small groups of students. However, the simple juxtaposition of the two spaces raises numerous issues for users: How can objects in the shared workspace be referenced from within the chat? How can users track and comprehend all the various simultaneous activities? How can participants coordinate their multifaceted actions? We present three steps toward integration of activities across separate interaction spaces: support for deictic references, implementation of a history feature and display of social awareness information.

The construction, modification, annotation and arrangement of shared artifacts are key activities in many collaborative learning settings. Software systems now exist that permit synchronous coordinated manipulation of such shared artifacts even for geographically distributed users, by providing a shared graphical workspace. A shared workspace in a collaborative environment is an area of the software interface that allows a participant to construct and manipulate a graphical object so that the object and the effects of the manipulation appear in the corresponding area of the other participants’ interfaces, essentially in real time. These shared workspaces may be used for creating and using external representations of knowledge (Whittaker, 2003), for collaboratively completing design tasks (Reimann & Zumbach, 2001), for working together with simulations (Jermann, 2004; Landsman & Alterman, 2003), or for solving math problems, as in VMT. The design of shared workspaces is an important topic in computer-supported collaborative learning (CSCL).

Learning at a distance requires a medium of communication. The medium can be auditory, audio-visual or text-based. For collaborative learning, textual synchronous communication with chat has two main advantages over audio and even face-to-face: For the chat poster, writing encourages a more careful planning of one’s contribution; it fosters reflection on the discourse. For the recipient, the communication is persistent and available in symbolic form that “may be searched, browsed, replayed, annotated, visualized, restructured and recontextualized” (Erickson, 1999).

The combination of a shared workspace with chat makes two regions for interaction available to a group in the form of a dual-interaction space (Dillenbourg, 2005). The chat provides a medium of communication for the exchange of textual messages; the shared workspace allows for the collaborative construction and manipulation of shared artifacts that are relevant to the task at hand. In most groupware systems for synchronous distance learning, the chat and graphical workspace simply appear next to each other as two visually distinct areas of the application that are largely functionally independent of each other. This introduces a number of problems for the users (Pata & Sarapuu, 2003; Suthers et al., 2003; van Bruggen, 2003). For instance, if a group of students want to create a concept map in the shared workspace consisting of arguments pro and con and their relationships to each other, this raises the following questions:

• How can objects and relationships within the workspace be referenced from a posting in the chat area?

• How can the participants grasp and understand the relationships among each other of the activities and messages that are part of a single collaborative interaction but are distributed across the two interaction spaces? For example, how can one establish that the message, “I agree,” is a response to the introduction of a particular new node in the argumentation graph?

• How can the participants coordinate their actions in the graphical workspace and in the chat with each other? For example, when and by whom should an argument introduced in the chat be added to the graphical concept map?

A better software integration of chat and workspace is needed to overcome such difficulties (Dimitracopoulou, 2005; McCarthy & Monk, 1994; Suthers, 2001). But from the perspective of software design the question, which functionalities must be provided to support the collaboration in dual-interaction spaces, remains unanswered; the claim for better integration is too general to guide the design of the learning environment. This became apparent in the workshop “Dual-interaction spaces” at CSCL 2005 in Taipei organized by Dillenbourg (2005) and the CSCL SIG of Kaleidoscope.

In this chapter we propose integration measures for three relevant aspects of the connection of chat and shared workspace:

• deictic referencing,

• coordinating simultaneous activities and

• understanding of past interactions.

These problems are analyzed in the next section. In a third section we will describe the integration measures. Then we will present experiences with ConcertChat—a collaboration tool that implements these measures and is part of the VMT environment.

For the sake of simplicity this chapter describes our development of the integration measures as a linear process starting with problem analysis that leads to certain functionalities. As we know from CSCL research, this idealized development seldom holds. Our system was developed over five years. We started with assumptions of what is needed by the users, developed first prototypes and used them in serious learning settings. The analysis of those real collaborations provided us insights into the complex nature of mediated collaborative meaning making in dual-interaction spaces. Our focus gradually shifted from an individual point of view (what is needed by a user) to a group cognition (Stahl, 2006b) perspective taking into account the creative, simultaneous, interwoven interactions among the team members.

Problems in Combined Interaction Spaces

A shared workspace can play at least two contrasting roles within a collaborative session. It can, for instance, provide the central location for the joint activity of the participants, with the chat playing a supportive role in discussing and disambiguating the activities that take place in the workspace. Conversely, the chat discourse can dominate, with the graphical workspace serving as a resource for clarification or for illustrating things that are hard to articulate in words. Which way communication is divided between the dual spaces depends upon the current task, the meta-communicative skills of the participants and the respective affordances of the two media (Dillenbourg & Traum, 2006; Pata & Sarapuu, 2003). The activities in the chat and the shared workspace are typically intimately interrelated. To the extent that the technology supports it, participants may coordinate their use of the dual spaces in creative and subtle ways (see Chapters 7 and 17).

A prominent characteristic of chat is the delay between the production of a message by its author and its presentation to others when it is complete. This has two main advantages: that the author can revise the message before sending it and that several people can be producing messages at the same time, unlike in spoken conversation (see Chapter 14). However, it also leads to the constant danger of sequential incoherence, which forces the participants to work additionally on explicitly coordinating the content and structure of their interactions (see Chapter 21). The problem is that, unlike in conversation, in chat the appearance of responses often do not immediately temporally follow the messages to which they are responding. The coherence of interaction is highly dependent upon the response structure between messages. But in the time it takes for someone to prepare and send a response to one note, a note from someone else can be posted, causing “interrupted turn adjacency” (Herring, 1999). A number of specific communication strategies may be evoked to deal with this (Fuks, Pimentel & Lucena, 2006; Lonchamp, 2006; Murray, 2000). In order to minimize the delay in responding, mistakes in syntax and wording are accepted and many abbreviations or acronyms are used (Garcia & Jacobs, 1999). Cohesive devices like explicitly naming the addressee of a contribution (Nash, 2005) are used to make references explicit.

The fact that several people can be producing messages at the same time means that the common conversational rules of turn taking (Sacks et al., 1974) do not apply. The resulting parallelism can scarcely be avoided, and must particularly be taken into account when multiple topics are discussed simultaneously.¹ This problem is eased by the fact that the flow of chat is documented in the persistent transcript, which is visible—at least for the last several postings. The chat window serves not only as the location of communications, but also as a representation of the temporal order of the messages. In contrast, the graphical workspace usually only shows the current state. All information about the actions and actors who brought about this state is ephemeral.

These problems resulting from the visual and functional juxtaposition of chat and workspace have the consequence that it is hard for users to track and specify relations of content and sequentiality between the textual contributions and the graphical activities. Specifically, there are three major problems:

Deictic references. An important means of communicative expression during collaboration with shared workspaces is deixis (Barnard, May & Salber, 1996; Clark & Wilkes-Gibbs, 1986)—the referencing of objects, relations and actions in the shared visual environment. When chat is used as the communication medium, deictic referencing is associated with high production costs and potentially also higher levels of ambiguity because gestural pointing is not possible. Purely textual descriptions of the object or of its specific position are obvious solutions, but there is no guarantee that such a description will be intelligible to others when they receive it because another user of the shared workspace may have moved or even deleted the object in the meantime.

Decontextualization of actions and messages. When collaborating in a dual-interaction space, participants interact with each other through chat messages and modifications of artifacts in the workspace. Whereas the persistent chat history represents the complete sequentiality of the discursive contributions, the same does not hold for the workspace. Both the ordering and the intermediate results of actions in the shared workspace are fleeting. This has two direct consequences. First, the necessary context for interpreting messages that reference artifacts in the workspace can quickly disappear. This defeats the important advantage of the persistent discourse history, which can support retrospective reflection. Second, the phenomenon of interrupted turn adjacency, described above, is heightened. During the time it takes for one person to respond, others can not only insert new messages but also modify referenced graphical artifacts.

The coordination of communication and interaction. In a dual-interaction space, different participants can simultaneously be typing and posting chat messages or producing objects in the workspace. In collaboration, these various activities are interrelated: a message can announce or comment upon an action in the shared workspace and a workspace action can respond to or clarify a chat message. The awareness of the activities of the other people is a prerequisite for the construction of common ground (Dillenbourg & Traum, 2006). In chat, the chat history documents the sequence of discursive activities of the participants and the usual system messages when someone enters of leaves the room provide basic information about who is present. A series of interface features have been established to support coordination in shared workspaces (Gutwin & Greenberg, 2002), helping with turn taking and the anticipation of actions by other participants. For instance, objects that were just selected by users might be color-coded to indicate who is using them and the location of the user’s mouse can be indicated (Stefik, Bobrow, Foster, Lanning & Tatar, 1987). Similarly, many chat systems display a message near the chat input area if someone is typing. However, if all these awareness techniques are combined in an environment with dual-interaction spaces, then they can overwhelm the limited attentional abilities of humans. The fleeting awareness messages scattered across the interface require users to pay constant attention to their whole screen.

Support Through Integration

People collaborating in a dual-interaction space are exposed to a series of problems that derive from the visually and functionally separated nature of the chat and workspace components. Three software mechanisms will now be presented that integrate these components with each other:

• An explicit referencing tool that makes possible deictic references from the chat to the workspace.

• An integrated history function that documents the on-going collaboration process consisting of the activities in the chat and in the shared workspace, and lets users review it.

• A visually integrated social awareness display that supports the perception of the simultaneous activities of the multiple participants in both areas.

To illustrate these integration measures, a shared whiteboard will be described as a common workspace for the collaborative creation of drawings, concept graphs and mind maps. See Fig. 15.1 for an example showing the most important interface elements. This screenshot shows the state of the VMT interface after the posting of a message with an explicit reference to a textbox in the shared workspace. Rtoledosj is currently working on the large textbox while Euclid is typing a chat message. The interface features for showing explicit references, the workspace history and awareness messages have been annotated.

Fig. 15.1

Functionality in the VMT interface

Mechanism 1: Explicit References

The concept of explicit references addresses the difficulty of deictic referencing in the textual medium of chat (Pfister & Mühlpfordt, 2002). Pointing gestures are frequently used in face-to-face conversation (Bekker, Olson & Olson, 1995), for instance to identify objects and to clarify relationships among objects. Similarly, explicit references in chat allow one to associate a chat contribution with objects in the shared workspace and with other chat messages using graphical connectors. A graphical reference to a chat message can point to the whole message, a single word or some portion of the message. A reference can also point to an object or a region in the workspace. In the simplest case, one might want to point to a particular object, but in other situations to just a specific part of the object or else to a spatial constellation of several objects. So a number of different forms of referencing must be supported.

For summary statements in the chat—e.g., “These two arguments contradict each other”—multiple references can be made to relevant messages and objects. Just as with gestural pointing, the effective meaning of a graphical reference is given only once both the gestural and verbal messages are given. Thus, a reference can be used to clarify a “response-to-that-message” relation as well as to indicate a “related-to-this-object” relation.

The usability of an explicit referencing tool depends upon its effect on the media-dependent costs of production and reception (Clark & Brennan, 1991). In order to keep these costs low, appropriate interaction possibilities must be available for the easy production of references and for the visualization of references.

In order to maintain the chronological order of the chat history—rather than threading it—with the associated advantages for retroactive reflection, a reference is represented by a graphical arrow going from the referencing chat message to the referenced object or message. As soon as the referencing message is displayed, the accompanying reference arrow is also displayed, as illustrated in Fig. 15.1.

Mechanism 2: Artifact History

In collaboration in dual-interaction spaces, the actions in the shared workspace and the messages in the chat are but two facets of a single activity. While the chat displays a persistent history of the collaborative discourse, there is no corresponding history display for the workspace, let alone an integrated history for the whole collaboration. In technical terms, an artifact history of the objects in the workspace is a chronological collection of the various different versions or circumstances of the workspace resulting from the manipulations of the participants. In a shared whiteboard, every creation, movement and editing of an object changes the state of the workspace. The provision of an artifact history has two goals: to preserve the workspace context at various times and to represent its evolutionary process. The context of the workspace at the time when a chat message was being produced is important to know in order to interpret the message—particularly if the message explicitly references artifacts in the workspace. The artifact history permits the reconstruction of that context and encodes that context in the software representation of the reference. As needed, the historical context corresponding to a message of interest can be reconstructed and displayed. The other goal is to allow the normally fleeting artifact history to be replayed. The chronologically ordered developmental steps can be played back like the frames of a film, making possible reflection on the whole collaborative construction. Reflection in the group discussion is facilitated by the combination of being able to review the past developmental stages of the shared workspace and being able to point to a particular stage with an explicit reference.

Mechanism 3: Integrated Activity Awareness

The integration of activity displays has the goal of making it easier to be aware of the simultaneous activity of the other participants. Awareness of these activities is a prerequisite for constructing and maintaining a mutual understanding of the chat messages and the changes to the graphical artifacts—and therefore provides a necessary foundation for collaboration. In a chat environment, the chat history documents all the activities—both the individual messages and information about participant presence. This chronological documentation of activity suggests that it could serve as a representation of all activity within a dual-interaction space as well.

With chat, the process of producing a message is not directly perceivable by the other participants. The extent to which a long lasting and cognitively strenuous activity in a shared workspace is observable for the other participants depends upon the nature of the workspace and the granularity of the operations that are displayed for everyone. For instance, the editing of a textbox annotation in the shared workspace may only become visible for the others when the edit is completed. Activity awareness notifications have been established to support the coordination of activities like joint editing, so someone knows not to try to edit an object that someone else is currently editing. In a dual-interaction space, however, it is necessary to visually integrate these notices that are associated with the locations of different individual activities. If one participant wants to post a chat message in response to a contribution from another (such as responding to an annotation in the shared workspace with: “I would say that differently”), then she might hold off doing this if she is informed that he has just begun to make a change in the workspace that might very well serve to clarify his original contribution. Conversely, if he is informed that she is typing a chat message, he may delay his change in anticipation of a new objection. Both cases of course presume that the information about the activities is perceived. This can be supported by displaying the awareness information at the appropriate location (see Fig. 15.1).

Integrated Dual-Interaction Spaces in Use

The described integration measures are implemented in ConcertChat, an open-source dual-interaction system developed by the chapter authors and colleagues in Germany; it has been adopted and adapted in the VMT Project. A detailed case study of how deictic referencing was conducted in this context using the ConcertChat functionality in the dual-interaction space is presented in Chapter 17. Further studies of the use of ConcertChat’s explicit referencing tool are reported by Mühlpfordt and Wessner (2005). These provide some evidence that the participants were able to employ effective communication strategies with the help of the explicit referencing.

For researchers, the persistence of all activities in a dual-interaction space provides the possibility of conducting fine-grained analyses of group interaction, as demonstrated in this volume. To support this, a Replayer version of ConcertChat has been developed that allows all the activities to be repeatedly reviewed, with the chat and workspace histories precisely coordinated. As mentioned in the introduction, the in-depth analysis of collaborative meaning making of groups learning together in the ConcertChat environment provided us insights in how the functionalities are used. The next three examples illustrate that.

The three examples are taken from the VMT Spring Fest 2006 (discussed in Chapters 7, 8, 10, 11 and 26). The collaborative context was set by organizing a contest: members of the most collaborative teams would win prizes. Students were recruited globally through teachers who were involved in other Math Forum activities. The teams in the excerpts consisted of students from Singapore (example 1) and from the US (Examples 2 and 3), as well as a facilitator from the Math Forum, who provided technical assistance. At the beginning of the first sessions the facilitators briefly explained the functionalities of the learning environment to the groups. Pedagogically, the topic for discussion was an open-ended exploration of geometric patterns. An initial pattern of squares formed from sticks was given. The students were to figure out the formulae for the number of squares and the number of sticks at stage N first, and then explore other patterns that they or other teams invented.

Example 1

The first example illustrates how the referencing tool is established by the group to ease deictic references. Figure 15.2 shows a screen shot of a VMT session with four participants, Amanda, Clarice, Wang and Dshia. The chat is reproduced in Log 15-1

Fig. 15.2

Explicit referencing must be learned

Table 1 Log 15-1

In this interaction the group reflects on what aspects of the mathematical problem at hand they already solved. Wang asks “so how many formulas have we come up with huh?” and both Amanda and Clarice respond in the subsequent messages. Here the interesting response is the textual graphic from Clarice: “<—-”. With that she textually simulates an explicit reference. In contrast to other group members, Clarice has never used ConcertChat’s graphical referencing tool before, so it might be that she does not know how to create a reference with it. Wang’s reply with two question marks (“??”) indicates a lack of understanding. Amanda, while providing an interpretation (“I think she meant look on the left at the text box?”), also closes the message with a question mark. With her subsequent message (“in the text box”), Clarice again tries to establish a reference to the textbox on the shared whiteboard. Amanda finally translates this into a posting with an explicit reference to the textbox with all the collected formulas.

Example 2

While Clarice is a novice in using the referencing tool, Bwang—in the second example—uses it creatively to incorporate a formula written on the shared whiteboard into his explanation of a derived formula for the number of white squares in the rectangular pattern on the left (see Fig. 15.3). In a first step he refers to an already found formula for the number of squares in one corner (“we can use the equation from session 1” and “n(n+1)/2”). Then in a second step he extends that to the number of squares in all four corners. This number must be subtracted from the number of all squares in the pattern. The group already found a formula for the latter number and documented that in a textbox on the whiteboard (“big square: (2n-1)/2”). Bwang’s posting of the final formula is linked to that box. In this case, the referencing tool is used not merely for a deictic reference, but for incorporating an intermediate step in his formula derivation.

Fig. 15.3

Bwang uses an explicit reference

Example 3

The third example is from the same group of students (see Log 15-2 for the excerpt of the chat log) and shows that for the groups it is sometimes not trivial to choose the appropriate interaction space. In line 1516 Aznx invites the others to “simplify their formula” (he is actually referring to a formula published by another group) and after Bwang’s request (“how did you simplify it,” line 1525) he posts five chat messages describing the transformation of the formula. But his team members Quicksilver and Bwang seem not to understand that (“im lost,” line 1533). Aznx now switches to the whiteboard (“I’ll do it on the board,” line 1536) and uses it for writing down the derivation. Figure 15.4 shows a screen shot of his final drawings. It also shows that Aznx’s drawings (each drawing step is indicated by a small square in the chat history on the right side) are interwoven with chat postings, even from himself (line 1542). The interactions of the group are distributed over both interaction spaces, but highly interrelated. In line 1546 (“whyd u multiply by the two”) we can see how the referencing tool is used by Quicksilver for establishing referential identity.

Fig. 15.4

Screen shot after message 1546

Table 2 Log 15-2

Conclusions and Future Work

The design of dual-interaction spaces for synchronous collaborative learning has to take into account the dynamic, tightly coupled and interwoven nature of the activities that are scattered across both media: the chat and the shared workspace. This demands (a) support for deictic referencing, (b) access to an integrated history and (c) integrated activity awareness. We exemplified the advantages offered by such integration measures.

Software developers like to think in modules, but when combining a shared workspace with a chat into one collaboration environment we have to think holistically about using the workspace in the context of a chat conversation and chatting in the context of working together in the workspace.

The experiences with ConcertChat to date suggest a series of further research questions:

• The storing of explicit references and the integrated representation of all activities make available additional structural and temporal information about the collaborative artifacts in the two interaction spaces. To what extent is it possible to use this information to construct a retrospective indexing, documentation or summarization of the collaboration that would facilitate future reflection or recall by the participants—for instance, when they return to the room for a subsequent session?

• An essential difference between a chat window and a shared whiteboard is the persistence of the artifacts. While a textbox in a shared whiteboard remains visible indefinitely (unless it is edited or deleted by a participant), the same is not true for chat contributions; they scroll out of sight with the appearance of the following discourse. Interesting questions arise when the additional possibility of audio communication offers a non-persistent medium. Can this supplementary mode of communication be substituted for chat to the advantage of the participants or will it be used as a secondary addition? What different communication strategies would result?

• How can the concepts of explicit referencing, integrated activity awareness and artifact history be applied to multiple interaction spaces, in which the collaboration environment provides even more than two primary workspaces?