Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

One advantage as well as challenge of multivocal approaches to analysis of collaborative learning interactions is that it reveals the ways in which our individual operationalizations of complex constructs are limited. In bringing together analyses from multiple perspectives addressing similar issues with the same dataset, our eyes are opened to the richness and complexity of how these constructs are manifest in language. In this chapter, we compare two multidimensional approaches to assessing collaborative learning processes, which are based on a similar theoretical foundation and sound superficially similar. However, when a line by line comparison is made between the specific codings, we find interesting differences that serve to highlight subtle nuances in the operationalization of these theories. We are left with a deeper appreciation for the difficulty of our task as analysts to capture the intricacies of the ways in which collaborative processes are displayed through the language that we see.

The scope of the analytical work we present in this chapter is defined by our theoretical assumptions regarding formative assessment of collaborative learning interactions (Howley, Mayfield, & Rosé, 2013; Strijbos, 2011). Specifically, we assume that collaborative learning processes are an integration of three orthogonal dimensions, namely, cognitive, relational, and motivational. Furthermore, we assume that each dimension can be operationalized as a set of mutually exclusive codes, each of which is defined at the level of an individual contribution to a conversation. Thus, assessments within each dimension are performed by analyzing sequences or distributions of these codes. In this chapter, we focus specifically on distributional analyses. Overall, the purpose of the analysis could be considered broadly to be that of identifying the discourse contributions that support or hamper the unfolding collaborative learning process. Automatic assessment of collaborative learning in real time using such approaches can be used to trigger support for collaborative learning in a context sensitive way, for example conversational agents triggered by detection of an attempt at an explanation that prompts other group members to respond with their evaluation (see (Kumar & Rosé, 2011) for a review of such context sensitive support techniques). Formative assessment of collaborative learning processes can also be used to measure progress within iterative development processes for design of CSCL environments or for supporting the facilitation efforts of instructors who work with collaborative groups.

In the remainder of this chapter we offer an overview of two multidimensional frameworks for assessment of collaborative learning interactions, each of which can be thought of as including separate dimensions for exploring group processes from the perspective of three core dimensions: cognitive, relational, and motivational (Strijbos, 2011). Along with each of those frameworks we offer a high level assessment of both group discussions included in the PTL Chemistry dataset (Sawyer, Frey, and Brown, Chap. 9, this volume).

Despite the similarity in conceptualization of the two multidimensional frameworks, we find that they make different predictions within one of the groups where we explore social positioning as it is negotiated through the style of information presentation. While our codes are assigned at the contribution level, we can consider that our analysis is used to do an assessment for each student in a group within the span of time it takes the group to solve one chemistry problem together. Important differences are those that emerge through comparisons across distributions of codes from the two analysis frameworks per student along single dimensions within problems. Thus the basic unit of interaction is the student within a group per problem, because each group in our data only worked on one problem. The coding schemes we use can be thought of as our analytic representations. They impose a structure on the stream of conversational contributions that we analyze. The subtlety in our analytic work comes in the operationalization of those codes. We do not employ any sophisticated transformations of our codings beyond simple statistical comparisons. Using a distributional approach, we are able to characterize behavior of students within a problem solving session in terms of what was most typical for them during that interaction. Within this approach, we can define a pivotal moment for a student as a moment in which that student diverges from this typical behavior and does something that is atypical.

The Souflé Framework

Howley et al. (2013) first introduced the Souflé framework as a linguistic analysis approach for studying small groups. The intention was to define the codes at the level of basic language processes without reference to theoretical constructs that are specific to a particular theory of learning or collaboration. More specifically, the aim was to provide a neutral way of describing collaborative processes that might serve as a boundary object for researchers from different theoretical perspectives. Here we define its Cognitive, Relational, and Motivational dimensions in turn.

The Cognitive Dimension

The Cognitive dimension of Souflé is distinct from the other two in that its definition is not strictly linguistic. However, the values underlying the construct of transactivity (Berkowitz & Gibbs, 1979) are not controversial. The simple idea behind the concept of transactivity is a value placed on making reasoning explicit and elaborating expressed reasoning by building on or evaluating instances of expressed reasoning that came earlier in the discussion. In our prior work, we have developed and applied machine learning techniques for automatic analysis of transactivity in discussion forums (Rosé et al., 2008), chat transcripts (Joshi & Rosé, 2007), transcribed group discussions (Ai, Sionti, Wang, & Rosé, 2010), and speech recordings of dyadic discussions (Gweon, Jain, McDonogh, Raj, & Rosé, 2012) (Table 11.1).

Table 11.1 Codes, definitions, and examples for the cognitive dimension
Full size table

The unit of analysis we have adopted in Souflé was first established for analysis of transactivity. In particular, one unit is the minimal amount of text required to express reasoning. Our formulation of what counts as a reasoning display comes from the Weinberger and Fischer (2006) notion of what counts as an “epistemic unit,” where what they look for is a connection between some detail from the given task (which in their case is the object of the case study analyses their students are producing in their studies) with a theoretical concept (which comes from the attribution theory framework, which the students are applying to the case studies). When they have seen enough text that they can see in it mention of a case study detail, a theoretical concept, and a connection between the two, they place a segment boundary. Occasionally, a detail from a case study is described, but not in connection with a theoretical concept. Or a theoretical concept may be mentioned, but not tied to a case study detail. In these cases, the units of text are considered degenerate, not quite counting as an epistemic unit. In our coding of the PLTL corpus, degenerate contributions are coded as “no” for “not reasoning.” Note that degenerate does not necessarily imply contentless or unimportant. For example, questions such as “Are you doing e photon?” do not count as reasoning displays, but they nevertheless serve an important function within the collaboration.

The simple way of thinking about what constitutes a reasoning display is that it has to communicate an expression of some causal mechanism or express an evaluation or comparison. Often that will come in the form of an explanation, such as X because Y. However, it can be more subtle than that, for example “Increasing the tension makes the spring springier.” The basic premise was that a reasoning statement should reflect the process of drawing an inference or conclusion through the use of reason. Note that in the example with the spring, although there is no “because” clause, one could rephrase this in the following way, which does contain a “because” clause: “The spring will be springier because we will increase the tension.” Reasoning statements stand in contrast to mere information sharing statements, which can be thought of as sharing of rote knowledge. An example of a reasoning display from the PLTL corpus is “and then you use the kinetic energy to get V.” Because face-to-face discussion frequently leaves much implicit, we do not require all portions of the reasoning display to be articulated if the context makes the full articulation of the expressed reasoning clear. Because of this, even some questions can count as reasoning displays. For example, “and then did you subtract the work function?” counts as a reasoning display in that it is an expression of a student checking that her understanding was correct about how another student just derived a recently articulated result.

In our work, we have needed to adjust our specific definition of what counts as a reasoning display each time we have applied our transactivity coding scheme to a new domain (Gweon et al., 2012). In each case, however, the thinking behind the operationalization was the same. In particular, when students are working on a given task or a project in a team, they typically receive a certain amount of information that would help them solve the problem, in the form of a task statement and training materials. In order to solve the given problem, students discuss the materials that are given to them and try to apply them to a potential solution. These shared materials provide a frame of reference for anchoring our definition of a reasoning display. The displayed reasoning that we are interested in capturing is what goes beyond what is given and displays some understanding of a causal mechanism. Typically some causal mechanism would be referenced in a discussion of how something works or why something is the way it is. It is important to note that what we are coding is attempts at displayed reasoning. Thus, we need to allow for displays of incorrect, incomplete, and incoherent reasoning to count as reasoning, as long as in our judgment we can believe an attempt at reasoning was made. That will necessarily be quite subjective—especially in the case of incoherent explanations.

Statements that display reasoning can be coded as either Externalizations, which represent a new direction in the conversation, not building on prior contributions, or Transactive contributions, which operate on or build on prior contributions. In our distinction between Externalizations and Transactive contributions, we have attempted to take an intuitive approach by determining whether a contribution refers linguistically in some way to a prior statement, such as through the use of a pronoun or deictic expression. In the PLTL corpus, we defined transactive contributions as those reasoning displays that were positioned as contingent on at least one earlier expression. For example, one student reported, “I didn’t get the same kinetic energy.” Then another student responded, “I didn’t subtract the work function.” The response is an explanation for the difference between the answers obtained by the two students. Thus, that expression of reasoning (i.e., explanation for the difference) was also an evaluation of the other student’s approach. Its contingent relationship with the prior utterance makes it transactive. The definition of transactive employed in the analysis of the PLTL corpus might be seen as reading a lot into the contributions of students that goes beyond what is literally found in the text. However, the terse nature of the majority of student contributions in the discussions necessitated such an approach in order for the distributions of codes on this dimension to exhibit a nontrivial amount of variance between students.

The Relational Dimension

The Relational dimension in Souflé is meant to capture the level of openness to the ideas of others that is communicated in a student’s framing of assertions. Whereas in the Cognitive dimension we adopted an approach in which we read into the text in order to identify expressions of reasoning and transactivity, in the Relational dimension, we base our work on the earlier work of Martin and White (2005), whose theoretical approach explicitly mandates not going beyond the evidence that is explicit in a text.

The important distinction in our application of the Martin and White’s Heteroglossia framework is the distinction between a monoglossic assertion that is framed as though it leaves no room for questioning, in contrast to those framed in a heteroglossic manner, where the assumed perspective of others is explicitly acknowledged within the framing. For example, whereas “For e photon it would be 6.6 × 10−19.” is monoglossic, “I would say that for e photon it would be 6.6 × 10−19.” would be heteroglossic.

The specifics of the definition for the heteroglossic versus monoglossic distinction are adapted from Martin and White’s (2005) original discussion. First, some propositional content must be being asserted in some form, although it may be done in such a way as to communicate extreme uncertainty. Thus, questions that are framed in such a way as the reader believes the speaker was asking an honest question, for which no specific answer seems to be supposed do not count as heteroglossic. Interjections, like “Yay,” that cannot be interpreted as ellipsis, and thus have no propositional content are not considered heteroglossic. However, fixed expressions like “no,” and “yes” that implicitly assert the propositional content of the yes/no question they are a response to do count as expressing propositional content. Other forms of ellipsis (e.g., “13.4 angstroms” in response to “What did you get?”) and do-anaphora (i.e., “I did.” In response to “Did you also get 13.4 angstroms?”) also count as having propositional content. Second, an awareness must be made visible to the presence of alternative perspectives than that represented by the propositional content of an utterance. Thus, bald claims, even if they are biased, do not acknowledge alternative perspectives. For example, “13.4 angstroms is the obviously the answer.” May be subjective, but it is not heteroglossic. It does not show any awareness that someone else might disagree. If a speaker goes on to give reasons to defend the statement, however, then that speaker is showing awareness of other perspectives. These cases will be caught by the third requirement. Third, in order to count as heteroglossic, the acknowledgement of other perspectives must be expressed grammatically (e.g., through a model auxiliary like “might”) or paraphrastically (e.g., “I think”) within the articulation of that propositional content. If it is implicit or signaled through the discourse structure, then that is not enough to count as heteroglossic in the Martin and White sense as represented by their Engagement system, although they would acknowledge it as heteroglossic “in spirit.”

There are two types of contributions we code as Heteroglossic, one type that shows openness to other perspectives, which we refer to as Heteroglossic Expand, and another that explicitly expresses a rejection of some other perspective, which we refer to as Heteroglossic Contract (Table 11.2).

Table 11.2 Codes, definitions, and examples for the relational dimension
Full size table

The Motivational Dimension

The Motivational dimension in Souflé is meant to capture conversational behavior that reflects the self-efficacy of students related to their ability to participate meaningfully in the collaborative learning interaction. In our prior work we have seen correlations between self-report measures of collective self-efficacy from collaborative groups and measures of authoritativeness of stance derived from our coding in this dimension (Howley et al., 2012). In short, on this dimension we consider that an authoritative presentation of knowledge is one that is presented without seeking external validation for that knowledge.

This dimension, which we have referred to as the Authoritativeness Framework, is rooted in Martin’s Negotiation Framework (Martin & Rose, 2003), from the systemic functional linguistics community. This framework highlights the moves that are made in a dialogue as they reflect the authoritativeness with which those moves were made, and gives structure to exchanges back and forth between participants. Previous work has studied the complexity of, for instance, the difference between authority to alter the direction of a conversation and authority to contribute new information to a conversation. We are interested in this framework because of its descriptiveness for social interactions, and how it boils down the intricacies of power management within an interaction to a few simple codes, making it easy to track shifts in positioning over time. An application of this framework to analysis of social shift in response to bullying in computer supported collaborative learning offers an example of that use (Howley et al., 2013).

While the Negotiation framework as formulated by Martin is highly descriptive for sociolinguists, and has been widely used by Martin himself as well as by other sociolinguistics, it is difficult to replicate reliably from the previously published formulations, as this was not a methodological goal of the original researchers. This makes its immediate use for quantitative analysis difficult without introducing threats to internal validity. To remedy this, we have worked iteratively on a coding manual that incorporates the insights from that framework that are relevant to our task and makes them precise and concrete enough to be reproducible. Our inter-rater agreement for this coding has achieved a Cohen’s Kappa of 0.78. A full treatment of the details of our development process is beyond the scope of this chapter. However, we would like to acknowledge that we have developed this Authoritativeness Framework through consultation with experts from a variety of backgrounds (sociocultural researchers, education researchers, sociolinguists, computational linguists, computer scientists, interaction analysts, learning scientists, etc.). As in our work on transactivity, we have had success with automating our analysis of authoritativeness with high reliability in transcriptions of face to face interactions (Mayfield & Rosé, 2011) as well as chat transcripts (Howley et al., 2012).

Our formulation of the Authoritativeness framework is comprised of two axes with six and three codes, respectively, and incorporates structural and pragmatic knowledge of language. To simplify our analysis for this chapter, we will focus on two moves in particular. The first is K1, or “primary knower,” and the second is K2, or “secondary knower.” A “primary knower” move includes a statement of fact, an opinion, or an answer to a factual question, such as “yes” or “no.” It only counts as “primary knower” if it is not presented in such a way as to elicit an evaluation from another participant in the discussion. Conversely, a “secondary knower” move includes statements where the speaker is not positioned as authoritative on the topic at hand, such as asking a question eliciting information, or presenting information in a context where evaluation is the expected response or formulated to elicit feedback (Table 11.3).

Table 11.3 Codes, definitions, and examples for the motivational dimension
Full size table

There is no strict form-function relationship between these codes and the text being analyzed. The simplest example of this is a line such as “yeah,” which could be authoritative in response to a question or could be non-authoritative response to someone else’s evaluation. Additionally, factual statements where the speaker is uncertain of their correctness and is looking for approval from a listener would be coded as a K2 move, even though it is structurally similar to most K1 moves. The roles that speakers take through these codes can shift rapidly within a conversation, and are dynamic, being heavily based on the context of what has happened leading up to an utterance, and how that utterance is responded to by other participants. Figure 11.1 displays an excerpt from the PLTL corpus where three students are discussing an intermediate result within the scope of their problem solving session. Here we see that M’s contributions and F5’s contributions are both entirely framed as authoritative, but F4 is positioned, both by her contribution, and by M’s response, as non-authoritative.

Fig. 11.1
figure 1

An example analysis using Martin and Rose’s (2003) Negotiation system, labeled as Authority

Full size image

Souflé Analysis

Now we apply the Souflé analysis to the two separate PLTL groups and interpret the distribution of codes, at the group level and at the individual level. These are the Gillian (G-group) and Matt (M-group) groups referred to throughout this section of the book.

Group Level Analyses (Table 11.4)

Table 11.4 Frequency and proportion for occurrences of each code in each of the three dimensions for both groups
Full size table

Based on the distribution of codes on each dimension for each group, we are able to see that the two groups behave quite differently. For example, on the cognitive dimension, only 24 % of the contributions in the G-group are some form of reasoning display, whereas in the M-group, 45 % fall into this category. Furthermore, while in both groups the split between Externalizations and Transactive contributions is not very skewed, in the G-group, more than half of the reasoning displays are Externalizations, whereas in the M-group more than half are Transactive. Thus, the M group is not only displaying more reasoning, but is also engaging in building complex reasoning more frequently. In both groups, roughly 70 % of the contributions are some form of assertion on the Relational dimension, but the distribution across the types of assertions was different in the two groups. In the G-group, the majority of assertions were coded as Monoglossic, whereas in the M-group, the majority of assertions were Heteroglossic Expand. And only the G-group had any heteroglossic contract assertions of either type. Thus, we see a more open attitude communicated in the G-group. In both groups, we see a similar distribution of codes on the Motivational dimension.

Individual Level Analysis

For the next phase of the analysis, we examine more closely the inner workings of teams at the student level.

G-Group

The G-group is composed of five women. Two of them each only contribute once in the discussion. So we focus our analysis on the other three participants.

  • Cognitive dimension: Most of the reasoning contributions come from two of the three participants, which we refer to as F1 and F4. F1 has twice as many Externalizations as Transactive contributions, whereas F4 has equal numbers of both. In contrast, F2 contributes about 25 % as many reasoning contributions as the other two. Thus, we see F2 as less engaged in the active reasoning process. And F1 may be an idea leader.

  • Relational dimension: On the Relational dimension, we also see a contrast between F1 and F4 on the one hand and F2 on the other. For F1 and F4, the dominant code on this dimension is Monoglossic, even more so for F4 than F1, whereas for F2 it is no assertion. Furthermore, F4 is roughly balanced between Heteroglossic Expand and Contract, whereas the other two have twice as many Heteroglossic Expand as Contract. On this dimension we see F4 start to distinguish herself as a little more of a firm leader than F1, whereas F1 appears to be more of a supportive and open leader. In combination with the findings at the cognitive level, we can see F1 as an idea leader who does not push her own view, but places her ideas on the table for discussion.

  • Motivational dimension: The Motivational dimension echoes the view of F4 as a more dominant leader in that F4 has the highest ration of K1 to K1 + K2 contributions of the three participants, and F2 has the lowest.

M-Group

The M-group has three members, one of which is male, referred to as M, and two of which are female, referred to as F4 and F5. Roles are much less pronounced in the M-group than in the G-group.

  • Cognitive dimension: In the Cognitive dimension, the three participants utter close to the same percentage of reasoning display contributions. F5 is the lowest at 41 %, whereas F4 is the highest at 55 %. However, whereas F5 has a lower percentage of reasoning displays overall, her percentage of Transactive contributions is roughly twice that of each of the other two (i.e., 41 % in contrast to 22 % for F4 and 21 % for M). This could be seen as a way in which F5 places herself in less of a leadership position by recognizing the idea leadership of others.

  • Relational dimension: There is little distinction between participants on the Relational dimension. In all cases, Heteroglossic Expand is the dominant category.

  • Motivational dimension: There was little distinction between participants on the Motivational dimension, however, complementary to the finding on the Cognitive dimension, we see F4 as slightly more authoritative than the other two, with a K1 to K1 + K2 ratio of 86 % in comparison for 75 % for each of the other two. Thus we see a slight pattern where F4 is taking more leadership, and F5 is taking a little more of a follower role.

When comparing the two groups, we see a stronger leadership pattern in the G-group, with less openness, less reasoning and fewer contributions, whereas we see a richer and longer discussion in the M-group, with a greater level of equality between participants, more reasoning displays, more openness, and more collaborative knowledge building.

CRM Coding Scheme

Reviewing the literature on (CS)CL, it is apparent that cognitive outcomes are central to the assessment of learning in past and present (CS)CL studies, however, cognitive outcomes are not the only outcomes of collaborative learning. Slavin (1996) already identified three major perspectives in cooperative learning research—the motivational, social (cohesion), and cognitive—and stated that they “…may be seen as complementary, not contradictory” (p. 52) and that there are many other outcomes like “…intergroup relations, self-esteem, acceptance of mainstreamed classmates, pro-social norms, and so on” (p. 64). Social (cohesion) aspects, such as intergroup relations, are typically emphasized in the “Learning Together” approach (Johnson & Johnson, 1994) and the “Group Investigation” approach (Sharan & Sharan, 1992). In the context of Group Investigation, there also appear to be positive effects in relation to aspects commonly associated with intrinsic motivation, such as interest, enjoyment, and (mutual) encouragement (Ryan & Deci, 2000).

The CRM coding scheme is a tentative conceptualization of the “Group Experience” (GE) metaphor applied to the analysis of collaborative interaction. The GE metaphor contends that (a) both the individual and group level should be analyzed, (b) that the collaborative interaction cannot solely be reduced to the cognitive plane, and (c) that concurrent strands of experience exist, that is, cognitive, relational and motivational processes are affected differentially within as well as between individuals (Strijbos, 2011). The present coding scheme draws from various coding schemes and coding dimensions in previous and forthcoming publications.

The Cognitive Dimension

The Cognitive dimension focuses on the content plane. Within the context of this chapter, the Cognitive dimension is operationalized in terms of feedback processes. Widely investigated types of feedback are (a) simple feedback types providing outcome-related information, and (b) elaborated feedback types providing additional information besides outcome-related information. Narciss (2008) developed a content-related classification providing a structured overview of simple and elaborated components. Simple feedback components evaluate the performance level achieved—i.e., knowledge of performance, knowledge of result, and knowledge of the correct response (also referred to as the “verification” component; Kulhavy & Stock, 1989). An elaborated feedback component (also referred to as the “informational” component) depends on the elaborated information provided, which might address: (a) knowledge on task constraints, (b) knowledge about concepts, (c) knowledge about mistakes, (d) knowledge on how to proceed, and (e) knowledge on meta-cognition.

The present codes for the Cognitive dimension were developed as part of a study on peer feedback (Strijbos, Van Goozen, & Prins, 2012). The codes are a subset of that coding scheme. More specifically, the Verification and Elaboration codes. Krippendorff’s alpha for the entire coding scheme (24 categories) was 0.67 and 0.73 for the Verification and Elaboration distinction. Table 11.5 provides an overview of the Cognitive dimension codes, their definitions and examples from the present dataset.

Table 11.5 Codes, definitions and examples for the cognitive dimension
Full size table

The Relational Dimension

The Relational (or social) dimension and its associated outcomes are considered to a certain degree in recent literature, e.g., Gillies (2007), Strijbos and Stahl (2007), and Sarmiento and Shumar (2010). A recent study by Tolmie, Kenneth, Topping, Christie, Donaldson, Howe, Jessiman, Livingston, and Thurston (2010) investigated social effects of collaborative learning among 575 primary schools students (aged 9–12) and revealed that (a) collaborative learning leads to a dual impact in terms of cognitive and social gains, (b) collaborative skills improve alongside understanding and optimal social relations need not be in place prior to collaboration, (c) social context (rural versus urban schools) did not affect cognitive or social gains; rather the engagement in collaborative learning raises both cognitive and social gains counteracting prior social differences, and (d) the convergence over time between transactive dialogue and collaborative skills (in terms of work relations) suggests that “…cognitive and social gains would appear to be interlinked, if distinguishable, outcomes” (p. 188). In the context of (CS)CL, however, social interaction is still often taken for granted, or restricted to cognitive processes (Kreijns, Kirschner, & Jochems, 2003).

The present codes for the relational dimension were inspired from the coding scheme by Kumpulainen and Mutanen (1999) and more particularly the “social processing” dimension, consisting of six codes: collaborative, tutoring, argumentative, individualistic, dominative, conflict, and confusion. The present scheme operationalizes the Relational dimension in terms of “social climate.” The codes Collaborative Orientation and Individual Orientation were previously used in the social dimension of the VMT coding scheme (Strijbos & Stahl, 2007). The single dominance code by Kumpulainen and Mutanen (1999) was extended in terms of Positive and Negative Dominance. Table 11.6 provides an overview of the Relational dimension codes, their definitions and examples from the present dataset.

Table 11.6 Codes, definitions and examples for the relational dimension
Full size table

The Motivational Dimension

The Motivational dimension and associated outcomes have also received increased attention in recent (CS)CL literature, see for example Boekaerts and Minnaert (2006), Dillenbourg, Järvelä, and Fischer (2009), and Järvelä, Volet, and Järvenoja (2010). In contrast to an extrinsic operationalization of motivation in early studies on cooperative and collaborative learning (e.g., rewards), present motivational perspectives, such as the “dual processing self-regulation model” (Boekaerts & Niemivirta, 2000), “self-determination theory” (Ryan & Deci, 2000), and “person-object theory” (Krapp, 2005), share the premise that students have multiple goals with their subsequent motivations, actions, and affective responses. Likewise, students have multiple goals and motivations in the context of collaborative learning. Hijzen, Boekaerts, and Vedder (2007) found that mastery goals (“I want to learn new things”) and social responsibility goals (“I want help my peers”) prevail in effective collaborative learning groups. Furthermore, belongingness goals (e.g., “I want my peers to like me”) were more important than mastery goals in ineffective collaborative groups, whereas the opposite was observed for effective groups.

The present scheme operationalizes the Motivational dimension as “motivation/affect.” The codes Encouragement and Performance were taken from the peer feedback coding scheme discussed in the section on the Cognitive dimension (Strijbos et al., 2012). The codes Interest and Enjoyment were added based on recent insight in motivation research (Krapp, 2005; Ryan & Deci, 2000). Table 11.7 provides an overview of the Motivational dimension codes, their definitions and examples from the present dataset.

Table 11.7 Codes, definitions and examples for the motivational dimension
Full size table
Table 11.8 Frequency and proportion for occurrences of each code in each of the three dimensions for both groups
Full size table

Analyses and Initial Interpretations

Group Level Analyses

In this analysis, a first striking difference between both groups is visible with respect to the proportion of the overall number of statements for the Cognitive dimension (48 % in the G-group versus 65 % in the M-group) and more specifically with respect to Elaboration codes, which are solely of the Suggestion type in the M-group. This is very consistent with the findings from the Souflé analysis. Furthermore, although the groups have an almost equal proportion of statements in the Relational dimension (36 % in the G-group and 42 % in the M-group), more statements with an Individual Orientation are made in the M-group (20 % versus 9 %) and slightly more Collaborative Orientation in the G-group (19 % versus 16 %). This is the opposite of the finding from the Souflé analysis where we see a more open atmosphere in the M-group than in the G-group. Finally, the Motivational dimension is virtually nonexistent in the M-group (2 %), whereas these types of the statements constituted 10 % of all statements in the G-group. This distinction was not evident in the Souflé analysis.

Individual Level Analysis

The individual analysis complements the group level analysis, adopting more of a qualitative flavor and focusing on pivotal moments during the collaborative learning episode (Table 11.8).

G-Group

  • Cognitive dimension: Consistent with the Souflé analysis, at the cognitive plane there is collaborative dialogue and it is interspersed with short argumentative instances. There is a roughly equal input by F4 and F1, whereas F2 is “on the side” for most of the collaborative episode.

  • Relational dimension: On the Relational dimension it is evident that F2 is not especially involved and this student’s early suggestions (“no wait that’s the de Broglie. That’s to find the ejected electron” and “oh, they gave us the work function because we have to find out if the kinetic energy is greater than zero so that we can say that 1 electron is equal to 1 proton”) are not taken up by F1 and F4. In fact a Negative Verification is given by F1 “no it’s ejected. We do need the work function to find kinetic energy” to the suggestion by F2. Then F2 replies with “oh ok” and subsequently stays on the side line for quite some time. In this case the reply “no it’s ejected” also signals that there is a Negative Dominance by F1, which results in F2 “staying out of the way.” Further on there is a Positive Dominance and Encouragement by F4 to involve F2 again in a tutoring episode (“are you with us?”, “ok, we have ek”, “We know that ek equals 1/2mv2”), and F2 responds with appreciation for F4’s competence in solving the task, “you are awesome.” A couple of lines later there is again Negative Dominance since the statement “we are so smart” seems to refer to F4 and F1 working on the problem in a dyadic mode with F2 on the side. F4 appears to be the “pack leader” from this analysis, as in the Souflé analysis. However the impression we get about Negative and Positive Dominance is the opposite of what we might have expected given that F4 has many more Heteroglossic Contract and Monoglossic moves than F1 in the Souflé analysis. We will address this in our conclusion.

  • Motivational dimension: On the Motivational dimension there are expressions of interest and enjoyment during the task, although mostly at the end and by F1 and F4.

M-Group

  • Cognitive dimension: Within the Cognitive plane there are mostly Suggestions provided, as well as the comparison of calculations. In most cases this is followed by a Positive or Negative Verification. We do not see any of the students distinguish themselves at this level.

  • Relational dimension: Within the Relational plane none of the students is dominant (if any could be considered dominant, then F5 enacts the most Positive Dominance reflected by the statements “and then you subtract the work function?”, “yeah, and then you use the kinetic energy to get V”, “would you use that as the kinetic energy to get V”). This is the opposite impression than we got from the Souflé analysis, but in neither analysis is the distinction very pronounced.

  • Motivational dimension: Within the Motivational plane there are no expressions of either Interest or Enjoyment.

When comparing both groups from the perspective of this analysis, it appears that the G-group resembles a much more interactive group, whereas the M-group is focused on finding the answer as soon as possible, working individually interspersed with short episodes of sharing and checking answers. This difference is also visible in the lack of expressions of encouragement, interest or enjoyment.

Conclusions

In this chapter we have explored two separate three-dimensional analysis frameworks for formative assessment of collaborative learning processes in the PLTL dataset. Within each framework, the separate dimensions provide distinct lenses through which collaborative processes can be viewed. Students have the opportunity to take leadership along any one of these dimensions independent of the others.

The analysis along the relational dimensions across is most interesting comparison from the perspective of multivocality. Here we see the subtleties behind the idea of dominance and the different ways that positive versus negative may be viewed. In the Souflé framework, contributions are characterized as expanding or contracting the set of ideas that remain up for consideration. In the other framework, contributions are characterized as either enacting a positive or negative polarity. We see that there is a many-to-many correspondence between these distinctions. The question is where this leaves us in terms of defining the Relational dimension. Within both frameworks, one is viewed as more imposing (contracting, negative) and the other less imposition (expanding, positive). We leave it to future work to determine how these differing conversational constructs can be validated through correlation with external measures of power relations, leadership, and social roles, etc. that are sensitive enough to measure the impact of conversational positioning in collaborative groups.