An Empirical Study on Adaptive Scripting

Abstract

The objective of this chapter is to provide empirical evidenced for a comprehensive investigation which compiled some classic constructive thoughts to understand the underlying principles which are mainly outlined by the Script Theory of Guidance and associated with successful collaboration scripts in the setting of computer-supported collaborative learning. Based on the success of preceding efforts on conceptualizing, designing and assessing collaboration scripts an empirical study involving ninety-six students was conducted to test the effects of computer-supported epistemic and argumentative collaboration scripts on the processes and outcomes of argumentative knowledge construction.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The empirical study which I am going to describe here aims at investigating the effects of computer-supported epistemic and argumentative collaboration scripts on the processes and outcomes of argumentative knowledge construction. While the studies relative to the theme have been published (Weinberger et al. 2005, 2007), the current study is more concerned with a comprehensive investigation which compiled some classic constructive thoughts to understand the underlying principles associated with successful collaboration scripts in the setting of computer-supported collaborative learning. Based on the preceding efforts in the dissertation to identify the key elements and principles that determine the effectiveness of collaboration scripts, mainly outlined by the Script Theory of Guidance (Fischer et al. 2013), it is clear that a more rigorous and carefully conducted research is required to be complementary with theoretical insights what we have explored.

4.1 Designing Adaptive Collaboration Scripts—An Example

Despite the fact that the theoretical review revealed the attractiveness and effectiveness of collaboration scripts (as can be seen above), more knowledge is desperately needed when we attempt to apply the novel scaffolding approach to real learning settings, which often pose, in turn, substantial barriers for teachers, educators and researchers alike. General speaking, the CSCL community often faces two methodological challenges with respect to research on collaboration scripts.

This enriched view of learning from constructivist perspective makes the task of designing instructional supports in terms of collaboration scripts more challenging than ever before. While the increasingly large literature on the specifics of collaboration scripts in varying experimental conditions, there still remains little systematic research which explicitly considers how to design effective collaboration scripts in a systematic structure and format that is compatible with both cognitive and social sciences (Strijbos et al. 2004). One way to approach these challenges is to take a somewhat in-depth look at the theoretical mechanisms which underpin the effective collaborative learning, and to check whether these underlying mechanisms can be reproduced in computer-supported collaborative learning (Dillenbourg and Schneider 1995). In particular, a deeper understanding of the theoretical underpinning of the collaboration scripts is important to explaining why a scaffolding tool reflecting these guidelines would benefit learners (Reiser 2004) and further provide important clues to designing more effective computer-supported collaborative learning environments (Cohen 1994). The book is beginning to address both of the critical remarks about designing the collaboration scripts. Before the empirical study which might be mainly concerned here can be further discussed, a detailed approach to a systematic framework for designing collaboration scripts is proposed here.

In light of this problem, the Script Theory of Guidance (Fischer et al. 2013) presented a conceptual framework of collaboration scripts which also offers the possibility of guiding users through a certain series of components to design the collaboration scripts: in external collaboration scripts the play is reached by engaging in a set of scenes distributed among the learners in the form of roles. These scenes are further constituted by specific scriptlets. It is argued that these basic thoughts and underlying principles are also some common mechanisms about how to design external collaboration scripts more productive for learners. I am aiming at enriching the understanding about the chosen framework by specifying the collaboration scripts implemented in the present research to foster argumentative knowledge construction.

4.1.1 Play: Argumentative Dialogue

Collaborative learning can be so broad and vague in topics that it is difficult to sustain a common focus among learner partners (Koschmann 2003). Thus, the first critical element that needs to be considered for developing collaboration scripts is to provide a general task definition detailing the main goal of the collaboration. In the current context, I strived to investigate a theme that is now of increasing importance in the field of CSCL: Argumentation Dialogue. Through argumentation, students learned to think critically, articulate their own views, and negotiate their own thoughts with others’ different perspectives (Andriessen et al. 2003).

Although one primary purpose of argumentation is to convince someone else of one’s own position or belief, engaging in such constructive argumentation usually promotes learning (Kuhn et al. 1997). Principal requirements for successful computer-supported collaborative learning are to enhance the cognitive conflict and resolved it collaboratively (Dillenbourg and Schneider 1995). Active engagement in collaborative argumentation seems to fit this principle. The field of computer-supported collaborative learning has, in particular, been interested in argumentation and how students can benefit from it (Baker 2009; de Vries et al. 2002; Muller et al. 2009; Schwarz and De Groot 2007; Stegmann et al. 2007).

From the cognitive perspectives, the production of high-quality argumentation is frequently thought to promote deeper understanding about the topic or issue being argued (Nussbaum 2008), increase students’ understanding of challenging concepts (Andriessen et al. 2003), promote self-explanations (Baker 2009), foster deep cognitive elaboration (Stegmann et al. 2012), and hence contribute to the individual knowledge acquisition (Leitão 2000; Wright 1995).

From the point of view of social interactivity, argumentation fundamentally involves an interaction carried out among two or more individuals, in which a dialog emerges as a response to doubts or divergences of positions (Osborne et al. 2004). Such a conflicting situations are of particular interest with respect to collaborative interaction because they induce socio-cognitive conflict (Andriessen et al. 2003; Doise and Mugny 1984). As Von Glaserfeld (1989) has noted, other people are the greatest source of alternative views to challenge our current views and hence to serve as the source of cognitive conflict that stimulates learning.

To sum up, the ubiquity of argumentation makes it an appealing subject within the field of education. The basic assumption is that learners need to construct arguments appropriately in order to benefit from collaborative learning environments (Weinberger et al. 2007). And the ability to appreciate and engage in sound argument is among the most widely valued educational objectives for students of middle-school age and beyond (Kuhn and Udell 2007).

4.1.2 Scene and Scriptlet: Argumentation and Counter-Argumentation

As I clarified earlier, Scriptlet scaffolds prompt the learner to apply available knowledge of the sequence of activities within particular scenes, and the activities in sequence constitute a situation which can be further guided by Scene scaffolds. Therefore, the second critical step with respect to the conceptual framework is to decide which specific discourse activities and activity sets in certain sequence can be implemented and further facilitate the construction of argumentative knowledge.

Weinberger and Fischer (2006) have developed a framework to demonstrate the critical dimensions (namely, participation, epistemic, argumentative, social mode) of the argumentative activity which has been shown to foster argumentative knowledge construction in online discussions (Stegmann et al. 2007; Weinberger et al. 2007). Drawing on this conceptual framework, the approach to scaffold argumentative knowledge construction in the current study focused on two dimensions, (i) the epistemic dimension which structures what learners discussed to handle the group task and (ii) the argumentation dimension which aims to facilitate how learners argue and interacted with each other. I did not additionally support the collaborative learning on the dimension socio model, since the structured of interaction has been illustrated by the argumentation activities and the role distributions. Research has suggested that argumentation can be nurtured if both the epistemological and social discourse structures are taken into account (Clark et al. 2003). The detailed descriptions with respect to the dimensions will be given in the following paragraphs.

4.1.2.1 Epistemic Activities

The epistemic dimension refers to the question of how learners work on the tasks they are confronted with the goal to (re-)construct knowledge (Fischer et al. 2002). As epistemic activities aim to guide the attention of the learners towards the task, learners may more frequently engage in specific task-oriented activities, which in turn have been reported to foster knowledge acquisition (Cohen 1994). In this respect, the epistemic dimension of arguments is an important indicator for the extent to which learners are able to adequately apply knowledge (Weinberger et al. 2005). More recent studies have distinguished several epistemic activities with a differentiated effect on the learning outcome. Fischer et al. (2002) differentiated three types of epistemic activities in collaborative knowledge construction, which include the construction of problem space, the construction of conceptual space, and the construction of relations between conceptual and problem space.

The importance of the construction of the problem space has been outlined (Fischer et al. 2002). The construction of the problem space is a prerequisite for successfully solving a complex problem, which includes selecting, evaluating and relating individual components of problem case information. However, a focus on the concrete level of problem case information may hint at an engagement of learners in the learning task on a low level (Salomon and Perkins 1998). In this way, learners may retell rather than interpret a problem. Accordingly, it has been shown that discourse beyond a concrete level of the problem space may reflect better strategies in learning scenarios based on complex problems (Fischer et al. 2002; Hogan et al. 1999).

Within collaborative knowledge construction environments, the construction of conceptual space has been argued to be essential for successful problem-solving (Grave et al. 1996). In order to solve problems on the grounds of theoretical concepts, learners need to share the understanding of a theory. During this process, theoretical terms or principles are being defined and categorized by the learners (Pontecorvo and Girardet 1993). The construction of conceptual space also implies to distinguish concepts from each other.

The construction of relations between conceptual and problem space can be regarded as the main task in problem-oriented learning environments (Grave et al. 1996), which indicate how learners approach a problem in detail, as well as to what extent learners are able to apply knowledge adequately.

The collaborative application of theoretical concepts to a problem space may also indicate the internalization of these relations between conceptual and problem space (Palincsar et al. 1993). In other words, learners who apply theoretical concepts to problems collaboratively may be able to transfer this knowledge to future problem cases and apply theoretical concepts individually (Vygotsky 1978). Therefore, the frequency of the construction of relations between conceptual and problem space may thus indicate knowledge acquisition. To sum up, epistemic scripts can assist the structuring of subject matter knowledge, which can be more or less specific to the domain that is to be discussed (Stegmann et al. 2007).

4.1.2.2 Argumentation Activities

The activity of argumentation can be described by at least two sub-dimensions, i.e. the construction of single arguments and the construction of argumentation sequences (Weinberger and Fischer 2006). Argumentative script components aim to help learners construct formally adequate arguments and thus better elaborate the argumentative knowledge construction (Andriessen et al. 2003).

Typically, conducting a single argument includes the processes such as identifying alternative perspectives, developing solutions and supporting a solution with adequate and convincing evidence or reasoning to support one’s claims, statements and other assertions (Kuhn et al. 1997; Voss 1988). Toulmin’s Model (1958) has been widely used in recent literature (Bell 2000; Osborne et al. 2004), which puts forth a single-argument structure consisting of multiple components: Claim is a conclusion that is being presented and justified in the argument. The component of the claim is based on a ground, which is a fact that is supposed to support the claim. Additionally, a warrant specifies the underlying theoretical presumptions of how the ground supports the claim. This is reasonable for distinguishing between ground and warrants: ground is appealed to in order to be explicit, while warrant is more implicit. Sometimes arguments may optionally also provide a qualifier, which indicates the extent to which the datum warrants the claim or may limit the validity of a claim. Backing refers to the theoretical assumptions on which the warrants rest, and rebuttal serves to anticipate parts of a counterargument that attack the data or the warrant.

The attraction of Toulmin’s model is to pose an alternative to formal logic, which is closer to reasoning in real-world situations (Yeh 1998). Besides, from a pedagogical perspective, the model is the explicit procedural representation of the basic layout of arguments (van Eemeren et al. 1996). However, Toulmin’s model has been criticized for difficulties in distinguishing between the single components of the model in everyday argumentation. For example, it is challenging for the learner to differentiate between a qualifier and a rebuttal (Voss and Van Dyke 2001). I decided to simplify the Toulmin’s model instead of using it completely in order to make it more closely accessible to target participants. Thus, the single argument dimension comprises how learners construct arguments with regard to defining formal connection among the three basic and core components: Claim, Ground and Warrant.

Construction of argumentation sequences captures the dynamic of argumentative dialogue consisting of arguments, counterarguments, and integrations (Leitão 2000; Weinberger and Fischer 2006). In this argumentation sequence, learning partners first try to justify their (initial) position by constructing arguments. Then, counterarguments challenge this position and may lead to a reconsideration of the initial argument. The counterargument is not necessarily the opposite of the initial argument, but calls it into question (Stegmann et al. 2007). As a minimum, a counterargument makes the acceptability of the initial position less certain. To some extend, conducting a counter-argument is integrated with the peer monitoring process which served as ‘a starting point’ to justify the initial position which is unanimous or conflicting with other peer partners. Finally, learners construct replies and may possibly synthesize their initial positions in integration or decide which alternatives fit the best (Stegmann et al. 2007).

4.1.3 Role: Analyst, Critic and Replier

The learners take different roles in formulating an initial argument and criticizing it with a counter-argument. Crossing different case-based problem solving tasks, the roles can be switched. The script provided guidance on the roles (namely, analyst, critics and replier) that the students had to follow during the activity. The analyst is responsible for constructing arguments. The critic constitutes a counter-argument that goes against the argument produced by the analyst. The replier combines the initial argument and the counterargument to the integrated argument.

4.1.4 Overview of the Designed External Collaboration Scripts

It is essential to accumulate what has been presented about argumentative knowledge construction as a systematic configuration before designing the external collaboration scripts to facilitate the targeted collaborative practice. The theoretical framework upon which the design is based was developed by Fischer and colleagues (Fischer et al. 2013) which highlighted a few essential elements accounting for effectiveness of collaboration scripts. Figure 4.1 displays the general architecture for representing the entire desired collaboration scripts to foster argumentative knowledge construction.

Fig. 4.1

Overall conceptual framework for the argumentative and epistemic activities

The architecture illustrates three layers of collaboration scripts that aim at promoting argumentative knowledge construction, which differs in the levels of scripting components. As the target collaborative practice, the Play for learners is to take part in an argumentative dialog with the purpose of resolving three case-based problems by applying a psychological theory (see in Chap. 5 for more details). The second layer is to specify the Scene in which learners work together to construct arguments, counter-arguments and integrate their statements. To conduct a single argument, a set of scriptlets are identified to scaffold the process of constituting claim, ground and warrant according to Tulmin’s model (1958). I further argue that the hierarchical and dynamic architecture of the collaboration scripts can further specify ‘tiny’ unit of activities, which is presented on the sub-layer of scriptlets (Sub-scriptlet). In this case, a claim is composing the construction and consequently the connection conceptual and problem spaces.

In order to scaffold the desired argumentative activities at the ‘bottom’ levels of the scriptlet and sub-scriptlet, a rather detailed framework of external collaboration scripts was developed in the present CSCL research. The collaboration scripts identify detailed steps to guide learners in contrasting single argument, counter argument and integrated argument. Learners are expected to respond to specific questions about the respective scripts which assumed to contribute to individual learners constructing single arguments (e.g. What is your claim regarding the subject and concept? What is your ground to support the claim, and what is the warrant to connect the claim with the ground?) In addition, counterarguments can be contrived through a defined sequence of interaction phases (e.g. Do you agree with the claim of your partner? If yes, have you any other ground to support the claim?).

The entire collaborative practice can be divided into ten phases, so that a set of specific activities for each phase can be undertaken by learners either individually or collaboratively. The computer program presented the participants at first with one case requiring independent problem solving. Each learner was responsible for the analysis of one of the three cases in the first phase to compose the first analysis consisting of single argument(s). By following the external collaboration scripts (as shown in Fig. 4.2), each learner was required to combine epistemic subject (from problem space) and concept (from conceptual space) to state a claim, provide ground to support the claim, and then a warrant is required to connect the ground to claim. Consequently, five steps are used to formulate one single and complete argument.

Fig. 4.2

Flowchart to outline the designed collaboration scripts

In the next two phases, each learner switched to other cases to provide counterarguments for the analysis constituted by other partners. Drawing on Walton’s analysis (1989), two goals of argumentation have been identified: one is to secure commitments from the opponent that can be used to support one’s own argument; the other is to undermine the opponent’s position by identifying and challenging weaknesses in his or her argument. In this manner, both of these goals require attention to the opponent’s positions. Due to the unique feature of argumentation, the scriptlets on the argument sequence dimension to some extent function as peer monitoring to justify the reasonableness of the claim, ground and warrant produced by partner learners. Besides the judging steps, learners are required to build new argument components, if conflicting opinions exist among peers.

In the following, the learner shifted back to the initial case to integrate the counterarguments given by learning partners into the initial analysis. Phases from two to four were repeated twice for the purpose of promoting more collaborative practices. In addition, each step of the script includes an explanation and example to further clarify the task in details. At the tenth phase of the discussion, each group had to find a consensus and outline a final analysis for each of the three cases.

Strijbos et al. (2004) have pointed out that one of the major problems of current research in the field of computer-supported collaborative learning is the lack of design principles for the learning settings. For practitioners, the components and mechanisms elaborated here may serve as a checklist for the design of the scripted CSCL environments. I also expect the detailed examples to be useful as models of good practice. More empirical testing is clearly required, and numerous other issues with respect to implementing the collaboration scripts in a real online learning environment remain to be discussed in the following chapter.

4.2 Dependent Variable: What Can We Learn from Argumentation

Knowledge plays a vital role as the learning objectives in the literature on learning and instruction. Generally, it comprises collecting facts about the world and procedures for how to solve problems (Sawyer 2006). Recently, there has been a shift in the views of what education is for, with an appealing emphasis on the need to enable and support not only the acquisition of concepts and facts, but also to develop the skills necessary to engage with the emerging social and technological changes, and to continue learning throughout life. Therefore, knowledge can be thought of having two dimensions: domain-specific and domain-general knowledge (Penner and Klahr 1996). The domain-specific knowledge lives in books and in our brains as concepts and facts and deals with the ‘know-what’, while the domain-general knowledge deals with the ‘know-how’ that is best manifested in practices and skills (Carud 1997).

Indeed, knowledge is characterized by strong links between elements, a high degree of abstraction, and a hierarchical nature (de Jong and Ferguson-Hessler 1996). However, it is rare to find a CSCL research on collaboration scripts which has well-articulated learning objectives. It is much more common to encounter a final and singular goal such as ‘learning outcome’ or ‘achievement’. Not much thought has been given to the multiple aspects of knowledge or to the entire knowledge construction. In the present study, it is noteworthy to explore the properties of different knowledge components and the interrelationship among them.

As I began addressing the general question, I quickly realized that it was well beyond my reach to clarify all the issues pertaining to knowledge. For this reason, I concentrate on an in-depth discussion of Argumentative Knowledge Construction (AKC) in which learners construct arguments in interaction with their learning partners in order to acquire knowledge about argumentation as well as knowledge of the content under consideration (Andriessen et al. 2003) (Fig. 4.3).

Fig. 4.3

Argumentative knowledge construction

Andriessen et al. (2003) further identified the difference between ‘Learning to Argue’ and ‘Arguing to Learn’ knowledge. The first kind of knowledge involves the acquisition of general skills such as justifying, challenging, counterchallenging, or conceding. In contrast ‘Arguing to Learn’ often fits a specific goal fulfilled through argumentation, and in an educational framework, the (implicit) goal is to understand or to construct domain-specific knowledge. By scaffolding good argumentation practices, the scripts employed in this study attempt to not only support students in ‘Learning to Argue’ but also help students learn about specific domain topics through argumentation (‘Arguing to Learn’). In addition, I further differentiate the knowledge of ‘Learning about Argumentation’, which refers to the conceptual knowledge on argumentation, and thus is different from the procedural knowledge of ‘Learning to Argue’. When we tend to identify the learning outcomes of argumentative knowledge construction in the studies undertaken, indeed, the three categories of knowledge are often mutually dependent (Koschmann 2003) since both declarative (subject matter) and procedural (skill or procedure) knowledge are essential to solving problems (Larkin and Reif 1979).

To exemplify the hierarchical knowledge structure, the learning objectives of the current study are presented in details. Specifically, the designed scripts aim at three kinds of learning outcomes, namely: (i) the acquisition of ‘Arguing to Learn’ knowledge about the domain-specific topic at hand (e.g. Attribution theory, specifically used in the current study) which refers to specific subject matter knowledge that can be attained by the learner as the predefined goals (e.g. Attribution theory) (ii) the acquisition of ‘Learning about Argumentation’ knowledge which basically means becoming better acquainted with the argumentation (e.g. A single argument may consist of claim, ground and warrant). This kind of learning outcome can be seen as the declarative knowledge (factual information) on argumentation (Alexander and Judy 1988) (iii) the acquisition ‘Learning to Argue’ knowledge, which is closely associated the general procedural skills of argumentation (compilation of declarative knowledge into functional unites). This kind of procedural knowledge is not described as a definitive object, but rather as an activity state of cognitive structures when individuals or groups perform tasks (de Jong and Ferguson-Hessler 1996). Corresponding with what I had emphasized in the Script Theory of Guidance, it makes the distinction more clear that the knowledge of ‘Learning to Argue’ can be regarded as internalized collaboration scripts.

4.3 Independent Variable: Degrees of Scripting

Dillenbourg’s comments (2002) are echoed in the contemporary concerning the degree of scripting. He argued that different scripts vary regarding their degrees of coercion, that is, the extent to which they force users into specific actions. This introduces a serious problem in collaboration scripts because of a dilemma between maximum flexibility on the one hand and minimal freedom on the other. Too much external collaboration scripts may result in ‘forced’ artificial interaction, but no structure may result in ‘fragmented’ interaction.

With respect to the ‘degree of scripting’, external scripts can differ substantially. An unresolved issue is to what degree is ‘optimal’ for a learner (free or coerced) to follow a structure given by external collaboration scripts. Instead of investigating the polarized degrees of with or without collaboration scripts, three degrees of scripting from high to low are distinguished (as shown in Fig. 4.2) in the current study to examine their impacts on learning processes and outcomes.

4.3.1 Low Degree of Scripting

With an interest in the thinking that occurs in daily lives, Kuhn (1991) pursued the notion of thinking as argumentative reasoning. While argumentation is a discursive practice which forms part of everyday experience, it is rarely spontaneous in professional life (Muller et al. 2009). As a result of the complexity of the argumentation activities, unfortunately, even adults rarely engage in constructing adequate arguments and interacting productively without specific prompting and scaffolding (Kuhn 1991). Specifically, a review of recent literature on argumentation pointed out that adult discussants have difficulty articulating and justifying their claims (Sadler 2004), rarely ground or warrant their claims and thus rarely construct complete arguments with logical reasoning (Kuhn et al. 1997); and often fails to embrace the dual objectives of argumentative discourse —to identify weaknesses in the opponent’s arguments and to secure commitments from the opponent that can be used to support one’s own claims (Walton 1989). Because of these challenges, it has become clear that simply asking learners to collaborate is not sufficient for fostering argumentative knowledge construction (Stegmann et al. 2007). Learners therefore need to be explicitly scaffolded in order to be successful in this social practice.

As a control condition, the low degree of scripting was manipulated by providing only general Play scaffold to ask learner to attend the argumentative dialog in the present study, which is assumed to cause the negative effects of under-scripting.

4.3.2 High Degree of Scripting

The other side of the coin in designing well-defined instructions is the risk of over-scripting. Designing and implementing collaboration scripts requires addressing a pedagogical dilemma that is very classical but particularly salient in CSCL: if the scaffolding is too weak, it will not produce the expected interactions; if it is too strong, it will increase the cognitive load of the learner (Dillenbourg 2002), spoil the natural richness of free collaboration (Dillenbourg and Tchounikine 2007), cause stronger negative motivational reactions as compared to the traditional script approaches (Rummel and Spada 2007), inhibits the learner’s self-regulated application of appropriate higher-level internal collaboration script components (Fischer et al. 2013) and actually impede knowledge construction (Larson and Dansereau 1986).

However, these assumed obviously negative effects must be thought through carefully. In contrast to the above statements, Stegmann et al. (2011) claimed that negative effects of scripts on collaborative learning should not any longer simply be classified as over-scripting. Rather, the problems of ‘too much script’ exist only if learners have sufficient access to internal collaboration scripts that can be disturbed. There is not much of a worry about over-scripting for novices in particular.

In order to address the complex problem of ‘over-scripting’, the high degree of scripting was designed, which includes the scaffolds on the levels of play, scene, scriptlet and even sub-scriptlet (as present in Chap. 3). And for each specific component of external collaboration scripts, external explanation and example were additionally provided to further scaffold learners in the collaborative practices.

4.3.3 Medium Degree of Scripting

Many of the classical script approaches that were developed to facilitate collaborative learning are built on the assumption that through extended practice with the external collaboration script, a learner would, little by little, internalize relevant elements of the script so that the external scaffolding provided by the script could be faded out over time (Palincsar and Brown 1984).

One especially acute issue that needs to be clarified here is how to implement the adaptive collaboration scripts. In the current study, the components of external collaboration scripts were removed gradually overtime, and on average, the degree of scripting stay at the medium level. I call this ‘limited adaptive collaboration scripts’ and it differs from ‘really adaptive collaboration scripts’ because it was not responsive to what was happening in the collaboration, but faded with fixed time interval and therefore limited to be really adaptive to the individual learners. Particularly, the collaboration scripts were faded from highest degree of scripting (backward fading), and crossing various layers (vertical fading) (Fig. 4.4).

Fig. 4.4

Framework to outline the three degrees of scripting

Therefore, the empirical research aims to build in-depth knowledge of how the expected argumentative knowledge acquisition is related to degrees of scripting. The learning effects of adaptive collaboration scripts (fading condition with medium degree of scripting) are compared to other two conditions: a condition in which the collaboration scripts maintain at the high lever (over-scripting) and a condition without additional scaffolding supports as the control condition (under-scripting).

4.4 Research Questions

The Script Theory of Guidance for computer-supported collaborative learning provides a systematic way of describing and interpreting the internal collaboration scripts, external collaboration scripts provided and the dynamic interaction between them (Fischer et al. 2013). By studying this theoretical framework, the present work goes beyond the previous studies to explore the underlying socio-cognitive mechanism account for effectiveness of collaborative learning, a set of research questions can be formulated to implement the underlying mechanism of the interactive and intricate relationship among those decisive components, e.g. internal collaboration scripts retained by individual learner, external scripts manipulated with various degrees, learning processes and outcomes covering multiple perspectives of the target cognitive-social activities, and last but not least, the influence from motivational variables.

4.4.1 RQ 1: To What Extent Does the Degree of Scripting Affect Students’ Participation?

The original social-constructivist assumption of collaboration scripts is to design a set of instructions which directly impact the learning processes of how the group members should interact and collaborate. As a consequence the enhanced learning outcomes are expected.

Students’ engagement in computer-based learning environments often has problems. A learner often does not engage enough (Cohen and Lotan 1995). Without some form of explicit instruction, it is rare for learners to spontaneously use effective interaction procedures and match them to the task at hand (King 2007). Indeed, even when given instructions to work collaboratively on a task, learners generally tend to interact with each other at a very basic level (Webb et al. 1986). For this reason, numerous attempts have been made to promote learning by structuring and regulating the interaction with collaboration scripts.

The first research question mainly serves as a treatment check of whether the designed collaboration scripts can foster the expected activities during the collaborative practices. Participation becomes the key concept (as contrasted with acquisition and conceptual change), serving as both the process and the goal of learning (Sfard 1998). Even though student engagement is not the best measure of educational value (Dewey 1963), constructivists stressed that active participation in a task leads to deeper and richer understanding and use of knowledge (Moallem 2003); students need to actively participate to construct their own learning (Anderson 2008) and active participation and interaction is critical to the success of online collaborative learning (Lave and Wenger 1991; Vygotsky 1978).

However, the active participation should not only be determined by the number of messages sent by group members (Harasim 1993) but also by a deeper (contrast to the surface level characteristics of the communication) content analysis to figure out the amount of off-task activities. Drawing on the analytical framework of Henri (1992), participation can be considered at the two levels: (i) Overall Participation, which is the total number of messages and contributions to the online discussion made by learners, while (ii) the Active Participation in the learning process is the number of statements directly related to learning (non-off-task activities). External collaboration scripts have proven to be powerful strategies for supporting collaboration in learning and problem-solving contexts (Kollar et al. 2007; Rummel and Spada 2007; Weinberger et al. 2002). Schwarz and Glassner (2003) took one more step and acknowledged that appropriate interface and interaction designs caused students to use more relevant claims and arguments and less off-task expressions.

Therefore, I expect (H1a) that the collaboration scripts manipulated in the current study can help to ensure the participation by instructing the collaboration and cognitive activities, and learners supported by collaboration scripts will contribute more to the collaboration and conduct less off-task activities (H1b).

4.4.2 RQ 2: How Does the Degree of Scripting Affect Learning Outcomes of Argumentative Knowledge Construction?

There is a growing body of evidence that shows that collaborative learning is only beneficial with respect to the acquisition of knowledge and skills when appropriate instructional support is provided (Cohen 1994). Fading is promising for removing some of the explicit instructions or restrictions of scripts in the course of the learning process, as learners internalize the script more and more. Therefore, I expect that the medium degree of scripting, which provides limited adaptive collaboration scripts is expected to be able to most effectively foster the acquisitions of knowledge. Qualitative content analysis was conducted in the post/transfer case to estimate the knowledge acquisition of ‘Learning to Argue’ as the one of the indicators of learning outcomes. In the post case, the procedural knowledge can be regarded as internalized scripts on how to argue and collaborate. This study aims to provide more empirical evidence on this issue by manipulating three degrees of scripting to figure out the ‘optimal’ collaboration scripts to foster collaborative learning.

4.5 Methods

4.5.1 Participants

Ninety-six (96) students of educational science at the University of Munich participated in this study during the summer term 2010. The experimental sample consisted of 19 males and 74 females. The age of the participants ranged from 18 to 44 years old, and the average age of the participants was M = 23.48 years (standard deviation SD = 3.87). The participants were volunteers who are varied in majors, and were randomly assigned to groups of three.

4.5.2 Study Design

A one-factorial quasi-experimental design with three experimental conditions differing in the ‘degree of scripting’ was implemented. During the entire learning process, groups were randomly assigned to one of the three experimental conditions. As shown in Table 4.1, there were 11 triads in the experimental conditions (with medium and high degree of scripting) and 10 triads in the control condition with low degree of scripting.

Table 4.1 Experimental conditions and participants

4.5.3 Learning Materials

The subject of the learning environment was Weiner’s attribution theory (1985) and its application in education. The students read the text of this theory and the text of introducing argumentation individually before the experimental session.

The task of the participants was to apply the attribution theory of Weiner (1985) to five problem cases (see Table 4.2 as an example case) and reach agreement on a final analysis for each case. In this empirical study the problem-based cases are designed to be varying similar to the problems the participants encountered within real-life studying contexts. The case ‘Math’ describes the attributions of a student with respect to his poor performance in mathematics. In the case ‘Class reunion’ a math tutor talks about how he tries to help female students deal with success and failure in assignments. The case ‘Between-culture variance’ describes differences in school performance between Asian and American/European students that were explained by attribution theory. Another two cases were used in the pre and post test, which mainly concern the factors that affect a student’s choice of a major at the university and student’s explanation for failure in the exam of ‘Text analysis’.

Table 4.2 One of the three problem cases, namely the ‘math case’, learners needed to analyze and discuss

4.5.4 Learning Environment

A computer-based learning environment served as an asynchronous, text-based discussion board for the delivery of the learning materials, instructional contents and the exchange of text messages that resembled emails. The online learning environment was developed using a tool called S-COL (Scripting for Collaborative Online Learning), which allows the sustainable development of scripts and scaffolds that can be used with a broad variety of content and platforms (Wecker et al. 2010). Learners accessed the learning environment directly via the Internet. While three students in each group were together in a laboratory room (but not sitting next to each other), all communication in the study was computer-mediated.

Supervision was kept standard and was represented in the form of videos which contained basic and concise instructions. In general, video A is a generic instruction that was given to all participant. It was presented before all the learning phases to introduce (i) the fundamental goal of the research, (ii) features of the interface, such as how to type into the input box and how to submit the text message, (iii) required steps for proceed with the learning phases, and (iv) other details with timing and scaffolding information. Video B presented before the collaborative learning phase is specified with various conditions, which differentiated mainly in their introductions of the different degrees of scripting.

Collaboration scripts were dynamically integrated into the computer environment. Three conditions of implementing the various degrees of collaboration scripts are described below in detail.

4.5.4.1 Low Degree of Scripting

Participants in this condition received no additional support in solving the three problem cases. However, they were advised to argue well according to the text on argumentation they had to read before the experimental session.

Figure 4.5 presents the screenshot of treatment condition with low degree of scripting. The main screen is divided into three areas: The scripting bar in the left-hand panel was to define the script elements, the reading area in the top part of the right-hand panel served to present the on-working case, and the typing area just below the case material used to crate a discourse text.

Fig. 4.5

Interface example—low degree of scripting

While in the control condition the left-hand fields of the collaboration scripts were blank and hidden, the web-based discussion boards provided selection bars with an overview of all message headers as well as the timing information. The program allowed the participants to select and work on the cases freely by clicking the case buttons only in the control condition. The descriptions of the problem cases were embedded into the web-based learning environment, so that the participants could study the problem case while composing new messages on the web-based discussion board. The interface allowed the exchange of text messages that resemble emails. Each typing area consisted of two parts: text and title. These areas have the look and function of most e-mail systems. If no title was entered, the system default title was ‘No Title’.

All submitted messages were recorded on a central database and typically represented in discussion threads (Fig. 4.6). These threads started with one particular message that was indicated in a message overview by its title, the author, and the time of entry. In this overview, answers to original messages appeared in an outline form. The learners could read the full text of all messages, reply to the messages, compose and post new messages. In the replies, the original messages were quoted out with > as in standard newsreaders programs. Any response to a message is graphically connected to a message that initiates a discussion by a ‘thread’. Thus, an increasingly indented discussion thread is built in which the discussants are supposed to continue the specific subject which was initialized with the very first message.

Fig. 4.6

Discussion tree structure example

4.5.4.2 High Degree of Scripting

In the high degree of scripting condition, a set of written prompts were presented in the left-hand part of the text field. As already illustrated in Chap. 3, the collaborative learning session was divided into ten phases. For each phase, written prompts were delivered to guild the constructions of claim, ground, warrant, counter-argument and the integrated statements by providing highly detailed information (as shown in Fig. 4.6). The prompt were further accompanied by an illustrating example and explanation.

The progression from phase to phase was designed in advance, i.e. after a certain time, learners were automatically forwarded to the next phase. The participant was not allowed to switch cases as he/she will. While the activities and activity sequences were scripted in a fixed way, it is worthy of note that learner may choose freely whether or not to follow the steps to argue and collaborate. In other words, they always had the chance to write statements in the message box ‘supported with low degree of scripting’.

4.5.4.3 Medium Degree of Scripting

The medium degree of scripting was implemented through continuous fading-out of the prompts of the external script with fixed time interval (as shown in Fig. 4.7). The environment handles the fading levels according to a designed sequence specified by the researchers in advance. Specifically, the written prompts in first phase were identical to the condition with high degree of scripting. After each turn point, which is also the point of switching phases and cases for learner in the groups with high degree of scripting, the degree of the provided collaboration scripts were gradually reduced by slightly fading out as well as fading in. For example, in the end of the tenth phase, learners received only one textboxes for creating a whole argumentation neither specifying the argument components, nor providing explanation and example.

Fig. 4.7

Interface example—high degree of scripting

4.5.5 Procedure

The study was comprised of four main learning stages. Table 4.3 provides an overview of the procedure of the experimental study. Prior to the lab experiment, the randomization of participants was successfully controlled using individual questionnaires and tests, for example, on motivation, computer literacy, and other personal information. After the introductory section with the video A, the participants proceeded to practice the learning environment task with one pre-case. During this learning phase, learners’ individual domain-specific prior knowledge and the internal collaboration scripts were assessed by the automatic approach of content analysis. Before the real treatment, the students received a standard introduction via video B, which differed with respect to the computer-supported script components that were implemented in the interface of the online learning environment. The collaborative learning phase, in which learners communicated with each other via the text and web-based discussion board, was 80 min for all experimental groups. During the 80 min learners in triads were required to solve each of the three cases. All discourse data within the web-based discussion board were recorded and served as the sources for analyzing the processes of argumentative knowledge construction (Fig. 4.8).

Table 4.3 Illustration of dummy variable coding schema

Fig. 4.8

Fading procedure

4.5.6 Variable Measurements

Both quantitative questionnaire data as well as qualitative content analysis of discourse data were collected. In order to put the pieces together, it is essential to have an overview of the data obtained from the multiple sources.

4.5.6.1 Variables Measured by Questionnaires

Several individual learning prerequisites have been discussed as important factors for collaborative knowledge construction. These individual learning prerequisites may be categorized as cognitive (Computer Literacy, and Learning Strategies) and motivational (Intrinsic and Extrinsic Motivation). All of these variables measured with individual instruments will be described in the following sections.

4.5.6.2 Computer Literacy

Computer literacy was measured by four items to rate the perceived computer skill and experiences, with 1 representing very much below average and 5 representing very much above average.

4.5.6.3 Learning Strategy

The learning strategies were measured with the German language scale (Inventars zur Erfassung von Lernstrategien im Studium) developed by Wild and Schiefele (1994). Learning strategies, which describe how learners deal intentionally with learning material, may influence how well learners understand theoretical texts (Wild and Schiefele 1994). Learners who read through learning texts critically, for instance, may have advantages over learners with inadequate learning strategies when applying theoretical concepts to a problem. The 6-item-scales was sufficiently reliable to be used in the current study (Cronbach’s alpha = 0.76).

4.5.6.4 Academic Motivation (Pre Test)

Motivation after the intervention was measured using the Academic Motivation Scale (Vallerand et al. 1993), which further consists of two subscales, which separately address Intrinsic Motivation (IM) and Extrinsic Motivation (EM). Three types of IM can be identified as intrinsic motivation to know, to accomplish things and to experience stimulation. Specifically, IM to know can be defined as the fact of performing an activity for the pleasure and the satisfaction that one experiences while learning, exploring, or trying to understand something new (e.g. Because I experience pleasure and satisfaction while learning new things), while IM to accomplish focuses on engaging in an activity for the pleasure and satisfaction experienced when one attempts to accomplish or create something (e.g. For the pleasure that I experience while I am surpassing myself in one of my personal accomplishments). Finally, IM to experience stimulation is present when someone engages in an activity in order to experience stimulating sensations derived from one’s engagement in the activity (e.g. For the intense feelings I experience when I am communicating my own ideas to others).

According to the self-determination theory (Deci and Ryan 1985, 1991), three types of EM can be ordered along a self-determination continuum. From lower to higher levels of self-determination, they are: external regulation is to regulate behaviour through external rewards and constrains (e.g. Because with only a high-school degree I would not find a high-paying job later on); with introjected regulation, the individual begins to internalize the external reasons for his or her action (e.g. To show myself that I am an intelligent person); and the internalization of extrinsic motive become regulated through identification (e.g. Because eventually it will enable me to enter the job market in a field that I like).

Each of the subscales lists 12 items that may represent reasons why students go to college. These reasons are scored on a seven point scale from ‘not at all’ to ‘Exactly’ with a midpoint at 4 (‘Moderately’). Cronbach’s alpha was 0.92 for the Intrinsic Motivation Subscale and 0.91 for the Extrinsic Motivation Subscale. An average score for each subscale can be derived by summing the subscale items and dividing by the number of items that make up the dimensions. The higher the score, the more motivated the learner is.

4.5.6.5 Variables Measured by Content Analysis

• Participation

Initial studies attempted to analyze the effectiveness of collaboration scripts by counting a few superficial features of discourse data, such as the student participation rates (e.g. raw number of messages sentences or words). While the drawbacks of using such superficial features to indicate the complex learning processes is obvious, the content analysis on the participation dimension provides basic information. Since the general idea of collaborative learning is to encourage student participation and promote peer interaction, it is imperative to establish a pattern of participation in the online discussion.

Quantitative data on participation, in terms of the Overall Participation and Off-task Activities, were collected as part of a preliminary analysis. The Overall Participation in the learning setting can be easily measured by counting the number of segments the discussants actually produced during the collaborative learning phase. Off-Task Activity can be identified by the automatic approach to content analysis, which has been presented in detail in Chap. 3. Such an analysis on the participation provided first-hand evidence of the success of the collaboration scripts for facilitating argumentative knowledge construction.

4.6 Statistics

It is argued that the assumption of independence between the scores of members in the same group is violated in the small group research –which makes the use of ANOVA and OLS regression inappropriate. In this manner multilevel analysis appeared to be the best suited technique to analyze the cluster data collected in the present study. A set of HLM models with two levels was performed to detect the potential affects of scripts on the small group collaboration. In addition, a few statistics issues are addressed in the section before I can proceed to the results of the experimental study.

Coding systems typically use two different dummy-coding variables to represent the three levels of experimental treatments. Through the use of dummy codes, the categorical information can be rendered into quantitative forms. Table 4.3 presents alternative dummy-variable coding schemes that could be used for the current study.

In section A, three treatment conditions are represented by two dummy codes with the control group as the reference group. Having chosen a low degree of scripting as the reference group, each of the other group is given a value of 1 on the dummy-coded variable that will contrast it with the reference group in the regression analysis and a value of 0 on the other dummy-coded variables. As is illustrated in Table 4.3, SCRIPT contrasts a high with a low degree of scripting, FADE contrasts a medium with a low degree of scripting. However, I am also interested in a comparison between high and medium degree of scripting but neither of them is the reference group. The easiest way to accomplish this comparison between two non-reference-groups is to rerun the analysis after the data have been recoded by using another dummy coding system in which one of the comparison conditions is taken as the reference group (Cohen and Cohen 1983). Section B in the table shows the alternative dummy codes in which a medium degree of scripting is taken as the reference group. In this way, the full comparisons among all three treatment conditions are available now.

4.7 Results

4.7.1 Preliminary Analyses

To control for differences prior to the treatments within the various conditions, I compared the three different conditions using a ONEWAY ANOVA. No significant effects of the degree of script in the online discussion on the pre-test case were found regarding the concerned control variables. Table 4.4 includes a more detailed description of the variables used in the analysis. More specifically, some descriptive statistics, such as the means, standard deviations as well as the ANOVA results of each variable are presented.

Table 4.4 Mean and standard deviations of control variables by experimental conditions

However, the students disposed of extremely little applicable prior knowledge on argumentation. In total, 6881 segments (syntactically meaningful sentences) were analyzed with the automatic content analysis model presented in this chapter. Table 4.5 presents an overview of the descriptive results. During the online discussion, students included many cognitive activities to constitute claims and grounds, but fewer warrants.

Table 4.5 Mean and standard deviations of participation by experimental conditions

4.7.2 RQ 1: To What Extent Does the Degree of Scripting Affect Students’ Participation?

The process variables I present here are the Overall Participation and Off Task Activities, a measure of student’s active participation in the online discussion. For each process variable, the HLM analyses were performed in three stages. In the first stage, a null model was tested in which no independent variables were included in the analysis. In the second stage, the group-level independent variables were added to the model, while in the third stage the group-level independent variables were re-runed with alternative dummy codes. All results present in this section are meant to answer the first research question.

4.7.2.1 The Effects of Degree of Scripting on Overall Participation

Table 4.6 presents the parameter estimate and standard errors for three models. In this table, the intercept-only model estimates the intercept as 49.49, which is simply the average value of Overall Participation across all groups and individual learners. The variance of the group level residual errors, symbolized by $\sigma_{u0}^{2}$, is estimated as 25.83, which is not significantly different from zero (χ² = 37.51, df = 31, p = 0.20). The variance of the individual learner level residual errors, symbolized by $\sigma_{e0}^{2}$, is estimated as 450.33. The intra-class correlation (ICC), calculated by the equation of $\rho = \sigma_{u0}^{2} /(\sigma_{u0}^{2} + \sigma_{e0}^{2} )$, is 0.054. Thus 5.40% of the variances of the Overall Participation are at the group level, which is not so high. Since the intercept-only model contains no explanatory variables, the residual variances represent unexplained error variance. The deviance reported here as −2*log likelihood function value, is a measure of model misfit. In general, the larger the deviance, the poorer the fit to the data (Raudenbush and Bryk 2002). Therefore, when more explanatory variables are added, a lower level of deviance is expected.

Table 4.6 Fixed effects and variance estimates for models of the degree of scripting affecting learning processes in terms of overall participation

The second model includes two dummy variables (SCRIPT and FADE) as explanatory variables. The intercept of 54.07 in this model represents the estimated average amount of segments contributed by learners only in control groups. The regression coefficient for SCRIPT is −11.83, which marginally significant (p = 0.06). Since the dummy variables are coded as 1 = High Degree of Scripting, and 0 = Low Degree of Scripting, this means that on average students receiving scripts with high degree contribute 11.83 unit of analysis (segments) fewer than students in the control condition (Mean_High = 54.07 − 11.83 = 42.24).

Rerunning the analysis with the alternative dummy coding system, in which medium degree of scripting is taken as the reference group, will yield that the estimated coefficient is negative and significant (β = −10.32, p < 0.05), indicating that the mean of Overall Participation in the groups with high degree of scripting is significantly lower than the groups with medium degree of scripts (Mean_Medium = 52.57).

The group-level variance is still not significantly different from zero (χ² = 31.93, df = 29, p = 0.32) in the model. However, note that the level 2 variances are dramatically reduced, reflecting the importance of the Degree of Scripting as the predictor. It isn’t possible to obtain a true R-squared value in HLM; however, there are statistics that provide a value of the total variance that can be explained by the model at the different level, and they are often referred to as R-squared values (Kreft and De Leeuw 1998). The results of this analysis suggested that 87% of the variance in Overall Participation across groups could be explained by the predictors of Degree of Scripting at the group level. And only 1% of variances within groups are explained in the model.

In addition, it is generally preferable to carry out a likelihood ratio test by estimating the ‘deviance’ for the full model and the null model omitting the level 2 variables (Raudenbush and Bryk 2002). The deviances are 864.08 and 858.57 respectively, with a difference of 5.51 which is referred to the chi-square distribution with the difference in number of parameters as degrees of freedom (here, df = 2) and is marginally significant (p = 0.06). According to the likelihood ratio test, I can conclude that a better model-fit can be achieved by adding the explanatory variable at the group level.

4.7.2.2 The Effects of Degree of Scripting on Off-task Activity

Table 4.7 presents the results of a series of models for Off-task Activities. Model 1 is a null model that contains only an intercept term. The intercept of 6.80 in this model, which is the average frequency for off task behaviour, occurred during the collaborative learning phases. The null model estimates the group-level variances as 22.62, which is significantly different from zero (χ² = 239.85, df = 31, p = 0.00). Hence, the proportion variance at the group level is estimated as 0.684, which is very high, and means that only one thirds of variances are situated within groups.

Table 4.7 Fixed effects and variance estimates for models of the degree of scripting affecting learning processes in terms of off task activity

In model 2, two dummy variables (SCRIPT and FADE) are added to explain the large variances at the group level. The model predicts a mean value of 10.70 for the control group, which reduces to 7.01 (p < 0.01), and 4.34 (p < 0.05) respectively for the learner in the groups with a high and medium degree of scripting. By rerunning the dummy coding variable, it can be estimated that the mean value of Off-task Activities in groups with high degree of scripts (Mean_High = 10.70 − 7.01 = 3.69) is even slightly lower (p = 0.05) than groups with fading-out scripts (Mean_Medium = 10.70 − 4.34 = 6.36).

By comparing the variance components of model 1 with model 2, it is clear that entering the dummy variables representing various degrees of scripting decreases the group-level variances considerably, while it is still significantly different from zero (χ² = 165.10, df = 29, p = 0.00). The effect of the Degree of Scripting is clearly significant, which suggests that 36% of the variances could be explained by the treatments. In addition, the model 2 (same with model 3) achieves a better fit than the null model for the difference in deviance of both models is highly significant (χ² = 11.95, df = 2, p = 0.00).

4.7.3 RQ 2: How Does the Degree of Scripting Affect Learning Outcomes of Argumentative Knowledge Construction?

The quantitative findings show that collaboration scripts can influence the participation. More detailed analysis has been conducted to examine how does the degree of Scripting affect learning outcomes of Argumentative Knowledge Construction with respect to the knowledge acquisition of ‘Learning to Argue’.

The following three excerpts provide examples illustrating various patterns of discussion threads in experimental conditions. Specifically, Fig. 4.9 shows the flow charts that derives communication threads out of the structure of relating messages. Messages with a reference to themselves initiate new threads, each of which is assigned a unique thread number. The index number shows the exact position in the thread tree. Messages referencing other messages inherit their thread number. The thread number is part of the index number, based on the position in the resulting thread tree. Threads can contain branched trees as well as chains, or they can be single entries (Figs. 4.10 and 4.11).

Fig. 4.9

Discussion threads with low degree of scripting

Fig. 4.10

Discussion threads with medium degree of scripting

Fig. 4.11

Discussion threads with high degree of scripting

4.8 Summary of Results and Discussion

As has already been stated above, the study dealt with the questions of to what extend the degree of scripting affects the learning processes and outcomes of Argumentative Knowledge Construction. In short, a number of notable findings have been produced and summarised under the following headings in which the compound arguments are evidenced empirically and further warranted by drawing on the interpretative Script Theory of Guidance (Fischer et al. 2013) and other theoretical principles.

4.8.1 RQ 1: To What Extent Does the Degree of Scripting Affect Learning Processes of Argumentative Knowledge Construction?

The first research question is to examine the effects of the designed collaboration scripts on various aspects of learning processes.

4.8.1.1 Without Scripts, Learners Do Participate but Engage in Off-task Activities

The empirical study revealed that collaboration scripts can influence the participation. However, it yielded mixed results with respect to Overall Participation and Off-Task Activities on task. Apparently, without scripting stimulated free coordination, and as a result, learners in control groups tended to simply express but frequently ignore or fail to concentrate on the target tasks, indicating a need for support. Conversely, while learners supported by either high or medium degree of scripting contributed less text, the number of unit of analysis relevant to arguments produced in the intervention conditions was significantly higher than in the control groups with low degree of scripting. In sum, the results confirmed the hypotheses, that is, scripting students’ activities positively contributed to active participation (favourable behaviour relevant to arguments) and suppressed unexpected behaviour (off-task activities).

The surface methods of counting messages to determine students’ participation can provide a useful initial orientation, but more detailed analysis is needed to understand the underlying mechanisms of group interaction (Strijbos and Stahl 2007). Taken together, the results with respect to the first research question partially supported the hypotheses that the external collaboration scripts employed in the present study are effective to guide learners to actively engage in argumentative and collaborative activities.

4.8.2 RQ 2: To What Extent Does the Degree of Scripting Affect Learning Outcomes of Argumentative Knowledge Construction?

4.8.2.1 Medium Degree of Scripting Did Not Promote the Internalization of Scripts in Terms of ‘Learning to Argue’ Knowledge as Expected

First of all, the results proved the effects of collaboration scripts in the intended direction by offering evidence in favour of the medium degree of scripting. The superiority of adaptively scripted collaboration over under-scripted and over-scripted collaboration, however, was observed only on certain learning outcomes.

Specifically, the results did not prove the assumed effects of adaptive scripts with respect to ‘Learning to Argue’ knowledge (Internalized Collaboration Scripts). The empirical finding was against this initial assumption in which fading was expected to foster the internalization of collaboration scripts.

4.8.2.2 High Degree of Scripting Fostered the Internalization of Collaboration Scripts—‘Learning to Argue’ Knowledge

The negative effect of ‘over-scripting’ on the knowledge acquisition of ‘Learning to Argue’ did not occur. The empirical study showed that high degree of scripting indeed increased the internalization of ‘Learning to Argue’ knowledge. Specifically, learners with detailed external collaboration scripts produced more formally complete arguments in the transferable post-case than learners in other two conditions.

In order to explain the complex results, it appears to be necessary to utilize the optimal external scripting level principle outlined in the Script Theory of Guidance: “an external collaboration script is most effective for knowledge acquisition if it is directed at the highest possible hierarchical level of internal collaboration script components for which subordinate components are already available to the learner” (Fischer et al. 2013). From this point of view, there have appeared to have not only one optimal scripting for all kinds of knowledge acquisitions, since even single individual learners might have different prior knowledge or ‘already available’ script components for each kind of knowledge acquisition.

Different categories of knowledge are hardly compared or contrasted with one another. However, learners often lack procedural knowledge of how to construct and interpret arguments (Kollar et al. 2007). Compared with the conceptual knowledge acquisition goals, therefore, it might be more challenging for learners to construct a formally complete argument/argument sequence due to little ‘available’ script components (smaller Zone of Proximal Development, ZPD), and thus the failure of fading on knowledge acquisition of ‘Learning to Argue’ might be due to the un-individualized scripting fading out too quickly, which led to the scaffolding residing outside of learner’s ZPD and hence the unexpected effects of under-scripting.

4.9 Overall Discussions

This experimental study has unique value in that it outlines a systematic investigation by involving multi variables and their interactions, which extends our understanding of the role of internal/external collaboration scripts in the computer-supported collaborative learning.

First, this study contributes to the growing empirical literature on the effectiveness of adaptive collaboration scripts by providing a direct comparison between low, high and medium (limited adaptive) degree of scripting in a one-factorial quasi-experiment.

In doing so, this study directly addresses the call for a more rigorous investigation of the various degree of scripting, rather than the polarized with/without distinction. The empirical evidence from the present study gave clear supports for the hypothesis that the groups supported by collaboration scripts achieved performances superior to those learner in control groups on their own. The findings suggested that even though the adaptive collaboration scripts manipulated with fixed fading do not offer the real-individualized scripts, the limited adaptive fading provided learners with the opportunity to achieve a deeper understanding of the problem domain. Learner also benefited from the self-regulation and knew more ‘about’ the skilled collaboration.

Second, previous studies have shown that scaffolding improves student learning outcomes (Kollar et al. 2007; Rummel and Spada 2005; Schellens et al. 2007; Stegmann et al. 2007). However, few of these studies are accompanied by empirical evidence whether over-scripting exist and why and how the unexpected effects of collaboration scripts occurs. The current study indicates that an absolute threshold (above which the scripting degree will be ‘too much’ for learners) indeed does not exist. Rather, whether an external collaboration script being ‘too much’ or not, is relative to and depending on the available resource stored in human mind as prior knowledge/internal collaboration scripts. Therefore, a certain degree of scripting might be ‘over’ the optimal level for an experienced learner but still ‘under’ the required scaffolds for other learner with less competence. Furthermore, due to the dynamic feature of knowledge construction, the delicate balance between internal and external collaboration scripts is temporal, which means that the pervious ‘optimal’ scripting might impede following learning after internalizing practices. In this manner, adaptive collaboration scripts are not simply dichotomous alternatives to full collaboration scripts, and the key distinction between them is not seen in the specific degree provided at a given time, but in how the degree changes as the learner changes.

Generally, the purpose of the empirical study is to emphasize the need for a systematic conceptual framework underlying the effects of adaptive collaboration scripts, rather than focusing on the intelligent part of particular technologies or special pedagogical methods. This finding can help us investigate different types of adaptive collaboration support in more detail in order to increase our knowledge of when and why adaptive collaboration support is effective. The issue addressed in the current study is not to demonstrate the inherent superiority of one degree of scripting over another, but to emphasize the recognition of the scaffolding approaches toward providing appropriate supports to specific learner and particular learning goals (facets of knowledge) at a given point in time.

The recent emphasis on exploring effective scaffolding approaches to foster collaborative learning has revitalized the research interest in the most closely-studied differences between the adaptable and adaptive collaboration scripts. In a typical setting which provides adaptive collaboration scripts, the instructional variations are controlled by the teacher, program or other kind of authority, while the control is based on the needs of learners. An obvious alternative is to place the same instructional options under the control of learners (Hannafin and Land 1997). In particular, the factor of control refers to the tendency of learners to control what happens to them and direct his/her own learning process (Lawless and Brown 1997; Shyu and Brown 1992), by giving learners control they gain the opportunity to determine many aspects of their learning such as depth of study, range of content, and time spent on learning (Hadjileontiadou et al. 2012; Kirschner et al. 2004).

Learner control is appearing in recent instructional strategies because learner’s control is assumed to help learners to feel capable of engaging in an educational experience (Reeve 2006; Shroff and Vogel 2009), to use the provided choices toward his/her self-selected goals (Ryan and Deci 2000), to increase the learner’s motivation to learn (Schnackenberg and Sullivan 2000; Steinberg 1989), and ultimately to lead to improved academic performance (Kinzie et al. 1988). However, inconsistent research findings with respect to the effects of learner control have incurred benefits as well as raised problems (Lin and Hsieh 2001). For instance, it has been found that not all students are capable of monitoring and regulating their learning, Many don’t seek help when they need it (Hadwin and Winne 2001). Learner, especially novice, may not be capable of judging when they need help (Aleven and Koedinger 2000). In earlier days, Hannafin (1984) concluded that compared with program control learner control is likely to be most successful when the following conditions are met: (i) the learners are older and more skilled; (ii) the educational objective is to impart a higher order of skills rather than factual information; (iii) the content is familiar; and (iv) advice is provided to assist learners in making decisions. Similarly, Tennyson and Rothen (1979) suggested that learner control seems to work best for tasks that require minimal prerequisite knowledge and that have a simple content structure.

Given the various findings regarding the effects of learner control, one conclusion is that in order to have beneficial learner control, learners need to have at least some minimal knowledge to make authentic choices (Jones and Issroff 2005), a well established conceptual understanding of the content domain (Gay 1986), high meta-cognitive skills or high prior domain-specific knowledge (Clark and Mayer 2003) and they need to be capable of acting appropriately on that knowledge (Ross and Morrison 1989).

Recently, Wang et al. (2011) found that compared to the unscripted collaboration and non-adaptable script, an adaptable script improved individual knowledge and skill acquisition by providing learners with control over whether they would like to use the interaction-oriented prompts and the time they would like to spend on each learning task. However, it is worthy of note that the positive effects of adaptable collaboration scripts were observed after a 44-minute training phase, which helped learner “get a first experience on how to handle the learning environment and how the collaboration script worked on” (Wang et al. 2011, p. 384). From this perspective, the participants were already experienced and with the assumed sufficient prior knowledge to make appropriate use of learner control options. Therefore, they benefited from learner-controlled collaborative learning. In other words, before providing the ‘freedom’ and delivering the ‘control’ to learners, they might need to be ‘forced’ to accumulate their knowledge and skill in the relevant domain. Strictly speaking, the adaptable script employed in the study is a mixed learning approach rather than a pure controlling by learners.

Swaak and de Jong (2001) explore the issue of how much control the system should provide versus how much control the student should have. However, they do not provide a conclusive answer on what is the ideal delicate balance between student control and system control. Learner control must be approached more cautiously, since one level of control does not fit all learners. While some experienced students may perform best when learner control options are available and gain educational benefit from this freedom, other novices may “suffer as a consequence of being handed such control over their learning” (Large 1996, p. 104).

In this respect, it is important to emphasize that in designing collaboration scripts to support learner either with adaptive or adaptable script, the educator must keep in mind that the ultimate goal is to remove the script components in order to move toward appropriating self-regulated learning. The combination of the both the scaffolding approaches (adaptive and adaptable collaboration scripts), which can be further elaborated in the future work, is potentially practical and effective.

With respect to the practical implementation, I present a promising approach which identifies the core elements in designing collaboration scripts and argue that the educational program must be integrally linked with constructivist thoughts as the basis in order for significant progress in learning and instruction to occur in computer-supported learning settings. Specifically, the work also has direct implications for designing an exemplar of computer scripts that can inspire future research on argumentative knowledge construction.

Effective instruction design should be based on proven and sound learning theories (Inaba et al. 2001) for allowing effective collaborative learning to occur. Therefore, systematic instruction design with a theory-based analysis are needed to achieve actual facilitation of collaborative knowledge construction (Strijbos et al. 2004). A number of theorists have discussed the ways in which constructivist values influence instructional design and have proposed several principles of the ‘constructivist instructional design model’ (Jonassen 1994; Lebow 1993; Willis 1995). Compared to traditional instructional systems approaches to designing instruction, constructivism makes a different set of assumptions about learning and suggests new instructional principles. The challenge instructional designers face is caused by the difficulty of translating the philosophy of constructivism into actual practice (Karagiorgi and Symeou 2005) due to the lack of common and solid background concepts for collaborative learning.

In the Script Theory of Guidance, the hierarchical approach to represent internal/external collaboration scripts provides some critical advantages over the traditional method of decomposing scripts, the first of which is that the hierarchical decomposition method gives the researcher a much richer view on how collaboration scripts (internal/external) are structured across levels. The four components representing different levels can be used to decompose nearly any scaffolding approach applied in the earlier days into a hierarchical model. In this way, the outlined components can be regarded as a shared language to explicitly interpret and compare the variety of collaboration scripts at different levels of detail.

The second advantage of the hierarchical approach lies in the underlying models of Script Theory of Guidance. The scripts theory does not just place all collaboration scripts into the polarized categories of macro and micro collaboration scripts (Dillenbourg and Tchounikine 2007), but argues that there is a continuum of scripting levels, from coarse-grained play, moderate-grained scene to finer-grained scriptlets. The script architecture is actually quite divers and flexible, which provides the possibility for abstracting from a set of low-level components to build a higher-level component. From this perspective, a learner’s level of expertise is a critical factor in determining the size of component on each layer of the script architecture.

Central to the Script Theory of Guidance is not just a demonstration of the differences between a set of separate components but also the gradual transformation in learners’ knowledge, skill, and strategic processing that bring about those differences. In fact, the trajectories plotted in the Scripts Theory of Guidance offer an internalization process versus product look at collaboration scripts. Dillenbourg and Schneider found that the process of internalization is rather vague and somewhat mysterious (Dillenbourg and Schneider 1995). The hierarchical structure of collaboration scripts makes the process (typically via fading) explicit and potentially efficient.

When confronted with a specific configuration of collaborative practice (e.g. counter-argumentation), experts are able to abstract or mentally represent the pattern as a familiar internal script and act on the whole configuration as a single unit at higher level (play or scene). These might be processes that don’t require much effort but which map the collaborative practice(s) onto the most appropriate schemata more effectively (Alexander and Judy 1988). Therefore, those experienced learners may not need any additional instructional guidance because their internal scripts provide the full guidance. In contrast, novices lack sophisticated internal collaboration scripts associated with a practice or situation at hand.

As a consequence, external collaboration scripts that serve as instructional guidance that maybe essential for novices may have negative consequences for more experienced learners (over-scripting). Besides the horizontal differences among different learners, it is also vertically different, particularly when internal collaboration scripts of the same learner may be ‘sublimated’ to a higher level after further intense interactions take place. For more experienced learners, rather than risking the counterproductive redundancy of external collaboration scripts, it may be preferable to eliminate the scripts (fading). Thus, the most important instructional implication of this flexible collaboration script is that to be efficient, instructional design should be dynamically tailored to the level of experience of the individual learners, not only horizontally but also vertically. By understanding the developing process of collaboration scripts, educators can better conceptualize and carry out their roles as guides for learners at different points in this journey.