Problem statement

Research on Computer-Supported Collaborative Learning (CSCL) investigates the learning of complex topics in technology-supported social contexts, such as text-based asynchronous online discussions (Hew et al. 2010; Schellens and Valcke 2006), or web-based science courses (e.g., Linn and Eylon 2011). However, collaborating effectively in CSCL settings is not a trivial task. As Weinberger et al. (2010) have found, for example, learners in small groups may even acquire less knowledge than learners who deal with the same topic individually in a computer-supported learning environment. One possible reason for this may be that collaboratively learning about complex and challenging topics in CSCL requires learners to constantly make necessary adjustments regarding their goals and behaviours (Azevedo 2007), and in addition to coordinate their goals and behaviours with those of their learning partners (Järvelä and Hadwin 2013). Thus, regulation skills are regarded as a critical component of effective collaborative learning in CSCL settings (Dillon and Greene 2003; Quintana et al. 2004; Rummel and Spada 2005).

However, not all learners possess the regulation skills that would guarantee smooth collaboration in CSCL settings. For learners with low collaborative learning skills, collaboration needs to be structured externally, and collaboration scripts are a way to provide such structure, as they specify, sequence and distribute learning activities and roles among learners in a small group (Kollar et al. 2006; Rummel and Spada 2005).

Providing learners with collaboration scripts can however be a double-edged sword: If designed in a too coercive way, they may take too much regulation responsibility away from the learners, having the effect that learners have only very few opportunities to practice the strategies that are instructed by the script in a self-regulated way (e.g. Dillenbourg 2002; Hesse 2007; Rummel et al. 2009). The challenge for instructional designers therefore is how to design collaboration scripts in a way that they keep a balance between self-regulation and external structure (Wecker et al. 2010b); to guarantee that they on the one hand provide guidance to collaborate at a higher level than without assistance, but on the other hand also provide learners with opportunities to internalize these higher-level collaboration strategies to possibly apply them in future collaborative learning. This challenge is the focus of this article.

Regulation in CSCL

Although self-regulation has traditionally mainly been studied in individual learning settings, there currently is an increased research interest in considering regulation processes in CSCL settings (Azevedo and Hadwin 2005; Järvelä and Hadwin 2013; Volet et al. 2009). Regulation research in CSCL on the one hand frequently refers to “classical” theories of self-regulated learning (SRL) that emphasize how individuals regulate both cognitive and affective-motivational aspects of their own learning (Zimmerman and Schunk 2011). In our work, however, we only take cognitive aspects of regulation into account and ignore the regulation of affective-motivational aspects of the learning process.

On the other hand, CSCL research has widened the perspective on regulation to include not only an individual’s attempt to regulate his or her learning, but also to address regulatory processes that may happen at the group level (Hadwin et al. 2010; Järvelä et al. 2016). Järvelä and Hadwin (2013) have proposed two different ways in which groups may engage in such a regulation. First, co-regulation refers to interactions in which an individual learner attempts to regulate his/her learning partner’s learning, e.g. by evaluating other’s learning progress or by giving suggestions on what that learner might turn to next. Theoretically, working with a peer who is more proficient in SRL offers opportunities for a peer with less regulation skills to learn regulation strategies and skills transferable to varied conditions (DiDonato 2013). Alternatively, when two or more peers jointly engage in regulation activities, Järvelä and Hadwin (2013) speak of shared regulation. In this case, regulation responsibility is distributed equally among all group members and the regulation process is negotiated within the group.

A large body of research has shown that students gain significant benefits in terms of higher level performance on tasks and achievement outcomes when they are able to effectively regulate their own, their learning partner’s, or their group’s learning in CSCL settings (e.g., Hew et al. 2010; Rummel and Spada 2005). For example, Vauras et al. (2003) examined fourth-graders’ co−/shared regulation as they worked on mathematical problems in a CSCL environment and found that effective co−/shared regulation led to better performance on mathematical tasks, as well as more use and transfer of learning strategies. Similarly, research by Järvelä et al. (2008) found that when solving a challenging educational psychology task, successful college students employed a number of co−/shared regulation strategies to sustain their collaboration within a small group.

Collaboration scripts: Facilitators or barriers for regulation in CSCL?

Research that investigates how individuals and groups regulate their learning in CSCL (as well as other computer-mediated learning environments such as learning with hypermedia; Azevedo and Feyzi-Behnagh 2010; Bannert and Reimann 2012) has revealed that students often do not possess high-level regulation skills. In CSCL, one way to compensate for the lack of well-developed self-regulation skills is to present collaboration scripts to provide “other-regulation” (Kollar et al. 2006). That way, regulation-related decisions during the learning process are taken away from the learners, so that they can focus on the enactment of particular high-level socio-cognitive activities without possibly being overwhelmed by the need to plan or monitor their learning process on their own. Collaboration scripts are scaffolds that are explicitly directed towards structuring the collaborative and interactive processes that occur within small groups; they specify, sequence and distribute learning activities and/or collaboration roles among the members of a small group (Kollar et al. 2006). Such scripts have however been criticized for possible over-scripting effects: they may take over too much regulation responsibility and thus hinder learners’ self-regulation (e.g. Hesse 2007; Rummel et al. 2009). For example, a study by Mäkitalo et al. (2005) found that a script which provided step-by-step instruction impeded college students’ knowledge acquisition relative to unscripted collaborative learning in an asynchronous online discussion. The authors suggested that students in their study may already have been able to regulate their own online discussions in an effective way and that the script therefore may have forced them to follow a learning process that did not match the one they preferred on their own. Dillenbourg (2002), in addition, warns that an “over-scripting” of collaborative learning may impede “fun and the richness of group interactions” (p. 61).

Yet, other scripts have been developed that, rather than taking regulation opportunities completely away from learners, actually support students in regulating their learning processes in a shared way. For instance, after the partners of a dyad have read a text, the “ASK to THINK-TEL WHY” script (King 1991) prompts learners to ask each other metacognitive questions. In another study by Weinberger et al. (2005), a peer-review script was developed to structure case-based asynchronous online discussions. In their study, learners worked in groups of three to analyze three educational problem cases by applying Attribution Theory (Weiner 1985). The peer-review script distributed the roles of analyst and critic, sequenced sub-activities through guiding learners through creating a first case analysis, critiquing their learning partners’ case analyses, replying to critique they received, to crafting a final analysis, and facilitated the learners’ adoption of the roles by providing interaction-oriented prompts (realized in their study as incomplete sentences). The results showed that the peer-review script supported co-regulations and co-construction of domain-specific knowledge when compared to collaborative learning without an external script.

In another study by Rummel and Spada (2005), dyads consisting of a psychology student and a medical student consulting about the diagnosis for a fictitious patient were invited to collaborate in an unfamiliar video-conferencing environment. In one condition, dyads were supported by a collaboration script that was designed to improve the quality of their co-regulation, e.g. by prompting them to clarify their learning partner’s questions or to divide labor in a meaningful way. This script condition was compared to a control condition in which no collaboration script was provided. The authors found that dyads who had received the collaboration script in a first task made better plans for a subsequent unscripted collaboration task than dyads whose collaboration on the first task had been unscripted. This can be taken as evidence for an improvement in learners’ regulation skills in this specific setting, which seemingly has been caused by the collaboration script.

It thus seems that collaboration scripts that do not take all regulation opportunities away from the learners, but rather help them engage in regulation activities at a level that would be out of reach for them without support, are promising to improve learners‘regulation skills. Yet, a tricky question is how to determine what regulation skills are out of reach for a particular learner or a particular group even more if one acknowledges that regulation skills will evolve over time as learners are collaborating (i.e. the skills that are out of reach will probably become fewer over time). In fact, providing learners with a fixed script (i.e. a script that always stays the same during the learning process), might yield more and more over-scripting the longer groups collaborate (Dillenbourg and Tchounikine 2007; Fischer et al. 2013). Thus, research is needed that investigates how to design scripts in a flexible way so that they more smoothly ‘fit’ the learners’ current regulatory needs.

Flexible scripting approaches

At least three approaches to providing flexibility in scripting can be distinguished. Firstly, some studies have looked at the effects of ‘fading out’ collaboration scripts over time, based on a predefined fading algorithm. For example, Wecker and Fischer (2011) implemented fading as a gradual reduction in the script prompts that were displayed to learners who engaged in writing a critical response to a given educational case analysis. The fading algorithm was realized as a gradual removal of script prompts after a predefined number of messages. The results showed that the fading script was no better than an unfading (i.e. “fixed”) script in supporting learners’ acquisition of regulation strategies. However, when fading was combined with distributed monitoring (i.e. when a peer was asked to check whether the learner was carrying out the activities for which the script prompts had already been faded out), strategy acquisition was significantly improved. These results show that monitoring and reflection of self-regulation play an important role for the acquisition of self-regulation skills.

Others have explored the effects of ‘adaptive’ scripting (e.g., Karakostas and Demetriadis 2011) on relevant learning processes and outcomes. A script is termed adaptive when an intelligent computer or tutoring system is available that reliably assesses learners’ performance on the fly and adjusts the collaboration script based on the level at which the group is currently collaborating. This is achieved by building a model of optimal collaboration and comparing it to a constantly updated user model that captures the quality of the collaborative process (Diziol et al. 2010). Early studies have provided evidence for the effectiveness of adaptive scripts. For instance, Walker et al. (2011) found that an adaptive script designed to improve help-giving behavior led groups to show higher levels of collaboration as compared to groups whose collaboration was supported by a fixed support. Individual learning outcomes, however, were no different between participants in the adaptive and fixed support conditions.

A third and yet under-researched way of supporting self-regulation in CSCL is to provide learners with “adaptable” scripts. A learning environment is termed adaptable when users can adjust the external support that is provided by the script to meet their self-perceived needs (Leutner 2009; Shute and Towle 2003). When learning with an adaptable system, it is thus the users who make decisions about the script support that is displayed, rather than the system or the designer (as is the case in adaptive scripting). Applied to learning with collaboration scripts, an adaptable script may grant learners the opportunity to make conscious decisions on whether or not they would like to receive support for example concerning what activity prompts to receive or concerning what roles to distribute within the group. An effective adaptation of the script may actually be seen as a prototypical regulatory learning activity (see Zimmerman and Kitsantas 1997), since it requires learners to reflect upon their past experiences with the script to make an informed decision on what script components they would need in order to reach or keep a high level of collaboration during their further collaboration. That way, students can be enabled to increasingly “appropriate” (Tchounikine 2016) the script by “making it their own” during collaboration through the continuous adjustment of the support they receive to their own self-perceived needs. Adaptable scripting may thus on the one hand be a promising way of improving the quality of the within-group regulation process, and on the other hand a powerful way of helping learners to acquire regulation skills that they may be able to use in future (i.e. unscripted) situations as well. Yet, adaptable scripting may also be problematic as it requires learners to already possess a certain level of regulation strategies. Although there is no direct evidence from CSCL studies on this matter, research from hypermedia learning has shown that by no means are all learners competent in this respect (see Azevedo 2005; Scheiter and Gerjets 2007).

Research question and hypotheses

This study examined how adaptable collaboration scripts (compared to non-adaptable scripts and unscripted collaboration) influence regulation processes and learners’ acquisition of regulation skills for learners varying in their prior knowledge on regulation strategies in CSCL. In the context of this study, the adaptable script that was used during the treatment phase (and compared to a fixed script and an unscripted condition) granted learners the opportunity to make conscious decisions on (1) what roles to distribute within the group and (2) whether or not they would like to receive and apply activity prompts and thus represents opportunities for regulation processes regarding the groups’ learning processes. Specifically, we aimed to answer three research questions:

  1. RQ 1:

    To what extent does the type of collaboration script (adaptable script, non-adaptable script vs. unscripted collaboration) affect regulation processes during treatment in a CSCL environment (main effect)?

  2. RQ 2:

    To what extent does the type of collaboration script (adaptable script, non-adaptable script vs. unscripted collaboration) affect regulation skills demonstrated in a subsequent transfer phase in a CSCL environment (main effect)?

  3. RQ 3:

    To what extent does the effect (if any) of the type of the collaboration script on regulation skills students display in a subsequent transfer phase be explained by the learners’ engagement in regulation processes during the treatment phase (mediation effect)?

As indicated above, we expected learning with an adaptable collaboration script to encourage students to engage in shared regulation processes (during treatment) and to facilitate students’ internalization of regulation skills, which should become evident by an increased engagement in regulatory processes in a subsequent, unscripted discussion, relative to students who had learned with a non-adaptable script or learning without a script during the treatment phase. In addition, we expected the hypothesized positive effects of the adaptable script on the individual acquisition of regulation skills (as measured in the subsequent transfer phase) to be mediated by the amount to which learners engaged in regulation processes during the treatment phase.

Method

Participants and design

Participants were N = 87 students (68 females, 19 males; M (age) = 24.07, SD (age) = 4.23) from a Southern German university. Most of them (N = 54) were from the department of Educational Science, and their participation was part of an assignment in a lecture. The remaining students (N = 33) were mostly from the departments of Psychology, Sociology or Communication Sciences and were evenly distributed across the three experimental conditions. These participants received either a certificate or an hourly wage for their participation. To test our hypotheses, a one-factorial pre-post test design was used, in which we experimentally varied the independent variable ‘type of collaboration script’ during the treatment phase (see below): no script, non-adaptable script and adaptable script. All participants were randomly assigned to triads, and each triad was randomly assigned to one of the three experimental conditions (see Table 1). Experiments took place at six time points within one month during the semester, with four to six triads simultaneously showing up at one time point. Each triad only participated in one learning session. Although the majority of the students came from the same program, the randomized group formation made it at least unlikely that the groups had already prior collaboration experience with each other. Also, in the asynchronous and text-based online learning environment we used (see below), students worked with nick names to not make visible to the learners with whom they collaborated. Those who were assigned to the same group were seated in different parts of the room (separated by wall panels) in order to make sure that their communication only happened online.

Table 1 Design of the experimental study
Full size table

Task, setting and learning environment

Each group was asked to go through three successive collaboration cycles (cycle 1: exposure phase; cycle 2: treatment phase; cycle 3: subsequent transfer phase). During cycles 1 and 2, each group was asked to solve three authentic problem cases. In cycle 3, only one case was to be solved within each group (for the content of the cases, see below). All problem cases were complex and ambiguous; they required students to apply concepts from a psychological theory which was presented to them on paper (during exposure phase: Cognitive Theory of Multimedia Learning, Mayer 2001; during treatment and subsequent transfer phase: Attribution Theory; Weiner 1985) and to negotiate to produce a theory-based solution for each case. Group-work on case analyses took place in asynchronous online discussions that were realized in the CASSIS environment (see Fig. 1; Stegmann et al. 2007). In CASSIS, there is one forum for each problem case. In the upper left part of the screen (see Fig. 1) is the description of the task, which is to analyze the problem case with the help of the according theory and discuss with peers (#1). At the middle left of the screen is a timer that tells the students how much time is left for the current task (#2). At the lower left of the screen is an orientation map depicting which case the student is currently working on (#5). In the upper right part of the screen, the students can read the case information (#3). The lower right of the screen contains the discussion board where students can post their messages (#4).

Fig. 1
figure 1

Screenshot of the learning environment CASSIS

Full size image

Procedure

The experiment took over three hours and comprised five phases:

  1. (1)

    Pre-test and individual learning (about 15 mins.): First, students completed online questionnaires on demographic and control variables. Also, they individually read two four-pages texts that introduced the two theories they were about to apply during the later phases of the experiment. The first text introduced the Cognitive Theory of Multimedia Learning (CTML; Mayer 2001), and the second one introduced attribution theory (Weiner 1985). Students were allowed to underline and take notes on the theory texts. Both theory texts remained at their desks throughout the experiment.

  2. (2)

    Exposure phase (44 mins): During the exposure phase, all student groups were asked to solve three cases related to multimedia design based on CTML. In all three experimental conditions, group discussions were supported by a peer-review script (Weinberger et al. 2005). The peer-review script distributed the roles of analyst and critic, sequenced the sub-activities create first case analysis, critique, reply to critique and create final analysis, and facilitated the adoption of the roles by providing interaction-oriented prompts that had also been used by Weinberger et al. (2005). While being in the role analyst, the learner assumed responsibility for the preliminary and concluding analyses on that case and responded to the critiques of his or her learning partners. While being in the critic role, students were required to constructively criticize their partners’ analyses of a case. As the time that was allocated for each step ran out, students were automatically moved from one case forum to the next and instructed to act either as analyst or critic in the subsequent forum. The execution of both roles was supported by interaction-oriented prompts (exemplary prompt for role “constructive critic”: “We have not reached consensus concerning these aspects:”; exemplary prompt for the role “case analyst”: “Regarding your suggestions for modification:“) that were automatically inserted into the message field in order to help students play their roles successfully. The peer-review script was implemented with S-COL (Wecker et al. 2010a; b), a browser plug-in that allows for a sustainable development of scripts and scaffolds that can be used with a broad variety of content and platforms. The purpose of the exposure phase was to give students the opportunity to gain experience on how to handle the learning environment and to learn how the peer-review collaboration script worked. This was necessary as without such a phase, students in the adaptable script condition would later-on not be able to make informed decisions on how to adapt the script without having collected experiences with it.

  3. (3)

    Chat (4 mins.): After a short break, students were invited to participate in a short online chat with their group mates during which they were asked to reflect on their group’s discussion during the exposure phase and plan for the subsequent discussion (treatment phase). In addition, students in the adaptable script condition (but not students in the other two conditions) were given the opportunity to adapt a first feature of the collaboration script, namely to choose the case for which they would like to play the role of analyst during the following treatment phase. In a sense, both the script and the learners themselves thus jointly regulated the learning process because the script defined two obligatory roles (analyst and critic) and learners’ made their own choices of which role to play (see Table 2 for the operationalization of experimental conditions).

  4. (4)

    Treatment phase (70 mins.): During the treatment phase, the students’ task was to discuss three problem cases (see Appendix) related to attribution theory (Weiner 1985). During this phase, our experimental variation was realized: In the unscripted condition, students worked collaboratively on the case analyses without support. They were allowed to switch between the three discussion boards and could work freely on any of the three problem cases by clicking on the relevant hyperlinks on the navigation bar. Within each discussion board, new contributions (initial messages) and responses to existing messages could be posted to start a new discussion thread or to continue an existing discussion. In this condition, none of those messages carried any pre-fabricated prompts. In the non-adaptable script condition, students analyzed the three problem cases with the help of the same peer-review script as in the exposure phase. In the adaptable script condition, finally, the same peer-review script was used, but a second feature of it was adaptable as students could choose which of the interaction-related prompts that were presented by the script they wanted to use; see Table 2). By doing this, the adaptable script was supposed to increase learners’ awareness of and participation in the regulation processes of monitoring and reflection.

  5. (5)

    Subsequent transfer phase (20 mins.): The collaborative learning phase was followed by a subsequent transfer phase that asked the same triads of students how collaborated during the treatment phase to engage in an unstructured discussion about an additional case related to attribution theory. Students’ collaboration process during this task was used to assess their acquisition of regulation skills.

Table 2 Operationalization of experimental conditions during treatment phase
Full size table

Dependent variables

The main dependent variables referred to (a) regulation processes (observed regulation activities during the chat and the treatment phase) and (b) students’ acquisition of regulation skills (observed regulation activities during the subsequent transfer phase when the peer-review script was removed).

Regulation activities during chat and treatment phase

Discourse data from the chat and the treatment phase was segmented and coded. Two independent coders segmented 10% of the individual contributions to the group discussion (Chi 1997) into meaningful message segments. Inter-rater agreement on segmentation was 94%, Cohen’s Kappa κ = .88. The remaining 90% of the material for a given group discussion was then segmented by one of the two trained coders. Subsequently, each of the resulting segments was assessed with respect to the application of the regulation skills. Beside the regulation processes, learners’ co-construction of domain-specific knowledge (knowledge related to attribution theory) was also coded, but has been reported elsewhere (Wang et al. 2011). The focus of the current work was to investigate the team-related, shared regulation processes that would be observable in the adaptable script condition compared to the other two conditions and to look at the extent to which they contribute to the acquisition of regulation skills that students would display in a new situation. More specifically, the coding scheme differentiated between contributions which represented instances of (a) planning, (b) monitoring and (c) reflection (see Table 3 for the description of codes and examples).

Table 3 Codes of shared regulation processes
Full size table

Utterances of each of these three types were counted for each student. The resulting sum scores were used as indicators for the frequency by which students engaged in regulation processes during treatment. Inter-rater agreement for coding of regulation processes was 96%, with Cohen’s κ = .85. In the end, the resulting scores included regulatory activities from both the chat and the treatment phase. To conduct preliminary analyses that checked whether students in the adaptable script condition actually made use of the opportunity to adapt the script regarding the question who would take what role during the treatment phase (see “preliminary analyses” below), we also compared the number of regulatory activities that occurred during chat separately.

Regulation skills during subsequent transfer phase

As regulation skills can be considered an important learning outcome (beyond domain-specific knowledge) in CSCL which is often ignored in CSCL research (Järvelä and Hadwin 2013), we specifically looked at students’ acquisition of regulation skills as individual learning outcomes in this work. These were measured during the subsequent transfer phase during which each group was supposed to solve one further case by aid of attribution theory. For this discussion, no script was provided, i.e. all groups engaged in unscripted discussions. The resulting (written) discourse data was segmented and coded in the same way as for the data from the treatment phase, again looking at (a) planning, (b) monitoring and (c) reflection. Inter-rater agreement on coding of regulation skills was 87%, Cohen’s κ = .71. We used the overall sum of instances each group engaged in these regulatory activities during the subsequent transfer phase as the basis for our analyses.

Control variables

We measured two control variables: (1) intrinsic motivation, and (2) prior knowledge on regulation strategies.

Intrinsic motivation

Intrinsic motivation has been shown to influence the way learners engage in learning activities and attainment of learning outcomes (Schoor and Bannert 2011). Also in CSCL research, intrinsic motivation has lately been regarded as an important factor that may have an influence on learning outcomes (Rienties et al. 2009). Students’ intrinsic motivation was assessed with the ‘intrinsic motivation’ subscale of Vallerand et al. (1989) ‘Academic Motivation Scale’ (AMS) during the pre-test phase. The scale includes twelve items (e.g. “I go to college for the pleasure I experience while surpassing myself in my studies.”), with the degree to which students agreed or disagreed with the respective statements to be provided on a seven-point Likert scale (1 = “Disagree completely”; 7 = “Agree completely”). Reliability of the intrinsic motivation scale was very good (Cronbach’s α = .92).

Prior knowledge on regulation strategies (pre test)

Students’ prior knowledge on regulation strategies was assessed by an open-ended question that was asked prior to the exposure phase. Students were asked to describe how they would structure their group’s work, what steps they would take to organize group activities and why. Students’ responses were assessed the same way as the regulation processes during the treatment phase and the regulation skills displayed in the subsequent transfer phase. Inter-rater agreement for coding of prior knowledge on regulation (pre-test) was 87%, and inter-rater reliability reached Cohen’s κ = .71.

Statistical analyses

The data in the present study has a hierarchical structure. Individual students within a group cannot be treated as completely independent from each other because of their shared group experiences. In this respect, the independence assumption for unilevel statistic techniques is violated. Hierarchical linear modelling was therefore used to account for the hierarchical structure of the data (Hox and Kreft 1994). We computed a random intercept model. Random slopes were not included, to prevent overidentification of models. Hence, only random effects between conditions are reported. All predictors and criteria except for the dummy-coded independent variables (see Table 4) were z-standardized to enable interpretation of regression coefficients as effect sizes (similar to standardized beta coefficients or correlation coefficients).

Table 4 Dummy coding of experimental conditions
Full size table

R (version 3.3.2) was used to perform all statistical analyses. The multilevel linear modeling was done using the R-package “lme4” (version 1.1–12; Bates et al. 2015). To analyze causal mediations of effects of regulation processes on individual acquisition of regulation skills, the R-package “mediation” was used (version 4.4.5; Tingley et al. 2014). The statistical tests for the preliminary analyses were performed regarding a two-sided 80% confidence interval to reduce type-II error probability. All other statistical tests were performed regarding a two-sided 95% confidence interval. The confidence intervals of the regressions coefficients of the multi-levels were estimated using the bootstrapping method.

Results

Preliminary analyses

Prior knowledge on regulation strategies and intrinsic motivation were measured to perform a randomization check as well as to increase the precision of analyses. We found significant pre-test differences between our three experimental conditions both with respect to intrinsic motivation and prior knowledge on regulation strategies (see Table 5). Therefore, both variables were statistically controlled for in all subsequent analyses. Descriptives for dependent variables are also reported in Table 5.

Table 5 Estimated means (EM) and standard errors (SE) of all control and dependent measures per condition
Full size table

Before running further analyses, we first checked whether students in the adaptable script condition actually used the opportunities to adapt the script (treatment check). We found that during chat, groups in the adaptable script condition showed more regulatory activities (M = 4.93; SD = 1.04) than groups in the fixed condition (M = 4.03, SD = 1.46). This difference was, however, not significant (t(18) = 1.59, p = 0.06, one-tailed, d = 0.72), but had a medium to large effect size. In addition, we checked how many of the messages that were sent within these two conditions actually responded to a prompt that was part of the (full) collaboration script. While in the non-adaptable script condition the percentage of prompted messages was 78%, it was only 28% in the adaptable condition. This means that students working with the adaptable script skipped many prompts that they perceived as unnecessary. Taking these results together, we interpret this as evidence that the adaptable script “worked” in that sense that students in this condition made more or less extensive use of the different opportunities to adapt the script.

Effects of collaboration scripts on regulation processes during treatment (RQ1)

To answer the first research question, multi-level regression of the type of collaboration script on regulation processes during the treatment phase was performed. Results are presented in Table 6.

Table 6 Standardized regression coefficients and standard errors (SE) of multi-level analysis on regulation processes during treatment phase
Full size table

Learners without a collaboration script engaged more in planning, but less in reflection activities during the treatment phase than those who learned with a collaboration script, either adaptable or non-adaptable. Furthermore, learning with an adaptable script enhanced learners’ participation in monitoring activities compared to learning without a script and increased the frequency of planning activities compared to learning with a non-adaptable script. Learners’ prior knowledge on regulation strategies and their intrinsic motivation had no significant effect on regulation processes of planning, monitoring, and reflection (for statistical coefficients, see Table 6).

Effects of collaboration scripts on regulation skills displayed during subsequent transfer phase (RQ2)

To answer the research question regarding the effect of the type of collaboration script on students’ engagement in regulation skills in the subsequent, unscripted transfer phase, linear mixed models were fit by REML. Results are presented in Table 7.

Table 7 Standardized regression coefficients and standard errors (SE) of multi-level analysis on regulation skills during subsequent transfer phase
Full size table

As shown in Table 7, the adaptable script had a positive effect on students’ acquisition of regulation skills as compared to the unscripted condition (p < 0.05) as well as the non-adaptable script condition (p < 0.05). There was no significant difference regarding regulation outcomes between the no script and the non-adaptable script condition. Control variables had no significant effect on learners’ acquisition of regulation skills (for statistical coefficients see Table 7).

Mediation effects (RQ3)

Research question 3 asked for the mediating role that students’ engagement in regulation activities during the treatment phase might have played for the effect of the script types on the students’ engagement in regulatory processes in the subsequent transfer phase. To answer this question, we ran a mediation analysis with the type of collaboration scripts as independent variable, engagement in regulatory processes during the subsequent transfer phase as dependent variable, and regulation processes of planning, monitoring, and reflection during the treatment phase as mediators. Results are shown in Table 8.

Table 8 Standardized regression coefficients and standard errors (SE) of regulation processes on regulation skills (subsequent transfer phase)
Full size table

Of the three regulation processes “planning”, “monitoring”, and “reflection”, only “reflection” played a significant role as mediator. More specifically, as the significant average causal mediation effect (ACME; see Imai et al. 2010) shows, reflection during treatment explained a significant proportion of the difference between the non-adaptable script condition and the unscripted condition as well as of the difference between the adaptable script and the unscripted condition in the subsequent transfer phase. Reflection suppressed (i.e. reversed) a stronger negative effect of the non-adaptable script compared to the unscripted condition. While the suppressor effect is significant (see Table 8), the negative effect of the non-adaptable script did not reach significance. The effect of the adaptable script condition compared to the unscripted condition is explained about 28% by reflection. Reflection did not explain a significant proportion of the difference between the adaptable and the non-adaptable condition (for statistical coefficients see Table 8).

Discussion

The question of how to implement flexible scripting has recently drawn more and more research attention (Dillenbourg and Tchounikine 2007; Diziol et al. 2010; Gweon et al. 2006; Fischer et al. 2013). In a recent review, Järvelä and Hadwin (2013) argued that shared regulation processes in CSCL need to be considered as a prerequisite for an effective co-construction of knowledge. Our research adds to this claim by investigating whether adaptable scripting which encourages shared regulation between an external regulation agency (the script provided by the researcher) and students in a small group is an effective way to support learners’ reflection upon the appropriation of the script and that way to increase their engagement in regulatory processes and a subsequent, unscripted group discussion.

Regulatory processes of scripted CSCL did not receive much attention until very recently (Volet et al. 2009). In this work we implemented an adaptable script and found that learners made more or less extensive use of it (see treatment check). With regard to the question how the different script types we used affected regulation processes during treatment (RQ1), results of the current work showed that collaboration scripts, both a non-adaptable and an adaptable one, had negative effects on planning, compared to learning without a script. Both types of collaboration scripts, the adaptable as well as the non-adaptable script, provide a certain level of structure and may thus have reduced the need of planning that was present in the unscripted condition. Also, this result might indicate that training learners about the script components during the exposure phase may have led to profound internalization of the script, so that students in the unscripted condition were able to take the whole responsibility of planning during the treatment phase. Learners in the two scripted conditions, in contrast, may have simply “let the script do the regulation” and thus not engage much in planning by themselves.

Nevertheless, planning was still more frequent in the adaptable than in the non-adaptable condition. This might have to do with the higher degree of structure that was realized in the non-adaptable script condition. The observed pattern seems to be that the higher the provided structure, the lower the need for planning activities. This also provides evidence for our argument that compared to the non-adaptable script, the adaptable one relieved the group at least of some of the regulation responsibilities and therefore reduced students’ engagement in planning. As a side note, it would be interesting to see how groups would engage in planning who would not receive a script during the treatment phase. Following our line of argumentation, we would hypothesize that planning activities should be the highest in such a condition, as the external support is the lowest. On the other hand, it might be that during a first unscripted phase, such groups would already establish some common standards on how to collaboration, which might then make planning in the second phase more or less superfluous. Future research needs to look into this.

Even though both, the non-adaptable as well as the adaptable script, decreased the amount of planning processes when compared to the unscripted condition, they at the same time substantially increased regulatory processes of reflection. This supports the argument that when learning without scaffolding, inexperienced learners rarely realize the importance of reflection and fail to reflect thoughtfully on what they did and how to improve (Woodward-Kron and Remedios 1998). The two script types enhanced reflection possibly because they introduced the role of a critic, whose task was mainly to comment and evaluate the contributions from the analyst (Weinberger et al. 2005). This is in line with a previous study from Wecker and Fischer (2011), who found that the assignment of the role of an evaluator gave rise to a greater amount of reflective activities compared to the same script condition without role assignment. To assign a role of critic is also used in other collaboration scripts such as the ASK to THINK – TEL WHY script by King (1991) and has been described as an important trigger for learning.

Concerning monitoring, only the adaptable script had a positive effect relative to unscripted collaboration. It seems reasonable that the adaptable script required continuous efforts towards decision making regarding when, how, and what to do during learning on the one hand (Leutner 2009), which makes an increased engagement in monitoring processes necessary to arrive at an optimal adaptation decision. The adaptable script might in this case serve as external regulation agency for students to perform regulation activities (Azevedo et al. 2008).

Our results also showed that adaptable scripts had a positive effect on regulation skills that learners showed in a subsequent new collaboration situation (RQ2), relative to both unscripted collaboration and learning with the non-adaptable script. Adaptable scripting is therefore a promising approach to implementing flexible scaffolding in CSCL to promote the learners’ autonomous regulation during learning and the acquisition and transfer of corresponding skills to new situations. It seems to be a promising alternative to fixed fading regimes such as the one used by Wecker and Fischer (2011) and the labor- and cost-intensive development of adaptive systems (Diziol et al. 2010).

To gain deeper insight into the mechanisms of how adaptable scripting promotes learners’ regulation skills, mediation analyses were performed. Results indicated the important role of reflection in individual acquisition of regulation skills. First, reflection mediated the positive effect of the adaptable script on individual acquisition of regulation skills compared to the unscripted condition; second, reflection significantly suppressed the negative effect of the non-adaptable script on individual acquisition of regulation skills compared to the unscripted condition. Yet, no indication was found that the difference between the adaptable script condition and the non-adaptable script condition with respect to regulation skills was mediated by reflection processes. Thus, reflection seem to be an important, but not sufficient factor to explain the mechanisms by which regulation skills are acquired.

While both types of scripts, the adaptable as well as the non-adaptable script, triggered reflection processes, only the adaptable script required learners to be responsible for their enactment. Possibly the interaction of reflection and self-regulated enactment is needed to facilitate acquisition of regulation skills. This explanation is in line with the perspective of self-regulated learning (Dillenbourg et al. 2009). Adaptability thus maybe increases opportunities for self-regulation, which in turn is often discussed as a significant predictor for strategy acquisition since it allows for autonomous application of the target knowledge (Zimmerman and Schunk 2001).

Limitations and implications

As with any empirical research, our study has some limitations. Firstly, the peer-review script was implemented as a short-term intervention aiming to achieve immediate effects on the outcomes. Our study therefore only permits conclusions about the effects of adaptability in short-term interventions. The fact that all participants were from the same program and their prior knowledge and motivation differed among conditions may further constrain the ecological validity of this work. It may be possible to detect more pronounced effects of adaptable scripting by using the script over the course of a longer intervention, and in a more authentic setting. Especially with respect to the question how students “internalize” or “appropriate” (Tchounikine 2016) such scripts, it seems necessary to do research that realized scripting in longer time frames. Secondly, the current study is in some ways an exploratory study. Although we compared the effects of an adaptable script to those of a non-adaptable script and unscripted collaboration, we did not vary the degree of adaptability. Further studies that manipulate the degree of adaptability (e.g. allow learners to switch the whole script off from the very beginning of their collaborative learning phase) may lead to a clearer understanding of the type of instructional design that will provide instructional support on the one hand, and give learners the freedom to self-regulate their learning on the other hand. Thirdly, we focused in the current work on the effects of adaptable scripting on regulation processes and outcomes without examining whether it had any effect on learning outcomes beyond the acquisition of regulation skills. This is because that in a previous pilot work (Wang et al. 2011) we reported positive effects of the same adaptable collaboration script on individual acquisition of domain-specific and domain-general knowledge. The aim of the current work was to explore the shared regulation processes through which adaptability adds value to non-adaptable scripts. Fourthly, as most research on self-regulated learning, we concentrated in this study on frequency analysis of regulation processes. Further studies using various methods, for example process mining techniques, might give a more holistic and qualitative view of the regulation processes (Bannert et al. 2014; Cascallar et al. 2006). A more holistic view on regulation would also be achieved with an investigation of how the participants may have dealt with motivational and/or emotional problems (see Järvelä and Hadwin 2013). Such problems were not focused on in our analyses, but should of course be taken into account in future research on adaptable scripting. Finally, it needs to be acknowledged that randomization of our participants across experimental condition only worked in a suboptimal way, as there were significant differences regarding both intrinsic motivation and prior knowledge on regulation strategies. Although both variables were treated as covariates in all analyses, the results need to be taken with caution. It would certainly be worthwhile to replicate this study with a truly comparable set of students in each condition.

Despite these limitations, our findings indicate that adaptable scripts are a promising approach to promote self-regulation in order to maximize the effectiveness of collaboration scripts in CSCL. Adaptable scripting seems to be a valuable alternative to the design and implementation of costly adaptive systems that have however been shown to have positive effects on important learning parameters in recent research as well (e.g., Walker et al. 2011).

The concept of ‘adaptable scripting’ is not yet widely recognized by CSCL researchers; important theoretical assumptions and empirical evidence for the putative advantages of an adaptable script over non-adaptable scripts and unscripted collaboration had to be taken mainly from other research areas, such as self-regulated learning (Zimmerman and Schunk 2001), and learning with hypermedia (Scheiter and Gerjets 2007). This study highlights the need for theoretical and empirical CSCL research to extend our understanding of using more adaptable forms of instructional support, beyond scripting, and perhaps to less technology-loaded environments.