An Empirical Study on Adaptable Scripting

Wang, Xinghua; Mu, Jin

doi:10.1007/978-981-10-4020-7_5

Xinghua Wang⁵ &
Jin Mu⁶

Part of the book series: Perspectives on Rethinking and Reforming Education ((PRRE))

320 Accesses

Abstract

87 university students participated in this study which compared three experimental conditions (unscripted CSCL, CSCL with a non-adaptable script, CSCL with an adaptable script). Results showed that the adaptable script applied in the current work triggered learners’ employ of internal collaboration script components that are functional for social processes on one hand; on the other hand, the adaptable script inhibited students’ application of dysfunctional internal script components for cognitive processes of collaborative knowledge construction, which in turn, enhances the quality of cognitive processes in comparison to unscripted CSCL.

Access provided by CONRICYT-eBooks. Download chapter PDF

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

5.1 Learning Environment

Computer-supported collaborative learning (CSCL) describes a variety of educational practices in which technologies are used primarily to create conditions under which effective group interactions, which constitute the most important factor in collaborative learning, are expected to occur (Dillenbourg et al. 2009). CSCL occurs at various social levels (e.g., small group, class, knowledge community), across different contexts (e.g., laboratory, classroom, field trips) and media (e.g., video, audio, text-based). In the reported work asynchronous online discussions forum, which has been regarded as a typical CSCL scenario (De Wever et al. 2006), will be introduced as a prototypical CSCL-environment. More specifically, the learning scenario that is going to be studied in this work includes learning with complex problem cases in asynchronous online discussions. In this section three main issues will be illustrated. First, how would asynchronous online discussions influence collaborative learning? Second, how would learning with problem cases facilitate favorable interactions? Third, what are the desirable cognitive and social processes in asynchronous online discussions that would yield individual gain in learning outcomes?

5.1.1 Asynchronous Online Discussions

An asynchronous online discussion is defined as “a text-based computer-mediated communication environment that allows individuals to interact with one another without the constraint of time and place” (Hew et al. 2010, p. 572). Asynchronous online discussions provide the means for discussions to occur. Discussions have been identified as a key component of CSCL by educators and researchers (Ertmer et al. 2007) on the one hand. On the other hand, students also regarded online discussions as one of the most beneficial activities to their learning (Richadson and Swan 2003). There are different ways how asynchronous online discussions would afford certain benefits from collaborative learning.

Firstly, asynchronous online discussions encourage active and equal participation in expressing one’s own ideas. Discussants in asynchronous online discussion often express their opinions in writing. This form of communication facilitates participation of those who hesitate to participate in spoken discussions (Vonderwell 2003). In addition, messages in asynchronous online discussions are often posted anonymously or with code names, which further encourages participation (Fabos and Young 1999). The written messages anonymously posted in asynchronous online discussions reduce social cues, such as dialect, gender, skin colour etc. “Because of the lack of social cues, it is more likely that people will pay more attention to the content of the messages, thus creating an environment of equal opportunity and reciprocity in roles” (Sugar and Bonk 1998, p. 3).

Secondly, asynchronous online discussions afford deep-level thinking. Communication in writing is believed to promote higher levels of thinking than in face-to-face discussion where the interactions happen spontaneously and quickly, leaving little time for in-depth thinking (Marttunen and Laurinen 2001; Newman et al. 1997). The very process of writing in itself allows time and also helps students carefully construct their ideas. Furthermore, the reviewability and revisability of messages in asynchronous online discussions would also support in-depth feedback and reflective contribution (Suthers et al. 2008). For example, a study by Marttunen and Laurinen (1999) found students in written discussions provided more structured opinions than students in face-to-face discussions. Another study by Hawkes (2001) found that asynchronous online discussion encouraged discussants’ critical reflection on content.

Thirdly, threaded discussions in asynchronous online discussions assist learning from a socio-cultural perspective. Socio-cultural perspectives suggest that individuals learn by exchanging opinions or viewpoints with one another (Palincsar 1999). For the exchange of opinions to take place, sustained online discussions, typically characterized by long discussion threads, should ideally be the pattern since in long threads there are many exchanges of postings or notes for individuals to share ideas, explore different perspectives, negotiate issues and create mutual understandings (Hewitt 2005).

5.1.2 Learning with Complex Problem Cases

Learning with complex problem cases in asynchronous online discussions further assists learning, as the problem/case-based method has been identified as an effective practice in collaborative learning (Flynn and Klein 2001; Hmelo-Silver 2004). In case-based asynchronous online discussions, carefully constructed problem cases are presented to discussion groups. These cases often consist of a description of observable phenomena that are to be understood in terms of their underlying theories central to a particular domain of study.

Proponents of the case-based method argue that “case[s] make[s] learning relevant and meaningful to the student through active participation in analyzing, discussing, and solving real problems in a specific field of inquiry” (Flynn and Klein 2001, p. 71). For example, in one of her statements for case-based reasoning, Kolodner (1997) argued that a case-based reasoner learns by acquiring cases and indexing them. The experience of solving cases or problems is an important resource for students as they learn how to identify issues to pay attention to, how to move forward, and how to project the effects of solutions they have come up with. Initial analyses of the cases help students activate whatever knowledge, formal or informal, they may have about the cases, which in turn, will facilitate the comprehension of subsequently processed information. When a group of students comes together to discuss their different perspectives (if any), the critical reflection upon their understanding of the cases help students deepen and elaborate their knowledge (Schmidt et al. 2007). Furthermore, solving real problems shift the focus of learning away from memorization of information to the application of theories, principles, and techniques to practical situations. It helps prepare students to be lifelong learners and adaptive experts (Hmelo-Silver et al. 2007).

5.1.3 Desirable Collaborative Knowledge Construction Processes

Although there is evidence for positive effects of cased-based discussion groups on learning attitude and performance (Flynn and Klein 2001), it has been argued that the effectiveness of group discussions depends very much upon the extent to which group members actually engage in productive interactions (e.g., exchanging and negotiating opinions; Hmelo-Silver and Barrows 2008). From cognitive and socio-cultural perspectives, the underlying cognitive and social processes of overt activities during collaborative learning are the mechanisms for the advantage (or disadvantage, when the processes are dysfunctional) of collaborative learning over individual learning (van Blankenstein et al. 2011).

Cognitive processes of learning take place within the individual when learners modify their own thinking and restructure their own knowledge. Social processes are induced by joint activity where learners jointly construct and negotiate meanings with each other. In the following, desirable collaborative knowledge construction processes for learning in case-based asynchronous online discussions will be introduced in more details.

5.1.3.1 Cognitive Processes

Cognitive processes describe how learners process learning materials and construct knowledge individually. For example, to solve the collaborative task, learners may activate their prior knowledge to interpret and process the learning materials. They might also refer to specific new concepts that they ought to learn. Although cognitive processes take place within individuals, opinions or output from peers could be taken as input for individually cognitive processing, for example to re-structure their own knowledge, when learning in a collaborative setting (Fischer et al. 2002; Webb et al. 1995).

According to Chi’s taxonomy, cognitive processes could either be active, constructive, or interactive (Chi 2009). Being active means doing something while learning, for example, posting messages during asynchronous online discussions. Doing something is necessary, but not sufficient in online learning to acquire knowledge. Constructive processes are characterized by learners’ production of additional outputs. Learners are engaging in constructive processes only if they undertake activities where the outputs contains ideas that go beyond and are not explicitly presented in the learning materials. For example, if a self-explanation is either nonsensical, or a verbatim utterance, then the underlying process is merely being active. But if the generated self-explanation is a meaningful elaboration that goes beyond the learning materials, then the underlying process is constructive. Being interactive describes the social aspect of cognitive processes and will be introduced later on. As Chi (2009) stated, constructive processes are more likely to encourage modification and restructuring of one’s own knowledge and therefore, represent high-level cognitive processing.

Similarly, Weinberger and Fischer (2006) differentiated in learning with complex cases to what extent learners relate to case information, to what extent they relate to theoretical concepts, and to what extent they construct relations between theoretical concepts and case information. In their framework, constructing relations between theoretical concepts and case information describes high-level cognitive processing, since it requires learners’ application of theory to solve problem cases. This framework was taken in the current work to guide analysis, as the same learning scenario, which is learning with complex problem cases in asynchronous online discussions, was used in the current study as in their previous work (Weinberger et al. 2002, 2005). The cognitive processes identified in the framework are construction of problem space, construction of conceptual space, and construction of relations between conceptual space and problem space. More details concerning cognitive processes in this framework will be provided in the following.

The construction of problem space is required for learners to gain an understanding of the problem, which is a prerequisite to successfully solve a complex problem. In order to understand the problem, learners select, evaluate, and relate single components of problem case information. To construct problem space is necessary, it has been shown however, to go beyond a concrete level of the problem space may foster knowledge construction in learning scenarios based on complex problems (Fischer et al. 2002).

In order to solve problems on the ground of theoretical concepts, learners need to acquire an understanding of the theory. “The construction of conceptual space comprises summarizing, rephrasing, and discussing theoretical concepts and principles” (Weinberger and Fischer 2006, p. 75). Learners construct relations between individual theoretical terms or principles or they make distinction between concepts. Constructing the conceptual space is essential to understand the theory to be learned.

In order to apply a theory adequately and to solve a problem efficiently, the main task in knowledge construction when learning with complex problems is to construct relations between problem and conceptual space. Relations between conceptual and problem space that learners construct can indicate to what extent learners are able to apply knowledge adequately, as well as to what extent learners approach a problem in detail. The construction of relations between conceptual space and problem space indicates which concepts or principles learners resort to in order to solve the problem. This type of cognitive activity represents a higher level of knowledge construction, the constructive processes (Chi 2009), in problem-oriented collaborative learning and has been found to be predictive for individual knowledge acquisition (Weinberger and Fischer 2006).

5.1.3.2 Social Processes

From socio-cultural perspectives, discourse activities between an individual learner and another person, who can be a peer, a teacher, a tutor, constitute the most important factor in collaborative learning (Vygotsky 1978). In the field of CSCL, interactions among peers have been given particular emphasis although without excluding other interactions (Dillenbourg et al. 2009). It has been argued that peer interactions make it more likely for learners to engage in negotiation of multiple perspectives (Hogan et al. 2000), as peer interactions are more equal and horizontal than the hierarchical or vertical interactions with teachers. A reconstruction of cognitive structure can be initiated more easily in peer interactions than in interactions with teachers (Webb and Mastergeorge 2003). Social processes describe group learners’ co-construction of knowledge which means when learning in small group learners construct knowledge together by applying individually hold knowledge and negotiating the solutions to complex problems (Weinberger and Fischer 2006).

But of course not all discourse activities are the same, concerning their contribution to knowledge co-construction. As classified in Chi’s taxonomy, being interactive is the highest level of knowledge co-construction processes underlying collaborative learning (Chi 2009). There, being interactive means more than just interacting in dialogues, as some dialogue patterns are in fact not interactive at all. For example, it is often the case that one learner dominates and makes most of the contributions and the other learning partners merely agree with a response like “ok” or “great”. Interactive processes take place only if “both peers make substantive contributions to the topic or concept under discussion, such as by building on each other’s contribution, defending and arguing a position, challenging and criticizing each other on the same concept or point” (Chi 2009, p. 83).

The social processes described in Weinberger and Fischer’s framework likewise depict how learners interact with each other, and how they relate their contributions to contributions from their learning partners in solving the task (Fischer 2001; Weinberger and Fischer 2006). Specific social activities vary in the degree of transactivity, which is defined as reasoning that operates on the reasoning of another (Teasley 1997). In the following paragraphs five social activities with increasing degree of transactivity, which were identified in the framework from Weinberger and Fischer (2006), will be introduced. They are namely externalization, elicitation, quick consensus building, integration-oriented consensus building, and conflict-oriented consensus building.

Externalization (of knowledge) means that learners explicate their knowledge without reference to other contributions. Learners externalize what they know; this may make (mis-)conceptions accessible to learning partners and bring about discussions. To externalize opinions to each other makes co-construction of knowledge possible. Externalization may indicate prior differences between learners but cannot be made responsible for variance resulting from collaborative learning.

“Elicitation has been described as using learning partners as a resource by asking questions” (Weinberger and Fischer 2006, p. 78). Elicitation aims at initiating a reaction and receiving information from the learning partners. Elicitation may foster externalization and inspire further exploration when learners find gaps of understanding (Fischer 2001). Past research showed, however, that elicitation appeared to facilitate knowledge construction only if learners asked task-related questions, received help, and applied the help in the situation themselves (King 1994; Webb 1989).

Learners need to build consensus regarding the learning task in the process of social negotiation, in order to reach a common goal, for instance, to solve a complex problem. There are different styles in reaching consensus. Quick consensus building has been described as learners simply pretending to accept the contributions of their learning partners in order to continue discourses (Weinberger and Fischer 2006). In this way, quick consensus building may not represent an actual change of perspectives, but is rather a coordinating discourse type. Quick consensus building may be detrimental to knowledge construction when learners disregard other forms of consensus building in favour of quick consensus building.

In contrast to quick consensus building, integration-oriented consensus building has been regarded as taking over and operating the perspectives of learning partners. Integration-oriented consensus building indicates that learners “show a willingness to actively revise or change their own views in response to persuasive arguments” (Keefer et al. 2000, p. 77). Learners may come to better understanding by adopting and integrating each other’s perspectives. Thus, integration-oriented consensus building has been regarded as favourable social activity with high-level underlying social processes. By building integration-oriented consensus, learners may eventually establish and maintain shared conceptions of a subject matter. Previous studies found, however, integration-oriented consensus building appeared to take place rarely in collaborative knowledge construction (Weinberger et al. 2003).

Conflict-oriented consensus building has been considered as an influential component in collaborative knowledge construction (Teasley 1997). Conflict-oriented consensus building has been described as disagreeing and modifying the perspectives of learning partners (Weinberger and Fischer 2006). By facing critique, learners may be pushed to test multiple perspectives to solve the conflicts in the process of social negotiation. This leads to more closely operation on the reasoning of their learning peers and more elaborated arguments for their positions. This reflective and constructive resolution of conflicts has been related to learning (Chan et al. 1997).

These five types of social activities in the processes of collaborative knowledge construction differ in the degree to which learners refer to the contributions of their learning partners. Integration-oriented and conflict-oriented consensus building are regarded as the most favourable types of social activities that relate to knowledge construction. These two types of consensus building are in compliance with what Chi (2009) identified as being interactive in collaborative learning. Prior findings suggested, however, without instructional support, learners often engaged in quick and superficial consensus building (Weinberger et al. 2003).

Summary

Asynchronous online discussions, as often used technology for CSCL, encourage active participation and afford individual information processing as well as social exchange. Learning in a CSCL environment, such as asynchronous online discussions, involves discourse activities with multiple underlying processes, including cognitive and social processes (King 2007).

5.2 Research Questions and Hypotheses

In the empirical study presented in this chapter, triads of learners collaborated in a case-based online discussion environment in which they had to solve psychological problem cases by aid of previously selected theories. The learning experience consisted of two parts.

In the first phase (training phase), all triads were supported by aid of a collaboration script similar to the script provided by Weinberger et al. (2005) (see Chap. 2). In a second phase (treatment phase), the same triads had to solve other cases (based on a different theory than in the training phase). During this phase, we implemented three collaboration conditions: (a) a condition in which learners received no script (NS); (b) a condition in which learners continued to receive the script from the training phase, without having the opportunity to adapt it (non-adaptable script condition; NAS); and (c) a condition in which learners were allowed to repeatedly adjust the script based on their self-perceived needs (adaptable script condition; AS). This study aimed to answer two main research questions:

RQ1: To what extent does the adaptable collaboration script have effects on cognitive processes of collaborative knowledge construction, compared to a non-adaptable collaboration script or learning without script?
RQ2: To what extent does the adaptable collaboration script have effects on social processes of collaborative knowledge construction, compared to a non-adaptable collaboration script or learning without script?

As indicated in Chap. 3, we expect that an adaptable collaboration script will facilitate cognitive and social processes of collaborative knowledge construction, relative to learning with a non-adaptable script or learning without script.

In addition, we are interested in a third research question:

RQ3: Which pattern of discussion threads can be identified when learning without script, with a non-adaptable script or with an adaptable script in a CSCL environment?

A case study aims to explore how the pattern of discussion threads are changed or shaped regarding collaborative knowledge construction processes by inducing a collaboration script (a non-adaptable or an adaptable one) to a collaborative online discussion is conducted. No hypotheses are established for the case study due to its explorative character.

5.3 Methods

This section serves to illustrate the methodology of the empirical study, including sample and design, learning environment, experimental phases, and measurement of all the variables.

5.3.1 Design and Sample

To answer the research questions listed above, a one-factorial experimental design with three conditions was used. The factor ‘adaptability’ (CSCL with a non-adaptable script vs. CSCL with an adaptable script) was experimentally varied (see Table 5.1). There was also a reference condition in which learners received no collaboration script.

Table 5.1 Design of the experimental study

Full size table

87 students from Ludwig Maximilians University (LMU) of Munich participated in this study. The sample can be described as follows (see Table 5.2).

Table 5.2 Demographic data of the participants

Full size table

Most of the participating students (54) were from educational science and their participation was required and counted as part of an assignment in a lecture. The rest were from psychology, sociology or communication sciences. They got either a certificate or hourly pay for their voluntary participation. The non-educational participants were equally distributed across the three experimental conditions. All participants were randomly assigned to small groups of three.

5.3.2 Learning Material and CSCL Environment

The students’ task was to analyze three authentic educational problem cases on the basis of Weiner’s (1985) Attribution Theory in asynchronous online discussion boards. This section will introduce the theoretical texts learners had to read as a preparation, the problem cases that were used during collaboration as well as the individual components of the online learning environment.

5.3.2.1 Theoretical Texts and Problem Cases

The learning subject was Weiner’s Attribution Theory (1985) and its application in education. Students got a short description of the theory beforehand, which they were asked to learn on their own. The theory mainly addresses the question how students seek for causes for their academic success or failure. The theory allocates causes for attribution to two dimensions, namely locality and stability. Locality means that students attribute their success or failure internally (e.g., effort) or externally (e.g., difficulty). Stability describes whether attributed causes are temporally stable (e.g., talent) or variable (e.g., luck).

This classification system explains functional or dysfunctional attributions with respect to learning motivation. Weiner (1985) assumes that in order to sustain learning motivation, failures should be attributed to variable causes such as chance, while success should be attributed to internal, stable factors such as talent. Besides the attribution of the concerned student him- or herself, attribution of other persons, such as parents or teachers, may have equivalent effects on learning motivation. The short theoretical text introduces also re-attribution trainings, which from the practical point of view may change the inappropriate attribution pattern and thereby foster learning motivation. The theoretical text is from a previous study (Weinberger et al. 2005). It is about one thousand words in length.

In the collaborative learning phase, students were asked to analyze three problem cases from practical contexts that can be considered to be familiar to students. These problem cases are complex and ambiguous, which require students to apply Attribution Theory and to negotiate upon. These cases are from the previous study from Weinberger et al. (2005) as well, each of which is about 150 words in length. The description of problem cases was embedded into the online learning environment, so that they were available to students while they posted their messages and exchanged their opinions in small groups.

Since there was a training phase before students were about to learn the Attribution Theory, atheoretical text that was about the Cognitive Theory of Multimedia Learning (Mayer 2001), which was irrelevant to the learning subject, was handed out to students as well. During the training phase, the task of the students was to analyze three other problem cases with the help of the Cognitive Theory of Multimedia Learning.

5.3.2.2 CSCL Environment: CASSIS

Group discussions were led in a web-based CSCL environment, which was a revised version of the CASSIS environment (Stegmann et al. 2007; see Fig. 5.1). CASSIS is an asynchronous discussion board in which three participants can post messages that, apart from the experimenters, only the members of the learning group could read. The participants were logged in with code names in an effort to warrant anonymity, i.e. students from the same triad were unlikely to personally know their peer members.

Upper left is the description of the task, which is to analyze the problem case with the help of the according theory and discuss with peers. Middle left is a timer that tells the students how much time left for the current task. Lower left is an orientation map depicting which case the students is currently working on. Upper right is case information and lower right is the discussion board where students can post their messages.

5.3.3 Procedure

The experiment spanned over three hours and included four phases. (1) Individual learning: participants read two four-page theory sheets and filled out questionnaires concerning demographic information and control variables. (2) Training: by applying the Cognitive Theory of Multimedia Learning students learned how to handle the learning environment and how the collaboration script works. (3) Chat: before the collaborative learning phase, students chatted online to plan for the coming phase. (4) Collaborative learning: students learned Attribution Theory and its application in education either with the help of a non-adaptable script (the same one they got during training), an adaptable script, or without script.

5.3.3.1 Individual Learning

First of all, students were introduced to the learning goals, which were to experience forms of virtual learning with new media and to learn a prominent theory of Educational Science together with two learning partners. After that, they were invited to an online questionnaire of demographic and control variables. At this phase, participants were randomly assigned to small groups of three, and each group was randomly assigned to one of the two experimental conditions. One week before the experiment, students received the theoretical texts and information about the general procedure and task of the online learning session. They were asked to learn the theory before hand, individually. However, it was impossible to control the time and effort students put into individual learning. Thus, domain-specific knowledge was therefore measured at the very beginning of the collaborative learning phase, (see Sect. 5.3.3.4) to avoid bias caused by individual learning.

5.3.3.2 Training

To start the online learning session, each student was equipped with a standard MacBook with web-browser (Firefox). With the help of this, students could communicate with each other within small groups via CASSIS. Students in triads were given socio-emotionally neutral code names (Ahorn, Birke, and Pinie). Immediately after students logged in CASSIS with code names, they were informed about the individual components of the learning environment (task description, timer, orientation map, etc.) by a standard video instruction. After that, they were asked to read cases and write messages against the Cognitive Theory of Multimedia Learning, which is unrelated to the learning subject (Attribution Theory). Their discussion was supported by a peer-review script. The training phase aimed to help students get familiar with the online learning environment and get to know how the collaboration script works, which was important especially for the realization of adaptability, since students could not be expected to adapt the collaboration script appropriately without having made experiences on how the script worked. Students’ discussions during the training phase were not assessed.

5.3.3.3 Online Chat

After a short break, students were guided to the collaborative learning phase in the same environment. Before their group work, there was a 4-min online Chat, within which students were asked to make strategic planning for the coming collaborative learning phase. In addition, students in the adaptable scripted condition were provided with opportunities in the Chat to adapt parts of the peer-review script (see Sect. 5.3.4.3), which was the role distribution.

5.3.3.4 Collaborative Learning

In this 70-min collaborative learning phase, the task of the students was to discuss three problem cases on grounds of the Attribution Theory. Here students got another standard video instruction by which they were introduced to the specifics of the individual experimental conditions. From the moment that all three participants in the same group finished watching the video instruction, the learning environment worked automatically, depending on the experimental condition. During this collaborative learning phase, a copy of the theory text was available to each student to support them in analyzing the cases. The whole discourse was recorded by means of the discussion boards within which the participants communicated.

5.3.4 Experimental Conditions

The implementation of CSCL without script, collaboration with a non-adaptable script and collaboration with the aid of an adaptable script will be illustrated below.

5.3.4.1 CSCL Without Script

Students in the NS condition worked on the case analyses without support of a collaboration script. They were allowed to switch between the three discussion boards and freely work on any of the three problem cases through navigation (see Fig. 5.2).

Within each discussion board new contributions (initial messages) that start a discussion thread could be posted or existing messages could be answered in order to continue a discussion thread.

5.3.4.2 CSCL with a Non-adaptable Script

Students in the non-adaptable script condition worked on the case analyses with the help of a peer-review script (Weinberger 2005; see Table 5.3), which assigned two different roles (role A: analyst for one of the three cases and role B: constructive critic for the other two cases) to individual learners in a small group.

Table 5.3 Peer-review script for one of the three cases with respective time limits

Full size table

Role A (analyst) took over the responsibility for the preliminary and concluding analysis on one case and responding to critiques from his or her learning partners on the same case. In the role of critic (role B) students were required to constructively criticize their partners’ analyses of the two other cases. Each student took the analyst role for one of the three problem cases and the critic role for the other two cases. The execution of the two roles was supported by interaction-oriented prompts (see Table 5.4), which were automatically inserted into the message field in order to help students play their roles successfully. In addition, there was a time limit for each sub-activity.

Table 5.4 Prompts to support the roles of critic and analyst

Full size table

5.3.4.3 CSCL with an Adaptable Script

In the adaptable script condition the peer-review script introduced above was adaptable. “Adaptability” in the current study was operationalized by (1) providing students self-control over role distribution, which means, distribution of responsibilities for case analyses was not determined by the collaboration script, but was based on students’ group decision during Chat (see Fig. 5.2) and (2) providing students self-control over their use of interaction-oriented prompts, which means, students were allowed to switch on/off the prompts according to their own perceived needs (see Fig. 5.3). It is students’ individual decision whether they would like to use the prompts or not (Fig. 5.4).

5.3.5 Operationalization of Dependent Variables

Discourse data, which was recorded by means of the discussion boards within which the participants communicated during the collaborative learning phase, was assessed from different dimensions. Cognitive and social processes of collaborative knowledge construction were coded with the help of the coding system developed by Weinberger and Fischer (2006).

First of all, two independent coders segmented 10% of all discourse data into units of analysis (Chi 1997), which were meaningful pieces of stated or declared messages in the current study (most often is a sentence with a punctuation mark or a question mark, but it could be also a single word or a group of sentences). Accuracy on segmentation was 93%, reaching to a Cohen’s Kappa of κ = 0.85. The remaining 90% of the material were then segmented by the trained coders individually.

Secondly, each of the resulting segments was rated as either on-task or off-task. On-task means the learners attempt to solve the task at hand (e.g., “Should we start now to apply what we learned from the attribution theory?”) while off-task means learners talk something unrelated (e.g., “What bad weather it is!”). Accuracy of rating was 96%, κ = 0.83 (based on 10% of all segments; henceforth the same criterion for calculating inter-rater agreement was applied).

In a last step, each on-task utterance was coded from both a cognitive and a social dimension based on the framework from Weinberger and Fischer (2006). Both coding schemes are introduced in more detail below.

5.3.5.1 Measure of Cognitive Processes

According to Weinberger and Fischer (2006), when learning with complex cases, cognitive processes describe how learners construct the problem space, the conceptual space, and the relation between problem and conceptual space. An utterance is coded as construction of problem space when learners try to gain an understanding of the case by selecting, evaluating, and relating individual components of case information (e.g., “The student in the case thought that he failed in an exam because of inability”). Construction of conceptual space is defined as when learners try to gain understanding of the theory by constructing relations between individual theoretical terms or principles (e.g., “Internal stable attribution of failure has negative effects on learning motivation.”). Construction of relations between problem space and conceptual space describes discourse activities when learners resort to theoretical concepts in order to solve problems (e.g., “The student is attributing internally stable when he took ability as the reason of his failure.”).

The category “construction of relations between problem space and conceptual space” is considered as representing the highest quality of knowledge construction, since by applying theoretical concepts to a problem, students undertake activities that as an output produce ideas that go beyond the presented learning materials. As individuals vary in the amount of overall contributions, the percentage of utterances coded as construction between problem space and conceptual space in all cognitive utterances was taken as the indicator of the quality of cognitive processes of collaborative knowledge construction for each individual learner in order to avoid biases caused by more or less individual contributions in general (Weinberger 2003). Accuracy of coding on cognitive processes was 94%, Cohen’s κ = 0.73.

5.3.5.2 Measure of Social Processes

Social processes of collaborative knowledge construction describe to what extent learners refer to contributions of their learning partners (Fischer et al. 2002). Externalization (of knowledge) means that learners explicate their knowledge without reference to other contributions. Most often the initiating message to start a discussion thread is coded as externalization (e.g., “The attribution theory says…”). Elicitation has been described as taking peers as a resource by asking questions. Elicitation aims at provoking reactions from the learning partners (e.g., “What kind of attribution is ‘luck and chance’, would you like to give an example?”).

Quick consensus building is described as learners simply make agreement in the form of a short sign of approvals or affirmatively repeat utterances (e.g., “I agree with you in every respect”). Integration-oriented consensus building indicates that learners adopt and integrate each other’s perspectives to gain a better understanding of the learning material (e.g., “And of course not only the teacher, but also the parents’ attitude matters, I agree with this point”). Conflict-oriented consensus building describes how learners disagree and modify the perspectives of learning partners (e.g., “but I think it’s not the right thing to do when you try to persuade Michael that his failure results from the environment”).

Integration-oriented and conflict-oriented consensus building are regarded as the most favourable types of social activities that relate to collaborative knowledge construction (Weinberger and Fischer 2006). Percentages of utterances coded as integration-oriented and conflict-oriented consensus building in all utterances coded as social processes were taken as indicators of the quality of social processes. Accuracy of coding on social processes was 94%, Cohen’s κ = 0.91.

5.3.6 Control Variables

Although not being manipulated, the control variables are regarded as moderators that might explain the variance in collaborative knowledge construction processes and individual learning outcomes when learning with collaboration script in a CSCL environment.

5.3.6.1 Domain-Specific Prior Knowledge

Students’ domain-specific prior knowledge was assessed by the initial individual analysis of one of the three problem cases before their group discussion during collaborative learning phase. The initial unsupported case analysis by an individual student was coded against a check list from a previous study (Stegmann et al. 2007), which employed the same cases in a CSCL environment. The check list itemizes possible correct relations between the case and the attribution theory (e.g., “Her friend is attributing externally when she took difficulty of the task as the reason of her failure.”). By checking the individual analysis the number of correct relations an individual learner pointed out was calculated as the indicator of individual’s domain-specific prior knowledge. Inter-rater agreement was 91%, amounting to a Cohen’s Kappa of κ = 0.78.

To avoid biases caused by discrepancy in difficulty among the three problem cases, z-values within each case were calculated to make students’ prior knowledge comparable across different cases.

5.3.6.2 Internal Collaboration Scripts

Students’ internal collaboration scripts were assessed by an open ended question at the very beginning of the training phase, after the video instruction. The question asked learners to describe how they would like to organize their group work, more specifically what steps they would like to take and why.

More specifically, students’ answers to this open question were differentiated whether a contribution represented “task specification” (e.g., “I suggest that we read and summarize our analyses in the discussion thread.”), “role distribution” (e.g., “each of us takes one case”) or “sequencing” (e.g., “We should firstly work on our own analyses.”). The inter-rater agreement regarding coding on internal collaboration scripts was 87%, and the inter-rater reliability was Cohen’s κ = 0.71. The occurrence of these codes was counted and the resulting sum scores were used as an indicator of internal collaboration scripts.

5.3.6.3 Initial Intrinsic Motivation

Students’ initial intrinsic motivation before they undertook this experiment was assessed by the “Academic Motivation Scale” (AMS) from Vallerand, Blais, Brière, and Pelletier (1989) during the pre-test phase (see Appendix 5), which measures a relatively stable construct of motivation towards education amongst college and university students. The intrinsic motivation sub-scale of AMS included twelve items (e.g., “I go to college for the pleasure I experience while surpassing myself in my studies.”), with Cronbach’s α = 0.92 in the reported study.

5.3.7 Statistical Analyses

The data in the present study had a hierarchical structure. The individual students and the randomly formed small groups could be defined at separate levels of a hierarchical system (Hox 2010). In this respect, the assumption of independency for using unilevel statistic techniques was violated. This meant that in the present study, the data from individual students within a group could not be treated as completely independent because of their shared group experiences (Hox 2010). Therefore, hierarchical linear modelling was applied.

To investigate the research questions aforementioned, two-level models were built. A random intercept null model was calculated as the very first step, which only estimates the intercept for the specific dependent variable, without involving any explanatory variables. Random intercept models other than random slope models were taken for the reason that it was not the random variance between groups but the systematic variance between experimental conditions which was of interest in the current work. Therefore, random intercept models were taken to control the random variance between groups rather than to estimate it, as done by random slope models. In the null model, the total variance of the according dependent variable was decomposed into between-group, and between-student variance. As a next step, predictive variables were added to the null model to test the hypotheses.

The descriptive results were calculated with PASW statistic 18. HLM 6.08 (Raudenbush et al. 2004) was used to perform the multilevel modeling. Model estimation was based on the Restricted Maximum Likelihood (RML) solution. At individual level ground centering was used. At group level no centering was used. All analyses assume a 95% confidence interval.

5.3.8 Qualitative Approach

The quantitative approach to analysis of collaborative knowledge construction processes quantified the cognitive and social processes with the aid of respective coding scheme. Qualitative analyses might further reveal the differences that a non-adaptable script and an adaptable script brought about to learning processes in collaboration in a CSCL environment, which was asynchronous online discussion in the current study.

One group from each of the two experimental conditions was randomly selected for case studies. In general, case analysis means a search for patterns in data (Neuman 1997). According to Yin (2003) there are three general analytic strategies for analyzing cases: relying on theoretical propositions, thinking about rival explanations, and developing a case description. Here the third strategy, developing a case description, was used to provide further information about the collaboration processes in addition to the quantitative analysis.

Within the selected group, one of the three problem cases, which was case ‘Math’, was analyzed qualitatively. Discourse activities on case ‘Math’ were described with respect to all available information, including the number of messages, the author of each message, the length of each message, character of each message (prompted or not), original text of each message, number of discussion threads, and the length of each discussion thread. To keep the originality of the messages, texts were not translated.

The information from the detailed described group was used to explore the pattern of discussion threads in a specific experimental condition. Once the pattern was identified or uncovered, two other groups from the same experimental condition were randomly selected aiming at providing evidence for reliability (Yin 2009). The original messages of the other two groups in each condition were spared from presentation but the structure of discussion threads, which helped validate the pattern of discussion threads identified in the detailed presented group in the same condition.

5.4 Results

Quantitative results of the study will be reported on grounds of all 60 participants. The results will be reported following the sequence of research questions in Sect. 5.2. After reporting results from quantitative analyses, results from the case studies on grounds of three groups in each experimental condition will illustrate the pattern of discussion threads when learning with or without script. At the end of this section, there will be a short summary of all empirical findings.

5.4.1 Preliminary Analyses

Before performing statistical analyses related to the research questions, it was checked whether learners in the three experimental conditions were comparable with respect to their learning prerequisites, including domain-specific prior knowledge, internal collaboration scripts and initial intrinsic motivation (see Table 5.5).

Table 5.5 Descriptive analyses of control variables

Full size table

On initial intrinsic motivation there was a significant difference across the three experimental conditions (F _(2.84) = 3.85, p < 0.05). Students in the AS condition had higher initial intrinsic motivation than those in the NAS condition (p < 0.05; with Bonferroni Post Hoc test). On none of the other measures significant difference across the three conditions was found (F _(2.84) = 0.05, n.s., for domain-specific prior knowledge; F _(2.84) = 0.97, n.s., for internal collaboration scripts). To avoid biases of effects of the treatment on the post-test measures, however, all aforementioned individual learning prerequisites were taken as control variables in all of the following analyses.

5.4.2 Effects of Adaptable Script on Cognitive Processes (RQ1)

First, results from descriptive analyses concerning the cognitive processes of collaborative knowledge construction in each experimental condition are reported in Table 5.6. As can be seen from the descriptives, cognitive processes are clearly unevenly distributed. The main cognitive process of students across all three experimental conditions was the construction of relations between conceptual space and problem space. Students engaged least in cognitive processes to construct the conceptual space (5% and below).

Table 5.6 Frequencies and percentages of the three cognitive processes

Full size table

Descriptive statistics suggest differences among experimental conditions. The adaptable script reduced the frequency of overall cognitive processes, compared to the fixed script. The percentage of high level cognitive processes (construction of relations between conceptual space and problem space), however was higher when learning by aid of an adaptable script (76%), when contrasted to learning without script (57%).

To test RQ1 about the effect of the adaptable script on cognitive processes, results from multilevel modelling are presented in Table 5.7.

Table 5.7 Multilevel regression for the effects of adaptable script on cognitive processes

Full size table

The null model showed that the between group variance was significantly different from zero (χ ² = 68.41, df = 28, p < 0.001), and explained 33% of the total variance on the dependent variable (Intra class correlation (ICC) = 0.33). ICC indicates that the between group variance should be taken into account, multilevel modeling is hence an appropriate method for hypotheses testing.

In model 1, the independent variable (adaptability of collaboration script) was computed with possible explanatory variables at individual level serving as covariates. Shown in model 1, the quality of cognitive processes was significantly higher when learning with an adaptable script, in comparison to that when learning without script (β = 0.21, p < 0.05). There was no significant difference between the NAS condition and the NS condition with respect to the quality of cognitive processes.

Model 4 was basically the same as model 1, with the NAS condition as reference. Results in model 4 revealed that learning with an adaptable script led to no higher quality of cognitive processes when compared to a non-adaptable script.

Results reported in model 2 and model 5 showed that there was no interaction between domain-specific prior knowledge and adaptable script with respect to the quality of cognitive processes.

It was shown in model 3 and model 6 that there was an interaction between internal collaboration scripts and external collaboration script. Compared to the NS condition, a non-adaptable script or an adaptable script inhibited students’ internal collaboration scripts from playing a positive role in their performance on cognitive processes of collaborative knowledge construction.

5.4.3 Effects of Adaptable Script on Social Processes (RQ2)

With respect to social processes of collaborative knowledge construction, firstly, results from descriptive analyses in each experimental condition are reported in Table 5.8. The most often occurring social processes across all three experimental conditions were the integration-oriented consensus building and externalization. Students engaged least in social processes of elicitation.

Table 5.8 Frequencies and percentages of social processes

Full size table

Descriptive results suggest differences among experimental conditions. The adaptable script reduced overall social processes relative to NS and NAS condition. The percentage of higher level social process (integration-oriented consensus building) was, however, higher when learning by aid of a collaboration script, especially by an adaptable script (61%), when contrasted with learning without script (54%). Regarding conflict-oriented consensus building, which also represents a high level social activity of collaborative knowledge construction, there was descriptively no difference across the three experimental conditions (9% in all three conditions).

To test RQ2 about the effect of adaptable script on the quality of social processes of collaborative knowledge construction (percentage of integration-oriented consensus building as well as conflict-oriented consensus building), multilevel modeling was performed and results are reported in Tables 5.9 and 5.10.

Table 5.9 Multilevel regression for the effects of adaptable script on social processes of integration-oriented consensus building

Full size table

Table 5.10 Multilevel regression for the effects of adaptable script on social processes of conflict-oriented consensus building

Full size table

The null model shows that the between group variance was not significantly different from zero (χ ² = 34.28, df = 28, n.s.). Between group variance explained 11% of the total variance on the dependent variable (ICC = 0.11).

Results presented in model 1 and model 3 showed that an adaptable script had neither positive nor negative effects on social processes of integration-oriented consensus building, compared to a non-adaptable script. When contrasted to NS condition, learning with the aid of a collaboration script, either a fixed one or an adaptable one, did not give rise to a higher quality of social processes with respect to integration-oriented consensus building.

Tested in model 2 and model 4, there was no interaction between internal and external collaboration scripts on social processes of integration-oriented consensus building. Internal collaboration scripts had no significant effect on social processes of integration-oriented consensus building in any of the three experimental conditions.

As for integration-oriented consensus building, both the non-adaptable script and the adaptable one had no effect on social processes of conflict-oriented consensus building (shown in model 1 and model 3 in Table 5.10).

Shown in model 2 and model 4, there was interaction between internal and external collaboration scripts on social processes of conflict-oriented consensus building (β = 0.09, p < 0.05). Internal collaboration scripts had negative effects on conflict-oriented consensus building when learning without script. When learning with an adaptable script, internal collaboration scripts had a positive effect on social processes of conflict-oriented consensus building.

Seeing from the results reported above in Tables 5.9 and 5.10, collaboration scripts, neither a non-adaptable nor an adaptable script, had any effect on social processes, relative to learning without script. Internal collaboration scripts interacted with adaptable script on social processes of conflict-oriented consensus building.

5.4.4 Effects of Collaboration Script on the Pattern of Discussion Threads (RQ3)

In this section, three groups from each experimental condition were randomly selected and presented in the following. Within each group, discourse on one of the three problem cases (case ‘Math’) was analyzed qualitatively. One of the three groups in each experimental condition will be presented with all the available information, including the number of messages, the author of each message, the length of each message, character of each message (prompted or not), number of discussion threads, length of each discussion thread, structure of discussion threads, and content of each message (original text). The original text of the other two groups from the same condition will be saved from presentation. This detailed presented group in each condition is for exploration, while the other two groups in the same condition provide evidence of reliability.

5.4.4.1 Discussion Threads in the NS Condition

In the following, discourses (on case Math) of a learning group without support of a collaboration script (NS) will be analyzed. The problem case the students were required to analyze and discuss was about a fictional student who is subject to a variety of attributions regarding his in-class-failure in mathematics. Ahorn (nickname) in this NS group was a 29 year old female; Birke was a 23 year old male; and Pinie was a 29 year old female. The three participants were 2nd- semester students at the University of Munich from educational science, when they undertook this experiment.

As Fig. 5.5 shows, there were seven messages posted on case ‘Math’. Four out of the seven messages were from Pinie, one from Ahorn, two from Birke (Fig. 5.6).

Pinie started discussions by providing a first analysis of the problem case. There were 187 words in this message (Fig. 5.7).

Ahorn developed a discussion thread by responding to the first message. There were 136 words in this message (Fig. 5.8).

Pinie further developed this discussion thread by feeding Ahorn’s suggestion back. There were 24 words in this message (Fig. 5.9).

The second discussion thread was developed by Birke, also by answering Pinie’s first analysis. There were 42 words in this message (Fig. 5.10).

The fourth message was the response from Pinie to Birke. There were 18 words in the message (Fig. 5.11).

The sixth message was Birke’s feedback to Pinie. There were 21 words in this message (Fig. 5.12).

The seventh message was the last message on case ‚Math’, posted by Pinie. It was the final analysis of the problem case based on the first analysis and her discussions with Ahorn and Birke. There were 256 words in this message.

On average there were 48 words in each message during discussion. The first analysis was not counted as a message during discussion as it was posted before discussion started. The final analysis was not counted as discussion as well, because it was rather an individual product after discussion. Although there was no support from the peer-review script during their group work, students in this group organized their discussion the way the peer-review script required. As there was a training phase before their collaborative learning (see Sect. 5.3.3.2), students in this NS group obviously internalized the script quite well and were able to apply the internalized script implicitly to guide their collaboration. Pinie in this unscripted group implicitly played the role of analyst, who was responsible for the first and final analysis and response to critic. Ahorn and Birke implicitly played the role of critic, whose task was mainly to provide constructive critique to the first analysis and negotiate with the analyst on different opinions. This group can be regarded as the ‘trained scripted’ group.

To provide evidence for reliability of the discourse structure when learning without script, two other NS groups were randomly selected and the discourse structures in these two groups will be presented below. Figure 5.13 showed how discussions proceed in the second NS group.

In the second NS group, five messages were posted altogether. There were two discussion threads, one with two messages, and the other with three. On average there were 62 words in each message during discussion. In this NS group, discussions were not that interactive as Ahorn’s first analysis got only one response from each learning partner. In addition, only one response (from Birke) got a further answer from Ahorn. This group was characterized as a “quick-consensus building” group.

Figure 5.14 showed the discourse structure in the third NS group.

In the third NS group, nine messages were posted altogether, four from Ahorn, two from Birke and three from Pinie. There were three discussion threads, with three messages within each thread. The first discussion thread was so developed that Ahorn posted the first analysis, Pinie provided feedback and Ahorn agreed with Pinie. The second discussion thread started with a first analysis from Pinie, and ended up with responses from both Birke and Ahorn. The third thread started with a summary message from Pinie, and followed by the agreement from both Ahorn and Birke. On average there were 20 words in each message during discussion. In this NS group there was no clear role distribution, as both Ahorn and Pinie contributed a first analysis to group discussion. Seeing from the discourse structure, this group was characterized as an “equally distributed” group.

As reported above, there was no consistent structure of discussion threads across different unscripted groups. Group learners could either follow a trained script, play the roles implicitly, they could quickly reach consensus without too many interactions, or they could distribute tasks relatively equally among peers. There could be two discussion threads with mediate length (three to four messages), two threads either short (two messages) or mediate (three messages), or three discussion threads with the same length (three messages).

5.4.4.2 Discussion Threads in the NAS Condition

Discourse of a learning group facilitated by the non-adaptable script will be analyzed in the following. Ahorn in this NAS group was a 25 year old female; Birke was a 26 year old male; while Pinie was a 23 year old female. They were all 2nd-semester students from educational science.

As Fig. 5.15 showed, there were eight messages posted on case ‘Math’. Four out of the eight messages were from Ahorn, two from Birke, two from Pinie.

Within these eight messages, two discussion threads were developed, with four messages in each (Fig. 5.16).

Discussions started with the first analysis of the problem case posted by Ahorn. There were 145 words in this message (Fig. 5.17).

Birke developed a discussion thread by responding to the first message. There were 69 words in this message. One of the three interaction-oriented prompts (My proposal for an adjustment of the analysis is:) was used by Birke (Fig. 5.18).

Ahorn further developed this discussion thread by answering Birke’s critique. There were 60 words in this message. Also one prompt (Regarding the modification proposals:) was applied (Fig. 5.19).

The first discussion thread ended up with Birke’s second response to Ahorn. There were 22 words in this message. One prompt (We have not reached consensus concerning these aspects:) was used (Fig. 5.20).

The second message on case ‘Math’, which developed the second discussion thread, was posted by Pinie. It was a response to Ahorn’s first message, the case analysis. There were 23 words in the message. One prompt was used (Fig. 5.21).

The fifth message was Ahorn’s response to Pinie. There were 75 words in this message. It was a prompted message (Fig. 5.22).

The sixth message was the second critique from Pinie to Ahorn. There were 58 words in this message. One prompt was used (Fig. 5.23).

The seventh message was the last one on this case. It was the final case analysis posted by Ahorn. There were 121 words in this message.

On average there were 51 words in each message during discussion. Ahorn in this FS group played the role of analyst (as predefined by the script), who was responsible for the first, final analysis and response to critic or suggestions from Birke and Pinie. Birke and Pinie played the role of critic, whose task was mainly to provide constructive critique to Ahorn’s analyses of the case. All of the messages during discussion were prompted. Altogether there were six (out of 18) prompts being used by students. This group adhered to the script quite well. The structure of their discussion threads was exactly the same as what the script required.

Figure 5.24 showed how discussions proceed in the second NAS group.

In the second NAS group, eight messages were posted altogether. There were two discussion threads, with four messages in each. The structure of discussion thread was similar to that in the first NAS group. The difference was that each discussion thread included messages contributed by all three learners in this group instead of two (in the first NAS group). But it should be mentioned that the two end-up messages (the message titled ‘Rückantwort’ from Pinie and the message titled ‘Rückantwort ahorn’ from Birke) were no more than short yes-messages (‘genau lass deine analyse, war gut!’ and‘ ich finde du sollst auch deine analyse so lasen’). Therefore, the interactive discourses actually took place between two peers within each discussion thread. On average there were 34 words in each message during discussion. Two out of the six messages during discussion were prompted.

Figure 5.25 showed how discussions proceed in the third NAS group.

In the third NAS group, eight messages were posted altogether, four from Ahorn, two from Birke and two from Pinie. There were two discussion threads; three messages within the first while five within the second. The structure of discussion threads was also similar to the first two NAS groups. The difference was that Birke contributed his second message to the second thread instead of the first one, which was the case in the first NAS group. It made the second thread more interactive in a way that all three learners engaged in it. All messages during discussion were prompted. On average there were 60 words in each message during discussion.

Discourses in these three NAS groups suggested that the non-adaptable script introduced a consistent structure to online discussions. One of the three learners played the role of analyst while the other two acted as critics. Interactions took place mainly between analyst and critic, but rarely between critics. There were two discussion threads in each NAS group, with four messages each, or one with three the other with five. Most of the messages were prompted (78%).

5.4.4.3 Discussion Threads in the AS Condition

In the following, discourses of a learning group facilitated by an adaptable script will be analyzed and presented. Ahorn, Birke, and Pinie were all 20 year old females in this group. Ahorn and Pinie were from educational science and at their 2nd semester. Birke was from special education, at her 2nd semester as well. Figure 5.26 showed there were one main discussion thread and a following final analysis.

As Fig. 5.26 showed, there were seven messages posted on case ‘Math’. Three out of the seven messages were from Ahorn, two from Birke, two from Pinie.

Within these seven messages, there was one main discussion thread with six messages (Fig. 5.27).

Discussions started with the first analysis of the problem case posted by Ahorn. There were 78 words in this message (Fig. 5.28).

Pinie developed this discussion thread by responding to the first message. There were 80 words in Pinie’s message. Pinie did not use any of the interaction-oriented prompt (Fig. 5.29).

Birke further developed this discussion thread by answering Pinie’s critique. There were 119 words in this message. This message was not prompted (Fig. 5.30).

Ahorn extended the discussion thread by answering the two comments from Pinie and Birke together. There were 117 words in this answer-message. It was not prompted (Fig. 5.31).

Pinie’s second message in this discussion thread was a quick-consensus building. There were thirteen words in the message (Fig. 5.32).

The sixth message was Birke’s response to Pinie. There were one word and a smiling face in this message (Fig. 5.33).

The seventh message was the last one on this case. It was the final case analysis posted by Ahorn. There were 121 words in this message.

On average there were 66 words in each message during discussion. Ahorn in this AS group played the role of analyst while Birke and Pinie played the role of critic. But the activities attached to these roles were adapted or flexible in a sense that for example Birke (the critic) answered to Pinie’s (the other critic) response and Ahorn’s (analyst) initial analysis at the same time (the message titled ‘kritik’), rather than answered only to Ahorn’s first analysis, as prescribed in the script. So did Ahorn and Pinie. It made the discussion thread so condense that it engaged all three learners in one main discussion thread instead of two, as in NAS groups. None of the posted message was prompted.

Figure 5.34 showed how discussions proceed in the second AS group.

In the second AS group, eight messages were posted altogether. There was one main discussion thread with seven messages. This discussion thread was started by Ahorn. Birke and Pinie commented, and then Ahorn responded. This cycle repeated one more time. In addition, there was a response message (which was a quick agreement) from Birke (the message titled ‘bin durch’) got no further response. On average there were 34 words in each message during discussion. None of these messages was prompted.

Figure 5.35 showed how discussions proceed in the third AS group.

In the third AS group, nine messages were posted altogether. There were two discussion threads, one with six messages and the other with three. The first discussion thread was so developed that Ahorn posted the first analysis, Birke provided feedback and Ahorn responded and clarified the unclear points. Response from Ahorn was followed by comments from both Pinie and Birke. The second discussion thread started also with the first analysis from Ahorn. Due to technical problem the response from Pinie was not saved (the message titled ‘Kein Titel’ posted by Pinie at 14:38 was an empty message). This thread ended up with a question from Ahorn which was about the technique failure (the message titled ‘Kritik???’). On average there were 43 words in each message during discussion (with the empty message excluded). Five messages were prompted.

When online discussion was supported by an adaptable script, there was also a consistent structure across groups, which was, however, obviously different from that when learning with a non-adaptable script. In adaptable script groups, one main discussion thread developed, which was long (with six to seven messages) and with contributions and interactions from all three learners within the group. In one of the three selected groups most messages were prompted (the third AS group) while in the other two groups (the first and the second), no prompt was used at all. On average, 28% messages were prompted when learning with an adaptable script.

Summary of Results

The adaptable script enhanced the quality of cognitive processes relative to NS condition;
Compared to learning without script, both a non-adaptable and an adaptable script inhibited learners’ application of internal collaboration scripts to support cognitive processes;
Internal collaboration scripts contributed positively to social processes of conflict—oriented consensus building only when learning with an adaptable script;
Case studies showed that collaboration scripts had effects on the pattern of discussion threads.

5.5 Discussion and Conclusions

How to implement flexible scripting to maximize the effectiveness of CSCL has recently drawn more and more attention in related research (Dillenbourg and Tchounikine 2007; Diziol et al. 2010; Gweon et al. 2006). In this study, the question whether adding adaptability to the scripting approach is an effective way to reduce the coercion of collaboration scripts without losing the benefit from the process structuring was explored.

In this section, results presented in Sect. 5.4 will be interpreted on grounds of the theoretical background and previous findings. Some limitations will be discussed to avoid overgeneralization. Implications, both theoretical and practical, will be discussed as well. At the end of this section, conclusions will be drawn.

5.5.1 Effects of Adaptable Script on Cognitive Processes

It was found in the reported study that a non-adaptable collaboration script had no effect on cognitive processes of collaborative knowledge construction relative to unscripted collaboration. As the peer-review script applied in the reported study was by nature a communication-oriented script (Schellens et al. 2007) which targeted at stimulating productive collaboration rather than provoking content elaboration, it was not surprising that this script had no effect on cognitive processes (Weinberger and Fischer 2006).

According to Dillenbourg (1999), a collaboration script of a high degree of coercion might hinder cognitive processes when it segments collaboration into too many sub-processes, especially for learners with high prior knowledge, for the reason that a fine-grained script may conflict with the well chunked prior knowledge and therefore, cause unnecessary work load.

The cognitive over-scripting explanation was supported by the evidence that when the script was adaptable, quality of cognitive processes was enhanced. When learning by aid of an adaptable script the restriction on cognitive processes caused by the relatively high degree of coercion provided by a fixed script was avoided (Fischer et al., in press). When learning with an adaptable script, the opportunity was provided to learners to get rid of parts of the script that were subjectively regarded as unnecessary, for example, by switching off the prompts.

Furthermore, to adapt the external script, students were required to reflect on their individual and collaborative performance and to monitor the learning processes accordingly (Scheiter and Gerjets 2007). These regulatory processes might to some extent influence the effectiveness of collaborative learning in CSCL environments (Zimmerman and Tsikalas 2005), for the reason that learning about complex and challenging topics in a computer-based environment requires students to make necessary adjustments regarding their background and the learning context (Azevedo 2007).

In addition to the main effect, adaptable script interacted with learners’ internal collaboration scripts on cognitive processes. The interaction effect could be explained by the internal script configuration principle and the external script guidance principle (see Chap. 2; Fischer et al., in press), which state that learners’ dynamic configuration of internal collaboration scripts might be influenced by an externally induced collaboration script. In the current work, both a non-adaptable script and an adaptable script inhibited learners’ employment of their internal collaboration scripts to support cognitive processes of collaborative knowledge construction, compared to unscripted collaboration. However, as learners’ internal collaboration scripts could be dysfunctional or functional, it is not necessary that the suppression of an external collaboration script on the employment of internal collaboration scripts would bring about negative effects on learning processes and outcomes (Fischer et al., in press). As discussed above, despite its constraints on learners’ application of internal collaboration scripts, the adaptable script enhanced the overall quality of cognitive processes, compared to unscripted collaboration.

5.5.2 Effects of Adaptable Script on Social Processes

Although the evidence for a positive effect of the scripting approach on social processes is ample (Stegmann et al. 2007; Vogel et al., accepted), it was found that the non-adaptable script applied in the reported study had no significant effect on social processes of collaborative knowledge construction relative to unscripted collaboration, neither did the adaptable script.

As the same script was found to have positive effects on social processes of collaborative knowledge construction in a previous study (Weinberger et al. 2005), we had in the current study a so-called missing effect. A possible explanation of the missing effect is due to the training phase, which did not occur in the previous study. The training phase might lead to some extent internalization of the script so that learners in the unscripted condition would still implicitly use parts of the script during their collaborative learning. Evidence for the internalization assumption is supported by findings from qualitative analyses reported in Sect. 5.4.4, where the ‘trained scripted’ group structured their online discussion exactly the way introduced by the external collaboration script, although the external script was not available to them during their collaborative learning. But of course there was reason to include such a training phase in the current work. As we argued, the training phase was necessary for realizing adaptability, since without knowing about the mechanism of the collaboration script, adequately adapting it would be impossible.

The optimal external scripting level principle claims that an external script should provide a high or low scripting level based on learners’ internal collaboration scripts (Fischer et al., in press). Learners’ high structured internal collaboration scripts might be inhibited from being applied when learning with a coercive script (over-scripting; Fischer et al., in press). External scripts provided at an optimal level would contra wise induce or trigger students’ employment of appropriate internal script components (Fischer et al., in press).

The interaction between adaptable script and students’ internal collaboration scripts supports the optimal scripting argument. When learning with an adaptable script, internal collaboration script components that were functional for social processes were triggered rather than inhibited (which was the case in the non-adaptable script condition) and therefore, high structured internal collaboration scripts (with appropriate components accessible) contributed positively to social processes of conflict-oriented consensus building. When learning without script, internal collaboration scripts had no effect on social processes probably because there was no affordance from the social context to induce students to apply the appropriate internal script components. In other words, no script was not of an optimal scripting level as there was a lack of affordance (Fischer et al., in press).

Results of the interactions between internal and external collaboration scripts on social and cognitive processes revealed that the adaptable script triggered learners’ employment of internal collaboration scripts components that are helpful for interactive processes but impeded their application of internal collaboration scripts components that are dysfunctional for cognitive elaboration (Fischer et al., in press). Through the requirement for adaptation, the adaptable script shifted learners’ efforts and their use of internal collaboration scripts away from cognitive aspects toward social aspects. However, the support provided by the adaptable script itself compensated this side-effect, as the adaptable script enhanced the overall quality of cognitive processes for all learners, regardless of their internal collaboration scripts.

5.5.3 Effects of Collaboration Script on the Pattern of Discussion Threads

As reported in Sect. 5.4.4 findings from case studies suggested that collaboration scripts not only influenced the quantity of discourses, but also had effect on the pattern of discussions threads.

When learning without script, there was no consistent pattern of discussion threads across groups. Students in the unscripted condition could be seen as guided by their high or low structured internal collaboration scripts (Fischer et al., in press). For example, the ‘trained script’ group was guided by the internalized collaboration script through their collaborative learning phase.

The non-adaptable script, on the other hand, introduced a consistent structure to online discussions across groups. The structure that the non-adaptable script posed to online discussions is exactly the one predefined by the script. Students in the fixed script condition were guided by the external script provided to them. Although the structure shaped by the external script was to some extent artificial in a way that interactions took place only between analyst and critic, rarely between critics, students did adhere to it.

The adaptable script introduced an obviously different structure to online discussions. When learning with an adaptable script, learners developed a long discussion thread rather than two small ones as did students in the non-adaptable script condition. Moreover, interactions in adaptable script condition took place among all learning partners other than between analyst and critic only, as did students in the non-adaptable script condition. Long discussion threads, in which there are many exchanges of ideas, often characterize sustained online discussions for productive collaboration to occur (Hewitt 2005; Palincsar 1999). It suggested that the adaptable script allowed to a certain extent learners’ employment of internal collaboration scripts (Fischer et al., in press) which made their discussions more interactive and more effective.

Fewer messages (28%) were prompted when learning with the adaptable script compared to the non-adaptable script condition (78%), but the quality of collaborative knowledge construction remains the same, if not better (seeing from the quantitative analyses). It indicated firstly that the adaptable script was all and all more effective than was the non-adaptable script to structure collaborative learning and secondly, learners were capable of purposely fade out unnecessary parts of the scripts without losing the benefit from process structuring.

However, due to the exploratory nature of the case studies and the lack of statistic analysis, findings from qualitative analysis only provided information additional to the quantitative process analysis. Explanation of the pattern of discussion threads identified when learning with a specific type of collaboration script requires further systematic and theory-based investigation.

5.5.4 Limitations and Implications

This section discusses the limitations of the reported study before discussing the theoretical and practical implications of the reported findings.

5.5.4.1 Limitations

The present work had some limitations. Firstly, the peer-review script was implemented in a short term directing toward immediate effects on collaborative knowledge construction processes. A more long-term intervention and a delayed post-test would be helpful in understanding the long-term effects of collaboration scripts on collaborative knowledge construction and the transferability of the ‘adaptation’ skills.

Secondly, the measurement of internal collaboration scripts, which was an open question measuring declarative knowledge rather than a performance test that measures the applicable internal collaboration scripts, might not be optimal, since declarative knowledge may not be the best predictor of the internal collaboration scripts that learners actually applied during collaborative learning.

Thirdly, the current work is by some means an exploratory one on adaptable scripting in CSCL. Although the effects of an adaptable script on collaborative knowledge construction processes were compared to a non-adaptable script and unscripted collaboration, degree of adaptability has not been manipulated. Further studies that vary the degree of adaptability systematically would contribute to answer the question how to design CSCL scripts to provide the optimal scripting level based on individualized needs (Fischer et al., in press).

Furthermore, operationalization of the non-adaptable script and the adaptable script left the possibility that the difference between these two conditions was ignorable. Learners were not forced to use the interaction-oriented prompts when learning with the non-adaptable script (see Sect. 5.4.4, 78% messages were prompted), so that it was also possible for them to ‘switch off’ the prompts as was it for learners in the adaptable script condition. The difference could have been enlarged by forcing learners to use every single prompt when learning with the non-adaptable script. But our goal was to investigate whether adding adaptability to a script that has been used in practice and proved to be effective would bring about further benefits for scripted collaboration other than to purposely vary the degree of coercion. Future studies that compare adaptable script with a more coercive non-adaptable script than the peer-review script used in the current study would help generalize the effect of adaptability to a more coercive setting or limit it to a setting with a medium degree of coercion, such as the peer-review script in our study.

A further limitation was that the sample size of the reported study was not large (87 students in 29 triads). Although it was acceptable in CSCL research, for example, the study from Demetriadis et al. (2011) had a sample size of 63 (nine dyads and 15 triads), it was not large to perform multilevel modeling, which requires a large sample at both individual (e.g., 10 individuals in each group) and at group level (e.g., 50 groups), statistically (Hox 2010). Moreover, case-based asynchronous online discussion was selected as the learning scenario in our study; it was still unclear whether adding adaptability to scripted collaboration would have the same effects in other CSCL environments, such as synchronous discussion. Therefore, the interpretation of the reported findings should not be over-generalized.

5.5.4.2 Implications

Despite of the aforementioned limitations, the findings of the current study indicated that an adaptable script is a promising approach to realizing flexibility in order to maximize the effectiveness of collaboration scripts in CSCL.

Efforts have been put into realizing flexible scripting in CSCL through adaptivity, for example the use of intelligent tutoring (Diziol et al. 2010) and natural language processing technology (Mu et al., in press). Although the concept ‘adaptability’ (Leutner 2009) or ‘learner control’ (Scheiter and Gerjets 2007) has drawn some attention recently in the field of learning with hypermedia, adaptable scripting is quite new a topic in the field of CSCL research. The reported study was rather an exploratory one on adaptable scripting in CSCL. Theories and empirical evidences for the possible advantages of an adaptable script over a non-adaptable one and unscripted collaboration were mainly from other research areas, such as learning with hypermedia (Scheiter and Gerjets 2007). However, results of the reported study were encouraging. The study yielded that it was possible to facilitate collaborative knowledge construction processes in CSCL with an adaptable script. An adaptable script could immediately support students to construct relations between conceptual and problem space (cognitive processes).

The study also showed that an adaptable script influenced students’ configuration of internal collaboration scripts. The internal script configuration principle claims that learners’ dynamic configuration of internal collaboration script components is influenced by their perceived situational characteristics (Fischer et al., in press). The external script guidance principle states that external collaboration scripts guide learners in collaborative learning situations by inhibiting their automated use or by inducing their application of internal script components (Fischer et al., in press). Following these two principles in the Script Theory of Guidance (Fischer et al., in press), the adaptable script applied in the reported study influenced students’ configuration of internal collaboration script components in a way that it inhibited students’ employment of internal script components dysfunctional for cognitive processes but induced their application of internal script components beneficial for social processes. A non-adaptable script, on the other hand, inhibited students’ application of internal collaboration script components for both cognitive and social processes. The results suggested that adaptable scripts should be carefully designed in order to induce students’ application of different internal collaboration script components to fulfil specific instructional goals.

Given that the adaptable script was more effective than the non-adaptable script and unscripted collaboration on collaborative learning processes, the adaptable scripting approach is a practical example of realizing flexibility in scripted CSCL, at least for fostering cognitive processes and students’ application of internal scripts for social processes of collaborative knowledge construction. Design of adaptable learning environments or adaptable instructional supports would be of success in other educational practices outside CSCL, for example, formal classrooms, since a learning environment or an instructional approach can be adaptable not only to students but also to teachers (Leutner 2009).

5.5.5 Conclusions

How to make the scripting approach more flexible is an increasingly interesting topic in CSCL (Dillenbourg and Tchounikine 2007; Diziol et al. 2010; Fischer et al., in press).

In the present study, an adaptable script, which means students can adjust the external script based on their perceived needs (Leutner 2009), was implemented to realize flexible scripting in CSCL. Results of the study revealed that an adaptable script was overall advantageous over a non-adaptable script and unscripted collaboration with respect to collaborative knowledge construction processes in case-based asynchronous online discussions. However, to fulfil specific instructional goals, adaptable scripts should be carefully designed because that different internal collaboration script components might be inhibited or induced by an adaptable script (Fischer et al. 2013).

Author information

Authors and Affiliations

Beijing Normal University, Beijing, China
Xinghua Wang
Department of Statistics, Ludwig Maximilian University of Munich, Munich, Germany
Jin Mu

Authors

Xinghua Wang
View author publications
You can also search for this author in PubMed Google Scholar
Jin Mu
View author publications
You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

About this chapter

Cite this chapter

Wang, X., Mu, J. (2017). An Empirical Study on Adaptable Scripting. In: Flexible Scripting to Facilitate Knowledge Construction in Computer-supported Collaborative Learning. Perspectives on Rethinking and Reforming Education. Springer, Singapore. https://doi.org/10.1007/978-981-10-4020-7_5

Download citation

DOI: https://doi.org/10.1007/978-981-10-4020-7_5
Published: 28 April 2017
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-4019-1
Online ISBN: 978-981-10-4020-7
eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

An Empirical Study on Adaptable Scripting

Abstract

Keywords

5.1 Learning Environment

5.1.1 Asynchronous Online Discussions

5.1.2 Learning with Complex Problem Cases

5.1.3 Desirable Collaborative Knowledge Construction Processes

5.1.3.1 Cognitive Processes

5.1.3.2 Social Processes

Summary

5.2 Research Questions and Hypotheses

5.3 Methods

5.3.1 Design and Sample

5.3.2 Learning Material and CSCL Environment

5.3.2.1 Theoretical Texts and Problem Cases

5.3.2.2 CSCL Environment: CASSIS

5.3.3 Procedure

5.3.3.1 Individual Learning

5.3.3.2 Training

5.3.3.3 Online Chat

5.3.3.4 Collaborative Learning

5.3.4 Experimental Conditions

5.3.4.1 CSCL Without Script

5.3.4.2 CSCL with a Non-adaptable Script

5.3.4.3 CSCL with an Adaptable Script

5.3.5 Operationalization of Dependent Variables

5.3.5.1 Measure of Cognitive Processes

5.3.5.2 Measure of Social Processes

5.3.6 Control Variables

5.3.6.1 Domain-Specific Prior Knowledge

5.3.6.2 Internal Collaboration Scripts

5.3.6.3 Initial Intrinsic Motivation

5.3.7 Statistical Analyses

5.3.8 Qualitative Approach

5.4 Results

5.4.1 Preliminary Analyses

5.4.2 Effects of Adaptable Script on Cognitive Processes (RQ1)

5.4.3 Effects of Adaptable Script on Social Processes (RQ2)

5.4.4 Effects of Collaboration Script on the Pattern of Discussion Threads (RQ3)

5.4.4.1 Discussion Threads in the NS Condition

5.4.4.2 Discussion Threads in the NAS Condition

5.4.4.3 Discussion Threads in the AS Condition

Summary of Results

5.5 Discussion and Conclusions

5.5.1 Effects of Adaptable Script on Cognitive Processes

5.5.2 Effects of Adaptable Script on Social Processes

5.5.3 Effects of Collaboration Script on the Pattern of Discussion Threads

5.5.4 Limitations and Implications

5.5.4.1 Limitations

5.5.4.2 Implications

5.5.5 Conclusions

Author information

Authors and Affiliations

Rights and permissions

Copyright information

About this chapter

Cite this chapter

Download citation

Share this chapter

Publish with us

Search

Navigation