Introduction

One of the essential components of twenty-first century learning is collaborative learning (Häkkinen et al. 2017). Past studies have substantiated many benefits of collaborative learning (Dillenbourg 1999; Järvelä et al. 2016). From the cognitive perspective (Dillenbourg 1999; Ludvigsen et al. 2016), collaborative learning facilitates better mastery of content knowledge (Wen et al. 2015; Lin et al. 2014), stimulates the development of higher-order thinking skills (Yulianto et al. 2019), and promotes greater academic achievement and positive attitudes toward learning (Chen and Looi 2011; Stahl 2016). From the social perspective, collaborative learning promotes an active learning process (Järvelä et al. 2016) with improved learning through peer interactions (Dillenbourg 1999). CSCL creates an opportunity for the sharing and construction of knowledge among participants (Popov et al. 2017).

Effective CSCL requires students to discuss and solve problems, activate joint strategies by interacting with their peers, reflect on their differences in perspectives, and/or modify their respective initial points-of-view (Järvelä et al. 2016). However, effective CSCL is rarely a spontaneous phenomenon (Nussbaum et al. 2009) as students often fail to interact productively in groups (Chen et al. 2010; Järvelä et al. 2016). The social interaction among group members does not automatically take place just because a CSCL environment is created (Kreijns et al. 2003). Successful collaboration to foster knowledge construction can also be constrained by a dearth of high-level collaborative processes of meaning negotiation and a narrowing of distinctions between their expressed opinions (Popov et al. 2013).

In an authentic classroom with many groups of students working together during a lesson period (e.g., one hour), collaborative learning becomes more complex and challenging. Students rarely achieve a deepening of knowledge in class-wide collaboration practices (Jeong et al. 2019). Issues such as free-riding and unequal level of collaborative effort persist (Hämäläinen and Arvaja 2009). Adopting a divide-and-conquer approach is common (Barron 2003). When individuals and groups engage in dysfunctional social processes, it can be stressful for the teacher and cause distractions to the rest of the class (Borge and Mercier 2019). Other issues related to classroom collaboration include silent participants (Persico et al. 2010), information overload (Morgil et al. 2004), and time management difficulties (Dillenbourg 2016). The depth of understanding from generating and comparing alternative solutions and resolving conflicts is thereby lost (Barron 2003), leading to unproductive CSCL processes and outcomes. Positive inter-dependence (Prins et al. 2005) is thus important for the purpose of enabling students to work and coordinate with others towards knowledge improvement.

Therefore, how to leverage the collective intelligence of the groups and the whole class in the design of meaningful CSCL environments and experiences is a pertinent issue. Traditional pedagogical approaches with comparatively fixed goals and routines cannot be applied directly within a CSCL networked classroom (Wen et al. 2012). CSCL researchers have proposed various pedagogical and instructional approaches, strategies and scripts. Dillenbourg (2013) have advocated for classroom orchestration, which refers to the manner in which a teacher manages in real-time, multi-layered activities under multiple constraints. These multi-layered activities could be at the individual level (e.g., reading), group level (e.g. problem-solving) or class-wide level (e.g. lectures). Dillenbourg (2013) stated that managing these multi-layered collaborative learning activities under multiple constraints is not easy. How to address the specific challenges of these multi-layered collaborative learning activities in a networked classroom is the focus of this study.

This study presents a pedagogical model named the Spiral Model of Collaborative Knowledge Improvement (SMCKI). This model is meant to support the multi-layered CSCL process and collaborative knowledge improvement in an authentic networked classroom. The model was informed by the “low floor, high ceiling and wide walls” (Resnick et al. 2009) for learning task design. In a networked classroom, the knowledge generation, sharing and improvement also have these characteristics (low floor – everybody can generate ideas easily; wide walls – easy sharing of ideas enables a large amount of knowledge to be shared; high ceiling- participants can work on increasingly sophisticated ideas and knowledge).

This study aims to examine how participants collaboratively improve their knowledge throughout the various phases of the model. Specifically, the context of this study was on pre-service teacher education, where participants were tasked with designing technology-enhanced learning (TEL) lessons collaboratively. TEL design is a complex task. Designing TEL by integrating technology, pedagogy, and content (TPACK) is not easy. Collaborative knowledge improvement is suitable for TEL design. These complex tasks create a great diversity of outcomes (Reeves et al. 2004) and can benefit from a collaboration-based approach (Zambrano et al. 2019) By engaging participants in the SMCKI phases, we can examine how this pedagogical model supports participants in the complex collaborative lesson design process. Through unpacking the multi-layered CSCL process, we attempt to respond to the concern highlighted by Borge and Mercier (2019) regarding “how we use learning theories to understand complex collaborative systems and prioritize the use of theory for helping to make these processes more accessible to teachers and facilitators” (Borge and Mercier 2019, p. 11).

Literature review

Orchestrating CSCL can be difficult, and even expert teachers face problems (Rodríguez-Triana et al. 2015). Optimizing these process both at the small group level and the class level is challenging. Changing collaborative processes within a CSCL setting is more complicated than changing individual processes (Jeong et al. 2019). To support teachers and students in this endeavor, different pedagogical models and strategies have been developed in CSCL. These collaborative pedagogical models and strategies from past research studies include the Communicative model of collaborative learning, which emphasizes meaning-making, knowledge sharing and co-creation of knowledge during the collaborative learning process (Cecez-Kecmanovic and Webb 2000); the Online interaction learning model, which focuses on computer-mediated communication based on an input-process-output model with feedback loops (Benbunan-Fich et al. 2005); and the Funnel Model (Chen et al. 2011), which encourages cognitive diversity for the synergy of ideas, and consensus-seeking that leads to knowledge advancement of individuals, groups and the class as a whole. These models seek to enhance the process of collaborative learning and knowledge advancement. Typical collaborative learning processes commence with the group or end at the group or class level. Commencing with an individual phase to capitalize on individual prior knowledge or integrating a class-wide collaborative learning activity between groups is uncommon.

While CSCL research often aims at the learning of individuals and sometimes at learning of groups as a whole, studies discussing the learning and context of greater units like the classes the collaborating groups are part of are still rare (Kirschner and Erkens 2013). In addition, few studies have been concerned with the manner in which groups collaborate with other groups in a class. How to design and implement effective collaborative learning in a multi-layered authentic networked classroom that leverages the interplay of individual ideation, within-group collaboration, and between-group collaboration is underexplored, and even more so when the collaborative learning seeks shared-meaning making in order to achieve knowledge improvement.

Past research studies have claimed that collaboration scripts support collaborative knowledge improvement (Kollar et al. 2006; Stegmann et al. 2007). Collaboration scripts help participants engage in smooth collaboration processes and reach specific task or learning objectives as they are goal-oriented and afford specific practices (Kollar et al. 2006). When individuals interact as suggested by the collaboration script, they acquire more knowledge from collaborative learning tasks than those who lack that guidance and support in their CSCL environment (Weinberger et al. 2005).

There are two aspects to collaboration scripts, the external versus internal script dimension, and the macro versus micro script dimension. Based on Kollar et al. (2007), a script is deemed as external when it relates to the external orientation of the CSCL situation rather than the participants’ cognitive systems. Internal scripts, on the other hand, refer to learners’ internalized collaboration skills (Fischer et al. 2013). Dillenbourg and Hong (2008) refer to scripts as adopting one of two approaches, the macro and micro script approach, which share the same goal but operate a different levels of granularity in their implementation. The macro-script seeks to sequence different learning activities or phases such as organizing the interlacement of classroom, individual and collaborative learning phases (Näykki et al. 2017). On the other hand, the micro-script refers to the finer-grained scaffolding that follows a more psychological approach and emphasizes the activities of individual learners within those activities and phases (Dillenbourg and Hong 2008). The macro-script is tasked with supporting the teacher in implementing CSCL scenarios within the classroom, orchestrating individual and collaborative learning phases (Näykki et al. 2017). According to Dillenbourg and Hong (2008), a macro-script is also described as a pedagogical model. Adopting a pedagogical model as a macro-script helps to promote interaction and cognitive elaboration, and to foster knowledge construction (Bouta et al. 2012).

Despite the benefits of scripting on collaborative learning, the literature has warned of the dangers of over-scripting, which are believed to reduce learner autonomy and motivation and to hinder their knowledge acquisition (Dillenbourg 2002; Fischer et al. 2013). On the contrary, the meta-analysis conducted by Vogel et al. (2017) indicated that “on average, CSCL scripts have a small but statistically significant positive effect on learners’ domain-specific knowledge” (p. 496). The effect of scripts was higher on collaboration while the effect on domain learning was less clear (Radkowitsch et al. 2020). To address this issue, the design of a CSCL script needs to help students reach high levels of knowledge and skill acquisition (Vogel et al. 2017). Since leveraging the positive effects of collaboration scripts is a promising way to facilitate an effective CSCL scenario (Popov et al. 2013), we propose a pedagogical model, which can also be accorded as a macro-script, named the Spiral Model of Collaborative Knowledge Improvement (SMCKI) to scaffold and structure the collaborative learning process for knowledge improvement at the individual, the group and the class levels in an authentic networked classroom.

The SMCKI is designed as a learning framework that offers a low floor, high ceiling, and wide walls (Gadanidis 2014). The concept of low floor and high ceiling was first proposed by Papert (1993). Resnick (2017) reckoned that just a single route from low floor to high ceiling would not be sufficient. There is a need to widen walls, as this can provide many paths and many styles for making powerful ideas salient (Shneiderman et al. 2006). With widening walls, sharing and communication of learning increases (English 2017). Leveraging this as the design principle, the low floor to the high ceiling of the individual path in SMCKI begins with individual ideation and reaches the high ceiling with individual achievement. At the group level, the low ceiling commences with intra-group synergy and arrives at its high ceiling at intra-group refinement. To motivate participants during the collaborative process, the intra-group synergy phase requires participants to produce a group artifact that represents the highest quality of the group’s knowledge to offer for class-wide critique that will take place. The wide walls in SMCKI are the class-wide inter-group critiques, which then provide an avenue for the individual and group level low floors to reach the high ceilings. The explicit presentation of the groups’ respective artifacts to members of other groups to critique helps motivate each group to present the best quality design they have to offer.

In this study, we present an exploratory study illustrating how SMCKI was used in a face-to-face (F2F) CSCL environment where a class of Singapore pre-service teachers’ collaboratively design technology-enhanced learning (TEL) activities. We examine how the pre-service teachers improve their TEL designs by leveraging individual ideation, group collaboration and class-wide peer-critique in the CSCL environment and how collaborative knowledge improvement takes place. This conception of idea improvement over a short duration of class time is drawn from the research on Rapid Collaborative Knowledge Improvement (RCKI) (Looi et al. 2010a, 2011). RCKI refers to the notion of democratizing participation and idea refinement in the context of live dynamic classroom settings, that is face-to-face (F2F) collaborative knowledge construction and improvement throughout a class session, supported by certain technologies for lightweight instant interaction (Looi et al. 2010b; Ng et al. 2008). The development of SMCKI is informed through the RCKI process.

The Pedagogical Model: The Spiral Model of Collaborative Knowledge Improvement (SMCKI).

The Spiral Model of Collaborative Knowledge Improvement (SMCKI) is a five-phase pedagogical model to support the collaborative learning of a networked classroom. It focuses on democratic knowledge sharing as well as the individual, group and class level knowledge improvement processes (see Fig. 1).

Fig. 1
figure 1

A spiral model of collaborative knowledge improvement

Full size image

SMCKI commences with a phase of individual ideation, followed by a phase of intra-group idea synergy. The third phase then maximizes the benefit of class-level participation through inter-group critique. Comments from peers at this phase stimulate phase four intra-group idea refinement, heightening collaborative knowledge improvement. Phase five concludes the collaborative learning activity by returning to engagement of cognitive processes culminating with the individual achievement phase. SMCKI aims to 1) facilitate students’ social and cognitive development by maximizing idea generation, sharing, and improvement, and 2) address collaborative learning challenges and issues at both the group level (e.g., free-riding, silent participant) and the class level (e.g., learning from and with other groups within a short period of time).

What makes SMCKI unique lies in two specific phases: the first phase focused on individual ideation and the third phase focused on inter-group critique. To the best of our knowledge, SMCKI is the only pedagogical model that includes an individual ideation component within a collaborative learning model. It is designed with an intentional individual ideation phase prior to group-level activities as this facilitates the processing and recall of prior knowledge. For situating the individual phase within a collaborative learning environment, not only does it promote higher levels of participation (Isohätälä et al. 2017), the presence of free-riders and unequal level of collaboration can be discouraged. In addition, the individual experience encourages more ideas and thus creating a wider pool of ideas to benefit the group synergy phase. The next characteristic of SMCKI is phase three, namely, inter-group critique. Peer-assessment and feedback is not a new area. Past research studies have advocated that peer critique enhances students’ sense of judgment, objectivity and eventually develops autonomous thinking (Akanmu 2016; Yuan and Kim 2018). Critically reviewing one another’s work and providing constructive criticism accelerates knowledge improvement (Looi et al. 2010a; Wen et al. 2012). Peer-critique, assessment, and feedback have been widely adopted in view of their benefits from both the receiving end and the critique end (Prins et al. 2005). In a collaborative learning environment, peer-critique in the form of social interaction enhances metacognition (Frith 2012), which brings about knowledge improvement. Therefore, having a peer-critique phase should not be taken for granted (Kreijns et al. 2003). In SMCKI, peer-critique is provided at a class-wide level at phase three for active and simultaneous interactions. Knowledge generated at this phase will add value to the knowledge refinement in phase four.

In summary, by respecting and encouraging cognitive diversity, SMCKI phase one encourages the generation of diverse ideas for phase two to seek synergy through consensus-seeking, leading to knowledge improvement within the group. Phase three promotes class-wide knowledge contribution giving the necessary ingredients for phase four refinement. A final recollection during phase five individual achievement allows the consolidation of knowledge improvement spiraling through from the individual to the group, to the class, and back to the group and the individual. Therefore SMCKI is a spiraled spindle-shaped model where 1) the largest number of ideas generated is at the middle phase (phase three), 2) the knowledge build-up beginning from individual prior knowledge contribution and completes at individual achievement and 3) the quality of ideas grows higher and higher when the collaborative learning moves to a later phase, forming a continual process of knowledge improvement. SMCKI operates as a scaffold for CSCL in a networked classroom, bringing about the transformation of pedagogical practice. To investigate the applicability and effectiveness of SMCKI in a networked classroom, the present study is motivated by two research questions:

  1. 1.

    Does SMCKI support pre-service teachers’ collaborative knowledge improvement on TEL design?

  2. 2.

    How do the pre-service teachers improve their knowledge of TEL design (as demonstrated by artifacts generated) over the five SMCKI phases?

Methodology

This study adopts an exploratory case study design because it is one of the first studies examining the design and implementation of SMCKI in an authentic classroom. Therefore, a case study methodology was appropriate to explore a concept for which there has been little or no previous research in a given context (Yin 1992). The case chosen is a lesson where a class of pre-service teachers in Singapore collaboratively design TEL activities. Employing collaborative learning in design is not new. Mishra and Koehler (2003) reckoned that in a collaborative lesson design environment, pre-service teachers can be given the opportunity to explore their ideas among peers with ease. Through this process, they can uncover the inadequacies of their current teaching theories and methods (Chitpin and Evers 2012).

Participants and research context

A class of 19 pre-service teachers enrolled in the course “The use of ICT in Chinese Language and Chinese Literature” at the National Institute of Education, Nanyang Technological University Singapore, participated in this study. The gender composition was 79% female and 21% male participants. 42% of the participants were aged between 20 and 25 years old, 37% were between 26 and 30 years old, and the rest were above 30 years old. 84% of them possessed a Bachelor degree, and 16% had a Master’s degree. The course lecturer had four years of teaching experiences in this area. The participants knew each other and had prior group learning experiences but not facilitated by the pedagogical model under investigation in a classroom setting.

A total of six lessons on collaborative TEL lesson design were conducted. The course’s past collaborative TEL lesson design activities were set in a laissez-faire mode. The participants were left to manage their group work, and issues with silent participants and unequal level of participation in the collaboration were common. The benefits of collaborative learning were hardly reached. Knowledge sharing among participants and knowledge acquisition was compromised as a result. It was almost impossible to identify the source of shared knowledge within groups. To address this issue, this study employed the SMCKI model throughout the collaborative TEL lesson design task process in class. How each SMCKI phase helped with knowledge sharing and knowledge improvement from one phase to the next can be rigorously examined. Investigating the process and the outcome of the knowledge improvement with TPACK was the intended outcome of the study.

SMCKI was used in all the six lessons with the same design and implementation. We randomly chose one out of the six lessons as the case for the data analysis of the present study. This was the fifth lesson. In this lesson, the participants were randomly grouped into five groups, with four groups of four members and one group of three members for the CSCL task. It happened that the groupings were new such that participants had the opportunity to work with people they had not worked with previously. The class was a networked class where all participants were connected with computers with Internet access (see Fig. 2).

Fig. 2
figure 2

Grouping in the networked classroom

Full size image

Research intervention

The duration of the lesson was 80 min. The lecturer commenced with 15 min of task requirement briefing and 10 min of technical setup. In between each SMCKI phase was five minutes of instruction by the lecturer. The participants were tasked with collaboratively designing one technology-enhanced learning (TEL) activity for Chinese language teaching and learning for secondary schools based on a selected text in Singapore’s Chinese Language curriculum. The online collaborative platform used in this study was Google Slides, which well supports learning activities in the networked classroom (e.g., sharing of a common document, a simultaneous editing function, extensive multi-version support such that users can easily revert to any previous version of their document). Furthermore, it possesses the comment feature, which is vital to the within and between-group peer assessment and feedback processes. The feature of adding, deleting and editing slides supports the revision and improvement of TEL design well. In addition, Borge and Mercier (2019) stated that if a technology is too difficult to use, it might frustrate the user, and therefore increase negativity experienced within the team, as well as interfering with task completion. Since Google Slides is a familiar platform for the participants, it was adopted, as it is reckoned as a good fit for this study.

Figure 3 shows the phases of the research intervention informed by SMCKI. The entire collaborative lesson design procedure took approximately 35 min to complete. During the phase one individual ideation, the group leader of each group was tasked with setting up a Google Slide deck and sharing a link to it with other group members and the lecturer at the online class page (Edmodo). Each participant worked individually on one slide illustrating a design for the TEL activities, without influencing other members. There were no discussions during this phase.

Fig. 3
figure 3

Research intervention informed by SMCKI

Full size image

Figure 4 shows a screenshot of Group one participants’ activities during the phase one individual ideation, which lasted 10 min. Thereafter, the lecturer briefed the class to commence phase two intra-group synergy, which lasted for 10 min. During the briefing, the lecturer reminded the participants to consider each group member’s ideas before integrating the group’s ideas in order to be able to present the best quality TEL design from the group in preparation for the inter-group critique at phase three. They were also asked to construct their synergised lesson idea on a new slide within the deck. During phase two intra-group synergy, F2F discussion among group members took place, where members within the group consolidated and synergised lesson ideas into one synergised slide within the same shared Google Slide deck.

Fig. 4
figure 4

Screenshot of activities during phase one

Full size image

Figure 5 shows the screenshot of Group one’s synergised artifact after phase two intra-group Synergy. In phase three, all groups went to one other group board (Group one went to Group two, Group two went to Group three, and so on) to provide constructive feedback for five minutes. This phase operated at the group level. That is, all members within the group viewed another group’s synergised slide together and discussed (F2F) their consensus feedback before one or two representing member(s) commented using the comment boxes on the Google slide. After phase three, each group returned to their own group slide to view the comments. F2F discussion took place, and responses or refinements to the synergised slide were made in response to the critiques. Participants were either to make amendments directly on the lesson ideas on the synergised slide or indicate how would they refine their lesson idea by responding to the comments.

Fig. 5
figure 5

Screenshot of activities during phase two

Full size image

Figure 6 shows a screenshot of participants’ inter-group critique via the comments feature in Group one’s Google Slides deck and TEL activities refinement by group members at phase four. The participants generated and improved the TEL design online over the four phases of SMCKI in class. Phase five was conducted outside of the class where the participants designed and submitted an individual TEL design to the lecturer.

Fig. 6
figure 6

Screenshot of phase three and phase four

Full size image

Figure 7 shows a screenshot of a segment of one participant’s individual achievement at Phase five.

Fig. 7
figure 7

Screenshot of a segment of phase five (Sean, Group one). All Chinese texts translated to English for Fig. 4 to Fig. 7.

Full size image

Data collection and instrument design

Multi-faceted data were collected from this lesson to examine if SMCKI supports collaborative knowledge improvement and how knowledge improves over the SMCKI phases. Data collected in this study included all online artifacts (deleted or edited content) generated by pre-service teachers in the Google Slides deck. TEL design slides from phase one included one slide from each participant; phase two and phase four comprises one slide each from each group. Phase three peer-critiques were each of the comments (contributed at the group level by the critiquing group) within the entire Google slide deck. All Google slides were shared with the lecturer with edit permissions. Therefore, the lecturer had full access to the artifacts, including visibility into the version history, which records the CSCL process during the first four phases of the SMCKI. The TEL designs submitted at phase five were collected via an individual document submission through the Assignment dropbox created in the Virtual Edmodo classroom where each participant had a presence. There were two video cameras placed within the computer-lab (front and back) where all movements within the class were captured. Classroom observation videos enabled validation of the fidelity of the implementation.

To answer the first research question, content analysis was conducted to code the quantity and quality of participant-generated and group-generated artifacts (TEL design and peer-critiques) inside the online workspace during the various phases. Two coding schemes were used to code the online artifacts from the Google Slides. As the nature of the artifacts were related to TEL design, the Technological Pedagogical Content Knowledge (TPACK) framework (Koehler and Mishra 2009) was adopted to analyze the TEL activities. A peer-critique coding scheme was used to analyze the peer comments in this study.

The coding scheme for TEL design in phase one, two, four, and five

TPACK is a well-acknowledged and dependable guide to evaluate the quality of TEL design (Chai et al. 2011; Koh 2013). The TEL design artifacts were coded with the seven TPACK dimensions: Content knowledge (CK, the knowledge about the subject matter to be learned or taught), Pedagogical Knowledge (PK, the knowledge about the processes and practices or methods of teaching and learning); Technology Knowledge (TK, the knowledge of using information technology to accomplish a variety of different tasks in different ways), Pedagogical Content Knowledge (PCK, the transformation of the subject matter for teaching using various pedagogies), Technological Content Knowledge (TCK, an understanding of the manner in which technology and content influence and constrain one another), Technological Pedagogical Knowledge (TPK, an understanding of how teaching and learning can change when particular technologies are used in particular ways), and Technological Pedagogical Content Knowledge (TPACK, an understanding that emerges from interactions among content, pedagogy, and technical knowledge).

The unit of analysis of the content analysis was a complete TEL design presented in each slide in the Google Slide deck. The quality of the TEL design on each slide in terms of each TPACK dimension was measured by a five-point scale ranging from one to five, with one being the lowest quality and five being the highest quality. The explanation of each scale was defined within the rubric for assessing TPACK for meaningful learning with ICT (Koh 2013), and wordings were refined over successive coding trials. This study adopted a total score to compute the TPACK quality of lesson design, as recommended in TPACK performance-based evaluation from a past research study (Abbitt 2011). Please see Appendix Table 2 for the coding scheme for TEL design. Two coders were trained to code the TEL design using the coding scheme. The inter-rater reliability for the seven TPACK dimensions were (Cronbach alpha) 0.75 for TK, 0.71 for PK, 0.73 for CK, 0.75 for TPK, 0.73 for PCK, 0.8 for TCK, and 0.89 for TPACK, respectively.

The coding scheme for peer-comments at phase three

A coding scheme adapted from Clark and Sampson (2007) was used to analyze the peer comments at phase three. The unit of analysis was one comment posted to the group Google Slide deck. Since phase three was operated at the group level, the comments posted were the consolidated opinion of each group. There was no restriction on how many comments each group was able to post. There were a total of 17 comments posted at this phase from the five groups. Each comment was coded as one of the five categories: support (comments that are supportive), suggestion (comments that gave alternatives to enhance the TEL activities), rebuttal (comments that disagree with the TEL activities), query (comments that ask for clarification of the TEL activities) and emotive appeal (comments that express emotion). Two coders coded the peer comments. There was substantial agreement between the coders, with a Cohen’s Kappa value of 0.762.

To answer the second research question, uptake analysis was employed. The concept of uptake deals with the manner in which participants take up and build on prior contributions (Suthers 2006) during the CSCL process. Through the abstraction of log files (via version history of Google Slides), to directed graphs to observe relationships (contingencies) between events, we can interpret the evidence of interactions and possible influences between actors (Suthers and Medina 2011) during the CSCL process. This helps to inform how knowledge improvement takes place over the phases of the SMCKI.

Findings

SMCKI supports pre-service teachers’ knowledge improvement in TEL design

A comparative analysis was done on the quality of the generated TEL design of the class over the SMCKI phases one, two, four and five to answer the first research question. Figure 8 shows the analysis results in terms of the quality of the TEL design for all the five groups at phases one, two, four and five. The mean score of the quality of the TEL activities measured over the seven TPACK dimensions improved consistently over phases one, two, four and five. The improvement from phase one to two (34%) is the highest, followed by phase four to five (23%) and from phase two to four (14%). Figure 9 shows the quality of TEL design measured by the TPACK coding scheme for each group. All groups improved in the quality of TEL design through the SMCKI process. Results are consistent with the understanding that SMCKI supports the participants’ knowledge improvement on the TEL design.

Fig. 8
figure 8

Quality of TEL design for the class

Full size image
Fig. 9
figure 9

Quality of TEL design by group

Full size image

To further investigate the groups’ knowledge improvement demonstrated in the TEL design, we randomly selected one group (Group one) for a more detailed analysis of the improvement on the seven dimensions of TPACK. The quality of the TEL design of Group one was analyzed at three levels of TPACK: namely, the single-dimension level (TK, PK, and CK), the double-dimension level, which focuses on the interactions between two single dimensions (TPK, PCK, and TCK), and the triple-dimension level (TPACK), which represents the interaction between the three single dimensions.

Figure 10 displays the radar graphs representing the quality of TEL design in terms of the three-level seven-dimension TPACK scores for Group one at phase one, two, four and five, respectively. The dotted-line triangle represents the quality of TEL design at the single-dimension level (TK, PK, and CK) and the solid-line triangle represents the quality of TEL design at the double-dimension level (TPK, PCK, and TCK). The middle circle represents the quality of TEL design in terms of the triple-dimension level (TPACK). It is clear from the radar graphs that the area of triangles and circles were bigger and more prominent over the sequence of phases. In addition, the acute triangles at phase one transformed to be more like equilateral triangles by phase five. These results show that the quality of Group one’s TEL design in all seven dimensions at the three levels of TPACK improved from phase one to two, four and five. Moreover, the TEL design became more balanced in terms of the different dimensions of TPACK, which is also an indicator of effective TEL design.

Fig. 10
figure 10

Phase one, two, four, and five for the TPACK dimension (for Group one)

Full size image

How do the pre-service teachers improve their knowledge of TEL activities over the SMCKI phases?

To answer the second research question, a descriptive qualitative analysis was conducted. To investigate how participants improve in their TEL design over the SMCKI phases, we examined the TEL idea uptake pattern of Group one in order to understand 1) how phase one individual ideation contributed to phase two within-group synergy where the group artifact aimed at representing the highest quality of TEL design; 2) how phase three class-wide peer-critique contributed to knowledge improvement at phase four within-group refinement and 3) how phase four within-group refinement contributed to the final individual achievement at phase five. The essence of SMKCI, which is the knowledge improvement when transiting over the phases, provided a focus for answering the second research question.

The uptake analysis (Suthers and Medina 2011) approach was adopted to make explicit the uptake patterns during the TEL design over the SMCKI phases. Uptake happens when a group member takes up and builds on previous contributions to the discourse, and when group members reuse someone’s phrasing or re-express their ideas (Suthers and Medina 2011). In this study, uptake took place during phases two, four and five, serving as evidence of knowledge improvement. The uptake analysis approach helps us trace how the ideas were developed and transformed and enables the visualization of the process of knowledge improvement across SMCKI phases.

We extracted the 35-min process logs of the online collaborative TEL design activity using the version history feature of Google Slides (see Fig. 11). Google Slides automatically stores all edits (add, edit, delete) made by all users within a shared document. The version history logs within a shared document can be easily retrieved for viewing by clicking on the “File” ➔ “Version history” ➔ “See version history” (see Fig. 11). The authors manually exported the process logs stored in Google Slides for this study into an Excel file by extracting the timestamp and the type of edits traced through the slides. This Google Slide document was shared with full edit rights with the authors. This sharing permission setting made the viewing of the version history possible after extraction.

Fig. 11
figure 11

Phase one, two, four, five for the TPACK dimension (for Group one)

Full size image

The unit of the uptake analysis was one TEL procedure that is expressed by one of the TPACK domains (TK, PK, CK, TPK, PCK, TCK, TPACK) in the TEL design. Each TEL design generated on one Google Slide was segregated into many TEL procedures according to the TPACK dimensions. The TEL procedures were coded according to the behaviour transitions (e.g., add, edit, delete, read) in chronological event order. From the coded events, contingency graphs were constructed. As a starting point, the quality of the phase one TEL design for all Group one members: Sean, Jay, Yves and Zac (pseudonyms), was presented in radar graphs to demonstrate the group members’ prior knowledge of TEL design.

Figure 12 shows an unequal level of TPACK across the four members as measured based on their respective individual TEL designs.

Fig. 12
figure 12

Quality of TEL design at Phase one for members of Group 1

Full size image

Among the four members, Sean demonstrated the highest knowledge across all the TPACK dimensions, and his knowledge within the different TPACK dimensions was balanced. Jay and Zac had relatively higher knowledge on CK whereas Yves had relatively higher knowledge on PK and TK.

To examine how the phase one individual ideation contributed to phase two within-group synergy where the group product aimed at representing the highest quality of ideas, a contingency graph (Fig. 13) was constructed to display the process of the improvement of TEL design from phase one to phase two.

Fig. 13
figure 13

Contingency graph and the events from phase one to two (for Group 1)

Full size image

The contingency graph was constructed by working backwards from the events (actions taken by actors) in which the group’s synergised TEL design (box with the text “Group one’s synergised TEL activities”) was finalized before phase three inter-group critique took place. The four boxes with the text “participant’s name + Phase one TEL activities” (e.g. Sean’s Phase one TEL activities) represents the TEL design generated by group members during phase one. The backward dotted arrow shows phase two contingency to phase one individual ideation, the event prior to it. For ease of representation in this graph, only the minute and seconds segment of the time stamp was extracted. The timestamp for each event is illustrated on top of each event box before the participants’ name. The events commenced from 1001 and ended at 1908, with the first two digits representing minutes, and the last two digits representing seconds. The arrows between the boxes indicate contingencies. Dotted arrows represent intra-subjective contingencies, where interactions take place within a participants’ personal slide. Solid arrows represent intersubjective contingencies where interactions involve other elements (e.g. group slide, other group members’ slide).

Figure 13 shows that Jay first created the synergised slide at 1001 with a message “Group One’s Synergised TEL activities” added. Before the commencement of phase two group synergy at 1401, various events took place. These events included the reading of other members’ individual TEL designs (e.g., Jay reading Yves’s design, Zac reading Jay’s design). The affordances of Google Slides allow easy access to each individual’s phase one TEL activities, which were used as references for phase two group discussion. Therefore, these events can be interpreted as the sharing and integration of information. Sharing and integration are essential indicators of collaborative knowledge improvement. Individual ideation at phase one supports sharing of diverse knowledge, which is the basis for collaborative knowledge improvement.

The event of synergising TEL activities commenced at 1401. The TEL design of Sean was copied over to the group synergised slide. This event should not be treated as signifying dominance of one group member nor free-riding of group members as traces from the process logs, and classroom video observations, showed that the construction of the synergised TEL design was a group effort. The events of Jay and Yves reading the synergised TEL design at 1415 showed that they collectively contributed to the group TEL design via verbal discussion. The video analysis of Group one’s work revealed that all group members were involved in the group discussion during the phase two period. This verified that the group members were actively participating during the intra-group synergy phase. As Sean’s phase one TEL design yielded the highest quality in terms of TPACK (refer Fig. 12), the strong inclination of group members to adopt the majority of Sean’s TEL design during phase two intra-group synergy was predictable. To improve the group TEL design, the group deleted two TEL procedures at 1701. This improved the PK and PCK dimensions of the TEL design. Additional edits were made to the TEL design at 1904, which improved the TPACK dimension. In a nutshell, from phase one to phase two, the first decision was to adopt a TEL design from one of the members, followed by further improvement to the group design to achieve the best quality possible.

At phase two, the synthesised TEL design benefitted all group members, including Sean, who had the highest TPACK knowledge among the group members. To examine the zone of proximal development (the distance between what a student can do without help, and what they can do with support from a more knowledgeable other”) (Vygotsky 1978) of Sean, we examined the difference on the TPACK scores between Sean’s TEL design at phase one and the group’s TEL design at phase two. There was an improvement on the single-dimension level for PK and CK (PK = 3.0 for phase one and 3.5 for phase two; CK = 3.0 for phase one and 3.5 for phase two) and the double-dimension level PCK (PCK = 2.5 for phase one and 3.0 for phase two). These results suggest that a deeper understanding of CK could have triggered improvement to PK and PCK. Among the group members, Jay possessed the highest CK, suggesting that the more knowledgeable one does contribute to group knowledge improvement. Among the revision events, there was an edit made to improve the TPACK dimension of the TEL design activity. However, the revision to this TEL design did not arrive at a deeper improvement to the double-dimension level (interactions among TK, PK and CK).

At phase three, the group’s synergised TEL design was critiqued by another group in the class. To examine how phase three between-group peer-critique contributed to knowledge improvement at phase four refinement, we coded the nature of the peer comments. Out of the total of 17 constructive comments generated by the five groups, the majority of the comments fell under the Query categories (N = 9), followed by Suggestion (N = 8), Support (N = 4), Rebuttal (N = 3) and Emotive Appeal (N = 1). Next, we analysed the Group one comments received (from Group five) and provided (to Group two) for a detailed qualitative analysis. To investigate how phase three contributed to knowledge improvement, we examined the contingencies between events through the Group one contingency graph (Fig. 14).

Fig. 14
figure 14

Contingency graph and the events from phase three to four (for Group one)

Full size image

The upper segment of the timeline of Fig. 14 shows the events on the Group one workspace while the bottom segment shows the group activities on Group two’s workspace during the same period. The vertical axis separates the timeline between phase three and phase four. There were a total of two suggestive comments within the PCK dimension provided by Group one to Group two. In terms of the double-dimension level, Group one members possessed higher PCK as compared to TPK and TCK, which could explain why Group one members gave PCK suggestions. In total, Group one received four comments from Group five: “The TEL design is not bad” (coded as “support” on all TPACK dimensions), “You may consider reducing the load of students with fewer number of online platforms” (coded as “suggestion” on TPK), “There were too many online platforms” (coded as “rebuttal” on TPK), and “Will details of the peer assessment rubric be given to students?” (coded as “query” on PK). The contingency graph (Fig. 14) shows that the comments given led to substantial improvement of the TPACK dimension of the group receiving comments. As in the case of Group one, the highest score of the TPACK dimension at phase two was PK and CK (3.5) followed by PCK (3.0), which could be explained by the given comments on PCK during phase three. In contrast, two of the received comments helped with knowledge improvement to Group one’s weakness in the TPK dimension. This resulted in improvement from phase two to phase four (phase two: TPK = 2.0; phase four: TPK = 3.0), suggesting that class-wide peer-critique positively impacted collaborative knowledge improvement.

To further examine how SMCKI phase three contributes to phase four refinement, we examined the qualitative data of each critique comment given and how the critique helped with the TEL lesson design improvement during phase four. Besides the two comments given by Group five, the lecturer contributed one comment to Group one during phase three as well. The uptake graph displayed in Fig. 15 illustrates how each critique (green boxes) contributed to the corresponding action at phase four (blue boxes) signifying how Group one leveraged the critique comments to level up their knowledge of TPACK at phase four. Table 1 shows the detailed qualitative evaluation of the knowledge improvement process from phase two to phase four. There are three columns, as shown in Table 1. The first column details phase two lesson procedures. The second column shows the peer-critique comments from Group five members and the lecturer. The third column shows the refined lesson procedures in phase four. The words in italics indicate the relation between phase three critique and phase four refinement.

Fig. 15
figure 15

Uptake graph from Phase three to four (for Group one)

Full size image
Table 1 Evaluation of the knowledge improvement process from phase two to phase four
Full size table

As shown in Table 1, Group one members responded to the first query critique comment by improving the TEL design at phase four. A peer-critique rubric (PK) for the peer-critique procedure (see last procedure at phase four) was added. In response to the second rebuttal comment and the third query comment, the refinements made to the phase four TEL design were on TPK and TPACK respectively. By deleting the procedure on Plickers and include the use of Quizlet for vocabulary teaching (TK, TCK) instead, the refinement to the technology tool used brought about improvement to the TPK of the TEL design. The inclusion of Nearpod next to Padlet showed an attempt to streamline the number of platforms used. The inclusion of targeted peers during the peer-critique task demonstrated improvement to the PK as well. Through the process of refinement, content knowledge was also re-defined. This refinement was shown through the lesson procedure “know the difference between fact and opinion” and “distinguish between fact and opinion”. While the former targeted at CK at a lower cognitive level, the later targeted at the application from understanding the CK.

These refinements brought about improvement to the quality of phase four TEL design with higher scorings for TK, PK, CK, TPK, TCK and TPACK dimensions. Over all, comments from other groups evidently supported the improvement of the TEL design. In particular, rebuttal and query comments triggered knowledge improvement to the weaker TPACK dimensions of the receiving group. These findings suggest that phase three between-group peer critique supported collaborative knowledge improvement. The comments from other groups helped the group improve the quality of TEL design further, especially on those weaker TPACK dimensions.

To examine how phase four group refinement helps with the final individual achievement at phase five, we examined how events were taken up through a high-level uptake graph (Fig. 16). Figure 16 shows the abstraction of information that are only related to the final artifacts of Group one’s phase five TEL design. Among the group members, Sean was the participant most responsible for the double-dimension TPACK scores because of his uptake in phase four. For the other three group members, the common uptake single-dimension level was PK with an addition of a double-dimension level PCK for Yves. This could have suggested that Sean had acquired a better understanding of TPACK than other members. The uptake pattern in Fig. 16 shows that the group’s refinement at phase four positively affects the individual knowledge improvement reflected in the TEL design at phase five.

Fig. 16
figure 16

Uptake graph for the phase five artifact for Group one members

Full size image

To illustrate the extent of the knowledge improvement of the different TPACK dimensions at phase five for each group member, we constructed radar graphs (Fig. 17) for each of them. By comparing the radar graphs of TPACK of group members between phase one (Fig. 12) and phase five (Fig. 17), it is clear that Zac gained the highest TPACK knowledge improvement from phase one to phase five (phase one = 7.5; phase five = 25.5) followed by Jay (phase one = 10.0; phase five = 22.5), Yves (phase one = 11.0; phase five = 24.5) and Sean (phase one = 13.5; phase five = 26.5). The highest knowledge improvement for individual group member (Zac) suggests that SMCKI is effective in scaffolding lower performing participants for knowledge improvement. The uptake graphs show that SMCKI helped higher performers as well in terms of achieving a higher level of complex knowledge. The wide opportunities provided for the high performing participants to share their knowledge is a possible contributing factor for the group’s knowledge advancement. As a group, the improvement on the single-dimension level (mean difference = 1.92) was higher than the double-dimension level (mean difference = 1.21) and the triple-dimension level (mean difference = 1.25). These results show that SMCKI helps support pre-service teachers’ TPACK knowledge improvement at the single-dimension level to a larger extent as compared to the double-dimension and triple-dimension levels of TPACK.

Fig. 17
figure 17

Phase five TPACK measures for members of Group one

Full size image

In summary, SMCKI commenced with phase one individual ideation, where participants explicitly presented their individual TEL design in the Google Slide deck based on their prior knowledge. This phase provided participants simultaneous access to the work of others with diversified ideas and perspectives, and they were able to appropriate these ideas and perspectives, which were different or more advanced than their own. Based on the findings from the three levels of TPACK dimensions shown in the radar graphs, the participants possessed a lower capacity of TPACK knowledge and uneven distribution of prior knowledge at phase one. The entry phase engaged all participants with whatever prerequisite knowledge they had, yet with opportunities to explore different perspectives and ideas.

Phase two intra-group synergy provided the opportunity for each group to explore further their individual ideas to rise above to a higher quality of TEL design. The results show that group-level TPACK knowledge for all TPACK dimensions superseded that of the mean value of phase one. At this point, each group aimed to showcase the best quality TEL design for class-wide critique at phase three. The attempt to exploit the more complex and advanced dimensions (PCK, TPACK) of TPACK was evident through the uptake analysis of Group one.

Phase three inter-group critique exposed each group’s TEL design to a wider audience for critique within the class. Results from this study show that this phase provided the group wider perspectives of TPACK dimensions. At phase four, participants further improved the quality of TEL design by addressing comments from other groups. The uptake analysis shows that the inter-group critiques triggered group one’s previously underexplored dimensions, such as TPK. At phase four, Group one improved on more complex and advanced knowledge (TPK, TPACK dimension) by addressing the rebuttal and query comments provided during phase three.

Phase five rounds up the SMCKI phases with an individual achievement to recollect the knowledge improvement via the preceding four SMCKI phases. The uptake graph demonstrates the effect of class-wide collaborative learning, which resulted in phase four refinement and subsequently impacted the individual uptake of ideas at Phase five. In addition, the comparison of the radar graphs vividly illustrates the knowledge advancement between phase one and phase five. The results show that SMCKI was able to improve the participants’ knowledge at the single-dimension level (TK, PK and CK), and even higher improvement at the double-dimension level (TPK, PCK, TCK) and the triple-dimension level (TPACK).

Taken together, the SMCKI structures CSCL into five phases: Phases one to two emphasize the presentation of diverse prior knowledge before seeking knowledge improvement through group interactions. Phase three leverages a between-group peer critique when wider perspectives were invited from the class. Phase four provides the opportunity for group interactions, which brought knowledge improvement to the next level. Finally, phase five focuses on individual knowledge improvement by consolidating the learning points from the previous four phases of collaboration. Over all, the quality of TEL design was seen as improving throughout the five phases of SMCKI.

Discussion and conclusion

This study proposed a Spiral Model of Collaborative Knowledge Improvement to scaffold and structure the multi-layered CSCL process in an authentic networked classroom. With SMCKI, collaborative learning at the small group level and the class level created deep meaning shared among participants, which resulted in collaborative knowledge improvement. The results in terms of the TPACK knowledge improvement of the participants contributes to the cognitive aspects of the CSCL process.

Results of this study show that the quality of TEL design at each phase was always higher than those of the preceding phases (e.g. improving the diverse ideas from phase one to enable emergence of the highest quality group TEL design in phase two), which was particularly evident in the transition from phase two to the refinement at phase four as a result of addressing comments from the inter-group critique. SMCKI helps create a learning environment that has a “low floor, high ceiling and wide walls” (Resnick et al. 2009). “Low floor” in SMCKI marks the initial individual prior knowledge contributions to the group from phase one to phase two. It provides easy ways for novice and less-experienced participants to get started. “High ceiling” provides experts or more knowledgeable ones an opportunity to work on increasingly sophisticated ideas. These ideas are first presented during individual ideation, followed by the group level and back to the individual phase. During phase two within-group synergy, participants within the group came together to synthesize a group product that best represents the group’s ability as a whole. The purposeful synthesizing phase where the group product was presented to the entire class for peer-critique eliminates potential “cooperation bias” (Witherspoon et al. 2013) whereby one may intentionally withhold knowledge and not share with the group. Such undesirable social-emotional factors were minimized through the SMCKI.

SMCKI phase three encompasses both the effects of “wide walls” and “high ceilings”. Peer-critique is known to be able to raise the participants’ level of confidence and understanding of the subject matter involved (Akanmu 2016). During the class-wide peer-critique, the actions of each participant were interdependent. These critiques jointly contributed to the development of the TEL design. While the SMCKI phase two brings opportunities to participants to seek consensus or knowledge improvement during the intra-group discussions, resulting in levelling up the knowledge among group members; phase three supports the discussion and collaboration at the class level, leveraging the intelligence of all participants in the networked class. This “wide wall” structure creates opportunities for sharing of broader perspectives in the TEL design that enabled knowledge improvement at the individual, group and class levels. Essentially, the “wide walls” effects created by SMCKI phase three supported the participants in exploring a spectrum of ideas and perspectives from other classmates from floor to ceiling so that participants with different prior knowledge can all be engaged and levelled up.

At phase four, each group reviewed the critique from the class-wide level to advance their group’s ideas. At phase five, participants had the opportunity to tap each other’s ideas within-group and beyond their group, exploring more complex and advanced designs in greater depth, in order to reach the high ceiling. Overall, the results of the analysis show that the pre-service teachers improved the quality of TEL design from the simpler TPACK dimensions to increasingly more complex and advanced dimension levels by communicating, negotiating and critiquing through SMCKI. The participants who have a higher level of understanding of TPACK helped the less advanced members learn within their zone of proximal development (Vygotsky 1978). SMCKI enabled the participants to continually improve their knowledge and quality of artifacts by interacting with classmates within and beyond the group.

SMCKI is a flexible model that can be adapted based on the nature of collaborative learning tasks and characteristics of the networked classroom. All five phases of SMCKI need not be carried out within a lesson facilitated by the teacher. Phase one could be conducted prior to the lesson, and phase five could be conducted after the lesson as a self-directed learning activity. Initially implemented in a science classroom for collaborative argumentation in the secondary school context (Chen et al. 2019), this model has been employed in various contexts (grades and subjects) in Singapore. In this study, we examine its application for pre-service teachers’ collaborative TEL lesson design. In essence, SMCKI can be applied in a wide collaborative learning environment, regardless of whether it is a blended CSCL environment, in a F2F collaborative learning environment without computers, or a purely online environment where students are not collocated in a physical classroom.

Despite the flexibility of implementation, possible challenges during each SMCKI phase do exist. During the individual ideation phase, which aims to promote innovation and facilitate knowledge sharing (Thorsteinsson and Page 2012), it was observed that some participants tended to initiate the discussion (verbal, or online chat) with fellow group members. This kind of habit of collaborative learning needs to be addressed when implementing SMCKI. Another challenge of SMCKI is that it requires all groups in the class to work at the same pace. All individuals transitioned from phase one to phase two at the same time, even though some finished the work earlier. Likewise, all the groups transitioned from phase two to phase three at the same time for efficient peer-critique. The participants need to have good time management skills.

This study contributes to the CSCL literature in the following ways. First, the SMCKI model expands the pedagogical models and approaches of CSCL, especially in a multi-layered networked classroom setting. This study is one of the few studies addressing the pedagogical and instructional support for CSCL at the class-level. Second, this study addresses an underexplored research area in CSCL - the role of individual ideation in collaborative learning. The positive findings from individual ideation in the study have implications for future collaborative learning design, at both small group and class level. Third, SMCKI has the potential to be applied broadly across collaborative learning contexts, in different subjects and among different profiles of learners. Fourth, with the integration of both content analysis and uptake analysis for unpacking the process of collaborative knowledge improvement, this study echoes the recommendation by Stahl et al. (2014) for adopting a more process-oriented account to show the interactions of individuals within a CSCL environment. As stated by Häkkinen (2013), integrating the process-oriented dimension into the analysis can help to find the highest levels of knowledge construction activities. The effect of the SMCKI script on domain learning was explicitly illustrated through the uptake of knowledge shared at both the individual and group level. This analysis helps address the issue that the effect of CSCL scripts has been less clear on domain learning in past literature (Radkowitsch et al. 2020).

In the same way that a collaboration script supports the collaborative process within the CSCL environment, Yilmaz and Yilmaz (2019) found that pedagogical agent support in the CSCL environment has the ability to affect students’ motivation indirectly. Radkowitsch et al. (2020) noted that the design of CSCL scripts might affect the motivation of learners as well. Thus, we propose future research to be done to examine how SMCKI motivates learners throughout the CSCL process. In addition, future research needs to be conducted to validate and scale-up the use of SMCKI. Given the smaller number and the specific profile of the participants in the current study, further studies on the application of SMCKI in other contexts in terms of different profiles and levels of students, different nature of learning tasks, and different learning environments will be conducted. Since this study is exploratory in nature, by adopting a case study approach, it did not have a control group to be compared with the experimental group to confirm what specific improvement was caused by SMCKI. We will employ the experimental design in the future to confirm the effect of SMCKI. In addition, more studies are needed to examine the mechanism of specific phases of SMCKI to better support students’ collaborative knowledge improvement in a networked classroom. As discussed, SMCKI is a flexible model that has a few variations (e.g., without phase one individual ideation, without phase five individual achievement). Future studies are needed to examine the variations of SMCKI in the multi-layered CSCL classroom.