Computer-Based Scaffolding Targeting Individual Versus Groups in Problem-Centered Instruction for STEM Education: Meta-analysis

Abstract

Computer-based scaffolding (CBS) has been regarded as an effective way to help individual students complete and gain skill at completing complex tasks beyond their current ability level. Previous meta-analyses also have demonstrated that CBS for collaborative learning leads to positive cognitive outcomes in problem-centered instruction for STEM education. However, while separate synthesis efforts have been conducted on CBS and collaboration guidance, little work has examined the intersection of these approaches. This study addresses this gap by examining the extent to which the effect of CBS is moderated by the group size in which students work, which type of CBS intervention was used in groups or individually, and whether CBS includes supports for both individual and group works or only individual learning. Results from 145 studies indicate that CBS leads to statistically significant cognitive learning effects when students solve problems individually, as well as working in pairs, triads, and small groups. Moderator analyses indicated that (a) effect sizes are higher when students worked in pairs than when they worked in triads, small groups, or individually; (b) the effect size of metacognitive scaffolding on group activity is higher than other types of scaffolding intervention; and (c) the effect size is higher for groups when scaffolding was present but collaboration support was absent. These results suggest that elaborated design and integration of CBS and collaboration guidance are considered to maximize students’ learning in problem-centered instruction within STEM education.

Fostering problem-solving and collaboration skills are important goals of twenty-first-century education (Casner-Lotto and Barrington 2006; Greiff et al. 2014; Porter et al. 2011). These skills include organizing resources, interpreting data, drawing hypotheses, and representing information (Hannafin et al. 1999), as well as working effectively with others (Lazakidou and Retalis 2010). To facilitate the development of these skills, there has been an increased emphasis on Science, Technology, Engineering, and Mathematics (STEM) education and a shift to problem-centered instructional (PCI) models. The shift from passive learning to more active problem solving is difficult for students who are already familiar with traditional teacher-led instruction, and learners often need extra support for individual and collaborative learning to be successful in this complex learning context. Computer-based scaffolding can offload some of the instructional load of supporting students in problem-based instructional approaches (Roschelle et al., 2010b).

The effectiveness of computer-based scaffolding in enhancing problem-solving skills and other cognitive skills (Belland et al. 2017a, 2017b) and in improving collaboration (Chen et al. 2018; Vogel et al. 2016b) is well documented. While separate synthesis efforts have confirmed the potential of computer-based scaffolding for individual and collaborative learning, and one synthesis investigated the intersection of scaffolding and collaboration support (Vogel et al. 2016b), no synthesis has compared the use of scaffolding and collaboration support to the use of scaffolding by itself in the context of problem-centered instruction. To address these gaps, this article uses meta-analysis to examine the extent to which the effect of computer-based scaffolding in the context of PCI for STEM education is moderated by whether learning was done in groups with different sizes or individually, which type of scaffolding intervention was used in groups or individually, and whether scaffolding is used alone or in conjunction with collaboration support.

Problem-Centered Instruction (PCI) Viewed Through the Lens of Activity Theory

Centering instruction around problems is one way to improve students’ knowledge and skills for solving the complex problems that have no right or wrong answers (ill-structured) and authentic issues that they can face in everyday life (Hmelo-Silver and Barrows 2015). In STEM education, variations of PCI include problem-based learning, case-based learning, inquiry-based learning, project-based learning, and problem-solving (see Fig. 1).

Fig. 1

The variations of problem-centered instruction

Although such models have slightly different learning processes and teacher roles, PCI involves both individual and group activities, and learners engage in reflection on the process of modifying and supplementing their opinions in the context of group problem-solving (Bell 2010). Interactions between group members can help develop not only collaboration and communication skills but also emotional capabilities in solving problems (Stephanou et al. 2013). Therefore, success in PCI hinges to a great extent on relatedness among group members, communication skills, and social and cultural attributes of group members, and the role of tools to support individual and collaborative learning is important for successful PCI. For this reason, it can make sense to use a cultural-historical activity theory lens to view the complex interactions among learners, teachers, and tools, including cultural knowledge, which occurs during PCI (Jonassen 2011). Activity theory views the basic unit of analysis of human behavior as activity and describes the interrelationships between the elements of the activity system that occur during the process of achieving the objects by using the tools in the socio-cultural context (Engeström 2001; Leont'ev 1974; Luria 1976). The activity system consists of six elements—subject, tools, object, rules, community, and division of labor. In general, when the learning activity occurs at the individual level, the subject, the tool, and the object elements are included in the activity system to analyze the effect of interrelationships of these factors on learning outcomes. When the activity is extended to the group or social level, the rules, the community, and the division of labor are added in the activity system (see Fig. 2).

Fig. 2

Six elements in activity system for PCI

Within PCI, ‘the subject’ refers to learners who participate in PCI. Each subject has an ‘object,’ or motive when engaging in PCI. Within PCI, this is often to solve an authentic/ ill-structured problem related to their real-life through individual and collaborative learning. Mediating the relationship between ‘subjects and the object’, ‘Tools’ (i.e., scaffolding) can help learners accomplish the object by providing support for individual and collaborative learning in PCI (Hung and Wong 2000). ‘The rules’ refer to the guidelines of the learning activities and process, which vary in the different PCI models (e.g., problem-based learning, inquiry-based learning, case-based learning). Informed by such rules, learners try to solve the given task individually or through group activities at each learning stage. ‘Division of labor’ defines the detailed activities or tasks to be specified by the objects. The completion of these tasks normally requires students’ self-directed learning and collaborative learning skills; that is, a role and subtasks are assigned to each student through discussion between group members and should be solved through individual learning. ‘Community’ indicates a group that engages in activities with the subject for the same purpose. Depending on how advanced interaction and communication skills this community has, learning objects and students’ different roles (i.e., division of labor) are determined. Interaction and communication skills can be largely affected by group size (Fuchs et al. 2000; Strijbos et al. 2004); therefore, group size can be a factor that defines the character of a learning community (Sung et al. 2017). Finally, in activity theory, learning is a process in which knowledge is continuously transformed by the interaction of six elements in the activity system. Much research has been conducted to demonstrate the effects of scaffolding as a tool in relationship with other elements in PCI (e.g., subjects–tool–objects (Belland et al. 2017b; Kim at al. 2019), subjects–rules (e.g., different PCI models)–tool–objects (Belland et al. 2017a) at the individual level. However, few studies have considered collaboration (e.g., scaffolding interventions used by groups as they engage in problem-solving and collaboration guidance) and group size (the effects of scaffolding in different groups sizes) as characteristics of ‘the community’ and ‘the division of labor’.

Scaffolding

Scaffolding was originally defined as the process by which teachers provide cognitive and motivational support to help learners (e.g., preschoolers) carry out tasks (e.g., build pyramids out of wooden blocks) that they would not be able to complete unaided (Wood et al. 1976). In problem-centered instruction, students address authentic and ill-structured problems that require more advanced problem-solving skills, but face-to-face tutoring from an expert alone (usually a teacher) cannot bear the entire burden of scaffolding support (McNeill and Krajcik 2009; Ustunel and Tokel 2018). To overcome this problem, scaffolding in problem-centered instruction is often provided through computer technology. Computer-based scaffolding is defined as support for individuals from computer tools that allow learners to engage in and gain skill at complex problem-solving (e.g., addressing a multifaceted problem that requires a novel solution) (Belland 2014). Key to this definition is that scaffolding both supports current performance and also enables learners to solve similar complex problems independently in the future. In this way, scaffolding as defined in this paper aligns with the definition of second-order scaffolding (Noroozi et al. 2018).

Computer-based scaffolding has led to increases in students’ cognitive outcomes, ranging from (a) acquisition of deep content knowledge (Bulu and Pedersen 2010; Chang and Linn 2013) to (b) abilities to develop and integrate problem-solving strategies (Bulu and Pedersen 2010; Raes et al. 2012). Computer-based scaffolding to enhance students’ cognitive outcomes can be classified into four types (i.e., conceptual, metacognitive, strategic, and motivational (Hannafin et al. 1999; Rienties et al. 2012). Conceptual scaffolding guides students in terms of what to consider when addressing the problem (Hannafin et al. 1999). One example of conceptual scaffolding consists of a video of an expert who mentions what she would consider when attempting to locate a suitable planet for stranded aliens (Kang et al. 2017). Strategic scaffolding supports students in complex problem-solving by bootstrapping an overall strategy to solve the problem or build an argument (Hannafin et al. 1999) as in the Looi and Lim study (Looi and Lim 2009) where the ALGEBAR intelligent tutoring system invited and supported students to model a word problem, represent it algebraically, and solve it. In contrast, metacognitive scaffolding invites and supports students to plan, monitor, and self-assess their work during problem-based inquiry (Hannafin et al. 1999; Molenaar et al. 2011). For example, one scaffold consisted of a planning sheet, a targeted information collection space, and a project reflection sheet (Su 2007). Motivation scaffolding addresses motivational variables such as autonomy, self-efficacy, and interest to engage with the problem-solving skills and to enhance content knowledge (Belland et al. 2013). For instance, Ker-Splash embedded goal-directed activity within a game setting and provided visual cues to illustrate potential paths of rolling balls, resulting in a better understanding of content (Ke and Abras 2013). Previous meta-analysis work showed that these types of scaffolding improve individual learners’ academic achievement (e.g., cognitive outcomes) (Belland et al. 2017b; Kim 2018), but it is unclear if computer-based scaffolding’s effectiveness is consistent across levels of other important moderators in problem-centered instruction, including presence or absence of collaboration and group activity, which can lead to better learning outcomes by reducing students’ cognitive load in complex and ill-structured tasks (Kirschner et al. 2009).

Collaboration

In the classroom, collaborative learning refers to students working together to address a common goal, resulting in more effective solutions than one would find by summing up individual efforts of groupmates (Jeong and Hmelo-Silver 2016; Johnson and Johnson 2009). Data suggests that students learn more effectively, achieving higher levels of thought and longer retention of knowledge, when they work collaboratively (Johnson and Johnson 1989; Slavin 1980). This can be explained by two theories of Piaget and Vygotsky (Fawcett and Garton 2005; Sills et al. 2016). First, Piaget claimed that learners can experience cognitive conflict from working with groupmates who have different knowledge bases and opinions. In turn, the same learners are incentivized to pursue cognitive development so as to regain cognitive equilibrium (Piaget 1932). Therefore, collaboration with peers can activate individuals’ disequilibrium of knowledge. Second, according to the zone of proximal development, which refers to the difference between the actual level of development and the level of potential development, collaborative learning with more capable peers or heterogeneous group members is helpful for cognitive development (Vygotsky 1978). In addition, intersubjectivity is a process by which participants with different perspectives help group members reach a shared understanding by aligning their views with the other’s through constant communication (Vygotsky 1978). Based on these two theories, the benefits of collaborative problem-solving include interactions that trigger cognitive mechanisms; stimulation of constructive processing; elaboration of and reflection on knowledge; student questions, explanations, and justification of opinions; verbalized thinking; and increased possibility of students solving or examining problems in a variety of ways (Dillenbourg 1999; Jeong and Chi 2006; Soller et al. 1998).

Group Size in Collaborative Learning

To maintain productive collaboration in PCI, it is critical that each student identify groupmates’ learning issues, existing knowledge, perspectives, and experience through active interaction and communication, and that the group members together leverage such in collaboratively addressing the problem (Janssen et al. 2010). In other words, success in PBL depends on how effectively the interaction and communication between the group members are achieved. Research shows that group size matters for achievement in collaborative learning in that group size affects the equal opportunity of interaction and participation in shared problem-solving (Strijbos et al. 2004). Theoretically, between a dyad and a four-member group performing the same task within a given time frame, each learner’s contribution to problem-solving is different, and the interaction patterns and learning benefits in these two groups with different sizes cannot be identical (Fuchs et al. 2000). There has been a diversity of opinions among scholars about the ideal number of group members, as research on the effect of group size was not actively conducted in collaborative learning and problem-centered instruction. Generally, with a larger number of group members (e.g., more than 4), there tends to be more cognitive conflict such as negotiating disagreement and drawing a conclusion from an expanded pool of knowledge and experience, so small group activities are advantageous for students with above-average academic performance or with professional prior knowledge (Fuchs et al. 2000). For example, Barrows (1992), who created problem-based learning, recommended that the most ideal number of members in one group is five to seven in medical education, which requires more advanced problem-solving skills and active exchange of diverse views about the treatment. On the other hand, among low-achieving students or lower grade students who are unfamiliar with collaborative learning, learning in dyads can often result in equal interaction and participation in problem-solving (Strijbos et al. 2004). In the case of large groups (seven or more members), group activities often can be seen as the sum of interactions between small groups, which are newly organized within one large group (Forsyth 2018). In other words, individuals belong to a large group, but the actual learning activity is done by separate small groups within a large group. Therefore, it is difficult to specify the ideal educational context and characteristics of group members for large groups (Rogoff 1995; Strijbos et al. 2004). Despite this proposal for group size, empirical research on group size has failed to show consistent results. For example, in a meta-analysis, college-level healthcare and nursing students who engaged in problem-centered instructional approaches in groups of 4 or less had significantly larger effect sizes than those who worked in groups of 5 or larger in terms of achievement scores as cognitive outcomes (Kalaian and Kasim 2017). On the other hand, in a study in a Dutch medical school, students who worked in groups of 5 perceived greater satisfaction and great participation among groupmates than those who worked in groups with other sizes (Kooloos et al. 2011). Nevertheless, many studies consistently reported more positive outcomes in smaller groups than in larger groups in various aspects: more creative and cognitive activities (Palmérus and Hägglund 1991), and active interaction with teachers and individual students (Smith and Connolly 1980). However, studies that reported positive results of small groups revealed one major limitation. Group size itself can impact collaborative learning, but collaborative learning can be greatly influenced by the presence of scaffolding for individual and group activity (Chen and Law 2016; Vogel et al. 2016b). In other words, successful collaborative learning requires consideration of both the group size and the design of scaffolding. However, few studies indicated to what extent the effectiveness of scaffolding is moderated by group size in problem-centered instruction.

Collaboration Guidance

With the advent of instructional models emphasizing collaboration, interaction, and social constructivism, the role of scaffolding, which originally focused on individual learning situations, has expanded into enhancing collaborative learning activities (Kollar et al. 2006; Laru 2012). Scaffolding for individual learning aims at improving students’ deep understanding of content and developing problem-solving skills (Hannafin et al. 1999). On the other hand, scaffolding for collaborative learning encourages social interactions, helps heterogeneous group members achieve a single and unified goal, and guides students to reach consensus on conflicting opinions (Nussbaum et al. 2009). This scaffolding can play a role in reinforcing the ‘intersubjectivity’ process and creating ‘cognitive conflict’ that are required for successful collaborative learning based on Vygotsky and Piaget’s theories (Fawcett and Garton 2005; Nussbaum et al. 2009). In this study, collaboration guidance is operationally defined as support and prompts for students’ collaborative activities such as argumentation and discussion by assigning specific and integrated collaborative learning activities to each group or showing how to interact with groupmates in PCI. Some examples of collaboration guidance types include CSCL scripts (Fischer et al. 2013; Tchounikine 2016), group monitoring tools (Janssen et al. 2011; Wise and Schwarz 2017), guiding systems (Järvelä and Hadwin 2013), and collaborative scaffolding (Nussbaum et al. 2009). CSCL scripts bootstrap effective collaboration strategies and processes (Fischer et al. 2013). Group monitoring tools enable learners to visualize and otherwise monitor groupmates’ participation (Janssen et al. 2011). Guiding systems use automated analytic tools to detect collaboration problems and guide learners on how to remedy such (Järvelä and Hadwin 2013). Collaborative scaffolding encourages group decision-making activities through the process of a cognitive conflict created by technology (Nussbaum et al. 2009).

As technology has advanced and becomes a common part of educational environments, researchers have begun experimenting with ways that computers can support the collaborative learning of multiple students. One example is Computer-Supported Collaborative Learning (CSCL), defined as the use of networked computer technologies to support collaboration to share resources and (co)construct knowledge (Jeong and Hmelo-Silver 2016; Kirschner and Erkens 2013). CSCL can do this by providing collaboration guidance, enabling learners to (a) access and add to shared resources; (b) interact with each other through discussion, argument, and explanation; (c) manage, record, and monitor collaborative learning progress; and (d) reach a consensus on questions addressed during problem-solving (Jeong and Hmelo-Silver 2016; Rienties et al. 2012). For example, CSCL name removed (Belland et al. 2016) consists of six steps of PCI to address a scientific issue (i.e., water quality issue) and includes a group monitoring tool. Specifically, collaboration guidance allowed middle-school students to see and share each group member’s collected information related to water quality and assigned a unique stakeholder role (e.g., environmentalist, local residents, scientist) to each group member. This process invited students to actively argue about water quality issues from the perspective of their stakeholder and to create various opinions on water quality issues. Another example is Collpad, a digital system to support collaboration on open-ended tasks (i.e., elementary math problems) (Nussbaum et al. 2009). When students collaborate, they draw on collaboration guidance that enhances their interactions with others through different roles that can be performed by each participant. In addition, collaboration guidance helps students distribute tasks, balance group member perspectives, respond to groupmates’ articulations, and synthesize results. The common thing in the above examples is that they reflected well Vygotsky and Piaget’s claims for successful collaborative learning through the role assignment and active interaction.

Some empirical studies have investigated the effectiveness of collaboration guidance on various outcomes. Collaboration guidance fostered students’ knowledge sharing and transfer in multidisciplinary problem-solving settings (Noroozi et al. 2013a). When collaborative learners received collaboration guidance, they had better argumentative skills than those who did not receive the guidance or worked individually (Weinberger et al. 2010). A meta-analysis has shown collaboration guidance to be effective in supporting learners’ collaborative skills (Vogel et al. 2016b).

Nevertheless, one challenge within collaboration guidance has been that students may not follow precisely what was intended by the guidance designer, but rather follow the process reflected in their appropriation of the guidance (Tchounikine 2016). Additionally, the guidance can be leveraged differently by different students depending on their goals (Fischer et al. 2013). Group awareness tools can often be used to a greater or lesser extent by different learners (Janssen and Bodemer 2013). Last, collaboration guidance sometimes improves the objective nature of collaboration, but not cognitive outcomes (Wever et al. 2015). If the purpose of collaboration guidance focuses more on support for sequencing knowledge construction than about how the learners worked through the learning task, this guidance may not be effective in enhancing the individual acquisition of knowledge (Weinberger et al. 2005). Furthermore, the modification of groupwork processes due to collaboration guidance may not necessarily impact cognitive outcomes such as task performance and knowledge acquisition, which may be in part due to group member’s different characteristics (Kollar et al. 2006; Noroozi et al. 2013a). These studies show an imbalance in personal knowledge construction and collaborative skills as a result of collaborative learning supported by collaboration guidance. Given that eliciting strong cognitive development is a key objective of problem-centered instruction in STEM education, it would be logical to provide both (a) individual scaffolding to help each group member address the assigned tasks or acquire content knowledge and (b) collaboration guidance to support the groupwork. Some research has suggested effective scaffolding strategies for individual learning and collaboration guidance for collaborative learning (Kim 2019; Nussbaum et al. 2009), but few empirical studies compared students’ cognitive achievement between groups supported by both scaffolding and collaboration guidance and groups supported by either scaffolding or collaboration guidance in PCI. This is important to investigate because scaffolding and collaboration guidance can each offer opportunities and constraints for learning (Overdijk et al. 2012), but one cannot assume that such opportunities and constraints can be added in a linear manner.

The Goal of the Present Meta-analysis

There have been several meta-analyses of computer-based scaffolding in problem-centered instruction in STEM education. A comprehensive traditional meta-analysis indicated that effect sizes for computer-based scaffolding were higher among adult learners than among college, secondary, middle level, or primary students in terms of learners’ content-specific knowledge (Belland et al. 2017a). Another meta-analysis indicated that scaffolding leads to better cognitive outcomes (e.g., problem-solving skills) among traditional students than among underperforming students (Belland et al. 2014). Cognitive effects of computer-based scaffolding were highest when scaffolding can be added and faded according to students’ current learning status and abilities in the context of problem-based learning in STEM education (Kim et al. 2018). Moreover, meta-analyses have indicated effectiveness of specific types of scaffolding including dynamic assessment for students with special education needs (Swanson and Deshler 2003; Swanson and Lussier 2001) and scaffolding embedded within intelligent tutoring systems used by various age levels (Kulik and Fletcher 2016; Ma et al. 2014; Steenbergen-Hu and Cooper 2013; Steenbergen-Hu and Cooper 2014; VanLehn 2011).

Still, there is less clarity about how the contexts in which scaffolding is used influences scaffolding’s effectiveness (Van der Kleij et al. 2015; VanLehn 2011). Of note, collaboration has been shown to help students develop complex problem-solving skills and successful collaborative learning requires consideration of both the group size and the design of scaffolding. However, few studies indicated to what extent the effectiveness of scaffolding is moderated by group size in problem-centered instruction. In addition, more research is needed to guide the design and implementation of collaboration support (Mende et al. 2017; Splichal et al. 2018). For example, providing collaboration guidance may lead to the type of behaviors (e.g., considering, analyzing, and integrating groupmates’ arguments) that can lead to better argumentation outcomes (Noroozi et al. 2013b; Weinberger et al. 2013). Scaffolding can be used to support individual learning outcomes, problem-solving processes, and collaborative learning processes (Kollar et al. 2006), but more research is needed to disambiguate which of these types of scaffolding supports leads to the greatest cognitive gains. Furthermore, some research suggests that combining scaffolding and collaboration guidance leads to the best learning outcomes in some contexts (Scheuer et al. 2014), and other authors have highlighted the importance of studying how support for collaboration and other support can be combined (Gijlers et al. 2013). The purpose of this meta-analysis was to guide the future design of computer-based scaffolding by addressing the following research questions:

• 1. How is cognitive learning affected when computer-based scaffolds are used by students working in different group sizes in problem-centered instruction for STEM education?

• 2. How does scaffolding intervention type (conceptual, strategic, and metacognitive scaffolding) affect the cognitive learning of groups versus individuals in problem-centered instruction for STEM education?

• 3. How does cognitive learning differ between students working in groups who are supported by scaffolding and collaboration guidance and students working in groups who are supported by only scaffolding in problem-centered instruction for STEM education?

Method

Literature Search

A three-pronged approach of (a) a database search, (b) a targeted hand search of relevant journals, and (c) an ancestral search of references from included studies was employed. First, we searched the following databases for studies published between January 1, 1993 and December 31, 2015: Education Source, PsychINFO, Digital Dissertations, CiteSeer, Proquest, ERIC, PubMed, Academic Search Premier, IEEE, and Google Scholar. These databases were searched using the following Boolean operators “(scaffold OR scaffolding OR scaffolds) AND ( intelligent tutoring systems OR computer OR computers)” . To avoid unintentionally constraining our search, we chose not to use key terms such as “problem-solving” and/or “collaboration” despite their relevance to this study. Rather, we opted to cast as wide a net as possible and then follow up our database search with complementary searches of targeted journals and an ancestral search of included studies. Next, the 5415 database search results were analyzed for coverage in each of the STEM topics. Because we found that the search had produced few studies in mathematics and engineering, we conducted targeted hand searches of the following relevant journals: Journal for Research in Mathematics Education, International Journal of Mathematical Education in Science and Technology, Journal of Professional Issues in Engineering Education and Practice, Journal of Geoscience Education, and Computer Applications in Engineering Education, which yielded an additional 1613 potential studies. Finally, referrals from reference sections of included studies produced an additional 514 studies.

Application of Inclusion Criteria

The combined search results from our three approaches yielded 7543 articles. Next, we conducted a step by step procedure for (a) pre-pass: abstract check/1st pass (see Fig. 3) and (b) full-text screen. Specifically, four researchers, who have expertise in computer-based scaffolding, problem-centered instruction, and CSCL, read the abstract of selected studies and looked for work that is qualitative (e.g., ethnography, case study) without mention of quantitative results. Studies with only qualitative research were eliminated as candidates. In the first-pass process, researchers checked the remaining articles to make sure (a) their definition of computer-based scaffolding fit, (b) the study happened in the context of STEM disciplines, and (c) the study involved control and experimental groups. Through the pre-pass inspection, we excluded 6791 out of 7543 studies and the agreement across four researchers about the exclusion of studies by pre-pass was high (82%); any disagreements were addressed through the discussion and consensus. Researchers investigated the full-text version of the study recommended by the initial screening for determination of final eligibility with the following inclusion criteria:

• (a) participants addressed an ill-structured problem in STEM. Ill-structured problems were defined as problems for which there are multiple solution paths and no one correct answer (Jonassen 2011). Studies in which students did not address this kind of problem were excluded.

• (b) The study should have at least one control and one experimental condition (with scaffolding in PCI). Control refers to students who received a lecture, participated in PCI without scaffolding, or received an alternative intervention in PCI. The detailed coding process is explained in the next section. Control group participants should be independent of those in the experimental group. When given a choice between multiple control groups, a control group that was as close to the treatment as possible while withholding the most scaffolding elements was selected. Studies having a single group with only pre-post measurements were excluded.

• (c) The study should report cognitive learning outcomes affected by computer-based scaffolding and collaboration guidance, or computer-based scaffolding alone. In this meta-analysis, we defined cognitive outcomes as knowledge or skill that could be used to describe entities, describe relationships among entities, and solve problems when such activity is not entirely intrapersonal (Davis and Linn 2000; Ge and Land 2003). This, therefore, excludes metacognitive knowledge and skills, and motivational knowledge and skills, because such are applied intrapersonally. Also excluded were opinions and attitudes, because these cannot objectively report cognitive outcomes. Considering the scope of this meta-analysis, we excluded studies that reported affective, psychomotor or metacognitive outcomes but failed to report cognitive outcomes.

• (d) Sufficient data for effect size calculation included (i) experimental and control mean, SD, and sample size; (ii) binary 2 × 2 frequencies; (iii) ANOVA and sample size; and (iv) t test and sample size. Among the 751 eligible studies, we arrived at a final total of 146 studies with 333 outcomes that satisfied all criteria, involving a total of 27,203 students (see Table 1).

Fig. 3

Pre-pass inspection

Table 1 Inclusion criteria and the number of studies and students in this meta-analysis

Coding Scheme

Articles were coded for the following independent variables—control group, group size, scaffolding intervention, collaboration guidance, and the following dependent variable—cognitive learning outcome.

Control Group

Participants in the control condition engaged with the same or similar learning contents as those in the experimental condition (a) in lecture-based instruction, which was coded as ‘business as usual’ (BAU); (b) in PCI that removed computer-based scaffolding for individual and collaboration guidance coded as ‘Without Scaffolding’ (WOS); and (c) in PCI with alternative learning support tool (e.g., paper-based learning materials and computer’s simple response (e.g., right or wrong) to students’ answers) instead of computer-based scaffolding coded as ‘With Alternative Support (WAS)’ (see Appendix 1). In most included studies (n = 239, 72%), the control condition was WOS. Next came studies in which the control condition was BAU (n = 86, 26%). Studies with WAS as the control condition were a minority (n = 7, 2%). Each control condition can potentially generate different baselines for the subgroup analysis of moderators, resulting in making a direct comparison of effect sizes between studies difficult. Therefore, it was necessary to check if there is a significant difference between the effect sizes comparing scaffolding with each of the control conditions. The effect sizes comparing experimental condition with each of the control condition types were as follows: BAU: g = 0.50, CI [0.41, 0.59], WOS: g = 0.45, CI [0.39, 0.51], WAS: g = 0.329, CI [0.10, 0.56]. No significant difference among the control conditions was found, Q_b(2) = 3.69, p = 0.16. Thus, this meta-analysis did not need to consider bias according to different control conditions.

Group Size

This study originally coded five group size subcategories (i.e., individual, pairs, triad, small group (4–6), large group (7–9)) based on literature about group size (Baines et al. 2003; Del Marie Rysavy and Sales 1991; Ward 1987), but ‘large group’ was excluded from the final analysis and studies with ‘large group’ were coded as ‘small group’ for the following reasons: (1) there were only two studies within the category of ‘large group’ and (2) unique characteristics for large groups do not exist because the actual learning activity in a large group is done by separate small groups (Strijbos et al. 2004).

Group size was coded as individual if students worked individually to address the central problem with the help of scaffolding. For example, elementary students engaged as individuals with My Science Tutor, an intelligent tutoring system covering such topics as electricity (Ward et al. 2013). Group size was coded as pairs if students worked in groups of two students. For example, elementary school students worked in pairs to engage in problem-solving supported by scaffolding embedded in the Aliens software (Ulicsak 2004). When three students within one group tried to solve the given task, this group was coded as triads. For example, students in groups of three could interactively generate collective responses to posted questions through digital augmentation and knowledge building scaffolding (Yoon et al. 2012). Group size was coded as small groups if students worked in groups of 4–6 students. For example, undergraduate students worked in groups of 4 to address an ill-structured problem designing a functional desktop computer with a limited budget in hypermedia software, called All You Need is a Screwdriver (Su and Klein 2010).

Scaffolding Intervention

Scaffolding intervention can be categorized into four types (i.e., conceptual, metacognitive, strategic, and motivational scaffolding) according to how scaffolding tries to assist the learners. To distinguish clearly among the different types of scaffolding intervention, we operatively defined these four types of scaffolding based on scaffolding functions in online learning environments claimed by Hannafin et al. (1999).

Conceptual scaffolding helped students with what to consider (i.e., learning content) when addressing the problem. We coded conceptual scaffolding when the scaffolding in the study played one of the following roles: (a) recommending the use of manipulation or communication tools at certain times; (b) providing targeted expert hints, prompts, and visualization about the content; and (c) helping learners structure their content knowledge through concept mapping.

Metacognitive scaffolding invited and supported students to question their own understanding and/or learning processes through self-evaluative processes (e.g., reflection rubric) and self-regulative strategies (e.g., reflection prompts). For example, high school students were invited to and supported in reflecting on how they performed each step of the self-determined design process for the novel science content (Deters 2008).

Strategic scaffolding was designed to inform learners of resources that can be used in solving the problem or suggest approaches to solving a problem. Strategic scaffolding can be distinguished from conceptual scaffolding in that conceptual scaffolding can help learners think about what to consider when approaching a problem, while strategic scaffolding tells students what processes (e.g., access knowledge, generate a hypothesis, and construct understanding) to use when solving a problem. For example, middle school science students were provided text fragments they could use in discussion posts that helped them key in on important content and ask the right questions (Tan et al. 2005).

Motivation scaffolding was coded when the scaffold aimed to enhance such motivational factors as self-efficacy, autonomy, relatedness, interest, and/or perceptions of the value of the skill or process being supported. For example, we coded motivation scaffolding when the following prompts were used: “How are you feeling right now in dealing with this problem?” and “How are you feeling compared to when you got started” (Koenig 2008a; Koenig 2008b).

Sometimes, multiple scaffolding interventions were used within the same study. We coded multiple scaffolding interventions independently if separate experimental groups used each scaffolding intervention. Some studies used scaffolding integrating two or more interventions, but in most cases, one scaffolding intervention was mainly used and the other intervention was used intermittently and when needed. We only coded one main scaffolding intervention after considering the role, time of use, and importance of each scaffolding intervention. For example, conceptual scaffolding was mainly provided to help learners organize their ideas to solve an ill-structured problem related to pollution (Zydney 2008). Metacognitive reflection prompts were also provided to help learners monitor their knowledge construction process, but the time provided for this metacognitive scaffolding was only 10 min. Therefore, we coded conceptual scaffolding as the main scaffolding intervention in this study.

Collaboration Guidance

Collaboration guidance was coded as yes when students addressed the central problem in pairs, triad, or small groups with scaffolding and collaboration guidance. For example, knowledge-building scaffolds in Knowledge Forum included a set of peer ideas and strategies for considering peer ideas and coming to a consensus as well as the prompts to support individual student’s scientific reasoning (Yoon et al. 2012). Collaboration guidance was coded as no when students addressed the central problem in pairs, triad, or small groups, but there was only scaffolding designed to support individual work and/or the scaffolding did not address collaborative processes. For example, in Raes et al. (2012), students worked in pairs; the scaffolding was designed to support students’ definition of the task, integration of strategies to seek information, the judgment of the relevance of information, and synthesis of information, but did not specifically support collaboration. Collaboration guidance was coded as not applicable when students worked individually. For example, students using SPIRUT worked individually and the system did not contain specific support for collaboration (Ruzhitskaya 2011).

Cognitive Outcome

The cognitive outcome can be categorized according to three main assessment types (i.e., concepts, principles, and application) (Gijbels et al. 2005; Sugrue 1995). At the concept level, students’ basic knowledge and facts (e.g., biology definitions, and order of the species) are measured. The principle-level assessment focuses on knowledge of direction and magnitude of relationships (e.g., presenting a hypothetical situation to students and asking them to predict the outcomes). In a principle-level assessment in an educational game for mathematical word problem-solving (Kajamies et al. 2010), students had to identify critical data, analyze the nature of the relationship among the data, and finally perform the calculation. At the application level, students solve holistic and authentic problems with the use of both types of knowledge: concept and principles. As an example, students can be asked to design a way in which to apply Newton’s law in various realistic contexts through the methods specifying related concepts and their relationship (Reif and Scott 1999). We coded all studies with the three levels of assessment (see Appendix 1), but did not use these categories as separate dependent variables of this meta-analysis for the following reasons: (1) we are only interested in students’ general cognitive outcomes, instead of the specific subcategories of cognitive outcomes according to the assessment levels, and (2) the differences among cognitive outcomes measured by three levels of assessment were not statistically significant, Q(2) = 5.23, p > 0.05. Coding for each included study is presented in Appendix 1.

Coding Process

Three coders who have expertise in scaffolding and problem-centered instruction independently coded each study with the above-mentioned coding scheme. If coding between three coders was inconsistent, consensus codes were determined through discussion. To identify the level of consistency in coding between the three coders, Krippendorff’s alpha was used to assess agreement among the initial codes, as the metric addresses chance agreement and is robust across assessment levels and number of observers (Hayes and Krippendorff 2007) (see Fig. 4).

Fig. 4

Interrater reliability. Note: 0.667 represents minimum acceptable reliability (Krippendorff 2004)

All Krippendorff’s alpha values for the moderators in this study were above the minimum acceptable value, indicating that coding consistency between coders was acceptable.

Analysis

We used the metan package of STATA 15. A random-effects model was adopted given the diversity of scaffolding approaches and research settings in the included studies (Borenstein et al. 2009).

Many studies included in this meta-analysis provided more than one effect size due to (a) multiple measurements within a study, (b) more than one comparison group within a study (i.e., multiple experimental groups and/or control groups), and (c) multiple independent subgroups within one study (Borenstein et al. 2009). We considered two methods to address multiple effect sizes from one study. In the case of (a) and (b), because these effect sizes were not independent and multiple effect sizes from one study can threaten the validity of results due to the bias, we averaged multiple effect sizes from a study. By averaging the multiple effect sizes, we reduced 182 outcomes. In the case of (c), if the multiple effect sizes within one study are generated from the independent experiments and different participants, treating multiple effect size as independent cases is reasonable to avoid loss of unique information (Borenstein et al. 2009; Fisher and Tipton 2015). In this meta-analysis, we expected that the effects of computer-based scaffolding can vary by different group size, scaffolding interventions, and participants; as a result, multiple independent effect sizes from one study were regarded as independent cases. We used robust variance estimation to check the independence of multiple outcomes from one study (Hedges et al. 2010). This approach does not require specific information about within-study covariance and can be applied to any type of weighted effect size estimation (Tanner-Smith and Tipton 2014). Based on sensitivity analysis, there were no differences in the weighted effect sizes and τ² in a range of rho values (within-study effect size correlation) between 0 and 1. Therefore, multiple effect sizes from a single study that were included in this meta-analysis were not dependent. For this reason, the total number of outcomes (N = 333) was larger than the number of studies (k = 145).

Effect sizes were calculated based on data provided in each study such as the mean differences, change scores from pre, post, and retention tests, T test, F test, and precise intra-class correlation, using an online tool (name and link removed for blind review). Hedges’s g was used for effect size estimates because it has the potential to reduce bias through the use of pooled standard deviations and sample size weighting (Hedges 1982).

We assessed publication bias using two methods—visual examination (i.e., funnel plot) and statistical testing (i.e., Egger’s regression and fill and trim testing). First, in the funnel plot (see Fig. 5), which shows the relationship between standard error of treatment effect and effect size, individual studies were distributed around the whole mean of Hedges’s g, but in the case of studies with standard errors higher than 0.4, there were many missing studies in the lower left side, which resulted in the potential asymmetrical distribution of studies. However, interpretations of funnel plots can be subjective, so further statistical testing was conducted to confirm the publication bias. First, after the asymmetry of the effect size distribution was corrected by the Trim and Fill method, the adjusted effect size was g = 0.38 (95% CI [0.32, 0.45]), but there was no statistical difference (p > 0.05) from the observed effect size (g = 0.46, 95% CI [0.41, 0.53]). Furthermore, an Egger’s regression test assuming that there would be no correlation between the effect size and the standard error without the publication bias was not significant, coefficient = 0.336, p > 0.05. Therefore, we concluded that there is little possibility of publication bias although a slight asymmetrical distribution was detected.

Fig. 5

Funnel plot with pseudo 95% confidence limits

Results

RQ1. How Is Cognitive Learning Affected When Computer-Based Scaffolds Are Used in Individual, Pairs, Triad, or Small Groups?

The overall mean effect size (g = 0.46) was greater than 0 at a statistically significant level and heterogeneity statistics were significant for the effectiveness of scaffolding according to group size, Q(332) = 1096.96, I² = 69.7, p < 0.01 (see Fig. 6). This means that 69.7% of the variance can be explained by true heterogeneity across studies, not simple sampling error, allowing us to do a subgroup analysis for the group size moderator.

Fig. 6

Comparison of effect sizes according to group size

The diamonds in Fig. 5 illustrate effect size estimates (represented by the apex) and 95% confidence intervals (lower and upper limit represented by the right and left point, respectively) for scaffolding used by individuals, pairs, triad, and small groups (groups of 4–6 students). Hedges’s g estimates were significantly greater than zero across all group sizes. This means that computer-based scaffolding positively impacts student learning for all coded group sizes. Scaffolding’s effect size differed significantly according to group size,Q_b(3) = 35.92, p < 0.01. Z tests were used to conduct pairwise comparisons between different group sizes. The effect size of scaffolding was nominally greater when used by students working in paired groups than when used by students working individually and in triads but the only statistically significant difference was between pairs (g = 0.59) and small groups (g = 0.34), z = 2.30, p < 0.05.

Students who worked individually showed a higher average effect size (g = 0.47) than other group sizes (i.e., small group [g = 0.34], triad group [g = 0.41]), but there were no statistically significant differences between small groups and individuals, z = 1.67, p > 0.05, as well as triad group and individuals, z = 0.69, p > 0.05. In addition, there were no significant differences between triad group and small group, z = 0.65, p > 0.05).

RQ2. How Does Scaffolding Intervention Type (Conceptual, Strategic, and Metacognitive Scaffolding) Affect the Cognitive Learning of Groups Versus Individuals?

Effect size estimates associated with different scaffolding intervention types varied according to when students were working in pairs/triad/small groups or individually (see Fig. 7). The heterogeneity statistics were significant for scaffolding intervention on both individual activity, Q(235) = 694.33, I² = 66.2, p < 0.01, and group activity, Q(96) = 383.94, I² = 75.0, p < 0.01.

Fig. 7

Effect size comparison of scaffolding intervention in individual versus group activity

Hedges’s g estimates were significantly greater than zero in all cases except for motivation scaffolding used by students working individually (note: we found no includable study that involved motivation scaffolding used by students working in pairs, triad, or small groups). Conceptual scaffolding (g = 0.48) had a nominally higher effect than strategic (g = 0.47) and metacognitive scaffolding (g = 0.38) among students working individually, but there were no statistically significant differences among scaffolding intervention types when students worked individually,Q_b(3) = 4.74, p > 0.05.

When students worked in groups, metacognitive scaffolding had a nominally higher effect size (g = 0.48) than conceptual (g = 0.46) and strategic scaffolding (g = 0.39). But there were no statistically significant differences among scaffolding intervention types,Q_b(2) = 0.94, p > 0.05.

RQ3. How Does Cognitive Learning Differ Between Students Working in Groups Who Are Supported by Scaffolding and Collaboration Guidance and Students Working in Groups Who Are Supported by Only Scaffolding?

Hedges’s g estimates were significantly greater than zero in the case of support and no support for collaboration (see Fig. 7). The interesting finding was that group activity in only 56% of the studies was supported by both scaffolding and collaboration guidance. Significant heterogeneity was found between studies, Q(96) = 383.94, p < 0.01, with a large amount of variation (I² = 75.0), justifying the subgroup analysis.

Among students who worked in groups, those who received collaboration guidance and scaffolding had a lower effect size estimate (g = 0.29) than those who just received scaffolding (g = 0.63; see Fig. 8). This difference was significant,Q_b(1) = 38.87, p < 0.01.

Fig. 8

Comparison of the effect size of scaffolding with collaboration guidance versus without collaboration guidance used by groups

Discussion

This meta-analysis of 333 outcomes extracted from 145 studies with a total of 27,203 students indicates that computer-based scaffolding is effective whether students are working individually or in groups. In all cases, students using scaffolding performed substantially better than control students regardless of the control conditions, but there are some intriguing differences in effect sizes, most notably that outcomes were stronger when scaffolding was present but collaboration guidance was absent (g = 0.63) than when both were present (g = 0.29), and when students worked in pairs (g = 0.59) as opposed to individually (g = 0.47).

Larger Effect Size When Collaboration Guidance Was Absent

Perhaps the most intriguing finding was that effect sizes were higher when groups were provided scaffolding without collaboration guidance than when they received collaboration guidance and scaffolding. With both computer-based scaffolding and collaboration guidance proven effective, it would be logical to expect that combining the two would lead to enhanced learning. Still, it is important to note that the extent to which students learn in the context of problem-centered instruction models supported by scaffolding and collaboration guidance cannot be seen as a simple linear combination of the opportunities and constraints afforded by these elements. Rather, scaffolding and collaboration support are tools that are part of a complex system through which students engage in goal-directed activity (Leont'ev 1974). Note: it would be difficult to fully understand the complex interrelations among scaffolding, collaboration support, problem-centered instructional models, and cognitive outcomes among the included studies because there was little codeable information about student goals. Still, it is important to investigate why the effect size estimate for scaffolding without collaboration guidance is more than double that of scaffolding and collaboration guidance. Another meta-analysis indicated that having a type of collaboration guidance (e.g., CSCL scripts) plus scaffolding led to larger effect size in terms of domain-specific knowledge than having the CSCL script alone (Vogel et al. 2016b). But interestingly, the effect size point estimate for CSCL scripts plus scaffolding (d = 0.3; Vogel et al. 2016b) was almost exactly the same as that for collaboration guidance plus scaffolding in this paper (g = 0.29). But the Vogel et al. 2016b) paper did not include any studies in which scaffolding was present but not CSCL scripts, while the current paper did not include any studies in which collaboration guidance was present but not scaffolding. It is critical to carefully unpack potential reasons why, in this paper, collaboration guidance plus scaffolding led to significantly and substantially lower effect sizes than scaffolding alone. First, collaboration guidance is a more general term that includes CSCL scripts but also other forms of collaboration support such as group awareness tools (Janssen and Bodemer 2013), guiding systems (Järvelä and Hadwin 2013), and collaboration scaffolding (Nussbaum et al. 2009). Next, included studies in the Vogel et al. 2016a, 2016b) meta-analysis did not necessarily involve problem-solving, while all included studies in the current paper do. This is important because problem-centered instructional models that involve groupwork often contain support for collaboration that direct students to collaborate in a particular way. So, studies of students engaging in problem-centered instruction supported by scaffolding and collaboration guidance may have an additional form of support for collaboration that is inherent to the problem-centered model (Dole et al. 2017; Laru 2012). Problem-based learning (PBL), for example, presents inherent rules for collaboration (Hmelo-Silver 2004; Schmidt et al. 2011). Such rules can influence students’ objects (motives) while engaging in PBL. If the same students also receive separate collaboration guidance, such may result in two different potential problems: (a) the assumption of different objects and (b) overlapping opportunities and constraints introduced by the coexistence of collaboration guidance, the rule for collaboration inherent in PCI, and scaffolding. First, with two competing objects, a student’s work may be highly conflicted, and result in poorer quality outcomes, and knowledge and skill gain, than if only one focused object were present. When one compounds this with co-existence with what is likely different collaboration guidance among the group members, different group members can have divergent motives when engaging in problem-solving. This, in turn, can lead to less than ideal problem solving and learning outcomes. In short, with different support for collaboration coming from different sources, students may understandably experience what may be termed guidance dissonance, in that different support invites students to do different things. In the end, more work may be exerted by students to compensate for the guidance dissonance and equilibrize collaboration, and thereby cognitive outcomes suffer. One way to avoid such guidance dissonance may be to assure guidance alignment, defined as alignment between the problem-centered model processes and the processes promoted by collaboration guidance. This is indeed one of the key tenets of the Script Theory of Guidance (Fischer et al. 2013).

Next, collaboration guidance, collaboration rules inherent in the problem-centered model, and scaffolding all present opportunities and constraints (Overdijk et al. 2012). To seize the opportunities afforded by such, groupmates need to engage in a process of negotiation, but some of the possible opportunities afforded may be canceled out by constraints posed by other artifacts (Overdijk et al. 2012). Thus, while the combination of opportunities posed by all of the artifacts may be rich, there may be considerably less actionable opportunities due to conflicts with constraints. Still, being invited to do many different things by different guidance requires that learners make choices about which guidance to follow. Autonomy is, of course, a positive element in most learning environments, as it can enhance motivation (Deci and Ryan 1987; Stefanou et al. 2004) and propensity to engage in lifelong learning (Dunlap and Grabinger 2003). However, simply allowing students to make choices does not in itself constitute fostering autonomy (Deci and Ryan 1987). Rather, it is critical to invite students to make meaningful choices and provide criteria to help them choose (Belland et al 2013a; Chinn et al. 2013; Rogat et al. 2014). The coded papers do not contain enough information to definitively conclude that the choice provided to students was not meaningful and that choosing criteria was not provided. But providing for meaningful choices and providing choosing criteria is difficult, and it is not something that instructors naturally do (Reeve 2009). There is a clear need for more research on how and why scaffolding and collaboration guidance opportunities and constraints interact.

Another possible explanation for the larger effect size among students who did not receive collaboration guidance is that students receiving the collaboration guidance were guided by their appropriation of the guidance (Tchounikine 2016). The appropriation of the guidance may have been to simply acknowledge and integrate peers’ work into the group work, rather than critically engage with the work in a dialectic manner. If the former is the case, then it would make sense that achievement is lower than with no collaboration guidance (Vogel et al. 2016a). That is, much scaffolding at its core fosters critical engagement with information in the interest of dialectically approaching the truth (Ford and Wargo 2012; Pea 2004; Saye and Brush 2002; Weinberger and Fischer 2006). If the push toward critical engagement is contradicted by an appropriation of collaboration guidance that says that students need to integrate groupmate ideas in some other way, then that can hamper good learning outcomes.

Furthermore, within collaborative groups, collaboration guidance could conceivably be appropriated differently by different group members. For example, in a triad, all three group members could appropriate collaboration guidance differently. When taken together with what would likely be three different forms of collaboration guidance, and the collaboration guidance inherent in PCI, the group would have to reconcile up to six different forms collaboration guidance that may all say to do different things. On top of that, the scaffolding would instruct students to do different things still and could potentially be appropriated differently by different group members (Belland et al. 2013b). It is easy to imagine that all of these competing messages would lead to relatively little student attention remaining to learn the content at hand.

One way forward may allow for greater learner instrumentalization of collaboration guidance (Lonchamp 2012). That is, rather than present the guidance as hard and fast rules to be enforced automatically, collaboration guidance can be seen as general rules to guide learner activity and be adapted as learners see fit (Lonchamp 2012). After all, allowing for learner autonomy is important to student success in problem-centered instruction (Rotgans and Schmidt 2011; Wijnia et al. 2011).

The Effects of Computer-Based Scaffolding for Students Working in Groups or Alone

Computer-based scaffolding plays a role in developing individual students’ problem-solving skills, engagement, and deep content knowledge in problem-centered instruction (Steenbergen-Hu and Cooper 2014; VanLehn 2011). Our results indicated that scaffolding enhanced cognitive outcomes among students working in different group sizes or alone; however, the effect sizes of scaffolding varied according to group sizes. Most scaffolding in this meta-analysis (k = 236 out of a total of 333 outcomes) was designed to support students working individually. The effect size (g = 0.47) of scaffolding used in individual students’ work was similar to that of existing meta-analyses with similar topics and contexts (Belland et al 2017b, g = 0.46; Ma et al. 2014, g = 0.41; VanLehn 2011; ES = 0.40). That scaffolding was more effective when students work in pairs (g = 0.59) than when students work individually (g = 0.47) is not surprising given the substantial literature that reports students solve problems better when working in groups than when working individually (Azmitia 1988; Heller et al. 1992). Interestingly, the effects of scaffolding according to group size show a similar pattern with the different effects of collaborative learning according to group size. The effect size of computer-based scaffolding in the pair group (g = 0.59) was higher than in the other group sizes (i.e., triads [g = 0.41] and small groups [g = 0.29]). The collaborative learning literature consistently states that simply putting students in a group does not make a collaborative group (Johnson and Johnson 2009; Slavin 1980). Rather, one needs to establish social interdependence among groupmates, and in this way, the product of their collaboration will be greater than the sum of their individual contributions. It is possible, given the literature that demonstrated increased difficulties arriving at common goals and coordinating work as group sizes become larger (Cummings et al. 2013), that much scaffolding already serves to divide up the overall problem-solving task into sub-components that can be completed by separate groupmates. If this is the case, such scaffolding may already serve to foster social interdependence, albeit without the explicit goal of supporting collaboration. As such, it might work better in smaller groups (i.e., pairs).

This may also point to a problem in the design of scaffolding for students working in groups. The original metaphor of scaffolding described support for the problem solving of individual learners (Wood et al. 1976). Many of the bedrock principles of scaffolding (e.g., the use of modeling, dynamic assessment, and fading) remain the same even when applied to scaffolding used by students working collaboratively (Puntambekar 2015). Given the benefits of collaborative learning, it may be possible that scaffolding would work better among learners working in groups if such scaffolding’s design were grounded in a specialized model for scaffolding for collaborative groups. By this, we do not mean simply a new conceptual framework, but rather a new conceptual foundation for collaborative scaffolding that would clarify for example what dynamic assessment and corresponding customization looks like for collaborative groups, how one would know when transfer of responsibility happens, and what scaffolding mechanisms are most valuable to support group problem solving.

Scaffolding Intervention Type Used in Groups and Individually

It is intriguing that scaffolding largely has strong effects across scaffolding types both when it is used individually and in pairs or larger groups. The exception was with motivation scaffolding, which had an effect not statistically greater than zero, as in our previous work (Belland et al. 2017a), and was not used in an included study in which students worked cooperatively. While there were certainly variations in strategies used in the different scaffolding systems across the studies, there was no difference in the effectiveness of conceptual, metacognitive, or strategic scaffolding based on group or individual learning. Individual learners and those working in groups ended with statistically the same effect size, and ones that were nominally at most 0.1 SDs apart from each other. This is an interesting contrast to the meta-analysis done by Lou et al. (2001), in which students using computer technologies—tutorial learning, drill and practice, exploratory environment (e.g., simulation), tool (e.g., word processor or programming language)—had a higher effect size when working in groups versus individually. This may have been due to better perseverance and use of better learning strategies among students working in groups than among students working individually (Lou et al. 2001). While we did not code for perseverance and use of learning strategies in the current meta-analysis, one may imagine that all scaffolding should at least attempt to promote the use of strong learning strategies. Many computer technology types (e.g., word processor, programming language) included in Lou et al. (2001) do not likely address learning strategies at all. Another reason that one might find statistically equivalent effect sizes across scaffolding types between individual and group learning, but not among computer technologies in general is related to the critical importance of feedback within learning (Black and Wiliam 1998; Shute 2008). One natural advantage of collaborative learning is the need to constantly articulate ideas and receive feedback on such (Baker 2015). This is often done in the negotiation of shared goals and work plans (Van den Bossche et al. 2006). Much scaffolding contains feedback (Proske et al. 2012; Roschelle et al., 2010a; Ulicsak 2004) and invites negotiation of goals (Cahill et al. 2010); thus, even if one is working with scaffolding individually, one is still articulating ideas and receiving feedback. When working with a word processor, one would not get feedback except from a teacher, while when working collaboratively, one would receive feedback from groupmates as well.

Yet, it is still somewhat perplexing that students who worked in triads or larger groups while being supported by computer-based scaffolding would achieve at the same level as those working individually using the same scaffolding types. Work on distributed scaffolding suggests that making scaffolding accessible in diverse forms (e.g., computer-based and peer) causes performance to increase (McNeill and Krajcik 2009; Puntambekar and Kolodner 2005; Tabak 2004). But it is not accurate to assume that all students working in triads or larger groups receive peer scaffolding, as assigning a learning task to two or more learners working together does not automatically lead to either (a) the generation and provision of peer scaffolding, or (b) collaborative learning (Belland 2014; Baker 2015; Pata et al. 2006). Clearly, more research is needed to disentangle the contributions of groupwork, collaboration, and computer-based scaffolding to cognitive outcomes.

Limitations and Suggestions for Future Research

First, as we used broad search terms related to computer-based scaffolding, we had a chance to find more potential studies (N = 7543) including both individual and collaborative learning. And in consultation with our grant advisory board, our search terms were validated to find a broad range of studies related to scaffolding. However, it is possible that our search terms could unintentionally miss some studies covering CSCL or collaborative learning if those studies did not include any scaffolding characteristics in their interventions. Therefore, authors of future meta-analyses with a similar topic should carefully consider the appropriateness of search terms. Furthermore, through a sufficient range of databases, we could include a sufficient number of studies, but including others (e.g., Scopus) might lead to even more relevant publications.

Second, scaffolding has many different forms (e.g., hints, feedback, questioning prompts, and expert modeling) and types (i.e., conceptual, metacognitive, procedural, and strategic) (Belland et al 2017b; Kim et al. 2018; Hannafin et al. 1999). Numerous meta-analyses have been conducted to investigate the effects of different scaffolding characteristics (Belland et al 2014; Belland et al 2017b; Kim et al. 2018; Kulik and Fletcher 2016; Ma et al. 2014; Steenbergen-Hu and Cooper 2013, 2014; VanLehn 2011) and there has been a consensus among scholars on the positive effects of scaffolding. However, it is difficult to categorize collaboration guidance into universal forms and features due to a relative lack of detailed description of the design and role of collaboration guidance. A recent meta-analysis (Vogel et al. 2016b) showed effects of CSCL scripts based on different levels (play, scene, and scriptlet), but still had the above-mentioned limitations. To estimate the effects of collaboration guidance more accurately, future CSCL studies should have more detailed descriptions of their supports, making it possible to categorize collaboration guidance.

Third, there is much valuable research on scaffolding and support for collaboration outside of STEM education. Further research should expand to other areas such as history and English education, diverse educational settings (e.g., other cities in USA, other countries), different ability levels (e.g., low, middle, and high-achieving level), and different kinds of outcomes (e.g., metacognition, motivation, and subcategories of cognitive outcomes), which are conducive to problem-centered instruction, scaffolding, and support for collaboration (Fitzgerald and Graves 2004; Nussbaum 2002; Proctor et al. 2007; Saye and Brush 2002).

Conclusion

Results of this meta-analysis indicate that computer-based scaffolding leads to positive effects when students are solving problems individually, as well as in pairs, triad, and small groups. Effect sizes were larger when students worked in pairs and individually than when working in triads and small groups, and the effect size was significantly greater when specific guidance for collaboration was not provided in addition to scaffolding, versus when it was. Suggestions for further refining of the scaffolding metaphor include developing a stronger theoretical foundation for scaffolding used by student groups, assuring alignment between collaboration guidance and the processes promoted by problem-centered instructional models, and integrating scaffolding and collaboration guidance more seamlessly.

Appendix 1

Summary of included outcomes

Study	Group size	Scaffolding for collaboration	Scaffolding intervention	Assessment level	Education level	Participants (n)	Effect size	Lower CI	Upper CI	Control group
(Hmelo and Day 1999)	individual	no	conceptual	principles	graduate/pro	35	0.82	0.12	1.49	BAU
	individual	no	conceptual	principles	graduate/pro	34	0.83	0.12	1.50	BAU
(Chen et al. 2003)	pair	no	conceptual	concept	primary	43	1.06	0.41	1.67	BAU
(Mayer et al. 2002)	individual	no	conceptual	principles	C/V/T	48	0.84	0.24	1.41	BAU
	individual	no	conceptual	principles	C/V/T	94	0.68	0.22	1.12	BAU
	individual	no	strategic	principles	C/V/T	95	0.47	0.03	0.91	BAU
	individual	no	conceptual	principles	C/V/T	73	0.85	0.37	1.32	BAU
(Chang et al. 2001)	individual	no	conceptual	concept	middle	32	0.4	− 0.30	1.08	WOS
	individual	no	conceptual	concept	middle	33	0.25	− 0.43	0.92	WOS
(Linn and Eylon 2000)	small	no	conceptual	principles	middle	144	0.53	0.18	0.87	WOS
	small	no	conceptual	principles	middle	144	0.36	0.02	0.70	WOS
	small	no	conceptual	principles	middle	144	0.32	− 0.03	0.66	WOS
(MacGregor and Lou 2004)	individual	no	conceptual	concept	primary	22	0.94	0.03	1.79	WOS
	individual	no	conceptual	concept	primary	22	1.22	0.27	2.08	WOS
(Revelle et al. 2002)	pair	no	strategic	application	primary	99	0.77	0.36	1.17	WOS
	pair	no	strategic	application	primary	96	0.79	0.37	1.20	WOS
(Lane 2004)	individual	no	strategic	application	C/V/T	21	0.91	− 0.02	1.76	BAU
(Puntambekar et al. 2003)	individual	no	conceptual	principles	middle	36	0.82	0.13	1.47	WOS
	individual	no	conceptual	concept	middle	32	0.52	− 0.19	1.21	WOS
	individual	no	conceptual	concept	middle	36	− 0.4	− 1.05	0.26	WOS
(Zhang et al. 2004)	individual	no	strategic	concept	middle	80	− 0.16	− 0.60	0.28	WOS
	individual	no	strategic	principles	middle	80	− 0.34	− 0.77	0.10	WOS
	individual	no	strategic	application	middle	80	− 0.1	− 0.53	0.34	WOS
	individual	no	strategic	concept	middle	30	− 0.35	− 1.06	0.37	WOS
	individual	no	strategic	principles	middle	30	− 0.21	− 0.91	0.51	WOS
	individual	no	strategic	application	middle	30	0.06	− 0.65	0.76	WOS
	individual	no	strategic	principles	middle	30	0.87	0.11	1.59	WOS
	individual	no	strategic	principles	middle	30	0.64	− 0.10	1.35	WOS
(Ulicsak 2004)	pair	collaboration	strategic	concept	primary	51	0.21	− 0.34	0.75	BAU
	pair	collaboration	strategic	concept	primary	51	0.13	− 0.42	0.67	BAU
	pair	collaboration	strategic	concept	primary	51	0.13	− 0.42	0.67	BAU
	pair	collaboration	strategic	concept	primary	51	0.41	− 0.14	0.96	BAU
	pair	collaboration	strategic	concept	primary	51	0.22	− 0.33	0.76	BAU
(Vreman de Olde and de Jong 2006)	individual	no	strategic	application	secondary	35	0.77	0.04	1.46	WOS
(Zydney 2005)	individual	no	conceptual	principles	middle	30	1.01	0.24	1.74	WOS
	individual	no	metacognitive	principles	middle	30	0.19	− 0.53	0.89	WOS
	individual	no	conceptual	principles	middle	30	0.75	0.00	1.46	WOS
	individual	no	metacognitive	principles	middle	30	0.82	0.06	1.53	WOS
	individual	no	conceptual	principles	middle	30	0.6	− 0.14	1.30	WOS
	individual	no	metacognitive	principles	middle	30	− 0.01	− 0.71	0.70	WOS
(Siegel 2006)	individual	no	strategic	principles	secondary	47	0.43	− 0.16	1.01	WOS
(Manlove et al. 2007)	pair	no	conceptual	principles	secondary	35	− 1.26	− 1.95	− 0.51	WOS
	pair	no	conceptual	concept	secondary	35	1.62	0.82	2.35	WOS
(Fund 2007)	individual	no	conceptual	principles	middle	154	0.95	0.61	1.28	WOS
	individual	no	strategic	principles	middle	147	0.67	0.34	1.00	WOS
	individual	no	conceptual	principles	middle	151	0.65	0.32	0.97	WOS
	individual	no	conceptual	principles	middle	165	0.21	− 0.10	0.51	WOS
	individual	no	conceptual	principles	middle	154	1.14	0.80	1.48	WOS
	individual	no	strategic	principles	middle	147	0.91	0.57	1.25	WOS
	individual	no	conceptual	principles	middle	151	0.67	0.34	1.00	WOS
	individual	no	conceptual	principles	middle	165	0.29	− 0.02	0.59	WOS
(Graesser et al. 2007)	individual	no	conceptual	concept	C/V/T	33	0.16	− 0.52	0.83	WOS
	individual	no	conceptual	concept	C/V/T	33	− 0.36	− 1.03	0.33	WOS
(Koenig 2008a, 2008b)	individual	no	motivational	concept	adult	38	0.87	0.20	1.52	WOS
(Su 2008)	small	collaboration	conceptual	application	C/V/T	65	− 0.11	− 0.60	0.37	WOS
	small	collaboration	conceptual	application	C/V/T	63	− 0.17	− 0.66	0.33	WOS
	small	collaboration	conceptual	concept	C/V/T	216	0.15	− 0.12	0.41	WOS
	small	collaboration	conceptual	concept	C/V/T	208	0.24	− 0.04	0.51	WOS
(Etheris and Tan 2004)	small	collaboration	strategic	application	middle	9	0.67	− 0.68	1.86	WOS
(Tan et al. 2005)	large	collaboration	strategic	principles	middle	68	0.61	0.12	1.09	WOS
	large	collaboration	strategic	principles	middle	68	0.7	0.20	1.18	WOS
	large	collaboration	strategic	principles	middle	68	0.36	− 0.12	0.83	WOS
(Demetriadis et al. 2008)	individual	no	conceptual	concept	C/V/T	32	0.74	0.02	1.43	WOS
	individual	no	conceptual	concept	C/V/T	32	0.86	0.12	1.55	WOS
(Belland 2009)	small	collaboration	strategic	application	middle	37	− 0.25	− 0.90	0.41	WAS
	small	collaboration	strategic	application	middle	49	− 0.09	− 0.65	0.47	WAS
(Pifarré et al. 2006)	pair	no	strategic	principles	middle	89	0.63	0.20	1.05	BAU
(Simons and Klein 2007)	small	no	conceptual	application	middle	70	0.82	0.30	1.33	WOS
	small	no	conceptual	application	middle	64	0.99	0.45	1.52	WOS
	small	no	conceptual	principles	middle	70	0.64	0.13	1.14	WOS
	small	no	conceptual	principles	middle	64	0.36	− 0.16	0.87	WOS
(Lee et al. 2008)	individual	no	conceptual	principles	G/P	38	− 0.27	− 0.90	0.36	WOS
(Zydney 2008)	individual	no	conceptual	principles	secondary	41	0.47	− 0.15	1.08	WOS
	individual	no	conceptual	principles	secondary	39	0	− 0.62	0.62	WOS
	individual	no	conceptual	principles	secondary	41	0.43	− 0.19	1.04	WOS
	individual	no	conceptual	concept	secondary	40	0.03	− 0.58	0.64	WOS
	individual	no	conceptual	concept	secondary	38	0.52	− 0.13	1.15	WOS
	individual	no	conceptual	concept	secondary	40	0.03	− 0.59	0.64	WOS
	individual	no	conceptual	concept	secondary	41	− 0.18	− 0.78	0.43	WOS
	individual	no	conceptual	concept	secondary	39	− 0.03	− 0.65	0.60	WOS
	individual	no	conceptual	concept	secondary	41	0.03	− 0.57	0.64	WOS
(Looi and Lim 2009)	pair	no	conceptual	principles	middle	68	1.07	0.55	1.56	BAU
(Yeh et al. 2010)	individual	no	conceptual	principles	C/V/T	163	1.56	1.21	1.91	WOS
	individual	no	conceptual	principles	C/V/T	162	1.38	1.03	1.72	WOS
(Mendicino et al. 2009)	individual	no	strategic	concept	primary	56	0.62	0.08	1.15	WOS
(Gijlers 2005)	pair	collaboration	conceptual	principles	secondary	44	0.69	0.08	1.28	BAU
	pair	collaboration	conceptual	concept	secondary	44	0.61	0.00	1.20	BAU
	pair	collaboration	conceptual	principles	secondary	44	− 0.15	− 0.73	0.44	BAU
	pair	collaboration	conceptual	concept	secondary	44	− 0.27	− 0.85	0.32	BAU
	pair	collaboration	strategic	principles	secondary	24	0.38	− 0.42	1.16	WOS
	pair	collaboration	strategic	concept	secondary	24	− 0.49	− 1.27	0.32	WOS
	pair	collaboration	strategic	principles	secondary	24	0.73	− 0.11	1.52	WOS
(Ross and Bruce 2009)	individual	no	conceptual	principles	middle	178	0.27	− 0.03	0.56	BAU
	individual	no	conceptual	principles	middle	217	0.08	− 0.27	0.42	BAU
(Kajamies et al. 2010)	pair	no	strategic	principles	primary	16	0.7	− 0.32	1.64	BAU
	pair	no	strategic	principles	primary	16	0.58	− 0.42	1.52	BAU
	pair	no	strategic	principles	primary	16	0.78	− 0.25	1.73	WOS
	pair	no	strategic	principles	primary	16	0.51	− 0.49	1.45	WOS
(Sun et al. 2011)	individual	no	strategic	concept	middle	46	0.32	− 0.26	0.89	WOS
(Toth et al. 2002)	small	collaboration	strategic	concept	secondary	10	1.06	− 0.30	2.21	WOS
	small	collaboration	metacognitive	concept	secondary	10	− 1.03	− 2.19	0.32	WOS
(Yoon et al. 2012)	triad	collaboration	conceptual	concept	middle	119	0.16	− 0.34	0.66	WOS
	triad	collaboration	conceptual	principles	middle	34	0.93	0.21	1.61	WOS
(Reid et al. 2003)	individual	no	conceptual	principles	middle	38	− 0.35	− 0.98	0.29	WOS
	individual	no	conceptual	concept	middle	38	− 0.32	− 0.95	0.32	WOS
	individual	no	conceptual	application	middle	38	− 0.03	− 0.66	0.60	WOS
	individual	no	strategic	concept	middle	38	0.08	− 0.55	0.71	WOS
	individual	no	strategic	application	middle	38	0.75	0.08	1.38	WOS
	individual	no	strategic	principles	middle	38	0.59	− 0.07	1.22	WOS
	individual	no	strategic	concept	middle	38	0.38	− 0.27	1.01	WOS
	individual	no	strategic	principles	middle	38	0.6	− 0.05	1.23	WOS
	individual	no	strategic	application	middle	38	0.53	− 0.12	1.16	WOS
(Ward et al. 2013)	individual	no	conceptual	concept	primary	1098	0.18	− 0.05	0.40	BAU
(Clark et al. 2012)	individual	no	conceptual	concept	middle	50	0.41	− 0.15	0.96	WAS
	individual	no	conceptual	concept	middle	50	0.59	0.02	1.15	WAS
	individual	no	conceptual	application	middle	50	0.52	− 0.05	1.07	WAS
(Raes et al. 2012)	pair	no	conceptual	concept	secondary	135	0.02	− 0.32	0.36	WOS
(Liu et al. 2013)	individual	no	conceptual	concept	secondary	128	1.37	0.97	1.74	WOS
(Stark 2013)	individual	no	conceptual	principles	C/V/T	42	0.13	− 0.50	0.77	WOS
	individual	no	conceptual	principles	C/V/T	37	− 0.78	− 1.44	− 0.08	WOS
(Tan 2000)	small	collaboration	strategic	application	C/V/T	53	− 0.2	− 0.75	0.36	WOS
	small	collaboration	strategic	application	C/V/T	53	0.35	− 0.21	0.90	WOS
(Li 2001)	individual	no	conceptual	principles	C/V/T	36	0.93	0.23	1.59	WOS
	individual	no	conceptual	principles	C/V/T	36	0.1	− 0.55	0.74	WOS
	individual	no	conceptual	principles	C/V/T	35	1.17	0.43	1.85	WOS
	individual	no	conceptual	principles	C/V/T	35	0.24	− 0.42	0.89	WOS
	individual	no	conceptual	principles	C/V/T	36	0.6	− 0.07	1.25	WOS
	individual	no	conceptual	principles	C/V/T	36	− 0.21	− 0.85	0.44	WOS
(Chu et al. 2010)	individual	no	conceptual	concept	primary	13	1.72	0.37	2.83	BAU
(Ge et al. 2010)	small	collaboration	conceptual	principles	C/V/T	96	1.83	1.34	2.29	WOS
(Ching 2009)	individual	no	strategic	principles	C/V/T	49	0.33	− 0.24	0.88	WOS
	individual	no	metacognitive	principles	C/V/T	50	0.21	− 0.35	0.76	WOS
	individual	no	strategic	principles	C/V/T	50	0.53	− 0.04	1.08	WOS
(Thomas 2011)	individual	no	strategic	concept	G/P	18	0.85	− 0.13	1.75	WOS
	individual	no	strategic	concept	G/P	20	2.29	1.09	3.29	BAU
(Ruzhitskaya 2011)	pair	no	conceptual	concept	C/V/T	132	0.39	0.04	0.73	BAU
	pair	no	conceptual	concept	C/V/T	131	0.44	0.10	0.79	BAU
(Clarebout and Elen 2006)	individual	no	strategic	application	C/V/T	128	0.48	0.13	0.83	WOS
	individual	no	strategic	application	C/V/T	121	0.33	− 0.03	0.68	WOS
(Chen et al. 2005)	pair	no	conceptual	concept	primary	12	1.9	0.44	3.06	WOS
(Barab et al. 2009)	pair	no	conceptual	concept	C/V/T	25	1.38	0.48	2.19	WOS
	pair	no	conceptual	application	C/V/T	25	1.53	0.60	2.35	WOS
(Hickey et al. (2008)	individual	no	conceptual	concept	C/V/T	26	0.93	0.10	1.70	WOS
	individual	no	conceptual	application	C/V/T	26	0.51	− 0.28	1.26	WOS
(Lee 2010)	triad	no	conceptual	concept	C/V/T	247	0.73	0.47	0.98	WOS
	triad	no	conceptual	concept	C/V/T	248	0.48	0.22	0.73	WOS
	triad	no	conceptual	concept	C/V/T	248	0.37	0.12	0.62	WOS
(Su and Klein 2010)	small	collaboration	conceptual	concept	C/V/T	109	0.14	− 0.23	0.52	WOS
	small	collaboration	conceptual	concept	C/V/T	99	0.33	− 0.08	0.72	WOS
	small	collaboration	metacognitive	concept	C/V/T	104	0.05	− 0.34	0.43	WOS
	small	collaboration	metacognitive	concept	C/V/T	112	− 0.34	− 0.71	0.03	WOS
	small	collaboration	conceptual	principles	C/V/T	63	− 0.71	− 1.20	− 0.19	WOS
	small	collaboration	metacognitive	principles	C/V/T	65	− 0.12	− 0.60	0.37	WOS
(Schrader and Bastiaens 2012)	individual	no	conceptual	concept	middle	59	1.17	0.61	1.71	WOS
	individual	no	conceptual	principles	middle	59	0.95	0.40	1.47	WOS
(Chen et al. 2013)	individual	no	conceptual	concept	C/V/T	28	0.83	0.02	1.60	BAU
(Holland 2009)	individual	no	conceptual	principles	C/V/T	35	− 0.04	− 0.72	0.64	WOS
(Kumar 2005)	individual	no	conceptual	concept	C/V/T	64	0.26	− 0.23	0.75	WOS
	individual	no	conceptual	concept	C/V/T	48	0.48	− 0.10	1.04	WOS
(Bornas and Llabrés 2001)	individual	no	conceptual	concept	primary	30	0.71	0.50	0.92	BAU
(Barak and Dori 2005)	individual	no	conceptual	concept	C/V/T	215	0.68	0.40	0.95	BAU
(Ronen and Eliahu 2000)	pair	no	conceptual	application	secondary	42	1.64	0.87	2.35	BAU
	pair	no	conceptual	application	secondary	34	1.04	0.29	1.74	BAU
(Deters 2008)	individual	no	metacognitive	application	secondary	51	0.33	− 0.22	0.88	WOS
	individual	no	metacognitive	application	secondary	52	0.34	− 0.22	0.88	WOS
(Conati and Vanlehn 2000)	individual	no	metacognitive	principles	C/V/T	56	0.1	− 0.42	0.62	WOS
(Dori and Belcher 2005)	triad	collaboration	conceptual	concept	C/V/T	811	0.55	0.35	0.74	BAU
(Kaberman and Dori 2009)	individual	no	strategic	application	secondary	241	0.64	0.33	0.94	WOS
	individual	no	strategic	application	secondary	176	0.58	0.27	0.88	WOS
(Dori and Sasson 2008)	individual	no	conceptual	principles	secondary	661	0.84	0.57	1.10	WOS
(Nichols et al. 2011)	triad	collaboration	conceptual	principles	C/V/T	269	− 0.07	− 0.31	0.17	WOS
	triad	collaboration	conceptual	principles	C/V/T	254	− 0.15	− 0.40	0.10	WOS
	triad	collaboration	conceptual	concept	C/V/T	254	0.24	− 0.01	0.49	WOS
	triad	collaboration	conceptual	concept	C/V/T	269	0.15	− 0.09	0.39	WOS
(Leutner 1993)	individual	no	conceptual	concept	middle	32	0.63	− 0.09	1.32	WOS
	individual	no	conceptual	principles	middle	32	− 0.99	− 1.69	− 0.24	WOS
	individual	no	conceptual	concept	C/V/T	38	0.84	0.16	1.48	WOS
	individual	no	conceptual	principles	C/V/T	38	− 0.56	− 1.19	0.09	WOS
	individual	no	conceptual	concept	middle	40	0.23	− 0.39	0.84	WOS
	individual	no	conceptual	principles	middle	40	0.19	− 0.43	0.80	WOS
	individual	no	conceptual	concept	middle	40	0.19	− 0.42	0.81	WOS
	individual	no	conceptual	principles	middle	40	− 0.17	− 0.78	0.45	WOS
	individual	no	conceptual	concept	middle	32	− 0.01	− 0.69	0.68	WOS
	individual	no	conceptual	concept	middle	40	0.27	− 0.35	0.88	WOS
	individual	no	conceptual	principles	middle	32	0.36	− 0.34	1.04	WOS
	individual	no	conceptual	concept	middle	40	0.1	− 0.51	0.71	WOS
(Vanlehn et al. 2005)	individual	no	conceptual	principles	C/V/T	912	0.25	0.03	0.47	BAU
	individual	no	conceptual	application	C/V/T	1066	0.5	0.38	0.62	BAU
(Parchman et al. 2000)	individual	no	conceptual	concept	C/V/T	37	0.49	− 0.19	1.16	BAU
	individual	no	conceptual	principles	C/V/T	37	0.14	− 0.53	0.80	BAU
	individual	no	conceptual	concept	C/V/T	47	0.27	− 0.30	0.84	BAU
	individual	no	conceptual	principles	C/V/T	47	0.3	− 0.27	0.87	BAU
(Renkl 2002)	individual	no	conceptual	principles	C/V/T	48	0.5	− 0.09	1.07	WAS
(Rieber et al. 2004)	individual	no	conceptual	principles	C/V/T	26	1.61	0.68	2.42	WOS
(Wiley et al. 2009)	individual	no	conceptual	principles	C/V/T	60	0.51	− 0.01	1.02	WOS
	individual	no	conceptual	concept	C/V/T	60	0.63	0.11	1.14	WOS
	individual	no	conceptual	principles	C/V/T	60	1.05	0.50	1.57	WOS
	individual	no	conceptual	principles	C/V/T	60	0.77	0.24	1.29	WOS
	individual	no	conceptual	principles	C/V/T	60	0.74	0.21	1.25	WOS
(Ardac and Akaygun 2004)	individual	no	conceptual	concept	middle	49	0.88	0.25	1.48	BAU
(Chang et al. 2010)	small	collaboration	conceptual	concept	middle	110	0.47	0.08	0.85	WOS
	small	collaboration	conceptual	principles	middle	110	0.63	0.24	1.01	WOS
	small	collaboration	conceptual	concept	middle	114	− 0.49	− 0.87	− 0.11	WOS
	small	collaboration	conceptual	principles	middle	114	− 0.2	− 0.57	0.18	WOS
(Frailich et al. 2009)	triad	collaboration	conceptual	concept	secondary	233	0.76	0.48	1.03	BAU
(Hundhausen et al. 2011)	individual	no	conceptual	principles	C/V/T	21	− 0.22	− 1.05	0.63	WOS
(Dori et al. 2003)	individual	no	conceptual	principles	C/V/T	215	1.03	0.74	1.31	BAU
(Finkelstein et al. 2005)	small	no	conceptual	principles	C/V/T	231	0.43	0.17	0.70	WOS
	small	no	conceptual	principles	C/V/T	231	0.25	− 0.02	0.51	WOS
(Adair and Jaeger 2014)	individual	no	strategic	concept	C/V/T	81	0.68	0.22	1.12	BAU
	individual	no	strategic	principles	C/V/T	81	0.13	− 0.31	0.56	BAU
(Mitrovic and Ohlsson 1999)	individual	no	conceptual	concept	C/V/T	46	0.75	0.14	1.33	BAU
(Huang et al. 2013)	individual	no	conceptual	principles	C/V/T	86	0.54	0.11	0.96	WOS
(Martín-Gutiérrez et al. 2013)	individual	no	conceptual	principles	C/V/T	40	0.22	− 0.40	0.84	BAU
	individual	no	conceptual	principles	C/V/T	40	0.09	− 0.53	0.70	BAU
(Aydin and Cagiltay 2012)	large	no	conceptual	principles	C/V/T	112	1.43	1.00	1.84	WOS
(Katai 2011)	individual	no	conceptual	principles	C/V/T	43	1.06	0.40	1.67	BAU
(Van Eck and Dempsey 2002)	individual	no	conceptual	principles	middle	35	0.66	− 0.03	1.32	WOS
	individual	no	conceptual	principles	middle	35	0.2	− 0.46	0.86	WOS
(Rodriguez et al. 2006)	individual	no	conceptual	concept	C/V/T	11	1.09	− 0.22	2.21	WOS
	individual	no	conceptual	principles	C/V/T	11	0.67	− 0.55	1.78	WOS
	individual	no	conceptual	principles	C/V/T	11	0.92	− 0.36	2.03	WOS
(Pfahl et al. 2004)	individual	no	strategic	concept	graduate/pro	34	0.63	− 0.06	1.30	WOS
	individual	no	strategic	principles	graduate/pro	34	0.08	− 0.58	0.75	WOS
(Hwang et al. 2010)	individual	no	conceptual	concept	primary	45	0.34	− 0.25	0.91	WOS
(Roschelle et al. 2010a, Roschelle et al. 2010b, Roschelle et al. 2010)	triad	collaboration	conceptual	concept	primary	158	0.32	0.01	0.63	WOS
(Marbach-Ad et al. 2008)	individual	no	conceptual	concept	secondary	132	0.22	− 0.13	0.56	WOS
	individual	no	conceptual	principles	secondary	132	0.56	0.21	0.91	WOS
	individual	no	conceptual	concept	secondary	132	0.7	0.35	1.05	BAU
	individual	no	conceptual	principles	secondary	132	1.92	1.49	2.32	BAU
(Pareto et al. 2011)	individual	no	metacognitive	concept	primary	153	0.38	0.05	0.70	BAU
(Hwang and Hu 2013)	small	collaboration	conceptual	principles	primary	58	0.59	0.06	1.10	WOS
(Hulshof and de Jong 2006)	individual	no	conceptual	concept	C/V/T	25	0.61	− 0.20	1.39	WOS
(Swaak et al. 1998)	individual	no	conceptual	concept	C/V/T	42	0.1	− 0.50	0.69	WOS
	individual	no	conceptual	principles	C/V/T	42	0.77	0.14	1.38	WOS
(Manlove et al. 2006)	triad	collaboration	conceptual	principles	secondary	17	0.92	− 0.10	1.85	WOS
(Ardac and Sezen 2002)	individual	no	conceptual	concept	secondary	39	0.66	0.01	1.29	BAU
	individual	no	conceptual	principles	secondary	43	0.13	− 0.47	0.72	BAU
(Zhang et al. 2000)	individual	no	conceptual	concept	middle	26	− 0.08	− 0.84	0.67	WOS
	individual	no	conceptual	principles	middle	26	− 0.45	− 1.20	0.33	WOS
	individual	no	conceptual	application	middle	26	− 0.18	− 0.93	0.59	WOS
	individual	no	conceptual	concept	middle	26	− 0.75	− 1.51	0.06	WOS
	individual	no	conceptual	principles	middle	26	− 0.29	− 1.04	0.48	WOS
	individual	no	conceptual	application	middle	26	0.21	− 0.55	0.97	WOS
	individual	no	conceptual	concept	middle	26	0.65	− 0.15	1.41	WOS
	individual	no	conceptual	principles	middle	26	0	− 0.76	0.76	WOS
	individual	no	conceptual	application	middle	26	− 0.11	− 0.87	0.65	WOS
	individual	no	conceptual	application	middle	26	− 0.23	− 0.98	0.54	WOS
	individual	no	conceptual	application	middle	26	− 0.12	− 0.87	0.64	WOS
	individual	no	conceptual	application	middle	26	− 0.03	− 0.79	0.73	WOS
(Leemkuil and de Jong 2012	individual	no	conceptual	concept	C/V/T	194	0.05	− 0.23	0.33	WOS
	individual	no	conceptual	principles	C/V/T	194	0.22	− 0.06	0.50	WOS
(Mulder et al. 2011)	individual	no	conceptual	concept	secondary	58	0.02	− 0.49	0.53	WOS
	individual	no	conceptual	principles	secondary	58	1.05	0.49	1.58	WOS
	individual	no	conceptual	concept	secondary	56	0.31	− 0.22	0.83	WOS
	individual	no	conceptual	principles	secondary	56	0.07	− 0.45	0.59	WOS
(Atkinson et al. 2003)	individual	no	strategic	principles	C/V/T	39	0.93	0.26	1.57	WOS
	individual	no	strategic	principles	C/V/T	39	0.31	− 0.33	0.92	WOS
	individual	no	strategic	principles	C/V/T	39	0.74	0.08	1.37	WOS
(Hundhausen and Brown 2008)	pair	collaboration	conceptual	principles	C/V/T	79	0.47	0.02	0.91	WAS
(Kramarski and Hirsch 2003)	individual	no	metacognitive	principles	middle	43	0.95	0.31	1.56	WOS
(Teong 2003)	pair	no	metacognitive	principles	middle	40	0.59	− 0.05	1.20	WOS
	pair	no	metacognitive	principles	middle	40	0.74	0.09	1.36	WOS
(Kramarski and Gutman 2006)	pair	no	metacognitive	concept	secondary	65	0.51	0.01	0.99	WOS
	pair	no	metacognitive	application	secondary	65	0.84	0.33	1.34	WOS
	pair	collaboration	metacognitive	principles	middle	43	1.95	1.20	2.64	WOS
	pair	collaboration	metacognitive	concept	middle	43	1.39	0.70	2.02	WOS
(Zydney et al. 2014)	individual	no	conceptual	concept	primary	30	0.54	− 0.19	1.25	WOS
(Galleto and Refugio 2012)	individual	no	conceptual	principles	C/V/T	95	0.48	0.07	0.88	BAU
(Kong 2011)	individual	no	conceptual	concept	G/P	68	0.51	0.01	0.99	BAU
(Graesser et al. 2003)	individual	no	conceptual	concept	C/V/T	48	1.56	0.86	2.20	BAU
(Pareto et al. 2012)	pair	collaboration	conceptual	concept	primary	38	0.76	0.10	1.40	BAU
(Chin et al. 2013)	individual	no	conceptual	concept	primary	133	0.97	0.57	1.36	WOS
(Hwang et al. 2012)	individual	no	conceptual	concept	primary	43	0.64	0.02	1.24	WOS
(Corbett and Anderson 2001)	individual	no	strategic	principles	C/V/T	20	0.73	− 0.19	1.59	WOS
	individual	no	strategic	principles	C/V/T	20	0.95	0.01	1.82	WOS
	individual	no	strategic	principles	C/V/T	20	1.14	0.16	2.01	WOS
	individual	no	strategic	concept	C/V/T	20	0.58	− 0.32	1.43	WOS
	individual	no	strategic	concept	C/V/T	20	0.79	− 0.13	1.65	WOS
	individual	no	strategic	concept	C/V/T	20	0.9	− 0.04	1.76	WOS
(Girault and d’Ham 2013)	individual	no	strategic	concept	C/V/T	23	0.59	− 0.27	1.40	WOS
(Korganci et al. 2014)	individual	no	conceptual	concept	secondary	30	1.7	0.83	2.47	WOS
	individual	no	conceptual	concept	secondary	32	0.75	0.02	1.44	WOS
(Zucker et al. 2013)	small	no	conceptual	principles	middle level	781	0.28	0.14	0.42	BAU
(Reif and Scott 1999)	individual	no	conceptual	application	C/V/T	30	1.33	0.51	2.08	BAU
(Hung et al. 2013)	individual	no	conceptual	principles	middle	49	0.62	0.04	1.18	WOS
(Ifenthaler 2012)	individual	no	metacognitive	concept	C/V/T	58	0.83	0.29	1.36	WOS
	individual	no	metacognitive	concept	C/V/T	66	− 0.02	− 0.51	0.47	WOS
	individual	no	metacognitive	principles	C/V/T	58	0.52	− 0.01	1.04	WOS
	individual	no	metacognitive	principles	C/V/T	66	− 0.19	− 0.68	0.30	WOS
(Yin et al. 2013)	small	collaboration	conceptual	concept	G/P	41	1.07	0.40	1.70	WOS
(Osman and Lee 2013)	individual	no	strategic	principles	secondary	127	0.52	0.17	0.87	BAU
(Moreno and Mayer 2005)	individual	no	conceptual	concept	C/V/T	54	0.32	− 0.22	0.86	WOS
	individual	no	conceptual	principles	C/V/T	54	1.19	0.60	1.76	WOS
(Kereluik 2013)	individual	no	metacognitive	principles	secondary	45	0.07	− 0.54	0.69	WOS
(Butz et al. 2006)	individual	no	conceptual	principles	C/V/T	39	0.97	0.30	1.61	BAU
	individual	no	conceptual	principles	C/V/T	39	1.31	0.59	1.97	BAU
(Philpot et al. 2005)	individual	no	conceptual	principles	C/V/T	114	0.64	0.17	1.10	BAU
	individual	no	conceptual	principles	C/V/T	78	0.8	0.29	1.29	BAU
(Segedy 2014)	individual	no	conceptual	concept	middle	65	0.01	− 0.47	0.50	WOS
(Kinnebrew et al. 2014)	individual	no	strategic	principles	middle	35	0.15	− 0.52	0.81	WOS
	individual	no	metacognitive	principles	middle	32	0.71	− 0.02	1.40	WOS
	individual	no	strategic	principles	middle	35	0.18	− 0.48	0.84	WOS
	individual	no	metacognitive	principles	middle	32	0.06	− 0.63	0.75	WOS
(Hwang et al. 2014)	individual	no	conceptual	concept	middle	66	0.65	0.15	1.14	WOS
Rosen and Tager 2014)	individual	no	strategic	principles	secondary	190	0.65	0.35	0.94	WOS
(Chen 2014)	individual	no	conceptual	principles	middle	170	0.94	0.62	1.25	WOS
	individual	no	conceptual	principles	middle	170	− 0.16	− 0.46	0.14	WOS
	individual	no	conceptual	principles	middle	170	1.47	1.12	1.80	WOS
(Zacharia 2005)	triad	no	conceptual	concept	C/V/T	88	0.69	0.26	1.12	WOS
	triad	no	conceptual	principles	C/V/T	88	0.79	0.35	1.22	WOS
(Rouinfar et al. 2014)	individual	no	conceptual	principles	C/V/T	80	0.74	0.28	1.18	WOS
(Madsen et al. 2013)	individual	no	conceptual	principles	C/V/T	37	0.69	0.02	1.33	WOS
(Siler et al. 2010)	individual	no	strategic	principles	middle	28	0.8	0.02	1.53	BAU
	individual	no	strategic	principles	middle	25	0.84	0.01	1.62	BAU
Woo et al. 2006)	individual	no	conceptual	concept	G/P	50	− 0.48	− 1.03	0.09	BAU
	individual	no	conceptual	principles	G/P	50	1.22	0.60	1.80	BAU
(Weusijana et al. 2004)	triad	collaboration	strategic	concept	C/V/T	54	0.55	0.00	1.08	WOS
(Koedinger et al. 1997)	small	no	conceptual	principles	secondary	169	0.66	0.30	1.01	BAU
(Koedinger et al. 1997)	small	no	conceptual	concept	secondary	169	0.32	− 0.03	0.67	BAU
(Lin and Lehman 1999)	individual	no	metacognitive	principles	C/V/T	45	0.6	− 0.01	1.18	WOS
	individual	no	conceptual	principles	C/V/T	45	0.1	− 0.49	0.68	WOS
	individual	no	motivational	principles	C/V/T	45	0.06	− 0.52	0.64	WOS
	individual	no	metacognitive	application	C/V/T	45	1.41	0.74	2.04	WOS
	individual	no	conceptual	application	C/V/T	45	0.6	0.00	1.19	WOS
	individual	no	motivational	application	C/V/T	46	0.4	− 0.19	0.97	WOS
(Kumar et al. 2007)	pair	collaboration	conceptual	principles	C/V/T	58	0.6	0.07	1.11	WOS
(Ge and Land 2003)	small	no	conceptual	principles	C/V/T	24	1.77	0.78	2.64	WOS
	small	no	conceptual	principles	C/V/T	31	1.21	0.43	1.94	WOS
(Dancik and Kumar 2003)	individual	no	conceptual	principles	C/V/T	47	0.59	0.00	1.16	BAU
Kumar 2002)	individual	no	conceptual	concept	C/V/T	33	− 0.16	− 0.83	0.52	BAU
(Beal et al. 2010)	individual	no	conceptual	concept	middle	23	0.71	− 0.21	1.58	BAU
	individual	no	conceptual	concept	middle	32	− 0.28	− 1.10	0.56	BAU

1. The following acronyms are used: graduate/professional (G/P); college/vocational/technical (C/V/T); business as usual (BAU); problem solving without scaffolding (WOS); problem solving with scaffolding (WAS)