Abstract
CSCL as a field incorporates diverse methodological practices. This chapter provides an overview of research methods and practices in CSCL. Research methods are designed to answer those questions central to CSCL, whereas the practices refer to how these methods are used in practice. The chapter considers the diversity of methodological practices that are used to address the questions that CSCL researchers ask. In particular, research designs, settings, data sources, and analysis methods are reviewed using a literature corpus of CSCL research in STEM fields from 2005 to 2014. The results of this review show the range of practices used in CSCL research and the common practice of mixing methods. Finally, future trends related to visualization, automated analysis, and multimodal data are considered. These future trends address the complexity and diversity of CSCL environments as well as the challenges in analyzing the vast amount of data in seeking to support and understand collaborative sensemaking.
Access provided by Autonomous University of Puebla. Download chapter PDF
Similar content being viewed by others
Keywords
1 Definitions and Scope: CSCL Methodological Practices
CSCL is an active and growing field that embraces diverse methodological practices. In this chapter, we provide an overview of research methods and methodological practices in CSCL. Research methods are the specific techniques designed to answer those questions central to CSCL. Methodological practices are how those techniques are applied. These questions revolve around how technology supports and mediates collaborative learning and sensemaking (Jeong & Hmelo-Silver, 2016; Suthers, 2006). We consider research methods and practices to include types of questions being asked, research designs, settings in which the research is conducted, sources of data that are used, and approaches taken to analyze the data. Research methods are, at their very essence, ways to produce knowledge that a professional community considers to be legitimate (Kelly, 2006). These methods are the ways that we gather and analyze data and produce evidence that help make inferences that address research questions and problems (Shavelson & Towne, 2002). We consider qualitative, quantitative, and mixed-method approaches. That said, the choice of method (and resulting methodological practice) is often driven by the overriding theoretical commitments of the authors (see Stahl & Hakkarainen, this volume) as well as the goals of research.
This chapter describes the results from an examination of an expanded corpus of CSCL research from 2005 to 2014 that includes 693 articles (McKeown et al., 2017), updating the results of similar work of the authors that covers 400 articles from 2005 to 2009 (Jeong, Hmelo-Silver, & Yu, 2014), that we refer to as “the earlier results.” This article corpus includes systematically collected journal articles with CSCL research from 2005 to 2014. The earlier results included a review of both STEM and non-STEM domains, but the updated corpus is focused on STEM domains as well as use in education programs. In the earlier results, these disciplines accounted for 78% of the papers reviewed (Jeong & Hmelo-Silver, 2012) suggesting that narrowing the focus to STEM and education disciplines will not have a substantial effect on the review presented here. To update this, we include some newer examples, selecting empirical articles from recent ijCSCL volumes to use as running examples. These examples were selected because they were about CSCL environments that used mixed methods or multiple measures as well as new analytic tools. We consider overall historical trends and the challenges in CSCL research in formal educational settings, with a particular focus on STEM and Education disciplines.
In considering methodological practices in CSCL, it is important to consider the nature of questions being asked, the research designs used, the settings in which research is conducted, the types of data collected, and the analytical tools used. Research designs indicate the plan for addressing the research question. They differ depending on whether the study goals are descriptive or explanatory. Research settings refer to the contexts in which the research is conducted, generally either classrooms or controlled laboratory settings. Data are defined as the sources and materials, which are analyzed in the research. Analysis methods are the processes conducted on the data sources that include both qualitative and quantitative approaches. These are defined further in their respective sections. Within all these different aspects, there is considerable diversity—a theme to which this chapter returns. CSCL research methods are unique in that they address written and spoken language as well as nonverbal aspects of interaction in examining collaborative processes and outcomes.
2 History and Development
Research methods have been an important topic of discussion in the CSCL community. This is not surprising given the complexity of CSCL environments with the interplay of pedagogies, technology, and modes of collaboration (Kirschner & Erkens, 2013; Major, Warwick, Rasmussen, Ludvigsen, & Cook, 2018; McKeown et al., 2017). It has been the subject of numerous workshops at CSCL conferences over the years that resulted in a special issue of Computers and Education in 2003 on “Documenting Collaborative Interactions: Issues and Approaches” (Puntambeker & Luckin, 2002) with another special issue on “Methodological Issues in CSCL” in 2006 (Valcke & Martens, 2006). In 2007, a special issue of Learning and Instruction focused on “Methodological Challenges in CSCL” (Strijbos & Fischer, 2007). The interest has continued in the CSCL book series with two methods-focused volumes in that series (Analyzing interactions in CSCL: Methodology, approaches, and issues, Puntambekar, Erkens, & Hmelo-Silver, 2011; Productive Multivocality in the Analysis of Group Interactions, Suthers, Lund, Rosé, & Teplovs, 2013).
The methods of CSCL have been derived from psychology, education, cognitive science, linguistics, anthropology, and computer science. Over time, the kinds of research questions have become more diverse. In the earlier research, most studies focused on examining the effects of technology and instruction, and although these were still a major focus in the later years of the larger corpus, researchers also addressed more questions that included effects of learner characteristics and affective outcomes. There have also been trends over time to a smaller percentage of qualitative studies that are not connected to a well-described research method and a higher percentage of studies that used inferential statistics and content analysis.
3 State of the Art: Current Methodological Practices in CSCL
3.1 Research Questions in CSCL
Research methods are designed to help answer specific types of research questions. Such questions organize research activities. CSCL researchers use research questions to determine the appropriateness of data collection and analysis methods and evaluate the relevance and meaningfulness of results (Onwuegbuzie & Leech, 2006). Research questions can be distinguished from research problems or goals (Creswell & Creswell, 2017; Onwuegbuzie & Leech, 2006). A research problem is an issue or dilemma within the broad topic area that needs to be addressed or investigated such as shared regulation in online learning. A research purpose or objective follows from the research problem and specifies the intent of the study such as whether it intends to describe variable relationships, explain the causality of the relationships, or explore a phenomenon (e.g., whether to search for causes or seek a remedy). A research question is a specific statement of specific inquiry that the researcher seeks to investigate (e.g., whether shared regulation can be improved in certain conditions). A hypothesis, unique to quantitative research, is a formal prediction that arises from a research question (e.g., group awareness tools can improve shared regulation).
In the corpus reviewed from 2005 to 2014, most research questions were, not surprisingly, organized around examining technology interventions (37% of studies) or instructional interventions (24%). In addition, 13% of studies looked at the effects of learner characteristics such as motivation. But looking at these general questions in isolation does not tell the whole story. Often CSCL research asks questions about knowledge outcomes, learner, and/or group characteristics such as gender and group cohesion, as well as different kinds of knowledge outcomes and collaborative processes. The CSCL studies reviewed often asked multiple questions in a given study. Similarly, a review that focused on classroom discourse and digital technologies found that similar themes emerged in examining the affordances of technologies and learning environments more broadly (Major et al., 2018). However, they also found a theme focused on how digital technologies enhanced dialogic activities and support knowledge co-construction. Even without the emphasis on STEM, Major et al. (2018) found similar themes providing converging evidence for the generality of the kinds of questions asked in CSCL. A recent study exemplifies this trend to use multiple questions as Borge, Ong, and Rosé (2018) ask questions about whether and how a pedagogical framework and technology support can affect group regulation as well as how individual reflective scripts affect collaborative processes. The Borge et al. study also epitomize the challenges in categorizing the kinds of research questions that CSCL researchers ask—it is testing a theoretically grounded framework, but this framework is embodied in a pedagogical approach and technology that are being studied.
3.2 Research Designs and Settings
Research designs refer to strategies for inquiry that allow researchers to address their questions (Creswell & Creswell, 2017). Traditionally, they tend to be divided into three types: qualitative, quantitative, and mixed methods and within each of these they can be descriptive or explanatory. Descriptive designs focus on what is happening whereas explanatory designs can investigate causal processes or mechanisms (Shavelson & Towne, 2002).
Quantitative designs tend to test objective theories by looking at the relationships among variables that can be measured (Creswell & Creswell, 2017) but they can also be used for exploratory data analysis to generate and refine theories and hypotheses (Behrens, 1997). Experimental designs describe studies where researchers actively manipulate variables to examine causal relationships among variables (e.g., whether the use of particular kinds of scaffolds increases interaction; see Janssen & Kollar, this volume). Statistical tests would be used to determine if any difference between conditions was greater than might occur by chance. Experiments can be further classified as randomized or quasi-experimental (e.g., assignments to conditions are nonrandom as in assigning different classes to different conditions; Shadish, Cook, & Campbell, 2002; What Works Clearinghouse, 2017). Both randomized experimental or quasi-experimental designs allow causal inferences to be made. In contrast, pre-post designs look at change between a variable measured before an intervention and measured again after an intervention. In such designs, it is harder to draw causal conclusions about the effects of an intervention. In that sense, such designs as well as correlational studies are descriptive. Although descriptive quantitative research can show what happened and experimental designs can explain why, they may not be able to explain how the change occurred and/or what would happen when these variables covary simultaneously. Qualitative designs may be better suited to address such “how” questions about phenomena that emerge from a complex set of interactions.
Qualitative research designs involve emerging questions and are conducted in natural settings (Creswell & Creswell, 2017). These designs can be either descriptive or explanatory. They are interpretive and tend to involve emergent themes. Such designs include ethnographies and case studies (see also, Uttamchandani & Lester, this volume; Koschmann & Schwarz, this volume). There is often a focus on the meaning that participants bring to the setting as researchers try to construct holistic accounts. Such designs may focus on trying to explain the how and why in their account (so they might be explanatory) or they might be more descriptive. Descriptive designs are studies that aimed at providing an account of a phenomenon or intervention. Such studies seek to uncover regularities in the data without actively manipulating variables (Creswell & Creswell, 2017). Case studies, observational studies, and surveys are examples of descriptive designs. Case studies are detailed analyses of a program, event, or activity that are clearly bounded in some way. In contrast, ethnographic designs may seek to be more explanatory, showing how processes unfold as researchers study a group in a natural setting over an extended time frame. Many of these forms of research involve collecting rich forms of data such as interviews, observations, and artifacts.
Mixed methods research designs integrate qualitative and quantitative methods. An important research design for CSCL is design-based research (DBR). DBR methods are a research strategy in which theoretically-driven CSCL designs and interventions are enacted and which are refined progressively over several iterations (Brown, 1992; Collins, 1992; Sandoval, 2014). DBR refers to a research framework or strategy that can transcend the design of individual iterations (e.g., Zhang, Scardamalia, Reeve, & Messina, 2009). The goal of DBR is to design learning environments that succeed in natural settings and advance theories of learning and design of learning environments. This involves developing theory-driven designs of learning environments and iteratively refining both theory and design. In particular, DBR aims to understand what works for whom, how, and under what circumstances (Design-based Research Collective, 2003). Such programs of research may stretch over several years (e.g., Zhang et al., 2009). All of these research designs have philosophical underpinnings that are beyond the scope of this chapter (see Creswell & Creswell, 2017 for further details).
Of the studies examined in the corpus, descriptive (including qualitative) designs were most frequent (50%), followed by randomized experiments (25%), quasi-experiments (16%), DBR (7%), and pre-post designs (3%). These relative frequencies are generally consistent with the earlier results and demonstrate that CSCL continues to use a range of research designs. DBR is somewhat infrequent in this dataset. This likely signals difficulties and challenges associated with DBR and yet it is also not clear when and how a program of research might be represented as several separate articles rather than as a single program of design-based CSCL research. These difficulties may be due to both publication pressures and limitations to the length of articles.
An important characteristic of CSCL research studies is that they tend to be conducted “in the wild.” Classroom settings are formal learning situations that are guided by teachers. Laboratory settings are controlled settings where data collection is carried out outside the context of classrooms or other authentic learning situations. Other settings include CSCL outside laboratories or classrooms such as workplace, online communities, or informal learning environments (e.g., teacher workshops, professional conferences). In the corpus, 80% of the research was conducted in classroom settings (up from 74% in the earlier research). Of the classroom studies, 52% were descriptive, 20% were randomized experiments, 17% were quasi-experimental, and 8% were DBR. As anticipated, the DBR studies were almost exclusively set in classrooms. For example, Zhang et al. (2009) study used Knowledge Forum and examined how different ways of forming groups affected collective knowledge building over three iterations of instructions. As part of a design-based classroom study, Looi, Chen, and Ng (2010) examined the effectiveness of Group Scribbles (GS) in two Singapore science classrooms codesigned by teachers and found that the GS classroom performed better than the traditional classroom by traditional assessments. In an experimental classroom study, Borge, Ong, and Rosé (2018) compared the effects of two different individual reflective scripts on group regulation in threaded discussions as part of a course on information sciences and technology. It is clear that CSCL research focuses on ecologically valid settings that are oriented toward being useful.
3.3 Data Sources and Analysis in CSCL
To make sense of the messiness of the classroom context (Kolodner, 2004), CSCL researchers use different types and sources of data. For example, text, video, and log process data can reveal CSCL learning processes (Derry et al., 2010; Strijbos & Stahl, 2007). Outcomes are data about student or group performance, achievement, or other artifacts. Outcome data provide information about change in knowledge measured in a test or through artifacts that learners construct. There are also miscellaneous data providing evidence about noncognitive and/or situational aspects of CSCL such as questionnaires that assess perception and motivation of students, interviews, or researcher field notes. In our corpus, most use multiple data sources.
Analysis methods refer to the ways that CSCL researchers make sense of the data. CSCL research uses a variety of data ranging from video to synchronous and asynchronous messages. Process data as well as outcome data are collected. Analysis of such diverse range data requires application of quite different methodological approaches. The general categories of quantitative and qualitative analysis are often used to differentiate analysis methods, but each has several subcategories. Quantitative analyses are typically applied to analyze test, survey, or questionnaire data or other data in numeric forms. Code and count, often called verbal analysis or (quantitative) content analysis, are used to quantify qualitative data such as texts or dialogues. The outcome of the code and count analysis can then be subjected to inferential statistics or other more advanced quantitative analysis (Chi, 1997; Jeong, 2013; Neuendorf, 2002). Simple descriptive statistics are another commonly used approach and include common data reduction methods such as frequencies or means. Inferential statistics can be used to make inferences about group differences, whereas modeling refers to more complex analytic techniques such as multilevel analyses that seek to explain causal relationships among different variables. One challenge in doing quantitative analyses is that individuals within groups are not independent of each other which requires either using the group as the unit of analysis or multilevel modeling (Cress, 2008; Paulus & Wise, 2019). Note that the last three types of quantitative analyses are hierarchically related. Modeling statistics presumes the use of inferential statistics, which also presumes the use of descriptive statistics. When properly combined, they can serve as a useful tool to analyze a range of data collected in CSCL settings (see Chiu & Reimann, this volume).
Of the studies in the earlier corpus, 40% of the papers used two or more different analytic techniques. In reporting on the research, 88% used at least one quantitative analysis and 47% used at least one qualitative analysis. Of the quantitative analysis approaches used, inferential statistics were most common, followed by code and count and simple descriptive statistics. In some cases, elaborate statistical analyses were used. Hong et al. (2013) designed archaeology games around digital archives in Taiwanese museums and collected survey data from teams of high school game participants after they played the game. Here the structural equation modeling technique was used to test a model about the effects of gameplay self-efficacy on game performance and perceived ease of gameplay. Such sophisticated modeling techniques are not common which may be because small sample sizes are often a limiting factor in CSCL.
Not all data can be usefully quantified with code and count. Coding and counting the frequency of certain codes, even when it can be done, may not reveal what is going on during collaboration. A deeper and more holistic analytic approach is needed to analyze data sources such as interview data or field notes which were collected in 25% and 13% of the studies in the corpus, respectively. Within qualitative analyses (qualitative), content analysis refers to systematic text analysis (Mayring, 2000). Conversation or discourse analysis refers to analyses that analyze conversations or discourse, but can vary considerably in their approaches and techniques (e.g., Gee & Green, 1998; Uttamchandani & Lester, this volume). Grounded theory refers to qualitative analytic techniques that emphasize the discovery of theory based on the systematic analysis of data. Codes, concepts, and/or categories can be formed in the process of formulating a theory, but interpreted quite differently from the way they are used in the quantitative analysis (Straus & Corbin, 1990). Interaction analysis examines the details of social interaction as they occur in practice and generally relies on collaborative viewing of video (Derry et al., 2010; Jordan & Henderson, 1995). There are also several other established qualitative methods such as narrative analysis, thematic analysis, or phenomenography. Qualitative methods are not merely about analysis, but often refer to a whole approach to inquiry that prescribes research objective, design, data collection method, as well as analysis. Boundaries of different qualitative analyses are not always clear-cut. In many cases, the approach to qualitative analysis was what Jeong et al. (2014) referred to as loosely defined—that which did not appear to be linked to any specific analytic traditions.
For qualitative analyses, loosely defined qualitative analyses remained the most common, appearing in 25% of the studies (consistent with the earlier results). In addition, qualitative content analysis appeared in 12% of the papers; other well-defined techniques were reported in less than 5% of the papers each (e.g., interaction analysis, conversation analysis, and grounded theory, in decreasing order of frequency). The quality of “loosely defined” analyses was quite variable. Some studies used these to complement statistical analysis and as a tool to illustrate and explore the observed differences that were identified quantitatively (Schwarz & Glassner, 2007). In studies that received a loosely defined code, these analyses were often verbatim examples or other excerpts from data supporting the researchers’ observations and/or conclusions (e.g., Minocha, Petre, & Roberts, 2008). Another form of loosely defined qualitative analysis was qualitative summaries of data, often supplemented with simple descriptive statistics (Menkhoff, Chay, Bengtsson, Woodard, & Gan, 2015; Rick & Guzdial, 2006). Some of the limited specificity in the description of qualitative methods may be due to the limited journal space. We need to explore ways to report these qualitative findings while making the rigor of the research clear.
3.4 Mixing Methods in CSCL
Consistent with prior work, the results from this corpus suggest that CSCL research mixes analytical methods and multiple data sources. Although more than half of the papers used exclusively quantitative analysis and some only qualitative analysis, 35% used multiple analytic methods. These choices of analytic tools were related to the particular research designs as shown in Fig. 1. Mixed methods were most often used in descriptive research designs and quantitative methods only were mostly used in experimental designs. Design-based research studies were more likely to use mixed methods than a single method. Although the scope of the study was slightly different, Major et al. (2018) also found that 46% of the studies were quantitative studies but an almost equal number used mixed methods. One typical example of the mixed-method study is a study by Zhang, Tao, Chen, Sun, Judson, and Naqvi (2018) in which they used quantitative methods to compare the scientific sophistication of explanations and topics discussed between two different classroom interventions with the Idea Thread Mapper, a tool to help students and teachers organize and monitor their collective inquiry. Here, the inferential statistics allowed them to determine where there were differences. Qualitative methods allowed the researchers to see how the collective inquiry unfolded with the different instructional designs. Video analysis documented the temporal unfolding and elaboration of collaborative inquiry structures. In a study of CSCL in threaded discussions, Borge, Ong, and Rosé (2018) counted indicators related to regulation from the group discourse, used inferential statistics to compare across conditions, and used a qualitative case study to demonstrate how a group’s collaborative activity and regulation changed over time. In both of these examples, quantitative analysis allowed researchers to see what changes occurred, and qualitative analyses were used to examine how change unfolded.
Other ways of mixing methods do not always show up in this kind of analysis. For example, Suthers et al. (2013) brought together CSCL researchers from different methodological traditions to engage in what they called Productive Multivocality in the Analysis of Group Interactions. This series of workshops demonstrated how different methods might be applied to the same datasets and illuminate new understandings that the original researcher might not have considered (Suthers et al., 2013). For example, one dataset was about peer-led team learning in chemistry; the different methods applied to it included ethnographic analysis, two different multidimensional approaches to coding and counting, as well as a social network analysis. The application of different methods revealed tensions in how data need to be prepared for analysis as well as how to conceptualize a given learning event. Different analytic approaches often driven by different research questions contributed toward constructing a richer understanding of the events. The process of resolving differences led to deeper elaboration of the underlying collaborative mechanisms as well. These different approaches provided new insights on the construct of leadership in groups.
3.5 On the Relation Between Theory and Method
Although other chapters focus on theory (Stahl & Hakkarainen, this volume), this section refers to the diversity of theories and the specific relationship to research methods. Consistent with the earlier review in Jeong et al. (2014), analysis of the expanded corpus from 2005 to 2014 (McKeown et al., 2017) shows that CSCL uses diverse theoretical frameworks that include information processing, socio-cognitive, constructivist, sociocultural, communication, social psychology, motivation, and other theoretical frameworks. This aligns well with the argument in Wise and Schwarz (2017) that CSCL does have multiple explanatory frameworks. Some of these differences are related to the multidisciplinary nature of CSCL (Strijbos & Fischer, 2007).
Jeong, Hmelo-Silver, and Yu (2014) identified several clusters organized around theory and method. These clusters represent patterns of theoretical perspectives and research designs. One pattern identified is that socio-cultural frameworks tended to be used in qualitative studies, contextualized in classrooms, and with descriptive research designs (e.g., Ares, 2008; Berge & Fjuk, 2006). Another pattern used general constructivist perspectives with quasi-experimental classroom studies (e.g., Dori & Belcher, 2005; Van Drie, van Boxtel, Jaspers, & Kanselaar, 2005). Other patterns were more eclectic with multiple theoretical orientations guiding either descriptive classroom designs or experimental laboratory designs.
3.6 Challenges in CSCL Research Methods
There are many challenges in conducting research in CSCL environments. The technology is only a piece of a complex system of CSCL that also includes pedagogy and collaborative groups in particular contexts (Arnseth & Ludvigsen, 2006). In such environments, one challenge is identifying the appropriate unit of analysis. For studies arising from cognitive and socio-cognitive perspective, this might be the individual nested within a group but studies coming from a socio-cultural perspective, the group itself and the emergent dialog might be the appropriate unit of analysis (Janssen, Cress, Erkens, & Kirschner, 2013; Ludvigsen & Arnseth, 2017; Stahl, 2006). For others, the overall activity system would be the focus of analytic interest as in Danish’s (2014) study of young children learning about complex systems through collaborative engagement with a simulation in a carefully designed activity system. The curriculum unit was designed around four key activities and the analyses demonstrated how these activities organized students’ collective engagement with the target concepts.
Related to the challenge of identifying appropriate units of analysis is that of segmentation and coding. For example, in CSCL when chat and threaded discussions are used, reconstructing the response structure can be a challenge as it is not always clear what participants are responding to (Strijbos & Stahl, 2007). The ability to appropriately segment units for analysis also presents challenges in terms of reliability of coding.
The multidisciplinarity of CSCL creates additional challenges for the field. The field is not as cumulative as it might be because researchers tend to ignore results from methodologies unlike their own rather than looking to triangulate across studies with different kinds of methods (Strijbos & Fischer, 2007). The diversity of methods, and even hybrid methods, leads to a lack of standards for how research results are reported which makes the rigor of studies difficult to evaluate. This highlights the importance of documenting how methods have been adapted and combined so that other researchers can use these methods in the future. Developing standards for such research is what Kelly (2004) has called the argumentative grammar for evidentiary support needed to warrant claims. An argumentative grammar is a clear and explicitly stated logic of inquiry that provides the basis for the particular methods selected and how claims can be warranted. This remains a challenge because of the multiple theoretical frames and methodological tools used in CSCL research.
Another important challenge in CSCL and much learning sciences research is the challenge of incorporating situational and contextual factors into analysis of CSCL. Arvaja, Salovaara, Häkkinen, and Järvelä (2007) combined individual and group level perspectives as a way to account for context but this remains one of the tensions between qualitative and quantitative research methods. Qualitative methods tend to be excellent at providing rich descriptions of contexts but are also focused very much on the particular. Mixed methods become especially important in accounting for different aspects of cognition, learning, and the learning contexts. Nonetheless, many current approaches to accounting for context are highly labor-intensive and require many person-hours for data collection, management, and analysis, whether it is video data sources or thousands of lines of chat data when trying to understand processes in CSCL settings.
4 The Future: Addressing Challenges and New Horizons
This corpus and literature analyzed here present recent methodological trends in CSCL but it has not necessarily accounted well for certain aspects of CSCL research. In this last section, we highlight several important trends in methods for analyzing CSCL: temporality, visualizations, automation, and units of analysis. These trends address challenges related to the messy, contextualized, labor-intensive, and collaborative nature of CSCL research.
One way of dealing with the situated nature of learning in CSCL is for the research methods to be able to account for the temporal dimension. CSCL researchers have the opportunities to study learning processes that unfold over extended periods of time (Reimann, 2009). A number of researchers have argued that this dimension needs to be addressed explicitly in CSCL research (Järvelä, Malmberg, & Koivuniemi, 2016; Kapur, 2010; Reimann, 2009; Suthers, Dwyer, Medina, & Vatrapu, 2010; Zhang et al., 2018). To accomplish that, Reimann (2009) has argued for events being considered as a central unit of analysis in which entities participate. Reimann notes the importance of formalizing methods of analyzing these kinds of events. Suthers et al. (2010) argue for interactions and uptake as a way of capturing temporal relationships. Uptake is defined as “the relationship present when a participant’s coordination takes aspects of prior or ongoing events as having relevance for an ongoing activity” (Suthers et al., 2013, p. 13). This latter approach considers the importance of contingencies between events, which can include both temporal and spatial coordination across actors and media. Approaches to dealing the temporal nature of CSCL can take highly sophisticated statistical forms (e.g., Chiu, 2018; Csanadi, Egan, Kollar, Shaffer, & Fischer, 2018; Reimann, 2009; Kapur, 2010) as well as more qualitative approaches. For example, quantitative approaches such as sequential data analysis, can analyze the probabilities of how likely one event is to be followed by another. In CSCL, these events could be different kinds of dialogue moves, such as argumentation (Jeong, Clark, Sampson, & Menekse, 2011). To model dynamic collaboration processes, statistical discourse analysis (SDA, Chiu, 2018) estimates the likelihood of particular discourse moves during each turn of talk or online message and the influences of explanatory factors at multiple levels (e.g., individual, group, class). SDA detects pivotal moments that substantially change an interaction and models factors affecting the likelihood of these moments (Chiu, Molenaar, Chen, Wise & Fujita, 2013). The analyses of these temporal dimensions of CSCL can help identify events critical to idea development or failures in social regulation, which in turn can help teachers and designers to determine when and perhaps how to intervene (Oshima, Oshima, & Fujita, 2018).
Visualizations can provide both temporal and relational information that could aid in qualitative interpretation of complex CSCL data that go beyond coding and counting utterances (Csanadi et al., 2018; Hmelo-Silver, Liu, & Jordan, 2009; Suthers et al., 2010). This helps address the challenge of dealing with the rich contextual information found in CSCL environments. One way of integrating across different kinds of data and multidimensional coding schemes is through the use of visualizations. Visualizations take advantage of human perceptual capabilities as they allow one to view and search a large amount of information in a single glance (Larkin & Simon, 1987). They can support perceptual inference and pattern recognition. When computer tools are used to create these visualizations, they can be used to manipulate the representations created, allowing an analyst to examine different parts of interactions, zooming in and zooming out as needed (Hmelo-Silver, Liu, & Jordan, 2009; Huang et al., 2018; Howley, Kumar, Mayfield, Dyke, & Rosé, 2013). For example, Huang et al. used CORDTRA (Chronologically-oriented Representations of Discourse and Tool-Related Activity) diagrams to analyze collaborative modeling practices in a citizen science community, showing their tool-mediated interaction on a longer time scale (a working session) and then used it to zoom into a 5-min excerpt of interest when the modeling tool was particularly salient as a boundary object to support collaboration. Howley et al. (2013) used TATIANA’s sliding window to look at how distributions of discourse codes changed over time. Suthers et al. (2010) constructed contingency graphs that demonstrate how interaction is distributed among participants and tools over time to support their framework for uptake analysis. More recently, Csanadi et al. (2018) have introduced epistemic network analysis to visualize and quantify temporal co-occurrence of codes. This latter approach is unique in allowing statistical comparisons across networks.
Many of the approaches described in this chapter require intensive work to code verbal discourse of observational data, online interactions, and patterns of activity from a variety of sources (Law, Yuen, Wong, & Leng, 2011). Work in the future of CSCL will include automated approaches to data analysis and multimodal learning analytics (see Schneider, Worsley, & Martinez-Maldonado, this volume; Wise, Knight, & Buckingham Shum, this volume). Law, Yuen, Wong, and Leng (2011) developed automated coding and visualization to focus on indicators of high-quality asynchronous discussions in Knowledge Forum. Howley and Rosé (2016) have used automated systemic functional linguistic analysis of discussions to illuminate social dimensions of interaction. Nistor et al. (2015) used automated dialogue analysis to identify clusters of central and peripheral participation in virtual communities of practice. Moreover, they found substantial correlation between automated and hand coding of the discourse data. In addition to making sense of discourse and computer-generated data, multimodal learning analytics can be used to make sense of CSCL processes that go beyond what can be captured by a computer (Blikstein & Worsley, 2016; Noroozi, Alikhani, Järvelä, Kirschner, Juuso, & Seppänen, 2019). Blikstein and Worsley noted that in addition to text analysis, similar to the work on automated discourse coding, there are technological capabilities for analysis of speech, handwriting, gesture, and movement. In addition, sensors detecting different physiological markers can be used to infer affective states (e.g., Malmberg et al., 2019). Eye-tracking data can be used to track what learners are attending to and strategies that they are using. Indeed Blikstein and Worsley (2016) argue that the most promising use of gaze data is in small groups to track joint visual attention. Noroozi et al. (2019) note the importance of coordinating multimodal data with video to support studying such complex phenomena as socially shared regulation. Buckingham-Shum and Ferguson (2012) argue that opportunities for designing learning analytics that involves social aspects of learning are particularly timely, challenging, and need to consider the ethical as well as technical issues.
5 Conclusion
The methodological landscape of CSCL is complex, and like other aspects of the learning sciences often uses multiple methodological approaches and techniques (Major et al., 2018; Yoon & Hmelo-Silver, 2017). This chapter builds on the review of Jeong et al. (2014) with an update of an additional 5 years of literature. This suggests that the research methods used in the field are stable overall; however, some newer techniques have been introduced in the time since the corpus reported here was generated. These multiple research methods in CSCL are consistent with a survey of learning scientists that demonstrated the learning sciences researchers are involved in research that uses a broad range of methods (Yoon & Hmelo-Silver, 2017). It was surprising to not see design-based research more prominently featured in the sample given that it is the signature methodology taught in many programs in the learning sciences (Sommerhoff, Szameitat, Vogel, Chernikova, Loderer, & Fischer, 2018). Although many challenges remain, CSCL researchers are developing new ways of mixing methods and new techniques to help deal with the messy real-world contexts that characterize research in the learning sciences more generally (e.g., Brown, 1992; Kolodner, 2004).
Although this review covers a broad range of CSCL literature, it primarily focuses on STEM domains and education and the years covered in the systematic review. It is important to learn more about the generality of these research methods across other different disciplinary contexts. This review also only focuses on research published in peer-reviewed journals, which may miss some current trends in research methods represented in CSCL conference proceedings. However, given the limited space in such proceedings, it would be challenging to extract the necessary methodological features from studies reported in such venues.
In general, CSCL tends to ask a range of research questions, but these are often organized around specific technologies/and or pedagogical interventions. Although most studies have clear theoretical orientation, there were few studies that engaged explicitly in theory development and testing. This provides challenges in building a cumulative science of CSCL. Encouraging however is the range of data sources and analytic tools being used to address these questions. Although CSCL has many challenges in terms of being resource-intensive to analyze and complex to understand, CSCL research methods are an active topic for reflection and research within this research community. These reflective discussions acknowledge the challenges and provide impetus for the development and appropriation of new methods for understanding learning in CSCL environments.
References
Ares, N. (2008). Cultural practices in networked classroom learning environments. International Journal of Computer-Supported Collaborative Learning, 3, 301–326.
Arnseth, H. C., & Ludvigsen, S. (2006). Approaching institutional contexts: Systemic versus dialogic research in CSCL. International Journal of Computer-Supported Collaborative Learning, 1(2), 167–185. https://doi.org/10.1007/s11412-006-8874-3.
Arvaja, M., Salovaara, H., Häkkinen, P., & Järvelä, S. (2007). Combining individual and group-level perspectives for studying collaborative knowledge construction in context. Learning and Instruction, 17, 448–459.
Behrens, J. T. (1997). Principles and procedures of exploratory data analysis. Psychological Methods, 2(2), 131.
Berge, O., & Fjuk, A. (2006). Understanding the roles of online meetings in a net-based course. Journal of Computer Assisted Learning, 22, 13–23.
Blikstein, P., & Worsley, M. (2016). Multimodal learning analytics and education data mining: Using computational technologies to measure complex learning tasks. Journal of Learning Analytics, 3(2), 220–238.
Borge, M., Ong, Y., & Rosé, C. P. (2018). Learning to monitor and regulate collective thinking processes. International Journal of Computer-Supported Collaborative Learning, 13, 61–92.
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2, 141–178.
Buckingham-Shum, S., & Ferguson, R. (2012). Social learning analytics. Educational Technology & Society, 15(3), 3–26.
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6, 271–315.
Chiu, M. M. (2018). Statistically modelling effects of dynamic processes on outcomes: An example of discourse sequences and group solutions. Journal of Learning Analytics, 5, 75–91.
Chiu, M. M., Molenaar, I., Chen, G., Wise, A. F., & Fujita, N. (2013). Micro-analysis of collaborative processes that facilitate productive online discussions: Statistical discourse analyses of three cases. In M. Clara & E. B. Gregori (Eds.), Assessment and evaluation of time factors in online teaching and learning (pp. 232–263). Hershey, PA: IGI Global.
Chiu, M. M., & Reimann, P. (this volume). Statistical and stochastic analysis of sequence data. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning. Cham: Springer.
Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology (pp. 15–22). Berlin, Heidelberg: Springer.
Cress, U. (2008). The need for considering multilevel analysis in CSCL research—An appeal for the use of more advanced statistical methods. International Journal of Computer Supported Collaborative Learning, 3, 69–84.
Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage.
Csanadi, A., Eagan, B., Kollar, I., Shaffer, D. W., & Fischer, F. (2018). When coding-and-counting is not enough: Using epistemic network analysis (ENA) to analyze verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13(4), 419–438.
Danish, J. A. (2014). Applying an activity theory lens to designing instruction for learning about the structure, behavior, and function of a honeybee systems. Journal of the Learning Sciences, 23, 100–148.
Derry, S. J., Pea, R. D., Barron, B., Engle, R. A., Erickson, F., Goldman, R., Hall, R., Koschmann, T., Lemke, J. L., Sherin, M. G., & Sherin, B. L. (2010). Conducting video research in the learning sciences: Guidance on selection, analysis, technology, and ethics. Journal of the Learning Sciences, 19, 3–53.
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8.
Dori, Y. J., & Belcher, J. (2005). How does technology-enabled active learning affect undergraduate students’ understanding of electromagnetism concepts? Journal of the Learning Sciences, 14, 243–279.
Gee, J. P., & Green, J. L. (1998). Discourse analysis, learning, and school practice: A methodological study. Review of Research in Education, 23, 119–169.
Hmelo-Silver, C. E., Liu, L., & Jordan, R. (2009). Visual representation of a multidimensional coding scheme for understanding technology-mediated learning about complex natural systems. Research and Practice in Technology Enhanced Learning Environments, 4, 253–280.
Hong, J. C., Hwang, M. Y., Chen, Y. J., Lin, P. H., Huang, Y. T., Cheng, H. Y., & Lee, C. C. (2013). Using the saliency-based model to design a digital archaeological game to motivate players’ intention to visit the digital archives of Taiwan’s natural science museum. Computers & Education, 66, 74–82.
Howley, I., Kumar, R., Mayfield, E., Dyke, G., & Rosé, C. P. (2013). Gaining insights from sociolinguistic style analysis for redesign of conversational agent based support for collaborative learning. In D. D. Suthers, K. Lund, C. P. Rosé, C. Teplovs, & N. Law (Eds.), Productive multivocality in the analysis of group interactions (pp. 477–494). New York: Springer.
Howley, I. K., & Rosé, C. P. (2016). Towards careful practices for automated linguistic analysis of group learning. Journal of Learning Analytics, 3(3), 239–262.
Huang, J., Hmelo-Silver, C. E., Jordan, R., Gray, S., Frensley, T., Newman, G., & Stern, M. (2018). Scientific discourse of citizen scientists: Objects for collaborative problem solving. Computers in Human Behavior, 87, 480–492.
Janssen, J., Cress, U., Erkens, G., & Kirschner, P. A. (2013). Multilevel analysis for the analysis of collaborative learning. In C. E. Hmelo-Silver, C. Chinn, C. K. K. Chan, & A. M. O’Donnell (Eds.), International handbook of collaborative learning (pp. 112–125). New York: Routledge.
Janssen, J., & Kollar, I. (this volume). Experimental and quasi-experimental research in CSCL. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning. Cham: Springer.
Järvelä, S., Malmberg, J., & Koivuniemi, M. (2016). Recognizing socially shared regulation by using the temporal sequences of online chat and logs in CSCL. Learning and Instruction, 42, 1–11.
Jeong, A., Clark, D. B., Sampson, V. D., & Menekse, M. (2011). Sequential analysis of scientific argumentation in asynchronous online discussion. In S. Puntambekar, G. Erkens, & C. E. Hmelo-Silver (Eds.), Analyzing interactions in CSCL (pp. 207–232). New York: Springer.
Jeong, H. (2013). Verbal data analysis for understanding interactions. In C. E. Hmelo-Silver, C. K. K. Chan, C. Chinn, & A. M. O’Donnell (Eds.), International handbook of collaborative learning (pp. 168–181). New York: Routledge.
Jeong, H., & Hmelo-Silver, C. (2012). Technology supports in CSCL. In J. van Aalst, K. Thompson, M. J. Jacobson, & P. Reimann (Eds.), The future of learning: Proceedings of the 10th international conference of the learning sciences (ICLS 2012)—Volume 1, full papers (pp. 339–346). Sydney: ISLS.
Jeong, H., & Hmelo-Silver, C. E. (2016). Seven affordances of CSCL technology: How can technology support collaborative learning. Educational Psychologist, 51, 247–265.
Jeong, H., Hmelo-Silver, C. E., & Yu, Y. (2014). An examination of CSCL methodological practices and the influence of theoretical frameworks 2005–2009. International Journal of Computer-Supported Collaborative Learning, 9(3), 305–334.
Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. Journal of the Learning Sciences, 4, 39–103.
Kapur, M. (2010). Temporality matters: Advancing a method for analyzing problem-solving processes in a computer-supported collaborative environment. International Journal of Computer-Supported Collaborative Learning, 6, 39–56.
Kelly, A. (2004). Design research in education: Yes, but is it methodological? Journal of the Learning Sciences, 13(1), 115–128.
Kelly, G. J. (2006). Epistemology and educational research. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Handbook of complementary methods in education research (pp. 33–56). Washington, DC: American Educational Research Association.
Kirschner, P. A., & Erkens, G. (2013). Toward a framework for CSCL research. Educational Psychologist, 48(1), 1–8.
Kolodner, J. L. (2004). The learning sciences: Past, present, future. Educational Technology, 44(3), 34–40.
Koschmann, T., & Schwarz, B. B. (this volume). Case studies in theory and practice. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning. Cham: Springer.
Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65–99.
Law, N., Yuen, J., Wong, W. O. W., & Leng, J. (2011). Understanding learners’ knowledge building trajectory through visualizations of multiple automated analyses. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methods, approaches and issues (pp. 47–82). Boston, MA: Springer.
Looi, C.-K., Chen, W., & Ng, F.-K. (2010). Collaborative activities enabled by GroupScribbles (GS): An exploratory study of learning effectiveness. Computers & Education, 54, 14–26.
Ludvigsen, S., & Arnseth, H. C. (2017). Computer-supported collaborative learning. In E. Duval, M. Sharples, & R. Sutherland (Eds.), Technology enhanced learning: Research themes (pp. 47–58). New York: Springer International Publishing.
Major, L., Warwick, P., Rasmussen, I., Ludvigsen, S., & Cook, V. (2018). Classroom dialogue and digital technologies: A scoping review. Education and Information Technologies, 23(5), 1995–2028.
Malmberg, J., Järvelä, S., Holappa, J., Haataja, E., Huang, X., & Siipo, A. (2019). Going beyond what is visible: What multichannel data can reveal about interaction in the context of collaborative learning? Computers in Human Behavior, 96, 235–245.
Mayring, P. (2000). Qualitative content analysis. Forum: Qualitative Social Research, 1(2). Art. 20. Retrieved from http://nbn-resolving.de/urn:nbn:de:0114-fqs0002204.
McKeown, J., Hmelo-Silver, C. E., Jeong, H., Hartley, K., Faulkner, R., & Emmanuel, N. (2017). A meta-synthesis of CSCL literature in STEM education. In B. K. Smith, M. Borge, E. Mercier, & K.-Y. Lim (Eds.), Proceedings of CSCL 2017. Philadelphia, PA: International Society of the Learning Sciences.
Menkhoff, T., Chay, Y. W., Bengtsson, M. L., Woodard, C. J., & Gan, B. (2015). Incorporating microblogging (“tweeting”) in higher education: Lessons learnt in a knowledge management course. Computers in Human Behavior, 51, 1295–1302.
Minocha, S., Petre, M., & Roberts, D. (2008). Using wikis to simulate distributed requirements development in a software engineering course. International Journal of Engineering Education, 24(4), 689–704.
Neuendorf, K. A. (2002). The content analysis: Guidebook. London: Thousand Oaks.
Nistor, N., Trăuşan-Matu, S., Dascălu, M., Duttweiler, H., Chiru, C., Baltes, B., & Smeaton, G. (2015). Finding student-centered open learning environments on the internet: Automated dialogue assessment in academic virtual communities of practice. Computers in Human Behavior, 47, 119–127.
Noroozi, O., Alikhani, I., Järvelä, S., Kirschner, P. A., Juuso, I., & Seppänen, T. (2019). Multimodal data to design visual learning analytics for understanding regulation of learning. Computers in Human Behavior, 100, 298–304.
Onwuegbuzie, A. J., & Leech, N. L. (2006). Linking research questions to mixed methods data analysis procedures. The Qualitative Report, 11(3), 474–498.
Oshima, J., Oshima, R., & Fujita, W. (2018). A mixed-methods approach to analyze shared epistemic agency in jigsaw instruction at multiple scales of temporality. Journal of Learning Analytics, 5(1), 10–24.
Paulus, T. M., & Wise, A. F. (2019). Looking for insight, transformation, and learning in online talk. New York: Routledge.
Puntambekar, S., Erkens, G., & Hmelo-Silver, C. E. (Eds.). (2011). Analyzing interactions in CSCL: Methodology, approaches, and issues. New York: Springer.
Puntambeker, S. & Luckin, R. (2002). Documenting collaborative interactions: Issues and approaches workshop. CSCL 2002. Boulder, CO.
Reimann, P. (2009). Time is precious: Variable- and event-centred approaches to process analysis in CSCL research. International Journal of Computer Supported Collaborative Learning, 4, 239–257.
Rick, M., & Guzdial, M. (2006). Situating CoWeb: A scholarship of application. International Journal of Computer-Supported Collaborative Learning, 1, 89–115.
Sandoval, W. A. (2014). Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences, 23, 18–36.
Schneider, B., Worsley, M., & Martinez-Maldonado, R. (this volume). Gesture and gaze: Multimodal data in dyadic interactions. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning. Cham: Springer.
Schwarz, B. B., & Glassner, A. (2007). The role of floor control and of ontology in argumentative activities with discussion-based tools. International Journal of Computer-Supported Collaborative Learning, 2, 449–478.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. New York: Houghton Mifflin Company.
Shavelson, R. J., & Towne, L. (2002). Scientific research in education. Washington, DC: National Academy Press.
Sommerhoff, D., Szameitat, A., Vogel, F., Chernikova, O., Loderer, K., & Fischer, F. (2018). What do we teach when we teach the learning sciences? A document analysis of 75 graduate programs. Journal of the Learning Sciences, 27, 319–351. https://doi.org/10.1080/10508406.2018.1440353.
Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. Cambridge MA: MIT Press.
Stahl, G., & Hakkarainen, K. (this volume). Theories of CSCL. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning. Cham: Springer.
Straus, A., & Corbin, J. (1990). Basics of qualitative research. Newbury Park: Sage.
Strijbos, J., & Fischer, F. (2007). Methodological challenges for collaborative learning research. Learning and Instruction, 17, 389–393. https://doi.org/10.1016/j.learninstruc.2007.03.004.
Strijbos, J.-W., & Stahl, G. (2007). Methodological issues in developing a multi-dimensional coding procedure for small-group chat communication. Learning and Instruction, 17, 394–404.
Suthers, D. D., Lund, K., Rosé, C. P., & Teplovs, C. (2013). Achieving productive multivocality in the analysis of group interactions. In D. D. Suthers, K. Lund, C. P. Rosé, C. Teplovs, & N. Law (Eds.), Productive multivocality in the analysis of group interactions (pp. 577–612). Springer US. https://doi.org/10.1007/978-1-4614-8960-3_31
Suthers, D., Lund, K., Rosé, C. P., Teplovs, C., & Law, N. (2013). Productive multivocality in the analysis of group interactions. New York: Springer.
Suthers, D. D. (2006). Technology affordances for intersubjective meaning making. International Journal of Computer Supported Collaborative Learning, 1, 315–337.
Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R. (2010). A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer-Supported Collaborative Learning, 5(1), 5–42.
Suthers, D. D., Rosé, C. P., Lund, K., & Teplovs, C. (2013). A Reader’s guide to the productive multivocality project. In D. D. Suthers, C. P. Rosé, K. Lund, C. Teplovs, & N. Law (Eds.), Productive multivocality in the analysis of group interactions (pp. 37–59). New York: Springer.
Uttamchandani, S., & Lester, J. N. (this volume). Qualitative approaches to language in CSCL. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning. Cham: Springer.
Valcke, M., & Martens, R. (2006). The problem arena of researching computer supported collaborative learning: Introduction to the special section. Computers & Education, 46, 1–5.
Van Drie, J., van Boxtel, C., Jaspers, J., & Kanselaar, G. (2005). Effect of representational guidance on domain specific reasoning in CSCL. Computers in Human Behavior, 21, 575–602.
What Works Clearinghouse. (2017). Standards handbook version 4.0. Retrieved March 5, 2019, from https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_standards_handbook_v4.pd
Wise, A. F., Knight, S., & Buckingham Shum, S. (this volume). Collaborative learning analytics. In U. Cress, C. Rosé, A. F. Wise, & J. Oshima (Eds.), International handbook of computer-supported collaborative learning. Cham: Springer.
Wise, A. F., & Schwarz, B. B. (2017). Visions of CSCL: Eight provocations for the future of the field. International Journal of Computer-Supported Collaborative Learning, 12, 423–467.
Yoon, S., & Hmelo-Silver, C. E. (2017). What do learning scientists do? Journal of the Learning Sciences, 26, 167–183.
Zhang, J., Scardamalia, M., Reeve, R., & Messina, R. (2009). Designs for collective cognitive responsibility in knowledge building communities. Journal of the Learning Sciences, 18, 7–44.
Zhang, J., Tao, D., Chen, M.-H., Sun, Y., Judson, D., & Naqvi, S. (2018). Co-organizing the collective journey of inquiry with idea thread mapper. Journal of the Learning Sciences, 27(3), 390–430.
Further Readings
Arnseth, H. C., & Ludvigsen, S. (2006). Approaching institutional contexts: Systemic versus dialogic research in CSCL. International Journal of Computer-Supported Collaborative Learning, 1(2), 167–185. This paper contrasts systemic and dialogic approach in CSCL research. Systemic approach aims at generating and testing models of the system. Dialogic approach on the other hand focuses on understanding how meaning is constituted in social practice. The paper argues that differences in their analytical practices have consequences for the generation and assessment of findings.
Jeong, H., Hmelo-Silver, C. E., & Yu, Y. (2014). An examination of CSCL methodological practices and the influence of theoretical frameworks 2005–2009. International Journal of Computer-Supported Collaborative Learning, 9(3), 305–334. This study reports on a content meta-analysis of CSCL research methods that this chapter builds on. It reports in detail on the methodology for doing the systematic review of the CSCL literature as well as how the papers were selected and screened.
Major, L., Warwick, P., Rasmussen, I., Ludvigsen, S., & Cook, V. (2018). Classroom dialogue and digital technologies: A scoping review. Education and Information Technologies, 23(5), 1995–2028. This article is a systematic review of the relation between classroom dialogue and educational technology. It considers the nature of the evidence related to how digital technologies mediate productive classroom discourse and the nature of the questions that different studies ask.
Reimann, P. (2009). Time is precious: Variable- and event-centred approaches to process analysis in CSCL research. International Journal of Computer Supported Collaborative Learning, 4, 239–257. This article argues for the importance of considering temporality in CSCL research and provides suggestions for addressing these considerations.
Suthers, D., Lund, K., Rosé, C. P., Teplovs, C., & Law, N. (2013). Productive multivocality in the analysis of group interactions. New York: Springer. This volume provides empirical examples of how different methodologies can be used on the same datasets, foster cross-disciplinary discussions, and the development of new insights on CSCL.
Acknowledgments
This research was funded by the National Science Foundation under Grant DRL # 1439227.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Hmelo-Silver, C.E., Jeong, H. (2021). An Overview of CSCL Methods. In: Cress, U., Rosé, C., Wise, A.F., Oshima, J. (eds) International Handbook of Computer-Supported Collaborative Learning. Computer-Supported Collaborative Learning Series, vol 19. Springer, Cham. https://doi.org/10.1007/978-3-030-65291-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-65291-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65290-6
Online ISBN: 978-3-030-65291-3
eBook Packages: EducationEducation (R0)