Introduction

As the impact of the pandemic has changed the landscape of economy, society, and culture, it also made drastic changes in education, such as tentatively closing schools and transforming modes of instruction. Under these circumstances, online learning is considered a necessity or even a panacea for the crisis (Dhawan, 2020). The societal need for distance learning is bringing more attention to educational practice and research situated in online settings (e.g., Shin & Hickey, 2021). One of the challenges that instructional designers and teachers encounter in the transition from face-to-face to online instruction is ensuring that social interactions take place (Rannastu-Avalos & Siiman, 2020). Although the benefits and importance of online collaborative learning are well established in the literature (e.g., Graham & Misanchuk, 2004), collaborative learning is accompanied by the possibilities of social loafing and requiring more resources for group coordination in task management, thus increasing cognitive demands (Janssen et al., 2010; Järvelä & Hadwin, 2013). One viable option to address these challenges is to implement asynchronous collaboration, which does not require real-time interactions as synchronous collaboration does. Asynchronous online communication has advantages in allowing learners flexibility for their learning, more time for reflections that lead to cogitated problem-solving (Hron & Friedrich, 2003), and encouragement for more explicit, concrete output (Schellens & Valcke, 2006). However, an asynchronous online environment itself does not guarantee the generation of critical thinking and knowledge building among students (Abawajy, 2012). Therefore, task structure should be deliberately designed for advanced learning outcomes, for which technological aids could play a critical role (So, 2009).

One of the frequently employed activities to foster knowledge building and social interactions in online collaborative learning is collaborative argumentation, which is often coupled with asynchronous communications. Research indicates that collaborative argumentation enables in-depth knowledge construction in groups as it engages learners in a variety of reasoning processes for interpreting claims or evidence made by others, producing arguments, and refining meanings (Andriessen et al., 2003). Nevertheless, previous studies have shown that collaborative argumentations inherently encompass various viewpoints and a large amount of information to process, which tends to make learners overladen and challenged in constructing a common ground (Kirschner et al., 2008). Therefore, collaborative argumentations situated in asynchronous online environments need additional instructional support to overcome the latent concerns of asynchronous communications and argumentation. For example, groupware that visualizes the structure or main arguments of a discussion could assist learners in enhancing individual cognitive processes and constructing an integrated knowledge set collaboratively.

Online groupware and representational systems

Groupware enables learners to be aware of the other members with whom they are interacting in the “shared environment” and who are involved in “a common task” (Ellis et al., 1991, p 40). Ellis et al. (1991) identified types of groupware in terms of application functionality. Of their terminology, group decision support systems facilitate group communication and collaboration when exploring a shared problem due to their distinct structures designed to enhance idea generation and problem analysis (Ellis et al., 1991). Kirschner and Erkens (2013) have suggested a framework for computer-mediated tools or instructional support that assists online collaborative learning. Among their taxonomy of pedagogical measures, which include interactive, representational, and guiding tools, representational measures provide structure with which learners can organize pieces of information or scaffolding to help them track the status of common tasks or interplays between group members. Examples of representational tools include argument maps, tables, or voting systems (Kirschner & Erkens, 2013). Suthers and Hundhausen (2003) determined three roles shared representational systems in group learning processes perform: collaboratively constructed representations (1) provide a ground for meaning negotiations in a group, (2) serve as a reference or a proxy that users can point by gestural deixis, and (3) remind previously mentioned ideas by which users can share a collective memory.

A series of recent studies indicate that the type of representational notation used shapes how learners recognize information and produces different effects on individual learning outcomes and the quality of collective works. For example, Suthers and Hundhausen (2003) contrasted the effects of three representational guides, a map, a matrix, and text, on undergraduate students’ understanding of scientific concepts and their application to scientific inquiry. The results showed that the type of representational system influenced individual learning outcomes and collaborative inquiry. During the individual learning session, the students who used a matrix representational aid most extensively addressed possible evidential relations but got distracted by insignificant ideas that were frequently revisited during collaboration. The graph users, on the other hand, came up with the fewest scientific ideas during individual learning, but used most of those ideas productively when they engaged in collaborative writing after the learning session (Suthers & Hundhausen, 2003).

Kwon and Park (2017) presented a social and cognitive diagram to facilitate asynchronous online discussions in a graduate-level course. They found that students expressed more disagreements when given cognitive diagrams. On the other hand, with social diagrams, they tended to agree more with their peers’ opinions. Another study by Ouyang et al. (2021) revealed results that appear consistent with prior research. In their study comparing graduate students’ social and cognitive engagements with three network visualizations, they found that the social network visualization increased students’ responsiveness, while the topic network promoted the number of perspective expressions and the cognitive network increased information-seeking (Ouyang et al., 2021). This line of research indicates that representational systems have different affordances in cognitive and social engagement in online collaborative learning, which accordingly affects individual information processing and the quality of collective knowledge building.

Graphic organizers are one of the representational systems, visually displaying thought processes or relations of ideas and evidence. There exists a considerable body of literature on the pedagogical effects of using graphic organizers, which shows that they are conducive for individual learning. Research indicates that graphic organizers help learners comprehend texts more easily (Gardill & Jitendra, 1999) and construct an integrated set of knowledge (Nussbaum & Schraw, 2007; Robinson & Kiewra, 1995). For example, Nussbaum and Schraw (2007) found that undergraduate students who used a graphic organizer in writing reflective opinion essays demonstrated more strategies for argument integrations such as evaluating and synthesizing.

Yet, there is scant research on representational artifacts that are both (1) developed by learners instead of instructors or automated tools, and (2) collaboratively constructed by more than one learner. Most research on student-developed representations is centered on workflow coordination systems that are used for individual writing or the preliminary process before engaging in the actual interactive collaboration process (e.g., Mochizuki et al., 2019; Suthers & Hundhausen, 2003). One of the few exceptions is an experimental study conducted by Kwon et al. (2018) where they contrasted the effects of instructor-provided and student-generated group graphic organizers. According to Kwon et al.’s (2018) findings, although these two graphic organizers did not vary in terms of knowledge construction levels, student-generated graphic organizers included fewer topic clusters yet more nodes per cluster. This suggests when students are creating a group graphic organizer rather than receiving one, they tend to selectively tune in a subset of potential topics. Drawing on the activity theory, the authors maintained that the act of developing a graphic organizer based on the previously shared ideas solidified the links between similar nodes and allowed learners to focus on fewer topics (Kwon et al., 2018). It is implied that having students generate graphic organizers themselves has distinct pedagogical implications beyond receiving and making sense of them. Nevertheless, as previously stated, few studies have focused on how different types of student graphic organizers affect the scope and depth of online discussion. To fill this literature gap, this study aims at finding the affordances of distinct graphic organizers collaboratively constructed by learners, on their social, cognitive, and affective engagements in online collaborative argumentation. To that end, the researchers designed an online curriculum that involved asynchronous discussion activities with the use of three graphic organizers as follows.

Three types of graphic organizers

The three graphic organizers addressed in this study are t-charts, tree charts, and maps. A t-chart allows for aligning main reasons, facts, and examples into one of two sides (advantages vs. disadvantages or pros vs. cons). Participants add their reasons or examples to strengthen an existing view or present an opposing view to counter them. A t-chart helps a group review what has been suggested on each side and affords salient contrast of the two sides, allowing students to discover the main structure of the argumentation. T-charts have the simplest structure of the three graphic organizers used in this study, so they might require users to elaborate more with text to compensate for the lack of classification and spatial information. Lumping different topics and ideas together by one criterion might be a potential downside of using t-charts. This makes users exert additional efforts to recognize ideas or to compare arguments side-by-side in terms of a subtopic, which may decrease the heuristic adequacy of the representation (Andriessen et al., 2003). The t-charts were constructed in Google Docs (Fig. 1).

Fig. 1
figure 1

An example of a student-authored pros/cons T-chart in Google Docs

Full size image

The second graphic organizer is a tree chart, and it offers ground to decompose a problem into topics and elaborate on corresponding examples, possible influences, or any other relevant supporting arguments. A topic is an overarching element that encompasses its subtopics and provides multiple viewpoints to a subject. A tree chart demonstrates different points grouped under several topics, regardless of whether they were made for or against the claim. Therefore, it assists in identifying structures or hierarchies of topics. Users can list their ideas in the existing column or create a new column by adding another topic. The topical tree charts were made in Padlet (Fig. 2).

Fig. 2
figure 2

An example of a student-authored topical tree chart in Padlet

Full size image

Lastly, the third graphic organizer is a map that visualizes the relationships between claims. It shows how different ideas are associated (e.g., supporting, opposing, or exemplifying other ideas). Maps help learners recognize the connections between varied ideas in discussions, explore new topics in consideration of existing ideas, and generate more ideas. The maps were created within Bubbl (Fig. 3). The types of graphic organizers employed in this study are summarized in Table 1 in tandem with their target structures and authoring tools.

Fig. 3
figure 3

An example of a student-authored map in Bubbl

Full size image
Table 1 Three types of graphic organizers and their target structures
Full size table

Research questions

We aimed to examine the effects of graphic organizers in online collaborative argumentation by answering the following research questions.

R1:

Do the different types of graphic organizers affect learners’ participation and interactions in asynchronous online discussions?

R2:

Do the different types of graphic organizers affect the quality of collaborative knowledge construction in asynchronous online discussions?

R3:

How do learners perceive their use of different graphic organizers in online collaborative learning?

Method

This study took place in an online graduate course for a semester (13 weeks) where students participated in four asynchronous discussions using the three different graphic organizers. To discover the influence of graphic organizers in online collaborative learning, the researchers collected discussion posts and conducted surveys as well as interviews.

Participants

The students engaged in four online asynchronous discussions taking place every 2–3 weeks and each lasting for a week. The 36 students were grouped as nine groups of four students at a time. They were assigned to a discussion group for each ID case according to the following rules: (1) every student used a different type of graphic organizer for each discussion, (2) for each ID case, students were assigned to different groups so that no pairs of students belong to the same group more than once, and (3) the order of using graphic organizers was different across the students.

At the beginning of the semester, the students were informed that the discussions would be accounting for 20% of their final grades and are required for their course completion. If a student did not agree to participate in the study, they were guided to opt-out of the study by contacting the first author who had no impact on a student’s grade. It was also noted to students in advance that their decision would not be acknowledged by the instructor or affect their standing in the class. Four students did not participate in a discussion at least once, and were excluded from the data analysis, which makes up 32 participants in total.

Treatment design

Out of the four discussions, the first group discussion did not involve any graphic organizers, and the remaining three discussions asked the students to develop respective graphic organizers (Fig. 4). On every discussion from the second to the fourth, three groups used pros/cons T-charts, another three groups employed topical tree charts, and the last three worked with maps. Though the sequence of graphic organizers used was different depending on each student, they used all three graphic organizers in the end. Intending to avoid threats to the internal validity, the order of the graphic organizers used was randomly shuffled to rule out confounders like familiarity to a certain ID case or maturation effects from becoming more skilled at discussions. As stated, members of a group were rearranged, and each student worked with new members at each discussion to rule out potential impacts of group dynamics.

Fig. 4
figure 4

Summary of the designs for the experiments

Full size image

To enable learners to collaboratively construct a graphic organizer in a group, three web 2.0 tools were used. The authoring tools were Google Docs for t-charts, Padlet for tree charts, and Bubbl for maps. The researchers considered several criteria to determine the tools for authoring graphic organizers, such as affordances of real-time collaboration, ease of use, and compatibility to the existing learning management system (e.g., Can they be embedded in the Canvas discussion forum?). All the graphic organizers made with the web 2.0 tools were embedded in the Canvas discussion forum where students would leave their posts (see Fig. 5). Placed at the very top of a discussion forum, a graphic organizer was updated in real-time, showcasing collective group work in one place. In addition, the students were presented with video tutorials that instructed them how to use each tool step-by-step to collaboratively develop a group graphic organizer.

Fig. 5
figure 5

A picture of the online discussion forum with a group’s graphic organizer and posts

Full size image

Materials

Through the weekly instructional design (ID) case discussions, students were expected to engage in case-based reasoning (Kolodner et al., 2005) and apply their knowledge in ID to real-world problems. The ID cases described authentic design problems where instructional resources would be limited, stakeholders had different opinions, or learning theories were conflicting. Each ID case was 3000 to 5000 words long and excerpted from the ID CaseBook (Ertmer et al., 2014). The instructional design cases were selected based on the setting (K-12, post-secondary, and corporate), the agent role (researchers and practitioners), and the stage in the ADDIE model (Dick et al., 2014) in order to provide comprehensive problems that students could relate themselves to.

The researchers deliberately chose cases situated in dilemmas to yield contentious viewpoints, and hence, more interactions. In tandem with the dilemma cases, pros and cons discussion questions were given to foster active exchanges of thoughts and horizontal interactions in which there is no right authoritative answer (Zhu, 1996). Some of the example questions are: “Do you agree or disagree with Chef Reiner’s daily Checklist evaluations? Provide suggestions considering Beth’s perceived conflict between constructivist and behavioral approaches to instruction”; “Do you agree or disagree with Craig’s recommendation of embedding test items in the game to help learners prepare for the standardized test and achieve more buy-in from schools?” Along with the main discussion questions, four facilitation questions were additionally provided to encourage students to consider various aspects of instructional design. The students were requested to employ instructional design principles applicable to the situations. Video tutorials about the contexts of each ID case were provided to enhance the understanding of each case and increase motivation to participate in the discussions. The students were to write at least two initial posts and reply to two others’ posts. The first author monitored their interactions and provided feedback through elaboration-encouraging and perspective-widening comments (Kwon et al., 2019). The time and amount of feedback were controlled across the groups and ID cases.

Data collection and analysis

The dataset of this study incorporates social and cognitive engagement in discussions, as well as perceptions of graphic organizers in one’s learning. The social aspects refer to how actively a participant engages in a discussion and interacts with others. The cognitive dimension explores how profoundly and extensively a participant builds their arguments.

Social engagement: participation and interaction

The social aspect of collaborative argumentation was gauged by learners’ participation and interactions.

Participation

For the participative dimension, the total number of messages, the number of idea units, and the number of words in an idea unit were counted and then compared across the treatment groups (Henri, 1992).

Interaction

To account for the interactive dimension, the number of interactions and individual posts was contrasted (Henri, 1992). Specifically, the density of interaction was quantified by the ratio of the number of interactions (i.e., links) to the total possible combinations (Density = 2a/N(N − 1), where a is the number of observed links and N is the total combinations) (Berkowitz, 1982; Fahy et al., 2001; Zhu, 2006). In addition, another measure of interaction intensity was used in this study. The intensity of the interactions was operationally defined as the persistence of a topic or thread, measured by the numbers of messages appearing in a thread (Fahy et al., 2001; Hewitt, 2005; So, 2009).

Cognitive engagement: knowledge construction levels and breadth of topics

Cognitive engagement was examined through the content appearing in the discussion messages. To investigate the level of knowledge construction and the breadth of the topics, two coders content-analyzed students’ discussion messages. The unit of analysis was a thematic unit (idea unit) that presents one purpose, holding a chunk of consistent points in a message (Henri, 1992). An idea unit consists of one to several sentences in a message. In this study, all the messages included 116,886 words and 2600 idea units in total. Taking off the comments that were unrelated to the discussion left 116,393 words and 2480 idea units, which take up 99.6% and 95.4% of the entire dataset, respectively.

Knowledge construction levels

Knowledge construction was assessed by a three-phase analytical framework by Kwon et al. (2019) for classifying ideas in discussions. This framework was established by synthesizing the analytic frameworks by Gunawardena et al. (1997), Garrison et al. (2001), Veerman and Veldhuis-Diermanse (2001), and Järvelä and Häkkinen (2002). The primary category for knowledge construction levels includes initiation, development, and construction. The initiation level is the most elementary level in the scheme and sets up a ground for discussion. Initiation is then subdivided into two secondary categories: coming up with an idea (new) and restating an earlier statement without adding any original ideas (restate). The second level, development, expands the scope or depth of the discussion by presenting supporting evidence (elaborate), putting forward an alternative aspect (alternative), or seeking additional information (inquiry). Construction, the final level, produces a higher level of knowledge by drawing on shared knowledge or metacognitive strategies. Construction levels include judging the quality of an argument (evaluate), synthesizing opposing views (synthesize), and reflecting on one’s own learning process (reflect). The social-emotional discourse such as remarks of recognition or social references without relevance to the topic discussed were excluded. The coding scheme of the three primary and eight secondary categories is presented in Table 2. Two researchers independently coded the students’ messages with the established coding scheme. The inter-rater reliability index by Krippendorff’s Alpha (2018) was estimated to be .87.

Table 2 The coding Scheme for coding the knowledge construction levels
Full size table
Breadth of topics

It was of our interest to see how extensively and profoundly topics are discussed and whether this breadth of discussion varies depending on the graphic organizer used. Therefore, two of the researchers implemented a thematic analysis of the students’ writings for each ID case (Braun & Clarke, 2006). Both researchers had experience in conducting qualitative data analysis for educational research and were versed in the research context. One researcher read through the posts in a group, identified patterns, and developed generative codes. Based on the open coding process, sets of codes were grouped into themes. A preliminary codebook included definitions, examples, non-examples, and notes for themes. The two researchers then pilot-tested the codebook by separately coding the same subset of data (11%). The two researchers met regularly to discuss uncertainties during the pilot test. Comparing the coding results by two researchers, discrepancies between them were discussed and reflected in the second codebook. Through this process, a codebook was established in which each ID case incorporates 11 themes (see Table 3). The two researchers separately coded another 33% of the data and computed inter-rater reliability. With the inter-rater reliability over .8, we assumed that the codebook was reliable. The Krippendorff’s Alpha (2018) was estimated to be .9.

Table 3 Example of the coding scheme for thematic analysis (ID case 3)
Full size table

Perceptions of graphic organizers

How students perceived their use of graphic organizers in their collaborative argumentations was asked through a survey and interviews. After the four discussions, the class was invited to take a survey and an interview.

Survey

Inspired by Han (2019) and Kwon and Park (2017), the survey items were developed in terms of three main categories probing how learners evaluate the graphic organizers for their learning (Table 4). The first category was the utility of graphic organizers for assisting collaborative argumentation processes. Secondly, the usability category asked the degrees to which the three authoring tools of Google Doc, Padlet, and Bubbl, were easy to use. Lastly, the survey intended to solicit students’ attitudes toward using graphic organizers in their online collaborative learning. The survey items included whether they would enjoy making contributions and noticing updates to the graphic organizers or want to persist in using them for their future learning. The survey items were provided with four choices on a 4-point Likert scale. A couple of open-ended questions were included, asking what they liked or disliked from their experience using the graphic organizers. Of those 26 participants who started the survey, 24 completed the survey answering all the questions, and the response rate was 66%.

Table 4 Survey items to investigate learners’ perceptions of using graphic organizers in collaborative argumentations
Full size table
Interview

To gather in-depth information as to how the learners perceived the graphic organizers, semi-structured interviews were conducted. The interview request was sent to all the participants and four students volunteered. The participants were interviewed separately, each of which lasted between 20 to 35 min and was led by one researcher. The questions included, but were not limited to, when and how frequently they used graphic organizers, their perceived differences between the graphic organizers, and scaffolding they needed for online collaborative learning. With the participant’s consent, the interviews were recorded, transcribed, and went through thematic analysis (Braun & Clarke, 2006). Since the interviews were organized around predetermined questions, what themes would emerge was expected to a certain extent, which made the process of determining themes “more heuristic rather than analytic” (Given, 2008, p 867). The coding guide was established by first settling the coding categories that pivot to the questions in the interview, and then fleshed out with meaningful themes which provided significant insights to the research questions and recurred across participants.

Results

Social engagement: participation and interaction

Participation

The analysis of the quantified result of the content analysis revealed that there was no significant difference in participation between different graphic organizers. Table 5 summarizes the number of messages, idea units, and word counts per group or student across graphic organizers. A group produces about 19 messages on average. Each member generally contributed 5.4 messages, including two initial messages and 3.4 replies. Given that the instruction required each student to write two initial messages and two replies, there were students who wrote more replies than required. One student, on average, developed 18 idea units per discussion and each idea unit was made up of 47 words.

Table 5 Number of messages, idea units, word counts per group and student
Full size table

Interaction

The density of interaction between the members in a group was calculated by observed links divided by the total possible combinations within the members in a group. The density for the groups with a t-chart was 81.5%, which means that 81.5% of the participants interacted with each other at least once in a discussion. For trees and maps, the density was 88.9% (see Table 6). There was no statistically significant difference in the densities.

Table 6 Interaction measures: density and intensity of interactions in discussions
Full size table

The intensity of interaction, referring to thread sustainability measured by the number of messages in a thread, was 2.61 for t-charts, 2.63 for trees, 2.65 for maps (see Table 6). These intensity levels were not significantly different across conditions.

Cognitive engagement: knowledge construction levels and breadth of topics

The cognitive aspects of a discussion investigate the depth and breadth of written argumentation. The depth of cognitive engagement was gauged by knowledge construction levels of idea units while the breadth of the discussion was measured by the number of topics addressed.

Knowledge construction levels

Knowledge construction levels were measured by the numbers of idea units falling into each level and their proportions. A student, on average, developed 4.7 initiating ideas, 12.1 developing ideas, and 1.3 constructing ideas, which individually account for 26.2%, 66.9%, and 6.7% of their entire contribution. With all messages in the account, we did not find any significant difference in the graphic organizers in the number and proportion of knowledge construction levels (see Fig. 6). The numbers and proportions of knowledge construction levels faceted by the type of treatments are presented in Table 7 and Fig. 7.

Fig. 6
figure 6

Total number of idea units according to graphic organizers

Full size image
Table 7 Mean number and proportion of knowledge levels by a student (all messages)
Full size table
Fig. 7
figure 7

Number of idea units for each knowledge level in all messages

Full size image

We further investigated the knowledge construction levels across the graphic organizers by separating out message types, initial messages, and replies, since replies more aptly illustrate the quality of their interactions. Even though no significant difference across conditions was observed in their initial messages, replies displayed noticeable differences in knowledge construction levels, indicated by the descriptive statistics of idea units (see Table 8 and Fig. 8).

Table 8 Mean number and proportion of idea units by a student (Replies)
Full size table
Fig. 8
figure 8

Number of idea units for each knowledge level in replies

Full size image

Figure 9 illustrates how the proportions of different knowledge construction levels in the replies are distributed with the probability density. Specifically, initiating and constructing ideas are more likely to appear in small proportions, while developing ideas tend to occur in relatively high proportions.

Fig. 9
figure 9

Kernel density estimates for the proportion of knowledge construction levels in replies

Full size image

A considerable difference in the replies was not found with the three primary categories of knowledge construction as shown in Fig. 9. However, the three conditions demonstrated different proportions with the eight secondary categories, particularly the alternatives in the development knowledge construction level. Such difference can be seen clearly in Fig. 10 in which the probability density for the t-chart lies higher than that of the map to the right. This demonstrates that the t-chart is likely to have more alternative ideas than the map.

Fig. 10
figure 10

Kernel density estimates for the proportion of the development-alternative level in replies

Full size image

When using a t-chart, students wrote .66 (SD = 1.41) alternative ideas on average, while only writing .56 (1.01) alternative ideas when they were using a map. A one-way analysis of variance test revealed that the difference between the graphic organizers was significant in that \(F\left(2, 93\right)=3.55, p= .03\). The effect size, Cohen’s \(f\) was estimated to be 0.28 and \({\eta }^{2}=.07\), which is considered moderate (Cohen, 1988). The post hoc analysis with Tukey’s multiple comparisons of means showed that the difference between t-charts and the maps had an acceptable level by the adjusted p-value (\(p= .02\)). The main difference was estimated to be .44, which means that a t-chart yields .44 more alternative idea units than a map type does.

Breadth of topics

The number of topics discussed by a group or an individual provides information on the breadth of the discussion. Table 9 shows the number of distinct topics by a group and an individual. Given that the total number of themes was 11 for all ID cases, a group tends to deal with 84% of the determined themes. Considering that an individual addresses 54% of the themes, we can see that the students were exposed to more themes than they would be working alone. We intended to examine whether a graphic organizer has an influence in addressing more topics but did not find any differences in the number of topics between the treatment groups.

Table 9 Mean number and proportion of the themes discussed within a group and by an individual
Full size table

Perceptions of graphic organizers

Through a survey, students’ judgments on graphic organizers were captured, particularly on the perceived utility of graphic organizers, the authoring tools’ usability, and their overall attitudes toward using graphic organizers in online collaborative learning. For each category, there were four to five items, and they were combined to indicate a category by being averaged. The Cronbach’s alphas were estimated to be .93, .72 ~ .78, and .89 ~ .94 which shows the acceptable levels of internal consistency in the survey items (Taber, 2018). Table 10 provides the mean scores and standard deviations for each category.

Table 10 Perceptions of using graphic organizers in collaborative argumentations
Full size table

On the perceived utility, students responded slightly positively: t-chart (M = 2.73), tree (M = 2.64), and map (M = 2.67). They tended to believe that the graphic organizers helped them participate in the discussions by guiding them to be more attentive, understand what’s being discussed, develop one’s ideas, and regulate group efforts. Pertaining to the authoring tools’ usability, on the other hand, students considered them to be weak: t-chart (M = 2.03), tree (M = 2.02), and map (M = 2.12). This indicates that the students did not feel it was easy to use and comprehend the authoring tools. The students’ attitudes toward using graphic organizers were shown to be neutral: t-chart (M = 2.69), tree (M = 2.56), and map (M = 2.48). The items included whether they enjoyed using graphic organizers in online collaborative learning or have prolonged motivation to use them in similar circumstances. There was no significant difference between the graphic organizers, for these three categories.

The students’ preferences on types of graphic organizers were also asked. Thirteen participants answered this question, by which 46.2% chose a t-chart, coming out at the top, followed by a tree chart (30.8%) and then a map (23.1%). In the following, the findings from the analysis of the open-ended questions and the interviews will be revealed with detailed reflections shared by the learners.

Utility for learning

The majority of the students perceived that the graphic organizers helped their learning in summarizing and comprehending discussions, generating and organizing their own ideas, and extending their knowledge of relevant facts or principles. Some stated that it was convenient to glance at everyone’s thoughts in one place, “highlight the trends in the conversation” (Student Q, survey), and convey the main ideas discussed in a condensed way. They also mentioned that graphic organizers helped them organize and clarify their thoughts, and thus assisted in their understanding of the concepts and principles related to the discussion topics. “They help you understand others’ ideas and … better understanding and organizing your ideas. … some of my statements, they were very wordy, but for the graphic organizers, they chop a lot of that stuff down and just give the point, make it as concise as possible.” (Student I, personal communication, December 14, 2020).

However, there were a few students who questioned the efficacy of using graphic organizers in this context. Some of the students said they felt it was time-consuming, “tacked-on busy work” (Student B, survey), and repetitive as they had to use a graphic organizer in addition to writing posts. A student commented that it was unnecessary, claiming graduate students have enough ability to follow the trends in discussions.

Usability

With respect to the usability of the authoring tools, the students tend to feel that using a t-chart in Google Docs is easier, while a map in Bubbl is the most difficult to create and follow. A tree chart in Padlet was also considered malleable compared to a map. The reasons for their perceived difficulty are summarized in the four points as follows. First, the students responded that they were more familiar with the t-chart type, with one of them saying they used them since K-12, and it felt like second nature. Second, a map has the most options to choose from followed by a tree chart then a t-chart. As a map has various visual features, such as colors, lines, and shapes, creating group work might have been overwhelming to the learners. Third, the difference in the perceived usability of the tools may have become more salient because of the insufficient time to learn to use a tool. The students used a new tool for every case, and there were some comments from the students saying one week was not enough to get accustomed to a new tool. Lastly, unlike the other graphic organizers, a map has a unique form, mostly of radial shape, which makes it hard to decide where to start reading. A student remarked “When you look at the t-chart, it's easy to go from top to bottom, or even the other one [tree] from left to right like the sequential order… But for the map, it's a lot more entropy in a way and it's just not as easy to follow where the story begins and where it ends” (Student I, personal communication, December 14, 2020).

Perceived affordances for collaborations

Some students expressed that the group discussions and the use of graphic organizers helped exchange ideas during the collaboration process, and thus expanded the scope of perspectives and the knowledge set. One student said, “it’s a good place for our paths to cross and exchange information and ideas … They really kind of had different career goals where a lot of them were like teachers” (Student E, personal communication, December 7, 2020). Another student said the graphic organizers supported the ideation process during which he was exploring new ideas in association with others’ contributions. “So, it let me think outside the box a little bit and can, and therefore kinda just collaborate more, like just jump off the ideas of others in a good way. Because I think it's important for there always to be different viewpoints … it led me to see how I can relate to them in a better way that we could allow for more inclusive ideas for all of us” (Student I, personal communication, December 14, 2020).

However, some complained that they didn’t like that there were already earlier comments that included the same or similar idea to theirs. “If my initial comment was similar to that of another student–and that student posted to the graphic organizer before I did–I was left with little to contribute” (Student K, survey). They also didn’t like when other members didn’t update the graphic organizers early enough even though they could have connected more ideas and led to more active interactions. “There wasn't much to go with you, waiting for other people to have things to respond to or read through. And that kind of slowed down the ability to have things to build upon” (Student G, personal communication, December 7, 2020).

Some students alleged that the graphic organizers lacked room for negotiating meanings and that it hindered communication as well as collaboration within a group. “There’s no way to really comment in some of the tools… I mostly went back to focusing on the discussion forum and responding to comments there. Because that’s where I felt like it was easier to discuss and actually interact with people. Whereas I didn't feel like it was as easy to interact with people through the graphic organizer” (Student L, personal communication, December 9, 2020). The students felt it was hard to communicate within the organizer itself, which might have deterred their participation and interactions in the graphic organizers.

Discussion

This section summarizes the findings of this study and their implications. The result of this study shows that t-charts led to more alternative ideas in replies than a map. This finding suggests that the type of graphic organizers affects interactions between individuals by making an environment that is conducive to producing more opposing ideas. This indicates that an individual’s ways of thinking and making an argument are partially influenced by a group graphic organizer. The finding is in line with the study by Nussbaum and Schraw (2007) where they found that graphic organizers that illustrate contrasting arguments, like the t-chart in this study, led to more rebuttals. It has been acknowledged from previous research that alternative ideas are comparatively difficult to produce in collaborative argumentations due to social pressure and the need for higher-order thinking skills such as refining and negotiating (Kwon & Park, 2017; Kwon et al., 2018). The socio-conflict theory by Doise and Mugny (1984) suggests learning occurs when students notice conflicts, convince others by demonstrating opposing views, and modify their knowledge or belief. Taken together, the finding supports the notion that certain types of graphic organizers encouraged learners to engage in collective knowledge construction through shaping discourse functions.

Particularly, the difference in the knowledge construction levels was observed between a t-chart and a map, which may be attributable to the following potential reasons. First, a t-chart’s structure with two columns is more straightforward than a map. For a map, one should take a closer look at each node and linkage to figure out their meanings and relations. On the other hand, a t-chart is easier to recognize ideas with because contextual cues coming from the simple structure generate anticipation of which sides a point would stand for. The self-evident pattern and learners’ familiarity with the tabular structure could have elevated the effect of graphic organizers in the collective knowledge construction process.

Another possible explanation for the observed difference is different usability according to the authoring tools. The students commented that a t-chart (Google Docs) was way easier to deal with than a map type (Bubbl). The authoring tools for a map contained more options in design features as noted in the survey report (see Usability in Result). When learners engaged in a group discussion with Bubbl, they may have experienced excessive degrees of freedom in creating a map, and this could have distracted their attention from the discussion toward the development of the graphic organizer itself. From this standpoint, the structural simplicity and the usability of the authoring tools could have made variations in how easily one can access and interpret the ideas on a graphic organizer and increased knowledge construction.

Lastly, a t-chart might have been most fit for the activity suggesting arguments for and against as it displays opposing ideas side by side. In this study, the cases and the discussion topics were deliberately chosen and designed for pro and con debates. The affordances of a t-chart, manifesting what to attack from the other point of view, could have made a difference in producing more alternative ideas in the discussions. For these reasons, results might change depending on the type of discussion. For example, if the task was not a pro and con debate but finding multiple, viable solutions in a complex, multi-faceted problem, a map or a tree chart could be as, or more, effective in generating knowledge construction.

Implications

This experiment adds to a growing corpus of computer-supported collaborative learning research comparing the types of graphic organizers in terms of multiple domains of learning: social (participation, interaction), cognitive (knowledge construction, scope of themes), and attitudes (utility, usability). The results showed that graphic organizers partly influence learners’ knowledge constructions; the pros and cons t-chart promoted a number of alternative ideas compared with the map types. The results in the other areas were found to be non-effective, but they provide the research community with empirical evidence and suggestions for future research.

These findings also provide implications in practice. First, an appropriate type of graphic organizer should be chosen or designed in accordance with a discussion topic, case, or learner characteristic. For example, in this study, the main problems were all contextualized in dilemma, and the arguments tend to be divided into two sides: agree vs. disagree, or advantages vs. disadvantages. It was one of the potential reasons why the t-chart was most effective under the circumstances. Depending on the context, instructional designers should be able to develop a tailored group organizer that best accommodates the contents discussed and represents the expected main discourse functions (e.g., classifying, defining, describing, evaluating, explaining, exploring, reporting) (Dalton-Puffer, 2016; Jiang & Grabe, 2007).

Secondly, when introducing a group collaboration tool, it is important to consider the workload on the learner’s end. Task designs should interlock a main task and group coordination efforts so that learners do not feel that using groupware is an add-on task attached without purpose. In this study, the students wrote more words in meaning units in the first discussion. This might be an indication that the introduction of graphic organizers diverted their cognitive resources to the graphic organizers’ interface rather than the discussions. This could be supported by student responses, as some students felt using graphic organizers was burdensome. Further, giving enough time to get used to tools would help adjust the load on the students. Though it may sound obvious, “representational guidance may not be a factor until the students started using the tool” (Suthers & Hundhausen, 2003, p 214).

Third, to improve the effect of graphic organizers, there needs to be scaffolding or facilitation for developing graphic organizers and increasing social interactions. Though this study provided the students with tutorials for the cases and the tools and feedback on the discussion board, it did not provide an intervention specifically targeted for the graphic organizer. We observed, from some of the students, late or no contributions to the graphic organizers, which other students complained about as the delayed input in the groupware kept them from more actively interacting with other members. Based on the participant’s needs, a facilitator can add soft and hard scaffolding (Saye & Brush, 2002). Particularly, a facilitator may provide soft scaffolding by initiating comments or questions in the organizer, as priming water into the group. For example, Ouyang and Xu (2022) showed that student interaction in a concept map improved when they were given encouragement and social support. In an online asynchronous setting, equipping rooms for social annotation in a group organizer might help a facilitator intervene and foster negotiations within a group discussion. In addition, hard scaffolding for group regulation can be given by assigning explicit roles to members (De Wever et al., 2010). In a graduate-level course, Hara et al. (2000) designated for each group, a starter who initiates a discussion by raising key points of the discussion or asking questions, and a wrapper who summarizes what has been addressed in the group and calls attention to significant points made. They found that this starter-wrapper technique increased interactions among members and enhanced higher level knowledge construction. Although these examples were directly given to the main medium of discussion (e.g., a threaded discussion board or forum) and not to group organizers, these scaffolding designs are expected to have the potential to improve students’ engagement with a group organizer, which needs further investigation.

Limitations

This result ties well with previous studies that confirmed the affordances of graphic organizers in promoting cognitive engagement. However, contrary to the findings of increased social engagement (Kwon et al., 2018; Ouyang et al., 2021), we did not find any significant difference in participation and interactions between the graphic organizers. Further, we did not observe any differences between the type of graphic organizers in the other knowledge levels except for alternative ideas. This might have happened because the effects of graphic organizers might have been minimal to advanced learners like graduate students as in this study. Additionally, the non-significant result in the other domains can be illustrated by the survey result where the usability scores were relatively low. The low usability of the authoring tools implies that the technology was not well integrated into this online learning environment, and this could have led to an impediment to achieving learning outcomes. One of the possible reasons for the low level of perceived usability is insufficient instructional support. As the students articulated from the survey and interviews, time to learn new tools was considered short, and there could be more encouragement for early participation and contributing to graphic organizers.

There’s still another possibility in which the null hypotheses incorrectly failed to be rejected. One of the potential reasons for this is that the experimental design of this study inherently requests large sample sizes. With the limited sample size of this study, the tests are underpowered (Visentin et al., 2020). The post-hoc power analysis shows that the statistical power of the test with the achieved effect size is 67%, which means with the 33% probability, the test does not reject the null hypothesis when the effect indeed exists. Given that the desirable power for an experiment is 80% and above, the current experiment was underpowered. For this conventional power level of 80%, the smallest effect size that would be able to be detected is 0.32, which indicates that the test is not sensitive enough to capture smaller differences than that (Cohen, 1988). The effect size revealed from this study could be leveraged and taken into account for experimental designs of future research.

Future works

One of the remaining issues observed during the study and left unaddressed was learners using patterns for graphic organizers. Some students remarked that they had organized their main ideas in a graphic organizer and fleshed them out within their posts, whereas others responded that they had written their posts first and then added their summary into a graphic organizer. We speculate that the sequence of when learners use a graphic organizer could be an element that influences an individual’s or a group’s knowledge construction levels. For example, when learners need to work on a graphic organizer during the ideation phase, it might be more likely that before building out their ideas, they will explore what contributions were made by others. This will increase the likelihood that the contents in a graphic organizer get noticed by the discussants.

Another realm that could be worth exploring is representational artifacts authored by learners. Although this study examined different dimensions of learning outcomes based on the learners’ written argumentations, the individual students’ contributions to the group graphic organizer were not an object of analysis. Some related research on graphic organizers investigated individual learners’ contributions to the group organizers (Suthers et al., 2003, 2008), but did not conjoin them directly with the collaborative process or the outcomes dealt in this study (i.e., participation, interaction, knowledge construction, and scope of discussion). As learners’ achievements shown in their written argumentations could be associated with the degree of participation for graphic organizers, students’ work on the graphic organizers should be examined further in future studies.