Introduction

The threaded discussion forum is one of the most common instructional methods adopted to facilitate higher order thinking in online learning environments. There are several benefits to using the threaded discussion forum to support asynchronous communication modes: They provide learners enough time to think about others’ opinions and reflect on their own arguments before posting (Heckman and Annabi 2005). Through delayed peer interactions, learners can be more reflective and critical, which is rarely achieved in teacher-centered face-to-face classrooms (Rovai and Jordan 2004). Threaded discussion forums allow learners to participate in discussion equally since they do not need to wait until the other learners stop talking as they should in a face-to-face classroom discussion. Threaded discussion also displays the structure of the discussion, which helps learners recognize the depth of messages and easily capture “message-reply” sequences (Guzdial and Turns 2000; Scheuer et al. 2010).

Despite these benefits, there are some limitations to threaded discussion forums. One of the main concerns is the passive participation of students. Several studies have reported low participation rates. For example, Fung (2004) found that students posted only the minimum number of messages when an instructor emphasized a certain number of posts for grades. This might be explained by pointing to low motivation for the academic activity or unwillingness to post a perceived (self-defined) “ridiculous” message that can be reviewed by others over time (e.g., Hara et al. 2000, p. 145).

The quality of messages also matters to instructors. Students tend to exchange superficial messages rather than sufficient elaboration or critical debates (Gunawardena et al. 1997; Heckman and Annabi 2005; McLoughlin and Luca 2000). Hara et al. (2000) mentioned disappointedly, “There was never a sense of real heated or seminal online discussions with students negotiating meaning, taking sides on issues, or coming to compromise” (p. 141). From a social constructivist perspective, learning occurs when students construct their own meaning through interactions with others, by negotiating different ideas, and reflecting on their own (Jonassen et al. 1995). Online discussion activity has been promoted by educators for its potential to facilitate higher order thinking processes. However, it is difficult to obtain the higher order thinking outcomes within online discussions and requires well-designed instructional interventions.

Additionally, the discussion structure illustrated by the threaded forums may increase cognitive load (Murphy and Coleman 2004). A key factor in identifying discussion structure is the linkage between an initial post and its replies; however, the linkage does not necessarily represent the relationship between messages. For example, when an initial post discusses three topics (Topics 1, 2, and 3), with the first reply elaborating on Topic 1 and the second reply debating Topic 2 from a different perspective, the discussion structure links these two replies to the initial message. However, it does not display the content of the linkages. In this case, students need to figure out by themselves the relationship between messages and the flow of discussion, and this requires extra cognitive effort (Eryilmaz et al. 2009).

Last but not least, a characteristic of online forums is the emphasis on “new” or “unread” posts, which can encourage the undesirable habit of disregarding old posts. Hewitt (2005) found that students tended to ignore messages that were posted early but were not replied to and paid more attention to replies (or new initial posts) marked as unread. It is possible that students overlook some discussion topics unintentionally while focusing on only a few, not because of their content but because they are marked as unread. Hewitt (2005) described this phenomenon: “Once a thread slipped out of the communal spotlight, it was unlikely to return” (p. 580).

Creating representations of discussions has been suggested as an effective instructional strategy to overcome the limitations and facilitate higher order thinking (Jyothi et al. 2012; Nussbaum et al. 2007; Suthers et al. 2008). While these studies supported the educational effects of visualization of discussion, it was not easy for instructors to adopt the tools because they were not built into common learning management systems (LMS). Online instructors especially need convenient instructional tools that can be easily integrated into their LMS without complicated technical knowledge. Considering this need, the current study designed simple visualization tools available in a common LMS. The two tools invented were a cognitive diagram, emphasizing the flow of discussion topics, and a social diagram, emphasizing interactions among students. Given the different emphases of the two tools, different learning behaviors and learning outcomes were expected. The following section is an overview of the general effects of visual representations of discussion based on the Community of Inquiry model (Garrison et al. 1999) and the features of previously used discussion representation tools.

Literature review

(Garrison et al. 1999) suggested a conceptual framework that identified cognitive, social, and teaching presence as crucial components of a successful Community of Inquiry (CoI) in online learning environments. The CoI model emphasizes that instructors consider social processes as well as cognitive structures when practicing instructional design, facilitating discourse, and providing instruction (Anderson et al. 2001). As the current study aimed to examine the effects of instructional intervention (strategy) on knowledge-building in online discussion, the representation tools were developed in consideration of two aspects: cognitive presence and social presence.

Cognitive presence is related to the quality of discourse shared in a community of inquiry (Garrison et al. 2001). It provides a theoretical framework for evaluating the process of critical thinking, and it reflects the quality of discourse and the degree of knowledge-building as a result. There have been many studies that have assessed quality of discourse by means of content analysis (see De Wever et al. 2006). The common interest among these studies is the suggestion of levels of discourse and instructional strategies to enhance the overall quality of discourse.

Social presence emphasizes social aspects that support cognitive presence (Garrison et al. 1999). The CoI model suggests three categories of social presence: emotional expression, open communication, and group cohesion. For the purposes of this study, the researchers are most interested in the category of open communication, which describes reciprocal and respectful exchanges of discourse. On the basis of the CoI model, students will be more respectful and responsive to others if they have mutual awareness and recognition of each other’s contributions; this can be facilitated by visual representation of their participation. The following subsections discuss studies that have explored the effects of visual representation from cognitive and social perspectives.

Visual representation to enhance cognitive presence

Without instructors’ specific interventions or instructional strategies, the quality of students’ online discussions may be lower than satisfactory (Rovai 2007). In alignment with Curtis and Lawson’s (2001) study, Pawan et al. (2003) found that their students did not go further than “simply sharing their thoughts”, not reaching the further phases of “challenging and questioning each others points of view” (p. 127). Guzdial and Turns (2000) suggested that “simply making a discussion forum available does not mean that it will be used effectively to enable learning” (p. 437). Considering the need for instructional intervention, visual representation of online discussion could be a good option for facilitating better discussion activities, such as considering alternative perspectives and negotiating different opinions (Nussbaum et al. 2007), providing evidence for arguments (Suthers and Hundhausen 2003), and changing initial opinions (Nussbaum et al. 2007), because it effectively contrasts different perspectives and encourages students to develop “balanced reasoning” (Nussbaum et al. 2007, p. 493).

Since one purpose of discussion is not to win an argument but to elaborate on one’s own ideas, listening to the opinions of others is crucial to the process of developing better arguments and learning from discussion (Wise et al. 2013). Also, it is important to learn how to develop coherent arguments and consistently support them with evidence. However, without explicit guidance from an instructor, students tend to focus on their own arguments and are reluctant to consider alternative opinions (Nussbaum and Schraw 2007). In order to encourage discussion activities, (Nussbaum et al. 2007) invented online argumentation vee diagrams (AVDs), which contrasted two opposite opinions, and encouraged students to integrate them. Using AVDs forced students to consider alternative views and validate their initial opinion in consideration of them, which facilitated balanced reasoning. As a result, students who used the AVDs found it was easier to compromise between two different opinions during discussion. The ability to consider alternative views and change one’s opinion after validating an argument is a type of cognitive engagement highly desired in discussion, and one that is not achieved easily without proper instructional intervention (Baker 2003; Koschmann 2003).

The integration of knowledge is the ultimate goal of discussion. To achieve this, students need to understand the relationships between concepts, refer to concepts to explain phenomena, and restructure prior knowledge in a coherent way, which requires a high level of cognitive engagement (Alavi and Tiwana 2002; Davis 2000; Thomas 2002). A visual representation can support the cognitive process of students by visualizing the elements of arguments, highlighting relations of arguments, and contrasting opposite perspectives (Novak 1990; Nussbaum et al. 2007; Suthers 2001). Bell and Linn (2000) implemented the SenseMaker tool for online discussions. The SenseMaker demonstrated expert models, allowed students to monitor and reflect their arguments in visual mode, and helped them, together, to compare individual ideas. By visualizing multiple items (evidence) that supported their arguments, students could consider various perspectives and generalize their knowledge during “scientifically authentic discussion” (Bell and Linn 2000, p. 810).

In the process of integrating knowledge and sustaining discussion, maintaining focus on topics is a necessary condition for effectively leveraging discussion activity for learning (Guzdial and Turns 2000). To ensure this condition, Guzdial and Turns (2000) designed CaMILE to (1) display discussion threads as well as notes illustrating the flows of discussion and discourse interactions, (2) provide procedural guidance by prompting types of discussion and suggesting structures based on the type selected, and (3) present anchors at the beginning of discussion threads where relevant content was located. In particular, the first and third functions were used in visual representations to attract students’ attention and illustrate discourse structures. As a result, the use of CaMILE sustained longer discussions, broadened participation, and kept discussions on topic.

Visual representation to enhance social presence

It is not easy to monitor (or realize) the level of engagement among students in a threaded discussion. This negatively affects students’ awareness of their own interactivity and can reduce motivation to engage in discussion as discussed below. In a “collaborative” learning environment, revealing contributions by members encourages more active engagement among students (Kerr and Bruun 1983; Lam 2015). Garrison and his colleagues have suggested that mutual awareness “begins with evidence that others are present and attending to messages” (Garrison et al. 1999, p. 100).

Research regarding computer-supported collaborative learning (CSCL) has provided the theoretical understanding that group awareness can be enhanced through visual representation of individual behaviors such as number of messages posted or contributions of each member (Janssen and Bodemer 2013; Järvelä et al. 2014). A growing body of evidence has shown that well-shared group awareness can promote interactivity among students in collaborative learning (e.g., Fransen et al. 2011; Kwon et al. 2013; Lambropoulos et al. 2012; Phielix et al. 2011). Although a considerable number of studies have been carried out regarding group awareness in CSCL settings, not enough research has been done on online discussion to examine the educational effects of visual representations that illustrate the interactivity of students.

From the social presence framework, a myriad of studies has been done to understand interactivity among students in online discussion through social network analysis (e.g., Aviv et al. 2003; Hernández-García et al. 2015; Stewart and Abidi 2012). This research provides useful information to educators and researchers regarding patterns of interactions (Heo et al. 2010), individual student’s roles and power (Aviv et al. 2003), and, most importantly, who is lurking or needs the instructor’s attention (Jyothi et al. 2012). By shedding light on how students interact and how networks grow, the studies have guided instructors in designing instruction for better discussion. However, there is a dearth of research on how visual representation of interaction affects student participation at the moment of engagement in online discussion.

Considering the interactions and contributions of members affect the interaction patterns of online students, it is reasonable to assume that a social diagram illustrating individual student interactions would facilitate active engagement among students. However, it is not certain that the quality of discussion can be encouraged with a social diagram because it does not emphasize the content of discussion. Thus, this study aimed to examine the effect of social diagrams on student interaction and quality of discussion.

Different effects of different representations

Different types of representation will have different effects on learning activities. For example, a concept map illustrating relationships between concepts in a two-dimensional diagram fosters knowledge acquisition processes by explicitly representing the structures underlying a domain without adding cognitive demands on students to extract the inherent structures (Novak 1990). Graphics, another example, can efficiently represent information by comparing values and adding attributes in the form of colors or shapes. Compared to spreadsheets, which present data in cells coordinated by columns and rows, graphics organize the same data using dimensional modality, which reduces cognitive load and promotes learning as a result (Keller et al. 2006).

In collaborative learning contexts that include online discussion, students’ learning outcomes and interactions may be influenced differently by different representation modes. Suthers and Hundhausen (2003) examined the effects of three representation types (graph, matrix, and text) on collaborative learning processes. In their experiment, students were asked to generate external representations illustrating the relationships between a hypothesis and data. A matrix naturally represents all possible relationships by aligning the hypothesis and data in rows and columns, while graphs and text are limited to illustrating certain selected relationships. As a result, students who generated matrices considered a wider range of relationships than those who generated graphs or text representations. Compared to the matrix users, the graph users focused on a few main relationships and were highly engaged in understanding them, which affected their knowledge construction. In general, graphs make the relationships between nodes salient and intuitive by connecting the nodes with relational links, which allows users to understand the relationships deeply (Scheuer et al. 2010). Although the main advantage of a graph lies in the fact that it organizes relations effectively, it may become too complex as the relations are explored, limiting its usefulness (van Drie et al. 2005) and distracting students from the main contents of discussion (Suthers 2001).

Even when the modality of representations is the same, the content can vary depending on the focus and interest of users. For example, knowledge maps illustrate relationships between data and hypothesis (Suthers et al. 2008) and social diagrams describe interaction patterns among students (e.g., Jeong 2003; Liu and Tsai 2008). Although they share the same graphical presentation modality, the contents they represent are different. In particular, the former has typically been used to facilitate meaningful discussions among students, whereas the latter has been used to understand the nature of student interactions. Providing social diagrams to students during discussion would affect their discussion differently than knowledge maps would because they emphasize interaction patterns as opposed to content, which knowledge maps do. Therefore, we hypothesize that if visual representations emphasize different aspects of discussion, as did the tools we invented, they will guide students’ attention to the aspects emphasized. To our knowledge, no direct comparison between cognitive and social diagrams has been done yet to validate this hypothesis in online discussion settings.

Purpose

The purpose of this study was to investigate the effects of diagrams emphasizing either content (cognitive diagram) or interactivity (social diagram) in online discussion. We hypothesized that the two types of diagrams would affect students’ discussion behavior differently: The cognitive diagram would encourage more profound development of ideas while the interaction diagram would facilitate more active interaction among students. The following research questions were addressed:

  1. 1.

    Will the cognitive diagram encourage students to develop more thoughtful messages compared to the social diagram?

  2. 2.

    Will the social diagram facilitate more active interactions among students compared to the cognitive diagram?

Method

Participants

Thirteen participants were recruited from an online instructional design course at the graduate level. Participants were not given compensation for their participation in the study. Six were enrolled in online master’s or doctoral degree programs, six in online graduate certificate programs, and one in a residential doctoral degree program. The gender of participants was balanced (seven females and six males). The study was approved by the University Institutional Review Board (#1506210201).

Context of learning

Participants were enrolled in a 12-week online course during a summer semester. They were asked to analyze an instructional design (ID) case (Ertmer et al. 2013) and discuss the case in an online forum over one week. The forums opened on Mondays and closed on Sundays. A total of five ID cases were distributed over five nonconsecutive weeks (Weeks 2, 3, 5, 6, and 9). The instructor provided participants with leading questions that initiated discussion but did not limit the topics to them. The instructor and a teaching assistant graded the discussion activities based on a course rubric which encouraged participants to engage in higher order thinking and knowledge building. The discussion took place in an online forum embedded in the CANVAS learning management system. The forum allowed threaded discussions and a rich text edit mode. Students had their own group discussion forum and could not access other groups’ discussion forums. The experiment was part of the participants’ regular course work.

Instructional interventions

The instructor and the teaching assistant generated two different visual representations of discussion—social and cognitive diagrams—and presented one of them at the top of each discussion thread. The social and cognitive diagrams illustrated whether discussion flow focused on the interactions between students or the evolution of topics, respectively.

Social diagram

The social diagram described who responded to whose messages by linking the images of authors, but they did not display the topics of the messages. The diagram also illustrated when messages were posted by aligning them with the days they were posted. The social diagrams were updated daily and accumulated interactions for the whole week. Figure 1 is an example of the social diagram.

Fig. 1
figure 1

Social diagram

Full size image

Cognitive diagram

The cognitive diagram summarized how discussion topics evolved. Once an initial post mentioned a topic, it was linked to following posts that elaborated, criticized, or synthesized the topic. Due to the expanding volume of the discussion and space limitations, sometimes the evolutions of topics could not be displayed in their entirety at the end of the discussion period. In that case, the instructor would display only a few topics that had been discussed recently. Figure 2 is an example of the cognitive diagram.

Fig. 2
figure 2

Cognitive diagram

Full size image

Research design

For the discussions, participants were randomly assigned to one of two groups, each consisting of six or seven students. In order that students’ initial discussion activity could be observed and measured by the researchers, the first case study was without instructional treatment. In the following two case studies (2 and 3), participant groups were shuffled. One group received the interaction diagram for Case Study 2 and the topic diagram for Case Study 3, while the other group received the diagrams in adverse order. For the last two case studies (4 and 5), the instructor reassigned group membership randomly and provided discussion diagrams in the same way as in Cases 2 and 3. Figure 3 describes the research design.

Fig. 3
figure 3

Grouping and instructional interventions

Full size image

Measurement

In order to answer the research questions, we collected data that represented the interactivity of students (post days, centrality, and number of messages) as well as knowledge building (quality of messages). The perceived usefulness of the discussion diagrams was also collected from students.

Post days

Discussion is reciprocal communication. So, the contents and directions of a prior post affect the following posts (Hara et al. 2000). The timing of posts—when an initial message is posted and when responses follow—also affects the quantity and quality of discussion (Hewitt 2005; Jeong and Frazier 2008). A message posted early can be expected to get more responses than one posted late; regardless of content, messages posted early have more of a chance of being exposed to learners. Moreover, regarding early messages, learners tend to raise more challenging arguments or provide more evidence, which improves the quality of discussion (Jeong and Frazier 2008). In this sense, one may argue that encouraging learners to post initial messages early can induce more meaningful interactions. All discussions began on Monday and ended on Sunday. Researchers converted the days to numerical values so that Monday was 1 and Sunday, 7, and so that the average post days could represent the timing of posted messages.

Centrality of students

The proportion of actual links to potential links, called centrality, measures a student’s degree of interaction with other students (Aviv et al. 2003; Freeman 1978). If a student had a high degree of centrality, she would have active interactions with others. We can measure centrality in terms of incoming links (in-degree) and outgoing links (out-degree) (de Laat et al. 2007). Hanneman and Riddle (2005) explained that those who had high in-degree centrality were perceived as “important” by others while those who had high out-degree centrality “influenced” others. In an online discussion forum, the centrality of each person can indicate to what degree opinions were shared with others. For example, if a student posted a message and all the other students responded to it while she did not respond to others, she would have high in-degree centrality (100%) but low out-degree centrality (0%).

Researchers measured centrality on two levels: thread and forum. On the thread level, the centrality of each student who posted an initial message was measured. Thread level centrality indicated the importance and influence of each initial message. On the forum level, the centralities of all students were measured from the accumulation of threads in each case discussion. Forum level centrality indicated the overall interactions of students in a discussion group.

Quantity of messages

As studying “interactivity-as-process”, the current study focused on human interactions (Stromer-Galley 2004). Rafaeli and Sudweeks (1997) suggests that “the condition of communication in which simultaneous and continuous exchanges occur” indicates the interactivity in computer-mediated communication. To measure the degree of communication, we counted the number of initial messages and replies to them. The average number of initial messages and replies per student indicated the degree of interactivity (Heo et al. 2010).

Quality of messages

Based on the guidelines offered by Chi (1997) and Graneheim and Lundman (2004), a coding scheme was developed, and the researchers conducted a content analysis using the following steps:

Coding scheme development

The development of a coding scheme was a recursive process. After reading samples of the messages, the researchers met and discussed a tentative coding scheme that each individual researcher would use independently until a consensus could be reached. In order to refine the coding scheme, each researcher carried out content analysis with the tentative scheme on messages in a pre-determined discussion, then refined it until there was no conflict.

Based on the literature review (De Wever et al. 2006; Garrison et al. 2001) and the recursive refinement process, the coding scheme was developed to analyze the quality of discussion messages. It included six types of responses: initial message, agree/disagree, question, answer, social, and off topic. Among the response types, we were most interested in the agree/disagree and answer categories because they reflected the students’ knowledge building process. In order to examine how students built knowledge during discussion, we identified ways to develop ideas, ranging from low to high level development: no elaboration, summary, elaboration, and conceptual change (see Table 1). On the basis of Toulmin’s (1958) argumentation framework, for further examination, the elaboration was divided into four types: new argument, evidence, rebuttal, and counter argument (see Table 2).

Table 1 Coding scheme
Full size table
Table 2 Coding scheme of elaboration types
Full size table
Segmentation

Graneheim and Lundman (2004) defined the idea unit as a “constellation of words or statements that relate to the same central meaning” (p. 106). In this study, the idea unit served as the unit of analysis. The researchers divided discussion messages into idea units containing one meaning that matched with the subcategories.

Coding and inter-rater reliability

After the discussion messages were divided into idea units, two independent coders carried out a content analysis based on the coding scheme. The initial inter-rater reliabilities reached were outstanding in their agreement on types of responses (Cohen’s kappa = .87) and substantial on developing ideas (Cohen’s kappa = .72) (Landis and Koch 1977).

Representation and identification of behavior patterns

After completion of coding, statistical analyses testing for differences between groups (a χ 2 test and analysis of variance) and describing group characteristics (percentage and frequency) were used to identify patterns that emerged from the data.

Perception of intervention

An online survey tool was developed to investigate the usefulness of the discussion diagrams as perceived by students. Eight questions on the usefulness of each diagram were presented and revealed high reliability (social diagram: α = .93, cognitive diagram: α = .94). Two of the questions asked students to directly compare the two types of diagrams in order to gauge their preferences. All responses were measured with a 5-point Likert scale that ranged from 1 (strongly disagree) to 5 (strongly agree). Perceived usefulness was measured by averaging the responses to the eight questions.

Results

Overview of discussions

Table 3 shows the number of messages posted and the total word count for each discussion for each ID case. The results of the first ID case revealed that there were considerable differences between groups even when no instructional treatment was given. In addition, the quantity of postings and interactions varied in accordance with the ID cases for the five discussion periods. Considering the difference was based on ID cases rather than experimental conditions, the researchers decided to convert the numbers to standardized scores for each ID case in order to equalize the impact of the ID cases on the discussions.

Table 3 Number of initial messages and responses in each ID case discussion
Full size table

Types of responses

Table 4 describes the frequency of responses to the initial messages and other replies. In order to check if there were significant differences between instructional treatments and time periods, the researchers ran a two-way repeated measures analysis of variance (ANOVA) for each response type. First, for the agree/disagree response type, the results revealed that there was a significant main effect of treatment on agreement, F(1, 12) = 19.74, p = .001, \(\eta^{2}\) = .62. This result tells us that students were more likely to agree with their peers’ opinions when they received the social diagram as compared to the cognitive diagram. Interestingly, there was also a significant main effect of the treatment on disagreement, F (1, 12) = 9.81, p = .009, \(\eta^{2}\) = .45. This result reveals that students were more likely to disagree with their peers’ opinions when they received the cognitive diagram as compared to the social diagram. From the results, it is apparent that the cognitive diagram encouraged students to disagree with others’ opinions while the social diagram encouraged them to agree.

Table 4 Frequency of responses to initial messages and replies in each ID case discussion
Full size table

Second, for the social response type, the results revealed that there was a marginal main effect of treatment, F (1, 12) = 4.33, p = .06, \(\eta^{2}\) = .27. This result tells us that social diagrams can potentially effect facilitation of social interactions among students.

Third, for the question and answer response types, the results revealed that there was a marginal main effect of treatment on questioning, F (1, 12) = 3.94, p = .07, \(\eta^{2}\) = .25 but no effect on answering, F (1, 12) = .34, p = .57. This result tells us that social diagrams can also potentially effect facilitation of questioning activity among students.

Finally, the results of the overall interactions revealed that students more actively replied to others when they received social diagrams than when they received cognitive diagrams, F (1, 12) = 11.21, p = .006, \(\eta^{2}\) = .48.

All of the results revealed that the time period did not have any effect on students’ discussion activities. In other words, all the findings were consistent throughout the two time periods. Researchers excluded the off topic response type from the analysis due to its lack of frequency.

Elaboration of discussion

Table 5 shows the frequency and proportion of response types and the ways that ideas were developed in discussions The development of ideas was analyzed by each response type. Overall, elaboration was found to be the most frequent. In addition, students always elaborated on their ideas when they disagreed with their peers’ opinions. It is worth noting that a meaningful number of responses (21% in total) did not elaborate on ideas when they agreed with a peers’ opinion, and this occurred quite often in the social diagram condition (28%).

Table 5 Frequency and proportion of response types and ways to develop ideas
Full size table

To investigate the effects of discussion diagrams on the quality of developing ideas, the researchers separated the methods of development into two categories: low level and high level. Considering their contributions to the development of discussion, ideas comprising either no elaboration or a summary were identified as low level while ideas comprising elaboration or conceptual change fell into high level. Considering the insufficient frequency of both question and answer types, the researchers decided to exclude them from the analysis. Social and off topic categories were also excluded for being unrelated to the development of ideas.

A χ 2 test of independence was performed after combining the agree and disagree response types to examine the relation between the discussion diagrams and the response levels. The relation between these variables was significant, χ 2 (1, N = 288) = 11.77, p = .001. Students were more likely to post high level responses (cognitive: 90% vs. social: 74%) and less likely to post low level ones (cognitive: 10% vs. social: 26%) when receiving the cognitive diagrams as compared to the social diagrams (See Fig. 4).

Fig. 4
figure 4

Proportion of high and low level of responses in two discussion diagrams

Full size image

Table 6 describes the types of elaborations. Students most frequently elaborated on a discussion by adding new arguments (overall 61%). The researchers found only a few cases of counter argument (5%) and rebuttal (9%) from their elaborations. No further statistical analysis was done for low frequency and similarly proportioned categories across conditions.

Table 6 Frequency and proportion of elaboration types
Full size table

Post days

Table 7 describes the post days. As explained in the Method section, researchers assigned numeric values to the days of the week. For example, if a message was posted on the first day of discussion, it was assigned the number 1; the second day, the number 2; and so on. In this way, the researchers calculated the average day for postings in the discussion.

Table 7 Average post days
Full size table

An ANOVA showed that there was no significant main effect from the day of the initial messages in both Cases 2 and 3 and Cases 4 and 5. However, for the days of the replies to the initial messages, the results revealed a significant main effect in both Cases 2 and 3, F(1, 129) = 5.35, p = .022, \(\eta^{2}\) = .04, and Cases 4 and 5, F(1, 93) = 9.41, p = .003, \(\eta^{2}\) = .09. This result indicates that students responded to other’s messages earlier when they received the cognitive diagram than when they received the social diagram.

Centrality of members

Table 8 describes the centrality of students. An ANOVA revealed there was no significant main effect on the thread level centrality. However, the result showed a significant (or marginal) main effect on the forum level centrality of in-degree, F(1, 50) = 3.35, p = .07, \(\eta^{2}\) = .063 and out-degree F(1, 50) = 4.62, p = .037, \(\eta^{2}\) = .085. This result indicates that students received (in-degree) and provided (out-degree) more responses from and to other students when they were provided with the social diagrams.

Table 8 Centrality of students
Full size table

Students’ perception of discussion diagrams

Table 9 describes the students’ responses to a survey asking about the perceived usefulness of the two discussion diagrams. Although the cognitive diagram was perceived as more useful when compared to the social diagram, no statistically significant difference was found except with Question 6. A paired-samples t test indicated that the students perceived the cognitive diagram (M = 3.4, SD = 1.12) to be more helpful than the social diagram (M = 2.3, SD = .65) in understanding the issues discussed, t(10) = 2.96, p = .014, d = 1.2. The result implies that the cognitive diagram sufficiently illustrated not only the discussion topics but also the interactions between students, at a level comparable to the social diagram (see Questions 5 and 6). Direct comparisons between the two diagrams also revealed the students’ preference for the cognitive diagram over the social, but the difference was not statistically significant (see Questions 9 and 10).

Table 9 Survey responses
Full size table

Discussion

The purpose of this study was to investigate the effects of discussion diagrams on online discussion behaviors. The researchers hypothesized that the discussion behaviors would be affected differently based on the emphasis of the discussion diagrams. To test the hypothesis, two different discussion diagrams were invented and provided to online students. The main findings of the study can be summarized as follows: The social diagram and the cognitive diagram influenced students’ online discussion behaviors differently because they emphasized distinct features—interactivity among students and elaboration of discussion topics, respectively.

In summary, when students received the social diagrams, they tended to be more active in responding to others’ messages. Researchers found that the social diagrams affected almost every student’s centrality during discussion, not just a few active ones. However, the social diagrams did not encourage students to challenge each other’s ideas to enhance the quality of discussion. Considering the “agreeing” behavior and frequent social interactions, the social diagrams facilitated socially desirable responses.

In contrast, when students received cognitive diagrams, they tended to be more thoughtful toward one another’s ideas and elaborated on them or raised alternatives that were cognitively desirable responses. We also found an unexpected result which was that students responded to each other’ messages earlier in that condition, proving they did not rush into the discussion at the last moment but prepared their arguments relatively early. It also suggested that students in the social diagram condition were more active in the later discussion period.

Quality of discussion

Gunawardena et al. (1997) suggested a five phase analytic model for examining the co-construction of knowledge in online discussion. They distinguished lower and higher mental functions based on cognitive demands and meaning making. The lower mental functions included sharing and comparing information, and exploring cognitive dissonance. The higher mental functions were more closely related to group learning, which included negotiation of meaning, co-construction of knowledge, validation of proposed synthesis, and construction of shared knowledge. Facilitating the higher mental functions can be a target instructional goal to be achieved in online discussion.

The participants in this study engaged in higher mental functions quite often (79% of valid responses fell in the elaboration or change categories) compared to participants in other studies (e.g. Gunawardena et al. 1997). This could be due to the instruction of discussion given by the instructor and the characteristics of participants. In this study, a small number of graduate students majoring in educational technology were asked to engage in discussion. They also understood that the instructor would evaluate the quality of their messages based on the rubric, encouraging higher order thinking and the co-construction of knowledge.

In comparing the results of the instructional treatments, we found that the cognitive diagram strongly facilitated the higher mental functions compared to the social diagram. It is worth noting that the impact of the cognitive diagrams was associated with the “disagreeing” behavior. As the results revealed, the disagreeing behavior required the elaboration of arguments and triggered the negotiation of meaning. In real terms, the difference in disagreeing behaviors between the two treatments was considerable. Students disagreed 25 times when they received the cognitive diagram which was more than double the number of disagreements with the social diagram, and four times more than the proportion of whole responses (16 vs. 4%). The findings suggested that disagreeing behavior encouraged by the cognitive diagram had facilitated higher mental functions in the discussion.

An increased number of higher mental functions was also observed in the agreeing behaviors. As many as 88% of the responses that agreed with previous messages in the cognitive diagram condition fell in the higher mental function category. On the other hand, the social diagram condition had 31% of the responses simply agreeing without further elaboration or with a brief summary of the previous argument. The study revealed that, given the cognitive diagram, students more actively elaborated their ideas or changed them while engaging in the negotiation of meaning, even when agreeing with previous messages. The cognitive diagram seemed to facilitate students’ higher mental functions in both agreeing and disagreeing arguments.

Then, what features of the cognitive diagram affected the discussion behaviors? To answer this question, we need to examine how students perceived the cognitive diagram and how the instructor designed it. Based on the survey responses, the cognitive diagram helped students understand the issues being discussed. In this study, the instructor and the teaching assistant paid attention to main discussion topics and how they evolved as discussion went on. They captured the main points and illustrated them in a simple form. They considered information overload in the diagram and reduced information by limiting the text to bullets of discussion topics and arguments. As a result of these efforts, students easily understood the diagram and enjoyed it when it was presented/updated.

Interaction during discussion

Many researchers have agreed that having enough interactivity is a prerequisite to constructive discussion (Dennen 2005; Jeong and Frazier 2008). In this sense, the social diagram was expected to facilitate interactivity among students in online discussion. In this study, interactivity was measured by the number of responses to initial messages, the days of posting, and the centrality of each student. From a quantitative aspect, the social diagram increased interactivity more than the cognitive diagram did, as was expected.

We need to note that the informational aspects of students’ interactions were presented by the cognitive diagram as well. While the social diagram represented all of the interactions without describing content, the cognitive diagram illustrated specific interactions related to certain discussion topics. Based on the survey responses, both the social and cognitive diagrams helped students understand the interactions among themselves, as was intended. However, compared to the cognitive diagram, the social diagram affected student interactivity more. This may have happened because the social diagram emphasized interactivity by representing solely the number of interactions rather than content.

The information illustrated by the social diagram must be considered when interpreting the effects. The social diagram differentiated initial posts and replies by icon size (initial post = big, reply = small). It also aligned all the icons with a day column so students could tell when the messages were posted. Initial posts and replies were connected by arrows, which revealed who responded to whom. The instructor updated the diagram daily and added information until the last day of each discussion. Thus, it is plausible that students would see the progress of interactions as the discussion went on and perceive the information in the diagram as a formative, rather than summative, assessment of their performance.

Limitations

The findings of this study should be interpreted with two limitations kept in mind. First, the participants were graduate students with considerable work experience in a professional field. The topics discussed in this study were related to their professional experiences, and this would have affected their discussion behaviors positively (Johnsen et al. 2004). Second, the number of participants was quite small and the motivation to learn might have been high, which limits the generalizability of the study. Considering the limitations, the researchers carefully analyzed online discussion behaviors from qualitative aspects and quantified the results to see the effects of treatments.

Conclusion

This study revealed that the two discussion diagrams—social and cognitive—affected students’ online discussion behaviors differently, as they emphasized certain distinct features of discussion. Although the study suggests the different effects of the two diagrams, instructors may want to integrate them for better quality discussions as well as more active interaction during discussion. Further research on how the two diagrams can work together in online discussion will give instructors greater insight into the design of discussion tools.