Introduction

Computer-supported collaborative learning (CSCL) has emerged as a research domain focusing on how individuals learn together using a computer (Stahl and Hesse 2006). Stahl (2006) defines CSCL as a pedagogical approach in which learners put co-ordinated efforts to structure knowledge while solving a problem or completing a task. During this process, learners communicate and collaborate using computers and other technological tools in the same or different places and times to accomplish a learning task together and conduct collaborative learning activities. As a consequence of the technological integration in education, CSCL provides considerable contributions to supporting of the in- and out-of-class learning processes as well as enabling students to gain knowledge and competencies such as collaborative learning which is considered as one of the twenty-first century skills, communicating, problem-solving and using digital tools actively (Chen et al. 2018; Kreijns et al. 2003; Kreijns and Kirschner 2004; Resta and Laferriere 2007; Stahl et al. 2006). Therefore, there occurs a growing interest in CSCL.

In order for CSCL to be effective, it is of great importance among the group members to have developed task and group awareness (TaGA). During the collaborative learning process, learners may have difficulties for such reasons as a lower level of TaGA. Students without TaGA may have difficulty in accomplishing what is expected from them in the process of completing group tasks in CSCL, knowing what to do, communicating and collaborating with the group members. In the event of low level of TaGA, group members may develop negative attitude towards collaborative learning in time and experience problems in developing self-regulated learning skills (SRLS) during collaborative learning process (Kirschner et al. 2015; Korkmaz 2012; Miller and Hadwin 2015; Phielix et al. 2011; Yilmaz et al. 2017). Therefore, it is important to design interventions to raise students’ TaGA in CSCL environments (Mühlpfordt and Wessner 2009). One of the design interventions used in CSCL environments is the use of TaGA tools.

The goal of using TaGA tools is to enable learners to assess if they fulfill their responsibilities in terms of group tasks, and their contribution to group performance. Thus, they can compare their own and other group members’ contributions in terms of student-group collaboration by means of the TaGA tools. Additionally, students have a chance of completing/developing group tasks by realizing the shortcomings related to task and group development and eliminate the coordination problems (Dehler et al. 2011). Thus, providing TaGA support to group members allows learners to find solutions for common problems that occur in CSCL environments such as developing a negative attitude towards collaborative learning in CSCL environments and not being able to develop SRLS.

Previous research has established that pedagogical agent (PA) used to provide metacognitive guidance support in online learning environments have various positive effects on the learning outcomes (Yilmaz and Kilic-Cakmak 2012). Similarly, it is suggested that metacognitive support provided to group members by PA increases students’ TaGA and fosters the students’ attitudes towards collaborative learning and SRLS. In this regard, this study aims to examine the effect of group and awareness support provided to the group members via the PA in CSCL environments on students’ attitudes towards collaborative learning and SRLS. This study is expected to contribute to the literature by investigating the use of PA as a TaGA tool in CSCL environments and be a guide for instructional designers, instructors and researchers about the design of a CSCL environment.

Theoretical background, literature review and research hypotheses

The need for task and group awareness tools in computer-supported collaborative learning environment

Task awareness is the knowledge and awareness of an individual among the collaborative learning group on collaborative learning activities. Behaviors expected from individuals in a group when achieving a task and individuals’ knowledge and awareness related to task steps are examples of task awareness. Group awareness, on the other hand, is defined as the existing knowledge of each individual on the engagement and status of other group members to coordinate and complete some part of the group task (Liccardi et al. 2007).

TaGA can be regarded as the essentials for group members to work in coordination and to increase productivity in CSCL environments (Fransen et al. 2011). One of the basic problems in CSCL environments is that students have limited interaction with each other because they are physically separated from each other. This, in turn, may lead to a decrease in the TaGA of the group members. Therefore, researchers point out the importance of using tools to develop group TaGA in CSCL environments (Engelmann et al. 2009). In this regard, researchers have been working on the design of visual tools such as Radar and Reflector, as well as scripts that improve TaGA in CSCL environments. Besides, researchers point to the use of PA to provide awareness support in CSCL environments by considering the advantages provided by the PA in online learning environments (Järvelä et al. 2015, 2016; Strijbos et al. 2006).

Learners often need assistance to stay focused on virtual learning environments (Karaoglan Yilmaz and Yilmaz 2019). It is, therefore, important to guide learners through the PA in these settings. There is evidence to suggest that using PA in virtual environments contributes to learning outcomes (Karaoglan Yilmaz et al. 2018; Kim et al. 2017; van der Meij et al. 2015). Similarly, it can be argued that students can be supported to increase their TaGA, using the PA in CSCL environments.

The role of the PA in CSCL environments is important, as well. A review of the literature has revealed that metacognitive support of the PA in online learning environments yields fruitful learning outcomes (Azevedo and Hadwin 2005; Baylor 2002; Karaoglan Yilmaz et al. 2018). It is noteworthy that the role of metacognitive support may also be crucial for CSCL environments. With the help of metacognitive support related to the TaGA in the planning phase of the CSCL, students can adopt strategies to share the tasks, responsibilities, and coordination, and create an effective plan before they start working on weekly-based tasks. With the metacognitive support of the PA to be provided in the monitoring phase of the CSCL, new strategies can be developed to determine whether the work at the beginning of the process has been going well or not. During the evaluation phase of the CSCL, evaluations can be made about the status of the planned works, accomplishment levels of the planned works and responsibilities of the group (Karaoglan Yilmaz et al. 2018). Thus, the TaGA of the group members will remain high throughout the process with the metacognitive support for planning, monitoring, and evaluation phases.

Attitudes towards computer-supported collaborative learning process and the use of group support systems

Attitudes towards CSCL are of great importance in the process and the use of group support systems. According to Thurstone (1946), attitude is the degree of positive or negative emotions linked to a psychological object. Morgan (1961) suggests that attitudes impact our positive or negative reactions to certain individuals, objects or situations. Therefore, attitudes can be regarded as one of the components that directly affect the learning process and outcomes. Attitudes are also considered as one of the most crucial determinants of the efficiency of CSCL (Chen et al. 2018; Liaw et al. 2008). Attitudes towards CSCL encompass attitudes towards the use of technological tools and environments used in the implementation of collaborative learning activities, as well as the feelings towards collaborative learning activities and group collaboration processes carried out using these tools and environments (Korkmaz 2012, 2013). Recent research provides evidence to suggest that attitudes towards CSCL environment and processes can be effective on several variables related to learning process and outcomes including active participation of students, group efficiency, and product quality (Phielix et al. 2011; Kirschner et al. 2015; Korkmaz 2012). Therefore, it is important to conduct studies to determine the students’ attitudes towards CSCL and to develop their attitudes.

Previous research reveals that one of the important factors for the success of applications used for CSCL is the use of technology and the attitudes towards collaborative learning activities (Korkmaz 2012). However, several studies reveal that students’ unwillingness to actively participate in group discussions in CSCL environments may have an adverse effect on the CSCL process (Nam and Zellner 2011; Korkmaz and Yeşil 2011; Korkmaz 2012). In this regard, interventions to the CSCL design to improve students’ attitudes towards CSCL environments and processes are of crucial importance. However, attitude towards collaborative learning is not investigated sufficiently (Chen et al. 2018; Korkmaz 2012, 2013; Lin et al. 2016) and many researchers suggest that new studies provide important contributions to current literature (Korkmaz 2012, 2013; Nam and Zellner 2011; Yilmaz et al. 2017). Most of the previous research on CSCL has focused on the use of tools and support to optimize attitudes towards knowledge competence, learning, and co-operation (Chen et al. 2018; Korkmaz 2012). Kimmerle and Cress (2008) argue that it is possible to control attitudes utilizing a group awareness tool visually representing the individual contribution, and this can even lead individuals to actively participate in collaborative learning.

A review of the literature has revealed that several design interventions such as the use of scripts, social and cognitive awareness tools, and scaffolding support have existed in CSCL environments. These design interventions seem to have different effects on attitudes. Studies were conducted by Phielix et al. (2010) and Kirschner et al. (2015) who found that groups of students who used social and cognitive awareness tools in CSCL environments exhibited a more positive attitude towards collaborative problem-solving than other groups of students who didn’t use these tools. Phielix et al. (2011) revealed that using awareness tools increased team development, group satisfaction, and attitude. On the other hand, Tsovaltzi et al. (2014) found that providing group awareness support led students to develop negative attitudes. In this sense, there is a limited number of studies in CSCL focusing on the effect of group support systems on attitude. Further research on the current topic is therefore required to fill this gap and provide new insights.

The analysis of the literature has indicated that tools such as scripts, Radar, and Reflector are used as tools of social and cognitive awareness. These tools are graphs employed to indicate the interaction and collaboration among group members and to illustrate them in CSCL. Group members can observe the contribution made by each member, by looking at these graphs. Tsovaltzi et al. (2014) suggest that methods such as promoting group awareness with verbal instruction in a change of attitude may be more effective in the learning process. In this context, it can be observed that PA is used in studies. Previous studies (Dincer and Doganay 2017; Duffy and Azevedo 2015; Park 2015; van der Meij et al. 2015) reveal that PA positively affects students’ attitudes towards learning and their task performance (Yilmaz and Kilic-Cakmak 2012). However, there is no information regarding how using the PA in collaborative learning such as CSCL environments affects students’ attitudes towards collaborative learning. Therefore, further investigation is needed to see the effect of PA on attitudes towards collaborative learning.

With metacognitive support provided to learners in CSCL through a PA, learners can benefit from PA contributing to collaboration, identifying missing points, and eliminating these shortcomings when planning the collaboration process, during the process and afterward. Thus, the support provided to the group will be more coordinated and efficient, thereby developing the attitudes of the members of the group towards a CSCL environment. However, when the existing literature is examined, this hypothesis has not been adequately investigated. Based on the evidence mentioned above about the TaGA support in the CSCL environments through the PA, the first hypothesis of the study has been formulated as:

H1:

There is a statistically significant difference in students’ online collaborative learning attitude scores obtained from the CSCL environment where metacognitive support with the PA is provided.

Self-regulated learning and use of group support systems in computer-supported collaborative learning

Pintrich (2000) describes self-regulated learning (SRL) as a dynamic and constructive process by which students can set goals according to their previous experiences and environment where they work, observe, regulate and control their cognition, motivation, and behavior in line with these goals (p. 453). Schunk and Zimmerman (2007) point out that teachers can promote the development of SRLS through directing, modeling and providing feedback to students. Yilmaz et al. (2017), on the other hand, indicate that it is crucial for learners to develop their SRLS in learning environments including CSCL environments in which students show greater autonomy and responsibility for regulating their learning process. Students may have difficulties in planning the learning process and maintaining it without having SRLS in such environments. It is therefore important that certain interventions should be taken into consideration to improve the SRLS of learners in virtual environments.

In CSCL environments, it is seen that the learners use scripts, Radar, and Reflector as social and cognitive awareness tools to support their development of SRLS (Miller and Hadwin 2015). The aim of using these tools is supporting to coordinate the collaboration among group members by increasing the awareness of group members about tasks and groups in CSCL environments (Bodemer and Dehler 2011; Miller and Hadwin 2015). Several researches have shown that awareness support has positive effects on the self-regulation and knowledge structure of students (Buder and Bodemer 2008). Lin and Tsai (2016) found that awareness support in the online project-based learning environment had temporary positive effects on students who had low self-regulation skills, whereas it had sustainable positive effects on students who had high SRLS. Gijlers et al. (2013) uncovered that providing TaGA assisted in the collaboration environment provided better learning outcomes and, increased non-task and coordination-related activities. Kirschner et al. (2015) revealed that using group awareness support in CSCL increased awareness and group satisfaction reducing the level of conflict among group members (lower levels of conflicts). However, the use of these tools did not effect outcome quality according to their study. According to another study conducted by Pifarré et al. (2014), the use of group awareness tools in CSCL enhanced cognitive, metacognitive and participation in learning activities of students. Tsovaltzi et al. (2014) revealed that the use of group awareness support and argumentation scripts in a social network-based discussion environment developed individual argument elaboration. Kimmerle and Cress (2008) found that using group awareness tools enhanced students’ awareness about the individual contribution to group processes and provided the opportunity for self-recognition. There is also evidence that learners benefit from PA in the development of SRLS in online learning environments. Karaoglan Yilmaz et al. (2018) suggest that the metacognitive support provided to the learners with the PA at the planning, monitoring, and evaluation purposes develop their SRLS.

Researchers propose a variety of strategies to develop learners’ SRLS. One of these strategies is related to the regulation of cognition. The regulation of cognition is metacognitive activities that help one to control thinking or learning (Schraw and Moshman 1995). The regulation of cognition consists of three steps: planning, monitoring and regulation strategies (Hofer et al. 1998; Pintrich 1999; Schraw and Moshman 1995). Considering a CSCL environment and processes, these strategies can be provided by the PA. With the metacognitive support provided by PA in CSCL, it is possible to enable the students to plan, monitor and evaluate their learning process. Through metacognitive support to be provided to the students during the planning stage of the CSCL, students have heightened awareness of the task and the group and make feasible plans related to work and coordination of group tasks. By providing metacognitive support during the monitoring phase, awareness of fulfilling individual and group-based responsibilities is increased and whether or not the works are in progress as planned can be determined. By providing the metacognitive support during the evaluation phase, it is possible to determine the extent to which the responsibilities related to the task are carried out, reveal unfulfilled responsibilities and incomplete tasks and develop new strategies for overcoming these deficiencies. Thus, the student can master the entire collaboration process on an individual and group basis and has opportunities to develop his/her SRLS. This is because PA will be able to provide feedback to group members to develop awareness as individuals and groups, and they will be able to see the deficiencies in the process and take measures to eliminate them. The continuity of this situation may add to the augmentation of SRLS of group members in the process. It is an assumption that the metacognitive support to be granted by the PA in the process of CSCL will develop the SRLS of learners. Upon reviewing the existing literature, the second hypothesis of the study has been formulated as:

H2:

There is a statistically significant difference in students’ SRLS scores arising from the CSCL environment where metacognitive support with the PA is provided.

Purpose of the study

Based on an extensive review of the literature on PA-based awareness support in CSCL environments and self-regulation, it can be noted that PA-based awareness support can be useful in the development of attitudes and SRLS. However, further research is required to examine the effects of PA-based metacognitive support in CSCL settings on the learning process and outcomes. In this regard, this study aims to investigate the effect of the metacognitive TaGA support to group members through the PA in a CSCL environment on the members’ attitudes towards collaborative learning and their SRLS.

Method

Research model and participants

This study was conducted using a quasi-experimental mixed-method research design through pretest and posttest control groups. The population consisted of 42 first-year Political Science students enrolled in the Computing II course at a public university in Turkey. After getting their consent, the participants were randomly assigned to two groups, which included 22 participants in the experimental group. The male participants accounted for 35.7% (n = 15) of the study group in the study, and the rest were female 64.3% (n = 27). Their ages ranged from 18 to 20. Each student participating in the study had internet access, a laptop computer and a smartphone. In the previous semester, the students took Computing I course and learned basic knowledge on electronic presentation, internet, and communication, etc. Therefore, it can be stated that students had basic computing skills and knowledge. Figure 1 summarizes the experimental process.

Fig. 1
figure 1

Research model

Full size image

Data collection instruments

The SRL scale was used as a data collection instrument in both pretest and posttest, whereas the online collaborative learning attitude scale was only used in the posttest. Additionally, a semi-structured interview form was developed to reveal students’ opinions following the procedure.

The online collaborative learning attitude scale developed by Korkmaz (2012) was used to determine students’ attitudes. The scale is a 5-point Likert type and includes 17 items. Each item in the scale is scored between 1 and 5 according to the criteria of “Never: 1” and “Always: 5”. It has a two-factor structure, including positive and negative attitudes. The Cronbach’s alpha value for the whole scale was measured as 0.90. The Cronbach’s alpha values of sub-dimensions were found to be 0.89 for a positive attitude and 0.91 for a negative attitude. The reliability values were calculated for this study. The higher scores on the scale indicate that the attitudes of students towards collaborative learning are positive. Since the questions in the factor called “negative attitude” were negative, these questions were analyzed by reverse coding. As a result, the increase in the scores on the “positive attitude” factor increases the positive attitude, while the increase in the scores on the “negative attitude” factor means the decrease in the negative attitude. The scale was used in the posttest in the study.

The SRL scale generated by Haslaman and Askar (2015) was employed to compare students’ SRLS. The scale is a 10-point Likert type and has 59 items. Each item in the scale is scored between 1 and 10 according to the criteria of “It does not reflect me at all: 1” and “It reflects me exactly: 10”. The Cronbach’s alpha value for the whole scale was measured as 0.97. The Cronbach’s alpha values for the sub-dimensions are as follows; forethought 0.90, self-control/self-regulation 0.91, self-observation 0.90, and self-reflection 0.93. The reliability values were calculated for this study. The higher scores on the scale show higher levels of SRLS, whereas lower scores on scale refer to lower SRLS. The scale was employed in the pretest and posttest phases of the study. A review of the literature has revealed that several measurement tools on students’ SRLS were adapted into the Turkish language, and their validity and reliability tests were conducted. However, it should be noted that most of them were suitable for younger students, such as primary or elementary. There are few scales adapted into the Turkish language which are suitable for adults. This study was designed to investigate students’ SRLS in CSCL environments so the only appropriate scale was Haslaman and Askar’s (2015) scale that was developed to measure the SRLS of adults in online learning environments. As a result, this instrument was used in this study.

To obtain in-depth qualitative data on CSCL environments, a semi-structured interview form was developed by the researchers. The questions were formed by the review of the related literature. Additionally, the sub-dimensions of two data collection instruments in this study were consulted while preparing the questions. To ensure the validity of the interview form, expert opinion was asked, and the final version of the form was designed.

Procedures and computer-supported collaborative learning environments

In the current research, Moodle mobile learning management system (MLMS) was employed as a CSCL environment. Besides, a TaGA tool supported by pedagogical and a wiki tool were integrated into Moodle MLMS. The research was implemented during the Computing II course and lasted for 8 weeks. At the beginning of the research period, participants in the experimental and control groups were asked to form groups of 3–5 persons on their own. After the subgroups were formed in the first week of the research period, the participants in the experimental and control groups were notified about this environment and tasks they would carry out. In this regard, students were asked to prepare and submit a weekly assignment on digital citizenship (access, communication, health, etc.) in a wiki environment. At the beginning of the process, the researchers explained what students should do and what they should pay attention to such as requiring students to apply the material design principles to construct the subject and contents. In this sense, the participants were provided with the necessary explanations about the multimedia design principles (Mayer 2001) and the conditions of learning theory (Gagne et al. 1992) at the beginning of the procedure. The participants were required to stick to the multimedia design principles (coherence principle, signaling principle, etc.) while designing the wiki pages and the contents (writing, pictures, animations, video, etc.) placed on these pages (Mayer 2001). Additionally, the participants were required to adhere to these principles (gaining attention, informing learners of the objective, etc.) while designing the wiki pages in terms of the conditions of learning theory. The groups collaboratively worked during this knowledge construction process. To enhance and maintain the students’ engagements and motivations both in the experimental and control groups for the collaborative learning process, it was declared that the outcomes of the research would be evaluated as course performance. Thus, it was attempted to make the students focus on the product and to enable them to transfer the theoretical knowledge into the practice.

Awareness tool supported by the PA was applied individually to each student to help them plan tasks and group works before working in a group at the beginning of each week and to help them assess tasks and group works at the end of each week. At the beginning of each week, students were given a link “Let’s Have a Look before the Group Work!” in MLMS and answered several questions supported by metacognitive guidance by PA. The aim of these questions was enhancing the awareness of participants about their tasks of the related week, their collaboration with the group, and the planning of tasks and group. After the groups responded to these questions on the preliminary form, they started their knowledge construction in the wiki. This applied for a week. Then, each team examined each other’s wiki environment and discussed both material design principles and structured topic contents using the discussion board of the wiki. At the end of each week, the TaGA tool supported by PA was reapplied to the students by the link “Let’s Evaluate the Group Work” in MLMS. To raise group awareness, questions were asked about how well the members of the group fulfill their responsibilities, how group cohesion is, good and bad aspects of the group, solutions to problems, etc. questions were asked to raise group awareness. A sample of metacognitive awareness support via the PA used to increase group and task awareness is given in Fig. 2. Similarly, metacognitive awareness support continued with the PA each week in the experimental group. CSCL groups in the control group did not receive the support of the PA.

Fig. 2
figure 2

An example of task and group awareness support with the pedagogical agent and learning environment

Full size image

The TaGA support by a PA illustrated in Fig. 2 is an example of the support provided at the end of the weekly collaborative learning process. Accordingly, after the students in the collaborative learning group completed their tasks in the wiki environment in line with their weekly schedule, a link “Let’s Evaluate the Group Work” became active in the Moodle environment. By clicking this link, the students in the collaborative learning group individually answered the questions related to weekly group work and collaboration. In this sense, the students were asked these questions by the PA: “Have you completed your weekly tasks in line with your weekly schedule?”, “Have you had any impediments during this week’s task and group process?”, If so, what strategies have you developed to cope with them?”, “What are the most difficult situations for you during the task and group collaboration process?”, “Are there any points to be completed/improved considering this week’s collaboration process?”, “If so, what will you do to improve these?”. In each week, these questions were changed to avoid making the activity boring. It was attempted to have the students to evaluate the weekly tasks and collaboration, notice any barriers that impeded carrying out the tasks and making collaboration and overcome these barriers, if they exist, by bringing solutions to them. To complete the “Let’s Evaluate the Group Work” activity, the students continued to work together until the next week’s theme. Therefore, this activity helped students to realize the drawbacks of their completed tasks and accordingly redo incomplete parts of the tasks. Figure 3 summarizes the weekly workflow diagram of the CSCL process planned for students in the experimental group.

Fig. 3
figure 3

Flow diagram for each of the levels

Full size image

The interfaces of the PA used in the CSCL environment were designed in Toondoo and Photoshop programs. Then, gestures and mimics were added to these designed interfaces, and CrazyTalk program was used to add an animation feature. Voice recordings and thought bubbles were also added to the PA. So, the PA interacted with students in the form of voice and text while asking metacognitive questions.

Data analysis

To test the normal distribution of the scores obtained from the scales, the Kolmogorov–Smirnov test was employed. It was concluded that the data set followed a normal distribution (p > 0.05). Since the samples met certain preconditions of parametric tests, they were employed in the statistical analyses. The pretest scores of the participants obtained from the SRL scale were controlled to check if there was a significant difference between posttest scores. To make comparisons, the ANCOVA test was employed. The T-test was conducted to compare online collaborative learning attitude scale scores obtained from posttest measurements. To analyze the qualitative data, a content analysis method was used. Opinions of the students on the support provided by the PA in the collaborative learning process were analyzed according to their responses given in the interview forms. The opinions of participants obtained through interview forms were transcribed, coded and analyzed.

The questions related to the PA’s contribution to TaGA, the disadvantages of its use in the CSCL environment, and what can be done for a more effective and efficient PA support were directed to the experimental group students in the interview form. These interview questions were designed to reveal the students’ opinions on the support of the PA. A content analysis method was employed to analyze the data obtained through semi-structured questions. To perform content analysis, obtained data were classified following a comprehensive examination and tentative codes were formed by the researchers. Obtained data were transcribed by one researcher, and the data set was re-coded by another researcher to achieve reliability. The researchers who performed the content analysis have expertise in the field of educational technology. All researchers coded each segment of qualitative data. The reliability of the encoding was calculated by dividing the number of same codes given by both researchers by the total number of codes (Miles and Huberman 1994). The coding process displayed 92% reliability. The researchers tried to reach an agreement for the 8% difference. The statements of the participants were revisited to check the difference and it was identified that 8% difference stemmed from the fact that some of the answers can be categorized under different sub-themes. The following interview questions were framed for the qualitative section of the study:

(1) What is the contribution of the PA in the CSCL environment to the group work and collaboration? (2) Are there any handicaps or restrictions of using the PA in the CSCL environment? If yes, what are these? (3) Which regulations (What amendments) could be applied to increase the effectiveness of the support provided by the PA in the CSCL environment?

Findings

Analysis of online collaborative learning attitude scores

In order to test the first hypothesis, the study tested if there was a statistically significant difference in online collaborative learning attitude scores obtained from the experimental and control groups. To do this, the posttest scores of the participants in both groups were checked through the T-test. Table 1 summarizes the T-test results concerning online collaborative learning attitudes.

Table 1 The results of T-test analysis related to online collaborative learning attitude posttest scores
Full size table

As shown in Table 1, there is a statistically significant difference in the online collaborative learning attitude scores of the participants between the experimental group (M = 69.86, SD = 10.42) and the control group (M = 61.25, SD = 11.80). Accordingly, it can be noted that participants in the experimental group had higher means in online collaborative learning attitude scores of the students with a medium to large effect size when compared to the control group (t(40) = 2.37; p < 0.05; Cohen’s d = 0.73). In other words, the online collaborative learning attitude showed differences depending on the PA with metacognitive support. The results concluded that the online collaborative learning attitude of the experimental group differed significantly from those of the control group.

Regarding the items of the online cooperative learning attitude scale, item-based comparisons were made according to the responses given by the students in experimental and control groups. The results of item-based comparisons were consistent with the T-test results. In other words, it is seen that the online collaborative learning attitude mean scores of the students in the experimental group were higher than those of the students in the control group. In this sense, the metacognitive support provided to group members by the PA increased the students’ TaGA and fostered the students’ attitudes. The item-based comparison tables of the scales are presented in the appendix section of the manuscript (Appendix 1).

Analysis of self-regulated learning scores

To test the second hypothesis, the study tested if there was a statistically significant difference in SRLS scores obtained from the experimental and control groups. The covariance analysis was employed by using SRLS pretest scores as the covariant, SRLS posttest scores as dependent and CSCL environment as independent variable to explore for differences experimental and control groups’ SRLS.

The mean of the experimental group was M = 389.36 from the SRL scale pretest (SD = 110.95) and M = 446.59 (SD = 76.69) from the SRL scale posttest. Also, the mean of the control group was M = 338.95 (SD = 113.74) from the SRL scale pretest and M = 418.1 (SD = 98.99) from the SRL scale posttest.

As a result of the ANCOVA test, it was found that the adjusted means of SRLS were M = 440.69 for the experimental group, while it was M = 424.60 for the control group. ANCOVA was carried to check for a significant difference in SRL posttest scores. The obtained results are displayed in Table 2.

Table 2 The results of covariance analysis related to students’ self-regulated learning skills
Full size table

As shown in Table 2, upon examining students’ SRL scale pretest–posttest scores, no significant difference between experimental and control groups [F(1, 39): 0.36; p = 0.552 > 0.05; \({\eta^2}\) = 0.009; observed value: 0.090] was found. In other words, SRLS does not differ statistically depending on the PA with metacognitive support.

Regarding the items of the SRL scale, item-based comparisons were made according to the responses given by the students in experimental and control groups. There was evidence to suggest that the students in both experiment and control groups had higher scores on the posttest when compared to pretest scores. However, this increase in posttest scores according to pretest measurements was close to each other in the experiment and control groups. The results of item-based comparisons were consistent with the ANCOVA results. In other words, the online collaborative learning attitude mean scores of the students in the experimental group were close to those of the students in the control group. The item-based comparison tables of the scales are presented in the appendix section of the manuscript (Appendix 2).

Findings related to qualitative data

In the qualitative section of the study, students’ opinions about the effects of metacognitive support granted by the PA on the group work process were investigated. The participants’ voices revealed that agent support during the learning process helped them make evaluations about the group, increase their awareness on tasks and collaboration, as well as enabled them to have closer relations with other group members. Additionally, it was found that the support of the PA increased the sense of responsibility for the tasks and collaboration, ensuring that the tasks and collaboration were made more regularly and that the knowledge sharing between the group members was improved. Moreover, it was concluded that the support of PA in the CSCL provided an opportunity to see and manage learning deficiencies/errors as an individual and also as a group. On the other hand, when the disadvantage of the PA support system was asked, no disadvantages were uttered by the participants except a few ones who mentioned that preparing a report every week for the PA’s questions and directions could be tedious. Some of the participant views are as follows:

Q1::

“It provided information to the people in the group. It made some tasks easier to do as a group.”

Q2::

“It made us see how helpful our friends are and this contributed to our group cohesion.”

Q3::

“It has allowed us to see how we should approach a topic and how we can get through something together.”

Q4::

“It made everyone realize their mistakes.”

Q5:

“It was boring to report every week.”

When asked how PA support might be better for students, most of the students’ responses show their satisfaction with the use of the PA support system at CSCL. Also, other student responses were that the questions asked by the PA should be individualized, that individual guidance should be made according to the given answers, that scaffolding support can be provided for the applications. Some of the participant views are as follows:

Q1::

“I would like it to be instructive and evaluative as we are currently using.”

Q2::

“Short hands-on demonstration videos could also be added for our missing points.”

Discussion and theoretical implications

The purpose of this study was to investigate the effects of TaGA support provided by the PA to group members in a CSCL environment on students’ attitudes towards collaborative learning and SRLS. The results of the analyses revealed that the students in the experimental group had significantly higher scores in terms of online collaborative learning attitude than the control group. This evidence suggested that TaGA support with the PA in the CSCL environment was useful in developing collaborative learning attitudes. The results of the qualitative analysis supported this finding. Qualitative findings indicated that TaGA support with PA provided more connection among group members, increased awareness of tasks and collaboration, increased responsibility for tasks and collaboration, made tasks and collaboration more organized, and fostered the knowledge sharing between group members. In this sense, the group atmosphere and cohesion may have contributed to the development of group members’ attitudes towards collaborative learning. In the literature, some studies align with this interpretation. In the works performed by Phielix et al. (2010) and Kirschner et al. (2015), it was found that groups using social and cognitive awareness tools in CSCL environments displayed a more positive attitude towards collaborative problem-solving. Phielix et al. (2011) also concluded that using TaGA tool enhanced team development, group satisfaction, and improvement in a positive attitude.

Several studies on CSCL found that various support strategies such as collaboration scripts (Miller and Hadwin 2015; Noroozi et al. 2013; Vogel et al. 2017), peer feedback (Phielix et al. 2010) and role assignment (De Wever et al. 2007, 2009; Yilmaz and Karaoglan Yilmaz 2019) were useful in various contexts. The results of these researches suggested that using support strategies made an assessment of group easier, developed awareness of task and collaboration, enabled the group members to connect more easily, facilitated the social structuring of knowledge, improved knowledge sharing and coordination, and directed the students’ perceptions about the group in a positive way.

In our study, guidance and support were provided for the effective planning of tasks and collaboration before the PA-based collaborative learning process started. This enabled participants to monitor the development of their plans, define the deficiencies, and develop strategies for eliminating them. Since PA support was provided similarly throughout the process, the students stated that they started to act more responsibly and constructively about the tasks and collaboration in time. In the meantime, depending on the way the group work, group members began to become more connected to each other, adopted group tasks and collaboration in a better way. Additionally, the group cohesion among the group members and the group atmosphere began to develop positively. These positive emotions related to the group and collaboration was positively reflected on the students’ attitudes towards online collaborative learning. Accordingly, it was observed that the attitudes of experimental group students toward online collaborative learning were statistically significant in a positive way at the end of the CSCL process.

From data analysis drawing upon qualitative analysis, the support of the PA in the CSCL process contributed to make evaluations about the group, raised awareness about the task and the group, ensured that the group members were connected, developed a sense of responsibility, enabled participants to perform the tasks in an organized way, and improved the knowledge sharing among the members. These were mostly related to increasing TaGA, improving collaboration and creating a positive group atmosphere. In other words, it was observed that the support of the PA allowed group members to increase their awareness of the group and to contribute to the collaboration. Members of a group planed their tasks and group collaboration concerning the support of the PA and reviewed to what extent they performed their responsibilities. However, this review activity was in the form of completing the tasks according to the questions asked by the PA and responding to the questions of the PA, by reviewing their actions in the group contribution to the questions. In other words, students’ self-evaluation was based on their self-reports. Since students were not informed on the results of an evaluation that objectively compared the contributions of each student to the task and group, students in the group may have thought that they did their best to carry out their responsibilities. In this case, the feedback provided by the PA may have inadequately improved their SRLS. As in visual tools such as the peer feedback tool (Radar) and the reflection tool (Reflector) used in collaboration environment (Phielix et al. 2010; Pifarré et al. 2014; Kirschner et al. 2015; Sangin et al. 2011), a system of PA comparing students’ contributions to the task and group, and accordingly, giving feedback to students were more effective in developing their SRLS. The effectiveness of this assumption can be further studied in future research. Besides, feedback provided by the PA in this study aimed to increase the TaGA of the group members and make the group coordination and collaboration more effective. Therefore, the content of the feedback provided by the PA was created in this direction. Depending on the content of the feedback, this may not have had a direct impact on students’ SRLS. Indeed, in the study of Karaoglan Yilmaz et al. (2018), it was found that feedback provided by the PA to develop students’ SRLS in online learning settings for planning, monitoring and evaluation purposes promoted developing those skills. Therefore, it is thought that the contents of the feedback to be provided in CSCL environments in future studies will be useful for both improving students’ SRLS and increasing the TaGA.

Another finding was that TaGA support with the PA in the CSCL environment did not result in a significant difference in the SRLS of the experimental group students. In other words, no significant difference between SRLS of the experiment and control group students was found at the end of the CSCL process although the related literature suggests that providing awareness support in the CSCL environment mostly helps to improve SRLS. Buder and Bodemer (2008) concluded that awareness support positively influenced students’ SRLS. Lin and Tsai (2016) also put forward that awareness support had temporary positive effects on students with low SRLS and sustainable positive effects on students with high SRLS.

There may be several reasons why the findings from the study on the effect of PA support on SRLS were different from the literature. First, the difference was because of the type of tool used for awareness support. In the existing literature, visual tools such as Radar and Reflector are employed as awareness tools to show the collaboration and interaction behaviors of the group members. In other words, all the members of the group can see their interaction behavior. In this research, on the other hand, the PA asked questions to the group members individually in the CSCL process and received their responses based on self-reported. In other words, students observed their learning processes, but they could not see the contributions of others to group processes and others’ interactions with each other. Learners may have believed that they were performing their tasks properly and that their contributions to the CSCL process were adequate because they gave appropriate responses to the PA’s questions about the TaGA. For this reason, students may have had difficulties in improving their SRLS. Indeed, findings from qualitative data analysis revealed that even if the PA provides self-monitoring and regulated opportunities for learners while contributing to group co-operation processes, it does not lead to such a significant difference in SRLS. Although the findings obtained from the qualitative data analysis showed that the PA enabled learners to monitor and regulate themselves and contributed to the group collaboration processes, it was observed that this did not make a significant difference in the scores of SRLS. Yilmaz et al. (2017) verified this result by reaching the conclusion that a group transactive memory system did not add to boost SRLS of individual group members in the collaborative learning process, even if group cohesion and atmosphere were well developed.

Another reason why learners did not improve their SRLS may be that the guidance of the PA on tasks and group processes was generic and not personalized/adaptive. Students may not be able to see where they failed in the process from the general guidance and directions of the PA, and therefore may not be able to resolve their learning deficiencies/mistakes. For this reason, students may not be able to improve their SRLS at the expected level. However, since awareness tools such as Radar and Reflector visually indicate collaboration behaviors of team members and their contributions to the team in the CSCL process, learners can deduce their individual information from this and can arrange their learning behaviors accordingly. Indeed, in the analysis of the qualitative data obtained from the students, it was found that students expected the PA to give personalized feedback. In further CSCL studies, exploring the effect of providing personalized feedback with artificial intelligence supported PA would be useful.

Practical implications

The findings of the current study have considerable implications. Several recommendations can be done for the development and utilization of CSCL. First, it is important to benefit from PA support in CSCL environments as well as in personalized learning environments. The PA plays a significant role for students, so it is useful in providing metacognitive support to organize their cognitive processes. While the PA provides metacognitive support, it would be appropriate to provide planning purposes for determining tasks, planning collaboration and ensuring coordination at the beginning of the CSCL process and also at the end of the CSCL process, for the evaluation purposes of assessing the fulfillment of the tasks, evaluation of collaboration and coordination, determining the deficiencies and addressing them. The support granted by the PA in the context of the study is mostly linked to the development of TaGA. This support appears to have raised students’ attitudes towards collaboration positively. Besides, the content of PA feedback needs to be diversified and differentiated accordingly to boost individual SRLS of the participants. In this regard, PA in a learning environment is not merely in the role of increasing TaGA, but in the role of diversified support and guidance such as facilitator, information provider, consultant/mentor, critic, motivational support as has been suggested in the literature as well (Baylor and Kim 2003; Chou et al. 2003; Yilmaz and Kilic Cakmak 2012).

Limitations of the study and suggestions for further research

Our research may have several limitations and it is plausible that they could have influenced the results. The first limitation is that the study group was a small one. In further studies, a similar research design can be used on larger samples to increase the generalizability of research results. Additionally, the participants were college students, so it can be noted that they are comparatively more autonomous and self-regulated learners than K-12 students. Future research, therefore, may recruit younger ages of students, including primary or secondary levels. As a result of such studies, it would be possible to obtain ample evidence for making comparisons between different grades of students. Thus, the impact of metacognitive support provided by the PA can be compared for different study groups. Another limitation of the research is related to the lack of interaction among participants in the experimental and control groups. Even so, it could have been possible for the participants to have interacted with each other during the process. For this reason, it is suggested to research distance education courses in which students in experimental and control groups do not have the chance to have face-to-face interaction with each other. A further limitation of the study is related to the failure to measure attitudes through a pretest. This is because the participants did not have a collaborative learning experience in the CSCL environment and had not developed positive or negative attitudes towards online collaborative learning. We suggest that further research should focus on participants with CSCL experience to control and measure pretest attitude scores. The present research is related to metacognitive support granted by the PA during the CSCL process aimed to increase students’ TaGA. Although quantitative research findings revealed that this did not yield expected benefits to SRLS of students, the content of the metacognitive support granted by the PA is believed to be effective on SRLS. This is because the content of this support is intended to increase the TaGA rather than to direct SRLS. In future research, the effectiveness of providing PA support to develop students’ SRLS can be studied in the CSCL. Additionally, future research should explore the impact of attitudes and SRL on learning performance, by examining the relations between learning performance results and students’ attitudes towards collaborative learning as well as SRLS.

It is important to provide personalized/adaptive feedback and guidance by the PA to enhance the effectiveness of TaGA support. It is also important that the feedback provided by the PA to be presented in visual feedback, such as learning analytics to show the social interaction of the group members. Thus, group members will be able to visually see their contributions to group processes, the interaction, and contributions of other members. In future research, these design elements should be integrated into the CSCL environment to investigate their effectiveness. Data drawn from the qualitative analysis suggest that students expected the scaffolding support in some tasks from the PA. In this regard, future research should investigate the effectiveness of scaffolding support provided by the PA, particularly in performance and application-based tasks.

Conclusion

The effect of the use of PA-based metacognitive awareness support in the CSCL environment was investigated to students’ attitudes towards online collaborative learning and SRLS in this study. The findings from the study clearly showed that providing PA-based metacognitive awareness support positively affected students’ attitudes towards collaborative learning in the CSCL environment. The study, on the other hand, concluded that the metacognitive support provided to increase the students’ awareness of the task and group in the CSCL environment did not reveal a statistically significant difference in the SRLS of the students. Qualitative analysis of the data, the findings highlighted that the primary benefits of providing PA-based metacognitive awareness support in CSCL were as follows; helping to make an assessment of the group, increasing awareness of their tasks and collaboration, increasing the group members’ connection with each other, increasing their sense of responsibility for their tasks and collaboration, providing more organized tasks and collaboration, improving knowledge sharing among group members, giving an opportunity to see other’s contributions to the group, giving the opportunity to see and eliminating learning deficiencies as individuals and as a group. This study broadened the current understanding of guiding instructional designers and practitioners in the setting of the CSCL environment.