Keywords

1 Introduction

Thanks to various studies, it has been proven that the computer-supported collaborative learning [1] (CSCL) it favors individual learning and fosters social skills, it has also been proven that students who work collaboratively develop better attitudes towards the learning process, dedicate more time to the task of learning, are more tolerant, listen more to the opinions of others and have better negotiation skills. In addition, from the teacher’s viewpoint, the use of computers as a learning tool allows for a more detailed follow-up of the process, since the different tools and applications can incorporate a record of the activities. In this way, the teacher can review the process that each student has followed in their learning, guide the process and observe the mistakes made [2]. On the other hand, the analysis of the collaboration allows measuring the interactive process that takes place during a collaborative activity [3]. From this viewpoint, it is important to be able to detect what degree of collaboration is having or has taken place during the group learning process [4]. To guide the learning process, according to [5], is divided into three phases: Pre-Process, Process and Post-Process. The first phase Pre-Process begins with the activity design and specification, in addition, the activity content, the main tasks and the objectives to be achieved by participating groups are designed. In the Process phase, the collaboration activity is executed to achieve the objectives, where each member develops collaborative skills such as explanation, argumentation, regulation, negotiation, communication among others. At the end of the activity, in the Post-Process phase, the activity coordinator performs an individual and collective review to verify the achievement of the proposed objective. Currently, there are proposals that include some of the elements or techniques that constitute the collaborative learning ambit [6,7,8], which does not cover all the design needs of collaborative activities and much less monitoring and evaluation mechanisms. Consequently, the need arises to create a proposal that, with technological accompaniment, guides the collaborative learning activities execution, to improve the collaboration outcomes of the students involved process [9]. For this reason, in this paper, we propose a refinement to the Process stage through of its elements (activities, roles, inputs, outputs and, monitoring and evaluation mechanisms), also, we validate this refinement through its usefulness, applicability, and ease of use. According to the validation, it can be considered that our proposal is useful and applicable, but despite this, it was found that due to the amount of information necessary for its application it does not ease of use.

This paper is structured, section two: related work, the section three: redefinition of the Process phase elements, which shows the evolution of each element, section four show the application of this phase through a case study and finally the section five the conclusions and future work section.

2 Related Work

Ramirez et al. [10] presented a guide for the computer-supported collaborative learning activities design called CSCoLAD, which used a web tool that provided a mechanism to support all collaborative learning process. The guide design was based on theoretical references, such as those proposed by Johnson et al. [11] regarding the phases that are incorporated in the collaborative learning process and the definition that it proposes about collaborative activity and its characteristics, adding activities given by the expert’s experience. This work helped as a basis for the redefinition of the Process phase elements show in this paper and for the addition of monitoring and evaluation elements, thus allowing increased collaboration among the activity members, concepts that were not considered in this first version presented in CSCoLAD.

On the other hand, the method presented by Chacón [12], enable the collaborative activities structuring and thus stimulate the technology incorporation more efficiently in teaching and learning processes, taking advantage of Web 2.0 tools to design and develop collaborative activities. Lund [13] proposed a model for designing teaching-learning activities, in collaborative and geographically distributed environments, in an experimental format. This model, called CODILA+A, was oriented to universities that dictate computer science careers, this model has a template that guides the teacher in the activities design, its exposed guidelines or characteristics and revision for each collaborative activity carried out. A script called Strategic Guide for Problem Solving proposed by King [14] was designed to foment student interactions when solving complex problems, which was based on a strategic question sequence that guides the participants in the problem-solving activity, controlling the content of their interactions while solving the activity together. Fischer et al. [15] proposed a study on the convenience of using scripts in the collaborative activities design and development. In this study a series of experiments with different groups was done, obtaining that some of these development activities with the scripts support and others without such help. Gallardo et al. [16] designed a computational environment to support one of the most commonly used techniques in collaborative learning environments such as JIGSAW [4]. This type of tool was linked to the collaborative learning technique and has not been structured in a generic way so that it can adapt to any other technique. Hernández et al. [17] presented a high-level tool for the collaborative activities design based on patterns, called COLLAGE, created with the purpose of helping teachers in the process of creating their collaborative designs through the reuse and patterns customization, in order to that they are effective and adapted to the needs of a learning situation.

3 Redefining the Process Phase Elements

As shown, Ramírez et al. [10], presented the first version of the activities for each process phase set forth in [5], which generated a guide for the collaborative learning activities design assisted by computer. The steps of creating the guide followed by Ramirez et al. [10] was: the first step was the study of the process for the collaborative activities design proposed by Johnson et al. [18], after, they analyze the classification of these activities in the Pre-Process phases, Process and Post-Process. With the support of the authors such as Dillenbourg [1], Johnson et al. [11], Lund [13], and Collazos et al. [5], a proposal is carried out, obtaining a CSCoLAD guide first version. This proposal is repeatedly evaluated by experts in the collaborative learning area, who propose improvements and new recommendations that will be adapted to the guide, in order to obtain a CSCoLAD guide final version. Once this final version is available, it is described in the format of patterns, how to execute each one activity exposed in the guide, in order to facilitate the interpretation, within a collaborative learning environment.

Starting from the guide defined in CSCoLAD project and the deficiencies related to maintaining collaboration during the activity execution according by Collazos [9], it was observed the need to monitor and evaluate the collaborative learning process, through by incorporation of new elements at process for increasing the collaboration between the participants, it is for this reason that a redefinition of the Process phase elements and of the CSCoLAD guide was carried out. For the definition of our proposal of refinement, we define some stages, the launch stage: where approval was given and the proposal to be made communicated, after, the definition stage: the characteristics at the group level were defined, activities, and technology. Evaluation mechanisms for the activities in each phase were defined for CSCoLAD and templates were created for data collection. A formulation stage: conceptual models were developed of how the collaborative processes were executed with its activities, groups, tools where the process possible improvements were identifying. A metrics stage: a set of indicators and collaboration metrics were described. It was defined which factors that must be considered for the creation of the groups, the activities design, and the tools. A mechanisms stage: mechanisms that allow the evaluation and monitoring of the collaborative process were created which were presented in [19]. According to the previous stages, we obtained the first refinement, then, it was evaluated by experts in the collaborative learning area, who proposed some improvements and recommendations that were incorporated. In this first refinement was implemented a software tool that supported each process phase. An improvement stage: it was defined an improved conceptual model of the collaborative learning process that allowed us solving the shortcomings found in activities, roles, and tools that were part of collaborative learning, in addition to having a formal specification of the activities of Process phase presented in [20], a test stage: within order to evaluate and validate the proposed conceptual infrastructure field tests were elaborated in various classrooms, using like supporting the defined tool in [21]. Finally, we obtained recommendations and results that allowed us to improve our proposal.

According to the stages that were shown previously, as one of the results, some deficiencies and improvement opportunities in CSCoLAD were identified (See Table 1):

Table 1. Improvement opportunities
Full size table

To support the improvement opportunities identified in the previous table, each activity of the process phase was assigned mechanism to execution, an application strategy, responsible, inputs, outputs, evaluation mechanisms and monitoring mechanisms. Table 2 shows a summary of the most important items assigned to each activity.

Table 2. Monitoring and evaluation mechanisms for the Process phase
Full size table

In order to verify that the proposed elements were adequate, seven experts in the area were asked to do an evaluation where they were requested that each activity, each monitoring and evaluation mechanisms are given a score: 1–5 considering the relevance that each one of them (1 is little relevant and 5 highly relevant). With the results obtained by the experts, the arithmetic calculation of these values is applied, those activities that have an inferior value to those defined in the limits, they were discarded in the final proposal of this work (In Table 3 shows the value obtained by the experts in each activity):

Table 3. Arithmetic calculation of the experts
Full size table

The criterion of Statistical Fashion (Mo) was used to find the value that has the highest absolute frequency in the distribution of data, and thus define the threshold with which it is statistically determined which elements of the refinement are included and which will not be included according to expert validation. The threshold value was Mo = 3.72, the activities that have a value lower than this will not be included in the refinement. For this reason, it can be determined that all activities defined are relevant to the Process phase, considering that recommendations were given for some activities which allowed to enrich the phase elements. Finally, for each activity was specified, subtasks, steps, roles, inputs, outputs, monitoring, and evaluation mechanisms. In Fig. 1. is shown each activity with its subtasks associated:

Fig. 1.
figure 1

Activities with its subtasks associated

Full size image

As an example, below is shown only the information specification of activity “Describe activity”, the same was done with each of the other activities proposed, in addition to each task the steps that comprise it was defined, and for each step were defined a monitoring mechanism, an evaluation mechanism, and its description (See Table 4).

Table 4. Describe activity information
Full size table

4 Case Study

According to refinement that was shown above, a case study was developed in which MEPAC [22], was used as a support tool, which allowed to support the teacher in the collaborative learning process in each phase (Pre-process, Process, Post-process), and we are focusing the support of monitoring and evaluation in the Process Phase, applying each element redefined in this research.

4.1 Context

This case study was developed in an academic context, applying the refinement in two university courses of a systems engineering program, specifically in the Object-Oriented Programming (See Fig. 2), and Data Base Modeling (See Fig. 3) courses. The first course was constituted by 16 undergraduate students of first and second year. The second course was constituted by 10 students of second and third year.

Fig. 2.
figure 2

Course object-oriented programming

Full size image
Fig. 3.
figure 3

Course data base modeling

Full size image

4.2 Objective

The objective was to validate the elements refinement (activities, roles, inputs, outputs, monitoring and evaluation mechanisms) of the collaborative learning process in the Process stage. Therefore, the question for this case study is: Does the refinement of the collaborative learning elements of the Process phase is useful, applicable and easy to use, in the undergraduate academic environment?

4.3 Selection of the Case Study

The unit of analysis is the academic environment within a collaborative learning process in an academic environment, the primary source of information is the teacher and the participating groups, who are responsible for the Process phase, his selection responded to availability criteria [23] of a subject with interest in applying collaborative activities to his course. According to Benbasat et al. [23], the case study is Holistic, the collaborative activity selected was because it is a real case in teaching (it is a sufficiently complete case to evaluate the applicability of the proposal).

4.4 Metrics and Indicators of the Case Study

The indicators detailed description and its metrics are the following:

Utility: The utility is defined as the property by which the refinement of the Process phase achieves the proposed improvement objectives for the collaborative learning process. The metrics that have been established to calculate the utility are:

  • The range of students who approve the activity must be between 80% and 100%.

  • The range of students who consider that the refinement of the elements is a positive support for the course should be between 80% and 100%.

  • The percentage of the number of questions that have a positive impact obtained from the perception of the teacher between level four and five (five being the highest degree of utility) must be greater than or equal to 80%.

Applicability: Applicability is defined as the property by which refinement can be easily employed to obtain favorable improvement results for the collaborative learning process. The metrics that have been established to determine the applicability are:

  • The answers average about applicability obtained from the teacher perception must be greater than or equal to 80%. With values between four and five, where five is the highest degree of applicability.

  • The effort to apply the refinement must be on average of a collaborative activity (average duration of 3 to 4 h).

Ease of use: Ease of use is defined as the degree of ease with which a person can understand and apply the Process phase refinement. The metrics that have been established to determine the ease of use are:

  • The average from the teacher perception of ease of use obtained that is between 1 and 5 (5 being the highest degree of ease of use) must be greater than or equal to 80%.

  • The number of explanation questions for using that the teacher and the students made to the formulator of the refinement to apply the elements, should be below 3 questions per hour.

4.5 Case Study Execution

The case study application began with the Pre-process phase, in which the teacher used MEPAC tool and with a form performed the collaborative activity design. In the following session, the teacher performed the second form related to the Process phase, considering some activities already defined in the Pre-process, and taking the help given to execute this phase. The form presented in the tool for this phase contains the following elements:

  • Describe the activity:

    Clear explanation of the activity to be carried out:

    Define the activity limits (to be achieved with the activity):

    Steps sequence to be followed in the activity:

  • Groups formation:

    Group formation (participant’s names in each group created):

  • To assign roles:

    Students names with the assignment of each role created in the groups:

    Description for each role and work justification performed by each student:

  • Materials distribution:

    Groups name with the material definition delivered to each of them:

  • Activity start:

    Activity start time and date:

    Estimated time and date of activity completion:

    Description of each task with the estimated duration:

  • Maintain the collaboration moment:

    Define means to answer doubts, queries, and problems:

    Resources suggestion and information sources regarding the activity designed:

    Strategies to increase collaboration among students, during the activity execution:

  • Test the success criteria:

    Describe activity success criteria:

  • Conduct a formative evaluation:

    Define the evaluation about the activity theme carried out:

    Field to locate the evaluation to be carried out:

  • Feedback:

    Mechanisms to be handled to provide feedback on the activities carried out:

  • Present the activity closing:

    Mechanisms to be used for the teacher to guide the activities completion carried out:

    Strategy within each group to complete the activities carried out (for example: sharing within each group in pairs what was learned within the class):

  • To do the groups compare their results with each other:

    Strategy for groups to share the results:

After the teacher filled out each field of the Process phase, the software tool MEPAC generates a PDF of this second phase, and with this guide the teacher executes the activity designed on the course students. In order to carry out this Process phase, the monitoring and evaluation mechanisms of the activities carried out that allowed the teacher’s intervention at appropriate times were applied to achieve the proposed objective.

After executing the activity with the students, the teacher makes the form for the Post-process. For all these processes the times were measured, and the observations were recorded according to the protocols and templates established at each process phase. At the session end, a survey was given to the investigated subject to measure the satisfaction degree, utility, and ease of use of the application of the refined elements.

4.6 Results

As a result of the execute case study we obtain the following values for the indicators defined:

Utility

  • The percentage of the students who approved the activity before the refinement was 70.8%, while after the application of the refinement this percentage was 88.2%. Bearing in mind, that the activity carried out before the refinement and afterward was the same, to compare the obtained grades.

  • The survey conducted to students allows determining that the refinement is positive support for the course development, with an average of 86% of the students that classify the refinement at a high level of utility.

  • 80% of the questions answered by the teacher determine that the refinement is at a high level of positive impact on the course.

Applicability

  • The teacher perception regarding the applicability of the refinement of the elements is 85%.

  • The effort involved in the application was on average 4 h per person.

Ease of Use

  • The teacher perception of ease of use was in 20%.

  • The number of explanation questions by the teacher per hour was 5, and by the student was 4.

Discussion

In summary, the results show that the application of the Process phase redefined elements is simple; the terminology used is very close to the teaching environment, considering the perception of both the teacher and the student. The students perceived that the tools provided to them support the development of their activities. This was evident through students’ communication, via the MEPAC software tool, as they had to listen to each other’s viewpoints and to work as an effective team. Furthermore, from the teacher’s perspective, it is possible to classify the phase elements as useful taking into consideration the positive impact that was generated on the activity performed, and the positive impact that generates the forms use to guide collaborative activity definition and execution. Regarding the level of ease of use, the results specify that refinement is not classified as easily use because it requires a lot of information to follow each of the activities and their specifications, but it does provide mechanisms for its application in this context. Likewise, from the perspective of the teacher, it can be observed that in order to comply with all the stages a great effort is needed in terms of time per person.

5 Conclusions and Future Work

The refinement is useful to achieve the goals proposed according to the perception of the teacher, the students and when obtaining satisfactory results in the execution of the activity. In addition to being applicable to the collaborative learning processes area. Although the teacher requires a considerable time amount the first time it is applied, due to the lack of knowledge and inexperience of the needs to define a collaborative activity. The teacher needs an explanation of the use of some elements defined in the refinement that was not clear enough and some additional information about the description.

To achieve the success of the refinement application, MEPAC tool was essential to support this process and to allows the monitoring of each one activity, providing the necessary mechanisms to achieve the collaboration among the participants.

As future work, it is necessary to execute more case studies in order to refine the improvement of the collaborative learning process, considering the teacher and student roles in any of the phases of the process (Pre-Process, Process, Post-Process), with the support of MEPAC.