Keywords

1 Introduction

Orchestrating Computer-Supported Collaborative Learning (CSCL) scripts may be seen as intertwining scripting, i.e., making a priori design decisions and then adapting them at run-time as necessary, and conducting, i.e., monitoring the students’ progress and helping students when required [7]. In this article, we introduce a simple model to design monitoring tools for tablet-based CSCL scripts in classrooms. The model is based on the idea that orchestration in classroom is both digital and physical: teachers are provided with quantitative and qualitative data captured from students’ activities that they can use to physically go and help the students. The model is voluntarily simple because teacher adoption of new technologies depends on the ability to appropriate and integrate the tools in their classrooms [5]; tools should thus be designed to be user-friendly and intuitive.

The model has been tested by developing and testing monitoring tools for a tablet-based negotiated dictation CSCL script, which setting is as follows (see [8] for details). In the initial individual phase, students listen to a dictation (audio-recorded) and type it on tablets. Then, teachers align students’ dictations to facilitate word by word comparisons and create groups of two or three students. Finally, in the collective phase, groups are asked to discuss on specific words for which they have to choose a spelling and justify it. The research question we considered is: does the model support (1) defining what should be displayed on the teacher’s tablet and (2) implementing tools that are simple enough to be easily used by not-specifically-trained teachers while allowing them to satisfactorily monitor and conduct the activity?

2 A Model to Support Monitoring for Orchestration

The activity and monitoring model we propose is illustrated in Fig. 1. With respect to the learning activity, we adopt a model that builds on the classic CSCL script structure. An activity is composed of several phases associated with actors (students or groups). In each phase, one or several tasks are performed by the corresponding actor, who produces some data (productions). With respect to monitoring, the design rationale is to provide teachers with information related to the actors’ progress and productions, being respectively quantitative and qualitative data. This double perspective is consistent with our idea that monitoring requires both a coarse-grained view (a synthesized overall classroom view) and a fine-grained view of how the activity is unfolding. “Additional data” corresponds to other monitoring data made available to teachers and which is not directly related to the progress of the activity.

Fig. 1.
figure 1

The model: offering both quantitative and qualitative monitoring data.

Full size image

In the case of negotiated dictations, the model is instantiated as follows. For the individual dictation phase (see Fig. 2): quantitative data corresponds to the part of the dictation completed by students (displayed as a progress bar); qualitative data corresponds to the dictation itself; additional data corresponds to the number of student corrections and the number of times each audio track was played. For the collective phase: quantitative data corresponds to the number of words justified by each group; qualitative data corresponds to the different students’ votes and their justifications; additional data is the time spent voting and justifying each word (duration of a few seconds may denote limited interactions).

Figure 2 illustrates the abstract interfaces and the implementation we developed for the individual phase and, to some extent, the general look and feel. By swiping the screen left or right, the teacher can have a quick glance at the quantitative data (classroom progress) and qualitative data (each student’s dictation). Additional information is made available by touching the corresponding row, which opens a pop-up window. The monitoring tools for the collective phase follow the same pattern. As dictations are linear exercises, it may seem that qualitative data would be sufficient to monitor the activity as the dictation itself can illustrate the students’ progress. However, with a larger set of students and multi-line dictations, separating quantitative and qualitative information proves necessary to offer an immediately clear perception of the students’ progress and avoid overwhelming teachers with too much data. If the list of students does not fit on the screen, the teacher simply has to swipe the screen up and down.

Fig. 2.
figure 2

The abstract interface (top) is instantiated to provide quantitative and qualitative perspectives for the individual phase of the negotiated dictation (bottom).

Full size image

From a technical perspective, each tablet is connected to a dedicated WLAN and the MQ Telemetry Transport (MQTT) protocol is used to enable communication between the devices. Using this protocol, the students’ tablets continuously send feedback about the ongoing activity to the teacher’s tablet, and vice-versa when the teacher needs to send instructions to students. A MySQL server is used to log each user’s actions, outputs and progress in the activity. This feature is essential for the research (analyzing data), but also allows recovery of a student’s activity in case of failure.

3 Tests and Results

Tests were conducted with four teachers in two Grade 4 (age 9-10) and two Grade 5 (age 10-11) classes. Classrooms ranged from 20 to 29 students for a total of exactly 100 students. Though the tools were designed with the help of teachers, the ones involved in the tests had actually no part in the design decisions. Hence, they were quickly briefed on how to use the tablet and the monitoring tools before the session. The class was split in halves with one engaged in the negotiated dictation and the other one on another task, and the activity was conducted twice so each half could participate. After the second session, teachers were asked via a questionnaire if and how they had satisfactorily monitored and conducted the sessions, and how they perceived the usability and efficiency of the tools offered (using a Likert scale). Three points were studied: usability of the student application, usability of tablets for monitoring the activity and usability and efficiency of the monitoring tools. As the results related to our concerns (usability and efficiency) were consistent, no more tests were run after these eight sessions.

We first considered the usability of the student application, which may influence teacher monitoring. Actually, students encountered no major difficulty in using the system. All four teachers agreed that the individual phase went smoothly and that the collective phase produced interesting justifications, which supports the idea that students were comfortable with the device and focused on the pedagogical tasks.

Regarding usability of tablets for monitoring the activity, three features were analyzed: the general look-and-feel of the tablets and the interest of touch screen interfaces, the size of the screen (we used “10.5” tablets) and the interest of the mobility offered by tablets. All four teachers fully agreed that tablets were easy to use and that the screen was big enough for the tools offered with only two of them being familiar with tablets. With respect to mobility, they all mentioned that it was an advantage as they all benefited from it by walking around the classroom during the individual and collective phases and providing hints to students. This behavior suggests that it could be interesting to study if and how such monitoring devices improve teachers’ proactivity, in opposition to solving issues only.

Our central research question was about usability and efficiency of the monitoring tools. Many differences in students’ task performance and individual behavior emerged from both the individual phase and the collective phase, which resulted in various durations required to finish each task. Such contrast provided enough occasions for teachers to use the tools and conduct the setting. A first general question addressed the usefulness of the tools offered for monitoring the students, in particular given these differences. All four teachers strongly agreed that the monitoring tools were necessary as they could simultaneously help students or groups in trouble and keep an eye on the overall course of the activity. A second question addressed if teachers had satisfactorily monitored the quantitative and qualitative progress of each actor (student by student and group by group). Altogether, the answer was positive (three strongly agreed and one agreed) for 99 out of the 100 students and for all 37 groups. The usability of the different tools was demonstrated by the ease with which the teachers used the tools right from the first session, and confirmed by the open discussion during the debriefing session. These findings suggest the different tools are highly usable and demonstrate the efficiency of both these tools (collectively) and the underlying model.

Additional questions addressed the usefulness of each individual tool. Here, some answers differ. For instance, one teacher found the qualitative monitoring tool very helpful and the quantitative one rather superfluous. Another did not use the diagnosis information. The debriefing discussion allowed us to understand that this was related to different monitoring strategies (see discussion in next section).

4 Discussion

The results suggest that the model and its implementation for negotiated dictations provide easy-to-use and efficient monitoring tools for teachers. It also validates the interest of our approach to monitoring as a mixed (digital and non-digital) activity and the usefulness of tablet mobility for this purpose.

The model of activity we used is fairly general and traditional in CSCL. Our future research agenda includes developing and testing instantiation with other learning scenarios. This implies identifying, for the considered scenario, a way to describe the actors’ progress and productions with quantitative and qualitative information. Depending on the nature of the tasks, an actor’s quantitative data can be illustrated with either a continuous or discrete (in the mathematical sense) progress bar while the qualitative data can reuse the same type of interface to show the answers. For instance, for a scenario based on a suite of exercises to be addressed individually and then collectively, reusing a discrete representation to display the quantitative and qualitative information of each exercise seems appropriate.

Another aspect to be further considered is the extent to which the model can be scaled, i.e., the maximum number of students or groups that can be managed simultaneously and the quality of the management. With more students, tablets might be too small to provide efficient monitoring tools. At this level, however, let us recall that we are considering monitoring for classroom orchestration.

An interesting observation that became apparent during the experiment, and was confirmed by the questionnaire and the debriefing session, is that teachers implemented different monitoring strategies. These different monitoring strategies can be explained by the teachers own work practices [6] (let us remember that they were not involved in the design process of the tools nor trained to use them). Given our research goals, this observation suggests that teachers seemed to succeed in using the tools according to their strategy and were satisfied with the way they monitored the activity. This also suggests that the simplicity of the model and tools, coupled with the overall mixed perspective of monitoring, allows for a variety of strategies. Keeping in mind this diversity of teacher own practices is a crucial issue when designing orchestration tools, and more generally, when designing educational software [6].

The work presented in this article is in line with other works having shown that mobile technologies may be used in classrooms to scaffold face-to-face interactions and foster social interactions between students [9]. This is the rationale for using technology to enhance an activity with which teachers are already familiar (here, dictation). Using tablets provides teachers with the necessary mobility to move around the classroom and intervene among students, which would be more difficult if computers were used. With respect to orchestration research themes presented by the synthesis proposed in [4], our work addresses regulation issues and awareness. With respect to the orchestration design principles presented in [1], our work implements control, visibility, physicality and, last but not least, minimalism. The design principle of proposing a dual perspective (progress bars and students’ productions) is a well-founded pattern. For instance, the monitoring tool presented in [3] uses progress bars to show the students’ relative involvement in their groups during a collective phase. In [2], the groups’ actions are shown on a public display allowing both teacher and students to be aware of all the productions. The work presented distinguishes itself from these by using a single model to monitor both individual and collective phases and by delivering both quantitative and qualitative feedback to ensure whole class and individual monitoring.

5 Conclusions and Perspectives

In this article, we presented a model to support teachers’ monitoring actions for the orchestration of CSCL activities in a classroom. An instantiation has been implemented and tested, and results suggest the value of the approach. The experiments also suggest that it would be worth studying teachers’ monitoring strategies and the interplay between these strategies and the model and tools. The model, the abstract interfaces and the implementation architecture and framework are generic. Our research agenda also includes developing other instances with other tablet-based CSCL activities consistent with our activity model. Developing new instances will help define the precise scope of application of the model and framework. Finally, another perspective of this work is to enhance the monitoring device with orchestration tools, such as means to move students from one group to another at run-time.