Keywords

1 Introduction

The term “orchestration” has been used in Education to describe the real-time management of multiple classroom activities, various learning processes and involving numerous teaching actions [1]. In technology-enhanced learning, orchestration technologies are the digital tools that support teachers in the orchestration of complex learning activities [2]. Such tools have been especially proposed to support teachers in orchestrating student collaboration across learning flows, in the sense of guiding, the managing and coordinating, activity sequences, group formation, resource distribution, etc. [3]. In alignment with the concept of orchestration technologies, the field of Computer-Supported Collaborative Learning (CSCL) studies the use digital tools to design and deploy collaborative learning activities [4]. In this context, teacher orchestration refers to three dimensions of a distributed CSCL environment: cognitive (e.g., regulating individual, small-group and class-wide interactions), pedagogical (e.g., real-time adaptation of the designed activities to the classroom needs), and technological (e.g., management of the transactions between software components) [5].

Orchestration technologies are being mostly designed for classroom, with the most salient part of the scenario occurring face-to-face [1]. Thus, the implementation of CSCL activities in fully online environments can be challenging for teachers and students at both levels, educationally and technologically. Several studies have discussed the difficulties the students face when performing online collaborative activities [e.g., 6, 7]. Major challenges include ineffective communication, conflict among group members, and negative behavior toward group work [6]. Less attention has been paid to understand how teachers’ orchestration actions differ across different learning settings, e.g., in-class and online setting.

Therefore, in this paper we explore the teacher’s orchestration actions in two settings namely in-class and online in the use of PyramidApp [8], a web-based tool that allows teachers to deploy Pyramid collaborative learning flow pattern based scripted collaborative learning activities. PyramidApp consists of an authoring space which facilitates activity authoring, activity enactment space for students and a teacher-facing dashboard that provides orchestration support, e.g., information about students’ activity participation as well as functionalities to adapt the flow of script in real-time. The activity flow is as follows: First students require to provide an individual option to a given task. Then they join in small groups and later in larger groups to discuss and improve individual options and to reach a consensus at the end of the activity.

We analyzed data collected from a single teacher across ten sessions, five of which were online and five in-class. We used a mixed-methods approach to answer the following research question: To what extent do the teacher orchestration actions differ in online sessions when compared to in-class sessions of computer-supported-collaborative-learning activities?

We posit that the contribution of this study, as a work in progress, to the field of technology-enhanced learning would advance the examination of how different learning settings, i.e., online and in-class, influence teachers’ orchestration actions which could also help us to explain better the orchestration load experienced by the teachers in future studies.

The rest of the paper is structured as follows. In Sect. 2, describes the provides methodology followed to answer the research question. In Sect. 3, presents study findings and lastly Sect. 4, discuss the results, limitations, and future work.

2 Method

2.1 Data Collection

A female teacher from a public university in Spain has participated in this study. She had over 17 years of teaching experience and had previous experience in authoring and orchestrating CSCL activities. The main criteria for selecting the participant were the existence of teaching experience, prior knowledge, and experience in using PyramidApp in both online and in-class settings. The teacher conducted ten Pyramid activities five of which were online and the other five were in-class sessions.

Fig. 1.
figure 1

Teacher-facing dashboard used by the teacher.

Full size image

Data was collected through capturing audio data from each session, screen- recording the teacher-facing dashboard (see Fig. 1) and taking observation notes while the teacher was orchestrating the activity. Moreover, the log data that indicated the relevant details were extracted from the PyramidApp database (e.g., the number of students participated in the activity, duration of the task, the task given for each session and the actions taken by the teacher in the dashboard). The screen and audio recordings, the observations notes, and the log data were analyzed to explore how teacher’s orchestration actions differ in two settings (i.e., Online and In-class) using PyramidApp tool.

The tasks for the five online sessions were the same as those for the five in-class. However, the design of each collaborative learning activity differed depending on the teacher’s requirements for conducting CSCL activities in each session. Table 1 presents the tasks given by the teacher and the number of students who participated in each session. In addition, tasks A and B were conducted in an undergraduate class and tasks C and D were conducted in a master class. Task B was used in four sessions (i.e., Online1, In-class1, Online2 and In-class2), while each of the other three tasks were used in two sessions (i.e., Online1 and In-class1). Each activity lasted around 9 to 19 min.

Table 1. A Summary of Collaborative-Learning Activities Conducted".
Full size table

2.2 Coding Teacher’s Orchestration Actions

To be able to answer the research question, we analyzed orchestration actions of the teacher across the ten sessions. Teacher’s orchestration actions were coded following a coding scheme defined in [9]. This coding scheme includes six codes as follows: 1) Teacher-individual interaction 2) Teacher class interaction 3) Announcements to class 4) Check responses tab 5) Check participation tab and 6) Dashboard interventions. More details about the codes are presented in Table 2.

Table 2. Codes defined to describe teacher’s actions.
Full size table

3 Findings

This section presents the results obtained after the analysis of ten sessions distributed to four collaborative learning tasks. We compare the number of teacher’s actions in each task of both settings (i.e., Online and In-class). Figure 2 shows two graphs, one for the actions taken during the online sessions and one for the actions taken during the in-class sessions. Then we present and compare the aggregated actions for all the tasks in different settings. (Table 3).

Fig. 2.
figure 2

Teacher’s actions in both online and in-class settings.

Full size image

As shown in Fig. 2, in all tasks there were differences in the teacher-individual interaction. The individual students interacted more with the teacher in the in-class sessions when compared to the online sessions. In tasks A and D, the teacher conducted more class interactions in the online sessions. Moreover, actions from announcements to class, check responses tab, check participation tab and dashboard interventions occurred more in the in-class sessions than in the online. Task B was used in two different sessions. The first one (i.e., Online1 and In-class1), actions such as teacher class interaction, announcements to class, check responses tab and dashboard interventions occurred more in online sessions. However, check participation tab actions occur more in in-class sessions. The second session (i.e., Online2 and In-class2), the teachers conducted more class interactions and checked the responses tab in the online session. The number of announcements to the class were the same in the online and in-class sessions. In addition, the teacher conducted more dashboard interventions during in-class sessions when compared to the online sessions. In task C, the teacher interacted with the class and made more announcements in online sessions when compared to the in-class sessions, while actions from check response/participation tabs and dashboard intervention happened more in in-class sessions.

Table 3 shows the difference between aggregated actions of each code in the two settings. The findings show that during the in-class setting there were more teacher-individual interactions, announcements, check responses tab, check participation tab and dashboard interventions. In the online setting, however, the teachers conducted more class interactions and fewer individual interactions when compared to the in-class context (Table 3). It is also interesting that the teacher was not using less the monitoring features of the classroom in the In-class condition, but the contrary. Differences in the number of times that the teacher decided to check student participation are substantial.

Table 3. Teacher’s actions in all online sessions and all in-class sessions.
Full size table

4 Discussion and Future Work

Teacher-individual interactions occurred less often in the online sessions, even though there were more participants in this setting (n = 77) than in the in-class sessions (n = 56). The lack of interactions with individual students might indicate less workload to the teacher. This might be due to a communication issue connected to the students’ willingness to raise questions during online sessions, which is consistent with the literature suggesting that communication has shown to be the biggest challenge in online collaboration. [6]. Also, we assume that the number of teacher-class interactions in the online setting indicates the need for more explanations about how to use the facilitating CSCL tool when compared to the same interactions in the in-class setting. Most of the actions in this category (31 out of 45 in the online setting, and 20 out of 23 in the in-class) were technology-related, i.e., the teacher is giving directions to the students about the use of the facilitating tool. To further investigate such assumptions in the future, we are working on analyzing the students’ performance during online and in-class sessions (e.g., the total number of students who completed the task in each session, quality of their outcomes).

The data collected for this study is limited due to the criteria of data collection, and the differences between learning designs across sessions. More data will be collected in the future from other teachers who taught the same course to enable for more in-depth analysis and generalizable findings.

The implications of this study and its continuation are related to the consideration of the learning environment in the design, redesign, and evaluation processes of orchestration technologies, and how they can impact the teacher orchestration load as well as the student learning and collaboration. This ongoing research would also further the investigation of how orchestration tools could facilitate teachers to regulated CSCL activities in different settings. It can be of interest to practitioners who teach in distance, online and hybrid settings and other stakeholders in the wider TEL field.