Keywords

1 Introduction

This paper builds on recent research exploring the application of analytics to measure SRL (e.g. [3, 4]). We explore how process mining can be used to track the influence of tutor practices on learner SRL in online one-to-one human tutoring settings. Our key Research Questions are presented below:

  1. 1.

    Which signifiers can be used to track the influence of tutoring practices on learner SRL, in a Virtual Classroom Environment (VCE)?

  2. 2.

    How can we use process mining to detect variations between high and low performing groups of tutors’ practice, regarding their influence on students’ SRL?

2 Methodology and Analysis

We have followed the methodology set out in [3] to i) Develop a framework of signifiers using a mixed-methods approach, ii) Implement the framework of signifiers using process mining. For our empirical work, we partnered with an industrial supplier named Third Space Learning, which delivers mathematics tutoring for primary school children aged 10 years old. Students and tutors log into a shared online environment, and the student works through a pre-designed online set of questions, with the guidance of a human tutor on an interactive whiteboard. The data available for analysis includes log data from the virtual classroom environment (VCE), learner audio files and tutor audio files. More information on the methods and findings can be found here.

2.1 Developing the Framework of Signifiers

We have built on the research conducted in [7] and adopted the Winne and Hadwin model of SRL [8] as the foundation for our Framework of Signifiers. We have interpreted the model based on the context of our research. More specifically, tutors are regarded as an external Condition, which influence the cognitive and metacognitive Operations of the student across the four phases of SRL. Micro-level processes and signifiers have been identified using a mixed-methods approach. This included a literature review and an empirical review of 50 randomly selected tutoring sessions (Tables 1 and 2).

Table 1. A framework of signifiers for tutor practices
Full size table
Table 2. A framework of signifiers for student operations
Full size table

2.2 Implementing the Framework of Signifiers, Using Process Mining

We collected samples of 55 online tutoring sessions conducted by high and low performing tutors. Tutor performance was determined using human evaluator rankings, with the top fifteenth percentile and bottom fifteenth percentile of tutors being selected respectively. We coded data based on the framework of signifiers, using the approach presented in [4]. We implemented our framework using a proprietary adaptation of Fuzzy Miner called Disco, configured to show only the most significant events and` transitions adjusting the parameters accordingly (Figs. 1 and 2).

Fig. 1.
figure 1

The process model of low ranked tutor dataset, mined with Disco algorithm

Full size image
Fig. 2.
figure 2

The process model of high ranked tutor dataset, mined with Disco algorithm

Full size image

3 Discussion

Our analysis showed that the process maps for the low ranked tutor group differed from the process maps for the high ranked tutor group in terms of both the structure of the map and also the types of prominent events and transitions. Overall, there is a relatively narrow range of events that are considered significant in the low-ranked cluster, with tutor-led practices, such as instruction and explanation, being the most prominent. In contrast, the high ranked tutor group exhibits a broader range of significant events and transitions, and the process map is more complex. Tutoring practices extend beyond the platform and include open-ended questions and metacognitive prompts. These findings align with past research [4, 6, 7] and have the potential to be used to i) design intelligent support for tutors in VCEs, ii) tutor training on these behaviours to improve their practice and student SRL.