Keywords

1 Section Overview

To make an unobtrusive observation requires a quiet detection of features, a detection that does not disturb the natural actions of the interacting learner. For example, a sensor system might be collecting near-real-time data about someone’s physiological states during some task or activity while, at the same time, also collecting information about the tools used or communications with team members. Some of these detected features are then combined into indicators of states (e.g. engagement, deliberate pause) or trajectories (e.g. increased skill, changes in emotional valence) of interest. Along with the primary features and indicators, an observation also requires a bounded context, a surrounding set of nodes labelled as entities and edges labelled as relationships or processes if the analysis uses a network model.

Regarding the unobtrusive data collection about learning processes, both features and their context need to be engineered, which entails answering some critical questions. How are the indicators combined into features? What is the role of the extracted features in the learning process? How does the learner’s awareness of the features and indicators impact their performance? What are the limitations of the affordances in the designed experience to elicit evidence of the constructs of interest? Added to this are a host of potential noncognitive influences like the emotional states, motivations, and social capital of facing a variety of learning tasks as a team member. In the following chapters, you will find discussions of features such as:

  • Emotion, including emotional variability, instability, inertia, cross-lags, and emotional patterns (Chap. 2)

  • Problem-solving, e.g. deliberate pause (Chap. 3)

  • Soft skills, e.g. leadership skills in a workplace learning context (Chap. 4)

  • Motivation, particularly that changes over time and entails changing contexts that require thinking about ongoing feature redefinition (Chap. 5)

  • Self-regulated learning (Chap. 6)

So, the picture that is developing for unobtrusive observation is one that is both dynamic and contextual that requires multiple and wide-ranging measurements over time. Several data challenges arise, including dealing with differences in measures per minute and quality of the measures and aggregations from sensors collected using different time windows. Data must be integrated and clustered meaningfully to link to the indicators, a process that, at this time, requires both human and machine learning techniques. Understanding dynamic context requires a complex system perspective, for example, to determine the unit of analysis, the surrounding context, and the influences on the dynamics from the surround as well as how the unit of analysis influences its surround.

The following brief introductions provide a quick glimpse of how these authors view the unobtrusive observation of learning processes.

Chapter 2

Juan Zheng, Shan Li, Susanne P. Lajoie. A Review of Measurements and Techniques to Study Emotion Dynamics in Learning

Emotion states are dynamic and contextual across learning environments. Learners who experience similar levels of emotions can differ substantially in the fluctuation of emotions in a task or throughout a course. The authors introduce a taxonomy of emotion dynamics features, i.e. emotional variability, emotional instability, emotional inertia, emotional cross-lags, and emotional patterns. They discuss emotion detection methods that can unobtrusively capture longitudinal and time-series data, including experience sampling methods, emote aloud, facial expressions, vocal expressions, language and discourse, and physiological sensors. They also present several emerging techniques for assessing emotion dynamics, including entropy analysis, growth curve modelling, time series analysis, network analysis, recurrence quantification analysis, and sequential pattern mining.

Chapter 3

Karen D. Wang, Shima Salehi, Carl Wieman. Applying Log Data Analytics to Measure Problem Solving in Simulation-Based Learning Environments

This chapter presents the research team’s efforts towards understanding how the log data of students’ interactions within an educational simulation can be translated into meaningful evidence about their problem-solving process. Features extracted from log data were found to be both significant predictors of students’ problem-solving outcomes and indicators of specific problem-solving practices. Deliberate pauses during the problem-solving process, in particular, were identified as an important and generalizable feature associated with problem-solving competencies across different tasks.

Chapter 4

Abhinava Barthakur, Vitomir Kovanovic, Srecko Joksimovic, Abelardo Pardo. Challenges in Assessments of Soft Skills: Towards Unobtrusive Approaches to Measuring Student Success

This chapter outlines a multi-tiered case study that used a novel blended methodology, marrying measurement models and learning analytics techniques, to mitigate some of the challenges of unobtrusively measuring leadership skills in a workplace learning context. Using learners’ reflection assessments, several leadership-defining course objectives were quantified using a blend of natural language and structured data approaches. Student progress was assessed over time in relation to course learning outcomes. The chapter discusses the implications of their evidence-based assessment approach, informed by theory, to measure and model soft skills acquisition.

Chapter 5

Heeryung Choi, Philip H. Winne, Christopher Brooks. Proposal and Critiques of Measuring Motivational Constructs Using State-Revealing Trace Data

This chapter examines opportunities afforded by trace data to capture dynamically changing latent states and trajectories spanning states in self-regulated learning (SRL). The authors catalogue and analyse major challenges in temporally investigating SRL constructs related to a prominent motivational factor, achievement goals. The chapter summarizes three recent studies addressing these challenges and characterizes learning analytics designed to promote SRL and motivation formed from unobtrusive traces. The authors propose a research agenda for learning analytics focusing on guiding and supporting SRL.

Chapter 6

Sambit Praharaj, Maren Scheffel, Marcus Specht, Hendrik Drachsler. Measuring Collaboration Quality Through Audio Data and Learning Analytics

This chapter addresses the unobtrusive detection and measurement of collaboration quality based on audio recordings of student interactions. Using two indicators, time and content of communications, the team aimed to move towards an automated measure of collaboration quality. The authors explain the design of a sensor-based automatic analysis system and show their analysis using meaningful visualizations to gain insights into the quality of student collaboration.

To summarize, the detection methods discussed in the section include latent variable detection (Chap. 2), log traces becoming semantically meaningful units of analysis (Chap. 3), automated content analysis of learners’ reflection assessments (Chap. 4), and sensors systems and data handling of noisy information (Chap. 6).

Analysis methods discussed in the chapters include entropy analysis, growth curve modelling, time series analysis, network analysis, recurrence quantification analysis, sequential pattern mining, quantitative association rule mining, cognitive diagnostic model machine scoring of natural language products for depth of reflection on leadership skills, and temporal challenges of dynamic and contextual data, to name a few.

As noted by these authors, the path from unobtrusively acquiring log data to analysing semantically meaningful evidence of learning processes is an interdisciplinary effort that joins personality psychology, developmental science, learning science, and neuroscience. We trust that you will find this collection useful.