Keywords

1 Introduction

Meaningful Learning Theory (MLT) suggests that when we learn something new, it is most effective if we can relate it to our existing knowledge and experiences. This way, we can integrate the new information in a way that is relevant and meaningful to us, rather than just memorizing it superficially without really understanding it. This approach to learning is more beneficial than the superficial approach, where the focus is only on memorization without any real comprehension or connection to existing knowledge [1].

The MLT was developed by educational psychologist David Ausubel. This theory is concerned with the active construction of knowledge by the learner. Some key features of meaningful learning include the connection with prior knowledge - this means that new information is related to what the student already knows and understands. By doing this, the new learning is based on a solid foundation and creates deeper connections. In addition, deep understanding means that the learner not only memorizes information but also comprehends the underlying concepts. [2].

From this perspective, effective learning involves the ability to explain, apply and relate new knowledge in various contexts. The relevance and applicability of new knowledge are crucial in giving it meaning and practicality. This enables learners to apply it in real-world situations, making the learning experience more valuable. Additionally, meaningful learning involves the cognitive process of reflection, analysis, and critical thinking, which engages the learner in a deeper mental process. Finally, the integration and construction of knowledge means that new information is integrated with previous knowledge, creating a network of interconnected concepts [1].

A classic example of meaningful learning is when someone learns a new scientific concept by relating it to everyday situations or other already known concepts. By making these connections, the learner creates a deeper, more lasting understanding. Educators often seek to promote meaningful learning in their teaching practices by encouraging students to actively participate in the learning process, relate content to their own experiences, and build an authentic understanding of concepts. This contributes to more lasting and meaningful learning over time [2].

An important element of meaningful learning is monitoring student learning. It is of fundamental importance for the teacher to know that students are paying attention in class and not wandering or absent. This monitoring, which in face-to-face teaching can be done with a simple look at the class, in distance education becomes an additional challenge to teaching practice. This problem is even more pressing in synchronous remote classes, which have become common in recent years.

In other words, understanding and mapping the effective results of the teaching and learning process in remote and distance learning, ensuring the addition of new knowledge to previous ones and, consequently, meaningful learning, are a categorical challenge today and give rise to the main question of this study: how can we improve learning by using new resources to map student attention and engagement, enabling greater effectiveness and meaningful distance learning? The central hypothesis of the research is that with the use of human computer interaction (HCI) technologies it is possible to monitor student attention and participation more accurately during remote or distance classes.

This article explores the application of two popular technologies in the HCI field for monitoring students during remote learning. These technologies are Electroencephalography (EEG) and eye-tracking. EEG measures the student's level of attention during classes, while eye-tracking tracks the student's gaze and records points of focus on the computer screen or any shift of attention away from the screen.

Initial tests were conducted with five students, using two different videos. The findings suggest that students have a preference for paying more attention to one video profile and feeling more bored while watching the other. By analyzing this data, educators can customize their teaching methods, adjusting them to individual students’ needs, which can lead to more meaningful and engaging learning experiences.

2 Theoretical Reference

Distraction, also known as mental wandering, can be a hindrance in the field of education and the teaching and learning processes. In this study, we define mind wandering as a state where the mind wanders or strays away from the task or focus at hand. It occurs when a person's attention is diverted to thoughts, ideas, or concerns that are not directly related to their current activity or train of thought [3].

Mind wandering can occur in various situations, such as when an individual tries to concentrate on work, studying, or a conversation but instead ends up lost in unrelated thoughts. This can happen due to boredom, external distractions, mental fatigue, or the intrinsically active nature of the human mind, which frequently jumps between random thoughts.

In education, students’ wandering can pose a challenge in the teaching-learning process [4,5,6]. When teaching in-person, a teacher has various methods for observing a student's level of engagement and focus in learning activities. However, in remote or distance learning situations, interactions are virtual, and new strategies are necessary to assess the effectiveness of student learning. Considering the principles of meaningful learning, two techniques that can help teachers are eye tracking and electroencephalography (EEG).

Eye-tracking technology is a well-known and widely used method in Human-Computer Interaction (HCI) and various other fields, including marketing, cognitive psychology experiments, and entertainment, such as in games [7, 8]. It works by detecting the user's eye movements during interaction, which can be achieved through different tools like webcams, infrared cameras, or head-mounted displays. The data collected through this technology can provide valuable insights about users, such as their attention to a specific object on the screen.

When mapping a user's gaze, the specific points that they look at are called “gaze points” or “specific points in the image”. A group of gaze points that are fixed on a certain area for a period of time is called a “fixation” [9]. This region is crucial for our study, as it is during this time that the user's main cognitive processes such as understanding, and memory occur.

The quick movement made between the “fixation” points is called a “saccade”. During this movement, which lasts between 30 to 80 ms, visual information is blocked. A group of “fixations” can be grouped together based on proximity, forming a “gaze”, which can then be divided into “areas of interest” (AOI) [10]. The amount of time spent on each AOI (known as “dwell time”) can determine the level of interest the user has in a particular stimulus on the screen. A longer dwell time can indicate a higher level of interest. One way to observe this phenomenon is through the use of heatmaps, which display on the screen the points that captured the user's attention [8].

The article presents a second technique called EEG, which involves analyzing the spontaneous electrical activity of the brain. The technique is used to detect patterns and abnormalities in brain waves [11]. EEG involves placing electrodes on specific points on the user's skull according to the internationally recognized 10/20 system. This enables the recording of brain waves in the cortex. While the technique has good temporal resolution, meaning it captures information in real time, it has low spatial resolution. It's important to select the relevant information and remove noise when processing the data extracted from the brain using EEG.

Studies have shown that midline-frontal beta waves correspond to individual preferences [12]. This means that the higher the amplitude of oscillation in the beta wave frequency observed through EEG when watching a film trailer, the higher the score given by research participants to films related to the same theme [12]. Additionally, the desynchronization of alpha waves in the left-frontal side of the brain is positively related to the level of pleasure and satisfaction perceived when watching commercials [11]. Finally, an increase in the power of theta waves in the frontal midline is associated with feelings of pleasure [13].

Using eye tracking and EEG, it is possible to diagnose a user's perception of visual stimuli. This helps us to better understand their preferences and the emotional and cognitive processes that occur naturally as they consume content on a screen. Initially, we can be highly certain about where the student is focusing their gaze, which should be on the computer screen. At the same time, we can measure their attention, engagement with the class content, and mental memorization processes with the EEG.

In other words, EEG and eye tracking can help measure students’ engagement and attention. These technologies are aligned with MLT and aim to connect new content with prior knowledge. The use of these resources could optimize the learning process, creating a more adaptive and student-oriented educational environment.

3 Related Work

There have been several research studies that have focused on using the techniques of EEG and eye tracking [14,15,16,17,18]. For instance, GuruTutor [15] is an avatar developed to provide virtual instructions to students. It was tested and developed by analyzing visual focus through eye-tracking and patterns of engagement and comprehension through EEG. Another relevant research study that combined the two techniques was conducted using the Neuroscan SynAmps2 system and Curry 8.0 recording software [18]. In this study, EEG data was collected while a conductive gel was used to reduce the impedance at each electrode. Simultaneously, eye movement data was collected using an infrared video-based eye tracker, the EyeLink Portable Duo from SR Research.

There are several studies that have used eye-tracking technology to identify students’ concentration and attention levels during learning activities [16, 18,19,20,21,22,23,24,25]. Some of these studies have used WebGazer, which can detect cognitive states during online reading comprehension tasks. These cognitive states can be related to both task-related and non-task-related comprehension. Other studies have used eye-tracking glasses, which have shown that multimedia technologies need to be carefully chosen for effective learning [20].

Although there are other indicators, such as physiology and gestures, eye-tracking is considered the best short-term indicator for identifying the direction of visual attention. However, detecting mind wandering can be difficult, as it involves “looking without seeing” and directing attention to other content or subjects. Despite this limitation, gaze tracking can effectively identify when the mind begins to wander [24]. This is because conventional eye movement patterns change when someone is mentally distracted. For example, when someone is not focused, they are less likely to hold their gaze on parts of the text they have already read. Additionally, blink frequency increases when the mind is wandering, perhaps because there is less processing of visual information during blinks.

Research in the field of education has explored the use of EEG in addition to eye tracking, using different technologies [18, 22, 26,27,28,29,30,31,32,33,34]. By using machine learning methods [28, 33, 34], EEG measurements were utilized to predict mind wandering. It was observed that the Support Vector Machine (SVM) model is more effective than the logistic regression model in classifying mind wandering states within and between subjects. This finding is consistent with previous studies that demonstrate the effectiveness of nonlinear models in determining the boundaries between attentional states.

Real-world environments have also been studied, where recordings of EEG activity during live lectures were analyzed, and power measurements in the theta, alpha and beta frequency bands achieved an average detection accuracy of 80% to 83%. It was discovered that students who demonstrated satisfaction [26, 30, 31] showed greater power in alpha and beta frequencies when subjected to tests in which the video instructor used pointing gestures. These results suggest less sensorimotor involvement in the processing of this information.

Although the topic of mapping attention in the teaching and learning process, with a focus on distance education, is present in scientific research in different parts of the world, no research was found that takes into account both the direction of the gaze and the attention and concentration of students. This data becomes even more relevant if we include the use of a webcam for eye tracking as a requirement.

4 Methods

This article utilized eye-tracking software to identify the user's points of interest in remote classes, while the Emotiv Insight 2.0 headset was utilized to map neural patterns. The research is based on Design Science Research (DSR), which justifies the development of problem-solving-oriented artifacts as a way of generating scientific and technological knowledge [35,36,37]. DSR was considered suitable for this study after identifying gaps in technical and scientific production involving EEG and eye tracking for attention mapping in remote classes. The methodological process consists of six activities:

  1. 1.

    Identification and motivation of problems: This step aims to define the research problem and justify the value of a solution.

  2. 2.

    Defining the objectives of a solution: The objectives of a solution are inferred, considering the definition of the problem, and analyzing what is possible and viable within the scope of the research.

  3. 3.

    Design and development: At this stage, the artifact is created.

  4. 4.

    Demonstration: The researcher presents the created artifact and shows how it solves the problem, or part of it.

  5. 5.

    Evaluation: In this activity, the researcher observes the use of the artifact and evaluates how well it solves the problem. A comparison is made between the objectives and the results observed during the use of the artifact in the demonstration.

  6. 6.

    Communication: this is the final stage that closes the DSR cycle. In this activity, the problem, along with its context, relevance, and the created artifact's usefulness, quality, and novelty, rigor of the design, and effectiveness must be disclosed to impacted professionals, researchers, scientists, and other relevant audiences.

5 Objectives and Development of the Artifact

Considering Activity 2 of the DSR and the central research problem of identifying attention in distance synchronous classes, the following objectives were established for the artifact:

  1. 1.

    Use of non-invasive, easily accessible, and low-cost devices.

  2. 2.

    Reading and identification of neural wave patterns.

  3. 3.

    Availability and access to real-time data processing and subsequent treatment using proprietary software.

  4. 4.

    Storage of data for future retrieval.

  5. 5.

    Data retrieval for study and analysis of neural patterns.

  6. 6.

    Graphical visualization of neural patterns.

  7. 7.

    Integration of the system with the individuals’ focus of attention.

  8. 8.

    Integrated analysis of attention during classes.

An initial description of the system requirements is in [38]. The subsequent sections describe the design, implementation, and testing of the artifact, along with the lessons learned so far.

For this stage, the Emotiv Insight 2.0 headset was used, which meets objectives 1 and 2 entirely and objective 3 partially. A Python script was developed for data processing and visualization, which completes objective 3. The Cortex API [39], developed by the headset manufacturer and available online for free, was also used.

The Emotiv Insight 2.0 [40] is a device that measures EEG signals using wireless Bluetooth technology. It has 5 reading channels located at positions AF3, AF4, T7, T8 and Pz of the international 10/20 system. It also has Driven Right Leg (DRL) connections and Common Mode Sense (CMS) on the left mastoid, which act as the system reference. The device collects samples sequentially at a frequency of 2048 Hz, which are then filtered and reduced to a sampling rate of 128 samples per second. The collected data is processed by a 14-bit analog-to-digital converter, with 2 bits of instrumental noise floor being discarded. One Least Significant Bit (LSB) represents 51μV, and the dynamic range for the inputs is 8400μV. The headset is equipped with a fifth order Sinc filter, digital notch filters for 50 Hz and 60 Hz, AC coupling, and can recognize bandwidths for brain waves in the range of 0.5–43 Hz. The Emotiv Insight is also capable of detecting movements using the accelerometer, magnetometer, and gyroscope present in the ICM-1 IMU attached to the headset.

The user is exposed to different classes using the Emotiv Insight device. The system architecture is shown in Fig. 1. While the electromagnetic pulses are read via the headset, the data is interpreted in a cloud server environment through constant exchange of information with the Cortex API. The connection is made using the Websockets Secure Protocol on a Websocket server and the JSON-RPC 2.0 protocol [41]. After the application authentication process using credentials validated by the Emotiv Launcher application, it is possible to extract different data streams from the readings. The records are immediately displayed in a graph dynamically built in a Python language program using the Matplotlib library. This research presents information using power readings in μV2/Hz in the Alpha, Low Beta, High Beta, Gamma, and Theta frequency bands, transmitted at a frequency of 8Hz, in accordance with the specifications of the aforementioned API [39].

Fig. 1.
figure 1

Diagram of the functioning of the developed artifact [Authoral].

Full size image

Simultaneously a CSV file is created and fed with the obtained information. The application can be fed back with the generated files to study and analyze the recorded neural patterns. To facilitate more in-depth readings of the extracted data, the graph exploration features of the Matplotlib library are utilized in a similar way to real-time visualization.

Additionally, the GazeRecorder tool [42], a free online software for eye tracking experiments based on the computer's camera, is used to mitigate the issue of gaze focus and attention. In previous tests, it was observed that emotions vary depending on people and moments, but patterns or causal relationships could not be identified [38, 43]. The relationship between macro and micro expressions can be influenced by the environment, focus, attention, and concentration of the user, resulting in different emotional frames [43]. By allowing synchronization between the readings of brain waves and the direction of the user's gaze, an eye tracking system can evaluate which elements caught the user's attention the most. This information can then be associated with an analysis carried out using the EEG to make more detailed conclusions about what exactly influences the emotion felt during enjoyment.

An experiment was conducted to demonstrate and validate the effectiveness of two video lessons. Five participants, three men and two women aged between 21 and 54, attended the classes which were held in the Laboratory of Interaction and Media. The laboratory was set up to simulate an office environment with a table and a Macbook computer. Prior to the commencement of the classes, participants were asked to fill out pre-test questionnaires to identify their physical and emotional state. Post-test questionnaires were administered at the end of the day to evaluate the two classes attended.

The experiment began with an initial calibration step where participants had to follow a red dot that moved across the screen with their vision. After calibration, the classes were displayed on the screen, and the tool mapped the vision of the participants. This allowed the tool to identify the regions of the video that the participant had the greatest concentration on. Finally, the software generated a heat map over the film scenes, providing the results of the experiment for later analysis. The software used had an accuracy of 1.05°, a precision of 0.129°, and a sampling frequency of 30 Hz [44].

Two videos were used in the experiment, each with a different approach. The first video was a didactic video about how the Solar System moves around the Milky Way. It was divided horizontally into two parts, with the upper part being a 3D animation representing the movement described by the author of the video. The lower part describes the movements of the animation. The second video was a traditional class without any visual appeal.

Video 1:

Description: This is a didactic video explaining how the Solar System moves around the Milky Way. The video is divided horizontally into two sections. The upper section is a 3D animation representing the movement described by the author, and the lower section describes the movements of the animation.

Author: The video was published by the YouTube channel “aindanaosei” on August 23, 2022.

Duration: The video lasts for 1 min.

Video 2:

Description: This is a didactic video about polynomial equations. The video uses drawings to demonstrate how algebraic calculations are performed with more than one type of variable. The exercise is solved on a sheet of paper.

Author: The author of the video is unknown.

Publication: The publication date and source of the video are unknown.

Duration: The video lasts for 1 min.

6 Outcomes and Discussions

The study aimed to prove that the quality of content produced can be identified or mapped based on brain waves. Five users were tested, with a focus on alpha and beta waves. The alpha waves are closely related to spatial, semantic, and social attention [13, 45,46,47]. They have several functional correlations that reflect sensory, motor, and memory functions. During physical and mental relaxation with eyes closed, the power levels of this frequency increase. However, the level of alpha waves is reduced during mental or bodily activity with the eyes open. Alpha suppression is a reliable indicator of mental engagement and activity states, particularly during focused attention on various stimuli, which is the focal point of this study. This means that during attentive moments with eyes open, one would expect a low or non-existent rate of alpha waves.

On the other hand, beta waves are related to active, busy, or anxious thinking and high levels of concentration. The level of beta waves becomes stronger as we plan or execute movements, particularly when reaching or grasping requires precise finger movements and focused attention. This increase in beta waves is also noticeable when we observe the body movements of others. Our brain apparently imitates other people's limb movements, indicating that there is an intricate “mirror neuron system” in our brain that is coordinated by beta frequencies [13, 48, 49].

Fig. 2.
figure 2

Eye tracking heat concentration in 3D simulation [Authoral].

Full size image

Beta waves can be divided into two categories: low beta and high beta. Low beta waves are associated with negative feelings such as anxiety, fright, and surprise, while high beta waves are linked to positive feelings of surprise, anxiety, and excitement [50]. Therefore, a high rate of high beta waves is expected when the student is enjoying the content and is satisfied with the class.

Fig. 3.
figure 3

Low Beta, High Beta and Alpha levels in the left front channel of a specific participant during video 1 enjoyment [Authoral].

Full size image

In the YouTube Shorts video experiment, the first video demonstrated that participants had a greater focus on the 3D animation when it was mentioned in the video author's speech, resulting in a quick unconscious response from the student. This caused the participants to pay more attention to the content being presented (Fig. 2). When the 3D modeling was not referenced in the video, participants often returned their gaze to the author's face, which detracted from the fact that they were actually paying attention to the content. These results demonstrate that using supporting figures and videos can be an excellent solution to capture student attention.

During the analysis of the brain waves of a user in a video about the movement of the Solar System in the Milky Way, high levels of alpha waves were detected (the blue ones in Fig. 3). The alpha wave can give information about the participant's level of attention. In this case, it can be concluded that the participant was mentally scattered during the class, despite focusing on the graphic elements on the screen. This pattern was observed in all other study participants, and post-test interviews confirmed that this video was preferred due to its interesting content and high-quality production. High beta waves and little gaze wandering were also observed in the participants during the video.

During the video lesson on polynomial equations, researchers observed the participants’ eye movements and found that they focused on the specific element that the teacher referred to. Figure 4 shows that sometimes, the participants’ gaze shifted, possibly indicating that they were trying to anticipate the result of the equation. Since the equations were simple, it's likely that the participants became bored or anxious and tried to predict the result with their eyes.

Fig. 4.
figure 4

Eye tracking heat concentration in anticipating results [Authoral].

Full size image

The video on polynomial equations showed higher levels of low beta waves (in yellow) as seen in Fig. 5. Low beta waves are associated with negative emotions such as surprise, fright, and anxiety, which suggests that users negatively experienced anxiety while watching the video. This could be due to the fact that users already knew what the expected result of the polynomial equation would be, and the video was a slower teaching method without engaging visuals.

In contrast to the first video, the alpha levels in this one were slightly lower, implying that the student might have focused more on an abstract topic. Mental calculations demand greater effort and concentration compared to the mental retrieval of images from graphic elements. The post-test interviews confirmed this fact and indicated that the second video was perceived as less interesting and had less educational value compared to the first one.

Studies have shown that educators can use EEG and eye tracking together to adjust their teaching strategies and materials based on the identified patterns. This approach provides useful data for both synchronous classes and teaching material preparation. For instance, if eye tracking reveals that students tend to focus on certain parts of the text or image, while EEG indicates high levels of brain activity associated with attention, educators can emphasize those sections during explanation or discussion in the classroom.

Fig. 5.
figure 5

Low Beta, High Beta and Alpha levels in the left front channel of a specific participant during video 2 fruition [Authoral].

Full size image

Conversely, EEG signals and eye-tracking patterns can indicate problematic pedagogical strategies if they show moments of distraction, disinterest or mind wandering. In such cases, educators can intervene by incorporating interactive activities or breaks for discussions to engage the students. This data-driven approach promotes more meaningful learning by adapting the content to the student's individual needs and preferences, resulting in deeper and longer-lasting absorption of knowledge.

The use of computational processes aligns with the principle of selective attention, which is a fundamental part of the MLT [1]. This theory emphasizes that learning occurs when students focus on relevant and meaningful information. Detailed analysis of fixations and eye movements can help identify which parts of the material are receiving the most attention, enabling educators to adapt teaching methods to emphasize these areas of focus. This, in turn, increases the likelihood of students making connections and developing deeper knowledge foundations. It could also avoid the mind wandering phenomenon.

According to MLT, activating students’ cognitive processes requires reflection, analysis, and critical thinking. The presence of these elements can be identified by alpha and high beta waves, which allow for an inference about the student's engagement with the content being taught.

Eye tracking tools have certain limitations when used for experiments as they rely on webcam recordings, which may not be as accurate as recordings made using an infrared camera. Infrared cameras are more suitable for environments where the user is not stationary in front of the screen. Moreover, the free version of the GazeRecorder software has restrictions on the number of tests that can be carried out.

For future work, we recommend obtaining an infrared camera to enhance accuracy in eye-tracking results and integrating open software to analyze it. We also suggest implementing software that can visualize EEG waves. In terms of wave analysis, we suggest the need for more powerful EEG readers that are not prone to interference from hair fibers, like the Emotiv Insight. Additionally, we require more data from a larger number of users to further understand issues related to taste patterns and affinity with the content of the classes as a whole.

7 Final Considerations

The article discussed the use of eye tracking and EEG to monitor students’ attention during distance learning. The GazeRecorder website was utilized for eye tracking, while the Emotiv Insight headset and Python software were used to read brain waves. The study aimed to enhance synchronous classes, but the gathered data can also be used to evaluate and redesign teaching materials available asynchronously.

The study confirmed the hypothesis that the use of Human-Computer Interaction technologies, specifically eye-tracking and EEG-based brain-machine interfaces, can accurately monitor student attention and participation during remote or distance classes. The test results showed that neural patterns and eye tracking were compatible with the dynamics of the videos used in the tests. The patterns identified were confirmed by post-test interviews.

It's important to note that the study was conducted in a laboratory and utilized specific technologies, which may not be feasible on a larger scale. While eye tracking based on computer webcams can be replicated on a large scale, using the Emotiv headset may not be practical for larger groups of students. However, one alternative could be the MN8 headset, developed and marketed by Emotiv, which has two reading channels and is available for purchase at $399.00 on the company's website.

Although the study produced results, certain limitations were identified during the research. However, these limitations were mitigated throughout the process. As explained in the text, the visual analysis of the results of waves obtained through electroencephalography remains one of the key challenges faced. It is a manual process that is time-consuming and open to interpretation, which can vary depending on the evaluator.

The Emotiv Insight device has certain limitations that affected the development of the experiment. Although the device is portable and provides a stable connection via Bluetooth, it fails to obtain stable readings in individuals with voluminous or thick fiber hair. This creates difficulties in stabilizing brainwave reading flows. During tests with unstable signal acquisition, an attempt to normalize the results obtained was noticed, either by the device or by the API provided by the manufacturer. This resulted in readings that were incompatible with the participant's actual mental state. Additionally, the type of results obtained, and the format of the output files are limited by the manufacturer's business model. Certain features are blocked unless one subscribes to paid plans, such as obtaining unprocessed EEG data and exporting the results in one's own formats.

A significant limitation is the issue with audio, as students can look at things outside the screen without losing their focus in class. This means that changes in neural patterns may not be identified. Attention remains constant and focused on the class content. However, if something external grabs the student's attention, dopamine levels naturally increase. This shift in attention can be detected through alpha waves, which indicate changes in dopamine levels and their relationship with shifts in focus [51,52,53,54,55]. In the study discussed in this article, no changes in alpha waves were found. One possible reason for this is the short duration of the videos and the characteristics of the tests, which did not cause shifts in attention. This should be taken into account in future tests.

Moreover, it's important to note that the results described are from an initial study intended to highlight potential uses of these technologies in the context of distance learning. The results indicate the potential for mapping student attention and engagement during classes. However, it's important to recognize that there is still work to be done before this research becomes accessible technology for teachers. It's crucial to find ways to visualize the data in an easy and understandable manner for any teacher through an online graphical interface, without having to deal with technical elements related to EEG or eye tracking. This aspect is also a suggestion for future work.