Keywords

1 Introduction

In recent years, the number of students with learning disabilities has increased in Japan. According to a survey by the Ministry of Education, Culture, Sports, Science and Technology (MEXT), it was 18.4% in 2012, but it increased up to 28.7% in 2022 [1]. Special educational support is required for these children, and as a part of this, special instructional classrooms have been established to provide that [2]. However, special instructional classrooms and the special educational support there have not spread sufficiently, so MEXT prepared guidance web for unfamiliar teachers [3] and published a collection of practical examples [4] in 2020.

The students there suffered from various kinds of learning disabilities, such as difficulties in learning and behavioral aspects. In this study, we focused on eye-movement training, which is expected to be effective for learning such as reading and writing. Eye-movement training is reported to be effective in improving learning and behavior by promoting eye movements in addition to visual acuity [5].

Authors have already developed an eye movement assessing and training system using a PC and an eye tracker, developed an original index to evaluate eye movements during eye-movement training, and demonstrated its effectiveness by comparison with assessments by experts [6, 7]. In this study, we propose a system that allows students to perform eye-movement training alone in special instructional classrooms.

2 Concept

The availability of eye-movement training using PCs and eye tracking devices in special education classrooms would be an effective solution for students with learning difficulties. However, to propagate this kind of system, it is necessary to consider that teachers are not accustomed to special instructional classrooms and that the teaching materials used there are created through trial and error to fit each student. In addition, contrary to the national standard, one teacher is assigned to every 13 students [8], but, currently, a teacher must be in charge of an immense number of students [9] – for example, about 90 students in our case. Based on this, we propose the following eye-movement training system for special instructional classrooms.

  1. 1.

    Teachers install an eye-movement training system on the desks of special instructional classrooms.

  2. 2.

    Students there conduct eye movement assessment/training by their own will and by following teachers’ advice.

  3. 3.

    The students who want to use the system can use it alone.

  4. 4.

    In 3, students can use the system in an enjoyable way as if they are watching attractive videos or playing games but not in the assessment procedure.

  5. 5.

    In 3, we make the assessment accurate and precise in order to measure daily eye movement and the training procedure.

  6. 6.

    In 3–5, we make it available to record a log of system usage in each class and for each student.

Item 1 enables us to operate the system in typical special instructional classrooms without preparing specific facilities. Items 2 and 3 are essential in ensuring little or no increase in teachers’ workload because students can use the system alone or with slight teacher assistance. Item 4 is used to promote the system’s use by students themselves. Item 5 is necessary to make the system as an assessment and training tool. Item 6 is useful for personal instruction by recording, checking, and analyzing students’ attitudes in daily use.

3 System Development

3.1 Hardware Development

Before developing the system proposed in Sect. 2, we first considered item 1 so that we could install the system on a desk in a regular elementary school classroom. Here, we checked the desk size (600 mm × 400 mm according to the old JIS standard, 650 mm × 450 mm according to the new JIS standard) and used a 17.3-inch display (cocopar, JSJ-173). We used a mini-PC (Intel, NUC11PAHI5) that could be attached to the VESA mount on the back of the display. An eye tracker (Tobii, Eye Tracker 5) and a web camera (Logitech, C922n) were connected to the PC, making it possible to capture gaze and videos.

Additionally, we developed an enclosure to house this hardware. This enclosure was designed to minimize sharp corners for safety during use in special instructional classrooms, evoke a sense of desire for use by students, as proposed in item 2, and eliminate the appearance of a typical inspection device as in item 4. Consequently, we designed a friendly, rounded enclosure as depicted in Fig. 1. This enclosure was 3D printed and housed components such as the aforementioned display and PC. Additionally, we applied a soft, warm acrylic coating as shown in Fig. 2.

Fig. 1.
figure 1

Hardware design.

Full size image
Fig. 2.
figure 2

Overview of the eye-movement training systems.

Full size image

3.2 Software Development and Operation Confirmation

We installed Windows 10 as an OS on the mini-PC. The development was done using Unity 2020.3.18f1 from Unity Technologies and Tobii Unity SDK for Desktop. For items 3 and 4, we developed a software capable of playing MP4-format videos and measuring gaze during eye-movement training while watching videos. Instead of introducing gaze interaction, where gaze may be lost for some time, we adopted video watching. This decision was aimed at encouraging continuous training by making the system enjoyable during daily use. Furthermore, the software allowed users to store multiple videos, enabling them to freely select from a list of videos they wished to play (Fig. 3).

Fig. 3.
figure 3

A screenshot of video list.

Full size image

Figures 4 and 5 illustrate examples measured in using this hardware and software. The display resolution was 1920 × 1080 pixels, with the top-left corner of the display serving as the origin for plotting the coordinates of the gaze point. In Fig. 4, the plotted values represent the x and y coordinates of the gaze point when the object of fixation jumps from right to left, showing the movement of the gaze point. Similarly, in Fig. 5, the plotted values represent the x and y coordinates of the gaze point when the object of fixation moves from right to left, demonstrating the movement of the gaze point.

Fig. 4.
figure 4

An example of measurement smooth saccade.

Full size image
Fig. 5.
figure 5

An example of smooth-pursuit measurement.

Full size image

Furthermore, we implemented a system in which one account is issued per classroom where the system is installed. This allows each student to create and select their own user profile, enabling the storage of individual usage history and other related data to comply with item 6. The video contents of the saccadic eye-movement training and pursuit eye-movement training are created, as shown in Figs. 6 and 7, respectively. These videos feature targets such as characters that participants can track with their eyes to engage in the respective training exercises.

Fig. 6.
figure 6

Examples of the video for saccade training (up: fox, down: demon slaying)

Full size image
Fig. 7.
figure 7

Examples of the video for pursuit training (up: ghost, down: Valentine’s Day)

Full size image

3.3 Mini-game for Gaze Measurement

To ensure accurate eye tracking measurements, even when a child is using the system by themselves, as proposed in item 5, it is crucial to maintain the head within the range where eye tracking is feasible. Therefore, we develop mini-games to guide the position of the head appropriately, as shown in Figs. 8 and 9, because many children in special instruction classrooms tend to have ADHD. In Fig. 8, the head position is represented by the meat and frying pan, with the range of measurement displayed by the stove. By moving the head to the position where the meat is cooking on the stove, the head position is appropriately adjusted. Similarly, in Fig. 9, the facial puzzle panel is used to guide the movement of the head position, encouraging it to fit into the panel accurately.

Fig. 8.
figure 8

Implemented head position guidance scene (grilled meat).

Full size image
Fig. 9.
figure 9

Implemented head position guidance scene (face panel).

Full size image

On December 13 and 14, 2022, a two-day experiment was conducted in a special instructional classroom in Izumi City Kokufu Elementary School, Osaka, Japan, to determine whether the system could be adequately used, especially whether the developed mini-game could be properly integrated into eye tracking. One of the authors described this experiment to the participants, and parents’ consent was obtained. Over the course of the experiment, over 80 children came to the classroom. About 20 of the students participated in this experiment.

During this experiment, the authors acted as experimenters, providing verbal instructions and explanations of the system without adjusting the positions of chairs or tilting the displays. As a result, all children were able to position their heads within the designated range and performed the training (Fig. 10). Additionally, it was observed that the children enjoyed the interactive experience of objects moving in correspondence with their head movements (Fig. 11).

Fig. 10.
figure 10

Position of the students’ head.

Full size image
Fig. 11.
figure 11

Measurement experiment by experimenter.

Full size image

4 System Operation Evaluation

4.1 Overview

In Sect. 3, we developed a system that satisfies items 1–6. We installed the system in a special instructional classroom of Izumi City Kokufu Elementary School and conducted an operation evaluation. Here, after only explaining how to use the system in advance, we did not go to the site during the period, and we allowed the children to use the system by themselves without specifying the type of video, number of training sessions, and training time (Fig. 12). We prepared one video each for training and measurement, as shown in Figs. 6 and 7, respectively, with three types each for saccadic eye movements and smooth-pursuit eye movements. The operational period spanned 7 weeks from January 30, 2023, to March 20, 2023, totaling 35 days, excluding weekends and holidays.

Fig. 12.
figure 12

Experimental scenery in Kokufu Elementary School.

Full size image

4.2 Evaluation of Measurement Accuracy

First, we evaluated the measurement accuracy from January 30 to February 13. Here, we calculated the ratio of measurement (measured frames/total frames) by judging whether the users’ heads were in the range of measurement. We also compared the results with the cases of full assistance (experimenters adjusted chair positions and display tilts, performed in July 2022) and verbal assistance (experimenters offered advice for properly using the system, performed in December 2022). In full assistance, 23 training sessions were conducted by 23 participants, and in verbal assistance, 100 training sessions were conducted with 24 participants. In this no-assistance evaluation, 135 training sessions were conducted with 24 participants. The results are shown in Fig. 13. The average ratios of measurement for full assistance, verbal assistance, and no-assistance were 93.3%, 88.7%, and 75.8%, respectively. Excluding the three individuals who exhibited extremely low ratio of measurement in the no-assistance evaluation, the average was 81%. Figure 14 illustrates the actual measurement examples. According to the introduction of the mini-game developed in Sect. 3, eye tracking accuracy was at the same level compared to full or verbal assistance. Thus, the development of an eye-movement training system that children could use by themselves proved to be successful.

Fig. 13.
figure 13

Results of the ratio of measurement depending on types of assistance.

Full size image
Fig. 14.
figure 14

Examples of gaze measurement results with or without assistance.

Full size image

4.3 Operation Results

As shown in Sect. 4.2, accurate gaze tracking was achieved. Consequently, the system was operated until March 20, 2023. Throughout this operational period, 31 individuals participated in the training, totaling 368 training sessions. Figure 15 illustrates the number of training sessions per user. Seventeen children utilized the system seven times or more (approximately once a week) on average. Additionally, approximately 90 children attended the special needs education class that year.

Fig. 15.
figure 15

Number of training sessions per user. (Color figure online)

Full size image

To assess the training status, we conducted an analysis focusing on the measurement data of eight students (highlighted in orange in Fig. 15) who had a high number of training sessions and consistently used the system throughout the period. The results of this analysis are summarized in Table 1.

Table 1. Eye movement characteristics of frequently used users.
Full size table

Here, we labeled the eight individuals from the collaborators with a high number of training sessions as A to H. We utilized the Seo, developed in previous research [6, 7], as an index for the effectiveness of training. Additionally, we calculated the correlation coefficient between the number of training sessions and the values of the index for each student. Here direction 1 was from top right to bottom left. Direction 2 was the opposite.

Because a lower Seo value indicates better eye movement, it is desirable to observe a negative correlation with the training period. However, in the results of this operational evaluation, small correlation was observed. In this special instructional classroom, eye-movement trainings without our system have been performed, and this experiment was conducted at the end of the fiscal year. So, the eye movement abilities of the students may have already improved beyond the level of those with learning difficulties.

Another reason for this was the lack of concentration of students in training. This resulted in the inability to accurately measure participants’ real capabilities. For instance, Table 1 shows several instances of data that were not analyzable; for example, only 75% of the data were suitable for the analysis of training effects in the case of participant C, and only 63% of the data were suitable for participant H. To verify whether the children were concentrating during the measurements, we compared the data when the eye movement error values were small and when they were large in the daily measurements of participant A (Fig. 16). The comparison suggested that the eyes may not have moved the specified number of times on certain days.

Fig. 16.
figure 16

Comparison of measurement results when the value of eye movement error is at its minimum and maximum (saccade).

Full size image

5 Conclusion

In this study, we proposed an eye-movement training system that can be used by individual students in special instructional classrooms, developed the system, conducted operational evaluation experiments, and demonstrated its effectiveness. In particular, we proposed a system where students could perform eye movement assessment and training while watching videos using a PC, and an eye tracking device was effective. According to the repeated operational evaluations and ongoing problem-solving efforts, we successfully developed a system that students can use alone easily. Although the teaching materials in special instructional classrooms are still evolving, we hope this system will be widely used to overcome difficulties in reading and sports skills at an early stage.