Keywords

5.1 Introduction

With the development of information technology, the accuracy and performance of measurement instruments for biological signals have improved, and new analysis methods have been presented [1]. Such measurement instruments are used extensively in research and medical settings. For example, techniques such as electroencephalography, functional MRI, and near-infrared spectroscopy (NIRS) are used in the field of brain physiology [2,3,4]. Electrocardiograms (ECGs) and electrogastrograms (EGGs), which measure action potentials in the body, are also carried out using specialized equipment [5, 6]. Moreover, eye movements are an important type of biological signal data [7].

The eye perceives light energy from the outside world through the photoreceptors of the retina. The received information is transmitted through the nervous system to the brain center, where it is perceived as vision. Two types of photoreceptor cells exist in the retina: rods and cones. The rods are a function of high sensitivity to light and operate while observing objects in the dark. The cones operate in bright areas and respond selectively to different light wavelengths. The three types of human cone cells are classified as the L-cone, M-cone, and S-cone, based on the longer light wavelengths that they absorb. Furthermore, an area of concentration of cones exists, which is known as the central fossa. Therefore, for a human to obtain a clear view of an object, the object must be captured in the central fossa. Six external eye muscles, namely the external rectus, internal rectus, superior rectus, inferior rectus, inferior oblique, and superior oblique, move the eye to capture an object in the central fossa. Thus, eye movements occur based on the structure and function of the eye [8, 9].

The human gaze is controlled by the action of the external eye muscles that move the eye. In this chapter, we outline the types of eye movements. Moreover, we introduce several methods for measuring eye movements.

5.2 Types of Eye Movements

Eye movements can be categorized as saccade (saccadic eye movement), smooth pursuit eye movement, vergence (convergence and divergence) eye movement, optokinetic nystagmus (OKN), vestibulo-ocular reflex (VOR), and rotational eye movement [8, 9].

5.2.1 Saccade (Saccadic Eye Movement)

The saccade is a rapid eye movement, whereby the maximum velocity increases with the amplitude (amplitude: 0.5°–40°, speed: 30–700°/s) [10, 11]. A saccade cannot be intentionally stopped once the exercise begins. In general, the saccade stops slightly ahead of the target, and in such cases, a small saccade (corrective saccade) will eventually bring it to a halt at the target position. Furthermore, saccades require an interval of at least 200 ms if they occur consecutively, and no visual information is entered during the saccade.

5.2.2 Smooth Pursuit Eye Movement

Smooth pursuit eye movement is a smooth, gradual conjugate eye movement that captures and follows the target in the central fossa. Smooth pursuit eye movement is prominent in primates with developed fovea centralis, such as humans and monkeys. In humans, the eye can be moved smoothly up to approximately 50°/s. A visual object is constantly required for smooth pursuit eye movement to occur and the ratio (gain) of the eye velocity to the visual object velocity is 0.7–0.9 in humans. Moreover, the correction of the position is adjusted for saccades.

5.2.3 Vergence (Convergence and Divergence) Eye Movement

Convergence is a condition in which the left and right eyes move close together, as when looking at a nearby object. Convergence is a reflexive and voluntary movement. Divergence is shifting the point of the gaze from near to far. The reaction time of convergence/divergence eye movements is 160 ms and the maximum speed is 20°/s, which is slower than the other eye movements.

Convergence/divergence eye movements differ from other eye movements in that the movements of both eyes exhibit an opposite directional component. When gazing at near and distant objects, parallax occurs in both limits. The fused convergence eye movements attempt to eliminate the retinal shift of the gazing point. In contrast, another mechanism allows the retinal misalignment (parallax) outside the gaze point to be fused by processing in the brain. In general, the latter function is a major factor in stereopsis, whereas accuracy is not significantly involved in depth perception.

5.2.4 OKN

One of the eye movements that fixates a moving visual target on the retina is the OKN. VOR also has a similar function of stabilizing the moving visual object on the retina. However, OKN fine-tunes the slip, which the VOR fails to compensate. The two components of the OKN eye movement are rapid and slow responses; the tracking of a moving visual target is typically achieved by a combination of slow and rapid movements [12].

5.2.5 VOR

The VOR is an eye movement that automatically compensates for the change in the gaze direction when the head is rotated without tilting. Head and eye movements are linked to one another while observing an object, and the eyes can remain fixed on the object. The VOR also occurs while the eye is closed, but with a smaller amount of eye movement.

The VOR is controlled by a head movement sensor known as the semicircular canal in the ear and by a neural pathway known as the vestibule that controls the sense of balance. Moreover, signals from sensors that detect the stretching of muscles in the neck and other parts of the body affect eye movement. The VOR is a reflex mechanism for rotational stimuli because the reflexes associated with the rotation of the head and body occur in response to input from the semicircular canals. When the head is tilted to the left or right, rotational eye movement occurs.

5.2.6 Rotational Eye Movement

Rotational eye movement is the movement of the eye as it rotates around a gaze axis. It is generated by two types of stimuli: those from the vestibule (especially the otolithic organ) and visual stimuli.

Rotational eye movements that are generated by vestibular stimulation are referred to as vestibular counter-rolling (vestibular torsional counter-rolling). When the body (head) is tilted in either direction, the eyeball rotates in the opposite direction to that of the body (head) tilt, thereby preserving the vision [13].

Rotational eye movements are also a reflection of linear acceleration from the otolithic organ.

It has been suggested that ocular torsional counter-rolling occurs in motion sickness, whereas a postural wobble occurs as a physical symptom of motion sickness [14].

5.3 Measurement of Eye Movement

The modern scientific observation and measurement of eye movements are believed to have started with Muller [15, 16]. The two methods used at the time were the direct observation method, which allowed the naked eye to observe the iris pattern in the cornea (black eye) and the movement of capillaries on the sclera (white eye), and the after-image method, which quantitatively measured the movement of the after-image generated on the retina by projecting it onto a screen. Subsequently, Huey covered the eye up to the sclera with a plaster contact and used a technique that mechanically magnified the movement of the bar connected to the contact by using a lever and recorded the movement on paper [17].

At present, methods used to measure eye movements include the magnetic search coil method, pupil center corneal reflection (PCCR) method, limbus tracking method, image analysis method, and electrooculography (EOG). In this section, we introduce these methods and the related research.

5.3.1 Magnetic Search Coil Method

In the magnetic search coil method, an external magnetic field is created, a contact lens-shaped coil is placed in close contact with the eye (usually the sclera), and the eye movement is measured as the potential induced by the coil [18, 19]. Therefore, it offers high accuracy and a wide measurement range. Furthermore, it exhibits the ability to measure the eye rotation motion [20, 21].

The magnetic search coil method is used in the diagnosis and measurement of pathological nystagmus owing to its high accuracy. For example, it is known that congenital nystagmus is a common cause of nystagmus in children and that the nystagmus diminishes over the natural course of time. The magnetic search coil method has been used to inspect nystagmus with high accuracy [22, 23]. In recent years, the magnetic search coil method has been used in virtual reality (VR) and augmented reality (AR) scenarios. VR and AR are used for eye tracking using wearables, headsets, and other devices. Many of these techniques rely on optical tracking, with an accuracy of approximately 0.5°–1°. In contrast, devices for VR and AR using the magnetic search coil method can estimate the orientation of the eyes with an average accuracy of 0.094° [24].

5.3.2 PCCR Method

In PCCR, a light source is shone onto the cornea to identify the light reflection point and pupil on the cornea, and the eye direction is calculated based on the light reflection point and other geometric features [7, 25, 26]. The PCCR method has been used for a long time owing to its large amount of reflected light, and it remains the most widely used method. In principle, when the cornea is irradiated by a light source, four reflection images (the first to fourth Purkinje images) appear. These images are reflected at the anterior surface of the cornea, posterior surface of the cornea, anterior surface of the lens, and posterior surface of the lens. Among these, the reflectivity of the anterior surface of the cornea is large, at approximately 2.4%, making it the brightest reflection image. Compared to the reflected image on the anterior surface of the cornea, the other reflected images are darker and therefore negligible in the measurement of eye movement. Furthermore, a “glint-free” method that does not use these Purkinje images has been proposed, but it is not yet at the practical stage [27]. In recent years, instruments that are capable of capturing a wide range of corneal reflections have been developed to study the limiting points of corneal reflections and to derive the movement range of the camera and light source to capture the cornea [28], which is expected to expand the use of the system further.

5.3.3 Limbus Tracking Method

The limbus tracking method is a technique in which a light source is irradiated on the cornea (black eye) and sclera (white eye), and the eye movement is detected based on the difference in reflectivity between the two [7, 29].

The limbus is the boundary between the cornea and sclera. The reflectivity from the cornea and sclera is significantly greater than the reflectivity from the sclera. When infrared light is shone near the limbus, the amount of reflected light also changes as the ratio of the area covered by the cornea to that covered by the sclera changes owing to the movement of the eyeball. The eye movement is detected by the Limbus tracking method according to the amount of change.

The limbus tracking method uses infrared light instead of visible light to avoid glare for the subject. The infrared light is irradiated below the center of the cornea and the amount of reflected light is measured. Therefore, the limbus tracking method mainly focuses on horizontal eye movements, and it can also measure rapid movements such as saccades.

A study using the limbus tracking method assessed the level of arousal based on the measurement of eye movements of reading horizontal text strings [30]. The results of this study are expected to be developed for the assessment of driver arousal levels in automated driving. Research has also been conducted on security systems using eye gaze measurement technology [31].

5.3.4 Image Analysis Method

The accuracy of image analysis has improved dramatically with the development of technologies and methods relating to data science. The image analysis technique is also commonly used to measure eye movements by capturing a photograph of the eye and analyzing the image with a computer [32, 33].

The most convenient image analysis method is detecting and recording the center position of the pupil. Because the pupil area is dark, and only needs to be binarized by the thresholding and calculation of the center of gravity coordinates, it is also possible to measure the eye movements in real time. Moreover, if the center of the pupil and corneal reflection image are analyzed simultaneously, it is possible to cancel the movement of the head. Previous studies have measured the ocular rotational movements by including the pattern of photophores in the acquisition data [34].

5.3.5 EOG

EOG is a method for recording the electrical phenomena related to eye movements. The eye is positively charged on the corneal side and negatively charged on the retinal side. In EOG, the potential difference between the cornea and retina is measured by stretching electrodes around the eye [35, 36].

People with disabilities, particularly total paralysis, are often unable to use biological communication channels such as voice and gestures, resulting in the need for digital communication channels. These methods for assisting people include the acquisition of eye movement and gaze data by EOG, and communication by inputting information to information devices [37]. Certain studies have specifically addressed the real-time measurement of cancer through uvula motion and gaze data by EOG [38, 39].

5.4 Development of an Index for Evaluating VIMS Using Gaze Data

In the following sections, we introduce studies on the development of an evaluation index for visually induced motion sickness (VIMS) using eye gaze data.

VIMS is considered as a type of agitation illness that is thought to be caused by the disharmony between visual information and the vestibular system, such as the tricuspid canal. Therefore, the eye movement control system is expected to be involved. The physical symptoms of VIMS include feeling unwell, nausea (upset stomach), dizziness, and vertigo when standing up. The indices used to evaluate VIMS are the results of simulator sickness questionnaires (SSQs), which are the main evaluation method, and measurements of the gravity center sway of the participants while standing [40, 41]. It is also known that if motion sickness causes unsteadiness, rotational eye movement will occur in the direction opposite to the unsteadiness, so that the individual’s eyes do not tilt the field of view [14]. This rotational eye movement can be used as a physiological index of VIMS, and evaluations have been carried out based on electromyographic data from near the eyes. Furthermore, a method for evaluating nausea based on EGG measurements is currently being studied as a new physiological index. However, studies using physiological indices have not obtained consistent results for the complex changes that occur with the progression of motion sickness. As electrodes must be attached to test participants and specialized equipment must be used to obtain electromyograms or EGGs, these options are only feasible in limited circumstances. Although SSQ-based introspective reports permit evaluations with a considerable amount of liberty, in certain cases where the symptoms of VIMS have progressed to the point of subjective assessment, such symptoms persisted for approximately a week.

5.4.1 VIMS in Driving Simulator

With the increasing number of elderly drivers on roads, the prevention of accidents involving elderly drivers has become a serious challenge. Research on elderly drivers includes studies on their visual and cognitive functions as well as their driving characteristics to help them to drive more safely. While research on elderly drivers primarily involves actual vehicles on ordinary roads, several studies using driving simulators (DSs) exist. The merits of experiments using DSs include a low risk of accidents or other operational problems, easy setting and reproduction of specific conditions and situations, and the ability to adjust the experimental conditions. In contrast, the demerits of using DSs include the occurrence of VIMS, lack of a sense of reality, and high cost. In recent years, the performance of DS hardware has improved and its cost has decreased; thus, the reality and cost problems are gradually being resolved, but an effective VIMS prevention measure has not yet been developed. Inversely, as the field of view expands, the symptoms of VIMS are intensified. Therefore, as display screens become larger, the risk of VIMS may also increase. When a DS experiment is performed, it is necessary to detect VIMS precursors promptly before the symptoms become severe.

The use of a noncontact eye-tracking system is considered a low burden method for measuring biological signals. The goal of this study was to develop a VIMS evaluation index that uses a noncontact eye-tracking system for DS experiments.

As rotational eye movement is caused by unsteadiness resulting from VIMS, the following hypotheses are presented.

  • Hypothesis 1: VIMS symptoms occur; the locus of the eye-tracking data lengthens.

  • Hypothesis 2: VIMS symptoms occur; the eye-tracking data are diffused.

The participants were nine elderly people who had visual and balance functions that did not interfere with their daily life. The gaze data were measured at rest before and after DS driving (5 min of driving, 5 trials). The resting gaze data were obtained by the participants gazing at the center of the DS screen for 1 min during the measurement. SSQs were conducted before and after the start and end of the experiment.

The instrument used for measuring the gaze data and the analysis software were the Tobii Pro X2–30 (sampling rate: 60 Hz) and Tobii Pro Studio (ver. 3.3.2), respectively. The gaze data were plotted corresponding to the resolution (640 × 480 pixels) of the scene camera (Logitech HD Webcam C270).

The participants were divided into two groups based on the SSQ results. One group experienced VIMS during the DS driving (4 people, average of 79.0 years old). The other group did not experience VIMS during the DS driving (5 people, average of 71.2 years old).

5.4.2 Evaluation of VIMS Using Gaze Data

Figure 5.1 presents the calculation results of the total locus length for each participant based on the eye-tracking data at rest to determine the average for each case of “no previous or previous experience” of VIMS before and after (pre-/post-) DS driving. The total locus length results indicate that, for both the pre-DS and post-DS driving, the total locus length data were longer for those with than for those without previous experience of VIMS. Furthermore, among those without previous experience, no difference was observed between the total locus lengths of the pre-DS and post-DS driving. However, among those with previous experience, the total locus length tended to be longer for the post-DS driving (p < 0.1).

Fig. 5.1
figure 1

Total locus length of gaze

Full size image

Similar to the total locus length, the sparse density was calculated based on the eye-tracking data to obtain the average for each set of experimental conditions, as illustrated in Fig. 5.2. The sparse density is a quantification index that is represented by a scatterplot of the data on a plane, and the diffusion of the data increases its value. In the sparse density results, the post-trial value was higher than the pre-trial value for both the participants with and without experience of VIMS. Among those without any experience of VIMS, no difference was observed in the sparse density between the pre-DS and post-DS driving results. Among those with previous experience of VIMS, the sparse density value was significantly higher after the DS driving (p < 0.05).

Fig. 5.2
figure 2

Sparse density of gaze

Full size image

The two hypotheses concerning the results of VIMS symptoms were confirmed: the loci of the eye-tracking data were lengthened and the eye-tracking data were diffused. Moreover, the experiments demonstrated the usefulness of sparse density as a quantification index for eye-tracking data in evaluating VIMS (Fig. 5.2).

The results of the total locus length in the eye-tracking data revealed that the locus was longer after the DS driving in the group with previous VIMS experience, suggesting that among elderly people who are susceptible to VIMS, the unsteadiness of the line of vision may have been normalized.

Regarding the application of the findings of this study, it is believed that if an eye-tracking data-based VIMS evaluation index can be used, it will be easier to detect VIMS caused by DS operations, thereby permitting detection of the symptoms while they are still at the developmental stage. Furthermore, by coding the VIMS evaluation index algorithm into a program, it will be possible to develop a real-time automatic VIMS detection system to help to reduce the load on participants after viewing stereoscopic motion images or while wearing head-mounted display devices. Moreover, the findings will not be limited to VIMS; they can also be applied to the treatment of disorders relating to rotational eye movements (nystagmus).

5.5 Conclusions

The prevention of VIMS and the treatment of nystagmus disorders are important topics in the field of hygiene. In this chapter, we have outlined the types of eye movements and have introduced several methods for measuring eye movements. The use of gaze data is a highly effective method for acquiring biological signals. In the next step, gaze data are expected to be used for evaluating the performance of new human–machine interface devices as in VR and AR.