Keywords

1 Introduction

Distraction is a general phenomenon in daily life. In some circumstances that require a driver’s close attention to road conditions, distraction degrades safety and traffic flow [22], and occurs frequently at all ages [14]. To reduce traffic accidents caused by distractions, many studies have investigated distracted driving. Most studies specifically examine distracted driving caused by external factors such as mobile phones [4, 15, 16], in-vehicle information systems [2], passengers [3, 10, 19], and music and auditory materials [6, 7, 24].

There is one type of distraction that has nothing to do with external factors: inattentive driving [11]. This distraction is difficult to examine, but it is extremely dangerous and difficult to control. Based on National Police Agency of Japan data, careless driving (inattentive driving) is the most common reason reported for all fatal traffic accidents attributed to a violation of safe driving practices (Fig. 1). To reduce traffic accidents and to resolve traffic difficulties of all kinds, Intelligent Transport Systems (ITS) have become a mainstream field of study, especially to alleviate inattentive driving.

Fig. 1
figure 1

Annual fatal accidents in Japan data [17] by type of violation involving primary parties (based on the National Police Agency of Japan data)

Full size image

Because it is difficult to ascertain and control the timing of an inattentive state, subtasks while driving, such as mathematical operations, are generally used to study inattentive driving. When examiners ask participants verbal questions while driving, drivers blink more frequently, and their fixation time is reduced [23]. Harbluk et al. [5] used easy tasks and difficult arithmetic tasks to assess distracted driving, and found that the mean number of braking events with difficult tasks during driving was higher than that without a task. When driving while following other vehicles, brake reaction times are longer if drivers are doing mental arithmetic [8]. If numerous vehicles are on the road and the driver is doing mental arithmetic while driving, then the frequency of eye fixation time is increased at the 0.1–1 s time interval, and eyes move more often. With few other vehicles on the road, the result is the opposite [1]. Louie and Mouloua [9] used mathematical problems that are commonly encountered in life (e.g., the total price of 2 boxes of $3 turkey). Slower braking reactions occurred when a yellow traffic light appeared and during the sudden appearance of vehicles when distracted. Some studies have used subtasks such as memories. Akiyama et al. [1] used numerical memorization tasks, and Ross et al. [20] studied the effect of a verbal working memory load task on driving behavior. Driving performance deteriorated with increasing verbal working memory load in terms of the mean deviation in the lane change path, lane change initiation, and the percentage of correct lane changes.

Some studies have measured eye movement during distracted driving [13, 25]. Metz et al. [12] compared the visual attention in driving for two visual secondary tasks and found that drivers had a longer fixation in a driving simulation. Savage et al. [21] studied the effects of mobile phone conversations on eye movement during driving and found higher saccade peak velocities, increased blink frequencies, and a reduction in the spread of fixation along the horizontal axis. Wang et al. [27] designed an experiment with and without a mobile phone and found that compared to drivers without mobile phones, drivers with them had higher numbers of glance transitions and shorter on-road glance duration during distracted driving.

Drivers will often think or recall something during inattentive driving. Furthermore, an image might appear in the person’s mind during recall. To ascertain the manner by which and the degree to which this process affects drivers, we designed two subtasks of mental arithmetic and map recall to simulate thinking and recall processes during driving.

An inattentive state has a significant impact on eye movement [26], and driving behavior has a direct relationship with traffic accidents. We examined the influence of mental arithmetic and map recall on driver behavior when a car or a pedestrian suddenly appeared in front of the vehicle. Driving behavior and eye movements were captured using the driving simulation and eye movement measurement apparatus. We measured brake reaction time, the number of steering wheel rotations, fixation time, angular eyeball speed, and the direction of focal point movement.

2 Methods

2.1 Participants

Thirteen participants (10 men, 3 women) from Kyushu University participated in this experiment. Participants were 22–29 years old, and each participant held a valid driver’s license.

2.2 Equipment

Hardware. The simulator was controlled using a PC (Precision T1700; Dell Inc.). The image was displayed on a 27-inch liquid crystal monitor (T 270 W; Hyundai IBT Co. Ltd.) with a resolution of 1,920 × 1,080 pixels. A simulated vehicle steering wheel, accelerator pedal, and brake pedal were used (Driving Force GT, Logicool; Logitec Corp.). An eye movement measurement apparatus (EMR-9; NAC Image Technology Inc.) was used to measure eye movements.

Software. The driving simulator control program was created using software (UC-win Road Ver. 6.00.02; Forum 8). Data related to brake operation, steering operation, and other vehicle parameters were measured using the UC-win Road Drive Log plug-in, which we created in our laboratory. The system frame rate was about 50 fps in all experiments. Reaction time accuracy was about 20 ms. Eye movements were analyzed using software (EMR-dFactory; NAC Image Technology Inc.).

2.3 Design

To find driving behavior and eye movement differences between participants driving while doing a subtask and without doing a subtask, four types of roads were designed: S1, S2, C1, and C2. Roads S1 and S2 were straight lines with three intersections. Because the buildings on road S1 were taller and closer to the road, the view of the driver on road S1 was narrower than that on road S2. At the intersection, another vehicle might suddenly cross the vehicle path. At this point, we measured the driver’s emergency braking reaction time. Roads C1 and C2 were curved roads: road C1 was more curved than road C2. The purpose of the curved roads was to study the direction change, the right and left fine adjustment, of the steering wheel. In this experiment, all participants drove six road combinations with a driving simulator. All road combinations comprised the four roads: S1, S2, C1, and C2 (Table 1). A three-second break was inserted between roads.

Table 1 Road combinations of six types
Full size table

2.4 Tasks

Main task. Figure 2 presents some captured monitor screens. All participants were required to drive at up to 60 km/h. They were able to brake when any possibility of a road incident was presented.

Fig. 2
figure 2

Three road incident situations

Full size image

Subtasks. The subtasks were for participants to do mental arithmetic and drive during map recall.

Mental arithmetic. With an increase in the difficulty level of arithmetic calculations, the brake reaction time increases gradually [29]. To prevent the calculations from being too simple to affect driving, six sets of mental arithmetic of double digit sums and subtractions were used in the six road combinations. The answers to mental arithmetic questions were 0–99 (Table 2). Participants had 3 s to answer each question before the next question was asked.

Table 2 Examples of questions related to mental arithmetic
Full size table

Recall the map. Six maps were used in the six road combinations. Each map had markings for seven locations, several roads, and north (Fig. 3). The map was displayed on the monitor for two and a half minutes before each driving test for the driver to memorize. The map then disappeared and driving started. While driving, the participant was asked map-information-related questions continuously. The questions were about directions and distances between several locations, location on the map, and the number of intersections between locations (Table 3). Participants had five seconds to answer before the next question was asked.

Fig. 3
figure 3

Example of maps in the map recall experiment

Full size image
Table 3 Examples of questions related to map recall
Full size table

2.5 Procedure

The participants were divided into two groups. The first group started with the mental arithmetic subtask. A week later, they were asked to continue the experiment for the map recall subtask. The other group participated in the experiment in the opposite order, and each participant spent around forty minutes a day for the experiment.

On the first day after the experiment was explained, participants were asked for their consent to participate. After an eye movement measurement apparatus was applied, participants practiced using the driving simulator. No sudden incidents were presented during practice mode, which was intended for participants to become accustomed to the system and road combinations. The requirements below were given to the participants.

  • The accelerator pedal should be pushed to the end, always maintaining driving speed at 60 km/h on smooth roads: 60 km/h was set as the maximum speed.

  • The brake pedal must be pressed immediately when a pedestrian or vehicle suddenly enters the road.

  • After pushing the brake pedal to allow a pedestrian or vehicle to pass, release the brake pedal and press the accelerator pedal to the end to continue the driving simulation.

The experiment started after completion of the process described above. First, we let participants drive through all six road combinations in the order of with-subtask—without-subtask—with-subtask—without-subtask—with-subtask—without-subtask. Second, we reversed the order of the with/without subtasks, and let participants drive through all road combinations. These two steps were to ensure that with-subtask and without-subtask situations had been met in all road combinations. We eliminated possible influences that the subtask order may have had on the experiment results.

We used R (3.5.0) software for statistical processing. Anovakun (4.7.0) was used for analysis of variance (ANOVA). Mendoza’s Multisample Sphericity Test was used to test for the assumption of sphericity. The degrees of freedom were corrected using the Greenhouse–Geisser method if the assumption was not valid. Multiple comparisons were used when a significant difference was found using Shaffer’s F-Modified Sequentially Rejective Bonferroni Procedure. Average values were used for without-subtask conditions because the without-tasks were conducted both days.

3 Results

3.1 Driving Behavior

Reaction time of braking. The reaction time was the time from the appearance of a vehicle or a pedestrian appearing on the screen to the time when the driver started to brake. Figure 4 shows the average of all participants’ reaction times for the three types of incidents during driving with subtasks, mental arithmetic and map recall, and during driving without the subtasks. Data were calculated using two-factor ANOVA according to the incident and the subtask type. A significant difference was found for the main effects of an incident (F (2, 24) = 38.5888, MSE = 0.0065, \( {\eta_{p}}^{2} \) = 0.7628, p = 0.0000) and for the subtask type (F (1.21, 14.54) = 10.4468, MSE = 0.0285, \( {\eta_{p}}^{2} \) = 0.4654, p = 0.0041). Furthermore, a marginally significant effect was found for the interaction of those two factors (F (4, 48) = 2.5277, MSE = 0.0032, \( {\eta_{p}}^{2} \) = 0.1740, p = 0.0527). For driving without a subtask, the reaction times of braking to the sudden appearance of a car on road S2 (0.657 s) and to the appearance of a pedestrian (0.701 s) were longer than that of the appearance of a car on road S1 (0.540 s). When driving with mental arithmetic, the reaction times of braking to the car appearance on road S2 (0.642 s) and the appearance of a pedestrian (0.668 s) were longer than that to a car appearance on road S1 (0.561 s). When driving during map recall, the reaction time of braking to a car appearance on road S2 (0.797 s) and the reaction time of braking to the appearance of a pedestrian (0.813 s) were longer than that to a car appearance on road S1 (0.628 s). For the car appearance on road S1, the reaction time of braking when driving during map recall (0.628 s) was longer than that when driving with no subtask (0.540 s) or when driving with mental arithmetic (0.561 s). For the car appearance on road S2, the reaction time when driving during map recall (0.797 s) was longer than that when driving without a subtask (0.657 s) or with mental arithmetic (0.642 s). After the appearance of a pedestrian, the reaction time when driving during map recall (0.813 s) was significantly longer than that when driving without a subtask (0.701 s) or when driving with mental arithmetic (0.668 s).

Fig. 4
figure 4

Reaction time of braking for all incidents

Full size image

Frequency of steering wheel fine adjustment. The number of times that the steering wheel was manipulated clockwise or counterclockwise to keep the car in the lane on a curved road. Figure 5 shows the average number of times that all participants were driving through roads C1 and C2. Data were calculated using two-factor ANOVA according to the type of road and subtask type. A significant difference was found for the main effect of roads (F (1, 12) = 69.1188, MSE = 117.6255, \( {\eta_{p}}^{2} \) = 0.8521, p = 0.0000) and for the subtask type (F (2, 24) = 21.8786, MSE = 38.5584, \( {\eta_{p}}^{2} \) = 0.6458, p = 0.0000). Furthermore, a significant difference was found for the interaction of those two factors (F (2, 24) = 13.4338, MSE = 16.9968, \( {\eta_{p}}^{2} \) = 0.5282, p = 0.0001). With regard to the type of road when driving without a subtask, the frequency of changes in the direction of the steering wheel on road C1 (29.4 times) was higher than that for road C2 (15.6 times). When driving while doing mental arithmetic, the frequency of changes in the direction of the steering wheel on road C1 (29.4 times) was more than that for road C2 (19.4 times). When driving during map recall, the frequency of changes in the steering wheel direction on road C1 (43.8 times) was more than that on road C2 (21.6 times). Regarding the subtask type, when driving with mental arithmetic (44.7 times) and during map recall (43.8 times), the participants rotated the steering wheel more frequently than without a subtask (29.4 times) on road C1. For driving with mental arithmetic (19.4 times) and during map recall (21.6 times), the average number of times that participants rotated the steering wheel was more than that without the subtask (15.6 times) on road C2.

Fig. 5
figure 5

Frequency of rotating the steering wheel on roads C1 and C2

Full size image

3.2 Eye Movement Characteristics

Fixation time frequency distribution. Figure 6 shows the average frequency distribution of different fixation times. Data were used with three-factor ANOVA according to the six types of road combinations, the subtask type, and the fixation time. A significant difference was found for the main effect of fixation time (F (15, 180) = 147.4924, MSE = 85.9559, \( {\eta_{p}}^{2} \) = 0.9197, p = 0.0000). Furthermore, a significant difference (F (30, 360) = 1.7999, MSE = 29.4200, \( {\eta_{p}}^{2} \) = 0.1304, p = 0.0072) was found for the interaction of the subtask type and fixation time. Compared to the without-subtask figure, the fixation time when driving with mental arithmetic had a lower frequency in the 0.5–0.7 s and 0.8–0.9 s time intervals; the fixation time when driving during map recall had a higher frequency in the 0.2–0.3 s time interval, but a lower frequency in the 0.7–0.8 s time interval. The frequency of the fixation time with map recall was higher than other tasks in the 0.2–0.3 s time interval.

Fig. 6
figure 6

Fixation time frequency distribution

Full size image

Traveling speed frequency distribution. Figure 7 shows the average frequency distribution of all participants’ angular eyeball speed. Data were calculated using three-factor ANOVA according to the six types of road combinations, the subtask type, and the angular eyeball speed. A significant difference was found for the main effect of the angular eyeball speed (F (1.14, 13.68) = 63.3727, MSE = 3087.5676, \( {\eta_{p}}^{2} \) = 0.8408, p = 0.0000). Furthermore, a significant difference was found (F (18, 216) = 2.9505, MSE = 87.0523, \( {\eta_{p}}^{2} \) = 0.1974, p = 0.0001) for the interaction of the subtask type and the angular eyeball speed. Compared to driving without a subtask, the angular eyeball speed with mental arithmetic was found to have a higher frequency in the 0–30 deg/s speed interval, but a lower frequency in the 150–180 deg/s speed interval. In the case of map recall, a higher frequency was found in the 0–30 deg/s speed interval, but a lower frequency was found in the 60–120 and 150–180 deg/s speed intervals. The frequency of traveling speed with mental arithmetic was higher in the 60–90 deg/s speed interval than that with map recall.

Fig. 7
figure 7

Angular speed frequency distribution

Full size image

Traveling direction frequency distribution. Figure 8 shows the average frequency distribution of all participants’ eye movement directions. Data were calculated using three-factor ANOVA according to the six types of road combinations, the subtask type, and the fixation direction. A significant difference was found for main effects of the angular speed of fixation direction (F (1.27, 15.22) = 89.9521, MSE = 1537.8463, \( {\eta_{p}}^{2} \) = 0.8823, p = 0.0000). Furthermore, a significant difference was found (F (14, 168) = 5.0601, MSE = 41.3128, \( {\eta_{p}}^{2} \) = 0.2966, p = 0.0000) for the interaction of the subtask type and fixation direction. Compared to driving without a subtask, the fixation with mental arithmetic had a lower frequency in the left and right directions, but a higher frequency was found in the up/right, up, and down directions. Compared with no subtask and with map recall, a lower frequency was found in the left and right directions with map recall, but the frequency was higher in the up/right, down/left, and down directions with map recall. For subtasks between driving with mental arithmetic and driving during map recall, the frequency of traveling direction was higher in the up/right direction.

Fig. 8
figure 8

Fixation direction frequency distribution

Full size image

4 Discussion

We investigated the influence of mental arithmetic and map recall on eye movement and driving performance. Each task weakened driving ability, increased driver burden, and altered eye movements.

Brake reaction times were longer for map recall than for driving without a subtask and driving with mental arithmetic in response to the sudden appearance of a car or pedestrian. When following a lead vehicle, memory tasks also increase brake reaction times [28]. Compared to driving with no subtask, no significant difference in reaction time was found with mental arithmetic during driving. This result demonstrates that mental arithmetic does not affect a driver’s ability to brake, which differs from other results [8]. This result may be true because of the lower difficulty of mental arithmetic, such as ample time for thinking, which has a lower burden on participants. Map memory had a strong effect, and drivers needed more time to slow. Under the same subtask condition, brake reaction time was significantly different for different incidents.

For all subtasks, brake reaction times to a car appearance on road S2 and a pedestrian appearance were longer than that to a car appearance on road S1. The brake reaction time was shorter for narrow field of view intersections (Road S1). The brake reaction time from a pedestrian appearance was longer than that from a car appearance on road S1. One likely reason is that the participants did not anticipate pedestrians entering into the road. Because of the narrow field of view for intersections on road S1, participants may have had increased vigilance. In contrast, the wide field of view intersections on road S2 may have resulted in decreased vigilance. Although the participants could not estimate which intersections will have vehicles suddenly entering the intersection, they understood that vehicles could only enter at intersections. Pedestrians could enter the road at any point, which was unpredictable. Therefore, the overall reaction time was longer when pedestrians entered the road.

The number of times rotating the steering wheel was higher for driving with mental arithmetic and driving during map recall than for without-subtask driving. Therefore, both subtasks increased driver burden, reduced driver attention to road conditions, and increased actions for correct driving.

Furthermore, a significant difference was found between two different curved roads, which suggested that driving on a more curved road was more difficult and required more steering wheel maneuvers.

Eye movement was expressed as a frequency of distribution. Both subtasks decreased eye fixation time, and eyes had more movement during with-subtask driving, which suggests that thinking or recall will increase driver burden and make it difficult for a driver to fix on one point for long periods. However, the frequency of the fixation time when driving during map recall was higher than with the other subtasks in the 0.2–0.3 s time interval. The influence of map recall was stronger than that of mental arithmetic. Recarte and Nunes [18] showed that fixation time was longer during a spatial-imagery task. Therefore, an image-related task increases the maximum fixation time, but the frequency of a long fixation time was not high.

Angular eyeball speed was lower for driving with mental arithmetic and driving during map recall. When driving without a subtask, the distribution of angular eyeball speed was highest in the 0–30 deg/s speed interval, and subtasks further improved this distribution. A slower angular speed might increase the time to recognize pedestrians or vehicles entering an intersection, which may increase brake reaction times. Wakui and Hirata [26] showed a horizontal saccadic movement with a peak speed less than 40 deg/s and at time intervals of less than 0.2 s increased while driving in an inattentive state. Combining their results and ours, the frequency of fixation time in the 0.1–0.3 s interval and travel speed in the 0–40 deg/s interval may be an important indicator of the inattentive state.

Eye direction frequency was lower in the left/right directions and higher in the up/down directions for mental arithmetic and map recall, which indicates that thinking or recall required the eyes to move frequently in the up/down directions. The frequency of the traveling direction with map recall was higher than when driving with mental arithmetic in the up/right direction. Subtasks increased driver burden and reduced driver awareness in the left/right direction.

We found that map recall decreased driving ability, which suggests that mental images impair real ones. Mental arithmetic and map recall subtasks require numerical analysis and comparison, and eye movements are produced throughout the whole driving process. Whether eye movements at intersections differ from periods between intersections needs more investigation.