Keywords

1 Introduction

Despite a solid foundation for the national economy and transportation industry has been providing, the rapid development of the transportation system has also caused a lot of traffic accidents and resulted in great casualties and economic losses [1]. Drivers are the main factors in the transportation system. Statistics show that 90% of all the traffic accidents are directly related to drivers [2] and 25%–37% of road traffic accidents are related to the distraction of drivers [3]. On average, drivers will be distracted every six minutes. The main reason for distraction is to adjust the facilities inside the car through the buttons [4], in other words, to do the secondary task through the central control platform.

The intelligence vehicles reduce the driving difficulty, but add more secondary tasks. Mobile phones, navigation and mobile office will make drivers “information overload”. Drivers’ attention is distributed due to more than one electronic equipment, which is not conducive to driving safety [5,6,7]. Integration and management of the information is the best way to reduce the risk of driving distraction [5]. In order to use as many electronic devices as possible in the limited space of the car, combining them into an integral human-computer interaction interface becomes an inevitable trend. A touch screen, similar to a tablet, can better integrate smart phones and web services into cars.

Touch screen has the advantages of strong expansibility. It can solve the problem relating to increasing buttons while the traditional mechanical key could only hold the post of one function. In addition, the touch screen gained the consumers’ affection with the high-tech feeling. The control platform is an important part of the interior decoration of the vehicle. Its appearance, texture and functionality directly affect the quality of the vehicle interior decoration, which has a great impact on customer satisfaction and user experience. Touch screens also meet the people’s enjoyment needs. Current drivers are beginning to become more youthful, and many young drivers said they are willing to accept text messages and voice mail in their cars [8].

However, it is undeniable that compared with mechanical buttons, there are some disadvantages of the virtual buttons of the in-car touch screen, which are mainly reflected on the characteristics of poor precision and lack of feedback. Virtual buttons lack muscle memory, a driver cannot distinguish keys through the button’s material, shape or fixed position, causing a poor precision. When necessary, the driver has to rely on visual to complete the secondary task. Focusing on the screen for a long time has a great impact on the driving safety. Secondly, lack of feedback from the virtual buttons makes the driver unable to determine whether the button is pressed and whether the instruction is executed, resulting in a much lower user experience. The literature shows that the difference of tactile feedback greatly influences user experience, and good tactile feedback can make task difficulty decrease [9].

Driving is a complicated task characterized by multiple tasks. When drivers do driving-related secondary tasks through in-car touchscreen, how to improve the user experience, meanwhile, distract drivers as little as possible and ensure driving safety becomes a matter of concern. In previous studies [10, 11], researchers have studied the input and output of hearing, touch, but the multimodal concept has not been used in the in-car touch screens to the best knowledge of the author. In this study, we design an interactive system of the in-car touch screen. Besides optimizing the visual design, we add auditory channel and haptic channel to output information as well as providing feedbacks, filling the research blank. The design is evaluated and validated through the simulated driving experiment after completed, at last, this study gives some suggestions for future designs.

2 Methods

2.1 Prototype Design

We exploited an application named “Eyes free” which could be installed in a tablet computer with Android system to simulate in-vehicle touchscreens. In this App, icons with the form of a large-area tile (similar with the windows 8’ start interface), was placed in the upper part of the touch screen because the upper part is the same height with driver’s glimpse. It is beneficial to the driver operation compared to the lower part. The functions like adjust volume, control air conditioning wind size are traditionally conducted through the “+”, “−” buttons. While they are now replaced by sliding gesture on any position of touch screen, which do not need to gaze to find a specific button and improve the fault tolerance rate (see Fig. 1). For functions such as return and exit, we adopt the method of long-time press (about 2 s) on any position of the screen to return or exit current function without clicking the specific return key. The meaning of gesture is to keep with people’s cognition and to make it convenient for memory. Sliding and long-time press gestures could be completed at any position of the screen with no eye focus, avoiding the driver to find and click on the button, reducing the visual resources utilization greatly.

Fig. 1.
figure 1

Sliding gesture interaction

Full size image

To reduce the occupation of visual resources, our design makes full use of auditory channel to convey information and provide feedback. There will be a voice broadcast to tell drivers what button he has chosen. In addition, there are different non-speech sounds to provide feedbacks after driver’s operation like adjusting volume, return and exit. The auditory feedbacks remind drivers the operation has been completed, enhancing the user experience.

Similar to the auditory channel, we also used haptic channel. We divided the vibration to two groups: position-based and gesture-based. The position-based vibration means the vibration motor will show different vibration intensity and frequency based on the position, in other words, based on the icons. The gesture-based vibration means vibrating differently depend on different gestures such as click, long-time press and sliding. The position-based vibration could convey the information what icon the driver has clicked, and the gesture-based vibration could provide feedbacks to tell the driver that the operation has been executed. In consideration of the in-vehicle touchscreen is not handheld device, based on our experience, it is vibration is hard to detect by our hands. Thus, we choose an external wearable vibrate motor to provide vibration. The vibrate motor communicates with touchscreens via XY-SWFS20 wi-fi module. The different vibration intensity and frequency are controlled by STC12C5A60S2 microcontroller with 9014 triodes and different resistances. In our design, we adopted three obviously distinguishable vibration intensities, defined as weak, medium and strong, to achieve position-based and gesture-based vibration. The schematic diagram is shown as Fig. 2.

Fig. 2.
figure 2

Vibration schematic diagram

Full size image

2.2 Simulation Driving Experiment Design

To investigate the design’s effectiveness, we conducted simulation driving experiments. Driving simulation systems based on human-computer interaction has the advantage of low cost, high safety, designable scene and repeatable experiment. We designed both highway and city driving simulated tracks using UC/win-road. For the highway, its speed limitation is 100 km/h with six-lane in both directions while for the city track, its speed limitation is 40 km/h with four-lane in both directions. The participants used a Logitech G27 steering and foot pedals to control their vehicle. The participants were required to drive without changing lanes and complete the secondary-tasks via touch screens during the driving. The secondary-task instructions like “Open the air conditioner”, “Play music”, “Turn the volume up” will appear on the top of screen in text meanwhile verbal form during the driving.

The whole experiment was divided into five trials. The first and the last trial were completed in non-driving condition. In the first trial, participants only operate the in-vehicle touchscreen using vision, no auditory and haptic channel. It is a blank experiment, which data of task-completion time could as a baseline. In the second trial, visual and auditory channel were provided to test the effect of auditory channel while in the third trial, visual and haptic channel were provided to test the effect of haptic channel. In the fourth trial, all the channels were provided to test the multimodal effectiveness. For the trial 2, 3, 4, a Latin square design was used to arrange the order to avoid the sequence effect. At last, participants would be blindfolded using an eyeshade to complete the last trial only using auditory and haptic channels. The fifth trial is to test whether multimodal design is effective in extreme cases, whether it can liberate the eyes of the driver, and realize the eyes free. During the driving, we used SmartEye AB, a camera-based eye tracker, to record the participant gaze fixation points and the duration of their gaze fixation in milliseconds (ms).

2.3 Procedure

Each participant completed five trials mentioned above, and the whole experiment lasted 1 h on average. With participant consent and pre-test questionnaire including general demographics and previous experience completed, participant seated in the simulator seat and adjusted to a comfortable position. After that, under the help of experimenter, the participant would be familiar with the operation of in-vehicle touchscreens (A tablet PC with the app “Eyes free”), feel the different intensity and frequency of vibration. Then, the experimenter demonstrated the functionality of the wheel and pedal set, then, allowed participants to drive a practice track till they could successfully control the vehicle. The last two minutes’ data will be used as a benchmark to evaluate their driving levels and behaviors. Before the formal experiment, participants were fitted with the eye tracker and gaze calibration was adjusted for each participant.

Participants were told the aim of the experiment is to complete the secondary-task under the premise of ensuring the safety of driving, if possible, to use vision as little as possible during the operation of the touch screen. In addition, the experimenter instructed participant to obey speed limits, avoid collision, and maintain vehicle in the specific lane without changing lanes when driving. The entire experiment process will be videotaped by the camera for research purpose only.

3 Results

We recruited 23 participants who have a valid driver’s license from campus, 14 males and 9 females with an average of 22 years old (SD = 4). All of them had more than one year’s driving experience, and have had using touch screen phones or tablets more than three years, nearly half of them using more than five years. Only one of the 23 participants was left-handed, and the rest were right-handed.

3.1 The Accuracy and Completion Time

The accuracy of secondary tasks was recorded by the experimenter in the process of the experiment. When complete the secondary tasks, drivers are allowed to make an error as long as they correct it in time. As long as they complete the task independence finally, it is deemed right. The correction process will be reflected on the completion time. The Raw data show that the accuracy of secondary tasks was more than 95% under different experimental conditions, indicating that the interactive design of the in-vehicle touch screen is easy to learn and easy to operate.

The completion time was analyzed from video after the experiment, and the time was accurate to milliseconds. We only compared the completion time of trial one and trial five, because the secondary task instructions in these two trials were exactly the same. The main difference between the two trials was whether to use vision. On average, in trial five, participants used 1.112 s for each task (SD = 0.040 s), which is a little longer than trial one (M = 1.048 s, SD = 0.033 s), however, the t-test result did not show a significant difference between these two trials (p = 0.367). Also, there was no significant difference between these two trails (p = .819) in terms of accuracy.

3.2 Lane Deviation

To better assess driver distraction, we measured participants’ lane deviation. Lane deviation was acquired from UC/win-road driving logs. As prescribed by the experimenter, drivers could not change lanes, so the lane deviation value should be between [−1, 1]. The value which is greater than 1 because of the unexpected situation will be eliminated. Because each driver’s driving habits were different also the vehicle starting position was different, we calculated the root mean squared error (RMSE) of lane deviation instead of raw data to measure stability of the driver’s lateral control. A repeated measures ANOVA was conducted to analyze the effects of Trial (2, 3, 4) on lane deviations. Although, the average in trial 4 is smaller than trial 2 and 3, there was no significant main effect of trial on lane deviations.

3.3 Glance Time off Road

Due to limitation of the eye tracking software, gaze data could not be reliably collected from participants wearing eyeglasses. Therefore, the glance time off road was composed of 15 participants. Trial 4 shown much lower glance time off road than trial 1. A paired-samples t-test suggested a significant difference (p = .000) in glance time off road between trial 4 (M = 0.397 s, SD = 0.021) and trial 1 (M = 0.934 s, SD = 0.102).

3.4 Perception Questionnaire

Participants completed an online questionnaire at the end of the experiment. The responses (N = 23) were summarized in Table 1 and covered four categories: satisfaction with page design (icon size, spacing, aesthetic), satisfaction with interactive design (sliding gestures, long-time press), satisfaction with audio channel, satisfaction with haptic channel.

Table 1. Data of satisfaction. (Five-point Likert Scale, 1 = Most negative, 5 = Most positive)
Full size table

4 Discussion

In general, the data of accuracy and completion time between trial 1 and trial 5 shown that the design of multimodal in-vehicle touchscreen could achieve “eyes free”. Even if do not use vision, with our multimodal design, the accuracy will not decrease, meanwhile, the completion time will not change significantly. The average glance time off road of each task in trial 4 was 0.397 s, which is lower than the recommended maximum time (1.5 s) spent looking at an input device [12], thus, there will have no significant impact on traffic safety. Furthermore, data collected from the post-test questionnaire gave us participants’ opinions of each channel. Most of participants were satisfied with the multimodal design expect the haptic channel was rated a little lower. At same time, the RMSE of trial 3 (Haptic channel-only) was higher than trial 2 and 4. We suspect that it will occupy cognitive resource when decode the information conveyed by vibration, consequently, affecting the driving performance [13].

Due to the limitation of eye tracker, we eliminated 8 participant’s data. This may have had an impact on our glance time off road. However, we were still able to analyze gaze data from over 65% of the participants. In addition, the number of participants was small and they were mainly youth. Maybe in the future we will do more experiments to cover more people. Although driving simulators are routinely used as a research tool in traffic psychology and its validity has been accepted, there is a great difference between simulation and reality such as no risk and low user experience.

By evaluating our design with simulation driving experiment, we could conclude that, completing secondary task with multimodal touch screens could not have an influence on driving safety, meanwhile, improve the user experience. Based on our results, we propose two recommendations for the in-vehicle touchscreen design:

  • Sliding gestures should be adopted which could improve the fault tolerance rate and decrease distraction comparing to finding a specific icon;

  • Provide audio and haptic feedback to tell the driver what the icon he has clicked and whether the operation has been executed, which could improve the user experience.