Keywords

1 Human-Machine Shared Control Context

1.1 From Driver-Led to Human-Machine Shared Control

Smart cars will not be able to achieve fully autonomous driving in the foreseeable future, so humans are still participants in car driving, and this new driving context is called human-machine shared control. The traditional car driving context is centered on people, especially the driver, where the "center" refers to both the center of the user experience and the center of vehicle control, and the driver is the main initiator of driving behavior and interaction. But in the intelligent car driving context, the driver and the vehicle intelligent system are in a “co-driving” state, and the human and intelligent system work together to complete the driving task.

The task allocation in the intelligent vehicle Human-machine Shared Control Context differs according to the different driving controllers, so here we discuss the task allocation in the Human-machine Shared Control Context with the driver as the driving controller and the in-vehicle intelligent system as the driving controller.

The driver-intelligent system is usually the driver in an ideal road situation, where the car obtains real-time information about the surrounding environment based on pre-programmed algorithms and sensors, and intelligently handles the overall task using the "experience" learned as artificial intelligence. In this case, the driver gradually changes from a user to a supervisor. The driver is the car controller mainly for non-ordinary driving contexts, where the in-vehicle intelligent system uses the above-mentioned technology to guide the driver in driving. This task distribution is similar to that of a driver and a navigator in a car rally. The in-vehicle intelligent system provides information to the driver through real-time monitoring of the environment and the vehicle, and assists the driver in completing the driving task.

1.2 Situational Awareness and Haptic Feedback

According to Walker G.H. et al., people perform more than 1600 tasks in the car,[1] and in the human-vehicle context, people are freed from the main driving task to perform other behaviors, which makes the context of smart cars tend to be diverse and complex. The complexity of the context dictates that the human in the car needs to understand the changes in the context, that is situational awareness. [2] Situational awareness refers to the user's awareness of what is dynamically changing in the environment, and it explains how the user manages the relationship between long-term goals of driving (e.g., reaching a destination) and short-term goals (e.g., accident avoidance, entertainment communication, etc.).

Haptic feedback, as an effective physical information transfer pathway in the smart cockpit, can play an important role in facilitating a shift in the driver's situational awareness. First, the Human-machine Shared Control Context in smart cars requires the driver and the car's intelligent system to be involved simultaneously and to switch the driving state and the dominant player according to the actual situation. Therefore, appropriate haptic feedback can alert the driver during the process of switching between driving state and dominant, trigger the driver's situational awareness, and avoid traffic accidents caused by negligence. Secondly, haptic feedback is more efficient than visual and auditory feedback in conveying simple information. The intelligent car cockpit can replace the less efficient visual and auditory feedback with haptic feedback to improve the efficiency of the driver's processing of changes in the driving situation. Finally, haptic feedback can be combined with other sensory information to form a multimodal and integrated intelligent driving experience.

2 Haptic Feedback in Smart Cars

Various tactile receptors and nerve endings distributed on the human skin transmit external information to the central nervous system to obtain tactile perception. Compared with other senses, haptics is a direct, zero-distance physical contact between the cognizer and the object. Haptics allows the cognizer to recognize the surface features, spatial characteristics, material properties, and other characteristics of an object. And in the process of recognition, tactile sensation can spawn emotional tactile sensations such as surprise, fear, and pleasure. Thus, haptics has both material and mental properties. Accordingly, the current research on tactile sensation is divided into two aspects: the direct sensation based on the actual material and the actual structure of the material surface, and the psychological sensation based on the imagery of the material surface organization. Usually, the two complement each other in haptic experience, and the study of the two is not isolated.[3].

Tactile perception consists of multimodal information about the shape, size, texture, hardness, roughness, material, temperature, weight, and force of an object, and can be divided into the sensation of touch felt by the skin and the sensation of force felt by the joint ligaments. The complete haptic experience is presented by the mutual integration of touch and force sensation. The simulation of haptic feedback is also achieved by computer simulation of haptic feedback and force feedback. Among them, the tactile feedback is through the vibration, texture and temperature of the object surface to identify the object of touch. Force feedback, on the other hand, discriminates the object of contact by factors such as weight, hardness, and friction characteristics.[4].

In smart cars, the environment and driving conditions around the car can be sensed through a wealth of sensors, and haptic feedback can be provided to the driver through interactive smart devices within the smart cockpit. According to the type of haptic feedback can be divided into force feedback, tactile feedback and pseudo-haptic feedback.

2.1 Force Feedback in Smart Cars

Force feedback is widely used in current automobiles, such as steering wheel steering, throttle and brake down resistance, etc. There are also many studies trying to optimize the driving safety and driving experience of smart cars through force feedback. Bosch developed in '16 to add haptic feedback to a prototype gas pedal to communicate with the driver during acceleration. This active pedal system uses an integrated motor set in the pedal to provide feedback. The system generates resistance and vibrations at specific pedal depths to remind the driver to save fuel and be safe, depending on the driving situation. In driving, if the driver presses the gas pedal too deep, then the system will increase the pedal resistance appropriately to remind the driver to pay attention to the fuel consumption, through the experiment shows that this system can reduce the fuel consumption by about 7%. When the vehicle is in a dangerous situation, such as cornering too fast, the pedal will vibrate to remind the driver to improve driving behavior. In addition, the system can be personalized according to the driver's preference to meet the needs of different users. In addition, Bosch has partnered with Ultra Haptics in Bristol, UK, to create a prototype motion gesture function to provide haptic feedback. As the hand floats above the console, the driver makes gestures and ultrasonic waves hit the user's hand and provide feedback, making it feel as if the user is actually touching the physical knob. Similarly, BMW's Holo Active haptic interface uses a haptic gesture control system (sensors "see" the hand, and through the use of ultrasonic technology, the driver can "feel" the controls), so the driver doesn't have to touch the screen. Simply move in the air. Such haptic feedback and interaction not only improves the efficiency of human-vehicle interaction, but also improves the driver's dangerous driving behavior.

2.2 Tactile Feedback in Smart Cars

Haptic perception consists of multimodal information such as the shape, size, texture, hardness, roughness, material, temperature, weight, and force of an object, which can be divided into touch sensation felt by skin and force sensation felt by joint ligaments. [5] The complete haptic experience is presented by the mutual integration of touch and force sensation. The simulation of haptic feedback is also achieved by computer simulation of haptic feedback and force feedback. Among them, tactile feedback is to identify the object to be touched by the vibration, texture and temperature of the object surface.

If force feedback is the in-vehicle intelligent system to provide assistance to the driver's driving behavior according to the driving situation, then tactile feedback is to give the driver a rich experience through different devices in the intelligent cockpit. In particular, the development of intelligent interactive surfaces has brought new possibilities for the haptic experience of smart cars. Smart interactive surfaces break the boundaries between interior decorative surfaces and touch panels, embedding electronic components and smart sensors in textile materials to achieve interactive functions. For example, with a combination of decorative lighting elements, capacitive switching technology and haptic actuation integrated with force sensors embedded under the textile material, users can interact with touch seat gestures to switch on and off seat memory and seat heating. Most recently, BMW also presented an intelligent surface for interior and exterior surfaces, called "Shy Tech". This smart surface contains cameras, radar and many sensors, has digital functions and is self-healing. This textile-mediated smart interactive surface can be used for automotive seat covers, seat belts, roof, door panels and dashboard components. The smart interactive surface combines traditional surface materials with smart components and information technology to provide the driver with a rich tactile experience through technology while retaining the traditional material tactile experience.

3 Multimodal Pseudo-haptic Experiences

Multimodality refers to the integration of multiple senses. Multimodal interaction is the fusion of multiple senses such as human vision, hearing, and touch, and the computer responds to input using multiple communication channels and simulates human interaction. Now it is widely used in the interaction design of various intelligent products. Humans input information through voice, gestures, expressions and other modalities, and the computer responds and gives feedback through computer vision and auditory channels, which is a typical multimodal interaction. Currently, multimodal interaction and related technologies have been commonly used in the interaction design of intelligent vehicles, and multimodal design related to haptic interaction and haptic feedback has also attracted much attention. Research on multimodal haptic feedback is mainly focused on two areas. The first is the combination of existing haptic feedback and other sensory modalities to form a comprehensive haptic feedback. The second is the combination of multimodal sensory feedback to form pseudo-haptic feedback.

3.1 Pseudo-haptic Feedback

With the advancement of interaction technology and the enrichment of interaction methods, more and more attention has been paid to the study of pseudo-haptic feedback. The so-called pseudo-haptic feedback refers to the haptic experience that is different from the traditional physical haptic feedback simulation and does not even require contact with physical objects.[6] It is a technique for presenting tactile perception through cross-sensory information, which is an aid and compensation for the formation of tactile sensation through the experience of other senses using the principle of fluency. The purpose of pseudo-haptic feedback is to generate haptic illusions, i.e., haptic perceptions under the influence of other senses. Pseudo-haptic feedback based on multisensory illusions has been used to simulate various haptic properties, such as the stiffness of a virtual spring, the texture or quality of an object. In the current situation where smart products are generally equipped with touch displays and speakers and other devices are constantly improved and upgraded, pseudo-haptic feedback formed by visual, auditory and other perceptions has a broad application prospect.

3.2 Pseudo-haptic Feedback Formed by Different Senses

First, in the formation of pseudo-haptic feedback, the visual senses can play a significant role. This is because there is a large degree of overlap between tactile and visual senses in terms of the types of information represented and the neural basis on which they rely.[7] It can be said that humans have innate characteristics of visual haptic perception.[8] In the intelligent design of haptic experiences, the construction of visual for haptic information can be done either through sensory substitution, converting visual information into haptic information, or through pseudo-haptic feedback to simulate haptic information as a technical complement to real haptic feedback. At present, it has been possible to form different pseudo-haptic feedback through vision, such as pseudo-haptic simulation by adjusting the speed of visual stimuli to characterize friction, and pseudo-haptic simulation by distorting and deforming video images to characterize wind resistance.

Second, hearing also plays an important role in the formation of pseudo-haptic feedback. The conversion of a specific modality to haptics in pseudo-haptic feedback requires that this sensory modality has the type of data specified by the biophysiological characteristics of haptics. For both hearing and touch, the vibration and sound information generated by touching an object are wave signals, which dictate that hearing and touch share physical properties such as vibration frequency and amplitude, which enables a mapping relationship between the two in terms of properties such as intensity, frequency, velocity, and roughness. [9] Therefore, in the case where the tactile sensation cannot reach a satisfactory state, the auditory modality can be used in an intelligent way to provide tactile illusion.

3.3 Pseudo-haptic Feedback for Smart Cars

Haptic feedback in smart cars never exists in isolation, especially pseudo-haptic feedback, a feedback system that can be formed through other senses. For example, in the cockpit of a smart car, different driving contexts can be simulated through the in-car display, intelligent interactive surfaces, ambient lights, interior aromatherapy and audio devices. When the display and ambience present images and colors related to the ocean and the beach, and the audio device simulates the sound of the sea breeze, it can make the driver feel as if he or she is on the seaside highway, at which time, although the driver is still in the closed cockpit, the pseudo-haptic feedback experience of the sea breeze can be formed through the multi-sensory simulation. At present, there are relatively few pseudo-haptic feedback designs actually used in smart cars, and the related research and technology are in their infancy. However, under the context of human-machine shared control in smart cars, the identity of human gradually changes from driver to supervisor, and the focus of interaction design of smart cars also starts to transition from function to user experience, and the experience and entertainment value of smart cars increases.

4 Conclusion

Different brands of smart cars are constantly appearing in the market, and artificialintelligence and different sensing technologies make smart cars more and more feature-rich. However, it is undeniable that the traditional haptic feedback experience is gradually failing, and the haptic feedback interaction system adapted to new technologies and new driving contexts has not yet been fully formed. Therefore, this paper distinguishes different forms of haptic feedback, describes the three main types of haptic feedback: force feedback, tactile feedback and pseudo-haptic feedback, and discusses the use of different forms of haptic feedback in smart cars in conjunction with smart car Human-machine Shared Control Context s, and proposes design recommendations to further improve the design quality from the perspective of safety and experience.