Abstract
Since 2007, when the world's first self-driving car was tested on real roads, smart cars have gradually become the main direction of automotive development. Nowadays, with the development of smart car technology, the car driving mode has seen a huge change, and haptic feedback research of human-computer interaction in human-machine shared control and autonomous driving have gradually become the main contexts of future smart cars. The International Society of Automotive Engineering (ISAE) has divided different autonomous vehicle behaviors into six levels from L0 to L5. In the automation level L0-L3 smart cars, people are indispensable in the whole driving process, while the automation level L4-L5 smart cars do not need people to participate in driving tasks at all. However, due to various factors such as automotive technology, laws and regulations, and morality and ethics, humans will remain the participants and dominators of car driving in the foreseeable future. At the same time, the new iterations of smart car in-vehicle devices are leading to the gradual failure of classical car interaction feedback. Among these interactive feedback, haptic is a real physical feedback, which is an important way for humans to receive physical information about objective things. Haptic feedback allows the driver to obtain timely information about the proximal environment and vehicle movement, which can directly affect the driver's next driving behavior. In addition, haptic feedback also has the ability to carry emotion. The haptic feedback of different interaction objects will bring different experiences to the driver.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Keywords
1 Human-Machine Shared Control Context
1.1 From Driver-Led to Human-Machine Shared Control
Smart cars will not be able to achieve fully autonomous driving in the foreseeable future, so humans are still participants in car driving, and this new driving context is called human-machine shared control. The traditional car driving context is centered on people, especially the driver, where the "center" refers to both the center of the user experience and the center of vehicle control, and the driver is the main initiator of driving behavior and interaction. But in the intelligent car driving context, the driver and the vehicle intelligent system are in a “co-driving” state, and the human and intelligent system work together to complete the driving task.
The task allocation in the intelligent vehicle Human-machine Shared Control Context differs according to the different driving controllers, so here we discuss the task allocation in the Human-machine Shared Control Context with the driver as the driving controller and the in-vehicle intelligent system as the driving controller.
The driver-intelligent system is usually the driver in an ideal road situation, where the car obtains real-time information about the surrounding environment based on pre-programmed algorithms and sensors, and intelligently handles the overall task using the "experience" learned as artificial intelligence. In this case, the driver gradually changes from a user to a supervisor. The driver is the car controller mainly for non-ordinary driving contexts, where the in-vehicle intelligent system uses the above-mentioned technology to guide the driver in driving. This task distribution is similar to that of a driver and a navigator in a car rally. The in-vehicle intelligent system provides information to the driver through real-time monitoring of the environment and the vehicle, and assists the driver in completing the driving task.
1.2 Situational Awareness and Haptic Feedback
According to Walker G.H. et al., people perform more than 1600 tasks in the car,[1] and in the human-vehicle context, people are freed from the main driving task to perform other behaviors, which makes the context of smart cars tend to be diverse and complex. The complexity of the context dictates that the human in the car needs to understand the changes in the context, that is situational awareness. [2] Situational awareness refers to the user's awareness of what is dynamically changing in the environment, and it explains how the user manages the relationship between long-term goals of driving (e.g., reaching a destination) and short-term goals (e.g., accident avoidance, entertainment communication, etc.).
Haptic feedback, as an effective physical information transfer pathway in the smart cockpit, can play an important role in facilitating a shift in the driver's situational awareness. First, the Human-machine Shared Control Context in smart cars requires the driver and the car's intelligent system to be involved simultaneously and to switch the driving state and the dominant player according to the actual situation. Therefore, appropriate haptic feedback can alert the driver during the process of switching between driving state and dominant, trigger the driver's situational awareness, and avoid traffic accidents caused by negligence. Secondly, haptic feedback is more efficient than visual and auditory feedback in conveying simple information. The intelligent car cockpit can replace the less efficient visual and auditory feedback with haptic feedback to improve the efficiency of the driver's processing of changes in the driving situation. Finally, haptic feedback can be combined with other sensory information to form a multimodal and integrated intelligent driving experience.
2 Haptic Feedback in Smart Cars
Various tactile receptors and nerve endings distributed on the human skin transmit external information to the central nervous system to obtain tactile perception. Compared with other senses, haptics is a direct, zero-distance physical contact between the cognizer and the object. Haptics allows the cognizer to recognize the surface features, spatial characteristics, material properties, and other characteristics of an object. And in the process of recognition, tactile sensation can spawn emotional tactile sensations such as surprise, fear, and pleasure. Thus, haptics has both material and mental properties. Accordingly, the current research on tactile sensation is divided into two aspects: the direct sensation based on the actual material and the actual structure of the material surface, and the psychological sensation based on the imagery of the material surface organization. Usually, the two complement each other in haptic experience, and the study of the two is not isolated.[3].
Tactile perception consists of multimodal information about the shape, size, texture, hardness, roughness, material, temperature, weight, and force of an object, and can be divided into the sensation of touch felt by the skin and the sensation of force felt by the joint ligaments. The complete haptic experience is presented by the mutual integration of touch and force sensation. The simulation of haptic feedback is also achieved by computer simulation of haptic feedback and force feedback. Among them, the tactile feedback is through the vibration, texture and temperature of the object surface to identify the object of touch. Force feedback, on the other hand, discriminates the object of contact by factors such as weight, hardness, and friction characteristics.[4].
In smart cars, the environment and driving conditions around the car can be sensed through a wealth of sensors, and haptic feedback can be provided to the driver through interactive smart devices within the smart cockpit. According to the type of haptic feedback can be divided into force feedback, tactile feedback and pseudo-haptic feedback.
2.1 Force Feedback in Smart Cars
Force feedback is widely used in current automobiles, such as steering wheel steering, throttle and brake down resistance, etc. There are also many studies trying to optimize the driving safety and driving experience of smart cars through force feedback. Bosch developed in '16 to add haptic feedback to a prototype gas pedal to communicate with the driver during acceleration. This active pedal system uses an integrated motor set in the pedal to provide feedback. The system generates resistance and vibrations at specific pedal depths to remind the driver to save fuel and be safe, depending on the driving situation. In driving, if the driver presses the gas pedal too deep, then the system will increase the pedal resistance appropriately to remind the driver to pay attention to the fuel consumption, through the experiment shows that this system can reduce the fuel consumption by about 7%. When the vehicle is in a dangerous situation, such as cornering too fast, the pedal will vibrate to remind the driver to improve driving behavior. In addition, the system can be personalized according to the driver's preference to meet the needs of different users. In addition, Bosch has partnered with Ultra Haptics in Bristol, UK, to create a prototype motion gesture function to provide haptic feedback. As the hand floats above the console, the driver makes gestures and ultrasonic waves hit the user's hand and provide feedback, making it feel as if the user is actually touching the physical knob. Similarly, BMW's Holo Active haptic interface uses a haptic gesture control system (sensors "see" the hand, and through the use of ultrasonic technology, the driver can "feel" the controls), so the driver doesn't have to touch the screen. Simply move in the air. Such haptic feedback and interaction not only improves the efficiency of human-vehicle interaction, but also improves the driver's dangerous driving behavior.
2.2 Tactile Feedback in Smart Cars
Haptic perception consists of multimodal information such as the shape, size, texture, hardness, roughness, material, temperature, weight, and force of an object, which can be divided into touch sensation felt by skin and force sensation felt by joint ligaments. [5] The complete haptic experience is presented by the mutual integration of touch and force sensation. The simulation of haptic feedback is also achieved by computer simulation of haptic feedback and force feedback. Among them, tactile feedback is to identify the object to be touched by the vibration, texture and temperature of the object surface.
If force feedback is the in-vehicle intelligent system to provide assistance to the driver's driving behavior according to the driving situation, then tactile feedback is to give the driver a rich experience through different devices in the intelligent cockpit. In particular, the development of intelligent interactive surfaces has brought new possibilities for the haptic experience of smart cars. Smart interactive surfaces break the boundaries between interior decorative surfaces and touch panels, embedding electronic components and smart sensors in textile materials to achieve interactive functions. For example, with a combination of decorative lighting elements, capacitive switching technology and haptic actuation integrated with force sensors embedded under the textile material, users can interact with touch seat gestures to switch on and off seat memory and seat heating. Most recently, BMW also presented an intelligent surface for interior and exterior surfaces, called "Shy Tech". This smart surface contains cameras, radar and many sensors, has digital functions and is self-healing. This textile-mediated smart interactive surface can be used for automotive seat covers, seat belts, roof, door panels and dashboard components. The smart interactive surface combines traditional surface materials with smart components and information technology to provide the driver with a rich tactile experience through technology while retaining the traditional material tactile experience.
3 Multimodal Pseudo-haptic Experiences
Multimodality refers to the integration of multiple senses. Multimodal interaction is the fusion of multiple senses such as human vision, hearing, and touch, and the computer responds to input using multiple communication channels and simulates human interaction. Now it is widely used in the interaction design of various intelligent products. Humans input information through voice, gestures, expressions and other modalities, and the computer responds and gives feedback through computer vision and auditory channels, which is a typical multimodal interaction. Currently, multimodal interaction and related technologies have been commonly used in the interaction design of intelligent vehicles, and multimodal design related to haptic interaction and haptic feedback has also attracted much attention. Research on multimodal haptic feedback is mainly focused on two areas. The first is the combination of existing haptic feedback and other sensory modalities to form a comprehensive haptic feedback. The second is the combination of multimodal sensory feedback to form pseudo-haptic feedback.
3.1 Pseudo-haptic Feedback
With the advancement of interaction technology and the enrichment of interaction methods, more and more attention has been paid to the study of pseudo-haptic feedback. The so-called pseudo-haptic feedback refers to the haptic experience that is different from the traditional physical haptic feedback simulation and does not even require contact with physical objects.[6] It is a technique for presenting tactile perception through cross-sensory information, which is an aid and compensation for the formation of tactile sensation through the experience of other senses using the principle of fluency. The purpose of pseudo-haptic feedback is to generate haptic illusions, i.e., haptic perceptions under the influence of other senses. Pseudo-haptic feedback based on multisensory illusions has been used to simulate various haptic properties, such as the stiffness of a virtual spring, the texture or quality of an object. In the current situation where smart products are generally equipped with touch displays and speakers and other devices are constantly improved and upgraded, pseudo-haptic feedback formed by visual, auditory and other perceptions has a broad application prospect.
3.2 Pseudo-haptic Feedback Formed by Different Senses
First, in the formation of pseudo-haptic feedback, the visual senses can play a significant role. This is because there is a large degree of overlap between tactile and visual senses in terms of the types of information represented and the neural basis on which they rely.[7] It can be said that humans have innate characteristics of visual haptic perception.[8] In the intelligent design of haptic experiences, the construction of visual for haptic information can be done either through sensory substitution, converting visual information into haptic information, or through pseudo-haptic feedback to simulate haptic information as a technical complement to real haptic feedback. At present, it has been possible to form different pseudo-haptic feedback through vision, such as pseudo-haptic simulation by adjusting the speed of visual stimuli to characterize friction, and pseudo-haptic simulation by distorting and deforming video images to characterize wind resistance.
Second, hearing also plays an important role in the formation of pseudo-haptic feedback. The conversion of a specific modality to haptics in pseudo-haptic feedback requires that this sensory modality has the type of data specified by the biophysiological characteristics of haptics. For both hearing and touch, the vibration and sound information generated by touching an object are wave signals, which dictate that hearing and touch share physical properties such as vibration frequency and amplitude, which enables a mapping relationship between the two in terms of properties such as intensity, frequency, velocity, and roughness. [9] Therefore, in the case where the tactile sensation cannot reach a satisfactory state, the auditory modality can be used in an intelligent way to provide tactile illusion.
3.3 Pseudo-haptic Feedback for Smart Cars
Haptic feedback in smart cars never exists in isolation, especially pseudo-haptic feedback, a feedback system that can be formed through other senses. For example, in the cockpit of a smart car, different driving contexts can be simulated through the in-car display, intelligent interactive surfaces, ambient lights, interior aromatherapy and audio devices. When the display and ambience present images and colors related to the ocean and the beach, and the audio device simulates the sound of the sea breeze, it can make the driver feel as if he or she is on the seaside highway, at which time, although the driver is still in the closed cockpit, the pseudo-haptic feedback experience of the sea breeze can be formed through the multi-sensory simulation. At present, there are relatively few pseudo-haptic feedback designs actually used in smart cars, and the related research and technology are in their infancy. However, under the context of human-machine shared control in smart cars, the identity of human gradually changes from driver to supervisor, and the focus of interaction design of smart cars also starts to transition from function to user experience, and the experience and entertainment value of smart cars increases.
4 Conclusion
Different brands of smart cars are constantly appearing in the market, and artificialintelligence and different sensing technologies make smart cars more and more feature-rich. However, it is undeniable that the traditional haptic feedback experience is gradually failing, and the haptic feedback interaction system adapted to new technologies and new driving contexts has not yet been fully formed. Therefore, this paper distinguishes different forms of haptic feedback, describes the three main types of haptic feedback: force feedback, tactile feedback and pseudo-haptic feedback, and discusses the use of different forms of haptic feedback in smart cars in conjunction with smart car Human-machine Shared Control Context s, and proposes design recommendations to further improve the design quality from the perspective of safety and experience.
References
Walker, G.H.F., Stanton, N.A.S., Young, M.S.T.: Hierarchical task analysis of driving: a new research tool. Contempor. Ergon. 435, 40 (2001)
Hao, T., Zhang, F., Yingli, S.: Safety-oriented intelligent vehicles information & interaction design research. Zhuangshi 352(08), 22–27 (2022)
Ma, X., Tong, Q., Li, Y.: Research on the construction of intelligent design haptic experience based on multimodal haptic and pseudo-haptic Feedback. Zhuangshi 353(09), 28–33 (2022)
Qi, B., Hu, F., Zhu, F.: Tactile interaction service framework for blind readers and application research. Libr. Inf. Work. 14, 20–29 (2019)
Wang, D., Ohnishi, K., Xu, W.: Multimodal Haptic Display for Virtual Reality: A Survey. IEEE Transactions on Industrial Electronics 67(1), 610–623 (2020). https://doi.org/10.1109/TIE.2019.2920602
Lécuyer A,F., Coquillart, S., Kheddar, A.T.: Pseudo-haptic Feedback: can isometric input devices simulate force Feedback?. Proceedings of the IEEE International Conference on Virtual Reality (2000)
Lécuyer, A.: Simulating haptic feedback using vision: a survey of research and applications of pseudo-haptic Feedback. Presence Teleoper. Virtual Environ. 18(1), 39–53 (2009). https://doi.org/10.1162/pres.18.1.39
Yu, W., Liu, Y., Fu, X.: Cognitive mechanisms of haptic two-dimensional image recognition. Adv. Psychol. Sci. 4, 611–622 (2019)
Yang, Z., Li, H., Ma, S., Shu, T.: Applying "Audible" tactile experiences: cues from multi-sensory channel integration. Adv. Psychol. Sci. 3, 580–590 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xiao, J. (2023). Haptic Feedback Research of Human-Computer Interaction in Human-Machine Shared Control Context of Smart Cars. In: Stephanidis, C., Antona, M., Ntoa, S., Salvendy, G. (eds) HCI International 2023 Posters. HCII 2023. Communications in Computer and Information Science, vol 1836. Springer, Cham. https://doi.org/10.1007/978-3-031-36004-6_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-36004-6_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36003-9
Online ISBN: 978-3-031-36004-6
eBook Packages: Computer ScienceComputer Science (R0)