Introduction

The vast amount of people’s fears about robots as a device, social phenomenon, or industrial development phase take many forms. These forms of fears are associated with robots themselves, stemming from concerns about their capabilities, impacts, and the potential consequences of their integration into society (Porpora, D. 2021). These fears reflect the uncertainty and unknown outcomes associated with rapidly advancing robotic technologies and their increasing presence in daily life. Subsequently, robots’ technical and aesthetic aspects have a marginal influence on robophobia (Davey, 1997). A particular fear of losing a job from the robotization of the industry mainly comes from the difference between humans and robots in the field of emotion and intellectual activity (Porpora, 2021). Humans have doubts and uncertainties regarding the artificial intellect means used by robots. Even though it may sound paradoxical, acceptance of robot decisions results in better mental achievements in humans (Hayashi & Wakabayashi, 2018). When a robot’s decisions look reasonable, the public accepts them better because people’s social culture and habits are very distinct and significantly impact the robot’s acceptance (Maccarini, 2021). Humans can train their behavior and acceptance of robots. Therefore, in areas such as nursing and medicine, a methodology rounding the corners about extreme human emotional reactions to robots is necessary (Archer, 2021), as well as a special robot control methodology preventing humans from losing social competencies due to long-lasting human–robot relations (Kempt, 2022).

The “pragmacentric”—practical behavior-based approach to robot action acceptance develops higher robot decision acceptance quality concerning human opinion (Kempt, 2020). In circumstances where robot decisions and actions cause serious outcomes, like medical injections or nursing actions, the human reaction becomes tenser and, depending on human experience, knowledge, feelings, and emotional state, varies from strongly positive to negative (Bhattacharya, 2021). Thus, it is necessary to focus on developing comprehensive social, educational, and technological solutions for human–robot interaction (HRI) to minimize unreasonable fears.

Conversely, robophobia lies in the vast area between standard human behavior concerning unknown objects or their actions. Typically, humans are conservative toward new and unknown processes or objects. Human reaction to robots differs from many factors, including geographical location. For instance, society’s agreement to use robots in Greece (57%) is much lower than in Denmark (95%) (Hofstede Insights, 2023).

Research on human–robot interaction is often considered to be closely related to human–computer interaction. Still, in contrast to the general computer science and human interfaces to it, the concept of robotics involves many technical, psychological, and even social aspects besides electronics, electrotechnics, software, and artificial intelligence (AI), which generates effects of robophobia as well. The recent appearance of social robotics as a separate product development field has sparked interest and opened new niches of investigation, bringing important questions about HRI to a new light.

This research aims to highlight the progress of human–robot interaction and qualify levels of such cooperation and their limits. The paper proposes a system of HRI evaluation and reveals the complexity of the area. Nevertheless, a systematic approach is delivered from the engineering point of view.

In our review, we hypothesize:

  1. 1.

    Human-robot cooperation and collaboration can be effective in industry and possibly in other partially predefined environments;

  2. 2.

    Communication and messaging in HRI are at slow progress and lack of systematic approach;

  3. 3.

    HRI reveals new socio-psychologic phenomena;

  4. 4.

    The general acceptance of robots over the entire population is mixed and underexplored;

  5. 5.

    Emotional communication is very effective between humans, but better understanding and classification are necessary for improved human-robot interaction.

Materials and methods

This overview of research conducted in the area of human–robot interaction provides a multi-criteria analysis covering research questions related to available hardware and software limitations, methodological issues, and humans’ social and psychological reactions to the robots. Analysis was conducted using 106 scientific research papers selected from Google Scholar, ScienceDirect, and IEEExplore databases during three stage inclusion process. In the first stage, more than 560 publications from the last five years (except a few older ones containing fundamental statements) were selected according to the title and keywords. The following keywords were used to filter the articles: Human–robot interaction, Human emotions definition, Instrumental emotions detection methods, Human safety, Psychological comfort, Robophobia, Human motion detection/prediction, Human–machine communication, and Human reaction to robot/machine. In the second stage, after the screening, almost 320 papers were excluded from the analysis as not suitable due to the out of scope research problems, lack of validation, and low quality. In the third stage, after removing the repetitive records, 116 papers were classified into four categories (human–robot collaboration—60 papers; human–robot communication—17; human emotions and physical state evaluation in HRI—11; human perception of robots—28) and analyzed in detail. The main criteria for including the paper were: clear formulation of the research problem and proposed solution, the applicability of the results in the HRI area, and the reliability of provided results.

Review outcomes

Performed analysis delivers a vast amount of data; therefore, the outcomes of our research beg for structured presentation. In our opinion, these findings are split into four fields: human–robot collaboration, human–robot communication, human emotion, and physical state evaluation as input for robot controls and human perception of robots. This classification covers both technical and psychological issues.

Human–robot collaboration

Human–robot collaboration covers various research areas, many of which are well-explored in the scientific literature. In the majority of analyzed references, two typical approaches can be noticed:

  • Study of the possibilities of robot-human collaboration in specific fields such as medicine, agriculture, machine manufacturing, shipbuilding, waste management, etc.;

  • Analysis of the characteristics of robot-human collaboration directly unrelated to a particular specific field. For example, it is common to think that a robot can perform simple repetitive actions. Still, applying robots in areas such as shipbuilding is challenging due to many unique designs and technical solutions (Zacharaki et al., 2022).

The main way for evaluating the intensity of human–robot collaboration is done with a universally recognizable classification into five levels: (i) no collaboration; (ii) coexistence; (iii) synchronization; (iv) cooperation; (v) collaboration (Dzedzickis et al., 2021). The issue of human–robot collaboration quality regarding human understanding or psychological status stays outside the collaboration quality evaluation; the paper focuses on the technical solutions in various aspects of HRI. Human behavior evaluation belongs to the HRI realization, which is covered in many types of research.

The classification presented in Fig. 1 evaluates the possibilities of sharing workspaces, work objects, and tasks. The lowest HRI level is no collaboration—the robot remains inside a closed work cell, and workspace sharing is strictly forbidden. The second level is coexistence—a case when closed cells are removed, but workspaces between humans and robots remain strictly separated. The third level is synchronization, where robots and humans can share part of the workspace and work objects, but never simultaneously. The fourth level is cooperation—shared tasks and workspace are acceptable, but physical interaction is forbidden. Collaboration is the highest level of interaction when physical interaction and common operations between humans and robots are allowed.

Fig. 1
figure 1

Levels of human–robot collaboration intensity (Matheson et al., 2019)

Full size image

Various types and levels of human–robot interaction in manufacturing were examined to develop a typical robot-human interaction methodology according to established conditions (Malik & Bilberg, 2019). The authors provide a synthesis of a human–robot collaboration architecture based on three aspects: team composition, engagement level, and safety, allowing them to describe collaboration using a 3-dimensional reference scale. A review presented by (Li et al., 2023) provides a detailed analysis of safety standards and methods ensuring human safety in HRI.

Despite different attempts to classify the intensity of human–robot collaboration, it remains one of the fundamental factors affecting the required features of HRI. Collaboration intensity typically correlates with the technological development level of HRI; the higher the collaboration level—the more advanced HRI is required.

General issues in the field of human–robot interaction

The research on HRI for common operations between robots and humans faces many scientific uncertainties. From the research reports provided in the last five years, we defined that the main research interests are general HRI issues, the possibility of adapting robots to individual humans, and the robotization of specific industries or research areas. Extensive literature analysis provided by (Faccio et al., 2023) revealed the five most important human factors impacting the success of long-term human–robot interaction. According to the authors, the main factors are physical ergonomics, mental workload, trust, acceptance, and usability. In addition, the presence of complex machines and their relations with the technological process also impacts human–robot collaboration. Balancing robotic assembly lines containing complex machines, robots, and humans sharing a common workpiece has remained an actual problem for over 30 years (Chutima, 2022, Bänziger 2020).

Another actual HRI research area is the issue of robot-human interaction in unexpected situations. Paper (Gualtieri et al., 2022) presents a virtually simulated solution for HRI in unforeseen situations and provides guidelines that effectively support non-expert users in designing and improving collaborative assembly systems from a security perspective. The authors declare that minimizing human motion amplitude and optimizing the assembly process could reduce the risk of accidents by 33%. Situation awareness is crucial not only for humans but also for robots. Research (Müller et al., 2023) presents an attempt to develop a metric capable of evaluating situation awareness by the robot using a digital twin. Research presented by (Kousi et al., 2019) introduces an augmented reality-based software suite to assist operators in manufacturing systems using mobile robots when it is challenging to predict cooperation between a robot and a human due to the presence of various tasks. The developed tool was tested in a case study inspired by the automotive industry, showing that it can facilitate communication between humans and mobile robots, increasing the work quality of human operators and supporting the assembly that connects them. A study presented by (Murata et al., 2017) focuses on analyzing the robots’ ability to train each other in a special neural network that evaluates error probabilities.

Table 1 provides a summarized overview of research on general HRI issues reported in the last five years.

Table 1 The research summary of general HRI issues reported in the last five years
Full size table

Adaptive human–robot interaction

Robots’ capability to adapt to individual needs and human emotional or physical states is a widely studied issue (Umbrico et al., 2022). Robot adaptation can facilitate human physical work and perform the social functions of robots, but on the other hand, it results in complex structures requiring specialized hardware and software. Figure 2 provides an example of the architecture of an adaptive human–robot interaction case.

Fig. 2
figure 2

Example of adaptive human–robot interaction architecture (Umbrico et al., 2022)

Full size image

Adaptive robotic solutions are especially preferred in medical or rehabilitation applications. One example could be exoskeletons used to ease human physical exertion when lifting heavy objects. Their purpose is to duplicate the movements of the human body parts while creating a correspondingly greater force (than the parts of the human body produce).

Research provided by (Huang et al., 2019) examines such issues as the suitability of the exoskeleton for people of various body types, the comfort issues of the exoskeleton, and the ability of the exoskeleton control algorithms to interact with the human harmoniously. The research authors noted that relevant parameters, such as height, mass, and body mass index, could describe human body composition. Therefore, they developed an exoskeleton model that evaluates human complexity and provides data for exoskeleton adaptation.

Another issue of exoskeleton adaptation is its ergonomics. Research performed by (Ballen-Moreno et al., 2022) described a method that quantifies the difference in orientation between a user’s limb and the exoskeleton joint. This method brings a better understanding of human–robot interactions in implementing exoskeletons. In addition, the method proposed in the article determines the performance indicator of the physical interfaces of the exoskeleton.

Apart from physical adaptation, the question of perception is also relevant. In various applications, there is a problem of communication between humans and robots when the need to provide the necessary tools or equipment to humans in time arises. An experimental study providing a method of how a human can use gestures to request one or another tool for the assembly operation was conducted by (Neto et al., 2019). In the presented approach, the data captured by the robot is divided into static and dynamic blocks that are recognized using unsupervised machine learning. The proposed method demonstrated 98% accuracy in recognizing eight static and four dynamic gestures.

Cooperation between humans and robots may not necessarily be based on targeted physical tasks. It could also be based on psychological reasoning. The robot can partially perform the social function of a colleague, friend, or pet. In such cases, the social characteristics of a person and how the robot adapts to a person become essential for successful human–robot interaction. Paper (Lavit Nicora et al., 2021) describes the developed interaction model that proves the possibility of creating software that facilitates a robot’s adaptation to a person’s characteristics. Meanwhile, the research provided by (Oliveira et al., 2021) analyses the difference in HRI characteristics between a single person and a group of people. The authors suggest some avenues and future methodological trends that can help assess human behavior in human–robot interaction situations by increasing ways to assess these interactions in groups. In (Bajcsy et al., 2018) presented a model for human–robot interaction that evaluates extraneous physical factors in a case where two robots interact with two humans. Case, when few human operators interact with one collaborative robot, is also possible (Boschetti, 2021). Research provided by (Cacace et al., 2023) reveals issues of interactive physical cooperation between humans and collaborative robots. Their approach is based on the idea that robots should be able to estimate human intentions and adjust initially defined tasks and motions. About 80% of 40 undergraduate students with different experiences related to robots stated that interaction with a collaborative system seems safe and straightforward.

Studying robots’ global and local information needs is another research topic. In (Yu et al., 2022), the modeling of an information network to provide rational information for the robot is described. Addressing the problem of robot reaction speed deficiency, (Mugisha et al., 2022) presented an experimental study of improving the movement prediction time and reducing the time required for the robot to reach the desired position. The experimental results of this study revealed that eye gaze-based prediction significantly improved system performance. The detection time was reduced by 37%, and the time required to reach the target was reduced by 27%.

A summary of the research focused on the robots’ adaptation to the individual humans’ needs in the last five years period is provided in Table 2.

Table 2 Research in the field of HRI focused on the robot adaptation to humans
Full size table

Summarizing the information described above, it should be noted that it is necessary to emphasize questions such as the robot’s need for global and local information, human–robot interaction in assessing extraneous physical factors, the laws of the lack of robot reaction speed, and the ability of robots to train each other. As well as the question of whether it is possible to create a typical robot-human interaction methodology according to the established conditions demands special attention. To answer this query, studying the development of human–robot interaction research methodologies is required.

Human–robot interaction in specific application areas

The number of areas in which robots can be applied is not defined or thought to be finite. For that reason, new application areas are constantly appearing, and thus, the research question of possibilities to robotize one or another process becomes more actual. There are several application areas where robots are not widely used or have yet to be fully explored. Despite advances in medical robots and assistive technologies, the application of robots in healthcare and robots used for education is still in its infancy. The application of robots in agriculture is still limited, although there is increasing interest in using robots for tasks such as crop monitoring and harvesting.

Moreover, robots have yet to be widely adopted in the service industry, although certain customer service and hospitality tasks show growing demand for them. Additionally, robots used for environment monitoring and cleaning pollution are instrumental fields, as the potential of using robots in environmental applications remains largely untapped. Figure 3 shows a few examples of different levels of human–robot collaboration in various applications.

Fig. 3
figure 3

Several examples of human–robot interaction and collaborative human–robot tasks (Hentout et al., 2019)

Full size image

Shipbuilding is one of the industrial areas in which questions about the possibility of automation of the processes often appear. Major challenges in shipbuilding are the large variety of weights and dimensions of the elements to be installed—from several kilograms to tens of tons. There is also a diversity of required precision of movements. One robot cannot be adapted to such a wide range of needs. The paper’s authors (Zacharaki et al., 2022) proposed an algorithm for grouping operations, creating robot operation zones, and ensuring the safety of their interaction with humans while considering the above mentioned operations.

Another problematic field from the point of view of robot-human cooperation is the field of equipment and components utilization. On the one hand, a lot of routine work can be automated in this field. On the other hand, the equipment used is diverse and creates many limitations (Hjorth & Chrysostomou, 2022; Qu, 2023). The main difficulties identified by the authors are high variability of the technical conditions of the used parts; insufficient information about the recycled products; increasing complexity of recycled products; short product life cycle and a large variety of products; increasing quality requirements for regenerated materials, components and varying requirements for utilization efficiency.

In contrast to shipbuilding or equipment utilization, agriculture faces a lot of the same repetitive work, so the application of robots here should be effective. Such operations as land plowing, harrowing, and fertilizing should be mechanized and sometimes even robotic. The question of harvesting is more complicated, thus gathering attention in the scientific literature (Vasconez et al., 2019). The paper evaluates the complexity of various tasks in terms of robot-human cooperation. Operations, such as grain crop harvesting, are massive and straightforward processes conventionally agreed to be relatively simple. However, operations with fruits and vegetables are much more complicated. There is a considerable variety of them (for example, some with dice, others with stems, etc.), different sizes, masses, and different requirements for handling them. Pruning and thinning of fruit trees is a separate issue. According to the article’s authors, these operations can be performed by joining robots and human abilities. However, such integration requires special algorithms and software for the robots.

Another problematic area similar to agriculture is forestry. Large areas of forests and predictable cases characterize this area. One of them is forest fires. One of the most effective measures to facilitate the extinguishing of forest fires is fire detection at an early stage (before the fire has covered large forest areas). A research paper (Lim et al., 2021) discusses optimizing the distribution of decision-making between humans and artificial intelligence by implementing a robotic control module.

A similar issue of optimizing decision-making processes is reported in medicine, where robots are used not only in treatment and rehabilitation processes but also to help organize the work of a medical facility, such as dispensing drugs or tools (Lestingi et al., 2021). Also, in a medical institution, prioritization of work is very important. Research presented in (Wan et al., 2020) examines a developed algorithm that allows a medical robot to distinguish priority tasks from others.

Robot implementation for treatment and rehabilitation also remains a relevant research topic—for example, using a robot to treat autism. Paper (Katsanis & Moulianitis, 2021) presents a taxonomy of child-robot interactions in autism interventions, explaining its entire framework. Interactions are modeled according to this taxonomy, where an interaction case is used to define the structure of the interaction. Based on this, a safety architecture is proposed to be integrated into the robot controller. Scientific articles also address the suitability of robots for rehabilitating human limbs (Shi et al., 2021). The study proposes a human-centered adaptive control of a lower limb rehabilitation robot based on a dynamic human–robot interaction mode. A dynamic human–robot system model is developed based on the HRI model. An equivalent spring model in three-dimensional space is proposed.

A summary of the research focused on the robots’ implementation issues in the specific application areas in the last five years period is provided in Table 3.

Table 3 Summary of research focused on implementing robots for the specific case
Full size table

Human motion detection and prediction/behavior prediction

Human–robot collaboration is impossible without the use of specific methods preventing humans from rough/dangerous interactions during the operation. Human motion detection and its trajectory prediction are some of the most preferred and researched methods in HRI. Those methods are typically used to solve two major issues in robotics: prevent contact between the robot and human, or vice versa—synchronize the motion for smooth common action. However, the unequivocal assignment of the research task to one or another case is not simple since both cases include the part of motion detection. Methodology and equipment for human motion detection should account for humans’ accidental reactions caused by fear and general robophobia. Therefore, classification based on the implemented technique seems to be more reliable. The following parts provide a detailed analysis of research on human motion and detection issues, classifying the proposed approach according to the implemented methods. An overview of available reports revealed three main approaches used for motion detection and prediction: implementation of predefined algorithms (Table 4), use of physical sensors (Table 5), and application of machine learning algorithms (Table 6).

Table 4 Human motion detection using predefined algorithms
Full size table
Table 5 Human motion detection using physical sensors
Full size table
Table 6 Motion trajectory prediction using databases and machine learning
Full size table

As seen from Table 4, human motion detection and prediction based on prescribed algorithms require initial references such as posture, eye contact, or workspace distribution into smaller parts. The main problem with using prescribed algorithms is existing application constraints due to the possible unexpected human reaction. Nevertheless, there are cases where human motion prediction is impossible without implementing prescribed algorithms. Analysis and prediction of human gait is a complicated process due to the complex neuromusculoskeletal system and cannot be performed without using predictive models and experimental data. A detailed review conducted by (De Groote & Falisse, 2021) presents modern methods and models suitable for human gait analysis, motion, and trajectory prediction. Such methods are essential when it is necessary not only to avoid contact with a person but also to synchronize the movements of the human and the device precisely.

Human motion detection based on physical sensors is more precise than the methods based on prescribed algorithms, but this method also comes with its own issues. The major one is the restriction created by contact sensors. Such sensors limit humans’ ability to move freely and can even distract their attention from the main focus point, as the sensors cause discomfort. An alternative to this is the application of non-contact methods, for example, based on computer vision. However, computer vision-based methods are not so accurate and reliable. Moreover, they require stable environmental conditions (light intensity, the position of a light source, etc.) and higher computational power than methods based on contact sensors, for example, electromyography.

The most advanced approach nowadays to detecting and predicting human motion is the implementation of various machine learning algorithms (Table 6). It combines the advantages of both previously described methods and simultaneously allows us to avoid their drawbacks. Such algorithms can be trained using accurate data collected using various sensors in predefined/controlled conditions and later implemented in systems equipped only with basic sensors. For example, algorithms can be trained using precise data about motion obtained from accelerometers or electromyography and implemented in systems equipped with average-resolution vision sensors.

In terms of complexity in human motion detection and prediction, the most challenging tasks require motion synchronization (Table 7). A successful solution to such problems must include human motion detection, prediction models (Vianello et al., 2023), and real-time trajectory generation capabilities.

Table 7 Human motion detection and trajectory definition to synchronize for common operations
Full size table

Concerning Table 7, the most recent research on human–robot motion synchronization indicates that the latter is more applicable in home appliances and general service robots than in industrial ones. Such a situation could be explained by the industry’s low popularity of collaborative robots and strict work safety regulations. Nevertheless, pressure in the labor market develops an increasing number of collaborative robot installations in the industries, thus fostering an interest in the highest degree of human–robot collaboration. An excellent example of human–robot collaboration is provided in (Rahman, 2021), where a robotic manipulator defines the weight of the box lifted by a human.

Human–robot communication

Proper understanding of the situation and future actions plays an essential role in the life of humans, clever life beings, and intellectual equipment. Communication between humans and robots is artificial; therefore, an intuitive understanding of signals and signaling back requires additional effort. The advantages of such communication bring high benefits—remote control mode, diminishing of technical breaks and interoperation stops, enhancement of human comfort level, allowing the operator to control a robot with lower stress level. While communication between robots and humans is still artificially driven, robot behavior toward humans is disclosed by technical means. Depending on the implemented technique, three main types of human–robot communication techniques can be distinguished: speech-based communication (Table 8), sensors-based communication (Table 9), and symbolic language/gest-based communication (Table 10).

Table 8 Voice/speech-based human–robot communication
Full size table
Table 9 Sensor based human–robot communication
Full size table
Table 10 Symbolic/gest-based human–robot communication
Full size table

The voice and speech recognition-based technique (Richards & Matuszek, 2021) promises a powerful tool for robot control and expression of natural human reactions. The proposed speech recognition technique reaches recognition accuracy up to 90.3% using a dataset containing more than 7000 descriptions of 300 items.

Research provided by (Maggioni, 2023) proves the importance of verbal reaction. Authors experimentally defined that implementing verbal functions to the robot makes it possible to achieve a higher robot acceptance ratio since it becomes more attractive for interaction.

Efforts to teach natural language are most prospective from the point of intuitive control, but they contain a lot of obstacles and technicalities. Firstly, language understanding anchor differs from language to language. Secondly, the lexicon of the operator should be limited by a set of keywords. Training of language with a particular operator (Higgins et al., 2021) solves a task with more limitations, but it is not transferable to another operator directly. This technique works well in individual cases, but operator change brings extra expenses and resources in the long run. Voice recognition tasks in a noisy environment remain complicated, especially when signal/noise power and spectrum are in a similar range. Interesting selective voice recognition is described in (Fukumori et al., 2022), where the Doppler-effect sensor distinguishes voice signals from environmental noise. Miscommunication with robots due to accident fear or general robophobia can be detected by instrumental methods since the speed of human movement and their type differs from standard human operation mode. (Richardson, 2020).

Currently, sensors-based human–robot communication is one of the fastest emerging research trends in human–robot interaction. Such an increase is mainly caused by the increasing need for more advanced communication methods and significant developments in sensing, signals acquisition, and data processing technologies. Simultaneously, with the development of techniques intended to transform human action or reaction into a robot control command, much attention is dedicated to the inverse case—feedback from the robot to the human as a response to the actual action. Haptic devices that use force, vibration, or motion to create a sense of touch are popular in the HRI (Kuhail, 2023). Their main development trend is optimizing design and enhancing functionality to achieve more realistic and intuitive responses from the machine. The robot’s appearance also plays a significant role in human–robot communication (Song, 2022). It could provide feedback for the human, for example, by changing the robot’s eye color or providing associative images on the robot control screen.

The communication system between robots and humans using human expressions like mimic, hand, and body gestures has a particular perspective on human–robot cooperation and collaboration. For this purpose, most solutions distinguish gesture language as a special command language (Canuto et al., 2022), which is not intuitively developed and requires human education. Nevertheless, such language generates better and clearer commands due to their active character. Special gestures, which significantly differ from the natural human reaction, were developed and presented in (Mazhar et al., 2019).

Another way of human–robot communication lays in the robot’s understanding of natural human gestures. It is a more technically complicated option, but it eliminates the need to educate operators or bypass personnel about special gestures. One of these methods translates gestures into control commands (Obo & Takizawa, 2022). Another method trains robots to understand gestures (Caccavale et al., 2019). Both ways are prospective, but their reliability and dependency on human personality require additional research. Misunderstanding human gestures by robots in case of unnatural fear or general robophobia requires a separate robot operation mode, but no certain research is available. Social behavior for fear of robots is analyzed better (Fraune et al., 2019).

Human emotion and physical state evaluation as input for robot controls

Emotions and subconscious body language play a crucial role in inter-human communication, as they are the fundamental modes of communication in the animal world. Natural emotional messaging is unconscious and is known as emotional contagion (Hatfield et al., 2014). Despite the traditional belief that robots generally are ill-suited for emotional communication, there is plenty of ongoing research about computer recognition of human emotions and the emotional appearance of the robots (Kulke et al., 2020; Lim et al., 2020; Noroozi et al., 2021; Park & Whang, 2022; Ruhland et al., 2015; Toichoa Eyam et al., 2021; Weis & Herbert, 2022). While the latter aspect is more relevant for social and service robots, reading human operators’ emotional contagion and non-verbal cues is becoming an important control input if a human–robot collaborative environment is established.

Safety is supposed to be a priority over other human–robot collaborative aspects, such as production effectiveness or speed. Boredom, fatigue, and stress are human-specific variables that can lead to physical and psychological accidents during human–robot interaction. Biometric artificial intelligence methods, such as facial, speech, and body language recognition, can be applied to read these variables. Commonly, reading electrical signals of the human body employing electrocardiography (ECG), electromyography (EMG), and electroencephalography (EEG) remain fundamental non-invasive methods for emotional state detection (Dzedzickis et al., 2020). Brain activity measurement by EEG was recently shown to be adequate for measuring the emotional state of humans’ joint work with the collaborative robot during the assembly process (Toichoa Eyam et al., 2021). Moreover, the detected emotional state of the human operator (such as stress level) was used as an input for the robot velocity adjustment. It was found that emotional feedback to the robot increased the trust and comfort levels of the human operator but reduced the engagement due to routine settlement. Also, the authors admit that EEG is the most accurate and reliable technique of human emotion measurement among the other widely known techniques.

Still, EEG can be uncomfortable and even unacceptable in many practical situations. Therefore, biometric emotion measurement methods are gaining importance, especially as they are backed by the rapid development of artificial intelligence and augmented reality algorithms. The human face is one of the most important instruments of emotional communication, and the eyes are central to conveying emotional information between humans (Ruhland et al., 2015). There are plenty of parameters to be tracked to utilize the eyes for emotion detection: eyeball position and movements, eyelid position and movements, pupil diameter and its variation, fixation duration, saccade, and many others (Lim et al., 2020; Ruhland et al., 2015). The taxonomy of emotion recognition using eye-tracking is shown in Fig. 4. Desktop and wearable (mounted onto glasses) eye trackers were recently created to read most of the important eye parameters, and they are often used in augmented reality applications.

Fig. 4
figure 4

Taxonomy of emotion recognition using eye-tracking (Lim et al., 2020)

Full size image

Although there is some basic knowledge about the relationship between the above mentioned eye parameters and particular emotions, there is an obvious lack of up-to-date experimental research demonstrating any practical application of eye tracking for emotional feedback to the robots. At present, eye tracking is seen only as a component of multimodal emotion measurement systems (Lim et al., 2020). In their latest study from 2022, (Lewandowska et al., 2022) employed eye-tracking technology to gauge user focus and emotional reactions to both negative and positive webpage content. However, it’s worth noting that the findings from Lewandowska’s research do not offer any real-time feedback based on the eye-tracker data.

Rapidly developing AI-backed facial movement and facial expression analysis algorithms, based on the Facial Action Coding System proposed by American psychologist Paul Eckman (Ekman, 1992), is another trend for recognition of basic human emotions, such as “happy”, “angry” and “neutral”. Recently it was found that AI-detected emotional states correlate well with the results of the parallel emotion measurement done by interpreting EMG data (Kulke et al., 2020). However, the conditions under which this research was performed are quite far off from the practical situations of human–robot collaborative environments since participants of the study (twenty students) were explicitly instructed to imitate emotions.

Similarly, AI-backed classification of body postures and their relationships with intense emotions are currently being researched by behavioral and technology scientists. The pipelined concept of automatic emotion detection from body postures is illustrated in Fig. 5. It involves several detection stages, starting from capturing the person in a video stream or a photo, estimating its body pose, which is based on the part-based skeletal or kinematic model of the human body, and recognition of emotion based on several emotions models: categorical, dimensional and componential. Still, the system’s output remains of limited reliability and doubtful value since many personal, cultural, and gender related disturbances may cause serious misinterpretations (Noroozi et al., 2021).

Fig. 5
figure 5

General overview of an Emotion Body Gesture Recognition system

Full size image

The topic of emotional feedback from humans to robots remains at the basic research and demonstration level. Most authors admit the lack of a more general understanding of the importance of the relationships between human emotions and the parameters of the robot control programs. The measurement of emotions, in general, remains technologically complicated and of low reliability since the variability between the subjects is one of the unsolved challenges in this field. Also, different efforts are needed to measure or classify emotions since negative or neutral emotion recognition brings more challenges than the measurement or classification of positive ones. Furthermore, recognition of emotions often leads to big data issues, and a successful solution requires edge computing available by using centralized cloud resources. On the other hand, such an approach raises cybersecurity issues (Yao et al., 2022). A summary of research on human emotions evaluation data as an input variable for HRI is presented in Table 11.

Table 11 Human emotion evaluation as input for robot control systems
Full size table

Human perception of robots

Human perception of robots defines human reaction to the robot from psychological, social, and economic perspectives.

Socializing with a robot is not science fiction nowadays; several social robots with empathetic functionalities have been recently released to the market. The most widely known models are Softbanks Pepper and Jibo by NTT Disruption, which were created almost a decade ago. Several startups followed with similar products, loading them with valuable functions ranging from remote presence and entertainment to shopping assistance. Still, despite many early promises and heavy marketing, social robots lack true popularity. One of the most probable reasons is limited human trust in robots and artificial intelligence in general. Sometimes, this is expressed as an anxiety disorder called robophobia, which is specific for almost 20% of the world’s population, according to (Davey, 1997). Much greater numbers of humans are possibly deeply biased against the robots in many ways, not only because of their experiences of limited efficiency and performance during previous application and/or collaboration attempts but also because of some irrational disbelief. This is presently identified as a social problem, preventing the general humankind from more efficient technological development. Therefore, there is a number of ongoing research directed towards exploring perceptional, cognitive, behavioral, cultural, existential, or economic concerns of humans related to the application of robots and artificial intelligence (Tables 12, 13, 14 and 15).

Table 12 Human reaction to the robot
Full size table
Table 13 Human trust and safety analysis
Full size table
Table 14 Instrumental human reaction evaluation
Full size table
Table 15 Analysis of robots’ suitability to social applications
Full size table

The term “empathy” is employed to summarize, understand, and explore the affective (emotional, primitive) and cognitive (associated with the ability to understand the mental state or perspective of another person) parameters of human–robot interaction (HRI) (Park & Whang, 2022). A model of empathy in HRI is illustrated in Fig. 6. Generally, the study revealed the absence of the cognitive responses of humans during interaction with robots, while affective response dominates. The authors note the importance of sophisticated emotional models in robotic software for improved human perception and admit the lack of research that would disclose the models with the potential of a more positive perception of the robot by a human user.

Fig. 6
figure 6

Conceptual model of empathy of HRI (Park & Whang, 2022)

Full size image

One of the few more or less successful emotional models for improved HRI is related to dog-like (canine) social robots, such as Sony Aibo (De Visser et al., 2022; Krueger et al., 2021). While investigating the emotional reaction of humans to the robots (Table 12), authors hypothesize that framing a robot as a puppy, which has a corresponding appearance and mimics a dog, the learning process will have a higher potential for positive acceptance than simply a robot, such as Spot by Boston Dynamics (De Visser et al., 2022). Authors identify the “uncanine valley” phenomenon in the emotional reaction versus dog-likeness graph (Fig. 7). Experiments with the appearance of a canine robot by dressing it in fur showed that the presence or absence of fur changed the emotional reaction of human participants. However, these changes depend on whether a robot is framed as a puppy or just a robot. Overall, it was concluded that framing a canine robot as a learning puppy will lead to a richer interactive pattern and human perceptions in HRI, and this experience can be elaborated for human and robot collaborative environments and situations.

Fig. 7
figure 7

Representation of the “uncanine” valley, a hypothesized adaption of the uncanny valley (De Visser et al., 2022)

Full size image

Social touch is another research track of HRI and robot social acceptance improvement. The phenomenon of social touch is investigated mainly in behavioral sciences, and it is widely known that it can elicit a vast range of emotional and behavioral responses. Recently it was shown that a social touch from a robot could be perceived positively by a human participant, reducing stress and improving the sense of intimacy between a human and a robot (Willemse & van Erp, 2019). Still, this research focuses on human psychology, and robot operation was only imitated via the master–slave configuration involving a human moderator at the master controls.

Present research on the human reactions to robots introduced new concepts or paradigms known as anthropomorphism (Fischer, 2022) and “computers are social actors” (J.-E. R. Lee & Nass, 2010). Anthropomorphizing behavior is a specific psychological phenomenon when people tend to attribute human-like traits to robots. Anthropomorphizing behavior was observed in many psychological and physiological studies, but this phenomenon still lacks more generalized explanations. Systematic qualitative research (Fischer, 2022) demonstrated significant intra- and interpersonal variation in the responses of human participants to identical robot behavior patterns, with easy switching from anthropomorphizing behavior to technical behavior (treating the robot as a machine). These observations are taken as arguments that the paradigm “computers are social actors” does not hold in general since anthropomorphizing behavior is temporal and will be different for different persons. However, the conditions under which this behavior was studied in this particular research were not universal since the appearance and behavior of a robot used in the experiment were far from anthropomorphic.

Furthermore, human reactions and behavior with respect to robots strongly depend on the actual robot implementation use case (Etemad-Sajadi et al., 2022) and the robot’s structure as well as external look (Jørgensen et al., 2022). Questioning respondents after reviewing Peper robots in various actions brought answers that robot acceptance by humans is mainly affected by safety and trust as well as by robot behavior scenarios. Evidently, the more similar to humans it is, the higher acceptance can be achieved (Etemad-Sajadi et al., 2022). However, according to (Van Maris et al., 2021), higher acceptance of robots leads to positive consequences only if it is related to broader implementations of the robots. In other cases, it can lead to stronger emotional attachment, dependence on robots, and self-isolation, especially when using service robots to assist older people. A similar involvement in affection is demonstrated in (Lee & Liang, 2019), where authors have proved that a foot-in-door strategy—smaller requests followed by large ones could be successfully implemented in HRI to persuade humans to perform required actions.

The study provided by (Jørgensen et al., 2022) defined that human reaction and behavior patterns differ in the case of interaction with conventional and soft robots. Nevertheless, the experiment participants could not specify which robot type seemed more natural. Such behavior proves the complexity of human perception and the need for further extensive multifactorial research in this field.

Human trust and safety in robotic installations were analyzed in many references, which are embraced in Table 13. Direct evaluation of human trust in the robot operation is a complicated task, and the indirect definition of trust level experimental research is provided in (Story et al., 2022) and (Horstmann & Krämer, 2022). In addition to experimental methodology, a simulation of the robot’s impression to the examined person in the working environment exists (Akalin et al., 2022). A modern augmented reality method and game environment are useful for human trust research and bring outstanding results (Alarcon et al., 2021). The game environment can attract young people into robotic and industrial action circumstances and help promote industrial careers.

Theoretical analysis of human behavior and comfort level in HRI by using meta-analysis and a broad view in the available literature presented in (Hancock et al., 2011). Meta-analysis can predict HRI at an early stage and provide fast results for frequent cases, but local influence (human habits, societal opinion, etc.) opens a broad space for further analysis.

Human acceptance of the robot environment develops a certain human reaction to them. The reaction can be expressed as acceptance of different degrees. Evaluation of this degree was analyzed in the paper (Roesler et al., 2022). It can be based on the level of anthropomorphism (Xiao et al., 2022)(Páez & González, 2022b)n mode (Páez & González, 2022) or general sociality (Londoño et al., 2022).

Another issue limiting the implementation of emotional models into HRI is related to the challenges of evaluating real human emotions: subconscious (instrumental) and conscious (questionnaires) evaluation often provide opposite or noncorrelating responses. Therefore, it is necessary to foster the development of instrumental human emotions evaluation models suitable for HRI (Table 14).

Recent research (Y. Hu et al., 2022) involving 35 participants proved the existence of the relationships between humans’ physical and psychological data and their age, gender, perception, and personality during HRI. Such dependencies are vital in developing adaptive HRI frameworks capable of responding to the physical and mental state of the robot operator. Furthermore, it has been experimentally proved that humans misunderstanding robot intentions is one of the major issues (Khoramshahi & Billard, 2019). The authors performed experiments on developing a task-based HRI, where the robot must recognize the human intention, switch to a corresponding task, or adjust motion parameters. Nevertheless, it was found that sometimes humans falsely assume that robot has recognized their intention, and as a result, humans disturb the process by themselves. Therefore, reliable adaptive HRI requires not only instrumental evaluation of human intentions but also feedback from the robot. Furthermore, this study defined that a different data update rate is required to ensure stable operation: 1 kHz for the haptic device, 200 Hz for the lightweight robotic arm, and 125 Hz for the mobile robot. The benefits of the feedback from the robot to the human are discussed in (Willemse & van Erp, 2019). Researchers defined that the touch initiated by the service robot has a similar positive effect as the friendly touch of a human. It minimizes psychological stress, creates stronger bonds between robots and humans, and could extend non-verbal communication capabilities.

Implementing robots in social-based applications brings other HRI issues (Table 15). Humans’ reactions to robots are highly hidden due to their complexity and frequent possibility of avoiding undesired interactions. Outstanding research (Choi et al., 2020) on laymen’s human social behavior reveals a reaction to the robots serving the hotel. Interviews with hotel guests reveal quite a strong reaction vector to robots’ existence in the hotel; a study with 400 participants showed that human staff services are perceived as having slightly higher interaction quality than the service of service robots. Theoretical research on human expectations from robots and robotic technologies is provided in the review (Saunderson & Nejat, 2019), where some specialists’ survey brings the main direction on the acceptance degree of robot non-verbal behavior. Special robot social conditions in operation in the social area of aged persons are defined experimentally (Moro et al., 2019) using a very intelligent robot, Casper. Based on the findings, it was confirmed that human emotion expression capability raises engagement of HRI and brings a general positive effect.

To summarize, all the recent research on the human reaction to robots is at the beginning of a quest to discover the complexity of human-to-robot reactions and corresponding behavior models. The complexity of certain technological developments can make it difficult to fully understand their causes and consequences. Such a lack of understanding can lead individuals or society as a whole to resist or accept these technologies. However, it is also important to note that other factors, such as ethical issues, privacy concerns, or social impact, may contribute to this unacceptability. Scientists and technology developers must establish open and transparent communication with individuals, communities, and society to better understand their concerns and find solutions that meet everyone’s needs. This may include research to evaluate the technology, developing ethical guidelines to ensure its responsible use, or considering alternative methods that address problems and promote acceptance.

Provided theoretical and experimental research in HRI opens the free space for the psychological impact of robots on human measurement. Another big issue is the improvement of human attitudes toward robots and their installations. These issues stimulate new research and development of new technologies in the future.

Discussion

The recent research on HRI and related sociopsychological phenomena, such as robophobia and anthropomorphism, is at the beginning of the process of disclosing the vast complexity of its context and indicates the necessary contributions from a wide variety of disciplines. Here, engineering is taking just a minor part of the whole, while psychology, sociology, humanities, and design are becoming equally important.

  1. 1.

    Human–robot cooperation and collaboration can be effective in industry and possibly in other partially predefined environments;

  2. 2.

    Communication and messaging in HRI are at slow progress but lack a systematic approach;

  3. 3.

    HRI reveals new socio-psychologic phenomena;

  4. 4.

    The general acceptance of robots over the entire population is mixed and underexplored;

  5. 5.

    Emotional communication is very effective between humans, but achieving better understanding and classification is necessary for improved human-robot interaction.

Dynamic intrusion and the limited success of social robotics during the last decade are significant motivators for investments in further HRI research. In our review, we found the confirmation that HRI reveals some new socio-psychologic phenomena. As our review has shown, not many publications describe successful collaboration between different disciplines, which defines the context of such HRI research. On the contrary, technology research continues to progress rapidly at its own pace, while psychology and social research are often organized with outdated and obsolete technical equipment, without explicit practical value.

Even in the wide public exists Grimwade’s Syndrome known as the effect of robophobia. Relations between robophobia and access to robots have a multiverse connection. The performed analysis discovered that robophobia mostly affects the broad public with minimal access to robots. Regarding levels of HRI, the first level (isolated robotic cell) is mostly safe and causes fear for untrained people. Higher levels of interaction typically happen with trained personnel, with minimal tendencies to robophobia. On the other hand, this research doesn’t reveal the cause of such phenomena—possibly, that Grimwade syndrome can be cured by contact with robots or personnel with such conditions avoid robots.

Our review found a sound positive backing for our hypothesis about the effectiveness of human–robot cooperation and collaboration in industry and other environments. The collaborative approach between humans and robots is being demonstrated as productive in industry and medicine, although reliable and unambiguous feedback from humans to robots remains an issue. This also supports our hypothesis about the slow progress of communication and messaging in HRI. While several works demonstrate promising results in applying and developing tactile sensors, others target the development of gesture languages. We found that high adaptivity based on effective machine learning of various types and corresponding AI is essential when humans and robots share the workplaces, synchronize their work, cooperate, and collaborate.

We found support for the hypotheses about emotional feedback as another promising field of HRI research. However, we identify it as still remaining at the basic research and demonstration level. It appears that there is a lack of a more general understanding of human emotions to be considered as important feedback inputs for robot algorithms. Also, the measurement of emotions remains technologically complicated and of low reliability, mainly because of the variability between human subjects.

In the current discussion, we would like to emphasize the complexity of the human perception of the robots. We found support for our hypothesis about the mixed acceptance of robots over the entire population. Although early work identified computers as potential social actors, more recent research has demonstrated mixed and easily switchable social perceptions and acceptance of robots. There are situations in which the same robot can be perceived as an anthropomorphic entity. At the same time, it can be treated as just a machine if the situation has changed. For example, a robot created for entertainment purposes, such as a humanoid or quadrupedal robot that can dance or play games, may be perceived by audiences as an anthropomorphic or zoomorphic entity. In contrast, a robot for industrial use, such as a task to assemble a robot arm, may be perceived by operators and maintenance personnel as just a machine.

Future research in HRI will develop intuitively predictable robot signaling to humans, discovering intended actions before they are estimated rather than factual operations; this will add some trust to robot perception by laymen in interference with robots or robotic complexes. Human emotion evaluation by the robotic system will ass flexibility to robot operation, especially in the mobile mode. All these enhancements require new HRI conception and backing of such conception by hardware and software. Pure emotional factors of robot perception, like design, coloration, and sound, significantly impact HRI, so a broad area is open for new activities and design findings.

Social science and psychology have their own challenges: early robophobia detection and prevention, suggestive phobia treatment, or social prevention of phobia-induced people to access robots on a physical level. Psychologists should initially develop questionaries for robophobia detection; there are existing ones for agoraphobia, social phobia or other types of phobias. Special cases with children’s phobias raise requests for education methodologies or even social animation material, including graphic games. Reconsidering recently existing games and proper visualization of robots, there will be an aim to reduce robotic fear in general because the maturing of young generation will not be infected with some mysterious-born phobias. Treatment of specific phobias lacks a methodology for robotic phobia as well; there are no references pointing to robophobia treatment methodology so far.

Additionally, people’s perceptions of robots can be influenced by factors such as their design, the way they interact with humans, and the tasks they perform. Also, the same situation is valid for humans, and different perceptions can result from education. These interpersonal variations are similar in principle to those mentioned in the emotional feedback context and, therefore, similarly challenge the robot developers. While emotional feedback can be a highly effective data source for improving the collaborating robots, our hypothesis about the need for better understanding and interpretation of human emotions has also found good support.

Conclusions

Many areas in daily life show potential for robotization; however, many factors must be assessed and successfully combated to employ robots working alongside humans seamlessly. Different intensity levels of robot-human interaction propose their own benefits and drawbacks, signifying the increase in relational complexity as we move up the intensity scale. To achieve efficient human–robot collaboration, a lot of work in defining guidelines, safety measures, and design is yet to be done. Considering these factors, further research on human trust, reaction, and response must be conducted to provide more data for generalizations.

There are several recent demonstrations of successful human–robot cooperation with sensory and emotional feedback. However, the extension of these achievements to wider application areas, except in industry and medicine, remains limited.

Effective human-to-robot communication remains an issue despite several examples of gesture language, AI-backed speech and face recognition, and emotion recognition algorithms. New gesture, body language, or speech recognition features for robots are required, but such research is not available in public sources.

New sociopsychological phenomena such as robophobia and sporadic anthropomorphism are gaining importance in HRI research. In contrast to others, these phenomena are still new and indicate very indirect effects on the human psyche in the absence of robots.

Emotional feedback of humans to robots is assumed to be the preferable adaptive input to cooperative/collaborative robot behavior. However, realistic human emotion recognition remains a big issue and has low reliability. Moreover, general acceptance of robots and caused emotion as such requires deeper analysis.

Human perception of robots is very complex and presently can be regarded as sporadic, i.e., hardly predictable, easily spreading over the audience, and having strong interpersonal variability. Emotional human–robot communication is and will continue to remain limited; forthcoming progress will decrease communication limits.

Future HRI will intensify and cover the broader public, which implies some challenges in this field. Authors suppose that industrial robots should be classified into more than now existing two groups (robots and cobots), according to operation intensity. As a result, safety standards will appear, describing all categories of robots, thus developing optical and other markings to distinct security levels of this robot, defining human behavior in robot environments and safety levels where robots can enter as service. There is a prediction for big society preparation for robotic safety rules, like behavior in the street with traffic. Therefore, all these changes must be naturally understandable and clear for everybody. Specialized production areas will keep no-enter zones for the public, and the robot interaction level will still require training and education. Ultimately, authors would encourage leaving space for humans in HRI and unlimited robot development for a comfortable human life, unshaded by massive robophobia.