1 Introduction

In a world where humans and intelligent machines are expected to share the same environment, be it public or domestic, and to communicate and collaborate, it is crucial that their interaction takes place in a way that is satisfying for humans and efficient. As shown by many studies [1–3], the understanding of what makes an interaction successful, socially adapted and pleasant for humans, is exposed to interpersonal [4, 5] and intercultural variations [6]. For instance, the degree to which a human-like nature for a robot is needed is not yet understood to a sufficient degree and studies that focus on such a human-like nature are rare [7, 8]. As regarded to the “Uncanny Valley” [9], some studies showed that humanoids that are too much similar to humans [10, 11] but also robots with a high mechanical appearance [12, 13] tend to be negatively perceived; whereas other recent studies brought solid and daring proofs that invalidate the hypothesis of the uncanny valley and the common reference to it as a general truth [14]. Most studies [10, 15, 16] agree on the fact that further research is needed to better understand and determine which aspects and degrees of similarity and likeability are required in order to enable more empathic and intuitive Human–Robot Interaction (HRI).

The way a “socially adapted” interaction builds up and the way social acceptance and social well-being occur are difficult to comprehend in a human-human interaction, knowing that what might be acceptable or satisfying for one individual is likely to be differently perceived by another [17]. Few social experiences happen without experiencing some ambiguity or ambivalence, or even strangeness, especially in a first encounter. When it comes to human-robot interaction, questions of social acceptance and of “natural” and successful interactions seem more crucial since the difference between humans and robots is fundamental and ontological. The quality of such interactions depends strongly on the robot, not only on its appearance, but also on its abilities, features, and autonomy degree. The interaction quality depends also on the humans’ perception and appreciation of the robot and their readiness to adapt to it, thus taking its abilities and limitations into account and compensating for them in order for the interaction to happen [7, 18].

Robots are gradually appearing in the society and their presence in homes, schools, hospitals, companies, and museums is still new for humans. To get a closer, more subjective, perspective on the human experience of encountering and interacting with a robot for the first time, we propose to use the notion of the familiar as an affective state. When working on social acceptance and socially adapted robot behavior, numerous reviews in HRI use terms such as “to familiarize with”, “familiarity” or “familiar”, all generally referring to what is known, well acquainted, often seen, or to what becomes known and usual due to repeated exposure and habituation. Nevertheless, we think that beyond its common uses as an adjective or verb to qualify what seems known or habitual, the familiar can be experienced as an affective state, related to specific emotions and thoughts. Furthermore, we believe that experiencing an affective state of familiarity might have interesting impacts on the human involvement and readiness to interact during an encounter with a robot. We are interested in the way people would “feel familiar” with a robot when meeting it for the first time or when having to interact with it as a partner for the first time. Considered from a qualitative point of view, “the familiar” taken as an affective inner state is a notion that lacks definition and precision. However, it is possible from a quantitative point of view, to address it and measure it precisely by asking participants to assess how familiar they felt with a robot at different points of a first-time short interaction with it (we used here the Likert scale, e.g. “You felt familiar with the robot in the beginning of the interaction” or “You felt familiar with the robot all along the entire interaction”). We believe that investigating the reliability of considering familiarity as an affective state is a challenging thesis. If validated, it would provide a comprehensive way to access the human experience during HRI as we believe it to have a major impact on the interaction quality and efficiency. The familiar state, once better understood, could be used as a reliable indicator for measuring human behavior when sharing a common space and interacting with a robot.

We aim to gradually work on bringing more precision and a better understanding to the conceptualization of the “familiar” through different studies using humanoid robots. In the future, what we might learn about the familiar state will probably inspire the way robots are conceived, possibly enabling them to recognize in human partners states of familiarity or unfamiliarity, thus adapting their actions towards a more familiar presence. Therefore, we started an innovative study [19] investigating directly the familiar and aiming at better defining it in relation to other important concepts in HRI such as anthropomorphism or the robot’s social skills. Our study is meant to be lead to in a cross-disciplinary collaborative approach involving robotics (mainly work on motion) and humanities (mainly psychology and psychoanalysis). Though very rare studies in the robotic field explicitly refer to psychoanalytical concepts [2] we find inspiration, among other sources, in some of these concepts and believe in the original perspective brought by an interdisciplinary approach to HRI studies.

We do not know of prior studies directly addressing the familiar as an affective state, particularly in HRI. Therefore, the novelty our study tries to bring is, in the same time, weakened by the lack of prior solid references which would have given a reliable structure to work within. Nevertheless, we connect our experiment to other works which tackle ideas and concepts related in many ways to it and which might enable us to show that HRI studies would clearly benefit from better understanding and defining the familiar. Our hypotheses do not arise from prior works but from general observations and questions we had when reading and reflecting on HRI studies and on the use of robots in different environments. Our hypotheses are also formulated based on our main concern which is starting to identify which aspects of an interaction ought to be considered when it is about evaluating and measuring the familiarity of an interaction. We are tackling what could become a main field of theory and research. Its freshness may also make it vulnerable and vague.

In particular, we aim at:

  1. 1.

    showing that different variables essential to HRI are strongly connected to the familiar, which encourages us to work, in future studies, towards grouping these aspects under the frame of the familiar (e.g. experiencing the familiar relies on certain levels of: pleasure, comfort, safety, predictability, sociability…) thus possibly assessing different levels of an interaction quality through the familiar,

  2. 2.

    testing, on the methodological level, the interest and efficiency of mixing different data (observable reactions, answers to a questionnaire and motion capture data) in bringing a more original and reliable point of view on the participants’ experience and reactions.

In our previous study on the familiar state [19], using the same experimental set-up and procedure described here but testing different hypotheses, results have shown that the familiar state can be experienced in a new and unexpected situation, during an interaction with a humanoid robot. These results underlined strong connections between experiencing an intense state of familiarity and perceiving the interaction as comfortable, secure and pleasant. Also, exploring the familiar state experienced while interacting with a robot cannot but be dependent on the human appreciation and perception of the robot they are in contact with. Therefore, we investigated the participants’ perception of the robot NAO which showed to be clearly positive (mostly positive adjectives were highly scored (Fig. 6); the negative adjectives have all very low scores); also we studied the possible relations existing between the appreciation of the robot and the intensity of the familiar state. This showed for half our participants that the more sociable the robot is perceived, the more intense is the familiar state experienced.

In the current paper, we explore the possible associations existing between experiencing familiarity, perceiving the sociable character of the robot proactive partner and the adequacy and intensity of the human response to its engaging actions depending on these actions’ target: strictly social vs. useful and practical.

2 Theory and Related Works

2.1 The Familiar in HRI: A Positive and Sometimes ‘Strange’ Experience

Joy pointed out [20], in the human attitude towards the new, what he called a bias towards instant familiarity and unquestioning acceptance. The recent years have showed an increasing interest in questioning and exploring human acceptance of robots. Nevertheless, the familiar as a topic has been cited or evoked in HRI studies mostly in indirect ways. For instance a study conducted on the implementation of a conversational robot in an elderly care center [21] showed that it is possible to help humans interact with the robot in a way that is familiar to them even though the robot itself was not familiar; the reference to the familiar here was not more explicitly explained or furthermore explored.

One source of inspiration for our work is the “Unheimlichkeit Gefühl” [22]—its translations vary: “The uncanny”, “feeling of strangeness”, “incredible familiarity”—theorized in psychoanalysis and later on in philosophy notably in relation to the theme of the double. The “Unheimlichkeit Gefühl” together with Jentsch’s elaboration of it [23], has inspired Mori’s concept of the “Uncanny valley” [9]. Beyond that, it is the subtle ambivalence it brings to what is known or unknown, new or acquainted that interests us: indeed this concept describes a bizarre feeling of strangeness, a feeling of being uncomfortably familiar, experienced when encountering a person or an object that seems familiar yet foreign and new at the same time. This ambivalent association of the strange and the familiar within a feeling that is triggered by a new situation but still draws from past experience can be useful when working on the interaction between a human and its artificial humanoid double.

In our first study [19] we were able to show the coexistence of moderate to high familiarity with feelings of strangeness and thoughts describing the interaction as being “absurd”. We also proved the familiar to be associated with positive and pleasant affective and mental states when interacting with a robot. The familiar was strongly correlated with a secure, comfortable, meaningful, and easy interaction. Further analysis showed in our second study [24] that the more humans experience the familiar during an interaction with a robot, the more they are prone to react to its engaging actions.

2.2 The Familiar Experienced During a Real and New Encounter

In this research we chose to consider the human perception of the robot as well as the human experience of collaborating with the robot in the frame of a real first encounter. Some rare participants might have met with a robot in the past, but the scenario of the interaction and its environment are likely to be completely new to them. We believe that focusing on the first encounter, rich with spontaneity will give a more genuine perspective of the interaction, a perspective that is still not reviewed/corrected by more elaborated cognitive processes or by habituation. Also, research based on Lorenz imprinting theory [25] proved the influence of first and early experiences on shaping social behavior. In our research, we are interested in the familiar as a state that may be experienced in a relatively limited time frame and may occur during new encounters, drawing possibly information from past experience but mainly forming itself in the present. This differs fundamentally from the “familiarity principle” also known as the “mere exposure effect” [26, 27] which focuses on familiarity built up through repeated exposure. As for our idea of a “real” encounter, it questions the generalization of results of experiments which isolate a fraction out of what would be an entire interaction in real life. By “real encounter”, we mean a situation where humans are able to freely experience, on a subjective and affective level, an encounter with a robot. The fact that the robot’s intervention is not announced and is therefore unexpected, the fact that this particular robot is new to them (at least “meeting it in real life” is new to them) and that no instructions were given to direct the human attitude and reactions towards the robot, contribute to support the idea of an encounter that is more “real” and genuine than artificial, as well as more open to spontaneous attitudes than to ones directly related to social expectations or experimental instructions. Of course, each individual has his/her own way to relate to explicit or implicit expectations and instructions, and an experimental setting can only try to ensure conditions which enhance genuine and natural reactions in the participant. Only few experiments have been conducted in real situations, and have succeeded in realizing valid and representative results [7, 28]. In our first study, [19] we successfully showed that moderate to high levels of familiarity were experienced during first-time interactions with a robot, within a short time frame. We also showed that this first contact with NAO had a clear and efficient impact on the participants’ perception of the robot.

2.3 Social and Affective Impact of Greeting Gestures: Social vs. Useful Interaction

Though movement has been from the beginning a core theme in HRI, it is only with more recent and few studies that its social and psychological impacts (e.g. greeting gestures) have been directly targeted. It has been shown that embodied non-verbal interactions are fundamental to regulate human-human social interactions [29] and that simple social conventions such as daily greetings can have a strong and direct impact on the perceptions of others’ feelings thus playing an important role in maintaining social ties [21]. Daily greetings performed by a robot in an elderly care center [21] proved to bring positive effects like pleasure, comfort and interest to the elderly. Our experimental situation begins and closes, as in real social situations, with welcoming and goodbye (non-verbal) gestures performed by the robot.

In our first study [19] we showed that the social gestures, more precisely the greetings performed by the robot, promote its polite and sociable character, two aspects that were strongly associated. We also proved for half our sample that the more participants perceive the robot as being sociable, the more their interaction with it is highly familiar and associated with positive pleasant affective states. As for the adequacy of the human response, it was clearly higher regarding the robot engaging action to accomplish a useful joint-task than it was regarding its engaging gesture to socially interact.

2.4 Unpredictability in HRI, Anthropomorphism and the Familiar

Studies [30–32] show that humans tend to draw inferences about the robot’s mental states, abilities, and personality in a way going clearly beyond its observable actions. A recent research [33] showed that both the robot’s behavior and appearance are important in influencing its perception by humans, but when behavior and appearance are contradictory, the robot’s behavior is more powerful than the robot’s appearance in the perception of the robot as more machine-like or human-like. The tendency to draw inferences about the robot beyond its observable features, follows a psychological strategy to satisfy a need for control over one’s environment in order to reduce psychological stress [34] and favor a more pleasant and efficient HRI. Effectance motivation, the third of the “Three-Factor Model of Anthropomorphism” [34, 35], represents one of the key psychological determinants of anthropomorphic inferences, its function being to resolve feelings of uncertainty and maintain or re-establish a feeling of predictability of one’s social environment. Our second study [24] showed that the more participants understand and make sense of the robot’s engaging actions while interacting with it, the more they react adequately to these actions. Results also showed for half our samples that the more humans understand and make sense of the robot’s actions while interacting with it, the stronger they experience it as familar.

Our study features an unannounced and unexpected interaction with a robot, possibly perceived as an unpredictable agent for some of the participants. This interaction might probably generate surprise and uncertainty at least, also possibly excitement and/or stress, depending on the participants’ personality and previous exposure to robots. A recent study [36] provided the first evidence for the fact that unpredictability of a robot’s actions does not necessarily lead to less acceptance or less liking; and that anthropomorphism, by its effectance motivation factor, facilitates HRI by increasing people’s ability to subjectively make sense of a robot’s behavior. Results of our first study [19] showed moderate to high levels of experiencing the familiar to be strongly correlated with high anthropomorphization of the robot.

2.5 ‘Useful’ vs. ‘Hedonic’ HRI

A recent study [3] explored the differential effects of hedonic vs. utilitarian robots. Results revealed that hedonic robot users perceived more enjoyment than utilitarian robot users, whereas utilitarian robot users perceived more usefulness and ease-of use than the others. These study results also indicate that individual differences play a significant moderating role on user attitudes toward hedonic and utilitarian robots. The authors of this study strongly suggest to robot developers and marketers to seriously consider labeling the robots as hedonic or utilitarian, and to also consider users’ individual differences in order to improve the benefits of human-robot interactions. The functional dichotomy between pleasure and utility, usually applied to most products [37] may be applied to robots [38] that could then be considered as hedonic (pleasure is emphasized) or utilitarian (practicality is emphasized) [3].

Of course, the dichotomy between pleasure and utility is different from the one that might exist between what could be considered by users as ‘useful’ in a robot’s behavior and what might be perceived as strictly ‘social’ with no clearly identified practical finality. But this study certainly underlines users’ tendency to have subjective expectations and therefore different attitudes as well as different emotional and mental experiences when interacting with a robot. These variations in users’ preferences have consequences on their perception of the robot and on their attitudes towards it. Based on these results, studying the impact on the participant’s response to social behavior displayed by the robot (A) vs. a useful task-oriented action with practical effect (B), seems an interesting theme to explore. Distinguishing between social and useful at the experimental level would enable us to learn more about social acceptance and efficiency in HRI, as well as about the presence or absence of distinction and categorization in humans’ mind, between socially oriented or usefully oriented interactions with a robot.

As previously introduced, this study aims at learning more about the familiar state in its relation to a satisfying, efficient and reciprocal HRI. Therefore, our research relies not only on testing hypotheses but also on analyzing and reflecting on collected data provided by the experiments. Based on the sources and allegations cited, as well as on our preliminary results, we hypothesize the following:

  • (H1) Experiencing a high level of familiarity when interacting with a robot co-occurs with high levels of adequate responses to its proactive engaging actions (H1/A social actions; and H1/B utilitarian actions).

  • (H2) The change of behavior of the robot from one participant to the other regarding a similar interaction has an effect on the intensity of the familiarity experienced during this interaction as well as on human readiness to interact with the robot afterwards.

  • (H3) The higher the appreciation of the robot’s sociable character is, the more intense is the human response to its engaging actions (H3/A: social actions; and H3/B utilitarian actions).

  • (H4) The more the participants tend to engage in and to react adequately to the robot in a useful joint task initiated by it, the more they tend to respond to the robot’s social solicitations (greeting gestures).

3 Methodology and Experiments

3.1 Participants and Design

Twenty pairs of students, 40 students in total (14 women, 26 men), were recruited on the campus of Tokyo Univ. of Agriculture and Technology, and volunteered to participate in a study on the perception of robots and human robot interaction.

Though previous exposure to robots was not controlled when recruiting the participants, candidates were mainly students from agriculture, biology, and chemistry departments. Furthermore, having seen a robot in videos or having been exposed to a robot does not necessarily mean exposure to a humanoid robot or to the same robot used in our experiment. Even persons who have some experience in handling robots have mainly worked ‘on’ the robot or on parts of the robot. Having seen or technically intervened on a robot does not say how a person would react once in an interactive relation ‘with’ a proactive robot. The interactive and relational (and possibly reciprocal) dimensions involved are more subjective than rational and a person who might be used to manipulating robots might, once the robot manifests and presents itself as an interaction proactive partner, not behave with the same comfort or detachment or obviousness.

3.2 The Robot

NAO (Aldebaran Robotics) is a 57-cm tall commercial humanoid robot. Its body has 25 degrees of freedom (DOF) whose key elements are electric motors and actuators. We used the programming software delivered with the robot to control it.

3.3 Experimental Set-up and Procedure

(1) The experiment involves a triad: NAO and two participants (X and Y) at a time. The participants are only told they are invited to answer a questionnaire on the perception of robots and human robot interaction. They are also informed that the set is filmed and that sensors will be placed around their head and wrist for motion capture. They do not know about the robot’s intervention and their possible interaction with it. Once the experience starts, there is no further intervention from the staff. Participants are not instructed about what they ought to do, it is all upon their own judgment. The scenario’s duration involving NAO is about 1 minute. Then, the questionnaire requires 5 to 10 minutes to be filled.

(2) The experiment set-up consists of a rectangular area limited by colored boards. It is furnished with a carpet, a low table equipped with pens, and two cushions directly put on the floor on each side of the table, providing therefore a comfortable Japanese-style ambiance, closer to a cozy space rather than an anonymous lab. Also when seated on the cushions, participants are positioned on a low level which, given NAO’s small height, enables face-to-face contact (Fig. 1 and additional video material).

Fig. 1
figure 1

The experimental setup when X and Y exchange the envelope with NAO

Full size image

(3) The experiment starts with NAO entering the room, facing the table and holding in each hand an envelope with the word “Questionnaire” obviously written down on it. NAO walks towards the participants, then stops a few centimeters away from the table and greets (Social-oriented behavior) them by bowing (its head bends with a slight forward bending of the upper torso). Then NAO turns towards participant X sitting to its left and extends its left arm holding the envelope in their direction (Useful-oriented behavior). After a few seconds, its fingers release tension and the envelope is then ready to fall down in the participant’s hand or on the floor, depending on the participant’s reaction. Then NAO turns towards participant Y, extends its right arm holding the second envelope in their direction (Useful-oriented behavior). NAO is slightly more distant from participant Y than it was from participant X so in order for the envelope exchange to happen, Y has not only to extend his/her arm like for participant X, but in addition to lean forward and reduce the distance from NAO. Another difference from the interaction with X, is that NAO will now keep the envelope 4 seconds between its fingers before releasing it: NAO is hesitating in handing the envelope. Having delivered both envelopes, NAO waves goodbye with its right hand (Socially oriented behavior), its position facing participant Y more than X, then turns around, walks back towards the door, its mission completed. Participants are free to start filling the questionnaire anytime after receiving the envelope. We chose to ask the participants to answer the questionnaire at the end of the interaction and not after each key-moment. We wanted the interaction to be uninterrupted, in order to remain as natural as possible. We also wanted the participants to be able to behave as spontaneously as possible without being disturbed by the experimenters’ interventions and frequent pausing. Asking them to give their perceptions and opinions once the interaction with the robot ends was the only possible choice, especially that the whole situation lasts for one minute and that the questionnaire is filled directly after the robot gets out of the room. At that time, participants’ memory is still fresh and the short time they needed (5 to 10 minutes) to fill the entire questionnaire supported reporting spontaneous thoughts as faithful as possible to their experience, rather than a feedback possibly distorted by reflection or by experimenters’ interventions.

(4) Our experimental situation was created with the intention to avoid to the maximum a lab-situation mimicry of a social interaction, thus allowing a real social situation to occur. Having two participants at a time might bring some uncontrolled variable, but it brings at the same time a feeling that this is not really an experiment. Candidates in a pair know each other and chat lightly while waiting for the questionnaire. The whole situation, which has a beginning and an end, punctuated by greeting and goodbye gestures performed by NAO, allows no anticipation or prepared reactions for the participants as well as puts them directly in a socially significant atmosphere. Furthermore, the robot has a real function and an essential task to accomplish which consists in handing out the questionnaires to them. The encounter, and later on the interactive collaborative action that could possibly happen between it and the two persons have real chances of happening or not, since the reactions and choices made by humans are not dictated by an obligation to follow experimental instructions. The only must formulated to participants was, as stated before, to answer the questionnaire. Finally, the robot shows a slightly different behavior with each participant, which allows limiting its repeatability and predictability (and mechanical functioning as a machine). It also allows seeing how this difference of behavior would be interpreted by both participants: social preference, teasing, program dysfunction, or other, and how it might affect their reactions and their perception of the robot as well as of the whole interaction.

This situation is a particular illustration of what could happen in the future in public (or even domestic) spaces where the robot has precise tasks to accomplish and is prone to interact with different users that are not inevitably aware of its intervention and are relatively free to interact with it. Another particularity is that NAO is not presented here as an experimental object but rather as having a proactive role which makes it a potential interaction partner, thus reducing their comfort position since an interaction is proposed to them and giving more emphasis to their decision-making and reactions (or absence of reactions).

The choice of two participants at a time is motivated by two main reasons: First, allowing NAO to manifest different—possibly perceived as ‘subjective’ and still not predictable—behaviors and not one perfectly repeated action, thus permitting different more complex and more global perceptions/interpretations regarding NAO and regarding the interaction with it from the participants’ side (e.g. NAO has a preference for the other participant, NAO is not well functioning as a machine, NAO likes to tease). The second reason lies in the possibility for each participant to be likely influenced and nourished in their experience and actions by the different aspects of NAO’s behavior but also by the behavior and expressions of the other participant, thus allowing various obvious and less obvious interactions to occur. Furthermore, we felt that the stress that might be generated from the unpredictable factors proper to the situation as well as from a close encounter with a robot might be counter-balanced or at least eased by the complicity and shared-experience of being two persons facing the robot (all the pairs were recruited together and consequently knew each other). Overall, this choice makes the situation’s interactive potential richer, more surprising and more challenging for the participants.

3.4 Protocol Design Choices

Participants were randomly assigned to one of two sitting positions in a 1(X) × 1(Y) between-subjects design. Each pair of participants experienced the same encounter with NAO, the only variation being its changing behavior when handing the envelope to the participant: Smooth (with X) vs. Resisting (with Y). The video in the supplementary materials shows a typical encounter.

Participants range in age from 19 to 35 years (Participants X: M=24, SD=3.5/Participants Y: M=23, SD=1.7). The gender was not controlled in assigning the position, resulting in: 11 males seated in X, nine females seated in X and 15 males seated in Y and five females seated in Y.

We deliberately chose feed-forward control of the robot so that each pair of participants’ experience the same encounter with NAO, the only variations being brought by its changing behavior when handing the envelope: Smooth (with X) vs. Resisting (with Y). Resisting behavior refers to the fact that NAO stands slightly farther from participant Y than it was from participant X when delivering the envelope to Y. Also it refers to NAO keeping the envelope for four seconds in its fingers before releasing it, whereas the release when facing X is immediate.

3.5 Data Collection and Data Analysis

In this study, it was important to use distinct but complementary tools in order to have a more accurate and faithful access on what was really experienced by the participants as well as to limit ambiguity in the results.

(1) Each experiment session is video recorded using two stable cameras: One is filming the set from behind and gives images of the robot entering the set and of its interaction with the participants. The other is facing the participants, and providing images of their movements and facial expressions. This tool is particularly used to collect data on the participants’ verbal and non-verbal behavior, especially their facial expressions (laughter, surprise, eye contact between the participants…), but moreover to collect data on their reactions (answer back or not) to NAO’s greeting and goodbye gestures as well as their reactions when NAO extends its arm to hand them the envelope. The recorded data is reinforced with observation notes taken by the psychologist of our team.

(2) Two IMU (Inertial Measurement Unit) sensors (Fig. 1) are used for each participant (one fixed on the forehead to capture the head and upper torso movements; the other on the arm—the right arm for X participants and the left for Y participants, which are the closest arms to the robot’s position and most likely (from our observations on a pilot study of 20 candidates) to be used by the participants to fetch the envelope). The IMU sensors measure the longitudinal accelerations and the rotational velocities around 3-axes. Thus, more discrete micro-movement data is recorded, giving us another level of information regarding the participants experience and reactions (proactive behavior, readiness to interact, absence of action) to NAO’s greetings and attempt to engage them in an interaction. Data for two pairs of candidates are unavailable.

(3) The questionnaire proposed to the participants consists of three parts/methods addressing different topics but also sometimes the same topic considered from different perspectives, as can be seen from the appendix. The questionnaire is written in Japanese to avoid possible confusion and misunderstanding in the nuances that an insufficient level of English could bring. The first part uses the 7-point Likert scale. Participants are asked to indicate their feeling/opinion about series of statements, 1 meaning “strongly disagree”, 7 “strongly agree”. We added 0 for “Irrelevant statement” to allow a more precise expression. Statements are, respectively, about: NAO; Interacting with NAO; Understanding and interpreting NAO’s actions, personality and behavior; Experience of the familiar during the different moments of the interaction; Earlier exposure to robots. To assess participants’ perception of NAO, we presented them with a list of personality traits that mainly reflect human nature (civility, morality, emotions, cognitive abilities, temperament); the other traits can for one part apply to animals and for the other part likely to be perceived as machine-like (“efficient”, “useful”). The second part consists of Multiple Choice Questions regarding the nature of earlier exposure to robots; Understanding NAO’s action when it handed them the envelope; Deciding how to react to NAO; Possible Confusion when deciding how to react; Perceiving NAO as familiar in relation to its appearance, movement and behavior. The third very short part consists of two open-ended questions, about describing NAO and about describing the interacting with NAO in the participants’ own words. In the present study we principally focus on the answers about the familiar, earlier exposure, and the ratings of NAO’s sociability and usefulness. The measurement of the familiar experienced by the participants consists in a personal evaluation (from 0 to 7 on a Likert scale) of one’s affective experience at different moments of the interaction.

From the IMU data, only three components of the rotational velocity ω are post-processed to obtain two types of information. The first one is called “the intensity of the movement” Im which corresponds to an integration over the time of the interaction motion, which we finally normalized to adjust to the Likert scale for the rating of the intensity ranging from 0 to 7 as shown by (1), where ts is the starting time of the considered movement, te is the ending time.

$$ \mathit{Im} = 7 \times \frac{\int_{ts}^{te} \Vert \boldsymbol{\omega} \Vert dt}{\max_{\mathit{all}\ \mathit{movements}} ( \int_{\mathit{ts}}^{\mathit{te}} \Vert \boldsymbol{\omega} \Vert dt )} $$
(1)

This parameter was computed for the motion during the exchange of the envelope, and for the goodbye greetings when they occurred.

The second uses the feature vector obtained by forming a vector of the columns of the matrix M given by (2), and its PCA decomposition to find tendencies by cluster formation in the motion data [39], as can be seen from Fig. 2. Density-based clustering is used here to find the clusters. The most dense cluster is isolated in the right corner of the figure. Its center is computed and the Euclidean distance and the direction from the horizontal for each trial data to this center are calculated to give information about similarity in the motion data of each participant when taking the envelope.

$$ M = \frac{1}{T}\sum_{k = 1}^{T} \boldsymbol{\omega} [k]\boldsymbol{\omega}^{\mathrm{T}} [k - 2] $$
(2)
Fig. 2
figure 2

Motion data decomposition in feature vector and PCA during the envelope exchange

Full size image

We calculated the descriptive statistics related to the participants’ responses to the robot’s engaging actions to interact. We also calculated Chronbach’s α reliability for certain items in the questionnaire and the descriptive statistics based on their answers to all the parts of the Questionnaire except for the open-ended questions part from which answers were used when clarification was needed. The Chronbach’s α reliability tests are given in Table 1. They indicate that the questionnaire is valid and has a good internal reliability. Evaluating dependencies between variables (T-test, Pearson correlation and Spearman correlation) of the questionnaire, of the participants’ observed behavior and of the measured motion, showed interesting and strong associations between many criteria, and also revealed the absence of any correlation between some others. The most important results are summarized in Tables 1, 2, 3 and 4.

Table 1 Chronbach’s α reliability test for selected items in the questionnaire
Full size table
Table 2 Significant Spearman correlation results between two parameters: Reactions to NAO’s greeting and NAO’s handing the envelope, and associated 2D-plot
Full size table
Table 3 Significant Spearman correlation results between two parameters: Reactions to NAO’s actions and Familiarity while interacting
Full size table
Table 4 Significant Pearson correlation between the distance to cluster from motion data when exchanging the envelope and familiarity when taking the envelope, and associated 2D-plot
Full size table

4 Main Results

4.1 Participants’ Responses to NAO’s Proactive Socially Oriented and Usefully Oriented Actions

When comparing X participants’ and Y participants’ respective reactions to NAO’s greetings, before exchanging the envelope (greeting hello) as well as after the exchange, at the end of the encounter (greeting goodbye), we found the difference in their response to be not statistically significant, thus being due to chance. T-test results showed similar lack of statistical significance when comparing X participants’ reactions and Y participants’ reactions to NAO handing them the envelope, and when reacting to NAO’s greetings, as well as when comparing X and Y participants’ movement intensity (Table 6) when accomplishing each of these two actions (greeting back goodbye and taking the envelope from NAO). No statistical proof was found to assert that X and Y reacted differently—frequency of adequate response and movement intensity when reacting—to NAO, nor to assume that the difference of behavior showed by NAO when handing the envelope to X and Y participants had a relevant impact on their respective reactions and on the movement intensity measured for these reactions (to greetings and to the envelope exchange).

Based on this lack of statistical significance regarding the impact of the robot changing behavior on X and Y movement intensity when reacting to NAO, we considered all the 40 participants as one group and compared between their movement intensity when greeting NAO back goodbye (M=0.4, SD=0.8, N=31) and their movement intensity when taking the envelope from it (M=2.2, SD=1.4, N=36). The movement intensity when exchanging the envelope is clearly higher than when greeting back goodbye and T-Test results proved this difference to be highly significant (P≤0.0001; CI=95 %; t=6.1; DF=65; SED=0.3).

High and positive correlations (Table 2) were found between the participants’ readiness to adequately respond to NAO’s action to achieve a joint and useful task together (handing the envelope) and their readiness to respond to its hello as well as its goodbye greetings. This strongly suggests that readiness and adequacy to interact socially and readiness and adequacy to interact usefully are dependent, even though the tendency to interact usefully was clearly higher than to interact socially for both participants.

4.1.1 Social HRI: Responses to NAO’s Greeting Gestures

45 % of X and 35 % of Y answered back NAO’s hello greeting. 30 % of X and 35 % of Y answered back NAO’s goodbye greeting. Less than half X participants greeted NAO hello and their number fell to 35 % out of 20 participants when it was time to greet NAO goodbye. As for Y participants, the same number, almost one third of them, greeted NAO hello and goodbye (Fig. 5).

4.1.2 Useful HRI: Responses to NAO handing the envelope

Without having been informed about the interaction or about what they ought to do, 80 % of X and 85 % of Y were proactive towards NAO’s arm movement to hand them the envelope. Of course, Y had seen NAO already performing the same movement with X participants’ which might have facilitated participants Y participants’ reaction. A very strong positive correlation between X and Y responses to NAO handing the envelope (Table 2) was found, which means that the more X reacted adequately to the robot’s engaging action and took the envelope, the more Y had the same tendency to respond to NAO’s action and react accordingly. This finding suggests that there might be an influence of X on Y, possibly a Chameleon effect. Nevertheless, the interpretation of this dependency between the two participants’ behavior was not addressed at this stage of the study.

Considering the novelty of NAO’s action (resisting before handing the envelope to Y), the large number of participants who adequately reacted is to be noted. Only 5 % were passive while NAO was still holding the envelope and waited for it to fall on the floor to take it (10 % of X did the same), whereas 10 % of Y (and 10 % of X) were totally passive and waited, after NAO went out of the room, for the experimenter to come and give them the envelope.

4.2 Participants’ Perception of NAO’s Sociability and Usefulness in Relation to NAO’s Changing Behavior and to the Intensity of Their Response (Movement) to It

As stated before, we investigated the participants’ perception of NAO in order to study the possible correlations existing between some of its aspects and the familiar state experienced by the participants as well as their proneness and adequacy in responding to the robot. The participants’ general perception of NAO shows to be positive since mostly positive adjectives are highly scored to describe it (Fig. 6) and since the participants’ answers to open-ended questions showed a positive enthusiastic appreciation of it. Mostly, negative adjectives have very low scores for both X and Y, and the lowest score is obtained for both participants for ‘hostile’ (X: M=1.3, SD=0.7, SEM=0.1, Y: M=1.4, SD=1.5, SEM=0.3). The data analysis of the participants’ movement during the envelope exchange (Fig. 2 and Table 4) showed a strong correlation between perceiving NAO as hostile and the distance to cluster. In other terms, the more the participants perceived NAO as hostile when it was handing them the envelope, the more their reaction was far different (and the distance to the cluster larger) from the dominating tendency of the group that is to react adequately and take the envelope from NAO (Table 3).

‘Sociable’ was amongst the most highly scored adjectives to describe NAO (X: M=4.9, SD=1.8, SEM=0.4, Y: M=4.8, SD=2.1, SEM=0.5) whereas ‘Useful’ obtained a medium score (X: M=3.7, SD=1.9, SEM=0.4, Y: M=4.1, SD=1.6, SEM=0.3). Paradoxically, X and Y ratings for ‘Sociable’ and ‘Polite’ are higher than for ‘Useful’, even though they reacted more frequently to NAO’s useful handing of the envelope than to NAO’s social greetings. Still, results showed a positive Spearman correlation between describing NAO as highly sociable and participants’ movement intensity (motion data) when greeting goodbye, which means that the more participants found NAO to be sociable, the more intense was their movement when greeting it farewell at the end of the encounter (Table 5). No similar dependency was validated regarding participants’ movement intensity (motion data) when taking the envelope from NAO (no statistically valid correlations were found between the motion intensity and NAO’s useful-oriented action).

Table 5 Significant Pearson Correlations between the Intensity of motion data when greeting goodbye and NAO’s Sociable character, and associated 2D-plot
Full size table
Table 6 Movement intensity results (normalized to adjust to the Likert scale from 0 to 7) for X & Y when greeting NAO back Goodbye & when taking the envelope
Full size table

More generally, when comparing X participants’ and Y participants’ ratings for the following positive adjectives to describe NAO: polite, sociable, interesting, useful, efficient; as well as for the following negative adjectives: stupid, hostile, confusing, clumsy (this adjective which was amongst the highly rated, might not have been considered as negative by the participants, but still it is not a compliment); T-test results showed the difference between them to be not statistically significant. NAO’s difference of behavior from participant X to participant Y, during the envelope exchange, seems not to have had an impact on the participants’ perception of NAO and not to have been negatively interpreted by them.

The main impact of NAO’s changing behavior on the participants perception of it, is observed for ‘unpredictable’ whereas Y found NAO to be more unpredictable than X (X: M=3.6, SD=2.2, SEM=0.5, Y: M=3.8, SD=1.4, SEM=0.3); as well as for ‘interesting’ which was the most highly rated adjective for both X and Y, with a higher rating for X (X: M=6.5, SD=0.9, SEM=0.2, Y: M=5.7, SD=1.3, SEM=0.3). Indeed, T-test results for ‘unpredictable’ (P=0.0006; CI=95 %; t=3.7; DF=38; SED=0.5) showed that the difference between Y and X ratings is statistically extremely significant. Also, T-test results for ‘interesting’ (P=0.029; CI=95 %; t=2.3; DF=38; SED=0.3) showed that the difference between Y and X ratings is statistically significant.

4.3 The Familiar: About NAO and About Interacting with It

Most participants experienced medium-to-high familiarity while interacting with NAO (X: M=4.9, SD=2.0, SEM=0.4, Y: M=4.9, SD=1.6, SEM=0.4). As for the different moments of the encounter, most participants reported feeling familiar with NAO all along the interaction, gave the highest scores for “the envelope exchange” moment and for the end of the encounter (Fig. 4).

In this study, data analysis of the participants’ movement during the envelope exchange (Fig. 2) showed that: the more the participants felt familiar when exchanging the envelope with NAO, the more their reaction was to be close/similar to the dominating tendency of the group that is to react adequately and take the envelope from NAO (Table 4); and the less they were to display a response that was different and distant from the main tendency showed by the cluster.

The series of high positive Spearman correlations (Table 3) obtained for both participants between experiencing the familiar while interacting with NAO and two main moments of the interaction, meaning: NAO handing the envelope and NAO greeting goodbye; showed the familiar state and the participants’ reactions to NAO’s utilitarian and social actions to be strongly associated.

When comparing X participants’ and Y participants’ ratings of experiencing familiarity when NAO handed the envelope as well as when NAO greeted goodbye afterwards, the difference in their responses showed to be not statistically significant, thus due to chance. T-Test results failed to validate the influence of NAO’s changing behavior from X to Y on the intensity of the familiar state experienced by both participants.

4.4 Participants’ Earlier Exposure to Robots in Relation to the Novelty of Their Interaction with NAO

40 % of X reported having never been exposed to robots, 45 % said to be familiar with robots from movies and literature. 40 % of Y reported having been exposed at least once to a real robot, 55 % said to be familiar with robots from movies and literature (Fig. 3). Earlier exposure to real robots was low for X (M=2.5, SD=2.1, SEM=0.4) and medium to low for Y (M=3.8, SD=2.4, SEM=0.5). T-test results showed that the difference between X and Y previous exposure to robots is not statistically significant. Also, most participants found interacting with NAO ‘New’ (X: M=5.6, SD=2.0, SEM=0.4, Y: M=5.1, SD=1.7, SEM=0.4). Even though the number of participants who had seen real robots was higher in Y than in X, and even though X participants rated “interacting with NAO is new” slightly higher than Y, T-test results showed no statistically relevant difference. This suggests that in spite of the difference (not statistically relevant) of exposure to real robots between X and Y, both experienced the interaction with NAO as new. This is very important for our study. Indeed, we are working on the familiarity as an affective state experienced in the frame of a new encounter with a humanoid robot and not on familiarity in relation to previous exposures and prior experiences that might make the interaction seem already known and/or usual, as well as expected for the participants.

Fig. 3
figure 3

Earlier exposure to robots and source of exposure: movies, books, real robots

Full size image
Fig. 4
figure 4

Familiarity at different moments of the interaction

Full size image
Fig. 5
figure 5

Reaction to the robot gestures

Full size image

5 Discussion and Conclusion

Generally, participants highly and adequately reacted to NAO’s engaging action to exchange the envelope in opposition to their low response to NAO’s greetings in the beginning and at the end of the encounter [19]. In spite of this notable difference of reaction, the participants’ readiness to socially interact showed to be strongly correlated for both hello and goodbye greetings to their readiness to adequately respond to a useful interaction -here the envelope exchange (H4 validated).

Furthermore, results from the previous stages of our study [19] and [24], showed strong positive dependencies between ‘experiencing the familiar for a human interacting with a robot’ and ‘reacting adequately to the robot’s engaging actions’, whether these actions were social gestures or gestures inviting to achieve a practical joint-task together. Nevertheless, high levels of familiarity failed to co-occur with high levels of adequate responses to the robot’s invitations to socially interact since few participants answered clearly back NAO’s greetings (H1/A: infirm). Only high levels of adequate responses to the robot’s gesture consisting in handing the envelope containing the questionnaire participants needed to achieve the experiment, were clearly observed (H1/B: confirmed); as the familiarity experienced by the participants at the envelope handing moment was very high. This might possibly explain the level of adequate responses.

These particular findings might of course be affected by the population size or by an uncontrolled variable such as participants’ previous exposure to robots. Yet these findings also indicate that participants were more prone to respond to the robot when its engaging action had a practical and useful target. Could this mean that participants were more interested in interacting usefully (taking the questionnaire to be filled) with the robot than they were in exchanging social signals which importance is more symbolic than practical and concrete? Or did the participants have a similar experience towards both the social and useful actions of the robot but were more inclined to take the envelope because they were instructed to answer the questionnaire it contained, and that they, consequently, needed?

In the first stage of our study [19], participants gave high scores to “NAO is sociable” and “polite” and were able to identify and appreciate the social meaning of its gestures as results showed that social gestures expressed by it played a major role in promoting its sociable character. Results proved more particularly for participants X that the more NAO was perceived as sociable, the more interacting with it was familiar and pleasant. Nevertheless, more than half the participants did not feel obliged to act towards NAO with equivalent politeness or sociability. Though their evaluation of NAO shows they thought it to be more sociable then useful or efficient (Fig. 6), most of them seemed more willing to engage in a target-oriented interaction than in a pure social interaction (only 30 % to 45 % of them greeted NAO back; smiling and nodding were not considered as adequate responses to NAO’s greetings). Studies have shown that though humans do respond socially to robots [40], it is probable that they will not exactly react to them as they would to other humans [41]. It is also important to keep in mind that participants were not instructed to interact with the robot. This might have given them as much freedom as discomfort. Another explanation could lie in the fact that participants’ comfort zone was disturbed by a close face-to-face with NAO. More generally, these results show the inter-individual differences related to how each person reacts to and copes with an unpredictable encounter with a robot and with the different actions displayed by it: Social vs. Useful. We might be needing more than enabling robots to perform socially engaging gestures to get to socially engage and interest human partners efficiently.

Fig. 6
figure 6

NAO’s evaluation

Full size image

This stage of the study brings a novelty through validating a strong correlation between the intensity of the gesture in those who answered back NAO’s greeting and the appreciation of NAO as being a sociable partner (H3/A: validated; H3/B: infirmed). The emotional and expressive dimensions that might be more involved in social interaction than in practical and usefully oriented interaction, could explain the validation of such intensity in the greeting movements even if these responses were less frequent than taking the envelope. In the first stage of our study, we only used results from the questionnaire for our data analysis and could not prove the hypothesis stating that “The more the participants perceive the robot as socially-engaging, the more they are likely to interact with it”. The light brought by the motion data analysis, in relation to the intensity of the movement, gives an insight on a dimension of the participants’ reaction that is not observable or possible to detect by other means. Movement intensity can be related to emotions, motivation and interpersonal variations. Our results, based on a combined analysis of motion capture and participants’ answer to a section of the questionnaire, are encouraging to explore furthermore the relations existing between uncontrolled physiological reactions and more reflected opinions during HRI.

The change of behavior displayed by NAO from participant X to participant Y seemed not to be interpreted as a preference or as a negative aspect in NAO [19]. Both participants reported high scores for a seductive, clumsy, funny, clever NAO and low scores for an efficient, stupid, hostile NAO; the differences between X and Y ratings were not statistically significant. The only (statistically) significant impact of NAO’s changing behavior is observed in the participants’ description of it as ‘unpredictable’ and ‘interesting’. ‘Interesting’ was the most highly rated adjective to describe NAO for both X and Y. But X found NAO clearly more interesting than Y. The exact reason of this difference is not certain: it is probably due to the fact that NAO was more precise and efficient in handing X the envelope which might have improved its interest and quality from X perspective. It also might have been influenced by the fact that more Y participants than X participants reported having seen once a real robot (not necessarily NAO) which might have moderated their appreciation of NAO’s interest in comparison to X. As for perceiving NAO as ‘unpredictable’, it seems only logical and fair that Y participants find it more unpredictable than X, since NAO’s changing resisting behavior happened while interacting with them.

More generally, these results show, as proven in [36] that unpredictability of a robot’s actions does not necessarily lead to less liking from its human partners. NAO’s change of behavior had an impact on specific characteristics of the robot in the participants’ opinion without affecting their general perception and appreciation of it: NAO’s resistance to Y participants seems to be more interpreted as a sign of clumsiness rather than a sign of hostility or stupidity. This also suggests that having seen two variants of NAO’s behavior regarding the envelope exchange seemed to have influenced both participants in forming a global perception of the interaction rather than an opinion only based on their one-to-one interaction with it.

A strong dependency, which interpretation was not furthermore explored in this paper, was validated between X and Y responses to NAO handing the envelope: the more X was prone to take the envelope, the more Y was to have the same behavior. This reinforces the probable impact of being two persons encountering the robot at the same time. And this also suggests that further work is needed to better appreciate the interest of experiments where more than a person interacts with the robot and to define the specificities of such a setting in comparison to a one-to-one HRI.

Most importantly, these experimental results failed to enable us to validate a possible effect of the robot’s change of behavior on the familiarity experienced by the participants and on their responses to the robot’s goodbye (socially engaging) gesture at the end of the interaction (H2 infirmed). Does this mean that the robot’s changing behavior did not really affect the participants or does it say that a human response to a robot depends on more complex variables than the ones directly connected to the robot’s features and efficiency? Could it be that the social dimension of the interaction was considered by the human partners independently from the practical joint-task one of the encounter? Obviously, comparing these results to the ones of an experiment where only one participant meets the robot at a time and where two controlled protocols with two different behaviors of the robot are tested, would be mostly useful to our study.

Most of the participants described the interaction with NAO as new and most of them felt familiar while interacting with it. Results proved that experiencing a high level of familiarity when interacting with a robot strongly associates with high levels of adequate responses to its proactive useful-oriented actions. This correlation suggests a possible impact of the familiar on the human readiness to adequately interact with a humanoid robot as well as on the success of this interaction especially when its aim is useful and practical-oriented. In addition, neither the intensity of the familiarity experienced during this interaction nor the human readiness to interact seemed to be significantly affected by the robot’s changing behavior.

As for the distinction between social-oriented and useful-oriented actions performed by the robot, it brings to light some interesting and surprising aspects of the human experience and response during HRI. Though the participants found the robot more sociable and polite than useful, and even though their readiness to engage in a useful joint task with it was strongly associated to their readiness to respond to its social solicitations, not only did they react more frequently to its useful-oriented action than to its social-oriented ones, but they also reacted to it with more intense movements. This difference of intensity could be seen as a sign of motivation and of putting more involvement in their reaction to the robot, as well as a sign of certainty when responding. On another hand, even if the movement intensity was lower when greeting NAO goodbye than when taking the envelope from it, our results showed that the higher the appreciation of the robot’s sociable character was, the more intense was the human response (movement) to its socially engaging actions. As for not finding a statistically valid association between describing the robot as useful and the intensity of the movement when taking the envelope from it, one possible interpretation could be that the lack of statistical significance might be due to the sample size knowing that the data for the intensity movement was not available for all the 40 participants. Another possible reason could be that the participant distinguished between considering NAO as generally useful (which explains the average rates ‘useful’ in the answers to the questionnaire), and finding it useful for them to react to it when its arm extended to hand them the envelope, since they needed the questionnaire. Having observed that NAO had mainly two kinds of actions (greeting and handing objects) in its repertoire and having witnessed two variants of the handing (smoothly vs. hesitating when handing) might have cast doubt on NAO’s general usefulness and efficiency.

All these findings support the interest of distinguishing between the robot’s actions and studying human response to them in relation not only to their perception and appreciation of the specific robot they are interacting with, but also in relation to their perception and representation of their own needs and objectives during the interaction (e.g. it might be useful for them to interact with a robot for specific reasons in a certain context and they still might judge the robot as averagely useful in general). Moreover, these findings lead us to draw a complex sketch of the relations and differences existing between what humans think and what humans do, and between their appreciation and liking of certain aspects of the humanoid robot partner and their reaction to it in regards to certain goals they want to achieve by interacting with it. The distinction made between “social” and “useful” interaction ought to be furthermore investigated as our results underline its interest and utility for understanding what makes HRI efficient and reciprocal.

This third stage of our research gave us more insight on the complexity of studying the human experience and responses to a robot partner during HRI. Some previous results were enriched and clarified by the current findings whereas some others were questioned. These uncertainties and complexities open up to further exploration and show, as much as validated hypotheses, that the familiar sketches a challenging path for studying the inner experience as well as the observable actions and reactions of a human interacting with a robot.

6 Limitations

We are aware that our study comports certain limitations. In particular a pilot study using a human instead of a robot, a larger participants sample as well as introducing “earlier exposure to robots” as a controlled variable in a future research, might allow us to get more accurate results. As mentioned in the introduction, the intercultural aspect should be taken into account. The study being conducted in a Japanese setting, our preliminary findings will gain by being compared to results from experiments conducted in other cultures, which will give us a better insight on the impact of intercultural variations. We are also aware that this study has been conducted with a specific robot: NAO. Its particular design features probably facilitate human acceptance and appreciation (most of the participants have reported finding NAO cute and seductive [19]). Conducting the same study with a different robot (e.g. taller, bigger, more mechanical looking) would also be of interest.