Introduction

The iCat is a robotic research platform that has been introduced by Philips to investigate human robot interaction with the aim to apply the results of this research in smart home environments. Its face has 11 actuators, one for each eyebrow and eyelid, one for the vertical eye movement, two for horizontal eye movement and four to control each corner of the mouth. These actuators enable it to show a wide variety of expressions, see Fig. 1.

Fig. 1
figure 1

Left The iCat has various sensors including touch, vision and sound sensors to perceive our physical world with. Right The iCat is able to show various facial expressions. From left to right happy, angry and surprise

Full size image

The neck and body movements are controlled by two DC motors. Input devices include a camera in the nose, a proximity sensor in the left paw and two microphones to record stereo sound. It also has multicolour LEDs in its ears and paws and a speaker as output devices. The fact that iCat lacks mobility is the reason that the focus of research is on aspects of human-robot interaction, such as believability.

The iCat is mainly controlled by “playing” animations, predefined sequences of actuator values. This precise control over motor positions results in more lifelike behavioural animations than traditional feedback control loops that are normally used in robots (van Breemen 2004). The animation tools and programming environment for the iCat are provided by the Open Platform for Personal Robots (OPPR).

In this article, we focus on the role of gaze behaviour for increasing the believability and likability of the iCat (and similar robotic devices). Gaze behaviours differ with respect to when, where and how somebody looks at something. When to look is investigated in studies where the function of gaze during social interactions is looked at. See for instance the research done in the AMI project (http://www.amiproject.org/).

When do we look at a speaker, when do we look away and what information do we convey by doing this? Where to look is based in the area of cognitive psychology and covers aspects such as which cues from the environment attract our visual attention. How to look has its roots in the biological and neurophysiological fields where the movements of the eyes and combined head movements are studied. As mentioned above, the main objective of this work is to increase the believability and likability of the iCat through enhancing the naturalness of the gaze behaviour. In this study, we have focussed on improving the way the iCat looks (that is the how question, and to a minor extent the where question) and measured what effect this more natural behaviour has on the perception of the iCat by users.

In order to measure believability and likability, we should have clear definitions of these concepts. Both criteria that are used to evaluate the gaze behaviour system are abstract concepts that cannot be measured absolutely. The evaluation methodology will therefore be a user test comparing a person’s perception of an iCat with gaze behaviour to one without it.

A useful definition of believability is given by Dautenhahn (1998): because humans seem to be naturally biased towards identifying self-propelled movement and intentional behaviour, believability is not necessarily dependent on complex or realistic behaviour, but more influenced by a user’s subjective perception of a character’s behaviour (i.e., can a person match the character’s behaviour to what they have observed in living beings). Believability cannot be accomplished by focussing on a single behavioural aspect, but should rather be seen as a combination of various attributes (including but not limited to intelligence, reactivity, emotion, etc.) blended together. An overview of requirements that contribute to the believability of a character is given in Loyall (1997).

Likability is a more straightforward concept. When using and/or interacting with a system the capabilities of the system and the feedback it provides influence a user experience. We hypothesize that the gaze behaviour system, which adds an element of natural communication to the iCat, will elicit positive emotions in a user during the interaction. The reported level of positive emotions is called a user level of likability of the iCat. We use the following definition of likability. A character is likable if, from the perspective of a person interacting with it, the character elicits positive emotional experiences within them. From studies such as Nass and Moon (2000) we see that people have an innate inclination to react in a social and natural manner to those things we perceive as capable of having personality. This means we can build technology that allows us to communicate naturally with them. By natural communication we do not only mean speech, but also all aspects of the way humans exchange information while interacting with each other. This also includes facial expressions, body posture, gestures, gaze direction and tone of voice. Because there should be a certain degree of consistency between a robot’s embodiment and its behaviour, the fact that the iCat has eyes causes people to think that it can see. We contribute to the iCat’s communicative capabilities and attempt to convince people that the iCat can see by equipping it with a gaze behaviour system.

Functions and effects of gaze

In this section, we summarize some of the functions and effects of gaze with regard to personality, emotions, reactivity and responsive and self motivation. The major function of gaze is clearly to exchange information through the visual channel. The direction of gaze is mainly determined by the choice of the focus of attention. Gaze may also be turned away from attention-seeking objects to avoid distraction, for instance when one has to concentrate on thinking. However, gaze has many side-effects and functions that build on this basic determinant. A study of eye contact during two person social interactions is given by Argyle and Dean (1965). This study states that eye contact may serve a number of functions. First, it has a feed-back seeking function. A speaker will often look away while talking to avoid distraction and seek eye contact on completion of the speech to observe the reactions of listeners. Second, it can signal that the interaction can proceed. Other examples of information communicated through gaze are given in Argyle and Cook (1976). One of the more notable examples is that there is a relation between the liking of another person and the amount of gaze. Feelings of embarrassment, guilt and sorrow often result in aversion of gaze. Several experiments have been done to evaluate the perception of eye gaze. They show that the information perceived through gaze is largely similar to the information that was conveyed. An interesting exception is that large amounts of gaze are perceived as dominant whereas dominant people gaze less at people they interact with. An experiment by Argyle et al. (1974) investigates the influence of the amount of gaze (from no gaze to continuous gaze) during a conversation. The results show that higher amounts of gaze lead to a higher score on a dominant–submissive scale. Also more gaze increases the score on an agreeable–disagreeable scale up to a certain point after which it is evaluated less agreeable. Given the many effects of gaze behaviours on conversational flow, emotions and interpersonal variables, it plays an important role in the appreciation of a person or an agent in terms of believability and likability.

Effects of gaze on believability and likability

Believability and likability of a (robotic) character cannot be accomplished by focusing on a single behavioural aspect, but require a well-balanced blend of various attributes (Loyall 1997). The aspects to which gaze behaviour contributes are: personality, emotion, reactivity and self motivation.

Personality

The single most important aspect of believability is personality. All the details of a character’s behaviour and emotional responses to situations, and the (unusual) traits a character has, are what define the individual and it is especially these that make the individual interesting. Somebody’s gaze behaviour in a social context says a lot about his/her personality. Argyle (1967) says that the eyes may provide clues to aspects of personality that determine a person’s behaviour in similar situations. Libby and Yaklevich (1973) and Larsen and Shackelford (1996) provide an overview of personality determinants that are related to different types of gaze behaviour. They find that gaze is mainly associated with personality traits such as shyness, dominance and needs to give or avoid affection and self-esteem.

Emotion

From the traditional character-based arts and the field of animation it is clear that emotion also plays an important role in believability. Therefore a character should appear to have appropriate emotional reactions and be able to show them in some way and at appropriate times. Research into the role of gaze in social contexts has shown that it is one of the most significant forms of nonverbal communication and previous research into gaze shows that clues about various emotions are communicated to an interaction partner through one’s gaze (Argyle and Cook 1976; Kimble and Olszewski 1980; Libby and Yaklevich 1973; Emery 2000). Primary methods for conveying emotion (mainly sexual attraction, love, shame and sorrow) through gaze is the amount of time spent looking at an interaction partner and the direction to which the eyes are averted. To be believable a character needs to communicate the same (emotional) message with all its body parts. The face, body, eyes, voice, etc., must work together to express the same message. If a character has eyes then they should also be used in the expressiveness of the character. Consider this: how convincing is a character’s happiness, when re-encountering an old friend, to an observer if the character does not even look at this friend? This notion is confirmed in a study by Adams et al. (2003) which shows that the direction of somebody’s gaze actually influences how strongly our brain reacts to the emotions they display. That this is consistent with what is conveyed by people is concluded from an experiment by Kimble and Olszewski (1980). They show that more direct gaze is maintained when emotionally heavy messages are conveyed.

Reactive and responsive

A believable character should appear to react to changes and stimuli from its environment and its responses should also occur at a reasonable speed. If the response lags behind the stimuli too much, the reactiveness of the character will likely have a negative influence on the believability. Humans and animals often exhibit involuntary gaze shifts as a reaction to (unexpected) environmental stimuli (Carpenter 1988). This type of behaviour should also be exhibited by a believable character. To show that it has become aware of something that has happened in the environment it should direct its visual attention in the direction of the stimulus.

Self motivation

Apart from reacting to stimuli from its environment, a character should do things of its own accord because this reflects its internal drives and desires. A person’s belief in a character will quickly diminish if the character sits still when the environment ceases to provide suitable stimuli. If a character in this situation were to inspect its surroundings or casually gaze out of the window, this would probably lead to an increased level of belief because this gazing then reveals some degree of self motivation in a character. This idea has been the subject of a variety of research projects. Salvucci (1999) looked at mapping eye movements to cognitive processes based on the idea that eye movements provide a window into a person’s thoughts and intentions. He proposes a set of algorithms that are able to interpret eye-movements as accurately as human experts. Libby and Yaklevich (1973) say that when a person is engaged in deep thought they will often look in an upward direction to avoid visual distraction and Williams (2001) mentions that saccadic eye movements can show shifts in a character’s chain of thought.

Design of a gaze behaviour system for the iCat

In this section, we focus on the design of the gaze behaviour system of the iCat. Special attention is paid to vestibulo ocular reflex, gaze shifts, vergence and smooth pursuit.

Gaze movements

Eye and head movement is important to humans because the density of photo receptors on the retina is highest in the fovea, a spot located nearly directly on the optical axis. A person’s gaze must be such that the region of interest falls on the fovea (Purves et al. 2001). This enables our high visual acuity when focusing on a certain point and is called foveated vision. There are four major types of eye and head movement that are important for gaze behaviour and must be simulated by the gaze behaviour system:

  • Gaze shifts are movements that change the focus of gaze from one point to another. They can be extremely small or very large, requiring the coordination of the head. Small gaze shifts without head coordination are called saccades. They can be induced voluntarily but often occur reflexively.

  • Vergence movements are when the eyes move in opposite directions in the horizontal plane in order to bring an image near the fovea of both eyes. They are mainly voluntary and relatively slow.

  • The vestibulo ocular reflex generates eye movements that compensate for head movement. It utilizes information from the balance system to move the eyes in an opposite direction to the head.

  • Smooth pursuit is the visual tracking of a moving object. This system uses visual feedback and prediction to generate coordinated eye and head movements stabilizing the image of the object on the retina.

Requirements

The gaze behaviour system must fulfil the following three requirements. Each is elaborated on and worked out in the remaining sections.

  1. 1.

    A point in space where the iCat’s gaze should be directed is an input for the gaze system because the functions of gaze (the visual as well as the social function) are realized by altering or maintaining the point where gaze is directed.

  2. 2.

    In order to alter or maintain the gaze target, the system must implement models of the four major types of eye and head movement: vergence, vestibulo ocular reflex, smooth pursuit movements and gaze shifts.

  3. 3.

    The system needs to fit into and interact with the current software environment and control the iCat hardware so that it can be used to create complex interaction scenarios.

Integration into existing environment

The gaze behaviour system will drive certain iCat actuators that correspond with the body parts involved with gaze movements. Because of this it needs to interact with the existing iCat control software, the animation engine, which also drives the iCat hardware. Animations are rendered onto the iCat hardware by the animation engine at fixed speed of 10 frames per second (fps).

The engine has a number of channels in which animations can be “played”, and is able to merge animations together or to fade one animation into the next. Apart from predefined animations, the engine also has the ability to render so-called behaviours which are “dynamic” animations written in the interpreted programming language Lua.

A number of different body parts are involved with gaze movements, namely: the eyes, head, upper and lower eyelids. The iCat has actuators for the following body parts:

  • Eyes: Controlled by the left eye, right eye and eyes vertical actuators.

  • Head: Controlled by the body and neck actuators.

  • Upper eyelids: Controlled by the left eyelid and right eyelid actuators.

The output of the animation engine can be seen as the current state of the embodiment and is an input for the gaze behaviour system. This is a vector of actuator positions as shown in (1) (note that only the actuators that are relevant for the gaze behaviour system are specified).

$$ \vec{x} = [x_{\rm{body}} ,x_{\rm{neck}} ,x_{\rm{left-eye}} ,x_{\rm{right-eye}} ,x_{\rm{eyes-vertical}} ,x_{\rm{left-eye-lid}} ,x_{\rm{right-eye-lid}} ,x_{\rm{body}} , \ldots ]^{t} $$
(1)

The output of the system is a modified vector: only actuator values relevant for gaze movements are altered. How the system should integrate into the existing environment is depicted in Fig. 2.

Fig. 2
figure 2

The integration of the gaze behaviour system with the existing soft- and hardware environment

Full size image

The gaze target point

The gaze behaviour system requires an input signal that specifies where the iCat’s gaze should be directed. This signal is shown in Fig. 2 as \( \vec{p}. \) To simulate the iCat gazing at specific points in three-dimensional (3D) space we use a standard right-handed Cartesian coordinate system. Often different situations require a different coordinate system given a reference point. For the gaze position of the eyes one has to use a different coordinate system than for the gaze position of the head, since the eyes can rotate. To accommodate this we need the following coordinate systems.

World reference coordinates

This is a fixed coordinate space that enables us to refer to the world with the iCat as a reference point. The origin of this space is located in the centre of the base of the iCat.

Camera coordinate space

This coordinate system makes it possible to ascertain in which direction the camera is looking and where, from the viewpoint of iCat, an object is located. The origin of this space is located at the lens of the camera.

Head gaze coordinate space

This coordinate system is needed in order to determine the head gaze direction. The origin of this space is located at the centre between the eyes.

Eye gaze coordinate space

Correspondingly we have the same needs for eye gaze direction. This coordinate system has the same origin as the head space, but is influenced by the vertical and horizontal rotations of the eyes (i.e., if the eyes rotate, this coordinate space also rotates).

Gaze movement models

The gaze behaviour system implements models of the four major types of eye and head movement, vergence, vestibulo ocular reflex, smooth pursuit movements and gaze shifts as sub-systems. Vestibulo ocular reflex, smooth pursuit and gaze shift movements alter or maintain the direction of gaze whereas vergence movements focus both eyes on the same point somewhere along the line of gaze. Therefore, the vergence sub-system is always active and only one of the other three sub-systems is active at any given time. The gaze behaviour system has a control signal (see Fig. 2) to switch between these three sub-systems. More detailed information on the gaze system can be found in Meulemans (2007).

Vestibulo ocular reflex

Recall that the vestibulo ocular reflex (VOR) generates eye movements that compensate for head movement. It utilizes information from the balance system to move the eyes in an opposite direction to the head. The effected embodiment parameters are the horizontal position of the left and right eye and the vertical position of both eyes simultaneously. The algorithmic procedure is as follows: the coordinates of target point are first transformed into head gaze coordinates called \( \vec{p}. \)

Afterwards, the following angles are computed:

$$ \begin{aligned} &x_{\rm{left-eye}} = x_{\rm{right-eye}} = \arctan (p_{x} /p_{y} ) \\ &x_{\rm{eyes-vertical}} = \arctan (p_{z} /p_{y} ) \\ \end{aligned} $$
(2)

Equation 2: Calculating the eye angles from the head gaze coordinate \( \vec{p}. \)

If these eye angles lie outside the achievable scope, due to hardware limitations of the iCat, the nearest achievable eye angles are chosen. From a study of Collewijn and Smeets (2000) it follows that latency for VOR is around 10 ms. Hence we do not model this latency.

Smooth pursuit

Recall that smooth pursuit is the visual tracking of a moving object. This system uses visual feedback and prediction to generate coordinated eye and head movements stabilizing the image of the object on the retina. First, the target point is transformed to body coordinates and afterwards the dynamical model depicted in Fig. 3 is used for tracking. The required angle is computed by transforming world coordinates to head gaze coordinates.

Fig. 3
figure 3

The control diagram for the smooth pursuit

Full size image

The model is just a simple moving-average model for which the parameters, 0.2 and 0.8, are determined by the work of Freedman (2001) and experimentation. For high-speed tracking this model is too slow, but it should be observed that small errors are compensated by fast eye movements.

Gaze shifts and saccades

Gaze shifts and saccades change the direction of visual attention from one point to another. For humans, they range from very small to very large amplitudes and can take between 20 and 300 ms to complete. Also for large gaze shifts where the coordination of the head is required, there are variations in the amount of head contribution and in the latencies before the head and eyes start moving. These dynamic properties are influenced mainly by the gaze shift if it is a reflexive response or a voluntary shift of visual attention. Our model is an adaptation of the model proposed in Freedman (2001), it incorporates the dynamics of a real gaze shift system as much as possible. The gaze shift model is split into a number of elements, namely a system that determines head contribution and head and eye movement simulators. The input is the target point in world coordinates. The coordinates of this target point \( \vec{p} \) is transformed into head gaze coordinates (this determines where the target point is w.r.t. the direction of the head) and eye gaze coordinates (this determines where the target point is w.r.t. the direction of the eyes). From this, one can easily calculate the horizontal and vertical gaze displacement angles and the initial eye positions. Using these, the contribution of the eyes and the head to the gaze shift can be calculated. For gaze shifts smaller than a threshold there is no head contribution to the gaze shift, only eye gaze shift. The movement dynamics for the head and eyes are depicted in Figs. 4 and 5.

Fig. 4
figure 4

The control diagram for the head movement for a gaze shift of the head

Full size image
Fig. 5
figure 5

The control diagram for the eye movement for a gaze shift of the eyes

Full size image

Freedman (2001) determined the parameters of the models experimentally by analysing head and eye movements in monkeys. An example of a horizontal gaze shift can be found in Fig. 6.

Fig. 6
figure 6

An example of a horizontal gaze shift

Full size image
Vergence control

The other gaze models will calculate the horizontal angle of the eyes as if their origins were equal to the origin of their parent joint (eyes vertical), i.e., discarding their offsets. This means that the eyes will be looking in a certain direction, but focused at infinity. The vergence control system modifies the horizontal eye angles to converge on the target point distance. Irrespective of the above sub-systems are active the gaze behaviour system goes through the same processing steps:

  • Step 1: Transform 3D target point to required embodiment angles. The 3D target gaze point and the current state of the embodiment are used to calculate the embodiment angles required to gaze in the direction of the target. The current state of the embodiment, represented as in (1), is another input for the gaze behaviour system. It is provided by the animation engine. The animation engine outputs actuator positions as values for the embodiment state vector which are transformed into joint angles by the gaze behaviour system.

  • Step 2: Simulate gaze movement dynamics. The gaze shift and smooth pursuit sub-systems simulate the movement from the current embodiment state to the required embodiment state over a period of time. This requires the current and required states of the embodiment and the target point signal. The vestibulo ocular reflex model is a direct response to the current embodiment state and therefore does not simulate movement dynamics.

  • Step 3: Alter angles to converge on target distance. At this stage the horizontal eye angles are oriented in the correct direction, but their focus is on infinity. The vergence sub-system alters the angles (which are initially equal) to converge on the target point distance. This requires the 3D target point.

  • Step 4: Adjust eyelids for vertical gaze direction and blinks. This step alters the position on the eyelids so that the pupils are not covered by them when the gaze is pointed in the upward direction. This system also implements blinking.

Evaluation of the believability and likability

In this section we focus on the methodology behind the user test and evaluate the results in order to asses the believability and likability of the gaze behaviour system. Because both believability and likability are abstract concepts and can therefore not be measured on an absolute scale, we need to compare the perceived believability and likability of a character with gaze behaviour with one without gaze behaviour. More precisely, we try to answer the following question:

What is the effect of the iCat gaze behaviour on a person’s belief in the iCat and their liking of the iCat?

The hypothesis is that the gaze behaviour system for the iCat will have a significant influence on these factors, i.e., the iCat will be perceived to be more believable and/or likable. Furthermore, we also expect that people deem the iCat more likeable when it exhibits more natural gaze behaviour.

Methodology

To test the hypothesis an experiment was conducted in which participants engaged in two interaction sessions with the iCat. During one condition the gaze behaviour system was used and during the other condition static animations were used.

Participants

A total of 18 participants (8 males and 10 females) for the experiment were recruited internally at the High Tech Campus facilities where Philips Research is located. The criteria used during the selection process were that participants had no prior interaction experience with the iCat and that they were sufficiently adept in the English language. All participants signed a consent form at the beginning of the experiment and during the experiment they were offered chocolates as gratitude for their participation.

Design

The experiment was a within-subject design with two conditions, with (test condition) and without (control condition) the implemented gaze behaviour system. Participants interacted with both versions of the iCat and this was done in a balanced order, i.e., half of the participants first interacted with the iCat in the experimental condition followed by the control condition and half of the participants started with the control condition followed by the experimental condition.

Setting

The experiment took place in the Philips Homelab. During the experiment, participants were seated at the dining table in the living room behind a laptop computer. The iCat was situated on the table, within the user’s peripheral field of view. During the experiment, the experimenter was seated in the control room, from which it is possible to monitor all activity in the lab through many cameras and microphones. The experiment setting is shown in Fig. 7.

Fig. 7
figure 7

In the setting of the experiment the participants were located at the dining table in the living room of the Homelab with the iCat to their left

Full size image

The evaluation of the believability and likability focussed on a person’s perception of the character which was measured with a paper-and-pencil questionnaire and a semi-structured interview. Both the questionnaire and interview were in English.

As there seems to be no literature concerning the (subjective) measurement of believability, a new questionnaire has been designed that focuses on those aspects of believability that are generally influenced by gaze: personality, emotion, reactiveness and self motivation (Sect. 2.1). For each of these aspects we have formulated four to eight items per aspect, in the form of statements. The participants were asked to rate agreement with each item on a five-point Likert scale (1: strongly disagree, 2: disagree, 3: neither agree nor disagree, 4: agree, 5: strongly agree). Table 1 gives an overview of all the items and the factor they belong to.

Table 1 Each aspect of believability has a group of items associated with it
Full size table

During the interview, the participants were asked more direct questions about how they perceived the lifelikeness and their likability of the iCat. Participants were also asked what they thought of the task they were asked to perform during the experiment in order to filter out possibly biased responses.

Procedure

During each session, the participants were asked to answer five to ten multiple choice general knowledge questions that appeared on the screen of the laptop in front of them. They were allowed to use the computer to look up the answers to the questions. The questions have been where chosen such that on average 60–70% of the answers will need looking up. Answers were then checked with the iCat who indicated if they were correct or incorrect so that the user’s attention was forcibly drawn towards the iCat a number of times during the experiment. Independent of the correctness of the answer, the user continued with the next question. A script of the experiment scenario is given in Table 2.

Table 2 For both the test and the control conditions the following scenario was used
Full size table

The experiment has a so-called ‘Wizard of Oz’ setup where the iCat and the users interact, which is partly controlled by the experimenter.

The evaluation protocol

For the control condition the iCat’s gaze movements are animated using static animations. In the test condition, the models implemented by the gaze behaviour system will replace most animations of the control condition. The animations displaying happiness and sorrow will be kept the same and are rendered over the gaze behaviour using the merging capabilities of the animation engine. The activation of the animations/behaviour as a response to the environment will be controlled by the experimenter. For example, in step 2 of the scenario, when the participant seeks the attention of the iCat the experimenter will start the animation or signal the behaviour that focuses the attention of the iCat on the participant. We set up the experiment so that the different modes of gaze fit adequately in the interaction.

The following sections describe how and when the models are applied and what difference in the participants’ perception we attempt to invoke.

Vestibulo ocular reflex

When a participant informs the iCat of his or her answer to the question, the iCat responds by nodding or shaking its head. During this movement in the test condition the eyes will remain fixed on the participant, such as is normal when humans perform this type of gesture. For the control condition the eyes will not counter rotate in relation to the head movement, but remain still. Because the iCat expresses some emotion during this movement we expect this emotion to be perceived more strongly when the eyes remain focused on the participant.

Smooth pursuit

The smooth pursuit will be used to keep focus on the participant’s face during various parts of the experiment. For example, when the iCat and the participant are paying attention to each other, in the test condition the iCat’s eyes stay focused on the participant when the participant moves. We expect this to have an effect on self motivation (the iCat has some interest in the participant) and on reactivity and responsiveness (the iCat is aware of the participant). During the control condition the iCat’s gaze will not follow the user, but stay in the pre-animated position.

Gaze shifts

From time to time the iCat’s gaze will shift between different points of visual attention in the room. The iCat will often shift its gaze from the participant to the screen of the laptop and visa versa. For example, after indicating if a question was right or wrong the iCat will look from the participant to the screen to indicate the next question. Because the user is likely to be looking at the iCat at this time the user should have a good view of this behaviour (which is naturally also the case for the previous two models). When the user is busy using the laptop the iCat will lose interest in the screen and the participant and look around the room. Also, once during the experiment, the iCat will respond with a gaze shift to something that happens outside. When the iCat’s focus is on a particular object, it will simulate “scanning” the object by making small gaze shifts quickly after each other (on average every second). During the control condition, the iCat will display the same type of behaviours (except “scanning”), but using static animations which do not include movement dynamics. We expect gaze shifts to have an effect on the perceived reactive and responsiveness and on the degree of self motivation of the iCat.

Blinking

In both conditions the iCat will blink its eyes. In the control condition this is every three and a half seconds and in the test condition the period between blinks and the duration of one blink vary. Also, in the test condition, gaze shifts will often be preceded by a blink. We expect blinking to have an overall effect on believability, because it introduces movement at regular (and not fixed as in the control condition) intervals.

General gaze behaviour

Personality is to some degree affected by all types of gaze movements, but mainly by smooth pursuit and VOR because the amount of gaze the iCat focusses on the user is a determinant for this. Likability is also affected by the gaze behaviour system as a whole because it introduces “something extra” to the iCat’s capabilities.

Results

In this section, we will elaborate on the outcomes of the experiment.

Quantitative data

First we calculated the Cronbach’s Alpha for each factor of the questionnaire. (The Cronbach’s Alpha is a measurement for testing the reliability of items that belong to the same scale.) The results are shown in Table 3. Correlation between items in the personality scale was very low. This likely has to do with the fact that the questions are directed towards certain personality traits and not to personality in general. Because of this we did not evaluate items of personality as a group. Items 7, 9, and 13 (see Table 1) did not have a corrected item-total correlation higher than 0.20 and we decided to remove them from the list. The Cronbach’s Alpha for emotion is on the low side, but we consider it acceptable for the type of test performed. The task to be performed distracted the users from analysing the facial expressions of the iCat and the results are far from significant. Hence we are only looking for an indication.

Table 3 Cronbach’s Alpha for emotion, reactiveness, self motivation and likability factors
Full size table

To find out whether there is a significant variance in the way participants perceived the differences between the gaze behaviour of the two iCats, a Wilcoxon signed rank test was conducted for all items of the questionnaire. This is a non-parametric test to compare related samples. The results of this test per item for significant (P < 0.05) and approaching significant (P ≈ 0.05) P values are given in Table 4.

Table 4 Items that are statistically significant with their Z-score, P value and the direction of the difference
Full size table

The Wilcoxon signed rank test was conducted for the factors emotion, reactiveness, self motivation and likability. The results show far from significant results cf. Table 5. There is, however, in all cases a slight tendency towards the test condition scoring higher than the control condition.

Table 5 Wilcoxon signed rank results per factor including Z score, P value and the direction of the difference
Full size table

Qualitative data

In the semi-structured interview a large amount of qualitative data was collected. The users were directly asked their opinion on the believability and likability of the iCat. These results are given in Table 6.

Table 6 Answers given to the questions asked in the interview
Full size table

The main reason given by the participants for why they thought the test iCat displayed more lifelike behaviour than the control version was that the iCat seemed to look around more (6 times) and in some cases (4) it was said to show more expression.

The participants that had no preference mostly thought that both versions showed some lifelike behaviour. Most participants expressed no preference in liking for a particular iCat, but the most frequently reported reason for liking the control condition iCat more was that it seemed more attentive (3) to them. Participants that liked test condition iCat more mainly said that this was because it seemed more lively (3). The participants also gave some common statements about the iCat and the experiment in general. Eight participants reported that the task was distracting them from paying attention to the iCat and five participants said that they “forgot” to notice the iCat during the first session because all of their attention was focussed on performing the task (this ‘problem’ was also noted by the experimenter who observed the experiment from the control room). Six participants noted that the iCat sometimes reacted slowly after they told it their answer. Four participants said that the iCat made a lot of noise which reminded them that it was a robot. Only a single participant mentioned that in the test condition the iCat’s eyes remained focused on him when the iCat nodded or shook its head.

Because a large number of participants mentioned that they found the task somewhat demanding, we also investigated any order effect. We found that when analysing the quantitative data samples by comparing a participant’s first session with the data from the second session 20 items moved closer to significant values (i.e., had lower P values). This indicates that the experiment suffers from a considerable order effect. When looking at the actual values, the answers given in the first session are more extreme (i.e., there is a tendency towards strongly agree or strongly disagree) than in the second session.

Discussion

In this experiment the main question was whether the manipulation of the iCat’s gaze behaviour results in an increased believability and/or likability. The main conclusion that can be drawn from the results of the experiment is that the participants were unable to see significant differences between the two conditions. We believe this can be mainly attributed to the design of the experiment.

Because of the considerable order effect the differences between the two conditions are levelled out. The order effect is probably caused by the fact that participants were unfamiliar with the setup of the experiment. Because they were completely unaware of the kind of questions asked in the questionnaire and their focus was not primarily on the iCat, they did not know what to pay attention to during the first session. After answering the questionnaire they had some idea what to look for in the iCat but had little to compare it with, because they had missed these things during the first session. During the interview, participants did, however, report that they saw differences between the two iCats. It is probable that they could not exactly identify what the difference was. Introducing a “dummy” session before the two conditions in which participants perform a similar task and complete a similar questionnaire would likely help to reduce the effect the order of the conditions has on the results.

The fact that the iCat was more part of the setting of the experiment than part of the task is probably another reason as to why participants were unable to report significant differences. Because the multiple choice questions of the task were quite difficult, a lot of the participant’s attention, in both sessions, went toward answering these questions correctly (even though participants were specifically told that this was not so important and were asked to pay attention to the iCat).

Additionally, the differences between the two iCats were subtle. Both iCats displayed similar gaze behaviour but differed in the actual movement. For example, the head of the control condition iCat moved mostly at a constant speed as opposed to the simulated movements (varying speed) in the test condition. Due to the possible lack of attention for the iCat the participants may not have specifically noticed this.

That two of four significant items (1 and 6) relate to personality factors shows that gaze is important with regards to how people perceive personality. But because one item is in favour of the control condition and the other of the test condition, it is difficult to say how gaze behaviour relates to believability. To say more about this would require additional experiments that focus on the impact of gaze on specific personality discriminants.

The quantitative data show few significant differences between the iCat with and without the gaze behaviour. But the fact that despite the considerable order effect there is a slight tendency towards more emotion, reactiveness, self motivation and likability in the test condition warrants further investigation into the effect of gaze behaviour on believability and likability. This is also reinforced by the qualitative data where nearly half the participants said that the test iCat showed more lifelike behaviour.

Conclusions and future work

The main objective of this work was to increase the believability and likability of the iCat by designing a gaze behaviour system. We studied gaze behaviour from a number of perspectives to see what is required if such a system is to have an effect on the believability and likability of a character. We found that besides vision gaze has an expressive and functional purpose in social interaction; mainly conveying emotion and personality and mediating conversation flow. Also when applied to robots and virtual characters gaze can portray reactiveness to the environment and self motivation.

We designed a gaze behaviour system that implements models for the four most important types of gaze behaviour in species with binocular foveated vision: vergence, vestibulo ocular reflex, gaze shifts and smooth pursuit. The models realistically simulate these types of gaze movements on the iCat. This was achieved by using models and empirically obtained data of gaze behaviour in humans and primates provided in literature. The gaze behaviour system can be used by application developers to create complex interaction scenarios for the iCat. The modular design of the system is such that the movement models are fairly independent of the embodiment.

Because of this the system can be easily ported to other robotic platforms. The only requirements for these embodiments is that they have a head and eyes with two degrees of freedom.

The problem of measuring the effect of the gaze behaviours on believability and likability of the iCat was also tackled. Believability is a term often used in literature on virtual and robotic characters, but literature lacks a clear and common definition of the term. Additionally, actual established measurement methods for believability seem to be non-existent. Our approach was to take a number of attributes that make a character more believable and relate them to gaze behaviour.

We found that personality, emotion, reactiveness and self motivation are influenced by gaze. The effect of gaze behaviour on these four factors together with likability was tested in an experiment. We measured the effect of gaze behaviour on believability by comparing quantitative and qualitative information about the user’s perception of an iCat with gaze behaviour to an iCat without gaze behaviour. The results did not yield any statistically significant results, but do warrant further effort in this direction. We also gained some valuable insight into testing the effect of the type of gaze behaviours we created on the believability and likability of the iCat. In reflection, the user test might have required more time than was available, but has nonetheless turned out to be very interesting and valuable learning experience.

Overall, this study is a step towards gaining more insight into gaze behaviours for robotic characters. We presented an approach to creating realistic models of gaze behaviour that can be further utilized to create a more socially competent iCat. We consider the observable gaze behaviour that has been designed for the iCat quite natural even though the user test does not fully confirm this.