Introduction

With an estimated prevalence of one in 68 (Blumberg et al. 2013), autism spectrum disorder (ASD) is an important and increasingly serious public health concern. The most important ASD characteristic and the most challenging for treatment is the social interaction deficit. Children with ASD have impairments in verbal and nonverbal communication, in social–emotional reciprocity and relationships, and have a restrictive pattern of activities (American Psychiatric Association 2000). One of the most typical impairment in non-verbal communication is the tendency to avoid eye contact (Jones and Klin 2013), the unconventional use of gestures and difficulties in the interpretation of body language (Reed et al. 2007). On the level of social–emotional reciprocity, individuals with ASD experience difficulties in understanding thoughts, feelings, intentions and preferences of others (Baron-Cohen et al. 2013; Bons et al. 2013). In addition, impairments in the language development, in initiating actions, sharing and turn-taking limit the possibilities of individuals with ASD to engage in meaningful reciprocal interactions (Zwickel et al. 2011; Huskens et al. 2013). They also have a poor understanding of the complexity of the social world and therefore engage less in social interactions (Scattone 2007).

Given the impairments mentioned above, all treatments have difficulties in motivating and engaging children with ASD in (social) learning tasks (Weiss and Harris 2001). Nevertheless, several studies suggest that children with ASD can successfully engage in social interaction if social information is presented in an attractive manner (i.e. in a manner that is easily understood and clearly identifies the expected behaviors) (Quirmbach et al. 2009; Chevallier et al. 2012; Vismara and Lyons 2007; Deckers et al. 2014).

In this paper, we explored the potential added value of social robots as motivating tools in social tasks with children with ASD. The current literature on this area of research is fragmented: a wide range of social skills were addressed, using different designs of robots and different scenarios. On top, recent reviews (Diehl et al. 2012; Scassellati et al. 2012) reported mixed and inconclusive results.

Broadly speaking, three assumptions are held in robot assisted therapy (RAT) for children with ASD.

(1) Some children with ASD have a preference for robotic toys over non-robotic-toys and humans; this preference can be used to increase the effectiveness of ASD treatment programs. When the targeted social skills are modeled by a robot instead of a human, children with ASD may better understand the task due to the simplicity and the transparency of the (programmed) robot.

For example, Tapus et al. (2012) reported that preschoolers with ASD manifested eye gaze and smile/laughter behaviors in interaction with a the robot more frequently compared to the interaction with a human partner. In addition, Vanderborght et al. (2012) found that a Social Story Intervention conducted by a robot was more motivating and led to a lower level of prompt to perform the targeted social skill (saying thank you) compared to the intervention conducted by a human therapist. Kim et al. (2013) found no differences between the adult and robot condition in the amounts of speech children directed to interaction partners, but the robot triggered more speech than a computer screen. Also, in Pop et al. (2013a, b) it was investigated whether social stories presented by a social robot have a greater effect than those presented on a computer display on the expression of independent social abilities of children with ASD. 20 children with ASD were involved in the study (seven in the control group, six in the computer-presented social stories group and seven in the RAT group). The results showed that using the social robot to implement a Social Story Intervention was more effective for improving the independence of expression of social abilities for the participants, than the computer screen. Huskens et al. (2013) revealed that the number of self-initiated questions did not differ between a robot or a human trainer. Pop et al. (2014) found that children with ASD did not perform better in a functional play task with the robot compared to an adult partner. The same study also revealed that children initiated less verbal communication in the robot group compared to the adult group. However, they exhibited more collaborative play with the robot compared to play situations with the human partner. Finally, in Wainer et al. (2010), children with ASD alternated between playing a cooperative, dyadic video game with an adult and playing the same game with an autonomous humanoid robot. They found that when the robot was their play partner, children had a poorer performance compared to the human–partner.

(2) Although there is not yet a clear answer regarding which category of children from the autism spectrum answer the best to RAT interactions, the robot can function as a social mediator in triadic interactions: interacting with a robot does not only trigger social responses to the robot in children with ASD but also to others accompanying the child.

For example, Robins et al. (2004) showed that for low-functioning children with ASD the robot was a mediator for joint attention. Kozima et al. (2008) found that 2–4 years old children with ASD spontaneously approached the robot and engaged in dyadic interaction, which then was extended to triadic interactions, where they exchanged with caregivers/other children pleasure and surprise they found in the robot. Also, Kim et al. (2013) reported that the interaction with a social robot elicited speech directed toward an adult confederate, not only toward the robot. Similarly, robot-based activities (i.e. working with peers on programming the robots in an afterschool program) elicited social interactions in 12 years old high-functioning children with ASD (Wainer et al. 2010). Finally, Costa et al. (2010) provided qualitative observations that two children with ASD continued to play a ball game with each other after learning it from a robot.

(3) A number of studies found positive responses of individuals with ASD to different types of advanced interactive technologies, such as computer technology (Bernard-Opitz et al. 2001), virtual reality environments (Mitchell et al. 2007) and robotic systems (Dautenhahn and Werry 2004; Kozima and Nakagawa 2006; Kim et al. 2013; Scassellati et al. 2012; Diehl et al. 2014). Therefore, the third commonly held assumption in the RAT field is that children with ASD enjoy interacting with robots due to their interest in technology and consequently they are more motivated to engage in social tasks assisted by robots compared to tasks with humans. As a result, robots might be useful in eliciting social behaviors in children with ASD.

For example, children with ASD had more eye contact and physical contact with the robot than with a comparison toy (Dautenhahn and Werry 2004). Robins et al. (2006) found that three children with ASD were more interested in the robot when its appearance was more robot-like. Although participants showed an initial preference for robot-like characteristics, this preference disappeared over a 6 month period of time. De Silva et al. (2009) found that five individuals with ASD were able to follow social referencing behaviors made by a robot. Also, several studies focusing on play tasks, showed that robots can elicit turn-taking with children with ASD who manifest difficulties in engaging in such behavior (Kozima et al. 2007; Dautenhahn et al. 2009; Ferrari et al. 2009). Warren et al. (2013) found that, across a series of four sessions, five of the six children with ASD exhibited lower average levels of prompt in their ability to orient to joint attention prompts administered by the robot. On the other hand, Anzalone et al. (2014) reported that during a joint attention elicitation task with a robot and a human agent, children with ASD had a lower performance with the robotic agent than with the human.

As shown above, positive results regarding the effectiveness, motivational power and added value as a social mediator of social robots in RAT are questionable. In addition, it is not clear if the positive results will hold in bigger samples. The present study is designed to answer this question by using a repeated measures design with (to our knowledge) one of the largest sample of children with ASD (N = 30). Most of the existing studies, as reported by two of the most relevant reviews in the field (Scassellati et al. 2012; Diehl et al. 2012) are reporting case-studies or small groups of children with ASD as participants of RAT studies. Very few studies reported comparable (e.g. Kim et al. 2013) or bigger samples of children with ASD (e.g. Costescu et al. 2014) in interaction with social robots. The following research questions were addressed:

(1) Does the performance of children with ASD in a social task with a robot differ from their performance in the same task with an adult?

The experimental task was adjusted from Kim et al. (2013) and it was designed to elicit a host of social perception, reasoning, and interaction behaviors from participants. These included taking turns with the interaction partner; identifying the interaction partner’s emotions or expressions of preference for one particular fruit or another. We explored whether or not the performance of detecting preferences in a play task (i.e. making a fruit salad for each of the two partners based on their preferences) with the robot differed from the same play task with a human. A detailed description of the experimental task can be found in the Procedure section and also the exact protocol that was followed can be read in the “Appendix”. Based on the interaction studies with the social robot Probo (Pop et al. 2013a, b; Simut et al. 2012; Vanderborght et al. 2012) and the characteristics that robots have for being good social partners for children with ASD (Baron-Cohen 2010), we expect children to easily understand the robot’s behaviors. However, due to the inconsistency of existing results, we are not able to predict the direction of the differences.

(2) Do children with ASD differ in social and asocial behaviors when interacting with an adult or a robot?

We investigated whether or not the frequency of social (e.g. making eye contact, initiating joint attention, verbal utterances and positive affect) and asocial (e.g. non-response and evading the task) behaviors differ in interaction with the robot compared to the interaction with the human. Based on the reported increased engagement levels (e.g. Kozima et al. 2008; Scassellati et al. 2012; Kim et al. 2013; Diehl et al. 2014)of children with ASD in tasks assisted by robots, we expect less non-response and task evasion behaviors in the robot condition compared to the adult condition.

(3) Does the type of interaction partner (adult or robot) have an impact on the way how children with ASD interact with another adult present in the room?

Behaviors such as eye contact and initiating joint attention were also measured in relation to the experimenter and not only with the interaction partner (robot or adult). Based on the existing research, we expect that the robot condition will trigger more social behaviors between the child and the experimenter than the adult condition.

Method

Participants

35 children with ASD were recruited from three Belgian schools with classes specifically for children with ASD. In order to be included in the study, the participants had to fulfill the following inclusion criteria:

  1. 1.

    A chronological age from 5 to 7 years.

  2. 2.

    A diagnosis of ASD according to the criteria outlined in the diagnostic and statistical manual of mental disorders (American Psychiatric Association 2000). The ASD diagnosis was retrieved from the child’s case file and in each case confirmed by a psychiatrist.

  3. 3.

    An IQ of ≥70. The IQ scores were assessed with the Wechsler Preschool and Primary Scale of Intelligence-III (WPPSI-III-NL, (Hendriksen and Hurks III 2009) (N = 23), and the Snijders-Oomen Non-verbal Intelligence Test Revised Version (SON-R 2½–7), (Tellegen et al. 1998), (N = 7). The mean IQ score was 91.23 (min = 70 and max = 119).

  4. 4.

    No other psychiatric diagnoses;

  5. 5.

    A performance level of 80 % in a preference understanding task. This selection task was equivalent to the experimental task. A detailed description of this selection task is provided in Procedure section. The aim of the selection task was to ensure that the participants had the ability to detect preferences of human partners, thus, they could engage appropriately in the experimental task (see Table 1). When children were not able to detect preferences, differences between the two conditions could be the result of random errors of the children in the two conditions as a consequence of not being able to detect preferences. The procedure used gave us the certainty that differences were not the result of not being able to detect preferences. Also, the age and the IQ criteria were established like this in order to fit with the difficulty level of the task, thus, the task is appropriate for the targeted population.

    Table 1 Age of the 30 children included in the study and their performance during the selection tasks
    Full size table

Thirty-two children had the desired performance level for the selection task. One of them was excluded due to medical reasons and one did not complete the robot condition. Finally, 30 participants (27 boys and three girls) performed both of the experimental conditions (see Fig. 1). Parental consent was obtained for each participant before the study took place. Parents had the right to stop their child’s participation at any time during the experiment.

Fig. 1
figure 1

The experimental setting

Full size image

Experimental Design

A repeated measures design was used to compare the robot interaction with the adult interaction. Each of the 30 participants was exposed randomly to both conditions (with an interval of 7–10 days between the two conditions): the adult condition (AC), where the interaction partner was a human agent, and the robot condition (RC) in which they had to interact with the social robot Probo. A program created by the engineer of the research team took care of the randomization (e.g., computer generated randomization program). Thus, 16 children started with the robot condition and ended with the adult condition and the other 14 started with the adult condition and ended with the robot condition.

Treatment Fidelity

An external person, the first author, was always present in the experimental room during both conditions, to check if the protocol was respected. It was investigated if the exact words, the order and the length of the lines were respected and if the time interval in between the lines was respected. For this, a checklist with the procedural steps was used for both conditions. No errors were observed.

Setting

The study took place in different therapy rooms (minimum 20 m2) in the three selected schools. A false wall divided these rooms. One half was the control room, where an operator controlled the robot using a set of buttons on a computer screen. The operator was able to see into the therapy room via a video camera inside the robot’s head, giving them a general overview of the setting. The interaction partner (the adult or the robot) with the child, always accompanied by the experimenter, were located in the other half of the room. Two pillows were placed in front of the robot: one pillow for the experimenter and one for the child. The placement of the supports required the interaction partner to make a 45° turn to align with the lateral supports for the task (see Fig. 2). The salad bowl was placed in front of the interactional partner and thus also in front of the child. The boxes with the pairs of fruit were placed behind the experimenter, in order not to distract the attention of the child during the task. Two digital cameras were used to record the interaction: a frontal camera recorded the child’s upper torso and face, and a lateral camera captured the child, the interaction partner and the experimenter (see Fig. 3).

Fig. 2
figure 2

Different sequences of the robot condition

Full size image
Fig. 3
figure 3

Robot Control Center (RCC) is used to control Probo in an intuitive and scenario specific way: expressed emotion depicted in the 2dof emotion circumplex model of affect defined by Russell (1980) (left top); the virtual 3D model of Probo in order to visualize the motion behavior (middle top); video image and mosaic image constructed to obtain a full area view, covering the full visual range of the robot (left bottom); the required buttons in order to follow the requested protocol of making a fruit salad (right)

Full size image

Procedure

Each of the 30 participants was exposed randomly, one at a time to each of the two conditions of the study, in different orders as previously described (i.e. 14 adult-robot and 16 robot-adult), with an interval of 7–10 days between the two conditions. The experimental task lasted for 15 min for both robot and adult conditions. As a result, each child had one session of 15 min with the adult and one session of 15 min with the robot. Five adults formed the team: four psychologists (lead author, experimenter, 2 master students as adults for selection and experimental tasks) and one engineer (operator). A training regarding the implementation of the protocol for both selection and experimental task was performed by the lead author for each of the team members. The selection task and the two conditions were guided by the experimenter under the supervision of the lead author, who was hidden behind the false wall together with the operator. The unfamiliar adult used for the selection task was different from the adult used in the adult condition, and was not present during the robot condition. Each child had three meetings with the research team, as follows:

During the first meeting the selection task was performed, followed by a 5-min familiarization phase with the robot. The selection task consisted of a detecting preference task performed with an unfamiliar adult. It was built equivalent with the experimental task which means that the structure of the task and the behaviors (i.e. gazing at the objects, making specific sounds for like and dislike and expressing the facial expressions for happiness and respectively, for sadness) that the adult had to manifest in order to express his preferences were identical with the ones from the experimental task. The children had to build a tower with Lego® blocks preferred by the unfamiliar adult. In order to select the correct blocks to build the tower, the children had to be able to detect the preferences of the unfamiliar adult.

During the familiarization phase, the robot was programmed to stay in alive mode (looking around, blinking and answering questions). The children were allowed to touch or/and talk to the robot for a pre-determined period of time (5 min).

The second and third meetings included performing the experimental task once with the robot and once with the adult, in a randomized order, for each participant. The same experimental task was established for both conditions and the two interaction partners were trained to follow the same standardized protocol (see “Appendix”). The only difference between the two conditions was the type of interaction partner (i.e. the adult or the Probo robot). The experimental task lasted 15 min. The experimental task consisted in making a fruit salad for the interaction partner. The experimental task started with an introductory standardized session, in which the interaction partner welcomed the experimenter and then the child. Some specific questions were asked by the interaction partner in order to give the child time to feel comfortable with the task. If the child did not answer these questions spontaneously (within the next 3 s) the experimenter helped the child to answer and encouraged him/her to feel safe and comfortable by touching the interaction partner. However, none of these behaviors were not of interest for the aim of this study, so they were not included in the analysis. After the introduction, the demonstration started. This sequence had two parts: one in which the experimenter demonstrated to the child how to perform the task and one in which the child had to do it alone, helped by the experimenter. The demonstration sequence was followed by eight more tests that respected the structure of the two demonstrations: the experimenter presented a new pair of (plastic) fruit for each of the eight tests and the child had to select the fruit that the interaction partner preferred, based on the direction of his/her gaze, verbal utterances and facial expressions. The game continued until the eight pairs of fruit were chosen. No reinforcements were provided to the participants by the experimenter. The interaction partner (adult or robot) gave a feedback after the child placed the chosen fruit in the salad, as follows: when the child choose the right fruit, the interaction partner showed a happy face and said “MMM” and for the wrong one, she showed a sad face and said “OOOH”. However, each condition followed the same protocol, so every child received the same amount of reinforcements from each interaction partner. At the end, the interaction partner greeted the child and the experimenter took the child back to the classroom.

The role of the experimenter in the task was to connect different moments of the protocol (i.e. bringing the child from his classroom and bringing him/her back afterwards, making the demo phase for the child, placing each time the two pair of fruits in the supports and asking at the end the child which fruit he/she considers the interaction partner preferred). The interaction partner (robot or adult) had the role of presenting each trial by following the same repetitive pattern of behaviors: calling the child’s name, looking at each fruit and expressing the pre-established facial expression and providing an answer at the end after the child placed a fruit in the salad bowl. Therefore, the child had to follow the behaviors of the interaction partner and had to answer to the questions of the experimenter. The experimenter did not give any prompts to the child to answer to the interaction partner behaviors. However, no prompts were necessary, since the age and the IQ criteria were established like this in order to fit with the difficulty level of the task, thus, the task is not too challenging or too boring for the participants.

The Social Robot Probo

The social robot Probo was used in this study (see Fig. 3). Probo serves as a multidisciplinary research platform for human–robot interaction in order to develop robot-assisted therapies aimed at children. Previous interactions with Probo and children with ASD, showed that they can engage in the tasks and have no difficulties understanding and responding to Probo’s behavior. The robot was used as a storyteller agent showing that the social story said by a robot has a greater impact on the social skills of children with ASD, compared when the story is read by a human therapist (Vanderborght et al. 2012), or when the story is displayed on a computer screen (Pop et al. 2013a). Another study used the robot Probo in an emotion recognition intervention study, reporting that a small group of children with ASD improved with moderate to large effect sizes in identifying basic emotions, such as sadness and happiness (Pop et al. 2013b). Also, Probo was used in a playing-doctor task, showing that children with ASD exhibit more collaborative play when the robot is the patient compared with the situation in which the patient in the doctor game was a human partner (Pop et al. 2014).

The robot is designed to act as a social interface by employing human-like social cues and communication modalities. With 20 motors in its head, Probo is able to direct its gaze (eyes and head), express emotions, movements, facial expressions and is capable of verbal communication (Goris et al. 2011a). A lip-synch module allows its lips to move in time to the voice. To avoid expectations about the robot’s behavior, Probo’s appearance does not resemble a human or animal but is an imaginary entity (Saldien et al. 2008). To guarantee safe physical interaction between the robot and the child and to provide a huggable form, compliant actuation systems and a layered structure of foam and fabric are used.

A user-friendly Robot Control Center enables the operator to intuitively control Probo in a Wizard of Oz (WoZ) mode. WoZ refers to a person who remotely operates the robot, controlling a number of things along the autonomy spectrum, from fully autonomous to fully tele-operated, as well as mixed initiative interaction (Riek 2012). It is a well-known design paradigm in the human–robot interaction field and is used to elicit participants’ belief that the robot is behaving and responding autonomously. For this study, the WoZ paradigm was used in real-time, with the operator hidden behind the false wall dividing the experiment room in two.

To allow a more fluent interaction and rigorous scenario execution, specific buttons for each animation were created on the computer screen. Each button corresponded to each line of the protocol (see “Appendix”). One button usually corresponded to a sequence of verbal behaviors synchronized with non-verbal behaviors (direction of gaze, facial expressions, movements of different parts of the head, etc.) (see Fig. 3).

For the display of emotions, most of the facial movements are based on action units defined by the Facial Action Coding System developed by Ekman et al. (2013). The action units express a motion of mimic muscles as 44 categories of basic operation, with 14 expressing the six basic emotions: anger, disgust, fear, joy, sadness and surprise. Figure 4 shows the action units implemented in Probo to express the happy and sad emotions used for this study. More information about the hardware and software involved can be found in Goris et al. (2011b) and in Saldien (2009).

Fig. 4
figure 4

The happy and sad emotions of the social robot, with and without cover

Full size image

Instruments

All variables were manually coded using the program Elan—Linguistic Annotator, version 4.5 (Lausberg and Sloetjes 2009). Variables were only assessed during the task and not during the introduction or demonstration phases (see “Appendix”). The frontally recorded films, capturing the face and upper torso of the child, were used for the coding process. Two master students independently coded the recordings of the two conditions. The coders were blind to the experiment conditions. They were trained by the first author in data collection procedure. The training consisted in giving clear definitions of the dependent variables, in offering examples and non-examples for each category of behavior and in observing appropriate behavior of typically developing peers. Training continued until the inter-observer agreement reached 80 % on two successive observations An inter-observer agreement was calculated for 60 % of the recordings, with a Cohen’s kappa > .74 for each of the variables.

The extent in which preferences were detected was measured with the variable Detecting Preference (DP). The DP was defined as providing the right answer (verbal or gestural/facial expression) to the question Which fruit does Probo like? In order to provide the right answer, the child had to make a host of actions, such as following the head-turns of the interaction partner, recognizing its/her happy and sad emotions, making the difference between the facial expressions manifested for the “like”-happy emotion and “dislike”-sad emotion “. This variable was measured in frequency and scored with 1 for every correct answer. The child received eight opportunities with eight different pairs of fruit. Therefore, the DP score ranged from 0 to 8.

The quality of the child–partner interaction was examined with variables that are measuring social and asocial behaviors. Social behaviors included eye contact (EC) and initiating joint attention (IJA). Both were measured in frequency and direction (directed either at the experimenter or the interaction partner). EC was defined as looking at the face (upper region, not necessarily at the eyes) of the experimenter (EC_E) or interaction partner (EC_P) for at least 1 s. IJA was defined as the initiation of the child of coordinated attention with another partner. The IJA was scored in frequency and direction whenever the child showed and/or said something to the interaction partner (IJA_P) or to the experimenter (IJA_E). Social behaviors also included as secondary measurements positive affect (PA) and verbal utterances (VU). These variables were measured only in frequency. PA was scored for every physical gesture the child made to comfort the adult or the robot, as well as smiling during the experimental task. VU was defined as a verbal production that either expressed a complete proposition (subject + predicate) or different sounds expressed for communication that were followed by more than 2 s of silence. Asocial behaviors included non-responses (NR) and evading the task (ET). NR was defined as not responding verbally or non-verbally within 3 s to a question or a proposal made by the interaction partner. Every time the child did not answer or did not do as he/she was asked within 3 s, NR was scored. ET was defined as every behavior indicating that the child was not interested in the task or the action (e.g. suddenly standing up during the activity, doing something unrelated to the task, etc.).

Data Analysis

Analyses were performed by SPSS 20. The Wilcoxon signed-rank test was used to analyze the differences between the children’s performance in the two conditions: adult (AC) and robot (RC). Firstly, a Mann–Whitney test (for the robot-adult and the adult-robot groups) was used to investigate whether or not the order of the two conditions had an impact on children’s performances. Results showed no order effect for majority of the variables, meaning that performance did not vary in function of the order of the experimental conditions. A significant difference was obtained for the initiations of JA with the experimenter variable (M RC = 1.77, M AC = 1.27, U = 65.5, z = −2.14, p < .05) with a medium effect size r = −.39, in favor of the adult-robot group. Cohen’s guidelines for social sciences were used, which classifies effect sizes as being small when under or equal to .10, medium when equal or higher than .30, and large for values higher than .50 (Cohen 2001).

Results

Eye contact with the interaction partner differed between the two conditions, being significantly higher in the RC compared to the AC (M RC = 32.60, M AC = 25.73, z = −2.48, p < .05) with a medium effect size (r = −.45). No other significant differences were obtained for the other variables (see Table 2).

Table 2 The Wilcoxon signed-ranks test values for each of the study variables between the two conditions
Full size table

The Wilcoxon signed-rank test was performed also separately for the IJA_E variable (for which the order of the conditions proved to have an effect), in order to detect whether or not there was a significant difference between the adult and robot conditions in each of the two groups (adult-robot and robot-adult). No significant difference was found for initiating JA with the experimenter between the two conditions of each of the groups (see Table 3).

Table 3 The Wilcoxon values for the IJA with the experimenter variable, for both of the two groups (Robot_Adult and Adult_Robot)
Full size table

Discussion

In this study we addressed three research questions: (1) Does the performance of children with ASD in a social task with a robot differ from their performance in the same task with an adult? (2) Do children with ASD differ in social and asocial behaviors when interacting with an adult or a robot? And (3) Does the type of interaction partner (adult or robot) have an impact on the way how children with ASD interact with another adult present in the room? As response patterns were similar in both the robot-adult and the adult-robot groups, it the findings are summarized.

The variables used to answer the research questions were: performance in a detecting preference task, social behaviors (i.e. eye-contact, initiations of joint attention, verbal utterances, positive affect) and asocial behaviors (i.e. non-response and evading task behaviors). Regarding the performance of detecting preferences, the analyses showed that children with ASD did not differ in detecting the preferences of the robot or the adult. Consequently the first research question is answered negatively. The findings are in line with (Tapus et al. 2012; Vanderborght et al. 2012; Pop et al. 2014), who also did find that children with ASD do not perform better in the robot condition compared to the adult condition. A potential explanation for this outcome could be the fact that the task was too simple for the studied children (although the age and the IQ criteria were established in order to fit with the difficulty level of the task), As the final aim of RAT is to offer alternative approaches for some children with ASD, a similar performance with an adult or robot partner is sufficient. The aim of future RAT interventions is to reduce the workload of the therapist, not to replace the therapist by the robot. Therefore, when using a partner such as the social robot Probo who can assure a similar performance in the task, the therapist can focus on the child’s needs (i.e. guidance in case of need, prompting, comforting) and not on implementing the task itself.

When the focus of the analysis is switched to the variables selected as descriptors of the quality of interaction between the child and the other adult present in the room, a surprising and interesting result is that the robot condition did not trigger more social behaviors compared to the adult condition, excepting for one variable. Thus, an important outcome of this study is that the eye contact differed between the two conditions. The children had more eye contact with the robot than with the human partner. This result is in line with previous studies that found a significant difference in the level of eye contact in favor of a robotic partner when compared with a human partner (Dautenhahn and Werry 2004; Tapus et al. 2012; Kim et al. 2013). However, in the actual study children’s behaviors were measured only in one session, maybe after several sessions with the robot the eye contact would have decreased. Alternatively, having more eye contact with the robot than with the human could also be caused by a novelty effect. Although we exposed each participant to the robot during a familiarization phase, the novelty effect can still be present. With regard to the other social behaviors of children with ASD, no differences were found between the two conditions. These findings are interesting to discuss. Several studies found that children with ASD initiated and manifested more positive affect with a robot than with a human partner (Kim et al. 2013; Kozima et al. 2008; Robins et al. 2006, 2004). Regarding the positive affect, the way this concept was measured (by frequency) can explain the non-significant difference. We observed indeed that the children showed positive affect behaviors for longer periods of time and more constantly (i.e. a constant smile, hugging different parts of the robot) in the robot condition. Outcomes regarding the verbal utterances are in line with some previous research which reported that children with ASD manifest or a similar frequency (Kim et al. 2013) or less frequently verbal behaviors (Pop et al. 2014) with a robot. Consequently, the second research question exploring whether the robot condition elicit more social and less asocial behaviors than the adult condition is positively answered, however only regarding the eye contact. The fact that the robot condition did not elicit more social behaviors is in line with other research results (e.g., Anzalone et al. 2014; Kim et al. 2013). However, these outcomes remain surprising in what it concerns the level of engagement in the task. Mostly the idea that robot assisted tasks are more engaging and motivating for the children than traditional tasks assisted by humans is supported (Dautenhahn and Werry 2004; Diehl et al. 2012; Scassellati et al. 2012; Pop et al. 2014). Therefore, the fact that we did not find differences regarding the frequency of asocial behaviors (i.e. non-response and evading task behaviors) is surprising. A potential explanation can be the structure and standardization of the experimental task, tailored to children with autism and consequently based on a highly repetitive and predictive scenario. These asocial behaviors normally manifest when children with autism are overwhelmed by the complexity of a task (i.e. social complexity) or by a too large number of stimuli. Alternatively, an accurate explanation can be also that the task was too simple for the studied population. Consequently, this can explain why the children manifested almost no asocial behaviors during the two conditions.

Based on our analyses, the third research question was also negatively answered. The result is surprising since preliminary data have shown that when a robot is present in the interaction, not only do the social behaviors addressed to the robot increase, but also behaviors such as initiations of joint attention or verbal utterances directed to another adult present in the experiment setting (Costa et al. 2010; Kim et al.2013; Wainer et al. 2013). It is possible that the way the task was built, is a potential explanation for the results: the pre-established protocol used identically in both conditions severely limited the social behaviors that the experimenter was allowed to produce, in order to match the limited verbal capabilities of the robot.

Limitations and Future Directions

Firstly, some methodological issues need to be specified. For example, one of the limitations could be the single exposure of the participants in the two conditions. Therefore, in future longitudinal studies it could be interesting to explore in more depth whether the significant difference in eye contact obtained between the two conditions could also be obtained after several exposures. Moreover, in the future the period of the study will be extended to ensure that any novelty effect of the robot will have worn off.

Also, the technical constraints of the robot can also represent an important limitation of the study. The WoZ setup used in this study is indeed not efficient, requiring an additional human dedicated to controlling the robot. Since this is only an explorative study and not an intervention one, this issue is not yet so problematic. However, future work will focus on going beyond WoZ setups and towards robots that can operate somewhat autonomously (while of course remaining under the supervision of the therapist by using a remote), so that an operator is no longer required (Thill et al. 2012). Moreover, at present the robot used is only capable of showing facial expressions and moving its head, eyes, ears, neck and mouth. Different results may be obtained when its communication expressivity is also supported by gestures (i.e. the robot being able to move its arms and body). More social behaviors could also be elicited if the robot was capable of reacting to social advances made by the children with gestures (e.g. pointing gestures). A new version of the robot with a gestural system is currently being developed. Nevertheless, in order to reduce errors that may appear during the coding process (even if this is performed using specialized coding programs) incorporating sensors into the robot could be a useful way to register different behavior (such as eye-contact) of the children and to appropriately adapt the social behavior of the robot to the children.

However, this is one of the first controlled study, over a statistically powerful sample, to address the most commonly held assumptions in RAT field. Other strengths of this study are thatareas where no significant differences occur are presented, even when that contradicts prior work. The difference between the outcomes of this and previous studies could be explained by: (1) the positive outcomes are mostly obtained from engineer-driven work and when studies are more controlled and with more clinical guidance, the same positive outcomes cannot be statistically revealed; (2) the technological limitations of the Probo robot meant it was not perfectly suitable for triggering some of the assessed behaviors, and (3) the characteristics of the children participating were different from samples included in previous studies that had more positive results. More controlled studies are required in order to extract final conclusions about the potential added value of using robots in therapies for children with ASD.

Conclusions

This exploratory study aimed to put robotics in a clinical framework, using one of the largest sample of children with autism until now. We aimed to address three commonly held assumptions in the RAT field. Thus, we investigated whether children with ASD will perform better in a task assisted by a robot compared to the same task with a human, whether they will manifest more social behaviors in interaction with the robot than with the human and if the robot functions as a social mediator between the child and another human. We have demonstrated that for a sample population of 5–7 years old children with ASD (N = 30) with an IQ > 70, a social robot could not elicit greater performance in a detecting preference task than a human, but a similar performance. We have shown that for these children the social robot used in the study did not function as a social mediator for the child and the experimenter. Finally, the robot elicited more eye-contact behaviors than the human, but for all the other variables (initiation of joint attention, verbal utterances, positive affect, no-response and evading task behaviors) the robot had no greater impact then the human partner. These findings suggest that social robots may have potential to be good triggers for some social skills among some of the children with ASD. However, this is a research field still in its infancy. Firstly there is a need for more controlled studies, with more appropriate clinical guidance, paving the way for effective robot assisted therapies on the long term.