Introduction

Difficulties with social interaction are a hallmark characteristic of autism spectrum disorder (ASD). These vulnerabilities include challenges with social-emotional reciprocity and impairments in emotion recognition and expression (American Psychiatric Association 2013; Baron-Cohen 1997). Children with ASD frequently exhibit delays and deviations in their ability to recognize emotions in themselves and others (Harms et al. 2010). Even as adults, many individuals with ASD struggle to recognize complex emotions, have trouble expressing and regulating their own emotions, and show evidence of atypical eye movements when processing emotional faces (APA 2013; Baron-Cohen 1997; Klin et al. 2002; Pelphrey et al. 2002; Samson et al. 2012). Accordingly, social cognitive accounts of autism include emotion understanding and recognition among the core deficits in autism (Hadjikhani et al. 2006).

A related construct that is important to social cognitive theories of autism is the concept of mentalizing deficits in ASD (Frith 2001). Mentalizing, a concept which emerged from the Theory of Mind literature, refers to the ability to attribute mental states, such as beliefs, thoughts, feelings, plans, and intentions, to oneself and others and to recognize that others’ mental states may be different from one’s own (Baron-Cohen et al. 1985). This understanding allows individuals to predict and explain the behaviors of others. The ability to pass tasks assessing this skill in typically developing children (TD) is associated with language development as well as emerging math and literacy skills (Blair and Razza 2007; Bloom 2002). However, studies show that mentalizing is impaired or delayed in individuals with ASD (Baron-Cohen 1997; Buitelaar et al. 1999; Frith 2001).

Because those with ASD have difficulties with emotion processing and mentalizing, they often fail to accurately interpret the dynamics of social interactions. One proposed explanation for this deficit is that individuals with ASD are less adept at social referencing, including seeking out emotional information from another’s face (Moore and Corkum 1994). Studies have shown children with autism have difficulty “reading” facial expressions, matching facial expressions with verbal messages, and comprehending emotion-laden words (Hobson 1993). Thus, atypical face processing is a trait considered by some to be a fundamental characteristic of individuals with autism (Rutishauser et al. 2013).

The deficits in face processing, emotion recognition, and mentalizing abilities observed in ASD may be interrelated. Studies utilizing multiple paradigms, including pictures of faces, eyes, social scenes, and animated objects, in addition to voices of varying intonation and pitch, indicate that individuals with ASD have difficulties interpreting more complex emotions and mental states compared to their typically developing peers (Amenta et al. 2014; Daou et al. 2014; Shic et al. 2011; Xu and Tanaka 2014). Neuroanatomical observations have shown that areas of the brain that are critical for engaging in social cognition (i.e., thinking about others’ thoughts, feelings, and intentions) are also implicated in perceiving and interpreting nonverbal social signals such as facial expressions, social gestures, and eye gaze (Hadjikhani et al. 2006; Schultz et al. 2003; Zilbovicius et al. 2013). Abnormalities in these brain regions have been shown to relate to many of the behavioral symptoms observed in ASD (Chevallier et al. 2012).

Accordingly, social cognitive theories propose that core emotion and social processing deficits observed in individuals with ASD may account for some of the observed behavioral symptoms of autism (Baron-Cohen 1997; Frith 2001). For example, problems recognizing, labeling and understanding the emotional and mental states of others, coupled with an inability to discern the appropriate empathetic and congruent response, can obstruct communication and precipitate social misunderstandings. Offering interventions that provide children with the face-processing skills requisite for mentalizing and emotion recognition may also attempt to remedy a potentially causal factor of many of the pervasive social skills deficits exhibited by children on the autism spectrum.

Computer Assisted Instruction in ASD

For those with ASD, computers and computer-assisted instruction (CAI) provide a useful method for receiving instruction and engaging in reciprocal interactions (Golan et al. 2007; Moore et al. 2000; Smith and Sung 2014). Computers are multimodal, repetitive, predictable, and consistent systems; they require fewer social demands, and they can be used at one’s own pace and difficulty level (Golan et al. 2007). CAI provides multisensory interactions, controlled and structured environments, multilevel interactive functions, and the ability to individualize instruction, all of which have been found to be successful in interventions for children with ASD (Bernard-Opitz et al. 1990; Chen and Bernard-Opitz 1993; Panyan 1984; Yamamoto and Miya 1999). In addition, these programs are often purposefully designed to create an intrinsically motivating environment, a feature that may especially appeal to children with ASD (Chen and Bernard-Opitz 1993; Heimann et al. 1995; Moore and Calvert 2000). Creating interesting learning environments involves using perceptually salient production features, such as sound effects and action, which are likely to elicit both children’s attention to information and their subsequent processing of that information (Calvert 1999). For example, CAI featuring actions or animations increases poor readers’ memory of nouns by providing a visual, iconic mode that children can use to represent content (Calvert et al. 1990).

Although CAI has been shown to be a relevant method to train and develop vocabulary knowledge and language learning for individuals with ASD (Bosseler and Massaro 2003; Whalen et al. 2010; Whalen et al. 2006; Yamamoto and Miya 1999), research on the use of CAI to teach complex social skills, such as emotion recognition or affect recognition, to individuals with disabilities is still emerging. Generally, studies have shown that a time-limited use of computer interventions with individuals with various disabilities was sufficient to teach basic recognition of emotions such as happiness, sadness, anger and fear (Blocher and Picard 2002; Bölte et al. 2006; Moore et al. 2005; Silver and Oakes 2001). More recently, results of several studies have suggested that basic and complex emotion recognition can improve with computer intervention (Golan et al. 2010; Golan and Baron-Cohen 2006; Lacava et al. 2007; Young and Posselt 2012). Notably, a randomized clinical trial found that 20 hours of training with a program called Let’s Face It! led to improvements in facial recognition and processing skills in children with ASD (Tanaka et al. 2010; Wolf et al. 2008). Another randomized control trial using an alternate computerized intervention to improve emotion processing in children with ASD found improvements in mental state identification, suggesting that it is indeed feasible to target mentalizing skills through CAI (Silver and Oakes 2001). Although CAI has been instrumental in teaching specific skills, such as emotion recognition and processing, to children with ASD, most existing programs demonstrate limited generalizability of acquired improvements to social behaviors and environments (Golan et al. 2010; Smith and Sung 2014; Young and Posselt 2012).

However, a particular CAI program, FaceSay™, has had some success teaching focused skills to children with ASD that appear to lead to more generalized improvements in social behavior. Hopkins et al. (2011) showed in an earlier randomized-control trial that children with low-functioning autism (LFA) who received a 6-week intervention with FaceSay™ showed improvements in emotion recognition and social interactions. This same study found that children with high-functioning autism (HFA) who received the FaceSay™ intervention demonstrated improvement in facial recognition, emotion recognition, and social interactions compared to the control group. In blinded observations of peer interactions on the playground, children with HFA and LFA who received the FaceSay™ intervention initiated more social interactions with their peers, made more eye contact, and exhibited fewer negative behaviors than those children in a control group. Nevertheless, further work must be done to both evaluate the utility of the FaceSay™ software as a therapeutic tool for children with ASD and to understand the mechanisms by which FaceSay™ training impacts social information processing and social cognition more broadly. The previous study did not address any other specific skills related to face processing, particularly mentalizing, nor did it look at the relations between these skills. Furthermore, it is critical to replicate findings from intervention studies in order to fully assess their efficacy in the target population.

Aims and Predictions

The present study aims to expand upon initial results concerning the efficacy of FaceSay™ as an intervention tool (Hopkins et al. 2011). It has already been demonstrated that FaceSay™ training improves facial recognition in children with autism. We aim to replicate these findings and improve upon them in several ways. First, by examining additional skills, such as mentalizing ability, this study will clarify the central impact of face processing on multiple, potentially downstream components essential to social cognition. Second, this study will more closely examine patterns of change associated with the FaceSay intervention. In addition, the present study uses teachers or student aides to administer the intervention alongside a different, educational control condition, providing a more authentic experience than before. This study will thus advance our theoretical appreciation of the interconnected nature of face processing, emotion recognition, and mentalization, as well as our practical understanding of CAI for intervening in individuals with ASD.

To this end, we evaluated five different hypotheses in a group of children receiving the FaceSay™ intervention (experimental group) and a control group that was administered a CAI intervention focused on more standard academic content, such as mathematics and reading:

Hypothesis 1

Participants in the experimental group will have a significantly higher pre-to post-intervention mean score on affect recognition as compared to participants in the control condition.

Hypothesis 2

Participants in the experimental group will have a significantly higher pre- to post-intervention mean score on mentalizing assessments as compared to participants in the control condition.

Hypothesis 3

Participants in the experimental group will have significantly lower post-intervention scores on teacher report measures assessing the participant’s social impairment as compared to participants in the control condition.

Hypothesis 4

Participants in the experimental group will have increased positive interactions with peers post-intervention based on social skills observation ratings as compared to participants in the control condition.

Hypothesis 5

Participants in the experimental group will have decreased negative interactions with peers post-intervention based on social skills observation ratings as compared to participants in the control condition.

Methods

Participants

This study was conducted with Institutional Review Board approval from the California Graduate Institute of the Chicago School of Professional Psychology. The administration of the participating school district provided written consent to conduct research and collect data at their facility. Parents of qualifying students were contacted by mail with a description of the study, parental consent and child assent forms for the students, along with stamped, self-addressed return envelopes and contact information in case they had questions about the study or required further information. After all participants were recruited, they were randomly assigned to a study group.

Participants included 31 elementary school students in Ventura County, California, ranging in age from 5 to 11 years (M = 7.77), who were eligible for special education services under the educationally-based handicapping condition of autism in California. The population was divided into 16 students receiving the experimental intervention and 15 receiving the control. 28 of the students were male. This population was 71.9 % Caucasian, 9.4 % African American, 9.4 % Hispanic, and 6.3 % Asian. Subjects received either the WISC-III or WISC-IV as a measure of cognitive functioning. All participants were considered high functioning, with FSIQ > 70 (M = 101, SD = 14.45). For additional information regarding participants, see Table 1.

Table 1 Sample characterization (means and standard deviations)
Full size table

Design and Instruments

We designed a randomized, controlled experiment to determine the effects of the FaceSay™ computer program on the ability of children with ASD to recognize emotions, understand another’s perspective, and improve their social skills in comparison to other ASD children not receiving the intervention. This study thus involved a 2 (Training) × 2 (Time) mixed factorial design. The two levels of the between-subjects factor, training, were experimental (FaceSay™ program) and control (SuccessMaker ® program, see Procedures). The two levels of the within-subjects factor, time, were pre- and post-intervention.

Materials

Intervention Materials (FaceSay™)

Once all of the pre-intervention measures were completed and just prior to beginning the computer sessions, the participants in the experimental condition underwent a brief training session on the FaceSay™ program with one of the authors on this manuscript (LR) and a paraeducator or specialty teacher in the child’s school in order to ensure that the children could access the program and navigate through the games.

In FaceSay™, various games are designed to teach specific face-processing skills for social cognition. The “Amazing Gazing” game was designed to teach children to attend to eye gaze and respond to joint attention, given that children with ASD have shown deficits in these areas, and these skills can be taught through interventions (Leekam et al. 2000; Mundy et al. 2010). Because research studies have indicated orienting difficulties to social and nonsocial stimuli, with even greater problems in response to social stimuli (Dawson et al. 1998), “Amazing Gazing” includes both social and nonsocial pictures. In the game, an avatar is surrounded by an array of objects, numbers, or faces (see Fig. 1a). The participant is asked to look at the avatar’s eyes and indicate which object, number, or face the avatar is attending to. If the participant is correct, the item will light up and a verbal reinforcement will be given (e.g., “Good job, Johnny!”); if the participant is incorrect, a verbal and visual prompt is given to indicate the correct answer.

Fig. 1
figure 1

Screenshots from three games within FaceSay. a “Amazing Gazing” asks participants to follow the eye movements of the avatar in the center and select the object to which she is attending; b “Band Aid Clinic” requires participants to complete the picture an obscured face by choosing one of the available picture matches; and c “Follow the Leader” requires participants to judge whether the two avatars shown are making the same facial expression

Full size image

Joseph and Tanaka (2003) demonstrated that children with ASD do not spontaneously view whole faces; rather, they tend to process each feature separately. The second game within FaceSay, “Band Aid Clinic,” was developed to teach more holistic strategies for face recognition by scaffolding upon more local, feature-based techniques. In the “Band Aid Clinic,” participants are asked to select the appropriate face “band aid” to fit over an obscured portion of an avatar’s face (see Fig. 1b). Band aids include pictures of features, such as eyes or a mouth, and the avatar’s face is reconstructed by identifying the correct band aid. The potential band aids from which participants can choose increase in number and similarity as the game progresses, making the game increasingly more difficult. The goal of the “Band Aid Clinic” is to encourage processing facial expressions in terms of their features (e.g., eyes and mouth) and configuration (i.e., their location on the face).

The third FaceSay™ game, “Follow the Leader,” was designed to address well-known difficulties in ASD involving recognition and identification of emotional expressions (Hobson 1986; for a review see Harms et al. 2010), especially expressions indicated only by pictures of people’s eyes (Baron-Cohen et al. 2001). This game specifically emphasizes how subtle changes in eye information can alter the perception of facial expressions and is designed to teach participants to look at and derive emotional information from the eyes. In the first level of “Follow the Leader”, the participant is asked to identify identical facial expressions and emotions by selecting “Yes” for same and “No” for different (see Fig. 1c). The similarity of the two faces increases as the game progresses. As the game levels advance, the participant is asked to make an avatar’s face match another avatar’s expression by selecting the appropriate eyes from a selection. Similarly, as the game continues, the facial expressions change and become increasingly subtle. The game thus provides practice both in more passive comparisons between facial expressions as well as more active online adjustment of an avatar’s facial expressions.

As participants began each session, the computer program “coach” would describe the activity and the participants were cued by the program to solve the tasks. Upon selecting an answer, the computer program “coach” would either praise the participant for the correct answer or provided the participant with a verbal or visual cue if the response was incorrect. Levels within the program would begin where the participant stopped during the previous session.

Control Materials (SuccessMaker®)

The control group participants received SuccessMaker ®, a set of computer-based courses used to supplement regular classroom reading instruction in grades K-8 (What Works Clearinghouse 2009). Participants in the control condition underwent training sessions with specialty teachers prior to the current study. Using adaptive lessons tailored to the participant’s reading level, SuccessMaker ® aims to improve understanding in areas such as phonological awareness, phonics, fluency, vocabulary, comprehension, and concepts of print. The courses aim to help participants develop and maintain reading skills as well as to provide opportunities for exploration, open-ended instruction and development of analytical skills. As the students interacted with the program, the computer analyzed each participant’s skill development, assigned specific segments of the program, and introduced new skills as appropriate. Individualization allowed the participant to progress on his/her own schedule. The control participants utilized SuccessMaker ® for the same time and duration as the intervention group utilized FaceSay™.

Emotion/Affect Recognition (AR)

Affect recognition was assessed using standard scores on the NEPSY-II Affect Recognition subtest (Korkman et al. 2007). This subtest is designed to assess the ability to identify affect from photographs of children’s faces. The tasks progress from affect identification to recognition and memory for affect. Low scores on this task suggest difficulties with recognition and discrimination of facial affect. Participants were given a raw score equivalent to the number of correct responses on the subtest. The raw score was then converted to a scaled score based on age norms, and this was defined as the participant’s AR score.

Mentalizing/Theory of Mind (ToM)

Raw score on the NEPSY-II Theory of Mind subtest was used to measure each participant’s mentalizing skills, which includes the ability to understand mental functions, such as belief, intention, deception, emotion, imagination, and pretending, and to understand that others have their own thoughts, ideas, and feelings that may differ from one’s own (Korkman et al. 2007). The subtest is comprised of two types of tasks designed to assess the ability to understand mental functions and another’s point of view. For the first task, children are shown pictures of social situations or read vignettes and asked questions that probe for an understanding of the characters’ points of view. For the final few items, children are required to identify someone’s emotion from pictures of faces. Participants’ scores were defined as a raw score equivalent to the number of correct responses on the subtest. The NEPSY-II has been shown to have adequate test–retest reliability (Korkman et al. 2007). For the age groups tested here, reliability coefficients range from r = 0.52 to r = 0.60 for the Affect Recognition subtest and r = 0.77 for Theory of Mind.

Social Skills Ratings

The Social Responsiveness Scale, Second edition (SRS-2; Constantino and Gruber 2002) was used as one index of social skills. The SRS-2 is a parent/teacher questionnaire consisting of 65 items that measure the type and severity of ASD-specific social deficits in children and adolescents, such that high scores indicate greater impairment. Teachers’ numerical ratings yield a total raw score, which was converted to a standard score based upon age and gender norms. Participants received a total score and scores for each of five subscales: Social Awareness, Social Cognition, Social Communication, Social Motivation, and Restricted Interests and Repetitive Behaviors.

A behavior coding scheme and rating system that has been developed and used in previous studies (Hauck et al. 1995) was adapted for use in this study. The rating scale is comprised of 14 items which assess specific social skills and are grouped according to the following general descriptions: positive interactions, in which the child spontaneously initiated and engaged in positive interactions with a peer, and negative interactions, in which the participant exhibited controlling or avoidant behaviors that discouraged social interactions.

Each time a defined behavior was observed during the observation interval, the rater placed a tally mark in the correct row and column on the observation form. Examples of positive interaction items include: “the child looks into the eyes of another child,” “the child approaches another child with a social intention,” and “the child says hello.” Examples of negative interaction items are: “the child behaves in malicious, intrusive ways toward peers (e.g., teases them),” “the child dominates other children,” “the child avoids social overtures made toward him/her by peers,” and “the child actively avoids contact by looking away from the initiator.”

Composite raw scores for both positive and negative domains were calculated. In order to ensure adequate inter-rater reliability, the observers were trained through practice sessions until 90 % agreement had been reached consistently for the coding of data between the observers and one of the authors (LR).

Procedures

Both the experimental and control groups utilized their respective software in their school setting during their scheduled time in the computer lab. Each participant attended one 25 min intervention session per week for a total of 10 weeks. The sessions followed a predictable and fixed schedule. The software was programmed to advance through each of the three games in a pre-determined order, and progress was saved at the end of each session. All three games included a number of levels of gameplay, with several trials per level. Game levels lasted 10 min at the onset of the program and up to 25 min towards the end of treatment as they progressed within the program to more complex tasks. Each participant advanced through one or two levels per week within the allotted 25 min time frame.

Each participant’s attention to and interaction with the program was carefully monitored by a paraeducator at each session for the duration of the entire study. No verbal interactions were required nor allowed between the paraeducator and participants during the sessions, other than to facilitate the participant’s attending to task. Each paraeducator was trained by the one of the researchers (LR) prior to the onset and was told to only refer the participant back to the program (e.g., “What is the program asking you to do?” “What is next?”). Each paraeducator was monitored on a weekly basis, either during or directly following scheduled sessions, to ensure the students were accessing the game and participating appropriately.

To measure the participants’ emotion recognition and mentalizing skills, pre- and post-test measures were administered by research staff in the school psychologist’s office at each school site. Each participant was assessed in a one-on-one format for approximately 20 min.

Social skills information was collected from the teachers using the SRS-2. The forms were given to the teachers directly by the examiner, along with a return envelope to ensure confidentiality and contact information in the event that they required additional instruction or support from research staff. Teachers were blinded to the participants’ training group membership (FaceSay™ treatment or control).

Social skills observations were also conducted at baseline and post-intervention by two observers on the playground during recess. These observers were also blinded to training group membership. Both observers were employed by the school district, held master’s degrees in mental health and/or education, and had experience and training in gathering observational data. The observations took place for approximately 10 min during regularly scheduled recess and lunch times. The participants were observed independently by each rater at separate time points, for a total of 20 min. For each 10-min session, the observer recorded the behaviors of a single participant. The observers maintained fairly close proximity to the participants; however, they did not interact with the participants and politely declined any overtures made towards them. The participants were informed that the observer was simply interested in watching them play if the participants questioned the observer or other adult.

Results

Primary outcome variables for analyses were pre-post difference scores in dependent variables as described previously. Correlations between these difference scores (i.e. correlations between observed changes) are presented in Table 2. Affect Recognition and Theory of Mind scores were positively correlated (r = 0.52, p < 0.01) with each other and negatively correlated with SRS-2 scores (r = −0.53, p < 0.01; r = −0.42, p < 0.05, respectively). No other correlations were found to be significant. Means and standard deviations of pre- and post- measures of all dependent variables are presented in Table 3.

Table 2 Correlations between difference scores for dependent variables for all groups
Full size table
Table 3 Means (standard deviations) of dependent variables pre- and post-intervention for all groups
Full size table

Hypothesis 1: Affect Recognition

We used an analysis-of-covariance (ANCOVA) approach in which the independent variable was Group (intervention or control), and the dependent variable was post-test score on the NEPSY Affect Recognition subtest. Pre-test NEPSY scores were entered as covariates in order to allow for individual differences prior to the intervention. No other covariates were entered.

There was a significant difference in post-test affect recognition score between the experimental and control groups after controlling for pre-test score, F(1,28) = 20.45, p < 0.001, partial η 2 = 0.42. (The adjusted Ms for the experimental and control groups were 12.59 and 8.50, respectively).

Hypothesis 2: Mentalizing

As with Hypothesis 1, we used an ANCOVA approach with post-test NEPSY Theory of Mind score as a dependent variable, Group as an independent variable, and pre-test NEPSY Theory of Mind score as a covariate.

There was a significant difference in post-test Theory of Mind score between the experimental and control groups after controlling for pre-test score, F(1,28) = 37.35, p < 0.001, partial η 2 = 0.57 (adjusted Ms: 12.39 and 16.85, respectively).

Hypothesis 3: Social Skills

We conducted another ANCOVA with post-intervention SRS-2 score as the dependent measure, Group (intervention or control) as the independent measure, and the pre-test SRS-2 scores as a covariate.

There was a significant difference in post-test SRS-2 score between the experimental and control groups after controlling for pre-test score, F(1,28) = 4.523, p < 0.05, partial η 2 = 0.14 (adjusted Ms: 67.7 and 62.3, respectively).

Hypothesis 4: Positive Interactions

An ANCOVA approach analogous to previous analyses showed no significant differences in the number of positive social skills observations between the experimental and control groups following the intervention after controlling for pre-test numbers, F(1,28) = 0.61, p > 0.05 (adjusted Ms: 6.71 and 7.61, respectively). The covariate, pre-test score was the only significant predictor of post-test positive observations, F(1,28) = 17.24, p < 0.01.

Hypothesis 5: Negative Interactions

An ANCOVA approach like those listed above found no significant difference in the number of post-test negative social skills observations between the experimental and control groups following the intervention, after controlling for pre-test numbers, F(1,28) = 0.61, p > 0.05 (adjusted Ms: 0.18 and 0.55, respectively). The covariate, pre-test score, did not significantly predict post-test negative observations either, F(1,28) = 0.627, p > 0.05. Results of this and the preceding analyses are shown in Fig. 2.

Fig. 2
figure 2

Differences in post-test means for dependent variables

Full size image

Discussion

The purpose of this study was to investigate the extent to which FaceSay™ improves affect recognition, mentalizing, and social skills in school-aged children with ASD. The results of the present study suggest that by practicing simulated activities addressing eye gaze, joint attention, emotional cognition, and facial recognition skills on a computer, participants were able to improve their ability to recognize basic emotions such as happiness, sadness, neutrality, anger, disgust and fear. Furthermore, as suggested by improvements in the NEPSY-II Theory of Mind subtest, participants in the FaceSay™ condition improved their mentalization abilities. Finally, this study indicates that training through this software program is related to a pattern of fewer autism symptoms as assessed by teacher ratings. Thus, the results of the present study support previous work demonstrating that this software improves teacher-observed social function and emotion-processing skills (Hopkins et al. 2011). In addition, significant correlations observed among changes in affect recognition ability, mentalization skills, and teacher-observed social behaviors provide evidence that FaceSay™ may impact core skills that, in turn, influence multiple domains implicated in atypical social cognition in individuals with ASD.

These findings are particularly encouraging because they demonstrate the ways that FaceSay™ may improve general elements of social functioning above and beyond those directly targeted by the intervention. The results suggest that we may be able to simultaneously improve emotion recognition, mentalizing skills, and social skills related to autism symptomatology through an intervention that primarily addresses face processing. FaceSay™ does not directly teach mentalizing, nor does it explicitly label any emotions. Rather, all of the constituent games address attention to eye gaze, joint attention bids, facial recognition, and the ability to distinguish and create emotionally valenced facial expressions in avatars. Nevertheless, the program does target mentalizing in very subtle ways. For example, joint attention bids are accompanied with questions like, “What does Rebecca want next?” that imply that others’ mental states can be deduced through an understanding of facial expressions and eye gaze.

While not evidence of a causal relationship, the relationships found between improvements in multiple domains related to social cognition offers potential insights into the structure of deficits in ASD. Face processing is a core skill for developing proficiency in social interactions, but is a relatively low-level process compared to the complex demands of live social interactions. Nevertheless, by targeting this ability alone, FaceSay™ appears to effect change in some aspects of real world social ability. Changes in Theory of Mind were strongly correlated with changes in affect recognition, and changes in both of these domains were negatively correlated with changes in SRS scores; that is, greater improvements in each of these domains were associated with decreased symptoms of autism as measured by the SRS-2. Theory of Mind, in addition to emotion recognition, may comprise an intermediary through which face processing and emotion recognition may influence the real-world demand of social situations.

Although the FaceSay™ intervention did produce observable changes in social skills, as evidenced by improvements in SRS-2 scores, unlike Hopkins et al. (2011), these changes did not translate into broader improvements in prosocial and antisocial behaviors observed on the playground. While it is possible that FaceSay intervention did not translate to more directly observable real-world behaviors, it may also be the case that the observations made here were simply not an effective measure of generalizability. Prosocial behavior requires an array of complex social and communicative skills that go beyond face processing, emotion recognition, or mentalizing (Eisenberg and Mussen 1989). Furthermore, the number of negative observations in both groups was initially quite low, especially in the context of the high variability of those outcome measures. Perhaps a more telling post-intervention measure would be to observe children’s ability to understand and respond to another person’s perspective in real-life situations.

Nevertheless, the current findings suggest that the FaceSay™ program and programs of similar design may represent promising, efficient, and cost-effective strategies for teaching affect recognition and mentalizing constructs to higher-functioning, elementary school-aged children with ASD. It further suggests that by addressing elementary face processing skills, we may be able to effect positive changes in real-world behaviors outside the direct scope of the intervention. Future work should attempt to disentangle these relationships, using more advanced statistical models to explore possible causal relationships and using additional experimental modalities, such as eye-tracking and neuroimaging, to clarify the mechanisms underlying observed changes.

Limitations and Future Directions

There are a few noteworthy limitations to the present study. Although the NEPSY-II is a standardized and relatively sophisticated measure of affect recognition, this study specifically assessed the recognition of six basic emotions from static two-dimensional representations of children’s faces. In some cases, high functioning individuals with ASD can recognize basic emotions relatively well; however, their emotion deficits become apparent when the recognition of more complex emotions and mental states is required (Adolphs et al. 2001). Generally, complex emotions, guilt for example, involve attributing a cognitive state as well as an emotion, and are more nuanced than basic emotions, such as fear. Barriers to social referencing can impede the interpretation of social dynamics; thus, further work should address the effectiveness of FaceSay™ on the ability to recognize and identify more complex emotions and mental states from static as well as dynamic (e.g., video) facial expressions. In addition, details related to the protocol surrounding FaceSay™ play (e.g., the fidelity of monitoring of the students’ attention to FaceSay™ or the number of redirections provided by the paraeducators) were, unfortunately, not recorded, reflecting the complexities of real-world deployment of FaceSay™.

Although the study involved a number of social skills assessments, only the teacher-report measure suggested some generalizability of social skills. This intervention was implemented in a computer lab rather than the classroom setting. Generalizability could be improved by providing in-person reviews of what the child learned, implementing self-monitoring techniques, or using the intervention in a more natural setting. Additionally, the use of a self-report measure may be useful in determining outcome efficacy; possibilities include assessments that tap into reduction of anxiety relative to social situations or changes in peer networks (Locke et al. 2013).

Finally, expanding the number of participants in general, and including preschool, secondary school, and specialized educational settings would greatly enhance the generalizability of results to the broader ASD population. The addition of an existing social skills intervention as a control, instead of a more general academic training condition, would strengthen claims about the specific efficacy of FaceSay™, and CAI more broadly, in improving social skills in children with ASD.

The results of this study can benefit parents, psychologists, educators, and specialists who live and work with children on the autism spectrum. As the prevalence of ASD increases, the identification of more evidence-based and cost-effective methods to augment the education of children with ASD is warranted. This study demonstrates that CAI can provide some benefits when helping children with ASD understand the social world, and future studies should continue this work.

Conclusions

Our results indicate that by practicing simulated activities addressing eye gaze, joint attention skills, emotional cognition, and facial recognition skills on the computer, students were able to significantly increase their affect recognition capabilities and mentalizing skills and reduce their teacher-observed social impairment. Although these improvements were based on standardized assessments of emotion recognition and social cognition skills, the hypotheses that social interactions in the school environment would also improve were not fully supported.

This study demonstrates that computer technology can be an effective tool for helping children with ASD understand the mental states of others. The computer software program FaceSay™ improves the ability of children with ASD to recognize emotions and understand another’s perspective, and shows great promise in enhancing these skills in the more general school environment. We hope these results will be useful for parents, psychologists, educators, and specialists who live and work with children with ASD.