Introduction

Social emotional interaction is a process underpinned by encoding and interpreting social cues, informing the generation and evaluation of social responses (Lemerise and Arsenio 2000). Emotion recognition is fundamental to social-emotional reciprocity, depending on multi-modal processing of emotional information from the face, body language and/or verbal intonations to determine the mental states of others (Banziger et al. 2009; Darwin 1965). This process enables adaptive behavioural responses aligned with social situations (Grossmann and Johnson 2007; Lemerise and Arsenio 2000).

Difficulties associated with emotion recognition processing are widely reported amongst autistic individualsFootnote 1 (Uljarevic and Hamilton 2013), even when compared with similar conditions (Berggren et al. 2016). These difficulties appear to become more apparent during tasks with increasing complexity such as recognising complex emotions or tasks with restricted response time (Clark et al. 2008; Harms et al. 2010; Nuske et al. 2013). Challenges in social emotional skills are arguably impacting on the social participation difficulties observed in autistic individuals (Tobin et al. 2014; Williams and Gray 2012).

To date, interventions aiming to remediate the emotion recognition difficulties of autistic adults have employed various modalities, including social skills groups (Spain and Blainey 2015), typically involving small groups of autistic individuals led by a facilitator, focussing on teaching skills, discussions, role playing and individualised feedback (Reichow et al. 2012). While social skills groups demonstrate promising results in improving the social cognition skills of autistic adults, including emotion recognition (Spain and Blainey 2015), most of these groups cover multiple domains of social functioning, limiting the time available for autistic adults to meaningfully absorb the social emotional content (Spain and Blainey 2015). Additionally, social skills groups can be time and resource consuming, with the group context limiting the ability of facilitators to support individualised learning (Rao et al. 2008).

Research has increasingly explored computer based interventions (CBI) as a modality for remediating emotion recognition difficulties in autism (Grossard et al. 2017; Ramdoss et al. 2012). Autistic individuals are widely reported to have a preference for CBI, underpinned by the opportunities it provides for autonomous learning in an environment with minimal distractions (Mazurek et al. 2012; Shane and Albert 2008). While CBI demonstrates effectiveness in remediating emotion recognition difficulties in autism, there is limited evidence demonstrating its efficacy in supporting the generalisation of learnt skills to real world contexts (Ramdoss et al. 2012; Tang et al. 2019a, b). Improving emotion recognition outcomes beyond the training context is arguably essential if these interventions are to contribute to meaningful social functioning changes.

Limited generalisation of skills to everyday social contexts features as a consistent outcome of studies examining the efficacy of social emotional interventions in autism (Fletcher-Watson et al. 2014). It is widely recognised that autistic individuals have difficulties in filtering and coping in everyday situations with high social demands, as a result of their difficulties with multisensory integration (Magnée et al. 2011; Wass and Porayska-Pomsta 2014). While research suggests that autistic adults demonstrate good acquisition of social emotional skills, their difficulties become increasingly more apparent in unstructured social environments (Ponnet et al. 2004; Roeyers et al. 2001; Scheeren et al. 2013). Improving our understanding of those approaches and strategies which facilitate the transfer of emotion recognition skills beyond the training context is key if these interventions are to facilitate meaningful improvements in social functioning.

Research has begun to explore the value of combining CBI for autistic individuals with other intervention approaches including complementing CBI with social skills group training or involving a facilitator as a strategy towards facilitating the generalisation of emotion recognition skills (Lopata et al. 2016; Thomeer et al. 2015). These combined facilitator-mediated approaches generally build on the skills learnt during CBI through analysing facial features and vocal intonation, modelling, imitation and role plays (Russo-Ponsaran et al. 2016). Several interventions have incorporated opportunities for autistic individuals to apply the emotion recognition skills learnt within CBI to everyday scenarios, through analysing feature films or problem solving within a group environment (Golan and Baron-Cohen 2006; Lopata et al. 2016; Thomeer et al. 2015). With facilitator support, providing direct opportunities to practice learnt social emotional skills in naturalistic and unfamiliar settings was discussed as an useful strategy for promoting skill generalisation (Rao et al. 2008). Combining CBI with facilitator-mediated interventions has resulted in greater improvements in emotion recognition skills than participating in social skills group programs alone (Golan and Baron-Cohen 2006; Lopata et al. 2016). However, research examining the potential effectiveness of combining facilitator-mediated approaches with CBI is in its infancy, with further research needed to understand the utility of these approaches in improving the emotion recognition skills of autistic individuals.

Current emotion recognition interventions in autism are generally designed to target children and adolescents, with a paucity of research focussed on designing interventions specifically for autistic adults (Ramdoss et al. 2012). The learning content of existing emotion recognition CBI largely centres on presenting static or dynamic face stimuli, requiring participants to identify emotions in a multiple choice format (Golan and Baron-Cohen 2006; Russo-Ponsaran et al. 2016). However, the learning format and content of these approaches has been criticised as being too simplistic for autistic adults, who would likely require more advanced social emotional content reflecting the complexities of social encounters in adulthood. Understanding the relevance of CBI in improving the emotion recognition skills of autistic adults depends on the evaluation of programs incorporating content more relevant to the social emotional encounters of adulthood.

The aim of this paper is to describe the development process and systematically assess the feasibility of MindChip™ (MC), an online facilitator-mediated program delivered as an adjunct to a CBI targeting emotion recognition skills, Mind Reading© (MR) (Baron-Cohen et al. 2004). The MR program was selected given that it was the only commercially available computer program targeting the emotion recognition skills of autistic adults. The Medical Research Council Framework for developing complex interventions framed the process of developing MC, outlining an iterative intervention development process for considering the available evidence, theoretical frameworks, community consultation and feasibility and pilot testing (Craig et al. 2013). Areas of feasibility were operationalised based on the focus areas outlined by Bowen et al. (2009), including the (1) preliminary effectiveness of the MC program and CBI on the social emotional outcomes of autistic adults from pre to post intervention, (2) satisfaction of the participants during the MC program and fidelity of the intervention, (3) appropriateness of the MC program prior to further evaluation in a larger experimental trial.

Design

The development of the social emotional program, MC was guided by the Medical Research Council (MRC) Framework (Craig et al. 2013), with feasibility testing informed by the focus area as described by Bowen et al. (2009). This study was undertaken in three stages. Stage 1, the development of the MC manual, guided by the MRC framework, identifying evidence, theoretical underpinning and modelling process and outcomes. Stage 2, employed a two armed randomised controlled pilot trial examining the feasibility and preliminary efficacy of MC intervention combined with the MR program (Baron-Cohen et al. 2004), in comparison with the MR program only, in targeting emotion recognition skills of autistic adults. Stage 3 revised the MC manual following the results of the pilot study. Figure 1 illustrates the three stages of this study.

Fig. 1
figure 1

MindChip™ development process based on the Medical Research Council Framework for developing and evaluating complex interventions

Full size image

Stage 1: MindChip™ Manual Development

The MC intervention design and modelling involved establishing an evidence base through conducting a systematic review and meta-analysis of the social emotional CBI in autism (Tang et al. 2019a), which also informed the measurement framework for evaluating social emotional interventions in autism. Following the review, two case controlled studies were conducted aiming to evaluate the discriminant validity of two social emotional ecologically valid assessments, the Reading in the Mind in Films test and Movie Assessment of Social Cognition (Tang et al. 2019b; Tang et al. 2020). Focus groups were conducted with 22 stakeholders including autistic youth, allied health professionals and educators, informing the content of the MC manual (Tang et al. 2018). The draft version of the MC manual was reviewed by two autistic adults and pilot tested with one non-autistic individual and two autistic adults prior to conducting the feasibility trial.

Results

Identifying the Evidence Base

A literature search was conducted with the goal of identifying existing social emotional interventions for autistic adults. Identified social emotional interventions utilised various modalities in delivering their programs including working with individuals in social skills groups and computer based platforms (Fletcher-Watson et al. 2014; Reichow et al. 2012). While some promising findings emerged in relation to the efficacy of previous social emotional interventions in remediating the social emotional skills in autism, these were limited to improvements in close generalisation outcomes (Ramdoss et al. 2012). An updated systematic review of social emotional CBI in autism, including meta-analysis evaluating the role of serious gaming design principles (Whyte et al. 2015) in moderating the transfer of skills to close and distant generalisation outcomes was conducted (Tang et al. 2019a). Findings revealed that overall existing CBI in autism had rarely engaged or considered the needs of autistic individuals in designing their programs and that incorporating serious gaming design principles improved the efficacy of these programs on distant generalisation outcomes. Combining social emotional CBI with mentoring support was indicated as a likely promising strategy in improving the design and delivery of social emotional interventions, and promoting distant generalisation outcomes.

Developing Theoretical Underpinnings

Overall findings from the literature review pointed to a need for a social emotional CBI, underpinned by a strong theoretical framework, targeting the needs and aligning with the preferences of autistic adults. In response to this need, the MC program was developed, underpinned by a theoretical framework drawing from self-efficacy, social development, self-management and person-centred practice theories. Underpinning performance mastery, increasing an individual’s self-belief in their capabilities in executing an action plan is key in successful goal attainment (Bandura 1986). Within the social development theory, Vygotsky (1978) emphasises the dialectical relationship between the individual and their social environment in promoting learning and development. Addressing both the social and emotional targets of managing a health condition, self-management advocates for approaches which maintain or adjust behaviours and emotional responses to a condition (Corbin and Strauss 1988; Lorig and Holman 2003). Action planning is a core component of self-management, enacted by the process of goal setting, and determining specific steps required in achieving a desired goal (Lenzen et al. 2017; Lorig 2006). Person-centred approaches regard the individuals perspective, values and goals, as central to the decision making process, and key in mitigating against a ‘state of incongruence’ (Kilbane and McLean 2008; Rogers 1957). The key principles of each of these theories and their application to the theoretical framework underpinning the MC program are detailed in Table 1.

Table 1 Application of self-efficacy, social development, self-management and person centred practice in the MindChip™ program
Full size table
Modelling Process and Evaluating Outcomes

The MRC framework outlines a process for developing interventions, from mapping ‘active ingredients’ to evaluating outcomes (Craig et al. 2008b; Medical Research Council, 2000). In addition to identifying the relevant literature and theoretical frameworks relating to emotion recognition CBI in autism, consultations with stakeholders were conducted with the goal of obtaining information regarding appropriate content for an emotion recognition program targeting the needs of autistic adults (Tang et al. 2018). Key stakeholders included autism researchers, and clinicians experienced in delivering social skills interventions in autism (n = 11) and autistic youth (n = 11). In summary findings from the focus group study revealed that stakeholders perceived emotion recognition interventions in autism as having two overarching goals, increasing skills and knowledge in understanding and in responding to emotions. The researchers and therapists participating in this study discussed the limitations of existing social emotional interventions, specifically stimuli drawn from static images of facial expressions, valuing interventions addressing integrating emotional cues, such as facial expressions, body language, vocal intonation and the social context. Interventions targeting responding to emotions, developing an understanding of appropriate emotion responses supported by a ‘toolbox’ of strategies for managing social situations, were highly valued by both groups. Autistic youth emphasised the importance of embedding motivational aspects in an intervention and linking social emotional content to everyday situations. Drawing from this stakeholder consultation, the intervention objectives of MC were to:

  1. 1.

    create an awareness of the various components necessary in recognising emotions;

  2. 2.

    identify personally relevant strategies for responding to everyday situations where emotion recognition is needed;

  3. 3.

    increase participants understanding of the everyday situations requiring emotion recognition;

  4. 4.

    support participants in creating and identifying opportunities for applying learnt emotion recognition skills to everyday situations; and,

  5. 5.

    foster participants’ confidence in managing everyday situations requiring emotion recognition.

Developing the MindChip™ Manual

Following a systematic review and meta-analysis of the relevant literature (Tang et al. 2019a) and stakeholder consultation (Tang et al. 2018), the MC manual was developed. Four learning tools were developed and embedded into the MC program. The first learning tool, MC describes a computer microchip containing modules pertaining to emotion recognition clues such as eye gazes, face, body, voice and contextual information. The second learning tool, Mind Reading Toolbox describes four different strategies or tools to support problem solving potentially challenging social emotional situations. This includes visual tools, compensatory approaches when a certain emotion cue may be challenging to observe or understand, verbal tools describing using appropriate questions or statements, social tools exploring individual’s support network and self-care tools, describing an action or activities to cope with overwhelming situations. The Mind Reading Lightbulb is a five step reflective process on the influence of behaviours on emotions adapted from cognitive behaviour therapy. Finally, the action planning component of the MC program is completed weekly, encouraging participants to observe or practice the social emotional tool learnt in each session. A participant’s action plan is then divided into four main levels of difficulty, providing options for selecting goals based on their level of comfort or social emotional level. An example of a level one goal could be observing emotion clues from a video clip, and a goal based in a new or unfamiliar social context is considered as a level four action plan. Figure 2 illustrates the four learning tools of the MC program.

Fig. 2
figure 2

MindChip™ program learning tools

Full size image

The initial draft of the MC manual was reviewed by two autistic adults considering the aims of the MC program, its readability and the appropriateness of the manualised activities. Raising concerns that social skills training programs could be perceived as aligning with the medical model (Kapp et al. 2013), by aiming to impact autistic symptomatology with the underlying aim of ‘normalising’ an autistic individual, these autistic adults suggested adopting a neurodiversity perspective, acknowledging both strengths and difficulties of the autistic individuals. It was suggested that this approach could be enacted in a program focussing on providing and identifying opportunities for observing typical emotion recognition processes and reflecting on alternative strategies for managing various social situations, rather than overtly teaching emotion recognition skills. Cautioning against presenting characters portraying a stereotypical image of autistic individuals, these autistic adults emphasised the importance of developing a program aiming to support the self-determination of autistic individuals, with the goal of achieving personally meaningful social goals. Achieving these goals was underpinned by an approach considering participants current social preferences and scaffolding current strengths in working towards personal goals.

Pilot testing of the 10 MC sessions with a non-autistic individual revealed a need for increased clarity in presenting aspects of the learning content, particularly in regard to interpreting contextual cues, and using verbal, social or self-care strategies. Further refinements of the manual led to the inclusion of additional visual resources and instructions. Further pilot testing of the MC program with two autistic adults evaluating the depth and acceptability of the intervention content revealed that given the delivery of the program relied on the functionality of a video conferencing platform, it was important to support participants in resolving any technical difficulties as they arose. This testing also highlighted the importance of tailoring MC to individual participants, considering their interest and communication style, in facilitating their engagement.

Development of MindChip™ Evaluation Framework

An earlier systematic review and meta-analysis of social emotional CBI in autism informed the development of a measurement framework aimed at evaluating the preliminary efficacy of the MC program (Tang et al. 2019a, b). This review identified three domains relevant to understanding the efficacy of social emotional interventions in autism, close generalisation, distal generalisation and transferability to other skills. Measures assessing aspects of close generalisation were conceptualised as those measuring task performance on emotion recognition stimuli similar to those presented within the intervention. Distal generalisation measures assessed performance of tasks different to those presented within the intervention context, including concepts such as theory of mind and recognition of emotions in realistically depicted scenarios. Measures capturing transferability outcomes evaluated skills or factors beyond the stated focus of the social emotional interventions, such as their impact on autism symptomology (Tang et al. 2019a, b). Table 2 summarises the process of modelling of the intervention objectives to the MC intervention and development of the evaluation framework.

Table 2 Modelling of MindChip intervention components and evaluation measures
Full size table
MindChip™ Content

MC is a 10 week facilitator-mediated intervention, aiming to encourage autistic individuals to observe emotions in everyday social contexts. MC emphasises developing personalised strategies for recognising emotions, rather than correctly identifying emotions per se, targeting ‘understanding emotions’ and ‘responding to emotions’ via the MC and Mind Reading Toolbox components of the program, respectively. Following an initial introductory session in week one, the subsequent 5 weeks focus on exploring the ‘MindChip’ module, comprising of five emotion recognition cues. The remaining 4 weeks cover the ‘Mind Reading Toolbox’ module, exploring strategies for managing challenging social situations. Prompting personal reflections on recognising emotions, the Mind Reading Lightbulb provides iconic representation of taught strategies. The final week of the MC program supports individuals to reflect on the content of the program and develop a personalised action plan for applying emotion recognition strategies to everyday situations.

Delivery of the MC program was supported by a series of video clips, taken from talk shows or TV shows and static pictorial representations of various emotions. Participants were assigned a mentor acting as a facilitator, supporting participants’ mind reading journey, exploring aspects of the MR program, formulating personal goals for observing or practicing emotion recognition strategies aligned with their ability and comfort levels. As a strategy for individualising the MC program for each participant, the facilitator adopted a person-centred approach by enquiring and acknowledging the social emotional strengths of each individual (Rogers 1957). Rather than imposing a set of strategies, the facilitator was encouraged to support the autistic adult to discover their own personally meaningful strategies through enquiring about their own experiences and seeking feedback. Additionally, the selection of picture and video based discussions could be adapted based on the autistic adults’ skill level, interests and preferences. Table 3 provides an outline of the MC sessions.

Table 3 MindChip™ sessions
Full size table

Stage 2: Feasibility Study

Methods

Participants and Recruitment

Recruitment for this study commenced in June 2018 and was completed in March 2019. Participants were recruited via multiple autism organisations and university institutions based in Western Australia. Advertising materials such as flyers were distributed via social media platforms, websites and email. Participants were eligible to participate if they were adults aged 18 years and above, residing in Western Australia, with sufficient English language and computer skills to use the MC and/or MR program. Participants were required to have a formal clinical diagnosis of Autism Spectrum Disorder under the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5) (American Psychiatric Association, 2013) or Autistic Disorder, Asperger’s Syndrome, or Pervasive Developmental Disorder- Otherwise Not Specified (PDD-NOS) diagnoses specified in the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) (American Psychiatric Association, 1994). Participants were required to have a stable medication regime, i.e. have not recently commenced taking new medication, nor have participated in a social skills intervention in the last 3 months. Participants were excluded if their intellectual functioning was assessed as less than 70 on the Wechsler Abbreviated Scale of Intelligence, second edition (WASI-2) (Wechsler 2011) on the Verbal Comprehension and Perceptual Reasoning Index domains or they reported a recent neurological disorders and/or an acute mental health condition, including epilepsy, stroke, traumatic brain injury, bipolar disorder and personality disorders.

Ethics

Ethical approval for this study was obtained from the University Research Ethics Board with the protocol registered with the Australian and New Zealand Clinical Trial Registry.

Demographic and IQ Data

Baseline characteristics of participants were obtained via an online questionnaire requesting participants’ demographic information, employment status and computer gaming experience. The WASI-2 was administered to obtain baseline estimation of general intellectual capacity. The WASI-2 contains four subtests, with the Block Design and Matrix Reasoning combined providing a Perceptual Reasoning Index, and the Vocabulary and Similarities, providing a Verbal Comprehension Index score. The WASI-2 demonstrated strong validity and excellent internal consistency, ranging from 0.90 to 0.92 (McCrimmon and Smith 2013).

Feasibility Related Measures

The feasibility of the MC program was assessed against the key feasibility focus areas as outlined by Bowen et al. (2009), assessing the acceptability, demand, implementation, practicality, adaptation, integration and expansion and preliminary efficacy of the intervention. Intervention satisfaction was evaluated using a questionnaire (Appendix A), scored on a Likert scale ranging from 1 (strongly disagree) to 4 (strongly agree). The questionnaire included some open-ended questions relating to their experiences with the program and suggestions for improvement. The Serious Game Scale (Appendix B) was developed to evaluate participants’ level of agreement of the MR program in applying the five serious game principles outlined in Whyte et al. (2015). Intervention fidelity was assessed using checklists completed by the facilitator, covering the core elements of the MindChip™ program such as resources, recap, weekly content and action plans. Table 4 outlines the data collection methods aligned with each key feasibility focus areas.

Table 4 Feasibility methodology based on the focus areas outlined by Bowen et al. (2009)
Full size table
Efficacy Related Measures

Assessment of the efficacy of the MC and MR programs was assessed on two main primary outcomes, close and distant generalisation (Tang et al. 2019a). Close generalisation was measured via an emotion recognition skills assessment presenting tasks in a format similar to the MR intervention, with distant generalisation measures utilising a stimuli distinctly different to the MR intervention, representing in vivo emotion recognition tasks.

The close generalisation measure for this study was derived from the Mind Reading Emotion Library (Baron-Cohen et al. 2004). This library contains a total of 412 short silent dynamic facial videos classified into 24 emotion groups. A subtest of 40 emotion videos was taken from this library, ensuring an equal representation of gender and ethnicity of actors, and the complexity and valance of displayed emotions. The videos were presented in random order. Following the presentation of each video, a four multiple choice option was presented, with one option representing the emotion portrayed in the video. Multiple choice options were randomly selected from the Mind Reading Battery. Participants were provided with a 10 s time limit to respond to each question. The Mind Reading Battery was administered with EEG and eye tracking measurements.

The Movie for the Assessment of Social Cognition (MASC)- English version is an ecological emotion recognition and theory of mind assessment (Bölte et al. 2014; Dziobek et al. 2006). In this assessment, participants are presented with a 15 min social scenario involving four friends gathering on a Saturday. The scenario is segmented into 43 video segments with each segment immediately followed by a question regarding the character’s thoughts, intentions and feelings. Eye tracking measurements were recorded while participants completed the MASC. Previous evaluation of the MASC demonstrates its ability to discriminate between autistic adults and their typically developing controls (Isaksson et al. 2019; Muller et al. 2016).

Secondary outcomes measures pertained to emotion recognition self-efficacy, autistic traits, depression and anxiety symptoms. Emotion recognition self-efficacy was measured using the Emotion Recognition Self-Efficacy Scale (ERSES) (Appendix C), adapted from an existing Self-Efficacy Scale previously employed as an outcome measure for health care interventions (Lorig et al. 1996). This scale evaluates an individuals’ self-perception of their knowledge, skills and abilities in recognising emotions. The emotion recognition skills targeted in this scale included identifying specific emotions, emotional cues (face, voice, body posture and contextual cues) and problem solving strategies in social environments. Autism traits were measured using the Social Responsiveness Scale-2 (SRS-2), consisting of 65 questions relating to social awareness, social cognition, social communication, social motivation, restricted interests and repetitive behaviour as measured on a four-point Likert scale (Constantino and Gruber 2011). The DASS provided a self-reported measure of mental health, comprising 42 statements relating to negative emotional states of depression, anxiety and stress (Lovibond and Lovibond 1995). The LSAS provided a self-report assessment of social anxiety across various daily situations (Liebowitz 1987). This assessment describe 24 situations rated on two four-point Likert scales relating to the individual’s perception of their level of anxiety (fear) and frequency in avoiding the situation (avoidance). Higher scores on the DASS and LSAS reflects poorer mental health outcomes and increased social anxiety.

Randomisation

This study adopted an open and pragmatic design. A 1:1 randomisation procedure was used based on an online random number generator allocating participants to either the intervention group (receiving both the MC and MR program) or the control group (receiving the MR program only). As a strategy to minimise contamination between the study groups, participants with familial relations were allocated to the same group (Portney and Watkins 2009). Outcome measures were administered via computer based assessments and online questionnaires.

Intervention Group

The MC program was run over a 10 week period. Each participant met with a facilitator via Zoom Video Communication for an hour each week over the 10 week study period. The facilitator was a psychologist with 8 years’ experience in the area of autism. Before the commencement of the MC program, the facilitator was provided with the manual and the PowerPoint slides and videos pertaining to each session. The facilitator attended a 4 h training session, involving a discussion regarding the overall aim of the program, reviewing the activities of each session, and attended one pilot session.

Participants receiving the MC program were provided with a package, containing a copy of the MR program and a participant workbook. Both paper and electronic copies of the participant workbook were provided. Participants required access to a computer device and an active internet connection. Participants were encouraged to use the MR program for 10 h over the 10 weeks. The hours were tracked using a tracking statistics embedded in the MR program. Technical support was provided with installing the MR program and the video conferencing software, ZOOM, on the participants’ personal computers.

Control Group

Participants allocated to the control group received the MR program only. Initial support was provided to install the MR program. Fortnightly emails were sent to each participant, regarding their progress with the program and any support need relating to the use of the MR program. Participants were encouraged to use the MR program for at least 10 h over the 10 week period, approximating 1 hour of usage per week, as recorded using the inbuilt recording statistics of the MR program.

Procedure

Participants indicating their interest in the study were provided with a participant information sheet and consent form, with all participants reminded that their participation in the study was voluntary and they had the right to withdraw at any point in time. Following the obtaining of consent, participants were sent a password protected electronic survey link via Qualtrics, containing a demographic questionnaire and the secondary outcome measures of this study. Participants were requested to complete the questionnaire, prior to their baseline assessment at a university research laboratory. At baseline, the primary outcome measures, the Mind Reading Battery and the MASC were obtained during a two and a half hours data collection session. The MASC was always administered first followed by the Mind Reading Battery. The tasks for both assessments were presented on a 22″ computer screen with the two WASI-2 subtests administered between each of these assessments. Following the baseline assessment, participants were informed of their group allocation and provided with a brief demonstration of the MR program. Participants allocated to the MC program were given a copy of the MC participant workbook (paper-based and electronic), and a re-occurring weekly booking was made to complete the MC sessions.

Following the 10 week intervention period, participants were contacted and asked to complete the post-test assessment. Participants were requested to again complete the online questionnaire, which included the outcome measures obtained in this manner at pre-test with the addition of a series of open ended questions (Appendix A) aimed at obtaining their experiences and perceptions of the programs. Participants were further requested to specify a time convenient for them to attend a session at the university laboratory to obtain post-test measures of the Mind Reading Battery and MASC. The post-test assessment was scheduled within 2 months after the 10 week intervention period. All participants were provided with a $60 shopping voucher at post-test and allowed to retain the provided copy of the MR program.

Statistical Analyses

Data was managed and analysed using SPSS Statistics Version 26 (IBM Corp 2016). Kolmogorov–Smirnov test assessed the normality of continuous variables. Baseline differences between groups in regard to demographic variables and outcome measures were compared using independent t-tests for continuous variables and Chi square test for categorical data. The assessment of preliminary efficacy of the combination of the MC and MR program in improving close and distant generalisation as well as the secondary outcomes measures were conducted using a group (MC and MR vs. MR only) by time point (baseline vs. post intervention) factorial repeated measures ANOVA. To account for differences in sample sizes and missing data, the close and distant generalisation accuracy scores were converted to percentages. Effect size calculations were expressed using partial eta squared, η2, 0.01, 0.06 and 0.14 indicating small, moderate and large effect size, respectively (Cohen 1988). Alpha levels were applied at p < 0.05.

Intent to treat analysis was applied using the Last Observation Carried Forward method, whereby the last observed score was used as post-intervention data, accounting for any missing data resulting from attrition (Streiner 2010). Per protocol analysis used data from participants completing the study as intended. Descriptive statistics summarised the process evaluation measures, evaluating the overall agreement relating to the usefulness, usability and satisfaction of the MC and MR programs. Qualitative feedback was analysed using content analysis using NVivo Version 12.

Results

A total of 30 individuals expressed their interest in participating in this study and were screened for eligibility. Two participants were excluded due to low intellectual functioning scores and three participants declined to participate in the intervention, due to scheduling commitments and disinterest in participating. Subsequently, 25 participants were randomised to the MR and MC intervention group (n = 11) and the MR only control group (n = 14). Participant enrolment, allocation and dropout rates are illustrated in Fig. 3.

Fig. 3
figure 3

CONSORT Diagram for MindChip™ and Mind Reading© feasibility study

Full size image

Between group baseline comparisons revealed no significant differences between groups in regard to age, gender, intellectual functioning scores, diagnosis, medication intake, employment status and time spent on computer games per week. The most prevalent comorbid diagnoses were anxiety and depression (n = 6) and Attention Deficit Hyperactivity Disorder (n = 5), and other medical conditions such as asthma and diabetes (n = 4). Table 5 describes the baseline characteristics of all participants. No between group differences were observed on all baseline outcome measures.

Table 5 Demographic characteristics for intervention and control group
Full size table
Feasibility Measures

Of the 25 participants completing the baseline assessment, three participants assigned to the MR only group did not complete the intervention as intended due to difficulties with installing the software and disengagement. One participant allocated to the MC and MR group withdrew from the intervention citing time constraints, with another participant allocated to this group completing the intervention as intended but failing to attend the follow up assessment. Overall, only one participant in the MR only arm reported using the program for the suggested 10 h or more, with participants averaging 4.27 h (SD = 5.48) usage, citing technical issues, disinterest in the program and competing commitments as limiting their capacity to complete the suggested 10 h. Participants in the MC and MR group completed the MR program for an average of 6.76 h (SD = 3.76), with two participants completing the program for more than 10 h. Several participants reported encountering difficulties with installing the MR program, heightening frustration with the program. Investigation of these issues revealed that they resulted from software compatibility issues, largely resolved by installing the MR program on a computer with older operating system or via an emulator.

MindChip™ Program Feedback

A total of 7 participants in the MC and MR group completed the feedback questionnaires. Participants receiving the MC and MR programs agreed that it was realistic to complete the MC program for an hour per week, over 10 weeks. All participants completed all 10 MC sessions, with some sessions rescheduled due to family and university commitments. On average participants completed the MC program within 12 weeks, indicating that the MC program was practicable. While most participants found the Zoom application easy to use (86%), several participants encountered technical issues, resulting from poor internet connection, resulting in video and audio not syncing or in disconnection.

Participants’ satisfaction ratings relating to the MC program are outlined in Fig. 4, with all participants (n = 7) agreeing that the MC program was a helpful tool for learning emotions. Eighty six percent of participants in the MC program regarded the MC program as a meaningful and relevant emotion recognition learning tool, finding the content understandable and applicable to everyday life. Participants regarded the video discussions as “fun”, finding the Mind Reading Lightbulb a useful tool in facilitating their emotion recognition reflective skills:

Fig. 4
figure 4

MindChip™ satisfaction ratings

Full size image

“I found the videos most useful, despite them being exaggerated compared to the real world, because we could explore a whole range of possible interpretations that I wouldn’t have thought of, for any of the social incidents shown there.” (Participant 15)

Participants agreed that it was helpful to have a facilitator, enjoying the informal nature of the sessions and “sensitive and responsive” approach of the mentor.

“All in all it remind me of having sessions with a good therapist that could actually help. Yes, at times it was hard, but the mentor did her best to readjust our sessions to my needs and the discussions helped motivate me and to understand the lessons.” (Participant 7)

Participants stated that the MC program enabled them to achieve their social emotional goals, increasing their awareness of emotions and social interactions, giving them more confidence in deciphering the subtleties of emotions:

“I gained more confidence and self-esteem in myself, bringing my ‘being hard on yourself’ behaviour to a minimum, grasped the concept of others’ emotions and make more friends for the future…” (Participant 9)

Overall, 86% of participants agreed that they enjoyed the MC program. However, some participants found some MC content too simplistic, especially during the first few weeks of the program, suggesting future tailoring of content to individual’s needs and knowledge:

“Some of the tools were stuff I already do based on life experience, and did not really add on to what I already knew…. Maybe a test to see where we stand before going over things we actually have a great understanding of already.” (Participant 11)

Across the MC sessions, the facilitator achieved 100% fidelity as measured via intervention fidelity checklists. Individual tailoring of sessions included enabling participants to attend the sessions with a support person present (n = 2) and establishing ‘ground rules’ including ensuring that participants were not interrupted when voicing their opinions. One participant experiencing difficulties in verbally expressing her thoughts opted to use the ‘chat’ function or instant messaging of Zoom as an alternative communication strategy. Several participants (n = 3) completed some MC sessions via their smart phone rather than on a computer, because of convenience or persistent technical issues (audio not working on computer). The facilitator reported providing regular check-ins and encouraged participants to take micro-breaks during the session as required.

Participants accessed the MC program for a total of 10 h throughout the intervention period. Based on the National Disability Insurance Agency guidelines for funding therapy sessions for autistic individuals, the cost for an hour MC session for each participant would be about $193.99 Australian Dollars (AUD) (National Disability Insurance Agency 2019a). Each MR program costs $129 AUD, however this program is no longer in production and is no longer stocked by the publisher. Thus, the total cost for each participant receiving both the MC and MR program over 10 weeks was equivalent to $3038.82 AUD.

Mind Reading© Program Feedback

Feedback of the MR program was obtained from participants in both groups (N = 19). More than half (59%) of the participants, including the group receiving both the MC and MR programs agreed that it was helpful, with 44% regarding it as a meaningful and relevant emotion recognition learning tool. The majority of the participants (83%) agreed that it was easy to use the MR program. Most participants (89%) agreed that a story underpinning the MR program would make it more motivating, with half of the participants (53%) agreeing that parallel mentoring support would improve the program. Overall, only 35% of participants were satisfied and enjoyed the MR program. Overall satisfaction ratings of the MR program are shown in Fig. 5. Participants reported technical issues, disengagement with the program and other competing commitments as limiting their capacity to complete the 10 h, suggesting low levels of success rate in implementing the MR program. Two thirds (66%) of participants thought it was realistic to use the MR program for an hour a week, over 10 weeks.

Fig. 5
figure 5

Mind Reading© satisfaction ratings

Full size image

Figure 6 summarise the serious game agreement ratings of the MR program. Only a quarter of participants agreed with the statement that “The MR program was motivating”, with most reporting feeling disinterest in completing the activities, likely as a result of the outdated nature of the content and interface:

Fig. 6
figure 6

Mind Reading© serious game agreement ratings

Full size image

“I feel that the characters used in the program are too outdated, along with the rest of the program, to really captivate users or create much motivation to keep exploring it. It is simply too old for today’s standards to be regarded as a good way to help people learn emotions.” (Participant 18)

The majority of the participants agreed that the MR program had clearly defined goals (84%) and good individualisation of features (81%), stating the program had “good intentions” and could be beneficial if updated. Participants stated that the goals were “clear and easy to understand”, enjoying the flexibility of exploring emotions within the ‘Emotions Library’ which built their knowledge. Participants valued being able to customise their own quizzes as it gave them some “creative control”. However, some participants felt that there was too much information, finding the lack of structure overwhelming:

“It was too broad. Having too many choices of what to do overwhelmed me. Personally, I feel it would be better to have a standard circuit to work through all the different tasks, rather than being given the option of doing…them all at once.” (Participant 22)

Serious Game ratings for the MR program relating to the level of difficulty, rewards and feedback were similar, with agreement scores of 47% and 57%, respectively. While participants reported enjoying the rewards system which involved collecting various items as rewards others found it uninteresting and “belittling”. Several participants reported appreciating the robot character guide who provided verbal prompts perceiving it as helpful in guiding them when they were in need of assistance:

“I do like the timer [in] first initial voice over…hearing a friendly voice at the start played more than once is very good. I also really like it whenever the GUI [Graphic User Interface] flashes during a voice over…it’s a really effective method for way finding assistance.” (Participant 4)

While for some participants the opportunity to reattempt incorrect task enabled them “not dwell on the incorrect answers”, others felt that feature increased their frustration:

“I found myself questioning and picking answers at random…when I got it wrong instead of it coming up red and showing me the right way or the right one it would get me to redo it, if I didn’t get it the first time I’m not going to get it for the next few attempts.” (Participant 23)

Other views included that the MR program was too “basic” and that it would be more appropriate to younger participants, that the emotion expressions were too “exaggerated” and were an inaccurate portrayal of the emotions. One participant remarked:

“It’s unrealistic for people to always react that way …Some of the depictions of specific of emotions were frankly so over the top, if I’d encountered them in public, I’d assume I was being taunted sarcastically.” (Participant 28)

Others valued the range of emotional expressions, particularly those embedded within everyday scenarios and depicting more subtle emotions:

“I enjoyed the facial expressions which were somewhat amusing in how over-the-top they were. The more subtle ones were helpful in that they sometimes helped me pick up on things in real people’s expressions.” (Participant 8)

Overall, participants’ satisfaction and level of engagement with the MR program was low. Participants provided several suggestions for adapting the MR program aimed at increasing its relevance to autistic adults including embedding themes relating to ‘interface updates’, a ‘storyline’, ‘goals’, ‘rewards and feedback’ and ‘level of difficulty’. These suggestions are outlined in Table 6 along with exemplar quotes.

Table 6 Participants’ suggestions for improving the Mind Reading© program
Full size table
Preliminary Efficacy

Intention-to-treat analyses (Table 7) indicated improvements across close and distant generalisation measures, self-efficacy and reductions in autistic traits over time. Significant group by time point was only observed for distant generalisation measure, F(1, 193,99) = 6.80, p = 0.02, partial η2 = 0.23, with the group receiving the MC program demonstrating greater improvement in distant generalisation skills from baseline to post-intervention (p < 0.01). Depression and anxiety outcomes did not reveal a significant main effect over time.

Table 7 Baseline and post-intervention outcome measures, MindChip™ (n = 11) and Mind Reading© only group (n = 14)
Full size table

Similarly, on protocol analyses (Table 8) revealed a significant main effect from baseline to post-intervention for close and distant generalisation measures, self-efficacy and autistic trait scores. However, group by time point interactions did not achieved significant differences. Although, a near significant trend was observed for group by time point interaction for distant generalisation skills, F(1, 135.172) = 4.44, p = 0.05, partial η2 = 0.20.

Table 8 On protocol results-Baseline and post intervention outcome measures, MindChip™ (n = 9) and Mind Reading© only group (n = 11)
Full size table

Stage 3: Manual Revision

Stage 3 summarised the main findings from the feasibility study, informing recommendations for revisions of the MC program. Overall, the MC facilitator component of the program was acceptable to participants, but limited adherence was observed for the MR program, suggesting limited feasibility in implementing MC in combination with the MR program. Findings suggested that the MC manual could be improved by adding suggestions for individualised tailoring of content, including an assessment of individual social emotional skills and adapting the content based on their perceived skills. Additionally, MC participants suggested permitting access to various methods of delivery such as utilising chat functions or mobile phone access.

Discussion

This study described the development and evaluated the feasibility and preliminary efficacy of a social emotional online facilitator-mediated program for autistic adults in a pragmatic pilot trial. The MC program was designed to address the gap in existing interventions targeting the social emotional skills of autistic adults, with the specific goal of generalising learnt skills to real life environments (Bishop-Fitzpatrick et al. 2013; Ramdoss et al. 2012; Tang et al. 2019a). Developing interventions has several dimensions of complexity, requiring careful consideration of several interacting components impacting on the standardisation, implementation and acceptability of the intervention at the societal and organisational level (Craig et al. 2008a; Kastner and Straus 2012) The MRC framework provided a useful framework in guiding the development of the MC program, enabling establishment of evidence and theoretical based understanding of the essential ingredients of the intervention, and elucidating potential logistical issues in evaluation (Craig et al. 2008a).

The key feasibility focus areas outlined by Bowen et al. (2009) guided the methodology of the present feasibility study before proceeding to larger experimental trials, mainly (1) participants’ satisfaction and perceived appropriateness of the MC and MR programs (acceptability), (2) amount of expression of interests received for the interventions (demand), (3) extend to which the program was delivered in accordance to the manual (implementation), (4) participants’ agreement on the ease or difficulty in accessing the intervention (practicality), (5) suggested improvements for the intervention to expand to a larger population (adaptation), (6) perceived fit and expansion of the intervention within the Australian disability system (integration and expansion), and (7) preliminary effects of the interventions (limited efficacy).

Findings indicated partial acceptability of the MC and MR programs. Good engagement was observed for the facilitator-mediated components of the MC program, with participants considering it a useful tool in developing their social emotional skills. In addition, recruitment rates and retention (demand) in this study suggest that the MC program was acceptable to the autistic adults. Intervention adherence appeared to improve as a result of a facilitator-mediated approach as participants in the MC program demonstrated higher retention rate (> 80%), and increased MR usage in comparison to the MR program only group. These findings are consistent with previous studies demonstrating the involvement of a facilitator has a positive influence in improving the social relationship outcomes of autistic adults in various settings (Martin et al. 2017; Siew et al. 2017).

In the MC group, the feedback ratings suggest high levels of satisfaction with involving a facilitator in supporting the social emotional learning of autistic adults. However, approximately half of the participants in the CBI only group (53%) stated they would prefer support from a facilitator when completing the MR program. Collectively, findings suggest that autistic adults have divergent social emotional learning preferences, a conclusion supported by the noted variability of functioning profiles in autism (Masi et al. 2017). Future studies could consider embedding specific questions enquiring autistic individuals’ social emotional learning preferences prior to commencing an intervention, further enabling evaluation of the acceptability of different approaches in targeting the social emotional skills of autistic adults. Feedback questions could focus on autistic adults’ perspectives on the acceptability of the facilitator mediated approach, focus of the program, and willingness to invest in the program itself.

High intervention fidelity suggests ease in implementing the manualised MC content (implementation). Slight deviations from the manual were made in relation to adapting the activities and discussions to align with the interest and goals of the participants, aligning with the principles of the person-centred framework (Rogers 1957). Findings suggested the facilitators’ intuitive observation and adaptability in individualising the learning content promoted an engaging and positive learning environment for the participants. There is further scope to improve the individualisation of the MC program, with some participants finding the content covered in the first 5 weeks too simplistic, recommending more personalisation of content based on individuals’ knowledge (adaptation). Further work with autistic individuals in co-producing the MC program, including establishing guidelines for individualising and tailoring the content, would likely improve the program. The moderate levels of satisfaction observed for the MR program activities, with participants citing outdated interface and poor integration of serious game elements as main reasons for their disengagement with the program, align with the most recent published study evaluating the MR program (LaCava et al. 2007). Acceptability is likely to improve as a result of integrating more serious gaming principles (Whyte et al. 2015).

Providing teleconferencing option was perceived as a practical option for participants to access the MC intervention, despite the technical difficulties experienced due to limited internet connection (practicality). In Australia, barriers in transportation are prevalent given its widely dispersed population, increasing the cost and inconvenience associated with accessing face-to-face interventions, especially for those living in remote and regional areas (Bradford et al. 2016). Based on the price guidelines established by the National Disability Insurance Scheme in Australia (National Disability Insurance Agency 2019b), travel claims are limited to $58 AUD per hour and to only 30–60 min depending on the individual’s geographical location, potentially requiring families to cover these excess expenses. Given MC is delivered via a teleconferencing software participating in the program incurs no travel costs contributing to the cost effectiveness and convenience of the intervention for families and service providers. Issues relating to software compatibility and the low levels of engagement observed among participants in response to the MR program suggest that in its current format, the feasibility and acceptability of this program is limited. Participants receiving the MC program reported some minor technical difficulties, resulting from poor internet connectivity, highlighting the importance of addressing this issue before expanding the reach of the MC program in the Australian context.

Preliminary efficacy results revealed that those receiving the MC program with CBI demonstrated greater improvements on the MASC. This finding suggests the potential for combining a facilitator-mediated approach with CBI for improving distant generalisation outcomes, suggesting an improvement in social emotional skills beyond the specific intervention context, arguably an important outcome in intervention evaluation (Berggren et al. 2018). The other outcome measures assessing close generalisation social emotional skills, self-efficacy, autistic traits and mental health outcomes demonstrated no significant differences between groups. These preliminary findings aligned with those of previous research demonstrating that the MR program contributes to improvements in close generalisation outcomes, consistent with previous studies (Golan and Baron-Cohen 2006; LaCava et al. 2007) which may reflect a practice effect, given that the assessment stimuli was similar to the intervention itself (Tang et al. 2019a). While further research is needed, these preliminary findings suggest that receiving the combination of the MC and MR programs likely improves the social emotional skills of autistic adults.

As this was the first evaluation of the MC program, this study sought to initially assess the preliminary efficacy of the MC program in comparison to an active control group. Given the initial promising findings of combining the MC program with a CBI, future research may now seek to elucidate the specific mechanisms facilitating the social emotional skills of autistic adults. Future studies could compare the relative contribution of the facilitator-mediated and computer based components of the MC program, and identify the most helpful aspects of the facilitator-mediated approach for autistic adults.

Limitations

Although the MC program yielded promising feasibility, the small sample size limited the statistical power for formulating definitive conclusions in regard to the relative effectiveness of the intervention arms. Additionally, a single facilitator delivered the MC program, limiting the generalisability of the findings. Further research with larger sample of participants and facilitators is warranted to increase the confirmability of the findings.

Differences in treatment compliance in the MC and MR only groups may have potentially accounted for the significant findings observed, thus limiting the certainty of the results. Disengagement was cited as the main reason for discontinuing the MR program, highlighting difficulties in implementing this program for autistic adults. In addition to improving the motivational value of CBI, future experimental trials evaluating the MC program should control for intervention dosage, enabling better comparison of the effects of facilitator support in improving generalisation outcomes.

The MASC assessment was chosen as an emotion recognition distant generalisation measure in this study, given its demonstrated sensitivity in distinguishing between autistic and non-autistic individuals (Muller et al. 2016). Additionally, the MASC assessed skills in a different context to the MC and MR, enabling assessment of possible transference of skills to real life social emotional contexts (Tang et al. 2019a). While evaluation using the MASC demonstrated positive outcomes, it is ultimately a lab-based assessment, leaving open the question as to its capacity to capture skill generalisation to real life situations. Further evaluation into the external validity of the MASC is warranted. Future research may also seek to incorporate broader measures of social emotional change, such as via observational ratings of social emotional behaviours, or assessment of social emotional aspects of adaptive functioning.

Conclusion

This paper described the development of a social emotional program for autistic adults. The MC program was developed, incorporating four theoretical perspectives and community feedback underpinning the essential mechanisms for targeting the social emotional understanding of autistic adults. Findings suggest combining the MC program with CBI demonstrated partial acceptability and feasibility as a social emotional program for autistic adults. Participants provided suggestions for improvement, including strategies for individualising social emotional content and increasing the motivational value of the CBI. The MC program combined with a CBI demonstrated preliminary efficacy for improving emotion recognition generalisation outcomes. While these early findings are promising, further research employing an adequately powered larger randomised controlled trial design is recommended to evaluate the effectiveness of the MC program.