Keywords

1 Introduction

Children with Autism Spectrum Disorder (ASD) [1] usually exhibit certain social communication deficits in various ways, including difficulties in verbal and non-verbal communication, deficits in social-emotional reciprocity, inability in interpreting facial expressions correctly, difficulties in self-emotion control, etc. Statistical evidences also show that ASD commonly co-occurs with other mental health issues, such as anxiety disorders [2], Attention-Deficit Hyperactivity Disorder (ADHD) [3], etc. These characteristics of children with ASD, especially school-aged children, significantly affect their learning in the inclusive education setting of Hong Kong.

Virtual Reality (VR) has been considered as a promising tool to help children with ASD. The early adoption of Virtual Reality Environment (VRE) for psycho-educational training of children with ASD can be traced all the way back to the 90s [4]. The training and learning environment for children with ASD usually needs to be authentic, safe, controllable and manipulable [5], in order to meet the special learning needs of children with ASD. Virtual Reality Learning Environment (VRLE), a VRE with explicit educational objectives and pedagogical design, well meets the requirements of being used as the training and learning environment for children with ASD. With the advent and lowering costs of VR technologies in recent years, it is becoming possible to provide training and learning activities via VR to the mass population for education in general [6] and those with ASD [7].

In this paper, we present a VR-enabled system to facilitate social adaptation training for school-aged children with clinical diagnosis or suspected diagnosis of Autism Spectrum Disorders (ASD) in the inclusive education setting of Hong Kong. Six unique VR scenarios with corresponding facilitations are designed and implemented. Four of the six VR scenarios cover various social scenarios and social occasions of school life, including preparing for school in the morning, taking school bus to go school, having classes, reading and studying in the school library and buying food from the tuck shop. Besides that, one consolidation scenario helps children with ASD generalize what they have learned in the four training scenarios to other social occasions, and one relaxation scenario helps them to get used to the VR environment, learn coping skills and practice self-emotion control. The VR-enabled learning experience is delivered through a fully immersive 4-side CAVE™ installation [8] with head position and orientation tracking for perspective adaptation, which makes the scenarios extremely authentic to the children in terms of visual stimuli.

2 Related Work

Although the empirical and interdisciplinary studies on using VR and its enabling technologies for educational or therapeutic purposes only become popular in the recent years, the underlying theories and design guidelines have been proposed in the past.

Moreno [9] investigated empirically the affective factors on learning process. The author proposed a model that extends the traditional information processing model by introducing motivation, affect and self-regulation, three factors that are believed to be in close relation with our learning process. The model was then tested based on a set of studies conducted on various media, of which two studies are based on VR to deliver learning contents [10–12], the authors also investigated the interplay between emotion and learning. The novel learning model proffered in [12] is considered as one of the fundamentals in the research domain of affective learning. To study whether VR in combination with other sensory stimulations, as a media for content delivery, could induce better affective experience during learning, resulting better learning effectiveness, Kwok et al. at City University of Hong Kong used a multi-sensory multi-modal smart ambience environment called SAMAL for learning de Bono’s six-hat thinking in an undergraduate course of information management [13]. Results from this empirical study showed that the VR experience positively influences the affective experience perceived by learners, which positively influence the learners’ learning engagement, resulting in better learning effectiveness. Subsequently, Ip et al., proposed a pedagogical model for affective learning called The SAMAL model [14], which considered the incorporation of body, mind and emotion during the learning process.

In recent years, using VR and its enabling technologies for special education [15] and therapy for children with ASD has drawn a great attention of interdisciplinary scholars. Cheng et al. [16] designed a collaborative virtual learning environment to explore the efficacy of using such environments for empathy training of children with ASD. Three autistic children with relatively good verbal IQ, performance IQ, and full-scale IQ participated in the study. The virtual learning environment recreated several social occasions that could commonly appear in restaurants. Results showed that all three children improved in terms of understanding empathy after participating in the study, and such improvements could be generalised to their daily lives based on long-term evaluation and observation by their care takers. Lorenzo [17] investigated using VR as a tool to facilitate the acquisition of knowledge, improving social skills and improving performance of school tasks of children with Asperger syndrome (i.e., high functioning autism). The educational contents of this study were designed in the context of daily lives of school-aged young adolescents, and were delivered via an immersive L-shape two-screen stereoscopic projection system. Ten children from primary school and ten children from secondary school all with Asperger syndrome participated in this 40-week long study. By comparing pre- and post-assessments, the authors demonstrated the children’ significant improvements on both executive functioning and social skills. Observations from the tutors of both schools also confirmed the children’ knowledge generalisation and transferring from virtual reality environment to their daily lives. Similar positive results of using VR as a tool facilitating learning and therapy of children with ASD could also be found in [18–20]. However, none of these studies involved trials of the specifically developed virtual reality contents on relatively large number of participants with ASD. In this paper, we present our work in developing a series of virtual reality learning scenarios specifically designed for social adaptation training for school-aged children with ASD in inclusive education settings, and delivering such VR-enabled learning experience to a relatively great number of autistic children in Hong Kong.

3 Design and Methodology

3.1 Environment and Setting

The VR learning scenarios are delivered via a four-side CAVE-like immersive VR environment with head tracking for perspective adjustment. The whole software system is supported by two workstations for graphics rendering and motion tracking respectively, which are connected via a high speed local area network (LAN) for data exchange. During training sessions, the trainer is able to control the learning content via a wireless Xbox 360 game controller, while the children will mainly interact with the environment via body gestures and communication with the trainer. Figure 1 shows the overall setting of the VR software and hardware environment; Fig. 2 illustrates the four-side CAVE-like immersive VR environment.

Fig. 1.
figure 1

Overall setting and data flow of the VR software and hardware environment

Full size image
Fig. 2.
figure 2

Illustration of the four-side CAVE-like immersive VR environment

Full size image

3.2 Scenario Design

The virtual reality learning scenarios are designed to provide an authentic, safe, controllable and manipulable environment, in which the school-aged children with ASD can practice their social skills and coping skills while avoid unnecessary embarrassment. Specifically, each of the six unique learning scenarios is designed with elements that could guide the children better understand their internal emotions, express their thoughts and feelings, appropriately response to challenging social occasions, develop empathy and generalise the knowledge and skills they are expected to gain in the VR to their daily lives with the facilitation of the programme trainers. Scenario 1 simulates the preparation of going to school in the early morning. The children will experience a series of preparation steps with a checklist as the visual cue, including wake up, use the toilet, wash hands, brush teeth, all the way to get to the lobby and wait for school bus. This scenario focuses on training the children’s executive functioning [21]. Scenario 2 simulates the social occasions that could happen on the way to school and in classroom. A facial expression matching game adapted from [22] is included in this scenario, in order to let the children practice their facial expression recognition skills. The scenario also contains a series of routines that school-aged children will experience every day in an inclusive education environment.

Scenario 3 creates a virtual reality library, in which appropriate social behaviours are quite important. The children will be asked to keep quiet, look for an interesting book, share a reading seat with their peer children and check out the book at the library circulation counter. The children will also be challenged by inappropriate social behaviours of others. Scenario 4 simulates a tuck shop, from where children can purchase snacks and food. The children will be required to queue up and they may face challenging situations, such as the snack or food they choose has been sold out. Scenario 5 is designed for consolidation, in which the social skills, coping skills, other knowledge and skills the children have learned in the previous four scenarios will be tested and generalised to a new scene - having PE lesson on the playground. In scenario 6, children will be able to experience four different seasons with peaceful background sounds. This scenario functions as both an environment for new comers to experience and get used to the fully immersive VR setting, and a safe and relax place in which the children will be guided to calm down and express their internal feelings and thoughts. Figure 3 shows some of the virtual sceneries of the learning scenarios.

Fig. 3.
figure 3

The VR-enabled learning contents are delivered to children with ASD via the four-side CAVE-like immersive VR environment under the facilitation of the trainers; (a) executive functioning training and social skills training; (b) children learn to express themselves, practice coping skills and relax in scenario 6 - four seasons.

Full size image

3.3 Method

Sample.

Children from mainstream primary schools were referred to the team by the school social worker or special education needs coordinator if they have a diagnosis or a suspected diagnosis of autism spectrum disorder (ASD). The inclusion criteria specified that children must have a diagnosis or a suspected diagnosis of ASD and a full-scale Intelligence Quotient (IQ) of more than 70. All children were fluent in Cantonese or English. The children usually had poor social and communication skills and have difficulties recognizing and regulating their emotions.

20 children were recruited for pilot study that started in July 2015. There was one drop out due to inability to commit to the training. 19 children completed the training in October 2015. The children ranged in age from 6 to 9 years (n = 20, mean age: 7.00). Children were randomly assigned into one of five groups.

33 children were recruited for group 1 that started in October 2015. All 33 children completed the training in February 2016. The children ranged in age from 6 to 11 years old (n = 33, mean age: 8.67). Children were randomly assigned into one of ten groups. There were more children recruited and scheduled for group 2 and 3 which are yet to be completed.

Training Sessions.

The training consisted of 28 one-hour sessions which spread through approximately 14 weeks. Each group consisted 3-4 children. Each session is broken into three parts: (1) briefing, (2) interactive training scenario in the CAVE™, and (3) debriefing. Each session begins in the discussion zone where the trainer reviews previous concepts and introduce the learning objectives and tasks of the day. After that, children will enter the CAVE™ and work with the training individually on the virtual reality scenario for that session. After all children have completed the training, the group returns to the discussion zone to discuss on what they have just experienced and to bridge the virtual learning experience into the real life. There are ongoing observations and weekly parental feedbacks throughout the training sessions in order to monitor the children’ progress.

Measures.

All children entering the study need to complete Raven Progressive Matrices Test, a nonverbal test of analytic intelligence [23]. Parents are also required to complete Childhood Autism Spectrum Test (CAST) to assess the severity of autism spectrum symptoms in children [24].

For the pilot group, 9 other assessments were administered at pre-assessment. Children completed 6 different assessments, including the, Faces test [25], Eyes test [26], Psychoeducational Profile - 3rd Edition (PEP-3) [27], Spence Children’s Anxiety Scale-child version (SCAS-C) [28], Faux Pas Test [29] and Social Attribution Task (SAT) [30]. There were also 4 parent-report measures, including Spence Children’s Anxiety Scale-parent version (SCAS-P) [31], Children’s Communication Checklist (CCC-2) [32], and Adaptive Behavior Assessment System (ABAS-II) [33]. These assessments assess the children’s ability in emotion recognition, social perception, theory of mind, and adaptive skills. The SCAS-C and SCAS-P were administered in order to identify any possible anxiety that may inhibit their performances in the training. All of the above assessments were repeated at the end of the training program.

4 Preliminary Results

Paired-sample t-tests were conducted to compare the scores across different assessments before and after the training program. In terms of emotion recognition, there was a significant difference in the Eyes Test scores before training (M = 12.4, SD = 3.41) and after training (M = 14.3, SD = 2.50); t(15) = -2.23, p = .041, d = .635, where t denotes the t-value, indicating the test statistic, p denotes the p-value, indicating the level of marginal significance, and d denotes the Cohen’s d, indicating the effect size between the two means. There was no significant difference in the Faces Test. For affective expression, there was a significant difference in the score before training (M = 17.4, SD = 2.31) and after training (M = 19.1, SD = 3.22); t(15) = -2.87, p = .012, d = .607. For social reciprocity, there was also a significant difference in the score before training (M = 19.3, SD = 3.38) and after training (M = 21.4, SD = 3.48); t(15) = -2.52, p = .023, d = .612. There were also overall significant differences for this PEP-3 score before training (M = 36.7, SD = 5.44) and after training (M = 40.6, SD = 6.20); t(15) = -3.21, p = .006, d = .669.

5 Future Work

The empirical study and its preliminary results presented in this paper clearly demonstrate the great potentials of using VR and its enabling technologies as a tool to facilitate social adaptation training for school-aged children with ASD. The study will continue and the scenario design, accompanied by the training protocols, will be further polished in order to better serve the psycho-educational purposes and to achieve better results. We expect the number of beneficiaries could pass 100 by the end of this study. Under the framework of SAMAL model and affective learning, we also plan to further expand and adapt the VR-enabled psycho-educational contents to cover other aspects of training and learning for children with ASD.