1 Introduction and Description of the Emerging Technology

Autism is a lifelong neurodevelopmental condition that is typically associated with difficulties in socio-communicative interactions as well as stereotyped behaviors (DSM-5 American Psychiatric Association. Diagnostic & Statistical Manual of Mental Disorders, 2013). Autistic individuals can also present with psychopathological comorbidities such as cognitive disabilities, epilepsy, anxiety disorders, and sensory problems (Müller et al., 2008). As a result, individuals with autism often face increased challenges as they try to adapt to living in a predominant neurotype society (Woods, 2017), which in turn can impact an individual’s quality of life (Hedley et al., 2017). Current estimates suggest that between 1 in 54 in the USA (Baio et al., 2018) and approximately 2% of people worldwide have autism (Roman-Urrestarazu et al., 2021). Several societal-wide obstacles exist that complicate community functioning and integration for autistic groups (Woods, 2017). Responding to this, researchers have increasingly sought to create effective and appropriate supports that can help people with autism to develop the skills needed to thrive in social situations (Rao et al., 2008). In particular, technology supports have received considerable attention, as they are believed to have useful affordances that align with the needs and strengths of autistic users (Grynszpan et al., 2014).

1.1 Autism and Virtual Reality Technology

Autistic individuals have been reported as exhibiting a natural affinity to computers (Parsons, 2016). One technology that is considered to be particularly promising and used with this population is virtual reality (VR) and Head-Mounted Displays (HMDs). As outlined by Bradley and Newbutt (2018), this specific technology is considered to be well-aligned to autistic people as it can help to provide a highly realistic and supportive environment, free of real-life consequences and other obstacles (e.g. overstimulation). An example of VR being used with autistic people is provided in Fig. 1.

Fig. 1
figure 1

Example of virtual reality being used by a young autisitc person at school (left) and a transition-aged autistic adult in a university program (right)

Full size image

While the benefits of VR systems for individuals with autism have been broadly explored, definitive reports of effectiveness remain elusive, with the promise of this technology not yet being fully realized (Parsons, 2016). This is in part due to the significant challenges of designing VR experiences for people with autism (Glaser et al., 2021; Schmidt et al., 2019), which includes the need for highly technical development skills and, in some cases, prohibitive hardware costs (Grynszpan et al., 2014; Schmidt & Glaser, 2021). Due to the challenges associated with creating fully immersive VR systems (IVR), some researchers are beginning to explore the use of spherical, video-based virtual reality (SVVR; Schmidt et al., 2019). Also referred to as 360-degree videos or immersive videos, this type of VR system places users centrally within a spherical environment in which they interact with the content and context via their head movements as captured by an attached HMD (Schmidt et al., 2019). This emerging technology could hold promise for autistic users due to simplicity, low cost and ease of use, while still providing potentially useful affordances related to immersion. Developing 360 videos is easier and requires fewer specialized skills than developing IVR experiences in digitally-modeled environments, as the primary skills required are the ability to operate a 360 video camera and the ability to prepare those videos using video editing software. SVVR therefore holds promise for the field of autism research in that the simplicity of production provides a conduit for non-experts (i.e., teachers, service providers, therapists, etc.) to actively design and deploy VR in their own context or settings. This provides an opportunity to bridge a research-to-practice gap in that practitioners, caregivers, and end-users could now have the opportunity to develop and implement immersive learning experiences for austic people.

Research suggests a need exists for a more robust evidence-base in the area of VR applied to autistic users. For example, Mesa-Gresa et al. (2018) conducted a systematic review to evaluate results related to the effectiveness of VR programs in the application to autistic youth. Thirty-one research papers (published between 2010 and 2018) were located and reviewed. Findings revealed that studies located on social and emotional skills were most common followed by daily living and then communication. These focus areas are grounded on the premise that VR allows “the emulation of everyday life situations so that scheduled training can be conducted in a therapist-controlled and safe environment” (p. 11). Despite Messa-Gresa and colleagues’ finding that there is a need to better control VR studies and to build more robust evidence to inform practice, these authors note that interventions conducted within an autistic person's home may also be of interest in future work. They go on to suggest that “this would strengthen the learning obtained during therapy and help to improve the interaction [of autisic people with their caregivers] reducing the overload and stress suffered by them” (p. 12). This last point of working in the home is something we next turn to for the possible use and application of video-based VR for autistic users.

The overarching goal of this emerging technology report, therefore, is to consider how SVVR might be a viable option when used in naturalistic settings (as opposed to clinical or research settings) to support autistic users. First, we describe the emerging technology in detail and contrast it to fully immersive VR systems which have long been used in this line of research. We then describe how the technology can contribute to learning, instruction, and assessment with an emphasis on how video-based VR might be implemented in more naturalistic settings with a specific focus on the application of evidence-based practices. Following this, we include references to projects that adopt the emerging technology and describe how they have been used. Finally, associated challenges with adoption and implementation are considered.

1.2 Description of Emerging Technology

VR systems vary greatly in terms of hardware requirements, interaction possibilities, and complexity of software development. Among these are SVVR systems, which are designed to present a spherical, 360-degree view to a user who is centrally positioned within the scene. These 360-degree videos are recordings that combine perspectives in every direction. SVVR provides three degrees of freedom (3DOF). That is, users of SVVR are able to look around and interact with the environment in a limited way by moving their heads while wearing a HMD, but are unable to directly manipulate the environment or move of their own volition. In contrast, IVR provides six degrees of freedom (6DOF), meaning that users are not only able to freely view the environment in 360 degrees but also to move within the environment and to interact with and manipulate virtual objects. Relative to IVR, possibilities for interaction are different in SVVR. Users are unable to directly navigate the virtual world and must remain in the position of the video-maker. The constraints of a 3DOF VR system render SVVR objectively less immersive than IVR (Miller & Bugnariu, 2016; Slater, 2009) in terms of both user interaction and sensory engagement (see Slater & Wilbur, 1997). Nonetheless, the fundamental characteristics of SVVR environments still hold the potential to provide sufficient photographic fidelity, realism, and immersion to help promote effective, efficient, and appealing learning experiences. This assertion is supported in part by Miller and Bugnariu’s (2016) research suggesting that even low-immersion conditions can result in measurable changes in learning for autistic individuals.

IVR environments are comprised of 3D assets that are developed using computationally-intensive 3D modelling software or computer aided design (CAD) software. This software requires high-end computing machinery and advanced video game and software development expertise (Glaser et al., 2021). In contrast, SVVR systems can be rapidly developed and deployed, as they require far fewer computational resources and, importantly, technical knowledge. 360-degree videos are captured using specialized cameras, which include low-end omnidirectional devices that can be attached to a smartphone up to high-end collections of cameras that are mounted on a rig. After capturing the videos, they are stitched together to create the 360-degree view. Because SVVR utilizes common video formats, it does not require the development of bespoke systems to deliver the videos, but can run on pre-existing platforms such as YouTube. Further, using IVR environments typically requires access to high-end computers equipped with powerful graphics cards and costly HMD (e.g., Valve Index, HTC Vive), or standalone HMD (e.g., Oculus Quest 2). Lower cost solutions such as Google Cardboard also can be used for delivering IVR experiences; however this platform’s limitations render it incapable of delivering similar experiences to more sophisticated HMD. In contrast, SVVR can be deployed across a significantly broader range of hardware with no discernable difference in user experience, from very low-end and inexpensive Google Cardboard HMDs to the high-end and costlier HMD mentioned above. It is therefore unsurprising to see a growing interest in using SVVR in educational contexts in general (Chang et al., 2020), and with autistic groups in particular (Schmidt et al., 2019).

2 Relevance for Learning, Instruction, and Assessment

Support for the implementation of SVVR in some ways can be derived from the wealth of research around video-based instruction as an evidence-based pedagogical strategy for teaching social and life opportunities for autistic individuals (Delisio & Isenhower, 2020). Video-based instruction is frequently used in the field of special education and is widely considered to be an evidence-based practice (Wong et al., 2015). Further, autistic individuals often have exceptional visual processing skills, which makes video an ideal method of content delivery (McCoy & Hermansen, 2007). In the following sections, we present how SVVR systems maintain relevance for learning, instruction, and assessment for individuals with autism with a focus on video-based instructional strategies and theories that have been found to be useful in the literature. We discuss the technology’s relevance by (1) positioning the work within video modeling, (2) discussing how video prompting strategies can be designed, and (3) discussing how theories of telepresence and immersion further support the medium.

2.1 Video Modeling

The premise of video modeling rests on the notion that learners can acquire a range of skills by watching another individual perform skills/modelling and ways to achieve successful outcomes. Video modeling typically involves a demonstration of the target skills through a pre-recorded representation of a skill that uses either the individual as the model or a peer (Bellini & Akullian, 2007). Video modeling is supported by not only empirical research (Bellini & Akullian, 2007; Bross et al., 2020; McCoy & Hermansen, 2007), but also theories such as Bandura’s social learning theory which posits that learning can take place through observing, modelling, and imitating the behaviors and reactions of others in naturalistic settings (Bandura & Walters, 1977). In alignment with this theory, through the use of video multimedia, an agent or actor is able to model the skill within authentic contexts that allow for viewers to observe the skill being demonstrated and then mimic what was observed.

2.2 Video Prompting

Autistic people tend to present a range of challenges in relation to executive function that can impact learning processes (DSM-5 American Psychiatric Association. Diagnostic & Statistical Manual of Mental Disorders, 2013). The instructional practice known as video prompting has been found useful in addressing this issue (Vries et al., 2015). Video prompting aligns well with the use of SVVR applications because video information can be designed to promote learning in individuals who may struggle with working memory (Vries et al., 2015) and recalling multi-step tasks (Berenguer et al., 2020). Within SVVR systems, a video lobby can be presented where information is broken down into short, straightforward segments (Fig. 2).

Fig. 2
figure 2

SVVR smartphone app showing buttons used to access segmented video content (Schmidt et al., 2019; used with permission)

Full size image

SVVR applications can also assist with the tendency of individuals with autism to fixate or attend to irrelevant features while receiving instruction (Shipley-Benamou et al., 2002). This is achieved by utilizing video editing techniques and advancements in SVVR development software, where the user's gaze or attention can be drawn towards or diverted to points of interest. Gaze can be influenced using less-intrusive techniques such as hotspots and visual cues or more-intrusive techniques such as shifting the user’s field-of-view to include the intended focal point. By positioning these frameworks within 360 degree videos, autistic users can make connections to social environments and can witness authentic, observable examples of skills they are working to learn. In addition, SVVR can be designed to promote individualization by including characters, scenes, and skills specifically targeted at the individual, or even perhaps video recorded by the individual.

While evidence suggests video-based intervention modalities are effective, (Bross et al., 2020; McCoy & Hermansen, 2007), research on generalization and long-term effects remains limited. Because of this, researchers have begun to explore theories of presence as a way of supporting development of skills. Presence is “the ‘sense of being there’ in the environment depicted by the virtual reality system” (Slater, 2009, p. 3551). When users don a VR headset (e.g., Google Cardboard, Oculus Quest 2), the device blocks out external visual and auditory stimuli, which can promote a sense of presence, depending on the quality of the hardware, underlying software, video resolution, video production, and other factors (e.g., Slater, 2009). By using high-definition omni-directional cameras to shoot SVVR footage, the resultant SVVR videos can then be used in conjunction with HMDs to immerse users within virtual scenes. The ability for users to experience the illusion of being present within SVVR applications implies novel and potentially effective possibilities for designing and delivering training materials for people with autism (Malihi et al., 2020).

3 Emerging Technology in Practice

Due to the relatively recent re-emergence of interest in VR in general and increasing interest in using SVVR applications for individuals with autism specifically, there is a dearth of peer-reviewed research on the matter. However a handful of preliminary findings and case studies can be found in the literature. Five of these are reviewed in the following paragraphs.

First, Gelsomini et al. (2017) describe an in-development Google Cardboard system called Wildcard that could be used to help individuals with various neurodevelopmental disorders. The authors of this study sought to explore the learning potential and possible difficulties with using wearable VR technologies. In Wildcard, various virtual environments were displayed inside a Google Cardboard that autistic users could interact with through gaze focus and direction. The authors of this project cite the low-cost, high degree of customization, and the ability to control stimuli as being critical factors in their decision to use SVVR. Findings suggest that the usability and acceptance of the technology varied between users and that there was some observable discomfort with wearing the HMD. The authors state that generalization effects in VR should be explored in the future and that there is a need to provide highly individualized approaches to autistic users.

Second, as part of a graduate student’s thesis project, Wickham (2016) describes another Google Cardboard system that was developed to promote behavioral skills concerning grocery shopping. Findings from this research indicate that fourteen participants were able to complete the training, but not without varying levels of prompting. The participants also reported largely positive feedback on the experience. However, findings regarding efficacy were not reported.

Third, a public transportation training intervention called Virtuoso also utilized SVVR technologies as part of a staged instructional strategy. In Virtuoso-SVVR, autistic users engaged with the application through a Google Cardboard or Google Daydream to view videos that demonstrated a four-step process to boarding a university shuttle. Authors cited cost, ease of development and implementation, photographic fidelity, and concerns around cybersickness as support for their use of this technology (Schmidt et al., 2019). Findings from expert review (n = 4) and a usage test with autistic users (n = 5) supported the usability, feasibility, and relevance of the system for promoting training objectives. However, like other research in this area, efficacy was not reported.

Fourth, Meindl et al. (2019) explored SVVR as a way of addressing phobias and to provide gradual exposure as a treatment mechanism. In one example, researchers developed a single-user SVVR application to gradually expose participants with autism to the process of having their blood drawn. In this system, participants would watch a 360-degree video within a Tzumi Dream Vision HMD. In the video, participants would find themselves in an immersive scenario that mirrored a doctor’s office. They would be able to look around as a doctor talked them through the procedure and began to prepare to draw their blood. When the doctor in the video began the process of actually drawing the blood then a real-world therapist would use an Apple pencil to simulate the needle on the participant's arm. The authors report that their SVVR-based solution provided benefits to traditional exposure therapy because specific environments, including medical staff and offices, can be duplicated through easy to use and cost effective 360-degree video hardware and software. They also cite the ability to control stimuli and to create individualized solutions for autistic users as a benefit of the technology. The participant who underwent exposure therapy with this intervention presented evidence of acceptance and generalization across medical providers and their willingness to have their blood drawn.

Fifth and finally, Dixon et al. (2019) describe the evaluation of a SVVR pedestrian street crossing intervention designed for children with autism. Their system utilized YouTube to deliver videos that were created using a low-end omnidirectional camera. Using an Oculus Rift, participants (n = 3) viewed a diverse array of short, 10-s video clips depicting safe and unsafe street-crossing conditions. Participants took part in multiple training sessions, which were around three to five minutes long. Findings suggest that participants were able to demonstrate mastery of the skill using SVVR. Of particular note, participants demonstrated generalization of the skill to the natural environment with no additional training outside the SVVR experience. This is remarkable, as generalization in autism VR research is a particular challenge with many calls for further research. However, as detailed in this section, early research on the use of SVVR for autistic users is promising.

4 Significant Challenges

Though the benefits of SVVR technology seem well aligned with supporting learning and development of skills for people with autism, several significant challenges exist. First and foremost, autism is considered to be a ‘spectrum’ disorder (Wing et al., 2011) which means that symptoms present differently for each individual. For example, the DSM 5 (2013) refers to differing symptomology as levels of severity, ranging from level 1 (requiring support) to level 3 (requiring very substantial support). Each severity level represents broad diversity in needs and abilities, and therefore introduces clear challenges for advanced technology integration such as SVVR. While autistic individuals do exist on a spectrum, there are still a number of socio-communicative challenges that are present across all levels of autism due to societal constraints and accessibility issues. Because of this, designers of technology-driven systems for autistic users must intentionally include the end-users to create accessible and relevant solutions that consider the unique needs of their diagnosis (e.g., Schmidt et al., 2019). Although challenges due to autistic diversity are rather obvious, perhaps less obvious are the opportunities presented by SVVR. For example, fully immersive IVR systems such as the Oculus Quest 2 are complex, require considerable skill and fluency, and introduce substantial cognitive load during usage—all of which are potential barriers for more severely impacted autistic individuals. Conversely SVVR is typically less interactive, requires fewer inputs and has more simplified controls, and can even be controlled by an external operator. The simplicity of operation and utilization of SVVR therefore could reduce barriers to utilization, thereby providing opportunities for immersive learning and associated affordances that would otherwise be unreachable.

A further challenge is that of the problematic evidence-base in the area of SVVR. To date this remains a limitation, and researchers’ understanding of SVVR’s learning affordances lag behind the more well-known advantages of IVR (e.g., Glaser & Schmidt, 2018; Schmidt et al., in-press). A need exists to explore technologies such as SVVR that can be more readily designed and deployed (Newbutt et al., 2022). A need also exists to explore possible adverse effects (i.e., cybersickness, eye strain, feeling dizzy) of working with an emerging technology and ethical challenges of working with autistic groups to decide on the most appropriate ways to use this form of technology (Schmidt et al., 2021).

A further challenge stems from the foundational use of video modeling techniques to support the implementation of SVVR technologies, and the extent to which the evidence base of using traditional video might extend to the use of 360 video. While there is some evidence of effectiveness with video modeling (Bross et al., 2020; McCoy & Hermansen, 2007), findings suggest that generalization and maintenance outcomes are often under-reported (Gunning et al., 2019). Future research is warranted in this area, as the extent to which the medium of video is flexible enough to fully support the generalization of complex skills (e.g., social and communicative) for individuals with autism remains an open question (Allen et al., 2010; Nikopoulos & Nikopoulou-Smyrni, 2008). However, the advent of no-cost SVVR development platforms which enable the modification of materials so as to create flexigle, branching scenarios holds promise. Such scenarios could add realism and adaptivity by reinforcing the variability of the real-world while also providing opportunities to individualize the application to the unique needs of each user. Issues of generalization and evidence-based design related to SVVR remain areas for future research.

5 Conclusions

This emerging technology report has briefly outlined the potential of SVVR for autistic users to support a range of skills. We have pointed to potential advantages of using SVVR and its relevance for learning, instruction, and assessment for individuals with autism. Emerging research suggests that SVVR is suitable for autistic users, with some evidence supporting the generalization of skills from the SVVR to the natural environment. SVVR empowers not only researchers but also practitioners, clinicians, caregivers, and even end-users to develop training and education that provides individual and unique experiences where people can develop a range of knowledge and skills. Due to the promising benefits of SVVR, such as cost, ease-of-use, quick and easy development, and potential effectiveness, we assert that this promising emerging technology warrants further consideration. This includes explication of perceived and actual technology affordances, identification of training and therapeutic needs that effectively leverage these affordances, and theoretically-inspired designs that meet these needs. Further, we argue that these directions for future research should be guided by the voices of the autistic users who will use the technology and solutions co-designed in collaboration with the broader autistic community.