Keywords

1 Introduction

Autism Spectrum Disorder (ASD) has two main groups of symptoms: issues related to social communication and social interaction; and repetitive patterns of behavior, interests or activities [1]. The number of children diagnosed with ASD has continued to grow in recent decades, with estimates as high as one in 68 in the US [2] and one in 100 in the UK [3] suffering from this condition. Up to 90% of autistic children have auditory, visual and/or tactile hypersensitivity [4], and SGs may provide significant assistance in moderating these hypersensitivities. In this paper, we present an SG for autistic children with auditory hypersensitivity. We have designed an adventure SG, Sinbad and the Magic Cure, to be played on mobile android devices, and investigate the following research questions:

  • What sounds appeared to be the most critical for children with ASD?

  • Would an SG, based on artistic representations and user-centered design, confronting ASD children with sound hypersensitivity to unpleasant auditory stimuli, be effective in reducing their sensitivity to sounds?

  • To what extend this adventure game is appropriate for that specific purpose?

2 Related Works

We have categorized this analysis of related work into two sections exploring the goals and effectiveness of SGs that have been developed for autistic children. This analysis relates to studies concerning efficiency of SGs in moderating sensory hypersensitivity in ASD children.

2.1 Purpose of Serious Games for Children with ASD

In the past decade, the development of SGs for children with ASD has gradually increased [5]. A large number of studies have demonstrated significant improvements in the social interaction, communication and conversation skills of children with ASD after playing SGs [6,7,8,9]. For instance, PixTalk [8] is a smart phone application, with many pictures designed to help children express their feelings and emotions by browsing and selecting images. Likewise, ECHOES [9] is an intelligent application, in which autistic children interact with an avatar verbally, non-verbally, or both via a multi-touch display.

SGs for autistic children have two main purposes: education and therapy [11, 12]. According to our previous review of SGs for ASD children [10], there is a need for more research on therapy [11, 12]. One example targets autistic children that speak very quickly, who are likely to be unintelligible, by developing their speech to become clearer and more understandable [11]. Another SG combines tangible and graphical user interfaces on a PC, aimed at enhancing children’s social and cognitive skills [12]. Although these SGs show significant promise for ASD therapy, further research is required.

2.2 Effectiveness of Serious Games in Moderating Sensory Hypersensitivity in Children with ASD

Children with ASD often struggle with Moderating Sensory Hypersensitivity (MSH) in one or multiple sensory modalities, including tactile, visual, vestibular and auditory modalities [13, 14]. Despite increased research on ASD, there is still a lack of understanding concerning the functioning of SGs when used in the context of ASD with MSH [10]. Nevertheless, to the best of our knowledge, there is only one published study in this area, focusing on vision [15]. This study used virtual reality as a sensory integration therapy (SIT), studying the relationship between sensory experiences and motor and behavioral performance. The study considers social skills training, visuomotor coordination ability assessment, and SIT. The game measures “visuomotor coordination ability”; for example, the player must break out of a virtual balloon using a tangible (real) stick. Additionally, the study has another session also created for SIT purposes. The application design was based on photos and recorded videos of an amusement park and playground. The game (EASe Funhouse Treasure HuntFootnote 1) uses 3D game technology to help ASD children and all other children diagnosed with auditory hypersensitivity, and sensory integration disorder cope with noise and improve sensory processing. Several studies have suggested the effectiveness of therapeutic games for children with ASD [16, 17], although more research is required.

3 Our Work

Although many autistic children are thought to suffer from auditory hypersensitivity [18] according to our present knowledge, no studies have discussed what the most relevant noises are, since there are large individual differences [19]. Accordingly, our research is categorized into two experiments: (1) Preliminary research which aims at determining the major characteristics of the sounds that affect sound sensitivity of children with ASD, and (2) A study which implements those sounds into a custom-designed SG, in order to explore the effectiveness of this game to familiarize and desensitize autistic children with sound hypersensitivity to critical sound cues.

3.1 Preliminary Research: Identify the Most Critical Sounds for Children with ASD

Ten male children already diagnosed with ASD, from small specialized autism organizations in Glasgow, UK, were asked to rate their perception of specific sounds using a Likert scale. Based on literature review and expert opinion (authors MP and DS), fifteen types of sounds were selected from [20], representing the most disturbing sounds according to people with auditory hypersensitivity, and considered as two independent variables; low-frequency sounds (drilling, traffic noise, dog barking) and high-frequency sounds (rattling of dishes, child crying, rustling of paper, applause, birdsong). Most of the audio resources were collected from the Simulation and Visualization library of the Glasgow School of Art, while a few audio samples were used from the Freesound websiteFootnote 2. In terms of ensuring auditor level safety, we used Reaper 2016 softwareFootnote 3 to measure all 30 sound levels to be no more than 85 dB (normal hearing) [21], whilst Adobe Audition CC 2016 was utilized for editing, such as duplicating some audio samples (doors and school bells). A series of emoji were designed through Adobe Illustrator CS6 and Photoshop CS6, to translate the Likert scale items into graphical information that children could easily understand. We developed an Audio Interactive Questionnaire (AIQ) for Mac/Windows computers via Unity 2016Footnote 4 software (Fig. 1). The order of the sounds was coded and randomized, in order to avoid any possible order effect.

Fig. 1.
figure 1

Audio Interactive Questionnaire

Full size image

The AIQ data is presented in Table 1. The auditory questionnaire determines the independent variables to be considered in the primary experiment. As indicated in the following table, seven sounds have been highlighted as those being most critical.

Table 1. Most critical Sounds for ASD children aged 8–11 years old
Full size table

Most critical sounds were:

  1. 1.

    Baby crying 2: sound of a new-born baby crying

  2. 2.

    Bell 2: sound of school bell ringing twice

  3. 3.

    Drilling 1: sound of drilling teeth

  4. 4.

    Drilling 2: sound of drilling wood

  5. 5.

    Fire engine 2: sound of three fire engine sirens

  6. 6.

    Traffic noise 1: sound of three cars passing on a highway

  7. 7.

    Train 1: sound of a moving train

These sounds were implemented in the SG described in the next sections.

3.2 Serious Game Study: Research Methods

An SG, Sinbad and the Magic Cure, was developed for Android devices. Prior to experimentation, the game was pilot-tested on typical children (n = 28) of the same age range as our target audience (8–11) at the Glasgow Science Center for three days (Fig. 2). The goal of this test was to determine the appropriate experimental duration, explore any difficulties or issues while playing the game on a tablet, assess interface accessibility by exploring the usability of touched buttons, and test the sound volume. Overall, the pilot was successful, with no technical issues or complications. Eventually, children were handed star stickers as a reward for their participation. Pilot outcomes were:

  1. 1.

    The game must be divided into three difficulty modes: easy, medium and difficult.

  2. 2.

    Control buttons need to be bigger, as they were too small for adequate control.

  3. 3.

    Sound volume was too low.

Fig. 2.
figure 2

Pilot-test at Glasgow Science Centre, 17-19/06/2017

Full size image

3.2.1 Participants

The main study was performed at a local primary school for children with special needs in Glasgow, UK. A new cohort of participants was built using seven children aged 8–11 (6 males, 1 female) with ASD and hypersensitivity to sounds but noun of them had sensitivity to visual stimuli, that were involved in our preliminary research (Sect. 3.1). The main researcher, along with the gate-keepers (teachers and teaching assistants from the school), observed the children throughout the experiment. Due to space shortage, participants were divided into two groups of three and four. Both groups were exposed successively to the same experimental treatment.

3.2.2 Materials

The game’s story and design are inspired by the Arabic fairy tale Sinbad the Sailor from 1001 NightsFootnote 5, and combine Islamic and contemporary art. The development mixes artistic and game-engine platforms such as SketchBookPro 2016Footnote 6, Photoshop CS6, Illustrator CS6 and Unity 3D 2016 software. The participants were required to play the game on a Fire tablet, 7.0 in. display, with headphones (Fig. 3).

Fig. 3.
figure 3

Sinbad and the Magic Cure, a SG for autistic children with audio hypersensitivity

Full size image

3.2.3 Procedure

Children were required to play with the SG throughout multiple sessions. Having previously obtained formal consent from their caregivers, prior to each session, the purpose of the study was explained to the children and they were informed about their rights as participants to withdraw from the session at any point. Participants were given the opportunity to ask questions or raise any concerns throughout the study. In the first session, children were asked to fill out the AIQ in the same way as for session 3.1 on a laptop for about 10 min, and they were then given tablets and headphones. Also, the aim of playing the game, and how to play it, was explained to participants. The tasks during each session were:

  1. 1.

    Mood table: children were asked how they felt prior to playing the game, in order to fill out the mood table in (Fig. 4); this step is important as it could affect participants’ answers.

    Fig. 4.
    figure 4

    Mood table

    Full size image
  2. 2.

    Gameplay: Playing our Sinbad and the Magic Cure game for approximately 25 min. The game involves six levels, each having a different number of sounds (minimum of two and maximum of four sounds per level). The aim of the game is for the avatar (Sinbad) to collect various sound objects in order to exchange them with a wicked witch for a magic cure – as a result, his friend Yasmin turns back into a girl again. The objects appear for a few seconds before vanishing and emerging again in new locations in the digital environment of the game (Fig. 5).

    Fig. 5.
    figure 5

    Group 1 during playing the game Sinbad and the Magic Cure

    Full size image
  3. 3.

    AIQ: At the end of every session, participants rated the 30 sounds based on their feeling in the same way as for session 3.1 on a laptop for about 10 min.

Every group took around 30 min in total to complete playing the game and filling out the AIQ. Finally, the children were thanked for their contribution and given a small gift (star stickers) as appreciation.

3.3 Data Analysis

A typical Likert scale (Very Happy = 1, Somewhat Happy = 2, OK = 3, Somewhat Unhappy = 4 and Not Happy at All = 5) using iconic faces as in the preliminary research (Sect. 3.1), was employed to rate the statements proposed through the questionnaires.

4 Results

Generally, we observed that participants enjoyed playing the game very much. They tended to keep playing even after the required time. Additionally, they were keen to have the game installed on their mobile devices in order to play it at home; for example, they asked when the game would be published in Google Play so that they could download it to their own devices. Indeed, they were looking forward to having more levels for the game after they finished, and kept asking whether there was a new level available for playing. They engaged well with the game during the duration of the experiment. Interestingly, although DM mentioned that he preferred to play 3D games, he nevertheless kept playing our (2D) game. Participants were competitive and enthusiastic to finish the game. Six out of seven were able to do this in approximately 20 min, and only one participant could not finish the game.

The AIQ mean and standard deviation data are presented in the Table 2. All participants, as mentioned earlier, voluntarily played the game for seven days; however, one participant (with co-morbid Down Syndrome) was excluded from data analysis since he was happy with all seven sounds and found the game too difficult to complete in the easy mode, although he never actually stopped playing. The sound tolerance of these six participants was measured on a day-to-day basis after finishing playing the SG (Table 2). Overall, the results support the suggestion that playing SGs over a long period of time can be an effective way for audio hypersensitive autistic children to develop tolerance to sounds. Interestingly, ratings for the sounds of baby crying 2, bell 2, drill 1, drill 2, and fire engine 2 decreased. In addition, traffic noise 1 statistics were unstable in contrast with train 2 ratings (Table 2). Although, none of these changes suggested a drastic variation of trend in the context of this experiment, these results remain nonetheless promising for further experimentation.

Table 2. Mean and Standard Deviation of the critical sounds
Full size table

5 Discussion and Future Work

There is much evidence to suggest that serious games provide an effective intervention therapy tool. However, more studies utilizing SGs for ASD children with moderating sensory hypersensitivity are required. Although further research is needed to explore the eventual effect of order and increase exposure to the game, this study provides evidence that using an SG in the longer term could improve sound tolerance in children with ASD. The results confirm that mobile devices may provide a suitable supportive tool for children suffering from ASD and sound hypersensitivity. Participating children reported feeling more accustomed to the sounds by the end of the experimental week. Moreover, they showed interest and engagement with the game. Indeed, as most children are accustomed to playing games on mobile devices, they become familiar with such games very quickly. One limitation of this study is the small sample size and the short duration of the study. In the future, we will repeat the experiment with larger groups for longer periods of time. We will also further develop the game, adding more levels to maximize gameplay duration. The game will also be customizable, and based on children’s needs, so that parents can select the most unpleasant sounds to implement in the game, in order to increase its effectiveness.