Keywords

1 Introduction

Autism spectrum disorders (ASD), characterized by deficits in communication and social interaction, consist of a range of neurodevelopmental disorders [1]. Although not considered as the core symptoms of ASD, motor deficits and atypical handedness have been documented in a number of reports [2,3,4,5]. Several studies have shown higher incidence of non-right-handedness (including left handedness and ambiguous handedness) in children with ASD as compared to their typically developing (TD) peers [6, 7]. Handedness, as an indication of cerebral lateralization, suggests the link between the development of the dominant hemisphere and functional skills [8]. Generally, the left hemisphere is predominant in motor and language skills. A growing number of literature has suggested that non-right-handedness is associated with several disorders (e.g., language disorders, developmental learning disorders and poor motor functioning) [5, 9, 10]. In addition, children with ASD have been found to display a dissociation of hand preference and skill such that they prefer to use the hand which is less skilled [11, 12].

Handedness is always assessed by handedness measure tests, such as the Almli Handedness Assessment [13] and the Hand Preference Demonstration Test [14], or questionnaires. However, these methods usually involve laborious work or subjective evaluation. The technology-assisted systems applied in multiple types of ASD interventions [15, 16] suggest the potential use of these systems as the novel intervention platforms to engage children with ASD in an interesting, low-cost, efficient and objective intervention environment with real-time feedback. Especially, the applications based on tablet systems grow exponentially with the advantages of providing convenient learning and training environments [17]. As far as we know, few technology-assisted systems are available for hand preference assessment and hand skill training of children with ASD [18, 19]. In this context, we focus on developing a tablet game system that can motivate children with ASD to perform tasks requiring hand manipulations as well as record objective performance measures for hand preference and hand skill evaluation.

In this paper, we propose a tablet game system on the Android platform that aims to assess the hand usage of children with ASD in collaborative games. Here the term “collaborative” refers to cooperation between two hands as well as between two players. To play these games, the players should coordinate the manipulations of two hands (of one/two players) to hit or avoid contacting the moving bubbles in the game space, which requires precise and quick hand movement manipulations, eye-hand coordination skill as well as interaction skills to communicate and cooperate with the partners. The system is capable of collecting objective and quantitative performance data of the players, which are used to analyze the hand usage of the players in the games. We expect that this system would provide an effective and efficient method to evaluate the hand usage and hand preference of children with ASD, and eventually enhance the hand skills and interaction skills among the children with ASD through the collaborative hand-control games.

The rest of this paper is organized as follows. Section 2 introduces the design and implementation of the system. Section 3 describes the modes of playing the games in this system. Section 4 presents a small usability study. Section 5 discusses the study results followed by the summary of contributions and future work in Sect. 6.

2 System Design

The tablet game system was developed using Unity 3D [20]. The core of the system is the Bubble Game as shown in Fig. 1. In the game, a number of blue bubbles and/or pink bubbles randomly move in the game space. The players are required to manipulate the collaborative tool by placing fingers on the touch plates to make the pin (the red point) of the tool hit the blue bubbles and simultaneously avoid contacting the pink bubbles, in order to achieve higher scores within a given time. Since the type, number, movement speed and direction of the bubbles in the game space could vary, the players should schedule their manipulations in an optimal order and adjust these manipulations flexibly. For example, when two blue bubbles move across the game space in different speeds, the player(s) should decide which bubble to hit first considering the distance of each bubble from the cross point, the distance of each bubble from the border and the manipulation efficiency.

Fig. 1.
figure 1

The Bubble Game interface. (Color figure online)

Full size image

The Bubble Game was implemented with five major modules as represented by the block diagram in Fig. 2. The Main Controller addresses the communication and synchronization of all the components that comprise this game. In the following sections, we will describe the other five modules in detail.

Fig. 2.
figure 2

The Bubble Game architecture.

Full size image

2.1 Bubble Manager

The Bubble Manager controls the behaviors of the bubbles by adjusting a number of bubble parameters. The parameters for one bubble are listed below:

  • Color (c): it can be blue or pink. The blue bubble is a rewarding bubble that can improve the score when being hit, while hitting the pink bubble will reduce the score;

  • Speed (v): a bubble’s movement speed is randomly assigned from a specified speed range \( (v_{min} , v_{max} ) \);

  • Start Point \( (x_{0} ,y_{0} ) \): it is the location from which the bubble moves into the game space. Thus the start point will be near the borders of the game space (11.74 × 11.74). With the origin of the coordinate frame at the center of the game space, the values for the coordinate \( (x_{0} ,y_{0} ) \) of the start point are randomly chosen from \( ( \pm 6, range\left( { - 6, 6} \right)) \) or \( (range\left( { - 6, 6} \right), \pm 6) \). For example, if \( \left( {x_{0} ,y_{0} } \right) = ( - 6, 0) \), the bubble will enter the game space from the center of the left border;

  • Destination Point \( \left( {x_{d} ,y_{d} } \right) \): it is the location toward which the bubble moves. Destination point and start point together decide the motion trail of one bubble. For the blue bubbles, the destination points are located near the opposite border of the start point to make sure that the blue bubbles move across the game space and the players will have certain time to hit the blue bubbles. For instance, if the start point locates at \( \left( {x_{0} ,y_{0} } \right) = ( - 6,0) \), the coordinate of the destination point will be \( \left( {x_{d} ,y_{d} } \right) = (6, range( - 6, 6)) \) near the right border. For all the pink bubbles, there is only one destination point which is the location of the pin of the collaborative tool when the pink bubbles are spawned. The players thus need to be alert and protect the pin from the approaching pink bubbles.

Except for the above individual parameters, several group parameters are developed to make bubbles spawn in an orderly manner. These parameters are explained as follows:

  • Spawn period (t s ): it is the period for spawning a wave of bubbles;

  • Spawn interval (t i ): it is the interval between spawning a bubble in a wave;

  • Blue number (N b ): it is the number of blue bubbles in a spawn wave;

  • Pink number (N p ): it is the number of pink bubbles in a spawn wave. So the total number of the bubbles in a spawn wave will be \( (N_{b} + N_{p} ) \);

The Bubble Manager also performs the interaction logic of the bubble with the pin of the collaborative tool. When the pin touches one bubble, the bubble will burst and disappear from the game space. Accordingly, the player’s score will increase (hitting the blue bubble) or decrease (hitting the pink bubble) by Fig. 1.

2.2 Touch Manager

The Touch Manger obtains the input data in terms of finger location on the tablet screen and uses the data to control the collaborative tool. The collaborative tool consists of two touch plates (left touch plate and right touch plate), each of which is allowed to move along one direction within the game space (horizontal direction: \( x \in ( - 5.72, 5.72) \) or vertical direction: \( y \in ( - 5.72, 5.72)) \). Once the location of the finger touching the screen falls within the area of one touch plate, the touch plate will move following the finger movement in horizontal or vertical direction. For instance, when one finger touches the left touch plate, the horizontal location of the left touch plate will be same as that of the finger, while its vertical location will not change.

2.3 Text and Audio Manger

To make the game more lively and understandable, we added some simple visual and auditory feedback. For instance, when one blue bubble is touched, a “+1” text is displayed in the game space and a bubble-burst audio is played to tell the players that they have scored a point (Fig. 3). Similarly, when one pink bubble is contacted, a “−1” text is shown. However, a different bubble-burst audio is played to warn the players that they touched the dangerous pink bubbles.

Fig. 3.
figure 3

An example of the visual feedback. The “+1” text would display (in the right picture) when one blue bubble (the blue bubble near the pin in the left picture) is hit. (Color figure online)

Full size image

2.4 Configuration Module

The Configuration Module specifies the game settings, such as the bubble parameters mentioned in Sect. 2.1. The data about the game settings and codes for game configuration update are saved and executed in this module. By adjusting the bubble parameters in this module, we developed and used three types of Bubble Games in the usability study: (1) Blue Bubble Game: only blue bubbles exist in the game space and the players are expected to hit all the blue bubbles; (2) Pink Bubble Game: only pink bubbles exist in the game space and the players are expected to avoid contacting all the pink bubbles; (3) Blue-Pink Bubble Game: both blue and pink bubbles exist in the game space. Every game lasts 2 min. Table 1 shows the detailed information about the game configurations for these three games.

Table 1. Game configurations
Full size table

2.5 Data Logging Module

The Data Logging Module records game-related data and generates performance measures for offline analysis. These measures include:

  • Score (S): it is computed as (the number of touched blue bubbles + the number of avoided pink bubbles)/(the total number of produced blue bubbles + the total number of produced pink bubbles) in one game. For example, we assume that in one Blue Bubble Game 98 blue bubbles and 49 pink bubbles were produced. The player successfully hit 80 blue bubbles and avoided touching 40 pink bubbles (touching 9 pink bubbles). The final score for this player would be (80 + 40)/(98 + 49) = 0.81;

  • Distances per motion \( (d_{l} ,d_{r} ) \): it is computed as (the sum of the displacements of left/right touch plate)/(the movement times of the left/right touch plate) in one game. For instance, we assume that a player moved the left touch plate 34 times with the total movement displacements equaling 366.74. Then the average left distance per motion was d l  = 366.74/34 = 10.79. The average distances per motion reflect how many efforts the player spends during each motion manipulation.

3 Play Modes

Every Bubble Game can be played by one player or two players. For one player, he/she should use two hands, each of which controls one touch plate of the collaborative tool. Due to the influence of the handedness, we hypothesize that movement manipulations would be easier to perform using the dominant hand rather than the non-dominant hand. The players would spend more time and greater efforts to complete non-dominant hand manipulations though two hands are supposed to undertake equal work. For two players, each player is allowed to control one touch plate using either the dominant hand or the non-dominant hand. In this mode, the players’ performances are affected by the usage of hand as well as the communicative and collaborative skills.

4 Usability Study

4.1 Participants

We recruited 4 children with ASD and 4 TD children for this study. All the participants were right-handed. They were divided into 4 ASD-TD pairs as shown in Table 2. Every participant was given a $25 gift card as rewards. The study was approved by the Vanderbilt University Institutional Review Board. Both sets of experiments were conducted after obtaining the assent of the participants, consents from their parents and under the supervision of trained ASD therapists and experimenters.

Table 2. Participants’ characteristics
Full size table

4.2 Experimental Procedure

The main procedure for one single experiment with one pair of participants is shown in Fig. 4. First, the paired participants separately played the three types of Bubble Game alone in different experimental rooms. As mentioned before, each of them used two hands to play these games. Next, two participants came to the same room and played the Blue-Pink Bubble Game together on one tablet. They were required to use their dominant hands first and then use their non-dominant hands to play the same game. At the end of the experiment, participants completed a survey with two questions to express their feedback regarding the experience of using this system.

Fig. 4.
figure 4

The experimental procedure.

Full size image

5 Results and Discussions

All the participants completed the entire experiment. They could understand the game rules quickly and figure out how to play the games easily with a small amount of practice. Participants’ answers to the survey showed that most of the participants (6 out of 8) liked the games “very much”, while the other two participants (one participant with ASD and one TD participant) liked the games “a little”. From our observation, the participants were engaged in the games and seemed to enjoy the entire experiment. In addition, all the participants preferred to play with a partner, except for one child with ASD who preferred to play by himself. From the game video records, we found that all the paired participants would talk with their partners spontaneously even though they did not know each other before. And they talked more when they played the second two-player game.

The performance results are shown in Fig. 5. First, we can see that on the average participants played better in Pink Bubble Games than in Blue Bubble Games. It is reasonable since it is easier to avoid touching bubbles than to hit bubbles. When the games became more difficult and complex in Blue-Pink Bubble Games, both the average scores of participants with ASD and TD participants decreased a little bit. However, when they played the same games in the two-player mode, their scores increased a little bit even in the game requiring to use non-dominant hand. It might indicate that it was easier for participants to perform one-hand manipulations. And good communication and cooperation might also contribute to the increased scores.

Fig. 5.
figure 5

Performance results of participants. The top graph shows the average scores and the bottom graph shows the average distances per motion of each hand in different games.

Full size image

Second, participants spent great efforts to perform non-dominant hand (left hand) manipulations in the one-player mode. The Wilcoxon Signed-ranks Tests indicated that the distances per motion of non-dominant hand for participants with ASD (mean = 8.527, median = 9.021) were significantly greater than those of dominant hand (mean = 4.937. median = 4.014), W = 78, Z = 3.059, p < .001, r = 0.624. The difference regarding the distances per motion between non-dominant hand (mean = 5.985, median = 6.176) and dominant hand (mean = 4.232, median = 3.7) for TD participants was also significant, W = 69, Z = 2.353, p = .019, r = 0.480. In addition, no significant difference was found between the distances per motion of non-dominant hand for participants with ASD and for TD participants (W = 61, Z = 1.726, p = .084, r = 0.352), as well as regarding the dominant hand (W = 53, Z = 1.098, p = .272, r = 0.224).

By computing the Pearson correlation coefficients, we found that the non-dominant hand and dominant hand of participants with ASD had strong positive relationship (r(10) = .716), while the relationship for TD participants was weak and positive (r(10) = .201). It indicated that participants with ASD tended to increase the intensity of one hand’s manipulations as the other hand’s manipulations increased. However, both hands’ manipulations of TD participants were relatively independent. It might demonstrate that participants with ASD performed redundant or inefficient manipulations during playing the games. This assumption could be supported by the results of two-player games. We could see that the average distances per motion of non-dominant hand and dominant hand for participants with ASD were twice as many as those of one-player games though the achieved scores of two-player games increased just a little bit. And TD participants’ still remained at a similar level of the distances per motion in the two-player games.

6 Conclusion and Future Work

In this paper, we have presented a tablet game system which is able to efficiently assess the hand usage of users in movement manipulations and to spontaneously induce communicative and interactive activities between the users via hand-control games. The preliminary usability study showed that the proposed system was attractive for participants with ASD and their TD peers. The participants also tended to play with their partners and communicated in a natural way when they played the games together. The performance results based on the logged data reflected the hand usage behaviors of the participants. From the results, we found that participants performed a little better in two-player games only requiring one-hand manipulations than in the one-player games requiring two-hand manipulations. Participants also took greater efforts to perform non-dominant hand manipulations in the one-player games. In addition, participants with ASD showed a strong and positive relationship between their two hands, while the relationship of both hands was weak among TD participants. The study results were limited but promising, which suggested the potential of this proposed system for assessing the hand usage and promoting collaborative hand manipulations of children with ASD. In the future, we will expand the sample size of the user study to further test the effectiveness of this proposed system as well as to investigate the relationship between the performance, hand manipulation and interactive skills.