Exploring Collaboration Patterns in a Multitouch Game to Encourage Social Interaction and Collaboration Among Users with Autism Spectrum Disorder

Abstract

In this paper, we present a design and evaluation of four Collaboration Patterns on a multitouch collaborative game designed to encourage collaboration among people diagnosed with ASD (autism spectrum disorder). We define Collaboration Patterns as collaborative interaction strategies on elements in a multiuser interface. The patterns presented here were designed according to both recommendations from experts in ASD and requirements of a group of youths with high ASD-related impairment in their social interactions and were inspired by collaborative methods used in other studies. The proposed Collaboration Patterns were evaluated using research criteria relating to social interaction actions and collaborative tasks achieved by users during a multitouch game. The evaluation results suggest that each Collaboration Pattern motivates the need for collaboration and encourages creation of social interaction expressions among users. The applied sequence of patterns gradually encouraged collaborative activities and verbal and gestural interaction expressions among users. The significant characteristics of the proposed Collaboration Patterns allow us to suggest that they might be used in other collaborative applications aimed at fostering social interaction and collaboration among people with ASD.

1 Introduction

Autism spectrum disorder (ASD) is characterized by deficits in the development of social interaction and communication skills. People with ASD may be dependent on routines, intensely focused on inappropriate activities, or highly sensitive to change (WHO 1992; APA 2000; Swedo 2013). Difficulty in social interaction is one of the most significant problems of people with ASD. This difficulty is made more significant by additional problems with speech and language. ASD also creates problems in interpreting what another person might be thinking or feeling. They may not be able to anticipate what others will say or do in different situations. It also affects how individuals with ASD make sense of the world around them (WHO 1992; APA 2000).

The degree of autistic impairment varies for each individual according to the intensity of impairment in his or her language, imagination and social interaction. Low autistic impairment is known as high-functioning autism (HFASD), or Asperger’s Syndrome (Wing 1988). At the other extreme of the autism spectrum, people with high autistic impairment exhibit characteristics of intellectual disabilities, absence of language, indifference to social interaction, and a strong degree of isolation (Wing 1988). People with ASD usually do not participate in collaborative activities; they show little emotion, little sympathy and little empathy for others. Often, as they grow, they can develop a greater connection, but social relationships remain generally superficial and immature (Salle et al. 2005).

Different computing applications have been developed to mitigate some of the difficulties of people with ASD (Gillette et al. 2007; Millen et al. 2011; Chen 2012; Noor et al. 2012; Kientz et al. 2013). Among these, we note the advantages of collaborative applications in multitouch tabletop interfaces over desktop computers or interfaces with input devices (Sitdhisanguan et al. 2007). This technology provides inherent multi-user features to encourage collaborative work among multiple users (Tse et al. 2007; Chen 2012). It can be sufficiently flexible to adapt to the difficulties of users, to contribute in learning social and collaborative skills (Farr et al. 2010; Chen 2012), and to enable users to be creative, express themselves, and understand emotions (Hourcade et al. 2012).

Beyond the benefits of multitouch technology, it is still necessary to answer several questions related to the development of software that introduces notions of collaborative work between users with impairments in communication and social skills. For example, which type of software features must be considered by developers to create effective collaborative applications for this type of user? Generally, the developer of a collaborative application assumes that the users want or at least know how to collaborate. How can we provide collaborative learning opportunities within software aimed at this special group of people who do not know how to collaborate in the typical sense?

Although studies in collaborative applications for people with ASD present important results in introducing social skills and collaboration techniques to users, there is still a lack of design methods or standards that could serve as a basis for developing efficient collaborative applications for these users. Both this lack of design methods and the potential benefits of collaborative applications for these users motivated our research toward the search for these design methods.

The study presented here is part of a larger research project that aims to propose collaboration methods to guide the design of multitouch collaborative applications to encourage the development of collaborative and social interaction skills among users with ASD.

This research started with the study of different collaboration methods in multitouch interfaces. This led to the design of strategies that we called Collaboration Patterns. We define Collaboration Patterns as interaction strategies on elements in a multiuser interface that gradually encourage collaboration among people with ASD. Our goal in this article is to propose Collaboration Patterns that may be used as a guide to develop collaborative applications for multitouch tabletops (regardless of the multitouch technology used) designed for people with ASD.

In this work, we present four Collaboration Patterns designed according to the following methodology. (a) We selected a group of youths with ASD who experience high impairment in their social interactions to analyze their specific characteristics, such as their interests, difficulties, and needs. (b) We followed recommendations of experts responsible for the therapy of these users. (c) We studied previous work about collaborative applications for people with ASD/HFASD. (d) We studied theories about collaboration mechanisms in applications for typically developing people. Then, we proposed the four Collaboration Patterns. Finally, we developed a collaborative game called PAR using a proposed sequence of Collaboration Patterns and evaluated it with the selected group of youths.

This study led us to two main results: (a) The defined Collaboration Patterns used in the proposed sequence helped us to gradually encourage collaboration among the selected users, contributing to their cooperative skills and generating social interactions among them. (b) The main characteristics of the Collaboration Patterns proposed in this paper allow us to suggest that they might be used in other collaborative applications aimed at helping the social interaction and collaboration skills of people with ASD.

This paper is organized as follows. In section 2, we present studies on collaboration mechanisms used in multitouch tabletop applications, and aspects considered in the design of collaboration patterns intended for people with ASD. In section 3, we detail the proposed Collaboration Patterns and their application in the developed game PAR. The evaluation process with a group of youths with high autistic impairment is presented in section 4. Finally, in section 5, we present the conclusions of this work.

2 Related Work

Collaborative applications in multitouch tabletop settings are often developed to contribute to the social and collaboration skills of people with ASD, with HFASD in particular (Chen 2012; Noor et al. 2012). The applications use different types of constraints or strategies, both on the touch surface and on interface elements, to motivate/force collaborative activities among the users (Piper et al. 2006; Bauminger et al. 2007; Gal et al. 2009; Battocchi et al. 2009; Giusti et al. 2011; Zancanaro et al. 2011).

Giusti et al. (2011) proposed a set of constraints called collaborative patterns (“Choosing together”, “Constraints on objects”, “Different role”, “Ownership”) intended to encourage interaction among two or more users with HFASD on multitouch interfaces. These patterns have special features that require collaborative interaction with the objects in the interface. For example, “Choosing together” requires that two or more users touch an object to select it, “Constraints on objects” requires two or more users to move an object, “Different role” assigns different roles for each user, and in “Ownership”, each user has ownership of different objects that must be negotiated.

Giusti et al. (2011) and Zancanaro et al. (2011) included these collaborative patterns in a set of tabletop games, where a therapist controls the interface from a panel on one side of the tabletop to mediate the collaborative activity of two users who interact from the other side. The authors conclude that users with HFASD learn and understand the importance of collaboration as they advance in the games, adding that they created options for coordinating their collaboration, sometimes using verbal language (COSPATIAL 2011) and responding properly to the collaboration strategies applied in the games (Weiss et al. 2011).

Bauminger et al. (2007), Battocchi et al. (2009), and Gal et al. (2009) developed two games based on what the authors called an “Enforced Collaboration paradigm”, which forces simultaneous actions of users on objects on multitouch surface tabletops. This paradigm is similar to the “Choosing together pattern” (Giusti et al. 2011).

“Collaborative Puzzle Game” (Battocchi et al. 2009) consists of movement of the pieces of a puzzle simultaneously by two users, and “StoryTable” (Bauminger et al. 2007; Gal et al. 2009) is a game for building a common history between pairs of users. “Collaborative Puzzle Game” (Battocchi et al. 2009) was comparatively tested with typically developing users and users with ASD. The authors mention that users with ASD needed more help from therapists to get involved in the game; also, these users requested more negotiation and coordination during collaboration. This suggests that the “Enforced Collaboration paradigm” increases the opportunity for behavioral therapy that involves social interaction and encourages the need for collaboration. In “Story Table” (Bauminger et al. 2007; Gal et al. 2009), the authors found that the design of this type of application for multi-user tabletop interfaces has considerable potential for the improvement of social behavior and acquisition of some language skills.

“SIDES” (Piper et al. 2006) is another collaborative game for users with HFASD that includes tasks that motivate negotiation skills and involves ‘enforced turn-taking’ by four users around the tabletop. The authors suggest that games on tangible tabletop interfaces provide great motivation for users while learning group work skills because they found that users develop a great ability to communicate with colleagues.

Regarding strategies of collaborative applications in multiuser tabletop interfaces for typically developing people, we describe here only those that are more closely related to strategies found in studies for people with ASD/HFASD.

Initially, we note the study of Eva Hornecker (2005). This study indicates that the use of constraints in collaborative systems indirectly induce users to collaborate because the constraints provide implicit suggestions to act in certain ways or to adopt interaction patterns that indirectly foster collaboration among users. Constraints should be applied in collaborative work that requires distributed tasks, mutual assistance among users, and coordinated actions. In this study, the author found that constraints led users to encourage coordination, cooperation, and group awareness.

On the other hand, Goh et al. (2012) described four general design patterns for collaborative tabletop applications. They are more directly related to the physical environment: (a) “Using a Large Physical Space. Spatial separation afforded by large physical interactive spaces has strategic collaborative potential”; (b) “Promoting Large-Group Collaboration. Anyone working in a large team knows that it is more challenging to accomplish a task when every member in the team is required to work together. Such a situation often provides greater opportunities to teach social skills such as leadership, negotiation and conflict-management”; (c) “Using Sound. Appropriate use of sound and its timely rendition can support collaborative behavior”; and (d) “Using Multiple Touch Capabilities. The ability of an interactive tabletop to detect multiple touch points provides interesting affordances in interaction design”.

These patterns are based on the strategies used in “Collaborative Puzzle Game” (Battocchi et al. 2009) and “SIDES” (Piper et al. 2006). According to Goh et al. (2012), these patterns can be applied in multi-touch tabletop collaborative games regardless of target users, i.e., people with or without ASD.

A different set of patterns or mechanisms was proposed by Yuill and Rogers (2012) in a framework of collaboration mechanisms and constraints that allows identifying what needs to be constrained on a multi-user interface to help people interact more smoothly. According to these authors, it is necessary to consider three different characteristics in collaborative applications: the degree of “awareness” of other users’ actions and intentions, the degree of each user’s “control over” the interface, and the degree of availability of relevant “background information”.

2.1 Issues Considered in the Design of Collaboration Patterns

Despite the encouraging results achieved in the use of collaborative applications for people with ASD, we found that the majority of these applications were tested only with users with HFASD (Piper et al. 2006; Bauminger et al. 2007; Gal et al. 2009; Giusti et al. 2011). Therefore, it is not yet clear if these applications are also effective for those with high autistic impairment. In other words, it is not clear whether these applications and their collaborative strategies would present good results for people without a minimum degree of ‘collaboration know-how’. Therefore, we analyzed the different strategies mentioned above together with experts in ASD to determine how these could contribute to the design of collaborative patterns for applications aimed at people with ASD.

Multitouch interface support of collaborative work allows high levels of awareness and more-fluid interactions (Hornecker et al. 2008). When people interact on these interfaces, they have face-to-face awareness of the actions, intentions, emotions, and mental states of their partners. Users show signals that allow one to anticipate their actions and motions. This awareness could encourage users to “disseminate their own knowledge in the workspace” (Belkadi et al. 2013).

One of the main problems of people with ASD is their difficulty with identifying others’ mental state, limiting their capacity to understand the signals and implicit information that are essential for ‘awareness’ and, consequently, for social interaction and the realization of collaborative tasks. Thus, to develop collaborative applications in multitouch interfaces aimed at this group, it is necessary to find ways to provide maximum ‘background information’ and to gradually encourage users to identify the actions of their partners. With this information, users should gradually recognize aspects of their collaborative environment such as how to identify their own role in the workspace, how to interact with other members of the group, what he/she should expect from their partners, and what roles will their partners assume (Belkadi et al. 2013).

We opted to use constraints in the design of collaborative strategies due their advantages for encouraging collaboration (Hornecker et al. 2008). From the set of collaborative patterns proposed by Giusti et al. (2011), we considered “Different role pattern” appropriate for these people so long as it will be provided gradually, thereby generating awareness of the roles of each user and, thus, encouraging interaction and cooperation between the partners. We expected to see more interest and ease of adaptation in the users adopting this pattern in a collaborative application. We also considered “Constraints on objects pattern” appropriate because it motivates collaborative activity between users through restrictions that require joint interaction of users on certain objects.

In turn, “Ownership pattern” was not considered due to the high degree of behavioral impairment of our target users (detailed in Section 3), which does not necessarily allow them to perform negotiation actions.

“Choosing together”, which is a pattern similar to “Enforced Collaboration” applied in “Collaborative Puzzle Game” (Battocchi et al. 2009) and “SIDES” (Piper et al. 2006), is dependent on a technology that recognizes who is touching where on the surface (e.g., DiamondTouch tabletop (Dietz and Darren 2001)). It cannot be completely reproduced with other multitouch tabletops because a single user could easily simulate this pattern using two fingers.

In turn, from the set of design patterns proposed by Goh et al. (2012) (“Using a Large Physical Space”, “Promoting Large Group Collaboration”, “Using Sound”, and “Using Multiple Touch Capabilities”), we considered the “Using a Large Physical Space” pattern. It would be appropriate for the target users for several reasons: to enhance co-located face-to-face collaboration with different options for the user’s location around the tabletop, to assign interaction spaces for both the interaction of each user (‘owner’) and the joint-interaction (‘shared space’), and to include tasks to encourage communication, social interaction expressions and enforced collaboration. In addition to the size of the surface, it is important to consider its orientation (‘horizontal or vertical’), which affects the collaboration (Yuill and Rogers 2012), and the orientation of the application contents for each user around a horizontal tabletop to prevent users from viewing these contents from different angles (Kruger et al. 2003). However, we cannot consider this last aspect for the design of collaboration patterns for people with ASD because they have difficulties understanding changes and easily adapting to them.

We emphasize that this physical space should not be large enough to impede closer contact with the partners around the tabletop. It is important to provide an appropriate physical space size to allow physical contact between users because this is one of the aspects that needs to be stimulated in people with ASD. Together with the physical space, it is important to provide an environment that contributes to easy adaptation and manipulation of the elements on the interface. Cognitive workload should be minimized (Haller et al. 2010).

In turn, we consider that the “Promoting Large Group Collaboration” pattern would depend greatly on the impairments, difficulties and abilities of each user. In a large group, more-‘advanced’ users could be encouraged to help weaker ones; otherwise, those ‘advanced’ users might choose not to wait for the participation of the weaker ones, instead choosing to do all of the work individually and ignoring the actions of the other. It would be interesting to modify the size of the group according to the extent that users learn how to work collaboratively.

The “Using Sound” pattern is very important in collaborative applications for people with ASD, especially to continuously guide user interaction. We recommend applying different types of sounds to give feedback on how users should interact on the interface, indicating the effects of actions performed and presenting success, errors and help messages.

“Using Multiple Touch Capabilities” is related to specific tabletop technologies that recognize individual touches and, therefore, is not considered in our research.

In the following section, we present the methodological process used to define the set of collaboration patterns intended for people with high autistic impairment.

3 Design Methodology

The design process of the proposed Collaboration Patterns involved the following steps. Initially, we chose a group of youths with high autistic impairment and identified their needs and specific characteristics. After that, we followed both recommendations of experts responsible for the therapy of these youths and theories about collaborative applications (mentioned in Section 2). Then, we proposed Collaboration Patterns, a set of collaborative strategies to motivate collaboration among these youths. Finally, we developed the game PAR, applied a proposed sequence of Collaboration Patterns and evaluated the game with our group of youths.

3.1 Target Users

We selected five youths with ages between 10 and 17 years. They were selected for their high level of impairment in social, interaction, and collaboration skills. According to information provided by recommending experts, these youths (boys and girls) enjoy sports, especially soccer; they enjoy technology and innovative tools to learn; and they require interesting and easy activities to understand and do. They have difficulties in verbal communication, visual contact, social skills, and repetitive behavior. They need to improve their social interaction, communication and collaboration skills, and giving and receiving help from others. Their characteristics are described in Table 1.

Table 1 Description of each target user.

The designs of both the Collaboration Patterns and the game address the specific characteristics of these users, allowing them to maximize interest in the application to increase ease of learning and interacting and to help these users with social behavior.

3.2 Experts’ Recommendations

Experts responsible for the therapy of users with autism gave us recommendations about important aspects that designers should consider when developing collaborative applications for people with high autistic impairment, according both to the users’ specific characteristic and collaborative strategies used in other applications. Many recommendations are related to human-computer communication, but we present here only those related to the collaborative aspects of the interaction, which were the most important in designing the Collaboration Patterns:

• It is not enough to consider general or theoretical aspects of people with high autistic impairment. It is necessary to consider individual impairments and specific skills to identify what type of game is more suitable for them.

• Help information should be offered through voice messages, avoiding written messages. In addition, these messages should be offered with appropriate terminology for users, preferably using words from their everyday context.

• Coordination situations are very complex for them. Therefore, such must be carefully and gradually encouraged, e.g., the way “Ownership pattern” was applied in other studies is not recommended for our users with ASD because they have difficulties performing negotiation and coordination activities. They need to learn this gradually as recommended by “Different role pattern”.

• Despite their difficulties, people with high autistic impairment have other skills that need to be stimulated even more. Designers should be careful with ‘excesses’ such as information aids and constraints, among others. Sometimes, by focusing only on the difficulties of these people, intending to facilitate their interaction, designers may provide more features than are necessary in the system, and the system ends up doing everything for them, which is not productive.

3.3 Collaboration Patterns

In this section, we present four Collaboration Patterns that we considered appropriate to guide the design of collaborative applications for users with high autistic impairment. In addition to the difficulties, needs and skills of the target users, we considered different aspects in the design of the Collaboration Patterns, which are summarized in Table 2.

Table 2 Summary of aspects considered in collaboration patterns design.

As mentioned in the introduction, we defined these Collaboration Patterns as interaction strategies on elements in a multiuser interface that gradually encourage collaboration among people with ASD. The main goal of these Collaboration Patterns applied to collaborative applications is to encourage the social interaction, communication and development of collaboration skills of people diagnosed with ASD, particularly in social interactions among users with a greater degree of autistic impairment.

The challenge posed by the target group starts in slowly motivating users to understand the necessity of being aware of a partner’s action. So, closely related to the definition of Collaboration Patterns, it is very important that they be used in a specific sequence. For this, we also defined a continuum with the four Collaboration Patterns proposed (Figure 1). Three of these Collaboration Patterns impose restrictions on interaction to gradually motivate collaboration and social interaction between users. A fourth Collaboration Pattern has no restrictions, allowing identification of collaboration tasks performed by users in an environment with free interaction on the interface.

Figure 1

Continuum of collaboration patterns.

Initially we assigned a minimum number of tasks for each user to enable them to understand and to adapt themselves in realizing their role. This number of tasks is gradually increased to encourage the need for collaboration, while the users with more difficulties adapt to the notion of sharing resources to achieve certain goals. Following this idea, we designed the first two Collaboration Patterns: “Passive Sharing” and “Active Sharing”.

To motivate a further understanding of collaboration, it is necessary to include tasks that require joint performance of users on application objects. Thus, we obtained the third Collaboration Pattern, “Joint-Performance”, joining the “Different role” and “Constraints on objects” patterns (Giusti et al. 2011) and including notions of synchronous collaboration.

The last Collaboration Pattern proposed poses no restrictions on interaction. It is called “Unrestricted Interaction”; it allows collaborative actions among users in an unconstrained environment after collaboration has been motivated by the previous three Collaboration Patterns.

The four Collaboration Patterns proposed are described below:

• Passive Sharing Pattern: The only constraint applied in this pattern is the different role assigned to each user. The tasks are just action and response from one user to another. Users receive information about when and how to execute their own tasks as a result of the action of their partners, but they are not particularly aware that they are doing a collaborative work. To ‘share resources’, each user has only to realize his/her own task and know the result of the tasks of the partner, independently of identifying who executed the task and how it was executed (Figure 2a). This pattern is intended to introduce the notion of ‘awareness’.

  Figure 2

  

  The proposed collaboration patterns.

• Active Sharing Pattern: This pattern is intended to introduce the importance of recognizing the role of the partner in collaboration. This is achieved by requiring ‘information exchange’ between users, in addition to sharing resources. Each user receives information from the partner about how to cooperate. Because of users’ impairments and the increasing complexity of the collaboration, it is necessary that the system provide additional sound and voice support to facilitate this information exchange.

  The constraints applied in “Active Sharing Pattern” are the roles assigned to the users and the attention that each user must pay to the partner’s actions. Users begin to recognize that their activity requires ‘communication’ (Figure 2b).

• Joint-Performance Pattern: This pattern uses the same principles as “Active Sharing Pattern” and introduces cooperative ‘simultaneous actions’. The constraints of this pattern are the roles, the necessity of information exchange, and the constant attention to help the other when needed. Users now realize that the participation of both is strictly necessary to achieve a goal, recognizing that they are involved in a ‘collaborative activity’ (Figure 2c).

• Unrestricted Interaction Pattern: this pattern does not assign roles or constraints to users, allowing them to collaborate in a ‘free interaction’. This pattern is offered to users after they have interacted with the restricted patterns. The intention of this pattern is that users develop ‘coordination strategies’ for sharing information, cooperating, and helping each other. It is expected that users perform the activity together, although this cooperation is not strictly required (Figure 2d). The four Collaboration Patterns are summarized in Figure 3.

  Figure 3

  

  Main features of the collaboration patterns.

3.4 Applying Collaboration Patterns in a Multitouch Game

Finally, we developed a collaborative game, called PAR, to evaluate the Collaboration Patterns continuum.

PAR has three collaborative phases. Each phase includes one of the three restricted interaction patterns and the unrestricted interaction pattern. The phases of PAR are based on a single objective, which is to get the pieces of uniforms and to dress the soccer players of a team. The sports pieces, shirt, shorts and sneakers, are randomly distributed on three higher shelves of a warehouse. These pieces need to be taken down so the players can wear them. On the side of each shelf, there is a box. This box serves as a container into which the user must put each piece of the uniform to be taken down. In the lower part of the warehouse, there is a cart with three spaces; each space is able to receive any piece of a uniform. When the cart is full, it is necessary to take it to the parking lot and give the pieces one by one to the player in a row. After the parts are delivered, the user must return with the cart to receive three more parts for the next player to wear and so on until all players are wearing the team uniform and are ready for the game.

To obtain the pieces of the uniforms, the cooperation of two users is required; each user will have a different role according their place around the multitouch tabletop. One user is located at the upper side of the table (User 1), and the second user is located at the lower side of the table (User 2). At the time of dressing each player, users may move to the right side of the table. The phases of the game are described below.

• First phase: User 1 must take a piece of the uniform and put it in the box on the shelf; the box will descend with the piece. User 2 should move the cart and take it to the shelf of the descending box to receive the piece sent by User 1. Likewise, User 1 must send the second piece of a player uniform from any shelf, while User 2 moves the cart to receive it (Figure 4a). When the cart is filled with three pieces, User 2 should move the cart to the parking lot.

  Figure 4

  

  a “Passive sharing pattern” in the first phase b “Active sharing pattern” in the second phase of the game PAR.

  The “Passive Sharing Pattern” is applied in this phase because User 2 need only take the cart to the position where the box is descending. This position results from the action of User 1, but User 2 might not recognize that User 1 is responsible for this.

• Second phase: User 1 must have information about the three sport pieces that should be sent. User 2 should ask for any piece by pressing a button. User 1 should then put the piece requested by the partner in the box; this piece can be on any of the three shelves, so User 1 must locate the piece, take it and put it in the box. User 2 should move the cart to the shelf of the piece being sent to be able to receive it (Figure 4b). Similarly, User 2 should ask for the two missing pieces to fill the cart and take it to the parking lot.

  The “Active Sharing Pattern” is applied in this phase because User 1 needs to know which uniform piece User 2 asked for to execute his/her own action of putting that piece in the box.

• Third phase: In addition to the tasks of the previous phases, in this phase the boxes are already closed at the time that each piece is requested. Therefore, it is necessary that User 2 helps by pressing a button to open the boxes while User 1 takes and puts the piece requested in the box. At the moment User 1 takes the piece, a voice is heard stating, “Help me by pressing the yellow button.” User 2 must press that button while User 1 puts one piece into the box (Figure 5).

  Figure 5

  

  “Joint-performance pattern” in third phase of PAR. a Selection of piece b Helping the partner.

  The “Joint-Performance Pattern” is applied in this phase because the box must be opened by both users. User 2 must press the yellow button while User 1 put the piece of uniform in the box.

• Three phases: in the three previous phases, after the cart is placed in the parking area, the “Unrestricted Interaction Pattern” is applied, where both users can take the pieces of the cart to dress the soccer player. Each user may take any piece in any order to dress the soccer player (Figure 6). At this moment, a message is displayed to users informing them of the number of soccer players dressed and offering an option of continuing to dress the next soccer player. User 2 must then take the cart from the parking lot and return to the lower part of the warehouse to continue asking and receiving sports pieces.

  Figure 6

  

  Each user may take pieces to dress the soccer player in the “unrestricted interaction pattern”.

4 Evaluation and Results

In this section, we present both the evaluation process of the PAR Game with the group of target users and the results of this evaluation.

4.1 Evaluation Methods

PAR was tested with the five youths shown in Table 1. We conducted a pre-training stage over a period of nine days spread over a month. During this stage, the functioning of the game was explained to users and they became familiarized with the interaction and direct manipulation of game objects on a multitouch table. After this stage, both the place and the necessary equipment to perform the tests were prepared, as well as the evaluation criteria.

Tests were performed in a computer laboratory. We installed the multitouch tabletop and three cameras located at different angles focusing on the table (Figure 7). This arrangement allowed recording both user-system and user-user interactions. Tests were applied during 15 days spread over 6 weeks. Each test session lasted between 5 and 15 min. The number of tests differs among user pairs because it was determined by the emotional and behavioral characteristics of each user in each test. Fifty-one test sessions were conducted in this period.

Figure 7

Multitouch tabletop and cameras to record users’ interaction.

In each session, two users participated in addition to the researcher in the role of evaluator of the system and a therapist who accompanied the activity performed. Eight therapists took turns performing the monitoring. Each was willing to offer help to the users and to motivate them if needed, and in some ways maintained control of the behavioral characteristics of the users.

We delivered audio instructions via speakers, specifying the user to whom the message was addressed. We did not provide headphones because the users might experience difficulty in keeping headphones on their ears during the game due to their intense sensibility to sound and to their behavior difficulties. When required, the therapist guided the users to follow the audio instructions.

The test routine was conducted so that each user interacted with the game in the order of phases: 1, 2 and 3. During each game test, the pairs of users and their respective roles in the game were exchanged. In these tests, the major objective was to evaluate the effect of each Collaboration Pattern on social interaction undertaken by users during the game to accomplish collaborative work.

4.2 Quantitative Measures

The analysis of each phase of the game was based on aspects related to the collaborative tasks and social interaction expressions shown by users. In the first three Collaboration Patterns, we assessed collaboration by the number of actions and responses from one user to another on interface elements to achieve a collaborative goal. In the Unrestricted Interaction Pattern, collaboration is assessed by the different strategies created by each user to cooperate with their partner to achieve a goal. These are called “Collaborative Strategies”.

We categorize social interaction expressions as verbal interactions—rectify, guide, ask question, answer, encourage, thank, ask for help, complain, commemorate, and reject—or gestural interactions—see, smile, laugh, perform task in the game, and physical contact. For analysis, we organize social interaction expressions as follows:

“Interactive Situations” (INSs) refer to verbal or gestural interactive expressions in which a user interaction leads to a response from the partner. INSs are classified by type according to the number of interactive expressions occurring between a pair of users (Table 3). This is because an interactive action may generate an interactive response, but a response may generate a new interactive action, which may generate another interactive response, and so on. These types of INSs indicate continuous interaction among users (See example in Figure 8). Types with bigger INSs indicate more collaboration.

Table 3 Classification of INS.

Figure 8

Example INS type II. a IA user 1: guide; b IR user 2: see, IA user 1: guide, encourage; c IR user 2: Perform task.

“Interaction Intentions” (IINs) refer to verbal or gestural interactive expressions performed by a user without getting an interactive response from the partner. They indicate the intentions of users to collaborate with their partner.

Considering these definitions, we evaluated the following points to identify the contribution of the Collaboration Patterns. During the application of the sequence of Collaboration Patterns, is it possible to observe a collaboration process growing? What is the collaborative behavior (INSs and IINs) of each user in each Collaboration Pattern? Which verbal and/or gestural social interaction expressions are obtained during the game? What strategies were created by users to collaborate?

4.3 Results

As discussed in the previous section, users must perform actions and responses from one to another to achieve the goal in every phase of the game. The interest of the users increased with each phase of the game. Each new Collaboration Pattern applied generated a greater need for collaboration among users and therefore greater motivation to perform the tasks and to guide their partners through verbal or gestural expressions.

• First phase of PAR Game (Passive Sharing Pattern). We observed that during this phase, users A, C, and E tried to cooperate with their partners B and D. However, B and D were more passive, sometimes were not concentrated on the game, and did not respond to the collaborative tasks, causing disinterest and frustration in their partners (A, C, and E). Their partners answer with negative attitudes such as desire to perform the task of the partner and aggression to the partner when he/she does not respond in the desired time.

  The average number of collaborative tasks performed by each user is shown in Figure 9. In this figure, the success of each collaborative cycle applying the pattern in the game (i.e., from the beginning until the car is placed in the parking lot) was quantified. For instance, if a user accomplished all tasks assigned to him/her in that cycle, a 100 % level was achieved. The graphic shows the average percentage of each user during the various cycles in each phase of the game.

  Figure 9

  

  Level of collaborative tasks performed by users in the three restricted collaboration patterns.

  This phase presented INSs of types I and II and one of type III (Figure 10). Users engaged in different INSs, even with their difficulties in social interaction, because each user needed help and motivation to act at different levels. Some users (A, C, and E) were more motivated to perform their tasks and to guide their partners, while others (B, D) were more receptive but responded to requests only when motivated by their partners.

  Figure 10

  

  Types of INSs by collaboration pattern.

  In IINs, the interaction expressions performed more frequently by users A, B, C, and E were verbal and/or gestural guides to encourage their partners to perform the tasks. In addition, some users performed, to a lesser degree, other interaction expressions in response to their partner such as to “smile”, “ask for help”, “commemorate”, “complain”, and “rectify” (Figure 11). Conversely, user D smiled at his partner only once to request a response.

  Figure 11

  

  Number of verbal and gestural interaction expressions performed by users to collaborate with their partner in the three restricted collaboration patterns.

• Second phase of PAR Game (Active Sharing Pattern). In this phase, users were more active and there was a decrease in negative attitudes (users A, B, and C) due mainly to the augmentation of tasks in this phase. This required more concentration by users and led them to respond properly to the collaborative tasks, reaching a higher level of collaboration (Figure 9). User D initially had difficulty in adapting to this phase of the game but, due to his high motivation to play, gradually managed to perform the activity together with his partner. In turn, user E had difficulty working cooperatively; she was too impatient to wait for the response of her partner and performed both her own tasks and that of the partner.

  There was a general increase of INSs and IINs in relation to those presented in the previous phase. The INSs presented here had a greater number of interactive expressions involved and a greater number of interactive actions and responses between users, increasing the INSs of types I, II and III (Figure 10). These interactive expressions particularly occurred when some users tried to adapt to the new Collaboration Pattern and their partners guided and helped them to get the right answer.

  In this phase, users needed greater collaboration than in the previous one. This allowed them to increase the diversity of interaction expressions to encourage their partners to perform the tasks in the game (Figure 11). As observed in the first phase of the game, users A, C, and E performed a greater number of actions and responses, while users B and D were still receptive but more attentive to understanding the helps and responses of their partners. User D, because of his difficulty in initiating an interactive situation, had the lowest number of IINs; however, in this phase, he tried to interact with his partners with actions such as “see”, “smile” and “commemorate” a success in the game, expressions that were not observed in the first phase of the game (Figure 11).

  These results suggest that the “Active Sharing Pattern” motivates the users to develop collaborative work, encouraging them to communicate with their partners and to perform different collaborative tasks through interactive expressions that became INSs or IINs.

• Third phase of PAR Game (Joint-Performance Pattern). The number of INSs and IINs decreased in this phase for most users. However, INSs that appeared in this phase are more complex than those presented in previous phases. This complexity is defined both by the greater number of interactive expressions involved in each INS and by the type of INSs presented. This phase achieved INSs of types I, II, III and the first of type IV during the game (Figure 10). Furthermore, these INSs involved a greater number of interactive expressions than occurred in previous phases.

  This phase demanded greater collaboration due to the requirement for greater concentration to perform the tasks in the game. We also observed some low values in collaborative activity (Figure 9) in situations such as delay to respond and impatience for waiting an answer. The task of “helping each other” had become very interesting for the users. For some of them, it was easy, but for others, it required a process of adaptation. This allowed generation of more collaboration with interactive actions and IINs by users more involved in the game and the wrong and right answers of those who were adapting. These wrong answers generated a new interaction that allowed, after several attempts in some cases, one right answer finally to reach the goal. This explains the number of interactive expressions in each INS and the number of actions and responses achieved, reaching up to the type IV of INSs.

  In this phase, users performed a greater number of other social interaction expressions than they did in previous phases, including “physical contact”, “see”, “rectify”, “smile”, “encourage” and “reject” (Figure 11). All users improved their initiative to start a collaborative task and to play cooperatively; user E learned to respect rules in group activities and waited for the response of the partner, and user D strove to understand collaborative activities. It is therefore suggested that the Collaboration Patterns motivate generation of social interaction expressions between users and contribute to their collaborative tasks.

• Three phases (Unrestricted Interaction Pattern). In the three phases of the game, during the “Unrestricted Interaction Pattern”, only INSs of types I and II were presented (Figure 10). Users A and C performed a greater number of interactive expressions to get INSs, and users B and D were more attentive to requests from their partners. User E tried, on various occasions, to begin INSs, but had no answers, achieving the realization of several IINs during the game.

  The greatest number of IINs was performed by user C, who was continually seeking help from her partners, and to user E, who was interested in guiding her partners, but without getting answers. The most frequent interaction expressions were “perform task”, “physical contact” and verbal and/or gestural “guide” (Figure 12). The results suggest that the “Unrestricted Interaction Pattern” stimulates the need to seek the help of partners, thus motivating users to perform interaction actions.

  Figure 12

  

  Number of verbal and gestural interaction expressions performed by users to collaborate with their partner in the “unrestricted interaction pattern”.

  In the three restricted Collaboration Patterns, the number of INSs and IINs varied for each user because each one has a different impairment level in their interaction. However, the results show that the Collaboration Patterns foster verbal and/or gestural interaction expressions in each user in different ways. The interaction behavior of user D stands out because he is the user with a greater disability in his social interaction; during the game, he showed interest in answering INSs and starting an interaction by performing IINs.

  It was also important to analyze the different strategies employed between the users to perform tasks during this unrestricted collaboration phase to understand whether they had started to acquire notions of coordination. These strategies were classified as “Collaborative Strategies (CS)” and “Non-Collaborative Strategies (NS)”. CS includes different ways users perform the task together. NS are those situations where there is no interest in collaborating, and none of the users or only one performs the task.

  In the first phase of the game, users performed the same number of NS and CS, but we note that this changed as the game progressed (Table 4). In the second phase, more pairs of users performed CS to achieve the task and the number of NS was reduced. Table 4 shows that some pairs changed their strategies favorably. User D changed from just paying attention to acting and taking turns with his partner. Pair BE changed from a competitive strategy to taking turns to suit the soccer player in the game. The aggressiveness observed previously in pair BC was not present in the second phase and changed to a CS of offering and asking for help.

  Table 4 Collaborative and non-collaborative strategies performed by users in “unrestricted interaction pattern”.

  In the third phase, only a single NS could be observed. Users also increased the number of CS, and some users could perform different CS. Most pairs changed their collaborative strategies in comparison to previous phases, as shown in Table 4.

  After the tests with the users, we interviewed the therapists about the game and its effect on the users. They mentioned two main advantages regarding social interaction among the users. The first one is the interactive participation of more than one user, and the second is the fact that it is a collaborative and not a competitive game, motivating equitable participation of both users, regardless of the experience or skills of each user. Additionally, the therapists mentioned that the structure of the multitouch table provides the users mobility to play, motivating interactive situations among them. They mentioned that, in the restricted Collaboration Patterns, the opportunity for interaction in turns was important and, in “Unrestricted Collaboration Pattern”, the most skilled player was able to learn the skills of stopping and waiting for the interaction of their partner.

  One of the therapists summarized the contribution of the game to some users’ abilities to collaborate. To user B, who uses a computer at home for playing alone, it was very positive to learn to wait for the response of his partner, who played with a longer response time. User C was very interested; her euphoria was seen several times. User D started to have better answers, and his attention and interest improved. User E showed a great deal of interest and learned the commands of the game very fast. Her initial difficulty was to wait for the contribution of the other user. The therapists’ comments indicate the benefits of the Collaboration Patterns for users with high autistic impairment when they are designed to take into account users’ specific characteristics.

5 Conclusions

In this paper, we proposed a set of four collaborative strategies (“Passive Sharing”, “Active Sharing”, “Joint-Performance” and “Unrestricted Interaction”) that may be used as a guide to develop multitouch collaborative applications intended for people with ASD to contribute to their social interaction, communication and coordination skills. We called these strategies Collaboration Patterns. They were designed following the specific requirements of a group of youths with high autistic impairment, recommendations of experts in ASD, and collaborative strategies used in other studies. We also proposed a specific sequence of these Collaboration Patterns to gradually encourage users to identify the importance of a collaborative activity. Then, we evaluated this sequence in a multitouch collaborative game called PAR with our target group.

The results indicate that both the interaction on the multitouch interface and the aspects considered in the proposed Collaboration Patterns allowed engaging users in an attractive experience, gradually encouraging social interaction and collaborative work.

We emphasize the motivation generated in the users to interpret the intentions and actions of their partners throughout the sequence of Collaboration Patterns. More-active users performed activities faster and learned to respect the rules of the game. They also learned to help and motivate a partner to cooperate by means of orientation situations, physical contact, encouragement, complaints, and even by using verbal expressions. More-receptive users, with greater difficulty in the game, tried any way they could to perform the required actions; they asked for help with several interactive expressions, and they commemorated when they succeeded in performing the action and rejected their action when they did not.

The results indicate the importance of offering a multitouch collaborative application, which initially requires a minimum number of collaborative tasks and then increases these tasks to the extent that users advance in learning. We observed that this increase in the number of collaborative tasks involves the users in an attractive experience and encourages their need to collaborate and to be attentive to send and receive responses to/from their partner.

The results shown that in “Passive Sharing Pattern”, after sharing resources repeatedly, more-active users realized that their partner was responsible for executing the other action; they then tried to guide the partner to execute his/her task. More-receptive users needed more help to execute their tasks and presented aggressive behavior when they did not receive the other’s action. They asked for help but without realizing that this help could be offered by their partner. In “Active Sharing Pattern”, the aggressive behavior of users was reduced, and they were more motivated to execute their tasks. Users were more aware of the need of another’s action to execute their own action. In “Joint-Performance Pattern”, the users paid more attention to executing their tasks by synchronous action, it being less necessary to perform interactive expressions to motivate or help their partners.

The “Unrestricted Interaction Pattern” allowed us to determine that users with high autistic impairment can find ways to collaborate in an unrestricted environment after they learned the collaboration technique in restricted environments. The users accomplished a collaborative activity through gestural and/or verbal coordination expressions.

The Collaboration Patterns proposal allowed the appearance among users of gestural and interaction expressions such as perform tasks in the interface, guide the partner, have physical contact, see, ask for help, answer, rectify, complain, smile, laugh, commemorate, reject, encourage, and thank. As the game progressed through the three restricted Collaboration Patterns, users realized new types of interaction expressions and more extensively engaged in social situations, thus contributing to the therapy for the difficulties experienced by users.

The significant characteristics of the Collaboration Patterns proposed in this paper allow us to suggest that they might be used not only in other multitouch collaborative applications but also in ‘conventional’ (non-multitouch) collaborative applications aimed at helping the social interaction and collaboration skills of people with high autistic impairment. An interesting point for future work is to investigate whether the patterns could be applied in physical games without any technology at all. This would make the idea more accessible to less affluent institutions.

It is advisable to apply these Collaboration Patterns in other collaborative applications and to perform more studies with other groups of people with ASD to identify possible generalizations and limitations of this approach. Nevertheless, these patterns and the experiment with PAR Game shed some light on the challenge of providing collaborative technology to people who do not know the very basic notions behind collaboration.