COBO: A Card-Based Toolkit for Co-Designing Smart Outdoor Experiences with People with Intellectual Disability

Abstract

This paper presents a co-design toolkit that aims to involve people with Intellectual Disability (ID) in the ideation of smart outdoor experiences. We conducted a series of workshops with young adults with ID and special-education professionals of a social-care center, to identify the physical and digital experience that could favour reflection and engagement of the addressed users, and empower them in solving daily-life challenges. Multiple refinements of iterative prototypes led to the design of an interactive toolkit, COBO, that integrates inspirational cards, interactive smart objects and multimedia contents to guide users during the conception of novel ideas. This paper illustrates the main insights of the conducted workshops and the design of COBO. It also reports on a preliminary evaluation we conducted to validate some design choices for the integration of physical and digital components within COBO.

1 Introduction

Intellectual Disability (ID), also defined as Disorder of Intellectual Development by the International Classification of Diseases (ICD-11 [1], code: 6A00), is a condition arising during the developmental period characterized by often co-occurring challenges in the cognitive, social, communicative, motor, behavioral and emotional spheres [8]. The disorder includes deficits in intellectual functioning (e.g., reasoning, problem solving, planning, abstract thinking, judgment, academic learning) and adaptive behavior (communication, social participation), affecting autonomy in everyday life [31].

Several studies investigated the use of technology [27] to support people with ID in enhancing their cognitive, behavioral, social, and sensorimotor capabilities. Some works specifically focused on the development of “phygital" interfaces [6, 13, 36], a specific class of Tangible User Interfaces (TUIs) characterized by the combination of physical and digital contents. The use of such integrated paradigm for people with ID is regarded as a promising approach that is not meant to replace current treatments making use of manipulative interactions, but rather to enhance them with new meanings provided by virtual scenarios. Its potential is driven by theoretical approaches that posit learning as both an intellectual and a physical process. This implies performing tasks through the use of physical materials and emphasizes the formative role of embodiment - i.e., the way sensorimotor capacities enable interacting with the physical environment successfully - in the development of cognitive skills, such as mental imagery, memory, reasoning and problem solving [15, 41]. The work presented in this paper explores the potential for young adults with ID of phygital interfaces, and in particular addresses the following research question: Can a structured co-design method, based on the adoption of phygital material, empower individuals with ID to reflect on and ideate smart outdoor experiences?

To our knowledge, these aspects have scarcely been addressed in previous studies. The literature mainly reports on the adoption of phygital interfaces as a means to support learning experiences. Our work focuses instead on using this technology to make people with ID protagonists in the ideation of smart outdoor experiences for themselves. These consist in orchestrations of smart devices and behaviors situated outdoors, which can occur in situations users might encounter in their everyday life [7].

For this purpose, our research proposes COBO (COllaborative BOard), a phygital toolkit that we designed by involving a group of young adults with ID and special-education professionals of a social-care center. Based on SNaP [22], a collaborative card-based board game addressing the co-design of smart objects by children, COBO introduces a paradigm that combines multimedia elements (images, animations, sounds) shown on a tablet device, and physical items (deck of cards, an interactive board, some smart objects) that are manipulated by the users. This material is used within a flow of structured activities that favours not only users’ reflection, but also group discussion and collaboration. Overall, the users are guided in the ideation of smart interactive artefacts.

This paper illustrates the user-centered design of COBO. It also tries to outline insights related to the adoption of phygital interfaces for engaging people living with ID in the design of interactive technologies, and the potential of these technologies for empowering these users in the design of new interactive solutions. The results of the conducted studies are promising and let us think that, although further experimentation is needed, our work could pave the ground for the definition of methods for the co-design of interactive technologies with people with ID. The paper is organized as follows. After describing our research context in social-care centers and the rationale behind our work, Sect. 2 discusses relevant studies on tangible and phygital interfaces for people with ID. Section 3 describes the user-centered process that led to the design of the COBO toolkit; the main features of COBO are then illustrated in Sect. 4. Section 5 reports on a preliminary evaluation that allowed us to validate some design choices and Sect. 6 discusses the main insights gained throuhg the whole process of design and evaluation. Sections 7 finally draws our conclusions and future directions.

2 Rationale and Background

The research discussed in this paper was conducted in collaboration with young adults living with ID attending a social-care center, and their special-education professionals. Social-care centers, as the one our work is linked with, offer services that reinforce the development of skills such as the physical, cognitive, behavioral and social abilities needed to engage in daily life activities. They provide support for employment skills, self-care (e.g., grooming, dressing, feeding, bathing), leisure (e.g., knitting, playing games), cognitive stimulation.

We worked in close cooperation with Fraternità & Amicizia (F&A), an accredited private non-profit organization that manages social day-care centers in Milan (Italy), offering numerous services to people with ID. In F&A, as well as in most public and private social care institutions in Italy, guests usually work in groups with the intent to promote integration and well-being. Although these services are very valid, the needs of users with ID are often remarkably different and would require customized tools. Therefore, special-education professionals are always looking for innovative and accessible solutions whose use is transverse for different people with ID. Traditionally, most of the materials used are paper-based, yet, in the last decade, digital solutions such as smartphones, tablets and notebooks have been used to provide higher engagement and motivation [40].

In our activities conducted at F&A we wanted to adopt a Participatory Design (PD) approach to understand to which extent the use of co-design toolkits could empower users with ID to become protagonists in the design of technology. PD enables the design of products and applications directly with the final users and, more recently, even with users with ID [17] in order to truly develop accessible products. If planned properly, PD can lead to the development of a sense of empowerment and a feeling of competence, which bring beneficial effects on the individuals: they derive satisfaction and fun while feeling useful through their participation [16, 29]. We therefore involved young adults with ID throughout the entire process of our toolkit design, with the assumption that their inclusion could have an impact not only on the quality of the final product but also on their experience. We decided to focus on physical interactive materials with the aim to increase their interest and offer, by means of a tangible indicator of progression, a sense of achievement and confidence. The choice of such material also derived from previous research experiences [18, 21], which led us to focus our attention on phygital toolkits as a viable solution to involve those users in the ideation of technologies for themselves.

A dimension that characterizes our research is thus the interaction with smart physical materials, in the form of Tangible User Interfaces (TUIs). TUIs are claimed to be more natural and familiar than other types of interfaces [30], lowering the threshold of participation, enhancing e-accessibility and inclusion, and fostering independent exploratory, assistive and collaborative learning [33, 42]. Research on TUIs, and in particular on their applications for users with ID, is still at a preliminary stage, but initial studies with this population indicate positive effects on engagement, collaboration and initiative [4, 5, 42] when this technology is meant as a support to professionals in their education activities [11, 13, 20].

Within the context outlined above, our research aims to define a co-design method, supported by an interactive toolkit, acting on two different levels. At the level of reflection, it aims to identify how to lead users with ID to acquire the awareness on their everyday-life challenges, and to reflect on how technology can enhance the situations in which these challenges occur. Also, the mechanisms for smart-object ideation the users are exposed with can help them understand how the surrounding environments they live in, which very often are dense of smart technologies, work. At the level of empowerment, we also aim to identify methods that can make people with ID protagonists in the design of smart experiences that are inclusive and adapted to their needs.

2.1 Tangible User Interfaces for People with ID

Tangible User Interfaces (TUIs) enhance physical manipulation, physical-digital mappings, and multisensory exploration, providing richer sensory and learning experiences through the interweaving of computation and physical materials [5, 6, 13]. They extend the intellectual and emotional potential of interactive artifacts and integrate compelling and expressive aspects of traditional technologies with creative and valuable properties of physical objects [11]. Few empirical studies highlighted the benefits of TUIs for people with ID. Gelsomini et al. explored the use of a system called Reflex [18, 21, 35]. Inspired by a commercial system named Osmo [39], Reflex provided a number of game-based educational activities, involving the users’ manipulation of multimedia contents and physical items. The study unveiled some benefits in the use of TUI-based methods for the development of cognitive and social skills.

Some other studies have shown promising effects of the use of TUIs with children with ID. Falcao [14] studied children with ID playing with different tangible artifacts (Augmented Objects, LightTable, Drum Machine, Sifteo cubes). The author points out the effectiveness of tangible interaction for exploratory learning and suggests that the most efficient gaming paradigm is the one with a clear mapping between specific physical objects and their meanings. A similar mapping characterizes the design of other prominent TUIs. Polipo is a 3D printed smart toy co-designed with special-education professionals, which provides various manipulatory affordances and offers feedbacks and rewards by means of lights, sounds, and music integrated in its body [37]. An exploratory study highlighted its benefits for children with ID in improving fine motor skills and encouraging children’s communication with their educators. Similar benefits were also observed with Poma [3], a tangible user interface to improve social and cognitive skills of Sri-Lankan children with autism spectrum disorder (ASD); the authors also identified guidelines to design TUIs for children with ID. The Magic Room is then a more extensive example of physical/digital interaction in a smart space designed specifically for children with special needs, and installed at social-care centers and inclusive schools [2, 9, 19]. In the Magic Room, multiple connected physical objects and digitally-enhanced soft toys are used as interaction devices and are mapped to digital representations in wall or floor projections.

Recently, some works have proposed TUIs for introducing users with ID to coding. The Magic Cube [28] is an interactive physical toolkit for learning the basics of computer concepts, that allows children with ID to discover, through the interaction with a smart cube, the effects of available input and output. TapeBlocks [12] is then a toolkit for building circuits that aims to be a first point of contact with making for young adults with ID.

In summary, current research mainly highlights the benefits of TUIs for children [43]. With the exception of very few works (e.g., [12]), there is instead a lack of studies on the effects in adolescents and adults with ID. Our work advocates for a better understanding of the advantages and drawbacks of using TUIs with these users, and in particular aims to understand whether the use of TUIs can increase their engagement in co-design processes for technology ideation.

2.2 Co-design Toolkits

Card toolkits are often used in the generative phase of co-design methods to collaboratively build prototype artefacts [34]. They favour the engagement of non-experts in the design process and make it tangible and in some cases playful. In recent years, a number of toolkits have been proposed to help people ideating interactive applications. Tiles Inventor is a general-purpose card toolkit for engaging anybody in the ideation of IoT devices. Khandu [26] is then a card game that aims to bring design thinking to children. It is designed to help children in problem solving and ideation for different contexts. Khandu’s approach, however, is non-technology oriented; children are guided to design and prototype ideas without technology. LocaLudo, instead, is a card-based game to design interactive architectures [24] where the outcome are ideas for local-communities. Güldenpfennig [23] created a co-design toolkit to provide senior users with early tangible experiences of their future systems and to iteratively convert them into the final implementations. Tango cards [10] then supports the design of tangible learning games. Overall, to our knowledge, there is a lack of work focusing on people with ID.

2.3 SNaP

Our work takes inspiration from SNaP (Smart Nature Protagonists), a collaborative board game for children aged between 10 and 14 years, designed for 2 to 4 players. As reported in Fig. 1, the main elements of the toolkit are three decks of cards and a game board that aim to inspire the design of augmented environment objects [22].

Fig. 1.

SNaP toolkit: the game board (left) and the environment, input and output cards (right) [22].

During the game-play, participants have the role of Designers while the game is facilitated by a moderator playing the role of a City Mayor. Each player has a mission to accomplish in order to make the elements of the outdoor environment interactive. The game structure recalls the most classic board games and consists of a game board with pawns, dice, coins. There are two types of cards: the mission cards and the nature cards. While the mission cards provide specific objectives that guide the design during the game (e.g., “make the park enjoyable for the visitors”), the nature cards are the central element of the game since they inspire the ideation of interactive artefacts. They consists of 18 environment cards, representing objects that can be found in parks or outdoor environments; 20 input cards, representing sensors and buttons that trigger an interaction; 5 output Cards, representing actuators that react to interactions through sensors and buttons.

In SNaP each player places a token in one corner of the board and tries to reach the central circle by throwing the dice and moving the token. Depending on the space the token reaches on the board, players may be entitled to get one or more cards from the Mayor’s deck of cards, get coins to be used to buy further cards, try to combine the cards and take notes, exchange or buy cards. In particular, when discussion boxes are reached, the players have rounds of discussion: they have to present their ideas to the rest of the team and to the Mayor. Overall, the goal of the game is to collect and combine different nature cards. At the end, all players “win” the game collaboratively, by presenting to the team their idea inspired by the cards gained during the game-play. The game ends when each player has designed at least an interactive object fulfilling his/her mission.

Given the SNaP capability of facilitating reflection through common discussion, its focus on physical material (i.e., cards, board), and its regular scheme guiding the design activities, we wanted to investigate its potential with people with ID. Trying the SNaP game with the young adults of F&A was the first step that led us to identify the feasibility of adopting such a method, although some adaptations were needed. The following section will describe the process that we followed to frame the opportunities and the limits of SNaP and identify possible benefits deriving from the introduction of TUIs.

3 Design Process

We conducted two workshops involving four users with ID attending the F&A social-care services, supported by two special-education professionals. Through an online survey, we then involved 27 professionals working at F&A. The two professionals attending the workshops were also interviewed to discuss in more details some themes emerged during the workshops and through the survey.

Table 1. Users with ID participating to the Workshops (P1 to P4).

The inclusion criteria for the participants with ID were: mild intellectual disability, no communication disorders, previous experience with experimental technological tools and in working together at F&A (with a consequence of having a strong and efficient relationship)¹. We thus involved four young adults living with a mild to moderate ID (see Table 1), who are used to working together at F&A, and had participated in other workshops with the use of technological devices. In relation to their level of intellectual functioning, each of them uses public transportation to get to the day-care center; they spend time with friends on the weekends, and they were, or are, all doing internships with the aim of a job placement in protected categories, according to Italian Law 68/99.

The four participants were asked whether they wanted to be involved in the study; after they agreed, we collected informed consents from their legal tutor reporting information about the study goal, the procedure, the planned data treatment and the option to withdraw at any point of the study.

The workshops were held in the day-care center the users with ID generally attend, so to offer a familiar environment and make them feel comfortable. The workshops were moderated by one senior researcher; the two special-education professionals attending the workshops were the ones supervising the activities of the participants at F&A. The two workshops and the interview with the two professionals were audio-video recorded to facilitate the analysis of data.

3.1 Workshop 1: Familiarization with the Game

The first workshop was conducted to favour the familiarization of the participants with the original SNaP board game (see Sect. 2.3); the literature indeed considers familiarization an important element to improve participants’ contributions to the final design [30]. The entire session lasted 1 h. The participants (P1–P4) played SNaP following its standard rules (Fig. 2). The two professionals attended the workshop with the aim of assisting the four players in case of need.

An expert researcher played the role of the city Mayor inviting the participants to act as Designers with the mission of making the city park more engaging for the visitors and for themselves. During the game-play, all the players were able to come out with some ideas on which they had discussion rounds. In turn, each one presented his/her ideas to the rest of the team and to the Mayor; the discussion helped refine the initial ideas, for example bringing participants to understand how to combine cards they initially discarded, or how to exchange useless cards with the other participants. The two special-education professionals observed the game, but there were no need for them to support the participants. They encouraged participants’ exploration when they encountered a new feature or function that stimulated their interest. They rewarded the participants to increase their confidence in their abilities in ideation.

At the end of the game, the four players in turn presented to the Mayor their final ideas of smart objects supporting playful activities in outdoor environments, as inspired by the cards gained during the game-play. P1 thought of “street signs that rotate every time someone passes by, to give directions to reach interesting, hidden spots of the park”. P2 ideated “a street sign that lights up to indicate the right direction to those who cannot see very well”. P3 thought of “a bridge that detects the distance of a person and lights up before s/he passes, to help identify the slope”. P4 ideated an interactive experience for deaf-blind visitors: “flowers must be able to sense the proximity of a person and emit scents to offer an olfactory experience".

Fig. 2.

Participants familiarizing with the SNaP game in Workshop 1.

3.2 Online Survey with Special-Education Professionals

After Workshop 1, we invited special-education professionals working at F&A to participate to an online survey. 27 professionals (20 female, with an average age of 40) filled in a questionnaire that was organized in three different parts². The first part shortly explained the SNaP game. The second asked for demographic data and experience with technology-based materials. The third one focused on gathering opinions about the adoption of SNaP within the activities held at the day-care center, and how to make it easy to learn, engaging and effective for participants with ID. To give them an idea of possible TUIs, this part also included links to videos of other interactive toolkits proposed in the literature (for example Magic Cube [28]).

The results highlighted that the strongest point of SNaP was the common goal to be achieved collaboratively by the users, which was positively evaluated also due to the participation and fun it can generate. The educators did not report on weaknesses for the game elements, but some of them observed that four players could be too many given the constant and active support that an educator has to devote to assist some users with ID. In this respect, they suggested that the integration of a digital support guiding the game could be useful (also for themselves) to explain the game rules and guide the players step by step.

In their answers to the final open questions, the professionals remarked that the physicality of cards was a strength, because users with ID need to see and touch physical material to improve their understanding of the context they have to focus on. They suggested introducing interactive, physical-digital elements to further help players understand how to combine cards and what the concrete effect of using the depicted sensors and actuators could be. Finally they discouraged the introduction of apps running on the participants’ smartphones, as this would distract them from the game and especially from the reflection process.

3.3 Interview with Special-Education Professionals

To deepen the themes emerged with the survey, we conducted an interview with the two professionals participating to Workshop 1³. They confirmed what resulting from the survey. In addition, they emphasized the opportunity for the participants to reflect on their needs and possible solutions. Very often, education activities focus on getting people with ID become independent in everyday tasks, especially those to be accomplished outdoor. Experiences at the park can favour the well-being of individual with ID, and can encourage them to go outside; the professionals were therefore in favour of adopting the game for their education activities. They also suggested extensions (e.g., new cards) focusing on further domains: the grocery store, the pharmacy, the best friend’s house.

3.4 Workshop 2: Gaining Feedback on the Extensions

The professionals’ comments informed the conception of some physical-digital extensions with two main functions: i) showing concretely the effect of card combinations, i.e., how sensors and actuators work; ii) guiding the different steps of the game and recommending how to combine cards. As a result, we started designing an interactive game board, complementing the original paper board, that could integrate both these functions, with different sensors and actuators to be plugged in to see the effect of card combinations, and a single visual display, also equipped with a conversational UI, offering recommendations to guide the game-play. We therefore wanted to conduct a new workshop to gather the opinions of the users with ID on the introduction of these new elements.

To let the participants understand our idea of tangible interactive board, we created a smart object prototype that, thanks to embedded electronics, could demonstrate the interactive behaviours of the objects depicted on the environment cards, thus giving a concrete feedback on what the game participants would have defined through card combinations. As illustrated in Fig. 3, this new object had an external cardboard case covered by soft plastic grass. Environment elements (e.g., trees, benches and ponds) were positioned on top of the board. Two connectors allowed the participants to plug in one sensor and one actuator at a time to see them work in combination. An Arduino board controlled the behaviour of the plugged digital components.

Fig. 3.

Paper prototype of the interactive board used during Workshop 2.

Workshop 2 lasted one hour and a half and started by asking players to collect cards in the same way they did with the card-only toolkit in Workshop 1. Once the players reached the phase where they had to generate ideas, we proposed them to have a try with the board prototype to see the effect of their choices on card combinations. Each of them orchestrated different inputs and outputs. At the end of the game, we showed participants a prototype of a tablet app aimed to give recommendations on alternative combinations, additional cards to buy, and how to progressively plug corresponding elements into the board.

3.5 Outcomes

We gathered qualitative data from the transcripts of the workshop, from the interview recordings and from the survey answers. We then conducted a thematic analysis to identify opportunities and challenges of adopting SNaP and its phygital extensions. We report below some emerging themes.

Theme 1: Challenges in learning the game. The participants with ID liked a lot manipulating the cards. Even if with different levels of complexity, they were able to come out with meaningful ideas. The involved professionals observed that the game would fit the abilities of users with a moderate ID: in Workshop 1 “P2 had an initial hesitation in finding card combinations, but she appeared more confident as the game progressed”. Also, they observed that all the participants were able to understand the dynamics and to make reasonable combinations. Therefore, they did not ask for any simplification of the method, rather they suggested extensions that could lower the entry barriers, for example to demonstrate concretely how electronic components work.

In Workshop 2, the comments of the participants with ID on the new materials highlighted that the meaning of card combinations was clearer than in Workshop 1. P1 for example thought of combining a button as an input, a fountain as an environment object and a light as an output. When trying the combination with the interactive board, P1 said “So, if I press the button the water lights up in red. It’s incredible! Can I decide to make the fountain blue too?”. Especially the final question let us think that the interactive board stimulated the participants’ curiosity on the way things worked, and this can trigger discussions that can facilitate a deeper comprehension of the game and of the designed smart interactive experiences.

Theme 2: Playfulness for engagement. All participants appeared very engaged and enthusiastic of “playing” with SNaP in both the two workshops. The game-play encouraged them all to try ideating interactive objects to enhance their experience at the park, for themselves and also for other visitors. Each one wanted to be protagonist and presented his/her own idea. We did not observe any hesitation, both with thinking about possible combinations of cards, and with sharing their ideas with the others. The meaningfulness of the resulting ideas can be interpreted first of all as an indication of the participants’ understanding of the design context, but also as an expression of their engagement in the game-play that motivated them to fulfill the game objective. Indeed, the workshop participants also suggested to “include tricks during the game to further engage the players (for example, traps or going back a number of positions).”

Theme 3: Structured game-play and collaboration. The professionals appreciated the regular scheme that the participants must follow to complete the game and the balanced alternation of discussions supporting divergent exploration of possible solutions and convergent refinement of selected ideas. In Workshop 1, the collaboration promoted by these discussion phases guided the participants to identify their ideas and refine them also with the help of their peers. They offered help to each other, and enriched the ideas of the other participants if they were encountering difficulties in combining the cards gained during the game-play. Thanks to this spirit, very few cards remained unused, i.e., outside a combination: every participant tried hard to define combinations also for the others. Participants also liked having missions, because they provide guidance through the game and set a goal. P2 suggested: “Why don’t you add a new (super-)mission for helping the other participants to conceive their new ideas?”. This is also an indication that they grasped the collaborative spirit of the game, till the point to think of a specific role to encourage ideation by every participant.

Fig. 4.

The interactive board prototype.

Theme 4: Reflection for education. Given the capability of the game to motivate participants to show and discuss their ideas, we identified some evidence for meaningful ideation and social connections. In addition to the benefits that can be achieved from the discussion with the others during the game-play, the two professionals participating to the workshops stressed the mentoring role that the mediator can play. The professionals also highlighted that even if the discussion and the deriving reflections focus on technology and on the role of digital devices in enhancing different activities, “indirectly this argumentation becomes a means to reflect on the related adversities; the autonomy as a goal would positively influence the user experience of the game”.

Theme 5: Phygital as a plus. Phygital material facilitated the ability for participants to discover by doing. In Workshop 2, the participants were positive to the introduction of tangible objects representing the park elements as they liked the idea of positioning the objects on the table to mimic the interactive situations they wanted to design. They enjoyed the concrete, immediate feedback provided by the digital devices about the interactive behaviours they were designing (e.g., playing sounds, turning on lights); and expressed their preference to soft sounds. Moreover, manipulating digital components helped them understand better what they were made of and how they worked. They also appreciated having a visual touch point, i.e., the app running on the tablet, to represent in an ordered way the card combinations and to receive guidance on the use of digital devices. They did not dislike having two boards (the paper one, originally devised for the SNaP game, and the new physical-digital one) and they understood very well the aim of having these two elements. This can be considered an indication of the benefit that the proposed paradigm can have on the users’confidence with the game.

4 COBO

Based on the collected information, we expanded the previous design of the toolkit and developed a working prototype: COBO (COllaborative BOard). COBO proposes the same game dynamics as SNaP, but introduces new elements in the toolkit: an interactive board, showing the effect of card combinations, a Web application, running on a tablet integrated into the interactive board and guiding the players in the different steps of the game, and augmented cards, equipped with a 2D code that can be read by visual recognition software embedded in the Web application.

Figure 4 illustrates the components of the interactive board:

• 1. Actuator bases: These are the components enabling the multisensory feedback representing the effect of card combinations. They are positioned in the upper side of the board (Fig. 4.1). Their role is to detect up to 8 different plugged-in objects (elements of a park environment) that can be inserted on the top through a 3.5 mm 3 pole male/female stereo audio jack connection. For each object, the bases can generate sound, light and vibration output.

• 2. Sensors: On the front side (Fig. 4.2), the interactive board integrates physical input devices and sensors: a button, a switch, a rotary potentiometer, a distance sensor, a brightness and a motion sensor.

• 3. Tablet slot: The interactive board also includes a housing slot for a 10-inches tablet (Fig. 4.3) for the execution of the Web application.

These components are controlled by an Arduino Mega 2560 board, which is connected to the tablet through an ESP8266 WiFi Module. The external input and output components are instead directly connected to the board.

The cards and the paper board have been re-designed to give a new visual identity to the game (Fig. 5). In addition, on the back, each card has now an AZTEC code that can be recognized by the Web application.

The Web application then includes functionality for:

• 1. Visual card recognition: Through the tablet front camera, it recognizes the card AZTEC code.

• 2. Voice-based UI: A controller allows participants to interact by voice with the app.

• 3. Speech recording: The users can also record and store voice comments.

Fig. 5.

COBO restyled cards and game board.

Given these components and functions, the users interacts with COBO through the following steps:

Fig. 6.

COBO user experience flow.

• 1. On-boarding and continuous assistance: The moderator prepares the playground and positions the deck of cards (Fig. 6A). The role of the moderator (Major) is complemented by Virginia, an avatar/conversational agent provided by the Web application, which supports players both in learning the game rules and in the crucial phases of ideation through card combination (Fig. 6B). Virginia is depicted as a woman university researcher acting as a reference point for interactions. She can assist the players through visual and auditory channels, activated by pressing an always-active help button.

• 2. Card recognition: When the player collects enough cards, she can have COBO recognizing them by simply placing their AZTEC code in front of the tablet camera (Fig. 6C). When the card code is recognized, the card name is told and its virtual representation is shown on the screen (Fig. 6D).

• 3. Combination feedback: Once a combination of 3 cards is added (input, environment and output), the user can plug the environment physical objects into the actuator bases (Fig. 6E); while interacting with buttons and sensors, the user sees in real-time a “live” version of the card combination, which is conveyed through the multisensory functions of the board.

• 4. Idea recording: Players can finally voice-record the idea they had in mind. Once the registration is completed, it can be re-played and saved for future listenings (Fig. 6F). In case of any failure (e.g., wrong mounting of the objects on the interactive bases) the application is able to guide the players to undertake the right actions through spoken and written recommendations.

5 Remote Exploratory Study

We run an exploratory study to gather participants feedback on the latest COBO extensions and evaluate their level of engagement and likability of the interactive prototype. In order to get feedback consistent with the previous evaluation sessions, we involved the same four participants with ID. Due to COVID-19 restrictions we could not access the social-care center, thus we organized a Web call on the Google Meet platform. The participants were located at F&A, and were coordinated by one of the two special-education professionals who assisted them in the first two workshops. Two researchers, remotely connected from their offices, guided the whole experience: one acted as moderator, the other provided technical support to ensure the high quality of the streaming. In addition, two other researchers silently attended the call and took notes.

Setting and Materials. A reflex camera was used to frame the moderator, the interactive board prototype, and the cards. A second computer was used by the second researcher to run and share the Web application. The session was audio-video recorded.

During the session, the two researchers guiding the activities used the final prototype of the interactive board, the paper board and the augmented cards. The participants with ID were provided with the cards only.

Procedure. Once all the participants connected to the online room, the moderators briefly recalled the board game rules and illustrated the new features of the augmented cards. The moderator then proceeded by explaining the main functions of the interactive board and the Web application. Since the session was held online, the game was not replicated entirely, while the focus was kept on the phases that most required the use of the new interactive elements (Fig. 7):

Fig. 7.

A) Users become familiar with the available cards; B) One of the users shows the chosen card; C) Users observe the moderator interacting with the interactive board by showing the effect of the chosen card combination.

• 1. First of all we showed how the board could recognize the cards. We started from a card combination consisting of an input card (button), an object card (sign) and an output card (color light). While the main moderator with the interactive board were showing to the users how to make the app recognize a card, the second moderator were sharing her screen where the Web app was running. The same procedure was followed for reading other cards and for showing the participants the effect on the Web app.

• 2. We then proceeded by showing two other card combinations, and how they could be read and managed in the application. Users were asked to interpret the cards, before activating them within the app, and to think of an idea that could refer to the combination they were going to activate.

• 3. We then illustrated how the conversational assistant, Virginia, was able to recognize questions such as: “Is there a correct order to add the cards?” or “How do I cancel a card?”. For the demonstration, participants were asked if they had any questions on how to proceed; their questions were vocally asked to the conversational agent by the moderator.

• 4. We showed the procedure to vocally record their ideas, and to save and replay the audio.

• 5. At the end of the session users were asked whether they had any further questions or concerns about the game and whether they found the game interesting and easy to use.

Final Questionnaire. After the study, participants were asked to fill out a questionnaire to evaluate how much they liked the game and appreciated the various components of the new toolkit⁴. The questionnaire was composed of seven questions to be answered through an iconic Likert scale inspired to the smiley-o-meter representation [32]. One question asked the users to express how much they liked the game in general. Then six questions asked how much they appreciated the different elements of the toolkit: the interactive board, its knobs and sensors, the multisensory effects, the Web application, the card reading mechanisms, the conversational agent. The three last questions, to be answered with “yes” or “no”, asked if they perceived the game as easy to use, if it stimulated collaboration among them, and if they wanted to play it again.

Fig. 8.

Answers of the questionnaire submitted to the participants after Workshop 2.

Results. The results of the questionnaire, the video-recordings and the taken notes were then analyzed to collect insights. The four participants gave positive feedback about the material, especially the interactive board. As illustrated in Fig. 8, in a scale from 1 to 5, participants, evaluated all the toolkit components with scores equal to 4 and 5 (“satisfying” and “very satisfying”). Only one participant evaluated with 3 the way of reading the cards. Two participants then gave the maximum score to all the questions concerning each specific element of the game. The maximum rating given to the question “Do you like the buttons and sensors of the board” lets us think that the interaction with tangibles was appealing for the users, and this is in line with the results of others studies [14]. Participants also gave the maximum rate for the question “Do you like the game?”, as an expression of their interest in the phygital extension of the game.

In relation to usability, the positive responses received for the question on the “ease of use” confirm that the interactive game, grounded on the accessibility guidelines [38], allowed the users to understand how to use the new features. While observing them it emerged that, by exploring card combinations and getting direct feedback, participants were able to build their understanding of how the game works, and this also made interaction easier.

The interactive mechanisms in general were considered easy to use. After illustrating how the application recognizes cards, one of the users said: “This game is very easy, I like it!”. All the participants also said they would love to play again with COBO in the future.

6 Discussion

From an educational perspective, we gathered positive feedback during the whole process of design and evaluation. The four participants with ID showed us to feel involved in the design of new interactive artefacts, and at each workshop they asked us to organize further sessions. They also asked if the smart experiences designed in the different sessions had been implemented by some expert developers. We interpret this request as their engagement with the design process.

Considering individual characteristics, we did not observe specific hesitations of the participants to start playing during our workshops. It seemed they did not feel evaluated: they were always willing to express and share their ideas with the other palyers, the researchers and the educators.

The ideas that emerged during the whole process (see Sect. 3.1) also confirmed the willingness of users to think about solutions related to the accessibility of the park for people with disabilities. This can be interpreted as an indication of their sense of empowerment, as they felt responsible in designing interactive experiences that could be useful for other fragile people, a feeling that could lead them to a greater sense of citizenship and social participation. This also makes us think that the use of COBO could induce users to reflect on their own needs. More in general, a relevant aspect is that our co-design method allowed participants to make concrete proposals to real problems that could arise in everyday situations, as designing a new inclusive urban park.

It is worth noticing that COBO currently focuses on the ideation of interactive ideas for the city-park environment and provides participants with cards related to this context. However, COBO was designed to be easily expanded to new environments by simply adding new sets of cards and plug-in objects. The flexibility in expanding the domain of the game is coherent with both the requests of the users participating to the study, who asked to be involved in ideation sessions for other situations, and with the suggestions provided by the professionals in the online survey carried out after the first workshop. Varying the addressed domain would expand the potential of the design process for smart activities, providing people with ID with the opportunity to focus on other environments where they generally act in their everyday life, and where they might potentially encounter situations that can be digitally enhanced.

6.1 Limitations

Although the research had positive findings, there were some unavoidable limitations. First of all, the self-reported data could have introduced biases, such as telescoping, attribution, exaggeration. The difficulty in observing the participants during the remote study might have also induced professionals and researchers to hypothesize some facts [25] leading to further biases. To generalize the results, the study should be replicated with a larger group of participants and for a longer period to observe post-novelty and generalization of effects.

The participants involved had already experienced working together both at the day-care center and with technological devices; furthermore, their communication skills are preserved, both in terms of ideation and phonoarticulatory apparatus, and that made playing with COBO quite easy for them. As a consequence, although we feel COBO played an important role as a social mediator, we are not able to ascribe whether the method was the principal factor enabling collaboration between them, as the group dynamics were already established.

Finally, to preserve participants’ safety during the COVID-19 pandemic, we opted not to enter the center. Giving participants the possibility to share the playground with us would have potentially enhanced the collaboration.

7 Conclusions and Future Work

Our design approach allowed us to collect several insights from the involved young adults with ID, but it also gave them the chance to feel empowered in solving a concrete problem, i.e., the design of smart outdoor experiences. Participants played a crucial role during the process for revising the SNaP method and introducing the phygital elements; they contributed with very important feedback about the game dynamics and the interactive board prototype. Educators’ suggestions were valuable as well: they helped us identify tangible and digital interactive material as a viable solution to enhance participants’ ability to understand the method and ideate. The final version of the interactive board seemed to allow the participants to enhance their self-accomplishment. Their response during the workshops suggests that it could be helpful for enhancing both the learning and the creativity process.

Our future work will focus on systematically assessing the observed benefits through user studies involving a larger sample of users and in a longer period, also expanding the number of city-related environments available in the game. We plan to involve users who have never worked together in order to verify and evaluate how group dynamics and communication skills evolve during the game. Moreover, it would be interesting to involve people with lower IQ levels and/or with communication disorders, to see whether COBO, together with group dynamics, could promote communication between them.

A long-term goal could be to investigate the use of COBO to help people with ID to improve their communication and logic skills, and to enhance their creative process, which is usually lacking. This would be in line with the results of some recent works (e.g., [12]) that promote tangible toolkits for the playful discovery of electronics, a further form of empowerment that could introduce young adults with ID to making and thinkering practices.