Keywords

1 Introduction

Information technology is transforming many different areas, even the area of rehabilitation. Serious games are finding a use as an alternative for generating meaningful experiences in different contexts of use. A serious game can be defined as a balance between the entertainments and pedagogical [2]. Auditory Therapy (AT) is the process whereby the child learns to use his hearing to the fullest. The goal of rehabilitation is to help children learn to extract or take information from the stimuli they perceive via the cochlear implantFootnote 1. The incorporation of digital games in therapy can make a healthy contribution in such a way that they bring together entertainment and education [1, 2] and can be integrated into the rehabilitation process.

Today, games have adapted formal Human Computer Interaction techniques in order to assess interaction and product quality [3,4,5,6,7,8]. Usability is a quality attribute that determines user satisfaction and consequently the product. This is defined by ISO 9241 as “a product can be used by specified users to achieve goals in a specified use context” [9]. According to Nielsen [10] usability comprises five attributes: ease of learning (Learnability), efficiency (Efficiency), ease of memorization (Memorability), low error rate (Errors – Low Rate) and satisfaction (Satisfaction). The HCI approach furthermore involves a design philosophy that aims to create products that meet specific needs of the end users, achieving a better satisfaction and user experience.

User experience (UX) can meanwhile be understood as a set of feelings and emotions that are produced in a user on interacting with an interactive product [11], such as a serious game. Thus, the experience children have with a product depends on the skills they have for interacting with it more easily or with more difficulty [12, 13]. Therefore, the evaluation of a product focused on children is a different process compared with adults [14] and the focus of the technology is different [15]. Thus, the design and development of a therapeutic game for children is not a simple task, because with children come particular requirements.

This article is structured as follows. Section 2 begins with a brief description of children with cochlear implants and the challenges that they must overcome. Section 3 discusses different methods for assessing usability that have been applied to children and how these could be adapted to the characteristics of a child. Meanwhile in Sect. 4 is applied a study case to assess games oriented to auditory-verbal therapy for children between 7–11 years old with a group of children with cochlear implants from the Institute for Blind and Deaf Children from the Cauca Valley department of Colombia is applied. Finally, conclusions and future work are described.

2 Children with Cochlear Implants

Deaf children can benefit from fitting a cochlear implant, an electronic device that picks up sounds from the environment (noise, sounds, words) and transforms them into electrical energy that can directly stimulate the auditory nerve endings, producing auditory sensations in the brain [16]. Children with cochlear implants are beneficiaries of auditory-verbal therapy, where they must learn to listen and identify sounds to learn to speak [16, 17].

A child who receives auditory-verbal therapy must extract information from the stimuli picked up by the cochlear implant. Rehabilitation is supported with educational material aimed at stimulating the senses of the patient with the implant. The children meanwhile are not able to develop skills at the same pace as hearing children. This makes it difficult to identify problems in the development of their cognitive skills and can affect their progress in the acquisition of learning [18]. This indicates that they need more attention and, in turn, motivation for learning, as they are easily distracted, especially when they lose interest in the task or the difficulty level of the task is high in comparison to their ability level.

3 Assessment with Children

When technology is evaluated with children it is important to define the purpose of the evaluation and to understand the data gathered. Existing evaluation methods tend to focus on usability and user experience. ISO 9421-210 defines “user experience as the perception of a person and responses that result from the use of a product or service” [11], i.e. when emotions produced in the children are observed directly. User experience is subjective and cannot be captured using traditional usability metrics such as time tasks or errors. Also, if we measure the use of a tool for children with hearing disabilities, we must evaluate it based on basic skills. This indicates that these metrics may vary depending on the user profile and the purpose of the evaluation. This implies the need to establish evaluation methods adapted to the level of difficult; i.e. if a child with a cochlear implant is only just acquiring skills in speech, the Thinking Aloud method is not the most appropriate for them.

This method is aimed at children who can establish a channel of communication through speech, so it becomes quite complicated for a child with a cochlear implant wherever speech is involved, since a lot of the children are in the process of language acquisition and learning how to listen: some research [19] works do not recommend this method, because the child must carry out two simultaneous actions – complete the task and give verbal information on the activity, since the children make very few comments. As such, there are methods based on drawings [20] as an alternative to the verbalization methods, such as Drawing Intervention [21], a method that is used to elicit visual information from the child by means of drawing.

Furthermore, several studies [21, 28,29,30] have analyzed different methods for the evaluation of interactive products applied in children. Based on these studies, Table 1 shows some of the advantages and disadvantages found in each evaluation method. Of the methods that have been analyzed (Table 1), it ought to be noted that capturing the attention of a child is not an easy task. A communication channel must be established that makes it possible to establish greater concentration on the activities to be carried out during the test tasks. In turn, this channel may vary depending on whether a disability is present and the type of disability. Thus, a child with hearing impairment is more visual, and if they have a cochlear implant they can learn to speak, so that their objective is to further strengthen channels of communication through listening without losing the visual channel.

Table 1. UX assessment methods for children.
Full size table

Meanwhile, there are measurement instruments of qualitative (subjective) and quantitative (objective), which correspond to non-verbal and verbal instruments [15, 32]. These instruments are used with the evaluation methods to capture the emotional, physical and aesthetic experiences of the user [31]. The majority of the methods are especially adapted to the experience of the user, for example: Fun Toolkit [6], identification of images on cards (Picture Cards) [24], simple observation, thinking in a loud voice (Thinking Aloud) [23], and Laddering [26], among others. While many of the methods are not applied to all contexts of use and need to be well understood to know when to apply them and when not to [33], because many of those are not adapt to children with special needs and not considering if children have some type disability, it indiques that many of methods proposed are designed for children in normal conditions.

Direct observation and verbalization methods such as Thinking Aloud, Picture Card are responsible for gathering information about the experience of the user as they interact with the system. However, the method may prove difficult at times, so that the children on feeling that they are being watched change their attitude in the moment of the test and can at times be considered intrusive [34]. The child’s cognitive skills can affect how easily they understand each of the methods presented to them. The attention span of a child, in most cases, is limited and they are able to focus for about 30 min, so that activities are limited by time [7]. As a result, the method called Fun Toolkit [6] is designed for children who require a reduced cognitive skill level, since to be able to respond, the child must fill in or select one of the options presented to them. Smileyometer [35] is a visual scale tool (VAS) based on the Likert scale with ratings from 1 to 5, where each level on the scale is represented by facial expressions, i.e. a disappointed face corresponds to (1) and a very happy face to (5).

This technique has been used in previous studies to measure satisfaction [8] and fun. However, if a child is unfamiliar with some of the emotions that are presented on the scale, the selection of the child will perhaps not be correct. One possible alternative is EMODIANA [36], a visual tool based on 10 graphic representations of different emotions of a character and the intensity of the emotions by means of a target, by which evaluation of emotions for children 7–12 years was conducted. The objective of Picture Card [34], meanwhile, is to find usability problems and images are used to verbalize sentences. It is used for children who have language problems, as a way of establishing a communication between the child and the evaluator.

The channel of communication of children with cochlear implants in the early stages of learning how to listen is primarily visual. Teachers therefore base their teaching of concepts on pictograms accompanied by sounds [16]. In the therapeutic context, children with cochlear implants must develop auditory qualities, for which they need to follow a process that involves a number of stages, such as: detection, discrimination, identification, and understanding. Many children have trouble pronouncing certain specific phonemes in the middle of words. Many also have a particular problem. The audiologists therefore perform therapies individually, which takes more time.

Suitable evaluation methods are sought for this reason, that make it possible to identify the needs of children with cochlear implants, in such a way that, according to the level and characteristics of the learning, suitable evaluation techniques can be adapted to involve the child in the design of the serious game.

There are a number of different methods of evaluation which in turn must be adapted to suit the profile of the child, since not all methods are adapted in the same way for them to use. Evaluation methods provide support for measuring usability, user experience, or both. Theses methods applied will be either subjective or objective, depending on the type of activity to be conducted with each child. The evaluation methods that have been used for children with cochlear implant, are: Direct observation, fun toolkit, test of usability, video analysis, interviews, drawing intervention and picture card.

4 Case Study

The case study is applied to 11 children with cochlear implants - eight boys and three girls between the ages of 7–11 years in the Institute for Blind and Deaf Children, in Cali, Colombia. All eleven children are profoundly deaf in both ears. Auditory-verbal therapy for these children is currently carried out with the support of the speech therapist by means of a number of different activities that involve very little technology.

The purpose of this evaluation is to identify different aspects of user experience and usability following the work proposal by Cano et al. [42, 43], with the aim of designing an interface that is useful to the audiologist in being integrated into therapies that use technology. Among the activities for evaluating the inclusion of technology in user experience, three games oriented to voice and speech therapy were selected, e.g. Talking with Teo [13] Pre-lingua [14] and Vivoso [15]. The three games are used in PCs, where Talking to Teo supports speech therapy, while Pre-lingua and Vivoso are aimed at voice work skills such as pitch, timbre, intensity, breath, and vowel articulation. The different activities carried out with the children lasted 20–36 min. The objective is to evaluate usability for each game in order to identify needs and improvements in a game oriented to auditory-verbal therapy.

4.1 UX Evaluation Methods for Children

An evaluation method is defined as those procedures, objective or subjective, used to obtain information in relation to the performance objectives. To determine the aspects that are most relevant to the child with cochlear implant, methods of inquiry are used, such as interviews and questionnaires to teachers and speech therapists in the interest of extracting information about the child such as learning styles, behaviors, interests, and others. In turn, the technique of direct observation was applied, to observe the interaction between teacher-child and speech therapist-child.

Children with cochlear implants tend to communicate visually; so appropriate methods of verbalization were selected. Fun Toolkit was used in the beginning, adapting nonverbal expressions to formulate question and answer combinations. Only two rating scales are used - This or That. The questions are related to the experience they feel on interacting with the game. As a result, use was made of an instrument called Smileyometer or VAS (Visual Analogue Scale), where the scale is modified to include just two faces and this changes according to the gender of the child. The faces show either a happy face to indicate a positive response or a sad face for a negative response (Fig. 1). In the early stages of the evaluation, all five faces represented in the Likert scale were used, but on being shown these different expressions some of the children became confused either because some of the expressions were unknown to them or they were too similar to each other. When teachers introduced them to a new evaluation framework, they were filled with doubts and became confused, so this also influence the decision to continue using just the two faces for the rating scale. This method is helpful for use as a form of communication response, where children express their opinions by identifying themselves with the feelings.

Fig. 1.
figure 1

Proposal for improving Smileyometer tool, adapted to include just two emotional states, depending on the gender of the child

Full size image

Meanwhile, another method called drawing intervention [21] communicates the user experience through drawings, where the children are not required to speak or comment. This method is based on the observation about what the children draw and has been used to understand children’s thinking. A modification was made to this method so that the objective for each child was to construct an animated character in order to be more involved in the game (Fig. 2). Therefore, using a set of body parts supplied to each child, two characters were built using all the pieces as they pleased.

Fig. 2.
figure 2

Applying the drawing intervention method for children from 7–11 years

Full size image

Finally, each of these interactions was recorded on video. To evaluate the user experience using the proposed tools, two videos were recorded: one in front of the children, in order to observe their facial expressions and one behind, to observe the interaction of the child with the game. Notes were taken by two assessors, where one interacts with the child during the activities, while the other observes and records his observations. The information obtained is finally analyzed.

Selection of the methods depended on the level of schooling of the children. For example, for children in prekindergarten whose ages range from 5–9 years, the only method that could be applied was Drawing Intervention, because they don’t yet know how to write, so that mainly the visual channel was used. Some of them vocalize, but lack oral clarity and are not easily understood. Moreover, many of them have been using the implant for less than a year. They also have a very poor vocabulary and don’t manage to write their name well, which is why the interaction with them was visual only.

4.2 Experiment Design

One experiment consisted of selecting a set of interactive tools for speech and voice therapy. These tools interact with the child through the microphone, into which they pronounce a given phoneme and voice aspects such as intensity, pitch, and timbre are evaluated. Tools such as QUIS [37] that relates to technology, USE [38], to user experience, GEQ [39], and UEQ [40] are used in order to evaluate the established metrics, as shown in Fig. 3, so as to assess usability.

Fig. 3.
figure 3

Applying the direct observation method for children aged 7 to 11 years.

Full size image

The “Talking with TEO” game meanwhile consisted in using a microphone to capture the phonemes “da-de-di-do-du” with a number of configurable repetitions for each phoneme pronounced correctly, where the child receives a star as a rating. Elsewhere, the Pre-lingua tool helps the child to acquire some sound characteristics such as pitch, timbre, and breath. Pre-lingua uses a set of mini-games to evaluate sound characteristics. The activity consisted in the child pronouncing a certain vowel, where through play they detect the presence of sound with a suitable voice timbre and the car moves along until it reaches its target destination.

Other kinds of activities are also done on paper, such as the children drawing themselves, or putting together a character from pieces handed out to them, where they are required to choose parts of the body and face, and go on to piece them together (Fig. 2).

5 Conclusions and Future Work

The purpose of this article is to identify elements of entertainment that can be incorporated into games for rehabilitation. It is therefore important to include the children in the design process, to find out their opinions. Moreover, not all children have the same abilities, so a means of communicating with them ought to be established. Existing methods of evaluation were adapted, in such a way that the motivation of the children could be captured and thus their views.

The evaluation provides support for following required guidelines and choosing techniques adapted to the characteristics of the child. In addition, follows user-centered design philosophy, in such a way that takes account of human and demographic factors of the children.

As future work, it is intended to incorporate evaluation methods that make it possible to abstract cognitive skills of the children and thus automatically adapt these evaluation methods according to the characteristics of each child.