Keywords

1 Introduction

There is a general agreement on the idea that playing can boost brain functions and improve well-being [2]. Playing is a pervasive cognitively demanding activity that, as shown in a recent study [8], can yield to structural changes in the brain. Serious Games (SG) designed on a hand-to-hand collaboration between game developers and neuroscientists can have a more effective impact on cognitive training than purely leisure games [1].

In particular, SG potentially represent new and effective tools in the management and treatment of cognitive impairments in the elderly [4]. Indeed, there are evidences that adults engaged in computer activities have decreased odds of developing Mild Cognitive Impairments (MCI) [13], and that virtual reality-based memory training can contribute to prevent memory decline [14] and reduce depressive symptoms [7] in elderly adults.

Our main hypothesis is that in addition to rehabilitation and training, SG can also be an effective tool for the detection of cognitive impairments and dementia. Therefore, they can be used to perform large-scale, cheaper screening of the population over 50 yielding to earlier detection of cognitive impairments and anticipated enrollment in rehabilitation programs. A secondary hypothesis is that the efficacy of this tool can be higher if it is based on daily life activities in a realistic 3D scenario in which the user’s virtual skills are representative of the actual user skills.

On the basis of this hypothesis and with the ambition of partially substituting pen-and-paper-based tests, we have designed and implemented SmartAgeing [17], a web-based electronic test of Mild Cognitive Impairments (MCI) based on the SG technology. In the development of this new technology, we have faced various challenges to make it accessible to a diversity of users most of them without computer skills. We describe our solution in the next sections. A preliminary description of this work have been presented as an extended abstract [17] in Rehab2014. We herein describe it in depth and present and discuss the results of the usability tests.

2 Scenario

The scenario represents a 3D home-like space (loft) providing a diversity of ambiances in one room: dining room, sitting room, bedroom and kitchen, plus a separate bathroom. It is filled with furniture and objects that users can manipulate virtually (see Fig. 1).

The choice of the scenario is motivated by two major reasons: first to be a familiar place that anyone can recognize, and second to be an open space with specialized areas (room, kitchen, sitting room) where a variety of exercises can be done without having to navigate through doors and corridors, because navigation is difficult for most users unfamiliar with computers and games.

The objects that appear in the scenario are commonly used, as international as possible, so that anyone can recognize them. Their design is a trade-off between realism and shape. For many objects, such as food cans, the geometry is very simple and the identification of the product is based mostly on the texture mapped on the model. To make easier the internationalization of the test, we have avoided to overuse text in the textures, and have tried to apply images with a high iconic value, be them drawings or real-life pictures.

The location of the objects in the scenario is configurable, so that the task can be realized several times with different challenges. In all configurations, however, the location of the objects is of common sense in order to avoid unnecessary traps in the tasks of finding objects.

Fig. 1.
figure 1

A view of the virtual scenario

Full size image

3 User Interface

3.1 Dimension

The game is designed in a first-person perspective so the player experiences the action through the eyes of an invisible avatar which reinforces his/her involvement in the game. The actions are dimensionally congruent [5]: navigation and manipulation are done in 3D, but actions that are inherently 2D such as reading and writing are done using 2D interfaces launched from the 3D virtual environment. As an example, in order to dial a telephone number, users interact with the 3D model of the telephone. This causes the display of a 2D view of the telephone where users can dial the numbers very easily (see Fig. 2). Once finished, the game comes back to the last view of the 3D scenario. In this way, the game naturally accommodates the two visualization and interaction models within a same framework.

Fig. 2.
figure 2

An example of integration of 2D tasks in the 3D environment

Full size image

3.2 Degrees-of-Freedom

In order to simplify as much as possible the interface, the interaction has been reduced to three degrees of freedom: two for camera orientation control and one for selection. Locomotion is semi-automatic, based on the Point Of Interest (POI) paradigm [9]. Users click on a POI. Since the scenario is a closed space, all selections fall onto a surface of an environment. If the object at the POI is within the user’s avatar reach, the game performs the action associated with the object at that moment of the play, for instance picking, dropping or opening. Otherwise, the best path to reach the POI avoiding collisions is computed [10]. Then, the path is followed automatically at a constant height over the floor’s plane simulating a smooth walking. During the automatic navigation, all interactions are disabled. At still positions, users can perform a view-around action implemented as rotations of the pitch and yaw angles (elevation and axial rotation of the head). To prevent exaggerate elevations, a maximum pitch angle is fixed so that at still positions and orientation, after a delay, the system smoothly recovers the horizontality. In this way, users have all the advantages of a 3D environment, but they are not required to master locomotion. More complex movements such as kneeling or stretching are not allowed, since they would complicate the interface unnecessarily. Objects too high for the user’s reach are not accessible and just used for decoration.

3.3 Input Devices

Two types of input devices have been tested: with mouse and with a touch screen. In the mouse version, the cursor is always centered in the middle of the graphical area. The mouse movement is mapped to pitch and yaw rotations, and the selection is done by clicking the left mouse button. The object selected is, thus, always at the center of the screen. The preliminary user tests performed during the development of the project [11] showed that this navigation was very intuitive for young users and seniors with computer literacy. However, elderly users not familiarized with computers had difficulties in mapping the mouse movement with the cursor, and having to move the camera center to select an object already visible in the current view confused them. Therefore, we implemented a touch-screen version of the interface. In order to keep the simplicity of the three degrees of freedom, we use a single-touch interface. The cursor is free, so users can select any POI with a simple hit on the screen. For the camera orientation, we chose the widget-based paradigm rather than a gesture-only model [12] in order to clearly separate orientation from selection. Instead of the traditional mini-map of the navigation controls at a corner of the screen, that seemed too abstract, we preferred to surround the 3D view by a partially transparent 2D frame. Users touch the upper frame to elevate the head (pitch angle) and the bottom frame to lower it. They touch the left and right frame to rotate it (yaw angle). Figure 3 shows the interface with the two types of input devices.

Fig. 3.
figure 3

Three-degrees of freedom interface. At left with a mouse input device: the cursor (shown as a hand) is centered, the mouse movement is mapped to the yaw and pitch angles, and selection is always done at the center of the window where the cursor is fixed. At right with single-touch screen, the frame is used for camera orientation, and the cursor is free for selection at any point of the window. Users do not need to change the orientation to select an object visible in the current frame.

Full size image

3.4 Manipulation

The available actions with selected objects can be classified in two categories: instantaneous actions and time-lasting actions. The former ones are done immediately, when an interaction is detected. They are to pick, to drop, to turn on and off, to open and close doors. Time-lasting actions such as dragging an object, playing music or filling a can are also available. Picking yields to attaching to the cursor a miniaturized version of the objects that do not collide with the environment. Dragging an object is thus done implicitly through navigation and viewing around with a picked object attached to the cursor. Dropping is then achieved by clicking onto the desired surface.

4 Tasks

The test consists of five tasks addressing different cognitive aspects: executive functions (reasoning and planning), attention (selected and divided), memory (short and long term, perspective), orientation (visuo-spatial). Table 1 summarizes the tasks.

Before starting the tasks, the game presents a traveling through the environment to familiarize the user with it. During this traveling, all the doors and drawers of the cupboards and furniture of the loft are opened one after the other to show their content. Next, a familiarization task is proposed in which users practice the navigation and the selection of objects. They are asked to go to a specific place, to pick and drag objects and to drop them.

The first task consists of finding in the 3D environment the objects shown in a 2D panel at the left of the screen, in total 12 objects. The image in the panel is a representative view of the objects from which they are clearly identifiable. All the objects have been shown in their current location in the initial traveling. Some of them are visible at a simple glance, but most of them require to open drawers or doors. The task requires memory, spatial orientation and attention at the same time.

Table 1. Tasks of the test
Full size table

In the second task, users are asked to turn on the radio and listen. They must perform a click each time they listen a specific word. With the mouse-driven input, the click is performed with the left button of the mouse, and in the touch screen version, a 2D button is shown at the bottom part of the screen, above the navigation bar. After a while, users are required to water the flowers while keeping clicking when they listen the word. Watering is a sequential activity: picking the watering can, filling it in the tap (which requires to first turn on the faucet and after having filled the can, recalling to turn it off), and finally, watering the flowers. Thus, the task involves divided attention and executive planning functions.

In the third task, users are first warned to turn on the television at the end of the task. Then, they must select the agenda that is on top of the table near the bed. The selection deploys a 2D interface where users must find open the agenda at the right page and then, click on the proper name. Then, the telephone number is shown. Users learn it during a maximum lapse of time that they can shorten freely. After that, the game comes back to the 3D environment where users select the telephone yielding to the deployment of a 2D interface for dialing. At the end of the task users must remember to turn on the television. This task involves short-term and long-term memory and executive functions and selective attention.

Finally tasks 4 and 5 are related to task 1. In task 4, a 2D panel with 20 objects is shown. Users must identify in the panel the 12 objects selected in task 1 among the impostors. This task tests memory. In task 5 the user must find the objects in 3D as in task 1, but without following any instruction. It is therefore a long-term memory exercise coupled with spatial orientation and attention.

5 Indices

The game records all users actions. From these data, it computes a set of indices for each task of the game separately. At the end of the game these indices are parsed to give an overall score of the patient’s cognitive skills. Currently, clinicians are working on the evaluation model, with the goal of making it equivalent to standardized measures based on pen and paper screening tests. Recording all the indices provides flexibility in adjusting the model during the validation stage.

Table 2. Main indices registered for each task of the game
Full size table

The parameters registered by the game are summarized in Table 2. We differentiate between correct and incorrect actions. Correct actions can be done on time or within a pre-defined delay. For instance, in task 1, identifying an object is correct and on time if the picture of the object is shown in the panel when the identification is done. It is correct but not in time when it is identified after the object has been shown and within a given delay. It is incorrect if the object has never been shown before or when the delay has expired.

6 Validation

6.1 Usability Validation

The development of the game was done on the basis of a strong collaboration between clinicians and game developers and following a user-centric development paradigm. Thus, throughout the development of the project, the game was periodically evaluated by clinicians and tested by small groups of users including patients with cognitive decline.

At the end of the development we performed a usability test on 16 users aged from 55 to 82 (average age of 64.4), 7 women and 9 men, without any 3D computer game experience but with different computer skills. None of the participants were diagnosed of cognitive decline. The participants were volunteers, recruited in the social network of the team and had no relationship with the project. Each test was performed separately and conducted by a member of the team acting as facilitator. Seven of them used a touch screen, while the other ten used the mouse. At the beginning of the experiment, the facilitator clarified the aims of the experiment and stated clearly that only the usability of the game was going to be measured, not the cognitive skills of the participants. During the experiment the facilitator encouraged the participants to “think aloud” and freely comment on the interaction techniques. At the end, the participants filled a 10-questions SUS test [3] extended with 4 specific questions (SUS-E, see Table 3). The minimum, maximum and average values of each question are shown in Fig. 4. Table 4 shows the extended SUS scores for all participants, sorted by age. This SUS-E score corresponds to the 4 additional questions. It was computed similarly to the SUS score by giving for positive questions (P11, P12, P13 and P5) the selected value in the Likert scale minus one and, for negative questions, five minus the selected value. The sum of points has been multiplied by 5 to have a scale from 0 to 100. Finally, Table 5 shows the maximum, minimum average and standard deviation of the SUS and SUS-E scores.

Table 3. The SUS test extended with 4 additional questions.
Full size table
Table 4. Usability analysis results
Full size table
Fig. 4.
figure 4

Minimum, average and maximum values of the extended SUS quest.

Full size image
Table 5. Usability test scores
Full size table
Table 6. Time required to perform the tasks
Full size table

All participants could finish the four tasks with only very few errors. The time needed to complete the tasks were rather similar, except for Task 4 that some users completed very quickly (see Table 6). Participants were nervous at the beginning, fearing not being able to play, but they felt quickly confident. Although none of them had any experience with 3D environments, they immediately understood the first-person perspective and had a feeling of immersion in the environment. In general, all users liked the game and found it usable and fun. The SUS and SUS-E scores are in general very high. Participants that were tested with the touch screen were invited to test also the mouse input version. All them referred to feel more comfortable with the touch screen that they found easier to master, even if they were used to the mouse. The average SUS and SUS-E scores of these users is higher that the score of the other users: 95.36 and 92.14 versus 85.83 and 85. This result coincides with the preliminary tests done on neuropsychological patients that yield to the development of the touch screen version. Some of touch screen users, that were familiar with tablet devices, tried to drag the objects with the finger and to apply gesture to rotate the camera. This may indicate that a gesture version of the game could also be useful. A few of them commented that the design of a couple of objects (banana and toasts) was not enough realistic. These objects have now been changed.

7 Scientific Validation

The scientific validation stage is currently in process with a primary target group represented by 1000, 50 years old and older persons, 50 to 80 of which are already diagnosed with MCI and/or neurodegenerative dementia. MCI patients are recruited from IRCCS Mondino and Don Gnocchi Foundation, which are two important centers for the diagnosis and treatment of neurological disorders. Subjects are already diagnosed as affected by MCI according to [15] criteria and selected from the institutions’ patient registries. The inclusion criteria are a Mini Mental State Examination - MMSE [6] score \(>\) 24 and the memory domain subscale of the Clinical Dementia Rating Scale \(<\) 0.5.

The sample of 1000 users are recruited from aggregation centers, public entities as well as through newspaper advertisements. The sample will be stratified according to gender (female/male), age (50–60, 61–70, 71–80), and education (primary, middle, high, university). The inclusion criteria are a negative history for neurologic and psychiatric diseases, a MMSE score \(>\) 24 and a performance within normative ranges for age and education at the pen and pencil tests. It is expected that at least 8 % [16] of these participants will be excluded from the validation stage failing to pass the inclusion criteria, and then resulting as potentially affected by any neurological disorders. As shown in Fig. 5 in the validation stage, the evaluation model will be refined to compute a score equivalent to standardized scores for pen and pencil test.

Fig. 5.
figure 5

Schema of the scientific validation process

Full size image

8 Conclusions

The SmartAgeing serious game is a novel methodology to detect cognitive impairments while performing 3D real-life tasks. It brings an automatic way of reporting the users performance through the registration of many run-time parameters. The usability results of the tests performed throughout the development of the game and at its end show that the game is easy-to-use and fun, specially the touch screen version. After the scientific validation stage, we expect to be able to show its efficacy and ecological value. Once deployed for large-scale screening campaigns, it will require less resources both in terms of time and human activity that current methodologies. Finally, the SmartAgeing scenario can also be used to perform rehabilitation tasks for diagnosed patients. In this case, it will provide a coherent environment that will allow a continuous tracking of the patients evolution. We are currently working on this extension of the system.