Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Exergames for Rehabilitation

In the last years, National Health Service Providers have become overly saturated and are forced to shorten the duration of the rehabilitation service, increasing the period in which patients have to rehabilitate outside the hospital [1]. Enabling patients to exercise at home seems a good answer and serious games are believed to be the most adequate tool to provide this opportunity due to their motivational benefits. Therefore, much research from both academics and the industry has been directed towards creating games for rehabilitation, with the underlying purpose of providing a correct rehabilitation session while entertaining the patient [2-3]. The exergames combine the benefits of exercising with the engagement, motivation and fun typical of gaming applications.

Many games for rehabilitation are nowadays available for a wide range of dysfunctions and there is a broad agreement that they are beneficial [4] although few systematic clinical studies have been carried out and almost none for their use in a home environment without supervision of a therapist. One of the first works to be completed has been published this year [5] and it is based on the use of the Wii Fit games for postural rehabilitation. In the same study, however, several adverse events were registered, namely related to pain at the hip and knee joints developed during the training; the major risk highlighted in the study is that patients do not execute the exercises correctly and develop maladaptation that, in turn, may easily lead to training-related injuries.

This calls for methods to monitor the patient during her training: this is a fundamental feature that distinguishes any exergame developed for fitness from the ones developed for rehabilitation. Moreover, a clear feedback on wrong doing should also be given to the patient while gaming so that he can correct wrong postures and attitudes. Besides this, clinicians should also have the possibility to analyse the outcome and trend of the rehabilitation to advice and tune the therapy and to adapt the exercise’s difficulty to the patient both to make the exercise more effective and to make it interesting: a difficult exergame wold never be completed and an easy one would be boring.

Fig. 1
figure 1

: The IGER system and its inner workings: the patient interacts with the games, while the control unit makes sure that the rehabilitation is correctly performed.

Full size image

This situation calls for a further advancement, as the exergames that can be currently found on the market or in research laboratories are often thought for a single patient condition and thus aim to treat a single dysfunction, using a fixed platform and input device. In addition, the complete set of features that are required for a correct rehabilitation cannot be found in all exergames, making the use of multiple, different games for rehabilitation harder for the therapist.

We propose here a game engine, IGER (Intelligent Game Engine for Rehabilitation – see Fig. 1), specifically tailored to rehabilitation that addresses these issues. A few games developed to address posture rehabilitation will be described along with their shared capabilities. IGER leverages computational intelligence to provide all the assistance needed during exergaming. In section 2, we detail the features of the game engine. In section 3, we introduce some of the games we have developed using IGER and our results. In section 4, we draw our conclusions.

2 IGER: A Game Engine for Rehabilitation

A game engine allows developers to focus on a higher level on the design of games, their gameplay and graphics style, while the engine takes care of the game’s inner workings (rendering, input handling, physics, collision detection and so forth), thus making the creation of games much easier. In the case of games for rehabilitation, the game engine can be even more useful as it can also provide all the features required for a correct rehabilitation to all the games created with it: the engine can provide shared components that limit maladaptation, adapt the game difficulty to the patient and allow the therapists to configure and review each rehabilitation session.

The features of a game engine for rehabilitation can be grouped in three macro- areas: efficacy, accessibility and motivation. For each area, we detail what the engine should provide and our implementation inside IGER.

2.1 Efficacy

Effectiveness must be the main focus of any exergame used to guide rehabilitation, as a correct therapy is more important than the entertainment side, although the former should not overshadow the latter (see section 2.3). To make sure that each exergame fulfils this important requirement, the game engine should provide components that aid the developers and the therapists in this task.

Configurable and Adaptable Exergames. To design a rehabilitation session, a set of exercises is defined by the therapist and tailored to the patient and her residual functional ability. A session is constituted of a mix of exercises often involving both coordination and strength [3]. The exercises usually consist of repeated movements and actions that aim at stimulating the recovery of the impaired function.

Exergames are defined to guide the patient through these exercises: a variety of games is therefore required to implement a whole rehabilitation session through exergames. Moreover, different patients have varying degrees of impairment and this requires that each exergame is tailored specifically to their residual abilities: for this reason, range of motion and intensity of the exergame should be defined by the therapist. This calls for exergames that need to be fully parameterized so that they can be set-up by the therapists for each specific patient, also defining a proper difficulty level by setting the value of a few adequate parameters, such as the movement range or speed of targets [6]. This operation can be carried out by the therapist in the hospital and it can be facilitated by a graphical display on how the game would be played with a set of parameters. Such parameters can also be modified automatically during play to achieve an autonomous day-to-day adaptation of the patient’s status as described in section 2.3.

Monitoring of Patient’s Movement. While exercising, the patient must be monitored constantly. This is another important requisite for a correct rehabilitation, because the execution of erroneous movements may trigger maladaptation. In a classic rehabilitation scenario, the therapist usually accompanies the patient during the whole training session and corrects her movements where needed. When the therapist is not present, the exergames should provide a suitable replacement for the monitoring of the patient’s movements.

IGER features a fully configurable set of monitors that can be attached to any game. They are implemented as a set of fuzzy rules that check in real-time the data gathered from input devices and compare them to the constraints defined by the therapists. These constraints can be related, for example, to the straightness of the back, the bending of the legs, or the distribution of the weight between the right and left side of the body.

Whenever a monitor detects wrong actions, it issues an alarm level that increases in severity the farther the action is from the constraints. An immediate qualitative feed-back is given to the patient by a change in colour of the avatar’s body part that is violating the constraint. For instance, if the patient is tilting his back during an exergame, the avatar’s back would change the colour from green to yellow, orange and then to red in the most severe cases. When the maximum limit of the constraint is violated, the game is paused and the patient is given advice about the correct execution of the exercise through her virtual therapist (section 2.3). We refer to [7] for further details on the fuzzy engine.

Assessment of the Exercise Goal. The game engine must be able to record the movement data of the patient and the data that describes the interaction with the games (for example, the time of interaction with game objects) while the patient plays her exergames. The therapist can then use these data to review the session and assess the progression. From these data a few parameters can be extracted to portray the rehabilitation outcome to the therapist who can evaluate the rehabilitation effectiveness, advice the patient and possibly tune the therapy. Most commercial games discard any input data after using it for controlling the game, therefore they lack this feature.

A clear distinction between assessment and monitoring is required: while assessment is related to the exercise goal, monitoring is related to the correctness of the exercise. For instance, during weight shift exercises, required in posture rehabilitation, while the goal of the exercise is to learn to shift the center of pressure of the body (COP) laterally with little sagittal oscillations,, correctness requires that the body is kept straight during the motion.

Assessment data constitute the documentation of the rehabilitation activity and are required by clinicians for evaluation. Moreover, they may be used to replay the whole exercise or extract specific information; extracted parameters can be, for instance, the trajectory or the variance of the COP, the covariance between trunk and legs and so forth. The same parameters can be shown to the patient at the end of each session, along with the values obtained in the previous sessions, as a feedback on his effort.

2.2 Accessibility

The majority of commercial games are unsuitable for rehabilitation also because of their cognitive complexity and of the speed and physical skills required to play them [8]. There have been attempts to make developers aware of accessibilityFootnote 1, but the application of guidelines is still rare. While entertainment games are designed for non-impaired people, as they must appeal to a wide market, for which accessibility may not be a high-priority issue, rehabilitation games must instead be tailored to a specific niche of patients for which accessibility is a very important matter.

Natural User Interfaces. Natural User Interfaces (NUIs) represent a good asset to provide accessibility to most users, who can be of any age and come from any background and thus not necessarily attuned to technology and videogames. NUIs allow the user to interact with a software application using natural means, such as body gestures or speech, making the actual human-computer interface transparent and in turn making the interaction more comfortable for the user.

Using suitable input devices such as cameras or microphones, innovative control schemes such as gesture control or speech control have already been introduced [9]. In IGER they have been implemented for navigating the rehabilitation sessions and playing the games. We use the Microsoft Kinect sensor in IGER for gesture or speech navigation through the menus but any other suitable device can be used to control an on-screen cursor for the same purpose.

Input Abstraction. To obtain the most effective exercising capabilities, a wide range of input devices should be supported by the game engine. Patients possess very diverse impairments and no input device exists that can be effectively used to track all important movements for each condition. A pressure board may be indispensable for balance and posture rehabilitation, but it would not be of any use to a patient who is regaining her cognitive functions or when the patient is required to make steps. On the other hand, that same patient may benefit from the use of haptic devices that could give additional feedback through force and vibration.

To address this need, we have created a middle layer between IGER and the input devices, so that any device suitable for a specific pathology can be used. Such layer, called IDRA (Input Devices for Rehab Abstraction layer), matches the animation data required by the game with the movements required by the exercise and the features supported by the physical device. Moreover, it avoi ds conflicts between multiple devices and allows users to play all games regardless of the chosen device. IDRA also makes the introduction of NUIs easier, since the most natural device for the specific patient condition can be used.

At present, IGER has been used with several input devices: the Microsoft Kinect sensor, the Nintendo Wii Balance Board (Fig. 2), the Tyromotion Tymo plate, the Moticon OpenGo Insoles, the Sony PS3 Camera, the PM10 robotic arm and the Novint Falcon haptic device.

Fig. 2
figure 2

: Playing the Fruit Catcher game (see section 3) using a Nintendo Wii Balance Board

Full size image

Instructions and Tutorials. There are many things that the patient needs to keep in mind while playing rehabilitation exergames: the therapeutic goal of the each exercise, the gameplay objective, the setup of the play space and of the input devices, and the monitoring constraints. Especially in the case of at-home personal rehabilitation, such instructions should be clearly provided to the patient.

We provide the patient with different means to get this information and the preferred mean can be configured by the therapist. Each game is provided with written, spoken and video instructions, localized in different languages (we tested English, Italian and German versions). We also allow the patient to play a tutorialversion of each game, during which she can experiment with the game mechanics prior to playing it and is thus taught how to play. This will be used by the clinician inside the hospital to train the patient but it could also be used autonomously by the patient at home.

2.3 Motivation

Games have a great motivational impact on the rehabilitation experience. At their base, games require that a lot of care is put into crafting meaningful play into even the most basic mechanics [10-11]. In addition, suitable feedback and plenty of immediate rewards can contribute to a fun experience and to allow the patient to enter a state of flow, in which the focus on the game distracts from the actual impairment [12].

Rehabilitation exercises often require multiple, repetitive, mechanical movements to be performed for a correct therapy, hence why the motivational aspects brought forward by a game can be so beneficial. Rehabilitation games must thus be designed with great care as the game mechanics are constrained by the underlying rehabilitation exercise. It can be very hard to come up with good game ideas and implementation for some exercises as the prolonged period of time in which rehabilitation extends may make a single game, although entertaining per-se, boring on the long run.

Other motivational mechanisms have been added inside IGER through very different means that can cooperate to achieve a shared effect, some of which are introduced in the next sections.

Adaptation Mechanisms. Commercial games create challenges for the player to complete, and this is an integral part of the fun experience of playing. The challenge needs to be neither too easy (resulting in a tedious experience) nor too hard (resulting in anxiety): game developers should aim for the sweet spot between these two extremes as thoroughly explained using flow theory [13]. The balance between the player’s skill and the game difficulty is created, in games for entertainment, during the development phase by the game designer through heavy play-testing, with the specific user target in mind of an average-skilled person. This cannot be achieved in exergames for rehabilitation. Patients may have a wide variety of residual skills; moreover, they are expected to improve these skills during the rehabilitation sessions and thus greatly modify their abilities.

Rehabilitation games need therefore adaptation mechanisms that automatically balance the game to the patient’s skill while she is playing. Balance can be achieved by tuning a few game parameters that directly determine the difficulty of the exergame. By choosing suitable parameters for adaptation, the exercise can require the same actions but the difficulty may be largely increased, for example by requiring faster movements. Inside IGER, we implemented a Bayesian approach in which the parameters are modified in real-time by analysing the actual success rate and the a-priori parameters deemed appropriate by the therapist. Such adaptation acts on a set of parameters that are defined by the therapist in the hospital at configuration time and therefore allows adapting the game to the specific status of the each patient. More details are reported in [14].

The adaptation mechanism is tightly connected with monitoring: a repeated intervention of the monitor has an impact also on the level of difficulty of the game. When errors are detected, the increase in difficulty is disabled and when repeated errors are detected over time (up to 30 % of the time), difficulty is progressively made easier to let the patient return to a correct execution of the exercises.

Long term Motivation. While short-term motivation deals with the immediate feedback given to the patient that makes her enjoying the single gaming instance, long-term motivation deals with capturing the attention and the focus of the patient in the long run, since the time extension of rehabilitation can be several months or even years. There are many methods for achieving long-term motivation that should be integrated into the game engine to help link the different games and even the different sessions of the same game together.

A basic mechanism to provide long-term motivation, widely explored by the entertainment industry, is a balanced scoring system. This mechanism can be used by the patient to evaluate her performance during a single session compared to previous sessions, thus motivating her to play again to improve in an attempt to reach a higher score and therefore exercise more intensively. A balanced score should depend both on the gameplay successes and on the correctness of the execution of the movements required by the exercise. In our implementation, the correctness acts as a multiplier of the score earned during each game-related action.

More advanced mechanisms are related to the social competition when games are played by multiple players, to the introduction of a larger theme common to all games (e.g. the farm theme in our case) and to the randomization of the game assets to increase variability. Such mechanisms are currently subject to further investigation.

We have already introduced a balanced scoring system in IGER and therefore in all implemented games and we are currently adding the other motivational mechanisms.

The Virtual Therapist. In the case of at-home rehabilitation, the central figure for rehabilitation, that of the therapist, is missing. In IGER, the therapist is partially replaced by a Virtual Therapist (VT) that provides feed-back similar to the one given in routine sessions by a real therapist.

The VT is an avatar that accompanies the patient during her rehabilitation sessions throughout the life of the application and advices her. This character can be useful for multiple purposes: it can explain how to navigate the interfaces and how to play the exergames, it can introduce options, congratulate on achievements and motivate during challenges. It also explains to the patient how to correctly perform the exercises when wrong movements are detected by the monitors. Even if the character is virtual, having a face to refer to can be beneficial for the patient. Many entertainment games use similar figures as guides for the player, such as the Wii Fit games and their cartoon-like animated balance board.

3 Results

We show here how we have designed and implemented a set of exergames using IGER, we also show some preliminary results following their use. The exergames we designed regard the rehabilitation of posture and balance and the rehabilitation of neglect. The games can be played with any of our input devices and we detail a few differences when a specific device is chosen. All these games share the same theme: life in the farm.

The Balloon Popper game (fig. 3A) represents a refinement of DuckNeglect [15], a game developed for neglect rehabilitation. The game requires the user to reach a set of balloons and pop them. In the native version, the game has been designed to be played with a 2D camera: the silhouette of the patient is extracted from the images and pasted inside the virtual scenario. Very good results are obtained when the background can be controlled and is stationary. When using a Kinect sensor, thanks to the superimposed skeleton, the position of the hand can be located and the interaction with the target and distractors determined also when the hand passes in front of the body and with any constraint on the background. This game is meant to allow patients to explore the neglected field following balloons that move from right to left. However, the same game could be played also with a haptic device: we have adopted the Novint Falcon as an input device, allowing the patient to reach the balloons on the screen with its cursor. In this case, there is no silhouette of the patient, but a visual cursor moving on the screen. We could also play the same game with the Nintendo Wii Balance Board: the lateral movement is in this case mapped on lateral weight shift and the up/down movements on the front-posterior weight shift. We also remark that the game has been tested for use with the PM10 robotic arm for upper-arm rehabilitation.

Fig. 3
figure 3

: Some of the games built with the IGER system. From the top left: Bal’loon Popper (A), Fruit Catcher (B), Scare Crow (C), Hay Collect (D), Mix Soup (E), Animal Hurdler (F).

Full size image

A second game, Fruit Catcher (fig. 3B), has been developed to train lateral weight shift. The game asks the patient to catch fruits as they fall from a tree, using a basket placed over the head of the player’s avatar. The game is played in third person, with the user viewing the scene from behind the avatar’s back. The player is required to catch the fruits as they fall laterally in a range specified by the therapist. The Fruit Catcher game can be run with two different exercises. For the first exercise (i) the patient is required to shift his body to the left and to the right, while keeping the feet still on the ground. For the second exercise (ii) the patient must instead step laterally inside the play area. The game can be played using either a Wii Balance Board, a Tymo Board or the Kinect sensor for the first exercise (i), while for the second exercise (ii) only Kinect can be used as a tracking device since foot tracking is required for lateral movement.

The Scare Crow game (fig. 3C) is designed to train static equilibrium and asks the patient to stand still while birds fly over the shoulders of her avatar. The Hay Collect game (fig. 3D) is used to train reflexes and perception; it asks the patient to drive a tractor across a field while collecting hay bales. The Mix Soup game (fig. 3E) can be used for balance rehabilitation, upper-arm rehabilitation (for example using a robotic arm) and even for cognitive rehabilitation; it requires the user to touch a set of bubbles that appear on the surface of the liquid in a cauldron. A cognitive load can be added by generating bubbles of different colours and requiring that only bubbles of a given colour have to be touched. The Animal Hurdler game (fig. 3F) is designed for in-place stepping exercises and asks the patient to rise one foot when small creatures try to pass below it.

The IGER system and the games were tested on seven elder people (75 ± 7 years old) to analyse usability and accessibility. All subjects reported a very good reaction to the games. They did not get tired while playing them and did not report any fatigue or interaction difficulty. The monitoring through colour coding was rated particularly useful as it allowed them to focus immediately on the wrong features of the implemented movements and to correct them. The same was true for adaptation as each subject could see the game slow down when the pace was too high or to increase progressively the speed and therefore become progressively more and more interesting to the user. The interaction was rated extremely natural and easy to perform. No particular preference between speech and gesture was found. Further validation is currently being performed at the Neurological clinic of Zurich hospital, at the Virgin de Rocio hospital in Seville and a pilot study is scheduled to start in January 2014.

4 Discussion and Conclusion

IGER starts from the observation that commercial games are indeed too hard for most rehabilitation patients. At its core, IGER is built upon the open-source Panda3D game engine, but it has been expanded with a set of modules tailored to rehabilitation: configuration, adaptation, monitoring, logging of the data and feedback through a virtual therapist. Without these additional components the effectiveness and reliability of any game-based exergame for rehabilitation at home can be severely questioned. Moreover, these additional components cannot be developed as stand-alone but should work in synergy: adaptation has a prominent role in motivation, and monitoring has an influence on the scoring system and modulates adaptation.

As far as adaptation is concerned, the Bayesian approach followed here does not make any hypothesis on the particular data domain as for instance in [16] in which a specific heuristic for arm motion has been proposed, but it is general and it can be applied to any parameter or combination of them. It also does not require any particular model that would require calibration by testing the games on a significant number of homogeneous patients as required by the RGS system [17].

Another characteristic of IGER is the abstraction layer that couples the input devices with the game. Such layer allows different devices be used with the same game thus making the engine extremely flexible. To highlight this, our current selection of games spans a range of rehabilitation domains: from full-body postural and balance physical rehabilitation to cognitive rehabilitation for neglect patients to arm rehabilitation. The same game can be used for different rehabilitation domains, changing the tracking device or modality. This can be a great asset for rehabilitation.

Components that take care of monitoring, of configuration, of adaptation to the patient’s skills, of accessibility and usability by patients with diverse skill ranges require a considerable development time and the advantage of reuse is thus great. Using a specifically developed game engine, the designers can use the methods made available to build their own games without having to write them from scratch.

Moreover, the rehabilitation game engine can also be used to enforce a shared design intent and thus provide design guidelines for both games and exercises, thus making the process of creating efficacious exergames easier and, in the meantime, helping in guaranteeing their validity. The game engine can also, as in our case, suggest a shared theme in both visuals and narrative to promote cohesion between different exergames, provide a framework to insert motivational elements and ultimately increase the user’s immersion in the game world.

IGER is not meant as a stand-alone system. It has been embedded into a Patient Station (PS) developed for the REWIRE projectFootnote 2, financed by the European Union, to be deployed at patient’s home to support their rehabilitation therapy. The PS integrates the IGER with bidirectional communication through Internet with the reference hospital. Such communication has a dual role: transfer the configuration of each rehabilitation session from the hospital to the patient’s home and open a video-communication with the reference clinicians in case of need.

We explicitly remark that the PS has been designed to be used autonomously by the patient at her own home, without needing the presence of a therapist. For this reason, all exercises have been carefully designed in collaboration with the therapists to be performed without their presence. We also remark that the IGER engine is used to support the therapists in their absence, but their role is still of utmost importance in defining and configuring the exercises, following remotely the rehabilitation progression, advising and directing patients and reviewing and validating the results. Morever, we believe that rehabilitation sessions carried out in specialized centers with the help of the therapist intermingled with rehabilitation at home would make the approach most effective.

Given all these characteristics, the IGER system can be a good candidate to be successfully used as a game engine upon which rehabilitation games can be built to make rehabilitation at home possible and valid from the clinical point of view. This would represent a large step forward in rehabilitation, that could be taken out, at least partially, from the clinics and enable patients, discharged from the hospital, to continue their treatment intensively at home, where they feel most comfortable.