Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Approximately 15 % of the global population suffers from some form of musculoskeletal disorders that often times hinders the execution of daily personal and work-related tasks [49]. Aside from the loss of productivity that this leads to, it can also lead to a reduction in the quality of life. Treatment for musculoskeletal disorders often includes preventive, corrective, and maintenance exercises, each performed during a specific stage of the disorder treatment, often times under the care of a medical expert. Preventive exercises aim to reduce risk, improve and adjust a movement or posture, while maintenance-based exercises focus on physical activity required to sustain functionality in order to carry out daily activities [41]. When prevention has failed and an injury has occurred, patients are typically required to engage in some form of physical therapy/exercise (cryotherapy, electrotherapy, and kinesiotherapy). Often times, this is in conjunction with pain killers, and anti-inflammatory medication, although in some cases, surgical intervention may be required [32]. When pain persists and medication loses its effect, distraction and endorphin stimuli resulting from engaging activities may reduce pain perception; the endomorphin stimuli can be obtained from the production of dopamine that a user/player experiences while winning and overcoming challenges within a virtual environment that incorporate gaming elements [24].

Physiotherapy-based exercise requires the execution of repetitive movements (often times within a time limit), whose ranges of movement are set by a medical expert. The practice of physiotherapy exercise is primarily constrained to medical centres/clinics with specialized equipment and capable personnel (e.g., physiotherapists). Attendance at such facilities often requires transportation (assisted or not assisted, depending on the musculoskeletal disorder or disability). Sometimes the exercises are prescribed to be executed at home without any supervision or feedback regarding how the exercises (movements) are actually being performed. An additional problem relates to assessing a patient’s performance within an exercise program. Traditionally, assessment has been mostly qualitative including observation by the medical expert/physiotherapist (when present), and the results of questionnaires that the patient completes. Given these considerations, patients are often not motivated (or lose motivation quickly), to engage in physical activities even though these physical activities can help prevent, correct, or maintain their quality of life. However, the degree that a patient adheres to physiotherapy exercise treatment requirements (home- or clinic-based) can be a large factor in the success of the physiotherapy program [5]. Other factors that may hinder a patient from maintaining a rehabilitation-based exercise program includes lack of time, confusing guides, lack of knowledge, lack of interest, fear of incorrectly performing the exercises, and pain associated with performing the exercises [16].

Developing an understanding of human locomotion has been a longstanding topic of interest, driving research towards creating solutions to quantify movement for different purposes (e.g., movement pattern identification, joint force and energy expenditures, and treatment of war veterans, among many others) [10]. In recent years, technological advances have introduced various consumer level motion sensing technologies to many homes. For example, the Microsoft Kinect motion sensing vision-based sensor is capable of detecting and tracking user movements and allowing interactions with the games using natural gestures thus eliminating the game controller. Motion capture technology provides a suitable solution to the challenges associated with observation-based assessment (whether the exercises are performed at home or in the clinic). Capturing exercise-based motions can provide quantitative information regarding a patient’s movements while exercising. This information can then be provided to the patient thus providing them with (dynamic) feedback regarding their movements that can ultimately be used to correct any problematic movements.

Tracking technologies can be grouped into three categories [12]: (i) active-target systems which incorporate powered signal emitters (e.g., magnetic [25], mechanical [14], optical, radio [21], and acoustic signals [46]), and sensors placed in a calibrated environment, (ii) passive-target systems employ ambient or naturally occurring signals (e.g., vision-based systems [48]), and (iii) inertial systems which rely on sensing linear acceleration and angular motion (e.g., accelerometers, and gyroscopes). Each of these three approaches has its own share of issues and challenges, particularly when considering motion capture for physiotherapy-based exercise. More specifically, active-target systems are prone to signal degradation and interference [63]. For example, the accuracy of a magnetic tracker degrades as the distance between the emitter and receiver increases while mechanical trackers typically require a direct physical connection with the object (e.g., patient’s limb) being tracked. Aside from the physical attachment to the tracking device, a major problem with mechanical trackers is the limited range of motion [63]. Passive-target systems generally require the user to wear markers (often times these are incorporated into a special suit worn by the user), and this can potentially lead to possible alterations of the movements that are relevant in the diagnosis and assessment of musculoskeletal disorders [1]. Furthermore, many optical-based tracking systems are prone to changes in lighting conditions. Finally, inertial sensors will typically measure acceleration or motion rates, and these signals must then be integrated to produce position or orientation which requires constant calibration and are also susceptible of error accumulation during measurements [18]. Despite the associated issues, the most frequently used motion capture systems are passive and employ optical sensors that require the user to wear markers on the body segments that are to be detected and tracked by infrared cameras [7, 15]. That being said, the majority of such systems are not consumer-ready but are rather targeted to research and development (mainly in the entertainment industry). As previously described, there are a variety of motion capture systems available. Most common systems employ passive markers attached to the body of the person (or object) being tracked. The camera is capable of detecting the markers and through the use of image analysis, a skeleton and its movements depicting the person (or object) can be reconstructed. Given the setup required for such marker-based solutions, they are often used in movies, gait research labs, sports labs or medical facilities with physiotherapy programs.

Virtual reality- (VR-) based rehabilitation and training can improve the user experience by providing immersive and interactive scenarios that are compelling due to user interfaces (UIs), and narrative. In addition, recent advances in 3D stereoscopic vision with affordable consumer-level devices, can help to increase a patient’s motivation when participating in any exercise program [11]. Early applications of VR-focused on stroke rehabilitation by assisting patients in recovering mobility and functionality in order to perform tasks and ultimately improve their quality of life [26]. These early applications were generally cost prohibitive, and limited technologically, and although these systems did offer clear goals and tasks, their success relied on the user’s commitment and motivation to follow through the process. With the current advances in technology, particularly with respect to computing hardware and graphical-based rendering, VR has evolved tremendously and has become accessible to the general public through affordable consumer-level (gaming-based) devices such as the Microsoft Kinect, and the Oculus Rift [9], that are bringing VR into the average living room. Such devices are placing a greater emphasis on interaction. For example, Johnson and Winters modified a commercial joystick to extend the arm movement to accommodate a therapeutic range of motion and thus provide and accessible tool for rehabilitation performance evaluation [30]. This increased emphasis on interaction can be traced back to the introduction of the Nintendo Wii gaming console that included the Wii remote (also known as the “Wiimote”), as its primary controller. Through the use of accelerometers and optical sensing, the Wiimote provides the user/player the opportunity to interact with and manipulate items on the screen using gesture recognition and pointing. The Nintendo Wii (and the Wiimote in particular), demonstrated the popularity of motion-based casual gaming and was responsible for bringing together gamers from all ages (from the very young to the very old) into scenarios where the “interactions where everything” [61]. VR is being increasingly used in a variety of applications as it provides users the opportunity to interact with scenarios otherwise impossible in the real world due to safety, ethical, or cost concerns. An example can be seen in tourism-based applications where a user can visit a destination not easily available to them [23], or post-traumatic stress therapy where patients are exposed to simulated, safe, and controlled environments to help them overcome their health state [34].

Despite the benefits of VR-based exercise approaches, the novelty of VR can wear off as the user becomes acquainted with the equipment and the virtual environment [24]. Exergaming can help overcome this challenge by increasing and maintaining motivation to exercise. Exergaming couples video games and exercise whereby playing a video game becomes a form of physical activity allowing healthy behaviours and learning to be promoted [56]. Exergames take advantage of the engagement, interaction, and fun inherent in video games to promote physical activity and engagement within a training or rehabilitation program [35, 47], which can be mundane and repetitive [31]. When compared to traditional exercise, exergaming has proven effective in terms of increasing user motivation regarding the practice of physical activity whether for fitness, rehabilitation, training, or entertainment. Exergaming has been linked to greater frequency and intensity of physical activity and enhanced health outcomes [2], and more recently with commercial exergames, where heart rate and oxygen consumption equals real physical exercising [42]. According to Skiba, there are three emerging trends in exergaming [56]: (i) the increased use of games for therapy and rehabilitation, (ii) the use of games in gyms and other settings to promote physical activity, and (iii) the greater involvement in gaming of corporations and health care providers. With respect to physical rehabilitation, the integration of video games can provide greater guidance for teaching patients how to properly move their limbs as UIs based on motion interactions can track and provide feedback [59]. As Janarthanan describes [28], video games can help a patient learn a pattern of limb movement and reapply it. In other words, “what we learn with these types of games is how to get well soon”. Exergaming has also been beneficial to those suffering from traumatic brain injury and cerebral palsy [4], and exergames have been employed to assist patients with their stroke rehabilitation, and to assist young adults with physical and intellectual disabilities. Recent evidence also suggests that exergaming may improve cognitive function and thus augment traditional rehabilitation of motor symptoms in people with Parkinson’s disease [4].

In a review of the literature related to the use of video games in health care, Kato highlights the positive effects regarding the use of exergames, and interactive media in general, for example, patients undergoing physical therapy were able to better cooperate with their required physical therapy program [31]. Another review of the literature that was conducted by Primack et al., who examined whether video games are useful in improving health outcomes [44]. They found that video games improved 69 % of psychological therapy outcomes, 59 % of physical therapy outcomes, 50 % of physical activity outcomes, 46 % of clinician skills outcomes, 42 % of health education outcomes, 42 % of pain distraction outcomes, and 37 % of disease self-management outcomes leading to the conclusion that although greater rigour is required, there is potential promise for video games to improve health outcomes, particularly in the areas of psychological therapy and physical therapy [44].

In this work, we describe two independent upper-limb exergames for the purpose of shoulder rehabilitation. We have chosen to focus on shoulder rehabilitation given that shoulder-related injuries and problems are widespread. More specifically, it is estimated that approximately 7.5 million people have some form of shoulder problems with 4.1 million of them related to the rotator cuff [39]. There are also approximately 1.4 million shoulder-based arthroscopies performed each year [40], and according to the World Health Organization (WHO), 40 % of the working population suffers from some form of shoulders and upper limb ailment caused by repetitive strain on ligaments, muscles, bone or tendons surrounding the joint [27]. Shoulder injuries can be acute or painful (from sudden movements), or chronic and everlasting (from overuse). Common afflictions can happen throughout the lifespan of a person as a result of repetitive tasks and bad postures or static shoulder loads resulting in tendinitis. However, shoulder pain and tendinitis can also be the product of aging and sports practices [45]. Therefore, motivation in therapy is a topic of interest among physicians as common exercises focus on stretching and strengthening the shoulder’s muscles to provide flexibility and support. These exercises require commitment as observation and follow-up is only performed during the early stages of recovery [50]. To facilitate proper exercising, both of the exergames presented here stimulate exercising through kayak-based motions and map the resulting motions using a vision-based sensor (Microsoft Kinect), into a kayak-based game world providing the user (player) an interactive and immersive environment where they are in control of a kayak moving through a number of courses (rivers, etc.).

The chapter begins with an overview of shoulder-based exergame development starting with a characterization of the shoulder anatomy, movement, and exercising followed by a discussion of the issues involved in the shoulder-based exergame design and development. As with any exergame and serious game in general, the instructional component of exergames must not be overlooked and clear objectives must be devised early in the development process. Each of the exergames will then be described. This will include a discussion regarding our experiences in developing the games (e.g., what worked, what did not work, what we could have done differently, and lessons learned). We also discuss the results of several experiments that were conducted to test the usability of the exergames. There are a number of issues and difficulties in conducting user-based experiments to gauge the effectiveness of exergames and as a result, quantifiable data providing evidence that the game does indeed meet its intended objectives is sparse. The information presented in this chapter will be useful to those interested in the field of exergaming and to those interested in developing and using their own exergame.

2 Exergame Development

To develop the shoulder-based exergames, the upper limb was characterized to determine the body parts involved, their ranges of motion, and the exercises suitable to be incorporated within the exergames. This characterization provides information that serves as an initial step in determining the hardware and software development requirements. Greater details regarding the characterization of the upper limb are provided in the following sections.

2.1 Anatomy Characterization

The upper limb is comprised of bones, muscles and ligaments; it is segmented into two main parts: (i) the arm (humerus), and (ii) the forearm (radius and ulna) joint at the elbow. Human motion is performed in a three-dimensional space and movements are described in terms of the body planes [55]. As presented in Fig. 6.1, three planes segment the human body: (i) the sagittal plane, (ii) the frontal plane, and (iii) the horizontal plane. Within these planes there are several combinations of motion depending on the body part and their degrees of freedom (DOF), resulting in flexion/extension and adduction/abduction movements.

Fig. 6.1
figure 1

Human body planes of movement

Full size image

2.2 The Shoulder

The shoulder is comprised of seven joints, each of which is important to achieve fully functional movements [52]: (i) glenohumeral, (ii) superhumeral, (iii) acromioclavicular, (iv) claviculae sternal, (v) scapulocostal, (vi) sternocostal and (vii) costovertebral. Among the joints, the glenohumeral wears faster than the others as it allows rotations on the three coordinate axes, making it susceptible to wear and tear that may result in pain and limited motion. Muscles around the rotator cuff allow flexion/extension, adduction/abduction, circumduction, medial and lateral rotation movements and help keep the humerus head at the right place; this is the reason that while lifting heavy objects, abrupt movements can deteriorate the joint [17].

Shoulder movements involve flexion rotations with average angles varying from 0\(^\circ \) to 180\(^\circ \) moving the arm from the body along the sagittal plane as presented in Fig. 6.2. It also involves extension rotations ranging from 0\(^\circ \) to 60\(^\circ \) also shown in Fig. 6.2. It is worth noting that some people are capable of performing hyperflexion and hyperextension rotations that the average person cannot. Within the frontal plane, the shoulder allows abduction rotations from 0\(^\circ \) to 80\(^\circ \) and adduction from 0\(^\circ \) to 45\(^\circ \), similarly to flexion/extension some people may exceed these ranges. Since the shoulder can rotate about all three axes, there are additional combinations that can be achieved, such as the lateral and medial rotations commonly used when exercising the rotator cuff.

Fig. 6.2
figure 2

Shoulder and elbow movements

Full size image

2.3 Common Afflictions and Exercises

Musculoskeletal disorders can result from overuse, bad posture, low muscular demanding activities, static work, recurrent vibrations, manipulation of heavy loads and repetitive actions all of which can result in pain, discomfort, and mobility limitations [27]. Exercises to prevent, correct, or maintain the shoulder are focused on moving the arm across the planes with or without weights depending on the condition of the patient. These movements can be related to several activities performed in multiple scenarios that can be within a VR environment. The goal of these movements (exercises) is to strengthen the muscles and tendons and thus help avoid strains and ruptures, and minimize the risk of muscle and tendon irritation, degeneration of bone and cartilages. Figure 6.3 illustrates two shoulder-based exercises recommended for the shoulder rehabilitation [32].

Fig. 6.3
figure 3

Example shoulder exercises for medial and lateral rotations

Full size image
Fig. 6.4
figure 4

Example Exergame system architecture

Full size image

2.4 Design Overview

A general exergame system architecture is presented in Fig. 6.4. As shown, the game receives the camera-based motion capture information (here, using a Microsoft Kinect visual sensor), of the player performing the movements. This input information is then processed, typically mapped to game play (e.g., to corresponding motions of a virtual canoe), and compared to one or more reference motions. Based on this comparison, the player interacts with the virtual world through the game rules and goals, while being assessed on through motion capture through feedback (typically graphical and sound-based) to improve exercise execution. This cycle is continually repeated until the game ends.

2.5 Motion Capture

A cost effective, yet limited tool for measuring limb movements is the goniometer, which allows angle rotation measurements between two joints by being positioned alongside the body part to measure and its reference. However, currently, the most common form of assessment is through observation which leads to subjective analysis and becomes a challenge when the exercises are prescribed to be done at home without any supervision. These unsupervised exercises rely on the capacity of the patient to execute all movements as explained or as indicated by available media (e.g., pamphlets), without any feedback. Affordable solutions include the use of webcams and built-in device cameras. For example, BitGym is a tablet oriented application that uses the tablet’s camera to capture body movements and allow for low latency tracking [8]. Prange et al. employed a webcam with a television to track horizontal movements in patients with hemiparesis [43]. For both of these examples, a quantitative assessment was performed outside the exergame due to the system’s low accuracy resulting from the use of a single webcam. However, although inexpensive, these solutions present major challenges due to the current use of a single integrated webcam which requires tilting the computer screen (reducing visibility of the contents), and the lack of proper 3D motion tracked data associated with upper-limb movements (due to the abstraction of depth information from only one image). To overcome the difficulties of motion tracking with a single camera, stereoscopic cameras provide a potential solution as they can obtain depth information through a combination of red/green/blue and infrared cameras. Several depth cameras are available at the consumer level including the Microsoft Kinect [37], AsusXtion Prolive [3], SoftKinectic [58], and the ZED 2K Stereo Camera [33], among others, with varying features regarding camera resolution, motion capture area, size, prices and ultimately availability on the market.

Fig. 6.5
figure 5

Game system

Full size image

Both of our exergames employ a Microsoft Kinect sensor due to its low cost, decent specifications, and widespread use in addition to the freely available resources/information regarding it. Within three years of its launch in 2010, 24 million Microsoft Kinect (V1) units have been sold worldwide [57]. The Microsoft Kinect features have been widely studied and they have been incorporated into many applications across a variety of areas [64]. Taking advantage of the Microsoft Kinect’s motion-capture capabilities, the Jintronix system for physiotherapy was patented and approved by the Federal Drug and Administration office in the USA [29]. This commercial solution provides configurable scenarios for physiotherapy and telerehabilitation with custom designed exercises and assessment based on metrics capture during the interaction. For the development of the motion capture subsystem, the Microsoft Kinect SDK [36] and Unity3D [60] were used to program user interactions and responses from the system within the game mechanics. Ranges of motion were considered and included within the code to provide feedback and quantifiable data from the performed movements.

2.6 Game Design

The design and development of an exergame follow a process similar to the design of an entertainment game; define the goals, rules, and feedback along with its formal, dramatic and dynamic elements [19]. However, designers and developers of exergames may not necessarily be afforded as much freedom and flexibility with respect to adjusting various aspects of the design throughout the development process if these adjustments interfere with the rehabilitation requirements and goals. For example, if an exergame requires the user to move their arm following an up and down motion, the designers and developers may not necessarily be able to change these motions to side-to-side. Rather than providing simple interactive environments, our approach to the development of the two exergames involved brainstorming for scenarios where a person requires shoulder movements to accomplish a goal. From this process, two kayaking shoulder exergames were developed by two different teams independently.

With the participation of medical experts, the design process resulted in identifying key elements to encourage a series of properly executed shoulder movements. The information allows diagraming the game system in terms of the rules, goals and feedback as presented in Fig. 6.5. A key element while developing any type of game is the fun component, which in our case was approached through challenges promoting self-competition and improvement once users actually play the game.

2.7 The Two Exergames

The first exergame focuses on lateral and medial rotations, which are important movements for strength and functionality while the second exergame focuses on more complex movements involving the shoulder’s three DOF to kayak with a paddle-like tool that brings more reality to the game interaction. A description of each exergame along with a discussion of preliminary tests that were conducted to gauge user perceptions regarding the exergames is provided in the following subsections.

3 Exergame 1: Kayaking Through Lateral and Medial Rotations

The lateral and medial rotation exergame was developed as a tool for encouraging shoulder care. The goal of the game is to paddle from an origin (starting) position in a virtual lake to the finish line. Correct paddling results in swift navigation that allows collecting power bars to restore energy depletion caused by movement. To reach the finish line, appropriate paddling and power bars are required as otherwise the player stays adrift in the lake [38].

The exergame system receives several inputs from the user as follows: (i) menu navigation and selections are accomplished through hand tracking across the screen. To access the game, settings, and quit buttons, the user needs to hold the position over the menu for five seconds. (ii) once in the game, the user moves the virtual kayak through the upper limb’s lateral and medial rotations mapped as suitable inputs representing kayaking movements to going forward, backward and steering left or right; and (iii) finally, after completing the exergame, the user can access the motion data and score stored in a plain text file separated by commas.

The engine receives the user interactions and executes the kayak animations from a third person point of view across the lake. The collision system provides feedback when collecting the energy capsules and also limits where the user can navigate within the lake. A collision may cause the kayak to sink and may prevent the user from capturing collectibles that are required to reach the goal. All interactions are programmed in response to xyz joint information obtained from the Microsoft Kinect’s SDK while performing the medial and lateral rotations to recognize the number of repetitions, forearm trajectories, and the time spent while playing the game. Various audio and visual cues were introduced to the game to provide immersion within the environment. More specifically, visual cues were added to account for water reflections on the surface of the lake, vegetation in the bottom, an animated kayaking character, energy bars to boost the kayaker stamina, rocks around the lake, and energy status. Audio cues included paddling sounds, movement across the water and collision sounds with the energy capsules.

The player’s paddling motion data (the upper-limb medial and lateral tracked rotation data), is captured and deemed to be valid if it was within 10 % of the actual (correct) motion. error from, This 10 % threshold was chosen in consultation with health care (rehabilitation) specialists after reviewing samples and measurements obtained with a virtual skeleton captured with the Microsoft Kinect of various users going through the required motions can account for light variations in user movements in addition to slight changes in lighting conditions. When a captured movement is classified as valid, they are mapped to the game’s avatar (e.g., the game executes the corresponding animation to move the kayaker). If three consecutive invalid movements are detected a pop-up window appears to remind the user to: (i) check the movements and posture, (ii) check room lighting, and (iii) check the feedback motion graphics placed at the bottom of the screen; if working properly, these should render a rotation curve of the forearm. To encourage the player to pay attention to shoulder care and how proper movement execution is achieved, checkpoints were programmed displaying random information about anatomy, shoulder care, and the purpose of the exercise, so once the game is completed, a pop-up window prompts the user to answer a question related to the presented information. Figure 6.6 presents a screen-shot of the exergame with the required movements to paddle and the motion capture data.

Fig. 6.6
figure 6

Visual game elements (avatar, energy bar, rock obstacle, arm feedback and energy status)

Full size image

3.1 Exergame Presentation

Once the exergame was developed, to gauge user reactions regarding the exergame and its use, a preliminary user study was conducted with 22 participants (ages between 19 and 35). All of the participants were required to perform shoulder exercises to diminish the risk of developing rotator cuff afflictions due to continuous heavy lifting objects, or due to a sedentary lifestyle. Participants played the game for a brief period of time (minimum 1 min, maximum 2 min for every 45 min for one full work day (8 h)) as indicated by occupational health care guidelines to prevent musculoskeletal disorders [22].

Before playing the game, participants completed a survey that gauged their shoulder care awareness. Results of the survey showed that 77 % were aware of the importance of shoulder care, while the rest were not; 23 % believed shoulder pain is a common problem, 36 % occasional, 32 % frequent and 10 % non-existent; when asked about how to prevent shoulder afflictions, all participants ranked the solutions acknowledging physiotherapy as the best, followed by occupational health care, surgery, sports and exergames. Finally, with respect to their frequency of exercise, one person exercised every day, nine frequently, nine rarely, and two never. Figure 6.7 presents a user interacting with the exergame where medial and lateral rotations result in kayak movement with the corresponding motion capture data.

Fig. 6.7
figure 7

Medial and lateral rotations for paddling with motion data

Full size image

The exergaming session was overseen by two health care specialists from an occupational health and a fitness facility. After using the exergame, participants completed a second survey to gauge their perceptions of the game. Seventy seven percent found the exergame easy to use while the rest found it difficult to use. Those that found it difficult to use mentioned that it was the result of their lack of familiarity with the Microsoft Kinect sensor, and that they were not gamers. When asked about any future goals that they identified after playing the exergame, their responses were: to exercise, learn about shoulder care, learn how to properly exercise, make the best score, and finally reach the goal. Once all of the participants completed their game play session, the two health care specialists supervising the activity provided feedback on the experience. They highlighted the importance of motion tracking to better assess patient progression and the use of game elements to motivate the user (patient) into engaging physical activity.

4 The Rapid Recovery Exergame

The Rapid Recovery exergame was developed to facilitate shoulder rehabilitation and physical fitness. It couples the strong levels of engagement and interactivity inherent in video games with the Spincore Inc. Helium 6 baton [51, 54]. Players take on the role of a kayaker who must paddle their canoe through a pre-defined course. The game is played from a third-person perspective so that the player can see their real-world actions affecting the player avatar. Prior to beginning, the player must choose one of the three modes of operation (see Fig. 6.8): (i) kayaking (sitting or standing), (ii) canoeing, and (iii) stand-up paddling. Finally, the player chooses the length of their workout session in minutes. The player is then placed at the beginning of the course, the timer begins, and then they must paddle their canoe through various checkpoints throughout the level until the timer expires. The player grips the Helium 6 baton (physically, in the real world), just as they would grip a kayak paddle, and then goes through the identical motions as they would if they were paddling a real kayak. The player’s Helium 6 baton movements are continuously captured using a Microsoft Kinect video sensor and mapped to movements in the game world. A sample screenshot (canoeing mode of operation), is provided in Fig. 6.9. Included in Fig. 6.9 is an inset which illustrates the player holding the Helium 6 baton. In addition to the kayaking motions, the game is also responsive to the speed of motions. In other words, faster motions of the Helium 6 baton will result in faster movement of the virtual kayak.

Fig. 6.8
figure 8

Rapid recovery supports three modes of operation: kayaking (sitting or standing), canoeing, and stand-up paddling. The user must choose one mode of operation at the start of the game

Full size image
Fig. 6.9
figure 9

Sample gameplay screenshot (canoeing mode of operation) along with the inset that illustrates the use holding the Spincore Fitness Helium 6 baton

Full size image

The player is provided with feedback throughout the duration of the game (see Fig. 6.9). This feedback provides them with information regarding the elapsed time on the course (game timer), current score, and the direction indicator that lets the player know the direction that they should be steering towards to reach the next checkpoint in the game. The player’s score is linked to the player’s physical motions with the Helium 6 baton. More specifically, the score is increased depending on whether the player’s motions match (within a predefined threshold value), the reference motions (performed by expert kayakers and canoers), and how many laps they have completed. The reference motions were collected from two experienced kayakers who completed the required motions for each of the three modes of operation (seated and standing kayaking, canoeing, and stand-up paddling) while being recorded via a Natural Point Optitrack motion capture system (with eight infrared cameras).

Rapid Recovery also includes a tutorial level which the player may complete at any time from the main menu. This helps ensure that the player has a basic understanding of the game requirements (Helium 6 baton motions, etc.). The tutorial level includes a video of a professional kayaker demonstrating the proper paddle rotation (using the Helium 6 baton). This video is followed by a test round where a series of paddle motions are presented to the player (individually) which the player must then replicate. The player is allowed to proceed to the next motion in the test round only if their motions match the reference motion to within a certain threshold. Throughout the test round, feedback is provided to the player letting them know how they are doing (whether their motions are correct or incorrect, and if they are incorrect, how they can improve on a per limb basis).

4.1 Alpha Testing of Rapid Recovery

A preliminary alpha test of Rapid Recovery was conducted to examine the initial functionality of the game and to gauge the game’s clarity of content, ease of use, and the user interface [51]. Participants completed the tutorial level and then played the game for a 15-min exploratory period. Playing the game involved using the game and exploring its interface/options (no restrictions were placed on what they did within the game during this period). After completion of this exploratory period, participants completed a brief questionnaire comprised of a subset of questions from the Questionnaire for User Interaction Satisfaction (QUIS); whose purpose is to “assess users’ subjective satisfaction with specific aspects of the human-computer interface and several open-ended questions” [53]. In addition to the QUIS-based questions, the survey contained 12 open-ended questions where participants were asked for feedback regarding the Rapid Recovery exergame. Results showed that generally, the game was enjoyable and well received. Participants also provided valuable feedback that will assist in future development of Rapid Recovery.

5 Conclusions

In this chapter, we presented an approach to develop upper-limb exergames focused on shoulder physical activity, and rehabilitation. This process yielded to two different and independently developed kayak-based exergames where a player’s motions are captured via a Microsoft Kinect vision-based sensor and mapped to motions of a virtual kayaker within the game. Although the Microsoft Kinect sensor proved to be reliable for both of the games, with any vision-based sensor), the lighting within the environment it is being used in and (perhaps to a lesser degree depending on the colour of the players’ clothing can have serious consequences on motion tracking, and thus negatively affect the exergaming experience. Although for the two exergames presented here lighting issues were not a big problem, both exergames were tested in settings with relatively controlled lighting. However, if these games are to be used in homes or health care centres, there is no guarantee regarding how consistent the lighting will be (Microsoft recommends that the Kinect is kept away from direct sunlight). Testing of an exergame should thus take into account the environment(s) it will be used in and testing should consider a wide variety of clothing colours. Aside from lighting issues, other problems can include space requirements and constraints. For both of our exergames, space was not a constraint and the distance between the Microsoft Kinect and the user was approximately 2 m although, when used in the home, this cannot always be guaranteed (greater details regarding the correct placement of the Microsoft Kinect sensor are available by Microsoft via Xbox support websiteFootnote 1). Furthermore, regarding the Microsoft Kinect, it has a limited accuracy (approximately 10 % of the movement [20]). If more accurate measures are required, additional sensors such as accelerometers and gyroscopes may have to be added and the data between the various sensors integrated. This may also lead to the use of more intrusive marker-based approaches were the user, for example, must wear some form of tracking device; this can, of course, affect the user’s natural movements. Such a sensor integration approach may also lead to greater computational requirements.

Preliminary usability studies conducted with the exergames provided us with valuable information that will guide future iterations of the games. Aside from the feedback that was specific to each game (e.g., comments regarding the user interface, etc.), these preliminary tests have also shown that an important aspect of an exergame is the availability of multiple scenarios to limit the possibility of users becoming bored with the game and thus losing interest over time. A possible solution to this is the development of an exergame framework that allows the rehabilitation experts (and perhaps the end-user to some degree), to develop their own scenarios or modify existing scenarios. Of course, developing such a framework is not a trivial task as one must balance ease of use with respect to scenario development and the technical (programming) skills of the user developing them; there is no on-size-fits-all solution here.

Although designing and developing exergames is in many respects similar to designing and developing traditional “entertainment games”, there are also important differences. More specifically, designers and developers of exergames must generally adhere to the corresponding content/knowledge base, while ensuring that their game is not only fun and engaging, but is also an effective rehabilitation tool. In other words, the motions/movements that a player/patient must perform within the game cannot necessarily be modified as this could lead to injury. Designers and developers must also consider the implications of game mechanics within the exercise context that allows for the addition of goals and rules such that in combination with computer generated environments, provide users (patients) with a far more engaging experience that can help overcome some of the limitations associated with traditional exercise and rehabilitation approaches (e.g., the inherent decrease in motivation to maintain an exercise and rehabilitation program over time). To this end, care must be taken to ensure that exergames are properly designed and developed to meet their intended goals. Part of this implies that a proper and thorough needs/task analysis be conducted. A needs/task analysis involves a significant research component typically involving interviews with subject matter experts, and the potential users of the exergame and is often done poorly or neglected altogether. Furthermore, once developed, their effectiveness must be tested to ensure that they do indeed meet their intended goals.

Assessing the effectiveness of an exergame is not a trivial task but it should be considered early on in the design process. We believe this involves a two-stage process, an initial “alpha” test to examine the usability and user interface of the exergame, followed by an assessment of its effectiveness. Assessing the effectiveness of an exergame can be aided by the availability of common off-the-shelf technology. For example, microphones and cameras can be used to capture any audio (e.g., voice of the participant if think-aloud techniques are employed), and the video of the participant as they are performing in the test. Furthermore, various sensors can be used to monitor various physiological responses (e.g., heart rate, galvanic skin response, amongst others), that can provide further insight into a player’s state and engagement during gameplay. A complete discussion regarding the assessment of exergames is beyond the scope of this chapter but greater details are provided in a review article regarding the assessment of serious games by Bellotti et al. [6].

In addition to knowledge and expertise in game design and development, exergame designers/developers must also be knowledgeable in the specific content area covered by the exergame and should ideally possess some knowledge of teaching methods and instructional design. In other words, the development of effective exergames is not a trivial task and knowledge in game design solely is not sufficient to develop an effective exergame. As a result of potentially competing for interests between the rehabilitation (and potentially educational) and entertainment value of the product, exergame development is an inherently interdisciplinary process, bringing together experts from a variety of fields including game design and development, and experts from the content domain in question (e.g., medical professionals, rehabilitation and physical fitness experts). Although exergame designers are not expected to be experts in medicine and rehabilitation/physical fitness, possessing some knowledge in these areas will, at the very least, promote effective communication between members of the interdisciplinary team.

The future of exergames is rapidly evolving. Technological advancements such as open electronics, and 3D printing, are providing designers and developers of exergames with great opportunities to develop immerse and interactive experiences. However, care must be taken to ensure that the end product (the exergame) does indeed meet its intended objectives. To this end, although not a trivial task, future work should examine the development of a set of rules and guidelines (e.g., best practices) that can guide exergame designers and developers. Finally, rather than focusing on the visual (graphical) scene and perhaps to a smaller degree, the auditory scene, designers and developers should also consider, and take advantage of, multiple sensory channels [13]. Technological advancements have provided us with consumer level devices that allow for a variety of non-traditional cues (e.g., haptics) to be incorporated into exergames and video games in general and help create more compelling user experiences. An example of such rapid change is the launch of the Kinect V2 that exceeds the first version used in both exergames described here. Improvements can be seen in characteristics such as accuracy distribution, depth resolution, depth entropy, edge noise and structural noise, providing grounds for future works [62].