Introduction

When a person suffers a physical injury, medical recommendations usually involve physical therapy to attempt to recover from the impairment. The main goal of physical therapy is to restore, develop or maintain the maximum amount of possible body movement. Physical therapy must be directed and supervised by a physical therapist. Moreover, it involves a series of phases that must be carried out including: injury assessment, evaluation, diagnosis and prognosis, treatment, and re-examination [1].

This article focuses on physical therapy for knee rehabilitation, in the strengthening phase, for injuries like cruciate ligament injuries [2] that cause pain and inflammation, immobilizing the natural movement of the knee [3]. One of the main issues with physical therapy patients (especially knee patients) is that they usually suffer from reduced mobility. Even though injury assessment, evaluation and diagnosis cannot be performed remotely, treatment can. However, physical therapists require a way to remotely monitor their patient’s progress, and patients need a way to ensure they are exercising properly [4], as these exercises can often require precise movements and repetition that patients may unknowingly perform incorrectly, being discouraged from continuing remote treatment from subpar results.

A second scenario stemming from the one described above is that patients stop performing their physical therapy because exercises become monotonous [5]. This paper presents a wearable device and a mobile application that allows physical therapists to monitor and assess knee rehabilitation exercises and their execution through the use of in-game representations, reducing the tediousness usually associated with exercise monotony. Nielsen’s usability heuristics [6] were applied with ten physical therapists. Moreover, the cognitive walkthrough method [7] was also performed to evaluate the system.

The wearable device, presented in this paper, is similar to a knee band that has an embedded processor, an accelerometer a gyroscope, and a Bluetooth adapter. The system connects with a mobile application in order to provide feedback on user’s performance and their progress on the exercise program. Both the exercise guidelines and feedback are provided in the form of an interactive game. The system also prevents users from overdoing an exercise, by alerting the patient when the exercise sequence is done. This system (wearable device and mobile game) is called FunTherapy.

To exemplify the functioning of FunTherapy, imagine a patient that has suffered a knee injury. The physical therapist meets with the patient to assess the injury, evaluate and diagnose it. Then, the patient is given a band that they could easily place on their calf. They install the mobile application on their phone. The therapist configures the application for the patient, including the angle of elevation, the session duration and number of repetitions required for each exercise. At home, the patient executes the mobile application, which explains how to place the wearable device, and starts playing level 1. Level 1 is a tutorial to understand how the system works. Afterwards, the game uses the therapist’s configuration to guide the patient exercises and the performance information is recorded. This information is available for therapists at their office. The system monitors that the patient executes the exercise movements correctly and would only count them as valid if the established angle is achieved. Different levels in the game are aligned with different exercises or levels of difficulty to allow progress during the therapy sessions. After a while, the patient and the physical therapist meet again for re-examination.

The rest of this article is structured as follows: the approach to the overall system is first described, followed by a brief review of prior work. Following this, a thorough description of the developed system’s design is presented, including hardware modules and software components. This is followed by a report of the usability evaluations. Finally, conclusions pertaining to this evaluation are presented along with future work.

Approach and contribution

A system evaluation was performed by experts in physical therapy. The evaluation involved ten professionals in the physical therapy field. The computational evaluation was performed by usability and Human-Computer Interaction researchers. The evaluators assessed usability and alignment with traditional therapy exercises.

The research discovered that wearable devices and mobile games can be used to encourage adherence to physical therapy exercises. Also, the physical therapists that evaluated the system believe that the game approach is a ludic solution to engage patients with their therapy and that it helps them understand each exercise easily.

Research on related projects confirmed Fun Therapy’s uniqueness as it incorporates entertaining interfaces, similar to those on mobile games, to help patients perform exercises, and to allow physical therapists to monitor their patients’ progress.

Prior work

Significant research has been conducted to create accurate devices that measure physical movement and help during rehabilitation: from assistive robots [8, 9] to wearable devices and more rudimentary technologies. A literature review was performed in order to determine if there were other initiatives on wearable knee rehabilitation devices. Each paper was evaluated in terms of feedback mechanisms provided, amount of sensors used, their locations in the patient’s body, and whether they included an entertaining way to perform the exercises. Table 1 shows the main results of this review.

Table 1 Description of system initiatives on wearable knee rehabilitation devices
Full size table

Figure 1 shows the location of the sensors on each one of the systems found during the literature review. It is assumed that the right leg is under treatment in all scenarios. As it can be seen in Fig. 1 the trend is that the younger the project is, the least amount of sensors are used. Both initiatives from 2015 use only one sensor each. Lastly, none of the reviewed systems propose a way to make physical therapy exercises entertaining. It should be noted that due to the scope of the research presented in this article, only human-computer interface aspects were considered for this review, excluding clinical analysis.

Fig. 1
figure 1

Sensor layout for studies considered for literature review. Numbers below each figure correspond to “Ref #” on Table 1

Full size image

None of the papers found during this related work review present medical assessments. They are all focused on the computational conclusions. Furthermore, the medical assessment of a system like the one presented in this paper would require medical experimentation. In most countries, this type of experimentation is very strict and regulated. At the most, some of the results presented in the related work are based on surveys with patients. In this paper, the approach was to evaluate the system’s accuracy with physical therapists rather than with patients.

System design

FunTherapy consists of a wearable device and a mobile application working together. The wearable device incorporates sensors for motion tracking (accelerometer and gyroscope), communications (Bluetooth), processing (microprocessor) and power supply (lithium battery). The mobile application is a game that can only be played using the wearable device placed in the patient’s leg as interface. The system is designed to be usable, intuitive and to be operated by both patients and physical therapists. Figure 2 shows FunTherapy’s architecture.

Fig. 2
figure 2

Funtherapy system architecture

Full size image

Physical therapist interface

The system includes an interface that allows the therapists to customize the angle of each exercise, the number of repetitions for a series of exercises and the number of series per day. Therapists must set these parameters according to the muscular ability of their patients, allowing them to increase the difficulty as the leg strengthens. Figure 3 shows the interface through which each exercise’s angle requirement can be set.

Fig. 3
figure 3

On preferences, there are options to define the target angle for each exercise

Full size image

The interface showed in Fig. 3 also allows therapists to visualize their patients’ daily work, and determine whether the proposed therapy is consistent with the patient’s capacity so they can make the respective adjustments for their next session.

Wearable device

A wearable medical device was built (i.e., an autonomous, non-invasive system that performs a specific medical function such as monitoring or supporting). Wearable medical devices are usually composed of: a) sensors to monitor surrounding conditions, b) processing unit and hardware for input and output, and c) customized clothing to attach the components to the user [17].

In this case, the wearable device is the mechanism that patients use to control the mobile application (i.e., the game). It has a button to turn it on, which immediately links it to the application. The only feedback provided to the user is a LED that indicates if it is on or off and when it is linked.

An Accelerometer/Compass/Gyroscope LSM9DS0 Sensor [18] was used, with a 3-axis measurement system (16 mm diameter × 0.8 mm thick). Communications were achieved with a generic Bluetooth module (39.5 mm × 20.5 mm × 1.6 mm). The processor is an Adafruit Flora wearable platform (45 mm diameter × 7 mm thick), and is powered by a 3.7v lithium battery, with a total weight of 53 g. Figure 4 shows the electronic components of our wearable device.

Fig. 4
figure 4

Electronic components of the wearable device

Full size image

The multi-sensor module is responsible for obtaining movement information to be sent to the mobile device through the Bluetooth module, in the form of a sequence of Euler angles [19] in a three-dimensional system: roll (x-axis), pitch (y-axis), and heading (z-axis), through the equations depicted in Fig. 5 x, y and z correspond to data gathered from the accelerometer in each axis. Euler angles were used to transform the gathered data to the proper format in order to be assessed.

Fig. 5
figure 5

Euler angle gathering equations, as performed by the multi-sensor module

Full size image

Mobile application

The patient’s application runs on a mobile device. It is a video game that guides the user to perform the exercise planned by their physical therapist. The only way to control the game is by using the wearable device. The game is based on the oscillation of a spaceship, and the movement of the leg allows the spaceship to go up or down. The game is planned with four levels per exercise. Figure 6 shows the interface in its four levels. Levels 1 and 2 have the same interface but the amount of stars to reach is higher in level 2.

Fig. 6
figure 6

Mobile application (game) levels

Full size image

The first level was designed for patients beginning their rehabilitation and who are on a strengthening phase. The goal is to learn the game flow, which consists on catching a star on the highest point of the exercise (target angle) and another one at the lowest point (starting position). The position of the star is determined by the exercise, and whenever a target is reached, by completing a stage of the exercise, a new target appears at the bottom of the screen, corresponding to the height of the initial position of the leg.

The main spaceship is controlled by the movement of the leg (only within y-axis). A repetition of the exercise is completed when two stars have been caught, increasing by one the number of exercises performed. At this level, the star moves only along with the movement of the leg, if patients are unable to reach the therapist’s proposed target angle the star won’t be reached. This allows therapists to perform parameter calibrations, which can have an impact on reducing patient frustration and quitting rates.

The second level has the same objective as the previous one with the difference that the star moves independently and not along with the patients’ movement, forcing them to calculate when to make the move to reach the target angle at the same time the star does. If patients fail to reach the star (target angle), it reappears at the same height it was at the time of the failure.

The third level increases the game’s complexity on the assumption that patients have increased their muscle power and have a better handle on their leg movements. In addition to catching the two stars to complete the exercise, patients must avoid obstacles that move along with said targets and force them to perform greater muscle work.

The fourth level merges the logic of the third level and an additional control factor: patients must hold their leg at the target angle for a given time period, during which a progress bar appears indicating the time that the limb must be held up at a given position in order for a point to be awarded.

Game design

Two main aspects characterize a videogame: the gameplay and the eye candy [20]. The gameplay refers to the techniques used by the designers in order to keep players engaged with the game: game rules and strategies. The eye candy is the graphical interface. According to Prensky [20], the gameplay is the key factor. A large amount of addictive games with very simple graphical interfaces were found. In this case, the gameplay was defined by patients needs (i.e., physical exercises). On the other hand, as the game was designed for a wide range of patients, a minimalistic and user friendly appearance was developed.

The FunTherapy design was based on the model proposed by Lohse et al. [21]. The system is divided in three subject areas: neuroscience, motor learning and game design. According to this model, every level of the game and the gameplay itself were designed according to the specifications of the exercises and the cognitive abilities of the patients.

To evaluate this game, the Sweetser and Wyeth [22] method for game designs was used, and its adaptation to “exergames” [5]. In this adaptation authors propose eight characteristics a game must have to be considered “good”: concentration, challenge, considers player skills, control, clear goals, feedback, immersion and social interaction.

This game is based on an old game called Xenon 2 Megablast, released in 1989. However, instead of being a vertically scrolling, the game is horizontal, and instead of shooting and avoiding objects, the player must gather them. The decision to use this game was based on the fact that most of the consulted individuals who suffer impairments have had some type of interaction with this kind of games before. Therefore, it is hoped that the similarity to a well-known game helps to have people-engaged in the gameplay so they can provide the desired levels of concentration and immersion.

The challenge (gameplay) is configured by physical therapists. As users play with their treating leg, the challenge increases and the amount of stars and position variations change -according to their capabilities.

The goal is clear just as the game visuals are simple: catch the stars. Also, the feedback is in real time (i.e., as soon as users moves their leg, the spaceship moves in their mobile device screen).

Two types of feedback are provided to patients: visual and auditory. When a star is caught, it flashes and disappears along with a sound. If the spaceship collides with an obstacle, the position of the screen elements resets, accompanied by another sound. For the last level, progress is shown with a charging energy bar that emits a sound when charged to 100 % (see Fig. 6).

The game control is achieved by synchronizing the movements of the patient’s leg with the spaceship. This allows patients to concentrate on the video game by catching the stars. At the end, patients are only focused in the stars and not counting the exercises, achieving the immersion in the gameplay.

Implementation

The first stage of the research consisted on the development of the wearable device and a software package for testing. The wearable device uses the accelerometer and gyroscope with the Arduino Flora platform for the purpose of measuring the movement of the leg, which is transmitted via Bluetooth to the mobile device and is validated according to the selected exercise. The wearable works like a goniometer device, which is used by therapists to measure the angles on the limbs.

The design of the wearable prototype is focused on being non-intrusive for the execution of the exercise (total weight: 142 g). All the circuitry is enclosed within a container mounted over a shin guard (as shown in Fig. 7), allowing for a natural placement of the device on the leg as can be seen in Fig. 8.

Fig. 7
figure 7

Wearable prototype

Full size image
Fig. 8
figure 8

Wearable prototype on patient’s leg

Full size image

The development of the mobile application was performed using the game-development framework libgdx, and implemented for the Android operating system. The structure of the application consists of two layers. The first layer is responsible for the Bluetooth connection between the phone and the wearable device for receiving the movement data. The second layer is responsible for rendering the screens and for the logic of each level on the game.

Evaluation

This section describes the evaluation of the FunTherapy system. Ergonomics and design evaluations were conducted. Afterwards, usability tests were performed. Finally, a cognitive walkthrough method was applied by physical therapists.

Wearable device ergonomics

To evaluate the wearable device ergonomics, the Gemperle, et al. [23] method was used. In this method authors describe 13 guidelines for wearability: placement, form language, human movement, proxemics, sizing, attachment, containment, weight, accessibility, sensory interaction, thermal aspects, aesthetics, and long-term effects. Table 2 shows how the implementation considered each proposed guideline.

Table 2 Wearable device ergonomics assessment throught the application of guidelines
Full size table

Expert evaluation and user experience

Ten professional therapists used and evaluated FunTherapy. They were asked about interfaces issues and the main results were: the instructions to use the tool were understandable and in the case of the first exercise the target angle indicators were fine. About the game interface, they said that it was motivating because patients focused on catching the stars rather than on counting exercise repetitions.

Experts suggested improvements to the videogame such as: adding music, more levels with higher difficulty, and an option to configure the speed of the exercise, so therapists can adapt them to the muscle capability of each patient.

About the wearable device, experts said it is not intrusive for the body, is lightweight (142 g), and the position does not restrict movements for the knee rehabilitation therapy.

First, to test the usability of FunTherapy, therapists were shown how the system worked, and the they tried it. Each therapist played all levels, and set each angle of the exercises as they liked. After that, they were introduced to Nielsen’s Heuristics, explained the concept of each heuristic, and each participant indicated whether or not they considered heuristics were satisfied.

Table 3 shows the results of the evaluations, and the heuristics best qualified were:

  • Visibility of system status.

  • Match between system and real world.

  • User control and freedom.

  • Consistency and standards.

  • Help users recognize, diagnose, and recover from errors.

Table 3 Nielsen’s heuristic evaluation results
Full size table

The evaluation shows that therapists identified as useful elements that provide feedback to the user. They appreciated that the system is simple to understand, and that it keeps a constant game flow that does not distract patients from their therapy. Visually, it is very easy to detect if patients are not doing the exercises correctly, facilitating the communication between patient and therapist.

Help and documentation was the heuristics with less rating because the application does not have instructions or a user manual. Some therapists emphasized the importance of teaching patients how to use the application, for example with some game instructions at the beginning of each level.

Regarding feedback received from therapists about the system and its use as a complement to physical therapy, following comments can be highlighter (originally in Spanish):

  • “I think it could be a very good complement. It allows us to prescribe exercises properly and keep a historical record on the activities completed by each person throughout their recovery”.

  • “Generally with knee injuries, one starts with passive or active movements. Personally, I like the app to start in active ones. I do think the application is a good complement in physical therapy and there are no obstacles to add on weight or to treat isotonic contractions.”

  • “The patient is as focused on the exercise as on the application. Since it is a game, the patient will want to complete it, which is beneficial because it prevents the exercising routine to go dull, and the therapy’s goals are achieved”.

  • “Yes, the game is useful because through technology users can perceive exercising as a game and not as just another therapy task” as through the use of technology the user can see the exercises as a game and not just as therapy assignments.

  • “The game is very passive, so it might not motivate the patient” The therapist states that the stars are moving very slowly; the therapist feels that for a patient at an early stage it would work well but a more advanced patient would requires more challenges. The therapist suggests adding parameters for time measurement when the stars appear and move, as well as the charging time of the energy bar at Level 4.

  • “The device is very light, but for a person suffering knee pathology it might be of significance”: Since there are many causes for knee injuries, the therapist should consider whether patients are too weak to carry the weight of the wearable and if it will affect their rehabilitation process. As further work, we will test smaller sensors in order to have a lighter device.

Summarizing, therapists agreed that FunTherapy system serves as a tool for improving knee rehabilitation therapy by providing a recreational component that allows patients to motivate their daily work. However, for it to be more functional for therapists, it still needs to provide more options of customization and to validate the movement from the rest of the limb to ensure that each exercise is being performed correctly.

In the cognitive walkthrough, the first task was:

  1. 1.

    Select Videogame option.

  2. 2.

    Select Level 3.

  3. 3.

    Leg raises the spaceship and catches the first star, successfully avoiding obstacles.

  4. 4.

    Leg lowers, controlling the spacecraft to catch the second star, completing one repetition of the exercise.

  5. 5.

    Repeat 3 and 4 until indicated repetitions are completed.

Four questions were asked, focusing on action visibility, user recognition of correct actions, user feedback, and comparing if actions taken by users were the same they were trying to achieve.

Results show that in the first step users weren’t lost in the system and successfully achieved selecting Level 3. But once the second step was completed, users were unable to tell which level they were playing because there were no indications to it; it is necessary to improve feedback in this sequence, although indications of the completed exercises are very understandable.

The second task was to configure Level 3 at a 45 degrees angle and test it. The actions sequence was:

  1. 1.

    Select Preferences option.

  2. 2.

    Select the angle of exercise for Level 3.

  3. 3.

    Enter number 45 for the new angle.

  4. 4.

    Save new angle.

  5. 5.

    Return to Main Menu.

The same questions were asked, and the results were: for the first three steps users were not lost in the system, but in the fourth step, the system did not provide any notifications if angle changes were saved.

Conclusions and future work

Based on the results described above, it has been proved that it is possible to create an electronic device that works like a goniometer and allows therapists to obtain data from therapy exercises, even when performed remotely. It has been also established that therapy patients will be more encouraged to engage in therapy when supported with the developed system.

Therapists evaluations indicated that elements on the system interface are adequate and can help users understand therapy requirements through implemented in-game analogies, such as target angles. Research also suggests that for each exercise it is important to properly configure angles and speed conditions in an individual patient basis.

As for future work, several potential avenues of research have been identified. Developing different videogame themes (according to the exercises recommended by the therapists) and incorporating feedback elements for achievements during therapy, as well as supporting and creating opportunities for social interaction to increase user adherence are some of these. Also, more research in Physical Therapy needs to be conducted, including patient clinical trials and validating the use of the device from a clinical perspective, through evaluation tools such as the Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST) [24] or the Physical Impact of Assistive Device Scale (PIADS) [25].

During the evaluation process, a case of a blind student of the Physical Therapy career was brought up. It was discovered that it is also possible to use a wearable system to create a digital tool for blind therapists, by adding audio alerts when movement measures meet the target, so the actual system can be modified to incorporate this function.