Keywords

1 Introduction

Multiple Sclerosis (MS) is an inflammatory, debilitating, demyelinating disorder of the central nervous system which has a particularly high prevalence in Scotland (Mackenzie et al. 2014). The disease severity and symptoms are highly variable between individuals, but typically fatigue, loss of bladder control, and sensory and motor dysfunction are the most common (Kister et al. 2013). Motor deficits can specifically include loss of balance, spasticity, difficulties with gait and coordination, muscle weakness and loss of fine motor skills. Overall, these symptoms have a significant damaging effect on the individual’s quality of life due to becoming increasingly more dependent on others, and patients often develop depression as a result (Fernández-Jiménez and Arnett 2015). Rehabilitation is one of the main forms of treatment to improve motor deficits and activities of daily living.

Novel approaches are becoming increasingly popular within rehabilitation and now often include the use of gamification strategies along with enhanced interaction into their approaches. There is the desire for balanced rehabilitation approaches that are accessible and affordable, but also effective and motivational. Despite being an ongoing healthcare issue, research into upper limb involvement and management MS is sparse. Research in recent years has highlighted the potential role of robotics of MS upper limb rehabilitation. Whilst studies have demonstrated its success (Gijbels et al. 2011; Carpinella et al. 2012), there have been issues with accuracy and some argue it may not be more effective than traditional methods (Maciejasz et al. 2014). Furthermore, these robotic systems require specific facilities, specialised apparatus and therefore are not as easily accessible to patients compared with other approaches. They are more expensive than alternative techniques and it has been disputed if the reported successful results are sufficiently cost effective to cause a substantial shift in using robotics (Van der Loos et al. 2008). This is where computer-generated environments often referred to as Virtual Environments (VE) are believed to tackle this issue. As well as being easily accessible, the advantages over conventional methods can include higher motivation, the provision of real-time user feedback and versatility (Lange et al. 2012). Commercially available gaming platforms in particular have been extensively studied, such as the Nintendo Wii system. However, their suitability for motor impaired individuals, such as MS, was argued not be sufficient due to participants reportedly feeling discouraged with negative feedback (Plow and Finlayson 2014) as these gaming systems target healthy individuals and with no consideration of differing ability levels. This highlights the importance of the creation of approaches based on games specifically targeted at improving functionality within disability-related conditions.

Literature has debated the effectiveness of virtual approaches in comparison to other low costing rehabilitation techniques (Saposnik et al. 2010). Whilst it is unclear if virtual rehabilitation is significantly more effective, it is argued that it offers benefits to the user including an immersive but safe environment; positive reinforcements using feedback; engaging and enjoyable experience; and these are believed to be important for patient compliance and thus recovery within MS (Massetti et al. 2016).

Motion capture systems are a more recent development in this research area. For instance, the potential of Leap Motion (LM), a powerful cost-effective hand tracking system, has recently been investigated for virtual rehabilitation research. Despite the technology not being extremely accurate with tracking, there are other multiple benefits to using it from a patient and healthcare standpoint. It is more accurate than e.g. Microsoft Kinect for hand tracking and cheaper than the robotics system; is compatible with most computers and is portable – making it ideal for use at home (Taylor and Curran 2015). It has been primarily researched with regards to stroke but has not yet been fully explored with MS patients. Studies have found it to be successful in terms of improving mobility, allowing excellent feedback to the user with no noted adverse effects in stroke patients (Khademi et al. 2014; Iosa et al. 2015). Therefore, it is possible these benefits could transfer to MS patients and has the potential to become a technique for MS upper limb motor rehabilitation.

To date, there is no published study on the usability or creation of serious games using LM technology for upper limb improvements in MS rehabilitation. This could be due to difficulties in game design in accommodating the wide range of symptoms and varying disability levels within patients. Therefore, this study used a co-creation approach to develop a series of its rehabilitation exercises using the LM technology. This involvement of participants from the early stages would potentially highlight common themes within this target audience and input these into the game design.

1.1 Aims and Objectives

The main aim of this project was to create a user-friendly collection of rehabilitation exercises to facilitate the improvement of hand dexterity in MS patients. Another aim was to determine the common hand problems people with MS experience and investigate if the rehabilitation of these movements could be incorporated into these exercises using LM. In addition, this project aimed to determine the opinion of MS patients on the developed exercises, the benefits of using LM in a rehabilitation context and of potentially using it at home. Adopting a co-design approach with early and continuous user input aimed to make the overall product more desirable and successful for use. Achieving these aims involved collecting qualitative data using focus groups (FG), to highlight common hand issues or their user preferences and input this into the game design and evaluate the developed game to strengthen it further. The evaluation will also highlight the suitability of this technology for this specific target group. Theme-Based Content Analysis (Neale and Nichols 2001) was used as an inspiration to determine user opinion and common themes amongst their comments.

2 Materials and Methods

2.1 Materials

A demonstration set up of the LM controller connected to a PC was used for the co-design FG. “Playground” was the demonstration game used to stimulate discussion among members of the FG (https://gallery.leapmotion.com/v2-playground/). The additional materials required for the evaluation stage was a videorecorder to record the hand movements of participants during use of the exercises and a digital audio-recorder to record the focus group interaction.

In terms of the rehabilitation gamified exercises, the graphical models of various objects were created using Autodesk 3ds Max, a powerful 3D computer modelling platform. The game engine Unity was used to construct the actual rehabilitation exercises. The device used to motion track the user’s hand was a LM controller connected to a desktop PC with a conventional 2D monitor.

2.2 Methods

This study used a co-creation approach in the development of the rehabilitation exercises. This involved the user input from the beginning of the development and evaluation after the first stage of development. This technique has increased in popularity within product development due to the advantages of early intended user input and can potentially create a more successful product (Kristensson et al. 2008). This collaborative approach, with early user input, before the production of the game design, is beneficial in establishing user needs and create a more successful design.

2.2.1 Predevelopment Focus Group

The participants for this study were recruited through a local MS social group in Glasgow. There were no exclusion criteria for recruiting theses participants, however to be included participants had to self-report some degree of upper limb dysfunction. Five participants, four females one male, all gave written consent to participate. A FG methodology was selected to gain information due to its benefits in acquiring a wide range of detailed information and evaluating group opinion (Rabiee 2004). The aim of this focus group was to gather information regarding the participants’ MS symptoms such as what upper limb dysfunction participants experienced, their comfortability with this hand tracking technology and what would be motivating to use LM. This session lasted approximately 2 h and a transcript of the focus group taken from the audio-recording was produced. The participants were asked the following questions:

  1. 1.

    How does multiple sclerosis affect your daily function in life?

  2. 2.

    What do you struggle with your hands – particularly around your home?

  3. 3.

    Are there any actions which are impossible to do?

  4. 4.

    Do you do any hand exercises at the moment, if so what?

  5. 5.

    What do you like about this LM technology?

  6. 6.

    What concerns do you have about potentially using this technology?

  7. 7.

    What would motivate you to use it?

2.2.2 Development of Rehabilitation Exercises

Using the results from the FG, a functional and non-functional analysis of requirements was conducted to define relevant functions or system characteristics to be implemented. Certain functions and actions were chosen to be incorporated into the exercises based on mutual problems highlighted by the FG. A total of four exercises were created, with each incorporating a different hand movement, referred to as “Practice”, “Pinch”, “Type” and “Two Hand Interaction”.

Practice

The “Practice” exercise’s purpose was to allow the user to become comfortable with using the LM device as well as interacting with a virtual environment. The assets used within the practice scene reflected what would be used within the other rehabilitation exercises and household items the participants expressed difficulty with using. The objects modelled were a wine glass, coffee mug, door key and ball as these were objects discussed as difficult to hold in the initial FG. A “Restart” user interface (UI) button was added for the user to set the models back into their default position.

Pinch

The “Pinch” rehabilitation game was designed for the user to grasp bubbles in the virtual environment. It was scripted such that if the user successfully made the correct movement at the bubble model, the user would score a point. Consequently, a new bubble model would appear in a different positional transform after each successful movement. Audio feedback was added, with a popping sound after each successful hand movement. The scoring system was displayed on the UI along with a count down timer of 120 s for the user to track their progress. A “Restart” UI button was added to this scene to allow the option for the user to try again at this exercise.

Type

The “Type” exercise included a model of a piano keyboard for the user to train their individual finger movement. This was created to replicate the typing action some participants struggled with. Audio feedback of a piano key note was implemented to convey to the user they had performed a successful movement. The piano keys were numbered corresponding to the finger the user was to use for a specific key.

Two-Hand Interaction

The final rehabilitation exercise “Two-Hand Interaction” was developed due to the lack of LM studies incorporating the movement of both hands into therapeutic gaming. This scene included a tennis ball model and a net model which had an added animation to move up and down. The goal of this exercise was to throw the ball between the hands, with the added challenge of avoiding the net.

The application included a start scene, a main menu, an ‘about application’ page so the user could navigate easily through the exercises. Calming music was added to each exercise which was downloaded from the Unity store. The series of exercises were gathered as a game entitled “Handy Rehab”.

The realistic hand model was chosen due to the disconnection a participant had with the mechanical one. The intractable elements of the scene were all ensured to be within the LM controller’s range, as the user would use their hand to navigate through the application and interact with the virtual environment.

2.2.3 Evaluation and Post-development Focus Group

The participants were from the same group that took part in the predevelopment FG. However, due to scheduling difficulties, only three of the five participants, two females and one male, could attend the date for the post-development FG. Each participant individually tried out the exercises and were invited to go through each one and give an initial response as they participated. During this, their hand movements were video recorded to observe their own hand actions and compare to the game play. Participants were also informed that they could stop at any time and were offered arm support if required, because fatigue of the hand/forearm had been identified as a possible concern earlier. After each participant had taken part, the FG was conducted to gather qualitative data on their opinion was of the exercises.

The aim was to attain participant opinion and feedback of the developed exercises. As before the session was audio recorded and a transcript was produced. This FG lasted 35 min and the questions asked were:

  1. 1.

    What was the difficultly level when tasked with picking up objects?

  2. 2.

    Which objects were the most challenging to pick up?

  3. 3.

    What improvements or additions would you like to see in the exercises?

  4. 4.

    What was your favourite exercise? Why?

  5. 5.

    What was your least favourite aspect? Why?

  6. 6.

    Did you find the exercises engaging? Why?

  7. 7.

    Did you feel like that was your own hand displayed on the screen?

  8. 8.

    Would you use this at home? Explain.

  9. 9.

    What would further motivate you to use it?

  10. 10.

    What was your opinion on the layout, easy or difficult to understand?

  11. 11.

    Thoughts on the music?

3 Results

3.1 Predevelopment Focus Group

The data from the predevelopment FG were the transcript from the session. There were group discussions for each of the main seven questions, with participants often sharing similar problems with regards to their MS. There were differing opinions with regards to views of using the LM technology and motivational factors. Common thoughts or main opinions were highlighted for each question.

When the researched asked: “How does multiple sclerosis affect your daily function in life?”: there was common agreement of symptoms experienced such as sensory discomfort, mobility issues, spasms, pain but also mention of how fatigue was a huge factor in their daily functioning and how it can alter their motor function rather than their own mobility levels. Furthermore, one participant stated that they needed to be in that mindset to do something mobility-wise and that readiness varied from day to day. One participant summarised what they felt the most common issues were to their daily living:

Bladder issues and fatigue are main aspects but also fine dexterity. But everyone is different.

The participants also noted that they would have to “compensate” with their actions but also often depended on others in the home to assist. With regards to the question: “What do you struggle with your hands – particularly around your home”, there were numerous actions and tasks the participants particularly struggled with, which are displayed in the Table 1 below.

Table 1 List of difficult actions relating to everyday function
Full size table

Actions that the participants noted as impossible to do were very small finger movement such as: putting on jewellery, fastening buttons, using zips or tying laces. Participants expressed differences with hand mobility levels or which hand or fingers were more mobile. With regards to doing hand exercises at home, only one participant stated that they did an exercise such as hand stretching. Another stated:

You are meant to [do home exercises], but I don’t.

When asked “What do you like about this technology?” many participants stated that it was a “cool” and different views regarding gaming rehabilitation. They felt it could be good for muscle control and using the same muscles in the games as you could to do movements in a real environment. They reported enjoyment of the calm and relaxing music within the LM demo. Another positive response to LM demonstration was that “you can see your own hand and you are the interaction”. The demo LM game used a robotic hand, which received different opinions from participants. One participant argued that it was beneficial as you could visualise the joints of your hands, while another felt a “disconnect” with their hand:

I would like it [hand model] to look like a hand… My brain just wasn’t connecting that it [robotic hand] was my hand on screen.

Another concern about the LM technology was the potential to cause strain or discomfort with having the forearm raised for periods of time. Most participants additionally stated that they were not interested in gaming nor used gaming technology frequently. Only one participant mentioned previously using gaming technologies stating:

It reminds me of the Wii, but this would be good for dexterity.

In terms of motivational factors, participants differed on what aspects would motivate them to use LM. They agreed there needed to be different games with instant gratification or feedback such as time or a score. Other motivators included progression such as unlocking new levels after completion or a multifunctional approach:

I would use it if I knew it was something I really had to do…Or I would need to do that to access my emails or something.

3.2 The Developed Rehabilitation Exercises

The developed application was built in Unity specifically for Windows. The first display scene is the ‘Introduction’ page with three buttons to navigate through by using their hands instead of a mouse cursor. It gives the user the option to read the instructional rules to these exercises or to proceed straight to choosing a game to play if they are already familiar with what to do. Additionally, the user has the option to exit the application at this stage.

Four exercises were created in this game application: practising picking up household objects; pinching the bubbles; hitting the keys on the piano keyboard and throwing a ball between both hands. Actions were chosen based on FG data. Due to participants saying they struggled with grabbing or picking up household items, the “Practice” scene offered the user the opportunity to interact and grab these (Fig. 1). The “Pinch” game stimulates fine dexterity issues the participantss with fine finger actions they struggled with. The user can faciliate this action onto the bubbles and keep track of progress. The user can use either hand during this activity. The “Type” exericse simulates typing an activity participants reported they had difficulty with. This activity also faciliated independent finger movement. The user can use either hand during this activity. The “Two-Hand Interaction” exercise involved the overall coordination between both hands but also incorporated wirst action by throwing (Fig. 2).

Fig. 1
figure 1

The practice game play, along with image showing set up and user hand in correspondence with onscreen

Full size image
Fig. 2
figure 2

The two-hand interaction game play

Full size image

3.3 Results from Evaluation and Post-development Focus Group

Each participant tried every developed rehabilitation exercise and gave their reactions and feedback. Whilst it was clear what do to in each game, the corresponding hand actions were not as obvious to the participants initially.

Practice

The participants initially struggled picking up objects in the scene but after being shown the specific hand action required for LM to recognise a successful grasp, it became easier for the participants. All participants experienced objects often dropping from their grasp or even disappearing from view. One participant stated they enjoyed the visuals of this scene and said it was a realistic representation of a home environment.

Pinch

It was observed that participants found it difficult to successfully pinch the bubbles closer to the screen edge. This is not ideal as one participant expressed arm tiredness during this game and had to switch arms. Figure 3 shows a participant demonstrating the successful action within this exercise.

Fig. 3
figure 3

Participant No. 3 user testing the “Pinch” exercise

Full size image

Type

This exercise was received positively during testing by all participants, with comments including enjoyability of the overall task. All participants expressed difficulty in hitting individual keys with fingers other than the index finger (Fig. 4).

Fig. 4
figure 4

Participant 2 user testing “Type” game

Full size image

Two-Hand Interaction

Participants liked the overall design and reported it was pleasant aesthetically. They enjoyed an outside element to this exercise. During game play, the participants often struggled to control the rolling ball and would compensate their action by grabbing and placing the ball over the net. It was noted that the hand models would often distort when both hands were tracked.

Focus Group Data

The transcript data were analysed by using an approach inspired by theme-based content analysis as described in Neale and Nichols (2001). This approach analysis for qualitative data regarding the user’s opinion throughout evaluation and has proven useful within VE research contexts. Comments from the FG were assigned into common themes, which then related to a higher order theme category. The Table 2 below demonstrates common and higher order themes in the FG, along with a quantified number of responses by participants.

Table 2 Theme based content analysis of the evaluation FG data from MS participants
Full size table

3.4 Participant Input into Game Design

Further minor developments were undertaken to the “Handy Rehab” exercises due to the feedback given by the participants as this was a co-creation process. These changes included the addition of count-up timers on all exercises in order to provide time feedback; changing the physical properties of the ball models to avoid rolling too much; and widening the piano keys making it easier to successfully strike the key. Other changes to the UI were made making the cursor easier to visualise and interact with the VE.

4 Discussion

This study aimed to develop VE rehabilitation exercises using a co-creation design with input of MS patients, to potentially improve hand motor skills and be used as part of their home-based rehabilitation. Overall, the participants found the developed exercises to be immersive, engaging and enjoyable. This study has therefore demonstrated a possible technique for upper limb rehabilitation in MS. Each participant had a different favourite exercise, suggesting co-creation was a successful approach, enabling the exercises to be enjoyable for all participants and also in highlighting concerns which could interfere with compliance in future use. The differing opinion in preferable exercise also demonstrates the need for choice and personalisation in rehabilitation game design. It was beneficial to have initial input as requirements in the game that reflected the needs as found in the predevelopment FG.

From the evaluation, participants were keen to use this approach at home if there was evidence for improving their dexterity and would treat it as part of a physiotherapy regime. Rehabilitation benefits was found to be a high order theme from the evaluation FG and was discussed to potentially be the greatest motivational factor in this group for rehabilitation exercises.

However, concerns with this LM technology were expressed by participants. Usability was another common theme in the post-development FG and there were noticeable problems during user testing. They expressed the movements they made during the exercises did not correspond exactly to the natural actions of the hand when picking up such objects – which they thought could be detrimental when trying to improve dexterity. In the Practice scene, even if they were making the appropriate action, grasping was not always successful. Though the difficulty aspect of this level made them “determined”, they also commented that it was frustrating. This could be due to the tracking abilities of LM and should be addressed in future work.

Since this study is one of the first to investigate LM in MS patients, it is difficult to compare to other similar works. Literature is sparse in terms of upper limb motion capture-based technology for this branch of rehabilitation. Therefore, this study has provided first steps in developing rehabilitation gaming for this specific target audience. It has pinpointed the potential of an optional relaxing approach to serious gaming, as well as potential user discomfort and concerns – from feedback and input from MS users.

Realistic and interactive virtual environments have proven effective in improving upper limb dysfunction in stroke patients and are believed keep the patient engaged (Choi et al. 2014), but this is not as well documented in MS. Although this feedback came from a small sample of participants, it could be suggested that this group of individuals with MS would prefer engaging, realistic simulations in their virtual environment compared to abstract gamification.

The developed exercises aimed to offer a variety of different hand actions or movements which is reflected in the four separate exercises. However, the picking up objects and pinching bubbles within the different exercises involved the same movement detection within Unity. It involved fine opposition of all fingers at once which aimed to reflect the fine motor actions in Table 1, however, it was often difficult to grasp objects – even when facilitating this movement correctly. The participants often felt the interaction when picking up objects was not the same as in a normal setting. This highlights the restriction of the LM tracking in this rehabilitation approach. Contrastingly, a study by Gieser et al. (2015) found the LM controller to offer high accuracy using Unity for game development for cerebral palsy rehabilitation. However, this involved the detection of static gestures only and did not include interaction with virtual environments which the present study did. This suggests that LM could be restrictive when it comes to interacting with virtual objects and could be better suited to using gesture-based movements. Nevertheless, the challenge with this would be applying the engaging factor into this new approach that this present study offered.

4.1 Limitations

The main limitation of this study was using the LM technology itself. There is a lack of documentation with regards to this technology, Unity assets and involvement in rehabilitation studies. In terms of tracking, the user would often lose track of hand or object. Further disadvantages included frequent non-detection even with correct movements, and it was easily susceptible to occlusion interfering with tracking. Ebert et al. (2014) found similar weaknesses of LM, along with reported tracking time delays, in their study but suggests this could be short term with the increasing developments in refining tracking technology. Despite this there are advantages to LM over other more expensive tracking systems. This illustrates the idea of ‘give and take’ with technology. For usage by professionals in future, they must be aware there will be drawbacks that come with cheaper, more accessible technologies.

Whilst having input from individuals with MS was a positive aspect for this study, the number of participants was small - with a smaller number returning for the evaluation session. Opinions from a small, select group of individuals, from the same geographical location is not representative of all individuals with MS. It is hard to determine if these findings are representative of a wide field of patients and if the exercises’ design would be as well received in a larger audience. Future applications should involve a larger number of participants to overcome this limitation.

4.2 Future Work

Although additions were made after the evaluation FG, these were limited due to time constraints. Therefore, future alterations could include the addition of different levels of difficulties and milestone achievements as these were found to be a potential motivator as well as adding more feedback values. Scoring and time feedback was difficult to implement into every game design and was not possible due to time constraints of the study. With regards to the Practice scene, this had the least positive response. The participants desired this to have an end result – rather than just picking up objects. This could include interaction between the objects themselves, such as placing them inside each other. This emphasised a clear purpose was needed for the exercises or for component parts of each exercise.

As highlighted by the evaluation FG, the participants would be keen to use this approach if this lead to improvements in their hand mobility. Therefore, future research could involve investigating, using a longitudinal study, if this method of LM would improve hand functionality long term. Additionally, this type of study would highlight potential adverse effects of using LM long term and the motivation the user possesses after continued use. To further investigate the benefits of LM over other methods, a comparative study could also be pursued. This would identify any advantages over other rehabilitation techniques and what LM can uniquely offer to patients.

5 Conclusion

This project achieved the main aim of making a collection of interactive and enjoyable rehabilitation exercises for MS patients made possible due to their early input in the design process. This study successfully inputted user opinion into the design to create a successful, promising product. The results from the evaluation highlight the limitations in hand tracking, thus LM may not be able to offer a wide range of movements that reflect the upper limb difficulties of people with MS. Although restoring hand mobility was incredibly important for these participants, they did not partake in any rehabilitation exercises at home. The positive responses to aspects of the created exercises, along with participants stating they would use this approach at home if there were reported benefits, suggests this approach possesses potential in rehabilitation. Therefore, more extensive research is necessary to determine the relevance and rehabilitation benefits of such approach in MS.