Keywords

1 Introduction

Stroke is a brain attack that occurs when blood flows to an area of brain is cut off, and it represents a leading cause of disability [1]. This situation causes death of brain cells that are deprived of oxygen. Since a stroke involves brain cells, their death has important consequences on the functions of the human body; in particular, functional deficit depends on the brain region affected by dead cells. If the stroke occurs on the right side of the brain, it will affect the left side of the body, including the left side of face. In this case, paralysis on the left side of the body, poor eyesight, quick and inquisitive behavioural style, memory loss can arise too. When the stroke occurs on the left side of the brain, the right side of the body will be affected, producing paralysis on the right side of the body, speech disorder, slow and cautious behavioural style, memory loss. In some cases, stroke can occur in brain stem, where, depending on the severity of injury, it can affect both sides of the body and may leave someone in a locked-in state. When this state occurs, the patient is generally unable to speak or achieve any movement below the neck [2].

Rehabilitation of people affected by a stroke is probably one of the most important phases of recovery for many stroke survivors. Rehabilitation can bring subject to his independent life, if properly addressed. To achieve this goal, monitoring EMG (electromyography) signals during stroke rehabilitation therapy can provide valuable insights on neuromuscular system, leading to increased recovery effectiveness.

In this work, we propose a novel approach to forearm rehabilitation that makes use of Myo armband, a low cost gesture controller, to acquire electromyographic signals. Our approach relies on the execution of rehabilitation tasks of a serious game in Augmented Reality (AR) using a simple video see-through setup. This choice reduces equipment and national healthcare system costs, making the system portable and allowing a true home rehabilitation.

The Myo Gesture Control Armband from Thalmic Labs (Fig. 1) is a chain of plastic, or rectangular “pods”, where each of them has one medical grade stainless steel EMG (ElectroMyoGraphic) sensor able to detect the electrical pulses of muscles. Myo armband has 8 distinct sEMG (surface EMG) sensors. It communicates via Bluetooth and sends the 8 samples (one per pod) with a frequency of 50 Hz. EMG is referred to as myoelectric activity. Muscle tissue conducts electrical potentials similar to the way nerves do and the name given to these electrical signals is the muscle action potential. Surface EMG (sEMG) is a method of recording the information present in these muscle action potentials [3].

The muscle activity and the EMG signal associated may vary according to the number of Motor Units (MU) recruited and their activation frequency. A MU is the smallest functional unit, which describes the neural control of muscle contraction. During voluntary muscle contraction, two independent parameters modulates the force applied: the first one represents the number of recruited MUs and the second one is MU activation frequency. Considering an experiment, which involves the same muscle activity and the same applied force, is unlikely to observe the same pattern signal. The main parameters that influences EMG signals are:

  • tissue kind

  • tissue thickness

  • user’s temperature

  • user’s physiological state

Due to the non-reproducibility of EMG signals when performing the mentioned experiment, it is appropriate to apply particular smoothing algorithms to filter signals. Using appropriate processing techniques, such as Average Rectified Value (AVR) or Root Mean Square (RMS), it is necessary to evaluate the average signal trend that represents the applied force.

2 Related Works

Various approaches, different from classical therapy, have been used formerly on post-stroke rehabilitation such as virtual reality and other techniques [4, 5]. In [6], it is used an active robot exoskeleton to support patients in their rehabilitation phase doing reaching/grasping tasks. Moreover, they have also developed a real time tracking system in order to let the patient interact with real objects. In [7] an arm rehabilitation robot has been studied in order to perform task-oriented repetitive exercises in patients with neurological and orthopaedic lesions, training patients with everyday life activities. In [8] an arm rehabilitation system based on Augmented Reality was developed; they use an haptic sensor to measure the wrist impedance in order to have strength feedbacks, while the AR system is used as hand tracking system. A similar work could be found in [9], where they perform different tasks for different upper art parts (hand, wrist and arm) in order to evaluate different parameters for each task.

In some cases, it is important to migrate the rehabilitation phase to patients’ home where it is possible to adapt the actual environment to personal needs after medical staff understood the best living conditions, for example using Virtual Reality (VR) simulations [10, 11] and Augmented Reality (AR) using video or optical See-Through Devices [12, 13].

We developed a serious game to evaluate the relationship between the force applied by the user and the difficulty level of augmented reality serious game used in rehabilitation tasks. Compared to state of the art our system offers a low cost and portable augmented reality setup.

Fig. 1.
figure 1

Serious game setup

Full size image

In the next sections, we describe the serious game, the acquisition protocol for sEMG signals and their processing in order to find the aforementioned relationship.

3 Methodology

3.1 Experiment Description

The augmented reality serious game designed is inspired by an arcade game developed and published by Atari, Inc. in 1976 and known all over the world. In the game Breakout, shown in Fig. 2, a layer of bricks lines the top third of the screen. A ball travels across the screen, bouncing off the top and side walls of the screen. When a brick is hit, the ball bounces away and the brick is destroyed. The player loses a turn when the ball touches the bottom of the screen. To prevent this from happening, the player has a movable paddle to bounce the ball upward, keeping it in play. The game difficulty is strictly correlated to ball speed.

Fig. 2.
figure 2

The original Breakout game by Atari, Inc (1976)

Full size image

Our serious game incorporates AR technology and tangible interaction techniques to provide an innovative interface to rehabilitation tasks. The system augments the captured image of the real environment with computer-generated graphics to present a variety of game or task-driven scenarios to the user [14]. The tangible interaction (through Tangible User Interface or TUI [15, 16]) facilitate to interact with digital information using the physical environment. TUIs have emerged as an alternative paradigm to conventional GUIs allowing users to manipulate objects in virtual space using real, thus tangible, objects.

The designed AR serious game involves the use of two tangible markers, on which 3D objects, that are part of game, are attached. The system, via a webcam, tracks the position and orientation of each tangible markers as it is moved. The used optical tracking system is based on the natural feature tracking algorithm implemented in Vuforia [17]. The “A marker” is associated to the line of bricks and to the game area, while the real TUI “B marker” is associated to the paddle used to hit the moving ball in the game.

In order to simplify the user’s movement to achieve certain results, a single target represented by a virtual cube (or cube target) was considered (Fig. 3) in the upper right corner of the “A marker”. The initial position of the sphere is in the upper left corner of the scene and, during the game, it moves toward the cube (paddle) that the user grabs in her/his hand (Fig. 4).

Fig. 3.
figure 3

The initial positions of sphere, paddle and target

Full size image
Fig. 4.
figure 4

Sphere trajectories

Full size image

The goal is to hit the cube target with the sphere. Unlike the original game, in this version the user does not move the paddle left-right, but s/he can only rotate the paddle on its vertical axis. This way, we consider only a wrist rotation.

3.2 Testing

System Setup. Starting the experiment, it is necessary to place ARTags in a correct way. The “A marker” must be placed on a table in front of the user, while the “B marker” must be placed between the “A marker” and the user. The user should seat in a way that both, the centre of the marker attached on the cube and the shoulder relative to the forearm under rehabilitation, lie in the same sagittal plane as shown in Fig. 5. The user wears the Myo armband, s/he has to relax the arm to warm up the device. During this phase, the Myo establishes a strong electrical connection with the muscles. It is not necessary to perform the usual calibration. In fact, in this work, the Myo device is used as an array of sensors to record the sEMG signals and not as a gesture controller.

Fig. 5.
figure 5

Handle and movement

Full size image

In the initial stage of rehabilitation at home, users have to be trained in order to handle and to move the cardboard cube in the correct way. This is necessary to reduce the noise caused by the tension of other muscles registered involuntarily in the event.

Experiment Start. When the application is started, a setting panel is shown, where an operator may insert:

  • IP Server: the address on which game sends message to start and stop process that saves samples;

  • Name: the user’s name;

  • Initial Speed: the sphere speed at first attempt;

  • Speed Increase: the value that is added to sphere speed of the previous attempt to get the sphere speed of the actual attempt;

  • Levels number: the number of trials in an experiment

An experiment consists of two trials. In particular in the first attempt the sphere has a speed equal to 0.17 m/s while in the second trial it increases to 0.39 m/s. When all the objects are framed on the scene, the sphere, starting from initial position as described previously, moves towards the paddle; the user has to hit the sphere in such a way that it bounces off pointing to the virtual target and s/he has to hold final position until the sphere hits the target or exits the scene.

In order to make trials having the same duration, the sphere speed is decreased after hitting the paddle. For each trial we recorded 150 samples.

4 Experimental Results

For this study 5 healthy persons aged between 24 and 30 years (mean 27,2 sd 1,9) were recruited. For each subject the sEMG signal related to the two trials was registered. First of all, collected samples were processed to detect the two sensors with the greatest variance on the measured signal, let name them sensor A and sensor B. In Table 1 the values of variance for each user and each sensor are shown for each attempt.

Table 1. Variance for each user, attempt and sensor
Full size table

Then it was performed an amplitude analysis on the samples recorded by the two sensors identified in the previous step. It was calculated the RMS which represents an evaluation of the force applied by the user during the test [18]. Table 2 shows the results. Finally, it was normalized the forces on the value obtained in the trial with higher sphere speed (Table 3).

Table 2. RMS values for each sensors
Full size table

5 Discussion and Results

In order to consider only the sensors activated during the movement, it was calculated samples variance for each sensor. It were chosen the first two sensors on which was registered greatest variance (sensor A and sensor B). Samples registered by these sensors were used to calculate the mean power of the signal which represents an evaluation of force applied by user during experiment. The value considered to compare the force applied in the two trials is an average between mean power of signal registered by sensor A and sensor B.

In the Table 2 it is possible to observe that the applied force is proportional to the sphere speed (Fig. 6).

Fig. 6.
figure 6

Result

Full size image
Table 3. Normalization
Full size table

In a hypothetical rehabilitation scenario a supervisor could set a game level suited to the physiological conditions of the user and then monitor the rehabilitation performance in terms of applied force.

6 Conclusion

In this work, we propose a system for forearm rehabilitation based on an AR serious game using a video see-through device.

We use a low cost and portable setup including a gesture controller device to monitor the rehabilitation task performances recording and processing sEMG signals. The samples are processed to find an evaluation of the applied force during the rehabilitation task related to level of difficulty of the game. In a rehabilitation scenario the results could be used to evaluate the improvements analysing the performance history.