Keywords

1 Introduction

Conventional diagnosis techniques in Parkinson’s disease (PD) rely on the clinical assessment tools such as Unified Parkinson’s Disease Rating Scale (UPDRS) [1]. However, these tools are based on the subjective evaluation of some disease factors and are not fully interpretable. Another shortcoming is that traditional rehabilitation treatment is often slow, monotonous, disempowering and a non-motivational process, from which patients can easily give up. Due to all these facts, there is a clear need to introduce new techniques into the rehabilitation processes able to enhance both, the therapist evaluation procedure and the patient’s motivation.

Our ultimate goal is to build a multifunctional, affordable and portable system, to be used as a support to the conventional rehabilitation practice as well as home rehabilitation. In this work, we present an important part of the future rehabilitation structure – the analysis of patient’s movements along with the decision-making (diagnosis support) scheme.

The main limitation of our approach results from the relatively modest accuracy of the Kinect and its inability for tracking finger joint trajectories without additional equipment.

The remaining of the paper is structured as follows. Section 2 reviews the state of the art of rehabilitation techniques applied to neurological disorders with special emphasis on those used in Parkinson’s disease. Section 3 explains the procedure of data acquisition using the Kinect. Section 4 describes the experimental group of subjects used in this study and the performed exercises/movements. Section 5 explains the algorithm we developed for extracting relevant measurements. Section 6 summarizes the results of classification between patients and controls and the analysis of the most informative features to support the decision-making (diagnosis support) process. In Sect. 7 we draw some conclusions and propose future extensions of this work.

2 Related Work

Recent approaches for rehabilitation treatment, combining the medical background and new sensing and data-analysis technologies, have a large potential to improve the diagnosis and rehabilitation therapy. However, they still face substantial challenges regarding the practical integration and application in a clinical setting. Some of the main challenges concern the inclusion of the technical systems into medical protocols, implying lengthy certification procedures, costly equipment and software and overall system complexity.

Marker-based motion capture (mocap) systems [2] are often used for movement acquisition during rehabilitation sessions, as they can be extremely accurate but also extremely costly. Other alternatives include the integration of different sensor types attached to the patient’s body [3] or hand (data glove) and, more recently, low-cost marker-free mocap systems such as Kinect and Xtion [4]. The performance of lower-cost systems has been tested and shown to possess a satisfactory accuracy for the application in the rehabilitation therapy [5–7]. While some examples of Kinect-based rehabilitation systems are described in [8–11], little attention has been devoted to the specific case of Parkinson’s disease [12, 13]. Recently, authors in [12] have studied the Kinect accuracy for measuring movements of Parkinson’s patients, but they did not implement the movement analysis. They compared the Kinect to the VICON mocap system through a set of rehabilitation exercises. Their results suggest similar temporal accuracy between the two systems, when measuring the movement duration and spatial accuracy regarding to the upper body movements. Their general conclusion is that the Kinect has potential to be used for movement analysis in Parkinson’s disease and promising application in the future for home rehabilitation.

To raise a patient’s motivation during therapy, some studies have introduced virtual environments into data acquisition and processing procedures for Parkinson’s disease [13, 14]. Main limitations with the use of virtual environments and rehabilitation games are the lack of official safety-evidence and proof of clinical effectiveness.

Rehabilitation studies for neurological disorders usually concentrate on the analysis of particular body functionalities such as postural control [13], gait [15], upper body movements [16] or even the observation of the behavior of a concrete joint [17]. Our work incorporates the analysis of such main human – gait and upper body – functionalities, based on the characteristic skeleton joint coordinates acquired with a Kinect during rehabilitation exercises. So far, most rehabilitation studies have been developed in order to monitor the treatment and observe a patient’s progress [8–11, 13–17]. Our approach extends these aspects by including a comparison between patients and control subjects in order to determine benchmark ranges of relevant rehabilitation measurements (features) for assisting the diagnosis. We describe in detail how to define a set of measurements to characterize the movements of a subject. In addition, we propose the symmetry ratio, widely used as a validity criterion for models in biomechanics and motor control [18, 19], a new feature for rehabilitation. In fact, it has been shown that the symmetry of kinematic speed profiles results exclusively from neurological mechanisms [20, 21], without any interference from changes of conditions or variables of the performed task. In this paper, we introduce the full potential and the importance of this new proposed measurement together with standard rehabilitation features for diagnosis and monitoring in Parkinson’s disease.

3 System Description and Data Acquisition

All data-acquisition techniques in the rehabilitation practice can be broadly categorized in two main approaches, depending on the adopted sensor technologies: (i) body-fixed or (ii) vision based sensing. Due to the complicated process of attaching the sensors or special markers to the body, as well as the discomfort felt by the subjects, patients often manifest the resistance according to the techniques based on the body-fixed sensors. On the other hand, visual systems composed by a large set of cameras can be quite complex in terms of system integration, costly and require a technician’s presence. For those reasons, several recent research efforts in the rehabilitation field have explored the use of motion sensing devices, such as the Kinect and Xtion [4]. These low-cost devices may offer a suitable alternative to more expensive and complex vision-based mocap systems used today in rehabilitation practice.

The use of the Kinect has advantages, compared to the traditional rehabilitation techniques, in terms of the patient’s motivation during the therapy and opening the possibility of home rehabilitation. The process of the data acquisition is based on the visual skeleton tracking and the possibility of storing 3D positions of characteristic joints for every frame using marker free based technique. The maximum frame rate for the Kinect is 30 frames per second (30 Hz), but in our case due to additional processing in order of data collection, frame rate drops down to 27 Hz. The acquired data consist of 3D positions (x, y and z coordinates) of characteristic skeleton joints, collected during the movement performance, along with RGB and depth video sequences (Fig. 1).

Fig. 1.
figure 1

(a-d) RGB stream and (e-h) depth stream from Kinect with detected skeleton and collected joints

Full size image

A few papers have discussed the Kinect accuracy and justified its use in the rehabilitation practice [5–7]. Results suggest that the Kinect accuracy is sufficient for the rehabilitation treatment of the Parkinson’s disease [12], but they also report the variations of the data accuracy depending on the distance to the subject, tracked body-part and the range of movement. Since the accuracy drops down significantly with the increase of distance to the Kinect [5], we have chosen the smallest possible distance that would still allow full body tracking, which was around 1.5 m.

4 Experimental Groups and Performed Exercises

Our experimental group consists of twelve patients with Parkinson’s disease of the first, second and third stage [22]. In addition to commonly used rehabilitation features, we measure the symmetry ratio from specific upper body movements and it will be shown later in the paper that this measure contains significant information relevant for the movement analysis and to discriminate among healthy and non-healthy subjects.

Following the therapist’s advice, all rehabilitation exercises are designed to recover or enhance one of the three main human functionalities – balance, mobility in the sense of normal gait and upper body movements [23]. Due to clinical protocol, the gait test is fairly present in the majority of rehabilitation procedures and it can have different forms depending on the equipment used and the measured gait features [23]. In our work, the gait test is carried out in accordance with the available Kinect range [6], with the starting and end points placed at 3.5 m and 1.5 m away from the Kinect, respectively. Under the gait test, patients have crossed the selected distance of 2 m six times with normal and natural gait rhythm (Fig. 1-a). The relevant measurements extracted from the gait are the gait speed and a measure of hand rigidity during walking. The procedure for extracting relevant measurements is explained in detail in the next section.

The rest of the tested exercises belong to a group of 3 upper body movements: adjusted shoulder abduction-adduction (SAA) (Fig. 1-b) until maximum possible range of motion, shoulder flexion-extension (SFE) (Fig. 1-c) and movements of the right-left hand between the boundaries (hereinafter HBM, Fig. 1-d). The first two exercises are well known in the rehabilitation practice and the third one is introduced in order to test the patient’s movement performance when focusing on a target. Regarding the relevant measures of the first two exercises, range of motion, speed of the movement and symmetry ratio are measured. From the hand boundary movements, we chose only the speed as relevant performance measure, due to the specific nature of the test.

5 Approach for Characterizing Movements

We have used several measurements (features) that represent the motion of the different body parts of the subject. The choice of features was partly resulting from discussions with doctors, therapists and other domain experts. All together we have used 9 different features that result from the combination of four categories of measurements (speed, rigidity, range of motion and symmetry) applied to 4 categories of lower/upper body movements, as illustrated in Table 1.

Table 1. The features used result from a combination of 4 body movements and 4 different measurements (speed, rigidity, range and symmetry).
Full size table

Measurements extracted from gait movements are commonly used in the rehabilitation practice and treatment [23]. From our gait movements, we consider two measurements – speed of the gait and hand rigidity during walking. We have observed the mean gait speed V (1) during each two-meter sequence. Due to possible deviations of the starting and end point of the gait test, and in order to improve the accuracy, the path length (the numerator in (Eq. 1)) has been calculated as the total trajectory of the torso during each gait sequence, instead of setting the path length of 2 m. The total trajectory is obtained by summing up the Euclidean distances (d) between the torso joint coordinates Xi (xi, yi, zi) and Xi-1 (xi-1, yi-1, zi-1) for each consecutive frames i and i-1 during the gait sequence. The time duration of the gait sequence (denominator in (Eq. 1)) is expressed in seconds and has been computed as the ratio between the total number of frames (m and n denote respectively the starting and last frame of the sequence) and the frame rate, f = 27 Hz.

$$ V = \frac{{\sum\nolimits_{i = m}^{n} {d(X_{i} ,X_{i - 1} )} }}{(n - m + 1)/f} $$
(1)

The position of the arms during walking can reveal rigidity, one of the main indicators of the Parkinson’s disease [1]. In the case of healthy subjects, the arms usually swing in a certain rhythm during gait activity, in contrast to the Parkinson’s patients. We compute a measure of rigidity, based on the hands position during the gait test. The rigidity symptom can be noticed in the variation of the distance between the hip and hand during the gait sequence. For healthy subjects, the temporal evolution of these distances is approximately periodic, due to normal arms swinging. In contrast, for patients with one rigid arm, the distance between the rigid hand and the closest hip does change significantly over time (Fig. 2-a). The measure of rigidity is calculated in two steps. First, we record the difference between the left/right hand-hip distances, during the gait movement.

Fig. 2.
figure 2

(a) The difference between the left/right hand-hip distances shows the rigidity symptom. (b) evolution of the shoulder angle profiles during shoulder abduction movements.

Full size image

In patients with rigid arm, the difference signal is larger, because the healthy arm performs a normal swing and the rigid arm remains more or less static. Instead, healthy subjects display a lower-amplitude signal, due to the normal swing of both hands. Finally, we take the highest value of the (absolute) difference signal as an indicator of rigidity.

Inspired by well-known and widely used rehabilitation measure for upper body movements, we have computed the range of motion [23] for the shoulder abduction-adduction and shoulder flexion-extension exercise. The range of motion represents an angle of the movement relative to a specific body axis, which can be measured at various joints such as elbow, shoulder, knee, etc. In our case, we measure the evolution of the shoulder angle during the movement in relation to the longitudinal body axis (Fig. 3). Depending on the relative positions of the shoulder (x1, y1) and elbow (x2, y2) joints (z coordinate is not relevant), the shoulder angle may take the value of α (Fig. 3-a), 90° + α (Fig. 3-b) and 180° + α (Fig. 3-c) in the first, second and third quadrant, respectively.

Fig. 3.
figure 3

Definition of the shoulder angle for different scenarios depending on the arm position

Full size image

As a specific movement descriptor we have used the range of motion (maximum achieved shoulder angle). Examples of the shoulder angle profiles of both normal subjects and patients for the shoulder abduction movement are shown in Fig. 2-b. The range of motion is higher for healthy subjects (more than 180°) than for patients (142°, 150°). In addition, the trajectory of shoulder angle is steeper for healthy subjects, indicating a higher speed of movement. We calculated the mean movement speed for all three tested upper body exercises. The applied procedure was the same for the gait speed (Eq. 1), setting the path length to the total length of hand trajectory during the movement.

The comparison between relevant left/right body-side movement descriptors can suggest which side or limb is more affected by the neurological disorder. For healthy subjects, these differences are usually negligible, while they can become quite large for Parkinson patients, depending on the disease stage. Important movement descriptors such as profiles of joint angles (Fig. 2-b) and angular velocity profiles (Fig. 4 - a) can reveal the symmetry of the movements. In order to quantitatively assess the movement symmetry, we have extracted symmetry ratio from the shoulder abduction-adduction and shoulder flexion-extension exercises. In motor control, the symmetry ratio (SR) [18–21] (Fig. 4-b) is defined as the ratio between acceleration (tACC) and deceleration (tDEC) times, during one movement. Figure 4-a shows that the maximum angular velocity of the shoulder abduction movement is higher for healthy subjects than it is for Parkinson patients. In addition, healthy subjects reach the maximum angular velocities of the left/right arm movements approximately at the same time as opposed to non-healthy subjects, where a difference of about 20 frames is typical. The consequence is unbalance in symmetry ratios between left and right arm for the same movement. Thus, in our experiments we obtained larger left-right differences of the symmetry ratios for Parkinson patients than in healthy subjects.

Fig. 4.
figure 4

(a) Evolution of the shoulder angular velocity profiles during shoulder abduction movements and (b) symmetry ratio calculation

Full size image

6 Results

The normalized ranges of the relevant measurements extracted from patients and healthy subjects data are illustrated in the boxplot on the Fig. 5. Values of range of motion and gait/movement speed are lower in the patient group, while the left-right arm differences of the symmetry ratio, during shoulder movements, are much larger in patients, as expected. By adopting the proposed relevant measurements, we obtain 9-dimensional feature vectors (Fig. 5), which can be used in a classification system to assist diagnosis. We applied a process of dimensionality reduction to the 9-dimensional feature space, in order to improve the classifiers efficiency and accuracy, as well as to determine the most informative features for rehabilitation. Dimensionality reduction is performed using Linear Discriminant Analysis (LDA) [24], that allowed us to transform our original 9-dimensional data set into a new 2-dimensional feature space.

Fig. 5.
figure 5

Normalized relevant measurement ranges

Full size image

As a side-result, the LDA method also ranks the original features in terms of their contribution to the reduced 2-dimensional feature space, and five features (1, 9, 5, 7 and 8 from the Fig. 5) have shown significantly higher impact compared to the other ones. This result suggests that, in addition to the speed of the gait and upper body movements, both symmetry ratio features are associated to a group of the most relevant features.

Classification between healthy and non-healthy subjects is performed based on the original feature set, the five most relevant features and the two new obtained features from LDA, using four different classifiers (Fig. 6): SVM – support vector machines with RBF kernel (bandwidth of the RBF kernel, \( \sigma \) and regularization parameter, C: \( 0.01 < \sigma < 1 \), \( 0.01 < C < 10 \)), KNN (number of nearest neighbors, \( {\text{k}} \in 1,3,5 \)) and two types of neural networks (various structures with different number of hidden layers and nodes): MLP – multilayer perceptron and RB – radial basis (bandwidth of the RBF kernel: \( 0.01 < \sigma < 2 \)). Parameters of classifiers are chosen from listed ranges in a validation procedure in order to achieve the highest accuracy rate. The best results for all classifiers are reported in the case of new created 2-D feature vectors, whereby SVM and MLP have shown higher accuracy compared to KNN and RB.

Fig. 6.
figure 6

Classification accuracy for each classifier

Full size image

7 Conclusion

We presented an approach for therapeutic movement analysis based on the low-cost vision system (Kinect), to support the traditional rehabilitation procedures both for diagnosis and monitoring purposes. Our results have shown significant differences between experimental (patients) and control (healthy subjects) groups for the majority of proposed relevant measures and the possibility of successful classification based on the observed measures. For reducing the computational cost, we have applied a dimensionality-reduction procedure and determined the most informative features in terms of assisting the medical diagnosis process. This result underlines the significant role of new measurement we proposed – the symmetry ratio feature for classification procedure. For the future work we plan to extend this research and to introduce additional devices, such as data gloves in order to design a comprehensive movement analysis system.