1 Introduction

According to the World Health Organization (WHO) [1], more than one billion people in the world suffer from some type of disability. In Mexico, according to the results of the 2014 Demographic Dynamics National Survey, 6% of 119.9 million inhabitants suffer from some type of disability, 33% of which, approximately, have motor disability in their arms or hands [2].

New rehabilitation technologies are increasingly used to enhance the quality of life of patients while simultaneously improving therapy techniques [3, 4]. In this sense, haptic virtual interfaces can set multiple different rehabilitation scenarios [3], thus allowing medical experts to design personalized rehabilitation routines and thus adapt therapy treatments to both real environments and patient needs. Moreover, virtual objects in haptic systems reduce patient fatigue. Another advantage of haptic systems is that they provide therapists with quantitative data on patient progress and performance and therapy exercises, including the force exerted in each body movement. In other words, haptic systems record and process electromyographic (EMG) signals to obtain information directly related to muscle fatigue and power, thus assessing the performance of patients during rehabilitation treatments.

The Myo armband is a portable movement recording device (see Fig. 1). It has eight medical grade EMG sensors, a gyroscope, an accelerometer, and a three-axis magnetometer [5]. The Myo armband detects EMG activity in the different muscles of the forearm. Also, it can amplify and filter the detected signals. Once myoelectric signals from the forearm are detected, they are transferred to the target device using Bluetooth 4.0 technology. This article proposes a haptic system consisting of a hand exoskeleton, a graphical user interface, and EMG signal acquisition and processing capabilities.

Fig. 1
figure 1

Myo armband: a EMG sensor numbering, b Placement of the armband and visualization of EMG signals

Full size image

2 Methodology

2.1 Exoskeleton development

We used five actuators to give the exoskeleton movement. Mechanisms A and B required two actuators each to move the proximal and middle phalanges, while mechanism C required only one actuator for the metacarpal joint (see Fig. 2). The actuators of the device are direct current gearmotors with angular position sensors, both from the Pololu Corporation. Such position sensors are quadrature magnetic encoders of 12 pulses per revolution (ppr), thus being 1200 ppr in total, taking into account the gearmotor’s 100:1 gear ratio.

Fig. 2
figure 2

Exoesqueleton: a Mechanical structure, mechanisms A, B, and C, respectively. b Placement on user hand

Full size image

2.2 Design of the graphical user interface

We designed a graphical user interface (GUI) in the Processing Development Environment (PDE), in which a virtual object can be “touched” by means of the exoskeleton. Figure 3 illustrates the main screen of the GUI. Notice that a virtual object (a ball) appears in the middle, whereas the end-effector positions for the exoskeleton mechanisms are represented by red cursors labeled A, B, and C, respectively. The cursors from mechanisms A and B are located above the ball, whereas the cursor representing mechanism C is located in the bottom-left. Therefore, the positions of mechanisms A and B approach the virtual object when they move downwards, while mechanism C approaches the object when it moves to the right. Based on the information obtained from the coders, mechanism positions were calculated using their direct kinematic model [6] and mapped to place them within the virtual environment.

Fig. 3
figure 3

Design of the graphical user interface in the Processing Development Environment

Full size image

Figure 3 also depicts three bars on the right side of the ball. These bars indicate the reaction force being applied in each mechanism – i.e. A, B, and C. The bottom of the GUI screen shows a stopwatch, which begins counting the time when pressing the start button and stops counting when pressing the exit button. Three graphs are also located on the bottom part of the screen and show the EMG signals coming from channels 1, 2, and 3 from the Myo armband. During the tests, we recorded all the data on mechanism positions, reaction forces being applied, and the signals from the EMG channels.

The interface allows users to choose among four levels of virtual object rigidity using the buttons from the left side of the screen. These texture or stiffness options include concrete (maximum stiffness), rubber (moderate stiffness), foam (minimum stiffness), and personalized (i.e. stiffness manually modified with a scroll bar within the maximum-minimum range). Additionally, three types of grip were included: spherical, power, and precision. These options are located on the right side of the interface. Finally, the protocol button leads to a pre-configured routine to analyze user performance. This routine combines the different types of textures and grips available, following the sequence presented in Table 1. The test lasts for 11 minutes and comprises two repetitions of the protocol described in Table 1.

Table 1 Protocol for user performance analysis
Full size table

2.3 Haptic system integration

We used an Arduino Mega 2560 development board to control the exoskeleton. Five pulse width modulation (PWM) outputs were used to control the exoskeleton actuators, and ten digital inputs were used to read the corresponding encoders to calculate the positions. The positional data were sent to the visual interface through the serial port, where the values to be represented within the GUI are mapped.

We compared the end-effector position for each mechanism with the ball’s surface to verify whether there was interaction (i.e. contact) between the two. When interaction was detected, we estimated displacement associated with deformation in the direction of the center of the ball. Then, we calculated the force required to perform such a displacement according to the stiffness of the virtual object. Then, the resulting force value is sent to the Arduino board through the serial port. In the Arduino board, the torque value is calculated for each motor to generate the force. To this end, we created a Jacobian matrix of the mechanism. Then, the corresponding voltage was applied through a power amplifier circuit.

2.4 EMG signal processing

The data provided by the Myo armband is type int8; that is, each channel gives an integer between -128 and 127 representing muscle activation levels. The acquired EMG signals are converted to millivolts by means of a third-degree polynomial regression as described below:

$$ \begin{array}{@{}rcl@{}} y_{i} &=& 4.084\times10^{-3} + 1.239\times10^{-3}x_{i} - 1.861\times10^{-5} {x_{i}^{2}}\\ &&+ 1.457\times10^{-6}{x_{i}^{3}} \end{array} $$
(1)

where xi represents the i-th sample of the EMG signal acquired and yi the corresponding value in millivolts. This relationship was obtained by comparing the EMG signals acquired with the Myo armband and those corresponding to the same myoelectric activity with the Biopac Student Lab system. Afterward, the signal was rectified and segmented for each channel in windows of one second and with a 50% overlap.

To carry out a quantitative evaluation, the following indices were considered:

The root-mean-square (RMS) envelope

provides a measure of muscle strength [7] and is calculated as follows:

$$ RMS = \sqrt{\frac{1}{N}\sum\limits_{k=1}^{N} {y_{k}^{2}}} $$
(2)

where N represents the number of points (window length).

The cumulative wavelength (wL)

provides a measure of waveform amplitude, frequency, and duration [7]. It is computed as follows:

$$ wL = \sum\limits_{k=1}^{N-1} \left| y_{k+1} - y_{k}\right| $$
(3)

The mean absolute value (MAV)

is estimated as indicated below:

$$ MAV = \frac{1}{N}\sum\limits_{k=1}^{N} \left| y_{k} \right| $$
(4)

The performance tests were conducted with ten healthy subjects, who tested between 15 and 20 seconds for strength data and 30 seconds for EMG signals with different grip and stiffness settings.

3 Results

During the experimental tests, we recorded the data collected on mechanism positions, applied forces, and EMG signals. Some of the results obtained are depicted in Figs. 4, 5, and 6, in terms of relationship between the applied force and the displacement generated in each mechanism with respect to the surface of the virtual object. Similarly, we introduce the comparative results of two test subjects, which were selected to analyze their significant difference in terms of user performance. The positions and forces corresponding to subject 1 are shown in blue, whereas those from subject 2 are colored in green. The position of the virtual object’s surface is depicted in red, and it is always constant.

Fig. 4
figure 4

Comparative results of two test subjects: spherical grip with concrete stiffness for mechanism A.

Full size image
Fig. 5
figure 5

Comparative results of two test subjects: spherical grip with concrete stiffness for mechanism B

Full size image
Fig. 6
figure 6

Comparative results of two test subjects: spherical grip with concrete stiffness for mechanism C

Full size image

Figure 4 shows a comparison of the results obtained in the spherical grip test with concrete stiffness for mechanism A, which controls the movements of the little, middle, and ring fingers. On the other hand, Fig. 5 introduces the results on mechanism B, which controls only the index finger. Finally, Fig. 6 presents the results on mechanism C, which controls the thumb.

Additionally, Table 2 summarizes the results obtained in one protocol participant in terms of the EMG signal indices acquired on channel 6 from the Myo armband.

Table 2 Results obtained from EMG indices
Full size table

4 Discussion

As regards the analysis of positions and forces depicted in Fig. 4, we concluded that subject 1 correctly performed the grip almost all the time, except for a short interval between 5 seconds and 10 seconds. Conversely, subject 2 failed to exert adequate force to maintain the mechanism in position, thus causing intermittent interaction with the virtual object. In addition, it can be observed that subject 2 only correctly performed the grip between 15 and 20 seconds of the test.

Figure 5 depicts once more that subject 1 correctly held the grip for almost all the time. However, subject 2 seemed to have more difficulty the grip with mechanism B than with mechanism A. This might be due to the fact that it is more difficult to exert force to hold the grip with a single finger (index). As for mechanism C, Fig. 6 demonstrates that subject 1 was able to hold the grip throughout the test, whereas subject 2 had problems to perform the test correctly. Finally, Table 2 shows differences across grip types and textures. In most cases, the highest value is found in spherical grip, which indicates a greater myoelectric activity for this type of grip with respect to the others.

5 Conclusion

In this work, we create the haptic perception of a virtual object through an exoskeleton and data collected on mechanism positions and forces and EMG signals. This quantitative information on hand movements and applied forces complements evaluations that therapists make on patients being rehabilitated and provides additional information on the activity of the muscles responsible for the movements during the different types of grip, thus recording muscle effort. This information can be particularly useful for therapists to assess patient progress throughout the rehabilitation process.

The EMG signal indices computed in this work are proposed as a measure of muscle strength and contraction. Therefore, the resulting information can be used to quantitatively assess increases in muscle tone or determine the presence of fatigue. Finally, as its main advantage, our haptic system allows users to modify object rigidity, thus adapting to individual patient needs throughout the entire rehabilitation process. The system also enables users to perform different exercises (e.g. lateral grip, finger extension) to work on several hand movements.