A 3-DOF hemi-constrained wrist motion/force detection device for deploying simultaneous myoelectric control

Abstract

For describing the state of the wrist, either the force or movement of wrist can be measured as the training target in the simultaneous electromyography control. However, the relationship between the force and movement is so complex that only the force or movement is not precise enough to describe its actual situations. In this paper, we propose a novel platform that can acquire three degrees of freedom (DOF) wrist motion/force synchronously with multi-channel electromyography signals in a hemi-constraint way. The self-made wrist force-movement mapping device establishes a stable relationship between the wrist movement and force. Meanwhile, the elicited wrist movement can be directly fed back to the subjects via laser cursor. The information of the cursor can directly reflect the 3-DOF movement of the wrist without any decoupling algorithms. Through this platform, the support vector regression model learned from the training data can well predict the arbitrary combinations of 3-DOF wrist movements. The cross-validation result indicates that the regression accuracy of free 3-DOF movements can reach a similar performance to that of 2-DOF regular movements (in terms of R², regular movement vs. free movement, p > 0.1).

1 Introduction

For regulating hand movements, our central nervous system sends action potentials (APs) to the targeted motor units (MUs) that further contract the muscles on the forearm accordingly. At the same time, the myoelectric signal (electromyography, EMG) can be measured by detecting the electric potentials on the skin or inside the muscle using surface (sEMG) or intramuscular electrodes (iEMG), respectively. As an electric manifestation of the neuromuscular activities, the EMG signals can well reflect the level of muscle contractions, as well as the pattern of movement intentions. It has been widely used in muscle neuropathy testing, motion analysis, rehabilitation, and prosthetic control.

At present, by processing the EMG signals collected from the forearm muscles, the patterns of several hand and wrist movements can be well recognized in the literature. On the other hand, commercial products like Myo Armband (Thalmic Labs, USA), multi-functional EMG controller (Coapt, LLC, Chicago, USA) [1], etc., also start to be available. Through a series of steps including signal segmentation, feature extraction and classification, the motion-related information can be detected from EMG signals; thus, a discriminative model for classifying a variety of hand gestures can be well established [2,3,4,5,6]. However, despite the diversity of various pattern recognition approaches, their performance may have no significant difference in terms of its real practical usability.

After reviewing many EMG control studies of these years, C. Castellini et al. pointed out that, to improve the prosthetic hand’s functionality, the simultaneous EMG control of multiple degrees of freedom (DOFs) were urgently needed [7]. Relative investigation on this direction just starts recently and has very limited achievements. One big different is that, for hand gesture recognition, the motion classes used for training can be directly observed and recorded; while for the simultaneous EMG control, the high-precision multi-DOF movement labels are very difficult to measure by hand. At present, the methods for acquiring the multi-DOFs wrist activities (either motion or force) can be divided to four types.

• 1. Vision prompt-based A moving cursor was displayed on the screen to prompt the degree of force, and the subject was instructed to elicit corresponding forces of multiple DOFs [8,9,10] according to the movements of the cursor. This approach is easy to achieve but its performance was largely dependent on the subject’s control experience. The force elicited was so subjective that it might be both deviated and delayed from the prompt cursor. Some influencing factors are highly related to the subjects’ states (reaction speed, fatigue, etc.), which are very difficult to detect, resulting in the low accuracy and repeatability. Besides, when the subjects attempted to elicit multi-DOFs forces, it was very difficult for them to plan these forces synchronously. Rather, the subjects would like to perform only simple combinations of independent DOFs. Another shortage is that, the wrist motion detected by this method was not rich enough to cover its whole workspace.

• 2. Position-based This method placed position trackers on the forearm of the subject, and calculated the wrist motion according to the relative position between the trackers and the signal source [11]. Although the rotation angles of the wrist could be exactly measured, they can be only used to reflect the spatial movement, but not the elicited force of the wrist. The relationship between the movement and the force is ambiguous that requires lots of manual operation to establish and maintain.

• 3. Motion capture-based The motion capture device (Vicon, PTI, etc.) was employed to detect the movements of the joints on the forearm (wrist, hand, and fingers) [12, 13]. Similar to the position-based method, this method can also measure the kinematic information of the forearm simultaneously. However, the forces elicited by the joints could not be explicitly given according to the kinematic information. The relationship between the force and movement of the joints still needed the manual operation to establish and maintain.

• 4. Force-based A six-dimentional force transducer was utilized for detecting the multi-DOFs force elicited by the wrist indirectly (the force/torque output by the hand was recorded and then transformed into the wrist’s coordinate) [14,15,16]. The repeatability of this method was high, and its accuracy was largely depended on the performance of the transducer. However, the subjects could only know the force level through their own perception, or by observing the results displayed on the screen. In addition, the testing hands were always fixed throughout the experiments, having very limited movement range that differentiates from its nature condition, resulting in a gap between the research and real practice of this method.

All of the methods above tried to utilize existing sensors for acquiring multi-DOFs wrist motion/force. In general, either the forces or the motions of the wrist are measured as the training target in the simultaneous EMG control. Comparing these two means, although multi-DOFs movements are more visible and convenient for the subjects to plan, it is the force manifestation that can better represent the degree of the control wills sent from our central nervous system (CNS) to a specific group of muscles. However, none of them can acquire the force and motion of a joint synchronously, or establish a stable relationship between them. In both (2) position-based and (3) motion capture-based methods, there could be no force getting involved in the motion planning phase that makes the joint in test totally unconstrained. On the contrary, in (1) vision prompt-based and (4) force-based method methods, there could be no movement during force eliciting that makes the joint in test fully constrained. Both fully unconstrained and constrained conditions neglect the complex relationship between the joint’s movements and force output, which are different from their application scenario. Meanwhile, these methods were not intuitive enough and it was difficult for subjects to perform complex motions of multiple DOFs. The most complex motions the subjects performed in these methods were some simple combinations of two DOFs. Also, these measurements were not precise enough to describe the actual situations. All these factors led to limit achievements in the field of simultaneous EMG control, since reliable targets for effective regression cannot be obtained.

In this paper, we propose a novel platform that can acquire 3-DOF wrist motion/force in a hemi-constraint way, as well as, collect multi-channel EMG signals at the same time. Within the platform, we design an elastic structure with a group of springs for transforming the 3-DOF wrist force into a measurable movement. The collected wrist movement (via machine vision) can keep a stable relationship with the force information, and thus can be utilized directly as the learning target for the simultaneous EMG control. Meanwhile, the elicited wrist force (include both the orientation and scale) can be directly fed back to the subjects via graphic demonstration. Subjects are able to observe and control their wrist’s movement/force in real-time, and do not need to pay too much attention to coordinate different wrist DOFs. The EMG control experiment conducted with this device is intuitive and repeatable. The motions that the subjects performed during the experiments are more similar to their nature counterpart. The wrist movements and multi-channel EMG signals measured on this platform are synchronous, thus the EMG samples with reliable, supervised labels can be acquired in the study of simultaneous EMG control for finding the inner relationship between the EMG signals and 3D wrist force. Besides prostheses control, this platform could also be applied to human robot interaction and virtual reality technology.

2 Materials and methods

2.1 Materials

2.1.1 Platform design

The platform proposed in this paper should be designed according to three main principles: (1) the platform should not restrict the subject’s wrist to allow free movements, (2) the platform should be able to establish a stable relationship between the 3-DOFs wrist force and motion, and (3) the platform should be able to give an intuitive feedback to the subject for improving the simultaneous control performance.

The diagrammatic sketch of the whole platform is shown in Fig. 1. The platform mainly consists of a self-made wrist force-movement mapping device (WFMMD), an EMG acquisition device (Trigno Wireless, Delsys, USA), a camera (avA1000-12kc, Basler, Germany), a computer (Dual Core 3.4GHz, 8GB RAM), a projector (PT-X600, Panasonic, Japan), and a display screen. The subject’s hand and forearm are respectively fixed on two separate parts (frames) of the WFMMD, and a total of eight EMG electrodes are placed on the forearm in a circumambient configuration. The WFMMD provides suitable resistance forces to any wrist movements in a hemi-constraint manner; by doing so, the multi-DOF wrist force can be quantified by measuring the wrist’s movements. The laser installed on the WFMMD emits a cross cursor onto the screen to display the degree of the wrist movement as an intuitive feedback to the user. Meanwhile, the camera captures the pictures of the cursor on the screen and streams them into the computer, within which image processing algorithms are implemented to extract the exact position of the cursor. This detected position/orientation of the cursor is also projected on the screen using a ring-shape indicator, which is easy to follow for the subject. The EMG signals are collected and synchronized with the wrist movements (within the software). The projector also displays the target (final position/orientation) and expected trajectory of the cursor, which can be used to validate the effectiveness of various simultaneous control methods.

Fig. 1

The diagrammatic sketch of the platform. The movement of the wrist will influence the position/orientation of laser cursor on the screen. The computer calculates the position and orientation from picture, and then saves this information with EMG signals simultaneously. The calculated result could be shown on the screen through the projector

2.1.2 The wrist force-movement mapping device (WFMMD)

When human wrist moves, the rotation axes of its three DOFs (pronation/supination, flexion/extension, and ulnar/radial deviation) have a deviation of 6.8 mm [17]. On designing the WFMMD, we considered these rotation axes to be coincided, and set the coincided point as the center of the device. The schematic diagram and pictures of the WFMMD are shown in Fig. 2. The device is composed of two cuboid frames as in its inner side (for fixing the hand) and outer side (for fixing the forearm), respectively. The connecting points fixed on these two frames constitute two cubes, and the center points of these two cubes are coincided, as shown in Fig. 2a). Eight springs were installed for connecting the vertices of the inner cube and the outer cube. For lightening the operation load, the inner frame was welded by square aluminum alloy. In the inner frame, the handle and fixtures were installed for fixing the hand. The placement of the handle on the inner frame is adjustable, thus the rotation point of the wrist can coincide to the center of the two cubes. The fixture installed on the outer frame is also an adjustable ring-shape structure composed of pieces of bandages. It can help fixing the forearm on the outer frame and keeping it horizontal during the experiment. The outer frame is fixed on the table, while the inner frame is a hemi-constraint part (through springs) that can move along with the hand caused by the 3-DOF wrist movements.

Fig. 2

The wrist force-movement mapping device

During the experiments, the forearm and hand of the subject were fixed at the two frames of the WFMMD, respectively. All subjects moved their wrists freely, which in turn drove the inner frame to rotate. To improve the comfort, only one fixture was applied to fix the forearm, and thus the forearm was not completely fixed and might move in a small range. This movement would lead to the translation of the inner frame, and it was very difficult to avoid when the subjects were instructed to move their wrists only. The springs were arranged according to the symmetrical structure of the cubes. When the inner frame was driven to translate (or rotation) alongside a single wrist DOF, outside the moving direction the resultant force (or moment) produced by the springs was zero. For example, when the inner frame rotates along the −Z-axis, all the springs would extend the same length and the resultant moment is on the exactly opposite direction of the rotation (+Z-axis). On the other hand, when the inner frame translates along −Y-axis, the right four springs will be shortened and the left four springs will be extended, as shown in Fig. 3b). The resultant force along the +Y-axis is a compound effect of these changes. The relationship between the 1-DOF translation (or rotation) and the resultant force (or moment) is shown in Fig. 4. The size of the outer cube and inner cube are 310 and 155 mm, respectively, and the stiffness of springs is 0.2 N/mm. The 1-DOF rotational movement of the inner frame (∆ϕ, Fig. 4a) would reach 15~20°; while the translational movement of the inner frame (∆y, Fig. 4b) seldom exceeded 40 mm. When the subject produced requested wrist movements, the force arms were highly related to their hand’s size and did not exceed 80 mm at present. According to Fig. 4, the resistance force to rotational movements is about an order of magnitude larger than that to translation movements.

Fig. 3

The diversification of springs while controlling the inner frame. When the inner frame rotates, all the springs will be extended. But when the inner frame translates, four springs will be shortened and the other four springs will be extended

Fig. 4

The calculated resultant moment and force while inner frame rotating or moving

For acquiring wrist motion, our method does not use any inertial measurement unit (IMU) sensors on the inner frame. Our motivation is that, from the WFMMD itself, an intuitive feedback could be given to its user during the operation. The subject should be able to directly observe the resultant motions and thus instantly adjust their wrist. On the other hand, this feedback information could be also measured precisely by the computer at the same time. As shown in Fig. 1, a laser transmitter was installed on the inner frame of the WFMMD. This laser transmitter can display a cursor in the shape of cross on the screen. Through acquiring and processing the image of the cursor on the screen, we can get the cursor’s exact position and orientation. This information can reflect the posture of the inner frame (or, the hand). The laser transmitter contains two pieces of lens for scattering the point laser to two line lasers that constitutes the cross. The scattering angle δ is influenced by the parameter of lens, and the angle we selected is 30°. The distance from the laser to the screen (S) determines the size of the cursor. During our experiment, the S was set to 4.5 m, and the length of the line of cross cursor reached 2 m.

Because the cursor was projected onto the screen, the translational motion of the inner frame would be displayed with the same size. However, its rotational movement would be enlarged along with the distance S. As shown in Fig. 5a), when the inner frame rotates α (or θ) along x (or y-axis), the center of the cross cursor would produce the movement of c_x (or c_y), which keeps a linear relationship with the distance S, as

$$ {c}_x=S\tan \alpha, $$

(1)

$$ {c}_y=S\tan \theta . $$

(2)

Fig. 5

The movement of the cross laser cursor while rotating the inner frame

During the experiment, both c_x and c_y can reach about 1 m. But the translation of the inner frame could only displace the cursor a maximum 40 mm on the screen.

When the inner frame rotates ϕ degree along the Z-axis, as shown in Fig. 5b), the projected cross cursor would also rotate the same degree. The length of the line of the cross, u, and the movement of the end point of the line (along Y-axis), c_z, keeps a linear relationship with the distance S, as

$$ u=S\tan \left(\delta /2\right) $$

(3)

$$ {c}_z=u\sin \phi =S\tan \left(\delta /2\right)\sin \phi $$

(4)

where, the δ denotes the scattering angle (δ = 30°). Thus, the c_z would also increase along with the distance S. Giving different S, an identical wrist motion would result in different cursor movements, in terms of c_x, c_y, and c_z. The resolution ratio of the motion could be improved by giving a reasonably large distance S.

As mentioned before, to operate the WFMMD, the subjects need to elicit an appropriate force to overcome the resistance force of the springs, which produces a hemi-constraint condition to the subjects’ wrist. Even the subjects’ forearms were loosely fixed by the fixture and required to keep static during the operation, the inner frame would inevitably translate because of human body’s synergistic movement (it very still difficult for the subject to rotate his wrist purely). For the translation part, it was not the information we need. In this paper, the translation of the inner frame can be neglected because (1) the movement of the cross cursor on the screen caused by translation was trivial enough to be considered as deviation (40 vs. 1000 mm, 4%) and (2) the force requested by translation was much lighter than the one requested by wrist rotation. By magnifying the rotation effect through linear projection, the WFMMD can well suppress the influence of the inner frame’s translation to its rotation, which largely improves the detection quality. Through the image processing, we can obtain each motion component (supination, flexion, or ulnar deviation) of the 3-DOF wrist movements without any complex decoupling algorithms. This kinematic information can be directly used to describe the multi-DOF force output by the wrist, on account of the stable force-motion scheme provided by the WFMMD. In this paper, we do not try to detail the relationship, especially the non-linear part, between the wrist’s force and the cursor’s motion, because an intelligent machine with appropriate learning capability (like ANN, CNN etc.) can well address this issue. Further experiments were conducted to directly evaluate the relationship between the EMG signals and the cursor’s movements, as well as to find whether the 3-DOF wrist motion control is intuitive or not.

2.1.3 Motion detection

For detecting the 2D position (translate along X-axis and Y-axis) and 1D posture (rotate along Z-axis) of the cursor, image processing methods were applied. The resolution of the camera (avA1000-12kc, Basler) is 1024 × 1024 and the update frequency can reach 120 fps. In addition, the exposure time of the camera is adjustable and can be tuned according to the environment. In practice, the exposure time was set to 5 ms for decreasing the brightness of the background, so that clear pictures of the cursor could be obtained. The camera communicated with the frame grabber (Xcelera-CL_PX4, DALSA) through CameraLink, and the communication frequency was set to 65 MHz. The CameraLink could enlarge the clock frequency seven times and contained four differential signals for transmission. These all reduced the time needed for transferring the picture (14 ms/frame).

While detecting the position of the cursor, the applied wavelength of the laser was 650 nm and the color was red. Besides, the environmental brightness was reduced and the exposure time of the camera was shortened. Thus, only the red channel of the picture was able to collect light while the other channels were nearly black, as shown in Fig. 6. After image binarization, segmentation, and erosion, the key points of the two lines constituting the cursor were well extracted. The Hough transform [18] was applied to distinguish which line these points belong to. Through the least square fitting, the equations of these two lines can be obtained, so as the 2D position and 1D posture of the cursor. The steps of the image processing were detailed in Fig. 6. The resolution of the position can reach sub-pixel and the resolution of the posture can reach 0.02°. Our algorithm first tried to directly call the functions within the OpenCV library, which needed at least 25 ms that did not fit to the control frequency (at least 50 Hz). The image was a high contrast picture of the cross cursor with a black background. Thus, the steps of image binarization and segmentation were combined within traversing the pixels of the picture. Besides, during the step of image erosion, only the key points of the cursor were extracted and saved in the array. In the steps of Hough transform and least square fitting, the coordinates of these core points could be directly acquired from the array without searching the picture. After the optimization, the detection of the position and posture could be completed within 5 ms for one frame picture.

Fig. 6

The flowchart of image processing. Through controlling the light of environment, the background is nearly black. The Hough transform step detects the line in low accuracy and determines which line these points belong to. Then, the position and posture of the laser cursor could be calculated exactly through the least square fitting

By using this method, the calculated cursor’s position and posture inevitably have deviations with their real targets. Thus, the results needed calibration to establish a reliable relationship with the actual cursor. This calibration was accomplished after setting up the camera. At first, the window, within which the predicted cursor was displayed, was fixed during the experiment. This window needed to firstly display a chessboard for the camera to detect. Then, the border of the chessboard was detected and an affine transformation between the border of the displayed chessboard and the window was established. The predicted cursor indicating the calculated result should rapidly follow the movement of the cursor projected by the laser after calibration.

After optimization, the predicted cursor delayed nearly 30 ms (exposure time, picture transference, and image processing) with relative to the laser cursor. In addition, since the display frequency of the projector was 60 Hz, the predicted cursor could not be displayed on the screen at once. Thus, the delay between the displayed cursor and the actual cursor would be longer. But when saving the motion information into offline datasets, each frame was added with a label indicating on which time the exposure signal was sent. The whole program applied the structure of assembly line. The sampling frequency was determined by the longest part of the assembly line and could reach 50 Hz stably. This parameter exceeds the natural, distinguishable frame rate of human eyes, and satisfies the demand of the EMG control.

By using the WFMMD, the wrist movements were sampled simultaneously with the EMG signals that provided reliable samples with supervised labels for training the model. An intuitive online control experiment could be achieved by employing this platform.

2.2 Subjects, data collection, and algorithm

For testing the stability of the relationship between the wrist motion detected by the platform and the EMG signals, the experiment on simultaneously sampling the EMG signals and the wrist motion was conducted.

A total of eight wireless surface EMG electrodes (Trigno, Delsys INC, USA) were applied in the experiment. Those electrodes were arranged around the forearm, approximately 1/3 of the forearm length near the olecranon. The sampling frequency was set to 1926 Hz. The acquisition program was compiled in VS2010, which also contained the detection and display of the cursor, synchronous sampling, and storage of the EMG signals and wrist motion. The analysis of the data was performed in MATLAB (Version 8.0, Dual Core 3.4GHz, 8GB RAM). The EMG signals were firstly filtered by a fourth-order Butterworth band pass filter (20–500 Hz) and a notch filter (50 Hz). The feature selected was the root mean square (RMS) of each channel with a moving window of 200 ms. The overlap was configured to 20 ms, according to the sampling rate of the wrist motion (50 Hz). The support vector regression was applied to establish the models, and each DOF had an independent support vector regression (SVR) model [19]. The input of the SVR model was the eight RMS features, and the features were normalized along each channel before feeding into the model. The raw data of wrist flexion/extension and abduction/adduction motions used the pixel as the unit, while the raw data of wrist supination/pronation motion used the rotational degree (angle) of the cursor. Both types of the raw data were normalized into the same scale before use. Then, the wrist motions were selected as the output data for the SVR models of each DOF. The SVR algorithm was realized by the libsvm package [20]. The parameters were selected according to the grid search method (C = 16, gamma = 4). The kernel of radial basis function (RBF) was selected to map the RMS feature into infinity space. It has the capability to describe the mapping from the multi-channel EMG signals to the wrist forces and then to the 3-DOF cursor movements. The stability of this model was tested in this paper.

Six subjects were invited to participate in this experiment (age 26.0 ± 1.2 years, girth of forearm 25.0 ± 0.7 cm). Four subjects placed the electrodes on their right hands, another two placed electrodes on their left hands (non-dominant hand). The subjects needed to move their wrists according to the required cursor movements listed in the Fig. 7. The rhythm of the movement was determined according to their own habits. However, the speed could not be too fast so that the subjects could observe the state of the cursor clearly and control it exactly at any time. The frequency of the movement was about 0.2–0.5 Hz. Before the experiment, subjects were given enough time to be familiar with the system. Each trial of the movement lasted 10 s, wherein 500 samples with labels were collected. In a data-collection session, 20 trials of different movements were performed and an enough rest was given between two trials to avoid fatigue. Each session cost nearly 20 min, and the rest between two sessions needed about 10 min. The subjects needed to avoid excessive exercise during the rest to get rid of electrode shifting. Each subjects collected six sessions during the experiment. The 20 trials of movements within a session were divided into two parts according to their implementation difficulty, as shown in Fig. 7. The first ten movements were the 1-DOF movements and 2-DOF movements, which had high repeatability for the subjects to perform. For these movements, the subjects were requested not to actuate the third DOF deliberately. The other ten movements were the free movements of all three wrist DOFs. The subjects were required to activate all three DOFs simultaneously without a specified track. These movements expanded the workspace of the wrist and increased the motion’s detection complexity.

Fig. 7

Diagrammatic sketch of the movements. (a–j) were the ten regular movements, within, (a–d) were the 1-DOF movements, (e–j) were the 2-DOF movements, (k) was the sample of the free movement, subjects were required to activate all three DOFs simultaneously without a specified track

The performance of SVR was evaluated by the coefficient of determination (R²),

$$ {R^2}_i=1-\frac{\sum \limits_{t=1}^N{\left({p}_i\left(\mathrm{t}\right)-{f}_i(t)\right)}^2}{\sum \limits_{t=1}^N{\left({f}_i(t)-\overline{f_i(t)}\right)}^2} $$

(5)

where, f_i(t) is the detected motion of the ith DOF, p_i(t) is the motion of ith DOF predicted by the SVR model, R²_i is the determination coefficient of the ith DOF, and N is the number of the samples in discussion.

In this paper, we attempt to evaluate the stability and repeatability of the SVR method on mapping the EMG signals to wrist motion. The training data of the model is gradually increased and the diversification of the results is analyzed. We try to find whether and what wrist motions are repeatable, in terms of regression accuracy, by giving enough training data. The 10-fold cross-validation was utilized for evaluating the performance. All the samples were divided into ten groups randomly. The SVR models were trained by only one group data at first and then validated using the remaining groups. The number of the training groups was gradually increased (from 10 to 90%) in the following performance evaluations.

3 Results

As mentioned before, the wrist motions conducted in the experiments included both regular movements (the first ten) and free movements (the last ten). In general, the regular movements had higher repeatability and better classification performance than the free movements. Thus, in this paper, two types of models were trained, as one learned of regular movements (denoted as the regular model) and the other learned from all movements (denoted as the free model, hereafter), respectively. For the last model, the wrist movements were much more voluntary and the working space for these free movements was larger than the one for regular movements. The results on the regression accuracy and the ratio of support vectors of different models are shown in Figs. 8 and 9, respectively. The regression target, either regular movements or free movements, and their regression results are shown in Fig. 10.

Fig. 8

The regression result R² on the two different models (regular and free). Note that the diameters of the bubble indicate the size of the standard deviations of the R²

Fig. 9

The ratio of the support vectors of two different models (regular and free)

Fig. 10

The measured and detected wrist motions of a representative subject. The subjects were required to move their wrist in a comfortable range. Due to the simultaneous movement of 3-DOF is difficult to plan, we required subjects to activate all three DOFs freely instead

From Fig. 8, it shows that, as the percentage of the training data increases (10–90%), the regression accuracy (R²) increases and the standard deviation decreases. For the regular model, the best regression results (R², the percentage of the training data all needs to reach 90%) can arrive at 0.884 ± 0.052, 0.891 ± 0.040, and 0.883 ± 0.028, on wrist flexion/extension, abduction/adduction, and supination/pronation, respectively. When including the other ten free movements, the corresponding results obtained by the free model can reach 0.871 ± 0.030, 0.867 ± 0.027, and 0.859 ± 0.034, respectively. Only the regression result on wrist supination/pronation is slightly different when comparing these two models (p < 0.01). Compared with the other studies [10, 12, 15, 16], our results show that, there is no significant difference (p > 0.1) on the regression accuracy even when increase the control DOFs from two to three. In the Fig. 10, the measured motion could rapidly follow the transformation of the detection. The errors mostly exist in the passive DOFs, and the measured motion of the passive DOF could be influenced by the movements of other DOFs.

In SVR, the support vectors represent the similarity of dataset and describe the inner structure of the model in high-dimension space. To inspect the complexity of the SVR model, we examined the ratio of the support vectors over its total training samples (RSV). From Fig. 9, the number of the support vectors increases, but the RSV (both average and standard deviation) decreases, along with the percentage of the training data. It clearly indicates that, by adopting more training sessions, the repeatability of these wrist motions can be improved. The decreasing speed of the RSV tends to become moderate at the last several growths of the training data, indicating that there may exists an optimized training dataset on comprehensively considering the training’s completeness, the subject’s learning curve of the system’s usability. On the other hand, the RSVs obtained by the regular model are 27.5% ± 2.0%, 29.3% ± 1.6%, and 21.8% ± 4.0% for wrist flexion/extension, abduction/adduction, and supination/pronation, respectively, which are fewer than the ones obtained by the free model, as 37.8% ± 2.9% (p < 0.001), 38.9% ± 2.9% (p < 0.001), and 33.5% ± 6.4% (p < 0.005). This result indicates that the complexity of the free model may increase sharply with complex wrist movements, possibly the 3-DOF ones. There would be some intrinsic features existing in the 3-DOF wrist movements that cannot be learned from simple 2-DOF motions that a specialized device needs to be developed to fully exhibit the strength of the simultaneous control.

4 Discussions

The platform proposed in this paper bridges the multi-channel EMG signals and multi-DOF wrist movements, which can be used to leverage the study on simultaneous EMG control. The wrist force-movement mapping device keeps a stable relationship between the 3-DOF wrist force and movement. The laser installed on the device gives an intuitive visual feedback to the subjects while collecting the wrist motion. After implementing an algorithm, like SVR, for predicting these motions in real-time, online control experiments can be tested in this platform. The performance on predicting various wrist motions, including the 3-DOF ones, could reach the similar level as in other studies. On the screen, the projector can display the cursor (wrist movement) that detected by the machine vision, as well as, the one predicted by the EMG signals synchronously. The subjects can control the detected cursor (or the predicted cursor, two different protocols) through adjusting their wrist movements (or EMG signals) to track a given target or trajectory and then evaluate the control performances online. On comparing different approaches implemented on the platform, a stable model for predicting the wrist motions could be established by giving the superior performance. Once this mode had been well built, it could be work in a more natural, free wrist condition, that is, no external forces are applied anymore. To output similar EMG signals, whether different wrist conditions (hemi-constraint and non-constraint) do matter in the simultaneous EMG control will be fully examined in our future study.

According to the feedback from the subjects, the wrist supination/pronation needs more strength to achieve compared with the other two movements. This phenomenon is caused by the difference on force arms. For the wrist flexion/extension and abduction/adduction, the force arm is the length between the handle and the rotational center; while for wrist supination/pronation, the force arm is nearly the half width of the palm that is shorter than the other two. Thus, to rotate the inner frame with the same angle, the wrist supination/pronation needs more force. In other words, the same force output by wrist supination/pronation only give a smaller motion range comparing to the other two. However, the information relating to the wrist supination/pronation is the rotational degree of cursor. The unit of this DOF is different with the other two and does not need to plan a specific range of the movement. Besides, the detection resolution (< 0.02°) is good enough for catching any small changes of the movement. On this DOF, it was not necessary for subjects to reach the same amplitude like the other two DOFs during the experiment. The subjects were only requested to try different combinations of various wrist DOFs without fatigue.

Because the springs of large stiffness were used, the maximum rotating angle of the inner frame under a reasonable wrist force is relatively small (15–20°). This value could be enlarged by replacing the springs with a smaller stiffness. However, the inner frame will interfere with the forearm if a large rotating angle is given. Besides, the distance between the laser and the screen should be shortened; otherwise, the cursor of the laser would exceed the screen. The short distance will decrease the detection resolution of the movements especially the wrist supination/pronation. We will consider redesigning the structure of the inner frame to enlarge the rotational angle for providing more free moving space for the wrist.

From the results, we can also conclude that the performance of the predicting model generally improves as the training data increases. Meanwhile, the standard deviation of the coefficients and the ratio of support vectors are reduced. When the percentage of the training data is low, the training data for establishing the model cannot fully cover the feature space. Those data cannot well describe the wrist motions and the prediction performance is largely affected by the subjects’ experiences. When increasing the training data gradually, more detailed information about the wrist motions could be learned thus the prediction performance improves. Besides, the decreasing standard deviation indicates that the divergence of performance caused by individual experiences reduces at the same time. When the percentage of training data reaches 70%, the performance improvement will become unclear (lower than 0.02, p > 0.1). This trivial promotion may also be influenced by the decreasing of the testing data. The regression accuracy may seem low comparing with the accuracy of gesture recognition, but the measured motion in the Fig. 10 could describe the major information of the detected motion. Increasing the percentage of training data will decrease the ratio of support vectors clearly, while increase the number of support vectors slightly. This phenomenon indicates that the wrist motions can be well described by the model learned from sufficient training samples, and the relationship between the EMG signals and wrist motions established by the WFMMD is stable.

When the training data is insufficient (< 50%), the predictive model trained with regular movements performs better than the one trained with free movements (p < 0.05). The regular movements are easily repeated and the feature space is easier to be covered, as shown by the lower ratio of the support vectors. Besides, the ratio of support vectors of the wrist supination/pronation is lower than the ones of the other two DOFs. As shown in Fig. 10, the wrist supination/pronation is less complex than other motions. It is because these movements are related to the posture of the cursor, and it is difficult for subjects to plan these motions together with the position of cursor. Meanwhile, the wrist supination/pronation needs relatively more force to perform that the subjects often attempted to avoid changing this activity. However, the gap of the performance achieved by different models (regular and free) is decreasing while increasing the percentage of the training data. With sufficient training data, free wrist movements of all its three DOFs could also obtain equivalent regression accuracy like the regular movements do. The motion component of each DOF can be well decoupled from the free movements and keep a similar relationship with the EMG signals like the regular movements do. On this platform, the established relationship between EMG signals and wrist movements is repeatable and seldom influenced by the motion types. Ultimately, the platform proposed in this paper presents an available device for establishing the map between wrist’s 3D force and movement, as well as, interfacing the multi-channel EMG signals together with the multi-DOF wrist movements.

To be frank, there is still a long way from the offline results to its online practice of the simultaneous control. If the training samples are arranged according to their chronological sequence in the experiment, the regression performance of the model will be worse. The feature employed in this paper is the RMS with a 200 ms moving window. The information contained in the feature may not be rich enough for detailing the motion characteristics. Our future research will concentrate on other features, like the wavelet coefficient in the time-frequency domain, for abstracting the information within the EMG signals for multi-DOF wrist motion decomposition.

5 Conclusions

In this paper, a novel platform is proposed to provide multi-channel EMG samples with explicit labels, that is, the multi-DOF wrist force target is intuitively expressed as the position and posture of a visible cursor. The subjects can plan multi-DOFs wrist movements according to the position and posture of the cursor, and the controlling experience is more intuitive. To achieve this, the wrist is on a hemi-constraint condition that is more similar to its nature situation compared with the traditional non-constraint or fully constraint ones. The data collected by using this platform has been tested by a cross-validation experiment on the SVR model. The model learned from the training data collected by the platform can well predict the arbitrary combinations of 3-DOF wrist movements, and the result shows stable and repeatable.

In the future work, we will put our study forward the online control scenario, including new features, predictive models, and training protocols in the simultaneous EMG control. Metrics for evaluating the online control performance should be also established.