Abstract
Studies on brain-machine interface techniques have shown that electrocorticography (ECoG) is an effective modality for predicting limb trajectories and muscle activity in humans. Motor control studies have also identified distributions of “extrinsic-like” and “intrinsic-like” neurons in the premotor (PM) and primary motor (M1) cortices. Here, we investigated whether trajectories and muscle activity predicted from ECoG were obtained based on signals derived from extrinsic-like or intrinsic-like neurons. Three participants carried objects of three different masses along the same counterclockwise path on a table. Trajectories of the object and upper arm muscle activity were predicted using a sparse linear regression. Weight matrices for the predictors were then compared to determine if the ECoG channels contributed more information about trajectory or muscle activity. We found that channels over both PM and M1 contributed highly to trajectory prediction, while a channel over M1 was the highest contributor for muscle activity prediction.
Similar content being viewed by others
Introduction
Studies on brain-machine interface (BMI) have demonstrated success using electrocorticography (ECoG) to predict hand trajectory in monkeys1,2,3,4 and hand5,6,7,8 and finger9,10,11,12,13,14 trajectory in humans, decode joint angle15 and grasping force16 and estimate muscle activity17,18. Advances in ECoG electrodes have allowed for more precise prediction of human hand and finger movements19,20. Control of prosthetic limbs using ECoG has also been achieved21,22,23. Therefore, ECoG offers considerable potential in rehabilitation as a control modality for neuro-prostheses.
A topic of debate has arisen, however, on whether activity from the primary motor cortex (M1) and premotor cortex (PM) represents “extrinsic” parameters related to the location of a target, or “intrinsic” parameters related to a set of muscles24,25,26,27,28,29,30. For example, Kakei et al. showed that most neurons in the monkey ventral premotor cortex exhibited “extrinsic-like” encoding (~94%), while M1 neurons exhibited both “intrinsic-like” (~32%) and “extrinsic-like” (~61%) encoding29.
This disparity gives rise to further questions: When predicting hand and finger trajectories using ECoG, are the predictions based on extrinsic or intrinsic information? When estimating arm muscle activity, is only intrinsic information exploited? It is unlikely that ECoG activity can be divided into extrinsic and intrinsic components based on signal amplitude or other features, because the ECoG signal is a summation of signals from a huge number of neurons beneath an electrode. We expect that ECoG signals from PM and some parts of M1 would mainly result from firing of extrinsic-like neurons, while other signals from M1 would result from both extrinsic-like and intrinsic-like neurons firing.
Seeking answers to the above questions, here we performed trajectory and muscle activity predictions for object-carrying tasks designed to vary muscle activity but keep trajectory unchanged. We obtain decoders trained with ECoG signals of various object masses and compare spatial distributions of weight matrices for trajectory prediction with those for muscle activity prediction. If the predictor is constructed based on ECoG signals derived from both extrinsic-like and intrinsic-like neurons, predicted trajectories will stray from actual trajectories given objects of different mass. Conversely, if the weight matrix for muscle activity prediction is obtained based on signals derived from extrinsic-like neurons, the predicted muscle activity will not change with varying object mass. Similar results would be expected if trajectory were varied while muscle activities were kept constant. We also trained predictors using task data for a single object mass to observe the effects of object mass on prediction.
Materials and Methods
Ethical approval
The study protocol was approved by the ethics committee of Osaka University Hospital (Approval No. 14353, UNIM ID: UMIN000017900) and carried out in accordance with the Declaration of Helsinki. The ECoG electrodes in this study were implemented as part of the patients’ treatment. All patients or their guardians gave written informed consent before participating in the study.
Participants and electrodes
Three female patients aged 66, 11 and 23 years, hereinafter referred to as patients 1, 2 and 3, respectively, participated in this study. All patients were diagnosed with intractable epilepsy. The sensorimotor cortices of patients 1 and 2 were undamaged. Patient 3 had a cerebral congenital lesion on her right frontal lobe (green area in Fig. 1C). Subdural planar platinum electrode arrays (Unique Medical Co., Tokyo, Japan) covering the precentral and central sulcus were implanted in all patients to localize foci of epilepsy and functional mapping (Fig. 1A–C). Each electrode had an exposed surface diameter of 3 mm. Inter-electrode distance was 10 mm for patients 1 and 3, and 7 mm for patient 2. The electrodes were implanted for 9 days in patient 1 and 14 days in patients 2 and 3.
Behavioral tasks
Patients 1, 2 and 3 conducted behavioral tasks 6 days, 13 days and 10 days after implantation, respectively. Sitting upright on their hospital room beds, the patients performed carrying tasks with a plastic bottle on a table (Fig. 1D). Patients 1 and 3 handled the bottle with their right arms, and patient 2 used her left arm. Four points were marked on the table with masking tape (red and green rectangles in Fig. 1D). Patients put the bottle at a start point (left red marker) and carried it to an end point (right red marker) via a green marker on a path denoted as path 1 (blue arc). The X and Y coordinates (mm) of the bottle changed from (200, 0) to (0, 150) and then from (0, 150) to (−200, 0). Carrying height for the bottle was not specified, but patients were required to place it firmly on the red markers for an instant. After a brief stop on the end point of path 1, they were to continue carrying along path 2 (red arc). The path 1 and 2 carrying tasks were alternately repeated about 20 times per session. Bottles with three different masses were prepared: an empty bottle (25 g), a half-full bottle (250 g) and a full bottle (500 g). Each patient performed three sessions, one with each of the three bottle masses.
ECoG, EMG and motion recordings
Patient 1 was implanted with a 4 × 5 electrode array on the left cerebral hemisphere (Fig. 1A), patient 2 was implanted with a 5 × 6 and a 2 × 5 array on the right hemisphere (Fig. 1B) and patient 3 was implanted with a 4 × 5 array and two 2 × 5 arrays on the left hemisphere (Fig. 1C). A scalp electrode on the nasion of each patient was used as a reference for all electrodes. EMG signals of biceps and triceps brachii and anterior and posterior deltoid were recorded for all patients. ECoG and EMG were synchronously recorded with a 128-channel digital electroencephalography (EEG) system (Neurofax EEG 1200; Nihon Koden Corporation, Tokyo, Japan) at a sampling rate of 1000 Hz.
The 3D motion of the bottle was recorded with a Kinect for Windows sensor (Microsoft, Redmond, WA) at 30 fps, using the bottle’s cap as the reference point (Fig. 1D). Motion recording durations (average across three sessions ± standard deviation) and total number of trials across sessions with the 25-g, 250-g and 500-g bottles were as follows: 1.72 ± 0.17 s (40 trials), 1.53 ± 0.09 s (40 trials) and 1.52 ± 0.13 s (42 trials) for patient 1; 1.86 ± 0.29 s (51 trials), 1.99 ± 0.31 s (39 trials) and 1.91 ± 0.28 s (44 trials) for patient 2; and 2.97 ± 0.32 s (42 trials), 3.14 ± 0.4 s (40 trials) and 2.97 ± 0.33 s (42 trials) for patient 3; respectively. There were no significant differences in duration among weights (F2,371 = 0.25, p = 0.779, the two-way ANOVA with unweighted-means analysis).
To synchronize the Kinect images and ECoG/EMG signals, brief flashes of LED light were displayed at random intervals and recorded with the Kinect camera, while trigger signals of the flashes were recorded with the EEG system.
ECoG and EMG signal processing
We applied the following ECoG pre-processing methods, which we previously proposed. (1) Raw ECoG signals were re-referenced to a common average reference. (2) The re-referenced signals were decomposed into nine frequency bands with fourth-order bandpass Butterworth filters (δ: ~4 Hz, θ: 4~8 Hz, α: 8~14 Hz, β1: 14~20 Hz, β2: 20~30 Hz, γ1: 30~60 Hz, γ2: 60~90 Hz, γ3: 90~120 Hz and γ4: 120~150 Hz). (3) These component signals were full-wave rectified. (4) They were smoothed with a Gaussian window (cutoff frequency: 3.98 Hz, time constant: 0.04 s). (5) The smoothed signals were down-sampled to 100 Hz. (6) Standard z-score zi(t) were obtained by normalization of the signals xi(t) (i = 1, 2, …. n × 9) at time t, such that
where n. μi and σi denote the number of electrodes, the mean value of xi(t) and the standard deviation of xi(t), respectively. We also used this method in our previous studies to predict trajectory2,7, muscle activation17, and force18, though not in our 3D motion study14, which used only unrectified ECoG. In the current study, we used both rectified and unrectified features in the weight selection.
EMG signals were band-pass filtered from 30 to 400 Hz and full-wave rectified. The signals were smoothed with a second-order low-pass filter with a cut-off frequency of 2.2 Hz31,32 and down-sampled from 1000 Hz to 100 Hz.
Decoding method
The bottle position or muscle activity Yp(t) at a time t was predicted with sparse linear regression (SLiR)32,33,34,35 as follows.
where m denotes the number of sampling points for the z-score used for prediction of Yp, and Δt represents the time interval between sampling points. w0 and wij are a bias term and members belonging to a weight matrix. In this study, we assigned m = 100 points and Δt = 0.01 s. We then performed a leave-one-out cross validation (LOO-CV) to verify our results. Finally, we obtained root-mean-square error (RMSE) and Pearson’s correlation coefficient (CC) by comparing predicted Yp and actual values in LOO-CV.
Results
Actual bottle trajectories and muscle activity
Figure 2 shows actual bottle trajectories by patient 1 for all trials from the 25-g, 250-g and 500-g sessions (Fig. 2A–C) and averaged across trials after resampling each trial period to 200 points (Fig. 2D). Observed differences between sessions were slight, particularly in X coordinates. Actual trajectories for patients 2 and 3 are shown in Supplementary Figs S1 and S2, respectively. Supplementary Table S1 shows p and F values for a one-way ANOVA comparing coordinates for all trials and patients at 0.25 T, 0.5 T and 0.75 T, where T denotes each trial period. Patient 1 had the smallest differences among the three patients.
Figure 3 shows actual biceps brachii, triceps brachii and anterior deltoid activity (pre-processed EMG signals) for all trials (Fig. 3A,C) and averaged across trials (Fig. 3B,D). Although EMG of the posterior deltoid was also measured, the signals were highly uniform across trials and sessions. Among the three muscles, the biceps brachii showed the largest difference in activity across sessions. The p and F values for a one-way ANOVA comparing actual muscle activity for all trials and patients are shown in Supplementary Table S2.
Predicted bottle trajectories and muscle activity
Bottle trajectories were predicted for all trials, sessions and patients using sparse linear regression and LOO-CV. Prediction results were evaluated with RMSE and CC. We trained decoders based on subsets of ECoG and coordinate data from each session, as well as from all three sessions lumped together, and used them to predict X, Y and Z coordinates. Figure 4A shows RMSE between predicted and actual trajectories. Note that the proposed method can also estimate the complete opposite direction with the same decoders even without all lumped data. Predictions for patient 1 showed relatively low RMSE values and were typically the best among the three patients. Patient 1 also showed higher CC values than the others (Supplementary Fig. S3A).
Muscle activity was also predicted in the same way as trajectory prediction. Patient 1 showed the lowest RMSE values for anterior deltoid activity as well as biceps brachii activity with the 250 g and 500 g bottles (Fig. 4B).
X- and Y-coordinate predictions for patient 1 are shown in Fig. 5. Average predicted trajectories (Fig. 5D, thick lines) for the 250 g and 500 g bottles fitted well with actual trajectories (thin lines). Figs. 5E and 5F show spatial distributions of normalized weight matrices |wij |/|wmax| for the X- and Y-coordinate predictors, where |wmax| denotes the maximum among absolute values of all members |wij| in matrix [W]. For the X-coordinate predictor, channels labeled a, b and c for ECoG without full-wave rectification showed relatively high weight values. Weight averages for areas PM and M1 were not significantly different from each other (p = 0.829, t-test). Weight matrices for the Y-coordinate predictor showed relatively high values at channels c through g, and average weight for PM was significantly higher than that for M1 (p = 1.15 × 10−23).
Figure 6 shows predicted biceps brachii activity for patient 1. Significant differences between predicted muscle activities were observed (thick lines in Fig. 6D). The p and F values for a one-way ANOVA comparing predicted muscle activity for all trials and patients are shown in Supplementary Table S3 (cf. Table S2). Muscle activity for patient 1 typically showed significant differences according to bottle weight.
Distributions for weight matrix [W] are shown in Fig. 6E. Channels a, c, h, i and j for ECoG with full-wave rectification had high weights. Average weight for M1 was significantly higher than that for PM (p = 0.00916). Predicted triceps brachii activity also yielded significantly higher average weights for M1 than PM, but those of the anterior deltoid did not (Supplementary Fig. S4). Though it is not clear whether M1 had more information than PM, the highest contributor for muscle activity prediction was channel a in M1. This could mean that intrinsic-like neurons for generating movement are more prominent than extrinsic-like neurons under the area of channel a.
Weights and weight ratios of the frequency bands for XY coordinates and biceps brachii activity predictors are shown in Supplementary Fig. S5. Weight values using full-wave rectified ECoG were higher at all frequency bands for the muscle activity predictor than the trajectory predictor. This shows that the higher frequency bands had slightly more influence on muscle activity prediction than trajectory prediction. The δ weights using unrectified ECoG and the δ and θ weights using rectified ECoG were higher than any other band. The ratio of δ frequency bands (dark blue and blue) notably favored XY coordinates, while ratios for θ and γ bands favored muscle activity. Particularly the γ bands, such as those for channels a and h, were greater in the M1 area.
Effect of bottle mass on trajectory and muscle activity predictions
We trained a decoder with complex data involving different masses and opposite trajectories to study its feasibility for neuroprosthetic control. We compared our decoder with a decoder designed for a single mass by applying weight matrices [W] trained using a single bottle mass to predict trajectories for all three masses. Figure 7A shows X and Y coordinates predicted using a decoder trained with a data set from patient 1 performing trials with the 25-g bottle (selected due to it having the lowest RMSE). Using the 250-g and 500-g decoders (Fig. 7B,C), prediction for 25 g (cyan/blue lines) showed deficient trajectory compared with actual trajectory. This implies that the weights of the single mass decoder included intrinsic information for the given mass. This mass-specific decoder could not generate trajectory for other conditions. Figure 7D and E show spatial distributions of weight matrices [W] used in the three mass-specific decoders. For the 25-g decoder with full-wave rectification, channel h showed higher weight values than those of the 250-g and 500-g decoders. Since many intrinsic- and extrinsic-like neurons exist under the area of channel h, intrinsic-like neurons may have influenced the training for trajectory decoding.
Predicted biceps brachii activity for patient 1 using one of the decoders obtained with LOO-CV in each session is shown in Supplementary Fig. S6. Although predicted curves slightly differed from each other, they did not fit well with actual muscle activity as compared with the decoders in Fig. 6. Triceps brachii and anterior deltoid activity of patient 1 predicted with mass-specific decoders are also shown in Supplementary Fig. S7.
We also sought to determine how much channels with high weights contributed to prediction. We predicted biceps brachii activity for patient 1 with one of the decoders obtained after LOO-CV using data for all three masses and two paths to determine the difference in predicted muscle activity for the 25 g and 500 g bottles (Fig. 8A for path 1 and Fig. 8F for path 2). The difference from 0.4 T to 0.6 T was averaged (green area in Fig. 8A and F). To evaluate contribution of each channel, we also predicted muscle activity again using ECoG signals of only one channel and obtained average differences between predicted muscle activities for the 25 g and 250 g bottles at each channel (for example, Fig. 8C–E for path 1, Fig. 8H–J for path 2). Figure 8B and G express percentages of the average difference for each channel compared to those in Fig. 8A and F. Channel h showed the highest contribution rate.
We also calculated contribution of each channel to trajectory prediction for the 500 g bottle (Supplementary Fig. S8). Predicted trajectories for paths 1 and 2 were compared (green area). Channels a and b in M1, and c, e and f in PM showed high contribution rates.
Discussion
We used sparse linear regression to predict both bottle trajectory and muscle activity based on ECoG signals. To vary intrinsic parameters for muscle activity while keeping extrinsic parameters unchanged, we asked our patients to carry bottles of different masses along specified paths. Predicted trajectories and muscle activities were evaluated with RMSE and CC. To reduce stress and discomfort, recordings were performed in short durations and in their hospital rooms rather than an electromagnetically shielded room. Their postures were also unstable, because they sat on their beds. In addition, patient 1 had only 20 channels in her electrode array around the sensorimotor cortex (Fig. 1A). However, patient 1 typically showed the best RMSE and CC values (Fig. 4 and Supplementary Fig. S3) among the three patients. Patient 2 was not always able to keep the bottle along the specified path (see left three columns in Supplementary Fig. S1) and changed how she supported it in each session. We surmise that she was learning to carry it using as little energy as possible during the course of the three sessions. This learning likely made our prediction difficult. Although the results of patient 2 showed similar tendencies to those of patient 1, the features were less clear (Supplementary Figs S9 and S10). Patient 3 had the lowest RMSE and CC values among the three patients for trajectory prediction. One possible explanation for this is the trajectory patient 3 applied. She tried to pause at the via-points every trial (Supplementary Fig. S2). This complex movement may have affected decoding accuracy. Another explanation might be the site of patient 3’s brain lesion. Patient 3 had a cerebral congenital lesion near PM and M1, which might have influenced the ECoG. Therefore, results for patient 3 were difficult to interpret (Supplementary Figs S11 and S12).
For our decoders, we used both rectified and unrectified features in the weight selection. We found that using both provided the highest decoding performance (Supplementary Fig. S5). At the least, all rectified frequency bands and unrectified low frequency bands are necessary to obtain satisfactory performance.
Previous studies reported that weight values from δ and γ bands were effective and informative in trajectory decoding36,37,38. Crone et al. showed that high γ activity in response to a wide variety of experimental tasks was useful as a general electrophysiological index of cortical processing. Hammer et al. showed that the δ and high γ bands were superior to other bands for the prediction of direction, position, velocity, and acceleration. They further proposed the neuronal population activity model and stated that large neuronal population activities such as those of local field potential and ECoG provided better prediction performance than single or multi-unit activity. Pistohl et al. reported that the accuracy of grasp-type classification using power spectrums in high (75~170 Hz) and low (<4 or 5 Hz) frequency bands was greater than that using intermediate frequency bands (6~15 Hz and 17~40 Hz). In the current study, weights for the δ, θ and γ bands were higher than those of the other bands. This result is consistent with previous studies decoding for arm movement7, finger movement14, velocity37, and grasp type38.
Weight values for the γ bands tended to influence decoding for muscle activity, while those of the unrectified δ band influenced decoding for XY coordinates (Supplementary Fig. S5). In particular, weight values of γ bands from channels a and h (M1 area) were considerably higher for muscle activity decoding compared with XY-coordinate decoding. This finding matches well with previous studies on high δ contribution to movement7,14,37 and high γ contribution to muscle activity17 and object weight38.
Since the predicted results were the sum of the product of the frequency features and their weights for both XY-coordinate and muscle activity decoding, components with higher weight values were considered more informative for a specific type of decoding. This is consistent with the conceptual schematic model36 and the neuronal population activity model37. When neurons in PM and M1 spike for motor planning or motor control, diverse changes in neuronal population activities occur, according to the balance of excitatory and inhibitory synaptic input. These changes may be represented in the frequency bands of local field potential and ECoG. This could be why all frequency bands were needed for effective prediction, and also emphasizes the importance of selecting ECoG electrodes of appropriate size.
The decoder weights trained with each path showed similar patterns for X and Y coordinates (Supplementary Fig. S13). The highest decoder channels a and b for path 1 were located near each other. Channel l decoders for path 2 were located in the same area. However, weights for the decoder trained on all bottle masses and paths showed different distributions (Fig. 5E,F). It was for this reason that the high weights in both paths, such as channel a, stood out. Other weights, such as those for channel k, were low, as they could not satisfy the demand of the different trajectories. Consequently, the decoder weights trained with all data were more useful to our analysis than the task-specific weights.
Kakei et al. indicated that (1) neurons in M1 exhibited intrinsic-like properties, and (2) neurons in both the ventral PM and M1 exhibited extrinsic-like properties29. We compared products of temporal frequency features and weights to evaluate whether channels contributed more to XY-coordinate prediction or muscle activity prediction. Our analysis showed results consistent with those of Kakei et al. For example, the weight at channel h (upper part of M1) was much higher in muscle activity prediction (Fig. 6) than in trajectory prediction (Fig. 5). This could imply that channel h included a large amount of EMG information. Figures 8J and G showed that channel h gave the highest contribution compared to other channels. Therefore, this channel included the most information about the difference in muscle activity for the 25-g and 500-g bottles along path 2. Channel a (right side of channel h) had high weights for X coordinates in both paths (Supplementary Fig. S13). This channel also had high weights for muscle activity (Fig. 6E) along path 2, but not along path 1 (Fig. 8). Therefore, the area of channel a area may include strong extrinsic-like neurons for trajectory and intrinsic-like neurons for movement.
The spatial distribution of the weight matrix for X-coordinate prediction was not significantly high in PM (Fig. 5E). The differences among channels in contribution for the biceps brachii along path 1 were vague, although a few channels along path 2 had high contribution rates (Fig. 8B and G). Natural hand movement has been shown to be roughly straight with a gentle convex curve39,40,41. Uno et al. reported that the speed profile of convex curved paths had only one peak, while those of concave curved path had two peaks. They also reported that horizontal free movements between two targets were always slightly convex curved, because of the complex kinematics and dynamics of the musculoskeletal system. Indeed, the path 1 task was relatively easy for participants, as it involved a natural combination of joint movements, such as internal rotation of the shoulder joint, flexion of the elbow joint and so forth. To carry the bottle along path 2, patients had to draw the object up toward their chest while simultaneously carrying it from their left to right side. This motion is not as common in everyday life and may have forced patients to plan directions and routes before movement onset. These differences for paths 1 and 2 may have influenced the distributions in Fig. 5E and Fig. 8B.
From the viewpoint of neuroprosthetic control, it is ideal that trajectory and muscle activity prediction are based on extrinsic and intrinsic coordinates, respectively. Otherwise, the prosthesis would deviate from the user’s intended course when given an object of different mass, as shown in Fig. 7. When predicting muscle activity from ECoG, inaccuracies are usually inevitable because ECoG signals are summations of activity from a huge number of neurons under each electrode. Accuracy, however, could be improved by training decoders on ECoG recorded during manipulation of objects with various masses.
Additional Information
How to cite this article: Nakanishi, Y. et al. Mapping ECoG channel contributions to trajectory and muscle activity prediction in human sensorimotor cortex. Sci. Rep. 7, 45486; doi: 10.1038/srep45486 (2017).
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
Chao, Z. C., Nagasaka, Y. & Fujii, N. Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys. Front Neuroeng 3, 3, doi: 10.3389/fneng.2010.00003 (2010).
Chen, C. et al. Prediction of Hand Trajectory from Electrocorticography Signals in Primary Motor Cortex. PLoS ONE 8, e83534, doi: 10.1371/journal.pone.0083534 (2013).
Shimoda, K., Nagasaka, Y., Chao, Z. C. & Fujii, N. Decoding continuous three-dimensional hand trajectories from epidural electrocorticographic signals in Japanese macaques. Journal of neural engineering 9, 036015, doi: 10.1088/1741-2560/9/3/036015 (2012).
Eliseyev, A. & Aksenova, T. Stable and artifact-resistant decoding of 3D hand trajectories from ECoG signals using the generalized additive model. J Neural Eng 11, 066005, doi: 10.1088/1741-2560/11/6/066005 (2014).
Pistohl, T., Ball, T., Schulze-Bonhage, A., Aertsen, A. & Mehring, C. Prediction of arm movement trajectories from ECoG-recordings in humans. J Neurosci Methods 167, 105–114, doi: 10.1016/j.jneumeth.2007.10.001 (2008).
Gunduz, A., Sanchez, J. C., Carney, P. R. & Principe, J. C. Mapping broadband electrocorticographic recordings to two-dimensional hand trajectories in humans Motor control features. Neural Netw 22, 1257–1270, doi: 10.1016/j.neunet.2009.06.036 (2009).
Nakanishi, Y. et al. Prediction of three-dimensional arm trajectories based on ECoG signals recorded from human sensorimotor cortex. PLoS One 8, e72085, doi: 10.1371/journal.pone.0072085 (2013).
Bundy, D. T., Pahwa, M., Szrama, N. & Leuthardt, E. C. Decoding three-dimensional reaching movements using electrocorticographic signals in humans. J Neural Eng 13, 026021, doi: 10.1088/1741-2560/13/2/026021 (2016).
Miller, K. J., Zanos, S., Fetz, E. E., den Nijs, M. & Ojemann, J. G. Decoupling the cortical power spectrum reveals real-time representation of individual finger movements in humans. J Neurosci 29, 3132–3137, doi: 10.1523/JNEUROSCI.5506-08.2009 (2009).
Kubanek, J., Miller, K. J., Ojemann, J. G., Wolpaw, J. R. & Schalk, G. Decoding flexion of individual fingers using electrocorticographic signals in humans. J Neural Eng 6, 066001, doi: 10.1088/1741-2560/6/6/066001 (2009).
Acharya, S., Fifer, M. S., Benz, H. L., Crone, N. E. & Thakor, N. V. Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand. J Neural Eng 7, 046002, doi: 10.1088/1741-2560/7/4/046002 (2010).
Liang, N. & Bougrain, L. Decoding Finger Flexion from Band-Specific ECoG Signals in Humans. Front Neurosci 6, 91, doi: 10.3389/fnins.2012.00091 (2012).
Chen, W., Liu, X. & Litt, B. Logistic-weighted regression improves decoding of finger flexion from electrocorticographic signals. Conf Proc IEEE Eng Med Biol Soc 2014, 2629–2632, doi: 10.1109/EMBC.2014.6944162 (2014).
Nakanishi, Y. et al. Decoding fingertip trajectory from electrocorticographic signals in humans. Neurosci Res 85, 20–27, doi: 10.1016/j.neures.2014.05.005 (2014).
Benz, H. L. et al. Connectivity analysis as a novel approach to motor decoding for prosthesis control. IEEE Trans Neural Syst Rehabil Eng 20, 143–152, doi: 10.1109/TNSRE.2011.2175309 (2012).
Flint, R. D. et al. Extracting kinetic information from human motor cortical signals. Neuroimage 101, 695–703, doi: 10.1016/j.neuroimage.2014.07.049 (2014).
Shin, D. et al. Prediction of muscle activities from electrocorticograms in primary motor cortex of primates. PLoS One 7, e47992, doi: 10.1371/journal.pone.0047992 (2012).
Chen, C. et al. Decoding grasp force profile from electrocorticography signals in non-human primate sensorimotor cortex. Neuroscience Research 83, 1–7, doi: 10.1016/j.neures.2014.03.010 (2014).
Kellis, S. et al. Decoding hand trajectories from micro-electrocorticography in human patients. Conf Proc IEEE Eng Med Biol Soc 2012, 4091–4094, doi: 10.1109/EMBC.2012.6346866 (2012).
Hotson, G. et al. Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject. J Neural Eng 13, 026017, doi: 10.1088/1741-2560/13/2/026017 (2016).
Yanagisawa, T. et al. Electrocorticographic control of a prosthetic arm in paralyzed patients. Ann Neurol 71, 353–361, doi: 10.1002/ana.22613 (2012).
Morishita, S. et al. Brain-machine interface to control a prosthetic arm with monkey ECoGs during periodic movements. Front Neurosci 8, 417, doi: 10.3389/fnins.2014.00417 (2014).
Thakor, N. V. et al. Neuroprosthetic limb control with electrocorticography: approaches and challenges. Conf Proc IEEE Eng Med Biol Soc 2014, 5212–5215, doi: 10.1109/EMBC.2014.6944800 (2014).
Evarts, E. V. Contrasts between activity of precentral and postcentral neurons of cerebral cortex during movement in the monkey. Brain Res 40, 25–31 (1972).
Thach, W. T. Timing of activity in cerebellar dentate nucleus and cerebral motor cortex during prompt volitional movement. Brain Res 88, 233–241 (1975).
Georgopoulos, A. P., Kalaska, J. F., Caminiti, R. & Massey, J. T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J Neurosci 2, 1527–1537 (1982).
Kalaska, J., Cohen, D., Hyde, M. & Prud’homme, M. A comparison of movement direction-related versus load direction- related activity in primate motor cortex, using a two-dimensional reaching task. J. Neurosci. 9, 2080–2102 (1989).
Kakei, S., Hoffman, D. S. & Strick, P. L. Muscle and movement representations in the primary motor cortex. Science 285, 2136–2139 (1999).
Kakei, S., Hoffman, D. S. & Strick, P. L. Direction of action is represented in the ventral premotor cortex. Nat Neurosci 4, 1020–1025, doi: 10.1038/nn726 (2001).
Overduin, S. A., d’Avella, A., Roh, J., Carmena, J. M. & Bizzi, E. Representation of Muscle Synergies in the Primate Brain. J Neurosci 35, 12615–12624, doi: 10.1523/JNEUROSCI.4302-14.2015 (2015).
Koike, Y. & Kawato, M. Estimation of Dynamic Joint Torques and Trajectory Formation from Surface Electromyography Signals Using a Neural-Network Model. Biol Cybern 73, 291–300 (1995).
Yoshimura, N., Dasalla, C. S., Hanakawa, T., Sato, M. A. & Koike, Y. Reconstruction of flexor and extensor muscle activities from electroencephalography cortical currents. Neuroimage 59, 1324–1337, doi: 10.1016/j.neuroimage.2011.08.029 (2012).
Sato, M. Online model selection based on the variational bayes. Neural Computation 13, 1649–1681 (2001).
Ting, J. A. et al. Variational Bayesian least squares: an application to brain-machine interface data. Neural Netw 21, 1112–1131, doi: 10.1016/j.neunet.2008.06.012 (2008).
Nambu, I. et al. Single-trial reconstruction of finger-pinch forces from human motor-cortical activation measured by near-infrared spectroscopy (NIRS). Neuroimage 47, 628–637, doi: 10.1016/j.neuroimage.2009.04.050 (2009).
Crone, N. E., Korzeniewska, A. & Franaszczuk, P. J. Cortical gamma responses: searching high and low. Int J Psychophysiol 79, 9–15, doi: 10.1016/j.ijpsycho.2010.10.013 (2011).
Hammer, J. et al. Predominance of Movement Speed Over Direction in Neuronal Population Signals of Motor Cortex: Intracranial EEG Data and A Simple Explanatory Model. Cereb Cortex 26, 2863–2881, doi: 10.1093/cercor/bhw033 (2016).
Pistohl, T., Schulze-Bonhage, A., Aertsen, A., Mehring, C. & Ball, T. Decoding natural grasp types from human ECoG. Neuroimage 59, 248–260, doi: 10.1016/j.neuroimage.2011.06.084 (2012).
Morasso, P. Spatial control of arm movements. Exp Brain Res 42, 223–227 (1981).
Abend, W., Bizzi, E. & Morasso, P. Human arm trajectory formation. Brain 105, 331–348 (1982).
Uno, Y., Kawato, M. & Suzuki, R. Formation and control of optimal trajectory in human multijoint arm movement. Minimum torque-change model. Biol Cybern 61, 89–101 (1989).
Acknowledgements
We thank the participants for their efforts in this study. We also thank the clinicians and researchers at Osaka University Hospital for their contributions. This work was supported in part by the Japan Science and Technology Agency, Precursory Research for Embryonic Science and Technology (JST PRESTO), KAKENHI grants (26560467, 26282165, 15H05710, 15H01659, 16K01572), the Brain Mapping by Integrated Neurotechnologies for Disease Studies (Brain/MINDS) from AMED, Development of BMI Technologies for Clinical Application from the Strategic Research Program for Brain Sciences from AMED, and Development and application of an implantable device for brain-machine interfaces using high-speed wireless data transfer and big data analyses of neural information by the Commissioned Research of NICT.
Author information
Authors and Affiliations
Contributions
Y.K. and M.H. supervised the study and experiment design. H.Ki. performed operations on the patients. H.Ki. and T.Y. recruited the participants. T.Y. and D.S. designed the experiments. T.Y., M.T. and R.Y. executed the experiments. D.S. and Y.N. analyzed the data. Y.N. wrote the paper. H.Ka. and N.Y. provided conceptual advice regarding the experiments and results. All authors discussed the results and commented on the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Rights and permissions
This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/
About this article
Cite this article
Nakanishi, Y., Yanagisawa, T., Shin, D. et al. Mapping ECoG channel contributions to trajectory and muscle activity prediction in human sensorimotor cortex. Sci Rep 7, 45486 (2017). https://doi.org/10.1038/srep45486
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/srep45486
- Springer Nature Limited
This article is cited by
-
Why brain-controlled neuroprosthetics matter: mechanisms underlying electrical stimulation of muscles and nerves in rehabilitation
BioMedical Engineering OnLine (2020)
-
Cortico-Spinal Neural Interface to Restore Hindlimb Movements in Spinally-Injured Rabbits
Neurophysiology (2020)