Abstract
Predicting and accurately classifying intentions for human hand gestures can be used not only for active prosthetic hands, rehabilitation robots and entertainment robots but also for artificial intelligence robots in general. In this paper, first of all, source data of three hand gestures of grasping and three hand gestures of sign language are acquired by using the armband combined with eight sEMG (surface Electromyography) sensors. To classify these hand gestures, basically simple CNN (convolutional neural network) models with raw data, short-time Fourier transform (STFT), wavelet transform (WT), and scale average wavelet transform (SAWT) are applied, and their performances are compared. Finally, it is shown that by using a CNN with SAWT images, the accuracy can be improved up to 94.6% for selected hand gestures with higher accuracy and lower computational burden than conventional multi-channel STFT or WT.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
U. C. Allard, C. L. Fall, A. Drouin, A. C. Lecours, C. Gosselin, K. Glette, F. Laviolette, and B. Gosselin, “Deep learning for electromyographic hand gesture signal classification using transfer learning,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 27, no. 4, pp. 760–771, 2019.
K. Asai and N. Takase, “Finger motion estimation based on frequency conversion of EMG signals and image recognition using convolutional neural network,” Proc. of 17th International Conference on Control, Automation and Systems (ICCAS), Oct. 18–21, Jeju, Korea, 2017.
M. G. B. Fonseca, A. G. S. Conceicao, and E. F. S. Filho, “Artificial neural networks applied to the classification of hand gestures using electromyo- graphic signals,” Proc. of Latin American Robotics Symposium (LARS) and 2017 Brazilian Symposium on Robotics, Curitiba, Brazil, IEEE 2017.
D. Na, L.-Z. Liu, X.-J. Yu, Q. Li, and S.-C. Yeh, “Classification of multichannel surface-electromyography signals based on convolutional neural networks,” Journal of Industrial Information Integration, vol. 15, pp. 201.206, Sept. 2019.
I. Batzianoulis, S. El-Khoury, E. Pirondini, M. Coscia, S. Micera, and A. Billard, “EMG-based decording of grasp gestures in reaching-to-grasping motions,” Robotics and Automation Systems, vol. 91, pp. 59–70, Jan. 2017.
J. Luo, C. Liu, and C. Yang, “Estimation of EMG-based force using a neural-network-based approach,” IEEE Access, vol. 7, pp. 64856–64865, 2019.
A. Boyali and N. Hashimoto, “Spectral collaborative representation based classification for hand gestures recognition on electromyography signals,” Biomedical Signal Processing and Control, vol. 24, pp. 11–18, 2016.
M. Atzori, M. Cognolato, and H. Muller, “Deep learning with convolutional neural networks applied to electromyography data: A resource for the classification of movements for prosthetic hands,” Frontiers in Neurorobotics, vol. 10, no. 9, Sept. 2016.
P. Xia, J. Hu, and Y. Peng, “EMG-based estimation of limb movement using deep learning with recurrent convolutional neural networks,” Artificial Organs, vol. 42, no. 5, pp. 67–77, Oct. 2017.
H. Alaskar, “Deep learning of EMG time-frequency representations for identifying normal and aggressive actions,” International Journal of Computer Science and Network Security, vol. 18, no. 12, Dec. 2018.
Y. U. Jo and D. C. Oh, “Study on the EMG-based grasp gesture classification using deep learning and application to active prosthetics,” Journal of Institute of Control, Robotics and Systems (in Korean), vol. 25, no. 3, pp. 229–234, 2019.
G. Ruffini, D. Lbanez, M. Castellano, L. D. Vall, A. S. Frisch, R. Postuma, J. F. Gagnon, and J. Montplaisir, “Deep learning with EEG spectrograms in rapid eye movement behavior disorder,” Frontiers in Neurology, vol. 10, Article 806, July, 2019.
S. Chauhan and L. Vig, “Anomaly detection in ECG time signals via deep long short-term memory networks,” Proc. of IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 1–7, 2015.
L. Yuan and J. Cao, “Patients’ EEG data analysis via spectrogram image with a convolution neural network,” Proc. of International Conference on Intelligent Decision Technologies, pp. 13–21, 2017.
A. Vilamala, K. H. Madsen, and L. K. Hansen, “Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring,” Proc. of IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), 2017.
H. Chen, P. Zhang, H. Bai, Q. Yuan, and Y. Yan, “Deep convolutional neural network with scalogram for audio scene modeling,” Interspeech 2018, pp. 3304–3308, Hyderabad, India, 2018.
J. Kilby and K. Prasad, “Continuous wavelet analysis and classification of surface electromyography signals,” International Journal of Computer and Electrical Engineering, vol. 5, no. 1, February 2013.
A. Strazza, F. Verdini, L. Burattini, S. Fioretti, and F. D. Nardo, “Time-frequency analysis of surface EMG signals for maximum energy localization during walking,” EM-BEC & NBC 2017, IFMBE Proceedings, vol. 65, 2017.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Recommended by Associate Editor Jong-Han Kim under the direction of Editor Doo Yong Lee. This paper was supported by the Konyang University Research Fund in 2019.
Do-Chang Oh received his B.S., M.S., and Ph.D. degrees in electronics from Kyungpook National University in 1991, 1993, and 1997, respectively. He was with the University of Florida as a Courtesy Associate Professor for one year from July 2007. He is currently a professor at the School of Biomedical Engineering, Konyang University. His research interests include robust control, model reduction, rehabilitation robot, and biomedical applications of deep learning.
Yong-Un Jo received his B.S. degree in biomedical engineering from Konyang University in 2019. He is currently studying for a master’s degree at Konyang University. His research interests include ANN(artificial neural network), rehabilitation robot, and biomedical applications of deep learning.
Rights and permissions
About this article
Cite this article
Oh, DC., Jo, YU. Classification of Hand Gestures Based on Multi-channel EMG by Scale Average Wavelet Transform and Convolutional Neural Network. Int. J. Control Autom. Syst. 19, 1443–1450 (2021). https://doi.org/10.1007/s12555-019-0802-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12555-019-0802-1