Abstract
Human-robot control interfaces have received increased attention during the past decades for conveniently introducing robot into human daily life. In this paper, a novel Human-machine Interface (HMI) is developed, which contains two components. One is based on the surface electromyography (sEMG) signal, which is from the human upper limb, and the other is based on the Microsoft Kinect sensor. The proposed interface allows the user to control in real time a mobile humanoid robot arm in 3-D space, through upper limb motion estimation by sEMG recordings and Microsoft Kinect sensor. The effectiveness of the method is verified by experiments, including random arm motions in the 3-D space with variable hand speed profiles.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
M. Chen, C.-S. Jiang, and Q.-X. Wu, “Disturbanceobserver-based robust flight control for hypersonic vehicles using neural networks,” Advance Science Letter, vol. 4, no. 4, pp. 1771–1775, 2011
M. Chen, S. S. Ge, and B. How, “Robust adaptive neural network control for a class of uncertain MIMO nonlinear systems with input nonlinearities,” IEEE Trans. on Neural Networks, vol. 21, no. 5, pp. 796–812, 2010.
M. Chen, S. S. Ge, and B. Ren, “Robust attitude control of helicopters with actuator dynamics using neural networks,” IET Control Theory Appl., vol. 4, no. 12, pp. 2837–2854, 2010.
H. Li, M. Chow, Z. Sun, and F. Sun, “Gain scheduling based state feedback integral control for networked control systems,” IEEE Trans. on Industrial Electronics, vol. 58, no. 6, pp. 2465–2472, 2011.
H. Li, F. Sun, Z. Sun, and J. Du, “Optimal state feedback integral control using network-based measurements,” IEEE Trans. on Instrumentation and Measurement, vol. pp, no. 99, pp. 1–9, 2012.
H. Li, M. Chow, and Z. Sun, “EDA-based speed control of a networked DC motor system,” IEEE Trans. on Industrial Electronics, vol. 56, no. 5, pp. 1727–1735, 2009.
H. Li, Z. Sun, H. Liu, and F. Sun, “Stabilization of networked control systems using delay dependent control gains,” IET Control Theory & Applications, vol. 6, no. 5, pp. 698–706, 2012.
T. Felzer and B. Freisleben, “HaWCoS: the handsfree wheelchair control system,” Proc. of the ACM SIGCAPH Conference on Assistive Technologies, Edinburgh, Scotland, ACM Press, pp. 127–134, 2002.
K. Kiguchi, T. Tanaka, K. Watanabe, and T. Fukuda, “Design and control of an exoskeleton system for human upper-limb motion assist,” Proc. of IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 926–931, 2003.
P. Ju, L. P. Kaelbling, and Y. Singer, “State-based classification of finger gestures from electromyographic signals,” Proc. of the 17th International Conference on Machine Learning, San Francisco, USA, pp. 439–446, 2000.
B. Hudgins, P. Parker, and R. Scott, “A new strategy for multifunction myoelectric control,” IEEE Trans. on Biomedical Engineering, vol. 40, no. 1, pp. 82–94, January 1993.
K. Englehart, B. Hudgins, and P. A. Parker, “A wavelet-based continuous classification scheme for multifunction myoelectric control,” IEEE Trans. on Biomedical Engineering, vol. 48, no. 3, pp. 302–310, March 2001.
A. T. C. Au and R. F. Kirsch, “EMG-based prediction of shoulder and elbow kinematics in ablebodied and spinal cord injured individuals,” IEEE Trans. on Rehabilitation Engineering, vol. 8, no. 4, pp. 471–480, December 2000.
P. Shenoy, K. J. Miller, B. Crawford, and R. P. N. Rao, “Online electromyographic control of a robotic prosthesis,” IEEE Trans. on Biomedical Engineering, vol. 55, no. 3, pp. 1128–1135, March 2008.
S. Jamie, F. Andrew, C. Mat, S. Toby, F. Mark, M. Richard, K. Alex, and B. Andrew, “Real-time human pose recognition in parts from single depth images,” Proc, of the 24th IEEE Conference on Computer Vision and Pattern Recognition, pp. 1297–1304, January 2011.
D. Tkach, H. Huang, and T. A. Kuiken, “Study of stability of time-domain features for electromyographic pattern recognition,” Journal of Neuroengineering and Rehabilitation, vol. 7, no. 1, pp. 21, 2010.
OpenNI software, www.OpenNI.org.
Y. Kim and S. Yi, “Articulated body motion tracking using illumination invariant optical flow,” International Journal of Control, Automation, and Systems, vol. 8, no. 1, pp. 73–80, 2010.
G. Rodolfo and P. Vicente, “Task-space neurosliding mode control of robot manipulators under Jacobian uncertainties,” International Journal of Control, Automation, and Systems, vol. 9, no. 5, pp. 895–904, 2011.
B. Hudgins, P. Parker, and R. Scott, “A new strategy for multifunction myoelectric control,” IEEE Trans. Biomedical Engineering, vol. 40, no. 1, pp. 82–94, January 1993.
Author information
Authors and Affiliations
Corresponding author
Additional information
Recommended by Editor Shuzhi Sam Ge.
This work is supported by Natural Science Foundation of China under Grant Nos. 60804003, 61174045, 61111130208 and 60935001, and International Science & Technology Cooperation Program of China, No 0102011DFA10950.
Baocheng Wang received his B.S. degree in Automation from Qingdao University of Science and Technology, China, in 2010. From 2010, he has been with the Department of Automation, Shanghai Jiaotong University, China. Currently, he is working toward a Master degree. His research interests include the application of machine learning techniques to the understanding of EMG signals and control of bio-robot.
Zhijun Li received his Dr. Eng. degree in Mechatronics, Shanghai Jiao Tong University, P. R. China, in 2002. From 2003 to 2005, he was a postdoctoral fellow in Department of Mechanical Engineering and Intelligent Systems, The University of Electro-Communications, Tokyo, Japan. From 2005 to 2006, he was a research fellow in the Department of Electrical and Computer Engineering, National University of Singapore, and Nanyang Technological University, Singapore. From 2007–2012, he is an Associate Professor in the Department of Automation, Shanghai Jiao Tong University, P. R. China. He is serving as an Associate Editor of Journal of Intelligent & Robotic System. Dr. Li’s current research interests include adaptive/robust control, mobile manipulator, teleoperation system, etc.
Wenjun Ye received his B.E. degree in Naval Architecture and Ocean Engineering and his B.E. degree in Computer Application from Shanghai Jiao Tong University, China, in 2012. Currently, he is working with the Department of Automation, Shanghai Jiao Tong University, China. His study interests include the rehabilitation robot, adaptive control, biped robot, etc.
Qing Xie received her Bachelor of Medicine and Master Degree in Medical Rehabilitation, Hubei Medical University, P. R. China in 1987 and 1999, respectively. From 1987 to 2002, she was a Doctor in the Department of Rehabilitation Medicine of Renming Hospital, Wuhan University, P. R. China. From 2002 to now, she was a Chief Doctor in the Department of Rehabilitation Medicine of Ruijin Hospital, Shanghai Jiao Tong University, P. R. China. And she is the Director of the Rehabilitation Medicine and Physical Therapy Department of Rui Jin Hospital, attached to Medicine School of Shanghai Jiao Tong University. Dr. Xie has devoted to the neurological rehabilitation and orthopedic rehabilitation for 25 years, such as how to treat the paralysis of limbs or control the spasticity after Stroke. She is the both Editor of Chinese Journal of Physical Medicine and Rehabilitation and Chinese Journal of Rehabilitation.
Rights and permissions
About this article
Cite this article
Wang, B., Li, Z., Ye, W. et al. Development of human-machine interface for teleoperation of a mobile manipulator. Int. J. Control Autom. Syst. 10, 1225–1231 (2012). https://doi.org/10.1007/s12555-012-0617-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12555-012-0617-9