Keywords

1 Introduction

Human–machine interfaces and brain–machine interfaces are developing technologies with a bright future ahead of them. These devices have gained a lot of importance in recent years because they not only minimize human effort in various industries, but they also provide a great help for disabled people to work normally. Bio-signals produced by our bodies serve as the basis for these technologies. The most important bio-signals to remember are electroencephalography (EEG), electrooculography (EOG), electromyography (EMG) and electrocardiography. Paralyzed or disabled patients need specially designed medical equipment, such as EEG-, EOG- or EMG-controlled robotic hands, prosthetics and smart wheelchairs, that is controlled by these signals. An EEG brain headset is used to monitor a wheelchair, but it could only move in one direction using brain waves, while other directions are chosen using EOG. Advanced bio-sensors with multiple channels are used to achieve 2D regulation of the same. However, these bio-sensors are costly and may not be suitable for real-time applications, with accuracy being the primary concern. So, for this prototype, a smart wheelchair, which will be powered by a low-cost single channel EMG sensor is developed.

Few muscles are regulated actively and willingly by our will, i.e., voluntary muscles for voluntary acts, while others function involuntarily but play an important role in our survival. We normally focus on skeletal muscles in order to support and assist a disabled person. In a few special cases, such as quadriplegia, we must use other muscles such as facial muscles or anterior and posterior side muscles in the leg. Skeletal muscles are linked to the skeleton, which functions under the direction of the person and assists in the movement of various body parts. Muscle fibers in the skeleton are grouped together to form skeletal muscles. The nervous system sends signals to the muscles, which cause them to contract or relax. A muscle can generate enough force to shift the body when its fibers contract in unison, allowing us to move our hand. The EMG records these movement thresholds, which can be used to monitor a variety of devices depending on the application. The use of EMG in HMI is common because EMG recording can be done with surface electrodes and needs very little hardware and software resources, making it a suitable low-cost solution.

A smart wheelchair based on surface EMG signal is designed [1]. Cyber-link sensing device is used to obtain the forehead surface EMG signals generated by facial movements. The autoregressive model is used to compress the data and extract surface EMG features. Then, backpropagation artificial neural network (BPANN) improved by Levenberg–Marquardt algorithm is used to identify different facial movement patterns. The wheelchair makes simple movements like forward, left, right, backward and stop based on facial movements. But, face movements during talking and looking may cause adverse effects.

Innovative driving assistance system of wheel chair, for paralyzed people, is developed using laser dots [2]. A pan-tilt mounted laser is used to track user’s head posture for projecting a colored spot on the ground ahead. The wheelchair is provided with depth-camera to model a traversability map. The user can travel to the spot marked by laser pointer but that spot should also be present in the traversability map; otherwise, the wheel chair will not move and laser dot will glow in red color. If reachable, the laser pointer will turn from red to green color. The wheelchair is then controlled by micro-controller to follow the path to reach the destination.

In [3], the wheel chair is controlled by joystick. Initially, the joystick is in exact middle position so that the motor will not run. When the joystick is moved, the potentiometer sends analog values to Arduino board. Arduino converts it into digital signal. The digital signal is sent to the motor drive for controlling the DC motors. It makes the motors to move forward, backward, left or right based on the signal. This requires a person at least to move his forehand forcefully. Stroke patients’ mobility is limited as some of their limbs do not function properly. But the brain signals of stroke patients are observed as normal. Hence, EEG signals are used as input for wheelchair to rehabilitate post-stroke patient (Prajitno et al. [4]). A technology called brain–computer interface (BCI) is used to allow computer to make action based on brain signals. Electroencephalography is used to record brain waves. The Neurosky Mindwave Mobile 2 headset is used as input signal. The obtained signal is processed and classified by MATLAB software. It classifies the signal for moving wheelchair in four directions. The output signal is fed to Arduino board to control the motors of wheelchair.

The wheelchair-assisted system based on voice of a person is developed [5]. The user needs to download a voice recognition application in android phone. It should be connected to a Bluetooth device. Then, the user can speak specific commands which will be processed by the application. The word command is converted into text by google voice service. The text format is processed by micro-controller; it checks for valid input and gives command to motor drive for moving wheelchair forward, left, backward right and stop. As second part, the user can use an Android device with a GUI app to submit commands by pressing a specific button, and the wheelchair will react appropriately. The disadvantage of this system is that it is sensitive to pronunciations accuracy and clarity of voice of the user.

A wearable electromyography device is made to interact with computing systems and associated electrical devices [6]. Sensing, preprocessing, feature extraction and classification are the four main components [7,8,9]. EMG electrodes are placed on the arm muscles, mainly the flexor carpi radialis and the palmaris longus, for sensing signal. Preprocessing of signal is done for reducing noises. The extraction of features including common statistical features is done, and the patterns that represent gesture movements are matched by using the minimum distance classifier technique. The wristwatch can measure biological impedance body movements, heart beat rate, respiration.

The EMG electrodes are mounted on the upper trapezium muscles for signal acquisition. The trapezium EMG signals are analyzed and converted into control movements like forward and reverse that are fed into a PIC microcontroller. The wheel chair considers two forms of neck muscle rotations when navigating: flexion and lateral rotations. EMG-based wheelchair with robotic manipulator is designed for persons with transradial amputation [10]. EMG-controlled wheelchair for disable patients is available, but they use computationally expensive neural network for classification [11]. In the literature, even though a number of works are available to control wheelchair signals [4, 12,13,14], in this work, two simple hand gestures are used to control the movement of wheelchair and its implementation is easier.

2 Materials and Methods

Smart wheelchair consists of two parts: wheelchair transmitter and wheelchair receiver. Myoware muscle sensor and accelerometer are used as input sensors in wheelchair transmitter. Arduino Uno compares the signal from these sensors with threshold value and sends command through RF transmitter. In wheelchair receiver, RF receiver captures the transmitted signal and sends it to Arduino Mega for processing of signal. After processing the signal, Arduino Mega sends command to motor drive to run the motor in desired direction. Ultrasonic sensor is used to avoid obstacle by smart wheelchair.

In EEG-based driving assistance, more than ten electrodes are placed in forehead to sense the EEG signal. The electrode can be used for at least two times, after that it losses adhesive capacity. This increases the maintenance cost. Before placing the electrodes, surface of skin should be cleaned and hair should be removed. This makes the user to feel difficult in using the wheelchair for day-to-day life. In accelerometer-based driving assistance, the hand should not be tilted unwantedly which may trigger the signal of accelerometer. Hence, a combination of Myoware sensor, accelerometer and ultrasonic sensor is proposed as an upgrade to wheelchair.

2.1 Wheelchair Transmitter Module

The wireless control of wheel chair through hand gesture is the methodology used here. In transmitter part, accelerometer and Myoware muscle sensor acts as input, Arduino uno as motherboard, RF transmitter as wireless transmitter. Figures 1 and 2 show the physical and functional block diagram of wheel chair transmitter module. Figure 3 presents the actual hardware implemented for the transmitter. Myoware sensor and accelerometer are used as sensing elements. Myoware sensors are placed in forearm on the brachioradialis muscle for EMG signal acquisition. Contraction of these muscles generates an EMG signal which can be acquired through the Myoware sensors. Index finger movement and hand closure are the two movements used to move the wheelchair in left and right directions. The signal acquired from Myoware sensors is fed to the Arduino Uno which is microcontroller board based on ATmega328P. This board is used as motherboard to process the captured EMG signal. RF transmitter module is connected to Arduino Uno for wireless transmission of the signal.

Fig. 1
The block diagram has input components Myoware sensor, power supply, accelerometer, and switch, the motherboard Arduino Uno, and the output component the R F transmitter.

Physical architecture of transmitter module

Full size image
Fig. 2
A flowchart consists of the following steps. Data acquisition from Myoware sensor and accelerometer, processing of the E M G signal using Arduino Uno, and wireless data transmission by R F transmitter.

Functional block diagram wheelchair transmitter module

Full size image
Fig. 3
2 photos of a breadboard with wires attached to it and a setup with components such as wires and motor controller boards.

Actual hardware of wheel chair transmitter module

Full size image

Arduino Uno is uploaded with predefined threshold for four movements of wheel chair. When the intended movement takes place, the acquired EMG signals are processed and compared with predefined threshold by Arduino Uno. The EMG signal value corresponding to the identified movement is sent wirelessly to the wheelchair through RF transmitter.

2.2 Wheelchair Receiver Module

Figure 4 shows the physical architecture of wheel chair receiver module, and Fig. 5 shows the functional block diagram of the receiver. Figure 6 presents the actual hardware used for implementing the module. The receiver part of wheel chair has RF receiver, Arduino Mega and ultrasonic sensor. The signal transmitted from transmitter module is received by the RF receiver module that consists of RF tuned circuit, a pair of operational amplifiers, PLL and a decoder. The received signal is amplified by the operational amplifiers and fed into PLL, which allows the decoder to lock into a stream of digital bits, resulting in improved decoded output and noise immunity. Arduino Mega is used as motherboard to process the incoming signal obtained through RF receiver module. After processing, to control the movement of wheelchair, signal from Arduino mega is fed to motor drive which is responsible for driving DC motors connected to the wheelchair. L298N motor drive used is able to control the speed and direction of two DC motors simultaneously. The DC motors enable the wheelchair to be moved in all four directions.

Fig. 4
The block diagram has the input components R F receiver, power supply, and ultrasonic sensor, the motherboard Arduino Mega, and output components motor drive and D C motors.

Physical architecture of wheelchair receiver module

Full size image
Fig. 5
The flowchart of the wheelchair receiver module. The R F receiver receives data and Arduino mega processes data. A signal is sent to move the wheel in the desired direction. If some obstacle is detected, a signal is sent to move the wheelchair backward.

Functional block diagram of wheelchair receiver module

Full size image
Fig. 6
2 photos of the setup of wheelchair receiver module hardware viewed from the front and the back. The majority of components like wires and boards are located on the back of it.

Actual hardware of wheel chair receiver module (left: frontside of wheelchair, right: backside of wheelchair)

Full size image

Module for Obstacle Avoidance: The ultrasonic sensor monitors the environment and provides distance between wheel chair and surrounding. If there is an obstacle, Arduino Mega sends signal to motor drive to stop the wheel chair and move in reverse direction till the distance increases to threshold value. Now, user will decide the direction that avoids obstacle. User can also use switch in wheel chair transmitter which makes Arduino Mega send signal to motor drive that moves wheel chair in left, forward, right sequence to avoid the obstacle.

3 Results and Discussion

The Ardunio IDE handles all of the programming, and the application is loaded onto the Arduino board. Forward and reverse movements are carried out using accelerometer, whereas right and left movements are performed using different hand gestures. Training is carried out to decide the threshold level for performing each movement using corresponding direction of tilt of the accelerometer and gesture.

3.1 Forward Movement

When the accelerometer is tilted forward at the transmitter side as in Fig. 7, the X-axis value gets increased. Arduino Uno compares the obtained value with predefined X-axis value. If present value is higher than value 15 (corresponding command output shown in right side of Fig. 7), signal to move wheel chair forward is send through RF transmitter. The threshold values are obtained by proper training with the desired gesture. Arduino Mega processes the received signal and sends signal to motor drive to move the wheel chair forward.

Fig. 7
A photo of a breadboard with wires attached to it tilted to the left. A snippet of the output code is on the right.

Accelerometer tilted forward (left) and output (right)

Full size image

3.2 Backward Movement

When the accelerometer is tilted backward as in Fig. 8, the X-axis of the accelerometer value gets increased. Arduino Uno compares the obtained value with predefined X-axis value. If present value is lesser than value 15 (corresponding command output shown in right side of Fig. 8), signal to move wheel chair backward is send through RF transmitter. Arduino Mega processes the received signal and sends signal to motor drive to move the wheel chair backward.

Fig. 8
A photo of a breadboard with wires on the left and the output code snippet produced by it on the right.

Accelerometer tilted backward (left) and output (right)

Full size image

3.3 Left Movement

When hand gesture shown in Fig. 9 is applied, the sensor value of Myoware muscle sensor gets increased. Arduino Uno compares the obtained value with predefined sensor value, and if value is between 60 and 150, it sends signal to move wheel chair left through RF transmitter. Here also, the threshold values by proper training with the desire gesture. RF receiver receives the transmitted signal, and Arduino Mega processes the received signal, thereby sending signal to motor drive to move the wheelchair left.

Fig. 9
A photo of a hand gesture for left movement in which the index finger touches the thumb, while the other three fingers are stretched outward. A snippet of the output code is on the right.

Hand gesture for left movement (left) and output (right)

Full size image

3.4 Right Movement

When hand gesture shown in Fig. 10 is applied, the sensor value gets increased. Arduino Uno compares that value with predefined sensor value. If it is between 160 and 300, signal to move wheel chair right through RF transmitter is sent. The signal received by RF receiver is processed by Arduino Mega and signal sent to move the wheel chair right.

Fig. 10
A photo on the left of a hand gesture for right movement depicts all four fingers coiled into a fist and the thumb bent inward but not under the fingers. A snippet of the output code is on the right.

Hand gesture for right movement (left) and output (right)

Full size image

3.5 Obstacle Avoidance

Ultrasonic sensor continuously monitors the environment for obstacle and provides distance between wheel chair and surrounding. If the distance value less than 10, it sends signal to Arduino that an obstacle is present in the path. Arduino Mega sends command to motor drive to take reverse direction till the distance value gets 10. Now, user will decide the direction that avoids obstacle for the wheelchair. The user can press switch in transmitter which makes wheelchair to move in left, forward and right direction to avoid obstacle. Figure 11 is used to depict this process.

Fig. 11
A photo of the setup of the transmitter module and a photo of a breadboard with wires.

Detection of obstacle (left), switch pressed in transmitter module (right)

Full size image

4 Conclusion and Future Work

Hand gesture recognition has become a very active research subject in recent years, because of its possible use in human–machine interaction. A novel EMG-based HMI device is designed for gesture-based control of an intelligent wheelchair. The benefit of a surface electromyogram is that it is simple to record and non-invasive. The system is used to control the simple movements of the intelligent wheelchair, such as forward, left, right, backward and stop. The method is real-time and has a high rate of recognition. It has an important research value in many aspects of real-world applications such as assisting the elderly and disabled for their rehabilitation. It is provided with obstacle avoidance mechanism which helps the user to avoid obstacle.

The prototype is cost-effective and the mechanism helps user to control the wheelchair, it has a few drawbacks. The patient must be trained to operate the above-mentioned mechanisms for the wheelchair to work properly, which is a drawback of this model. If these limitations are resolved, these types of assisting wheelchairs can be very useful for patients on a daily basis. The Myoware sensor is used for controlling the wheelchair in left and right directions. In future, Myoware sensor can be used for controlling all the four directions with help of machine learning algorithm.