Keywords

1 Introduction

Attention can be considered as a focal point to capture the brain waves of a person that can further be used to process and analyze signals for utilization in the implementation of autonomous smart wheelchairs [1,2,3], where the human makes decisions and the smart control technology helps in the automation of motion. The primary contribution of this research is fourfold: The first fold is to observe the brain activity. The second is to create a firmly established environment with different situations. The third is to be evaluated through a machine learning algorithm, and the fourth is to deploy the idea in the smart wheelchair to perform the task. Brain waves control an autonomous smart wheelchair, in which the human makes decisions to move the wheelchair (forward, backward, right, left, and stop) with smart controls and automates the motion. The advancements in a smart wheelchair are equipped with environmental sensors, HD camera, 3D scanner, Bluetooth device, and a computer-based system as the main processing unit and place a strong emphasis on computer cluster architecture. Besides their disability, intelligence technique is added to the wheelchair platform around user control.

The goal of our study is to develop a BCI framework that may be extended to offer more natural and involving control. This involves the creation, testing, evaluation, and deployment of a new data analysis method for the study of human cognitive function along with the psychological and behavioral effects of a BCI system on the cognitive capacity of a human user. In this section, we have discussed the architecture of the model and working process of the model and the method of collection of data and analysis and implementation in machine learning and how to convert it into the command of a motor in the wheelchair. Furthermore, the accuracy of various data methods has been examined.

2 Background

Soleymani [4] proposed synchronized stimuli videos about unusual degrees of freedom. The author discovered two concepts for continuous emotion [5]: “long short-term memory recurrent neural network” (LSTM-RNN) and CCRF. They studied contamination and the effects of facial muscles using EEG signals. The authors have taken facial expressions to carry out the complementary observation of that EEG signal. The previous frame was used to track the supervised descent method while using the landmark (SDM). A conditional random field (CRF) had been used to build a probability-based model to classify sequential data and segment support. The concentration of the signal in the EEG [6] is used as the control signal for the wheelchair, and the signal is sent to the STM32 [7] to allow forward movement of the wheelchair. The emotion reflection in real time was collected and interpreted using a brain–computer interface (BCI) [8]. Negative and positive emotions were used to classify EEG data using k-NN [9] algorithm and multilayer perception neural network (MLPNN) [9]. The classifier algorithm sets up the channel selection. Individual participants used the EEG channel to perform various classifications. The resulting feature vector is being used to classify related negative and positive emotions using a related classifier.

k-NN classifier along with a new object was selected for feature extraction and distance calculation using Euclidean distance, Minkowski distance, and Hamming code. In all of the preceding research, the authors analyzed their observations in the context of an advertisement, a short film, or a non-biased situation.

Table 1 contains the various types of EEG signals with their respective frequency range and amplitude. These signals have been used further in developing and simulating the proposed BCI framework to capture perceptual and sensitive information from the temporal lobe, frontal lobe, parietal lobe, and occipital lobe [9].

Table 1 Comparison of various types of classified signals [6, 9]
Full size table

3 Methodology

An experimental protocol was designed to capture participants’ attention and meditation level which is shown in Figs. 1 and 2. Wheelchair will be moved, controlled, and stopped as per attention and mediation level of person. Attention and mediation level of person will be obtained from EEG response data received from brain using the NeuroMAX-32 instrument. The duration of the experiment is 900 s including preparation and rest. The experiment is based on visual movements tests and facial expression tests. These two factors were chosen because they have a great influence on participants’ selective attention and meditation-related area. The experiment was conducted to study the paired habituation of children and adults and when the external force or anybody pinched the body that reflects in the attention and some change in facial expression which reflects the motion of a person getting hike. When the external force was applied to the human body, it was raised till the human body gets calm. When the human body gets relaxed and the mind is in a stable state, at that time the meditation of the human body gets hiked, and like this, the other lobes get hiked, and after some moment, it is normalized.

Fig. 1
figure 1

Architecture of model

Full size image
Fig. 2
figure 2

Working process

Full size image

3.1 Architecture for Proposed Model

As per Fig. 1, the data was collected by the EEG machine (NeuroMax-32) from hundred participants and analyzed through machine learning algorithms, viz. SVM, decision tree (J48), and naive Bayes classifier. It, in turn, gives the command to the motor of the wheelchair, and according to command data, the wheelchair starts to work and the process will continue symmetrically.

3.2 Functionality of Smart Wheelchair

As per Fig. 2, the data has been collected from the human brain using EEG machine and analyzed and trained  through machine learning teachniques. The trained data send to the microcontroller through Bluetooth of a wheelchair. Then the Microcontroller gives the command to the motor of the wheelchair and the wheelchair started working as per the data given by the Bluetooth.

3.3 Data Collection Framework

In this part, the proposed paradigm for categorizing emotions in various situations is described.

First, data from EEG signals was obtained from several lobe points of the brain using the NeuroMax-32 device. After that, the data was preprocessed in anticipation of extraction in the future. Then, by selecting the proper EEG channel, it generated a feature vector. The data was then examined and grouped into positive and negative.

Data collected from various lobes will be paired and analyzed. The right sides lobes, FP2-F8, F8-T4, T4-T6, T6-O2, and left sides lobes, FP1-F7, F7-T3, T3-T5, P3-O1 can be paired and analysed for different applications. In this work, two pairs such as, P3-O1 and T4-T6 provide EEG response data that reflects meditation and attention level of person, and it is in the range of 0–100.

Digital wavelet transform (DWT) is used to convert the extracted data and EEG signal. The model’s last job involves categorization and prediction. It has been used during the analysis phase. Then, accuracy is obtained using the classifier.

3.4 Experimental Setup

The working mechanism of the approach has been expressed in three stages as per Fig. 3. In the first stage, the NeuroMax-32 channel headset has been used to capture the brain wave by using 32 electrodes placed in the human skull. It has 32 channel represented by 32 red and black connecting wires. The connecting wires have two ends; one end has been connected to the NeuroMax board, and another end has been connected to the electrodes. Out of 32, 16 electrodes are placed in right side of brain represented by red color wire and another 16 electrode are placed in left side of the brain represented by black color wire.

Fig. 3
figure 3

Experimental setup for the proposed model

Full size image

In the second stage, EEG signals will be received from brain by EEG headset through electrodes placed on the head and it will be sent to the NeuroMax board which in turn sent the captured signals to the connected computer through USB serial cable. The EEG signals are captured by NeuroMax software running in the computer, and output is represented in the graphical view.

In the third stage, the data in the graphical view is converted into an excel format that has been used for analysis in the proposed work.

3.5 Data Analysis and Modeling

The final step to control the wheelchair on the decision of the appropriate movement of a person is classification; this can be done with the implementation of machine learning algorithm, which is an application of AI. Machine learning models and applications can be categorized into supervised learning, unsupervised learning, and reinforcement learning.

In supervised learning, input data (X) and output data (Y) are a function of Y = f(X). This has been resolved through the use of classification and regression. Regression problems occur when output data is used instead of a real or continuous value. Classification problems occur when data is output on a category basis. Decision tables, like decision tree classification models, are primarily used for machine learning algorithms.

Where W1 and W2 are the weight factor given to attention label AL and meditation label ML, respectively. We have given 70% focus to the attention level, and that is why we consider W1 = 0.7, and whereas 30% focus is given to meditation level, and that is why we consider W2 = 0.3 collectively to define the status of the person using attention and meditation using Eqs. 1 and 2.

AL:

 data obtained from T4-T6 lobe points (set of lobes considered for attention).

ML:

data obtained from P3-O1 lobe points (set of lobes considered for meditation)

 

$${\text{Att}}\_{\text{med}} = A_L *W_1 + M_L *W_2$$
(1)
$${\text{Att}}\_{\text{med}}_{{\text{status}}} = \left\{ {\begin{array}{*{20}l} {{\text{low}}} \hfill & {{\text{for}}\,{\text{Att}}\_{\text{med}} < 35} \hfill \\ {{\text{medium}}} \hfill & {{\text{for}}\,35 < {\text{Att}}\_{\text{med}} < 55} \hfill \\ {{\text{high}}} \hfill & {{\text{for}}\,{\text{Att}}\_{\text{med}} > 55} \hfill \\ \end{array} } \right.$$
(2)

In Eq. 1, Att_med represents mental status of person based on his attention and mediation level, and further, it has been categorized into low, medium, and high as reflected in Eq. 2. As per this, mental status of person speed of wheelchair will be controlled. Further, it can also control, stop, and move wheelchair. In this work, the wheelchair will stop when Att_medstatus is low, it will move slowly in medium status, and it will move in a steady way for high status. For this, we have studied the data using various machine learning approaches to judge the accuracy of model.

Processes to be followed

P1: Calculate the prior probability based on the given class labels.

P2: Find the likelihood probability for each class individually.

P3: Calculate the posterior probability using the naive Bayes formula.

P4: Find the class with the highest probability and label it as the probability class.

3.6 Machine Learning Algorithm Used for Classification

Machine learning can be helpful in the prediction of outcomes based on the inputs collected from various previous data without the need for explicit programming. It comprises sets of algorithms that can take data as input and can automatically improve user experience.

3.6.1 Naive Bayes Algorithm

Naive Bayes is a collection of “probabilistic classifiers” based on Bayes’ theorem with a high amount of statistical data independence between features or elements taken for classification.

3.6.2 SVM

A support vector machine (SVM) is a learning algorithm based on a supervised learning method that can be helpful for data analysis, pattern recognition, and regression analysis. It is a classifier for the separation of the hyperplane. SVM is a non-probabilistic binary linear classifier. It is a supervised learning method associated with other learning algorithms. It can also be used as a discriminative classifier defined by separating hyperplane. It plays a significant role in the categorization between text and hypertext by accepting two possible classes from the input.

3.6.3 Decision Tree (J48)

A decision tree is another popular machine learning tool that uses a non-backtracking or greedy approach. The decision trees are constructed in a top-down or bottom-up manner. Initially, training is conducted with a set of tuples along with their associated class labels which are further recursively partitioned into smaller subsets.

3.7 Performance Evaluation Parameter

True positive (TP) rate, false positive (FP) rate, recall, precision, F-measure, mean absolute error (MAE), root mean squared error (RMSE), relative absolute error (RAE), and root relative squared error (RRSE) are the various parameters used for evaluation of machine learning algorithms used for classification in the proposed work.

For any potential parameter value chosen to differentiate between two classes or different cases, certain data will be correctly classified as positive (TP = true positive fraction) and some may be erroneously classified as negative (FN = false negative fraction).

The FP rate is calculated as FP/FP + TN, where FP stands for false positives and TN for true negatives (the total number of negatives is FP + TN). It is the chance that a false alarm will be triggered; a positive result will be returned when the true value is negative.

$${\text{Recall}} = \frac{{{\text{TPR}}}}{{{\text{FN}} + {\text{TPR}}}}$$
(3)
$${\text{Precision}} = \frac{{{\text{TPR}}}}{{{\text{FPR}} + {\text{TPR}}}}$$
(4)
$$F\,{\text{Measure}} = \frac{{2*{\text{Precision}}*{\text{Recall}}}}{{{\text{Precision}} + {\text{Recall}}}}$$
(5)

Precision, recall, and F-measure have been determined as per Eqs. (3), (4), and (5). In statistical analysis of binary classification, the F-score or F-measure is a measure of a test's accuracy. An F-score can have a maximum value of 1.0, signifying perfect precision and recall, and a minimum value of zero if the precision or recall is both zero. Correctly classified instances are used for the compilation of true positive and true negative occurrences (TP + TN). For both false negative and false positive classifications (FP + FN), the same incorrectly classified instances were used.

The graph for performance evaluation that can be helpful in the classification of performance from the application of multiple classifiers is termed receiver operating characteristic (ROC). This is helpful in the demonstration of the relationship between the rates of true positives and false positives. ROC is a machine learning prediction concept that represents a diagonal approach. There are various parameters associated with the performance evaluation namely, mean absolute error (MAE), root mean square error (RSME), and relative absolute error (RAE). The MAE quantifies the average measurement of erroneous items in a given set of values where values are the distribution of probabilistic data items. The standard deviation or the root mean square error (RSME) provided information about the deviation of individual data items from the mean or in others words it gives a mathematical idea about how close the data points are toward the mean. Relative absolute error (RAE) is a method for calculating the accuracy of an analytical model. RAE is an abbreviation for the absolute rate.

4 Result and Discussion

In NeuroMax-32, the lobe points T4, T6, P3, and O1 are primarily taken as emotion and sentimental analysis. The data has been analyzed and computed using these lobe points. The data was gathered from various nerves of the brain from data pair points T4-T6 and P3-O1. As a result, the data amalgamation technique can be a suitable mechanism for more accurate analysis. The data amalgamation was performed with the T4-T6 pair of points and the P3-O1 pair of points. Multiple paired lobes points are calculated from the 32 port NMX32 device. P3, O1, T4, and T6 are primarily designed to assess emotion and sentiment. The data has been analyzed using these four lobes as a starting point. The data aggregation has been computed from these two pairs, as shown in Figs. 4 and 5. In these graphs, at the time of pinching on the skin of the person or heating to the skin of a person, the emotion may be observed through the segments of entire result.

Fig. 4
figure 4

Data aggression between T4 and T6

Full size image
Fig. 5
figure 5

Data aggression between P3 and O1

Full size image

As per Fig. 4, it has been observed that the resultant data is unstable due to outside activities on the human body, till the human body gets calm. From 185 to 227 of X-axis showing more value  as a person gets pinched which reflects the emotion of a person getting hiked. Like this, the other points get hiked and normalized, and so on. The process will continue till the human body gets some external forces.

As per Fig. 5, it has been observed that the data becomes hike because the external force was applied on the human body and then it raised till the human body gets calm. Like in 1981–2080, the human body gets relaxed with the concentrated assignment. So, the meditation of the human body gets hiked. Like this, the other points get hiked and normalized, and so on.

In Fig. 6, it is shown that the range of data in all methods is mostly medium-range data which was collected when the input body in a normal position, when the input body gets hurt by some external body or pinched by someone or surprised or scared the data level in all methods became high, and when the input body gets calm or sleepy the mind was in peace the data level becomes low.

Fig. 6
figure 6

Based on category

Full size image

In Fig. 7, we observed the kappa statistic, mean absolute error, and root mean squared error for naïve Bayes, SVM, and decision tree (J48) and we find that naïve Bayes is more kappa statistic value than SVM and decision tree. And, also we find that SVM has very little error in comparison with naïve Bayes and decision tree.

Fig. 7
figure 7

Error detection

Full size image

In Fig. 8, we have analyzed the TP rate, FP rate, precision, recall, F-measure, MCC, ROC, and PRC area and we find that in SVM all the classes are the same except the FP rate. In SVM, the FP rate is zero and the decision tree is almost the same in all parameters expect FP rate; the FP rate is zero here. And, in the naïve Bayes, all the parameters are less than one.

Fig. 8
figure 8

Classifier analysis

Full size image

As per Fig. 9, we can say that the accuracy level of SVM is higher in comparison with naïve Bayes and decision tree. The accuracy level in SVM is 99.96%, whereas in decision tree it is 99.80% and in Naïve Bayes it is 94.72%. The incorrectly classified instance is closer to 0 in both SVM and decision tree and a little in naïve Bayes, and also, we find that in SVM we get very less error in comparison with the decision tree and naïve Bayes.

Fig. 9
figure 9

Accuracy level

Full size image

5 Conclusion

We have tried to develop a smart wheelchair that can work on behalf of an incapacitated human being by the methods discussed in this work. We have used various machine learning algorithms to design an artificial intelligence-based experience for people on wheelchairs. For the same, we have captured EEG response data from human brain using NeuroMax-32 headset attached with the skull to detect brain waves. Further, attention and meditation level has been measured using T4-T6 and P3-O1 lobe points of EEG response data. It has been used to determine mental status of person that will be used to move, control, and stop the wheel chair. In the future, we will try to move the wheelchair on left and right side using eye movement.