Keywords

1 Introduction

Neurodegenerative disease (NDD) is a term which results in death of neurons by blocking of the nervous system which includes brain and spinal cord and this is often incurable as neurons do not reproduce [1], NDDs are the main cause of the breakdown of communications among human brain cells. It can change balance, movement (called ataxias), speech, breathing, memory (called dementias), intelligence, and much more in human body [2, 3]. Parkinson’s, Alzheimer’s, Huntington’s, etc. are the most frequently diagnosed NDDs. NDDs are mostly considered incurable to disease progression without successful treatments, efficient therapies; patients could even die. A report generated by World Health Organization (WHO) represents that minimum 1 billion people across the world have been affected by neurological disorders such as multiple sclerosis, neuroinfections, headache, Parkinson diseases etc. [1]. It also shows that more than 50 million people are suffering form Alzheimer and other dementias which will be double in next 5 years. After the heart diseases it is the second leading cause of death with minimum 9 million deaths and 16.5% of global deaths [4]. A research show that 6% of total diseases are NDDs and these rate is high in developed and developing countries [5]. Due to the extensive popularity, Machine Learning (ML) methods have been used in biological data mining [6, 7] image analysis [8], decision support system [9,10,11,12,13]. In the arena of management Of NDD, deep learning approaches are powerful tools that enable systems to learn from the measured data in order to develop ways of making smarter decisions that can lead to better management of these types of patients. It can help to process medical data with multi-layer neural networks which results in improved prediction capabilities for several specific applications in management of NDDs [6, 7]. On the other hand, Internet of Things (IoT) devices are being used to monitor, manage and motivate a new generation of health care with the concept of smart home appliances for aged patients [14,15,16]. Recent IoT studies focus primarily on smart homes and communication technologies that support remote control of electrical, heating and lighting devices [17].

In this paper a RNN based fall detection framework for patient with NDD has been designed. The activity data from patient with NDD is collected using IoT sensor nodes (such as wearable, smartphone and camera), these information is pre-processed, and analysed in cloud and thereby differentiate fall and normal routine activities. The system can also be used for detecting an anomaly in the patient activity data and send the anomaly information to the doctor at the hospital and fall event to the caregiver/family member.

The rest of the paper is arranged as below: Sect. 2 outlined the related articles; Sect. 3 discussed system model, the methodology and results are explained in Sect. 4 and Sect. 5 respectively. Finally the paper concluded in Sect. 6.

2 Literature Review

This section discuss existing literature related to fall/activity detection and patient management. Sase et al. [18] proposed a method to detect fall using depth videos. By substracting background from extracted frames and doing some preprocessing like filter and analysis of connected components, Region of Interested (ROI) is calculated which helps to detect fall by comparing it to calculated threshold. Mostarac et al. [19] describes a system which can detect fall by using three axis accelerometric data. In this system at least two sensors are needed to collect personal data which contains information about treatment efficiency, mobility of patients which will be sent to the server by local receivers. System will alert the caregiver if fall is detected. Not only early detection of fall but also patient monitoring in real time will be served. Ali et al. [20] proposed a quick and precised system to detect fall by videos captured by surveillance camera. This system is represented on the basis of spatial based features and novel temporal which includes discriminatory prejudicial movement, individual location and geometric orientation. Doulamis et al. [21] proposed a system to detect fall by a single camera which is independent of direction of fall which using the background subtraction approach using hierarchical motion estimation and Gaussian Mixtures. The accuracy of this system to differentiate between fall and normal activities like sitting, bending is very high. Tzallas et al. [22] proposed a model called PERFORM for the real-times remote monitoring, assessment and management of patient’s with PD which can be used for personalized treatment (such as therapeutic treatment) and motor status for PD based on recorded data. Pereira et al. [23] has developed a mobile application design for the assistance of people suffering from PD. The main focus of this application is to provide knowledge and professional support for both patients and care givers to improve healthcare assistance. Baga et al. [24] proposed a system which can minimize the wearable sensors for monitoring and develop quantification algorithm and detect symptoms to help the clinical and caregivers for taking decisions. Punin et al. [25] has developed non-invasive hardware-based wireless system to collect data from PD patients with FOG to induce progress of walking, avoid falling which will be collected by a processor and transferred to mobile through bluetooth and enhance the lifestyles of patients. Magariño et al. [26] has introduced a novel technology that could potentially support and monitor people with NDDs and focuses on the application of Fog computing to ease the bandwidth uses. LeMoyne et al. [27] aims to incorporate a Smartphone as a platform for wireless accelerometers Machine learning to identify deep brain characteristics Stimulator for the ultimate tremor. Three mature systems like a smartphone and machine learning has been successfully to improve the efficiency of deep brain stimulation treatment.

3 System Model

Figure 1 shows the proposed framework for the management of Neurodegenerative disease using machine learning and IoT. The NDD management is a vast process and due to the page limitation, we have considered the fall detection module and the pre/post-fall management using ML and IoT. The expected data can be collected via camera sensors and wearable devices such as mobile, portable devices, smartwatch etc. Accelerometer or Gyroscope is being used to collect data of action like shaking and spinning and camera sensor is being used for detecting the person’s position. A smart home system is also connected to the mobile phone for giving immediate comfort such as turning on/off the light or fan etc for the patient’s emergency. These acceleration, orientation data and the data from the camera sensor are being sent to the cloud and e-health records via wifi or cellular net or Ethernet for local access for the specified caretaker and doctors of the patients in the time of emergency. The data is being received and stored in the cloud. Due to page limit, we are not discussing about the processes how the computational cost will be managed in real time detection though cloud. The sent data is being pre-processed such as labeling the data set and feature selection. Then it trains and tests the data set as it expected and applies machine learning algorithms on the collected medical data for analyzing the data and processing patient’s clinical assessment, assisting Decision support system and anomaly detection. By this, we can decide which data we should be stored and which not and make a decision. Here, repetitive data is not being saved. It helps to reduce storage size. Thus, the memory will be efficiently used. If an anomaly is detected that means the patient is fallen from chair or standing or any other way and it will automatically notify the caregiver/family member about patient’s fall. The flow chart of the proposed fall detection process is illustrated in Fig. 2.

Fig. 1.
figure 1

A function model for Neurodegerative disease management using machine learning and IoT. The physiological, voice and video data collected from the patient using IoT system are sent the collected information to a cloud based platform. The ML algorithm in the cloud convert the data into actionable insight or detect anomaly from the data. The knowledge extracted from data can be sent to personal doctor, caregiver or even the family members.

Full size image
Fig. 2.
figure 2

Flowchart for proposed fall detection system.

Full size image

4 Methodology

In this section, we represent our ML framework which includes data collection, data preprocessing feature extraction and ML algorithm to detect a fall. We have used two dataset named UR dataset and Open labeled dataset. These datasets have been preprocessed to extract features. From them test set and training set have been divided where train data have been classified by ML classifiers. Figure 3 shows Fall detection architecture and datasets used to train the model. We used two datasets named UR dataset and open labelled dataset in our model. Firstly batch normalization is applied in both datasets. Batch Normalization is a technique to improve the speed at which the network trains, allows higher learning rate by re-scaling and re-centering the input layer. Then RNN layer is applied on both datasets. Then Fully Connected layer applied on both dataset which converts RNN outputs to our desired shape. Softmax is implemented just before the output layer which assign decimal probabilities that must sum to 1. At last knowledge fusion from both datasets are used to detect the fall appropriately.

4.1 Dataset Description

Two datasets named UR dataset and Open labeled dataset have been used in our system (see Table 1). UR fall detection dataset are developed by kepski et al. [28] used seventy sequences where thirty are falls and fourty are activities of daily living (ADL). Two camera are used. One is front facing and other is from ceiling which provides the top views of the scene. Kinect cameras and corresponding accelerometric data are used to record fall and one device(camera 0) are used to record ADL. IMU and PS Move devices are used to collect sensor data. Two types of falls, one from standing and other while sitting on a chair are described here. Besides picking object from ground, lying on the sofa and floor, normal walking, sitting down are the ADL. Data needed to extract features of UR are

  • Height/Width- Bounding box height to width ratio

  • l/w - Major to minor axis ratio

  • H/Hmax - A proportion expressing the height of the person’s surrounding box in the current frame to the physical height of the person, projected onto the depth map and

  • Area - A ratio expressing the person’s area in the image to the area at assumed distance to the camera.

Wertner et al. [29] has created a labelled dataset which can be used for mobile phone with the data of accelerometer and gyroscope sensor. An orientation software based sensor is used to derive data from the accelerometer and geomagnetic field sensor which is attached to the mobile phone and the data is recorded by the mobile phone. Data needed to extract features of Open labelled are:

  • Acceleration of devices: Acceleration is stored as 3D vector indicating acceleration along each device axis, not including gravity. It can be calculated as

    $$\begin{aligned} a_{xyz}=\sqrt{x^2+y^2+z^2} \end{aligned}$$
    (1)

  • Orientation of devices: Orientation is stored as 3D vector of angles azimuth, pitch and roll.

Fig. 3.
figure 3

Proposed Fall Detection Architecture and datasets used to train the model. (A) shows features used in the proposed RNN model and (B) illustrate the RNN architecture

Full size image
Table 1. Information about the dataset used in this study.
Full size table

4.2 Feature Extraction

A feature extraction module performs a significant role in the fall detection system. To enhance the fall detection rate, our focus was on the generation and selection of features. In this research we have extracted five features from UR and open labeled dataset.

It represents the variety of change of the magnitude of acceleration in x, y and z axis.

Variance of CV Acceleration:If we divide the standard deviation of acceleration by it’s mean (\(\mu \)) we can get the coefficient of variation.

$$\begin{aligned} \sigma _{xyz}=\frac{\sqrt{\sigma _x^2+\sigma _y^2+\sigma _z^2 }}{\mu } \end{aligned}$$
(2)

Variation in Motion Vector:During the fall the body is in a motion and magnitude of the motion variation will be high when fall occur. The variation will be 0 when fall occur. We can calculate the magnitude of motion by:

$$\begin{aligned} m_{xyz}=\sqrt{(m_x^2 +m_y^2+m_z^2 )} \end{aligned}$$
(3)

Polar Angle Ratio:Polar angle ratio from accelerometer data can be calculated as follows:

$$\begin{aligned} \cos ^{-1}\left( \frac{z}{\sqrt{x^2+y^2+z^2}}\right) \end{aligned}$$
(4)

This polar angle reflects the sudden transition and body angle, suggesting a fall. In addition, sudden change is represented by the ratio of instant angle and it’s previous values within a short period of time.

Difference Between Polar Angle: It is represented by \(\Delta \theta \) which helps to cover large tilt angle variations.

4.3 ML Algorithm

Recurrent Neural Network (RNN): RNN mainly used for supervised time series analysis is a machine learning algorithm where outputs of the previous step are used for the inputs of next step. Hidden state are the most important feature of RNN. RNN with convolutionary layers are used to expand the successful neighborhood of pixels.

Random Forest (RF): Random forest is a supervised learning algorithm which is a combination of decision trees where the forest is build by an assemble of decision trees to increase the overall result by combining learning models. Here the input is evaluated by the decision tree forest and the output class is measured as the tree’s response class.

Support Vector Machine (SVM): SVM which is mostly used for classification is a machine learning algorithm which helps to solve pattern recognition. Coordinates of individual observations are represented by support vector. It is a frontier that separates both classes at its best. Each data item is ploted in an n-dimensional space where n indicates the number of features we have and the value of each feature represent the value of a particular coordinates.

4.4 Model Training

For training purpose both the datasets are splitted into two parts: 70% data from each dataset is used for training and the remaining 30% is used for testing. For the 5-fold cross validation, we used random partition from the datasets.

5 Numerical Results

This section discussed the numerical results obtained using state-of-the-art classifiers. We have utilized Weka for evaluating the performance of the classification algorithms (RF, SVM, and RNN) which can be used to detect falls as well as the normal daily activities of the people with neurological diseases. For each classifier, we use precision, sensitivity, specificity, and F-1 score.

The proposed RNN based fall detection architecture contains two parallel structure, each consists of a batch normalization layer, an RNN layer and a fully-connected layer followed by softmax output layers. The model was trained using Adam optimizer and for 30 epochs with a learning rate of 0.001, batch size of 32 and RNN dropout of zero on the training dataset. After training, the model was tested using the separated test dataset.

Table 2 shows the classification performance of RF, SVM and RNN.

Table 2. Performance comparison of RNN with RF and SVM
Full size table

From the table (see Table 2), we can depicted that RNN and SVM have better accuracy, Precision, specificity and F1-Score than the RF. But the sensitivity is better in RF then the SVM and RNN. From overall analysis, we can see that the Fused algorithm gives better Accuracy, precision, sensitivity, specificity, and F1-score, whereas Open labelled gives the lowest value of these performance matrices.

6 Conclusion

Management of neurodegenerative diseases is a vast and condemnatory process. As falls are the second leading cause of accidental or unintentional injury deaths worldwide among elderly people having a neurological disorder, in our proposed model, we have worked on designing a recurrent neural network-based framework which can detect any occurrence of fall/daily activity of a patient using IoT and then send this data to the specified doctor and also notify caregivers/family members about the fall events through the available communication line easily. In our RNN based fall detection Architecture, fused knowledge from wearable/portable and imaging devices (camera) has been used for the fall detection. Open-labeled and UR data-sets are used to train the preferred ML method, RNN and the performance is compared with two classifier like RF and SVM for these two data-sets. The comparison of the performance evaluation shows that the proposed RNN based fall detection framework is worthwhile and excels its counterparts. In future, we will integrate other management features with this model to extend the scope and enhance the quality of experiences.