1 Introduction

Advances in Internet of Things (IoT), wireless communication, big data and machine learning technologies have led to a plethora of useful applications that are being used for improving individuals’ quality of life and benefitting businesses. The integration of IoT with paradigms of edge and fog computing has further provided an impetus to enriching these applications. Computing at the network edge removes the barriers of latency that were typically created in sending data from IoT devices to cloud-based servers for analytics. One domain that can truly benefit from the above mentioned technologies is that of human daily activity monitoring. Activity monitoring is a significant task in e-health applications where physical activities of individuals requiring special care, such as elderly, chronic patients etc. are continually monitored. Individuals’ data from wearable or environmental sensors is collected for reasoning, decision making or providing personalized recommendations.

The percentage of elderly in the world population is increasing over the years and it is expected to reach 17 percent by 2050 according to a new report, “An Aging World: 2015” [1]. A large population of elderly are staying alone and can be provided better health-care facilities by monitoring their daily life activities. Their activity data can be used to monitor their routine vitals and deduce if they need further support. Any untoward incident can be detected as an anomalous activity so that immediate help can be provided to them. Such monitoring can also support critical patients by real-time detection of any deviation from normal parameters.

Activity monitoring has been possible with the increase in number of IoT devices. This number is expected reach 75.44 billion by 2025, as per Statista [2]. These devices are continuously generating large amount of streaming data which will only be helpful when it is transformed into meaningful information. IoT devices cannot, in general, process this data locally because of low storage and limited processor speed capability. This gives rise to the need of transferring the data to cloud for further processing. However, high speed internet is required to send the data to cloud for further processing and retrieve information according to user requirements. It also takes time to process the data over internet cloud and increases the latency of response. Fog based processing makes use of computing power present at the nodes located in local area network (LAN) of the IoT devices. Instead of sending the generated data to the cloud, it is processed at the fog devices with improved processing capabilities to reduce latency. The LAN based fog devices are able to process the huge amount of generated data and send the aggregated processed data to the cloud periodically. They generate alarm or send information to the user in case of detection of any anomaly or discrepancies. Hence, fog-based devices work as an interface between IoT devices and cloud or user.

This work presents XLAAM, a daily activity data monitoring and anomaly detection framework within a fog-enhanced smart home. A time series data set related to daily activities of two residences is monitored for activity classification and identification of anomalies, if any. An explainable Long short-term memory (LSTM) [3] recurrent neural network is employed for the same. LSTM is a deep learning neural network (DNN) with feedback connections that can process sequences of data and hence, is suitable for the streaming data of activity monitoring. Further, the use of explainable AI is especially important in the application under consideration since it involves human health care. The inferences or explanations obtained from the system will aid in building the users’ trust in the DNN. It will also help the practitioners to improve the functioning of the health-care system. The paper makes the following contributions:

  • A fog-enhanced architecture of a smart home is presented

  • Explainable LSTM is used for activity monitoring and anomaly detection

  • Extensive streaming data analytics is performed for verifying the efficacy of the proposed framework

The rest of the paper is organised as follows: Sect. 2 discusses the state of art in domain of activity monitoring and use of XAI in the same. Section 3 presents the fog-enhanced architecture that is employed in the work. We discuss the XLAAM framework in Sect. 4. Results of experiments carried on a real-world data set are presented in Sect. 5. Lastly, we conclude the presentation.

2 Related work

Streaming data is continuously generated from various sources such as IoT sensors, servers, security logs, web applications etc. Data streams can be processed, stored, analysed, and acted upon as it is generated in real time using stream processing technologies. Amazon Kinesis is a cloud-based managed, scalable service that allows for real-time processing of large data streams. [4]. Apache Kafka is a distributed publish-subscribe messaging system which integrates applications and data streams [5]. Apache Flink is a data-flow streaming engine that allows for distributed computing over data streams. [6]. Apache Storm is a distributed real-time computation system that's commonly used for distributed machine learning, real-time analytics, and a variety of other applications, particularly when there's a lot of data. [7].

One important application of data streams is identification of suspicious patterns or real-time threats by processing and analysing the continuous data [8]. Classification and clustering are two methods of pattern recognition used in machine learning. While classification employs supervised learning to assign predefined classes to input data, clustering uses unsupervised learning to identify similarities between data points and group them according to the common characteristics. These groups are called clusters. Clustering algorithms are often used in recommendation systems and classification is used in applications like fraud detection, where financial transactions may be classified as correct or fraud using historical data. Some of most popular clustering algorithms are K-means, Density-based spatial clustering of applications with noise (DBSCAN), Ordering Points to Identify Cluster Structure (OPTICS) and Hierarchical clustering etc. Types of classification algorithms are logistic regression, Naïve Bayes, K- Nearest Neighbours, Decision tree, Random forest and support vector machine [3].

Data can be either static (stored in some database) or dynamic data (that is continuously generating from different sources). Clustering algorithms take data and find patterns in the data may be the shape, size, colour which can be used to group data items or create clusters basically. Hence, clustering algorithm provides the labels on historical data. Classification algorithms assign label to the new data according to the training and test phase. Both clustering and classification are required to analyse the data. Deulkar Shweta S. et al. (2021) [9] have used clustering followed by classification to evaluate the external feature based quality of tomato in to three grades. They used K-means clustering to extract the external features of a tomato include RGB percentage, shape of fruit and texture (size and weight are not considered). These features have been used in support vector machine(SVM) classification for training and testing [9]. Many researchers have used clustering followed by classification in health care such as to classify heart electrocardiogram (ECG) wave as healthy, non-healthy and not defined [10] and to check human eyes as healthy or unhealthy [11]. Norma et al. (2020) [12] proposed the heart disease prediction model for clinical decision support system to diagnose the heart disease. Jin Xiao et al. (2020) [13] proposed hybrid model which consists of clustering and classification. They used two clustering algorithms (K-means and DBSCAN) and C4.5 Decision tree classification model for experiment on the UCI (University of California at Irvine) datasets and achieved better performance. There are many real-world applications for time series forecasting. For example, it can predict call arrivals [14], how much electricity will be used [15], and how bad the air will be [16]. In time series forecasting, Autoregressive Integrated Moving Average (ARIMA) is widely used. Conteras et al. (2003) [17] used ARIMA to figure out how much electricity will cost the next day. In time series forecasting, neural network based methods have also been used. Kong et al. [18] predicted residential load using a Long Short-Term Memory Network (LSTM). Rangapuram et al. [19] came up with a way to predict time series. They used a model called deep-state space. Yi et al. [16] proposed a distributed fusion network for air quality prediction. Moreover, The Convolutional Neural Network (CNN) is a potentially useful Deep Learning approach that has shown promising results in speech recognition, image classification, and text analysis. To recognise human activity from time series data, CNNs outperform other traditional machine learning approaches because of their local dependence and scale invariance [20]. One-dimensional CNNs have been shown to be more effective than traditional machine learning models in solving the human activity recognition problem using performance metrics [21]. Recurrent Neural Network (RNN) [22] is a special type of neural network that aims to exploit the temporal information in the input data. As a result of the shortcomings of Multi-Layer Perceptron (MLP), which ignores the relationship between the current input data and the previous and subsequent input data, RNN was developed. When compared to MLP, RNN is more capable of transmitting temporal information through the network. It is used in time series data tasks like machine translation, speech recognition, and emotion classification.

Another application of streaming processing has been put forth by Zhixiang et al. (2020) [23] that calculates the dynamic air quality index based on streaming data for Chinese cities [23]. The authors have done empirical studies and applied three-month width window Data Envelopment Analysis (DEA) models to evaluate the air pollution problem in all over China. Taegong Kim et. al. (2020) [24] proposed a method for anomaly pattern detection in a data stream by applied K-means clustering for binary classification for outliers, i.e., 0 tends to normal data and 1 tends to outlier, and performed statistical tests on a data with binary labels [24]. The detection of an anomaly is closely linked to the concept of drift detection on a data flow whose objective is to understand a change of current data stream [25].

Activity Monitoring is essential to proactively detect as well as handle unwanted threats or losses. Hence, activity monitoring finds its applications in various areas like business activity monitoring [26], human activity monitoring [27], social activity monitoring [28] [29] and health monitoring [30] etc. It can be used for cost benefit analysis of business activity to know the positive and negative views of any social activity and take care of elderly and patients by physical activity monitoring.

There are numerous works proposed to address the health monitoring in IoT environment such as ECG monitoring [31], sleep monitoring [32] and human activity recognition [33]. ECG of patients have been monitored by IOT based embedded platform. Author has used Particle Swarm Optimization (PSO) based twin support vector machine (TSVM) to classify the time frequency features and detect the irregular heartbeat [34]. Researchers have attempted to find the best position to wear sensors to get higher accuracy and monitor the physical activity in IoT environment [35]. Various machine learning algorithms such as Bayes networks, SVM, Artificial Neural Networks (ANN) and Hidden Markov Model (HMM) have been utilized to train and test physical activity data. Few efforts have also addressed issues such as handling time series data, limited memory and processing power of devices, security and privacy to deploy the devices in IoT environment [36].

Through continuous monitoring and sensing, millions of Internet of Things devices around the world generate enormous amounts of data every minute. Now the challenge is raised for data transmission as whether to send all the generated data directly to the cloud, or to pre-process such data at the edge node and only send necessary data to the cloud [23]. Through frequently monitoring to any activities, if data is being transferred to the cloud it will required good internet speed, processing power but many IoT sensors have resource constrained have limited energy and storage. To overcome this transmission overhead and limited storage, fog network came in the picture. Fog is interface between IoT sensors and cloud. Data is analysed and pre-processed at the fog node. Only unwanted behaviour of data or some data capture by random sampling or periodically is sent to the cloud. Y. Chen et al. (2020) [37] proposed an Internet of Things (IoT)-based pet care system with sensors and actuators on the food feeder, water dispenser, and litter box. They created a smart app for pet owners to access information about their pets and track their food and water consumption. MU Sohang et al. (2020) [38] has been proposed IoT based application for smart garbage management to monitor the status of garbage bin. When the garbage bin is full, the communication system notifies the appropriate authority to collect the waste. Other IoT based applications are smart parking system [39], smart agriculture [40], smart health monitoring [41] etc.

This work utilizes an explainable deep learning model in a fog-based environment for activity and anomaly detection. The proposed approach differs from many recent works on human activity recognition such as [42, 43] which are only IoT-cloud based and apply classification on batch data. The present work employs a fog-based layer which performs anomaly detection in real time on streaming data. Another highlight of the work, as compared to the previous literature, is the use of the cloud layer for explainability analysis of the data.

3 System model

Internet of things (IoT) based devices or sensors are generating huge data continuously, but are not able to process it locally because of poor storage and processor limitations. This gives rise to the need of transferring the data to cloud for further processing. IoT based devices require high internet speed to send the data to cloud for further processing and send back information to the user as per requirements. In the traditional approach (as shown in Fig. 1), IoT devices or sensors send streaming data to the cloud directly. Any processing or analytics is performed at cloud-based servers. Processed result or any notification is sent from the cloud node to the end user or actuators.

Fig. 1
figure 1

Traditional approach

Full size image

In comparison, current systems are relying on fog-based processing to reduce latency of data transmission and improve system performance. As shown in Fig. 2, fog-based processing works on the idea that generated data should be processed at devices located in local area network (LAN). The LAN based fog devices are able to acquire and process huge amounts of generated data swiftly. These are able to generate an alarm or send notification to the user in case of detection of any anomaly or discrepancies without involving any delays. The summary of the processed data is transferred to the cloud periodically for further analytics. Hence, fog-based devices work as an interface between IoT devices and cloud or user.

Fig. 2
figure 2

Fog based IoT System

Full size image

In this paper we propose XLAAM, a fog-enhanced smart home architecture for daily activity data monitoring and anomaly detection. As illustrated in the Fig. 3, the proposed architecture comprises of three layers. The first layer is responsible to collect the raw data from the various sensors or IoT nodes which are available in a smart home. Monitoring of the daily activities and their classification with LSTM occurs at the fog nodes of the second layer. Detection of any anomalous activity is also performed along with. If an anomaly is detected, then a health worker or caretaker is informed by some notification or alarm. The daily activity data, after classification, is also sent to cloud-based servers. The cloud layer is responsible for any required data analysis that may not involve time constraints. Moreover, Explainable AI is employed in the cloud layer to obtain the features importance in the developed LSTM model.

Fig. 3
figure 3

XLAAM Architecture for activity monitoring and anomaly detection

Full size image

4 Proposed methodology

The proposed method recognises and distinguishes between normal and anomalous activities in a smart home. In addition, anomalous days are indicated by the number of activities completed in each day and the duration of each activity. We use the Long Short-Term Memory (LSTM) model to recognise activity and use the LSTM autoencoder to detect an anomaly. Anomalies are further classified based on criteria of missing or extra activities and unusual duration of an activity. Boxplots have been utilized to check and verify the anomalous day for the loss that represents the difference between the predicted and the actual values. Additional statistical evaluation of anomaly detection is also performed. We have calculated the anomaly detection rate, also known as the recall. The value has come approximately as 20%. The low value of recall is due to the limited size of the data considered and can be improved by increasing the training data.

The proposed method has been applied to smart home ARAS data sets, which are publicly available [44]. Table 1 summarises the information on this data set with regard to the type of house, sensors, number of residents, age, duration and number of captured activities. In the dataset, binary sensors like photo cells, pressure mattresses, contact sensors, proximity sensors, sonar distance, temperature and infrared sensors/receiver are used for both smart homes. Recorded activities include going out, using toilet, sleep, preparing food, eating, having a snack, dressing, music listening, studying, watching TV, etc.

Table 1 Introduction about datasets [44]
Full size table

Our proposed approach, XLAAM, an eXplainable LSTM-based framework to classify the activities of daily life and detect anomalies within a fog-enhanced smart home, comprises of the following three functions:

  1. i.

    Activity Monitoring

  2. ii.

    Anomaly detection

  3. iii.

    Explainability

4.1 Activity monitoring

The considered data set comprises of data collected from different sensors over days that has been classified and labelled with various activities. This data is split into training and testing data and subsequently, classified incrementally to emulate streaming data. An LSTM network was trained using the first 10 days of data whereby the model learns patterns specific to each class and uses those patterns to predict or classify the activities of 11th day. Subsequently, the model was retrained with 11 days of data to predict the 12th day, and so on. The output of the LSTM model gives exactly one activity as the output class, i.e., the predicted answer is the activity (as a numeric label).

Figure 4 shows the Long Short-Term Memory (LSTM) model [46] as a chain of repeated neural network modules. Each module includes a memory cell, an input gate, an output gate, and a forget gate. Gating control what passes through and what gets blocked. The performance of LSTM is typically superior to that of Recurrent Neural Network (RNN) or the Hidden Markov Model [3]. Gradient descent can be used to reduce training error by adjusting the weight of each iteration based on the error. When we activate the LSTM block, the error is calculated backwards from output to each gate in the input gate stage until it is filtered out. As a result, using a standard backward pass neuron to train LSTM blocks causes it to remember long-term values. LSTMs are capable of looking back numerous time steps and using that information to make accurate predictions about what will happen next. There are many practical applications for this when translating from one language to another. LSTMs work very well even though translation is not just a word for word process, but a phrase to phrase or even sentence-to-sentence process. LSTMs can represent the grammar structures that are unique to each language, and they do so by locating the higher level idea and translating it from one mode expression to the next. LSTMs convert speech to text well [45]. Speech is nothing more than a collection of time-varying signals that can be used to infer the text being spoken and the word that is being said. For a more accurate prediction of what is to come, consider the most recent history of words. LSTMs are an excellent fit for any information that is time-stamped [47]. We used LSTM model for prediction of activity in which we took ten days activities data and predict the eleventh day activity data and so on. The advantages of applying the aforementioned machine learning method will be further discussed in the result section.

Fig. 4
figure 4

LSTM Structure (X: input, O: Output, M: Memory cell)

Full size image

4.2 Anomaly detection

Anomalies are errors in activity execution that can be detected by identifying activities taking place for irregular durations or by detecting deviated activity patterns [48]. Therefore, anomalies are classified into three types: activity instances with unusual characteristics, unusual durations of activities, and irregular frequencies of activities performed in a day. The activities identified from a dataset are analysed to identify anomalies, which are usually rare and unexpected deviations from standard patterns. The proposed system makes use of the LSTM auto-encoder learning algorithm for the detection of anomalies.

LSTM autoencoder is a type of neural network model that attempts to learn a compressed representation of an input. The LSTM encoder-decoder architecture compresses data with an encoder and decodes it with a decoder to keep the original structure. Flow diagram of an autoencoder is depicted in Fig. 5, where X is an input vector, X’ is the reconstruction vector and Z is the latent vector. Encoder is used to compress input vector into a lower dimensional representation, i.e., the latent vector. The Latent vector retains the most important features of the input data. The Decoder is used to decompress it back to the original one. Reconstruction error E (X, X’) is the difference between input data and output data. The goal is to minimise reconstruction error using a loss function, such as the Mean Absolute Error (MAE). MAE can be used to determine the accuracy of various forecasts based on statistics. The absolute error is the difference between the forecasted and actual values. MAE tells us how much error we can expect on average from the forecast. Mean Absolute Error is a metric for evaluating regression models. The average absolute error of a model relative to a test set is equal to the average of the absolute values of the individual prediction errors across all instances of the test set [49].

Fig. 5
figure 5

Block diagram of basic autoencoder

Full size image

The trained LSTM model is used to detect anomalies in the newly identified activities. We calculate the Mean Absolute Error (MAE) using the training data. The reconstruction error threshold depicts the maximum MAE loss value in the training data. A data point in the test set will be labelled as an anomaly if the reconstruction loss is greater than the reconstruction error threshold value.

The ARAS smart home dataset used for implementation consists of binary sensor values and associated activities for each second of the day. The dataset [44] contains data of thirty days containing 86,400 activity records per day. For anomaly detection, the data has been processed and converted into duration wise activity data. This processed data set contains an activity’s label (i.e., Activity Id) and its duration (in seconds). Table 2 shows the description of each activity id. The numerical activity id is used as the response value for the classification algorithm. Table 3 displays the durations, in seconds, of a subset of activities for day 1 of two residents (R1 and R2).

Table 2 Activity ID description [44]
Full size table
Table 3 Duration of activities of R1 and R2
Full size table

Anomalies are detected by calculating the MAE loss in the training set and setting the threshold using MAE as it gives a good representation of how far a test point is from its expected value, i.e., the calculated loss in the training set. We took the maximum value of MAE and set the threshold as reconstruction error. This ensures that the threshold is set higher than the "noise level," preventing false positives. The reasoning behind doing so is that if a smart home resident’s duration of performing an activity lies outside of the threshold values, it can be regarded as an anomaly. For e.g. If the sleeping time is higher than the usual time, anomaly can be detected. Next, we calculated the MAE loss in the testing set. If MAE in the testing set is greater than the threshold, this data is marked anomalous data. This is also demonstrated using a boxplot in a result section. The occurrences of each activity class that fall within the whiskers of the boxplot representations are labelled as normal, while the remainder are labelled as anomalous, where whiskers are Q3 + 1.5(Q) and Q1-1.5(Q3- Q1), and Qi is the ith quartile.

4.3 Explaining the model

An eXplainable LSTM-based framework to classify the activities of daily life and detect anomalies within a fog-enhanced smart home. The explanatory AI refers to artificial intelligence (AI) methods so that human experts can understand the solution results. AI takes some inputs to generate an output while Explainable AI also helps the users understand the impact of input variables on the output. The adoption of AI systems for critical applications such as activity monitoring depends on the people’s ability to trust the AI output. Explainable AI makes AI reliable so that it can be accounted for and make certain that the use of AI will not cause any harm.

We are using LSTM model to predict the activity; however, this model behaves like a black box and it is difficult to understand the how the model made a given decision. We are able to understand how the model is arriving at a decision using Explainable LSTM. We have employed XAI library [50] to make LSTM explainable. XAI is a Machine Learning library built around AI explainability. It includes a number of tools that help with data and model analysis along with their evaluation. The XAI library is built using the three steps of explainable machine learning: data analysis, model evaluation, and production monitoring. XLAAM uses XAI library to show the importance of each sensor in detection of a particular activity. Reliability is important when a ML model’s decisions are related to users’ safety and directly affect their lives. Therefore, the use of explanations is significant in the current work to help users understand the results of the classification model.

5 Results

We use a publicly available smart home dataset [44] that includes two homes with multiple occupants. The dataset contains thirty days of data that includes binary values from multiple sensors labelled with various activities. The dataset has 20 attributes and is divided into 27 separate classes corresponding to the 27 activities such as going out, having breakfast, sleeping, watching TV and studying etc. This class is represented by the activity attribute in the dataset and this attribute is the target label for classification. The activity labels have been mapped to numeric values, 1,2.0.27 before being provided to the LSTM for modelling, i.e., label encoding has been done. One day activities of multiple residents of home A and home B have been shown in Fig. 6. It shows activities on Y-axis and time in seconds on X-axis. Flat line (day1: activity of resident 1 of smart home A) means activity whose number is 11 (Sleeping) going on, so resident of that home A is sleeping at that time.

Fig. 6
figure 6

Activity diagram of resident1 and resident2 of smart home A and smart home B

Full size image

We combined ten days of data of Home A and divided it into 80 percent training data and 20 percent testing data for analysis. Then, as described earlier, a LSTM model is used to predict the value of the testing data. The output of the LSTM model gives exactly one activity as the output class, i.e., the predicted answer is the activity (as a numeric label).

The training and test loss with different epochs are plotted in Fig. 7.

Fig. 7
figure 7

Training and testing loss over different epochs

Full size image

The confusion matrix obtained after applying the LSTM classification is shown in Table 4. The average F1-score obtained is 0.7900592985.

Table 4 Confusion matrix of each activity of R1 of Home A
Full size table

The LSTM Model is evaluated and compared using the performance metric based on mean absolute error (MAE). This represents the absolute difference between the actual and predicted values in the dataset.

$$MAE = \frac{1}{N}\sum_{\dot{i}=1}^{N}\left|\widehat{{y}_{i}}-{y}_{i}\right|$$

where \(\widehat{{y}_{i}}\) is predicted value and yi true value and N is the total number of instances.

The proposed LSTM-based approach is compared with basic four machine learning methods: Naïve Based, Support vector machine (SVM), K-Nearest Neighbour (KNN) and Random Forest (RF). The performance of various methods is evaluated based on MAE. It is observed that the MAE of the proposed approach is low in comparison to others, as shown in Fig. 8.

Fig. 8
figure 8

Activity Prediction of Home A

Full size image

The proposed LSTM-based approach is also compared with other models more suited to time series data: Auto Regression Integrated Moving Average (ARIMA), 1D Convolutional Neural Network (1DCNN) and Recurrent Neural Network(RNN). In comparison to other approaches, the MAE of the proposed approach is low, as shown in Fig. 9. Improved results of LSTM are due to its ability to capture long-term temporal dependencies and learn long-term sequences of observations.

Fig. 9
figure 9

Activity Prediction of Home A

Full size image

The proposed approach, XLAAM also identifies any anomaly if it happens using the LSTM encoder. The ARAS data set has been processed and converted it into duration wise activity data. This data set contains an activity’s label and its duration (in seconds). The anomaly score threshold is set equal to the maximum reconstruction loss during the final stages of training [51, 52]. Figures 10 and 11 illustrate the test loss vs threshold for days 1–10 and 11–20 respectively. The anomalies are detected on 10th day and 16th day, as can be observed from Figs. 12 and 13.

Fig. 10
figure 10

Test loss Vs Threshold [Home A_R1_1to 10 Days]

Full size image
Fig. 11
figure 11

Test Loss Vs Threshold [Home A_R1_11 to 20 days]

Full size image
Fig. 12
figure 12

Anomalous Activity Detection on 10th day

Full size image
Fig. 13
figure 13

Anomalous Activity Detection on 16th day

Full size image

We use boxplot to verify the occurrence of anomalous data. The data that fall within the whiskers of the boxplot representations are labelled normal, while the rest are labelled anomalous, where the whiskers are Q3 + 1.5(Q) and Q1-1.5(Q3- Q1), and Qi is the ith quartile. Test loss vs. Threshold is depicted in Fig. 10, while same loss is shown by the boxplot in Fig. 14.

Fig. 14
figure 14

Boxplot representing Test Loss

Full size image

While activity recognition and anomaly detection are functions of fog layer due to their low latency requirements, interpretation of the developed LSTM models is performed at the cloud servers. Figures 15, 16, 17, 18, 19, 20 illustrate the importance of each sensor input to the model for recognition of a few activities in the considered data set. It can be observed that the value obtained from sensor Fo3 is the most important feature for activity 1 prediction, the value obtained from sensor Fo1 is the most important feature for activity 2, and so on.

Fig. 15
figure 15

Activity number 1

Full size image
Fig. 16
figure 16

Activity number 2

Full size image
Fig. 17
figure 17

Activity number 3

Full size image
Fig. 18
figure 18

Activity number 4

Full size image
Fig. 19
figure 19

Activity number 7

Full size image
Fig. 20
figure 20

Activity number 27

Full size image

Observing the importance of each sensor input helps the system users to understand how the LSTM model is making classification decisions.

6 Conclusion

The paper has presented a novel deep learning based approach to human activity recognition using a LSTM-based framework. Data collected from IoT nodes is processed at fog nodes present in the local network itself to improve the response of the system in comparison to transferring it to the cloud servers for processing. It has been found that the LSTM-based framework outperforms various machine learning models by reducing the Mean Absolute Error. Further, anomaly detection has been performed by an LSTM autoencoder and verified with boxplot graphs. The fog layer ensures that any anomalous activity triggers an alarm to a user, such as the health-care worker, for immediate action. The entire data is also streamed to the cloud layer for interpretation or explainability of the developed LSTM model. An explainable LSTM model helps in building a reliable system that enhances the confidence of the system users in comparison to a black box implementation. The proposed approach, XLAAM can be employed for developing an effective health-care system in a fog-enhanced smart environment.