Keywords

1 Introduction

Currently, 703 million people over 65 years of age inhabit the world. The number of elderly people is projected to double up to 1.5 billion in 2050. Globally, the share of the population aged over 65 has increased from 6% in 1990 to 9% in 2019. This proportion is projected to rise further to 16% by 2050, so that one in six people in the world will be aged 65 years or older [1]. Given the existence of advanced technologies that could improve the living conditions and independence of these people, we face the challenge of creating new models of healthcare technology.

As the average age of the population continues to increase, a large number of associated diseases are becoming more prominent, such as dementia, hypertension, diabetes, gait problems, etc., which mainly affect elderly people. This work has focused on the Prevention of Pressure Ulcers (PPU), an age-related complication in people with low or no mobility which is defined as injuries located in the skin or underlying dermal tissue, usually over a bony prominence, as a result of pressure [2]. They deeply affect the quality of life of patients due to reduction of independence and self-esteem, and even causing death in extreme cases [2].

Changing posture and maintaining a correct body position is the best and most efficient tool to prevent the appearance of PPUs [3]. In any case, a given in-bed position must not be held for more than two hours [4]. Change of posture is the main method of prevention and control. However, carrying out these practices continuously for a long time is an arduous task for caregivers, causing physical and emotional stress [5]. Regarding the cost of treatment, a study on the cost analysis of PUs [6] concluded that cost increases with the severity of the ulcer because it requires a greater number of healing processes, in addition to expenditure on materials, nursing time and hospitalization costs for the healthcare system. Investing in techniques to prevent these situations before they occur is widely accepted to be of paramount importance among the health and technology community.

The contribution of our approach is focused on the detection of in-bed positions by means of a smart system where convolutional neural networks (CNN) learn from the fuzzy visual representation of data from pressure mat sensors. Namely, non-intrusive pressure mat sensors are embedded in a smart-blanket within a smart home environment. This device is a more attractive option than other kinds of devices for recognizing posture because it enables the privacy of the person to be preserved at all times. Moreover, the body of the patient can be covered with sheets and blankets given that the sensor is deployed under the patient.

The remainder of the paper is organized as follows: in Sect. 1.1 we offer a review of works related to our proposal, Sect. 2 presents the proposed methodology to classify in-bed human posture for the prevention of pressure ulcers using pressure mat sensors, and Sect. 3 introduces the results of a dataset evaluated using our approach. Finally, in Sect. 4, conclusions and ongoing works are discussed.

1.1 Related Works

In recent years, academic and industrial research initiatives on smart systems have increased rapidly worldwide [7]. In order to improve quality of life for the elderly, different home-based detection technologies are used to monitor their activity. In this way, we can provide personalized services and support future health care demands, complementing specific medical services [8].

In the context of PU prevention, in [9] authors analyze the effectiveness of the use of special pressure management surfaces in the prevention and treatment of PU patients via authorized access to patient photographs and records. In [10], an integrative literature review is carried out by identifying documents in the main databases related to health sciences using reverse search in the identified articles, including prospective studies, systematic reviews or other types of studies that evaluate the effectiveness of postural changes.

In [11], a commercial pressure mapping system is used to create a time-stamped, whole-body pressure map of the patient. An ad hoc image-based processing algorithm is proposed to keep an unobtrusive and informative record of the in-bed posture of patients over time. In [12], a bedsore (PU) monitoring method using Pictorial Structure models is introduced to localize pressure distributions of the body. A pilot study including 12 subjects reveals that the proposed method enables reliable localization of body parts with 89.8% accuracy in common lying postures. In [13], a sensor matrix made of a multi-walled carbon nanotube (MWCNT)-polydimethylsiloxane (PDMS)-composite was fully printed to enable pressure distribution measurements. A final application was a printed insole manufactured with six single MWCNT-PDMS pressure sensors that were situated on characteristic points of the insole to detect unhealthy rollover patterns. Another specific software application for PU management [14] was developed and used for assessing its acceptability, usefulness and applicability by professional nurses from three residential homes for the elderly over a one year period. Three centres and 69 residents took part in this project. During this process, 27 nutritional status screens, and 22 PU risk assessments, as well as the monitoring of 230 PUs, were performed.

In another study conducted by Dresch [15], the measured area of venous ulcers was compared using the software AutoCAD (r) and Image Tool. It was concluded that both software applications are suitable to measure ulcers; however, their accuracy decreased dramatically when the wounds had an \(area <10\) cm\(^2\). In [16], the authors present an electronic prototype to assist patients in the process of PU rehabilitation. The system identifies body movements and positions using a video device located in front of the bed. The results indicated that the prototype has 95% reliability but makes mistakes and activates false alarms in low lighting conditions or vibrations.

The proposed methodology aims to gather the best features of previous works, and in turn, as mentioned earlier, to offer a degree of privacy which is not provided by traditional camera methods, thus providing more positive acceptance than visible spectrum cameras [17].

2 Methodology

In this section, we detail the methodology proposed to classify in-bed human posture for the prevention of pressure ulcers using pressure mat sensors. To this end, we propose a straightforward data collection method to reduce system deployment time in the sensitive context of patients with pressure ulcers.

From the data collected from the pressure mat sensors, we first provide a visual representation using fuzzy processing from raw data to grayscale. Second, we include ad hoc data augmentation techniques to increase the dataset, generating new synthetic images by means of geometric transformations. Third, we evaluate 2 types of CNN to analyze the impact of the number of layers and kernel size on learning in-bed human posture. Finally, the classification of in-bed human posture enables real-time computation of the time each position is maintained, which allows caregivers to be notified when the time limits are reached.

In Fig. 1 we show the architecture of components described in the approach, which are addressed more in depth in the next sections.

Fig. 1.
figure 1

Architecture of the components proposed in the methodology: limited dataset from pressure mat sensors, fuzzy visual representation, data augmentation and learning from CNNs.

Full size image

2.1 Limited Dataset from Pressure Mat Sensors

The use of Deep Learning approaches requires a large amount of data [18]. However, in many contexts, obtaining a labelled dataset for learning purposes is an arduous task which interferes with the real deployment and configuration of intelligent systems in home-based health solutions. To overcome this limitation, we propose the agile collection of a limited amount of data that will be processed using augmentation methods to provide high performance in deep learning approaches.

The dataset evaluated in this work has been obtained from [19], based on data collected in experiment II. The data collected proceeds from the pressure mat sensor Vista Medical BodiTrak BT3510. This device provides a pressure data matrix of points \(M^{w,h}\) whose size is \(w=27,h=64\). Each value \(m_{i,j}\) of the matrix \(M^{w,h}\) represents a point of pressure in the range [0, 500]. A picture of Vista Medical BodiTrak BT3510 is provided in Fig. 2.

Fig. 2.
figure 2

Left, the Vista Medical BodiTrak pressure mat sensor selected to collect the dataset is shown. Right, the in-bed postures based on the prevention of pressure ulcers and position labels are described.

Full size image

In this limited dataset, only 28 positions were collected for each of the eight participants. Three positions were labeled in the dataset: supine, left and right, together with the data matrix for each experiment from the pressure mat sensor. Moreover, the dataset includes several configurations for each basic posture based on the prevention of pressure ulcers, which is shown in Fig. 2.

This dataset has been selected to include ad hoc in-bed positions for pressure ulcers and contains a small number of samples for learning purposes (232 samples).

2.2 Fuzzy Processing of Pressure Mat Sensor Data

The data collected by the pressure mat sensor represents human body pressure on the surface of the sensor mat which is defined by a matrix of points \(M^{w,h}\). In order to provide a visual representation from the raw data, fuzzy processing into grayscale values has been proposed.

Thus, we define a fuzzy set to represent relevant body pressure in fuzzy color [20] by means of a membership function \(\mu _{M}(m_{i,j})\), which relates the pressure values \(m_{i,j}\) to a degree of relevance between 0 and 1 \(\mu _{M}(m_{i,j}): R \rightarrow [0,1]\), where 0 represents absence of pressure and 1 maximal pressure. A trapezoidal right function has been selected to determine the shape of the membership function which is defined by a lower limit \(l_1\) and an upper limit \(l_2\).

$$ \begin{array}{r} \mu _{M}(m_{i,j})=TR([l_1,l_2],m_{i,j})\\ TR([l_1,l_2],x) = {\left\{ \begin{array}{ll} 0 &{} \text {if } x < l_1 \\ (l_2-x)/(l_2-l_1) &{} \text {if } l_1 \le x \le l_2 \\ 1 &{} \text {if } x > l_2 \\ \end{array}\right. }\end{array} $$

In addition, a fuzzy modifier \(\alpha \) is proposed to model the progression of the linear function \( TR([l_1,l_2],x)^\alpha \) [21]. If \(\alpha _m<0\), we obtain a weak modifier, such as light; and a heavy modifier when \(\alpha _m>0\), such as strong.

The aim of including fuzzy data processing in pressure values from the sensor mat is to provide: (i) a filter for non-relevant information and (ii) the reduction of noise from non-feasible values [22, 23].

In Sect. 3, we describe the parameters and integration of fuzzy processing with the rest of the system components. In Fig. 3, we show an example of fuzzy representation of the pressure data matrix from the pressure mat sensor.

Fig. 3.
figure 3

Left, example of data matrix for supine position. Right, visual representation of the left data using fuzzy representation in grayscale.

Full size image

2.3 Data Augmentation

Collecting data on different orientations and positions in patients with low or no mobility with the aim of increasing learning performance is inappropriate due to the sensitive health conditions of the patients. Fortunately, the data augmentation method in [24] provides a solution to enlarge the number of learning cases from a limited dataset [25] to generate longer synthetic images and therefore reduce over-fitting [26].

In this work, we have applied the following geometric transformations to augment and enlarge the image data from the original dataset:

  • Translation. The original image is relocated within a maximal window size \([t_x,t_y]^+\) using a random process which generates a random translation transformation \([t_x,t_y], t_x \in [0,t_x^+], t_y \in [0,t_y^+]\).

  • Rotation. A rotation transformation is defined by a maximal rotation angle \(\alpha ^+\), which generates a random rotation within an angle \(\alpha \in [0,\alpha ^+]\) which is applied in the center of the image.

  • Scale. A maximal scale factor \(s^+\) is defined to generate random scale \(s \in [1-s^+,1+s^+]\), reducing the impact of the height of the training sample.

We note that flipping the images horizontally is not suitable in this context, due to the need to differentiate between left and right positions in the classification carried out by the convolutional neural networks. Likewise, as the pressure sensor mat is oriented in a given position on the patient’s bed, vertical flipping is also not recommended.

2.4 Design of the Convolutional Neural Network

In this section we describe 2 types of Convolutional Neural Networks (CNN) to classify the in-bed posture of patients. In the dataset there are three target classes for the CNN to learn: supine, left and right positions.

Based on the visual information of the images, the CNNs learn kernels which are related to key visual patterns, whose similarity with source visual regions is defined by a convolutional operation. They enable us to summarize and reduce the spatial information defined by a bi-dimensional kernel size whose convolutional size is defined by \([K_x, K_y]\). So each convolutional layer i is defined by the number of kernels L and its size \([K_x, K_y]^l\), represented as \(K_i=[K_x, K_y]\times L\). In addition, some methods/sub-layers are usually included to configure the network and reduce the number of parameters: (i) dropout reduces the number of network nodes to reduce over-fitting [27], (ii) pooling provides spatial reduction in the size of the outputs within the layers, which is defined by a window size \([P_x,P_y]\) and an aggregation function, such as maximal or average, [25] and (iii) activation function filters the last output layer to train the neural network several times faster [26].

In this work, two types of CNN configurations are evaluated to classify in-bed human posture: (i) X and (ii) Y. In Table 1, we describe the layers of the two CNN models.

Table 1. Convolutional Neural Network configurations to classify in-bed human posture from augmented pressure mat sensor data
Full size table

3 Results

In this section, we describe the results obtained from the proposed methodology applied to a dataset of in-bed posture derived from a pressure ulcer prevention program to classify sleeping positions.

3.1 Dataset Description

For the purposes of evaluating the proposed methodology, we start with the dataset developed in [19] (experiment II), which was collected by the device Vista Medical BodiTrak BT3510.

This experiment was carried out with 8 participants aged between 19,34. The height and weight of the participants were in the range of [169,63] and [186,100], respectively. In experiment II, a sponge pressure sensor mat was used to collect data on the in-bed posture of the participants. The sampling rate was 1 Hz and each file includes the average pressure of around 20 2-min frames (around 120 frames). Supine posture was defined by 8 different positions. Left posture included 3 different positions and right posture also included 3.

For evaluating purposes, we include a leave-one-participant-out cross-validation which leads to arduous testing given that unseen participant data are used to evaluate models generated for other participants, without including personalized learning.

3.2 Dataset Pre-processing: Fuzzy Visual Representation and Data Augmentation

In this section, we describe the data processing from raw values as points of pressure on the mat sensor in order to generate visual augmented images which configure the learning of convolutional neural networks.

First, a trapezoidal function has been used to define the membership degree of relevant body pressure. We have defined the membership function based on expert criteria using the minimal and maximal threshold \(l_1=50, l_2=400\). To smooth the impact of body and joints on pressure values, a fuzzy modifier \(\alpha =1/4\) is applied to compute the final membership function: \(TR([l_1,l_2],m_{i,j})^\alpha \). In Fig. 3, we show an example using this fuzzy visual representation.

Second, we have applied the following geometric transformations for augmenting and enlarging the image data from the original dataset: a translation defined by the window size \([t_x,t_y]^+=[5,5]\), a rotation defined by a maximal rotation angle \(\alpha ^+=\pi /5\) and, a scale defined by a random scale in the range \(s \in [1-s^+,1+s^+]=[0.8,1.2]\). An augmentation ratio of 25 images per original one (232 images) provides a suitable dataset (6.032 images) for deep learning purposes.

3.3 Evaluation of Convolutional Neural Networks

In this section, we evaluate the two proposed convolutional neural network models. Thesss configuration of the two CNN models was detailed in Sect. 2.4. The learning stage was configured over 50 steps defining a non-extensive learning process with an average duration of 5m43s in model A and 8 m 27 s in model B (using a i5 CPU and 8 GB RAM computer).

In Table 2, we show the results of evaluating the convolutional neural network models, where precision, recall, f1-score and accuracy metrics were computed for each participant and model. In addition, in 4 we show the confusion matrices of the aggregated results obtained from participants for each model.

Table 2. Table summarizing the results of single and multi-occupancy data with raw and fuzzy representation for the best configuration \(CNN_3\). In Fig. 4, we also include a confusion matrix for the best models in single and multi-occupancy.
Full size table
Fig. 4.
figure 4

Confusion matrix for aggregated data from participant in models A (left) and B (right).

Full size image

Based on the results described in Sect. 3.3, we observe that both models provide encouraging results with accuracy and f1-score of up to 98%. This is sterling performance given the highly demanding learning context and validation process: (i) a small number of samples was used as training data, (ii) a leave-one-participant-out cross-validation with unseen user data was used for evaluation.

There are no significant differences between the performance of models A and B, which both describe an agile CNN model with quick learning capabilities which can also be integrated into embedded devices [28] to provide real-time evaluation in the edge-computing of smart mat devices.

4 Conclusions and Ongoing Works

In this work, in-bed posture classification models from non-invasive pressure mat sensor data have been proposed and evaluated for the prevention of pressure ulcers using Convolutional Neural Networks. First, fuzzy processing from raw values provided a straightforward visual representation. Second, we enabled the generation of a large dataset from limited data using ad hoc data augmentation.

Two convolutional neural network models were evaluated from a dataset which included ad hoc in-bed positions for pressure ulcers and contained a small number of samples for learning purposes. Based on the results, data processing with fuzzy logic and augmentation has enabled suitable data learning from agile CNNs with 2 and 3 convolutional layers.

In future works, we will focus on collecting a dataset with a wide range of participants with low mobility to evaluate the approach in a real context. In addition, we will integrate the classification into a fuzzy rule-based system to define the temporal limits for each in-bed position of patients in an interpretable and flexible way, enabling real-time notifications to caregivers.