1 Introduction

The elderly population increases considerably with changes in the quality of life and the various support services. In 2000, there were already 420 million people with more than 65 years old (about 7% of the world population), and the statistics estimated that this number will reach 1500 million (about 16% of the world population) in the 2050 (Steg et al. 2006; Destatis 2011). The fall and injuries that result are a major health problem for the elderly. National Safety Council (NSC) estimates that the highest mortality rate among persons over 65 years is due to falls.

Heart rate is an important factor for cardiovascular disease. It is also related to an increase in mortality due to heart failure to the elderly (Fox et al. 2007; Maddox et al. 2008). Heart rate monitoring and falls detection can give a good indication of the health status of the elderly. This information can help to provide the necessary medical service. The rising cost of healthcare requires an adapted method for reducing hospital readmissions. Home monitoring in real-time or near-real is the solution to follow the elderly in their home and send information to the doctor.

The monitoring systems of elderly people in their environment provide several services (Foko et al. 2013):

  • extend the time people can live in their preferred environment by increasing their autonomy;

  • support maintaining health and functional capability of the elderly individuals;

  • promote a better and healthier life style for individuals at risk;

  • support Caregiver, families and care organizations;

  • increase the efficiency and productivity of used resources in the ageing societies.

This paper proposes a multimodal monitoring system of elderly in their environment which provides three ambient assistance services for fall detection, cardiac problem detection and the location of elderly. This system included also an emergency service that provides data fusion of the three ambient assistance services and sends detailed information about the condition of the followed person to the caregiver.

In this paper, we review the state of the art of the technologies involved in the field in Sect. 2, then, we present our proposed architecture in Sect. 3. The methodology of our architecture and its hardware and software implementation is presented in Sect. 4. In Sect. 5, the model with CPNTools is shown. The experimental results and system evaluation are discussed in Sects. 6 and 7. Sections 8 and 9 give the concluding remarks and identifies areas for future work.

2 Related work

In recent years, the continuous monitoring of the elderly has become an important factor as they prefer to live freely in their environment but always remain in safe. In this section, we will present a state of the art in this field.

2.1 Heart rate monitoring

Today, there are a growing number of technological devices on the market to help and save people reached a cardiac disease. Continuous monitoring of heart rate can reduce the risk to return in cardiac failure. To achieve this goal we will need to the heart rate detectors, a detector based on a piezoelectric sensor (Torres-Pereira et al. 1997), a watch (Segerståhl and Oinas-Kukkonen 2011), a pulsometer placed at the finger (Huang et al. 2013), a belt placed around the chest (Tetzlaff et al. 2014; Rotariu et al. 2011), and an optical detector based on the calculation of the blood volume of the finger (Miah et al. 2013).

The first researches in this domain have been designed especially for athletes; Pike et al. (2012) has developed a system which supervises the heart rate of an athlete during exercises using a pulsometer placed at the finger; and sends collected information to a bracelet which displays a color code using LEDs. So that, the athletes knows if he will slow down or accelerate his speed. Another study was made on a parachutist (Hermans et al. 2005), by supervising his heart rate, its altitude to know its position during jumping.

Researchers in this field have developed systems to detect and prevent cardiac failure in sleep. Bradley et al. (2005) proposed an intelligent cushion to monitor heart rate in sleep, and another study (Furman et al. 2008) is based on the detection if a driver falls asleep at the wheel. Valle et al. (2008) tested the heart rate monitoring system on 166 patients who have cardiac failure to reduce cardiac arrest.

Monitoring the heart rate of the fetus and his mother is also important; Ahmed et al. (2002) have developed a portable device for long-term monitoring of the heart rate of fetus and his mother. Yang et al. (2014) and Nageotte (2015) monitor the heart rate of the fetus using a portable stethoscope.

Recently, several researches have been developed for monitoring heart rate by developing Smartphone applications (Wagner et al. 2012; Diab et al. 2013). Lee et al. (2017) validated four Smartphone applications using data taken by a Holter monitor. They compared these values with the values taken by Smartphone applications that measure heart rate during physical exercise and found a 95% confidence interval.

In addition, Smartphone applications are based on two methods: contact-based method and non-contact-based method (Coppetti et al. 2017). Contact-based method is when the subject places a finger on the built-in camera of the phone directly on the skin and the built in flash provides the necessary light source in the visible range for reflection by blood cells. Non-contact-based method use the camera in the classical way by holding the camera in front of the patients face without the need for direct skin contact. Yan et al. (2017) proved that the contact-based method gives information closer to the value taken by a pulsometer than the non-contact-based method.

2.2 Fall detection

In the last years, there has been a lot of research work in the field of daily life activities monitoring for the elderly. Fall can be defined as a sudden and unexpected change of the body position in which static and dynamic balance mechanisms fail and the voluntary responses are inadequate to correct the lack of balance (Fortino and Gravina 2015); we can divide the fall depending to the scenarios to four types:

  • Fall from walking or standing.

  • Fall from sitting.

  • Fall from sleeping.

  • Fall from standing on support.

Detection of fall can be even divided according to the sensors used in three methods: ambience device, camera-based and wearable device (Mubashir et al. 2013; Van De Ven et al. 2010). Fall detection methods are depicted in Fig. 1.

Fig. 1
figure 1

Fall detection methods

Full size image

2.2.1 Ambience device method

Ambient sensors monitor people in a closed environment (care home, Intelligent Habitat for Health, etc.). Among these sensors, pressure sensors installed in the carpet is called “Smart floor”, to track the trajectory of people in a Habitat equipped by it and detect the fall (Alwan et al. 2006; Klack et al. 2010; Lauterbach et al. 2013).

There are other ambient sensors for detecting the fall; such that the work of Li et al. (2014), have developed a fall detection system based on the audio, using a microphone installed in home. Another example is the work of Humenberger et al. (2012); they use optical sensors sensitive to intensity of light related to the change of body posture.

The smart floor is a good fall detector, although the cost of his installation is very high and it does not make the difference between the monitored person and others in Habitat.

2.2.2 Camera-based method

The camera is a multiple function tool as: tracking trajectories, detecting the fall ... etc. Ozcan et al. (2013) proposed a fall detection system based on a camera placed to the person belt; this system was tested in a closed and open environment with all three types of fall.

Other systems (Yu et al. 2013, 2012) used one or more cameras installed in Habitat to monitor the person, and from the analysis of human body shape deformation (Rougier et al. 2011; Yu et al. 2010) and calculates the volume distributed around the vertical axis of the body (Auvinet et al. 2011), they can detect the fall. Choi and Youm (2017) detects falls of elderly living alone using only image information obtained from a low-cost camera module without using a separate sensor or equipment in order to guarantee the freedom of foreign objects and movement to the monitoring object.

Vision sensors (camera) have a very high cost and very limited coverage area. In addition, privacy and life style of individuals is hindered.

2.2.3 Wearable device method

The method is based on sensors worn by the individual for detecting posture or body movement of patient to detect fall. Researchers have developed the posture detection systems based on a 3-axis accelerometer (Bourke et al. 2010; Vallejo et al. 2013). This research use data from three axes of human body with the aid of an accelerometer attached at the belt (Kangas et al. 2012; Chen et al. 2011), or a gilet (Bourke et al. 2008; Baek et al. 2013) worn by the patient. Wang et al. (2014, 2017) proposes a low-power fall detector using tria-xial accelerometer and barometric pressure sensing to optimize false alarms. Another method is to collect information of the three axes of human body and the angle between the vertical axis and the floor (Liu and Lockhart 2014, Ye and Xiang-Yu 2013; Lemay et al. 2013; Charlon et al. 2013).

Other systems use integrated accelerometer in Smartphones (Abbate et al. 2012; Aguiar et al. 2014; Colon et al. 2014) and sends message or email in case of fall detection. However, Hakim et al. (2017) detects human fall utilizing the built inertial measurement unit sensors of a smartphone attached to the body with the signals wirelessly transmitted to remote PC for processing used the classify activity daily living.

2.3 Heart rate monitoring and fall detection

There exist systems that detect the heart rate and fall, as the work of Chan et al. (2013). He developed a patch composed of two electrodes and a tri-axial accelerometer. In the work of Zhou et al. (2014), it is presented a new device worn on the wrist that detects the heart rate and the fall using a 3-axis accelerometer. On the other hand, Khawandi et al. (2013) used a webcam for fall detection and a belt for the heart rate, Valenti and Westerterp (2013) and Hui (2010) detect just the movement of the person with his heart rate.

In this paper, we propose a system that detects simultaneously the heart rate and the fall from walking or standing, in addition, it checks if there was a heart problem at the time of fall or a fall at the time of heart problem detection and it detects also the location of the patient in Intelligent living. This system collects the complete information about the state of the person and sends it to the doctor.

3 Proposed architecture

The elderly often suffer from heart problems, which are sometimes the cause of a fatal fall. The Objective of our work is to monitor continually the health status of old people to detect their different disorders and rescue them in case of emergencies.

Figure 2 shows a general view of our architecture. It is composed of two parts; Input modalities (Absolute acceleration of the body, Heart beat per min and Room number) and Fusion Center (Ambient assistance services and Emergency service).

Fig. 2
figure 2

Proposed architecture

Full size image

3.1 Input modalities

We take the data generated from heterogeneous sources: Absolute acceleration of the body, Heart beat per min and Room number. These three data can be taken from a continuous monitoring of daily activities of a person living alone.

3.2 Fusion center

Our architecture is based on a centralized fusion (Gravina et al. 2017). The input modalities send the information to a processing unit (Computer). This unit processes the data of each modality separately at the feature-level to ensure tree ambient assistance service: Fall detection service, Heart trouble detection service and Location service. The fusion of the collected data by these services triggers an emergency service. With Decision-level fusion, this service classifies the emergency order and produces according to the statue of the person: a call to the emergency center or to the doctor, it sends a message to the patient’s phone ... etc. Our system keeps a historic record because elderly and people with disabilities are often afraid or ashamed to report falls.

4 Methodologies and models background

The proposed architecture is dedicated to elderly people living alone; the implementation of this architecture is made in an environment consisting of four pieces: a bathroom, a kitchen, a bedroom and a living room. Figure 3 shows the proposed environment with the sensors placed in the habitat (part A) and worn by the person (part B) such as: the Accelerometer, the Pulse Oxymeter and the Presence detectors. We propose to use the least cumbersome sensors because the elderly prefer to be free in their lives and at the same time in safe.

Fig. 3
figure 3

Health Intelligent Habitat

Full size image

4.1 Fall detection service

Our system detects the fall for only one single scenario (from Walking or Standing). We choose to use the Wearable device method so we detect the fall from a tri-axial accelerometer placed discreetly at the belt that is sufficient to respond at this scenario. To complete the information we check with the patient by sending him a message before sending an alert.

Based on the cost and availability of sensors on the market we choose the Accelerometer 3-axis (ADXL362). It is placed at the belt of the person and measures the absolute acceleration (At). For each 500 ms there is a data record At (Makhlouf et al. 2017).

Data from a three-dimensional accelerometer is indicated in terms of three orthogonal axes (X, Y, and Z). The absolute acceleration (At) is obtained from the calculation of the root of the sum of the square of three axes of the accelerometer (Ax, Ay, Az) (Shinde and Chawan 2014).

$$\left| {{A_T}} \right|=\sqrt {\left( {A_{X}^{2}+A_{Y}^{2}+A_{Z}^{2}} \right)}$$

In a fall, we detect three successive states: free fall, impact and extended position (Fig. 4):

Fig. 4
figure 4

Acceleration prototype of a fall

Full size image

  • free fall (T0 < At < T1): the fall begins with a free fall phase where the acceleration is less than 1G (Bourke et al. 2007). The lap times are between 300 and 500 ms;

  • impact (T1 < t < T2): when the body reaches the ground with an acceleration greater than 3G (Chen et al. 2006);

  • extended position (t > T2): the body remains lying with an orientation close to 90° relative to the start of free fall (T0).

The definition of the fall presented in this section is based on the absolute acceleration values of a standard fall from standing or walking. This service is based on unsupervised approach.

4.2 Heart disorder detection service

Our choice is to use the Grove-Ear-clip Heart Rate Sensor, it placed at the ear of the person. The measured information is the number of heart beats per minute (Fc).

Taking the pulse allows to evaluate the heart rate regulation in a simple way, that is mean, the heart rate (heart beats per minute) and pulse amplitude.

The evaluation of the person general state can monitor the heart disease evolution, prevent and/or detect a complication (rhythm disease). Without heart disease proved, it can still remove some anomalies in the frequency outside the norms.

  • Bradycardia: decreased heart rate;

  • Tachycardia: increased heart rate;

  • Cardiac arrest: no pulse.

The Table 1 shows the standards of heart rate for a person in good health (Bauer et al. 2008).

Table 1 Heart rate (Standards)
Full size table

4.3 Location service

The location is additional information and as we made our tests in a house with only four pieces, we propose to use two photoelectric sensors (two emitters and two receivers) placed at each door of the habitat. The role of this sensor is to detect a target, which can be an object or a person using the light beam. Its two basic components are therefore an emitter and a light receiver. The detection is carried out when the target enters the light beam and sufficiently alters the amount of light received by the receiver (Lai et al. 2014), to cause a change of state of the output.

Two photoelectric sensors (two emitters and two receivers) placed at each door of the habitat, to detect the movement of the person from a room to another. One detector (emitter-receiver) placed on the door frame on the side of the hall and the other placed on the side of the piece. Its output is 0 if the person passes through the door where it is placed or it is always 1. We said that the person is crossed the door of the piece if and only if both detectors are set to 0 for two successive samples. The retrieved information is the number of the room where the person is located (Rnb) and is recorded every 500 ms.

4.4 Emergency service

Our proposed architecture is based on two levels of fusion (1) feature-level fusion, we take the data from the orthogonal three axis of the body to detect the fall and heart rate to detect cardiac disorders; (2) decision-level fusion, we use the method of rules (IF THEN ELSE) and send to the doctor the final result.

The information sent to the doctor is shown in Fig. 5, includes the detection of falls and heart disorders with the location of the person being followed.

Fig. 5
figure 5

Functional diagram

Full size image

Our system is based on a centralized approach. The Accelerometer, the Pulse oximeter and the eight Photoelectric sensors send the data of absolute acceleration, heart beats per minute and the room number respectively to a central processing unit (computer). The “feature-level fusion” will be done at this unit to detect the fall, heart disorder and location of the followed person. The “decision-level fusion” use the rule method (IF THEN ELSE) to generate the adequate decision. If the system detects a fall, it checks for the existence of a heart disorder and it extracts the final information: Fall with Heart disorder or Fall without Heart disorder. The same for the heart disorder detection.

5 Modelisation

For modeling this architecture, we used the simulation tool CPNTools (Computer Tool for Colored Petri Nets). CPNTools (Jensen and Kristensen 2009) was developed at the University of Aarhus in Denmark; it is used by a large community of academics and industrialists which allowed its development. The specification can be in the form of hierarchical networks, which facilitates the description and understanding (Milner 1997).

The tool offers two modes of simulations: the step-by-step simulation and automatic simulation. In each mode, the graphical interface assists the user in the comprehension of behavior specified by an activation system by coloring components. Stop criteria can be specified to control the automatic mode. Automatic verification generates a trace in text format, but a mechanism for extending the tool associate user functions with system state changes in a given trace. These extensions must be written in the language SML (Standard Meta Language) (Makhlouf et al. 2015). Our simulation is performed with the version of CPNTools v4.0.0 1.

1http://cpntools.org/.

Our architecture implementation on CPNTools is shown in Fig. 6. We used the hierarchical network based on four essential modules: Fall detection Service, Heart trouble detection Service, Location Service and Emergency Service.

Fig. 6
figure 6

Hierarchical network model

Full size image

  • Fall detection service This module is responsible for monitor the daily activities of people. As shown (Fig. 6), it receives the information of the absolute acceleration from the tri-axial accelerometer placed at the belt of the person.

The places “Read Ax”, “Read Ay” and “Read Az” send as data the number of the sample n with the value of the accelerations along the three orthogonal axes (X, Y and Z) of the human body. For n = 2, Ax = 0.93 g, Ay = − 0.02 g and Az = 0.34 g.

The transition “Fall detection service” represents a sub- network that receives in input the data Ax, Ay and Az and sends it at any sample n the detection of fall or not.

  • Heart trouble detection service From a continuous monitoring of the heart rate with a pulse Oxymeter, this module can detect heart disorders.

The place “Read Fc” transmits the number of heart beat (Fc) in simple n (for n = 2, Fc = 74beat/min).

Sub- network represented at the transition “Heart trouble detection service” receives this data and send in outputs the detection of cardiac disorder in case of existence.

  • Location service From the eight presence detectors placed two by two at each door of the home, we can detect the room where the person is located.

The place “Living room input” sends the number of the sample n and the values of the two presence sensors placed on the living room door (for n = 2, C1 = 1, C2 = 0). On the other hand, the values of the sensors placed on the other rooms doors represented by the place “Bedroom input”, “Kitchen input” and “Bathroom input” places equal to 1 for the same sample n = 2, which means the person being followed is crossing the living room door.

The transition «Location service» receives as input the values of all sensors in the same sample and transmits as outputs the location of the person.

  • Emergency service From the data sent by the Accelerometer, the Pulse Oxymeter and the Presence detectors and after a fusion this module is in charge of transmitting the information to the doctor.

The transition “Fusion” collects the information and sends it to a place “n At Fc Rnb” in a single message for example n = 1, Fc = 51beat/min, At = 1.23 g and the person in the living room. This place sends this message to the sub- network represented by the transition “Emergency service” that sends out a code with the health status of the monitored person and its location in the home.

5.1 Fall detection service

Figure 7 represents the fall detection module. As it is shown, at the beginning there is the reading of absolute acceleration (At) from the data collected by the tri-axial accelerometer (Ax, Ay and Az) placed on the belt of the followed person. The place “Treatment” receives this data in packet, as indicated on the arc “getPackets1”.

After the fall detection, our system counts the time to know if the person is still on the floor after two minutes from the moment of the fall. In this case a sub-network “Person on the floor” will load to send the information. In another case, as shown in Fig. 7 (after simulation) for the simple n = 544 the person fell and for n = 674 it got up (544, 674, “Person Stands up “).

Fig. 7
figure 7

Fall detection service module

Full size image

5.2 Heart rate trouble detection service

Figure 8 shows the heart trouble detection module. After the information processing of heart beat per minute, the system detects all heart rate anomalies (the transition “Trouble detection”).

Fig. 8
figure 8

Heart trouble detection service module

Full size image

According to the anomaly, one of these three cases can be active:

  • Heart rate trouble 1 The sub-network “Bradycardia” will be activated which means that the system detected a bradycardia at sample n.

  • Heart rate trouble 2 The system detects a tachycardia and the sub-network “Tachycardia” sends the information.

  • Heart rate trouble 3 this is the case of cardiac arrest detection. The system sends a message with a code number 1 directly to the doctor. A call will be made to the phone of the person followed and at the same time an emergency of maximum order (order 1) will be triggered.

After the simulation, the heart rate detected for the sample n = 5 is Fc = 109beats/min and the system identifies a tachycardia.

5.3 Location service

Figure 9 represents the Location module. As shown, there are four places in input “Bathroom”, “Kitchen”, “Living room” and “Bedroom”. Each place sends in packets the values of the two receivers of the presence detectors placed on the room door.

Fig. 9
figure 9

Location service module

Full size image

The transition “Data” receives to each sample n the information of the sensors transmitted by the places and sends all this data to the treatment. A sub-network “Location” processes this data and send in outputs the location of the person to any sample n (for n = 544, “Person in living room”).

5.4 Emergency service

After the collection and the processing of the modalities data, the Ambient Assistance Services produces specific information about the situation of the person (in case of fall or heart rate trouble) and its location in the Habitat.

Figure 10 shows the emergency service module for the two cases: fall detection without cardiac disorder detection and cardiac disorder detection without fall.

Fig. 10
figure 10

Emergency service module (fall detection without cardiac disorder detection and cardiac disorder detection without fall)

Full size image

The accelerometer values are sent from the place “At for 10 min” to the sub-networks “absolute acceleration” to process data and join the oximeter values “Fc for 10 min” in transition “n, At, Fc “.

In the case of heart trouble detection without fall, the system detects the location of the person at the moment of the anomaly detection in transition “T3”. In this moment the person receives a message on his mobile phone. According to his answer (positive or negative), the doctor receives the code number. If the answer is positive, the code 2 will be sent and the emergency order is minimal (order 0). Otherwise, the operator will decide the emergency order.

For the case of fall detection without a heart rate trouble, the system detects like the other case the location of the person at the time of fall in the transition “T2”. The transition “T5” will be active in case when the person remained on the floor after 2 min from the time of the fall detection. A call will be made to the followed person. If he does not respond, an emergency with a maximum order (order 1) and the code number will be sent to the doctor. If he responds, the operator decides the emergency order and the code number (code 6 or code 7).

The transition “T4” is active in case when the person gets up before 2 min, a message will be send to the followed person. Three other codes (code 8, code 9 and code 10) with the emergency order can be sent to the doctor.

Figure 11 illustrates the emergency service module for the case of fall detection with heart rate disorder. For this network, we have in input four places “Bradycardia”, “Tachycardia”, “Fall” and “Location”. The moment of fall from the time of bradycardia or tachycardia is very important as information. Code 11 is an emergency of maximum order (order 1) because in this case the system has detected a fall after a tachycardia (treatment 1) or after a bradycardia (treatment 2).

Fig. 11
figure 11

Emergency service module (fall detection with heart rate disorder)

Full size image

In case of bradycardia or tachycardia detection after a fall, the system checks if the person gets up before 2 min from the time of the fall “T7”. The action for this case is to send a message to the person and according to his answer the doctor can receive the codes 15, 16 or 17 with the emergency order and its location.

If the person is on the floor after 2 min from the time of fall “T6”, a call will be made to the phone of the followed person. According to the response, the system sends the code number and the location to the doctor.

Table 2 resumes all codes that the doctor can receive after the fusion of data collected by the ambient assistance services.

Table 2 Message code number
Full size table

6 Tests and results

This section presents the evaluation and performance of the proposed architecture. The objective is to validate this system with performance tests for each assistance service. We divide our tests into two parts; the first is to simulate the model using random input modalities. The second is an experimental test, using real information from sensors placed on a volunteer.

We finish this section with an evaluation of our system in terms of accuracy for each service: the Fall detection service, Heart disorder detection service, Location service and Emergency services.

6.1 Simulation tests

In order to demonstrate the effectiveness of the proposed system, we started by using the random input modalities along 10 min. This simulation generates even non-real results to check all possible alarms.

The input modalities retrieved from the accelerometer (At) and the photoelectric sensor (Rnb) are recorded every 500 ms which results 120 samples in one minute. On the other hand, the information recovered from the pulse oximeter (Fc) is recorded every one minute. Our simulation is performed along 10 min which consists 1200 samples for the two modalities At and Rnb and 10 samples for the modality Fc.

The purpose of this system is to send a detailed message to the doctor and specify the time of fall detection in relation to the detection of the anomaly of heart rate. As these two modalities have different behaviors, we have proposed to compare 1 sample of Fc with 120 samples of At and Rnb (60 samples of At before and 60 samples after the sample of Fc), in order to check the time of the fall detection before or after the cardiac problem.

Figure 12 illustrates the random simulation of the input modality: number of heart beats as a function of time. For 10 samples, the system detected a tachycardia in sample n = 1 with Fc = 109 beats/min and another in sample n = 5 with Fc = 118 beats/min. In addition, it detected bradycardia at sample n = 8 with Fc = 25 beats/min and another at sample n = 10 with Fc = 30 beats/min.

Fig. 12
figure 12

Heart rate of the person (Random simulation)

Full size image

Figure 13 shows three zone (a), (b) and (c) of the graph recovered after a random simulation of the absolute acceleration (At) with a 60 sample interval for each part. In zone (a) and (b), the system didn’t detect a fall. On the other hand, as indicated in zone (c), the system detected the three successive states of fall at samples n = 948, n = 949 and n = 950.

Fig. 13
figure 13

Absolute acceleration of the person (Random simulation)

Full size image

The random simulation of the input modality Rnb is presented in Fig. 14. As shown, the location of the person in sample 120 is code 4 which indicates the bathroom. In samples n = 600, n = 950 and n = 1200, the location is the code 0 which indicates the Hall.

Fig. 14
figure 14

Location in the home (Random simulation)

Full size image

The system detects four anomalies during the random simulation of the three input modalities, as shown in Fig. 15.

Fig. 15
figure 15

Message number (Random simulation)

Full size image

In samples n = 120 and n = 1200, code 2 was triggered and the doctor received the message: At time 120, the person has a Tachycardia without fall but he is in normal situation with emergency order 0–code 2–he is in the Bathroom.

In sample n = 600, the doctor receives the message: At time 600, the person has a bradycardia without fall but he is in normal situation, after a call to the emergency center the operator decides the emergency order–code 4–he is in the hall.

In Sample n = 950, that means at the time of the fall. The message sent to the doctor is: At time 950, the person fell before bradycardia and after a call the operator decides that he is in dangerous situation – code 16-he is in the hall.

6.2 Experimental tests

In order to verify the robustness of the model, experimental tests were carried out. The accelerometer was placed on the belt of a volunteer who does not have cardiac disease and the pulse oximeter on his ear. This volunteer purposely made some free falls and he moved in an environment included four rooms and a hall for 10 min.

The volunteer is healthy, does not have any cardiac disease. Figure 16 shows the number of the heart beat recovered using pulse oximeter placed on the ear of this person over 10 min. The heart rate values (Fc) collected in that period are within a standard margin defined by doctors (minimum = 50 beat/min and maximum = 100 beat/min).

Fig. 16
figure 16

Heart rate of the person (Experimental test)

Full size image

The volunteer did these daily activities (walking, sitting, falling…etc.) over 10 min. The tri-axial accelerometer placed at the belt of this voluntaire sends an information every 0.5 s so for 10 min we recover 1200 values of its absolute acceleration (At). Figure 17 shows the values of At between samples n = 450 and n = 850.

Fig. 17
figure 17

Absolute acceleration of the person (Experimental test)

Full size image

The graph shows a variation of At between 0.7 and 1.5 g to the sample n = 520. For n = 521 we have At = 0.56 g, for n = 522, At = 3.15 g and for n = 523, At = 1 g. the success of these three samples means a fall detection. A stability of the absolute acceleration between n = 524 and n = 819, the voluntary was on the floor for this period. At n = 820, the voluntary stands up and At = 1.58 g.

The input modality (Rnb) specifies the location of the person in the habitat. Figure 18 shows the location of the volunteer in an environment of four rooms and a hall. These results were recovered from the eight photoelectric sensors placed at the door of each room. The volunteer moved from the kitchen (room 3) to the bedroom (room 2) ... etc. At the moment of fall (sample n = 523), the person was in the living room (room 1).

Fig. 18
figure 18

Location in the home (Experimental test)

Full size image

The result of the fusion of information retrieved from the three ambient assistance services after an experimental test on a volunteer who does not have a cardiac disease for 10 min is shown in Fig. 19.

Fig. 19
figure 19

Message number (Experimental test)

Full size image

In this case, the message sent to the doctor is: At time 523, the person fell without heart rate disorder—he is on the floor in the living room—order of emergency is maximal (order 1)—code 5.

6.3 Evaluation

To evaluate the quality of our system, it was necessary to carry out a statistic analysis on a series of tests for each service. We tested our system on 10 participants (3 women and 7 men) with ages between 30 and 70 years old who don’t have a heart problem or physical disabilities. Each participant made 10 different scenarios to detect every possible case.

6.3.1 Fall detection service

We placed the tri-axial accelerometer on the volunteer’s belt one by one. We collected the data from 100 tests with two scenarios:

  • 50 tests of emulated falls from standing.

  • 50 tests during the activity daily living.

During these tests we can find four possible cases (Fortino and Gravina 2015):

  • True positive (TP) a fall occurs and the system detects it;

  • False positive (FP) the system announces a fall that didn’t occur;

  • True negative (TN) a normal movement (not a fall) and the system doesn’t declare a fall;

  • False negative (FN) a fall occurs but the system doesn’t detect it.

The results of our tests are presented in Table 3.

Table 3 Results of performance tests
Full size table

To evaluate the responses to these four situations, we used tree criteria:

$$Sensitivity\;=\;~\frac{{TP}}{{TP+FN}} ;\;\;specificity\;=\;~\frac{{TN}}{{TN+FP}} ;\;\;accuracy\;=\;~\frac{{TP+TN}}{{P+N}}$$

The fall detection service proposed in this paper has:

  • Accuracy = 87%.

  • Sensitivity = 82%.

  • Specificity = 92%.

It can be seen that our system offer a very good sensitivity of 92% and a high accuracy of 87%. It can identify most of the fall events, but there were some false judgments for the complexes activities. Our system differentiates with a high specificity of 82% the false result from the true fall detection.

6.3.2 Heart disorder detection service

The volunteers didn’t have heart disease; they carried the pulse oxymeter to the ear for a supervision period. To detect tachycardia, we asked the participants to make physical efforts such as: climbing the stairs, jumping, running...

Each one of the 10 volunteer has done 10 tests (100 tests) with different scenarios and each time they make a physical effort, the number of heart beats increases and our system detects a tachycardia.

Bradycardia and cardiac arrest are delicate cases and can’t be verified by doing experimental tests.

The results obtained during the tests validate the detection of tachycardia with 100% of success.

6.3.3 Location service

Two photoelectric sensors are placed at each door of the environment. The fall detection service tests were done in this environment.

For 10 participants who made a total of 100 tests during a period, we had:

  • True positive, TP = 94;

  • False negative, FN = 6;

  • Sensitivity = 94%.

Our location service has a high sensitivity, it minimize the false information.

6.3.4 Emergency service

The emergency service merges the data collected by the three services. It has the following performance:

  • It detects if the fall is at the time, before or after the detection of cardiac disorders.

  • It announces the case of fall detection without the detection of cardiac disorder and the case of heart disorder detection without the fall detection.

After merging all this data, this service will send this information to the doctor with the location. In addition it classified the order of emergency.

The emergency service has been tested only in simulation with real input data from sensors placed in the test environment and carried by the volunteers.

7 Discussion

A continuous monitoring system of the elderly was presented. This system is installed in an environment of four pieces, it collect the information from a tri-axial accelerometer for the fall detection, an pulse oxymeter for the cardiac disorders detection and eight photoelectric sensors for the location.

We validated this system with the petri network simulation tool CPNTools in two kinds of tests:

  • Simulation with random input values The aim of random simulations is to see the system response for each service and to improve it as appropriate.

For a random simulation of 1200 samples for the fall detection service and the location service and 10 samples for the cardiac disorders detection service, we distinguished four anomalies: two tachycardia without a fall, one bradycardia without a fall and one fall before a bradycardia that triggered an emergency. The codes detected for this simulation are code 4 and code 2 with an emergency order equal to 0 and code 16 with operator decision for the emergency order.

With this simulation we were able to detect even complicated cases with experimental tests such as bradycardia and cardiac arrest.

  • Test with real input values We tested our system on 10 volunteers (3 women and 7 men) with ages between 30 and 70 years old who don’t have a heart problem or physical disabilities, each participant made 10 different scenarios.

We present the case of monitoring a volunteer with good health during 10 min who simulated a fall from standing. The system detected this fall at sample n = 523 and sent a code message 5 with a maximum order of emergency.

An evaluation of the system was presented for each assistance services. For the fall detection service, we have: 87% of accuracy, 82% of sensitivity and 92% of specificity. According to these high values, we can say that this service minimize the false positive and the false negative.

The cardiac disorders detection service was checked only with the case of tachycardia and we had 100% of success. The location was done with 94% of sensitivity; the location service minimizes the false negative. The verification of the merge responses of these three data (emergency service) was done with simulation.

The system proposed in (Chan et al. 2013) detects the fall and monitors the ECG of the patients. Zhou et al. (2014) detects heart problems and falls with 83% of sensitivity. In addition to the detection of heart trouble and falls, our system offers more features such as: the location at the time of fall and followed remote medical by sending the information of the person’s state to the doctor.

8 Conclusion

The paper described a multimodal system for the monitoring of heart rate, fall detection and location. Our system is based on three ambient assistance services and an emergency service to detect anomalies in the health status of the elderly in their family environments.

  • The proposed fall detection service monitors the absolute acceleration from a tri-axial accelerometer placed at the belt of the person. It detects the fall from walking or standing.

  • The heart disorder detection service takes the heart rate from a pulse oxymeter placed at the ear of the followed person.

  • The location service uses eight photoelectric sensors placed on the four room’s door.

  • The emergency service merges the data from these heterogeneous sensors with a centralized approach in two fusion levels: feature and decision-level fusion. This service sends to the doctor a detailed message including the status of the monitored person, their answers, the emergency order and the location; each received message has an accurate code number. The information about the person state will be saved in a folder at his doctor.

Two kinds of tests were carried out to check the effectiveness of the proposed system; a random simulation and an experimental test. The aim of random simulations is to see the system response for each service and it detects even complicated case with experimental tests.

We performed experimental tests on 10 subjects (3 women and 7 men) with ages between 30 and 70 years old who don’t have a heart problem or physical disabilities, each participant made 10 different scenarios.

For the 100 tests performed, we identified 87% of accuracy, 82% of sensitivity and 92% of specificity for the fall detection service and 94% of sensitivity for the location service. The heart problem detection service was tested only for the case of tachycardia detection with 100% of success.

9 Future works

Based on the results obtained, further work will focus on:

  • Improve the fall detection service to define the fall from all possible scenarios from: standing, setting and sleeping. Make other tests with more scenarios on older people to validate the operation of the system.

  • Perfecting the system proposed by adding the possibility of monitoring people who have physical disabilities.

  • Validate the operation of the emergency service with real tests.

  • Perform a satisfaction investigation with elderly and doctors to find out about their acceptability of our system.