1 Introduction

In recent years, there has been a variety of studies related to the safety of elementary school students in cloud computing environments, as safety concerns about students commuting to school has become an issue. For examples, the research to calculate the degree of the dangerous situation based on the measured values of the sensors attached to a smart phone is introduced [1]. The measured values of the sensors are multiplied by the corresponding weights and then all multiplied values are summed to deduct a mixed value, which expresses the degree of dangerous situation. There is the system design research based on the above mixed values to recognize the dangerous situation of elementary school students [2]. In this research, the dangerous situation is altered to the elementary school students by argument reality-based user interfaces. In addition, given that the smart phone is easily connected to cloud computing environments through the internet, diverse kinds of services after altering the dangerous situation are possible to be provided for handling the situation.

Behavior network [3,4,5,6] can be utilized for detecting dangerous situations. Behavior network is utilized for recognizing the dangerous situation of elementary school students [7]. Given that the criteria to recognize dangerous situations depending on each student is different to be built, each criteria can be customized by being defined based on the measured sensor values by a smart phone and is utilized for risk recognition. To perform the risk recognition, Bayesian probability and behavior network are integrated and applied. Another study is introduced for dividing the dangerous situation into two levels and recognizing each by one behavior network [8]. Two levels are for risky situation and safety situation. This study decides one of two levels and performs the behavior network corresponding to the decided level to decide the degree of dangerous situation. However, it is necessary to further subdivide and process the dangerous situation in detail to respond to a variety of situations. It is possible to divide dangerous situation into multiple levels more than two, which enables to handle each dangerous situation accurately [9].

In the existing multilevel behavior networks, the concept was not defined for moving between behavior networks in the process of expanding the behavior network. A detailed method of moving between multileveled behavior networks is required for the movement of the behavior networks.

This paper proposes a method to divide the dangerous situation into three or more levels and recognize dangerous situations with behavior networks based on Bayesian probability. In the experiments, the proposed method was applied to a smart phone and then validated. Given that the dangerous situation can be divided into multiple levels, it is possible to handle the dangerous situation accurately.

This paper is organized as follows. Section 2 introduces the research of recognizing dangerous situations. Section 3 proposes a multi-level situation recognition scheme based on behavior networks. Section 4 illustrates an example of implementation and application of the above proposed method. Section 5 discusses the proposed method. Finally, Sect. 5 discusses the proposed method and a conclusion is made.

2 Related work

In recent years, with the development of a wide variety of smart devices and increased internet speed, various internet-based services are being provided. In particular, a wide range of services with sensors are being provided for users to recognize and avoid dangerous situations.

There has been research, which designed the system that informs the parents of an infant the hazards by inferring the possible dangerous situations under a home network environment [10]. This is accomplished by, based on OSGi, using various sensors and RFID. An accurate situation is recognized by analyzing the data acquired via various sensors and RFID through the ontology database and the inference engine. The researchers suggest a system which a guardian can recognize the current status of the child.

There is a study that identify the situation by employing the mobile embedded systems equipped with a GPS receiver and CDMA modem [11]. It informs a guardian the location of elderly people who needs protection to go out of the resident area. When a guardian requests, the location information of an elderly can be checked using SMS. The sending cycle for location information, and message destination can be changed. In order to reduce the amount of communication data with the server when the elderly person travels, the embedded system judges the situation with GPS coordinates.

Table 1 Problems of existing studies
Full size table

There is a study on the system that enables a guardian to monitor the status of protected person in real time, and informs both the guardian and medical institutions the emergency situation if any [12]. RFID is used to detect proximity distance and an infrared sensor is additionally used to analyze the activity amount of the protected. The message processed employing a OSGi, a middleware for home network, and the processed result shows the state of protection by using a graph and a webcam. This system makes it easy to accurately check the health of the elderly, and lets the guardian know the situation in case of an emergency.

There is also a research that enables the headphone user to perceive a dangerous situation in advance, thus allowing the pedestrian with the headphone to respond to external sounds [13]. Sound is used to detect the direction of occurrence and movement of dangerous risk elements. In addition, if any risk element occurs, it immediately stops the flow of music the pedestrian is listening via the headphone and informs the occurrence of a dangerous situation.

In order to detect dangerous situations indoors, there is a study on behavior and recognition of the subject with regard to the dangerous situation using subject size ratio [14]. This study proposes the algorithms that recognizes human basic behavior and detects dangerous situations. In order to identify movement of the object is extracted by using single camera. In order to accurately judge a threat, a size ratio is primarily used and a dangerous situation is perceived by detecting the movement of an object in the continuous input frames.

In this paper, the data measured by the sensors of a smart phone are utilized to recognize dangerous situations, and the proposed method learns according to the result of recognition. This automatically perceives the user’s dangerous situation in the event of emergency. Table 1 describes the problems of existing studies and their solutions.

3 Behavior network for dangerous situation recognition

This paper proposes a method that processes the whole dangerous situation by segmenting it into multiple levels via Bayesian probability-based behavior networks. In addition, this paper introduces a transition method that moves between the ‘high level’, representing a more dangerous situation, ‘low level’, representing a relatively less dangerous situation, and ‘intermediate level’, representing a middle level between ‘high level’ and ‘low level’.

The dangerous situation includes, for example, the unfortunate situations of drowning, gas and pesticide poisoning accidents, and hazardous material accidents. The dangerous situations that can be measured by smartphone sensors include both possible dangerous situation and present dangerous situation. Possible dangerous situations include, when entering a dangerous area such as a place of frequent traffic accidents, a frequent sexual assault zone, a designated danger zone by the guardian, a construction site, or a crime ridden district. For such possible dangerous situations, it is possible to recognize the position of elementary school students by GPS, and then to inform them of the possibility of danger. Present dangerous situations include skirmish, abduction, fire, traffic accidents, fainting, and accidents during sports activities. It is possible to perceive dangerous situations such as scuffles, traffic accidents, and kidnappings by detection of elementary school student’s sudden motion. The sudden motions can be perceived by the acceleration sensor and the Gyrosensor imbedded in the smartphone. In addition, a dangerous situation can be recognized through a camera or microphone attached to the smartphone. This paper proposes a method to determine the safety risk of elementary school students based on various sensors.

The proposed approach is processed as follow. First, the target subject needs to be set. For example, elementary school students are very active. Therefore, the possibility of accidents is high, while they are engaged in some kinds of activity. It can be applied to elementary school students as well as seniors. In case of the elderly, the dangerous situation of the physical disease rather than the activity is set.

Second, danger levels are designed. In order to handle a dangerous situation, it is necessary to model the dangerous situation in advance. It is possible to deal with the dangerous situation by dividing the dangerous situation into multiple levels and processing the multiple levels. Therefore, the danger level can be further defined into safe, caution, warning, and danger levels. The safe level confirms that the situation can be managed at the safe level. If a danger level of the situation is graver than the level that can be processed at the safe level, it transits to the caution level after executing the countermeasure. At the caution level, it is checked if it can be managed at the caution level. If the situation is more serious than what the caution level can process, it transits to the warning level, after the countermeasure is taken; however, it transits back to the safe level if it is not that grave. At the warning level, it is checked to determine whether the danger level of the situation is manageable. If the situation is more serious than that it can be processed at the warning level, it transits to the danger level after the countermeasure is taken; however, it transits to the caution level if it is not as grave. At the danger level, it is confirmed that the situation is manageable. If it is less than the manageable at the danger level, it transits to the warning level. The dangerous situation can be managed according to the corresponding danger level. In addition, different countermeasures can be applied for each danger level. The behavior network is defined as sensing, Bayesian, critical, transition learning, retention learning, and processing states as shown in Fig. 2 in order to process it level by level. In the proposed method, it is assumed that the mixed value \({m}_{\mathrm{t}}\) [5] is utilized, which is a single numerical value calculated at time t and the modified value of multiple sensor values into one value in the situation \({s}_{\mathrm{t}}\) at the time t in order to process the situations with Bayesian probability. The mixed value \({m}_{\mathrm{t}}\) is utilized to control the behavior network.

Fig. 1
figure 1

Proposed multilevel-based behavior network. The state \({s}_{\mathrm{t}}\) and the mixed value \({m}_{\mathrm{t}}\) are received in the sensing state. Next, in the Bayesian state, the Bayesian probability \({p}_{\mathrm{i,st}}\) is calculated for the ith situation \({s}_{\mathrm{t}}\) based on the level transition count (LTC) \({q}_{\mathrm{i,st}}\) and level retention counts (LRC) \({r}_{\mathrm{i,st}}\) as shown in Fig. 2. Next, in the critical state, the dangerous situation is recognized by comparing the UCV \(\delta \) and the LCV \(\zeta \) with respect to the mixed value \({m}_{\mathrm{t}}\). Next, LTC \({q}_{\mathrm{i,st}}\) and LRC \({r}_{\mathrm{i,st}}\) are updated for the state \({s}_{\mathrm{t}}\) in the transition learning state and the retention learning state. Next, in the processing state, a countermeasure corresponding to a dangerous situation is performed, and then the situation transits to an upward level or a downward level

Full size image
Fig. 2
figure 2

Bayesian probability-based behavior network

Full size image

Third, behavior networks are defined for danger levels and performed. Figure 1 shows the behavior networks of ‘Safe’, ‘Caution’, ‘Warning’ and ‘Danger’. At the ‘Safe’, it is checked whether it can transit to the next level, either in the Bayesian state or in the critical state. However, at the ‘Caution’ and ‘Warning’ levels, there are two Bayesian states and two critical states. In the Bayesian state of the ‘Caution’ and ‘Warning’ levels, there exists a high Bayesian state and a low Bayesian state. In ‘high Bayesian state’, the probability of going up to the ‘high level’ is compared. If the condition to climb to the ‘high level’ is satisfied, it transits to ‘high transition learning state’. However, in case, the condition is not satisfied, it transits to ‘low Bayesian state’. In the ‘low Bayesian probability state’, the probability of going down to the ‘low level’ is assessed. If the condition to go down to the ‘low level’ is satisfied, then it moves to the ‘low transition learning state’. However, if the condition is not met, it transits to the ‘high critical state’.

The critical state of ‘the ‘Caution’ and ‘Warning’ levels has a ‘high critical state’ and a ‘low critical state’. In the ‘high critical state’, if the mixed value \({m}_{\mathrm{t}}\) is greater than upward critical value (UCV) \(\delta \), it moves to the ‘high transition learning state’. However, if the mixed value \({m}_{\mathrm{t}}\) is small, it transits to a ‘low critical state’. In the low critical state, if the mixed value \({m}_{\mathrm{t}}\) is lesser than downward critical value (LCV) \(\zeta \), it moves to ‘low transition learning state’. However, if the mixed value \({m}_{\mathrm{t}}\) is greater, it transits to the ‘retention learning state’.

In the Bayesian state, as shown in Fig. 3, if the sum of LTC \({q}_{\mathrm{i,st}}\) and LRC \({r}_{\mathrm{i,st }}\)is equal to 0; then, it transits to the ‘critical state’ as there is no learning. However, if the sum is not 0, the Bayesian probability \({p}_{\mathrm{i,st }}\) is calculated. The Bayesian probability \({p}_{\mathrm{i,st}}\) is the sum of LTC \({q}_{\mathrm{i,st}}\) and LRC \({r}_{\mathrm{i,st }}\)divided by LTC \({q}_{i,st. }\) Based on the Bayesian probability \({p}_{i,st, }\) it is decided whether to move to the ‘transition learning state’ or the ‘retention learning state’. The coefficient \(\beta \) for moving to the higher state than the current behavior network and the coefficient \(\alpha \) for moving to the lower state than the current behavior network are set by the user in advance. In case the Bayesian probability \({p}_{\mathrm{i,st}}\) is greater than \(\beta \) or less than \(\alpha \), it transits to the ‘transition learning state’ and in case it is less than or equal to \(\beta \) and greater than or equal to \(\alpha \), it transits to the ‘retention learning state’.

Fig. 3
figure 3

Bayesian state flowchart

Full size image

4 Experiments

Dangerous situations, which can occur when elementary school students are walking through the streets, were perceived and verified using multilevel-based behavior networks. The sensor values from the smartphone of an elementary school student were measured. In this experiment, four levels were defined, and the levels are termed as ‘safe’, ‘caution’, ‘warning’, and ‘danger’. In order to verify the proposed method, Bayesian state was applied only to ‘warning’ and ‘danger’ levels, and only the critical state was applied to the ‘safe’ and ‘caution’ levels. To verify the proposed method, the method that employs the critical value only and the method that uses behavior network were compared.

The experiment was conducted by considering the possible dangerous situations that could happen to elementary school students. The situations included: ‘way to and from the school (1.1 km), the road to the market (0.9 km), meeting with a bully (0.9 km), the kidnapping involving a vehicle (0.9 km) and the way to a friend’s house (1 km). A picture of the six situations on the map is shown Fig. 4. The experiment was carried out with ordinary situations and prearranged (i.e., staged or ‘mock’) dangerous situations.

Fig. 4
figure 4

Situation map

Full size image

The signals were collected that had occurred while a user with a smart phone was walking along the street. Each situation involved the user with a smart phone walking along the street for about 20 min and a total of six situations were collected. Galaxy S4 was used by applying the proposed method as an app. Gyrosensor, acceleration sensor, temperature sensor, GPS, and microphone were used in Galaxy S4. The values from each sensor to determine the current situation of the elementary school student were used.

First, Fig. 5 is a graph of acceleration sensor measurement values. (A) is x of the acceleration sensor. (B) is y of the acceleration sensor. (C) is z of the acceleration sensor. The acceleration sensor score was calculated by measuring the change in the magnitude of the acceleration sensor value. It was able to know that where there was a lot of change in the acceleration sensor of his smart phone, elementary school student was running at the time.

Fig. 5
figure 5

Acceleration sensor measurement values. a x measured by acceleration sensor. b y measured by acceleration sensor. c z measured by acceleration sensor

Full size image

Second, Fig. 6 is a graph of Gyrosensor measurement values. (a) shows the yaw variation of Gyrosensor, (b) shows the pitch variation of Gyrosensor, and (c) shows the roll variation of the Gyrosensor. In order to identity the condition of the elementary school student, pitches and rolls of Gyrosensor were used to calculate A Gyrosensor score by measuring the change in the pitches and rolls of the Gyrosensor.

When an elementary school student held a mobile phone by hand and moved, a change was detected. When the amount of change was large, it was possible to tell that the mobile phone shook sharply.

Fig. 6
figure 6

Measurement of Gyrosensor. a The amount of yaw changes measured by Gyrosensor. b The amount of pitch changes measured by Gyrosensor. c The amount of roll changes measured by Gyrosensor

Full size image

Third, Fig. 7 is a graph of measured temperature sensor values. Given that the app needs some time before receiving temperature values in the beginning after it starts, it showed a value of 0. It was possible to see the temperature change of the mobile phone with time. Using the temperature sensor, the ambient temperature of the elementary school student could be checked. In addition, it was able to detect the fire by checking the ambient temperature. However, the temperature was constant because it was not related to the fire in current situation.

Fig. 7
figure 7

Measurement value of temperature sensor

Full size image

Fourth, Fig. 8 is a graph of scores obtained by voice recognition. The score for recognition data of an elementary school student’s voice increased.

Fig. 8
figure 8

Measurement value of voice sensor

Full size image

Fifth, Fig. 9 is a graph showing the conversion of GPS measurement values into speed. The speed value was measured according to the changes of GPS value. The speed value per time was available for reading. At this point, the elementary school student stopping after running could be perceived.

Fig. 9
figure 9

Measurement value of speed sensor

Full size image

Sixth, Fig. 10 recorded GPSs. how close the elementary school student was to the danger area was checked, while moving. It was possible to see that the elementary school student was located in a place of Gyeongsan, Gyeongsangbuk-do, South Korea (Fig. 11).

Fig. 10
figure 10

Measurement value of GPS

Full size image
Fig. 11
figure 11

Measurement of level value per sensor. a Acceleration sensor level. b Gyro level. c Temperature level. d Voice level. e Speed level. f Dangerous area level

Full size image
Fig. 12
figure 12

Danger score

Full size image
Fig. 13
figure 13

Danger level

Full size image

It was possible to obtain the level calculated by each sensor using the acceleration sensor, Gyrosensor, temperature sensor and GPS of a smart phone that the elementary school student carried. Figure 12 shows the level value received from each sensor. (A) shows the level calculated by the acceleration sensor. In the early stage, an acceleration sensor level increased and decreased according to acceleration sensor score. In (B), the level value calculated by Gyrosensor is shown. It was possible to observe that the value of Gyrosensor increased and continuously | maintained at the highest level. (C) exhibits the level calculated by the temperature sensor. The temperature sensor was not detected and was found to be at the lowest level. In (d), the level value calculated for the speech of the elementary school student is shown. At this point, the voice value for the dangerous situation was recognized and gradually the level dropped. (E) illustrates the level calculated from the speed based on GPS. Since the speed score have increased, it maintained its position at the highest level. (F) illustrates the measurements converted to level values, via the GPS, according to the distance when there is a dangerous area near by.

Figure 12 shows the calculated danger score by employing the level of the sensor for the experimental situation encountering with the bullies. Overall, a high danger score can be seen. However, even though it was a dangerous situation, the danger score decreased as the target subject moved away from the danger zone. In Fig. 13, it was confirmed that the danger level changed according to the danger score. Overall, it was returning a danger level.

Figure 14 shows the value of calculated danger level using the level value, which was calculated by the sensor when the bullies were encountered in an experimental ‘mock test’ situation. The situation occurred a little after the experiment started when the students encountered bullies. The perceived result was a safe situation at first, but it had to be recognized as a dangerous situation when they met bullies. In case of recognizing a dangerous situation using only critical value, there was a part, which was not perceived as a dangerous situation. However, it was confirmed that the recognition rate using the Bayesian probability was higher than the recognition using only the critical value. In Fig. 15, we can confirm the danger level for level value calculated by the sensor that occurred on the way to and from school.

The safety situations were recognized as a safe situation. When playing with friends on the way to and from school, the danger score increased by the critical value, but it was not perceived as a danger.

Fig. 14
figure 14

Dangerous situation

Full size image
Fig. 15
figure 15

Safe situation

Full size image

Through the proposed method, all of them were recognized as a ‘safe situation’ in a safe situation. In the dangerous situation, a dangerous situation was perceived to be a bit more dangerous than when recognized using critical values.

5 Discussions and conclusion

Existing research that processed the dangerous situation as two behavior networks failed to deal with the dangerous situation in detail given that the danger level was only divided into two levels. In this paper, we proposed a method to recognize dangerous situations by dividing them into multiple levels based on the behavior networks. Each behavior network processed the dangerous situation considering the corresponding level. The method proposed in this paper was applied and validated on the smartphone of an elementary school student. When the proposed method was used to recognize dangerous situations, the undetected dangerous situation when the critical values only were used could be recognized using the proposed method.

There have been studies to recognize a user’s dangers using the behavior network [7,8,9]. However, they did not recognize dangerous situations in multilevel because of the limitations of single or two behavior networks. To apply the behavior network to the real domain, multiple behavior networks and a method of recognizing dangerous situations using the multiple behavior networks are required. In this paper, we have utilized multiple behavior networks for multilevel dangerous situations and have verified the operation of the multiple behavior networks according to each dangerous situation.

It is difficult to recognize dangerous situations in complex real-life situations using a single or two behavior networks. The need to recognize dangerous situations using multiple behavior networks has been mentioned, but no detailed methods have been suggested. The recognition rates of dangerous situations are high when dangerous situations are recognized in multilevel in conjunction with behavior networks to recognize complex real-life situations. Algorithms such as neural networks and support vector machines could be used to classify each dangerous situation, but there is a need to reduce the amount of computation for mobile devices with limited resources. The Bayesian probability has been introduced to reduce the amount of computation for classifying dangerous situations and the amount of learning for modeling dangerous situations based on each level and user.

The proposed method is different from the existing research methods in that it has expanded dangerous situations into multilevel dangerous situations and provided a method of moving between expanded behavior networks. It is possible to move to higher and lower behavior networks except for the highest and lowest behavior networks. The method of recognizing dangers according to dangerous situations showed higher recognition rates than the previous methods.

The proposed algorithm is proper to a lightweight solution for the smart phone environment in the Bayesian probability state of multilevel expansion. In the future, learning algorithms such as neural networks and support vector machines can be applied to recognize more various dangerous situations, if the amount of the computation of those algorithms is reduced. In addition, multiple behavior networks can identify multiple dangerous situations in multilevel. However, a novel learning processes are required to apply the proposed method to the dangerous situations in an environment that has not been learned.