1 Introduction

In recent years, the world’s population is increasing year by year. At the same time, the proportion of the aging population is increasing in more and more countries. As society ages, the burden of medical expenses increases. According to the Universal Health Coverage (UHC) study, global healthcare costs for 2015 are estimated to be around $10 trillion and expected to increase to $20 trillion by 2040. The burden of medical expenses is painful for individuals as well, but it may be considered for the country to control the rise in medical expenses.

In order to reduce the burden of medical expenses, it is important for the general public, especially elderly people, to actively maintain their health. Maintenance of health leads to reducing various burdens of medical treatment and brings benefits to both individuals and society. One of the means to achieve that is to review lifestyle habits that lead to illness. To date, many researchers and healthcare professionals have suggested lifestyles to maintain good health. However, ordinary people find it difficult to assess whether their lifestyle habits are appropriate for maintaining good health. For example, there is a method of taking a meal to check whether the eating habits are correct. It will be useful to measure how much energy and nutrients can be obtained. However, I think that it is necessary to observe whether the person’s age or daily activity level is also good with the energy and nutrients. To that end, we consider one of the important approaches to maintaining health, to be aware of both the daily diet and life.

In addition, another approach to maintaining health is to work on treatment and life improvement while the symptoms are mild by detecting the signs of the disease in advance before progressing to a serious illness. This approach also has the advantage of reducing the economic, physical and psychological burden of medical treatment by detecting and treating the disease early. However, in order to realize these, it is necessary to continue to check vital data that is an indicator of health on a regular basis. That could even increase the burden on consumers. However, even if the burden on consumers increases slightly, I think that maintaining the health by continuing to check my condition on a regular basis has great benefits for both individuals and society. As it turns out, the best way to reduce the risk of causing illness is to continue the lifestyle of maintaining good health every day and to detect early whether it is a sign of illness or not from unusual circumstances Think. As a method of detecting signs of illness, for example, the state of accumulation and distraction of the fatigue of the consumer, change in physical condition or change in mood, rapid fluctuation of biological data, etc. are important in determining the health status of the consumer. I think that it is a judgment material. In order to maintain health and detect signs of illness, we believe that it is necessary to comprehensively detect the status of consumers in all areas and judge the degree of risk based on them.

In these researches on health maintenance and disease detection, various researches and developments have been conducted for the purpose of monitoring the condition of patients who are already suffering from a disease. Many medical institutions have already reduced the burden on medical professionals by actually using this equipment to monitor patients. At present, research and development is also underway on devices that monitor biological information of patients in remote locations in real time. However, most of the sensing devices used in these systems need to be worn directly on the human body. For healthy people who are not yet ill, it is considered that they are resistant to wearing such devices in their lives. In addition, there is a risk that the quality of life (quality of life) will be degraded by the equipment of those devices. Therefore, it is necessary to develop a method to acquire the behavior and vital data of the living person without attaching the device directly, so that the health condition of the person living normally can be monitored indirectly.

In order to solve those various problems at once, we propose to use non-contact bio-information sensing comprehensively. Our proposed system prepares a smart home where multiple sensors are installed inside the house. When the subject lives in the interior, the subject’s body movements, facial expressions, voice, etc., non-contact biological information that can be acquired without contact, indoor temperature changes, humidity changes, air component changes, etc. It is hoped that they will be able to estimate their health status with a certain degree of accuracy from such information. Although these partial studies have already been conducted, I believe that they have not achieved a reliable system. It is thought that there is a high novelty in searching for a method to use for estimating health conditions by integrating and using multiple observation data and exploring possibilities

2 Related Work

Many research projects have been conducted on how to look at people to realize home health care. In research on methods of acquiring human activity data and biological data using sensors, various methods are used as means for acquiring data. Heretofore, sensors generally worn on the subject’s body have been used. Those sensors have the function to connect to the Internet, and by using such IoT devices, sensor data is collected in the server. Research is also in progress to determine the state of the observation target by analyzing those data.

In the study of home health care, it is divided into four parts considered to be important. The first is research on how to acquire human activity data and biological data using sensors. There are many studies and practical applications of wearable devices that can be worn on the human body. There are also various researches on how to present information to observation objects. Next is the research on how to collect and analyze those acquired data. Furthermore, the question is who and how to present the information obtained as a result of analyzing the data. Finally, it is about the research to realize the reuse of knowledge using the big data and information obtained by those mechanisms. So far, we have been working on research to provide useful information for safe driving by keeping watch over the people driving the car by various sensors [1,2,3]. Figure 1 is an image of a study in which a sensor was installed in the car to observe the movement of a driver or a car.

Fig. 1.
figure 1

The sensors and a controller installed in the vehicle.

Full size image

There are various researches to realize human health by observing people and the environment. Gao et al. propose to use wearable accelerometer data to predict human personality characteristics [4]. Lowe et al. survey technologies for monitoring human health behavior in the living environment [5]. Arshad et al. introduce recent advances in health monitoring systems [6]. Mshali et al. survey on health monitoring system using health smart home mechanism [7]. Chaaraoui et al. present related technologies and services on how to implement Health Monitoring System (HMS) with Health Smart Home [8]. Dias et al. introduce vital sign monitoring techniques and systematization using wearable health devices (WHDs) [9]. Majumder et al. survey several low-cost and non-invasive health and activity monitoring systems and textile-based sensors [10]. Abuella et al. propose a wireless vital sign monitoring system using visible light sensing (VLS) [11]. Li et al. propose a way to detect lifestyle by wearing a wearable device that monitors the acoustic activity of the oral cavity [12].

There are many researches on telemedicine that aim to realize advanced services by using sensors and networks. Kakria et al. propose an online interactive telemedicine system that monitors the heart in real time [13]. Malasinghe et al. provide a review of recent advances in remote health care and monitoring in both contactless and contactless methods. [14]. Ha et al. proposes wearable sensors based on functional materials with a unique sensing function for detecting vital signs [15]. Tamura et al. introduce various types of wearable thermometers such as touch, patch and invisible (radiometric analysis) [16]. Wang et al. propose a non-contact infrared thermometer (NCIT) that measures the temperature of children quickly and accurately. [17]. Al-Hamry et al. health monitoring of human respiratory health with graphene oxide-based sensors [18]. Adib et al. propose a smart home sensor that monitors the respiration and heart rate of indoor people using radio waves. [19]. Güntner et al. study the current status and selection of respiratory markers and respiratory sampling to realize advanced respiratory sensors. [20]. Khan et al. propose a method for accurately measuring the heart rate of people in the room using passive Wi-Fi sensing [21].

We will use the findings from these studies to build a system to monitor people living and dining in the house. The method to watch is the observation data obtained from various sensors, and the goal is to construct a system that detects slight changes that may impair health by analyzing those data. Then, based on the detected results, provide advice and warnings to the target person. If this system detects an indoor accident, contact not only the person but also the family, caregivers, security companies and medical institutions. The mechanism for observing the health condition of the target person and presenting advice to promote a healthy life by observing the living space of the target person proposed in this study is called Ambient Assisted Living (AAL) in recent years. Bygholm et al. direct methodological behavior that should be considered in AAL studies [22]. Nawaz et al. propose sensor edge computing that realizes sensor level security and privacy for use in AAL [23]. Lloret et al. introduce intelligent communication architecture use in AAL [24]. Almeida et al. propose an inconspicuous system to monitor the behavior of the elderly and detect changes [25]. Shulman et al. introduce the Pregnancy Risk Assessment Monitoring System (PRAMS) and how to use it [26]. Polsky et al. recommend continuous glucose monitoring (CGM) for pregnant women [27]. Wang et al. show that using remote cardiac monitoring of fetuses in late pregnancy increases the detection rate of neonatal asphyxia [28]. El Murabet et al. report the results of a survey of AAL system requirements and implementation issues [29]. Julia et al. survey on the potential needs of the technologies that will enable AAL and the technologies [30]. With the initiative, researchers and companies around the world are working towards realization. Figure 2 is a diagram showing components of the AAL. In this system, the daily life performed by the target person in the living space is measured by a smart home using various sensors. As a result, data on daily lifestyles, movements and voices of the target person is acquired. Those data are sent to a hospital, a family, a carer, etc. through a network, and not only the system but also the mechanism that people of various positions and various devices watch together.

Fig. 2.
figure 2

Image of ambient assist living

Full size image

3 System Design

In this chapter, we introduce the design of a system that has the function of observing consumers with passive sensors and the function of advising consumers using the observation results. As an observation by a passive sensor, a method of analyzing the shape of a person photographed by a camera and detecting the motion of the hand and foot has already been put to practical use. In our research, we use sensors to track how people move and how they are changing. Multiple microphones are used to accurately grasp the movement of the living person. By using this, it is possible to obtain the distance between the sensor and the observation object, the positional relationship, and the relative velocity. Furthermore, in recent years, research on vital sensing that measures human heart rate and respiration using radar sensors is underway. Using an optical camera to detect human movement and facial expression, to detect temperature with a far-infrared sensor, to obtain voice data and living sound using a microphone, and to analyze components in the air in the room with a gas detection sensor.

3.1 Data Collection Using Non-wearable Sensors with No Waves

In the present system, the sensor used to acquire action data and biological data of the subject is not worn by the subject. And those sensors do not emit waves and observe their reflected waves. The waves are sound, light, electromagnetic waves, etc., and they are not used at all in this system. All sensors used in this system are only passive sensors that capture what originates from the subject. Components that make up the system include a camera and a microphone placed in the living space, a thermography, a sensor that observes the component of air in the living room, a pressure sensor, a potential sensor, and the on/off of the room. And all kinds of data such as body movement and sounds, temperature and humidity of each living room. In this system, sensors are installed in all rooms except private limited areas. The system aggregates observations obtained by those sensors into one information integration server. The information integration server analyzes the obtained data in real time to determine whether it is in a state of high urgency such as a serious accident or disease onset. If there is a high possibility of a high degree of urgency, immediately notify in advance (Fig. 3).

Fig. 3.
figure 3

Home healthcare support system by non-wearable sensors with no waves.

Full size image

3.2 Get Information Using Collected Sensor Data

This system consolidates the observation values obtained by the sensors described in the previous section into one information integration server. The information integration server analyzes the obtained data in real time. As a result of analysis, it is judged whether it is in a state of high urgency such as a serious accident or the onset of a disease. If it is determined that the system is highly likely to be in a state of high urgency, a preset notification is issued immediately. If the level of urgency is not high, acquired data is sent to the database on the cloud through the network after securing anonymity. By analyzing sensor data acquired by this system collected on a database using a general rule base, we aim to detect the occurrence of an accident immediately and also detect signs of illness. In addition, by accumulating daily life, by reading unusual movements, voices, and atmospheres, and aiming at detecting changes in the body and mind by noticing minute changes, it is also aimed. Based on the analysis results, this system provides the subject with appropriate advice. In addition, in the case of an emergency such as an accident or a sudden illness, a system is also planned to realize notification immediately to an appropriate emergency contact. The system also analyzes the data and estimates the health hazards. And we aim to control the possibility of getting sick by improving the lifestyle based on the estimation. The system also observes whether the subject has changed their lifestyle after the advice. The system learns the results by evaluating on its own whether the advice is appropriate and continues to improve to give better advice.

3.3 Safe Driving Support Including Driver Monitoring by Radar

Figure 4 shows the components of the system that observes the safety of consumers and provides appropriate advice for a healthy life by observing the observation target with the sensor installed in the smart home, and the processing flow. This system determines whether the subject is in a dangerous state based on data acquired from a plurality of sensors installed in each room of the smart home. This determination uses a rule-based algorithm or a trained neuron. If it is determined to be in danger, notify appropriate contacts. If it is judged that the condition is not dangerous, the observation data is output to the server, and the server side predicts signs of poor health and illness. At the same time, provide advice for the subject to lead a healthy life. The advice is given to the user and the family in another home through an interface device or device installed at home. We improve the effectiveness of our advice through these processes.

Fig. 4.
figure 4

System components and processing flow

Full size image

4 Preliminary Experiments

In order to realize this system, we performed preliminary experiments. The sensors used in this experiment were two common webcams and two microphones. The videos acquired from the webcam detected human body movements and expressions. Two microphones were used to capture voice and life sounds. The data observed by these sensors was continuously transmitted to the information aggregation server, and experiments were conducted while observing the data in real time. As sensors, Raspberry Pi 3B and Arduino Uno were used. We used Nvidia ADG Jetson Xavier as the information aggregation server. We connected Grove Base HAT to the Raspberry Pi. And we connected various Grove sensors to the HAT. Figure 5 shows the HAT with Grove sensors. It connects Light sensor, Sound sensor, Multichannel Gas Sensor (CO, NO2, H2, NH3, CH4), Ultrasonic Ranger, PIR Motion Sensor, Temperature & Humidity Sensor, LiDAR (Light Detection and Ranging) Sensor, GSR (Galvanic Skin Response) Sensor, and ear clip type heart rate monitor. We have created a program that continuously acquires data from these sensors. We also created a program that displays the acquired data on the console and a program that writes it to a file. In addition, we have created a simple crisis detection routine and data analysis and advice routine program as shown in the design. As an interface device for advice, we created a program to output to the communication robot Sota, in-vehicle display, speakers, etc.

Fig. 5.
figure 5

Raspberry Pi & Grove Base HAT with Grove sensors

Full size image

5 Conclusions

In this paper, healthy and safe life support aimed at reducing the number of fatalities due to household accidents and sudden illness by monitoring the target person in the living space and providing appropriate support and information presentation according to the situation. I proposed a system. In this research, by using non-contact and non-wave generating sensors as a monitoring method of the target person, it is ensured that the quality of life of the target person is not degraded and the maximum safety. Compared with conventional monitoring by wearable sensors and radars, it is difficult to precisely identify the physical condition of the body because it can’t be expected for accurate acquisition of biological data. However, if the subject is normally healthy and does not have chronic illness, we think that loose observation without wearable sensors is preferable. For example, when it is judged that there is a problem in health condition, it is desirable to shift to acquiring and observing detailed biological data. By observing the subject’s life for a long time using these sensors, it is possible to identify the subject’s health and mental condition and provide appropriate support for improving the quality of life. Think of it. Furthermore, I think that being able to usually acquire the health condition of the target person using sensors leads to leading correlation of life cycle and health. In addition, I think that it leads to the improvement of the accuracy which predicts the possibility that the accident and sudden illness occur in the life.