Keywords

1 Introduction

Along with the increasing number of elderly people, one must note that the number of those elderly people who are no longer able to look after themselves will also increase proportionally. For instance, many of them will lose the ability to live independently because of limited mobility, frailty or other physical or mental health problems [1, 2]. In Japan, the increasing number of elderly people who live alone or independently has required a large forms of nursing care to support them. However, since the number of caregivers is always limited, it is important to introduce another solution to tackle this problem. One of the solutions is the introduction of the human-friendly robot partner and intelligent room to support the elderly people in their daily life.

In the recent years, health care support system [3,4,5] based on application mainly for disease management, health and wellness management, and aging independently has developed rapidly (Fig. 1). This condition is also affected by the result of cooperation between companies on the integration of the digitalization of health and medical equipment. Now, we can acquire various information, starting from low dimensional measurement sensor data until high dimensional data to produce lifelog data. However, by recent technology, it is shown that we only can acquire data, but it is also not less important that the foundation system which connects the measurement system until support system foundation is also needed. In order to realize this, the fusion between robot technology, smart home, information technology and network technology is very important.

Fig. 1.
figure 1

Health care for elderly

Full size image

Information and communication technology is one of the promising approaches to extend healthy life expectancy of elderly people. Steve Jobs explained that a Mac, in a short time, could serve as the Digital Hub that unites those disparate points in our digital life (2001). Based on the concept of Digital Hub, we proposed the concept of Life Hub that unites a person with physical and virtual information in addition to real world, e.g., people, communities, events, places, goods, environmental data, other robots, Internet information, and personal information (Fig. 2) [6, 7]. Based on Life Hub, people are able to interact with life environment by conversional interfaces [8]. Furthermore, the environment system can use Internet of Things (IoT) and Ambient Assisted Living (AAL) technologies to actively support people [9, 10]. A wireless sensor network system can measure the number of people, human motions and behaviors as well as environmental state surrounding people. Furthermore, robot partners can ask elderly people about their daily activities through verbal communication. However, we have to deal with huge size of measurement data gathered from different types of sensors simultaneously. Therefore, the environment surrounding people and robots should have a structured platform for gathering, storing, transforming, and providing information. Such an environment is called Informationally Structured Space (ISS) [11, 12].

Fig. 2.
figure 2

The concept of Life Hub

Full size image

2 Activity Classification for Lifelog

Various types of activity classification methods have been proposed until now [13]. Szalai conducted a multinational comparative time budget project in mid 1960s. The project used a two-digit coding system composed of 96 activity categories. In this section, we discuss human behaviors for lifelog. We select activity classification by NHK, because the granularity is much less than ICF.

2.1 International Classification of Functioning, Disability, and Health (ICF)

International Classification of Functioning, Disability, and Health is useful to evaluate human body functions, structures, and disability, while taking into account both environmental factors and personal factors. The advantage of ICF is in the hierarchical structure of coding [14, 15]. In ICF, a person’s health and health-related states are given in an array of codes that encompass the two parts of the classification. Thus the maximum number of codes per person can be 34 at the one-digit level (8 body functions, 8 body structures, 9 performance and 9 capacity codes).

2.2 Activity Classification by NHK

NHK (Nippon Hoso Kyokai) has conducted Japanese Time Use Survey aimed at youths and adults aged 10 and older every five years since 1960 [16, 17]. Three-layers of categorization is used for the survey by NHK; classification, sub-classification, and minor classification. The classification is categorized to (1) Necessary activities (sleep, meals, personal chores etc.), (2) Obligatory activities (work, schoolwork, housework, commutation, etc.), (3) Free-time activities (leisure activities, conversation, mass media use, etc.), and (4) Other activities. The granularity of NHK activity classification is much less than ICF.

2.3 Social Rhythm Metric (SRM)

Social Rhythm Metric [18, 19] gives a score based on the timing of 15 specific and 2 built-in activities that are thought to constitute a social rhythm of a person. If the timing of an activity that occurs at least three times a week is within 45 min of the average or habitual time it is considered a “hit” for daily routine. The total number of hits of these activities divided by the total number of activities occurring at least three times a week gives the SRM-score.

3 Computation Model for Elderly Care

We explain ISS in introduction. Here, we will discuss the ISS which consists of three layers: (1) sensing layer, (2) feature extraction layer, and (3) monitoring layer (Fig. 3). In sensing layer, each sensor node measures human and environmental states while changing its sampling interval. (2) In feature extraction layer, human behavior estimation is conducted according to sensing data from the sensor node and robot. (3) In monitoring layer, spatiotemporal changing patterns are extracted from time-series of behaviors. This layer also generates individual daily life model and detects sensor failure as well as human anomaly life pattern. Additionally, the monitoring layer is able to control sampling interval of sensor node according to the change of the environmental condition.

Fig. 3.
figure 3

Computation model for elderly care

Full size image

4 Lifelog Generation

Base on computation model in session 3, we introduce several human behavior measurement method for lifelog. And we show lifelog generation in indoor, outdoor and by using robot partner.

4.1 Indoor Lifelog Generation by Distributed Sensing System

Many approaches have been proposed for human behavior modeling so far, for example Hidden Markov Model, Bayesian methods, Support Vector Machines and neural networks are widely applied [20,21,22]. However, they require supervised training data, they cannot handle the unknown states well. In my previous work, spiking neural network was applied to localize human, object and sensor device according to local and global sensor specification [12]. The important feature of spiking neural networks is the capability of temporal coding and resistance to noise [23]. In order to reduce the computational cost, a simple spike response model is used. Figure 4 shows simulation result for human behavior on iPad. Blue line and orange line are human tracks. Here, we use Kinect sensor to measure human position. Red circle defines the local measurement sensor position. When the local sensor is fired, we can use human position to localize the sensor position. we can also localize the sensor installed on the furniture and consumer electronics. Through this simulation, we can understand the spatial-temporal pattern of the human behavior [24, 26].

Fig. 4.
figure 4

Simulation result for human behavior

Full size image

4.2 Outdoor Lifelog Generation by Smart Device

In outdoor measurement, different with indoor measurement, smart watch, smart band, and smart phone are used as sensor devices. However, one issue is the trade-off between energy consumption and measurement accuracy. In outdoor measurement, we also integrate two types of measurement; the global measurement (GPS) and local measurement (Accelerometer). This research estimates the smartphone state by consider the trade-off between energy consumption and measurement accuracy. Thus, we combine the global and local measurement to estimate human transport modes [25].

4.3 Lifelog Generation by Robot Partner

It is enable to measure the human behavior in both indoor and outdoor environments flexibly. However, it is difficult to estimate complex human activity by the proposed methods. We propose a method of complementarily using behavior information measured by the distributed sensing system and behavior information estimated by the robot partner in the feature extraction layer. The measurement components consist of device control, human activity estimation, and environment state estimation. Here, we propose to conduct the human activity estimation as a subcomponent. Figure 5 shows the behavior measurement algorithm for the fusion of distributed sensor system and robot partner system. In this algorithm there are some conditions that should be fulfilled such as unknown and sit on chair activity, then the robot partner will start to do active measurement. An example of scenario conversation (human sits on the chair) is show in Table 1. This example shows (Figs. 6 and 7) the conversation between robot partner and human, where the human actions were previously registered and the questions are made according to these. After the robot partner checked the human answer, the action is registered into the database.

Fig. 5.
figure 5

The behavior measurement algorithm

Full size image
Table 1. Conversation contents example for active measurement.
Full size table
Fig. 6.
figure 6

Snapshot of behavior estimation by robot partner

Full size image
Fig. 7.
figure 7

An experimental result of behavior estimation by robot partner

Full size image

5 Conclusions

In this paper, we proposed a lifelog generation based on informationally structured space. First we showed different types of classification methods for human activity. We explained our computation model for elderly care. Next, we introduced several human behavior measurement method for lifelog. And we showed lifelog generation in indoor, outdoor and by using robot partner. As a future work, we intend to conduct experiments of the proposed method to elderly people.