Keywords

1 Introduction

Aging is the reality of the future world. With improvements in healthcare, life expectancy has drastically improved in the last decade. Demographic trends indicate that the aging population world over is growing at an enormous rate with more than 20% of the population in the US expected to be above the age of 65 in 2060 [1]. This is important because aging has profound consequences on a broad range of economic, political and social processes. As more and more people enter old age, it has become a priority to promote their wellbeing as aging is often accompanied by several chronic conditions and susceptibility to injuries due to cognitive impairment and loss of motor control. According to the U.S. Centers for Disease Control and Prevention, one out of every three adults (above the age of 65) falls each year and these falls are among the leading cause of fatal and non-fatal injuries. In 2013 alone, the number of emergency cases (due to falls) equaled 2.5 million with more than 700,000 hospitalized 25,000 dead [2]. In 2013, the direct medical costs of falls were $34 billion and these costs will only accelerate as more and more senior citizens opt to “age at home” with no functional support system in place to aid it. Moreover, the responsibility will now lie on family members and caregivers as the number of dependent individuals increase but the number of people who can support them will either remain the same or decrease due to self or government imposed family planning measures. In China, the previously 1-child and now 2-child policy has resulted in a significant increase in the dependent to caregiver ratio. Family caregiving is both emotionally and physically demanding and is generally unpaid. According to a study, the estimated value of this unpaid care is about $257M dollars annually [3]. As most senior citizens prefer to “age in place,” the number of older adults living alone continues to increase with at least one out of three non-institutionalized senior citizens living alone [4]. Independent living (e.g., private households) will be an important housing option for the future, particularly for the newly aged [5] and the applications of in-home monitoring technologies will have enormous potential for assuaging the emotional and monetary burden on caregivers/family members.

To provide older adults with the required level of independence in terms of care, methods to detect cognitive and physical decline that put them at risk must be in place. There are several potential technologies under development for remote health monitoring. These technologies range from in-house lifestyle monitoring to fall detection and monitoring of health vitals such as blood pressure, etc. [6]. The major limitations of current products are the high cost of technology, lack of flexibility in use, and limited one-dimensional data collection and analytics to “intelligently” monitor health status of senior citizens at-home. Even with the recent developments, there is a need for an affordable but smart and non-invasive health monitoring system. The proposed system, SilverLink (SL), aims at developing, evaluating, and commercializing an easy to use, all encompassing smart and connected home monitoring system that will allow users to have extended functionalities at a more affordable cost.

2 Literature Review and Related Systems

2.1 Remote Health Monitoring Techniques

Remote health monitoring has broadly widened its scope in the last few years. Remote monitoring (via telemedicine), previously employed as means for a follow up patient consult is now a means to support prevention (of falls, etc.), medication adherence, early diagnosis, disease management and home rehabilitation. Using remote health monitoring, especially for those suffering from chronic conditions and in need of long-term care, could lead to a significant reduction in healthcare costs by avoiding unnecessary hospitalizations and ensuring that those in need of urgent care receive it sooner. Latest developments in micro- and nanotechnologies as well as in information processing and wireless communication offer, today, the possibility for smart miniaturization and non-invasive biomedical measurement as well as for wearable sensing, processing and communication. Remote health monitoring systems are designed to gather data about patients’ status and relay it efficiently to healthcare providers/caregivers/physicians on a regular basis. They pave a path for communicating with patients (or users) beyond the acute care setting. Most such devices/systems fall under one or more of the following sub categories: mobility tracking, patient support portals, and advanced health analytics.

2.1.1 Mobility Tracking

Mobility tracking in terms of health monitoring is capturing human motion or movements. The manner in which a person performs a physical activity is highly indicative of her/his health and quality of life. Quantification and reliable measurement of daily physical activity can allow an effective assessment of a person’s daily activities as well as the effects of numerous medical conditions and treatments, especially in people suffering from chronic diseases such as arthritis, cardiovascular or neurodegenerative diseases that can often affect gait and mobility [7]. Many researchers have focused their attention towards gait analysis for health progression monitoring and fall detection. However, due to technological constraints such as the need to use multiple wearable sensors for accurate data gathering in case of gait analysis have resulted in limiting this critical research to the laboratory with no real-life applications in place.

2.1.2 Health Activity Portals and Support

Health and mobility restrictions often result in shrinking a senior citizen’s physical and social reach. Digital technology has an obvious role to play here by connecting people virtually when being together is difficult or impossible. Research shows that “persuasive technology” [8] in the form of personal messages, frequent communication via photos, videos, and other means can often help motivate people to change their attitudes, and in turn better manage their health. For example, portals such as DiabeticLink provide a platform for diabetics to track and easily visualize health data on the portal and improve health outcomes by monitoring how one health factor can affect another [9].

2.1.3 Advanced Health Analytics

With the advancements in sensor technology, it is now possible to collect data about any person in a home-based environment. Data (for healthcare) collected can range from movement of objects (e.g., displacement of a pillbox) to human motion (e.g., walking, jogging, sitting). In-depth analysis of the collected data patterns can prove useful in predicting patient behavior and health outcomes, enabling the development of a more personalized healthcare solution [10]. One limitation of existing tools is that they lack monitoring capabilities for progression of frailty (slow and natural health deterioration) in older adults. The existing solutions to home health monitoring are divided into two main categories:

  1. (1)

    Personalized Emergency Response Systems (PERS): One of the most widely used technology-based home care solutions today, PERS provides an easy way to summon assistance in case of an emergency. Advanced PERS also possess fall detection and fall prediction capabilities; however, they do not provide all the components of home health monitoring, e.g., activity monitoring, medicine reminders, etc. Examples of these devices include Alert 1 and Philips Medical Alert System. In addition, Internet-connected fitness wristbands (e.g., FitBit) and health-monitoring smart watches (e.g., Apple Watch) are gaining traction with the youth. Despite their emerging popularity, such devices do not target home care or activity monitoring for the aging population.

  2. (2)

    Home-Use Monitoring with Sensors: Some of the more mature but basic home monitoring systems such as home security systems (e.g., ADT home security) and home video surveillance (e.g., Nest Cam) do not adopt or leverage advanced multi-sensor technologies or cloud-based intelligent analytics services. Some Smart Home researchers used object sensors with pressure sensors on the floor to recognize users’ daily activities, which is not easily applicable in real home settings [11].

Thus, we propose a system called SilverLink that is unique in its capabilities and encompasses all the aspects of remote monitoring, i.e., mobility tracking, lifestyle monitoring, online health support and advanced analytics. SilverLink aims to overcome both, technological and functional challenges faced by existing remote monitoring tools.

3 System Design

3.1 SilverLink Architecture

The SilverLink system developed with funding support from National Science Foundation (NSF), US, consists of both hardware and software components. The hardware components include multiple motion sensors, a gateway and a wearable sensor with an SOS alarm button. The software components consist of data collection API, a database, an analytics engine and a web portal. The overall service architecture is shown in Fig. 1. The SL system consists of 4 object sensors and 1 human (wearable) sensor. The object sensors can be attached to relevant household objects to indicate user activity or health status, e.g., pillbox (indicating medication compliance), refrigerator door (indicating food intake), etc. The wearable human sensor can be used to capture different human motions, such as walking, falling, etc. A pre-configured gateway is designed to use BLE and 3G/WiFi communication techniques to receive and transmit data collected from the sensors to the Datacenter and into our analytics engine.

Fig. 1.
figure 1

SilverLink Architecture: Hardware (Sensors/Gateway), Software (Analytics/Portal), and Services

Full size image

Our advanced analytics engine processes the data to make deductions based on pattern recognition and in turn generate notifications/alerts when a shift in pattern is detected. The wearable device with the SOS button can be activated (by the user) to alert a response team in case of an emergency. The SilverLink web portal will also provide a platform (on devices such as laptops, tablets, mobile phones, etc.) to visualize the health information collected by the sensors.

3.2 Hardware Design for Home Activity Sensors and Gateway

SilverLink uses two types of activity sensors: (1) object sensors and (2) human sensors. Both types of sensors have high sensitivity, high frequency of data sampling and are comprised of accelerometers and gyroscopes. Each sensor communicates wirelessly via Bluetooth. It periodically emits signals to indicate sensor status and to synchronize with other components of the monitoring system. Because of Bluetooth, the range on the sensors is about 10–12 m. The sensor, powered by a coin cell battery, is enclosed in a light but durable casing with an attachment mechanism so it can be fixed onto a variety of different objects. For human motion monitoring, the sensor can be easily attached to a user’s belt/keychain or worn as a pendant. The home gateway is an Android Smartphone with Bluetooth, 3G and WiFi capabilities.

3.3 Data Collection API and Activity Database

The data collection API is used to accept data from the different sensors placed in a user’s home. The datacenter is configured to store raw data collected from activity sensors and sent via the gateway. Examples of the types of data stored in the tables include gateway, sensor and system information; raw sensor log data; processed data representing user activities; and web portal management data such as user login and profile, links, notifications, etc.

3.4 Process Design for Advanced Analytics Engine

SilverLink’s novel analytics engine is configured to process and analyze data obtained by other components of the system. Accordingly, the analytics engine employs an algorithm (e.g., an abnormal pattern detection algorithm) to perform such tasks as advanced pattern recognition. Data is sourced from the remote sensors and transmitted through the monitoring system to the data collection API such that a set of raw sensor data is generated and is subjected to data transformation and integration steps for noise reduction and sanitization. Various analytics approaches including pattern recognition and signal detection to generate user activity data and define signal patterns are then used and this processed data is then either recorded and stored in user activity tables or will trigger notifications to family members/caregivers.

3.5 Design for SilverLink Web Portal

The SilverLink web portal is an online monitoring and data visualization tool designed to allow family members/caregivers to remotely monitor senior citizens.

The SilverLink web portal offers utilities such as user sign in/registration, user dashboard to view monitoring data (Fig. 2), sensor configuration (sensor status and location of the sensor), notifications, notification settings (selection of the threshold for notification/alert generation) and administrative options (adding or editing a new user profile). The web portal provides password-protected access to registered users and is accessible from computers, tablets or smartphones. The portal also generates alerts, (via text/email) in case of activity pattern anomalies and; a summary page, which displays the daily average activity count over a period of 7 days. This summary is sent out weekly via email to keep the user/caregivers updated. The organizational dashboard can effectively reduce operational cost by allocating more attention to high-risk elderlies.

Fig. 2.
figure 2

SilverLink’s user dashboard displaying a user’s activity summary for the day

Full size image

Once our hardware and software development was complete, the next step was a to perform a comprehensive system evaluation in order to determine how the system fares in real world settings. Our evaluation included extensive internal and external testing procedures, which are discussed in detail in the following sections.

4 Preliminary System Evaluation

In order to determine the technical feasibility of our system, we designed and conducted a user study, which was approved by University of Arizona’s Institutional Review Board. The main objective of the study, its methodology and results are as follows:

4.1 Objective

The aim of this study was to evaluate the SilverLink system based on several parameters including form factor, user comfort, technical feasibility and system stability to uncover potential opportunities for system improvement and gather data for advancing our analytics research.

4.2 System Evaluation Methodology

This study was conducted in two phases. For the first phase of system evaluation (or internal testing), we used the following usability testing methodologies: expert review and remote testing to evaluate factors such as the operating distance between the Gateway and the sensors (range), battery life, data transmission rate, data loss rate, system errors, stability of the portal, the capabilities of the analytics engine and other system heuristics. This internal testing was carried out in the US, Taiwan and China to determine system compatibility with different networks in varied settings.

For the second phase or external testing, we screened and selected candidates that best matched our target population. The participants were asked to participate in a short pre-test interview (qualitative) before we installed the SilverLink system in their houses and then a longer post-test interview (qualitative) after 4–6 weeks of continuously using the system. Both, the pre and post-test questionnaires were designed to gauge user’s interest and comfort in using the SilverLink system.

5 Preliminary Findings

5.1 Internal Testing

As mentioned previously, the internal testing was carried out by our teams in the US, Taiwan and China. Each team tested 10–15 sets in different settings with randomly selected participants. Participants included domain experts, students, working professionals and retired citizens. Each randomly selected user was asked to install and use the SilverLink system (following set guidelines) for at least a month and provide feedback on different aspects of the system as discussed under system evaluation methodology. Some of the technical issues reported and resolved during this evaluation phase are summarized below.

5.1.1 Improvements in Sensor Functionality

During our research into the home health monitoring market, we realized that battery life is a major cause for concern among most devices serving this segment. Even a simple step counter like Fitbit has to be charged once every week. During our internal testing, we faced similar battery issues with our sensors. To ensure the sustainable use of our system in the field, we strived to extend the battery life of the sensors.

Most human sensors on the market either only store and transmit the summarized data (e.g., step numbers and total distances), which doesn’t support detailed analytics; or continuously stream movement data to the cloud for analytics which drains battery quickly. We aimed to balance data granularity and power consumption in our use cases. To begin with, we examined the rate of power consumption at different stages of sensor functionality, such as configuration, transmission, etc. We then analyzed and designed optimal battery consumption procedures at each stage and identified several reasons for a high initial battery consumption (2–3/10 days for human/object sensors). These were continuous BLE (Bluetooth Low Energy) connection (also known as streaming mode), incompatible firmware update and download of unnecessary stationary data. To address these issues, we implemented, in our sensors, a ‘logging mode’ that only triggers data transmission from the sensor to the gateway when a connection is established between the two. In addition, by applying the ‘any motion detection’ logic, the battery life of human sensor increased to 4–5 weeks and that of the object sensors increased to 10 weeks. Lastly, we increased the BLE advertising interval to further extend object sensor’s battery life to more than 4 months. Among other improvements, to optimize sensor sensitivity, we implemented a filter to reduce noise from picking up undesirable events. The next section discusses the development of the gateway app.

5.1.2 Improvements in Gateway App

For our gateway, we selected a low-cost Android smartphone with 3G, WiFi and BLE capabilities that allows development of a customized SilverLink app with sensor’s Android SDK (Software Development Kit). SL’s gateway app is run as a foreground service to boost its robustness as select background services are terminated by the Android system when low on resources. One drawback of the device is the low number (2–4) of concurrent BLE connections that can be established at any point of time. To work around this constraint, we employed a rotating connection mechanism to ensure that all 5 sensors could connect to the gateway.

We also found that our app would not attempt to reconnect to a WiFi network automatically upon a disconnection. To resolve this, we developed a mechanism to automatically re-establish WiFi connectivity if it was dropped. Wi-Fi availability and stability is one challenge we faced especially in Taiwan and China. Our Android phone-based gateway can utilize 3G to solve this connectivity issue and hence we are exploring local 3G options for our next version.

In a real world setting, our system should require very little user involvement. For this purpose, our gateway has been designed with a built-in auto-update. The gateway also has an auto crash report system and can reset itself, should any error occur.

5.1.3 Improvements in Battery Gauge/Meter

Another major concern with IoT devices is inaccurate indication of the battery percentage (charge remaining). This issue occurs because the battery percentage meter for an IoT device is voltage-based. The IoT system converts the effective working voltage to a percentage. However, the working voltage is sensitive to the ambient temperature. Thus, the battery percentage is unstable and non-linear. To solve this, we measured the voltage of a battery under different temperature conditions for the same period of time and used linear regression to generate a mapping table, which is used to map the voltage percentage to a stable and linear value in different temperatures.

After completing extensive internal testing, we moved to the second phase of our system evaluation. The results of this study are summarized in the section below.

5.2 SilverLink US Evaluation

The external usability study was conducted with 6 participants that matched our study criteria. These participants were in the age group of 76–89 years and were living in independent housing, either alone or with their spouse. Our selected participants suffered from various health conditions ranging from arthritis, fibromyalgia, sepsis due to a prior infection, Parkinson’s disease and other heart conditions. This made them a suitable fit for our usability study.

5.2.1 User Feedback

Before setting up the system for our participant (user), we conducted a pretest interview which gave us information about their daily routines, use of alternate home monitoring systems and general level of comfort with smartphones, iPads and tablets. We also asked them about their biggest fear when it came to living alone. This helped us assess the need for a system like SilverLink and its best possible use for each individual participant.

At the end of 6 weeks, we conducted another round of interviews to find out what the users thought about the SilverLink system. The questions addressed several aspects of the system including system design, user comfort, portal usability, etc. The initial feedback that we received was as follows:

  • 6/6 users said the sensors are light weight, non-obtrusive and comfortable to wear 24*7

  • 6/6 users said the human sensor is better designed than their PERs system

  • 6/6 users said that an SOS button is a must-have for them

  • 3/6 users said they would use the system for home security in addition to health monitoring

  • 3/6 users suggested implementing GPS on the human sensor

  • 2/6 users said that the system was useful in monitoring sleeping patterns

  • 2/6 users said that activity monitoring (using object sensors) was not a must-have feature whereas gait monitoring using the human sensor was.

  • 1/6 users said the device was useful for monitoring eating habits

  • 1/6 users said that the SilverLink portal motivated her to set goals and be more active

During this study, no family members or caregivers were involved due to which it was not possible to accurately judge the usefulness of the entire system (including the online activity portal). We aim target participants including family members and retirement community staff members in the next phase of the study.

In terms of technical feasibility, the SilverLink system faced two major issues: (1) the loss of WiFi connectivity when using a public network and (2) unexpectedly low battery life (2–3 weeks) of the human sensor when used continuously.

5.3 SilverLink China Evaluation

To evaluate the effectiveness and robustness of the system outside the US, we conducted a user study in China as well. The 7 participants (users) were mostly IT technicians who have senior family members living with them but have never used home health monitoring devices before. We selected families with several different house floor plans and home settings, to ensure that different conditions have been considered in the test.

5.3.1 User Feedback

Overall, the users were satisfied with the system. The feedback received was:

  • 5/7 users said that the system “lets them know about their family members whenever they want”

  • 4/7 users said that is was “fit for seniors who live alone”

  • 3/7 users said the dashboard charts were difficult to understand

  • 2/7 users said that sensor range was a problem in houses bigger than 100 m2

Other suggestions made included extending the system functionality to record outdoor activities, changing the human sensor from a pendant into a wristband (5/7) and extending the remote functionality to cater to both senior and child care (1/7).

Our immediate next steps, based on the results of the user study, are to improve sensor battery life and incorporate the SOS (PERs) functionality. We will also continue testing our system with a larger group of people including family members/caregivers of senior citizens and further our mobile sensor research on fall detection and activity of daily living recognition. The preliminary results of this research are described in Sect. 6.

6 Preliminary Mobile Sensor Research and Results

6.1 Fall Detection Research Design and Preliminary Findings

We conducted a preliminary study on designing a fall detection system using the SiverLink human sensor. This sensor contains a tri-axial accelerometer to track the user’s movements and send acceleration signals to the gateway. Analytic algorithms were used to aggregate the signals to high-level parameters (e.g., walking speed and step count) and make meaningful inferences.

A fall is an event, which results in a person coming to rest inadvertently on the ground or floor or other lower level [1]. Three phases can be identified during a fall event: collapse, impact, and inactivity [12]. There have been studies on fall detection attempting to capture the features for one or more of the three phases, by measuring maximum acceleration, vertical velocity, posture change, etc. [13, 14]. However, most of the prior studies used threshold-based or classifier-based systems based on signal processing or fall specific features, which did not take the time precedence of the three phases into account. The absence or order change of any phases may lead to different scenarios other than fall events. For instance, missing the inactivity phase may indicate that the fall is of no or minimal damage. Furthermore, prior studies fixed the sensor orientation and position on human body for their experiments, which is a void assumption in real world scenarios. Real users typically wear the human sensor in any arbitrary orientation. To resolve the two issues, we proposed a fall detection system based on hidden Markov models (HMMs) with sensor orientation calibration techniques. HMMs are a temporal pattern recognition models that characterizes human motor states (e.g., standing upright still, falling, impact, lying down still, etc.). Figure 3 illustrates the human motor states during a simulated fall event.

Fig. 3.
figure 3

Human motor states during a simulated fall event

Full size image

Each HMM can characterize a single category of fall events or normal activities, e.g., fall forward, or walking. We construct multiple HMMs to model various categories of those activities, including fall forward, fall backward, fall laterally left, fall laterally right, quiet standing, quiet sitting, quiet lying, walking, stand-up, sit-down, lie-down, and get-up from bed. To evaluate our system, we collected signal samples from five student volunteers (average age: 27 years). Five sensors were attached to each volunteer at five positions on human body, including neck, chest, waist, left side and right side. Each sensor was considered a separate event (we did not use multiple sensors for one event at a time). We collected 100 simulated fall events and 185 normal activities. We evaluated our model in ten-fold cross validation and achieved a sensitivity of 0.990 and a specificity of 0.984, which means only one missed fall for every 100 falls, and less than two false alarms for every 100 normal activities. Our next step is to evaluate our system using real fall events, and deploy our system to real users, to further validate its performance and effectiveness.

6.2 Activity of Daily Living (ADL) Recognition Research Design and Findings

Some researchers introduced Activity of Daily Living, such as functional mobility, and food preparing, to evaluate senior’s self-care ability. Our preliminary research is focused on evaluating how the use of object sensors with a human sensor (with an advanced algorithm) can give us a better understanding of a user’s ADL and help in ADL recognition. We proposed a deep learning based approach (see Fig. 4) to process raw accelerometer readings with SilverLink’s human and object sensors and extract the ADLs. The approach utilized CNN layers to extract locally dependent sensor interactions and a sequence-to-sequence Recurrent Neural Networks (RNN) layer to extract ADLs from the recognized interaction sequences.

Fig. 4.
figure 4

Fall detection research design

Full size image

The research test-bed was collected with two 3-axes accelerometers in a controlled home environment. The human sensor was configured with 12.5 Hz sampling rate and a sensitivity of ±4G and the object sensor was configured at 12.5 Hz sampling rate with a sensitivity of ±2G. The participants wore the human sensor as a pendant. In these settings, we collected four target human-object interactions (using object sensors placed on refrigerator/restroom/cabinet door and salt containers/pillboxes), including pull, push, pick up, and put down. In total, the data set is separated into the training set with 3600 data frames, the validation set with 600 data frames, and the Test set with 600 data frames. 1000 ADL sequences were bootstrapped from these data. According to the location information of the object sensors, each interaction in these sequences was labeled with the following four ADLs: medication, food intake, personal hygiene, and no activity. The training set, validation set, and test set has 800, 100, and 100 sequences.

6.2.1 Preliminary Findings

For interaction extraction task, we evaluate our CNN against SVM and kNN. The accuracy of CNN method was 99.5%, which exceeded the accuracy level of the baseline method SVM with RBF (98.36%) and kNN (k = 1) (96.42%). We compared the label-wise accuracy for the ADL recognition task against a 4-hidden-state HMM model with 2000 EM iterations. The result showed that with only 4 hidden nodes and 100 training epochs, RNN reached the similar accuracy level (46.57%) as HMM (48.71%). When we increased the number of RNN hidden nodes to 128, the accuracy increased to 61.81%. Overall, our deep learning (CNN + RNN) approach is a viable framework to recognize ADLs from raw sensor readings without manual feature identification and extraction.

7 Conclusion

There is a growing need for a smart and connected home health monitoring system that reduces manual effort and makes aging at home easier and safer for senior citizens. Our current research has helped validate the need for such a system and provided improvement opportunities both in terms of social and technical feasibility. The next step will be to conduct further user experience research in the US, China and Taiwan (with over 150 participants in total) in order to obtain further customer validation and determine system enhancement opportunities. We will also continue to improve hardware and software functionalities such as battery life, SOS alarm design, system stability, and data visualization on the SilverLink web portal.