Integration of Stationary and Wearable Support Services for an Actively Assisted Living of Elderly People: Capabilities, Achievements, Limitations, Prospects—A Case Study

Abstract

Within the recent three years, a stationary home assistance system has been developed, continuously optimized and operated for supporting seniors of very high age. In the last year, the scope of the system has been extended by function and beyond the spatial borders of the familiar home by a smartwatch with integrated cellular radio (Samsung Gear™ S) as a wearable device. All condensed data from the different stationary and mobile sensors are transferred to and collected by a central server for long-term analysis. The technical structure of the system is presented and its capabilities will be described, especially with respect to the variation of collected data over time in the course of a progressing dementia of one of the inhabitants. The different achievements and perceived value, which the system delivers to its users and their relatives over the course of the years will be presented. But also the limitations of the currently available technology in comparison the actual demand of the inhabitants and their relatives will be characterized which defines the boundary conditions and guidelines for further research.

1 Problem Description

For most elderly people staying in their familiar own home as long as possible in a self-determined and safe manner way is a very high-ranking target. But on the other hand technical support systems for an assisted active living (AAL) have not gained a broad acceptance among the elderly. In our case study work we describe (a) the technical assistance solution implemented and its delivered benefits against several concerns of the elderly and (b) the actual usage and perceived benefits for a senior couple of high age in a typical care-giving situation where one spouse gives care to the other in the presence of beginning and progressing (vascular) dementia. The duration of our study covers a service period of nearly three years, for which the assistance system with its increasing functional coverage has been in uninterrupted 24/7 operations.

We have chosen the case study method because a key goal of our research was to perform an in-depth analysis of the usage of such a system in a real live scenario for several years thus combining both a qualitative and quantitative approach. A main concern about applying AAL technologies (cf. [1]) deals with not really involving the elderly, but just focusing on the technical implications rather than including also the psychological and social aspects of such AAL systems. The relatively long study period of nearly three years—including the chance to investigate the upcoming and integration of a new technology like smartwatches—gave us the unique opportunity to get new insights into this difficult area of applying new technological innovations. Thus the study is based both on qualitative data derived from individual experiences and talks with the involved parties and quantitative data derived from the technological systems used, also combining both data when necessary.

In order to monitor the wellbeing of persons in need of support, care, for their daily living and health we use an established technical approach, the detection of the activities and events of daily living (ADLs, EDLs) (see Sect. 3.1 for details).¹ EDLs may be »falls « or events like »retiring to bed at nighttime « or »getting up in the morning «, which determine the beginning, ending and duration of the ADL »bedtime « rsp. »nightly sleep «. The ADL and EDL detection is performed via stationary sensors deployed within the home or sensors, smartwatches, worn by the residents on their wrist. ADL/EDL detection typically includes sensor fusion from multiple sensor devices. After possibly combining matching EDLs into single ADLs, the duration, presence and intervals between ADLs over time is analyzed. These parameters are combined in a wellbeing function w(t) (see Sect. 3.2 below for details), which characterizes the assumed wellbeing resp. health state of a monitored person at a specific time t. Whenever w(t) falls below a predefined threshold, a health hazard alert will be issued. Before external help will be called, the person in need of support resp. their present relatives or caregivers always receive a local pre-alert giving them the opportunity to cancel the upcoming external alert and to filter out occasional false alerts. External help can be provided or organized by distant family members or home emergency call centers (HECCs). In order to clarify the specific hazardous situation, external alert handling always includes establishing a direct speech connection to the home resp. the person in need of support.

The acceptance barriers of elderly people against the utilization of such assistance systems have multiple aspects:

• Fear of stigmatization. Visible AAL systems clearly demonstrate to outsiders, but also for the users themselves, that the user indeed needs support for organizing the daily living, a situation that typically everyone want to camouflage as long as possible. A potential solution is the use of “dual use” devices like smartwatches, which cannot be easily identified in their additional assistive usage.

• Resistance to change, especially if the installation of the assistance system at home is combined with demolition work. This general human attitude grows with increasing age and can only be compensated by minimizing the necessary construction work for the assistance system.

• Privacy concerns. Of course, the continuous monitoring of one’s daily living by an assistance system produces a constant feeling of discomfort and raises concerns about the potential misusage of the accumulated data. Especially imaging technologies—despite their technological achievements—are rejected.

• Costs. High installation costs, esp. for stationary systems, are another important key point for not using assistance systems. For a broad acceptance and mass distribution of such assistance systems, an acceptable level of assistance services has to be provided at reasonable costs. [4] reports that the average costs Germans would accept are around 20 € service costs per month. In [5] this cost estimation is also confirmed for the US with monthly costs of US $25 reported for the California based Lively service (with initial setup costs of US $40).

These aspects all condense in the self-insight that with the usage of the assistance system the final phase in one’s life has begun, foreseeable followed by the death. People tend to delay this type of introspection as long as possible. Thus these technological solutions are mainly perceived through these negative connotations, not by the positive aspects associated. It has been argued [6] that the user concerns can be alleviated and a buying decision for such a technology can be boosted by stressing the multivalent utility of the assistance system not only for support services, but also for improving comfort and safety at home as well as for improving the energy efficiency of the home (by reduced heating, cooling costs).

2 System Structure

The system developed over the course of the years consists of the following three components (see Fig. 1):

Fig. 1

System structure with components: 1. Stationary assistance subsystem (bottom), 2. Smartwatches (top) and 3. LTAS (middle left). Involved in the communication processes, but not part of the assistance system, is the home emergency call center or family members on duty (middle right), which react to alerts and smartphones (of family members, relatives) as endpoints of LTAS services (upper left corner)

• 1. A stationary assistance subsystem in the home based on high quality presence sensors—multisector PIR sensors—in each room (cf. [7] for a detailed description, Figs. 2, 3). Sensor fusion and the local monitoring is performed by a Siemens LOGO™ SPS/PLC. Reporting of detected EDLs, ADLs to the LTAS server (via http) and alerting (via E-Mail, SMS) is done in a highly availably way via WAN and also cellular network by an INSYS IMO-1™ GRPS router/rule-based fault transmitter. The assistance subsystem simultaneously also acts as the center of a local home automation system (Fig. 4).

  Fig. 2

  

  Two channel presence sensors used for the stationary assistance subsystem (Theben Office™)—1st channel for (local) automatic room lighting control, 2nd channel for presence signaling to assistance system

  Fig. 3

  

  Center of stationary assistance subsystem with Siemens LOGO™ SPS/PLC and insys IMO-1™ rule-based fault transmitter and GPRS router (upper row), power supply, industry IP switch (middle row) and electric power sensor and contactor for the electric stove (lower row) in a standard 3-row junction box

  Fig. 4

  

  Three layer smartwatch app architecture

• 2. Smartwatches as wearables devices together with our developed assistance app on the wrist of the person in need of support for their daily living/health. Here we use the Samsung Gear™ S smartwatch with its large 2’’ AMOLED display, GPS and integrated cellular radio, which can operate independently from a coupled smartphone and is equipped with our Tizen™ assistance app (cf. [8] for a detailed description and Fig. 5). The cellular radio transmits the recognized ADLs, EDLs to the LTAS server via http. In addition to communicating the wearer’s geographic position by SMS the smartwatch app also establishes speech connections to the wearer of the smartwatch. The smartwatch app will also directly communicate with the stationary assistance subsystem, e.g. to inform it about the departure of the wearer from home and his return.

  Fig. 5

  

  Smartwatch Samsung Gear™ S with assistance app displaying communication and orientation information (holidays, birthdays, ….) on the left and an advice to return to home after leaving an agreed area (geofencing) on the right

• 3. The long-term analysis server LTAS, which collects the ADLs, EDLs transmitted by 1, 2 and performs the long-term statistical analysis of the data. A smartwatch app accessing the server shows the last performed ADLs and vital signs of the person in need of support on demand. The LTAS will proactively inform relatives and/or authorized persons about substantial deviations of the usual lifecycle of the person in need of support. If configured, the server will also provide a daily/weekly summary of all health-relevant activities of the monitored person (see Fig. 6).

  Fig. 6

  

  Services for family members delivered by the LTAS: on demand requests for the last vital signs from the person in need of support/care (left), warning of substantial deviations of the used circadian cycle (middle), regular reports summarizing the last day (right)

Historically, the stationary subsystem (1) has started its productive use in the early spring of 2013, the smartwatch app (2) went into daily use in spring of 2015 and the LTAS (3) started its 24/7 operation on June 1, 2015 (Fig. 7).

Fig. 7

Typical agility for a 24 h period based on an α = 0.1 (giving heavy weights for historical values). As can been seen the agility value is far above the alert threshold indicating that no agility problems are present. Left scale denotes the accumulated steps (orange actual steps of the day, blue the estimated accumulated steps for the period, grey the agility value ß₃ and yellow the alert threshold for ß₃)

3 System Implementation

For the stationary assistance subsystem, many sensors and commercially available smart home systems have been evaluated and tested. For sensors, finally the decision in favor of high-end, two channel presence sensors was taken because standard movement sensors typically issued an unacceptable high rate of false-negatives, if people do not constantly move heavily (cf. [7] for details). For a discussion of potential alternative sensor technologies, see Sect. 5 of this document. One channel of the sensors is used for direct control of the ambient lighting; the second channel is used for sensing the presence of a person to the assistance subsystem.

The local usage of the sensors also for room lighting control was deliberately chosen to demonstrate the comfort value of the system to the residents on a daily basis. This automatic ambient lighting control was initially in fact judged by the inhabitants of the home as the most important and valuable feature of the new technology for them, although they were aware of the much more sophisticated assistance and monitoring technology in the background. The presence of this subsidiary monitoring technology for health hazards in the household was creating only awareness when health hazard pre-alert/alerts occurred from time to time.

Typically, the available sensor basis of commercially available (end user) smart home systems, (e.g. RWE Smart Home …), is not reliable enough compared to our presence sensors chosen. More important, it was not possible to implement the sensor fusion algorithms and finite state machines which are at the heart of the monitoring process (cf. [7, 9]) in a necessary fine-grained and precise way. Therefore we finally decided for an implementation based on the Siemens LOGO™ SPS/PLC and Insys IMO™ GPRS router/rule-based fault transmitter, which—as proven industrial components—work very reliable also in the presence of casual power brownouts and blackouts.

The knowledge for ADL, EDL recognition and the local handling of health hazards on the smartwatch is empirical, best practice knowledge which is growing and changing on a daily basis. The maintenance of the software encoding this knowl-edge with economic costs is a severe challenge, especially because all health hazards must be dealt with simultaneously and the hazard handling is intertwined in its execution.

We have developed a three layer architecture for smartwatch apps which allows to separate this knowledge into independent, small manageable chunks (cf. [10] and Figs. 4, 8):

Fig. 8

Finite state machine for joint handling the health hazards resulting from the ADLs: »absence from home«, »runaway situation«

• The lower layer contains the ADL and EDL detection based on sensor fusion. For simple EDLs, like leaving resp. reentering an agreed vicinity around the home, this detection will be done by trigonometric math calculations based on the current GPS sensor data. Complex ADL detection, for example for the detection of fluid ingestion, drinking (see [11–13]) utilizes neuronal networks or statistic regression methods for achieving the task. The nets resp. statistic parameters have to be trained before by several hundreds of supervised samples, in order to achieve the targeted precision and recall rate of a least 90% (cf. [13] for details of this process, which includes data conditioning and mining of the sample data on standard PCs). In addition this software layer contains the necessary functionality for managing speech connections, calls with the smartwatch and provides the necessary orientation information presented to the smartwatch wearer. Such orientation information—see Fig. 5—has been proven to be of high substantial value especially for persons with MCI or beginning dementia, in order to compensate the effect of their failing memory in communication situations with other persons.

• The medium layer on top of the ADL/EDL detection layer comprises finite state machines for recognition and handling of health hazards. A single state machine represents, in a declarative way, the processing of an individual health hazard by the smartwatch (cf. [10] for details). The finite state machine is described by its state transition table, including the corresponding actions to be executed when entering a state, and the state contents describing the output of the machine on the smartwatch screen while being in a specific state. For example, Fig. 8 describes the handling of an excessive absence from home either by (i) leaving a agreed vicinity around the home and/or (ii) extending the outside stay beyond an agreed maximum duration.

• The upper layer contains the central scheduler for synchronizing the simultaneous operations of the finite state machines for the individual hazards, thus allowing the smartwatch to monitor different health hazards simultaneously (e.g. leaving agreed areas, excessive absence, falls, insufficient liquid ingestion, abnormal heart rates, …). The scheduling algorithm selects the content to be displayed resp. interaction sequence with the wearer to be performed at a specific point in time, selecting the actual state with highest priority from all state machine (cf. [10] for details of the scheduling algorithm).

With the current generation of smartwatches, the high power consumption of the GPS sensor and the limited computational power of the CPU in continuously condensing all the sensor signals and comparing them against the trained patterns for ADL/EDL recognition limits the usage of the smartwatch to at most 18 h before the watch needs to be recharged. The smartwatch therefore is not a 24 h assistance device, but can only be used between rising up in the morning and retiring to bed at night. At night time the smartwatch device will be typically recharged. Assistance during this time can only be provided via the stationary assistance subsystem.

A direct communication link between the smartwatches and the stationary assistance subsystem assures that the departure from home and a later arrival can be managed without involving the long-term analysis server LTAS. The presence of the smartwatch wearer at home is detected via accessibility of the home Wi-Fi with a known SSID, thus allowing to detect the departure from home by loss of the Wi-Fi signal and later return by reconnecting to the home Wi-Fi. This is much more energy preserving than using GPS. The stationary assistance system—acting as a home automation system—can then switch-off critical electric loads, e.g. the electric stove, during the period of absence.

The LTAS records all EDLs/ADLs reported from the stationary assistance subsystems and the smartwatch in a relational database. Matching EDLs like departure from home and later arrival will be further condensed to a common ADL, e.g. the period of absence from home. In order to recognize substantial deviations from daily routine resp. the learned circadian cycle, the LTAS needs to be trained first for at least one week. During this week it will observe and record the nominal values for the presence, the duration of ADLs and the intervals between different ADLs on a weekday specific basis. Later on the nominal values will be adapted by a time series analysis with the actual values measured in subsequent weeks (see Sect. 3.3). Based on this information, the LTAS can then deliver three subtypes of services (cf. Fig. 6) to family members, relatives, caregivers or agents on duty in a home emergency call center during an incoming call from the home or from a smartwatch wearer.

3.1 EDLs and ADLs

Currently, the system recognizes and analyzes the following EDLs (event of daily living):

• E₁ fall, tumbling

• E₂ leaving home

• E₃ returning to home

• E₄ leaving a agreed vicinity around the home

• E₅ returning into a agreed vicinity around the home

• E₆ getting-up in the morning

• E₇ retiring to bed at nighttime

• E₈ falling asleep for a nap

• E₉ awaking from a nap

• E₁₀ low battery situation of the smartwatch.

All EDLs have as their characteristic attribute the time t, at which they happen, but may have additional attributes (e.g. the actual battery level for E₁₀).

An ADL (activity of daily living) A_i is either atomic (e.g. A₅, A₆) or structured and then characterized by two EDLs e₁, e₂ happening at times t1, t2 with t1 < t2 as their start and end events, which we denote as A_i[e₁, e₂] and thus A_i has a duration ta = t2−t1. If for such a structured A_i, its characteristic start event e₁ has been recently recognized, but the end event e₂ is pending, A_i will be denominated as ongoing. See Fig. 9 for a depiction of the structured ADLs A₁ to A₄.

Fig. 9

Some EDLs and ADLs, including their specific starting and ending events within the daily routine

• A₁ Bedtime, nightly sleep, defined by A₁[E₇, E₆]

• A₂ Absence from home, defined by A₂[E₂, E₃]

• A₃ Runaway situation, defined by A₃[E₄, E₅]

• A₄ (Midday) nap, defined by A₄[E₈, E₉]

• A₅ Visit to the toilet

• A₆ Fluid ingestion, drinking.

Also ADLs have attributes, in general their actual duration ta, but also more specific attributes, as the time spend in the toilet room (excluding the way to and from the toilet) for A₅ or the amount for fluid ingested for A_6, (the determination of the type of ingested fluid still an unsolved issue in our work).

3.2 Wellbeing Calculation and Monitoring

For the wellbeing function w we propose a (functional) combination of at least the three wellbeing aspects measuring the inactivity (ß₁) of the person in need of support, the excess duration of ongoing activities (ß₂) and the agility of the person in need of support (ß₃). For these aspects we define three sub-functions ß₁, ß₂, ß₃ mapping SensoricEvents → [0, 1], [0, 1] ⊂ ℜ. A function value of 1 denotes ideal wellbeing. A health situation is judged the more dangerous the more those values decrease. If the w function value falls below a defined threshold, e.g. 0.5, an automatic alert will be issued. With respect to the following definitions this threshold value of 0.5 will be reached if the related current value for ADLs deviates by 70% from the specified nominal value.

When no recognized ADL in the household is taking place, ß₁, the wellbeing sub-function for inactivity measurement, is applied based on the definition in [3], by:

$${\text{e}}^{{ - {\text{t}}/2*{\text{T}}}}$$

where t is the current (time) duration of inactivity since completion of the last ADL, and T is the specific average inactivity between ADLs learned from the past for the current day of the week.²

On the opposite, as long as a recognized ADL is ongoing, ß₂, the wellbeing sub-function for the measurement of excess duration of this specific ADLs will be applied, which has been defined in [3] to:

$$\begin{array}{*{20}l} {{\text{e}}^{{({\text{TN}} - {\text{ta}})/{\text{TN}}}} ,} \hfill & {{\text{for}}\,{\text{ta}} > {\text{TN}}} \hfill \\ {1,} \hfill & {\text{otherwise}} \hfill \\ \end{array}$$

where ta is the actual duration of the (ongoing) ADL and TN is the specific maximum duration of the corresponding recognized ADL in a normal situation learned from the past for the current day of the week (Assuming at least a typical one week cycle for calculating the specific, possibly varying T, TN, SN, F nominal values for the individual days of a week. If the current day under consideration is a national or local holiday, based on the Western cultural context, the values for the last Sunday will be used instead.). For simplicity reasons of calculating ß₂, it is not always necessary to know which ADL exactly is happening, but the set of all ADLs will be partitioned into categories with respect to similar typical execution times, and then all ADLs within the respective partition will have the same maximum execution time TN. ß₂ will then be calculated with the specific TN value based on the category of the recognized ADL.

Independent from detected EDLs, ADLs occurring resp. being carried, we propose for the new wellbeing sub-function ß₃ determining the agility of the elderly subject:

$$\begin{array}{*{20}l} {{\text{e}}^{{\left( {{\text{stp}}({\text{t}}) - {\text{STP}}({\text{t}})} \right)/{\text{STP}}({\text{t}})}} ,} \hfill & {{\text{for}}\,{\text{stp}}({\text{t}}) < {\text{STP}}({\text{t}})\,{\text{and}}\,{\text{NOT}}({\text{E}}_{ 1} )} \hfill \\ {1,} \hfill & {{\text{for}}\,{\text{stp}}({\text{t}}) \ge {\text{STP}}({\text{t}})\,{\text{and}}\,{\text{NOT}}({\text{E}}_{ 1} )} \hfill \\ {0,} \hfill & {{\text{if}}\,{\text{event}}\,{\text{E}}_{ 1} \,{\text{has}}\,{\text{been}}\,{\text{detected}}} \hfill \\ \end{array}$$

where stp(t) is the sum of steps performed during the current day until actual time t, F(t) is the cumulative distribution function of steps over the day, SN is the specific total number of steps learned from the past for the current day of the week and with STP(t) = SN * F(t) estimated from the nominal step sum for the current day at time t. ß₃ will be calculated all over the day. An advantage of ß₃ is that it does not rely on ADL detection and thus counterbalances the up-to-date dependency of the wellbeing calculation from the plenitude of recognized ADLs. The sub function ß₃ will depend primarily on the (counted) steps measured by the smartwatch over the course of the day and taking into account the cumulative distribution of those steps until time t of the specific day learned from the past. A recognized fall will immediately effect an alert (see below).

Finally, the wellbeing function w: SensoricEvents → [0, 1], [0, 1] ⊂ ℜ, will be formally defined as:

$${\text{w}} = \hbox{min} \left\{{\ss_{ 1} ,\,\ss_{ 2} ,\,\ss_{ 3} } \right\}$$

This means that whenever the inactivity (missing any recognized ADL) or the excess duration of an ongoing activity category or the lacking agility gets critical and the w value falls below 0.5, a health hazard alert will be issued.

3.3 Learning the Nominal Values

For the acquisition of the nominal values for T, the TNs for each ADL category, SN and the cumulative movement distribution function F(t) a seasonal cycle of one week and a preceding initial training phase of a week will be used. For each day s∈Nat, s = 1, 2… 7 in the training phase and the following weeks, the (weekday) specific T_s, the TN_s for each ADL category, SN_s and F_s(t) values resp. distributions are estimated:

• Inactivity: For all ADLs A₁ … A_X detected on this day s, for T_s the average (I₁ + I₂ … + I_x−1)/X−1 of all inactivity periods I₁ … I_x−1 will be used, where I_n denotes the inactivity period between A_n and A_n+1.

• Excess duration: For all activity (time) durations d₁, … d_X of those ADL instances A₁, … A_X detected on this day s for a specific ADL category, for TN_s for this ADL category the maximum duration MAX _{i=1,…, x} (d_i) will be used.

• Agility: For each hour h = 0… 23 of the day, the sum of all counted steps achieved by the end of a hour (from the beginning of the day) will be tabulated for computing the initial F_s(h) distribution. SN_s will be the total number of all steps counted for this day s.

For projecting these values for the next time for day s, s > 7, we use the exponential moving average already proposed in [14] for MA = T, TN, SN, F:

$${\text{MA}}_{\text{s}} =\upalpha^{*} {\text{O}}_{{{\text{s}} - 7}} + (1 -\upalpha)^{*} \left( {{\text{MA}}_{{{\text{s}} - 7}} + {\text{TR}}_{{{\text{s}} - 1}} } \right)$$

where O_x is the observed value for day x (computed by the method described above) and MA_x is the computed moving average for day x and α, 0 ≤ α ≤ 1, α∈ℜ, is a “smoothing constant”, which may give more relative weight either to observed values in the week before or the predicted value, moving average, for the specific day in the week before. Initially MA_i = O_i for all i = 1, …, 7. The seasonality factor TR_x covers a potential data trend by accounting the differences in O_x values for subsequent days of the last week (cf. [3]) and will be contributing starting from the third week (s = 15), for all prior days i ≤ 14: TR_i = 0. TR_x is defined by:

$$\begin{aligned} & 1/7\,^{*} \,(\left( {{\text{Ox}} - {\text{O}}_{{{\text{x}} - 7}} } \right)/7 + \left( {{\text{O}}_{{{\text{x}} - 1}} - {\text{O}}_{{{\text{x}} - 8}} } \right)/7 + \ldots \\ & \quad + \left( {{\text{O}}_{{{\text{x}} - 6}} - {\text{O}}_{{{\text{x}} - 13}} } \right)/7) \\ \end{aligned} .$$

Figure 7 shows a typical example of an elderly for the agility parameter:

4 Experiences

4.1 Development of Functionality Over the Years

During the initial years of operation, when the stationary assistance subsystem was the only available component, the main use of the assistance subsystem was: (i) the monitoring of inactivity periods via ß₁ at day time (“inactivity analysis”, cf. [6, 15]) and (ii) the ß₂ monitoring of an excessive duration of the ADLs » bedtime « and » visit to the toilet «. At that time, we used an additional ADL » motion at home « in order to have a more structured daytime. Although, in a multi-person household—the person in need of support and his spouse, guests in the home—this ADL did not really allowed to conclude specifically about the health state of the person in need of support. The introduction of the smartwatch therefore brought a substantial progress, because the movement information from now on could be assigned directly to the person in need of support and supervised by ß₃; thus the recognition of this initial ADL » motion at home « was abandoned with the availability of the new smartwatches. For the recognition of ADLs A₁ » bedtime «, A₅ » visit to the toilet «, the stationary assistance subsystem achieved a durable precision rate of about 98% based on the false positive and negative alert elimination techniques described in [7]. In essence, the utilized finite state machine did so in heuristically applying the fact that persons cannot arbitrarily appear or disappear in the specific rooms of the home other than enabled by (i) the connectivity of the rooms and (ii) based on their room specific location inside the home (or outside) up to now.

With the finite state machines in the background, especially the sequencing of activities for a nightly visit of the toilet, ADL A₅, starting and ending from and in the bedroom was supervised. The transit times from the bedroom to the toilet and back again were so characteristic for the individual persons that these toilet visits could be clearly assigned to the individual persons. We used this information to automatically fill-up the daily nursing record requested by the health insurance. We further observed in the course of the progressing dementia illness of the person in need of support that the transit time necessary to pass a distance of about 10 m grew within the operation time of the assistance system nearly by factor of ten due to the proliferating disorientation of the patient. Pre-alerts of the assistance system at that time primarily addressed resp. targeted to give attention to resp. to wake-up the care-giving spouse or family member present in the home in order to verify that everything was ok with the person in need of support. With the progressing dementia and the noticeable exhaustion of the care-giving persons by the more and more demanding care-giving task, the nightly ß₂ induced pre-alerts gained more and more practical importance over the course of the years. The exhaustion of the caregivers was not only caused by number of nightly toilet visits continuously increasing during the progress of the dementia illness and accompanied by an increasing frequency of ß₂ excess duration alerts for those visits. Also the increasing frequency of nightly unrest periods in the bedroom, where the person in need of support was not able to sleep continuously more than 3 h, contributed to this. Similar scenarios have been reported to us by many care-giving relatives with respect to their family members in need of support and impaired by dementia. At this stage of the dementia illness, two years after the introduction of the assistance technology, the real value of the installed technology in providing much more support than automatic ambient lighting was fully understood and perceived by the care-giving spouse and family relatives.

If a local pre-alert remains unanswered, the stationary assistance subsystem places an automatic alert via SMS and E-Mail. The following external handling of such alerts suffers from the same limitations as for the traditional home emergency call devices: if the check-back telephone call to the home also will not be answered, complicated research has to be initiated by the alert serving agent. This includes a potential questionnaire of local neighbors by telephone …, in order to decide whether a costly on-site emergency intervention shall be done rsp. an intervention team shall be send out to the alerting home.

Now, with the availability of the LTAS, the alert serving party can immediately retrieve contextual information about the recent history of ADLs in the corresponding home, even if the check-back call will not be answered. This allows faster and more fact based decisions about sending out the emergency intervention team, especially at nighttime, when no neighbors can be asked. Moreover, with the cellular telephone included in the smartwatch worn at the wrist of the person in need of support, the likelihood of establishing a successful speech connection between the alert serving agent and the person in need of support is substantially increased. If a health hazard has been detected by the smartwatch and a local pre-alert has not been answered by the wearer of the watch (terminating false alerts) the smartwatch app initiates the telephone call. For such an automatically established direct speech connection from the person in need of support, the presence of a severe health hazard typically has to be assumed, if the person is not able to respond verbally anymore.

For the computation of the wellbeing function w, the computation of its constituents and—partially—the adaption of the nominal values has been located to the stationary subsystem (for ß₁, ß₂) and the smartwatch (for ß_2, ß₃), in order to guarantee the functionality and the issuing of corresponding pre-alerts/alerts in a most reliable way, even if no other system component is momentarily at hand. The notion of a finite state machine has been proven to be a suitable implementation of the abstract wellbeing definitions above, for example see Fig. 8 (and cf. [10]) for a local implementation for the ß₂ joint health hazard handling of the ADLs A₂, A₃ on the smartwatch.

Overall the smartwatches were welcomed and have been accepted by the elderly people of our case study as non-stigmatizing, multipurpose devices with a readable, large enough display. In contrast to wireless remote units of the classic home emergency call devices, which were categorically rejected by the elderly because they felt (with a real age of 85 and more) “too young” in order to use these devices! Especially the orientation information provided by the app (see Fig. 5) was valued as the feature with highest value for the person in need of support because it enables a continued participation in social interaction without directly disclosing the mental disability; an unobtrusive glance at the watch compensates—or at least camouflages—the failing personal memories.

From the assistance system perspective the smartwatch app complements the functionality of the system in that EDLs like » falls « can be directly recognized which so far could only be indirectly observed via its potential consequences by the stationary assistance subsystem. In addition, the smartwatch app enables the detection of more fine-grained ADLs like » midday naps « or » liquid ingestion « , ADLs, for which it was so far impossible to observe them via the presence sensors of the stationary subsystem. And for outdoor stays, the direct intertwining of the stationary subsystem and the smartwatch allows the fully automatic recognition of the ADL » absence from home « . For the stationary subsystem alone, so far the subsystem had to be informed manually at least about the beginning of an outdoor stay in order to prevent false alarms. In practice, this often was forgotten and consequently caused such irritating false alarms. And, last but not least, the smartwatch app enables the reach of the assistance beyond the spatial borders of the home and includes protection for strolling, shopping, visits to the doctor. Especially a » runaway situation « , which not infrequently results in a deathly outcome and produces extensive and costly public search actions, can be dealt with in very short-term. On the other hand the current smartwatch app is of productive use only between getting up in the morning and retiring to bed at nighttime, while the stationary assistance is on duty at nighttime when the smartwatch needs to be recharged.

From the perspective of the care-giving persons and distant family member, the main advantage of the smartwatch app is the permanent reachability of the persons in need of support due to the integrated cellular radio of the smartwatch. Especially for elderly people of high age the general tendency is not to carry their mobile phones with them when they leave their home because they have not been acquainted to this during their working life. With the smartwatch typically continuously worn during the whole day, a forgotten or deliberately not carried along smartphone has no consequences with respect to reachability and the functionality of the assistance services.

4.2 Usability

The stationary assistance subsystem and the smartwatch differ substantially in their usability. As a subsidiary system, the stationary assistance subsystem will not actively operated by the person in need of support. At most, the person in need of support has to react to prealerts in order to signalize that everything is currently ok by cancelling those prealerts. The person in need of support needs not to be aware of the assistance technology “in the background” rsp. is not required to have any in-depth understanding of the stationary assistance subsystem. This makes the technology especially suited also for people with mental disabilities, e.g. caused by MCI or dementia.

In contrast, use of the smartwatch requires a basic “digital competence”, in order to gain a beneficial use by the wearer. This starts with insight in the necessity of recharging the electronic device each night (as it is self-evident practice for most smartphones). It includes basic capabilities of operating the GUI of the smartwatch and circumventing the use of the default installed, but partially complex, manufacturer provided system apps on the watch. Also, the wearer needs to have a basic understanding about the many assistance functions of the app, which only can deliver effective support, if he/she are really wearing the device. Otherwise, the additional value of wearing the smartwatch in contrast to the used smartphone will not be perceived. And, for example, the wearer should have a basic understanding about the operation principles of GPS, by which the app can only track his/her exact position after a few moments with free sight to the open sky. It turned out also for the field test,³ which followed our case study, that the prior knowledge or sharing of this “digital competence” was a mandatory precondition and decisive factor for the successful use of smartwatches. This recommends the application of smartwatches for elderly persons primarily not of very high age, which ideally already have a mobile phone usage background from their prior active work life.

4.3 Practicability

Another aspect concerns the charging procedure, which in our case of the Samsung Gear™ S requires a mechanical precision attachment of the watch to a charging adapter. This requires fine motoric capabilities typically not present anymore for elderly people and caused regular handling problems during our study. We therefore included a specific SMS to the caregivers and also an alert to the LTAS requesting recharging in case of an ongoing low battery situation (event E₁₀)! Fortunately, the manufacturers have addressed this problem meanwhile. An inductive charging without mechanical contacting of the watch—as proposed by the Moto 360™ and Apple Watch™—will be the future general standard (now also featured by the Samsung successor model Gear™ S2).

4.4 Synergies by Integrating Stationary and Wearable Information in the LFAS

The integration of delivered information from the stationary subsystem and the smartwatches in the LFAS has been used especially for mutually completing the information from both sources and thus reducing false alerts. For example, if the smartwatch wearer has left his home and the smartwatch battery would fade out during his absence, no return information will be later generated by the smartwatch. In such a situation, if the stationary presence sensor signals the presence of a person at home at a time conforming with the usual return time of the smartwatch wearer for this day of the week, it will be assumed by the LFAS that this present person in the household will be in fact the smartwatch wearer and no ß₂ excessive absence alert for A₂ will be communicated. In the opposite case, if no one returns to home in due time, even in case of an empty battery (and therefore a silent smartwatch), the LFAS will communicate an excessive ß₂ absence alert based on the prior E₂ information about leaving the home. Also E₁ »fall « EDLs occurring in the household and eventually not detected by a worn smartwatch, will be finally detected by the presence sensors due to overtime presence at the location of the fall, if the person in need of support should not move resp. leave the location after the fall.

5 Discussion

The choice of the most suitable sensor technology for an assistance system is an ongoing discussion based on the respective technologic progress. In addition to the stigmatization and cost/value benefits aspects mentioned above, additional aspects like the estimated maturity of a sensor technology and its foreseeable support and maintenance costs as well as the subjective fear of (especially German) users of “electro smog” had to be additionally taken into account.

In [6, 18] comprehensive overviews of stationary, wearables and image-based sensor projects are presented. The up-to-date overview in [18] has been elaborated in the scope of the SPHERE project in order to explore the strength and limitations of the different sensor technologies for monitoring various health conditions. [19] focuses on the usage of such sensors especially for the purpose of fall detection. In our estimation, the reported slightly decreased recognition rate of falls by wearables, smartwatch sensors is (more than) counterbalanced by their universality of use also outside the home, the substantially reduced price of wearables in contrast to floor sensors for fall detection and the non existing privacy concerns and installation problems at the room ceiling in contrast to any kind of image based fall sensors.

In [20], an 80% recognition rate of 19 fine-grained ADLs is reported, if active sensors are not only placed on the wrist, but additionally on the legs, waist and back of persons, and active Bluetooth beacons had been placed in each room of the home. In our estimation and from a practical perspective, the unquestionable achievements in ADL recognition by this plenitude of sensors has to be questioned against the additional 2, 4 GHz radiation emitted by the beacons and the burden of attaching and wearing so many sensors on a daily basis. Our approach builds on the opposite principle: identifying only wellbeing and health critical ADLs with a minimal number of sensors, but with a maximum precision. The non-intrusive (electric) appliance load monitoring of [21, 22], which requires only one central digital load monitor at the electricity meters in the central junction box of the household would much more comply with our spirit.⁴

In [23] an interesting approach focusing on things in contrast to our movement of persons centered approach for ADL detection is presented. In the thing based approach all relevant things (dishwasher and washing-machine safe) in the household are equipped with low cost RFID tags which can be attached to most tableware. In this approach, the ADLs will be concluded from the specific movements of things caused by human activity. The RFID tags attached to the things will be periodically powered by multiple antennas covered in the walls of the home. These antennas also receive the response from the RFID tags, including their individual ids, in a very short time frame after inductively powering the tags. From the response signals received by different antennas, the position of the things will be monitored by trilateration. The sequence of moving positions will then be used for activity recognition via the help of Bayesian networks. Although this technology is limited to the home by its nature, its strength is a fine-grained analysis of eating behavior, one of the weak aspects of our approach.

In [24, 25] a monitoring approach based on a wireless sensor network (WSN) for the home is presented, which utilizes specific pressure resp. occupancy sensors for chairs and beds in addition to standard PIR movements sensors. Our view is that the costs for such dedicated sensor devices, including initial installation in the furniture and regular maintenance for providing electric power to the sensors, will not be justified for the future by the gain in the precision of ADL recognition. With our high-quality presence sensors and in combination with the smartwatch sensors and the finite state machines in the background, occupancy in bed and sitting in a room can be typically recognized resp. concluded from these more universal devices.

6 Conclusions and Future Work

The amendment of the original stationary assistance subsystems by smartwatches and the LTAS indeed increases the perceived value of the assistance technology:

• for the users, by providing more mobile services, e.g. the wearable orientation information in Fig. 5, and extending the reach of support services beyond the spatial boundaries of the home,

• for the system, by recognizing more ADLs (e.g. » runaway situation « , » midday nap « , » liquid ingestion «), detecting EDLs directly rsp. in a shorter time frame (e.g. » fall «) or recognizing ADLs now in a more reliable and fully automatic way (e.g. » absence from home «) and by integrating wearable and stationary sensor information aiming at a reduced number of false alerts,

• for parties in charge of reacting to external alerts (distant family members, home emergency call center agents), by providing contextual information for more fact based decisions about what happened to the person in need of support at the distant home or on the go (e.g. services in Fig. 6).

Our future work will be directed towards the elaboration of the developed data mining based activity recognition technology ([11–13]) for improving the resource efficient and stable execution on the smartwatch, for a more reliable detection of falls and for detecting more fine grained ADLs on the basis of this technology like » teeth-brushing «, » hand washing « and » combing « as symptoms of a well-managed life. Another focal point will be the enforced utilization of the LTAS for identifying and filtering—in result: reducing—potential false alerts by heuristically combining all relevant information from wearables and stationary sensors. This combination of historical and current information about EDLs, ADLs will also be used by the LTAS for producing fact-based, trustworthy and comprehensible natural language justifications for communicated alerts.

Also hardware progress on the sensor equipment of the smartwatches, for example for pulse oximetry carried out within the smartwatch on the wrist, would be highly welcome. In the long-term, we do foresee the evolution of the smartwatch app towards a regulated medical software product, at least, when together with today’s heart rate measurement and the pulse-oximetry in the future the blood pressure could be also measured with the smartwatch resp. its wrist band. The basic research for this, namely to utilize the transition time of the arterial pressure valve from the left edge of the wrist band to its right edge, a time period which is strongly related to the individual blood pressure, is already on the way (ETH Zürich, cf. [26]). With (i) current heart rate, (ii) the arterial oxygen saturation of the blood and iii) the blood pressure, the most essential three medical parameters for a diagnosis of vital health hazards would be available for the smartwatch app. This regulated medical software product would run on a smartwatch consumer hardware and thus would be required to cope with potentially wrong and/or inconsistent hardware sensor values of the consumer hardware without leading to a wrong diagnosis (cf. [27] for those requirements).

This will elevate the added value of the smartwatch app and level of possible assistance by the app to a complete new quality.