1 Introduction

Ubiquitous and pervasive computing is a paradigm in which computing technology becomes virtually invisible by being embedded in our environments. As embedded computers get smaller and become integrated into noncompeting artifacts, they can effectively physically disappear. With the addition of communication capabilities and pervasive networking, such artifacts can be associated together and remotely accessed in both familiar and novel arrangements to make highly personalized systems (Doctor et al. 2005). Ambient intelligence (AmI) is a new information paradigm, which combines pervasive computing with intelligent data representation and learning systems. AmI enables the environment to be “aware” of the presence and context of inhabitance, their activities and preferences using transparent, unobtrusive intelligent, personalized interconnected systems and services (Ducatel et al. 2003). The health and social care domain is an important area where pervasive computing and ambient intelligence research is being increasingly applied to provide eHealth related services for our ageing population (Chiarugi et al. 2006). An increase in life expectancy has produced an older population that needs continuous monitoring and assistance, especially in case of progressive age-related illnesses such as dementia.

Dementia is a collective term used to describe various different brain disorders that have in common a loss of brain function that is usually progressive and eventually severe (Alzehimer’s Society 2011). The observable characteristics of dementia-related cognitive decline is generally: gradual loss of memory, difficulty in performing familiar tasks or complex tasks, changes in mood or behaviour and disorientation. In the UK there is a gradual increase in the number of elderly people aged 65 and above who suffer from dementia-related cognitive disorders. The number of patients with dementia in the UK is estimated to be 821,884, representing 1.3 % of the UK population (Dementia 2010). The majority of dementia sufferers tend to be supported by carers and live independently in their own homes. Due to the progressive and degenerative nature of diseases such as Alzheimer’s, long-term remote management of patients is essential in order to track the stages of the disease, adjust the course of therapy and understand the changing care needs of the patient.

Observing a patient’s activities can be used to assess stages of the disease. For example, early stages of cognitive impairment may cause forgetting certain steps in familiar tasks or difficulty in performing new and complex tasks. Whereas in moderate stages the patient may exhibit a lack of awareness in the time of day leading to pronounced changes in behaviour from day to day, or impaired communication (PDE-3193 2004). The patient’s perception of their environment may also be affected due to their cognitive deterioration. Dementia can significantly change the interpretation of sensing and as a consequence perception (Bakker 2003). People with dementia have an altered sensitivity for indoor environmental conditions, which can induce problematic behavior that may form a serious burden for professional and family carers day and night (Hoof et al. 2009). Patients can express themselves more reactively emotionally and behaviorally to their environment rather than acting upon it (Weaverdyck 1991).

Taking the example of to thermal comfort, due to the pathology of many dementia patients involving atrophy of brain tissue, the perception of the thermal environment as well as the thermoregulation may be different from older adults without dementia. A patient may not realize that a bathroom is too cold, only that he/she is uncomfortable and may not associate the room’s temperature with discomfort, which can result in frustration or anger (Warner 2000). Patients may also over compensate the control of heating or cooling systems by setting them to very high or very low temperatures, and then forget that they have manipulated the system. It is therefore important to monitor how changes in the states of environment are affecting a patient’s actions and build up a model of the patient’s individual comfort perceptions, which can then be analyzed and tracked over time to detect abnormal changes that may be caused due to cognitive decline.

In this paper, we propose an ambient intelligent system which could be deployed in a home-based environment to monitor and track the disease progression of dementia patients, see Fig. 1. Our approach would create a Personalized Home Sensor Network (PHSN) for monitoring key aspects of a patient’s cognitive behavior and activities. The PHSN would consist of Local Sensor Groups (LSG) of simple unobtrusive wireless sensor devices, which would be retrofitted to positions and items at locations around the home, and accessed using an open middleware component based software architecture. The placement of each LSG would be determined by a variety of patient-related factors alongside the advice of care staff and health professionals. For each LSG we propose using an intelligent agent learning and adaptation technique we had previously developed in (Doctor et al. 2005) that learns fuzzy rules modelling the particular behaviours of the patient in the environment. The fuzzy rules can be adapted online and used to track medium- to long-term behaviour changes, which could be due to patient’s cognitive decline. We extend the fuzzy rules to incorporate rule quality and importance measures, which we use to help identify the associations between groups of firing rules to determine prominent rule groupings that tend to be associated with the same behaviour and how these change overtime. We will refer to these as macro level changes. We also propose extending our approach to focus on individual environmental parameters and their associations to the patient’s actions and quantify their perceptions of comfort using fuzzy sets to track subtle perception changes, which we will refer to as micro level changes.

Fig. 1
figure 1

Flow diagram showing the proposed AmI system comprising of the PHSN and the intelligent fuzzy agent learning and adaptation stages for a given LSG

Full size image

In Sect. 2 we present a literature review discussing environmental sensors and wireless sensor network technologies, intelligent agents and fuzzy systems for realising AmI and previous work on the development of assistive technologies for monitoring dementia patients. Section 3 briefly introduces an open middleware framework called OSGi with reference to existing work in which the architecture has been used to integrate wireless sensor devices into ambient intelligent environments for elderly care monitoring. In Sect. 4 we describe our fuzzy learning and online adaptation approach for the intelligent agent to enable the tracking of cognitive decline in dementia patients. In Sect. 5, we present experiments and results. Finally conclusions and future work are presented in Sect. 6.

2 Literature review

Research has shown that people who develop cognitive decline seem to experience subtle changes in daily activities before the diagnosis of dementia is confirmed (Gustafsson et al. 1995). However periodic observations and assessments have been found to be ineffective in detecting or diagnosing cognitive impairment in early stages of the disease (Binegar et al. 2009). Therefore it is important to continually monitor everyday activities over time in order to provide a more effective way of identifying activity changes that can signify cognitive deterioration.

It can be difficult for relatives and carers to continually observe a patient and recognise the subtle changes in a patient’s behaviour and daily activities, which may signal progressively worsening stages of the disease. It can also be difficult for health professionals to accurately track the patterns of a patient’s cognitive impairment and the impact this is having on daily living and wellbeing. Wireless sensor networks comprise of clusters of sensor devices equipped with a radio transceiver, and small microprocessor deployed over a specific area, which wirelessly communicate data to a central system. They are based on emerging technologies such as communication technologies (RF communication, ad hoc networking routing), semiconductor, embedded systems and micro sensor technologies (PureLink Technology 2012). The prevalence of relatively low cost off-the-shelf pervasive wireless sensor devices can be used for sensing motion, light level, temperature, occupancy, chair/bed pressure or appliance hazards such as checking if the cooker has been left on (Woolham et al. 2007). Simple switch sensors can be used to determine if drawers and cupboards have been opened/closed, and whether specific electronic devices have been switched on or off. Radio Frequency Identification (RFID) tagging of daily personal items can be used to track how the patient interacts with and places movable items in the home during routine activities. For the purpose of this paper we will divide these sensors into passive input sensors such as temperature, light level or occupancy, and active output actuation based sensors such as appliance states, sensors for light switches or doors and cupboards states and RFID tracking of movable objects (Matic et al. 2008). The vast majority of wireless sensor devices will support one of several commercial wireless sensor networking standards such as Zigbee, ANT, Z-Wave, ONE-NET to name a few. The development of open middleware architectures such as OSGi (OSGi 2011) can be used to dynamically integrate heterogeneous commercial proprietary sensor devices and network technologies together for easy deployment in a residential environment. This can allow AmI systems to seamlessly record data simultaneously from various sensor devices in context of the resident’s interactions with the environment.

Intelligent agents can be embedded in the user’s environment to realize AmI systems, which can recognize the patient’s interactions and program themselves to learn their behaviours (Doctor et al. 2005). Soft computing approaches can be used to identify and model behaviours based on the patterns of association between sensed environmental conditions (light level, occupancy, time of day etc.) and specific actions being performed by the patient (switching on the lights, changing channels on the TV or moving an object to a specified location). In particular Fuzzy Logic Systems (FLSs) have been credited with being an adequate methodology for designing robust systems that are able to deliver a satisfactory performance when contending with the uncertainty and imprecision attributed to real world pervasive computing environments. An FLS represents a nonlinear mapping of input states to specific output conditions. FLSs provide transparent and flexible model representations, which can be generated from data, and are based on linguistic quantifiers and human readable rules. Fuzzy sets provide a methodology for computing with words using linguistic quantifiers such as ‘High’ or ‘Near’ (Mendel 2001). They also provide a means for quantifying abstract concepts and perceptions such as ‘Cold’, ‘Comfortable’ or ‘Morning’ based on associations between environmental input sensor states and the users behaviour or actions. Fuzzy sets can be combined with fuzzy If–Then rules which can approximate independent local models of a patient’s particular behaviours that map sets of input sensor states to sets of output user action states. The flexibility of FLS’s means that on a macro level individual fuzzy rules can be adapted to approximate short-term changes in the patient’s behaviours and identify medium- to long-term behaviour variations which may be the consequence of progressive cognitive decline. On a micro-level we can also monitor and quantify patient’s perceptions of comfort and time of day using fuzzy sets, and track if these perceptions change due to either normal variations or abnormal changes as a consequence of dementia.

The importance of pervasive continuous and reliable long-term monitoring of dementia is emphasized in (Arnrich et al. 2010). Significant results have been obtained in the development of smart environments for monitoring hand washing (Peters et al. 2009) (Mihailidis et al. 2004), cooking (Mynatt et al. 2004), taking medications (Hayes et al. 2006), dressing activities (Matic et al. 2008), as well as behaviours such as agitation (Biswas 2006) pain and discomfort (Bonroy 2008). These approaches however tend to focus on very specific activities. They rely heavily on the use of cameras and computer vision in detecting and classifying activity patterns, and have tended to be conducted under very controlled settings over short periods of time. Such systems may not necessarily be practical to use in a patient’s home due to logistical and privacy related reasons. A study was undertaken on the use of simple motion sensors around the home to spot subtle changes in mobility and behaviour of Alzheimer’s suffers over time that may precede memory loss (Neergaard 2007). However this study was again limited to using a small set of sensory inputs, which focused on one indicator of cognitive decline.

3 OSGi middleware framework for wireless device integration

The Open Services Gateway initiative (OSGi) framework defines a standardized component and service oriented computing environment in Java, allowing applications to be constructed from small, reusable and collaborative components. The framework provides a service-oriented architecture that enables these components to dynamically discover and collaborate with each other (OSGi 2011). Applications or components are known as ‘bundles’ and can be remotely installed, started, stopped, updated and uninstalled without requiring a reboot of the system. In (Conde et al. 2010) an OSGi-based solution is described to integrate wireless sensor devices into ambient assisted living environments. The approach is based on using a layered architecture in which services offered by different wireless devices are bundled together in an application layer that allows other applications and system to uniformly access the services of each device. In lower layers, application bundles are associated with the sending and receiving of data from the proprietary wireless networked devices (Conde et al. 2010).

OSGi is well suited to our residential context where groups of different sensor devices need to be configured and retrofitted to specific locations within a patient’s home. The framework can provide a service oriented interface mechanism with which each intelligent agent can homogeneously capture data received from different heterogeneous sensor devices configured within each LSG.

4 Fuzzy agent approach for dementia monitoring

The placement of the sensors selected for each LSG would be based on factors such as the physical layout of the residence, the aspects of a patient’s cognitive behaviors that need to be monitored, how advanced their dementia already is, and the kind of unassisted activities the patient is able to currently perform. For each LSG the intelligent agent learns and adapts to the user behaviour in the environment using a variation of our previously developed Adaptive Online Fuzzy Inference System (AOFIS) approach described in (Doctor et al. 2005). Our approach is an unsupervised data-driven approach for generating a FLS from data collected through monitoring the patient’s interactions with the LSG over a period of time. The generated FLS models the patient’s habitual behaviours using fuzzy rules that map sets of input sensor states to sensed output states of devices and utilities based on the patient’s interactions with them. The adaptive FLS will passively compare its learning model with the user interactions in the environment. The system will allow the short-term tuning of the learnt fuzzy rules to better approximate the patient’s behaviours. Over the medium- to long-term, the agent will record subtle persistent changes in the patient’s behaviour as new and adapted fuzzy rules. These can be compared with original tuned behaviour rules to determine how specific behaviours have changed over time in each LSG. The agent can present behaviour changes to health professionals who can make an informed assessment of the cognitive state of the patient. Our approach comprises of five stages as follows (and illustrated in Fig. 1).

4.1 Capturing input/output data

The agent passively monitors the user’s states and actions within the LSG. Either periodically or when the user performs an action in the environment, the agent will record a ‘snapshot’ of the current inputs (passive sensor states) and outputs (active sensor states adjusted or previously set by the patient). These ‘snapshots’ are accumulated over a period of time so that the agent observes as much of the patient’s interactions within the environment as possible. The agent learns a descriptive model of the patient’s behaviours from the accumulated data. Therefore, given a set of multi-input multi-output data pairs:

$$ (x^{(t)} ;y^{(t)} ),t = 1,2, \ldots ,N $$
(1)

where N is the number of data instances, \( x^{(t)} \, \in \,R^{n} \) and \( y^{(t)} \in \,R^{k} \). The agent extracts rules which describe how the k output variables \( y\, = \,(y_{1} , \ldots ,y_{k} )\,\, \in \,\,R^{k} \) are influenced by the n input variables \( x\, = \,(x_{1} , \ldots ,x_{n} )\,\, \in \,\,R^{n} \) based on the sampled data.

4.2 Generation of FLS modelling patient behaviours

It is necessary to be able to categorise the accumulated user input/output data into a set of fuzzy membership functions which quantify the raw crisp values of the passive input and active output sensors into linguistic labels such as ‘normal’, ‘cold’ or ‘hot’. As we want to learn the particular behaviours of the user, the agent should define the membership functions from the user’s recorded input/output data. In our previous work, (Doctor et al. 2005) we developed a technique for generating type-1 membership functions from data that was based on using a Double Clustering approach combining Fuzzy-C-Means (FCM) and agglomerative hierarchical clustering (Castellano et al. 2002). We can equally also use FCM over each dimension of the input and output space to generate approximate membership functions (MFs) directly from the fuzzy partition matrix generated by the algorithm.

The defined set of membership functions are combined with the existing user input/output data to extract the rules defining the patient’s behaviours. The fuzzy rule extraction approach used is based on an enhanced version of the Mendel Wang (MW) method developed by Wang (Wang 2003). This is a one pass technique for extracting fuzzy rules from the sampled data. The fuzzy sets for the antecedents and consequents of the rules divide the input and output space into fuzzy regions.

The approach extracts multi-input multi-output rules which describe the relationship between y = (y 1 ,..,y k ) and x = (x 1 ,…,x n ) T, and take the following form:

$$ IF\;x_{1} \;{\text{is}}\;A_{1}^{(l)} \ldots \quad {\text{and}}\quad x_{n} \;{\text{is}}\;A_{n}^{(l)} \,,THEN\,y_{1} \;{\text{is}}\;B_{1}^{(l)} \ldots \quad {\text{and}}\quad y_{k} \;{\text{is}}\,B_{k}^{(l)} $$
(2)

\( l\, = \,1,2, \ldots ,\,M \), where M is the number of rules and l is the index of the rules. There are \( V_{n}^{{}} \) fuzzy sets \( A_{s}^{q} ,q\, = \,1, \ldots ,\,V_{n} , \) defined for each input \( x_{s} \) where \( (s\, = \,1, \ldots ,\,n) \). There are \( W_{k}^{{}} \) fuzzy sets \( B_{c}^{h} ,h\, = \,1, \ldots ,\,W_{k} , \) defined for each output \( y_{c} \) where \( (c\, = \,1, \ldots ,\,k) \). The agent now extracts rules in the form of Eq. (2) from the data. Once the agent has extracted the membership functions and the set of rules from the user input/output data, it has then learnt the FLS that captures the patient’s LSG based behaviours. The extracted fuzzy rules represent local models that map a set of inputs to the set of outputs without the need for formulating any mathematical model. Individual rules can therefore be adapted online influencing only specific parts of the descriptive model learnt by the agent (Doctor et al. 2005).

4.3 Passive monitoring and short-term online tuning of behaviour rules

The FLS will passively monitor the patient’s behaviours associated with their interactions with the LSG. Rules whose antecedent fuzzy sets match the current sensed state of the environment will be fired and each rule will also keep a frequency of the number of times it has fired over its lifetime. This can be used to prune rules that have fired infrequently and therefore contributed the least to the patient’s learnt behaviour model. This will allow the rule base to be kept to a manageable size reducing memory overheads. We also record the online fuzzy support of each unique rule in the rulebase. A rule’s fuzzy support is calculated as the product of the rule’s support and firing strength. The support of a rule refers to coverage of input patterns, which map to it (Agrawal and Srikant 1994), while the it’s firing strength measures the degree to which the rule matches those input patterns (Mendel 2001). The rule’s fuzzy support can be used to identify the most dominant rules associated with a specific behavior. They can further be used to help identify the associations between groups of firing rules to recognise rule groupings that tend to be activated together and therefore associated with the same behaviour.

The patient may make small adjustments to their behaviours within the LSG and the FLS will need to be tuned over a short period of time to allow the system to better approximate the patient’s behaviours. Whenever the patient performs an action within the environment (e.g., switching the TV on/off), the agent will check its learnt behaviour rules online and either adapt or add new rules if the patient’s action(s) have changed. A snapshot \( t^{'} \) of the state of the input/output sensors in the environment is recorded and passed to the rule adaptation routine. Each input sensor state is compared to each of the antecedent sets \( A_{s}^{(l)} \) of a given rule in the rule base to determine its membership values. The weight \( wi^{(l)} \) of the rule is then calculated to determine the degree of firing of the rule as shown in Eq. (3).

$$ wi^{{(t^{'} )}} \, = \prod\limits_{s = 1}^{n} {\mu_{{A_{s}^{q} }} (x_{s} (t^{'} ))} $$
(3)

where the weight of a rule \( wi^{{(t^{'} )}} \) is a measure of the strength of the points \( x^{{(t^{'} )}} \) belonging to the fuzzy region covered by the rule. If the weight of the rule is \( > 0 \), then the rule has fired and matched the sensed input states of the environment when the actuation occurred.

The consequent fuzzy sets that give the highest membership to each of the current output sensor states are selected as follows:

$$ \mu_{{B_{c}^{h*} }} (y_{c} ) \ge \mu_{{B_{c}^{h} }} (y_{c} ) $$
(4)

for \( h = 1,2, \ldots ,\,W. \) The \( B_{c} \) is chosen as \( B_{c}^{{^{h*} }} \). Where c = 1,2..,k.

These are compared to the existing consequent sets of the fired rule(s) to determine if there has been a change in the patient’s actions (output states). If a change has occurred the new consequent fuzzy sets \( B_{c}^{{^{h*} }} \)will replace the consequent sets of the fired rule(s), adapting the rule(s) to reflect the patient’s behaviour changes.

If none of the existing rules fired, new rules are added based on forming rules from the input fuzzy sets. For each input sensor state \( x_{s} \) the fuzzy sets that give a membership value where \( \mu_{{A_{s}^{q} }} (x_{s}^{(t')} ) \) \( > tr \), where \( tr \) is a firing threshold, are identified. This leads to a grid of identified fuzzy set(s) for each input parameter. New rules can be constructed based on each unique combination of consecutive input fuzzy sets. The consequent fuzzy sets for each of the new rules are determined using Eq. (4). This allows new rules to be gradually added to the rule base. The agent will also add new rules when the currently monitored environmental state is undefined by the existing rules in the rule base; i.e. none of the existing rules fired. In this case the agent will create new rules where the antecedent sets reflect the current input states of the environment and the consequent fuzzy sets are based on the current state of the outputs. As each rule represents a local model of a specific patient behaviour, adapting the rule will only change a specific part of the descriptive model learnt by the agent without requiring the remodelling of all the patient’s learnt behaviours.

4.4 Long-term rule adaptation and generation

The agent can adapt existing rules and add new rules based on long-term changes in the patient’s behaviour within the LSG that may be a direct result of the progressive stages of dementia. The mechanism used to adapt or generate new rules is the same as in Sect. 4.3. The difference here is that rule changes and rule additions will be recorded as new rules in a separate ‘probationary’ rule base and hence will not change the current behaviour model. As in Sect. 4.3 the individual fuzzy support of fired rules for both the tuned model’s rules and the new probationary rules will be recorded. Probationary rule(s) that indicate a persistence and relevant behaviour change over time will then be compared, (on the basis of their antecedent and consequent fuzzy sets), to the current LSG behaviour rules to determine which behaviours have been changed.

These long-term rule changes will then be fed back to update the patient learnt behaviours rules, allowing the system to adapt its behaviour model to account for how the patient’s decline in their cognitive abilities is affecting daily activities over time. It is conceivable that over very long periods of time patient’s behaviour models may need to be rebuilt from monitoring the patient interactions again. This is likely to occur due to significant changes in the patient’s daily routines, or reconfiguring the LSG by adding or removing sensor devices.

Because our approach decomposes the pervasive environment into units of LSGs, it allows each agent-based behaviour model to be associated with a specific sensing area in the home, which can be individually monitored to track the patient’s cognitive state. Each behaviour model is also more compact due to smaller more manageable input sizes and hence tractable in terms of the size of the generated rule base. This also means that the fuzzy behaviour rules are more easily interpretable to end users of the system.

4.5 Assessment of LSGs marco level behaviour changes and micro level perception drift

For each LSG, macro level behaviour changes will be reported to health professionals and care providers in the form of linguistically interpretable rule comparison reports outlining behaviour changes that have taken place. This will allow observers to make an informed assessment of whether or not the patient’s condition has worsened due to cognitive decline, their state of mind and ability to perform routine activities in their home. This will enable a judgment to be made on how their care plan needs to be updated to address changes in their care needs.

Micro-level analysis can also be conducted on the captured input/output data described in Sect. 4.1. Here we can take the data for a single or set of the LSG input devices and analyse there individual or collective associations to controlled or observed output states which can represent some simple or composite perceptions of comfort. For instance the perceptions of ambient light level could be divided into linguistic labels such as bright, moderate, or dark. These linguistic notions of brightness can be quantified based on the sensed ambient Lux levels of the room or environment. We can therefore quantify such perceptions of based on analysing the distribution of input sensor values in the data associated with a give output state that represents the perception, i.e. input sensors values for internal light level associated with the switching on state of the lights representing the perception of darkness. From these labelled sensor value distributions we can construct fuzzy sets that can be used to quantify and visualise these perception states. The fuzzy sets can be used to model the current micro-level comfort perceptions of the patient and be tracked to see how they might change over time as a result of normal fluctuation or the effects of cognitive impairment.

5 Experiments

Experiments have been conducted in a real pervasive computing environment test bed demonstrating the capability of our intelligent agent approach at learning a fuzzy behaviour model from monitoring a user in the environment over a period of time (Doctor et al. 2005). The experiments also demonstrate how the initially learnt behaviour model can be tuned to better approximate the user behaviours over a short period of time (Doctor et al. 2005). The test bed used was a room-based student bedroom environment comprising of a large number of embedded sensors, concealed actuators (e.g. buried in walls and ceiling) with the intention that the user would be completely unaware of the intelligent infrastructure of the room. The sensor and actuator devices were based around three networks, Lonworks, 1-wire (TINI) and IP which were accessed using a purpose-built Java middleware platform allowing the agent homogeneous access to the devices in the room. The student bedroom environment consisted of a variety of sensors and actuators. The sensors in the room sensed: internal light level; external light level; internal temperature; external temperature; bed pressure; chair pressure; occupancy; time measured as a continuous input on an hourly scale. The actuators in the room consisted of: 4 variable intensity spot lights located around the ceiling; desk lamp; bedside lamp; motorised window blinds; fan heater; fan cooler; and two PC based applications comprising of a media playing and word processing software application. The test bed environment demonstrates a bedroom/study LSG that would be typically configured for our proposed system. Experiments were performed in which a single user lived in the environment for a period of five consecutive days. During the monitoring phase that lasted for three consecutive days, the agent passively recorded the user interactions and activities within the environment.

The online performance of the agent was evaluated on how well the generated behaviour model could capture and emulate the user’s behaviours over a period of time. Data acquired from the user’s interactions in the environment over the initial period of three days was used to train an initial Fuzzy Logic Controller (FLC) comprising of a rule base of fuzzy If–Then rules of the form shown in Fig. 2a. Figure 2b also shows an example of the fuzzy sets generated for one of the environmental inputs to the system.

Fig. 2
figure 2

a Example If–Then fuzzy rule modelling users behaviour. b Fuzzy sets for a system input

Full size image

5.1 Short-term online tuning

The agent was then run online for a further 3 days during which it monitored the environment and user’s activities, and it produced the appropriate control responses based on its learnt rules. During this time the user was allowed to override and adapt the agent’s learnt control responses if it was necessary to modify and tune them further. Whenever the user made changes to controls, the agent received the request, generated new rules or adjusted previously learnt rules and allowed the action through. The plot in Fig. 3a shows how the initially learnt behaviour model was tuned (based on the number of rule adaptations) until the model reached a degree of stability, by the end of day one, when the user no longer needed to override the agent’s automatic control responses. This demonstrates how the adaptation of each rule is able to gradually tune specific behaviours without affecting the entire learnt model. The agent continued to be run over the second day simulating a period of longer term constant monitoring during which the user behaviours remain constant and unchanged with only slight adaptive tuning required to the agent’s rule base. Figure 3b shows a histogram that provides a view of the individual online rulebase changes that occurred over the initial 2 days. The histogram depicts how there is an initial period of high frequency of rule adaptations over day 1 which then reduces to over the second day as the system stabilizes.

Fig. 3
figure 3

a Number of online rule adaptations based on short term tuning and long term online adaptation of behaviour rules. b Histogram of individual rule adaptations based on short term and long term online adaptation

Full size image

5.2 Long-term behaviour change

As described in Sect. 4.4, our approach further extends on our previous work in (Doctor et al. 2005) by recording long-term changes in patient’s behaviours, which are then compared with the patient’s current behaviour model to identify behaviour changes. Specific behaviour changes would then be reported to care providers to help them assess the level of a patient’s cognitive impairment. The current behaviour model could then be updated to reflect the behaviour changes that occurred. To demonstrate the agent’s ability to provide long term monitoring of the patient’s behaviour changes, the occupant simulated a long-term behaviour change during the third day of the agent being run online. This simulated change is indicated on the plot in Fig. 3a by a sudden stepped increase in the number on online rule adaptations to the rulebase during the late afternoon on day 3. This can also be seen in Fig. 3b as two corresponding points of high frequency rule changes.

A separate experiment was conducted to show the actual effects on individual rules from simulating a subtle behaviour change on the agent’s initially trained rulebase. For simplicity in this scenario we focused on the occupant’s behaviours associated with their operation of a single output device, namely the fan heater in the room. One of the normal monitored behaviours of the occupant was to switch the fan heater ‘on’ during the night while reading in bed due to a reduction in the internal temperature of the room. We simulated the occupant suddenly changing their normal behaviour by switching the fan heater in the room ‘off’ during this time. Table 1a shows the initial user behaviour rules generated by the agent for both the ‘on’ and ‘off’ state of the output fan heater device. The rules are shown ordered according to their fuzzy support. The set of core user behaviour rules associated with each state of the fan heater are shown shaded the darkest. It is clear from these initial rules that the user’s normal behaviour is to switch the fan heater ‘on’ either at night when the internal and external temperatures are low and the occupant is laying on the bed, or in the morning or evening if the room temperature is low to medium. The top most highlighted heater ‘on’ rule with the highest fuzzy support is the specific behaviour that was then changed. Table 1b shows the updated user behaviour rules after the simulated behaviour change takes place. As can be seen from the adapted rules the original top most highlighted heater ‘on’ state rule has now become a least important heater ‘off’ state rule. This illustrates how specific localised changes to the user’s normal behaviour can be tracked by the system. These subtle changes could be the result of memory loss or lack of awareness in the time of day, or other effects caused from cognitive impairment. These rule changes can be reported to health professionals and care providers in the form of linguistically interpretable comparison reports outlining the specific behaviour changes that have taken place. This can help to make more informed assessments of disease progression in context of daily activity, and help to identify necessary changes to the patient’s care needs.

Table 1 (a) Initial generated user behaviour rules for the fan heater ‘on’ and ‘off’ states. (b) Adapted user behaviour rules for the fan heater ‘on’ and ‘off’ states simulating a localised behaviour change
Full size table

In the system proposed in this paper we extend from our previous work in (Doctor et al. 2005) to use a similar adaptation mechanism for adapting the patient’s learnt behaviours online. However our new system will only passively monitor and adapt the rule base online based on observing the patient’s interactions. This is because we are trying to detect subtle changes in the patient’s own interactions, which could be difficult to distinguish from automated control responses produced by the agent. An infrastructure for wireless automated control of devices may also be additionally challenging to configure and install within the patient’s home environment.

5.3 Long-term perception drift

In order to illustrate how a patient’s comfort perception might be captured and modelled in our system we referred back to the initial data acquired from the user’s interactions in the environment over the initial period of 3 days. We focused once again on analysing occupant’s interactions in the environment associated with their operation of the fan heater in the room. The operation of the fan heater can be used to represent the thermal comfort preferences of the user, which we divided into three linguistic labels: ‘cold’, ‘comfortable’ and ‘warm’. When the fan heater was switched on it represented the fact the user found the room to be cold. Alternatively when the fan heater was switched off, the temperature in the room was perceived to be warm. We made the assumption that within the periods in between consecutive switching on and off’s of the heater; the temperature in the room would have reached a comfortable level for the user. We quantified these perceptions based on analysing the distribution of internal temperature sensor values from the data that were associated with the periodic consecutive on/off output states of the fan heater. The Table 2 shows columns for internal temperature values for consecutive on/off fan heater states along with the time of day the values were recorded, represented as a continuous value on a hourly scale. For each pair of recorded temperatures for on/off heater states, the mean temperature was calculated as is shown in a separate column in Table 2. This represented a logical approximation of the comfort temperature for each pair of associated on/off temperatures. The overall mean standard deviation and min/max values for each column of temperature values were also calculated. Three fuzzy sets represented by overlapping gaussian MFs were generated, where the mean and standard deviation temperature values from each column represented the centre and spread of each set as is shown in Fig. 4. The three fuzzy sets represented the user’s linguistic perceptions for ‘cold’, ‘comfortable’, and ‘warm’. The fuzzy sets provide an approximate quantification of the user’s thermal comfort perceptions based on the sampled internal temperature ranges over a given period of time. As the overlapping fuzzy sets allow a gradual assessment of the membership of temperature values ‘belongingness’ to each set, they can represent the uncertainty that exists in the distribution of temperatures values, which were found to be associated with each perceptual state. The fuzzy sets can be used to visualise how numerical sensor and control data is approximated to different kinds of measurable comfort perceptions, which are also linguistically interpretable to an end user. These fuzzy sets can be periodically remodelled overtime and compared to determine if there are significant change is the patient’s perceptions, which could also be having an effect on their behaviours as a consequence of their progressive cognitive impairment.

Table 2 Internal temperature data associated with user control of fan heater recorded over 3 days
Full size table
Fig. 4
figure 4

Generated Fuzzy sets representing user’s thermal comfort perceptions modelled from sampled data based on user control of fan heater recorded over 3 days

Full size image

6 Conclusion

In this paper, we propose an ambient intelligent system to monitor and track the disease progression of dementia patients. The system could be deployed in a patient’s home environment where sensors are placed in grouped units pertaining to specific locations and factors related to the patient’s disease severity and the opinions of care providers. We suggest the use of a middleware architecture for providing standardized interfaces for communicating with various proprietary wireless sensor devices. For each LSG we propose using an intelligent agent employing a fuzzy learning and adaptation technique for generating fuzzy rules that model the normal habitual behaviours of the patient in the environment. The fuzzy rules can be adapted online and used to track medium- to long-term macro-level behaviour changes, which could be due to patient’s cognitive decline. We also propose a method of using fuzzy sets to quantify and visualise the patient’s specific comfort perceptions within their environment, which can be used to monitor micro-level cognitive changes due to perception drift. These changes are presented to care providers to enable them to make more accurate assessments of the patient’s cognitive abilities in performing daily tasks, and the kind of additional care support the patient needs. We have demonstrated through previously conducted experiments in a real pervasive computing environment, that our intelligent agent approach can learn to model the user’s behaviours and provide short-term tuning of the model to better approximate the learnt behaviours. We have shown that the agent can be used to identify and track simulated longer-term macro-level behavioural changes, which could be analogous to the progressive effects of cognitive impairment. We have further demonstrated an illustration of how the user’s thermal comfort perceptions may be captured and modelled using fuzzy sets, which can provide a visual means of monitoring micro-level perception changes which may be caused by the progressive worsening of the patient’s condition. Although the experiments we have presented have been conducted with non-dementia suffers they demonstrate the capability of our proposed framework to passively monitor individuals’, identify behaviour changes and model perception drift, attributed to the effects of cognitive decline in real suffers.

For our current and future work we intend to carry out experiments to test the stability and robustness of our approach based on long-term monitoring of users and its ability to detect and report subtle long-term behaviour changes based on the user’s activity variations that would further simulate effects of dementia-related cognitive decline. We also aim to investigate type-2 Fuzzy sets that could be used to better approximate and represent the quantification of user comfort perceptions, and model their uncertainty and variation with regard to behaviour. We eventually aim to deploy a prototype system within a residential care setting to monitor a patient with dementia to evaluate the system’s ability in providing insight into long-term behaviour variation to help care providers assess the changing care needs of the patient.