Activity modeling under uncertainty by trace of objects in smart homes

Abstract

A typical resident of a smart home can be an Alzheimer patient that forgets sometimes to complete the activities that he begins. The key point to assist the smart home resident is to model the activities and discover correct realization patterns of activities. To accomplish this task, we apply sensors to provide primary data about realization patterns of actions, operations, plans, goals and generally any objective that the smart home resident may desire to do. In the consequence, by applying fuzzy clustering techniques, we are able to mine sensor data to retrieve the realization patterns of activities, and so the prediction patterns of intentions are recognizable. Comparing the realization patterns with prediction patterns of activities, we would be able to predict the intention of the resident about the activity that the resident considers to realize. In this way, we would be able to provide hypotheses about the resident goals and his possible goal achievement’s defects. Spatiotemporal aspects of daily activities such as movement of objects are surveyed to discover the patterns of activities realized by the smart homes residents. In this research, uncertainty is considered as a property of activity recognition.

1 Introduction

Smart home is a home-similar environment that is surrounded and ambient by a collection of sensors (Roy et al. 2010). Embedded sensors in smart home provide observations about several aspects and features of activities of daily living (ADLs) in home (see Fig. 1). We analyze the gained data applying artificial intelligence (AI) techniques to assist the resident of the smart home to accomplish his ADLs independently or to provide health services to the patients. In this context, Alzheimer disease is justified as one of the applications of AI in healthcare domain.

Fig. 1

The kitchen area in LIARA smart home. The embedded sensors in this environment would capture accomplishment of actions such as opening the cabinets, turning the oven on, and movement of objects

Fig. 2

Fuzzy clustering the data using subtractive clustering technique in MATLAB. The less cluster radius be desired, the more variant classes are defined. Maximum number of classes would be number of individuals

Fig. 3

Increasing the influence range (ratio) or cluster radius rate, the clusters are chosen with broader constraints and include more individuals in fewer clusters

Primary goal to design smart home is to observe the actions and activities that the resident of the Smart home performs (Amirjavid et al. 2011b). These observations are done through the environment-embedded sensors and would be interpreted by AI techniques. For instance, Acampora et al. (2010) proposed a fuzzy logic based approach, Biswas (2011) and Nazerfard et al. (2010) introduced a probabilistic approach and Roy et al. (2010) applied a possibilistic method to interpret the observations and make inferences about the home and its resident states.

To provide primary data to do activity recognition, temporal data from movement of objects, accomplishment of actions like opening the doors or turning on the lights and many other features are captured.

A possible ultimate goal of smart homes such as LIARA¹ is to recognize the normality of home and its habitant state (Roy et al. 2010). Another intended goal can be assistance provision for the habitant to make him able to live independently at home (Bouchard et al. 2007). One other possible ultimate goal of smart home application is to manage better the resources (Cook et al. 2003), such as electricity, water and gaz. Other objectives such as home maintenance in absence of resident or automation provision to serve the habitant are surveyed to increase its resident’s quality of life (Bouchard et al. 2007). Moreover, a particular application of smart home is concerning to healthcare problems (Bouchard et al. 2007; Jakkula and Cook 2007).

In order to deal with such mentioned problems, the ADLs—realizing by a human—are surveyed in smart home environment. In this context, recognition of ADLs is one major difficulty that would be determined by analyzing the sensors’ generated events resulting from accomplishment of actions by the habitant, in order to predict the future possible contexts and so in the consequence provision of appropriate assistance. Intention recognition (Amirjavid et al. 2011a) and plan recognition (Roy et al. 2010; Bouchard et al. 2007) are two favorite artificial intelligence (AI) subjects that are applied to deal with prediction in smart homes and to recognize correct realization of activities.

A possible resident of the smart home is a patient that suffers from the Alzheimer disease that typically forgets the finalizing or continuing the activities that begin (Roy et al. 2010; Bouchard et al. 2007). The reason of this selection is that, at one hand, statistics illustrate a noticeable increscent in number of Alzheimer patients for the near future years (Bouchard et al. 2007) and at the other hand, considering this type of residents would reveal better the complexities in design of smart homes.

Providing facts, we refer to (Diamond 2007), in which, it is indicated that by 2,031 more than 500,000 Canadians would face the Alzheimer disease; however, today the number of present patients is not less than 280,000. Considering non-automated assistance provision for this amount of patients is a time consuming task for the caregivers, and considering human-oriented assistance would not let them live independently at home, so automatic assistance provision is desired. In this context, activity recognition is highlighted as a key point to achieve the final goal of smart home design, which is provision of automatic assistance for its resident.

Recognition of activities has attracted attention of several computer science communities in recent decades, since it provides personalized support for many different applications concerning to different fields of study such as medicine, psychology and human–computer interaction (Biswas 2011).

The goal of the activity recognition is to interpret the series of observations that are resulted from accomplishment of human actions and then to make inferences about the possible goals (activities) that are being achieved in the environment (Roy et al. 2010).

Activity recognition is a hard issue to be performed automatically without use of human-supervision. The reason is that the behavior of human is not easily predictable and the human does not often repeat his activities exactly as like as past realizations (there is uncertainty in behavior of human). For example, to realize the activity of “drinking”, a human does not always take a cup from a special area and he can drink water while seated on a chair or while standing anywhere else. Furthermore, the patients suffering from the Alzheimer disease probably make erroneous actions, activities or behaviors among their ADLs. Therefore, not only correct activities should be recognized, but also erroneous ones should be detected among them. Considering the fact that activities are possible to be realized in many different ways so, normality recognition of home and its resident would be a complex task.

The number of all possible normal states and normal ways that activities can be realized is not countable and so knowing and learning the mentioned states by simple definitions or observations is rather impossible. Furthermore, the same problems exist for the abnormal states and erroneous ways of activities realization. To solve such mentioned problem, we would need to do partial observation and reasoning would be done to make inferences about unobserved and inexperienced situations. Considering this fact, AI techniques are applied by researchers to solve the activity recognition problem. These techniques generally observe the environment through sensors. Then with or without help and supervision of human, activities are recognized

With the notion that, the Smart home provides large amount of data, it can be presumed and resembled as a big data warehouse (Jakkula and Cook 2007; Nazerfard et al. 2010; Biswas 2011). Applied traditional data mining approaches that deal with activity recognition, provide statistical and historical information about realization of activities as a sort of resulted knowledge. For example, temporal data mining approaches provide statistical information about the beginning times of activities (Nazerfard et al. 2010). A regular problem with traditional approaches to do activity recognition is that they do not consider the quality of realization of activities and so they do not confirm certainly the normal realization of activities. Furthermore, it can be said that they do not consider the activity recognition issue as a problem that a relatively big number of variables may help to solve it.²

Another noticeable problem with traditional quantitative approaches such as hidden markov model based ones (Singla et al. 2008) or dynamic Bayesian network based ones (Biswas 2011) is that the activities must begin from especial states called (initial states) which makes the Activity Recognition rather difficult to be applied in real situations. For example, to recognize the “drinking” activity, the glass should be taken from a particular area on a table (Biswas 2011).

Most of the introduced non-vision based approaches of activity recognition such as (Singla et al. 2008; Biswas 2011) are quantitative and their performance depend directly on the quantity of the training tests (Quinlan and Ghosh 2006). We find that quantitative approaches would need a relatively big number of tests to be trained and the inferred knowledge would depend on the quantity of different activities regarding to the total number of training samples. In contrast, our idea is that different styles of activity realization should be surveyed and quantity of activities realizations in a training set should not determine (at least directly) the chance of accomplishment of an activity by the resident.

In this research, the activities are observed through sensors. By use of the proposed fuzzy temporal data mining techniques, the primary data is analyzed and realization patterns of activities (induced from trace of objects) are retrieved. The mentioned pattern describes complete details of scenarios. By comparing all of the learned realization patterns, prediction pattern is retrieved. Prediction pattern explains the possible intentions of the resident when he accomplishes a few actions to achieve a goal or realize an activity. Our contribution in this paper is to propose a theoretical framework to model the activities and predict the intention of a smart home resident when he performs a few actions. Furthermore we propose to both conceptual and experimental modeling of activities. An extension of fuzzy logic is applied to model the activities and we conducted an implementation to show the potentials of validation of this research.

This paper is organized as the following: After introduction of our research at the first section, we propose a fuzzy logic based approach to model the activities in section two. In the consequence in section three, the “fuzzy event” concept, which is applied in the modeling process, is explained and then in section four, we represent the implementation and two case studies. At the fifth section, the conclusion is performed, then the future researches are introduced and finally the list of applied references is available at the end of this document.

2 Modeling the realization and prediction patterns of activities

Here, we introduce a model to learn the activities (scenarios) realization patterns (ARP) and the secondary goal would be learning of the APP. The first model provides complete and detailed patterns that activities (Scenarios) would be realized according to them; however, the second model predicts the goal by observation of accomplishment of a few actions or operations. In this way, we would be able to make hypotheses about the goals, goal achievement defects and anomalies.

2.1 Comparison as basic knowledge generation operator

In the proposed mode, comparison is applied as a basic and primary technique of the knowledge discovery. By comparison, we would be able to generate hierarchies of knowledge. In the case study, we compare each sensor’s generated data (in different states) to each other by the use of fuzzy logic (Zadeh 1968) and subtractive clustering (Chiu 1997). Therefore, different states of each sensor are defined and the knowledge would be inferable by trace of changes in sensors generated values. Different levels of knowledge are generated by comparison technique. The mentioned patterns are regarded as applicable knowledge for activity (Scenario) and intention (goal) recognition.

2.2 Definitions

Here we introduce some definitions that are applied in our learning model. The “world” of the proposed learning problem is observed through set of applied sensors “S”. “S_i” represents sensor “i” from the set of applied sensors.

Definition 1

Observation: an observation is considered as value generated from sensors. At each time we refer to sensors, they generate a special value. Observations represent raw and primary data that are inputted to the learner model. Observation through applied sensors is mentioned as $O_{S,G}$; in which “S” is the set of applied sensors and G is the concerning goal. $O_{S,G}$ represents set of observations through sensors set “S” and concerning to achievement of goal G (an activity in activity recognition). Observation of sensor “i” in time “t” is referred to as $O_{i,t}$. In fact, $O_{i,t} \subseteq O_{S,G}$.

Definition 2

Significant difference or change in observation of sensor “i”: it is referred to as “$e_{i}$” and indicates a noticeable difference within consecutive observations concerning to sensor “i”. In fact, $E_i = \left\{ {<x_{i,t}, x_{i,t + 1}>| <x_{i,t}, x_{i,t + 1}> \in O_{i}^{2}, \quad |x_{i,t} - x_{i,t + 1} |> \varepsilon ,\quad \varepsilon> 0} \right\}$ and $E_{i} \subseteq O_{i}$. The interpretation of the term “noticeable difference” ($\varepsilon$) depends to the problem circumstances and the process of its detection can be designed through use of experience of expert. Whenever the mentioned changed is observed, an “event” is inferred. In the proposed case study, $\varepsilon$ is dependable to the cluster radius factor.

Definition 3

Activities (scenarios) realization patterns (ARP): we define activities realization patterns as set of couples constituting from events and their occurrence order. ARP definition can be demonstrated as $A_{j} = \{ < e_{i} ,t > |e_{i} \in E_{i} \;and\;t \in {\mathbb{N}}\}$ and t concerns to the order of occurrence of event “i”. The occurrence orders of events are inferred according to the real time that they occur.

Proposition 1

Activities (scenarios) Prediction Patterns (APP): It expresses the most important parameters and values to infer the activities. In fact, its consisting elements are the fuzzy events that are ordered based on their information entropy ³ of being accomplished in all ARPs. Each ARP can be hypothesis that the inferred fuzzy events may be a part of it. Therefore, APP is the collection of all ARPs (hypotheses) that are ordered based on information entropy of their concerning fuzzy events. ARP can be created during a classification process and can be represented by a decision tree (Gray 2011 ).

The prediction patterns, inferred from the decision patterns represent the most important signs and information to recognize the intention of activity realization. The fuzzy event that has the biggest entropy would be placed in the highest node of the decision tree and the lower nodes of the tree have lower importance to recognize the intended activity.

By accomplishment of simple actions (operations), different activities (scenarios) would be realized. In fact, actions are subsets of activities and one action concerning to a special activity can be found in realization of other activities too.

Proposition 2

Conceptual objects: any subset of each ARP can be considered as a conceptual object (CO) that possibly would happen in other different scenarios. For example, in activity recognition field of research, by accomplishment of simple actions (operations), different activities (scenarios) would be realized. By this example, it can be said that, actions can be recognized as subsets of activities and they can be found in realization of other different activities.

3 Fuzzy event

Realization of activities is recognizable through recognition of activities statuses. These statuses do not have clear, definite and certain specifications. Quantitative and traditional data-driven machine learning approaches like HMM (Singla et al. 2008) and Bayesian networks (Biswas 2011) try to find absolute and certain statuses of activities; however, activities do not follow definitive and certain realization ways. For example, in (Biswas 2011) to recognize the activity of “drinking water”, the glass should be taken from a definite point (called “initial state”). If “initial state” of activities is not recognized then the activity is not recognizable. In (Nazerfard et al. 2010) in some cases only 13 % of confidence from the inferred knowledge is expected which is clearly not reliable.

We suggest mentioning the “fuzzy event” as the status of activities. Fuzzy events are the statuses of activities that are defined depending to the interrelations of variables that exist within the variables. Fuzzy event is a conceptual object that makes the basis of the proposed model. Fuzzy event is inferred by comparison of each sensor’s generated value to its last values. At the case of noticeable change, observation in sensor’s generated value, the fuzzy event is reported. The time (delay between two fuzzy events generation) can be also fuzzified. Fuzzy event entity depends on the cluster definition. In another viewpoint, fuzzy event can be defined as a switch from a fuzzy cluster (class) to another one. For example, if an object is moved from table to cabinet, then its distance from table changes (switches) from “near” (fuzzy class) to “far” (fuzzy class).

3.1 Fuzzy clustering

A cluster is a set of similar objects, where similarity is defined by some distance measure. Clustering is a critical task for temporal data mining because the similarity between objects change temporally and there should be defined temporal and dynamic criteria for distance measuring. Clustering via partitioning, hierarchical clustering, density-based clustering, and fuzzy c-means are the traditional methods that find the clusters considering predefined and static criteria.

Clusters of data are formed when data generated from each sensor is compared to its history of observations. For example, from a viewpoint, the distance concerning to a special point⁴ in space can be considered as “far” and “closed”. Another way to form the clusters is to compare data generated from a sensor to other sensor’s data. Temporal information is formed when data generated from each sensor be compared to daily time or absolute time. In this way, we can consider the clock as a timer sensor.

3.2 Subtractive clustering

Some authors work with fuzzy clustering methods in the product space of the input–output space in order to detect the interaction between the input and output variables. Others have extended the use of fuzzy clustering to detect multidimensional fuzzy sets in the product space of the input variables to identify the premise of the fuzzy rules and then assigning a linear consequent to each rule. The identification of fuzzy models can be improved using these multi-dimensional reference fuzzy sets (Priyono and Ridwan 2005). Hence, fuzzy clusters give rise to “local” regression models. The model is then structured into a set of IF–THEN statements.

The TSK model is composed of IF–THEN rules of the following form:

$$ \begin{gathered} R_{(r)} :if\quad x_{1} \;is\;A_{r}^{1} \quad and\quad x_{2} \;is\;A_{r}^{2} \quad and, \ldots ,and\quad x_{m} \;is\;A_{r}^{m} \quad then\;y_{r} \;is\;f_{r} (x) \hfill \\ where:f_{r} (x) = a_{r}^{0} + a_{r}^{1} x_{1} + , \ldots , + a_{r}^{m} x_{m} \hfill \\ \end{gathered} $$

The idea of fuzzy clustering is to divide the data space into fuzzy clusters, each representing one specific part of the system behavior. After projecting the clusters onto the input space, the antecedent parts of the fuzzy rules can be found. The consequent parts of the rules can then be simple functions. In this way, one cluster corresponds to one rule of the TSK model.

Using a fuzzy clustering algorithm, membership functions can be determined according to two possible methods. In the first method, the clusters are projected orthogonally onto the axes of the antecedent variables, and the membership functions are fitted to these projections. The second method uses multi-dimensional antecedent membership functions, i.e. the fuzzy clusters are projected onto the input space (Priyono and Ridwan 2005).

3.3 Fuzzy event inference

A fuzzy event is inferred if a sensors generated value concerning to a fuzzy cluster changes to a value concerning to a different fuzzy cluster. The sensitivity to fuzzy event depends directly to the cluster radius factor. The more cluster radius value causes the less sensitivity to the fuzzy event detection (Fig. 4) and in contrast, the fewer clusters radius value causes the more sensitivity to the fuzzy event detection (Fig. 2). Figure 3 illustrates immediate state is formed if influence range or cluster radius rate is selected as 0.5. Therefore, by determination of fuzzy event sensitivity, different inferences about fuzzy event are considered about an observation.

Fig. 4

By selecting the highest cluster radius rate we would be able to choose a cluster center for all the data

3.4 Temporal fuzzy event inference

Many factors affect on realization of activities and scenarios. Neglecting the inclusion of less important factors (sensors) makes uncertainties about realization of scenarios. The mentioned uncertainties are about reasoning in correction realizations of scenarios and world state normality. Imprecision of sensors in measurement of world features causes uncertainty in activity recognition. One other important source of uncertainty in recognition of scenarios is the lack of knowledge and the fact that we do not know everything about the possible ways of scenarios realization and the intentions (goals). Another reason of uncertainty in activity recognition is ignorance of some world features (data or variables) to avoid from process complexity.

All the mentioned uncertainties appear as temporal uncertainties in realization of scenarios. The more we know the less temporal uncertainty would be caused and the less we know, more temporal uncertainties would be caused. The more we know, the more influencing factors are taken into account and the more possible scenarios are inferred. The reason is that more different states from real world states (contexts and situations) is considered, so more precise information about elapsing time (delay) for each action and operation accomplishment is considered.

In brief, the author’s belief is that the temporal constraints can be expressed more precisely if more awareness and knowledge about the surveyed scenario is provided. In contrast, temporal entropy is increased if less knowledge is considered (Gray 2011).

Similar to regular fuzzy event inference, possible delays between two events are clustered and temporal fuzzy event concerning to possible delays between events is inferred. $\varepsilon$ depends on the cluster radius and temporal event sensitivity.

$$ E_{{t_{i} }} = \left\{ { < t_{{e_{i + 1} }} - t_{{e_{i} }} > | < t_{{e_{i + 1} }} - t_{{e_{i} }} > \in e_{i} ,\quad |t_{{e_{i + 1} }} - t_{{e_{i} }} | > \varepsilon ,\quad \varepsilon > 0} \right\} $$

4 Case study

We have organized two case studies for this section. At the first case-study we survey a single activity (“making coffee”) and concentrate on pattern recognition from it. More than 500 features of this activity are observed in LIARA. For the second section, three ADLs are studied and we discuss how we made ARPs and APP from the data. For the second case, we applied limited number of sensors.

4.1 Case study1: pattern recognition from the activity of “making coffee”

In this section we would verify and discuss the activity of “coffee making” as a case of pattern recognition. Two correct and wrong realization of this activity are compared by a fuzzy inference system (FIS).⁵ For this case we do not apply fuzzy events and we try to discover the positions that objects are mostly located in them, while this activity is realized. The objective of this case study is to show the potential of the fuzzy logic to model and evaluate the activities. The approach of this case study cannot evaluate the runtime observations and needs complete realization of activities to be evaluated. Application of fuzzy event improves this weakness.

4.1.1 Observation

In LIARA, 560 features of the activity of “making coffee” are observed for 253 s and per each second, one observation from the world was done.

4.1.2 Inference of fuzzy classes

At the influence range of 0.5, we did the fuzzy clustering to the observations and we reduced the data to a 246 in 560 matrix, which holds the cluster centers of the observations. A fuzzy inference system is made based on the available knowledge from the activity. FIS is also trained with a wrong realization of the activity, in which the sugar is forgotten to be applied. This realization is taken 170 s. According to the available knowledge, by training the system with one normal and one wrong realization, 22 variables were selected as effective variables to recognize the normal realization of the activity. The concerning fuzzy rule is illustrated in Fig. 5.

Fig. 5

22 fuzzy classes to explain a normal realization of “making coffee”

To evaluate the reasoning system, we decreased the number of world features from 560 to 300 and the number of inferred fuzzy classes reduced to 19 for this state and again by decreasing to 125 variables, the number of inferred fuzzy classes reduced to six. See Fig. 6.

Fig. 6

FIS at influence range of 0.5 with different number of training variables

The FIS is tested by evaluating two similar normal realizations of the “coffee making” activity. The evaluated similarity degrees of the correct realizations and the learned patterns are illustrated in Fig. 6. By decreasing the quantity of the training variables, the similarity degree was reduced. The evaluation process was done by calling the “evalfis” function in MATLAB.⁶

According to the experimental results, it can be inferred that the quantity of the calculated fuzzy classes are dependent to the number of training variables. When the more fuzzy classes are calculated, then the more states for the world state is considered and so more precision in reasoning to separate the normal and abnormal world is expected.

4.1.3 Discussion on the first case study

When an object is positioned in a special location for a longer time rather than other locations, then the data of the object position would concentrate around it. This position is calculated by the fuzzy subtractive clustering algorithm and it is a meaningful signal to recognize the activity. We expect that the object be positioned again in the same area(s) at the future repetitions. For example, the “sugar” is mostly positioned next to the glass, and also the table. These positions are significant positions to recognize the activity of “coffee making”.

In the mentioned case study, we could model the activity of “coffee making” without use of fuzzy event concept and instead, we used the possible positions of objects to infer the activities. It is illustrated that, as the number of observing variables (sensors) increases in the learning and reasoning process, the more fuzzy classes are defined and so more criteria to judge about correct realization of activities are formed. The training process for each of three steps was done in less than a second by a system with “windows 7 operating system”, Quad core 2.66 GHz processor and 4 GB of RAM.

This case study, reveals that the proposed approach welcomes more sensors in training and its process complexity is not increased significantly when the number of inputs are increased. Furthermore, we can observe that this system is not completely dependent to the sensors positions and sensors elimination or addition will not cause the invalidity of the existing knowledge.

The problem with this activity modeling is that we can evaluate the activities when they are completely done and we do not have the possibility to reason in the current situation (at the runtime) and the order of performed actions. To consider the world dynamicity, actions’ occurrence orders and being able to reason at the run time we should regard the world through the chain of fuzzy events.

4.2 Case syudy2: inference of ARP and APP

For this case study we organized several tests to recognize ARPs and APP concerning to three ADLs (“drinking water”, “making tea” and “making coffee”). To do that, applied objects like sugar, coffee, coffee maker, tea maker, spoon, glass, etc. in these activities are tagged by RFID tags and traced by RFID antennas. Each activity is done twelve times in different manners. Sensors observe actions concerning to each activity and by trace of objects and resident, we are able to express how the objects change their positions in realization of activities (also the resident can be traced as an object that is concerning to realization of activities). The concerning ARPs and APP are made. We have organized this test by limited number of sensors.

4.2.1 Observation

Observation is the first step to learn the patterns of activities realizations. Embedded sensors in environment do observation. The observation is done rather permanently but treated (or registered) frequently. For example, in LIARA the computers that are connected to the sensors and register their generated values do each one second an observation.

4.2.2 Inference of activities realization patterns (ARP)

ARPs are inferred by trace of objects displacements. Objects displacements are inferred if objects get close (or get far) relative to special points in the environment.

In this way, activities are defined by displacement of the objects in space. Position of objects to special geographical points can be described by partial membership of their distances to specific fuzzy classes. For example, “near”, “intermediate distance” and “far”; however, in the implementation two simple concepts; “far” and “near” are applied. The movement of the objects regarding to each geographical point can be done into two ways; “getting closer” or “getting farther”.

All the mentioned definitions are fuzzy terms and can be defined through fuzzy functions (Zadeh 1978). Therefore, at the implementation, instead of directly applying the “integers representing the distance of objects”, we applied fuzzy membership measures of the distances to the fuzzy classes. It should be mentioned that we do not care the position of object in x–y page, or their precise distance, but their movement and displacement regarding to some specific positions (in fact antennas’ positions) are noticeable.

Implementation of the fuzzy approach lies on the concept of “fuzzy event” (already discussed in previous sections). In this approach, events are inferred from each meaningful change in position of objects regarding to some already known geographical positions. Observance of special events could mean recognition of a special activity. To infer the occurred event we can apply the following membership function to be prepared for the classifier:

$$ \mu (far) = \left\{{\begin{array}{ll} 1&{if\;d>b}\\{\frac{d - a}{b - a}}&{if\;a< d<b} \\ 0 & {if\;d<a} \\ \end{array} } \right\},\quad \mu (close) = \left\{ {\begin{array}{ll} 1&{if\;d<a} \\ {\frac{b - a}{d - a}} & {if\;a<d<b} \\ 0 & {if\;d>b} \\ \end{array} } \right\} $$

$$ \begin{aligned} \mu (get\_far) & = \mu^{1}_{Object} (far) - \mu^{2}_{Object} (far) \\ \mu (get\_near) & = \mu^{1}_{Object} (near) - \mu^{2}_{Object} (near) \\ \end{aligned} $$

The mentioned approach to define fuzzy event is valid only if the expert has idea about the possible positions of objects regarding to the antennas, which are “far” or “near” positions in the mentioned definition.

In the case, that expert does not have any idea about possible states of data records or at least he intends not to involve his idea to the pattern recognition process, then we can describe the position of objects to special geographical points (antennas locations) as partial membership of their distances to specific fuzzy classes. For example, “near”, “intermediate distance” and “far”. The movement of the objects regarding to each geographical point can be done into two ways; “getting closer” or “getting farther”. Here, a tuple combined from couples of “events” and their “occurence order” indicates the Realization pattern for an activity named activity 1. In this example, O_i represents an object that is applicable in realization of activity 1, A_i represents one of the geographical points that are fixed, and displacements of objects are defined relative to them.

In this model, non-fuzzy events like “turn oven on or off” can be merged to complement and provide supplementary information.

$$ \begin{gathered} Realization\;Pattern\;for\;activity\_1 = \{ <get\_close(O1\_A1),1>,\hfill \\ <get\_far(O1\_A2),2>, <get\_close(O2\_A1),3>, \hfill \\ <get\_close(O3\_A1),4> , <get\_close(O3\_A2),5>, \hfill \\ <get\_close(O4\_A1),6> , <get\_far(O4\_A2),7> \}. \hfill \\ \end{gathered} $$

In the proposed model to learn spatiotemporal patterns of activities, at first values that are observed from the world are compared to each other and then the displacements of objects are inferred as events. To derive the order of events, we assign a number to the events, according to their occurrence order. ARP is in fact the ordered inferred events concerning to the activities.

4.2.3 Inference of activities prediction patterns (APP)

Creation of prediction pattern resulted from classification of activities is the final goal of our research. APPs can be calculated by comparing all the ARPs together by use of a classification process. For example, by application of C4.5 algorithm (Quinlan and Ghosh 2006), decision tree that predicts the intended activity can be drawn (Fig. 7).

Fig. 7

Decision tree illustrating an APP about three activities. Fuzzy logic is applied to infer the objects displacements. For example, if object 1 gets farther from antenna 1, then the only hypothesis that explains the observations is the intention to realize the activity 1

By use of APPs, we can predict the intention of the resident. In this way, when resident starts realization of activities by accomplishment of a few actions, we would be able to recognize the intention of resident about his final goal.

In Fig. 6, each node of the decision tree indicates the relation between the object and the RFID antenna and each branch indicates the occurrence of a fuzzy event on its upper node. The rectangular indicate the possible intended objectives. For example, in this case study if the Object 1 (cup) gets nearer (operation performed by the Smart home resident) to the antenna 1 (tea maker) and then if Object 2 (sugar) gets nearer to the antenna 1 then the system infers that the resident would intend to realize the activity of “making tea”. The reason is that no activity is available in the knowledgebase or no activity would be recognized as normal, unless the resident intends to complete the activity of “drinking water” and so he should follow one of the ARPs concerning to the activity of “drinking water”.

5 Conclusion

In this paper, we introduced the smart home as an ambient environment that as like as a big data warehouse provides a lot of records of data about realization of activities. We applied the proposed fuzzy temporal data mining approach to retrieve the fuzzy models of activities. The models consider existing uncertainty of activities on behind of the applied fuzzy clustering technique. The proposed approach welcomes application of more sensors as they help it to increase its reasoning precision. We modeled activities realization patterns, which provides us complete explanation about actions and their accomplishment order. Then the information is classified and prediction patterns of activities are retrieved. Applying activities prediction patterns we are able to guess and predict the intention of the resident when he accomplished a few actions. Therefore, comparing the mentioned information, we may be able to infer the assistance needed by the resident.

6 Future researches

In this research it was proposed how to model an activity under uncertainty using fuzzy events. In our future research, we would present a research on assistance provision and judgment about world actuation.