Keywords

1 Introduction

According to the Oxford dictionary habit is defined as “a thing that you do often and almost without thinking, especially something that is hard to stop doing”. Now by the quick interpretation of this concept, we can infer that there are patterns that we are running throughout our lives. Recognizing these standards and acting accordingly can give people a better quality of life.

In this sense, concepts such as smart devices, intelligent environments, are crucial in building such systems. These bring systems that are very useful as they are able to feel the environment around them.

Well-being and comfort have had a major impact on society. In the case of well-being, higher levels mean less risk of disease and more longevity [2]. Consequently, comfort is also a vogue concern. Universities, hospitals, institutions spend most of their time promoting increased comfort and consequently reducing discomfort. Several studies address that employees with higher comfort have a better performance on work [9]. However, it is quite difficult to understand what these terms are and how they may be affected. In fact, one of the main questions that come up when we talk about monitoring comfort and well-being is: What to measure? The environment condition alone is correlated to well-being and comfort status [17]. The heterogeneity of the environment and of the population makes difficult to perceive what may be affecting us in a given place. However, these days it is increasingly easy to explore these terms. Effectively more and more people have access to smart devices. These devices make the use of sensors easy and inexpensive since, for example, when we buy a smartphone it comes with a huge variety of sensors. This will allow some globalization, making possible to gather more heterogeneous information and give the possibility to answer this and other questions. In short, this article aims to demonstrate the main results obtained in the search for a solution that allows foreseeing comfort and well-being.

2 State of Art

As one of the focus of this project is comfort and well-being it is important to define these concepts in order to clarify their meaning. In this sense, on the next sub-chapters, is presented a little about these and other concepts crucial to the understanding of the present work.

2.1 Well-Being and Comfort

On the one hand, the term welfare is quite difficult to define, and its refinement has been the subject of many studies over the years. It is a term that is often used by the community, but not always with a correct connotation. In fact, because of the associated confusion, we do not always know the meaning of being well. Several authors seek to define this concept. Elena Alatarsteva and Galina Barysheva [1] subdivide this term into two strands: one objective and one subjective. In the case of the objective strand, well-being could be characterizing by wage levels, residence conditions, educational opportunities, social quality, the environment, security and civil rights. In the case of the subjective strand, well-being is conceptualized only as an internal state of an individual. However, this is not the only vision about the definition of this concept. Through the consulting of various articles published by authors from different branches it is possible to arrive at the definition of this concept more specifically, that is, by categorizing it into different classes. Some of this classes may be Community Well-being [14], Economic Well-being, Emotional Well-being [4], Physical Well-being [14], Social Well-being [3] and Work Well-being [15]. There are several ways to categorize well-being, however, in general, they all end up touching the same points.

In the case of comfort, looking for and maintaining it is common to all people. Many people quickly identify loss of comfort and work diligently to restore it.

In our days, the term comfort is often seen related to the marketing of products like chairs, cars, clothing, hand tools, and a lot of others  [20]. Evidence from the literature suggests that comfort, as a concept, has historical and more recent relevance for nursing [19]. It is, therefore, a concept that needs further exploration to increase knowledge in this area because it is difficult to reach a consensus on its definition. In order to establish the definition of comfort, this article will use the meaning present in the Oxford dictionary. According to this, comfort is a broader concept that can be defined from a physical and a psychological perspective. In the first case, it is seen as a “state of physical ease and freedom from pain or constraint”. In the second it is defined as: “The easing or alleviation of a person’s feelings of grief or distress”. However, despite all these definitions, they all note the existence of factors that influence comfort. Some papers show that different activities during measurements can influence comfort, concluding that characteristics of the environment and the context, can change how people feel [20]. Although comfort is often associated with a synonym for well-being, a clear difference can be demonstrated between both concepts. By the previous definitions, it is noticeable that comfort is most used to classify the atmosphere that surrounds the human being. This implies that comfort is associated with momentary aspects. However, well-being arises characterized by an exhaustive variety of groups, which makes it associated with a long-term context. By way of example, a person may find himself comfortable but unhappy (and vice versa). In this sense, well-being is referred to in many studies as being progressively improved, while comfort is something that can be improved on time. As we can see, the mental health organization in the UK said that “it is important to realize that well-being is a much broader concept than moment-to-moment happiness” [5]. Although comfort and well-being are distinct terms, this does not imply that they exist separately. Indeed, both terms arise intrinsically linked. Comfort at a certain level contributes to well-being.

2.2 Learning Techniques

One of the areas of artificial intelligence is machine learning which is known for the capacity to automatically learn and improve from experience without the need of being explicitly programmed [18]. With the definition of machine learning comes another, deep learning. Deep learning is a subset of machine learning. It is considered the next evolution of machine learning algorithms. The designation deep comes because of the many layers that usually this type of technique has. The main difference between these two areas is that deep learning can automatically discover the features to be used while in machine learning the features have to be provided manually [7].

Another term that arises with deep learning is the term of artificial neural network. These types of algorithms were drawn to simulate the way the human brain analyzes and processes information [6]. There are many types of neural networks that can be used depending on what is necessary. In this work, these mechanisms can be useful to make predictions. When we speak of the term predictions, we speak of the outputs that are achieved from the training that a given algorithm had on a given dataset. In many cases, the term does not refer to future issues. It can be used, for example, to predict if there has been fraud in a given transaction that has already happened. Predictions are very important as they allow highly accurate guessing, thus helping in various aspects. However, for good guessing happen an algorithm has to learn. In fact, machine and deep learning are bounded by the concept of learning. There are different types of learning: supervised learning, unsupervised learning and reinforcement learning. Each of these types has several machines and deep learning algorithms that, depending on problem, can be applied. Although there many types of learning, this article will focus on supervised learning. In this type of learning is given to the machine what each data block means. This is, the model is trained on a labeled dataset [11]. A label dataset is one that has both parameters: input and output parameters. In this case, learning is based on regression and classification techniques. If the output variable is a category, such as “yes” or “no”, it is a classification problem. If the output variable is a real value, such as “weight” it is a regression problem. In this article, will be useful, a category technique because one of the main purposes is to identify well-being and comfort levels which can be represented by classes.

There are many algorithms that can be applied in this work. One of these algorithms that can be really useful explore the concept Recurrent Neural Networks (RNN). This is because RNN “can model sequence of data so that each sample can be assumed to be dependent on previous ones” [12]. One of the most popular versions of RNN is Long Short-Term Memory (LSTM) Network which is used to process and predict time series given time lags of unknown duration [12]. In fact, comfort and well-being can be dependent on what happened in a previous time.

2.3 Related Work

Generally speaking, there are several solutions we use every day that are related to the concept of data collection and intelligent suggestions. Take for example the case of virtual assistants such as Siri, Cortana, and others. This type of system collects as much information as possible about the user and the environment around them, and from there suggests and performs actions that simplify people’s lives. The use of smart devices for data collection is not new, and there are a lot of applications in this field. One of the main applications is, for example, context recognition [13]. In the case of comfort and well-being, there are advances with the use of smart devices sensors. One example of an application that has this purpose is BeWell [10]. It is an application designed to make people care about certain concepts of their well-being, concepts that are often overlooked. This is the case of, for example, lack of sleep, hygiene or high social isolation. The authors believe that this situation is caused by an absence of adequate tools for effective self-management of overall well-being and health. The main idea of this application is to be capable of monitoring multiple dimensions of human behavior, encompassing physical, mental and social dimensions of well-being. Comparing to what is presented in this article, this application will be useful as a support for this work since the main objective is similar to what is presented. However, this application only makes use of some device sensors whereby only a few pre-established well-being metrics are evaluated.

In another way, in the field of comfort, there are equally some studies and consequently some applications. One example of that is the ThermOOstact [16]. This is an application that has with main purpose study the thermal comfort inside a University Campus. Nevertheless, it is really specific since only is used concepts associated with temperature and only works on the University of California occupants. The purpose of this article is more comprehensive because a lot of different data will be collected and according to that data will be possible to explore a wide range of concepts. This brings a great advantage because it allows the creation of a broader application. Furthermore, the proposal application will allow to explore comfort and well-being at the same time.

3 Architecture

According to the system architecture, it is possible, in addition to fundamental components such as the database, to divide the application into two strands. A strand oriented only to data collection so that the data can be stored and processed later and another strand more related to the building of predictive models. In Fig. 1 could be seen this process.

Fig. 1.
figure 1

Application workflow

Full size image

3.1 Data Collector Application Strand

Since for built machine learning algorithms data is needed, an application that collects data has to be built. With an application developed for smart devices, it is possible to use inherent sensors of this type of system. Some information that could be extracted is the number of steps, ambient temperature, light level, magnetometer values, gyroscope values, accelerometer values, humidity, and location. However other information can be collected, like the environment type (indoor or outdoor), Wi-Fi information (SSID, RSSID, and Mac address of the provider), battery percentage, call number and duration, during a day, the number of events for the day and activity performed by the user by the time that the data is collected. Some of this information is sensible and for more security, it has to be saved through hash key, furthermore, no information identifying the user could be saved. In addition, it is intended to use APIs so that the information gathered was even wider. This includes, for example, the OpenWeather API that allows collecting a vast amount of data associated with weather conditions (such as weather state, wind speed, ultraviolet radiation index, pressure, and temperature) and the Foursquare API that allows capturing location-related data (such as local category). In addition to these, an API to capture data about air quality (such as AQI and the dominant polluter) can also be used.

One decision to make is how users will be asked about their comfort and well-being. In this sense, to avoid being too intrusive, two situations were taken into account. On the one hand, the questions about comfort could be made by context analysis. That is, when it is detected that the user has moved, questions will be asked more frequently. Another situation is that these questions can appear at specific times. In this case, it could be given to the user the possibility to choose how often the questions occur. Three options could be taken into account: low, medium, or high frequency. In the case of the well-being question, it is intended to appear only less frequently. This is due to what was defined in the previous section. Indeed, it is assumed that well-being throughout the day will not have as many variations as comfort. In these cases, the notification could show up every two hours. On the other hand, the user could also be allowed to answer a question about comfort or well-being voluntarily without waiting for a notification. To make both processes faster a response scheme through a smiley face could be used (Fig. 2). This idea came about over an adaptation of the Likert scale, commonly used in opinion polls. As the following figure demonstrate, the questionnaire for well-being is more complex to be possible to associate the different type of data collected to different types of well-beings (defined in state of art). These questionnaires were designed with the aim of being short answer so that users can answer quickly and without effort.

Fig. 2.
figure 2

Application questions

Full size image

3.2 Learning Workflow Strand

After an initial analysis of the data, the basic procedures of machine learning and, more particularly deep learning, can be used to build predictive models. Such procedures result from the application of frameworks that arise from different studies. One example of an approach was the one described by Yufeng Guo [8]. This involves, after the data collection, a pre-processing phase. This is a complex phase, as it involves a set of steps (missing values treatment, data-type conversions, data normalization, among others) where various strategies could be explored. Then the objective is to choose a model and train it, trying to improve it in relation to a base value. Finally, it is necessary to evaluate the model by making optimizations regarding its hyper-parameters. The studies applied below could be applied to comfort and well-being.

4 Case Study

The case study that will be presented next involves an application created for Android, with the assumptions stated above, with the participation of ten users who used the application during the period from 15 March 2020 to 20 April 2020. Users are located in Portugal and due to this factor, the data collection period coincided with a mandatory quarantine period due to the global pandemic. In this sense, there was a change in the final objective of the work so that it fit the conditions that were felt. In short, the collected dataset contains 1954 lines and a total of 43 features. In the following steps, we summarize some of the steps taken. This work of building a model was done in the Python programming language, using some libraries such as TensorFlow and sklearn. Some of the data processing done was:

  • Elimination of some irrelevant variables: Variables like the user’s key, which is not relevant to the prediction, were deleted;

  • Handling of missing values: Since there were a lot of missing values, we started by using some methods for their treatment. What seemed more adequate was the replacement of these values by the immediately previous value. This was due to the fact that once the data is captured sequentially, this technique could result in less introduction of variability in the dataset. In addition, in the case of some features, such as the case of meteorology, for example, it is expected that they will not have major variations over the course of a day. However, other techniques were also explored here, for example, the use of the mean and the median. Since the number of lines in the dataset is not very high, it was decided not to eliminate lines with missing values in order not to further reduce the sample number;

  • Treatment of the encoding of non-numeric variables: Several features of the dataset were represented by strings. Since deep learning models only accept numbers, it was necessary to proceed with some type of pre-processing. For this purpose, the One Hot Encoder method was used;

  • Data Normalization: In order to reduce the discrepancy, some techniques for framing the values were also applied, among them the Standardization and Normalization methods were tested;

  • Target Encoding: Since the target presents values in a certain way sorted from 1 to 5, a label encoding technique was used to normalize these values (thus transforming these values into classes 0 to 4).

We want to study if the previous data are relevant to the prediction of physical well-being. For that case were used RNN, more in particular LSTM. As this is a neural network where the order of the data is quite relevant, no shuffle has been done. Although, other special precautions regarding the way the data had to be processed were taken. In addition to pre-processing, the data were transformed according to a sliding window principle. Thus, the previous data are organized so that they serve as input to predict the target of the next line. Since we want to classify the well-being some precautions have to be taken. As this is a multiclass classification problem, the loss function used was therefore categorical_crossentropy. Furthermore, in the final layer, a softmax activation function was used. Here we have to take into account the type of this activation function and the loss function, as incorrect use of these can lead to false results. Some studies were carried out in order to understand what is the method that could give best results. Essentially two approaches were built:

  1. 1.

    LSTM using data from a single user

    1. a.

      Only one user was used for training the rest were used for testing.

  2. 2.

    LSTM without separating by user

    1. b.

      There was no separation of each user (only two users are saved for the test);

    2. c.

      The transformation into a sliding window occurred in all data, so the data was sorted by user key and date-time in order to reduce inconsistencies.

The following are some of the results obtained in both approaches. It is although important to refer that the following experiments were made on physical well-being. However, the study could be extended to other well-being types.

First, a study was done on what should be the best treatment for missing values. The results could be seen summarized on Table 1.

Table 1. Nan treatment experiments
Full size table

Comparing the two approaches is possible to see that the first one, gives in a first instance better result than the second one. This could be for the fact that the first one, is simpler and the data is less. However, test accuracy does not have big changes in the two approaches. In these experiments seem like using the mean values to solve the Nan problem is the most appropriate in the first approach (less loss) and the median is better in the second approach.

With the use of values normalization techniques, it was possible to obtain a big improvement in the accuracy of the second approach, as shown in Table 2. However, in the first one, no improvement has been achieved. Despite all, generally, a value normalization is better. The MinMaxScaler technique was the chosen one.

Table 2. Value normalization experiments
Full size table

Since the number of data is low, the next step was to test resampling mechanisms. The method varies depending on the approach. In the first one, the data was resampling in an interval of 15 min because it was the minimum time that notification is thrown. In the case of the second approach, the resampling was done taking into account some considerations. First, the data were grouped by user, after that the resampling was applied for each user individually. This is, for every user was generated samples that correspond to the interval of 15 min. However, in this case, it was not possible to improve the accuracy. The results could be consulted in Table 3.

Table 3. Resample experiments
Full size table

The final step was to validate and tuning the model. In these approaches the objective was to experiment some combinations in order to find a good fit. The number of layers, the number of neurons, the windows size, epochs, batch size, among other, were tested together. It is, therefore, important to refer that a good fit is not that one that has high accuracy. In fact, after the application of some of these techniques, it was possible to improve the results and consequently combat problems such as underfitting and overfitting which can lead to a leak of performance on the final model despite the accuracy value.

In the first approach, the model seems to converge after a few Epochs. The learning curves of the best model could be seen in Fig. 3. The windows size used was 8 and the batch size was 6.

Fig. 3.
figure 3

Approach 1: learning curves

Full size image

Despite these results, this approach has some problems. The main one is in choosing a user who is the most possible representative of the problem. For example, if user A is used for training and user B is used for testing the model will fail if the classes in B are not the same as in A (user A has chosen 1 and 2 for physical well-being, and user B has chosen 2 and 3, as the model was not trained in class 3 this could be a problem). Furthermore, each user individually has few rows of data. This makes the model be more fallible in a real scenario since it will depend on that one user used for the training. The next image (Fig. 4) represents two confusion matrixes from two different users. The one on the right (Fig. 4b) has a higher accuracy than the one on the left (Fig. 4a). This could be explained for what was said before.

Fig. 4.
figure 4

Approach 1: confusion matrixes

Full size image

The validation of the model and consequently the final accuracy was obtained by using all the remaining users from the dataset. The mean accuracy obtained was 84.22%.

In a search for finding a good model the same steps following before were applied in the second approach. The final results could be seen on Fig. 5.

Fig. 5.
figure 5

Approach 2: final model

Full size image

The windows size used was 12. The best batch size found was 14. Although this approach would have the same problem as the first one, since the number of data is bigger the probability of that problem happen is lower.

Despite all this, this approach has also some other problems. First, by joining the data of all users in the same file, we may be creating some inconsistencies mainly about the creation of sliding windows. Second, the classes are unbalanced, this combined with the impossibility to shuffle the dataset could lead to a leak of performance. However, the second approach seems to be one with better results.

5 Conclusion

In this article, initially, it was described how smartphones and the sensors they have could be used to identify the context in which a given user is inserted. The experiments carried out have shown good results. In fact, according to the results obtained, it is possible to answer in an affirmative way the question “Is it possible to predict well-being in times of a pandemic?”. Thus, it was allowed to build a model capable of mainly predicting well-being at the moment in which we live. Indeed, it has been shown that physical well-being is dependent on previous conditions. Despite in this article only some of the results regarding physical well-being have been shown, the process is more comprehensive and allows the extension of these experiences to other types of well-being. In the case of comfort, it is intended to apply the same studies.

In the future, and after having overcome the contingencies of the times in which we live, we hope to proceed to a new collection of data and thus not only be able to predict well-being and comfort during quarantine but also for different environments and different spaces. Furthermore, it is intended to create a more comprehensible study and in that way fight against the problems mentioned. Such a study proves to be quite useful both from a personal point of view, as well as from a more professional point of view. From a personal point of view, in a future work, notifications could be launched that provide advice to users so that they can consciously act on their well-being and comfort. In the professional case, on the other hand, the development of an application that can cope with changes in the conditions of comfort and well-being in a given space can be useful. So, the next steps are to improve this study and apply it to help the needs of different populations.