Keywords

1 Introduction

For both scientists and the public, sleep is hot topic nowadays. Medical literature is pointing towards disturbed sleep as underlying cause for health related issues that challenge modern society. In hospitals, there are long waiting lists for people who want or need their sleep to be studied in a sleep center, leaving a growing group of people undiagnosed. As these specialized centers use traditional and expensive techniques, hospitals face some challenges to make use of the available capacity in the most efficient way. In addition to the increased medical attention to sleep related problems, with initiatives like the Sleep Awareness Week or the international World Sleep Day, the debate on disturbed sleep is brought to the attention of the broad public as well. Topics like sleep, sleep related disorders, and solutions to improve your sleep are also attracting increased attention from newspapers, magazines, and public broadcasting.

Along with this increased attention for sleep, we see a growing market for products that are meant to improve sleep quality: from intelligent alarm clocks for smartphones that wake the sleeper in light sleep to state-of-the-art sleep monitoring devices to be used at home. One type of products that has the purpose to assist people in monitoring or improving their sleep is the smart wristband. These wearables track activity 24/7, counting steps during the day and measuring sleep patterns at night, and make promises to improve the quality of the user’s sleep. For example, the Fitbit Flex product website states: “Flex never sleeps, even when you do. Continue to wear it all night and it’ll measure your sleep quality. Your dashboard will reveal how long you slept and the number of times you woke up to help you learn how to sleep more soundly” [3]. As we find similar statements on the websites and product packaging of the Jawbone Up, Misfit Shine, and Polar Loop, it is clear that these tools make big promises and set high expectations for buyers regarding sleep. In 2015, with the launch of the Apple Watch, the success of smart watches is expected to be at the expense of smart wristbands [4]. And although neither of the two is widely adopted, this new wave of wearables comes with new technology, so we can expect even more extensive features related to sleep in the near future.

Prior evaluations of wearables focus on the fitness and activity tracking features, e.g. [6]. From a technical point of view, alternatives for existing sleep monitoring techniques are suggested, e.g. using smartphones [1]. And from a medical point of view, the effectiveness of popular activity tracking products has been evaluated [7]. The social impact of wearables is also being studied, mainly concerning privacy aspects [8]. But despite Choe’s call for work on sleep in the HCI community [2], to our knowledge, no work on these sleep-tracking tools exists from the user’s perspective.

The present study looks at the user experience when monitoring sleep at home through smart wristbands. We don’t make a verdict on the accuracy, effectiveness, or the sense or nonsense of these products from a technical or medical perspective. We contribute to the research on sleep-tracking by sharing experiences of users using popular tools and summarizing design recommendations for sleep monitoring features and tools to be used at home from a user’s perspective.

2 The Study

We performed a diary study to sensitize the participants to their own sleep patterns and habits. After the diary study, we asked the participants to use two different smart wristbands to track their sleep, and then asked them to report on their experiences with these products by comparing them. We recruited six participants via a call that was shared on Facebook. The participants were between 21 to 60 years old, and five of the six participants were female. The participants had a moderate to advanced knowledge of technology. They all used (laptop) computers and smartphones daily, and two participants used a tablet regularly. Some of the participants were familiar with the selected tools, but no one had actually used one before. All participants received a gift card of €50 as incentive after the concluding interview.

Inspired by cultural probes [5], we composed a niary – a diary to record the night – for the participants. As with the original probes, the niary wasn’t meant for extensive analysis in the first place. Our niary had the purpose to sensitize the participants about their own sleep patterns. By completing the niary, people were made more conscious about their evening rituals, their sleep, and the night, before they started using sleep-tracking tools. This made the evaluation of the activity trackers more focused on sleep-tracking features of the tools.

The niary (see Fig. 1) consisted of a booklet to record each night for one week, three postcards with a question related to sleep, and a to-do list. In the booklet, participants were asked to write down when they went to sleep and when they woke up, and how they were feeling (as in how tired they were, and their mood). Also, we asked them to use a metaphor for their sleep and for waking up (e.g. a song, movie, or object). Finally, we also asked them to sketch the night (this could be a drawing, a graph, a written text, or a collage). The cards listed the questions: “The most important thing in my bedroom is…”, “Sometimes, I can’t sleep because…”, and “I prefer to get awakened…”. On the to-do list, people could write down their evening ritual.

Fig. 1.
figure 1

An impression of the niary as given to the participants (left), and completed (right).

Full size image

After the diary study, we evaluated the user experience when using a smart wristband. We selected four popular wristbands with sleep-tracking functionalities: the Fitbit Flex, Jawbone Up, Misfit Shine, and Polar Loop (see Fig. 2).

Fig. 2.
figure 2

From left to right: the Fitbit Flex, Jawbone Up, Misfit Shine, and Polar Loop.

Full size image

These four products have the same working principle and offer similar functionality regarding sleep-tracking. Although other, more specific sleep-tracking devices exist, these are more expensive and require the latest smartphones. In order to keep the study feasible, we decided to limit our set of devices to those with a higher compatibility with computers or smartphones.

We visited our participants three times in two weeks. During the first visit we explained the overall course of our study. After signing an informed consent, we handed over the niary, and explained every aspect of it in detail with the participant. Then, the participants completed the niary in one week, after which we had a second conversation to discuss the completed niary. These conversations were not analyzed, but it allowed the participants to formulate their story and to give thought to their sleep and sleep habits once again, in order to become more conscious of it. We then gave the participants two wristbands for a pairwise comparison, and explained how they could use the devices. We chose the wristbands for each participant based on the available technology (e.g. only a few participants had access to an iPhone or iPad required for the Misfit Shine, while all participants had a computer that could be used for the Fitbit Flex). We also distributed the devices to have them equally used among the participants. We limited our instructions to the bare minimum (how to install, how to synchronize, and how to view the data). Afterwards, we also sent the instructions to the participant by e-mail. Then, the participants could use the two trackers for one week, after which we had a third and final encounter. We interviewed the participants and focused on their experiences with the products, and we asked them to compare the two systems. This interview was recorded for further analysis. We transcribed the recorded interviews, and made an affinity diagram from the experiences people reported to identify recurring patterns.

3 Results

The interviews with participants of our study gave interesting insight in the user experience of the sleep-tracking features, which we bundled in five themes: the bracelet, night mode, synchronization, viewing results, and value.

3.1 The Bracelet

The participants reported that wearing the bracelet did not bother them. They also said that they didn’t feel like it had any influence on their sleep. The only negative remark we received was that the bracelets were hurting others, especially partners in bed.

The different products have other ways to let the user interact with the bracelet. The Fitbit Flex and Polar Loop require the user to tap the bracelet, and the Jawbone Up and Polar Loop have physical button. The users did regard the tapping as a fun way to interact with the bracelet, but as the bracelets don’t always react from the first time, this form of interaction can be time consuming and cause frustrations.

“When I had to tap the bracelet to get the lights, it didn’t always respond in the way I wanted it. I knew that I should have two, so I kept tapping until I saw the two lights. Sometimes, this could take a while.” – U02

Confusing situations where the bracelet did not respond as expected are enforced by unclear or missing feedback. Figure 2 shows the different displays: 5 small lights on the Fitbit Flex, a sun and moon icon on the Jawbone Up, 12 lights as a clock on the Misfit Shine, and digits (which can also show text) on the Polar Loop. During the study, two participants switched day and night mode (for counting steps and measuring sleep) on the Fitbit Flex for some days because they misinterpreted the displayed patterns of lights, which resulted in no data and caused frustration. On the Jawbone Up, the feedback was clearer, but people were wondering why day mode is shown as a flower. As one of the participants noted, the bracelets are useless until they are synchronized with the app or website.

3.2 Night Mode

The bracelets have two modes: day mode for counting steps and night mode for measuring sleep. Some devices require a manual action to switch modes. The participants thought it was nice that they could indicate “now I go to sleep”, but they didn’t understand there was no data recorded if they forgot this. And our study showed that people do forget to put the bracelet in night mode. As some devices are able to switch day and night mode automatically, the participants expected this behavior from all products.

“I forgot to put it in night mode a few times. The bracelet could detect it automatically, or measure both in case you forget to change it. […] If you forget it now, all data is lost and you can not retrieve it anymore.” – U03

People found a fully automatic switch not ideal, as they thought that the device was not always right. The Misfit Shine detects sleep automatically and allows people to correct the data, but then all measured data is removed so the user only has the manually entered times left, which is not better than having slightly incorrect data.

3.3 Synchronization

To view the measured nights, people first have to synchronize the data. Some bracelets have small extras to do this: dongles, cables, … Participants didn’t bother using these extras, but were afraid to lose them, as they are quite small. People did have a clear preference to synchronize the data via their smartphone to their computer, especially in the morning after waking up.

“I always have my smartphone with me, as well as my tracker, so synchronizing goes much smoother. With the dongle and the computer it was more cumbersome.” – U04

Some products provide automatic synchronization, but our participants didn’t regard this as a necessary function. As people have the goal of looking at their data, they do take time for it and take their smartphone or computer with them. As opposed to switching day and night mode, people are unlikely to forget synchronizing before viewing the results.

“Manual sync may make it less handy, but it’s not bad either. You know you have to log in on the website so you can not forget to sync, as opposed to put the tracker in night mode manually.” – U06

Also, participants did perform a manual sync every time before looking at the results even with an automatic synchronizing product, just to be sure to have the most recent data, especially in the morning when they were curious about their night.

3.4 Viewing Results

As with synchronization, people preferred viewing results on their smartphone to their computer.

“The phone is really interesting. It was quicker to take a look at my phone more regularly. With the computer, you have to start it every time.” – U05

Regarding data visualization, people preferred linear graphs to circular graphs, as 24 h circles were confused with 12 h clocks. The use of color could also be improved, as some websites used colors with very little difference (e.g. different shades of blue, which makes it hard to distinguish restful and restless sleep). People did expect a more detailed view on their night, and were disappointed by the lack of details.

When viewing the results, people were wondering whether the measurements were correct. People were missing explanation on how the bracelets decide whether the user was awake or sleeping.

“It is not clear how they make up their conclusions. How does the band know when I’m sleeping? Maybe I’m watching television in the sofa. If you look at the results the day after, you immediately see the error. But if I don’t look at the results for over a week, I don’t remember this anymore. I would like to know how they define when I am sleeping, and how they calculate the ‘time to sleep’.” – U06

We also got a lot of questions on how people should interpret the data. People want to know what the data means, and what is normal.

“I would like to have some tips that help me to interpret the data, what is good and what is bad. Maybe some more explanation would be better, what is sound sleep and light sleep.” – U01

People were missing tips and advice to improve their sleep, based on the measurements.

3.5 Value

During our interviews, the participants said the trackers were fun to use, and we did not see any differences between trackers on this aspect. People liked to have an overview of the times they went to bed, the times they woke up, and how long it took before they actually fell asleep. Our participants said it was nice to see the difference between light and deep sleep. However, most participants found the activity trackers very basic. The information was found descriptive and rather limited. People were not convinced of the value of the trackers, because the devices gave too little information and the explanation was too brief. People had the impression that you cannot do much with these devices.

“You can tell that sleep is not the company’s core business. But it is good it is included next to steps and food, as sleep has an impact on your health.” – U04

People told us they learned that a good night doesn’t necessarily mean a long night, and a short night can be very restful.

“I noticed that sleep is very subjective. You can have a very bad night, while the graph is good.” – U05

People also felt that there is more than numbers when talking about sleep. Having an app that tells your night was good, while you feel terrible, is damaging for the credibility of the system. So where these products offer very little guidance to make interpretations and meaning from the data, they do make simplistic statements based on numbers. People do feel that emotions play a big role as well.

4 Discussion

Based on the experiences from the users in our study, we formulate three recommendations for sleep-tracking tools: find a good balance between automation and control, make data intelligible, and don’t ignore the role of emotions.

People want control, they want to be able to tell the system when they’re going to sleep. But, we must take into account that people do forget to push the button to do this, so an automatic fallback is no luxury. This ensures that there is data in the morning. However, if the automatic fallback would be incorrect, the user should be able to fine-tune the data without losing valuable data. On the other hand, synchronization can be manual, as this doesn’t seem to bother users. We should keep in mind that people don’t have a computer nearby while waking up, so it is more convenient to use a smartphone or tablet to synchronize the data.

It’s obvious that graphs should be clear. By using distinct colors, the users can easily spot disturbances and patterns in their measured sleep. Users will want to compare nights, so the interface should facilitate this. Providing an overview of multiple days, or at least by using the same timescale for each graph can do this. It is important to inform people how the data was measured, and on what ground statements were made. Also, we need to teach users how to make meaning from the measured data, so how they should interpret the signals.

People feel that numbers are not everything, that emotions play a big role as well in sleep quality. Therefore, tools should be more careful when judging how sleep was. Judging a night as long enough is better than saying the night was perfect, purely based on numbers.

As with every study, there are some limitations that should be taken into account when discussing these results. We only had six participants, who tested two devices in a pairwise comparison, for the duration of one week. Of course, a longer study with more participants would be very interesting. Then, a more quantitative approach could be taken as well, using standardized questionnaires to assess the usability of these systems. Also, by having a more correct gender balance, the importance of emotions might be nuanced or new themes might arise.

5 Conclusion

While the existing sleep trackers can help people to assess their sleep and improve their sleep quality, the users experienced some frustration while using these tools. We think that the tools should be more transparent. By finding the right balance between automation and control, trackers can be more effortless to use and forgiving to the user. By telling users what they see, and teaching them what they could do with this information, data can be more intelligible. Then, people can take the next step beyond measuring, and actually improve their sleep. And to conclude, trackers should also broaden their view on sleep by including emotions. Numbers are valuable to quantify sleep, but there is more. By giving room to emotions, users can get a more nuanced view on their sleep. With these small efforts, which are feasible, existing tools and technology can address user needs in a more regarding sleep-tracking at home.