1 Introduction

A lot is staked today on the ability of ordinary people to make the right choices, from environmentally sustainable consumption behavior to personal health and wellness. Although many are cognizant of the benefits of ideal choices and want to follow them, in practice, they find that habits are difficult to change.

It is difficult to resist temptations. If we find delicious food in a party while on a diet, it is hard to fight against the impulse. Similarly, many drive a car to save commuting time even though they know that using the available public transportation would be better in the long term. The tragedy of the commons, where resources with common access tend to be over-exploited, is likewise frequent across society [31]. Even though the ideal behavior in each of these cases is widely recognized, there are psychological and socioeconomic limitations to the ability of education alone to bring about changes in behavior. Especially in contemporary urban metropolises such as Tokyo, many people find themselves leading hectic, unhealthy, and unsustainable lifestyles, while feeling powerless to do anything about it. This article deals with question of how computing technologies can be designed to help people to align their behavior with their ideals.

Technologies that aim to change behavior have been studied under such rubrics as persuasive technology, captology [18], serious games [6], and most recently, gamification [24, 64, 79]. Technologies that aim to help people make ideal choices are also studied as decision support systems [77]. A common theme in all these fields is providing users with feedback regarding their choices and actions. Although people’s choices are always to some extent constrained and guided by their social and economic contexts, these systems have shown that with suitable support, individuals can often achieve significant changes in their habits.

In this article, we use the metaphor of a mirror to examine the art of effective feedback, especially visual feedback. A mirror provides its user with a new angle to perceive the self. It shows what shape we are in and how our clothes fit. It reveals details that would otherwise go unnoticed. Thanks to its visuality, it can present large amounts of information in a calm and unobtrusive manner, a key concern in ubiquitous computing research [29, 35]. The information contains various incentives to persuade people who use the mirror. Unlike an actual mirror, the metaphorical persuasive mirror does not need to consist of a single feedback surface; when the application requires it, it may be distributed across devices and environments. For this reason, we term our approach the persuasive ambient mirror.

A persuasive ambient mirror can be more selective about its feedback than a conventional optical mirror. It can apply various transformations and also integrate data from other sources. This opens up a number of dimensions for feedback design. The aesthetic expression can contain metaphors to show the importance of the targeted behavior. The feedback can also attempt to engage emotions and empathy. Virtual pets are very popular in many online services, as are social robot pets among some people. Such pets evoke empathy and encourage the caretaker to adapt their behavior when the pet expresses negative affects. Emotional impact is very effective in bringing about behavioral change. The artificial mirror can also provide amplified and enhanced feedback. In some cases the effect of the desired behavior does not appear quickly and may even be negative in the short term. Current technology may not able to measure the actual effect. For example, the effect of small eco-friendly actions on the actual CO2 output of the society is impossible to measure in real time. Some outcomes, such as health and wellness, are not easily quantified. A persuasive ambient mirror can present feedback based on predicted rather than actual outcomes.

The mirror is by no means a new metaphor in computer engineering. Galernter famously described a “mirror world” where a display acts as a window into the real world augmented with digital information [26]. Fujinami et al. [22] developed an actual mirror that superimposes useful information on the reflection. All these mirrors can be categorized using Moere’s taxonomy of ambient displays [58]. The taxonomy consists of three types of information visualization approaches: visualization as transformation, visualization as augmentation, and visualization as embodiment. Visualization as transformation refers to a traditional ambient display that represents information in an abstract form. Visualization as augmentation refers to ambient information that reflects and relates to the context, actions, and surroundings of the user. Visualization as embodiment refers to the notion of representing information as embodied objects.

There is a wealth of previous research on ambient displays that focus on the transformation and embodiment categories [17, 53, 63]. In these systems, the information does not necessarily have any direct feedback link with the user. The augmentation category is less examined; some of the best examples are cited above. Feedback in itself is studied in specialized disciplines such as captology [18], but the specific questions of visual feedback design are addressed only incidentally. We have developed a number of persuasive ambient mirrors like systems in the past [61, 69]. In this article, we present insights from the design and evaluation of these systems to advance the understanding of visual feedback design and visualization as augmentation.

The rest of the paper is structured as follows. In the following section, we present four case studies of persuasive systems developed in our project. In Sect 3, we draw together the lessons learned from the systems’ design and evaluation and draw implications from these lessons to the design of computing technologies that help people align their behavior with their ideals; these are presented in the form of a design guideline for persuasive ambient mirrors. In the final section, we examine future research directions in the area.

2 Four case studies

In this section, we first introduce a set of three a priori design principles that we used in developing our persuasive ambient mirrors. These are based on earlier cases and literature. Then, four case studies of persuasive ambient mirrors are presented, along with their corresponding user studies and results.

2.1 Design principles

The persuasive ambient mirror systems we developed are implemented using ubiquitous computing techniques, such as sensors and ambient display devices. Most implementation details are determined by the needs of the particular application and what behavior it targets. But, at the outset, we decided on three underlying design principles, based on the literature discussed in the previous section. Before presenting the case studies, we briefly introduce these principles and their rationales.

2.1.1 Implicit interaction

One of the key factors limiting the applicability of some earlier solutions is the various burdens they place on a user, either in the form of time or effort. For example, traditional commercial systems require users to set aside time for usage sessions or devote effort to entering data to the system. To avoid these burdens, our systems should be able to passively observe the user’s behavior. To eliminate the need to set aside time or go to a special place, the system should be integrated with normal daily activities. Thus, our first design principle is to use observations of the users’ behavior through the use of smart daily objects, as opposed to using keystrokes or some other proxy behavior.

2.1.2 Ambient persuasive visual expression

To complete the integration of the system into a daily living environment, we must also make sure that the output produced by the system is appropriate. A loud or disruptive feedback system might find itself thrown out of the house or workplace. Calm technology, such as informative art [35] and slow technology [29], that is able to leverage peripheral perception to deliver information, as opposed to constantly demanding direct attention, is a promising approach to seamlessly integrate information into daily living.

Persuasive ambient mirrors show information reflecting a user’s behavior as an ambient visual expression that is blended into the user’s daily environment. One of the important issues to design persuasive ambient mirrors is that the meaning of the expressions should be easily understood by users although the existence of the mirrors should be calm. The issue is as same as the existing ambient displays, but the expressions of persuasive ambient mirrors should in addition offer persuasiveness to the users.

2.1.3 Emotional engagement

The fact that feedback is delivered in a non-disruptive way must not mean that it ends up being irrelevant to the user. To provoke a change in behavior, we must be able to administer some sort of meaningful consequences to the user. For example, following a simple behaviorist model of learning, we might want to reward or punish the user in some way [65]. In a visualization system, it is impractical to provoke changes in physical reality, but would it be possible to make the user care about changes in the internal state of a computer system? Computer games seem to be able to do this. Good games are able to provoke a range of emotional responses, from fun and satisfaction to guilt and discontent. By mimicking the techniques used in computer games, we should be able to build an emotionally engaging persuasive system, allowing us to administer punishments and rewards without any physical resource. By emotional engagement, we do not necessarily mean strong and deep emotional responses, but the simple kicks that make many games interesting and addictive.

2.2 Case 1: Persuasive Art

Decorating walls with paintings, posters, and other pictures is common at home. Pictures enhance the aesthetic quality of daily surroundings. Persuasive Art uses a painting in an attempt to motivate the user to walk at least 8,000 steps every day to stay fit. The number of steps is monitored automatically using an electronic pedometer and stored into a computer. The painting shows the feedback of the current status of the user’s exercise to motivate him to maintain desirable habits.

Persuasive Art currently offers the following four types of paintings as shown in Fig. 1: (1) the figure painting is the portrait of Mona Lisa, (2) the landscape painting includes a tree that grows and withers, (3) the abstract painting has objects that change in size and complexity, and (4) the still life painting contains a changing number and size of orbs in a bottle. Each painting is a kind of a mirror to reflect the current status of a user’s exercise. Of course, in the real exercise, the effect like reducing a user’s weight may not be appeared soon and the fact may decrease the user’s incentive to continue the exercise. On the other hand, Persuasive Art shows the expression reflecting the amount of a user’s exercise. This will maintain the user’s incentive because the expression is changed according to the amount of the exercise.

Fig. 1
figure 1

Four alternative virtual paintings

Full size image

When using the landscape painting, the tree’s growth is varied according to the user’s behavior. When the user maintains desirable habits, the tree will grow, but if he stops the desirable habits, the tree will get sick. The painting adopts the following metaphor. The increase in healthy activities makes the tree healthier, but the neglect of the exercise makes the tree sick. When using the figure painting, Mona Lisa gets older and younger according to the users behavior. The increase in healthy activities makes Mona Lisa younger, and the neglect of the exercise makes Mona Lisa older. When using the abstract painting, the blue objects change in size and complexity according to user’s behavior. If the user maintains desirable habits, the objects “grow” significantly. When using the still life painting, the number and size of orbs in the bottle changes according to user’s behavior. Even if users do not maintain desirable habits enough, one orb is added into the bottle, but the size is small. If users do maintain desirable habits enough, one big orb is added. If users do not maintain at all, the bottle is cracked.

2.2.1 Experiments and key results

To understand the effects of each painting, we first conducted a simple experiment. We hired 6 participants from our laboratory (age 22–24) in 10 days. Participant’s steps were counted by Persuasive Art and they appraised those four types of paintings installed in the laboratory. In this experiment, any paintings did not increase users’ steps significantly, although the participants became more conscious of their exercises. Notably, all participants claimed that they felt touched by the tree expression. Most of them said this was because (1) they were connected emotionally to the tree and wanted to nurture it, and (2) they felt guilty when the tree died. Of course, the Mona Lisa painting also had an empathetic intention, but lacked unexpected expressions like the growth of a tree. From these impressions, we conclude that an important factor is applying empathetic expressions such as animate things.

Secondly, we conducted another experiment to investigate the effects of positive and negative expressions. We hired 8 participants (M:4, F:4, all are our university students not belonging to our laboratory) in 3 weeks. Just like the former experiment, each participant’s steps were counted by the application. In this experiment, one half of the participants appraised positive expressions without any negative expressions, other half of the participants appraised positive and negative expressions. In the former case, the tree just grew when participants did well. In the latter case, the tree grew when participants did walk well and got morbid when participants did not walk well. In this experiment, there was no significant difference in the numbers of steps. When we interviewed the latter participants how they felt the negative expression, 5 out of 6 participants commented they felt that they must walk more. However, the negative expression did not translate into the actual act. The other one participant emphasized that the painted tree looked revolting and she failed to continue the experiment. This shows that excessively negative feelings and discomfort may result in stopping the whole target behavior rather than leading to improvements in it. Thus, it is extremely important to choose the valence of expressions carefully.

2.3 Case 2: Virtual Aquarium

Virtual Aquarium [61] has the objective of improving users’ dental hygiene by promoting correct toothbrushing practices. The system is set up in the bathroom, where it turns a mirror into a simulated aquarium. Fish living in the aquarium are affected by the users’ toothbrushing activity. If users brush their teeth properly, the fish prosper and procreate. If not, they are weakened and may even perish.

In this system, we used a 3-axis accelerometer sensor that is attached to each toothbrush in the household. Since toothbrushes are usually personal and each sensor has a unique identification number, we are able to infer which user is using the system at a given time. Toothbrushing patterns are recognized by analyzing the acceleration data. A user brushes his teeth in front of the Virtual Aquarium using a brush with a sensor attached. The toothbrush is able to observe how the user brushes their teeth passively without requesting extra actions to play the game.

The objective of Virtual Aquarium is to promote good toothbrushing practices. In this prototype, the ideal behavior was defined as follows: (1) users should brush their teeth at least twice per day; (2) one session should involve at least 3 min of brushing; and (3) brushing should involve patterns that ensure the teeth are properly cleaned. User behavior is compared to this ideal and translated to feedback as described below. We believe that the existence of an aquarium is in-artificial in a lavatory, but the aquarium enriches our daily life.

When a user begins to brush her teeth, a scrub inside the aquarium starts cleaning algae off the aquarium wall. At the same time, a set of fishes associated with the user starts moving in the aquarium in a playful manner. When the user has brushed for a sufficient time, the scrub finishes cleaning and the fishes’ dance turns to a more elegant pattern. When the user finishes brushing, the fish ends their dance and resumes their normal activities. Both the activity of the fish and the movement of the scrub are designed in such a way as to give the user hints regarding the correct method of toothbrushing. The pictures in Fig. 2 show a scene from the aquarium during brushing. However, if a user does not brush their teeth sufficiently, the aquarium becomes dirty and the fish in the aquarium become sick. The feedback information is returned immediately according to the movement of a user’s toothbrush. We call the feedback immediate feedback.

Fig. 2
figure 2

Persuasive expressions in Virtual Aquarium

Full size image

The fishes’ health is visibly affected by how clean the aquarium is. If the user neglects to brush her teeth, some fish fall ill and may even die. In contrast, faithful brushing may result in the fish laying eggs as shown in the right pictures in Fig. 2. At first, the eggs are not very likely to hatch. If the user continues to brush consistently for a number of days in row, the incubation ratio increases. This way, the long-term feedback gives clues to the correct behavior and attempts to maintain motivation over a period of time. The long-term feedback is called accumulated feedback.

While designing the prototype, we consider the association between a user’s healthy lifestyle and the cleanness of the aquarium. Also, our design attempts to invoke empathy for the virtual fish figures.

2.3.1 Experiments and key results

After we had designed and built Virtual Aquarium, we obviously wanted to test whether such a system would encourage changes in people’s lifestyle patterns as hypothesized. To this end, we designed a user study that evaluates the effectiveness of the Virtual Aquarium in improving users’ toothbrushing patterns. The study consists of three phases, briefly described below:

  1. 1.

    For a set of test households representative of the general population, an accelerometer sensor is attached to each person’s personal toothbrush. The households are then asked to continue their normal activities, while daily toothbrushing patterns are recorded. Once the patterns have stabilized over time (after possibly spiking slightly due to the intervention), the study is moved to the next phase.

  2. 2.

    The Virtual Aquarium feedback components are introduced to the households’ bathrooms while toothbrushing patterns continue to be recorded. Once the patterns have stabilized to a predetermined degree over time, the study is moved to the final phase.

  3. 3.

    The feedback components are removed while toothbrushing patterns continue to be recorded. Once the patterns have stabilized to some predetermined degree over time, the data collection is concluded. Brushing records are analyzed to find out the effect that the introduction of feedback had on the patterns and to what extent the effect seems persistent.

We did not have the resources to carry out the study in a scale necessary for statistical validity, but a 12-day pilot study was conducted. The participants were three male and four female volunteers from three households associated with the researchers. All of them are Japanese. Figure 3 shows how the recorded toothbrushing time per session for each participant changed during the study. In all cases, the brushing times were less than 3 min per session at the beginning of the study and increased to over 3 min when Virtual Aquarium was introduced. When the feedback system was removed, the times fell back, yet mostly remained higher than their initial levels for the remainder of the study. How well the sensor data describe actual behavior obviously critically affects the validity of the results. Users could have cheated the system by waving the brush in their hands. Whether this should be seen as a problem depends on who is using the system on whom. In a concluding interview, the participants of this study self-reported that their brushing patterns had in fact changed, at least for the time being.

Fig. 3
figure 3

Brushing time for each morning (mo) and night (ni) of the study for each participant. Total brushing time is the interval between start and end times of accelerometer use, while actual brushing time is the number of seconds the accelerometer sensed movement

Full size image

During the study, participants were not always able to use the system in the way they wanted to due to one reason or the other. Failing to wake up in time and having little time to spend on toothbrushing was a common occurrence. Users could also not use the system when spending a night elsewhere than one’s own home. In these cases, they indicated that they were frustrated by having to neglect the system. Future persuasive ambient mirrors should be able to support increasingly nomadic lifestyles to be useful, meaning that feedback should be available anywhere at any time.

It is also worth mentioning that five of the users said that their families became interested in the status of the aquariums, e.g., following the changes in the number of fish. Virtual Aquarium, and by proxy the behavior of the user, became something of a topic of daily discussion in these families. In the future, we might wish to investigate the use of persuasive ambient mirrors as a social media that brings users closer to their family members. In any case, the results suggest that persuasive ambient mirrors are quite acceptable to most people. There is even reason to believe that comparable systems are becoming widespread.

2.4 Case 3: Mona Lisa Bookshelf

Resources shared by a number of people, such as a public bathroom or a bookshelf in a research laboratory tend to deteriorate quickly in a process called the tragedy of the commons. This happens because each individual derives a personal benefit from using the resource, while any costs are shared between all the users, leading to reckless use. Garret Hardin, the ecologist who popularized the concept, noted that this belongs to the category of problems that cannot be solved by technology alone, requiring instead a change in human behavior [31]. Mona Lisa Bookshelf [61] is aimed at keeping a communal bookshelf organized. It tries to encourage users to keep books in order and to return missing books, but also to take books out every now and then for reading. Each book in the shelf is linked with a piece of a digital image of the Mona Lisa. Like a picture puzzle, the image changes according to how the books are positioned. The Mona Lisa picture is a kind of a mirror to reflect the situation of the public bookshelf. A high-quality flat display placed near the bookshelf shows the image to the users.

The tracking of a user’s behavior is based on optically detecting books in the shelf. In the prototype system, visual tags are attached to the spines of the books to facilitate their detection and identification. Visual tags are also attached to the corners of the shelf to determine its dimensions. The detection system comprises the following hardware: a digital video camera, a high-resolution digital camera, and two infrared distance detectors. The distance sensors and the digital video camera are used to detect whether the user is manipulating books in the shelf. As soon as the user is seen leaving the shelf, the high-resolution still camera takes a picture of it and all the books contained within it. Images captured by the still camera are analyzed by the VisualCodes software library, which recognizes the visual tags attached to the books. Each visual code yields data regarding its position, alignment, and identity. This is then translated into context information that describes the bookshelf’s width and height, which books are currently contained in shelf, and how they are aligned and ordered. This information is then passed to the feedback logic component. The above approach is able to observe how the user uses her bookshelf passively without requesting extra actions to use the system.

In this system, the feedback logic aims to encourage the following ideal behavior: (1) books should be arranged correctly and aligned neatly and (2) at least one of the books should be read at least once per week. The correct arrangement of the books is pre-programmed and could be, e.g., alphabetical. User behavior is compared to this ideal and translated to feedback as described below.

Mona Lisa Bookshelf features two types of feedback that aims to encourage users to keep the bookshelf in order and to take books out for reading. When a book is removed from the shelf, the corresponding piece of the Mona Lisa image also disappears. If books are lying on their face or otherwise misaligned, the pieces of the image also become misaligned, distorting the painting. When the books are arranged neatly, Mona Lisa smiles contently. The assumption is that users are aware of how da Vinci’s Mona Lisa is supposed to look like, and as when completing a picture puzzle, inherently prefer the correct solution to a distorted image. The immediate feedback thus provides clues and motivation for keeping the bookshelf organized. The top right image in Fig. 4 is an example of a distorted image. Also, there is an accumulated feedback mechanism that attempts to encourage users to read the books once in a while. If none of the books are removed from the shelf for over a week, Mona Lisa starts getting visibly older. The bottom left image is an example of an aged portrait. As soon as one of the books is removed from the shelf, she regains her youth.

Fig. 4
figure 4

Persuasive expressions in Mona Lisa Bookshelf

Full size image

2.4.1 Experiments and key results

We conducted a simple experiment in our laboratory whether users could change their behavior to deal with shared resources. One 61-inch plasma display showed Mona Lisa related to one big bookshelf in the laboratory in 2 weeks. Unfortunately, this was not effective at all. Firstly, someone who noticed the relationship between the picture and books kept book organized. However, after few days, many of those researchers and students lost interest in the picture and Mona Lisa was left apart. Besides, some participants commented that the apartness and missing were fun to see, so they changed the order of books intentionally. This failure is because the presentation could not solve the tragedy of the commons. Even if Mona Lisa was distorted, anyone could not find out who did not keep the bookshelf organized. Besides, the participants commented that they assumed somebody else read books when the Mona Lisa got older. In short, the presentation could not encourage a sense of ownership.

2.5 Case 4: EcoIsland

Public and private efforts to change individual behavior toward more environmentally friendly practices usually rely on education, but there are psychological limits to the ability of education alone to effect behavioral change. Even when a person fully well knows that a particular behavior is detrimental enough to their long-term well-being to offset any possible short-term benefits, they may still irrationally choose the short-term indulgence. Future consequences, while widely known, are easily ignored in the present.

EcoIsland [69] is a game-like application intended to be used as a background activity by an ecologically minded family in the course of their normal daily activities. A display installed in the kitchen or another prominent place in the household presents a virtual island. The display showing the virtual island is a kind of mirror reflecting a family’s cooperation status of eco-friendly activities. Each family member is represented on the island by an avatar (Fig. 5). The family sets a target carbon dioxide emission level (e.g., national average minus 20%), and the system tracks their approximate current emissions using sensors and self-reported data. This means that a user needs to report the user’s action explicitly, so the case deviates from the first principle described at the beginning of the section because it is hard to automatically detect the user’s action accurately. If the emissions exceed the target level, the water around the island begins to rise, eventually sweeping away the avatars’ possessions and resulting in a game over.

Fig. 5
figure 5

Persuasive expression in EcoIsland

Full size image

On their mobile phones, the participants have a list of actions that they may take to reduce the emissions: turning down the air conditioning by one degree, taking the train instead of the car, etc. Upon completing an action, a participant reports using the phone and the water level reacts accordingly. Reported activities are also shown in speech bubbles above the corresponding avatars. A lack of activity causes the avatars to suggest actions.

The general approach is again to provide a feedback loop for user behavior. The virtual island shown in the display acts as a metaphor and makes the participants conscious of the ecological consequences of their choices and activities. We also tap into social psychology, attempting to exploit social facilitation and conforming behavior to encourage the desired behavior. EcoIsland’s design facilitates these by involving the whole family and by presenting the participants’ activity reports in the speech bubbles and providing contribution charts and activity histories. On the other hand, the fact that the system is used by a family unit instead of an individual means that participants can also agree to assign tasks to certain members. The goal to reduce carbon dioxide emission and speech bubbles presents immediate feedback, and the situation of each family’s island reflects accumulated feedback.

The application also includes a trading system, which is based on the same principle as industry level emissions trading systems: reductions should be carried out in places where it is easiest to do so. A family that finds it easy to make significant reductions can sell emission rights to households that find it difficult due to, for example, their location or occupation. This should make it possible to achieve the same total reductions with a lower social cost. The credits are also used to buy improvements and decorations to the island; so, successful sellers can afford to decorate their island more, while heavy emitters have to spend their allowance on emission rights.

2.5.1 Experiments and key results

We conducted an experiment to investigate whether social factors and economical notion were effective. We recruited 6 families (20 persons, aged 15–58) who were interested in environmental issues. The experiment lasted for 4 weeks. In the first week, we equipped participants’ air conditioners with a simple electricity usage meter to compare the readings between experiment weeks. In the second week, the application was installed and only one family member from each household used it. In the third week, all family members used the application. Comparing the results of the second and third week provides insights regarding the social psychological effects. In the fourth week, we introduced the emission trading system. After the experiment, we conducted a survey in the form of a questionnaire asking about changes in the participants’ attitude and feedback.

From the questionnaires, 17 out of 20 participants said that they were more conscious of environmental issues after the experiment than before. Some families said that the sinking virtual island contributed to a change in their consciousness, suggesting that the metaphor works well. Our system log of the reported emission reduction shows that 5 out of 6 persons reported more actions in the third week than the second week, lending support to the hypothesis that social facilitation and conforming behavior can be used effectively. As for the air heater electricity usage, however, there was no observable correlation with the reported emission reducing activities. While this is an alarming result, it did reflect that the experiment period was short considering ordinary day-to-day variance in electricity use.

Meanwhile, during the fourth week, only 2 out of 6 families used the emissions trading system. 10 participants commented that the target reduction levels were so easy to achieve that there was no need to resort to emissions trading. This shows that these kinds of systems need appropriate “parameters,” just like video games, to motivate users. In this case, the parameters were target levels, effects of actions, and currency allowances. Extensive play testing is a typical way to find correct parameters.

3 Design implications

In the design and evaluation of the cases presented above, four recurrent issues proved to present a challenge in every case. The first issue is how to design incentives to change a user’s behavior. The second issue is how to design visual expressions that somehow touch the user to achieve the desired impact. The third issue is how to control the timing of the feedback to users. The fourth issue is how to manage the interaction between the systems and users. Below, we explain these challenges in detail and draw on our case studies and elementary concepts in psychology and social sciences to outline feasible solutions to them. The resulting generalized design guideline can be applied in the design of persuasive ambient mirrors and other persuasive systems.

The design space outlined above is summarized in Fig. 6.

Fig. 6
figure 6

Persuasive ambient mirror design guidelines

Full size image

3.1 Choosing an incentive system

When designing a persuasive ambient mirror that aims to motivate the user to carry out their desired behavior, a number of different strategies for motivating or incentivizing the user are possible. We have extended Jordan’s pleasure framework [39]Footnote 1 to classify these strategies into five categories: physical incentives, psychological incentives, social incentives, economic incentives, and ideological incentives.

3.1.1 Physical incentives

Utilizing the physical design and shape of objects and environments to suggest or even force certain behaviors is generally studied under the rubric of affordances and, more recently, persuasive architecture. However, the physical incentives referred to here are more dynamic, capable of being adjusted and working through bodily sensations, such as comfort and discomfort. The five senses can be directly stimulated through physical actuation. The mechanism through which such incentives work is obvious, but most users probably dislike receiving discomfort as a negative feedback. Our case studies did not use physical incentives, but there are opportunities to use them in smart spaces that incorporate physical actuators, such as motors that adjust the height of a desk or a chair.

3.1.2 Psychological incentives

Broadly speaking, psychological incentives can refer to any motivational techniques that relate to the users’ mental and emotional landscape. In our case studies, we used two varieties. Virtual Aquarium and Mona Lisa Bookshelf used visual representations to evoke affects or mental states that were intended to influence user behavior. In Virtual Aquarium, the cheerful dancing fish encourage the user to persist in their targeted behavior, and in Mona Lisa Bookshelf, the unattractive Mona Lisa is intended to push the user to change their behavior through evoking negative affects. Affects and emotional states are often used as a tool to influence behavior in marketing. There are various categorizations and models, but one model identifies the following eight emotions that can be targeted in industrial product design: happiness, joy, contentment, satisfaction, anger, irritation, disappointment, and dissatisfaction [14]. How to evoke such affects through digital representations is discussed in the next section.

The second variety of psychological incentivizing used in our case studies can be classified under the often cited “flow” family of motivation theories [65]. Like many computer games, Virtual Aquarium presents the user with arbitrary goals; in this case, removing all the algae from the aquarium by means of moving the toothbrush. Even though the goal itself is arbitrary, if the path toward reaching it is sufficiently yet not too challenging, users are tempted to pursue it. The challenge for designers is to develop dynamically adjusting challenges that keep users attracted. As with many computer games, the risk with systems like Virtual Aquarium is that once the novelty wears off, the challenge may no longer be attractive in itself. Designing captivating computer games remains as much an art as it is an exercise in applied psychology.

3.1.3 Social incentives

Another set of strategies toward persuading users can be found in the literatures of social psychology and sociology. Being social animals, humans react to the actions, expressions, and mere presence of other humans. Two concrete strategies explored in our case studies are social facilitation and conforming behavior [25]. Social facilitation refers to the phenomenon where people perform better at a task when someone else, such as a colleague or a supervisor, is watching. This applies to simple tasks that are easy for the subject; in complex tasks, performance can instead suffer. In the user study of Virtual Aquarium, one participant said that family members’ presence encouraged their performance of the task. Social facilitation and inhibition effects have been demonstrated to take place even when physical human presence is replaced with digital avatars [32]. In EcoIsland, family members are represented by avatars, so that their presence is felt even when they are not physically present. The idea is that this would facilitate the quite simple energy-saving tasks, although the effect was not explicitly tested in the study.

Conforming behavior is the desire not to act against group consensus, colloquially known as peer pressure. Within social formations such as families and workplaces, there are also power relationships that exert similar pressures. In Virtual Aquarium, children’s toothbrushing behavior is revealed visually through the health of the virtual fish to other family members. In EcoIsland, each users’ recent actions are displayed to others in speech bubbles, revealing any slackers to others and thus subjecting them to possible social punishments. In a larger group, a phenomenon called social loafing may take over. Social loafing occurs when people make less effort to achieve a goal when working in a group than when working alone. This happens because an individual feels less personal responsibility for the result when part of a group.

One additional effect that could be utilized is reciprocity or the desire to reciprocate gifts and favors received from others. Virtual gifts are frequently exchanged in online environments, strengthening the relationships between users [47]. In EcoIsland, users can buy virtual items to decorate their island if they work hard enough to achieve the energy-saving goals they have set. If these virtual items could be exchanged among families, this could lead to a gift exchange culture that works as one additional motivation toward reaching the goals.

We also need to consider how we can make each member’s situation anonymous, if the number of users is increased. In this case, the system needs to offer a way to take into account the information explosion produced by a large number of users’ visual expressions. For example, a visual expression for his rival may highlight the current situation. The statistical information can be represented in a visual expression to know whether a target user’s effort is above or below the average.

Finally, when considering social incentives, it is important to account for cultural differences [43]. For example, the effect of peer pressure may be stronger in so-called collectivist cultures and settings than in more individualist ones.

3.1.4 Economic incentives

Not surprisingly, we also found that using economic incentives is a powerful technique to motivate people to change their behavior. An economic incentive is a tangible reward that the users consider valuable, yet not necessarily actual money or goods. In online games, millions of players work hard to obtain rare and valuable virtual goods and even trade these goods for real money at a rate $3 billion per year [48]. In EcoIsland, we sought to tap this phenomenon by including virtual items in the system that users can decorate their islands with. The items are bought with virtual currency that users must earn by meeting their energy-saving goals. Another example of economic incentives in persuasive technology is the activity-based billing system [76] that uses automatic micropayments and microrewards to coax users toward desired behaviors.

Although economic incentives are a powerful tool to motivate behavior, they may lead to unpredictable results if not used carefully [49]. EcoIsland could be modified to use only economic benefits to motivate families, but that approach would lack the pedagogic aspect of reminding users of the importance of environmental sustainability through metaphors. The desirable changes would be reversed if the economic incentives are removed. Lacking respect for the cause underlying the incentives, users might also see no problem in cheating the system, that is, obtaining the benefits without the actual behavior.

3.1.5 Ideological incentives

What is here referred to as ideological incentives is the notion of influencing user behavior through influencing their attitudes and values, in other words, educating the user on a deeper level. Attitudes and values influence the users’ behavior in the long term. The ideological incentive makes it possible to motivate the user by himself. The user raising the ideological incentive has a belief called self-efficacy that makes him believe he is able to achieve his goal. In our current case studies, we choose simple metaphors to be understood by the user easily, but the metaphors have shallow impact on the understanding of the importance of desirable lifestyle.Footnote 2 The more detailed discussions about metaphors can be found in the next section.

The ideological incentive is brought by intellectual stimulation. For maintaining desirable behavior, it is important that the user is aware of the importance of the desirable behavior. The association between the effect of desirable behavior and the persuasive ambient mirror offered to the user as feedback information is an effective intellectual stimulation. Other incentives described in the previous paragraphs cause an extrinsic motivation that makes it difficult to maintain the user’s desirable behavior for a long time. On the other hand, the ideological incentive may cause an intrinsic motivation and the changed behavior is maintained for a long time. In order to use the ideological incentive, the user needs to perceive self-efficacy. Since a positive attitude is necessary to grow the user’s self-efficacy, it is essential to use positive feedback and the user should be aware of his ability to change his current lifestyle.

3.2 Making digital representations meaningful

3.2.1 Communicating meanings through metaphors

In the user study of Persuasive Art, we found that people preferred the tree picture and Mona Lisa over the abstract and the still life paintings. The reason given was that more figurative paintings were considered to be more “intuitive” and suggest empathy. While any visual representation can be used to relay information, shapes that come with pre-attached meanings (e.g., “a tree withering is a negative thing”) are more capable of evoking emotional engagement. The meaning attached to an expression is essential to design the persuasiveness in our approach.

Attaching the meanings to the forms of products is necessary to design the user experience [34].Footnote 3 When designing a persuasive ambient mirror, the metaphor visualizing the user’s lifestyle needs to be attached to the visual expression. Also, the metaphor shows the attitude of the user’s desirable lifestyle. In Virtual Aquarium, a clean aquarium is a metaphor for clean teeth. In Mona Lisa Bookshelf, beautiful Mona Lisa is a metaphor for a well-organized bookshelf. In EcoIsland, a sinking island is a metaphor of the effect of global warming, and the environmental wellness is associated with the situation of a family’s island. In our case studies, the user needs to know the associations before starting to use the systems. The metaphor used in the visual expressions is easily understandable, but he/she may not be aware of the association without learning the meaning of expressions. Of course, we can use much direct associations between the effect of the user’s behavior and the visual expression. For example, in Virtual Aquarium, we may use the presentation of virtual teeth. Also, in EcoIsland, we may use a real photo or a more realistic synthesized 3D graphic showing the current environmental situation.

Another way to represent lifestyle goals is to use an art form. Some visual expressions may represent numbers showing goals in an art form or a portrait photo showing what the user expects to become in the near future. We believe that art thinking will help to design effective persuasive expressions that offer more stimulative experiences to consider the important of desirable lifestyle. For example, contemporary conceptual art uses complex metaphors to provoke deep reflections on issues like sustainability and peace in the world. Such presentation styles could be used to obtain a deeper impact in persuasive ambient mirrors, but with the cost of more effort required from the user [7].

However, users sometimes misunderstand the meaning of the visual expression, and this is one of the dangers of relying on metaphors [27]. Users may find unintended meanings in a visual expression [70]. For example, if a supposedly unattractive picture is used to discourage undesirable behavior, that picture may actually have the opposite effect on an avant-garde or ironic art consumer. Also, in Japan, people usually do not like realistic expressions. This depends strongly on the cultures and personalities of the users. It is not easy for a designer to attach a single meaning to a specific expression for all people. The interpretation of the expression could be left to the user. This open interpretation [68] allows the user to feel pleasure or good surprise, but it is not easy to predict the effect of the interpretation by the user in a controllable way.

3.2.2 Aesthetics and empathetic expression

The chosen incentives are associated with persuasive expressions. Two types of visual expressions are used in persuasive ambient mirrors. The first type is an aesthetic expression.Footnote 4 The type is used to associate the effect of the user’s behavior with the metaphor displayed in the expression. The ideological incentive is usually associated with the aesthetic expression. The goal of the lifestyle can be encoded into an aesthetical expression by using various metaphors. On the other hand, the loss of aesthetics can be used as the psychological incentive. In Mona Lisa Bookshelf, a Mona Lisa picture associates its beauty and the integrity with the well-organized public bookshelf.

The user feels empathy to the empathetic expression. The emotional engagement evoked by persuasive ambient mirrors involves liking and disliking the feedback expressions. These feelings are essential to make the effects of persuasive ambient mirrors effective. This means that we need to design the expression so that users feel empathy for them. Expressions using virtual creatures and anthropomorphic characteristics are good at this. However, people’s tastes vary and cultural differences also matter. In general, human evokes empathy when a creature is growing by his care.Footnote 5 For example, the sickness of virtual fish evokes negative emotion; then, the user is motivated to clean up the aquarium in Virtual Aquarium.

In our case studies, we chose figures or living creatures as empathetic expressions. In Persuasive Art, a growing virtual tree is chosen. In Virtual Aquarium, virtual fishes are used. In Mona Lisa Bookshelf, a figure of Mona Lisa is used, and each user’s avatar is used in EcoIsland. As people usually like to bring up a virtual tree and fish, they feel empathy for the virtual creatures. It is a cultural issue whether people feel empathy toward virtual creatures or not.

Cheok [12] showed that a real creature is found to rise more empathy than a virtual creature. However, especially in Japan, people also feel empathy toward virtual creatures. Fujinami [23] showed that Japanese users feel even empathy for virtual creatures represented as abstract symbols. Moreover, people usually like avatars as used in EcoIsland more than real figure pictures. We sometimes assign different meanings to a real creature and a virtual creature because we know the difference between them.

An empathetic expression using avatars makes it possible to use social incentives effectively. The avatar can evoke other users’ emotion by changing its facial expression or the tone of their spoken voice.

When we cannot use virtual creatures, it is necessary to employ another approach to evoke empathetic emotion. Decorating personal belongings is effective to develop empathetic feeling of the user [60] because it becomes a specialized product that they feel is the only product for them. For example, EcoIsland enables the user to decorate their island with virtual items. The decoration makes the island unique and special to the user, and the incentive to protect the island is increased. This is comparably to how Japanese teenage girls decorate their mobile phones and iPods extensively to make them personalized and their “own.”

3.3 Designing the feedback loop

3.3.1 Positive and negative stimuli

A key issue in designing persuasive ambient mirrors is when and how feedback is returned to the user. In our case studies, we used two types of feedback: immediate feedback and accumulated feedback. Immediate feedback is returned immediately according to the user’s current behavior, and the feedback expression visualizes his current behavior. For immediate feedback, we adopt the basic approach from behavioral psychology, known as operant conditioning, to encourage or discourage the user’s behavior [65]. It is based on a naïve model of human motivation, but it is nevertheless useful in shaping simple behaviors.

In operant conditioning, feedback content can be divided into reinforcement and punishment depending on whether behavior is encouraged or discouraged. Reinforcement and punishment are further divided into four types:

  1. 1.

    Positive reinforcement: encouraging the user’s behavior by providing a positive stimulus.

  2. 2.

    Negative reinforcement: encouraging the user’s behavior by removing an aversive stimulus.

  3. 3.

    Positive punishment: discouraging the user’s behavior by providing an aversive stimulus.

  4. 4.

    Negative punishment: discouraging the user’s behavior by removing a positive stimulus.

Our case studies use the combination of the above four types of the feedback. In most of our case studies, the user’s behavior is changed due to a control loop created by positive reinforcement and negative punishment. The visual expression in a persuasive ambient mirror evoking positive emotion makes the user feel positive reinforcement that encourages him, and the visual expression evoking negative emotion makes the user feel negative punishment that discourages him. For example, in Virtual Aquarium, dancing fishes make users excited and increase their positive emotion, but when Mona Lisa is getting old in Mona Lisa Bookshelf, the negative emotion is increased and they start feeling anxious. Emotional engagement is a very powerful tool to change the user’s undesirable behavior and to keep desirable behavior.

3.3.2 Short-term and long-term goals

The interval of accumulated feedback is usually a few days or a week. In Virtual Aquarium, a fish produces an egg that hatches to reveal an attractive fish, if the user brushes their teeth consistently for a week. Sinking an island in EcoIsland is also accumulated feedback to reflect family members’ lazy activities. We believe that it is more effective to incorporate a long-term goal in accumulated feedback. A long-term goal makes the user aware of the merit of a target desirable behavior and develops intrinsic motivation to maintain the desirable behavior. In Persuasive Art, the growth of the tree can be reinitialized every week to start a new goal, but it may reduce the sense of achievement in a long term. The short-term goal should be a part of the long-term goal. When a short-term goal is achieved, a new short-term goal that is a little bit closer to the long-term goal is defined. The user feels that he/she is closing to the long-term goal gradually. Once the user achieves a long-term goal, maintaining the effect will become more important.

3.3.3 Balancing positive and negative feedback

Motivating humans can be classified into two approaches. One is to make users aware of their current situation and the other is to enhance the user’s willingness to change his habits. Motivating a change of habits can also be classified into two types. The positive expression style increases the user’s positive emotion to motivate a change in the user’s undesirable habits. The user feels happy when changing their undesirable habits even if the change is challenging and hard. Another type is the negative expression style. This promotes negative emotion to feel a sense of crisis that motivates to change the user’s undesirable habits. For instance, if a user looks at himself in a mirror and finds that he is significantly overweight, this may motivate him to do more exercise.

The persuasive ambient mirror needs to be designed when returning feedback information. There are two issues in terms of timing. The first issue is whether positive or negative feedback is used. Our approach uses operand condition to control users’ behavior. The positive feedback assigns positive meaning to an expression, and the negative feedback associates the negative meaning with the visual expression in a persuasive ambient mirror. As described in [56], it is important to consider the balance between positive feedback and negative feedback to design successful persuasive visual expressions. In similar systems like UbiFit Garden [13], only positive feedback is chosen to design persuasive ambient mirrors. In the systems, their negative feedback is very weak, but if negative emotion is evoked too strongly, the user may feel helplessness and the effect will become ineffective eventually. When only positive emotion is evoked, the user will feel bored quickly. The strength of positive and negative emotion is relative. This means that strong negative feedback is necessary when strong positive feedback is selected. The strength of feedback is taken into account the phase that a user changes his attitude.

The visual expression associated with the positive meaning evokes positive emotions like happiness, joy, contentment, and satisfaction. For evoking happiness and joy, an empathetic expression is used, and for evoking contentment and satisfaction, an aesthetic expression is used. On the other hand, visual expressions associated with negative meanings evoke negative emotions like anger, irritation, disappointment, and dissatisfaction, as described in [14]. For evoking anger and irritation, an empathetic expression is used, and for evoking disappointment and dissatisfaction, an aesthetic expression is used. For example, in Virtual Aquarium, dancing fishes evoke joyful emotion and sick fishes evoke disappointed emotion. Also, a clean aquarium evokes satisfied emotion, and a dirty aquarium evokes dissatisfied emotion.

The feeling as information theory is useful to consider how an expression evokes the user’s emotion [67]. The theory indicates that it is difficult to think rationally during a positive feeling. On the other hand, the user tends to think rationally when she feels to be in a negative mood. The results indicate that positive stimuli to evoke positive emotions are effective in early stages of behavior modification, but in latter stages, negative stimuli to evoke negative emotions are desirable. Virtual Aquarium provides a positive stimulus when the user’s current behavior is desirable, but a negative stimulus is returned when she behaves undesirably. One of key issues is that negative stimulus alone are not effective. Because the user becomes rational, he/she considers the effectiveness of his behavior. She needs more information to think the importance of the activity in a rational way.

In the positive psychology, thinking positively makes people to feel a self-efficacy and to try to challenge hard tasks [20]. The fact tells us to use negative stimulus carefully because strong negative stimulus offer a possibility to lose self-efficacy from the user and to feel helplessness. Finally, he gives up to change his current undesirable behavior. However, the weak negative stimulus is effective, if the user is already believing that a target activity is important for the user. They act as a kind of a reminder to encourage continuing their efforts. Social pressure is also a kind of negative stimulus, but it works well in EcoIsland. In EcoIsland, negative stimuli based on empathy are effective to increase the user’s eco-friendly behavior. In Virtual Aquarium, when the user neglects to wash his teeth, negative stimulus, e.g., virtual fish’s sickness is returned.

3.3.4 Stages of behavior change

The so-called transtheoretical model of behavior change (TTM) used in studies of health and behavior change posits that there are five stages through which a person progresses as their undesirable behavior is changed [73]. Controlling a feedback loop needs to take into account the stage that the user changes his attitude. In the early stage, the user is not aware of the importance of the desirable lifestyle. The metaphor noticing the user the importance does not have enough impact on his attitude. The empathetic expression is useful in this case, because the expression makes the user enjoyed by evoking their positive emotion. In an early stage of behavior change, strong positive feedback is desirable, but negative feedback should not be too strong because in this stage, the user should not be discouraged to change his attitude. It is also useful to set up the goal that the user has a fun time. For evoking strong emotion, a user needs to be conscious of focusing on the achievement of the goal.

In the next stage, the user needs to be aware of the importance of the desirable lifestyle. The aesthetic expression gives a metaphor to show the importance of the lifestyle, while enjoyable stimulus encourages the user to continue the desirable behavior. In this stage, the user starts to discover the importance of the desirable behavior, and their attitude is also changed gradually according to the user’s cognitive level to understand the importance. This stage is motivated by the ideological incentives, and the incentive offers the long-term feedback to encourage the user to maintain the desirable lifestyle.Footnote 6 Designing the ideological incentive in the visual expression in a persuasive ambient mirror has been demonstrated in conceptual art and design. The direction offers possibilities to integrate information technologies and art/design knowledge to design the mapping between information and its meaning of the information more objectively.

When the user changes his attitude and maintains his desirable lifestyle for a long time, it is important to notice a user when their behavior is not sufficient as a light-weight reminder. In this case, strong positive feedback is annoying and the negative feedback should be sufficiently strong to trigger awareness reliably. The user needs to improve the performance of his behavior to shape more desirable lifestyle in this stage. The user needs to know detailed information for making a plan to analyze the current behavior to improve the efficiency of the behavior. The persuasive ambient mirror is not enough in this stage, and the expression containing detailed information for analyzing the current behavior becomes essential. The user needs to spend some time to analyze the information for analyzing their behavior.

Therefore, for designing persuasive ambient mirrors to be used for a long time, it is important to change the visual expressions and timing according to the user’s current stage to change his attitude.

When designing incentives based on TTM, the incentives should be change smoothly according to each stage. When the economic incentive is chosen in the first stage, the incentive is very hard to replace to other incentives. Therefore, the economic incentive may not be suitable to be used with TTM.

3.4 Interaction management

Embedding computers into daily environments makes it possible to use implicit interaction between human and environments [66]. Virtual Aquarium uses a 3D accelerometer to recognize the movement of the user’s toothbrush to observe their behavior without interacting explicitly. Our experiences show that recognizing the user’s behavior with sensors implicitly has limited reliability. In Virtual Aquarium and Mona Lisa Bookshelf, we chose to analyze a very simple context that can be implemented in a reliable way. However, it is very difficult to analyze the user’s complex behavior correctly even if using heavy-weighted learning-based algorithms. Automatic reporting using sensor devices may be possible, but this cannot make users fully aware of possible desirable lifestyle. For example, EcoIsland uses a self-reporting method to input what kind of actions the user takes in order to avoid complex behavior analysis. EcoIsland encourages users to input their actions to reduce CO2 emission since they are recognized as eco-conscious persons. A self-reporting method requires the user to motivate to input their activities explicitly. Acting in a green way is more complicated than some of the other desirable activities, e.g., not smoking or walking more. In the meanwhile, people tend to be lazy when they do not know what they should do or can do. Suggestions will make a participant conscious of the set of activities desirable for reducing CO2 emissions.

Generally, when starting to learn something new, we often need to repeat activities (e.g., writing a spelling) many times. Self-reporting generates the same phenomenon, requiring users to repeat the eco-friendly activities and they need to be aware of the activities explicitly. Repeating the same activity will possibly lead users to learn the importance of each activity and forms intrinsic motivation for doing the desirable activities.

Because ubiquitous computing technologies are improving, complete manual input will be old-fashioned and it is desirable to offer a more light-weighted method to report activities. Most of the existing persuasive applications encourage simple activities that can be easily detected using current technologies, but in EcoIsland, multiple complex activities should be identified, and the activity detection is not easy to implement in the current sensor-based activity detection technologies. We need to investigate a more light-weight method like a gesture to report activities by combining both implicit sensor-based interaction and explicit interaction. The user will be able to input his current action with a minimum cognitive effort. Lightweight interaction can also be used to correct mistakes of behavior analysis. Gesture analysis is easier to compare than general behavior analysis. The combination of explicit interaction with gesture analysis and implicit interaction with sensor data analysis is a very interesting topic and has a critical role in our future researches.

One of the problems in the current case studies is that the user may cheat the analysis of the sensors consciously. For example, in Virtual Aquarium, some users imitated the movement of their toothbrushes in order to make the fishes dance. There are two approaches to solve this problem. The first approach is to prohibit cheating by increasing the accuracy of the movement analysis. The second approach is to encourage the user not to cheat to use sensors. From our experiences, the user is encouraged to keep desirable behavior if they deeply thinks about the merit behind the desirable lifestyle. As described in the previous section, incorporating a long-term goal to develop intrinsic motivation is promising to solve the problem.

4 Related work

There are many studies on prototype applications that aim to influence user behavior [2, 9–11, 13, 16, 28, 37, 38, 51, 52, 72, 76]. For example, quit meters [52] provide smokers with constant feedback on how much money is wasted and how many minutes of life are lost. Carb counters [51] provide instant feedback on meal choices. Playful toothbrush [9] shows a set of virtual teeth and visualizes the cleansing effect that brushing has on them. Similar to our persuasive ambient mirrors, these applications focus on to change one target activity like unhealthy behavior. The difference between Playful toothbrush and Virtual aquarium is that the former shows the actual current situation of a user’s teeth, while the latter uses a more metaphorical, emotional, and ambient mode of feedback. From an efficacy point of view, the literal expression is not always the most engaging and natural to the user.

At the same time, from the perspective of usability and affordances, it is important that feedback can be easily associated with the behavior and its consequences. For example, UbiFit Garden [13] reflects the user’s daily exercise in the form of a flower garden, but it is not granted that the user sees a connection between the flowers and the achievements in their exercise. Playful Bottle [11] represents a growing tree that reflects the user’s water drinking activity. Growing a tree is a useful expression that users can feel empathy toward, but they might not see association between the tree and water drinking activity. The Activity-based billing system [76] motivates behavior changes with economic incentives. This is a strong incentive, but it lacks pedagogical content and may thus fail to build intrinsically sustained behavior. If economic incentives are removed, the changes in behavior are likely to be reversed.

UbiFit Garden [13] uses a mobile phone to reflect a user’s dairy exercise on flowers shown on a display of his/her mobile phone. The application is very similar to Persuasive Art, but Persuasive Art uses both positive and negative expression. Thus, it is possible to use the visual expression as a metaphor to a user’s health condition. Also, we believe that a mobile phone is not a practical place to show ambient expressions, because many people use the mobile phone for self-expression. This suggests that the placement of ambient expressions is also a very important research topic in the future.

In [18], Fogg proposed a framework to design applications based on persuasive technologies. Since the framework offers a very abstract guideline to developing persuasive applications, most persuasive applications described in this paper did not use the framework. We believe that our framework is easier to use in the practical design of persuasive applications.

Recently, environmental conservation has become one of the most important topics in persuasive applications, and there are a lot of studies that aim to encourage ecological behavior with persuasive applications [4, 15, 21, 44, 54, 55]. Midden et al. [55] showed that findings from social psychology can be useful in changing a user’s energy consumption behavior. For example, they claimed that setting goals is very important in achieving reductions. Also, comparing efforts to reduce energy consumption between people is an effective way to motivate eco-friendly behavior. However, the effects depend on the culture that the user belongs to. Midden et al. [57] evaluated EcoIsland in a Dutch context and argued that the results differ in individualistic and collective cultures. In [43], Khaled et al. developed two games that are customized for different cultures. Also, in [19], Fogg et al. showed that social networking services reflect cultural differences. Finally, in [45], Kimura et al. proposed a framework to design persuasive applications based on social incentives.

Feedback information needs to be modified according to the current stage of the user’s attitude, skills, and knowledge. The transtheoretical model proposes a process involving five stages to change a user’s undesirable behavior [73]. There are some persuasive applications that use the transtheoretical model [33, 50, 78]. He et al. [33] claimed that each stage requires different strategies to persuade people. In earlier stages, the user prefers emotional reinforcement to not to give up his current efforts. On the other hand, for the user who is near the final stage, accurate information for making a better decision through rational thinking is more suitable. However, it is not obvious which stage a user belongs. Yoshii et al. [78] proposed that the current stage could be determined through a conversation between the user and a software agent.

Persuasive ambient mirrors adopt various strategies from computer games [1, 3, 8, 36, 59]. Computer games usually use positive and negative feedback to control a user’s motivation [36]. Goal setting is also central in computer games. Recently, it has become increasingly possible to embed computer gaming experiences into daily life by using ubicomp technologies. Such games are called pervasive games, and they offer new types of user experiences by integrating real and virtual worlds [59].

A “serious game,” on the other hand, is a game designed for a primary purpose other than traditional entertainment [6]. Many serious games have been developed for solving real-world problems in education, health care, and politics. Recently, the term “gamification” has become a popular term to describe the notion of applying gaming concepts to accelerate business [24, 64, 79]. In our framework, we categorized incentives into five types. These five incentives can also be used to “gamify” business activities.

One field that could be applied more to develop persuasive technologies further is behavioral economics, which studies human decision making empirically. Various biases in decision making have been identified that could be useful in designing persuasive ambient mirrors. For example, the framing of a choice situation influences the choices made [40]. This approach was already adopted by Yamabe et al. in [76]. On the other hand, adequate feedback information can reduce biases by making users more aware of them and providing sufficient information to employ better heuristics [71].

When designing persuasive applications, we need to take into account ethics [5]. When an application has the potential to change a user’s attitude, it is very important to consider whether the application can be used from the ethical point of view. This is especially the case for applications that can change behaviors unconsciously [30]. Persuasive technologies are unproblematic when individuals use them on themselves to achieve their own goals, but considerably more problematic when used in the fields of politics and marketing, for example.

5 Future directions

Health, nutrition, energy conservation, peace—there are many places where people need help in changing their lifestyles and behaviors toward better. Each of our case studies focused on only one target activity, and the systems were installed in users’ homes. Yet, people’s time is split between different places and contexts. Technological development and societal change in countries like Japan may give rise to even more mobile, almost nomadic lifestyles. A person with such a lifestyle cannot see a tree growing in their house or fish dancing in their bathroom. Indeed, some participants in our Virtual Aquarium user study said that they wanted to take care of the fish even when they were staying elsewhere. Thus, we believe that feedback information should be available in any place, from hotels to public spaces. The visual expressions reflecting persuasive information will become a part of smart daily objects located in various places. These smart objects will be installed in any place to enhance daily environments [41]. End users might even be able to configure do-it-yourself ambient mirrors into smart daily objects according to their preferences [42]. In this case, we can use feedback information as a design material to build new smart daily objects, and these objects make it possible to persuade the user anytime, anywhere.

Such pervasive persuasive ambient mirrors can be blended into the environments of our daily life, but we also need to consider whether this is our dream or just a nightmare. Is this really a better lifestyle for the future? Pervasive ambient lifestyle feedback systems everywhere could take control of our attitudes, causing serious ethical problems. Users should have a right to control which behaviors are reflected in expressions that shape their behavior. Yet, especially marketing practitioners can be expected to try to assert as much control over such infrastructure as possible.

Especially, in Tokyo, our surrounding environment is already flooded by waves of information through advertisements, signboards, and public screens. After the Great East Japan Earthquake and resulting power outages, many of these screens were turned off temporarily, but now they are recovering. When every surface in an urban environment becomes smart, pervasive persuasive ambient mirrors are a design material for building intelligent urban surroundings.

6 Conclusion

In this paper, we presented four example systems and corresponding user studies of persuasive ambient mirrors. In each case study, we implemented a kind of mirror that reflects a user’s current behavior, while being calmly blended into the environment. Of course, a mirror is sometimes not calm to the user. The mirror is a window to a virtual world that represents a virtualized reality. Such persuasive ambient mirrors will be a key technology in harmonizing real and virtual worlds in the future.

Based on the experiences of the case studies, we developed a framework to design persuasive ambient mirrors. The most important aspect of the framework is a taxonomy of five incentive types: physical, psychological, social, economic, and ideological. These incentives form the building blocks of “gamifying” everyday experiences.