Introduction

Applications of behavioral economics are an increasingly popular means to influence a range of behaviors, such as alcohol and drug use, gambling, and investments in preventative health care (Shafir 2013; Thaler and Sustain 2008). For example, both the US and UK governments have established Behavioral Insights Teams (a.k.a. the “Nudge Squads”). These teams work with policymakers to apply insights from the behavioral sciences in order to encourage and enable individuals to make decisions that improve private and social welfare. In the private sector, there has been a rapid growth of firms providing strategies grounded in the decision sciences for both private and public clients. For example, Opower, C3 Energy, and WaterSmart Software help utilities meet their efficiency goals through the use of targeted messages designed to promote reductions in residential energy or water use.

In the environmental domain, the use of normative messaging—particularly those including a social comparison—has become a popular way for practitioners to put basic behavioral principles to work. Such messages build upon Festinger’s (1954) social comparison theory, which posits that individuals validate the appropriateness of an action through comparisons to others. Studies in both social psychology and economics provide empirical evidence that interventions based on this theory provide an effective means to promote environmental conservation (see, e.g., Allcott 2011; Cialdini et al. 2006; Ferraro and Price 2013; Goldstein et al. 2008; Kurz et al. 2005; Nolan et al. 2008; Schultz et al. 2007). For example, Allcott (2011) evaluates data from 17 natural field experiments targeting more than 600,000 residential energy users and finds an approximate 1.4 to 3.3% reduction in average monthly energy consumption among households receiving a Home Energy Report that compares their energy use over the past 12 months to the energy use of a group of like neighbors.Footnote 1

Despite the apparent success of such interventions, the existing literature has focused largely on contemporaneous behavioral change and outcomes over the short-run.Footnote 2 The paucity of evidence on longer-run effects is symptomatic of the wider literature on behavioral nudges. In the health domain, for example, Bonell et al. (2011) claim that “…to date, few nudging interventions have been evaluated for their effectiveness in changing behavior in general populations and none…has been evaluated for its ability to achieve sustained change.”Footnote 3 Moreover, the literature on normative messaging in the environmental domain has provided little evidence on the channels through which targeted messages impact behavior.

From a policy perspective, uncovering the channels through which “nudges” impact behavior and identifying the persistence of their impacts are important. Before one can advance such strategies as viable options to fight climate change or promote healthier living, it is critical to understand whether and how they influence choice over the long-run. The goal of this study is to extend the analysis presented in Ferraro et al. (2011) to examine how social comparisons influence longer-run patterns of residential water use and to shed more light on the channels through which the observed treatment effects are achieved.

Our study uses data generated from a randomized control trial implemented in May 2007 by a water utility in metropolitan Atlanta. Residential households were assigned into one of four treatments: a control group, a group that received a message containing technical advice on reducing water use, a group that received both technical advice and an appeal to pro-social preferences, and a group that received the advice, the appeal, and a social comparison contrasting the household’s water use in the prior summer to that of the utility’s median residential consumer. The messaging campaign was designed to promote conservation efforts during a period of extreme drought. As reported in Ferraro and Price (2013), the technical advice message had little impact, but the appeal to pro-social preferences and the appeal augmented with a social comparison reduced water use by 2.7% and 4.8%, respectively, relative to the control group.

Following Ferraro et al. (2011), we use data on initial treatment assignment and subsequent water use to examine the longer-run impacts of this one-time nudge. We extend their earlier analysis by including four additional years of data and transforming the estimated treatment effects to control for temporal changes in the variance of use among households in the control group. Doing so allows us to explore how the longer-run effects of treatment compare to the utility’s minimum desired impact. Moreover, we follow Ferraro and Miranda (2013) and use information on movers to uncover whether the observed treatment effects arise through the adoption of new technologies in the home or the creation of new habits among residents living in the home at the time of intervention. Finally, we update Ferraro and Price’s cost-effectiveness analysis to take into account the observed persistence of our nudge.

The empirical results are striking; we find that our nudge has a surprisingly persistent effect. While the estimated effect size declines by nearly 50% after 1 year, we find that it remains detectable and policy-relevant 4 years later. Moreover, if we restrict the sample to the subset of households that did not move during the entirety of the panel, the effect of treatment remains detectable in the seventh year. Moreover, we find that the total reduction in water use achieved after the 4-month period targeted by the intervention is larger than the total reduction achieved during the target period (the target period was Jun-Sept 2007).

Such persistence is notable and makes our intervention significantly more cost-effective than previously assumed. Specifically, we find that the cost per 1,000 gal saved is almost 60% lower than the figure derived by Ferraro and Price (2013) using only contemporaneous treatment effects. For policymakers, this result confirms the potential of behavioral nudges as part of comprehensive environmental policy—nudges provide a cost-effective way to reduce water use among residential households and help promote broader environmental objectives.

Exploring the channels through which our nudge affects water use, we find mixed evidence. Given that we observe an approximate 50% reduction in the estimated effect size after 1 year, our data suggest an important role for short-lived behavioral adjustments that wane rapidly.Footnote 4 However, the observed persistence of our intervention suggests a role for longer-lived adjustments to habits or physical capital. To disentangle these channels, we contrast water use in homes from which treated consumers have moved out with their counterparts in the control group. Conceptually, if treatment results in changes to the capital stock of the home, we would expect such homes to use less water than those in the control even after the originally treated customer has moved. Although we find no difference in the treatment effect of movers and non-movers immediately following our initial intervention in summer 2007, the two household types respond in a statistically different manner in 2010. Moreover, we cannot reject the null hypothesis of zero treatment effect in summer 2010 among the subset of households for which the treated customers have moved. In other words, the treatment effect disappears when the treated customers disappear. This empirical pattern suggests that the treatment effects arise through the creation of new habits or the adoption of mobile technologies, rather than changes to the capital stock of the home.

Study Design

Located in metropolitan Atlanta, the Cobb County Water System (CCWS) is an agency of the county government and distributes water to about 170,000 residential customers. The county is second largest user of Georgia’s public water supply (Fanning 2003). During a drought in 2007, CCWS implemented a targeted, residential information campaign within a randomized evaluation design. Its goal was to test the effectiveness of messages aimed at inducing voluntary reductions in water consumption during the summer months (June–September). Ferraro and Price (2013) describe the experimental design and treatments in detail. We outline the key elements here.

In late May 2007, three messages were randomly assigned to single-family, detached dwellings whose customers had lived in the home since May 2006 and who were above the 22nd percentile of use June–October 2006 (see appendix for message examples):

  1. (i)

    A technical information message, which presented customers with a double-sided “tip sheet” that explained ways in which the household could reduce its water consumption;

  2. (ii)

    A weak social norm message, which augmented the tip sheet with a personalized letter signed by a CCWS employee on official stationary that explained the drought conditions, reiterated historical use information on the customer’s bill, and, using norm-laden language, encouraged the customer to act on the enclosed tips;

  3. (iii)

    A strong social norm message, which augmented the weak social norm message with a social comparison, in which the customer’s own consumption during the previous year’s summer was compared to median county consumption during the same period, and the customer’s percentile was reported. County residents are referred to as the customer’s “neighbors.” An example of a social comparison is the following:

    Your own total consumption June to October 2006: 52,000 gal

    Your neighbors’ average (median) consumption June to October 2006: 35,000 gal

    You consumed more water than 73% of your Cobb County neighbors.

Prior to randomizing the messages, the CCWS staff determined that if a message could induce a reduction in water use of 2% during the 2007 summer watering season (June–September), the message would be deemed cost-effective. Accounting for the variance in baseline water use, this reduction corresponds to an effect size of 0.025. The small effect size reflects a common feature of behavioral nudges: they are so inexpensive that even small effect sizes are policy-relevant. In our application, the estimated cost of treatment was approximately $0.997 per household.

Each treatment group comprises approximately 11,700 households, and the control group comprises approximately 71,600 households—numbers determined via power calculations designed to detect the targeted 2% reduction with 90% power. All messages were randomized within almost 400 meter route units (small neighborhoods) and were mailed first class on the same day in late May 2007. Ferraro and Price (2013) show that randomization was effective at balancing pre-treatment water use across treatment arms (despite the large sample, differences were statistically insignificant) and that there was unlikely to have been any interference among units (i.e., violations of the Stable Unit Treatment Value Assumption, SUTVA) or treatment noncompliance.

Based on the analysis in Ferraro and Price (2013), the effect of the technical information message was statistically indistinguishable from zero and well below the policy-relevant threshold identified by the Water System (trimming 780 observations from the top and bottom 0.25 percentile increases the precision of this estimate, but it remains small and policy-irrelevant). The weak social norm message reduced water use by, on average, 990 gal (2.7% reduction; p < 0.01) and the strong social norm message reduced water use by, on average, 1,740 gal (4.8% reduction). The treatment effects for the two norm-based messages are immediately detectable in the month after treatment assignment and still detectable 4 months later.

Subsequent analysis by Ferraro et al. (2011) and Ferraro and Miranda (2013) highlights substantial differences in the long-run impacts of treatment. Whereas Ferraro et al. (2011) find that the effect of the strong social norm remains detectable in summer 2009; Ferraro and Miranda (2013) find that the effect of the weak social norm is no longer detectable by December 2007. Given that previous studies failed to detect persistent impacts in the technical information and weak social norm messages, we focus our study on the persistence of the strong social norm message. In doing so, we restrict the analysis to households initially assigned to either the strong social norm treatment or the control.

We extend previous analyses in several ways. First, we elaborate on the nature of the persistence observed in Ferraro et al. (2011), who report treatment effects in gallons only. Reporting results in gallons ignores changes in the mean and variance of water use over time in the control group (representing counterfactual water use). We believe that reporting percentage reductions and effect sizes, and how these measures compare to the CCWS’s policy-relevant threshold, is more informative. Second, we extend their analysis for four more summer watering seasons (2010–2013), which is important because of the time-varying institutional changes that were occurring between 2007 and 2010. At the end of September 2007 (the end of the period of analysis in Ferraro and Price), a complete outdoor watering ban was instituted in metropolitan Atlanta. The ban was in force until mid-June 2009, with only two small exceptions: in March 2008, the state government allowed hand watering for 25 min a day between midnight and 10 a.m. (defined as one person with one hose with a shut off nozzle) and, in April 2008, 30 days of watering between midnight and 10 a.m. for new professional landscape installations. Summer 2010 was the first full summer without an outdoor water ban.

Moreover, we extend previous analyses to think more deeply about the potential channels through which households changed their behaviors in response to the strong social norm message. In the context of water consumption, as in other environmental contexts like energy use or toxic waste generation, consumers can adjust their behaviors in cheap-to-reverse ways (e.g., shorter showers; re-use water) or costly-to-reverse ways (e.g., let the outdoor vegetation die during drought). Some behavioral adjustments require constant vigilance to maintain (e.g., only run full loads of laundry), while others can be done and forgotten about (e.g., put programmable irrigation system or dishwasher on eco-friendly setting). Consumers can also invest in technology capital that requires higher up-front fixed costs but lower variable costs (e.g., fix leaks; buy low-flow toilets or high-efficiency irrigation systems).

Such physical capital-based adjustments, like a persistent shock to water demand, would be expected to induce more persistent treatment effects. Yet some of these adjustments may be mobile and follow the treated household members (e.g., an efficient above-ground sprinkler system or washing machine). Others may be immobile and stay with the home (e.g., the purchase of a more efficient below-ground sprinkler, repaired leaks, or the installation of low-flow toilets). Thus the analysis is equivalent to asking whether treatment affects the home or the homeowner. A treatment response based on investments in immobile, physical capital might be expected to be more persistent than a response based on behavioral adjustments or investments in mobile capital.

Without observing behavior inside the households over time, we cannot identify specific mechanisms like “purchased low-flow toilets,” but we can probe the nature of the potential channels. To do so, we look both at the pattern of treatment effects over time and contrast the heterogeneous responses over time of homes from which the customers moved during the post-treatment assignment period and homes from which the customers did not move. For example, if the treatment effect observed in 2007 wanes over time, such a pattern would not be consistent with all of the effect coming from costly-to-reverse behavioral adjustments, or cheap-to-reverse behavioral adjustments that require no vigilance to maintain over time by the customer, or technology investments that lead to persistent demand shocks. Furthermore, if the treatment effects are persistent for non-movers, but disappear in homes in which the originally treated customers have moved, the pattern would be inconsistent with treatment effects being achieved through investment in immobile forms of physical capital. If investments in immobile forms of physical capital were an important driver of changes in water use, the treatment effect should depend on the home, not who lives in the home.

Results

Persistence of Impacts

The average treatment effects of the strong social norm message for each summer are presented in Table 1 (we restrict attention here to the treatment effect estimates and suppress the coefficients on pre-treatment water use variables and the 400 meter route dummies). As measured by gallons of water, the observed treatment effect in 2007 declines by 63% (p < 0.05) in 2008. However, in 2008, outdoor watering was banned and thus the counterfactual water use was much smaller than in 2007. Thus, in terms of percent reduction or effect size, the treatment effect declined by less than 50% in 2008. After 2008, the treatment effect in both absolute and relative terms remains roughly similar and can still be detected in summer 2011. We cannot, however, reject the null of zero treatment effect in the 2012 and 2013 summer seasons. We return to this issue in Revisiting the Longer-run Effects Section.

Table 1 Average treatment effect for summer (June–September) 2007–2013
Full size table

Before proceeding, it is worthwhile to note that the effect size of the impact in 2010 and 2011 (0.021 and 0.015, respectively) differs only slightly from the desired effect size of 0.025 for the original 2007 target period. Thus, although the water utility believed that the behavioral nudge would be cost-effective if it had an effect size of 0.025 in 2007, the nudge persisted in having a similar effect size four summers later. In fact, the estimated total reduction in water use achieved after the 4-month period targeted by the intervention is larger than the reduction during the target period.

Implications for Our Understanding of Behavioral Channels

The large decline in the estimated treatment effects after 2007 implies an important role for short-lived behavioral adjustments. However, the persistence of an effect also suggests the presence of longer-lived adjustments to either habits or capital stock in the home. To evaluate the relative importance of investments in capital stock and the development of new habits, we consider three sources of evidence. Although neither source by itself identifies a unique channel, when combined they offer a consistent picture.

First, we note that Ferraro and Price (2013) find the treatment effects for the two norm-based messages are immediately detectable at full strength in the month immediately after treatment assignment (June). Although it is possible that households quickly adopted new capital stock into their homes in such a rapid time frame, it seems less plausible than purely behavioral adjustments.

Second, Ferraro and Miranda (2013) detect effects only in summers, when outdoor watering forms a large part of the water budget, but not in winter, when most of the water use is indoors. This pattern suggests that the treatment effects are largely being realized through changes in outdoor watering. In Cobb County, in-ground, immobile irrigation systems are much less prevalent than in other regions, such as the arid western states in the USA (Kathy Nguyen, pers. comm.). CCWS believed that an important water saving channel was reducing improper, over-watering of lawns, and this reduction would require a change in habits. Thus, changes in outdoor watering in Cobb County seem more likely to happen through changes in watering habits (e.g., watering 1 day a week in the early morning, rather than 3 days per week during the middle of the day). Nevertheless, investments in more efficient, but mobile, irrigation systems (e.g., above-ground sprinklers) are also a potential channel.

Third, Ferraro and Miranda (2013) show that owner-occupied homes are much more responsive than renter-occupied homes.Footnote 5 In fact, they cannot reject the null hypothesis that renters do not respond to treatment. This result is relevant because owners are more likely to invest in physical capital stock for two reasons. First, Davis (2012) has shown that renters are significantly less likely to have energy efficient appliances, like clothes washers and dishwashers. This pattern is consistent with the hypothesis that when tenants pay the utility bills, landlords may buy cheap inefficient appliances. In our sample, almost all renters are directly billed (apartment buildings are not in the sample). Second, owner-occupants have a greater incentive to invest in high fixed-cost, water conservation technologies that are capitalized into the value of the home, like low-flow toilets or high-efficiency in-ground sprinkler systems. Thus, if some of the persistence of the treatment effect comes from investment in physical capital stock that is incorporated into the home, the most likely subgroup to see such investment is the owner-occupied homes.

We hypothesize that if owner-occupied homes have invested in immobile physical capital—capital that becomes part of the home—rather than through changes in habits or investments in mobile physical capital, we should see the treatment effect persist even after the customer has moved from the home. From the water utility data, we can observe whether the customer name on the bill changes at some point between treatment assignment and May 2010, the month before the summer 2010 period. We call these homes “movers.” Non-movers have the same customer name on the bill at treatment assignment and in May 2010.

In Table 2, the non-movers are the omitted group, and thus the estimated coefficient on “Strong Social Norm Treatment” is the average treatment effect on mover homes. The sum of this coefficient and the estimated coefficient on the interaction of treatment and non-movers (Treatment*Non-movers) is the average treatment effect on non-mover homes.

Table 2 Average treatment effects for movers and non-movers
Full size table

The 2007 column shows the estimated treatment effects on mover and non-mover homes before anyone has moved. The estimated coefficients and associated hypothesis tests imply that, immediately after the treatment assignment and before anyone has moved, both non-movers and movers are responsive, on average, to the treatment message, and there is no statistical difference between their mean responses (−1,525 vs −2,078 gal).

In contrast, the 2010 column shows the treatment effects on mover and non-mover homes after people have moved. The patterns in 2010 are quite different from what was observed in 2007. In mover homes, the estimated average treatment effect coefficient is positive (591 gal), and we cannot reject the null hypothesis of zero average treatment effect. We can, however, reject the null hypothesis of zero treatment effect for the non-mover homes (−726 gal). We can also reject the null hypothesis that movers and non-movers respond equally. In other words, before people move out (2007), we detect similar treatment responses in mover and non-mover homes, but after people move out of the mover homes (2010), only the non-mover homes have a detectable treatment effect, and this estimated effect is statistically different from the estimated effect of mover homes.Footnote 6

The combined weight of the three sources of evidence suggests that changes in water use as a result of receiving the strong social norm message appear to arise through the creation of new habits among treated customers (or mobile capital stock) rather than changes in the capital stock of the treated homes. This analysis provides some evidence for the conjecture advanced in Allcott and Rogers (2012), that persistence in the treatment effects from normative messaging that includes social comparisons reflects habit formation.

Revisiting the Longer-Run Effects

Using our definition, almost one in four households in the treatment and control groups (23%) had moved by summer 2013. The analysis in the previous section suggests that one reason why we may be unable to detect a treatment effect in 2013 in Table 1 is because treated homeowners have left the treated homes. Therefore, we replicate Table 1, but restrict the sample to the subset of 51,846 owner-occupied homes for which the customer name on the bill did not change between initial treatment assignment and May 2013. We acknowledge that we did not randomize treatment within this subgroup, but randomization was done within small neighborhood units and the sample remains large. To mitigate concerns about hidden biases in this subgroup analysis, we also conduct a placebo test using pre-treatment summer 2006 water use.

Table 3 reports the results. Our placebo test using summer 2006 use supports the claim that randomization was effective within this subgroup: the treatment effect is small and insignificantly different from zero. After 2006, however, we can reject the null hypothesis of zero effect for every year except 2012. Moreover, the estimated effect size is similar in 2009, 2010, 2011, and 2013. Thus, for the subgroup of homes that remain treated throughout the entire panel, we find that a treatment effect persists through 2013. After seven summers, the difference between the water use in the treatment group and the control group remains policy-relevant and statistically different from zero.

Table 3 Average treatment effect: owner-occupied, non-mover households (2007–2013)
Full size table

Implications for Our Understanding of Total Impacts and Cost-Effectiveness

Given the persistence of the nudge’s impact on water use, we update Ferraro and Price’s cost-effectiveness analysis that only looked at the first 4 months after treatment assignment. The nudge is far more cost-effective than their analysis implies. If we use the estimated treatment effects from Table 1, the nudge costs an estimated $0.235 per thousand gallons reduced, which is almost 60% lower than Ferraro and Price’s original estimate ($0.58).Footnote 7 If we make the more conservative assumption that the average treatment effect was zero in 2012 and 2013, the cost is $0.25 per thousand gallons. Had the message been sent to all homes in the experimental sample (which is all customers with a pre-treatment water history from the year earlier), the utility could have expected to reduce water consumption by 453 million gallons over the 2007–2013 period (426 million 2007–2011). For comparison, the U.S. Geological Survey estimates that the average American uses between 80–100 gal of water per day (http://ga.water.usgs.gov/edu/qa-home-percapita.html).

Conclusion

Economists have only recently begun to explore the effects of norm-based messages as a means to promote behavioral change. To date, this literature has focused largely on short-run effects. This study contributes to a growing body of work that explores whether and how such messages influence behavior over the long-run. We do so by investigating the effectiveness of social comparisons in a randomized control trial carried out in conjunction with a water utility system in the metropolitan Atlanta area.

Our analysis builds upon prior work by Ferraro et al. (2011) and Ferraro and Miranda (2013), but extends this earlier work along two important dimensions. First, we include data for a longer time horizon than the earlier work and report results in terms of effect sizes rather than absolute levels. Reporting impacts in effect sizes allows us to control for temporal variations in baseline water use among control households and explore how the long-run effects of treatment compare to the utility’s minimum desired impact. Second, we use updated data on movers and post-treatment water use to uncover whether persistent changes in water use reflect longer-lived adjustments in the habits or mobile technologies of water users, or fundamental changes in the capital stock of the home.

The empirical results are striking. We find that our nudge has a surprisingly persistent effect despite the fact that households received only a single message. Although the estimated effect size declines by approximately 50% in the year after intervention, it remains detectable and policy-relevant four years later in the overall sample and six years later in the subgroup of owner-occupied homes in which the originally treated owners had not moved. Moreover, we find that the long-run impacts of our intervention exceed those observed in the initial, 4-month target period. Adjusting for such effects, we find that our intervention is significantly more cost-effective than previously calculated. Specifically, we find that the cost per 1,000 gal saved is almost 60% lower when one accounts for the persistence of treatment ($0.24 per thousand gallons saved).

For policymakers, these results are promising and suggest a potentially important role for behavioral nudges in environmental policy—they provide a low cost way to reduce residential consumption levels. Nevertheless, water utilities may find the persistence of the nudges undesirable if they are intended to have only a short-run effect on demand during environmental emergencies like a drought. Given that many utilities are regulated, a structural reduction in demand may force them to raise prices, which could subsequently anger customers who had voluntarily changed their water use for the public (environmental) good as a result of the normative messaging campaign.Footnote 8

Turning to the channels through which our strong social norm message affects water use, we find mixed evidence. The rapid decline in the estimated treatment effects suggests an important role for short-lived behavioral adjustments. However, the observed persistence of our intervention suggests the presence of longer-lived adjustments to either habits or capital stock in the home. Using data on movers, we find that changes in use appear to arise through the creation of new habits (or mobile capital stock) among the treated customers rather than changes in the capital stock of the treated homes. Future work should push this analysis further by tracking treated customers as they move from one home to another. Whether the patterns of effects we find in Georgia would be found in other contexts is unknown. For example, one might expect that customers in the arid West, where households rely upon in-ground sprinklers, might have more opportunities to adjust physical capital stocks that are incorporated into the home. Likewise, more studies are needed to determine if the patterns we observe are also observed in other domains in which normative messaging has been shown to have short-run effects, like energy use and recycling. Finally, we see great promise in studies that elucidate the welfare implications of behavioral nudges—a topic which is currently absent in the literature but one for which economists are well placed to address.