Do The Effects of Nudges Persist? Theory and Evidence from 38 Natural Field Experiments

This paper asks why the Home Energy Report (HER) — a widely deployed social-comparison nudge that shows households how their electricity consumption compares to their neighbors — produces behavioral changes that persist long after the nudge is discontinued, while analogous nudges in other domains (charitable giving, financial savings, voter turnout, tax compliance) fade almost entirely within a year or two. The authors formalize a research design to decompose the HER’s long-run effectiveness into two channels: technology adoption (a change in the stock of energy-efficient capital in the home) and habit formation (a change in the stock of habits or skills in the resident).

The identifying strategy exploits the administrative rule that when the initial resident in an HER experiment moves out, HER mailings stop immediately — but electricity consumption in the home continues to be observed as new residents occupy it. Under three assumptions — (1) treatment assignment did not influence the initial resident’s decision to move; (2) treatment assignment did not influence the type of resident who moved in; and (3) energy-efficient technology adopted in response to the HER remained in the home after the move — the post-move HER effect identifies the fraction of the long-run treatment effect attributable to technology adoption (ATK), and the remainder identifies the fraction attributable to habit formation (ATH).

Data come from 38 natural field experiments administered by Opower between 2008 and 2013 across 21 U.S. residential energy providers, comprising 61,310,166 electricity bills for 1,810,096 homes. The mover sample, restricted to homes where the initial resident deactivated service at or after the receipt of their fourth HER, contains 5,890,855 bills for 139,908 homes. Treatment and control homes enter the mover sample at statistically indistinguishable rates and have similar baseline electricity consumption.

The main findings: the HER reduced electricity consumption by 2.1 percent in the long run (the pre-move ATE). After the initial resident moved and the HER was discontinued, 1.1 percent of the reduction persisted in the home — attributable to technology. The habit channel accounts for the remaining 1.0 percent reduction. Normalizing by the ATE, 51.4 percent (s.e. = 13.1) of the long-run effectiveness is attributable to technology adoption and 48.6 percent to habit formation. The persistence of the post-move effect is robust across alternative specifications, different HER-receipt cutoffs, balanced panels, and exclusion of low-consumption move-period homes. A falsification test using rental homes — where tenants do not typically own appliances and the technology channel is therefore shut down — yields a null post-move effect, consistent with the balanced-habits assumption.

The authors use these results to explain a broader empirical pattern: one year after discontinuation, social comparison nudges targeting compliance, charitable giving, savings, and voter turnout retain on average only 4 percent of their initial effect, while nudges targeting energy and water conservation retain 65 percent. The paper argues this divergence reflects the relative abundance of enabling technologies in conservation contexts versus their absence in compliance or voting contexts. The findings also have cost-benefit implications: ignoring HER-induced technology adoption overstates net benefits by as much as 65 percent, depending on assumed technology cost per kWh saved (ranging from $0.03 per kWh saved per Gillingham et al. 2018 to $0.12 per kWh saved per Billingsley et al. 2014).

Scope conditions: results are specific to electricity-consumption nudges in the U.S. residential sector; the technology channel identification requires that adopted equipment stays in the home after a move; the decomposition rests on a linear production function for outcomes in habits and technology.

Q: What is the Home Energy Report and how was it administered in these experiments? A: The HER is a mailed social-comparison report that contrasts a household’s electricity consumption with that of similar neighbors. In each of the 38 waves, homes were observed for a 12-month baseline, then randomly assigned to treatment (receiving HERs) or control. HERs were mailed monthly, bimonthly, or quarterly; generation ceased when the initial resident deactivated electricity service.

Q: What is the paper’s central identification strategy? A: The authors exploit a discontinuity created when the initial treated resident moves out: HER mailings stop, but the home’s electricity consumption continues to be measured as new residents move in. Under three assumptions about non-interference of treatment with moving decisions, balanced habits of subsequent residents, and stability of adopted technology, the post-move HER effect point-identifies the technology-adoption component (ATK) of the long-run average treatment effect (ATE). The habit-formation component (ATH) is then inferred as ATE minus ATK.

Q: What are the three identifying assumptions and how are they tested? A: Assumption 1 (no effect of treatment on moving rates) and Assumption 2 (balanced habits of subsequent residents) are tested with the data; treatment and control homes enter the mover sample at statistically indistinguishable rates and have similar baseline consumption, supporting Assumption 1. The rental-home falsification test supports Assumption 2: rental homes show a null post-move effect, consistent with renters having balanced habits because the technology channel is inactive in rentals. Assumption 3 (stable technology after a move) is untestable from the data; the authors note that violation of this assumption would imply the post-move effect is a lower bound on ATK, making the technology-adoption estimate conservative.

Q: What are the main quantitative estimates of the decomposition? A: The pre-move (long-run) ATE is -2.1 percent of baseline electricity consumption. The post-move effect (ATK) is -1.1 percent, and the habit-formation component (ATH) is -1.0 percent. Normalizing by the ATE, 51.4 percent (s.e. = 13.1) is attributed to technology adoption and 48.6 percent to habits.

Q: How large is the HER effect in absolute terms during the comparison period? A: During the comparison period, the HER reduced average daily electricity consumption by approximately -1.8 to -2.3 percent in the first year and -1.5 to -2.0 percent in the second year, with 95 percent confidence intervals excluding zero. In levels, these correspond to roughly -0.6 to -0.9 kWh per day — equivalent to using 2 to 4 sixty-watt incandescent bulbs for 5 fewer hours per day.

Q: How persistent is the HER effect during the move period? A: In the first year of the move period the HER continues to produce reductions of -1.7 and -1.4 percent; more than a year after the initial resident’s departure the estimated effect is -1.2 percent. All move-period estimates are statistically significant at conventional levels.

Q: How does the paper explain variation in persistence across social-comparison nudge contexts? A: One year after discontinuation, nudges targeting compliance, charitable giving, savings, and voter turnout retain on average only 4 percent of their initial effect, while nudges targeting energy or water conservation retain 65 percent on average. The paper argues the divergence reflects the relative availability of enabling technologies: households can adopt long-lived, input-efficient technologies (appliances, fixtures) to reduce energy and water use, but analogous technologies to facilitate compliance, donations, or voting are largely unavailable or absent.

Q: How does this paper’s finding about technology adoption compare to Allcott and Rogers (2014)? A: Allcott and Rogers (2014) used participation in utility-sponsored energy-efficiency programs as a proxy for technology adoption and found it explained no more than 2 percent of the HER’s long-run effectiveness. The authors reject this conclusion: their decomposition attributes 51.4 percent to technology, which is estimated precisely enough to statistically reject the 2 percent figure from Allcott and Rogers (2014). They attribute the discrepancy to the imperfect proxy used by Allcott and Rogers and low statistical power in analogous analyses.

Q: What are the cost-benefit implications of accounting for HER-induced technology adoption? A: Assuming monthly HERs for one year, a household electricity price of $0.10/kWh, and benefits accruing over two years, the baseline net benefit (ignoring technology costs) is $32.38 per household (electricity savings of $44.38 minus $12 administration cost). Using a technology cost of $0.03/kWh saved (Gillingham et al. 2018), net benefits fall to $27.14. Using $0.12/kWh saved (Billingsley et al. 2014), net benefits drop to $11.43 — a reduction of up to 65 percent from the baseline estimate. The HER still passes cost-benefit analysis but prior evaluations that ignore technology costs overstate net benefits substantially.

Q: How robust are the decomposition results to alternative sample definitions and specifications? A: The qualitative findings are stable across: alternative sets of control variables (Table A1); mover samples defined by receiving as few as 1 or as many as 5 HERs before moving (Table A2, with pre-move effects of -2.08 and post-move effects of -0.93 to -1.04 across cutoffs); balanced panels requiring fixed observation windows in each period (Table A3); and exclusion of homes showing unusually low consumption in the move period (Table A4, post-move effects of -1.19 to -1.48).

Q: What policy implications does the paper draw for nudge design? A: Policymakers seeking persistent nudge effects should target behaviors that can be augmented by readily available technologies, or pair social-comparison nudges with opportunities to adopt new technologies. In voting contexts, combining social-comparison nudges with opt-in mail-in or online ballot defaults could produce more persistent effects. In savings and charitable giving, pairing social comparisons with automatic contribution-rate defaults (as in Madrian and Shea 2001; Thaler and Benartzi 2004) is predicted to produce longer-lived effects than the nudge alone.

Q: What methodological contribution does the paper offer beyond the HER application? A: The mover-based decomposition is a generalizable research design for separating human capital (habits, skills) from physical capital (technology, infrastructure) as channels of policy effectiveness. The authors suggest it can be applied using other natural separation events — such as student graduation or employee departure — to assess the extent to which nudges build human capital in both recipients and the organizations in which they are embedded.

Technology adoption channel (ATK): The component of the HER’s long-run average treatment effect attributable to increases in the stock of energy-efficient technologies in the home — identified empirically as the post-move HER effect that persists after the treated resident departs and the HER is discontinued.

Habit formation channel (ATH): The component of the HER’s long-run treatment effect attributable to changes in the habits or skills of the resident — inferred as the residual after netting the technology component (ATK) from the total long-run effect (ATE).

Post-move effect: The estimated difference in electricity consumption between treatment and control homes after the initial resident has moved out, the HER has been discontinued, and a new resident has taken occupancy; under the paper’s identifying assumptions this equals ATK.

Balanced-habits assumption: The identifying assumption that treatment assignment did not influence the characteristics or habits of residents who subsequently moved into homes in the experimental sample, so that the habits of incoming residents are comparable across treated and control homes.

Stable-technology assumption: The identifying assumption that energy-efficient technologies adopted in response to the HER remain in the home after the initial resident moves; relaxing this assumption implies the post-move effect is a lower bound on ATK.

Home Energy Report (HER): A mailed social-comparison report that contrasts a recipient household’s electricity consumption with that of similar neighboring households; the treatment studied across all 38 experiments in this paper.

Enabling technologies: Long-lived, input-efficient capital goods (appliances, lighting, insulation) that reduce the marginal cost of conservation and thereby lock in behavioral changes induced by a nudge; their relative abundance in energy and water conservation contexts — versus their absence in voting, giving, or compliance contexts — is the paper’s proposed explanation for cross-context variation in nudge persistence.