Nonresponse Bias in Household Inflation Expectations Surveys

Layer 1: Overview

Research question and motivation: Inflation expectations measured from household surveys are central inputs to monetary policy, but roughly half of respondents to the RBNZ Household Inflation Expectations survey decline to answer the quantitative inflation-expectations question. Because these item non-responses are not random across demographic groups, aggregate and subgroup measures derived only from those who answer can be systematically biased. The paper quantifies that non-response bias and proposes a simple, operational method to correct aggregate and subgroup inflation-expectation indices and disagreement measures.

Data and strategy: Micro-data from the RBNZ Household Inflation Expectations survey, quarterly, achieving about 1,000 household responses per wave, covering 1998Q2 to 2022Q4 with 89,834 individual responses treated as repeated cross-sections. The focal question asks the expected annual rate of inflation/deflation over the next 12 months. The survey switched from telephone to online mode starting 2018Q3. Outliers are removed using a 1.5xIQR rule (excluding 4,535 observations in the baseline). The empirical approach has three steps: (1) Probit models of the probability of responding on demographics (gender, age, region, ethnicity, income, employment) plus macro controls (lagged inflation and its square, a year trend, seasonal dummies, an online-mode dummy); (2) a Heckman sample selection model (selection equation = the baseline Probit extended with online-mode interactions; outcome equation = inflation-expectation bias regression) with four exclusion restrictions dropped from the outcome equation (region, employment, year trend, lagged inflation squared); (3) a regression-on-quarter-dummies index that adds the inverse Mills ratio to deliver bias-adjusted average and dispersion series. Estimates use survey weights, extending Heckman estimators to weighted form.

Main quantitative findings: Item non-responses average about 44% over the full sample, falling to about 24% after the move to online mode. Non-responses artificially raise average one-year-ahead inflation expectations by about 0.3 percentage points; the average selection adjustment is -0.288 over the full sample, ranging from -0.385 (2018Q1) to -0.138 (2022Q3). Females are about 20% less likely to respond than men; older, employed, higher-income individuals respond more; Maori and Pacific Islanders respond less. Online mode raises response probability by about 33%. Response rates rise non-linearly with lagged inflation: moving from 2% to 7% raises average response probability by about 12%, while it barely changes over the 0-4% range, with the slope turning steeply positive in the 5-7% range. There is a downward trend in response of about 1% more item non-response per year. The online switch narrowed the female-male response gap from 24.4% (telephone) to 5.5% (online) and rendered most ethnicity gaps insignificant. In the bias (outcome) regressions without selection (weighted), respondents over 25 show bias more than 0.23 pp above the under-25 base; Pacific Islanders 0.34 pp, Maori 0.15 pp, Asians 0.12 pp above the base ethnic group. After the Heckman correction, gender, ethnicity, and income differences become insignificant or shrink substantially, while age effects strengthen (older respondents over-predict; under the two-step estimator, bias for those over 35 is more than double the no-selection estimate). The online dummy in the outcome equation lowers predicted expectations by more than 2.27 pp (interpreted cautiously, as it also captures large 2020Q3-onward negative biases).

Implications: Survey weights correct unit non-response but not item non-response, so published aggregates overstate expectations by ~0.3 pp. The correction lowers all subgroup means, decreases cross-subgroup disagreement for gender/income/ethnicity (increases it across age), and generally decreases within-subgroup dispersion. Correcting also makes the household-vs-professional-forecaster intercept gap statistically insignificant. Policy: online survey modes and inclusive, layered communication (especially during high-inflation periods of greater public attention) can reduce measurement error.

Layer 2: Deep Dive

What is the identification strategy and what are the main threats to it?

Identification rests on a Heckman sample selection model. A Probit selection equation models the probability of answering the inflation-expectations question; its predicted probabilities yield the inverse Mills ratio, added to the outcome (bias) regression to correct for selection-as-omitted-variable bias. Identification is sharpened by exclusion restrictions: four variables (region, employment status, year trend, lagged inflation squared) enter the selection equation but are dropped from the outcome equation. The authors justify these because region and employment were found statistically insignificant in the outcome equation, and year trend and lagged inflation squared induced collinearity/variance inflation. The selection equation also includes online-mode interaction terms to better identify heterogeneity in response rates. Threats: the validity of the exclusion restrictions (the assumption that these variables affect participation but not the level of expectations bias) and the known sensitivity of the full-information ML Heckman estimator to collinearity; the authors address the latter by also reporting the two-step estimator.

What are the main mechanisms and how are they distinguished empirically?

Two mechanisms drive non-response. First, demographic propensity: young, female, low-income, and minority-ethnicity (Maori, Pacific Islander, Asian) respondents are less likely to answer, documented via Probit average partial effects. Second, state dependence on the inflation environment: response rates rise non-linearly when lagged inflation moves away from the target range (steeply positive slope at 5-7%), consistent with a ‘rational inattention’ interpretation where agents notice inflation only when it becomes salient, and with the finding that inflation uncertainty co-moves with the inflation level (Binder, 2017). The authors also test whether non-response reflects lack of understanding using a 2018Q3-2021Q4 sub-question: only 5% of respondents indicated not understanding inflation, so 81% of non-responses are not due to lack of understanding, pointing instead to factors like cultural norms/uncertainty rather than literacy.

What heterogeneity is documented?

Response heterogeneity: females respond ~20% less than males; response probability rises with age; Maori and Pacific Islanders respond markedly less; higher income and employment raise response; households with dependent children and non-freehold owners respond less; being the main grocery shopper slightly lowers response. Bias heterogeneity before correction: age, ethnicity (Pacific Islanders 0.34 pp, Maori 0.15 pp, Asian 0.12 pp), and income show differences. After Heckman correction, gender, ethnicity, and income differences become insignificant or shrink substantially, while age effects strengthen (older respondents over-predict inflation, with an upward-sloping age profile). Online mode reduces demographic gaps: the female-male response gap fell from 24.4% to 5.5%, and most ethnicity gaps became insignificant online.

What robustness checks are run?

(1) Four Probit specifications with progressively richer covariates (occupation, grocery shopping, dependent children, home ownership) across sub-periods, with baseline effects stable. (2) Two Heckman estimators, two-step and ML, mostly consistent (the main divergence is gender, insignificant under two-step). (3) Comparison against random imputation, which reproduces the distorted no-selection picture. (4) Six outlier-detection rules (fixed -2/15 interval, 1.5xIQR, 3xIQR, hybrid IQR, top/bottom 5% by quarter, top/bottom 5% overall): Probit estimates are insensitive to the outlier definition. (5) A separate Probit on outlier responses shows similar demographic patterns (low-income young minority females give more outlier responses) but with differing magnitudes and trend/inflation effects, indicating outlier responses and non-responses are related but distinct. (6) An Appendix-E forward-looking Phillips curve exercise where adjusted subgroup expectations are always preferred to unadjusted.

How does this paper relate to and differ from closely related prior work?

It builds on the heterogeneity-of-expectations literature (Bruine de Bruin et al. 2010; Pfajfar and Santoro 2010; Malmendier and Nagel 2016; D’Acunto et al. 2023) documenting demographic differences in expectations, and on studies finding non-response from young/female/low-income groups (Blanchflower and MacCoille 2009; Leung 2009). Its distinctive contribution is showing that part of the observed gender/ethnicity/income differences in expectations is an artifact of non-response (selection) rather than true belief differences, and proposing an operational correction. Unlike imputation methods (e.g., the US Michigan Survey’s distribution-based imputation), the Heckman approach accounts for the socio-demographic composition of responders. Unlike methods requiring randomized incentives or special survey-design features (McGovern et al. 2018; Comerford 2023), it works on long-running repeated cross-sections lacking such features. It differs from attrition-focused work (Burgi 2023) by addressing item non-response in repeated cross-sections rather than panel attrition.

What are the policy implications and their scope conditions?

First, because survey weights correct only unit non-response, published aggregates overstate expectations by ~0.3 pp; central banks should apply an item-non-response correction. Second, response engagement rises when inflation deviates from target, so central banks could leverage high-inflation periods of elevated public attention for broader communication beyond financial-market audiences, using layered messaging. Third, moving surveys online substantially reduces non-response bias and improves representativeness, but requires ensuring digital accessibility to avoid new selection bias. Scope conditions: the non-linear inflation-response relationship is based on few episodes of out-of-range inflation, possibly confounded by Covid/recessions, so it should be interpreted with caution; the large online-mode coefficient on expectations also captures the post-2020Q3 negative biases from sluggish expectation adjustment; and RBNZ owns the survey and could change methodology accordingly.

How is the adjusted index constructed operationally, and why is it attractive?

Average expectations are obtained by regressing micro inflation-expectations on quarter dummies (WLS); adding the inverse Mills ratio from the baseline Probit as an extra regressor yields the bias-adjusted average. Subgroup indices interact subgroup dummies with time dummies; an adjusted disagreement (dispersion) measure replaces the dependent variable with squared deviations from the quarterly mean. The approach is attractive operationally because updating each quarter only requires a new inverse Mills ratio from the pre-fitted, relatively stable Probit model, so the adjustment is unlikely to undergo severe revisions.

What does the comparison with professional forecasters show?

Regressing one-year-ahead Survey of Professional Forecasters expectations on household expectations, the unadjusted household series gives a negative, significant intercept (-0.294, confirming households’ upward divergence), but using the adjusted household average makes the intercept insignificant (-0.019), suggesting the household-professional gap is partly a non-response artifact. The slope remains below one (0.759 unadjusted, 0.740 adjusted), consistent with Carroll (2003), so household expectations still do not scale one-to-one with professional forecasters.

Key Concepts