The Effect of Omitted Variables on the Sign of Regression Coefficients

Masten and Poirier demonstrate a previously unrecognized asymmetry in the coefficient stability literature: depending on how omitted variable bias is measured, it can be substantially easier for omitted variables to flip a regression coefficient’s sign than to drive it to zero. The paper focuses specifically on Oster (2019b), a widely used robustness framework with approximately 5,500 Google Scholar citations as of December 2025, and shows that Oster’s sensitivity parameter δ — commonly interpreted as the ratio of selection on unobservables to selection on observables — exhibits a structural problem when used to assess sign robustness.

The core theoretical result (Theorem 2) is that, in Oster’s sensitivity analysis, the sign change breakdown point is bounded above by 1 for any value of R²_long. Since researchers typically treat |δ| = 1 as the cutoff for a robust result, this implies that no empirical result is robust to sign changes under Oster’s framework, even when the explain away breakdown point is far larger than 1. The mechanism is a vertical asymptote in the identified set for βlong that occurs precisely at δ = 1, arising from near multicollinearity between the treatment X and the covariates. At this asymptote, the bias-adjusted estimand becomes discontinuous: βlong can jump from a positive to a negative value as δ crosses 1, even when δ is changed by a negligible amount.

The paper illustrates this with the bias-adjusted estimand formula. Under Oster’s Proposition 1 (which requires δ = 1 plus an auxiliary proportionality assumption), the point estimate for the social capital application is 0.532. But if δ = 1 without the auxiliary assumption, the identified set becomes {−0.0855, 1.8947}. For δ = 0.99, the identified set includes {−18.66, −0.0868, 1.736}. The baseline OLS estimate is 0.17, and the explain away breakdown point (correct) is −32.0, while the sign change breakdown point is only 0.586 — well below the conventional robustness threshold of 1.

The authors propose a modified robustness measure that adds Assumption A5: an explicit bound M on the magnitude of omitted variable bias (|βlong − βmed| ≤ M). Under this restriction, the sign change breakdown point can exceed 1, making robust sign conclusions possible. The choice of M requires substantive justification by the researcher.

Two meta-analyses covering 58 empirical papers document the practical extent of the problem. For papers published in top-five journals from 2019–2021 that cite Oster (2019), the median explain away breakdown point is 2.65, while the median sign change breakdown point (with M = 10|β̂med|) is 1.15 and without the M restriction is 0.96. At the 90th percentile, the explain away point is 13.22, while the sign change point (M = 10|β̂med|) is only 1.66. Across both meta-analytic samples, more than 50% of regressions require that the sign of βlong must be assumed a priori in order to interpret the explain away breakdown point as evidence of sign robustness.

Scope conditions: The results apply specifically to Oster’s linear regression coefficient stability framework under the assumption of exogenous controls (cov(W1, W2) = 0, Assumption A4). The authors note this exogeneity assumption is strong in many applications. The paper does not claim the results extend to other sensitivity analysis frameworks (e.g., Cinelli and Hazlett 2020). The methods are implemented in the companion Stata module regsensitivity.

Q: What is the central finding of the paper?

A: The sign change breakdown point for Oster’s δ is bounded above by 1 (Theorem 2), regardless of how large the explain away breakdown point is. Since |δ| = 1 is the conventional robustness threshold, this implies that, under Oster’s framework, no result is ever robust to a sign change. The explain away breakdown point can simultaneously be very large — e.g., −32.0 in the social capital application — while the sign change breakdown point is only 0.586.

Q: What are the two kinds of breakdown points the paper distinguishes?

A: The explain away breakdown point answers: what is the smallest |δ| required for the data to be consistent with a zero causal effect? The sign change breakdown point answers: what is the smallest |δ| required for the data to be consistent with a causal effect of opposite sign? These two quantities are often equal but are not generally equivalent, and the sign change breakdown point can be strictly smaller than the explain away breakdown point.

Q: What is the mechanism behind the sign change breakdown point being bounded above by 1?

A: The identified set for βlong has a vertical asymptote precisely at δ = 1, arising because the sensitivity analysis allows treatment X and the covariates (W1, W2) to approach near multicollinearity. Near this asymptote, omitted variable bias can be arbitrarily large while δ remains close to 1. This discontinuity allows the bias-adjusted estimand to jump across zero — changing sign — even as δ is changed by an infinitesimal amount near 1.

Q: How sensitive is Oster’s bias-adjusted point estimator near δ = 1?

A: Extremely sensitive. In the social capital application, Oster’s Proposition 1 formula (which assumes δ = 1 with the auxiliary proportionality condition) yields an estimate of 0.532. But without the auxiliary assumption, at δ = 1 the identified set is {−0.0855, 1.8947}; at δ = 0.99 it includes {−18.66, −0.0868, 1.736}; at δ = 1.01 it includes {−0.0843, 2.133, 15.64}. These are not minor perturbations — the estimand is discontinuous in δ at the value that Oster’s formula evaluates it.

Q: What modification do the authors propose to recover sign robustness?

A: They propose adding Assumption A5, which bounds the magnitude of omitted variable bias: |βlong − βmed| ≤ M for a researcher-specified M ≥ 0. Under this restriction, the identified set BI(δ, R²_long, M) is intersected with [βmed − M, βmed + M], and it becomes possible for the sign change breakdown point to exceed 1. The practical difficulty is that M must be chosen with substantive justification, and the authors show via meta-analysis that the conventional choice M = |βmed| (equivalent to assuming the sign of βlong is already known) applies to more than 50% of regressions in their sample.

Q: What do the meta-analyses show about the gap between explain away and sign change breakdown points in practice?

A: For 34 primary regressions from top-five journal papers (2019–2021) with R²_long = 1, the median explain away breakdown point is 2.65 while the median sign change breakdown point (M = 10|β̂med|) is 1.15 and without the M restriction is 0.96. At the 90th percentile, the explain away point is 13.22 versus a sign change point (M = 10|β̂med|) of only 1.66. The second meta-analysis (141 regressions from 55 papers, 2008–2013) produces qualitatively similar results.

Q: Why does the paper flag the implicit sign assumption embedded in many applications of Oster’s method?

A: Using the explain away breakdown point as evidence of sign robustness implicitly requires that M = |βmed|, which is equivalent to constraining βlong ∈ [0, 2βmed] — that is, assuming the sign of βlong is the same as the sign of βmed. The paper shows (Table 4) that across both meta-analytic samples, more than 50% of regressions make this implicit sign assumption in order to interpret the explain away breakdown point as informative about sign robustness.

Q: What is δ, and what are its interpretive limitations?

A: δ is the ratio of (cov(X, γ′2,long W2)/var(γ′2,long W2)) to (cov(X, γ′1,long W1)/var(γ′1,long W1)), measuring the relative magnitude of selection on unobservables versus observables. As Cinelli and Hazlett (2020) show, it is a double ratio: the ratio of the treatment-unobservable association to the treatment-observable association, divided by the ratio of their outcome effects. This double-ratio structure leads to counter-intuitive behavior: a single omitted variable that is only modestly related to treatment can produce δ values far from 1 if the observable control is also only weakly related to treatment, even if the omitted variable is not strongly confounding in an absolute sense.

Q: What assumption is required for the entire sensitivity analysis framework, and how restrictive is it?

A: Assumption A4 requires that all observed covariates W1 are uncorrelated with all unobserved covariates W2 (exogenous controls). The authors note this is a strong assumption in many empirical settings. A companion paper (Diegert, Masten, and Poirier 2025a) addresses the case where controls are endogenous.

Q: What do the authors recommend as best practice?

A: They recommend two practices: (1) plotting the full estimated identified set for the coefficient of interest across a range of assumptions about omitted variables, rather than relying on a single bias-adjusted point estimate; and (2) reporting sign change breakdown points as robustness summary statistics in addition to (or instead of) explain away breakdown points. Both are implemented in the companion Stata module regsensitivity.

Explain Away Breakdown Point: The smallest value of the sensitivity parameter |δ| required for the data to be consistent with a zero causal effect (βlong = 0). This is the quantity computed by Oster’s Proposition 2 and commonly reported as “Oster’s delta.”

Sign Change Breakdown Point: The smallest value of |δ| required for the data to be consistent with a causal effect of opposite sign from the baseline estimate. The paper proves this is bounded above by 1 in Oster’s framework, regardless of the magnitude of the explain away breakdown point.

Oster’s δ: The ratio of the regression of treatment X on the omitted variable index (γ′2,long W2) to the regression of X on the observed covariate index (γ′1,long W1), measuring relative selection on unobservables versus observables. Interpreted as a double ratio: (treatment-unobservable association / treatment-observable association) ÷ (outcome effect of unobservable index / outcome effect of observable index).

Identified Set BI(δ, R²_long): The set of values of βlong consistent with the observed data and a given value of δ and R²_long. Characterized as roots of a cubic polynomial. Has a vertical asymptote at δ = 1, meaning the set can include arbitrarily large or small values of βlong as δ approaches 1.

Bias Magnitude Restriction (Assumption A5): A bound M ≥ 0 on the magnitude of omitted variable bias: |βlong − βmed| ≤ M. Adding this assumption intersects the identified set with [βmed − M, βmed + M], allowing the sign change breakdown point to potentially exceed 1 and making sign robustness conclusions possible.

Coefficient Stability Analysis: A class of empirical methods that assess omitted variable bias by comparing regression coefficients across specifications that include different sets of covariates. The intuition is that if adding observed controls substantially raises R² but barely moves the coefficient, further omitted variable bias is likely small. Formalized by Altonji, Elder, and Taber (2005) and extended by Oster (2019b).

Near Multicollinearity (in this context): The situation in which treatment X and the combined covariate vector (W1, W2) are nearly collinear. In Oster’s framework, this arises precisely at δ = 1 and produces the vertical asymptote in the identified set, making the bias-adjusted estimand discontinuous and potentially unbounded near this value.