Do Stock Prices Move Too Much to Be Justified by Subsequent Changes in Dividends?
📄 Summarized from the full manuscript · Human-reviewed for faithfulness before publication
In brief
Shiller compares actual U.S. stock prices since 1871 with the "ex post rational price" -- the present value of the dividends a stock actually went on to pay -- and shows that under the efficient-markets model, the smoother ex post series should be the more volatile of the two, not the other way around. In the data the inequality is violated by a factor of five to thirteen: observed prices swing far more than any subsequent dividend stream can justify, and the gap cannot plausibly be closed by letting real interest rates vary instead, since that would require interest-rate swings far larger than any actually observed.
What this paper finds — and why it matters
Shiller tests the “efficient markets model” of stock prices – that the real price equals the present value of rationally expected future real dividends discounted at a constant real rate – by comparing the actual detrended real price series p to the “ex post rational price” p*, the present value of the dividends that were actually paid afterward. Because p is the market’s optimal forecast of p* under the model, and an optimal forecast’s error must be uncorrelated with the forecast itself, the model implies the variance of p cannot exceed the variance of p*; using U.S. data on the S&P Composite (1871-1979) and a modified Dow Jones Industrial Average (1928-1979), Shiller finds this and two related inequalities are violated dramatically, with the observed volatility of prices five to thirteen times too large relative to the bound implied by the actual subsequent variation in dividends. He shows the gap cannot plausibly be closed by allowing the real discount rate to vary over time instead of holding it constant: the required standard deviation of the ex ante real interest rate would have to be at least 4.4-7.4 percentage points, implying a real rate ranging as wide as -8 to +17 percent over the sample, far more volatile than observed nominal interest rates. Shiller concludes that this volatility evidence is a different and, in his view, more robust way of rejecting the simple efficient-markets model than standard tests for forecastable returns, since it does not require confining the data to short, carefully measured samples, though he acknowledges the remaining “academic” escape route – that the market feared much larger dividend movements than ever materialized – cannot be evaluated statistically because it rests on an unobservable.
Summary of a classic paper, AI-assisted and human-reviewed. See the linked original for the authoritative claims and full conditions.
Questions & answers
Q1. What is the “efficient markets model” the paper tests, and what motivates the shift from standard profitability tests to a volatility test?
Shiller defines the model as P_t = Σ_{k=0}^∞ γ^(k+1) E_t[D_{t+k}], with 0<γ<1 a constant real discount factor – real stock price equals the present value of rationally expected future real dividends. He distinguishes this from the broader efficient-markets literature, which mostly investigates whether nominal “profit opportunities” survive transaction costs; “very little of the efficient markets literature bears directly on the characteristic feature of the model considered here: that expected real returns for the aggregate stock market are constant through time (or approximately so).” His motivation for a volatility test rather than a profitability test is that the latter has “limited power” in a world where dividends are nearly constant and prices drift, since testing whether returns are forecastable in that setting is really testing for stationarity, for which short-sample power is low – so a model could fail dramatically on volatility grounds while surviving standard forecastability regressions.
Q2. What is the “ex post rational price” p*, and what is the key statistical inequality it generates?
pt is defined as the present value of the dividends actually subsequently paid, Σ γ^(k+1) d{t+k}, computed recursively from an assumed terminal value; the model implies p_t = E_t(p_t), i.e., the observed price is the optimal forecast of this ex post rational price.** Because an optimal forecast’s error u_t = p*_t - p_t must by construction be uncorrelated with the forecast p_t itself (else the forecast could be improved), and the variance of a sum of uncorrelated variables is the sum of their variances, var(p) = var(u) + var(p) ≥ var(p), so the model implies σ(p) ≤ σ(p)** (eq. 1) – the reverse of what one might naively guess, since it is the smoothed, actual-outcome series p* that must be the more volatile of the two, not the forecast p.
Q3. What do Figures 1 and 2 show, and what intuition do they convey before any formal test?
Plotting the detrended real S&P Composite price (1871-1979) and modified Dow (1928-1979) against their respective p series, Shiller finds p is “struck[ingly]” smooth and stable compared to the actual price series, because the present-value relation makes p* a long weighted moving average of dividends, and dividends “did not vary long enough or far enough” to generate major movements in p* – even the Great Depression left real dividends only 10-38 percent below trend, for a few years, which a long moving average smooths away.** This leads to the paper’s central rhetorical challenge: “Clearly the stock market decline beginning in 1929 and ending in 1932 could not be rationalized in terms of subsequent dividends;” Shiller’s goal is to show, through the persistent recurrence of such large price movements across the whole sample, that this cannot simply be dismissed as one large rational forecast error.
Q4. What do the paper’s empirical volatility comparisons (Table 2) show, numerically?
For the S&P data (1871-1979), the standard deviation of the detrended price series is σ(p) = 50.12 against σ(p) = 8.968 – more than five times larger than the inequality permits; for the modified Dow data (1928-1979), σ(p) = 355.9 against σ(p) = 26.80.** Across all three inequalities reported (on price levels, on price innovations, and on price changes), “the left-hand side of the inequality is always at least five times as great as the right-hand side, and as much as thirteen times as great,” and the lower bound of a one-sided 95 percent confidence interval for the observed price volatility remains “over five times higher than the upper bound allowed by” the dividend-volatility-implied bound – a gap Shiller argues is too large to attribute to data errors, price-index construction, or tax-law changes.
Q5. Could a constant real discount rate be the wrong assumption – might time-varying real interest rates rescue the model?
Shiller derives (eq. 17) a lower bound on the standard deviation of the ex ante real interest rate needed to reconcile the model with the observed excess volatility of price changes, given the model is modified to allow a time-varying but still rationally-expected real rate. Computing this bound, he finds the real discount rate would need a standard deviation of at least 4.36 percentage points for the S&P data and 7.36 points for the Dow data – implying, over a plus-or-minus-two-standard-deviation range around the sample mean real rate, that the real interest rate would have to have ranged from about -3.9 percent to +13.5 percent (S&P) or -8.2 percent to +17.3 percent (Dow). Shiller notes these are “very large numbers,” “much larger than the movements in nominal interest rates over the sample period,” and that they represent the most conservative (lowest) standard deviations consistent with the model, since the bound assumes the real rate’s shocks are perfectly negatively correlated with dividend shocks – the single most favorable case for the discount-rate explanation.
Q6. What ordinary regression-based test does the paper also run, and why does Shiller consider the volatility comparison more informative?
A regression of the price innovation on the lagged price level yields a significant negative coefficient (-0.1521, t = -3.218, R² = 0.089 for the S&P; -0.2421, t = -2.631, R² = 0.124 for the Dow), and a regression of the one-period holding return on the dividend-price ratio is also significant, confirming that stock returns are to some degree forecastable, contrary to the model. But Shiller argues the volatility comparisons are the more revealing and more robust diagnostic: they are “insensitive to misalignment of price and dividend series” and to changes in the composition of a price index over time, whereas the forecastability regressions could in principle have low power in any period dominated by a long, smooth upward price drift, even when the model is badly wrong – a hypothetical he illustrates with a stylized world of perfectly constant dividends, where the model is obviously false on volatility grounds but a regression test could still fail to reject it.
Q7. What two “escape routes” for saving the efficient-markets model does Shiller consider, and why is he skeptical of each?
The first escape route – attributing the price movements to changes in expected real interest rates – Shiller addresses directly with the calculation in Q5, showing the required interest-rate volatility is implausibly large relative to observed nominal rate movements. The second – that the sample standard deviation of actual dividend movements around trend understates the true uncertainty the market rightfully feared, even though those larger feared movements never materialized – he calls “academic,” since it “relies fundamentally on unobservables and cannot be evaluated statistically”; he notes skeptically that after a century of data, nothing happened that could justify fears many times larger than the actual Great Depression dividend declines, though he concedes one cannot be certain the market was wrong to have worried about events that, as it happened, did not occur.
Key terms in this paper
Definitions below follow the paper's own usage.
- Efficient markets model (as defined in this paper)
- the specific present-value model P_t = Σ γ^(k+1) E_t[D_{t+k}] with a constant real discount factor γ, under which real stock price is the mathematical expectation, conditional on all information at time t, of the present value of all future real dividends -- distinguished explicitly by Shiller from the broader efficient-markets literature concerned with the exploitability of nominal trading profits, which this paper does not address.
- Ex post rational price (p*)
- the present value of the dividends actually subsequently paid, computed (since the infinite sum cannot be observed exactly) recursively backward from an assumed terminal value; under the model the observed price p is the optimal forecast of p*, so p* can be compared with p to test the model without needing to observe the market's unobservable expectations directly.
- Variance (volatility) inequality
- the paper's central testable implication, σ(p) ≤ σ(p*), which follows purely from the statistical fact that an optimal forecast's error must be uncorrelated with the forecast itself; because moving averages smooth the series they average, the model requires the forecast p to be no more volatile than the smoother, ex post, actual-outcome series p* -- the opposite of the pattern Shiller finds in the data.