The Nature of Long-Term Unemployment: Predictability, Heterogeneity, and Selection

Q1. What is the conceptual framework for decomposing duration dependence?

The paper develops a statistical framework that decomposes the observed change in average job-finding rates over the spell into true duration dependence — the individual-level within-spell decline — and dynamic selection, the change in pool composition as higher-job-finding workers exit first. The key insight is that dynamic selection is identified by the persistent covariance in job-finding probabilities across durations: if a worker who has a high job-finding probability early in a spell also has a high probability later, selection of such workers out of unemployment will lower the average for the remaining pool even if no individual’s rate changes. The framework shows that the hold-out R-squared of a prediction model provides a lower bound for the share of variance in outcomes that is ex ante determined, and that the cross-duration covariance of predictions identifies the persistent heterogeneity component that drives selection.

The paper further shows that combining prediction-based identification (which recovers observable plus transitory heterogeneity) with multiple-spell identification (which recovers persistent observable and unobservable heterogeneity) yields a tighter lower bound on overall heterogeneity. Observable and unobservable approaches are complementary: the multiple-spell approach identifies persistent unobservable heterogeneity that observables miss, while the observables approach captures transitory heterogeneity that changes across spells.

Q2. What is the predictive power of observable characteristics?

The baseline prediction model using standard administrative variables achieves a hold-out R-squared of 15%, measuring the share of variance in job-finding outcomes that is predictable from characteristics determined before the spell begins. This estimate is more than twice as large as a model using only basic socio-demographics (age, gender, education, marital status, citizenship, number and age of children). Prior employment history — even if available for only one or two years — is the most powerful predictor, potentially proxying for unobservable worker characteristics. Additional variables available for limited samples or years (occupation, assets, IQ, union membership) add only modest further predictive power beyond the baseline, suggesting saturation of the observable signal. The predictive power of a linear model is nearly as high as the ensemble of LASSO, gradient-boosted trees, and random forests, indicating that the gains come from data richness rather than nonlinearities exploited by machine learning algorithms.

The unobserved heterogeneity estimated using repeated unemployment spells corresponds to roughly half of the estimated observable heterogeneity. The combined lower bound on the variance that is ex ante determined is at least 19% of total variation in job-finding outcomes.

Q3. How much of duration dependence is explained by dynamic selection?

Applying the prediction model at multiple durations within the spell, the paper finds that nearly three-quarters of predictable heterogeneity is persistent over the spell of unemployment, implying that dynamic selection accounts for at least 49% of the observed aggregate decline in job-finding rates from the start to 6 months into the spell. In 2006, the aggregate 6-month job-finding rate fell from 70% at spell start to 55% at 6 months of ongoing unemployment — a 15 percentage point decline. The lower bound from persistent observable heterogeneity accounts for a decline from 70% to 62.7%, or 49% of the total observed decline. Under a proportionality assumption that unobservable selection mirrors observable selection, the paper estimates that dynamic selection can explain as much as 88% of the observed decline.

The paper also finds substantial heterogeneity in the individual-level dynamics across workers with different observable characteristics: the individual-level decline in job-finding over the spell is strongly negatively correlated with the job-finding probability at the start of the spell, meaning that workers who start with lower job-finding chances also experience stronger within-spell declines, further compressing the heterogeneity in job-finding over time.

Q4. What does the analysis imply about the proportional hazard assumption?

The paper tests and rejects the key assumption in proportional hazard models that job-finding rates decline at the same proportional rate across all workers. The individual predictions at different durations reveal significant heterogeneity in the dynamics across workers with different observable characteristics: workers do not all experience the same proportional decline with duration. This rejection is robust to corrections for sampling error and to non-parametric tests. The heterogeneity in dynamics is empirically distinguishable from the level heterogeneity in job-finding probabilities, and exists over and above what dynamic selection alone would generate.

Q5. Can compositional changes in the pool of unemployed explain LTU risk in recessions?

Despite the paper’s finding that rich observable characteristics explain substantial duration dependence over the spell, the same data reject the heterogeneity hypothesis for cyclicality: compositional changes in the observable characteristics of workers who become unemployed in recessions do not translate into higher predicted LTU risk for the average unemployed worker. The distribution of predicted job-finding risk changes over the business cycle, but the direction is not consistent with the hypothesis that recessions selectively pull in high-LTU-risk workers. Instead, unemployed workers are exposed to substantial changes in LTU risk over the business cycle that operate through within-individual declines in job-finding chances, not through composition. Recessions disproportionately hurt the job-finding prospects of workers with lower education and income, but this is a general worsening of individual prospects rather than a compositional shift in the pool.

Q6. What are the policy implications?

The policy implications differ sharply between genuine state dependence and dynamic selection as sources of LTU. If duration dependence at the individual level is large — skill atrophy, scarring, signaling stigma — early intervention is warranted to interrupt the deterioration process before it becomes irreversible. The paper’s evidence that selection is quantitatively dominant over the spell of unemployment suggests that much of the observed fall in aggregate job-finding with duration does not reflect structural deterioration of individual prospects, limiting the case for early intervention aimed at preventing skill erosion. However, the paper also finds that individual-level declines are heterogeneous and concentrated among workers with already-low job-finding chances, suggesting that targeted risk-profiling — already used by Public Employment Services in many countries including Sweden — is well motivated. The separate finding that recessions generate genuine within-individual worsening of prospects (rather than compositional shift) implies that countercyclical support is warranted on different grounds.

Key Concepts

dynamic selection

the mechanical change in the composition of the unemployment pool as workers with higher job-finding probabilities exit unemployment first, leaving an increasingly low-job-finding-probability remainder; identified in the paper as the dominant driver of the observed aggregate decline in job-finding rates with unemployment duration, accounting for at least 49% and potentially up to 88% of the observed decline.

genuine state dependence (true duration dependence)

the within-individual structural deterioration of a worker’s job-finding rate from the experience of unemployment itself — skill erosion, employer stigma signaling — as distinct from compositional selection; the paper finds this is a minor contributor to the observed aggregate duration dependence pattern in Sweden.

persistent heterogeneity

the covariance of individual job-finding probabilities across different durations (or spell cohorts); the component of overall heterogeneity that drives dynamic selection, identified empirically by the cross-duration covariance of prediction model outputs in hold-out samples.

LTU risk profiling

the practice of predicting workers’ probability of becoming long-term unemployed at spell entry using observable characteristics, to target active labor market programs; the paper provides the statistical foundation and quantifies the gains from richer administrative data relative to standard survey-based characteristics.