Natural Experiment

A natural experiment is a situation in which some event outside the researcher's control, such as a new law, a lottery, a border, a birth date cut-off or a sudden migration, exposes one group of people to a change while a similar group is left unaffected. Because the split happens for reasons unrelated to the outcome being studied, comparing the two groups can reveal cause and effect, much as a planned experiment would. Economists use natural experiments to study questions where deliberate experiments would be impractical, unethical or too costly, such as the effects of the minimum wage, immigration, schooling or military service.

Natural experiments are a core tool of modern econometrics and of the "credibility revolution" in empirical economics from the 1990s onwards. They sit between purely observational studies, which can confuse correlation with causation, and randomized controlled trials, in which researchers assign treatment themselves. Their pioneers, David Card, Joshua Angrist and Guido Imbens, shared the 2021 Nobel Memorial Prize in Economic Sciences.

Imagine two neighbouring towns that are alike in almost every way: similar shops, similar jobs, similar people. One day a new rule applies only to one of them because the boundary of a district happens to run between them. Nobody chose which town would be affected based on how it was doing; it was an accident of the map.

If the affected town then changes in some way and its neighbour does not, it is reasonable to think the rule caused the difference. A natural experiment is simply spotting such lucky accidents in the real world and using them as if they were a planned test.

Three methods are especially common:

• Difference-in-differences. Compare how an outcome changes over time in the affected group with how it changes in an unaffected group.
• Instrumental variables. Use a factor that shifts the treatment, such as a lottery number, but has no direct link to the outcome, to isolate the causal effect.
• Regression discontinuity. Compare people just above and just below a cut-off, such as an exam score, an age limit or an income threshold, who are otherwise very similar.

The difference-in-differences estimate is:

• $\bar{Y}_{T}$ is the average outcome in the treated group, before and after the change.
• $\bar{Y}_{C}$ is the average outcome in the comparison (control) group, before and after.

For example, suppose average employment per shop rises from 20 to 21 in a region that raised its minimum wage, and falls from 20 to 19 in a neighbouring region that did not. The estimated effect is $(21 - 20) - (19 - 20) = 1 - (-1) = 2$ jobs per shop. The key assumption is that, without the policy, both regions would have followed parallel trends.

• Minimum wage. Card and Alan Krueger compared fast-food restaurants in New Jersey, which raised its minimum wage in 1992, with those in neighbouring eastern Pennsylvania, and found no fall in employment, a result that challenged the textbook prediction and sparked a long debate.
• Immigration. Card studied the 1980 Mariel boatlift, when about 125,000 Cubans arrived in Miami within months, and found little effect on the wages and unemployment of existing workers. Later re-analyses by George Borjas and others disputed parts of this finding.
• Schooling. Angrist and Krueger used the fact that compulsory schooling laws make people born early in the year leave school with slightly less education, to estimate the earnings return to schooling and thus to human capital.
• Military service. Angrist used the US Vietnam-era draft lottery to measure the effect of service on later earnings.

Natural experiments are also used in finance and macroeconomics, for example studying sudden changes in monetary policy announced between data releases, regional differences in fiscal policy spending to estimate the multiplier effect, or windfall gains such as lottery wins to study how households respond to extra income.

The main strength is credibility: when the split is genuinely as good as random, the results are much harder to dismiss than simple correlations from observational economic models. Natural experiments also use real-world settings and large populations.

The main criticisms are:

• Hidden differences. The groups may differ in ways that are not obvious, breaking the "as good as random" assumption.
• Local results. A finding from one place and time may not apply elsewhere, a problem called limited external validity.
• Narrow questions. Critics argue that the hunt for clean natural experiments can push research towards questions that happen to have one, rather than the most important questions.
• Small effects and noise. Some studies have too little data to detect realistic effects, and the results can be sensitive to how the comparison group is chosen.

Despite these limits, natural experiments have become a standard source of evidence in labour economics, public finance, development and health economics, and their findings regularly inform debate about poverty policy and taxation.

• Nobel Prize: The Prize in Economic Sciences 2021