Start studying
Module 11 · Causal Inference Module contents
Module 11  ·  Card 13 / 13intermediate

Natural Experiments

A natural experiment is a situation in which something outside the researcher's control assigns treatment almost at random.

Lotteries, sudden policy changes, borders and weather events are common sources.

For example, a visa lottery lets researchers compare winners and losers who applied in exactly the same way.

Because chance or an outside event decided the treatment, the groups are similar in ways that a choice would not produce.

The key question is always whether the assignment really was as good as random, which needs checking, not assuming.

Learn in depth ↓
Module 11 · Causal Inference · 13 concepts
In depth  /  15 lines

Natural Experiments, in more detail

Natural experiments are situations where the world assigns a treatment in a way close to random.

They give causal evidence on questions where a deliberate trial would be impossible or unethical.

The researcher's job is to find them and check that the assignment really was as good as random.

Lotteries are the cleanest source, as with visa lotteries and lotteries for school places.

Sudden policy changes that affect one group and not another create comparisons over time.

Borders can split similar people under different rules, such as neighbouring towns with different minimum wages.

Weather, timing and other outside events can shift treatment without any choice by the people affected.

A natural experiment often feeds one of the designs above, such as difference in differences or instrumental variables.

The strength of the evidence depends entirely on how the assignment happened.

Comparing characteristics measured before the event shows whether the groups were alike.

Anything else that changed at the same moment, or in the same place, can undermine the comparison.

Results apply to the people and setting of the event, which may be unusual.

Searching through many possible events and reporting the one that worked brings back the multiple testing problem.

In practice, natural experiments supply much of the best causal evidence in economics and public health.

The usual mistake is calling any before and after comparison a natural experiment, when nothing about the assignment was random.

Want the full technical article? Wikipedia ↗
Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
A visa lottery picks winners at random from all applicants. Why is comparing winners with losers a strong natural experiment?
Causal Inference

13 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close