Start studying
Module 11 · Causal Inference Module contents
Module 11  ·  Card 4 / 13beginner

Selection Bias

Selection bias happens when the people or cases studied are chosen in a way that is linked to the outcome.

For example, judging a training course by its graduates ignores everyone who dropped out because it was not working for them.

Hospital studies can mislead the same way, since admitted patients are sicker than people with the same condition at home.

It differs from confounding: the problem is who ends up in the data, not a hidden shared cause.

It can make a useless programme look effective, so asking who is missing from the data comes before any analysis.

Learn in depth ↓
Module 11 · Causal Inference · 13 concepts
In depth  /  15 lines

Selection Bias, in more detail

Selection bias appears when the route into a dataset depends on the outcome being studied, or on something tied to it.

It is especially hard to spot because the data looks complete, while the missing cases leave no trace.

It can create links that do not exist or hide links that do.

Survivorship is a common form, such as studying only funds still open, which hides the ones that closed after losing money.

Self-selection is another, where people who choose to take part differ from those who do not.

Attrition, where people leave a study over time, biases results if those who leave differ from those who stay.

Hospital data can make 2 unrelated conditions look linked, because having either one raises the chance of admission.

Selecting on a shared consequence like admission is one of the most common ways false links arise.

Unlike confounding, selection bias can affect even a randomised trial if many people drop out.

Analysing everyone in the group they were originally assigned to, called intention to treat, protects trials against it.

In business data, studying only active customers hides the ones who left, often the most informative group.

Weighting or modelling the selection can help when the reasons people were included are known and measured.

When the reasons are unknown, the safest step is to describe the limitation clearly.

In practice, tracing the path each case takes into the data, and who drops out at each step, reveals most selection problems.

The usual mistake is analysing whatever data happens to exist without asking who is missing and why.

Want the full technical article? Wikipedia ↗
Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
A coding course reports that 90% of its graduates found jobs. Half of those who enrolled dropped out. What is the main problem with the claim?
Causal Inference

13 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close