Start studying
Module 14 · Data Science in Practice Module contents
Module 14  ·  Card 12 / 12intermediate

Goodhart's Law

Goodhart's law says that when a measure becomes a target, it stops being a good measure.

People find ways to move the number that do not improve the thing it was meant to track.

For example, a call centre judged on call length may rush or drop difficult callers to keep calls short.

The more pressure placed on one metric, the further it tends to drift from the real goal.

Pairing metrics, such as speed with customer satisfaction, and reviewing them regularly makes gaming harder.

Learn in depth ↓
Module 14 · Data Science in Practice · 12 concepts
In depth  /  15 lines

Goodhart's Law, in more detail

Goodhart's law is named after the economist Charles Goodhart, who described it in the context of monetary policy.

It applies anywhere numbers are used to manage people or systems.

The core idea is that pressure on a measure changes behaviour in ways that break its link to the goal.

A measure works because it moves along with the goal under normal conditions.

Once people are rewarded for the measure, they look for the cheapest ways to move it.

Those ways often do nothing for the goal, or harm it.

For example, schools judged on exam pass rates may steer weaker pupils away from sitting exams.

Websites judged on page views may split articles across many pages.

Some gaming is deliberate, but much of it is people sincerely optimising what they are told matters.

Measuring several things at once makes it harder to improve one at the expense of the goal.

Rotating or refreshing metrics stops people learning to game a single fixed number.

Keeping some measures for observation only, not as targets, preserves them as honest signals.

Qualitative checks, such as reading a sample of support tickets, catch gaming that numbers miss.

In practice, any metric tied to a reward should be paired with one that would reveal gaming.

The common misconception is that the fix is a better single metric, when every single metric can be gamed.

Want the full technical article? Wikipedia ↗
Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
A support team is rewarded for closing tickets fast. Ticket times fall sharply, but complaints rise. What is happening?
Data Science in Practice

12 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close