Start studying
Module 14 · Data Science in Practice Module contents
Module 14  ·  Card 2 / 12beginner

Defining Good Metrics

A good metric is a number that moves when the thing you care about changes, and is hard to move any other way.

It needs a precise definition: what is counted, over what period, and for which users.

For example, active users must say what counts as active, such as opening the app at least once in 7 days.

Good metrics are easy to understand, comparable over time and tied to a decision someone will make.

Vanity metrics, such as total sign-ups ever, only go up and say little about whether things are improving.

Learn in depth ↓
Module 14 · Data Science in Practice · 12 concepts
In depth  /  15 lines

Defining Good Metrics, in more detail

Metrics turn goals into numbers, and the choice of number shapes what a team pays attention to.

A bad metric can make a team work hard on the wrong thing.

Defining one carefully is often the most valuable part of an analysis.

A good definition states the numerator, the denominator, the time window and any exclusions.

For example, conversion rate might be orders divided by unique visitors per day, excluding staff and bots.

Rates and averages per user are usually more informative than raw totals, which grow with size alone.

A metric should be sensitive enough to move when things change and stable enough not to swing on noise.

It should be hard to game, or it will improve without the goal improving.

Leading metrics move early and predict later results, such as trial sign-ups predicting revenue.

Lagging metrics, such as yearly revenue, confirm results but arrive too late to steer by.

Counter metrics, tracked alongside, catch harm that the main metric would miss.

Changing a metric's definition breaks comparisons with the past, so changes should be rare and recorded.

Writing definitions in one shared place stops teams calculating the same metric in different ways.

In practice, a metric that no decision depends on is usually not worth tracking.

The common mistake is choosing whatever is easy to count rather than what reflects the goal.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
A product team reports total sign-ups since launch, which rises every month. Why is this a weak metric?
Data Science in Practice

12 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close