F1 Score in machine learning, explained in 5 lines

Module 01 · Machine Learning · 55 concepts

In depth / 15 lines

F1 Score, in more detail

F1 exists because precision and recall pull against each other.

Reporting one alone lets you hide a terrible score on the other.

F1 folds them into a single number so that cannot happen.

It uses the harmonic mean rather than the ordinary average.

The harmonic mean is dragged down hard by whichever number is smaller.

A precision of one and a recall of nearly zero averages to about a half.

The F1 of that same pair is close to zero, which is the honest answer.

So a high F1 means both numbers are genuinely decent.

It is the usual summary metric when the positive class is rare.

It treats both kinds of mistake as equally costly, which is often untrue.

When they are not, the F-beta version lets you weight one more heavily.

F2 leans toward recall and F0.5 leans toward precision.

Like its two parts, F1 depends on the threshold you picked.

Sweeping the threshold and reporting the best F1 is common and slightly optimistic.

It ignores true negatives entirely, which is usually what you want for rare events.

Want the full technical article? Wikipedia ↗

Deep dive

F_1 = 2 \cdot \frac{\mathrm{Precision} \cdot \mathrm{Recall}}{\mathrm{Precision} + \mathrm{Recall}}

That is the harmonic mean of the two, and the choice of mean is the entire content of this metric.

F_1 = \frac{2TP}{2TP + FP + FN}

Written in raw counts it is the same number, and it makes visible that $TN$ never appears: F1 ignores the true negatives entirely.

Take a model with precision 1.0 and recall 0.01: out of 100 positives it flagged one, got it right, and missed the other 99.

\text{arithmetic: } \frac{1.0 + 0.01}{2} \approx 0.5 \qquad \text{harmonic: } 2 \cdot \frac{1.0 \times 0.01}{1.0 + 0.01} \approx 0.02

The arithmetic mean calls that a mediocre model. The harmonic mean calls it nearly worthless, which it is.

The harmonic mean is dominated by the smaller of the two numbers, so it cannot be rescued by making the other one large.

That is exactly the behaviour you want from a summary of a pair where both have a trivial way to be maximised.

F_\beta = (1 + \beta^2) \cdot \frac{\mathrm{Precision} \cdot \mathrm{Recall}}{\beta^2 \cdot \mathrm{Precision} + \mathrm{Recall}}

$F_2$ weights recall higher, for screening problems where a miss is worse than a false alarm.

$F_{0.5}$ weights precision higher, for problems where acting on a false alarm is the expensive part.

$F_1$ is just $\beta = 1$ : the claim that the two errors cost the same. That is a claim about your problem, and it is usually made by accident.

Choose a threshold.
Count $TP$ , $FP$ and $FN$ at that threshold.
Compute precision and recall.
Take their harmonic mean.
Note the threshold alongside the score, because without it the number is not reproducible.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1

Why use the harmonic mean of precision and recall rather than the plain average?

Machine Learning

55 concepts in this module, then a graded test with every answer explained.