The primary metric turns the goal of a test into a single number that decides the outcome.
Choosing one, and only one, in advance protects the test from the multiple testing problem.
It also forces the team to agree on what success means before any results appear.
A good primary metric is tied to the change, sensitive enough to move during the test, and linked to long-term value.
Purchase rate per visitor is sensitive and close to value, which makes it a common choice for shops.
Clicks are very sensitive but can rise without any real benefit, such as when users click around because they are lost.
Revenue per user matters most to the business but is noisy, because a few big orders swing it.
Teams sometimes choose a less noisy measure that is known to predict the one they care about.
Rates need a clear denominator, since per visitor, per session and per order give different answers.
The denominator should match the unit of randomisation, so per user metrics suit tests assigned by user.
Averages of averages can mislead when some users are far more active than others.
A metric that is easy to game, such as time on page, can reward designs that frustrate users.
In practice, companies often keep a short list of approved primary metrics so that tests can be compared.
Other metrics are still tracked, but they explain the result rather than decide it.
The usual mistake is changing the primary metric after the test because another one looks better.