Sample size determination is done before a test, because the size needed depends on choices made in advance.
It replaces a guess about how long to run with a calculation anyone can check.
Too small, and the test cannot see the effect it was built for; too large, and traffic is wasted.
The inputs are the significance threshold, the power, the minimum detectable effect and the variance of the metric.
For a conversion rate the variance comes from the rate itself, so rates near 50% are the noisiest.
For revenue and other continuous measures, the variance is estimated from past data.
For a 2-sided test at 0.05 with 80% power, a handy rule gives about 16 times the variance divided by the effect squared, per group.
For example, a metric with a standard deviation of 10 and a target effect of 1 needs about 1,600 users per group.
Uneven splits, such as 90/10, need more users in total than 50/50 for the same precision.
Reducing noise, by capping extreme values or adjusting for each user's past behaviour, can cut the sample needed sharply.
Run time comes from dividing the total by the eligible daily traffic, then rounding up to whole weeks.
Whole weeks matter because weekend users often behave differently from weekday users.
Most experimentation tools include a calculator, but its inputs still need the team's judgement.
In practice the calculation is written into the test plan, and the test stops at the planned size.
The usual mistake is choosing the run time first, such as 1 week, and never checking whether it can detect anything.