Multi-armed bandits take their name from a gambler choosing between slot machines with unknown payouts.
The question is how to win the most while still learning which machine pays best.
In testing, each version is an arm, and each visitor is a pull.
A fixed A/B test explores evenly until the end, then exploits the winner.
A bandit mixes the 2 continuously, sending more traffic to versions that look better.
Epsilon-greedy sends a small fixed share of traffic to random versions and the rest to the current leader.
Thompson sampling picks each version with the probability that it is the best, given the data so far.
As evidence builds, the leader gets most of the traffic, and weak versions are starved.
The total cost of showing weak versions, called regret, is lower than in a fixed split.
The trade-off is weaker inference: the losing versions get few visitors, so their results are imprecise.
Changing traffic shares over time also mixes in any change in user behaviour, which can bias the estimates.
Bandits suit short-lived choices with many options, such as headlines, offers or banner images.
They are a poor fit for decisions that need a reliable effect size, such as a new pricing model.
In practice, some teams run a bandit for speed, then confirm the winner with a short fixed test.
The usual misconception is that a bandit is a better A/B test, when it answers a different question: what to show now.