The law of large numbers provides the mathematical foundation connecting theoretical probability to empirical observation.
It guarantees that the average result obtained from a large number of independent trials converges to the true expected value.
As the count of trials increases, the random variations inherent in individual outcomes cancel each other out in the aggregate.
The theorem exists in two distinct mathematical formulations: the weak law and the strong law of large numbers.
The weak law of large numbers states that the probability of the sample average differing from the expectation by any small epsilon approaches zero.
The strong law states that the sample average converges almost surely to the expected value as the number of trials tends to infinity.
Both laws require that individual observations are independent and identically distributed with a finite mathematical expectation.
If the underlying distribution has an infinite mean, like the Cauchy distribution, the law of large numbers fails completely.
The law explains why commercial casinos and insurance underwriters maintain profitable operations despite huge uncertainty on individual policies.
While a single gambler may win a jackpot, millions of independent wagers ensure that the casino net margin matches its mathematical edge.
In machine learning, the law justifies estimating model loss functions by averaging errors over finite mini-batches of training data.
The law does not state that past deviations will be compensated for by future opposite deviations to restore balance.
If a coin lands heads ten times in a row, future tosses do not favour tails: rather, the early ten heads are diluted by thousands of future tosses.
Dilution, rather than compensation, is the mechanism that drives empirical convergence toward the true expectation.
Analysts must remember that the law of large numbers describes long-run aggregate behaviour and provides zero guarantees for individual trials.