A batch is the group of examples processed in one update.
An iteration is a single update using one batch.
An epoch is one full pass over the whole training dataset.
10,000 examples with a batch size of 100 gives 100 iterations per epoch.
Batch size affects memory use, training speed and the quality of the result.
These three words describe how the data is fed to the network, and they are easy to confuse.
A batch is the group of examples processed together in one step.
An iteration is one such step: a forward pass, a backward pass, and a weight update.
An epoch is one complete pass over the entire training set.
With 10,000 rows and a batch size of 100 an epoch is 100 iterations.
Batch size decides how much memory the forward pass needs, since every intermediate is kept.
Running out of GPU memory is nearly always solved by halving the batch size.
Larger batches give a steadier gradient and use the hardware more efficiently.
Smaller batches are noisier, and that noise often generalises slightly better.
Powers of two are conventional because they map neatly onto how GPUs are built.
Doubling the batch size usually means raising the learning rate to match.
Gradient accumulation fakes a large batch by summing gradients over several small ones.
That is how a large model is trained on a GPU that cannot hold the batch at once.
Typical training runs for tens of epochs on small data and less than one on huge data.
Large language models see most of their corpus exactly once.
A batch is how many examples one gradient is computed from. An iteration, or step, is one parameter update. An epoch is one full pass over the data.
With 50,000 examples, a batch of 128 and 10 epochs, that is 391 steps per epoch and 3,910 updates in total.
The model learns per step, not per epoch. An epoch is a bookkeeping unit, and it is the wrong unit as soon as the dataset size changes.
Doubling the batch size halves the number of updates in the same number of epochs, so a run that was tuned at batch 64 takes half as many steps at batch 128 and underfits.
The usual correction is to scale the learning rate with the batch, so that fewer, less noisy steps each travel further.
Large language models are not measured in epochs at all. They are measured in tokens seen, because most of the data is visited once and a second pass would be a different decision entirely.
The reshuffle at step one matters. Without it the model sees the same batches in the same order every epoch, and the gradient noise becomes periodic rather than random.
The final batch of an epoch is usually smaller, and a smaller batch has a noisier gradient than the rest.
With batch normalisation a final batch of size 1 fails outright, because the variance of one example is zero. Dropping the remainder is the standard fix.
Gradient accumulation goes the other way: run batches, sum the gradients, and step once, which buys a large effective batch on a small GPU.
The step is then one iteration made of forward and backward passes, and the learning rate should be set for the effective size rather than the physical one.
Loss against steps, not loss against epochs, because two runs with different batch sizes are not comparable on an epoch axis.
Wall clock time is worth the third axis. A configuration that needs fewer steps but spends longer per step has not saved anything.
One question on this card. Every option explains itself once you pick, right or wrong.
25 concepts in this module, then a graded test with every answer explained.