Detection as one regression problem
Earlier detectors proposed regions and then classified each one, which meant running a classifier thousands of times per image.
- : the grid the image is divided into.
- : boxes predicted per cell.
- : the number of classes.
YOLO predicts the whole thing in a single forward pass. The network outputs a tensor of that shape, and every number in it is a box coordinate, a confidence or a class score.
The original was 7 by 7 with 2 boxes and 20 classes, so the output was 7 by 7 by 30, produced in one pass rather than two thousand.
What each cell predicts
The cell containing an object's centre is responsible for it, which is the assignment rule the whole loss is built on.
Position is predicted relative to the cell, squashed by a sigmoid so a cell can never claim a centre outside itself.
Size is predicted as a multiplier on an anchor box , in log space, so the network learns a correction to a typical shape rather than an absolute size from scratch.
Objectness and class are kept separate, so the model can say there is something here without yet committing to what.
The loss, and its weights
Almost every cell is empty, so without those weights the gradient is dominated by cells learning to say nothing, and the model collapses to predicting no objects at all.
The original regressed the square root of width and height, so a 10 pixel error on a small box costs more than the same error on a large one.
One image, in order
- Resize the image to the network's fixed input size.
- One forward pass through the backbone, producing feature maps at several scales.
- Read boxes, objectness and class scores from each cell of each scale.
- Drop everything below the confidence floor.
- Run non-maximum suppression per class on what remains.
What the versions actually changed
Anchors and multiple scales came early, and they fixed the original's weakness on small objects and on grouped ones.
Predicting from three feature maps lets large objects be found in the coarse map and small ones in the fine map, which a single 7 by 7 grid could never do.
Recent versions dropped anchors again in favour of predicting box distances directly, and replaced the hand-tuned assignment rule with one that assigns targets by cost during training.
The constant across every version is the shape of the trade: one pass, fixed cost per image, and a few points of accuracy given up against detectors that look twice.