Start studying
Module 3 · Computer Vision Module contents
Module 3  ·  Card 17 / 22beginner

YOLO

YOLO stands for You Only Look Once.

It divides the image into a grid and predicts boxes and classes in one forward pass.

That single pass is what makes it fast enough for real-time video.

It historically traded a little accuracy on small objects for that speed.

It is the standard choice for live detection on cameras and edge devices.

Learn in depth ↓
Module 03 · Computer Vision · 22 concepts
In depth  /  15 lines

YOLO, in more detail

YOLO stands for you only look once, which describes exactly what it changed.

Earlier detectors ran a classifier over thousands of candidate regions.

YOLO passes the whole image through the network once.

It divides the image into a grid and each cell predicts boxes and class scores.

Every prediction is made with the whole image in view, which reduces false positives on background.

One pass makes it fast enough for real-time video, which is the entire point.

Frame rates in the tens or hundreds are achievable depending on version and hardware.

The early versions traded accuracy for that speed, especially on small clustered objects.

Each cell could only claim a limited number of objects.

Later versions closed most of that gap with anchor boxes and better backbones.

The name now covers a family maintained by several different groups.

It is the default choice for cameras, robots, drones and anything on the edge.

It runs on modest hardware, which matters when there is no GPU in the field.

Training on a custom set is well-trodden and needs a few thousand labelled images.

Check the licence of the specific version before shipping it in a product.

Want the full technical article? Wikipedia ↗
Deep dive

What is underneath YOLO

Detection as one regression problem

Earlier detectors proposed regions and then classified each one, which meant running a classifier thousands of times per image.

S×S×(B⋅5+C)S \times S \times (B \cdot 5 + C)
  • SS: the grid the image is divided into.
  • BB: boxes predicted per cell.
  • CC: the number of classes.

YOLO predicts the whole thing in a single forward pass. The network outputs a tensor of that shape, and every number in it is a box coordinate, a confidence or a class score.

The original was 7 by 7 with 2 boxes and 20 classes, so the output was 7 by 7 by 30, produced in one pass rather than two thousand.

What each cell predicts

(tx,ty,tw,th,pobj)(t_x, t_y, t_w, t_h, p_{\mathrm{obj}})

The cell containing an object's centre is responsible for it, which is the assignment rule the whole loss is built on.

bx=σ(tx)+cx,by=σ(ty)+cyb_x = \sigma(t_x) + c_x, \qquad b_y = \sigma(t_y) + c_y

Position is predicted relative to the cell, squashed by a sigmoid so a cell can never claim a centre outside itself.

bw=pwetw,bh=phethb_w = p_w e^{t_w}, \qquad b_h = p_h e^{t_h}

Size is predicted as a multiplier on an anchor box pp, in log space, so the network learns a correction to a typical shape rather than an absolute size from scratch.

score=pobj⋅p(class∣object)\mathrm{score} = p_{\mathrm{obj}} \cdot p(\mathrm{class} \mid \mathrm{object})

Objectness and class are kept separate, so the model can say there is something here without yet committing to what.

The loss, and its weights

L=λcoordLbox+Lobj+λnoobjLnoobj+Lcls\mathcal{L} = \lambda_{\mathrm{coord}} \mathcal{L}_{\mathrm{box}} + \mathcal{L}_{\mathrm{obj}} + \lambda_{\mathrm{noobj}} \mathcal{L}_{\mathrm{noobj}} + \mathcal{L}_{\mathrm{cls}}
λcoord=5,λnoobj=0.5\lambda_{\mathrm{coord}} = 5, \qquad \lambda_{\mathrm{noobj}} = 0.5

Almost every cell is empty, so without those weights the gradient is dominated by cells learning to say nothing, and the model collapses to predicting no objects at all.

w,  h\sqrt{w}, \; \sqrt{h}

The original regressed the square root of width and height, so a 10 pixel error on a small box costs more than the same error on a large one.

One image, in order

  1. Resize the image to the network's fixed input size.
  2. One forward pass through the backbone, producing feature maps at several scales.
  3. Read boxes, objectness and class scores from each cell of each scale.
  4. Drop everything below the confidence floor.
  5. Run non-maximum suppression per class on what remains.

What the versions actually changed

Anchors and multiple scales came early, and they fixed the original's weakness on small objects and on grouped ones.

grid 13×13,  26×26,  52×52\text{grid } 13 \times 13, \; 26 \times 26, \; 52 \times 52

Predicting from three feature maps lets large objects be found in the coarse map and small ones in the fine map, which a single 7 by 7 grid could never do.

Recent versions dropped anchors again in favour of predicting box distances directly, and replaced the hand-tuned assignment rule with one that assigns targets by cost during training.

The constant across every version is the shape of the trade: one pass, fixed cost per image, and a few points of accuracy given up against detectors that look twice.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
What makes YOLO fast enough for live video?
Computer Vision

22 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close