Transfer learning starts from a model already trained on a huge dataset.
The early layers have already learned generic edges and textures.
You replace the final classification layer with one for your own classes.
You then fine-tune, often freezing the early layers at first.
It gets strong results from a few thousand images instead of millions.
Transfer learning starts from a model that has already learned something useful.
A network trained on a million images has learned edges, textures and shapes.
None of that is specific to its original classes, since every image has edges.
Only the last layer or two are truly task-specific.
So you keep the body of the model and replace the final classification layer.
The new layer has one output per class you actually care about.
Freezing the body first and training only the new head is the usual first step.
That is fast and needs very little data, because you are fitting few parameters.
Then you unfreeze the upper layers and fine-tune with a much smaller learning rate.
A large rate at this point would destroy the very features you came for.
Freezing the early layers is normal, since edges do not need relearning.
This gets strong results from a few thousand images rather than millions.
It also trains in minutes on one GPU rather than days on a cluster.
The closer the original task is to yours, the better it transfers.
It is the default approach for nearly all practical vision work, and the same idea underlies fine-tuning language models.
One question on this card. Every option explains itself once you pick, right or wrong.
22 concepts in this module, then a graded test with every answer explained.