Supervised fine-tuning continues training a pretrained model on your own labelled examples.
Each example is an input paired with the exact output you want.
The learning rate is kept very small so existing knowledge is not destroyed.
A few hundred to a few thousand high-quality examples is often enough.
It teaches behaviour and format far more reliably than it teaches new facts.
Supervised fine-tuning continues training a pretrained model on examples you provide.
Each example is an input paired with the output you want for it.
The model has already learned language, so you are adjusting behaviour rather than teaching from scratch.
The learning rate is kept very small, often a hundredth of what pretraining used.
A large rate here overwrites general ability, which is catastrophic forgetting.
A few hundred to a few thousand clean examples is often enough.
That is a surprising number to people expecting to need millions.
It teaches format and behaviour reliably and facts unreliably.
Facts learned this way are absorbed inconsistently and cannot be cited or updated.
Use retrieval for knowledge and fine-tuning for how the model should behave.
The typical wins are a consistent output shape, a house style, or a narrow classification task.
It also lets a much smaller model match a large one on that one task.
That is where the money is, since inference cost falls by an order of magnitude.
Always compare against a well-prompted base model before concluding it worked.
Most fine-tuning projects that fail were solvable with a better prompt.
Supervised fine-tuning is next-token prediction again, run on curated pairs of a prompt and the response you want.
Nothing about the objective changes from pretraining. What changes is that the text is chosen rather than scraped, and that only part of it is scored.
The loss on the prompt tokens is zero.
The prompt is context, not output. Training the model to predict the user's question teaches it to generate questions, which is not the behaviour being bought.
In practice this is a label mask: the prompt positions are set to an ignore index and contribute no gradient.
Get the mask wrong and the model repeats the instruction back.
That failure is common enough to be worth checking directly, by printing the first batch's labels and confirming the prompt region is masked.
A template looks like "<|user|> ... <|assistant|> ... <|end|>".
Special tokens mark who is speaking, and the model learns to emit the end token to stop.
The exact template used in training must be used at inference, character for character.
A mismatched template is the single most common cause of a fine-tuned model behaving strangely in production.
The rate is small because the model is already good. A pretraining rate erases the pretrained behaviour in a few hundred steps.
A thousand excellent examples beat a hundred thousand mediocre ones.
This is the most reliably reproduced result in the area.
A small, carefully written set beats a large scraped one, and the failure mode of the large set is that the model learns its average quality.
Past about 3 epochs it starts to memorise.
The dataset is small and the model is enormous, so it will memorise rather than generalise if given the chance, and the sign is a training loss that collapses towards zero.
SFT teaches the model what a good answer looks like. It never shows it a bad one, so it has no signal about what to avoid.
That is the gap preference tuning fills. A ranked pair says this response is better than that one, which is information the maximum likelihood objective structurally cannot express.
It also does not add knowledge reliably. A fact stated in a hundred examples is learned as a phrasing pattern, and the model will produce the same confident shape for facts it does not have.
One question on this card. Every option explains itself once you pick, right or wrong.
15 concepts in this module, then a graded test with every answer explained.