Instruction tuning trains a base model to follow instructions.
The data is a broad mix of tasks phrased as instructions with good responses.
It converts a raw next-token predictor into something conversational.
This is the stage that makes a model feel like an assistant.
It normally runs before any preference tuning such as RLHF or DPO.
A base model out of pretraining does not answer questions, it continues text.
Ask it a question and it may reply with more questions, because that is what documents do.
Instruction tuning is the stage that converts it into something conversational.
The training data is a broad mix of tasks phrased as instructions with good answers.
Summarise this, translate that, write code for this, answer this question.
The breadth is the point, since the model learns the general shape of following an instruction.
It is ordinary supervised fine-tuning, distinguished by what the data contains.
The effect is dramatic and out of proportion to the amount of data involved.
Tens of thousands of examples change a model that rambles into one that responds.
It usually runs before preference tuning such as RLHF or DPO.
Instruction tuning teaches the format of helpfulness, and preference tuning refines the taste.
Datasets such as FLAN and Alpaca made this reproducible outside the large labs.
Generating instruction data with a stronger model became standard practice.
That raises licensing questions, since many providers forbid training competitors on their output.
Every chat model you have used has been through this stage.
One question on this card. Every option explains itself once you pick, right or wrong.
15 concepts in this module, then a graded test with every answer explained.