Start studying
Module 6 · LLMs Module contents
Module 6  ·  Card 6 / 22beginner

Base vs Instruct Models

A base model has only been trained to continue text, so it answers a question with more questions.

It is a document completer, and a chat transcript is just one document shape among many.

An instruct model is that base model further trained to treat your text as a request.

That extra stage is instruction tuning followed by preference training such as RLHF.

Almost everything you use is an instruct model; base models are for further fine-tuning.

Learn in depth ↓
Module 06 · LLMs · 22 concepts
In depth  /  15 lines

Base vs Instruct Models, in more detail

Pretraining produces a base model, and a base model is a very strange thing to talk to.

It has learned one skill: given some text, produce text that plausibly follows.

Ask it what the capital of France is and a reasonable continuation is a list of quiz questions.

That is not a failure. A page containing that question very often contains more questions.

The base model is doing exactly what it was trained to do, which is why it feels useless in a chat box.

Turning it into an assistant takes at least two further stages of training.

Instruction tuning is supervised fine-tuning on thousands of instruction and response pairs.

It teaches the shape of the interaction: this text is a request, and what follows should satisfy it.

Preference training then ranks candidate responses by human judgement, using RLHF or DPO.

That second stage is where helpfulness, tone and most refusal behaviour come from.

The result is an instruct or chat model, and it is what every API and product ships.

Base models are still published, because they are the better starting point for your own fine-tuning.

Fine-tuning a chat model can fight the alignment already baked into it; a base model has none to fight.

Base models are also cleaner for research, since their behaviour is not shaped by an unpublished preference dataset.

If a model card names both a base and an instruct variant, the instruct one is nearly always what you want.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
You ask a base model a question and it replies with three more questions. Why?
LLMs

22 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close