In-context learning is the model picking up a task from examples in the prompt alone.
No weights change, and nothing learned this way survives the end of the conversation.
The pattern is worked out during the forward pass and applied to the tokens that follow.
It emerged on its own from next-token prediction; nobody trained the model to do it.
It is the mechanism that makes few-shot prompting work at all.
In-context learning is a model picking up a task from the prompt alone.
Show it three examples of a format and it will follow that format for the fourth.
No weights change, and no training step of any kind occurs.
Everything it worked out is discarded when the context ends.
This is what separates prompting from fine-tuning in one sentence.
Fine-tuning changes the model, and prompting changes only what the model is currently looking at.
The behaviour was not designed, and it emerged from next-token prediction at scale.
Small models barely do it, and it strengthens noticeably as models grow.
The likely explanation is that the training corpus is full of patterns that continue.
A list of translated pairs continues with more translated pairs, so continuing the pattern is the task.
It is why few-shot prompting works and why examples beat descriptions so often.
It is also why the examples you choose matter more than their number.
The format of the examples is copied faithfully, mistakes included.
Because nothing persists, every call must carry its examples again and pay for them.
When the examples grow large enough to cost real money, that is the signal to fine-tune instead.
One question on this card. Every option explains itself once you pick, right or wrong.
22 concepts in this module, then a graded test with every answer explained.