The model's entire job is to predict what token comes next.
It outputs a probability distribution across the whole vocabulary.
One token is chosen, appended to the text, and the process repeats.
That loop is all there is to text generation.
Everything the model appears to do is built on this one operation.
The model has exactly one operation: given some text, predict what comes next.
It outputs a probability for every token in its vocabulary.
One of those is chosen, appended to the text, and the whole thing is run again.
That loop, repeated, is everything you have ever seen a language model do.
Answering a question is predicting the tokens that follow the question.
Writing code is predicting the tokens that follow the description of the code.
There is no plan for the sentence, and each token is chosen with no commitment to the next.
Coherence emerges because every choice conditions the ones after it.
The model sees its own output as input, so a mistake is now part of the context.
That is why an early wrong turn tends to be confidently continued rather than corrected.
Training is the same task at scale, on text where the true next token is known.
The loss is how much probability the model gave to the token that actually came next.
This one objective is enough to force the model to learn grammar, facts and reasoning.
It is also why models produce confident wrong answers, since plausible text is the target.
Understanding this loop explains most surprising model behaviour.
One question on this card. Every option explains itself once you pick, right or wrong.
22 concepts in this module, then a graded test with every answer explained.