The decoder generates an output sequence one token at a time.
Its self-attention is masked so a token cannot see the future.
That mask is what makes autoregressive generation possible.
Each generated token is fed back in to produce the next.
Without masking, the model would trivially cheat during training.
A decoder generates a sequence one token at a time.
Each token it produces becomes part of the input for the next.
That loop is called autoregression, and it is how all text generation works.
Its self-attention is masked so that a position cannot attend to later positions.
The mask sets those scores to negative infinity, which softmax turns into zero.
Without it, training would be trivial and useless, since the answer would be visible.
The model would learn to copy the next token rather than to predict it.
With the mask, training can still process a whole sequence in parallel.
Every position predicts its own next token while seeing only what precedes it.
That is a large efficiency win over generating one token at a time during training.
At inference there is no such shortcut, since the future genuinely does not exist yet.
Each new token requires another pass through the whole model.
The KV cache stores the keys and values already computed so history is not redone.
A decoder in an encoder-decoder model also has a cross-attention layer.
A decoder-only model has just the masked self-attention and the feed-forward layer.
2 questions on this card. Every option explains itself once you pick, right or wrong.
19 concepts in this module, then a graded test with every answer explained.