Attention
Attention lets a model decide which parts of the input matter for each output.
Instead of one fixed summary it computes a weighted blend of all positions.
Positions that are relevant get a high weight, the rest get a low one.
This removed the fixed-vector bottleneck of old seq2seq models.
It is the single idea the entire transformer architecture is built around.