A GRU is a simplified LSTM with two gates instead of three.
The reset gate decides how much past state to ignore.
The update gate decides how much new information to let in.
It has no separate cell state, so it has fewer parameters.
It trains faster than an LSTM and often performs about as well.
A GRU is a simplified LSTM that keeps most of the benefit for less machinery.
It has two gates instead of three and no separate cell state.
Memory lives entirely in the hidden state, which is passed along as usual.
The update gate decides how much of the old state to keep versus overwrite.
It does the job the forget and input gates split between them in an LSTM.
The reset gate decides how much of the past to ignore when proposing new content.
Setting it low lets the unit drop history and start fresh, which suits a sentence boundary.
Fewer gates means about three quarters of the parameters of an LSTM.
That trains faster and needs less data to fit, which matters on small datasets.
On most benchmarks the two perform within noise of each other.
LSTMs hold a small edge on very long sequences where the separate cell state helps.
GRUs are often preferred when data or compute is tight.
Neither is now the default for text, since transformers took that ground.
Both remain reasonable for short numeric sequences and low-power devices.
The general lesson held: a gated path is what lets a recurrent model remember.
A GRU has two gates rather than three, and no separate cell state. The hidden state is the memory.
The last line is the design. Keeping and writing are tied together by one gate, so whatever fraction of the state is replaced is exactly the fraction that was forgotten.
In an LSTM, and are independent.
An LSTM can forget without writing and write without forgetting, which gives it a strictly larger set of behaviours.
A GRU cannot. The constraint removes a gate's worth of parameters and, in practice, almost none of the capability.
That is 25 percent fewer parameters and 25 percent less arithmetic per step, which is the practical reason to choose one.
There is no output gate, so the full state is exposed at every step. A GRU cannot hold something privately the way an LSTM's cell state can.
The honest answer from a decade of comparisons is that they perform about the same, and the difference is smaller than the difference between two random seeds.
GRUs train faster and need less data, which makes them the better default on small datasets and on anything latency-bound.
LSTMs hold an edge on tasks requiring precise long-term counting or copying, where the separate cell state and the output gate genuinely buy something.
Both share the limitation that ended the era. The state at each step depends on the previous one, so neither can be parallelised across time.
Where the two are still chosen over a transformer, it is for the same reason: a fixed-size state means constant memory per token, however long the sequence runs.
One question on this card. Every option explains itself once you pick, right or wrong.
20 concepts in this module, then a graded test with every answer explained.