Feed-Forward Layers in transformers

Module 05 · Transformers · 19 concepts

In depth / 15 lines

Feed-Forward Layers, in more detail

Every transformer block has two parts, and attention is only the first.

The second is a small feed-forward network applied to each position separately.

It is the same network at every position, applied independently to each.

It expands the vector to about four times its size, applies a non-linearity, and projects back.

A 4,096-dimension model expands to roughly 16,000 in the middle of this layer.

The division of labour is clean and worth remembering.

Attention moves information between tokens.

The feed-forward layer processes what each token now holds.

Without attention, no token could learn anything from any other.

Without the feed-forward layer, the model would be little more than repeated averaging.

These layers hold roughly two thirds of the parameters in a typical language model.

That makes them the main place where knowledge appears to be stored.

Research treats them as a key-value memory, with patterns retrieving learned facts.

Editing a specific fact in a model usually means editing weights here.

Mixture-of-experts replaces this layer with many copies and routes each token to a few of them.

Want the full technical article? Wikipedia ↗

Deep dive

What is underneath Feed-Forward Layers

Two matrices and a nonlinearity

\mathrm{FFN}(x) = W_2 \, \sigma\!\left( W_1 x + b_1 \right) + b_2

$W_1$ : expands, $d_{\text{model}} \times d_{ff}$ .
$W_2$ : contracts, $d_{ff} \times d_{\text{model}}$ .
$\sigma$ : the activation, originally ReLU, now usually GELU or a gated variant.

d_{ff} = 4 \, d_{\text{model}}

The expansion factor of four is a convention from the original paper that has survived almost unchanged.

It is applied per position

This is the part most often misread. The same two matrices are applied to every position independently, with no mixing between them.

\mathrm{FFN}(X)_i = \mathrm{FFN}(x_i)

Attention moves information between positions. The feed-forward layer transforms each position on its own.

That division of labour is the whole structure of a transformer block: one sublayer to look around, one to think about what was found.

Where the parameters actually are

\text{FFN: } 2 \, d_{\text{model}} \, d_{ff} = 8 \, d_{\text{model}}^2 \qquad \text{attention: } 4 \, d_{\text{model}}^2

The feed-forward layer holds twice as many parameters as the attention it follows.

Roughly two thirds of a transformer's weights are in these layers, which surprises people who think of the architecture as being about attention.

It is also why mixture-of-experts replaces this sublayer and not the attention: it is where the parameters are, so it is where sparsity pays.

The gated variant

\mathrm{SwiGLU}(x) = \left( \mathrm{Swish}(W_1 x) \odot W_3 x \right) W_2

$\odot$ : elementwise multiplication.

A third matrix produces a gate that multiplies the activation elementwise, letting the layer suppress channels rather than only scale them.

Because there are now three matrices instead of two, $d_{ff}$ is usually cut to about $\frac{8}{3} d_{\text{model}}$ to keep the parameter count equal.

What it is doing

The expand-then-contract shape gives the layer room to compute something in a wider space before compressing back.

One reading is that these layers behave like key-value memories: $W_1$ detects patterns and $W_2$ retrieves what to add when they fire.

That matches the evidence that most of a model's factual knowledge can be located in these layers rather than in attention.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1

What does the feed-forward layer do that attention does not?

Transformers

19 concepts in this module, then a graded test with every answer explained.

Study this module →