Pooling shrinks a feature map by summarising small neighbourhoods.
Max pooling keeps the largest value in each window.
Average pooling keeps the mean instead.
It reduces computation and gives slight tolerance to small shifts.
Pooling has no learnable parameters of its own.
Pooling shrinks a feature map by summarising each small neighbourhood into one number.
A 2 by 2 window with a stride of two halves the height and the width.
Max pooling keeps the largest value in the window.
The reasoning is that the strongest response is what matters, not where exactly it was.
Average pooling takes the mean, which is gentler and blurs the signal a little.
Max pooling is far more common in practice.
The saving is real, since a quarter of the values means a quarter of the downstream work.
It also grants a little tolerance to small shifts in the input.
Move an object two pixels and the pooled output is often unchanged.
That is helpful for classification and unhelpful for anything needing exact position.
Pooling has no learnable parameters at all, so it adds nothing to model size.
Global average pooling collapses each channel to a single number at the end of a network.
It replaced the large dense layers old architectures used and cut parameters dramatically.
Some modern designs drop pooling entirely in favour of strided convolution.
Segmentation networks have to undo it later, which is what the upsampling half of a U-Net does.
Max pooling takes the largest value in each window and discards the rest. A 2 by 2 window with stride 2 quarters the number of positions.
Average pooling takes the mean instead. Neither has a single weight to learn, which is why pooling layers are invisible in a parameter count and very visible in a memory profile.
The output size follows the same rule as a convolution with no padding, and the usual halves each spatial dimension.
Max pooling routes the entire gradient to the one input that won, and gives nothing to the others.
Average pooling splits it evenly across the window.
So the forward pass needs the values and the backward pass needs the indices of the winners, which is the one thing a pooling layer has to remember.
Each pooling layer doubles the reach of everything above it, so a small kernel in a late layer covers a large part of the original image.
It also quarters the work of every layer that follows, which is most of why a deep image network is affordable at all.
Small translations of the input often leave the maximum of a window unchanged, so the representation is slightly invariant to where a feature sits.
Five halvings take an image to a 7 by 7 grid, and everything finer than 32 pixels of the original has been thrown away.
That is fatal for segmentation and detection, which need to say where something is to the pixel. It is why U-Net keeps skip connections and why detectors read from several depths at once.
Modern architectures increasingly replace pooling with a strided convolution, which downsamples with learned weights instead of a fixed rule, at the cost of parameters and arithmetic.
Global average pooling collapses each channel to a single number, turning a feature map into a vector the classifier can read.
The alternative, flattening a 7 by 7 by 512 map into a dense layer, costs 25 million parameters and ties the network to one input size.
Global pooling costs none, accepts any input size, and generalises better, which is why every architecture after 2015 uses it.
One question on this card. Every option explains itself once you pick, right or wrong.
22 concepts in this module, then a graded test with every answer explained.