A convolutional neural network stacks convolution, activation and pooling layers.
Early layers capture local detail and later layers capture global structure.
The spatial map shrinks while the number of channels grows.
Dense layers at the end turn the final features into a prediction.
CNNs were the architecture that made modern computer vision work.
A convolutional neural network is convolution, activation and downsampling, stacked.
The pattern repeats: convolve to find features, apply ReLU, then shrink the map.
Through that stack the spatial size falls while the channel count rises.
An image might go from 224 by 224 by 3 to 7 by 7 by 512.
That is the network trading where something is for what it is.
Early layers hold precise positions of simple things such as edges.
Deep layers hold vague positions of complicated things such as faces.
A classifier head at the end turns the final features into class scores.
Global average pooling followed by one dense layer is the modern way to do that.
The architecture builds in two assumptions about images.
The first is that a useful pattern is useful anywhere, which gives weight sharing.
The second is that nearby pixels are related, which gives small local kernels.
Those assumptions are why a CNN needs far less data than a dense network on the same task.
They are also why vision transformers, which assume less, need much more data to compete.
For most practical vision work with limited data a CNN remains the sensible starting point.
A convolutional network is a dense network with two constraints imposed on the weights, and both constraints are statements about images.
Locality says a pixel is explained by its neighbours. Sharing says a pattern is worth finding wherever it appears.
Those constraints are what make the parameter count independent of image size, and they are the entire reason images are tractable.
The usual pattern is a block of convolutions, then a downsample, repeated. Spatial size halves and channel count doubles at each stage.
That halves the activation volume per stage while keeping the arithmetic per layer roughly level, which is the trade the design is built around.
Early layers are cheap in parameters and expensive in arithmetic, because the maps are large. Late layers are the reverse.
A 3 by 3 layer from 256 to 256 channels on a 28 by 28 map is 590,000 parameters and about 925 million operations per image.
Training memory is dominated by activations rather than weights, because every map is held for the backward pass and the batch multiplies all of them.
That is why batch size is the first thing to cut when a convolutional model does not fit.
Weight sharing gives equivariance rather than invariance: shifting the input shifts the feature map, and it takes pooling or augmentation to stop caring where.
Nothing in a convolution is invariant to rotation or scale, so those have to come from data augmentation. A network trained on upright faces does not recognise upside down ones.
A vision transformer drops both constraints and learns the relationships instead, which needs far more data to reach the same place and then keeps going past it.
One question on this card. Every option explains itself once you pick, right or wrong.
22 concepts in this module, then a graded test with every answer explained.