TracksStudyAbout
Sign in
Start studying
Module 3 · Computer Vision Module contents
Module 3  ·  Card 7 / 22beginner

Receptive Field

The receptive field of a neuron is the patch of the original image that can affect its value.

In the first layer that patch is just the filter, so three by three pixels and no more.

Stack another layer and each neuron now sees a patch of patches, so the field grows with depth.

Stride and pooling grow it faster, because each step downward covers more ground per unit.

A network cannot recognise anything larger than its receptive field, which is why depth buys context.

Back Next
Learn in depth ↓
Module 03 · Computer Vision · 22 concepts
In depth  /  15 lines

Receptive Field, in more detail

Ask what a single number deep inside a convolutional network actually depends on.

Trace it backwards through the layers and you arrive at a rectangle of the input image.

That rectangle is the receptive field, and its size decides what the neuron can possibly know about.

In the first layer it is exactly the filter: a three by three convolution sees nine pixels.

A second three by three layer sees a three by three grid of those windows, so five by five pixels.

Each further layer of the same kind adds two pixels to each side, which is slow growth.

Stride and pooling accelerate it, because one step in a downsampled map is several steps in the original.

After a stride-two pooling layer, every subsequent three by three filter covers six input pixels, not three.

This is why classic architectures alternate convolution and downsampling rather than stacking convolutions alone.

The practical rule is that a network cannot recognise a structure larger than its receptive field.

A model whose field covers forty pixels will never see a whole face in a high-resolution photograph.

Dilated convolutions insert gaps in the filter to widen the field without adding parameters or layers.

Segmentation models care about this most, since a pixel's label often depends on distant context.

Self-attention takes the opposite approach and gives every position a receptive field of the entire image.

That difference in default reach is much of why vision transformers behave unlike convolutional networks.

Want the full technical article? Wikipedia ↗
Deep dive

What is underneath Receptive Field

How much of the image one output actually sees

A single number deep in a network was computed from a patch of the original image. The size of that patch is the receptive field, and it decides what the unit can possibly detect.

rl=rl−1+(kl−1)∏i=1l−1sir_{l} = r_{l-1} + (k_l - 1) \prod_{i=1}^{l-1} s_i
  • rlr_l: the receptive field after layer ll.
  • klk_l: that layer's kernel size.
  • sis_i: the stride of each earlier layer.

Each layer adds its kernel size minus one, multiplied by every stride below it. The product is why a stride early in the network matters far more than a stride late.

r0=1r_0 = 1

The input pixel sees itself, and the recurrence builds from there.

Worked, on a small stack

  1. A 3 by 3 convolution with stride 1 gives a receptive field of 3.
  2. A second 3 by 3 gives 5, since the strides below it are all 1.
  3. A 2 by 2 pool with stride 2 gives 6, and doubles the multiplier for everything above.
  4. A third 3 by 3 now adds 2 rather than 2 by 1, giving 10.
  5. A second pool gives 12, and the multiplier becomes 4.

After five layers the field is 12 pixels wide. After a typical 50-layer network it is larger than the input, which is the intended state for a classifier.

Why 3 by 3 twice beats 5 by 5 once

two 3×3:r=5,params=2×9C2=18C2\text{two } 3 \times 3 : r = 5, \quad \text{params} = 2 \times 9 C^2 = 18 C^2
one 5×5:r=5,params=25C2\text{one } 5 \times 5 : r = 5, \quad \text{params} = 25 C^2

The same reach costs 28 percent fewer parameters, and it passes through two nonlinearities instead of one.

That single observation is the design of VGG and of almost every convolutional network since. Large kernels went away because stacking small ones is strictly better.

Effective, not theoretical

The formula gives the patch that can influence the output. It does not say that every pixel in it matters equally.

influence∼N(0,σ2)\text{influence} \sim \mathcal{N}(0, \sigma^2)

Contributions from the centre pass through many more paths than contributions from the corners, so the actual influence falls off roughly as a Gaussian.

reffective≈r/Lr_{\mathrm{effective}} \approx r \big/ \sqrt{L}

The effective receptive field grows only with the square root of depth, so it is substantially smaller than the theoretical one and stacking layers has diminishing reach.

When it is the bug

A model that cannot detect large objects usually has a receptive field smaller than they are, and no amount of training fixes an architecture that never sees them whole.

dilated:rl=rl−1+(kl−1)dl∏si\text{dilated} : r_l = r_{l-1} + (k_l - 1) d_l \prod s_i

Dilated convolutions insert gaps in the kernel, multiplying its reach by the dilation rate without adding a single parameter.

That is how segmentation networks obtain a large field while keeping the output at full resolution, since they cannot afford to pool the detail away.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
What happens to the receptive field as you stack more convolutional layers?
Concepts these five lines lean on
Self-AttentionAttentionEnsembling
Computer Vision

22 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close