TracksStudyAbout
Sign in
Start studying
Module 3 · Computer Vision Module contents
Module 3  ·  Card 3 / 22beginner

Filters

A filter is the set of learned weights inside a convolution.

Early filters typically learn edges, corners and simple colour patches.

Middle layers combine those into textures and repeated motifs.

Deep layers respond to whole objects or object parts.

A single convolutional layer learns many filters in parallel, each looking for something different.

Back Next
Learn in depth ↓
Module 03 · Computer Vision · 22 concepts
In depth  /  15 lines

Filters, in more detail

A filter is the set of learned weights inside one convolution.

It is a small pattern detector, and the network decides what pattern it detects.

Before deep learning these were designed by hand, with known kernels for edges and blurs.

Now they are learned, and the learned ones consistently beat the designed ones.

A single convolutional layer holds many filters running in parallel.

64 or 128 filters in a layer is typical, each producing its own feature map.

Look at the first layer of a trained vision network and you see edges at various angles.

You also see simple colour blobs, which is remarkably consistent across models and datasets.

The second layer combines those into corners, curves and simple textures.

Deeper layers respond to recognisable parts, such as an eye or a wheel.

The deepest respond to whole objects, though rarely as cleanly as the diagrams suggest.

Nobody assigns any of this, since the hierarchy falls out of training on the task.

A filter has depth as well as width, matching the channels of its input.

So a 3 by 3 filter on a colour image is really 3 by 3 by 3 numbers plus a bias.

The number of filters in a layer becomes the number of channels for the next one.

Want the full technical article? Wikipedia ↗
Deep dive

What is underneath Filters

A filter is a pattern written as numbers

Kvertical=[−101 −202 −101]K_{\text{vertical}} = \begin{bmatrix} -1 & 0 & 1 \ -2 & 0 & 2 \ -1 & 0 & 1 \end{bmatrix}

That is the Sobel filter for vertical edges. Slid across an image it answers, at every position, how much brighter the right side is than the left.

On a flat region the positive and negative halves cancel and the output is zero. On an edge they do not, and the output is large.

Kblur=19[111 111 111],Ksharpen=[0−10 −15−1 0−10]K_{\text{blur}} = \frac{1}{9}\begin{bmatrix} 1 & 1 & 1 \ 1 & 1 & 1 \ 1 & 1 & 1 \end{bmatrix}, \qquad K_{\text{sharpen}} = \begin{bmatrix} 0 & -1 & 0 \ -1 & 5 & -1 \ 0 & -1 & 0 \end{bmatrix}

The same nine numbers blur when they average and sharpen when they subtract the surroundings from the centre. Nothing else about the operation changes.

Why the weights sum to what they sum to

∑m,nK(m,n)=1  ⟹  brightness preserved\sum_{m,n} K(m,n) = 1 \implies \text{brightness preserved}

A kernel whose weights sum to 1 leaves the average brightness of the image alone, which is what a blur or a sharpen wants.

∑m,nK(m,n)=0  ⟹  flat regions vanish\sum_{m,n} K(m,n) = 0 \implies \text{flat regions vanish}

A kernel summing to zero outputs nothing on any uniform patch, which is what every edge and texture detector wants.

Reading the sum of a learned kernel is therefore a fast way to tell whether it detects something or transmits something.

The part that changed in 2012

Before deep learning these kernels were designed by hand, and a decade of computer vision research was largely the design of better ones.

∂L∂K(m,n)=∑i,j∂L∂O(i,j) I(i+m, j+n)\frac{\partial \mathcal{L}}{\partial K(m,n)} = \sum_{i,j} \frac{\partial \mathcal{L}}{\partial O(i,j)} \, I(i+m, \, j+n)

In a convolutional network the kernel is just parameters, so the same gradient that trains a weight matrix trains the filter itself.

The network is not given edge detectors. It discovers them, and the first layer of a trained image model reliably contains something close to Sobel.

What each layer's filters end up detecting

  1. Layer one: oriented edges, single colours, and simple gradients.
  2. Layer two: corners, curves and junctions built from those edges.
  3. Middle layers: textures, repeated motifs and parts such as wheels or eyes.
  4. Late layers: whole objects, and configurations of parts.
  5. The classifier: a weighted vote over which of those appeared.

How many filters a layer has

Cout∈{32,64,128,256,512}C_{\mathrm{out}} \in \{32, 64, 128, 256, 512\}

Each output channel is one filter, so a layer with 64 output channels is looking for 64 different patterns in parallel.

Channel counts usually double as the spatial size halves, which keeps the arithmetic per layer roughly constant while the representation becomes less about where and more about what.

filter shape=K×K×Cin\text{filter shape} = K \times K \times C_{\mathrm{in}}

A filter in a later layer is not a picture. It spans every incoming channel, so it is a pattern over detected features rather than over pixels.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
How does a Vision Transformer handle an image?
Concepts these five lines lean on
Dataset
Computer Vision

22 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close