Convolution: how networks see edges

01Why it matters

A network that knows where to look

A 28 × 28 photo of a digit has 784 pixels. A fully connected layer with 100 neurons needs 78,400 weights just to look at it once, and it has to learn separately that a stroke in the top-left corner and the same stroke in the bottom-right are the same thing.

Convolutional networks fix both problems with one idea: a tiny pattern detector of 9 weights, slid over every position of the image. This lesson shows what that detector computes, why the same 9 weights work everywhere, and what happens to its output next.

02Key parts

The pieces you will see

kernel: A small grid of weights, here 3 × 3. It is the pattern the layer looks for.
feature map: The output: one number per window position, large where the image looks like the kernel's pattern.
stride: How far the window jumps between positions. Stride 2 skips every other position and roughly halves the map.
ReLU: The activation $\max(x, 0)$ : keep positive responses, turn the rest into 0.
max-pooling: Shrink the map by keeping only the largest value in each 2 × 2 block.

03See it work

Slide, multiply, add

The vertical-edge kernel weighs the right column +1, +2, +1 and the left column −1, −2, −1. Each output is the sum of the nine products under the window.

At every position the window computes the same thing, a dot product of the patch with the kernel:

\text{out}[i][j] = \sum_{a=0}^{2}\sum_{b=0}^{2} \text{image}[i + a][j + b] \cdot \text{kernel}[a][b]

Where the image is darker on the left and brighter on the right, the positive weights win and the output is large. Over a flat patch, bright or dark, the left and right columns cancel and the output is 0. So the feature map is a picture of where vertical edges are, and of which way they go.

04See it work

Other kernels, other patterns

Rotate the weights a quarter turn and the kernel finds horizontal edges instead: the top of the 7 lights up, its slanted stroke only a little.

The horizontal-edge kernel: positive where the picture gets brighter going down, negative where it gets darker.

A kernel of nine equal weights of $\tfrac19$ is an average, so it blurs; a big centre weight minus its neighbours sharpens. Classic image editors used exactly these hand-made kernels. A convolutional network starts from random ones and learns its own by backpropagation, usually ending up with edge and colour detectors in its first layer.

05Predict

Your turn: positive, negative or nothing?

Make 5 predictions: step forward and answer each question before the next step plays.

06Try it

Change the window

Try these in the visualiser below, using the buttons or your own input:

Stride 2: the window jumps two pixels at a time. Predict the feature map's size before you press play: an $n \times n$ image with a $k \times k$ kernel and stride $s$ gives $\lfloor (n - k)/s \rfloor + 1$ positions across.
Your own kernel: type picture = square, kernel = [0 1 0; 1 -4 1; 0 1 0], stride = 1. This kernel (a Laplacian) responds to edges in every direction. Which parts of the square light up?
Blur: every weight is positive, so ReLU has nothing to remove. Why?

07Check yourself

Check yourself

Question 1 of 5
A 12 × 12 image is convolved with a 3 × 3 kernel, stride 1, no padding. How big is the feature map?
Your answer comes back in Review in a few days, so it sticks.
Question 2 of 5
The vertical-edge kernel slides over a patch that is white everywhere. What does it output?
Question 3 of 5
Why does a convolution layer need so few weights compared with a fully connected layer?
Question 4 of 5
A 2 × 2 block of a ReLU output holds 0, 2.5, 1 and 0. What does max-pooling keep?
Question 5 of 5
What does max-pooling mainly buy a convolutional network?

08Explain it back

Say it in your own words

What does a convolution layer compute, and why does it need so few weights?

09Recap

What to remember

A convolution slides a small kernel over the image; each output is the dot product of a patch with the kernel.
The feature map is large where the image looks like the kernel's pattern; edge kernels respond to changes, not brightness.
The same weights at every position (weight sharing) means few parameters and a pattern found wherever it is.
Output size: $\lfloor (n - k)/s \rfloor + 1$ . ReLU keeps the positive responses; max-pooling keeps the strongest per block and shrinks the map.
Stacked layers detect patterns of patterns: edges, then corners and curves, then whole shapes.