Neural networks: why one neuron isn’t enough

01Why it matters

Why one neuron isn't enough

A single artificial neuron draws a straight line through its inputs and says which side a point is on. That is enough to separate two clusters, but not to tell the inside of a ring from the outside, or to compute XOR (one or the other, but not both). In 1969 this limitation stalled the field for a decade.

The fix is to stack neurons in layers. Each hidden neuron draws its own soft straight edge, and the next layer combines those edges into curves. This lesson shows the failure, then the fix.

02Key parts

The pieces you will see

neuron: A weighted sum of inputs plus a bias, passed through a squashing function: $\tanh(w_1x_1 + w_2x_2 + b)$ or a sigmoid.
perceptron: The oldest learning rule: when a point is misclassified, add it to (or subtract it from) the weights.
hidden layer: Neurons between the inputs and the output. Each one becomes a new feature the output can use.
decision boundary: Where the network's output crosses ½: the line, or curve, between the two classes.

03See it work

One neuron draws one line

Every mistake nudges the line towards the misclassified point. On separable data the mistakes eventually stop.

On data like this the perceptron always finds a separating line in a finite number of mistakes. On XOR it never does: no straight line puts the two orange corners on one side and the two blue corners on the other.

XOR: the line keeps moving forever, because no line can be right about all four corners.

04See it work

Hidden units bend the boundary

Four hidden tanh units, trained by gradient descent with backpropagation. Edge width is the weight's size; colour is its sign.

Each hidden unit learns a soft straight edge; the output weighs them, so the boundary becomes a combination of edges that can bend around the corners. The training is the same recipe as linear regression: forward, loss, backward, step.

05Predict

Your turn: will it keep improving?

Make 5 predictions: step forward and answer each question before the next step plays.

06Try it

Break it and fix it

Use the slider to change the number of hidden units, and the buttons to change the data:

XOR, no hidden layer: the network is back to a single neuron, and accuracy stalls.
Circles with 1 or 2 hidden units: too few edges to surround the inner ring. How many do you need?
Step too big: the loss jumps around instead of falling smoothly.

07Check yourself

Check yourself

Question 1 of 5
The perceptron misclassifies a point x whose label is +1. What happens to the weights?
Your answer comes back in Review in a few days, so it sticks.
Question 2 of 5
Why can’t a single neuron learn XOR?
Question 3 of 5
What does a hidden layer add?
Question 4 of 5
Two hidden units can’t surround the inner ring of Circles. What is the right fix?
Question 5 of 5
Two layers with no activation are stacked: y = W₂(W₁x + b₁) + b₂. What can it do that a single layer can’t?

08Explain it back

Say it in your own words

Why can’t a single neuron learn XOR, and how does a hidden layer fix it?

09Recap

What to remember

A neuron draws one straight boundary; the perceptron rule finds one whenever it exists.
XOR and rings need curved boundaries, which a single neuron can't draw.
Hidden layers create new features that combine into curves; backpropagation trains every weight at once.
Next: the same building blocks, arranged differently, become convolutional networks and transformers.