Logistic regression: a line that gives probabilities

01Why it matters

A line that says how sure it is

Is this email spam? Will this customer cancel? A yes/no answer isn't enough: you want to know how likely, so you can act differently at 51% and at 99%.

Logistic regression draws a straight line through the data and turns "which side, and how far" into a probability. It is also a single neuron with a sigmoid, the building block that neural networks stack by the thousand.

02Key parts

The pieces you will see

score: $z = w_1 x + w_2 y + b$ : positive on one side of a line, negative on the other, larger further away.
sigmoid: $\sigma(z) = \frac{1}{1 + e^{-z}}$ squashes any score into a probability between 0 and 1, with $\sigma(0) = \tfrac12$ .
decision boundary: Where $z = 0$ , so $p = \tfrac12$ : always a straight line.
cross-entropy loss: The average of $-\ln p$ of the true class: small when the model is confidently right, huge when it is confidently wrong.

03See it work

Watch the line swing into place

The shading is p(class 1) at every spot; the line is where p = ½. Each step moves the weights against the gradient of the loss.

At the start every weight is 0, so $p = \tfrac12$ everywhere and the loss is $\ln 2 \approx 0.693$ : a coin flip. The gradient is remarkably simple:

\nabla_{w} \text{loss} = \text{mean of } (p - \text{label}) \cdot x

Each point pushes the weights by its own error $p - \text{label}$ . A point the model already gets right with confidence barely pushes; a point on the wrong side pushes hard. As the classes separate, the weights grow and the shading sharpens from a gentle blur into a crisp edge.

04Predict

Your turn: overlapping classes

Make 6 predictions: step forward and answer each question before the next step plays.

05Try it

Make it fail

Pick the datasets one after another:

Overlapping: no line separates every point, so the model hedges with $p$ near ½ in the middle.
XOR: opposite corners share a class. The loss barely moves from $\ln 2$ , and accuracy stays far from 100%.
Circles: one class surrounds the other; any straight line cuts through both.

The last two aren't a training problem: more steps won't help, because no straight line fits. Curved boundaries need new features, or hidden layers, which is where neural networks come in.

06Check yourself

Check yourself

Question 1 of 5
A point sits exactly on the decision boundary. What probability does the model give it?
Your answer comes back in Review in a few days, so it sticks.
Question 2 of 5
Two class-1 points: the model gives one p = 0.9 and the other p = 0.1. Which pushes the weights harder?
Question 3 of 5
Before any training, all weights are 0. What is the loss?
Question 4 of 5
Why can’t logistic regression learn XOR, however long it trains?
Question 5 of 5
A spam filter built with logistic regression outputs 0.8 for an email. What does that 0.8 mean?

07Explain it back

Say it in your own words

How does logistic regression turn a straight line into a probability, and how does training move that line?

08Recap

What to remember

$p = \sigma(w \cdot x + b)$ : a linear score squashed into a probability.
The boundary $p = \tfrac12$ is always a straight line; confidence grows with distance from it.
The loss is cross-entropy; its gradient is the mean of (p − label) times the input.
Curved boundaries are out of reach without new features or hidden layers.