Skip to content
Code it · Write it

Backpropagate through a neuron by hand

A neuron computes z = w1·x1 + w2·x2 + b and outputs out = tanh(z). Write neuron_grads(x, w, b, out, grad) (neuronGrads in JavaScript and TypeScript), where x and w are lists of two numbers and out is the output, already worked out for you.

Fill grad with how out changes with each parameter: grad[0] = d out/d w1, grad[1] = d out/d w2, grad[2] = d out/d b. Return grad.

Chain two local gradients: tanh’s, which is 1 − out², and z’s with respect to each parameter, which is the input it multiplies (or 1 for the bias).

def neuron_grads(x, w, b, out, grad):
    # out = tanh(w[0] * x[0] + w[1] * x[1] + b), already worked out
    # Fill grad with d out/d w1, d out/d w2 and d out/d b.
    return grad