A MISTAKE · A GRADIENT · AN UPDATE How one mistake Backpropagation, from the forward values to the five numbers that tell a tiny network how its loss can change.
Start reading → Two paths add before moving upstream The same loss sensitivity contributes minus 0.5 via a and minus 0.25 via d. Sum minus 0.75 at h. Multiply by ReLU slope one and input two to obtain w gradient minus 1.5. Loss sensitivity at prediction: −0.5 via a: −0.5 −0.5 × v via d: −0.25 −0.5 × u h sensitivity = −0.75 ADD the two contributions z sensitivity = −0.75 × ReLU slope 1 w gradient = −1.5 × input x = 2 Two contributions. One shared value.
Five short chapters. Basic arithmetic is enough to start; notation and derivatives arrive when needed. One inspectable synthetic network, a working update lab, and a complete script-free reading edition.
01 One mistake, five parameters A forward pass turns inputs into a loss. Keep its intermediate values: the backward pass will need them.
↗ 02 Local slopes, global consequences At each operation, multiply the sensitivity arriving from the loss by a derivative of that operation.
↗ 03 When paths meet, add Multiply along a path. Add across paths. Reusing a value creates another contribution to its sensitivity.
↗ 04 A gradient is not a step The backward pass reports sensitivities. A learning rate turns them into a finite proposal—and the next forward pass judges it.
↗ 05 Trust the trace. Check the derivative. Hand arithmetic, small perturbations and shape checks catch different failures.
↗