Who should change?

A small network predicts 1.5 when the target is 2. The prediction is too low. That tells us the direction of the output error, but it does not yet tell us how to change an earlier weight. That weight’s effect passes through other operations—and may reach the output by more than one route.

Backpropagation computes how sensitive one scalar loss is to each parameter. An optimizer uses those sensitivities to propose new parameters. We will do both operations by hand before letting the lab repeat them.

You need arithmetic and the idea that a model has adjustable numbers. We introduce derivatives and vector shapes when they become useful. If you already know losses, treat the half-squared error below as our fixed measuring rule.

A network you can hold in your head

Imagine a normalized electricity-demand feature $x$ and a target $y$. Our numbers are invented and dimensionless. There is no timestamp, real grid data, fitted forecast or physical unit behind them. The demand story motivates inspecting a learning mechanism.

The five adjustable parameters are $w,b,v,u,c$. Every variable here is a scalar: one number. First form an affine value $z$ (multiply, then add); apply a rectified linear unit, or ReLU, to get the hidden activation $h$:

ReLU passes a positive input unchanged and outputs zero for a negative input. The same $h$ is used in two branches, $a$ and $d$, which rejoin at prediction $\hat y$:

This deliberately redundant readout could be collapsed to $(v+u)h+c$. Keeping its branches separate makes one shared value’s two uses visible. These are not two different hidden neurons.

Start with $x=2,\ y=2$ and $w=0.5,\ b=0,\ v=1,\ u=0.5,\ c=0$. Forward values are $z=h=1,\ a=1,\ d=0.5,\ \hat y=1.5$. Arrows record dependencies. PyTorch’s autograd mechanics describes how recorded operations form the graph used for differentiation. BP-C01, BP-C09–10

End at one number

Define residual $e=\hat y-y$ and loss $L$:

Here $e=-0.5$, so $L=0.125$. The factor one-half is a convenience: differentiating cancels the square’s factor two. This is a loss for one example, not a batch mean.

Record intermediate values rather than only the final answer. To differentiate $a=vh$, we will need the values of both $v$ and $h$ from this forward pass. Backpropagation travels backward through dependencies; it does not run the prediction formula in reverse to recover the input.

What a derivative says

A derivative is a local sensitivity. Writing $\partial L/\partial w=-1.5$ means that a tiny increase in $w$, with other parameters fixed, changes loss at about −1.5 times that increase. It is neither a weight value nor a command to add −1.5.

For $\Delta w=0.001$, the first-order estimate is $\Delta L\approx-0.0015$. This approximation concerns sufficiently small changes in a differentiable region; it is not an exact finite-step promise. BP-C15

We use compact notation: $\bar q=\partial L/\partial q$. Read “q-bar” as the sensitivity of the loss to q. It is an intermediate derivative, often called an adjoint; it is not the forward value $q$.

Multiply along a route

Suppose $r=f(q)$, and the loss depends on $r$. The chain rule connects the small operation’s derivative to the loss:

Seed the calculation with $\bar L=1$: the sensitivity of $L$ to itself is one. For $L=e^2/2$, the local derivative is $e$. Thus $\bar e=1(-0.5)=-0.5$. Since $e=\hat y-y$, its derivative with respect to $\hat y$ is one, so $\bar{\hat y}=-0.5$.

At the addition $\hat y=a+d+c$, each local derivative is one. Each branch receives the same incoming sensitivity, −0.5. The forward sum does not divide it between its inputs. At multiplication $a=vh$, the local derivative with respect to $v$ is $h$; with respect to $h$ it is $v$.

The official autograd explanation describes reverse-mode differentiation as applying the chain rule over recorded operations. BP-C01–02

Keep the two derivatives separate

The local derivative $\partial a/\partial h=v=1$ says how $a$ changes with $h$. The loss sensitivity contributed through that branch is $\bar a\,v=-0.5$. Forgetting the incoming factor confuses an operation’s behavior with its effect on the objective.

There is another issue: $h$ also changes the second branch. We have found one contribution, so we are not ready to propagate backward through ReLU yet.

One value, two uses

Through $d=uh$, the contribution to the hidden value is $\bar d\,u=-0.5(0.5)=-0.25$. Changing $h$ affects both branches at once:

This is the multivariable chain rule. If a value $q$ feeds several direct consumers $r_j$, accumulate every contribution before propagating farther upstream:

Conceptually, initialize a node’s sensitivity to zero and add contributions. Overwriting −0.5 with −0.25 discards a real dependence. Add first, then continue in reverse dependency order. BP-C02

Through the activation

Our $z=1$ is in ReLU’s positive region, whose local slope is one. Hence $\bar z=\bar h(1)=-0.75$. For $z=wx+b$, the local derivatives are $x$ with respect to $w$ and one with respect to $b$:

The gradient collects all five partial derivatives in the declared order $(w,b,v,u,c)$. A negative gradient for $w$ predicts that a small increase in $w$ lowers loss. Gradient descent subtracts that negative gradient. BP-C11, BP-C15

A zero needs a diagnosis

For $z<0$, ReLU stays zero under sufficiently small perturbations. Its slope is zero, blocking the signal to $w$ and $b$ for this example. With $h=0$, gradients of $v$ and $u$ are also zero. Output bias $c$ can still have a nonzero gradient. A blocked hidden route does not imply every parameter stopped learning.

At exactly $z=0$, ReLU has no ordinary derivative. Our code uses a zero backward rule there, consistent with PyTorch’s minimum-norm subgradient convention for this convex operation. This is an implementation choice, not an ordinary derivative. Autograd’s nondifferentiable-function rules give the convention. BP-C04, BP-C14

Another cause of zero is cancellation. If $v=1,u=-1$ while $h=1$, contributions to $\bar h$ cancel. The activation is positive. Inspect the routes before diagnosing an inactive activation.

Three kinds of accumulation

The path sum belongs to one graph. If a batch loss is a sum of example losses, its gradient is their sum; if it is a mean over a fixed batch of $N$ examples, the gradient includes the same $1/N$ factor. Reduction changes the objective you differentiate.

There is also framework storage behavior: PyTorch accumulates gradients into parameter .grad buffers across backward calls. A usual fresh training step clears them with optimizer.zero_grad() before computing new gradients. That housekeeping does not replace the chain rule’s path sum. The optimization tutorial shows the ordering. BP-C06

Make the proposal

Write parameters as $\theta=(w,b,v,u,c)$ and their gradient as $g=\nabla_\theta L$. Plain gradient descent chooses a nonnegative learning rate $\eta$:

We use the same single example each time: deterministic gradient descent. This is the no-momentum, no-weight-decay case of PyTorch’s SGD rule. All five deltas use the same old forward pass and gradient, then all parameters change together. Updating one before computing the remaining gradients would be a different procedure. BP-C05

At $\eta=0.05$, the first weight delta is $-0.05(-1.5)=+0.075$, so $w_{\text{next}}=0.575$. The complete new vector is $(0.575,0.0375,1.025,0.525,0.025)$. A fresh forward pass gives prediction 1.865625 and loss about 0.00902832.

Try the next step

Before revealing a proposal, predict: with the initial example, will rate 0.5 lower the loss? The gradient is still correct; the question is how far its local information should be trusted.

At the initial parameters, changing only the rate never changes the gradient. Rate 0.05 lowers this loss; rate 0.5 produces prediction 6 and loss 8. A finite step can cross the target and make the error larger. These are toy consequences, not recommended rates for another network. BP-C12

After an update, compute the next gradient at the new parameters. Reusing the initial gradient for every step would follow a fixed direction. Large rates in this tiny system may later close ReLU and let the output bias alone fit the example; the first loss increase does not prove permanent divergence. In a closed or boundary case, that bias can move while the hidden route stays blocked.

Where autograd ends

This schematic PyTorch sequence explains responsibilities. The complete runnable companion for this guide is the dependency-free JavaScript lab linked in the notes.

optimizer.zero_grad()
prediction = model(x)
loss = 0.5 * (prediction - target).square()
loss.backward()
optimizer.step()

backward() computes and accumulates derivatives for a scalar loss. step() applies the optimizer’s rule. Calculating gradients does not itself update model parameters. See Optimizing model parameters. Our browser lab implements the equations directly so you can inspect each term. BP-C03

These updates fit one invented pair. A falling loss here says nothing about a real demand forecast, held-out examples, data leakage or research validity. Those require a different experiment and evidence.

Perturb one parameter

A numerical check provides another route to a derivative. Perturb only parameter $\theta_i$ by a small amount $\varepsilon>0$ in both directions and recompute the complete forward loss:

Here $e_i$ is the coordinate vector that changes only the selected parameter, unrelated to our residual $e$. At the default smooth point, $\varepsilon=10^{-6}$ gives agreement with all five analytic gradients within the lab’s absolute tolerance $10^{-8}$.

That agreement is separate from the hand-computed values. Both matter: a checker can miss a shared bug if its “numerical” loss reuses the analytic derivative. Our finite-difference path calls the forward loss alone. Gradcheck mechanics explains the comparison. BP-C07

import { DEFAULT_PARAMS, DEFAULT_EXAMPLE,
  backward, finiteDifference } from './model.mjs';

console.log(backward(DEFAULT_PARAMS,
  DEFAULT_EXAMPLE).gradients);
console.log(finiteDifference(DEFAULT_PARAMS,
  DEFAULT_EXAMPLE, { epsilon: 1e-6 }));

The expected analytic gradient is $(-1.5,-0.75,-0.5,-0.5,-0.5)$. Run the full companion for recorded outputs and assertions. Small final-digit differences in the numerical estimate are floating-point arithmetic.

The corner is different

Set $b=-1$ with other defaults unchanged. Now $z=0$. Perturbing $w$ in opposite directions samples different sides of ReLU’s corner. The centered estimate approaches −3; our chosen backward gradient at the corner is zero. This disagreement is expected: there is no ordinary derivative for both methods to recover.

Even away from a corner, a large perturbation may cross it. An excessively small perturbation can lose precision through subtraction. PyTorch’s gradcheck reference warns about nondifferentiable points and notes its default settings target double precision. Our tests exclude corners from derivative-agreement assertions and check the convention separately. BP-C04, BP-C08

When numbers become vectors

For column input $x\in\mathbb R^D$, weights $W\in\mathbb R^{H\times D}$ and bias $b\in\mathbb R^H$, an affine layer gives $z=Wx+b$. Let $\bar z=\partial L/\partial z$ be an $H$-coordinate column of sensitivities.

The transpose collects contributions from every output coordinate into each input coordinate. The outer product gives one derivative for every weight entry. A gradient has the same shape as its parameter. These equations use one column-vector example; batch dimensions and reductions need explicit treatment. BP-C13

Diagnose a new case

Try this without the lab. Keep the architecture but change values: $x=3,\ w=1/3,\ b=1,\ v=2,\ u=-2,\ c=1,\ y=3$. A colleague reports “the early-layer gradient is zero, so ReLU must be inactive.” Is that justified? Find $h,L,\bar h$ and all five parameter gradients. Will every parameter stay fixed under a positive-rate step?

When a gradient surprises you, inspect saved forward values, local derivatives, all downstream routes and the loss reduction. Then inspect the optimizer separately. These questions make “the model learned” a sequence of checkable operations.