Explainer 1 of 6 · Training · interactive
Backpropagation
How does a wrong guess turn into better weights?
Watch
The idea
A network is a chain of simple steps: multiply by weights, add, squash. The forward pass runs the chain and produces a guess; the loss measures how far the guess is from the answer.
The chain rule runs the same chain backwards. Starting from the loss, each step passes back how much its output mattered, so every weight receives a gradient: the direction and size of its share of the blame.
Training is then one line repeated millions of times: move each weight a little against its gradient. Too small a step and learning crawls; too large and the loss jumps around instead of falling.
Try it
loss 0.2650
Line thickness is a weight's size; cyan is positive, violet negative. Each step moves every weight against its gradient. Push the learning rate to 8 and watch the loss start to jump instead of fall.