Training & backpropagation

What it is

Training means adjusting a network's weights so that its answers get closer to the correct ones. Backpropagation is the method that works out, for every weight, how much it contributed to the error, by passing the error backwards through the layers.

What it is for

It is how nearly every modern neural network is trained, from small classifiers to large language models. Knowing it explains why training needs many examples and a lot of computing, and what words like learning rate and epoch mean.

How to do it

The process, step by step. Libraries and tools change; these steps stay.

1. Pick a loss: a single number that says how wrong the network is on an example.
2. Forward pass: run examples through the network and compute the loss.
3. Backward pass: backpropagation computes the gradient, that is, for each weight the direction that would reduce the loss.
4. Update: an optimizer (for example SGD or Adam) moves every weight a small step, sized by the learning rate.
5. Repeat over the whole dataset many times (epochs), watching the loss on data the network has not trained on.
6. In practice, libraries such as PyTorch compute the backward pass for you: you define the model and the loss.