Backpropagation trains a neural network by calculating how much each weight contributed to the model’s error and then adjusting those weights to reduce that error.
The process works in four main steps:
- Forward pass: The input moves through the network, producing a prediction.
- Calculate the error: The prediction is compared with the correct answer using a loss function.
- Backward pass: Backpropagation uses the chain rule of calculus to calculate the gradient of the loss with respect to each weight, working backward from the output layer toward the input.
- Update the weights: An optimizer such as gradient descent changes the weights in the direction that reduces the loss.
For example, if a network incorrectly predicts whether an image contains a cat, backpropagation identifies which internal weights contributed most to the incorrect prediction and calculates how they should change.
After repeating this process across many training examples, the network gradually adjusts its weights and becomes better at making predictions.
In simple terms, backpropagation tells the neural network what went wrong and how each weight should change, while the optimizer uses that information to improve the model.
