What it is and the problem it solves
Backpropagation is an efficient gradient-computation method for feedforward neural networks. It solves the problem of calculating weight-space gradients of a loss function quickly enough to make iterative parameter updates feasible.
How it works
It applies the chain rule in reverse, layer by layer, from output to input, computing gradients of the loss with respect to each weight without redundant calculations.
What works
It correctly computes the exact gradient of the loss with respect to all weights for a single example, given differentiable layers and a defined loss. Its reverse accumulation avoids exponential recomputation inherent in naive chain-rule application.
What does not
It does not specify how gradients are used—no optimiser, no learning rate, no regularisation. It does not generalise to arbitrary computation graphs without modification. It does not solve vanishing gradients, overfitting, or data scarcity.
What it changes
It enables practical training of multilayer perceptrons by making gradient computation tractable for deep networks—shifting neural network research from theoretical curiosity to empirical engineering.
Is it worth your time
Yes—if you train feedforward neural networks and need efficient gradient computation. It is not a learning algorithm itself, and offers no convergence guarantees or architectural insight.
