Back to Glossary
Artificial Intelligence
Backpropagation
Backpropagation computes gradients of the loss function with respect to model weights for training.
Definition
Backpropagation (backward propagation of errors) is the fundamental algorithm for training neural networks. It computes the gradient of the loss function with respect to each weight in the network by applying the chain rule of calculus, working backwards from the output layer to the input layer. These gradients are then used by an optimizer (SGD, Adam) to update weights. Backpropagation requires: (1) Forward pass — compute predictions, (2) Loss calculation, (3) Backward pass — compute gradients, (4) Weight update. This cycle repeats millions of times during training. Modern GPUs accelerate backpropagation via Tensor Cores.
Related Keywords
backpropagationbackpropgradient computationneural network training
Related Terms
AI Training
AI training is the process of optimizing model parameters using labeled data and compute resources.
Transformer Architecture
The Transformer is a neural network architecture using self-attention, the basis for modern LLMs.
Fine-Tuning
Fine-tuning adapts a pre-trained model to a specific task using domain-specific data.