AI Blog

AI Blog

by Michele Laurelli

Back to Glossary

Gradient

Concept
Definition

Direction and magnitude of steepest increase in loss function with respect to parameters.

Gradients indicate how to adjust weights to reduce loss. Computed via backpropagation. Move in opposite direction (gradient descent).

Examples

1

Weight gradient: -0.5

2

Gradient flow through network

3

Vanishing gradients

Related Terms

Gradient Descent

An optimization algorithm that iteratively adjusts parameters to minimize a loss function by following the gradient.

Backpropagation

An algorithm for training neural networks by calculating gradients of the loss function with respect to weights.

Michele Laurelli - AI Research & Engineering