by Michele Laurelli
An optimization algorithm that iteratively adjusts parameters to minimize a loss function by following the gradient.
Gradient descent updates parameters in the direction opposite to the gradient. Variants include batch, mini-batch, and stochastic gradient descent (SGD). Learning rate controls step size.
Training neural network weights
Optimizing linear regression
Fine-tuning model parameters