by Michele Laurelli
An activation function that outputs the input if positive, otherwise zero: f(x) = max(0, x).
ReLU is the most popular activation function in deep learning due to its simplicity and effectiveness. It helps mitigate vanishing gradient problems and enables faster training compared to sigmoid or tanh.
Hidden layers in CNNs
Deep neural networks
Most modern architectures