by Michele Laurelli
A mathematical function applied to a neuron's output to introduce non-linearity into the network.
Activation functions enable neural networks to learn complex patterns. Common functions include ReLU, sigmoid, tanh, and softmax. They determine whether a neuron should be activated based on input.
ReLU for hidden layers
Sigmoid for binary classification
Softmax for multi-class classification
An activation function that outputs the input if positive, otherwise zero: f(x) = max(0, x).
An activation function that maps inputs to values between 0 and 1: f(x) = 1/(1 + e^(-x)).
A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) that process information.