by Michele Laurelli
Methods for setting initial values of neural network weights before training begins.
Proper initialization is crucial for effective training. Random initialization breaks symmetry. Methods like Xavier/Glorot (for tanh/sigmoid) and He initialization (for ReLU) ensure gradients flow well initially.
Xavier initialization for tanh layers
He initialization for ReLU layers
Zero initialization for biases