Random Initialization

Random Initialization

How to pick initial value for parameter $\Theta$?
- In neural network, initialize to all zeros doesn't work because $a_1^{(2)} = a_2^{(2)} = 0.5$. Also $\delta_1^{(2)} = \delta_1^{(2)}$. This is also known as symmetric weights problem

- Random initialization to a random value in $[-\epsilon, \epsilon]$

Resources:
- https://www.coursera.org/learn/machine-learning/supplement/KMzY7/random-initialization