Random Initialization

Random Initialization

How to pick initial value for parameter \(\Theta\)?
- In neural network, initialize to all zeros doesn't work because \(a_1^{(2)} = a_2^{(2)} = 0.5\). Also \(\delta_1^{(2)} = \delta_1^{(2)}\). This is also known as symmetric weights problem

- Random initialization to a random value in \([-\epsilon, \epsilon]\)

Resources:
- https://www.coursera.org/learn/machine-learning/supplement/KMzY7/random-initialization