Unrolling Parameters

Unrolling Parameters

Usually advanced optimization algorithms assume that \(\theta \in R^{n+1}\) which is a vector. In neural network these \(theta\) are no longer vectors, they are matrices. Unrolling parameter method will help us unroll matrices into vectors then later on convert them back into matrices.

Note: image above missing one hidden layer: \(s_1 = s_2 = s_3 = 10, s_4 = 1\)
thetaVec is in \(R^{231}\)

Advantage of Matrix representation:
- Vectorized implementation of Forward and Back Propagation very easy and clean
Advantage of Vector representation (unrolling parameter):
- When using optimization algorithms, they tend to assume that you pass them a long Vector instead of Matrix

Resources:
- https://www.coursera.org/learn/machine-learning/supplement/v88ik/implementation-note-unrolling-parameters