Unrolling Parameters

Unrolling Parameters

Usually advanced optimization algorithms assume that $\theta \in R^{n+1}$ which is a vector. In neural network these $theta$ are no longer vectors, they are matrices. Unrolling parameter method will help us unroll matrices into vectors then later on convert them back into matrices.

Note: image above missing one hidden layer: $s_1 = s_2 = s_3 = 10, s_4 = 1$
thetaVec is in $R^{231}$

Advantage of Matrix representation:
- Vectorized implementation of Forward and Back Propagation very easy and clean
Advantage of Vector representation (unrolling parameter):
- When using optimization algorithms, they tend to assume that you pass them a long Vector instead of Matrix

Resources:
- https://www.coursera.org/learn/machine-learning/supplement/v88ik/implementation-note-unrolling-parameters