Gradient Checking

Numerical Gradient Checking

Help eliminating all problems and bugs related to implementation of back propagation.

Numerical estimation of gradients: 2 sided epsilon

Note: numerical gradient checking is a very inefficient method to obtain gradient. It is much much slower than back propagation algorithm, so we must turn off gradient checking before using back propagation.

Resources:

- https://www.coursera.org/learn/machine-learning/supplement/fqeMw/gradient-checking