Normal Equation

Normal Equation

Method to solve for $a$ analytically: take partial derivatives with respect to each $a_i$ then set it to 0 to solve for $a$

Octave: pinv(X' * X) * X' * y

Feature scaling is no longer needed for normal equation.

Gradient Descent vs. Normal Equation:

Future/more complex machine learning algorithm, we'll see that normal equation won't be able to solve them, gradient descent will.

Normal Equation Non-invertibility:

What happened when $X^T X$ is non-invertible (single, degenerate matrix)?

• This should happen pretty rarely
• If you use pinv in octave, it will always find the inverse correctly

Possible Causes of non-invertibility:

Resources:

- https://www.coursera.org/learn/machine-learning/lecture/2DKxQ/normal-equation

- https://www.coursera.org/learn/machine-learning/lecture/zSiE6/normal-equation-noninvertibility