Normal Equation

Normal Equation

Method to solve for \(a\) analytically: take partial derivatives with respect to each \(a_i\) then set it to 0 to solve for \(a\)
Octave: pinv(X' * X) * X' * y

Feature scaling is no longer needed for normal equation.

Gradient Descent vs. Normal Equation:

Future/more complex machine learning algorithm, we'll see that normal equation won't be able to solve them, gradient descent will.

Normal Equation Non-invertibility:

What happened when \(X^T X\) is non-invertible (single, degenerate matrix)?
  • This should happen pretty rarely
  • If you use pinv in octave, it will always find the inverse correctly
Possible Causes of non-invertibility:


Resources:

Comments

Post a Comment