Feature Scaling and Mean Normalization

Feature Scaling

Idea: Make sure features are on a similar scale

Example: below data, the contour will be very skewed hence gradient descent might take a long time to complete. We should convert them into similar scale, ie. similar range of values (in picture below they're now in the range of [0, 1] (if possible in the range of [-1, 1])
\(x_1\) = size (0-2000 feet²)
\(x_2\) = number of bedrooms (1-5)

Goal: get every feature into approximately [-1, 1] or [-0.5, 0.5] range
Note: denominator is actually a range (= max - min), ie. 2000 - 0 = 2000

Mean normalization:

Idea: replace \(x_i\) with \(x_i - \mu_i\) where \(\mu_i\) is the average of all the values for feature, \(s_i\) is the range of values (max - min) or \(s_i\) is the standard deviation.
Formula:

\(x_i := \frac{x_i - \mu_i}{s_i}\)

Resources:

- https://www.coursera.org/learn/machine-learning/lecture/xx3Da/gradient-descent-in-practice-i-feature-scaling