Feature Scaling and Mean Normalization

Feature Scaling

Idea: Make sure features are on a similar scale

Example: below data, the contour will be very skewed hence gradient descent might take a long time to complete. We should convert them into similar scale, ie. similar range of values (in picture below they're now in the range of [0, 1] (if possible in the range of [-1, 1])
$x_1$ = size (0-2000 feet²)
$x_2$ = number of bedrooms (1-5)

Goal: get every feature into approximately [-1, 1] or [-0.5, 0.5] range
Note: denominator is actually a range (= max - min), ie. 2000 - 0 = 2000

Mean normalization:

Idea: replace $x_i$ with $x_i - \mu_i$ where $\mu_i$ is the average of all the values for feature, $s_i$ is the range of values (max - min) or $s_i$ is the standard deviation.
Formula:

$x_i := \frac{x_i - \mu_i}{s_i}$

Resources:

- https://www.coursera.org/learn/machine-learning/lecture/xx3Da/gradient-descent-in-practice-i-feature-scaling