Model Selection and Train/Validation/Test Sets

Model Selection and Train/Validation/Test Sets

Model selection problem: choose degree of polynomial, features or regularization parameter

As you can see, if we try to decide polynomial parameter d, we have to use our training set to learn \(\theta\), then to pick the best polynomial parameter d, we have to evaluate our hypothesis using test set. Once we pick a d (above =5), we have no more samples to evaluate.

This means we have to split our samples into 3 sets:

• 60% training set
• 20% cross validation set (cv)
• 20% test set

Resources:

- https://www.coursera.org/learn/machine-learning/supplement/XHQqO/model-selection-and-train-validation-test-sets