K-fold Cross-Validation is a widely used approach for estimating test error. The idea is to randomly divide the data into K equal-sized parts. We leave out part k, fit the model to the other K-1 parts, and then obtain predictions for the left-out kth part. This is done in turn for each part k = 1,2…..k, and then the results are combined. We cannot apply cross-validation in step 2 directly without applying it in step 1. Because it would ignore the fact that in step 1, the procedure has already seen the labels of the training data, and made use of them. This is a form of training and must be included in the validation process.