In the last section of this notebook, we're going to show you how we can use Grid Search CV to take everything that we just learnt with all the four loops, how we went with four loops for the Kfolds, and how we pass that through to cross val predict to make that more succinct. Now we are using the four loop through each one of our hyper-parameters, and then passing through cross val predicts, how we can do all of that in just one step using Grid Search CV. So we're going to import from sklearn.model selection our Grid Search CV. We are going to initiate our estimator objects using pipeline. So our pipeline is going to be the polynomial features, then the scalar, then ridge regression here. Then we are going to say what our parameters are going to be. Now, what are going to be the hyper-parameters that we want to loop over in order to figure out which one of the hyper-parameters, again, those hyper-parameters being the ones that we choose rather than what the model learns that will optimize the results for a holdout set. So we are actually going to look at both polynomial features as well as the different Alpha values that we saw. So higher polynomial features will mean more complexity, and then a higher Alpha value will mean less complexity, and we will look at looping through every single one of those values. So if there is three values here and there is 30 here, we are actually going to be looking at 90 different values, and we're going to run Grid Search CV. We have our estimator that we have defined here. So that's going to be our model, and then our parameters. Just so you know, just all practice when you're using Grid Search CV is going to be a dictionary where you're going to have as each one of the values, the different hyper-parameters that you can loop through. So the different hyper-parameters that we can loop through here are the different degrees. First we have to use the name that we defined up here, which is polynomial features, and then two underscores, and then we can pass in any of the hyper-parameters that are available for polynomial features. So that's the way that we decide how to name this. Then the same thing goes for ridge where we have to just use hyper-parameters that are available within the initiating ridge. We're going to use, again, the name ridge regression, two underscores, and then we're going to pass the hyper-parameter Alpha. So we pass in our estimator as well as the dictionary of each one of the different values that we want to loop through, and then we say we want to be, again, looking at holdout sets, how are we going to split up our data-set? We're going to use that k folds object that we defined earlier. As I mentioned, when we were doing cross val predict, you can also pass in a number if you don't have a k folds object created, but we create our grid objects. Now that we have our grid objects, that grid object is similar to many of the sklearn objects that we've been using so far or it's going to have this fit method and it's going to fit that pipeline, but running through each one of these different hyper-parameters. So we fit it, and that's going to take a bit a minute as it's going through 90 different hyper-parameters. It's going to figure out what the best score and the best parameters were. We are going to print that out and that's going to be, the best score they got was 0.85 and the best degree was degree two with a ridge regression Alpha of 4.0. We can then actually use that grid in order to predict the x-values. Recall we were not able to do that with cross val predicts. The reason why we are able to do that now is that with grid search, it's going to test against all of the different holdout sets, and then once it's tested against all of the different holdout sets and it finds the best hyper-parameters that will fit for generalization for data-sets that we've never seen. It's then going to use those hyper-parameters to learn the parameters on the entire data-set because the more data that you have, the better you're going to be able to predict, and the idea is that you are now able to use this to predict as new data comes in. It's ready for production. So we're going to run grid.predicts since it's now fit on the entire data-set. Again, we don't have actual holdout data or new data, so it should perform fairly well, and we get an r2 score of 0.89. Then with the grid, we were going to call out our best estimator, that's an attribute that's available within the grid if we want to look at different attributes within our actual estimator that we're looking at. So we have our estimator, we do grid.estimator, best estimator is specifying the estimator that we had passed into our Grid Search CV. Then from there, we can pull out name steps such as ridge regression and get our coefficients. Then if you do grid.cv results, this is actually a little bit cleaner if we do data-frame, and you can see for each one, if you want all the details of what grid search did, we are going through each one of the parameters, what the different scores were for each one of the splits. Then it looks at the mean score, the standard deviation of each one of those scores, and ultimately, picks the best mean score. So if you want to look at more details, you can look at the CV results. In conclusion, in our notebook, we learned first just how to do those general K folds. We then moved on to using cross val predicts so that we can see the scores for each one of those K folds in assessing fashion. Then we talked about how we can use hyper-parameters in order to look at multiple versions of the same model, reducing or increasing the complexity of that model to see which one performs best on our holdout set. We passed that then using grid search CV. That's going to find that the best hyper-parameters, given the hyper-parameters that we gave it the options to run such as degree 1, 2, or 3, or Alphas within this range of 4-20. Then it will ultimately fit on all the data so that you will have the best hyper-parameters which are chosen and then tested across each one, as well as all of the best parameters as they will be learned on the entire data-set. That closes out our notebook. I look forward to seeing you back in lecture. Thank you.
