[MUSIC] For any specific value of lambda, we get some balance between this residual sum of squares, and this two norm. And so what I'm gonna do, is in this movie, I'm gonna add these two contour plots together. I'm gonna add, so let me write this down. Add contour plots together, where I'm getting residual sum of squares of w plus lambda 2 norm of w. Where here the residual sum of squares were these ellipses, centered about my least squares solution, and there's to norm, where these circles centered about zero. And lambda is some weighting on how much I'm including that two norm penalty or the cost. And what I'm going to do is I'm going to show a movie as a function of lambda. So movie, function of increasing lambda, where I have my ellipses, and I'm weighting more and more heavily these contours that are coming from the circle. The circle terms from this two norm penalty. Okay, so this is the movie right here, and my lovely assistant Carlos, will click the mouse to play the movie. [LAUGH] Since I don't know how to control it from the tablet, unfortunately. [LAUGH] Thank you Fanna. That reference was probably lost on most people. And in doing so, you didn't get me describing the movie so let's watch it again. But what we see this x, let me be clear, that the x is going to mark the optimal solution, For a specific lambda and we're varying lambda so this x is gonna move. Okay where's the x gonna start? Well when lambda's equal to zero, we're starting out our lee squared solution and as lambda increases we know that as lambda goes to infinity the coefficients are gonna shrink to zero. But let's visualize the path that it takes as we increase lambda. Okay so let's play this movie again, gonna start it early square the solution and we see that the magnitude of our coefficients W0 and W1 are shrinking smaller and smaller towards zero. So, maybe we'll play that once more just to visualize this and what we say. Again, this was just the tail end of the movie. Is this shrinking magnitude of the coefficients? Carlos is very excited about this movie, so we're gonna watch it one more time. It's pretty cool. We've never actually seen somebody do this visualization. We think it's really intuitive. So again, as that land of penalty is increasing, the magnitude of the coefficients are getting shrunk. Okay, well now let's talk about what the solution looks like for a given value of lambda. Oops, sorry let me turn my pen on. So for a specific lambda value. We have some balance between residual sum of squares and the magnitude of our coefficients. Lambda's automatically doing some trade-off between the two. So some balance Between RSS and our two norm. And specifically, in this plot, this is our solution. So, it has some RSS that happens to be, five thousand two hundred fifteen, that's what the number on this contour is indicating and it has some tune arm, which has value 4.75, and so this lambda has chosen the specific trade off and we see that our solution is somewhere here, which has shrunk from where our least square solution was. Let's remember our least square solution was somewhere around here. And the optimal for lambda equals infinity was that zero. So it's somewhere in between these two values. And if we had chosen a different value of lambda, let's say a larger value of lambda We would of had a different solution. And when I'm drawing all these contours, what I'm saying is, let me just go back to the original one before this this drawing. What I'm saying is, every other point along this circle, has exactly the same residual sum of squares. But larger l2 norm of w and everywhere along this circle has exactly the same w2 norm, sorry, l2 norm of w, but it has larger residual sum of squares. So that's why this is the optimal trade off for this lambda. Then, like I drew here, if I chose a larger lambda, I will get a solution that preferred a smaller two norm and a larger residual sum of squares. So, this would be solution for a larger lambda value. Okay, so this is just a little visualization of what a ridge regression solution looks like. [MUSIC]