Now, when we have learned how to calculate false positive and false negative error types, we can go to benefit curve analysis. What is this? Look, we have historical data, we can sort this data by probability of target event ascending, and we have a table with our clients and historical events about these clients. In the upper part of the table, we will have clients with rather low probability of target event in our case of default and in the lower part of the table, on the contrary, we will have clients with a higher probability of target event. Again, we can come up with some threshold level A and we will have certain acceptance rate C. Well, we can vary this threshold level A, not only 0.5, but we can try 0.4, 0.3, et cetera. When we iterate through these numbers, we have different numbers of false positive and false negative. We can combine them in a benefit curve. This is how it looks like. Benefit curve has x axis, which is acceptance rate, and the y axis, which is a benefit or financial results that we experience by implementing this model with a certain acceptance rate. Let's discuss some certain points of this curve. The left one is the starting point. We have threshold level a equals zero and the same as acceptance rate c. It's also zero. It means that we reject every client who has probability of default higher than zero. Obviously, every client has probability of default higher than zero because it ranges from zero to one. In this case we will have no loss and no gain as well. Therefore we have zero benefit. Then we can go to some middle point. For instance, c equals 0.3. Well, we calculate this point by coming up with false positive and false negative for given c. Then we take weights for this false positive and false negative error types. Which are LGD in case of credit scoring and credit margin. We plot on the y axis the resulting formula. Which you can see above the benefit curve. We can reiterate these steps from first to the third for different c levels. Therefore, we will get point for each c. This is how this benefit curve is built. Well, there is another interesting point, the right point. It's called end points. We have threshold level a, which is one, and acceptance rate c, which is also one. It means that we reject every client who has probability higher than one. Well, obviously, no one can have anything like probability higher than one because it ranges from zero to one. It means that we accept every loan application. It means that we do not reject any client base our model. Regardless the value of probability of model prediction, we accept the loan. Let's calculate precisely what benefit we're going to get from this endpoint. Well, you can put this into the formula and you have a false positive from one and false negative at one too. Obviously, false positive at one is zero because we do not reject anyone. We cannot make a mistake by rejecting a good borrower. So it's zero. False negative from one, is p. Which is total number of positive clients. Because if there are some positive clients in sense of being default client, we accept everyone, so we accept all of them. It means that our formula gets to the N multiplied it by the error cost of false positive error and we subtract number of positive clients multiplied by error cost of false-negative. Well, this is rather interesting in number. Because it can be either less than zero or higher than zero or equal to zero. Actually in real business processes, we can have different situation. It depends on the parameters of portfolio of clients that we deal with. In the first case, which I would say is rather typical for credit scoring, is that if we accept all loan applications, we will have negative benefit. This holds true when ratio of negative and positive clients is less than the ratio of the error types costs. On the contrary, when this point is higher than zero, we have negative two positive clients ratio higher than ratio between error types. This happens when our Client inflow is very high-quality I would say. It means that maybe there were some certain checks for those clients and some of them were rejected previously, before given to the model. It can be, but it's rather rare situation. Theoretically there is possibility to have the situation when those ratios are equal and both end points on the left and on the right are zero. You have to consider your class balance within your data and the ratio between one and two type errors. Well, what's so interesting about this benefit curve? How can we use it? Well, the very simple option is to analyze an optimal decision level. It means that we can vary our threshold level a and corresponding acceptance rate c. We can calculate the optimal level of this decision rule. In this case, we have a situation when we apply acceptance rate c equals 0.3. But at the same time, there is a point b where c is 0.4. and for the point b, we have higher benefit compared to point a. In this case, we can observe a sub-optimal decision-making and we can make a recommendation to change your acceptance rate to increase it. This is one aspect of benefit curve. How it can make our decisions better. In the later topics, we will see other ways we can use it. Now let's go for some practice. Let's make a small summary of what we have learned. Well, decision-making scheme has a huge impact on financial result. Even if you have an ideal model which is 100 percent accurate, you can easily ruin it by taking a wrong threshold level a or applying the wrong acceptance rate c. You have to choose them optimally. Sometimes, it's easier and more efficient to improve decision strategy than the model itself just by changing your acceptor level c.