Artificial intelligence, and in particular Machine Learning, is used nowadays to support decisions in many different fields: from automated hiring systems to commercial tools for automatically predicting some categories of future crimes used in sentencing decisions. Automated Decision Systems process data about people and combine human and automated decision making. Using AI to support human decisions is not new: for example starting from the 1980s several systems to support diagnosis of diseases and their possible treatments have been created. What is new now is the approach. While traditional systems adopt a top-down approach, that is based on knowledge and rules established by humans, current systems use a bottom-up approach. This is based on the ability of the system to learn from experience those rules to infer new knowledge. This approach typically uses Machine Learning techniques. Learning is one of the key abilities to characterize intelligence. A program able to learn from experience can adapt to novel situations and perform tasks without explicit instructions. A Machine Learning program does not possess the precise instructions to be executed in a given situation; rather the program, starting from a large enough set of data, is able to elaborate a mathematical model. This model can automatically predict the most accurate answer according to the received input. Classification is a possible form of learning. A classification algorithm is capable of predicting to which class an unknown object belongs, by choosing from a set of known classes. This mechanism is similar to that of a child who wants to learn the animals in a zoo. At the beginning her teacher will show her the animals and say: “This is a swan, this is a duck”. Then, after some other examples, the child will try to correctly classify new animals. Errors will help her to improve this learning process. Decisions can be seen as problems of classification. A classification algorithm can be applied to many different contexts. What is important is: First, that the number of examples used to train the algorithm is large enough; And second, that these examples are representative of the population on which the algorithm will decide. Machine Learning can be applied to very simple decisions, such as the next video to watch or the next purchase to make. This is the case of the so-called recommender systems. In this case the algorithm learns on the basis of a similarity principle: users showing similar behaviors will maintain it also in the future. However, the same mechanism can be applied to more controversial decisions, as the mathematician Cathy O’Neil has discussed in her best-seller “Weapons of Math Destruction”. For example, O’Neil describes an Automated Decision System, named IMPACT, to automatically evaluate teachers in the schools of Washington DC. A score was assigned to every teacher on the basis of the results achieved by their students in standardized tests. Unfortunately this resulted in the firing of some teachers who were indeed very much esteemed by students and their families. Similar systems are also used in recruiting processes, where they are adopted to pre-screen CVs and evaluate job interviews. More problematic applications are those of the so-called predictive justice. These Automated Decision Systems are used for example to predict recidivism, that is the possibility that an arrested person can re-offend. For example, in the US the software called COMPAS is widely used to automatically evaluate the risk of recidivism; in UK a similar software is HART. The use of such systems is simpler in those situations where decisions are based on a prediction and the event or the category to be predicted is well defined and measurable. However, notwithstanding some opportunities to improve decision processes, many applications of these technologies are problematic. Algorithms are subjected to a positive prejudice, that is they are neutral and produce objective results. However, using mathematical and statistical tools is not enough to guarantee that a decision is impartial. A mathematical model indeed is always a representation of reality, so it contains assumptions, simplifications and even possible errors. These assumptions can limit the application of Machine Learning algorithms and raise ethical and social issues. A possible assumption can concern the mechanism used to generate data samples to train the algorithm. If these data samples are not representative enough of the population that is of interest in that particular situation, the risk is to consider correlations that are not significant or even misleading. These and similar assumptions bring to a well-known problem in Machine Learning, that is called overfitting. Overfitting occurs when the model is accurate only with the examples used in the training phase, whereas its performance decreases in the test phase. In other words, an algorithm, that is overfitted with respect to the examples, is not able to generalize. It’s like a map which is very precise with respect only to a specific area of a city, but not very much detailed for other areas and, therefore, not very useful to orient us. In conclusion, we can say that not only these algorithms must be accurate, robust and safe; they also must be compliant from an ethical, social and legal point of view. In the next lecture we will illustrate some of the most common problems and the possible approaches to deal with them.
