In this session, we're going to take a closer look at variance, how variance is computed, and how you can structure your program to make them variance correct. The material in this session is a bit more advanced than the other sessions, so you can also skip it in a first pass through the course. We've seen in the previous session that some types should be covariant, whereas other types should not. Roughly speaking, a type that accepts mutations of its elements, like array did, should not be covariant. But immutable types can be covariant if some conditions on methods are met. Before we go on, let's define what we mean by variance. Say C of T is a parameterized type, and A, B are types such that A is a subtype of B. In general, the three possible relationships between C of A and C of B. C of A could be a subtype of C of B, we say then C is covariant. Or C of A could be a supertype of C of B, then we say C is contravariant. Or neither C of A, nor C of B is a subtype of the other, then we say C is nonvariant. Covariant means the application with C varies in the same direction as the elements, contravariant means it varies in the other direction, and nonvariant means the two types are unrelated. In Scala, you can declare the variance of a type by annotating the type parameter. You can declare a class to be covariant by writing a plus in front of the type parameter, A in this case, or for contravariant classes, you would write a minus in front of the type parameter. If you don't write anything, then the class is assumed to be nonvariant. Here's an exercise that requires some thought. Let's say we have a type hierarchy fruit with subclasses for apples and oranges. Then we have two functions. One converts a fruit to an orange, and the other takes an apple and returns a fruit. According to the Liskov Substitution Principle, which of the following should be true? Should fruit to orange be a subtype of apple to fruit? Or should apple to fruit be a subtype of fruit to orange? Or should the two types be unrelated? Let's look at the first subtyping relationship. What can I do with the second function, apple to fruit? Well, I can pass it an apple and obtain a fruit. Can I do the same thing with the first function? Yes, you can pass an apple to the first function, apples are fruit, and you will obtain a fruit because oranges are fruit as well. This one seems to be true. What about the second relationship? Is apple to fruit a subtype of fruit to orange? Well, what can I do with a fruit to orange function? I can pass it a fruit and obtain an orange. Can I do the same thing with the second function? No, because the second function requires an apple. To the first function, I could also pass an orange, oranges are fruit, but oranges are not apples. I can't pass the same things to the second function. Furthermore, the second function actually returns a type that is less precise than the first one. The first guarantees to give back an orange, whereas the second only guarantees to give back some fruit. It could be an orange, but it could also be an apple. That relationship is not true, and so the correct answer is 1 in this case. We can generalize from that reasoning to arrive at a rule for subtyping between arbitrary function types. Here's the rule. We say the function type A1 arrow B1 is a subtype of the function type A2 arrow B2 if B1 is a subtype of B2, so that's covariant. The arguments there it goes in the other way, A2 must be a subtype of A1. Functions are contravariant in their argument types and covariant in their result type. That means that if we want to come up with a revised definition of the Function1 trait that takes account of variance, we have to write it like this, Function1 is contravariant in its argument type T and covariant in its result type U. How does the compiler verify whether your variance annotations are correct? We've seen in the array example that the combination of covariance with certain operations is unsound. In this case, the problematic operation was the update operation on an array. If we turn array into a class and update into a method, it would look like this, class Array, let's assume it should be covariant, and then we have the update method that takes a parameter x of type T. The problematic combination here is the covariant type parameter T, which appears in parameter position of the method update. That led to problems, and that's probably something that the compiler should disallow. The Scala compiler will check that there are no problematic combinations when compiling our class with variants annotations. Roughly, covariant type parameters can only appear in method results, contravariant type parameters can only appear in method parameters, and invariant type parameters can appear anywhere. The precise rules are a bit more involved, but fortunately the Scala compiler checks them for us. Let's see where the rules work out for our function type that we've just defined with variances. Here is the function type. Here, T is contravariant and it appears only as a method parameter type. That's okay. U is covariant and appears only as the result type of the apply method. That's also okay, and the method checks out correctly. Let's get back to the previous implementation of lists. One shortcoming was that Nil had to be a class because it needed a type parameter that indicates the type of the list. Whereas we would prefer it to be an object. After all, there's only one empty list. Can we change that? The answer is yes because we can make list covariant. Here the essential modifications, we gifted this trait a covariant type parameter T. We have already seen that a list of apples is a subtype of list of fruit. Then the object empty can be a list of nothing. We can write object empty extends list of nothing. Now that's beautiful because it works on two different levels. On the one hand, we have nothing is a subtype of T. For any T and lists are covariants or list of nothing can be used for any list type T. On the other side, the type list of nothing really conveys the information that there's nothing in the list. List of nothing on the one hand says, well, there's nothing in the list, and on the other hand, ensures that, that object is a subtype of any list type that the user might care to give. So far we have followed in this course then no magic rule. Everything you see should be constructable from first principles and those principles are stated explicitly. Now we have enough machinery to apply the no magic rules to lists as well. We can show you how one could write lists as they're used in Scala interlibrary, at least roughly and approximately. Lists would be a trait with a covariant type parameter. This empty method would do a pattern match and say, well, if the list is nil, then it's true and otherwise it's false. The two string method is a little bit more involved. It would use a local function recur that takes a prefix and the list of elements that should be still converted to a string and it will turn the string. It would say, okay, if the list is empty, then I close my string with a closing parents. If it's non-empty, then I print the prefix, I print the head element, and I follow with recursive call to recur, where I say the new prefix that should follow x if there are more elements is a comma. I start the recurrence with the prefix list open parents. The first time we print an element, we printed after list open parents and every other time we print it after a comma. To continue, there would be a subclass and a sub-object of lists. The subclass is called cons. It's also called covariant. It's a case class and it has a head parameter of type T, and a tail parameter of type list of T. The other case is a case object nil that extends list of nothing, as we have seen. The final thing we need is cons as a method we want to write one cons nil has an infix method, and we do that by defining an extension method. We say, cons is written like this. It takes a first parameter x, which is of type T. A second parameter xs, which is of type list of T. Its right-hand side would be the class creation, which now is written prefix. That now here we create a new object with the parameters x and xs. Finally, we still need to implement calls like list of 1, 2, 3, things like that. For that, it turns out that we don't have quite the machinery yet to do it in general. Essentially we do a fallback and we just give you an overloaded apply method with several cases for essentially lists of parameters of smaller size. Apply without parameters would be Nil, of course. Apply with a single parameter would be x colon Nil, two parameter cases here and you would go on with apply methods as many are needed. Later on we'll see that we can do with just a single apply method that uses a vararg parameter. But to introduce viral parameters, we need a little bit more machinery for them, so for the moment, we should leave it at that. Sometimes we have to put in a bit of work to make a class covariant. For instance, consider adding a prepend method to list which prepends a given element, yielding a new list. A first implementation of prepend could look like this. We are in trait list with covariant type parameter T. Prepend takes an element of type T, returns a list of T, and is implemented by essentially calling a new cons node with the given element and the current list as the tail. But that doesn't work. What do you think? Why does the following code not type check? Because prepend turns list into immutable class and that violates variants. Because prepend fails variance checking in other ways, or because prepend's right-hand side contains a type error. Well, let's look at these possibilities. Prepend gives you back a new cons node. So no, there's no mutation involved. It doesn't turn list into immutable class. But it still fails variants checking. Why? Well, because the type parameter T is used as the parameter of the prepend method, and we have seen that's legal only for contravariant type parameters or invariant type parameters, but T is declared covariant. That's a variance error. Is the compiler overzealous in rejecting prepend? In fact, no. Indeed, the compiler is right to throw out list with prepend because it violates the LSP, the Liskov Substitution Principle. How? Well, here's something one can do with a list xs of type, List of fruit. Prepend an orange. You would still get a list of fruit. At the same operation on a list ys of type, List of apple would lead to a type error. If I do the same operation, then the compiler would complain and said, "Well, I required an apple, but I found an orange." I can't prepend an orange to a list of apples, which means list of apple cannot be a subtype of a list of fruit. But you might argue, prepend is such a natural method to have on immutable lists. Is there a way to make it variance correct? In fact, yes, we can use a lower bound. We can write prepend like this. Prepend takes a new type parameter U, which is now a supertype of T and an alum of type U and returns a list of U and it has the same body with the cons as before. This passes variance checks, because covariant type parameters may appear in lower bounds of method type parameters, whereas contravariant type parameters may appear only in upper bounds. In essence we have a double flip there. We go to a parameter, but then we go to the lower bound of that parameter and that turns variance around again and makes covariant parameters legal to appear. But is this more than just a trick to make the variance checker happy? In fact, yes, we have added some useful functionality to prepend. If we look at prepend again, the question is, what is the result type of this function? We take a list of apples and an orange, and we call xs prepend x. Previously, that wouldn't have worked because we said, I can only prepend an apple to a list of apples, but now I can prepend an orange. What do I get? Well, it does type check, and I get a list of Apple. No. The list would contain an orange. A list of orange? Surely not. A list of fruit? Yes. That's the answer. I would obtain a list of fruit. How does that work? Well, I'm in a list of apples. I get the element which is an orange. The compiler will have to find a type U, which is a supertype of apple and can take an orange. The smallest such type is indeed fruit. The compiler will instantiate my type U with fruit and that's the result type list of fruit that I get back. But there's a simpler way to obtain the same functionality. We've seen it already. It's called extension methods. In fact, the need for a lower bound was essentially to decouple the parameter of the class and the parameter of the newly created object. Could be a fruit where the class was list of apple. Using an extension method such that the cons method that we've already constructed sidesteps the problem and leads to the same result. If we define an extension method here, then the compiler will also instantiate the type T to be a supertype of the type of the element that we prepend and the type of the tail that we pass here. If we prepend, an orange to a list of apples we would again, obtain a list of fruit.
