Variance in Generics

Variance in generics refers to how subtyping relationships behave when they are wrapped in a generic container or function. E.g. if Cat is a subclass of Animal, then subtype polymorphism which you may already be familiar with, explains how a Cat can transform (morph) into, and be used i place of an Animal. In a similar way, variance tells us how e.g a set[Cat] can transform (vary) and be used in place of a set[Animal] or vice versa.

There are three types of variance:

Covariance enables you to use a more specific type than originally specified. Example: If Cat is a subclass of Animal, you can assign an instance of Iterable[Cat] to a variable of type Iterable[Animal].
Contravariance enables you to use a more general type than originally specified. Example: If Cat is a subclass of Animal, you can assign an instance of Callable[[Animal], None] to a variable of type Callable[[Cat], None].
Invariance means that you can only use the type originally specified. An invariant generic type parameter is neither covariant nor contravariant. Example: If Cat is a subclass of Animal, you cannot assign an instance of list[Animal] to a variable of type list[Cat] or vice versa.

If we look at Python types, there are already many types and combinations of types that have different variance:

Function types i.e. callables are covariant on their return type, and contravariant on their arguments.
Mutable containers like list and dict are invariant.
Immutable containers like tuple and set are covariant.
Union types are covariant. This means that optional types are also covariant because they are equivalent to T | None.

Understanding variance helps you writing more flexible and type-safe code, especially when working with container types, generics and inheritance.

Covariance

Covariance (co- = together) means the subtype relationship goes in the same direction i.e. transform (vary) together with the wrapped type. For example, if Cat is a subtype of Animal, then set[Cat] is a subtype of set[Animal]. The type parameter varies together and in the same direction as the inheritance relationship.

As we mentioned in the introduction, covariance is a type of variance that allows you to use a more specific type than originally specified. For example, if Cat is a subclass of Animal, you can assign an instance of Iterable[Cat] to a variable of type Iterable[Animal], and if you have a method that takes an Iterable[Animal], you can safely pass in an Iterable[Cat].

The definition is that a generic type GenericType[T] is covariant in type parameter T if:

Derived is subtype of Base.
GenericType[Derived] is a subtype of GenericType[Base]

Examples of covariant types in Python are Iterable, set, tuple, and return types of callables.

Before we start we need to import a few types that we will use in the examples.

from abc import abstractmethod
from collections.abc import Callable, Iterable
from typing import Generic, TypeVar

Let's define a few classes. We will use Animal as a base class, and define Cat and Dog as subclasses of Animal.

class Animal:
    @abstractmethod
    def say(self) -> str: ...


class Cat(Animal):
    def say(self) -> str:
        return "Meow"


class Dog(Animal):
    def say(self) -> str:
        return "Woof"

Let's first see how this works with basic assignments.

cats = [Cat(), Cat()]
xs: Iterable[Animal] = cats  # Ok
ys: list[Animal] = cats  # Error: ...
# Type parameter "_T@list" is invariant, but "Cat" is not the same as "Animal"

So we see for the first assignment everything is ok, since the type of xs is Iterable[Animal], which is indeed a subtype of Iterable[Cat].

But for the second assignment, we get an error. This is because list[Animal] is invariant, and list[Cat] is not a subtype of list[Animal].

The problem is that lists are mutable. Think for a minute what would happen if we appended a Dog to the list ys. This is fine for ys since the type is list[Animal], but this means that cats would also be modified and would now contain a Dog, which is not allowed and should come as a surprise to any code still using cats.

cats: list[Cat] = [Cat(), Cat()]
ys: list[Animal] = cats  # type: ignore
ys.append(Dog())

print([cat.say() for cat in cats])
# Output: ['Meow', 'Meow', 'Woof']

Covariance And Function Return Types

Now let's see how this works with function return types for callables. We will define some "getter" functions that returns an instance of Animal or Cat, and also a "setter" function just to show that this does not work.

def get_animal() -> Animal:
    return Cat()  # Cat, but returned as Animal


def get_animals() -> Iterable[Animal]:
    return iter([Cat()])


def get_cat() -> Cat:
    return Cat()


def set_cat(cat: Cat) -> None:
    pass


def get_cats() -> Iterable[Cat]:
    return iter([Cat()])


# Cat -> Animal
var1: Animal = Cat()  # Ok, polymorphism
var2: Callable[[], Animal] = get_cat  # Ok, covariance,
var3: Callable[[], Iterable[Animal]] = get_cats  # Ok, covariance
var4: Callable[[Animal], None] = set_cat  # Error: ...
# Parameter 1: type "Animal" is incompatible with type "Cat"

The first assignment of var1 is just normal polymorphism. This is just to show the similarity between polymorphism and covariance. For the second and third assignments we see that covariance works for return types in callables since a function that returns a Cat is compatible with a function that returns an Animal.

For the last assignment, we get an error, since set_cat is a function that takes a Cat, and set_cat is not compatible with a function that takes an Animal. This is because callables are not covariant on parameter types.

In the next example, we will see how this works when assigning a general type e.g. Animal to a more specific type e.g Cat.

# Animal -> Cat
var5: Cat = Animal()  # Error: "Animal" is incompatible with "Cat"
var6: Callable[[], Cat] = get_animal  # Error: ...
var7: Callable[[], Iterable[Cat]] = get_animals  # Error: ...
# Type parameter "_T_co@Iterable" is covariant, but "Animal" is not a subtype of "Cat"

For the first assignment of var5, we get an error, since Animal is not a subtype of Cat. This is because a function that returns an Animal is not compatible with a function that returns a Cat. This is because the function might return a Dog.

For the second assignment of var6, and third assignments of var7, we also get errors, since Animal is not a subtype of Cat, hence a Dog might be returned from get_animal or get_animals which is incompatible with Cat.

Custom Covariant Generic Classes

We can also define our own covariant generic classes. Let's see how this works. To define a covariant generic class, we need to use the covariant keyword argument for the T_co type variable.` This is by the way similar to how we declared type variables before Python 3.12.

We will make a Rabbit class that is a subclass of Animal, and a Hat class that is covariant in its type parameter. This means that a Hat[Rabbit] is a subtype of Hat[Animal].

Let's see how this looks in code.

T_co = TypeVar("T_co", covariant=True)


class Rabbit(Animal):
    def say(self) -> str:
        return "Squeak"


class Hat(Generic[T_co]):
    def __init__(self, value: T_co) -> None:
        self.value = value

    def pull(self) -> T_co:
        return self.value

One way to think about covariance is that covariant types are "out" types and can only be used as return types. If a class were to allow inserting and setting values of the generic type, it would violate the principle of covariance and could lead to type safety issues.

Let's see what happens if we try to add a method that takes the generic type as input for a class with the covariant type variable.

class InvalidHat(Hat[T_co]):
    """Just to show that in parameters are not allowed"""

    def put(self, value: T_co) -> None:  # Error: Covariant type variable cannot be used in parameter type
        self.value = value

As we see in the example above, we get an error when we try to add a method that takes the generic type in the parameter. This is because covariant types can only be used as return types. If a class were to allow inserting and setting values of the generic type, it could lead to type safety issues.

But wait a minute. The constructor or the initializer __init__ method is taking the generic type as an argument. Why isn't that a problem? The reason why this is okay is that the object is being created at that point, and the type is being established. Once the object is created, its type is fixed and won't change.

But for abstract covariant classes or protocols that do not have constructors, we can only use the generic type as on out type, i.e. only in the return type of a method.

hat_with_animal: Hat[Animal] = Hat[Rabbit](Rabbit())


def fetch_rabbit_hat() -> Hat[Rabbit]:
    return Hat(Rabbit())


fetch_animal_hat: Callable[[], Hat[Animal]] = fetch_rabbit_hat

In the example above we defined Hat[Rabbit] and assign it to a variable that has the type Hat[Animal]. This would have given an error if the generic type T_co was not covariant.

Note: we specify Hat[Rabbit](Rabbit()) to avoid that the constructor uses polymorphism from Rabbit to Animal so we do create a Hat[Rabbit] and not a Hat[Animal].

Covariance summarized

Covariance is in may ways similar to polymorphism in the way we think about and use the type in our code. We use it for "out" types we have in our methods, i.e. methods that returns the generic type like Iterable, Set, Tuple, and also return types of Callable.

When we define a type to be covariant, we are able to assign a container i.e. generic class of e.g. Rabbit, to a variable that is annotated as a generic class of Animal. This would not have been possible if the type was not covariant.

Contravariance

Contravariance (contra- = against/opposite) means the subtype relationship goes in the opposite direction. If Cat is a subtype of Animal, then e.g Observer[Animal] is a subtype of Observer[Cat]. The type parameter varies in the opposite direction from the inheritance relationship of the wrapped type.

As mentioned introduction, contravariance is a type of variance that allows you to use a more general type in place of a more specific type. This might sound counterintuitive at first. It usually goes against what you would expect, and it's safe to say that this is something most developers don't know about.

In Python, contravariance is typically experienced for function arguments in callables, or push-based containers such as observables (RxPY). This means that it might help to think in terms of callbacks when you try to understand contravariance. This is because callbacks are usually functions that usually takes one or more arguments and returns nothing.

The definition is that a generic type GenericType[T] is contravariant in type parameter T if:

Derived is subtype of Base.
GenericType[Base] is a subtype of GenericType[Derived].

Examples of contravariant types in Python are callables, and function arguments. The Observer class in RxPY and similar classes using the "consumer"" pattern e.g (send, throw, close) style of methods that take generic type T as an argument and return nothing.

First, we need to import a few types that we will use in the examples.

from abc import abstractmethod
from collections.abc import Callable, Iterable
from typing import Generic, TypeVar

Let's define a few classes, the same as we used with covariance so we can compare the two. We will use Animal as a base class, and define Cat and Dog as subclasses of Animal.

class Animal:
    @abstractmethod
    def say(self) -> str: ...


class Cat(Animal):
    def say(self) -> str:
        return "Meow"


class Dog(Animal):
    def say(self) -> str:
        return "Woof"

Example: Function Arguments

Let's see how this works with function arguments for callables. Callables are generic types that are contravariant in their argument types, e.g. you can assign an instance of Callable[[Animal], None] to a variable of type Callable[[Cat], None]

We can define a few setter functions that takes an argument of type Animal or Cat and returns nothing. These are the opposites of the getter functions we defined for covariance.

def set_animal(animal: Animal) -> None:
    pass


def set_animals(animals: Iterable[Animal]) -> None:
    pass


def set_cat(cat: Cat) -> None:
    pass


def set_cats(cats: Iterable[Cat]) -> None:
    pass


def get_animal() -> Animal:
    return Cat()  # Cat, but returned as Animal


# Cat -> Animal
var1: Animal = Cat()  # Ok, polymorphism
# This works because a function that takes a Cat is compatible with a function that
# takes an Animal.
var2: Callable[[Cat], None] = set_animal  # Ok, since Callable is contravariant for arguments
var3: Callable[[Iterable[Cat]], None] = set_animals  # Ok, contravariance
var4: Callable[[], Cat] = get_animal  # Error: "Animal" is incompatible with "Cat"

We start in a similar way as we did with covariance. We see that for the first assignment, everything is ok, since the type of var1 is Animal, which is a base class of Cat. This is normal polymorphism.

For the second and third assignments, we start to see how contravariance works for function arguments. This works because a function that takes an Animal can be assigned to a variable that is annotated as a callable that takes a Cat. We can always call a callback that takes an Animal with a Cat.

For the last assignment, we get an error, since get_animal is a function that returns an Animal, and get_animal is not compatible with a function that returns a Cat. This is because callables are not contravariant on return types.

# Animal -> Cat
var5: Cat = Animal()  # Error: "Animal" is incompatible with "Cat"
# We get an error here because a function that takes an Animal is not compatible with
# a function that takes a Cat. This is because the function might take a Dog.
var6: Callable[[Animal], None] = set_cat  # Error: ...
var7: Callable[[Iterable[Animal]], None] = set_cats  # Error: ...
# (*) Type parameter "_T_co@Iterable" is covariant, but "Animal" is not a subtype of "Cat"

For the first assignment, we get an error, since Animal is not a subtype of Cat. For the second and third assignments, we get an error because Animal is not a subtype of Cat. If you think about the callable as a callback, then it's easier to see that you cannot give an Animal e.g. a Dog to a function that takes a Cat.

Custom Contravariant Generic Classes

We can also define our own contravariant generic classes, similar to how we made covariant classes. To define a contravariant generic class, we need to use the contravariant keyword argument for the T_contra type variable.` This is by the way similar to how we declared type variables before Python 3.12.

We will make a Rabbit class that is a subclass of Animal, and a Hat class that is contravariant in its type parameter. This means that a Hat[Animal] is a subtype of Hat[Rabbit].

Let's see how this looks in code.

T_contra = TypeVar("T_contra", contravariant=True)


class Rabbit(Animal):
    def say(self) -> str:
        return "Squeak"


class Hat(Generic[T_contra]):
    def __init__(self, value: T_contra) -> None:
        self.value = value

    def put(self, value: T_contra) -> None:
        self.value = value


class Callable_(Generic[T_contra]):
    def __call__(self, value: T_contra) -> None: ...

Let's see what happens if we try to add a method that returns the generic type for a class with a contravariant type variable.

class InvalidHat(Hat[T_contra]):
    def __init__(self, value: T_contra) -> None:
        self.value = value

    def get(self) -> T_contra:  # Error: Contravariant type variable cannot be used as a return type
        return self.value

As we see in the example above, we get an error when we try to add a method that returns the generic type. This is because contravariant types can only be used as function argument types. If a class were to allow returning values of the generic type, it could lead to type safety issues.

One way to think about contravariance is that contravariant types are "in" types and can only be used as function argument types. If a class were to allow returning values of the generic type, it would violate the principle of contravariance and could lead to type safety issues.

We see that we get the opposite of what we saw with covariance. The get_value method now has an error, since we cannot return a value of type T_contra. But the set_value method works, since we can set the value to a value of type T_contra.

animal: Animal = Rabbit()
hat_with_rabbit: Hat[Rabbit] = Hat[Animal](animal)


def fetch_animal_hat() -> Hat[Animal]:
    return Hat(Rabbit())


fetch_rabbit_hat: Callable[[], Hat[Rabbit]] = fetch_animal_hat

In the example above we defined Hat[Animal] and assign it to a variable that has the type Hat[Rabbit]. This would have given an error if the generic type T_contra was not contravariant.

Summary

Contravariance is the opposite of covariance, and this makes it quite a bit harder to understand since Hat[Rabbit] is not a subtype of Hat[Animal] perhaps as you might expect. It is actually the other way around. With contravariance Hat[Animal] becomes a subtype of Hat[Rabbit].

When we define a type to be contravariant, we are able to assign a container i.e. generic class of e.g. Animal, to a variable that is annotated as a generic class of Rabbit. This would not have been possible if the type was not contravariant.

Invariance in Generics

Invariance (in- = un/not) means that the type is not variant, and will not transform (vary) together with the wrapped type. This means that you can use only the type originally specified, and neither a more specific nor a more general type. This is the default behavior for generic types in Python.

An invariant generic type parameter is neither covariant nor contravariant. You cannot assign an instance of Hat[Animal] to a variable of type Hat[Rabbit] or vice versa.

from abc import abstractmethod


class Animal:
    @abstractmethod
    def render(self) -> str: ...


class Rabbit(Animal):
    def render(self) -> str:
        return "Rabbit!"


class Hat[T]:
    @abstractmethod
    def put(self, value: T) -> None: ...

    @abstractmethod
    def pull(self) -> T: ...


class RabbitHat(Hat[Rabbit]):
    def put(self, value: Rabbit) -> None:
        print(f"Putting {value.render()} in the hat")

    def pull(self) -> Rabbit:
        """Pull a Rabbit out of the hat"""
        return Rabbit()


# This will not work due to invariance
animal_hat: Hat[Animal] = RabbitHat()  # Error: ...
# Type parameter "T@Hat" is invariant, but "Rabbit" is not the same as "Animal"

# This also will not work due to invariance
rabbit_hat: Hat[Rabbit] = Hat[Animal]()  # Error: ...
# Type parameter "T@Hat" is invariant, but "Animal" is not the same as "Rabbit"

# This is the only valid assignment
rabbit_hat: Hat[Rabbit] = RabbitHat()

# We can only put Rabbits in a RabbitHat
rabbit_hat.put(Rabbit())

# This will not work, even though Rabbit is a subclass of Animal
rabbit_hat.put(Animal())  # Error: ...
# "Animal" is incompatible with "Rabbit"

# We can only take Rabbits from a RabbitHat
rabbit: Rabbit = rabbit_hat.pull()

Invariance restricts us to use exactly the type specified. This happens when we use the generic type as both "in" and "out" types, meaning that methods of the type use the generic type both in the parameters, and return types.

This hints that the generic type may be some kind of mutable container and we cannot allow assigning a Hat[Animal] to a Hat[Rabbit] or vice versa since that could easily lead to code adding a Cat into a Hat[Rabbit].

Python Type Annotations (part 3)

Table of contents