Type checking vs runtime
After writing this, I finally understood @Alexander point in first comment: whatever you write in annotations, it does not affect runtime, and your code is executed in the same way (sorry, I missed that you're looking just not from type checking perspective). This is core principle of python typing, as opposed to strongly typed languages (which makes it wonderful IMO): you can always say "I don't need types here - save my time and mental health". Type annotations are used to help some third-party tools, like mypy (type checker maintained by python core team) and IDEs. IDEs can suggest you something based on this information, and mypy checks whether your code can work if your types match the reality.
Generic version
T = TypeVar('T')
class Stack(Generic[T]):
def __init__(self) -> None:
self.items: list[T] = []
def push(self, item: T) -> None:
self.items.append(item)
def pop(self) -> T:
return self.items.pop()
def empty(self) -> bool:
return not self.items
You can treat type variables like regular variables, but intended for "meta" usage and ignored (well, there are some runtime traces, but they exist primary for introspection purpose) on runtime. They are substituted once for every binding context (more about it - below), and can be defined only once per module scope.
The code above declares normal generic class with one type argument. Now you can say Stack[int] to refer to a stack of integers, which is great. Current definition allows either explicit typing or using implicit Any parametrization:
# Explicit type
int_stack: Stack[int] = Stack()
reveal_type(int_stack) # N: revealed type is "__main__.Stack[builtins.int]
int_stack.push(1) # ok
int_stack.push('foo') # E: Argument 1 to "push" of "Stack" has incompatible type "str"; expected "int" [arg-type]
reveal_type(int_stack.pop()) # N: revealed type is "builtins.int"
# No type results in mypy error, similar to `x = []`
any_stack = Stack() # E: need type annotation for any_stack
# But if you ignore it, the type becomes `Stack[Any]`
reveal_type(any_stack) # N: revealed type is "__main__.Stack[Any]
any_stack.push(1) # ok
any_stack.push('foo') # ok too
reveal_type(any_stack.pop()) # N: revealed type is "Any"
To make the intended usage easier, you can allow initialization from iterable (I'm not covering the fact that you should be using collections.deque instead of list and maybe instead of this Stack class, assuming it is just a toy collection):
from collections.abc import Iterable
class Stack(Generic[T]):
def __init__(self, items: Iterable[T] | None) -> None:
# Create an empty list with items of type T
self.items: list[T] = list(items or [])
...
deduced_int_stack = Stack([1])
reveal_type(deduced_int_stack) # N: revealed type is "__main__.Stack[builtins.int]"
To sum up, generic classes have some type variable bound to the class body. When you create an instance of such class, it can be parametrized with some type - it may be another type variable or some fixed type, like int or tuple[str, Callable[[], MyClass[bool]]]. Then all occurrences of T in its body (except for nested classes, which are perhaps out of "quick glance" explanation context) are replaced with this type (or Any, if it is not given and cannot be deduced). This type can be deduced iff at least one of __init__ or __new__ arguments has type referring to T (just T or, say, list[T]), and otherwise you have to specify it. Note that if you have T used in __init__ of non-generic class, it is not very cool, although currently not disallowed.
Now, if you use T in some methods of generic class, it refers to that replaced value and results in typecheck errors, if passed types are not compatible with expected.
You can play with this example here.
Working outside of generic context
However, not all usages of type variables are related to generic classes. Fortunately, you cannot declare generic function with possibility to declare generic arg on calling side (like function<T> fun(x: number): T and fun<string>(0)), but there is enough more stuff. Let's begin with simpler examples - pure functions:
T = TypeVar('T')
def func1() -> T:
return 1
def func2(x: T) -> int:
return 1
def func3(x: T) -> T:
return x
def func4(x: T, y: T) -> int:
return 1
First function is declared to return some value of unbound type T. It obviously makes no sense, and recent mypy versions even learned to mark it as error. Your function return depends only on arguments and external state - and type variable must be present there, right? You cannot also declare global variable of type T in module scope, because T is still unbound - and thus neither func1 args nor module-scoped variables can depend on T.
Second function is more interesting. It does not cause mypy error, although still makes not very much sense: we can bind some type to T, but what is the difference between this and func2_1(x: Any) -> int: ...? We can speculate that now T can be used as annotation in function body, which can help in some corner case with type variable having upper bound parameterizing an invariant class used covariantly (imagine needing a list[Parent] with .count() method and __getitem__, but still allowing list[Child] to be passed in and ignoring both Sequence existence and custom protocol option). Similar example is even explicitly referenced in PEP as valid.
The third and fourth functions are typical examples of type variables in functions. The third declares function returning the same type as it's argument.
The fourth function takes two arguments of the same type (arbitrary one). It is more useful if you have T = TypeVar('T', bound=Something) or T = TypeVar('T', str, bytes): you can concatenate two arguments of type T, but cannot - of type str | bytes, like in the below example:
T = TypeVar('T', str, bytes)
def total_length(x: T, y: T) -> int:
return len(x + y)
The most important fact about all examples above in this section: T doesnot have to be the same for different functions. You can call func3(1), then func3(['bar']) and then func4('foo', 'bar'). T is int, list[str] and str in these calls - no need to match.
With this in mind your second solution is clear:
T = TypeVar('T')
class Stack:
def __init__(self) -> None:
# Create an empty list with items of type T
self.items: list[T] = [] # E: Type variable "__main__.T" is unbound [valid-type]
def push(self, item: T) -> None:
self.items.append(item)
def pop(self) -> T: # E: A function returning TypeVar should receive at least one argument containing the same TypeVar [type-var]
return self.items.pop()
Here is mypy issue, discussing similar case.
__init__ says that we set attribute x to value of type T, but this T is lost later (T is scoped only within __init__) - so mypy rejects the assignment.
push is ill-formed and T has no meaning here, but it does not result in invalid typing situation, so is not rejected (type of argument is erased to Any, so you still can call push with some argument).
pop is invalid, because typechecker needs to know what my_stack.pop() will return. It could say "I give up - just have your Any", and will be perfectly valid (PEP does not enforce this). but mypy is more smart and denies invalid-by-design usage.
Edge case: you can return SomeGeneric[T] with unbound T, for example, in factory functions:
def make_list() -> list[T]: ...
mylist: list[str] = make_list()
because otherwise type argument couldn't have been specified on calling site
For better understanding of type variables and generics in python, I suggest you to read PEP483 and PEP484 - usually PEPs are more like a boring standard, but these are really good as a starting point.
There are many edge cases omitted there, which still cause hot discussions in mypy team (and probably other typecheckers too) - say, type variables in staticmethods of generic classes, or binding in classmethods used as constructors - mind that they can be used on instances too. However, basically you can:
- have a TypeVar bound to class (
GenericorProtocol, or someGenericsubclass - if you subclassIterable[T], your class is already generic inT) - then all methods use the sameTand can contain it in one or both sides - or have a method-scoped/function-scoped type variable - then it's useful if repeated in the signature more than once (not necessary "clean" - it may be parametrizing another generic)
- or use type variables in generic aliases (like
LongTuple = tuple[T, T, T, T]- then you can dox: LongTuple[int] = (1, 2, 3, 4) - or do something more exotic with type variables, which is probably out of scope
how to define python generic classes - Stack Overflow
When to make classes & functions generic in python?
PEP 646 (Variadic Generics) has been accepted for Python 3.11
Let's talk about python 3.12's new Type Parameter Syntax.
I have been generally making my methods generic mostly for the following reasons:
I am subclassing an abstract class and want to override a method and narrow the type hinting in the arguments, which would otherwise violate the Liskov substitution principle
I am not subclassing/overriding, but would like return values, attributes, etc. of a class to be more narrow than the type hints it is currently bound to, since I may be using that class in many different places with different types.
In particular for the second case, I have realized that there are actually two approaches to tackle this:
make the class generic and provide the type arguments for the instance type hints.
do not make the class generic, but subclass simply for the purpose of updating the type hints
With projects I am working on, with "context" and "manager" classes, there can be many different attributes and methods with many different types (5+), hence, making the class generic on all of them is too verbose. If I do make a class generic, if any other attributes (containing instances of other classes) return those same generic types, I have to propagate the generic type down the entire chain when I am using composition instead of inheritance. This is something I would like to avoid. If I choose option 2, there would be an explosion of subclasses just to override the type hints.
When should I choose 1 or 2? Is there a better way to do this?
Option 1 example:
from typing import Self, TypeVar, Generic
BazT = TypeVar('BazT')
class Bar(Generic[BazT]):
def method1(self: Self) -> BazT:
...
class Foo(Generic[BazT]):
bar: Bar[BazT]
def method1(self: Self, baz: BazT) -> None:
...
def method2(self: Self) -> Bar[BazT]:
...Option 2 example:
from typing import Self, Any, TypeVar
Baz = Any
class Bar:
def method1(self: Self) -> Baz:
...
class Foo:
bar: Bar
def method1(self: Self, baz: Baz) -> None:
...
def method2(self: Self) -> Bar:
...
BazNarrowed = TypeVar('BazNarrowed', bound=Baz) # (doesn't matter what this is just some more narrow type)
class BarSubclass(Bar):
def method1(self: Self) -> BazNarrowed:
...
class FooSubclass(Foo):
bar: BarSubclass
def method1(self: Self, baz: BazNarrowed) -> None:
...
def method2(self: Self) -> BarSubclass:
...