I'm guessing you wrote code that looked roughly like this?
class Matrix:
T = TypeVar('T')
def _matrix_map(self, mapper: Callable[[Tile], T]) -> Dict[Coordinate, T]:
# ...snip...
There's actually no issue with this at runtime/no issue for mypy, but it seems pylint isn't happy with you putting that "T" inside the scope of the class definition. What you can do to satisfy pylint is to instead do this:
T = TypeVar('T')
class Matrix:
def _matrix_map(self, mapper: Callable[[Tile], T]) -> Dict[Coordinate, T]:
# ...snip...
This is exactly equivalent to the former code snippet, and will also satisfy pylint.
(As a tangent, I'd argue that the fact that pylint isn't happy with the former is actually a bug. To be fair though, there aren't really any established style guides on best practices for type hints yet/putting a typevar inside of a nested scope is a somewhat uncommon use case, so I guess I can't really blame them for not accounting for this case.)
The only downside is that having that TypeVar on the outside does seem vaguely untidy/seems like it'll pollute the global namespace. You can partially mitigate this by renaming it to something like _T for example, to indicate it's private. You can also reuse this typevar whenever you need another generic class or function, which should also hopefully help keep the namespace pollution down.
For more information about using generics with Python type hints, see http://mypy.readthedocs.io/en/stable/generics.html
Answer from Michael0x2a on Stack OverflowSpecifying type for Generic Function
Simplifying Parametrized Type Hints for Generic Functions in Python - Typing - Discussions on Python.org
Generics, Type[T], and TypeVarTuple
python - Use of Generic and TypeVar - Stack Overflow
Videos
I'm guessing you wrote code that looked roughly like this?
class Matrix:
T = TypeVar('T')
def _matrix_map(self, mapper: Callable[[Tile], T]) -> Dict[Coordinate, T]:
# ...snip...
There's actually no issue with this at runtime/no issue for mypy, but it seems pylint isn't happy with you putting that "T" inside the scope of the class definition. What you can do to satisfy pylint is to instead do this:
T = TypeVar('T')
class Matrix:
def _matrix_map(self, mapper: Callable[[Tile], T]) -> Dict[Coordinate, T]:
# ...snip...
This is exactly equivalent to the former code snippet, and will also satisfy pylint.
(As a tangent, I'd argue that the fact that pylint isn't happy with the former is actually a bug. To be fair though, there aren't really any established style guides on best practices for type hints yet/putting a typevar inside of a nested scope is a somewhat uncommon use case, so I guess I can't really blame them for not accounting for this case.)
The only downside is that having that TypeVar on the outside does seem vaguely untidy/seems like it'll pollute the global namespace. You can partially mitigate this by renaming it to something like _T for example, to indicate it's private. You can also reuse this typevar whenever you need another generic class or function, which should also hopefully help keep the namespace pollution down.
For more information about using generics with Python type hints, see http://mypy.readthedocs.io/en/stable/generics.html
With the introduction of type parameter lists in Python 3.12, it is no longer necessary to explicitly define a TypeVar. (Note also that typing.Dict was deprecated in Python 3.9 since dict itself can used instead.)
from collections.abc import Callable
class Matrix:
def _matrix_mapT -> dict[Coordinate, T]:
...
If the type parameter should be shared by all methods, it can also be declared at the class level.
class Matrix[T]:
def _matrix_map(self, mapper: Callable[[Tile], T]) -> dict[Coordinate, T]:
...
Type variables are literally "variables for types". Similar to how regular variables allow code to apply to multiple values, type variables allow code to apply to multiple types.
At the same time, just like code is not required to apply to multiple values, it is not required to depend on multiple types. A literal value can be used instead of variables, and a literal type can be used instead of type variables โ provided these are the only values/types applicable.
Since the Python language semantically only knows values โ runtime types are also values โ it does not have the facilities to express type variability. Namely, it cannot define, reference or scope type variables. Thus, typing represents these two concepts via concrete things:
- A
typing.TypeVarrepresents the definition and reference to a type variable. - A
typing.Genericrepresents the scoping of types, specifically to class scope.
Notably, it is possible to use TypeVar without Generic โ functions are naturally scoped โ and Generic without TypeVar โ scopes may use literal types.
Consider a function to add two things. The most naive implementation adds two literal things:
def add():
return 5 + 12
That is valid but needlessly restricted. One would like to parameterise the two things to add โ this is what regular variables are used for:
def add(a, b):
return a + b
Now consider a function to add two typed things. The most naive implementations adds two things of literal type:
def add(a: int, b: int) -> int:
return a + b
That is valid but needlessly restricted. One would like to parameterise the types of the two things to add โ this is what type variables are used for:
T = TypeVar("T")
def add(a: T, b: T) -> T:
return a + b
Now, in the case of values we defined two variables โ a and b but in the case of types we defined one variable โ the single T โ but used for both variables! Just like the expression a + a would mean both operands are the same value, the annotation a: T, b: T means both parameters are the same type. This is because our function has a strong relation between the types but not the values.
While type variables are automatically scoped in functions โ to the function scope โ this is not the case for classes: a type variable might be scoped across all methods/attributes of a class or specific to some method/attribute.
When we define a class, we may scope type variables to the class scope by adding them as parameters to the class. Notably, parameters are always variables โ this applies to regular parameters just as for type parameters. It just does not make sense to parameterise a literal.
# v value parameters of the function are "value variables"
def mapping(keys, values):
...
# v type parameters of the class are "type variables"
class Mapping(Generic[KT, VT]):
...
When we use a class, the scope of its parameters has already been defined. Notably, the arguments passed in may be literal or variable โ this again applies to regular arguments just as for type arguments.
# v pass in arguments via literals
mapping([0, 1, 2, 3], ['zero', 'one', 'two', 'three'])
# v pass in arguments via variables
mapping(ks, vs)
# v pass in arguments via literals
m: Mapping[int, str]
# v pass in arguments via variables
m: Mapping[KT, VT]
Whether to use literals or variables and whether to scope them or not depends on the use-case. But we are free to do either as required.
The whole purporse of using Generic and TypeVar (here represented as the X and Y variables) is when one wants the parameters to be as generic as possible. int can be used, instead, in this case. The difference is: the static analyzer will interpret the parameter as always being an int.
Using generics mean the function accepts any type of parameter. The static analyzer, as in an IDE for instance, will determine the type of the variables and the return type as the arguments are provided on function call or object instantiation.
mapping: Mapping[str, int] = {"2": 2, "3": 3}
name = lookup_name(mapping, "1", 1)
In the above example type checkers will know name will always be an int relying on the type annotations. In IDEs, code completion for int methods will be shown as the 'name' variable is used.
Using specific types is ideal if that is your goal. The function accepting only a map with int keys or values, and/or returning int in this case, for instance.
As X and Y are variable you can choose any name want, basically.
Below example is possible:
def lookup_name(mapping: Mapping[str, int], key: str, default: int) -> int:
try:
return mapping[key]
except KeyError:
return default
The types are not generic in the above example. The key will always be str; the default variable, the value, and the return type will always be an int. It's the programmer's choice. This is not enforced by Python, though. A static type checker like mypy is needed for that.
The Generic type could even be constrained if wanted:
import typing
X = typing.TypeVar("X", int, str) # Accept int and str
Y = typing.TypeVar("Y", int) # Accept only int
@MisterMiyagi's answer offers a thorough explanation on the use scope for TypeVar and Generic.
According to PEP-484, the right way to do this is to use Type[T] for the argument:
from typing import TypeVar, Type
T = TypeVar('T')
def my_factory(some_class: Type[T]) -> T:
instance_of_some_class = some_class()
return instance_of_some_class
It however seems like my editor does not (yet) support this.
Python 3.12 introduces type parameter lists, which allows generic functions to be defined without explicit TypeVars. Moreover, typing.Type was deprecated in Python 3.9 in favor of using type directly.
The function can be rewritten like so:
def my_factoryT -> T:
return some_class()
Not sure what your question is, the code you posted is perfectly valid Python code. There is typing.Type that does exactly what you want:
from typing import Type, TypeVar
class Animal: ...
class Snake(Animal): ...
T = TypeVar('T', bound=Animal)
def make_animal(animal_type: Type[T]) -> T:
return animal_type()
reveal_type(make_animal(Animal)) # Revealed type is 'main.Animal*'
reveal_type(make_animal(Snake)) # Revealed type is 'main.Snake*'
See mypy output on mypy-play.
Since Python 3.12, generic function declarations can be simplified with type parameter lists, which obviates the need to explicitly define TypeVars. (Also note that typing.Type was deprecated in Python 3.9 in favor of directly subscripting type.)
def make_animalT -> T:
return animal_type()
Python >= 3.8
As of Python3.8 there is typing.get_args:
print( get_args( List[int] ) ) # (<class 'int'>,)
PEP-560 also provides __orig_bases__[n], which allows us the arguments of the nth generic base:
from typing import TypeVar, Generic, get_args
T = TypeVar( "T" )
class Base( Generic[T] ):
pass
class Derived( Base[int] ):
pass
print( get_args( Derived.__orig_bases__[0] ) ) # (<class 'int'>,)
Python >= 3.6
As of Python 3.6. there is a public __args__ and (__parameters__) field.
For instance:
print( typing.List[int].__args__ )
This contains the generic parameters (i.e. int), whilst __parameters__ contains the generic itself (i.e. ~T).
Python < 3.6
Use typing_inspect.getargs
Some considerations
typing follows PEP8. Both PEP8 and typing are coauthored by Guido van Rossum. A double leading and trailing underscore is defined in as: "โmagicโ objects or attributes that live in user-controlled namespaces".
The dunders are also commented in-line; from the official repository for typing we can see:
- "
__args__is a tuple of all arguments used in subscripting, e.g.,Dict[T, int].__args__ == (T, int)".
However, the authors also note:
- "The typing module has provisional status, so it is not covered by the high standards of backward compatibility (although we try to keep it as much as possible), this is especially true for (yet undocumented) dunder attributes like
__union_params__. If you want to work with typing types in runtime context, then you may be interested in thetyping_inspectproject (part of which may end up in typing later)."
I general, whatever you do with typing will need to be kept up-to-date for the time being. If you need forward compatible changes, I'd recommend writing your own annotation classes.
As far as I know, there is no happy answer here.
What comes to mind is the __args__ undocumented attribute which stores this information:
list_of_ints.__args__
>>>(<class 'int'>,)
str_to_bool_dict.__args__
>>>(<class 'str'>, <class 'bool'>)
but there is no mention of it in the documentation of the typing module.
It is worth noting that it was very close to be mentioned in the documentation though:
Probably we should also discuss whether we need to document all keyword arguments for
GenericMeta.__new__. There aretvars,args,origin,extra, andorig_bases. I think we could say something about first three (they correspond to__parameters__,__args__, and__origin__and these are used by most things in typing).
But it did not quite make it:
I added
GenericMetato__all__and added docstrings toGenericMetaandGenericMeta.__new__following the discussion in the issue. I decided not to describe__origin__and friends in docstrings. Instead, I just added a comment at the place where they are first used.
From there, you still have three non-mutually exclusive options:
wait for the
typingmodule to reach full maturity and hope these features will be documented soonjoin the Python ideas mailing list and see if enough support can be gathered to make these internals public / part of the API
work in the meantime with the undocumented internals, making a gamble that there won't be changes to these or that the changes will be minor.
Note that the third point can hardly be avoided as even the API can be subject to changes:
The typing module has been included in the standard library on a provisional basis. New features might be added and API may change even between minor releases if deemed necessary by the core developers.
I have been generally making my methods generic mostly for the following reasons:
-
I am subclassing an abstract class and want to override a method and narrow the type hinting in the arguments, which would otherwise violate the Liskov substitution principle
-
I am not subclassing/overriding, but would like return values, attributes, etc. of a class to be more narrow than the type hints it is currently bound to, since I may be using that class in many different places with different types.
In particular for the second case, I have realized that there are actually two approaches to tackle this:
-
make the class generic and provide the type arguments for the instance type hints.
-
do not make the class generic, but subclass simply for the purpose of updating the type hints
With projects I am working on, with "context" and "manager" classes, there can be many different attributes and methods with many different types (5+), hence, making the class generic on all of them is too verbose. If I do make a class generic, if any other attributes (containing instances of other classes) return those same generic types, I have to propagate the generic type down the entire chain when I am using composition instead of inheritance. This is something I would like to avoid. If I choose option 2, there would be an explosion of subclasses just to override the type hints.
When should I choose 1 or 2? Is there a better way to do this?
Option 1 example:
from typing import Self, TypeVar, Generic
BazT = TypeVar('BazT')
class Bar(Generic[BazT]):
def method1(self: Self) -> BazT:
...
class Foo(Generic[BazT]):
bar: Bar[BazT]
def method1(self: Self, baz: BazT) -> None:
...
def method2(self: Self) -> Bar[BazT]:
...Option 2 example:
from typing import Self, Any, TypeVar
Baz = Any
class Bar:
def method1(self: Self) -> Baz:
...
class Foo:
bar: Bar
def method1(self: Self, baz: Baz) -> None:
...
def method2(self: Self) -> Bar:
...
BazNarrowed = TypeVar('BazNarrowed', bound=Baz) # (doesn't matter what this is just some more narrow type)
class BarSubclass(Bar):
def method1(self: Self) -> BazNarrowed:
...
class FooSubclass(Foo):
bar: BarSubclass
def method1(self: Self, baz: BazNarrowed) -> None:
...
def method2(self: Self) -> BarSubclass:
...