Yes, there is a difference. Although in Python 3, all objects are instances of object, including object itself, only Any documents that the return value should be disregarded by the typechecker.
The Any type docstring states that object is a subclass of Any and vice-versa:
>>> import typing
>>> print(typing.Any.__doc__)
Special type indicating an unconstrained type.
- Any object is an instance of Any.
- Any class is a subclass of Any.
- As a special case, Any and object are subclasses of each other.
However, a proper typechecker (one that goes beyond isinstance() checks, and which inspects how the object is actually used in the function) can readily object to object where Any is always accepted.
From the Any type documentation:
Notice that no typechecking is performed when assigning a value of type
Anyto a more precise type.
and
Contrast the behavior of
Anywith the behavior ofobject. Similar toAny, every type is a subtype ofobject. However, unlikeAny, the reverse is not true: object is not a subtype of every other type.That means when the type of a value is
object, a type checker will reject almost all operations on it, and assigning it to a variable (or using it as a return value) of a more specialized type is a type error.
and from the mypy documentation section Any vs. object:
The type
objectis another type that can have an instance of arbitrary type as a value. UnlikeAny,objectis an ordinary static type (it is similar toObjectin Java), and only operations valid for all types are accepted for object values.
object can be cast to a more specific type, while Any really means anything goes and a type checker disengages from any use of the object (even if you later assign such an object to a name that is typechecked).
You already painted your function into a an un-typed corner by accepting list, which comes down to being the same thing as List[Any]. The typechecker disengaged there and the return value no longer matters, but since your function accepts a list containing Any objects, the proper return value would be Any here.
To properly participate in type-checked code, you need to mark your input as List[T] (a genericly typed container) for a typechecker to then be able to care about the return value. Which in your case would be T since you are retrieving a value from the list. Create T from a TypeVar:
from typing import TypeVar, List
T = TypeVar('T')
def get_item(L: List[T], i: int) -> T:
return L[i]
or, using Python 3.12 or newer:
def get_itemT -> T:
return L[i]
Answer from Martijn Pieters on Stack OverflowVideos
Yes, there is a difference. Although in Python 3, all objects are instances of object, including object itself, only Any documents that the return value should be disregarded by the typechecker.
The Any type docstring states that object is a subclass of Any and vice-versa:
>>> import typing
>>> print(typing.Any.__doc__)
Special type indicating an unconstrained type.
- Any object is an instance of Any.
- Any class is a subclass of Any.
- As a special case, Any and object are subclasses of each other.
However, a proper typechecker (one that goes beyond isinstance() checks, and which inspects how the object is actually used in the function) can readily object to object where Any is always accepted.
From the Any type documentation:
Notice that no typechecking is performed when assigning a value of type
Anyto a more precise type.
and
Contrast the behavior of
Anywith the behavior ofobject. Similar toAny, every type is a subtype ofobject. However, unlikeAny, the reverse is not true: object is not a subtype of every other type.That means when the type of a value is
object, a type checker will reject almost all operations on it, and assigning it to a variable (or using it as a return value) of a more specialized type is a type error.
and from the mypy documentation section Any vs. object:
The type
objectis another type that can have an instance of arbitrary type as a value. UnlikeAny,objectis an ordinary static type (it is similar toObjectin Java), and only operations valid for all types are accepted for object values.
object can be cast to a more specific type, while Any really means anything goes and a type checker disengages from any use of the object (even if you later assign such an object to a name that is typechecked).
You already painted your function into a an un-typed corner by accepting list, which comes down to being the same thing as List[Any]. The typechecker disengaged there and the return value no longer matters, but since your function accepts a list containing Any objects, the proper return value would be Any here.
To properly participate in type-checked code, you need to mark your input as List[T] (a genericly typed container) for a typechecker to then be able to care about the return value. Which in your case would be T since you are retrieving a value from the list. Create T from a TypeVar:
from typing import TypeVar, List
T = TypeVar('T')
def get_item(L: List[T], i: int) -> T:
return L[i]
or, using Python 3.12 or newer:
def get_itemT -> T:
return L[i]
Any and object are superficially similar, but in fact are entirely opposite in meaning.
object is the root of Python's metaclass hierarchy. Every single class inherits from object. That means that object is in a certain sense the most restrictive type you can give values. If you have a value of type object, the only methods you are permitted to call are ones that are a part of every single object. For example:
foo: object = 3
# Error, not all objects have a method 'hello'
bar = foo.hello()
# OK, all objects have a __str__ method
print(str(foo))
In contrast, Any is an escape hatch meant to allow you to mix together dynamic and statically typed code. Any is the least restrictive type -- any possible method or operation is permitted on a value of type Any. For example:
from typing import Any
foo: Any = 3
# OK, foo could be any type, and that type might have a 'hello' method
# Since we have no idea what hello() is, `bar` will also have a type of Any
bar = foo.hello()
# Ok, for similar reasons
print(str(foo))
You should generally try and use Any only for cases where...
- As a way of mixing together dynamic and statically typed code. For example, if you have many dynamic and complex functions, and don't have time to fully statically type all of them, you could settle for just giving them a return type of Any to nominally bring them into the typechecked work. (Or to put it another way, Any is a useful tool for helping migrate an untypechecked codebase to a typed codebase in stages).
- As a way of giving a type to an expression that is difficult to type. For example, Python's type annotations currently do not support recursive types, which makes typing things like arbitrary JSON dicts difficult. As a temporary measure, you might want to give your JSON dicts a type of
Dict[str, Any], which is a bit better then nothing.
In contrast, use object for cases where you want to indicate in a typesafe way that a value MUST literally work with any possible object in existence.
My recommendation is to avoid using Any except in cases where there is no alternative. Any is a concession -- a mechanism for allowing dynamism where we'd really rather live in a typesafe world.
For more information, see:
- https://docs.python.org/3/library/typing.html#the-any-type
- http://mypy.readthedocs.io/en/latest/kinds_of_types.html#the-any-type
For your particular example, I would use TypeVars, rather then either object or Any. What you want to do is to indicate that you want to return the type of whatever is contained within the list. If the list will always contain the same type (which is typically the case), you would want to do:
from typing import List, TypeVar
T = TypeVar('T')
def get_item(L: List[T], i: int) -> T:
return L[i]
This way, your get_item function will return the most precise type as possible.
Type hints are great! But I was playing Devil's advocate on a thread recently where I claimed actually type hinting can be legitimately annoying, especially to old school Python programmers.
But I think a lot of people were skeptical, so let's go through a made up scenario trying to type hint a simple Python package. Go to the end for a TL;DR.
The scenario
This is completely made up, all the events are fictitious unless explicitly stated otherwise (also editing this I realized attempts 4-6 have even more mistakes in them than intended but I'm not rewriting this again):
You maintain a popular third party library slowadd, your library has many supporting functions, decorators, classes, and metaclasses, but your main function is:
def slow_add(a, b):
time.sleep(0.1)
return a + bYou've always used traditional Python duck typing, if a and b don't add then the function throws an exception. But you just dropped support for Python 2 and your users are demanding type hinting, so it's your next major milestone.
First attempt at type hinting
You update your function:
def slow_add(a: int, b: int) -> int:
time.sleep(0.1)
return a + bAll your tests pass, mypy passes against your personal code base, so you ship with the release note "Type Hinting Support added!"
Second attempt at type hinting
Users immediately flood your GitHub issues with complaints! MyPy is now failing for them because they pass floats to slow_add, build processes are broken, they can't downgrade because of internal Enterprise policies of always having to increase type hint coverage, their weekend is ruined from this issue.
You do some investigating and find that MyPy supports Duck type compatibility for ints -> floats -> complex. That's cool! New release:
def slow_add(a: complex, b: complex) -> complex:
time.sleep(0.1)
return a + bFunny that this is a MyPy note and not a PEP standard...
Third attempt at type hinting
Your users thank you for your quick release, but a couple of days later one user asks why you no longer support Decimal. You replace complex with Decimal but now your other MyPy tests are failing.
You remember Python 3 added Numeric abstract base classes, what a perfect use case, just type hint everything as numbers.Number.
Hmmm, MyPy doesn't consider any of integers, or floats, or Decimals to be numbers :(.
After reading through typing you guess you'll just Union in the Decimals:
def slow_add(
a: Union[complex, Decimal], b: Union[complex, Decimal]
) -> Union[complex, Decimal]:
time.sleep(0.1)
return a + bOh no! MyPy is complaining that you can't add your other number types to Decimals, well that wasn't your intention anyway...
More reading later and you try overload:
@overload
def slow_add(a: Decimal, b: Decimal) -> Decimal:
...
@overload
def slow_add(a: complex, b: complex) -> complex:
...
def slow_add(a, b):
time.sleep(0.1)
return a + b
But MyPy on strict is complaining that slow_add is missing a type annotation, after reading this issue you realize that @overload is only useful for users of your function but the body of your function will not be tested using @overload. Fortunately in the discussion on that issue there is an alternative example of how to implement:
T = TypeVar("T", Decimal, complex)
def slow_add(a: T, b: T) -> T:
time.sleep(0.1)
return a + bFourth attempt at type hinting
You make a new release, and a few days later more users start complaining. A very passionate user explains the super critical use case of adding tuples, e.g. slow_add((1, ), (2, ))
You don't want to start adding each type one by one, there must be a better way! You learn about Protocols, and Type Variables, and positional only parameters, phew, this is a lot but this should be perfect now:
T = TypeVar("T")
class Addable(Protocol):
def __add__(self: T, other: T, /) -> T:
...
def slow_add(a: Addable, b: Addable) -> Addable:
time.sleep(0.1)
return a + bA mild diversion
You make a new release noting "now supports any addable type".
Immediately the tuple user complains again and says type hints don't work for longer Tuples: slow_add((1, 2), (3, 4)). That's weird because you tested multiple lengths of Tuples and MyPy was happy.
After debugging the users environment, via a series of "back and forth"s over GitHub issues, you discover that pyright is throwing this as an error but MyPy is not (even in strict mode). You assume MyPy is correct and move on in bliss ignoring there is actually a fundamental mistake in your approach so far.
(Author Side Note - It's not clear if MyPy is wrong but it defiantly makes sense for Pyright to throw an error here, I've filed issues against both projects and a pyright maintainer has explained the gory details if you're interested. Unfortunately this was not really addressed in this story until the "Seventh attempt")
Fifth attempt at type hinting
A week later a user files an issue, the most recent release said that "now supports any addable type" but they have a bunch of classes that can only be implemented using __radd__ and the new release throws typing errors.
You try a few approaches and find this seems to best solve it:
T = TypeVar("T")
class Addable(Protocol):
def __add__(self: T, other: T, /) -> T:
...
class RAddable(Protocol):
def __radd__(self: T, other: Any, /) -> T:
...
@overload
def slow_add(a: Addable, b: Addable) -> Addable:
...
@overload
def slow_add(a: Any, b: RAddable) -> RAddable:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bAnnoyingly there is now no consistent way for MyPy to do anything with the body of the function. Also you weren't able to fully express that when b is "RAddable" that "a" should not be the same type because Python type annotations don't yet support being able to exclude types.
Sixth attempt at type hinting
A couple of days later a new user complains they are getting type hint errors when trying to raise the output to a power, e.g. pow(slow_add(1, 1), slow_add(1, 1)). Actually this one isn't too bad, you quick realize the problem is your annotating Protocols, but really you need to be annotating Type Variables, easy fix:
T = TypeVar("T")
class Addable(Protocol):
def __add__(self: T, other: T, /) -> T:
...
A = TypeVar("A", bound=Addable)
class RAddable(Protocol):
def __radd__(self: T, other: Any, /) -> T:
...
R = TypeVar("R", bound=RAddable)
@overload
def slow_add(a: A, b: A) -> A:
...
@overload
def slow_add(a: Any, b: R) -> R:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bSeventh attempt at type hinting
Tuple user returns! He says MyPy in strict mode is now complaining with the expression slow_add((1,), (2,)) == (1, 2) giving the error:
Non-overlapping equality check (left operand type: "Tuple[int]", right operand type: "Tuple[int, int]")
You realize you can't actually guarantee anything about the return type from some arbitrary __add__ or __radd__, so you starting throwing Any Liberally around:
class Addable(Protocol):
def __add__(self: "Addable", other: Any, /) -> Any:
...
class RAddable(Protocol):
def __radd__(self: "RAddable", other: Any, /) -> Any:
...
@overload
def slow_add(a: Addable, b: Any) -> Any:
...
@overload
def slow_add(a: Any, b: RAddable) -> Any:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bEighth attempt at type hinting
Users go crazy! The nice autosuggestions their IDE provided them in the previous release have all gone! Well you can't type hint the world, but I guess you could include type hints for the built-in types and maybe some Standard Library types like Decimal:
You think you can rely on some of that MyPy duck typing but you test:
@overload
def slow_add(a: complex, b: complex) -> complex:
...
And realize that MyPy throws an error on something like slow_add(1, 1.0).as_integer_ratio(). So much for that nice duck typing article on MyPy you read earlier.
So you end up implementing:
class Addable(Protocol):
def __add__(self: "Addable", other: Any, /) -> Any:
...
class RAddable(Protocol):
def __radd__(self: "RAddable", other: Any, /) -> Any:
...
@overload
def slow_add(a: int, b: int) -> int:
...
@overload
def slow_add(a: float, b: float) -> float:
...
@overload
def slow_add(a: complex, b: complex) -> complex:
...
@overload
def slow_add(a: str, b: str) -> str:
...
@overload
def slow_add(a: tuple[Any, ...], b: tuple[Any, ...]) -> tuple[Any, ...]:
...
@overload
def slow_add(a: list[Any], b: list[Any]) -> list[Any]:
...
@overload
def slow_add(a: Decimal, b: Decimal) -> Decimal:
...
@overload
def slow_add(a: Fraction, b: Fraction) -> Fraction:
...
@overload
def slow_add(a: Addable, b: Any) -> Any:
...
@overload
def slow_add(a: Any, b: RAddable) -> Any:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bAs discussed earlier MyPy doesn't use the signature of any of the overloads and compares them to the body of the function, so all these type hints have to manually validated as accurate by you.
Ninth attempt at type hinting
A few months later a user says they are using an embedded version of Python and it hasn't implemented the Decimal module, they don't understand why your package is even importing it given it doesn't use it. So finally your code looks like:
from __future__ import annotations
import time
from typing import TYPE_CHECKING, Any, Protocol, TypeVar, overload
if TYPE_CHECKING:
from decimal import Decimal
from fractions import Fraction
class Addable(Protocol):
def __add__(self: "Addable", other: Any, /) -> Any:
...
class RAddable(Protocol):
def __radd__(self: "RAddable", other: Any, /) -> Any:
...
@overload
def slow_add(a: int, b: int) -> int:
...
@overload
def slow_add(a: float, b: float) -> float:
...
@overload
def slow_add(a: complex, b: complex) -> complex:
...
@overload
def slow_add(a: str, b: str) -> str:
...
@overload
def slow_add(a: tuple[Any, ...], b: tuple[Any, ...]) -> tuple[Any, ...]:
...
@overload
def slow_add(a: list[Any], b: list[Any]) -> list[Any]:
...
@overload
def slow_add(a: Decimal, b: Decimal) -> Decimal:
...
@overload
def slow_add(a: Fraction, b: Fraction) -> Fraction:
...
@overload
def slow_add(a: Addable, b: Any) -> Any:
...
@overload
def slow_add(a: Any, b: RAddable) -> Any:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bTL;DR
Turning even the simplest function that relied on Duck Typing into a Type Hinted function that is useful can be painfully difficult.
Please always put on your empathetic hat first when asking someone to update their code to how you think it should work.
In writing up this post I learnt a lot about type hinting, please try and find edge cases where my type hints are wrong or could be improved, it's a good exercise.
Edit: Had to fix a broken link.
Edit 2: It was late last night and I gave up on fixing everything, some smart people nicely spotted the errors!
I have a "tenth attempt" to address these error. But pyright complains about it because my overloads overlap, however I don't think there's a way to express what I want in Python annotations without overlap. Also Mypy complains about some of the user code I posted earlier giving the error comparison-overlap, interestingly though pyright seems to be able to detect here that the types don't overlap in the user code.
I'm going to file issues on pyright and mypy, but fundamentally they might be design choices rather than strictly bugs and therefore a limit on the current state of Python Type Hinting:
T = TypeVar("T")
class SameAddable(Protocol):
def __add__(self: T, other: T, /) -> T:
...
class Addable(Protocol):
def __add__(self: "Addable", other: Any, /) -> Any:
...
class SameRAddable(Protocol):
def __radd__(self: T, other: Any, /) -> T:
...
class RAddable(Protocol):
def __radd__(self: "RAddable", other: Any, /) -> Any:
...
SA = TypeVar("SA", bound=SameAddable)
RA = TypeVar("RA", bound=SameRAddable)
@overload
def slow_add(a: SA, b: SA) -> SA:
...
@overload
def slow_add(a: Addable, b: Any) -> Any:
...
@overload
def slow_add(a: Any, b: RA) -> RA:
...
@overload
def slow_add(a: Any, b: RAddable) -> Any:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bSo infuriating because I feel like my function is lying to me when it ends up letting other objects through the parameter.
Now every function I created needs an instance check which is unintuitive, verbose, and easily forgotten.
def wtf(string: str, integer:int): return 'TYPE HINT DOESNT DO ANYTHING' print( wtf( 123, 'wtf') ) >> 'TYPE HINT DOESNT DO ANYTHING'
EDIT:
Turns out it is a noob mistake. Type hint only functions as a signal for IDE, but developers get to do whatever they want to your parameter.
If you really must enforce it, you got to have the line
if not isinstance(string, str): raise TypeError()
Still, I don't like that this behavior isn't taught until it happens. So much time wasted on debugging production code when you take for granted your function is accepting only restricted data when it doesn't.