As @jonrsharpe noted in a comment, this can be done with collections.abc.Callable:
from collections.abc import Callable
def my_function(func: Callable):
Note: Callable on its own is equivalent to Callable[..., Any].
Such a Callable takes any number and type of arguments (...) and returns a value of any type (Any). If this is too unconstrained, one may also specify the types of the input argument list and return type.
For example, given:
def sum(a: int, b: int) -> int: return a+b
The corresponding annotation is:
Callable[[int, int], int]
That is, the parameters are sub-scripted in the outer subscription with the return type as the second element in the outer subscription. In general:
Callable[[ParamType1, ParamType2, ..., ParamTypeN], ReturnType]
Answer from Dimitris Fasarakis Hilliard on Stack OverflowAs @jonrsharpe noted in a comment, this can be done with collections.abc.Callable:
from collections.abc import Callable
def my_function(func: Callable):
Note: Callable on its own is equivalent to Callable[..., Any].
Such a Callable takes any number and type of arguments (...) and returns a value of any type (Any). If this is too unconstrained, one may also specify the types of the input argument list and return type.
For example, given:
def sum(a: int, b: int) -> int: return a+b
The corresponding annotation is:
Callable[[int, int], int]
That is, the parameters are sub-scripted in the outer subscription with the return type as the second element in the outer subscription. In general:
Callable[[ParamType1, ParamType2, ..., ParamTypeN], ReturnType]
My specific use case for wanting this functionality was to enable rich code completion in PyCharm. Using Callable didn't cause PyCharm to suggest that the object had a .__code__ attribute, which is what I wanted, in this case.
I stumbled across the types module and...
from types import FunctionType
allowed me to annotate an object with FunctionType and, voilà, PyCharm now suggests my object has a .__code__ attribute.
The OP wasn't clear on why this type hint was useful to them. Callable certainly works for anything that implements .__call__() but for further interface clarification, I submit the types module.
Bummer that Python needed two very similar modules.
Videos
So infuriating because I feel like my function is lying to me when it ends up letting other objects through the parameter.
Now every function I created needs an instance check which is unintuitive, verbose, and easily forgotten.
def wtf(string: str, integer:int): return 'TYPE HINT DOESNT DO ANYTHING' print( wtf( 123, 'wtf') ) >> 'TYPE HINT DOESNT DO ANYTHING'
EDIT:
Turns out it is a noob mistake. Type hint only functions as a signal for IDE, but developers get to do whatever they want to your parameter.
If you really must enforce it, you got to have the line
if not isinstance(string, str): raise TypeError()
Still, I don't like that this behavior isn't taught until it happens. So much time wasted on debugging production code when you take for granted your function is accepting only restricted data when it doesn't.
Type hints are great! But I was playing Devil's advocate on a thread recently where I claimed actually type hinting can be legitimately annoying, especially to old school Python programmers.
But I think a lot of people were skeptical, so let's go through a made up scenario trying to type hint a simple Python package. Go to the end for a TL;DR.
The scenario
This is completely made up, all the events are fictitious unless explicitly stated otherwise (also editing this I realized attempts 4-6 have even more mistakes in them than intended but I'm not rewriting this again):
You maintain a popular third party library slowadd, your library has many supporting functions, decorators, classes, and metaclasses, but your main function is:
def slow_add(a, b):
time.sleep(0.1)
return a + bYou've always used traditional Python duck typing, if a and b don't add then the function throws an exception. But you just dropped support for Python 2 and your users are demanding type hinting, so it's your next major milestone.
First attempt at type hinting
You update your function:
def slow_add(a: int, b: int) -> int:
time.sleep(0.1)
return a + bAll your tests pass, mypy passes against your personal code base, so you ship with the release note "Type Hinting Support added!"
Second attempt at type hinting
Users immediately flood your GitHub issues with complaints! MyPy is now failing for them because they pass floats to slow_add, build processes are broken, they can't downgrade because of internal Enterprise policies of always having to increase type hint coverage, their weekend is ruined from this issue.
You do some investigating and find that MyPy supports Duck type compatibility for ints -> floats -> complex. That's cool! New release:
def slow_add(a: complex, b: complex) -> complex:
time.sleep(0.1)
return a + bFunny that this is a MyPy note and not a PEP standard...
Third attempt at type hinting
Your users thank you for your quick release, but a couple of days later one user asks why you no longer support Decimal. You replace complex with Decimal but now your other MyPy tests are failing.
You remember Python 3 added Numeric abstract base classes, what a perfect use case, just type hint everything as numbers.Number.
Hmmm, MyPy doesn't consider any of integers, or floats, or Decimals to be numbers :(.
After reading through typing you guess you'll just Union in the Decimals:
def slow_add(
a: Union[complex, Decimal], b: Union[complex, Decimal]
) -> Union[complex, Decimal]:
time.sleep(0.1)
return a + bOh no! MyPy is complaining that you can't add your other number types to Decimals, well that wasn't your intention anyway...
More reading later and you try overload:
@overload
def slow_add(a: Decimal, b: Decimal) -> Decimal:
...
@overload
def slow_add(a: complex, b: complex) -> complex:
...
def slow_add(a, b):
time.sleep(0.1)
return a + b
But MyPy on strict is complaining that slow_add is missing a type annotation, after reading this issue you realize that @overload is only useful for users of your function but the body of your function will not be tested using @overload. Fortunately in the discussion on that issue there is an alternative example of how to implement:
T = TypeVar("T", Decimal, complex)
def slow_add(a: T, b: T) -> T:
time.sleep(0.1)
return a + bFourth attempt at type hinting
You make a new release, and a few days later more users start complaining. A very passionate user explains the super critical use case of adding tuples, e.g. slow_add((1, ), (2, ))
You don't want to start adding each type one by one, there must be a better way! You learn about Protocols, and Type Variables, and positional only parameters, phew, this is a lot but this should be perfect now:
T = TypeVar("T")
class Addable(Protocol):
def __add__(self: T, other: T, /) -> T:
...
def slow_add(a: Addable, b: Addable) -> Addable:
time.sleep(0.1)
return a + bA mild diversion
You make a new release noting "now supports any addable type".
Immediately the tuple user complains again and says type hints don't work for longer Tuples: slow_add((1, 2), (3, 4)). That's weird because you tested multiple lengths of Tuples and MyPy was happy.
After debugging the users environment, via a series of "back and forth"s over GitHub issues, you discover that pyright is throwing this as an error but MyPy is not (even in strict mode). You assume MyPy is correct and move on in bliss ignoring there is actually a fundamental mistake in your approach so far.
(Author Side Note - It's not clear if MyPy is wrong but it defiantly makes sense for Pyright to throw an error here, I've filed issues against both projects and a pyright maintainer has explained the gory details if you're interested. Unfortunately this was not really addressed in this story until the "Seventh attempt")
Fifth attempt at type hinting
A week later a user files an issue, the most recent release said that "now supports any addable type" but they have a bunch of classes that can only be implemented using __radd__ and the new release throws typing errors.
You try a few approaches and find this seems to best solve it:
T = TypeVar("T")
class Addable(Protocol):
def __add__(self: T, other: T, /) -> T:
...
class RAddable(Protocol):
def __radd__(self: T, other: Any, /) -> T:
...
@overload
def slow_add(a: Addable, b: Addable) -> Addable:
...
@overload
def slow_add(a: Any, b: RAddable) -> RAddable:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bAnnoyingly there is now no consistent way for MyPy to do anything with the body of the function. Also you weren't able to fully express that when b is "RAddable" that "a" should not be the same type because Python type annotations don't yet support being able to exclude types.
Sixth attempt at type hinting
A couple of days later a new user complains they are getting type hint errors when trying to raise the output to a power, e.g. pow(slow_add(1, 1), slow_add(1, 1)). Actually this one isn't too bad, you quick realize the problem is your annotating Protocols, but really you need to be annotating Type Variables, easy fix:
T = TypeVar("T")
class Addable(Protocol):
def __add__(self: T, other: T, /) -> T:
...
A = TypeVar("A", bound=Addable)
class RAddable(Protocol):
def __radd__(self: T, other: Any, /) -> T:
...
R = TypeVar("R", bound=RAddable)
@overload
def slow_add(a: A, b: A) -> A:
...
@overload
def slow_add(a: Any, b: R) -> R:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bSeventh attempt at type hinting
Tuple user returns! He says MyPy in strict mode is now complaining with the expression slow_add((1,), (2,)) == (1, 2) giving the error:
Non-overlapping equality check (left operand type: "Tuple[int]", right operand type: "Tuple[int, int]")
You realize you can't actually guarantee anything about the return type from some arbitrary __add__ or __radd__, so you starting throwing Any Liberally around:
class Addable(Protocol):
def __add__(self: "Addable", other: Any, /) -> Any:
...
class RAddable(Protocol):
def __radd__(self: "RAddable", other: Any, /) -> Any:
...
@overload
def slow_add(a: Addable, b: Any) -> Any:
...
@overload
def slow_add(a: Any, b: RAddable) -> Any:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bEighth attempt at type hinting
Users go crazy! The nice autosuggestions their IDE provided them in the previous release have all gone! Well you can't type hint the world, but I guess you could include type hints for the built-in types and maybe some Standard Library types like Decimal:
You think you can rely on some of that MyPy duck typing but you test:
@overload
def slow_add(a: complex, b: complex) -> complex:
...
And realize that MyPy throws an error on something like slow_add(1, 1.0).as_integer_ratio(). So much for that nice duck typing article on MyPy you read earlier.
So you end up implementing:
class Addable(Protocol):
def __add__(self: "Addable", other: Any, /) -> Any:
...
class RAddable(Protocol):
def __radd__(self: "RAddable", other: Any, /) -> Any:
...
@overload
def slow_add(a: int, b: int) -> int:
...
@overload
def slow_add(a: float, b: float) -> float:
...
@overload
def slow_add(a: complex, b: complex) -> complex:
...
@overload
def slow_add(a: str, b: str) -> str:
...
@overload
def slow_add(a: tuple[Any, ...], b: tuple[Any, ...]) -> tuple[Any, ...]:
...
@overload
def slow_add(a: list[Any], b: list[Any]) -> list[Any]:
...
@overload
def slow_add(a: Decimal, b: Decimal) -> Decimal:
...
@overload
def slow_add(a: Fraction, b: Fraction) -> Fraction:
...
@overload
def slow_add(a: Addable, b: Any) -> Any:
...
@overload
def slow_add(a: Any, b: RAddable) -> Any:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bAs discussed earlier MyPy doesn't use the signature of any of the overloads and compares them to the body of the function, so all these type hints have to manually validated as accurate by you.
Ninth attempt at type hinting
A few months later a user says they are using an embedded version of Python and it hasn't implemented the Decimal module, they don't understand why your package is even importing it given it doesn't use it. So finally your code looks like:
from __future__ import annotations
import time
from typing import TYPE_CHECKING, Any, Protocol, TypeVar, overload
if TYPE_CHECKING:
from decimal import Decimal
from fractions import Fraction
class Addable(Protocol):
def __add__(self: "Addable", other: Any, /) -> Any:
...
class RAddable(Protocol):
def __radd__(self: "RAddable", other: Any, /) -> Any:
...
@overload
def slow_add(a: int, b: int) -> int:
...
@overload
def slow_add(a: float, b: float) -> float:
...
@overload
def slow_add(a: complex, b: complex) -> complex:
...
@overload
def slow_add(a: str, b: str) -> str:
...
@overload
def slow_add(a: tuple[Any, ...], b: tuple[Any, ...]) -> tuple[Any, ...]:
...
@overload
def slow_add(a: list[Any], b: list[Any]) -> list[Any]:
...
@overload
def slow_add(a: Decimal, b: Decimal) -> Decimal:
...
@overload
def slow_add(a: Fraction, b: Fraction) -> Fraction:
...
@overload
def slow_add(a: Addable, b: Any) -> Any:
...
@overload
def slow_add(a: Any, b: RAddable) -> Any:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bTL;DR
Turning even the simplest function that relied on Duck Typing into a Type Hinted function that is useful can be painfully difficult.
Please always put on your empathetic hat first when asking someone to update their code to how you think it should work.
In writing up this post I learnt a lot about type hinting, please try and find edge cases where my type hints are wrong or could be improved, it's a good exercise.
Edit: Had to fix a broken link.
Edit 2: It was late last night and I gave up on fixing everything, some smart people nicely spotted the errors!
I have a "tenth attempt" to address these error. But pyright complains about it because my overloads overlap, however I don't think there's a way to express what I want in Python annotations without overlap. Also Mypy complains about some of the user code I posted earlier giving the error comparison-overlap, interestingly though pyright seems to be able to detect here that the types don't overlap in the user code.
I'm going to file issues on pyright and mypy, but fundamentally they might be design choices rather than strictly bugs and therefore a limit on the current state of Python Type Hinting:
T = TypeVar("T")
class SameAddable(Protocol):
def __add__(self: T, other: T, /) -> T:
...
class Addable(Protocol):
def __add__(self: "Addable", other: Any, /) -> Any:
...
class SameRAddable(Protocol):
def __radd__(self: T, other: Any, /) -> T:
...
class RAddable(Protocol):
def __radd__(self: "RAddable", other: Any, /) -> Any:
...
SA = TypeVar("SA", bound=SameAddable)
RA = TypeVar("RA", bound=SameRAddable)
@overload
def slow_add(a: SA, b: SA) -> SA:
...
@overload
def slow_add(a: Addable, b: Any) -> Any:
...
@overload
def slow_add(a: Any, b: RA) -> RA:
...
@overload
def slow_add(a: Any, b: RAddable) -> Any:
...
def slow_add(a: Any, b: Any) -> Any:
time.sleep(0.1)
return a + bFor me it's :
from typing import Self
class Foo:
def __init__(self: Self) -> None:
...The second example is acceptable in my opinion, as the parameter are one type and the type hint for the actual attributes is for their entire lifetimes within the instance :
class Foo:
def __init__(self, par1: int, par2: tuple[float, float]):
self.par1: int = par1
self.par2: tuple[float, float] | None = par2Edit: changed the method in the first example from bar to __init__