Good on you for using type annotations! As the documentations says, if you are on Python 3.9+, you should most likely never use typing.Sequence due to its deprecation. Since the introduction of generic alias types in 3.9 the collections.abc classes all support subscripting and should be recognized correctly by static type checkers of all flavors.
So the benefit of using collections.abc.T over typing.T is mainly that the latter is deprecated and should not be used.
As mentioned by jsbueno in his answer, annotations will have no runtime implications either way, unless of course they are explicitly picked up by a piece of code (see my other answer here). They are just an essential part of good coding style. But your function would still work, i.e. your script would execute without error, even if you annotated your function with something absurd like def average(sequence: 4%3): ....
Proper annotations are still extremely valuable. Thus, I would recommend you get used to some of the best practices as soon as possible. (A more-or-less strict static type checker like mypy is very helpful for that.) For one thing, when you are using generic types like Sequence, you should always provide the appropriate type arguments. Those may be type variables, if your function is also generic or they may be concrete types, but you should always include them.
In your case, assuming you expect the contents of your sequence to be something that can be added with the same type and divided by an integer, you might want to e.g. annotate it as Sequence[float]. (In the Python type system, float is considered a supertype of int, even though there is no nominal inheritance.)
Another recommendation is to try and be as broad as possible in the parameter types. (This echoes the Python paradigm of dynamic typing.) The idea is that you just specify that the object you expect must be able to "quack", but you don't say it must be a duck.
In your example, since you are reliant on the argument being compatible with sum as well as with len, you should consider what types those functions expect. The len function is simple, since it basically just calls the __len__ method of the object you pass to it. The sum function is more nuanced, but in your case the relevant part is that it expects an iterable of elements that can be added (e.g. float).
If you take a look at the collections ABCs, you'll notice that Sequence actually offers much more than you need, being that it is a reversible collection. A Collection is the broadest built-in type that fulfills your requirements because it has __iter__ (from Iterable) and __len__ (from Sized). So you could do this instead:
from collections.abc import Collection
def average(numbers: Collection[float]) -> float:
return sum(numbers) / len(numbers)
(By the way, the parameter name should not reflect its type.)
Lastly, if you wanted to go all out and be as broad as possible, you could define your own protocol that is even broader than Collection (by getting rid of the Container inheritance):
from collections.abc import Iterable, Sized
from typing import Protocol, TypeVar
T = TypeVar("T", covariant=True)
class SizedIterable(Sized, Iterable[T], Protocol[T]):
... # Literal ellipsis, not a placeholder
def average(numbers: SizedIterable[float]) -> float:
return sum(numbers) / len(numbers)
This has the advantage of supporting very broad structural subtyping, but is most likely overkill.
(For the basics of Python typing, PEP 483 and PEP 484 are a must-read.)
Answer from Daniel Fainberg on Stack OverflowGood on you for using type annotations! As the documentations says, if you are on Python 3.9+, you should most likely never use typing.Sequence due to its deprecation. Since the introduction of generic alias types in 3.9 the collections.abc classes all support subscripting and should be recognized correctly by static type checkers of all flavors.
So the benefit of using collections.abc.T over typing.T is mainly that the latter is deprecated and should not be used.
As mentioned by jsbueno in his answer, annotations will have no runtime implications either way, unless of course they are explicitly picked up by a piece of code (see my other answer here). They are just an essential part of good coding style. But your function would still work, i.e. your script would execute without error, even if you annotated your function with something absurd like def average(sequence: 4%3): ....
Proper annotations are still extremely valuable. Thus, I would recommend you get used to some of the best practices as soon as possible. (A more-or-less strict static type checker like mypy is very helpful for that.) For one thing, when you are using generic types like Sequence, you should always provide the appropriate type arguments. Those may be type variables, if your function is also generic or they may be concrete types, but you should always include them.
In your case, assuming you expect the contents of your sequence to be something that can be added with the same type and divided by an integer, you might want to e.g. annotate it as Sequence[float]. (In the Python type system, float is considered a supertype of int, even though there is no nominal inheritance.)
Another recommendation is to try and be as broad as possible in the parameter types. (This echoes the Python paradigm of dynamic typing.) The idea is that you just specify that the object you expect must be able to "quack", but you don't say it must be a duck.
In your example, since you are reliant on the argument being compatible with sum as well as with len, you should consider what types those functions expect. The len function is simple, since it basically just calls the __len__ method of the object you pass to it. The sum function is more nuanced, but in your case the relevant part is that it expects an iterable of elements that can be added (e.g. float).
If you take a look at the collections ABCs, you'll notice that Sequence actually offers much more than you need, being that it is a reversible collection. A Collection is the broadest built-in type that fulfills your requirements because it has __iter__ (from Iterable) and __len__ (from Sized). So you could do this instead:
from collections.abc import Collection
def average(numbers: Collection[float]) -> float:
return sum(numbers) / len(numbers)
(By the way, the parameter name should not reflect its type.)
Lastly, if you wanted to go all out and be as broad as possible, you could define your own protocol that is even broader than Collection (by getting rid of the Container inheritance):
from collections.abc import Iterable, Sized
from typing import Protocol, TypeVar
T = TypeVar("T", covariant=True)
class SizedIterable(Sized, Iterable[T], Protocol[T]):
... # Literal ellipsis, not a placeholder
def average(numbers: SizedIterable[float]) -> float:
return sum(numbers) / len(numbers)
This has the advantage of supporting very broad structural subtyping, but is most likely overkill.
(For the basics of Python typing, PEP 483 and PEP 484 are a must-read.)
Actually, in your code you need neither of those:
Typing with annotations, which is what you are doing with your imported Sequences class is an optional feature, meant for (1) quick documentation; (2) checking of the code before it is run by static code analysers such as Mypy.
The fact is that some IDEs use the result of static checking by default in their recomented configurations, and they can make it look like code without annotations is "faulty": it is not - this is an optional feature.
As long as the object you pass into your function respect some of the Sequence interface it will need, it will work (it needs __len__ and __getitem__ as is)
Just run your code without annotations and see it work:
def average(myvariable):
return sum(myvariable) / len(myvariable)
That said, here is what is happening: list is "the sequence" by excellence in Python, and implements everything a sequence needs.
typing.Sequence is just an indicator for the static-checker tools that the data marked with it should respect the Sequence protocol, and does nothing at run time. You can't instantiate it. You can inherit from it (probably) but just to specialize other markers for typing, not for anything that will have any effect during actual program execution.
On the other hand collections.abc.Sequence predates the optional typing recomendations in PEP 484: it works as both a "virtual super class" which can indicate everything that works as a sequence in runtime (through the use of isinstance) (*). AND it can be used as a solid base class to implement fully functional cusotm Sequence classes of your own: just inherit from collections.abc.Sequence and implement functional __getitem__ and __len__ methods as indicated in the docs here: https://docs.python.org/3/library/collections.abc.html (that is for read only sequences - for mutable sequences, check collections.abc.MutableSequence, of course).
(*) for your custom sequence implementation to be recognized as a Sequence proper it has to be "registered" in runtime with a call to collections.abc.Sequence.register. However, AFAIK, most tools for static type checking do not recognize this, and will error in their static analysis)
Question regarding type-hinting Collection types
Question about python types: What's the difference between a types.Sequence and types.Iterable?
Feature Request: Type hint for Fixed Length Homogeneous Sequences
python - What exactly is a Sequence? - Stack Overflow
I'm trying to make sense of the current type-hinting convention regarding Collection types. For simplicity's sake, I'll use list, tuple and dict, but that applies to any of their variant. Note that none of the following lines of code actually raise an error when run. Here's my question:
When someone writes:
var: str | int
What I read is obviously that 'var' is either a string OR an integer.
Again, when someone writes:
lst: list[str]
What I read is obviously that 'lst' is a list that contains ONLY strings.
So, logic says that here:
list[str | int]
It means that it's a list that contains ONLY strings OR ONLY integers. Right? And in this following case:
list[str, int]
it means that it's a list that contains strings AND/OR integers. Perhaps would it make more sense if it used & instead of , but whatever. Hopefully I'm making sense.
However, when checking the types using mypy, I find out that list[str, int] is not conventional, and that rather you should use list[str | int] for both cases. My code editor also says this, and yours probably does too.
Now take a similarly-structured type, tuple. By similarly-structured, I mean that you access and iterate through tuples the same way, in fact they are both Sequence types. The thing with tuple is that its args are index sensitive. I understand why this is the case and why it makes sense (because of immutability), but what doesn't make sense to me is why mypy is fine now with the comma notation with tuple but not list.
tpl: tuple[str, int] = ("", 0) # fine
tpl: tuple[str, int] = (0, "") # not fine
tpl: tuple[str | int] = (0, "") # not fine
tpl: tuple[str | int] = (0, ) # fine
tpl: tuple[str | int] = ("", ) # fine
Last but not least, dict. Everybody knows that Mapping types are different from Sequence types in the way you access, set or delete their items. So why is the typing convention this:
dict[str, int] # comma notation like tuple
and not:
dict[str: int] # Mapping specific notation
The latter makes much more sense and is more readable to me, and makes the distinction between Sequence types and Mapping types. Furthermore, the use of commas in Mapping types kinda looks like you can insert as many of them as you want, just like tuple?
tpl: tuple[str, int] = ("", 0) # this is fine
tpl: tuple[str, int, bool] = ("", 0, False) # this is fine
dct: dict[str, int, bool] = {"": 0: False} # but this is not fineI know that slices are not hashable for a reason, and that dict[str: int] notation kinda implies that they are, but again, the interpreter will raise no errors if you do this, much like if you write list[str, int], so I don't see the problem.
Is there an actual, good reason why this is the way it is?
Can't really find info looking online. The python docs doesn't describe any difference between them, semantically they sound identical but I'm not sure I'm missing something here.
Brief introduction to typing in Python
Skip ahead if you know what structural typing, nominal typing and duck typing are.
I think much of the confusion arises from the fact that typing was a provisional module between versions 3.5 and 3.6. And was still subject to change between versions 3.7 and 3.8. This means there has been a lot of flux in how Python has sought to deal with typing through type annotations.
It also doesn't help that python is both duck-typed and nominally typed. That is, when accessing an attribute of an object, Python is duck-typed. The object will only be checked to see if it has an attribute at runtime, and only when immediately requested. However, Python also has nominal typing features. Nominal typing is where one type is declared to be a subclass of another. This can be through inheritance, or with the register() method of ABCMeta.
typing originally introduced its types using the idea of nominal typing. As of 3.8 it is trying to allow for the more pythonic structural typing.
Structural typing is related to duck-typing, except that it is taken into consideration at "compile time" rather than runtime. For instance, when a linter is trying to detect possible type errors -- such as if you were to pass a dict to a function that only accepts sequences like tuples or list. With structural typing, a class B should be considered a subtype of A if it implements the all the methods of A, regardless of whether it has been declared to be a subtype of A (as in nominal typing).
Answer
sequences (little s) are a duck type. A sequence is any ordered collection of objects that provides random access to its members. Specifically, if it defines __len__ and __getitem__ and uses integer indices between 0 and n-1 then it is a sequence. A Sequence (big s) is a nominal type. That is, to be a Sequence, a class must be declared as such, either by inheriting from Sequence or being registered as a subclass.
A numpy array is a sequence, but it is not a Sequence as it is not registered as a subclass of Sequence. Nor should it be, as it does not implement the full interface promised by Sequence (things like count() and index() are missing).
It sounds like you want is a structured type for a sequence (small s). As of 3.8 this is possible by using protocols. Protocols define a set of methods which a class must implement to be considered a subclass of the protocol (a la structural typing).
from typing import Protocol
import numpy as np
class MySequence(Protocol):
def __getitem__(self, index):
raise NotImplementedError
def __len__(self):
raise NotImplementedError
def __contains__(self, item):
raise NotImplementedError
def __iter__(self):
raise NotImplementedError
def f(s: MySequence):
for i in range(len(s)):
print(s[i], end=' ')
print('end')
f([1, 2, 3, 4]) # should be fine
arr: np.ndarray = np.arange(5)
f(arr) # also fine
f({}) # might be considered fine! Depends on your type checker
Protocols are fairly new, so not all IDEs/type checkers might support them yet. The IDE I use, PyCharm, does. It doesn't like f({}), but it is happy to consider a numpy array a Sequence (big S) though (perhaps not ideal). You can enable runtime checking of protocols by using the runtime_checkable decorator of typing. Be warned, all this does is individually check that each of the Protocols methods can be found on the given object/class. As a result, it can become quite expensive if your protocol has a lot of methods.
I think the most practical way to define a sequence in Python is 'A container that supports indexing with integers'.
The Wikipedia definition also holds:
a sequence is an enumerated collection of objects in which repetitions are allowed and order does matter.
To validate if an object is a sequence, I would emulate the logic from the Sequence Protocol:
hasattr(test_obj, "__getitem__") and not isinstance(test_obj, collections.abc.Mapping)
Not supported by typeshed
Apparently, this is not possible with type hints. PEP 484 can not distinguish between Sequence[str], Iterable[str] and str according to Guido van Rossum.
Source: https://github.com/python/mypy/issues/1965 and https://github.com/python/typing/issues/256
So far, none of the proposals to fix this in typeshed have made it.
Support by individual type checkers
There have been some discussions if individual type checkers should support the distinction, but so far, this has not happened because most type checkers prefer to only follow the typing specification (and thus ensure that the behaviour is common to all Python type checkers).
Since 2021, pytype is the only type checker to treat str as a special case that does not match against Sequence[str] or Iterable[str]. Source: pytype FAQ on str.
A request to implement this in pyright has been rejected, with the possibility to reopen once enough upvotes are cast (a thumbs up to the first post in https://github.com/microsoft/pyright/issues/4886).
I couldn't find anything about the type exclusion or the type negation, seems like it's not supported in current version of Python 3. So the only distinctive feature of strings that crossed my mind is that strings are immutable. Maybe it'll help:
from typing import Union
from collections.abc import MutableSequence
MySequenceType = Union[MutableSequence, tuple, set]
def foo(a: MySequenceType):
pass
foo(["09485", "kfjg", "kfjg"]) # passed
foo(("09485", "kfjg", "kfjg")) # passed
foo({"09485", "kfjg", "kfjg"}) # passed
foo("qwerty") # not passed
Until Python 3.9 added support for type hinting using standard collections, you had to use typing.Tuple and typing.List if you wanted to document what type the contents of the containers needed to be:
def f(points: Tuple[float, float]):
return map(do_stuff, points)
Up until Python 3.8, tuple and list did not support being used as generic types. The above example documents that the function f requires the points argument to be a tuple with two float values.
typing.Tuple is special here in that it lets you specify a specific number of elements expected and the type of each position. Use ellipsis if the length is not set and the type should be repeated: Tuple[float, ...] describes a variable-length tuple with floats.
For typing.List and other sequence types you generally only specify the type for all elements; List[str] is a list of strings, of any size. Note that functions should preferentially take typing.Sequence as arguments and typing.List is typically only used for return types; generally speaking most functions would take any sequence and only iterate, but when you return a list, you really are returning a specific, mutable sequence type.
If you still need to support Python 3.8 or older code, you should always pick the typing generics even when you are not currently restricting the contents. It is easier to add that constraint later with a generic type as the resulting change will be smaller.
If you are implementing a custom container type and want that type to support generics, you can implement a __class_getitem__ hook or inherit from typing.Generic (which in turn implements __class_getitem__).
From Python 3.9 (PEP 585) onwards tuple, list and various other classes are now generic types. Using these rather than their typing counterpart is now preferred. From Python 3.9 you can now just do:
def f(points: tuple[float, float]):
return map(do_stuff, points)
If you don't need to evaluate your type hints then you can use this syntax in Python 3.7+ due to PEP 563.
from __future__ import annotations
def f(points: tuple[float, float]):
return map(do_stuff, points)
You should always pick then non-typing generic whenever possible as the old typing.Tuple, typing.List and other generics are deprecated and will be removed in a later version of Python.
Importing those from
typingis deprecated. Due to PEP 563 and the intention to minimize the runtime impact of typing, this deprecation will not generate DeprecationWarnings. Instead, type checkers may warn about such deprecated usage when the target version of the checked program is signalled to be Python 3.9 or newer. It's recommended to allow for those warnings to be silenced on a project-wide basis.The deprecated functionality will be removed from the typing module in the first Python version released 5 years after the release of Python 3.9.0.