I am not aware of any possible way to achieve this in Python as you cannot provide such constraints in type hints.
However, probably the Collection type might be useful in your context as a workaround:
class collections.abc.CollectionABC for sized iterable container classes.
This requires objects to have a __len__, which is a more strict requirement than being finite. For example, finite generators don't count as Collection.
How to Implement Positional Type Hinting for Iterables in Python?
Python type hint for Iterable[str] that isn't str - Stack Overflow
Type hinting a non-iterator (non-consumable) iterable
Type hints for Sized Iterable in Python - Stack Overflow
The code seems to work with both, but perhaps one of these is deprecated...?
def get_squares(upper_range: int) -> Iterable[int]:
return (x**2 for x in range(upper_range + 1))
for i in get_squares(10):
print(i)As of March 2022, the answer is no.
This issue has been discussed since at least July 2016. On a proposal to distinguish between str and Iterable[str], Guido van Rossum writes:
Since
stris a valid iterable ofstrthis is tricky. Various proposals have been made but they don't fit easily in the type system.
You'll need to list out all of the types that you want your functions to accept explicitly, using Union (pre-3.10) or | (3.10 and higher).
e.g. For pre-3.10, use:
from typing import Union
## Heading ##
def operate_on_files(file_paths: Union[TypeOneName, TypeTwoName, etc.]) -> None:
for path in file_paths:
...
For 3.10 and higher, use:
## Heading ##
def operate_on_files(file_paths: TypeOneName | TypeTwoName | etc.) -> None:
for path in file_paths:
...
If you happen to be using Pytype, it will not treat str as an Iterable[str] (as pointed out by Kelly Bundy). But, this behavior is typechecker-specific, and isn't widely supported in other typecheckers.
pytype (the type checker from Google) by default treat str as non-iterable and will raise error if you pass a str to a function expecting Iterable[str]. If you do need to handle a str as Iterable[str] you'll have to pass iter(s).
See https://google.github.io/pytype/faq.html#why-doesnt-str-match-against-string-iterables
If you decide to yield something instead of returning, what will the function return? It's obviously not a None, you get a generator object. How can I annotate that? (What is the proper type hinting for functions that yield)
The Sequence and Iterable abstract base classes (can also be used as type annotations) mostly* follow Python's definition of sequence and iterable. To be specific:
- Iterable is any object that defines
__iter__or__getitem__. - Sequence is any object that defines
__getitem__and__len__. By definition, any sequence is an iterable. TheSequenceclass also defines other methods such as__contains__,__reversed__that calls the two required methods.
Some examples:
list,tuple,strare the most common sequences.- Some built-in iterables are not sequences. For example,
reversedreturns areversedobject (orlist_reverseiteratorfor lists) that cannot be subscripted.
* Iterable does not exactly conform to Python's definition of iterables — it only checks if the object defines __iter__, and does not work for objects that's only iterable via __getitem__ (see this table for details). The gold standard of checking if an object is iterable is using the iter builtin.
When writing a function/method with an items argument, I often prefer Iterable to Sequence.
Hereafter is why and I hope it will help understanding the difference.
Say my_func_1 is:
from typing import Iterable
def my_func_1(items: Iterable[int]) -> None:
for item in items:
...
if condition:
break
return
Iterable offers the maximum possibilities to the caller. Correct calls include:
my_func_1((1, 2, 3)) # tuple is Sequence, Collection, Iterator
my_func_1([1, 2, 3]) # list is MutableSequence, Sequence, Collection, Iterator
my_func_1({1, 2, 3}) # set is Collection, Iterator
my_func_1(my_dict) # dict is Mapping, Collection, Iterator
my_func_1(my_dict.keys()) # dict.keys() is MappingKeys, Set, Collection, Iterator
my_func_1(range(10)) # range is Sequence, Collection, Iterator
my_func_1(x**2 for x in range(100)) # "strict' Iterator, i.e. neither a Collection nor a Sequence
...
... because all areIterable.
The implicit message to a function caller is: transfer data "as-is", just don't transform it.
In case the caller doesn't have data as a Sequence (e.g. tuple, list) or as a non-Sequence Collection (e.g. set), and because the iteration breaks before StopIteration, it is also more performing if he provides an 'strict' Iterator.
However if the function algorithm (say my_func_2) requires more than one iteration, then Iterable will fail if the caller provides a 'strict' Iterator because the first iteration exhausts it. Hence use a Collection:
from typing import Collection
def my_func_2(items: Collection[int]) -> None:
for item in items:
...
for item in items:
...
return
If the function algorithm (my_func_3) has to access by index to specific items, then both Iterable and Collection will fail if the caller provides a set, a Mapping or a 'strict' Iterator.
Hence use a Sequence:
from typing import Sequence
def my_func_3(items: Sequence[int]) -> None:
return items[5]
Conclusion: The strategy is: "use the most generic type that the function can handle". Don't forget that all this is only about typing, to help a static type checker to report incorrect calls (e.g. using a set when a Sequence is required). Then it's the caller responsibility to transform data when necessary, such as:
my_func_3(tuple(x**2 for x in range(100)))
Actually, all this is really about performance when scaling the length of items.
Always prefer Iterator when possible. Performance shall be handle as a daily task, not as a firemen task force.
In that direction, you will probably face the situation when a function only handles the empty use case and delegates the others, and you don't want to transform items into a Collection or a Sequence. Then do something like this:
from more_itertools import spy
def my_func_4(items: Iterable[int]) -> None:
(first, items) = spy(items)
if not first: # i.e. items is empty
...
else:
my_func_1(items) # Here 'items' is always a 'strict' Iterator
return