If your code is designed to work with Python 3.10 or newer, you want to use the PEP 604 syntax, using ... | None union syntax, and not use typing.Optional:
def test(a: dict[Any, Any] | None = None) -> None:
#print(a) ==> {'a': 1234}
#or
#print(a) ==> None
def test(a: list[Any] | None = None) -> None:
#print(a) ==> [1, 2, 3, 4, 'a', 'b']
#or
#print(a) ==> None
Code that still supports older Python versions can still stick to using Optional. Optional[...] is a shorthand notation for Union[..., None], telling the type checker that either an object of the specific type is required, or None is required. ... stands for any valid type hint, including complex compound types or a Union[] of more types. Whenever you have a keyword argument with default value None, you should use Optional.
So for your two examples, you have dict and list container types, but the default value for the a keyword argument shows that None is permitted too so use Optional[...]:
from typing import Optional
def test(a: Optional[dict] = None) -> None:
#print(a) ==> {'a': 1234}
#or
#print(a) ==> None
def test(a: Optional[list] = None) -> None:
#print(a) ==> [1, 2, 3, 4, 'a', 'b']
#or
#print(a) ==> None
There is technically no difference between using Optional[] on a Union[], or just adding None to the Union[]. So Optional[Union[str, int]] and Union[str, int, None] are exactly the same thing.
Personally, I'd stick with always using Optional[] when setting the type for a keyword argument that uses = None to set a default value, this documents the reason why None is allowed better. Moreover, it makes it easier to move the Union[...] part into a separate type alias, or to later remove the Optional[...] part if an argument becomes mandatory.
For example, say you have
from typing import Optional, Union
def api_function(optional_argument: Optional[Union[str, int]] = None) -> None:
"""Frob the fooznar.
If optional_argument is given, it must be an id of the fooznar subwidget
to filter on. The id should be a string, or for backwards compatibility,
an integer is also accepted.
"""
then documentation is improved by pulling out the Union[str, int] into a type alias:
from typing import Optional, Union
# subwidget ids used to be integers, now they are strings. Support both.
SubWidgetId = Union[str, int]
def api_function(optional_argument: Optional[SubWidgetId] = None) -> None:
"""Frob the fooznar.
If optional_argument is given, it must be an id of the fooznar subwidget
to filter on. The id should be a string, or for backwards compatibility,
an integer is also accepted.
"""
The refactor to move the Union[] into an alias was made all the much easier because Optional[...] was used instead of Union[str, int, None]. The None value is not a 'subwidget id' after all, it's not part of the value, None is meant to flag the absence of a value.
Side note: Unless your code only has to support Python 3.9 or newer, you want to avoid using the standard library container types in type hinting, as you can't say anything about what types they must contain. So instead of dict and list, use typing.Dict and typing.List, respectively. And when only reading from a container type, you may just as well accept any immutable abstract container type; lists and tuples are Sequence objects, while dict is a Mapping type:
from typing import Mapping, Optional, Sequence, Union
def test(a: Optional[Mapping[str, int]] = None) -> None:
"""accepts an optional map with string keys and integer values"""
# print(a) ==> {'a': 1234}
# or
# print(a) ==> None
def test(a: Optional[Sequence[Union[int, str]]] = None) -> None:
"""accepts an optional sequence of integers and strings
# print(a) ==> [1, 2, 3, 4, 'a', 'b']
# or
# print(a) ==> None
In Python 3.9 and up, the standard container types have all been updated to support using them in type hints, see PEP 585. But, while you now can use dict[str, int] or list[Union[int, str]], you still may want to use the more expressive Mapping and Sequence annotations to indicate that a function won't be mutating the contents (they are treated as 'read only'), and that the functions would work with any object that works as a mapping or sequence, respectively.
Python 3.10 introduces the | union operator into type hinting, see PEP 604. Instead of Union[str, int] you can write str | int. In line with other type-hinted languages, the preferred (and more concise) way to denote an optional argument in Python 3.10 and up, is now Type | None, e.g. str | None or list | None.
If your code is designed to work with Python 3.10 or newer, you want to use the PEP 604 syntax, using ... | None union syntax, and not use typing.Optional:
def test(a: dict[Any, Any] | None = None) -> None:
#print(a) ==> {'a': 1234}
#or
#print(a) ==> None
def test(a: list[Any] | None = None) -> None:
#print(a) ==> [1, 2, 3, 4, 'a', 'b']
#or
#print(a) ==> None
Code that still supports older Python versions can still stick to using Optional. Optional[...] is a shorthand notation for Union[..., None], telling the type checker that either an object of the specific type is required, or None is required. ... stands for any valid type hint, including complex compound types or a Union[] of more types. Whenever you have a keyword argument with default value None, you should use Optional.
So for your two examples, you have dict and list container types, but the default value for the a keyword argument shows that None is permitted too so use Optional[...]:
from typing import Optional
def test(a: Optional[dict] = None) -> None:
#print(a) ==> {'a': 1234}
#or
#print(a) ==> None
def test(a: Optional[list] = None) -> None:
#print(a) ==> [1, 2, 3, 4, 'a', 'b']
#or
#print(a) ==> None
There is technically no difference between using Optional[] on a Union[], or just adding None to the Union[]. So Optional[Union[str, int]] and Union[str, int, None] are exactly the same thing.
Personally, I'd stick with always using Optional[] when setting the type for a keyword argument that uses = None to set a default value, this documents the reason why None is allowed better. Moreover, it makes it easier to move the Union[...] part into a separate type alias, or to later remove the Optional[...] part if an argument becomes mandatory.
For example, say you have
from typing import Optional, Union
def api_function(optional_argument: Optional[Union[str, int]] = None) -> None:
"""Frob the fooznar.
If optional_argument is given, it must be an id of the fooznar subwidget
to filter on. The id should be a string, or for backwards compatibility,
an integer is also accepted.
"""
then documentation is improved by pulling out the Union[str, int] into a type alias:
from typing import Optional, Union
# subwidget ids used to be integers, now they are strings. Support both.
SubWidgetId = Union[str, int]
def api_function(optional_argument: Optional[SubWidgetId] = None) -> None:
"""Frob the fooznar.
If optional_argument is given, it must be an id of the fooznar subwidget
to filter on. The id should be a string, or for backwards compatibility,
an integer is also accepted.
"""
The refactor to move the Union[] into an alias was made all the much easier because Optional[...] was used instead of Union[str, int, None]. The None value is not a 'subwidget id' after all, it's not part of the value, None is meant to flag the absence of a value.
Side note: Unless your code only has to support Python 3.9 or newer, you want to avoid using the standard library container types in type hinting, as you can't say anything about what types they must contain. So instead of dict and list, use typing.Dict and typing.List, respectively. And when only reading from a container type, you may just as well accept any immutable abstract container type; lists and tuples are Sequence objects, while dict is a Mapping type:
from typing import Mapping, Optional, Sequence, Union
def test(a: Optional[Mapping[str, int]] = None) -> None:
"""accepts an optional map with string keys and integer values"""
# print(a) ==> {'a': 1234}
# or
# print(a) ==> None
def test(a: Optional[Sequence[Union[int, str]]] = None) -> None:
"""accepts an optional sequence of integers and strings
# print(a) ==> [1, 2, 3, 4, 'a', 'b']
# or
# print(a) ==> None
In Python 3.9 and up, the standard container types have all been updated to support using them in type hints, see PEP 585. But, while you now can use dict[str, int] or list[Union[int, str]], you still may want to use the more expressive Mapping and Sequence annotations to indicate that a function won't be mutating the contents (they are treated as 'read only'), and that the functions would work with any object that works as a mapping or sequence, respectively.
Python 3.10 introduces the | union operator into type hinting, see PEP 604. Instead of Union[str, int] you can write str | int. In line with other type-hinted languages, the preferred (and more concise) way to denote an optional argument in Python 3.10 and up, is now Type | None, e.g. str | None or list | None.
Directly from mypy typing module docs.
Optional[str]is just a shorthand or alias forUnion[str, None]. It exists mostly as a convenience to help function signatures look a little cleaner.
Update for Python 3.10+
you can now use the pipe operator as well.
# Python < 3.10
def get_cars(size: Optional[str]=None):
pass
# Python 3.10+
def get_cars(size: str|None=None):
pass
Videos
Recently, one of my coworkers and I had a long debate on using Union[str, None] vs Optional[str]. I wrote a function that was supposed to return either a string or None. Based on the returned value my program was supposed to trigger some action. In my opinion, since None was a deciding factor it would be a better choice to use Union[str, None] for the sake of clarity. However, according to his opinion, Optional is a better choice in terms of simplicity/cleaner look. We both understand clearly that Optional is nothing but a shorthand of Union[..., None]. After a few searches, we figured people use both and both teams have their logic. In fact, the creator of Fast API, u/tiangolo also supports using Union (According to one of his recent tweets). My question is, what do you use and why?
The "modern" way to write this is str | None. Traditionally, I used Optional[str], mostly because Optional existed specifically for this purpose.
Assuming it's between these two, using the Union basic says Optional exists but should never be used.
I'm on team Optional here, that's exactly the use case for it so if we don't use it in that case there's no point for it having been accepted.
Sure, use the new syntax if you have it but if you don't then prefer Optional over Union with None.
By the way, None isn't a type - Union with NoneType also makes sense and this is a quibble that doesn't come up with Optional. We all know what Optional means.
I know that python essentially is a duck typed language and, in a way, asking for type checking is a no-brainer considering there is no plan to make cpython statically typed.
The point I am not understanding is, being optional and helpful in finding bug and better documentation, why is type hinting (PEP-484 and followups) not popular among tutorials and the community in general?
I am not talking about tutorials aimed at complete beginners. Consider function docstrings. There comes a state when we start documenting argument types for a function (e.g. see the numpy's guide on parameters), but most of us don't use type hints which can help both in documentation + type checking (using mypy, say) and instead use plain docstrings. Is it because of the idea being slightly contrary to python's dynamic typing rise to fame? Or is it simply not known by many?
EDIT: I mean type hinting (which translates to comment based hinting for Python 2) and not just type annotations for Python 3.
All I read is speculations.
I'd rather give you a feedback from practice (I'm a professional trainer and dev, I've been expose to a lot of code and coders, including using type hints).
-
type hints are not well know. A lot of people still don't know about it.
-
people that know about it didn't take time to learn about it. Typing is not that popular amongs Python dev. They are popular amongs dev in big teams or comming from stricter languages. The feature is here to accomodate them.
-
a lot of Python dev are not... dev. I work with teachers, geogramaticians, mathematicians, etc. They could not care less about code, it's just a mean to an end, and typing just get in their way.
-
a lot of dev are not... good dev. I works with a lot of people making loads of money 20k/month) with badly coded websites. They don't know a lot of about programming, just enough to run their business. They don't want to invest into something new, like type hints, that make things cleaners. Clean is not their problem.
Now there is a small minority of people, like me, who are interested in that feature. They used it. Here is there result:
-
the documentation sucks. The tutorials suck. You basically spend hours just to express anything more complex than the basic types.
-
it's not ready : mypy and pycharm support is partial, and sometimes buggy. I had sometime to use some very stupid hacks: https://github.com/Tygs/ww/blob/master/src/ww/types.py
-
setuping mypy is much more work that it should be.
-
the spec is incomplete: dunder methods are not detected yet, which means duck typing cannot be easily defined in your annotation.
-
it's takes a lot of time to figure out how to do anything.
-
even if you do everything right, the tooling doesn't give you that much safety net yet.
All in all, I'm still considering type hints a good things. It needs to happen. But in it's current state, it's a pain.
It's a fairly new feature that's only natively supported in recent versions of Python, while many production environments are stuck on 2.7, etc.
I am new to Pydantic. In one of the fastapi tutorials for validating the request body a pydantic model was declared. In one of the models for optional parameters typing.Optional was used.
# Optional approach id: Optional[int] = None # default value approach id: int = None
What is the difference between the above two approaches ?
I don't know whether TypeScript enforces type constraints at compile or run time. But Python does support type hints.
Python 3.0 introduced function annotations as defined in PEP 3107. They work like this:
def log_in(login: 'Login', password: 'Password') -> 'Return value':
# ...
log_in.func_annotation['login'] # => 'Login'
log_in.func_annotation['password'] # => 'Password'
log_in.func_annotation['return'] # => 'Return value'
You can annotate the parameters and the return value with any object.
This can be used to implement type hints. Python 3.5 introduces the typing module. You can implement a decorator that'd perform run-time type checking.
I'm sure the type specifications could be used to greatly speed up the runtime [โฆ]
No, they can't. The definition of an optional type system is that it does not impact the runtime behavior in any way. That's what makes it "optional" in the first place. If adding or removing types made the program change its behavior, the types wouldn't be optional.
Note also that you don't need to change Python in any way to typecheck it. You could just write your type annotations in comments, after all. What you do need is of course a type system with typing rules. Then, you need to prove this type system correct and sound (if you are interested in soundness, which, let's face it, you are, otherwise you wouldn't be interested in a type system to begin with). Then you need to write a typechecker for this typesystem. And once you have done that, and proven its worth, then you can talk about adding your type system to the language itself, and also adding type annotation syntax.
[Note that, at least for function parameters and return values, there already does exist support for type annotation syntax.]
This is already very hard. But it's not even the interesting part: you have now created a new, statically-typed language. But, you cannot expect everyone to statically type their Python code all at once. (Remember how long the transition to Python3 took โฆ ehm โฆ is still taking?) What you really want is a way to gradually migrate to static typing, or have parts of your code statically typed and parts dynamically.
One way to solve this problem, would be to find a type inference scheme which can automatically infer the correct static types from the program, so that you don't have to rely on legacy Python code to be annotated. Now, typechecking Python without violating its idioms is already going to be hard enough with the help of static type annotations, just imagine how hard it is going to be without!
So, the other way is what is called "gradual typing", a way to mix statically type-safe and dynamically typed code in such a way that
- they can interoperate seamlessly,
- the dynamically typed code cannot violate the type-safety of the statically type-safe code, and
- dynamically typed code can be gradually migrated to statically type-safe code through gradually adding annotations.
There is a lot of research into gradual typing, and in fact, a lot of the original research has been (and is being done) in and on Python, and a research implementation of gradual typing for typing in Jython has existed for quite a while.
You should give it some time. There is still debate over whether gradual typing or optional typing (or the route the Racket community took with their alternative take on gradual typing called soft typing), a combination of the two (and there are again different ideas on what such a combination would look like), or something entirely different is the way to go. Gradual typing is still an actively evolving research subject.
Besides Jeremy Siek's gradual typing in Jython there was also a research project for a statically typed and type-inferred Ruby (Diamondback Ruby), and there was ECMAScript4. There is Typed Racket (and clojure.typed, wich is based on the former). We had the optional Strongtalk typesystem for Smalltalk. But we still haven't figured it all out.
tl;dr:
Why python doesn't provide optional types?
Because nobody has done the work yet.