A new take on dependency injection in Python
Benchmarked: 10 Python Dependency Injection libraries vs Manual Wiring (50 rounds x 100k requests)
python - What is a Pythonic way for Dependency Injection? - Stack Overflow
Python Dependency Injection Framework - Stack Overflow
In case anyone's interested, I've put together a DI framework "pylayer" in python that's fairly different from the alternatives I'm aware of (there aren't many). It includes a simple example at the bottom.
https://gist.github.com/johnhungerford/ccb398b666fd72e69f6798921383cb3f
What my project does
It allows you automatically construct dependencies based on their constructors.
The way it works is you define your dependencies as dataclasses inheriting from an Injectable class, where upstream dependencies are declared as dataclass attributes with type hints. Then you can just pass the classes to an Env object, which you can query for any provided type that you want to use. The Env object will construct a value of that type based on the Injectable classes you have provided. If any dependency needed to construct the queried type, it will generate an error message explaining what was missing and why it was needed.
Target audience
This is a POC that might be of interest to anyone who is uses or has wanted to use dependency injection in a Python project.
Comparison
https://python-dependency-injector.ets-labs.org/ is but complicated and unintuitive. pylayer is more automated and less verbose.
https://github.com/google/pinject is not maintained and seems similarly complicated.
https://itnext.io/dependency-injection-in-python-a1e56ab8bdd0 provides an approach similar to the first, but uses annotations to simplify some aspects of it. It's still more verbose and less intuitive, in my opinion, than pylayer.
Unlike all the above, pylayer has a relatively simple, functional mechanism for wiring dependencies. It is able to automate more by using the type introspection and the automated __init__ provided by dataclasses.
For anyone interested, my approach is based on Scala's ZIO library. Like ZIO's ZLayer type, pylayer takes a functional approach that uses memoization to prevent reconstruction of the same values. The main difference between pylayer and ZIO is that wiring and therefore validation is done at runtime. (Obviously compile-time validation isn't possible in Python...)
Hi r/python!
DI gets flak sometimes around here for being overengineered and adding overhead. I wanted to know how much it actually adds in a real stack, so I built a benchmark suite to find out. The fastest containers are within ~1% of manual wiring, while others drop between 20-70%
Full disclosure, I maintain Wireup, which is also in the race. The benchmark covers 10 libraries plus manual wiring via globals/creating objects yourself as an upper bound, so you can draw your own conclusions.
Testing is done within a FastAPI + Uvicorn environment to measure performance in a realistic web-based environment. Notably, this also allows for the inclusion of fastapi.Depends in the comparison, as it is the most popular choice by virtue of being the FastAPI default.
This tests the full integration stack using a dense graph of 7 dependencies, enough to show variance between the containers, but realistic enough to reflect a possible dependency graph in the real world. This way you test container resolution, scoping, lifecycle management, and framework wiring in real FastAPI + Uvicorn request/response cycles. Not a microbenchmark resolving the same dependency in a tight loop.
Table below shows Requests per second achieved as well as the secondary metrics:
-
RPS (Requests Per Second): The number of requests the server can handle in one second. Higher is better.
-
Latency (p50, p95, p99): The time it takes for a request to be completed, measured in milliseconds. Lower is better.
-
σ (Standard Deviation): Measures the stability of response times (Jitter). A lower number means more consistent performance with fewer outliers. Lower is better.
-
RSS Memory Peak (MB): The highest post-iteration RSS sample observed across runs. Lower is better. This includes the full server process footprint (Uvicorn + FastAPI app + framework runtime), not only service objects.
Per-request injection (new dependency graph built and torn down on every request):
| Project | RPS (Median Run) | P50 (ms) | P95 (ms) | P99 (ms) | σ (ms) | Mem Peak |
|---|---|---|---|---|---|---|
| Manual Wiring (No DI) | 11,044 (100.00%) | 4.20 | 4.50 | 4.70 | 0.70 | 52.93 MB |
| Wireup | 11,030 (99.87%) | 4.20 | 4.50 | 4.70 | 0.83 | 53.69 MB |
| Wireup Class-Based | 10,976 (99.38%) | 4.30 | 4.50 | 4.70 | 0.70 | 53.80 MB |
| Dishka | 8,538 (77.30%) | 5.30 | 6.30 | 9.40 | 1.30 | 103.23 MB |
| Svcs | 8,394 (76.00%) | 5.70 | 6.00 | 6.20 | 0.93 | 67.09 MB |
| Aioinject | 8,177 (74.04%) | 5.60 | 6.60 | 10.40 | 1.31 | 100.52 MB |
| diwire | 7,390 (66.91%) | 6.50 | 6.90 | 7.10 | 1.07 | 58.22 MB |
| That Depends | 4,892 (44.30%) | 9.80 | 10.40 | 10.60 | 0.59 | 53.82 MB |
| FastAPI Depends | 3,950 (35.76%) | 12.30 | 13.80 | 14.10 | 1.39 | 57.68 MB |
| Injector | 3,192 (28.90%) | 15.20 | 15.40 | 16.10 | 0.58 | 53.52 MB |
| Dependency Injector | 2,576 (23.33%) | 19.10 | 19.70 | 20.10 | 0.75 | 60.55 MB |
| Lagom | 898 (8.13%) | 55.30 | 57.20 | 58.30 | 1.63 | 1.32 GB |
Singleton injection (cached graph, testing container bookkeeping overhead):
-
Manual Wiring: 13,351 RPS
-
Wireup Class-Based: 13,342 RPS
-
Wireup: 13,214 RPS
-
Dependency Injector: 6,905 RPS
-
FastAPI Depends: 6,153 RPS
The full page goes much deeper: stability tables across all 50 runs, memory usage, methodology, feature completeness notes, and reproducibility: https://maldoinc.github.io/wireup/latest/benchmarks/
Reproduce it yourself: make bench iterations=50 requests=100000
Wireup getting this close to manual wiring comes down to how it works: instead of routing everything through a generic resolver, it compiles graph-specific resolution paths and custom injection functions per route at startup. By the time a request arrives there's nothing left to figure out.
If Wireup looks interesting: github.com/maldoinc/wireup, stars appreciated.
Happy to answer any questions on the benchmark, DI and Wireup specifically.
» pip install dependency-injector
See Raymond Hettinger - Super considered super! - PyCon 2015 for an argument about how to use super and multiple inheritance instead of DI. If you don't have time to watch the whole video, jump to minute 15 (but I'd recommend watching all of it).
Here is an example of how to apply what's described in this video to your example:
Framework Code:
class TokenInterface():
def getUserFromToken(self, token):
raise NotImplementedError
class FrameworkClass(TokenInterface):
def do_the_job(self, ...):
# some stuff
self.user = super().getUserFromToken(...)
Client Code:
class SQLUserFromToken(TokenInterface):
def getUserFromToken(self, token):
# load the user from the database
return user
class ClientFrameworkClass(FrameworkClass, SQLUserFromToken):
pass
framework_instance = ClientFrameworkClass()
framework_instance.do_the_job(...)
This will work because the Python MRO will guarantee that the getUserFromToken client method is called (if super() is used). The code will have to change if you're on Python 2.x.
One added benefit here is that this will raise an exception if the client does not provide a implementation.
Of course, this is not really dependency injection, it's multiple inheritance and mixins, but it is a Pythonic way to solve your problem.
The way we do dependency injection in our project is by using the inject lib. Check out the documentation. I highly recommend using it for DI. It kinda makes no sense with just one function but starts making lots of sense when you have to manage multiple data sources etc, etc.
Following your example it could be something similar to:
# framework.py
class FrameworkClass():
def __init__(self, func):
self.func = func
def do_the_job(self):
# some stuff
self.func()
Your custom function:
# my_stuff.py
def my_func():
print('aww yiss')
Somewhere in the application you want to create a bootstrap file that keeps track of all the defined dependencies:
# bootstrap.py
import inject
from .my_stuff import my_func
def configure_injection(binder):
binder.bind(FrameworkClass, FrameworkClass(my_func))
inject.configure(configure_injection)
And then you could consume the code this way:
# some_module.py (has to be loaded with bootstrap.py already loaded somewhere in your app)
import inject
from .framework import FrameworkClass
framework_instance = inject.instance(FrameworkClass)
framework_instance.do_the_job()
I'm afraid this is as pythonic as it can get (the module has some python sweetness like decorators to inject by parameter etc - check the docs), as python does not have fancy stuff like interfaces or type hinting.
So to answer your question directly would be very hard. I think the true question is: does python have some native support for DI? And the answer is, sadly: no.
Spring Python is an offshoot of the Java-based Spring Framework and Spring Security, targeted for Python. This project currently contains the following features:
- Inversion Of Control (dependency injection) - use either classic XML, or the python @Object decorator (similar to the Spring JavaConfig subproject) to wire things together. While the @Object format isn't identical to the Guice style (centralized wiring vs. wiring information in each class), it is a valuable way to wire your python app.
- Aspect-oriented Programming - apply interceptors in a horizontal programming paradigm (instead of vertical OOP inheritance) for things like transactions, security, and caching.
- DatabaseTemplate - Reading from the database requires a monotonous cycle of opening cursors, reading rows, and closing cursors, along with exception handlers. With this template class, all you need is the SQL query and row-handling function. Spring Python does the rest.
- Database Transactions - Wrapping multiple database calls with transactions can make your code hard to read. This module provides multiple ways to define transactions without making things complicated.
- Security - Plugin security interceptors to lock down access to your methods, utilizing both authentication and domain authorization.
- Remoting - It is easy to convert your local application into a distributed one. If you have already built your client and server pieces using the IoC container, then going from local to distributed is just a configuration change.
- Samples - to help demonstrate various features of Spring Python, some sample applications have been created:
- PetClinic - Spring Framework's sample web app has been rebuilt from the ground up using python web containers including: CherryPy. Go check it out for an example of how to use this framework. (NOTE: Other python web frameworks will be added to this list in the future).
- Spring Wiki - Wikis are powerful ways to store and manage content, so we created a simple one as a demo!
- Spring Bot - Use Spring Python to build a tiny bot to manage the IRC channel of your open source project.
I like this simple and neat framework.
http://pypi.python.org/pypi/injector/
Dependency injection as a formal pattern is less useful in Python than in other languages, primarily due to its support for keyword arguments, the ease with which objects can be mocked, and its dynamic nature.
That said, a framework for assisting in this process can remove a lot of boiler-plate from larger applications. That's where Injector can help. It automatically and transitively provides keyword arguments with their values. As an added benefit, Injector encourages nicely compartmentalized code through the use of Module s.
While being inspired by Guice, it does not slavishly replicate its API. Providing a Pythonic API trumps faithfulness.