September 23, 2023 - This blog post will introduce you to the exciting new feature in Python 3.12 - Subinterpreters. We’ll start by explaining what subinterpreters are, their history, and how they compare to threads, processes, and greenlets. From there, we’ll delve into why subinterpreters were introduced, discussing the limitations of the Global Interpreter Lock (GIL) and how subinterpreters can improve performance and concurrency in Python.

Some simple and hopefully safe-ish bindings to Python 3.12's sub-interpreters. - ChillFish8/python12-subinterpreters

November 17, 2023 - Python 3.12 introduced a new API for “sub interpreters”, which are a different parallel execution model for Python that provide a nice compromise between the true parallelism of multiprocessing, but with a much faster startup time.

This was made available in 2023 as Python 3.12 came out to allow some experiments and sending objects back and forth subinterpreters.

May 3, 2024 - Free Bonus: Click here to download your sample code for a sneak peek at Python 3.12, coming in October 2023. Before you start thinking about subinterpreters, recall that an interpreter is a program that executes a script directly in a high-level language instead of translating it to machine code.

_xxsubinterpreters.run_string(): release the GIL: https://github.com/python/cpython/commit/fb2c7c4afbab0514352ab0246b0c0cc85d1bba53 · subprocess: close_fds=False, posix_spawn() is safe in subinterpreters

February 19, 2024 - For the time being, the "legacy version" of subinterpreters (i.e., using a global, shared GIL) should be used, since all Python extensions (including _tkinter and all extensions with single-phase initialization) should be supported. If I understand the docs correctly, using the legacy Py_NewInterpreter() method should preserve the existing behavior. Still, the following application crashes at shutdown: #include <Python.h> class PythonInterpreter { public: PythonInterpreter(PyThreadState* parent, int id) : mParent(parent) , mId(id) { PyEval_RestoreThread(mParent); mThread = Py_NewInterpreter();

February 29, 2024 - One of the most exciting features of Python 3.12 is the possibility of creating sub-interpreters programmatically, each with an independent per-interpreter lock. This feature relies on a new C API function Py_NewInterpreterFromConfig. Ne...

January 23, 2025 - I've removed any functions call even functions like sum(). It seems like SubInterpreter creations and even function starts are concurrent, but overall it's still synchronized on something (despite the fact that python 3.12+ should support per-interpreter GIL).

Enabling subinterpreters from Python code (PEP 554). ... See Tier 2 optimizer for an explanation of what the tier2 optimizer is. ... Our goal for 3.13 is to reduce the time spent in the interpreter by at least 50%. Issue. Execution engine. Detailed plan for copy-and-patch. Unlike the other tasks, which are mainly focused on single-threaded performance, this work builds on the per-interpreter GIL work that shipped in Python 3.12 to allow Python programmers to take advantage of better parallelism in subinterpreters from Python code (without the need to write a C extension).

March 3, 2023 - Additionally, PEP 554 will make it possible to create subinterpreters from Python (currently a C API-only feature), opening up true multi-threaded parallelism. We have done an analysis of which bytecodes would benefit the most from specialization ...

May 14, 2023 - We've found the sub-interpreters, but we will also need to borrow some helper functions from Python's test module that we will use to pass code to sub-interpreter: from textwrap import dedent import os # https://github.com/python/cpython/blob/ # 15665d896bae9c3d8b60bd7210ac1b7dc533b093/Lib/test/test__xxsubinterpreters.py#L75 def _captured_script(script): r, w = os.pipe() indented = script.replace('\n', '\n ') wrapped = dedent(f""" import contextlib with open({w}, 'w', encoding="utf-8") as spipe: with contextlib.redirect_stdout(spipe): {indented} """) return wrapped, open(r, encoding="utf-8") def _run_output(interp, request, channels=None): script, rpipe = _captured_script(request) with rpipe: interp.run(script, channels=channels) return rpipe.read()

August 19, 2024 - I started coding up a basic implementation to see if I could serve simple requests from subinterpreters. From the get go I found that requests were failing in Python version 3.12 but working for previous versions thanks to CI tests.

October 3, 2023 - I got shared this: GitHub - ChillFish8/python12-subinterpreters: Some simple and hopefully safe-ish bindings to Python 3.12's sub-interpreters. , which someone seems to have made to get access to subinterpreters from python before an official ...

CPython has supported multiple interpreters in the same process (AKA “subinterpreters”) since version 1.5 (1997). The feature has been available via the C-API. [c-api] Multiple interpreters operate in relative isolation from one another, ...

If you know the tp_new can be skipped safely, filter the deprecation warning out using :func:`warnings.catch_warnings` from Python. :c:var:`PyOS_InputHook` and :c:var:`PyOS_ReadlineFunctionPointer` are no longer called in :ref:`subinterpreters <sub-interpreter-support>`. This is because clients generally rely on process-wide global state (since these callbacks have no way of recovering extension module state).

December 27, 2020 - It should ease the development of isolated subinterpreters. It will be removed once subinterpreters will be fully isolated (once each interpreter will have its own GIL). Types declared in Python (class MyType: ...) are always "heap types": types dynamically allocated on the heap memory.

Python 3.12 exposes the subinterpreter functionality: after starting the Python main.exe you can spawn multiple subinterpreter threads. Questions: As the subinterpreter threads run in the same pro...

February 23, 2026 - syslog.openlog() and syslog.closelog() now fail if used in subinterpreters. syslog.syslog() may still be used in subinterpreters, but now only if syslog.openlog() has already been called in the main interpreter. These new restrictions do not apply to the main interpreter, so only a very small set of users might be affected.