When using functions exported by cdll.LoadLibrary, you're releasing the Global Interpreter Lock (GIL) as you enter the method. If you want to call python code, you need to re-acquire the lock.

e.g.

void someFunctionWithPython()
{
    ...
    PyGILState_STATE state = PyGILState_Ensure();
    printf("importing numpy...\n");
    PyObject* numpy = PyImport_ImportModule("numpy");
    if (numpy == NULL)
    {
        printf("Warning: error during import:\n");
        PyErr_Print();
        Py_Finalize();
        PyGILState_Release(state);
        exit(1);
    }

    PyObject* repr = PyObject_Repr(numpy);
    PyObject* str = PyUnicode_AsEncodedString(repr, "utf-8", "~E~");
    const char *bytes = PyBytes_AS_STRING(str);

    printf("REPR: %s\n", bytes);

    Py_XDECREF(repr);
    Py_XDECREF(str);

    PyGILState_Release(state);
    return;
}
(python3.9-config --includes --ldflags --embed) -shared -o mylibwithpy.so mylibwithpy.c
$ LD_LIBRARY_PATH=. python driver.py
opening mylibwithpy.so...
.so object:  <CDLL 'mylibwithpy.so', handle 1749f50 at 0x7fb603702fa0>
.so object's 'someFunctionWithPython':  <_FuncPtr object at 0x7fb603679040>
calling someFunctionWithPython...
python alread initialized.
importing numpy...
REPR: <module 'numpy' from '/home/me/test/.venv/lib/python3.9/site-packages/numpy/__init__.py'>

Also if you look at PyDLL it says:

Instances of this class behave like CDLL instances, except that the Python GIL is not released during the function call, and after the function execution the Python error flag is checked. If the error flag is set, a Python exception is raised.

So if you use PyDLL for your driver then you wouldn't need to re-acquire the lock in the C code:

from ctypes import PyDLL

if __name__ == "__main__":
    print("opening mylibwithpy.so...");
    my_so = PyDLL("mylibwithpy.so")

    print(".so object: ", my_so)
    print(".so object's 'someFunctionWithPython': ", my_so.someFunctionWithPython)

    print("calling someFunctionWithPython...");
    my_so.someFunctionWithPython()

UPDATE

Why do numpy's internal shared objects files not link to libpython3.8.so?

I believe numpy is setup this way because it expects to be called by the python interpreter where libpython will already be loaded and have the symbols made available.

That said, we can make the python libraries available for when mylibwithpy calls the import of numpy by using RTLD_GLOBAL.

The symbols defined by this shared object will be made available for symbol resolution of subsequently loaded shared objects.

The update to your code is simple:

void* mylibwithpy_so = dlopen("mylibwithpy.so", RTLD_LAZY | RTLD_GLOBAL);

Now all of the python libraries will be included because they are a dependency of mylibwithpy, meaning they will be available by the time that numpy loads its own shared libraries.

Alternatively, you could choose to load just libpythonX.Y.so with RTLD_GLOBAL to prior to loading mylibwithpy.so to minimize the amount symbols made globally available.

printf("opening libpython3.9.so...\n");
void* libpython3_so = dlopen("libpython3.9.so", RTLD_LAZY | RTLD_GLOBAL);
if (libpython3_so == NULL){
    printf("an error occurred during loading libpython3.9.so: \n%s\n", dlerror());
    exit(1);
}

printf("opening mylibwithpy.so...\n");
void* mylibwithpy_so = dlopen("mylibwithpy.so", RTLD_LAZY);
if (mylibwithpy_so == NULL){
    printf("an error occurred during loading mylibwithpy.so: \n%s\n", dlerror());
    exit(1);
}

Docker setup I used to recreate and test:

FROM ubuntu:20.04

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y \
   build-essential \
   python3.9-dev \
   python3.9-venv 

RUN mkdir /workspace
WORKDIR /workspace

RUN python3.9 -m venv .venv
RUN .venv/bin/python -m pip install numpy

COPY . /workspace

RUN gcc -o mylibwithpy.so mylibwithpy.c -fPIC -shared \
    $(python3.9-config --includes --ldflags --embed --cflags) 

RUN gcc -o cdriver driver.c -L/usr/lib/x86_64-linux-gnu -Wall -ldl

ENV LD_LIBRARY_PATH=/workspace
# Then run: . .venv/bin/activate && ./cdriver
Answer from flakes on Stack Overflow
🌐
Python
docs.python.org › 3 › c-api › index.html
Python/C API reference manual — Python 3.14.4 documentation
This manual documents the API used by C and C++ programmers who want to write extension modules or embed Python. It is a companion to Extending and Embedding the Python Interpreter, which describes...
🌐
Medium
medium.com › @wansac › the-python-c-api-a-brief-introduction-7926ea0ef488
The Python/C API: A Brief Introduction | by Ashley Wans | Medium
September 14, 2020 - The Python/C API: A Brief Introduction The Python/C API brings together two of my two favorite things: Python and C. What is it? The Python/C API allows C programmers to embed Python directly into C
🌐
Python Developer's Guide
devguide.python.org › developer-workflow › c-api
Changing Python’s C API
The C API is divided into these tiers: The internal, private API, available with Py_BUILD_CORE defined. Ideally declared in Include/internal/. Any API named with a leading underscore is also consid...
Top answer
1 of 1
3

When using functions exported by cdll.LoadLibrary, you're releasing the Global Interpreter Lock (GIL) as you enter the method. If you want to call python code, you need to re-acquire the lock.

e.g.

void someFunctionWithPython()
{
    ...
    PyGILState_STATE state = PyGILState_Ensure();
    printf("importing numpy...\n");
    PyObject* numpy = PyImport_ImportModule("numpy");
    if (numpy == NULL)
    {
        printf("Warning: error during import:\n");
        PyErr_Print();
        Py_Finalize();
        PyGILState_Release(state);
        exit(1);
    }

    PyObject* repr = PyObject_Repr(numpy);
    PyObject* str = PyUnicode_AsEncodedString(repr, "utf-8", "~E~");
    const char *bytes = PyBytes_AS_STRING(str);

    printf("REPR: %s\n", bytes);

    Py_XDECREF(repr);
    Py_XDECREF(str);

    PyGILState_Release(state);
    return;
}
(python3.9-config --includes --ldflags --embed) -shared -o mylibwithpy.so mylibwithpy.c
$ LD_LIBRARY_PATH=. python driver.py
opening mylibwithpy.so...
.so object:  <CDLL 'mylibwithpy.so', handle 1749f50 at 0x7fb603702fa0>
.so object's 'someFunctionWithPython':  <_FuncPtr object at 0x7fb603679040>
calling someFunctionWithPython...
python alread initialized.
importing numpy...
REPR: <module 'numpy' from '/home/me/test/.venv/lib/python3.9/site-packages/numpy/__init__.py'>

Also if you look at PyDLL it says:

Instances of this class behave like CDLL instances, except that the Python GIL is not released during the function call, and after the function execution the Python error flag is checked. If the error flag is set, a Python exception is raised.

So if you use PyDLL for your driver then you wouldn't need to re-acquire the lock in the C code:

from ctypes import PyDLL

if __name__ == "__main__":
    print("opening mylibwithpy.so...");
    my_so = PyDLL("mylibwithpy.so")

    print(".so object: ", my_so)
    print(".so object's 'someFunctionWithPython': ", my_so.someFunctionWithPython)

    print("calling someFunctionWithPython...");
    my_so.someFunctionWithPython()

UPDATE

Why do numpy's internal shared objects files not link to libpython3.8.so?

I believe numpy is setup this way because it expects to be called by the python interpreter where libpython will already be loaded and have the symbols made available.

That said, we can make the python libraries available for when mylibwithpy calls the import of numpy by using RTLD_GLOBAL.

The symbols defined by this shared object will be made available for symbol resolution of subsequently loaded shared objects.

The update to your code is simple:

void* mylibwithpy_so = dlopen("mylibwithpy.so", RTLD_LAZY | RTLD_GLOBAL);

Now all of the python libraries will be included because they are a dependency of mylibwithpy, meaning they will be available by the time that numpy loads its own shared libraries.

Alternatively, you could choose to load just libpythonX.Y.so with RTLD_GLOBAL to prior to loading mylibwithpy.so to minimize the amount symbols made globally available.

printf("opening libpython3.9.so...\n");
void* libpython3_so = dlopen("libpython3.9.so", RTLD_LAZY | RTLD_GLOBAL);
if (libpython3_so == NULL){
    printf("an error occurred during loading libpython3.9.so: \n%s\n", dlerror());
    exit(1);
}

printf("opening mylibwithpy.so...\n");
void* mylibwithpy_so = dlopen("mylibwithpy.so", RTLD_LAZY);
if (mylibwithpy_so == NULL){
    printf("an error occurred during loading mylibwithpy.so: \n%s\n", dlerror());
    exit(1);
}

Docker setup I used to recreate and test:

FROM ubuntu:20.04

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y \
   build-essential \
   python3.9-dev \
   python3.9-venv 

RUN mkdir /workspace
WORKDIR /workspace

RUN python3.9 -m venv .venv
RUN .venv/bin/python -m pip install numpy

COPY . /workspace

RUN gcc -o mylibwithpy.so mylibwithpy.c -fPIC -shared \
    $(python3.9-config --includes --ldflags --embed --cflags) 

RUN gcc -o cdriver driver.c -L/usr/lib/x86_64-linux-gnu -Wall -ldl

ENV LD_LIBRARY_PATH=/workspace
# Then run: . .venv/bin/activate && ./cdriver
🌐
Cornell Virtual Workshop
cvw.cac.cornell.edu › python-performance › intro › cpython-api
Cornell Virtual Workshop > Python for High Performance > Overview > CPython and the Python/C API
The Python/C API allows for compiled pieces of code to be called from Python programs or executed within the CPython interpreter.
Find elsewhere
🌐
Python.org
discuss.python.org › c › core-dev › c-api › 30
C API - Discussions on Python.org
This sub-category is for discussions specifically about the Python C API. Covers both the development of as well as the development with (usage of) the C API.
🌐
GitHub
github.com › fpoli › python-c-api
GitHub - fpoli/python-c-api: Python/C API quick example · GitHub
A quick example of Python 3.10 modules implemented in C using the Python/C API.
Starred by 21 users
Forked by 12 users
Languages   C 74.7% | Makefile 17.4% | Python 7.9%
🌐
Python.org
discuss.python.org › core development › c api
C API Working Group and plan to get a Python C API compatible with alternative Python implementations? - C API - Discussions on Python.org
April 24, 2025 - The C API Working Group produced a PEP (PEP 733 – An Evaluation of Python’s Public C API | peps.python.org) entitled “An Evaluation of Python’s Public C API”. Part of this PEP is dedicated to discuss issues related to t…
🌐
Real Python
realpython.com › build-python-c-extension-module
Building a Python C Extension Module – Real Python
June 27, 2023 - In this tutorial, you'll learn how to write Python interfaces in C. Find out how to invoke C functions from within Python and build Python C extension modules. You'll learn how to parse arguments, return values, and raise custom exceptions using the Python API.
🌐
Mintlify
mintlify.com › python › cpython › c-api › intro
Introduction to the C API - CPython
Meet the next generation of documentation. AI-native, beautiful out-of-the-box, and built for developers.
🌐
Snarky
snarky.ca › try-to-not-use-the-c-api-directly
Try to not use the Python C API directly
August 20, 2018 - The key takeaway I hope people have from this is that you should use a library/tool to help you interface Python with C code, whether it's for performance or FFI needs. While the C API for CPython helped it gain its reputation as a great glue language, that doesn't mean you need to continue to work with it directly with so many great projects out there to help make your life easier.
🌐
Python
python.org › doc
Our Documentation | Python.org
Python/C API · Using Python · Python HOWTOs · Glossary · Search the online docs · Download Current Documentation (multiple formats are available, including typeset versions for printing.) FAQ: Sunsetting Python 2 · Final Python 2.7 Release Schedule · Python 3 Statement ·
🌐
GitHub
github.com › topics › python-c-api
python-c-api · GitHub Topics · GitHub
Create Python native extensions using Zig and Python C API!
🌐
Python Tips
book.pythontips.com › en › latest › python_c_extension.html
22. Python C extensions — Python Tips 0.1 documentation
There are three key methods developers use to call C functions from their python code - ctypes, SWIG and Python/C API.
🌐
GitHub
github.com › python › cpython
GitHub - python/cpython: The Python programming language · GitHub
3 weeks ago - For more complete instructions on contributing to CPython development, see the Developer Guide. Installable Python kits, and information about using Python, are available at python.org.
Starred by 72.5K users
Forked by 34.5K users
Languages   Python 61.9% | C 36.0% | C++ 0.7% | M4 0.4% | HTML 0.3% | JavaScript 0.2%
🌐
FastAPI
fastapi.tiangolo.com › tutorial › first-steps
First Steps - FastAPI
3 weeks ago - FastAPI is a Python class that provides all the functionality for your API.