For anyone who finds this, in Python 3 string and bytes are now different. ctypes needs a bytes instance in order to correctly pass the parameter as char*.

test_lib.get_val(b'Test1', byref(a)) will pass the full 'Test1' as a char* to the c function

test_lib.get_val('Test1', byref(a)) will not pass 'Test1' as a char*, from my little bit of testing it passes the first letter of the python str followed by a null charcter.

This has nothing to do with ctypes; your C++ code is invalid in itself. You can't define an extern "C" function that returns a string.

In a quick test with a C++ program that uses the same library, it also prints garbage.

I also wrote a C program that defines something called string with the same layout as std::string just so I could compile it and see what happens; it also prints garbage, and then it segfaults in ~string.

So, it's not surprising that the Python program also prints garbage.

With a minor change, everything works:

extern "C"
const char *return_string(char* name){
    cout<<strlen(name)<<endl;
    cout<<name<<endl;
    static string s = "hello ";
    s += name;
    return s.c_str();
}

I get this output:

3
Tom
hello Tom
<ctypes.c_char_Array_4 object at 0x11011c7a0>

(Or, from the C++ version, the same thing but with "Tom" in the last line.)

Of course for obvious reasons this isn't a very good solution, but it shows that returning string is the problem.

Both g++-4.5 and clang-apple-4.0 warned me about exactly this problem when I tried to compile your C++ code (although g++-apple-4.2 didn't, unless I added an extra -W flag). When the compiler gives you a warning, that's often the answer to "why does my code do the wrong thing even though it compiles".

A few other things wrong with your code:

• You don't make any use of anything from Python in your .cpp file. And in general, the whole point of using ctypes is so your C or C++ code doesn't have to know anything about Python; it's your Python code that knows about it. So, don't include or link.
• It's generally a bad idea to take a non-const char* if you aren't planning to modify it. My C++ driver had to call it with const_cast<char*>(name.c_str()) instead of just name.c_str(). Also, this can prevent the compiler from noticing other things that you're doing.

Here's the C++ driver I mentioned above:

#include <iostream>
#include <string>
using namespace std;

extern "C" string return_string(char* name);

int main(int argc, char *argv[]) {
  string name("Tom");
  string s(return_string(const_cast<char *>(name.c_str())));
  cout << s << "\n";
  cout << name << "\n";
  return 0;
}

Also, if I play around with different optimization settings or reorganize the code a bit, in my C++ driver, sometimes your code actually works, sometimes it prints garbage, and sometimes it segfaults. My first guess would be that it depends where the ~string call gets inlined—but really, the details don't matter; the code shouldn't work, and it doesn't, so who cares why?

If you read the parsing format string docs, it's pretty clear.

s is solely for getting a NUL terminated UTF-8 encoded C-style string from a str object (so it's appropriate for your first argument, but not your second).

y* is specifically called out in the docs with (emphasized in original text):

This is the recommended way to accept binary data.

y# would also work, at the expense of requiring the caller to provide immutable bytes-like objects, excluding stuff like bytearray and mmap.mmaps.

The solution depends on whether you need the data type to be mutable (from Python's perspective).

If don't need the data to be mutable, then don't bother with the buffer constructor. Just pass the bytes object direct to the TEST_STRUCT constructor. eg.

from ctypes import *

class TEST_STRUCT(Structure):
    _fields_ = [("a", c_int),
                ("b", c_char_p)]

hello_args = TEST_STRUCT(1, b"test")

If you need a mutable buffer then you need to specify your char* type slightly differently in the TEST_STRUCT class. eg.

from ctypes import *

class TEST_STRUCT(Structure):
    _fields_ = [("a", c_int),
                ("b", POINTER(c_char))] # rather than c_char_p

name = create_string_buffer(b"test")
hello_args = TEST_STRUCT(1, name)

c_char_p versus POINTER(c_char)

Ostensibly, whilst these two appear similar they are subtly different. POINTER(c_char) is for any buffer of char data. c_char_p is specifically for NULL terminated strings. As a side effect of this condition all c_char_p objects are immutable (on the python side) so as to make this guarantee easier to enforce. eg. {65, 66, 67, 0} is a string ("ABC"), whereas {1, 2, 3} would not be.

Table of Contants Introduction When working with C libraries in Python using the ctypes module, you may encounter functions that accept a char** argument. This typically represents a pointer to a pointer to a character, which can be used to pass strings or arrays of strings to C functions. This guide will show you how to pass a string from Python to such a function. Passing a String to a C Library Function Accepting char** 1. Import the ctypes Module First, make sure to import the ctypes module. import ctypes 2. Define the C Function For demonstration purposes, let’s assume you have a simple C function that takes a char** argument. Below is a simple example of such a function: // example.c #include void print_strings(char** strings, int count) { for (int i = 0; i < count; i++) { printf("%s\n", strings[i]); } } Compile this C code into a shared library (e.g., example.so on Linux or example.dll on Windows). 3. Load the Shared Library In your Python code, load the shared library using ctypes: # Load the shared library example = ctypes.CDLL('./example.so') # Adjust the path as necessary 4. Define the Argument Types Next, define the argument types for the C function. In this case, the function accepts a char** and an int: example.print_strings.argtypes = (ctypes.POINTER(ctypes.c_char_p), ctypes.c_int) 5. Prepare the String Data To pass a string (or an array of strings) to the C function, you need to create an array of c_char_p. Here's how you can do that: # Prepare a list of strings strings = [b"Hello", b"World", b"from ctypes"] # Create a ctypes array of c_char_p string_array_type = ctypes.c_char_p * len(strings) c_string_array = string_array_type(*strings) 6. Call the C Function Now you can call the C function, passing the array of strings and the count: # Call the C function example.print_strings(c_string_array, len(strings)) 7. Full Example Here’s the complete code to illustrate the entire process: import ctypes # Load the shared library example = ctypes.CDLL('./example.so') # Define the argument types for the C function example.print_strings.argtypes = (ctypes.POINTER(ctypes.c_char_p), ctypes.c_int) # Prepare a list of strings strings = [b"Hello", b"World", b"from ctypes"] # Create a ctypes array of c_char_p string_array_type = ctypes.c_char_p * len(strings) c_string_array = string_array_type(*strings) # Call the C function example.print_strings(c_string_array, len(strings)) Conclusion Passing a string (or an array of strings) to a C library function that accepts char** can be easily accomplished in Python using the ctypes module. By defining the function's argument types, preparing your string data, and calling the function with the correct arguments, you can seamlessly interface with C libraries from Python. This capability allows for powerful integrations and can help leverage existing C code within Python applications.

Importing printf from libc.stdio fixes the problem for me:

from libc.stdio cimport printf

def go():
    py_string = """
a complicated string
with a few
newlines.
"""

    cdef bytes py_bytes = py_string.encode()
    cdef char* c_string = py_bytes
    printf(c_string)

    print "we actually got this far! :)"

The error is in the declaration of printf. That should have been, as stdio.pxd lists,

cdef extern from *:
    ctypedef char const_char "const char"

int printf(const_char *, ...)

whereas your version is implicitly object printf(char *); the default return value type is Python object rather than int as in C. Getting the right declaration turns off Cython's attempt to Py_XDECREF the return value from printf.

(Btw, in your "vanilla" C problem, you shouldn't be casting the return value from printf to void *.)

Aside from mistakes in the original code, I managed to make it work with the following source.pyx (translation between bytes type in Python3 and char* in C++):

cdef extern from "source.cpp":
  cdef char* fun(char* param)

def pyfun(mystring):
  mystring_b = mystring.encode('utf-8')
  rvalue = fun(mystring_b).decode('utf-8')
  return rvalue

If the memory is allocated using malloc within fun it will also need to be freed, otherwise there will be a memory leak (when working with C pointers it's always worth considering who owns the memory). A modified version that does this is:

from libc.stdlib cimport free

# cdef extern as before

def pyfun(mystring):
    cdef char* value_from_fun
    mystring_b = mystring.encode('utf-8')
    value_from_fun = fun(mystring_b)
    try:    
        return value_from_fun.decode('utf-8')
    finally:
        free(value_from_fun)

The type conversions act in the same way as before.

---

Edit

As per hpaulj's comment in the original question, here is the version with libcpp.string which maps C++ std::string defined in <string>:

from libcpp.string cimport string

cdef extern from "source.cpp":
  cdef string fun(string param)

def pyfun(mystring):
  mystring_ = mystring.encode('utf-8')
  rvalue = fun(mystring_).decode('utf-8')
  return rvalue

First, you import the ctypes module. Then you need to load your c dll containing the said function. After that you need to set the argument types and the result type of that function. Finally you create your destination buffer and call your function.

Python:

import ctypes as ct

MAX_BUFSIZE = 100
  
mycdll = ct.CDLL("path_to_your_c_dll")  # for windows you use ct.WinDLL(path)

mycdll.function_name.argtypes = [ct.c_char_p, ct.c_int,
                                 ct.c_char_p, ct.c_char_p]

mycdll.function_name.restype = ct.c_int

mystrbuf = ct.create_string_buffer(MAX_BUFSIZE)
result = mycdll.function_name(mystrbuf, len(mystrbuf), 
                              b"my_input", b"my_second_input")

Working example using strncpy:

import ctypes as ct

MAX_BUFSIZE = 100

mycdll = ct.CDLL("libc.so.6")  # on windows you use cdll.msvcrt, instead

mycdll.strncpy.argtypes = [ct.c_char_p, ct.c_char_p, ct.c_size_t]

mycdll.strncpy.restype = ct.c_char_p

mystrbuf = ct.create_string_buffer(MAX_BUFSIZE)
dest = mycdll.strncpy(mystrbuf, b"my_input", len(mystrbuf))

print(mystrbuf.value)

Python 3 output:

user@Mint20:~/Dokumente/Programmieren/sites/Stackoverflow$ python3 --version
Python 3.8.5

user@Mint20:~/Dokumente/Programmieren/sites/Stackoverflow$ python3 python_ctypes.py 
b'my_input'

Python 2 output:

user@Mint20:~/Dokumente/Programmieren/sites/Stackoverflow$ python2 --version
Python 2.7.18

user@Mint20:~/Dokumente/Programmieren/sites/Stackoverflow$ python2 python_ctypes.py 
my_input

There are a number of problems:

1. Define .argtypes for your functions. It will catch errors passing incorrect parameters. Add the line below and note it is plural and is a tuple of argument types. The comma makes a 1-tuple:

python_p_printme.argtypes = c_char_p,

2. Once you make that change, you'll get an error because this code:

   c_sends = pointer(c_char_p(CHAR.encode('utf-8')

is actually sending a C char** (a pointer to a c_char_p). Once you've set the argtypes properly, you can just call the function with a byte string and it will work. Your function becomes:

def printmeP(CHAR):
    print("In Print function")
    print(CHAR)
    python_p_printme(CHAR.encode())
    print("DONE function - Python")

3. There is one more subtle problem. While the program may appear to work at this point Python strings are immutable so if the function being called requires a mutable string, you must allocate a mutable buffer using either create_unicode_buffer (for c_wchar_p) or create_string_buffer (for c_char_p); otherwise, the strcat in your C code is going to corrupt the Python string.

Here's a full example:

test.cpp

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

// For Windows compatibility
#ifdef _WIN32
#   define API __declspec(dllexport)
#else
#   define API
#endif

// For C++ compiler compatibility
#ifdef __cplusplus
extern "C" {
#endif

API void alter_me(char* param, size_t length) {
    // truncates if additional info exceeds length
    strncat_s(param, length, " adding some info", length - 1);
}

#ifdef __cplusplus
}
#endif

test.py

from ctypes import *

lib = CDLL('./test')
alter_me = lib.alter_me
alter_me.argtypes = c_char_p,c_size_t
alter_me.restype = None

data = create_string_buffer(b'test',size=10)
alter_me(data,sizeof(data))
print(data.value)

data = create_string_buffer(b'test',size=50)
alter_me(data,sizeof(data))
print(data.value)

Output:

b'test addi'
b'test adding some info'

Note you do not need to use create_string_buffer if the C function does not alter the buffer, such as if the C parameter is const char*. Then you could just call printme(b'test string').

In python3 (and you are definitely using python3 as on python2 your code would luckily work) strings are stored as wchar_t[] buffers, so when you pass "myfirstfile.txt" the C function sees its arg as "m\0y\0..." which is obviously a C string of lenght one. Here is the problem manifested:

In [19]: from ctypes import cdll, c_char_p

In [20]: libc = cdll.LoadLibrary("libc.so.6")

In [21]: puts = libc.puts

In [22]: puts('abc')
a

You should pass to the C function a bytes object

In [23]: puts(b'abc')
abc

You can convert str to bytes like this:

puts(my_var.encode())

To avoid further confusion you may specify the argument types of C function:

In [27]: puts.argtypes = [c_char_p]

Now the function accepts bytes (ctypes converts it to char*):

In [28]: puts(b'abc')
abc

but not str:

In [30]: puts('abc')
---------------------------------------------------------------------------
ArgumentError                             Traceback (most recent call last)
<ipython-input-26-aaa5b59630e2> in <module>()
----> 1 puts('abc')

ArgumentError: argument 1: <class 'TypeError'>: wrong type