Interfacing C code with Python

Sometimes, you may want more performance out of your Python projects, and the size of your project size has grown so much that it would be more difficult to port it over to a more performant language.

Sometimes you may want the ability to perform low-level operations in your Python project and don’t want to rewrite the entire project.

In these cases, the best option would be to write some of your logic in the more performant language and then call it from the Python code. But how do you do that? I’ll explain this article.

But to understand how it will work you’ll need to understand what a foreign function interface is.

So what is a foreign function interface?

A foreign function interface (FFI) is a mechanism that allows a program written in one language to access functions, methods, and services that are written and compiled by another.

The primary purpose of a foreign function interface is to match the semantics and calling conventions of one language to another.

In a Python-C code, where you’re calling a C function from a Python script, the C code presents functions through a foreign function interface (a shared library, or dynamic-link library (DLL) in the case of Windows OS). The Python script can then call those functions through its built-in ctypes module. Let’s look more into that

Calling a shared library or DLL function in Python

As I mentioned, these functions can be called with Python’s ctypes module. More specifically, the CDLL class from this module.

So in your Python script, you’ll first need to import the module:

from ctypes import CDLL

Then you’ll need to create an instance of the class with a path to the shared library or DLL file:

file = "./func.so"
func = CDLL(file)

On Windows, DLLs usually have a .dll extension. So you’ll need to use func.dll instead of func.so.

You can now call the functions from the shared library or DLL like it’s a method of the CDLL instance. For example:

func.sayHello()
result = func.add(5, 5)
print(result)

The CDLL object takes care of any type conversion that needs to happen, so you don’t need to worry too much about it. However, it only works very seamlessly for simple types. For more, complex data structures, it can be a little more complicated, and I’ll discuss more later in this article.

Creating shared libraries and DLLs in C

Creating shared libraries and DLLs in C is straightforward because:

• Shared libraries and DLLs are generated by the compiler.

• By default, all functions in C codes are public if they are ever compiled to a shared library or DLL.

This means that you don’t need to make any changes (for example explicitly making functions “public”).

So, to make things consistent, here’s the C code for the static library that was imported into the Python script earlier:

 #include <stdio.h>

void sayHello() {
    printf("Hello, World!\n");
}

int add(int a, int b) {
    return a + b;
}

To compile the C code into a static library with gcc run this command:

gcc -fpic -shared -o func.so func.c

Here’s a breakdown of the options:

• -fpic: tells the compiler to generate position-independent code. Useful for being able to call the functions in the library by name. If you get an error with the linker about this option, you may need to use the -fPIC option instead.

• -shared: tells the compiler to generate a shared object instead of an executable.

• -o func.so: saves the compiled code to func.so.

• func.c: the c code.

Troubleshooting on MacOS

This is the system I’m using to test these examples. And while testing I ran into an issue that went like this:

user@My-MacBook python-project % python3 main.py
Traceback (most recent call last):
  File "/Path/to/main.py", line 4, in <module>
    func = CDLL(so_file)
  File "/Path/to/python/installation/python3.13/ctypes/__init__.py", line 390, in __init__
    self._handle = _dlopen(self._name, mode)
                   ~~~~~~~^^^^^^^^^^^^^^^^^^
OSError: dlopen(./func.so, 0x0006): tried: './func.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')),
'/Maybe/need/to/be/redacted/OS./func.so' (no such file), '/more/redactions/./func.so' (no such file), '/Some/more/redactons/./func.so' (no such file),
'/Another/redaction/./func.so' (no such file), '/Theres/so/much/to/redact/./func.so' (no such file), '/redact/./func.so' (no such file, not in dyld cache),
'./func.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/Redacted/user/func.so' (mach-o file, but is an incompatible
architecture (have 'arm64', need 'x86_64')), '/dactRe/serU/func.so' (no such file), '/yppaH/gnitcader/func.so' (mach-o file, but is an incompatible architecture
(have 'arm64', need 'x86_64'))

To solve this issue, you have to compile the c code with the x86_64 version of GCC:

arch -x86_64 gcc -fpic -shared -o func.so func.c

And then you can run it normally:

python3 main.py

Why does this happen? I’m not sure. I just had the idea that it would work, and it did for me.

What more can you do with FFIs?

Beyond just sending simple data structures, Python’s ctypes module, allows sending more complex data structures. In this section, I’ll explain how to pass strings, arrays, and structs from Python to C.

Strings

In this part, I’ll show you how to call this greet function if it was in the shared object:

void greet(char *name) {
    printf("Hello, %s\n", name);
}

To call it from Python, you’ll need these lines

from ctypes import c_char_p

name = "Chigozie"
b_name = name.encode('utf-8')
func.greet.argtypes = [c_char_p]
func.greet(b_name)

How it works is that:

1. Encode the string into bytes.

2. Specify that greet's argument is a C char pointer.

3. Finally, pass the bytes as arguments to greet.

Structs

Here’s a code example to demonstrate how this can work:

• func.c

#include <stdio.h>

typedef struct Person {
    char *name;
    int age;
} Person;

void printDetails(Person person) {
    printf("Name: %s\nAge: %d\n", person.name, person.age);
}

• main.py

from ctypes import Structure, c_char_p, c_int

class Person(Structure):
    _fields_ = [("name", c_char_p), ("age", c_int)]

func.printDetails.argtypes = [Person]

# b_name from the strings example
person = Person( c_char_p(b_name), 32)

func.printDetails(person)

To be able to pass struct types between Python and C, you need to define the struct type in C and create a representation of the struct in Python.

If you want to return a struct, from the C function to Python set restype to be the struct representation. For example:

createPerson = func.createPerson

createPerson.argtypes = [c_char_p, c_int]
createPerson.restype = Person
person2 = createPerson(c_char_p(b_name), 32)

print(person2.name)

With this createPerson function:

Person createPerson(char* name, int age) {
    Person p;
    p.name = name;
    p.age = age;
    return p;
}

Arrays

To pass and receive arrays from C functions, there are certain things you need to do:

1. Have the function ready. For this example, I’ll be using the printList function below:

void printList(int* arrp, int length) {
    for (int i = 0; i < length; i++) {
        printf("%d\n", arrp[i]);
    }
}

2. Import it into Python and prepare for calling:

printList = func.printList
printList.argtypes = [ POINTER(c_int), c_int ]

To pass pointer types, use the ctypes.POINTER class

3. Prepare the array:

arrp = (c_int * 5)(2, 4, 6, 8, 10)

(c_int * 5) returns a class of type <class 'main.c_int_Array_5'>. That I used to initialize the array.

4. Pass arrp and the length of the array to printList:

printList(arrp, 5)