Chapter 4: Handling package dependencies, entry points, and extensions

Creating a C extension for Python

• Cython is a superset of the Python language. Therefore, a python code is a valid Cython code.
• We use Cython to convert Cython code into optimized C code, which eventually will be compiled during the build process.
• Cython source files have the extension .pyx.

• At the current state (November-2023) we still need to add a setup.py if we have C extensions in our package.
• The editable-mode is not working with C extensions.

********************************************************************************
An error happened while installing `quick-notes` in editable mode.

The following steps are recommended to help debug this problem:

- Try to install the project normally, without using the editable mode.
  Does the error still persist?
  (If it does, try fixing the problem before attempting the editable mode).
- If you are using binary extensions, make sure you have all OS-level
  dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
- Try the latest version of setuptools (maybe the error was already fixed).
- If you (or your project dependencies) are using any setuptools extension
  or customization, make sure they support the editable mode.

After following the steps above, if the problem still persists and
you think this is related to how setuptools handles editable installations,
please submit a reproducible example
(see https://stackoverflow.com/help/minimal-reproducible-example) to:

    https://github.com/pypa/setuptools/issues

See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.
********************************************************************************

Important

I found the cause of the error. The issue was that the shared library generated during the build (compiled from the c files generated by Cython) were being stored in "quick_notes.api" instead of "danoan.quick_notes.api". The solution I found was to explicitly describe the c extensions in the setup.py file.

from setuptools import setup, Extension
from Cython.Build import cythonize

setup(
    ext_modules=cythonize(
        [
            Extension(name="danoan.quick_notes.api.to_markdown",
                      sources=["src/danoan/quick_notes/api/to_markdown.pyx"]
                      ),
            Extension(name="danoan.quick_notes.api.to_toml",
                      sources=["src/danoan/quick_notes/api/to_toml.pyx"]
                      ),
            Extension(name="danoan.quick_notes.api.model",
                      sources=["src/danoan/quick_notes/api/model.pyx"]
                      )
        ]
    )
)

Python API or Cython

You can write a C-extension yourself using the Python API. Here it is an snippet of an example from a post in RealPython:

#include <Python.h>


static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;


    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;

    }


    FILE *fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);

    fclose(fp);


    return PyLong_FromLong(bytes_copied);

}

However, it can be much more challenging that using Cython. Cython is a superset of the Python language, and you can get a good amount of optimization by simply change the python file extensions to .pyx.

Answer from ChatGPT

General Considerations:

• Project Size and Complexity: For smaller projects or projects with simpler performance needs, Cython's ease of use may be more beneficial. For larger projects with specific optimization requirements, writing C-extensions using the Python API may be preferred.

• Development Team Skillset: The skillset and experience of the development team can influence the choice. If the team is more comfortable with Python, Cython might be a more accessible option.

• Compatibility and Portability: Consider the compatibility and portability requirements of your project. Cython might offer more flexibility in terms of platform support.

In summary, the choice between Cython and writing a C-extension using the Python API depends on the specific requirements of your project, the development team's expertise, and the desired level of control and optimization. Many projects find Cython to strike a good balance between performance and ease of development.