Skip to content

Latest commit

 

History

History
138 lines (108 loc) · 4.39 KB

python.md

File metadata and controls

138 lines (108 loc) · 4.39 KB
Raw
Linux macOS Windows
Status Status Status

Introduction

To be compliant with PEP513 a python package should embbed all its C++ shared libraries.

Creating a Python native package containing all .py and .so (with good rpath/loaderpath) is not so easy...

Build the Binary Package

To build the Python wheel package, simply run:

cmake -S. -Bbuild -DBUILD_PYTHON=ON
cmake --build build --target python_package -v

note: Since python_package is in target all, you can also omit the --target option.

Testing

To list tests:

cd build
ctest -N

To only run Python tests:

cd build
ctest -R "python_.*"

Technical Notes

First you should take a look at the ortools/python/README.md to understand the layout.
Here I will only focus on the CMake/SWIG tips and tricks.

Build directory layout

Since Python use the directory name where __init__.py file is located and we want to use the CMAKE_BINARY_DIR to generate the Python binary package.

We want this layout:

<CMAKE_BINARY_DIR>/python
├── setup.py
└── ortools
    ├── __init__.py
    ├── algorithms
    │   ├── __init__.py
    │   ├── _pywrap.so
    │   └── pywrap.py
    ├── graph
    │   ├── __init__.py
    │   ├── pywrapgraph.py
    │   └── _pywrapgraph.so
    ├── ...
    └── .libs
        ├── libXXX.so.1.0
        └── libXXX.so.1.0

src: tree build --prune -U -P "*.py|*.so*" -I "build"

note: On UNIX you always need $ORIGIN/../../${PROJECT_NAME}/.libs since _pywrapXXX.so will depend on libortools.so.
note: On APPLE you always need "@loader_path;@loader_path/../../${PROJECT_NAME}/.libs since _pywrapXXX.so will depend on libortools.dylib.
note: on Windows since we are using static libraries we won't have the .libs directory...

So we also need to create few __init__.py files to be able to use the build directory to generate the Python package.

Why on APPLE lib must be .so

Actually, the cpython code responsible for loading native libraries expect .so on all UNIX platforms.

const char *_PyImport_DynLoadFiletab[] = {
#ifdef __CYGWIN__
    ".dll",
#else  /* !__CYGWIN__ */
    "." SOABI ".so",
#ifdef ALT_SOABI
    "." ALT_SOABI ".so",
#endif
    ".abi" PYTHON_ABI_STRING ".so",
    ".so",
#endif  /* __CYGWIN__ */
    NULL,
};

ref: https://github.com/python/cpython/blob/main/Python/dynload_shlib.c#L37-L49

i.e. pywrapXXX -> _pywrapXXX.so -> libortools.dylib

Why setup.py has to be generated

To avoid to put hardcoded path to SWIG .so/.dylib generated files, we could use $<TARGET_FILE_NAME:tgt> to retrieve the file (and also deal with Mac/Windows suffix, and target dependencies).
In order for setup.py to use cmake generator expression (e.g. $<TARGET_FILE_NAME:_pyFoo>). We need to generate it at build time (e.g. using add_custom_command()).

note: This will also add automatically a dependency between the command and the TARGET !

todo(mizux): try to use file(GENERATE ...) instead.