Build Linux wheels in GitHub CI (#60)

Apart from the usual benefit of CI for testing proposed changes, this
will eventually make installing python-poppler-qt5 much
easier. Currently, installing from PyPI compiles from source, which
results in a bad user experience since lots of tools and a somewhat
delicate setup are needed. The added CI job builds precompiled packages
("wheels") that can be uploaded to PyPI and should be compatible with
current Linux distributions. The eventual goal is to improve this to
build macOS and Windows wheels too.

Relates to #42

.github/workflows/build_wheels.yml

@@ -0,0 +1,28 @@
+on: [push, pull_request]
+
+
+# For now only Linux wheels with Python 3.11. TODO: cover other Python
+# versions and macOS / Windows.
+
+jobs:
+  build_wheels_linux:
+    name: Build Linux wheels
+    # The Docker container the job is run in.
+    container: quay.io/pypa/manylinux2014_x86_64
+    # The host system that runs Docker.
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: conda-incubator/setup-miniconda@v2
+        with:
+          miniconda-version: latest
+          environment-file: ci/environment/conda-linux-64.lock
+      - name: Build the wheels
+        # The following line here and elsewhere is needed for our Conda environment to
+        # be activated. See:
+        # https://github.com/marketplace/actions/setup-miniconda#important
+        shell: bash -el {0}
+        run: ci/build.sh
+      - uses: actions/upload-artifact@v3
+        with:
+          path: "fixed-wheel/*.whl"

README.rst

@@ -24,12 +24,6 @@ Documentation
 The Python API closely follows the Poppler Qt5 C++ interface library API,
 documented at https://poppler.freedesktop.org/api/qt5/ .
 
-Note: Releases of PyQt5 < 5.4 currently do not support the QtXml module,
-all methods that use the QDomDocument, QDomElement and QDomNode types are
-disabled. This concerns the ``Document::toc()`` method and some constructors
-and the ``store()`` methods in the ``Annotation`` subclasses. So, using
-PyQt5 >= 5.4 is recommended.
-
 Wherever the C++ API requires ``QList``, ``QSet`` or ``QLinkedList``, any
 Python sequence can be used. API calls that return ``QList``, ``QSet`` or
 ``QLinkedList`` all return Python lists.
@@ -71,4 +65,3 @@ functions:
     This is determined at build time. If at build time the Poppler-Qt5 version
     could not be determined and was not specified, an empty tuple might be
     returned.
-

ci/README.md

@@ -0,0 +1,36 @@
+The CI setup is made of this directory (`ci/`) as well as the GitHub workflow
+definition in `.github/workflows/`.
+
+This CI builds wheels (precompiled packages) of python-poppler-qt5, currently
+only for Linux. These wheels can be used on any reasonably recent Linux
+system.
+
+Building the wheels is an involved process. First, we need to get a version of
+PyQt5 that can be used for compilation, i.e., it must contain the C++ headers
+and CMake helper files. Unfortunately, this is not the case of the PyQt5-Qt5
+distribution on PyPI. This is why we use Conda to get Qt5 (as well as other
+stuff).
+
+Since Linux binaries link to the system glibc, which is binary
+backwards-compatible but not forwards-compatible, getting a wheel that is
+actually compatible with many Linux systems requires compiling against an old
+glibc. This is also nicely achieved by using the libc in Conda's build
+environment.
+
+(Note that this assumes that the compiler and compiler flags in use in the Conda
+environment are ABI-compatible with the PyQt5-Qt5 wheels on PyPI. If by
+misfortune that ceases being the case in the future, we will need to find
+another strategy.)
+
+We build Poppler itself from source because it links to a large number of
+external libraries by default and it would be impractical to bundle them all in
+the wheels: not only would it increase the size, but for some of these
+libraries, it is questionable that bundling them is the right thing to do (e.g.,
+graphics libraries that might need to be system-dependent). By turning off lots
+of features, we reduce the set of external libraries to link to only libjpeg and
+libopenjp2.
+
+Note: Searching for fonts on the system (for PDFs that don't embed their own
+fonts, which are rare) is disabled since it would require also shipping
+libfontconfig. Users who want this feature should get a different build of
+python-poppler-qt5.

ci/build.sh

@@ -0,0 +1,97 @@
+#!/bin/bash
+
+set -eu # exit on error
+set -o xtrace # show each command before running
+
+# Sync with pyproject.toml. POPPLER_VERSION is for the download of Poppler and may contain
+# leading zeroes, while PYTHON_POPPLER_QT5_VERSION does not.
+POPPLER_VERSION="21.03.0"
+PYTHON_POPPLER_QT5_VERSION="21.3.0"
+
+# These compiler flags add an RPATH entry with value $ORIGIN (literally),
+# to make the built shared libraries search for Qt and Poppler in the
+# same directory instead of using the system ones. --disable-new-dtags
+# is to generate an RPATH and not a RUNPATH; the latter doesn't apply
+# to transitive dependencies.
+RPATH_OPTION="-Wl,-rpath,'\$ORIGIN',--disable-new-dtags"
+# Shell quoting madness to survive through qmake and make ...
+RPATH_OPTION_2="-Wl,-rpath,'\\'\\\$\\\$ORIGIN\\'',--disable-new-dtags"
+
+# Download and extract the Poppler source code
+
+POPPLER=poppler-$POPPLER_VERSION
+curl -O https://poppler.freedesktop.org/$POPPLER.tar.xz
+tar -xf $POPPLER.tar.xz
+# Patch Poppler to avoid building the tests. Newer Poppler versions have a config
+# variable for this.
+sed -i 's/add_subdirectory(test)//g' $POPPLER/CMakeLists.txt
+
+
+pushd $POPPLER
+
+# Construct build options
+
+CMAKE_OPTIONS=
+# We don't need the Qt6, GLib (GTK) and C++ wrappers, only the Qt5 one.
+CMAKE_OPTIONS+="-DENABLE_QT5=ON"
+CMAKE_OPTIONS+=" -DENABLE_QT6=OFF"
+CMAKE_OPTIONS+=" -DENABLE_GLIB=OFF"
+CMAKE_OPTIONS+=" -DENABLE_CPP=OFF"
+# We don't need the command line utilities (pdfimages, pdfattach, etc.)
+CMAKE_OPTIONS+=" -DENABLE_UTILS=OFF"
+# We don't need libpng or libtiff. Apparently, only they're used in pdfimages.
+# However, the build is not smart enough to avoid searching them if the utilities
+# aren't built.
+CMAKE_OPTIONS+=" -DWITH_PNG=OFF"
+CMAKE_OPTIONS+=" -DWITH_TIFF=OFF"
+# Disable network stuff that's apparently used to validate signatures and the like.
+# We don't need this.
+CMAKE_OPTIONS+=" -DWITH_NSS3=OFF"
+CMAKE_OPTIONS+=" -DENABLE_LIBCURL=OFF"
+# Disable the use of Little CMS. (TODO: maybe this would actually be
+# useful? Investigate.)
+CMAKE_OPTIONS+=" -DENABLE_CMS=none"
+# Disable Cairo backend, it's (famously) not supported in the Qt5 wrapper anyway.
+CMAKE_OPTIONS+=" -DWITH_Cairo=OFF"
+# Disable the use of Fontconfig, it's only needed for PDFs that use external
+# fonts. We don't care to support these.
+CMAKE_OPTIONS+=" -DFONT_CONFIGURATION=generic"
+# Don't build the tests.
+CMAKE_OPTIONS+=" -DBUILD_QT5_TESTS=OFF"
+# Install locally
+CMAKE_OPTIONS+=" -DCMAKE_INSTALL_PREFIX==../../../installed-poppler"
+
+# Generate Poppler Makefile
+LDFLAGS=$RPATH_OPTION \
+PKG_CONFIG_LIBDIR=$CONDA_PREFIX/lib/pkgconfig \
+LIBRARY_PATH=$CONDA_BUILD_SYSROOT/lib:$CONDA_PREFIX/lib \
+    cmake -S . -B build $CMAKE_OPTIONS
+
+# Build Poppler
+pushd build
+make -j$(nproc)
+make install
+popd
+
+popd
+
+# Now build python-poppler-qt5. Add a RUNPATH just like for poppler.
+PKG_CONFIG_LIBDIR=installed-poppler/lib64/pkgconfig:$CONDA_PREFIX/lib/pkgconfig \
+LIBRARY_PATH=$CONDA_BUILD_SYSROOT/lib:$CONDA_PREFIX/lib \
+    sip-wheel --verbose --link-args=$RPATH_OPTION_2 --build-dir=build
+
+# Unpack wheel to tinker with it
+WHEEL=(python_poppler_qt5*.whl)
+wheel unpack $WHEEL
+pushd python_poppler_qt5-$PYTHON_POPPLER_QT5_VERSION
+
+# Vendor libopenjp2 and libjpeg
+cp ../installed-poppler/lib64/*.so* \
+   $CONDA_PREFIX/lib/libopenjp2.so* \
+   $CONDA_PREFIX/lib/libjpeg.so* \
+   PyQt5/Qt5/lib/
+
+# Repack the wheel
+popd
+mkdir fixed-wheel
+wheel pack --dest-dir=fixed-wheel python_poppler_qt5-$PYTHON_POPPLER_QT5_VERSION

ci/environment/README.md

@@ -0,0 +1,63 @@
+This directory contains a lock file that specifies the CI environment used to
+build python-poppler-qt5 wheels. This is used to make the build environment
+reproducible across CI runs, and makes the CI setup much faster.
+
+The `environment.yml` file contains the names of the packages to be
+installed. See [environment-doc] about the format of this file.
+
+The lock file is `conda-lock.yml` and is generated using [conda-lock]. To
+regenerate it:
+
+- Install [Miniconda],
+
+- Install conda-lock into the Conda base environment using
+
+  ```
+  conda install --channel=conda-forge --name=base conda-lock
+  ```
+
+  (you may also create a dedicated environment for it, different than `base`).
+
+- Optional but highly recommended: set the libmamba dependency solver.
+
+  ```
+  conda install --channel=conda-forge --name=base conda-libmamba-solver
+  conda config --set solver libmamba
+  ```
+
+  The dependency resolution takes a lot of time. The libmamba solver is not fast
+  (generating the lock file is expected to take several minutes), but it's
+  faster than conda's default solver.
+
+- Activate the Conda environment where you installed `conda-lock`, e.g.,
+
+  ```
+  conda activate base
+  ```
+
+- Run the command `conda-lock`, which updates `conda-lock.yml`.
+
+- Run `conda-lock render` to also update `conda-linux-64.lock`
+  from `conda-lock.yml`.
+
+To test your platform's environment locally, first run:
+
+```
+conda-lock install --name python-poppler-qt5-test-env conda-lock.yml
+```
+
+You may give the environment a different name than
+`python-poppler-qt5-test-env`.  Afterwards, activate the newly created
+environment with
+
+```
+conda activate python-poppler-qt5-test-env
+```
+
+(use the environment name you chose)
+
+
+
+[environment-doc]: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually
+[conda-lock]: https://github.com/conda/conda-lock
+[Miniconda]: https://docs.conda.io/en/latest/miniconda.html