Understanding maturin backend for wheel creation and dynamic dependencies

[DavidAntliff]

I'm not sure yet if this is a bug, a mistake I've made, or just something not supported.

I have a pyproject.toml that looks a bit like this:

[project]
name = my_project
...

dynamic = ["dependencies"]

...

[build-system]
requires = ["maturin>=1.5,<2.0"]
build-backend = "maturin"

[tool.maturin]
bindings = "pyo3"
compatibility = "linux"
python-packages = ["my_project"]
module-name = "my_project.native.my_project_native"

Specifically, the required dependencies specifications are delegated to the setup.py script, a la setuptools:

setup(
    install_requires=install_requires(),
)

def install_requires():
    version = get_version()    # local function that gets a local version + git info
    return [
        f"my_dependency1=={version}",
        f"my_dependency2=={version}",
        "semver==2.13.0",
        "numpy>=1.23",
        "boltons>=23",
    ]

I'm using this because I need to calculate the version of each dependency at build time based on a bunch of internal information like CI pipeline ID, git SHA, mono-repo version, etc.

Now that I'm using the maturin build-backend, instead of the setuptools-rust backend, I now find that the dynamic dependencies are not included in the built wheel, when running:

$ maturin build --release

In fact, looking at the resultant wheel's METADATA file, the Requires-Dist: entries are missing entirely.

Am I right in thinking that switching to the maturin backend disables the ability to use setup.py for dynamic dependency determination?

Is there another supported "dynamic" mechanism for the maturin back-end?

If this is not possible, as an improvement, could maturin issue an error if it encounters dynamic = ["dependencies"]?

I'm also wondering if I should revert back to setuptools-rust (which did work), but I made the switch to maturin because I needed Windows PyO3 cross-compilation, which I couldn't get working with that backend. But this cross-compilation does work with maturin so I'm at a cross-roads...

[DavidAntliff]

Perhaps this isn't supported by maturin.

As a workaround I will look at splitting the project into two packages - one architecturally-dependent package built by maturin for the Rust extension with no dynamic dependencies, and another that uses the setuptools back-end to depend on the Rust one, along with the dynamic dependencies via setup.py.

Does this sound reasonable?

[messense]

Perhaps this isn't supported by maturin.

Yes, maturin does not invoke setup.py because setup.py is a setuptools-thing and maturin does not depend on setuptools at all.

Your workaround sounds reasonable.

[DavidAntliff]

The workaround is working, it seems.

Is maturin likely to ever support dynamic version or dynamic dependencies? Obviously setup.py isn't the way to do this, that's for setuptools, but I haven't found any other mechanism either.

If not, I think maturin should raise an error if, say, dynamic = ["dependencies"] is encountered, because PEP-0621 states:

"Build back-ends MUST raise an error if the metadata specifies a field in dynamic but the build back-end was unable to provide the data for it."

Additionally in the Packaging spec, it adds to this slightly:

"Build back-ends MUST raise an error if the metadata specifies a key in dynamic but the build back-end was unable to determine the data for it (omitting the data, if determined to be the accurate value, is acceptable)."

Maybe "omitting the data, if determined to be the accurate value, is acceptable" is the way out for now, although I don't really understand that sentence!