Distribute libcuspatial wheels

[jameslamb]

Description

Contributes to rapidsai/build-planning#33

Adds libcuspatial wheels, and switches cuspatial wheels to using them.

Notes for Reviewers

Benefits of these changes

Faster CI runs and smaller total footprint on package repositories (because now libcuspatial no longer needs to be compiled once per Python version).

Smaller cuspatial wheels.

whee. l	size (before)	size (this PR)
libcuspatial	---	17.0M
cuspatial.	21.0M	4.1M
cuproj	0.9M	0.9M
TOTAL	21.9M	22.0M

NOTES: size = compressed, "before" = 2024-09-02 nightlies (1544e7b), CUDA = 12, Python = 3.11

how I calculated those (click me)

docker run \
    --rm \
    -v $(pwd):/opt/work:ro \
    -w /opt/work \
    --network host \
    --env RAPIDS_NIGHTLY_DATE=2024-09-02 \
    --env RAPIDS_NIGHTLY_SHA=1544e7b \
    --env RAPIDS_PR_NUMBER=1450 \
    --env RAPIDS_PY_CUDA_SUFFIX=cu12 \
    --env RAPIDS_REPOSITORY=rapidsai/cuspatial \
    --env WHEEL_DIR_BEFORE=/tmp/wheels-before \
    --env WHEEL_DIR_AFTER=/tmp/wheels-after \
    -it rapidsai/ci-wheel:cuda12.5.1-rockylinux8-py3.11 \
    bash

mkdir -p "${WHEEL_DIR_BEFORE}"
mkdir -p "${WHEEL_DIR_AFTER}"

py_projects=(
    cuspatial
    cuproj
)

for project in "${py_projects[@]}"; do
    # before
    RAPIDS_BUILD_TYPE=nightly \
    RAPIDS_PY_WHEEL_NAME="${project}_${RAPIDS_PY_CUDA_SUFFIX}" \
    RAPIDS_REF_NAME="branch-24.10" \
    RAPIDS_SHA=${RAPIDS_NIGHTLY_SHA} \
        rapids-download-wheels-from-s3 python "${WHEEL_DIR_BEFORE}"

    # after
    RAPIDS_BUILD_TYPE=pull-request \
    RAPIDS_PY_WHEEL_NAME="${project}_${RAPIDS_PY_CUDA_SUFFIX}" \
    RAPIDS_REF_NAME="pull-request/${RAPIDS_PR_NUMBER}" \
        rapids-download-wheels-from-s3 python "${WHEEL_DIR_AFTER}"
done

# after
RAPIDS_BUILD_TYPE=pull-request \
RAPIDS_PY_WHEEL_NAME="libcuspatial_${RAPIDS_PY_CUDA_SUFFIX}" \
RAPIDS_REF_NAME="pull-request/${RAPIDS_PR_NUMBER}" \
    rapids-download-wheels-from-s3 cpp "${WHEEL_DIR_AFTER}"

du -sh ${WHEEL_DIR_BEFORE}/*
du -sh ${WHEEL_DIR_BEFORE}
du -sh ${WHEEL_DIR_AFTER}/*
du -sh ${WHEEL_DIR_AFTER}

devcontainers job?

Once this PR is close to ready, let's merge the devcontainers PR and then re-run the devcontainers CI here.

devcontainers PR: rapidsai/devcontainers#387

rapids-metadata changes?

Not necessary, libcuspatial is already there: https://github.com/rapidsai/rapids-metadata/blob/9b6307e708511cd9a1990d8bb36606df53bc9e1b/src/rapids_metadata/__init__.py#L89

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[bdice]

Re-approving.

[harrism]

My review is requested but I have no expertise on wheels/pip.

[harrism]

Approving cmake changes.

[vyasr]

Could you remind me why we're setting this? Is it strictly necessary? Why aren't the install rules that come from add_subdirectory sufficient? If we do have to set this variable, why do we have to set it but also do a separate install below? It seems like either one or the other should be sufficient. I know this code is in the libcudf wheel now as well, but I don't understand why.

[jameslamb]

I copied this in here from libcudf because I think I misunderstood the purpose there.

There, this is used to ensure all the nvcomp libraries and libcudf.so all get put together into the same place: https://github.com/rapidsai/cudf/blob/26091a44b3dbf0f56fc0dfc5f081077f2d00681f/python/libcudf/CMakeLists.txt#L50-L53

That isn't necessary here, and you're right that the install rules from the add_subdirectory() are sufficient. Removed this stuff in fb307ba

[vyasr]

I suspect that the libcudf install rules are also a little overspecified now. Simultaneously setting both the CMake var and doing the install of cudf in particular seems redundant. The nvcomp installation might be fine as is. We can revisit those later, though.

[vyasr]

Mostly confused by the CMake installation, otherwise everything else looks OK to me.

[jameslamb]

I've merged rapidsai/devcontainers#387 and restarted the pip devcontainer CI job here. Will merge this if it passes.

build link: https://github.com/rapidsai/cuspatial/actions/runs/10705830747/job/29698846144?pr=1450

[jameslamb]

/merge