Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for Python 3.11, require NumPy 1.23+ #15111

Merged
merged 31 commits into from
Feb 29, 2024
Merged

Add support for Python 3.11, require NumPy 1.23+ #15111

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
bf0de6c
test shared-workflows matrix refactoring
jameslamb Feb 21, 2024
29b8b63
empty commit to re-trigger CI
jameslamb Feb 21, 2024
0b75781
Merge branch 'branch-24.04' of github.com:rapidsai/cudf into test-mat…
jameslamb Feb 22, 2024
d89ae6d
test python-3.11
jameslamb Feb 22, 2024
f10f2ba
Merge branch 'branch-24.04' into test-matrix-refactor
jameslamb Feb 23, 2024
971ee7a
apply Python 3.11 changes
jameslamb Feb 23, 2024
e2a6bda
Merge branch 'branch-24.04' of github.com:rapidsai/cudf into test-mat…
jameslamb Feb 26, 2024
e298c3a
update matrix filters
jameslamb Feb 26, 2024
37c675a
revert GitHub actions configs
jameslamb Feb 26, 2024
a8fd0d5
Update README.md
jameslamb Feb 27, 2024
7310cd7
Merge branch 'branch-24.04' into test-matrix-refactor
KyleFromNVIDIA Feb 27, 2024
cc4d677
Merge branch 'branch-24.04' into test-matrix-refactor
KyleFromNVIDIA Feb 27, 2024
e299cea
Merge branch 'branch-24.04' of github.com:rapidsai/cudf into test-mat…
jameslamb Feb 28, 2024
72e8e29
require NumPy 1.23+
jameslamb Feb 28, 2024
1815ddc
Require NumPy in Conda build reqs
jakirkham Feb 28, 2024
7ac7fec
Build with NumPy 1.23 & depend on NumPy 1.23+
jakirkham Feb 28, 2024
c1d0639
Simplify `dependencies.yaml`
jakirkham Feb 28, 2024
ba5e9dd
Drop unused `pyarrow` alias
jakirkham Feb 28, 2024
43b1283
Clean up comment
jakirkham Feb 28, 2024
f3f7d82
Bump tokenizers to 0.15.2
KyleFromNVIDIA Feb 28, 2024
351890d
Bump transformers to 4.38.1
KyleFromNVIDIA Feb 28, 2024
7fb5d39
Merge branch 'branch-24.04' into test-matrix-refactor
KyleFromNVIDIA Feb 28, 2024
a434cbd
Use re.U instead of accidentally using re.T
KyleFromNVIDIA Feb 28, 2024
016d1aa
Use pytorch>=2.1.0.
bdice Feb 28, 2024
6d4f1a9
Merge branch 'branch-24.04' into test-matrix-refactor
jakirkham Feb 29, 2024
2f0a8a8
Work around failing tests in PyTorch CUDA Array Interface.
bdice Feb 29, 2024
631e13b
Merge branch 'test-matrix-refactor' of github.com:jameslamb/cudf into…
bdice Feb 29, 2024
0ca1d9b
Merge branch 'branch-24.04' into test-matrix-refactor
bdice Feb 29, 2024
d7231a0
Update test_subword_tokenizer.py
galipremsagar Feb 29, 2024
c36707a
Merge branch 'branch-24.04' into test-matrix-refactor
galipremsagar Feb 29, 2024
dd013bd
copyright update
galipremsagar Feb 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/workflows/build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ concurrency:
jobs:
cpp-build:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@python-3.11
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -37,7 +37,7 @@ jobs:
python-build:
needs: [cpp-build]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@python-3.11
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -46,7 +46,7 @@ jobs:
upload-conda:
needs: [cpp-build, python-build]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-upload-packages.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-upload-packages.yaml@python-3.11
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -57,7 +57,7 @@ jobs:
if: github.ref_type == 'branch'
needs: python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.11
with:
arch: "amd64"
branch: ${{ inputs.branch }}
Expand All @@ -69,7 +69,7 @@ jobs:
sha: ${{ inputs.sha }}
wheel-build-cudf:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.11
with:
build_type: ${{ inputs.build_type || 'branch' }}
build-2_28-wheels: "true"
Expand All @@ -80,7 +80,7 @@ jobs:
wheel-publish-cudf:
needs: wheel-build-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@python-3.11
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -90,7 +90,7 @@ jobs:
wheel-build-dask-cudf:
needs: wheel-publish-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.11
with:
matrix_filter: map(select(.ARCH == "amd64" and .PY_VER == "3.10" and (.CUDA_VER == "11.8.0" or .CUDA_VER == "12.2.2")))
build_type: ${{ inputs.build_type || 'branch' }}
Expand All @@ -101,7 +101,7 @@ jobs:
wheel-publish-dask-cudf:
needs: wheel-build-dask-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@python-3.11
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand Down
38 changes: 19 additions & 19 deletions .github/workflows/pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,56 +32,56 @@ jobs:
#- pandas-tests-diff
#- pandas-tests-diff-comment
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/pr-builder.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/pr-builder.yaml@python-3.11
checks:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/checks.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/checks.yaml@python-3.11
with:
enable_check_generated_files: false
conda-cpp-build:
needs: checks
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@python-3.11
with:
build_type: pull-request
conda-cpp-checks:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-post-build-checks.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-post-build-checks.yaml@python-3.11
with:
build_type: pull-request
enable_check_symbols: true
conda-cpp-tests:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@python-3.11
with:
build_type: pull-request
conda-python-build:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@python-3.11
with:
build_type: pull-request
conda-python-cudf-tests:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@python-3.11
with:
build_type: pull-request
test_script: "ci/test_python_cudf.sh"
conda-python-other-tests:
# Tests for dask_cudf, custreamz, cudf_kafka are separated for CI parallelism
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@python-3.11
with:
build_type: pull-request
test_script: "ci/test_python_other.sh"
conda-java-tests:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.11
with:
build_type: pull-request
node_type: "gpu-v100-latest-1"
Expand All @@ -91,7 +91,7 @@ jobs:
conda-notebook-tests:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.11
with:
build_type: pull-request
node_type: "gpu-v100-latest-1"
Expand All @@ -101,7 +101,7 @@ jobs:
docs-build:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.11
with:
build_type: pull-request
node_type: "gpu-v100-latest-1"
Expand All @@ -111,37 +111,37 @@ jobs:
wheel-build-cudf:
needs: checks
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.11
with:
build_type: pull-request
build-2_28-wheels: "true"
script: "ci/build_wheel_cudf.sh"
wheel-tests-cudf:
needs: wheel-build-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.11
with:
build_type: pull-request
script: ci/test_wheel_cudf.sh
wheel-build-dask-cudf:
needs: wheel-build-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.11
with:
matrix_filter: map(select(.ARCH == "amd64" and .PY_VER == "3.10" and (.CUDA_VER == "11.8.0" or .CUDA_VER == "12.2.2")))
build_type: pull-request
script: "ci/build_wheel_dask_cudf.sh"
wheel-tests-dask-cudf:
needs: wheel-build-dask-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.11
with:
matrix_filter: map(select(.ARCH == "amd64" and .PY_VER == "3.10" and (.CUDA_VER == "11.8.0" or .CUDA_VER == "12.2.2")))
build_type: pull-request
script: ci/test_wheel_dask_cudf.sh
devcontainer:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/build-in-devcontainer.yaml@branch-24.04
uses: rapidsai/shared-action-workflows/.github/workflows/build-in-devcontainer.yaml@python-3.11
with:
build_command: |
sccache -z;
Expand All @@ -150,7 +150,7 @@ jobs:
unit-tests-cudf-pandas:
needs: wheel-build-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.11
with:
matrix_filter: map(select(.ARCH == "amd64" and .PY_VER == "3.10" and (.CUDA_VER == "11.8.0" or .CUDA_VER == "12.2.2")))
bdice marked this conversation as resolved.
Show resolved Hide resolved
build_type: pull-request
Expand All @@ -159,7 +159,7 @@ jobs:
# run the Pandas unit tests using PR branch
needs: wheel-build-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.11
with:
matrix_filter: map(select(.ARCH == "amd64")) | max_by(.CUDA_VER) | [.]
build_type: pull-request
Expand All @@ -171,7 +171,7 @@ jobs:
# needs: [pandas-tests-main, pandas-tests-pr]
# secrets: inherit
# # This branch exports a `job_output` output that the downstream job reads.
# uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.04
# uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.11
# with:
# node_type: cpu4
# build_type: pull-request
Expand Down
22 changes: 11 additions & 11 deletions .github/workflows/test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ on:
jobs:
conda-cpp-checks:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-post-build-checks.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-post-build-checks.yaml@python-3.11
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand All @@ -25,15 +25,15 @@ jobs:
enable_check_symbols: true
conda-cpp-tests:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@python-3.11
with:
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
conda-cpp-memcheck-tests:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.11
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand All @@ -45,7 +45,7 @@ jobs:
run_script: "ci/test_cpp_memcheck.sh"
conda-python-cudf-tests:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@python-3.11
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand All @@ -55,7 +55,7 @@ jobs:
conda-python-other-tests:
# Tests for dask_cudf, custreamz, cudf_kafka are separated for CI parallelism
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@python-3.11
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand All @@ -64,7 +64,7 @@ jobs:
test_script: "ci/test_python_other.sh"
conda-java-tests:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.11
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand All @@ -76,7 +76,7 @@ jobs:
run_script: "ci/test_java.sh"
conda-notebook-tests:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.11
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand All @@ -88,7 +88,7 @@ jobs:
run_script: "ci/test_notebooks.sh"
wheel-tests-cudf:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.11
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand All @@ -97,7 +97,7 @@ jobs:
script: ci/test_wheel_cudf.sh
wheel-tests-dask-cudf:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.11
with:
matrix_filter: map(select(.ARCH == "amd64" and .PY_VER == "3.10" and (.CUDA_VER == "11.8.0" or .CUDA_VER == "12.2.2")))
build_type: nightly
Expand All @@ -107,7 +107,7 @@ jobs:
script: ci/test_wheel_dask_cudf.sh
unit-tests-cudf-pandas:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.11
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand All @@ -117,7 +117,7 @@ jobs:
pandas-tests:
# run the Pandas unit tests
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.04
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.11
with:
matrix_filter: map(select(.ARCH == "amd64")) | max_by(.CUDA_VER) | [.]
build_type: nightly
Expand Down
Loading