cupy v13.0.0 #241

regro-cf-autotick-bot · 2024-01-18T17:16:03Z

Close #229.

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

Dependencies have been updated if changed: see upstream
Tests have passed
Updated license if changed and license_file is packaged

Information about this PR:

Feel free to push to the bot's branch to update this PR if needed.
The bot will almost always only open one PR per version.
The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
If you want these PRs to be merged automatically, make an issue with @conda-forge-admin,please add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Pending Dependency Version Updates

Here is a list of all the pending dependency version updates for this repo. Please double check all dependencies before merging.

Name	Upstream Version	Current Version
cudatoolkit	12.0.0

Dependency Analysis

We couldn't run dependency analysis due to an internal error in the bot, depfinder, or grayskull. :/ Help is very welcome!

_{This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/7573336059, please use this URL for debugging.}

conda-forge-webservices · 2024-01-18T17:16:14Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

leofang · 2024-01-18T18:10:18Z

To address #229, I wanna raise a question to @conda-forge/cupy: With CuPy v13 we can move all run dependencies to run_constrained because all CUDA libs are now optional. While this would help deployment on edge devices etc, I think it's a breaking change in terms of installation UX, as users need to explicitly install all dependencies manually.

My proposal:

Add a new output cupy-core which has everything run-constrained.
- FYI, XXXXX-core is a common naming scheme for packaging core/essential capabilities of an otherwise extended project on conda-forge
Keep cupy as before (with dependencies listed in run), to avoid breaking change
Document cupy-core: Update conda installation guide cupy/cupy#8129

Thoughts?

…nda-forge-pinning 2024.01.18.06.29.38

bdice · 2024-01-19T14:06:10Z

@leofang I like the cupy-core proposal. I agree the installation UX of including CUDA package dependencies by default is worth preserving.

vyasr · 2024-01-19T14:44:41Z

I concur with the benefits of having two separate packages. I think cupy-core would be especially helpful to other libraries that use a subset of cupy's functionality internally since they could specify the subset of required dependencies, while preserving the benefits of having all dependencies installed for cupy end-users.

leofang · 2024-01-19T15:09:46Z

@conda-forge-admin, please rerender

github-actions · 2024-01-19T15:12:30Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/cupy-feedstock/actions/runs/7585579292.

leofang · 2024-01-19T15:20:24Z

@conda-forge-admin, please rerender

…nda-forge-pinning 2024.01.19.13.31.20

leofang · 2024-01-19T15:29:23Z

@beckermr @jakirkham I am unable to disable Python 3.8 which CuPy v13 no longer supports. Could this be a bug in conda-smithy? Could this relate to the Python 3.12 migration issue that we were unable to fix (#234)?

jakirkham · 2024-01-20T01:43:09Z

@conda-forge-admin , please re-render

…nda-forge-pinning 2024.01.19.20.48.12

jakirkham · 2024-01-20T01:48:36Z

Add a dummy top-level requirements section with build and host.

Did essentially this and added python, which seems to do the trick

recipe/meta.yaml

jakirkham · 2024-01-20T01:52:19Z

recipe/meta.yaml

 requirements:
-  build:
-    - {{ compiler("c") }}
-    - {{ compiler("cxx") }}
-    - {{ compiler("cuda") }}
-    #- cuda-nvcc
-    - sysroot_{{ target_platform }} 2.17    # [linux]
-    - cross-python_{{ target_platform }}    # [build_platform != target_platform]
-    - python                                # [build_platform != target_platform]
-    - cython >=0.29.22,<3                   # [build_platform != target_platform]
-    # TODO: clean up
-    {% if cuda_major >= 12 %}
-    - cuda-driver-dev                       # [linux and build_platform != target_platform]
-    - cuda-cudart-dev                       # [build_platform != target_platform]
-    - cuda-nvrtc-dev                        # [build_platform != target_platform]
-    - cuda-nvtx-dev                         # [build_platform != target_platform]
-    - cuda-profiler-api                     # [build_platform != target_platform]
-    - cuda-cccl_{{ build_platform }}        # [build_platform != target_platform]
-    - libcublas-dev                         # [build_platform != target_platform]
-    - libcufft-dev                          # [build_platform != target_platform]
-    - libcurand-dev                         # [build_platform != target_platform]
-    - libcusolver-dev                       # [build_platform != target_platform]
-    - libcusparse-dev                       # [build_platform != target_platform]
-    {% endif %}
-    # optional dependencies for CUDA 11.2+
-    - cudnn >=8.0.*,<9                      # [build_platform != target_platform and (not ((aarch64 or ppc64le) and (cuda_compiler_version or "").startswith("12")))]
-    - nccl >=2.8,<3                         # [build_platform != target_platform]
-    - cutensor >=1.4,<2                     # [build_platform != target_platform]
-    - cusparselt ~=0.2.0.0                  # [build_platform != target_platform and (linux64 or aarch64 or win) and (cuda_compiler_version or "").startswith("11")]
-
  host:
    - python
-    - pip
-    - setuptools
-    - cython >=0.29.22,<3
-    - fastrlock >=0.5
-    - cuda-version {{ cuda_compiler_version }}
-    - nvtx-c  # [win64]
-    # TODO: clean up
-    {% if cuda_major >= 12 %}
-    - cuda-driver-dev  # [linux]
-    - cuda-cudart-dev
-    - cuda-nvrtc-dev
-    - cuda-nvtx-dev
-    - cuda-profiler-api
-    - cuda-cccl_{{ target_platform }}
-    - libcublas-dev
-    - libcufft-dev
-    - libcurand-dev
-    - libcusolver-dev
-    - libcusparse-dev
-    {% endif %}
-    # optional dependencies
-    # TODO: see https://github.com/conda-forge/cudnn-feedstock/issues/58
-    - cudnn >=8.0.*,<9      # [not ((aarch64 or ppc64le) and (cuda_compiler_version or "").startswith("12"))]
-    - nccl >=2.8,<3         # [linux]
-    - cutensor >=1.4,<2
-    - cusparselt ~=0.2.0.0  # [(linux64 or aarch64 or win) and (cuda_compiler_version or "").startswith("11")]


Think we may want to keep these top-level requirements at least for build & host

Why? TBH I don't quite understand the purpose of top-level requirements, when there are multiple outputs. Seems like just hacks to me for bowing to the limitations of conda-build & co? But what do we wish to achieve here, when it's working fine now?

The original idea of the multi-outputs recipe was to largely handle everything (like the build, dependencies at build time, etc.) at the top-level

The outputs sections themselves were mainly geared around taking the resulting contents of $PREFIX and slicing them up in different ways to create several (potentially interrelated) packages

Over time people have added more things to the subpackages sections (different requirement stages, more complicated build & install patterns, etc.). As we have seen these later additions are incomplete

This is why we see issues with Python and PIP_* environment variables, misunderstandings by the build tooling about how to build the recipe, etc.

Hewing close to the original model will minimize the chance of niche issues cropping up

Thanks, John, for sharing the background. My preference is to keep the recipe as-is and see what else issue we could run into, unless you have strong opinion otherwise. The status quo is cleaner and makes better sense to reflect the intention IMHO. I am also very curious to see how this would involve into, once the new recipe format is up and running.

MNT: Re-rendered with conda-build 3.23.1, conda-smithy 3.30.4, and conda-forge-pinning 2024.01.22.14.29.27

leofang · 2024-01-22T19:58:09Z

The PR is ready. Will merge later tonight / tomorrow unless there are objections.

Lessons learned in this PR

Cannot skip PY38:
- solution: Add python as a host requirement, as well as the skip condition, to the top-level and all relevant sub packages
- xref: cupy v13.0.0 #241 (comment)
cutensor 1.* seems to be cached somewhere
- solution: It's cached in the global pinning, just add a migrator for 2.x
- ~~TODO: Send the migrator to the global pinning feedstock~~ Update: Add cuTENSOR 2 migrator conda-forge-pinning-feedstock#5410
Jinja variable {{ PYTHON }} does not resolve in a multi-output recipe
- solution: Hard-code the host Python path manually
Included patch gets outdated (cc: @kmaehashi for vis)
- solution: Co-develop the patch with an upstream PR Patch the build system to better support conda-build cupy/cupy#7603
CUDA source compilation failed on Windows:
- solution: Increase the swap size via adding SET_PAGEFILE to conda-forge.yaml.

leofang · 2024-01-23T02:06:11Z

Merging, thanks @beckermr @jaimergp @jakirkham for help and @jakirkham @vyasr @bdice for reviewing the cupy-core proposal. In we go!

updated v13.0.0

620eb3b

regro-cf-autotick-bot requested review from asi1024, emcastillo, jakirkham, kmaehashi, leofang and toslunar as code owners January 18, 2024 17:16

leofang marked this pull request as draft January 18, 2024 18:06

This comment was marked as resolved.

Sign in to view

add cupy-core

91d1193

leofang force-pushed the 13.0.0_h238c25 branch from 19a916f to 91d1193 Compare January 19, 2024 04:24

This comment was marked as outdated.

Sign in to view

fix build_num

f86e03a

This comment was marked as outdated.

Sign in to view

conda-forge-webservices[bot] and others added 2 commits January 19, 2024 04:32

MNT: Re-rendered with conda-build 3.28.4, conda-smithy 3.30.4, and co…

2a1c246

…nda-forge-pinning 2024.01.18.06.29.38

fix

54f17b6

This comment was marked as outdated.

Sign in to view

Update dependency requirements

590d092

also copy skip/about to each output

5e09147

MNT: Re-rendered with conda-build 3.28.4, conda-smithy 3.30.4, and co…

332c6d3

…nda-forge-pinning 2024.01.19.13.31.20

jakirkham added 2 commits January 19, 2024 17:42

Drop extra blank line

6726661

Add top-level requirement/host python

6160059

MNT: Re-rendered with conda-build 3.28.4, conda-smithy 3.30.4, and co…

1c6d1bb

…nda-forge-pinning 2024.01.19.20.48.12

jakirkham reviewed Jan 20, 2024

View reviewed changes

recipe/meta.yaml Outdated Show resolved Hide resolved

jakirkham reviewed Jan 20, 2024

View reviewed changes

leofang force-pushed the 13.0.0_h238c25 branch from e26ee3d to 78be34d Compare January 20, 2024 03:07

fix static lib search when cross compiling

2cebe8b

leofang force-pushed the 13.0.0_h238c25 branch from 78be34d to 2cebe8b Compare January 20, 2024 03:15

leofang added 6 commits January 21, 2024 17:04

pin sub package exactly; pin host/run compatible; nit for patch

230b504

cccl is no longer a build-time dependency

65ec5db

fix missing tests

3d200d8

fix nvtx error exposed via the test

4c64e1c

nit: fix metadata copy/paste mistake

3f87da9

turn off artifact uploading

abf5c40

MNT: Re-rendered with conda-build 3.23.1, conda-smithy 3.30.4, and conda-forge-pinning 2024.01.22.14.29.27

leofang marked this pull request as ready for review January 22, 2024 17:51

This was referenced Jan 22, 2024

Add cuTENSOR 2 migrator conda-forge/conda-forge-pinning-feedstock#5410

Merged

Update conda installation guide cupy/cupy#8129

Merged

leofang merged commit 8b01da7 into conda-forge:main Jan 23, 2024
33 checks passed

leofang mentioned this pull request Jan 23, 2024

Support lazy import in v13 #229

Closed

regro-cf-autotick-bot deleted the 13.0.0_h238c25 branch January 23, 2024 02:07

This was referenced Jan 23, 2024

Patch the build system to better support conda-build cupy/cupy#7603

Merged

Add cuda-version to run #249

Merged

leofang mentioned this pull request Jan 31, 2024

Relax cuda-python run dependencies conda-forge/cuda-python-feedstock#66

Closed

1 task

weiji14 mentioned this pull request Jun 19, 2024

Enable API reference docs to show accessor methods xarray-contrib/cupy-xarray#44

Merged

leofang mentioned this pull request Jul 26, 2024

Refactor recipe + Relax CUDA 11 dependency + Drop PY312 dx builds + Kill redundant PY38 pipelines conda-forge/nvmath-feedstock#7

Merged

1 task

jakirkham mentioned this pull request Oct 7, 2024

Fix NumPy 2 upper bound for older CuPy packages conda-forge/conda-forge-repodata-patches-feedstock#873

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cupy v13.0.0 #241

cupy v13.0.0 #241

regro-cf-autotick-bot commented Jan 18, 2024 •

edited by leofang

Loading

conda-forge-webservices bot commented Jan 18, 2024

leofang commented Jan 18, 2024 •

edited

Loading

This comment was marked as resolved.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

bdice commented Jan 19, 2024

vyasr commented Jan 19, 2024 •

edited

Loading

leofang commented Jan 19, 2024

github-actions bot commented Jan 19, 2024

leofang commented Jan 19, 2024

leofang commented Jan 19, 2024

jakirkham commented Jan 20, 2024

jakirkham commented Jan 20, 2024

jakirkham Jan 20, 2024

leofang Jan 20, 2024

jakirkham Jan 20, 2024

leofang Jan 22, 2024

leofang commented Jan 22, 2024 •

edited

Loading

leofang commented Jan 23, 2024

cupy v13.0.0 #241

cupy v13.0.0 #241

Conversation

regro-cf-autotick-bot commented Jan 18, 2024 • edited by leofang Loading

Pending Dependency Version Updates

Dependency Analysis

conda-forge-webservices bot commented Jan 18, 2024

leofang commented Jan 18, 2024 • edited Loading

This comment was marked as resolved.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

bdice commented Jan 19, 2024

vyasr commented Jan 19, 2024 • edited Loading

leofang commented Jan 19, 2024

github-actions bot commented Jan 19, 2024

leofang commented Jan 19, 2024

leofang commented Jan 19, 2024

jakirkham commented Jan 20, 2024

jakirkham commented Jan 20, 2024

jakirkham Jan 20, 2024

Choose a reason for hiding this comment

leofang Jan 20, 2024

Choose a reason for hiding this comment

jakirkham Jan 20, 2024

Choose a reason for hiding this comment

leofang Jan 22, 2024

Choose a reason for hiding this comment

leofang commented Jan 22, 2024 • edited Loading

leofang commented Jan 23, 2024

regro-cf-autotick-bot commented Jan 18, 2024 •

edited by leofang

Loading

leofang commented Jan 18, 2024 •

edited

Loading

vyasr commented Jan 19, 2024 •

edited

Loading

leofang commented Jan 22, 2024 •

edited

Loading