deepspeed v0.12.0

[regro-cf-autotick-bot]

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

• Dependencies have been updated if changed: see upstream
• Tests have passed
• Updated license if changed and license_file is packaged

Information about this PR:

1. Feel free to push to the bot's branch to update this PR if needed.
2. The bot will almost always only open one PR per version.
3. The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
4. If you want these PRs to be merged automatically, make an issue with @conda-forge-admin,please add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
5. If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Closes: #33

Pending Dependency Version Updates

Here is a list of all the pending dependency version updates for this repo. Please double check all dependencies before merging.

Name	Upstream Version	Current Version
git	2.42.1

Dependency Analysis

Please note that this analysis is highly experimental. The aim here is to make maintenance easier by inspecting the package's dependencies. Importantly this analysis does not support optional dependencies, please double check those before making changes. If you do not want hinting of this kind ever please add bot: inspection: false to your conda-forge.yml. If you encounter issues with this feature please ping the bot team conda-forge/bot.

Analysis by grayskull shows a discrepancy between it and the the package's stated requirements in the meta.yaml.

Packages found by grayskull but not in the meta.yaml:

• accelerator
• pydantic
• ninja
• pynvml

Packages found in the meta.yaml but not found by grayskull:

• git
• pytorch ==cuda
• py-cpuinfo
• libaio
• pydantic <2.0.0

_{This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/6751503504, please use this URL for debugging.}

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[weiji14]

Getting this build error at https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=817391&view=logs&j=4f922444-fdfe-5dcf-b824-02f86439ef14&t=937c195f-508d-5135-dc9f-d4b5730df0f7&l=266:

Processing $SRC_DIR
  Added file://$SRC_DIR to build tracker '/tmp/pip-build-tracker-lw4wcbxq'
  Running setup.py (path:$SRC_DIR/setup.py) egg_info for package from file://$SRC_DIR
  Created temporary directory: /tmp/pip-pip-egg-info-76wqtttu
  Preparing metadata (setup.py): started
  Running command python setup.py egg_info
  No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
  [WARNING] Torch did not find cuda available, if cross-compiling or running with cpu only you can ignore this message. Adding compute capability for Pascal, Volta, and Turing (compute capabilities 6.0, 6.1, 6.2)
  DS_BUILD_OPS=1
  �[93m [WARNING] �[0m Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
  �[93m [WARNING] �[0m Filtered compute capabilities ['6.0', '6.1', '7.0', '7.5']
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/conda/feedstock_root/build_artifacts/deepspeed_1699059341031/work/setup.py", line 192, in <module>
      ext_modules.append(builder.builder())
                         ^^^^^^^^^^^^^^^^^
    File "/home/conda/feedstock_root/build_artifacts/deepspeed_1699059341031/work/op_builder/builder.py", line 632, in builder
      extra_link_args=self.strip_empty_entries(self.extra_ldflags()))
                                               ^^^^^^^^^^^^^^^^^^^^
    File "/home/conda/feedstock_root/build_artifacts/deepspeed_1699059341031/work/op_builder/inference_cutlass_builder.py", line 71, in extra_ldflags
      import dskernels
  ModuleNotFoundError: No module named 'dskernels'
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/conda/feedstock_root/build_artifacts/deepspeed_1699059341031/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/bin/python -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize
  
  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)
  
  __file__ = %r
  sys.argv[0] = __file__
  
  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"
  
  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/home/conda/feedstock_root/build_artifacts/deepspeed_1699059341031/work/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base /tmp/pip-pip-egg-info-76wqtttu
  cwd: /home/conda/feedstock_root/build_artifacts/deepspeed_1699059341031/work/
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
  Preparing metadata (setup.py): finished with status 'error'
Exception information:
Traceback (most recent call last):
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 64, in generate_metadata
    call_subprocess(
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
    raise error
pip._internal.exceptions.InstallationSubprocessError: python setup.py egg_info exited with 1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 76, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 513, in collect_root_requirements
    reqs = list(
           ^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 474, in _make_requirements_from_install_req
    cand = self._make_candidate_from_link(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 211, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
                                       ^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 293, in __init__
    super().__init__(
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
                ^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
    dist = self._prepare_distribution()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 304, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 525, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 640, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 71, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/distributions/sdist.py", line 67, in prepare_distribution_metadata
    self.req.prepare_metadata()
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/req/req_install.py", line 583, in prepare_metadata
    self.metadata_directory = generate_metadata_legacy(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$PREFIX/lib/python3.11/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 71, in generate_metadata
    raise MetadataGenerationFailed(package_details=details) from error
pip._internal.exceptions.MetadataGenerationFailed: metadata generation failed
Removed file://$SRC_DIR from build tracker '/tmp/pip-build-tracker-lw4wcbxq'
Removed build tracker: '/tmp/pip-build-tracker-lw4wcbxq'
Traceback (most recent call last):
  File "/opt/conda/bin/conda-mambabuild", line 10, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 256, in main
    call_conda_build(action, config)
  File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 228, in call_conda_build
    result = api.build(
  File "/opt/conda/lib/python3.10/site-packages/conda_build/api.py", line 180, in build
    return build_tree(
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 3078, in build_tree
    packages_from_this = build(metadata, stats,
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2198, in build
    utils.check_call_env(cmd, env=env, rewrite_stdout_env=rewrite_env,
  File "/opt/conda/lib/python3.10/site-packages/conda_build/utils.py", line 451, in check_call_env
    return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/conda_build/utils.py", line 427, in _func_defaulting_env_to_os_environ
    raise subprocess.CalledProcessError(proc.returncode, _args)
subprocess.CalledProcessError: Command '['/bin/bash', '-o', 'errexit', '/home/conda/feedstock_root/build_artifacts/deepspeed_1699059341031/work/conda_build.sh']' returned non-zero exit status 1.
##[error]Bash exited with code '1'.
##[section]Finishing: Run docker build

Not sure what this dskernels module is about at https://github.com/microsoft/DeepSpeed/blob/3437a5b1b6e6aa927add2720efceb5cc856694f9/op_builder/ragged_ops.py#L85C1-L85C1. It seems to have been added in microsoft/DeepSpeed#4604, might need to disable some more build flags...