Didn't find wheel for tensorrt-llm

[jasonngap1]

System Info

• CPU architecture: x86_84
• GPU: A100
• Python version: 3.10.13 on a virtual env

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

1. pip install -U --pre --extra-index-url https://pypi.nvidia.com/ tensorrt-llm

Expected behavior

TensorRT-LLM installed

actual behavior

Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com
Collecting tensorrt_llm
  Using cached tensorrt_llm-0.12.0.tar.gz (1.1 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [53 lines of output]
      INFO:nvidia-stub:Testing wheel tensorrt_llm-0.12.0-cp310-cp310-linux_x86_64.whl against tag cp310-cp310-linux_x86_64
      INFO:nvidia-stub:Testing wheel tensorrt_llm-0.12.0-cp310-cp310-win_amd64.whl against tag cp310-cp310-win_amd64
        File "/tmp/pip-build-env-81oreuoz/overlay/lib/python3.9/site-packages/nvidia_stub/wheel.py", line 177, in download_wheel
          return download_manual(wheel_directory, distribution, version)
        File "/tmp/pip-build-env-81oreuoz/overlay/lib/python3.9/site-packages/nvidia_stub/wheel.py", line 144, in download_manual
          raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
      Traceback (most recent call last):
        File "/tmp/pip-build-env-81oreuoz/overlay/lib/python3.9/site-packages/nvidia_stub/wheel.py", line 177, in download_wheel
          return download_manual(wheel_directory, distribution, version)
        File "/tmp/pip-build-env-81oreuoz/overlay/lib/python3.9/site-packages/nvidia_stub/wheel.py", line 144, in download_manual
          raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
      RuntimeError: Didn't find wheel for tensorrt-llm 0.12.0
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/anaconda/envs/azureml_py38/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/anaconda/envs/azureml_py38/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/anaconda/envs/azureml_py38/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
        File "/tmp/pip-build-env-81oreuoz/overlay/lib/python3.9/site-packages/nvidia_stub/buildapi.py", line 29, in build_wheel
          return download_wheel(pathlib.Path(wheel_directory), config_settings)
        File "/tmp/pip-build-env-81oreuoz/overlay/lib/python3.9/site-packages/nvidia_stub/wheel.py", line 179, in download_wheel
          report_install_failure(distribution, version, exception_context)
        File "/tmp/pip-build-env-81oreuoz/overlay/lib/python3.9/site-packages/nvidia_stub/error.py", line 63, in report_install_failure
          raise InstallFailedError(
      nvidia_stub.error.InstallFailedError:
      *******************************************************************************
      
      The installation of tensorrt-llm for version 0.12.0 failed.
      
      This is a special placeholder package which downloads a real wheel package
      from https://pypi.nvidia.com. If https://pypi.nvidia.com is not reachable, we
      cannot download the real wheel file to install.
      
      You might try installing this package via
      ```
      $ pip install --extra-index-url https://pypi.nvidia.com tensorrt-llm
      ```
      
      Here is some debug information about your platform to include in any bug
      report:
      
      Python Version: CPython 3.9.19
      Operating System: Linux 5.15.0-1064-azure
      CPU Architecture: x86_64
      Driver Version: 535.171
      CUDA Version: 12.2
      
      *******************************************************************************
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

additional notes

I have tried installing other version of tensorrt-llm but it does not work

[lfr-0531]

Just checked locally, it can work now. You can refer to https://nvidia.github.io/TensorRT-LLM/installation/linux.html#installing-on-linux to have a try again.

[jasonngap1]

Tried again and it works. Thanks!

[arivero]

The documentation should insist that pipy wheels are only built for python 3.10