langchain-nvidia-trt installation

[tapansstardog]

I have setup.py. When I try to install "langchain-nvidia-trt == 0.0.1rc0" through setup.py, I get the below error.

The conflict is caused by:
    tritonclient[all] 2.42.0 depends on cuda-python; extra == "all"
    tritonclient[all] 2.44.0 depends on cuda-python; extra == "all"
    tritonclient[all] 2.43.0 depends on cuda-python; extra == "all"
    tritonclient[all] 2.41.1 depends on cuda-python; extra == "all"
    tritonclient[all] 2.41.0 depends on cuda-python; extra == "all"
    tritonclient[all] 2.40.0 depends on cuda-python; extra == "all"

I am able to run:

pip install --no-deps langchain-nvidia-trt==0.0.1rc0 separately outside setup.py followed by pip install tritonclient[grpc]. And then I can connect to Triton server. But I am not able to do so within setup.py.

[mattf]

@tapansstardog will you share your setup.py or a minimal reproducer?

➜ python --version
Python 3.12.2

➜ pip install langchain-nvidia-trt==0.0.1rc0
Collecting langchain-nvidia-trt==0.0.1rc0
...

➜ pip freeze | grep -e triton -e cuda -e langchain
cuda-python==12.4.0
langchain-core==0.1.35
langchain-nvidia-trt==0.0.1rc0
tritonclient==2.44.0

[tapansstardog]

@mattf Here is the code:

from setuptools import find_packages, setup

setup(
    name="core",
    version="1.0.2",
    description="desc",
    author="author",
    url="https://github.com/",
    package_dir={"": "src"},
    packages=find_packages(where="src"),
    install_requires=[
        "langchain-nvidia-trt == 0.0.1rc0",
        "tritonclient[grpc] == 2.42.0",
    ],
    python_requires=">=3.10",
)

[mattf]

@tapansstardog i'm not able to reproduce, even when i specify 2.42.0 for tritonclient. it could be there's something else in your environment that is conflicting. is your environment fresh before you run the command? what is the command & full output where you have this error?

[tapansstardog]

Thanks, let me look into the environment.
Just wanted to confirm. Did you install it using setup.py and pip install . command?

[mattf]

yes but with a fresh environment.

➜ tree
.
├── setup.py
└── src

2 directories, 1 file


➜ cat setup.py
from setuptools import find_packages, setup

setup(
    name="core",
    version="1.0.2",
    description="desc",
    author="author",
    url="https://github.com/",
    package_dir={"": "src"},
    packages=find_packages(where="src"),
    install_requires=[
        "langchain-nvidia-trt == 0.0.1rc0",
        "tritonclient[grpc] == 2.42.0",
    ],
    python_requires=">=3.10",
)


➜ virtualenv -p 3.10 .venv
created virtual environment CPython3.10.13.final.0-64 in 115ms
  creator CPython3Posix(dest=/tmp/issue-11/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/matt/.local/share/virtualenv)
    added seed packages: pip==24.0, setuptools==69.1.1, wheel==0.42.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator


➜ source .venv/bin/activate


➜ pip install .            
Processing /tmp/issue-11
  Preparing metadata (setup.py) ... done
...
Successfully built core
Installing collected packages: cuda-python, brotli, zope.interface, zope.event, urllib3, typing-extensions, types-protobuf, tomlkit, tomli, tenacity, smmap, six, PyYAML, python-rapidjson, protobuf, platformdirs, packaging, orjson, numpy, multidict, mccabe, jsonpointer, isort, idna, grpcio, greenlet, frozenlist, dill, charset-normalizer, certifi, attrs, async-timeout, annotated-types, yarl, tritonclient, requests, pydantic-core, jsonpatch, gitdb, gevent, astroid, aiosignal, pylint, pydantic, gitpython, geventhttpclient, aiohttp, lint, langsmith, langchain-core, langchain-nvidia-trt, core
Successfully installed PyYAML-6.0.1 aiohttp-3.9.3 aiosignal-1.3.1 annotated-types-0.6.0 astroid-3.1.0 async-timeout-4.0.3 attrs-23.2.0 brotli-1.1.0 certifi-2024.2.2 charset-normalizer-3.3.2 core-1.0.2 cuda-python-12.4.0 dill-0.3.8 frozenlist-1.4.1 gevent-24.2.1 geventhttpclient-2.0.2 gitdb-4.0.11 gitpython-3.1.42 greenlet-3.0.3 grpcio-1.62.1 idna-3.6 isort-5.13.2 jsonpatch-1.33 jsonpointer-2.4 langchain-core-0.1.35 langchain-nvidia-trt-0.0.1rc0 langsmith-0.1.36 lint-1.2.1 mccabe-0.7.0 multidict-6.0.5 numpy-1.26.4 orjson-3.10.0 packaging-23.2 platformdirs-4.2.0 protobuf-3.20.3 pydantic-2.6.4 pydantic-core-2.16.3 pylint-3.1.0 python-rapidjson-1.16 requests-2.31.0 six-1.16.0 smmap-5.0.1 tenacity-8.2.3 tomli-2.0.1 tomlkit-0.12.4 tritonclient-2.42.0 types-protobuf-4.24.0.20240311 typing-extensions-4.10.0 urllib3-2.2.1 yarl-1.9.4 zope.event-5.0 zope.interface-6.2


➜ tree
.
├── build
│   └── bdist.linux-x86_64
├── setup.py
└── src
    └── core.egg-info
        ├── dependency_links.txt
        ├── PKG-INFO
        ├── requires.txt
        ├── SOURCES.txt
        └── top_level.txt

5 directories, 6 files


➜ pip freeze | grep -e triton -e cuda -e langchain
cuda-python==12.4.0
langchain-core==0.1.35
langchain-nvidia-trt==0.0.1rc0
tritonclient==2.42.0

[tapansstardog]

Thanks @mattf . Something is not right on my environment. I tried the same on other machine and didn't see any issue. Need to look into it. I am closing the ticket. Thanks.

[tapansstardog]

Hey @mattf . An update from my end.
I was able to run setup.py on ubuntu 22.04 boxes but facing the concerned issue on MacOS. Wanted to check with you the OS that you used to reproduce the issue. If not MacOS, can you reproduce on that?

[mattf]

@tapansstardog i used Fedora 39. i don't have a MacOS system available to me.