Error Code 9: API Usage Error (Target GPU SM 70 is not supported by this TensorRT release.)

[aliencaocao]

System Info

TensorRT-LLM version: 0.15.0.dev2024102900

Using Tesla V100 SXM2 16GB.
Following the official instructions and official wheel.
Building BLIP2-OPT failed with

[11/01/2024-03:33:08] [TRT] [E] IBuilder::buildSerializedNetwork: Error Code 9: API Usage Error (Target GPU SM 70 is not supported by this TensorRT release.)

Tried to build from source and specifying "70-real" using
python3 ./scripts/build_wheel.py --trt_root /home/ubuntu/TensorRT-LLM/TensorRT-10.5.0.18 --cuda_architectures "70-real;75-real"
Produced wheel is 900MB. Still same error.

The same wheel and command works on T4.

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Building BLIP2-OPT failed with official cmd shown in the example readme, with some of my own args.

path redacted
    --gemm_plugin float16 \
    --max_beam_width 5 \
    --max_batch_size 16 \
    --max_seq_len 100 \
    --max_input_len 48 \
    --context_fmha disable \
    --multiple_profiles disable \
    --max_multimodal_len 512 \
    --opt_num_tokens 576 \
    --profiling_verbosity detailed \
    --workers 8 \
    --log_level verbose

Expected behavior

Works on V100

actual behavior

Does not work

additional notes

NIL

[aliencaocao]

Wow ok...then anyway to get the old version's wheel? or Which is the commit right before removal of SM70 so I can build from src?

[nv-guomingz]

Wow ok...then anyway to get the old version's wheel? or Which is the commit right before removal of SM70 so I can build from src?

You may try this commit f14d1d4

[aliencaocao]

You mean https://github.com/NVIDIA/TensorRT-LLM/tree/3c46c2794e7f6df48250a68de6240994a77a26a7? I see that most of the code changes are after this

[aliencaocao]

Another related qns would be, is it possible to build the previous commit with TensorRT 10.5, instead of 10.4?

[nv-guomingz]

You mean https://github.com/NVIDIA/TensorRT-LLM/tree/3c46c2794e7f6df48250a68de6240994a77a26a7? I see that most of the code changes are after this

Yes, we release the code in weekly bias, so there're lots of changes.

[nv-guomingz]

Another related qns would be, is it possible to build the previous commit with TensorRT 10.5, instead of 10.4?

I don't have a try on that but I wouldn't recommend you have a try since it may raise unknown issues.

[aliencaocao]

Thank you.

[gyd-a]

I had the same problem.