trtllm-bench faild

[dingjingzhen]

System Info

tensorrt-llm 0.15.0

Who can help?

@kaiyux

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

python cpp/prepare_dataset.py --stdout --tokenizer mypath/Meta-Llama-3-8B-Instruct token-norm-dist --input-mean 500 --output-mean 128 --input-stdev 0 --output-stdev 0 --num-requests 3000 > ./synthetic_500_128.txt
trtllm-bench --model mypath/Meta-Llama-3-8B-Instruct throughput --dataset ./synthetic_500_128.txt --engine_dir mypath/engine_outputs_fp16_open_wo

Expected behavior

Performance result

actual behavior

Traceback (most recent call last):
File "/home/admin/miniconda3/envs/compare_quant/bin/trtllm-bench", line 8, in
sys.exit(main())
File "/home/admin/miniconda3/envs/compare_quant/lib/python3.10/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/home/admin/miniconda3/envs/compare_quant/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/admin/miniconda3/envs/compare_quant/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/admin/miniconda3/envs/compare_quant/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/admin/miniconda3/envs/compare_quant/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/admin/miniconda3/envs/compare_quant/lib/python3.10/site-packages/click/decorators.py", line 45, in new_func
return f(get_current_context().obj, *args, **kwargs)
File "/home/admin/miniconda3/envs/compare_quant/lib/python3.10/site-packages/tensorrt_llm/bench/benchmark/throughput.py", line 165, in throughput_command
metadata, requests = create_dataset_from_stream(
File "/home/admin/miniconda3/envs/compare_quant/lib/python3.10/site-packages/tensorrt_llm/bench/utils/data.py", line 91, in create_dataset_from_stream
logits = tokenize(prompt)["input_ids"] if logits is None else logits
File "/home/admin/miniconda3/envs/compare_quant/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2796, in call
raise ValueError("You need to specify either text or text_target.")
ValueError: You need to specify either text or text_target.

additional notes

no