New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Llama benchmark scripts for ROCm compatibility #18513

Draft

mindest wants to merge 6 commits into main from linmin/llama_rocm

Contributor

mindest commented Nov 20, 2023

Description

exclude some collective-related function definitions unavailable in ROCm
enable RoPE in ROCm
update Llama benchmark scripts.
- (workaround to be updated) on ROCm, --use-gqa should not be used
- (workaround to be updated) --disable_attn should be used for 70B, ROCm

mindest added 6 commits

November 20, 2023 11:14


          [ROCm] Enable RoPE; disable FuncMatMul related

d4d49c7


          limit hipBLASLt ws size at 32M; concat dtype

e665301


          ROCm related changes for benchmark scripts

afe3c9c


          update README.

29f16b1


          revert concat_impl related changes


          revert hipblaslt related.

3a9d59f

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet