Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vLLM deploy script and example client scripts #32

Merged
merged 9 commits into from
Nov 6, 2024
Merged

vLLM deploy script and example client scripts #32

merged 9 commits into from
Nov 6, 2024

Commits on Nov 5, 2024

  1. adding llama 3.1 70b benchmarking instructions

    @tstescoTT
    tstescoTT committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    24361b4 View commit details
    Browse the repository at this point in the history

  2. add doc link for vllm setup

    @tstescoTT
    tstescoTT committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    fac3d46 View commit details
    Browse the repository at this point in the history

  3. adding vllm-tt-metal-llama3-70b/src for vllm deployment run script an… 

    …d example clients
    @tstescoTT
    tstescoTT committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    aa5885a View commit details
    Browse the repository at this point in the history

  4. adding automatic default vvlm online server at startup as default CMD

    @tstescoTT
    tstescoTT committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    6c3ad48 View commit details
    Browse the repository at this point in the history

  5. update documentation with example client usage

    @tstescoTT
    tstescoTT committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    4a3c251 View commit details
    Browse the repository at this point in the history

  6. improve url handling in example clients

    @tstescoTT
    tstescoTT committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    83e7c5a View commit details
    Browse the repository at this point in the history

  7. adding docker build instructions

    @tstescoTT
    tstescoTT committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    3b45cd7 View commit details
    Browse the repository at this point in the history

  8. adding requirements for example clients scripts

    @tstescoTT
    tstescoTT committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    0bc9805 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2024

  1. Merge branch 'main' into tstesco/vllm-deploy-rebased

    @tstescoTT
    tstescoTT authored Nov 6, 2024
    Configuration menu
    Copy the full SHA
    6176bf7 View commit details
    Browse the repository at this point in the history