Code

Multinode batch inference#2

Code

Open

lizard-boy wants to merge 6 commits intomaingrepdemos/llm-engine:mainfrom seanshi/20240807-multinode-batch-inferencegrepdemos/llm-engine:seanshi/20240807-multinode-batch-inferenceCopy head branch name to clipboard

+94-4

Commits

Commits on Aug 7, 2024

• initial init_ray.py

  seanshi-scale

  committed
  00cc2a9

  Copy full SHA for 00cc2a9

• some notes

  seanshi-scale

  committed
  3e27a8a

  Copy full SHA for 3e27a8a

• use the env vars

  seanshi-scale

  committed
  e4ee24e

  Copy full SHA for e4ee24e

• add to docker image

  seanshi-scale

  committed
  be4dc00

  Copy full SHA for be4dc00

Commits on Aug 8, 2024

• notes

  seanshi-scale

  committed
  b3ba531

  Copy full SHA for b3ba531

• so I can test and not push to the actual repo

  seanshi-scale

  committed
  17439fb

  Copy full SHA for 17439fb