Code

Add OpenAI compatible v2 completion#1

Code

Open

lizard-boy wants to merge 10 commits intomaingrepdemos/llm-engine:mainfrom dmchoi/openai-completionsgrepdemos/llm-engine:dmchoi/openai-completionsCopy head branch name to clipboard

+907-160

Commits

Commits on Sep 30, 2024

• Add OpenAI compatible v2 completion

  dmchoiboi

  committed
  7b9b7ee

  Copy full SHA for 7b9b7ee

• Fix batch completion response

  dmchoiboi

  committed
  8418acf

  Copy full SHA for 8418acf

Commits on Oct 1, 2024

• fix test coverage

  dmchoiboi

  committed
  297561a

  Copy full SHA for 297561a

• Get model_name from v2 requests correctly

  dmchoiboi

  committed
  0be1647

  Copy full SHA for 0be1647

• Make post inference hook optional and add async version of forward

  dmchoiboi

  committed
  a15718d

  Copy full SHA for a15718d

• update predict method to be fully async

  dmchoiboi

  committed
  7f2471c

  Copy full SHA for 7f2471c

• woops forgot api route

  dmchoiboi

  committed
  e40bc61

  Copy full SHA for e40bc61

• Add async for stream forwarder

  dmchoiboi

  committed
  9e1efd0

  Copy full SHA for 9e1efd0

• Update to asyncgenerator

  dmchoiboi

  committed
  61b4884

  Copy full SHA for 61b4884

• Move aiohttp_sse_client to common to avoid unnecessary imports

  dmchoiboi

  committed
  636c0f3

  Copy full SHA for 636c0f3