Code

Followup with some fixes#77

Code

Draft

oandreeva-nv wants to merge 7 commits intomaintriton-inference-server/vllm_backend:mainfrom oandreeva_metrics_refactortriton-inference-server/vllm_backend:oandreeva_metrics_refactorCopy head branch name to clipboard

+355-382

Commits

Commits on Dec 5, 2024

• Remove vLLM 0.6.x version checks

  kthui

  committed
  8b17341

Commits on Dec 6, 2024

• Use ZMQ and some refactoring

  Show description for f9a31fc
  kthui

  committed
  f9a31fc

• Skip ZMQ process if metrics are enabled

  Show description for 050380b
  kthui

  committed
  050380b

• Update L0_check_health_vllm engine failure mock

  kthui

  committed
  96cece2

Commits on Dec 7, 2024

• Move metrics enable check to initialize()

  kthui

  committed
  94b80cd

• Fix engine args dependency issue

  kthui

  committed
  cd4cf06

Commits on Dec 20, 2024

• replasing asyncio event with threading event

  oandreeva-nv

  committed
  0e6f5ef