Skip to content

Commit

Permalink
[CI/Build] Fix CPU CI online inference timeout (#10314)
Browse files Browse the repository at this point in the history
Signed-off-by: Isotr0py <[email protected]>
  • Loading branch information
Isotr0py authored Nov 14, 2024
1 parent 29f3ef2 commit 03025c0
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions .buildkite/run-cpu-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ function cpu_tests() {
docker exec cpu-test bash -c "
set -e
export VLLM_CPU_KVCACHE_SPACE=10
export VLLM_CPU_OMP_THREADS_BIND=$CORE_RANGE
export VLLM_CPU_OMP_THREADS_BIND=$1
python3 -m vllm.entrypoints.openai.api_server --model facebook/opt-125m --dtype half &
timeout 600 bash -c 'until curl localhost:8000/v1/models; do sleep 1; done' || exit 1
python3 benchmarks/benchmark_serving.py \
Expand All @@ -75,4 +75,4 @@ function cpu_tests() {

# All of CPU tests are expected to be finished less than 25 mins.
export -f cpu_tests
timeout 25m bash -c "cpu_tests"
timeout 25m bash -c "cpu_tests $CORE_RANGE"

0 comments on commit 03025c0

Please sign in to comment.