Skip to content

Commit

Permalink
[sharktank] Evaluation - Update timeout for Perplexity CI test (#316)
Browse files Browse the repository at this point in the history
Update default CI timeout from 10 to 20 hrs for the first run and adjust
later.
  • Loading branch information
archana-ramalingam authored Oct 25, 2024
1 parent af4951e commit 8822d80
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 2 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/ci_eval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ concurrency:

jobs:
test_perplexity:
timeout-minutes: 600
timeout-minutes: 1000
name: "Evaluation Tests - perplexity"
strategy:
matrix:
Expand Down Expand Up @@ -59,4 +59,4 @@ jobs:
pip install --no-compile -r requirements.txt -r sharktank/requirements-tests.txt -e sharktank/
- name: Run perplexity test
run: pytest sharktank/tests/evaluate/perplexity_test.py --longrun
run: pytest -n 4 -v -s sharktank/tests/evaluate/perplexity_test.py --longrun
1 change: 1 addition & 0 deletions sharktank/sharktank/evaluate/perplexity.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,7 @@ def get_logits(self):
start = 0
for i in tqdm(
range(start, self.max_prompt_length - 1),
mininterval=300,
desc="eval: Calculating logits",
):
logger.debug(f"Iteration: {i}")
Expand Down

0 comments on commit 8822d80

Please sign in to comment.