Skip to content

Commit

Permalink
bump flashinfer to 0.1.5 (ModelCloud#376)
Browse files Browse the repository at this point in the history
* flashinfer 0.1.5

* Update unit_tests.yml

* Update unit_tests.yml
  • Loading branch information
CSY-ModelCloud authored Aug 17, 2024
1 parent 01bcf9e commit 182df2b
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 3 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/unit_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -302,7 +302,7 @@ jobs:
- name: Download wheel
continue-on-error: true
run: |
file_name=$(curl -s -F "runid=${{ github.run_id }}" -F "repo=${{ github.repository }}" -F "ref=${{ github.ref }}" -F "fuzz=1" "http://${{ needs.check-vm.outputs.ip }}/gpu/whl/download")
file_name=$(curl -s -F "runid=${{ needs.check-vm.outputs.run_id }}" -F "repo=${{ github.repository }}" -F "ref=${{ github.ref }}" -F "fuzz=1" "http://${{ needs.check-vm.outputs.ip }}/gpu/whl/download")
if echo "$file_name" | grep -q "gptqmodel"; then
mkdir dist || true
Expand Down Expand Up @@ -374,7 +374,7 @@ jobs:
- list-test-files
- check-vm
runs-on: self-hosted
if: always() && !cancelled() && (needs.build-intel.result == 'success' || needs.build-amd.result == 'success' || github.event.inputs.artifact_id != '')
if: always() && !cancelled() && (needs.build-intel.result == 'success' || needs.build-amd.result == 'success' || github.event.inputs.artifact_id != '') && needs.list-test-files.outputs.cpu-files != '[]'
timeout-minutes: 40
container:
image: ${{ needs.check-vm.outputs.ip }}:5000/modelcloud/gptqmodel:github-ci-v2
Expand Down
2 changes: 1 addition & 1 deletion tests/test_sglang.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ class TestLoadSglang(unittest.TestCase):
@classmethod
def setUpClass(self):
# sglang set disable_flashinfer=True still import flashinfer
subprocess.check_call([sys.executable, "-m", "pip", "install", "flashinfer==0.1.2", "-i", f"https://flashinfer.ai/whl/cu{torch.version.cuda.replace('.', '')}/torch{'.'.join(torch.__version__.split('.')[:2])}"])
subprocess.check_call([sys.executable, "-m", "pip", "install", "flashinfer==0.1.5", "-i", f"https://flashinfer.ai/whl/cu{torch.version.cuda.replace('.', '')}/torch{'.'.join(torch.__version__.split('.')[:2])}"])
subprocess.check_call([sys.executable, "-m", "pip", "install", "sglang[srt]>=0.2.7"])

self.MODEL_ID = "LnL-AI/TinyLlama-1.1B-Chat-v1.0-GPTQ-4bit"
Expand Down

0 comments on commit 182df2b

Please sign in to comment.