[Bug] The quantization process of Qwen/Qwen2-VL-7B-Instruct is getting killed without throwing error.

[vjaideep08]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

I am trying to AWQ 4-bit quantize Qwen/Qwen2-VL-7B-Instruct using LM DEPLOY awq quantization algorithm.

Reproduction

export HF_MODEL=Qwen/Qwen2-VL-7B-Instruct

export WORK_DIR = /Qwen/Qwen2-VL-7B-Instruct-4BIT-MODEL

lmdeploy lite auto_awq $HF_MODEL --calib-dataset 'ptb' --calib-samples 4 --calib-seqlen 64 --w-bits 4 --w-group-size 128 --batch-size 1 --work-dir $WORK_DIR

Environment

conda create -n lmdeploy python=3.8 -y
conda activate lmdeploy
pip install lmdeploy

Error traceback

No response

[AllentDan]

Qwen2-VL AWQ quantization is not supported yet. I will try to add the feature.

[AllentDan]

#2787 支持了，可以先用起来