[Badcase]: Fine tuning Qwen2-72b-Instruct using QLoRA with bnb, model indicators grad_norm=Nan; loss=0

[Jackmoyu001]

Model Series

Qwen2

What are the models used?

Qwen2-72B-Instruct

What is the scenario where the problem happened?

微调模型时，模型指标异常。使用了transformers、peft,

Is this badcase known and can it be solved using avaiable techniques?

• I have followed the GitHub README.
• I have checked the Qwen documentation and cannot find a solution there.
• I have checked the documentation of the related framework and cannot find useful information.
• I have searched the issues and there is not a similar one.

Information about environment

操作系统： Ubuntu 22.04
Python：Python 3.10
GPU：7 个 NVIDIA 3090 24G
NVIDIA 驱动程序：535
CUDA 编译器：12.2
PyTorch：2.1.2+cu121

Description

Steps to reproduce

This happens to Qwen2-72B-Instruct .
The badcase can be reproduced with the following steps:

1. My code and parameters are as follows

The executed instruction is python xxx. py

####The problem that has arisen is

Attempts to fix

I have tried several ways to fix this, including:

1. adjusting the sampling parameters, but not resolved

Anything else helpful for investigation

[errfgod]

same issue with qwen2.5_7B_instruct