Finetuned Phi3 built with the model builder give gibberish ouput due to lack of support for LoRA

[Pathos0925]

Has the model builder been proven to work with fine-tuned Phi3 models as suggested in the link in the HF card?
https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/resolve/main/sample_finetune.py

If it has then I am probably doing something wrong in the finetuning.

[natke]

Hi @Pathos0925, can you please share the model builder command and they command you are running and any errors that you get?

[kunal-vaishnavi]

The sample finetuning script appears to be using LoRA, which changes how the resulting ONNX model looks and runs. LoRA is not currently supported in the model builder but support for LoRA will be coming soon. A preview of how the APIs will look for running models with LoRA/QLoRA can be found here.

[Pathos0925]

The sample finetuning script appears to be using LoRA, which changes how the resulting ONNX model looks and runs. LoRA is not currently supported in the model builder but support for LoRA will be coming soon. A preview of how the APIs will look for running models with LoRA/QLoRA can be found here.

Alright, thank you!

[natke]

Closing this issue for now. Please check back for LoRA support