Load existing LORA and continue training it

[Nero10578]

⚠️ Please check that this feature request hasn't been suggested before.

• I searched previous Ideas in Discussions didn't find any similar feature requests.
• I searched previous Issues didn't find any similar feature requests.

🔖 Feature description

As far as I understand specifying the LORA directory only means you're loading the LORA ontop of the base model and then training a new LORA on top of that?

✔️ Solution

Be able to load a pre-trained LORA and continue training it. Unless Axolotl already does this and I'm misunderstanding the docs?

❓ Alternatives

No response

📝 Additional Context

No response

Acknowledgements

• My issue title is concise, descriptive, and in title casing.
• I have searched the existing issues to make sure this feature has not been requested yet.
• I have provided enough information for the maintainers to understand and evaluate this request.

[NanoCode012]

Hey, I think if you do load, it may work as intended and continue training. I believe I may have done this quite a while back.

Please let us know if my understanding is correct!

[Nero10578]

Hey, I think if you do load, it may work as intended and continue training. I believe I may have done this quite a while back.

Please let us know if my understanding is correct!

Do you mean it should work by just specifying the lora directory?

[NanoCode012]

Yes, it should continue training from that adapter. Please do let me know if there's some issue.