Add qlora to our current codebase

[samsja]

Context

we want to add qlora (lora + 4 bits int quant) to our codebase.

The goal is to reduce memory usage and the cost of finetuning without degrading quality.

paper
blog post

Steps

• activate 4bits in Peft. Should be as easy as turning on a flag
• run the modal on one GPU for one epoch and look at memory consumption compared to 8 bits training
• run a full training (3 epochs) with the same parameters as 8bits training and compare result to see if we don't degrade quality

[alaeddine-13]

you can get inspired by this PR: tloen/alpaca-lora#487