faq_en

FAQ

Question 1: What's the difference between this project and the first-gen project?

Answer: The first generation project, Chinese-LLaMA-Alpaca, was developed based on the "not fully open-source" first-generation LLaMA, and the related model cannot be used for commercial purposes and there are many restrictions on distribution. This project is an upgrade based on the relevant technology of the first-gen project, developed based on the Llama-2 (commercial and distributable), and the model performance has a significant advantage compared to its first generation.

Question 2: Can the model be used commercially?

Answer: Yes, but please carefully read the commercial licensing requirements of the original Llama-2 in advance. Developers should be responsible for the compliance of using related models and seek legal support when necessary. This project is not responsible for the results and associated losses caused by using related models.

Question 3: Do you accept third-party Pull Requests?

Answer: We welcome Pull Requests. This project mainly accepts PRs in areas such as new tool adaptation, script enhancement, bug fixes, usage tutorials, etc. We temporarily do not accept PRs that do not affect the normal use of the model, such as only correcting one or two typos (we still appreciate your comments).

Question 4: Why not perform full pre-training but use LoRA instead?

Answer: Considering factors such as training cost and efficiency, we chose to use LoRA for training based on Llama-2 (embedding/lm_head are fully trained. See training wiki). We believe that Llama-2 already has a certain understanding of Chinese, and incremental training through LoRA can quickly supplement Chinese understanding and generation capabilities. As for whether full pre-training on Llama-2 will be better than LoRA, there is currently no reference conclusion available. Therefore, the use of LoRA in this project is the result of weighing various factors, not just considering the performance of the model.

Question 5: Does Llama-2 series support tools that support the first-gen LLaMA?

Answer: The tools we adapted in the first-gen project will gradually migrate to this project, but this process will take some time. At the same time, we strongly recommend keeping an eye on the adaptation progress of the corresponding third-party tools for Llama-2. The main differences between the first and second-generation models are: 1) The instruction template of our Alpaca-2 is different from the first generation; 2) The 34B/70B models need to support GQA (this project does not involve these two model types now); There are also some minor differences. In summary, those with strong hands-on ability can adapt themselves, or refer to the adaptation of third-party tools to Llama-2.

Question 6: Is Chinese-Alpaca-2 trained from Llama-2-Chat?

Answer: No. All of our models are based on Meta's Llama-2 (non-chat version). Specifically, Chinese-LLaMA-2 is trained on Llama-2 with large-scale Chinese text. Chinese-Alpaca-2 is further trained on Chinese-LLaMA-2 with selected instruction data to make it capable of chatting and following instructions. All models are trained using LoRA, which is a type of efficient training method.

Question 7: Why does training with 24GB VRAM lead to an OOM error when fine-tuning chinese-alpaca-2-7b?

Answer: You can troubleshoot the issue from the perspectives of dependent versions, data, and trainable parameters:

• The required peft version is 0.3.0.dev0, which can be installed using pip install git+https://github.com/huggingface/peft.git@13e53fc.
• Currently, in the script, max_seq_length or block_size is set to 1024. You can delete the existing data cache and modify the length to 512 for continued training.
• per_device_train_batch_size is set to 1; setting it higher will result in an OOM error.