Constraint Back-translation Improves Complex Instruction Following of Large Language Models

Large language models (LLMs) struggle to follow instructions with complex constraints in format, length, etc. Following the conventional instruction-tuning practice, previous works conduct post-training on complex instruction-response pairs generated by feeding complex instructions to advanced LLMs. However, even advanced LLMs cannot follow complex instructions well, thus limiting the quality of generated data. In this work, we find that existing datasets inherently contain implicit complex constraints and propose a novel data generation technique, constraint back-translation. Specifically, we take the high-quality instruction-response pairs in existing datasets and only adopt advanced LLMs to add complex constraints already met by the responses to the instructions, which naturally reduces costs and data noise. In the experiments, we adopt Llama3-70B-Instruct to back-translate constraints and create a high-quality complex instruction-response dataset, named CRAB. We present that post-training on CRAB improves multiple backbone LLMs' complex instruction-following ability, evaluated on extensive instruction-following benchmarks. We further find that constraint back-translation also serves as a useful auxiliary training objective in post-training.

• 📖 Paper: Constraint Back-translation Improves Complex Instruction Following of Large Language Models
• 🦀 CRAB in the 🤗HuggingFace Hub: Datasets and Models

Crab: SFT Dataset Construction

🌟You can directly download the CRAB dataset for SFT from HF repo.

We also provide a pipeline for constructing high-quality complex instruction-following datasets, enabling you to generate your own customized Crab dataset using the following steps.

1. Prepare the seed data.

To prepare your environment for data processing, download the following datasets from the provided links and place them into the specified directory structure.

• Open-Assistant Dataset : OpenAssistant on HF
• WizardLM Dataset : WizardLM on HF
• Alpaca_GPT4 Dataset : Alpaca GPT-4 on Github
• Orca-Chat Dataset : Orca-Chat on HF
• ShareGPT Dataset : ShareGPT on HF, Note: Follow the repository’s instructions to generate a 53k training dataset.

Once downloaded, ensure that each dataset is organized into the designated directory structure:

data/
└── raw_data/
    ├── oasst2/
    ├── WizardLM/
    ├── GPT-4-LLM-main/
    ├── orca-Chat/
    └── ShareGPT_Vicuna_unfiltered

2. Data Generation

Execute the scripts in the scripts directory sequentially to generate the final dataset:

• Crab_0_Data_Collection.sh: Collects high-quality seed data.
• Crab_1_Constraint_Back_Translation.sh: Generates constraints through back-translation.
• Crab_2_Constraint_Combination.sh: Combines constraints to produce the final training data.

Training

For model training, we utilize different frameworks depending on the model architecture:

• Mistral-7B based models: Use the alignment-handbook for training guidance.
• LLaMA-3-8B based models: Follow the open-instruct repository for specific instructions.

Refer to their respective GitHub repositories for further training instructions.

Evaluation

The following evaluation benchmarks were used in our paper to measure model performance:

• IFEval
• FollowBench

Performance

Citation

@misc{qi2024constraintbacktranslationimprovescomplex,
      title={Constraint Back-translation Improves Complex Instruction Following of Large Language Models}, 
      author={Yunjia Qi and Hao Peng and Xiaozhi Wang and Bin Xu and Lei Hou and Juanzi Li},
      year={2024},
      eprint={2410.24175},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2410.24175}, 
}