Skip to content

Latest commit

 

History

History
33 lines (23 loc) · 1.3 KB

README.md

File metadata and controls

33 lines (23 loc) · 1.3 KB

multi_task-nlp-bert

This is a repo for NLP multi-task learning, which includes single-sentence classification, pairwise text similarity, pairwise text classification, and relevance ranking.

Downloading Datesets

Create a directory data/.

mkdir data

Download STS-B.zip to data/STS-B.zip.

Download QNLIv2.zip to data/QNLIv2.zip.

All the three datasets will be unzipped automatically so don't unzip these three files.

SNLI dataset will be downloaded automatically.

Experiments

This is a multi-task learning so data would take a lot of memory, so we should use a small batch size.

Train with max_grad_norm 1.0 with multi-task learning.

python -u main.py --batch_size=32 --gpu=0,1 --grad_max_norm=1 --multi_task
python -u main.py --batch_size=16 --gpu=0 --grad_max_norm=1 --multi_task

Train without multi-task learning. Now we have less datasets to train so we could increase the batch size.

python -u main.py --batch_size=64 --gpu=0,1
python -u main.py --batch_size=32 --gpu=0