BERT MULTI GPU

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

REQUIREMENT

python 3

tensorflow 1.12.0

0, edit the input and output file name in create_pretraining_data.py and run_pretraining_gpu.py

1, run create_pretraining_data.py

2, run run_pretraining_gpu.py

Edit n_gpus in run_pretraining_gpu.py

In sample_text.txt, sentence is end by \n, paragraph is splitted by empty line.

Name		Name	Last commit message	Last commit date
Latest commit History 17 Commits
tmp_data		tmp_data
.gitignore		.gitignore
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
__init__.py		__init__.py
bert_config.json		bert_config.json
create_pretraining_data.py		create_pretraining_data.py
extract_features.py		extract_features.py
modeling.py		modeling.py
modeling_test.py		modeling_test.py
multilingual.md		multilingual.md
optimization.py		optimization.py
optimization_gpu.py		optimization_gpu.py
optimization_test.py		optimization_test.py
requirements.txt		requirements.txt
run_classifier.py		run_classifier.py
run_pretraining.py		run_pretraining.py
run_pretraining_gpu.py		run_pretraining_gpu.py
run_squad.py		run_squad.py
sample_text.txt		sample_text.txt
tokenization.py		tokenization.py
tokenization_test.py		tokenization_test.py
vocab.py		vocab.py
vocab.txt		vocab.txt