Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

Commit

Permalink
update hash (#332)
Browse files Browse the repository at this point in the history
  • Loading branch information
eric-haibin-lin authored and szha committed Sep 11, 2018
1 parent b76bd0f commit a946474
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 7 deletions.
2 changes: 1 addition & 1 deletion gluonnlp/model/language_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,7 @@ def standard_lstm_lm_1500(dataset_name=None, vocab=None, pretrained=False, ctx=c
('700b532dc96a29e39f45cb7dd632ce44e377a752', 'standard_lstm_lm_200_wikitext-2'),
('45d6df33f35715fb760ec8d18ed567016a897df7', 'awd_lstm_lm_1150_wikitext-2'),
('7894a046f8286db0d5d2ed672b60f4f52b4bc3aa', 'awd_lstm_lm_600_wikitext-2'),
('92ed6f71c4b12f465b11d72e7271743d0c4c1a7a', 'big_rnn_lm_2048_512_gbw'),
('6bb3e991eb4439fabfe26c129da2fe15a324e918', 'big_rnn_lm_2048_512_gbw'),
]})

class BigRNN(Block):
Expand Down
10 changes: 5 additions & 5 deletions scripts/language_model/language_model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -179,18 +179,18 @@ The dataset used for training the models is Google's 1 billion words dataset.
+-----------------+------------------------------------------------------------------------------------------------------------------------------+
| Graident clip | 10.0 |
+-----------------+------------------------------------------------------------------------------------------------------------------------------+
| Test perplexity | 43.72 |
| Test perplexity | 43.62 |
+-----------------+------------------------------------------------------------------------------------------------------------------------------+
| Num epochs | 42 |
| Num epochs | 50 |
+-----------------+------------------------------------------------------------------------------------------------------------------------------+
| Training logs | `log <https://github.com/dmlc/web-data/blob/master/gluonnlp/logs/language_model/big_rnn_lm_2048_512_gbw.log>`__ |
+-----------------+------------------------------------------------------------------------------------------------------------------------------+
| Evaluation logs | `log <https://github.com/dmlc/web-data/blob/master/gluonnlp/logs/language_model/big_rnn_lm_2048_512_gbw-eval.log>`__ |
+-----------------+------------------------------------------------------------------------------------------------------------------------------+

[1] LSTM-2048-512 (Test PPL 43.72, based on `commit <https://github.com/eric-haibin-lin/gluon-nlp/tree/b3cfcc427fbceea2ce3c5fae63c61b93910326dc>`__)
[1] LSTM-2048-512 (Test PPL 43.62)

.. code-block:: console
$ python large_word_language_model.py --gpus 0,1,2,3 --epochs=42 --clip=10
$ python large_word_language_model.py --gpus 4 --eval-only --batch-size=1 --log-interval=999999
$ python large_word_language_model.py --gpus 0,1,2,3 --clip=10
$ python large_word_language_model.py --gpus 4 --eval-only --batch-size=1
2 changes: 1 addition & 1 deletion scripts/language_model/large_word_language_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@
help='list of gpus to run, e.g. 0 or 0,2,5. empty means using cpu.')
parser.add_argument('--log-interval', type=int, default=1000,
help='report interval')
parser.add_argument('--seed', type=int, default=1,
parser.add_argument('--seed', type=int, default=0,
help='random seed')
parser.add_argument('--lr', type=float, default=0.2,
help='initial learning rate')
Expand Down

0 comments on commit a946474

Please sign in to comment.