Sampling code

[nyaong7]

Hi, thank you for providing results from such a valuable research.

I'm trying to build word prediction model using your RNN code.
I successfully finished installation and built the first model!! Thank you.
Now I would like to see how it actually predict next word.
But I am having a hard time to write codes for such task, since I am very new to this domain.
In the Kaparthy's package, there is a script for generating texts (sample.lua).
Could you provide such script for your solution?

Thank you very much in advance.

[LiyuanLucasLiu]

Hi, I'm a rookie on torch too. However, I tried to write a sample function for this model myself (unfortunately it's super slow), and you can find it on https://github.com/LiyuanLucasLiu/lstm-char-cnn

[ghost]

Hi, another rookie here. The previous version using Squeeze.lua works fine.
I would like to test your sample.lua as per karpathy but it doesn't work with the previous version.
Unfortunately, when I run the new version without Squeeze.lua I get an error which references 'Squeeze': #18

Any suggestions to fix are appreciated.

Also, sample-beam.lua increases the quality of karpathy output dramatically (imho)

https://github.com/pender/char-rnn/blob/master/sample-beam.lua

Would be cool to see this working for this model.

cheers

[ghost]

Updated torch and everything working. Generated a quick .t7 file with lowest possible settings and only 5 epochs to test sample.lua. Following " batch_size' failed. " error generated.

Is there a value I can set to prevent this or do I have to run "main.lua" with the values given?

thanks

aaron@aaron-Macmini ~/Desktop/lstm-char-cnn-NEXT $ th sample.lua
loading data files...
Word vocab size: 4040, Char vocab size: 71
reshaping tensors...
data load done. Number of batches in train: 14, val: 2, test: 1
number of parameters in the model: 6073985
/home/aaron/torch/install/bin/luajit: /home/aaron/torch/install/share/lua/5.1/nn/THNN.lua:109: Assertion `THIndexTensor_(size)(target, 0) == batch_size' failed. at /home/aaron/torch/extra/nn/lib/THNN/generic/ClassNLLCriterion.c:43
stack traceback:
[C]: in function 'v'
/home/aaron/torch/install/share/lua/5.1/nn/THNN.lua:109: in function 'ClassNLLCriterion_updateOutput'
...ron/torch/install/share/lua/5.1/nn/ClassNLLCriterion.lua:41: in function 'forward'
sample.lua:216: in function 'sample'
sample.lua:236: in main chunk
[C]: in function 'dofile'
...aron/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x00406670

[LiyuanLucasLiu]

Hi, I'm not sure what caused this error (as a rookie...), but it seems that you ignored several necessary options while running sample.lua (the default value might not fit your checkpoint's setting), such as '-model'.

[ghost]

Thanks for your reply.

Could you supply an example of the terminal commands to run sample.lua ?

The commands should be based on running main.lua and generating the necessary .t7 file from the supplied PTB files.

This is easy with the karpathy original but not so much with this lstm version.

Thanks

[LiyuanLucasLiu]

Hi, I spent some time to train a new model and try sample.lua, I find that:

$ th main.lua -savefile char-large -EOS '+'
$ th sample.lua -model cv/lm_char-large_epoch25.00_81.97.t7 -gpuid 0 -cudnn 1 -length 20 -savefile tmpresult.t7 -temperature 0.5`

works fine with me.

hope it could help.