You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I got the following error while compiling
python train.py -src_data data/europarl-v7_de.txt -trg_data data/europarl-v7_en.txt -src_lang de -trg_lang en -SGDR -epochs 10 -checkpoint 10 -batchsize 128 -load_weights weights
loading spacy tokenizers...
loading presaved fields...
creating dataset and iterator...
The device argument should be set by using torch.device or passing a string as an argument. This behavior will be deprecated soon and currently defaults to cpu.
Traceback (most recent call last):
File "train.py", line 185, in
main()
File "train.py", line 97, in main
opt.train = create_dataset(opt, SRC, TRG)
File "Documents\transformers\Process.py", line 89, in create_dataset
opt.train_len = get_len(train_iter)
File "Documents\transformers\Process.py", line 95, in get_len
for i, b in enumerate(train):
File "envs\alexandria\lib\site-packages\torchtext\data\iterator.py", line 157, in iter
yield Batch(minibatch, self.dataset, self.device)
File "Anaconda3\envs\alexandria\lib\site-packages\torchtext\data\batch.py", line 34, in init
setattr(self, name, field.process(batch, device=device))
File "Anaconda3\envs\alexandria\lib\site-packages\torchtext\data\field.py", line 201, in process
tensor = self.numericalize(padded, device=device)
File "Anaconda3\envs\alexandria\lib\site-packages\torchtext\data\field.py", line 323, in numericalize
var = torch.tensor(arr, dtype=self.dtype, device=device)
RuntimeError: sizes must be non-negative
I am not sure why this is occurring but I had changed my source and training parallel corpus to a larger europarl dataset is such transfer learning supported? If not how would i go about doing that.
EDIT 1: I have subsequently trained it a model from scratch with a batchsize of 128 ( I am running on a GTX960M) and encounter the same problem.
The text was updated successfully, but these errors were encountered:
I got the following error while compiling
python train.py -src_data data/europarl-v7_de.txt -trg_data data/europarl-v7_en.txt -src_lang de -trg_lang en -SGDR -epochs 10 -checkpoint 10 -batchsize 128 -load_weights weights
loading spacy tokenizers...
loading presaved fields...
creating dataset and iterator...
The
device
argument should be set by usingtorch.device
or passing a string as an argument. This behavior will be deprecated soon and currently defaults to cpu.Traceback (most recent call last):
File "train.py", line 185, in
main()
File "train.py", line 97, in main
opt.train = create_dataset(opt, SRC, TRG)
File "Documents\transformers\Process.py", line 89, in create_dataset
opt.train_len = get_len(train_iter)
File "Documents\transformers\Process.py", line 95, in get_len
for i, b in enumerate(train):
File "envs\alexandria\lib\site-packages\torchtext\data\iterator.py", line 157, in iter
yield Batch(minibatch, self.dataset, self.device)
File "Anaconda3\envs\alexandria\lib\site-packages\torchtext\data\batch.py", line 34, in init
setattr(self, name, field.process(batch, device=device))
File "Anaconda3\envs\alexandria\lib\site-packages\torchtext\data\field.py", line 201, in process
tensor = self.numericalize(padded, device=device)
File "Anaconda3\envs\alexandria\lib\site-packages\torchtext\data\field.py", line 323, in numericalize
var = torch.tensor(arr, dtype=self.dtype, device=device)
RuntimeError: sizes must be non-negative
I am not sure why this is occurring but I had changed my source and training parallel corpus to a larger europarl dataset is such transfer learning supported? If not how would i go about doing that.
EDIT 1: I have subsequently trained it a model from scratch with a batchsize of 128 ( I am running on a GTX960M) and encounter the same problem.
The text was updated successfully, but these errors were encountered: