Question on how to set dataset size for training

[RBehr]

Dear Mr. Zbontar,

after having gone through the main.lua I can't seem to find the argument which to pass in order so set the size for the dataset on which to train the network. I do know that with "./main.lua kitti fast -a train_tr" I train the network on a subset of the kitti dataset but I would like to do a test just as stated in table 9, pg. 26 of your paper with say just 20% of the entire picture set.
Any help is much appreciated.

Regards,
Raphael.

[jzbontar]

You can use the -subset option.

https://github.com/jzbontar/mc-cnn/blob/master/main.lua#L621-L647

[RBehr]

so the command would be something like:
'./main.lua kitti2015 slow -a train_tr -subset -200'? for 50% of the kitti2015 dataset since the set itself contains 400 images, correct?
Thank you very much.

[jzbontar]

I think it's more like this: ./main.lua kitti2015 slow -a train_tr -subset 0.5

[RBehr]

thank you very much!

[RBehr]

Hallo once again Mr Zbontar, After successfully running through 14 epochs of training using the -subset option, the shell returns the following error: THCudaCheck FAIL file=/tmp/luarocks_cutorch-scm-1-6134/cutorch/lib/THC/generic/THCStorage.cu line=66 error=2 : out of memory luajit: ...torch/install/share/lua/5.1/cudnn/SpatialConvolution.lua:144: cuda runtime error (2) : out of memory at /tmp/luarocks_cutorch-scm-1-6134/cutorch/lib/THC/generic/THCStorage.cu:66 stack traceback: [C]: in function 'resize' ...torch/install/share/lua/5.1/cudnn/SpatialConvolution.lua:144: in function 'createIODescriptors' ...torch/install/share/lua/5.1/cudnn/SpatialConvolution.lua:188: in function 'updateOutput' ./main.lua:911: in function 'forward_free' ./main.lua:945: in function 'stereo_predict' ./main.lua:1198: in main chunk [C]: at 0x004064f0 Do you have any suggestions on how to solve this issue? I am training the net on the following GPU: +-----------------------------------------------------------------------------+ | NVIDIA-SMI 375.66 Driver Version: 375.66 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 Tesla K40c Off | 0000:04:00.0 Off | 0 | | 29% 53C P0 69W / 235W | 10939MiB / 11439MiB | 0% Default | +-------------------------------+----------------------+----------------------+ Best regards, Raphael. Von: Jure Zbontar [mailto:[email protected]] Gesendet: Dienstag, 11. Juli 2017 15:37 An: jzbontar/mc-cnn <[email protected]> Cc: Behrendt, Raphael Eric Alfred <[email protected]>; Author <[email protected]> Betreff: Re: [jzbontar/mc-cnn] Question on how to set dataset size for training (#33) I think it's more like this: ./main.lua kitti2015 slow -a train_tr -subset 0.5 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#33 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/Acc2ysXtx5XA3uHsqlDJqdgbpQKeOODjks5sM3pjgaJpZM4OLh2k>.