Hyperparameter Optimization

This page is for those wishing to optimize the hyperparameters of the different example scripts in dp.

Deep Inception

Hyper-parameter optimization using the deepinception.lua script.

Svhn

Hyper-parameter optimization of the deepinception.lua script for training the Google Street View House Numbers (SVHN) dataset.

base command :

th examples/deepinception.lua --batchSize 128 --accUpdate --progress --cuda

The following table contains different inflections of the above command

Hyper-parameters	Epoch	Train Accuracy	Valid Accuracy	Test Accuracy
--batchSize 64 --learningRate 0.01 --activation HardTanh --hiddenSize '{4000,4000}' --lecunlcn --dropout	598	0.8467	0.8948	0.8897
--convChannelSize '{32}' --dropout --lecunlcn	64	0.8108	0.8649	0.8529
--convChannelSize '{32}' --lecunlcn	164	0.9999	0.8600	0.8473