Best CIFAR-10, CIFAR-100 results with wide-residual networks using PyTorch

Pytorch Implementation of Sergey Zagoruyko's Wide Residual Networks

For Torch implementations, see here.

Requirements

See the installation instruction for a step-by-step installation guide. See the server instruction for server settup.

• Install cuda-8.0
• Install cudnn v5.1
• Download Pytorch 2.7 and clone the repository.

pip install http://download.pytorch.org/whl/cu80/torch-0.1.12.post2-cp27-none-linux_x86_64.whl
pip install torchvision
git clone https://github.com/meliketoy/wide-resnet.pytorch

How to run

After you have cloned the repository, you can train each dataset of either cifar10, cifar100 by running the script below.

python main --lr 0.1 resume false --net_type [lenet/vggnet/resnet/wide-resnet] --depth 28 --widen_factor 10 --dropout_rate 0.3 --dataset [cifar10/cifar100] 

Implementation Details

epoch	learning rate	weight decay	Optimizer	Momentum	Nesterov
0 ~ 60	0.1	0.0005	Momentum	0.9	true
61 ~ 120	0.02	0.0005	Momentum	0.9	true
121 ~ 160	0.004	0.0005	Momentum	0.9	true
161 ~ 200	0.0008	0.0005	Momentum	0.9	true

CIFAR-10 Results

Below is the result of the test set accuracy for CIFAR-10 dataset training.

Accuracy is the average of 5 runs

network	dropout	preprocess	GPU:0	GPU:1	per epoch	accuracy(%)
wide-resnet 28x10	0	ZCA	5.90G	-	2 min 03 sec	95.83
wide-resnet 28x10	0	meanstd	5.90G	-	2 min 03 sec	96.21
wide-resnet 28x10	0.3	meanstd	5.90G	-	2 min 03 sec	96.27
wide-resnet 28x20	0.3	meanstd	8.13G	6.93G	4 min 10 sec	96.55
wide-resnet 40x10	0.3	meanstd	8.08G	-	3 min 13 sec	96.31
wide-resnet 40x14	0.3	meanstd	7.37G	6.46G	3 min 23 sec	96.34

CIFAR-100 Results

Below is the result of the test set accuracy for CIFAR-100 dataset training.

Accuracy is the average of 5 runs

network	dropout	preprocess	GPU:0	GPU:1	per epoch	Top1 acc(%)	Top5 acc(%)
wide-resnet 28x10	0	ZCA	5.90G	-	2 min 03 sec	80.07	95.02
wide-resnet 28x10	0	meanstd	5.90G	-	2 min 03 sec	81.02	95.41
wide-resnet 28x10	0.3	meanstd	5.90G	-	2 min 03 sec	81.49	95.62
wide-resnet 28x20	0.3	meanstd	8.13G	6.93G	4 min 05 sec	82.45	96.11
wide-resnet 40x10	0.3	meanstd	8.93G	-	3 min 06 sec	81.42	95.63
wide-resnet 40x14	0.3	meanstd	7.39G	6.46G	3 min 23 sec	81.87	95.51