Pytorch implementation of ACLS

This is the implementation of the paper "ACLS:Adaptive and Conditional Label Smoothing for Network Calibration".

For detailed information, please check out our paper [arXiv].

Requirements

This repository has been tested with the following libraries:

• Python = 3.6
• PyTorch = 1.8.0

Getting started

Datasets

The structure should be organized as follows:

Tiny-ImageNet

└── /dataset/tiny-imagenet-200
    ├── test
    ├── train
    ├── val
    │   ├── images
    │   └── val_annotations.txt
    ├── wninds.txt
    └── words.txt

ImageNet

├── /dataset/ILSVRC2012
└── data
    ├── train
    ├── val
    └── ILSVRC2012_devkit_t12

Training

Tiny-ImageNet

bash scripts/run_tiny.sh

ImageNet

bash scripts/imagenet/run_imagenet.sh

Testing

Tiny-ImageNet

python tools/test.py data=tiny_imagenet hydra.run.dir=your/weight/directory test.checkpoint=your_weight.pth

ImageNet

python tools/test.py data=imagenet hydra.run.dir=your/weight/directory test.checkpoint=your_weight.pth

Pretrained models

NOTE: we train networks with one and four A5000 GPU for Tiny-ImageNet and ImageNet, respectively.

Tiny-ImageNet

• ResNet50

[2023-02-07 18:31:53,040 INFO][tester.py:124 - log_eval_epoch_info] - 
+---------+---------+---------+---------+
| samples | acc     | acc_5   | macc    |
+---------+---------+---------+---------+
| 10000   | 0.64840 | 0.85960 | 0.64840 |
+---------+---------+---------+---------+
[2023-02-07 18:31:53,040 INFO][tester.py:125 - log_eval_epoch_info] - 
+---------+---------+---------+---------+---------+
| samples | nll     | ece     | aece    | cece    |
+---------+---------+---------+---------+---------+
| 10000   | 1.42108 | 0.01050 | 0.01029 | 0.00135 |
+---------+---------+---------+---------+---------+

• ResNet101

[2023-01-21 05:55:53,096 INFO][trainer.py:214 - log_eval_epoch_info] - 
+---------+---------+---------+---------+
| samples | acc     | acc_5   | macc    |
+---------+---------+---------+---------+
| 10000   | 0.65780 | 0.86330 | 0.65780 |
+---------+---------+---------+---------+
[2023-01-21 05:55:53,096 INFO][trainer.py:215 - log_eval_epoch_info] - 
+---------+---------+---------+---------+---------+
| samples | nll     | ece     | aece    | cece    |
+---------+---------+---------+---------+---------+
| 10000   | 1.38089 | 0.01107 | 0.01151 | 0.00131 |
+---------+---------+---------+---------+---------+

ImageNet

• ResNet50

[2023-02-10 14:01:17,389 INFO][tester.py:149 - log_eval_epoch_info] -
+---------+---------+---------+---------+
| samples | acc     | acc_5   | macc    |
+---------+---------+---------+---------+
| 40000   | 0.75650 | 0.92363 | 0.75360 |
+---------+---------+---------+---------+
[2023-02-10 14:01:17,389 INFO][tester.py:150 - log_eval_epoch_info] -
+---------+---------+---------+---------+---------+
| samples | nll     | ece     | aece    | cece    |
+---------+---------+---------+---------+---------+
| 40000   | 1.01375 | 0.01021 | 0.01205 | 0.00028 |
+---------+---------+---------+---------+---------+

References

Our work is mainly built on FLSD, MbLS, and CALS. Thanks to the authors!