Skip to content

Latest commit

 

History

History
139 lines (107 loc) · 4.1 KB

README.md

File metadata and controls

139 lines (107 loc) · 4.1 KB
Raw

On-Device Saliency Prediction Based on Pseudoknowledge Distillation

This paper has been published to IEEE Transactions on Industrial Informatics (IEEE TII).

Paper: IEEE Transactions on Industrial Informatics

This offical implementation of PKD (Pseudoknowledge Distillation) from On-Device Saliency Prediction Based on Pseudoknowledge Distillation by Chakkrit Termritthikun.

PKD

This code is based on the implementation of EML-NET-Saliency, SimpleNet, MSI-Net, and EEEA-Net.

Prerequisite for server

  • Tested on Ubuntu OS version 20.04.4 LTS
  • Tested on Python 3.6.13
  • Tested on CUDA 11.6
  • Tested on PyTorch 1.10.2 and TorchVision 0.11.3
  • Tested on NVIDIA V100 32 GB (four cards)

Cloning source code

git clone https://github.com/chakkritte/PKD/
cd PKD
mkdir data

The dataset folder structure:

PKD
|__ data
    |_ salicon
      |_ fixations
      |_ saliency
      |_ stimuli
    |_ mit1003
      |_ fixations
      |_ saliency
      |_ stimuli
    |_ cat2000
      |_ fixations
      |_ saliency
      |_ stimuli
    |_ pascals
      |_ fixations
      |_ saliency
      |_ stimuli
    |_ osie
      |_ fixations
      |_ saliency
      |_ stimuli

Creating new environments

conda create -n pkd python=3.6.13
conda activate pkd
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch

Install Requirements

pip install -r requirements.txt --no-cache-dir

Salicon Pretrained models and Evaluation

  1. Download Salicon pretrained models from
bash download_pretrained.sh
  1. Change teacher parameter -> ofa595 or efb4 or pnas
python validate.py --dataset salicon --student eeeac2 --teacher ofa595

Results of single teacher method for student (EEEA-Net-C2) on Salicon dataset

Teacher Student CC↑ KL↓ NSS↑ Link
OFA595 EEEA-Net-C2 0.9062 0.1907 1.9298 Pretrained
EfficientNet-B4 EEEA-Net-C2 0.9055 0.1924 1.9346 Pretrained
PNASNet-5 EEEA-Net-C2 0.9044 0.1956 1.9319 Pretrained

Usage

Training on Salicon dataset (Teacher: OFA595, Student: EEEA-C2)

python main.py --student eeeac2 --teacher ofa595 --dataset salicon --model_val_path model_salicon.pt

Architecture Transfer

Training on MIT1003 dataset (Teacher: OFA595, Student: EEEA-C2)

python main.py --student eeeac2 --teacher ofa595 --dataset mit1003 --model_val_path model_mit1003.pt

Training on CAT2000 dataset (Teacher: OFA595, Student: EEEA-C2)

python main.py --student eeeac2 --teacher ofa595 --dataset cat2000 --model_val_path model_cat2000.pt

Training on PASCALS dataset (Teacher: OFA595, Student: EEEA-C2)

python main.py --student eeeac2 --teacher ofa595 --dataset pascals --model_val_path model_pascals.pt

Training on OSIE dataset (Teacher: OFA595, Student: EEEA-C2)

python main.py --student eeeac2 --teacher ofa595 --dataset osie --model_val_path model_osie.pt

Citation

If you use PKD or any part of this research, please cite our paper:

@ARTICLE{umer2022device,
  author={Umer, Ayaz and Termritthikun, Chakkrit and Qiu, Tie and Leong, Philip H. W. and Lee, Ivan},
  journal={IEEE Transactions on Industrial Informatics}, 
  title="{On-Device Saliency Prediction Based on Pseudoknowledge Distillation}", 
  year={2022},
  volume={18},
  number={9},
  pages={6317-6325},
  doi={10.1109/TII.2022.3153365}}

License

Apache-2.0 License