Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistency in pretrained mae vitb16 weights #35

Open
jafarinia opened this issue Dec 14, 2023 · 0 comments
Open

Inconsistency in pretrained mae vitb16 weights #35

jafarinia opened this issue Dec 14, 2023 · 0 comments

Comments

@jafarinia
Copy link

jafarinia commented Dec 14, 2023

Hi
I have a question about your pretreined mae vitb16 weights.
When I use "https://github.com/ShoufaChen/AdaptFormer/releases/download/v0.1/mae_pretrain_vit_b.pth" which is the one on your GitHub repo and train on Cifar100 for 1 epoch I generally get
Acc@1 37.330 Acc@5 67.820 loss 2.768
(the command is python main_image.py --batch_size=128 --cls_token --epochs=1 --finetune=mae_pretrain_vit_b.pth --dist_eval
--data_path=cifar100 --output_dir=output --num_workers=16 --drop_path=0.0 --blr=0.1 --dataset=cifar100 --ffn_adapt)
But
When I use "https://dl.fbaipublicfiles.com/mae/pretrain/mae_pretrain_vit_base.pth" which is the claimed weight on your paper's text and train on Cifar100 for 1 epoch I generally get
Acc@1 5.660 Acc@5 21.350 loss 4.295.
(the command is python main_image.py --batch_size=128 --cls_token --epochs=1 --finetune=mae_pretrain_vit_base.pth --dist_e
val --data_path=cifar100 --output_dir=output --num_workers=16 --drop_path=0.0 --blr=0.1 --dataset=cifar100 --ffn_adapt)
These results are consistent over multiple runs.
My questions are, what is the difference between these two pretrained weights? And what causes this huge difference in results?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant