-
Notifications
You must be signed in to change notification settings - Fork 243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reported accuracy mismatch for vgg11_bn
#313
Comments
Thanks for the report @nps1ngh . Good catch! For all torchivsion models, the correct reference for accuracies should be this table: https://pytorch.org/vision/main/models.html#table-of-all-available-classification-weights (or the specific model pages like the one you gave above, which has the same info). Unfortunately, the ones reported on the torchhub website may go out of date. I opened #318 to try to think of solutions / mitigations. Meanwhile, it's probably worth updating the VGG table with the latest correct values. LMK if you would you like to open a PR for that. Thank you! |
Sure, I'll open one! |
The
hub
documentation here: https://pytorch.org/hub/pytorch_vision_vgg/reports the
vgg11_bn
top-1 error to be26.70
. That is, a top-1 accuracy of73.30
.This is almost as good as the
resnet34
reported here: https://pytorch.org/vision/main/models/generated/torchvision.models.resnet34.htmlBut the main problem is that
torchvision
reports a top-1 accuracy of70.37
for the it here: https://pytorch.org/vision/main/models/generated/torchvision.models.vgg11_bn.htmlI guess the latter reported accuracy is correct (
70.37
) and the one reported here byhub
is incorrect?Or am I missing something? Is this possibly related to pytorch/vision#223? (It's a really old issue though.)
Thanks!
The text was updated successfully, but these errors were encountered: