Skip to content
This repository has been archived by the owner on Mar 19, 2024. It is now read-only.

[WIP] Byol Repro (v2) #454

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open

Conversation

iseessel
Copy link
Contributor

@iseessel iseessel commented Oct 17, 2021

Continues work done by @lluiscastrejonsubira.

Further Improvements and bug fixes by @iseessel and @pranavsinghps1.

We are still unable to reproduce the full results of the BYOL paper. Here is our current linear transfer results. That is running byol_8node_resnet.yaml and then running /eval_resnet_8gpu_transfer_in1k_linear.yaml and reporting the best test results:

top_1: 69.726
top_5: 89.408

However, the BYOL paper reports the following, so we are ~3% lower.

top_1: 72.5

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 17, 2021
@iseessel
Copy link
Contributor Author

Note this replaces: @lluiscastrejonsubira PR here: #290

@iseessel iseessel mentioned this pull request Oct 17, 2021
@iseessel iseessel changed the title [WIP] Byol v2 [WIP] Byol Repro (v2) Oct 17, 2021
@facebook-github-bot
Copy link
Contributor

@iseessel has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@iseessel has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@iseessel has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@iseessel has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@iseessel has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@iseessel has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@iseessel has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@iseessel has updated the pull request. You must reimport the pull request before landing.

@zhifanwu
Copy link

zhifanwu commented Mar 13, 2022

@iseessel, thanks for your great work. Can I know what accuracy does this version of BYOL achieve now?

@iseessel
Copy link
Contributor Author

@zhifanwu This will be finished in the next month or so. I believe it is currently fully reproduced, but will want to run some more experiments to verify.

@YannDubs
Copy link

Hi @iseessel are there any updates on that? Will you upload the weights to the model zoo?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants