-
Notifications
You must be signed in to change notification settings - Fork 262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add wandb logger init to hydra runners #894
Conversation
Codecov ReportAttention: Patch coverage is
|
This reverts commit f9760e7.
@@ -131,8 +156,8 @@ def main( | |||
rdzv_backend="c10d", | |||
max_restarts=0, | |||
) | |||
elastic_launch(launch_config, runner_wrapper)(cfg, args) | |||
elastic_launch(launch_config, runner_wrapper)(cfg) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just out of curiosity is the runner_wrapper
just to make elastic_launch
happy?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep, it can only deal with a top level function
Add instantiation for wandb logger to the new hydra runners. The logger is a singleton logger that is only instantiated on master and independent of the runners. The runners can then just log from anywhere (without having to know all the run information) assuming the logger is instantiated.
example logger config in the yaml:
Example run: https://fairwandb.org/rgao/fairchem/runs/202410-2422-3005-86f0