You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When tuning the base models, some variables are unused ( I presume the classification heads that we replace with our own classification head/seq2seq model). This breaks the implementation, making the code unrunnable.
This can be fixed by using ddp_spawn_find_unused_parameters_true strategy rather than ddp_spawn.
An issue discussing this can be found here
NOTE: this issue does not pop up when freezing the base model, leading more credence to the idea that the unused variables are the original classification head that the base model comes with
The text was updated successfully, but these errors were encountered:
When tuning the base models, some variables are unused ( I presume the classification heads that we replace with our own classification head/seq2seq model). This breaks the implementation, making the code unrunnable.
This can be fixed by using
ddp_spawn_find_unused_parameters_true
strategy rather thanddp_spawn
.An issue discussing this can be found here
NOTE: this issue does not pop up when freezing the base model, leading more credence to the idea that the unused variables are the original classification head that the base model comes with
The text was updated successfully, but these errors were encountered: