Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DeepSpeed and FairScale #78

Open
Mehrad0711 opened this issue Jan 19, 2021 · 0 comments
Open

DeepSpeed and FairScale #78

Mehrad0711 opened this issue Jan 19, 2021 · 0 comments
Labels
enhancement New feature or request P3 Nice to have, not working on it for now performance Performance issues: resource usage, scaling, GPU efficiency training Issues with training loop, optimization, GPU usage

Comments

@Mehrad0711
Copy link
Member

Huggingface recently started support for DeepSpeed and FairScale which supposedly give major speedups in both multi-gpu and single-gpu settings. The feature is still experimental but worthwhile to look into and see if we can get any performance boosts.

Documentation
@gcampax gcampax added enhancement New feature or request performance Performance issues: resource usage, scaling, GPU efficiency training Issues with training loop, optimization, GPU usage labels Jul 15, 2021
@nrser nrser added the P3 Nice to have, not working on it for now label Aug 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request P3 Nice to have, not working on it for now performance Performance issues: resource usage, scaling, GPU efficiency training Issues with training loop, optimization, GPU usage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nrser @gcampax @Mehrad0711