Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add automatic mixed precision support #62

Open
egillax opened this issue Mar 28, 2023 · 2 comments
Open

Add automatic mixed precision support #62

egillax opened this issue Mar 28, 2023 · 2 comments
Labels
enhancement New feature or request

Comments

@egillax
Copy link
Collaborator

egillax commented Mar 28, 2023

In the latest develop version of torch, automatic mixed precision has just been added. See this file and the unit test here to see how to use it

@egillax egillax added the enhancement New feature or request label Mar 28, 2023
@egillax
Copy link
Collaborator Author

egillax commented Apr 3, 2023

Here is a simple vignette describing how this works in torch:

https://torch.mlverse.org/docs/dev/articles/amp.html

@lhjohn
Copy link
Collaborator

lhjohn commented Apr 7, 2023

Benchmark results for RTX 3090

system.time({run(autocast = FALSE, scale = FALSE)})
user  system elapsed 0.75    0.09    1.88

system.time({run(autocast = TRUE, scale = FALSE)})
user  system elapsed 0.58    0.05    1.54

system.time({run(autocast = TRUE, scale = TRUE)})
user  system elapsed 0.38    0.11    1.64

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants