-
Notifications
You must be signed in to change notification settings - Fork 155
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WFRunner] Handle resource limits and CPU better #1532
[WFRunner] Handle resource limits and CPU better #1532
Conversation
REQUEST FOR PRODUCTION RELEASES:
This will add The following labels are available |
@sawenzel do you have ay comments here? If not, I would go ahead and merge. |
The commit message sounds reasonable. But is this necessary to test the dynamic system (just wondering how time-critical it is)? |
It's not necessary for testing right now. Tested and works just fine. |
* Account for relative CPU factor in case of sampling Studies have shown that being able to backfill tasks can have a difference for CPU efficiency, especially for transport. That is in particular important for high-efficiency jobs such as high-interaction-rate pp simulations. * Abort by default if estimated resources exceed limits * Run anyway, if --optimistic-resources is passed * Fix: Actually reset the overestimated resources to the limits as otherwise the runner would silently quit when nothing else can be done. * In case of dynamically sampled resources and if a corresponding task has been run already: Reset the assigned resources to the limits if they exceed the boundaries.
c4787ea
to
c01d0ad
Compare
@chiarazampolli
Anyway, first of all to be discussed with @sawenzel This is important to keep the CPU efficiency where it is in case it is already high by default. I will explain in an email. |
* Account for relative CPU factor in case of sampling Studies have shown that being able to backfill tasks can have a difference for CPU efficiency, especially for transport. That is in particular important for high-efficiency jobs such as high-interaction-rate pp simulations. * Abort by default if estimated resources exceed limits * Run anyway, if --optimistic-resources is passed * Fix: Actually reset the overestimated resources to the limits as otherwise the runner would silently quit when nothing else can be done. * In case of dynamically sampled resources and if a corresponding task has been run already: Reset the assigned resources to the limits if they exceed the boundaries. Co-authored-by: Benedikt Volkel <[email protected]>
* Account for relative CPU factor in case of sampling Studies have shown that being able to backfill tasks can have a difference for CPU efficiency, especially for transport. That is in particular important for high-efficiency jobs such as high-interaction-rate pp simulations. * Abort by default if estimated resources exceed limits * Run anyway, if --optimistic-resources is passed * Fix: Actually reset the overestimated resources to the limits as otherwise the runner would silently quit when nothing else can be done. * In case of dynamically sampled resources and if a corresponding task has been run already: Reset the assigned resources to the limits if they exceed the boundaries. Co-authored-by: Benedikt Volkel <[email protected]>
* Account for relative CPU factor in case of sampling Studies have shown that being able to backfill tasks can have a difference for CPU efficiency, especially for transport. That is in particular important for high-efficiency jobs such as high-interaction-rate pp simulations. * Abort by default if estimated resources exceed limits * Run anyway, if --optimistic-resources is passed * Fix: Actually reset the overestimated resources to the limits as otherwise the runner would silently quit when nothing else can be done. * In case of dynamically sampled resources and if a corresponding task has been run already: Reset the assigned resources to the limits if they exceed the boundaries. Co-authored-by: Benedikt Volkel <[email protected]>
* Account for relative CPU factor in case of sampling Studies have shown that being able to backfill tasks can have a difference for CPU efficiency, especially for transport. That is in particular important for high-efficiency jobs such as high-interaction-rate pp simulations. * Abort by default if estimated resources exceed limits * Run anyway, if --optimistic-resources is passed * Fix: Actually reset the overestimated resources to the limits as otherwise the runner would silently quit when nothing else can be done. * In case of dynamically sampled resources and if a corresponding task has been run already: Reset the assigned resources to the limits if they exceed the boundaries. Co-authored-by: Benedikt Volkel <[email protected]>
Account for relative CPU factor in case of sampling
Studies have shown that being able to backfill tasks can have a
difference for CPU efficiency, especially for transport.
That is in particular important for high-efficiency jobs such as
high-interaction-rate pp simulations.
Abort by default if estimated resources exceed limits
Run anyway, if --optimistic-resources is passed
otherwise the runner would silently quit when nothing else can be
done.
In case of dynamically sampled resources and if a corresponding task
has been run already:
Reset the assigned resources to the limits if they exceed the
boundaries.