v0.6.0: `OptimumSelector`

What's New

OptimumSelector

• Until know, the optimal power limit for GlobalPowerLimitOptimizer was the one that minimizes the Zeus time-energy cost. Not everyone would want that.
• Now, OptimumSelector is an abstract base class with which you can implement your own optimal power limit selection policy.
• Pre-implemented one are Time, Energy, ZeusCost, and MaxSlowdownConstraint. These are thoroughly tested.

wait_steps

• Now, you can specify wait_steps in GlobalPowerLimitOptimizer, and it'll wait for the specified number of steps before profiling and optimizing.
• wait_steps is set to 1 by default to because users may have torch.backends.cudnn.benchmark = True and DataLoader workers usually need time to warm up before ramping up to their normal fetch throughput.

Breaking Changes

• GlobalPowerLimitOptimizer now takes an instance of OptimumSelector in its constructor, instead of eta_knob. If you want to recover the functionality of v0.5.0, modify your code like this:

  # Before
  plo = GlobalPowerLimitOptimizer(..., eta_knob=0.5, ...)

  # After
  from zeus.optimizer.power_limit import ZeusCost
  
  plo = GlobalPowerLimitOptimizer(..., optimum_selector=ZeusCost(eta_knob=0.5), ...)