Can we set default values of hyper parameters for HPO?

[shiyf369]

Because we know some default configuration can reach good accuracy, we want to use these default configuration as startup parameters.
How do we set it? Thank you.

[QuanluZhang]

@shiyf369 , this is a very good feature request. currently, we have not supported specifying initial configurations yet. I have two proposals for this feature request. first, providing a new field in experiment config yaml file, e.g., initial_trial_configs which is list of trial configurations. These configurations must be included in the specified search space. Second, adding another field for each key in search space to specify the distribution of the candidate values of this key. users can specify high sampling probability for the candidate values that they think might perform better. The first one is easier to be supported, and easy to use. The second one complex but may cover more scenarios.

Feel free to give us your comments, proposals, or more information about your scenario and requirements :)

[shiyf369]

Thank you for your reply. Our requirement is very simply. The hyper parameters can be assigned a default value, NNI run the default value first in trials.

For example, the "learning_rate" can be assigned a default value 0.008.
{
"learning_rate": {"_type": "uniform", "_value": [0.0001, 0.1], "_default_value": 0.008}
}

[QuanluZhang]

@shiyf369 , is there any case that you want to specify more than one initial configs? if we put _default_value to each hyper-parameter, then it is not easy to specify two initial configs. for example, "learning_rate" has two default values, "batch_size" has two default values, then there would be 4 initial configs

[Major-333]

Has this feature been implemented yet? Or How can I make the scheduler use a default set of parameters?
Thank you.

[S-exypig]

Like the questioner, we hope to give a default value to each field in the search space, such as the original search space:
"dim": {"_type": "choice", "_value": [50,100,150]},
"lr": {"_type": "choice","_value": [0.001, 0.0001, 0.00001]},
I hope to select dim=50 and lr=0.001 by default for the first time. How to achieve this?