-
Notifications
You must be signed in to change notification settings - Fork 311
Training Data Collection
You can upload existing training data to a no-tuning session. If you do not have data, you can generate training data with random sampling or LHS sampling(recommended).
If you have training data collected previously, you can upload them to a new no-tuning session. The data path is specified as save_path in the driver configuration. The following command will upload all the data in save_path to the server.
fab upload_batch
OtterTune can generate random configurations if there is no training data. You can choose randomly generate session when creating a new session. It gets min/max value from the server-side knob fixture and randomly choose a value within the range. With the default knob fixture like ottertune/server/website/website/fixtures/postgres-96_knobs.json, it cannot work well because the min/max value of knobs in the fixture are not appropriate for tuning. e.g. min value of shared_buffers is 128kB and max value is several PB. Besides, it randomly generate configurations for all knobs in the fixture, instead of some important ones.
One possible way is to refactor the server-side knob fixture, remain the knobs you may tune, and change the min/max values to the reasonable range according to your hardware. Another better way is to use LHS sampling as described below, you can specify a LHS knob file in the client side for tuning.
When collecting training data from scratch, the latin-hypercube sampling(LHS) is a smarter way than the random sampling. LHS spreads the sample points more evenly across all possible values. We use pyDOE library, more details can be found here.
OtterTune supports LHS in the client side. You should specify lhs_knob_path and lhs_save_path in the driver configuration. LHS knob file includes the knobs you may tune and their tuning range. Tuning range (min/max values) is important. e.g. The maximum value of memory-related knobs should not exceed the total memory of your hardware. Also, we find setting default values as the minimum value works in most cases. Notice that this file is a different one from the server-side knob fixture. Knob type in this file is integer/float/bytes/time. The LHS knob file path is set as lhs_knob_path in the driver configuration. The output of LHS is located at lhs_save_path
Then you can generate LHS samples, 10 sample points by default.
fab lhs_samples # 10 samples by default
fab lhs_samples:20 # 20 samples
After generating LHS samples to lhs_save_path, you can run loops for all the samples in lhs_save_path. We recommend to use a no-tuning session. It will experiment on each of the LHS samples and update the results to the server.
fab run_lhs