Automate hyperparameter optimization

[thefirebanks]

• Look for information on automatic hyperparameter tuning optimization and its viability for our project
• Define hyperparameters to be optimized for
• Test new methods like population-based optimization or bayesian optimization

---

Bayesian Optimization explained

• Conceptually: https://towardsdatascience.com/a-conceptual-explanation-of-bayesian-model-based-hyperparameter-optimization-for-machine-learning-b8172278050f

Population Based Optimization

• Concept + comparison with Bayesian: https://medium.com/distributed-computing-with-ray/hyperparameter-optimization-for-transformers-a-guide-c4e32c6c989b

Huggingface + Ray + W&B implementation of hyperparameter tuning

• Blog detailing actual process: https://huggingface.co/blog/ray-tune

[jordiplanescutxi]

Fix training variability issue

[jordiplanescutxi]

Fix training variability issue