By Kyle Cranmer, Lukas Heinrich, Tim Head, Gilles Louppe
Based on earlier work with NYU CDS masters students Manoj Kumar, Phil Yeres, and Michele Ceru and discussions with Brenden Lake and Gilles Louppe.
The point of this notebook is to explore the synergy of three powerful techniques:
-
generic likelihood-free inference engines (eg. ABC, carl, etc.) that enable statistical inference on the parameters of a theory that are implicitly defined by a simulator
-
workflows that encapsulate scientific pipelines and extend the scope from reproducibility to reusability
-
active learning and sequential design algorithms (eg. Bayesian optimization) that balance exploration and exploitation to efficiently optimize an expensive black box objective function
Together, these three ideas can be combined to enable an efficient and automated loop of the scientific method -- at least, for a sufficiently well posed problem. The scientist's input to the system are:
- an external workflow that implements some experimental protocol (orange Exp component)
- an external workflow that implements a simulator for those experiments, which depends on some theoretical parameters that we would like to infer (orange Sim component).
Interesingly, we will use the simulator not only to perform inference on the parameters, but also to design the next experiment (this is where active learning comes in).
- demo_gaussian.ipynb Runs a fast demo with a simple Gaussian simulator
- demo_weinberg.ipynb Runs a fast demo with an approximate physics-inspired simulation locally
- demo_weinberg_yadage.ipynb Runs a slower demo with an approximate physics-inspired simulation through a workflow system. This measures the Weinberg angle in the standard model of particle physics.
- demo_weinberg_yadage_full.ipynb Runs a much slower demo with a real Physics simulator workflow.