Add Hyperplane Tree (#20)

requirements.txt

@@ -5,6 +5,7 @@ pytest
 pandas
 matplotlib
 tqdm
+torch
 pyomo
 omlt @ git+https://github.com/cog-imperial/OMLT
 linear-tree @ git+https://github.com/emsunshine/linear-tree

systems2atoms/__init__.py

@@ -1,2 +1,3 @@
 from .systems import *
-from .surrogates import *
+from .surrogates import *
+from .hyperplanetree import *

systems2atoms/hyperplanetree/README.md

@@ -0,0 +1,49 @@
+# hyperplanetree
+A Python library to build piecewise linear or piecewise quadratic models in multi-dimensional spaces.
+
+This code is a fork of [linear-tree](https://github.com/cerlymarco/linear-tree). Please see this repository for more information about Linear Model Decision Trees!
+
+## What does this fork include?
+
+The main features of this fork (compared to upstream) are as follows:
+
+1. Translate the mathematics of linear-tree into PyTorch tensor operations. This is roughly 1.5-2.5x faster on CPU and enables GPU calculations. (The exact speedup is highly dependent on your dataset, model hyperparameter choices, and probably even your CPU.)
+2. HyperplaneTree: Hyperplanes (linear combinations of features) are considered as splitting variables. This significantly increases the training cost of the tree, which motivated the PyTorch rewrite.
+3. "Mixed-integer linear program" (MIP) formulations for hyperplane trees via [OMLT](https://github.com/cog-imperial/OMLT) and [Pyomo](https://pyomo.org).
+
+## Installation
+This package can be installed as part of systems2atoms:
+
+```pip install git+https://github.com/LLNL/systems2atoms```
+
+Then, it can be imported as follows:
+
+```python
+from systems2atoms.hyperplanetree import LinearTreeRegressor, HyperplaneTreeRegressor
+```
+
+Alternatively, you can install hyperplanetree without installing the rest of systems2atoms:
+
+```pip install "hyperplanetree @ git+https://[email protected]/LLNL/systems2atoms#subdirectory=systems2atoms/hyperplanetree"```
+
+The import will then be:
+
+```python
+from hyperplanetree import LinearTreeRegressor, HyperplaneTreeRegressor
+```
+
+## Quickstart
+Please see the notebooks folder for basic tutorials on these models.
+
+## Why Hyperplanes?
+TLDR: Expanding the search space of possible splits can allow us to build trees with better accuracy for the same number of leaves.
+
+We use leaf-model decision trees as surrogates in optimization problems. See: [Ammari et al. Linear model decision trees as surrogates in optimization of engineering applications](https://www.sciencedirect.com/science/article/pii/S009813542300217X)
+
+Expanding the search space of possible splits can allow us to build trees with better accuracy for the same number of leaves. This is useful because when we translate the trees to optimization problems (via OMLT and Pyomo), each leaf becomes a binary variable. Optimization problems generally have poor scaling with the number of binary variables, so we cannot endlessly deepen our trees to achieve high accuracy.
+
+Hyperplanes are specifically useful because they are linear. When converted to optimization constraints, the problem will still be linear: a Mixed-Integer Linear Problem (MIP). 
+
+To further increase the accuracy of our models, we can also use quadratic terms in the splits and leaf regressions. This turns the problem into a Mixed-Integer Quadratic Constrained Problem (MIQCP). Quadric Trees are implemented as an experimental feature in this repository. We have not yet seen enough error metric improvement from these models to justify their use.
+
+Further increases in model complexity are possible. However, they are not implemented here as they quickly become computationally impractical even for small regression tasks.

systems2atoms/hyperplanetree/__init__.py

@@ -0,0 +1,26 @@
+from .hyperplanetree.lineartree import (
+    LinearTreeRegressor,
+)
+
+from .hyperplanetree.hyperplane_tree import (
+    HyperplaneTreeRegressor,
+)
+
+from .hyperplanetree.quadric_tree import (
+    QuadricTreeRegressor,
+)
+
+from .hyperplanetree.omlt import (
+    HyperplaneTreeDefinition,
+    HyperplaneTreeGDPFormulation,
+    HyperplaneTreeHybridBigMFormulation,
+    HyperplaneTreeOMLTFormulationMixin,
+)
+
+from .hyperplanetree.uq import (
+    calculate_uncertainty_metrics,
+)
+
+from .hyperplanetree._classes import (
+    plot_surrogate_2d,
+)

systems2atoms/hyperplanetree/hyperplanetree/__init__.py

@@ -0,0 +1,26 @@
+from .lineartree import (
+    LinearTreeRegressor,
+)
+
+from .hyperplane_tree import (
+    HyperplaneTreeRegressor,
+)
+
+from .quadric_tree import (
+    QuadricTreeRegressor,
+)
+
+from .omlt import (
+    HyperplaneTreeDefinition,
+    HyperplaneTreeGDPFormulation,
+    HyperplaneTreeHybridBigMFormulation,
+    HyperplaneTreeOMLTFormulationMixin,
+)
+
+from .uq import (
+    calculate_uncertainty_metrics,
+)
+
+from ._classes import (
+    plot_surrogate_2d,
+)