Merge pull request #142 from pnnl/develop

Merge develop (1.5) into master

README.md

@@ -3,7 +3,7 @@
   <img src="figs/Neuromancer.png" width="250">  
 </p>
 
-# NeuroMANCER v1.4.2
+# NeuroMANCER v1.5.0
 
 **Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations (NeuroMANCER)**
 is an open-source differentiable programming (DP) library for solving parametric constrained optimization problems, 
@@ -16,13 +16,21 @@ differentiable models and algorithms embedded with prior knowledge and physics.
 ![Static Badge](https://img.shields.io/badge/pip_install-neuromancer-blue)
  ![PyPI - Version](https://img.shields.io/pypi/v/neuromancer)
 
-### New in v1.4.2
-New feature: `GradientProjection` now supports updating violation energy at each gradient step, as in the original [DC3 paper](https://arxiv.org/pdf/2104.12225.pdf). See the full [release notes](RELEASE_NOTES.md) for more information.
 
-**New Colab Examples:**  
-> ⭐ [Physics-Informed Neural Networks (PINNs) for solving partial differential equations (PDEs) in NeuroMANCER](#physics-informed-neural-networks-pinns-for-partial-differential-equations-pdes)
+### New in v1.5.0
+![Lightning](https://img.shields.io/badge/-Lightning-792ee5?logo=pytorchlightning&logoColor=white)
+Now supports integration with PyTorch Lightning (https://lightning.ai/docs/pytorch/stable/), bringing:
+* User workflow simplifications: zero boilerplate code and increased modularity 
+* Ability for user to define custom training logic easily 
+* Easy support for distributed GPU training
+* Weights and Biases hyperparameter tuning 
 
-> ⭐ [System identification for ordinary differential equations (ODEs)](#ordinary-differential-equations-odes)
+Please refer to the Lightning folder and its [README](examples/lightning_integration_examples/README.md).
+
+**New Colab Examples:**
+> ⭐ [Various domain examples, such as system identification of building thermal dynamics, in NeuroMANCER](#domain-examples)
+
+> ⭐ [PyTorch lightning integration Examples ](#lightning-integration-examples)
 
 
 ## Features and Examples
@@ -111,8 +119,40 @@ Part 5: Using Cvxpylayers for differentiable projection onto the polytopic feasi
 + <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/control/Part_5_neural_Lyapunov.ipynb">
   <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 5: Learning neural Lyapunov function for a nonlinear dynamical system.
 
+### Domain Examples 
+
++ <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/domain_examples/DPC_building_control.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 1: Learning to Control Indoor Air Temperature in Buildings.
+
++ <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/domain_examples/DPC_PSH.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 2: Learning to Control an Pumped-Hydroelectricity Energy Storage System.
+
++ <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/domain_examples/NODE_building_dynamics.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 3: Learning Building Thermal Dynamics using Neural ODEs.
+
++ <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/domain_examples/NODE_rc_networks.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 4: Data-driven modeling of a Resistance-Capacitance network with Neural ODEs.
+
++ <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/domain_examples/NODE_swing_equation.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 5: Learning Swing Equation Dynamics using Neural ODEs.
+
++ <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/domain_examples/NSSM_building_dynamics.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 6: Learning Building Thermal Dynamics using Neural State Space Models.
+
+### Lightning Integration Examples 
+
++ <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/lightning_integration_examples/Part_1_lightning_basics_tutorial.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 1: Lightning Integration Basics.
+
++ <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/lightning_integration_examples/Part_2_lightning_advanced_and_gpu_tutorial.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 2: Lightning Advanced Features and Automatic GPU Support.
 
++ <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/lightning_integration_examples/Part_4_lightning_wanb_hyperparameter_tuning.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 3: Hyperparameter Tuning With Lightning & WandB
 
++ <a target="_blank" href="https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/lightning_integration_examples/other_examples/lightning_custom_training_example.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> Part 4: Defining Custom Training Logic via Lightning Modularized Code.
+
 ## Documentation
 The documentation for the library can be found [online](https://pnnl.github.io/neuromancer/). 
 There is also an [introduction video](https://www.youtube.com/watch?v=YkFKz-DgC98) covering 

RELEASE_NOTES.md

@@ -1,6 +1,14 @@
 
 ##  Release notes
 
+### Version 1.5.0 Release Notes 
++ New Feature: PyTorch Lightning Integration with NeuroMANCER core library. All these features are opt-in. 
+  + Code simplifications: zero boilerplate code, increased modularity 
+  + Added ability for user to define custom training logic 
+  + Easy support for GPU and multi-GPU training
+  + Easy Weights and Biases (https://wandb.ai/site) hyperparameter tuning and Tensorboard Logging
+
+
 ### Version 1.4.2 Release Notes 
 + New feature: Update violation energy for projected gradient #110 (based on idea #86).
 + Reverted `psl.nonautonomous.TwoTank` `(umin, umax)` bounds to `(0.5, 0.5)` for numerical stability #105

examples/ODEs/Part_1_NODE.ipynb

@@ -55,7 +55,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2dca093f",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {

examples/ODEs/Part_2_param_estim_ODE.ipynb

@@ -48,7 +48,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "901105f0",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {

examples/ODEs/Part_3_UDE.ipynb

@@ -54,7 +54,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5a386539",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {

examples/ODEs/Part_4_nonauto_NODE.ipynb

@@ -54,7 +54,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a7d0fa0d",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {

examples/ODEs/Part_5_nonauto_NSSM.ipynb

@@ -60,7 +60,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "99c494f8",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {

examples/ODEs/Part_6_NetworkODE.ipynb

@@ -47,6 +47,30 @@
     "Our task is to leverage the known structure of RC networks to estimate capacitances and coupling resistivities from data. We will use the NeruoMANCER package for this task; let's do our imports:"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Install (Colab only)\n",
+    "Skip this step when running locally."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,

examples/ODEs/Part_7_DeepKoopman.ipynb

@@ -63,7 +63,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1fd58f0c",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {

examples/ODEs/Part_8_nonauto_DeepKoopman.ipynb

@@ -62,7 +62,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "89ee6302",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {
@@ -1796,9 +1804,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "neuromancer",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "neuromancer"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -1810,7 +1818,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.4"
+   "version": "3.8.2"
   }
  },
  "nbformat": 4,

examples/PDEs/Part_1_PINN_DiffusionEquation.ipynb

@@ -43,10 +43,17 @@
    },
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\"\n",
+    "!pip install neuromancer\n",
     "!pip install pyDOE"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

examples/PDEs/Part_2_PINN_BurgersEquation.ipynb

@@ -41,10 +41,17 @@
    },
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\"\n",
+    "!pip install neuromancer\n",
     "!pip install pyDOE"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

examples/PDEs/Part_3_PINN_BurgersEquation_inverse.ipynb

@@ -41,10 +41,17 @@
    },
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\"\n",
+    "!pip install neuromancer\n",
     "!pip install pyDOE"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

examples/control/Part_1_stabilize_linear_system.ipynb

@@ -39,7 +39,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "54e902d0",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {

examples/control/Part_2_stabilize_ODE.ipynb

@@ -175,7 +175,15 @@
     }
    ],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "62795a33",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {

examples/control/Part_3_ref_tracking_ODE.ipynb

@@ -36,7 +36,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a3995d5f",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {

examples/control/Part_4_NODE_control.ipynb

@@ -44,7 +44,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install \"neuromancer[examples] @ git+https://github.com/pnnl/neuromancer.git@master\""
+    "!pip install neuromancer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "652c96fb",
+   "metadata": {},
+   "source": [
+    "*Note: When running on Colab, one might encounter a pip dependency error with Lida 0.0.10. This can be ignored*"
    ]
   },
   {