Merge pull request #24 from davidnabergoj/dev

Dev
davidnabergoj · Oct 19, 2024 · 01d3c14 · 01d3c14
2 parents f9fe25a + 38d29e1
commit 01d3c14
Show file tree

Hide file tree

Showing 60 changed files with 2,131 additions and 1,007 deletions.
diff --git a/docs/source/api/base_distributions.rst b/docs/source/api/base_distributions.rst
diff --git a/docs/source/api/bijections.rst b/docs/source/api/bijections.rst
diff --git a/docs/source/api/components.rst b/docs/source/api/components.rst
diff --git a/docs/source/api/multiscale_architectures.rst b/docs/source/api/multiscale_architectures.rst
diff --git a/docs/source/api/architectures.rst → ...source/architectures/general_modeling.rst b/docs/source/api/architectures.rst → ...source/architectures/general_modeling.rst
@@ -1,38 +1,86 @@
-Standard architectures
+API for standard architectures
 ============================
 We lists notable implemented bijection architectures.
 These all inherit from the Bijection class.
 
-.. _architectures:
+.. _autoregressive_architecture_api:
 
 Autoregressive architectures
 --------------------------------
 
 .. autoclass:: torchflows.architectures.RealNVP
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.InverseRealNVP
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.NICE
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.MAF
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.IAF
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.CouplingRQNSF
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.MaskedAutoregressiveRQNSF
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.InverseAutoregressiveRQNSF
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.CouplingLRS
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.MaskedAutoregressiveLRS
+    :members: __init__
+
+.. autoclass:: torchflows.architectures.InverseAutoregressiveLRS
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.CouplingDSF
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.UMNNMAF
+    :members: __init__
+
+.. _continuous_architecture_api:
 
 Continuous architectures
 -------------------------
 .. autoclass:: torchflows.architectures.DeepDiffeomorphicBijection
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.RNODE
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.FFJORD
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.OTFlow
+    :members: __init__
+
+.. _residual_architecture_api:
 
 Residual architectures
 -----------------------
 .. autoclass:: torchflows.architectures.ResFlow
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.ProximalResFlow
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.InvertibleResNet
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.PlanarFlow
+    :members: __init__
+
 .. autoclass:: torchflows.architectures.RadialFlow
-.. autoclass:: torchflows.architectures.SylvesterFlow
+    :members: __init__
+
+.. autoclass:: torchflows.architectures.SylvesterFlow
+    :members: __init__
diff --git a/docs/source/architectures/image_modeling.rst b/docs/source/architectures/image_modeling.rst
@@ -0,0 +1,56 @@
+API for multiscale architectures
+========================================================
+
+Multiscale architectures are suitable for image modeling.
+
+.. _multiscale_architecture_api:
+
+
+Classic multiscale architectures
+------------------------------
+
+.. autoclass:: torchflows.architectures.MultiscaleNICE
+    :members: __init__
+
+.. autoclass:: torchflows.architectures.MultiscaleRealNVP
+    :members: __init__
+
+.. autoclass:: torchflows.architectures.MultiscaleRQNSF
+    :members: __init__
+
+.. autoclass:: torchflows.architectures.MultiscaleLRSNSF
+    :members: __init__
+
+.. autoclass:: torchflows.bijections.finite.multiscale.architectures.MultiscaleDeepSigmoid
+    :members: __init__
+
+.. autoclass:: torchflows.bijections.finite.multiscale.architectures.MultiscaleDenseSigmoid
+    :members: __init__
+
+.. autoclass:: torchflows.bijections.finite.multiscale.architectures.MultiscaleDeepDenseSigmoid
+    :members: __init__
+
+
+Glow-style multiscale architectures
+------------------------------
+
+.. autoclass:: torchflows.architectures.AffineGlow
+    :members: __init__
+
+.. autoclass:: torchflows.architectures.ShiftGlow
+    :members: __init__
+
+.. autoclass:: torchflows.bijections.finite.multiscale.architectures.RQSGlow
+    :members: __init__
+
+.. autoclass:: torchflows.bijections.finite.multiscale.architectures.LRSGlow
+    :members: __init__
+
+.. autoclass:: torchflows.bijections.finite.multiscale.architectures.DeepSigmoidGlow
+    :members: __init__
+
+.. autoclass:: torchflows.bijections.finite.multiscale.architectures.DenseSigmoidGlow
+    :members: __init__
+
+.. autoclass:: torchflows.bijections.finite.multiscale.architectures.DeepDenseSigmoidGlow
+    :members: __init__
diff --git a/docs/source/architectures/index.rst b/docs/source/architectures/index.rst
@@ -0,0 +1,187 @@
+Full list of architectures (presets)
+=====================================================
+
+We list all implemented NF architectures and their respective class names below.
+Using these presets facilitates experimentation and modeling, however you can also modify each architecture and build new ones.
+
+.. _autoregressive_architecture_list:
+
+Autoregressive architectures
+-----------------------------
+
+We provide the list of autoregressive architectures in the table below.
+Click the architecture name to see the API and usage examples.
+Check the API for all autoregressive architectures :ref:`here <autoregressive_architecture_api>`.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Architecture
+     - Reference
+   * - :class:`NICE <torchflows.bijections.finite.autoregressive.architectures.NICE>`
+     - Dinh et al. `NICE: Non-linear Independent Components Estimation <http://arxiv.org/abs/1410.8516>`_ (2015)
+   * - :class:`RealNVP <torchflows.bijections.finite.autoregressive.architectures.RealNVP>`
+     - Dinh et al. `Density estimation using Real NVP <http://arxiv.org/abs/1605.08803>`_ (2017)
+   * - :class:`Inverse RealNVP <torchflows.bijections.finite.autoregressive.architectures.InverseRealNVP>`
+     - Dinh et al. `Density estimation using Real NVP <http://arxiv.org/abs/1605.08803>`_ (2017)
+   * - :class:`MAF <torchflows.bijections.finite.autoregressive.architectures.MAF>`
+     - Papamakarios et al. `Masked Autoregressive Flow for Density Estimation <http://arxiv.org/abs/1705.07057>`_ (2018)
+   * - :class:`IAF <torchflows.bijections.finite.autoregressive.architectures.IAF>`
+     - Kingma et al. `Improving Variational Inference with Inverse Autoregressive Flow <http://arxiv.org/abs/1606.04934>`_ (2017)
+   * - :class:`Coupling RQ-NSF <torchflows.bijections.finite.autoregressive.architectures.CouplingRQNSF>`
+     - Durkan et al. `Neural Spline Flows <http://arxiv.org/abs/1906.04032>`_ (2019)
+   * - :class:`Masked autoregressive RQ-NSF <torchflows.bijections.finite.autoregressive.architectures.MaskedAutoregressiveRQNSF>`
+     - Durkan et al. `Neural Spline Flows <http://arxiv.org/abs/1906.04032>`_ (2019)
+   * - :class:`Inverse autoregressive RQ-NSF <torchflows.bijections.finite.autoregressive.architectures.InverseAutoregressiveRQNSF>`
+     - Durkan et al. `Neural Spline Flows <http://arxiv.org/abs/1906.04032>`_ (2019)
+   * - :class:`Coupling LR-NSF <torchflows.bijections.finite.autoregressive.architectures.CouplingLRS>`
+     - Dolatabadi et al. `Invertible Generative Modeling using Linear Rational Splines <http://arxiv.org/abs/2001.05168>`_ (2020)
+   * - :class:`Masked autoregressive LR-NSF <torchflows.bijections.finite.autoregressive.architectures.MaskedAutoregressiveLRS>`
+     - Dolatabadi et al. `Invertible Generative Modeling using Linear Rational Splines <http://arxiv.org/abs/2001.05168>`_ (2020)
+   * - :class:`Inverse autoregressive LR-NSF <torchflows.bijections.finite.autoregressive.architectures.InverseAutoregressiveLRS>`
+     - Dolatabadi et al. `Invertible Generative Modeling using Linear Rational Splines <http://arxiv.org/abs/2001.05168>`_ (2020)
+   * - :class:`Coupling deep SF <torchflows.bijections.finite.autoregressive.architectures.CouplingDeepSF>`
+     -
+   * - :class:`Masked autoregressive deep SF <torchflows.bijections.finite.autoregressive.architectures.MaskedAutoregressiveDeepSF>`
+     -
+   * - :class:`Inverse autoregressive deep SF <torchflows.bijections.finite.autoregressive.architectures.InverseAutoregressiveDeepSF>`
+     -
+   * - :class:`Coupling dense SF <torchflows.bijections.finite.autoregressive.architectures.CouplingDenseSF>`
+     -
+   * - :class:`Masked autoregressive dense SF <torchflows.bijections.finite.autoregressive.architectures.MaskedAutoregressiveDenseSF>`
+     -
+   * - :class:`Inverse autoregressive dense SF <torchflows.bijections.finite.autoregressive.architectures.InverseAutoregressiveDenseSF>`
+     -
+   * - :class:`Coupling deep-dense SF <torchflows.bijections.finite.autoregressive.architectures.CouplingDeepDenseSF>`
+     -
+   * - :class:`Masked autoregressive deep-dense SF <torchflows.bijections.finite.autoregressive.architectures.MaskedAutoregressiveDeepDenseSF>`
+     -
+   * - :class:`Inverse autoregressive deep-dense SF <torchflows.bijections.finite.autoregressive.architectures.InverseAutoregressiveDeepDenseSF>`
+     -
+   * - :class:`Unconstrained monotonic neural network <torchflows.bijections.finite.autoregressive.architectures.UMNNMAF>`
+     -
+
+.. _multiscale_architecture_list:
+
+Multiscale architectures
+-----------------------------------------
+We provide the list of multiscale autoregressive architectures in the table below.
+These architectures are specifically made for image modeling, but can also be used for voxels or tensors with more dimensions.
+Click the architecture name to see the API and usage examples.
+Check the API for all multiscale architectures :ref:`here <multiscale_architecture_api>`.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Architecture
+     - Reference
+   * - :class:`MultiscaleNICE <torchflows.bijections.finite.multiscale.architectures.MultiscaleNICE>`
+     - Dinh et al. `NICE: Non-linear Independent Components Estimation <http://arxiv.org/abs/1410.8516>`_ (2015)
+   * - :class:`Multiscale RealNVP <torchflows.bijections.finite.multiscale.architectures.MultiscaleRealNVP>`
+     - Dinh et al. `Density estimation using Real NVP <http://arxiv.org/abs/1605.08803>`_ (2017)
+   * - :class:`Multiscale RQ-NSF <torchflows.bijections.finite.multiscale.architectures.MultiscaleRQNSF>`
+     - Durkan et al. `Neural Spline Flows <http://arxiv.org/abs/1906.04032>`_ (2019)
+   * - :class:`Multiscale LR-NSF <torchflows.bijections.finite.multiscale.architectures.MultiscaleLRSNSF>`
+     - Dolatabadi et al. `Invertible Generative Modeling using Linear Rational Splines <http://arxiv.org/abs/2001.05168>`_ (2020)
+   * - :class:`Multiscale deep SF <torchflows.bijections.finite.multiscale.architectures.MultiscaleDeepSigmoid>`
+     -
+   * - :class:`Multiscale dense SF <torchflows.bijections.finite.multiscale.architectures.MultiscaleDenseSigmoid>`
+     -
+   * - :class:`Multiscale deep-dense SF <torchflows.bijections.finite.multiscale.architectures.MultiscaleDeepDenseSigmoid>`
+     -
+   * - :class:`Shift Glow <torchflows.bijections.finite.multiscale.architectures.ShiftGlow>`
+     -
+   * - :class:`Affine Glow <torchflows.bijections.finite.multiscale.architectures.AffineGlow>`
+     -
+   * - :class:`RQS Glow <torchflows.bijections.finite.multiscale.architectures.RQSGlow>`
+     -
+   * - :class:`LRS Glow <torchflows.bijections.finite.multiscale.architectures.LRSGlow>`
+     -
+   * - :class:`Deep sigmoidal Glow <torchflows.bijections.finite.multiscale.architectures.DeepSigmoidGlow>`
+     -
+   * - :class:`Dense sigmoidal Glow <torchflows.bijections.finite.multiscale.architectures.DenseSigmoidGlow>`
+     -
+   * - :class:`Deep-dense sigmoidal Glow <torchflows.bijections.finite.multiscale.architectures.DeepDenseSigmoidGlow>`
+     -
+
+Residual architectures
+----------------------------
+We provide the list of iterative residual architectures in the table below.
+Click the architecture name to see the API and usage examples.
+Check the API for all residual architectures :ref:`here <residual_architecture_api>`.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Architecture
+     - Reference
+   * - :class:`Invertible ResNet <torchflows.bijections.finite.residual.architectures.InvertibleResNet>`
+     -
+   * - :class:`ResFlow <torchflows.bijections.finite.residual.architectures.ResFlow>`
+     -
+   * - :class:`ProximalResFlow <torchflows.bijections.finite.residual.architectures.ProximalResFlow>`
+     -
+
+We also list presets for some convolutional iterative residual architectures in the table below.
+These are suitable for image modeling.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Architecture
+     - Reference
+   * - :class:`Convolutional invertible ResNet <torchflows.bijections.finite.residual.architectures.ConvolutionalInvertibleResNet>`
+     -
+   * - :class:`Convolutional ResFlow <torchflows.bijections.finite.residual.architectures.ConvolutionalResFlow>`
+     -
+
+We finally list presets for residual architectures, based on the matrix determinant lemma.
+These support either forward or inverse transformation, but not both.
+This means they can be used for either sampling (and variational inference) or density estimation (and maximum likelihood fits), but not both at the same time.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Architecture
+     - Reference
+   * - :class:`Planar flow <torchflows.bijections.finite.residual.architectures.PlanarFlow>`
+     -
+   * - :class:`Radial flow <torchflows.bijections.finite.residual.architectures.RadialFlow>`
+     -
+   * - :class:`Sylvester flow <torchflows.bijections.finite.residual.architectures.SylvesterFlow>`
+     -
+
+Continuous architectures
+----------------------------
+We provide the list of continuous architectures in the table below.
+Click the architecture name to see the API and usage examples.
+Check the API for all continuous architectures :ref:`here <continuous_architecture_api>`.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Architecture
+     - Reference
+   * - :class:`DDNF <torchflows.bijections.finite.continuous.ddnf.DeepDiffeomorphicBijection>`
+     -
+   * - :class:`FFJORD <torchflows.bijections.finite.continuous.ffjord.FFJORD>`
+     -
+   * - :class:`RNODE <torchflows.bijections.finite.continuous.rnode.RNODE>`
+     -
+   * - :class:`OT-Flow <torchflows.bijections.finite.continuous.otflow.OTFlow>`
+     -
+
+We also list presets for convolutional continuous architectures in the table below.
+These are suitable for image modeling.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Architecture
+     - Reference
+   * - :class:`Convolutional DDNF <torchflows.bijections.finite.continuous.ddnf.ConvolutionalDeepDiffeomorphicBijection>`
+     -
+   * - :class:`Convolutional FFJORD <torchflows.bijections.finite.continuous.ffjord.ConvolutionalFFJORD>`
+     -
+   * - :class:`Convolutional RNODE <torchflows.bijections.finite.continuous.rnode.ConvolutionalRNODE>`
+     -