diff --git a/_images/examples_3_time_dependent_1_0.png b/_images/examples_3_time_dependent_1_0.png new file mode 100644 index 0000000..5ae41a1 Binary files /dev/null and b/_images/examples_3_time_dependent_1_0.png differ diff --git a/_images/examples_3_time_dependent_1_1.png b/_images/examples_3_time_dependent_1_1.png new file mode 100644 index 0000000..16ea9a3 Binary files /dev/null and b/_images/examples_3_time_dependent_1_1.png differ diff --git a/_images/examples_4_riesz_maps_1_1.png b/_images/examples_4_riesz_maps_1_1.png new file mode 100644 index 0000000..9f046f1 Binary files /dev/null and b/_images/examples_4_riesz_maps_1_1.png differ diff --git a/_images/examples_4_riesz_maps_3_1.png b/_images/examples_4_riesz_maps_3_1.png new file mode 100644 index 0000000..9f046f1 Binary files /dev/null and b/_images/examples_4_riesz_maps_3_1.png differ diff --git a/_images/examples_4_riesz_maps_3_2.png b/_images/examples_4_riesz_maps_3_2.png new file mode 100644 index 0000000..8b9fd32 Binary files /dev/null and b/_images/examples_4_riesz_maps_3_2.png differ diff --git a/_images/examples_4_riesz_maps_5_1.png b/_images/examples_4_riesz_maps_5_1.png new file mode 100644 index 0000000..9f046f1 Binary files /dev/null and b/_images/examples_4_riesz_maps_5_1.png differ diff --git a/_images/examples_4_riesz_maps_7_0.png b/_images/examples_4_riesz_maps_7_0.png new file mode 100644 index 0000000..40369da Binary files /dev/null and b/_images/examples_4_riesz_maps_7_0.png differ diff --git a/_images/examples_5_optimization_3_1.png b/_images/examples_5_optimization_3_1.png new file mode 100644 index 0000000..95a8cfd Binary files /dev/null and b/_images/examples_5_optimization_3_1.png differ diff --git a/_images/examples_5_optimization_3_2.png b/_images/examples_5_optimization_3_2.png new file mode 100644 index 0000000..31ac0a8 Binary files /dev/null and b/_images/examples_5_optimization_3_2.png differ diff --git a/_images/examples_6_custom_operations_1_0.png b/_images/examples_6_custom_operations_1_0.png new file mode 100644 index 0000000..e7fb2f7 Binary files /dev/null and b/_images/examples_6_custom_operations_1_0.png differ diff --git a/_images/examples_6_custom_operations_3_1.png b/_images/examples_6_custom_operations_3_1.png new file mode 100644 index 0000000..e7fb2f7 Binary files /dev/null and b/_images/examples_6_custom_operations_3_1.png differ diff --git a/_images/examples_6_custom_operations_3_2.png b/_images/examples_6_custom_operations_3_2.png new file mode 100644 index 0000000..b1558cd Binary files /dev/null and b/_images/examples_6_custom_operations_3_2.png differ diff --git a/_images/examples_7_jax_integration_1_2.png b/_images/examples_7_jax_integration_1_2.png new file mode 100644 index 0000000..bc018d3 Binary files /dev/null and b/_images/examples_7_jax_integration_1_2.png differ diff --git a/_images/examples_7_jax_integration_1_3.png b/_images/examples_7_jax_integration_1_3.png new file mode 100644 index 0000000..8ace9fd Binary files /dev/null and b/_images/examples_7_jax_integration_1_3.png differ diff --git a/_sources/acknowledgements.rst.txt b/_sources/acknowledgements.rst.txt new file mode 100644 index 0000000..f3e0084 --- /dev/null +++ b/_sources/acknowledgements.rst.txt @@ -0,0 +1,173 @@ +References and acknowledgements +=============================== + +Citing tlm_adjoint +------------------ + +tlm_adjoint is described in + +- James R. Maddison, Daniel N. Goldberg, and Benjamin D. Goddard, 'Automated + calculation of higher order partial differential equation constrained + derivative information', SIAM Journal on Scientific Computing, 41(5), pp. + C417--C445, 2019, doi: 10.1137/18M1209465 + +The automated assembly and linear solver caching applied by tlm_adjoint is +based on the approach described in + +- J. R. Maddison and P. E. Farrell, 'Rapid development and adjoining of + transient finite element models', Computer Methods in Applied Mechanics and + Engineering, 276, 95--121, 2014, doi: 10.1016/j.cma.2014.03.010 + +Checkpointing with tlm_adjoint, and mixed forward restart / non-linear +dependency data schedules defined by the code in +`tlm_adjoint/checkpoint_schedules/mixed.py +`_, are described in + +- James R. Maddison, 'On the implementation of checkpointing with high-level + algorithmic differentiation', https://arxiv.org/abs/2305.09568v1, 2023 + +References +---------- + +dolfin-adjoint +`````````````` + +tlm_adjoint implements high-level algorithmic differentiation using an +approach based on that used by dolfin-adjoint, described in + +- P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, 'Automated + derivation of the adjoint of high-level transient finite element programs', + SIAM Journal on Scientific Computing 35(4), pp. C369--C393, 2013, + doi: 10.1137/120873558 + +tlm_adjoint was developed from a custom extension to dolfin-adjoint. + +Taylor remainder convergence testing +```````````````````````````````````` + +The functions in `tlm_adjoint/verification.py +`_ implement Taylor remainder +convergence testing using the approach described in + +- P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, 'Automated + derivation of the adjoint of high-level transient finite element programs', + SIAM Journal on Scientific Computing 35(4), pp. C369--C393, 2013, + doi: 10.1137/120873558 + +Solving eigenproblems with SLEPc +```````````````````````````````` + +The `eigendecompose` function in `tlm_adjoint/eigendecomposition.py +`_ was originally developed +by loosely following the slepc4py 3.6.0 demo demo/ex3.py. slepc4py 3.6.0 +license information follows. + +.. code-block:: text + + ========================= + LICENSE: SLEPc for Python + ========================= + + :Author: Lisandro Dalcin + :Contact: dalcinl@gmail.com + + + Copyright (c) 2015, Lisandro Dalcin. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Differentiating fixed-point problems +```````````````````````````````````` + +The `FixedPointSolver` class in `tlm_adjoint/fixed_point.py +`_ derives tangent-linear and +adjoint information using the approach described in + +- Jean Charles Gilbert, 'Automatic differentiation and iterative processes', + Optimization Methods and Software, 1(1), pp. 13--21, 1992, + doi: 10.1080/10556789208805503 +- Bruce Christianson, 'Reverse accumulation and attractive fixed points', + Optimization Methods and Software, 3(4), pp. 311--326, 1994, + doi: 10.1080/10556789408805572 + +Binomial checkpointing +`````````````````````` + +The `MultistageCheckpointSchedule` class in +`tlm_adjoint/checkpoint_schedules/binomial.py +`_ implements the +binomial checkpointing strategy described in + +- Andreas Griewank and Andrea Walther, 'Algorithm 799: revolve: an + implementation of checkpointing for the reverse or adjoint mode of + computational differentiation', ACM Transactions on Mathematical Software, + 26(1), pp. 19--45, 2000, doi: 10.1145/347837.347846 + +The `MultistageCheckpointSchedule` class determines a memory/disk storage +distribution using an initial run of the checkpoint schedule, leading to a +distribution equivalent to that in + +- Philipp Stumm and Andrea Walther, 'MultiStage approaches for optimal offline + checkpointing', SIAM Journal on Scientific Computing, 31(3), pp. 1946--1967, + 2009, doi: 10.1137/080718036 + +The `TwoLevelCheckpointSchedule` class in +`tlm_adjoint/checkpoint_schedules/binomial.py +`_ implements the +two-level mixed periodic/binomial checkpointing approach described in + +- Gavin J. Pringle, Daniel C. Jones, Sudipta Goswami, Sri Hari Krishna + Narayanan, and Daniel Goldberg, 'Providing the ARCHER community with adjoint + modelling tools for high-performance oceanographic and cryospheric + computation', version 1.1, EPCC, 2016 + +and in the supporting information for + +- D. N. Goldberg, T. A. Smith, S. H. K. Narayanan, P. Heimbach, and M. + Morlighem,, 'Bathymetric influences on Antarctic ice-shelf melt rates', + Journal of Geophysical Research: Oceans, 125(11), e2020JC016370, 2020, + doi: 10.1029/2020JC016370 + +L-BFGS +`````` + +The file `tlm_adjoint/optimization.py +`_ includes an implementation of +the L-BFGS algorithm, described in + +- Jorge Nocedal and Stephen J. Wright, 'Numerical optimization', Springer, New + York, NY, 2006, Second edition, doi: 10.1007/978-0-387-40065-5 +- Richard H. Byrd, Peihuang Lu, and Jorge Nocedal, and Ciyou Zhu, 'A limited + memory algorithm for bound constrained optimization', SIAM Journal on + Scientific Computing, 16(5), 1190--1208, 1995, doi: 10.1137/0916069 + +Funding +------- + +Early development work leading to tlm_adjoint was conducted as part of a U.K. +Natural Environment Research Council funded project (NE/L005166/1). Further +development has been conducted as part of a U.K. Engineering and Physical +Sciences Research Council funded project (EP/R021600/1) and a Natural +Environment Research Council funded project (NE/T001607/1). diff --git a/_sources/autoapi/tlm_adjoint/adjoint/index.rst.txt b/_sources/autoapi/tlm_adjoint/adjoint/index.rst.txt new file mode 100644 index 0000000..e99e75e --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/adjoint/index.rst.txt @@ -0,0 +1,197 @@ +:orphan: + +:py:mod:`tlm_adjoint.adjoint` +============================= + +.. py:module:: tlm_adjoint.adjoint + + +Module Contents +--------------- + +.. py:class:: AdjointRHS(x) + + + The right-hand-side of an adjoint equation, for an adjoint variable + associated with an equation solving for a forward variable `x`. + + :arg x: The forward variable. + + .. py:method:: b(*, copy=False) + + Return the right-hand-side, as a variable. + + :arg copy: If `True` then a copy of the internal variable storing the + right-hand-side value is returned. If `False` the internal variable + itself is returned. + :returns: A variable storing the right-hand-side value. + + + .. py:method:: initialize() + + Allocate an internal variable to store the right-hand-side. + Typically need not be called manually. + + + .. py:method:: sub(b) + + Subtract a term from the right-hand-side. + + :arg b: The term to subtract. + :func:`.subtract_adjoint_derivative_action` is used to subtract the + term. + + + .. py:method:: is_empty() + + Return whether the right-hand-side is 'empty', meaning that the + :meth:`.AdjointRHS.initialize` method has not been called. + + :returns: `True` if the :meth:`.AdjointRHS.initialize` method has not + been called, and `False` otherwise. + + + +.. py:class:: AdjointEquationRHS(eq) + + + The right-hand-side of an adjoint equation, for adjoint variables + associated with an equation solving for multiple forward variables `X`. + + Multiple :class:`.AdjointRHS` objects. The :class:`.AdjointRHS` objects may + be accessed by index, e.g. + + .. code-block:: python + + adj_eq_rhs = AdjointEquationRHS(eq) + adj_rhs = adj_eq_rhs[m] + + :arg eq: An :class:`.Equation`. `eq.X()` defines the forward variables. + + .. py:method:: b(*, copy=False) + + For the case where there is a single forward variable, return a + variable associated with the right-hand-side. + + :arg copy: If `True` then a copy of the internal variable storing the + right-hand-side value is returned. If `False` the internal variable + itself is returned. + :returns: A variable storing the right-hand-side value. + + + .. py:method:: B(*, copy=False) + + Return variables associated with the right-hand-sides. + + :arg copy: If `True` then copies of the internal variables storing the + right-hand-side values are returned. If `False` the internal + variables themselves are returned. + :returns: A :class:`tuple` of variables storing the right-hand-side + values. + + + .. py:method:: is_empty() + + Return whether all of the :class:`.AdjointRHS` objects are 'empty', + meaning that the :meth:`.AdjointRHS.initialize` method has not been + called for any :class:`.AdjointRHS`. + + :returns: `True` if the :meth:`.AdjointRHS.initialize` method has not + been called for any :class:`.AdjointRHS`, and `False` otherwise. + + + +.. py:class:: AdjointBlockRHS(block) + + + The right-hand-side of multiple adjoint equations. + + Multiple :class:`.AdjointEquationRHS` objects. The + :class:`.AdjointEquationRHS` objects may be accessed by index, e.g. + + .. code-block:: python + + adj_block_rhs = AdjointBlockRHS(block) + adj_eq_rhs = adj_block_rhs[k] + + :class:`.AdjointRHS` objects may be accessed e.g. + + .. code-block:: python + + adj_rhs = adj_block_rhs[(k, m)] + + :arg block: A :class:`Sequence` of :class:`.Equation` objects. + + .. py:method:: pop() + + Remove and return the last :class:`.AdjointEquationRHS` in the + :class:`.AdjointBlockRHS`. + + :returns: A :class:`tuple` `(n, B)`. `B` is the removed + :class:`.AdjointEquationRHS`, associated with block `n`. + + + .. py:method:: is_empty() + + Return whether there are no :class:`.AdjointEquationRHS` objects in + the :class:`.AdjointBlockRHS`. + + :returns: `True` if there are no :class:`.AdjointEquationRHS` objects + in the :class:`.AdjointBlockRHS`, and `False` otherwise. + + + +.. py:class:: AdjointModelRHS(blocks) + + + The right-hand-side of multiple blocks of adjoint equations. + + Multiple :class:`.AdjointBlockRHS` objects. The :class:`.AdjointBlockRHS` + objects may be accessed by index, e.g. + + .. code-block:: python + + adj_model_rhs = AdjointModelRHS(block) + adj_block_rhs = adj_block_rhs[p] + + :class:`.AdjointEquationRHS` objects may be accessed e.g. + + .. code-block:: python + + adj_eq_rhs = adj_block_rhs[(p, k)] + + :class:`.AdjointRHS` objects may be accessed e.g. + + .. code-block:: python + + adj_rhs = adj_block_rhs[(p, k, m)] + + If the last block of adjoint equations contains no equations then it is + automatically removed from the :class:`.AdjointModelRHS`. + + :arg blocks: A :class:`Sequence` of :class:`Sequence` objects each + containing :class:`.Equation` objects, or a :class:`Mapping` with items + `(index, block)` where `index` is an :class:`int` and `block` a + :class:`Sequence` of :class:`.Equation` objects. In the latter case + blocks are ordered by `index`. + + .. py:method:: pop() + + Remove and return the last :class:`.AdjointEquationRHS` in the last + :class:`.AdjointBlockRHS` in the :class:`.AdjointModelRHS`. + + :returns: A :class:`tuple` `((n, i), B)`. `B` is the removed + :class:`.AdjointEquationRHS`, associated with equation `i` in block + `n`. + + + .. py:method:: is_empty() + + Return whether there are no :class:`.AdjointBlockRHS` objects in the + :class:`.AdjointModelRHS`. + + :returns: `True` if there are no :class:`.AdjointBlockRHS` objects in + the :class:`.AdjointModelRHS`, and `False` otherwise. + + + diff --git a/_sources/autoapi/tlm_adjoint/alias/index.rst.txt b/_sources/autoapi/tlm_adjoint/alias/index.rst.txt new file mode 100644 index 0000000..6d0fbb8 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/alias/index.rst.txt @@ -0,0 +1,41 @@ +:orphan: + +:py:mod:`tlm_adjoint.alias` +=========================== + +.. py:module:: tlm_adjoint.alias + + +Module Contents +--------------- + +.. py:function:: gc_disabled(fn) + + Decorator to disable the Python garbage collector. + + :arg fn: A callable for which the Python garbage collector should be + disabled. + :returns: A callable for which the Python garbage collector is disabled. + + +.. py:class:: Alias(obj) + + + An alias to an object. Holds a reference to the original object. + + :arg obj: Object to alias. + + +.. py:class:: WeakAlias(obj) + + + An alias to an object. Does *not* hold a reference to the original + object. + + Intended to be used in combination with `weakref.finalize`, so that object + attributes may be updated when the original object is destroyed, but object + methods may still be called after it is destroyed. + + :arg obj: Object to alias. + + diff --git a/_sources/autoapi/tlm_adjoint/cached_hessian/index.rst.txt b/_sources/autoapi/tlm_adjoint/cached_hessian/index.rst.txt new file mode 100644 index 0000000..79975e0 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/cached_hessian/index.rst.txt @@ -0,0 +1,101 @@ +:orphan: + +:py:mod:`tlm_adjoint.cached_hessian` +==================================== + +.. py:module:: tlm_adjoint.cached_hessian + + +Module Contents +--------------- + +.. py:class:: CachedHessian(J, *, manager=None, cache_adjoint=True) + + + + + Represents a Hessian associated with a given forward. Uses a cached + forward calculation. + + :arg J: A variable defining the Hessian. + :arg manager: The :class:`.EquationManager` used to record the forward. + This must have used `'memory'` checkpointing with automatic dropping of + variable references disabled. `manager()` is used if not supplied. + :arg cache_adjoint: Whether to cache the first order adjoint calculation. + + .. py:method:: compute_gradient(M, M0=None) + + Compute the (conjugate of the) derivative of a functional with + respect to a control using an adjoint. + + :arg M: A variable or a :class:`Sequence` of variables defining the + control. + :arg M0: A variable or a :class:`Sequence` of variables defining the + control value. `M` is used if not supplied. + :returns: The (conjugate of the) derivative. A variable or + :class:`Sequence` of variables, depending on the type of `M`. + + + .. py:method:: action(M, dM, M0=None) + + Compute (the conjugate of) a Hessian action on some :math:`\zeta` + using an adjoint of a tangent-linear. i.e. considering derivatives to + be row vectors, compute + + .. math:: + + \left( \frac{d}{dm} \left[ + \frac{d \mathcal{J}}{d m} \zeta \right] \right)^{*,T}. + + :arg M: A variable or a :class:`Sequence` of variables defining the + control. + :arg dM: A variable or a :class:`Sequence` of variables defining + :math:`\zeta`. The (conjugate of the) Hessian action on + :math:`\zeta` is computed. + :arg M0: A variable or a :class:`Sequence` of variables defining the + control value. `M` is used if not supplied. + :returns: A tuple `(J, dJ, ddJ)`. `J` is the value of the functional. + `dJ` is the value of :math:`\left( d \mathcal{J} / d m \right) + \zeta`. `ddJ` stores the (conjugate of the) result of the Hessian + action on :math:`\zeta`, and is a variable or a :class:`Sequence` + of variables depending on the type of `M`. + + + +.. py:class:: CachedGaussNewton(X, R_inv_action, B_inv_action=None, *, manager=None) + + + + + Represents a Gauss-Newton approximation to a Hessian associated with a + given forward. Uses a cached forward calculation. + + :arg X: A variable or a :class:`Sequence` of variables defining the state. + :arg R_inv_action: See :class:`.GaussNewton`. + :arg B_inv_action: See :class:`.GaussNewton`. + :arg manager: The :class:`.EquationManager` used to record the forward. + This must have used `'memory'` checkpointing with automatic dropping of + variable references disabled. `manager()` is used if not supplied. + + .. py:method:: action(M, dM, M0=None) + + Compute (the conjugate of) a Hessian action on some :math:`\zeta`, + using the Gauss-Newton approximation for the Hessian. i.e. compute + + .. math:: + + \left( H \zeta \right)^{*,T}. + + :arg M: A variable or a :class:`Sequence` of variables defining the + control. + :arg dM: A variable or a :class:`Sequence` of variables defining + :math:`\zeta`. The (conjugate of the) approximated Hessian action + on :math:`\zeta` is computed. + :arg M0: A variable or a :class:`Sequence` of variables defining the + control value. `M` is used if not supplied. + :returns: The (conjugate of the) result of the approximated Hessian + action on :math:`\zeta`. A variable or a :class:`Sequence` of + variables depending on the type of `M`. + + + diff --git a/_sources/autoapi/tlm_adjoint/caches/index.rst.txt b/_sources/autoapi/tlm_adjoint/caches/index.rst.txt new file mode 100644 index 0000000..7fddec4 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/caches/index.rst.txt @@ -0,0 +1,146 @@ +:orphan: + +:py:mod:`tlm_adjoint.caches` +============================ + +.. py:module:: tlm_adjoint.caches + + +Module Contents +--------------- + +.. py:class:: CacheRef(value=None) + + + A cache entry. Stores a reference to a cached value, which can later be + cleared. Calling a :class:`.CacheRef` returns the cached object, or `None` + if no object is referenced. + + :arg value: The object to reference. `None` may be supplied to indicate an + empty cache entry. + + .. py:method:: clear() + + Clear the cache entry. After calling this method, calling the + :class:`.CacheRef` will return `None`. + + + +.. py:function:: clear_caches(*deps) + + Clear caches entries. + + :arg deps: A :class:`Sequence` of variables. If non-empty then clear only + cache entries which depend on the supplied variables. Otherwise clear + all cache entries. + + +.. py:function:: local_caches(fn) + + Decorator clearing caches before and after calling the decorated + callable. + + :arg fn: A callable for which caches should be cleared. + :returns: A callable where caches are cleared before and after calling. + + +.. py:class:: Cache + + + Stores cache entries. + + Cleared cache entries are removed from the :class:`.Cache`. + + .. py:property:: id + + A unique :class:`int` ID associated with this :class:`.Cache`. + + + + .. py:method:: clear(*deps) + + Clear cache entries. + + :arg deps: A :class:`Sequence` of variables. If non-empty then only + clear cache entries which depend on the supplied variables. + Otherwise clear all cache entries. + + + .. py:method:: add(key, value, deps=None) + + Add a cache entry. + + :arg key: The key associated with the cache entry. + :arg value: A callable, taking no arguments, returning the value + associated with the cache entry. Only called to if no entry + associated with `key` exists. + :arg deps: A :class:`Sequence` of variables, defining dependencies of + the cache entry. + :returns: A :class:`tuple` `(value_ref, value)`, where `value` is the + cache entry value and `value_ref` is a :class:`.CacheRef` storing a + reference to the value. + + + .. py:method:: get(key, *args) + + Return the cache entry associated with a given key. + + :arg key: The key. + :returns: The cache entry or, if supplied, a default value. + + `args` should contain zero or one elements and defines the default + value. If there is no entry associated with the key then: + + - If `args` has no elements an exception is raised. + - If `args` has one element then this is returned. + + + +.. py:class:: Caches(x) + + + Multiple :class:`.Cache` objects, associated with a variable. + + Cache entries may depend on the variable. The variable also defines an + initial value, and the value is indicated by the variable ID and variable + state value. The value may be changed either by supplying a new variable + (changing the ID), or by changing the value of the current variable + defining the value (which should be indicated by a change to the variable + state value). Either change invalidates cache entries, in the + :class:`.Cache` objects, which depend on the original variable. + + The :meth:`.Caches.update` method can be used to check for cache entry + invalidation, and to clear invalid cache entries. + + :arg x: The variable defining a possible cache entry dependency, and an + initial value for that dependency. + + .. py:method:: clear() + + Clear cache entries which depend on the associated variable. + + + + .. py:method:: add(cache) + + Add a new :class:`.Cache` to the :class:`.Caches`. + + :arg cache: The :class:`.Cache` to add to the :class:`.Caches`. + + + .. py:method:: remove(cache) + + Remove a :class:`.Cache` from the :class:`.Caches`. + + :arg cache: The :class:`.Cache` to remove from the :class:`.Caches`. + + + .. py:method:: update(x) + + Check for cache invalidation associated with a possible change in + value, and clear invalid cache entries. + + :arg x: A variable which defines a potentially new value. + + + diff --git a/_sources/autoapi/tlm_adjoint/checkpoint_schedules/binomial/index.rst.txt b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/binomial/index.rst.txt new file mode 100644 index 0000000..3571773 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/binomial/index.rst.txt @@ -0,0 +1,129 @@ +:orphan: + +:py:mod:`tlm_adjoint.checkpoint_schedules.binomial` +=================================================== + +.. py:module:: tlm_adjoint.checkpoint_schedules.binomial + + +Module Contents +--------------- + +.. py:class:: MultistageCheckpointSchedule(max_n, snapshots_in_ram, snapshots_on_disk, *, trajectory='maximum') + + + + + A binomial checkpointing schedule using the approach described in + + - Andreas Griewank and Andrea Walther, 'Algorithm 799: revolve: an + implementation of checkpointing for the reverse or adjoint mode of + computational differentiation', ACM Transactions on Mathematical + Software, 26(1), pp. 19--45, 2000, doi: 10.1145/347837.347846 + + hereafter referred to as GW2000. + + Uses a 'MultiStage' distribution of checkpoints between RAM and disk + equivalent to that described in + + - Philipp Stumm and Andrea Walther, 'MultiStage approaches for optimal + offline checkpointing', SIAM Journal on Scientific Computing, 31(3), + pp. 1946--1967, 2009, doi: 10.1137/080718036 + + The distribution between RAM and disk is determined using an initial run of + the schedule. + + Offline, one adjoint calculation permitted. + + :arg max_n: The number of forward steps in the initial forward calculation. + :arg snapshots_in_ram: The maximum number of forward restart checkpoints + to store in memory. + :arg snapshots_on_disk: The maximum number of forward restart checkpoints + to store on disk. + :arg trajectory: When advancing `n` forward steps with `s` checkpointing + units available there are in general multiple solutions to the problem + of determining the number of forward steps to advance before storing + a new forward restart checkpoint -- see Fig. 4 of GW2000. This argument + selects a solution: + + - `'revolve'`: The standard revolve solution, as specified in the + equation at the bottom of p. 34 of GW2000. + - `'maximum'`: The maximum possible number of steps, corresponding + to the maximum step size compatible with the optimal region in + Fig. 4 of GW2000. + + The argument names `snaps_in_ram` and `snaps_on_disk` originate from the + corresponding arguments for the `dolfin_adjoint.solving.adj_checkpointing` + function in dolfin-adjoint (see e.g. version 2017.1.0). + + .. py:property:: is_exhausted + + Whether the schedule has concluded. Note that some schedules permit + multiple adjoint calculation, and may never conclude. + + + .. py:property:: uses_disk_storage + + Whether the schedule may use disk storage. If `False` then no disk + storage is required. + + + .. py:method:: iter() + + A generator which should be overridden in derived classes in order + to define a checkpointing schedule. + + + +.. py:class:: TwoLevelCheckpointSchedule(period, binomial_snapshots, *, binomial_storage='disk', binomial_trajectory='maximum') + + + + + A two-level mixed periodic/binomial checkpointing schedule using the + approach described in + + - Gavin J. Pringle, Daniel C. Jones, Sudipta Goswami, Sri Hari Krishna + Narayanan, and Daniel Goldberg, 'Providing the ARCHER community with + adjoint modelling tools for high-performance oceanographic and + cryospheric computation', version 1.1, EPCC, 2016 + + and in the supporting information for + + - D. N. Goldberg, T. A. Smith, S. H. K. Narayanan, P. Heimbach, and M. + Morlighem, 'Bathymetric influences on Antarctic ice-shelf melt + rates', Journal of Geophysical Research: Oceans, 125(11), + e2020JC016370, 2020, doi: 10.1029/2020JC016370 + + Online, unlimited adjoint calculations permitted. + + :arg period: Forward restart checkpoints are stored to disk every `period` + forward steps in the initial forward calculation. + :arg binomial_snapshots: The maximum number of additional forward restart + checkpointing units to use when advancing the adjoint between periodic + disk checkpoints. + :arg binomial_storage: The storage to use for the additional forward + restart checkpoints generated when advancing the adjoint between + periodic disk checkpoints. Either `'RAM'` or `'disk'`. + :arg binomial_trajectory: See the `trajectory` constructor argument for + :class:`.MultistageCheckpointSchedule`. + + .. py:property:: is_exhausted + + Whether the schedule has concluded. Note that some schedules permit + multiple adjoint calculation, and may never conclude. + + + .. py:property:: uses_disk_storage + + Whether the schedule may use disk storage. If `False` then no disk + storage is required. + + + .. py:method:: iter() + + A generator which should be overridden in derived classes in order + to define a checkpointing schedule. + + + diff --git a/_sources/autoapi/tlm_adjoint/checkpoint_schedules/h_revolve/index.rst.txt b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/h_revolve/index.rst.txt new file mode 100644 index 0000000..f6c0c5e --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/h_revolve/index.rst.txt @@ -0,0 +1,63 @@ +:orphan: + +:py:mod:`tlm_adjoint.checkpoint_schedules.h_revolve` +==================================================== + +.. py:module:: tlm_adjoint.checkpoint_schedules.h_revolve + + +Module Contents +--------------- + +.. py:class:: HRevolveCheckpointSchedule(max_n, snapshots_in_ram, snapshots_on_disk, *, wvect=(0.0, 0.1), rvect=(0.0, 0.1), uf=1.0, ub=2.0, **kwargs) + + + + + An H-Revolve checkpointing schedule. + + Converts from schedules as generated by the H-Revolve library, for the + case of two storage levels: RAM and disk. + + Offline, one adjoint calculation permitted. + + :arg max_n: The number of forward steps in the initial forward calculation. + :arg snapshots_in_ram: The maximum number of forward restart checkpoints + to store in memory. + :arg snapshots_on_disk: The maximum number of forward restart checkpoints + to store on disk. + :arg wvect: A two element :class:`tuple` defining the write cost associated + with saving a forward restart checkpoint to RAM (first element) and + disk (second element). + :arg rvect: A two element :class:`tuple` defining the read cost associated + with loading a forward restart checkpoint from RAM (first element) and + disk (second element). + :arg uf: The cost of advancing the forward one step. + :arg bf: The cost of advancing the forward one step, storing non-linear + dependency data, and then advancing the adjoint over that step. + + Remaining keyword arguments are passed to `hrevolve.hrevolve`. + + The argument names `snaps_in_ram` and `snaps_on_disk` originate from the + corresponding arguments for the `dolfin_adjoint.solving.adj_checkpointing` + function in dolfin-adjoint (see e.g. version 2017.1.0). + + .. py:property:: is_exhausted + + Whether the schedule has concluded. Note that some schedules permit + multiple adjoint calculation, and may never conclude. + + + .. py:property:: uses_disk_storage + + Whether the schedule may use disk storage. If `False` then no disk + storage is required. + + + .. py:method:: iter() + + A generator which should be overridden in derived classes in order + to define a checkpointing schedule. + + + diff --git a/_sources/autoapi/tlm_adjoint/checkpoint_schedules/memory/index.rst.txt b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/memory/index.rst.txt new file mode 100644 index 0000000..1619b81 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/memory/index.rst.txt @@ -0,0 +1,40 @@ +:orphan: + +:py:mod:`tlm_adjoint.checkpoint_schedules.memory` +================================================= + +.. py:module:: tlm_adjoint.checkpoint_schedules.memory + + +Module Contents +--------------- + +.. py:class:: MemoryCheckpointSchedule(max_n=None) + + + + + A checkpointing schedule where all forward restart and non-linear + dependency data are stored in memory. + + Online, unlimited adjoint calculations permitted. + + .. py:property:: is_exhausted + + Whether the schedule has concluded. Note that some schedules permit + multiple adjoint calculation, and may never conclude. + + + .. py:property:: uses_disk_storage + + Whether the schedule may use disk storage. If `False` then no disk + storage is required. + + + .. py:method:: iter() + + A generator which should be overridden in derived classes in order + to define a checkpointing schedule. + + + diff --git a/_sources/autoapi/tlm_adjoint/checkpoint_schedules/mixed/index.rst.txt b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/mixed/index.rst.txt new file mode 100644 index 0000000..316b80a --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/mixed/index.rst.txt @@ -0,0 +1,53 @@ +:orphan: + +:py:mod:`tlm_adjoint.checkpoint_schedules.mixed` +================================================ + +.. py:module:: tlm_adjoint.checkpoint_schedules.mixed + + +Module Contents +--------------- + +.. py:class:: MixedCheckpointSchedule(max_n, snapshots, *, storage='disk') + + + + + A checkpointing schedule which mixes storage of forward restart data and + non-linear dependency data in checkpointing units. Assumes that the data + required to restart the forward has the same size as the data required to + advance the adjoint over a step. + + Described in + + - James R. Maddison, 'On the implementation of checkpointing with + high-level algorithmic differentiation', + https://arxiv.org/abs/2305.09568v1, 2023 + + Offline, one adjoint calculation permitted. + + :arg max_n: The number of forward steps in the initial forward calculation. + :arg snapshots: The number of available checkpointing units. + :arg storage: Checkpointing unit storage location. Either `'RAM'` or + `'disk'`. + + .. py:property:: is_exhausted + + Whether the schedule has concluded. Note that some schedules permit + multiple adjoint calculation, and may never conclude. + + + .. py:property:: uses_disk_storage + + Whether the schedule may use disk storage. If `False` then no disk + storage is required. + + + .. py:method:: iter() + + A generator which should be overridden in derived classes in order + to define a checkpointing schedule. + + + diff --git a/_sources/autoapi/tlm_adjoint/checkpoint_schedules/none/index.rst.txt b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/none/index.rst.txt new file mode 100644 index 0000000..a60580d --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/none/index.rst.txt @@ -0,0 +1,40 @@ +:orphan: + +:py:mod:`tlm_adjoint.checkpoint_schedules.none` +=============================================== + +.. py:module:: tlm_adjoint.checkpoint_schedules.none + + +Module Contents +--------------- + +.. py:class:: NoneCheckpointSchedule + + + + + A checkpointing schedule for the case where no adjoint calculation is + performed. + + Online, zero adjoint calculations permitted. + + .. py:property:: is_exhausted + + Whether the schedule has concluded. Note that some schedules permit + multiple adjoint calculation, and may never conclude. + + + .. py:property:: uses_disk_storage + + Whether the schedule may use disk storage. If `False` then no disk + storage is required. + + + .. py:method:: iter() + + A generator which should be overridden in derived classes in order + to define a checkpointing schedule. + + + diff --git a/_sources/autoapi/tlm_adjoint/checkpoint_schedules/periodic/index.rst.txt b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/periodic/index.rst.txt new file mode 100644 index 0000000..182b46d --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/periodic/index.rst.txt @@ -0,0 +1,46 @@ +:orphan: + +:py:mod:`tlm_adjoint.checkpoint_schedules.periodic` +=================================================== + +.. py:module:: tlm_adjoint.checkpoint_schedules.periodic + + +Module Contents +--------------- + +.. py:class:: PeriodicDiskCheckpointSchedule(period) + + + + + A checkpointing schedule where forward restart checkpoints are stored + periodically to disk. Non-linear dependency data is recomputed for use by + the adjoint by re-running the forward from these checkpoints. If the + storage period is greater than one then non-linear dependency data for + multiple steps is recomputed and stored when advancing the adjoint. + + Online, unlimited adjoint calculations permitted. + + :arg period: Forward restart checkpoints are stored to disk every `period` + forward steps in the initial forward calculation. + + .. py:property:: is_exhausted + + Whether the schedule has concluded. Note that some schedules permit + multiple adjoint calculation, and may never conclude. + + + .. py:property:: uses_disk_storage + + Whether the schedule may use disk storage. If `False` then no disk + storage is required. + + + .. py:method:: iter() + + A generator which should be overridden in derived classes in order + to define a checkpointing schedule. + + + diff --git a/_sources/autoapi/tlm_adjoint/checkpoint_schedules/schedule/index.rst.txt b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/schedule/index.rst.txt new file mode 100644 index 0000000..63dc312 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/checkpoint_schedules/schedule/index.rst.txt @@ -0,0 +1,291 @@ +:orphan: + +:py:mod:`tlm_adjoint.checkpoint_schedules.schedule` +=================================================== + +.. py:module:: tlm_adjoint.checkpoint_schedules.schedule + + +Module Contents +--------------- + +.. py:class:: CheckpointAction(*args) + + + A checkpointing action. + + + .. py:property:: args + + Action parameters. + + + +.. py:class:: Clear(clear_ics, clear_data) + + + + + A checkpointing action which clears the intermediate storage. + + :arg clear_ics: Whether to clear stored forward restart data. + :arg clear_data: Whether to clear stored non-linear dependency data. + + .. py:property:: clear_ics + + Whether to clear stored forward restart data. + + + .. py:property:: clear_data + + Whether to clear stored non-linear dependency data. + + + +.. py:class:: Configure(store_ics, store_data) + + + + + A checkpointing action which configures the intermediate storage. + + :arg store_ics: Whether to store forward restart data. + :arg store_data: Whether to store non-linear dependency data. + + .. py:property:: store_ics + + Whether to store forward restart data. + + + .. py:property:: store_data + + Whether to store non-linear dependency data. + + + +.. py:class:: Forward(n0, n1) + + + + + A checkpointing action which indicates forward advancement. + + :arg n0: The forward should advance from the start of this step. + :arg n1: The forward should advance to the start of this step. + + .. py:property:: n0 + + The forward should advance from the start of this step. + + + .. py:property:: n1 + + The forward should advance to the start of this step. + + + +.. py:class:: Reverse(n1, n0) + + + + + A checkpointing action which indicates adjoint advancement. + + :arg n1: The adjoint should advance from the start of this step. + :arg n0: The adjoint should advance to the start of this step. + + .. py:property:: n0 + + The adjoint should advance to the start of this step. + + + .. py:property:: n1 + + The adjoint should advance from the start of this step. + + + +.. py:class:: Read(n, storage, delete) + + + + + A checkpointing action which indicates loading of data from a + checkpointing unit. + + :arg n: The step with which the loaded data is associated. + :arg storage: The storage from which the data should be loaded. Either + `'RAM'` or `'disk'`. + :arg delete: Whether the data should be deleted from the indicated storage + after it has been loaded. + + .. py:property:: n + + The step with which the loaded data is associated. + + + .. py:property:: storage + + The storage from which the data should be loaded. Either `'RAM'` or + `'disk'`. + + + .. py:property:: delete + + Whether the data should be deleted from the indicated storage after + it has been loaded. + + + +.. py:class:: Write(n, storage) + + + + + A checkpointing action which indicates saving of data to a checkpointing + unit. + + :arg n: The step with which the saved data is associated. + :arg storage: The storage to which the data should be saved. Either `'RAM'` + or `'disk'`. + + .. py:property:: n + + The step with which the saved data is associated. + + + .. py:property:: storage + + The storage to which the data should be saved. Either `'RAM'` or + `'disk'`. + + + +.. py:class:: EndForward + + + + + A checkpointing action which indicates the end of the initial forward + calculation. + + +.. py:class:: EndReverse(exhausted) + + + + + A checkpointing action which indicates the end of an adjoint + calculation. + + :arg exhausted: Indicates whether the schedule has concluded. If `True` + then this action should be the last action in the schedule. + + .. py:property:: exhausted + + Indicates whether the schedule has concluded. If `True` then this + action should be the last action in the schedule. + + + +.. py:class:: CheckpointSchedule(max_n=None) + + + + + A checkpointing schedule. + + Actions in the schedule are accessed by iterating over elements, and + actions may be implemented using single-dispatch functions. e.g. + + .. code-block:: python + + @functools.singledispatch + def action(cp_action): + raise TypeError(f"Unexpected checkpointing action: {cp_action}") + + @action.register(Forward) + def action_forward(cp_action): + logger.debug(f"forward: forward advance to {cp_action.n1:d}") + + # ... + + for cp_action in cp_schedule: + action(cp_action) + if isinstance(cp_action, EndReverse): + break + + Schedules control an intermediate storage, which buffers forward restart + data for forward restart checkpoints, and which stores non-linear + dependency data either for storage in checkpointing units or for immediate + use by the adjoint. For details see + + - James R. Maddison, 'On the implementation of checkpointing with + high-level algorithmic differentiation', + https://arxiv.org/abs/2305.09568v1, 2023 + + In 'offline' schedules, where the number of steps in the forward + calculation is initially known, this should be provided using the `max_n` + argument on instantiation. In 'online' schedules, where the number of steps + in the forward calculation is initially unknown, the number of forward + steps should later be provided using the + :meth:`.CheckpointSchedule.finalize` method. + + :arg max_n: The number of steps in the initial forward calculation. If not + supplied then this should later be provided by calling the + :meth:`.CheckpointSchedule.finalize` method. + + .. py:property:: is_exhausted + :abstractmethod: + + Whether the schedule has concluded. Note that some schedules permit + multiple adjoint calculation, and may never conclude. + + + .. py:property:: uses_disk_storage + :abstractmethod: + + Whether the schedule may use disk storage. If `False` then no disk + storage is required. + + + .. py:property:: n + + The forward location. After executing all actions defined so far in + the schedule the forward is at the start of this step. + + + .. py:property:: r + + The number of adjoint steps advanced in the current adjoint + calculation after executing all actions defined so far in the schedule. + + + .. py:property:: max_n + + The number of forward steps in the initial forward calculation. May + return `None` if this has not yet been provided to the scheduler. + + + .. py:property:: is_running + + Whether the schedule is 'running' -- i.e. at least one action has + been defined so far in the schedule. + + + .. py:method:: iter() + :abstractmethod: + + A generator which should be overridden in derived classes in order + to define a checkpointing schedule. + + + .. py:method:: finalize(n) + + Indicate the number of forward steps in the initial forward + calculation. + + :arg n: The number of steps in the initial forward calculation. + + + diff --git a/_sources/autoapi/tlm_adjoint/checkpointing/index.rst.txt b/_sources/autoapi/tlm_adjoint/checkpointing/index.rst.txt new file mode 100644 index 0000000..6f412f0 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/checkpointing/index.rst.txt @@ -0,0 +1,397 @@ +:orphan: + +:py:mod:`tlm_adjoint.checkpointing` +=================================== + +.. py:module:: tlm_adjoint.checkpointing + + +Module Contents +--------------- + +.. py:class:: CheckpointStorage(*, store_ics, store_data) + + + A buffer for forward restart data, and a cache for non-linear dependency + data. Contains three types of data: + + 1. References: Dependencies which are stored by reference. Variables + `x` for which `var_is_static(x)` is `True` are stored by reference. + 2. Forward restart / initial condition data: Dependencies which are + used to restart and advance the forward calculation. + 3. Non-linear dependency data: Non-linear dependencies of the forward + which are used to advance the adjoint. + + These may overlap -- for example non-linear dependency data may be stored + by reference. + + Non-linear dependency data has an associated key `(n, i)`, where `n` is an + :class:`int` indicating the block index and `i` is an :class:`int` + indicating the equation index within that block. Non-linear-dependency data + for an :class:`.Equation` can be accessed via, e.g. + + .. code-block:: python + + nl_deps = cp[(n, i)] + + where `cp` is a :class:`.CheckpointStorage`. Here `nl_deps` is a + :class:`tuple` of variables storing values associated with + `eq.nonlinear_dependencies()`, for :class:`.Equation` `i` in block `n`. + + :arg store_ics: Whether to enable storage of forward restart data. + :arg store_data: Whether to enable storage of non-linear dependency data. + + .. py:property:: store_ics + + Whether storage of forward restart data is enabled. + + + + .. py:property:: store_data + + Whether storage of non-linear dependency data is enabled. + + + + .. py:method:: configure(*, store_ics, store_data) + + Enable or disable storage of forward restart and non-linear + dependency data. + + :arg store_ics: Whether storage of forward restart data should be + enabled (`store_ics=True`) or disabled (`store_ics=False`). + :arg store_data: Whether storage of non-linear dependency data should + be enabled (`store_data=True`) or disabled (`store_data=False`). + + + .. py:method:: clear(*, clear_ics=True, clear_data=True, clear_refs=False) + + Clear stored data. + + :arg clear_ics: Whether forward restart data should be cleared. + :arg clear_data: Whether non-linear dependency data should be cleared. + :arg clear_refs: Whether references should be cleared. + + + .. py:method:: initial_condition(x, *, copy=True) + + Return the forward restart value associated with a variable `x`. + + :arg x: The variable, or the :class:`int` variable ID, for which the + forward restart value should be returned. + :arg copy: If `True` then a copy of the stored value is returned. If + `False` then an internal variable storing the value is returned. + :returns: A variable containing the forward restart value for `x`. + + + .. py:method:: initial_conditions(*, cp=True, refs=False, copy=True) + + Access stored forward restart data. + + :arg cp: Whether to include forward restart data that is stored by + value. + :arg refs: Whether to include data that is stored by reference. May + include non-linear dependency data that is not forward restart + data. + :arg copy: If `True` then a copy of the stored data is returned. If + `False` then internal variables storing the data are returned. + :returns: A :class:`.VariableStateLockDictionary`, with items `(x_id: + x_value)`, where `x_id` is the :class:`int` variable ID and + `x_value` is a variable storing the data. + + + .. py:method:: add_initial_condition(x, value=None) + + Store forward restart data / an initial condition dependency. + + :arg x: The initial condition dependency variable. + :arg value: A variable defining the initial condition dependency value. + `x` is used if not supplied. + + + .. py:method:: update_keys(n, i, eq) + + The :class:`.CheckpointStorage` keeps an internal map from forward + variables to equations in which values for those variables are + computed. Keys are updated automatically as needed. This method allows + keys to be updated manually. + + :arg n: The :class:`int` index of the block. + :arg i: The :class:`int` index of the equation. + :arg eq: An :class:`.Equation`, equation `i` in block `n`. + + + .. py:method:: add_equation(n, i, eq, *, deps=None, nl_deps=None) + + Store checkpoint data associated with an equation. + + :arg n: The :class:`int` index of the block. + :arg i: The :class:`int` index of the equation. + :arg eq: An :class:`.Equation`, equation `i` in block `n`. + :arg deps: Equation dependency values. `eq.dependencies()` is used if + not supplied. + :arg nl_deps: Equation non-linear dependency values. Extracted from + `deps` if not supplied. + + + .. py:method:: add_equation_data(n, i, eq, *, nl_deps=None) + + Store checkpoint data associated with an equation. As + :meth:`.CheckpointStorage.add_equation`, but adds only *non-linear* + dependency data. + + :arg n: The :class:`int` index of the block. + :arg i: The :class:`int` index of the equation. + :arg eq: An :class:`.Equation`, equation `i` in block `n`. + :arg nl_deps: Equation non-linear dependency values. + `eq.nonlinear_dependencies()` is used if not supplied. + + + .. py:method:: checkpoint_data(*, ics=True, data=True, copy=True) + + Extract checkpoint data. + + :arg ics: Whether to extract forward restart data. + :arg data: Whether to extract non-linear dependency data. + :arg copy: If `True` then a copy of the stored data is returned. If + `False` then internal variables storing the data are returned. + :returns: A :class:`tuple` `(cp, data, storage)`. Elements of this + :class:`tuple` are as for the three arguments for the + :meth:`.CheckpointStorage.update` method, and here `storage` is a + :class:`.VariableStateLockDictionary`. + + + .. py:method:: update(cp, data, storage, *, copy=True) + + Update the :class:`.CheckpointStorage` using the provided + checkpoint data. Used to update the :class:`.CheckpointStorage` from + loaded data. Note that the :class:`.CheckpointStorage` is *not* + cleared prior to updating using the provided data. + + :arg cp: A :class:`tuple` of keys. Forward restart data is defined by + `(storage[key] for key in cp)`. + :arg data: A :class:`dict`. Items are `((n, i), keys)`, indicating + that non-linear dependency data for equation `i` in block `n` is + `(storage[key] for key in keys)`. + :arg storage: The stored data. A :class:`Mapping` with items `((x_id, + x_indices), x_value)`. `x_id` is the :class:`int` ID for a variable + whose value `x_value` is stored. `x_indices` is either `None`, if + the variable value has not been computed by solving equations with + forward restart data storage enabled, or a tuple `(n, i, m)` + indicating that the variable value was computed as component `m` of + the solution to equation `i` in block `n`. + :arg copy: Whether the values in `storage` should be copied when being + stored in the :class:`.CheckpointStorage`. + + + +.. py:class:: ReplayStorage(blocks, N0, N1, *, transpose_deps=None) + + + Storage used when solving forward equations. + + A value for a forward variable can be accessed via + + .. code-block:: python + + x_value = replay_storage[x] + + and set via + + .. code-block:: python + + replay_storage[x] = x_value + + where here `x` is either a variable of an :class:`int` variable ID. + Containment can also be tested, + + .. code-block:: python + + if x in replay_storage: + [...] + + :arg blocks: A :class:`Sequence` or :class:`Mapping`, whose elements or + values are :class:`Sequence` objects containing :class:`.Equation` + objects. Forward equations. + :arg N0: An :class:`int`. `(blocks[n] for n in range(N0, N1))` defines the + forward equations which will be solved. + :arg N1: An :class:`int`. `(blocks[n] for n in range(N0, N1))` defines the + forward equations which will be solved. + :arg transpose_deps: A :class:`.TransposeComputationalGraph`. If supplied + then an activity analysis is applied. + + .. py:method:: is_active(n, i) + + Return whether the activity analysis indicates that an equation is + 'active'. + + :arg n: The :class:`int` index of the block. + :arg i: The :class:`int` index of the equation. + :returns: `True` if the equation is active, and `False` otherwise. + + + .. py:method:: update(d, *, copy=True) + + Use the supplied :class:`Mapping` to update forward values. + + :arg d: A :class:`Mapping`. Updates values for those keys in `d` + which are also in the :class:`.ReplayStorage`. + :arg copy: Whether the values in `d` should be copied when being stored + in the :class:`.ReplayStorage`. + + + .. py:method:: popleft() + + Remove the first equation. Used to deallocate forward variables + which are no longer needed as the solution of forward equations + progresses. + + :returns: A :class:`tuple` `(n, i)`, indicating that equation `i` in + block `n` has been removed. + + + +.. py:class:: Checkpoints + + + + + Disk checkpointing abstract base class. + + + .. py:method:: write(n, cp, data, storage) + :abstractmethod: + + Write checkpoint data. + + :arg n: The :class:`int` index of the block with which the checkpoint + data to be written is associated. + :arg cp: See :meth:`.CheckpointStorage.update`. + :arg data: See :meth:`.CheckpointStorage.update`. + :arg storage: See :meth:`.CheckpointStorage.update`. + + + .. py:method:: read(n, *, ics=True, data=True, ic_ids=None) + :abstractmethod: + + Read checkpoint data. + + :arg n: The :class:`int` index of the block with which the checkpoint + data to be read is associated. + :arg ics: Whether forward restart data should be included. + :arg data: Whether non-linear dependency data should be included. + :arg ic_ids: A :class:`Container`. If provided then only variables with + ID in `ic_ids` are included. + :returns: A :class:`tuple` `(cp, data, storage)`. Elements of this + :class:`tuple` are as for the three arguments for the + :meth:`.CheckpointStorage.update` method. + + + .. py:method:: delete(n) + :abstractmethod: + + Delete checkpoint data. + + :arg n: The :class:`int` index of the block with which the checkpoint + data to be deleted is associated. + + + +.. py:class:: PickleCheckpoints(prefix, *, comm=None) + + + + + Disk checkpointing using the pickle module. + + :arg prefix: Checkpoint files are stored at + `[prefix]_[root_pid]_[root_py2f]_[rank].pickle`. Here `prefix` is + defined by this argument, `root_pid` is the process ID on the root + process (i.e. process 0), `root_py2f` is the Fortran MPI communicator + on the root process, and `rank` is the process rank. + :arg comm: A communicator. + + .. py:method:: write(n, cp, data, storage) + + Write checkpoint data. + + :arg n: The :class:`int` index of the block with which the checkpoint + data to be written is associated. + :arg cp: See :meth:`.CheckpointStorage.update`. + :arg data: See :meth:`.CheckpointStorage.update`. + :arg storage: See :meth:`.CheckpointStorage.update`. + + + .. py:method:: read(n, *, ics=True, data=True, ic_ids=None) + + Read checkpoint data. + + :arg n: The :class:`int` index of the block with which the checkpoint + data to be read is associated. + :arg ics: Whether forward restart data should be included. + :arg data: Whether non-linear dependency data should be included. + :arg ic_ids: A :class:`Container`. If provided then only variables with + ID in `ic_ids` are included. + :returns: A :class:`tuple` `(cp, data, storage)`. Elements of this + :class:`tuple` are as for the three arguments for the + :meth:`.CheckpointStorage.update` method. + + + .. py:method:: delete(n) + + Delete checkpoint data. + + :arg n: The :class:`int` index of the block with which the checkpoint + data to be deleted is associated. + + + +.. py:class:: HDF5Checkpoints(prefix, *, comm=None) + + + + + Disk checkpointing using the h5py library. + + :arg prefix: Checkpoint files are stored at + `[prefix]_[root_pid]_[root_py2f].hdf5`. Here `prefix` is defined by + this argument, `root_pid` is the process ID on the root process (i.e. + process 0), and `root_py2f` is the Fortran MPI communicator on the root + process. + :arg comm: A communicator. + + .. py:method:: write(n, cp, data, storage) + + Write checkpoint data. + + :arg n: The :class:`int` index of the block with which the checkpoint + data to be written is associated. + :arg cp: See :meth:`.CheckpointStorage.update`. + :arg data: See :meth:`.CheckpointStorage.update`. + :arg storage: See :meth:`.CheckpointStorage.update`. + + + .. py:method:: read(n, *, ics=True, data=True, ic_ids=None) + + Read checkpoint data. + + :arg n: The :class:`int` index of the block with which the checkpoint + data to be read is associated. + :arg ics: Whether forward restart data should be included. + :arg data: Whether non-linear dependency data should be included. + :arg ic_ids: A :class:`Container`. If provided then only variables with + ID in `ic_ids` are included. + :returns: A :class:`tuple` `(cp, data, storage)`. Elements of this + :class:`tuple` are as for the three arguments for the + :meth:`.CheckpointStorage.update` method. + + + .. py:method:: delete(n) + + Delete checkpoint data. + + :arg n: The :class:`int` index of the block with which the checkpoint + data to be deleted is associated. + + + diff --git a/_sources/autoapi/tlm_adjoint/eigendecomposition/index.rst.txt b/_sources/autoapi/tlm_adjoint/eigendecomposition/index.rst.txt new file mode 100644 index 0000000..46d0387 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/eigendecomposition/index.rst.txt @@ -0,0 +1,104 @@ +:orphan: + +:py:mod:`tlm_adjoint.eigendecomposition` +======================================== + +.. py:module:: tlm_adjoint.eigendecomposition + + +Module Contents +--------------- + +.. py:function:: eigendecompose(space, A_action, *, B_action=None, arg_space_type='primal', action_space_type=None, N_eigenvalues=None, solver_type=None, problem_type=None, which=None, tolerance=1e-12, pre_callback=None, post_callback=None) + + Interface with SLEPc via slepc4py, for the matrix free solution of + eigenproblems + + .. math:: + + A v = \lambda v, + + or generalized eigenproblems + + .. math:: + + A v = \lambda B v. + + Originally developed by loosely following the slepc4py 3.6.0 demo + demo/ex3.py. slepc4py 3.6.0 license information follows: + + .. code-block:: text + + ========================= + LICENSE: SLEPc for Python + ========================= + + :Author: Lisandro Dalcin + :Contact: dalcinl@gmail.com + + + Copyright (c) 2015, Lisandro Dalcin. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + :arg space: The space for each eigenvector. + :arg A_action: A callable. Accepts a single variable argument, and returns + a variable containing the result after left multiplication of the input + by :math:`A`. + :arg B_action: A callable. Accepts a single variable argument, and returns + a variable containing the result after left multiplication of the input + by :math:`B`. + :arg arg_space_type: The space type of eigenvectors. `'primal'`, `'dual'`, + `'conjugate'`, or `'conjugate_dual'`. + :arg action_space_type: The space type of the result of multiplication by + :math:`A` or :math:`B`. `'primal'`, `'dual'`, `'conjugate'`, or + `'conjugate_dual'`. Defaults to the space type conjugate dual to + `arg_space_type`. + :arg N_eigenvalues: An :class:`int`, the number of eigenvalues to attempt + to compute. Defaults to the dimension of `space`. + :arg problem_type: The eigenproblem type -- see + `slepc4py.SLEPc.EPS.ProblemType`. Defaults to + `slepc4py.SLEPc.EPS.ProblemType.GNHEP` if `B_action` is supplied, or + `slepc4py.SLEPc.EPS.ProblemType.NHEP` otherwise. + :arg which: Which eigenvalues to attempt to compute -- see + `slepc4py.SLEPc.EPS.Which`. Defaults to + `slepc4py.SLEPc.EPS.Which.LARGEST_MAGNITUDE`. + :arg tolerance: Convergence tolerance. By default the convergence criterion + is defined using `slepc4py.SLEPc.EPS.Conv.REL`. + :arg pre_callback: A callable accepting a single `slepc4py.SLEPc.EPS` + argument. Used for detailed manual configuration. Called after all + other configuration options are set, but before the + `slepc4py.SLEPc.EPS.setUp` method is called. + :arg post_callback: A callable accepting a single `slepc4py.SLEPc.EPS` + argument. Called after the `slepc4py.SLEPc.EPS.solve` method has been + called. + :returns: A :class:`tuple` `(lam, V)`. `lam` is a :class:`numpy.ndarray` + containing eigenvalues. For non-Hermitian algorithms and a real build + of PETSc, `V` is a :class:`tuple` `(V_r, V_i)`, where `V_r` and `V_i` + are each a :class:`tuple` of variables containing respectively the real + and complex parts of corresponding eigenvectors. Otherwise `V` is a + :class:`tuple` of variables containing corresponding eigenvectors. + + diff --git a/_sources/autoapi/tlm_adjoint/equation/index.rst.txt b/_sources/autoapi/tlm_adjoint/equation/index.rst.txt new file mode 100644 index 0000000..d120e0e --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/equation/index.rst.txt @@ -0,0 +1,423 @@ +:orphan: + +:py:mod:`tlm_adjoint.equation` +============================== + +.. py:module:: tlm_adjoint.equation + + +Module Contents +--------------- + +.. py:class:: Equation(X, deps, nl_deps=None, *, ic_deps=None, ic=None, adj_ic_deps=None, adj_ic=None, adj_type='conjugate_dual') + + + + + Core equation class. Defines a differentiable operation for use as an + adjoint tape record. + + The equation is defined via a residual function :math:`\mathcal{F}`. The + forward solution is defined implicitly as the value :math:`x` for which + + .. math:: + + \mathcal{F} \left( x, y_0, y_1, \ldots \right) = 0, + + where :math:`y_i` are dependencies. + + This is an abstract base class. Information required to solve forward + equations, perform adjoint calculations, and define tangent-linear + equations, is provided by overloading abstract methods. This class does + *not* inherit from :class:`abc.ABC`, so that methods may be implemented as + needed. + + :arg X: A variable, or a :class:`Sequence` of variables, defining the + forward solution. + :arg deps: A :class:`Sequence` of variables defining dependencies. Must + define a superset of `X`. + :arg nl_deps: A :class:`Sequence` of variables defining non-linear + dependencies. Must define a subset of `deps`. Defaults to `deps`. + :arg ic_deps: A :class:`Sequence` of variables whose value must be + available prior to computing the forward solution. Intended for + iterative methods with non-zero initial guesses. Must define a subset + of `X`. Can be overridden by `ic`. + :arg ic: Whether `ic_deps` should be set equal to `X`. Defaults to `True` + if `ic_deps` is not supplied, and to `False` otherwise. + :arg adj_ic_deps: A :class:`Sequence` of variables whose value must be + available prior to computing the adjoint solution. Intended for + iterative methods with non-zero initial guesses. Must define a subset + of `X`. Can be overridden by `adj_ic`. + :arg adj_ic: Whether `adj_ic_deps` should be set equal to `X`. Defaults to + `True` if `adj_ic_deps` is not supplied, and to `False` otherwise. + :arg adj_type: The space type relative to `X` of adjoint variables. + `'primal'` or `'conjugate_dual'`, or a :class:`Sequence` of these. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: x() + + Return the forward solution variable, assuming the forward solution + has one component. + + :returns: A variable defining the forward solution. + + + .. py:method:: X(m=None) + + Return forward solution variables. + + :returns: If `m` is supplied, a variable defining the `m` th component + of the forward solution. If `m` is not supplied, a :class:`tuple` + of variables defining the forward solution. + + + .. py:method:: dependencies() + + Return dependencies. + + :returns: A :class:`tuple` of variables defining dependencies. + + + .. py:method:: nonlinear_dependencies() + + Return non-linear dependencies. + + :returns: A :class:`tuple` of variables defining non-linear + dependencies. + + + .. py:method:: initial_condition_dependencies() + + Return 'initial condition' dependencies -- dependencies whose value + is needed prior to computing the forward solution. + + :returns: A :class:`tuple` of variables defining initial condition + dependencies. + + + .. py:method:: adjoint_initial_condition_dependencies() + + Return adjoint 'initial condition' dependencies -- dependencies + whose value is needed prior to computing the adjoint solution. + + :returns: A :class:`tuple` of variables defining adjoint initial + condition dependencies. + + + .. py:method:: adj_x_type() + + Return the space type for the adjoint solution, relative to the + forward solution, assuming the forward solution has exactly one + component. + + :returns: One of `'primal'` or `'conjugate_dual'`. + + + .. py:method:: adj_X_type(m=None) + + Return the space type for the adjoint solution, relative to the + forward solution. + + :returns: If `m` is supplied, one of `'primal'` or `'conjugate_dual'` + defining the relative space type for the `m` th component of the + adjoint solution. If `m` is not supplied, a :class:`tuple` whose + elements are `'primal'` or `'conjugate_dual'`, defining the + relative space type of the adjoint solution. + + + .. py:method:: new_adj_x() + + Return a new variable suitable for storing the adjoint solution, + assuming the forward solution has exactly one component. + + :returns: A variable suitable for storing the adjoint solution. + + + .. py:method:: new_adj_X(m=None) + + Return new variables suitable for storing the adjoint solution. + + :returns: If `m` is supplied, a variable suitable for storing the `m` + th component of the adjoint solution. If `m` is not supplied, a + :class:`tuple` of variables suitable for storing the adjoint + solution. + + + .. py:method:: solve(*, manager=None, annotate=None, tlm=None) + + Compute the forward solution. + + :arg manager: The :class:`.EquationManager`. Defaults to `manager()`. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + + + .. py:method:: forward(X, deps=None) + + Wraps :meth:`.Equation.forward_solve` to handle cache invalidation. + + + + .. py:method:: forward_solve(X, deps=None) + :abstractmethod: + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint(adj_X, nl_deps, B, dep_Bs) + + Compute the adjoint solution, and subtract terms from other adjoint + right-hand-sides. + + :arg adj_X: Either `None`, or a :class:`Sequence` of variables defining + the initial guess for an iterative solve. May be modified or + returned. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: A sequence of variables defining the right-hand-side of the + adjoint equation. May be modified or returned. + :arg dep_Bs: A :class:`Mapping` whose items are `(dep_index, dep_B)`. + Each `dep_B` is an :class:`.AdjointRHS` which should be updated by + subtracting adjoint derivative information computed by + differentiating with respect to `self.dependencies()[dep_index]`. + + :returns: A :class:`tuple` of variables defining the adjoint solution, + or `None` to indicate that the solution is zero. + + + .. py:method:: adjoint_cached(adj_X, nl_deps, dep_Bs) + + Subtract terms from other adjoint right-hand-sides. + + :arg adj_X: A :class:`Sequence` of variables defining the adjoint + solution. Should not be modified. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_Bs: A :class:`Mapping` whose items are `(dep_index, dep_B)`. + Each `dep_B` is an :class:`.AdjointRHS` which should be updated by + subtracting adjoint derivative information computed by + differentiating with respect to `self.dependencies()[dep_index]`. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_X) + :abstractmethod: + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: subtract_adjoint_derivative_actions(adj_X, nl_deps, dep_Bs) + + Subtract terms from other adjoint right-hand-sides. + + Can be overridden for an optimized implementation, but otherwise uses + :meth:`.Equation.adjoint_derivative_action`. + + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_Bs: A :class:`Mapping` whose items are `(dep_index, dep_B)`. + Each `dep_B` is an :class:`.AdjointRHS` which should be updated by + subtracting adjoint derivative information computed by + differentiating with respect to `self.dependencies()[dep_index]`. + + + .. py:method:: adjoint_jacobian_solve(adj_X, nl_deps, B) + :abstractmethod: + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + :abstractmethod: + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: ZeroAssignment(X) + + + + + Represents an assignment + + .. math:: + + x = 0. + + The forward residual is defined + + .. math:: + + \mathcal{F} \left( x \right) = x. + + :arg X: A variable or a :class:`Sequence` of variables defining the forward + solution :math:`x`. + + .. py:method:: forward_solve(X, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_X) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_X, nl_deps, B) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: NullSolver(X) + + + + + Represents an assignment + + .. math:: + + x = 0. + + The forward residual is defined + + .. math:: + + \mathcal{F} \left( x \right) = x. + + :arg X: A variable or a :class:`Sequence` of variables defining the forward + solution :math:`x`. + + diff --git a/_sources/autoapi/tlm_adjoint/equations/index.rst.txt b/_sources/autoapi/tlm_adjoint/equations/index.rst.txt new file mode 100644 index 0000000..cbb0613 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/equations/index.rst.txt @@ -0,0 +1,652 @@ +:orphan: + +:py:mod:`tlm_adjoint.equations` +=============================== + +.. py:module:: tlm_adjoint.equations + + +Module Contents +--------------- + +.. py:class:: EmptyEquation + + + + + An adjoint tape record with no associated solution variables. + + + .. py:method:: forward_solve(X, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + +.. py:class:: Assignment(x, y) + + + + + Represents an assignment + + .. math:: + + x = y. + + The forward residual is defined + + .. math:: + + \mathcal{F} \left( x, y \right) = x - y. + + :arg x: A variable defining the forward solution :math:`x`. + :arg y: A variable defining :math:`y`. + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: Conversion(x, y) + + + + + Represents degree of freedom assignment + + .. math:: + + \tilde{x} = \tilde{y} + + where :math:`\tilde{x}` and :math:`\tilde{y}` are vectors of degrees of + freedom for :math:`x` and :math:`y` respectively. Can be used to convert + between different variable types. + + The forward residual is defined + + .. math:: + + \mathcal{F} \left( x, y \right) = \tilde{x} - \tilde{y}. + + :arg x: A variable defining the forward solution :math:`x`. + :arg y: A variable defining :math:`y`. + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: LinearCombination(x, *args) + + + + + Represents an assignment + + .. math:: + + x = \sum_i \alpha_i y_i. + + The forward residual is defined + + .. math:: + + \mathcal{F} \left( x, y_1, y_2, \ldots \right) + = x - \sum_i \alpha_i y_i. + + :arg x: A variable defining the forward solution :math:`x`. + :arg args: A :class:`tuple` of two element :class:`Sequence` objects. The + :math:`i` th element consists of `(alpha_i, y_i)`, where `alpha_i` is a + scalar corresponding to :math:`\alpha_i` and `y_i` a variable + corresponding to :math:`y_i`. + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: Axpy(y_new, y_old, alpha, x) + + + + + Represents an assignment + + .. math:: + + y_\text{new} = y_\text{old} + \alpha x. + + The forward residual is defined + + .. math:: + + \mathcal{F} \left( y_\text{new}, y_\text{old}, x \right) + = y_\text{new} - y_\text{old} - \alpha x. + + :arg y_new: A variable defining the forward solution :math:`y_\text{new}`. + :arg y_old: A variable defining :math:`y_\text{old}`. + :arg alpha: A scalar defining :math:`\alpha`. + :arg x: A variable defining :math:`x`. + + +.. py:class:: DotProduct(x, y, z, *, alpha=1.0) + + + + + Represents an assignment + + .. math:: + + x = \alpha z^T y. + + The forward residual is defined + + .. math:: + + \mathcal{F} \left( x, y, z \right) = x - \alpha z^T y. + + :arg x: A variable whose degrees of freedom define the forward solution + :math:`x`. + :arg y: A variable whose degrees of freedom define :math:`y`. + :arg z: A variable whose degrees of freedom define :math:`z`. May be the + same variable as `y`. + :arg alpha: A scalar defining :math:`\alpha`. + + +.. py:class:: InnerProduct(x, y, z, *, alpha=1.0, M=None) + + + + + Represents an assignment + + .. math:: + + x = \alpha z^* M y. + + The forward residual is defined + + .. math:: + + \mathcal{F} \left( x, y, z \right) = x - \alpha z^* M y. + + :arg x: A variable whose degrees of freedom define the forward solution + :math:`x`. + :arg y: A variable whose degrees of freedom define :math:`y`. + :arg z: A variable whose degrees of freedom define :math:`z`. May be the + same variable as `y`. + :arg alpha: A scalar defining :math:`\alpha`. + :arg M: A :class:`tlm_adjoint.linear_equation.Matrix` defining :math:`M`. + Must have no dependencies. Defaults to an identity matrix. + + +.. py:class:: MatrixActionRHS(A, X) + + + + + Represents a right-hand-side term + + .. math:: + + A x. + + :arg A: A :class:`tlm_adjoint.linear_equation.Matrix` defining :math:`A`. + :arg x: A variable or a :class:`Sequence` of variables defining :math:`x`. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: add_forward(B, deps) + + Add the right-hand-side term to `B`. + + :arg B: A variable if it has a single component, and a + :class:`Sequence` of variables otherwise. Should be updated by the + addition of this :class:`.RHS`. Subclasses may replace this + argument with `b` if there is a single component. + :arg deps: A :class:`Sequence` of variables defining values for + dependencies. Should not be modified. + + + .. py:method:: subtract_adjoint_derivative_action(nl_deps, dep_index, adj_X, b) + + Subtract the action of the adjoint of a derivative of the + right-hand-side term, on an adjoint variable, from `b`. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg deps_index: An :class:`int`. The derivative is defined by + differentiation of the right-hand-side term with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint variable. A variable if it has a single + component, and a :class:`Sequence` of variables otherwise. Should + not be modified. Subclasses may replace this argument with `adj_x` + if the adjoint variable has a single component. + :arg b: A variable storing the result. Should be updated by subtracting + the action of the adjoint of the right-hand-side term on the + adjoint variable. + + + .. py:method:: tangent_linear_rhs(M, dM, tlm_map) + + Construct tangent-linear right-hand-side terms obtained by + differentiation of this right-hand-side term. That is, construct + + .. math:: + + \sum_i \frac{\partial b}{\partial y_i} \tau_{y_i}, + + where :math:`b` is this right-hand-side term, and :math:`\tau_{y_i}` is + the tangent-linear variable associated with a dependency :math:`y_i`. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: A :class:`.RHS`, or a :class:`Sequence` of :class:`.RHS` + objects, defining the right-hand-side terms. Returning `None` + indicates that there are no terms. + + + +.. py:class:: DotProductRHS(x, y, *, alpha=1.0) + + + + + Represents a right-hand-side term + + .. math:: + + \alpha y^T x. + + :arg x: A variable whose degrees of freedom define :math:`x`. + :arg y: A variable whose degrees of freedom define :math:`y`. May be the + same variable as `x`. + :arg alpha: A scalar defining :math:`\alpha`. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: add_forward(b, deps) + + Add the right-hand-side term to `B`. + + :arg B: A variable if it has a single component, and a + :class:`Sequence` of variables otherwise. Should be updated by the + addition of this :class:`.RHS`. Subclasses may replace this + argument with `b` if there is a single component. + :arg deps: A :class:`Sequence` of variables defining values for + dependencies. Should not be modified. + + + .. py:method:: subtract_adjoint_derivative_action(nl_deps, dep_index, adj_x, b) + + Subtract the action of the adjoint of a derivative of the + right-hand-side term, on an adjoint variable, from `b`. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg deps_index: An :class:`int`. The derivative is defined by + differentiation of the right-hand-side term with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint variable. A variable if it has a single + component, and a :class:`Sequence` of variables otherwise. Should + not be modified. Subclasses may replace this argument with `adj_x` + if the adjoint variable has a single component. + :arg b: A variable storing the result. Should be updated by subtracting + the action of the adjoint of the right-hand-side term on the + adjoint variable. + + + .. py:method:: tangent_linear_rhs(M, dM, tlm_map) + + Construct tangent-linear right-hand-side terms obtained by + differentiation of this right-hand-side term. That is, construct + + .. math:: + + \sum_i \frac{\partial b}{\partial y_i} \tau_{y_i}, + + where :math:`b` is this right-hand-side term, and :math:`\tau_{y_i}` is + the tangent-linear variable associated with a dependency :math:`y_i`. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: A :class:`.RHS`, or a :class:`Sequence` of :class:`.RHS` + objects, defining the right-hand-side terms. Returning `None` + indicates that there are no terms. + + + +.. py:class:: InnerProductRHS(x, y, *, alpha=1.0, M=None) + + + + + Represents a right-hand-side term + + .. math:: + + \alpha y^* M x. + + :arg x: A variable whose degrees of freedom define :math:`x`. + :arg y: A variable whose degrees of freedom define :math:`y`. May be the + same variable as `x`. + :arg alpha: A scalar defining :math:`\alpha`. + :arg M: A :class:`tlm_adjoint.linear_equation.Matrix` defining :math:`M`. + Must have no dependencies. Defaults to an identity matrix. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: add_forward(b, deps) + + Add the right-hand-side term to `B`. + + :arg B: A variable if it has a single component, and a + :class:`Sequence` of variables otherwise. Should be updated by the + addition of this :class:`.RHS`. Subclasses may replace this + argument with `b` if there is a single component. + :arg deps: A :class:`Sequence` of variables defining values for + dependencies. Should not be modified. + + + .. py:method:: subtract_adjoint_derivative_action(nl_deps, dep_index, adj_x, b) + + Subtract the action of the adjoint of a derivative of the + right-hand-side term, on an adjoint variable, from `b`. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg deps_index: An :class:`int`. The derivative is defined by + differentiation of the right-hand-side term with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint variable. A variable if it has a single + component, and a :class:`Sequence` of variables otherwise. Should + not be modified. Subclasses may replace this argument with `adj_x` + if the adjoint variable has a single component. + :arg b: A variable storing the result. Should be updated by subtracting + the action of the adjoint of the right-hand-side term on the + adjoint variable. + + + .. py:method:: tangent_linear_rhs(M, dM, tlm_map) + + Construct tangent-linear right-hand-side terms obtained by + differentiation of this right-hand-side term. That is, construct + + .. math:: + + \sum_i \frac{\partial b}{\partial y_i} \tau_{y_i}, + + where :math:`b` is this right-hand-side term, and :math:`\tau_{y_i}` is + the tangent-linear variable associated with a dependency :math:`y_i`. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: A :class:`.RHS`, or a :class:`Sequence` of :class:`.RHS` + objects, defining the right-hand-side terms. Returning `None` + indicates that there are no terms. + + + diff --git a/_sources/autoapi/tlm_adjoint/fenics/backend_interface/index.rst.txt b/_sources/autoapi/tlm_adjoint/fenics/backend_interface/index.rst.txt new file mode 100644 index 0000000..c0f1ca1 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/fenics/backend_interface/index.rst.txt @@ -0,0 +1,50 @@ +:orphan: + +:py:mod:`tlm_adjoint.fenics.backend_interface` +============================================== + +.. py:module:: tlm_adjoint.fenics.backend_interface + + +Module Contents +--------------- + +.. py:class:: Function(*args, space_type='primal', static=False, cache=None, **kwargs) + + + + + Extends the DOLFIN `Function` class. + + :arg space_type: The space type for the :class:`.Function`. `'primal'`, + `'dual'`, `'conjugate'`, or `'conjugate_dual'`. + :arg static: Defines whether the :class:`.Function` is static, meaning that + it is stored by reference in checkpointing/replay, and an associated + tangent-linear variable is zero. + :arg cache: Defines whether results involving the :class:`.Function` may be + cached. Default `static`. + + Remaining arguments are passed to the DOLFIN `Function` constructor. + + +.. py:class:: ZeroFunction(*args, **kwargs) + + + + + A :class:`.Function` which is flagged as having a value of zero. + + Arguments are passed to the :class:`.Function` constructor, together with + `static=True` and `cache=True`. + + +.. py:function:: to_fenics(y, space, *, name=None) + + Convert a variable to a DOLFIN `Function`. + + :arg y: A variable. + :arg space: The space for the return value. + :arg name: A :class:`str` name. + :returns: The DOLFIN `Function`. + + diff --git a/_sources/autoapi/tlm_adjoint/fenics/caches/index.rst.txt b/_sources/autoapi/tlm_adjoint/fenics/caches/index.rst.txt new file mode 100644 index 0000000..9438423 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/fenics/caches/index.rst.txt @@ -0,0 +1,104 @@ +:py:mod:`tlm_adjoint.fenics.caches` +=================================== + +.. py:module:: tlm_adjoint.fenics.caches + +.. autoapi-nested-parse:: + + This module implements finite element assembly and linear solver data + caching. + + + +Module Contents +--------------- + +.. py:class:: AssemblyCache + + + + + A :class:`.Cache` for finite element assembly data. + + + .. py:method:: assemble(form, *, bcs=None, form_compiler_parameters=None, linear_solver_parameters=None, replace_map=None) + + Perform finite element assembly and cache the result, or return a + previously cached result. + + :arg form: The :class:`ufl.Form` to assemble. + :arg bcs: Dirichlet boundary conditions. + :arg form_compiler_parameters: Form compiler parameters. + :arg linear_solver_parameters: Linear solver parameters. Required for + assembly parameters which appear in the linear solver parameters. + :arg replace_map: A :class:`Mapping` defining a map from symbolic + variables to values. + :returns: A :class:`tuple` `(value_ref, value)`, where `value` is the + result of the finite element assembly, and `value_ref` is a + :class:`.CacheRef` storing a reference to `value`. + + - For an arity zero or arity one form `value_ref` stores the + assembled value. + - For an arity two form `value_ref` is a tuple `(A, b_bc)`. `A` + is the assembled matrix, and `b_bc` is a boundary condition + right-hand-side term which should be added after assembling a + right-hand-side with homogeneous boundary conditions applied. + `b_bc` may be `None` to indicate that this term is zero. + + + +.. py:class:: LinearSolverCache + + + + + A :class:`.Cache` for linear solver data. + + + .. py:method:: linear_solver(form, *, bcs=None, form_compiler_parameters=None, linear_solver_parameters=None, replace_map=None, assembly_cache=None) + + Construct a linear solver and cache the result, or return a + previously cached result. + + :arg form: An arity two :class:`ufl.Form`, defining the matrix. + :arg bcs: Dirichlet boundary conditions. + :arg form_compiler_parameters: Form compiler parameters. + :arg linear_solver_parameters: Linear solver parameters. + :arg replace_map: A :class:`Mapping` defining a map from symbolic + variables to values. + :arg assembly_cache: :class:`.AssemblyCache` to use for finite element + assembly. Defaults to `assembly_cache()`. + :returns: A :class:`tuple` `(value_ref, value)`. `value` is a tuple + `(solver, A, b_bc)`, where `solver` is the linear solver, `A` is + the assembled matrix, and `b_bc` is a boundary condition + right-hand-side term which should be added after assembling a + right-hand-side with homogeneous boundary conditions applied. + `b_bc` may be `None` to indicate that this term is zero. + `value_ref` is a :class:`.CacheRef` storing a reference to `value`. + + + +.. py:function:: assembly_cache() + + :returns: The default :class:`.AssemblyCache`. + + +.. py:function:: set_assembly_cache(assembly_cache) + + Set the default :class:`.AssemblyCache`. + + :arg assembly_cache: The new default :class:`.AssemblyCache`. + + +.. py:function:: linear_solver_cache() + + :returns: The default :class:`.LinearSolverCache`. + + +.. py:function:: set_linear_solver_cache(linear_solver_cache) + + Set the default :class:`.LinearSolverCache`. + + :arg linear_solver_cache: The new default :class:`.LinearSolverCache`. + + diff --git a/_sources/autoapi/tlm_adjoint/fenics/equations/index.rst.txt b/_sources/autoapi/tlm_adjoint/fenics/equations/index.rst.txt new file mode 100644 index 0000000..6637158 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/fenics/equations/index.rst.txt @@ -0,0 +1,500 @@ +:py:mod:`tlm_adjoint.fenics.equations` +====================================== + +.. py:module:: tlm_adjoint.fenics.equations + +.. autoapi-nested-parse:: + + This module implements finite element calculations. In particular the + :class:`.EquationSolver` class implements the solution of finite element + variational problems. + + + +Module Contents +--------------- + +.. py:class:: Assembly(x, rhs, *, form_compiler_parameters=None, match_quadrature=None) + + + + + Represents assignment to the result of finite element assembly: + + .. code-block:: python + + x = assemble(rhs) + + The forward residual :math:`\mathcal{F}` is defined so that :math:`\partial + \mathcal{F} / \partial x` is the identity. + + :arg x: A variable defining the forward solution. + :arg rhs: A :class:`ufl.Form` to assemble. Should have arity 0 or 1, and + should not depend on `x`. + :arg form_compiler_parameters: Form compiler parameters. + :arg match_quadrature: Whether to set quadrature parameters consistently in + the forward, adjoint, and tangent-linears. Defaults to + `parameters['tlm_adjoint']['Assembly']['match_quadrature']`. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: EquationSolver(eq, x, bcs=None, *, J=None, form_compiler_parameters=None, solver_parameters=None, adjoint_solver_parameters=None, tlm_solver_parameters=None, cache_jacobian=None, cache_adjoint_jacobian=None, cache_tlm_jacobian=None, cache_rhs_assembly=None, match_quadrature=None) + + + + + Represents the solution of a finite element variational problem. + + Caching is based on the approach described in + + - J. R. Maddison and P. E. Farrell, 'Rapid development and adjoining of + transient finite element models', Computer Methods in Applied + Mechanics and Engineering, 276, 95--121, 2014, doi: + 10.1016/j.cma.2014.03.010 + + The arguments `eq`, `x`, `bcs`, `J`, `form_compiler_parameters`, and + `solver_parameters` are based on the interface for the DOLFIN + `dolfin.solve` function (see e.g. FEniCS 2017.1.0). + + :arg eq: A :class:`ufl.equation.Equation` defining the finite element + variational problem. + :arg x: A DOLFIN `Function` defining the forward solution. + :arg bcs: Dirichlet boundary conditions. + :arg J: A :class:`ufl.Form` defining a Jacobian matrix approximation to use + in a non-linear forward solve. + :arg form_compiler_parameters: Form compiler parameters. + :arg solver_parameters: Linear or non-linear solver parameters. + :arg adjoint_solver_parameters: Linear solver parameters to use in an + adjoint solve. + :arg tlm_solver_parameters: Linear solver parameters to use when solving + tangent-linear problems. + :arg cache_jacobian: Whether to cache the forward Jacobian matrix and + linear solver data. Defaults to + `parameters['tlm_adjoint']['EquationSolver]['cache_jacobian']`. If + `None` then caching is autodetected. + :arg cache_adjoint_jacobian: Whether to cache the adjoint Jacobian matrix + and linear solver data. Defaults to `cache_jacobian`. + :arg cache_tlm_jacobian: Whether to cache the Jacobian matrix and linear + solver data when solving tangent-linear problems. Defaults to + `cache_jacobian`. + :arg cache_rhs_assembly: Whether to enable right-hand-side caching. If + enabled then right-hand-side terms are divided into terms which are + cached, terms which are converted into matrix multiplication by a + cached matrix, and terms which are not cached. Defaults to + `parameters['tlm_adjoint']['EquationSolver']['cache_rhs_assembly']`. + :arg match_quadrature: Whether to set quadrature parameters consistently in + the forward, adjoint, and tangent-linears. Defaults to + `parameters['tlm_adjoint']['EquationSolver']['match_quadrature']`. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: subtract_adjoint_derivative_actions(adj_x, nl_deps, dep_Bs) + + Subtract terms from other adjoint right-hand-sides. + + Can be overridden for an optimized implementation, but otherwise uses + :meth:`.Equation.adjoint_derivative_action`. + + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_Bs: A :class:`Mapping` whose items are `(dep_index, dep_B)`. + Each `dep_B` is an :class:`.AdjointRHS` which should be updated by + subtracting adjoint derivative information computed by + differentiating with respect to `self.dependencies()[dep_index]`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:function:: expr_new_x(expr, x, *, annotate=None, tlm=None) + + If an expression depends on `x`, then record the assignment `x_old = + x`, and replace `x` with `x_old` in the expression. + + :arg expr: A :class:`ufl.core.expr.Expr`. + :arg x: Defines `x`. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + :returns: A :class:`ufl.core.expr.Expr` with `x` replaced with `x_old`, or + `expr` if the expression does not depend on `x`. + + +.. py:function:: linear_equation_new_x(eq, x, *, annotate=None, tlm=None) + + If a symbolic expression for a linear finite element variational problem + depends on the symbolic variable representing the problem solution `x`, + then record the assignment `x_old = x`, and replace `x` with `x_old` in the + symbolic expression. + + :arg eq: A :class:`ufl.equation.Equation` defining the finite element + variational problem. + :arg x: A DOLFIN `Function` defining the solution to the finite element + variational problem. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + :returns: A :class:`ufl.equation.Equation` with `x` replaced with `x_old`, + or `eq` if the symbolic expression does not depend on `x`. + + +.. py:class:: Projection(x, rhs, *args, **kwargs) + + + + + Represents the solution of a finite element variational problem + performing a projection onto the space for `x`. + + :arg x: A DOLFIN `Function` defining the forward solution. + :arg rhs: A :class:`ufl.core.expr.Expr` defining the expression to project + onto the space for `x`, or a :class:`ufl.Form` defining the + right-hand-side of the finite element variational problem. Should not + depend on `x`. + + Remaining arguments are passed to the :class:`.EquationSolver` constructor. + + +.. py:class:: DirichletBCApplication(x, y, *args, **kwargs) + + + + + Represents the application of a Dirichlet boundary condition to a zero + valued DOLFIN `Function`. Specifically this represents: + + .. code-block:: python + + x.vector().zero() + DirichletBC(x.function_space(), y, + *args, **kwargs).apply(x.vector()) + + The forward residual :math:`\mathcal{F}` is defined so that :math:`\partial + \mathcal{F} / \partial x` is the identity. + + :arg x: A DOLFIN `Function`, updated by the above operations. + :arg y: A DOLFIN `Function`, defines the Dirichet boundary condition. + + Remaining arguments are passed to the DOLFIN `DirichletBC` constructor. + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: ExprInterpolation(x, rhs) + + + + + Represents interpolation of `rhs` onto the space for `x`. + + The forward residual :math:`\mathcal{F}` is defined so that :math:`\partial + \mathcal{F} / \partial x` is the identity. + + :arg x: The forward solution. + :arg rhs: A :class:`ufl.core.expr.Expr` defining the expression to + interpolate onto the space for `x`. Should not depend on `x`. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + diff --git a/_sources/autoapi/tlm_adjoint/fenics/fenics_equations/index.rst.txt b/_sources/autoapi/tlm_adjoint/fenics/fenics_equations/index.rst.txt new file mode 100644 index 0000000..51b34ec --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/fenics/fenics_equations/index.rst.txt @@ -0,0 +1,260 @@ +:py:mod:`tlm_adjoint.fenics.fenics_equations` +============================================= + +.. py:module:: tlm_adjoint.fenics.fenics_equations + +.. autoapi-nested-parse:: + + This module includes additional functionality for use with FEniCS. + + + +Module Contents +--------------- + +.. py:class:: LocalSolverCache + + + + + A :class:`.Cache` for element-wise local block diagonal linear solvers. + + + .. py:method:: local_solver(form, solver_type=None, *, replace_map=None) + + Construct an element-wise local block diagonal linear solver and + cache the result, or return a previously cached result. + + :arg form: An arity two :class:`ufl.Form`, defining the element-wise + local block diagonal matrix. + :arg local_solver: `dolfin.LocalSolver.SolverType`. Defaults to + `dolfin.LocalSolver.SolverType.LU`. + :arg replace_map: A :class:`Mapping` defining a map from symbolic + variables to values. + :returns: A :class:`tuple` `(value_ref, value)`. `value` is a + DOLFIN `LocalSolver` and `value_ref` is a :class:`.CacheRef` + storing a reference to `value`. + + + +.. py:function:: local_solver_cache() + + :returns: The default :class:`.LocalSolverCache`. + + +.. py:function:: set_local_solver_cache(local_solver_cache) + + Set the default :class:`.LocalSolverCache`. + + :arg local_solver_cache: The new default :class:`.LocalSolverCache`. + + +.. py:class:: LocalProjection(x, rhs, *, form_compiler_parameters=None, cache_jacobian=None, cache_rhs_assembly=None, match_quadrature=None) + + + + + Represents the solution of a finite element variational problem + performing a projection onto the space for `x`, for the case where the mass + matrix is element-wise local block diagonal. + + :arg x: A DOLFIN `Function` defining the forward solution. + :arg rhs: A :class:`ufl.core.expr.Expr` defining the expression to project + onto the space for `x`, or a :class:`ufl.Form` defining the + right-hand-side of the finite element variational problem. Should not + depend on `x`. + + Remaining arguments are passed to the + :class:`tlm_adjoint.fenics.equations.EquationSolver` constructor. + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: Interpolation(x, y, *, x_coords=None, P=None, tolerance=0.0) + + + + + Represents interpolation of the scalar-valued function `y` onto the + space for `x`. + + The forward residual :math:`\mathcal{F}` is defined so that :math:`\partial + \mathcal{F} / \partial x` is the identity. + + Internally this builds (or uses a supplied) interpolation matrix for the + local process *only*. This behaves correctly if the there are no edges + between owned and non-owned nodes in the degree of freedom graph associated + with the discrete function space for `y`. + + :arg x: A scalar-valued DOLFIN `Function` defining the forward solution. + :arg y: A scalar-valued DOLFIN `Function` to interpolate onto the space for + `x`. + :arg X_coords: A :class:`numpy.ndarray` defining the coordinates at which + to interpolate `y`. Shape is `(n, d)` where `n` is the number of + process local degrees of freedom for `x` and `d` is the geometric + dimension. Defaults to the process local degree of freedom locations + for `x`. Ignored if `P` is supplied. + :arg P: The interpolation matrix. A :class:`scipy.sparse.spmatrix`. + :arg tolerance: Maximum permitted distance (as returned by the DOLFIN + `BoundingBoxTree.compute_closest_entity` method) of an interpolation + point from a cell in the mesh for `y`. Ignored if `P` is supplied. + + +.. py:class:: PointInterpolation(X, y, X_coords=None, *, P=None, tolerance=0.0) + + + + + Represents interpolation of a scalar-valued function at given points. + + The forward residual :math:`\mathcal{F}` is defined so that :math:`\partial + \mathcal{F} / \partial x` is the identity. + + Internally this builds (or uses a supplied) interpolation matrix for the + local process *only*. This behaves correctly if the there are no edges + between owned and non-owned nodes in the degree of freedom graph associated + with the discrete function space for `y`. + + :arg X: A scalar variable, or a :class:`Sequence` of scalar variables, + defining the forward solution. + :arg y: A scalar-valued DOLFIN `Function` to interpolate. + :arg X_coords: A :class:`numpy.ndarray` defining the coordinates at which + to interpolate `y`. Shape is `(n, d)` where `n` is the number of + interpolation points and `d` is the geometric dimension. Ignored if `P` + is supplied. + :arg P: The interpolation matrix. A :class:`scipy.sparse.spmatrix`. + :arg tolerance: Maximum permitted distance (as returned by the DOLFIN + `BoundingBoxTree.compute_closest_entity` method) of an interpolation + point from a cell in the mesh for `y`. Ignored if `P` is supplied. + + .. py:method:: forward_solve(X, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_X) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_X, nl_deps, B) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + diff --git a/_sources/autoapi/tlm_adjoint/fenics/functions/index.rst.txt b/_sources/autoapi/tlm_adjoint/fenics/functions/index.rst.txt new file mode 100644 index 0000000..3c745eb --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/fenics/functions/index.rst.txt @@ -0,0 +1,139 @@ +:py:mod:`tlm_adjoint.fenics.functions` +====================================== + +.. py:module:: tlm_adjoint.fenics.functions + +.. autoapi-nested-parse:: + + This module includes functionality for interacting with FEniCS variables + and Dirichlet boundary conditions. + + + +Module Contents +--------------- + +.. py:class:: Constant(value=None, *args, name=None, domain=None, space=None, space_type='primal', shape=None, comm=None, static=False, cache=None, **kwargs) + + + + + Extends the DOLFIN `Constant` class. + + :arg value: The initial value. `None` indicates a value of zero. + :arg name: A :class:`str` name. + :arg domain: The domain on which the :class:`.Constant` is defined. + :arg space: The space on which the :class:`.Constant` is defined. + :arg space_type: The space type for the :class:`.Constant`. `'primal'`, + `'dual'`, `'conjugate'`, or `'conjugate_dual'`. + :arg shape: A :class:`tuple` of :class:`int` objects defining the shape of + the value. + :arg comm: The communicator for the :class:`.Constant`. + :arg static: Defines whether the :class:`.Constant` is static, meaning that + it is stored by reference in checkpointing/replay, and an associated + tangent-linear variable is zero. + :arg cache: Defines whether results involving the :class:`.Constant` may be + cached. Default `static`. + + Remaining arguments are passed to the DOLFIN `Constant` constructor. + + +.. py:class:: Zero + + + Mixin for defining a zero-valued variable. Used for zero-valued + variables for which UFL zero elimination should not be applied. + + +.. py:class:: ZeroConstant(*, name=None, domain=None, space=None, space_type='primal', shape=None, comm=None) + + + + + A :class:`.Constant` which is flagged as having a value of zero. + + Arguments are passed to the :class:`.Constant` constructor, together with + `static=True` and `cache=True`. + + +.. py:function:: eliminate_zeros(expr) + + Apply zero elimination for :class:`.Zero` objects in the supplied + :class:`ufl.core.expr.Expr` or :class:`ufl.Form`. + + :arg expr: A :class:`ufl.core.expr.Expr` or :class:`ufl.Form`. + :returns: A :class:`ufl.core.expr.Expr` or :class:`ufl.Form` with zero + elimination applied. May return `expr`. + + +.. py:class:: DirichletBC(V, g, sub_domain, *args, static=None, _homogeneous=False, **kwargs) + + + + + Extends the DOLFIN `DirichletBC` class. + + :arg static: A flag that indicates that the value for the + :class:`.DirichletBC` will not change, and which determines whether + calculations involving this :class:`.DirichletBC` can be cached. If + `None` then autodetected from the value. + + Remaining arguments are passed to the DOLFIN `DirichletBC` constructor. + + +.. py:class:: HomogeneousDirichletBC(V, sub_domain, *args, **kwargs) + + + + + A :class:`.DirichletBC` whose value is zero. + + Arguments are passed to the :class:`.DirichletBC` constructor, together + with `static=True`. + + +.. py:class:: Replacement(x, count) + + + + + Represents a symbolic variable but with no value. + + + +.. py:class:: ReplacementConstant(x, count) + + + + + Represents a symbolic DOLFIN `Constant`, but has no value. + + + +.. py:class:: ReplacementFunction(x, count) + + + + + Represents a symbolic DOLFIN `Function`, but has no value. + + + +.. py:class:: ReplacementZeroConstant(*args, **kwargs) + + + + + Represents a symbolic DOLFIN `Constant` which is zero, but has no value. + + + +.. py:class:: ReplacementZeroFunction(*args, **kwargs) + + + + + Represents a symbolic DOLFIN `Function` which is zero, but has no value. + + + diff --git a/_sources/autoapi/tlm_adjoint/firedrake/backend_interface/index.rst.txt b/_sources/autoapi/tlm_adjoint/firedrake/backend_interface/index.rst.txt new file mode 100644 index 0000000..a059c7a --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/firedrake/backend_interface/index.rst.txt @@ -0,0 +1,86 @@ +:orphan: + +:py:mod:`tlm_adjoint.firedrake.backend_interface` +================================================= + +.. py:module:: tlm_adjoint.firedrake.backend_interface + + +Module Contents +--------------- + +.. py:class:: Function(*args, space_type='primal', static=False, cache=None, **kwargs) + + + + + Extends :class:`firedrake.function.Function`. + + :arg space_type: The space type for the :class:`.Function`. `'primal'` or + `'conjugate'`. + :arg static: Defines whether the :class:`.Function` is static, meaning that + it is stored by reference in checkpointing/replay, and an associated + tangent-linear variable is zero. + :arg cache: Defines whether results involving the :class:`.Function` may + be cached. Default `static`. + + Remaining arguments are passed to the :class:`firedrake.function.Function` + constructor. + + +.. py:class:: ZeroFunction(*args, **kwargs) + + + + + A :class:`.Function` which is flagged as having a value of zero. + + Arguments are passed to the :class:`.Function` constructor, together with + `static=True` and `cache=True`. + + +.. py:class:: Cofunction(*args, space_type='conjugate_dual', static=False, cache=None, **kwargs) + + + + + Extends the :class:`firedrake.cofunction.Cofunction` class. + + :arg space_type: The space type for the :class:`.Cofunction`. + `'conjugate'` or `'conjugate_dual'`. + :arg static: Defines whether the :class:`.Cofunction` is static, meaning + that it is stored by reference in checkpointing/replay, and an + associated tangent-linear variable is zero. + :arg cache: Defines whether results involving the :class:`.Cofunction` may + be cached. Default `static`. + + Remaining arguments are passed to the + :class:`firedrake.cofunction.Cofunction` constructor. + + .. py:method:: equals(other) + + Check equality. + + + +.. py:class:: ReplacementCofunction(x, count) + + + + + Represents a symbolic :class:`firedrake.cofunction.Cofunction`, but has + no value. + + +.. py:function:: to_firedrake(y, space, *, name=None) + + Convert a variable to a :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction`. + + :arg y: A variable. + :arg space: The space for the return value. + :arg name: A :class:`str` name. + :returns: The :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction`. + + diff --git a/_sources/autoapi/tlm_adjoint/firedrake/block_system/index.rst.txt b/_sources/autoapi/tlm_adjoint/firedrake/block_system/index.rst.txt new file mode 100644 index 0000000..0ad5f9a --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/firedrake/block_system/index.rst.txt @@ -0,0 +1,823 @@ +:py:mod:`tlm_adjoint.firedrake.block_system` +============================================ + +.. py:module:: tlm_adjoint.firedrake.block_system + +.. autoapi-nested-parse:: + + This module implements solvers for linear systems defined in mixed spaces. + + The :class:`.System` class defines the block structure of the linear system, + and solves the system using an outer Krylov solver. A custom preconditioner can + be defined via the `pc_fn` callback to :meth:`.System.solve`, and this + preconditioner can itself e.g. make use of further Krylov solvers. This + provides a Python interface for custom block preconditioners. + + Given a linear problem with a potentially singular matrix :math:`A` + + .. math:: + + A u = b, + + a :class:`.System` instead solves the linear problem + + .. math:: + + \left[ (I - M U (U^* M U)^{-1} U^*) A (I - V (V^* C V)^{-1} V^* C) + + M U S V^* C \right] u = (I - M U (U^* M U)^{-1} U^*) b. + + Here + + - :math:`U` is a full rank matrix whose columns span the left nullspace for + a modified system matrix :math:`\tilde{A}`. + - :math:`V` is a full rank matrix with the same number of columns as + :math:`U`, whose columns span the nullspace for :math:`\tilde{A}`. + - :math:`V^* C V` and :math:`S` are invertible matrices. + - :math:`M` is a Hermitian positive definite matrix. + + Here the left nullspace for a matrix is defined to be the nullspace for its + Hermitian transpose, and the modified system matrix :math:`\tilde{A}` is + defined + + .. math:: + + \tilde{A} = (I - M U (U^* M U)^{-1} U^*) A (I - V (V^* C V)^{-1} V^* C). + + This has two primary use cases: + + 1. Where a matrix :math:`A` and right-hand-side :math:`b` are constructed + via finite element assembly on superspaces of the test space and trial + space. The typical example is in the application of homogeneous + essential Dirichlet boundary conditions. + + 2. Where the matrix :math:`A` is singular and :math:`b` is orthogonal to + the left nullspace of :math:`A`. Typically one would then choose + :math:`U` and :math:`V` so that their columns respectively span the left + nullspace and nullspace of :math:`A`, and the :class:`.System` then + seeks a solution to the original problem subject to the linear + constraints :math:`V^* C u = 0`. + + Function spaces are defined via Firedrake function spaces, and + :class:`Sequence` objects containing Firedrake function spaces or similar + :class:`Sequence` objects. Similarly functions are defined via + :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` objects, or :class:`Sequence` objects + containing :class:`firedrake.function.Function`, + :class:`firedrake.cofunction.Cofunction`, or similar :class:`Sequence` objects. + This defines a basic tree structure which is useful e.g. when defining block + matrices in terms of sub-block matrices. + + Elements of the tree are accessed in a consistent order using a depth first + search. Hence e.g. + + .. code-block:: python + + ((u_0, u_1), u_2) + + and + + .. code-block:: python + + (u_0, u_1, u_2) + + where `u_0`, `u_1`, and `u_2` are :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` objects, are both valid + representations of a mixed space solution. + + + +Module Contents +--------------- + +.. py:class:: MixedSpace(spaces) + + + Used to map between different versions of a mixed space. + + This class defines two representations for the space: + + 1. As a 'split space': A tree defining the mixed space. Stored using + Firedrake function space and :class:`tuple` objects, each + corresponding to a node in the tree. Function spaces correspond to + leaf nodes, and :class:`tuple` objects to other nodes in the tree. + 2. As a 'flattened space': A :class:`Sequence` containing leaf nodes of + the split space with an ordering determined using a depth first + search. + + Provides methods to allow data to be copied to and from a compatible + :class:`petsc4py.PETSc.Vec`. This allows, for example, the construction: + + .. code-block:: python + + u_0 = Function(space_0, name='u_0') + u_1 = Function(space_1, name='u_1') + u_2 = Function(space_2, name='u_2') + + mixed_space = MixedSpace(((space_0, space_1), space_2)) + + and then data can be copied to a compatible :class:`petsc4py.PETSc.Vec` via + + .. code-block:: python + + mixed_space.to_petsc(u_petsc, ((u_0, u_1), u_2)) + + and from a compatible :class:`petsc4py.PETSc.Vec` via + + .. code-block:: python + + mixed_space.from_petsc(u_petsc, ((u_0, u_1), u_2)) + + :arg spaces: The split space. + + .. py:property:: comm + + The communicator associated with the mixed space. + + + + .. py:property:: split_space + + The split space representation. + + + + .. py:property:: flattened_space + + The flattened space representation. + + + + .. py:property:: local_size + + The number of local degrees of freedom. + + + + .. py:property:: global_size + + The global number of degrees of freedom. + + + + .. py:method:: new_split() + + :returns: A new element in the split space. + + + .. py:method:: from_petsc(u_petsc, u) + + Copy data from a compatible :class:`petsc4py.PETSc.Vec`. + + :arg u_petsc: The :class:`petsc4py.PETSc.Vec`. + :arg u: An element of the split space. + + + .. py:method:: to_petsc(u_petsc, u) + + Copy data to a compatible :class:`petsc4py.PETSc.Vec`. Does not + update the ghost. + + :arg u_petsc: The :class:`petsc4py.PETSc.Vec`. + :arg u: An element of the split space. + + + +.. py:class:: Nullspace + + + + + Represents a matrix nullspace and left nullspace. + + + .. py:method:: apply_nullspace_transformation_lhs_right(x) + :abstractmethod: + + Apply the nullspace transformation associated with a matrix action + on :math:`x`, + + .. math:: + + x \rightarrow (I - V (V^* C V)^{-1} V^* C) x. + + :arg x: Defines :math:`x`. + + + .. py:method:: apply_nullspace_transformation_lhs_left(y) + :abstractmethod: + + Apply the left nullspace transformation associated with a matrix + action, + + .. math:: + + y \rightarrow (I - M U (U^* M U)^{-1} U^*) y. + + :arg y: Defines :math:`y`. + + + .. py:method:: constraint_correct_lhs(x, y) + :abstractmethod: + + Add the linear constraint term to :math:`y`, + + .. math:: + + y \rightarrow y + M U S V^* C x. + + :arg x: Defines :math:`x`. + :arg y: Defines :math:`y`. + + + .. py:method:: pc_constraint_correct_soln(u, b) + :abstractmethod: + + Add the preconditioner linear constraint term to :math:`u`, + + .. math:: + + u \rightarrow u + V \tilde{S}^{-1} U^* b, + + with + + .. math:: + + \tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}. + + :arg u: Defines :math:`u`. + :arg b: Defines :math:`b`. + + + .. py:method:: correct_soln(x) + + Correct the linear system solution so that it is orthogonal to + space spanned by the columns of :math:`V`. + + :arg x: The linear system solution, to be corrected. + + + .. py:method:: pre_mult_correct_lhs(x) + + Apply the pre-left-multiplication nullspace transformation. + + :arg x: Defines the vector on which the matrix action is computed. + + + .. py:method:: post_mult_correct_lhs(x, y) + + Apply the post-left-multiplication nullspace transformation, and add + the linear constraint term. + + :arg x: Defines the vector on which the matrix action is computed, and + used to add the linear constraint term. If `None` is supplied then + the linear constraint term is not added. + :arg y: Defines the result of the matrix action on `x`. + + + .. py:method:: correct_rhs(b) + + Correct the linear system right-hand-side so that it is orthogonal + to the space spanned by the columns of :math:`U`. + + :arg b: The linear system right-hand-side, to be corrected. + + + .. py:method:: pc_pre_mult_correct(b) + + Apply the pre-preconditioner-application nullspace transformation. + + :arg b: Defines the vector on which the preconditioner action is + computed. + + + .. py:method:: pc_post_mult_correct(u, b) + + Apply the post-preconditioner-application left nullspace + transformation, and add the linear constraint term. + + :arg u: Defines the result of the preconditioner action on `b`. + :arg b: Defines the vector on which the preconditioner action is + computed, and used to add the linear constraint term. If `None` is + supplied then the linear constraint term is not added. + + + +.. py:class:: NoneNullspace + + + + + An empty nullspace and left nullspace. + + + .. py:method:: apply_nullspace_transformation_lhs_right(x) + + Apply the nullspace transformation associated with a matrix action + on :math:`x`, + + .. math:: + + x \rightarrow (I - V (V^* C V)^{-1} V^* C) x. + + :arg x: Defines :math:`x`. + + + .. py:method:: apply_nullspace_transformation_lhs_left(y) + + Apply the left nullspace transformation associated with a matrix + action, + + .. math:: + + y \rightarrow (I - M U (U^* M U)^{-1} U^*) y. + + :arg y: Defines :math:`y`. + + + .. py:method:: constraint_correct_lhs(x, y) + + Add the linear constraint term to :math:`y`, + + .. math:: + + y \rightarrow y + M U S V^* C x. + + :arg x: Defines :math:`x`. + :arg y: Defines :math:`y`. + + + .. py:method:: pc_constraint_correct_soln(u, b) + + Add the preconditioner linear constraint term to :math:`u`, + + .. math:: + + u \rightarrow u + V \tilde{S}^{-1} U^* b, + + with + + .. math:: + + \tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}. + + :arg u: Defines :math:`u`. + :arg b: Defines :math:`b`. + + + +.. py:class:: ConstantNullspace(*, alpha=1.0) + + + + + A nullspace and left nullspace spanned by the vector of ones. + + Here :math:`V = U`, :math:`U` is a single column matrix whose elements are + ones, :math:`C = M`, and :math:`M` is an identity matrix. + + :arg alpha: Defines the linear constraint matrix :math:`S = \left( \alpha / + N \right)` where :math:`N` is the length of the vector of ones. + + .. py:method:: apply_nullspace_transformation_lhs_right(x) + + Apply the nullspace transformation associated with a matrix action + on :math:`x`, + + .. math:: + + x \rightarrow (I - V (V^* C V)^{-1} V^* C) x. + + :arg x: Defines :math:`x`. + + + .. py:method:: apply_nullspace_transformation_lhs_left(y) + + Apply the left nullspace transformation associated with a matrix + action, + + .. math:: + + y \rightarrow (I - M U (U^* M U)^{-1} U^*) y. + + :arg y: Defines :math:`y`. + + + .. py:method:: constraint_correct_lhs(x, y) + + Add the linear constraint term to :math:`y`, + + .. math:: + + y \rightarrow y + M U S V^* C x. + + :arg x: Defines :math:`x`. + :arg y: Defines :math:`y`. + + + .. py:method:: pc_constraint_correct_soln(u, b) + + Add the preconditioner linear constraint term to :math:`u`, + + .. math:: + + u \rightarrow u + V \tilde{S}^{-1} U^* b, + + with + + .. math:: + + \tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}. + + :arg u: Defines :math:`u`. + :arg b: Defines :math:`b`. + + + +.. py:class:: UnityNullspace(space, *, alpha=1.0) + + + + + A nullspace and left nullspace defined by the unity-valued function. + + Here :math:`V = U`, :math:`U` is a single column matrix containing the + degree-of-freedom vector for the unity-valued function, :math:`C = M`, + and :math:`M` is the mass matrix. + + :arg space: A scalar-valued function space containing the unity-valued + function. + :arg alpha: Defines the linear constraint matrix :math:`S = \alpha \left( + U^* M U \right)^{-1}`. + + .. py:method:: apply_nullspace_transformation_lhs_right(x) + + Apply the nullspace transformation associated with a matrix action + on :math:`x`, + + .. math:: + + x \rightarrow (I - V (V^* C V)^{-1} V^* C) x. + + :arg x: Defines :math:`x`. + + + .. py:method:: apply_nullspace_transformation_lhs_left(y) + + Apply the left nullspace transformation associated with a matrix + action, + + .. math:: + + y \rightarrow (I - M U (U^* M U)^{-1} U^*) y. + + :arg y: Defines :math:`y`. + + + .. py:method:: constraint_correct_lhs(x, y) + + Add the linear constraint term to :math:`y`, + + .. math:: + + y \rightarrow y + M U S V^* C x. + + :arg x: Defines :math:`x`. + :arg y: Defines :math:`y`. + + + .. py:method:: pc_constraint_correct_soln(u, b) + + Add the preconditioner linear constraint term to :math:`u`, + + .. math:: + + u \rightarrow u + V \tilde{S}^{-1} U^* b, + + with + + .. math:: + + \tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}. + + :arg u: Defines :math:`u`. + :arg b: Defines :math:`b`. + + + +.. py:class:: DirichletBCNullspace(bcs, *, alpha=1.0) + + + + + A nullspace and left nullspace associated with homogeneous Dirichlet + boundary conditions. + + Here :math:`V = U`, :math:`U` is a zero-one matrix with exactly one + non-zero per column corresponding to one boundary condition + degree-of-freedom, :math:`C = M`, and :math:`M` is an identity matrix. + + :arg bcs: A :class:`firedrake.bcs.DirichletBC`, or a :class:`Sequence` of + :class:`firedrake.bcs.DirichletBC` objects. + :arg alpha: Defines the linear constraint matrix :math:`S = \alpha M`. + + .. py:method:: apply_nullspace_transformation_lhs_right(x) + + Apply the nullspace transformation associated with a matrix action + on :math:`x`, + + .. math:: + + x \rightarrow (I - V (V^* C V)^{-1} V^* C) x. + + :arg x: Defines :math:`x`. + + + .. py:method:: apply_nullspace_transformation_lhs_left(y) + + Apply the left nullspace transformation associated with a matrix + action, + + .. math:: + + y \rightarrow (I - M U (U^* M U)^{-1} U^*) y. + + :arg y: Defines :math:`y`. + + + .. py:method:: constraint_correct_lhs(x, y) + + Add the linear constraint term to :math:`y`, + + .. math:: + + y \rightarrow y + M U S V^* C x. + + :arg x: Defines :math:`x`. + :arg y: Defines :math:`y`. + + + .. py:method:: pc_constraint_correct_soln(u, b) + + Add the preconditioner linear constraint term to :math:`u`, + + .. math:: + + u \rightarrow u + V \tilde{S}^{-1} U^* b, + + with + + .. math:: + + \tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}. + + :arg u: Defines :math:`u`. + :arg b: Defines :math:`b`. + + + +.. py:class:: BlockNullspace(nullspaces) + + + + + Nullspaces for a mixed space. + + :arg nullspaces: A :class:`.Nullspace` or a :class:`Sequence` of + :class:`.Nullspace` objects defining the nullspace. `None` indicates a + :class:`.NoneNullspace`. + + .. py:method:: apply_nullspace_transformation_lhs_right(x) + + Apply the nullspace transformation associated with a matrix action + on :math:`x`, + + .. math:: + + x \rightarrow (I - V (V^* C V)^{-1} V^* C) x. + + :arg x: Defines :math:`x`. + + + .. py:method:: apply_nullspace_transformation_lhs_left(y) + + Apply the left nullspace transformation associated with a matrix + action, + + .. math:: + + y \rightarrow (I - M U (U^* M U)^{-1} U^*) y. + + :arg y: Defines :math:`y`. + + + .. py:method:: constraint_correct_lhs(x, y) + + Add the linear constraint term to :math:`y`, + + .. math:: + + y \rightarrow y + M U S V^* C x. + + :arg x: Defines :math:`x`. + :arg y: Defines :math:`y`. + + + .. py:method:: pc_constraint_correct_soln(u, b) + + Add the preconditioner linear constraint term to :math:`u`, + + .. math:: + + u \rightarrow u + V \tilde{S}^{-1} U^* b, + + with + + .. math:: + + \tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}. + + :arg u: Defines :math:`u`. + :arg b: Defines :math:`b`. + + + +.. py:class:: Matrix(arg_space, action_space) + + + + + Represents a matrix :math:`A` mapping :math:`V \rightarrow W`. + + :arg arg_space: Defines the space `V`. + :arg action_space: Defines the space `W`. + + .. py:property:: arg_space + + The space defining :math:`V`. + + + + .. py:property:: action_space + + The space defining :math:`W`. + + + + .. py:method:: mult_add(x, y) + :abstractmethod: + + Add :math:`A x` to :math:`y`. + + :arg x: Defines :math:`x`. Should not be modified. + :arg y: Defines :math:`y`. + + + +.. py:class:: PETScMatrix(arg_space, action_space, a) + + + + + A :class:`tlm_adjoint.firedrake.block_system.Matrix` associated with a + :class:`petsc4py.PETSc.Mat` :math:`A` mapping :math:`V \rightarrow W`. + + :arg arg_space: Defines the space `V`. + :arg action_space: Defines the space `W`. + :arg a: The :class:`petsc4py.PETSc.Mat`. + + .. py:method:: mult_add(x, y) + + Add :math:`A x` to :math:`y`. + + :arg x: Defines :math:`x`. Should not be modified. + :arg y: Defines :math:`y`. + + + +.. py:function:: form_matrix(a, *args, **kwargs) + + Construct a :class:`.PETScMatrix` associated with a given sesquilinear + form. + + :arg a: A :class:`ufl.Form` defining the sesquilinear form. + :returns: The :class:`.PETScMatrix`. + + Remaining arguments are passed to the :func:`firedrake.assemble.assemble` + function. + + +.. py:class:: BlockMatrix(arg_spaces, action_spaces, blocks=None) + + + + + A matrix :math:`A` mapping :math:`V \rightarrow W`, where :math:`V` and + :math:`W` are defined by mixed spaces. + + :arg arg_spaces: Defines the space `V`. + :arg action_spaces: Defines the space `W`. + :arg block: A :class:`Mapping` defining the blocks of the matrix. Items are + `((i, j), block)` where the block in the `i` th and `j` th column is + defined by `block`. Each `block` is a + :class:`tlm_adjoint.firedrake.block_system.Matrix` or + :class:`ufl.Form`, or `None` to indicate a zero block. + + .. py:method:: mult_add(x, y) + + Add :math:`A x` to :math:`y`. + + :arg x: Defines :math:`x`. Should not be modified. + :arg y: Defines :math:`y`. + + + +.. py:class:: System(arg_spaces, action_spaces, blocks, *, nullspaces=None, comm=None) + + + A linear system + + .. math:: + + A u = b. + + :arg arg_spaces: Defines the space for `u`. + :arg action_spaces: Defines the space for `b`. + :arg blocks: One of + + - A :class:`tlm_adjoint.firedrake.block_system.Matrix` or + :class:`ufl.Form` defining :math:`A`. + - A :class:`Mapping` with items `((i, j), block)` where the matrix + associated with the block in the `i` th and `j` th column is defined + by `block`. Each `block` is a + :class:`tlm_adjoint.firedrake.block_system.Matrix` or + :class:`ufl.Form`, or `None` to indicate a zero block. + + :arg nullspaces: A :class:`.Nullspace` or a :class:`Sequence` of + :class:`.Nullspace` objects defining the nullspace and left nullspace + of :math:`A`. `None` indicates a :class:`.NoneNullspace`. + :arg comm: Communicator. + + .. py:method:: solve(u, b, *, solver_parameters=None, pc_fn=None, pre_callback=None, post_callback=None, correct_initial_guess=True, correct_solution=True) + + Solve the linear system. + + :arg u: Defines the solution :math:`u`. + :arg b: Defines the right-hand-side :math:`b`. + :arg solver_parameters: A :class:`Mapping` defining outer Krylov solver + parameters. Parameters (a number of which are based on FEniCS + solver parameters) are: + + - `'linear_solver'`: The Krylov solver type, default `'fgmres'`. + - `'pc_side'`: Overrides the PETSc default preconditioning side. + - `'relative_tolerance'`: Relative tolerance. Required. + - `'absolute_tolerance'`: Absolute tolerance. Required. + - `'divergence_limit'`: Overrides the default divergence limit. + - `'maximum_iterations'`: Maximum number of iterations. Default + 1000. + - `'norm_type'`: Overrides the default convergence norm definition. + - `'nonzero_initial_guess'`: Whether to use a non-zero initial + guess, defined by the input `u`. Default `True`. + - `'gmres_restart'`: Overrides the default GMRES restart parameter. + + :arg pc_fn: Defines the application of a preconditioner. A callable + + .. code-block:: python + + def pc_fn(u, b): + + The preconditioner is applied to `b`, and the result stored in `u`. + Defaults to an identity. + :arg pre_callback: A callable accepting a single + :class:`petsc4py.PETSc.KSP` argument. Used for detailed manual + configuration. Called after all other configuration options are + set, but before the :meth:`petsc4py.PETSc.KSP.setUp` method is + called. + :arg post_callback: A callable accepting a single + :class:`petsc4py.PETSc.KSP` argument. Called after the + :meth:`petsc4py.PETSc.KSP.solve` method has been called. + :arg correct_initial_guess: Whether to apply a nullspace correction to + the initial guess. + :arg correct_solution: Whether to apply a nullspace correction to + the solution. + :returns: The number of Krylov iterations. + + + diff --git a/_sources/autoapi/tlm_adjoint/firedrake/caches/index.rst.txt b/_sources/autoapi/tlm_adjoint/firedrake/caches/index.rst.txt new file mode 100644 index 0000000..88d300b --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/firedrake/caches/index.rst.txt @@ -0,0 +1,104 @@ +:py:mod:`tlm_adjoint.firedrake.caches` +====================================== + +.. py:module:: tlm_adjoint.firedrake.caches + +.. autoapi-nested-parse:: + + This module implements finite element assembly and linear solver data + caching. + + + +Module Contents +--------------- + +.. py:class:: AssemblyCache + + + + + A :class:`.Cache` for finite element assembly data. + + + .. py:method:: assemble(form, *, bcs=None, form_compiler_parameters=None, linear_solver_parameters=None, replace_map=None) + + Perform finite element assembly and cache the result, or return a + previously cached result. + + :arg form: The :class:`ufl.Form` to assemble. + :arg bcs: Dirichlet boundary conditions. + :arg form_compiler_parameters: Form compiler parameters. + :arg linear_solver_parameters: Linear solver parameters. Required for + assembly parameters which appear in the linear solver parameters. + :arg replace_map: A :class:`Mapping` defining a map from symbolic + variables to values. + :returns: A :class:`tuple` `(value_ref, value)`, where `value` is the + result of the finite element assembly, and `value_ref` is a + :class:`.CacheRef` storing a reference to `value`. + + - For an arity zero or arity one form `value_ref` stores the + assembled value. + - For an arity two form `value_ref` is a tuple `(A, b_bc)`. `A` + is the assembled matrix, and `b_bc` is a boundary condition + right-hand-side term which should be added after assembling a + right-hand-side with homogeneous boundary conditions applied. + `b_bc` may be `None` to indicate that this term is zero. + + + +.. py:class:: LinearSolverCache + + + + + A :class:`.Cache` for linear solver data. + + + .. py:method:: linear_solver(form, *, bcs=None, form_compiler_parameters=None, linear_solver_parameters=None, replace_map=None, assembly_cache=None) + + Construct a linear solver and cache the result, or return a + previously cached result. + + :arg form: An arity two :class:`ufl.Form`, defining the matrix. + :arg bcs: Dirichlet boundary conditions. + :arg form_compiler_parameters: Form compiler parameters. + :arg linear_solver_parameters: Linear solver parameters. + :arg replace_map: A :class:`Mapping` defining a map from symbolic + variables to values. + :arg assembly_cache: :class:`.AssemblyCache` to use for finite element + assembly. Defaults to `assembly_cache()`. + :returns: A :class:`tuple` `(value_ref, value)`. `value` is a tuple + `(solver, A, b_bc)`, where `solver` is the linear solver, `A` is + the assembled matrix, and `b_bc` is a boundary condition + right-hand-side term which should be added after assembling a + right-hand-side with homogeneous boundary conditions applied. + `b_bc` may be `None` to indicate that this term is zero. + `value_ref` is a :class:`.CacheRef` storing a reference to `value`. + + + +.. py:function:: assembly_cache() + + :returns: The default :class:`.AssemblyCache`. + + +.. py:function:: set_assembly_cache(assembly_cache) + + Set the default :class:`.AssemblyCache`. + + :arg assembly_cache: The new default :class:`.AssemblyCache`. + + +.. py:function:: linear_solver_cache() + + :returns: The default :class:`.LinearSolverCache`. + + +.. py:function:: set_linear_solver_cache(linear_solver_cache) + + Set the default :class:`.LinearSolverCache`. + + :arg linear_solver_cache: The new default :class:`.LinearSolverCache`. + + diff --git a/_sources/autoapi/tlm_adjoint/firedrake/equations/index.rst.txt b/_sources/autoapi/tlm_adjoint/firedrake/equations/index.rst.txt new file mode 100644 index 0000000..d8a8181 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/firedrake/equations/index.rst.txt @@ -0,0 +1,504 @@ +:py:mod:`tlm_adjoint.firedrake.equations` +========================================= + +.. py:module:: tlm_adjoint.firedrake.equations + +.. autoapi-nested-parse:: + + This module implements finite element calculations. In particular the + :class:`.EquationSolver` class implements the solution of finite element + variational problems. + + + +Module Contents +--------------- + +.. py:class:: Assembly(x, rhs, *, form_compiler_parameters=None, match_quadrature=None) + + + + + Represents assignment to the result of finite element assembly: + + .. code-block:: python + + x = assemble(rhs) + + The forward residual :math:`\mathcal{F}` is defined so that :math:`\partial + \mathcal{F} / \partial x` is the identity. + + :arg x: A variable defining the forward solution. + :arg rhs: A :class:`ufl.Form` to assemble. Should have arity 0 or 1, and + should not depend on `x`. + :arg form_compiler_parameters: Form compiler parameters. + :arg match_quadrature: Whether to set quadrature parameters consistently in + the forward, adjoint, and tangent-linears. Defaults to + `parameters['tlm_adjoint']['Assembly']['match_quadrature']`. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: EquationSolver(eq, x, bcs=None, *, J=None, form_compiler_parameters=None, solver_parameters=None, adjoint_solver_parameters=None, tlm_solver_parameters=None, cache_jacobian=None, cache_adjoint_jacobian=None, cache_tlm_jacobian=None, cache_rhs_assembly=None, match_quadrature=None) + + + + + Represents the solution of a finite element variational problem. + + Caching is based on the approach described in + + - J. R. Maddison and P. E. Farrell, 'Rapid development and adjoining of + transient finite element models', Computer Methods in Applied + Mechanics and Engineering, 276, 95--121, 2014, doi: + 10.1016/j.cma.2014.03.010 + + The arguments `eq`, `x`, `bcs`, `J`, `form_compiler_parameters`, and + `solver_parameters` are based on the interface for the DOLFIN + `dolfin.solve` function (see e.g. FEniCS 2017.1.0). + + :arg eq: A :class:`ufl.equation.Equation` defining the finite element + variational problem. + :arg x: A :class:`firedrake.function.Function` defining the forward + solution. + :arg bcs: Dirichlet boundary conditions. + :arg J: A :class:`ufl.Form` defining a Jacobian matrix approximation to use + in a non-linear forward solve. + :arg form_compiler_parameters: Form compiler parameters. + :arg solver_parameters: Linear or non-linear solver parameters. + :arg adjoint_solver_parameters: Linear solver parameters to use in an + adjoint solve. + :arg tlm_solver_parameters: Linear solver parameters to use when solving + tangent-linear problems. + :arg cache_jacobian: Whether to cache the forward Jacobian matrix and + linear solver data. Defaults to + `parameters['tlm_adjoint']['EquationSolver]['cache_jacobian']`. If + `None` then caching is autodetected. + :arg cache_adjoint_jacobian: Whether to cache the adjoint Jacobian matrix + and linear solver data. Defaults to `cache_jacobian`. + :arg cache_tlm_jacobian: Whether to cache the Jacobian matrix and linear + solver data when solving tangent-linear problems. Defaults to + `cache_jacobian`. + :arg cache_rhs_assembly: Whether to enable right-hand-side caching. If + enabled then right-hand-side terms are divided into terms which are + cached, terms which are converted into matrix multiplication by a + cached matrix, and terms which are not cached. Defaults to + `parameters['tlm_adjoint']['EquationSolver']['cache_rhs_assembly']`. + :arg match_quadrature: Whether to set quadrature parameters consistently in + the forward, adjoint, and tangent-linears. Defaults to + `parameters['tlm_adjoint']['EquationSolver']['match_quadrature']`. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: subtract_adjoint_derivative_actions(adj_x, nl_deps, dep_Bs) + + Subtract terms from other adjoint right-hand-sides. + + Can be overridden for an optimized implementation, but otherwise uses + :meth:`.Equation.adjoint_derivative_action`. + + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_Bs: A :class:`Mapping` whose items are `(dep_index, dep_B)`. + Each `dep_B` is an :class:`.AdjointRHS` which should be updated by + subtracting adjoint derivative information computed by + differentiating with respect to `self.dependencies()[dep_index]`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:function:: expr_new_x(expr, x, *, annotate=None, tlm=None) + + If an expression depends on `x`, then record the assignment `x_old = + x`, and replace `x` with `x_old` in the expression. + + :arg expr: A :class:`ufl.core.expr.Expr`. + :arg x: Defines `x`. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + :returns: A :class:`ufl.core.expr.Expr` with `x` replaced with `x_old`, or + `expr` if the expression does not depend on `x`. + + +.. py:function:: linear_equation_new_x(eq, x, *, annotate=None, tlm=None) + + If a symbolic expression for a linear finite element variational problem + depends on the symbolic variable representing the problem solution `x`, + then record the assignment `x_old = x`, and replace `x` with `x_old` in the + symbolic expression. + + :arg eq: A :class:`ufl.equation.Equation` defining the finite element + variational problem. + :arg x: A :class:`firedrake.function.Function` defining the solution to the + finite element variational problem. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + :returns: A :class:`ufl.equation.Equation` with `x` replaced with `x_old`, + or `eq` if the symbolic expression does not depend on `x`. + + +.. py:class:: Projection(x, rhs, *args, **kwargs) + + + + + Represents the solution of a finite element variational problem + performing a projection onto the space for `x`. + + :arg x: A :class:`firedrake.function.Function` defining the forward + solution. + :arg rhs: A :class:`ufl.core.expr.Expr` defining the expression to project + onto the space for `x`, or a :class:`ufl.form.BaseForm` defining the + right-hand-side of the finite element variational problem. Should not + depend on `x`. + + Remaining arguments are passed to the :class:`.EquationSolver` constructor. + + +.. py:class:: DirichletBCApplication(x, y, *args, **kwargs) + + + + + Represents the application of a Dirichlet boundary condition to a zero + valued :class:`firedrake.function.Function`. Specifically this represents: + + .. code-block:: python + + x.zero() + DirichletBC(x.function_space(), y, *args, **kwargs).apply(x) + + The forward residual :math:`\mathcal{F}` is defined so that :math:`\partial + \mathcal{F} / \partial x` is the identity. + + :arg x: A :class:`firedrake.function.Function`, updated by the above + operations. + :arg y: A :class:`firedrake.function.Function`, defines the Dirichet + boundary condition. + + Remaining arguments are passed to the :class:`firedrake.bcs.DirichletBC` + constructor. + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: ExprInterpolation(x, rhs) + + + + + Represents interpolation of `rhs` onto the space for `x`. + + The forward residual :math:`\mathcal{F}` is defined so that :math:`\partial + \mathcal{F} / \partial x` is the identity. + + :arg x: The forward solution. + :arg rhs: A :class:`ufl.core.expr.Expr` defining the expression to + interpolate onto the space for `x`. Should not depend on `x`. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + diff --git a/_sources/autoapi/tlm_adjoint/firedrake/firedrake_equations/index.rst.txt b/_sources/autoapi/tlm_adjoint/firedrake/firedrake_equations/index.rst.txt new file mode 100644 index 0000000..c163edb --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/firedrake/firedrake_equations/index.rst.txt @@ -0,0 +1,329 @@ +:py:mod:`tlm_adjoint.firedrake.firedrake_equations` +=================================================== + +.. py:module:: tlm_adjoint.firedrake.firedrake_equations + +.. autoapi-nested-parse:: + + This module includes additional functionality for use with Firedrake. + + + +Module Contents +--------------- + +.. py:class:: LocalSolverCache + + + + + A :class:`.Cache` for element-wise local block diagonal linear solver + data. + + .. py:method:: local_solver(form, *, form_compiler_parameters=None, replace_map=None) + + Compute data for an element-wise local block diagonal linear + solver and cache the result, or return a previously cached result. + + :arg form: An arity two :class:`ufl.Form`, defining the element-wise + local block diagonal matrix. + :arg form_compiler_parameters: Form compiler parameters. + :arg replace_map: A :class:`Mapping` defining a map from symbolic + variables to values. + :returns: A :class:`tuple` `(value_ref, value)`. `value` is a + :class:`firedrake.matrix.Matrix` storing the assembled inverse + matrix, and `value_ref` is a :class:`.CacheRef` storing a reference + to `value`. + + + +.. py:function:: local_solver_cache() + + :returns: The default :class:`.LocalSolverCache`. + + +.. py:function:: set_local_solver_cache(local_solver_cache) + + Set the default :class:`.LocalSolverCache`. + + :arg local_solver_cache: The new default :class:`.LocalSolverCache`. + + +.. py:class:: LocalProjection(x, rhs, *, form_compiler_parameters=None, cache_jacobian=None, cache_rhs_assembly=None, match_quadrature=None) + + + + + Represents the solution of a finite element variational problem + performing a projection onto the space for `x`, for the case where the mass + matrix is element-wise local block diagonal. + + :arg x: A :class:`firedrake.function.Function` defining the forward + solution. + :arg rhs: A :class:`ufl.core.expr.Expr` defining the expression to project + onto the space for `x`, or a :class:`ufl.form.BaseForm` defining the + right-hand-side of the finite element variational problem. Should not + depend on `x`. + + Remaining arguments are passed to the + :class:`tlm_adjoint.firedrake.equations.EquationSolver` constructor. + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: PointInterpolation(X, y, X_coords=None, *, tolerance=None, _interp=None) + + + + + Represents interpolation of a scalar-valued function at given points. + + The forward residual :math:`\mathcal{F}` is defined so that :math:`\partial + \mathcal{F} / \partial x` is the identity. + + :arg X: A scalar variable, or a :class:`Sequence` of scalar variables, + defining the forward solution. + :arg y: A scalar-valued :class:`firedrake.function.Function` to + interpolate. + :arg X_coords: A :class:`numpy.ndarray` defining the coordinates at which + to interpolate `y`. Shape is `(n, d)` where `n` is the number of + interpolation points and `d` is the geometric dimension. Ignored if `P` + is supplied. + :arg tolerance: :class:`firedrake.mesh.VertexOnlyMesh` tolerance. + + .. py:method:: forward_solve(X, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_X) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_X, nl_deps, B) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: ExprAssignment(x, rhs, *, subset=None) + + + + + Represents an evaluation of `rhs`, storing the result in `x`. Uses + :meth:`firedrake.function.Function.assign` or + :meth:`firedrake.cofunction.Cofunction.assign` to perform the evaluation. + + The forward residual :math:`\mathcal{F}` is defined so that :math:`\partial + \mathcal{F} / \partial x` is the identity. + + :arg x: A :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` defining the forward solution. + :arg rhs: A :class:`ufl.core.expr.Expr` defining the expression to + evaluate. Should not depend on `x`. + :arg subset: A :class:`pyop2.types.set.Subset`. If provided then defines a + subset of degrees of freedom at which to evaluate `rhs`. Other degrees + of freedom are set to zero. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + diff --git a/_sources/autoapi/tlm_adjoint/firedrake/functions/index.rst.txt b/_sources/autoapi/tlm_adjoint/firedrake/functions/index.rst.txt new file mode 100644 index 0000000..c795eed --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/firedrake/functions/index.rst.txt @@ -0,0 +1,139 @@ +:py:mod:`tlm_adjoint.firedrake.functions` +========================================= + +.. py:module:: tlm_adjoint.firedrake.functions + +.. autoapi-nested-parse:: + + This module includes functionality for interacting with Firedrake variables + and Dirichlet boundary conditions. + + + +Module Contents +--------------- + +.. py:class:: Constant(value=None, *args, name=None, domain=None, space=None, space_type='primal', shape=None, comm=None, static=False, cache=None, **kwargs) + + + + + Extends the :class:`firedrake.constant.Constant` class. + + :arg value: The initial value. `None` indicates a value of zero. + :arg name: A :class:`str` name. + :arg domain: The domain on which the :class:`.Constant` is defined. + :arg space: The space on which the :class:`.Constant` is defined. + :arg space_type: The space type for the :class:`.Constant`. `'primal'`, + `'dual'`, `'conjugate'`, or `'conjugate_dual'`. + :arg shape: A :class:`tuple` of :class:`int` objects defining the shape of + the value. + :arg comm: The communicator for the :class:`.Constant`. + :arg static: Defines whether the :class:`.Constant` is static, meaning that + it is stored by reference in checkpointing/replay, and an associated + tangent-linear variable is zero. + :arg cache: Defines whether results involving the :class:`.Constant` may be + cached. Default `static`. + + Remaining arguments are passed to the :class:`firedrake.constant.Constant` + constructor. + + +.. py:class:: Zero + + + Mixin for defining a zero-valued variable. Used for zero-valued + variables for which UFL zero elimination should not be applied. + + +.. py:class:: ZeroConstant(*, name=None, domain=None, space=None, space_type='primal', shape=None, comm=None) + + + + + A :class:`.Constant` which is flagged as having a value of zero. + + Arguments are passed to the :class:`.Constant` constructor, together with + `static=True` and `cache=True`. + + +.. py:function:: eliminate_zeros(expr) + + Apply zero elimination for :class:`.Zero` objects in the supplied + :class:`ufl.core.expr.Expr` or :class:`ufl.form.BaseForm`. + + :arg expr: A :class:`ufl.core.expr.Expr` or :class:`ufl.form.BaseForm`. + :returns: A :class:`ufl.core.expr.Expr` or :class:`ufl.form.BaseForm` with + zero elimination applied. May return `expr`. + + +.. py:class:: DirichletBC(V, g, sub_domain, *args, static=None, _homogeneous=False, **kwargs) + + + + + Extends the :class:`firedrake.bcs.DirichletBC` class. + + :arg static: A flag that indicates that the value for the + :class:`.DirichletBC` will not change, and which determines whether + calculations involving this :class:`.DirichletBC` can be cached. If + `None` then autodetected from the value. + + Remaining arguments are passed to the :class:`firedrake.bcs.DirichletBC` + constructor. + + +.. py:class:: HomogeneousDirichletBC(V, sub_domain, *args, **kwargs) + + + + + A :class:`.DirichletBC` whose value is zero. + + Arguments are passed to the :class:`.DirichletBC` constructor, together + with `static=True`. + + +.. py:class:: Replacement + + + Represents a symbolic variable but with no value. + + + +.. py:class:: ReplacementConstant(x, count) + + + + + Represents a symbolic :class:`firedrake.constant.Constant`, but has no + value. + + +.. py:class:: ReplacementFunction(x, count) + + + + + Represents a symbolic :class:`firedrake.function.Function`, but has no + value. + + +.. py:class:: ReplacementZeroConstant(*args, **kwargs) + + + + + Represents a symbolic :class:`firedrake.constant.Constant` which is + zero, but has no value. + + +.. py:class:: ReplacementZeroFunction(*args, **kwargs) + + + + + Represents a symbolic :class:`firedrake.function.Function` which is + zero, but has no value. + + diff --git a/_sources/autoapi/tlm_adjoint/firedrake/hessian_system/index.rst.txt b/_sources/autoapi/tlm_adjoint/firedrake/hessian_system/index.rst.txt new file mode 100644 index 0000000..47ff48b --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/firedrake/hessian_system/index.rst.txt @@ -0,0 +1,181 @@ +:orphan: + +:py:mod:`tlm_adjoint.firedrake.hessian_system` +============================================== + +.. py:module:: tlm_adjoint.firedrake.hessian_system + + +Module Contents +--------------- + +.. py:class:: HessianSystem(H, M, *, nullspace=None, comm=None) + + + + + Defines a linear system involving a Hessian matrix, + + .. math:: + + H u = b. + + :arg H: A :class:`.Hessian` defining :math:`H`. + :arg M: A :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction`, or a :class:`Sequence` of + :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` objects, defining the control. + :arg nullspace: A :class:`.Nullspace` or a :class:`Sequence` of + :class:`.Nullspace` objects defining the nullspace and left nullspace + of the Hessian matrix. `None` indicates a :class:`.NoneNullspace`. + :arg comm: A communicator. + + .. py:method:: solve(u, b, **kwargs) + + Solve a linear system involving a Hessian matrix, + + .. math:: + + H u = b. + + :arg u: A :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction`, or a :class:`Sequence` of + :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` objects, defining the + solution :math:`u`. + :arg b: A :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction`, or a :class:`Sequence` of + :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` objects, defining the + conjugate of the right-hand-side :math:`b`. + + Remaining arguments are handed to the base class + :meth:`.System.solve` method. + + + +.. py:function:: hessian_eigendecompose(H, m, B_inv_action, B_action, *, nullspace=None, problem_type=None, pre_callback=None, correct_eigenvectors=True, **kwargs) + + Interface with SLEPc via slepc4py, for the matrix free solution of + generalized eigenproblems + + .. math:: + + H v = \lambda B^{-1} v, + + where :math:`H` is a Hessian matrix. + + Despite the notation :math:`B^{-1}` may be singular, defining an inverse + operator only on an appropriate subspace. + + :arg H: A :class:`.Hessian`. + :arg m: A :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` defining the control. + :arg B_inv_action: A callable accepting a + :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` defining :math:`v` and + computing the conjugate of the action of :math:`B^{-1}` on :math:`v`, + returning the result as a :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction`. + :arg B_action: A callable accepting a :class:`firedrake.function.Function` + or :class:`firedrake.cofunction.Cofunction` defining :math:`v` and + computing the action of :math:`B` on the conjugate of :math:`v`, + returning the result as a :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction`. + :arg nullspace: A :class:`.Nullspace` defining the nullspace and left + nullspace of :math:`H` and :math:`B^{-1}`. + :arg problem_type: The eigenproblem type -- see + `slepc4py.SLEPc.EPS.ProblemType`. Defaults to + `slepc4py.SLEPc.EPS.ProblemType.GHEP` in the real case and + `slepc4py.SLEPc.EPS.ProblemType.GNHEP` in the complex case. + :arg pre_callback: A callable accepting a single `slepc4py.SLEPc.EPS` + argument. Used for detailed manual configuration. Called after all + other configuration options are set, but before the + `slepc4py.SLEPc.EPS.setUp` method is called. + :arg correct_eigenvectors: Whether to apply a nullspace correction to the + eigenvectors. + + Remaining keyword arguments are passed to :func:`.eigendecompose`. + + +.. py:function:: B_inv_orthonormality_test(V, B_inv_action) + + Check for :math:`B^{-1}`-orthonormality. + + Requires real spaces. + + :arg B_inv_action: A callable accepting a + :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` defining :math:`v` and + computing the action of :math:`B^{-1}` on :math:`v`, returning the + result as a :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction`. + :arg V: A :class:`Sequence` of :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` objects to test for + :math:`B^{-1}`-orthonormality. + :returns: A :class:`tuple` `(max_diagonal_error_norm, + max_off_diagonal_error_norm)` with + + - `max_diagonal_error_norm`: The maximum :math:`B^{-1}` + normalization error magnitude. + - `max_diagonal_error_norm`: The maximum :math:`B^{-1}` + orthogonality error magnitude. + + +.. py:function:: hessian_eigendecomposition_pc(B_action, Lam, V) + + Construct a Hessian matrix preconditioner using a partial spectrum + generalized eigendecomposition. Assumes that the Hessian matrix consists of + two terms + + .. math:: + + H = R^{-1} + B^{-1}, + + where :math:`R` and :math:`B` are symmetric. + + Assumes real spaces. Despite the notation :math:`R^{-1}` and :math:`B^{-1}` + (and later :math:`H^{-1}`) may be singular, defining inverse operators only + on an appropriate subspace. :math:`B` is assumed to define a symmetric + positive definite operator on that subspace. + + The approximation is defined via + + .. math:: + + H^{-1} \approx B + V \Lambda \left( I + \Lambda \right)^{-1} V^T + + where + + .. math:: + + R^{-1} V = B^{-1} V \Lambda, + + and where :math:`\Lambda` is a diagonal matrix and :math:`V` has + :math:`B^{-1}`-orthonormal columns, :math:`V^T B^{-1} V = I`. + + This low rank update approximation for the Hessian matrix inverse is + described in + + - Tobin Isaac, Noemi Petra, Georg Stadler, and Omar Ghattas, 'Scalable + and efficient algorithms for the propagation of uncertainty from data + through inference to prediction for large-scale problems, with + application to flow of the Antarctic ice sheet', Journal of + Computational Physics, 296, pp. 348--368, 2015, doi: + 10.1016/j.jcp.2015.04.047 + + See in particular their equation (20). + + :arg B_action: A callable accepting a :class:`firedrake.function.Function` + or :class:`firedrake.cofunction.Cofunction` defining :math:`v` and + computing the action of :math:`B` on :math:`v`, returning the result as + a :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction`. + :arg Lam: A :class:`Sequence` defining the diagonal of :math:`\Lambda`. + :arg V: A :class:`Sequence` of :class:`firedrake.function.Function` or + :class:`firedrake.cofunction.Cofunction` objects defining the columns + of :math:`V`. + :returns: A callable suitable for use as the `pc_fn` argument to + :meth:`.HessianSystem.solve`. + + diff --git a/_sources/autoapi/tlm_adjoint/fixed_point/index.rst.txt b/_sources/autoapi/tlm_adjoint/fixed_point/index.rst.txt new file mode 100644 index 0000000..27611a2 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/fixed_point/index.rst.txt @@ -0,0 +1,160 @@ +:orphan: + +:py:mod:`tlm_adjoint.fixed_point` +================================= + +.. py:module:: tlm_adjoint.fixed_point + + +Module Contents +--------------- + +.. py:class:: CustomNormSq(eqs, *, norm_sqs=None, adj_norm_sqs=None) + + + Defines the square of the norm of forward and adjoint solutions. + + Callables are used to define squared norms for the forward and adjoint + solutions of equations. The total squared norm is then the sum of the + squares. + + :arg eqs: A :class:`Sequence` of :class:`.Equation` objects. + :arg norm_sqs: A :class:`Sequence`. Each element is either a callable, or a + :class:`Sequence` of callables. The callables define the squared norm + associated with the corresponding components of the forward solution + for the corresponding :class:`.Equation` in `eqs`. Each callable + accepts a single variable and returns a :class:`float`. Defaults to the + square of the :math:`l_2` norm of the degrees of freedom vector. + :arg adj_norm_sqs: A :class:`Sequence`. Each element is either a callable, + or a :class:`Sequence` of callables. The callables define the squared + norm associated with the corresponding components of the adjoint + solution for the corresponding :class:`.Equation` in `eqs`. Each + callable accepts a single variable and returns a :class:`float`. + Defaults to the square of the :math:`l_2` norm of the degrees of + freedom vector. + + +.. py:class:: FixedPointSolver(eqs, solver_parameters, *, norm_sqs=None, adj_norm_sqs=None) + + + + + A fixed-point solver. Solves the given equations in sequence until + either an absolute or relative tolerance is reached. + + Derives tangent-linear and adjoint information using the approach described + in: + + - Jean Charles Gilbert, 'Automatic differentiation and iterative + processes', Optimization Methods and Software, 1(1), pp. 13--21, + 1992, doi: 10.1080/10556789208805503 + - Bruce Christianson, 'Reverse accumulation and attractive fixed + points', Optimization Methods and Software, 3(4), pp. 311--326, 1994, + doi: 10.1080/10556789408805572 + + :arg eqs: A :class:`Sequence` of :class:`.Equation` objects. One forward + iteration consists of computing, in order, a forward solution for all + :class:`.Equation` objects. + :arg solver_parameters: A :class:`Mapping` defining solver parameters. + Parameters (a number of which are based on KrylovSolver parameters in + FEniCS 2017.2.0) are: + + - absolute_tolerance: A :class:`float` defining the absolute + tolerance for a change in the solution in one iteration. + Required. + - relative_tolerance: A :class:`float` defining the relative + tolerance for a change in the solution in one iteration. + Required. + - maximum_iterations: An :class:`int` defining the maximum + permitted iterations. Defaults to 1000. + - nonzero_initial_guess: A :class:`bool` indicating whether to use + a non-zero initial guess in a forward solve. Defaults to `True`. + - adjoint_nonzero_initial_guess: A :class:`bool` indicating whether + to use a non-zero initial guess in an adjoint solve. Defaults to + `True`. + + :arg norm_sqs: Defines the squared norm used to test for convergence in a + forward solve. See :class:`.CustomNormSq`. + :arg adj_norm_sqs: Defines the squared norm used to test for convergence in + an adjoint solve. See :class:`.CustomNormSq`. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: forward_solve(X, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_jacobian_solve(adj_X, nl_deps, B) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: subtract_adjoint_derivative_actions(adj_X, nl_deps, dep_Bs) + + Subtract terms from other adjoint right-hand-sides. + + Can be overridden for an optimized implementation, but otherwise uses + :meth:`.Equation.adjoint_derivative_action`. + + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_Bs: A :class:`Mapping` whose items are `(dep_index, dep_B)`. + Each `dep_B` is an :class:`.AdjointRHS` which should be updated by + subtracting adjoint derivative information computed by + differentiating with respect to `self.dependencies()[dep_index]`. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + diff --git a/_sources/autoapi/tlm_adjoint/functional/index.rst.txt b/_sources/autoapi/tlm_adjoint/functional/index.rst.txt new file mode 100644 index 0000000..9992921 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/functional/index.rst.txt @@ -0,0 +1,42 @@ +:orphan: + +:py:mod:`tlm_adjoint.functional` +================================ + +.. py:module:: tlm_adjoint.functional + + +Module Contents +--------------- + +.. py:class:: Functional(*args, space=None, **kwargs) + + + + + A convenience class for defining functionals. + + Arguments are as for the :class:`.Float` class. + + .. py:method:: assign(y, *, annotate=None, tlm=None) + + Assign to the :class:`.Functional`. + + :arg y: The value. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + :returns: The :class:`.Functional`. + + + .. py:method:: addto(y, *, annotate=None, tlm=None) + + Add to the :class:`.Functional`. + + :arg y: The value to add. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + + + diff --git a/_sources/autoapi/tlm_adjoint/hessian/index.rst.txt b/_sources/autoapi/tlm_adjoint/hessian/index.rst.txt new file mode 100644 index 0000000..4f51a6b --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/hessian/index.rst.txt @@ -0,0 +1,209 @@ +:orphan: + +:py:mod:`tlm_adjoint.hessian` +============================= + +.. py:module:: tlm_adjoint.hessian + + +Module Contents +--------------- + +.. py:class:: Hessian + + + + + Represents a Hessian associated with a given forward. Abstract base + class. + + .. py:method:: compute_gradient(M, M0=None) + :abstractmethod: + + Compute the (conjugate of the) derivative of a functional with + respect to a control using an adjoint. + + :arg M: A variable or a :class:`Sequence` of variables defining the + control. + :arg M0: A variable or a :class:`Sequence` of variables defining the + control value. `M` is used if not supplied. + :returns: The (conjugate of the) derivative. A variable or + :class:`Sequence` of variables, depending on the type of `M`. + + + .. py:method:: action(M, dM, M0=None) + :abstractmethod: + + Compute (the conjugate of) a Hessian action on some :math:`\zeta` + using an adjoint of a tangent-linear. i.e. considering derivatives to + be row vectors, compute + + .. math:: + + \left( \frac{d}{dm} \left[ + \frac{d \mathcal{J}}{d m} \zeta \right] \right)^{*,T}. + + :arg M: A variable or a :class:`Sequence` of variables defining the + control. + :arg dM: A variable or a :class:`Sequence` of variables defining + :math:`\zeta`. The (conjugate of the) Hessian action on + :math:`\zeta` is computed. + :arg M0: A variable or a :class:`Sequence` of variables defining the + control value. `M` is used if not supplied. + :returns: A tuple `(J, dJ, ddJ)`. `J` is the value of the functional. + `dJ` is the value of :math:`\left( d \mathcal{J} / d m \right) + \zeta`. `ddJ` stores the (conjugate of the) result of the Hessian + action on :math:`\zeta`, and is a variable or a :class:`Sequence` + of variables depending on the type of `M`. + + + .. py:method:: action_fn(m, m0=None) + + Return a callable which can be used to compute Hessian actions. + + :arg m: A variable defining the control. + :arg m0: A variable defining the control value. `m` is used if not + supplied. + :returns: A callable which accepts a single variable argument, and + returns the result of the Hessian action on that argument as a + variable. Note that the result is *not* the conjugate of the + Hessian action on the input argument. + + + +.. py:class:: GeneralHessian(forward, *, manager=None) + + + + + Represents a Hessian associated with a given forward. Calls to + :meth:`.GeneralHessian.compute_gradient` or :meth:`.GeneralHessian.action` + re-run the forward. + + :arg forward: A callable which accepts one or more variable arguments, and + which returns a variable defining the forward functional. + :arg manager: An :class:`.EquationManager` used to create an internal + manager via :meth:`.EquationManager.new`. `manager()` is used if not + supplied. + + .. py:method:: compute_gradient(M, M0=None) + + Compute the (conjugate of the) derivative of a functional with + respect to a control using an adjoint. + + :arg M: A variable or a :class:`Sequence` of variables defining the + control. + :arg M0: A variable or a :class:`Sequence` of variables defining the + control value. `M` is used if not supplied. + :returns: The (conjugate of the) derivative. A variable or + :class:`Sequence` of variables, depending on the type of `M`. + + + .. py:method:: action(M, dM, M0=None) + + Compute (the conjugate of) a Hessian action on some :math:`\zeta` + using an adjoint of a tangent-linear. i.e. considering derivatives to + be row vectors, compute + + .. math:: + + \left( \frac{d}{dm} \left[ + \frac{d \mathcal{J}}{d m} \zeta \right] \right)^{*,T}. + + :arg M: A variable or a :class:`Sequence` of variables defining the + control. + :arg dM: A variable or a :class:`Sequence` of variables defining + :math:`\zeta`. The (conjugate of the) Hessian action on + :math:`\zeta` is computed. + :arg M0: A variable or a :class:`Sequence` of variables defining the + control value. `M` is used if not supplied. + :returns: A tuple `(J, dJ, ddJ)`. `J` is the value of the functional. + `dJ` is the value of :math:`\left( d \mathcal{J} / d m \right) + \zeta`. `ddJ` stores the (conjugate of the) result of the Hessian + action on :math:`\zeta`, and is a variable or a :class:`Sequence` + of variables depending on the type of `M`. + + + +.. py:class:: GaussNewton(R_inv_action, B_inv_action=None) + + + + + Represents a Gauss-Newton approximation for a Hessian. Abstract base + class. + + In terms of matrices this defines a Hessian approximation + + .. math:: + + H = J^T R_\text{obs}^{-1} J + B^{-1}, + + where :math:`J` is the forward Jacobian. In a variational assimilation + approach :math:`R_\text{obs}^{-1}` corresponds to the observational inverse + covariance and :math:`B^{-1}` corresponds to the background inverse + covariance. + + :arg R_inv_action: A callable which accepts one or more variables, and + returns the conjugate of the action of the operator corresponding to + :math:`R_\text{obs}^{-1}` on those variables, returning the result as a + variable or a :class:`Sequence` of variables. + :arg B_inv_action: A callable which accepts one or more variables, and + returns the conjugate of the action of the operator corresponding to + :math:`B^{-1}` on those variables, returning the result as a variable + or a :class:`Sequence` of variables. + + .. py:method:: action(M, dM, M0=None) + + Compute (the conjugate of) a Hessian action on some :math:`\zeta`, + using the Gauss-Newton approximation for the Hessian. i.e. compute + + .. math:: + + \left( H \zeta \right)^{*,T}. + + :arg M: A variable or a :class:`Sequence` of variables defining the + control. + :arg dM: A variable or a :class:`Sequence` of variables defining + :math:`\zeta`. The (conjugate of the) approximated Hessian action + on :math:`\zeta` is computed. + :arg M0: A variable or a :class:`Sequence` of variables defining the + control value. `M` is used if not supplied. + :returns: The (conjugate of the) result of the approximated Hessian + action on :math:`\zeta`. A variable or a :class:`Sequence` of + variables depending on the type of `M`. + + + .. py:method:: action_fn(m, m0=None) + + Return a callable which can be used to compute Hessian actions using + the Gauss-Newton approximation. + + :arg m: A variable defining the control. + :arg m0: A variable defining the control value. `m` is used if not + supplied. + :returns: A callable which accepts a single variable argument, and + returns the result of the approximated Hessian action on that + argument as a variable. Note that the result is *not* the conjugate + of the approximated Hessian action on the input argument. + + + +.. py:class:: GeneralGaussNewton(forward, R_inv_action, B_inv_action=None, *, manager=None) + + + + + Represents a Gauss-Newton approximation to a Hessian associated with a + given forward. Calls to :meth:`.GaussNewton.action` re-run the forward. + + :arg forward: A callable which accepts one or more variable arguments, and + which returns a variable or :class:`Sequence` of variables defining the + state. + :arg R_inv_action: See :class:`.GaussNewton`. + :arg B_inv_action: See :class:`.GaussNewton`. + :arg manager: An :class:`.EquationManager` used to create an internal + manager via :meth:`.EquationManager.new`. `manager()` is used if not + supplied. + + diff --git a/_sources/autoapi/tlm_adjoint/index.rst.txt b/_sources/autoapi/tlm_adjoint/index.rst.txt new file mode 100644 index 0000000..2a4ca2e --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/index.rst.txt @@ -0,0 +1,20 @@ +:orphan: + +:py:mod:`tlm_adjoint` +===================== + +.. py:module:: tlm_adjoint + + +Submodules +---------- +.. toctree:: + :titlesonly: + :maxdepth: 1 + + interface/index.rst + manager/index.rst + overloaded_float/index.rst + verification/index.rst + + diff --git a/_sources/autoapi/tlm_adjoint/instructions/index.rst.txt b/_sources/autoapi/tlm_adjoint/instructions/index.rst.txt new file mode 100644 index 0000000..22ee258 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/instructions/index.rst.txt @@ -0,0 +1,109 @@ +:orphan: + +:py:mod:`tlm_adjoint.instructions` +================================== + +.. py:module:: tlm_adjoint.instructions + + +Module Contents +--------------- + +.. py:class:: Instruction + + + + + An adjoint tape record which defines instructions to be performed during + forward or adjoint calculations. + + .. py:method:: adjoint_jacobian_solve(adj_X, nl_deps, B) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + +.. py:class:: GarbageCollection(comm=None, *, generation=2, garbage_cleanup=True) + + + + + An :class:`.Instruction` which indicates that garbage collection should + be performed during forward and adjoint calculations. + + :arg comm: Communicator to use for PETSc garbage cleanup. + :arg generation: Python garbage collection generation. If a value of `None` + is provided then Python garbage collection is not performed. + :arg garbage_cleanup: Whether to perform PETSc garbage cleanup. + + .. py:method:: forward_solve(X, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_jacobian_solve(adj_X, nl_deps, B) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + diff --git a/_sources/autoapi/tlm_adjoint/interface/index.rst.txt b/_sources/autoapi/tlm_adjoint/interface/index.rst.txt new file mode 100644 index 0000000..3862f12 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/interface/index.rst.txt @@ -0,0 +1,658 @@ +:py:mod:`tlm_adjoint.interface` +=============================== + +.. py:module:: tlm_adjoint.interface + +.. autoapi-nested-parse:: + + This module defines an interface for interaction with backend data types. + This is implemented via runtime binding of mixins. The + :class:`.VariableInterface` adds methods to 'variables' which can be used to + interact with backend variables. The :class:`.SpaceInterface` adds methods to + 'spaces' which define the vector spaces in which those 'variables' are defined. + + The extra methods are accessed using the callables defined in this module + (which also handle some extra details, e.g. related to cache invalidation and + space type checking). Typically these are prefixed with `space_` for spaces and + `var_` for variables. + + The interface distinguishes between original backend 'variables', which both + define symbolic variables and store values, and replacement 'variables', which + define the same variables but which need not store values. + + Variables have an associated 'space type', which indicates e.g. if the variable + is 'primal', meaning a member on an originating vector space, or 'conjugate + dual', meaning a member of the corresponding antidual space of antilinear + functionals from the originating vector space. Variables can also be 'dual', + meaning a member of the dual space of linear functionals, or 'conjugate', + meaning a member of a space defined by a conjugate operator from the primal + space. This conjugate operator is defined by complex conjugation of the vector + of degrees of freedom, and could e.g. correspond to complex conjugation of a + finite element discretized function. + + The space type associated with a variable is defined relative to an originating + vector space (e.g. a finite element discrete function space). A 'relative space + type' is defined relative to one of the 'primal', 'conjugate', 'dual', or + 'conjugate dual' spaces. For example the primal space associated with the dual + space is the dual space, and the dual space associated with the dual space is + the primal space. + + This module defines a default communicator `DEFAULT_COMM`, which is + `mpi4py.MPI.COMM_WORLD` if mpi4py is available. If mpi4py is not available a + dummy 'serial' communicator is used, of type :class:`.SerialComm`. + + + +Module Contents +--------------- + + + +.. py:function:: comm_dup_cached(comm, *, key=None) + + If the communicator `comm` with key `key` has previously been duplicated + using :func:`.comm_dup_cached`, then return the previous result. Otherwise + duplicate the communicator and cache the result. The duplicated + communicator is freed when the original base communicator is freed. + + :arg comm: A communicator, the base communicator to be duplicated. + :arg key: The key. + :returns: A communicator. A duplicated MPI communicator, or a previously + cached duplicated MPI communicator, which is freed when the original + base communicator is freed. + + +.. py:function:: garbage_cleanup(comm=None) + + Call `petsc4py.PETSc.garbage_cleanup(comm)` for a communicator, any + communicators duplicated from it, base communicators from which it was + duplicated, and any communicators duplicated from those base communicators. + + :arg comm: A communicator. Defaults to `DEFAULT_COMM`. + + +.. py:function:: add_interface(obj, interface_cls, attrs=None) + + Attach a mixin `interface_cls`, defining an interface, to `obj`. + + :arg obj: An object to which the mixin should be attached. + :arg interface_cls: A subclass of :class:`.SpaceInterface` or + :class:`.VariableInterface` defining the interface. + :arg attrs: A :class:`Mapping` defining any attributes. Used to set an + attribute `_tlm_adjoint__space_interface_attrs` (for a + :class:`.SpaceInterface`) or `_tlm_adjoint__var_interface_attrs` + (for a :class:`.VariableInterface`). + + +.. py:class:: SpaceInterface + + + A mixin defining an interface for spaces. Space types do not inherit + from this class -- instead an interface is defined by a + :class:`.SpaceInterface` subclass, and methods are bound dynamically at + runtime using :func:`.add_interface`. + + +.. py:function:: is_space(space) + + Return whether `space` is a space -- i.e. has had a + :class:`.SpaceInterface` attached. + + :arg space: An arbitrary :class:`object`. + :returns: `True` if `space` is a space, and `False` otherwise. + + +.. py:function:: space_comm(space) + + :arg space: A space. + :returns: The communicator associated with the space. + + +.. py:function:: space_dtype(space) + + :arg space: A space. + :returns: The data type associated with the space. Typically + :class:`numpy.double` or :class:`numpy.cdouble`. + + +.. py:function:: space_id(space) + + Return a unique :class:`int` ID associated with a space. + + :arg space: The space. + :returns: The unique :class:`int` ID. + + +.. py:function:: space_new(space, *, name=None, space_type='primal', static=False, cache=None) + + Return a new variable. + + :arg space: The space. + :arg name: A :class:`str` name for the variable. + :arg space_type: The space type for the new variable. `'primal'`, `'dual'`, + `'conjugate'`, or `'conjugate_dual'`. + :arg static: Defines whether the new variable is static, meaning that it is + stored by reference in checkpointing/replay, and an associated + tangent-linear variable is zero. + :arg cache: Defines whether results involving the new variable may be + cached. Default `static`. + :returns: The new variable. + + +.. py:function:: relative_space_type(space_type, rel_space_type) + + Return a relative space type. For example if `space_type` is `'dual'` + and `rel_space_type` is `'conjugate_dual'`, this returns `'conjugate'`. + + :arg space_type: An input space type. One of `'primal'`, `'conjugate'`, + `'dual'`, or `'conjugate_dual'`. + :arg rel_space_type: The relative space type to return. One of `'primal'`, + `'conjugate'`, `'dual'`, or `'conjugate_dual'`. + :returns: A space type relative to `space_type`. + + +.. py:function:: conjugate_space_type(space_type) + + Defines a map + + - `'primal'` :math:`\rightarrow` `'conjugate'` + - `'conjugate'` :math:`\rightarrow` `'primal'` + - `'dual'` :math:`\rightarrow` `'conjugate_dual'` + - `'conjugate_dual'` :math:`\rightarrow` `'dual'` + + :returns: The space type conjugate to `space_type`. + + +.. py:function:: dual_space_type(space_type) + + Defines a map + + - `'primal'` :math:`\rightarrow` `'dual'` + - `'conjugate'` :math:`\rightarrow` `'conjugate_dual'` + - `'dual'` :math:`\rightarrow` `'primal'` + - `'conjugate_dual'` :math:`\rightarrow` `'conjugate'` + + :returns: The space type dual to `space_type`. + + +.. py:function:: conjugate_dual_space_type(space_type) + + Defines a map + + - `'primal'` :math:`\rightarrow` `'conjugate_dual'` + - `'conjugate'` :math:`\rightarrow` `'dual'` + - `'dual'` :math:`\rightarrow` `'conjugate'` + - `'conjugate_dual'` :math:`\rightarrow` `'primal'` + + :returns: The space type conjugate dual to `space_type`. + + +.. py:function:: no_space_type_checking(fn) + + Decorator to disable space type checking. + + :arg fn: A callable for which space type checking should be disabled. + :returns: A callable for which space type checking is disabled. + + +.. py:function:: paused_space_type_checking() + + Construct a context manager which can be used to temporarily disable + space type checking. + + :returns: A context manager which can be used to temporarily disable + space type checking. + + +.. py:exception:: SpaceTypeError + + + + + Raised when an unexpected space type is encountered with space type + checking enabled. + + +.. py:function:: check_space_type(x, space_type) + + Check that a variable has a given space type. + + Raises a :class:`.SpaceTypeError` if the check fails and space type + checking is enabled. + + :arg x: A variable, whose space type should be checked. + :arg space_type: The space type. One of `'primal'`, `'conjugate'`, + `'dual'`, or `'conjugate_dual'`. + + +.. py:function:: check_space_types(x, y, *, rel_space_type='primal') + + Check that `x` and `y` have compatible space types. + + Raises a :class:`.SpaceTypeError` if the check fails and space type + checking is enabled. + + :arg x: A variable. + :arg y: A variable. + :arg rel_space_type: Check that the space type of `x` is `rel_space_type` + relative to `y`. For example if `rel_space_type='dual'`, and the + space type of `y` is `'conjuguate_dual'`, checks that the space type of + `x` is `'conjugate'`. + + +.. py:function:: check_space_types_conjugate(x, y) + + Check that `x` has space type conjugate to the space type for `y`. + + Raises a :class:`.SpaceTypeError` if the check fails and space type + checking is enabled. + + :arg x: A variable. + :arg y: A variable. + + +.. py:function:: check_space_types_dual(x, y) + + Check that `x` has space type dual to the space type for `y`. + + Raises a :class:`.SpaceTypeError` if the check fails and space type + checking is enabled. + + :arg x: A variable. + :arg y: A variable. + + +.. py:function:: check_space_types_conjugate_dual(x, y) + + Check that `x` has space type conjugate dual to the space type for `y`. + + Raises a :class:`.SpaceTypeError` if the check fails and space type + checking is enabled. + + :arg x: A variable. + :arg y: A variable. + + +.. py:class:: VariableInterface + + + A mixin defining an interface for variables. Variables types do not + inherit from this class -- instead an interface is defined by a + :class:`.VariableInterface` subclass, and methods are bound dynamically at + runtime using :func:`.add_interface`. + + +.. py:function:: is_var(x) + + Return whether `x` is a variable -- i.e. has had a + :class:`.VariableInterface` added. + + :arg x: An arbitrary :class:`object`. + :returns: `True` if `x` is a variable, and `False` otherwise. + + +.. py:function:: var_comm(x) + + :arg x: A variable. + :returns: The communicator associated with the variable. + + +.. py:function:: var_space(x) + + :arg x: A variable. + :returns: The space associated with the variable. + + +.. py:function:: var_space_type(x, *, rel_space_type='primal') + + Return the space type of a variable. + + :arg x: The variable. + :arg rel_space_type: If supplied then return a space type relative to the + variable space type. One of `'primal'`, `'conjugate'`, `'dual'`, or + `'conjugate_dual'`. + :returns: The space type. + + +.. py:function:: var_dtype(x) + + :arg x: A variable. + :returns: The data type associated with the variable. Typically + :class:`numpy.double` or :class:`numpy.cdouble`. + + +.. py:function:: var_id(x) + + Return a unique :class:`int` ID associated with a variable. + + Note that two variables share the same ID if they represent the same + symbolic variable -- for example if one variable represents both a variable + and stores a value, and a second the same variable with no value (i.e. is a + 'replacement'), then the two variables share the same ID. + + :arg x: The variable. + :returns: The :class:`int` ID. + + +.. py:function:: var_name(x) + + :arg x: A variable. + :returns: The :class:`str` name of the variable. + + +.. py:function:: var_state(x) + + Return the value of the state counter for a variable. Updated when the + value of the variable changes. + + :arg x: The variable. + :returns: The :class:`int` state value. + + +.. py:function:: var_lock_state(x) + + Lock the state of a variable. + + :arg x: The variable. + + +.. py:class:: VariableStateLockDictionary(*args, **kwargs) + + + + + A dictionary-like class. If a value is a variable and not a replacement + then the variable state is 'locked' so that a state update, with the lock + active, will raise an exception. + + State locks are automatically released when the + :class:`.VariableStateLockDictionary` is destroyed. Consequently objects of + this type should be used with caution. In particular object destruction via + the garbage collector may lead to non-deterministic release of the state + lock. + + +.. py:function:: var_update_state(*X) + + Ensure that variable state is updated, and check for cache invalidation. + + May delegate updating of the state to a backend library, in which case this + function checks that the state has been updated since the last + :func:`.var_update_state` call (or since instantiation on the first call), + and updates the backend state again only if needed. + + :arg X: A :class:`tuple` of variables. + + +.. py:function:: var_is_static(x) + + Return whether a variable is flagged as 'static'. A static variable + is stored by reference in checkpointing/replay, and the associated + tangent-linear variable is zero. + + :arg x: The variable. + :returns: Whether the variable is flagged as static. + + +.. py:function:: var_is_cached(x) + + Return whether results involving this variable may be cached. + + :arg x: The variable. + :returns: Whether results involving the variable may be cached. + + + +.. py:function:: var_caches(x) + + Return the :class:`.Caches` associated with a variable. + + :arg x: The variable. + :returns: The :class:`.Caches` associated with the variable. + + +.. py:function:: var_update_caches(*X, value=None) + + Check for cache invalidation associated with a possible change in value. + + :arg X: A :class:`tuple` of variables whose value may have changed. + :arg value: A variable or a :class:`Sequence` of variables defining the + possible new values. `X` is used if not supplied. + + +.. py:function:: var_zero(x) + + Zero a variable. + + :arg x: The variable. + + +.. py:function:: var_assign(x, y) + + Perform an assignment `x = y`. + + :arg x: A variable. + :arg y: A variable. + + +.. py:function:: var_axpy(y, alpha, x, /) + + Perform an in-place addition `y += alpha * x`. + + :arg y: A variable. + :arg alpha: A scalar. + :arg x: A variable. + + +.. py:function:: var_inner(x, y) + + Compute the :math:`l_2` inner product of the degrees of freedom vectors + associated with `x` and `y`. By convention if `y` is in the conjugate dual + space associated with `x`, this returns the complex conjugate of the + functional associated with `y` evaluated at `x`. + + :arg x: A variable. + :arg y: A variable. + :returns: The result of the inner product. + + +.. py:function:: var_linf_norm(x) + + Compute the :math:`l_\infty` norm of the degrees of freedom vector + associated with a variable. + + :arg x: The variable. + :returns: The :math:`l_\infty` norm of the degrees of freedom vector. + + +.. py:function:: var_local_size(x) + + Return the process local number of degrees of freedom associated with + a variable. This is the number of 'owned' degrees of freedom. + + :arg x: The variable. + :returns: The process local number of degrees of freedom for the variable. + + +.. py:function:: var_global_size(x) + + Return the global number of degrees of freedom associated with a + variable. This is the total number of 'owned' degrees of freedom, summed + across all processes. + + :arg x: The variable. + :returns: The global number of degrees of freedom for the variable. + + +.. py:function:: var_local_indices(x) + + Return the indices of process local degrees of freedom associated with + a variable. + + :arg x: The variable. + :returns: An :class:`Iterable`, yielding the indices of the process local + elements. + + +.. py:function:: var_get_values(x) + + Return a copy of the process local degrees of freedom vector associated + with a variable. + + :arg x: The variable. + :returns: A :class:`numpy.ndarray` containing a copy of the degrees of + freedom. + + +.. py:function:: var_set_values(x, values) + + Set the process local degrees of freedom vector associated with a + variable. + + :arg x: The variable. + :arg values: A :class:`numpy.ndarray` containing the degrees of freedom + values. + + +.. py:function:: var_new(x, *, name=None, static=False, cache=None, checkpoint=None, rel_space_type='primal') + + Return a new variable defined using the same space as `x`. + + :arg x: A variable. + :arg name: A :class:`str` name for the new variable. + :arg static: Defines whether the new variable is static, meaning that it is + stored by reference in checkpointing/replay, and an associated + tangent-linear variable is zero. + :arg cache: Defines whether results involving the new variable may be + cached. Default `static`. + :arg checkpoint: Deprecated. + :arg rel_space_type: Defines the space type of the new variable, relative + to the space type of `x`. + :returns: The new variable. + + +.. py:function:: var_new_conjugate(x, *, name=None, static=False, cache=None) + + Return a new conjugate variable. See :func:`.var_new`. + + :returns: A new variable defined using the same space as `x`, with space + type conjugate to the space type for `x`. + + +.. py:function:: var_new_dual(x, *, name=None, static=False, cache=None) + + Return a new dual variable. See :func:`.var_new`. + + :returns: A new variable defined using the same space as `x`, with space + type dual to the space type for `x`. + + +.. py:function:: var_new_conjugate_dual(x, *, name=None, static=False, cache=None) + + Return a new conjugate dual variable. See :func:`.var_new`. + + :returns: A new variable defined using the same space as `x`, with space + type conjugate dual to the space type for `x`. + + +.. py:function:: var_copy(x, *, name=None, static=False, cache=None) + + Copy a variable. See :func:`.var_new`. + + :returns: The copied variable. + + +.. py:function:: var_replacement(x) + + Return a variable, associated with the same variable as `x`, but + possibly without a value. + + :arg x: The variable. + :returns: A variable which symbolically represents the same variable as + `x`, but which may not store a value. May return `x` itself. + + +.. py:function:: var_is_replacement(x) + + Return whether a variable is a 'replacement', meaning that it has no + associated value. + + :arg x: The variable. + :returns: Whether `x` is a replacement. + + +.. py:function:: var_is_scalar(x) + + Return whether a variable defines a scalar variable. + + :arg x: The variable. + :returns: Whether `x` defines a scalar variable. + + +.. py:function:: var_scalar_value(x) + + If `x` defines a scalar variable, returns its value. + + :arg x: The variable, defining a scalar variable. + :returns: The scalar value. + + +.. py:function:: var_is_alias(x) + + Return whether a variable is an 'alias', meaning part or all of the + degree of freedom vector associated with the variable is shared with some + different aliased variable. A variable may not appear as an equation + dependency if it is an alias. + + :arg x: The variable. + :returns: Whether the variable is an alias. + + +.. py:function:: subtract_adjoint_derivative_action(x, y) + + Subtract an adjoint right-hand-side contribution defined by `y` from + the right-hand-side defined by `x`. + + :arg x: A variable storing the adjoint right-hand-side. + :arg y: A contribution to subtract from the adjoint right-hand-side. An + :meth:`.Equation.adjoint_derivative_action` return value. Valid types + depend upon the variable type. Typically this will be a variable, or a + two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value to subtract + defined by the product of `alpha` and `F`. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/_sources/autoapi/tlm_adjoint/jax/index.rst.txt b/_sources/autoapi/tlm_adjoint/jax/index.rst.txt new file mode 100644 index 0000000..c0ecb84 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/jax/index.rst.txt @@ -0,0 +1,316 @@ +:orphan: + +:py:mod:`tlm_adjoint.jax` +========================= + +.. py:module:: tlm_adjoint.jax + + +Module Contents +--------------- + +.. py:function:: set_default_jax_dtype(dtype) + + Set the default data type used by :class:`.Vector` objects. + + :arg dtype: The default data type. + + +.. py:class:: VectorSpace(n, *, dtype=None, comm=None) + + + A vector space. + + :arg dtype: The data type associated with the space. Typically + :class:`numpy.double` or :class:`numpy.cdouble`. + :arg comm: The communicator associated with the space. + + .. py:property:: dtype + + The data type associated with the space. + + + + .. py:property:: comm + + The communicator associated with the space. + + + + .. py:property:: local_size + + The number of local degrees of freedom. + + + + .. py:property:: global_size + + The global number of degrees of freedom. + + + + .. py:property:: ownership_range + + A tuple `(n0, n1)`, indicating that `slice(n0, n1)` is the range of + nodes in the global vector owned by this process. + + + .. py:method:: rdtype() + + The real data type associated with the space. + + + + +.. py:class:: Vector(V, *, name=None, space_type='primal', static=False, cache=None, dtype=None, comm=None) + + + + + Vector, with degrees of freedom stored as a JAX array. + + :arg V: A :class:`.VectorSpace`, an :class:`int` defining the number of + local degrees of freedom, or an ndim 1 array defining the local degrees + of freedom. + :arg name: A :class:`str` name for the :class:`.Vector`. + :arg space_type: The space type for the :class:`.Vector`. `'primal'`, + `'dual'`, `'conjugate'`, or `'conjugate_dual'`. + :arg static: Defines whether the :class:`.Vector` is static, meaning that + it is stored by reference in checkpointing/replay, and an associated + tangent-linear variable is zero. + :arg cache: Defines whether results involving the :class:`.Vector` may be + cached. Default `static`. + :arg dtype: The data type. Ignored if `V` is a :class:`.VectorSpace`. + :arg comm: A communicator. Ignored if `V` is a :class:`.VectorSpace`. + + .. py:property:: name + + The :class:`str` name of the :class:`.Vector`. + + + + .. py:property:: value + + For a :class:`.Vector` with one element, the value of the element. + + The value may also be accessed by casting using :class:`float` or + :class:`complex`. + + :returns: The value. + + + .. py:property:: space + + The :class:`.VectorSpace` for the :class:`.Vector`. + + + + .. py:property:: space_type + + The space type for the :class:`.Vector`. + + + + .. py:property:: vector + + A JAX array storing the local degrees of freedom. + + + + .. py:method:: new(y=None, *, name=None, static=False, cache=None) + + Return a new :class:`.Vector`, with the same :class:`.VectorSpace` + and space type as this :class:`.Vector`. + + :arg y: Defines a value for the new :class:`.Vector`. + :returns: The new :class:`.Vector`. + + Remaining arguments are as for the :class:`.Vector` constructor. + + + .. py:method:: assign(y, *, annotate=None, tlm=None) + + :class:`.Vector` assignment. + + :arg y: A :class:`numbers.Complex`, :class:`.Vector`, or ndim 1 array + defining the value. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + :returns: The :class:`.Vector`. + + + .. py:method:: addto(y, *, annotate=None, tlm=None) + + :class:`.Vector` in-place addition. + + :arg y: A :class:`numbers.Complex`, :class:`.Vector`, or ndim 1 array + defining the value to add. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + + + +.. py:class:: VectorEquation(X, Y, fn, *, with_tlm=True, _forward_eq=None) + + + + + JAX interface. `fn` should be a callable + + .. code-block:: python + + def fn(y0, y1, ...): + ... + return x0, x1, ... + + where the `y0`, `y1` are ndim 1 JAX arrays, and the `x0`, `x1`, are scalars + or ndim 1 JAX arrays. + + :arg X: A :class:`.Vector` or a :class:`Sequence` of :class:`.Vector` + objects defining outputs, whose value is set by the return value from + `fn`. + :arg Y: A :class:`.Vector` or a :class:`Sequence` of :class:`.Vector` + objects defining the inputs, whose values are passed to `fn`. + :arg fn: A callable. + :arg with_tlm: Whether to annotate an equation solving for the forward and + all tangent-linears (`with_tlm=True`), or solving only for the + forward (`with_tlm=False`). + + .. py:method:: solve(*, manager=None, annotate=None, tlm=None) + + Compute the forward solution. + + :arg manager: The :class:`.EquationManager`. Defaults to `manager()`. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + + + .. py:method:: forward_solve(X, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_jacobian_solve(adj_X, nl_deps, B) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: subtract_adjoint_derivative_actions(adj_X, nl_deps, dep_Bs) + + Subtract terms from other adjoint right-hand-sides. + + Can be overridden for an optimized implementation, but otherwise uses + :meth:`.Equation.adjoint_derivative_action`. + + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_Bs: A :class:`Mapping` whose items are `(dep_index, dep_B)`. + Each `dep_B` is an :class:`.AdjointRHS` which should be updated by + subtracting adjoint derivative information computed by + differentiating with respect to `self.dependencies()[dep_index]`. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:function:: call_jax(X, Y, fn) + + JAX interface. `fn` should be a callable + + .. code-block:: python + + def fn(y0, y1, ...): + ... + return x0, x1, ... + + where the `y0`, `y1` are ndim 1 JAX arrays, and the `x0`, `x1`, are scalars + or ndim 1 JAX arrays. + + :arg X: A :class:`.Vector` or a :class:`Sequence` of :class:`.Vector` + objects defining outputs, whose value is set by the return value from + `fn`. + :arg Y: A :class:`.Vector` or a :class:`Sequence` of :class:`.Vector` + objects defining the inputs, whose values are passed to `fn`. + :arg fn: A callable. + + +.. py:function:: new_jax(y, space=None, *, name=None) + + Construct a new zero-valued :class:`.Vector`. + + :arg y: A variable. + :arg space: The :class:`.VectorSpace` for the return value. + :arg name: A :class:`str` name. + :returns: The :class:`.Vector`. + + +.. py:function:: to_jax(y, space=None, *, name=None) + + Convert a variable to a :class:`.Vector`. + + :arg y: A variable. + :arg space: The :class:`.VectorSpace` for the return value. + :arg name: A :class:`str` name. + :returns: The :class:`.Vector`. + + +.. py:function:: new_jax_float(space=None, *, name=None, dtype=None, comm=None) + + Create a new :class:`.Vector` with one element. + + :arg space: The :class:`.VectorSpace`. + :arg name: A :class:`str` name. + :arg dtype: The data type. Ignored if `space` is supplied. + :arg comm: A communicator. Ignored if `space` is supplied. + :returns: A :class:`.Vector` with one element. + + diff --git a/_sources/autoapi/tlm_adjoint/linear_equation/index.rst.txt b/_sources/autoapi/tlm_adjoint/linear_equation/index.rst.txt new file mode 100644 index 0000000..b9a4743 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/linear_equation/index.rst.txt @@ -0,0 +1,420 @@ +:orphan: + +:py:mod:`tlm_adjoint.linear_equation` +===================================== + +.. py:module:: tlm_adjoint.linear_equation + + +Module Contents +--------------- + +.. py:class:: LinearEquation(X, B, *, A=None, adj_type=None) + + + + + Represents the solution of a linear equation + + .. math:: + + A x = \sum_i b_i, + + with left-hand-side matrix :math:`A` and right-hand-side terms :math:`b_i`. + The matrix and right-hand-side terms may depend on other forward variables + :math:`y_i`. + + The forward residual is defined + + .. math:: + + \mathcal{F} \left( x, y_1, y_2, \ldots \right) = A x - \sum_i b_i. + + :arg X: A variable or a :class:`Sequence` of variables defining the forward + solution `x`. + :arg B: A :class:`.RHS` or a :class:`Sequence` of :class:`.RHS` objects + defining the right-hand-side terms. + :arg A: A :class:`tlm_adjoint.linear_equation.Matrix` defining the + left-hand-side matrix. Defaults to an identity matrix if not supplied. + :arg adj_type: The space type relative to `X` of adjoint variables. + `'primal'` or `'conjugate_dual'`, or a :class:`Sequence` of these. + Defaults to `'primal'` if `A` is supplied and `'conjugate_dual'` + otherwise. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: forward_solve(X, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_jacobian_solve(adj_X, nl_deps, B) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_X) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: Matrix(nl_deps=None, *, ic=True, adj_ic=True) + + + + + Represents a matrix :math:`A`. + + This is an abstract base class. Information required by forward, adjoint, + and tangent-linear calculations is provided by overloading abstract + methods. This class does *not* inherit from :class:`abc.ABC`, so that + methods may be implemented as needed. + + :arg nl_deps: A :class:`Sequence` of variables, defining dependencies of + the matrix :math:`A`. + :arg ic: Whether solution of a linear equation :math:`A x = b` for + :math:`x` uses an initial guess. Defaults to `True`. + :arg adj_ic: Whether solution of an adjoint linear equation :math:`A^* + \lambda = b` for :math:`\lambda` uses an initial guess. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: nonlinear_dependencies() + + Return dependencies of the + :class:`tlm_adjoint.linear_equation.Matrix`. + + :returns: A :class:`Sequence` of variables defining dependencies. + + + .. py:method:: has_initial_condition() + + Return whether solution of a linear equation :math:`A x = b` for + :math:`x` uses an initial guess. + + :returns: `True` if an initial guess is used, and `False` otherwise. + + + .. py:method:: adjoint_has_initial_condition() + + Return whether solution of an adjoint linear equation :math:`A^* + \lambda = b` for :math:`\lambda` uses an initial guess. + + :returns: `True` if an initial guess is used, and `False` otherwise. + + + .. py:method:: forward_action(nl_deps, X, B, *, method='assign') + :abstractmethod: + + Evaluate the action of the matrix on :math:`x`, :math:`A x`. Assigns + the result to `B`, or adds the result to or subtracts the result from + `B`. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + dependencies. Should not be modified. + :arg X: Defines :math:`x`. A variable if it has a single component, and + a :class:`Sequence` of variables otherwise. Should not be modified. + Subclasses may replace this argument with `x` if there is a single + component. + :arg B: Stores the result. A variable if it has a single component, and + a :class:`Sequence` of variables otherwise. Subclasses may replace + this argument with `b` if there is a single component. + :arg method: If equal to `'assign'` then this method should set `B` + equal to the result. If equal to `'add'` then this method should + add the result to `B`. If equal to `'sub'` then this method should + subtract the result from `B`. + + + .. py:method:: adjoint_action(nl_deps, adj_X, b, b_index=0, *, method='assign') + :abstractmethod: + + Evaluate the action of the adjoint of the matrix on + :math:`\lambda`, :math:`A^* \lambda`. Assigns the `b_index` th + component to `b`, or adds the `b_index` th component to or subtracts + the `b_index` th component from `b`. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + dependencies. Should not be modified. + :arg adj_X: Defines :math:`\lambda`. A variable if it has a single + component, and a :class:`Sequence` of variables otherwise. Should + not be modified. Subclasses may replace this argument with `adj_x` + if there is a single component. + :arg b: A variable storing the result. Should be updated by this + method. + :arg b_index: The component of the result which should be used to + update `b`. + :arg method: If equal to `'assign'` then this method should set `b` + equal to the `b_index` th component of the result. If equal to + `'add'` then this method should add the `b_index` th component of + the result to `b`. If equal to `'sub'` then this method should + subtract the `b_index` th component of the result from `b`. + + + .. py:method:: forward_solve(X, nl_deps, B) + :abstractmethod: + + Solve the linear system :math:`A x = b` for :math:`x`. + + :arg X: The solution :math:`x`. A variable if it has a single + component, and a :class:`Sequence` of variables otherwise. May + define an initial guess. Subclasses may replace this argument with + `x` if there is a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + dependencies. Should not be modified. + :arg B: The right-hand-side :math:`b`. A variable if it has a single + component, and a :class:`Sequence` of variables otherwise. Should + not be modified. Subclasses may replace this argument with `b` if + there is a single component. + + + .. py:method:: adjoint_derivative_action(nl_deps, nl_dep_index, X, adj_X, b, *, method='assign') + :abstractmethod: + + Evaluate the action of the adjoint of a derivative of :math:`A x` on + an adjoint variable. Assigns the result to `b`, or adds the result to + or subtracts the result from `b`. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + dependencies. Should not be modified. + :arg nl_deps_index: An :class:`int`. The derivative is defined by + differentiation of :math:`A x` with respect to + `self.nonlinear_dependencies()[nl_dep_index]`. + :arg X: Defines :math:`x`. A variable if it has a single component, and + a :class:`Sequence` of variables otherwise. Should not be modified. + Subclasses may replace this argument with `x` if there is a single + component. + :arg adj_X: The adjoint variable. A variable if it has a single + component, and :class:`Sequence` of variables otherwise. Should not + be modified. Subclasses may replace this argument with `adj_x` if + the adjoint variable has a single component. + :arg b: A variable storing the result. Should be updated by this + method. + :arg method: If equal to `'assign'` then this method should set `b` + equal to the result. If equal to `'add'` then this method should + add the result to `b`. If equal to `'sub'` then this method should + subtract the result from `b`. + + + .. py:method:: adjoint_solve(adj_X, nl_deps, B) + :abstractmethod: + + Solve the linear system :math:`A^* \lambda = b` for + :math:`\lambda`. + + :arg adj_X: The solution :math:`\lambda`. A variable if it has a single + component, and a :class:`Sequence` of variables otherwise. May + define an initial guess. Subclasses may replace this argument with + `adj_x` if there is a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + dependencies. Should not be modified. + :arg B: The right-hand-side :math:`b`. A variable if it has a single + component, and a :class:`Sequence` of variables otherwise. Should + not be modified. Subclasses may replace this argument with `b` if + there is a single component. + + + .. py:method:: tangent_linear_rhs(M, dM, tlm_map, X) + :abstractmethod: + + Construct tangent-linear right-hand-side terms obtained by + differentiation of + + .. math:: + + \mathcal{G} \left( x, y_1, y_2, \ldots \right) = -A x + + with respect to dependencies of the matrix :math:`A`. i.e. construct + + .. math:: + + -\sum_i \frac{\partial \mathcal{G}}{\partial y_i} \tau_{y_i}, + + where :math:`\tau_{y_i}` is the tangent-linear variable associated with + the dependency :math:`y_i`. Note the *negative* sign. Does *not* + include the term :math:`-A \tau_x` where :math:`\tau_x` is the + tangent-linear variable associated with :math:`x`. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :arg X: Defines :math:`x`. A variable if it has a single component, and + a :class:`Sequence` of variables otherwise. Subclasses may replace + this argument with `x` if there is a single component. + :returns: A :class:`.RHS`, or a :class:`Sequence` of :class:`.RHS` + objects, defining the right-hand-side terms. Returning `None` + indicates that there are no terms. + + + +.. py:class:: RHS(deps, nl_deps=None) + + + + + Represents a right-hand-side term. + + This is an abstract base class. Information required by forward, adjoint, + and tangent-linear calculations is provided by overloading abstract + methods. This class does *not* inherit from :class:`abc.ABC`, so that + methods may be implemented as needed. + + :arg deps: A :class:`Sequence` of variables defining dependencies. + :arg nl_deps: A :class:`Sequence` of variables defining non-linear + dependencies. + + .. py:method:: drop_references() + + Drop references to variables which store values. + + + + .. py:method:: dependencies() + + Return dependencies of the :class:`.RHS`. + + :returns: A :class:`Sequence` of variables defining dependencies. + + + .. py:method:: nonlinear_dependencies() + + Return non-linear dependencies of the :class:`.RHS`. + + :returns: A :class:`Sequence` of variables defining non-linear + dependencies. + + + .. py:method:: add_forward(B, deps) + :abstractmethod: + + Add the right-hand-side term to `B`. + + :arg B: A variable if it has a single component, and a + :class:`Sequence` of variables otherwise. Should be updated by the + addition of this :class:`.RHS`. Subclasses may replace this + argument with `b` if there is a single component. + :arg deps: A :class:`Sequence` of variables defining values for + dependencies. Should not be modified. + + + .. py:method:: subtract_adjoint_derivative_action(nl_deps, dep_index, adj_X, b) + :abstractmethod: + + Subtract the action of the adjoint of a derivative of the + right-hand-side term, on an adjoint variable, from `b`. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg deps_index: An :class:`int`. The derivative is defined by + differentiation of the right-hand-side term with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint variable. A variable if it has a single + component, and a :class:`Sequence` of variables otherwise. Should + not be modified. Subclasses may replace this argument with `adj_x` + if the adjoint variable has a single component. + :arg b: A variable storing the result. Should be updated by subtracting + the action of the adjoint of the right-hand-side term on the + adjoint variable. + + + .. py:method:: tangent_linear_rhs(M, dM, tlm_map) + :abstractmethod: + + Construct tangent-linear right-hand-side terms obtained by + differentiation of this right-hand-side term. That is, construct + + .. math:: + + \sum_i \frac{\partial b}{\partial y_i} \tau_{y_i}, + + where :math:`b` is this right-hand-side term, and :math:`\tau_{y_i}` is + the tangent-linear variable associated with a dependency :math:`y_i`. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: A :class:`.RHS`, or a :class:`Sequence` of :class:`.RHS` + objects, defining the right-hand-side terms. Returning `None` + indicates that there are no terms. + + + diff --git a/_sources/autoapi/tlm_adjoint/manager/index.rst.txt b/_sources/autoapi/tlm_adjoint/manager/index.rst.txt new file mode 100644 index 0000000..3a2cf99 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/manager/index.rst.txt @@ -0,0 +1,128 @@ +:py:mod:`tlm_adjoint.manager` +============================= + +.. py:module:: tlm_adjoint.manager + +.. autoapi-nested-parse:: + + This module provides a simple :class:`.EquationManager` interface. Functions + defined here access and interact with the default manager. + + + +Module Contents +--------------- + +.. py:function:: manager() + + :returns: An :class:`.EquationManager`, the current default manager. + + +.. py:function:: set_manager(manager) + + Set the default manager. + + :arg manager: An :class:`.EquationManager` to use as the default manager. + + +.. py:function:: restore_manager(fn) + + Decorator to revert the default manager to the manager used prior to + calling the decorated callable. A typical use is + + .. code-block:: python + + @restore_manager + def decorated(*M): + set_manager(manager().new()) + forward(*M) + + :arg fn: A decorated callable. + :returns: A callable, where the default manager on entry is restored on + exit. + + +.. py:function:: configure_checkpointing(cp_method, cp_parameters) + + See :meth:`.EquationManager.configure_checkpointing`. + + + +.. py:function:: manager_info(*, info=print) + + See :meth:`.EquationManager.info`. + + + +.. py:function:: reset_manager(cp_method=None, cp_parameters=None) + + See :meth:`.EquationManager.reset`. + + + +.. py:function:: annotation_enabled() + + See :meth:`.EquationManager.annotation_enabled`. + + + +.. py:function:: start_manager(*, annotate=True, tlm=True) + + See :meth:`.EquationManager.start`. + + + +.. py:function:: stop_manager(*, annotate=True, tlm=True) + + See :meth:`.EquationManager.stop`. + + + +.. py:function:: paused_manager(*, annotate=True, tlm=True) + + See :meth:`.EquationManager.paused`. + + + +.. py:function:: manager_disabled(*, annotate=True, tlm=True) + + Decorator which can be used to disable processing of equations and + derivation and solution of tangent-linear equations. + + :arg annotate: Whether to disable processing of equations. + :arg tlm: Whether to disable derivation and solution of tangent-linear + equations. + + +.. py:function:: configure_tlm(*args, annotate=None, tlm=True) + + See :meth:`.EquationManager.configure_tlm`. + + + + +.. py:function:: tlm_enabled() + + See :meth:`.EquationManager.tlm_enabled`. + + + +.. py:function:: var_tlm(x, *args) + + See :meth:`.EquationManager.var_tlm`. + + + + +.. py:function:: compute_gradient(Js, M, *, callback=None, prune_forward=True, prune_adjoint=True, prune_replay=True, cache_adjoint_degree=None, store_adjoint=False, adj_ics=None) + + See :meth:`.EquationManager.compute_gradient`. + + + +.. py:function:: new_block() + + See :meth:`.EquationManager.new_block`. + + + diff --git a/_sources/autoapi/tlm_adjoint/markers/index.rst.txt b/_sources/autoapi/tlm_adjoint/markers/index.rst.txt new file mode 100644 index 0000000..d46d7e6 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/markers/index.rst.txt @@ -0,0 +1,120 @@ +:orphan: + +:py:mod:`tlm_adjoint.markers` +============================= + +.. py:module:: tlm_adjoint.markers + + +Module Contents +--------------- + +.. py:class:: ControlsMarker(M) + + + + + Represents + + .. math:: + + m = m_\text{input}, + + where :math:`m` is the control and :math:`m_\text{input}` the input value + for the control. The forward residual is defined + + .. math:: + + \mathcal{F} \left( m \right) = m - m_\text{input}. + + :arg M: A variable or a :class:`Sequence` of variables defining the + control :math:`m`. May be static. + + .. py:method:: adjoint_jacobian_solve(adj_X, nl_deps, B) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + +.. py:class:: FunctionalMarker(J) + + + + + Represents + + .. math:: + + J_\text{output} = J, + + where :math:`J` is the functional and :math:`J_\text{output}` is the output + value for the functional. The forward residual is defined + + .. math:: + + \mathcal{F} \left( J_\text{output}, J \right) = J_\text{output} - J. + + :arg J: A variable defining the functional :math:`J`. + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + diff --git a/_sources/autoapi/tlm_adjoint/optimization/index.rst.txt b/_sources/autoapi/tlm_adjoint/optimization/index.rst.txt new file mode 100644 index 0000000..ba337e3 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/optimization/index.rst.txt @@ -0,0 +1,285 @@ +:orphan: + +:py:mod:`tlm_adjoint.optimization` +================================== + +.. py:module:: tlm_adjoint.optimization + + +Module Contents +--------------- + +.. py:function:: minimize_scipy(forward, M0, *, manager=None, **kwargs) + + Provides an interface with :func:`scipy.optimize.minimize` for + gradient-based optimization. + + Note that the control is gathered onto the root process so that the serial + :func:`scipy.optimize.minimize` function may be used. + + All keyword arguments except for `manager` are passed to + :func:`scipy.optimize.minimize`. + + :arg forward: A callable which accepts one or more variable arguments, and + which returns a variable defining the forward functional. + :arg M0: A variable or :class:`Sequence` of variables defining the control, + and the initial guess for the optimization. + :arg manager: An :class:`.EquationManager` used to create an internal + manager via :meth:`.EquationManager.new`. `manager()` is used if not + supplied. + :returns: A :class:`tuple` `(M, return_value)`. `M` is variable or a + :class:`Sequence` of variables depending on the type of `M0`, and + stores the result. `return_value` is the return value of + :func:`scipy.optimize.minimize`. + + +.. py:class:: LBFGSHessianApproximation(m) + + + L-BFGS Hessian approximation. + + :arg m: Maximum number of vector pairs to retain in the L-BFGS Hessian + approximation. + + .. py:method:: append(S, Y, S_inner_Y) + + Add a step + gradient change pair. + + :arg S: A variable or a :class:`Sequence` of variables defining the + step. + :arg Y: A variable or a :class:`Sequence` of variables defining the + gradient change. + :arg S_inner_Y: The projection of the gradient change onto the step. + A separate argument so that a value consistent with + that used in the line search can be supplied. + + + .. py:method:: inverse_action(X, *, H_0_action=None, theta=1.0) + + Compute the action of the approximate Hessian inverse on some given + direction. + + Implements the L-BFGS Hessian inverse action approximation as in + Algorithm 7.4 of + + - Jorge Nocedal and Stephen J. Wright, 'Numerical optimization', + Springer, New York, NY, 2006, Second edition, + doi: 10.1007/978-0-387-40065-5 + + Uses 'theta scaling' as in equation (3.7) of + + - Richard H. Byrd, Peihuang Lu, and Jorge Nocedal, and Ciyou Zhu, + 'A limited memory algorithm for bound constrained optimization', + SIAM Journal on Scientific Computing, 16(5), 1190--1208, 1995, + doi: 10.1137/0916069 + + :arg X: A variable or a :class:`Sequence` of variables defining the + direction on which to compute the approximate Hessian inverse + action. + :arg H_0_action: A callable defining the action of the non-updated + Hessian inverse approximation on some direction. Accepts one or + more variables as arguments, defining the direction, and returns a + variable or a :class:`Sequence` of variables defining the action on + this direction. Should correspond to a positive definite operator. + An identity is used if not supplied. + :returns: A variable or a :class:`Sequence` of variables storing the + result. + + + +.. py:function:: l_bfgs(F, Fp, X0, *, m=30, s_atol, g_atol, converged=None, max_its=1000, H_0_action=None, theta_scale=True, delta=1.0, M_action=None, M_inv_action=None, c1=0.0001, c2=0.9, comm=None) + + Functional minimization using the L-BFGS algorithm. + + Implements Algorithm 7.5 of + + - Jorge Nocedal and Stephen J. Wright, 'Numerical optimization', + Springer, New York, NY, 2006, Second edition, + doi: 10.1007/978-0-387-40065-5 + + in a more general inner product space. + + By default uses 'theta scaling' to define the initial Hessian inverse + approximation, based on the approach in equation (3.7) and point 7 on p. + 1204 of + + - Richard H. Byrd, Peihuang Lu, and Jorge Nocedal, and Ciyou Zhu, 'A + limited memory algorithm for bound constrained optimization', SIAM + Journal on Scientific Computing, 16(5), 1190--1208, 1995, + doi: 10.1137/0916069 + + and with an initial value for the scaling parameter based on the discussion + in 'Implementation' in section 6.1 of + + - Jorge Nocedal and Stephen J. Wright, 'Numerical optimization', + Springer, New York, NY, 2006, Second edition, + doi: 10.1007/978-0-387-40065-5 + + Precisely the Hessian inverse approximation, before being updated, is + scaled by :math:`1 / \theta` with, on the first iteration, + + .. math:: + + \theta = \frac{\sqrt{\left| g_k^* M^{-1} g_k \right|}}{\delta} + + and on later iterations + + .. math:: + + \theta = \frac{y_k^* H_0 y_k}{y_k^* s_k}, + + where :math:`g_k`, :math:`y_k`, and :math:`s_k` are respectively the + gradient, gradient change, and step, and where :math:`M^{-1}` and + :math:`H_0` are defined by `M_inv_action` and `H_0_action` respectively. + + :arg F: A callable defining the functional. Accepts one or more variables + as arguments, and returns the value of the functional. Input arguments + should not be modified. + :arg Fp: A callable defining the functional gradient. Accepts one or more + variables as inputs, and returns a variable or :class:`Sequence` of + variables storing the value of the gradient. Input arguments should not + be modified. + :arg X0: A variable or a :class:`Sequence` of variables defining the + initial guess for the parameters. + :arg m: The maximum number of step + gradient change pairs to use in the + Hessian inverse approximation. + :arg s_atol: Absolute tolerance for the step change norm convergence + criterion. + :arg g_atol: Absolute tolerance for the gradient norm convergence + criterion. + :arg converged: A callable defining a callback, and which can be used to + define custom convergence criteria. Takes the form + + .. code-block:: python + + def converged(it, F_old, F_new, X_new, G_new, S, Y): + + with + + - `it`: The iteration number. + - `F_old`: The previous value of the functional. + - `F_new`: The new value of the functional. + - `X_new`: A variable or a :class:`Sequence` of variables defining + the new value of the parameters. + - `G_new`: A variable or a :class:`Sequence` of variables defining + the new value of the gradient. + - `S`: A variable or a :class:`Sequence` of variables defining the + step. + - `Y`: A variable or a sequence of variables defining the gradient + change. + + Input variables should not be modified. Returns a :class:`bool` + indicating whether the optimization has converged. + :arg max_its: The maximum number of iterations. + :arg H_0_action: A callable defining the action of the non-updated Hessian + inverse approximation on some direction. Accepts one or more variables + as arguments, defining the direction, and returns a variable or a + :class:`Sequence` of variables defining the action on this direction. + Should correspond to a positive definite operator. An identity is used + if not supplied. + :arg theta_scale: Whether to apply 'theta scaling', discussed above. + :arg delta: Controls the initial value of :math:`\theta` in 'theta + scaling'. If `None` then on the first iteration :math:`\theta` is set + equal to one. + :arg M_action: A callable defining a primal space inner product, + + .. math:: + + \left< x, y \right>_M = y^* M x, + + where :math:`x` and :math:`y` are degree of freedom vectors for primal + space elements and :math:`M` is a Hermitian and positive definite + matrix. Accepts one or more variables as arguments, defining the + direction, and returns a variable or a :class:`Sequence` of variables + defining the action of :math:`M` on this direction. An identity is used + if not supplied. Required if `H_0_action` or `M_inv_action` are + supplied. + :arg M_inv_action: A callable defining a (conjugate) dual space inner + product, + + .. math:: + + \left< x, y \right>_{M^{-1}} = y^* M^{-1} x, + + where :math:`x` and :math:`y` are degree of freedom vectors for + (conjugate) dual space elements and :math:`M` is as for `M_action`. + Accepts one or more variables as arguments, defining the direction, and + returns a variable or a :class:`Sequence` of variables defining the + action of :math:`M^{-1}` on this direction. `H_0_action` is used if not + supplied. + :arg c1: Armijo condition parameter. :math:`c_1` in equation (3.6a) of + + - Jorge Nocedal and Stephen J. Wright, 'Numerical optimization', + Springer, New York, NY, 2006, Second edition, + doi: 10.1007/978-0-387-40065-5 + + :arg c2: Curvature condition parameter. :math:`c_2` in equation (3.6b) of + + - Jorge Nocedal and Stephen J. Wright, 'Numerical optimization', + Springer, New York, NY, 2006, Second edition, + doi: 10.1007/978-0-387-40065-5 + + :arg comm: A communicator. + :returns: A :class:`tuple` `(X, (it, F_calls, Fp_calls, hessian_approx))` + with + + - `X`: The solution. A variable or a :class:`tuple` of variables. + - `it`: The number of iterations. + - `F_calls`: The number of functional evaluations. + - `Fp_calls`: The number of gradient evaluations. + - `hessian_approx`: The :class:`.LBFGSHessianApproximation`. + + +.. py:function:: minimize_l_bfgs(forward, M0, *, m=30, manager=None, **kwargs) + + Functional minimization using the L-BFGS algorithm. + + :arg forward: A callable which accepts one or more variable arguments, and + which returns a variable defining the forward functional. + :arg M0: A variable or :class:`Sequence` of variables defining the control, + and the initial guess for the optimization. + :arg manager: An :class:`.EquationManager` used to create an internal + manager via :meth:`.EquationManager.new`. `manager()` is used if not + supplied. + + Remaining arguments and the return value are described in the + :func:`.l_bfgs` documentation. + + +.. py:function:: minimize_tao(forward, M0, *, method=None, gatol, grtol, gttol=0.0, M_inv_action=None, pre_callback=None, post_callback=None, manager=None) + + Functional minimization using TAO. + + :arg forward: A callable which accepts one or more variable arguments, and + which returns a variable defining the forward functional. + :arg M0: A variable or :class:`Sequence` of variables defining the control, + and the initial guess for the optimization. + :arg method: TAO type. Defaults to `PETSc.TAO.Type.LMVM`. + :arg gatol: TAO gradient absolute tolerance. + :arg grtol: TAO gradient relative tolerance. + :arg gttol: TAO gradient norm change relative tolerance. + :arg M_inv_action: A callable defining a (conjugate) dual space inner + product, + + .. math:: + + \left< x, y \right>_{M^{-1}} = y^* M^{-1} x, + + where :math:`x` and :math:`y` are degree of freedom vectors for + (conjugate) dual space elements and :math:`M` is a Hermitian and + positive definite matrix. Accepts one or more variables as arguments, + defining the direction, and returns a variable or a :class:`Sequence` + of variables defining the action of :math:`M^{-1}` on this direction. + :arg pre_callback: A callable accepting a single + :class:`petsc4py.PETSc.TAO` argument. Used for detailed manual + configuration. Called after all other configuration options are set. + :arg post_callback: A callable accepting a single + :class:`petsc4py.PETSc.TAO` argument. Called after the + :meth:`petsc4py.PETSc.TAO.solve` method has been called. + :arg manager: An :class:`.EquationManager` used to create an internal + manager via :meth:`.EquationManager.new`. `manager()` is used if not + supplied. + :returns: A variable or a :class:`Sequence` of variables storing the + result. + + diff --git a/_sources/autoapi/tlm_adjoint/overloaded_float/index.rst.txt b/_sources/autoapi/tlm_adjoint/overloaded_float/index.rst.txt new file mode 100644 index 0000000..1e1cd2b --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/overloaded_float/index.rst.txt @@ -0,0 +1,305 @@ +:py:mod:`tlm_adjoint.overloaded_float` +====================================== + +.. py:module:: tlm_adjoint.overloaded_float + +.. autoapi-nested-parse:: + + This module defines types which allow for basic floating point level + algorithmic differentiation. The implementation is intended to be used for a + small number of calculations, for example after the calculation of a functional + obtained from a finite element model. + + For example, if annotation and operator overloading is enabled + + .. code-block:: python + + import numpy as np + + x = Float(np.pi, name='x') + y = x * np.sin(x) + + will lead to annotation of equations associated with the floating point + calculations. + + + +Module Contents +--------------- + +.. py:function:: set_default_float_dtype(dtype) + + Set the default data type used by :class:`.SymbolicFloat` objects. + + :arg dtype: The default data type. + + +.. py:class:: FloatSpace(float_cls=None, *, dtype=None, comm=None) + + + Defines the real or complex space. + + :arg float_cls: The :class:`.SymbolicFloat` class, in particular used to + instantiate new variables in :func:`.space_new`. Defaults to + :class:`.SymbolicFloat`. + :arg dtype: The data type associated with the space. Typically + :class:`numpy.double` or :class:`numpy.cdouble`. + :arg comm: The communicator associated with the space. + + .. py:property:: dtype + + The data type associated with the space. + + + + .. py:property:: comm + + The communicator associated with the space. + + + + .. py:property:: float_cls + + The :class:`.SymbolicFloat` class associated with the space. + + + + .. py:method:: rdtype() + + The real data type associated with the space. + + + + +.. py:function:: no_float_overloading(fn) + + Decorator to disable :class:`.OverloadedFloat` operator overloading. + + :arg fn: A callable for which :class:`.OverloadedFloat` operator + overloading should be disabled. + :returns: A callable for which :class:`.OverloadedFloat` operator + overloading is disabled. + + +.. py:function:: paused_float_overloading() + + Construct a context manager which can be used to temporarily disable + :class:`.OverloadedFloat` operator overloading. + + :returns: A context manager which can be used to temporarily disable + :class:`.OverloadedFloat` operator overloading. + + +.. py:class:: SymbolicFloat(value=0.0, *, name=None, space_type='primal', static=False, cache=None, dtype=None, comm=None, annotate=None, tlm=None) + + + + + A :class:`sympy.core.symbol.Symbol` which is also a 'variable', defining + a scalar variable. + + If constructing SymPy expressions then the :class:`.SymbolicFloat` class + should be used instead of the :class:`.OverloadedFloat` subclass, or else + :class:`.OverloadedFloat` operator overloading should be disabled. + + :arg value: A :class:`numbers.Complex` or :class:`sympy.core.expr.Expr` + defining the initial value. If a :class:`sympy.core.expr.Expr` then, if + annotation or derivation and solution of tangent-linear equations is + enabled, an assignment is processed by the :class:`.EquationManager` + `manager`. + :arg name: A :class:`str` name for the :class:`.SymbolicFloat`. + :arg space_type: The space type for the :class:`.SymbolicFloat`. + `'primal'`, `'dual'`, `'conjugate'`, or `'conjugate_dual'`. + :arg static: Defines whether the :class:`.SymbolicFloat` is static, meaning + that it is stored by reference in checkpointing/replay, and an + associated tangent-linear variable is zero. + :arg cache: Defines whether results involving the :class:`.SymbolicFloat` + may be cached. Default `static`. + :arg dtype: The data type associated with the :class:`.SymbolicFloat`. + Typically :class:`numpy.double` or :class:`numpy.cdouble`. + :arg comm: The communicator associated with the :class:`.SymbolicFloat`. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + + .. py:property:: value + + Return the current value associated with the + :class:`.SymbolicFloat`. + + The value may also be accessed by casting using :class:`float` or + :class:`complex`. + + :returns: The value. + + + .. py:method:: new(value=0.0, *, name=None, static=False, cache=None, annotate=None, tlm=None) + + Return a new object, which same type and space type as this + :class:`.SymbolicFloat`. + + :returns: The new :class:`.SymbolicFloat`. + + Arguments are as for the :class:`.SymbolicFloat` constructor. + + + .. py:method:: assign(y, *, annotate=None, tlm=None) + + :class:`.SymbolicFloat` assignment. + + :arg y: A :class:`numbers.Complex` or :class:`sympy.core.expr.Expr` + defining the value. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + :returns: The :class:`.SymbolicFloat`. + + + .. py:method:: addto(y, *, annotate=None, tlm=None) + + :class:`.SymbolicFloat` in-place addition. + + :arg y: A :class:`numbers.Complex` or :class:`sympy.core.expr.Expr` + defining the value to add. + :arg annotate: Whether the :class:`.EquationManager` should record the + solution of equations. + :arg tlm: Whether tangent-linear equations should be solved. + + + + +.. py:class:: OverloadedFloat(value=0.0, *, name=None, space_type='primal', static=False, cache=None, dtype=None, comm=None, annotate=None, tlm=None) + + + + + A subclass of :class:`.SymbolicFloat` with operator overloading. + + If constructing SymPy expressions then the :class:`.SymbolicFloat` class + should be used instead of the :class:`.OverloadedFloat` subclass, or else + :class:`.OverloadedFloat` operator overloading should be disabled. + + For argument documentation see :class:`.SymbolicFloat`. + + + +.. py:class:: Float(value=0.0, *, name=None, space_type='primal', static=False, cache=None, dtype=None, comm=None, annotate=None, tlm=None) + + + + + A subclass of :class:`.SymbolicFloat` with operator overloading. + + If constructing SymPy expressions then the :class:`.SymbolicFloat` class + should be used instead of the :class:`.OverloadedFloat` subclass, or else + :class:`.OverloadedFloat` operator overloading should be disabled. + + For argument documentation see :class:`.SymbolicFloat`. + + + +.. py:class:: FloatEquation(x, expr) + + + + + Represents an assignment to a :class:`.SymbolicFloat` `x`, + + .. math:: + + x = \mathcal{G} \left( y_1, y_2, \ldots \right), + + for some :math:`\mathcal{G}` defined by a :class:`sympy.core.expr.Expr`. + The forward residual is defined + + .. math:: + + \mathcal{F} \left( x, y_1, y_2, \ldots \right) + = x - \mathcal{G} \left( y_1, y_2, \ldots \right). + + :arg x: A :class:`.SymbolicFloat` defining the forward solution :math:`x` + :arg expr: A :class:`sympy.core.expr.Expr` defining the right-hand-side. + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: subtract_adjoint_derivative_actions(adj_x, nl_deps, dep_Bs) + + Subtract terms from other adjoint right-hand-sides. + + Can be overridden for an optimized implementation, but otherwise uses + :meth:`.Equation.adjoint_derivative_action`. + + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_Bs: A :class:`Mapping` whose items are `(dep_index, dep_B)`. + Each `dep_B` is an :class:`.AdjointRHS` which should be updated by + subtracting adjoint derivative information computed by + differentiating with respect to `self.dependencies()[dep_index]`. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:function:: to_float(y, *, name=None) + + Convert a variable to a :class:`.Float`. + + :arg y: A scalar variable. + :arg name: A :class:`str` name. + :returns: The :class:`.SymbolicFloat`. + + diff --git a/_sources/autoapi/tlm_adjoint/storage/index.rst.txt b/_sources/autoapi/tlm_adjoint/storage/index.rst.txt new file mode 100644 index 0000000..e938215 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/storage/index.rst.txt @@ -0,0 +1,228 @@ +:orphan: + +:py:mod:`tlm_adjoint.storage` +============================= + +.. py:module:: tlm_adjoint.storage + + +Module Contents +--------------- + +.. py:class:: Storage(x, key, *, save=False) + + + + + Used to save and load a forward solution value. + + With `save=True` the first forward solve saves the value of `x`. With + `save=False`, or on any subsequent forward solve, the value of the forward + solution is loaded into `x`. + + When processed by the :class:`.EquationManager` this is equivalent to an + assignment + + .. math:: + + x = x_\text{value}, + + where :math:`x_\text{value}` is the value which is saved or loaded. The + forward residual is defined + + .. math:: + + \mathcal{F} \left( x \right) = x - x_\text{value}. + + This is an abstract base class. Information required to save and load data + is provided by overloading abstract methods. This class does *not* inherit + from :class:`abc.ABC`, so that methods may be implemented as needed. + + :arg x: A variable defining the forward solution, whose value is saved or + loaded. + :arg key: A :class:`str` key for the saved or loaded data. + :arg save: If `True` then the first forward solve saves the value of `x`. + If `False` then the first forward solve loads the value of `x`. + + .. py:property:: key + + The key associated with saved or loaded data. + + + + .. py:method:: is_saved() + :abstractmethod: + + Return whether a value can be loaded. + + :returns: `True` if a value can be loaded, and `False` otherwise. + + + .. py:method:: load(x) + :abstractmethod: + + Load data, storing the result in `x`. + + :arg x: A variable in which the loaded data is stored. + + + .. py:method:: save(x) + :abstractmethod: + + Save the value of `x`. + + :arg x: A variable whose value should be saved. + + + .. py:method:: forward_solve(x, deps=None) + + Compute the forward solution. + + Can assume that the currently active :class:`.EquationManager` is + paused. + + :arg X: A variable if the forward solution has a single component, + otherwise a :class:`Sequence` of variables. May define an initial + guess, and should be set by this method. Subclasses may replace + this argument with `x` if the forward solution has a single + component. + :arg deps: A :class:`tuple` of variables, defining values for + dependencies. Only the elements corresponding to `X` may be + modified. `self.dependencies()` should be used if not supplied. + + + .. py:method:: adjoint_jacobian_solve(adj_x, nl_deps, b) + + Compute an adjoint solution. + + :arg adj_X: Either `None`, or a variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + defining the initial guess for an iterative solve. May be modified + or returned. Subclasses may replace this argument with `adj_x` if + the adjoint solution has a single component. + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg B: The right-hand-side. A variable (if the adjoint solution has a + single component) or :class:`Sequence` of variables (otherwise) + storing the value of the right-hand-side. May be modified or + returned. Subclasses may replace this argument with `b` if the + adjoint solution has a single component. + :returns: A variable or :class:`Sequence` of variables storing the + value of the adjoint solution. May return `None` to indicate a + value of zero. + + + .. py:method:: adjoint_derivative_action(nl_deps, dep_index, adj_x) + + Return the action of the adjoint of a derivative of the forward + residual on the adjoint solution. This is the *negative* of an adjoint + right-hand-side term. + + :arg nl_deps: A :class:`Sequence` of variables defining values for + non-linear dependencies. Should not be modified. + :arg dep_index: An :class:`int`. The derivative is defined by + differentiation of the forward residual with respect to + `self.dependencies()[dep_index]`. + :arg adj_X: The adjoint solution. A variable if the adjoint solution + has a single component, otherwise a :class:`Sequence` of variables. + Should not be modified. Subclasses may replace this argument with + `adj_x` if the adjoint solution has a single component. + :returns: The action of the adjoint of a derivative on the adjoint + solution. Will be passed to + :func:`.subtract_adjoint_derivative_action`, and valid types depend + upon the adjoint variable type. Typically this will be a variable, + or a two element :class:`tuple` `(alpha, F)`, where `alpha` is a + :class:`numbers.Complex` and `F` a variable, with the value defined + by the product of `alpha` and `F`. + + + .. py:method:: tangent_linear(M, dM, tlm_map) + + Derive an :class:`.Equation` corresponding to a tangent-linear + operation. + + :arg M: A :class:`Sequence` of variables defining the control. + :arg dM: A :class:`Sequence` of variables defining the derivative + direction. The tangent-linear computes directional derivatives with + respect to the control defined by `M` and with direction defined by + `dM`. + :arg tlm_map: A :class:`.TangentLinearMap` storing values for + tangent-linear variables. + :returns: An :class:`.Equation`, corresponding to the tangent-linear + operation. + + + +.. py:class:: MemoryStorage(x, d, key, *, save=False) + + + + + A :class:`.Storage` which stores the value in memory. + + :arg x: A variable defining the forward solution, whose value is saved or + loaded. + :arg d: A :class:`dict` in which data is stored with key `key`. + :arg key: A :class:`str` key for the saved or loaded data. + :arg save: If `True` then the first forward solve saves the value of `x`. + If `False` then the first forward solve loads the value of `x` + + .. py:method:: is_saved() + + Return whether a value can be loaded. + + :returns: `True` if a value can be loaded, and `False` otherwise. + + + .. py:method:: load(x) + + Load data, storing the result in `x`. + + :arg x: A variable in which the loaded data is stored. + + + .. py:method:: save(x) + + Save the value of `x`. + + :arg x: A variable whose value should be saved. + + + +.. py:class:: HDF5Storage(x, h, key, *, save=False) + + + + + A :class:`.Storage` which stores the value on disk using the h5py + library. + + :arg x: A variable defining the forward solution, whose value is saved or + loaded. + :arg h: An :class:`h5py.File`. + :arg key: A :class:`str` key for the saved or loaded data. + :arg save: If `True` then the first forward solve saves the value of `x`. + If `False` then the first forward solve loads the value of `x` + + .. py:method:: is_saved() + + Return whether a value can be loaded. + + :returns: `True` if a value can be loaded, and `False` otherwise. + + + .. py:method:: load(x) + + Load data, storing the result in `x`. + + :arg x: A variable in which the loaded data is stored. + + + .. py:method:: save(x) + + Save the value of `x`. + + :arg x: A variable whose value should be saved. + + + diff --git a/_sources/autoapi/tlm_adjoint/tangent_linear/index.rst.txt b/_sources/autoapi/tlm_adjoint/tangent_linear/index.rst.txt new file mode 100644 index 0000000..c148b01 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/tangent_linear/index.rst.txt @@ -0,0 +1,56 @@ +:orphan: + +:py:mod:`tlm_adjoint.tangent_linear` +==================================== + +.. py:module:: tlm_adjoint.tangent_linear + + +Module Contents +--------------- + +.. py:class:: TangentLinearMap(M, dM) + + + Defines a map from forward variables to associated tangent-linear + variables. + + The map is used via e.g. + + .. code-block:: python + + tau_x = tlm_map[x] + + where `x` is a forward variable. + + - If `x` defines a component of the control, then `tau_x` is a variable + defining the associated component of the direction. + - If `x` does not define a component of the control and is not + 'static', then `tau_x` is a tangent-linear variable. A new variable + is instantiated if needed. + - Otherwise `tau_x` is `None`, indicating that the tangent-linear + variable is zero. + + Containment can be tested + + .. code-block:: python + + if x in tlm_map: + [...] + + and returns `True` if `x` defines a component of the control, or a + tangent-linear variable associated with `x` has been instantiated. + + :arg M: A variable or :class:`Sequence` of variables defining the control. + :arg dM: A variable or :class:`Sequence` of variables defining the + derivative direction. The tangent-linear computes directional + derivatives with respect to the control defined by `M` and with + direction defined by `dM`. + + .. py:property:: id + + A unique :class:`int` ID associated with this + :class:`.TangentLinearMap`. + + + diff --git a/_sources/autoapi/tlm_adjoint/tlm_adjoint/index.rst.txt b/_sources/autoapi/tlm_adjoint/tlm_adjoint/index.rst.txt new file mode 100644 index 0000000..f20c746 --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/tlm_adjoint/index.rst.txt @@ -0,0 +1,394 @@ +:orphan: + +:py:mod:`tlm_adjoint.tlm_adjoint` +================================= + +.. py:module:: tlm_adjoint.tlm_adjoint + + +Module Contents +--------------- + +.. py:class:: EquationManager(*, comm=None, cp_method='memory', cp_parameters=None) + + + Core manager class. + + - Plays the role of an adjoint 'tape'. Records forward equations as + they are solved. + - Interacts with checkpointing schedules for adjoint checkpointing and + forward replay. + - Derives and manages tangent-linear equations. Tangent-linear + equations are processed as new forward equations, allowing higher + order adjoint calculations. + - Handles variable reference dropping, e.g. handles the dropping of + references to variables which store values, and their replacement + with symbolic equivalents, after :class:`.Equation` objects holding + those references have been destroyed. Internally the manager retains + a reference to a :class:`.WeakAlias` subclass so that the + :class:`.Equation` methods may be called after the original + :class:`.Equation` is destroyed. + + The manager processes forward equations (and tangent-linear equations) as + they are solved. Equations are collected into 'blocks' of equations, + corresponding to 'steps' in step-based checkpointing schedules. For + checkpointing schedule configuration details see + :meth:`.EquationManager.configure_checkpointing`. + + The configuration of tangent-linears is defined by a tangent-linear tree. + The root node of this tree corresponds to the forward. Following :math:`n` + edges from the root node leads to a node associated with an :math:`n` th + order tangent-linear. For tangent-linear configuration details see + :meth:`.EquationManager.configure_tlm`. + + On instantiation both equation annotation and tangent-linear derivation and + solution are *disabled*. + + :arg comm: The communicator associated with the manager. + :arg cp_method: See :meth:`.EquationManager.configure_checkpointing`. + :arg cp_parameters: See :meth:`.EquationManager.configure_checkpointing`. + + .. py:method:: comm() + + :returns: The communicator associated with the manager. + + + .. py:method:: info(*, info=print) + + Print manager state information. + + :arg info: A callable which accepts and prints a :class:`str`. + + + .. py:method:: new(cp_method=None, cp_parameters=None) + + Construct a new :class:`.EquationManager` sharing the communicator + with this :class:`.EquationManager`. By default the new + :class:`.EquationManager` also shares the checkpointing schedule + configuration with this :class:`.EquationManager`, but this may be + overridden with the arguments `cp_method` and `cp_parameters`. + + Both equation annotation and tangent-linear derivation and solution are + *disabled* for the new :class:`.EquationManager`. + + :arg cp_method: See :meth:`.EquationManager.configure_checkpointing`. + :arg cp_parameters: See + :meth:`.EquationManager.configure_checkpointing`. + + + .. py:method:: reset(cp_method=None, cp_parameters=None) + + Reset the :class:`.EquationManager`. Clears all recorded equations, + and all configured tangent-linears. + + By default the :class:`.EquationManager` *retains* its previous + checkpointing schedule configuration, but this may be overridden with + the arguments `cp_method` and `cp_parameters`. + + Both equation annotation and tangent-linear derivation and solution are + *disabled* after calling this method. + + :arg cp_method: See :meth:`.EquationManager.configure_checkpointing`. + :arg cp_parameters: See + :meth:`.EquationManager.configure_checkpointing`. + + + .. py:method:: configure_checkpointing(cp_method, cp_parameters) + + Provide a new checkpointing schedule configuration. + + The checkpointing schedule type is defined by the argument `cp_method`, + and detailed configuration options are provided by the :class:`Mapping` + argument `cp_parameters`. + + `cp_method` values: + + - `'none'`: No checkpointing. Can be used for tangent-linear only + calculations. Options defined by `cp_parameters`: + + - `'drop_references'`: Whether to automatically drop references + to variables which store values. :class:`bool`, optional, + default `False`. + + - `'memory'`: Store all forward restart data and non-linear + dependency data in memory. Options defined by `cp_parameters`: + + - `'drop_references'`: Whether to automatically drop references + to variables which store values. :class:`bool`, optional, + default `False`. + + - `'periodic_disk`: Periodically store forward restart data on + disk. Options defined by `cp_parameters`: + + - `'path'`: Directory in which disk checkpoint data should be + stored. :class:`str`, optional, default `'checkpoints~'`. + - `'format'`: Disk storage format. Either `'pickle'`, for + data storage using the pickle module, or `'hdf5'`, for data + storage using the h5py library. + - `'period'`: Interval, in blocks, between storage of forward + restart data. :class:`int`, required. + + - `'multistage'`: Forward restart checkpointing with checkpoint + distribution as described in: + + - Andreas Griewank and Andrea Walther, 'Algorithm 799: + revolve: an implementation of checkpointing for the reverse + or adjoint mode of computational differentiation', ACM + Transactions on Mathematical Software, 26(1), pp. 19--45, + 2000, doi: 10.1145/347837.347846 + + The memory/disk storage distribution is determined by an + initial run of the checkpointing schedule, leading to a + distribution equivalent to that in: + + - Philipp Stumm and Andrea Walther, 'MultiStage approaches + for optimal offline checkpointing', SIAM Journal on + Scientific Computing, 31(3), pp. 1946--1967, 2009, doi: + 10.1137/080718036 + + Options defined by `cp_parameters`: + + - `'path'`: Directory in which disk checkpoint data should be + stored. :class:`str`, optional, default `'checkpoints~'`. + - `'format'`: Disk storage format. Either `'pickle'`, for + data storage using the pickle module, or `'hdf5'`, for data + storage using the h5py library. + - `'blocks'`: The total number of blocks. :class:`int`, + required. + - `'snaps_in_ram'`: Maximum number of memory checkpoints. + :class:`int`, optional, default 0. + - `'snaps_on_disk'`: Maximum number of disk checkpoints. + :class:`int`, optional, default 0. + + The name 'multistage' originates from the corresponding + `strategy` argument value for the + `dolfin_adjoint.solving.adj_checkpointing` function in + dolfin-adjoint (see e.g. version 2017.1.0). The parameter names + `snaps_in_ram` and `snaps_on_disk` originate from the + corresponding arguments for the + `dolfin_adjoint.solving.adj_checkpointing` function in + dolfin-adjoint (see e.g. version 2017.1.0). + + - A callable: A callable returning a :class:`.CheckpointSchedule`. + Options defined by `cp_parameters`: + + - `'path'`: Directory in which disk checkpoint data should be + stored. :class:`str`, optional, default `'checkpoints~'`. + - `'format'`: Disk storage format. Either `'pickle'`, for + data storage using the pickle module, or `'hdf5'`, for data + storage using the h5py library. + - Other parameters are passed as keyword arguments to the + callable. + + + .. py:method:: configure_tlm(*args, annotate=None, tlm=True) + + Configure the tangent-linear tree. + + :arg args: A :class:`tuple` of `(M_i, dM_i)` pairs. `M_i` is a variable + or a :class:`Sequence` of variables defining a control. `dM_i` is a + variable or a :class:`Sequence` of variables defining a derivative + direction. Identifies a node in the tree (and hence identifies a + tangent-linear) corresponding to differentation, in order, with + respect to the each control defined by `M_i` and with each + direction defined by `dM_i`. + :arg annotate: If `True` then enable annotation for the identified + tangent-linear, and enable annotation for all tangent-linears on + which it depends. If `False` then disable annotation for the + identified tangent-linear, all tangent-linears which depend on it, + and any newly added tangent-linears. Defaults to `tlm`. + :arg tlm: If `True` then add (or retain) the identified tangent-linear, + and add all tangent-linears on which it depends. If `False` then + remove the identified tangent-linear, and remove all + tangent-linears which depend on it. + + + .. py:method:: tlm_enabled() + + :returns: Whether derivation and solution of tangent-linear equations + is enabled. + + + .. py:method:: var_tlm(x, *args) + + Return a tangent-linear variable. + + :arg x: A variable whose tangent-linear variable should be returned. + Cannot not be a replacement. + :arg args: Identifies the tangent-linear. See + :meth:`.EquationManager.configure_tlm`. + :returns: The tangent-linear variable. + + + .. py:method:: function_tlm(x, *args) + + + + + .. py:method:: annotation_enabled() + + :returns: Whether recording of equations is enabled. + + + .. py:method:: start(*, annotate=True, tlm=True) + + Start recording of equations and derivation and solution of + tangent-linear equations. + + :arg annotate: Whether recording of equations should be enabled. + :arg tlm: Whether derivation and solution of tangent-linear equations + should be enabled. + + + .. py:method:: stop(*, annotate=True, tlm=True) + + Stop recording of equations and derivation and solution of + tangent-linear equations. + + :arg annotate: Whether recording of equations should be disabled. + :arg tlm: Whether derivation and solution of tangent-linear equations + should be disabled. + :returns: A :class:`tuple` `(annotation_state, tlm_state)`. + `annotation_state` is a :class:`bool` indicating whether recording + of equations was enabled prior to the call to + :meth:`.EquationManager.stop`. `tlm_state` is a :class:`bool` + indicating whether derivation and solution of tangent-linear + equations was enabled prior to the call to + :meth:`.EquationManager.stop`. + + + .. py:method:: paused(*, annotate=True, tlm=True) + + Construct a context manager which can be used to temporarily disable + recording of equations and derivation and solution of tangent-linear + equations. + + :arg annotate: Whether recording of equations should be temporarily + disabled. + :arg tlm: Whether derivation and solution of tangent-linear equations + should be temporarily disabled. + :returns: A context manager which can be used to temporarily disable + recording of equations and derivation and solution of + tangent-linear equations. + + + .. py:method:: add_initial_condition(x, *, annotate=None) + + Process an 'initial condition' -- a variable whose associated value + is needed prior to solving an equation. + + :arg x: A variable defining the initial condition. + :arg annotate: Whether the initial condition should be recorded. + + + .. py:method:: add_equation(eq, *, annotate=None, tlm=None) + + Process an :class:`.Equation` after it has been solved. + + :arg eq: The :class:`.Equation`. + :arg annotate: Whether solution of this equation, and any + tangent-linear equations, should be recorded. + :arg tlm: Whether tangent-linear equations should be derived and + solved. + + + .. py:method:: drop_references() + + Drop references to variables which store values, referenced by + objects which have been destroyed, and replace them symbolic + equivalents. + + + .. py:method:: new_block() + + End the current block of equations, and begin a new block. The + blocks of equations correspond to the 'steps' in step-based + checkpointing schedules. + + + .. py:method:: finalize() + + End the final block of equations. Equations cannot be recorded, and + new tangent-linear equations cannot be derived and solved, after a call + to this method. + + Called by :meth:`.EquationManager.compute_gradient`, and typically need + not be called manually. + + + .. py:method:: compute_gradient(Js, M, *, callback=None, prune_forward=True, prune_adjoint=True, prune_replay=True, cache_adjoint_degree=None, store_adjoint=False, adj_ics=None) + + Core adjoint driver method. + + Compute the derivative of one or more functionals with respect to one + or more controls, using an adjoint approach. + + :arg Js: A variable or a sequence of variables defining the functionals + to differentiate. + :arg M: A variable or a :class:`Sequence` of variables defining the + controls. Derivatives with respect to the controls are computed. + :arg callback: Diagnostic callback. A callable of the form + + .. code-block:: python + + def callback(J_i, n, i, eq, adj_X): + + with + + - `J_i`: An :class:`int` defining the index of the functional. + - `n`: An :class:`int` definining the index of the block of + equations. + - `i`: An :class:`int` defining the index of the considered + equation in block `n`. + - `eq`: The :class:`.Equation`, equation `i` in block `n`. + - `adj_X`: The adjoint solution associated with equation `i` in + block `n` for the `J_i` th functional. `None` indicates that + the solution is zero or is not computed (due to an activity + analysis). Otherwise a variable if `eq` has a single solution + component, and a :class:`Sequence` of variables otherwise. + + :arg prune_forward: Controls the activity analysis. Whether a forward + traversal of the computational graph, tracing variables which + depend on the controls, should be applied. + :arg prune_adjoint: Controls the activity analysis. Whether a reverse + traversal of the computational graph, tracing variables on which + the functionals depend, should be applied. + :arg prune_replay: Controls the activity analysis. Whether an activity + analysis should be applied when solving forward equations during + checkpointing/replay. + :arg cache_adjoint_degree: Adjoint solutions can be cached and reused + across adjoints where the solution is the same -- e.g. first order + adjoint solutions associated with the same functional and same + block and equation indices are equal. A value of `None` indicates + that caching should be applied at all degrees, a value of 0 + indicates that no caching should be applied, and any positive + :class:`int` indicates that caching should be applied for adjoints + up to and including degree `cache_adjoint_degree`. + :arg store_adjoint: Whether cached adjoint solutions should be retained + after the call to this method. Can be used to cache and reuse first + order adjoint solutions in multiple calls to this method. + :arg adj_ics: A :class:`Mapping`. Items are `(x, value)` where `x` is a + variable or variable ID identifying a forward variable. The adjoint + variable associated with the final equation solving for `x` is + initialized to the value stored by the variable `value`. + :returns: The conjugate of the derivatives. The return type depends on + the type of `Js` and `M`. + + - If `Js` is a variable and `M` is a variable, returns a variable + storing the conjugate of the derivative. + - If `Js` is a :class:`Sequence`, and `M` is a variable, returns + a variable whose :math:`i` th component stores the conjugate of + the derivative of the :math:`i` th functional. + - If `Js` is a variable and `M` is a :class:`Sequence`, returns a + :class:`Sequence` of variables whose :math:`j` th component + stores the conjugate of the derivative with respect to the + :math:`j` th control. + - If both `Js` and `M` are :class:`Sequence` objects, returns a + :class:`Sequence` whose :math:`i` th component stores the + conjugate of the derivatives of the :math:`i` th functional. + Each of these is a :class:`Sequence` of variables whose + :math:`j` th component stores the conjugate of the derivative + with respect to the :math:`j` th control. + + + diff --git a/_sources/autoapi/tlm_adjoint/verification/index.rst.txt b/_sources/autoapi/tlm_adjoint/verification/index.rst.txt new file mode 100644 index 0000000..d994ccf --- /dev/null +++ b/_sources/autoapi/tlm_adjoint/verification/index.rst.txt @@ -0,0 +1,230 @@ +:py:mod:`tlm_adjoint.verification` +================================== + +.. py:module:: tlm_adjoint.verification + +.. autoapi-nested-parse:: + + This module implements Taylor remainder convergence testing using the + approach described in + + - P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, 'Automated + derivation of the adjoint of high-level transient finite element + programs', SIAM Journal on Scientific Computing 35(4), pp. C369--C393, + 2013, doi: 10.1137/120873558 + + Specifically for a sufficiently regular functional :math:`J`, via Taylor's + theorem we have, for some direction :math:`\zeta` and with perturbation + magnitude controlled by :math:`\varepsilon`, + + .. math:: + + \left| J \left( m + \varepsilon \right) - J \left( m \right) \right| + = O \left( \varepsilon \right), + + .. math:: + + \left| J \left( m + \varepsilon \right) - J \left( m \right) + - \varepsilon dJ \left( m; \zeta \right) \right| + = O \left( \varepsilon^2 \right), + + where here :math:`dJ \left( m; \zeta \right)` denotes the directional + derivative of :math:`J` with respect to :math:`m` with direction :math:`\zeta`. + Here we refer to the quantity appearing on the left-hand-side in the first case + as the 'uncorrected Taylor remainder magnitude', and the quantity appearing on + the left-hand-side in the second case as the 'corrected Taylor remainder + magnitude' + + A Taylor remainder convergence test considers some direction, and a number of + different values for :math:`\varepsilon`, and investigates the convergence + rates of the uncorrected and corrected Taylor remainder magnitudes, with the + directional derivative computed using a tangent-linear or adjoint. In a + successful verification the uncorrected Taylor remainder magnitude is observed + to converge to zero at first order, while the corrected Taylor remainder + magnitude is observed to converge to zero at second order. + + There are a number of ways that a Taylor remainder convergence test can fail, + including: + + - The computed derivative is incorrect. This is the case that the test is + designed to find, and indicates an error in the tangent-linear or adjoint + calculation. + - The considered values of :math:`\varepsilon` are too large, and the + asymptotic convergence orders are not observable. + - The considered values of :math:`\varepsilon` are too small, and iterative + solver tolerances or floating point roundoff prevent the convergence + orders being observable. + - The convergence order is higher than expected. For example if the + directional derivative is zero then the uncorrected Taylor remainder + magnitude can converge at higher than first order. + + In principle higher order derivative calculations can be tested by considering + more terms in the Taylor expansion of the functional. In practice the + corresponding higher order convergence rate can mean that iterative solver + tolerances or floating point roundoff effects are more problematic. Instead, + one can verify the derivative of a derivative, by redefining :math:`J` to be a + directional derivative of some other functional :math:`K`, with the directional + derivative computed using a tangent-linear. A successful verification then once + again corresponds to second order convergence of the corrected Taylor remainder + magnitude. + + The functions defined in this module log the uncorrected and corrected Taylor + remainder magnitudes, and also log the observed orders computed using a power + law fit between between consecutive pairs of values of :math:`\varepsilon`. + Logging is performed on a logging module logger, with name + `'tlm_adjoint.verification`' and with severity `logging.INFO`. The minimum + order computed for the corrected Taylor remainder magnitude is returned. + + A typical test considers tangent-linears and adjoints up to the relevant order, + e.g. to verify Hessian calculations + + .. code-block:: python + + min_order = taylor_test_tlm(forward, M, 1) + assert min_order > 1.99 + + min_order = taylor_test_tlm_adjoint(forward, M, 1) + assert min_order > 1.99 + + min_order = taylor_test_tlm_adjoint(forward, M, 2) + assert min_order > 1.99 + + + +Module Contents +--------------- + +.. py:function:: taylor_test(forward, M, J_val, *, dJ=None, ddJ=None, seed=0.01, dM=None, M0=None, size=5) + + Perform a Taylor remainder convergence test. Aims for similar behaviour + to the `taylor_test` function in dolfin-adjoint 2017.1.0. + + Uncorrected and corrected Taylor remainder magnitudes are computed by + repeatedly re-running the forward and evaluating the functional. The + perturbation direction :math:`\zeta` is defined by the `dM` argument. + :math:`\varepsilon` is set equal to + + .. math:: + + \varepsilon = 2^{-p} \eta \max \left( 1, + \left\| m \right\|_{l_\infty} \right) + \quad \text{ for } p \in \left\{ 0, \ldots, P - 1 \right\}, + + where the norm appearing here is defined to be the :math:`l_\infty` norm of + the control value degree of freedom vector. The argument `seed` sets the + value of :math:`\eta`, and the argument `size` sets the value of :math:`P`. + + :arg forward: A callable which accepts one or more variable arguments, and + which returns a variable defining the forward functional :math:`J`. + Corresponds to the `J` argument in the dolfin-adjoint `taylor_test` + function. + :arg M: A variable or a :class:`Sequence` of variables defining the control + :math:`m`. Corresponds to the `m` argument in the dolfin-adjoint + `taylor_test` function. + :arg J_val: A scalar defining the value of the functional :math:`J` for + control value defined by `M0`. Corresponds to the `Jm` argument in the + dolfin-adjoint `taylor_test` function. + :arg dJ: A variable or a :class:`Sequence` of variables defining a value + for the derivative of the functional with respect to the control. + Required if `ddJ` is not supplied. Corresponds to the `dJdm` argument + in the dolfin-adjoint `taylor_test` function. + :arg ddJ: A :class:`.Hessian` used to compute the Hessian action on the + considered perturbation direction. If supplied then a higher order + corrected Taylor remainder magnitude is computed. If `dJ` is not + supplied, also computes the first order directional derivative. + Corresponds to the `HJm` argument in the dolfin-adjoint `taylor_test` + function. + :arg seed: Defines the value of :math:`\eta`. Controls the magnitude of the + perturbation. Corresponds to the `seed` argument in the dolfin-adjoint + `taylor_test` function. + :arg dM: Defines the perturbation direction :math:`\zeta`. A direction with + degrees of freedom vector real and (in the complex case) complex parts + set by :func:`numpy.random.random` is used if not supplied. Corresponds + to the `perturbation_direction` argument in the dolfin-adjoint + `taylor_test` function. + :arg M0: Defines the value of the control at which the functional and + derivatives are evaluated. `M` is used if not supplied. Corresponds to + the `value` argument in the dolfin-adjoint `taylor_test` function. + :arg size: The number of values of :math:`\varepsilon` to consider. + Corresponds to the `size` argument in the dolfin-adjoint `taylor_test` + function. + :returns: The minimum order observed, via a power law fit between + consecutive pairs of values of :math:`\varepsilon`, in the calculations + for the corrected Taylor remainder magnitude. In a successful + verification this should be close to 2 if `ddJ` is not supplied, and + close to 3 if `ddJ` is supplied. + + +.. py:function:: taylor_test_tlm(forward, M, tlm_order, *, seed=0.01, dMs=None, size=5, manager=None) + + Perform a Taylor remainder convergence test for a functional :math:`J` + defined to the `(tlm_order - 1)` th derivative of some functional + :math:`K`. The `tlm_order` th derivative of :math:`K`, appearing in the + corrected Taylor remainder magnitude, is computed using a `tlm_order` th + order tangent-linear. + + :arg forward: A callable which accepts one or more variable arguments, and + which returns a variable defining the forward functional :math:`K`. + :arg M: A variable or a :class:`Sequence` of variables defining the control + :math:`m` and its value. + :arg tlm_order: An :class:`int` defining the tangent-linear order to + test. + :arg seed: Controls the perturbation magnitude. See :func:`.taylor_test`. + :arg dMs: A :class:`Sequence` of length `tlm_order` whose elements are each + a variable or a :class:`Sequence` of variables. The functional + :math:`J` appearing in the definition of the Taylor remainder + magnitudes is defined to be a `(tlm_adjoint - 1)` th derivative, + defined by successively taking the derivative of :math:`K` with respect + to the control and with directions defined by the `dM[:-1]` (with the + directions considered in order). The perturbation direction + :math:`\zeta` is defined by `dM[-1]` -- see :func:`.taylor_test`. + Directions with degrees of freedom vector real and (in the complex + case) complex parts set by :func:`numpy.random.random` are used if not + supplied. + :arg size: The number of values of :math:`\varepsilon` to consider. See + :func:`.taylor_test`. + :arg manager: An :class:`.EquationManager` used to create an internal + manager via :meth:`.EquationManager.new`. `manager()` is used if not + supplied. + :returns: The minimum order observed, via a power law fit between + consecutive pairs of values of :math:`\varepsilon`, in the calculations + for the corrected Taylor remainder magnitude. In a successful + verification this should be close to 2. + + +.. py:function:: taylor_test_tlm_adjoint(forward, M, adjoint_order, *, seed=0.01, dMs=None, size=5, manager=None) + + Perform a Taylor remainder convergence test for a functional :math:`J` + defined to the `(adjoint_order - 1)` th derivative of some functional + :math:`K`. The `adjoint_order` th derivative of :math:`K`, appearing in the + corrected Taylor remainder magnitude, is computed using an adjoint + associated with an `(adjoint_order - 1)` th order tangent-linear. + + :arg forward: A callable which accepts one or more variable arguments, and + which returns a variable defining the forward functional :math:`K`. + :arg M: A variable or a :class:`Sequence` of variables defining the control + :math:`m` and its value. + :arg adjoint_order: An :class:`int` defining the adjoint order to test. + :arg seed: Controls the perturbation magnitude. See :func:`.taylor_test`. + :arg dMs: A :class:`Sequence` of length `adjoint_order` whose elements are + each a variable or a :class:`Sequence` of variables. The functional + :math:`J` appearing in the definition of the Taylor remainder + magnitudes is defined to be a `(adjoint_order - 1)` th derivative, + defined by successively taking the derivative of :math:`K` with respect + to the control and with directions defined by the `dM[:-1]` (with the + directions considered in order). The perturbation direction + :math:`\zeta` is defined by `dM[-1]` -- see :func:`.taylor_test`. + Directions with degrees of freedom vector real and (in the complex + case) complex parts set by :func:`numpy.random.random` are used if not + supplied. + :arg size: The number of values of :math:`\varepsilon` to consider. See + :func:`.taylor_test`. + :arg manager: An :class:`.EquationManager` used to create an internal + manager via :meth:`.EquationManager.new`. `manager()` is used if not + supplied. + :returns: The minimum order observed, via a power law fit between + consecutive pairs of values of :math:`\varepsilon`, in the calculations + for the corrected Taylor remainder magnitude. In a successful + verification this should be close to 2. + + diff --git a/_sources/dependencies.rst.txt b/_sources/dependencies.rst.txt new file mode 100644 index 0000000..d3129a0 --- /dev/null +++ b/_sources/dependencies.rst.txt @@ -0,0 +1,45 @@ +Dependencies +============ + +Required +-------- + +tlm_adjoint requires: + + - `NumPy `_ + - `SymPy `_ + +Backend dependencies +-------------------- + +With the FEniCS backend tlm_adjoint requires: + + - `DOLFIN `_ + - `FFC `_ + - `UFL `_ + - `mpi4py `_ + - `PETSc `_ + +With the Firedrake backend tlm_adjoint requires: + + - `Firedrake `_ + - `PyOP2 `_ + - `UFL `_ + - `mpi4py `_ + - `PETSc `_ + +Optional +-------- + +Some features of tlm_adjoint require: + + - `JAX `_ + - `H-Revolve `_ + - `h5py `_ + - `SciPy `_ + - `SLEPc `_ + +While not required, if available some features of tlm_adjoint use: + + - `more-itertools `_ + - `Numba `_ diff --git a/_sources/examples/0_getting_started.ipynb.txt b/_sources/examples/0_getting_started.ipynb.txt new file mode 100644 index 0000000..1d56599 --- /dev/null +++ b/_sources/examples/0_getting_started.ipynb.txt @@ -0,0 +1,462 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "40c3ebad", + "metadata": {}, + "source": [ + "# Getting started with tlm_adjoint\n", + "\n", + "This notebook introduces derivative calculations using tlm_adjoint.\n", + "\n", + "tlm_adjoint is primarily intended for first derivative calculations using the adjoint method, and Hessian information calculations using the adjoint method applied to tangent-linear and forward calculations. However the approach used by tlm_adjoint generalizes to higher order.\n", + "\n", + "The approach used by tlm_adjoint for higher order differentiation is described in:\n", + "\n", + "- James R. Maddison, Daniel N. Goldberg, and Benjamin D. Goddard, 'Automated calculation of higher order partial differential equation constrained derivative information', SIAM Journal on Scientific Computing, 41(5), pp. C417–C445, 2019, doi: 10.1137/18M1209465\n", + "\n", + "## A floating point example\n", + "\n", + "We consider a simple floating point calculation:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0c79382a", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "\n", + "x = 1.0\n", + "y = 0.25 * np.pi\n", + "z = x * np.sin(y * np.exp(y))" + ] + }, + { + "cell_type": "markdown", + "id": "e82699c6", + "metadata": {}, + "source": [ + "tlm_adjoint is designed for high-level algorithmic differentiation, and not this type of low-level floating point calculation. However it *can* still process simple floating point calculations, so to introduce the key ideas we do that here. We consider differentiating `z` with respect to `x` and `y` – being precise, we mean computing the derivative of the function used to compute `z` with respect to the variables defined by `x` and `y`.\n", + "\n", + "## Adding tlm_adjoint\n", + "\n", + "We first modify the code so that tlm_adjoint processes the calculations:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dfb3955a", + "metadata": {}, + "outputs": [], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(x, y):\n", + " z = x * np.sin(y * np.exp(y))\n", + " return z\n", + "\n", + "\n", + "x = Float(1.0, name=\"x\")\n", + "y = Float(0.25 * np.pi, name=\"y\")\n", + "\n", + "start_manager()\n", + "z = forward(x, y)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "4bb98432", + "metadata": {}, + "source": [ + "The key changes here are:\n", + "\n", + "- To import tlm_adjoint.\n", + "- Controlling the 'manager' – an object tlm_adjoint uses to process equations. The manager is first reset using `reset_manager`. This clears any previous processing, and disables the manager. `start_manager` and `stop_manager` are then used to enable the manager just when it is needed.\n", + "- Defining `x` and `y` to be of type `Float`. Calculations involving `x` and `y` are recorded by the manager. The result of the calculations – here `z` – will have the same type, and we can access its value with `float(z)`.\n", + "\n", + "Let's display the information recorded by tlm_adjoint:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ba11e403", + "metadata": {}, + "outputs": [], + "source": [ + "manager_info()" + ] + }, + { + "cell_type": "markdown", + "id": "a0138110", + "metadata": {}, + "source": [ + "The key feature here is that there are four `FloatEquation` records, corresponding to the four floating point calculations – evaluation using `np.exp` and `np.sin`, and two multiplications.\n", + "\n", + "## Computing derivatives using an adjoint\n", + "\n", + "The `compute_gradient` function can be used to differentiate `z` with respect to `x` and `y` using the adjoint method:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "66f6f440", + "metadata": {}, + "outputs": [], + "source": [ + "dz_dx, dz_dy = compute_gradient(z, (x, y))\n", + "\n", + "print(f\"{float(dz_dx)=}\")\n", + "print(f\"{float(dz_dy)=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "1796fdfd", + "metadata": {}, + "source": [ + "For a simple check of the result, we can compare with the result from finite differencing, here using second order centered finite differencing:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "28d0c8a0", + "metadata": {}, + "outputs": [], + "source": [ + "def dJ_dm(J, m, *, eps=1.0e-7):\n", + " return (J(m + eps) - J(m - eps)) / (2.0 * eps)\n", + "\n", + "\n", + "print(f\"dz_dx approximation = {dJ_dm(lambda x: forward(x, float(y)), float(x))}\")\n", + "print(f\"dz_dy approximation = {dJ_dm(lambda y: forward(float(x), y), float(y))}\")" + ] + }, + { + "cell_type": "markdown", + "id": "4ef1b47b", + "metadata": {}, + "source": [ + "## Computing derivatives using a tangent-linear\n", + "\n", + "A tangent-linear computes directional derivatives with respect to a given control and with a given direction.\n", + "\n", + "Here we consider the forward to be a function of `x` and `y`, computing a value of `z`, denoted $z \\left( x, y \\right)$. We consider the control $m = \\left( x, y \\right)^T$, and a direction $\\zeta = \\left( 2, 3 \\right)^T$. We can then use a tangent-linear to compute the directional derivative\n", + "\n", + "$$\n", + " \\frac{dz}{dm} \\zeta = 2 \\frac{dz}{dx} + 3 \\frac{dz}{dy},\n", + "$$\n", + "\n", + "where vector derivatives are notated using row vectors.\n", + "\n", + "In most cases tlm_adjoint needs to be told what tangent-linear calculations to perform *ahead* of the forward calculations. `configure_tlm` provides this information to tlm_adjoint:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0220eb94", + "metadata": {}, + "outputs": [], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(x, y):\n", + " z = x * np.sin(y * np.exp(y))\n", + " return z\n", + "\n", + "\n", + "x = Float(1.0, name=\"x\")\n", + "y = Float(0.25 * np.pi, name=\"y\")\n", + "\n", + "m = (x, y)\n", + "zeta = (Float(2.0, name=\"zeta_x\"), Float(3.0, name=\"zeta_y\"))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "z = forward(x, y)\n", + "stop_manager()\n", + "\n", + "dz_dm_zeta = var_tlm(z, (m, zeta))\n", + "\n", + "print(f\"{float(dz_dm_zeta)=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "c00043a7", + "metadata": {}, + "source": [ + "There are three new changes:\n", + "\n", + "- The control $m$ and direction $\\zeta$ are defined using `m` and `zeta` respectively.\n", + "- Tangent-linear calculations are configured *before* the forward calculations are performed, using `configure_tlm`. Note that here, for this first derivative calculation, the argument is a single control-direction pair.\n", + "- We access the tangent-linear variable, containing the value of $\\left( dz/dm \\right) \\zeta$, using `var_tlm`, and using the same control-direction pair.\n", + "\n", + "In fact more has happened here – if we now display the information recorded by tlm_adjoint:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5b2a2150", + "metadata": {}, + "outputs": [], + "source": [ + "manager_info()" + ] + }, + { + "cell_type": "markdown", + "id": "527f3b8b", + "metadata": {}, + "source": [ + "we now see that there are *eight* `FloatEquation` records – the original four, and four new ones. The extra ones correspond to the tangent-linear calculations. tlm_adjoint has recorded the original forward calculations, and *also* recorded the tangent-linear calculations.\n", + "\n", + "## Computing second derivatives using an adjoint of a tangent-linear\n", + "\n", + "Since tlm_adjoint has recorded both forward and tangent-linear calculations, we can now compute second derivative information using an adjoint associated with a tangent-linear. Specifically we can compute a Hessian action on $\\zeta$,\n", + "\n", + "$$\n", + " H \\zeta = \\frac{d}{dm} \\left( \\frac{dz}{dm} \\zeta \\right)^T.\n", + "$$\n", + "\n", + "The inner directional derivative appearing here is computed using the tangent-linear method, and the outer derivative is computed by applying the adjoint method to the tangent-linear and forward calculations. In code we simply use `compute_gradient` to compute the derivative of the tangent-linear result:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8612e275", + "metadata": {}, + "outputs": [], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(x, y):\n", + " z = x * np.sin(y * np.exp(y))\n", + " return z\n", + "\n", + "\n", + "x = Float(1.0, name=\"x\")\n", + "y = Float(0.25 * np.pi, name=\"y\")\n", + "\n", + "m = (x, y)\n", + "zeta = (Float(2.0, name=\"zeta_x\"), Float(3.0, name=\"zeta_y\"))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "z = forward(x, y)\n", + "stop_manager()\n", + "\n", + "dz_dm_zeta = var_tlm(z, (m, zeta))\n", + "\n", + "print(f\"{float(dz_dm_zeta)=}\")\n", + "\n", + "d2z_dm_zeta_dx, d2z_dm_zeta_dy = compute_gradient(dz_dm_zeta, m)\n", + "\n", + "print(f\"{float(d2z_dm_zeta_dx)=}\")\n", + "print(f\"{float(d2z_dm_zeta_dy)=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "b4b0fac5", + "metadata": {}, + "source": [ + "## Computing second derivatives using a tangent-linear of a tangent-linear\n", + "\n", + "We can also compute second derivative information using a tangent-linear associated with a tangent-linear. For example if we define $e_1 = \\left( 1, 0 \\right)^T$, then we can find the first component of the previously computed Hessian action on $\\zeta$ via\n", + "\n", + "$$\n", + " e_1^T H \\zeta = \\left[ \\frac{d}{dm} \\left( \\frac{dz}{dm} \\zeta \\right) \\right] e_1.\n", + "$$\n", + "\n", + "That is, here we now want to compute a directional derivative of a directional derivative, and we compute this using a tangent-linear associated with the previous tangent-linear and forward calculations.\n", + "\n", + "tlm_adjoint handles this case by supplying more arguments to `configure_tlm`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5c02e84f", + "metadata": {}, + "outputs": [], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(x, y):\n", + " z = x * np.sin(y * np.exp(y))\n", + " return z\n", + "\n", + "\n", + "x = Float(1.0, name=\"x\")\n", + "y = Float(0.25 * np.pi, name=\"y\")\n", + "\n", + "m = (x, y)\n", + "zeta = (Float(2.0, name=\"zeta_x\"), Float(3.0, name=\"zeta_y\"))\n", + "e_1 = (Float(1.0, name=\"e_1_x\"), Float(0.0, name=\"e_1_y\"))\n", + "configure_tlm((m, zeta), (m, e_1))\n", + "\n", + "start_manager()\n", + "z = forward(x, y)\n", + "stop_manager()\n", + "\n", + "dz_dm_zeta = var_tlm(z, (m, zeta))\n", + "dz_dx = var_tlm(z, (m, e_1))\n", + "d2z_dm_zeta_dx = var_tlm(z, (m, zeta), (m, e_1))\n", + "\n", + "print(f\"{float(dz_dm_zeta)=}\")\n", + "print(f\"{float(dz_dx)=}\")\n", + "print(f\"{float(d2z_dm_zeta_dx)=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "88b3a4f1", + "metadata": {}, + "source": [ + "The first control-direction pair supplied to `configure_tlm` indicates that we seek to compute directional derivatives of the forward with respect to the control defined by `m` with direction defined by `zeta`. The second control-direction pair indicates that we seek to compute directional deriatives of *these* directional derivatives, with respect to the control defined by `m` and with direction defined by `e_1`. As a side-effect we find that we also compute the directional derivatives of the forward with respect to the control defined by `m` with direction defined by `e_1`.\n", + "\n", + "We then access the tangent-linear variables using `var_tlm`, supplying two control-variable pairs to access a second order tangent-linear variable.\n", + "\n", + "As before, tlm_adjoint has not just performed the tangent-linear calculations – if we display the information recorded by tlm_adjoint:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb6739a1", + "metadata": {}, + "outputs": [], + "source": [ + "manager_info()" + ] + }, + { + "cell_type": "markdown", + "id": "1e4fde63", + "metadata": {}, + "source": [ + "we now find that there are *sixteen* `FloatEquation` records, constituting the forward and all tangent-linear calculations.\n", + "\n", + "## Computing third derivatives using an adjoint of a tangent-linear of a tangent-linear\n", + "\n", + "We can now compute the derivative of the tangent-linear-computed second derivative by simply handing the second order tangent-linear variable to `compute_gradient`. This applies the adjoint method to the higher order tangent-linear calculations and the forward calculations, computing\n", + "\n", + "$$\n", + " \\frac{d}{dm} \\left( e_1^T H \\zeta \\right) = \\frac{d}{dm} \\left[ \\left[ \\frac{d}{dm} \\left( \\frac{dz}{dm} \\zeta \\right) \\right] e_1 \\right].\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c60bc4b8", + "metadata": {}, + "outputs": [], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(x, y):\n", + " z = x * np.sin(y * np.exp(y))\n", + " return z\n", + "\n", + "\n", + "x = Float(1.0, name=\"x\")\n", + "y = Float(0.25 * np.pi, name=\"y\")\n", + "\n", + "m = (x, y)\n", + "zeta = (Float(2.0, name=\"zeta_x\"), Float(3.0, name=\"zeta_y\"))\n", + "e_1 = (Float(1.0, name=\"e_1_x\"), Float(0.0, name=\"e_1_y\"))\n", + "configure_tlm((m, zeta), (m, e_1))\n", + "\n", + "start_manager()\n", + "z = forward(x, y)\n", + "stop_manager()\n", + "\n", + "dz_dm_zeta = var_tlm(z, (m, zeta))\n", + "dz_dx = var_tlm(z, (m, e_1))\n", + "d2z_dm_zeta_dx = var_tlm(z, (m, zeta), (m, e_1))\n", + "\n", + "print(f\"{float(dz_dm_zeta)=}\")\n", + "print(f\"{float(dz_dx)=}\")\n", + "print(f\"{float(d2z_dm_zeta_dx)=}\")\n", + "\n", + "d3z_dm_zeta_dx_dx, d3z_dm_zeta_dx_dy = compute_gradient(d2z_dm_zeta_dx, m)\n", + "\n", + "print(f\"{float(d3z_dm_zeta_dx_dx)=}\")\n", + "print(f\"{float(d3z_dm_zeta_dx_dy)=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "b1a3e919", + "metadata": {}, + "source": [ + "## Higher order\n", + "\n", + "The approach now generalizes.\n", + "\n", + "- Supplying further arguments to `configure_tlm` indicates directional derivatives of directional derivatives, defining a tangent-linear calculation of increasingly high order.\n", + "- Supplying these arguments to `var_tlm` accesses the higher-order tangent-linear variables.\n", + "- These higher order tangent-linear variables can be handed to `compute_gradient` to compute derivatives of the higher order derivatives using the adjoint method." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/_sources/examples/1_time_independent.ipynb.txt b/_sources/examples/1_time_independent.ipynb.txt new file mode 100644 index 0000000..ab4cc24 --- /dev/null +++ b/_sources/examples/1_time_independent.ipynb.txt @@ -0,0 +1,810 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "09bb9c5a", + "metadata": {}, + "source": [ + "# Time-independent example\n", + "\n", + "This notebook describes adjoint calculations, and Hessian calculations, using tlm_adjoint with the [Firedrake](https://firedrakeproject.org/) backend. A time-independent problem is considered, and importantly checkpointing is not used for the adjoint calculations. This notebook further describes how variables may be flagged to facilitate caching.\n", + "\n", + "The high-level algorithmic differentiation approach used by tlm_adjoint is based on the method described in:\n", + "\n", + "- P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, 'Automated derivation of the adjoint of high-level transient finite element programs', SIAM Journal on Scientific Computing 35(4), pp. C369–C393, 2013, doi: 10.1137/120873558\n", + "\n", + "The caching of data in tlm_adjoint uses an approach based on that described in:\n", + "\n", + "- J. R. Maddison and P. E. Farrell, 'Rapid development and adjoining of transient finite element models', Computer Methods in Applied Mechanics and Engineering, 276, 95–121, 2014, doi: 10.1016/j.cma.2014.03.010\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the solution of a linear time-independent partial differential equation, followed by the calculation of the $L^2$-norm of the solution. Extra non-linearity is introduced by allowing the right-hand-side of the partial differential equation to depend non-linearly on the control. We assume real spaces and a real build of Firedrake throughout.\n", + "\n", + "Specifically we consider the solution $u \\in V_0$ of\n", + "\n", + "$$\n", + " \\forall \\zeta \\in V_0 \\qquad \\int_\\Omega \\nabla \\zeta \\cdot \\nabla u = \\int_\\Omega \\zeta m^2,\n", + "$$\n", + "\n", + "where $V$ is a real $P_1$ continuous finite element space defining functions on the domain $\\Omega = \\left( 0, 1 \\right)^2$, with $m \\in V$, and where $V_0$ consists of the functions in $V$ which have zero trace. This corresponds to a discretization of the partial differential equation\n", + "\n", + "$$\n", + " -\\nabla^2 u = m^2 \\quad \\text{on } \\left( x, y \\right) \\in \\left( 0, 1 \\right)^2,\n", + "$$\n", + "\n", + "subject to homogeneous Dirichlet boundary conditions.\n", + "\n", + "A simple implementation in Firedrake, with $m = x y$, takes the form:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "227a8702", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "u = Function(space, name=\"u\")\n", + "solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + "J_sq = assemble(inner(u, u) * dx)\n", + "J = sqrt(J_sq)" + ] + }, + { + "cell_type": "markdown", + "id": "fa57fc17", + "metadata": {}, + "source": [ + "## Adding tlm_adjoint\n", + "\n", + "We first modify the code so that tlm_adjoint processes the calculations:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2a3dd1f9", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "c23c7102", + "metadata": {}, + "source": [ + "The key changes here are:\n", + "\n", + "- To import tlm_adjoint with the Firedrake backend.\n", + "- Controlling the 'manager' – an object tlm_adjoint uses to process equations.\n", + "- Using a `Functional` to compute the square of the $L^2$-norm of the solution of the (discretized) partial differential equation. This facilitates the calculation of simple functionals e.g. using finite element assembly.\n", + "- Taking the square root of the square of the $L^2$-norm using NumPy.\n", + "\n", + "Let's display the information recorded by tlm_adjoint:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a34fba5a", + "metadata": {}, + "outputs": [], + "source": [ + "manager_info()" + ] + }, + { + "cell_type": "markdown", + "id": "adc31bf5", + "metadata": {}, + "source": [ + "We see that there are three records.\n", + "\n", + "- Equation 0, an `EquationSolver`. This records the solution of the finite element variational problem for `u`.\n", + "- Equation 1, an `Assembly`. This records the calculation of the square of the $L^2$-norm.\n", + "- Equation 2, a `FloatEquation`. This records the calculation of the square root of the square of the $L^2$-norm.\n", + "\n", + "## Computing derivatives using an adjoint\n", + "\n", + "The `compute_gradient` function can be used to compute derivatives using the adjoint method. Here we compute the derivative of the $L^2$-norm of the resulting solution, considered a function of the control defined by `m`, with respect to this control:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d0a15072", + "metadata": {}, + "outputs": [], + "source": [ + "dJ_dm = compute_gradient(J, m)" + ] + }, + { + "cell_type": "markdown", + "id": "b7c4c3bc", + "metadata": {}, + "source": [ + "Here each degree of freedom associated with `dJ_dm` contains the derivative of the functional with respect to the corresponding degree of freedom for the control. `dJ_dm` represents a 'dual space' object, defining a linear functional which, given a 'direction' $\\zeta \\in V$, can be used to compute the directional derivative with respect to $m$ with direction $\\zeta$.\n", + "\n", + "For example we can compute the directional derivative of the functional with respect to the control $m$ with direction equal to the unity valued function via:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0772420c", + "metadata": {}, + "outputs": [], + "source": [ + "one = Function(space, name=\"one\")\n", + "one.interpolate(Constant(1.0))\n", + "\n", + "dJ_dm_one = var_inner(one, dJ_dm)\n", + "\n", + "print(f\"{dJ_dm_one=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "1b088023", + "metadata": {}, + "source": [ + "This result is the derivative of the $L^2$-norm of the solution with respect to the amplitude of a spatially constant perturbation to the control $m$. We can compare with the result from finite differencing:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b8e8f6ca", + "metadata": {}, + "outputs": [], + "source": [ + "def dJ_dm(J, m, *, eps=1.0e-7):\n", + " return (J(m + eps) - J(m - eps)) / (2.0 * eps)\n", + "\n", + "\n", + "print(f\"dJ_dm_one approximation = {dJ_dm(lambda eps: float(forward(m + eps * one)), 0.0)}\")" + ] + }, + { + "cell_type": "markdown", + "id": "1819f89e", + "metadata": {}, + "source": [ + "## Computing Hessian information using an adjoint of a tangent-linear\n", + "\n", + "### Single direction\n", + "\n", + "We next seek to compute the action of the Hessian of the functional on some direction $\\zeta \\in V$, using the adjoint method applied to tangent-linear and forward calculations. This can be handled directly, by configuring the relevant tangent-linear and computing the derivative using `compute_gradient`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "14f090f0", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta = var_tlm(J, (m, zeta))\n", + "\n", + "d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)" + ] + }, + { + "cell_type": "markdown", + "id": "5757194e", + "metadata": {}, + "source": [ + "The `Hessian` class applies the same approach, but handles several of the steps for us:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "49588ff6", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "H = Hessian(forward)\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "_, dJ_dm_zeta, d2J_dm_zeta_dm = H.action(m, zeta)" + ] + }, + { + "cell_type": "markdown", + "id": "a78eac8e", + "metadata": {}, + "source": [ + "### Multiple directions\n", + "\n", + "If we want to compute the Hessian action on *multiple* directions we can define multiple tangent-linears:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e23e1725", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta_0 = Function(space, name=\"zeta_0\")\n", + "zeta_0.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta_0))\n", + "\n", + "zeta_1 = Function(space, name=\"zeta_1\")\n", + "zeta_1.interpolate(sin(pi * X[0]) * sin(2.0 * pi * X[1]))\n", + "configure_tlm((m, zeta_1))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta_0 = var_tlm(J, (m, zeta_0))\n", + "dJ_dm_zeta_1 = var_tlm(J, (m, zeta_1))\n", + "\n", + "d2J_dm_zeta_0_dm, d2J_dm_zeta_1_dm = compute_gradient((dJ_dm_zeta_0, dJ_dm_zeta_1), m)" + ] + }, + { + "cell_type": "markdown", + "id": "94aa1773", + "metadata": {}, + "source": [ + "There are now calculations for two sets of tangent-linear variables, two sets of first order adjoint variables, and two sets of second order adjoint variables. However the two sets of first order adjoint variables have the same values – by default tlm_adjoint detects this and only computes them once.\n", + "\n", + "The above approach requires us to know the directions *before* the forward calculation. However some algorithms can generate the directions sequentially, and we do not know the next direction until a Hessian action on the previous direction has been computed. If possible we still want to avoid re-running the forward calculation each time we have a new direction.\n", + "\n", + "If we have sufficient memory available, and in particular so long as we do not need to use checkpointing for the adjoint calculations, we can make use of the `CachedHessian` class. This stores the forward solution and, by default, caches and reuses first order adjoint values. Here we do *not* need to configure the tangent-linear before the forward calculation – instead tlm_adjoint performs the tangent-linear calculations *after* the forward calculations:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9daddace", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "H = CachedHessian(J)\n", + "\n", + "zeta_0 = Function(space, name=\"zeta_0\")\n", + "zeta_0.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "zeta_1 = Function(space, name=\"zeta_1\")\n", + "zeta_1.interpolate(sin(pi * X[0]) * sin(2.0 * pi * X[1]))\n", + "\n", + "_, dJ_dm_zeta_0, d2J_dm_zeta_0_dm = H.action(m, zeta_0)\n", + "_, dJ_dm_zeta_1, d2J_dm_zeta_1_dm = H.action(m, zeta_1)" + ] + }, + { + "cell_type": "markdown", + "id": "dc0f8ae8", + "metadata": {}, + "source": [ + "## Assembly and solver caching\n", + "\n", + "### Using an `EquationSolver`\n", + "\n", + "The calculation for the Hessian action includes four discrete Poisson equations: one for the original forward calculation, one for the tangent-linear calculation, and one each for first and second order adjoint calculations. In this self-adjoint problem the finite element matrix – a stiffness matrix – is the *same* across all four calculations. Hence we can cache and reuse it. Moreover we can cache and reuse linear solver data – for example we can cache and reuse the Cholesky factorization.\n", + "\n", + "tlm_adjoint can apply such caching automatically, but we must interact directly with the object tlm_adjoint uses to record the solution of finite element variational problems – the `EquationSolver` previously seen when we used `manager_info()`. This looks like:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "04d2a68c", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "clear_caches()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " eq = EquationSolver(\n", + " inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " HomogeneousDirichletBC(space, \"on_boundary\"),\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"cholesky\"})\n", + " eq.solve()\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta = var_tlm(J, (m, zeta))\n", + "\n", + "d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)" + ] + }, + { + "cell_type": "markdown", + "id": "35c85f0d", + "metadata": {}, + "source": [ + "The key changes here are:\n", + "\n", + "- The use of `clear_caches`. This ensures that any previously cached data is cleared, avoiding memory leaks if the code is run more than once.\n", + "- The use of `HomogeneousDirichletBC`. This tells tlm_adjoint that the boundary condition is homogeneous, and helps it detect that forward and adjoint problems have the same boundary conditions.\n", + "- The instantiation of an `EquationSolver`, and the call to its `solve` method.\n", + "\n", + "If we query the relevant tlm_adjoint caches we find:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4023b690", + "metadata": {}, + "outputs": [], + "source": [ + "print(f\"{len(assembly_cache())=}\")\n", + "print(f\"{len(linear_solver_cache())=}\")\n", + "\n", + "assert len(assembly_cache()) == 1\n", + "assert len(linear_solver_cache()) == 1" + ] + }, + { + "cell_type": "markdown", + "id": "8c2a42f1", + "metadata": {}, + "source": [ + "and in particular we see that tlm_adjoint has cached data associated with a single matrix, and has cached a single assembled object (which turns out to be the matrix itself). The latter is a stiffness matrix, and the former stores its Cholesky factorization. The factorization is used four times: in the forward, tangent-linear, and first and second order adjoint calculations.\n", + "\n", + "### Flagging data for caching\n", + "\n", + "Now consider the slightly different calculation:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e8fbfb7c", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "clear_caches()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "one = Constant(1.0, name=\"one\")\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " eq = EquationSolver(\n", + " one * inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " HomogeneousDirichletBC(space, \"on_boundary\"),\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"cholesky\"})\n", + " eq.solve()\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta = var_tlm(J, (m, zeta))\n", + "\n", + "d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)\n", + "\n", + "print(f\"{len(assembly_cache())=}\")\n", + "print(f\"{len(linear_solver_cache())=}\")\n", + "\n", + "assert len(assembly_cache()) == 0\n", + "assert len(linear_solver_cache()) == 0" + ] + }, + { + "cell_type": "markdown", + "id": "78f750ac", + "metadata": {}, + "source": [ + "The only difference is the introduction of the multiplication by `one` on the left-hand-side of the finite element variational problem. However we now find that no matrix or linear solver data has been cached. The issue is that tlm_adjoint does not know that it should cache the results of calculations involving `one`.\n", + "\n", + "#### The 'cache' flag\n", + "\n", + "To resolve this, we can flag `one` for caching using `cache=True`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "41227987", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "clear_caches()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "one = Constant(1.0, name=\"one\", cache=True)\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " eq = EquationSolver(\n", + " one * inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " HomogeneousDirichletBC(space, \"on_boundary\"),\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"cholesky\"})\n", + " eq.solve()\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta = var_tlm(J, (m, zeta))\n", + "\n", + "d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)\n", + "\n", + "print(f\"{len(assembly_cache())=}\")\n", + "print(f\"{len(linear_solver_cache())=}\")\n", + "\n", + "assert len(assembly_cache()) == 2\n", + "assert len(linear_solver_cache()) == 1" + ] + }, + { + "cell_type": "markdown", + "id": "38406c1d", + "metadata": {}, + "source": [ + "We now see that tlm_adjoint has cached linear solver data associated with a single matrix. However assembly of two objects has been cached – it turns out there are now *two* cached matrices.\n", + "\n", + "The extra cached matrix appears in the tangent-linear calculations, involving the tangent-linear variable associated with `one` – a tangent-linear right-hand-side term has been converted into a matrix multiply using a *different* matrix. However in the above calculation we know that this tangent-linear variable must be zero, since the calculation for `one` doesn't depend on the control variable. The extra term in the tangent-linear calculation is similarly also known to be zero.\n", + "\n", + "#### The 'static' flag\n", + "\n", + "We can let tlm_adjoint know that `one` does not change, and resolve this inefficiency, by instead using `static=True`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2642124c", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "clear_caches()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "one = Constant(1.0, name=\"one\", static=True)\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " eq = EquationSolver(\n", + " one * inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " HomogeneousDirichletBC(space, \"on_boundary\"),\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"cholesky\"})\n", + " eq.solve()\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta = var_tlm(J, (m, zeta))\n", + "\n", + "d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)\n", + "\n", + "print(f\"{len(assembly_cache())=}\")\n", + "print(f\"{len(linear_solver_cache())=}\")\n", + "\n", + "assert len(assembly_cache()) == 1\n", + "assert len(linear_solver_cache()) == 1" + ] + }, + { + "cell_type": "markdown", + "id": "5c96f6ae", + "metadata": {}, + "source": [ + "Here `static=True` leads to `one` being flagged as a variable whose value is never updated. From this tlm_adjoint can infer that the relevant associated tangent-linear variable is zero, and avoid adding the zero-valued tangent-linear term.\n", + "\n", + "The key difference between using `cache=True` and `static=True` is that in the former the value of the variable *may* be updated. So long as tlm_adjoint is aware of the update (which happens, for example, when tlm_adjoint records a calculation) then updating the value of a variable invalidates cache entries, and invalidated cache entries are cleared automatically." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/_sources/examples/2_verification.ipynb.txt b/_sources/examples/2_verification.ipynb.txt new file mode 100644 index 0000000..33343cf --- /dev/null +++ b/_sources/examples/2_verification.ipynb.txt @@ -0,0 +1,294 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "92f4f706", + "metadata": {}, + "source": [ + "# Verifying derivative calculations\n", + "\n", + "This notebook describes the verification of derivative calculations using Taylor remainder convergence testing. A simple time-independent problem is considered, using tlm_adjoint with the [Firedrake](https://firedrakeproject.org/) backend.\n", + "\n", + "The Taylor remainder convergence testing method is described in:\n", + "\n", + "- P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, 'Automated derivation of the adjoint of high-level transient finite element programs', SIAM Journal on Scientific Computing 35(4), pp. C369–C393, 2013, doi: 10.1137/120873558\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the solution of a linear time-independent partial differential equation, followed by the calculation of the square of the $L^2$-norm of the solution. Non-linearity is introduced by defining the right-hand-side of the problem to be a non-linear function of the control. We assume real spaces and a real build of Firedrake throughout.\n", + "\n", + "Specifically we consider the solution $u \\in V$ of\n", + "\n", + "$$\n", + " \\forall \\zeta \\in V \\qquad \\int_\\Omega u \\zeta + \\alpha^2 \\int_\\Omega \\nabla u \\cdot \\nabla \\zeta = \\int_\\Omega \\left( \\sin^2 m \\right) \\zeta,\n", + "$$\n", + "\n", + "where $V$ is a real $P_1$ continuous finite element space defining functions on the domain $\\Omega = \\left( 0, 1 \\right)^2$, with $m \\in V$. This corresponds to a discretization of the partial differential equation\n", + "\n", + "$$\n", + " u - \\alpha^2 \\nabla^2 u = \\sin^2 m \\quad \\text{on } \\left( x, y \\right) \\in \\left( 0, 1 \\right)^2,\n", + "$$\n", + "\n", + "subject to boundary conditions $\\nabla u \\cdot \\hat{n} = 0$ on the boundary, where $\\hat{n}$ is an outward unit normal.\n", + "\n", + "A simple implementation in Firedrake, with $m = e^x \\sin \\left( \\pi x y \\right)$ and $\\alpha = 0.2$, takes the form:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "13fd669c", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "\n", + "mesh = UnitSquareMesh(50, 50)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(exp(X[0]) * sin(pi * X[0] * X[1]))\n", + "\n", + "alpha = Constant(0.2)\n", + "\n", + "u = Function(space, name=\"u\")\n", + "solve(inner(trial, test) * dx + (alpha ** 2) * inner(grad(trial), grad(test)) * dx\n", + " == inner(sin(m) ** 2, test) * dx, u)\n", + "\n", + "J = assemble(inner(u, u) * dx)" + ] + }, + { + "cell_type": "markdown", + "id": "29810cbf", + "metadata": {}, + "source": [ + "## Taylor remainder convergence testing: First order\n", + "\n", + "If we have a functional $J$ depending on a control $m$ then we have, given some perturbation direction $\\zeta$, via Taylor expansion,\n", + "\n", + "$$\n", + " \\left| J \\left( m + \\varepsilon \\zeta \\right) - J \\left( m \\right) \\right| = O \\left( \\varepsilon \\right), \\\\\n", + "$$\n", + "$$\n", + " \\left| J \\left( m + \\varepsilon \\zeta \\right) - J \\left( m \\right) - \\varepsilon \\frac{dJ}{dm} \\zeta \\right| = O \\left( \\varepsilon^2 \\right).\n", + "$$\n", + "\n", + "That is, $\\zeta$ is some direction in the same space as $m$, which we choose, and then we control the perturbation amplitude using the scalar $\\varepsilon$. The final term in the second absolute value is a directional derivative, which we can compute using the adjoint.\n", + "\n", + "This leads to a methodology for verifying a derivative computed using the adjoint method:\n", + "\n", + "1. Choose a direction $\\zeta$.\n", + "2. Choose a number of different values of $\\varepsilon$.\n", + "3. See if we have second order convergence of the second of the above, to zero.\n", + "\n", + "This verifies only the directional derivative with a single direction, but if we wish we can choose a new direction and repeat the test.\n", + "\n", + "We can use the `taylor_test` function to perform the test for us. By default this generates a pseudorandom direction and chooses a number of values of $\\varepsilon$. It then computes the quantities on the left-hand-sides of the above equations, computes the orders of convergence between consecutive pairs of values for $\\varepsilon$, and displays the results. It returns the *minimum* order computed for the second case, which in a successful verification should be close to two.\n", + "\n", + "Let's compute a derivative using the adjoint method, and apply a Taylor remainder convergence test:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1888394c", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import logging\n", + "import numpy as np\n", + "\n", + "np.random.seed(33582866)\n", + "\n", + "logger = logging.getLogger(\"tlm_adjoint\")\n", + "logger.setLevel(logging.INFO)\n", + "root_logger = logging.getLogger()\n", + "if len(logger.handlers) == 1:\n", + " if len(root_logger.handlers) == 1:\n", + " root_logger.handlers.pop()\n", + " root_logger.addHandler(logger.handlers.pop())\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(50, 50)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(exp(X[0]) * sin(pi * X[0] * X[1]))\n", + "\n", + "alpha = Constant(0.2)\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(trial, test) * dx + (alpha ** 2) * inner(grad(trial), grad(test)) * dx\n", + " == inner(sin(m) ** 2, test) * dx, u)\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u, u) * dx)\n", + " return J\n", + "\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm = compute_gradient(J, m)\n", + "\n", + "min_order = taylor_test(forward, m, J_val=J.value, dJ=dJ_dm, seed=1.0e-3)\n", + "assert min_order > 1.99" + ] + }, + { + "cell_type": "markdown", + "id": "dcdd1937", + "metadata": {}, + "source": [ + "The key changes here are:\n", + "\n", + "- To define a `forward` function. `taylor_test` uses this to repeatedly rerun the forward with different values of $\\varepsilon$.\n", + "- Using `seed` to control the considered values of $\\varepsilon$. If this is too large then the asymptotic orders of convergence may not be seen. If this is too small then the effect of roundoff or iterative solver tolerances may become too large.\n", + "- Seeding the NumPy pseudorandom number generator. The pseudorandom direction is generated using `numpy.random.random`, and we seed the pseudorandom number generator to improve reproducibility. We could alternatively supply a direction using the `dM` argument.\n", + "\n", + "We see the expected first and second orders of convergence.\n", + "\n", + "## Taylor remainder convergence testing: Second order\n", + "\n", + "Including the next order term in the Taylor expansion leads to\n", + "\n", + "$$\n", + " \\left| J \\left( m + \\varepsilon \\zeta \\right) - J \\left( m \\right) - \\varepsilon \\frac{dJ}{dm} \\zeta - \\frac{1}{2} \\varepsilon^2 \\left[ \\frac{d}{dm} \\left( \\frac{dJ}{dm} \\zeta \\right) \\right] \\zeta \\right| = O \\left( \\varepsilon^3 \\right).\n", + "$$\n", + "\n", + "Let's use this approach to test Hessian calculations using a `CachedHessian`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5b6ac48d", + "metadata": {}, + "outputs": [], + "source": [ + "np.random.seed(19986557)\n", + "\n", + "H = CachedHessian(J)\n", + "\n", + "min_order = taylor_test(forward, m, J_val=J.value, ddJ=H, seed=1.0e-3)\n", + "assert min_order > 3.00" + ] + }, + { + "cell_type": "markdown", + "id": "a0435ee4", + "metadata": {}, + "source": [ + "We now see the expected first and *third* orders of convergence, although there is a suggestion of roundoff affecting the results for the smallest $\\varepsilon$. Here the first order directional derivative is computed using a tangent-linear calculation, and the Hessian action on $\\zeta$ is computed by applying the adjoint method to the forward and tangent-linear calculations.\n", + "\n", + "## Taylor remainder convergence testing: Higher order\n", + "\n", + "We can test the derivative of a directional derivative, if we substitute\n", + "\n", + "$$\n", + " J \\rightarrow K = \\frac{dJ}{dm} \\zeta_0,\n", + "$$\n", + "\n", + "with some *new* direction $\\zeta_0$, which we choose. That is, we use\n", + "\n", + "$$\n", + " \\left| K \\left( m + \\varepsilon \\zeta \\right) - K \\left( m \\right) \\right| = O \\left( \\varepsilon \\right), \\\\\n", + "$$\n", + "$$\n", + " \\left| K \\left( m + \\varepsilon \\zeta \\right) - K \\left( m \\right) - \\varepsilon \\frac{dK}{dm} \\zeta \\right| = O \\left( \\varepsilon^2 \\right),\n", + "$$\n", + "\n", + "with\n", + "\n", + "$$\n", + " K = \\frac{dJ}{dm} \\zeta_0.\n", + "$$\n", + "\n", + "The new term\n", + "\n", + "$$\n", + " \\frac{dK}{dm} \\zeta\n", + "$$\n", + "\n", + "can be computed using either a higher order tangent-linear or higher-order adjoint calculation. This generalizes naturally to higher order, by replacing the functional with the directional derivative of a directional derivative.\n", + "\n", + "The function `taylor_test_tlm` performs such verification tests, considering directional derivatives of a given order, and computing all derivatives using tangent-linear calculations. Each directional derivative requires a new direction to be chosen – by default pseudorandom directions are generated.\n", + "\n", + "Let's apply this test up to fourth order:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4e5bf325", + "metadata": {}, + "outputs": [], + "source": [ + "np.random.seed(76149511)\n", + "\n", + "for order in range(1, 5):\n", + " min_order = taylor_test_tlm(forward, m, tlm_order=order, seed=1.0e-3)\n", + " assert min_order > 1.99" + ] + }, + { + "cell_type": "markdown", + "id": "df687375", + "metadata": {}, + "source": [ + "The function `taylor_test_tlm_adjoint` also performs such verification tests, but computes the highest order derivative information using the adjoint method.\n", + "\n", + "Let's apply this test up to fourth order:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9404e167", + "metadata": {}, + "outputs": [], + "source": [ + "np.random.seed(74728054)\n", + "\n", + "for order in range(1, 5):\n", + " min_order = taylor_test_tlm_adjoint(forward, m, adjoint_order=order, seed=1.0e-3)\n", + " assert min_order > 1.99" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/_sources/examples/3_time_dependent.ipynb.txt b/_sources/examples/3_time_dependent.ipynb.txt new file mode 100644 index 0000000..f0de5e9 --- /dev/null +++ b/_sources/examples/3_time_dependent.ipynb.txt @@ -0,0 +1,653 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "92f4f706", + "metadata": {}, + "source": [ + "# Time-dependent example\n", + "\n", + "This notebook describes the calculation of derivative information for a time-dependent problem using tlm_adjoint with the [Firedrake](https://firedrakeproject.org/) backend. Overheads associated with building the records of calculations are discussed, and a checkpointing schedule is applied.\n", + "\n", + "The binomial checkpointing schedule is based on the method described in:\n", + "\n", + "- Andreas Griewank and Andrea Walther, 'Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation', ACM Transactions on Mathematical Software, 26(1), pp. 19–45, 2000, doi: 10.1145/347837.347846\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the solution of a linear time-dependent partial differential equation, followed by the calculation of the square of the $L^2$-norm of the final time solution. We assume real spaces and a real build of Firedrake throughout.\n", + "\n", + "Specifically we consider the advection-diffusion equation in two dimensions, in the form\n", + "\n", + "$$\n", + " \\partial_t u + \\partial_x \\psi \\partial_y u - \\partial_y \\psi \\partial_x u = \\kappa \\left( \\partial_{xx} + \\partial_{yy} \\right) u,\n", + "$$\n", + "\n", + "where $\\psi$ vanishes on the domain boundary, and subject to zero flux boundary conditions. We consider the spatial domain $\\left( x, y \\right) \\in \\left( 0, 1 \\right)^2$ and temporal domain $t \\in \\left[ 0, 0.1 \\right]$, with $\\psi \\left( x, y \\right) = -\\sin \\left( \\pi x \\right) \\sin \\left( \\pi y \\right)$ and $\\kappa = 0.01$, and an initial condition $u \\left( x, y, t=0 \\right) = \\exp \\left[ -50 \\left( \\left( x - 0.75 \\right)^2 + \\left( y - 0.5 \\right)^2 \\right) \\right]$.\n", + "\n", + "The problem is discretized using $P_1$ continuous finite elements to represent both the solution $u$ at each time level and the stream function $\\psi$. The problem is discretized in time using the implicit trapezoidal rule.\n", + "\n", + "A simple implementation in Firedrake takes the form:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b29607a8", + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\")\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01)\n", + "\n", + "u_0 = Function(space, name=\"u_0\")\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "u_n = Function(space, name=\"u_n\")\n", + "u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + "u_h = 0.5 * (u_n + trial)\n", + "F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + "lhs, rhs = system(F)\n", + "\n", + "problem = LinearVariationalProblem(\n", + " lhs, rhs, u_np1,\n", + " constant_jacobian=True)\n", + "solver = LinearVariationalSolver(\n", + " problem, solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + "u_n.assign(u_0)\n", + "for n in range(N):\n", + " solver.solve()\n", + " u_n.assign(u_np1)\n", + "\n", + "J = assemble(inner(u_n, u_n) * dx)\n", + "\n", + "\n", + "def plot_output(u, title):\n", + " r = (u.dat.data_ro.min(), u.dat.data_ro.max())\n", + " eps = (r[1] - r[0]) * 1.0e-12\n", + " p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))\n", + " plt.gca().set_title(title)\n", + " plt.colorbar(p)\n", + " plt.gca().set_aspect(1.0)\n", + "\n", + "\n", + "plot_output(u_0, title=\"$u_0$\")\n", + "plot_output(u_n, title=\"$u_n$\")" + ] + }, + { + "cell_type": "markdown", + "id": "e2b78d66", + "metadata": {}, + "source": [ + "## Adding tlm_adjoint\n", + "\n", + "We first modify the code so that tlm_adjoint processes the calculations:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fd78c6fa", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "reset_manager(\"memory\", {})\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\")\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01)\n", + "\n", + "u_0 = Function(space, name=\"u_0\")\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "\n", + "def forward(u_0, psi):\n", + " u_n = Function(space, name=\"u_n\")\n", + " u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + " u_h = 0.5 * (u_n + trial)\n", + " F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + " lhs, rhs = system(F)\n", + "\n", + " problem = LinearVariationalProblem(\n", + " lhs, rhs, u_np1,\n", + " constant_jacobian=True)\n", + " solver = LinearVariationalSolver(\n", + " problem, solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + " u_n.assign(u_0)\n", + " for n in range(N):\n", + " solver.solve()\n", + " u_n.assign(u_np1)\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u_n, u_n) * dx)\n", + " return J\n", + "\n", + "\n", + "start_manager()\n", + "J = forward(u_0, psi)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "721fd3a1", + "metadata": {}, + "source": [ + "Later we will configure a checkpointing schedule. Resetting the manager resets the record of forward equations but does not reset the checkpointing configuration, and so in this example whenever we reset the manager we also return it to the default checkpointing configuration with `reset_manager(\"memory\", {})`.\n", + "\n", + "## Computing derivatives using an adjoint\n", + "\n", + "The `compute_gradient` function can be used to compute derivatives using the adjoint method. Here we compute the derivative of the square of the $L^2$-norm of the final timestep solution, considered a function of the control defined by the initial condition `u_0` and stream function `psi`, with respect to this control:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0e5b3bba", + "metadata": {}, + "outputs": [], + "source": [ + "dJ_du_0, dJ_dpsi = compute_gradient(J, (u_0, psi))" + ] + }, + { + "cell_type": "markdown", + "id": "98132cf9", + "metadata": {}, + "source": [ + "As a simple check of the result, note that the solution to the (discretized) partial differential equation is unchanged by the addition of a constant to the stream function. Hence we expect the directional derivative with respect to the stream function, with direction equal to the unity valued function, to be zero. This is indeed found to be the case (except for roundoff errors):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b5069278", + "metadata": {}, + "outputs": [], + "source": [ + "one = Function(space, name=\"one\")\n", + "one.interpolate(Constant(1.0))\n", + "\n", + "dJ_dpsi_one = var_inner(one, dJ_dpsi)\n", + "\n", + "print(f\"{dJ_dpsi_one=}\")\n", + "\n", + "assert abs(dJ_dpsi_one) < 1.0e-17" + ] + }, + { + "cell_type": "markdown", + "id": "d0ecd1a6", + "metadata": {}, + "source": [ + "## Computing Hessian information using an adjoint of a tangent-linear\n", + "\n", + "We next compute a Hessian action. Although the following calculation does work, it is inefficient – you may wish to skip forward to the optimized calculations.\n", + "\n", + "Here we compute a 'mixed' Hessian action, by defining a directional derivative with respect to the stream function, and then differentiating this with respect to the initial condition:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "95483f48", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "reset_manager(\"memory\", {})\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\")\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01)\n", + "\n", + "u_0 = Function(space, name=\"u_0\")\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "\n", + "def forward(u_0, psi):\n", + " u_n = Function(space, name=\"u_n\")\n", + " u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + " u_h = 0.5 * (u_n + trial)\n", + " F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + " lhs, rhs = system(F)\n", + "\n", + " problem = LinearVariationalProblem(\n", + " lhs, rhs, u_np1,\n", + " constant_jacobian=True)\n", + " solver = LinearVariationalSolver(\n", + " problem, solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + " u_n.assign(u_0)\n", + " for n in range(N):\n", + " solver.solve()\n", + " u_n.assign(u_np1)\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u_n, u_n) * dx)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.assign(psi)\n", + "configure_tlm((psi, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(u_0, psi)\n", + "stop_manager()\n", + "\n", + "dJ_dpsi_zeta = var_tlm(J, (psi, zeta))\n", + "\n", + "d2J_dpsi_zeta_du_0 = compute_gradient(dJ_dpsi_zeta, u_0)" + ] + }, + { + "cell_type": "markdown", + "id": "ddad9733", + "metadata": {}, + "source": [ + "## Optimization\n", + "\n", + "In the above we have successfully built a record of calculations, and used this to compute derivative information. However there are two issues:\n", + "\n", + "1. Building the record has a noticable cost – the forward calculation has slowed down. In the second order calculation overheads associated with the tangent-linear lead to substantial additional costs.\n", + "2. tlm_adjoint records the solution of the partial differential equation on all time levels. The memory usage here is manageable. However memory limits will be exceeded for larger problems with more fields, spatial degrees of freedom, or timesteps.\n", + "\n", + "Let's fix these issues in order.\n", + "\n", + "### Optimizing the annotation\n", + "\n", + "In the above code tlm_adjoint builds a new record for each finite element variational problem it encounters. Even though only one `LinearVariationalSolver` is instantiated, an `EquationSolver` record is instantiated on each call to the `solve` method. Building the record is sufficiently expensive that the forward calculation noticeably slows down, and this also leads to significant extra processing in the derivative calculations.\n", + "\n", + "Instead we can instantiate an `EquationSolver` directly, and reuse it. However if we do only that then the code will still be inefficient. A single `EquationSolver` will be used, but new linear solver data will be constructed each time its `solve` method is called. We need to also apply an optimization analogous to the `constant_jacobian=True` argument supplied to `LinearVariationalProblem`.\n", + "\n", + "A simple fix is to add `cache_jacobian=True` when instantiating the `EquationSolver`:\n", + "\n", + "```\n", + "eq = EquationSolver(\n", + " lhs == rhs, u_np1,\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"},\n", + " cache_jacobian=True)\n", + "```\n", + "\n", + "This works, but we can instead let tlm_adjoint detect that linear solver data can be cached. We can do that by adding `static=True` when instantiating variables whose value is unchanged throughout the forward calculation:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2856cf07", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "reset_manager(\"memory\", {})\n", + "clear_caches()\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N, static=True)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\", static=True)\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01, static=True)\n", + "\n", + "u_0 = Function(space, name=\"u_0\", static=True)\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "\n", + "def forward(u_0, psi):\n", + " u_n = Function(space, name=\"u_n\")\n", + " u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + " u_h = 0.5 * (u_n + trial)\n", + " F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + " lhs, rhs = system(F)\n", + "\n", + " eq = EquationSolver(\n", + " lhs == rhs, u_np1,\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + " u_n.assign(u_0)\n", + " for n in range(N):\n", + " eq.solve()\n", + " u_n.assign(u_np1)\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u_n, u_n) * dx)\n", + " return J\n", + "\n", + "\n", + "start_manager()\n", + "J = forward(u_0, psi)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "6fe47e03", + "metadata": {}, + "source": [ + "If we now query the relevant tlm_adjoint caches:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eb3638f2", + "metadata": {}, + "outputs": [], + "source": [ + "print(f\"{len(assembly_cache())=}\")\n", + "print(f\"{len(linear_solver_cache())=}\")\n", + "\n", + "assert len(assembly_cache()) == 2\n", + "assert len(linear_solver_cache()) == 1" + ] + }, + { + "cell_type": "markdown", + "id": "fee57222", + "metadata": {}, + "source": [ + "we find that linear solver data associated with a single matrix has been cached. We also find that two assembled objects have been cached – it turns out that there are two cached matrices. As well as caching the matrix associated with the left-hand-side of the discrete problem, a matrix associated with the *right-hand-side* has been assembled and cached. Assembly of the right-hand-side has been converted into a matrix multiply. If we wished we could disable right-hand-side optimizations by adding `cache_rhs_assembly=False`:\n", + "\n", + "```\n", + "eq = EquationSolver(\n", + " lhs == rhs, u_np1,\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"},\n", + " cache_rhs_assembly=False)\n", + "```\n", + "\n", + "### Using a checkpointing schedule\n", + "\n", + "To address the storage issue we enable checkpointing. Here we enable binomial checkpointing with storage of a maximum of $10$ forward restart checkpoints in memory:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "20b7fb3f", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import logging\n", + "\n", + "logger = logging.getLogger(\"tlm_adjoint\")\n", + "logger.setLevel(logging.DEBUG)\n", + "root_logger = logging.getLogger()\n", + "if len(logger.handlers) == 1:\n", + " if len(root_logger.handlers) == 1:\n", + " root_logger.handlers.pop()\n", + " root_logger.addHandler(logger.handlers.pop())\n", + "\n", + "reset_manager(\"memory\", {})\n", + "clear_caches()\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N, static=True)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\", static=True)\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01, static=True)\n", + "\n", + "u_0 = Function(space, name=\"u_0\", static=True)\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "\n", + "def forward(u_0, psi):\n", + " u_n = Function(space, name=\"u_n\")\n", + " u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + " u_h = 0.5 * (u_n + trial)\n", + " F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + " lhs, rhs = system(F)\n", + "\n", + " eq = EquationSolver(\n", + " lhs == rhs, u_np1,\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + " u_n.assign(u_0)\n", + " for n in range(N):\n", + " eq.solve()\n", + " u_n.assign(u_np1)\n", + " if n < N - 1:\n", + " new_block()\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u_n, u_n) * dx)\n", + " return J\n", + "\n", + "\n", + "configure_checkpointing(\"multistage\", {\"snaps_in_ram\": 10, \"blocks\": N})\n", + "start_manager()\n", + "J = forward(u_0, psi)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "c20a3dbb", + "metadata": {}, + "source": [ + "The key changes here are:\n", + "\n", + "- Configuration of a checkpointing schedule using `configure_checkpointing`. Here binomial checkpointing is applied, with a maximum of $10$ forward restart checkpoints stored in memory, indicated using the `\"snaps_in_ram\"` parameter. The total number of steps is indicated using the `\"blocks\"` parameter.\n", + "- The indication of the steps using `new_block()`.\n", + "\n", + "Extra logging output is also enabled so that we can see the details of the checkpointing schedule.\n", + "\n", + "### Computing derivatives\n", + "\n", + "We are now ready to compute derivatives. However a key restriction is that we can, with this checkpointing schedule, only perform the adjoint calculation *once* per forward calculation. We cannot call `compute_gradient` a second time, without first rerunning the entire forward calculation.\n", + "\n", + "In the following we compute both first and second derivative information using a single adjoint calculation:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "33534595", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import logging\n", + "\n", + "logger = logging.getLogger(\"tlm_adjoint\")\n", + "logger.setLevel(logging.DEBUG)\n", + "root_logger = logging.getLogger()\n", + "if len(logger.handlers) == 1:\n", + " if len(root_logger.handlers) == 1:\n", + " root_logger.handlers.pop()\n", + " root_logger.addHandler(logger.handlers.pop())\n", + "\n", + "reset_manager(\"memory\", {})\n", + "clear_caches()\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N, static=True)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\", static=True)\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01, static=True)\n", + "\n", + "u_0 = Function(space, name=\"u_0\", static=True)\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "\n", + "def forward(u_0, psi):\n", + " u_n = Function(space, name=\"u_n\")\n", + " u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + " u_h = 0.5 * (u_n + trial)\n", + " F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + " lhs, rhs = system(F)\n", + "\n", + " eq = EquationSolver(\n", + " lhs == rhs, u_np1,\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + " u_n.assign(u_0)\n", + " for n in range(N):\n", + " eq.solve()\n", + " u_n.assign(u_np1)\n", + " if n < N - 1:\n", + " new_block()\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u_n, u_n) * dx)\n", + " return J\n", + "\n", + "\n", + "zeta_u_0 = ZeroFunction(space, name=\"zeta_u_0\")\n", + "zeta_psi = Function(space, name=\"zeta_psi\", static=True)\n", + "zeta_psi.assign(psi)\n", + "configure_tlm(((u_0, psi), (zeta_u_0, zeta_psi)))\n", + "\n", + "configure_checkpointing(\"multistage\", {\"snaps_in_ram\": 10, \"blocks\": N})\n", + "start_manager()\n", + "J = forward(u_0, psi)\n", + "stop_manager()\n", + "\n", + "dJ_dpsi_zeta = var_tlm(J, ((u_0, psi), (zeta_u_0, zeta_psi)))\n", + "\n", + "dJ_du_0, dJ_dpsi, d2J_dpsi_zeta_du_0 = compute_gradient(\n", + " dJ_dpsi_zeta, (zeta_u_0, zeta_psi, u_0))" + ] + }, + { + "cell_type": "markdown", + "id": "283156e2", + "metadata": {}, + "source": [ + "The derivative calculation now alternates between forward + tangent-linear calculations, and adjoint calculations.\n", + "\n", + "If we wished we could perform higher order adjoint calculations, using a binomial checkpointing schedule, by supplying a higher order tangent-linear configuration and differentiating the result." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/_sources/examples/4_riesz_maps.ipynb.txt b/_sources/examples/4_riesz_maps.ipynb.txt new file mode 100644 index 0000000..5f7746f --- /dev/null +++ b/_sources/examples/4_riesz_maps.ipynb.txt @@ -0,0 +1,211 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "58eda45b-089d-4258-88fa-6ac36c24ed78", + "metadata": {}, + "source": [ + "# Visualizing derivatives\n", + "\n", + "When we differentiate a functional with respect to a finite element discretized function in some discrete function space $V$, we do not obtain a function in $V$, but instead obtain an element of the associated dual space $V^*$. This notebook describes how a Riesz map can be used to construct a function, associated with the derivative, which can be visualized.\n", + "\n", + "This example makes use of tlm_adjoint with the [Firedrake](https://firedrakeproject.org/) backend, and we assume real spaces and a real build of Firedrake throughout.\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the solution $u \\in V$ of a discretization of the Poisson equation subject to homogeneous Dirichlet boundary conditions,\n", + "\n", + "$$\\forall \\zeta \\in V_0 \\qquad \\int_\\Omega \\nabla \\zeta \\cdot \\nabla u = -\\int_\\Omega \\zeta m,$$\n", + "\n", + "where $V$ is a real $P_1$ continuous finite element space defining functions on the domain $\\Omega = \\left( 0, 1 \\right)^2$, with $m \\in V$, and where $V_0$ consists of the functions in $V$ which have zero trace. We define a functional\n", + "\n", + "$$J \\left( u \\right) = \\int_\\Omega \\left( 1 - x \\right)^4 u^2,$$\n", + "\n", + "where $x$ and $y$ denote Cartesian coordinates in $\\mathbb{R}^2$.\n", + "\n", + "We first solve the forward problem for\n", + "\n", + "$$m = \\mathcal{I} \\left[ \\sin \\left( \\pi x \\right) \\sin \\left( 2 \\pi y \\right) \\right],$$\n", + "\n", + "where $\\mathcal{I}$ maps to an element of $V$ through interpolation at mesh vertices." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f977ffe4-03f5-426b-a437-aa012b4a125a", + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test, trial = TestFunction(space), TrialFunction(space)\n", + "m = Function(space, name=\"m\").interpolate(sin(pi * X[0]) * sin(2 * pi * X[1]))\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == -inner(m, test) * dx,\n", + " u, DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(((1.0 - X[0]) ** 4) * u * u * dx)\n", + " return u, J\n", + "\n", + "\n", + "u, J = forward(m)\n", + "\n", + "print(f\"{J.value=}\")\n", + "\n", + "\n", + "def plot_output(u, title):\n", + " r = (u.dat.data_ro.min(), u.dat.data_ro.max())\n", + " eps = (r[1] - r[0]) * 1.0e-12\n", + " p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))\n", + " plt.gca().set_title(title)\n", + " plt.colorbar(p)\n", + " plt.gca().set_aspect(1.0)\n", + "\n", + "\n", + "plot_output(u, title=\"u\")" + ] + }, + { + "cell_type": "markdown", + "id": "be99c89b-29c5-492e-b3bd-6792d3c3f4b2", + "metadata": {}, + "source": [ + "## First order adjoint\n", + "\n", + "We can differentiate a functional respect to $m$ using `compute_gradient`. Specifically we compute the derivative of $\\hat{J} = J \\circ \\hat{u}$, where $\\hat{u}$ is the function which maps from the control $m$ to the solution to the discretized Poisson problem.\n", + "\n", + "When we compute a derivative of a functional with respect to a finite element discretized function using the adjoint method the result is not a function, but is instead a member of a dual space. Here we have $m \\in V$, and the derivative of $\\hat{J}$ with respect to $m$ is a member of the associated dual space $V^*$. In order to visualize this derivative we first need to map it to $V$. We can do this using a Riesz map. This is not uniquely defined, but here we choose to define a Riesz map using the $L^2$ inner product.\n", + "\n", + "Being precise the function we visualize, $g^\\sharp \\in V$, is defined such that\n", + "\n", + "$$\\forall \\zeta \\in V \\qquad \\left. \\frac{d \\hat{J} \\left( m + \\alpha \\zeta \\right)}{d \\alpha} \\right|_{\\alpha = 0} = \\int_\\Omega \\zeta g^\\sharp,$$\n", + "\n", + "where $\\alpha$ is a scalar." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "54f6e366-2cda-41f4-aa9c-d6c37d5bc26a", + "metadata": {}, + "outputs": [], + "source": [ + "reset_manager()\n", + "\n", + "start_manager()\n", + "u, J = forward(m)\n", + "stop_manager()\n", + "\n", + "print(f\"{J.value=}\")\n", + "plot_output(u, title=\"u\")\n", + "\n", + "\n", + "dJdm = compute_gradient(J, m)\n", + "plot_output(dJdm.riesz_representation(\"L2\"), title=r\"$g^\\sharp$\")" + ] + }, + { + "cell_type": "markdown", + "id": "16946cc4-64e1-438a-9f42-6a170decc5dc", + "metadata": {}, + "source": [ + "## Hessian action\n", + "\n", + "Next we seek to differentiate $\\hat{J}$ twice with respect to $m$. However the second derivative defines a *bilinear* operator. This can be represented as a matrix – a Hessian matrix – but the number of elements in this matrix is equal to the *square* of the number of degrees of freedom for $m$.\n", + "\n", + "Instead of computing the full second derivative we can compute its action on a given direction $\\zeta \\in V$. The degrees of freedom associated with the result define the action of the Hessian matrix on a vector – specifically the action of the Hessian matrix on the vector consisting of the degrees of freedom for $\\zeta$.\n", + "\n", + "We do this in two stages. First we compute a directional derivative,\n", + "\n", + "$$\\left. \\frac{d \\hat{J} \\left( m + \\alpha \\zeta \\right)}{d \\alpha} \\right|_{\\alpha=0},$$\n", + "\n", + "computed using the tangent-linear method. We consider the case where $\\zeta = 1$." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8e641946-4aeb-49e0-9415-6b86a822613e", + "metadata": {}, + "outputs": [], + "source": [ + "reset_manager()\n", + "\n", + "zeta = Function(space).interpolate(Constant(1.0))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "u, J = forward(m)\n", + "stop_manager()\n", + "\n", + "print(f\"{J.value=}\")\n", + "plot_output(u, title=\"u\")\n", + "\n", + "dJdm_zeta = var_tlm(J, (m, zeta))\n", + "print(f\"{dJdm_zeta.value=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "74f20f32-c109-4506-8fbf-0db3e4cdffa1", + "metadata": {}, + "source": [ + "Next we compute the derivative of *this* derivative using the adjoint method.\n", + "\n", + "Again we need to remember that the result is a member of the dual space $V^*$, and is not a function, so we again use a Riesz map to visualize it. Here we use the same Riesz map as before, defined using the $L^2$ inner product.\n", + "\n", + "Being precise the function we visualize, $h^\\sharp \\in V$, is defined such that\n", + "\n", + "$$\\forall \\chi \\in V \\qquad \\left. \\frac{\\partial^2 \\hat{J} \\left( m + \\alpha \\zeta + \\beta \\chi \\right)}{\\partial \\beta \\partial \\alpha} \\right|_{\\alpha = 0, \\beta = 0} = \\int_\\Omega \\chi h^\\sharp,$$\n", + "\n", + "where $\\zeta \\in V$ defines the direction on which the action is computed, and $\\alpha$ and $\\beta$ are scalars." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6107d209-7731-42c3-8d84-f61b92d30232", + "metadata": {}, + "outputs": [], + "source": [ + "d2Jdm2_zeta = compute_gradient(dJdm_zeta, m)\n", + "plot_output(d2Jdm2_zeta.riesz_representation(\"L2\"), title=r\"$h^\\sharp$\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/_sources/examples/5_optimization.ipynb.txt b/_sources/examples/5_optimization.ipynb.txt new file mode 100644 index 0000000..8ae0fe2 --- /dev/null +++ b/_sources/examples/5_optimization.ipynb.txt @@ -0,0 +1,231 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "258cd78c-55e3-40e6-97cb-057064281a61", + "metadata": {}, + "source": [ + "# Functional minimization\n", + "\n", + "Gradient-based optimization algorithms attempt to use derivative information, for example as computed using the adjoint method, to accelerate the solution of optimization problems. A typical example is to seek the minimum of an objective cost functional, constrained using one or more partial differential equations. This notebook describes how to solve such partial differential equation constrained optimization problems using tlm_adjoint.\n", + "\n", + "This example makes use of tlm_adjoint with the [Firedrake](https://firedrakeproject.org/) backend, and we assume real spaces and a real build of Firedrake throughout. The minimization problem is solved using the [Toolkit for Advanced Optimization (TAO)](https://petsc.org/release/manual/tao/).\n", + "\n", + "## Forward problem\n", + "\n", + "We first construct a partial differential equation constrained optimization problem where we know the answer – where we know the value of the control which minimizes the objective functional subject to the constraint that the forward problem is solved. To do this we consider the solution of the modified Helmholtz equation in the unit square domain,\n", + "\n", + "$$\\alpha^2 \\nabla^2 u - u = m \\qquad \\text{on} ~ \\Omega = \\left( 0, 1 \\right)^2,$$\n", + "\n", + "where $\\alpha \\in \\mathbb{R}$, subject to doubly periodic boundary conditions. We define an objective cost functional\n", + "\n", + "$$J \\left( u, m \\right) = \\frac{1}{2} \\int_\\Omega \\left( u - \\tilde{u} \\right)^2 + \\frac{1}{2} \\beta^2 \\int_\\Omega \\left( m - \\tilde{m} \\right)^2,$$\n", + "\n", + "where $\\tilde{u}$ and $\\tilde{m}$ are given functions and $\\beta \\ne 0$ is a real scalar. If we set\n", + "\n", + "$$\\tilde{m} \\left( x, y \\right) = -\\sin \\left( 2 \\pi x \\right) \\sin \\left( 2 \\pi y \\right),$$\n", + "$$\\tilde{u} \\left( x, y \\right) = -\\frac{1}{1 + 8 \\pi^2 \\alpha^2} m \\left( x, y \\right),$$\n", + "\n", + "where $x$ and $y$ are Cartesian coordinates in $\\mathbb{R}^2$, then $u = \\tilde{u}$ and $m = \\tilde{m}$ will be the minimum of $J$ where the modified Helmholtz problem is solved.\n", + "\n", + "We consider a continuous Galerkin finite element discretization, seeking $u \\in V$ such that\n", + "\n", + "$$\\forall \\zeta \\in V \\qquad \\alpha^2 \\int_\\Omega \\nabla \\zeta \\cdot \\nabla u + \\int_\\Omega \\zeta u = -\\int_\\Omega \\zeta m,$$\n", + "\n", + "where $V$ is a real continuous $P_1$ finite element space whose elements satisfy the doubly periodic boundary conditions. We now define $\\tilde{m} \\in V$ and $\\tilde{u} \\in V$ via interpolation, at mesh vertices, of the functions given above.\n", + "\n", + "We first use Firedrake to solve the forward problem. We consider $\\alpha = 0.1$, $\\beta = 0.1$, and an 'initial guess' where $m = -1$." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f41c40c7-d0c3-438e-b874-e35cfeee7d40", + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "alpha = Constant(0.1)\n", + "beta = Constant(0.1)\n", + "\n", + "\n", + "def forward(m):\n", + " space = m.function_space()\n", + " X = SpatialCoordinate(space.mesh())\n", + " test, trial = TestFunction(space), TrialFunction(space)\n", + "\n", + " m_tilde = Function(space, name=\"m_tilde\").interpolate(\n", + " -sin(2 * pi * X[0]) * sin(2 * pi * X[1]))\n", + " u_tilde = Function(space, name=\"u_tilde\").assign(\n", + " -(1.0 / (1.0 + 8.0 * pi * pi * alpha * alpha)) * m_tilde)\n", + "\n", + " u = Function(space, name=\"u\")\n", + " solve(alpha * alpha * inner(grad(trial), grad(test)) * dx + inner(trial, test) * dx\n", + " == -inner(m, test) * dx,\n", + " u, solver_parameters={\"ksp_type\": \"cg\",\n", + " \"pc_type\": \"sor\",\n", + " \"ksp_atol\": 1.0e-32,\n", + " \"ksp_rtol\": 1.0e-12})\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.addto(0.5 * inner(u - u_tilde, u - u_tilde) * dx)\n", + " J.addto(0.5 * beta * beta * inner(m - m_tilde, m - m_tilde) * dx)\n", + " return m_tilde, u, J\n", + "\n", + "\n", + "mesh = PeriodicSquareMesh(20, 20, 1.0)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "\n", + "m_0 = Function(space, name=\"m_0\").interpolate(Constant(-1.0))\n", + "m_tilde, u, J = forward(m_0)" + ] + }, + { + "cell_type": "markdown", + "id": "84b81c75-ceba-4e7e-a5ed-c3cbb521617c", + "metadata": {}, + "source": [ + "## Inverse problem\n", + "\n", + "We now seek to solve the inverse problem: to find the $m \\in V$ which minimizes $J$, subject to the discretized modified Helmholtz problem being solved.\n", + "\n", + "In the following we use the Toolkit for Advanced Optimization (TAO) to solve the inverse problem. We use the Limited-Memory Variable Metric (LMVM) approach with an absolute tolerance of $10^{-10}$. Noting that the adjoint computed derivative is an element of the dual space $V^*$, we need to define an appropriate dual space inner product. Here we define an inner product using the inverse mass matrix." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d7d2768b-5ff1-4f58-9370-1833e01cb612", + "metadata": {}, + "outputs": [], + "source": [ + "def forward_J(m):\n", + " _, _, J = forward(m)\n", + " return J\n", + "\n", + "\n", + "M_solver = LinearSolver(assemble(inner(TrialFunction(space), TestFunction(space)) * dx),\n", + " solver_parameters={\"ksp_type\": \"cg\",\n", + " \"pc_type\": \"sor\",\n", + " \"ksp_atol\": 1.0e-32,\n", + " \"ksp_rtol\": 1.0e-12})\n", + "\n", + "\n", + "def M_inv_action(x):\n", + " y = Function(space)\n", + " M_solver.solve(y, x.copy(deepcopy=True))\n", + " return y\n", + "\n", + "\n", + "def post_callback(tao):\n", + " print(f\"{tao.getIterationNumber()=}\")\n", + "\n", + "\n", + "m = minimize_tao(forward_J, m_0, gatol=1.0e-10, grtol=0.0,\n", + " M_inv_action=M_inv_action, post_callback=post_callback)\n", + "m.rename(name=\"m\")\n", + "m_tilde, u, J = forward(m)\n", + "\n", + "\n", + "def plot_output(u, title):\n", + " r = (u.dat.data_ro.min(), u.dat.data_ro.max())\n", + " eps = (r[1] - r[0]) * 1.0e-12\n", + " p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))\n", + " plt.gca().set_title(title)\n", + " plt.colorbar(p)\n", + " plt.gca().set_aspect(1.0)\n", + "\n", + "\n", + "plot_output(u, title=\"u\")\n", + "plot_output(m, title=\"m\")" + ] + }, + { + "cell_type": "markdown", + "id": "d2bf05e9-0e6e-4a4c-8b93-9e6019f689ee", + "metadata": {}, + "source": [ + "We now test the inverse procedure by checking that it converges to the expected result, considering meshes with decreasing element size. We compute the $L^2$ error in each case, and estimate the order of convergence by a power law fit between the error norms computed using subsequent pairs of meshes." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ebfbaff7-a877-4e9b-bac7-b7bdad621e5c", + "metadata": {}, + "outputs": [], + "source": [ + "Ns = np.array([20 * (2 ** p) for p in range(4)], dtype=int)\n", + "error_norms = []\n", + "for N in Ns:\n", + " mesh = PeriodicSquareMesh(N, N, 1.0)\n", + " X = SpatialCoordinate(mesh)\n", + " space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "\n", + " m_0 = Function(space, name=\"m_0\").interpolate(Constant(-1.0))\n", + "\n", + " M_solver = LinearSolver(assemble(inner(TrialFunction(space), TestFunction(space)) * dx),\n", + " solver_parameters={\"ksp_type\": \"cg\",\n", + " \"pc_type\": \"sor\",\n", + " \"ksp_atol\": 1.0e-32,\n", + " \"ksp_rtol\": 1.0e-12})\n", + "\n", + " def M_inv_action(x):\n", + " y = Function(space)\n", + " M_solver.solve(y, x.copy(deepcopy=True))\n", + " return y\n", + "\n", + " m = minimize_tao(forward_J, m_0, gatol=1.0e-10, grtol=0.0,\n", + " M_inv_action=M_inv_action, post_callback=post_callback)\n", + " m_tilde, u, J = forward(m)\n", + "\n", + " m_error_norm = sqrt(abs(assemble(inner(m - m_tilde, m - m_tilde) * dx)))\n", + " print(f\"{N=} {m_error_norm=}\")\n", + " error_norms.append(m_error_norm)\n", + "error_norms = np.array(error_norms, dtype=float)\n", + "\n", + "orders = -np.log(error_norms[1:] / error_norms[:-1]) / np.log(Ns[1:] / Ns[:-1])\n", + "print(f\"{orders=}\")\n", + "\n", + "assert (orders > 1.99).all()" + ] + }, + { + "cell_type": "markdown", + "id": "3a913c18-c76b-4119-aab5-db6176ddac04", + "metadata": {}, + "source": [ + "We find that we have close to second order convergence." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/_sources/examples/6_custom_operations.ipynb.txt b/_sources/examples/6_custom_operations.ipynb.txt new file mode 100644 index 0000000..8083989 --- /dev/null +++ b/_sources/examples/6_custom_operations.ipynb.txt @@ -0,0 +1,628 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "82be80f5-acad-4947-8f9a-c6a7a87a7ace", + "metadata": {}, + "source": [ + "# Defining custom operations\n", + "\n", + "This notebook describes the tlm_adjoint 'escape hatch' mechanism, which allows custom operations to be defined, recorded by the internal manager, and used in higher order derivative calculations.\n", + "\n", + "The approach used by tlm_adjoint to define custom operations is described in\n", + "\n", + "- James R. Maddison, Daniel N. Goldberg, and Benjamin D. Goddard, 'Automated calculation of higher order partial differential equation constrained derivative information', SIAM Journal on Scientific Computing, 41(5), pp. C417–C445, 2019, doi: 10.1137/18M1209465\n", + "\n", + "We assume real spaces and a real build of Firedrake throughout.\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the Poisson equation in the unit square domain\n", + "\n", + "$$\\nabla^2 u = m \\quad \\text{on} ~ \\Omega = \\left( 0, 1 \\right)^2,$$\n", + "\n", + "with\n", + "\n", + "$$\\int_\\Omega m = 0,$$\n", + "\n", + "subject to boundary conditions\n", + "\n", + "$$\\nabla u \\cdot \\hat{n} = 0,$$\n", + "\n", + "where $\\hat{n}$ is an outward unit normal on the boundary $\\partial \\Omega$ of the domain $\\Omega$.\n", + "\n", + "We consider a continuous Galerkin discretization, now seeking $u \\in V$ such that\n", + "\n", + "$$\\forall \\zeta \\in V \\qquad \\int_\\Omega \\nabla \\zeta \\cdot \\nabla u = -\\int_\\Omega \\zeta m,$$\n", + "\n", + "where $V$ is a real $P_1$ continuous finite element space defining functions on the domain $\\Omega = \\left( 0, 1 \\right)^2$. $m \\in V$ is now defined via\n", + "\n", + "$$m = \\mathcal{I} \\left[ \\cos \\left( \\pi x \\right) \\cos \\left( \\pi y \\right) \\right] - k,$$\n", + "\n", + "where $\\mathcal{I}$ maps to an element of $V$ through interpolation at mesh vertices, and where $k$ is defined so that $\\int_\\Omega m = 0$.\n", + "\n", + "As stated the discrete problem does not have a unique solution – given any solution $u_0$ to the problem we can define a new solution $u_1 = u_0 + c_1$ for any scalar $c_1$. We need to identify a solution by supplying an appropriate constraint. We will use the constraint\n", + "\n", + "$$\\sum_i \\tilde{u}_i = 0,$$\n", + "\n", + "where the $\\tilde{u}_i$ are the degrees of freedom associated with $u \\in V$.\n", + "\n", + "An implementation using Firedrake, solving for $u$ and then computing $\\frac{1}{2} \\int_\\Omega u^2$, takes the form" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "17707b1c-9237-46a3-8c70-f1e700c8ad00", + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test, trial = TestFunction(space), TrialFunction(space)\n", + "\n", + "\n", + "def forward(m):\n", + " k = Constant(assemble(m * dx) / assemble(Constant(1.0) * dx(mesh)))\n", + " m_tilde = Function(m.function_space(), name=\"m_tilde\").interpolate(m - k)\n", + " \n", + " u = Function(space, name=\"u\")\n", + " nullspace = VectorSpaceBasis(comm=u.comm, constant=True)\n", + "\n", + " solve(inner(grad(trial), grad(test)) * dx == -inner(m_tilde, test) * dx,\n", + " u, nullspace=nullspace, transpose_nullspace=nullspace,\n", + " solver_parameters={\"ksp_type\": \"cg\", \"pc_type\": \"hypre\",\n", + " \"pc_hypre_type\": \"boomeramg\",\n", + " \"ksp_rtol\": 1.0e-10, \"ksp_atol\": 1.0e-16})\n", + "\n", + " J = assemble(0.5 * inner(u, u) * dx)\n", + " return u, J\n", + "\n", + "\n", + "m = Function(space, name=\"m\").interpolate(cos(pi * X[0]) * cos(pi * X[1]))\n", + "\n", + "u, J = forward(m)\n", + "\n", + "\n", + "def plot_output(u, title):\n", + " r = (u.dat.data_ro.min(), u.dat.data_ro.max())\n", + " eps = (r[1] - r[0]) * 1.0e-12\n", + " p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))\n", + " plt.gca().set_title(title)\n", + " plt.colorbar(p)\n", + " plt.gca().set_aspect(1.0)\n", + "\n", + "\n", + "plot_output(u, title=\"u\")" + ] + }, + { + "cell_type": "markdown", + "id": "cc5e0b3d-426c-4b75-8b74-d46037c5e822", + "metadata": {}, + "source": [ + "## Adding tlm_adjoint\n", + "\n", + "Next we add the use of tlm_adjoint, which requires some minor modification of the forward code.\n", + "\n", + "`compute_gradient` is used to compute the derivative of $\\frac{1}{2} \\int_\\Omega u^2$ with respect to $m$, subject to the contraint that the forward problem is solved. The derivative is visualized using a Riesz map defined by the $L^2$ inner product. A Taylor remainder convergence test is used to verify the derivative.\n", + "\n", + "Note that, given an arbitrary $m \\in V$, we define\n", + "\n", + "$$\\tilde{m} = m - \\frac{\\int_\\Omega m}{\\int_\\Omega 1},$$\n", + "\n", + "so that $\\int_\\Omega \\tilde{m} = 0$, and use $\\tilde{m}$ in place of $m$ when solving the discrete Poisson problem." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fa48c271-64ab-4104-8d8d-2ebc46adb7f0", + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "np.random.seed(54151610)\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test, trial = TestFunction(space), TrialFunction(space)\n", + "\n", + "\n", + "def forward(m):\n", + " v = Constant(name=\"v\")\n", + " Assembly(v, Constant(1.0) * dx(mesh)).solve()\n", + " k = Constant(name=\"k\")\n", + " Assembly(k, (m / v) * dx).solve()\n", + " m_tilde = Function(m.function_space(), name=\"m_tilde\").interpolate(m - k)\n", + " \n", + " u = Function(space, name=\"u\")\n", + " nullspace = VectorSpaceBasis(comm=u.comm, constant=True)\n", + "\n", + " solve(inner(grad(trial), grad(test)) * dx == -inner(m_tilde, test) * dx,\n", + " u, nullspace=nullspace, transpose_nullspace=nullspace,\n", + " solver_parameters={\"ksp_type\": \"cg\", \"pc_type\": \"hypre\",\n", + " \"pc_hypre_type\": \"boomeramg\",\n", + " \"ksp_rtol\": 1.0e-10, \"ksp_atol\": 1.0e-16})\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(0.5 * inner(u, u) * dx)\n", + " return u, J\n", + "\n", + "\n", + "m = Function(space, name=\"m\").interpolate(cos(pi * X[0]) * cos(pi * X[1]))\n", + "\n", + "start_manager()\n", + "u, J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ = compute_gradient(J, m)\n", + "\n", + "\n", + "def plot_output(u, title):\n", + " r = (u.dat.data_ro.min(), u.dat.data_ro.max())\n", + " eps = (r[1] - r[0]) * 1.0e-12\n", + " p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))\n", + " plt.gca().set_title(title)\n", + " plt.colorbar(p)\n", + " plt.gca().set_aspect(1.0)\n", + "\n", + "\n", + "plot_output(u, title=\"u\")\n", + "plot_output(dJ.riesz_representation(\"L2\"), title=r\"$g^\\sharp$\")\n", + "\n", + "\n", + "def forward_J(m):\n", + " _, J = forward(m)\n", + " return J\n", + "\n", + "\n", + "min_order = taylor_test(forward_J, m, J_val=J.value, dJ=dJ)\n", + "assert min_order > 1.99" + ] + }, + { + "cell_type": "markdown", + "id": "23af6882-f085-4fa5-ae75-3ff04b367e3c", + "metadata": {}, + "source": [ + "## Defining a custom `Equation`\n", + "\n", + "Since the $u$ we have computed should satisfy\n", + "\n", + "$$\\sum_i \\tilde{u}_i = 0,$$\n", + "\n", + "we should have that the derivative of this quantity with respect to $m$, subject to the forward problem being solved, is zero. We will compute this derivative by defining a custom operation which sums the degrees of freedom of a finite element discretized function. We can do this by inheriting from the `Equation` class and implementing appropriate methods." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1d3bdc61-be2a-4e94-82f7-d8f6d5faa92d", + "metadata": {}, + "outputs": [], + "source": [ + "from tlm_adjoint import Equation\n", + "\n", + "\n", + "class Sum(Equation):\n", + " def __init__(self, x, y):\n", + " super().__init__(x, deps=(x, y), nl_deps=(), ic=False, adj_ic=False,\n", + " adj_type=\"conjugate_dual\")\n", + "\n", + " def forward_solve(self, x, deps=None):\n", + " if deps is None:\n", + " deps = self.dependencies()\n", + " _, y = deps\n", + " with y.dat.vec_ro as y_v:\n", + " y_sum = y_v.sum()\n", + " x.assign(y_sum)\n", + "\n", + " def adjoint_jacobian_solve(self, adj_x, nl_deps, b):\n", + " return b\n", + "\n", + " def adjoint_derivative_action(self, nl_deps, dep_index, adj_x):\n", + " if dep_index == 0:\n", + " return adj_x\n", + " elif dep_index == 1:\n", + " _, y = self.dependencies()\n", + " b = Cofunction(var_space(y).dual())\n", + " adj_x = float(adj_x)\n", + " b.dat.data[:] = -adj_x\n", + " return b\n", + " else:\n", + " raise ValueError(\"dep_index out of range\")\n", + "\n", + " def tangent_linear(self, M, dM, tlm_map):\n", + " tau_x, tau_y = (tlm_map[dep] for dep in self.dependencies())\n", + " if tau_y is None:\n", + " return ZeroAssignment(tau_x)\n", + " else:\n", + " return Sum(tau_x, tau_y)" + ] + }, + { + "cell_type": "markdown", + "id": "854d0055-7138-4728-a775-38cc8fecabf5", + "metadata": {}, + "source": [ + "We'll investgate how this is put together shortly.\n", + "\n", + "First, we check that the `Sum` class works. In the following we check that we can compute the sum of the degrees of freedom of a function, and then use Taylor remainder convergence tests to verify a first order tangent-linear calculation, a first order adjoint calculation, and a reverse-over-forward adjoint calculation of a Hessian action." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0ed98c56-fa77-499c-8243-8f83727c2aaf", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "np.random.seed(13561700)\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(m):\n", + " m_sum = Float(name=\"m_sum\")\n", + " Sum(m_sum, m).solve()\n", + "\n", + " J = m_sum ** 3\n", + " \n", + " return m_sum, J\n", + "\n", + "\n", + "def forward_J(m):\n", + " _, J = forward(m)\n", + " return J\n", + "\n", + "\n", + "mesh = UnitIntervalMesh(10)\n", + "x, = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Discontinuous Lagrange\", 0)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(exp(-x) * sin(pi * x))\n", + "with m.dat.vec_ro as m_v:\n", + " m_sum_ref = m_v.sum()\n", + "\n", + "start_manager()\n", + "m_sum, J = forward(m)\n", + "stop_manager()\n", + "\n", + "# Verify the forward calculation\n", + "\n", + "print(f\"{float(m_sum)=}\")\n", + "print(f\"{m_sum_ref=}\")\n", + "assert abs(float(m_sum) - m_sum_ref) == 0.0\n", + "\n", + "# Verify tangent-linear and adjoint calculations\n", + "\n", + "min_order = taylor_test_tlm(forward_J, m, tlm_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=2)\n", + "assert min_order > 1.99" + ] + }, + { + "cell_type": "markdown", + "id": "f6236cab-0eb6-4fad-ab87-e71e5ab071a2", + "metadata": {}, + "source": [ + "We can now use `Sum` with the Poisson solver. We find, as expected, that the derivative of the sum of the degrees of freedom for $u$ with respect to $m$, subject to the forward problem being solved, is zero." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3fa84206-fdc1-4672-83df-190fc974fa43", + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test, trial = TestFunction(space), TrialFunction(space)\n", + "\n", + "\n", + "def forward(m):\n", + " v = Constant(name=\"v\")\n", + " Assembly(v, Constant(1.0) * dx(mesh)).solve()\n", + " k = Constant(name=\"k\")\n", + " Assembly(k, (m / v) * dx).solve()\n", + " m_tilde = Function(m.function_space(), name=\"m_tilde\").interpolate(m - k)\n", + " \n", + " u = Function(space, name=\"u\")\n", + " nullspace = VectorSpaceBasis(comm=u.comm, constant=True)\n", + "\n", + " solve(inner(grad(trial), grad(test)) * dx == -inner(m_tilde, test) * dx,\n", + " u, nullspace=nullspace, transpose_nullspace=nullspace,\n", + " solver_parameters={\"ksp_type\": \"cg\", \"pc_type\": \"hypre\",\n", + " \"pc_hypre_type\": \"boomeramg\",\n", + " \"ksp_rtol\": 1.0e-10, \"ksp_atol\": 1.0e-16})\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(0.5 * inner(u, u) * dx)\n", + " \n", + " u_sum = Constant(name=\"u_sum\")\n", + " Sum(u_sum, u).solve()\n", + "\n", + " K = Functional(name=\"K\")\n", + " K.assign(u_sum)\n", + "\n", + " return u, J, K\n", + "\n", + "\n", + "m = Function(space, name=\"m\").interpolate(cos(pi * X[0]) * cos(pi * X[1]))\n", + "\n", + "start_manager()\n", + "u, J, K = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ, dK = compute_gradient((J, K), m)\n", + "\n", + "dK_norm = abs(dK.dat.data_ro).max()\n", + "print(f\"{dK_norm=}\")\n", + "\n", + "assert dK_norm == 0.0" + ] + }, + { + "cell_type": "markdown", + "id": "d7929c30-35c5-4ac7-bf43-3761685fd03b", + "metadata": {}, + "source": [ + "## `Equation` methods\n", + "\n", + "We now return to the definition of the `Sum` class, and consider each method in turn.\n", + "\n", + "### `__init__`\n", + "\n", + "```\n", + "def __init__(self, x, y):\n", + " super().__init__(x, deps=(x, y), nl_deps=(), ic=False, adj_ic=False,\n", + " adj_type=\"conjugate_dual\")\n", + "```\n", + "\n", + "The arguments passed to the base class constructor are:\n", + "\n", + "- The output of the forward operation, `x`.\n", + "- `deps`: Defines all inputs and outputs of the forward operation.\n", + "- `nl_deps`: Defines elements of `deps` on which the associated adjoint calculations can depend.\n", + "- `ic`: Whether the forward operation accepts a non-zero 'initial guess'.\n", + "- `adj_ic`: Whether an adjoint solve accepts a non-zero 'initial guess'.\n", + "- `adj_type`: Either `\"primal\"` or `\"conjugate_dual\"` defining the space for an associated adjoint variable.\n", + "\n", + "For this operation there is one output `x` and one input `y`. The operation is defined by the forward residual function\n", + "\n", + "$$F \\left( x, y \\right) = x - \\sum_i \\tilde{u}_i.$$\n", + "\n", + "Given a value for $y$ the output of the operation is defined to be the $x$ for which $F \\left( x, y \\right) = 0$. $F$ depends linearly on both $x$ and $y$, and so associated adjoint calculations are independent of both $x$ and $y$. Hence we set `deps=(x, y)` and `nl_deps=()`.\n", + "\n", + "`ic` and `adj_ic` are used to indicate whether non-zero initial guesses should be supplied to forward and adjoint iterative solvers respectively. Here we do not use iterative solvers, and so no non-zero initial guesses are needed. Hence we set `ic=False` and `adj_ic=False`.\n", + "\n", + "`adj_type` indicates whether an adjoint variable associated with the operation is:\n", + "\n", + "- In the same space as `x`. This is the case where $F$ maps to an element in the antidual space associated with $x$, and is indicated with `adj_x_type=\"primal\"`. A typical example of this case is an operation defined via the solution of a finite element variational problem.\n", + "- In the antidual space associated with the space for `x`. This is the case where $F$ maps to an element in same space as $x$, and is indiciated with `adj_x_type=\"conjugate_dual\"`.\n", + "\n", + "### `forward_solve`\n", + "\n", + "```\n", + "def forward_solve(self, x, deps=None):\n", + " if deps is None:\n", + " deps = self.dependencies()\n", + " _, y = deps\n", + " with y.dat.vec_ro as y_v:\n", + " y_sum = y_v.sum()\n", + " x.assign(y_sum)\n", + "```\n", + "\n", + "This method computes the output of the forward operation, storing the result in `x`. `deps` is used in rerunning the forward calculation when making use of a checkpointing schedule. If supplied, `deps` contains values associated with the dependencies as defined in the constructor (defined by the `deps` argument passed to the base class constructor). Otherwise the values contained in `self.dependencies()` are used.\n", + "\n", + "### `adjoint_jacobian_solve`\n", + "\n", + "```\n", + "def adjoint_jacobian_solve(self, adj_x, nl_deps, b):\n", + " return b\n", + "```\n", + "\n", + "Solves a linear problem\n", + "\n", + "$$\\frac{\\partial F}{\\partial x}^T \\lambda = b,$$\n", + "\n", + "for the adjoint solution $\\lambda$, returning the result. The right-hand-side $b$ is defined by the argument `b`. In this example $\\partial F / \\partial x$ is simply the identity, and so $\\lambda = b$.\n", + "\n", + "`adj_x` can contain an initial guess for an iterative solver used to compute $\\lambda$. `nl_deps` contains values associated with the forward dependencies of the adjoint, as defined in the constructor (defined by the `nl_deps` argument passed to the base class constructor).\n", + "\n", + "### `adjoint_derivative_action`\n", + "\n", + "```\n", + "def adjoint_derivative_action(self, nl_deps, dep_index, adj_x):\n", + " if dep_index == 0:\n", + " return adj_x\n", + " elif dep_index == 1:\n", + " _, y = self.dependencies()\n", + " b = Cofunction(var_space(y).dual())\n", + " adj_x = float(adj_x)\n", + " b.dat.data[:] = -adj_x\n", + " return b\n", + " else:\n", + " raise ValueError(\"dep_index out of range\")\n", + "```\n", + "\n", + "Computes\n", + "\n", + "$$\\frac{\\partial F}{\\partial v}^T \\lambda,$$\n", + "\n", + "where $v$ is the `dep_index`th dependency (for the dependencies defined by the `deps` argument passed to the base class constructor). Again `nl_deps` contains values associated with the forward dependencies of the adjoint, as defined in the constructor.\n", + "\n", + "### `tangent_linear`\n", + "\n", + "```\n", + "def tangent_linear(self, M, dM, tlm_map):\n", + " tau_x, tau_y = (tlm_map[dep] for dep in self.dependencies())\n", + " if tau_y is None:\n", + " return ZeroAssignment(tau_x)\n", + " else:\n", + " return Sum(tau_x, tau_y)\n", + "```\n", + "\n", + "Constructs a tangent-linear operation, for a tangent-linear computing directional derivatives with respect to the control defined by `M` with direction defined by `dM`. `tlm_map` is used to access tangent-linear variables.\n", + "\n", + "The new operation is defined by the residual function\n", + "\n", + "$$F_\\tau \\left( \\tau_x, \\tau_y \\right) = \\frac{\\partial F}{\\partial x} \\tau_x + \\frac{\\partial F}{\\partial y} \\tau_y,$$\n", + "\n", + "where $\\tau_x$ and $\\tau_y$ are the tangent-linear variables associated with $x$ and $y$ respectively. Given a value for $\\tau_y$ the output of the operation is defined to be the $\\tau_x$ for which $F_\\tau \\left( \\tau_x, \\tau_y \\right) = 0$.\n", + "\n", + "Note that this method returns an `Equation` instance – here either a `ZeroAssignment`, for the case where the tangent-linear operation sets a tangent-linear variable equal to zero, or another `Sum`. This new operation can then be recorded by the internal manager, so that reverse-over-forward algorithmic differentiation can be applied.\n", + "\n", + "### Reference dropping\n", + "\n", + "An important method, not defined in this example, is the `drop_references` method. This method is used to drop references to forward data. For example if an `Equation` subclass defines a `Function` attribute `_function`, then the `drop_references` method might look like\n", + "\n", + "```\n", + "def drop_references(self):\n", + " super().drop_references()\n", + " self._function = var_replacement(self._function)\n", + "```\n", + "\n", + "Here `var_replacement` returns a 'symbolic only' version of `self._function`, which can for example be used in UFL expressions, but which has no associated value.\n", + "\n", + "## Defining custom operations using JAX\n", + "\n", + "In some cases it may be more convenient to directly implement an operation using lower level code. tlm_adjoint integrates with [JAX](https://jax.readthedocs.io) to allow this. For example a Firedrake `Function` can be converted to a JAX array using\n", + "\n", + "```\n", + "v = to_jax(x)\n", + "```\n", + "\n", + "and the result converted back to a Firedrake `Function` using\n", + "\n", + "```\n", + "x = to_firedrake(v, space)\n", + "```\n", + "\n", + "Here we use JAX to compute the sum of the degrees of freedom for a Firedrake function (assuming a serial calculation)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c98042cd-c9b5-4ac8-abd5-8fbb1a5717ff", + "metadata": {}, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "np.random.seed(81976463)\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(m):\n", + " m_sum = new_jax_float(name=\"m_sum\")\n", + " call_jax(m_sum, to_jax(m), lambda v: v.sum())\n", + "\n", + " J = m_sum ** 3\n", + " \n", + " return m_sum, J\n", + "\n", + "\n", + "def forward_J(m):\n", + " _, J = forward(m)\n", + " return J\n", + "\n", + "\n", + "mesh = UnitIntervalMesh(10)\n", + "x, = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Discontinuous Lagrange\", 0)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(exp(-x) * sin(pi * x))\n", + "with m.dat.vec_ro as m_v:\n", + " m_sum_ref = m_v.sum()\n", + "\n", + "start_manager()\n", + "m_sum, J = forward(m)\n", + "stop_manager()\n", + "\n", + "# Verify the forward calculation\n", + "\n", + "print(f\"{float(m_sum)=}\")\n", + "print(f\"{m_sum_ref=}\")\n", + "assert abs(float(m_sum) - m_sum_ref) < 1.0e-15\n", + "\n", + "# Verify tangent-linear and adjoint calculations\n", + "\n", + "min_order = taylor_test_tlm(forward_J, m, tlm_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=2)\n", + "assert min_order > 1.99" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/_sources/examples/7_jax_integration.ipynb.txt b/_sources/examples/7_jax_integration.ipynb.txt new file mode 100644 index 0000000..31c2d69 --- /dev/null +++ b/_sources/examples/7_jax_integration.ipynb.txt @@ -0,0 +1,228 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "ea6ba877-3da2-4a5b-b878-8350e1c74b08", + "metadata": {}, + "source": [ + "# JAX integration\n", + "\n", + "This notebook introduces the use of [JAX](https://jax.readthedocs.io) with tlm_adjoint.\n", + "\n", + "JAX can be used e.g. with the Firedrake backend as an escape-hatch, allowing custom operations to be implemented using lower level code. However it can also be used independently, without a finite element code generator. Here JAX is used with tlm_adjoint to implement and differentiate a finite difference code.\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the diffusion equation in the unit square domain, subject to homogeneous Dirichlet boundary conditions,\n", + "\n", + "$$\\partial_t u = \\kappa \\left( \\partial_{xx} + \\partial_{yy} \\right) u + m \\left( x, y \\right) \\qquad \\text{on} ~ \\left( x, y \\right) \\in \\left( 0, 1 \\right)^2,$$\n", + "$$u = 0 \\qquad \\text{on} ~ \\partial \\Omega.$$\n", + "\n", + "This is discretized using a basic finite difference scheme, using second order centered finite differencing for the $x$ and $y$ dimensions and forward Euler for the $t$ dimension.\n", + "\n", + "Here we implement the discretization using JAX, for $\\kappa = 0.01$, $t \\in \\left[ 0, 0.25 \\right]$, a time step size of $0.0025$, and a uniform grid with a grid spacing of $0.02$. We consider the case where $m \\left( x, y \\right) = 0$, and compute the $L^4$ norm of the $t = T$ solution (with the $L^4$ norm defined via nearest neighbour interpolation of the finite difference solution, ignoring boundary points where the solution is zero)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4d51cefb-f6e5-4178-bf9e-07469c6410ec", + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import jax\n", + "\n", + "jax.config.update(\"jax_enable_x64\", True)\n", + "\n", + "N = 50\n", + "kappa = 0.01\n", + "T = 0.25\n", + "N_t = 100\n", + "\n", + "dx = 1.0 / N\n", + "dt = T / N_t\n", + " \n", + " \n", + "x = jax.numpy.linspace(0.0, 1.0, N + 1)\n", + "y = jax.numpy.linspace(0.0, 1.0, N + 1)\n", + "X, Y = jax.numpy.meshgrid(x, y, indexing=\"ij\")\n", + "\n", + "\n", + "def timestep(x_n, m):\n", + " x_np1 = jax.numpy.zeros_like(x_n)\n", + " x_np1 = x_np1.at[1:-1, 1:-1].set(\n", + " x_n[1:-1, 1:-1]\n", + " + (kappa * dt / (dx * dx)) * (x_n[1:-1, 2:]\n", + " + x_n[:-2, 1:-1]\n", + " - 4.0 * x_n[1:-1, 1:-1]\n", + " + x_n[2:, 1:-1]\n", + " + x_n[1:-1, :-2])\n", + " + dt * m[1:-1, 1:-1])\n", + " return x_np1\n", + "\n", + "\n", + "def functional(x_n):\n", + " x_n = x_n.reshape((N + 1, N + 1))\n", + " return (dx * dx * (x_n[1:-1, 1:-1] ** 4).sum()) ** 0.25\n", + "\n", + "\n", + "def forward(x_0, m):\n", + " x_n = x_0.copy()\n", + " for _ in range(N_t):\n", + " x_n = timestep(x_n, m)\n", + " J = functional(x_n)\n", + " return x_n, J\n", + "\n", + "\n", + "x_0 = jax.numpy.zeros((N + 1, N + 1), dtype=jax.numpy.double)\n", + "x_0 = x_0.at[1:-1, 1:-1].set(jax.numpy.exp(-((X[1:-1, 1:-1] - 0.5) ** 2 + (Y[1:-1, 1:-1]- 0.5) ** 2) / (2.0 * (0.05 ** 2))))\n", + "m = jax.numpy.zeros((N + 1, N + 1), dtype=jax.numpy.double)\n", + "\n", + "fig, ax = plt.subplots(1, 1)\n", + "p = ax.contourf(x, y, x_0.T, 32)\n", + "fig.colorbar(p)\n", + "ax.set_aspect(1.0)\n", + "ax.set_title(\"$x_0$\")\n", + "\n", + "x_n, J = forward(x_0, m)\n", + "print(f\"{J=}\")\n", + "\n", + "fig, ax = plt.subplots(1, 1)\n", + "p = ax.contourf(x, y, x_n.T, 32)\n", + "fig.colorbar(p)\n", + "ax.set_aspect(1.0)\n", + "ax.set_title(\"$x_{N_t}$\")" + ] + }, + { + "cell_type": "markdown", + "id": "3ff06fb2-5469-492f-95ce-7ce741375e30", + "metadata": {}, + "source": [ + "## Adding tlm_adjoint\n", + "\n", + "The tlm_adjoint `Vector` class wraps ndim 1 JAX arrays. The following uses the tlm_adjoint `call_jax` function to record JAX operations on the internal tlm_adjoint manager. Note the use of `new_block`, indicating steps, and enabling use of a step-based checkpointing schedule." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "55f03da3", + "metadata": {}, + "outputs": [], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import jax\n", + "\n", + "reset_manager()\n", + "\n", + "N = 50\n", + "kappa = 0.01\n", + "T = 0.25\n", + "N_t = 100\n", + "\n", + "dx = 1.0 / N\n", + "dt = T / N_t\n", + " \n", + " \n", + "x = jax.numpy.linspace(0.0, 1.0, N + 1)\n", + "y = jax.numpy.linspace(0.0, 1.0, N + 1)\n", + "X, Y = jax.numpy.meshgrid(x, y, indexing=\"ij\")\n", + "\n", + "\n", + "def timestep(x_n, m):\n", + " x_n = x_n.reshape((N + 1, N + 1))\n", + " m = m.reshape((N + 1, N + 1))\n", + " x_np1 = jax.numpy.zeros_like(x_n)\n", + " x_np1 = x_np1.at[1:-1, 1:-1].set(\n", + " x_n[1:-1, 1:-1]\n", + " + (kappa * dt / (dx * dx)) * (x_n[1:-1, 2:]\n", + " + x_n[:-2, 1:-1]\n", + " - 4.0 * x_n[1:-1, 1:-1]\n", + " + x_n[2:, 1:-1]\n", + " + x_n[1:-1, :-2])\n", + " + dt * m[1:-1, 1:-1])\n", + " return x_np1.flatten()\n", + "\n", + "\n", + "def functional(x_n):\n", + " x_n = x_n.reshape((N + 1, N + 1))\n", + " return (dx * dx * (x_n[1:-1, 1:-1] ** 4).sum()) ** 0.25\n", + "\n", + "\n", + "def forward(x_0, m):\n", + " x_n = Vector((N + 1) ** 2)\n", + " x_np1 = Vector((N + 1) ** 2)\n", + " x_n.assign(x_0)\n", + " for n_t in range(N_t):\n", + " call_jax(x_np1, (x_n, m), timestep)\n", + " x_n.assign(x_np1)\n", + " if n_t < N_t - 1:\n", + " new_block()\n", + " J = new_jax_float()\n", + " call_jax(J, x_n, functional)\n", + " return J\n", + "\n", + "\n", + "x_0 = jax.numpy.zeros((N + 1, N + 1), dtype=jax.numpy.double)\n", + "x_0 = x_0.at[1:-1, 1:-1].set(jax.numpy.exp(-((X[1:-1, 1:-1] - 0.5) ** 2 + (Y[1:-1, 1:-1]- 0.5) ** 2) / (2.0 * (0.05 ** 2))))\n", + "x_0 = Vector(x_0.flatten())\n", + "m = Vector((N + 1) ** 2)\n", + "\n", + "start_manager()\n", + "J = forward(x_0, m)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "5df893e2-47b5-4ba1-81c8-04542c344448", + "metadata": {}, + "source": [ + "We can now verify first order tangent-linear and adjoint calculations, and a second order reverse-over-forward calculation, using Taylor remainder convergence tests. Here we consider derivatives with respect to $m$." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "33eb96c6-8934-4c2c-b356-7b29962842e6", + "metadata": {}, + "outputs": [], + "source": [ + "min_order = taylor_test_tlm(lambda m: forward(x_0, m), m, tlm_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(lambda m: forward(x_0, m), m, adjoint_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(lambda m: forward(x_0, m), m, adjoint_order=2)\n", + "assert min_order > 2.00" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/_sources/index.rst.txt b/_sources/index.rst.txt new file mode 100644 index 0000000..74ec247 --- /dev/null +++ b/_sources/index.rst.txt @@ -0,0 +1,80 @@ +tlm_adjoint +=========== + +tlm_adjoint is a high-level algorithmic differentiation tool, principally for +use with `Firedrake `_. + +The primary aim of tlm_adjoint is to enable higher order adjoint calculations +-- and in particular to compute Hessian information -- while also using adjoint +checkpointing schedules, and allowing for caching of assembled finite element +data, and caching of linear solver data. + +Features +-------- + + - Integrates with the Firedrake automated code generation library, and + applies a high-level algorithmic differentiation approach to enable + tangent-linear and adjoint calculations. + - Applies a *reverse-over-multiple-forward* approach for higher order + adjoint calculations. For example a Hessian action on some given + direction is computed by deriving an adjoint associated with a combined + forward/tangent-linear calculation. + - Implements adjoint checkpointing schedules to enable large problems to + be tackled. + - Can apply automated finite element assembly and linear solver caching. + Cached data can be reused across the forward, tangent-linear, and adjoint + calculations. + - Provides a simple 'escape-hatch' to enable parts of the problem, not + otherwise supported by tlm_adjoint or the backend library, to be added, + with adjoint or tangent-linear information defined manually. + +Examples +-------- + +The following Jupyter notebooks introduce derivative calculations using +tlm_adjoint. + +- `Getting started with tlm_adjoint `__: + Introduces derivative calculations using tlm_adjoint. +- `Time-independent example `__: An example + including the solution of a time-independent partial differential equation + using Firedrake. +- `Verifying derivative calculations `__: + Introduces Taylor remainder convergence testing. +- `Time-dependent example `__: An example + including the solution of a time-dependent partial differential equation + using Firedrake. Introduces checkpointing. +- `Visualizing derivatives `__: The use of a Riesz + map to visualize the result of differentiating with respect to a finite + element discretized function using the adjoint method. +- `Functional minimization `__: The solution of + partial differential equation constrained optimization problems. +- `Defining custom operations `__: An + example using the tlm_adjoint 'escape-hatch' to define a custom operation. +- `JAX integration `__: A finite difference + example using JAX. + +Source and license +------------------ + +The source code is available from the +`tlm_adjoint GitHub repository `_. +tlm_adjoint is licensed under the GNU LGPL version 3. + +Indices +------- + +* :ref:`genindex` +* :ref:`modindex` + +.. toctree:: + :hidden: + + self + +.. toctree:: + :hidden: + :maxdepth: 0 + + dependencies + acknowledgements diff --git a/_static/_sphinx_javascript_frameworks_compat.js b/_static/_sphinx_javascript_frameworks_compat.js new file mode 100644 index 0000000..8141580 --- /dev/null +++ b/_static/_sphinx_javascript_frameworks_compat.js @@ -0,0 +1,123 @@ +/* Compatability shim for jQuery and underscores.js. + * + * Copyright Sphinx contributors + * Released under the two clause BSD licence + */ + +/** + * small helper function to urldecode strings + * + * See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/decodeURIComponent#Decoding_query_parameters_from_a_URL + */ +jQuery.urldecode = function(x) { + if (!x) { + return x + } + return decodeURIComponent(x.replace(/\+/g, ' ')); +}; + +/** + * small helper function to urlencode strings + */ +jQuery.urlencode = encodeURIComponent; + +/** + * This function returns the parsed url parameters of the + * current request. Multiple values per key are supported, + * it will always return arrays of strings for the value parts. + */ +jQuery.getQueryParameters = function(s) { + if (typeof s === 'undefined') + s = document.location.search; + var parts = s.substr(s.indexOf('?') + 1).split('&'); + var result = {}; + for (var i = 0; i < parts.length; i++) { + var tmp = parts[i].split('=', 2); + var key = jQuery.urldecode(tmp[0]); + var value = jQuery.urldecode(tmp[1]); + if (key in result) + result[key].push(value); + else + result[key] = [value]; + } + return result; +}; + +/** + * highlight a given string on a jquery object by wrapping it in + * span elements with the given class name. + */ +jQuery.fn.highlightText = function(text, className) { + function highlight(node, addItems) { + if (node.nodeType === 3) { + var val = node.nodeValue; + var pos = val.toLowerCase().indexOf(text); + if (pos >= 0 && + !jQuery(node.parentNode).hasClass(className) && + !jQuery(node.parentNode).hasClass("nohighlight")) { + var span; + var isInSVG = jQuery(node).closest("body, svg, foreignObject").is("svg"); + if (isInSVG) { + span = document.createElementNS("http://www.w3.org/2000/svg", "tspan"); + } else { + span = document.createElement("span"); + span.className = className; + } + span.appendChild(document.createTextNode(val.substr(pos, text.length))); + node.parentNode.insertBefore(span, node.parentNode.insertBefore( + document.createTextNode(val.substr(pos + text.length)), + node.nextSibling)); + node.nodeValue = val.substr(0, pos); + if (isInSVG) { + var rect = document.createElementNS("http://www.w3.org/2000/svg", "rect"); + var bbox = node.parentElement.getBBox(); + rect.x.baseVal.value = bbox.x; + rect.y.baseVal.value = bbox.y; + rect.width.baseVal.value = bbox.width; + rect.height.baseVal.value = bbox.height; + rect.setAttribute('class', className); + addItems.push({ + "parent": node.parentNode, + "target": rect}); + } + } + } + else if (!jQuery(node).is("button, select, textarea")) { + jQuery.each(node.childNodes, function() { + highlight(this, addItems); + }); + } + } + var addItems = []; + var result = this.each(function() { + highlight(this, addItems); + }); + for (var i = 0; i < addItems.length; ++i) { + jQuery(addItems[i].parent).before(addItems[i].target); + } + return result; +}; + +/* + * backward compatibility for jQuery.browser + * This will be supported until firefox bug is fixed. + */ +if (!jQuery.browser) { + jQuery.uaMatch = function(ua) { + ua = ua.toLowerCase(); + + var match = /(chrome)[ \/]([\w.]+)/.exec(ua) || + /(webkit)[ \/]([\w.]+)/.exec(ua) || + /(opera)(?:.*version|)[ \/]([\w.]+)/.exec(ua) || + /(msie) ([\w.]+)/.exec(ua) || + ua.indexOf("compatible") < 0 && /(mozilla)(?:.*? rv:([\w.]+)|)/.exec(ua) || + []; + + return { + browser: match[ 1 ] || "", + version: match[ 2 ] || "0" + }; + }; + jQuery.browser = {}; + jQuery.browser[jQuery.uaMatch(navigator.userAgent).browser] = true; +} diff --git a/_static/basic.css b/_static/basic.css new file mode 100644 index 0000000..30fee9d --- /dev/null +++ b/_static/basic.css @@ -0,0 +1,925 @@ +/* + * basic.css + * ~~~~~~~~~ + * + * Sphinx stylesheet -- basic theme. + * + * :copyright: Copyright 2007-2023 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ + +/* -- main layout ----------------------------------------------------------- */ + +div.clearer { + clear: both; +} + +div.section::after { + display: block; + content: ''; + clear: left; +} + +/* -- relbar ---------------------------------------------------------------- */ + +div.related { + width: 100%; + font-size: 90%; +} + +div.related h3 { + display: none; +} + +div.related ul { + margin: 0; + padding: 0 0 0 10px; + list-style: none; +} + +div.related li { + display: inline; +} + +div.related li.right { + float: right; + margin-right: 5px; +} + +/* -- sidebar --------------------------------------------------------------- */ + +div.sphinxsidebarwrapper { + padding: 10px 5px 0 10px; +} + +div.sphinxsidebar { + float: left; + width: 230px; + margin-left: -100%; + font-size: 90%; + word-wrap: break-word; + overflow-wrap : break-word; +} + +div.sphinxsidebar ul { + list-style: none; +} + +div.sphinxsidebar ul ul, +div.sphinxsidebar ul.want-points { + margin-left: 20px; + list-style: square; +} + +div.sphinxsidebar ul ul { + margin-top: 0; + margin-bottom: 0; +} + +div.sphinxsidebar form { + margin-top: 10px; +} + +div.sphinxsidebar input { + border: 1px solid #98dbcc; + font-family: sans-serif; + font-size: 1em; +} + +div.sphinxsidebar #searchbox form.search { + overflow: hidden; +} + +div.sphinxsidebar #searchbox input[type="text"] { + float: left; + width: 80%; + padding: 0.25em; + box-sizing: border-box; +} + +div.sphinxsidebar #searchbox input[type="submit"] { + float: left; + width: 20%; + border-left: none; + padding: 0.25em; + box-sizing: border-box; +} + + +img { + border: 0; + max-width: 100%; +} + +/* -- search page ----------------------------------------------------------- */ + +ul.search { + margin: 10px 0 0 20px; + padding: 0; +} + +ul.search li { + padding: 5px 0 5px 20px; + background-image: url(file.png); + background-repeat: no-repeat; + background-position: 0 7px; +} + +ul.search li a { + font-weight: bold; +} + +ul.search li p.context { + color: #888; + margin: 2px 0 0 30px; + text-align: left; +} + +ul.keywordmatches li.goodmatch a { + font-weight: bold; +} + +/* -- index page ------------------------------------------------------------ */ + +table.contentstable { + width: 90%; + margin-left: auto; + margin-right: auto; +} + +table.contentstable p.biglink { + line-height: 150%; +} + +a.biglink { + font-size: 1.3em; +} + +span.linkdescr { + font-style: italic; + padding-top: 5px; + font-size: 90%; +} + +/* -- general index --------------------------------------------------------- */ + +table.indextable { + width: 100%; +} + +table.indextable td { + text-align: left; + vertical-align: top; +} + +table.indextable ul { + margin-top: 0; + margin-bottom: 0; + list-style-type: none; +} + +table.indextable > tbody > tr > td > ul { + padding-left: 0em; +} + +table.indextable tr.pcap { + height: 10px; +} + +table.indextable tr.cap { + margin-top: 10px; + background-color: #f2f2f2; +} + +img.toggler { + margin-right: 3px; + margin-top: 3px; + cursor: pointer; +} + +div.modindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +div.genindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +/* -- domain module index --------------------------------------------------- */ + +table.modindextable td { + padding: 2px; + border-collapse: collapse; +} + +/* -- general body styles --------------------------------------------------- */ + +div.body { + min-width: 360px; + max-width: 800px; +} + +div.body p, div.body dd, div.body li, div.body blockquote { + -moz-hyphens: auto; + -ms-hyphens: auto; + -webkit-hyphens: auto; + hyphens: auto; +} + +a.headerlink { + visibility: hidden; +} + +a:visited { + color: #551A8B; +} + +h1:hover > a.headerlink, +h2:hover > a.headerlink, +h3:hover > a.headerlink, +h4:hover > a.headerlink, +h5:hover > a.headerlink, +h6:hover > a.headerlink, +dt:hover > a.headerlink, +caption:hover > a.headerlink, +p.caption:hover > a.headerlink, +div.code-block-caption:hover > a.headerlink { + visibility: visible; +} + +div.body p.caption { + text-align: inherit; +} + +div.body td { + text-align: left; +} + +.first { + margin-top: 0 !important; +} + +p.rubric { + margin-top: 30px; + font-weight: bold; +} + +img.align-left, figure.align-left, .figure.align-left, object.align-left { + clear: left; + float: left; + margin-right: 1em; +} + +img.align-right, figure.align-right, .figure.align-right, object.align-right { + clear: right; + float: right; + margin-left: 1em; +} + +img.align-center, figure.align-center, .figure.align-center, object.align-center { + display: block; + margin-left: auto; + margin-right: auto; +} + +img.align-default, figure.align-default, .figure.align-default { + display: block; + margin-left: auto; + margin-right: auto; +} + +.align-left { + text-align: left; +} + +.align-center { + text-align: center; +} + +.align-default { + text-align: center; +} + +.align-right { + text-align: right; +} + +/* -- sidebars -------------------------------------------------------------- */ + +div.sidebar, +aside.sidebar { + margin: 0 0 0.5em 1em; + border: 1px solid #ddb; + padding: 7px; + background-color: #ffe; + width: 40%; + float: right; + clear: right; + overflow-x: auto; +} + +p.sidebar-title { + font-weight: bold; +} + +nav.contents, +aside.topic, +div.admonition, div.topic, blockquote { + clear: left; +} + +/* -- topics ---------------------------------------------------------------- */ + +nav.contents, +aside.topic, +div.topic { + border: 1px solid #ccc; + padding: 7px; + margin: 10px 0 10px 0; +} + +p.topic-title { + font-size: 1.1em; + font-weight: bold; + margin-top: 10px; +} + +/* -- admonitions ----------------------------------------------------------- */ + +div.admonition { + margin-top: 10px; + margin-bottom: 10px; + padding: 7px; +} + +div.admonition dt { + font-weight: bold; +} + +p.admonition-title { + margin: 0px 10px 5px 0px; + font-weight: bold; +} + +div.body p.centered { + text-align: center; + margin-top: 25px; +} + +/* -- content of sidebars/topics/admonitions -------------------------------- */ + +div.sidebar > :last-child, +aside.sidebar > :last-child, +nav.contents > :last-child, +aside.topic > :last-child, +div.topic > :last-child, +div.admonition > :last-child { + margin-bottom: 0; +} + +div.sidebar::after, +aside.sidebar::after, +nav.contents::after, +aside.topic::after, +div.topic::after, +div.admonition::after, +blockquote::after { + display: block; + content: ''; + clear: both; +} + +/* -- tables ---------------------------------------------------------------- */ + +table.docutils { + margin-top: 10px; + margin-bottom: 10px; + border: 0; + border-collapse: collapse; +} + +table.align-center { + margin-left: auto; + margin-right: auto; +} + +table.align-default { + margin-left: auto; + margin-right: auto; +} + +table caption span.caption-number { + font-style: italic; +} + +table caption span.caption-text { +} + +table.docutils td, table.docutils th { + padding: 1px 8px 1px 5px; + border-top: 0; + border-left: 0; + border-right: 0; + border-bottom: 1px solid #aaa; +} + +th { + text-align: left; + padding-right: 5px; +} + +table.citation { + border-left: solid 1px gray; + margin-left: 1px; +} + +table.citation td { + border-bottom: none; +} + +th > :first-child, +td > :first-child { + margin-top: 0px; +} + +th > :last-child, +td > :last-child { + margin-bottom: 0px; +} + +/* -- figures --------------------------------------------------------------- */ + +div.figure, figure { + margin: 0.5em; + padding: 0.5em; +} + +div.figure p.caption, figcaption { + padding: 0.3em; +} + +div.figure p.caption span.caption-number, +figcaption span.caption-number { + font-style: italic; +} + +div.figure p.caption span.caption-text, +figcaption span.caption-text { +} + +/* -- field list styles ----------------------------------------------------- */ + +table.field-list td, table.field-list th { + border: 0 !important; +} + +.field-list ul { + margin: 0; + padding-left: 1em; +} + +.field-list p { + margin: 0; +} + +.field-name { + -moz-hyphens: manual; + -ms-hyphens: manual; + -webkit-hyphens: manual; + hyphens: manual; +} + +/* -- hlist styles ---------------------------------------------------------- */ + +table.hlist { + margin: 1em 0; +} + +table.hlist td { + vertical-align: top; +} + +/* -- object description styles --------------------------------------------- */ + +.sig { + font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace; +} + +.sig-name, code.descname { + background-color: transparent; + font-weight: bold; +} + +.sig-name { + font-size: 1.1em; +} + +code.descname { + font-size: 1.2em; +} + +.sig-prename, code.descclassname { + background-color: transparent; +} + +.optional { + font-size: 1.3em; +} + +.sig-paren { + font-size: larger; +} + +.sig-param.n { + font-style: italic; +} + +/* C++ specific styling */ + +.sig-inline.c-texpr, +.sig-inline.cpp-texpr { + font-family: unset; +} + +.sig.c .k, .sig.c .kt, +.sig.cpp .k, .sig.cpp .kt { + color: #0033B3; +} + +.sig.c .m, +.sig.cpp .m { + color: #1750EB; +} + +.sig.c .s, .sig.c .sc, +.sig.cpp .s, .sig.cpp .sc { + color: #067D17; +} + + +/* -- other body styles ----------------------------------------------------- */ + +ol.arabic { + list-style: decimal; +} + +ol.loweralpha { + list-style: lower-alpha; +} + +ol.upperalpha { + list-style: upper-alpha; +} + +ol.lowerroman { + list-style: lower-roman; +} + +ol.upperroman { + list-style: upper-roman; +} + +:not(li) > ol > li:first-child > :first-child, +:not(li) > ul > li:first-child > :first-child { + margin-top: 0px; +} + +:not(li) > ol > li:last-child > :last-child, +:not(li) > ul > li:last-child > :last-child { + margin-bottom: 0px; +} + +ol.simple ol p, +ol.simple ul p, +ul.simple ol p, +ul.simple ul p { + margin-top: 0; +} + +ol.simple > li:not(:first-child) > p, +ul.simple > li:not(:first-child) > p { + margin-top: 0; +} + +ol.simple p, +ul.simple p { + margin-bottom: 0; +} + +aside.footnote > span, +div.citation > span { + float: left; +} +aside.footnote > span:last-of-type, +div.citation > span:last-of-type { + padding-right: 0.5em; +} +aside.footnote > p { + margin-left: 2em; +} +div.citation > p { + margin-left: 4em; +} +aside.footnote > p:last-of-type, +div.citation > p:last-of-type { + margin-bottom: 0em; +} +aside.footnote > p:last-of-type:after, +div.citation > p:last-of-type:after { + content: ""; + clear: both; +} + +dl.field-list { + display: grid; + grid-template-columns: fit-content(30%) auto; +} + +dl.field-list > dt { + font-weight: bold; + word-break: break-word; + padding-left: 0.5em; + padding-right: 5px; +} + +dl.field-list > dd { + padding-left: 0.5em; + margin-top: 0em; + margin-left: 0em; + margin-bottom: 0em; +} + +dl { + margin-bottom: 15px; +} + +dd > :first-child { + margin-top: 0px; +} + +dd ul, dd table { + margin-bottom: 10px; +} + +dd { + margin-top: 3px; + margin-bottom: 10px; + margin-left: 30px; +} + +.sig dd { + margin-top: 0px; + margin-bottom: 0px; +} + +.sig dl { + margin-top: 0px; + margin-bottom: 0px; +} + +dl > dd:last-child, +dl > dd:last-child > :last-child { + margin-bottom: 0; +} + +dt:target, span.highlighted { + background-color: #fbe54e; +} + +rect.highlighted { + fill: #fbe54e; +} + +dl.glossary dt { + font-weight: bold; + font-size: 1.1em; +} + +.versionmodified { + font-style: italic; +} + +.system-message { + background-color: #fda; + padding: 5px; + border: 3px solid red; +} + +.footnote:target { + background-color: #ffa; +} + +.line-block { + display: block; + margin-top: 1em; + margin-bottom: 1em; +} + +.line-block .line-block { + margin-top: 0; + margin-bottom: 0; + margin-left: 1.5em; +} + +.guilabel, .menuselection { + font-family: sans-serif; +} + +.accelerator { + text-decoration: underline; +} + +.classifier { + font-style: oblique; +} + +.classifier:before { + font-style: normal; + margin: 0 0.5em; + content: ":"; + display: inline-block; +} + +abbr, acronym { + border-bottom: dotted 1px; + cursor: help; +} + +.translated { + background-color: rgba(207, 255, 207, 0.2) +} + +.untranslated { + background-color: rgba(255, 207, 207, 0.2) +} + +/* -- code displays --------------------------------------------------------- */ + +pre { + overflow: auto; + overflow-y: hidden; /* fixes display issues on Chrome browsers */ +} + +pre, div[class*="highlight-"] { + clear: both; +} + +span.pre { + -moz-hyphens: none; + -ms-hyphens: none; + -webkit-hyphens: none; + hyphens: none; + white-space: nowrap; +} + +div[class*="highlight-"] { + margin: 1em 0; +} + +td.linenos pre { + border: 0; + background-color: transparent; + color: #aaa; +} + +table.highlighttable { + display: block; +} + +table.highlighttable tbody { + display: block; +} + +table.highlighttable tr { + display: flex; +} + +table.highlighttable td { + margin: 0; + padding: 0; +} + +table.highlighttable td.linenos { + padding-right: 0.5em; +} + +table.highlighttable td.code { + flex: 1; + overflow: hidden; +} + +.highlight .hll { + display: block; +} + +div.highlight pre, +table.highlighttable pre { + margin: 0; +} + +div.code-block-caption + div { + margin-top: 0; +} + +div.code-block-caption { + margin-top: 1em; + padding: 2px 5px; + font-size: small; +} + +div.code-block-caption code { + background-color: transparent; +} + +table.highlighttable td.linenos, +span.linenos, +div.highlight span.gp { /* gp: Generic.Prompt */ + user-select: none; + -webkit-user-select: text; /* Safari fallback only */ + -webkit-user-select: none; /* Chrome/Safari */ + -moz-user-select: none; /* Firefox */ + -ms-user-select: none; /* IE10+ */ +} + +div.code-block-caption span.caption-number { + padding: 0.1em 0.3em; + font-style: italic; +} + +div.code-block-caption span.caption-text { +} + +div.literal-block-wrapper { + margin: 1em 0; +} + +code.xref, a code { + background-color: transparent; + font-weight: bold; +} + +h1 code, h2 code, h3 code, h4 code, h5 code, h6 code { + background-color: transparent; +} + +.viewcode-link { + float: right; +} + +.viewcode-back { + float: right; + font-family: sans-serif; +} + +div.viewcode-block:target { + margin: -1px -10px; + padding: 0 10px; +} + +/* -- math display ---------------------------------------------------------- */ + +img.math { + vertical-align: middle; +} + +div.body div.math p { + text-align: center; +} + +span.eqno { + float: right; +} + +span.eqno a.headerlink { + position: absolute; + z-index: 1; +} + +div.math:hover a.headerlink { + visibility: visible; +} + +/* -- printout stylesheet --------------------------------------------------- */ + +@media print { + div.document, + div.documentwrapper, + div.bodywrapper { + margin: 0 !important; + width: 100%; + } + + div.sphinxsidebar, + div.related, + div.footer, + #top-link { + display: none; + } +} \ No newline at end of file diff --git a/_static/css/badge_only.css b/_static/css/badge_only.css new file mode 100644 index 0000000..c718cee --- /dev/null +++ b/_static/css/badge_only.css @@ -0,0 +1 @@ +.clearfix{*zoom:1}.clearfix:after,.clearfix:before{display:table;content:""}.clearfix:after{clear:both}@font-face{font-family:FontAwesome;font-style:normal;font-weight:400;src:url(fonts/fontawesome-webfont.eot?674f50d287a8c48dc19ba404d20fe713?#iefix) format("embedded-opentype"),url(fonts/fontawesome-webfont.woff2?af7ae505a9eed503f8b8e6982036873e) format("woff2"),url(fonts/fontawesome-webfont.woff?fee66e712a8a08eef5805a46892932ad) format("woff"),url(fonts/fontawesome-webfont.ttf?b06871f281fee6b241d60582ae9369b9) format("truetype"),url(fonts/fontawesome-webfont.svg?912ec66d7572ff821749319396470bde#FontAwesome) format("svg")}.fa:before{font-family:FontAwesome;font-style:normal;font-weight:400;line-height:1}.fa:before,a .fa{text-decoration:inherit}.fa:before,a .fa,li .fa{display:inline-block}li .fa-large:before{width:1.875em}ul.fas{list-style-type:none;margin-left:2em;text-indent:-.8em}ul.fas li .fa{width:.8em}ul.fas li .fa-large:before{vertical-align:baseline}.fa-book:before,.icon-book:before{content:"\f02d"}.fa-caret-down:before,.icon-caret-down:before{content:"\f0d7"}.fa-caret-up:before,.icon-caret-up:before{content:"\f0d8"}.fa-caret-left:before,.icon-caret-left:before{content:"\f0d9"}.fa-caret-right:before,.icon-caret-right:before{content:"\f0da"}.rst-versions{position:fixed;bottom:0;left:0;width:300px;color:#fcfcfc;background:#1f1d1d;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;z-index:400}.rst-versions a{color:#2980b9;text-decoration:none}.rst-versions .rst-badge-small{display:none}.rst-versions .rst-current-version{padding:12px;background-color:#272525;display:block;text-align:right;font-size:90%;cursor:pointer;color:#27ae60}.rst-versions .rst-current-version:after{clear:both;content:"";display:block}.rst-versions .rst-current-version .fa{color:#fcfcfc}.rst-versions .rst-current-version .fa-book,.rst-versions .rst-current-version .icon-book{float:left}.rst-versions .rst-current-version.rst-out-of-date{background-color:#e74c3c;color:#fff}.rst-versions .rst-current-version.rst-active-old-version{background-color:#f1c40f;color:#000}.rst-versions.shift-up{height:auto;max-height:100%;overflow-y:scroll}.rst-versions.shift-up .rst-other-versions{display:block}.rst-versions .rst-other-versions{font-size:90%;padding:12px;color:grey;display:none}.rst-versions .rst-other-versions hr{display:block;height:1px;border:0;margin:20px 0;padding:0;border-top:1px solid #413d3d}.rst-versions .rst-other-versions dd{display:inline-block;margin:0}.rst-versions .rst-other-versions dd a{display:inline-block;padding:6px;color:#fcfcfc}.rst-versions.rst-badge{width:auto;bottom:20px;right:20px;left:auto;border:none;max-width:300px;max-height:90%}.rst-versions.rst-badge .fa-book,.rst-versions.rst-badge .icon-book{float:none;line-height:30px}.rst-versions.rst-badge.shift-up .rst-current-version{text-align:right}.rst-versions.rst-badge.shift-up .rst-current-version .fa-book,.rst-versions.rst-badge.shift-up .rst-current-version .icon-book{float:left}.rst-versions.rst-badge>.rst-current-version{width:auto;height:30px;line-height:30px;padding:0 6px;display:block;text-align:center}@media screen and (max-width:768px){.rst-versions{width:85%;display:none}.rst-versions.shift{display:block}} \ No newline at end of file diff --git a/_static/css/fonts/Roboto-Slab-Bold.woff b/_static/css/fonts/Roboto-Slab-Bold.woff new file mode 100644 index 0000000..6cb6000 Binary files /dev/null and b/_static/css/fonts/Roboto-Slab-Bold.woff differ diff --git a/_static/css/fonts/Roboto-Slab-Bold.woff2 b/_static/css/fonts/Roboto-Slab-Bold.woff2 new file mode 100644 index 0000000..7059e23 Binary files /dev/null and b/_static/css/fonts/Roboto-Slab-Bold.woff2 differ diff --git a/_static/css/fonts/Roboto-Slab-Regular.woff b/_static/css/fonts/Roboto-Slab-Regular.woff new file mode 100644 index 0000000..f815f63 Binary files /dev/null and b/_static/css/fonts/Roboto-Slab-Regular.woff differ diff --git a/_static/css/fonts/Roboto-Slab-Regular.woff2 b/_static/css/fonts/Roboto-Slab-Regular.woff2 new file mode 100644 index 0000000..f2c76e5 Binary files /dev/null and b/_static/css/fonts/Roboto-Slab-Regular.woff2 differ diff --git a/_static/css/fonts/fontawesome-webfont.eot b/_static/css/fonts/fontawesome-webfont.eot new file mode 100644 index 0000000..e9f60ca Binary files /dev/null and b/_static/css/fonts/fontawesome-webfont.eot differ diff --git a/_static/css/fonts/fontawesome-webfont.svg b/_static/css/fonts/fontawesome-webfont.svg new file mode 100644 index 0000000..855c845 --- /dev/null +++ b/_static/css/fonts/fontawesome-webfont.svg @@ -0,0 +1,2671 @@ + + + + +Created by FontForge 20120731 at Mon Oct 24 17:37:40 2016 + By ,,, +Copyright Dave Gandy 2016. All rights reserved. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/_static/css/fonts/fontawesome-webfont.ttf b/_static/css/fonts/fontawesome-webfont.ttf new file mode 100644 index 0000000..35acda2 Binary files /dev/null and b/_static/css/fonts/fontawesome-webfont.ttf differ diff --git a/_static/css/fonts/fontawesome-webfont.woff b/_static/css/fonts/fontawesome-webfont.woff new file mode 100644 index 0000000..400014a Binary files /dev/null and b/_static/css/fonts/fontawesome-webfont.woff differ diff --git a/_static/css/fonts/fontawesome-webfont.woff2 b/_static/css/fonts/fontawesome-webfont.woff2 new file mode 100644 index 0000000..4d13fc6 Binary files /dev/null and b/_static/css/fonts/fontawesome-webfont.woff2 differ diff --git a/_static/css/fonts/lato-bold-italic.woff b/_static/css/fonts/lato-bold-italic.woff new file mode 100644 index 0000000..88ad05b Binary files /dev/null and b/_static/css/fonts/lato-bold-italic.woff differ diff --git a/_static/css/fonts/lato-bold-italic.woff2 b/_static/css/fonts/lato-bold-italic.woff2 new file mode 100644 index 0000000..c4e3d80 Binary files /dev/null and b/_static/css/fonts/lato-bold-italic.woff2 differ diff --git a/_static/css/fonts/lato-bold.woff b/_static/css/fonts/lato-bold.woff new file mode 100644 index 0000000..c6dff51 Binary files /dev/null and b/_static/css/fonts/lato-bold.woff differ diff --git a/_static/css/fonts/lato-bold.woff2 b/_static/css/fonts/lato-bold.woff2 new file mode 100644 index 0000000..bb19504 Binary files /dev/null and b/_static/css/fonts/lato-bold.woff2 differ diff --git a/_static/css/fonts/lato-normal-italic.woff b/_static/css/fonts/lato-normal-italic.woff new file mode 100644 index 0000000..76114bc Binary files /dev/null and b/_static/css/fonts/lato-normal-italic.woff differ diff --git a/_static/css/fonts/lato-normal-italic.woff2 b/_static/css/fonts/lato-normal-italic.woff2 new file mode 100644 index 0000000..3404f37 Binary files /dev/null and b/_static/css/fonts/lato-normal-italic.woff2 differ diff --git a/_static/css/fonts/lato-normal.woff b/_static/css/fonts/lato-normal.woff new file mode 100644 index 0000000..ae1307f Binary files /dev/null and b/_static/css/fonts/lato-normal.woff differ diff --git a/_static/css/fonts/lato-normal.woff2 b/_static/css/fonts/lato-normal.woff2 new file mode 100644 index 0000000..3bf9843 Binary files /dev/null and b/_static/css/fonts/lato-normal.woff2 differ diff --git a/_static/css/theme.css b/_static/css/theme.css new file mode 100644 index 0000000..19a446a --- /dev/null +++ b/_static/css/theme.css @@ -0,0 +1,4 @@ +html{box-sizing:border-box}*,:after,:before{box-sizing:inherit}article,aside,details,figcaption,figure,footer,header,hgroup,nav,section{display:block}audio,canvas,video{display:inline-block;*display:inline;*zoom:1}[hidden],audio:not([controls]){display:none}*{-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}html{font-size:100%;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}body{margin:0}a:active,a:hover{outline:0}abbr[title]{border-bottom:1px dotted}b,strong{font-weight:700}blockquote{margin:0}dfn{font-style:italic}ins{background:#ff9;text-decoration:none}ins,mark{color:#000}mark{background:#ff0;font-style:italic;font-weight:700}.rst-content code,.rst-content tt,code,kbd,pre,samp{font-family:monospace,serif;_font-family:courier new,monospace;font-size:1em}pre{white-space:pre}q{quotes:none}q:after,q:before{content:"";content:none}small{font-size:85%}sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}sup{top:-.5em}sub{bottom:-.25em}dl,ol,ul{margin:0;padding:0;list-style:none;list-style-image:none}li{list-style:none}dd{margin:0}img{border:0;-ms-interpolation-mode:bicubic;vertical-align:middle;max-width:100%}svg:not(:root){overflow:hidden}figure,form{margin:0}label{cursor:pointer}button,input,select,textarea{font-size:100%;margin:0;vertical-align:baseline;*vertical-align:middle}button,input{line-height:normal}button,input[type=button],input[type=reset],input[type=submit]{cursor:pointer;-webkit-appearance:button;*overflow:visible}button[disabled],input[disabled]{cursor:default}input[type=search]{-webkit-appearance:textfield;-moz-box-sizing:content-box;-webkit-box-sizing:content-box;box-sizing:content-box}textarea{resize:vertical}table{border-collapse:collapse;border-spacing:0}td{vertical-align:top}.chromeframe{margin:.2em 0;background:#ccc;color:#000;padding:.2em 0}.ir{display:block;border:0;text-indent:-999em;overflow:hidden;background-color:transparent;background-repeat:no-repeat;text-align:left;direction:ltr;*line-height:0}.ir br{display:none}.hidden{display:none!important;visibility:hidden}.visuallyhidden{border:0;clip:rect(0 0 0 0);height:1px;margin:-1px;overflow:hidden;padding:0;position:absolute;width:1px}.visuallyhidden.focusable:active,.visuallyhidden.focusable:focus{clip:auto;height:auto;margin:0;overflow:visible;position:static;width:auto}.invisible{visibility:hidden}.relative{position:relative}big,small{font-size:100%}@media print{body,html,section{background:none!important}*{box-shadow:none!important;text-shadow:none!important;filter:none!important;-ms-filter:none!important}a,a:visited{text-decoration:underline}.ir a:after,a[href^="#"]:after,a[href^="javascript:"]:after{content:""}blockquote,pre{page-break-inside:avoid}thead{display:table-header-group}img,tr{page-break-inside:avoid}img{max-width:100%!important}@page{margin:.5cm}.rst-content .toctree-wrapper>p.caption,h2,h3,p{orphans:3;widows:3}.rst-content .toctree-wrapper>p.caption,h2,h3{page-break-after:avoid}}.btn,.fa:before,.icon:before,.rst-content .admonition,.rst-content .admonition-title:before,.rst-content .admonition-todo,.rst-content .attention,.rst-content .caution,.rst-content .code-block-caption .headerlink:before,.rst-content .danger,.rst-content .eqno .headerlink:before,.rst-content .error,.rst-content .hint,.rst-content .important,.rst-content .note,.rst-content .seealso,.rst-content .tip,.rst-content .warning,.rst-content code.download span:first-child:before,.rst-content dl dt .headerlink:before,.rst-content h1 .headerlink:before,.rst-content h2 .headerlink:before,.rst-content h3 .headerlink:before,.rst-content h4 .headerlink:before,.rst-content h5 .headerlink:before,.rst-content h6 .headerlink:before,.rst-content p.caption .headerlink:before,.rst-content p .headerlink:before,.rst-content table>caption .headerlink:before,.rst-content tt.download span:first-child:before,.wy-alert,.wy-dropdown .caret:before,.wy-inline-validate.wy-inline-validate-danger .wy-input-context:before,.wy-inline-validate.wy-inline-validate-info .wy-input-context:before,.wy-inline-validate.wy-inline-validate-success .wy-input-context:before,.wy-inline-validate.wy-inline-validate-warning .wy-input-context:before,.wy-menu-vertical li.current>a button.toctree-expand:before,.wy-menu-vertical li.on a button.toctree-expand:before,.wy-menu-vertical li button.toctree-expand:before,input[type=color],input[type=date],input[type=datetime-local],input[type=datetime],input[type=email],input[type=month],input[type=number],input[type=password],input[type=search],input[type=tel],input[type=text],input[type=time],input[type=url],input[type=week],select,textarea{-webkit-font-smoothing:antialiased}.clearfix{*zoom:1}.clearfix:after,.clearfix:before{display:table;content:""}.clearfix:after{clear:both}/*! + * Font Awesome 4.7.0 by @davegandy - http://fontawesome.io - @fontawesome + * License - http://fontawesome.io/license (Font: SIL OFL 1.1, CSS: MIT License) + */@font-face{font-family:FontAwesome;src:url(fonts/fontawesome-webfont.eot?674f50d287a8c48dc19ba404d20fe713);src:url(fonts/fontawesome-webfont.eot?674f50d287a8c48dc19ba404d20fe713?#iefix&v=4.7.0) format("embedded-opentype"),url(fonts/fontawesome-webfont.woff2?af7ae505a9eed503f8b8e6982036873e) format("woff2"),url(fonts/fontawesome-webfont.woff?fee66e712a8a08eef5805a46892932ad) format("woff"),url(fonts/fontawesome-webfont.ttf?b06871f281fee6b241d60582ae9369b9) format("truetype"),url(fonts/fontawesome-webfont.svg?912ec66d7572ff821749319396470bde#fontawesomeregular) format("svg");font-weight:400;font-style:normal}.fa,.icon,.rst-content .admonition-title,.rst-content .code-block-caption .headerlink,.rst-content .eqno .headerlink,.rst-content code.download span:first-child,.rst-content dl dt .headerlink,.rst-content h1 .headerlink,.rst-content h2 .headerlink,.rst-content h3 .headerlink,.rst-content h4 .headerlink,.rst-content h5 .headerlink,.rst-content h6 .headerlink,.rst-content p.caption .headerlink,.rst-content p .headerlink,.rst-content table>caption .headerlink,.rst-content tt.download span:first-child,.wy-menu-vertical li.current>a button.toctree-expand,.wy-menu-vertical li.on a button.toctree-expand,.wy-menu-vertical li button.toctree-expand{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}.fa-lg{font-size:1.33333em;line-height:.75em;vertical-align:-15%}.fa-2x{font-size:2em}.fa-3x{font-size:3em}.fa-4x{font-size:4em}.fa-5x{font-size:5em}.fa-fw{width:1.28571em;text-align:center}.fa-ul{padding-left:0;margin-left:2.14286em;list-style-type:none}.fa-ul>li{position:relative}.fa-li{position:absolute;left:-2.14286em;width:2.14286em;top:.14286em;text-align:center}.fa-li.fa-lg{left:-1.85714em}.fa-border{padding:.2em .25em .15em;border:.08em solid #eee;border-radius:.1em}.fa-pull-left{float:left}.fa-pull-right{float:right}.fa-pull-left.icon,.fa.fa-pull-left,.rst-content .code-block-caption .fa-pull-left.headerlink,.rst-content .eqno .fa-pull-left.headerlink,.rst-content .fa-pull-left.admonition-title,.rst-content code.download span.fa-pull-left:first-child,.rst-content dl dt .fa-pull-left.headerlink,.rst-content h1 .fa-pull-left.headerlink,.rst-content h2 .fa-pull-left.headerlink,.rst-content h3 .fa-pull-left.headerlink,.rst-content h4 .fa-pull-left.headerlink,.rst-content h5 .fa-pull-left.headerlink,.rst-content h6 .fa-pull-left.headerlink,.rst-content p .fa-pull-left.headerlink,.rst-content table>caption .fa-pull-left.headerlink,.rst-content tt.download span.fa-pull-left:first-child,.wy-menu-vertical li.current>a button.fa-pull-left.toctree-expand,.wy-menu-vertical li.on a button.fa-pull-left.toctree-expand,.wy-menu-vertical li button.fa-pull-left.toctree-expand{margin-right:.3em}.fa-pull-right.icon,.fa.fa-pull-right,.rst-content .code-block-caption .fa-pull-right.headerlink,.rst-content .eqno .fa-pull-right.headerlink,.rst-content .fa-pull-right.admonition-title,.rst-content code.download span.fa-pull-right:first-child,.rst-content dl dt .fa-pull-right.headerlink,.rst-content h1 .fa-pull-right.headerlink,.rst-content h2 .fa-pull-right.headerlink,.rst-content h3 .fa-pull-right.headerlink,.rst-content h4 .fa-pull-right.headerlink,.rst-content h5 .fa-pull-right.headerlink,.rst-content h6 .fa-pull-right.headerlink,.rst-content p .fa-pull-right.headerlink,.rst-content table>caption .fa-pull-right.headerlink,.rst-content tt.download span.fa-pull-right:first-child,.wy-menu-vertical li.current>a button.fa-pull-right.toctree-expand,.wy-menu-vertical li.on a button.fa-pull-right.toctree-expand,.wy-menu-vertical li button.fa-pull-right.toctree-expand{margin-left:.3em}.pull-right{float:right}.pull-left{float:left}.fa.pull-left,.pull-left.icon,.rst-content .code-block-caption .pull-left.headerlink,.rst-content .eqno .pull-left.headerlink,.rst-content .pull-left.admonition-title,.rst-content code.download span.pull-left:first-child,.rst-content dl dt .pull-left.headerlink,.rst-content h1 .pull-left.headerlink,.rst-content h2 .pull-left.headerlink,.rst-content h3 .pull-left.headerlink,.rst-content h4 .pull-left.headerlink,.rst-content h5 .pull-left.headerlink,.rst-content h6 .pull-left.headerlink,.rst-content p .pull-left.headerlink,.rst-content table>caption .pull-left.headerlink,.rst-content tt.download span.pull-left:first-child,.wy-menu-vertical li.current>a button.pull-left.toctree-expand,.wy-menu-vertical li.on a button.pull-left.toctree-expand,.wy-menu-vertical li button.pull-left.toctree-expand{margin-right:.3em}.fa.pull-right,.pull-right.icon,.rst-content .code-block-caption .pull-right.headerlink,.rst-content .eqno .pull-right.headerlink,.rst-content .pull-right.admonition-title,.rst-content code.download span.pull-right:first-child,.rst-content dl dt .pull-right.headerlink,.rst-content h1 .pull-right.headerlink,.rst-content h2 .pull-right.headerlink,.rst-content h3 .pull-right.headerlink,.rst-content h4 .pull-right.headerlink,.rst-content h5 .pull-right.headerlink,.rst-content h6 .pull-right.headerlink,.rst-content p .pull-right.headerlink,.rst-content table>caption .pull-right.headerlink,.rst-content tt.download span.pull-right:first-child,.wy-menu-vertical li.current>a button.pull-right.toctree-expand,.wy-menu-vertical li.on a button.pull-right.toctree-expand,.wy-menu-vertical li button.pull-right.toctree-expand{margin-left:.3em}.fa-spin{-webkit-animation:fa-spin 2s linear infinite;animation:fa-spin 2s linear infinite}.fa-pulse{-webkit-animation:fa-spin 1s steps(8) infinite;animation:fa-spin 1s steps(8) infinite}@-webkit-keyframes fa-spin{0%{-webkit-transform:rotate(0deg);transform:rotate(0deg)}to{-webkit-transform:rotate(359deg);transform:rotate(359deg)}}@keyframes fa-spin{0%{-webkit-transform:rotate(0deg);transform:rotate(0deg)}to{-webkit-transform:rotate(359deg);transform:rotate(359deg)}}.fa-rotate-90{-ms-filter:"progid:DXImageTransform.Microsoft.BasicImage(rotation=1)";-webkit-transform:rotate(90deg);-ms-transform:rotate(90deg);transform:rotate(90deg)}.fa-rotate-180{-ms-filter:"progid:DXImageTransform.Microsoft.BasicImage(rotation=2)";-webkit-transform:rotate(180deg);-ms-transform:rotate(180deg);transform:rotate(180deg)}.fa-rotate-270{-ms-filter:"progid:DXImageTransform.Microsoft.BasicImage(rotation=3)";-webkit-transform:rotate(270deg);-ms-transform:rotate(270deg);transform:rotate(270deg)}.fa-flip-horizontal{-ms-filter:"progid:DXImageTransform.Microsoft.BasicImage(rotation=0, mirror=1)";-webkit-transform:scaleX(-1);-ms-transform:scaleX(-1);transform:scaleX(-1)}.fa-flip-vertical{-ms-filter:"progid:DXImageTransform.Microsoft.BasicImage(rotation=2, mirror=1)";-webkit-transform:scaleY(-1);-ms-transform:scaleY(-1);transform:scaleY(-1)}:root .fa-flip-horizontal,:root .fa-flip-vertical,:root .fa-rotate-90,:root .fa-rotate-180,:root .fa-rotate-270{filter:none}.fa-stack{position:relative;display:inline-block;width:2em;height:2em;line-height:2em;vertical-align:middle}.fa-stack-1x,.fa-stack-2x{position:absolute;left:0;width:100%;text-align:center}.fa-stack-1x{line-height:inherit}.fa-stack-2x{font-size:2em}.fa-inverse{color:#fff}.fa-glass:before{content:""}.fa-music:before{content:""}.fa-search:before,.icon-search:before{content:""}.fa-envelope-o:before{content:""}.fa-heart:before{content:""}.fa-star:before{content:""}.fa-star-o:before{content:""}.fa-user:before{content:""}.fa-film:before{content:""}.fa-th-large:before{content:""}.fa-th:before{content:""}.fa-th-list:before{content:""}.fa-check:before{content:""}.fa-close:before,.fa-remove:before,.fa-times:before{content:""}.fa-search-plus:before{content:""}.fa-search-minus:before{content:""}.fa-power-off:before{content:""}.fa-signal:before{content:""}.fa-cog:before,.fa-gear:before{content:""}.fa-trash-o:before{content:""}.fa-home:before,.icon-home:before{content:""}.fa-file-o:before{content:""}.fa-clock-o:before{content:""}.fa-road:before{content:""}.fa-download:before,.rst-content code.download span:first-child:before,.rst-content tt.download span:first-child:before{content:""}.fa-arrow-circle-o-down:before{content:""}.fa-arrow-circle-o-up:before{content:""}.fa-inbox:before{content:""}.fa-play-circle-o:before{content:""}.fa-repeat:before,.fa-rotate-right:before{content:""}.fa-refresh:before{content:""}.fa-list-alt:before{content:""}.fa-lock:before{content:""}.fa-flag:before{content:""}.fa-headphones:before{content:""}.fa-volume-off:before{content:""}.fa-volume-down:before{content:""}.fa-volume-up:before{content:""}.fa-qrcode:before{content:""}.fa-barcode:before{content:""}.fa-tag:before{content:""}.fa-tags:before{content:""}.fa-book:before,.icon-book:before{content:""}.fa-bookmark:before{content:""}.fa-print:before{content:""}.fa-camera:before{content:""}.fa-font:before{content:""}.fa-bold:before{content:""}.fa-italic:before{content:""}.fa-text-height:before{content:""}.fa-text-width:before{content:""}.fa-align-left:before{content:""}.fa-align-center:before{content:""}.fa-align-right:before{content:""}.fa-align-justify:before{content:""}.fa-list:before{content:""}.fa-dedent:before,.fa-outdent:before{content:""}.fa-indent:before{content:""}.fa-video-camera:before{content:""}.fa-image:before,.fa-photo:before,.fa-picture-o:before{content:""}.fa-pencil:before{content:""}.fa-map-marker:before{content:""}.fa-adjust:before{content:""}.fa-tint:before{content:""}.fa-edit:before,.fa-pencil-square-o:before{content:""}.fa-share-square-o:before{content:""}.fa-check-square-o:before{content:""}.fa-arrows:before{content:""}.fa-step-backward:before{content:""}.fa-fast-backward:before{content:""}.fa-backward:before{content:""}.fa-play:before{content:""}.fa-pause:before{content:""}.fa-stop:before{content:""}.fa-forward:before{content:""}.fa-fast-forward:before{content:""}.fa-step-forward:before{content:""}.fa-eject:before{content:""}.fa-chevron-left:before{content:""}.fa-chevron-right:before{content:""}.fa-plus-circle:before{content:""}.fa-minus-circle:before{content:""}.fa-times-circle:before,.wy-inline-validate.wy-inline-validate-danger .wy-input-context:before{content:""}.fa-check-circle:before,.wy-inline-validate.wy-inline-validate-success .wy-input-context:before{content:""}.fa-question-circle:before{content:""}.fa-info-circle:before{content:""}.fa-crosshairs:before{content:""}.fa-times-circle-o:before{content:""}.fa-check-circle-o:before{content:""}.fa-ban:before{content:""}.fa-arrow-left:before{content:""}.fa-arrow-right:before{content:""}.fa-arrow-up:before{content:""}.fa-arrow-down:before{content:""}.fa-mail-forward:before,.fa-share:before{content:""}.fa-expand:before{content:""}.fa-compress:before{content:""}.fa-plus:before{content:""}.fa-minus:before{content:""}.fa-asterisk:before{content:""}.fa-exclamation-circle:before,.rst-content .admonition-title:before,.wy-inline-validate.wy-inline-validate-info .wy-input-context:before,.wy-inline-validate.wy-inline-validate-warning .wy-input-context:before{content:""}.fa-gift:before{content:""}.fa-leaf:before{content:""}.fa-fire:before,.icon-fire:before{content:""}.fa-eye:before{content:""}.fa-eye-slash:before{content:""}.fa-exclamation-triangle:before,.fa-warning:before{content:""}.fa-plane:before{content:""}.fa-calendar:before{content:""}.fa-random:before{content:""}.fa-comment:before{content:""}.fa-magnet:before{content:""}.fa-chevron-up:before{content:""}.fa-chevron-down:before{content:""}.fa-retweet:before{content:""}.fa-shopping-cart:before{content:""}.fa-folder:before{content:""}.fa-folder-open:before{content:""}.fa-arrows-v:before{content:""}.fa-arrows-h:before{content:""}.fa-bar-chart-o:before,.fa-bar-chart:before{content:""}.fa-twitter-square:before{content:""}.fa-facebook-square:before{content:""}.fa-camera-retro:before{content:""}.fa-key:before{content:""}.fa-cogs:before,.fa-gears:before{content:""}.fa-comments:before{content:""}.fa-thumbs-o-up:before{content:""}.fa-thumbs-o-down:before{content:""}.fa-star-half:before{content:""}.fa-heart-o:before{content:""}.fa-sign-out:before{content:""}.fa-linkedin-square:before{content:""}.fa-thumb-tack:before{content:""}.fa-external-link:before{content:""}.fa-sign-in:before{content:""}.fa-trophy:before{content:""}.fa-github-square:before{content:""}.fa-upload:before{content:""}.fa-lemon-o:before{content:""}.fa-phone:before{content:""}.fa-square-o:before{content:""}.fa-bookmark-o:before{content:""}.fa-phone-square:before{content:""}.fa-twitter:before{content:""}.fa-facebook-f:before,.fa-facebook:before{content:""}.fa-github:before,.icon-github:before{content:""}.fa-unlock:before{content:""}.fa-credit-card:before{content:""}.fa-feed:before,.fa-rss:before{content:""}.fa-hdd-o:before{content:""}.fa-bullhorn:before{content:""}.fa-bell:before{content:""}.fa-certificate:before{content:""}.fa-hand-o-right:before{content:""}.fa-hand-o-left:before{content:""}.fa-hand-o-up:before{content:""}.fa-hand-o-down:before{content:""}.fa-arrow-circle-left:before,.icon-circle-arrow-left:before{content:""}.fa-arrow-circle-right:before,.icon-circle-arrow-right:before{content:""}.fa-arrow-circle-up:before{content:""}.fa-arrow-circle-down:before{content:""}.fa-globe:before{content:""}.fa-wrench:before{content:""}.fa-tasks:before{content:""}.fa-filter:before{content:""}.fa-briefcase:before{content:""}.fa-arrows-alt:before{content:""}.fa-group:before,.fa-users:before{content:""}.fa-chain:before,.fa-link:before,.icon-link:before{content:""}.fa-cloud:before{content:""}.fa-flask:before{content:""}.fa-cut:before,.fa-scissors:before{content:""}.fa-copy:before,.fa-files-o:before{content:""}.fa-paperclip:before{content:""}.fa-floppy-o:before,.fa-save:before{content:""}.fa-square:before{content:""}.fa-bars:before,.fa-navicon:before,.fa-reorder:before{content:""}.fa-list-ul:before{content:""}.fa-list-ol:before{content:""}.fa-strikethrough:before{content:""}.fa-underline:before{content:""}.fa-table:before{content:""}.fa-magic:before{content:""}.fa-truck:before{content:""}.fa-pinterest:before{content:""}.fa-pinterest-square:before{content:""}.fa-google-plus-square:before{content:""}.fa-google-plus:before{content:""}.fa-money:before{content:""}.fa-caret-down:before,.icon-caret-down:before,.wy-dropdown .caret:before{content:""}.fa-caret-up:before{content:""}.fa-caret-left:before{content:""}.fa-caret-right:before{content:""}.fa-columns:before{content:""}.fa-sort:before,.fa-unsorted:before{content:""}.fa-sort-desc:before,.fa-sort-down:before{content:""}.fa-sort-asc:before,.fa-sort-up:before{content:""}.fa-envelope:before{content:""}.fa-linkedin:before{content:""}.fa-rotate-left:before,.fa-undo:before{content:""}.fa-gavel:before,.fa-legal:before{content:""}.fa-dashboard:before,.fa-tachometer:before{content:""}.fa-comment-o:before{content:""}.fa-comments-o:before{content:""}.fa-bolt:before,.fa-flash:before{content:""}.fa-sitemap:before{content:""}.fa-umbrella:before{content:""}.fa-clipboard:before,.fa-paste:before{content:""}.fa-lightbulb-o:before{content:""}.fa-exchange:before{content:""}.fa-cloud-download:before{content:""}.fa-cloud-upload:before{content:""}.fa-user-md:before{content:""}.fa-stethoscope:before{content:""}.fa-suitcase:before{content:""}.fa-bell-o:before{content:""}.fa-coffee:before{content:""}.fa-cutlery:before{content:""}.fa-file-text-o:before{content:""}.fa-building-o:before{content:""}.fa-hospital-o:before{content:""}.fa-ambulance:before{content:""}.fa-medkit:before{content:""}.fa-fighter-jet:before{content:""}.fa-beer:before{content:""}.fa-h-square:before{content:""}.fa-plus-square:before{content:""}.fa-angle-double-left:before{content:""}.fa-angle-double-right:before{content:""}.fa-angle-double-up:before{content:""}.fa-angle-double-down:before{content:""}.fa-angle-left:before{content:""}.fa-angle-right:before{content:""}.fa-angle-up:before{content:""}.fa-angle-down:before{content:""}.fa-desktop:before{content:""}.fa-laptop:before{content:""}.fa-tablet:before{content:""}.fa-mobile-phone:before,.fa-mobile:before{content:""}.fa-circle-o:before{content:""}.fa-quote-left:before{content:""}.fa-quote-right:before{content:""}.fa-spinner:before{content:""}.fa-circle:before{content:""}.fa-mail-reply:before,.fa-reply:before{content:""}.fa-github-alt:before{content:""}.fa-folder-o:before{content:""}.fa-folder-open-o:before{content:""}.fa-smile-o:before{content:""}.fa-frown-o:before{content:""}.fa-meh-o:before{content:""}.fa-gamepad:before{content:""}.fa-keyboard-o:before{content:""}.fa-flag-o:before{content:""}.fa-flag-checkered:before{content:""}.fa-terminal:before{content:""}.fa-code:before{content:""}.fa-mail-reply-all:before,.fa-reply-all:before{content:""}.fa-star-half-empty:before,.fa-star-half-full:before,.fa-star-half-o:before{content:""}.fa-location-arrow:before{content:""}.fa-crop:before{content:""}.fa-code-fork:before{content:""}.fa-chain-broken:before,.fa-unlink:before{content:""}.fa-question:before{content:""}.fa-info:before{content:""}.fa-exclamation:before{content:""}.fa-superscript:before{content:""}.fa-subscript:before{content:""}.fa-eraser:before{content:""}.fa-puzzle-piece:before{content:""}.fa-microphone:before{content:""}.fa-microphone-slash:before{content:""}.fa-shield:before{content:""}.fa-calendar-o:before{content:""}.fa-fire-extinguisher:before{content:""}.fa-rocket:before{content:""}.fa-maxcdn:before{content:""}.fa-chevron-circle-left:before{content:""}.fa-chevron-circle-right:before{content:""}.fa-chevron-circle-up:before{content:""}.fa-chevron-circle-down:before{content:""}.fa-html5:before{content:""}.fa-css3:before{content:""}.fa-anchor:before{content:""}.fa-unlock-alt:before{content:""}.fa-bullseye:before{content:""}.fa-ellipsis-h:before{content:""}.fa-ellipsis-v:before{content:""}.fa-rss-square:before{content:""}.fa-play-circle:before{content:""}.fa-ticket:before{content:""}.fa-minus-square:before{content:""}.fa-minus-square-o:before,.wy-menu-vertical li.current>a button.toctree-expand:before,.wy-menu-vertical li.on a button.toctree-expand:before{content:""}.fa-level-up:before{content:""}.fa-level-down:before{content:""}.fa-check-square:before{content:""}.fa-pencil-square:before{content:""}.fa-external-link-square:before{content:""}.fa-share-square:before{content:""}.fa-compass:before{content:""}.fa-caret-square-o-down:before,.fa-toggle-down:before{content:""}.fa-caret-square-o-up:before,.fa-toggle-up:before{content:""}.fa-caret-square-o-right:before,.fa-toggle-right:before{content:""}.fa-eur:before,.fa-euro:before{content:""}.fa-gbp:before{content:""}.fa-dollar:before,.fa-usd:before{content:""}.fa-inr:before,.fa-rupee:before{content:""}.fa-cny:before,.fa-jpy:before,.fa-rmb:before,.fa-yen:before{content:""}.fa-rouble:before,.fa-rub:before,.fa-ruble:before{content:""}.fa-krw:before,.fa-won:before{content:""}.fa-bitcoin:before,.fa-btc:before{content:""}.fa-file:before{content:""}.fa-file-text:before{content:""}.fa-sort-alpha-asc:before{content:""}.fa-sort-alpha-desc:before{content:""}.fa-sort-amount-asc:before{content:""}.fa-sort-amount-desc:before{content:""}.fa-sort-numeric-asc:before{content:""}.fa-sort-numeric-desc:before{content:""}.fa-thumbs-up:before{content:""}.fa-thumbs-down:before{content:""}.fa-youtube-square:before{content:""}.fa-youtube:before{content:""}.fa-xing:before{content:""}.fa-xing-square:before{content:""}.fa-youtube-play:before{content:""}.fa-dropbox:before{content:""}.fa-stack-overflow:before{content:""}.fa-instagram:before{content:""}.fa-flickr:before{content:""}.fa-adn:before{content:""}.fa-bitbucket:before,.icon-bitbucket:before{content:""}.fa-bitbucket-square:before{content:""}.fa-tumblr:before{content:""}.fa-tumblr-square:before{content:""}.fa-long-arrow-down:before{content:""}.fa-long-arrow-up:before{content:""}.fa-long-arrow-left:before{content:""}.fa-long-arrow-right:before{content:""}.fa-apple:before{content:""}.fa-windows:before{content:""}.fa-android:before{content:""}.fa-linux:before{content:""}.fa-dribbble:before{content:""}.fa-skype:before{content:""}.fa-foursquare:before{content:""}.fa-trello:before{content:""}.fa-female:before{content:""}.fa-male:before{content:""}.fa-gittip:before,.fa-gratipay:before{content:""}.fa-sun-o:before{content:""}.fa-moon-o:before{content:""}.fa-archive:before{content:""}.fa-bug:before{content:""}.fa-vk:before{content:""}.fa-weibo:before{content:""}.fa-renren:before{content:""}.fa-pagelines:before{content:""}.fa-stack-exchange:before{content:""}.fa-arrow-circle-o-right:before{content:""}.fa-arrow-circle-o-left:before{content:""}.fa-caret-square-o-left:before,.fa-toggle-left:before{content:""}.fa-dot-circle-o:before{content:""}.fa-wheelchair:before{content:""}.fa-vimeo-square:before{content:""}.fa-try:before,.fa-turkish-lira:before{content:""}.fa-plus-square-o:before,.wy-menu-vertical li button.toctree-expand:before{content:""}.fa-space-shuttle:before{content:""}.fa-slack:before{content:""}.fa-envelope-square:before{content:""}.fa-wordpress:before{content:""}.fa-openid:before{content:""}.fa-bank:before,.fa-institution:before,.fa-university:before{content:""}.fa-graduation-cap:before,.fa-mortar-board:before{content:""}.fa-yahoo:before{content:""}.fa-google:before{content:""}.fa-reddit:before{content:""}.fa-reddit-square:before{content:""}.fa-stumbleupon-circle:before{content:""}.fa-stumbleupon:before{content:""}.fa-delicious:before{content:""}.fa-digg:before{content:""}.fa-pied-piper-pp:before{content:""}.fa-pied-piper-alt:before{content:""}.fa-drupal:before{content:""}.fa-joomla:before{content:""}.fa-language:before{content:""}.fa-fax:before{content:""}.fa-building:before{content:""}.fa-child:before{content:""}.fa-paw:before{content:""}.fa-spoon:before{content:""}.fa-cube:before{content:""}.fa-cubes:before{content:""}.fa-behance:before{content:""}.fa-behance-square:before{content:""}.fa-steam:before{content:""}.fa-steam-square:before{content:""}.fa-recycle:before{content:""}.fa-automobile:before,.fa-car:before{content:""}.fa-cab:before,.fa-taxi:before{content:""}.fa-tree:before{content:""}.fa-spotify:before{content:""}.fa-deviantart:before{content:""}.fa-soundcloud:before{content:""}.fa-database:before{content:""}.fa-file-pdf-o:before{content:""}.fa-file-word-o:before{content:""}.fa-file-excel-o:before{content:""}.fa-file-powerpoint-o:before{content:""}.fa-file-image-o:before,.fa-file-photo-o:before,.fa-file-picture-o:before{content:""}.fa-file-archive-o:before,.fa-file-zip-o:before{content:""}.fa-file-audio-o:before,.fa-file-sound-o:before{content:""}.fa-file-movie-o:before,.fa-file-video-o:before{content:""}.fa-file-code-o:before{content:""}.fa-vine:before{content:""}.fa-codepen:before{content:""}.fa-jsfiddle:before{content:""}.fa-life-bouy:before,.fa-life-buoy:before,.fa-life-ring:before,.fa-life-saver:before,.fa-support:before{content:""}.fa-circle-o-notch:before{content:""}.fa-ra:before,.fa-rebel:before,.fa-resistance:before{content:""}.fa-empire:before,.fa-ge:before{content:""}.fa-git-square:before{content:""}.fa-git:before{content:""}.fa-hacker-news:before,.fa-y-combinator-square:before,.fa-yc-square:before{content:""}.fa-tencent-weibo:before{content:""}.fa-qq:before{content:""}.fa-wechat:before,.fa-weixin:before{content:""}.fa-paper-plane:before,.fa-send:before{content:""}.fa-paper-plane-o:before,.fa-send-o:before{content:""}.fa-history:before{content:""}.fa-circle-thin:before{content:""}.fa-header:before{content:""}.fa-paragraph:before{content:""}.fa-sliders:before{content:""}.fa-share-alt:before{content:""}.fa-share-alt-square:before{content:""}.fa-bomb:before{content:""}.fa-futbol-o:before,.fa-soccer-ball-o:before{content:""}.fa-tty:before{content:""}.fa-binoculars:before{content:""}.fa-plug:before{content:""}.fa-slideshare:before{content:""}.fa-twitch:before{content:""}.fa-yelp:before{content:""}.fa-newspaper-o:before{content:""}.fa-wifi:before{content:""}.fa-calculator:before{content:""}.fa-paypal:before{content:""}.fa-google-wallet:before{content:""}.fa-cc-visa:before{content:""}.fa-cc-mastercard:before{content:""}.fa-cc-discover:before{content:""}.fa-cc-amex:before{content:""}.fa-cc-paypal:before{content:""}.fa-cc-stripe:before{content:""}.fa-bell-slash:before{content:""}.fa-bell-slash-o:before{content:""}.fa-trash:before{content:""}.fa-copyright:before{content:""}.fa-at:before{content:""}.fa-eyedropper:before{content:""}.fa-paint-brush:before{content:""}.fa-birthday-cake:before{content:""}.fa-area-chart:before{content:""}.fa-pie-chart:before{content:""}.fa-line-chart:before{content:""}.fa-lastfm:before{content:""}.fa-lastfm-square:before{content:""}.fa-toggle-off:before{content:""}.fa-toggle-on:before{content:""}.fa-bicycle:before{content:""}.fa-bus:before{content:""}.fa-ioxhost:before{content:""}.fa-angellist:before{content:""}.fa-cc:before{content:""}.fa-ils:before,.fa-shekel:before,.fa-sheqel:before{content:""}.fa-meanpath:before{content:""}.fa-buysellads:before{content:""}.fa-connectdevelop:before{content:""}.fa-dashcube:before{content:""}.fa-forumbee:before{content:""}.fa-leanpub:before{content:""}.fa-sellsy:before{content:""}.fa-shirtsinbulk:before{content:""}.fa-simplybuilt:before{content:""}.fa-skyatlas:before{content:""}.fa-cart-plus:before{content:""}.fa-cart-arrow-down:before{content:""}.fa-diamond:before{content:""}.fa-ship:before{content:""}.fa-user-secret:before{content:""}.fa-motorcycle:before{content:""}.fa-street-view:before{content:""}.fa-heartbeat:before{content:""}.fa-venus:before{content:""}.fa-mars:before{content:""}.fa-mercury:before{content:""}.fa-intersex:before,.fa-transgender:before{content:""}.fa-transgender-alt:before{content:""}.fa-venus-double:before{content:""}.fa-mars-double:before{content:""}.fa-venus-mars:before{content:""}.fa-mars-stroke:before{content:""}.fa-mars-stroke-v:before{content:""}.fa-mars-stroke-h:before{content:""}.fa-neuter:before{content:""}.fa-genderless:before{content:""}.fa-facebook-official:before{content:""}.fa-pinterest-p:before{content:""}.fa-whatsapp:before{content:""}.fa-server:before{content:""}.fa-user-plus:before{content:""}.fa-user-times:before{content:""}.fa-bed:before,.fa-hotel:before{content:""}.fa-viacoin:before{content:""}.fa-train:before{content:""}.fa-subway:before{content:""}.fa-medium:before{content:""}.fa-y-combinator:before,.fa-yc:before{content:""}.fa-optin-monster:before{content:""}.fa-opencart:before{content:""}.fa-expeditedssl:before{content:""}.fa-battery-4:before,.fa-battery-full:before,.fa-battery:before{content:""}.fa-battery-3:before,.fa-battery-three-quarters:before{content:""}.fa-battery-2:before,.fa-battery-half:before{content:""}.fa-battery-1:before,.fa-battery-quarter:before{content:""}.fa-battery-0:before,.fa-battery-empty:before{content:""}.fa-mouse-pointer:before{content:""}.fa-i-cursor:before{content:""}.fa-object-group:before{content:""}.fa-object-ungroup:before{content:""}.fa-sticky-note:before{content:""}.fa-sticky-note-o:before{content:""}.fa-cc-jcb:before{content:""}.fa-cc-diners-club:before{content:""}.fa-clone:before{content:""}.fa-balance-scale:before{content:""}.fa-hourglass-o:before{content:""}.fa-hourglass-1:before,.fa-hourglass-start:before{content:""}.fa-hourglass-2:before,.fa-hourglass-half:before{content:""}.fa-hourglass-3:before,.fa-hourglass-end:before{content:""}.fa-hourglass:before{content:""}.fa-hand-grab-o:before,.fa-hand-rock-o:before{content:""}.fa-hand-paper-o:before,.fa-hand-stop-o:before{content:""}.fa-hand-scissors-o:before{content:""}.fa-hand-lizard-o:before{content:""}.fa-hand-spock-o:before{content:""}.fa-hand-pointer-o:before{content:""}.fa-hand-peace-o:before{content:""}.fa-trademark:before{content:""}.fa-registered:before{content:""}.fa-creative-commons:before{content:""}.fa-gg:before{content:""}.fa-gg-circle:before{content:""}.fa-tripadvisor:before{content:""}.fa-odnoklassniki:before{content:""}.fa-odnoklassniki-square:before{content:""}.fa-get-pocket:before{content:""}.fa-wikipedia-w:before{content:""}.fa-safari:before{content:""}.fa-chrome:before{content:""}.fa-firefox:before{content:""}.fa-opera:before{content:""}.fa-internet-explorer:before{content:""}.fa-television:before,.fa-tv:before{content:""}.fa-contao:before{content:""}.fa-500px:before{content:""}.fa-amazon:before{content:""}.fa-calendar-plus-o:before{content:""}.fa-calendar-minus-o:before{content:""}.fa-calendar-times-o:before{content:""}.fa-calendar-check-o:before{content:""}.fa-industry:before{content:""}.fa-map-pin:before{content:""}.fa-map-signs:before{content:""}.fa-map-o:before{content:""}.fa-map:before{content:""}.fa-commenting:before{content:""}.fa-commenting-o:before{content:""}.fa-houzz:before{content:""}.fa-vimeo:before{content:""}.fa-black-tie:before{content:""}.fa-fonticons:before{content:""}.fa-reddit-alien:before{content:""}.fa-edge:before{content:""}.fa-credit-card-alt:before{content:""}.fa-codiepie:before{content:""}.fa-modx:before{content:""}.fa-fort-awesome:before{content:""}.fa-usb:before{content:""}.fa-product-hunt:before{content:""}.fa-mixcloud:before{content:""}.fa-scribd:before{content:""}.fa-pause-circle:before{content:""}.fa-pause-circle-o:before{content:""}.fa-stop-circle:before{content:""}.fa-stop-circle-o:before{content:""}.fa-shopping-bag:before{content:""}.fa-shopping-basket:before{content:""}.fa-hashtag:before{content:""}.fa-bluetooth:before{content:""}.fa-bluetooth-b:before{content:""}.fa-percent:before{content:""}.fa-gitlab:before,.icon-gitlab:before{content:""}.fa-wpbeginner:before{content:""}.fa-wpforms:before{content:""}.fa-envira:before{content:""}.fa-universal-access:before{content:""}.fa-wheelchair-alt:before{content:""}.fa-question-circle-o:before{content:""}.fa-blind:before{content:""}.fa-audio-description:before{content:""}.fa-volume-control-phone:before{content:""}.fa-braille:before{content:""}.fa-assistive-listening-systems:before{content:""}.fa-american-sign-language-interpreting:before,.fa-asl-interpreting:before{content:""}.fa-deaf:before,.fa-deafness:before,.fa-hard-of-hearing:before{content:""}.fa-glide:before{content:""}.fa-glide-g:before{content:""}.fa-sign-language:before,.fa-signing:before{content:""}.fa-low-vision:before{content:""}.fa-viadeo:before{content:""}.fa-viadeo-square:before{content:""}.fa-snapchat:before{content:""}.fa-snapchat-ghost:before{content:""}.fa-snapchat-square:before{content:""}.fa-pied-piper:before{content:""}.fa-first-order:before{content:""}.fa-yoast:before{content:""}.fa-themeisle:before{content:""}.fa-google-plus-circle:before,.fa-google-plus-official:before{content:""}.fa-fa:before,.fa-font-awesome:before{content:""}.fa-handshake-o:before{content:""}.fa-envelope-open:before{content:""}.fa-envelope-open-o:before{content:""}.fa-linode:before{content:""}.fa-address-book:before{content:""}.fa-address-book-o:before{content:""}.fa-address-card:before,.fa-vcard:before{content:""}.fa-address-card-o:before,.fa-vcard-o:before{content:""}.fa-user-circle:before{content:""}.fa-user-circle-o:before{content:""}.fa-user-o:before{content:""}.fa-id-badge:before{content:""}.fa-drivers-license:before,.fa-id-card:before{content:""}.fa-drivers-license-o:before,.fa-id-card-o:before{content:""}.fa-quora:before{content:""}.fa-free-code-camp:before{content:""}.fa-telegram:before{content:""}.fa-thermometer-4:before,.fa-thermometer-full:before,.fa-thermometer:before{content:""}.fa-thermometer-3:before,.fa-thermometer-three-quarters:before{content:""}.fa-thermometer-2:before,.fa-thermometer-half:before{content:""}.fa-thermometer-1:before,.fa-thermometer-quarter:before{content:""}.fa-thermometer-0:before,.fa-thermometer-empty:before{content:""}.fa-shower:before{content:""}.fa-bath:before,.fa-bathtub:before,.fa-s15:before{content:""}.fa-podcast:before{content:""}.fa-window-maximize:before{content:""}.fa-window-minimize:before{content:""}.fa-window-restore:before{content:""}.fa-times-rectangle:before,.fa-window-close:before{content:""}.fa-times-rectangle-o:before,.fa-window-close-o:before{content:""}.fa-bandcamp:before{content:""}.fa-grav:before{content:""}.fa-etsy:before{content:""}.fa-imdb:before{content:""}.fa-ravelry:before{content:""}.fa-eercast:before{content:""}.fa-microchip:before{content:""}.fa-snowflake-o:before{content:""}.fa-superpowers:before{content:""}.fa-wpexplorer:before{content:""}.fa-meetup:before{content:""}.sr-only{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0,0,0,0);border:0}.sr-only-focusable:active,.sr-only-focusable:focus{position:static;width:auto;height:auto;margin:0;overflow:visible;clip:auto}.fa,.icon,.rst-content .admonition-title,.rst-content .code-block-caption .headerlink,.rst-content .eqno .headerlink,.rst-content code.download span:first-child,.rst-content dl dt .headerlink,.rst-content h1 .headerlink,.rst-content h2 .headerlink,.rst-content h3 .headerlink,.rst-content h4 .headerlink,.rst-content h5 .headerlink,.rst-content h6 .headerlink,.rst-content p.caption .headerlink,.rst-content p .headerlink,.rst-content table>caption .headerlink,.rst-content tt.download span:first-child,.wy-dropdown .caret,.wy-inline-validate.wy-inline-validate-danger .wy-input-context,.wy-inline-validate.wy-inline-validate-info .wy-input-context,.wy-inline-validate.wy-inline-validate-success .wy-input-context,.wy-inline-validate.wy-inline-validate-warning .wy-input-context,.wy-menu-vertical li.current>a button.toctree-expand,.wy-menu-vertical li.on a button.toctree-expand,.wy-menu-vertical li button.toctree-expand{font-family:inherit}.fa:before,.icon:before,.rst-content .admonition-title:before,.rst-content .code-block-caption .headerlink:before,.rst-content .eqno .headerlink:before,.rst-content code.download span:first-child:before,.rst-content dl dt .headerlink:before,.rst-content h1 .headerlink:before,.rst-content h2 .headerlink:before,.rst-content h3 .headerlink:before,.rst-content h4 .headerlink:before,.rst-content h5 .headerlink:before,.rst-content h6 .headerlink:before,.rst-content p.caption .headerlink:before,.rst-content p .headerlink:before,.rst-content table>caption .headerlink:before,.rst-content tt.download span:first-child:before,.wy-dropdown .caret:before,.wy-inline-validate.wy-inline-validate-danger .wy-input-context:before,.wy-inline-validate.wy-inline-validate-info .wy-input-context:before,.wy-inline-validate.wy-inline-validate-success .wy-input-context:before,.wy-inline-validate.wy-inline-validate-warning .wy-input-context:before,.wy-menu-vertical li.current>a button.toctree-expand:before,.wy-menu-vertical li.on a button.toctree-expand:before,.wy-menu-vertical li button.toctree-expand:before{font-family:FontAwesome;display:inline-block;font-style:normal;font-weight:400;line-height:1;text-decoration:inherit}.rst-content .code-block-caption a .headerlink,.rst-content .eqno a .headerlink,.rst-content a .admonition-title,.rst-content code.download a span:first-child,.rst-content dl dt a .headerlink,.rst-content h1 a .headerlink,.rst-content h2 a .headerlink,.rst-content h3 a .headerlink,.rst-content h4 a .headerlink,.rst-content h5 a .headerlink,.rst-content h6 a .headerlink,.rst-content p.caption a .headerlink,.rst-content p a .headerlink,.rst-content table>caption a .headerlink,.rst-content tt.download a span:first-child,.wy-menu-vertical li.current>a button.toctree-expand,.wy-menu-vertical li.on a button.toctree-expand,.wy-menu-vertical li a button.toctree-expand,a .fa,a .icon,a .rst-content .admonition-title,a .rst-content .code-block-caption .headerlink,a .rst-content .eqno .headerlink,a .rst-content code.download span:first-child,a .rst-content dl dt .headerlink,a .rst-content h1 .headerlink,a .rst-content h2 .headerlink,a .rst-content h3 .headerlink,a .rst-content h4 .headerlink,a .rst-content h5 .headerlink,a .rst-content h6 .headerlink,a .rst-content p.caption .headerlink,a .rst-content p .headerlink,a .rst-content table>caption .headerlink,a .rst-content tt.download span:first-child,a .wy-menu-vertical li button.toctree-expand{display:inline-block;text-decoration:inherit}.btn .fa,.btn .icon,.btn .rst-content .admonition-title,.btn .rst-content .code-block-caption .headerlink,.btn .rst-content .eqno .headerlink,.btn .rst-content code.download span:first-child,.btn .rst-content dl dt .headerlink,.btn .rst-content h1 .headerlink,.btn .rst-content h2 .headerlink,.btn .rst-content h3 .headerlink,.btn .rst-content h4 .headerlink,.btn .rst-content h5 .headerlink,.btn .rst-content h6 .headerlink,.btn .rst-content p .headerlink,.btn .rst-content table>caption .headerlink,.btn .rst-content tt.download span:first-child,.btn .wy-menu-vertical li.current>a button.toctree-expand,.btn .wy-menu-vertical li.on a button.toctree-expand,.btn .wy-menu-vertical li button.toctree-expand,.nav .fa,.nav .icon,.nav .rst-content .admonition-title,.nav .rst-content .code-block-caption .headerlink,.nav .rst-content .eqno .headerlink,.nav .rst-content code.download span:first-child,.nav .rst-content dl dt .headerlink,.nav .rst-content h1 .headerlink,.nav .rst-content h2 .headerlink,.nav .rst-content h3 .headerlink,.nav .rst-content h4 .headerlink,.nav .rst-content h5 .headerlink,.nav .rst-content h6 .headerlink,.nav .rst-content p .headerlink,.nav .rst-content table>caption .headerlink,.nav .rst-content tt.download span:first-child,.nav .wy-menu-vertical li.current>a button.toctree-expand,.nav .wy-menu-vertical li.on a button.toctree-expand,.nav .wy-menu-vertical li button.toctree-expand,.rst-content .btn .admonition-title,.rst-content .code-block-caption .btn .headerlink,.rst-content .code-block-caption .nav .headerlink,.rst-content .eqno .btn .headerlink,.rst-content .eqno .nav .headerlink,.rst-content .nav .admonition-title,.rst-content code.download .btn span:first-child,.rst-content code.download .nav span:first-child,.rst-content dl dt .btn .headerlink,.rst-content dl dt .nav .headerlink,.rst-content h1 .btn .headerlink,.rst-content h1 .nav .headerlink,.rst-content h2 .btn .headerlink,.rst-content h2 .nav .headerlink,.rst-content h3 .btn .headerlink,.rst-content h3 .nav .headerlink,.rst-content h4 .btn .headerlink,.rst-content h4 .nav .headerlink,.rst-content h5 .btn .headerlink,.rst-content h5 .nav .headerlink,.rst-content h6 .btn .headerlink,.rst-content h6 .nav .headerlink,.rst-content p .btn .headerlink,.rst-content p .nav .headerlink,.rst-content table>caption .btn .headerlink,.rst-content table>caption .nav .headerlink,.rst-content tt.download .btn span:first-child,.rst-content tt.download .nav span:first-child,.wy-menu-vertical li .btn button.toctree-expand,.wy-menu-vertical li.current>a .btn button.toctree-expand,.wy-menu-vertical li.current>a .nav button.toctree-expand,.wy-menu-vertical li .nav button.toctree-expand,.wy-menu-vertical li.on a .btn button.toctree-expand,.wy-menu-vertical li.on a .nav button.toctree-expand{display:inline}.btn .fa-large.icon,.btn .fa.fa-large,.btn .rst-content .code-block-caption .fa-large.headerlink,.btn .rst-content .eqno .fa-large.headerlink,.btn .rst-content .fa-large.admonition-title,.btn .rst-content code.download span.fa-large:first-child,.btn .rst-content dl dt .fa-large.headerlink,.btn .rst-content h1 .fa-large.headerlink,.btn .rst-content h2 .fa-large.headerlink,.btn .rst-content h3 .fa-large.headerlink,.btn .rst-content h4 .fa-large.headerlink,.btn .rst-content h5 .fa-large.headerlink,.btn .rst-content h6 .fa-large.headerlink,.btn .rst-content p .fa-large.headerlink,.btn .rst-content table>caption .fa-large.headerlink,.btn .rst-content tt.download span.fa-large:first-child,.btn .wy-menu-vertical li button.fa-large.toctree-expand,.nav .fa-large.icon,.nav .fa.fa-large,.nav .rst-content .code-block-caption .fa-large.headerlink,.nav .rst-content .eqno .fa-large.headerlink,.nav .rst-content .fa-large.admonition-title,.nav .rst-content code.download span.fa-large:first-child,.nav .rst-content dl dt .fa-large.headerlink,.nav .rst-content h1 .fa-large.headerlink,.nav .rst-content h2 .fa-large.headerlink,.nav .rst-content h3 .fa-large.headerlink,.nav .rst-content h4 .fa-large.headerlink,.nav .rst-content h5 .fa-large.headerlink,.nav .rst-content h6 .fa-large.headerlink,.nav .rst-content p .fa-large.headerlink,.nav .rst-content table>caption .fa-large.headerlink,.nav .rst-content tt.download span.fa-large:first-child,.nav .wy-menu-vertical li button.fa-large.toctree-expand,.rst-content .btn .fa-large.admonition-title,.rst-content .code-block-caption .btn .fa-large.headerlink,.rst-content .code-block-caption .nav .fa-large.headerlink,.rst-content .eqno .btn .fa-large.headerlink,.rst-content .eqno .nav .fa-large.headerlink,.rst-content .nav .fa-large.admonition-title,.rst-content code.download .btn span.fa-large:first-child,.rst-content code.download .nav span.fa-large:first-child,.rst-content dl dt .btn .fa-large.headerlink,.rst-content dl dt .nav .fa-large.headerlink,.rst-content h1 .btn .fa-large.headerlink,.rst-content h1 .nav .fa-large.headerlink,.rst-content h2 .btn .fa-large.headerlink,.rst-content h2 .nav .fa-large.headerlink,.rst-content h3 .btn .fa-large.headerlink,.rst-content h3 .nav .fa-large.headerlink,.rst-content h4 .btn .fa-large.headerlink,.rst-content h4 .nav .fa-large.headerlink,.rst-content h5 .btn .fa-large.headerlink,.rst-content h5 .nav .fa-large.headerlink,.rst-content h6 .btn .fa-large.headerlink,.rst-content h6 .nav .fa-large.headerlink,.rst-content p .btn .fa-large.headerlink,.rst-content p .nav .fa-large.headerlink,.rst-content table>caption .btn .fa-large.headerlink,.rst-content table>caption .nav .fa-large.headerlink,.rst-content tt.download .btn span.fa-large:first-child,.rst-content tt.download .nav span.fa-large:first-child,.wy-menu-vertical li .btn button.fa-large.toctree-expand,.wy-menu-vertical li .nav button.fa-large.toctree-expand{line-height:.9em}.btn .fa-spin.icon,.btn .fa.fa-spin,.btn .rst-content .code-block-caption .fa-spin.headerlink,.btn .rst-content .eqno .fa-spin.headerlink,.btn .rst-content .fa-spin.admonition-title,.btn .rst-content code.download span.fa-spin:first-child,.btn .rst-content dl dt .fa-spin.headerlink,.btn .rst-content h1 .fa-spin.headerlink,.btn .rst-content h2 .fa-spin.headerlink,.btn .rst-content h3 .fa-spin.headerlink,.btn .rst-content h4 .fa-spin.headerlink,.btn .rst-content h5 .fa-spin.headerlink,.btn .rst-content h6 .fa-spin.headerlink,.btn .rst-content p .fa-spin.headerlink,.btn .rst-content table>caption .fa-spin.headerlink,.btn .rst-content tt.download span.fa-spin:first-child,.btn .wy-menu-vertical li button.fa-spin.toctree-expand,.nav .fa-spin.icon,.nav .fa.fa-spin,.nav .rst-content .code-block-caption .fa-spin.headerlink,.nav .rst-content .eqno .fa-spin.headerlink,.nav .rst-content .fa-spin.admonition-title,.nav .rst-content code.download span.fa-spin:first-child,.nav .rst-content dl dt .fa-spin.headerlink,.nav .rst-content h1 .fa-spin.headerlink,.nav .rst-content h2 .fa-spin.headerlink,.nav .rst-content h3 .fa-spin.headerlink,.nav .rst-content h4 .fa-spin.headerlink,.nav .rst-content h5 .fa-spin.headerlink,.nav .rst-content h6 .fa-spin.headerlink,.nav .rst-content p .fa-spin.headerlink,.nav .rst-content table>caption .fa-spin.headerlink,.nav .rst-content tt.download span.fa-spin:first-child,.nav .wy-menu-vertical li button.fa-spin.toctree-expand,.rst-content .btn .fa-spin.admonition-title,.rst-content .code-block-caption .btn .fa-spin.headerlink,.rst-content .code-block-caption .nav .fa-spin.headerlink,.rst-content .eqno .btn .fa-spin.headerlink,.rst-content .eqno .nav .fa-spin.headerlink,.rst-content .nav .fa-spin.admonition-title,.rst-content code.download .btn span.fa-spin:first-child,.rst-content code.download .nav span.fa-spin:first-child,.rst-content dl dt .btn .fa-spin.headerlink,.rst-content dl dt .nav .fa-spin.headerlink,.rst-content h1 .btn .fa-spin.headerlink,.rst-content h1 .nav .fa-spin.headerlink,.rst-content h2 .btn .fa-spin.headerlink,.rst-content h2 .nav .fa-spin.headerlink,.rst-content h3 .btn .fa-spin.headerlink,.rst-content h3 .nav .fa-spin.headerlink,.rst-content h4 .btn .fa-spin.headerlink,.rst-content h4 .nav .fa-spin.headerlink,.rst-content h5 .btn .fa-spin.headerlink,.rst-content h5 .nav .fa-spin.headerlink,.rst-content h6 .btn .fa-spin.headerlink,.rst-content h6 .nav .fa-spin.headerlink,.rst-content p .btn .fa-spin.headerlink,.rst-content p .nav .fa-spin.headerlink,.rst-content table>caption .btn .fa-spin.headerlink,.rst-content table>caption .nav .fa-spin.headerlink,.rst-content tt.download .btn span.fa-spin:first-child,.rst-content tt.download .nav span.fa-spin:first-child,.wy-menu-vertical li .btn button.fa-spin.toctree-expand,.wy-menu-vertical li .nav button.fa-spin.toctree-expand{display:inline-block}.btn.fa:before,.btn.icon:before,.rst-content .btn.admonition-title:before,.rst-content .code-block-caption .btn.headerlink:before,.rst-content .eqno .btn.headerlink:before,.rst-content code.download span.btn:first-child:before,.rst-content dl dt .btn.headerlink:before,.rst-content h1 .btn.headerlink:before,.rst-content h2 .btn.headerlink:before,.rst-content h3 .btn.headerlink:before,.rst-content h4 .btn.headerlink:before,.rst-content h5 .btn.headerlink:before,.rst-content h6 .btn.headerlink:before,.rst-content p .btn.headerlink:before,.rst-content table>caption .btn.headerlink:before,.rst-content tt.download span.btn:first-child:before,.wy-menu-vertical li button.btn.toctree-expand:before{opacity:.5;-webkit-transition:opacity .05s ease-in;-moz-transition:opacity .05s ease-in;transition:opacity .05s ease-in}.btn.fa:hover:before,.btn.icon:hover:before,.rst-content .btn.admonition-title:hover:before,.rst-content .code-block-caption .btn.headerlink:hover:before,.rst-content .eqno .btn.headerlink:hover:before,.rst-content code.download span.btn:first-child:hover:before,.rst-content dl dt .btn.headerlink:hover:before,.rst-content h1 .btn.headerlink:hover:before,.rst-content h2 .btn.headerlink:hover:before,.rst-content h3 .btn.headerlink:hover:before,.rst-content h4 .btn.headerlink:hover:before,.rst-content h5 .btn.headerlink:hover:before,.rst-content h6 .btn.headerlink:hover:before,.rst-content p .btn.headerlink:hover:before,.rst-content table>caption .btn.headerlink:hover:before,.rst-content tt.download span.btn:first-child:hover:before,.wy-menu-vertical li button.btn.toctree-expand:hover:before{opacity:1}.btn-mini .fa:before,.btn-mini .icon:before,.btn-mini .rst-content .admonition-title:before,.btn-mini .rst-content .code-block-caption .headerlink:before,.btn-mini .rst-content .eqno .headerlink:before,.btn-mini .rst-content code.download span:first-child:before,.btn-mini .rst-content dl dt .headerlink:before,.btn-mini .rst-content h1 .headerlink:before,.btn-mini .rst-content h2 .headerlink:before,.btn-mini .rst-content h3 .headerlink:before,.btn-mini .rst-content h4 .headerlink:before,.btn-mini .rst-content h5 .headerlink:before,.btn-mini .rst-content h6 .headerlink:before,.btn-mini .rst-content p .headerlink:before,.btn-mini .rst-content table>caption .headerlink:before,.btn-mini .rst-content tt.download span:first-child:before,.btn-mini .wy-menu-vertical li button.toctree-expand:before,.rst-content .btn-mini .admonition-title:before,.rst-content .code-block-caption .btn-mini .headerlink:before,.rst-content .eqno .btn-mini .headerlink:before,.rst-content code.download .btn-mini span:first-child:before,.rst-content dl dt .btn-mini .headerlink:before,.rst-content h1 .btn-mini .headerlink:before,.rst-content h2 .btn-mini .headerlink:before,.rst-content h3 .btn-mini .headerlink:before,.rst-content h4 .btn-mini .headerlink:before,.rst-content h5 .btn-mini .headerlink:before,.rst-content h6 .btn-mini .headerlink:before,.rst-content p .btn-mini .headerlink:before,.rst-content table>caption .btn-mini .headerlink:before,.rst-content tt.download .btn-mini span:first-child:before,.wy-menu-vertical li .btn-mini button.toctree-expand:before{font-size:14px;vertical-align:-15%}.rst-content .admonition,.rst-content .admonition-todo,.rst-content .attention,.rst-content .caution,.rst-content .danger,.rst-content .error,.rst-content .hint,.rst-content .important,.rst-content .note,.rst-content .seealso,.rst-content .tip,.rst-content .warning,.wy-alert{padding:12px;line-height:24px;margin-bottom:24px;background:#e7f2fa}.rst-content .admonition-title,.wy-alert-title{font-weight:700;display:block;color:#fff;background:#6ab0de;padding:6px 12px;margin:-12px -12px 12px}.rst-content .danger,.rst-content .error,.rst-content .wy-alert-danger.admonition,.rst-content .wy-alert-danger.admonition-todo,.rst-content .wy-alert-danger.attention,.rst-content .wy-alert-danger.caution,.rst-content .wy-alert-danger.hint,.rst-content .wy-alert-danger.important,.rst-content .wy-alert-danger.note,.rst-content .wy-alert-danger.seealso,.rst-content .wy-alert-danger.tip,.rst-content .wy-alert-danger.warning,.wy-alert.wy-alert-danger{background:#fdf3f2}.rst-content .danger .admonition-title,.rst-content .danger .wy-alert-title,.rst-content .error .admonition-title,.rst-content .error .wy-alert-title,.rst-content .wy-alert-danger.admonition-todo .admonition-title,.rst-content .wy-alert-danger.admonition-todo .wy-alert-title,.rst-content .wy-alert-danger.admonition .admonition-title,.rst-content .wy-alert-danger.admonition .wy-alert-title,.rst-content .wy-alert-danger.attention .admonition-title,.rst-content .wy-alert-danger.attention .wy-alert-title,.rst-content .wy-alert-danger.caution .admonition-title,.rst-content .wy-alert-danger.caution .wy-alert-title,.rst-content .wy-alert-danger.hint .admonition-title,.rst-content .wy-alert-danger.hint .wy-alert-title,.rst-content .wy-alert-danger.important .admonition-title,.rst-content .wy-alert-danger.important .wy-alert-title,.rst-content .wy-alert-danger.note .admonition-title,.rst-content .wy-alert-danger.note .wy-alert-title,.rst-content .wy-alert-danger.seealso .admonition-title,.rst-content .wy-alert-danger.seealso .wy-alert-title,.rst-content .wy-alert-danger.tip .admonition-title,.rst-content .wy-alert-danger.tip .wy-alert-title,.rst-content .wy-alert-danger.warning .admonition-title,.rst-content .wy-alert-danger.warning .wy-alert-title,.rst-content .wy-alert.wy-alert-danger .admonition-title,.wy-alert.wy-alert-danger .rst-content .admonition-title,.wy-alert.wy-alert-danger .wy-alert-title{background:#f29f97}.rst-content .admonition-todo,.rst-content .attention,.rst-content .caution,.rst-content .warning,.rst-content .wy-alert-warning.admonition,.rst-content .wy-alert-warning.danger,.rst-content .wy-alert-warning.error,.rst-content .wy-alert-warning.hint,.rst-content .wy-alert-warning.important,.rst-content .wy-alert-warning.note,.rst-content .wy-alert-warning.seealso,.rst-content .wy-alert-warning.tip,.wy-alert.wy-alert-warning{background:#ffedcc}.rst-content .admonition-todo .admonition-title,.rst-content .admonition-todo .wy-alert-title,.rst-content .attention .admonition-title,.rst-content .attention .wy-alert-title,.rst-content .caution .admonition-title,.rst-content .caution .wy-alert-title,.rst-content .warning .admonition-title,.rst-content .warning .wy-alert-title,.rst-content .wy-alert-warning.admonition .admonition-title,.rst-content .wy-alert-warning.admonition .wy-alert-title,.rst-content .wy-alert-warning.danger .admonition-title,.rst-content .wy-alert-warning.danger .wy-alert-title,.rst-content .wy-alert-warning.error .admonition-title,.rst-content .wy-alert-warning.error .wy-alert-title,.rst-content .wy-alert-warning.hint .admonition-title,.rst-content .wy-alert-warning.hint .wy-alert-title,.rst-content .wy-alert-warning.important .admonition-title,.rst-content .wy-alert-warning.important .wy-alert-title,.rst-content .wy-alert-warning.note .admonition-title,.rst-content .wy-alert-warning.note .wy-alert-title,.rst-content .wy-alert-warning.seealso .admonition-title,.rst-content .wy-alert-warning.seealso .wy-alert-title,.rst-content .wy-alert-warning.tip .admonition-title,.rst-content .wy-alert-warning.tip .wy-alert-title,.rst-content .wy-alert.wy-alert-warning .admonition-title,.wy-alert.wy-alert-warning .rst-content .admonition-title,.wy-alert.wy-alert-warning .wy-alert-title{background:#f0b37e}.rst-content .note,.rst-content .seealso,.rst-content .wy-alert-info.admonition,.rst-content .wy-alert-info.admonition-todo,.rst-content .wy-alert-info.attention,.rst-content .wy-alert-info.caution,.rst-content .wy-alert-info.danger,.rst-content .wy-alert-info.error,.rst-content .wy-alert-info.hint,.rst-content .wy-alert-info.important,.rst-content .wy-alert-info.tip,.rst-content .wy-alert-info.warning,.wy-alert.wy-alert-info{background:#e7f2fa}.rst-content .note .admonition-title,.rst-content .note .wy-alert-title,.rst-content .seealso .admonition-title,.rst-content .seealso .wy-alert-title,.rst-content .wy-alert-info.admonition-todo .admonition-title,.rst-content .wy-alert-info.admonition-todo .wy-alert-title,.rst-content .wy-alert-info.admonition .admonition-title,.rst-content .wy-alert-info.admonition .wy-alert-title,.rst-content .wy-alert-info.attention .admonition-title,.rst-content .wy-alert-info.attention .wy-alert-title,.rst-content .wy-alert-info.caution .admonition-title,.rst-content .wy-alert-info.caution .wy-alert-title,.rst-content .wy-alert-info.danger .admonition-title,.rst-content .wy-alert-info.danger .wy-alert-title,.rst-content .wy-alert-info.error .admonition-title,.rst-content .wy-alert-info.error .wy-alert-title,.rst-content .wy-alert-info.hint .admonition-title,.rst-content .wy-alert-info.hint .wy-alert-title,.rst-content .wy-alert-info.important .admonition-title,.rst-content .wy-alert-info.important .wy-alert-title,.rst-content .wy-alert-info.tip .admonition-title,.rst-content .wy-alert-info.tip .wy-alert-title,.rst-content .wy-alert-info.warning .admonition-title,.rst-content .wy-alert-info.warning .wy-alert-title,.rst-content .wy-alert.wy-alert-info .admonition-title,.wy-alert.wy-alert-info .rst-content .admonition-title,.wy-alert.wy-alert-info .wy-alert-title{background:#6ab0de}.rst-content .hint,.rst-content .important,.rst-content .tip,.rst-content .wy-alert-success.admonition,.rst-content .wy-alert-success.admonition-todo,.rst-content .wy-alert-success.attention,.rst-content .wy-alert-success.caution,.rst-content .wy-alert-success.danger,.rst-content .wy-alert-success.error,.rst-content .wy-alert-success.note,.rst-content .wy-alert-success.seealso,.rst-content .wy-alert-success.warning,.wy-alert.wy-alert-success{background:#dbfaf4}.rst-content .hint .admonition-title,.rst-content .hint .wy-alert-title,.rst-content .important .admonition-title,.rst-content .important .wy-alert-title,.rst-content .tip .admonition-title,.rst-content .tip .wy-alert-title,.rst-content .wy-alert-success.admonition-todo .admonition-title,.rst-content .wy-alert-success.admonition-todo .wy-alert-title,.rst-content .wy-alert-success.admonition .admonition-title,.rst-content .wy-alert-success.admonition .wy-alert-title,.rst-content .wy-alert-success.attention .admonition-title,.rst-content .wy-alert-success.attention .wy-alert-title,.rst-content .wy-alert-success.caution .admonition-title,.rst-content .wy-alert-success.caution .wy-alert-title,.rst-content .wy-alert-success.danger .admonition-title,.rst-content .wy-alert-success.danger .wy-alert-title,.rst-content .wy-alert-success.error .admonition-title,.rst-content .wy-alert-success.error .wy-alert-title,.rst-content .wy-alert-success.note .admonition-title,.rst-content .wy-alert-success.note .wy-alert-title,.rst-content .wy-alert-success.seealso .admonition-title,.rst-content .wy-alert-success.seealso .wy-alert-title,.rst-content .wy-alert-success.warning .admonition-title,.rst-content .wy-alert-success.warning .wy-alert-title,.rst-content .wy-alert.wy-alert-success .admonition-title,.wy-alert.wy-alert-success .rst-content .admonition-title,.wy-alert.wy-alert-success .wy-alert-title{background:#1abc9c}.rst-content .wy-alert-neutral.admonition,.rst-content .wy-alert-neutral.admonition-todo,.rst-content .wy-alert-neutral.attention,.rst-content .wy-alert-neutral.caution,.rst-content .wy-alert-neutral.danger,.rst-content .wy-alert-neutral.error,.rst-content .wy-alert-neutral.hint,.rst-content .wy-alert-neutral.important,.rst-content .wy-alert-neutral.note,.rst-content .wy-alert-neutral.seealso,.rst-content .wy-alert-neutral.tip,.rst-content .wy-alert-neutral.warning,.wy-alert.wy-alert-neutral{background:#f3f6f6}.rst-content .wy-alert-neutral.admonition-todo .admonition-title,.rst-content .wy-alert-neutral.admonition-todo .wy-alert-title,.rst-content .wy-alert-neutral.admonition .admonition-title,.rst-content .wy-alert-neutral.admonition .wy-alert-title,.rst-content .wy-alert-neutral.attention .admonition-title,.rst-content .wy-alert-neutral.attention .wy-alert-title,.rst-content .wy-alert-neutral.caution .admonition-title,.rst-content .wy-alert-neutral.caution .wy-alert-title,.rst-content .wy-alert-neutral.danger .admonition-title,.rst-content .wy-alert-neutral.danger .wy-alert-title,.rst-content .wy-alert-neutral.error .admonition-title,.rst-content .wy-alert-neutral.error .wy-alert-title,.rst-content .wy-alert-neutral.hint .admonition-title,.rst-content .wy-alert-neutral.hint .wy-alert-title,.rst-content .wy-alert-neutral.important .admonition-title,.rst-content .wy-alert-neutral.important .wy-alert-title,.rst-content .wy-alert-neutral.note .admonition-title,.rst-content .wy-alert-neutral.note .wy-alert-title,.rst-content .wy-alert-neutral.seealso .admonition-title,.rst-content .wy-alert-neutral.seealso .wy-alert-title,.rst-content .wy-alert-neutral.tip .admonition-title,.rst-content .wy-alert-neutral.tip .wy-alert-title,.rst-content .wy-alert-neutral.warning .admonition-title,.rst-content .wy-alert-neutral.warning .wy-alert-title,.rst-content .wy-alert.wy-alert-neutral .admonition-title,.wy-alert.wy-alert-neutral .rst-content .admonition-title,.wy-alert.wy-alert-neutral .wy-alert-title{color:#404040;background:#e1e4e5}.rst-content .wy-alert-neutral.admonition-todo a,.rst-content .wy-alert-neutral.admonition a,.rst-content .wy-alert-neutral.attention a,.rst-content .wy-alert-neutral.caution a,.rst-content .wy-alert-neutral.danger a,.rst-content .wy-alert-neutral.error a,.rst-content .wy-alert-neutral.hint a,.rst-content .wy-alert-neutral.important a,.rst-content .wy-alert-neutral.note a,.rst-content .wy-alert-neutral.seealso a,.rst-content .wy-alert-neutral.tip a,.rst-content .wy-alert-neutral.warning a,.wy-alert.wy-alert-neutral a{color:#2980b9}.rst-content .admonition-todo p:last-child,.rst-content .admonition p:last-child,.rst-content .attention p:last-child,.rst-content .caution p:last-child,.rst-content .danger p:last-child,.rst-content .error p:last-child,.rst-content .hint p:last-child,.rst-content .important p:last-child,.rst-content .note p:last-child,.rst-content .seealso p:last-child,.rst-content .tip p:last-child,.rst-content .warning p:last-child,.wy-alert p:last-child{margin-bottom:0}.wy-tray-container{position:fixed;bottom:0;left:0;z-index:600}.wy-tray-container li{display:block;width:300px;background:transparent;color:#fff;text-align:center;box-shadow:0 5px 5px 0 rgba(0,0,0,.1);padding:0 24px;min-width:20%;opacity:0;height:0;line-height:56px;overflow:hidden;-webkit-transition:all .3s ease-in;-moz-transition:all .3s ease-in;transition:all .3s ease-in}.wy-tray-container li.wy-tray-item-success{background:#27ae60}.wy-tray-container li.wy-tray-item-info{background:#2980b9}.wy-tray-container li.wy-tray-item-warning{background:#e67e22}.wy-tray-container li.wy-tray-item-danger{background:#e74c3c}.wy-tray-container li.on{opacity:1;height:56px}@media screen and (max-width:768px){.wy-tray-container{bottom:auto;top:0;width:100%}.wy-tray-container li{width:100%}}button{font-size:100%;margin:0;vertical-align:baseline;*vertical-align:middle;cursor:pointer;line-height:normal;-webkit-appearance:button;*overflow:visible}button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}button[disabled]{cursor:default}.btn{display:inline-block;border-radius:2px;line-height:normal;white-space:nowrap;text-align:center;cursor:pointer;font-size:100%;padding:6px 12px 8px;color:#fff;border:1px solid rgba(0,0,0,.1);background-color:#27ae60;text-decoration:none;font-weight:400;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;box-shadow:inset 0 1px 2px -1px hsla(0,0%,100%,.5),inset 0 -2px 0 0 rgba(0,0,0,.1);outline-none:false;vertical-align:middle;*display:inline;zoom:1;-webkit-user-drag:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none;-webkit-transition:all .1s linear;-moz-transition:all .1s linear;transition:all .1s linear}.btn-hover{background:#2e8ece;color:#fff}.btn:hover{background:#2cc36b;color:#fff}.btn:focus{background:#2cc36b;outline:0}.btn:active{box-shadow:inset 0 -1px 0 0 rgba(0,0,0,.05),inset 0 2px 0 0 rgba(0,0,0,.1);padding:8px 12px 6px}.btn:visited{color:#fff}.btn-disabled,.btn-disabled:active,.btn-disabled:focus,.btn-disabled:hover,.btn:disabled{background-image:none;filter:progid:DXImageTransform.Microsoft.gradient(enabled = false);filter:alpha(opacity=40);opacity:.4;cursor:not-allowed;box-shadow:none}.btn::-moz-focus-inner{padding:0;border:0}.btn-small{font-size:80%}.btn-info{background-color:#2980b9!important}.btn-info:hover{background-color:#2e8ece!important}.btn-neutral{background-color:#f3f6f6!important;color:#404040!important}.btn-neutral:hover{background-color:#e5ebeb!important;color:#404040}.btn-neutral:visited{color:#404040!important}.btn-success{background-color:#27ae60!important}.btn-success:hover{background-color:#295!important}.btn-danger{background-color:#e74c3c!important}.btn-danger:hover{background-color:#ea6153!important}.btn-warning{background-color:#e67e22!important}.btn-warning:hover{background-color:#e98b39!important}.btn-invert{background-color:#222}.btn-invert:hover{background-color:#2f2f2f!important}.btn-link{background-color:transparent!important;color:#2980b9;box-shadow:none;border-color:transparent!important}.btn-link:active,.btn-link:hover{background-color:transparent!important;color:#409ad5!important;box-shadow:none}.btn-link:visited{color:#9b59b6}.wy-btn-group .btn,.wy-control .btn{vertical-align:middle}.wy-btn-group{margin-bottom:24px;*zoom:1}.wy-btn-group:after,.wy-btn-group:before{display:table;content:""}.wy-btn-group:after{clear:both}.wy-dropdown{position:relative;display:inline-block}.wy-dropdown-active .wy-dropdown-menu{display:block}.wy-dropdown-menu{position:absolute;left:0;display:none;float:left;top:100%;min-width:100%;background:#fcfcfc;z-index:100;border:1px solid #cfd7dd;box-shadow:0 2px 2px 0 rgba(0,0,0,.1);padding:12px}.wy-dropdown-menu>dd>a{display:block;clear:both;color:#404040;white-space:nowrap;font-size:90%;padding:0 12px;cursor:pointer}.wy-dropdown-menu>dd>a:hover{background:#2980b9;color:#fff}.wy-dropdown-menu>dd.divider{border-top:1px solid #cfd7dd;margin:6px 0}.wy-dropdown-menu>dd.search{padding-bottom:12px}.wy-dropdown-menu>dd.search input[type=search]{width:100%}.wy-dropdown-menu>dd.call-to-action{background:#e3e3e3;text-transform:uppercase;font-weight:500;font-size:80%}.wy-dropdown-menu>dd.call-to-action:hover{background:#e3e3e3}.wy-dropdown-menu>dd.call-to-action .btn{color:#fff}.wy-dropdown.wy-dropdown-up .wy-dropdown-menu{bottom:100%;top:auto;left:auto;right:0}.wy-dropdown.wy-dropdown-bubble .wy-dropdown-menu{background:#fcfcfc;margin-top:2px}.wy-dropdown.wy-dropdown-bubble .wy-dropdown-menu a{padding:6px 12px}.wy-dropdown.wy-dropdown-bubble .wy-dropdown-menu a:hover{background:#2980b9;color:#fff}.wy-dropdown.wy-dropdown-left .wy-dropdown-menu{right:0;left:auto;text-align:right}.wy-dropdown-arrow:before{content:" ";border-bottom:5px solid #f5f5f5;border-left:5px solid transparent;border-right:5px solid transparent;position:absolute;display:block;top:-4px;left:50%;margin-left:-3px}.wy-dropdown-arrow.wy-dropdown-arrow-left:before{left:11px}.wy-form-stacked select{display:block}.wy-form-aligned .wy-help-inline,.wy-form-aligned input,.wy-form-aligned label,.wy-form-aligned select,.wy-form-aligned textarea{display:inline-block;*display:inline;*zoom:1;vertical-align:middle}.wy-form-aligned .wy-control-group>label{display:inline-block;vertical-align:middle;width:10em;margin:6px 12px 0 0;float:left}.wy-form-aligned .wy-control{float:left}.wy-form-aligned .wy-control label{display:block}.wy-form-aligned .wy-control select{margin-top:6px}fieldset{margin:0}fieldset,legend{border:0;padding:0}legend{width:100%;white-space:normal;margin-bottom:24px;font-size:150%;*margin-left:-7px}label,legend{display:block}label{margin:0 0 .3125em;color:#333;font-size:90%}input,select,textarea{font-size:100%;margin:0;vertical-align:baseline;*vertical-align:middle}.wy-control-group{margin-bottom:24px;max-width:1200px;margin-left:auto;margin-right:auto;*zoom:1}.wy-control-group:after,.wy-control-group:before{display:table;content:""}.wy-control-group:after{clear:both}.wy-control-group.wy-control-group-required>label:after{content:" *";color:#e74c3c}.wy-control-group .wy-form-full,.wy-control-group .wy-form-halves,.wy-control-group .wy-form-thirds{padding-bottom:12px}.wy-control-group .wy-form-full input[type=color],.wy-control-group .wy-form-full input[type=date],.wy-control-group .wy-form-full input[type=datetime-local],.wy-control-group .wy-form-full input[type=datetime],.wy-control-group .wy-form-full input[type=email],.wy-control-group .wy-form-full input[type=month],.wy-control-group .wy-form-full input[type=number],.wy-control-group .wy-form-full input[type=password],.wy-control-group .wy-form-full input[type=search],.wy-control-group .wy-form-full input[type=tel],.wy-control-group .wy-form-full input[type=text],.wy-control-group .wy-form-full input[type=time],.wy-control-group .wy-form-full input[type=url],.wy-control-group .wy-form-full input[type=week],.wy-control-group .wy-form-full select,.wy-control-group .wy-form-halves input[type=color],.wy-control-group .wy-form-halves input[type=date],.wy-control-group .wy-form-halves input[type=datetime-local],.wy-control-group .wy-form-halves input[type=datetime],.wy-control-group .wy-form-halves input[type=email],.wy-control-group .wy-form-halves input[type=month],.wy-control-group .wy-form-halves input[type=number],.wy-control-group .wy-form-halves input[type=password],.wy-control-group .wy-form-halves input[type=search],.wy-control-group .wy-form-halves input[type=tel],.wy-control-group .wy-form-halves input[type=text],.wy-control-group .wy-form-halves input[type=time],.wy-control-group .wy-form-halves input[type=url],.wy-control-group .wy-form-halves input[type=week],.wy-control-group .wy-form-halves select,.wy-control-group .wy-form-thirds input[type=color],.wy-control-group .wy-form-thirds input[type=date],.wy-control-group .wy-form-thirds input[type=datetime-local],.wy-control-group .wy-form-thirds input[type=datetime],.wy-control-group .wy-form-thirds input[type=email],.wy-control-group .wy-form-thirds input[type=month],.wy-control-group .wy-form-thirds input[type=number],.wy-control-group .wy-form-thirds input[type=password],.wy-control-group .wy-form-thirds input[type=search],.wy-control-group .wy-form-thirds input[type=tel],.wy-control-group .wy-form-thirds input[type=text],.wy-control-group .wy-form-thirds input[type=time],.wy-control-group .wy-form-thirds input[type=url],.wy-control-group .wy-form-thirds input[type=week],.wy-control-group .wy-form-thirds select{width:100%}.wy-control-group .wy-form-full{float:left;display:block;width:100%;margin-right:0}.wy-control-group .wy-form-full:last-child{margin-right:0}.wy-control-group .wy-form-halves{float:left;display:block;margin-right:2.35765%;width:48.82117%}.wy-control-group .wy-form-halves:last-child,.wy-control-group .wy-form-halves:nth-of-type(2n){margin-right:0}.wy-control-group .wy-form-halves:nth-of-type(odd){clear:left}.wy-control-group .wy-form-thirds{float:left;display:block;margin-right:2.35765%;width:31.76157%}.wy-control-group .wy-form-thirds:last-child,.wy-control-group .wy-form-thirds:nth-of-type(3n){margin-right:0}.wy-control-group .wy-form-thirds:nth-of-type(3n+1){clear:left}.wy-control-group.wy-control-group-no-input .wy-control,.wy-control-no-input{margin:6px 0 0;font-size:90%}.wy-control-no-input{display:inline-block}.wy-control-group.fluid-input input[type=color],.wy-control-group.fluid-input input[type=date],.wy-control-group.fluid-input input[type=datetime-local],.wy-control-group.fluid-input input[type=datetime],.wy-control-group.fluid-input input[type=email],.wy-control-group.fluid-input input[type=month],.wy-control-group.fluid-input input[type=number],.wy-control-group.fluid-input input[type=password],.wy-control-group.fluid-input input[type=search],.wy-control-group.fluid-input input[type=tel],.wy-control-group.fluid-input input[type=text],.wy-control-group.fluid-input input[type=time],.wy-control-group.fluid-input input[type=url],.wy-control-group.fluid-input input[type=week]{width:100%}.wy-form-message-inline{padding-left:.3em;color:#666;font-size:90%}.wy-form-message{display:block;color:#999;font-size:70%;margin-top:.3125em;font-style:italic}.wy-form-message p{font-size:inherit;font-style:italic;margin-bottom:6px}.wy-form-message p:last-child{margin-bottom:0}input{line-height:normal}input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;*overflow:visible}input[type=color],input[type=date],input[type=datetime-local],input[type=datetime],input[type=email],input[type=month],input[type=number],input[type=password],input[type=search],input[type=tel],input[type=text],input[type=time],input[type=url],input[type=week]{-webkit-appearance:none;padding:6px;display:inline-block;border:1px solid #ccc;font-size:80%;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;box-shadow:inset 0 1px 3px #ddd;border-radius:0;-webkit-transition:border .3s linear;-moz-transition:border .3s linear;transition:border .3s linear}input[type=datetime-local]{padding:.34375em .625em}input[disabled]{cursor:default}input[type=checkbox],input[type=radio]{padding:0;margin-right:.3125em;*height:13px;*width:13px}input[type=checkbox],input[type=radio],input[type=search]{-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}input[type=search]::-webkit-search-cancel-button,input[type=search]::-webkit-search-decoration{-webkit-appearance:none}input[type=color]:focus,input[type=date]:focus,input[type=datetime-local]:focus,input[type=datetime]:focus,input[type=email]:focus,input[type=month]:focus,input[type=number]:focus,input[type=password]:focus,input[type=search]:focus,input[type=tel]:focus,input[type=text]:focus,input[type=time]:focus,input[type=url]:focus,input[type=week]:focus{outline:0;outline:thin dotted\9;border-color:#333}input.no-focus:focus{border-color:#ccc!important}input[type=checkbox]:focus,input[type=file]:focus,input[type=radio]:focus{outline:thin dotted #333;outline:1px auto #129fea}input[type=color][disabled],input[type=date][disabled],input[type=datetime-local][disabled],input[type=datetime][disabled],input[type=email][disabled],input[type=month][disabled],input[type=number][disabled],input[type=password][disabled],input[type=search][disabled],input[type=tel][disabled],input[type=text][disabled],input[type=time][disabled],input[type=url][disabled],input[type=week][disabled]{cursor:not-allowed;background-color:#fafafa}input:focus:invalid,select:focus:invalid,textarea:focus:invalid{color:#e74c3c;border:1px solid #e74c3c}input:focus:invalid:focus,select:focus:invalid:focus,textarea:focus:invalid:focus{border-color:#e74c3c}input[type=checkbox]:focus:invalid:focus,input[type=file]:focus:invalid:focus,input[type=radio]:focus:invalid:focus{outline-color:#e74c3c}input.wy-input-large{padding:12px;font-size:100%}textarea{overflow:auto;vertical-align:top;width:100%;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif}select,textarea{padding:.5em .625em;display:inline-block;border:1px solid #ccc;font-size:80%;box-shadow:inset 0 1px 3px #ddd;-webkit-transition:border .3s linear;-moz-transition:border .3s linear;transition:border .3s linear}select{border:1px solid #ccc;background-color:#fff}select[multiple]{height:auto}select:focus,textarea:focus{outline:0}input[readonly],select[disabled],select[readonly],textarea[disabled],textarea[readonly]{cursor:not-allowed;background-color:#fafafa}input[type=checkbox][disabled],input[type=radio][disabled]{cursor:not-allowed}.wy-checkbox,.wy-radio{margin:6px 0;color:#404040;display:block}.wy-checkbox input,.wy-radio input{vertical-align:baseline}.wy-form-message-inline{display:inline-block;*display:inline;*zoom:1;vertical-align:middle}.wy-input-prefix,.wy-input-suffix{white-space:nowrap;padding:6px}.wy-input-prefix .wy-input-context,.wy-input-suffix .wy-input-context{line-height:27px;padding:0 8px;display:inline-block;font-size:80%;background-color:#f3f6f6;border:1px solid #ccc;color:#999}.wy-input-suffix .wy-input-context{border-left:0}.wy-input-prefix .wy-input-context{border-right:0}.wy-switch{position:relative;display:block;height:24px;margin-top:12px;cursor:pointer}.wy-switch:before{left:0;top:0;width:36px;height:12px;background:#ccc}.wy-switch:after,.wy-switch:before{position:absolute;content:"";display:block;border-radius:4px;-webkit-transition:all .2s ease-in-out;-moz-transition:all .2s ease-in-out;transition:all .2s ease-in-out}.wy-switch:after{width:18px;height:18px;background:#999;left:-3px;top:-3px}.wy-switch span{position:absolute;left:48px;display:block;font-size:12px;color:#ccc;line-height:1}.wy-switch.active:before{background:#1e8449}.wy-switch.active:after{left:24px;background:#27ae60}.wy-switch.disabled{cursor:not-allowed;opacity:.8}.wy-control-group.wy-control-group-error .wy-form-message,.wy-control-group.wy-control-group-error>label{color:#e74c3c}.wy-control-group.wy-control-group-error input[type=color],.wy-control-group.wy-control-group-error input[type=date],.wy-control-group.wy-control-group-error input[type=datetime-local],.wy-control-group.wy-control-group-error input[type=datetime],.wy-control-group.wy-control-group-error input[type=email],.wy-control-group.wy-control-group-error input[type=month],.wy-control-group.wy-control-group-error input[type=number],.wy-control-group.wy-control-group-error input[type=password],.wy-control-group.wy-control-group-error input[type=search],.wy-control-group.wy-control-group-error input[type=tel],.wy-control-group.wy-control-group-error input[type=text],.wy-control-group.wy-control-group-error input[type=time],.wy-control-group.wy-control-group-error input[type=url],.wy-control-group.wy-control-group-error input[type=week],.wy-control-group.wy-control-group-error textarea{border:1px solid #e74c3c}.wy-inline-validate{white-space:nowrap}.wy-inline-validate .wy-input-context{padding:.5em .625em;display:inline-block;font-size:80%}.wy-inline-validate.wy-inline-validate-success .wy-input-context{color:#27ae60}.wy-inline-validate.wy-inline-validate-danger .wy-input-context{color:#e74c3c}.wy-inline-validate.wy-inline-validate-warning .wy-input-context{color:#e67e22}.wy-inline-validate.wy-inline-validate-info .wy-input-context{color:#2980b9}.rotate-90{-webkit-transform:rotate(90deg);-moz-transform:rotate(90deg);-ms-transform:rotate(90deg);-o-transform:rotate(90deg);transform:rotate(90deg)}.rotate-180{-webkit-transform:rotate(180deg);-moz-transform:rotate(180deg);-ms-transform:rotate(180deg);-o-transform:rotate(180deg);transform:rotate(180deg)}.rotate-270{-webkit-transform:rotate(270deg);-moz-transform:rotate(270deg);-ms-transform:rotate(270deg);-o-transform:rotate(270deg);transform:rotate(270deg)}.mirror{-webkit-transform:scaleX(-1);-moz-transform:scaleX(-1);-ms-transform:scaleX(-1);-o-transform:scaleX(-1);transform:scaleX(-1)}.mirror.rotate-90{-webkit-transform:scaleX(-1) rotate(90deg);-moz-transform:scaleX(-1) rotate(90deg);-ms-transform:scaleX(-1) rotate(90deg);-o-transform:scaleX(-1) rotate(90deg);transform:scaleX(-1) rotate(90deg)}.mirror.rotate-180{-webkit-transform:scaleX(-1) rotate(180deg);-moz-transform:scaleX(-1) rotate(180deg);-ms-transform:scaleX(-1) rotate(180deg);-o-transform:scaleX(-1) rotate(180deg);transform:scaleX(-1) rotate(180deg)}.mirror.rotate-270{-webkit-transform:scaleX(-1) rotate(270deg);-moz-transform:scaleX(-1) rotate(270deg);-ms-transform:scaleX(-1) rotate(270deg);-o-transform:scaleX(-1) rotate(270deg);transform:scaleX(-1) rotate(270deg)}@media only screen and (max-width:480px){.wy-form button[type=submit]{margin:.7em 0 0}.wy-form input[type=color],.wy-form input[type=date],.wy-form input[type=datetime-local],.wy-form input[type=datetime],.wy-form input[type=email],.wy-form input[type=month],.wy-form input[type=number],.wy-form input[type=password],.wy-form input[type=search],.wy-form input[type=tel],.wy-form input[type=text],.wy-form input[type=time],.wy-form input[type=url],.wy-form input[type=week],.wy-form label{margin-bottom:.3em;display:block}.wy-form input[type=color],.wy-form input[type=date],.wy-form input[type=datetime-local],.wy-form input[type=datetime],.wy-form input[type=email],.wy-form input[type=month],.wy-form input[type=number],.wy-form input[type=password],.wy-form input[type=search],.wy-form input[type=tel],.wy-form input[type=time],.wy-form input[type=url],.wy-form input[type=week]{margin-bottom:0}.wy-form-aligned .wy-control-group label{margin-bottom:.3em;text-align:left;display:block;width:100%}.wy-form-aligned .wy-control{margin:1.5em 0 0}.wy-form-message,.wy-form-message-inline,.wy-form .wy-help-inline{display:block;font-size:80%;padding:6px 0}}@media screen and (max-width:768px){.tablet-hide{display:none}}@media screen and (max-width:480px){.mobile-hide{display:none}}.float-left{float:left}.float-right{float:right}.full-width{width:100%}.rst-content table.docutils,.rst-content table.field-list,.wy-table{border-collapse:collapse;border-spacing:0;empty-cells:show;margin-bottom:24px}.rst-content table.docutils caption,.rst-content table.field-list caption,.wy-table caption{color:#000;font:italic 85%/1 arial,sans-serif;padding:1em 0;text-align:center}.rst-content table.docutils td,.rst-content table.docutils th,.rst-content table.field-list td,.rst-content table.field-list th,.wy-table td,.wy-table th{font-size:90%;margin:0;overflow:visible;padding:8px 16px}.rst-content table.docutils td:first-child,.rst-content table.docutils th:first-child,.rst-content table.field-list td:first-child,.rst-content table.field-list th:first-child,.wy-table td:first-child,.wy-table th:first-child{border-left-width:0}.rst-content table.docutils thead,.rst-content table.field-list thead,.wy-table thead{color:#000;text-align:left;vertical-align:bottom;white-space:nowrap}.rst-content table.docutils thead th,.rst-content table.field-list thead th,.wy-table thead th{font-weight:700;border-bottom:2px solid #e1e4e5}.rst-content table.docutils td,.rst-content table.field-list td,.wy-table td{background-color:transparent;vertical-align:middle}.rst-content table.docutils td p,.rst-content table.field-list td p,.wy-table td p{line-height:18px}.rst-content table.docutils td p:last-child,.rst-content table.field-list td p:last-child,.wy-table td p:last-child{margin-bottom:0}.rst-content table.docutils .wy-table-cell-min,.rst-content table.field-list .wy-table-cell-min,.wy-table .wy-table-cell-min{width:1%;padding-right:0}.rst-content table.docutils .wy-table-cell-min input[type=checkbox],.rst-content table.field-list .wy-table-cell-min input[type=checkbox],.wy-table .wy-table-cell-min input[type=checkbox]{margin:0}.wy-table-secondary{color:grey;font-size:90%}.wy-table-tertiary{color:grey;font-size:80%}.rst-content table.docutils:not(.field-list) tr:nth-child(2n-1) td,.wy-table-backed,.wy-table-odd td,.wy-table-striped tr:nth-child(2n-1) td{background-color:#f3f6f6}.rst-content table.docutils,.wy-table-bordered-all{border:1px solid #e1e4e5}.rst-content table.docutils td,.wy-table-bordered-all td{border-bottom:1px solid #e1e4e5;border-left:1px solid #e1e4e5}.rst-content table.docutils tbody>tr:last-child td,.wy-table-bordered-all tbody>tr:last-child td{border-bottom-width:0}.wy-table-bordered{border:1px solid #e1e4e5}.wy-table-bordered-rows td{border-bottom:1px solid #e1e4e5}.wy-table-bordered-rows tbody>tr:last-child td{border-bottom-width:0}.wy-table-horizontal td,.wy-table-horizontal th{border-width:0 0 1px;border-bottom:1px solid #e1e4e5}.wy-table-horizontal tbody>tr:last-child td{border-bottom-width:0}.wy-table-responsive{margin-bottom:24px;max-width:100%;overflow:auto}.wy-table-responsive table{margin-bottom:0!important}.wy-table-responsive table td,.wy-table-responsive table th{white-space:nowrap}a{color:#2980b9;text-decoration:none;cursor:pointer}a:hover{color:#3091d1}a:visited{color:#9b59b6}html{height:100%}body,html{overflow-x:hidden}body{font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;font-weight:400;color:#404040;min-height:100%;background:#edf0f2}.wy-text-left{text-align:left}.wy-text-center{text-align:center}.wy-text-right{text-align:right}.wy-text-large{font-size:120%}.wy-text-normal{font-size:100%}.wy-text-small,small{font-size:80%}.wy-text-strike{text-decoration:line-through}.wy-text-warning{color:#e67e22!important}a.wy-text-warning:hover{color:#eb9950!important}.wy-text-info{color:#2980b9!important}a.wy-text-info:hover{color:#409ad5!important}.wy-text-success{color:#27ae60!important}a.wy-text-success:hover{color:#36d278!important}.wy-text-danger{color:#e74c3c!important}a.wy-text-danger:hover{color:#ed7669!important}.wy-text-neutral{color:#404040!important}a.wy-text-neutral:hover{color:#595959!important}.rst-content .toctree-wrapper>p.caption,h1,h2,h3,h4,h5,h6,legend{margin-top:0;font-weight:700;font-family:Roboto Slab,ff-tisa-web-pro,Georgia,Arial,sans-serif}p{line-height:24px;font-size:16px;margin:0 0 24px}h1{font-size:175%}.rst-content .toctree-wrapper>p.caption,h2{font-size:150%}h3{font-size:125%}h4{font-size:115%}h5{font-size:110%}h6{font-size:100%}hr{display:block;height:1px;border:0;border-top:1px solid #e1e4e5;margin:24px 0;padding:0}.rst-content code,.rst-content tt,code{white-space:nowrap;max-width:100%;background:#fff;border:1px solid #e1e4e5;font-size:75%;padding:0 5px;font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;color:#e74c3c;overflow-x:auto}.rst-content tt.code-large,code.code-large{font-size:90%}.rst-content .section ul,.rst-content .toctree-wrapper ul,.rst-content section ul,.wy-plain-list-disc,article ul{list-style:disc;line-height:24px;margin-bottom:24px}.rst-content .section ul li,.rst-content .toctree-wrapper ul li,.rst-content section ul li,.wy-plain-list-disc li,article ul li{list-style:disc;margin-left:24px}.rst-content .section ul li p:last-child,.rst-content .section ul li ul,.rst-content .toctree-wrapper ul li p:last-child,.rst-content .toctree-wrapper ul li ul,.rst-content section ul li p:last-child,.rst-content section ul li ul,.wy-plain-list-disc li p:last-child,.wy-plain-list-disc li ul,article ul li p:last-child,article ul li ul{margin-bottom:0}.rst-content .section ul li li,.rst-content .toctree-wrapper ul li li,.rst-content section ul li li,.wy-plain-list-disc li li,article ul li li{list-style:circle}.rst-content .section ul li li li,.rst-content .toctree-wrapper ul li li li,.rst-content section ul li li li,.wy-plain-list-disc li li li,article ul li li li{list-style:square}.rst-content .section ul li ol li,.rst-content .toctree-wrapper ul li ol li,.rst-content section ul li ol li,.wy-plain-list-disc li ol li,article ul li ol li{list-style:decimal}.rst-content .section ol,.rst-content .section ol.arabic,.rst-content .toctree-wrapper ol,.rst-content .toctree-wrapper ol.arabic,.rst-content section ol,.rst-content section ol.arabic,.wy-plain-list-decimal,article ol{list-style:decimal;line-height:24px;margin-bottom:24px}.rst-content .section ol.arabic li,.rst-content .section ol li,.rst-content .toctree-wrapper ol.arabic li,.rst-content .toctree-wrapper ol li,.rst-content section ol.arabic li,.rst-content section ol li,.wy-plain-list-decimal li,article ol li{list-style:decimal;margin-left:24px}.rst-content .section ol.arabic li ul,.rst-content .section ol li p:last-child,.rst-content .section ol li ul,.rst-content .toctree-wrapper ol.arabic li ul,.rst-content .toctree-wrapper ol li p:last-child,.rst-content .toctree-wrapper ol li ul,.rst-content section ol.arabic li ul,.rst-content section ol li p:last-child,.rst-content section ol li ul,.wy-plain-list-decimal li p:last-child,.wy-plain-list-decimal li ul,article ol li p:last-child,article ol li ul{margin-bottom:0}.rst-content .section ol.arabic li ul li,.rst-content .section ol li ul li,.rst-content .toctree-wrapper ol.arabic li ul li,.rst-content .toctree-wrapper ol li ul li,.rst-content section ol.arabic li ul li,.rst-content section ol li ul li,.wy-plain-list-decimal li ul li,article ol li ul li{list-style:disc}.wy-breadcrumbs{*zoom:1}.wy-breadcrumbs:after,.wy-breadcrumbs:before{display:table;content:""}.wy-breadcrumbs:after{clear:both}.wy-breadcrumbs>li{display:inline-block;padding-top:5px}.wy-breadcrumbs>li.wy-breadcrumbs-aside{float:right}.rst-content .wy-breadcrumbs>li code,.rst-content .wy-breadcrumbs>li tt,.wy-breadcrumbs>li .rst-content tt,.wy-breadcrumbs>li code{all:inherit;color:inherit}.breadcrumb-item:before{content:"/";color:#bbb;font-size:13px;padding:0 6px 0 3px}.wy-breadcrumbs-extra{margin-bottom:0;color:#b3b3b3;font-size:80%;display:inline-block}@media screen and (max-width:480px){.wy-breadcrumbs-extra,.wy-breadcrumbs li.wy-breadcrumbs-aside{display:none}}@media print{.wy-breadcrumbs li.wy-breadcrumbs-aside{display:none}}html{font-size:16px}.wy-affix{position:fixed;top:1.618em}.wy-menu a:hover{text-decoration:none}.wy-menu-horiz{*zoom:1}.wy-menu-horiz:after,.wy-menu-horiz:before{display:table;content:""}.wy-menu-horiz:after{clear:both}.wy-menu-horiz li,.wy-menu-horiz ul{display:inline-block}.wy-menu-horiz li:hover{background:hsla(0,0%,100%,.1)}.wy-menu-horiz li.divide-left{border-left:1px solid #404040}.wy-menu-horiz li.divide-right{border-right:1px solid #404040}.wy-menu-horiz a{height:32px;display:inline-block;line-height:32px;padding:0 16px}.wy-menu-vertical{width:300px}.wy-menu-vertical header,.wy-menu-vertical p.caption{color:#55a5d9;height:32px;line-height:32px;padding:0 1.618em;margin:12px 0 0;display:block;font-weight:700;text-transform:uppercase;font-size:85%;white-space:nowrap}.wy-menu-vertical ul{margin-bottom:0}.wy-menu-vertical li.divide-top{border-top:1px solid #404040}.wy-menu-vertical li.divide-bottom{border-bottom:1px solid #404040}.wy-menu-vertical li.current{background:#e3e3e3}.wy-menu-vertical li.current a{color:grey;border-right:1px solid #c9c9c9;padding:.4045em 2.427em}.wy-menu-vertical li.current a:hover{background:#d6d6d6}.rst-content .wy-menu-vertical li tt,.wy-menu-vertical li .rst-content tt,.wy-menu-vertical li code{border:none;background:inherit;color:inherit;padding-left:0;padding-right:0}.wy-menu-vertical li button.toctree-expand{display:block;float:left;margin-left:-1.2em;line-height:18px;color:#4d4d4d;border:none;background:none;padding:0}.wy-menu-vertical li.current>a,.wy-menu-vertical li.on a{color:#404040;font-weight:700;position:relative;background:#fcfcfc;border:none;padding:.4045em 1.618em}.wy-menu-vertical li.current>a:hover,.wy-menu-vertical li.on a:hover{background:#fcfcfc}.wy-menu-vertical li.current>a:hover button.toctree-expand,.wy-menu-vertical li.on a:hover button.toctree-expand{color:grey}.wy-menu-vertical li.current>a button.toctree-expand,.wy-menu-vertical li.on a button.toctree-expand{display:block;line-height:18px;color:#333}.wy-menu-vertical li.toctree-l1.current>a{border-bottom:1px solid #c9c9c9;border-top:1px solid #c9c9c9}.wy-menu-vertical .toctree-l1.current .toctree-l2>ul,.wy-menu-vertical .toctree-l2.current .toctree-l3>ul,.wy-menu-vertical .toctree-l3.current .toctree-l4>ul,.wy-menu-vertical .toctree-l4.current .toctree-l5>ul,.wy-menu-vertical .toctree-l5.current .toctree-l6>ul,.wy-menu-vertical .toctree-l6.current .toctree-l7>ul,.wy-menu-vertical .toctree-l7.current .toctree-l8>ul,.wy-menu-vertical .toctree-l8.current .toctree-l9>ul,.wy-menu-vertical .toctree-l9.current .toctree-l10>ul,.wy-menu-vertical .toctree-l10.current .toctree-l11>ul{display:none}.wy-menu-vertical .toctree-l1.current .current.toctree-l2>ul,.wy-menu-vertical .toctree-l2.current .current.toctree-l3>ul,.wy-menu-vertical .toctree-l3.current .current.toctree-l4>ul,.wy-menu-vertical .toctree-l4.current .current.toctree-l5>ul,.wy-menu-vertical .toctree-l5.current .current.toctree-l6>ul,.wy-menu-vertical .toctree-l6.current .current.toctree-l7>ul,.wy-menu-vertical .toctree-l7.current .current.toctree-l8>ul,.wy-menu-vertical .toctree-l8.current .current.toctree-l9>ul,.wy-menu-vertical .toctree-l9.current .current.toctree-l10>ul,.wy-menu-vertical .toctree-l10.current .current.toctree-l11>ul{display:block}.wy-menu-vertical li.toctree-l3,.wy-menu-vertical li.toctree-l4{font-size:.9em}.wy-menu-vertical li.toctree-l2 a,.wy-menu-vertical li.toctree-l3 a,.wy-menu-vertical li.toctree-l4 a,.wy-menu-vertical li.toctree-l5 a,.wy-menu-vertical li.toctree-l6 a,.wy-menu-vertical li.toctree-l7 a,.wy-menu-vertical li.toctree-l8 a,.wy-menu-vertical li.toctree-l9 a,.wy-menu-vertical li.toctree-l10 a{color:#404040}.wy-menu-vertical li.toctree-l2 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l3 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l4 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l5 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l6 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l7 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l8 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l9 a:hover button.toctree-expand,.wy-menu-vertical li.toctree-l10 a:hover button.toctree-expand{color:grey}.wy-menu-vertical li.toctree-l2.current li.toctree-l3>a,.wy-menu-vertical li.toctree-l3.current li.toctree-l4>a,.wy-menu-vertical li.toctree-l4.current li.toctree-l5>a,.wy-menu-vertical li.toctree-l5.current li.toctree-l6>a,.wy-menu-vertical li.toctree-l6.current li.toctree-l7>a,.wy-menu-vertical li.toctree-l7.current li.toctree-l8>a,.wy-menu-vertical li.toctree-l8.current li.toctree-l9>a,.wy-menu-vertical li.toctree-l9.current li.toctree-l10>a,.wy-menu-vertical li.toctree-l10.current li.toctree-l11>a{display:block}.wy-menu-vertical li.toctree-l2.current>a{padding:.4045em 2.427em}.wy-menu-vertical li.toctree-l2.current li.toctree-l3>a{padding:.4045em 1.618em .4045em 4.045em}.wy-menu-vertical li.toctree-l3.current>a{padding:.4045em 4.045em}.wy-menu-vertical li.toctree-l3.current li.toctree-l4>a{padding:.4045em 1.618em .4045em 5.663em}.wy-menu-vertical li.toctree-l4.current>a{padding:.4045em 5.663em}.wy-menu-vertical li.toctree-l4.current li.toctree-l5>a{padding:.4045em 1.618em .4045em 7.281em}.wy-menu-vertical li.toctree-l5.current>a{padding:.4045em 7.281em}.wy-menu-vertical li.toctree-l5.current li.toctree-l6>a{padding:.4045em 1.618em .4045em 8.899em}.wy-menu-vertical li.toctree-l6.current>a{padding:.4045em 8.899em}.wy-menu-vertical li.toctree-l6.current li.toctree-l7>a{padding:.4045em 1.618em .4045em 10.517em}.wy-menu-vertical li.toctree-l7.current>a{padding:.4045em 10.517em}.wy-menu-vertical li.toctree-l7.current li.toctree-l8>a{padding:.4045em 1.618em .4045em 12.135em}.wy-menu-vertical li.toctree-l8.current>a{padding:.4045em 12.135em}.wy-menu-vertical li.toctree-l8.current li.toctree-l9>a{padding:.4045em 1.618em .4045em 13.753em}.wy-menu-vertical li.toctree-l9.current>a{padding:.4045em 13.753em}.wy-menu-vertical li.toctree-l9.current li.toctree-l10>a{padding:.4045em 1.618em .4045em 15.371em}.wy-menu-vertical li.toctree-l10.current>a{padding:.4045em 15.371em}.wy-menu-vertical li.toctree-l10.current li.toctree-l11>a{padding:.4045em 1.618em .4045em 16.989em}.wy-menu-vertical li.toctree-l2.current>a,.wy-menu-vertical li.toctree-l2.current li.toctree-l3>a{background:#c9c9c9}.wy-menu-vertical li.toctree-l2 button.toctree-expand{color:#a3a3a3}.wy-menu-vertical li.toctree-l3.current>a,.wy-menu-vertical li.toctree-l3.current li.toctree-l4>a{background:#bdbdbd}.wy-menu-vertical li.toctree-l3 button.toctree-expand{color:#969696}.wy-menu-vertical li.current ul{display:block}.wy-menu-vertical li ul{margin-bottom:0;display:none}.wy-menu-vertical li ul li a{margin-bottom:0;color:#d9d9d9;font-weight:400}.wy-menu-vertical a{line-height:18px;padding:.4045em 1.618em;display:block;position:relative;font-size:90%;color:#d9d9d9}.wy-menu-vertical a:hover{background-color:#4e4a4a;cursor:pointer}.wy-menu-vertical a:hover button.toctree-expand{color:#d9d9d9}.wy-menu-vertical a:active{background-color:#2980b9;cursor:pointer;color:#fff}.wy-menu-vertical a:active button.toctree-expand{color:#fff}.wy-side-nav-search{display:block;width:300px;padding:.809em;margin-bottom:.809em;z-index:200;background-color:#2980b9;text-align:center;color:#fcfcfc}.wy-side-nav-search input[type=text]{width:100%;border-radius:50px;padding:6px 12px;border-color:#2472a4}.wy-side-nav-search img{display:block;margin:auto auto .809em;height:45px;width:45px;background-color:#2980b9;padding:5px;border-radius:100%}.wy-side-nav-search .wy-dropdown>a,.wy-side-nav-search>a{color:#fcfcfc;font-size:100%;font-weight:700;display:inline-block;padding:4px 6px;margin-bottom:.809em;max-width:100%}.wy-side-nav-search .wy-dropdown>a:hover,.wy-side-nav-search>a:hover{background:hsla(0,0%,100%,.1)}.wy-side-nav-search .wy-dropdown>a img.logo,.wy-side-nav-search>a img.logo{display:block;margin:0 auto;height:auto;width:auto;border-radius:0;max-width:100%;background:transparent}.wy-side-nav-search .wy-dropdown>a.icon img.logo,.wy-side-nav-search>a.icon img.logo{margin-top:.85em}.wy-side-nav-search>div.version{margin-top:-.4045em;margin-bottom:.809em;font-weight:400;color:hsla(0,0%,100%,.3)}.wy-nav .wy-menu-vertical header{color:#2980b9}.wy-nav .wy-menu-vertical a{color:#b3b3b3}.wy-nav .wy-menu-vertical a:hover{background-color:#2980b9;color:#fff}[data-menu-wrap]{-webkit-transition:all .2s ease-in;-moz-transition:all .2s ease-in;transition:all .2s ease-in;position:absolute;opacity:1;width:100%;opacity:0}[data-menu-wrap].move-center{left:0;right:auto;opacity:1}[data-menu-wrap].move-left{right:auto;left:-100%;opacity:0}[data-menu-wrap].move-right{right:-100%;left:auto;opacity:0}.wy-body-for-nav{background:#fcfcfc}.wy-grid-for-nav{position:absolute;width:100%;height:100%}.wy-nav-side{position:fixed;top:0;bottom:0;left:0;padding-bottom:2em;width:300px;overflow-x:hidden;overflow-y:hidden;min-height:100%;color:#9b9b9b;background:#343131;z-index:200}.wy-side-scroll{width:320px;position:relative;overflow-x:hidden;overflow-y:scroll;height:100%}.wy-nav-top{display:none;background:#2980b9;color:#fff;padding:.4045em .809em;position:relative;line-height:50px;text-align:center;font-size:100%;*zoom:1}.wy-nav-top:after,.wy-nav-top:before{display:table;content:""}.wy-nav-top:after{clear:both}.wy-nav-top a{color:#fff;font-weight:700}.wy-nav-top img{margin-right:12px;height:45px;width:45px;background-color:#2980b9;padding:5px;border-radius:100%}.wy-nav-top i{font-size:30px;float:left;cursor:pointer;padding-top:inherit}.wy-nav-content-wrap{margin-left:300px;background:#fcfcfc;min-height:100%}.wy-nav-content{padding:1.618em 3.236em;height:100%;max-width:800px;margin:auto}.wy-body-mask{position:fixed;width:100%;height:100%;background:rgba(0,0,0,.2);display:none;z-index:499}.wy-body-mask.on{display:block}footer{color:grey}footer p{margin-bottom:12px}.rst-content footer span.commit tt,footer span.commit .rst-content tt,footer span.commit code{padding:0;font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;font-size:1em;background:none;border:none;color:grey}.rst-footer-buttons{*zoom:1}.rst-footer-buttons:after,.rst-footer-buttons:before{width:100%;display:table;content:""}.rst-footer-buttons:after{clear:both}.rst-breadcrumbs-buttons{margin-top:12px;*zoom:1}.rst-breadcrumbs-buttons:after,.rst-breadcrumbs-buttons:before{display:table;content:""}.rst-breadcrumbs-buttons:after{clear:both}#search-results .search li{margin-bottom:24px;border-bottom:1px solid #e1e4e5;padding-bottom:24px}#search-results .search li:first-child{border-top:1px solid #e1e4e5;padding-top:24px}#search-results .search li a{font-size:120%;margin-bottom:12px;display:inline-block}#search-results .context{color:grey;font-size:90%}.genindextable li>ul{margin-left:24px}@media screen and (max-width:768px){.wy-body-for-nav{background:#fcfcfc}.wy-nav-top{display:block}.wy-nav-side{left:-300px}.wy-nav-side.shift{width:85%;left:0}.wy-menu.wy-menu-vertical,.wy-side-nav-search,.wy-side-scroll{width:auto}.wy-nav-content-wrap{margin-left:0}.wy-nav-content-wrap .wy-nav-content{padding:1.618em}.wy-nav-content-wrap.shift{position:fixed;min-width:100%;left:85%;top:0;height:100%;overflow:hidden}}@media screen and (min-width:1100px){.wy-nav-content-wrap{background:rgba(0,0,0,.05)}.wy-nav-content{margin:0;background:#fcfcfc}}@media print{.rst-versions,.wy-nav-side,footer{display:none}.wy-nav-content-wrap{margin-left:0}}.rst-versions{position:fixed;bottom:0;left:0;width:300px;color:#fcfcfc;background:#1f1d1d;font-family:Lato,proxima-nova,Helvetica Neue,Arial,sans-serif;z-index:400}.rst-versions a{color:#2980b9;text-decoration:none}.rst-versions .rst-badge-small{display:none}.rst-versions .rst-current-version{padding:12px;background-color:#272525;display:block;text-align:right;font-size:90%;cursor:pointer;color:#27ae60;*zoom:1}.rst-versions .rst-current-version:after,.rst-versions .rst-current-version:before{display:table;content:""}.rst-versions .rst-current-version:after{clear:both}.rst-content .code-block-caption .rst-versions .rst-current-version .headerlink,.rst-content .eqno .rst-versions .rst-current-version .headerlink,.rst-content .rst-versions .rst-current-version .admonition-title,.rst-content code.download .rst-versions .rst-current-version span:first-child,.rst-content dl dt .rst-versions .rst-current-version .headerlink,.rst-content h1 .rst-versions .rst-current-version .headerlink,.rst-content h2 .rst-versions .rst-current-version .headerlink,.rst-content h3 .rst-versions .rst-current-version .headerlink,.rst-content h4 .rst-versions .rst-current-version .headerlink,.rst-content h5 .rst-versions .rst-current-version .headerlink,.rst-content h6 .rst-versions .rst-current-version .headerlink,.rst-content p .rst-versions .rst-current-version .headerlink,.rst-content table>caption .rst-versions .rst-current-version .headerlink,.rst-content tt.download .rst-versions .rst-current-version span:first-child,.rst-versions .rst-current-version .fa,.rst-versions .rst-current-version .icon,.rst-versions .rst-current-version .rst-content .admonition-title,.rst-versions .rst-current-version .rst-content .code-block-caption .headerlink,.rst-versions .rst-current-version .rst-content .eqno .headerlink,.rst-versions .rst-current-version .rst-content code.download span:first-child,.rst-versions .rst-current-version .rst-content dl dt .headerlink,.rst-versions .rst-current-version .rst-content h1 .headerlink,.rst-versions .rst-current-version .rst-content h2 .headerlink,.rst-versions .rst-current-version .rst-content h3 .headerlink,.rst-versions .rst-current-version .rst-content h4 .headerlink,.rst-versions .rst-current-version .rst-content h5 .headerlink,.rst-versions .rst-current-version .rst-content h6 .headerlink,.rst-versions .rst-current-version .rst-content p .headerlink,.rst-versions .rst-current-version .rst-content table>caption .headerlink,.rst-versions .rst-current-version .rst-content tt.download span:first-child,.rst-versions .rst-current-version .wy-menu-vertical li button.toctree-expand,.wy-menu-vertical li .rst-versions .rst-current-version button.toctree-expand{color:#fcfcfc}.rst-versions .rst-current-version .fa-book,.rst-versions .rst-current-version .icon-book{float:left}.rst-versions .rst-current-version.rst-out-of-date{background-color:#e74c3c;color:#fff}.rst-versions .rst-current-version.rst-active-old-version{background-color:#f1c40f;color:#000}.rst-versions.shift-up{height:auto;max-height:100%;overflow-y:scroll}.rst-versions.shift-up .rst-other-versions{display:block}.rst-versions .rst-other-versions{font-size:90%;padding:12px;color:grey;display:none}.rst-versions .rst-other-versions hr{display:block;height:1px;border:0;margin:20px 0;padding:0;border-top:1px solid #413d3d}.rst-versions .rst-other-versions dd{display:inline-block;margin:0}.rst-versions .rst-other-versions dd a{display:inline-block;padding:6px;color:#fcfcfc}.rst-versions.rst-badge{width:auto;bottom:20px;right:20px;left:auto;border:none;max-width:300px;max-height:90%}.rst-versions.rst-badge .fa-book,.rst-versions.rst-badge .icon-book{float:none;line-height:30px}.rst-versions.rst-badge.shift-up .rst-current-version{text-align:right}.rst-versions.rst-badge.shift-up .rst-current-version .fa-book,.rst-versions.rst-badge.shift-up .rst-current-version .icon-book{float:left}.rst-versions.rst-badge>.rst-current-version{width:auto;height:30px;line-height:30px;padding:0 6px;display:block;text-align:center}@media screen and (max-width:768px){.rst-versions{width:85%;display:none}.rst-versions.shift{display:block}}.rst-content .toctree-wrapper>p.caption,.rst-content h1,.rst-content h2,.rst-content h3,.rst-content h4,.rst-content h5,.rst-content h6{margin-bottom:24px}.rst-content img{max-width:100%;height:auto}.rst-content div.figure,.rst-content figure{margin-bottom:24px}.rst-content div.figure .caption-text,.rst-content figure .caption-text{font-style:italic}.rst-content div.figure p:last-child.caption,.rst-content figure p:last-child.caption{margin-bottom:0}.rst-content div.figure.align-center,.rst-content figure.align-center{text-align:center}.rst-content .section>a>img,.rst-content .section>img,.rst-content section>a>img,.rst-content section>img{margin-bottom:24px}.rst-content abbr[title]{text-decoration:none}.rst-content.style-external-links a.reference.external:after{font-family:FontAwesome;content:"\f08e";color:#b3b3b3;vertical-align:super;font-size:60%;margin:0 .2em}.rst-content blockquote{margin-left:24px;line-height:24px;margin-bottom:24px}.rst-content pre.literal-block{white-space:pre;margin:0;padding:12px;font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;display:block;overflow:auto}.rst-content div[class^=highlight],.rst-content pre.literal-block{border:1px solid #e1e4e5;overflow-x:auto;margin:1px 0 24px}.rst-content div[class^=highlight] div[class^=highlight],.rst-content pre.literal-block div[class^=highlight]{padding:0;border:none;margin:0}.rst-content div[class^=highlight] td.code{width:100%}.rst-content .linenodiv pre{border-right:1px solid #e6e9ea;margin:0;padding:12px;font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;user-select:none;pointer-events:none}.rst-content div[class^=highlight] pre{white-space:pre;margin:0;padding:12px;display:block;overflow:auto}.rst-content div[class^=highlight] pre .hll{display:block;margin:0 -12px;padding:0 12px}.rst-content .linenodiv pre,.rst-content div[class^=highlight] pre,.rst-content pre.literal-block{font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;font-size:12px;line-height:1.4}.rst-content div.highlight .gp,.rst-content div.highlight span.linenos{user-select:none;pointer-events:none}.rst-content div.highlight span.linenos{display:inline-block;padding-left:0;padding-right:12px;margin-right:12px;border-right:1px solid #e6e9ea}.rst-content .code-block-caption{font-style:italic;font-size:85%;line-height:1;padding:1em 0;text-align:center}@media print{.rst-content .codeblock,.rst-content div[class^=highlight],.rst-content div[class^=highlight] pre{white-space:pre-wrap}}.rst-content .admonition,.rst-content .admonition-todo,.rst-content .attention,.rst-content .caution,.rst-content .danger,.rst-content .error,.rst-content .hint,.rst-content .important,.rst-content .note,.rst-content .seealso,.rst-content .tip,.rst-content .warning{clear:both}.rst-content .admonition-todo .last,.rst-content .admonition-todo>:last-child,.rst-content .admonition .last,.rst-content .admonition>:last-child,.rst-content .attention .last,.rst-content .attention>:last-child,.rst-content .caution .last,.rst-content .caution>:last-child,.rst-content .danger .last,.rst-content .danger>:last-child,.rst-content .error .last,.rst-content .error>:last-child,.rst-content .hint .last,.rst-content .hint>:last-child,.rst-content .important .last,.rst-content .important>:last-child,.rst-content .note .last,.rst-content .note>:last-child,.rst-content .seealso .last,.rst-content .seealso>:last-child,.rst-content .tip .last,.rst-content .tip>:last-child,.rst-content .warning .last,.rst-content .warning>:last-child{margin-bottom:0}.rst-content .admonition-title:before{margin-right:4px}.rst-content .admonition table{border-color:rgba(0,0,0,.1)}.rst-content .admonition table td,.rst-content .admonition table th{background:transparent!important;border-color:rgba(0,0,0,.1)!important}.rst-content .section ol.loweralpha,.rst-content .section ol.loweralpha>li,.rst-content .toctree-wrapper ol.loweralpha,.rst-content .toctree-wrapper ol.loweralpha>li,.rst-content section ol.loweralpha,.rst-content section ol.loweralpha>li{list-style:lower-alpha}.rst-content .section ol.upperalpha,.rst-content .section ol.upperalpha>li,.rst-content .toctree-wrapper ol.upperalpha,.rst-content .toctree-wrapper ol.upperalpha>li,.rst-content section ol.upperalpha,.rst-content section ol.upperalpha>li{list-style:upper-alpha}.rst-content .section ol li>*,.rst-content .section ul li>*,.rst-content .toctree-wrapper ol li>*,.rst-content .toctree-wrapper ul li>*,.rst-content section ol li>*,.rst-content section ul li>*{margin-top:12px;margin-bottom:12px}.rst-content .section ol li>:first-child,.rst-content .section ul li>:first-child,.rst-content .toctree-wrapper ol li>:first-child,.rst-content .toctree-wrapper ul li>:first-child,.rst-content section ol li>:first-child,.rst-content section ul li>:first-child{margin-top:0}.rst-content .section ol li>p,.rst-content .section ol li>p:last-child,.rst-content .section ul li>p,.rst-content .section ul li>p:last-child,.rst-content .toctree-wrapper ol li>p,.rst-content .toctree-wrapper ol li>p:last-child,.rst-content .toctree-wrapper ul li>p,.rst-content .toctree-wrapper ul li>p:last-child,.rst-content section ol li>p,.rst-content section ol li>p:last-child,.rst-content section ul li>p,.rst-content section ul li>p:last-child{margin-bottom:12px}.rst-content .section ol li>p:only-child,.rst-content .section ol li>p:only-child:last-child,.rst-content .section ul li>p:only-child,.rst-content .section ul li>p:only-child:last-child,.rst-content .toctree-wrapper ol li>p:only-child,.rst-content .toctree-wrapper ol li>p:only-child:last-child,.rst-content .toctree-wrapper ul li>p:only-child,.rst-content .toctree-wrapper ul li>p:only-child:last-child,.rst-content section ol li>p:only-child,.rst-content section ol li>p:only-child:last-child,.rst-content section ul li>p:only-child,.rst-content section ul li>p:only-child:last-child{margin-bottom:0}.rst-content .section ol li>ol,.rst-content .section ol li>ul,.rst-content .section ul li>ol,.rst-content .section ul li>ul,.rst-content .toctree-wrapper ol li>ol,.rst-content .toctree-wrapper ol li>ul,.rst-content .toctree-wrapper ul li>ol,.rst-content .toctree-wrapper ul li>ul,.rst-content section ol li>ol,.rst-content section ol li>ul,.rst-content section ul li>ol,.rst-content section ul li>ul{margin-bottom:12px}.rst-content .section ol.simple li>*,.rst-content .section ol.simple li ol,.rst-content .section ol.simple li ul,.rst-content .section ul.simple li>*,.rst-content .section ul.simple li ol,.rst-content .section ul.simple li ul,.rst-content .toctree-wrapper ol.simple li>*,.rst-content .toctree-wrapper ol.simple li ol,.rst-content .toctree-wrapper ol.simple li ul,.rst-content .toctree-wrapper ul.simple li>*,.rst-content .toctree-wrapper ul.simple li ol,.rst-content .toctree-wrapper ul.simple li ul,.rst-content section ol.simple li>*,.rst-content section ol.simple li ol,.rst-content section ol.simple li ul,.rst-content section ul.simple li>*,.rst-content section ul.simple li ol,.rst-content section ul.simple li ul{margin-top:0;margin-bottom:0}.rst-content .line-block{margin-left:0;margin-bottom:24px;line-height:24px}.rst-content .line-block .line-block{margin-left:24px;margin-bottom:0}.rst-content .topic-title{font-weight:700;margin-bottom:12px}.rst-content .toc-backref{color:#404040}.rst-content .align-right{float:right;margin:0 0 24px 24px}.rst-content .align-left{float:left;margin:0 24px 24px 0}.rst-content .align-center{margin:auto}.rst-content .align-center:not(table){display:block}.rst-content .code-block-caption .headerlink,.rst-content .eqno .headerlink,.rst-content .toctree-wrapper>p.caption .headerlink,.rst-content dl dt .headerlink,.rst-content h1 .headerlink,.rst-content h2 .headerlink,.rst-content h3 .headerlink,.rst-content h4 .headerlink,.rst-content h5 .headerlink,.rst-content h6 .headerlink,.rst-content p.caption .headerlink,.rst-content p .headerlink,.rst-content table>caption .headerlink{opacity:0;font-size:14px;font-family:FontAwesome;margin-left:.5em}.rst-content .code-block-caption .headerlink:focus,.rst-content .code-block-caption:hover .headerlink,.rst-content .eqno .headerlink:focus,.rst-content .eqno:hover .headerlink,.rst-content .toctree-wrapper>p.caption .headerlink:focus,.rst-content .toctree-wrapper>p.caption:hover .headerlink,.rst-content dl dt .headerlink:focus,.rst-content dl dt:hover .headerlink,.rst-content h1 .headerlink:focus,.rst-content h1:hover .headerlink,.rst-content h2 .headerlink:focus,.rst-content h2:hover .headerlink,.rst-content h3 .headerlink:focus,.rst-content h3:hover .headerlink,.rst-content h4 .headerlink:focus,.rst-content h4:hover .headerlink,.rst-content h5 .headerlink:focus,.rst-content h5:hover .headerlink,.rst-content h6 .headerlink:focus,.rst-content h6:hover .headerlink,.rst-content p.caption .headerlink:focus,.rst-content p.caption:hover .headerlink,.rst-content p .headerlink:focus,.rst-content p:hover .headerlink,.rst-content table>caption .headerlink:focus,.rst-content table>caption:hover .headerlink{opacity:1}.rst-content p a{overflow-wrap:anywhere}.rst-content .wy-table td p,.rst-content .wy-table td ul,.rst-content .wy-table th p,.rst-content .wy-table th ul,.rst-content table.docutils td p,.rst-content table.docutils td ul,.rst-content table.docutils th p,.rst-content table.docutils th ul,.rst-content table.field-list td p,.rst-content table.field-list td ul,.rst-content table.field-list th p,.rst-content table.field-list th ul{font-size:inherit}.rst-content .btn:focus{outline:2px solid}.rst-content table>caption .headerlink:after{font-size:12px}.rst-content .centered{text-align:center}.rst-content .sidebar{float:right;width:40%;display:block;margin:0 0 24px 24px;padding:24px;background:#f3f6f6;border:1px solid #e1e4e5}.rst-content .sidebar dl,.rst-content .sidebar p,.rst-content .sidebar ul{font-size:90%}.rst-content .sidebar .last,.rst-content .sidebar>:last-child{margin-bottom:0}.rst-content .sidebar .sidebar-title{display:block;font-family:Roboto Slab,ff-tisa-web-pro,Georgia,Arial,sans-serif;font-weight:700;background:#e1e4e5;padding:6px 12px;margin:-24px -24px 24px;font-size:100%}.rst-content .highlighted{background:#f1c40f;box-shadow:0 0 0 2px #f1c40f;display:inline;font-weight:700}.rst-content .citation-reference,.rst-content .footnote-reference{vertical-align:baseline;position:relative;top:-.4em;line-height:0;font-size:90%}.rst-content .citation-reference>span.fn-bracket,.rst-content .footnote-reference>span.fn-bracket{display:none}.rst-content .hlist{width:100%}.rst-content dl dt span.classifier:before{content:" : "}.rst-content dl dt span.classifier-delimiter{display:none!important}html.writer-html4 .rst-content table.docutils.citation,html.writer-html4 .rst-content table.docutils.footnote{background:none;border:none}html.writer-html4 .rst-content table.docutils.citation td,html.writer-html4 .rst-content table.docutils.citation tr,html.writer-html4 .rst-content table.docutils.footnote td,html.writer-html4 .rst-content table.docutils.footnote tr{border:none;background-color:transparent!important;white-space:normal}html.writer-html4 .rst-content table.docutils.citation td.label,html.writer-html4 .rst-content table.docutils.footnote td.label{padding-left:0;padding-right:0;vertical-align:top}html.writer-html5 .rst-content dl.citation,html.writer-html5 .rst-content dl.field-list,html.writer-html5 .rst-content dl.footnote{display:grid;grid-template-columns:auto minmax(80%,95%)}html.writer-html5 .rst-content dl.citation>dt,html.writer-html5 .rst-content dl.field-list>dt,html.writer-html5 .rst-content dl.footnote>dt{display:inline-grid;grid-template-columns:max-content auto}html.writer-html5 .rst-content aside.citation,html.writer-html5 .rst-content aside.footnote,html.writer-html5 .rst-content div.citation{display:grid;grid-template-columns:auto auto minmax(.65rem,auto) minmax(40%,95%)}html.writer-html5 .rst-content aside.citation>span.label,html.writer-html5 .rst-content aside.footnote>span.label,html.writer-html5 .rst-content div.citation>span.label{grid-column-start:1;grid-column-end:2}html.writer-html5 .rst-content aside.citation>span.backrefs,html.writer-html5 .rst-content aside.footnote>span.backrefs,html.writer-html5 .rst-content div.citation>span.backrefs{grid-column-start:2;grid-column-end:3;grid-row-start:1;grid-row-end:3}html.writer-html5 .rst-content aside.citation>p,html.writer-html5 .rst-content aside.footnote>p,html.writer-html5 .rst-content div.citation>p{grid-column-start:4;grid-column-end:5}html.writer-html5 .rst-content dl.citation,html.writer-html5 .rst-content dl.field-list,html.writer-html5 .rst-content dl.footnote{margin-bottom:24px}html.writer-html5 .rst-content dl.citation>dt,html.writer-html5 .rst-content dl.field-list>dt,html.writer-html5 .rst-content dl.footnote>dt{padding-left:1rem}html.writer-html5 .rst-content dl.citation>dd,html.writer-html5 .rst-content dl.citation>dt,html.writer-html5 .rst-content dl.field-list>dd,html.writer-html5 .rst-content dl.field-list>dt,html.writer-html5 .rst-content dl.footnote>dd,html.writer-html5 .rst-content dl.footnote>dt{margin-bottom:0}html.writer-html5 .rst-content dl.citation,html.writer-html5 .rst-content dl.footnote{font-size:.9rem}html.writer-html5 .rst-content dl.citation>dt,html.writer-html5 .rst-content dl.footnote>dt{margin:0 .5rem .5rem 0;line-height:1.2rem;word-break:break-all;font-weight:400}html.writer-html5 .rst-content dl.citation>dt>span.brackets:before,html.writer-html5 .rst-content dl.footnote>dt>span.brackets:before{content:"["}html.writer-html5 .rst-content dl.citation>dt>span.brackets:after,html.writer-html5 .rst-content dl.footnote>dt>span.brackets:after{content:"]"}html.writer-html5 .rst-content dl.citation>dt>span.fn-backref,html.writer-html5 .rst-content dl.footnote>dt>span.fn-backref{text-align:left;font-style:italic;margin-left:.65rem;word-break:break-word;word-spacing:-.1rem;max-width:5rem}html.writer-html5 .rst-content dl.citation>dt>span.fn-backref>a,html.writer-html5 .rst-content dl.footnote>dt>span.fn-backref>a{word-break:keep-all}html.writer-html5 .rst-content dl.citation>dt>span.fn-backref>a:not(:first-child):before,html.writer-html5 .rst-content dl.footnote>dt>span.fn-backref>a:not(:first-child):before{content:" "}html.writer-html5 .rst-content dl.citation>dd,html.writer-html5 .rst-content dl.footnote>dd{margin:0 0 .5rem;line-height:1.2rem}html.writer-html5 .rst-content dl.citation>dd p,html.writer-html5 .rst-content dl.footnote>dd p{font-size:.9rem}html.writer-html5 .rst-content aside.citation,html.writer-html5 .rst-content aside.footnote,html.writer-html5 .rst-content div.citation{padding-left:1rem;padding-right:1rem;font-size:.9rem;line-height:1.2rem}html.writer-html5 .rst-content aside.citation p,html.writer-html5 .rst-content aside.footnote p,html.writer-html5 .rst-content div.citation p{font-size:.9rem;line-height:1.2rem;margin-bottom:12px}html.writer-html5 .rst-content aside.citation span.backrefs,html.writer-html5 .rst-content aside.footnote span.backrefs,html.writer-html5 .rst-content div.citation span.backrefs{text-align:left;font-style:italic;margin-left:.65rem;word-break:break-word;word-spacing:-.1rem;max-width:5rem}html.writer-html5 .rst-content aside.citation span.backrefs>a,html.writer-html5 .rst-content aside.footnote span.backrefs>a,html.writer-html5 .rst-content div.citation span.backrefs>a{word-break:keep-all}html.writer-html5 .rst-content aside.citation span.backrefs>a:not(:first-child):before,html.writer-html5 .rst-content aside.footnote span.backrefs>a:not(:first-child):before,html.writer-html5 .rst-content div.citation span.backrefs>a:not(:first-child):before{content:" "}html.writer-html5 .rst-content aside.citation span.label,html.writer-html5 .rst-content aside.footnote span.label,html.writer-html5 .rst-content div.citation span.label{line-height:1.2rem}html.writer-html5 .rst-content aside.citation-list,html.writer-html5 .rst-content aside.footnote-list,html.writer-html5 .rst-content div.citation-list{margin-bottom:24px}html.writer-html5 .rst-content dl.option-list kbd{font-size:.9rem}.rst-content table.docutils.footnote,html.writer-html4 .rst-content table.docutils.citation,html.writer-html5 .rst-content aside.footnote,html.writer-html5 .rst-content aside.footnote-list aside.footnote,html.writer-html5 .rst-content div.citation-list>div.citation,html.writer-html5 .rst-content dl.citation,html.writer-html5 .rst-content dl.footnote{color:grey}.rst-content table.docutils.footnote code,.rst-content table.docutils.footnote tt,html.writer-html4 .rst-content table.docutils.citation code,html.writer-html4 .rst-content table.docutils.citation tt,html.writer-html5 .rst-content aside.footnote-list aside.footnote code,html.writer-html5 .rst-content aside.footnote-list aside.footnote tt,html.writer-html5 .rst-content aside.footnote code,html.writer-html5 .rst-content aside.footnote tt,html.writer-html5 .rst-content div.citation-list>div.citation code,html.writer-html5 .rst-content div.citation-list>div.citation tt,html.writer-html5 .rst-content dl.citation code,html.writer-html5 .rst-content dl.citation tt,html.writer-html5 .rst-content dl.footnote code,html.writer-html5 .rst-content dl.footnote tt{color:#555}.rst-content .wy-table-responsive.citation,.rst-content .wy-table-responsive.footnote{margin-bottom:0}.rst-content .wy-table-responsive.citation+:not(.citation),.rst-content .wy-table-responsive.footnote+:not(.footnote){margin-top:24px}.rst-content .wy-table-responsive.citation:last-child,.rst-content .wy-table-responsive.footnote:last-child{margin-bottom:24px}.rst-content table.docutils th{border-color:#e1e4e5}html.writer-html5 .rst-content table.docutils th{border:1px solid #e1e4e5}html.writer-html5 .rst-content table.docutils td>p,html.writer-html5 .rst-content table.docutils th>p{line-height:1rem;margin-bottom:0;font-size:.9rem}.rst-content table.docutils td .last,.rst-content table.docutils td .last>:last-child{margin-bottom:0}.rst-content table.field-list,.rst-content table.field-list td{border:none}.rst-content table.field-list td p{line-height:inherit}.rst-content table.field-list td>strong{display:inline-block}.rst-content table.field-list .field-name{padding-right:10px;text-align:left;white-space:nowrap}.rst-content table.field-list .field-body{text-align:left}.rst-content code,.rst-content tt{color:#000;font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;padding:2px 5px}.rst-content code big,.rst-content code em,.rst-content tt big,.rst-content tt em{font-size:100%!important;line-height:normal}.rst-content code.literal,.rst-content tt.literal{color:#e74c3c;white-space:normal}.rst-content code.xref,.rst-content tt.xref,a .rst-content code,a .rst-content tt{font-weight:700;color:#404040;overflow-wrap:normal}.rst-content kbd,.rst-content pre,.rst-content samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace}.rst-content a code,.rst-content a tt{color:#2980b9}.rst-content dl{margin-bottom:24px}.rst-content dl dt{font-weight:700;margin-bottom:12px}.rst-content dl ol,.rst-content dl p,.rst-content dl table,.rst-content dl ul{margin-bottom:12px}.rst-content dl dd{margin:0 0 12px 24px;line-height:24px}.rst-content dl dd>ol:last-child,.rst-content dl dd>p:last-child,.rst-content dl dd>table:last-child,.rst-content dl dd>ul:last-child{margin-bottom:0}html.writer-html4 .rst-content dl:not(.docutils),html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple){margin-bottom:24px}html.writer-html4 .rst-content dl:not(.docutils)>dt,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt{display:table;margin:6px 0;font-size:90%;line-height:normal;background:#e7f2fa;color:#2980b9;border-top:3px solid #6ab0de;padding:6px;position:relative}html.writer-html4 .rst-content dl:not(.docutils)>dt:before,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt:before{color:#6ab0de}html.writer-html4 .rst-content dl:not(.docutils)>dt .headerlink,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt .headerlink{color:#404040;font-size:100%!important}html.writer-html4 .rst-content dl:not(.docutils) dl:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) dl:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt{margin-bottom:6px;border:none;border-left:3px solid #ccc;background:#f0f0f0;color:#555}html.writer-html4 .rst-content dl:not(.docutils) dl:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt .headerlink,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) dl:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt .headerlink{color:#404040;font-size:100%!important}html.writer-html4 .rst-content dl:not(.docutils)>dt:first-child,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple)>dt:first-child{margin-top:0}html.writer-html4 .rst-content dl:not(.docutils) code.descclassname,html.writer-html4 .rst-content dl:not(.docutils) code.descname,html.writer-html4 .rst-content dl:not(.docutils) tt.descclassname,html.writer-html4 .rst-content dl:not(.docutils) tt.descname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) code.descclassname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) code.descname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) tt.descclassname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) tt.descname{background-color:transparent;border:none;padding:0;font-size:100%!important}html.writer-html4 .rst-content dl:not(.docutils) code.descname,html.writer-html4 .rst-content dl:not(.docutils) tt.descname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) code.descname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) tt.descname{font-weight:700}html.writer-html4 .rst-content dl:not(.docutils) .optional,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .optional{display:inline-block;padding:0 4px;color:#000;font-weight:700}html.writer-html4 .rst-content dl:not(.docutils) .property,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .property{display:inline-block;padding-right:8px;max-width:100%}html.writer-html4 .rst-content dl:not(.docutils) .k,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .k{font-style:italic}html.writer-html4 .rst-content dl:not(.docutils) .descclassname,html.writer-html4 .rst-content dl:not(.docutils) .descname,html.writer-html4 .rst-content dl:not(.docutils) .sig-name,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .descclassname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .descname,html.writer-html5 .rst-content dl[class]:not(.option-list):not(.field-list):not(.footnote):not(.citation):not(.glossary):not(.simple) .sig-name{font-family:SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,Courier,monospace;color:#000}.rst-content .viewcode-back,.rst-content .viewcode-link{display:inline-block;color:#27ae60;font-size:80%;padding-left:24px}.rst-content .viewcode-back{display:block;float:right}.rst-content p.rubric{margin-bottom:12px;font-weight:700}.rst-content code.download,.rst-content tt.download{background:inherit;padding:inherit;font-weight:400;font-family:inherit;font-size:inherit;color:inherit;border:inherit;white-space:inherit}.rst-content code.download span:first-child,.rst-content tt.download span:first-child{-webkit-font-smoothing:subpixel-antialiased}.rst-content code.download span:first-child:before,.rst-content tt.download span:first-child:before{margin-right:4px}.rst-content .guilabel,.rst-content .menuselection{font-size:80%;font-weight:700;border-radius:4px;padding:2.4px 6px;margin:auto 2px}.rst-content .guilabel,.rst-content .menuselection{border:1px solid #7fbbe3;background:#e7f2fa}.rst-content :not(dl.option-list)>:not(dt):not(kbd):not(.kbd)>.kbd,.rst-content :not(dl.option-list)>:not(dt):not(kbd):not(.kbd)>kbd{color:inherit;font-size:80%;background-color:#fff;border:1px solid #a6a6a6;border-radius:4px;box-shadow:0 2px grey;padding:2.4px 6px;margin:auto 0}.rst-content .versionmodified{font-style:italic}@media screen and (max-width:480px){.rst-content .sidebar{width:100%}}span[id*=MathJax-Span]{color:#404040}.math{text-align:center}@font-face{font-family:Lato;src:url(fonts/lato-normal.woff2?bd03a2cc277bbbc338d464e679fe9942) format("woff2"),url(fonts/lato-normal.woff?27bd77b9162d388cb8d4c4217c7c5e2a) format("woff");font-weight:400;font-style:normal;font-display:block}@font-face{font-family:Lato;src:url(fonts/lato-bold.woff2?cccb897485813c7c256901dbca54ecf2) format("woff2"),url(fonts/lato-bold.woff?d878b6c29b10beca227e9eef4246111b) format("woff");font-weight:700;font-style:normal;font-display:block}@font-face{font-family:Lato;src:url(fonts/lato-bold-italic.woff2?0b6bb6725576b072c5d0b02ecdd1900d) format("woff2"),url(fonts/lato-bold-italic.woff?9c7e4e9eb485b4a121c760e61bc3707c) format("woff");font-weight:700;font-style:italic;font-display:block}@font-face{font-family:Lato;src:url(fonts/lato-normal-italic.woff2?4eb103b4d12be57cb1d040ed5e162e9d) format("woff2"),url(fonts/lato-normal-italic.woff?f28f2d6482446544ef1ea1ccc6dd5892) format("woff");font-weight:400;font-style:italic;font-display:block}@font-face{font-family:Roboto Slab;font-style:normal;font-weight:400;src:url(fonts/Roboto-Slab-Regular.woff2?7abf5b8d04d26a2cafea937019bca958) format("woff2"),url(fonts/Roboto-Slab-Regular.woff?c1be9284088d487c5e3ff0a10a92e58c) format("woff");font-display:block}@font-face{font-family:Roboto Slab;font-style:normal;font-weight:700;src:url(fonts/Roboto-Slab-Bold.woff2?9984f4a9bda09be08e83f2506954adbe) format("woff2"),url(fonts/Roboto-Slab-Bold.woff?bed5564a116b05148e3b3bea6fb1162a) format("woff");font-display:block} \ No newline at end of file diff --git a/_static/custom.css b/_static/custom.css new file mode 100644 index 0000000..c9cfdac --- /dev/null +++ b/_static/custom.css @@ -0,0 +1,3 @@ +div[role=contentinfo] { + display: none; +} diff --git a/_static/doctools.js b/_static/doctools.js new file mode 100644 index 0000000..d06a71d --- /dev/null +++ b/_static/doctools.js @@ -0,0 +1,156 @@ +/* + * doctools.js + * ~~~~~~~~~~~ + * + * Base JavaScript utilities for all Sphinx HTML documentation. + * + * :copyright: Copyright 2007-2023 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ +"use strict"; + +const BLACKLISTED_KEY_CONTROL_ELEMENTS = new Set([ + "TEXTAREA", + "INPUT", + "SELECT", + "BUTTON", +]); + +const _ready = (callback) => { + if (document.readyState !== "loading") { + callback(); + } else { + document.addEventListener("DOMContentLoaded", callback); + } +}; + +/** + * Small JavaScript module for the documentation. + */ +const Documentation = { + init: () => { + Documentation.initDomainIndexTable(); + Documentation.initOnKeyListeners(); + }, + + /** + * i18n support + */ + TRANSLATIONS: {}, + PLURAL_EXPR: (n) => (n === 1 ? 0 : 1), + LOCALE: "unknown", + + // gettext and ngettext don't access this so that the functions + // can safely bound to a different name (_ = Documentation.gettext) + gettext: (string) => { + const translated = Documentation.TRANSLATIONS[string]; + switch (typeof translated) { + case "undefined": + return string; // no translation + case "string": + return translated; // translation exists + default: + return translated[0]; // (singular, plural) translation tuple exists + } + }, + + ngettext: (singular, plural, n) => { + const translated = Documentation.TRANSLATIONS[singular]; + if (typeof translated !== "undefined") + return translated[Documentation.PLURAL_EXPR(n)]; + return n === 1 ? singular : plural; + }, + + addTranslations: (catalog) => { + Object.assign(Documentation.TRANSLATIONS, catalog.messages); + Documentation.PLURAL_EXPR = new Function( + "n", + `return (${catalog.plural_expr})` + ); + Documentation.LOCALE = catalog.locale; + }, + + /** + * helper function to focus on search bar + */ + focusSearchBar: () => { + document.querySelectorAll("input[name=q]")[0]?.focus(); + }, + + /** + * Initialise the domain index toggle buttons + */ + initDomainIndexTable: () => { + const toggler = (el) => { + const idNumber = el.id.substr(7); + const toggledRows = document.querySelectorAll(`tr.cg-${idNumber}`); + if (el.src.substr(-9) === "minus.png") { + el.src = `${el.src.substr(0, el.src.length - 9)}plus.png`; + toggledRows.forEach((el) => (el.style.display = "none")); + } else { + el.src = `${el.src.substr(0, el.src.length - 8)}minus.png`; + toggledRows.forEach((el) => (el.style.display = "")); + } + }; + + const togglerElements = document.querySelectorAll("img.toggler"); + togglerElements.forEach((el) => + el.addEventListener("click", (event) => toggler(event.currentTarget)) + ); + togglerElements.forEach((el) => (el.style.display = "")); + if (DOCUMENTATION_OPTIONS.COLLAPSE_INDEX) togglerElements.forEach(toggler); + }, + + initOnKeyListeners: () => { + // only install a listener if it is really needed + if ( + !DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS && + !DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS + ) + return; + + document.addEventListener("keydown", (event) => { + // bail for input elements + if (BLACKLISTED_KEY_CONTROL_ELEMENTS.has(document.activeElement.tagName)) return; + // bail with special keys + if (event.altKey || event.ctrlKey || event.metaKey) return; + + if (!event.shiftKey) { + switch (event.key) { + case "ArrowLeft": + if (!DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS) break; + + const prevLink = document.querySelector('link[rel="prev"]'); + if (prevLink && prevLink.href) { + window.location.href = prevLink.href; + event.preventDefault(); + } + break; + case "ArrowRight": + if (!DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS) break; + + const nextLink = document.querySelector('link[rel="next"]'); + if (nextLink && nextLink.href) { + window.location.href = nextLink.href; + event.preventDefault(); + } + break; + } + } + + // some keyboard layouts may need Shift to get / + switch (event.key) { + case "/": + if (!DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS) break; + Documentation.focusSearchBar(); + event.preventDefault(); + } + }); + }, +}; + +// quick alias for translations +const _ = Documentation.gettext; + +_ready(Documentation.init); diff --git a/_static/documentation_options.js b/_static/documentation_options.js new file mode 100644 index 0000000..7e4c114 --- /dev/null +++ b/_static/documentation_options.js @@ -0,0 +1,13 @@ +const DOCUMENTATION_OPTIONS = { + VERSION: '', + LANGUAGE: 'en', + COLLAPSE_INDEX: false, + BUILDER: 'html', + FILE_SUFFIX: '.html', + LINK_SUFFIX: '.html', + HAS_SOURCE: true, + SOURCELINK_SUFFIX: '.txt', + NAVIGATION_WITH_KEYS: false, + SHOW_SEARCH_SUMMARY: true, + ENABLE_SEARCH_SHORTCUTS: true, +}; \ No newline at end of file diff --git a/_static/file.png b/_static/file.png new file mode 100644 index 0000000..a858a41 Binary files /dev/null and b/_static/file.png differ diff --git a/_static/graphviz.css b/_static/graphviz.css new file mode 100644 index 0000000..8d81c02 --- /dev/null +++ b/_static/graphviz.css @@ -0,0 +1,19 @@ +/* + * graphviz.css + * ~~~~~~~~~~~~ + * + * Sphinx stylesheet -- graphviz extension. + * + * :copyright: Copyright 2007-2023 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ + +img.graphviz { + border: 0; + max-width: 100%; +} + +object.graphviz { + max-width: 100%; +} diff --git a/_static/jquery.js b/_static/jquery.js new file mode 100644 index 0000000..c4c6022 --- /dev/null +++ b/_static/jquery.js @@ -0,0 +1,2 @@ +/*! jQuery v3.6.0 | (c) OpenJS Foundation and other contributors | jquery.org/license */ +!function(e,t){"use strict";"object"==typeof module&&"object"==typeof module.exports?module.exports=e.document?t(e,!0):function(e){if(!e.document)throw new Error("jQuery requires a window with a document");return t(e)}:t(e)}("undefined"!=typeof window?window:this,function(C,e){"use strict";var t=[],r=Object.getPrototypeOf,s=t.slice,g=t.flat?function(e){return t.flat.call(e)}:function(e){return t.concat.apply([],e)},u=t.push,i=t.indexOf,n={},o=n.toString,v=n.hasOwnProperty,a=v.toString,l=a.call(Object),y={},m=function(e){return"function"==typeof e&&"number"!=typeof e.nodeType&&"function"!=typeof e.item},x=function(e){return null!=e&&e===e.window},E=C.document,c={type:!0,src:!0,nonce:!0,noModule:!0};function b(e,t,n){var r,i,o=(n=n||E).createElement("script");if(o.text=e,t)for(r in c)(i=t[r]||t.getAttribute&&t.getAttribute(r))&&o.setAttribute(r,i);n.head.appendChild(o).parentNode.removeChild(o)}function w(e){return null==e?e+"":"object"==typeof e||"function"==typeof e?n[o.call(e)]||"object":typeof e}var f="3.6.0",S=function(e,t){return new S.fn.init(e,t)};function p(e){var t=!!e&&"length"in e&&e.length,n=w(e);return!m(e)&&!x(e)&&("array"===n||0===t||"number"==typeof t&&0+~]|"+M+")"+M+"*"),U=new RegExp(M+"|>"),X=new RegExp(F),V=new RegExp("^"+I+"$"),G={ID:new RegExp("^#("+I+")"),CLASS:new RegExp("^\\.("+I+")"),TAG:new RegExp("^("+I+"|[*])"),ATTR:new RegExp("^"+W),PSEUDO:new RegExp("^"+F),CHILD:new RegExp("^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\("+M+"*(even|odd|(([+-]|)(\\d*)n|)"+M+"*(?:([+-]|)"+M+"*(\\d+)|))"+M+"*\\)|)","i"),bool:new RegExp("^(?:"+R+")$","i"),needsContext:new RegExp("^"+M+"*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\("+M+"*((?:-\\d)?\\d*)"+M+"*\\)|)(?=[^-]|$)","i")},Y=/HTML$/i,Q=/^(?:input|select|textarea|button)$/i,J=/^h\d$/i,K=/^[^{]+\{\s*\[native \w/,Z=/^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/,ee=/[+~]/,te=new RegExp("\\\\[\\da-fA-F]{1,6}"+M+"?|\\\\([^\\r\\n\\f])","g"),ne=function(e,t){var n="0x"+e.slice(1)-65536;return t||(n<0?String.fromCharCode(n+65536):String.fromCharCode(n>>10|55296,1023&n|56320))},re=/([\0-\x1f\x7f]|^-?\d)|^-$|[^\0-\x1f\x7f-\uFFFF\w-]/g,ie=function(e,t){return t?"\0"===e?"\ufffd":e.slice(0,-1)+"\\"+e.charCodeAt(e.length-1).toString(16)+" ":"\\"+e},oe=function(){T()},ae=be(function(e){return!0===e.disabled&&"fieldset"===e.nodeName.toLowerCase()},{dir:"parentNode",next:"legend"});try{H.apply(t=O.call(p.childNodes),p.childNodes),t[p.childNodes.length].nodeType}catch(e){H={apply:t.length?function(e,t){L.apply(e,O.call(t))}:function(e,t){var n=e.length,r=0;while(e[n++]=t[r++]);e.length=n-1}}}function se(t,e,n,r){var i,o,a,s,u,l,c,f=e&&e.ownerDocument,p=e?e.nodeType:9;if(n=n||[],"string"!=typeof t||!t||1!==p&&9!==p&&11!==p)return n;if(!r&&(T(e),e=e||C,E)){if(11!==p&&(u=Z.exec(t)))if(i=u[1]){if(9===p){if(!(a=e.getElementById(i)))return n;if(a.id===i)return n.push(a),n}else if(f&&(a=f.getElementById(i))&&y(e,a)&&a.id===i)return n.push(a),n}else{if(u[2])return H.apply(n,e.getElementsByTagName(t)),n;if((i=u[3])&&d.getElementsByClassName&&e.getElementsByClassName)return H.apply(n,e.getElementsByClassName(i)),n}if(d.qsa&&!N[t+" "]&&(!v||!v.test(t))&&(1!==p||"object"!==e.nodeName.toLowerCase())){if(c=t,f=e,1===p&&(U.test(t)||z.test(t))){(f=ee.test(t)&&ye(e.parentNode)||e)===e&&d.scope||((s=e.getAttribute("id"))?s=s.replace(re,ie):e.setAttribute("id",s=S)),o=(l=h(t)).length;while(o--)l[o]=(s?"#"+s:":scope")+" "+xe(l[o]);c=l.join(",")}try{return H.apply(n,f.querySelectorAll(c)),n}catch(e){N(t,!0)}finally{s===S&&e.removeAttribute("id")}}}return g(t.replace($,"$1"),e,n,r)}function ue(){var r=[];return function e(t,n){return r.push(t+" ")>b.cacheLength&&delete e[r.shift()],e[t+" "]=n}}function le(e){return e[S]=!0,e}function ce(e){var t=C.createElement("fieldset");try{return!!e(t)}catch(e){return!1}finally{t.parentNode&&t.parentNode.removeChild(t),t=null}}function fe(e,t){var n=e.split("|"),r=n.length;while(r--)b.attrHandle[n[r]]=t}function pe(e,t){var n=t&&e,r=n&&1===e.nodeType&&1===t.nodeType&&e.sourceIndex-t.sourceIndex;if(r)return r;if(n)while(n=n.nextSibling)if(n===t)return-1;return e?1:-1}function de(t){return function(e){return"input"===e.nodeName.toLowerCase()&&e.type===t}}function he(n){return function(e){var t=e.nodeName.toLowerCase();return("input"===t||"button"===t)&&e.type===n}}function ge(t){return function(e){return"form"in e?e.parentNode&&!1===e.disabled?"label"in e?"label"in e.parentNode?e.parentNode.disabled===t:e.disabled===t:e.isDisabled===t||e.isDisabled!==!t&&ae(e)===t:e.disabled===t:"label"in e&&e.disabled===t}}function ve(a){return le(function(o){return o=+o,le(function(e,t){var n,r=a([],e.length,o),i=r.length;while(i--)e[n=r[i]]&&(e[n]=!(t[n]=e[n]))})})}function ye(e){return e&&"undefined"!=typeof e.getElementsByTagName&&e}for(e in d=se.support={},i=se.isXML=function(e){var t=e&&e.namespaceURI,n=e&&(e.ownerDocument||e).documentElement;return!Y.test(t||n&&n.nodeName||"HTML")},T=se.setDocument=function(e){var t,n,r=e?e.ownerDocument||e:p;return r!=C&&9===r.nodeType&&r.documentElement&&(a=(C=r).documentElement,E=!i(C),p!=C&&(n=C.defaultView)&&n.top!==n&&(n.addEventListener?n.addEventListener("unload",oe,!1):n.attachEvent&&n.attachEvent("onunload",oe)),d.scope=ce(function(e){return a.appendChild(e).appendChild(C.createElement("div")),"undefined"!=typeof e.querySelectorAll&&!e.querySelectorAll(":scope fieldset div").length}),d.attributes=ce(function(e){return e.className="i",!e.getAttribute("className")}),d.getElementsByTagName=ce(function(e){return e.appendChild(C.createComment("")),!e.getElementsByTagName("*").length}),d.getElementsByClassName=K.test(C.getElementsByClassName),d.getById=ce(function(e){return a.appendChild(e).id=S,!C.getElementsByName||!C.getElementsByName(S).length}),d.getById?(b.filter.ID=function(e){var t=e.replace(te,ne);return function(e){return e.getAttribute("id")===t}},b.find.ID=function(e,t){if("undefined"!=typeof t.getElementById&&E){var n=t.getElementById(e);return n?[n]:[]}}):(b.filter.ID=function(e){var n=e.replace(te,ne);return function(e){var t="undefined"!=typeof e.getAttributeNode&&e.getAttributeNode("id");return t&&t.value===n}},b.find.ID=function(e,t){if("undefined"!=typeof t.getElementById&&E){var n,r,i,o=t.getElementById(e);if(o){if((n=o.getAttributeNode("id"))&&n.value===e)return[o];i=t.getElementsByName(e),r=0;while(o=i[r++])if((n=o.getAttributeNode("id"))&&n.value===e)return[o]}return[]}}),b.find.TAG=d.getElementsByTagName?function(e,t){return"undefined"!=typeof t.getElementsByTagName?t.getElementsByTagName(e):d.qsa?t.querySelectorAll(e):void 0}:function(e,t){var n,r=[],i=0,o=t.getElementsByTagName(e);if("*"===e){while(n=o[i++])1===n.nodeType&&r.push(n);return r}return o},b.find.CLASS=d.getElementsByClassName&&function(e,t){if("undefined"!=typeof t.getElementsByClassName&&E)return t.getElementsByClassName(e)},s=[],v=[],(d.qsa=K.test(C.querySelectorAll))&&(ce(function(e){var t;a.appendChild(e).innerHTML="",e.querySelectorAll("[msallowcapture^='']").length&&v.push("[*^$]="+M+"*(?:''|\"\")"),e.querySelectorAll("[selected]").length||v.push("\\["+M+"*(?:value|"+R+")"),e.querySelectorAll("[id~="+S+"-]").length||v.push("~="),(t=C.createElement("input")).setAttribute("name",""),e.appendChild(t),e.querySelectorAll("[name='']").length||v.push("\\["+M+"*name"+M+"*="+M+"*(?:''|\"\")"),e.querySelectorAll(":checked").length||v.push(":checked"),e.querySelectorAll("a#"+S+"+*").length||v.push(".#.+[+~]"),e.querySelectorAll("\\\f"),v.push("[\\r\\n\\f]")}),ce(function(e){e.innerHTML="";var t=C.createElement("input");t.setAttribute("type","hidden"),e.appendChild(t).setAttribute("name","D"),e.querySelectorAll("[name=d]").length&&v.push("name"+M+"*[*^$|!~]?="),2!==e.querySelectorAll(":enabled").length&&v.push(":enabled",":disabled"),a.appendChild(e).disabled=!0,2!==e.querySelectorAll(":disabled").length&&v.push(":enabled",":disabled"),e.querySelectorAll("*,:x"),v.push(",.*:")})),(d.matchesSelector=K.test(c=a.matches||a.webkitMatchesSelector||a.mozMatchesSelector||a.oMatchesSelector||a.msMatchesSelector))&&ce(function(e){d.disconnectedMatch=c.call(e,"*"),c.call(e,"[s!='']:x"),s.push("!=",F)}),v=v.length&&new RegExp(v.join("|")),s=s.length&&new RegExp(s.join("|")),t=K.test(a.compareDocumentPosition),y=t||K.test(a.contains)?function(e,t){var n=9===e.nodeType?e.documentElement:e,r=t&&t.parentNode;return e===r||!(!r||1!==r.nodeType||!(n.contains?n.contains(r):e.compareDocumentPosition&&16&e.compareDocumentPosition(r)))}:function(e,t){if(t)while(t=t.parentNode)if(t===e)return!0;return!1},j=t?function(e,t){if(e===t)return l=!0,0;var n=!e.compareDocumentPosition-!t.compareDocumentPosition;return n||(1&(n=(e.ownerDocument||e)==(t.ownerDocument||t)?e.compareDocumentPosition(t):1)||!d.sortDetached&&t.compareDocumentPosition(e)===n?e==C||e.ownerDocument==p&&y(p,e)?-1:t==C||t.ownerDocument==p&&y(p,t)?1:u?P(u,e)-P(u,t):0:4&n?-1:1)}:function(e,t){if(e===t)return l=!0,0;var n,r=0,i=e.parentNode,o=t.parentNode,a=[e],s=[t];if(!i||!o)return e==C?-1:t==C?1:i?-1:o?1:u?P(u,e)-P(u,t):0;if(i===o)return pe(e,t);n=e;while(n=n.parentNode)a.unshift(n);n=t;while(n=n.parentNode)s.unshift(n);while(a[r]===s[r])r++;return r?pe(a[r],s[r]):a[r]==p?-1:s[r]==p?1:0}),C},se.matches=function(e,t){return se(e,null,null,t)},se.matchesSelector=function(e,t){if(T(e),d.matchesSelector&&E&&!N[t+" "]&&(!s||!s.test(t))&&(!v||!v.test(t)))try{var n=c.call(e,t);if(n||d.disconnectedMatch||e.document&&11!==e.document.nodeType)return n}catch(e){N(t,!0)}return 0":{dir:"parentNode",first:!0}," ":{dir:"parentNode"},"+":{dir:"previousSibling",first:!0},"~":{dir:"previousSibling"}},preFilter:{ATTR:function(e){return e[1]=e[1].replace(te,ne),e[3]=(e[3]||e[4]||e[5]||"").replace(te,ne),"~="===e[2]&&(e[3]=" "+e[3]+" "),e.slice(0,4)},CHILD:function(e){return e[1]=e[1].toLowerCase(),"nth"===e[1].slice(0,3)?(e[3]||se.error(e[0]),e[4]=+(e[4]?e[5]+(e[6]||1):2*("even"===e[3]||"odd"===e[3])),e[5]=+(e[7]+e[8]||"odd"===e[3])):e[3]&&se.error(e[0]),e},PSEUDO:function(e){var t,n=!e[6]&&e[2];return G.CHILD.test(e[0])?null:(e[3]?e[2]=e[4]||e[5]||"":n&&X.test(n)&&(t=h(n,!0))&&(t=n.indexOf(")",n.length-t)-n.length)&&(e[0]=e[0].slice(0,t),e[2]=n.slice(0,t)),e.slice(0,3))}},filter:{TAG:function(e){var t=e.replace(te,ne).toLowerCase();return"*"===e?function(){return!0}:function(e){return e.nodeName&&e.nodeName.toLowerCase()===t}},CLASS:function(e){var t=m[e+" "];return t||(t=new RegExp("(^|"+M+")"+e+"("+M+"|$)"))&&m(e,function(e){return t.test("string"==typeof e.className&&e.className||"undefined"!=typeof e.getAttribute&&e.getAttribute("class")||"")})},ATTR:function(n,r,i){return function(e){var t=se.attr(e,n);return null==t?"!="===r:!r||(t+="","="===r?t===i:"!="===r?t!==i:"^="===r?i&&0===t.indexOf(i):"*="===r?i&&-1:\x20\t\r\n\f]*)[\x20\t\r\n\f]*\/?>(?:<\/\1>|)$/i;function j(e,n,r){return m(n)?S.grep(e,function(e,t){return!!n.call(e,t,e)!==r}):n.nodeType?S.grep(e,function(e){return e===n!==r}):"string"!=typeof n?S.grep(e,function(e){return-1)[^>]*|#([\w-]+))$/;(S.fn.init=function(e,t,n){var r,i;if(!e)return this;if(n=n||D,"string"==typeof e){if(!(r="<"===e[0]&&">"===e[e.length-1]&&3<=e.length?[null,e,null]:q.exec(e))||!r[1]&&t)return!t||t.jquery?(t||n).find(e):this.constructor(t).find(e);if(r[1]){if(t=t instanceof S?t[0]:t,S.merge(this,S.parseHTML(r[1],t&&t.nodeType?t.ownerDocument||t:E,!0)),N.test(r[1])&&S.isPlainObject(t))for(r in t)m(this[r])?this[r](t[r]):this.attr(r,t[r]);return this}return(i=E.getElementById(r[2]))&&(this[0]=i,this.length=1),this}return e.nodeType?(this[0]=e,this.length=1,this):m(e)?void 0!==n.ready?n.ready(e):e(S):S.makeArray(e,this)}).prototype=S.fn,D=S(E);var L=/^(?:parents|prev(?:Until|All))/,H={children:!0,contents:!0,next:!0,prev:!0};function O(e,t){while((e=e[t])&&1!==e.nodeType);return e}S.fn.extend({has:function(e){var t=S(e,this),n=t.length;return this.filter(function(){for(var e=0;e\x20\t\r\n\f]*)/i,he=/^$|^module$|\/(?:java|ecma)script/i;ce=E.createDocumentFragment().appendChild(E.createElement("div")),(fe=E.createElement("input")).setAttribute("type","radio"),fe.setAttribute("checked","checked"),fe.setAttribute("name","t"),ce.appendChild(fe),y.checkClone=ce.cloneNode(!0).cloneNode(!0).lastChild.checked,ce.innerHTML="",y.noCloneChecked=!!ce.cloneNode(!0).lastChild.defaultValue,ce.innerHTML="",y.option=!!ce.lastChild;var ge={thead:[1,"","
"],col:[2,"","
"],tr:[2,"","
"],td:[3,"","
"],_default:[0,"",""]};function ve(e,t){var n;return n="undefined"!=typeof e.getElementsByTagName?e.getElementsByTagName(t||"*"):"undefined"!=typeof e.querySelectorAll?e.querySelectorAll(t||"*"):[],void 0===t||t&&A(e,t)?S.merge([e],n):n}function ye(e,t){for(var n=0,r=e.length;n",""]);var me=/<|&#?\w+;/;function xe(e,t,n,r,i){for(var o,a,s,u,l,c,f=t.createDocumentFragment(),p=[],d=0,h=e.length;d\s*$/g;function je(e,t){return A(e,"table")&&A(11!==t.nodeType?t:t.firstChild,"tr")&&S(e).children("tbody")[0]||e}function De(e){return e.type=(null!==e.getAttribute("type"))+"/"+e.type,e}function qe(e){return"true/"===(e.type||"").slice(0,5)?e.type=e.type.slice(5):e.removeAttribute("type"),e}function Le(e,t){var n,r,i,o,a,s;if(1===t.nodeType){if(Y.hasData(e)&&(s=Y.get(e).events))for(i in Y.remove(t,"handle events"),s)for(n=0,r=s[i].length;n").attr(n.scriptAttrs||{}).prop({charset:n.scriptCharset,src:n.url}).on("load error",i=function(e){r.remove(),i=null,e&&t("error"===e.type?404:200,e.type)}),E.head.appendChild(r[0])},abort:function(){i&&i()}}});var _t,zt=[],Ut=/(=)\?(?=&|$)|\?\?/;S.ajaxSetup({jsonp:"callback",jsonpCallback:function(){var e=zt.pop()||S.expando+"_"+wt.guid++;return this[e]=!0,e}}),S.ajaxPrefilter("json jsonp",function(e,t,n){var r,i,o,a=!1!==e.jsonp&&(Ut.test(e.url)?"url":"string"==typeof e.data&&0===(e.contentType||"").indexOf("application/x-www-form-urlencoded")&&Ut.test(e.data)&&"data");if(a||"jsonp"===e.dataTypes[0])return r=e.jsonpCallback=m(e.jsonpCallback)?e.jsonpCallback():e.jsonpCallback,a?e[a]=e[a].replace(Ut,"$1"+r):!1!==e.jsonp&&(e.url+=(Tt.test(e.url)?"&":"?")+e.jsonp+"="+r),e.converters["script json"]=function(){return o||S.error(r+" was not called"),o[0]},e.dataTypes[0]="json",i=C[r],C[r]=function(){o=arguments},n.always(function(){void 0===i?S(C).removeProp(r):C[r]=i,e[r]&&(e.jsonpCallback=t.jsonpCallback,zt.push(r)),o&&m(i)&&i(o[0]),o=i=void 0}),"script"}),y.createHTMLDocument=((_t=E.implementation.createHTMLDocument("").body).innerHTML="
",2===_t.childNodes.length),S.parseHTML=function(e,t,n){return"string"!=typeof e?[]:("boolean"==typeof t&&(n=t,t=!1),t||(y.createHTMLDocument?((r=(t=E.implementation.createHTMLDocument("")).createElement("base")).href=E.location.href,t.head.appendChild(r)):t=E),o=!n&&[],(i=N.exec(e))?[t.createElement(i[1])]:(i=xe([e],t,o),o&&o.length&&S(o).remove(),S.merge([],i.childNodes)));var r,i,o},S.fn.load=function(e,t,n){var r,i,o,a=this,s=e.indexOf(" ");return-1").append(S.parseHTML(e)).find(r):e)}).always(n&&function(e,t){a.each(function(){n.apply(this,o||[e.responseText,t,e])})}),this},S.expr.pseudos.animated=function(t){return S.grep(S.timers,function(e){return t===e.elem}).length},S.offset={setOffset:function(e,t,n){var r,i,o,a,s,u,l=S.css(e,"position"),c=S(e),f={};"static"===l&&(e.style.position="relative"),s=c.offset(),o=S.css(e,"top"),u=S.css(e,"left"),("absolute"===l||"fixed"===l)&&-1<(o+u).indexOf("auto")?(a=(r=c.position()).top,i=r.left):(a=parseFloat(o)||0,i=parseFloat(u)||0),m(t)&&(t=t.call(e,n,S.extend({},s))),null!=t.top&&(f.top=t.top-s.top+a),null!=t.left&&(f.left=t.left-s.left+i),"using"in t?t.using.call(e,f):c.css(f)}},S.fn.extend({offset:function(t){if(arguments.length)return void 0===t?this:this.each(function(e){S.offset.setOffset(this,t,e)});var e,n,r=this[0];return r?r.getClientRects().length?(e=r.getBoundingClientRect(),n=r.ownerDocument.defaultView,{top:e.top+n.pageYOffset,left:e.left+n.pageXOffset}):{top:0,left:0}:void 0},position:function(){if(this[0]){var e,t,n,r=this[0],i={top:0,left:0};if("fixed"===S.css(r,"position"))t=r.getBoundingClientRect();else{t=this.offset(),n=r.ownerDocument,e=r.offsetParent||n.documentElement;while(e&&(e===n.body||e===n.documentElement)&&"static"===S.css(e,"position"))e=e.parentNode;e&&e!==r&&1===e.nodeType&&((i=S(e).offset()).top+=S.css(e,"borderTopWidth",!0),i.left+=S.css(e,"borderLeftWidth",!0))}return{top:t.top-i.top-S.css(r,"marginTop",!0),left:t.left-i.left-S.css(r,"marginLeft",!0)}}},offsetParent:function(){return this.map(function(){var e=this.offsetParent;while(e&&"static"===S.css(e,"position"))e=e.offsetParent;return e||re})}}),S.each({scrollLeft:"pageXOffset",scrollTop:"pageYOffset"},function(t,i){var o="pageYOffset"===i;S.fn[t]=function(e){return $(this,function(e,t,n){var r;if(x(e)?r=e:9===e.nodeType&&(r=e.defaultView),void 0===n)return r?r[i]:e[t];r?r.scrollTo(o?r.pageXOffset:n,o?n:r.pageYOffset):e[t]=n},t,e,arguments.length)}}),S.each(["top","left"],function(e,n){S.cssHooks[n]=Fe(y.pixelPosition,function(e,t){if(t)return t=We(e,n),Pe.test(t)?S(e).position()[n]+"px":t})}),S.each({Height:"height",Width:"width"},function(a,s){S.each({padding:"inner"+a,content:s,"":"outer"+a},function(r,o){S.fn[o]=function(e,t){var n=arguments.length&&(r||"boolean"!=typeof e),i=r||(!0===e||!0===t?"margin":"border");return $(this,function(e,t,n){var r;return x(e)?0===o.indexOf("outer")?e["inner"+a]:e.document.documentElement["client"+a]:9===e.nodeType?(r=e.documentElement,Math.max(e.body["scroll"+a],r["scroll"+a],e.body["offset"+a],r["offset"+a],r["client"+a])):void 0===n?S.css(e,t,i):S.style(e,t,n,i)},s,n?e:void 0,n)}})}),S.each(["ajaxStart","ajaxStop","ajaxComplete","ajaxError","ajaxSuccess","ajaxSend"],function(e,t){S.fn[t]=function(e){return this.on(t,e)}}),S.fn.extend({bind:function(e,t,n){return this.on(e,null,t,n)},unbind:function(e,t){return this.off(e,null,t)},delegate:function(e,t,n,r){return this.on(t,e,n,r)},undelegate:function(e,t,n){return 1===arguments.length?this.off(e,"**"):this.off(t,e||"**",n)},hover:function(e,t){return this.mouseenter(e).mouseleave(t||e)}}),S.each("blur focus focusin focusout resize scroll click dblclick mousedown mouseup mousemove mouseover mouseout mouseenter mouseleave change select submit keydown keypress keyup contextmenu".split(" "),function(e,n){S.fn[n]=function(e,t){return 0",d.insertBefore(c.lastChild,d.firstChild)}function d(){var a=y.elements;return"string"==typeof a?a.split(" "):a}function e(a,b){var c=y.elements;"string"!=typeof c&&(c=c.join(" ")),"string"!=typeof a&&(a=a.join(" ")),y.elements=c+" "+a,j(b)}function f(a){var b=x[a[v]];return b||(b={},w++,a[v]=w,x[w]=b),b}function g(a,c,d){if(c||(c=b),q)return c.createElement(a);d||(d=f(c));var e;return e=d.cache[a]?d.cache[a].cloneNode():u.test(a)?(d.cache[a]=d.createElem(a)).cloneNode():d.createElem(a),!e.canHaveChildren||t.test(a)||e.tagUrn?e:d.frag.appendChild(e)}function h(a,c){if(a||(a=b),q)return a.createDocumentFragment();c=c||f(a);for(var e=c.frag.cloneNode(),g=0,h=d(),i=h.length;i>g;g++)e.createElement(h[g]);return e}function i(a,b){b.cache||(b.cache={},b.createElem=a.createElement,b.createFrag=a.createDocumentFragment,b.frag=b.createFrag()),a.createElement=function(c){return y.shivMethods?g(c,a,b):b.createElem(c)},a.createDocumentFragment=Function("h,f","return function(){var n=f.cloneNode(),c=n.createElement;h.shivMethods&&("+d().join().replace(/[\w\-:]+/g,function(a){return b.createElem(a),b.frag.createElement(a),'c("'+a+'")'})+");return n}")(y,b.frag)}function j(a){a||(a=b);var d=f(a);return!y.shivCSS||p||d.hasCSS||(d.hasCSS=!!c(a,"article,aside,dialog,figcaption,figure,footer,header,hgroup,main,nav,section{display:block}mark{background:#FF0;color:#000}template{display:none}")),q||i(a,d),a}function k(a){for(var b,c=a.getElementsByTagName("*"),e=c.length,f=RegExp("^(?:"+d().join("|")+")$","i"),g=[];e--;)b=c[e],f.test(b.nodeName)&&g.push(b.applyElement(l(b)));return g}function l(a){for(var b,c=a.attributes,d=c.length,e=a.ownerDocument.createElement(A+":"+a.nodeName);d--;)b=c[d],b.specified&&e.setAttribute(b.nodeName,b.nodeValue);return e.style.cssText=a.style.cssText,e}function m(a){for(var b,c=a.split("{"),e=c.length,f=RegExp("(^|[\\s,>+~])("+d().join("|")+")(?=[[\\s,>+~#.:]|$)","gi"),g="$1"+A+"\\:$2";e--;)b=c[e]=c[e].split("}"),b[b.length-1]=b[b.length-1].replace(f,g),c[e]=b.join("}");return c.join("{")}function n(a){for(var b=a.length;b--;)a[b].removeNode()}function o(a){function b(){clearTimeout(g._removeSheetTimer),d&&d.removeNode(!0),d=null}var d,e,g=f(a),h=a.namespaces,i=a.parentWindow;return!B||a.printShived?a:("undefined"==typeof h[A]&&h.add(A),i.attachEvent("onbeforeprint",function(){b();for(var f,g,h,i=a.styleSheets,j=[],l=i.length,n=Array(l);l--;)n[l]=i[l];for(;h=n.pop();)if(!h.disabled&&z.test(h.media)){try{f=h.imports,g=f.length}catch(o){g=0}for(l=0;g>l;l++)n.push(f[l]);try{j.push(h.cssText)}catch(o){}}j=m(j.reverse().join("")),e=k(a),d=c(a,j)}),i.attachEvent("onafterprint",function(){n(e),clearTimeout(g._removeSheetTimer),g._removeSheetTimer=setTimeout(b,500)}),a.printShived=!0,a)}var p,q,r="3.7.3",s=a.html5||{},t=/^<|^(?:button|map|select|textarea|object|iframe|option|optgroup)$/i,u=/^(?:a|b|code|div|fieldset|h1|h2|h3|h4|h5|h6|i|label|li|ol|p|q|span|strong|style|table|tbody|td|th|tr|ul)$/i,v="_html5shiv",w=0,x={};!function(){try{var a=b.createElement("a");a.innerHTML="",p="hidden"in a,q=1==a.childNodes.length||function(){b.createElement("a");var a=b.createDocumentFragment();return"undefined"==typeof a.cloneNode||"undefined"==typeof a.createDocumentFragment||"undefined"==typeof a.createElement}()}catch(c){p=!0,q=!0}}();var y={elements:s.elements||"abbr article aside audio bdi canvas data datalist details dialog figcaption figure footer header hgroup main mark meter nav output picture progress section summary template time video",version:r,shivCSS:s.shivCSS!==!1,supportsUnknownElements:q,shivMethods:s.shivMethods!==!1,type:"default",shivDocument:j,createElement:g,createDocumentFragment:h,addElements:e};a.html5=y,j(b);var z=/^$|\b(?:all|print)\b/,A="html5shiv",B=!q&&function(){var c=b.documentElement;return!("undefined"==typeof b.namespaces||"undefined"==typeof b.parentWindow||"undefined"==typeof c.applyElement||"undefined"==typeof c.removeNode||"undefined"==typeof a.attachEvent)}();y.type+=" print",y.shivPrint=o,o(b),"object"==typeof module&&module.exports&&(module.exports=y)}("undefined"!=typeof window?window:this,document); \ No newline at end of file diff --git a/_static/js/html5shiv.min.js b/_static/js/html5shiv.min.js new file mode 100644 index 0000000..cd1c674 --- /dev/null +++ b/_static/js/html5shiv.min.js @@ -0,0 +1,4 @@ +/** +* @preserve HTML5 Shiv 3.7.3 | @afarkas @jdalton @jon_neal @rem | MIT/GPL2 Licensed +*/ +!function(a,b){function c(a,b){var c=a.createElement("p"),d=a.getElementsByTagName("head")[0]||a.documentElement;return c.innerHTML="x",d.insertBefore(c.lastChild,d.firstChild)}function d(){var a=t.elements;return"string"==typeof a?a.split(" "):a}function e(a,b){var c=t.elements;"string"!=typeof c&&(c=c.join(" ")),"string"!=typeof a&&(a=a.join(" ")),t.elements=c+" "+a,j(b)}function f(a){var b=s[a[q]];return b||(b={},r++,a[q]=r,s[r]=b),b}function g(a,c,d){if(c||(c=b),l)return c.createElement(a);d||(d=f(c));var e;return e=d.cache[a]?d.cache[a].cloneNode():p.test(a)?(d.cache[a]=d.createElem(a)).cloneNode():d.createElem(a),!e.canHaveChildren||o.test(a)||e.tagUrn?e:d.frag.appendChild(e)}function h(a,c){if(a||(a=b),l)return a.createDocumentFragment();c=c||f(a);for(var e=c.frag.cloneNode(),g=0,h=d(),i=h.length;i>g;g++)e.createElement(h[g]);return e}function i(a,b){b.cache||(b.cache={},b.createElem=a.createElement,b.createFrag=a.createDocumentFragment,b.frag=b.createFrag()),a.createElement=function(c){return t.shivMethods?g(c,a,b):b.createElem(c)},a.createDocumentFragment=Function("h,f","return function(){var n=f.cloneNode(),c=n.createElement;h.shivMethods&&("+d().join().replace(/[\w\-:]+/g,function(a){return b.createElem(a),b.frag.createElement(a),'c("'+a+'")'})+");return n}")(t,b.frag)}function j(a){a||(a=b);var d=f(a);return!t.shivCSS||k||d.hasCSS||(d.hasCSS=!!c(a,"article,aside,dialog,figcaption,figure,footer,header,hgroup,main,nav,section{display:block}mark{background:#FF0;color:#000}template{display:none}")),l||i(a,d),a}var k,l,m="3.7.3-pre",n=a.html5||{},o=/^<|^(?:button|map|select|textarea|object|iframe|option|optgroup)$/i,p=/^(?:a|b|code|div|fieldset|h1|h2|h3|h4|h5|h6|i|label|li|ol|p|q|span|strong|style|table|tbody|td|th|tr|ul)$/i,q="_html5shiv",r=0,s={};!function(){try{var a=b.createElement("a");a.innerHTML="",k="hidden"in a,l=1==a.childNodes.length||function(){b.createElement("a");var a=b.createDocumentFragment();return"undefined"==typeof a.cloneNode||"undefined"==typeof a.createDocumentFragment||"undefined"==typeof a.createElement}()}catch(c){k=!0,l=!0}}();var t={elements:n.elements||"abbr article aside audio bdi canvas data datalist details dialog figcaption figure footer header hgroup main mark meter nav output picture progress section summary template time video",version:m,shivCSS:n.shivCSS!==!1,supportsUnknownElements:l,shivMethods:n.shivMethods!==!1,type:"default",shivDocument:j,createElement:g,createDocumentFragment:h,addElements:e};a.html5=t,j(b),"object"==typeof module&&module.exports&&(module.exports=t)}("undefined"!=typeof window?window:this,document); \ No newline at end of file diff --git a/_static/js/theme.js b/_static/js/theme.js new file mode 100644 index 0000000..1fddb6e --- /dev/null +++ b/_static/js/theme.js @@ -0,0 +1 @@ +!function(n){var e={};function t(i){if(e[i])return e[i].exports;var o=e[i]={i:i,l:!1,exports:{}};return n[i].call(o.exports,o,o.exports,t),o.l=!0,o.exports}t.m=n,t.c=e,t.d=function(n,e,i){t.o(n,e)||Object.defineProperty(n,e,{enumerable:!0,get:i})},t.r=function(n){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(n,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(n,"__esModule",{value:!0})},t.t=function(n,e){if(1&e&&(n=t(n)),8&e)return n;if(4&e&&"object"==typeof n&&n&&n.__esModule)return n;var i=Object.create(null);if(t.r(i),Object.defineProperty(i,"default",{enumerable:!0,value:n}),2&e&&"string"!=typeof n)for(var o in n)t.d(i,o,function(e){return n[e]}.bind(null,o));return i},t.n=function(n){var e=n&&n.__esModule?function(){return n.default}:function(){return n};return t.d(e,"a",e),e},t.o=function(n,e){return Object.prototype.hasOwnProperty.call(n,e)},t.p="",t(t.s=0)}([function(n,e,t){t(1),n.exports=t(3)},function(n,e,t){(function(){var e="undefined"!=typeof window?window.jQuery:t(2);n.exports.ThemeNav={navBar:null,win:null,winScroll:!1,winResize:!1,linkScroll:!1,winPosition:0,winHeight:null,docHeight:null,isRunning:!1,enable:function(n){var t=this;void 0===n&&(n=!0),t.isRunning||(t.isRunning=!0,e((function(e){t.init(e),t.reset(),t.win.on("hashchange",t.reset),n&&t.win.on("scroll",(function(){t.linkScroll||t.winScroll||(t.winScroll=!0,requestAnimationFrame((function(){t.onScroll()})))})),t.win.on("resize",(function(){t.winResize||(t.winResize=!0,requestAnimationFrame((function(){t.onResize()})))})),t.onResize()})))},enableSticky:function(){this.enable(!0)},init:function(n){n(document);var e=this;this.navBar=n("div.wy-side-scroll:first"),this.win=n(window),n(document).on("click","[data-toggle='wy-nav-top']",(function(){n("[data-toggle='wy-nav-shift']").toggleClass("shift"),n("[data-toggle='rst-versions']").toggleClass("shift")})).on("click",".wy-menu-vertical .current ul li a",(function(){var t=n(this);n("[data-toggle='wy-nav-shift']").removeClass("shift"),n("[data-toggle='rst-versions']").toggleClass("shift"),e.toggleCurrent(t),e.hashChange()})).on("click","[data-toggle='rst-current-version']",(function(){n("[data-toggle='rst-versions']").toggleClass("shift-up")})),n("table.docutils:not(.field-list,.footnote,.citation)").wrap("
"),n("table.docutils.footnote").wrap("
"),n("table.docutils.citation").wrap("
"),n(".wy-menu-vertical ul").not(".simple").siblings("a").each((function(){var t=n(this);expand=n(''),expand.on("click",(function(n){return e.toggleCurrent(t),n.stopPropagation(),!1})),t.prepend(expand)}))},reset:function(){var n=encodeURI(window.location.hash)||"#";try{var e=$(".wy-menu-vertical"),t=e.find('[href="'+n+'"]');if(0===t.length){var i=$('.document [id="'+n.substring(1)+'"]').closest("div.section");0===(t=e.find('[href="#'+i.attr("id")+'"]')).length&&(t=e.find('[href="#"]'))}if(t.length>0){$(".wy-menu-vertical .current").removeClass("current").attr("aria-expanded","false"),t.addClass("current").attr("aria-expanded","true"),t.closest("li.toctree-l1").parent().addClass("current").attr("aria-expanded","true");for(let n=1;n<=10;n++)t.closest("li.toctree-l"+n).addClass("current").attr("aria-expanded","true");t[0].scrollIntoView()}}catch(n){console.log("Error expanding nav for anchor",n)}},onScroll:function(){this.winScroll=!1;var n=this.win.scrollTop(),e=n+this.winHeight,t=this.navBar.scrollTop()+(n-this.winPosition);n<0||e>this.docHeight||(this.navBar.scrollTop(t),this.winPosition=n)},onResize:function(){this.winResize=!1,this.winHeight=this.win.height(),this.docHeight=$(document).height()},hashChange:function(){this.linkScroll=!0,this.win.one("hashchange",(function(){this.linkScroll=!1}))},toggleCurrent:function(n){var e=n.closest("li");e.siblings("li.current").removeClass("current").attr("aria-expanded","false"),e.siblings().find("li.current").removeClass("current").attr("aria-expanded","false");var t=e.find("> ul li");t.length&&(t.removeClass("current").attr("aria-expanded","false"),e.toggleClass("current").attr("aria-expanded",(function(n,e){return"true"==e?"false":"true"})))}},"undefined"!=typeof window&&(window.SphinxRtdTheme={Navigation:n.exports.ThemeNav,StickyNav:n.exports.ThemeNav}),function(){for(var n=0,e=["ms","moz","webkit","o"],t=0;t0 + var meq1 = "^(" + C + ")?" + V + C + "(" + V + ")?$"; // [C]VC[V] is m=1 + var mgr1 = "^(" + C + ")?" + V + C + V + C; // [C]VCVC... is m>1 + var s_v = "^(" + C + ")?" + v; // vowel in stem + + this.stemWord = function (w) { + var stem; + var suffix; + var firstch; + var origword = w; + + if (w.length < 3) + return w; + + var re; + var re2; + var re3; + var re4; + + firstch = w.substr(0,1); + if (firstch == "y") + w = firstch.toUpperCase() + w.substr(1); + + // Step 1a + re = /^(.+?)(ss|i)es$/; + re2 = /^(.+?)([^s])s$/; + + if (re.test(w)) + w = w.replace(re,"$1$2"); + else if (re2.test(w)) + w = w.replace(re2,"$1$2"); + + // Step 1b + re = /^(.+?)eed$/; + re2 = /^(.+?)(ed|ing)$/; + if (re.test(w)) { + var fp = re.exec(w); + re = new RegExp(mgr0); + if (re.test(fp[1])) { + re = /.$/; + w = w.replace(re,""); + } + } + else if (re2.test(w)) { + var fp = re2.exec(w); + stem = fp[1]; + re2 = new RegExp(s_v); + if (re2.test(stem)) { + w = stem; + re2 = /(at|bl|iz)$/; + re3 = new RegExp("([^aeiouylsz])\\1$"); + re4 = new RegExp("^" + C + v + "[^aeiouwxy]$"); + if (re2.test(w)) + w = w + "e"; + else if (re3.test(w)) { + re = /.$/; + w = w.replace(re,""); + } + else if (re4.test(w)) + w = w + "e"; + } + } + + // Step 1c + re = /^(.+?)y$/; + if (re.test(w)) { + var fp = re.exec(w); + stem = fp[1]; + re = new RegExp(s_v); + if (re.test(stem)) + w = stem + "i"; + } + + // Step 2 + re = /^(.+?)(ational|tional|enci|anci|izer|bli|alli|entli|eli|ousli|ization|ation|ator|alism|iveness|fulness|ousness|aliti|iviti|biliti|logi)$/; + if (re.test(w)) { + var fp = re.exec(w); + stem = fp[1]; + suffix = fp[2]; + re = new RegExp(mgr0); + if (re.test(stem)) + w = stem + step2list[suffix]; + } + + // Step 3 + re = /^(.+?)(icate|ative|alize|iciti|ical|ful|ness)$/; + if (re.test(w)) { + var fp = re.exec(w); + stem = fp[1]; + suffix = fp[2]; + re = new RegExp(mgr0); + if (re.test(stem)) + w = stem + step3list[suffix]; + } + + // Step 4 + re = /^(.+?)(al|ance|ence|er|ic|able|ible|ant|ement|ment|ent|ou|ism|ate|iti|ous|ive|ize)$/; + re2 = /^(.+?)(s|t)(ion)$/; + if (re.test(w)) { + var fp = re.exec(w); + stem = fp[1]; + re = new RegExp(mgr1); + if (re.test(stem)) + w = stem; + } + else if (re2.test(w)) { + var fp = re2.exec(w); + stem = fp[1] + fp[2]; + re2 = new RegExp(mgr1); + if (re2.test(stem)) + w = stem; + } + + // Step 5 + re = /^(.+?)e$/; + if (re.test(w)) { + var fp = re.exec(w); + stem = fp[1]; + re = new RegExp(mgr1); + re2 = new RegExp(meq1); + re3 = new RegExp("^" + C + v + "[^aeiouwxy]$"); + if (re.test(stem) || (re2.test(stem) && !(re3.test(stem)))) + w = stem; + } + re = /ll$/; + re2 = new RegExp(mgr1); + if (re.test(w) && re2.test(w)) { + re = /.$/; + w = w.replace(re,""); + } + + // and turn initial Y back to y + if (firstch == "y") + w = firstch.toLowerCase() + w.substr(1); + return w; + } +} + diff --git a/_static/minus.png b/_static/minus.png new file mode 100644 index 0000000..d96755f Binary files /dev/null and b/_static/minus.png differ diff --git a/_static/nbsphinx-broken-thumbnail.svg b/_static/nbsphinx-broken-thumbnail.svg new file mode 100644 index 0000000..4919ca8 --- /dev/null +++ b/_static/nbsphinx-broken-thumbnail.svg @@ -0,0 +1,9 @@ + + + + diff --git a/_static/nbsphinx-code-cells.css b/_static/nbsphinx-code-cells.css new file mode 100644 index 0000000..a3fb27c --- /dev/null +++ b/_static/nbsphinx-code-cells.css @@ -0,0 +1,259 @@ +/* remove conflicting styling from Sphinx themes */ +div.nbinput.container div.prompt *, +div.nboutput.container div.prompt *, +div.nbinput.container div.input_area pre, +div.nboutput.container div.output_area pre, +div.nbinput.container div.input_area .highlight, +div.nboutput.container div.output_area .highlight { + border: none; + padding: 0; + margin: 0; + box-shadow: none; +} + +div.nbinput.container > div[class*=highlight], +div.nboutput.container > div[class*=highlight] { + margin: 0; +} + +div.nbinput.container div.prompt *, +div.nboutput.container div.prompt * { + background: none; +} + +div.nboutput.container div.output_area .highlight, +div.nboutput.container div.output_area pre { + background: unset; +} + +div.nboutput.container div.output_area div.highlight { + color: unset; /* override Pygments text color */ +} + +/* avoid gaps between output lines */ +div.nboutput.container div[class*=highlight] pre { + line-height: normal; +} + +/* input/output containers */ +div.nbinput.container, +div.nboutput.container { + display: -webkit-flex; + display: flex; + align-items: flex-start; + margin: 0; + width: 100%; +} +@media (max-width: 540px) { + div.nbinput.container, + div.nboutput.container { + flex-direction: column; + } +} + +/* input container */ +div.nbinput.container { + padding-top: 5px; +} + +/* last container */ +div.nblast.container { + padding-bottom: 5px; +} + +/* input prompt */ +div.nbinput.container div.prompt pre, +/* for sphinx_immaterial theme: */ +div.nbinput.container div.prompt pre > code { + color: #307FC1; +} + +/* output prompt */ +div.nboutput.container div.prompt pre, +/* for sphinx_immaterial theme: */ +div.nboutput.container div.prompt pre > code { + color: #BF5B3D; +} + +/* all prompts */ +div.nbinput.container div.prompt, +div.nboutput.container div.prompt { + width: 4.5ex; + padding-top: 5px; + position: relative; + user-select: none; +} + +div.nbinput.container div.prompt > div, +div.nboutput.container div.prompt > div { + position: absolute; + right: 0; + margin-right: 0.3ex; +} + +@media (max-width: 540px) { + div.nbinput.container div.prompt, + div.nboutput.container div.prompt { + width: unset; + text-align: left; + padding: 0.4em; + } + div.nboutput.container div.prompt.empty { + padding: 0; + } + + div.nbinput.container div.prompt > div, + div.nboutput.container div.prompt > div { + position: unset; + } +} + +/* disable scrollbars and line breaks on prompts */ +div.nbinput.container div.prompt pre, +div.nboutput.container div.prompt pre { + overflow: hidden; + white-space: pre; +} + +/* input/output area */ +div.nbinput.container div.input_area, +div.nboutput.container div.output_area { + -webkit-flex: 1; + flex: 1; + overflow: auto; +} +@media (max-width: 540px) { + div.nbinput.container div.input_area, + div.nboutput.container div.output_area { + width: 100%; + } +} + +/* input area */ +div.nbinput.container div.input_area { + border: 1px solid #e0e0e0; + border-radius: 2px; + /*background: #f5f5f5;*/ +} + +/* override MathJax center alignment in output cells */ +div.nboutput.container div[class*=MathJax] { + text-align: left !important; +} + +/* override sphinx.ext.imgmath center alignment in output cells */ +div.nboutput.container div.math p { + text-align: left; +} + +/* standard error */ +div.nboutput.container div.output_area.stderr { + background: #fdd; +} + +/* ANSI colors */ +.ansi-black-fg { color: #3E424D; } +.ansi-black-bg { background-color: #3E424D; } +.ansi-black-intense-fg { color: #282C36; } +.ansi-black-intense-bg { background-color: #282C36; } +.ansi-red-fg { color: #E75C58; } +.ansi-red-bg { background-color: #E75C58; } +.ansi-red-intense-fg { color: #B22B31; } +.ansi-red-intense-bg { background-color: #B22B31; } +.ansi-green-fg { color: #00A250; } +.ansi-green-bg { background-color: #00A250; } +.ansi-green-intense-fg { color: #007427; } +.ansi-green-intense-bg { background-color: #007427; } +.ansi-yellow-fg { color: #DDB62B; } +.ansi-yellow-bg { background-color: #DDB62B; } +.ansi-yellow-intense-fg { color: #B27D12; } +.ansi-yellow-intense-bg { background-color: #B27D12; } +.ansi-blue-fg { color: #208FFB; } +.ansi-blue-bg { background-color: #208FFB; } +.ansi-blue-intense-fg { color: #0065CA; } +.ansi-blue-intense-bg { background-color: #0065CA; } +.ansi-magenta-fg { color: #D160C4; } +.ansi-magenta-bg { background-color: #D160C4; } +.ansi-magenta-intense-fg { color: #A03196; } +.ansi-magenta-intense-bg { background-color: #A03196; } +.ansi-cyan-fg { color: #60C6C8; } +.ansi-cyan-bg { background-color: #60C6C8; } +.ansi-cyan-intense-fg { color: #258F8F; } +.ansi-cyan-intense-bg { background-color: #258F8F; } +.ansi-white-fg { color: #C5C1B4; } +.ansi-white-bg { background-color: #C5C1B4; } +.ansi-white-intense-fg { color: #A1A6B2; } +.ansi-white-intense-bg { background-color: #A1A6B2; } + +.ansi-default-inverse-fg { color: #FFFFFF; } +.ansi-default-inverse-bg { background-color: #000000; } + +.ansi-bold { font-weight: bold; } +.ansi-underline { text-decoration: underline; } + + +div.nbinput.container div.input_area div[class*=highlight] > pre, +div.nboutput.container div.output_area div[class*=highlight] > pre, +div.nboutput.container div.output_area div[class*=highlight].math, +div.nboutput.container div.output_area.rendered_html, +div.nboutput.container div.output_area > div.output_javascript, +div.nboutput.container div.output_area:not(.rendered_html) > img{ + padding: 5px; + margin: 0; +} + +/* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */ +div.nbinput.container div.input_area > div[class^='highlight'], +div.nboutput.container div.output_area > div[class^='highlight']{ + overflow-y: hidden; +} + +/* hide copy button on prompts for 'sphinx_copybutton' extension ... */ +.prompt .copybtn, +/* ... and 'sphinx_immaterial' theme */ +.prompt .md-clipboard.md-icon { + display: none; +} + +/* Some additional styling taken form the Jupyter notebook CSS */ +.jp-RenderedHTMLCommon table, +div.rendered_html table { + border: none; + border-collapse: collapse; + border-spacing: 0; + color: black; + font-size: 12px; + table-layout: fixed; +} +.jp-RenderedHTMLCommon thead, +div.rendered_html thead { + border-bottom: 1px solid black; + vertical-align: bottom; +} +.jp-RenderedHTMLCommon tr, +.jp-RenderedHTMLCommon th, +.jp-RenderedHTMLCommon td, +div.rendered_html tr, +div.rendered_html th, +div.rendered_html td { + text-align: right; + vertical-align: middle; + padding: 0.5em 0.5em; + line-height: normal; + white-space: normal; + max-width: none; + border: none; +} +.jp-RenderedHTMLCommon th, +div.rendered_html th { + font-weight: bold; +} +.jp-RenderedHTMLCommon tbody tr:nth-child(odd), +div.rendered_html tbody tr:nth-child(odd) { + background: #f5f5f5; +} +.jp-RenderedHTMLCommon tbody tr:hover, +div.rendered_html tbody tr:hover { + background: rgba(66, 165, 245, 0.2); +} + diff --git a/_static/nbsphinx-gallery.css b/_static/nbsphinx-gallery.css new file mode 100644 index 0000000..365c27a --- /dev/null +++ b/_static/nbsphinx-gallery.css @@ -0,0 +1,31 @@ +.nbsphinx-gallery { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(160px, 1fr)); + gap: 5px; + margin-top: 1em; + margin-bottom: 1em; +} + +.nbsphinx-gallery > a { + padding: 5px; + border: 1px dotted currentColor; + border-radius: 2px; + text-align: center; +} + +.nbsphinx-gallery > a:hover { + border-style: solid; +} + +.nbsphinx-gallery img { + max-width: 100%; + max-height: 100%; +} + +.nbsphinx-gallery > a > div:first-child { + display: flex; + align-items: start; + justify-content: center; + height: 120px; + margin-bottom: 5px; +} diff --git a/_static/nbsphinx-no-thumbnail.svg b/_static/nbsphinx-no-thumbnail.svg new file mode 100644 index 0000000..9dca758 --- /dev/null +++ b/_static/nbsphinx-no-thumbnail.svg @@ -0,0 +1,9 @@ + + + + diff --git a/_static/plus.png b/_static/plus.png new file mode 100644 index 0000000..7107cec Binary files /dev/null and b/_static/plus.png differ diff --git a/_static/pygments.css b/_static/pygments.css new file mode 100644 index 0000000..84ab303 --- /dev/null +++ b/_static/pygments.css @@ -0,0 +1,75 @@ +pre { line-height: 125%; } +td.linenos .normal { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; } +span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; } +td.linenos .special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; } +span.linenos.special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; } +.highlight .hll { background-color: #ffffcc } +.highlight { background: #f8f8f8; } +.highlight .c { color: #3D7B7B; font-style: italic } /* Comment */ +.highlight .err { border: 1px solid #FF0000 } /* Error */ +.highlight .k { color: #008000; font-weight: bold } /* Keyword */ +.highlight .o { color: #666666 } /* Operator */ +.highlight .ch { color: #3D7B7B; font-style: italic } /* Comment.Hashbang */ +.highlight .cm { color: #3D7B7B; font-style: italic } /* Comment.Multiline */ +.highlight .cp { color: #9C6500 } /* Comment.Preproc */ +.highlight .cpf { color: #3D7B7B; font-style: italic } /* Comment.PreprocFile */ +.highlight .c1 { color: #3D7B7B; font-style: italic } /* Comment.Single */ +.highlight .cs { color: #3D7B7B; font-style: italic } /* Comment.Special */ +.highlight .gd { color: #A00000 } /* Generic.Deleted */ +.highlight .ge { font-style: italic } /* Generic.Emph */ +.highlight .ges { font-weight: bold; font-style: italic } /* Generic.EmphStrong */ +.highlight .gr { color: #E40000 } /* Generic.Error */ +.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */ +.highlight .gi { color: #008400 } /* Generic.Inserted */ +.highlight .go { color: #717171 } /* Generic.Output */ +.highlight .gp { color: #000080; font-weight: bold } /* Generic.Prompt */ +.highlight .gs { font-weight: bold } /* Generic.Strong */ +.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */ +.highlight .gt { color: #0044DD } /* Generic.Traceback */ +.highlight .kc { color: #008000; font-weight: bold } /* Keyword.Constant */ +.highlight .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */ +.highlight .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */ +.highlight .kp { color: #008000 } /* Keyword.Pseudo */ +.highlight .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */ +.highlight .kt { color: #B00040 } /* Keyword.Type */ +.highlight .m { color: #666666 } /* Literal.Number */ +.highlight .s { color: #BA2121 } /* Literal.String */ +.highlight .na { color: #687822 } /* Name.Attribute */ +.highlight .nb { color: #008000 } /* Name.Builtin */ +.highlight .nc { color: #0000FF; font-weight: bold } /* Name.Class */ +.highlight .no { color: #880000 } /* Name.Constant */ +.highlight .nd { color: #AA22FF } /* Name.Decorator */ +.highlight .ni { color: #717171; font-weight: bold } /* Name.Entity */ +.highlight .ne { color: #CB3F38; font-weight: bold } /* Name.Exception */ +.highlight .nf { color: #0000FF } /* Name.Function */ +.highlight .nl { color: #767600 } /* Name.Label */ +.highlight .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */ +.highlight .nt { color: #008000; font-weight: bold } /* Name.Tag */ +.highlight .nv { color: #19177C } /* Name.Variable */ +.highlight .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */ +.highlight .w { color: #bbbbbb } /* Text.Whitespace */ +.highlight .mb { color: #666666 } /* Literal.Number.Bin */ +.highlight .mf { color: #666666 } /* Literal.Number.Float */ +.highlight .mh { color: #666666 } /* Literal.Number.Hex */ +.highlight .mi { color: #666666 } /* Literal.Number.Integer */ +.highlight .mo { color: #666666 } /* Literal.Number.Oct */ +.highlight .sa { color: #BA2121 } /* Literal.String.Affix */ +.highlight .sb { color: #BA2121 } /* Literal.String.Backtick */ +.highlight .sc { color: #BA2121 } /* Literal.String.Char */ +.highlight .dl { color: #BA2121 } /* Literal.String.Delimiter */ +.highlight .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */ +.highlight .s2 { color: #BA2121 } /* Literal.String.Double */ +.highlight .se { color: #AA5D1F; font-weight: bold } /* Literal.String.Escape */ +.highlight .sh { color: #BA2121 } /* Literal.String.Heredoc */ +.highlight .si { color: #A45A77; font-weight: bold } /* Literal.String.Interpol */ +.highlight .sx { color: #008000 } /* Literal.String.Other */ +.highlight .sr { color: #A45A77 } /* Literal.String.Regex */ +.highlight .s1 { color: #BA2121 } /* Literal.String.Single */ +.highlight .ss { color: #19177C } /* Literal.String.Symbol */ +.highlight .bp { color: #008000 } /* Name.Builtin.Pseudo */ +.highlight .fm { color: #0000FF } /* Name.Function.Magic */ +.highlight .vc { color: #19177C } /* Name.Variable.Class */ +.highlight .vg { color: #19177C } /* Name.Variable.Global */ +.highlight .vi { color: #19177C } /* Name.Variable.Instance */ +.highlight .vm { color: #19177C } /* Name.Variable.Magic */ +.highlight .il { color: #666666 } /* Literal.Number.Integer.Long */ \ No newline at end of file diff --git a/_static/searchtools.js b/_static/searchtools.js new file mode 100644 index 0000000..7918c3f --- /dev/null +++ b/_static/searchtools.js @@ -0,0 +1,574 @@ +/* + * searchtools.js + * ~~~~~~~~~~~~~~~~ + * + * Sphinx JavaScript utilities for the full-text search. + * + * :copyright: Copyright 2007-2023 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ +"use strict"; + +/** + * Simple result scoring code. + */ +if (typeof Scorer === "undefined") { + var Scorer = { + // Implement the following function to further tweak the score for each result + // The function takes a result array [docname, title, anchor, descr, score, filename] + // and returns the new score. + /* + score: result => { + const [docname, title, anchor, descr, score, filename] = result + return score + }, + */ + + // query matches the full name of an object + objNameMatch: 11, + // or matches in the last dotted part of the object name + objPartialMatch: 6, + // Additive scores depending on the priority of the object + objPrio: { + 0: 15, // used to be importantResults + 1: 5, // used to be objectResults + 2: -5, // used to be unimportantResults + }, + // Used when the priority is not in the mapping. + objPrioDefault: 0, + + // query found in title + title: 15, + partialTitle: 7, + // query found in terms + term: 5, + partialTerm: 2, + }; +} + +const _removeChildren = (element) => { + while (element && element.lastChild) element.removeChild(element.lastChild); +}; + +/** + * See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions#escaping + */ +const _escapeRegExp = (string) => + string.replace(/[.*+\-?^${}()|[\]\\]/g, "\\$&"); // $& means the whole matched string + +const _displayItem = (item, searchTerms, highlightTerms) => { + const docBuilder = DOCUMENTATION_OPTIONS.BUILDER; + const docFileSuffix = DOCUMENTATION_OPTIONS.FILE_SUFFIX; + const docLinkSuffix = DOCUMENTATION_OPTIONS.LINK_SUFFIX; + const showSearchSummary = DOCUMENTATION_OPTIONS.SHOW_SEARCH_SUMMARY; + const contentRoot = document.documentElement.dataset.content_root; + + const [docName, title, anchor, descr, score, _filename] = item; + + let listItem = document.createElement("li"); + let requestUrl; + let linkUrl; + if (docBuilder === "dirhtml") { + // dirhtml builder + let dirname = docName + "/"; + if (dirname.match(/\/index\/$/)) + dirname = dirname.substring(0, dirname.length - 6); + else if (dirname === "index/") dirname = ""; + requestUrl = contentRoot + dirname; + linkUrl = requestUrl; + } else { + // normal html builders + requestUrl = contentRoot + docName + docFileSuffix; + linkUrl = docName + docLinkSuffix; + } + let linkEl = listItem.appendChild(document.createElement("a")); + linkEl.href = linkUrl + anchor; + linkEl.dataset.score = score; + linkEl.innerHTML = title; + if (descr) { + listItem.appendChild(document.createElement("span")).innerHTML = + " (" + descr + ")"; + // highlight search terms in the description + if (SPHINX_HIGHLIGHT_ENABLED) // set in sphinx_highlight.js + highlightTerms.forEach((term) => _highlightText(listItem, term, "highlighted")); + } + else if (showSearchSummary) + fetch(requestUrl) + .then((responseData) => responseData.text()) + .then((data) => { + if (data) + listItem.appendChild( + Search.makeSearchSummary(data, searchTerms) + ); + // highlight search terms in the summary + if (SPHINX_HIGHLIGHT_ENABLED) // set in sphinx_highlight.js + highlightTerms.forEach((term) => _highlightText(listItem, term, "highlighted")); + }); + Search.output.appendChild(listItem); +}; +const _finishSearch = (resultCount) => { + Search.stopPulse(); + Search.title.innerText = _("Search Results"); + if (!resultCount) + Search.status.innerText = Documentation.gettext( + "Your search did not match any documents. Please make sure that all words are spelled correctly and that you've selected enough categories." + ); + else + Search.status.innerText = _( + `Search finished, found ${resultCount} page(s) matching the search query.` + ); +}; +const _displayNextItem = ( + results, + resultCount, + searchTerms, + highlightTerms, +) => { + // results left, load the summary and display it + // this is intended to be dynamic (don't sub resultsCount) + if (results.length) { + _displayItem(results.pop(), searchTerms, highlightTerms); + setTimeout( + () => _displayNextItem(results, resultCount, searchTerms, highlightTerms), + 5 + ); + } + // search finished, update title and status message + else _finishSearch(resultCount); +}; + +/** + * Default splitQuery function. Can be overridden in ``sphinx.search`` with a + * custom function per language. + * + * The regular expression works by splitting the string on consecutive characters + * that are not Unicode letters, numbers, underscores, or emoji characters. + * This is the same as ``\W+`` in Python, preserving the surrogate pair area. + */ +if (typeof splitQuery === "undefined") { + var splitQuery = (query) => query + .split(/[^\p{Letter}\p{Number}_\p{Emoji_Presentation}]+/gu) + .filter(term => term) // remove remaining empty strings +} + +/** + * Search Module + */ +const Search = { + _index: null, + _queued_query: null, + _pulse_status: -1, + + htmlToText: (htmlString) => { + const htmlElement = new DOMParser().parseFromString(htmlString, 'text/html'); + htmlElement.querySelectorAll(".headerlink").forEach((el) => { el.remove() }); + const docContent = htmlElement.querySelector('[role="main"]'); + if (docContent !== undefined) return docContent.textContent; + console.warn( + "Content block not found. Sphinx search tries to obtain it via '[role=main]'. Could you check your theme or template." + ); + return ""; + }, + + init: () => { + const query = new URLSearchParams(window.location.search).get("q"); + document + .querySelectorAll('input[name="q"]') + .forEach((el) => (el.value = query)); + if (query) Search.performSearch(query); + }, + + loadIndex: (url) => + (document.body.appendChild(document.createElement("script")).src = url), + + setIndex: (index) => { + Search._index = index; + if (Search._queued_query !== null) { + const query = Search._queued_query; + Search._queued_query = null; + Search.query(query); + } + }, + + hasIndex: () => Search._index !== null, + + deferQuery: (query) => (Search._queued_query = query), + + stopPulse: () => (Search._pulse_status = -1), + + startPulse: () => { + if (Search._pulse_status >= 0) return; + + const pulse = () => { + Search._pulse_status = (Search._pulse_status + 1) % 4; + Search.dots.innerText = ".".repeat(Search._pulse_status); + if (Search._pulse_status >= 0) window.setTimeout(pulse, 500); + }; + pulse(); + }, + + /** + * perform a search for something (or wait until index is loaded) + */ + performSearch: (query) => { + // create the required interface elements + const searchText = document.createElement("h2"); + searchText.textContent = _("Searching"); + const searchSummary = document.createElement("p"); + searchSummary.classList.add("search-summary"); + searchSummary.innerText = ""; + const searchList = document.createElement("ul"); + searchList.classList.add("search"); + + const out = document.getElementById("search-results"); + Search.title = out.appendChild(searchText); + Search.dots = Search.title.appendChild(document.createElement("span")); + Search.status = out.appendChild(searchSummary); + Search.output = out.appendChild(searchList); + + const searchProgress = document.getElementById("search-progress"); + // Some themes don't use the search progress node + if (searchProgress) { + searchProgress.innerText = _("Preparing search..."); + } + Search.startPulse(); + + // index already loaded, the browser was quick! + if (Search.hasIndex()) Search.query(query); + else Search.deferQuery(query); + }, + + /** + * execute search (requires search index to be loaded) + */ + query: (query) => { + const filenames = Search._index.filenames; + const docNames = Search._index.docnames; + const titles = Search._index.titles; + const allTitles = Search._index.alltitles; + const indexEntries = Search._index.indexentries; + + // stem the search terms and add them to the correct list + const stemmer = new Stemmer(); + const searchTerms = new Set(); + const excludedTerms = new Set(); + const highlightTerms = new Set(); + const objectTerms = new Set(splitQuery(query.toLowerCase().trim())); + splitQuery(query.trim()).forEach((queryTerm) => { + const queryTermLower = queryTerm.toLowerCase(); + + // maybe skip this "word" + // stopwords array is from language_data.js + if ( + stopwords.indexOf(queryTermLower) !== -1 || + queryTerm.match(/^\d+$/) + ) + return; + + // stem the word + let word = stemmer.stemWord(queryTermLower); + // select the correct list + if (word[0] === "-") excludedTerms.add(word.substr(1)); + else { + searchTerms.add(word); + highlightTerms.add(queryTermLower); + } + }); + + if (SPHINX_HIGHLIGHT_ENABLED) { // set in sphinx_highlight.js + localStorage.setItem("sphinx_highlight_terms", [...highlightTerms].join(" ")) + } + + // console.debug("SEARCH: searching for:"); + // console.info("required: ", [...searchTerms]); + // console.info("excluded: ", [...excludedTerms]); + + // array of [docname, title, anchor, descr, score, filename] + let results = []; + _removeChildren(document.getElementById("search-progress")); + + const queryLower = query.toLowerCase(); + for (const [title, foundTitles] of Object.entries(allTitles)) { + if (title.toLowerCase().includes(queryLower) && (queryLower.length >= title.length/2)) { + for (const [file, id] of foundTitles) { + let score = Math.round(100 * queryLower.length / title.length) + results.push([ + docNames[file], + titles[file] !== title ? `${titles[file]} > ${title}` : title, + id !== null ? "#" + id : "", + null, + score, + filenames[file], + ]); + } + } + } + + // search for explicit entries in index directives + for (const [entry, foundEntries] of Object.entries(indexEntries)) { + if (entry.includes(queryLower) && (queryLower.length >= entry.length/2)) { + for (const [file, id] of foundEntries) { + let score = Math.round(100 * queryLower.length / entry.length) + results.push([ + docNames[file], + titles[file], + id ? "#" + id : "", + null, + score, + filenames[file], + ]); + } + } + } + + // lookup as object + objectTerms.forEach((term) => + results.push(...Search.performObjectSearch(term, objectTerms)) + ); + + // lookup as search terms in fulltext + results.push(...Search.performTermsSearch(searchTerms, excludedTerms)); + + // let the scorer override scores with a custom scoring function + if (Scorer.score) results.forEach((item) => (item[4] = Scorer.score(item))); + + // now sort the results by score (in opposite order of appearance, since the + // display function below uses pop() to retrieve items) and then + // alphabetically + results.sort((a, b) => { + const leftScore = a[4]; + const rightScore = b[4]; + if (leftScore === rightScore) { + // same score: sort alphabetically + const leftTitle = a[1].toLowerCase(); + const rightTitle = b[1].toLowerCase(); + if (leftTitle === rightTitle) return 0; + return leftTitle > rightTitle ? -1 : 1; // inverted is intentional + } + return leftScore > rightScore ? 1 : -1; + }); + + // remove duplicate search results + // note the reversing of results, so that in the case of duplicates, the highest-scoring entry is kept + let seen = new Set(); + results = results.reverse().reduce((acc, result) => { + let resultStr = result.slice(0, 4).concat([result[5]]).map(v => String(v)).join(','); + if (!seen.has(resultStr)) { + acc.push(result); + seen.add(resultStr); + } + return acc; + }, []); + + results = results.reverse(); + + // for debugging + //Search.lastresults = results.slice(); // a copy + // console.info("search results:", Search.lastresults); + + // print the results + _displayNextItem(results, results.length, searchTerms, highlightTerms); + }, + + /** + * search for object names + */ + performObjectSearch: (object, objectTerms) => { + const filenames = Search._index.filenames; + const docNames = Search._index.docnames; + const objects = Search._index.objects; + const objNames = Search._index.objnames; + const titles = Search._index.titles; + + const results = []; + + const objectSearchCallback = (prefix, match) => { + const name = match[4] + const fullname = (prefix ? prefix + "." : "") + name; + const fullnameLower = fullname.toLowerCase(); + if (fullnameLower.indexOf(object) < 0) return; + + let score = 0; + const parts = fullnameLower.split("."); + + // check for different match types: exact matches of full name or + // "last name" (i.e. last dotted part) + if (fullnameLower === object || parts.slice(-1)[0] === object) + score += Scorer.objNameMatch; + else if (parts.slice(-1)[0].indexOf(object) > -1) + score += Scorer.objPartialMatch; // matches in last name + + const objName = objNames[match[1]][2]; + const title = titles[match[0]]; + + // If more than one term searched for, we require other words to be + // found in the name/title/description + const otherTerms = new Set(objectTerms); + otherTerms.delete(object); + if (otherTerms.size > 0) { + const haystack = `${prefix} ${name} ${objName} ${title}`.toLowerCase(); + if ( + [...otherTerms].some((otherTerm) => haystack.indexOf(otherTerm) < 0) + ) + return; + } + + let anchor = match[3]; + if (anchor === "") anchor = fullname; + else if (anchor === "-") anchor = objNames[match[1]][1] + "-" + fullname; + + const descr = objName + _(", in ") + title; + + // add custom score for some objects according to scorer + if (Scorer.objPrio.hasOwnProperty(match[2])) + score += Scorer.objPrio[match[2]]; + else score += Scorer.objPrioDefault; + + results.push([ + docNames[match[0]], + fullname, + "#" + anchor, + descr, + score, + filenames[match[0]], + ]); + }; + Object.keys(objects).forEach((prefix) => + objects[prefix].forEach((array) => + objectSearchCallback(prefix, array) + ) + ); + return results; + }, + + /** + * search for full-text terms in the index + */ + performTermsSearch: (searchTerms, excludedTerms) => { + // prepare search + const terms = Search._index.terms; + const titleTerms = Search._index.titleterms; + const filenames = Search._index.filenames; + const docNames = Search._index.docnames; + const titles = Search._index.titles; + + const scoreMap = new Map(); + const fileMap = new Map(); + + // perform the search on the required terms + searchTerms.forEach((word) => { + const files = []; + const arr = [ + { files: terms[word], score: Scorer.term }, + { files: titleTerms[word], score: Scorer.title }, + ]; + // add support for partial matches + if (word.length > 2) { + const escapedWord = _escapeRegExp(word); + Object.keys(terms).forEach((term) => { + if (term.match(escapedWord) && !terms[word]) + arr.push({ files: terms[term], score: Scorer.partialTerm }); + }); + Object.keys(titleTerms).forEach((term) => { + if (term.match(escapedWord) && !titleTerms[word]) + arr.push({ files: titleTerms[word], score: Scorer.partialTitle }); + }); + } + + // no match but word was a required one + if (arr.every((record) => record.files === undefined)) return; + + // found search word in contents + arr.forEach((record) => { + if (record.files === undefined) return; + + let recordFiles = record.files; + if (recordFiles.length === undefined) recordFiles = [recordFiles]; + files.push(...recordFiles); + + // set score for the word in each file + recordFiles.forEach((file) => { + if (!scoreMap.has(file)) scoreMap.set(file, {}); + scoreMap.get(file)[word] = record.score; + }); + }); + + // create the mapping + files.forEach((file) => { + if (fileMap.has(file) && fileMap.get(file).indexOf(word) === -1) + fileMap.get(file).push(word); + else fileMap.set(file, [word]); + }); + }); + + // now check if the files don't contain excluded terms + const results = []; + for (const [file, wordList] of fileMap) { + // check if all requirements are matched + + // as search terms with length < 3 are discarded + const filteredTermCount = [...searchTerms].filter( + (term) => term.length > 2 + ).length; + if ( + wordList.length !== searchTerms.size && + wordList.length !== filteredTermCount + ) + continue; + + // ensure that none of the excluded terms is in the search result + if ( + [...excludedTerms].some( + (term) => + terms[term] === file || + titleTerms[term] === file || + (terms[term] || []).includes(file) || + (titleTerms[term] || []).includes(file) + ) + ) + break; + + // select one (max) score for the file. + const score = Math.max(...wordList.map((w) => scoreMap.get(file)[w])); + // add result to the result list + results.push([ + docNames[file], + titles[file], + "", + null, + score, + filenames[file], + ]); + } + return results; + }, + + /** + * helper function to return a node containing the + * search summary for a given text. keywords is a list + * of stemmed words. + */ + makeSearchSummary: (htmlText, keywords) => { + const text = Search.htmlToText(htmlText); + if (text === "") return null; + + const textLower = text.toLowerCase(); + const actualStartPosition = [...keywords] + .map((k) => textLower.indexOf(k.toLowerCase())) + .filter((i) => i > -1) + .slice(-1)[0]; + const startWithContext = Math.max(actualStartPosition - 120, 0); + + const top = startWithContext === 0 ? "" : "..."; + const tail = startWithContext + 240 < text.length ? "..." : ""; + + let summary = document.createElement("p"); + summary.classList.add("context"); + summary.textContent = top + text.substr(startWithContext, 240).trim() + tail; + + return summary; + }, +}; + +_ready(Search.init); diff --git a/_static/sphinx_highlight.js b/_static/sphinx_highlight.js new file mode 100644 index 0000000..8a96c69 --- /dev/null +++ b/_static/sphinx_highlight.js @@ -0,0 +1,154 @@ +/* Highlighting utilities for Sphinx HTML documentation. */ +"use strict"; + +const SPHINX_HIGHLIGHT_ENABLED = true + +/** + * highlight a given string on a node by wrapping it in + * span elements with the given class name. + */ +const _highlight = (node, addItems, text, className) => { + if (node.nodeType === Node.TEXT_NODE) { + const val = node.nodeValue; + const parent = node.parentNode; + const pos = val.toLowerCase().indexOf(text); + if ( + pos >= 0 && + !parent.classList.contains(className) && + !parent.classList.contains("nohighlight") + ) { + let span; + + const closestNode = parent.closest("body, svg, foreignObject"); + const isInSVG = closestNode && closestNode.matches("svg"); + if (isInSVG) { + span = document.createElementNS("http://www.w3.org/2000/svg", "tspan"); + } else { + span = document.createElement("span"); + span.classList.add(className); + } + + span.appendChild(document.createTextNode(val.substr(pos, text.length))); + const rest = document.createTextNode(val.substr(pos + text.length)); + parent.insertBefore( + span, + parent.insertBefore( + rest, + node.nextSibling + ) + ); + node.nodeValue = val.substr(0, pos); + /* There may be more occurrences of search term in this node. So call this + * function recursively on the remaining fragment. + */ + _highlight(rest, addItems, text, className); + + if (isInSVG) { + const rect = document.createElementNS( + "http://www.w3.org/2000/svg", + "rect" + ); + const bbox = parent.getBBox(); + rect.x.baseVal.value = bbox.x; + rect.y.baseVal.value = bbox.y; + rect.width.baseVal.value = bbox.width; + rect.height.baseVal.value = bbox.height; + rect.setAttribute("class", className); + addItems.push({ parent: parent, target: rect }); + } + } + } else if (node.matches && !node.matches("button, select, textarea")) { + node.childNodes.forEach((el) => _highlight(el, addItems, text, className)); + } +}; +const _highlightText = (thisNode, text, className) => { + let addItems = []; + _highlight(thisNode, addItems, text, className); + addItems.forEach((obj) => + obj.parent.insertAdjacentElement("beforebegin", obj.target) + ); +}; + +/** + * Small JavaScript module for the documentation. + */ +const SphinxHighlight = { + + /** + * highlight the search words provided in localstorage in the text + */ + highlightSearchWords: () => { + if (!SPHINX_HIGHLIGHT_ENABLED) return; // bail if no highlight + + // get and clear terms from localstorage + const url = new URL(window.location); + const highlight = + localStorage.getItem("sphinx_highlight_terms") + || url.searchParams.get("highlight") + || ""; + localStorage.removeItem("sphinx_highlight_terms") + url.searchParams.delete("highlight"); + window.history.replaceState({}, "", url); + + // get individual terms from highlight string + const terms = highlight.toLowerCase().split(/\s+/).filter(x => x); + if (terms.length === 0) return; // nothing to do + + // There should never be more than one element matching "div.body" + const divBody = document.querySelectorAll("div.body"); + const body = divBody.length ? divBody[0] : document.querySelector("body"); + window.setTimeout(() => { + terms.forEach((term) => _highlightText(body, term, "highlighted")); + }, 10); + + const searchBox = document.getElementById("searchbox"); + if (searchBox === null) return; + searchBox.appendChild( + document + .createRange() + .createContextualFragment( + '

' + + '' + + _("Hide Search Matches") + + "

" + ) + ); + }, + + /** + * helper function to hide the search marks again + */ + hideSearchWords: () => { + document + .querySelectorAll("#searchbox .highlight-link") + .forEach((el) => el.remove()); + document + .querySelectorAll("span.highlighted") + .forEach((el) => el.classList.remove("highlighted")); + localStorage.removeItem("sphinx_highlight_terms") + }, + + initEscapeListener: () => { + // only install a listener if it is really needed + if (!DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS) return; + + document.addEventListener("keydown", (event) => { + // bail for input elements + if (BLACKLISTED_KEY_CONTROL_ELEMENTS.has(document.activeElement.tagName)) return; + // bail with special keys + if (event.shiftKey || event.altKey || event.ctrlKey || event.metaKey) return; + if (DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS && (event.key === "Escape")) { + SphinxHighlight.hideSearchWords(); + event.preventDefault(); + } + }); + }, +}; + +_ready(() => { + /* Do not call highlightSearchWords() when we are on the search page. + * It will highlight words from the *previous* search query. + */ + if (typeof Search === "undefined") SphinxHighlight.highlightSearchWords(); + SphinxHighlight.initEscapeListener(); +}); diff --git a/acknowledgements.html b/acknowledgements.html new file mode 100644 index 0000000..288caad --- /dev/null +++ b/acknowledgements.html @@ -0,0 +1,288 @@ + + + + + + + References and acknowledgements — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

References and acknowledgements

+
+

Citing tlm_adjoint

+

tlm_adjoint is described in

+
    +

  • James R. Maddison, Daniel N. Goldberg, and Benjamin D. Goddard, ‘Automated +calculation of higher order partial differential equation constrained +derivative information’, SIAM Journal on Scientific Computing, 41(5), pp. +C417–C445, 2019, doi: 10.1137/18M1209465

    • +
+

The automated assembly and linear solver caching applied by tlm_adjoint is +based on the approach described in

+
    +

  • J. R. Maddison and P. E. Farrell, ‘Rapid development and adjoining of +transient finite element models’, Computer Methods in Applied Mechanics and +Engineering, 276, 95–121, 2014, doi: 10.1016/j.cma.2014.03.010

    • +
+

Checkpointing with tlm_adjoint, and mixed forward restart / non-linear +dependency data schedules defined by the code in +tlm_adjoint/checkpoint_schedules/mixed.py, are described in

+ +
+
+

References

+
+

dolfin-adjoint

+

tlm_adjoint implements high-level algorithmic differentiation using an +approach based on that used by dolfin-adjoint, described in

+
    +

  • P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, ‘Automated +derivation of the adjoint of high-level transient finite element programs’, +SIAM Journal on Scientific Computing 35(4), pp. C369–C393, 2013, +doi: 10.1137/120873558

    • +
+

tlm_adjoint was developed from a custom extension to dolfin-adjoint.

+
+
+

Taylor remainder convergence testing

+

The functions in tlm_adjoint/verification.py implement Taylor remainder +convergence testing using the approach described in

+
    +

  • P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, ‘Automated +derivation of the adjoint of high-level transient finite element programs’, +SIAM Journal on Scientific Computing 35(4), pp. C369–C393, 2013, +doi: 10.1137/120873558

    • +
+
+
+

Solving eigenproblems with SLEPc

+

The eigendecompose function in tlm_adjoint/eigendecomposition.py was originally developed +by loosely following the slepc4py 3.6.0 demo demo/ex3.py. slepc4py 3.6.0 +license information follows.

+
=========================
+LICENSE: SLEPc for Python
+=========================
+
+:Author:  Lisandro Dalcin
+:Contact: dalcinl@gmail.com
+
+
+Copyright (c) 2015, Lisandro Dalcin.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+* Redistributions of source code must retain the above copyright
+  notice, this list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright
+  notice, this list of conditions and the following disclaimer in the
+  documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+
+
+

Differentiating fixed-point problems

+

The FixedPointSolver class in tlm_adjoint/fixed_point.py derives tangent-linear and +adjoint information using the approach described in

+
    +

  • Jean Charles Gilbert, ‘Automatic differentiation and iterative processes’, +Optimization Methods and Software, 1(1), pp. 13–21, 1992, +doi: 10.1080/10556789208805503

    • +

  • Bruce Christianson, ‘Reverse accumulation and attractive fixed points’, +Optimization Methods and Software, 3(4), pp. 311–326, 1994, +doi: 10.1080/10556789408805572

    • +
+
+
+

Binomial checkpointing

+

The MultistageCheckpointSchedule class in +tlm_adjoint/checkpoint_schedules/binomial.py implements the +binomial checkpointing strategy described in

+
    +

  • Andreas Griewank and Andrea Walther, ‘Algorithm 799: revolve: an +implementation of checkpointing for the reverse or adjoint mode of +computational differentiation’, ACM Transactions on Mathematical Software, +26(1), pp. 19–45, 2000, doi: 10.1145/347837.347846

    • +
+

The MultistageCheckpointSchedule class determines a memory/disk storage +distribution using an initial run of the checkpoint schedule, leading to a +distribution equivalent to that in

+
    +

  • Philipp Stumm and Andrea Walther, ‘MultiStage approaches for optimal offline +checkpointing’, SIAM Journal on Scientific Computing, 31(3), pp. 1946–1967, +2009, doi: 10.1137/080718036

    • +
+

The TwoLevelCheckpointSchedule class in +tlm_adjoint/checkpoint_schedules/binomial.py implements the +two-level mixed periodic/binomial checkpointing approach described in

+
    +

  • Gavin J. Pringle, Daniel C. Jones, Sudipta Goswami, Sri Hari Krishna +Narayanan, and Daniel Goldberg, ‘Providing the ARCHER community with adjoint +modelling tools for high-performance oceanographic and cryospheric +computation’, version 1.1, EPCC, 2016

    • +
+

and in the supporting information for

+
    +

  • D. N. Goldberg, T. A. Smith, S. H. K. Narayanan, P. Heimbach, and M. +Morlighem,, ‘Bathymetric influences on Antarctic ice-shelf melt rates’, +Journal of Geophysical Research: Oceans, 125(11), e2020JC016370, 2020, +doi: 10.1029/2020JC016370

    • +
+
+
+

L-BFGS

+

The file tlm_adjoint/optimization.py includes an implementation of +the L-BFGS algorithm, described in

+
    +

  • Jorge Nocedal and Stephen J. Wright, ‘Numerical optimization’, Springer, New +York, NY, 2006, Second edition, doi: 10.1007/978-0-387-40065-5

    • +

  • Richard H. Byrd, Peihuang Lu, and Jorge Nocedal, and Ciyou Zhu, ‘A limited +memory algorithm for bound constrained optimization’, SIAM Journal on +Scientific Computing, 16(5), 1190–1208, 1995, doi: 10.1137/0916069

    • +
+
+
+
+

Funding

+

Early development work leading to tlm_adjoint was conducted as part of a U.K. +Natural Environment Research Council funded project (NE/L005166/1). Further +development has been conducted as part of a U.K. Engineering and Physical +Sciences Research Council funded project (EP/R021600/1) and a Natural +Environment Research Council funded project (NE/T001607/1).

+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/adjoint/index.html b/autoapi/tlm_adjoint/adjoint/index.html new file mode 100644 index 0000000..4b59423 --- /dev/null +++ b/autoapi/tlm_adjoint/adjoint/index.html @@ -0,0 +1,347 @@ + + + + + + + tlm_adjoint.adjoint — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.adjoint

+
+

Module Contents

+
+
+class tlm_adjoint.adjoint.AdjointRHS(x)
+

The right-hand-side of an adjoint equation, for an adjoint variable +associated with an equation solving for a forward variable x.

+
+
Parameters:
+

x – The forward variable.

+
+
+
+
+b(*, copy=False)
+

Return the right-hand-side, as a variable.

+
+
Parameters:
+

copy – If True then a copy of the internal variable storing the +right-hand-side value is returned. If False the internal variable +itself is returned.

+
+
Returns:
+

A variable storing the right-hand-side value.

+
+
+
+ +
+
+initialize()
+

Allocate an internal variable to store the right-hand-side. +Typically need not be called manually.

+
+ +
+
+sub(b)
+

Subtract a term from the right-hand-side.

+
+
Parameters:
+

b – The term to subtract. +subtract_adjoint_derivative_action() is used to subtract the +term.

+
+
+
+ +
+
+is_empty()
+

Return whether the right-hand-side is ‘empty’, meaning that the +AdjointRHS.initialize() method has not been called.

+
+
Returns:
+

True if the AdjointRHS.initialize() method has not +been called, and False otherwise.

+
+
+
+ +
+ +
+
+class tlm_adjoint.adjoint.AdjointEquationRHS(eq)
+

The right-hand-side of an adjoint equation, for adjoint variables +associated with an equation solving for multiple forward variables X.

+

Multiple AdjointRHS objects. The AdjointRHS objects may +be accessed by index, e.g.

+
adj_eq_rhs = AdjointEquationRHS(eq)
+adj_rhs = adj_eq_rhs[m]
+
+
+
+
Parameters:
+

eq – An Equation. eq.X() defines the forward variables.

+
+
+
+
+b(*, copy=False)
+

For the case where there is a single forward variable, return a +variable associated with the right-hand-side.

+
+
Parameters:
+

copy – If True then a copy of the internal variable storing the +right-hand-side value is returned. If False the internal variable +itself is returned.

+
+
Returns:
+

A variable storing the right-hand-side value.

+
+
+
+ +
+
+B(*, copy=False)
+

Return variables associated with the right-hand-sides.

+
+
Parameters:
+

copy – If True then copies of the internal variables storing the +right-hand-side values are returned. If False the internal +variables themselves are returned.

+
+
Returns:
+

A tuple of variables storing the right-hand-side +values.

+
+
+
+ +
+
+is_empty()
+

Return whether all of the AdjointRHS objects are ‘empty’, +meaning that the AdjointRHS.initialize() method has not been +called for any AdjointRHS.

+
+
Returns:
+

True if the AdjointRHS.initialize() method has not +been called for any AdjointRHS, and False otherwise.

+
+
+
+ +
+ +
+
+class tlm_adjoint.adjoint.AdjointBlockRHS(block)
+

The right-hand-side of multiple adjoint equations.

+

Multiple AdjointEquationRHS objects. The +AdjointEquationRHS objects may be accessed by index, e.g.

+
adj_block_rhs = AdjointBlockRHS(block)
+adj_eq_rhs = adj_block_rhs[k]
+
+
+

AdjointRHS objects may be accessed e.g.

+
adj_rhs = adj_block_rhs[(k, m)]
+
+
+
+
Parameters:
+

block – A Sequence of Equation objects.

+
+
+
+
+pop()
+

Remove and return the last AdjointEquationRHS in the +AdjointBlockRHS.

+
+
Returns:
+

A tuple (n, B). B is the removed +AdjointEquationRHS, associated with block n.

+
+
+
+ +
+
+is_empty()
+

Return whether there are no AdjointEquationRHS objects in +the AdjointBlockRHS.

+
+
Returns:
+

True if there are no AdjointEquationRHS objects +in the AdjointBlockRHS, and False otherwise.

+
+
+
+ +
+ +
+
+class tlm_adjoint.adjoint.AdjointModelRHS(blocks)
+

The right-hand-side of multiple blocks of adjoint equations.

+

Multiple AdjointBlockRHS objects. The AdjointBlockRHS +objects may be accessed by index, e.g.

+
adj_model_rhs = AdjointModelRHS(block)
+adj_block_rhs = adj_block_rhs[p]
+
+
+

AdjointEquationRHS objects may be accessed e.g.

+
adj_eq_rhs = adj_block_rhs[(p, k)]
+
+
+

AdjointRHS objects may be accessed e.g.

+
adj_rhs = adj_block_rhs[(p, k, m)]
+
+
+

If the last block of adjoint equations contains no equations then it is +automatically removed from the AdjointModelRHS.

+
+
Parameters:
+

blocks – A Sequence of Sequence objects each +containing Equation objects, or a Mapping with items +(index, block) where index is an int and block a +Sequence of Equation objects. In the latter case +blocks are ordered by index.

+
+
+
+
+pop()
+

Remove and return the last AdjointEquationRHS in the last +AdjointBlockRHS in the AdjointModelRHS.

+
+
Returns:
+

A tuple ((n, i), B). B is the removed +AdjointEquationRHS, associated with equation i in block +n.

+
+
+
+ +
+
+is_empty()
+

Return whether there are no AdjointBlockRHS objects in the +AdjointModelRHS.

+
+
Returns:
+

True if there are no AdjointBlockRHS objects in +the AdjointModelRHS, and False otherwise.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/alias/index.html b/autoapi/tlm_adjoint/alias/index.html new file mode 100644 index 0000000..77a87b1 --- /dev/null +++ b/autoapi/tlm_adjoint/alias/index.html @@ -0,0 +1,156 @@ + + + + + + + tlm_adjoint.alias — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.alias

+
+

Module Contents

+
+
+tlm_adjoint.alias.gc_disabled(fn)
+

Decorator to disable the Python garbage collector.

+
+
Parameters:
+

fn – A callable for which the Python garbage collector should be +disabled.

+
+
Returns:
+

A callable for which the Python garbage collector is disabled.

+
+
+
+ +
+
+class tlm_adjoint.alias.Alias(obj)
+

An alias to an object. Holds a reference to the original object.

+
+
Parameters:
+

obj – Object to alias.

+
+
+
+ +
+
+class tlm_adjoint.alias.WeakAlias(obj)
+

An alias to an object. Does not hold a reference to the original +object.

+

Intended to be used in combination with weakref.finalize, so that object +attributes may be updated when the original object is destroyed, but object +methods may still be called after it is destroyed.

+
+
Parameters:
+

obj – Object to alias.

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/cached_hessian/index.html b/autoapi/tlm_adjoint/cached_hessian/index.html new file mode 100644 index 0000000..9cbac21 --- /dev/null +++ b/autoapi/tlm_adjoint/cached_hessian/index.html @@ -0,0 +1,233 @@ + + + + + + + tlm_adjoint.cached_hessian — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.cached_hessian

+
+

Module Contents

+
+
+class tlm_adjoint.cached_hessian.CachedHessian(J, *, manager=None, cache_adjoint=True)
+

Represents a Hessian associated with a given forward. Uses a cached +forward calculation.

+
+
Parameters:
+
    +

  • J – A variable defining the Hessian.

    • +

  • manager – The EquationManager used to record the forward. +This must have used ‘memory’ checkpointing with automatic dropping of +variable references disabled. manager() is used if not supplied.

    • +

  • cache_adjoint – Whether to cache the first order adjoint calculation.

    • +
+
+
+
+
+compute_gradient(M, M0=None)
+

Compute the (conjugate of the) derivative of a functional with +respect to a control using an adjoint.

+
+
Parameters:
+
    +

  • M – A variable or a Sequence of variables defining the +control.

    • +

  • M0 – A variable or a Sequence of variables defining the +control value. M is used if not supplied.

    • +
+
+
Returns:
+

The (conjugate of the) derivative. A variable or +Sequence of variables, depending on the type of M.

+
+
+
+ +
+
+action(M, dM, M0=None)
+

Compute (the conjugate of) a Hessian action on some \(\zeta\) +using an adjoint of a tangent-linear. i.e. considering derivatives to +be row vectors, compute

+
+\[\left( \frac{d}{dm} \left[ + \frac{d \mathcal{J}}{d m} \zeta \right] \right)^{*,T}.\]
+
+
Parameters:
+
    +

  • M – A variable or a Sequence of variables defining the +control.

    • +

  • dM – A variable or a Sequence of variables defining +\(\zeta\). The (conjugate of the) Hessian action on +\(\zeta\) is computed.

    • +

  • M0 – A variable or a Sequence of variables defining the +control value. M is used if not supplied.

    • +
+
+
Returns:
+

A tuple (J, dJ, ddJ). J is the value of the functional. +dJ is the value of \(\left( d \mathcal{J} / d m \right) +\zeta\). ddJ stores the (conjugate of the) result of the Hessian +action on \(\zeta\), and is a variable or a Sequence +of variables depending on the type of M.

+
+
+
+ +
+ +
+
+class tlm_adjoint.cached_hessian.CachedGaussNewton(X, R_inv_action, B_inv_action=None, *, manager=None)
+

Represents a Gauss-Newton approximation to a Hessian associated with a +given forward. Uses a cached forward calculation.

+
+
Parameters:
+
    +

  • X – A variable or a Sequence of variables defining the state.

    • +

  • R_inv_action – See GaussNewton.

    • +

  • B_inv_action – See GaussNewton.

    • +

  • manager – The EquationManager used to record the forward. +This must have used ‘memory’ checkpointing with automatic dropping of +variable references disabled. manager() is used if not supplied.

    • +
+
+
+
+
+action(M, dM, M0=None)
+

Compute (the conjugate of) a Hessian action on some \(\zeta\), +using the Gauss-Newton approximation for the Hessian. i.e. compute

+
+\[\left( H \zeta \right)^{*,T}.\]
+
+
Parameters:
+
    +

  • M – A variable or a Sequence of variables defining the +control.

    • +

  • dM – A variable or a Sequence of variables defining +\(\zeta\). The (conjugate of the) approximated Hessian action +on \(\zeta\) is computed.

    • +

  • M0 – A variable or a Sequence of variables defining the +control value. M is used if not supplied.

    • +
+
+
Returns:
+

The (conjugate of the) result of the approximated Hessian +action on \(\zeta\). A variable or a Sequence of +variables depending on the type of M.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/caches/index.html b/autoapi/tlm_adjoint/caches/index.html new file mode 100644 index 0000000..28b9fe7 --- /dev/null +++ b/autoapi/tlm_adjoint/caches/index.html @@ -0,0 +1,296 @@ + + + + + + + tlm_adjoint.caches — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.caches

+
+

Module Contents

+
+
+class tlm_adjoint.caches.CacheRef(value=None)
+

A cache entry. Stores a reference to a cached value, which can later be +cleared. Calling a CacheRef returns the cached object, or None +if no object is referenced.

+
+
Parameters:
+

value – The object to reference. None may be supplied to indicate an +empty cache entry.

+
+
+
+
+clear()
+

Clear the cache entry. After calling this method, calling the +CacheRef will return None.

+
+ +
+ +
+
+tlm_adjoint.caches.clear_caches(*deps)
+

Clear caches entries.

+
+
Parameters:
+

deps – A Sequence of variables. If non-empty then clear only +cache entries which depend on the supplied variables. Otherwise clear +all cache entries.

+
+
+
+ +
+
+tlm_adjoint.caches.local_caches(fn)
+

Decorator clearing caches before and after calling the decorated +callable.

+
+
Parameters:
+

fn – A callable for which caches should be cleared.

+
+
Returns:
+

A callable where caches are cleared before and after calling.

+
+
+
+ +
+
+class tlm_adjoint.caches.Cache
+

Stores cache entries.

+

Cleared cache entries are removed from the Cache.

+
+
+property id
+

A unique int ID associated with this Cache.

+
+ +
+
+clear(*deps)
+

Clear cache entries.

+
+
Parameters:
+

deps – A Sequence of variables. If non-empty then only +clear cache entries which depend on the supplied variables. +Otherwise clear all cache entries.

+
+
+
+ +
+
+add(key, value, deps=None)
+

Add a cache entry.

+
+
Parameters:
+
    +

  • key – The key associated with the cache entry.

    • +

  • value – A callable, taking no arguments, returning the value +associated with the cache entry. Only called to if no entry +associated with key exists.

    • +

  • deps – A Sequence of variables, defining dependencies of +the cache entry.

    • +
+
+
Returns:
+

A tuple (value_ref, value), where value is the +cache entry value and value_ref is a CacheRef storing a +reference to the value.

+
+
+
+ +
+
+get(key, *args)
+

Return the cache entry associated with a given key.

+
+
Parameters:
+

key – The key.

+
+
Returns:
+

The cache entry or, if supplied, a default value.

+
+
+

args should contain zero or one elements and defines the default +value. If there is no entry associated with the key then:

+
+
    +

  • If args has no elements an exception is raised.

    • +

  • If args has one element then this is returned.

    • +
+
+
+ +
+ +
+
+class tlm_adjoint.caches.Caches(x)
+

Multiple Cache objects, associated with a variable.

+

Cache entries may depend on the variable. The variable also defines an +initial value, and the value is indicated by the variable ID and variable +state value. The value may be changed either by supplying a new variable +(changing the ID), or by changing the value of the current variable +defining the value (which should be indicated by a change to the variable +state value). Either change invalidates cache entries, in the +Cache objects, which depend on the original variable.

+

The Caches.update() method can be used to check for cache entry +invalidation, and to clear invalid cache entries.

+
+
Parameters:
+

x – The variable defining a possible cache entry dependency, and an +initial value for that dependency.

+
+
+
+
+clear()
+

Clear cache entries which depend on the associated variable.

+
+ +
+
+add(cache)
+

Add a new Cache to the Caches.

+
+
Parameters:
+

cache – The Cache to add to the Caches.

+
+
+
+ +
+
+remove(cache)
+

Remove a Cache from the Caches.

+
+
Parameters:
+

cache – The Cache to remove from the Caches.

+
+
+
+ +
+
+update(x)
+

Check for cache invalidation associated with a possible change in +value, and clear invalid cache entries.

+
+
Parameters:
+

x – A variable which defines a potentially new value.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/checkpoint_schedules/binomial/index.html b/autoapi/tlm_adjoint/checkpoint_schedules/binomial/index.html new file mode 100644 index 0000000..5b034ba --- /dev/null +++ b/autoapi/tlm_adjoint/checkpoint_schedules/binomial/index.html @@ -0,0 +1,254 @@ + + + + + + + tlm_adjoint.checkpoint_schedules.binomial — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.checkpoint_schedules.binomial

+
+

Module Contents

+
+
+class tlm_adjoint.checkpoint_schedules.binomial.MultistageCheckpointSchedule(max_n, snapshots_in_ram, snapshots_on_disk, *, trajectory='maximum')
+

A binomial checkpointing schedule using the approach described in

+
+
    +

  • Andreas Griewank and Andrea Walther, ‘Algorithm 799: revolve: an +implementation of checkpointing for the reverse or adjoint mode of +computational differentiation’, ACM Transactions on Mathematical +Software, 26(1), pp. 19–45, 2000, doi: 10.1145/347837.347846

    • +
+
+

hereafter referred to as GW2000.

+

Uses a ‘MultiStage’ distribution of checkpoints between RAM and disk +equivalent to that described in

+
+
    +

  • Philipp Stumm and Andrea Walther, ‘MultiStage approaches for optimal +offline checkpointing’, SIAM Journal on Scientific Computing, 31(3), +pp. 1946–1967, 2009, doi: 10.1137/080718036

    • +
+
+

The distribution between RAM and disk is determined using an initial run of +the schedule.

+

Offline, one adjoint calculation permitted.

+
+
Parameters:
+
    +

  • max_n – The number of forward steps in the initial forward calculation.

    • +

  • snapshots_in_ram – The maximum number of forward restart checkpoints +to store in memory.

    • +

  • snapshots_on_disk – The maximum number of forward restart checkpoints +to store on disk.

    • +

  • trajectory –

    When advancing n forward steps with s checkpointing +units available there are in general multiple solutions to the problem +of determining the number of forward steps to advance before storing +a new forward restart checkpoint – see Fig. 4 of GW2000. This argument +selects a solution:

    +
    +
      +

    • ’revolve’: The standard revolve solution, as specified in the +equation at the bottom of p. 34 of GW2000.

      • +

    • ’maximum’: The maximum possible number of steps, corresponding +to the maximum step size compatible with the optimal region in +Fig. 4 of GW2000.

      • +
    +
    +

    • +
+
+
+

The argument names snaps_in_ram and snaps_on_disk originate from the +corresponding arguments for the dolfin_adjoint.solving.adj_checkpointing +function in dolfin-adjoint (see e.g. version 2017.1.0).

+
+
+property is_exhausted
+

Whether the schedule has concluded. Note that some schedules permit +multiple adjoint calculation, and may never conclude.

+
+ +
+
+property uses_disk_storage
+

Whether the schedule may use disk storage. If False then no disk +storage is required.

+
+ +
+
+iter()
+

A generator which should be overridden in derived classes in order +to define a checkpointing schedule.

+
+ +
+ +
+
+class tlm_adjoint.checkpoint_schedules.binomial.TwoLevelCheckpointSchedule(period, binomial_snapshots, *, binomial_storage='disk', binomial_trajectory='maximum')
+

A two-level mixed periodic/binomial checkpointing schedule using the +approach described in

+
+
    +

  • Gavin J. Pringle, Daniel C. Jones, Sudipta Goswami, Sri Hari Krishna +Narayanan, and Daniel Goldberg, ‘Providing the ARCHER community with +adjoint modelling tools for high-performance oceanographic and +cryospheric computation’, version 1.1, EPCC, 2016

    • +
+
+

and in the supporting information for

+
+
    +

  • D. N. Goldberg, T. A. Smith, S. H. K. Narayanan, P. Heimbach, and M. +Morlighem, ‘Bathymetric influences on Antarctic ice-shelf melt +rates’, Journal of Geophysical Research: Oceans, 125(11), +e2020JC016370, 2020, doi: 10.1029/2020JC016370

    • +
+
+

Online, unlimited adjoint calculations permitted.

+
+
Parameters:
+
    +

  • period – Forward restart checkpoints are stored to disk every period +forward steps in the initial forward calculation.

    • +

  • binomial_snapshots – The maximum number of additional forward restart +checkpointing units to use when advancing the adjoint between periodic +disk checkpoints.

    • +

  • binomial_storage – The storage to use for the additional forward +restart checkpoints generated when advancing the adjoint between +periodic disk checkpoints. Either ‘RAM’ or ‘disk’.

    • +

  • binomial_trajectory – See the trajectory constructor argument for +MultistageCheckpointSchedule.

    • +
+
+
+
+
+property is_exhausted
+

Whether the schedule has concluded. Note that some schedules permit +multiple adjoint calculation, and may never conclude.

+
+ +
+
+property uses_disk_storage
+

Whether the schedule may use disk storage. If False then no disk +storage is required.

+
+ +
+
+iter()
+

A generator which should be overridden in derived classes in order +to define a checkpointing schedule.

+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/checkpoint_schedules/h_revolve/index.html b/autoapi/tlm_adjoint/checkpoint_schedules/h_revolve/index.html new file mode 100644 index 0000000..0368697 --- /dev/null +++ b/autoapi/tlm_adjoint/checkpoint_schedules/h_revolve/index.html @@ -0,0 +1,169 @@ + + + + + + + tlm_adjoint.checkpoint_schedules.h_revolve — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.checkpoint_schedules.h_revolve

+
+

Module Contents

+
+
+class tlm_adjoint.checkpoint_schedules.h_revolve.HRevolveCheckpointSchedule(max_n, snapshots_in_ram, snapshots_on_disk, *, wvect=(0.0, 0.1), rvect=(0.0, 0.1), uf=1.0, ub=2.0, **kwargs)
+

An H-Revolve checkpointing schedule.

+

Converts from schedules as generated by the H-Revolve library, for the +case of two storage levels: RAM and disk.

+

Offline, one adjoint calculation permitted.

+
+
Parameters:
+
    +

  • max_n – The number of forward steps in the initial forward calculation.

    • +

  • snapshots_in_ram – The maximum number of forward restart checkpoints +to store in memory.

    • +

  • snapshots_on_disk – The maximum number of forward restart checkpoints +to store on disk.

    • +

  • wvect – A two element tuple defining the write cost associated +with saving a forward restart checkpoint to RAM (first element) and +disk (second element).

    • +

  • rvect – A two element tuple defining the read cost associated +with loading a forward restart checkpoint from RAM (first element) and +disk (second element).

    • +

  • uf – The cost of advancing the forward one step.

    • +

  • bf – The cost of advancing the forward one step, storing non-linear +dependency data, and then advancing the adjoint over that step.

    • +
+
+
+

Remaining keyword arguments are passed to hrevolve.hrevolve.

+

The argument names snaps_in_ram and snaps_on_disk originate from the +corresponding arguments for the dolfin_adjoint.solving.adj_checkpointing +function in dolfin-adjoint (see e.g. version 2017.1.0).

+
+
+property is_exhausted
+

Whether the schedule has concluded. Note that some schedules permit +multiple adjoint calculation, and may never conclude.

+
+ +
+
+property uses_disk_storage
+

Whether the schedule may use disk storage. If False then no disk +storage is required.

+
+ +
+
+iter()
+

A generator which should be overridden in derived classes in order +to define a checkpointing schedule.

+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/checkpoint_schedules/memory/index.html b/autoapi/tlm_adjoint/checkpoint_schedules/memory/index.html new file mode 100644 index 0000000..9d1677d --- /dev/null +++ b/autoapi/tlm_adjoint/checkpoint_schedules/memory/index.html @@ -0,0 +1,144 @@ + + + + + + + tlm_adjoint.checkpoint_schedules.memory — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.checkpoint_schedules.memory

+
+

Module Contents

+
+
+class tlm_adjoint.checkpoint_schedules.memory.MemoryCheckpointSchedule(max_n=None)
+

A checkpointing schedule where all forward restart and non-linear +dependency data are stored in memory.

+

Online, unlimited adjoint calculations permitted.

+
+
+property is_exhausted
+

Whether the schedule has concluded. Note that some schedules permit +multiple adjoint calculation, and may never conclude.

+
+ +
+
+property uses_disk_storage
+

Whether the schedule may use disk storage. If False then no disk +storage is required.

+
+ +
+
+iter()
+

A generator which should be overridden in derived classes in order +to define a checkpointing schedule.

+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/checkpoint_schedules/mixed/index.html b/autoapi/tlm_adjoint/checkpoint_schedules/mixed/index.html new file mode 100644 index 0000000..6ecd32d --- /dev/null +++ b/autoapi/tlm_adjoint/checkpoint_schedules/mixed/index.html @@ -0,0 +1,164 @@ + + + + + + + tlm_adjoint.checkpoint_schedules.mixed — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.checkpoint_schedules.mixed

+
+

Module Contents

+
+
+class tlm_adjoint.checkpoint_schedules.mixed.MixedCheckpointSchedule(max_n, snapshots, *, storage='disk')
+

A checkpointing schedule which mixes storage of forward restart data and +non-linear dependency data in checkpointing units. Assumes that the data +required to restart the forward has the same size as the data required to +advance the adjoint over a step.

+

Described in

+
+
+
+

Offline, one adjoint calculation permitted.

+
+
Parameters:
+
    +

  • max_n – The number of forward steps in the initial forward calculation.

    • +

  • snapshots – The number of available checkpointing units.

    • +

  • storage – Checkpointing unit storage location. Either ‘RAM’ or +‘disk’.

    • +
+
+
+
+
+property is_exhausted
+

Whether the schedule has concluded. Note that some schedules permit +multiple adjoint calculation, and may never conclude.

+
+ +
+
+property uses_disk_storage
+

Whether the schedule may use disk storage. If False then no disk +storage is required.

+
+ +
+
+iter()
+

A generator which should be overridden in derived classes in order +to define a checkpointing schedule.

+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/checkpoint_schedules/none/index.html b/autoapi/tlm_adjoint/checkpoint_schedules/none/index.html new file mode 100644 index 0000000..bdad97c --- /dev/null +++ b/autoapi/tlm_adjoint/checkpoint_schedules/none/index.html @@ -0,0 +1,144 @@ + + + + + + + tlm_adjoint.checkpoint_schedules.none — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.checkpoint_schedules.none

+
+

Module Contents

+
+
+class tlm_adjoint.checkpoint_schedules.none.NoneCheckpointSchedule
+

A checkpointing schedule for the case where no adjoint calculation is +performed.

+

Online, zero adjoint calculations permitted.

+
+
+property is_exhausted
+

Whether the schedule has concluded. Note that some schedules permit +multiple adjoint calculation, and may never conclude.

+
+ +
+
+property uses_disk_storage
+

Whether the schedule may use disk storage. If False then no disk +storage is required.

+
+ +
+
+iter()
+

A generator which should be overridden in derived classes in order +to define a checkpointing schedule.

+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/checkpoint_schedules/periodic/index.html b/autoapi/tlm_adjoint/checkpoint_schedules/periodic/index.html new file mode 100644 index 0000000..da49b6d --- /dev/null +++ b/autoapi/tlm_adjoint/checkpoint_schedules/periodic/index.html @@ -0,0 +1,153 @@ + + + + + + + tlm_adjoint.checkpoint_schedules.periodic — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.checkpoint_schedules.periodic

+
+

Module Contents

+
+
+class tlm_adjoint.checkpoint_schedules.periodic.PeriodicDiskCheckpointSchedule(period)
+

A checkpointing schedule where forward restart checkpoints are stored +periodically to disk. Non-linear dependency data is recomputed for use by +the adjoint by re-running the forward from these checkpoints. If the +storage period is greater than one then non-linear dependency data for +multiple steps is recomputed and stored when advancing the adjoint.

+

Online, unlimited adjoint calculations permitted.

+
+
Parameters:
+

period – Forward restart checkpoints are stored to disk every period +forward steps in the initial forward calculation.

+
+
+
+
+property is_exhausted
+

Whether the schedule has concluded. Note that some schedules permit +multiple adjoint calculation, and may never conclude.

+
+ +
+
+property uses_disk_storage
+

Whether the schedule may use disk storage. If False then no disk +storage is required.

+
+ +
+
+iter()
+

A generator which should be overridden in derived classes in order +to define a checkpointing schedule.

+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/checkpoint_schedules/schedule/index.html b/autoapi/tlm_adjoint/checkpoint_schedules/schedule/index.html new file mode 100644 index 0000000..83511b7 --- /dev/null +++ b/autoapi/tlm_adjoint/checkpoint_schedules/schedule/index.html @@ -0,0 +1,434 @@ + + + + + + + tlm_adjoint.checkpoint_schedules.schedule — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.checkpoint_schedules.schedule

+
+

Module Contents

+
+
+class tlm_adjoint.checkpoint_schedules.schedule.CheckpointAction(*args)
+

A checkpointing action.

+
+
+property args
+

Action parameters.

+
+ +
+ +
+
+class tlm_adjoint.checkpoint_schedules.schedule.Clear(clear_ics, clear_data)
+

A checkpointing action which clears the intermediate storage.

+
+
Parameters:
+
    +

  • clear_ics – Whether to clear stored forward restart data.

    • +

  • clear_data – Whether to clear stored non-linear dependency data.

    • +
+
+
+
+
+property clear_ics
+

Whether to clear stored forward restart data.

+
+ +
+
+property clear_data
+

Whether to clear stored non-linear dependency data.

+
+ +
+ +
+
+class tlm_adjoint.checkpoint_schedules.schedule.Configure(store_ics, store_data)
+

A checkpointing action which configures the intermediate storage.

+
+
Parameters:
+
    +

  • store_ics – Whether to store forward restart data.

    • +

  • store_data – Whether to store non-linear dependency data.

    • +
+
+
+
+
+property store_ics
+

Whether to store forward restart data.

+
+ +
+
+property store_data
+

Whether to store non-linear dependency data.

+
+ +
+ +
+
+class tlm_adjoint.checkpoint_schedules.schedule.Forward(n0, n1)
+

A checkpointing action which indicates forward advancement.

+
+
Parameters:
+
    +

  • n0 – The forward should advance from the start of this step.

    • +

  • n1 – The forward should advance to the start of this step.

    • +
+
+
+
+
+property n0
+

The forward should advance from the start of this step.

+
+ +
+
+property n1
+

The forward should advance to the start of this step.

+
+ +
+ +
+
+class tlm_adjoint.checkpoint_schedules.schedule.Reverse(n1, n0)
+

A checkpointing action which indicates adjoint advancement.

+
+
Parameters:
+
    +

  • n1 – The adjoint should advance from the start of this step.

    • +

  • n0 – The adjoint should advance to the start of this step.

    • +
+
+
+
+
+property n0
+

The adjoint should advance to the start of this step.

+
+ +
+
+property n1
+

The adjoint should advance from the start of this step.

+
+ +
+ +
+
+class tlm_adjoint.checkpoint_schedules.schedule.Read(n, storage, delete)
+

A checkpointing action which indicates loading of data from a +checkpointing unit.

+
+
Parameters:
+
    +

  • n – The step with which the loaded data is associated.

    • +

  • storage – The storage from which the data should be loaded. Either +‘RAM’ or ‘disk’.

    • +

  • delete – Whether the data should be deleted from the indicated storage +after it has been loaded.

    • +
+
+
+
+
+property n
+

The step with which the loaded data is associated.

+
+ +
+
+property storage
+

The storage from which the data should be loaded. Either ‘RAM’ or +‘disk’.

+
+ +
+
+property delete
+

Whether the data should be deleted from the indicated storage after +it has been loaded.

+
+ +
+ +
+
+class tlm_adjoint.checkpoint_schedules.schedule.Write(n, storage)
+

A checkpointing action which indicates saving of data to a checkpointing +unit.

+
+
Parameters:
+
    +

  • n – The step with which the saved data is associated.

    • +

  • storage – The storage to which the data should be saved. Either ‘RAM’ +or ‘disk’.

    • +
+
+
+
+
+property n
+

The step with which the saved data is associated.

+
+ +
+
+property storage
+

The storage to which the data should be saved. Either ‘RAM’ or +‘disk’.

+
+ +
+ +
+
+class tlm_adjoint.checkpoint_schedules.schedule.EndForward
+

A checkpointing action which indicates the end of the initial forward +calculation.

+
+ +
+
+class tlm_adjoint.checkpoint_schedules.schedule.EndReverse(exhausted)
+

A checkpointing action which indicates the end of an adjoint +calculation.

+
+
Parameters:
+

exhausted – Indicates whether the schedule has concluded. If True +then this action should be the last action in the schedule.

+
+
+
+
+property exhausted
+

Indicates whether the schedule has concluded. If True then this +action should be the last action in the schedule.

+
+ +
+ +
+
+class tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule(max_n=None)
+

A checkpointing schedule.

+

Actions in the schedule are accessed by iterating over elements, and +actions may be implemented using single-dispatch functions. e.g.

+
@functools.singledispatch
+def action(cp_action):
+    raise TypeError(f"Unexpected checkpointing action: {cp_action}")
+
+@action.register(Forward)
+def action_forward(cp_action):
+    logger.debug(f"forward: forward advance to {cp_action.n1:d}")
+
+# ...
+
+for cp_action in cp_schedule:
+    action(cp_action)
+    if isinstance(cp_action, EndReverse):
+        break
+
+
+

Schedules control an intermediate storage, which buffers forward restart +data for forward restart checkpoints, and which stores non-linear +dependency data either for storage in checkpointing units or for immediate +use by the adjoint. For details see

+
+
+
+

In ‘offline’ schedules, where the number of steps in the forward +calculation is initially known, this should be provided using the max_n +argument on instantiation. In ‘online’ schedules, where the number of steps +in the forward calculation is initially unknown, the number of forward +steps should later be provided using the +CheckpointSchedule.finalize() method.

+
+
Parameters:
+

max_n – The number of steps in the initial forward calculation. If not +supplied then this should later be provided by calling the +CheckpointSchedule.finalize() method.

+
+
+
+
+abstract property is_exhausted
+

Whether the schedule has concluded. Note that some schedules permit +multiple adjoint calculation, and may never conclude.

+
+ +
+
+abstract property uses_disk_storage
+

Whether the schedule may use disk storage. If False then no disk +storage is required.

+
+ +
+
+property n
+

The forward location. After executing all actions defined so far in +the schedule the forward is at the start of this step.

+
+ +
+
+property r
+

The number of adjoint steps advanced in the current adjoint +calculation after executing all actions defined so far in the schedule.

+
+ +
+
+property max_n
+

The number of forward steps in the initial forward calculation. May +return None if this has not yet been provided to the scheduler.

+
+ +
+
+property is_running
+

Whether the schedule is ‘running’ – i.e. at least one action has +been defined so far in the schedule.

+
+ +
+
+abstract iter()
+

A generator which should be overridden in derived classes in order +to define a checkpointing schedule.

+
+ +
+
+finalize(n)
+

Indicate the number of forward steps in the initial forward +calculation.

+
+
Parameters:
+

n – The number of steps in the initial forward calculation.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/checkpointing/index.html b/autoapi/tlm_adjoint/checkpointing/index.html new file mode 100644 index 0000000..0b4a871 --- /dev/null +++ b/autoapi/tlm_adjoint/checkpointing/index.html @@ -0,0 +1,642 @@ + + + + + + + tlm_adjoint.checkpointing — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.checkpointing

+
+

Module Contents

+
+
+class tlm_adjoint.checkpointing.CheckpointStorage(*, store_ics, store_data)
+

A buffer for forward restart data, and a cache for non-linear dependency +data. Contains three types of data:

+
+
    +

  1. References: Dependencies which are stored by reference. Variables +x for which var_is_static(x) is True are stored by reference.

    2. +

  2. Forward restart / initial condition data: Dependencies which are +used to restart and advance the forward calculation.

    3. +

  3. Non-linear dependency data: Non-linear dependencies of the forward +which are used to advance the adjoint.

    4. +
+
+

These may overlap – for example non-linear dependency data may be stored +by reference.

+

Non-linear dependency data has an associated key (n, i), where n is an +int indicating the block index and i is an int +indicating the equation index within that block. Non-linear-dependency data +for an Equation can be accessed via, e.g.

+
nl_deps = cp[(n, i)]
+
+
+

where cp is a CheckpointStorage. Here nl_deps is a +tuple of variables storing values associated with +eq.nonlinear_dependencies(), for Equation i in block n.

+
+
Parameters:
+
    +

  • store_ics – Whether to enable storage of forward restart data.

    • +

  • store_data – Whether to enable storage of non-linear dependency data.

    • +
+
+
+
+
+property store_ics
+

Whether storage of forward restart data is enabled.

+
+ +
+
+property store_data
+

Whether storage of non-linear dependency data is enabled.

+
+ +
+
+configure(*, store_ics, store_data)
+

Enable or disable storage of forward restart and non-linear +dependency data.

+
+
Parameters:
+
    +

  • store_ics – Whether storage of forward restart data should be +enabled (store_ics=True) or disabled (store_ics=False).

    • +

  • store_data – Whether storage of non-linear dependency data should +be enabled (store_data=True) or disabled (store_data=False).

    • +
+
+
+
+ +
+
+clear(*, clear_ics=True, clear_data=True, clear_refs=False)
+

Clear stored data.

+
+
Parameters:
+
    +

  • clear_ics – Whether forward restart data should be cleared.

    • +

  • clear_data – Whether non-linear dependency data should be cleared.

    • +

  • clear_refs – Whether references should be cleared.

    • +
+
+
+
+ +
+
+initial_condition(x, *, copy=True)
+

Return the forward restart value associated with a variable x.

+
+
Parameters:
+
    +

  • x – The variable, or the int variable ID, for which the +forward restart value should be returned.

    • +

  • copy – If True then a copy of the stored value is returned. If +False then an internal variable storing the value is returned.

    • +
+
+
Returns:
+

A variable containing the forward restart value for x.

+
+
+
+ +
+
+initial_conditions(*, cp=True, refs=False, copy=True)
+

Access stored forward restart data.

+
+
Parameters:
+
    +

  • cp – Whether to include forward restart data that is stored by +value.

    • +

  • refs – Whether to include data that is stored by reference. May +include non-linear dependency data that is not forward restart +data.

    • +

  • copy – If True then a copy of the stored data is returned. If +False then internal variables storing the data are returned.

    • +
+
+
Returns:
+

A VariableStateLockDictionary, with items (x_id: +x_value), where x_id is the int variable ID and +x_value is a variable storing the data.

+
+
+
+ +
+
+add_initial_condition(x, value=None)
+

Store forward restart data / an initial condition dependency.

+
+
Parameters:
+
    +

  • x – The initial condition dependency variable.

    • +

  • value – A variable defining the initial condition dependency value. +x is used if not supplied.

    • +
+
+
+
+ +
+
+update_keys(n, i, eq)
+

The CheckpointStorage keeps an internal map from forward +variables to equations in which values for those variables are +computed. Keys are updated automatically as needed. This method allows +keys to be updated manually.

+
+
Parameters:
+
    +

  • n – The int index of the block.

    • +

  • i – The int index of the equation.

    • +

  • eq – An Equation, equation i in block n.

    • +
+
+
+
+ +
+
+add_equation(n, i, eq, *, deps=None, nl_deps=None)
+

Store checkpoint data associated with an equation.

+
+
Parameters:
+
    +

  • n – The int index of the block.

    • +

  • i – The int index of the equation.

    • +

  • eq – An Equation, equation i in block n.

    • +

  • deps – Equation dependency values. eq.dependencies() is used if +not supplied.

    • +

  • nl_deps – Equation non-linear dependency values. Extracted from +deps if not supplied.

    • +
+
+
+
+ +
+
+add_equation_data(n, i, eq, *, nl_deps=None)
+

Store checkpoint data associated with an equation. As +CheckpointStorage.add_equation(), but adds only non-linear +dependency data.

+
+
Parameters:
+
    +

  • n – The int index of the block.

    • +

  • i – The int index of the equation.

    • +

  • eq – An Equation, equation i in block n.

    • +

  • nl_deps – Equation non-linear dependency values. +eq.nonlinear_dependencies() is used if not supplied.

    • +
+
+
+
+ +
+
+checkpoint_data(*, ics=True, data=True, copy=True)
+

Extract checkpoint data.

+
+
Parameters:
+
    +

  • ics – Whether to extract forward restart data.

    • +

  • data – Whether to extract non-linear dependency data.

    • +

  • copy – If True then a copy of the stored data is returned. If +False then internal variables storing the data are returned.

    • +
+
+
Returns:
+

A tuple (cp, data, storage). Elements of this +tuple are as for the three arguments for the +CheckpointStorage.update() method, and here storage is a +VariableStateLockDictionary.

+
+
+
+ +
+
+update(cp, data, storage, *, copy=True)
+

Update the CheckpointStorage using the provided +checkpoint data. Used to update the CheckpointStorage from +loaded data. Note that the CheckpointStorage is not +cleared prior to updating using the provided data.

+
+
Parameters:
+
    +

  • cp – A tuple of keys. Forward restart data is defined by +(storage[key] for key in cp).

    • +

  • data – A dict. Items are ((n, i), keys), indicating +that non-linear dependency data for equation i in block n is +(storage[key] for key in keys).

    • +

  • storage – The stored data. A Mapping with items ((x_id, +x_indices), x_value). x_id is the int ID for a variable +whose value x_value is stored. x_indices is either None, if +the variable value has not been computed by solving equations with +forward restart data storage enabled, or a tuple (n, i, m) +indicating that the variable value was computed as component m of +the solution to equation i in block n.

    • +

  • copy – Whether the values in storage should be copied when being +stored in the CheckpointStorage.

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.checkpointing.ReplayStorage(blocks, N0, N1, *, transpose_deps=None)
+

Storage used when solving forward equations.

+

A value for a forward variable can be accessed via

+
x_value = replay_storage[x]
+
+
+

and set via

+
replay_storage[x] = x_value
+
+
+

where here x is either a variable of an int variable ID. +Containment can also be tested,

+
if x in replay_storage:
+    [...]
+
+
+
+
Parameters:
+
    +

  • blocks – A Sequence or Mapping, whose elements or +values are Sequence objects containing Equation +objects. Forward equations.

    • +

  • N0 – An int. (blocks[n] for n in range(N0, N1)) defines the +forward equations which will be solved.

    • +

  • N1 – An int. (blocks[n] for n in range(N0, N1)) defines the +forward equations which will be solved.

    • +

  • transpose_deps – A TransposeComputationalGraph. If supplied +then an activity analysis is applied.

    • +
+
+
+
+
+is_active(n, i)
+

Return whether the activity analysis indicates that an equation is +‘active’.

+
+
Parameters:
+
    +

  • n – The int index of the block.

    • +

  • i – The int index of the equation.

    • +
+
+
Returns:
+

True if the equation is active, and False otherwise.

+
+
+
+ +
+
+update(d, *, copy=True)
+

Use the supplied Mapping to update forward values.

+
+
Parameters:
+
    +

  • d – A Mapping. Updates values for those keys in d +which are also in the ReplayStorage.

    • +

  • copy – Whether the values in d should be copied when being stored +in the ReplayStorage.

    • +
+
+
+
+ +
+
+popleft()
+

Remove the first equation. Used to deallocate forward variables +which are no longer needed as the solution of forward equations +progresses.

+
+
Returns:
+

A tuple (n, i), indicating that equation i in +block n has been removed.

+
+
+
+ +
+ +
+
+class tlm_adjoint.checkpointing.Checkpoints
+

Disk checkpointing abstract base class.

+
+
+abstract write(n, cp, data, storage)
+

Write checkpoint data.

+
+
Parameters:
+
+
+
+
+ +
+
+abstract read(n, *, ics=True, data=True, ic_ids=None)
+

Read checkpoint data.

+
+
Parameters:
+
    +

  • n – The int index of the block with which the checkpoint +data to be read is associated.

    • +

  • ics – Whether forward restart data should be included.

    • +

  • data – Whether non-linear dependency data should be included.

    • +

  • ic_ids – A Container. If provided then only variables with +ID in ic_ids are included.

    • +
+
+
Returns:
+

A tuple (cp, data, storage). Elements of this +tuple are as for the three arguments for the +CheckpointStorage.update() method.

+
+
+
+ +
+
+abstract delete(n)
+

Delete checkpoint data.

+
+
Parameters:
+

n – The int index of the block with which the checkpoint +data to be deleted is associated.

+
+
+
+ +
+ +
+
+class tlm_adjoint.checkpointing.PickleCheckpoints(prefix, *, comm=None)
+

Disk checkpointing using the pickle module.

+
+
Parameters:
+
    +

  • prefix – Checkpoint files are stored at +[prefix]_[root_pid]_[root_py2f]_[rank].pickle. Here prefix is +defined by this argument, root_pid is the process ID on the root +process (i.e. process 0), root_py2f is the Fortran MPI communicator +on the root process, and rank is the process rank.

    • +

  • comm – A communicator.

    • +
+
+
+
+
+write(n, cp, data, storage)
+

Write checkpoint data.

+
+
Parameters:
+
+
+
+
+ +
+
+read(n, *, ics=True, data=True, ic_ids=None)
+

Read checkpoint data.

+
+
Parameters:
+
    +

  • n – The int index of the block with which the checkpoint +data to be read is associated.

    • +

  • ics – Whether forward restart data should be included.

    • +

  • data – Whether non-linear dependency data should be included.

    • +

  • ic_ids – A Container. If provided then only variables with +ID in ic_ids are included.

    • +
+
+
Returns:
+

A tuple (cp, data, storage). Elements of this +tuple are as for the three arguments for the +CheckpointStorage.update() method.

+
+
+
+ +
+
+delete(n)
+

Delete checkpoint data.

+
+
Parameters:
+

n – The int index of the block with which the checkpoint +data to be deleted is associated.

+
+
+
+ +
+ +
+
+class tlm_adjoint.checkpointing.HDF5Checkpoints(prefix, *, comm=None)
+

Disk checkpointing using the h5py library.

+
+
Parameters:
+
    +

  • prefix – Checkpoint files are stored at +[prefix]_[root_pid]_[root_py2f].hdf5. Here prefix is defined by +this argument, root_pid is the process ID on the root process (i.e. +process 0), and root_py2f is the Fortran MPI communicator on the root +process.

    • +

  • comm – A communicator.

    • +
+
+
+
+
+write(n, cp, data, storage)
+

Write checkpoint data.

+
+
Parameters:
+
+
+
+
+ +
+
+read(n, *, ics=True, data=True, ic_ids=None)
+

Read checkpoint data.

+
+
Parameters:
+
    +

  • n – The int index of the block with which the checkpoint +data to be read is associated.

    • +

  • ics – Whether forward restart data should be included.

    • +

  • data – Whether non-linear dependency data should be included.

    • +

  • ic_ids – A Container. If provided then only variables with +ID in ic_ids are included.

    • +
+
+
Returns:
+

A tuple (cp, data, storage). Elements of this +tuple are as for the three arguments for the +CheckpointStorage.update() method.

+
+
+
+ +
+
+delete(n)
+

Delete checkpoint data.

+
+
Parameters:
+

n – The int index of the block with which the checkpoint +data to be deleted is associated.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/eigendecomposition/index.html b/autoapi/tlm_adjoint/eigendecomposition/index.html new file mode 100644 index 0000000..bce0c60 --- /dev/null +++ b/autoapi/tlm_adjoint/eigendecomposition/index.html @@ -0,0 +1,211 @@ + + + + + + + tlm_adjoint.eigendecomposition — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.eigendecomposition

+
+

Module Contents

+
+
+tlm_adjoint.eigendecomposition.eigendecompose(space, A_action, *, B_action=None, arg_space_type='primal', action_space_type=None, N_eigenvalues=None, solver_type=None, problem_type=None, which=None, tolerance=1e-12, pre_callback=None, post_callback=None)
+

Interface with SLEPc via slepc4py, for the matrix free solution of +eigenproblems

+
+\[A v = \lambda v,\]
+

or generalized eigenproblems

+
+\[A v = \lambda B v.\]
+

Originally developed by loosely following the slepc4py 3.6.0 demo +demo/ex3.py. slepc4py 3.6.0 license information follows:

+
=========================
+LICENSE: SLEPc for Python
+=========================
+
+:Author:  Lisandro Dalcin
+:Contact: dalcinl@gmail.com
+
+
+Copyright (c) 2015, Lisandro Dalcin.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+* Redistributions of source code must retain the above copyright
+  notice, this list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright
+  notice, this list of conditions and the following disclaimer in the
+  documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+
+
Parameters:
+
    +

  • space – The space for each eigenvector.

    • +

  • A_action – A callable. Accepts a single variable argument, and returns +a variable containing the result after left multiplication of the input +by \(A\).

    • +

  • B_action – A callable. Accepts a single variable argument, and returns +a variable containing the result after left multiplication of the input +by \(B\).

    • +

  • arg_space_type – The space type of eigenvectors. ‘primal’, ‘dual’, +‘conjugate’, or ‘conjugate_dual’.

    • +

  • action_space_type – The space type of the result of multiplication by +\(A\) or \(B\). ‘primal’, ‘dual’, ‘conjugate’, or +‘conjugate_dual’. Defaults to the space type conjugate dual to +arg_space_type.

    • +

  • N_eigenvalues – An int, the number of eigenvalues to attempt +to compute. Defaults to the dimension of space.

    • +

  • problem_type – The eigenproblem type – see +slepc4py.SLEPc.EPS.ProblemType. Defaults to +slepc4py.SLEPc.EPS.ProblemType.GNHEP if B_action is supplied, or +slepc4py.SLEPc.EPS.ProblemType.NHEP otherwise.

    • +

  • which – Which eigenvalues to attempt to compute – see +slepc4py.SLEPc.EPS.Which. Defaults to +slepc4py.SLEPc.EPS.Which.LARGEST_MAGNITUDE.

    • +

  • tolerance – Convergence tolerance. By default the convergence criterion +is defined using slepc4py.SLEPc.EPS.Conv.REL.

    • +

  • pre_callback – A callable accepting a single slepc4py.SLEPc.EPS +argument. Used for detailed manual configuration. Called after all +other configuration options are set, but before the +slepc4py.SLEPc.EPS.setUp method is called.

    • +

  • post_callback – A callable accepting a single slepc4py.SLEPc.EPS +argument. Called after the slepc4py.SLEPc.EPS.solve method has been +called.

    • +
+
+
Returns:
+

A tuple (lam, V). lam is a numpy.ndarray +containing eigenvalues. For non-Hermitian algorithms and a real build +of PETSc, V is a tuple (V_r, V_i), where V_r and V_i +are each a tuple of variables containing respectively the real +and complex parts of corresponding eigenvectors. Otherwise V is a +tuple of variables containing corresponding eigenvectors.

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/equation/index.html b/autoapi/tlm_adjoint/equation/index.html new file mode 100644 index 0000000..b9a862e --- /dev/null +++ b/autoapi/tlm_adjoint/equation/index.html @@ -0,0 +1,637 @@ + + + + + + + tlm_adjoint.equation — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.equation

+
+

Module Contents

+
+
+class tlm_adjoint.equation.Equation(X, deps, nl_deps=None, *, ic_deps=None, ic=None, adj_ic_deps=None, adj_ic=None, adj_type='conjugate_dual')
+

Core equation class. Defines a differentiable operation for use as an +adjoint tape record.

+

The equation is defined via a residual function \(\mathcal{F}\). The +forward solution is defined implicitly as the value \(x\) for which

+
+\[\mathcal{F} \left( x, y_0, y_1, \ldots \right) = 0,\]
+

where \(y_i\) are dependencies.

+

This is an abstract base class. Information required to solve forward +equations, perform adjoint calculations, and define tangent-linear +equations, is provided by overloading abstract methods. This class does +not inherit from abc.ABC, so that methods may be implemented as +needed.

+
+
Parameters:
+
    +

  • X – A variable, or a Sequence of variables, defining the +forward solution.

    • +

  • deps – A Sequence of variables defining dependencies. Must +define a superset of X.

    • +

  • nl_deps – A Sequence of variables defining non-linear +dependencies. Must define a subset of deps. Defaults to deps.

    • +

  • ic_deps – A Sequence of variables whose value must be +available prior to computing the forward solution. Intended for +iterative methods with non-zero initial guesses. Must define a subset +of X. Can be overridden by ic.

    • +

  • ic – Whether ic_deps should be set equal to X. Defaults to True +if ic_deps is not supplied, and to False otherwise.

    • +

  • adj_ic_deps – A Sequence of variables whose value must be +available prior to computing the adjoint solution. Intended for +iterative methods with non-zero initial guesses. Must define a subset +of X. Can be overridden by adj_ic.

    • +

  • adj_ic – Whether adj_ic_deps should be set equal to X. Defaults to +True if adj_ic_deps is not supplied, and to False otherwise.

    • +

  • adj_type – The space type relative to X of adjoint variables. +‘primal’ or ‘conjugate_dual’, or a Sequence of these.

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+x()
+

Return the forward solution variable, assuming the forward solution +has one component.

+
+
Returns:
+

A variable defining the forward solution.

+
+
+
+ +
+
+X(m=None)
+

Return forward solution variables.

+
+
Returns:
+

If m is supplied, a variable defining the m th component +of the forward solution. If m is not supplied, a tuple +of variables defining the forward solution.

+
+
+
+ +
+
+dependencies()
+

Return dependencies.

+
+
Returns:
+

A tuple of variables defining dependencies.

+
+
+
+ +
+
+nonlinear_dependencies()
+

Return non-linear dependencies.

+
+
Returns:
+

A tuple of variables defining non-linear +dependencies.

+
+
+
+ +
+
+initial_condition_dependencies()
+

Return ‘initial condition’ dependencies – dependencies whose value +is needed prior to computing the forward solution.

+
+
Returns:
+

A tuple of variables defining initial condition +dependencies.

+
+
+
+ +
+
+adjoint_initial_condition_dependencies()
+

Return adjoint ‘initial condition’ dependencies – dependencies +whose value is needed prior to computing the adjoint solution.

+
+
Returns:
+

A tuple of variables defining adjoint initial +condition dependencies.

+
+
+
+ +
+
+adj_x_type()
+

Return the space type for the adjoint solution, relative to the +forward solution, assuming the forward solution has exactly one +component.

+
+
Returns:
+

One of ‘primal’ or ‘conjugate_dual’.

+
+
+
+ +
+
+adj_X_type(m=None)
+

Return the space type for the adjoint solution, relative to the +forward solution.

+
+
Returns:
+

If m is supplied, one of ‘primal’ or ‘conjugate_dual’ +defining the relative space type for the m th component of the +adjoint solution. If m is not supplied, a tuple whose +elements are ‘primal’ or ‘conjugate_dual’, defining the +relative space type of the adjoint solution.

+
+
+
+ +
+
+new_adj_x()
+

Return a new variable suitable for storing the adjoint solution, +assuming the forward solution has exactly one component.

+
+
Returns:
+

A variable suitable for storing the adjoint solution.

+
+
+
+ +
+
+new_adj_X(m=None)
+

Return new variables suitable for storing the adjoint solution.

+
+
Returns:
+

If m is supplied, a variable suitable for storing the m +th component of the adjoint solution. If m is not supplied, a +tuple of variables suitable for storing the adjoint +solution.

+
+
+
+ +
+
+solve(*, manager=None, annotate=None, tlm=None)
+

Compute the forward solution.

+
+
Parameters:
+
    +

  • manager – The EquationManager. Defaults to manager().

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
+
+ +
+
+forward(X, deps=None)
+

Wraps Equation.forward_solve() to handle cache invalidation.

+
+ +
+
+abstract forward_solve(X, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint(adj_X, nl_deps, B, dep_Bs)
+

Compute the adjoint solution, and subtract terms from other adjoint +right-hand-sides.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a Sequence of variables defining +the initial guess for an iterative solve. May be modified or +returned.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – A sequence of variables defining the right-hand-side of the +adjoint equation. May be modified or returned.

    • +

  • dep_Bs – A Mapping whose items are (dep_index, dep_B). +Each dep_B is an AdjointRHS which should be updated by +subtracting adjoint derivative information computed by +differentiating with respect to self.dependencies()[dep_index].

    • +
+
+
Returns:
+

A tuple of variables defining the adjoint solution, +or None to indicate that the solution is zero.

+
+
+
+ +
+
+adjoint_cached(adj_X, nl_deps, dep_Bs)
+

Subtract terms from other adjoint right-hand-sides.

+
+
Parameters:
+
    +

  • adj_X – A Sequence of variables defining the adjoint +solution. Should not be modified.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_Bs – A Mapping whose items are (dep_index, dep_B). +Each dep_B is an AdjointRHS which should be updated by +subtracting adjoint derivative information computed by +differentiating with respect to self.dependencies()[dep_index].

    • +
+
+
+
+ +
+
+abstract adjoint_derivative_action(nl_deps, dep_index, adj_X)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+subtract_adjoint_derivative_actions(adj_X, nl_deps, dep_Bs)
+

Subtract terms from other adjoint right-hand-sides.

+

Can be overridden for an optimized implementation, but otherwise uses +Equation.adjoint_derivative_action().

+
+
Parameters:
+
    +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_Bs – A Mapping whose items are (dep_index, dep_B). +Each dep_B is an AdjointRHS which should be updated by +subtracting adjoint derivative information computed by +differentiating with respect to self.dependencies()[dep_index].

    • +
+
+
+
+ +
+
+abstract adjoint_jacobian_solve(adj_X, nl_deps, B)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+abstract tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.equation.ZeroAssignment(X)
+

Represents an assignment

+
+\[x = 0.\]
+

The forward residual is defined

+
+\[\mathcal{F} \left( x \right) = x.\]
+
+
Parameters:
+

X – A variable or a Sequence of variables defining the forward +solution \(x\).

+
+
+
+
+forward_solve(X, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_X)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_X, nl_deps, B)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.equation.NullSolver(X)
+

Represents an assignment

+
+\[x = 0.\]
+

The forward residual is defined

+
+\[\mathcal{F} \left( x \right) = x.\]
+
+
Parameters:
+

X – A variable or a Sequence of variables defining the forward +solution \(x\).

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/equations/index.html b/autoapi/tlm_adjoint/equations/index.html new file mode 100644 index 0000000..3e862e1 --- /dev/null +++ b/autoapi/tlm_adjoint/equations/index.html @@ -0,0 +1,888 @@ + + + + + + + tlm_adjoint.equations — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.equations

+
+

Module Contents

+
+
+class tlm_adjoint.equations.EmptyEquation
+

An adjoint tape record with no associated solution variables.

+
+
+forward_solve(X, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.equations.Assignment(x, y)
+

Represents an assignment

+
+\[x = y.\]
+

The forward residual is defined

+
+\[\mathcal{F} \left( x, y \right) = x - y.\]
+
+
Parameters:
+
    +

  • x – A variable defining the forward solution \(x\).

    • +

  • y – A variable defining \(y\).

    • +
+
+
+
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.equations.Conversion(x, y)
+

Represents degree of freedom assignment

+
+\[\tilde{x} = \tilde{y}\]
+

where \(\tilde{x}\) and \(\tilde{y}\) are vectors of degrees of +freedom for \(x\) and \(y\) respectively. Can be used to convert +between different variable types.

+

The forward residual is defined

+
+\[\mathcal{F} \left( x, y \right) = \tilde{x} - \tilde{y}.\]
+
+
Parameters:
+
    +

  • x – A variable defining the forward solution \(x\).

    • +

  • y – A variable defining \(y\).

    • +
+
+
+
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.equations.LinearCombination(x, *args)
+

Represents an assignment

+
+\[x = \sum_i \alpha_i y_i.\]
+

The forward residual is defined

+
+\[\mathcal{F} \left( x, y_1, y_2, \ldots \right) + = x - \sum_i \alpha_i y_i.\]
+
+
Parameters:
+
    +

  • x – A variable defining the forward solution \(x\).

    • +

  • args – A tuple of two element Sequence objects. The +\(i\) th element consists of (alpha_i, y_i), where alpha_i is a +scalar corresponding to \(\alpha_i\) and y_i a variable +corresponding to \(y_i\).

    • +
+
+
+
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.equations.Axpy(y_new, y_old, alpha, x)
+

Represents an assignment

+
+\[y_\text{new} = y_\text{old} + \alpha x.\]
+

The forward residual is defined

+
+\[\mathcal{F} \left( y_\text{new}, y_\text{old}, x \right) + = y_\text{new} - y_\text{old} - \alpha x.\]
+
+
Parameters:
+
    +

  • y_new – A variable defining the forward solution \(y_\text{new}\).

    • +

  • y_old – A variable defining \(y_\text{old}\).

    • +

  • alpha – A scalar defining \(\alpha\).

    • +

  • x – A variable defining \(x\).

    • +
+
+
+
+ +
+
+class tlm_adjoint.equations.DotProduct(x, y, z, *, alpha=1.0)
+

Represents an assignment

+
+\[x = \alpha z^T y.\]
+

The forward residual is defined

+
+\[\mathcal{F} \left( x, y, z \right) = x - \alpha z^T y.\]
+
+
Parameters:
+
    +

  • x – A variable whose degrees of freedom define the forward solution +\(x\).

    • +

  • y – A variable whose degrees of freedom define \(y\).

    • +

  • z – A variable whose degrees of freedom define \(z\). May be the +same variable as y.

    • +

  • alpha – A scalar defining \(\alpha\).

    • +
+
+
+
+ +
+
+class tlm_adjoint.equations.InnerProduct(x, y, z, *, alpha=1.0, M=None)
+

Represents an assignment

+
+\[x = \alpha z^* M y.\]
+

The forward residual is defined

+
+\[\mathcal{F} \left( x, y, z \right) = x - \alpha z^* M y.\]
+
+
Parameters:
+
    +

  • x – A variable whose degrees of freedom define the forward solution +\(x\).

    • +

  • y – A variable whose degrees of freedom define \(y\).

    • +

  • z – A variable whose degrees of freedom define \(z\). May be the +same variable as y.

    • +

  • alpha – A scalar defining \(\alpha\).

    • +

  • M – A tlm_adjoint.linear_equation.Matrix defining \(M\). +Must have no dependencies. Defaults to an identity matrix.

    • +
+
+
+
+ +
+
+class tlm_adjoint.equations.MatrixActionRHS(A, X)
+

Represents a right-hand-side term

+
+\[A x.\]
+
+
Parameters:
+
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+add_forward(B, deps)
+

Add the right-hand-side term to B.

+
+
Parameters:
+
    +

  • B – A variable if it has a single component, and a +Sequence of variables otherwise. Should be updated by the +addition of this RHS. Subclasses may replace this +argument with b if there is a single component.

    • +

  • deps – A Sequence of variables defining values for +dependencies. Should not be modified.

    • +
+
+
+
+ +
+
+subtract_adjoint_derivative_action(nl_deps, dep_index, adj_X, b)
+

Subtract the action of the adjoint of a derivative of the +right-hand-side term, on an adjoint variable, from b.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • deps_index – An int. The derivative is defined by +differentiation of the right-hand-side term with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint variable. A variable if it has a single +component, and a Sequence of variables otherwise. Should +not be modified. Subclasses may replace this argument with adj_x +if the adjoint variable has a single component.

    • +

  • b – A variable storing the result. Should be updated by subtracting +the action of the adjoint of the right-hand-side term on the +adjoint variable.

    • +
+
+
+
+ +
+
+tangent_linear_rhs(M, dM, tlm_map)
+

Construct tangent-linear right-hand-side terms obtained by +differentiation of this right-hand-side term. That is, construct

+
+\[\sum_i \frac{\partial b}{\partial y_i} \tau_{y_i},\]
+

where \(b\) is this right-hand-side term, and \(\tau_{y_i}\) is +the tangent-linear variable associated with a dependency \(y_i\).

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

A RHS, or a Sequence of RHS +objects, defining the right-hand-side terms. Returning None +indicates that there are no terms.

+
+
+
+ +
+ +
+
+class tlm_adjoint.equations.DotProductRHS(x, y, *, alpha=1.0)
+

Represents a right-hand-side term

+
+\[\alpha y^T x.\]
+
+
Parameters:
+
    +

  • x – A variable whose degrees of freedom define \(x\).

    • +

  • y – A variable whose degrees of freedom define \(y\). May be the +same variable as x.

    • +

  • alpha – A scalar defining \(\alpha\).

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+add_forward(b, deps)
+

Add the right-hand-side term to B.

+
+
Parameters:
+
    +

  • B – A variable if it has a single component, and a +Sequence of variables otherwise. Should be updated by the +addition of this RHS. Subclasses may replace this +argument with b if there is a single component.

    • +

  • deps – A Sequence of variables defining values for +dependencies. Should not be modified.

    • +
+
+
+
+ +
+
+subtract_adjoint_derivative_action(nl_deps, dep_index, adj_x, b)
+

Subtract the action of the adjoint of a derivative of the +right-hand-side term, on an adjoint variable, from b.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • deps_index – An int. The derivative is defined by +differentiation of the right-hand-side term with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint variable. A variable if it has a single +component, and a Sequence of variables otherwise. Should +not be modified. Subclasses may replace this argument with adj_x +if the adjoint variable has a single component.

    • +

  • b – A variable storing the result. Should be updated by subtracting +the action of the adjoint of the right-hand-side term on the +adjoint variable.

    • +
+
+
+
+ +
+
+tangent_linear_rhs(M, dM, tlm_map)
+

Construct tangent-linear right-hand-side terms obtained by +differentiation of this right-hand-side term. That is, construct

+
+\[\sum_i \frac{\partial b}{\partial y_i} \tau_{y_i},\]
+

where \(b\) is this right-hand-side term, and \(\tau_{y_i}\) is +the tangent-linear variable associated with a dependency \(y_i\).

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

A RHS, or a Sequence of RHS +objects, defining the right-hand-side terms. Returning None +indicates that there are no terms.

+
+
+
+ +
+ +
+
+class tlm_adjoint.equations.InnerProductRHS(x, y, *, alpha=1.0, M=None)
+

Represents a right-hand-side term

+
+\[\alpha y^* M x.\]
+
+
Parameters:
+
    +

  • x – A variable whose degrees of freedom define \(x\).

    • +

  • y – A variable whose degrees of freedom define \(y\). May be the +same variable as x.

    • +

  • alpha – A scalar defining \(\alpha\).

    • +

  • M – A tlm_adjoint.linear_equation.Matrix defining \(M\). +Must have no dependencies. Defaults to an identity matrix.

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+add_forward(b, deps)
+

Add the right-hand-side term to B.

+
+
Parameters:
+
    +

  • B – A variable if it has a single component, and a +Sequence of variables otherwise. Should be updated by the +addition of this RHS. Subclasses may replace this +argument with b if there is a single component.

    • +

  • deps – A Sequence of variables defining values for +dependencies. Should not be modified.

    • +
+
+
+
+ +
+
+subtract_adjoint_derivative_action(nl_deps, dep_index, adj_x, b)
+

Subtract the action of the adjoint of a derivative of the +right-hand-side term, on an adjoint variable, from b.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • deps_index – An int. The derivative is defined by +differentiation of the right-hand-side term with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint variable. A variable if it has a single +component, and a Sequence of variables otherwise. Should +not be modified. Subclasses may replace this argument with adj_x +if the adjoint variable has a single component.

    • +

  • b – A variable storing the result. Should be updated by subtracting +the action of the adjoint of the right-hand-side term on the +adjoint variable.

    • +
+
+
+
+ +
+
+tangent_linear_rhs(M, dM, tlm_map)
+

Construct tangent-linear right-hand-side terms obtained by +differentiation of this right-hand-side term. That is, construct

+
+\[\sum_i \frac{\partial b}{\partial y_i} \tau_{y_i},\]
+

where \(b\) is this right-hand-side term, and \(\tau_{y_i}\) is +the tangent-linear variable associated with a dependency \(y_i\).

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

A RHS, or a Sequence of RHS +objects, defining the right-hand-side terms. Returning None +indicates that there are no terms.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/fenics/backend_interface/index.html b/autoapi/tlm_adjoint/fenics/backend_interface/index.html new file mode 100644 index 0000000..a869107 --- /dev/null +++ b/autoapi/tlm_adjoint/fenics/backend_interface/index.html @@ -0,0 +1,161 @@ + + + + + + + tlm_adjoint.fenics.backend_interface — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.fenics.backend_interface

+
+

Module Contents

+
+
+class tlm_adjoint.fenics.backend_interface.Function(*args, space_type='primal', static=False, cache=None, **kwargs)
+

Extends the DOLFIN Function class.

+
+
Parameters:
+
    +

  • space_type – The space type for the Function. ‘primal’, +‘dual’, ‘conjugate’, or ‘conjugate_dual’.

    • +

  • static – Defines whether the Function is static, meaning that +it is stored by reference in checkpointing/replay, and an associated +tangent-linear variable is zero.

    • +

  • cache – Defines whether results involving the Function may be +cached. Default static.

    • +
+
+
+

Remaining arguments are passed to the DOLFIN Function constructor.

+
+ +
+
+class tlm_adjoint.fenics.backend_interface.ZeroFunction(*args, **kwargs)
+

A Function which is flagged as having a value of zero.

+

Arguments are passed to the Function constructor, together with +static=True and cache=True.

+
+ +
+
+tlm_adjoint.fenics.backend_interface.to_fenics(y, space, *, name=None)
+

Convert a variable to a DOLFIN Function.

+
+
Parameters:
+
    +

  • y – A variable.

    • +

  • space – The space for the return value.

    • +

  • name – A str name.

    • +
+
+
Returns:
+

The DOLFIN Function.

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/fenics/caches/index.html b/autoapi/tlm_adjoint/fenics/caches/index.html new file mode 100644 index 0000000..69a4781 --- /dev/null +++ b/autoapi/tlm_adjoint/fenics/caches/index.html @@ -0,0 +1,238 @@ + + + + + + + tlm_adjoint.fenics.caches — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.fenics.caches

+

This module implements finite element assembly and linear solver data +caching.

+
+

Module Contents

+
+
+class tlm_adjoint.fenics.caches.AssemblyCache
+

A Cache for finite element assembly data.

+
+
+assemble(form, *, bcs=None, form_compiler_parameters=None, linear_solver_parameters=None, replace_map=None)
+

Perform finite element assembly and cache the result, or return a +previously cached result.

+
+
Parameters:
+
    +

  • form – The ufl.Form to assemble.

    • +

  • bcs – Dirichlet boundary conditions.

    • +

  • form_compiler_parameters – Form compiler parameters.

    • +

  • linear_solver_parameters – Linear solver parameters. Required for +assembly parameters which appear in the linear solver parameters.

    • +

  • replace_map – A Mapping defining a map from symbolic +variables to values.

    • +
+
+
Returns:
+

A tuple (value_ref, value), where value is the +result of the finite element assembly, and value_ref is a +CacheRef storing a reference to value.

+
+
    +

  • For an arity zero or arity one form value_ref stores the +assembled value.

    • +

  • For an arity two form value_ref is a tuple (A, b_bc). A +is the assembled matrix, and b_bc is a boundary condition +right-hand-side term which should be added after assembling a +right-hand-side with homogeneous boundary conditions applied. +b_bc may be None to indicate that this term is zero.

    • +
+
+

+
+
+
+ +
+ +
+
+class tlm_adjoint.fenics.caches.LinearSolverCache
+

A Cache for linear solver data.

+
+
+linear_solver(form, *, bcs=None, form_compiler_parameters=None, linear_solver_parameters=None, replace_map=None, assembly_cache=None)
+

Construct a linear solver and cache the result, or return a +previously cached result.

+
+
Parameters:
+
    +

  • form – An arity two ufl.Form, defining the matrix.

    • +

  • bcs – Dirichlet boundary conditions.

    • +

  • form_compiler_parameters – Form compiler parameters.

    • +

  • linear_solver_parameters – Linear solver parameters.

    • +

  • replace_map – A Mapping defining a map from symbolic +variables to values.

    • +

  • assembly_cache – AssemblyCache to use for finite element +assembly. Defaults to assembly_cache().

    • +
+
+
Returns:
+

A tuple (value_ref, value). value is a tuple +(solver, A, b_bc), where solver is the linear solver, A is +the assembled matrix, and b_bc is a boundary condition +right-hand-side term which should be added after assembling a +right-hand-side with homogeneous boundary conditions applied. +b_bc may be None to indicate that this term is zero. +value_ref is a CacheRef storing a reference to value.

+
+
+
+ +
+ +
+
+tlm_adjoint.fenics.caches.assembly_cache()
+
+
Returns:
+

The default AssemblyCache.

+
+
+
+ +
+
+tlm_adjoint.fenics.caches.set_assembly_cache(assembly_cache)
+

Set the default AssemblyCache.

+
+
Parameters:
+

assembly_cache – The new default AssemblyCache.

+
+
+
+ +
+
+tlm_adjoint.fenics.caches.linear_solver_cache()
+
+
Returns:
+

The default LinearSolverCache.

+
+
+
+ +
+
+tlm_adjoint.fenics.caches.set_linear_solver_cache(linear_solver_cache)
+

Set the default LinearSolverCache.

+
+
Parameters:
+

linear_solver_cache – The new default LinearSolverCache.

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/fenics/equations/index.html b/autoapi/tlm_adjoint/fenics/equations/index.html new file mode 100644 index 0000000..8550451 --- /dev/null +++ b/autoapi/tlm_adjoint/fenics/equations/index.html @@ -0,0 +1,742 @@ + + + + + + + tlm_adjoint.fenics.equations — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.fenics.equations

+

This module implements finite element calculations. In particular the +EquationSolver class implements the solution of finite element +variational problems.

+
+

Module Contents

+
+
+class tlm_adjoint.fenics.equations.Assembly(x, rhs, *, form_compiler_parameters=None, match_quadrature=None)
+

Represents assignment to the result of finite element assembly:

+
x = assemble(rhs)
+
+
+

The forward residual \(\mathcal{F}\) is defined so that \(\partial +\mathcal{F} / \partial x\) is the identity.

+
+
Parameters:
+
    +

  • x – A variable defining the forward solution.

    • +

  • rhs – A ufl.Form to assemble. Should have arity 0 or 1, and +should not depend on x.

    • +

  • form_compiler_parameters – Form compiler parameters.

    • +

  • match_quadrature – Whether to set quadrature parameters consistently in +the forward, adjoint, and tangent-linears. Defaults to +parameters[‘tlm_adjoint’][‘Assembly’][‘match_quadrature’].

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.fenics.equations.EquationSolver(eq, x, bcs=None, *, J=None, form_compiler_parameters=None, solver_parameters=None, adjoint_solver_parameters=None, tlm_solver_parameters=None, cache_jacobian=None, cache_adjoint_jacobian=None, cache_tlm_jacobian=None, cache_rhs_assembly=None, match_quadrature=None)
+

Represents the solution of a finite element variational problem.

+

Caching is based on the approach described in

+
+
    +

  • J. R. Maddison and P. E. Farrell, ‘Rapid development and adjoining of +transient finite element models’, Computer Methods in Applied +Mechanics and Engineering, 276, 95–121, 2014, doi: +10.1016/j.cma.2014.03.010

    • +
+
+

The arguments eq, x, bcs, J, form_compiler_parameters, and +solver_parameters are based on the interface for the DOLFIN +dolfin.solve function (see e.g. FEniCS 2017.1.0).

+
+
Parameters:
+
    +

  • eq – A ufl.equation.Equation defining the finite element +variational problem.

    • +

  • x – A DOLFIN Function defining the forward solution.

    • +

  • bcs – Dirichlet boundary conditions.

    • +

  • J – A ufl.Form defining a Jacobian matrix approximation to use +in a non-linear forward solve.

    • +

  • form_compiler_parameters – Form compiler parameters.

    • +

  • solver_parameters – Linear or non-linear solver parameters.

    • +

  • adjoint_solver_parameters – Linear solver parameters to use in an +adjoint solve.

    • +

  • tlm_solver_parameters – Linear solver parameters to use when solving +tangent-linear problems.

    • +

  • cache_jacobian – Whether to cache the forward Jacobian matrix and +linear solver data. Defaults to +parameters[‘tlm_adjoint’][‘EquationSolver][‘cache_jacobian’]. If +None then caching is autodetected.

    • +

  • cache_adjoint_jacobian – Whether to cache the adjoint Jacobian matrix +and linear solver data. Defaults to cache_jacobian.

    • +

  • cache_tlm_jacobian – Whether to cache the Jacobian matrix and linear +solver data when solving tangent-linear problems. Defaults to +cache_jacobian.

    • +

  • cache_rhs_assembly – Whether to enable right-hand-side caching. If +enabled then right-hand-side terms are divided into terms which are +cached, terms which are converted into matrix multiplication by a +cached matrix, and terms which are not cached. Defaults to +parameters[‘tlm_adjoint’][‘EquationSolver’][‘cache_rhs_assembly’].

    • +

  • match_quadrature – Whether to set quadrature parameters consistently in +the forward, adjoint, and tangent-linears. Defaults to +parameters[‘tlm_adjoint’][‘EquationSolver’][‘match_quadrature’].

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+subtract_adjoint_derivative_actions(adj_x, nl_deps, dep_Bs)
+

Subtract terms from other adjoint right-hand-sides.

+

Can be overridden for an optimized implementation, but otherwise uses +Equation.adjoint_derivative_action().

+
+
Parameters:
+
    +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_Bs – A Mapping whose items are (dep_index, dep_B). +Each dep_B is an AdjointRHS which should be updated by +subtracting adjoint derivative information computed by +differentiating with respect to self.dependencies()[dep_index].

    • +
+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+tlm_adjoint.fenics.equations.expr_new_x(expr, x, *, annotate=None, tlm=None)
+

If an expression depends on x, then record the assignment x_old = +x, and replace x with x_old in the expression.

+
+
Parameters:
+
    +

  • expr – A ufl.core.expr.Expr.

    • +

  • x – Defines x.

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
Returns:
+

A ufl.core.expr.Expr with x replaced with x_old, or +expr if the expression does not depend on x.

+
+
+
+ +
+
+tlm_adjoint.fenics.equations.linear_equation_new_x(eq, x, *, annotate=None, tlm=None)
+

If a symbolic expression for a linear finite element variational problem +depends on the symbolic variable representing the problem solution x, +then record the assignment x_old = x, and replace x with x_old in the +symbolic expression.

+
+
Parameters:
+
    +

  • eq – A ufl.equation.Equation defining the finite element +variational problem.

    • +

  • x – A DOLFIN Function defining the solution to the finite element +variational problem.

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
Returns:
+

A ufl.equation.Equation with x replaced with x_old, +or eq if the symbolic expression does not depend on x.

+
+
+
+ +
+
+class tlm_adjoint.fenics.equations.Projection(x, rhs, *args, **kwargs)
+

Represents the solution of a finite element variational problem +performing a projection onto the space for x.

+
+
Parameters:
+
    +

  • x – A DOLFIN Function defining the forward solution.

    • +

  • rhs – A ufl.core.expr.Expr defining the expression to project +onto the space for x, or a ufl.Form defining the +right-hand-side of the finite element variational problem. Should not +depend on x.

    • +
+
+
+

Remaining arguments are passed to the EquationSolver constructor.

+
+ +
+
+class tlm_adjoint.fenics.equations.DirichletBCApplication(x, y, *args, **kwargs)
+

Represents the application of a Dirichlet boundary condition to a zero +valued DOLFIN Function. Specifically this represents:

+
x.vector().zero()
+DirichletBC(x.function_space(), y,
+            *args, **kwargs).apply(x.vector())
+
+
+

The forward residual \(\mathcal{F}\) is defined so that \(\partial +\mathcal{F} / \partial x\) is the identity.

+
+
Parameters:
+
    +

  • x – A DOLFIN Function, updated by the above operations.

    • +

  • y – A DOLFIN Function, defines the Dirichet boundary condition.

    • +
+
+
+

Remaining arguments are passed to the DOLFIN DirichletBC constructor.

+
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.fenics.equations.ExprInterpolation(x, rhs)
+

Represents interpolation of rhs onto the space for x.

+

The forward residual \(\mathcal{F}\) is defined so that \(\partial +\mathcal{F} / \partial x\) is the identity.

+
+
Parameters:
+
    +

  • x – The forward solution.

    • +

  • rhs – A ufl.core.expr.Expr defining the expression to +interpolate onto the space for x. Should not depend on x.

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/fenics/fenics_equations/index.html b/autoapi/tlm_adjoint/fenics/fenics_equations/index.html new file mode 100644 index 0000000..cce35a6 --- /dev/null +++ b/autoapi/tlm_adjoint/fenics/fenics_equations/index.html @@ -0,0 +1,432 @@ + + + + + + + tlm_adjoint.fenics.fenics_equations — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.fenics.fenics_equations

+

This module includes additional functionality for use with FEniCS.

+
+

Module Contents

+
+
+class tlm_adjoint.fenics.fenics_equations.LocalSolverCache
+

A Cache for element-wise local block diagonal linear solvers.

+
+
+local_solver(form, solver_type=None, *, replace_map=None)
+

Construct an element-wise local block diagonal linear solver and +cache the result, or return a previously cached result.

+
+
Parameters:
+
    +

  • form – An arity two ufl.Form, defining the element-wise +local block diagonal matrix.

    • +

  • local_solver – dolfin.LocalSolver.SolverType. Defaults to +dolfin.LocalSolver.SolverType.LU.

    • +

  • replace_map – A Mapping defining a map from symbolic +variables to values.

    • +
+
+
Returns:
+

A tuple (value_ref, value). value is a +DOLFIN LocalSolver and value_ref is a CacheRef +storing a reference to value.

+
+
+
+ +
+ +
+
+tlm_adjoint.fenics.fenics_equations.local_solver_cache()
+
+
Returns:
+

The default LocalSolverCache.

+
+
+
+ +
+
+tlm_adjoint.fenics.fenics_equations.set_local_solver_cache(local_solver_cache)
+

Set the default LocalSolverCache.

+
+
Parameters:
+

local_solver_cache – The new default LocalSolverCache.

+
+
+
+ +
+
+class tlm_adjoint.fenics.fenics_equations.LocalProjection(x, rhs, *, form_compiler_parameters=None, cache_jacobian=None, cache_rhs_assembly=None, match_quadrature=None)
+

Represents the solution of a finite element variational problem +performing a projection onto the space for x, for the case where the mass +matrix is element-wise local block diagonal.

+
+
Parameters:
+
    +

  • x – A DOLFIN Function defining the forward solution.

    • +

  • rhs – A ufl.core.expr.Expr defining the expression to project +onto the space for x, or a ufl.Form defining the +right-hand-side of the finite element variational problem. Should not +depend on x.

    • +
+
+
+

Remaining arguments are passed to the +tlm_adjoint.fenics.equations.EquationSolver constructor.

+
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.fenics.fenics_equations.Interpolation(x, y, *, x_coords=None, P=None, tolerance=0.0)
+

Represents interpolation of the scalar-valued function y onto the +space for x.

+

The forward residual \(\mathcal{F}\) is defined so that \(\partial +\mathcal{F} / \partial x\) is the identity.

+

Internally this builds (or uses a supplied) interpolation matrix for the +local process only. This behaves correctly if the there are no edges +between owned and non-owned nodes in the degree of freedom graph associated +with the discrete function space for y.

+
+
Parameters:
+
    +

  • x – A scalar-valued DOLFIN Function defining the forward solution.

    • +

  • y – A scalar-valued DOLFIN Function to interpolate onto the space for +x.

    • +

  • X_coords – A numpy.ndarray defining the coordinates at which +to interpolate y. Shape is (n, d) where n is the number of +process local degrees of freedom for x and d is the geometric +dimension. Defaults to the process local degree of freedom locations +for x. Ignored if P is supplied.

    • +

  • P – The interpolation matrix. A scipy.sparse.spmatrix.

    • +

  • tolerance – Maximum permitted distance (as returned by the DOLFIN +BoundingBoxTree.compute_closest_entity method) of an interpolation +point from a cell in the mesh for y. Ignored if P is supplied.

    • +
+
+
+
+ +
+
+class tlm_adjoint.fenics.fenics_equations.PointInterpolation(X, y, X_coords=None, *, P=None, tolerance=0.0)
+

Represents interpolation of a scalar-valued function at given points.

+

The forward residual \(\mathcal{F}\) is defined so that \(\partial +\mathcal{F} / \partial x\) is the identity.

+

Internally this builds (or uses a supplied) interpolation matrix for the +local process only. This behaves correctly if the there are no edges +between owned and non-owned nodes in the degree of freedom graph associated +with the discrete function space for y.

+
+
Parameters:
+
    +

  • X – A scalar variable, or a Sequence of scalar variables, +defining the forward solution.

    • +

  • y – A scalar-valued DOLFIN Function to interpolate.

    • +

  • X_coords – A numpy.ndarray defining the coordinates at which +to interpolate y. Shape is (n, d) where n is the number of +interpolation points and d is the geometric dimension. Ignored if P +is supplied.

    • +

  • P – The interpolation matrix. A scipy.sparse.spmatrix.

    • +

  • tolerance – Maximum permitted distance (as returned by the DOLFIN +BoundingBoxTree.compute_closest_entity method) of an interpolation +point from a cell in the mesh for y. Ignored if P is supplied.

    • +
+
+
+
+
+forward_solve(X, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_X)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_X, nl_deps, B)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/fenics/functions/index.html b/autoapi/tlm_adjoint/fenics/functions/index.html new file mode 100644 index 0000000..6feba30 --- /dev/null +++ b/autoapi/tlm_adjoint/fenics/functions/index.html @@ -0,0 +1,228 @@ + + + + + + + tlm_adjoint.fenics.functions — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.fenics.functions

+

This module includes functionality for interacting with FEniCS variables +and Dirichlet boundary conditions.

+
+

Module Contents

+
+
+class tlm_adjoint.fenics.functions.Constant(value=None, *args, name=None, domain=None, space=None, space_type='primal', shape=None, comm=None, static=False, cache=None, **kwargs)
+

Extends the DOLFIN Constant class.

+
+
Parameters:
+
    +

  • value – The initial value. None indicates a value of zero.

    • +

  • name – A str name.

    • +

  • domain – The domain on which the Constant is defined.

    • +

  • space – The space on which the Constant is defined.

    • +

  • space_type – The space type for the Constant. ‘primal’, +‘dual’, ‘conjugate’, or ‘conjugate_dual’.

    • +

  • shape – A tuple of int objects defining the shape of +the value.

    • +

  • comm – The communicator for the Constant.

    • +

  • static – Defines whether the Constant is static, meaning that +it is stored by reference in checkpointing/replay, and an associated +tangent-linear variable is zero.

    • +

  • cache – Defines whether results involving the Constant may be +cached. Default static.

    • +
+
+
+

Remaining arguments are passed to the DOLFIN Constant constructor.

+
+ +
+
+class tlm_adjoint.fenics.functions.Zero
+

Mixin for defining a zero-valued variable. Used for zero-valued +variables for which UFL zero elimination should not be applied.

+
+ +
+
+class tlm_adjoint.fenics.functions.ZeroConstant(*, name=None, domain=None, space=None, space_type='primal', shape=None, comm=None)
+

A Constant which is flagged as having a value of zero.

+

Arguments are passed to the Constant constructor, together with +static=True and cache=True.

+
+ +
+
+tlm_adjoint.fenics.functions.eliminate_zeros(expr)
+

Apply zero elimination for Zero objects in the supplied +ufl.core.expr.Expr or ufl.Form.

+
+
Parameters:
+

expr – A ufl.core.expr.Expr or ufl.Form.

+
+
Returns:
+

A ufl.core.expr.Expr or ufl.Form with zero +elimination applied. May return expr.

+
+
+
+ +
+
+class tlm_adjoint.fenics.functions.DirichletBC(V, g, sub_domain, *args, static=None, _homogeneous=False, **kwargs)
+

Extends the DOLFIN DirichletBC class.

+
+
Parameters:
+

static – A flag that indicates that the value for the +DirichletBC will not change, and which determines whether +calculations involving this DirichletBC can be cached. If +None then autodetected from the value.

+
+
+

Remaining arguments are passed to the DOLFIN DirichletBC constructor.

+
+ +
+
+class tlm_adjoint.fenics.functions.HomogeneousDirichletBC(V, sub_domain, *args, **kwargs)
+

A DirichletBC whose value is zero.

+

Arguments are passed to the DirichletBC constructor, together +with static=True.

+
+ +
+
+class tlm_adjoint.fenics.functions.Replacement(x, count)
+

Represents a symbolic variable but with no value.

+
+ +
+
+class tlm_adjoint.fenics.functions.ReplacementConstant(x, count)
+

Represents a symbolic DOLFIN Constant, but has no value.

+
+ +
+
+class tlm_adjoint.fenics.functions.ReplacementFunction(x, count)
+

Represents a symbolic DOLFIN Function, but has no value.

+
+ +
+
+class tlm_adjoint.fenics.functions.ReplacementZeroConstant(*args, **kwargs)
+

Represents a symbolic DOLFIN Constant which is zero, but has no value.

+
+ +
+
+class tlm_adjoint.fenics.functions.ReplacementZeroFunction(*args, **kwargs)
+

Represents a symbolic DOLFIN Function which is zero, but has no value.

+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/firedrake/backend_interface/index.html b/autoapi/tlm_adjoint/firedrake/backend_interface/index.html new file mode 100644 index 0000000..add7225 --- /dev/null +++ b/autoapi/tlm_adjoint/firedrake/backend_interface/index.html @@ -0,0 +1,198 @@ + + + + + + + tlm_adjoint.firedrake.backend_interface — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.firedrake.backend_interface

+
+

Module Contents

+
+
+class tlm_adjoint.firedrake.backend_interface.Function(*args, space_type='primal', static=False, cache=None, **kwargs)
+

Extends firedrake.function.Function.

+
+
Parameters:
+
    +

  • space_type – The space type for the Function. ‘primal’ or +‘conjugate’.

    • +

  • static – Defines whether the Function is static, meaning that +it is stored by reference in checkpointing/replay, and an associated +tangent-linear variable is zero.

    • +

  • cache – Defines whether results involving the Function may +be cached. Default static.

    • +
+
+
+

Remaining arguments are passed to the firedrake.function.Function +constructor.

+
+ +
+
+class tlm_adjoint.firedrake.backend_interface.ZeroFunction(*args, **kwargs)
+

A Function which is flagged as having a value of zero.

+

Arguments are passed to the Function constructor, together with +static=True and cache=True.

+
+ +
+
+class tlm_adjoint.firedrake.backend_interface.Cofunction(*args, space_type='conjugate_dual', static=False, cache=None, **kwargs)
+

Extends the firedrake.cofunction.Cofunction class.

+
+
Parameters:
+
    +

  • space_type – The space type for the Cofunction. +‘conjugate’ or ‘conjugate_dual’.

    • +

  • static – Defines whether the Cofunction is static, meaning +that it is stored by reference in checkpointing/replay, and an +associated tangent-linear variable is zero.

    • +

  • cache – Defines whether results involving the Cofunction may +be cached. Default static.

    • +
+
+
+

Remaining arguments are passed to the +firedrake.cofunction.Cofunction constructor.

+
+
+equals(other)
+

Check equality.

+
+ +
+ +
+
+class tlm_adjoint.firedrake.backend_interface.ReplacementCofunction(x, count)
+

Represents a symbolic firedrake.cofunction.Cofunction, but has +no value.

+
+ +
+
+tlm_adjoint.firedrake.backend_interface.to_firedrake(y, space, *, name=None)
+

Convert a variable to a firedrake.function.Function or +firedrake.cofunction.Cofunction.

+
+
Parameters:
+
    +

  • y – A variable.

    • +

  • space – The space for the return value.

    • +

  • name – A str name.

    • +
+
+
Returns:
+

The firedrake.function.Function or +firedrake.cofunction.Cofunction.

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/firedrake/block_system/index.html b/autoapi/tlm_adjoint/firedrake/block_system/index.html new file mode 100644 index 0000000..69f6b46 --- /dev/null +++ b/autoapi/tlm_adjoint/firedrake/block_system/index.html @@ -0,0 +1,1050 @@ + + + + + + + tlm_adjoint.firedrake.block_system — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.firedrake.block_system

+

This module implements solvers for linear systems defined in mixed spaces.

+

The System class defines the block structure of the linear system, +and solves the system using an outer Krylov solver. A custom preconditioner can +be defined via the pc_fn callback to System.solve(), and this +preconditioner can itself e.g. make use of further Krylov solvers. This +provides a Python interface for custom block preconditioners.

+

Given a linear problem with a potentially singular matrix \(A\)

+
+\[A u = b,\]
+

a System instead solves the linear problem

+
+\[\left[ (I - M U (U^* M U)^{-1} U^*) A (I - V (V^* C V)^{-1} V^* C) + + M U S V^* C \right] u = (I - M U (U^* M U)^{-1} U^*) b.\]
+

Here

+
+
    +

  • \(U\) is a full rank matrix whose columns span the left nullspace for +a modified system matrix \(\tilde{A}\).

    • +

  • \(V\) is a full rank matrix with the same number of columns as +\(U\), whose columns span the nullspace for \(\tilde{A}\).

    • +

  • \(V^* C V\) and \(S\) are invertible matrices.

    • +

  • \(M\) is a Hermitian positive definite matrix.

    • +
+
+

Here the left nullspace for a matrix is defined to be the nullspace for its +Hermitian transpose, and the modified system matrix \(\tilde{A}\) is +defined

+
+\[\tilde{A} = (I - M U (U^* M U)^{-1} U^*) A (I - V (V^* C V)^{-1} V^* C).\]
+

This has two primary use cases:

+
+
    +

  1. Where a matrix \(A\) and right-hand-side \(b\) are constructed +via finite element assembly on superspaces of the test space and trial +space. The typical example is in the application of homogeneous +essential Dirichlet boundary conditions.

    2. +

  2. Where the matrix \(A\) is singular and \(b\) is orthogonal to +the left nullspace of \(A\). Typically one would then choose +\(U\) and \(V\) so that their columns respectively span the left +nullspace and nullspace of \(A\), and the System then +seeks a solution to the original problem subject to the linear +constraints \(V^* C u = 0\).

    3. +
+
+

Function spaces are defined via Firedrake function spaces, and +Sequence objects containing Firedrake function spaces or similar +Sequence objects. Similarly functions are defined via +firedrake.function.Function or +firedrake.cofunction.Cofunction objects, or Sequence objects +containing firedrake.function.Function, +firedrake.cofunction.Cofunction, or similar Sequence objects. +This defines a basic tree structure which is useful e.g. when defining block +matrices in terms of sub-block matrices.

+

Elements of the tree are accessed in a consistent order using a depth first +search. Hence e.g.

+
((u_0, u_1), u_2)
+
+
+

and

+
(u_0, u_1, u_2)
+
+
+

where u_0, u_1, and u_2 are firedrake.function.Function or +firedrake.cofunction.Cofunction objects, are both valid +representations of a mixed space solution.

+
+

Module Contents

+
+
+class tlm_adjoint.firedrake.block_system.MixedSpace(spaces)
+

Used to map between different versions of a mixed space.

+

This class defines two representations for the space:

+
+
    +

  1. As a ‘split space’: A tree defining the mixed space. Stored using +Firedrake function space and tuple objects, each +corresponding to a node in the tree. Function spaces correspond to +leaf nodes, and tuple objects to other nodes in the tree.

    2. +

  2. As a ‘flattened space’: A Sequence containing leaf nodes of +the split space with an ordering determined using a depth first +search.

    3. +
+
+

Provides methods to allow data to be copied to and from a compatible +petsc4py.PETSc.Vec. This allows, for example, the construction:

+
u_0 = Function(space_0, name='u_0')
+u_1 = Function(space_1, name='u_1')
+u_2 = Function(space_2, name='u_2')
+
+mixed_space = MixedSpace(((space_0, space_1), space_2))
+
+
+

and then data can be copied to a compatible petsc4py.PETSc.Vec via

+
mixed_space.to_petsc(u_petsc, ((u_0, u_1), u_2))
+
+
+

and from a compatible petsc4py.PETSc.Vec via

+
mixed_space.from_petsc(u_petsc, ((u_0, u_1), u_2))
+
+
+
+
Parameters:
+

spaces – The split space.

+
+
+
+
+property comm
+

The communicator associated with the mixed space.

+
+ +
+
+property split_space
+

The split space representation.

+
+ +
+
+property flattened_space
+

The flattened space representation.

+
+ +
+
+property local_size
+

The number of local degrees of freedom.

+
+ +
+
+property global_size
+

The global number of degrees of freedom.

+
+ +
+
+new_split()
+
+
Returns:
+

A new element in the split space.

+
+
+
+ +
+
+from_petsc(u_petsc, u)
+

Copy data from a compatible petsc4py.PETSc.Vec.

+
+
Parameters:
+
+
+
+
+ +
+
+to_petsc(u_petsc, u)
+

Copy data to a compatible petsc4py.PETSc.Vec. Does not +update the ghost.

+
+
Parameters:
+
+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.block_system.Nullspace
+

Represents a matrix nullspace and left nullspace.

+
+
+abstract apply_nullspace_transformation_lhs_right(x)
+

Apply the nullspace transformation associated with a matrix action +on \(x\),

+
+\[x \rightarrow (I - V (V^* C V)^{-1} V^* C) x.\]
+
+
Parameters:
+

x – Defines \(x\).

+
+
+
+ +
+
+abstract apply_nullspace_transformation_lhs_left(y)
+

Apply the left nullspace transformation associated with a matrix +action,

+
+\[y \rightarrow (I - M U (U^* M U)^{-1} U^*) y.\]
+
+
Parameters:
+

y – Defines \(y\).

+
+
+
+ +
+
+abstract constraint_correct_lhs(x, y)
+

Add the linear constraint term to \(y\),

+
+\[y \rightarrow y + M U S V^* C x.\]
+
+
Parameters:
+
    +

  • x – Defines \(x\).

    • +

  • y – Defines \(y\).

    • +
+
+
+
+ +
+
+abstract pc_constraint_correct_soln(u, b)
+

Add the preconditioner linear constraint term to \(u\),

+
+\[u \rightarrow u + V \tilde{S}^{-1} U^* b,\]
+

with

+
+\[\tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}.\]
+
+
Parameters:
+
    +

  • u – Defines \(u\).

    • +

  • b – Defines \(b\).

    • +
+
+
+
+ +
+
+correct_soln(x)
+

Correct the linear system solution so that it is orthogonal to +space spanned by the columns of \(V\).

+
+
Parameters:
+

x – The linear system solution, to be corrected.

+
+
+
+ +
+
+pre_mult_correct_lhs(x)
+

Apply the pre-left-multiplication nullspace transformation.

+
+
Parameters:
+

x – Defines the vector on which the matrix action is computed.

+
+
+
+ +
+
+post_mult_correct_lhs(x, y)
+

Apply the post-left-multiplication nullspace transformation, and add +the linear constraint term.

+
+
Parameters:
+
    +

  • x – Defines the vector on which the matrix action is computed, and +used to add the linear constraint term. If None is supplied then +the linear constraint term is not added.

    • +

  • y – Defines the result of the matrix action on x.

    • +
+
+
+
+ +
+
+correct_rhs(b)
+

Correct the linear system right-hand-side so that it is orthogonal +to the space spanned by the columns of \(U\).

+
+
Parameters:
+

b – The linear system right-hand-side, to be corrected.

+
+
+
+ +
+
+pc_pre_mult_correct(b)
+

Apply the pre-preconditioner-application nullspace transformation.

+
+
Parameters:
+

b – Defines the vector on which the preconditioner action is +computed.

+
+
+
+ +
+
+pc_post_mult_correct(u, b)
+

Apply the post-preconditioner-application left nullspace +transformation, and add the linear constraint term.

+
+
Parameters:
+
    +

  • u – Defines the result of the preconditioner action on b.

    • +

  • b – Defines the vector on which the preconditioner action is +computed, and used to add the linear constraint term. If None is +supplied then the linear constraint term is not added.

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.block_system.NoneNullspace
+

An empty nullspace and left nullspace.

+
+
+apply_nullspace_transformation_lhs_right(x)
+

Apply the nullspace transformation associated with a matrix action +on \(x\),

+
+\[x \rightarrow (I - V (V^* C V)^{-1} V^* C) x.\]
+
+
Parameters:
+

x – Defines \(x\).

+
+
+
+ +
+
+apply_nullspace_transformation_lhs_left(y)
+

Apply the left nullspace transformation associated with a matrix +action,

+
+\[y \rightarrow (I - M U (U^* M U)^{-1} U^*) y.\]
+
+
Parameters:
+

y – Defines \(y\).

+
+
+
+ +
+
+constraint_correct_lhs(x, y)
+

Add the linear constraint term to \(y\),

+
+\[y \rightarrow y + M U S V^* C x.\]
+
+
Parameters:
+
    +

  • x – Defines \(x\).

    • +

  • y – Defines \(y\).

    • +
+
+
+
+ +
+
+pc_constraint_correct_soln(u, b)
+

Add the preconditioner linear constraint term to \(u\),

+
+\[u \rightarrow u + V \tilde{S}^{-1} U^* b,\]
+

with

+
+\[\tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}.\]
+
+
Parameters:
+
    +

  • u – Defines \(u\).

    • +

  • b – Defines \(b\).

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.block_system.ConstantNullspace(*, alpha=1.0)
+

A nullspace and left nullspace spanned by the vector of ones.

+

Here \(V = U\), \(U\) is a single column matrix whose elements are +ones, \(C = M\), and \(M\) is an identity matrix.

+
+
Parameters:
+

alpha – Defines the linear constraint matrix \(S = \left( \alpha / +N \right)\) where \(N\) is the length of the vector of ones.

+
+
+
+
+apply_nullspace_transformation_lhs_right(x)
+

Apply the nullspace transformation associated with a matrix action +on \(x\),

+
+\[x \rightarrow (I - V (V^* C V)^{-1} V^* C) x.\]
+
+
Parameters:
+

x – Defines \(x\).

+
+
+
+ +
+
+apply_nullspace_transformation_lhs_left(y)
+

Apply the left nullspace transformation associated with a matrix +action,

+
+\[y \rightarrow (I - M U (U^* M U)^{-1} U^*) y.\]
+
+
Parameters:
+

y – Defines \(y\).

+
+
+
+ +
+
+constraint_correct_lhs(x, y)
+

Add the linear constraint term to \(y\),

+
+\[y \rightarrow y + M U S V^* C x.\]
+
+
Parameters:
+
    +

  • x – Defines \(x\).

    • +

  • y – Defines \(y\).

    • +
+
+
+
+ +
+
+pc_constraint_correct_soln(u, b)
+

Add the preconditioner linear constraint term to \(u\),

+
+\[u \rightarrow u + V \tilde{S}^{-1} U^* b,\]
+

with

+
+\[\tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}.\]
+
+
Parameters:
+
    +

  • u – Defines \(u\).

    • +

  • b – Defines \(b\).

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.block_system.UnityNullspace(space, *, alpha=1.0)
+

A nullspace and left nullspace defined by the unity-valued function.

+

Here \(V = U\), \(U\) is a single column matrix containing the +degree-of-freedom vector for the unity-valued function, \(C = M\), +and \(M\) is the mass matrix.

+
+
Parameters:
+
    +

  • space – A scalar-valued function space containing the unity-valued +function.

    • +

  • alpha – Defines the linear constraint matrix \(S = \alpha \left( +U^* M U \right)^{-1}\).

    • +
+
+
+
+
+apply_nullspace_transformation_lhs_right(x)
+

Apply the nullspace transformation associated with a matrix action +on \(x\),

+
+\[x \rightarrow (I - V (V^* C V)^{-1} V^* C) x.\]
+
+
Parameters:
+

x – Defines \(x\).

+
+
+
+ +
+
+apply_nullspace_transformation_lhs_left(y)
+

Apply the left nullspace transformation associated with a matrix +action,

+
+\[y \rightarrow (I - M U (U^* M U)^{-1} U^*) y.\]
+
+
Parameters:
+

y – Defines \(y\).

+
+
+
+ +
+
+constraint_correct_lhs(x, y)
+

Add the linear constraint term to \(y\),

+
+\[y \rightarrow y + M U S V^* C x.\]
+
+
Parameters:
+
    +

  • x – Defines \(x\).

    • +

  • y – Defines \(y\).

    • +
+
+
+
+ +
+
+pc_constraint_correct_soln(u, b)
+

Add the preconditioner linear constraint term to \(u\),

+
+\[u \rightarrow u + V \tilde{S}^{-1} U^* b,\]
+

with

+
+\[\tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}.\]
+
+
Parameters:
+
    +

  • u – Defines \(u\).

    • +

  • b – Defines \(b\).

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.block_system.DirichletBCNullspace(bcs, *, alpha=1.0)
+

A nullspace and left nullspace associated with homogeneous Dirichlet +boundary conditions.

+

Here \(V = U\), \(U\) is a zero-one matrix with exactly one +non-zero per column corresponding to one boundary condition +degree-of-freedom, \(C = M\), and \(M\) is an identity matrix.

+
+
Parameters:
+
+
+
+
+
+apply_nullspace_transformation_lhs_right(x)
+

Apply the nullspace transformation associated with a matrix action +on \(x\),

+
+\[x \rightarrow (I - V (V^* C V)^{-1} V^* C) x.\]
+
+
Parameters:
+

x – Defines \(x\).

+
+
+
+ +
+
+apply_nullspace_transformation_lhs_left(y)
+

Apply the left nullspace transformation associated with a matrix +action,

+
+\[y \rightarrow (I - M U (U^* M U)^{-1} U^*) y.\]
+
+
Parameters:
+

y – Defines \(y\).

+
+
+
+ +
+
+constraint_correct_lhs(x, y)
+

Add the linear constraint term to \(y\),

+
+\[y \rightarrow y + M U S V^* C x.\]
+
+
Parameters:
+
    +

  • x – Defines \(x\).

    • +

  • y – Defines \(y\).

    • +
+
+
+
+ +
+
+pc_constraint_correct_soln(u, b)
+

Add the preconditioner linear constraint term to \(u\),

+
+\[u \rightarrow u + V \tilde{S}^{-1} U^* b,\]
+

with

+
+\[\tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}.\]
+
+
Parameters:
+
    +

  • u – Defines \(u\).

    • +

  • b – Defines \(b\).

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.block_system.BlockNullspace(nullspaces)
+

Nullspaces for a mixed space.

+
+
Parameters:
+

nullspaces – A Nullspace or a Sequence of +Nullspace objects defining the nullspace. None indicates a +NoneNullspace.

+
+
+
+
+apply_nullspace_transformation_lhs_right(x)
+

Apply the nullspace transformation associated with a matrix action +on \(x\),

+
+\[x \rightarrow (I - V (V^* C V)^{-1} V^* C) x.\]
+
+
Parameters:
+

x – Defines \(x\).

+
+
+
+ +
+
+apply_nullspace_transformation_lhs_left(y)
+

Apply the left nullspace transformation associated with a matrix +action,

+
+\[y \rightarrow (I - M U (U^* M U)^{-1} U^*) y.\]
+
+
Parameters:
+

y – Defines \(y\).

+
+
+
+ +
+
+constraint_correct_lhs(x, y)
+

Add the linear constraint term to \(y\),

+
+\[y \rightarrow y + M U S V^* C x.\]
+
+
Parameters:
+
    +

  • x – Defines \(x\).

    • +

  • y – Defines \(y\).

    • +
+
+
+
+ +
+
+pc_constraint_correct_soln(u, b)
+

Add the preconditioner linear constraint term to \(u\),

+
+\[u \rightarrow u + V \tilde{S}^{-1} U^* b,\]
+

with

+
+\[\tilde{S}^{-1} = + \left( V^* C V \right)^{-1} + S^{-1} + \left( U^* M U \right)^{-1}.\]
+
+
Parameters:
+
    +

  • u – Defines \(u\).

    • +

  • b – Defines \(b\).

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.block_system.Matrix(arg_space, action_space)
+

Represents a matrix \(A\) mapping \(V \rightarrow W\).

+
+
Parameters:
+
    +

  • arg_space – Defines the space V.

    • +

  • action_space – Defines the space W.

    • +
+
+
+
+
+property arg_space
+

The space defining \(V\).

+
+ +
+
+property action_space
+

The space defining \(W\).

+
+ +
+
+abstract mult_add(x, y)
+

Add \(A x\) to \(y\).

+
+
Parameters:
+
    +

  • x – Defines \(x\). Should not be modified.

    • +

  • y – Defines \(y\).

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.block_system.PETScMatrix(arg_space, action_space, a)
+

A tlm_adjoint.firedrake.block_system.Matrix associated with a +petsc4py.PETSc.Mat \(A\) mapping \(V \rightarrow W\).

+
+
Parameters:
+
    +

  • arg_space – Defines the space V.

    • +

  • action_space – Defines the space W.

    • +

  • a – The petsc4py.PETSc.Mat.

    • +
+
+
+
+
+mult_add(x, y)
+

Add \(A x\) to \(y\).

+
+
Parameters:
+
    +

  • x – Defines \(x\). Should not be modified.

    • +

  • y – Defines \(y\).

    • +
+
+
+
+ +
+ +
+
+tlm_adjoint.firedrake.block_system.form_matrix(a, *args, **kwargs)
+

Construct a PETScMatrix associated with a given sesquilinear +form.

+
+
Parameters:
+

a – A ufl.Form defining the sesquilinear form.

+
+
Returns:
+

The PETScMatrix.

+
+
+

Remaining arguments are passed to the firedrake.assemble.assemble() +function.

+
+ +
+
+class tlm_adjoint.firedrake.block_system.BlockMatrix(arg_spaces, action_spaces, blocks=None)
+

A matrix \(A\) mapping \(V \rightarrow W\), where \(V\) and +\(W\) are defined by mixed spaces.

+
+
Parameters:
+
    +

  • arg_spaces – Defines the space V.

    • +

  • action_spaces – Defines the space W.

    • +

  • block – A Mapping defining the blocks of the matrix. Items are +((i, j), block) where the block in the i th and j th column is +defined by block. Each block is a +tlm_adjoint.firedrake.block_system.Matrix or +ufl.Form, or None to indicate a zero block.

    • +
+
+
+
+
+mult_add(x, y)
+

Add \(A x\) to \(y\).

+
+
Parameters:
+
    +

  • x – Defines \(x\). Should not be modified.

    • +

  • y – Defines \(y\).

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.block_system.System(arg_spaces, action_spaces, blocks, *, nullspaces=None, comm=None)
+

A linear system

+
+\[A u = b.\]
+
+
Parameters:
+
+
+
+
+
+solve(u, b, *, solver_parameters=None, pc_fn=None, pre_callback=None, post_callback=None, correct_initial_guess=True, correct_solution=True)
+

Solve the linear system.

+
+
Parameters:
+
    +

  • u – Defines the solution \(u\).

    • +

  • b – Defines the right-hand-side \(b\).

    • +

  • solver_parameters –

    A Mapping defining outer Krylov solver +parameters. Parameters (a number of which are based on FEniCS +solver parameters) are:

    +
      +

    • ’linear_solver’: The Krylov solver type, default ‘fgmres’.

      • +

    • ’pc_side’: Overrides the PETSc default preconditioning side.

      • +

    • ’relative_tolerance’: Relative tolerance. Required.

      • +

    • ’absolute_tolerance’: Absolute tolerance. Required.

      • +

    • ’divergence_limit’: Overrides the default divergence limit.

      • +

    • ’maximum_iterations’: Maximum number of iterations. Default +1000.

      • +

    • ’norm_type’: Overrides the default convergence norm definition.

      • +

    • ’nonzero_initial_guess’: Whether to use a non-zero initial +guess, defined by the input u. Default True.

      • +

    • ’gmres_restart’: Overrides the default GMRES restart parameter.

      • +
    +

    • +

  • pc_fn –

    Defines the application of a preconditioner. A callable

    +
    def pc_fn(u, b):
+
    +
    +

    The preconditioner is applied to b, and the result stored in u. +Defaults to an identity.

    +

    • +

  • pre_callback – A callable accepting a single +petsc4py.PETSc.KSP argument. Used for detailed manual +configuration. Called after all other configuration options are +set, but before the petsc4py.PETSc.KSP.setUp() method is +called.

    • +

  • post_callback – A callable accepting a single +petsc4py.PETSc.KSP argument. Called after the +petsc4py.PETSc.KSP.solve() method has been called.

    • +

  • correct_initial_guess – Whether to apply a nullspace correction to +the initial guess.

    • +

  • correct_solution – Whether to apply a nullspace correction to +the solution.

    • +
+
+
Returns:
+

The number of Krylov iterations.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/firedrake/caches/index.html b/autoapi/tlm_adjoint/firedrake/caches/index.html new file mode 100644 index 0000000..2b93b9c --- /dev/null +++ b/autoapi/tlm_adjoint/firedrake/caches/index.html @@ -0,0 +1,238 @@ + + + + + + + tlm_adjoint.firedrake.caches — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.firedrake.caches

+

This module implements finite element assembly and linear solver data +caching.

+
+

Module Contents

+
+
+class tlm_adjoint.firedrake.caches.AssemblyCache
+

A Cache for finite element assembly data.

+
+
+assemble(form, *, bcs=None, form_compiler_parameters=None, linear_solver_parameters=None, replace_map=None)
+

Perform finite element assembly and cache the result, or return a +previously cached result.

+
+
Parameters:
+
    +

  • form – The ufl.Form to assemble.

    • +

  • bcs – Dirichlet boundary conditions.

    • +

  • form_compiler_parameters – Form compiler parameters.

    • +

  • linear_solver_parameters – Linear solver parameters. Required for +assembly parameters which appear in the linear solver parameters.

    • +

  • replace_map – A Mapping defining a map from symbolic +variables to values.

    • +
+
+
Returns:
+

A tuple (value_ref, value), where value is the +result of the finite element assembly, and value_ref is a +CacheRef storing a reference to value.

+
+
    +

  • For an arity zero or arity one form value_ref stores the +assembled value.

    • +

  • For an arity two form value_ref is a tuple (A, b_bc). A +is the assembled matrix, and b_bc is a boundary condition +right-hand-side term which should be added after assembling a +right-hand-side with homogeneous boundary conditions applied. +b_bc may be None to indicate that this term is zero.

    • +
+
+

+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.caches.LinearSolverCache
+

A Cache for linear solver data.

+
+
+linear_solver(form, *, bcs=None, form_compiler_parameters=None, linear_solver_parameters=None, replace_map=None, assembly_cache=None)
+

Construct a linear solver and cache the result, or return a +previously cached result.

+
+
Parameters:
+
    +

  • form – An arity two ufl.Form, defining the matrix.

    • +

  • bcs – Dirichlet boundary conditions.

    • +

  • form_compiler_parameters – Form compiler parameters.

    • +

  • linear_solver_parameters – Linear solver parameters.

    • +

  • replace_map – A Mapping defining a map from symbolic +variables to values.

    • +

  • assembly_cache – AssemblyCache to use for finite element +assembly. Defaults to assembly_cache().

    • +
+
+
Returns:
+

A tuple (value_ref, value). value is a tuple +(solver, A, b_bc), where solver is the linear solver, A is +the assembled matrix, and b_bc is a boundary condition +right-hand-side term which should be added after assembling a +right-hand-side with homogeneous boundary conditions applied. +b_bc may be None to indicate that this term is zero. +value_ref is a CacheRef storing a reference to value.

+
+
+
+ +
+ +
+
+tlm_adjoint.firedrake.caches.assembly_cache()
+
+
Returns:
+

The default AssemblyCache.

+
+
+
+ +
+
+tlm_adjoint.firedrake.caches.set_assembly_cache(assembly_cache)
+

Set the default AssemblyCache.

+
+
Parameters:
+

assembly_cache – The new default AssemblyCache.

+
+
+
+ +
+
+tlm_adjoint.firedrake.caches.linear_solver_cache()
+
+
Returns:
+

The default LinearSolverCache.

+
+
+
+ +
+
+tlm_adjoint.firedrake.caches.set_linear_solver_cache(linear_solver_cache)
+

Set the default LinearSolverCache.

+
+
Parameters:
+

linear_solver_cache – The new default LinearSolverCache.

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/firedrake/equations/index.html b/autoapi/tlm_adjoint/firedrake/equations/index.html new file mode 100644 index 0000000..e5c2e85 --- /dev/null +++ b/autoapi/tlm_adjoint/firedrake/equations/index.html @@ -0,0 +1,746 @@ + + + + + + + tlm_adjoint.firedrake.equations — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.firedrake.equations

+

This module implements finite element calculations. In particular the +EquationSolver class implements the solution of finite element +variational problems.

+
+

Module Contents

+
+
+class tlm_adjoint.firedrake.equations.Assembly(x, rhs, *, form_compiler_parameters=None, match_quadrature=None)
+

Represents assignment to the result of finite element assembly:

+
x = assemble(rhs)
+
+
+

The forward residual \(\mathcal{F}\) is defined so that \(\partial +\mathcal{F} / \partial x\) is the identity.

+
+
Parameters:
+
    +

  • x – A variable defining the forward solution.

    • +

  • rhs – A ufl.Form to assemble. Should have arity 0 or 1, and +should not depend on x.

    • +

  • form_compiler_parameters – Form compiler parameters.

    • +

  • match_quadrature – Whether to set quadrature parameters consistently in +the forward, adjoint, and tangent-linears. Defaults to +parameters[‘tlm_adjoint’][‘Assembly’][‘match_quadrature’].

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.equations.EquationSolver(eq, x, bcs=None, *, J=None, form_compiler_parameters=None, solver_parameters=None, adjoint_solver_parameters=None, tlm_solver_parameters=None, cache_jacobian=None, cache_adjoint_jacobian=None, cache_tlm_jacobian=None, cache_rhs_assembly=None, match_quadrature=None)
+

Represents the solution of a finite element variational problem.

+

Caching is based on the approach described in

+
+
    +

  • J. R. Maddison and P. E. Farrell, ‘Rapid development and adjoining of +transient finite element models’, Computer Methods in Applied +Mechanics and Engineering, 276, 95–121, 2014, doi: +10.1016/j.cma.2014.03.010

    • +
+
+

The arguments eq, x, bcs, J, form_compiler_parameters, and +solver_parameters are based on the interface for the DOLFIN +dolfin.solve function (see e.g. FEniCS 2017.1.0).

+
+
Parameters:
+
    +

  • eq – A ufl.equation.Equation defining the finite element +variational problem.

    • +

  • x – A firedrake.function.Function defining the forward +solution.

    • +

  • bcs – Dirichlet boundary conditions.

    • +

  • J – A ufl.Form defining a Jacobian matrix approximation to use +in a non-linear forward solve.

    • +

  • form_compiler_parameters – Form compiler parameters.

    • +

  • solver_parameters – Linear or non-linear solver parameters.

    • +

  • adjoint_solver_parameters – Linear solver parameters to use in an +adjoint solve.

    • +

  • tlm_solver_parameters – Linear solver parameters to use when solving +tangent-linear problems.

    • +

  • cache_jacobian – Whether to cache the forward Jacobian matrix and +linear solver data. Defaults to +parameters[‘tlm_adjoint’][‘EquationSolver][‘cache_jacobian’]. If +None then caching is autodetected.

    • +

  • cache_adjoint_jacobian – Whether to cache the adjoint Jacobian matrix +and linear solver data. Defaults to cache_jacobian.

    • +

  • cache_tlm_jacobian – Whether to cache the Jacobian matrix and linear +solver data when solving tangent-linear problems. Defaults to +cache_jacobian.

    • +

  • cache_rhs_assembly – Whether to enable right-hand-side caching. If +enabled then right-hand-side terms are divided into terms which are +cached, terms which are converted into matrix multiplication by a +cached matrix, and terms which are not cached. Defaults to +parameters[‘tlm_adjoint’][‘EquationSolver’][‘cache_rhs_assembly’].

    • +

  • match_quadrature – Whether to set quadrature parameters consistently in +the forward, adjoint, and tangent-linears. Defaults to +parameters[‘tlm_adjoint’][‘EquationSolver’][‘match_quadrature’].

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+subtract_adjoint_derivative_actions(adj_x, nl_deps, dep_Bs)
+

Subtract terms from other adjoint right-hand-sides.

+

Can be overridden for an optimized implementation, but otherwise uses +Equation.adjoint_derivative_action().

+
+
Parameters:
+
    +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_Bs – A Mapping whose items are (dep_index, dep_B). +Each dep_B is an AdjointRHS which should be updated by +subtracting adjoint derivative information computed by +differentiating with respect to self.dependencies()[dep_index].

    • +
+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+tlm_adjoint.firedrake.equations.expr_new_x(expr, x, *, annotate=None, tlm=None)
+

If an expression depends on x, then record the assignment x_old = +x, and replace x with x_old in the expression.

+
+
Parameters:
+
    +

  • expr – A ufl.core.expr.Expr.

    • +

  • x – Defines x.

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
Returns:
+

A ufl.core.expr.Expr with x replaced with x_old, or +expr if the expression does not depend on x.

+
+
+
+ +
+
+tlm_adjoint.firedrake.equations.linear_equation_new_x(eq, x, *, annotate=None, tlm=None)
+

If a symbolic expression for a linear finite element variational problem +depends on the symbolic variable representing the problem solution x, +then record the assignment x_old = x, and replace x with x_old in the +symbolic expression.

+
+
Parameters:
+
    +

  • eq – A ufl.equation.Equation defining the finite element +variational problem.

    • +

  • x – A firedrake.function.Function defining the solution to the +finite element variational problem.

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
Returns:
+

A ufl.equation.Equation with x replaced with x_old, +or eq if the symbolic expression does not depend on x.

+
+
+
+ +
+
+class tlm_adjoint.firedrake.equations.Projection(x, rhs, *args, **kwargs)
+

Represents the solution of a finite element variational problem +performing a projection onto the space for x.

+
+
Parameters:
+
    +

  • x – A firedrake.function.Function defining the forward +solution.

    • +

  • rhs – A ufl.core.expr.Expr defining the expression to project +onto the space for x, or a ufl.form.BaseForm defining the +right-hand-side of the finite element variational problem. Should not +depend on x.

    • +
+
+
+

Remaining arguments are passed to the EquationSolver constructor.

+
+ +
+
+class tlm_adjoint.firedrake.equations.DirichletBCApplication(x, y, *args, **kwargs)
+

Represents the application of a Dirichlet boundary condition to a zero +valued firedrake.function.Function. Specifically this represents:

+
x.zero()
+DirichletBC(x.function_space(), y, *args, **kwargs).apply(x)
+
+
+

The forward residual \(\mathcal{F}\) is defined so that \(\partial +\mathcal{F} / \partial x\) is the identity.

+
+
Parameters:
+
+
+
+

Remaining arguments are passed to the firedrake.bcs.DirichletBC +constructor.

+
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.equations.ExprInterpolation(x, rhs)
+

Represents interpolation of rhs onto the space for x.

+

The forward residual \(\mathcal{F}\) is defined so that \(\partial +\mathcal{F} / \partial x\) is the identity.

+
+
Parameters:
+
    +

  • x – The forward solution.

    • +

  • rhs – A ufl.core.expr.Expr defining the expression to +interpolate onto the space for x. Should not depend on x.

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/firedrake/firedrake_equations/index.html b/autoapi/tlm_adjoint/firedrake/firedrake_equations/index.html new file mode 100644 index 0000000..990fb13 --- /dev/null +++ b/autoapi/tlm_adjoint/firedrake/firedrake_equations/index.html @@ -0,0 +1,533 @@ + + + + + + + tlm_adjoint.firedrake.firedrake_equations — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.firedrake.firedrake_equations

+

This module includes additional functionality for use with Firedrake.

+
+

Module Contents

+
+
+class tlm_adjoint.firedrake.firedrake_equations.LocalSolverCache
+

A Cache for element-wise local block diagonal linear solver +data.

+
+
+local_solver(form, *, form_compiler_parameters=None, replace_map=None)
+

Compute data for an element-wise local block diagonal linear +solver and cache the result, or return a previously cached result.

+
+
Parameters:
+
    +

  • form – An arity two ufl.Form, defining the element-wise +local block diagonal matrix.

    • +

  • form_compiler_parameters – Form compiler parameters.

    • +

  • replace_map – A Mapping defining a map from symbolic +variables to values.

    • +
+
+
Returns:
+

A tuple (value_ref, value). value is a +firedrake.matrix.Matrix storing the assembled inverse +matrix, and value_ref is a CacheRef storing a reference +to value.

+
+
+
+ +
+ +
+
+tlm_adjoint.firedrake.firedrake_equations.local_solver_cache()
+
+
Returns:
+

The default LocalSolverCache.

+
+
+
+ +
+
+tlm_adjoint.firedrake.firedrake_equations.set_local_solver_cache(local_solver_cache)
+

Set the default LocalSolverCache.

+
+
Parameters:
+

local_solver_cache – The new default LocalSolverCache.

+
+
+
+ +
+
+class tlm_adjoint.firedrake.firedrake_equations.LocalProjection(x, rhs, *, form_compiler_parameters=None, cache_jacobian=None, cache_rhs_assembly=None, match_quadrature=None)
+

Represents the solution of a finite element variational problem +performing a projection onto the space for x, for the case where the mass +matrix is element-wise local block diagonal.

+
+
Parameters:
+
    +

  • x – A firedrake.function.Function defining the forward +solution.

    • +

  • rhs – A ufl.core.expr.Expr defining the expression to project +onto the space for x, or a ufl.form.BaseForm defining the +right-hand-side of the finite element variational problem. Should not +depend on x.

    • +
+
+
+

Remaining arguments are passed to the +tlm_adjoint.firedrake.equations.EquationSolver constructor.

+
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.firedrake_equations.PointInterpolation(X, y, X_coords=None, *, tolerance=None, _interp=None)
+

Represents interpolation of a scalar-valued function at given points.

+

The forward residual \(\mathcal{F}\) is defined so that \(\partial +\mathcal{F} / \partial x\) is the identity.

+
+
Parameters:
+
    +

  • X – A scalar variable, or a Sequence of scalar variables, +defining the forward solution.

    • +

  • y – A scalar-valued firedrake.function.Function to +interpolate.

    • +

  • X_coords – A numpy.ndarray defining the coordinates at which +to interpolate y. Shape is (n, d) where n is the number of +interpolation points and d is the geometric dimension. Ignored if P +is supplied.

    • +

  • tolerance – firedrake.mesh.VertexOnlyMesh tolerance.

    • +
+
+
+
+
+forward_solve(X, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_X)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_X, nl_deps, B)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.firedrake.firedrake_equations.ExprAssignment(x, rhs, *, subset=None)
+

Represents an evaluation of rhs, storing the result in x. Uses +firedrake.function.Function.assign() or +firedrake.cofunction.Cofunction.assign() to perform the evaluation.

+

The forward residual \(\mathcal{F}\) is defined so that \(\partial +\mathcal{F} / \partial x\) is the identity.

+
+
Parameters:
+
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/firedrake/functions/index.html b/autoapi/tlm_adjoint/firedrake/functions/index.html new file mode 100644 index 0000000..631de94 --- /dev/null +++ b/autoapi/tlm_adjoint/firedrake/functions/index.html @@ -0,0 +1,234 @@ + + + + + + + tlm_adjoint.firedrake.functions — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.firedrake.functions

+

This module includes functionality for interacting with Firedrake variables +and Dirichlet boundary conditions.

+
+

Module Contents

+
+
+class tlm_adjoint.firedrake.functions.Constant(value=None, *args, name=None, domain=None, space=None, space_type='primal', shape=None, comm=None, static=False, cache=None, **kwargs)
+

Extends the firedrake.constant.Constant class.

+
+
Parameters:
+
    +

  • value – The initial value. None indicates a value of zero.

    • +

  • name – A str name.

    • +

  • domain – The domain on which the Constant is defined.

    • +

  • space – The space on which the Constant is defined.

    • +

  • space_type – The space type for the Constant. ‘primal’, +‘dual’, ‘conjugate’, or ‘conjugate_dual’.

    • +

  • shape – A tuple of int objects defining the shape of +the value.

    • +

  • comm – The communicator for the Constant.

    • +

  • static – Defines whether the Constant is static, meaning that +it is stored by reference in checkpointing/replay, and an associated +tangent-linear variable is zero.

    • +

  • cache – Defines whether results involving the Constant may be +cached. Default static.

    • +
+
+
+

Remaining arguments are passed to the firedrake.constant.Constant +constructor.

+
+ +
+
+class tlm_adjoint.firedrake.functions.Zero
+

Mixin for defining a zero-valued variable. Used for zero-valued +variables for which UFL zero elimination should not be applied.

+
+ +
+
+class tlm_adjoint.firedrake.functions.ZeroConstant(*, name=None, domain=None, space=None, space_type='primal', shape=None, comm=None)
+

A Constant which is flagged as having a value of zero.

+

Arguments are passed to the Constant constructor, together with +static=True and cache=True.

+
+ +
+
+tlm_adjoint.firedrake.functions.eliminate_zeros(expr)
+

Apply zero elimination for Zero objects in the supplied +ufl.core.expr.Expr or ufl.form.BaseForm.

+
+
Parameters:
+

expr – A ufl.core.expr.Expr or ufl.form.BaseForm.

+
+
Returns:
+

A ufl.core.expr.Expr or ufl.form.BaseForm with +zero elimination applied. May return expr.

+
+
+
+ +
+
+class tlm_adjoint.firedrake.functions.DirichletBC(V, g, sub_domain, *args, static=None, _homogeneous=False, **kwargs)
+

Extends the firedrake.bcs.DirichletBC class.

+
+
Parameters:
+

static – A flag that indicates that the value for the +DirichletBC will not change, and which determines whether +calculations involving this DirichletBC can be cached. If +None then autodetected from the value.

+
+
+

Remaining arguments are passed to the firedrake.bcs.DirichletBC +constructor.

+
+ +
+
+class tlm_adjoint.firedrake.functions.HomogeneousDirichletBC(V, sub_domain, *args, **kwargs)
+

A DirichletBC whose value is zero.

+

Arguments are passed to the DirichletBC constructor, together +with static=True.

+
+ +
+
+class tlm_adjoint.firedrake.functions.Replacement
+

Represents a symbolic variable but with no value.

+
+ +
+
+class tlm_adjoint.firedrake.functions.ReplacementConstant(x, count)
+

Represents a symbolic firedrake.constant.Constant, but has no +value.

+
+ +
+
+class tlm_adjoint.firedrake.functions.ReplacementFunction(x, count)
+

Represents a symbolic firedrake.function.Function, but has no +value.

+
+ +
+
+class tlm_adjoint.firedrake.functions.ReplacementZeroConstant(*args, **kwargs)
+

Represents a symbolic firedrake.constant.Constant which is +zero, but has no value.

+
+ +
+
+class tlm_adjoint.firedrake.functions.ReplacementZeroFunction(*args, **kwargs)
+

Represents a symbolic firedrake.function.Function which is +zero, but has no value.

+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/firedrake/hessian_system/index.html b/autoapi/tlm_adjoint/firedrake/hessian_system/index.html new file mode 100644 index 0000000..2257d0e --- /dev/null +++ b/autoapi/tlm_adjoint/firedrake/hessian_system/index.html @@ -0,0 +1,301 @@ + + + + + + + tlm_adjoint.firedrake.hessian_system — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.firedrake.hessian_system

+
+

Module Contents

+
+
+class tlm_adjoint.firedrake.hessian_system.HessianSystem(H, M, *, nullspace=None, comm=None)
+

Defines a linear system involving a Hessian matrix,

+
+\[H u = b.\]
+
+
Parameters:
+
+
+
+
+
+solve(u, b, **kwargs)
+

Solve a linear system involving a Hessian matrix,

+
+\[H u = b.\]
+
+
Parameters:
+
+
+
+

Remaining arguments are handed to the base class +System.solve() method.

+
+ +
+ +
+
+tlm_adjoint.firedrake.hessian_system.hessian_eigendecompose(H, m, B_inv_action, B_action, *, nullspace=None, problem_type=None, pre_callback=None, correct_eigenvectors=True, **kwargs)
+

Interface with SLEPc via slepc4py, for the matrix free solution of +generalized eigenproblems

+
+\[H v = \lambda B^{-1} v,\]
+

where \(H\) is a Hessian matrix.

+

Despite the notation \(B^{-1}\) may be singular, defining an inverse +operator only on an appropriate subspace.

+
+
Parameters:
+
+
+
+

Remaining keyword arguments are passed to eigendecompose().

+
+ +
+
+tlm_adjoint.firedrake.hessian_system.B_inv_orthonormality_test(V, B_inv_action)
+

Check for \(B^{-1}\)-orthonormality.

+

Requires real spaces.

+
+
Parameters:
+
+
+
Returns:
+

A tuple (max_diagonal_error_norm, +max_off_diagonal_error_norm) with

+
+
    +

  • max_diagonal_error_norm: The maximum \(B^{-1}\) +normalization error magnitude.

    • +

  • max_diagonal_error_norm: The maximum \(B^{-1}\) +orthogonality error magnitude.

    • +
+
+

+
+
+
+ +
+
+tlm_adjoint.firedrake.hessian_system.hessian_eigendecomposition_pc(B_action, Lam, V)
+

Construct a Hessian matrix preconditioner using a partial spectrum +generalized eigendecomposition. Assumes that the Hessian matrix consists of +two terms

+
+\[H = R^{-1} + B^{-1},\]
+

where \(R\) and \(B\) are symmetric.

+

Assumes real spaces. Despite the notation \(R^{-1}\) and \(B^{-1}\) +(and later \(H^{-1}\)) may be singular, defining inverse operators only +on an appropriate subspace. \(B\) is assumed to define a symmetric +positive definite operator on that subspace.

+

The approximation is defined via

+
+\[H^{-1} \approx B + V \Lambda \left( I + \Lambda \right)^{-1} V^T\]
+

where

+
+\[R^{-1} V = B^{-1} V \Lambda,\]
+

and where \(\Lambda\) is a diagonal matrix and \(V\) has +\(B^{-1}\)-orthonormal columns, \(V^T B^{-1} V = I\).

+

This low rank update approximation for the Hessian matrix inverse is +described in

+
+
    +

  • Tobin Isaac, Noemi Petra, Georg Stadler, and Omar Ghattas, ‘Scalable +and efficient algorithms for the propagation of uncertainty from data +through inference to prediction for large-scale problems, with +application to flow of the Antarctic ice sheet’, Journal of +Computational Physics, 296, pp. 348–368, 2015, doi: +10.1016/j.jcp.2015.04.047

    • +
+
+

See in particular their equation (20).

+
+
Parameters:
+
+
+
Returns:
+

A callable suitable for use as the pc_fn argument to +HessianSystem.solve().

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/fixed_point/index.html b/autoapi/tlm_adjoint/fixed_point/index.html new file mode 100644 index 0000000..c874c58 --- /dev/null +++ b/autoapi/tlm_adjoint/fixed_point/index.html @@ -0,0 +1,304 @@ + + + + + + + tlm_adjoint.fixed_point — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.fixed_point

+
+

Module Contents

+
+
+class tlm_adjoint.fixed_point.CustomNormSq(eqs, *, norm_sqs=None, adj_norm_sqs=None)
+

Defines the square of the norm of forward and adjoint solutions.

+

Callables are used to define squared norms for the forward and adjoint +solutions of equations. The total squared norm is then the sum of the +squares.

+
+
Parameters:
+
    +

  • eqs – A Sequence of Equation objects.

    • +

  • norm_sqs – A Sequence. Each element is either a callable, or a +Sequence of callables. The callables define the squared norm +associated with the corresponding components of the forward solution +for the corresponding Equation in eqs. Each callable +accepts a single variable and returns a float. Defaults to the +square of the \(l_2\) norm of the degrees of freedom vector.

    • +

  • adj_norm_sqs – A Sequence. Each element is either a callable, +or a Sequence of callables. The callables define the squared +norm associated with the corresponding components of the adjoint +solution for the corresponding Equation in eqs. Each +callable accepts a single variable and returns a float. +Defaults to the square of the \(l_2\) norm of the degrees of +freedom vector.

    • +
+
+
+
+ +
+
+class tlm_adjoint.fixed_point.FixedPointSolver(eqs, solver_parameters, *, norm_sqs=None, adj_norm_sqs=None)
+

A fixed-point solver. Solves the given equations in sequence until +either an absolute or relative tolerance is reached.

+

Derives tangent-linear and adjoint information using the approach described +in:

+
+
    +

  • Jean Charles Gilbert, ‘Automatic differentiation and iterative +processes’, Optimization Methods and Software, 1(1), pp. 13–21, +1992, doi: 10.1080/10556789208805503

    • +

  • Bruce Christianson, ‘Reverse accumulation and attractive fixed +points’, Optimization Methods and Software, 3(4), pp. 311–326, 1994, +doi: 10.1080/10556789408805572

    • +
+
+
+
Parameters:
+
    +

  • eqs – A Sequence of Equation objects. One forward +iteration consists of computing, in order, a forward solution for all +Equation objects.

    • +

  • solver_parameters –

    A Mapping defining solver parameters. +Parameters (a number of which are based on KrylovSolver parameters in +FEniCS 2017.2.0) are:

    +
    +
      +

    • absolute_tolerance: A float defining the absolute +tolerance for a change in the solution in one iteration. +Required.

      • +

    • relative_tolerance: A float defining the relative +tolerance for a change in the solution in one iteration. +Required.

      • +

    • maximum_iterations: An int defining the maximum +permitted iterations. Defaults to 1000.

      • +

    • nonzero_initial_guess: A bool indicating whether to use +a non-zero initial guess in a forward solve. Defaults to True.

      • +

    • adjoint_nonzero_initial_guess: A bool indicating whether +to use a non-zero initial guess in an adjoint solve. Defaults to +True.

      • +
    +
    +

    • +

  • norm_sqs – Defines the squared norm used to test for convergence in a +forward solve. See CustomNormSq.

    • +

  • adj_norm_sqs – Defines the squared norm used to test for convergence in +an adjoint solve. See CustomNormSq.

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+forward_solve(X, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_jacobian_solve(adj_X, nl_deps, B)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+subtract_adjoint_derivative_actions(adj_X, nl_deps, dep_Bs)
+

Subtract terms from other adjoint right-hand-sides.

+

Can be overridden for an optimized implementation, but otherwise uses +Equation.adjoint_derivative_action().

+
+
Parameters:
+
    +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_Bs – A Mapping whose items are (dep_index, dep_B). +Each dep_B is an AdjointRHS which should be updated by +subtracting adjoint derivative information computed by +differentiating with respect to self.dependencies()[dep_index].

    • +
+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/functional/index.html b/autoapi/tlm_adjoint/functional/index.html new file mode 100644 index 0000000..6042845 --- /dev/null +++ b/autoapi/tlm_adjoint/functional/index.html @@ -0,0 +1,157 @@ + + + + + + + tlm_adjoint.functional — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.functional

+
+

Module Contents

+
+
+class tlm_adjoint.functional.Functional(*args, space=None, **kwargs)
+

A convenience class for defining functionals.

+

Arguments are as for the Float class.

+
+
+assign(y, *, annotate=None, tlm=None)
+

Assign to the Functional.

+
+
Parameters:
+
    +

  • y – The value.

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
Returns:
+

The Functional.

+
+
+
+ +
+
+addto(y, *, annotate=None, tlm=None)
+

Add to the Functional.

+
+
Parameters:
+
    +

  • y – The value to add.

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/hessian/index.html b/autoapi/tlm_adjoint/hessian/index.html new file mode 100644 index 0000000..4c8ffab --- /dev/null +++ b/autoapi/tlm_adjoint/hessian/index.html @@ -0,0 +1,366 @@ + + + + + + + tlm_adjoint.hessian — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.hessian

+
+

Module Contents

+
+
+class tlm_adjoint.hessian.Hessian
+

Represents a Hessian associated with a given forward. Abstract base +class.

+
+
+abstract compute_gradient(M, M0=None)
+

Compute the (conjugate of the) derivative of a functional with +respect to a control using an adjoint.

+
+
Parameters:
+
    +

  • M – A variable or a Sequence of variables defining the +control.

    • +

  • M0 – A variable or a Sequence of variables defining the +control value. M is used if not supplied.

    • +
+
+
Returns:
+

The (conjugate of the) derivative. A variable or +Sequence of variables, depending on the type of M.

+
+
+
+ +
+
+abstract action(M, dM, M0=None)
+

Compute (the conjugate of) a Hessian action on some \(\zeta\) +using an adjoint of a tangent-linear. i.e. considering derivatives to +be row vectors, compute

+
+\[\left( \frac{d}{dm} \left[ + \frac{d \mathcal{J}}{d m} \zeta \right] \right)^{*,T}.\]
+
+
Parameters:
+
    +

  • M – A variable or a Sequence of variables defining the +control.

    • +

  • dM – A variable or a Sequence of variables defining +\(\zeta\). The (conjugate of the) Hessian action on +\(\zeta\) is computed.

    • +

  • M0 – A variable or a Sequence of variables defining the +control value. M is used if not supplied.

    • +
+
+
Returns:
+

A tuple (J, dJ, ddJ). J is the value of the functional. +dJ is the value of \(\left( d \mathcal{J} / d m \right) +\zeta\). ddJ stores the (conjugate of the) result of the Hessian +action on \(\zeta\), and is a variable or a Sequence +of variables depending on the type of M.

+
+
+
+ +
+
+action_fn(m, m0=None)
+

Return a callable which can be used to compute Hessian actions.

+
+
Parameters:
+
    +

  • m – A variable defining the control.

    • +

  • m0 – A variable defining the control value. m is used if not +supplied.

    • +
+
+
Returns:
+

A callable which accepts a single variable argument, and +returns the result of the Hessian action on that argument as a +variable. Note that the result is not the conjugate of the +Hessian action on the input argument.

+
+
+
+ +
+ +
+
+class tlm_adjoint.hessian.GeneralHessian(forward, *, manager=None)
+

Represents a Hessian associated with a given forward. Calls to +GeneralHessian.compute_gradient() or GeneralHessian.action() +re-run the forward.

+
+
Parameters:
+
    +

  • forward – A callable which accepts one or more variable arguments, and +which returns a variable defining the forward functional.

    • +

  • manager – An EquationManager used to create an internal +manager via EquationManager.new(). manager() is used if not +supplied.

    • +
+
+
+
+
+compute_gradient(M, M0=None)
+

Compute the (conjugate of the) derivative of a functional with +respect to a control using an adjoint.

+
+
Parameters:
+
    +

  • M – A variable or a Sequence of variables defining the +control.

    • +

  • M0 – A variable or a Sequence of variables defining the +control value. M is used if not supplied.

    • +
+
+
Returns:
+

The (conjugate of the) derivative. A variable or +Sequence of variables, depending on the type of M.

+
+
+
+ +
+
+action(M, dM, M0=None)
+

Compute (the conjugate of) a Hessian action on some \(\zeta\) +using an adjoint of a tangent-linear. i.e. considering derivatives to +be row vectors, compute

+
+\[\left( \frac{d}{dm} \left[ + \frac{d \mathcal{J}}{d m} \zeta \right] \right)^{*,T}.\]
+
+
Parameters:
+
    +

  • M – A variable or a Sequence of variables defining the +control.

    • +

  • dM – A variable or a Sequence of variables defining +\(\zeta\). The (conjugate of the) Hessian action on +\(\zeta\) is computed.

    • +

  • M0 – A variable or a Sequence of variables defining the +control value. M is used if not supplied.

    • +
+
+
Returns:
+

A tuple (J, dJ, ddJ). J is the value of the functional. +dJ is the value of \(\left( d \mathcal{J} / d m \right) +\zeta\). ddJ stores the (conjugate of the) result of the Hessian +action on \(\zeta\), and is a variable or a Sequence +of variables depending on the type of M.

+
+
+
+ +
+ +
+
+class tlm_adjoint.hessian.GaussNewton(R_inv_action, B_inv_action=None)
+

Represents a Gauss-Newton approximation for a Hessian. Abstract base +class.

+

In terms of matrices this defines a Hessian approximation

+
+\[H = J^T R_\text{obs}^{-1} J + B^{-1},\]
+

where \(J\) is the forward Jacobian. In a variational assimilation +approach \(R_\text{obs}^{-1}\) corresponds to the observational inverse +covariance and \(B^{-1}\) corresponds to the background inverse +covariance.

+
+
Parameters:
+
    +

  • R_inv_action – A callable which accepts one or more variables, and +returns the conjugate of the action of the operator corresponding to +\(R_\text{obs}^{-1}\) on those variables, returning the result as a +variable or a Sequence of variables.

    • +

  • B_inv_action – A callable which accepts one or more variables, and +returns the conjugate of the action of the operator corresponding to +\(B^{-1}\) on those variables, returning the result as a variable +or a Sequence of variables.

    • +
+
+
+
+
+action(M, dM, M0=None)
+

Compute (the conjugate of) a Hessian action on some \(\zeta\), +using the Gauss-Newton approximation for the Hessian. i.e. compute

+
+\[\left( H \zeta \right)^{*,T}.\]
+
+
Parameters:
+
    +

  • M – A variable or a Sequence of variables defining the +control.

    • +

  • dM – A variable or a Sequence of variables defining +\(\zeta\). The (conjugate of the) approximated Hessian action +on \(\zeta\) is computed.

    • +

  • M0 – A variable or a Sequence of variables defining the +control value. M is used if not supplied.

    • +
+
+
Returns:
+

The (conjugate of the) result of the approximated Hessian +action on \(\zeta\). A variable or a Sequence of +variables depending on the type of M.

+
+
+
+ +
+
+action_fn(m, m0=None)
+

Return a callable which can be used to compute Hessian actions using +the Gauss-Newton approximation.

+
+
Parameters:
+
    +

  • m – A variable defining the control.

    • +

  • m0 – A variable defining the control value. m is used if not +supplied.

    • +
+
+
Returns:
+

A callable which accepts a single variable argument, and +returns the result of the approximated Hessian action on that +argument as a variable. Note that the result is not the conjugate +of the approximated Hessian action on the input argument.

+
+
+
+ +
+ +
+
+class tlm_adjoint.hessian.GeneralGaussNewton(forward, R_inv_action, B_inv_action=None, *, manager=None)
+

Represents a Gauss-Newton approximation to a Hessian associated with a +given forward. Calls to GaussNewton.action() re-run the forward.

+
+
Parameters:
+
    +

  • forward – A callable which accepts one or more variable arguments, and +which returns a variable or Sequence of variables defining the +state.

    • +

  • R_inv_action – See GaussNewton.

    • +

  • B_inv_action – See GaussNewton.

    • +

  • manager – An EquationManager used to create an internal +manager via EquationManager.new(). manager() is used if not +supplied.

    • +
+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/index.html b/autoapi/tlm_adjoint/index.html new file mode 100644 index 0000000..e4146a5 --- /dev/null +++ b/autoapi/tlm_adjoint/index.html @@ -0,0 +1,123 @@ + + + + + + + tlm_adjoint — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+ +
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/instructions/index.html b/autoapi/tlm_adjoint/instructions/index.html new file mode 100644 index 0000000..f7a560a --- /dev/null +++ b/autoapi/tlm_adjoint/instructions/index.html @@ -0,0 +1,243 @@ + + + + + + + tlm_adjoint.instructions — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.instructions

+
+

Module Contents

+
+
+class tlm_adjoint.instructions.Instruction
+

An adjoint tape record which defines instructions to be performed during +forward or adjoint calculations.

+
+
+adjoint_jacobian_solve(adj_X, nl_deps, B)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+ +
+
+class tlm_adjoint.instructions.GarbageCollection(comm=None, *, generation=2, garbage_cleanup=True)
+

An Instruction which indicates that garbage collection should +be performed during forward and adjoint calculations.

+
+
Parameters:
+
    +

  • comm – Communicator to use for PETSc garbage cleanup.

    • +

  • generation – Python garbage collection generation. If a value of None +is provided then Python garbage collection is not performed.

    • +

  • garbage_cleanup – Whether to perform PETSc garbage cleanup.

    • +
+
+
+
+
+forward_solve(X, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_jacobian_solve(adj_X, nl_deps, B)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/interface/index.html b/autoapi/tlm_adjoint/interface/index.html new file mode 100644 index 0000000..deb98c7 --- /dev/null +++ b/autoapi/tlm_adjoint/interface/index.html @@ -0,0 +1,1035 @@ + + + + + + + tlm_adjoint.interface — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.interface

+

This module defines an interface for interaction with backend data types. +This is implemented via runtime binding of mixins. The +VariableInterface adds methods to ‘variables’ which can be used to +interact with backend variables. The SpaceInterface adds methods to +‘spaces’ which define the vector spaces in which those ‘variables’ are defined.

+

The extra methods are accessed using the callables defined in this module +(which also handle some extra details, e.g. related to cache invalidation and +space type checking). Typically these are prefixed with space_ for spaces and +var_ for variables.

+

The interface distinguishes between original backend ‘variables’, which both +define symbolic variables and store values, and replacement ‘variables’, which +define the same variables but which need not store values.

+

Variables have an associated ‘space type’, which indicates e.g. if the variable +is ‘primal’, meaning a member on an originating vector space, or ‘conjugate +dual’, meaning a member of the corresponding antidual space of antilinear +functionals from the originating vector space. Variables can also be ‘dual’, +meaning a member of the dual space of linear functionals, or ‘conjugate’, +meaning a member of a space defined by a conjugate operator from the primal +space. This conjugate operator is defined by complex conjugation of the vector +of degrees of freedom, and could e.g. correspond to complex conjugation of a +finite element discretized function.

+

The space type associated with a variable is defined relative to an originating +vector space (e.g. a finite element discrete function space). A ‘relative space +type’ is defined relative to one of the ‘primal’, ‘conjugate’, ‘dual’, or +‘conjugate dual’ spaces. For example the primal space associated with the dual +space is the dual space, and the dual space associated with the dual space is +the primal space.

+

This module defines a default communicator DEFAULT_COMM, which is +mpi4py.MPI.COMM_WORLD if mpi4py is available. If mpi4py is not available a +dummy ‘serial’ communicator is used, of type SerialComm.

+
+

Module Contents

+
+
+tlm_adjoint.interface.comm_dup_cached(comm, *, key=None)
+

If the communicator comm with key key has previously been duplicated +using comm_dup_cached(), then return the previous result. Otherwise +duplicate the communicator and cache the result. The duplicated +communicator is freed when the original base communicator is freed.

+
+
Parameters:
+
    +

  • comm – A communicator, the base communicator to be duplicated.

    • +

  • key – The key.

    • +
+
+
Returns:
+

A communicator. A duplicated MPI communicator, or a previously +cached duplicated MPI communicator, which is freed when the original +base communicator is freed.

+
+
+
+ +
+
+tlm_adjoint.interface.garbage_cleanup(comm=None)
+

Call petsc4py.PETSc.garbage_cleanup(comm) for a communicator, any +communicators duplicated from it, base communicators from which it was +duplicated, and any communicators duplicated from those base communicators.

+
+
Parameters:
+

comm – A communicator. Defaults to DEFAULT_COMM.

+
+
+
+ +
+
+tlm_adjoint.interface.add_interface(obj, interface_cls, attrs=None)
+

Attach a mixin interface_cls, defining an interface, to obj.

+
+
Parameters:
+
    +

  • obj – An object to which the mixin should be attached.

    • +

  • interface_cls – A subclass of SpaceInterface or +VariableInterface defining the interface.

    • +

  • attrs – A Mapping defining any attributes. Used to set an +attribute _tlm_adjoint__space_interface_attrs (for a +SpaceInterface) or _tlm_adjoint__var_interface_attrs +(for a VariableInterface).

    • +
+
+
+
+ +
+
+class tlm_adjoint.interface.SpaceInterface
+

A mixin defining an interface for spaces. Space types do not inherit +from this class – instead an interface is defined by a +SpaceInterface subclass, and methods are bound dynamically at +runtime using add_interface().

+
+ +
+
+tlm_adjoint.interface.is_space(space)
+

Return whether space is a space – i.e. has had a +SpaceInterface attached.

+
+
Parameters:
+

space – An arbitrary object.

+
+
Returns:
+

True if space is a space, and False otherwise.

+
+
+
+ +
+
+tlm_adjoint.interface.space_comm(space)
+
+
Parameters:
+

space – A space.

+
+
Returns:
+

The communicator associated with the space.

+
+
+
+ +
+
+tlm_adjoint.interface.space_dtype(space)
+
+
Parameters:
+

space – A space.

+
+
Returns:
+

The data type associated with the space. Typically +numpy.double or numpy.cdouble.

+
+
+
+ +
+
+tlm_adjoint.interface.space_id(space)
+

Return a unique int ID associated with a space.

+
+
Parameters:
+

space – The space.

+
+
Returns:
+

The unique int ID.

+
+
+
+ +
+
+tlm_adjoint.interface.space_new(space, *, name=None, space_type='primal', static=False, cache=None)
+

Return a new variable.

+
+
Parameters:
+
    +

  • space – The space.

    • +

  • name – A str name for the variable.

    • +

  • space_type – The space type for the new variable. ‘primal’, ‘dual’, +‘conjugate’, or ‘conjugate_dual’.

    • +

  • static – Defines whether the new variable is static, meaning that it is +stored by reference in checkpointing/replay, and an associated +tangent-linear variable is zero.

    • +

  • cache – Defines whether results involving the new variable may be +cached. Default static.

    • +
+
+
Returns:
+

The new variable.

+
+
+
+ +
+
+tlm_adjoint.interface.relative_space_type(space_type, rel_space_type)
+

Return a relative space type. For example if space_type is ‘dual’ +and rel_space_type is ‘conjugate_dual’, this returns ‘conjugate’.

+
+
Parameters:
+
    +

  • space_type – An input space type. One of ‘primal’, ‘conjugate’, +‘dual’, or ‘conjugate_dual’.

    • +

  • rel_space_type – The relative space type to return. One of ‘primal’, +‘conjugate’, ‘dual’, or ‘conjugate_dual’.

    • +
+
+
Returns:
+

A space type relative to space_type.

+
+
+
+ +
+
+tlm_adjoint.interface.conjugate_space_type(space_type)
+

Defines a map

+
+
    +

  • ‘primal’ \(\rightarrow\) ‘conjugate’

    • +

  • ‘conjugate’ \(\rightarrow\) ‘primal’

    • +

  • ‘dual’ \(\rightarrow\) ‘conjugate_dual’

    • +

  • ‘conjugate_dual’ \(\rightarrow\) ‘dual’

    • +
+
+
+
Returns:
+

The space type conjugate to space_type.

+
+
+
+ +
+
+tlm_adjoint.interface.dual_space_type(space_type)
+

Defines a map

+
+
    +

  • ‘primal’ \(\rightarrow\) ‘dual’

    • +

  • ‘conjugate’ \(\rightarrow\) ‘conjugate_dual’

    • +

  • ‘dual’ \(\rightarrow\) ‘primal’

    • +

  • ‘conjugate_dual’ \(\rightarrow\) ‘conjugate’

    • +
+
+
+
Returns:
+

The space type dual to space_type.

+
+
+
+ +
+
+tlm_adjoint.interface.conjugate_dual_space_type(space_type)
+

Defines a map

+
+
    +

  • ‘primal’ \(\rightarrow\) ‘conjugate_dual’

    • +

  • ‘conjugate’ \(\rightarrow\) ‘dual’

    • +

  • ‘dual’ \(\rightarrow\) ‘conjugate’

    • +

  • ‘conjugate_dual’ \(\rightarrow\) ‘primal’

    • +
+
+
+
Returns:
+

The space type conjugate dual to space_type.

+
+
+
+ +
+
+tlm_adjoint.interface.no_space_type_checking(fn)
+

Decorator to disable space type checking.

+
+
Parameters:
+

fn – A callable for which space type checking should be disabled.

+
+
Returns:
+

A callable for which space type checking is disabled.

+
+
+
+ +
+
+tlm_adjoint.interface.paused_space_type_checking()
+

Construct a context manager which can be used to temporarily disable +space type checking.

+
+
Returns:
+

A context manager which can be used to temporarily disable +space type checking.

+
+
+
+ +
+
+exception tlm_adjoint.interface.SpaceTypeError
+

Raised when an unexpected space type is encountered with space type +checking enabled.

+
+ +
+
+tlm_adjoint.interface.check_space_type(x, space_type)
+

Check that a variable has a given space type.

+

Raises a SpaceTypeError if the check fails and space type +checking is enabled.

+
+
Parameters:
+
    +

  • x – A variable, whose space type should be checked.

    • +

  • space_type – The space type. One of ‘primal’, ‘conjugate’, +‘dual’, or ‘conjugate_dual’.

    • +
+
+
+
+ +
+
+tlm_adjoint.interface.check_space_types(x, y, *, rel_space_type='primal')
+

Check that x and y have compatible space types.

+

Raises a SpaceTypeError if the check fails and space type +checking is enabled.

+
+
Parameters:
+
    +

  • x – A variable.

    • +

  • y – A variable.

    • +

  • rel_space_type – Check that the space type of x is rel_space_type +relative to y. For example if rel_space_type=’dual’, and the +space type of y is ‘conjuguate_dual’, checks that the space type of +x is ‘conjugate’.

    • +
+
+
+
+ +
+
+tlm_adjoint.interface.check_space_types_conjugate(x, y)
+

Check that x has space type conjugate to the space type for y.

+

Raises a SpaceTypeError if the check fails and space type +checking is enabled.

+
+
Parameters:
+
    +

  • x – A variable.

    • +

  • y – A variable.

    • +
+
+
+
+ +
+
+tlm_adjoint.interface.check_space_types_dual(x, y)
+

Check that x has space type dual to the space type for y.

+

Raises a SpaceTypeError if the check fails and space type +checking is enabled.

+
+
Parameters:
+
    +

  • x – A variable.

    • +

  • y – A variable.

    • +
+
+
+
+ +
+
+tlm_adjoint.interface.check_space_types_conjugate_dual(x, y)
+

Check that x has space type conjugate dual to the space type for y.

+

Raises a SpaceTypeError if the check fails and space type +checking is enabled.

+
+
Parameters:
+
    +

  • x – A variable.

    • +

  • y – A variable.

    • +
+
+
+
+ +
+
+class tlm_adjoint.interface.VariableInterface
+

A mixin defining an interface for variables. Variables types do not +inherit from this class – instead an interface is defined by a +VariableInterface subclass, and methods are bound dynamically at +runtime using add_interface().

+
+ +
+
+tlm_adjoint.interface.is_var(x)
+

Return whether x is a variable – i.e. has had a +VariableInterface added.

+
+
Parameters:
+

x – An arbitrary object.

+
+
Returns:
+

True if x is a variable, and False otherwise.

+
+
+
+ +
+
+tlm_adjoint.interface.var_comm(x)
+
+
Parameters:
+

x – A variable.

+
+
Returns:
+

The communicator associated with the variable.

+
+
+
+ +
+
+tlm_adjoint.interface.var_space(x)
+
+
Parameters:
+

x – A variable.

+
+
Returns:
+

The space associated with the variable.

+
+
+
+ +
+
+tlm_adjoint.interface.var_space_type(x, *, rel_space_type='primal')
+

Return the space type of a variable.

+
+
Parameters:
+
    +

  • x – The variable.

    • +

  • rel_space_type – If supplied then return a space type relative to the +variable space type. One of ‘primal’, ‘conjugate’, ‘dual’, or +‘conjugate_dual’.

    • +
+
+
Returns:
+

The space type.

+
+
+
+ +
+
+tlm_adjoint.interface.var_dtype(x)
+
+
Parameters:
+

x – A variable.

+
+
Returns:
+

The data type associated with the variable. Typically +numpy.double or numpy.cdouble.

+
+
+
+ +
+
+tlm_adjoint.interface.var_id(x)
+

Return a unique int ID associated with a variable.

+

Note that two variables share the same ID if they represent the same +symbolic variable – for example if one variable represents both a variable +and stores a value, and a second the same variable with no value (i.e. is a +‘replacement’), then the two variables share the same ID.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

The int ID.

+
+
+
+ +
+
+tlm_adjoint.interface.var_name(x)
+
+
Parameters:
+

x – A variable.

+
+
Returns:
+

The str name of the variable.

+
+
+
+ +
+
+tlm_adjoint.interface.var_state(x)
+

Return the value of the state counter for a variable. Updated when the +value of the variable changes.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

The int state value.

+
+
+
+ +
+
+tlm_adjoint.interface.var_lock_state(x)
+

Lock the state of a variable.

+
+
Parameters:
+

x – The variable.

+
+
+
+ +
+
+class tlm_adjoint.interface.VariableStateLockDictionary(*args, **kwargs)
+

A dictionary-like class. If a value is a variable and not a replacement +then the variable state is ‘locked’ so that a state update, with the lock +active, will raise an exception.

+

State locks are automatically released when the +VariableStateLockDictionary is destroyed. Consequently objects of +this type should be used with caution. In particular object destruction via +the garbage collector may lead to non-deterministic release of the state +lock.

+
+ +
+
+tlm_adjoint.interface.var_update_state(*X)
+

Ensure that variable state is updated, and check for cache invalidation.

+

May delegate updating of the state to a backend library, in which case this +function checks that the state has been updated since the last +var_update_state() call (or since instantiation on the first call), +and updates the backend state again only if needed.

+
+
Parameters:
+

X – A tuple of variables.

+
+
+
+ +
+
+tlm_adjoint.interface.var_is_static(x)
+

Return whether a variable is flagged as ‘static’. A static variable +is stored by reference in checkpointing/replay, and the associated +tangent-linear variable is zero.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

Whether the variable is flagged as static.

+
+
+
+ +
+
+tlm_adjoint.interface.var_is_cached(x)
+

Return whether results involving this variable may be cached.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

Whether results involving the variable may be cached.

+
+
+
+ +
+
+tlm_adjoint.interface.var_caches(x)
+

Return the Caches associated with a variable.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

The Caches associated with the variable.

+
+
+
+ +
+
+tlm_adjoint.interface.var_update_caches(*X, value=None)
+

Check for cache invalidation associated with a possible change in value.

+
+
Parameters:
+
    +

  • X – A tuple of variables whose value may have changed.

    • +

  • value – A variable or a Sequence of variables defining the +possible new values. X is used if not supplied.

    • +
+
+
+
+ +
+
+tlm_adjoint.interface.var_zero(x)
+

Zero a variable.

+
+
Parameters:
+

x – The variable.

+
+
+
+ +
+
+tlm_adjoint.interface.var_assign(x, y)
+

Perform an assignment x = y.

+
+
Parameters:
+
    +

  • x – A variable.

    • +

  • y – A variable.

    • +
+
+
+
+ +
+
+tlm_adjoint.interface.var_axpy(y, alpha, x, /)
+

Perform an in-place addition y += alpha * x.

+
+
Parameters:
+
    +

  • y – A variable.

    • +

  • alpha – A scalar.

    • +

  • x – A variable.

    • +
+
+
+
+ +
+
+tlm_adjoint.interface.var_inner(x, y)
+

Compute the \(l_2\) inner product of the degrees of freedom vectors +associated with x and y. By convention if y is in the conjugate dual +space associated with x, this returns the complex conjugate of the +functional associated with y evaluated at x.

+
+
Parameters:
+
    +

  • x – A variable.

    • +

  • y – A variable.

    • +
+
+
Returns:
+

The result of the inner product.

+
+
+
+ +
+
+tlm_adjoint.interface.var_linf_norm(x)
+

Compute the \(l_\infty\) norm of the degrees of freedom vector +associated with a variable.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

The \(l_\infty\) norm of the degrees of freedom vector.

+
+
+
+ +
+
+tlm_adjoint.interface.var_local_size(x)
+

Return the process local number of degrees of freedom associated with +a variable. This is the number of ‘owned’ degrees of freedom.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

The process local number of degrees of freedom for the variable.

+
+
+
+ +
+
+tlm_adjoint.interface.var_global_size(x)
+

Return the global number of degrees of freedom associated with a +variable. This is the total number of ‘owned’ degrees of freedom, summed +across all processes.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

The global number of degrees of freedom for the variable.

+
+
+
+ +
+
+tlm_adjoint.interface.var_local_indices(x)
+

Return the indices of process local degrees of freedom associated with +a variable.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

An Iterable, yielding the indices of the process local +elements.

+
+
+
+ +
+
+tlm_adjoint.interface.var_get_values(x)
+

Return a copy of the process local degrees of freedom vector associated +with a variable.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

A numpy.ndarray containing a copy of the degrees of +freedom.

+
+
+
+ +
+
+tlm_adjoint.interface.var_set_values(x, values)
+

Set the process local degrees of freedom vector associated with a +variable.

+
+
Parameters:
+
    +

  • x – The variable.

    • +

  • values – A numpy.ndarray containing the degrees of freedom +values.

    • +
+
+
+
+ +
+
+tlm_adjoint.interface.var_new(x, *, name=None, static=False, cache=None, checkpoint=None, rel_space_type='primal')
+

Return a new variable defined using the same space as x.

+
+
Parameters:
+
    +

  • x – A variable.

    • +

  • name – A str name for the new variable.

    • +

  • static – Defines whether the new variable is static, meaning that it is +stored by reference in checkpointing/replay, and an associated +tangent-linear variable is zero.

    • +

  • cache – Defines whether results involving the new variable may be +cached. Default static.

    • +

  • checkpoint – Deprecated.

    • +

  • rel_space_type – Defines the space type of the new variable, relative +to the space type of x.

    • +
+
+
Returns:
+

The new variable.

+
+
+
+ +
+
+tlm_adjoint.interface.var_new_conjugate(x, *, name=None, static=False, cache=None)
+

Return a new conjugate variable. See var_new().

+
+
Returns:
+

A new variable defined using the same space as x, with space +type conjugate to the space type for x.

+
+
+
+ +
+
+tlm_adjoint.interface.var_new_dual(x, *, name=None, static=False, cache=None)
+

Return a new dual variable. See var_new().

+
+
Returns:
+

A new variable defined using the same space as x, with space +type dual to the space type for x.

+
+
+
+ +
+
+tlm_adjoint.interface.var_new_conjugate_dual(x, *, name=None, static=False, cache=None)
+

Return a new conjugate dual variable. See var_new().

+
+
Returns:
+

A new variable defined using the same space as x, with space +type conjugate dual to the space type for x.

+
+
+
+ +
+
+tlm_adjoint.interface.var_copy(x, *, name=None, static=False, cache=None)
+

Copy a variable. See var_new().

+
+
Returns:
+

The copied variable.

+
+
+
+ +
+
+tlm_adjoint.interface.var_replacement(x)
+

Return a variable, associated with the same variable as x, but +possibly without a value.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

A variable which symbolically represents the same variable as +x, but which may not store a value. May return x itself.

+
+
+
+ +
+
+tlm_adjoint.interface.var_is_replacement(x)
+

Return whether a variable is a ‘replacement’, meaning that it has no +associated value.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

Whether x is a replacement.

+
+
+
+ +
+
+tlm_adjoint.interface.var_is_scalar(x)
+

Return whether a variable defines a scalar variable.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

Whether x defines a scalar variable.

+
+
+
+ +
+
+tlm_adjoint.interface.var_scalar_value(x)
+

If x defines a scalar variable, returns its value.

+
+
Parameters:
+

x – The variable, defining a scalar variable.

+
+
Returns:
+

The scalar value.

+
+
+
+ +
+
+tlm_adjoint.interface.var_is_alias(x)
+

Return whether a variable is an ‘alias’, meaning part or all of the +degree of freedom vector associated with the variable is shared with some +different aliased variable. A variable may not appear as an equation +dependency if it is an alias.

+
+
Parameters:
+

x – The variable.

+
+
Returns:
+

Whether the variable is an alias.

+
+
+
+ +
+
+tlm_adjoint.interface.subtract_adjoint_derivative_action(x, y)
+

Subtract an adjoint right-hand-side contribution defined by y from +the right-hand-side defined by x.

+
+
Parameters:
+
    +

  • x – A variable storing the adjoint right-hand-side.

    • +

  • y – A contribution to subtract from the adjoint right-hand-side. An +Equation.adjoint_derivative_action() return value. Valid types +depend upon the variable type. Typically this will be a variable, or a +two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value to subtract +defined by the product of alpha and F.

    • +
+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/jax/index.html b/autoapi/tlm_adjoint/jax/index.html new file mode 100644 index 0000000..456456a --- /dev/null +++ b/autoapi/tlm_adjoint/jax/index.html @@ -0,0 +1,516 @@ + + + + + + + tlm_adjoint.jax — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.jax

+
+

Module Contents

+
+
+tlm_adjoint.jax.set_default_jax_dtype(dtype)
+

Set the default data type used by Vector objects.

+
+
Parameters:
+

dtype – The default data type.

+
+
+
+ +
+
+class tlm_adjoint.jax.VectorSpace(n, *, dtype=None, comm=None)
+

A vector space.

+
+
Parameters:
+
    +

  • dtype – The data type associated with the space. Typically +numpy.double or numpy.cdouble.

    • +

  • comm – The communicator associated with the space.

    • +
+
+
+
+
+property dtype
+

The data type associated with the space.

+
+ +
+
+property comm
+

The communicator associated with the space.

+
+ +
+
+property local_size
+

The number of local degrees of freedom.

+
+ +
+
+property global_size
+

The global number of degrees of freedom.

+
+ +
+
+property ownership_range
+

A tuple (n0, n1), indicating that slice(n0, n1) is the range of +nodes in the global vector owned by this process.

+
+ +
+
+rdtype()
+

The real data type associated with the space.

+
+ +
+ +
+
+class tlm_adjoint.jax.Vector(V, *, name=None, space_type='primal', static=False, cache=None, dtype=None, comm=None)
+

Vector, with degrees of freedom stored as a JAX array.

+
+
Parameters:
+
    +

  • V – A VectorSpace, an int defining the number of +local degrees of freedom, or an ndim 1 array defining the local degrees +of freedom.

    • +

  • name – A str name for the Vector.

    • +

  • space_type – The space type for the Vector. ‘primal’, +‘dual’, ‘conjugate’, or ‘conjugate_dual’.

    • +

  • static – Defines whether the Vector is static, meaning that +it is stored by reference in checkpointing/replay, and an associated +tangent-linear variable is zero.

    • +

  • cache – Defines whether results involving the Vector may be +cached. Default static.

    • +

  • dtype – The data type. Ignored if V is a VectorSpace.

    • +

  • comm – A communicator. Ignored if V is a VectorSpace.

    • +
+
+
+
+
+property name
+

The str name of the Vector.

+
+ +
+
+property value
+

For a Vector with one element, the value of the element.

+

The value may also be accessed by casting using float or +complex.

+
+
Returns:
+

The value.

+
+
+
+ +
+
+property space
+

The VectorSpace for the Vector.

+
+ +
+
+property space_type
+

The space type for the Vector.

+
+ +
+
+property vector
+

A JAX array storing the local degrees of freedom.

+
+ +
+
+new(y=None, *, name=None, static=False, cache=None)
+

Return a new Vector, with the same VectorSpace +and space type as this Vector.

+
+
Parameters:
+

y – Defines a value for the new Vector.

+
+
Returns:
+

The new Vector.

+
+
+

Remaining arguments are as for the Vector constructor.

+
+ +
+
+assign(y, *, annotate=None, tlm=None)
+

Vector assignment.

+
+
Parameters:
+
    +

  • y – A numbers.Complex, Vector, or ndim 1 array +defining the value.

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
Returns:
+

The Vector.

+
+
+
+ +
+
+addto(y, *, annotate=None, tlm=None)
+

Vector in-place addition.

+
+
Parameters:
+
    +

  • y – A numbers.Complex, Vector, or ndim 1 array +defining the value to add.

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
+
+ +
+ +
+
+class tlm_adjoint.jax.VectorEquation(X, Y, fn, *, with_tlm=True, _forward_eq=None)
+

JAX interface. fn should be a callable

+
def fn(y0, y1, ...):
+    ...
+    return x0, x1, ...
+
+
+

where the y0, y1 are ndim 1 JAX arrays, and the x0, x1, are scalars +or ndim 1 JAX arrays.

+
+
Parameters:
+
    +

  • X – A Vector or a Sequence of Vector +objects defining outputs, whose value is set by the return value from +fn.

    • +

  • Y – A Vector or a Sequence of Vector +objects defining the inputs, whose values are passed to fn.

    • +

  • fn – A callable.

    • +

  • with_tlm – Whether to annotate an equation solving for the forward and +all tangent-linears (with_tlm=True), or solving only for the +forward (with_tlm=False).

    • +
+
+
+
+
+solve(*, manager=None, annotate=None, tlm=None)
+

Compute the forward solution.

+
+
Parameters:
+
    +

  • manager – The EquationManager. Defaults to manager().

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
+
+ +
+
+forward_solve(X, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_jacobian_solve(adj_X, nl_deps, B)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+subtract_adjoint_derivative_actions(adj_X, nl_deps, dep_Bs)
+

Subtract terms from other adjoint right-hand-sides.

+

Can be overridden for an optimized implementation, but otherwise uses +Equation.adjoint_derivative_action().

+
+
Parameters:
+
    +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_Bs – A Mapping whose items are (dep_index, dep_B). +Each dep_B is an AdjointRHS which should be updated by +subtracting adjoint derivative information computed by +differentiating with respect to self.dependencies()[dep_index].

    • +
+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+tlm_adjoint.jax.call_jax(X, Y, fn)
+

JAX interface. fn should be a callable

+
def fn(y0, y1, ...):
+    ...
+    return x0, x1, ...
+
+
+

where the y0, y1 are ndim 1 JAX arrays, and the x0, x1, are scalars +or ndim 1 JAX arrays.

+
+
Parameters:
+
    +

  • X – A Vector or a Sequence of Vector +objects defining outputs, whose value is set by the return value from +fn.

    • +

  • Y – A Vector or a Sequence of Vector +objects defining the inputs, whose values are passed to fn.

    • +

  • fn – A callable.

    • +
+
+
+
+ +
+
+tlm_adjoint.jax.new_jax(y, space=None, *, name=None)
+

Construct a new zero-valued Vector.

+
+
Parameters:
+
    +

  • y – A variable.

    • +

  • space – The VectorSpace for the return value.

    • +

  • name – A str name.

    • +
+
+
Returns:
+

The Vector.

+
+
+
+ +
+
+tlm_adjoint.jax.to_jax(y, space=None, *, name=None)
+

Convert a variable to a Vector.

+
+
Parameters:
+
    +

  • y – A variable.

    • +

  • space – The VectorSpace for the return value.

    • +

  • name – A str name.

    • +
+
+
Returns:
+

The Vector.

+
+
+
+ +
+
+tlm_adjoint.jax.new_jax_float(space=None, *, name=None, dtype=None, comm=None)
+

Create a new Vector with one element.

+
+
Parameters:
+
    +

  • space – The VectorSpace.

    • +

  • name – A str name.

    • +

  • dtype – The data type. Ignored if space is supplied.

    • +

  • comm – A communicator. Ignored if space is supplied.

    • +
+
+
Returns:
+

A Vector with one element.

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/linear_equation/index.html b/autoapi/tlm_adjoint/linear_equation/index.html new file mode 100644 index 0000000..6b63b4d --- /dev/null +++ b/autoapi/tlm_adjoint/linear_equation/index.html @@ -0,0 +1,616 @@ + + + + + + + tlm_adjoint.linear_equation — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.linear_equation

+
+

Module Contents

+
+
+class tlm_adjoint.linear_equation.LinearEquation(X, B, *, A=None, adj_type=None)
+

Represents the solution of a linear equation

+
+\[A x = \sum_i b_i,\]
+

with left-hand-side matrix \(A\) and right-hand-side terms \(b_i\). +The matrix and right-hand-side terms may depend on other forward variables +\(y_i\).

+

The forward residual is defined

+
+\[\mathcal{F} \left( x, y_1, y_2, \ldots \right) = A x - \sum_i b_i.\]
+
+
Parameters:
+
    +

  • X – A variable or a Sequence of variables defining the forward +solution x.

    • +

  • B – A RHS or a Sequence of RHS objects +defining the right-hand-side terms.

    • +

  • A – A tlm_adjoint.linear_equation.Matrix defining the +left-hand-side matrix. Defaults to an identity matrix if not supplied.

    • +

  • adj_type – The space type relative to X of adjoint variables. +‘primal’ or ‘conjugate_dual’, or a Sequence of these. +Defaults to ‘primal’ if A is supplied and ‘conjugate_dual’ +otherwise.

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+forward_solve(X, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_jacobian_solve(adj_X, nl_deps, B)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_X)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.linear_equation.Matrix(nl_deps=None, *, ic=True, adj_ic=True)
+

Represents a matrix \(A\).

+

This is an abstract base class. Information required by forward, adjoint, +and tangent-linear calculations is provided by overloading abstract +methods. This class does not inherit from abc.ABC, so that +methods may be implemented as needed.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables, defining dependencies of +the matrix \(A\).

    • +

  • ic – Whether solution of a linear equation \(A x = b\) for +\(x\) uses an initial guess. Defaults to True.

    • +

  • adj_ic – Whether solution of an adjoint linear equation \(A^* +\lambda = b\) for \(\lambda\) uses an initial guess.

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+nonlinear_dependencies()
+

Return dependencies of the +tlm_adjoint.linear_equation.Matrix.

+
+
Returns:
+

A Sequence of variables defining dependencies.

+
+
+
+ +
+
+has_initial_condition()
+

Return whether solution of a linear equation \(A x = b\) for +\(x\) uses an initial guess.

+
+
Returns:
+

True if an initial guess is used, and False otherwise.

+
+
+
+ +
+
+adjoint_has_initial_condition()
+

Return whether solution of an adjoint linear equation \(A^* +\lambda = b\) for \(\lambda\) uses an initial guess.

+
+
Returns:
+

True if an initial guess is used, and False otherwise.

+
+
+
+ +
+
+abstract forward_action(nl_deps, X, B, *, method='assign')
+

Evaluate the action of the matrix on \(x\), \(A x\). Assigns +the result to B, or adds the result to or subtracts the result from +B.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +dependencies. Should not be modified.

    • +

  • X – Defines \(x\). A variable if it has a single component, and +a Sequence of variables otherwise. Should not be modified. +Subclasses may replace this argument with x if there is a single +component.

    • +

  • B – Stores the result. A variable if it has a single component, and +a Sequence of variables otherwise. Subclasses may replace +this argument with b if there is a single component.

    • +

  • method – If equal to ‘assign’ then this method should set B +equal to the result. If equal to ‘add’ then this method should +add the result to B. If equal to ‘sub’ then this method should +subtract the result from B.

    • +
+
+
+
+ +
+
+abstract adjoint_action(nl_deps, adj_X, b, b_index=0, *, method='assign')
+

Evaluate the action of the adjoint of the matrix on +\(\lambda\), \(A^* \lambda\). Assigns the b_index th +component to b, or adds the b_index th component to or subtracts +the b_index th component from b.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +dependencies. Should not be modified.

    • +

  • adj_X – Defines \(\lambda\). A variable if it has a single +component, and a Sequence of variables otherwise. Should +not be modified. Subclasses may replace this argument with adj_x +if there is a single component.

    • +

  • b – A variable storing the result. Should be updated by this +method.

    • +

  • b_index – The component of the result which should be used to +update b.

    • +

  • method – If equal to ‘assign’ then this method should set b +equal to the b_index th component of the result. If equal to +‘add’ then this method should add the b_index th component of +the result to b. If equal to ‘sub’ then this method should +subtract the b_index th component of the result from b.

    • +
+
+
+
+ +
+
+abstract forward_solve(X, nl_deps, B)
+

Solve the linear system \(A x = b\) for \(x\).

+
+
Parameters:
+
    +

  • X – The solution \(x\). A variable if it has a single +component, and a Sequence of variables otherwise. May +define an initial guess. Subclasses may replace this argument with +x if there is a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +dependencies. Should not be modified.

    • +

  • B – The right-hand-side \(b\). A variable if it has a single +component, and a Sequence of variables otherwise. Should +not be modified. Subclasses may replace this argument with b if +there is a single component.

    • +
+
+
+
+ +
+
+abstract adjoint_derivative_action(nl_deps, nl_dep_index, X, adj_X, b, *, method='assign')
+

Evaluate the action of the adjoint of a derivative of \(A x\) on +an adjoint variable. Assigns the result to b, or adds the result to +or subtracts the result from b.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +dependencies. Should not be modified.

    • +

  • nl_deps_index – An int. The derivative is defined by +differentiation of \(A x\) with respect to +self.nonlinear_dependencies()[nl_dep_index].

    • +

  • X – Defines \(x\). A variable if it has a single component, and +a Sequence of variables otherwise. Should not be modified. +Subclasses may replace this argument with x if there is a single +component.

    • +

  • adj_X – The adjoint variable. A variable if it has a single +component, and Sequence of variables otherwise. Should not +be modified. Subclasses may replace this argument with adj_x if +the adjoint variable has a single component.

    • +

  • b – A variable storing the result. Should be updated by this +method.

    • +

  • method – If equal to ‘assign’ then this method should set b +equal to the result. If equal to ‘add’ then this method should +add the result to b. If equal to ‘sub’ then this method should +subtract the result from b.

    • +
+
+
+
+ +
+
+abstract adjoint_solve(adj_X, nl_deps, B)
+

Solve the linear system \(A^* \lambda = b\) for +\(\lambda\).

+
+
Parameters:
+
    +

  • adj_X – The solution \(\lambda\). A variable if it has a single +component, and a Sequence of variables otherwise. May +define an initial guess. Subclasses may replace this argument with +adj_x if there is a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +dependencies. Should not be modified.

    • +

  • B – The right-hand-side \(b\). A variable if it has a single +component, and a Sequence of variables otherwise. Should +not be modified. Subclasses may replace this argument with b if +there is a single component.

    • +
+
+
+
+ +
+
+abstract tangent_linear_rhs(M, dM, tlm_map, X)
+

Construct tangent-linear right-hand-side terms obtained by +differentiation of

+
+\[\mathcal{G} \left( x, y_1, y_2, \ldots \right) = -A x\]
+

with respect to dependencies of the matrix \(A\). i.e. construct

+
+\[-\sum_i \frac{\partial \mathcal{G}}{\partial y_i} \tau_{y_i},\]
+

where \(\tau_{y_i}\) is the tangent-linear variable associated with +the dependency \(y_i\). Note the negative sign. Does not +include the term \(-A \tau_x\) where \(\tau_x\) is the +tangent-linear variable associated with \(x\).

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +

  • X – Defines \(x\). A variable if it has a single component, and +a Sequence of variables otherwise. Subclasses may replace +this argument with x if there is a single component.

    • +
+
+
Returns:
+

A RHS, or a Sequence of RHS +objects, defining the right-hand-side terms. Returning None +indicates that there are no terms.

+
+
+
+ +
+ +
+
+class tlm_adjoint.linear_equation.RHS(deps, nl_deps=None)
+

Represents a right-hand-side term.

+

This is an abstract base class. Information required by forward, adjoint, +and tangent-linear calculations is provided by overloading abstract +methods. This class does not inherit from abc.ABC, so that +methods may be implemented as needed.

+
+
Parameters:
+
    +

  • deps – A Sequence of variables defining dependencies.

    • +

  • nl_deps – A Sequence of variables defining non-linear +dependencies.

    • +
+
+
+
+
+drop_references()
+

Drop references to variables which store values.

+
+ +
+
+dependencies()
+

Return dependencies of the RHS.

+
+
Returns:
+

A Sequence of variables defining dependencies.

+
+
+
+ +
+
+nonlinear_dependencies()
+

Return non-linear dependencies of the RHS.

+
+
Returns:
+

A Sequence of variables defining non-linear +dependencies.

+
+
+
+ +
+
+abstract add_forward(B, deps)
+

Add the right-hand-side term to B.

+
+
Parameters:
+
    +

  • B – A variable if it has a single component, and a +Sequence of variables otherwise. Should be updated by the +addition of this RHS. Subclasses may replace this +argument with b if there is a single component.

    • +

  • deps – A Sequence of variables defining values for +dependencies. Should not be modified.

    • +
+
+
+
+ +
+
+abstract subtract_adjoint_derivative_action(nl_deps, dep_index, adj_X, b)
+

Subtract the action of the adjoint of a derivative of the +right-hand-side term, on an adjoint variable, from b.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • deps_index – An int. The derivative is defined by +differentiation of the right-hand-side term with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint variable. A variable if it has a single +component, and a Sequence of variables otherwise. Should +not be modified. Subclasses may replace this argument with adj_x +if the adjoint variable has a single component.

    • +

  • b – A variable storing the result. Should be updated by subtracting +the action of the adjoint of the right-hand-side term on the +adjoint variable.

    • +
+
+
+
+ +
+
+abstract tangent_linear_rhs(M, dM, tlm_map)
+

Construct tangent-linear right-hand-side terms obtained by +differentiation of this right-hand-side term. That is, construct

+
+\[\sum_i \frac{\partial b}{\partial y_i} \tau_{y_i},\]
+

where \(b\) is this right-hand-side term, and \(\tau_{y_i}\) is +the tangent-linear variable associated with a dependency \(y_i\).

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

A RHS, or a Sequence of RHS +objects, defining the right-hand-side terms. Returning None +indicates that there are no terms.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/manager/index.html b/autoapi/tlm_adjoint/manager/index.html new file mode 100644 index 0000000..b0150d3 --- /dev/null +++ b/autoapi/tlm_adjoint/manager/index.html @@ -0,0 +1,248 @@ + + + + + + + tlm_adjoint.manager — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.manager

+

This module provides a simple EquationManager interface. Functions +defined here access and interact with the default manager.

+
+

Module Contents

+
+
+tlm_adjoint.manager.manager()
+
+
Returns:
+

An EquationManager, the current default manager.

+
+
+
+ +
+
+tlm_adjoint.manager.set_manager(manager)
+

Set the default manager.

+
+
Parameters:
+

manager – An EquationManager to use as the default manager.

+
+
+
+ +
+
+tlm_adjoint.manager.restore_manager(fn)
+

Decorator to revert the default manager to the manager used prior to +calling the decorated callable. A typical use is

+
@restore_manager
+def decorated(*M):
+    set_manager(manager().new())
+    forward(*M)
+
+
+
+
Parameters:
+

fn – A decorated callable.

+
+
Returns:
+

A callable, where the default manager on entry is restored on +exit.

+
+
+
+ +
+
+tlm_adjoint.manager.configure_checkpointing(cp_method, cp_parameters)
+

See EquationManager.configure_checkpointing().

+
+ +
+
+tlm_adjoint.manager.manager_info(*, info=print)
+

See EquationManager.info().

+
+ +
+
+tlm_adjoint.manager.reset_manager(cp_method=None, cp_parameters=None)
+

See EquationManager.reset().

+
+ +
+
+tlm_adjoint.manager.annotation_enabled()
+

See EquationManager.annotation_enabled().

+
+ +
+
+tlm_adjoint.manager.start_manager(*, annotate=True, tlm=True)
+

See EquationManager.start().

+
+ +
+
+tlm_adjoint.manager.stop_manager(*, annotate=True, tlm=True)
+

See EquationManager.stop().

+
+ +
+
+tlm_adjoint.manager.paused_manager(*, annotate=True, tlm=True)
+

See EquationManager.paused().

+
+ +
+
+tlm_adjoint.manager.manager_disabled(*, annotate=True, tlm=True)
+

Decorator which can be used to disable processing of equations and +derivation and solution of tangent-linear equations.

+
+
Parameters:
+
    +

  • annotate – Whether to disable processing of equations.

    • +

  • tlm – Whether to disable derivation and solution of tangent-linear +equations.

    • +
+
+
+
+ +
+
+tlm_adjoint.manager.configure_tlm(*args, annotate=None, tlm=True)
+

See EquationManager.configure_tlm().

+
+ +
+
+tlm_adjoint.manager.tlm_enabled()
+

See EquationManager.tlm_enabled().

+
+ +
+
+tlm_adjoint.manager.var_tlm(x, *args)
+

See EquationManager.var_tlm().

+
+ +
+
+tlm_adjoint.manager.compute_gradient(Js, M, *, callback=None, prune_forward=True, prune_adjoint=True, prune_replay=True, cache_adjoint_degree=None, store_adjoint=False, adj_ics=None)
+

See EquationManager.compute_gradient().

+
+ +
+
+tlm_adjoint.manager.new_block()
+

See EquationManager.new_block().

+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/markers/index.html b/autoapi/tlm_adjoint/markers/index.html new file mode 100644 index 0000000..d1aaa19 --- /dev/null +++ b/autoapi/tlm_adjoint/markers/index.html @@ -0,0 +1,242 @@ + + + + + + + tlm_adjoint.markers — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.markers

+
+

Module Contents

+
+
+class tlm_adjoint.markers.ControlsMarker(M)
+

Represents

+
+\[m = m_\text{input},\]
+

where \(m\) is the control and \(m_\text{input}\) the input value +for the control. The forward residual is defined

+
+\[\mathcal{F} \left( m \right) = m - m_\text{input}.\]
+
+
Parameters:
+

M – A variable or a Sequence of variables defining the +control \(m\). May be static.

+
+
+
+
+adjoint_jacobian_solve(adj_X, nl_deps, B)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+ +
+
+class tlm_adjoint.markers.FunctionalMarker(J)
+

Represents

+
+\[J_\text{output} = J,\]
+

where \(J\) is the functional and \(J_\text{output}\) is the output +value for the functional. The forward residual is defined

+
+\[\mathcal{F} \left( J_\text{output}, J \right) = J_\text{output} - J.\]
+
+
Parameters:
+

J – A variable defining the functional \(J\).

+
+
+
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/optimization/index.html b/autoapi/tlm_adjoint/optimization/index.html new file mode 100644 index 0000000..c4084c4 --- /dev/null +++ b/autoapi/tlm_adjoint/optimization/index.html @@ -0,0 +1,436 @@ + + + + + + + tlm_adjoint.optimization — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.optimization

+
+

Module Contents

+
+
+tlm_adjoint.optimization.minimize_scipy(forward, M0, *, manager=None, **kwargs)
+

Provides an interface with scipy.optimize.minimize() for +gradient-based optimization.

+

Note that the control is gathered onto the root process so that the serial +scipy.optimize.minimize() function may be used.

+

All keyword arguments except for manager are passed to +scipy.optimize.minimize().

+
+
Parameters:
+
    +

  • forward – A callable which accepts one or more variable arguments, and +which returns a variable defining the forward functional.

    • +

  • M0 – A variable or Sequence of variables defining the control, +and the initial guess for the optimization.

    • +

  • manager – An EquationManager used to create an internal +manager via EquationManager.new(). manager() is used if not +supplied.

    • +
+
+
Returns:
+

A tuple (M, return_value). M is variable or a +Sequence of variables depending on the type of M0, and +stores the result. return_value is the return value of +scipy.optimize.minimize().

+
+
+
+ +
+
+class tlm_adjoint.optimization.LBFGSHessianApproximation(m)
+

L-BFGS Hessian approximation.

+
+
Parameters:
+

m – Maximum number of vector pairs to retain in the L-BFGS Hessian +approximation.

+
+
+
+
+append(S, Y, S_inner_Y)
+

Add a step + gradient change pair.

+
+
Parameters:
+
    +

  • S – A variable or a Sequence of variables defining the +step.

    • +

  • Y – A variable or a Sequence of variables defining the +gradient change.

    • +

  • S_inner_Y – The projection of the gradient change onto the step. +A separate argument so that a value consistent with +that used in the line search can be supplied.

    • +
+
+
+
+ +
+
+inverse_action(X, *, H_0_action=None, theta=1.0)
+

Compute the action of the approximate Hessian inverse on some given +direction.

+

Implements the L-BFGS Hessian inverse action approximation as in +Algorithm 7.4 of

+
+
    +

  • Jorge Nocedal and Stephen J. Wright, ‘Numerical optimization’, +Springer, New York, NY, 2006, Second edition, +doi: 10.1007/978-0-387-40065-5

    • +
+
+

Uses ‘theta scaling’ as in equation (3.7) of

+
+
    +

  • Richard H. Byrd, Peihuang Lu, and Jorge Nocedal, and Ciyou Zhu, +‘A limited memory algorithm for bound constrained optimization’, +SIAM Journal on Scientific Computing, 16(5), 1190–1208, 1995, +doi: 10.1137/0916069

    • +
+
+
+
Parameters:
+
    +

  • X – A variable or a Sequence of variables defining the +direction on which to compute the approximate Hessian inverse +action.

    • +

  • H_0_action – A callable defining the action of the non-updated +Hessian inverse approximation on some direction. Accepts one or +more variables as arguments, defining the direction, and returns a +variable or a Sequence of variables defining the action on +this direction. Should correspond to a positive definite operator. +An identity is used if not supplied.

    • +
+
+
Returns:
+

A variable or a Sequence of variables storing the +result.

+
+
+
+ +
+ +
+
+tlm_adjoint.optimization.l_bfgs(F, Fp, X0, *, m=30, s_atol, g_atol, converged=None, max_its=1000, H_0_action=None, theta_scale=True, delta=1.0, M_action=None, M_inv_action=None, c1=0.0001, c2=0.9, comm=None)
+

Functional minimization using the L-BFGS algorithm.

+

Implements Algorithm 7.5 of

+
+
    +

  • Jorge Nocedal and Stephen J. Wright, ‘Numerical optimization’, +Springer, New York, NY, 2006, Second edition, +doi: 10.1007/978-0-387-40065-5

    • +
+
+

in a more general inner product space.

+

By default uses ‘theta scaling’ to define the initial Hessian inverse +approximation, based on the approach in equation (3.7) and point 7 on p. +1204 of

+
+
    +

  • Richard H. Byrd, Peihuang Lu, and Jorge Nocedal, and Ciyou Zhu, ‘A +limited memory algorithm for bound constrained optimization’, SIAM +Journal on Scientific Computing, 16(5), 1190–1208, 1995, +doi: 10.1137/0916069

    • +
+
+

and with an initial value for the scaling parameter based on the discussion +in ‘Implementation’ in section 6.1 of

+
+
    +

  • Jorge Nocedal and Stephen J. Wright, ‘Numerical optimization’, +Springer, New York, NY, 2006, Second edition, +doi: 10.1007/978-0-387-40065-5

    • +
+
+

Precisely the Hessian inverse approximation, before being updated, is +scaled by \(1 / \theta\) with, on the first iteration,

+
+\[\theta = \frac{\sqrt{\left| g_k^* M^{-1} g_k \right|}}{\delta}\]
+

and on later iterations

+
+\[\theta = \frac{y_k^* H_0 y_k}{y_k^* s_k},\]
+

where \(g_k\), \(y_k\), and \(s_k\) are respectively the +gradient, gradient change, and step, and where \(M^{-1}\) and +\(H_0\) are defined by M_inv_action and H_0_action respectively.

+
+
Parameters:
+
    +

  • F – A callable defining the functional. Accepts one or more variables +as arguments, and returns the value of the functional. Input arguments +should not be modified.

    • +

  • Fp – A callable defining the functional gradient. Accepts one or more +variables as inputs, and returns a variable or Sequence of +variables storing the value of the gradient. Input arguments should not +be modified.

    • +

  • X0 – A variable or a Sequence of variables defining the +initial guess for the parameters.

    • +

  • m – The maximum number of step + gradient change pairs to use in the +Hessian inverse approximation.

    • +

  • s_atol – Absolute tolerance for the step change norm convergence +criterion.

    • +

  • g_atol – Absolute tolerance for the gradient norm convergence +criterion.

    • +

  • converged –

    A callable defining a callback, and which can be used to +define custom convergence criteria. Takes the form

    +
    def converged(it, F_old, F_new, X_new, G_new, S, Y):
+
    +
    +

    with

    +
    +
      +

    • it: The iteration number.

      • +

    • F_old: The previous value of the functional.

      • +

    • F_new: The new value of the functional.

      • +

    • X_new: A variable or a Sequence of variables defining +the new value of the parameters.

      • +

    • G_new: A variable or a Sequence of variables defining +the new value of the gradient.

      • +

    • S: A variable or a Sequence of variables defining the +step.

      • +

    • Y: A variable or a sequence of variables defining the gradient +change.

      • +
    +
    +

    Input variables should not be modified. Returns a bool +indicating whether the optimization has converged.

    +

    • +

  • max_its – The maximum number of iterations.

    • +

  • H_0_action – A callable defining the action of the non-updated Hessian +inverse approximation on some direction. Accepts one or more variables +as arguments, defining the direction, and returns a variable or a +Sequence of variables defining the action on this direction. +Should correspond to a positive definite operator. An identity is used +if not supplied.

    • +

  • theta_scale – Whether to apply ‘theta scaling’, discussed above.

    • +

  • delta – Controls the initial value of \(\theta\) in ‘theta +scaling’. If None then on the first iteration \(\theta\) is set +equal to one.

    • +

  • M_action –

    A callable defining a primal space inner product,

    +
    +\[\left< x, y \right>_M = y^* M x,\]
    +

    where \(x\) and \(y\) are degree of freedom vectors for primal +space elements and \(M\) is a Hermitian and positive definite +matrix. Accepts one or more variables as arguments, defining the +direction, and returns a variable or a Sequence of variables +defining the action of \(M\) on this direction. An identity is used +if not supplied. Required if H_0_action or M_inv_action are +supplied.

    +

    • +

  • M_inv_action –

    A callable defining a (conjugate) dual space inner +product,

    +
    +\[\left< x, y \right>_{M^{-1}} = y^* M^{-1} x,\]
    +

    where \(x\) and \(y\) are degree of freedom vectors for +(conjugate) dual space elements and \(M\) is as for M_action. +Accepts one or more variables as arguments, defining the direction, and +returns a variable or a Sequence of variables defining the +action of \(M^{-1}\) on this direction. H_0_action is used if not +supplied.

    +

    • +

  • c1 –

    Armijo condition parameter. \(c_1\) in equation (3.6a) of

    +
      +

    • Jorge Nocedal and Stephen J. Wright, ‘Numerical optimization’, +Springer, New York, NY, 2006, Second edition, +doi: 10.1007/978-0-387-40065-5

      • +
    +

    • +

  • c2 –

    Curvature condition parameter. \(c_2\) in equation (3.6b) of

    +
      +

    • Jorge Nocedal and Stephen J. Wright, ‘Numerical optimization’, +Springer, New York, NY, 2006, Second edition, +doi: 10.1007/978-0-387-40065-5

      • +
    +

    • +

  • comm – A communicator.

    • +
+
+
Returns:
+

A tuple (X, (it, F_calls, Fp_calls, hessian_approx)) +with

+
    +

  • X: The solution. A variable or a tuple of variables.

    • +

  • it: The number of iterations.

    • +

  • F_calls: The number of functional evaluations.

    • +

  • Fp_calls: The number of gradient evaluations.

    • +

  • hessian_approx: The LBFGSHessianApproximation.

    • +
+

+
+
+
+ +
+
+tlm_adjoint.optimization.minimize_l_bfgs(forward, M0, *, m=30, manager=None, **kwargs)
+

Functional minimization using the L-BFGS algorithm.

+
+
Parameters:
+
    +

  • forward – A callable which accepts one or more variable arguments, and +which returns a variable defining the forward functional.

    • +

  • M0 – A variable or Sequence of variables defining the control, +and the initial guess for the optimization.

    • +

  • manager – An EquationManager used to create an internal +manager via EquationManager.new(). manager() is used if not +supplied.

    • +
+
+
+

Remaining arguments and the return value are described in the +l_bfgs() documentation.

+
+ +
+
+tlm_adjoint.optimization.minimize_tao(forward, M0, *, method=None, gatol, grtol, gttol=0.0, M_inv_action=None, pre_callback=None, post_callback=None, manager=None)
+

Functional minimization using TAO.

+
+
Parameters:
+
    +

  • forward – A callable which accepts one or more variable arguments, and +which returns a variable defining the forward functional.

    • +

  • M0 – A variable or Sequence of variables defining the control, +and the initial guess for the optimization.

    • +

  • method – TAO type. Defaults to PETSc.TAO.Type.LMVM.

    • +

  • gatol – TAO gradient absolute tolerance.

    • +

  • grtol – TAO gradient relative tolerance.

    • +

  • gttol – TAO gradient norm change relative tolerance.

    • +

  • M_inv_action –

    A callable defining a (conjugate) dual space inner +product,

    +
    +\[\left< x, y \right>_{M^{-1}} = y^* M^{-1} x,\]
    +

    where \(x\) and \(y\) are degree of freedom vectors for +(conjugate) dual space elements and \(M\) is a Hermitian and +positive definite matrix. Accepts one or more variables as arguments, +defining the direction, and returns a variable or a Sequence +of variables defining the action of \(M^{-1}\) on this direction.

    +

    • +

  • pre_callback – A callable accepting a single +petsc4py.PETSc.TAO argument. Used for detailed manual +configuration. Called after all other configuration options are set.

    • +

  • post_callback – A callable accepting a single +petsc4py.PETSc.TAO argument. Called after the +petsc4py.PETSc.TAO.solve() method has been called.

    • +

  • manager – An EquationManager used to create an internal +manager via EquationManager.new(). manager() is used if not +supplied.

    • +
+
+
Returns:
+

A variable or a Sequence of variables storing the +result.

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/overloaded_float/index.html b/autoapi/tlm_adjoint/overloaded_float/index.html new file mode 100644 index 0000000..f0b780c --- /dev/null +++ b/autoapi/tlm_adjoint/overloaded_float/index.html @@ -0,0 +1,468 @@ + + + + + + + tlm_adjoint.overloaded_float — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.overloaded_float

+

This module defines types which allow for basic floating point level +algorithmic differentiation. The implementation is intended to be used for a +small number of calculations, for example after the calculation of a functional +obtained from a finite element model.

+

For example, if annotation and operator overloading is enabled

+
import numpy as np
+
+x = Float(np.pi, name='x')
+y = x * np.sin(x)
+
+
+

will lead to annotation of equations associated with the floating point +calculations.

+
+

Module Contents

+
+
+tlm_adjoint.overloaded_float.set_default_float_dtype(dtype)
+

Set the default data type used by SymbolicFloat objects.

+
+
Parameters:
+

dtype – The default data type.

+
+
+
+ +
+
+class tlm_adjoint.overloaded_float.FloatSpace(float_cls=None, *, dtype=None, comm=None)
+

Defines the real or complex space.

+
+
Parameters:
+
+
+
+
+
+property dtype
+

The data type associated with the space.

+
+ +
+
+property comm
+

The communicator associated with the space.

+
+ +
+
+property float_cls
+

The SymbolicFloat class associated with the space.

+
+ +
+
+rdtype()
+

The real data type associated with the space.

+
+ +
+ +
+
+tlm_adjoint.overloaded_float.no_float_overloading(fn)
+

Decorator to disable OverloadedFloat operator overloading.

+
+
Parameters:
+

fn – A callable for which OverloadedFloat operator +overloading should be disabled.

+
+
Returns:
+

A callable for which OverloadedFloat operator +overloading is disabled.

+
+
+
+ +
+
+tlm_adjoint.overloaded_float.paused_float_overloading()
+

Construct a context manager which can be used to temporarily disable +OverloadedFloat operator overloading.

+
+
Returns:
+

A context manager which can be used to temporarily disable +OverloadedFloat operator overloading.

+
+
+
+ +
+
+class tlm_adjoint.overloaded_float.SymbolicFloat(value=0.0, *, name=None, space_type='primal', static=False, cache=None, dtype=None, comm=None, annotate=None, tlm=None)
+

A sympy.core.symbol.Symbol which is also a ‘variable’, defining +a scalar variable.

+

If constructing SymPy expressions then the SymbolicFloat class +should be used instead of the OverloadedFloat subclass, or else +OverloadedFloat operator overloading should be disabled.

+
+
Parameters:
+
    +

  • value – A numbers.Complex or sympy.core.expr.Expr +defining the initial value. If a sympy.core.expr.Expr then, if +annotation or derivation and solution of tangent-linear equations is +enabled, an assignment is processed by the EquationManager +manager.

    • +

  • name – A str name for the SymbolicFloat.

    • +

  • space_type – The space type for the SymbolicFloat. +‘primal’, ‘dual’, ‘conjugate’, or ‘conjugate_dual’.

    • +

  • static – Defines whether the SymbolicFloat is static, meaning +that it is stored by reference in checkpointing/replay, and an +associated tangent-linear variable is zero.

    • +

  • cache – Defines whether results involving the SymbolicFloat +may be cached. Default static.

    • +

  • dtype – The data type associated with the SymbolicFloat. +Typically numpy.double or numpy.cdouble.

    • +

  • comm – The communicator associated with the SymbolicFloat.

    • +

  • annotate – Whether the EquationManager should record the +solution of equations.

    • +

  • tlm – Whether tangent-linear equations should be solved.

    • +
+
+
+
+
+property value
+

Return the current value associated with the +SymbolicFloat.

+

The value may also be accessed by casting using float or +complex.

+
+
Returns:
+

The value.

+
+
+
+ +
+
+new(value=0.0, *, name=None, static=False, cache=None, annotate=None, tlm=None)
+

Return a new object, which same type and space type as this +SymbolicFloat.

+
+
Returns:
+

The new SymbolicFloat.

+
+
+

Arguments are as for the SymbolicFloat constructor.

+
+ +
+
+assign(y, *, annotate=None, tlm=None)
+

SymbolicFloat assignment.

+
+
Parameters:
+
+
+
Returns:
+

The SymbolicFloat.

+
+
+
+ +
+
+addto(y, *, annotate=None, tlm=None)
+

SymbolicFloat in-place addition.

+
+
Parameters:
+
+
+
+
+ +
+ +
+
+class tlm_adjoint.overloaded_float.OverloadedFloat(value=0.0, *, name=None, space_type='primal', static=False, cache=None, dtype=None, comm=None, annotate=None, tlm=None)
+

A subclass of SymbolicFloat with operator overloading.

+

If constructing SymPy expressions then the SymbolicFloat class +should be used instead of the OverloadedFloat subclass, or else +OverloadedFloat operator overloading should be disabled.

+

For argument documentation see SymbolicFloat.

+
+ +
+
+class tlm_adjoint.overloaded_float.Float(value=0.0, *, name=None, space_type='primal', static=False, cache=None, dtype=None, comm=None, annotate=None, tlm=None)
+

A subclass of SymbolicFloat with operator overloading.

+

If constructing SymPy expressions then the SymbolicFloat class +should be used instead of the OverloadedFloat subclass, or else +OverloadedFloat operator overloading should be disabled.

+

For argument documentation see SymbolicFloat.

+
+ +
+
+class tlm_adjoint.overloaded_float.FloatEquation(x, expr)
+

Represents an assignment to a SymbolicFloat x,

+
+\[x = \mathcal{G} \left( y_1, y_2, \ldots \right),\]
+

for some \(\mathcal{G}\) defined by a sympy.core.expr.Expr. +The forward residual is defined

+
+\[\mathcal{F} \left( x, y_1, y_2, \ldots \right) + = x - \mathcal{G} \left( y_1, y_2, \ldots \right).\]
+
+
Parameters:
+
+
+
+
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+subtract_adjoint_derivative_actions(adj_x, nl_deps, dep_Bs)
+

Subtract terms from other adjoint right-hand-sides.

+

Can be overridden for an optimized implementation, but otherwise uses +Equation.adjoint_derivative_action().

+
+
Parameters:
+
    +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_Bs – A Mapping whose items are (dep_index, dep_B). +Each dep_B is an AdjointRHS which should be updated by +subtracting adjoint derivative information computed by +differentiating with respect to self.dependencies()[dep_index].

    • +
+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+tlm_adjoint.overloaded_float.to_float(y, *, name=None)
+

Convert a variable to a Float.

+
+
Parameters:
+
    +

  • y – A scalar variable.

    • +

  • name – A str name.

    • +
+
+
Returns:
+

The SymbolicFloat.

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/storage/index.html b/autoapi/tlm_adjoint/storage/index.html new file mode 100644 index 0000000..3e7977e --- /dev/null +++ b/autoapi/tlm_adjoint/storage/index.html @@ -0,0 +1,397 @@ + + + + + + + tlm_adjoint.storage — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.storage

+
+

Module Contents

+
+
+class tlm_adjoint.storage.Storage(x, key, *, save=False)
+

Used to save and load a forward solution value.

+

With save=True the first forward solve saves the value of x. With +save=False, or on any subsequent forward solve, the value of the forward +solution is loaded into x.

+

When processed by the EquationManager this is equivalent to an +assignment

+
+\[x = x_\text{value},\]
+

where \(x_\text{value}\) is the value which is saved or loaded. The +forward residual is defined

+
+\[\mathcal{F} \left( x \right) = x - x_\text{value}.\]
+

This is an abstract base class. Information required to save and load data +is provided by overloading abstract methods. This class does not inherit +from abc.ABC, so that methods may be implemented as needed.

+
+
Parameters:
+
    +

  • x – A variable defining the forward solution, whose value is saved or +loaded.

    • +

  • key – A str key for the saved or loaded data.

    • +

  • save – If True then the first forward solve saves the value of x. +If False then the first forward solve loads the value of x.

    • +
+
+
+
+
+property key
+

The key associated with saved or loaded data.

+
+ +
+
+abstract is_saved()
+

Return whether a value can be loaded.

+
+
Returns:
+

True if a value can be loaded, and False otherwise.

+
+
+
+ +
+
+abstract load(x)
+

Load data, storing the result in x.

+
+
Parameters:
+

x – A variable in which the loaded data is stored.

+
+
+
+ +
+
+abstract save(x)
+

Save the value of x.

+
+
Parameters:
+

x – A variable whose value should be saved.

+
+
+
+ +
+
+forward_solve(x, deps=None)
+

Compute the forward solution.

+

Can assume that the currently active EquationManager is +paused.

+
+
Parameters:
+
    +

  • X – A variable if the forward solution has a single component, +otherwise a Sequence of variables. May define an initial +guess, and should be set by this method. Subclasses may replace +this argument with x if the forward solution has a single +component.

    • +

  • deps – A tuple of variables, defining values for +dependencies. Only the elements corresponding to X may be +modified. self.dependencies() should be used if not supplied.

    • +
+
+
+
+ +
+
+adjoint_jacobian_solve(adj_x, nl_deps, b)
+

Compute an adjoint solution.

+
+
Parameters:
+
    +

  • adj_X – Either None, or a variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +defining the initial guess for an iterative solve. May be modified +or returned. Subclasses may replace this argument with adj_x if +the adjoint solution has a single component.

    • +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • B – The right-hand-side. A variable (if the adjoint solution has a +single component) or Sequence of variables (otherwise) +storing the value of the right-hand-side. May be modified or +returned. Subclasses may replace this argument with b if the +adjoint solution has a single component.

    • +
+
+
Returns:
+

A variable or Sequence of variables storing the +value of the adjoint solution. May return None to indicate a +value of zero.

+
+
+
+ +
+
+adjoint_derivative_action(nl_deps, dep_index, adj_x)
+

Return the action of the adjoint of a derivative of the forward +residual on the adjoint solution. This is the negative of an adjoint +right-hand-side term.

+
+
Parameters:
+
    +

  • nl_deps – A Sequence of variables defining values for +non-linear dependencies. Should not be modified.

    • +

  • dep_index – An int. The derivative is defined by +differentiation of the forward residual with respect to +self.dependencies()[dep_index].

    • +

  • adj_X – The adjoint solution. A variable if the adjoint solution +has a single component, otherwise a Sequence of variables. +Should not be modified. Subclasses may replace this argument with +adj_x if the adjoint solution has a single component.

    • +
+
+
Returns:
+

The action of the adjoint of a derivative on the adjoint +solution. Will be passed to +subtract_adjoint_derivative_action(), and valid types depend +upon the adjoint variable type. Typically this will be a variable, +or a two element tuple (alpha, F), where alpha is a +numbers.Complex and F a variable, with the value defined +by the product of alpha and F.

+
+
+
+ +
+
+tangent_linear(M, dM, tlm_map)
+

Derive an Equation corresponding to a tangent-linear +operation.

+
+
Parameters:
+
    +

  • M – A Sequence of variables defining the control.

    • +

  • dM – A Sequence of variables defining the derivative +direction. The tangent-linear computes directional derivatives with +respect to the control defined by M and with direction defined by +dM.

    • +

  • tlm_map – A TangentLinearMap storing values for +tangent-linear variables.

    • +
+
+
Returns:
+

An Equation, corresponding to the tangent-linear +operation.

+
+
+
+ +
+ +
+
+class tlm_adjoint.storage.MemoryStorage(x, d, key, *, save=False)
+

A Storage which stores the value in memory.

+
+
Parameters:
+
    +

  • x – A variable defining the forward solution, whose value is saved or +loaded.

    • +

  • d – A dict in which data is stored with key key.

    • +

  • key – A str key for the saved or loaded data.

    • +

  • save – If True then the first forward solve saves the value of x. +If False then the first forward solve loads the value of x

    • +
+
+
+
+
+is_saved()
+

Return whether a value can be loaded.

+
+
Returns:
+

True if a value can be loaded, and False otherwise.

+
+
+
+ +
+
+load(x)
+

Load data, storing the result in x.

+
+
Parameters:
+

x – A variable in which the loaded data is stored.

+
+
+
+ +
+
+save(x)
+

Save the value of x.

+
+
Parameters:
+

x – A variable whose value should be saved.

+
+
+
+ +
+ +
+
+class tlm_adjoint.storage.HDF5Storage(x, h, key, *, save=False)
+

A Storage which stores the value on disk using the h5py +library.

+
+
Parameters:
+
    +

  • x – A variable defining the forward solution, whose value is saved or +loaded.

    • +

  • h – An h5py.File.

    • +

  • key – A str key for the saved or loaded data.

    • +

  • save – If True then the first forward solve saves the value of x. +If False then the first forward solve loads the value of x

    • +
+
+
+
+
+is_saved()
+

Return whether a value can be loaded.

+
+
Returns:
+

True if a value can be loaded, and False otherwise.

+
+
+
+ +
+
+load(x)
+

Load data, storing the result in x.

+
+
Parameters:
+

x – A variable in which the loaded data is stored.

+
+
+
+ +
+
+save(x)
+

Save the value of x.

+
+
Parameters:
+

x – A variable whose value should be saved.

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/tangent_linear/index.html b/autoapi/tlm_adjoint/tangent_linear/index.html new file mode 100644 index 0000000..f8034fc --- /dev/null +++ b/autoapi/tlm_adjoint/tangent_linear/index.html @@ -0,0 +1,163 @@ + + + + + + + tlm_adjoint.tangent_linear — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.tangent_linear

+
+

Module Contents

+
+
+class tlm_adjoint.tangent_linear.TangentLinearMap(M, dM)
+

Defines a map from forward variables to associated tangent-linear +variables.

+

The map is used via e.g.

+
tau_x = tlm_map[x]
+
+
+

where x is a forward variable.

+
+
    +

  • If x defines a component of the control, then tau_x is a variable +defining the associated component of the direction.

    • +

  • If x does not define a component of the control and is not +‘static’, then tau_x is a tangent-linear variable. A new variable +is instantiated if needed.

    • +

  • Otherwise tau_x is None, indicating that the tangent-linear +variable is zero.

    • +
+
+

Containment can be tested

+
if x in tlm_map:
+    [...]
+
+
+

and returns True if x defines a component of the control, or a +tangent-linear variable associated with x has been instantiated.

+
+
Parameters:
+
    +

  • M – A variable or Sequence of variables defining the control.

    • +

  • dM – A variable or Sequence of variables defining the +derivative direction. The tangent-linear computes directional +derivatives with respect to the control defined by M and with +direction defined by dM.

    • +
+
+
+
+
+property id
+

A unique int ID associated with this +TangentLinearMap.

+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/tlm_adjoint/index.html b/autoapi/tlm_adjoint/tlm_adjoint/index.html new file mode 100644 index 0000000..1dea087 --- /dev/null +++ b/autoapi/tlm_adjoint/tlm_adjoint/index.html @@ -0,0 +1,616 @@ + + + + + + + tlm_adjoint.tlm_adjoint — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.tlm_adjoint

+
+

Module Contents

+
+
+class tlm_adjoint.tlm_adjoint.EquationManager(*, comm=None, cp_method='memory', cp_parameters=None)
+

Core manager class.

+
+
    +

  • Plays the role of an adjoint ‘tape’. Records forward equations as +they are solved.

    • +

  • Interacts with checkpointing schedules for adjoint checkpointing and +forward replay.

    • +

  • Derives and manages tangent-linear equations. Tangent-linear +equations are processed as new forward equations, allowing higher +order adjoint calculations.

    • +

  • Handles variable reference dropping, e.g. handles the dropping of +references to variables which store values, and their replacement +with symbolic equivalents, after Equation objects holding +those references have been destroyed. Internally the manager retains +a reference to a WeakAlias subclass so that the +Equation methods may be called after the original +Equation is destroyed.

    • +
+
+

The manager processes forward equations (and tangent-linear equations) as +they are solved. Equations are collected into ‘blocks’ of equations, +corresponding to ‘steps’ in step-based checkpointing schedules. For +checkpointing schedule configuration details see +EquationManager.configure_checkpointing().

+

The configuration of tangent-linears is defined by a tangent-linear tree. +The root node of this tree corresponds to the forward. Following \(n\) +edges from the root node leads to a node associated with an \(n\) th +order tangent-linear. For tangent-linear configuration details see +EquationManager.configure_tlm().

+

On instantiation both equation annotation and tangent-linear derivation and +solution are disabled.

+
+
Parameters:
+
+
+
+
+
+comm()
+
+
Returns:
+

The communicator associated with the manager.

+
+
+
+ +
+
+info(*, info=print)
+

Print manager state information.

+
+
Parameters:
+

info – A callable which accepts and prints a str.

+
+
+
+ +
+
+new(cp_method=None, cp_parameters=None)
+

Construct a new EquationManager sharing the communicator +with this EquationManager. By default the new +EquationManager also shares the checkpointing schedule +configuration with this EquationManager, but this may be +overridden with the arguments cp_method and cp_parameters.

+

Both equation annotation and tangent-linear derivation and solution are +disabled for the new EquationManager.

+
+
Parameters:
+
+
+
+
+ +
+
+reset(cp_method=None, cp_parameters=None)
+

Reset the EquationManager. Clears all recorded equations, +and all configured tangent-linears.

+

By default the EquationManager retains its previous +checkpointing schedule configuration, but this may be overridden with +the arguments cp_method and cp_parameters.

+

Both equation annotation and tangent-linear derivation and solution are +disabled after calling this method.

+
+
Parameters:
+
+
+
+
+ +
+
+configure_checkpointing(cp_method, cp_parameters)
+

Provide a new checkpointing schedule configuration.

+

The checkpointing schedule type is defined by the argument cp_method, +and detailed configuration options are provided by the Mapping +argument cp_parameters.

+

cp_method values:

+
+
    +

  • ‘none’: No checkpointing. Can be used for tangent-linear only +calculations. Options defined by cp_parameters:

    +
    +
      +

    • ‘drop_references’: Whether to automatically drop references +to variables which store values. bool, optional, +default False.

      • +
    +
    +
    • +

  • ‘memory’: Store all forward restart data and non-linear +dependency data in memory. Options defined by cp_parameters:

    +
    +
      +

    • ‘drop_references’: Whether to automatically drop references +to variables which store values. bool, optional, +default False.

      • +
    +
    +
    • +

  • ‘periodic_disk: Periodically store forward restart data on +disk. Options defined by cp_parameters:

    +
    +
      +

    • ‘path’: Directory in which disk checkpoint data should be +stored. str, optional, default ‘checkpoints~’.

      • +

    • ‘format’: Disk storage format. Either ‘pickle’, for +data storage using the pickle module, or ‘hdf5’, for data +storage using the h5py library.

      • +

    • ‘period’: Interval, in blocks, between storage of forward +restart data. int, required.

      • +
    +
    +
    • +

  • ‘multistage’: Forward restart checkpointing with checkpoint +distribution as described in:

    +
    +
      +

    • Andreas Griewank and Andrea Walther, ‘Algorithm 799: +revolve: an implementation of checkpointing for the reverse +or adjoint mode of computational differentiation’, ACM +Transactions on Mathematical Software, 26(1), pp. 19–45, +2000, doi: 10.1145/347837.347846

      • +
    +
    +

    The memory/disk storage distribution is determined by an +initial run of the checkpointing schedule, leading to a +distribution equivalent to that in:

    +
    +
      +

    • Philipp Stumm and Andrea Walther, ‘MultiStage approaches +for optimal offline checkpointing’, SIAM Journal on +Scientific Computing, 31(3), pp. 1946–1967, 2009, doi: +10.1137/080718036

      • +
    +
    +

    Options defined by cp_parameters:

    +
    +
      +

    • ‘path’: Directory in which disk checkpoint data should be +stored. str, optional, default ‘checkpoints~’.

      • +

    • ‘format’: Disk storage format. Either ‘pickle’, for +data storage using the pickle module, or ‘hdf5’, for data +storage using the h5py library.

      • +

    • ‘blocks’: The total number of blocks. int, +required.

      • +

    • ‘snaps_in_ram’: Maximum number of memory checkpoints. +int, optional, default 0.

      • +

    • ‘snaps_on_disk’: Maximum number of disk checkpoints. +int, optional, default 0.

      • +
    +
    +

    The name ‘multistage’ originates from the corresponding +strategy argument value for the +dolfin_adjoint.solving.adj_checkpointing function in +dolfin-adjoint (see e.g. version 2017.1.0). The parameter names +snaps_in_ram and snaps_on_disk originate from the +corresponding arguments for the +dolfin_adjoint.solving.adj_checkpointing function in +dolfin-adjoint (see e.g. version 2017.1.0).

    +
    • +

  • A callable: A callable returning a CheckpointSchedule. +Options defined by cp_parameters:

    +
    +
      +

    • ‘path’: Directory in which disk checkpoint data should be +stored. str, optional, default ‘checkpoints~’.

      • +

    • ‘format’: Disk storage format. Either ‘pickle’, for +data storage using the pickle module, or ‘hdf5’, for data +storage using the h5py library.

      • +

    • Other parameters are passed as keyword arguments to the +callable.

      • +
    +
    +
    • +
+
+
+ +
+
+configure_tlm(*args, annotate=None, tlm=True)
+

Configure the tangent-linear tree.

+
+
Parameters:
+
    +

  • args – A tuple of (M_i, dM_i) pairs. M_i is a variable +or a Sequence of variables defining a control. dM_i is a +variable or a Sequence of variables defining a derivative +direction. Identifies a node in the tree (and hence identifies a +tangent-linear) corresponding to differentation, in order, with +respect to the each control defined by M_i and with each +direction defined by dM_i.

    • +

  • annotate – If True then enable annotation for the identified +tangent-linear, and enable annotation for all tangent-linears on +which it depends. If False then disable annotation for the +identified tangent-linear, all tangent-linears which depend on it, +and any newly added tangent-linears. Defaults to tlm.

    • +

  • tlm – If True then add (or retain) the identified tangent-linear, +and add all tangent-linears on which it depends. If False then +remove the identified tangent-linear, and remove all +tangent-linears which depend on it.

    • +
+
+
+
+ +
+
+tlm_enabled()
+
+
Returns:
+

Whether derivation and solution of tangent-linear equations +is enabled.

+
+
+
+ +
+
+var_tlm(x, *args)
+

Return a tangent-linear variable.

+
+
Parameters:
+
    +

  • x – A variable whose tangent-linear variable should be returned. +Cannot not be a replacement.

    • +

  • args – Identifies the tangent-linear. See +EquationManager.configure_tlm().

    • +
+
+
Returns:
+

The tangent-linear variable.

+
+
+
+ +
+
+function_tlm(x, *args)
+
+ +
+
+annotation_enabled()
+
+
Returns:
+

Whether recording of equations is enabled.

+
+
+
+ +
+
+start(*, annotate=True, tlm=True)
+

Start recording of equations and derivation and solution of +tangent-linear equations.

+
+
Parameters:
+
    +

  • annotate – Whether recording of equations should be enabled.

    • +

  • tlm – Whether derivation and solution of tangent-linear equations +should be enabled.

    • +
+
+
+
+ +
+
+stop(*, annotate=True, tlm=True)
+

Stop recording of equations and derivation and solution of +tangent-linear equations.

+
+
Parameters:
+
    +

  • annotate – Whether recording of equations should be disabled.

    • +

  • tlm – Whether derivation and solution of tangent-linear equations +should be disabled.

    • +
+
+
Returns:
+

A tuple (annotation_state, tlm_state). +annotation_state is a bool indicating whether recording +of equations was enabled prior to the call to +EquationManager.stop(). tlm_state is a bool +indicating whether derivation and solution of tangent-linear +equations was enabled prior to the call to +EquationManager.stop().

+
+
+
+ +
+
+paused(*, annotate=True, tlm=True)
+

Construct a context manager which can be used to temporarily disable +recording of equations and derivation and solution of tangent-linear +equations.

+
+
Parameters:
+
    +

  • annotate – Whether recording of equations should be temporarily +disabled.

    • +

  • tlm – Whether derivation and solution of tangent-linear equations +should be temporarily disabled.

    • +
+
+
Returns:
+

A context manager which can be used to temporarily disable +recording of equations and derivation and solution of +tangent-linear equations.

+
+
+
+ +
+
+add_initial_condition(x, *, annotate=None)
+

Process an ‘initial condition’ – a variable whose associated value +is needed prior to solving an equation.

+
+
Parameters:
+
    +

  • x – A variable defining the initial condition.

    • +

  • annotate – Whether the initial condition should be recorded.

    • +
+
+
+
+ +
+
+add_equation(eq, *, annotate=None, tlm=None)
+

Process an Equation after it has been solved.

+
+
Parameters:
+
    +

  • eq – The Equation.

    • +

  • annotate – Whether solution of this equation, and any +tangent-linear equations, should be recorded.

    • +

  • tlm – Whether tangent-linear equations should be derived and +solved.

    • +
+
+
+
+ +
+
+drop_references()
+

Drop references to variables which store values, referenced by +objects which have been destroyed, and replace them symbolic +equivalents.

+
+ +
+
+new_block()
+

End the current block of equations, and begin a new block. The +blocks of equations correspond to the ‘steps’ in step-based +checkpointing schedules.

+
+ +
+
+finalize()
+

End the final block of equations. Equations cannot be recorded, and +new tangent-linear equations cannot be derived and solved, after a call +to this method.

+

Called by EquationManager.compute_gradient(), and typically need +not be called manually.

+
+ +
+
+compute_gradient(Js, M, *, callback=None, prune_forward=True, prune_adjoint=True, prune_replay=True, cache_adjoint_degree=None, store_adjoint=False, adj_ics=None)
+

Core adjoint driver method.

+

Compute the derivative of one or more functionals with respect to one +or more controls, using an adjoint approach.

+
+
Parameters:
+
    +

  • Js – A variable or a sequence of variables defining the functionals +to differentiate.

    • +

  • M – A variable or a Sequence of variables defining the +controls. Derivatives with respect to the controls are computed.

    • +

  • callback –

    Diagnostic callback. A callable of the form

    +
    def callback(J_i, n, i, eq, adj_X):
+
    +
    +

    with

    +
    +
      +

    • J_i: An int defining the index of the functional.

      • +

    • n: An int definining the index of the block of +equations.

      • +

    • i: An int defining the index of the considered +equation in block n.

      • +

    • eq: The Equation, equation i in block n.

      • +

    • adj_X: The adjoint solution associated with equation i in +block n for the J_i th functional. None indicates that +the solution is zero or is not computed (due to an activity +analysis). Otherwise a variable if eq has a single solution +component, and a Sequence of variables otherwise.

      • +
    +
    +

    • +

  • prune_forward – Controls the activity analysis. Whether a forward +traversal of the computational graph, tracing variables which +depend on the controls, should be applied.

    • +

  • prune_adjoint – Controls the activity analysis. Whether a reverse +traversal of the computational graph, tracing variables on which +the functionals depend, should be applied.

    • +

  • prune_replay – Controls the activity analysis. Whether an activity +analysis should be applied when solving forward equations during +checkpointing/replay.

    • +

  • cache_adjoint_degree – Adjoint solutions can be cached and reused +across adjoints where the solution is the same – e.g. first order +adjoint solutions associated with the same functional and same +block and equation indices are equal. A value of None indicates +that caching should be applied at all degrees, a value of 0 +indicates that no caching should be applied, and any positive +int indicates that caching should be applied for adjoints +up to and including degree cache_adjoint_degree.

    • +

  • store_adjoint – Whether cached adjoint solutions should be retained +after the call to this method. Can be used to cache and reuse first +order adjoint solutions in multiple calls to this method.

    • +

  • adj_ics – A Mapping. Items are (x, value) where x is a +variable or variable ID identifying a forward variable. The adjoint +variable associated with the final equation solving for x is +initialized to the value stored by the variable value.

    • +
+
+
Returns:
+

The conjugate of the derivatives. The return type depends on +the type of Js and M.

+
+
    +

  • If Js is a variable and M is a variable, returns a variable +storing the conjugate of the derivative.

    • +

  • If Js is a Sequence, and M is a variable, returns +a variable whose \(i\) th component stores the conjugate of +the derivative of the \(i\) th functional.

    • +

  • If Js is a variable and M is a Sequence, returns a +Sequence of variables whose \(j\) th component +stores the conjugate of the derivative with respect to the +\(j\) th control.

    • +

  • If both Js and M are Sequence objects, returns a +Sequence whose \(i\) th component stores the +conjugate of the derivatives of the \(i\) th functional. +Each of these is a Sequence of variables whose +\(j\) th component stores the conjugate of the derivative +with respect to the \(j\) th control.

    • +
+
+

+
+
+
+ +
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/autoapi/tlm_adjoint/verification/index.html b/autoapi/tlm_adjoint/verification/index.html new file mode 100644 index 0000000..7975d9e --- /dev/null +++ b/autoapi/tlm_adjoint/verification/index.html @@ -0,0 +1,348 @@ + + + + + + + tlm_adjoint.verification — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint.verification

+

This module implements Taylor remainder convergence testing using the +approach described in

+
+
    +

  • P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, ‘Automated +derivation of the adjoint of high-level transient finite element +programs’, SIAM Journal on Scientific Computing 35(4), pp. C369–C393, +2013, doi: 10.1137/120873558

    • +
+
+

Specifically for a sufficiently regular functional \(J\), via Taylor’s +theorem we have, for some direction \(\zeta\) and with perturbation +magnitude controlled by \(\varepsilon\),

+
+\[\left| J \left( m + \varepsilon \right) - J \left( m \right) \right| + = O \left( \varepsilon \right),\]
+
+\[\left| J \left( m + \varepsilon \right) - J \left( m \right) + - \varepsilon dJ \left( m; \zeta \right) \right| + = O \left( \varepsilon^2 \right),\]
+

where here \(dJ \left( m; \zeta \right)\) denotes the directional +derivative of \(J\) with respect to \(m\) with direction \(\zeta\). +Here we refer to the quantity appearing on the left-hand-side in the first case +as the ‘uncorrected Taylor remainder magnitude’, and the quantity appearing on +the left-hand-side in the second case as the ‘corrected Taylor remainder +magnitude’

+

A Taylor remainder convergence test considers some direction, and a number of +different values for \(\varepsilon\), and investigates the convergence +rates of the uncorrected and corrected Taylor remainder magnitudes, with the +directional derivative computed using a tangent-linear or adjoint. In a +successful verification the uncorrected Taylor remainder magnitude is observed +to converge to zero at first order, while the corrected Taylor remainder +magnitude is observed to converge to zero at second order.

+

There are a number of ways that a Taylor remainder convergence test can fail, +including:

+
+
    +

  • The computed derivative is incorrect. This is the case that the test is +designed to find, and indicates an error in the tangent-linear or adjoint +calculation.

    • +

  • The considered values of \(\varepsilon\) are too large, and the +asymptotic convergence orders are not observable.

    • +

  • The considered values of \(\varepsilon\) are too small, and iterative +solver tolerances or floating point roundoff prevent the convergence +orders being observable.

    • +

  • The convergence order is higher than expected. For example if the +directional derivative is zero then the uncorrected Taylor remainder +magnitude can converge at higher than first order.

    • +
+
+

In principle higher order derivative calculations can be tested by considering +more terms in the Taylor expansion of the functional. In practice the +corresponding higher order convergence rate can mean that iterative solver +tolerances or floating point roundoff effects are more problematic. Instead, +one can verify the derivative of a derivative, by redefining \(J\) to be a +directional derivative of some other functional \(K\), with the directional +derivative computed using a tangent-linear. A successful verification then once +again corresponds to second order convergence of the corrected Taylor remainder +magnitude.

+

The functions defined in this module log the uncorrected and corrected Taylor +remainder magnitudes, and also log the observed orders computed using a power +law fit between between consecutive pairs of values of \(\varepsilon\). +Logging is performed on a logging module logger, with name +‘tlm_adjoint.verification’ and with severity logging.INFO. The minimum +order computed for the corrected Taylor remainder magnitude is returned.

+

A typical test considers tangent-linears and adjoints up to the relevant order, +e.g. to verify Hessian calculations

+
min_order = taylor_test_tlm(forward, M, 1)
+assert min_order > 1.99
+
+min_order = taylor_test_tlm_adjoint(forward, M, 1)
+assert min_order > 1.99
+
+min_order = taylor_test_tlm_adjoint(forward, M, 2)
+assert min_order > 1.99
+
+
+
+

Module Contents

+
+
+tlm_adjoint.verification.taylor_test(forward, M, J_val, *, dJ=None, ddJ=None, seed=0.01, dM=None, M0=None, size=5)
+

Perform a Taylor remainder convergence test. Aims for similar behaviour +to the taylor_test function in dolfin-adjoint 2017.1.0.

+

Uncorrected and corrected Taylor remainder magnitudes are computed by +repeatedly re-running the forward and evaluating the functional. The +perturbation direction \(\zeta\) is defined by the dM argument. +\(\varepsilon\) is set equal to

+
+\[\varepsilon = 2^{-p} \eta \max \left( 1, + \left\| m \right\|_{l_\infty} \right) + \quad \text{ for } p \in \left\{ 0, \ldots, P - 1 \right\},\]
+

where the norm appearing here is defined to be the \(l_\infty\) norm of +the control value degree of freedom vector. The argument seed sets the +value of \(\eta\), and the argument size sets the value of \(P\).

+
+
Parameters:
+
    +

  • forward – A callable which accepts one or more variable arguments, and +which returns a variable defining the forward functional \(J\). +Corresponds to the J argument in the dolfin-adjoint taylor_test +function.

    • +

  • M – A variable or a Sequence of variables defining the control +\(m\). Corresponds to the m argument in the dolfin-adjoint +taylor_test function.

    • +

  • J_val – A scalar defining the value of the functional \(J\) for +control value defined by M0. Corresponds to the Jm argument in the +dolfin-adjoint taylor_test function.

    • +

  • dJ – A variable or a Sequence of variables defining a value +for the derivative of the functional with respect to the control. +Required if ddJ is not supplied. Corresponds to the dJdm argument +in the dolfin-adjoint taylor_test function.

    • +

  • ddJ – A Hessian used to compute the Hessian action on the +considered perturbation direction. If supplied then a higher order +corrected Taylor remainder magnitude is computed. If dJ is not +supplied, also computes the first order directional derivative. +Corresponds to the HJm argument in the dolfin-adjoint taylor_test +function.

    • +

  • seed – Defines the value of \(\eta\). Controls the magnitude of the +perturbation. Corresponds to the seed argument in the dolfin-adjoint +taylor_test function.

    • +

  • dM – Defines the perturbation direction \(\zeta\). A direction with +degrees of freedom vector real and (in the complex case) complex parts +set by numpy.random.random() is used if not supplied. Corresponds +to the perturbation_direction argument in the dolfin-adjoint +taylor_test function.

    • +

  • M0 – Defines the value of the control at which the functional and +derivatives are evaluated. M is used if not supplied. Corresponds to +the value argument in the dolfin-adjoint taylor_test function.

    • +

  • size – The number of values of \(\varepsilon\) to consider. +Corresponds to the size argument in the dolfin-adjoint taylor_test +function.

    • +
+
+
Returns:
+

The minimum order observed, via a power law fit between +consecutive pairs of values of \(\varepsilon\), in the calculations +for the corrected Taylor remainder magnitude. In a successful +verification this should be close to 2 if ddJ is not supplied, and +close to 3 if ddJ is supplied.

+
+
+
+ +
+
+tlm_adjoint.verification.taylor_test_tlm(forward, M, tlm_order, *, seed=0.01, dMs=None, size=5, manager=None)
+

Perform a Taylor remainder convergence test for a functional \(J\) +defined to the (tlm_order - 1) th derivative of some functional +\(K\). The tlm_order th derivative of \(K\), appearing in the +corrected Taylor remainder magnitude, is computed using a tlm_order th +order tangent-linear.

+
+
Parameters:
+
    +

  • forward – A callable which accepts one or more variable arguments, and +which returns a variable defining the forward functional \(K\).

    • +

  • M – A variable or a Sequence of variables defining the control +\(m\) and its value.

    • +

  • tlm_order – An int defining the tangent-linear order to +test.

    • +

  • seed – Controls the perturbation magnitude. See taylor_test().

    • +

  • dMs – A Sequence of length tlm_order whose elements are each +a variable or a Sequence of variables. The functional +\(J\) appearing in the definition of the Taylor remainder +magnitudes is defined to be a (tlm_adjoint - 1) th derivative, +defined by successively taking the derivative of \(K\) with respect +to the control and with directions defined by the dM[:-1] (with the +directions considered in order). The perturbation direction +\(\zeta\) is defined by dM[-1] – see taylor_test(). +Directions with degrees of freedom vector real and (in the complex +case) complex parts set by numpy.random.random() are used if not +supplied.

    • +

  • size – The number of values of \(\varepsilon\) to consider. See +taylor_test().

    • +

  • manager – An EquationManager used to create an internal +manager via EquationManager.new(). manager() is used if not +supplied.

    • +
+
+
Returns:
+

The minimum order observed, via a power law fit between +consecutive pairs of values of \(\varepsilon\), in the calculations +for the corrected Taylor remainder magnitude. In a successful +verification this should be close to 2.

+
+
+
+ +
+
+tlm_adjoint.verification.taylor_test_tlm_adjoint(forward, M, adjoint_order, *, seed=0.01, dMs=None, size=5, manager=None)
+

Perform a Taylor remainder convergence test for a functional \(J\) +defined to the (adjoint_order - 1) th derivative of some functional +\(K\). The adjoint_order th derivative of \(K\), appearing in the +corrected Taylor remainder magnitude, is computed using an adjoint +associated with an (adjoint_order - 1) th order tangent-linear.

+
+
Parameters:
+
    +

  • forward – A callable which accepts one or more variable arguments, and +which returns a variable defining the forward functional \(K\).

    • +

  • M – A variable or a Sequence of variables defining the control +\(m\) and its value.

    • +

  • adjoint_order – An int defining the adjoint order to test.

    • +

  • seed – Controls the perturbation magnitude. See taylor_test().

    • +

  • dMs – A Sequence of length adjoint_order whose elements are +each a variable or a Sequence of variables. The functional +\(J\) appearing in the definition of the Taylor remainder +magnitudes is defined to be a (adjoint_order - 1) th derivative, +defined by successively taking the derivative of \(K\) with respect +to the control and with directions defined by the dM[:-1] (with the +directions considered in order). The perturbation direction +\(\zeta\) is defined by dM[-1] – see taylor_test(). +Directions with degrees of freedom vector real and (in the complex +case) complex parts set by numpy.random.random() are used if not +supplied.

    • +

  • size – The number of values of \(\varepsilon\) to consider. See +taylor_test().

    • +

  • manager – An EquationManager used to create an internal +manager via EquationManager.new(). manager() is used if not +supplied.

    • +
+
+
Returns:
+

The minimum order observed, via a power law fit between +consecutive pairs of values of \(\varepsilon\), in the calculations +for the corrected Taylor remainder magnitude. In a successful +verification this should be close to 2.

+
+
+
+ +
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/dependencies.html b/dependencies.html new file mode 100644 index 0000000..80eb75a --- /dev/null +++ b/dependencies.html @@ -0,0 +1,175 @@ + + + + + + + Dependencies — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Dependencies

+
+

Required

+

tlm_adjoint requires:

+
+
+
+
+
+

Backend dependencies

+

With the FEniCS backend tlm_adjoint requires:

+
+
+

With the Firedrake backend tlm_adjoint requires:

+
+
+
+
+

Optional

+

Some features of tlm_adjoint require:

+
+
+

While not required, if available some features of tlm_adjoint use:

+
+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/examples/0_getting_started.html b/examples/0_getting_started.html new file mode 100644 index 0000000..d8fea76 --- /dev/null +++ b/examples/0_getting_started.html @@ -0,0 +1,690 @@ + + + + + + + Getting started with tlm_adjoint — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Getting started with tlm_adjoint

+

This notebook introduces derivative calculations using tlm_adjoint.

+

tlm_adjoint is primarily intended for first derivative calculations using the adjoint method, and Hessian information calculations using the adjoint method applied to tangent-linear and forward calculations. However the approach used by tlm_adjoint generalizes to higher order.

+

The approach used by tlm_adjoint for higher order differentiation is described in:

+
    +

  • James R. Maddison, Daniel N. Goldberg, and Benjamin D. Goddard, ‘Automated calculation of higher order partial differential equation constrained derivative information’, SIAM Journal on Scientific Computing, 41(5), pp. C417–C445, 2019, doi: 10.1137/18M1209465

    • +
+
+

A floating point example

+

We consider a simple floating point calculation:

+
+
[1]:
+
+
+
import numpy as np
+
+x = 1.0
+y = 0.25 * np.pi
+z = x * np.sin(y * np.exp(y))
+
+
+
+

tlm_adjoint is designed for high-level algorithmic differentiation, and not this type of low-level floating point calculation. However it can still process simple floating point calculations, so to introduce the key ideas we do that here. We consider differentiating z with respect to x and y – being precise, we mean computing the derivative of the function used to compute z with respect to the variables defined by x and y.

+
+
+

Adding tlm_adjoint

+

We first modify the code so that tlm_adjoint processes the calculations:

+
+
[2]:
+
+
+
from tlm_adjoint import *
+
+import numpy as np
+
+reset_manager()
+
+
+def forward(x, y):
+    z = x * np.sin(y * np.exp(y))
+    return z
+
+
+x = Float(1.0, name="x")
+y = Float(0.25 * np.pi, name="y")
+
+start_manager()
+z = forward(x, y)
+stop_manager()
+
+
+
+
+
[2]:
+
+
+
+
+(True, True)
+
+
+

The key changes here are:

+
    +

  • To import tlm_adjoint.

    • +

  • Controlling the ‘manager’ – an object tlm_adjoint uses to process equations. The manager is first reset using reset_manager. This clears any previous processing, and disables the manager. start_manager and stop_manager are then used to enable the manager just when it is needed.

    • +

  • Defining x and y to be of type Float. Calculations involving x and y are recorded by the manager. The result of the calculations – here z – will have the same type, and we can access its value with float(z).

    • +
+

Let’s display the information recorded by tlm_adjoint:

+
+
[3]:
+
+
+
manager_info()
+
+
+
+
+
+
+
+
+Equation manager status:
+Annotation state: AnnotationState.STOPPED
+Tangent-linear state: TangentLinearState.STOPPED
+Equations:
+  Block 0
+    Equation 0, FloatEquation solving for f_2 (id 2)
+      Dependency 0, f_2 (id 2), linear
+      Dependency 1, y (id 1), non-linear
+    Equation 1, FloatEquation solving for f_4 (id 4)
+      Dependency 0, f_4 (id 4), linear
+      Dependency 1, y (id 1), non-linear
+      Dependency 2, f_2 (id 2), non-linear
+    Equation 2, FloatEquation solving for f_6 (id 6)
+      Dependency 0, f_6 (id 6), linear
+      Dependency 1, f_4 (id 4), non-linear
+    Equation 3, FloatEquation solving for f_8 (id 8)
+      Dependency 0, f_8 (id 8), linear
+      Dependency 1, x (id 0), non-linear
+      Dependency 2, f_6 (id 6), non-linear
+Storage:
+  Storing initial conditions: yes
+  Storing equation non-linear dependencies: yes
+  Initial conditions stored: 2
+  Initial conditions referenced: 0
+Checkpointing:
+  Method: memory
+
+
+

The key feature here is that there are four FloatEquation records, corresponding to the four floating point calculations – evaluation using np.exp and np.sin, and two multiplications.

+
+
+

Computing derivatives using an adjoint

+

The compute_gradient function can be used to differentiate z with respect to x and y using the adjoint method:

+
+
[4]:
+
+
+
dz_dx, dz_dy = compute_gradient(z, (x, y))
+
+print(f"{float(dz_dx)=}")
+print(f"{float(dz_dy)=}")
+
+
+
+
+
+
+
+
+float(dz_dx)=0.9885002159138745
+float(dz_dy)=-0.592156957782821
+
+
+

For a simple check of the result, we can compare with the result from finite differencing, here using second order centered finite differencing:

+
+
[5]:
+
+
+
def dJ_dm(J, m, *, eps=1.0e-7):
+    return (J(m + eps) - J(m - eps)) / (2.0 * eps)
+
+
+print(f"dz_dx approximation = {dJ_dm(lambda x: forward(x, float(y)), float(x))}")
+print(f"dz_dy approximation = {dJ_dm(lambda y: forward(float(x), y), float(y))}")
+
+
+
+
+
+
+
+
+dz_dx approximation = 0.9885002155707312
+dz_dy approximation = -0.5921569579125929
+
+
+
+
+

Computing derivatives using a tangent-linear

+

A tangent-linear computes directional derivatives with respect to a given control and with a given direction.

+

Here we consider the forward to be a function of x and y, computing a value of z, denoted \(z \left( x, y \right)\). We consider the control \(m = \left( x, y \right)^T\), and a direction \(\zeta = \left( 2, 3 \right)^T\). We can then use a tangent-linear to compute the directional derivative

+
+\[\frac{dz}{dm} \zeta = 2 \frac{dz}{dx} + 3 \frac{dz}{dy},\]
+

where vector derivatives are notated using row vectors.

+

In most cases tlm_adjoint needs to be told what tangent-linear calculations to perform ahead of the forward calculations. configure_tlm provides this information to tlm_adjoint:

+
+
[6]:
+
+
+
from tlm_adjoint import *
+
+import numpy as np
+
+reset_manager()
+
+
+def forward(x, y):
+    z = x * np.sin(y * np.exp(y))
+    return z
+
+
+x = Float(1.0, name="x")
+y = Float(0.25 * np.pi, name="y")
+
+m = (x, y)
+zeta = (Float(2.0, name="zeta_x"), Float(3.0, name="zeta_y"))
+configure_tlm((m, zeta))
+
+start_manager()
+z = forward(x, y)
+stop_manager()
+
+dz_dm_zeta = var_tlm(z, (m, zeta))
+
+print(f"{float(dz_dm_zeta)=}")
+
+
+
+
+
+
+
+
+float(dz_dm_zeta)=0.2005295584792861
+
+
+

There are three new changes:

+
    +

  • The control \(m\) and direction \(\zeta\) are defined using m and zeta respectively.

    • +

  • Tangent-linear calculations are configured before the forward calculations are performed, using configure_tlm. Note that here, for this first derivative calculation, the argument is a single control-direction pair.

    • +

  • We access the tangent-linear variable, containing the value of \(\left( dz/dm \right) \zeta\), using var_tlm, and using the same control-direction pair.

    • +
+

In fact more has happened here – if we now display the information recorded by tlm_adjoint:

+
+
[7]:
+
+
+
manager_info()
+
+
+
+
+
+
+
+
+Equation manager status:
+Annotation state: AnnotationState.STOPPED
+Tangent-linear state: TangentLinearState.STOPPED
+Equations:
+  Block 0
+    Equation 0, FloatEquation solving for f_31 (id 31)
+      Dependency 0, f_31 (id 31), linear
+      Dependency 1, y (id 28), non-linear
+    Equation 1, FloatEquation solving for f_31_tlm((x,y),(zeta_x,zeta_y)) (id 33)
+      Dependency 0, f_31_tlm((x,y),(zeta_x,zeta_y)) (id 33), linear
+      Dependency 1, y (id 28), non-linear
+      Dependency 2, zeta_y (id 30), non-linear
+    Equation 2, FloatEquation solving for f_35 (id 35)
+      Dependency 0, f_35 (id 35), linear
+      Dependency 1, y (id 28), non-linear
+      Dependency 2, f_31 (id 31), non-linear
+    Equation 3, FloatEquation solving for f_35_tlm((x,y),(zeta_x,zeta_y)) (id 37)
+      Dependency 0, f_35_tlm((x,y),(zeta_x,zeta_y)) (id 37), linear
+      Dependency 1, y (id 28), non-linear
+      Dependency 2, zeta_y (id 30), non-linear
+      Dependency 3, f_31 (id 31), non-linear
+      Dependency 4, f_31_tlm((x,y),(zeta_x,zeta_y)) (id 33), non-linear
+    Equation 4, FloatEquation solving for f_39 (id 39)
+      Dependency 0, f_39 (id 39), linear
+      Dependency 1, f_35 (id 35), non-linear
+    Equation 5, FloatEquation solving for f_39_tlm((x,y),(zeta_x,zeta_y)) (id 41)
+      Dependency 0, f_39_tlm((x,y),(zeta_x,zeta_y)) (id 41), linear
+      Dependency 1, f_35 (id 35), non-linear
+      Dependency 2, f_35_tlm((x,y),(zeta_x,zeta_y)) (id 37), non-linear
+    Equation 6, FloatEquation solving for f_43 (id 43)
+      Dependency 0, f_43 (id 43), linear
+      Dependency 1, x (id 27), non-linear
+      Dependency 2, f_39 (id 39), non-linear
+    Equation 7, FloatEquation solving for f_43_tlm((x,y),(zeta_x,zeta_y)) (id 46)
+      Dependency 0, f_43_tlm((x,y),(zeta_x,zeta_y)) (id 46), linear
+      Dependency 1, x (id 27), non-linear
+      Dependency 2, zeta_x (id 29), non-linear
+      Dependency 3, f_39 (id 39), non-linear
+      Dependency 4, f_39_tlm((x,y),(zeta_x,zeta_y)) (id 41), non-linear
+Storage:
+  Storing initial conditions: yes
+  Storing equation non-linear dependencies: yes
+  Initial conditions stored: 4
+  Initial conditions referenced: 0
+Checkpointing:
+  Method: memory
+
+
+

we now see that there are eight FloatEquation records – the original four, and four new ones. The extra ones correspond to the tangent-linear calculations. tlm_adjoint has recorded the original forward calculations, and also recorded the tangent-linear calculations.

+
+
+

Computing second derivatives using an adjoint of a tangent-linear

+

Since tlm_adjoint has recorded both forward and tangent-linear calculations, we can now compute second derivative information using an adjoint associated with a tangent-linear. Specifically we can compute a Hessian action on \(\zeta\),

+
+\[H \zeta = \frac{d}{dm} \left( \frac{dz}{dm} \zeta \right)^T.\]
+

The inner directional derivative appearing here is computed using the tangent-linear method, and the outer derivative is computed by applying the adjoint method to the tangent-linear and forward calculations. In code we simply use compute_gradient to compute the derivative of the tangent-linear result:

+
+
[8]:
+
+
+
from tlm_adjoint import *
+
+import numpy as np
+
+reset_manager()
+
+
+def forward(x, y):
+    z = x * np.sin(y * np.exp(y))
+    return z
+
+
+x = Float(1.0, name="x")
+y = Float(0.25 * np.pi, name="y")
+
+m = (x, y)
+zeta = (Float(2.0, name="zeta_x"), Float(3.0, name="zeta_y"))
+configure_tlm((m, zeta))
+
+start_manager()
+z = forward(x, y)
+stop_manager()
+
+dz_dm_zeta = var_tlm(z, (m, zeta))
+
+print(f"{float(dz_dm_zeta)=}")
+
+d2z_dm_zeta_dx, d2z_dm_zeta_dy = compute_gradient(dz_dm_zeta, m)
+
+print(f"{float(d2z_dm_zeta_dx)=}")
+print(f"{float(d2z_dm_zeta_dy)=}")
+
+
+
+
+
+
+
+
+float(dz_dm_zeta)=0.2005295584792861
+float(d2z_dm_zeta_dx)=-1.7764708733484629
+float(d2z_dm_zeta_dy)=-49.4290736694207
+
+
+
+
+

Computing second derivatives using a tangent-linear of a tangent-linear

+

We can also compute second derivative information using a tangent-linear associated with a tangent-linear. For example if we define \(e_1 = \left( 1, 0 \right)^T\), then we can find the first component of the previously computed Hessian action on \(\zeta\) via

+
+\[e_1^T H \zeta = \left[ \frac{d}{dm} \left( \frac{dz}{dm} \zeta \right) \right] e_1.\]
+

That is, here we now want to compute a directional derivative of a directional derivative, and we compute this using a tangent-linear associated with the previous tangent-linear and forward calculations.

+

tlm_adjoint handles this case by supplying more arguments to configure_tlm:

+
+
[9]:
+
+
+
from tlm_adjoint import *
+
+import numpy as np
+
+reset_manager()
+
+
+def forward(x, y):
+    z = x * np.sin(y * np.exp(y))
+    return z
+
+
+x = Float(1.0, name="x")
+y = Float(0.25 * np.pi, name="y")
+
+m = (x, y)
+zeta = (Float(2.0, name="zeta_x"), Float(3.0, name="zeta_y"))
+e_1 = (Float(1.0, name="e_1_x"), Float(0.0, name="e_1_y"))
+configure_tlm((m, zeta), (m, e_1))
+
+start_manager()
+z = forward(x, y)
+stop_manager()
+
+dz_dm_zeta = var_tlm(z, (m, zeta))
+dz_dx = var_tlm(z, (m, e_1))
+d2z_dm_zeta_dx = var_tlm(z, (m, zeta), (m, e_1))
+
+print(f"{float(dz_dm_zeta)=}")
+print(f"{float(dz_dx)=}")
+print(f"{float(d2z_dm_zeta_dx)=}")
+
+
+
+
+
+
+
+
+float(dz_dm_zeta)=0.2005295584792861
+float(dz_dx)=0.9885002159138745
+float(d2z_dm_zeta_dx)=-1.7764708733484629
+
+
+

The first control-direction pair supplied to configure_tlm indicates that we seek to compute directional derivatives of the forward with respect to the control defined by m with direction defined by zeta. The second control-direction pair indicates that we seek to compute directional deriatives of these directional derivatives, with respect to the control defined by m and with direction defined by e_1. As a side-effect we find that we also compute the directional +derivatives of the forward with respect to the control defined by m with direction defined by e_1.

+

We then access the tangent-linear variables using var_tlm, supplying two control-variable pairs to access a second order tangent-linear variable.

+

As before, tlm_adjoint has not just performed the tangent-linear calculations – if we display the information recorded by tlm_adjoint:

+
+
[10]:
+
+
+
manager_info()
+
+
+
+
+
+
+
+
+Equation manager status:
+Annotation state: AnnotationState.STOPPED
+Tangent-linear state: TangentLinearState.STOPPED
+Equations:
+  Block 0
+    Equation 0, FloatEquation solving for f_103 (id 103)
+      Dependency 0, f_103 (id 103), linear
+      Dependency 1, y (id 98), non-linear
+    Equation 1, FloatEquation solving for f_103_tlm((x,y),(zeta_x,zeta_y)) (id 105)
+      Dependency 0, f_103_tlm((x,y),(zeta_x,zeta_y)) (id 105), linear
+      Dependency 1, y (id 98), non-linear
+      Dependency 2, zeta_y (id 100), non-linear
+    Equation 2, FloatEquation solving for f_103_tlm((x,y),(e_1_x,e_1_y)) (id 107)
+      Dependency 0, f_103_tlm((x,y),(e_1_x,e_1_y)) (id 107), linear
+      Dependency 1, y (id 98), non-linear
+      Dependency 2, e_1_y (id 102), non-linear
+    Equation 3, FloatEquation solving for f_103_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 110)
+      Dependency 0, f_103_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 110), linear
+      Dependency 1, y (id 98), non-linear
+      Dependency 2, zeta_y (id 100), non-linear
+      Dependency 3, e_1_y (id 102), non-linear
+      Dependency 4, zeta_y_tlm((x,y),(e_1_x,e_1_y)) (id 109), non-linear
+    Equation 4, FloatEquation solving for f_112 (id 112)
+      Dependency 0, f_112 (id 112), linear
+      Dependency 1, y (id 98), non-linear
+      Dependency 2, f_103 (id 103), non-linear
+    Equation 5, FloatEquation solving for f_112_tlm((x,y),(zeta_x,zeta_y)) (id 114)
+      Dependency 0, f_112_tlm((x,y),(zeta_x,zeta_y)) (id 114), linear
+      Dependency 1, y (id 98), non-linear
+      Dependency 2, zeta_y (id 100), non-linear
+      Dependency 3, f_103 (id 103), non-linear
+      Dependency 4, f_103_tlm((x,y),(zeta_x,zeta_y)) (id 105), non-linear
+    Equation 6, FloatEquation solving for f_112_tlm((x,y),(e_1_x,e_1_y)) (id 116)
+      Dependency 0, f_112_tlm((x,y),(e_1_x,e_1_y)) (id 116), linear
+      Dependency 1, y (id 98), non-linear
+      Dependency 2, e_1_y (id 102), non-linear
+      Dependency 3, f_103 (id 103), non-linear
+      Dependency 4, f_103_tlm((x,y),(e_1_x,e_1_y)) (id 107), non-linear
+    Equation 7, FloatEquation solving for f_112_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 118)
+      Dependency 0, f_112_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 118), linear
+      Dependency 1, y (id 98), non-linear
+      Dependency 2, zeta_y (id 100), non-linear
+      Dependency 3, e_1_y (id 102), non-linear
+      Dependency 4, f_103 (id 103), non-linear
+      Dependency 5, f_103_tlm((x,y),(zeta_x,zeta_y)) (id 105), non-linear
+      Dependency 6, f_103_tlm((x,y),(e_1_x,e_1_y)) (id 107), non-linear
+      Dependency 7, zeta_y_tlm((x,y),(e_1_x,e_1_y)) (id 109), non-linear
+      Dependency 8, f_103_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 110), non-linear
+    Equation 8, FloatEquation solving for f_120 (id 120)
+      Dependency 0, f_120 (id 120), linear
+      Dependency 1, f_112 (id 112), non-linear
+    Equation 9, FloatEquation solving for f_120_tlm((x,y),(zeta_x,zeta_y)) (id 122)
+      Dependency 0, f_120_tlm((x,y),(zeta_x,zeta_y)) (id 122), linear
+      Dependency 1, f_112 (id 112), non-linear
+      Dependency 2, f_112_tlm((x,y),(zeta_x,zeta_y)) (id 114), non-linear
+    Equation 10, FloatEquation solving for f_120_tlm((x,y),(e_1_x,e_1_y)) (id 124)
+      Dependency 0, f_120_tlm((x,y),(e_1_x,e_1_y)) (id 124), linear
+      Dependency 1, f_112 (id 112), non-linear
+      Dependency 2, f_112_tlm((x,y),(e_1_x,e_1_y)) (id 116), non-linear
+    Equation 11, FloatEquation solving for f_120_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 126)
+      Dependency 0, f_120_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 126), linear
+      Dependency 1, f_112 (id 112), non-linear
+      Dependency 2, f_112_tlm((x,y),(zeta_x,zeta_y)) (id 114), non-linear
+      Dependency 3, f_112_tlm((x,y),(e_1_x,e_1_y)) (id 116), non-linear
+      Dependency 4, f_112_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 118), non-linear
+    Equation 12, FloatEquation solving for f_128 (id 128)
+      Dependency 0, f_128 (id 128), linear
+      Dependency 1, x (id 97), non-linear
+      Dependency 2, f_120 (id 120), non-linear
+    Equation 13, FloatEquation solving for f_128_tlm((x,y),(zeta_x,zeta_y)) (id 131)
+      Dependency 0, f_128_tlm((x,y),(zeta_x,zeta_y)) (id 131), linear
+      Dependency 1, x (id 97), non-linear
+      Dependency 2, zeta_x (id 99), non-linear
+      Dependency 3, f_120 (id 120), non-linear
+      Dependency 4, f_120_tlm((x,y),(zeta_x,zeta_y)) (id 122), non-linear
+    Equation 14, FloatEquation solving for f_128_tlm((x,y),(e_1_x,e_1_y)) (id 134)
+      Dependency 0, f_128_tlm((x,y),(e_1_x,e_1_y)) (id 134), linear
+      Dependency 1, x (id 97), non-linear
+      Dependency 2, e_1_x (id 101), non-linear
+      Dependency 3, f_120 (id 120), non-linear
+      Dependency 4, f_120_tlm((x,y),(e_1_x,e_1_y)) (id 124), non-linear
+    Equation 15, FloatEquation solving for f_128_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 138)
+      Dependency 0, f_128_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 138), linear
+      Dependency 1, x (id 97), non-linear
+      Dependency 2, zeta_x (id 99), non-linear
+      Dependency 3, e_1_x (id 101), non-linear
+      Dependency 4, f_120 (id 120), non-linear
+      Dependency 5, f_120_tlm((x,y),(zeta_x,zeta_y)) (id 122), non-linear
+      Dependency 6, f_120_tlm((x,y),(e_1_x,e_1_y)) (id 124), non-linear
+      Dependency 7, f_120_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 126), non-linear
+      Dependency 8, zeta_x_tlm((x,y),(e_1_x,e_1_y)) (id 137), non-linear
+Storage:
+  Storing initial conditions: yes
+  Storing equation non-linear dependencies: yes
+  Initial conditions stored: 8
+  Initial conditions referenced: 0
+Checkpointing:
+  Method: memory
+
+
+

we now find that there are sixteen FloatEquation records, constituting the forward and all tangent-linear calculations.

+
+
+

Computing third derivatives using an adjoint of a tangent-linear of a tangent-linear

+

We can now compute the derivative of the tangent-linear-computed second derivative by simply handing the second order tangent-linear variable to compute_gradient. This applies the adjoint method to the higher order tangent-linear calculations and the forward calculations, computing

+
+\[\frac{d}{dm} \left( e_1^T H \zeta \right) = \frac{d}{dm} \left[ \left[ \frac{d}{dm} \left( \frac{dz}{dm} \zeta \right) \right] e_1 \right].\]
+
+
[11]:
+
+
+
from tlm_adjoint import *
+
+import numpy as np
+
+reset_manager()
+
+
+def forward(x, y):
+    z = x * np.sin(y * np.exp(y))
+    return z
+
+
+x = Float(1.0, name="x")
+y = Float(0.25 * np.pi, name="y")
+
+m = (x, y)
+zeta = (Float(2.0, name="zeta_x"), Float(3.0, name="zeta_y"))
+e_1 = (Float(1.0, name="e_1_x"), Float(0.0, name="e_1_y"))
+configure_tlm((m, zeta), (m, e_1))
+
+start_manager()
+z = forward(x, y)
+stop_manager()
+
+dz_dm_zeta = var_tlm(z, (m, zeta))
+dz_dx = var_tlm(z, (m, e_1))
+d2z_dm_zeta_dx = var_tlm(z, (m, zeta), (m, e_1))
+
+print(f"{float(dz_dm_zeta)=}")
+print(f"{float(dz_dx)=}")
+print(f"{float(d2z_dm_zeta_dx)=}")
+
+d3z_dm_zeta_dx_dx, d3z_dm_zeta_dx_dy = compute_gradient(d2z_dm_zeta_dx, m)
+
+print(f"{float(d3z_dm_zeta_dx_dx)=}")
+print(f"{float(d3z_dm_zeta_dx_dy)=}")
+
+
+
+
+
+
+
+
+float(dz_dm_zeta)=0.2005295584792861
+float(dz_dx)=0.9885002159138745
+float(d2z_dm_zeta_dx)=-1.7764708733484629
+float(d3z_dm_zeta_dx_dx)=0.0
+float(d3z_dm_zeta_dx_dy)=-48.244759753855064
+
+
+
+
+

Higher order

+

The approach now generalizes.

+
    +

  • Supplying further arguments to configure_tlm indicates directional derivatives of directional derivatives, defining a tangent-linear calculation of increasingly high order.

    • +

  • Supplying these arguments to var_tlm accesses the higher-order tangent-linear variables.

    • +

  • These higher order tangent-linear variables can be handed to compute_gradient to compute derivatives of the higher order derivatives using the adjoint method.

    • +
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/examples/0_getting_started.ipynb b/examples/0_getting_started.ipynb new file mode 100644 index 0000000..e7bea9d --- /dev/null +++ b/examples/0_getting_started.ipynb @@ -0,0 +1,805 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "40c3ebad", + "metadata": {}, + "source": [ + "# Getting started with tlm_adjoint\n", + "\n", + "This notebook introduces derivative calculations using tlm_adjoint.\n", + "\n", + "tlm_adjoint is primarily intended for first derivative calculations using the adjoint method, and Hessian information calculations using the adjoint method applied to tangent-linear and forward calculations. However the approach used by tlm_adjoint generalizes to higher order.\n", + "\n", + "The approach used by tlm_adjoint for higher order differentiation is described in:\n", + "\n", + "- James R. Maddison, Daniel N. Goldberg, and Benjamin D. Goddard, 'Automated calculation of higher order partial differential equation constrained derivative information', SIAM Journal on Scientific Computing, 41(5), pp. C417–C445, 2019, doi: 10.1137/18M1209465\n", + "\n", + "## A floating point example\n", + "\n", + "We consider a simple floating point calculation:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "0c79382a", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:03.644963Z", + "iopub.status.busy": "2023-12-20T16:44:03.644217Z", + "iopub.status.idle": "2023-12-20T16:44:03.720921Z", + "shell.execute_reply": "2023-12-20T16:44:03.720262Z" + } + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "\n", + "x = 1.0\n", + "y = 0.25 * np.pi\n", + "z = x * np.sin(y * np.exp(y))" + ] + }, + { + "cell_type": "markdown", + "id": "e82699c6", + "metadata": {}, + "source": [ + "tlm_adjoint is designed for high-level algorithmic differentiation, and not this type of low-level floating point calculation. However it *can* still process simple floating point calculations, so to introduce the key ideas we do that here. We consider differentiating `z` with respect to `x` and `y` – being precise, we mean computing the derivative of the function used to compute `z` with respect to the variables defined by `x` and `y`.\n", + "\n", + "## Adding tlm_adjoint\n", + "\n", + "We first modify the code so that tlm_adjoint processes the calculations:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "dfb3955a", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:03.723761Z", + "iopub.status.busy": "2023-12-20T16:44:03.723523Z", + "iopub.status.idle": "2023-12-20T16:44:06.281442Z", + "shell.execute_reply": "2023-12-20T16:44:06.280782Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(True, True)" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(x, y):\n", + " z = x * np.sin(y * np.exp(y))\n", + " return z\n", + "\n", + "\n", + "x = Float(1.0, name=\"x\")\n", + "y = Float(0.25 * np.pi, name=\"y\")\n", + "\n", + "start_manager()\n", + "z = forward(x, y)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "4bb98432", + "metadata": {}, + "source": [ + "The key changes here are:\n", + "\n", + "- To import tlm_adjoint.\n", + "- Controlling the 'manager' – an object tlm_adjoint uses to process equations. The manager is first reset using `reset_manager`. This clears any previous processing, and disables the manager. `start_manager` and `stop_manager` are then used to enable the manager just when it is needed.\n", + "- Defining `x` and `y` to be of type `Float`. Calculations involving `x` and `y` are recorded by the manager. The result of the calculations – here `z` – will have the same type, and we can access its value with `float(z)`.\n", + "\n", + "Let's display the information recorded by tlm_adjoint:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "ba11e403", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:06.283871Z", + "iopub.status.busy": "2023-12-20T16:44:06.283625Z", + "iopub.status.idle": "2023-12-20T16:44:06.287453Z", + "shell.execute_reply": "2023-12-20T16:44:06.286926Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Equation manager status:\n", + "Annotation state: AnnotationState.STOPPED\n", + "Tangent-linear state: TangentLinearState.STOPPED\n", + "Equations:\n", + " Block 0\n", + " Equation 0, FloatEquation solving for f_2 (id 2)\n", + " Dependency 0, f_2 (id 2), linear\n", + " Dependency 1, y (id 1), non-linear\n", + " Equation 1, FloatEquation solving for f_4 (id 4)\n", + " Dependency 0, f_4 (id 4), linear\n", + " Dependency 1, y (id 1), non-linear\n", + " Dependency 2, f_2 (id 2), non-linear\n", + " Equation 2, FloatEquation solving for f_6 (id 6)\n", + " Dependency 0, f_6 (id 6), linear\n", + " Dependency 1, f_4 (id 4), non-linear\n", + " Equation 3, FloatEquation solving for f_8 (id 8)\n", + " Dependency 0, f_8 (id 8), linear\n", + " Dependency 1, x (id 0), non-linear\n", + " Dependency 2, f_6 (id 6), non-linear\n", + "Storage:\n", + " Storing initial conditions: yes\n", + " Storing equation non-linear dependencies: yes\n", + " Initial conditions stored: 2\n", + " Initial conditions referenced: 0\n", + "Checkpointing:\n", + " Method: memory\n" + ] + } + ], + "source": [ + "manager_info()" + ] + }, + { + "cell_type": "markdown", + "id": "a0138110", + "metadata": {}, + "source": [ + "The key feature here is that there are four `FloatEquation` records, corresponding to the four floating point calculations – evaluation using `np.exp` and `np.sin`, and two multiplications.\n", + "\n", + "## Computing derivatives using an adjoint\n", + "\n", + "The `compute_gradient` function can be used to differentiate `z` with respect to `x` and `y` using the adjoint method:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "66f6f440", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:06.289907Z", + "iopub.status.busy": "2023-12-20T16:44:06.289707Z", + "iopub.status.idle": "2023-12-20T16:44:06.304876Z", + "shell.execute_reply": "2023-12-20T16:44:06.304232Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "float(dz_dx)=0.9885002159138745\n", + "float(dz_dy)=-0.592156957782821\n" + ] + } + ], + "source": [ + "dz_dx, dz_dy = compute_gradient(z, (x, y))\n", + "\n", + "print(f\"{float(dz_dx)=}\")\n", + "print(f\"{float(dz_dy)=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "1796fdfd", + "metadata": {}, + "source": [ + "For a simple check of the result, we can compare with the result from finite differencing, here using second order centered finite differencing:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "28d0c8a0", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:06.307362Z", + "iopub.status.busy": "2023-12-20T16:44:06.307161Z", + "iopub.status.idle": "2023-12-20T16:44:06.311545Z", + "shell.execute_reply": "2023-12-20T16:44:06.310928Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "dz_dx approximation = 0.9885002155707312\n", + "dz_dy approximation = -0.5921569579125929\n" + ] + } + ], + "source": [ + "def dJ_dm(J, m, *, eps=1.0e-7):\n", + " return (J(m + eps) - J(m - eps)) / (2.0 * eps)\n", + "\n", + "\n", + "print(f\"dz_dx approximation = {dJ_dm(lambda x: forward(x, float(y)), float(x))}\")\n", + "print(f\"dz_dy approximation = {dJ_dm(lambda y: forward(float(x), y), float(y))}\")" + ] + }, + { + "cell_type": "markdown", + "id": "4ef1b47b", + "metadata": {}, + "source": [ + "## Computing derivatives using a tangent-linear\n", + "\n", + "A tangent-linear computes directional derivatives with respect to a given control and with a given direction.\n", + "\n", + "Here we consider the forward to be a function of `x` and `y`, computing a value of `z`, denoted $z \\left( x, y \\right)$. We consider the control $m = \\left( x, y \\right)^T$, and a direction $\\zeta = \\left( 2, 3 \\right)^T$. We can then use a tangent-linear to compute the directional derivative\n", + "\n", + "$$\n", + " \\frac{dz}{dm} \\zeta = 2 \\frac{dz}{dx} + 3 \\frac{dz}{dy},\n", + "$$\n", + "\n", + "where vector derivatives are notated using row vectors.\n", + "\n", + "In most cases tlm_adjoint needs to be told what tangent-linear calculations to perform *ahead* of the forward calculations. `configure_tlm` provides this information to tlm_adjoint:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "0220eb94", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:06.314171Z", + "iopub.status.busy": "2023-12-20T16:44:06.313694Z", + "iopub.status.idle": "2023-12-20T16:44:06.376509Z", + "shell.execute_reply": "2023-12-20T16:44:06.375858Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "float(dz_dm_zeta)=0.2005295584792861\n" + ] + } + ], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(x, y):\n", + " z = x * np.sin(y * np.exp(y))\n", + " return z\n", + "\n", + "\n", + "x = Float(1.0, name=\"x\")\n", + "y = Float(0.25 * np.pi, name=\"y\")\n", + "\n", + "m = (x, y)\n", + "zeta = (Float(2.0, name=\"zeta_x\"), Float(3.0, name=\"zeta_y\"))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "z = forward(x, y)\n", + "stop_manager()\n", + "\n", + "dz_dm_zeta = var_tlm(z, (m, zeta))\n", + "\n", + "print(f\"{float(dz_dm_zeta)=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "c00043a7", + "metadata": {}, + "source": [ + "There are three new changes:\n", + "\n", + "- The control $m$ and direction $\\zeta$ are defined using `m` and `zeta` respectively.\n", + "- Tangent-linear calculations are configured *before* the forward calculations are performed, using `configure_tlm`. Note that here, for this first derivative calculation, the argument is a single control-direction pair.\n", + "- We access the tangent-linear variable, containing the value of $\\left( dz/dm \\right) \\zeta$, using `var_tlm`, and using the same control-direction pair.\n", + "\n", + "In fact more has happened here – if we now display the information recorded by tlm_adjoint:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "5b2a2150", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:06.379257Z", + "iopub.status.busy": "2023-12-20T16:44:06.378794Z", + "iopub.status.idle": "2023-12-20T16:44:06.382855Z", + "shell.execute_reply": "2023-12-20T16:44:06.382192Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Equation manager status:\n", + "Annotation state: AnnotationState.STOPPED\n", + "Tangent-linear state: TangentLinearState.STOPPED\n", + "Equations:\n", + " Block 0\n", + " Equation 0, FloatEquation solving for f_31 (id 31)\n", + " Dependency 0, f_31 (id 31), linear\n", + " Dependency 1, y (id 28), non-linear\n", + " Equation 1, FloatEquation solving for f_31_tlm((x,y),(zeta_x,zeta_y)) (id 33)\n", + " Dependency 0, f_31_tlm((x,y),(zeta_x,zeta_y)) (id 33), linear\n", + " Dependency 1, y (id 28), non-linear\n", + " Dependency 2, zeta_y (id 30), non-linear\n", + " Equation 2, FloatEquation solving for f_35 (id 35)\n", + " Dependency 0, f_35 (id 35), linear\n", + " Dependency 1, y (id 28), non-linear\n", + " Dependency 2, f_31 (id 31), non-linear\n", + " Equation 3, FloatEquation solving for f_35_tlm((x,y),(zeta_x,zeta_y)) (id 37)\n", + " Dependency 0, f_35_tlm((x,y),(zeta_x,zeta_y)) (id 37), linear\n", + " Dependency 1, y (id 28), non-linear\n", + " Dependency 2, zeta_y (id 30), non-linear\n", + " Dependency 3, f_31 (id 31), non-linear\n", + " Dependency 4, f_31_tlm((x,y),(zeta_x,zeta_y)) (id 33), non-linear\n", + " Equation 4, FloatEquation solving for f_39 (id 39)\n", + " Dependency 0, f_39 (id 39), linear\n", + " Dependency 1, f_35 (id 35), non-linear\n", + " Equation 5, FloatEquation solving for f_39_tlm((x,y),(zeta_x,zeta_y)) (id 41)\n", + " Dependency 0, f_39_tlm((x,y),(zeta_x,zeta_y)) (id 41), linear\n", + " Dependency 1, f_35 (id 35), non-linear\n", + " Dependency 2, f_35_tlm((x,y),(zeta_x,zeta_y)) (id 37), non-linear\n", + " Equation 6, FloatEquation solving for f_43 (id 43)\n", + " Dependency 0, f_43 (id 43), linear\n", + " Dependency 1, x (id 27), non-linear\n", + " Dependency 2, f_39 (id 39), non-linear\n", + " Equation 7, FloatEquation solving for f_43_tlm((x,y),(zeta_x,zeta_y)) (id 46)\n", + " Dependency 0, f_43_tlm((x,y),(zeta_x,zeta_y)) (id 46), linear\n", + " Dependency 1, x (id 27), non-linear\n", + " Dependency 2, zeta_x (id 29), non-linear\n", + " Dependency 3, f_39 (id 39), non-linear\n", + " Dependency 4, f_39_tlm((x,y),(zeta_x,zeta_y)) (id 41), non-linear\n", + "Storage:\n", + " Storing initial conditions: yes\n", + " Storing equation non-linear dependencies: yes\n", + " Initial conditions stored: 4\n", + " Initial conditions referenced: 0\n", + "Checkpointing:\n", + " Method: memory\n" + ] + } + ], + "source": [ + "manager_info()" + ] + }, + { + "cell_type": "markdown", + "id": "527f3b8b", + "metadata": {}, + "source": [ + "we now see that there are *eight* `FloatEquation` records – the original four, and four new ones. The extra ones correspond to the tangent-linear calculations. tlm_adjoint has recorded the original forward calculations, and *also* recorded the tangent-linear calculations.\n", + "\n", + "## Computing second derivatives using an adjoint of a tangent-linear\n", + "\n", + "Since tlm_adjoint has recorded both forward and tangent-linear calculations, we can now compute second derivative information using an adjoint associated with a tangent-linear. Specifically we can compute a Hessian action on $\\zeta$,\n", + "\n", + "$$\n", + " H \\zeta = \\frac{d}{dm} \\left( \\frac{dz}{dm} \\zeta \\right)^T.\n", + "$$\n", + "\n", + "The inner directional derivative appearing here is computed using the tangent-linear method, and the outer derivative is computed by applying the adjoint method to the tangent-linear and forward calculations. In code we simply use `compute_gradient` to compute the derivative of the tangent-linear result:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "8612e275", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:06.385270Z", + "iopub.status.busy": "2023-12-20T16:44:06.385075Z", + "iopub.status.idle": "2023-12-20T16:44:06.432869Z", + "shell.execute_reply": "2023-12-20T16:44:06.432244Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "float(dz_dm_zeta)=0.2005295584792861\n", + "float(d2z_dm_zeta_dx)=-1.7764708733484629\n", + "float(d2z_dm_zeta_dy)=-49.4290736694207\n" + ] + } + ], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(x, y):\n", + " z = x * np.sin(y * np.exp(y))\n", + " return z\n", + "\n", + "\n", + "x = Float(1.0, name=\"x\")\n", + "y = Float(0.25 * np.pi, name=\"y\")\n", + "\n", + "m = (x, y)\n", + "zeta = (Float(2.0, name=\"zeta_x\"), Float(3.0, name=\"zeta_y\"))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "z = forward(x, y)\n", + "stop_manager()\n", + "\n", + "dz_dm_zeta = var_tlm(z, (m, zeta))\n", + "\n", + "print(f\"{float(dz_dm_zeta)=}\")\n", + "\n", + "d2z_dm_zeta_dx, d2z_dm_zeta_dy = compute_gradient(dz_dm_zeta, m)\n", + "\n", + "print(f\"{float(d2z_dm_zeta_dx)=}\")\n", + "print(f\"{float(d2z_dm_zeta_dy)=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "b4b0fac5", + "metadata": {}, + "source": [ + "## Computing second derivatives using a tangent-linear of a tangent-linear\n", + "\n", + "We can also compute second derivative information using a tangent-linear associated with a tangent-linear. For example if we define $e_1 = \\left( 1, 0 \\right)^T$, then we can find the first component of the previously computed Hessian action on $\\zeta$ via\n", + "\n", + "$$\n", + " e_1^T H \\zeta = \\left[ \\frac{d}{dm} \\left( \\frac{dz}{dm} \\zeta \\right) \\right] e_1.\n", + "$$\n", + "\n", + "That is, here we now want to compute a directional derivative of a directional derivative, and we compute this using a tangent-linear associated with the previous tangent-linear and forward calculations.\n", + "\n", + "tlm_adjoint handles this case by supplying more arguments to `configure_tlm`:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "5c02e84f", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:06.435590Z", + "iopub.status.busy": "2023-12-20T16:44:06.435213Z", + "iopub.status.idle": "2023-12-20T16:44:06.520335Z", + "shell.execute_reply": "2023-12-20T16:44:06.519404Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "float(dz_dm_zeta)=0.2005295584792861\n", + "float(dz_dx)=0.9885002159138745\n", + "float(d2z_dm_zeta_dx)=-1.7764708733484629\n" + ] + } + ], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(x, y):\n", + " z = x * np.sin(y * np.exp(y))\n", + " return z\n", + "\n", + "\n", + "x = Float(1.0, name=\"x\")\n", + "y = Float(0.25 * np.pi, name=\"y\")\n", + "\n", + "m = (x, y)\n", + "zeta = (Float(2.0, name=\"zeta_x\"), Float(3.0, name=\"zeta_y\"))\n", + "e_1 = (Float(1.0, name=\"e_1_x\"), Float(0.0, name=\"e_1_y\"))\n", + "configure_tlm((m, zeta), (m, e_1))\n", + "\n", + "start_manager()\n", + "z = forward(x, y)\n", + "stop_manager()\n", + "\n", + "dz_dm_zeta = var_tlm(z, (m, zeta))\n", + "dz_dx = var_tlm(z, (m, e_1))\n", + "d2z_dm_zeta_dx = var_tlm(z, (m, zeta), (m, e_1))\n", + "\n", + "print(f\"{float(dz_dm_zeta)=}\")\n", + "print(f\"{float(dz_dx)=}\")\n", + "print(f\"{float(d2z_dm_zeta_dx)=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "88b3a4f1", + "metadata": {}, + "source": [ + "The first control-direction pair supplied to `configure_tlm` indicates that we seek to compute directional derivatives of the forward with respect to the control defined by `m` with direction defined by `zeta`. The second control-direction pair indicates that we seek to compute directional deriatives of *these* directional derivatives, with respect to the control defined by `m` and with direction defined by `e_1`. As a side-effect we find that we also compute the directional derivatives of the forward with respect to the control defined by `m` with direction defined by `e_1`.\n", + "\n", + "We then access the tangent-linear variables using `var_tlm`, supplying two control-variable pairs to access a second order tangent-linear variable.\n", + "\n", + "As before, tlm_adjoint has not just performed the tangent-linear calculations – if we display the information recorded by tlm_adjoint:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "cb6739a1", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:06.522820Z", + "iopub.status.busy": "2023-12-20T16:44:06.522563Z", + "iopub.status.idle": "2023-12-20T16:44:06.526609Z", + "shell.execute_reply": "2023-12-20T16:44:06.526003Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Equation manager status:\n", + "Annotation state: AnnotationState.STOPPED\n", + "Tangent-linear state: TangentLinearState.STOPPED\n", + "Equations:\n", + " Block 0\n", + " Equation 0, FloatEquation solving for f_103 (id 103)\n", + " Dependency 0, f_103 (id 103), linear\n", + " Dependency 1, y (id 98), non-linear\n", + " Equation 1, FloatEquation solving for f_103_tlm((x,y),(zeta_x,zeta_y)) (id 105)\n", + " Dependency 0, f_103_tlm((x,y),(zeta_x,zeta_y)) (id 105), linear\n", + " Dependency 1, y (id 98), non-linear\n", + " Dependency 2, zeta_y (id 100), non-linear\n", + " Equation 2, FloatEquation solving for f_103_tlm((x,y),(e_1_x,e_1_y)) (id 107)\n", + " Dependency 0, f_103_tlm((x,y),(e_1_x,e_1_y)) (id 107), linear\n", + " Dependency 1, y (id 98), non-linear\n", + " Dependency 2, e_1_y (id 102), non-linear\n", + " Equation 3, FloatEquation solving for f_103_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 110)\n", + " Dependency 0, f_103_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 110), linear\n", + " Dependency 1, y (id 98), non-linear\n", + " Dependency 2, zeta_y (id 100), non-linear\n", + " Dependency 3, e_1_y (id 102), non-linear\n", + " Dependency 4, zeta_y_tlm((x,y),(e_1_x,e_1_y)) (id 109), non-linear\n", + " Equation 4, FloatEquation solving for f_112 (id 112)\n", + " Dependency 0, f_112 (id 112), linear\n", + " Dependency 1, y (id 98), non-linear\n", + " Dependency 2, f_103 (id 103), non-linear\n", + " Equation 5, FloatEquation solving for f_112_tlm((x,y),(zeta_x,zeta_y)) (id 114)\n", + " Dependency 0, f_112_tlm((x,y),(zeta_x,zeta_y)) (id 114), linear\n", + " Dependency 1, y (id 98), non-linear\n", + " Dependency 2, zeta_y (id 100), non-linear\n", + " Dependency 3, f_103 (id 103), non-linear\n", + " Dependency 4, f_103_tlm((x,y),(zeta_x,zeta_y)) (id 105), non-linear\n", + " Equation 6, FloatEquation solving for f_112_tlm((x,y),(e_1_x,e_1_y)) (id 116)\n", + " Dependency 0, f_112_tlm((x,y),(e_1_x,e_1_y)) (id 116), linear\n", + " Dependency 1, y (id 98), non-linear\n", + " Dependency 2, e_1_y (id 102), non-linear\n", + " Dependency 3, f_103 (id 103), non-linear\n", + " Dependency 4, f_103_tlm((x,y),(e_1_x,e_1_y)) (id 107), non-linear\n", + " Equation 7, FloatEquation solving for f_112_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 118)\n", + " Dependency 0, f_112_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 118), linear\n", + " Dependency 1, y (id 98), non-linear\n", + " Dependency 2, zeta_y (id 100), non-linear\n", + " Dependency 3, e_1_y (id 102), non-linear\n", + " Dependency 4, f_103 (id 103), non-linear\n", + " Dependency 5, f_103_tlm((x,y),(zeta_x,zeta_y)) (id 105), non-linear\n", + " Dependency 6, f_103_tlm((x,y),(e_1_x,e_1_y)) (id 107), non-linear\n", + " Dependency 7, zeta_y_tlm((x,y),(e_1_x,e_1_y)) (id 109), non-linear\n", + " Dependency 8, f_103_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 110), non-linear\n", + " Equation 8, FloatEquation solving for f_120 (id 120)\n", + " Dependency 0, f_120 (id 120), linear\n", + " Dependency 1, f_112 (id 112), non-linear\n", + " Equation 9, FloatEquation solving for f_120_tlm((x,y),(zeta_x,zeta_y)) (id 122)\n", + " Dependency 0, f_120_tlm((x,y),(zeta_x,zeta_y)) (id 122), linear\n", + " Dependency 1, f_112 (id 112), non-linear\n", + " Dependency 2, f_112_tlm((x,y),(zeta_x,zeta_y)) (id 114), non-linear\n", + " Equation 10, FloatEquation solving for f_120_tlm((x,y),(e_1_x,e_1_y)) (id 124)\n", + " Dependency 0, f_120_tlm((x,y),(e_1_x,e_1_y)) (id 124), linear\n", + " Dependency 1, f_112 (id 112), non-linear\n", + " Dependency 2, f_112_tlm((x,y),(e_1_x,e_1_y)) (id 116), non-linear\n", + " Equation 11, FloatEquation solving for f_120_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 126)\n", + " Dependency 0, f_120_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 126), linear\n", + " Dependency 1, f_112 (id 112), non-linear\n", + " Dependency 2, f_112_tlm((x,y),(zeta_x,zeta_y)) (id 114), non-linear\n", + " Dependency 3, f_112_tlm((x,y),(e_1_x,e_1_y)) (id 116), non-linear\n", + " Dependency 4, f_112_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 118), non-linear\n", + " Equation 12, FloatEquation solving for f_128 (id 128)\n", + " Dependency 0, f_128 (id 128), linear\n", + " Dependency 1, x (id 97), non-linear\n", + " Dependency 2, f_120 (id 120), non-linear\n", + " Equation 13, FloatEquation solving for f_128_tlm((x,y),(zeta_x,zeta_y)) (id 131)\n", + " Dependency 0, f_128_tlm((x,y),(zeta_x,zeta_y)) (id 131), linear\n", + " Dependency 1, x (id 97), non-linear\n", + " Dependency 2, zeta_x (id 99), non-linear\n", + " Dependency 3, f_120 (id 120), non-linear\n", + " Dependency 4, f_120_tlm((x,y),(zeta_x,zeta_y)) (id 122), non-linear\n", + " Equation 14, FloatEquation solving for f_128_tlm((x,y),(e_1_x,e_1_y)) (id 134)\n", + " Dependency 0, f_128_tlm((x,y),(e_1_x,e_1_y)) (id 134), linear\n", + " Dependency 1, x (id 97), non-linear\n", + " Dependency 2, e_1_x (id 101), non-linear\n", + " Dependency 3, f_120 (id 120), non-linear\n", + " Dependency 4, f_120_tlm((x,y),(e_1_x,e_1_y)) (id 124), non-linear\n", + " Equation 15, FloatEquation solving for f_128_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 138)\n", + " Dependency 0, f_128_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 138), linear\n", + " Dependency 1, x (id 97), non-linear\n", + " Dependency 2, zeta_x (id 99), non-linear\n", + " Dependency 3, e_1_x (id 101), non-linear\n", + " Dependency 4, f_120 (id 120), non-linear\n", + " Dependency 5, f_120_tlm((x,y),(zeta_x,zeta_y)) (id 122), non-linear\n", + " Dependency 6, f_120_tlm((x,y),(e_1_x,e_1_y)) (id 124), non-linear\n", + " Dependency 7, f_120_tlm((x,y),(zeta_x,zeta_y))_tlm((x,y),(e_1_x,e_1_y)) (id 126), non-linear\n", + " Dependency 8, zeta_x_tlm((x,y),(e_1_x,e_1_y)) (id 137), non-linear\n", + "Storage:\n", + " Storing initial conditions: yes\n", + " Storing equation non-linear dependencies: yes\n", + " Initial conditions stored: 8\n", + " Initial conditions referenced: 0\n", + "Checkpointing:\n", + " Method: memory\n" + ] + } + ], + "source": [ + "manager_info()" + ] + }, + { + "cell_type": "markdown", + "id": "1e4fde63", + "metadata": {}, + "source": [ + "we now find that there are *sixteen* `FloatEquation` records, constituting the forward and all tangent-linear calculations.\n", + "\n", + "## Computing third derivatives using an adjoint of a tangent-linear of a tangent-linear\n", + "\n", + "We can now compute the derivative of the tangent-linear-computed second derivative by simply handing the second order tangent-linear variable to `compute_gradient`. This applies the adjoint method to the higher order tangent-linear calculations and the forward calculations, computing\n", + "\n", + "$$\n", + " \\frac{d}{dm} \\left( e_1^T H \\zeta \\right) = \\frac{d}{dm} \\left[ \\left[ \\frac{d}{dm} \\left( \\frac{dz}{dm} \\zeta \\right) \\right] e_1 \\right].\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "c60bc4b8", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:06.529193Z", + "iopub.status.busy": "2023-12-20T16:44:06.528992Z", + "iopub.status.idle": "2023-12-20T16:44:06.640859Z", + "shell.execute_reply": "2023-12-20T16:44:06.640275Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "float(dz_dm_zeta)=0.2005295584792861\n", + "float(dz_dx)=0.9885002159138745\n", + "float(d2z_dm_zeta_dx)=-1.7764708733484629\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "float(d3z_dm_zeta_dx_dx)=0.0\n", + "float(d3z_dm_zeta_dx_dy)=-48.244759753855064\n" + ] + } + ], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(x, y):\n", + " z = x * np.sin(y * np.exp(y))\n", + " return z\n", + "\n", + "\n", + "x = Float(1.0, name=\"x\")\n", + "y = Float(0.25 * np.pi, name=\"y\")\n", + "\n", + "m = (x, y)\n", + "zeta = (Float(2.0, name=\"zeta_x\"), Float(3.0, name=\"zeta_y\"))\n", + "e_1 = (Float(1.0, name=\"e_1_x\"), Float(0.0, name=\"e_1_y\"))\n", + "configure_tlm((m, zeta), (m, e_1))\n", + "\n", + "start_manager()\n", + "z = forward(x, y)\n", + "stop_manager()\n", + "\n", + "dz_dm_zeta = var_tlm(z, (m, zeta))\n", + "dz_dx = var_tlm(z, (m, e_1))\n", + "d2z_dm_zeta_dx = var_tlm(z, (m, zeta), (m, e_1))\n", + "\n", + "print(f\"{float(dz_dm_zeta)=}\")\n", + "print(f\"{float(dz_dx)=}\")\n", + "print(f\"{float(d2z_dm_zeta_dx)=}\")\n", + "\n", + "d3z_dm_zeta_dx_dx, d3z_dm_zeta_dx_dy = compute_gradient(d2z_dm_zeta_dx, m)\n", + "\n", + "print(f\"{float(d3z_dm_zeta_dx_dx)=}\")\n", + "print(f\"{float(d3z_dm_zeta_dx_dy)=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "b1a3e919", + "metadata": {}, + "source": [ + "## Higher order\n", + "\n", + "The approach now generalizes.\n", + "\n", + "- Supplying further arguments to `configure_tlm` indicates directional derivatives of directional derivatives, defining a tangent-linear calculation of increasingly high order.\n", + "- Supplying these arguments to `var_tlm` accesses the higher-order tangent-linear variables.\n", + "- These higher order tangent-linear variables can be handed to `compute_gradient` to compute derivatives of the higher order derivatives using the adjoint method." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/1_time_independent.html b/examples/1_time_independent.html new file mode 100644 index 0000000..4d99bc6 --- /dev/null +++ b/examples/1_time_independent.html @@ -0,0 +1,838 @@ + + + + + + + Time-independent example — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Time-independent example

+

This notebook describes adjoint calculations, and Hessian calculations, using tlm_adjoint with the Firedrake backend. A time-independent problem is considered, and importantly checkpointing is not used for the adjoint calculations. This notebook further describes how variables may be flagged to facilitate caching.

+

The high-level algorithmic differentiation approach used by tlm_adjoint is based on the method described in:

+
    +

  • P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, ‘Automated derivation of the adjoint of high-level transient finite element programs’, SIAM Journal on Scientific Computing 35(4), pp. C369–C393, 2013, doi: 10.1137/120873558

    • +
+

The caching of data in tlm_adjoint uses an approach based on that described in:

+
    +

  • J. R. Maddison and P. E. Farrell, ‘Rapid development and adjoining of transient finite element models’, Computer Methods in Applied Mechanics and Engineering, 276, 95–121, 2014, doi: 10.1016/j.cma.2014.03.010

    • +
+
+

Forward problem

+

We consider the solution of a linear time-independent partial differential equation, followed by the calculation of the \(L^2\)-norm of the solution. Extra non-linearity is introduced by allowing the right-hand-side of the partial differential equation to depend non-linearly on the control. We assume real spaces and a real build of Firedrake throughout.

+

Specifically we consider the solution \(u \in V_0\) of

+
+\[\forall \zeta \in V_0 \qquad \int_\Omega \nabla \zeta \cdot \nabla u = \int_\Omega \zeta m^2,\]
+

where \(V\) is a real \(P_1\) continuous finite element space defining functions on the domain \(\Omega = \left( 0, 1 \right)^2\), with \(m \in V\), and where \(V_0\) consists of the functions in \(V\) which have zero trace. This corresponds to a discretization of the partial differential equation

+
+\[-\nabla^2 u = m^2 \quad \text{on } \left( x, y \right) \in \left( 0, 1 \right)^2,\]
+

subject to homogeneous Dirichlet boundary conditions.

+

A simple implementation in Firedrake, with \(m = x y\), takes the form:

+
+
[1]:
+
+
+
from firedrake import *
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(X[0] * X[1])
+
+u = Function(space, name="u")
+solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,
+      DirichletBC(space, 0.0, "on_boundary"))
+
+J_sq = assemble(inner(u, u) * dx)
+J = sqrt(J_sq)
+
+
+
+
+
+

Adding tlm_adjoint

+

We first modify the code so that tlm_adjoint processes the calculations:

+
+
[2]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+reset_manager()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(X[0] * X[1])
+
+
+def forward(m):
+    u = Function(space, name="u")
+    solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,
+          DirichletBC(space, 0.0, "on_boundary"))
+
+    J_sq = Functional(name="J_sq")
+    J_sq.assign(inner(u, u) * dx)
+    J = np.sqrt(J_sq)
+    return J
+
+
+start_manager()
+J = forward(m)
+stop_manager()
+
+
+
+
+
[2]:
+
+
+
+
+(True, True)
+
+
+

The key changes here are:

+
    +

  • To import tlm_adjoint with the Firedrake backend.

    • +

  • Controlling the ‘manager’ – an object tlm_adjoint uses to process equations.

    • +

  • Using a Functional to compute the square of the \(L^2\)-norm of the solution of the (discretized) partial differential equation. This facilitates the calculation of simple functionals e.g. using finite element assembly.

    • +

  • Taking the square root of the square of the \(L^2\)-norm using NumPy.

    • +
+

Let’s display the information recorded by tlm_adjoint:

+
+
[3]:
+
+
+
manager_info()
+
+
+
+
+
+
+
+
+Equation manager status:
+Annotation state: AnnotationState.STOPPED
+Tangent-linear state: TangentLinearState.STOPPED
+Equations:
+  Block 0
+    Equation 0, EquationSolver solving for u (id 2)
+      Dependency 0, u (id 2), linear
+      Dependency 1, m (id 0), non-linear
+    Equation 1, Assembly solving for J_sq (id 9)
+      Dependency 0, J_sq (id 9), linear
+      Dependency 1, u (id 2), non-linear
+    Equation 2, FloatEquation solving for f_11 (id 11)
+      Dependency 0, f_11 (id 11), linear
+      Dependency 1, J_sq (id 9), non-linear
+Storage:
+  Storing initial conditions: yes
+  Storing equation non-linear dependencies: yes
+  Initial conditions stored: 2
+  Initial conditions referenced: 0
+Checkpointing:
+  Method: memory
+
+
+

We see that there are three records.

+
    +

  • Equation 0, an EquationSolver. This records the solution of the finite element variational problem for u.

    • +

  • Equation 1, an Assembly. This records the calculation of the square of the \(L^2\)-norm.

    • +

  • Equation 2, a FloatEquation. This records the calculation of the square root of the square of the \(L^2\)-norm.

    • +
+
+
+

Computing derivatives using an adjoint

+

The compute_gradient function can be used to compute derivatives using the adjoint method. Here we compute the derivative of the \(L^2\)-norm of the resulting solution, considered a function of the control defined by m, with respect to this control:

+
+
[4]:
+
+
+
dJ_dm = compute_gradient(J, m)
+
+
+
+

Here each degree of freedom associated with dJ_dm contains the derivative of the functional with respect to the corresponding degree of freedom for the control. dJ_dm represents a ‘dual space’ object, defining a linear functional which, given a ‘direction’ \(\zeta \in V\), can be used to compute the directional derivative with respect to \(m\) with direction \(\zeta\).

+

For example we can compute the directional derivative of the functional with respect to the control \(m\) with direction equal to the unity valued function via:

+
+
[5]:
+
+
+
one = Function(space, name="one")
+one.interpolate(Constant(1.0))
+
+dJ_dm_one = var_inner(one, dJ_dm)
+
+print(f"{dJ_dm_one=}")
+
+
+
+
+
+
+
+
+dJ_dm_one=0.02035145324320026
+
+
+

This result is the derivative of the \(L^2\)-norm of the solution with respect to the amplitude of a spatially constant perturbation to the control \(m\). We can compare with the result from finite differencing:

+
+
[6]:
+
+
+
def dJ_dm(J, m, *, eps=1.0e-7):
+    return (J(m + eps) - J(m - eps)) / (2.0 * eps)
+
+
+print(f"dJ_dm_one approximation = {dJ_dm(lambda eps: float(forward(m + eps * one)), 0.0)}")
+
+
+
+
+
+
+
+
+dJ_dm_one approximation = 0.020351453248329543
+
+
+
+
+

Computing Hessian information using an adjoint of a tangent-linear

+
+

Single direction

+

We next seek to compute the action of the Hessian of the functional on some direction \(\zeta \in V\), using the adjoint method applied to tangent-linear and forward calculations. This can be handled directly, by configuring the relevant tangent-linear and computing the derivative using compute_gradient:

+
+
[7]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+reset_manager()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(X[0] * X[1])
+
+
+def forward(m):
+    u = Function(space, name="u")
+    solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,
+          DirichletBC(space, 0.0, "on_boundary"))
+
+    J_sq = Functional(name="J_sq")
+    J_sq.assign(inner(u, u) * dx)
+    J = np.sqrt(J_sq)
+    return J
+
+
+zeta = Function(space, name="zeta")
+zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))
+configure_tlm((m, zeta))
+
+start_manager()
+J = forward(m)
+stop_manager()
+
+dJ_dm_zeta = var_tlm(J, (m, zeta))
+
+d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)
+
+
+
+

The Hessian class applies the same approach, but handles several of the steps for us:

+
+
[8]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+reset_manager()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(X[0] * X[1])
+
+
+def forward(m):
+    u = Function(space, name="u")
+    solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,
+          DirichletBC(space, 0.0, "on_boundary"))
+
+    J_sq = Functional(name="J_sq")
+    J_sq.assign(inner(u, u) * dx)
+    J = np.sqrt(J_sq)
+    return J
+
+
+H = Hessian(forward)
+
+zeta = Function(space, name="zeta")
+zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))
+
+_, dJ_dm_zeta, d2J_dm_zeta_dm = H.action(m, zeta)
+
+
+
+
+
+

Multiple directions

+

If we want to compute the Hessian action on multiple directions we can define multiple tangent-linears:

+
+
[9]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+reset_manager()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(X[0] * X[1])
+
+
+def forward(m):
+    u = Function(space, name="u")
+    solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,
+          DirichletBC(space, 0.0, "on_boundary"))
+
+    J_sq = Functional(name="J_sq")
+    J_sq.assign(inner(u, u) * dx)
+    J = np.sqrt(J_sq)
+    return J
+
+
+zeta_0 = Function(space, name="zeta_0")
+zeta_0.interpolate(sin(pi * X[0]) * sin(pi * X[1]))
+configure_tlm((m, zeta_0))
+
+zeta_1 = Function(space, name="zeta_1")
+zeta_1.interpolate(sin(pi * X[0]) * sin(2.0 * pi * X[1]))
+configure_tlm((m, zeta_1))
+
+start_manager()
+J = forward(m)
+stop_manager()
+
+dJ_dm_zeta_0 = var_tlm(J, (m, zeta_0))
+dJ_dm_zeta_1 = var_tlm(J, (m, zeta_1))
+
+d2J_dm_zeta_0_dm, d2J_dm_zeta_1_dm = compute_gradient((dJ_dm_zeta_0, dJ_dm_zeta_1), m)
+
+
+
+

There are now calculations for two sets of tangent-linear variables, two sets of first order adjoint variables, and two sets of second order adjoint variables. However the two sets of first order adjoint variables have the same values – by default tlm_adjoint detects this and only computes them once.

+

The above approach requires us to know the directions before the forward calculation. However some algorithms can generate the directions sequentially, and we do not know the next direction until a Hessian action on the previous direction has been computed. If possible we still want to avoid re-running the forward calculation each time we have a new direction.

+

If we have sufficient memory available, and in particular so long as we do not need to use checkpointing for the adjoint calculations, we can make use of the CachedHessian class. This stores the forward solution and, by default, caches and reuses first order adjoint values. Here we do not need to configure the tangent-linear before the forward calculation – instead tlm_adjoint performs the tangent-linear calculations after the forward calculations:

+
+
[10]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+reset_manager()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(X[0] * X[1])
+
+
+def forward(m):
+    u = Function(space, name="u")
+    solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,
+          DirichletBC(space, 0.0, "on_boundary"))
+
+    J_sq = Functional(name="J_sq")
+    J_sq.assign(inner(u, u) * dx)
+    J = np.sqrt(J_sq)
+    return J
+
+
+start_manager()
+J = forward(m)
+stop_manager()
+
+H = CachedHessian(J)
+
+zeta_0 = Function(space, name="zeta_0")
+zeta_0.interpolate(sin(pi * X[0]) * sin(pi * X[1]))
+
+zeta_1 = Function(space, name="zeta_1")
+zeta_1.interpolate(sin(pi * X[0]) * sin(2.0 * pi * X[1]))
+
+_, dJ_dm_zeta_0, d2J_dm_zeta_0_dm = H.action(m, zeta_0)
+_, dJ_dm_zeta_1, d2J_dm_zeta_1_dm = H.action(m, zeta_1)
+
+
+
+
+
+
+

Assembly and solver caching

+
+

Using an EquationSolver

+

The calculation for the Hessian action includes four discrete Poisson equations: one for the original forward calculation, one for the tangent-linear calculation, and one each for first and second order adjoint calculations. In this self-adjoint problem the finite element matrix – a stiffness matrix – is the same across all four calculations. Hence we can cache and reuse it. Moreover we can cache and reuse linear solver data – for example we can cache and reuse the Cholesky factorization.

+

tlm_adjoint can apply such caching automatically, but we must interact directly with the object tlm_adjoint uses to record the solution of finite element variational problems – the EquationSolver previously seen when we used manager_info(). This looks like:

+
+
[11]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+reset_manager()
+clear_caches()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(X[0] * X[1])
+
+
+def forward(m):
+    u = Function(space, name="u")
+    eq = EquationSolver(
+        inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,
+        HomogeneousDirichletBC(space, "on_boundary"),
+        solver_parameters={"ksp_type": "preonly",
+                           "pc_type": "cholesky"})
+    eq.solve()
+
+    J_sq = Functional(name="J_sq")
+    J_sq.assign(inner(u, u) * dx)
+    J = np.sqrt(J_sq)
+    return J
+
+
+zeta = Function(space, name="zeta")
+zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))
+configure_tlm((m, zeta))
+
+start_manager()
+J = forward(m)
+stop_manager()
+
+dJ_dm_zeta = var_tlm(J, (m, zeta))
+
+d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)
+
+
+
+

The key changes here are:

+
    +

  • The use of clear_caches. This ensures that any previously cached data is cleared, avoiding memory leaks if the code is run more than once.

    • +

  • The use of HomogeneousDirichletBC. This tells tlm_adjoint that the boundary condition is homogeneous, and helps it detect that forward and adjoint problems have the same boundary conditions.

    • +

  • The instantiation of an EquationSolver, and the call to its solve method.

    • +
+

If we query the relevant tlm_adjoint caches we find:

+
+
[12]:
+
+
+
print(f"{len(assembly_cache())=}")
+print(f"{len(linear_solver_cache())=}")
+
+assert len(assembly_cache()) == 1
+assert len(linear_solver_cache()) == 1
+
+
+
+
+
+
+
+
+len(assembly_cache())=1
+len(linear_solver_cache())=1
+
+
+

and in particular we see that tlm_adjoint has cached data associated with a single matrix, and has cached a single assembled object (which turns out to be the matrix itself). The latter is a stiffness matrix, and the former stores its Cholesky factorization. The factorization is used four times: in the forward, tangent-linear, and first and second order adjoint calculations.

+
+
+

Flagging data for caching

+

Now consider the slightly different calculation:

+
+
[13]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+reset_manager()
+clear_caches()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(X[0] * X[1])
+
+one = Constant(1.0, name="one")
+
+
+def forward(m):
+    u = Function(space, name="u")
+    eq = EquationSolver(
+        one * inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,
+        HomogeneousDirichletBC(space, "on_boundary"),
+        solver_parameters={"ksp_type": "preonly",
+                           "pc_type": "cholesky"})
+    eq.solve()
+
+    J_sq = Functional(name="J_sq")
+    J_sq.assign(inner(u, u) * dx)
+    J = np.sqrt(J_sq)
+    return J
+
+
+zeta = Function(space, name="zeta")
+zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))
+configure_tlm((m, zeta))
+
+start_manager()
+J = forward(m)
+stop_manager()
+
+dJ_dm_zeta = var_tlm(J, (m, zeta))
+
+d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)
+
+print(f"{len(assembly_cache())=}")
+print(f"{len(linear_solver_cache())=}")
+
+assert len(assembly_cache()) == 0
+assert len(linear_solver_cache()) == 0
+
+
+
+
+
+
+
+
+len(assembly_cache())=0
+len(linear_solver_cache())=0
+
+
+

The only difference is the introduction of the multiplication by one on the left-hand-side of the finite element variational problem. However we now find that no matrix or linear solver data has been cached. The issue is that tlm_adjoint does not know that it should cache the results of calculations involving one.

+
+

The ‘cache’ flag

+

To resolve this, we can flag one for caching using cache=True:

+
+
[14]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+reset_manager()
+clear_caches()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(X[0] * X[1])
+
+one = Constant(1.0, name="one", cache=True)
+
+
+def forward(m):
+    u = Function(space, name="u")
+    eq = EquationSolver(
+        one * inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,
+        HomogeneousDirichletBC(space, "on_boundary"),
+        solver_parameters={"ksp_type": "preonly",
+                           "pc_type": "cholesky"})
+    eq.solve()
+
+    J_sq = Functional(name="J_sq")
+    J_sq.assign(inner(u, u) * dx)
+    J = np.sqrt(J_sq)
+    return J
+
+
+zeta = Function(space, name="zeta")
+zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))
+configure_tlm((m, zeta))
+
+start_manager()
+J = forward(m)
+stop_manager()
+
+dJ_dm_zeta = var_tlm(J, (m, zeta))
+
+d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)
+
+print(f"{len(assembly_cache())=}")
+print(f"{len(linear_solver_cache())=}")
+
+assert len(assembly_cache()) == 2
+assert len(linear_solver_cache()) == 1
+
+
+
+
+
+
+
+
+len(assembly_cache())=2
+len(linear_solver_cache())=1
+
+
+

We now see that tlm_adjoint has cached linear solver data associated with a single matrix. However assembly of two objects has been cached – it turns out there are now two cached matrices.

+

The extra cached matrix appears in the tangent-linear calculations, involving the tangent-linear variable associated with one – a tangent-linear right-hand-side term has been converted into a matrix multiply using a different matrix. However in the above calculation we know that this tangent-linear variable must be zero, since the calculation for one doesn’t depend on the control variable. The extra term in the tangent-linear calculation is similarly also known to be zero.

+
+
+

The ‘static’ flag

+

We can let tlm_adjoint know that one does not change, and resolve this inefficiency, by instead using static=True:

+
+
[15]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+reset_manager()
+clear_caches()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(X[0] * X[1])
+
+one = Constant(1.0, name="one", static=True)
+
+
+def forward(m):
+    u = Function(space, name="u")
+    eq = EquationSolver(
+        one * inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,
+        HomogeneousDirichletBC(space, "on_boundary"),
+        solver_parameters={"ksp_type": "preonly",
+                           "pc_type": "cholesky"})
+    eq.solve()
+
+    J_sq = Functional(name="J_sq")
+    J_sq.assign(inner(u, u) * dx)
+    J = np.sqrt(J_sq)
+    return J
+
+
+zeta = Function(space, name="zeta")
+zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))
+configure_tlm((m, zeta))
+
+start_manager()
+J = forward(m)
+stop_manager()
+
+dJ_dm_zeta = var_tlm(J, (m, zeta))
+
+d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)
+
+print(f"{len(assembly_cache())=}")
+print(f"{len(linear_solver_cache())=}")
+
+assert len(assembly_cache()) == 1
+assert len(linear_solver_cache()) == 1
+
+
+
+
+
+
+
+
+len(assembly_cache())=1
+len(linear_solver_cache())=1
+
+
+

Here static=True leads to one being flagged as a variable whose value is never updated. From this tlm_adjoint can infer that the relevant associated tangent-linear variable is zero, and avoid adding the zero-valued tangent-linear term.

+

The key difference between using cache=True and static=True is that in the former the value of the variable may be updated. So long as tlm_adjoint is aware of the update (which happens, for example, when tlm_adjoint records a calculation) then updating the value of a variable invalidates cache entries, and invalidated cache entries are cleared automatically.

+
+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/examples/1_time_independent.ipynb b/examples/1_time_independent.ipynb new file mode 100644 index 0000000..7abb42c --- /dev/null +++ b/examples/1_time_independent.ipynb @@ -0,0 +1,1006 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "09bb9c5a", + "metadata": {}, + "source": [ + "# Time-independent example\n", + "\n", + "This notebook describes adjoint calculations, and Hessian calculations, using tlm_adjoint with the [Firedrake](https://firedrakeproject.org/) backend. A time-independent problem is considered, and importantly checkpointing is not used for the adjoint calculations. This notebook further describes how variables may be flagged to facilitate caching.\n", + "\n", + "The high-level algorithmic differentiation approach used by tlm_adjoint is based on the method described in:\n", + "\n", + "- P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, 'Automated derivation of the adjoint of high-level transient finite element programs', SIAM Journal on Scientific Computing 35(4), pp. C369–C393, 2013, doi: 10.1137/120873558\n", + "\n", + "The caching of data in tlm_adjoint uses an approach based on that described in:\n", + "\n", + "- J. R. Maddison and P. E. Farrell, 'Rapid development and adjoining of transient finite element models', Computer Methods in Applied Mechanics and Engineering, 276, 95–121, 2014, doi: 10.1016/j.cma.2014.03.010\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the solution of a linear time-independent partial differential equation, followed by the calculation of the $L^2$-norm of the solution. Extra non-linearity is introduced by allowing the right-hand-side of the partial differential equation to depend non-linearly on the control. We assume real spaces and a real build of Firedrake throughout.\n", + "\n", + "Specifically we consider the solution $u \\in V_0$ of\n", + "\n", + "$$\n", + " \\forall \\zeta \\in V_0 \\qquad \\int_\\Omega \\nabla \\zeta \\cdot \\nabla u = \\int_\\Omega \\zeta m^2,\n", + "$$\n", + "\n", + "where $V$ is a real $P_1$ continuous finite element space defining functions on the domain $\\Omega = \\left( 0, 1 \\right)^2$, with $m \\in V$, and where $V_0$ consists of the functions in $V$ which have zero trace. This corresponds to a discretization of the partial differential equation\n", + "\n", + "$$\n", + " -\\nabla^2 u = m^2 \\quad \\text{on } \\left( x, y \\right) \\in \\left( 0, 1 \\right)^2,\n", + "$$\n", + "\n", + "subject to homogeneous Dirichlet boundary conditions.\n", + "\n", + "A simple implementation in Firedrake, with $m = x y$, takes the form:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "227a8702", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:08.958446Z", + "iopub.status.busy": "2023-12-20T16:44:08.957183Z", + "iopub.status.idle": "2023-12-20T16:44:15.627967Z", + "shell.execute_reply": "2023-12-20T16:44:15.627249Z" + } + }, + "outputs": [], + "source": [ + "from firedrake import *\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "u = Function(space, name=\"u\")\n", + "solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + "J_sq = assemble(inner(u, u) * dx)\n", + "J = sqrt(J_sq)" + ] + }, + { + "cell_type": "markdown", + "id": "fa57fc17", + "metadata": {}, + "source": [ + "## Adding tlm_adjoint\n", + "\n", + "We first modify the code so that tlm_adjoint processes the calculations:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "2a3dd1f9", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:15.630955Z", + "iopub.status.busy": "2023-12-20T16:44:15.630594Z", + "iopub.status.idle": "2023-12-20T16:44:16.331869Z", + "shell.execute_reply": "2023-12-20T16:44:16.331190Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(True, True)" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "c23c7102", + "metadata": {}, + "source": [ + "The key changes here are:\n", + "\n", + "- To import tlm_adjoint with the Firedrake backend.\n", + "- Controlling the 'manager' – an object tlm_adjoint uses to process equations.\n", + "- Using a `Functional` to compute the square of the $L^2$-norm of the solution of the (discretized) partial differential equation. This facilitates the calculation of simple functionals e.g. using finite element assembly.\n", + "- Taking the square root of the square of the $L^2$-norm using NumPy.\n", + "\n", + "Let's display the information recorded by tlm_adjoint:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "a34fba5a", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:16.334425Z", + "iopub.status.busy": "2023-12-20T16:44:16.334121Z", + "iopub.status.idle": "2023-12-20T16:44:16.337913Z", + "shell.execute_reply": "2023-12-20T16:44:16.337188Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Equation manager status:\n", + "Annotation state: AnnotationState.STOPPED\n", + "Tangent-linear state: TangentLinearState.STOPPED\n", + "Equations:\n", + " Block 0\n", + " Equation 0, EquationSolver solving for u (id 2)\n", + " Dependency 0, u (id 2), linear\n", + " Dependency 1, m (id 0), non-linear\n", + " Equation 1, Assembly solving for J_sq (id 9)\n", + " Dependency 0, J_sq (id 9), linear\n", + " Dependency 1, u (id 2), non-linear\n", + " Equation 2, FloatEquation solving for f_11 (id 11)\n", + " Dependency 0, f_11 (id 11), linear\n", + " Dependency 1, J_sq (id 9), non-linear\n", + "Storage:\n", + " Storing initial conditions: yes\n", + " Storing equation non-linear dependencies: yes\n", + " Initial conditions stored: 2\n", + " Initial conditions referenced: 0\n", + "Checkpointing:\n", + " Method: memory\n" + ] + } + ], + "source": [ + "manager_info()" + ] + }, + { + "cell_type": "markdown", + "id": "adc31bf5", + "metadata": {}, + "source": [ + "We see that there are three records.\n", + "\n", + "- Equation 0, an `EquationSolver`. This records the solution of the finite element variational problem for `u`.\n", + "- Equation 1, an `Assembly`. This records the calculation of the square of the $L^2$-norm.\n", + "- Equation 2, a `FloatEquation`. This records the calculation of the square root of the square of the $L^2$-norm.\n", + "\n", + "## Computing derivatives using an adjoint\n", + "\n", + "The `compute_gradient` function can be used to compute derivatives using the adjoint method. Here we compute the derivative of the $L^2$-norm of the resulting solution, considered a function of the control defined by `m`, with respect to this control:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "d0a15072", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:16.340371Z", + "iopub.status.busy": "2023-12-20T16:44:16.340167Z", + "iopub.status.idle": "2023-12-20T16:44:17.629834Z", + "shell.execute_reply": "2023-12-20T16:44:17.629225Z" + } + }, + "outputs": [], + "source": [ + "dJ_dm = compute_gradient(J, m)" + ] + }, + { + "cell_type": "markdown", + "id": "b7c4c3bc", + "metadata": {}, + "source": [ + "Here each degree of freedom associated with `dJ_dm` contains the derivative of the functional with respect to the corresponding degree of freedom for the control. `dJ_dm` represents a 'dual space' object, defining a linear functional which, given a 'direction' $\\zeta \\in V$, can be used to compute the directional derivative with respect to $m$ with direction $\\zeta$.\n", + "\n", + "For example we can compute the directional derivative of the functional with respect to the control $m$ with direction equal to the unity valued function via:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "0772420c", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:17.632620Z", + "iopub.status.busy": "2023-12-20T16:44:17.632411Z", + "iopub.status.idle": "2023-12-20T16:44:17.919336Z", + "shell.execute_reply": "2023-12-20T16:44:17.918586Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "dJ_dm_one=0.02035145324320026\n" + ] + } + ], + "source": [ + "one = Function(space, name=\"one\")\n", + "one.interpolate(Constant(1.0))\n", + "\n", + "dJ_dm_one = var_inner(one, dJ_dm)\n", + "\n", + "print(f\"{dJ_dm_one=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "1b088023", + "metadata": {}, + "source": [ + "This result is the derivative of the $L^2$-norm of the solution with respect to the amplitude of a spatially constant perturbation to the control $m$. We can compare with the result from finite differencing:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "b8e8f6ca", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:17.922056Z", + "iopub.status.busy": "2023-12-20T16:44:17.921846Z", + "iopub.status.idle": "2023-12-20T16:44:19.034619Z", + "shell.execute_reply": "2023-12-20T16:44:19.034029Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "dJ_dm_one approximation = 0.020351453248329543\n" + ] + } + ], + "source": [ + "def dJ_dm(J, m, *, eps=1.0e-7):\n", + " return (J(m + eps) - J(m - eps)) / (2.0 * eps)\n", + "\n", + "\n", + "print(f\"dJ_dm_one approximation = {dJ_dm(lambda eps: float(forward(m + eps * one)), 0.0)}\")" + ] + }, + { + "cell_type": "markdown", + "id": "1819f89e", + "metadata": {}, + "source": [ + "## Computing Hessian information using an adjoint of a tangent-linear\n", + "\n", + "### Single direction\n", + "\n", + "We next seek to compute the action of the Hessian of the functional on some direction $\\zeta \\in V$, using the adjoint method applied to tangent-linear and forward calculations. This can be handled directly, by configuring the relevant tangent-linear and computing the derivative using `compute_gradient`:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "14f090f0", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:19.037004Z", + "iopub.status.busy": "2023-12-20T16:44:19.036810Z", + "iopub.status.idle": "2023-12-20T16:44:20.486281Z", + "shell.execute_reply": "2023-12-20T16:44:20.485560Z" + } + }, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta = var_tlm(J, (m, zeta))\n", + "\n", + "d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)" + ] + }, + { + "cell_type": "markdown", + "id": "5757194e", + "metadata": {}, + "source": [ + "The `Hessian` class applies the same approach, but handles several of the steps for us:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "49588ff6", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:20.488852Z", + "iopub.status.busy": "2023-12-20T16:44:20.488650Z", + "iopub.status.idle": "2023-12-20T16:44:20.615551Z", + "shell.execute_reply": "2023-12-20T16:44:20.614947Z" + } + }, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "H = Hessian(forward)\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "_, dJ_dm_zeta, d2J_dm_zeta_dm = H.action(m, zeta)" + ] + }, + { + "cell_type": "markdown", + "id": "a78eac8e", + "metadata": {}, + "source": [ + "### Multiple directions\n", + "\n", + "If we want to compute the Hessian action on *multiple* directions we can define multiple tangent-linears:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "e23e1725", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:20.618211Z", + "iopub.status.busy": "2023-12-20T16:44:20.618008Z", + "iopub.status.idle": "2023-12-20T16:44:21.284046Z", + "shell.execute_reply": "2023-12-20T16:44:21.283442Z" + } + }, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta_0 = Function(space, name=\"zeta_0\")\n", + "zeta_0.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta_0))\n", + "\n", + "zeta_1 = Function(space, name=\"zeta_1\")\n", + "zeta_1.interpolate(sin(pi * X[0]) * sin(2.0 * pi * X[1]))\n", + "configure_tlm((m, zeta_1))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta_0 = var_tlm(J, (m, zeta_0))\n", + "dJ_dm_zeta_1 = var_tlm(J, (m, zeta_1))\n", + "\n", + "d2J_dm_zeta_0_dm, d2J_dm_zeta_1_dm = compute_gradient((dJ_dm_zeta_0, dJ_dm_zeta_1), m)" + ] + }, + { + "cell_type": "markdown", + "id": "94aa1773", + "metadata": {}, + "source": [ + "There are now calculations for two sets of tangent-linear variables, two sets of first order adjoint variables, and two sets of second order adjoint variables. However the two sets of first order adjoint variables have the same values – by default tlm_adjoint detects this and only computes them once.\n", + "\n", + "The above approach requires us to know the directions *before* the forward calculation. However some algorithms can generate the directions sequentially, and we do not know the next direction until a Hessian action on the previous direction has been computed. If possible we still want to avoid re-running the forward calculation each time we have a new direction.\n", + "\n", + "If we have sufficient memory available, and in particular so long as we do not need to use checkpointing for the adjoint calculations, we can make use of the `CachedHessian` class. This stores the forward solution and, by default, caches and reuses first order adjoint values. Here we do *not* need to configure the tangent-linear before the forward calculation – instead tlm_adjoint performs the tangent-linear calculations *after* the forward calculations:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "9daddace", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:21.287092Z", + "iopub.status.busy": "2023-12-20T16:44:21.286877Z", + "iopub.status.idle": "2023-12-20T16:44:21.920712Z", + "shell.execute_reply": "2023-12-20T16:44:21.920124Z" + } + }, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "H = CachedHessian(J)\n", + "\n", + "zeta_0 = Function(space, name=\"zeta_0\")\n", + "zeta_0.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "zeta_1 = Function(space, name=\"zeta_1\")\n", + "zeta_1.interpolate(sin(pi * X[0]) * sin(2.0 * pi * X[1]))\n", + "\n", + "_, dJ_dm_zeta_0, d2J_dm_zeta_0_dm = H.action(m, zeta_0)\n", + "_, dJ_dm_zeta_1, d2J_dm_zeta_1_dm = H.action(m, zeta_1)" + ] + }, + { + "cell_type": "markdown", + "id": "dc0f8ae8", + "metadata": {}, + "source": [ + "## Assembly and solver caching\n", + "\n", + "### Using an `EquationSolver`\n", + "\n", + "The calculation for the Hessian action includes four discrete Poisson equations: one for the original forward calculation, one for the tangent-linear calculation, and one each for first and second order adjoint calculations. In this self-adjoint problem the finite element matrix – a stiffness matrix – is the *same* across all four calculations. Hence we can cache and reuse it. Moreover we can cache and reuse linear solver data – for example we can cache and reuse the Cholesky factorization.\n", + "\n", + "tlm_adjoint can apply such caching automatically, but we must interact directly with the object tlm_adjoint uses to record the solution of finite element variational problems – the `EquationSolver` previously seen when we used `manager_info()`. This looks like:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "04d2a68c", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:21.923784Z", + "iopub.status.busy": "2023-12-20T16:44:21.923564Z", + "iopub.status.idle": "2023-12-20T16:44:22.895863Z", + "shell.execute_reply": "2023-12-20T16:44:22.895266Z" + } + }, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "clear_caches()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " eq = EquationSolver(\n", + " inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " HomogeneousDirichletBC(space, \"on_boundary\"),\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"cholesky\"})\n", + " eq.solve()\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta = var_tlm(J, (m, zeta))\n", + "\n", + "d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)" + ] + }, + { + "cell_type": "markdown", + "id": "35c85f0d", + "metadata": {}, + "source": [ + "The key changes here are:\n", + "\n", + "- The use of `clear_caches`. This ensures that any previously cached data is cleared, avoiding memory leaks if the code is run more than once.\n", + "- The use of `HomogeneousDirichletBC`. This tells tlm_adjoint that the boundary condition is homogeneous, and helps it detect that forward and adjoint problems have the same boundary conditions.\n", + "- The instantiation of an `EquationSolver`, and the call to its `solve` method.\n", + "\n", + "If we query the relevant tlm_adjoint caches we find:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "4023b690", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:22.898624Z", + "iopub.status.busy": "2023-12-20T16:44:22.898412Z", + "iopub.status.idle": "2023-12-20T16:44:22.902200Z", + "shell.execute_reply": "2023-12-20T16:44:22.901660Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "len(assembly_cache())=1\n", + "len(linear_solver_cache())=1\n" + ] + } + ], + "source": [ + "print(f\"{len(assembly_cache())=}\")\n", + "print(f\"{len(linear_solver_cache())=}\")\n", + "\n", + "assert len(assembly_cache()) == 1\n", + "assert len(linear_solver_cache()) == 1" + ] + }, + { + "cell_type": "markdown", + "id": "8c2a42f1", + "metadata": {}, + "source": [ + "and in particular we see that tlm_adjoint has cached data associated with a single matrix, and has cached a single assembled object (which turns out to be the matrix itself). The latter is a stiffness matrix, and the former stores its Cholesky factorization. The factorization is used four times: in the forward, tangent-linear, and first and second order adjoint calculations.\n", + "\n", + "### Flagging data for caching\n", + "\n", + "Now consider the slightly different calculation:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "e8fbfb7c", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:22.904864Z", + "iopub.status.busy": "2023-12-20T16:44:22.904656Z", + "iopub.status.idle": "2023-12-20T16:44:24.937558Z", + "shell.execute_reply": "2023-12-20T16:44:24.936983Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "len(assembly_cache())=0\n", + "len(linear_solver_cache())=0\n" + ] + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "clear_caches()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "one = Constant(1.0, name=\"one\")\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " eq = EquationSolver(\n", + " one * inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " HomogeneousDirichletBC(space, \"on_boundary\"),\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"cholesky\"})\n", + " eq.solve()\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta = var_tlm(J, (m, zeta))\n", + "\n", + "d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)\n", + "\n", + "print(f\"{len(assembly_cache())=}\")\n", + "print(f\"{len(linear_solver_cache())=}\")\n", + "\n", + "assert len(assembly_cache()) == 0\n", + "assert len(linear_solver_cache()) == 0" + ] + }, + { + "cell_type": "markdown", + "id": "78f750ac", + "metadata": {}, + "source": [ + "The only difference is the introduction of the multiplication by `one` on the left-hand-side of the finite element variational problem. However we now find that no matrix or linear solver data has been cached. The issue is that tlm_adjoint does not know that it should cache the results of calculations involving `one`.\n", + "\n", + "#### The 'cache' flag\n", + "\n", + "To resolve this, we can flag `one` for caching using `cache=True`:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "41227987", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:24.940138Z", + "iopub.status.busy": "2023-12-20T16:44:24.939932Z", + "iopub.status.idle": "2023-12-20T16:44:26.188732Z", + "shell.execute_reply": "2023-12-20T16:44:26.188005Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "len(assembly_cache())=2\n", + "len(linear_solver_cache())=1\n" + ] + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "clear_caches()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "one = Constant(1.0, name=\"one\", cache=True)\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " eq = EquationSolver(\n", + " one * inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " HomogeneousDirichletBC(space, \"on_boundary\"),\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"cholesky\"})\n", + " eq.solve()\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta = var_tlm(J, (m, zeta))\n", + "\n", + "d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)\n", + "\n", + "print(f\"{len(assembly_cache())=}\")\n", + "print(f\"{len(linear_solver_cache())=}\")\n", + "\n", + "assert len(assembly_cache()) == 2\n", + "assert len(linear_solver_cache()) == 1" + ] + }, + { + "cell_type": "markdown", + "id": "38406c1d", + "metadata": {}, + "source": [ + "We now see that tlm_adjoint has cached linear solver data associated with a single matrix. However assembly of two objects has been cached – it turns out there are now *two* cached matrices.\n", + "\n", + "The extra cached matrix appears in the tangent-linear calculations, involving the tangent-linear variable associated with `one` – a tangent-linear right-hand-side term has been converted into a matrix multiply using a *different* matrix. However in the above calculation we know that this tangent-linear variable must be zero, since the calculation for `one` doesn't depend on the control variable. The extra term in the tangent-linear calculation is similarly also known to be zero.\n", + "\n", + "#### The 'static' flag\n", + "\n", + "We can let tlm_adjoint know that `one` does not change, and resolve this inefficiency, by instead using `static=True`:" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "2642124c", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:26.191398Z", + "iopub.status.busy": "2023-12-20T16:44:26.191185Z", + "iopub.status.idle": "2023-12-20T16:44:26.319693Z", + "shell.execute_reply": "2023-12-20T16:44:26.319063Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "len(assembly_cache())=1\n", + "len(linear_solver_cache())=1\n" + ] + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "clear_caches()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(X[0] * X[1])\n", + "\n", + "one = Constant(1.0, name=\"one\", static=True)\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " eq = EquationSolver(\n", + " one * inner(grad(trial), grad(test)) * dx == inner(m * m, test) * dx, u,\n", + " HomogeneousDirichletBC(space, \"on_boundary\"),\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"cholesky\"})\n", + " eq.solve()\n", + "\n", + " J_sq = Functional(name=\"J_sq\")\n", + " J_sq.assign(inner(u, u) * dx)\n", + " J = np.sqrt(J_sq)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.interpolate(sin(pi * X[0]) * sin(pi * X[1]))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm_zeta = var_tlm(J, (m, zeta))\n", + "\n", + "d2J_dm_zeta_dm = compute_gradient(dJ_dm_zeta, m)\n", + "\n", + "print(f\"{len(assembly_cache())=}\")\n", + "print(f\"{len(linear_solver_cache())=}\")\n", + "\n", + "assert len(assembly_cache()) == 1\n", + "assert len(linear_solver_cache()) == 1" + ] + }, + { + "cell_type": "markdown", + "id": "5c96f6ae", + "metadata": {}, + "source": [ + "Here `static=True` leads to `one` being flagged as a variable whose value is never updated. From this tlm_adjoint can infer that the relevant associated tangent-linear variable is zero, and avoid adding the zero-valued tangent-linear term.\n", + "\n", + "The key difference between using `cache=True` and `static=True` is that in the former the value of the variable *may* be updated. So long as tlm_adjoint is aware of the update (which happens, for example, when tlm_adjoint records a calculation) then updating the value of a variable invalidates cache entries, and invalidated cache entries are cleared automatically." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/2_verification.html b/examples/2_verification.html new file mode 100644 index 0000000..a5c8edf --- /dev/null +++ b/examples/2_verification.html @@ -0,0 +1,424 @@ + + + + + + + Verifying derivative calculations — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Verifying derivative calculations

+

This notebook describes the verification of derivative calculations using Taylor remainder convergence testing. A simple time-independent problem is considered, using tlm_adjoint with the Firedrake backend.

+

The Taylor remainder convergence testing method is described in:

+
    +

  • P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, ‘Automated derivation of the adjoint of high-level transient finite element programs’, SIAM Journal on Scientific Computing 35(4), pp. C369–C393, 2013, doi: 10.1137/120873558

    • +
+
+

Forward problem

+

We consider the solution of a linear time-independent partial differential equation, followed by the calculation of the square of the \(L^2\)-norm of the solution. Non-linearity is introduced by defining the right-hand-side of the problem to be a non-linear function of the control. We assume real spaces and a real build of Firedrake throughout.

+

Specifically we consider the solution \(u \in V\) of

+
+\[\forall \zeta \in V \qquad \int_\Omega u \zeta + \alpha^2 \int_\Omega \nabla u \cdot \nabla \zeta = \int_\Omega \left( \sin^2 m \right) \zeta,\]
+

where \(V\) is a real \(P_1\) continuous finite element space defining functions on the domain \(\Omega = \left( 0, 1 \right)^2\), with \(m \in V\). This corresponds to a discretization of the partial differential equation

+
+\[u - \alpha^2 \nabla^2 u = \sin^2 m \quad \text{on } \left( x, y \right) \in \left( 0, 1 \right)^2,\]
+

subject to boundary conditions \(\nabla u \cdot \hat{n} = 0\) on the boundary, where \(\hat{n}\) is an outward unit normal.

+

A simple implementation in Firedrake, with \(m = e^x \sin \left( \pi x y \right)\) and \(\alpha = 0.2\), takes the form:

+
+
[1]:
+
+
+
from firedrake import *
+
+mesh = UnitSquareMesh(50, 50)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(exp(X[0]) * sin(pi * X[0] * X[1]))
+
+alpha = Constant(0.2)
+
+u = Function(space, name="u")
+solve(inner(trial, test) * dx + (alpha ** 2) * inner(grad(trial), grad(test)) * dx
+      == inner(sin(m) ** 2, test) * dx, u)
+
+J = assemble(inner(u, u) * dx)
+
+
+
+
+
+

Taylor remainder convergence testing: First order

+

If we have a functional \(J\) depending on a control \(m\) then we have, given some perturbation direction \(\zeta\), via Taylor expansion,

+
+\[\begin{split}\left| J \left( m + \varepsilon \zeta \right) - J \left( m \right) \right| = O \left( \varepsilon \right), \\\end{split}\]
+
+\[\left| J \left( m + \varepsilon \zeta \right) - J \left( m \right) - \varepsilon \frac{dJ}{dm} \zeta \right| = O \left( \varepsilon^2 \right).\]
+

That is, \(\zeta\) is some direction in the same space as \(m\), which we choose, and then we control the perturbation amplitude using the scalar \(\varepsilon\). The final term in the second absolute value is a directional derivative, which we can compute using the adjoint.

+

This leads to a methodology for verifying a derivative computed using the adjoint method:

+
    +

  1. Choose a direction \(\zeta\).

    2. +

  2. Choose a number of different values of \(\varepsilon\).

    3. +

  3. See if we have second order convergence of the second of the above, to zero.

    4. +
+

This verifies only the directional derivative with a single direction, but if we wish we can choose a new direction and repeat the test.

+

We can use the taylor_test function to perform the test for us. By default this generates a pseudorandom direction and chooses a number of values of \(\varepsilon\). It then computes the quantities on the left-hand-sides of the above equations, computes the orders of convergence between consecutive pairs of values for \(\varepsilon\), and displays the results. It returns the minimum order computed for the second case, which in a successful verification should be close to two.

+

Let’s compute a derivative using the adjoint method, and apply a Taylor remainder convergence test:

+
+
[2]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import logging
+import numpy as np
+
+np.random.seed(33582866)
+
+logger = logging.getLogger("tlm_adjoint")
+logger.setLevel(logging.INFO)
+root_logger = logging.getLogger()
+if len(logger.handlers) == 1:
+    if len(root_logger.handlers) == 1:
+        root_logger.handlers.pop()
+    root_logger.addHandler(logger.handlers.pop())
+
+reset_manager()
+
+mesh = UnitSquareMesh(50, 50)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+m = Function(space, name="m")
+m.interpolate(exp(X[0]) * sin(pi * X[0] * X[1]))
+
+alpha = Constant(0.2)
+
+
+def forward(m):
+    u = Function(space, name="u")
+    solve(inner(trial, test) * dx + (alpha ** 2) * inner(grad(trial), grad(test)) * dx
+          == inner(sin(m) ** 2, test) * dx, u)
+
+    J = Functional(name="J")
+    J.assign(inner(u, u) * dx)
+    return J
+
+
+start_manager()
+J = forward(m)
+stop_manager()
+
+dJ_dm = compute_gradient(J, m)
+
+min_order = taylor_test(forward, m, J_val=J.value, dJ=dJ_dm, seed=1.0e-3)
+assert min_order > 1.99
+
+
+
+
+
+
+
+
+Error norms, no adjoint   = [1.83675325e-04 9.18679773e-05 4.59415781e-05 2.29726877e-05
+ 1.14868187e-05]
+Orders,      no adjoint   = [0.99952386 0.99976165 0.99988076 0.99994036]
+Error norms, with adjoint = [1.21373327e-07 3.03720079e-08 7.59658259e-09 1.89959287e-09
+ 4.74953355e-10]
+Orders,      with adjoint = [1.99863722 1.99931983 1.9996603  1.99983251]
+
+
+

The key changes here are:

+
    +

  • To define a forward function. taylor_test uses this to repeatedly rerun the forward with different values of \(\varepsilon\).

    • +

  • Using seed to control the considered values of \(\varepsilon\). If this is too large then the asymptotic orders of convergence may not be seen. If this is too small then the effect of roundoff or iterative solver tolerances may become too large.

    • +

  • Seeding the NumPy pseudorandom number generator. The pseudorandom direction is generated using numpy.random.random, and we seed the pseudorandom number generator to improve reproducibility. We could alternatively supply a direction using the dM argument.

    • +
+

We see the expected first and second orders of convergence.

+
+
+

Taylor remainder convergence testing: Second order

+

Including the next order term in the Taylor expansion leads to

+
+\[\left| J \left( m + \varepsilon \zeta \right) - J \left( m \right) - \varepsilon \frac{dJ}{dm} \zeta - \frac{1}{2} \varepsilon^2 \left[ \frac{d}{dm} \left( \frac{dJ}{dm} \zeta \right) \right] \zeta \right| = O \left( \varepsilon^3 \right).\]
+

Let’s use this approach to test Hessian calculations using a CachedHessian:

+
+
[3]:
+
+
+
np.random.seed(19986557)
+
+H = CachedHessian(J)
+
+min_order = taylor_test(forward, m, J_val=J.value, ddJ=H, seed=1.0e-3)
+assert min_order > 3.00
+
+
+
+
+
+
+
+
+Error norms, no adjoint   = [2.10559794e-04 1.05304003e-04 5.26580325e-05 2.63305245e-05
+ 1.31656394e-05]
+Orders,      no adjoint   = [0.9996697  0.99983476 0.99991736 0.99995867]
+Error norms, with adjoint = [9.17834456e-11 1.14548609e-11 1.43110530e-12 1.78548691e-13
+ 2.19004148e-14]
+Orders,      with adjoint = [3.00227403 3.0007582  3.00274037 3.02728746]
+
+
+

We now see the expected first and third orders of convergence, although there is a suggestion of roundoff affecting the results for the smallest \(\varepsilon\). Here the first order directional derivative is computed using a tangent-linear calculation, and the Hessian action on \(\zeta\) is computed by applying the adjoint method to the forward and tangent-linear calculations.

+
+
+

Taylor remainder convergence testing: Higher order

+

We can test the derivative of a directional derivative, if we substitute

+
+\[J \rightarrow K = \frac{dJ}{dm} \zeta_0,\]
+

with some new direction \(\zeta_0\), which we choose. That is, we use

+
+\[\begin{split}\left| K \left( m + \varepsilon \zeta \right) - K \left( m \right) \right| = O \left( \varepsilon \right), \\\end{split}\]
+
+\[\left| K \left( m + \varepsilon \zeta \right) - K \left( m \right) - \varepsilon \frac{dK}{dm} \zeta \right| = O \left( \varepsilon^2 \right),\]
+

with

+
+\[K = \frac{dJ}{dm} \zeta_0.\]
+

The new term

+
+\[\frac{dK}{dm} \zeta\]
+

can be computed using either a higher order tangent-linear or higher-order adjoint calculation. This generalizes naturally to higher order, by replacing the functional with the directional derivative of a directional derivative.

+

The function taylor_test_tlm performs such verification tests, considering directional derivatives of a given order, and computing all derivatives using tangent-linear calculations. Each directional derivative requires a new direction to be chosen – by default pseudorandom directions are generated.

+

Let’s apply this test up to fourth order:

+
+
[4]:
+
+
+
np.random.seed(76149511)
+
+for order in range(1, 5):
+    min_order = taylor_test_tlm(forward, m, tlm_order=order, seed=1.0e-3)
+    assert min_order > 1.99
+
+
+
+
+
+
+
+
+Error norms, no tangent-linear   = [2.14011610e-04 1.07017190e-04 5.35114451e-05 2.67564355e-05
+ 1.33783961e-05]
+Orders,      no tangent-linear   = [0.99984651 0.99992316 0.99996156 0.99998077]
+Error norms, with tangent-linear = [4.55807682e-08 1.14053926e-08 2.85262144e-09 7.13314473e-10
+ 1.78348619e-10]
+Orders,      with tangent-linear = [1.99870913 1.9993559  1.99967815 1.9998382 ]
+Error norms, no tangent-linear   = [6.50091607e-05 3.25462282e-05 1.62835054e-05 8.14434796e-06
+ 4.07282247e-06]
+Orders,      no tangent-linear   = [0.99815267 0.99907905 0.9995402  0.99977027]
+Error norms, with tangent-linear = [1.66371223e-07 4.15377915e-08 1.03775510e-08 2.59352448e-09
+ 6.48273139e-10]
+Orders,      with tangent-linear = [2.0019095  2.00095849 2.00048012 2.00024029]
+Error norms, no tangent-linear   = [1.37224954e-04 6.85404163e-05 3.42520880e-05 1.71215009e-05
+ 8.55961300e-06]
+Orders,      no tangent-linear   = [1.001516   1.00076302 1.00038276 1.0001917 ]
+Error norms, with tangent-linear = [2.89362317e-07 7.26204003e-08 1.81900660e-08 4.55188642e-09
+ 1.13851782e-09]
+Orders,      with tangent-linear = [1.99443027 1.99722411 1.99861431 1.99930769]
+
+
+
+
+
+
+
+tsfc:WARNING Estimated quadrature degree 11 more than tenfold greater than any argument/coefficient degree (max 1)
+
+
+
+
+
+
+
+Estimated quadrature degree 11 more than tenfold greater than any argument/coefficient degree (max 1)
+Error norms, no tangent-linear   = [2.86571468e-04 1.43890711e-04 7.20965028e-05 3.60860261e-05
+ 1.80524552e-05]
+Orders,      no tangent-linear   = [0.99392152 0.99697228 0.998489   0.99924522]
+Error norms, with tangent-linear = [2.41887274e-06 6.04459357e-07 1.51082379e-07 3.77665303e-08
+ 9.44112417e-09]
+Orders,      with tangent-linear = [2.00061762 2.00030994 2.00015525 2.00007769]
+
+
+

The function taylor_test_tlm_adjoint also performs such verification tests, but computes the highest order derivative information using the adjoint method.

+

Let’s apply this test up to fourth order:

+
+
[5]:
+
+
+
np.random.seed(74728054)
+
+for order in range(1, 5):
+    min_order = taylor_test_tlm_adjoint(forward, m, adjoint_order=order, seed=1.0e-3)
+    assert min_order > 1.99
+
+
+
+
+
+
+
+
+Error norms, no adjoint   = [1.96925244e-04 9.84831481e-05 4.92467146e-05 2.46246436e-05
+ 1.23126435e-05]
+Orders,      no adjoint   = [0.99969928 0.9998494  0.99992464 0.9999623 ]
+Error norms, with adjoint = [8.22009804e-08 2.05743897e-08 5.14661321e-09 1.28703002e-09
+ 3.21803907e-10]
+Orders,      with adjoint = [1.99830597 1.99915437 1.99957766 1.99979196]
+Error norms, no adjoint   = [6.06217536e-05 3.03568452e-05 1.51898950e-05 7.59781316e-06
+ 3.79962268e-06]
+Orders,      no adjoint   = [0.99781372 0.99890997 0.99945576 0.99972807]
+Error norms, with adjoint = [1.83663729e-07 4.58635021e-08 1.14593026e-08 2.86400265e-09
+ 7.15897696e-10]
+Orders,      with adjoint = [2.00164832 2.00082728 2.00041451 2.00020749]
+Error norms, no adjoint   = [1.77271516e-04 8.85216211e-05 4.42321703e-05 2.21089118e-05
+ 1.10526609e-05]
+Orders,      no adjoint   = [1.00185896 1.00093384 1.00046801 1.00023428]
+Error norms, with adjoint = [4.57678459e-07 1.14702543e-07 2.87109831e-08 7.18216298e-09
+ 1.79609281e-09]
+Orders,      with adjoint = [1.99643702 1.99822274 1.99911243 1.99955649]
+
+
+
+
+
+
+
+tsfc:WARNING Estimated quadrature degree 11 more than tenfold greater than any argument/coefficient degree (max 1)
+
+
+
+
+
+
+
+Estimated quadrature degree 11 more than tenfold greater than any argument/coefficient degree (max 1)
+Error norms, no adjoint   = [2.68076588e-04 1.34645957e-04 6.74748172e-05 3.37753585e-05
+ 1.68971655e-05]
+Orders,      no adjoint   = [0.99347432 0.99674984 0.99837808 0.99918983]
+Error norms, with adjoint = [2.42982799e-06 6.07250783e-07 1.51786806e-07 3.79434579e-08
+ 9.48545885e-09]
+Orders,      with adjoint = [2.00048984 2.00024606 2.00012332 2.00006169]
+
+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/examples/2_verification.ipynb b/examples/2_verification.ipynb new file mode 100644 index 0000000..c5e640a --- /dev/null +++ b/examples/2_verification.ipynb @@ -0,0 +1,661 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "92f4f706", + "metadata": {}, + "source": [ + "# Verifying derivative calculations\n", + "\n", + "This notebook describes the verification of derivative calculations using Taylor remainder convergence testing. A simple time-independent problem is considered, using tlm_adjoint with the [Firedrake](https://firedrakeproject.org/) backend.\n", + "\n", + "The Taylor remainder convergence testing method is described in:\n", + "\n", + "- P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes, 'Automated derivation of the adjoint of high-level transient finite element programs', SIAM Journal on Scientific Computing 35(4), pp. C369–C393, 2013, doi: 10.1137/120873558\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the solution of a linear time-independent partial differential equation, followed by the calculation of the square of the $L^2$-norm of the solution. Non-linearity is introduced by defining the right-hand-side of the problem to be a non-linear function of the control. We assume real spaces and a real build of Firedrake throughout.\n", + "\n", + "Specifically we consider the solution $u \\in V$ of\n", + "\n", + "$$\n", + " \\forall \\zeta \\in V \\qquad \\int_\\Omega u \\zeta + \\alpha^2 \\int_\\Omega \\nabla u \\cdot \\nabla \\zeta = \\int_\\Omega \\left( \\sin^2 m \\right) \\zeta,\n", + "$$\n", + "\n", + "where $V$ is a real $P_1$ continuous finite element space defining functions on the domain $\\Omega = \\left( 0, 1 \\right)^2$, with $m \\in V$. This corresponds to a discretization of the partial differential equation\n", + "\n", + "$$\n", + " u - \\alpha^2 \\nabla^2 u = \\sin^2 m \\quad \\text{on } \\left( x, y \\right) \\in \\left( 0, 1 \\right)^2,\n", + "$$\n", + "\n", + "subject to boundary conditions $\\nabla u \\cdot \\hat{n} = 0$ on the boundary, where $\\hat{n}$ is an outward unit normal.\n", + "\n", + "A simple implementation in Firedrake, with $m = e^x \\sin \\left( \\pi x y \\right)$ and $\\alpha = 0.2$, takes the form:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "13fd669c", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:29.288616Z", + "iopub.status.busy": "2023-12-20T16:44:29.287891Z", + "iopub.status.idle": "2023-12-20T16:44:34.017886Z", + "shell.execute_reply": "2023-12-20T16:44:34.017254Z" + } + }, + "outputs": [], + "source": [ + "from firedrake import *\n", + "\n", + "mesh = UnitSquareMesh(50, 50)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(exp(X[0]) * sin(pi * X[0] * X[1]))\n", + "\n", + "alpha = Constant(0.2)\n", + "\n", + "u = Function(space, name=\"u\")\n", + "solve(inner(trial, test) * dx + (alpha ** 2) * inner(grad(trial), grad(test)) * dx\n", + " == inner(sin(m) ** 2, test) * dx, u)\n", + "\n", + "J = assemble(inner(u, u) * dx)" + ] + }, + { + "cell_type": "markdown", + "id": "29810cbf", + "metadata": {}, + "source": [ + "## Taylor remainder convergence testing: First order\n", + "\n", + "If we have a functional $J$ depending on a control $m$ then we have, given some perturbation direction $\\zeta$, via Taylor expansion,\n", + "\n", + "$$\n", + " \\left| J \\left( m + \\varepsilon \\zeta \\right) - J \\left( m \\right) \\right| = O \\left( \\varepsilon \\right), \\\\\n", + "$$\n", + "$$\n", + " \\left| J \\left( m + \\varepsilon \\zeta \\right) - J \\left( m \\right) - \\varepsilon \\frac{dJ}{dm} \\zeta \\right| = O \\left( \\varepsilon^2 \\right).\n", + "$$\n", + "\n", + "That is, $\\zeta$ is some direction in the same space as $m$, which we choose, and then we control the perturbation amplitude using the scalar $\\varepsilon$. The final term in the second absolute value is a directional derivative, which we can compute using the adjoint.\n", + "\n", + "This leads to a methodology for verifying a derivative computed using the adjoint method:\n", + "\n", + "1. Choose a direction $\\zeta$.\n", + "2. Choose a number of different values of $\\varepsilon$.\n", + "3. See if we have second order convergence of the second of the above, to zero.\n", + "\n", + "This verifies only the directional derivative with a single direction, but if we wish we can choose a new direction and repeat the test.\n", + "\n", + "We can use the `taylor_test` function to perform the test for us. By default this generates a pseudorandom direction and chooses a number of values of $\\varepsilon$. It then computes the quantities on the left-hand-sides of the above equations, computes the orders of convergence between consecutive pairs of values for $\\varepsilon$, and displays the results. It returns the *minimum* order computed for the second case, which in a successful verification should be close to two.\n", + "\n", + "Let's compute a derivative using the adjoint method, and apply a Taylor remainder convergence test:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "1888394c", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:34.020747Z", + "iopub.status.busy": "2023-12-20T16:44:34.020378Z", + "iopub.status.idle": "2023-12-20T16:44:35.242766Z", + "shell.execute_reply": "2023-12-20T16:44:35.242285Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [1.83675325e-04 9.18679773e-05 4.59415781e-05 2.29726877e-05\n", + " 1.14868187e-05]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [0.99952386 0.99976165 0.99988076 0.99994036]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [1.21373327e-07 3.03720079e-08 7.59658259e-09 1.89959287e-09\n", + " 4.74953355e-10]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [1.99863722 1.99931983 1.9996603 1.99983251]\n" + ] + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import logging\n", + "import numpy as np\n", + "\n", + "np.random.seed(33582866)\n", + "\n", + "logger = logging.getLogger(\"tlm_adjoint\")\n", + "logger.setLevel(logging.INFO)\n", + "root_logger = logging.getLogger()\n", + "if len(logger.handlers) == 1:\n", + " if len(root_logger.handlers) == 1:\n", + " root_logger.handlers.pop()\n", + " root_logger.addHandler(logger.handlers.pop())\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(50, 50)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(exp(X[0]) * sin(pi * X[0] * X[1]))\n", + "\n", + "alpha = Constant(0.2)\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(trial, test) * dx + (alpha ** 2) * inner(grad(trial), grad(test)) * dx\n", + " == inner(sin(m) ** 2, test) * dx, u)\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u, u) * dx)\n", + " return J\n", + "\n", + "\n", + "start_manager()\n", + "J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ_dm = compute_gradient(J, m)\n", + "\n", + "min_order = taylor_test(forward, m, J_val=J.value, dJ=dJ_dm, seed=1.0e-3)\n", + "assert min_order > 1.99" + ] + }, + { + "cell_type": "markdown", + "id": "dcdd1937", + "metadata": {}, + "source": [ + "The key changes here are:\n", + "\n", + "- To define a `forward` function. `taylor_test` uses this to repeatedly rerun the forward with different values of $\\varepsilon$.\n", + "- Using `seed` to control the considered values of $\\varepsilon$. If this is too large then the asymptotic orders of convergence may not be seen. If this is too small then the effect of roundoff or iterative solver tolerances may become too large.\n", + "- Seeding the NumPy pseudorandom number generator. The pseudorandom direction is generated using `numpy.random.random`, and we seed the pseudorandom number generator to improve reproducibility. We could alternatively supply a direction using the `dM` argument.\n", + "\n", + "We see the expected first and second orders of convergence.\n", + "\n", + "## Taylor remainder convergence testing: Second order\n", + "\n", + "Including the next order term in the Taylor expansion leads to\n", + "\n", + "$$\n", + " \\left| J \\left( m + \\varepsilon \\zeta \\right) - J \\left( m \\right) - \\varepsilon \\frac{dJ}{dm} \\zeta - \\frac{1}{2} \\varepsilon^2 \\left[ \\frac{d}{dm} \\left( \\frac{dJ}{dm} \\zeta \\right) \\right] \\zeta \\right| = O \\left( \\varepsilon^3 \\right).\n", + "$$\n", + "\n", + "Let's use this approach to test Hessian calculations using a `CachedHessian`:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "5b6ac48d", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:35.245302Z", + "iopub.status.busy": "2023-12-20T16:44:35.244944Z", + "iopub.status.idle": "2023-12-20T16:44:37.101242Z", + "shell.execute_reply": "2023-12-20T16:44:37.100589Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [2.10559794e-04 1.05304003e-04 5.26580325e-05 2.63305245e-05\n", + " 1.31656394e-05]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [0.9996697 0.99983476 0.99991736 0.99995867]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [9.17834456e-11 1.14548609e-11 1.43110530e-12 1.78548691e-13\n", + " 2.19004148e-14]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [3.00227403 3.0007582 3.00274037 3.02728746]\n" + ] + } + ], + "source": [ + "np.random.seed(19986557)\n", + "\n", + "H = CachedHessian(J)\n", + "\n", + "min_order = taylor_test(forward, m, J_val=J.value, ddJ=H, seed=1.0e-3)\n", + "assert min_order > 3.00" + ] + }, + { + "cell_type": "markdown", + "id": "a0435ee4", + "metadata": {}, + "source": [ + "We now see the expected first and *third* orders of convergence, although there is a suggestion of roundoff affecting the results for the smallest $\\varepsilon$. Here the first order directional derivative is computed using a tangent-linear calculation, and the Hessian action on $\\zeta$ is computed by applying the adjoint method to the forward and tangent-linear calculations.\n", + "\n", + "## Taylor remainder convergence testing: Higher order\n", + "\n", + "We can test the derivative of a directional derivative, if we substitute\n", + "\n", + "$$\n", + " J \\rightarrow K = \\frac{dJ}{dm} \\zeta_0,\n", + "$$\n", + "\n", + "with some *new* direction $\\zeta_0$, which we choose. That is, we use\n", + "\n", + "$$\n", + " \\left| K \\left( m + \\varepsilon \\zeta \\right) - K \\left( m \\right) \\right| = O \\left( \\varepsilon \\right), \\\\\n", + "$$\n", + "$$\n", + " \\left| K \\left( m + \\varepsilon \\zeta \\right) - K \\left( m \\right) - \\varepsilon \\frac{dK}{dm} \\zeta \\right| = O \\left( \\varepsilon^2 \\right),\n", + "$$\n", + "\n", + "with\n", + "\n", + "$$\n", + " K = \\frac{dJ}{dm} \\zeta_0.\n", + "$$\n", + "\n", + "The new term\n", + "\n", + "$$\n", + " \\frac{dK}{dm} \\zeta\n", + "$$\n", + "\n", + "can be computed using either a higher order tangent-linear or higher-order adjoint calculation. This generalizes naturally to higher order, by replacing the functional with the directional derivative of a directional derivative.\n", + "\n", + "The function `taylor_test_tlm` performs such verification tests, considering directional derivatives of a given order, and computing all derivatives using tangent-linear calculations. Each directional derivative requires a new direction to be chosen – by default pseudorandom directions are generated.\n", + "\n", + "Let's apply this test up to fourth order:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "4e5bf325", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:37.103788Z", + "iopub.status.busy": "2023-12-20T16:44:37.103590Z", + "iopub.status.idle": "2023-12-20T16:44:57.683003Z", + "shell.execute_reply": "2023-12-20T16:44:57.682286Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no tangent-linear = [2.14011610e-04 1.07017190e-04 5.35114451e-05 2.67564355e-05\n", + " 1.33783961e-05]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no tangent-linear = [0.99984651 0.99992316 0.99996156 0.99998077]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with tangent-linear = [4.55807682e-08 1.14053926e-08 2.85262144e-09 7.13314473e-10\n", + " 1.78348619e-10]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with tangent-linear = [1.99870913 1.9993559 1.99967815 1.9998382 ]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no tangent-linear = [6.50091607e-05 3.25462282e-05 1.62835054e-05 8.14434796e-06\n", + " 4.07282247e-06]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no tangent-linear = [0.99815267 0.99907905 0.9995402 0.99977027]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with tangent-linear = [1.66371223e-07 4.15377915e-08 1.03775510e-08 2.59352448e-09\n", + " 6.48273139e-10]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with tangent-linear = [2.0019095 2.00095849 2.00048012 2.00024029]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no tangent-linear = [1.37224954e-04 6.85404163e-05 3.42520880e-05 1.71215009e-05\n", + " 8.55961300e-06]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no tangent-linear = [1.001516 1.00076302 1.00038276 1.0001917 ]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with tangent-linear = [2.89362317e-07 7.26204003e-08 1.81900660e-08 4.55188642e-09\n", + " 1.13851782e-09]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with tangent-linear = [1.99443027 1.99722411 1.99861431 1.99930769]\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "tsfc:WARNING Estimated quadrature degree 11 more than tenfold greater than any argument/coefficient degree (max 1)\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Estimated quadrature degree 11 more than tenfold greater than any argument/coefficient degree (max 1)\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no tangent-linear = [2.86571468e-04 1.43890711e-04 7.20965028e-05 3.60860261e-05\n", + " 1.80524552e-05]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no tangent-linear = [0.99392152 0.99697228 0.998489 0.99924522]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with tangent-linear = [2.41887274e-06 6.04459357e-07 1.51082379e-07 3.77665303e-08\n", + " 9.44112417e-09]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with tangent-linear = [2.00061762 2.00030994 2.00015525 2.00007769]\n" + ] + } + ], + "source": [ + "np.random.seed(76149511)\n", + "\n", + "for order in range(1, 5):\n", + " min_order = taylor_test_tlm(forward, m, tlm_order=order, seed=1.0e-3)\n", + " assert min_order > 1.99" + ] + }, + { + "cell_type": "markdown", + "id": "df687375", + "metadata": {}, + "source": [ + "The function `taylor_test_tlm_adjoint` also performs such verification tests, but computes the highest order derivative information using the adjoint method.\n", + "\n", + "Let's apply this test up to fourth order:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "9404e167", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:44:57.685737Z", + "iopub.status.busy": "2023-12-20T16:44:57.685404Z", + "iopub.status.idle": "2023-12-20T16:45:12.076547Z", + "shell.execute_reply": "2023-12-20T16:45:12.075917Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [1.96925244e-04 9.84831481e-05 4.92467146e-05 2.46246436e-05\n", + " 1.23126435e-05]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [0.99969928 0.9998494 0.99992464 0.9999623 ]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [8.22009804e-08 2.05743897e-08 5.14661321e-09 1.28703002e-09\n", + " 3.21803907e-10]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [1.99830597 1.99915437 1.99957766 1.99979196]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [6.06217536e-05 3.03568452e-05 1.51898950e-05 7.59781316e-06\n", + " 3.79962268e-06]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [0.99781372 0.99890997 0.99945576 0.99972807]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [1.83663729e-07 4.58635021e-08 1.14593026e-08 2.86400265e-09\n", + " 7.15897696e-10]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [2.00164832 2.00082728 2.00041451 2.00020749]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [1.77271516e-04 8.85216211e-05 4.42321703e-05 2.21089118e-05\n", + " 1.10526609e-05]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [1.00185896 1.00093384 1.00046801 1.00023428]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [4.57678459e-07 1.14702543e-07 2.87109831e-08 7.18216298e-09\n", + " 1.79609281e-09]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [1.99643702 1.99822274 1.99911243 1.99955649]\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "tsfc:WARNING Estimated quadrature degree 11 more than tenfold greater than any argument/coefficient degree (max 1)\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Estimated quadrature degree 11 more than tenfold greater than any argument/coefficient degree (max 1)\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [2.68076588e-04 1.34645957e-04 6.74748172e-05 3.37753585e-05\n", + " 1.68971655e-05]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [0.99347432 0.99674984 0.99837808 0.99918983]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [2.42982799e-06 6.07250783e-07 1.51786806e-07 3.79434579e-08\n", + " 9.48545885e-09]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [2.00048984 2.00024606 2.00012332 2.00006169]\n" + ] + } + ], + "source": [ + "np.random.seed(74728054)\n", + "\n", + "for order in range(1, 5):\n", + " min_order = taylor_test_tlm_adjoint(forward, m, adjoint_order=order, seed=1.0e-3)\n", + " assert min_order > 1.99" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/3_time_dependent.html b/examples/3_time_dependent.html new file mode 100644 index 0000000..c103dae --- /dev/null +++ b/examples/3_time_dependent.html @@ -0,0 +1,1192 @@ + + + + + + + Time-dependent example — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Time-dependent example

+

This notebook describes the calculation of derivative information for a time-dependent problem using tlm_adjoint with the Firedrake backend. Overheads associated with building the records of calculations are discussed, and a checkpointing schedule is applied.

+

The binomial checkpointing schedule is based on the method described in:

+
    +

  • Andreas Griewank and Andrea Walther, ‘Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation’, ACM Transactions on Mathematical Software, 26(1), pp. 19–45, 2000, doi: 10.1145/347837.347846

    • +
+
+

Forward problem

+

We consider the solution of a linear time-dependent partial differential equation, followed by the calculation of the square of the \(L^2\)-norm of the final time solution. We assume real spaces and a real build of Firedrake throughout.

+

Specifically we consider the advection-diffusion equation in two dimensions, in the form

+
+\[\partial_t u + \partial_x \psi \partial_y u - \partial_y \psi \partial_x u = \kappa \left( \partial_{xx} + \partial_{yy} \right) u,\]
+

where \(\psi\) vanishes on the domain boundary, and subject to zero flux boundary conditions. We consider the spatial domain \(\left( x, y \right) \in \left( 0, 1 \right)^2\) and temporal domain \(t \in \left[ 0, 0.1 \right]\), with \(\psi \left( x, y \right) = -\sin \left( \pi x \right) \sin \left( \pi y \right)\) and \(\kappa = 0.01\), and an initial condition +\(u \left( x, y, t=0 \right) = \exp \left[ -50 \left( \left( x - 0.75 \right)^2 + \left( y - 0.5 \right)^2 \right) \right]\).

+

The problem is discretized using \(P_1\) continuous finite elements to represent both the solution \(u\) at each time level and the stream function \(\psi\). The problem is discretized in time using the implicit trapezoidal rule.

+

A simple implementation in Firedrake takes the form:

+
+
[1]:
+
+
+
%matplotlib inline
+
+from firedrake import *
+
+import matplotlib.pyplot as plt
+import numpy as np
+
+T = 0.1
+N = 100
+dt = Constant(T / N)
+
+mesh = UnitSquareMesh(128, 128)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+psi = Function(space, name="psi")
+psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))
+
+kappa = Constant(0.01)
+
+u_0 = Function(space, name="u_0")
+u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))
+
+u_n = Function(space, name="u_n")
+u_np1 = Function(space, name="u_np1")
+
+u_h = 0.5 * (u_n + trial)
+F = (inner(trial - u_n, test) * dx
+     + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx
+     + dt * inner(kappa * grad(u_h), grad(test)) * dx)
+lhs, rhs = system(F)
+
+problem = LinearVariationalProblem(
+    lhs, rhs, u_np1,
+    constant_jacobian=True)
+solver = LinearVariationalSolver(
+    problem, solver_parameters={"ksp_type": "preonly",
+                                "pc_type": "lu"})
+
+u_n.assign(u_0)
+for n in range(N):
+    solver.solve()
+    u_n.assign(u_np1)
+
+J = assemble(inner(u_n, u_n) * dx)
+
+
+def plot_output(u, title):
+    r = (u.dat.data_ro.min(), u.dat.data_ro.max())
+    eps = (r[1] - r[0]) * 1.0e-12
+    p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))
+    plt.gca().set_title(title)
+    plt.colorbar(p)
+    plt.gca().set_aspect(1.0)
+
+
+plot_output(u_0, title="$u_0$")
+plot_output(u_n, title="$u_n$")
+
+
+
+
+
+
+
+../_images/examples_3_time_dependent_1_0.png +
+
+
+
+
+
+../_images/examples_3_time_dependent_1_1.png +
+
+
+
+

Adding tlm_adjoint

+

We first modify the code so that tlm_adjoint processes the calculations:

+
+
[2]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+reset_manager("memory", {})
+
+T = 0.1
+N = 100
+dt = Constant(T / N)
+
+mesh = UnitSquareMesh(128, 128)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+psi = Function(space, name="psi")
+psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))
+
+kappa = Constant(0.01)
+
+u_0 = Function(space, name="u_0")
+u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))
+
+
+def forward(u_0, psi):
+    u_n = Function(space, name="u_n")
+    u_np1 = Function(space, name="u_np1")
+
+    u_h = 0.5 * (u_n + trial)
+    F = (inner(trial - u_n, test) * dx
+         + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx
+         + dt * inner(kappa * grad(u_h), grad(test)) * dx)
+    lhs, rhs = system(F)
+
+    problem = LinearVariationalProblem(
+        lhs, rhs, u_np1,
+        constant_jacobian=True)
+    solver = LinearVariationalSolver(
+        problem, solver_parameters={"ksp_type": "preonly",
+                                    "pc_type": "lu"})
+
+    u_n.assign(u_0)
+    for n in range(N):
+        solver.solve()
+        u_n.assign(u_np1)
+
+    J = Functional(name="J")
+    J.assign(inner(u_n, u_n) * dx)
+    return J
+
+
+start_manager()
+J = forward(u_0, psi)
+stop_manager()
+
+
+
+
+
[2]:
+
+
+
+
+(True, True)
+
+
+

Later we will configure a checkpointing schedule. Resetting the manager resets the record of forward equations but does not reset the checkpointing configuration, and so in this example whenever we reset the manager we also return it to the default checkpointing configuration with reset_manager("memory", {}).

+
+
+

Computing derivatives using an adjoint

+

The compute_gradient function can be used to compute derivatives using the adjoint method. Here we compute the derivative of the square of the \(L^2\)-norm of the final timestep solution, considered a function of the control defined by the initial condition u_0 and stream function psi, with respect to this control:

+
+
[3]:
+
+
+
dJ_du_0, dJ_dpsi = compute_gradient(J, (u_0, psi))
+
+
+
+

As a simple check of the result, note that the solution to the (discretized) partial differential equation is unchanged by the addition of a constant to the stream function. Hence we expect the directional derivative with respect to the stream function, with direction equal to the unity valued function, to be zero. This is indeed found to be the case (except for roundoff errors):

+
+
[4]:
+
+
+
one = Function(space, name="one")
+one.interpolate(Constant(1.0))
+
+dJ_dpsi_one = var_inner(one, dJ_dpsi)
+
+print(f"{dJ_dpsi_one=}")
+
+assert abs(dJ_dpsi_one) < 1.0e-17
+
+
+
+
+
+
+
+
+dJ_dpsi_one=1.8775546878197248e-18
+
+
+
+
+

Computing Hessian information using an adjoint of a tangent-linear

+

We next compute a Hessian action. Although the following calculation does work, it is inefficient – you may wish to skip forward to the optimized calculations.

+

Here we compute a ‘mixed’ Hessian action, by defining a directional derivative with respect to the stream function, and then differentiating this with respect to the initial condition:

+
+
[5]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+reset_manager("memory", {})
+
+T = 0.1
+N = 100
+dt = Constant(T / N)
+
+mesh = UnitSquareMesh(128, 128)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+psi = Function(space, name="psi")
+psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))
+
+kappa = Constant(0.01)
+
+u_0 = Function(space, name="u_0")
+u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))
+
+
+def forward(u_0, psi):
+    u_n = Function(space, name="u_n")
+    u_np1 = Function(space, name="u_np1")
+
+    u_h = 0.5 * (u_n + trial)
+    F = (inner(trial - u_n, test) * dx
+         + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx
+         + dt * inner(kappa * grad(u_h), grad(test)) * dx)
+    lhs, rhs = system(F)
+
+    problem = LinearVariationalProblem(
+        lhs, rhs, u_np1,
+        constant_jacobian=True)
+    solver = LinearVariationalSolver(
+        problem, solver_parameters={"ksp_type": "preonly",
+                                    "pc_type": "lu"})
+
+    u_n.assign(u_0)
+    for n in range(N):
+        solver.solve()
+        u_n.assign(u_np1)
+
+    J = Functional(name="J")
+    J.assign(inner(u_n, u_n) * dx)
+    return J
+
+
+zeta = Function(space, name="zeta")
+zeta.assign(psi)
+configure_tlm((psi, zeta))
+
+start_manager()
+J = forward(u_0, psi)
+stop_manager()
+
+dJ_dpsi_zeta = var_tlm(J, (psi, zeta))
+
+d2J_dpsi_zeta_du_0 = compute_gradient(dJ_dpsi_zeta, u_0)
+
+
+
+
+
+

Optimization

+

In the above we have successfully built a record of calculations, and used this to compute derivative information. However there are two issues:

+
    +

  1. Building the record has a noticable cost – the forward calculation has slowed down. In the second order calculation overheads associated with the tangent-linear lead to substantial additional costs.

    2. +

  2. tlm_adjoint records the solution of the partial differential equation on all time levels. The memory usage here is manageable. However memory limits will be exceeded for larger problems with more fields, spatial degrees of freedom, or timesteps.

    3. +
+

Let’s fix these issues in order.

+
+

Optimizing the annotation

+

In the above code tlm_adjoint builds a new record for each finite element variational problem it encounters. Even though only one LinearVariationalSolver is instantiated, an EquationSolver record is instantiated on each call to the solve method. Building the record is sufficiently expensive that the forward calculation noticeably slows down, and this also leads to significant extra processing in the derivative calculations.

+

Instead we can instantiate an EquationSolver directly, and reuse it. However if we do only that then the code will still be inefficient. A single EquationSolver will be used, but new linear solver data will be constructed each time its solve method is called. We need to also apply an optimization analogous to the constant_jacobian=True argument supplied to LinearVariationalProblem.

+

A simple fix is to add cache_jacobian=True when instantiating the EquationSolver:

+
eq = EquationSolver(
+    lhs == rhs, u_np1,
+    solver_parameters={"ksp_type": "preonly",
+                       "pc_type": "lu"},
+    cache_jacobian=True)
+
+
+

This works, but we can instead let tlm_adjoint detect that linear solver data can be cached. We can do that by adding static=True when instantiating variables whose value is unchanged throughout the forward calculation:

+
+
[6]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+reset_manager("memory", {})
+clear_caches()
+
+T = 0.1
+N = 100
+dt = Constant(T / N, static=True)
+
+mesh = UnitSquareMesh(128, 128)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+psi = Function(space, name="psi", static=True)
+psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))
+
+kappa = Constant(0.01, static=True)
+
+u_0 = Function(space, name="u_0", static=True)
+u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))
+
+
+def forward(u_0, psi):
+    u_n = Function(space, name="u_n")
+    u_np1 = Function(space, name="u_np1")
+
+    u_h = 0.5 * (u_n + trial)
+    F = (inner(trial - u_n, test) * dx
+         + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx
+         + dt * inner(kappa * grad(u_h), grad(test)) * dx)
+    lhs, rhs = system(F)
+
+    eq = EquationSolver(
+        lhs == rhs, u_np1,
+        solver_parameters={"ksp_type": "preonly",
+                           "pc_type": "lu"})
+
+    u_n.assign(u_0)
+    for n in range(N):
+        eq.solve()
+        u_n.assign(u_np1)
+
+    J = Functional(name="J")
+    J.assign(inner(u_n, u_n) * dx)
+    return J
+
+
+start_manager()
+J = forward(u_0, psi)
+stop_manager()
+
+
+
+
+
[6]:
+
+
+
+
+(True, True)
+
+
+

If we now query the relevant tlm_adjoint caches:

+
+
[7]:
+
+
+
print(f"{len(assembly_cache())=}")
+print(f"{len(linear_solver_cache())=}")
+
+assert len(assembly_cache()) == 2
+assert len(linear_solver_cache()) == 1
+
+
+
+
+
+
+
+
+len(assembly_cache())=2
+len(linear_solver_cache())=1
+
+
+

we find that linear solver data associated with a single matrix has been cached. We also find that two assembled objects have been cached – it turns out that there are two cached matrices. As well as caching the matrix associated with the left-hand-side of the discrete problem, a matrix associated with the right-hand-side has been assembled and cached. Assembly of the right-hand-side has been converted into a matrix multiply. If we wished we could disable right-hand-side optimizations by +adding cache_rhs_assembly=False:

+
eq = EquationSolver(
+    lhs == rhs, u_np1,
+    solver_parameters={"ksp_type": "preonly",
+                       "pc_type": "lu"},
+    cache_rhs_assembly=False)
+
+
+
+
+

Using a checkpointing schedule

+

To address the storage issue we enable checkpointing. Here we enable binomial checkpointing with storage of a maximum of \(10\) forward restart checkpoints in memory:

+
+
[8]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import logging
+
+logger = logging.getLogger("tlm_adjoint")
+logger.setLevel(logging.DEBUG)
+root_logger = logging.getLogger()
+if len(logger.handlers) == 1:
+    if len(root_logger.handlers) == 1:
+        root_logger.handlers.pop()
+    root_logger.addHandler(logger.handlers.pop())
+
+reset_manager("memory", {})
+clear_caches()
+
+T = 0.1
+N = 100
+dt = Constant(T / N, static=True)
+
+mesh = UnitSquareMesh(128, 128)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+psi = Function(space, name="psi", static=True)
+psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))
+
+kappa = Constant(0.01, static=True)
+
+u_0 = Function(space, name="u_0", static=True)
+u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))
+
+
+def forward(u_0, psi):
+    u_n = Function(space, name="u_n")
+    u_np1 = Function(space, name="u_np1")
+
+    u_h = 0.5 * (u_n + trial)
+    F = (inner(trial - u_n, test) * dx
+         + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx
+         + dt * inner(kappa * grad(u_h), grad(test)) * dx)
+    lhs, rhs = system(F)
+
+    eq = EquationSolver(
+        lhs == rhs, u_np1,
+        solver_parameters={"ksp_type": "preonly",
+                           "pc_type": "lu"})
+
+    u_n.assign(u_0)
+    for n in range(N):
+        eq.solve()
+        u_n.assign(u_np1)
+        if n < N - 1:
+            new_block()
+
+    J = Functional(name="J")
+    J.assign(inner(u_n, u_n) * dx)
+    return J
+
+
+configure_checkpointing("multistage", {"snaps_in_ram": 10, "blocks": N})
+start_manager()
+J = forward(u_0, psi)
+stop_manager()
+
+
+
+
+
+
+
+
+forward: forward advance to 9223372036854775807
+forward: forward advance to 21
+forward: save checkpoint data at 0 in RAM
+forward: forward advance to 40
+forward: save checkpoint data at 21 in RAM
+forward: forward advance to 57
+forward: save checkpoint data at 40 in RAM
+forward: forward advance to 72
+forward: save checkpoint data at 57 in RAM
+forward: forward advance to 79
+forward: save checkpoint data at 72 in RAM
+forward: forward advance to 85
+forward: save checkpoint data at 79 in RAM
+forward: forward advance to 90
+forward: save checkpoint data at 85 in RAM
+forward: forward advance to 94
+forward: save checkpoint data at 90 in RAM
+forward: forward advance to 97
+forward: save checkpoint data at 94 in RAM
+forward: forward advance to 99
+forward: save checkpoint data at 97 in RAM
+forward: forward advance to 100
+
+
+
+
[8]:
+
+
+
+
+(True, True)
+
+
+

The key changes here are:

+
    +

  • Configuration of a checkpointing schedule using configure_checkpointing. Here binomial checkpointing is applied, with a maximum of \(10\) forward restart checkpoints stored in memory, indicated using the "snaps_in_ram" parameter. The total number of steps is indicated using the "blocks" parameter.

    • +

  • The indication of the steps using new_block().

    • +
+

Extra logging output is also enabled so that we can see the details of the checkpointing schedule.

+
+
+

Computing derivatives

+

We are now ready to compute derivatives. However a key restriction is that we can, with this checkpointing schedule, only perform the adjoint calculation once per forward calculation. We cannot call compute_gradient a second time, without first rerunning the entire forward calculation.

+

In the following we compute both first and second derivative information using a single adjoint calculation:

+
+
[9]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import logging
+
+logger = logging.getLogger("tlm_adjoint")
+logger.setLevel(logging.DEBUG)
+root_logger = logging.getLogger()
+if len(logger.handlers) == 1:
+    if len(root_logger.handlers) == 1:
+        root_logger.handlers.pop()
+    root_logger.addHandler(logger.handlers.pop())
+
+reset_manager("memory", {})
+clear_caches()
+
+T = 0.1
+N = 100
+dt = Constant(T / N, static=True)
+
+mesh = UnitSquareMesh(128, 128)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test = TestFunction(space)
+trial = TrialFunction(space)
+
+psi = Function(space, name="psi", static=True)
+psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))
+
+kappa = Constant(0.01, static=True)
+
+u_0 = Function(space, name="u_0", static=True)
+u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))
+
+
+def forward(u_0, psi):
+    u_n = Function(space, name="u_n")
+    u_np1 = Function(space, name="u_np1")
+
+    u_h = 0.5 * (u_n + trial)
+    F = (inner(trial - u_n, test) * dx
+         + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx
+         + dt * inner(kappa * grad(u_h), grad(test)) * dx)
+    lhs, rhs = system(F)
+
+    eq = EquationSolver(
+        lhs == rhs, u_np1,
+        solver_parameters={"ksp_type": "preonly",
+                           "pc_type": "lu"})
+
+    u_n.assign(u_0)
+    for n in range(N):
+        eq.solve()
+        u_n.assign(u_np1)
+        if n < N - 1:
+            new_block()
+
+    J = Functional(name="J")
+    J.assign(inner(u_n, u_n) * dx)
+    return J
+
+
+zeta_u_0 = ZeroFunction(space, name="zeta_u_0")
+zeta_psi = Function(space, name="zeta_psi", static=True)
+zeta_psi.assign(psi)
+configure_tlm(((u_0, psi), (zeta_u_0, zeta_psi)))
+
+configure_checkpointing("multistage", {"snaps_in_ram": 10, "blocks": N})
+start_manager()
+J = forward(u_0, psi)
+stop_manager()
+
+dJ_dpsi_zeta = var_tlm(J, ((u_0, psi), (zeta_u_0, zeta_psi)))
+
+dJ_du_0, dJ_dpsi, d2J_dpsi_zeta_du_0 = compute_gradient(
+    dJ_dpsi_zeta, (zeta_u_0, zeta_psi, u_0))
+
+
+
+
+
+
+
+
+forward: forward advance to 9223372036854775807
+forward: forward advance to 21
+forward: save checkpoint data at 0 in RAM
+forward: forward advance to 40
+forward: save checkpoint data at 21 in RAM
+forward: forward advance to 57
+forward: save checkpoint data at 40 in RAM
+forward: forward advance to 72
+forward: save checkpoint data at 57 in RAM
+forward: forward advance to 79
+forward: save checkpoint data at 72 in RAM
+forward: forward advance to 85
+forward: save checkpoint data at 79 in RAM
+forward: forward advance to 90
+forward: save checkpoint data at 85 in RAM
+forward: forward advance to 94
+forward: save checkpoint data at 90 in RAM
+forward: forward advance to 97
+forward: save checkpoint data at 94 in RAM
+forward: forward advance to 99
+forward: save checkpoint data at 97 in RAM
+forward: forward advance to 100
+reverse: adjoint step back to 99
+reverse: load checkpoint data at 97 from RAM and keep
+reverse: forward advance to 98
+reverse: forward advance to 99
+reverse: adjoint step back to 98
+reverse: load checkpoint data at 97 from RAM and delete
+reverse: forward advance to 98
+reverse: adjoint step back to 97
+reverse: load checkpoint data at 94 from RAM and keep
+reverse: forward advance to 95
+reverse: forward advance to 96
+reverse: save checkpoint data at 95 in RAM
+reverse: forward advance to 97
+reverse: adjoint step back to 96
+reverse: load checkpoint data at 95 from RAM and delete
+reverse: forward advance to 96
+reverse: adjoint step back to 95
+reverse: load checkpoint data at 94 from RAM and delete
+reverse: forward advance to 95
+reverse: adjoint step back to 94
+reverse: load checkpoint data at 90 from RAM and keep
+reverse: forward advance to 91
+reverse: forward advance to 92
+reverse: save checkpoint data at 91 in RAM
+reverse: forward advance to 93
+reverse: save checkpoint data at 92 in RAM
+reverse: forward advance to 94
+reverse: adjoint step back to 93
+reverse: load checkpoint data at 92 from RAM and delete
+reverse: forward advance to 93
+reverse: adjoint step back to 92
+reverse: load checkpoint data at 91 from RAM and delete
+reverse: forward advance to 92
+reverse: adjoint step back to 91
+reverse: load checkpoint data at 90 from RAM and delete
+reverse: forward advance to 91
+reverse: adjoint step back to 90
+reverse: load checkpoint data at 85 from RAM and keep
+reverse: forward advance to 86
+reverse: forward advance to 87
+reverse: save checkpoint data at 86 in RAM
+reverse: forward advance to 88
+reverse: save checkpoint data at 87 in RAM
+reverse: forward advance to 89
+reverse: save checkpoint data at 88 in RAM
+reverse: forward advance to 90
+reverse: adjoint step back to 89
+reverse: load checkpoint data at 88 from RAM and delete
+reverse: forward advance to 89
+reverse: adjoint step back to 88
+reverse: load checkpoint data at 87 from RAM and delete
+reverse: forward advance to 88
+reverse: adjoint step back to 87
+reverse: load checkpoint data at 86 from RAM and delete
+reverse: forward advance to 87
+reverse: adjoint step back to 86
+reverse: load checkpoint data at 85 from RAM and delete
+reverse: forward advance to 86
+reverse: adjoint step back to 85
+reverse: load checkpoint data at 79 from RAM and keep
+reverse: forward advance to 80
+reverse: forward advance to 81
+reverse: save checkpoint data at 80 in RAM
+reverse: forward advance to 82
+reverse: save checkpoint data at 81 in RAM
+reverse: forward advance to 83
+reverse: save checkpoint data at 82 in RAM
+reverse: forward advance to 84
+reverse: save checkpoint data at 83 in RAM
+reverse: forward advance to 85
+reverse: adjoint step back to 84
+reverse: load checkpoint data at 83 from RAM and delete
+reverse: forward advance to 84
+reverse: adjoint step back to 83
+reverse: load checkpoint data at 82 from RAM and delete
+reverse: forward advance to 83
+reverse: adjoint step back to 82
+reverse: load checkpoint data at 81 from RAM and delete
+reverse: forward advance to 82
+reverse: adjoint step back to 81
+reverse: load checkpoint data at 80 from RAM and delete
+reverse: forward advance to 81
+reverse: adjoint step back to 80
+reverse: load checkpoint data at 79 from RAM and delete
+reverse: forward advance to 80
+reverse: adjoint step back to 79
+reverse: load checkpoint data at 72 from RAM and keep
+reverse: forward advance to 73
+reverse: forward advance to 74
+reverse: save checkpoint data at 73 in RAM
+reverse: forward advance to 75
+reverse: save checkpoint data at 74 in RAM
+reverse: forward advance to 76
+reverse: save checkpoint data at 75 in RAM
+reverse: forward advance to 77
+reverse: save checkpoint data at 76 in RAM
+reverse: forward advance to 78
+reverse: save checkpoint data at 77 in RAM
+reverse: forward advance to 79
+reverse: adjoint step back to 78
+reverse: load checkpoint data at 77 from RAM and delete
+reverse: forward advance to 78
+reverse: adjoint step back to 77
+reverse: load checkpoint data at 76 from RAM and delete
+reverse: forward advance to 77
+reverse: adjoint step back to 76
+reverse: load checkpoint data at 75 from RAM and delete
+reverse: forward advance to 76
+reverse: adjoint step back to 75
+reverse: load checkpoint data at 74 from RAM and delete
+reverse: forward advance to 75
+reverse: adjoint step back to 74
+reverse: load checkpoint data at 73 from RAM and delete
+reverse: forward advance to 74
+reverse: adjoint step back to 73
+reverse: load checkpoint data at 72 from RAM and delete
+reverse: forward advance to 73
+reverse: adjoint step back to 72
+reverse: load checkpoint data at 57 from RAM and keep
+reverse: forward advance to 59
+reverse: forward advance to 61
+reverse: save checkpoint data at 59 in RAM
+reverse: forward advance to 63
+reverse: save checkpoint data at 61 in RAM
+reverse: forward advance to 65
+reverse: save checkpoint data at 63 in RAM
+reverse: forward advance to 67
+reverse: save checkpoint data at 65 in RAM
+reverse: forward advance to 69
+reverse: save checkpoint data at 67 in RAM
+reverse: forward advance to 71
+reverse: save checkpoint data at 69 in RAM
+reverse: forward advance to 72
+reverse: adjoint step back to 71
+reverse: load checkpoint data at 69 from RAM and keep
+reverse: forward advance to 70
+reverse: forward advance to 71
+reverse: adjoint step back to 70
+reverse: load checkpoint data at 69 from RAM and delete
+reverse: forward advance to 70
+reverse: adjoint step back to 69
+reverse: load checkpoint data at 67 from RAM and keep
+reverse: forward advance to 68
+reverse: forward advance to 69
+reverse: adjoint step back to 68
+reverse: load checkpoint data at 67 from RAM and delete
+reverse: forward advance to 68
+reverse: adjoint step back to 67
+reverse: load checkpoint data at 65 from RAM and keep
+reverse: forward advance to 66
+reverse: forward advance to 67
+reverse: adjoint step back to 66
+reverse: load checkpoint data at 65 from RAM and delete
+reverse: forward advance to 66
+reverse: adjoint step back to 65
+reverse: load checkpoint data at 63 from RAM and keep
+reverse: forward advance to 64
+reverse: forward advance to 65
+reverse: adjoint step back to 64
+reverse: load checkpoint data at 63 from RAM and delete
+reverse: forward advance to 64
+reverse: adjoint step back to 63
+reverse: load checkpoint data at 61 from RAM and keep
+reverse: forward advance to 62
+reverse: forward advance to 63
+reverse: adjoint step back to 62
+reverse: load checkpoint data at 61 from RAM and delete
+reverse: forward advance to 62
+reverse: adjoint step back to 61
+reverse: load checkpoint data at 59 from RAM and keep
+reverse: forward advance to 60
+reverse: forward advance to 61
+reverse: adjoint step back to 60
+reverse: load checkpoint data at 59 from RAM and delete
+reverse: forward advance to 60
+reverse: adjoint step back to 59
+reverse: load checkpoint data at 57 from RAM and keep
+reverse: forward advance to 58
+reverse: forward advance to 59
+reverse: adjoint step back to 58
+reverse: load checkpoint data at 57 from RAM and delete
+reverse: forward advance to 58
+reverse: adjoint step back to 57
+reverse: load checkpoint data at 40 from RAM and keep
+reverse: forward advance to 42
+reverse: forward advance to 44
+reverse: save checkpoint data at 42 in RAM
+reverse: forward advance to 46
+reverse: save checkpoint data at 44 in RAM
+reverse: forward advance to 48
+reverse: save checkpoint data at 46 in RAM
+reverse: forward advance to 50
+reverse: save checkpoint data at 48 in RAM
+reverse: forward advance to 52
+reverse: save checkpoint data at 50 in RAM
+reverse: forward advance to 54
+reverse: save checkpoint data at 52 in RAM
+reverse: forward advance to 56
+reverse: save checkpoint data at 54 in RAM
+reverse: forward advance to 57
+reverse: adjoint step back to 56
+reverse: load checkpoint data at 54 from RAM and keep
+reverse: forward advance to 55
+reverse: forward advance to 56
+reverse: adjoint step back to 55
+reverse: load checkpoint data at 54 from RAM and delete
+reverse: forward advance to 55
+reverse: adjoint step back to 54
+reverse: load checkpoint data at 52 from RAM and keep
+reverse: forward advance to 53
+reverse: forward advance to 54
+reverse: adjoint step back to 53
+reverse: load checkpoint data at 52 from RAM and delete
+reverse: forward advance to 53
+reverse: adjoint step back to 52
+reverse: load checkpoint data at 50 from RAM and keep
+reverse: forward advance to 51
+reverse: forward advance to 52
+reverse: adjoint step back to 51
+reverse: load checkpoint data at 50 from RAM and delete
+reverse: forward advance to 51
+reverse: adjoint step back to 50
+reverse: load checkpoint data at 48 from RAM and keep
+reverse: forward advance to 49
+reverse: forward advance to 50
+reverse: adjoint step back to 49
+reverse: load checkpoint data at 48 from RAM and delete
+reverse: forward advance to 49
+reverse: adjoint step back to 48
+reverse: load checkpoint data at 46 from RAM and keep
+reverse: forward advance to 47
+reverse: forward advance to 48
+reverse: adjoint step back to 47
+reverse: load checkpoint data at 46 from RAM and delete
+reverse: forward advance to 47
+reverse: adjoint step back to 46
+reverse: load checkpoint data at 44 from RAM and keep
+reverse: forward advance to 45
+reverse: forward advance to 46
+reverse: adjoint step back to 45
+reverse: load checkpoint data at 44 from RAM and delete
+reverse: forward advance to 45
+reverse: adjoint step back to 44
+reverse: load checkpoint data at 42 from RAM and keep
+reverse: forward advance to 43
+reverse: forward advance to 44
+reverse: adjoint step back to 43
+reverse: load checkpoint data at 42 from RAM and delete
+reverse: forward advance to 43
+reverse: adjoint step back to 42
+reverse: load checkpoint data at 40 from RAM and keep
+reverse: forward advance to 41
+reverse: forward advance to 42
+reverse: adjoint step back to 41
+reverse: load checkpoint data at 40 from RAM and delete
+reverse: forward advance to 41
+reverse: adjoint step back to 40
+reverse: load checkpoint data at 21 from RAM and keep
+reverse: forward advance to 23
+reverse: forward advance to 25
+reverse: save checkpoint data at 23 in RAM
+reverse: forward advance to 27
+reverse: save checkpoint data at 25 in RAM
+reverse: forward advance to 29
+reverse: save checkpoint data at 27 in RAM
+reverse: forward advance to 31
+reverse: save checkpoint data at 29 in RAM
+reverse: forward advance to 33
+reverse: save checkpoint data at 31 in RAM
+reverse: forward advance to 35
+reverse: save checkpoint data at 33 in RAM
+reverse: forward advance to 37
+reverse: save checkpoint data at 35 in RAM
+reverse: forward advance to 39
+reverse: save checkpoint data at 37 in RAM
+reverse: forward advance to 40
+reverse: adjoint step back to 39
+reverse: load checkpoint data at 37 from RAM and keep
+reverse: forward advance to 38
+reverse: forward advance to 39
+reverse: adjoint step back to 38
+reverse: load checkpoint data at 37 from RAM and delete
+reverse: forward advance to 38
+reverse: adjoint step back to 37
+reverse: load checkpoint data at 35 from RAM and keep
+reverse: forward advance to 36
+reverse: forward advance to 37
+reverse: adjoint step back to 36
+reverse: load checkpoint data at 35 from RAM and delete
+reverse: forward advance to 36
+reverse: adjoint step back to 35
+reverse: load checkpoint data at 33 from RAM and keep
+reverse: forward advance to 34
+reverse: forward advance to 35
+reverse: adjoint step back to 34
+reverse: load checkpoint data at 33 from RAM and delete
+reverse: forward advance to 34
+reverse: adjoint step back to 33
+reverse: load checkpoint data at 31 from RAM and keep
+reverse: forward advance to 32
+reverse: forward advance to 33
+reverse: adjoint step back to 32
+reverse: load checkpoint data at 31 from RAM and delete
+reverse: forward advance to 32
+reverse: adjoint step back to 31
+reverse: load checkpoint data at 29 from RAM and keep
+reverse: forward advance to 30
+reverse: forward advance to 31
+reverse: adjoint step back to 30
+reverse: load checkpoint data at 29 from RAM and delete
+reverse: forward advance to 30
+reverse: adjoint step back to 29
+reverse: load checkpoint data at 27 from RAM and keep
+reverse: forward advance to 28
+reverse: forward advance to 29
+reverse: adjoint step back to 28
+reverse: load checkpoint data at 27 from RAM and delete
+reverse: forward advance to 28
+reverse: adjoint step back to 27
+reverse: load checkpoint data at 25 from RAM and keep
+reverse: forward advance to 26
+reverse: forward advance to 27
+reverse: adjoint step back to 26
+reverse: load checkpoint data at 25 from RAM and delete
+reverse: forward advance to 26
+reverse: adjoint step back to 25
+reverse: load checkpoint data at 23 from RAM and keep
+reverse: forward advance to 24
+reverse: forward advance to 25
+reverse: adjoint step back to 24
+reverse: load checkpoint data at 23 from RAM and delete
+reverse: forward advance to 24
+reverse: adjoint step back to 23
+reverse: load checkpoint data at 21 from RAM and keep
+reverse: forward advance to 22
+reverse: forward advance to 23
+reverse: adjoint step back to 22
+reverse: load checkpoint data at 21 from RAM and delete
+reverse: forward advance to 22
+reverse: adjoint step back to 21
+reverse: load checkpoint data at 0 from RAM and keep
+reverse: forward advance to 2
+reverse: forward advance to 4
+reverse: save checkpoint data at 2 in RAM
+reverse: forward advance to 6
+reverse: save checkpoint data at 4 in RAM
+reverse: forward advance to 8
+reverse: save checkpoint data at 6 in RAM
+reverse: forward advance to 10
+reverse: save checkpoint data at 8 in RAM
+reverse: forward advance to 12
+reverse: save checkpoint data at 10 in RAM
+reverse: forward advance to 14
+reverse: save checkpoint data at 12 in RAM
+reverse: forward advance to 16
+reverse: save checkpoint data at 14 in RAM
+reverse: forward advance to 18
+reverse: save checkpoint data at 16 in RAM
+reverse: forward advance to 20
+reverse: save checkpoint data at 18 in RAM
+reverse: forward advance to 21
+reverse: adjoint step back to 20
+reverse: load checkpoint data at 18 from RAM and keep
+reverse: forward advance to 19
+reverse: forward advance to 20
+reverse: adjoint step back to 19
+reverse: load checkpoint data at 18 from RAM and delete
+reverse: forward advance to 19
+reverse: adjoint step back to 18
+reverse: load checkpoint data at 16 from RAM and keep
+reverse: forward advance to 17
+reverse: forward advance to 18
+reverse: adjoint step back to 17
+reverse: load checkpoint data at 16 from RAM and delete
+reverse: forward advance to 17
+reverse: adjoint step back to 16
+reverse: load checkpoint data at 14 from RAM and keep
+reverse: forward advance to 15
+reverse: forward advance to 16
+reverse: adjoint step back to 15
+reverse: load checkpoint data at 14 from RAM and delete
+reverse: forward advance to 15
+reverse: adjoint step back to 14
+reverse: load checkpoint data at 12 from RAM and keep
+reverse: forward advance to 13
+reverse: forward advance to 14
+reverse: adjoint step back to 13
+reverse: load checkpoint data at 12 from RAM and delete
+reverse: forward advance to 13
+reverse: adjoint step back to 12
+reverse: load checkpoint data at 10 from RAM and keep
+reverse: forward advance to 11
+reverse: forward advance to 12
+reverse: adjoint step back to 11
+reverse: load checkpoint data at 10 from RAM and delete
+reverse: forward advance to 11
+reverse: adjoint step back to 10
+reverse: load checkpoint data at 8 from RAM and keep
+reverse: forward advance to 9
+reverse: forward advance to 10
+reverse: adjoint step back to 9
+reverse: load checkpoint data at 8 from RAM and delete
+reverse: forward advance to 9
+reverse: adjoint step back to 8
+reverse: load checkpoint data at 6 from RAM and keep
+reverse: forward advance to 7
+reverse: forward advance to 8
+reverse: adjoint step back to 7
+reverse: load checkpoint data at 6 from RAM and delete
+reverse: forward advance to 7
+reverse: adjoint step back to 6
+reverse: load checkpoint data at 4 from RAM and keep
+reverse: forward advance to 5
+reverse: forward advance to 6
+reverse: adjoint step back to 5
+reverse: load checkpoint data at 4 from RAM and delete
+reverse: forward advance to 5
+reverse: adjoint step back to 4
+reverse: load checkpoint data at 2 from RAM and keep
+reverse: forward advance to 3
+reverse: forward advance to 4
+reverse: adjoint step back to 3
+reverse: load checkpoint data at 2 from RAM and delete
+reverse: forward advance to 3
+reverse: adjoint step back to 2
+reverse: load checkpoint data at 0 from RAM and keep
+reverse: forward advance to 1
+reverse: forward advance to 2
+reverse: adjoint step back to 1
+reverse: load checkpoint data at 0 from RAM and delete
+reverse: forward advance to 1
+reverse: adjoint step back to 0
+
+
+

The derivative calculation now alternates between forward + tangent-linear calculations, and adjoint calculations.

+

If we wished we could perform higher order adjoint calculations, using a binomial checkpointing schedule, by supplying a higher order tangent-linear configuration and differentiating the result.

+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/examples/3_time_dependent.ipynb b/examples/3_time_dependent.ipynb new file mode 100644 index 0000000..24bfadc --- /dev/null +++ b/examples/3_time_dependent.ipynb @@ -0,0 +1,4120 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "92f4f706", + "metadata": {}, + "source": [ + "# Time-dependent example\n", + "\n", + "This notebook describes the calculation of derivative information for a time-dependent problem using tlm_adjoint with the [Firedrake](https://firedrakeproject.org/) backend. Overheads associated with building the records of calculations are discussed, and a checkpointing schedule is applied.\n", + "\n", + "The binomial checkpointing schedule is based on the method described in:\n", + "\n", + "- Andreas Griewank and Andrea Walther, 'Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation', ACM Transactions on Mathematical Software, 26(1), pp. 19–45, 2000, doi: 10.1145/347837.347846\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the solution of a linear time-dependent partial differential equation, followed by the calculation of the square of the $L^2$-norm of the final time solution. We assume real spaces and a real build of Firedrake throughout.\n", + "\n", + "Specifically we consider the advection-diffusion equation in two dimensions, in the form\n", + "\n", + "$$\n", + " \\partial_t u + \\partial_x \\psi \\partial_y u - \\partial_y \\psi \\partial_x u = \\kappa \\left( \\partial_{xx} + \\partial_{yy} \\right) u,\n", + "$$\n", + "\n", + "where $\\psi$ vanishes on the domain boundary, and subject to zero flux boundary conditions. We consider the spatial domain $\\left( x, y \\right) \\in \\left( 0, 1 \\right)^2$ and temporal domain $t \\in \\left[ 0, 0.1 \\right]$, with $\\psi \\left( x, y \\right) = -\\sin \\left( \\pi x \\right) \\sin \\left( \\pi y \\right)$ and $\\kappa = 0.01$, and an initial condition $u \\left( x, y, t=0 \\right) = \\exp \\left[ -50 \\left( \\left( x - 0.75 \\right)^2 + \\left( y - 0.5 \\right)^2 \\right) \\right]$.\n", + "\n", + "The problem is discretized using $P_1$ continuous finite elements to represent both the solution $u$ at each time level and the stream function $\\psi$. The problem is discretized in time using the implicit trapezoidal rule.\n", + "\n", + "A simple implementation in Firedrake takes the form:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "b29607a8", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:45:14.781301Z", + "iopub.status.busy": "2023-12-20T16:45:14.780581Z", + "iopub.status.idle": "2023-12-20T16:45:22.039055Z", + "shell.execute_reply": "2023-12-20T16:45:22.038343Z" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgoAAAGyCAYAAACFsOzQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABdTUlEQVR4nO3df1xUdb4/8NfMAAOIoIaAECup5Y9NwWBlsd/FSptrtrdbZn6T3NKtcNdkd1OzxGoL+7Fet7Iok3KzVlfXftx0KcPIdWWzi5Fpapma1gaiJgTIADPn+4fNcc5wzplzzpyZgeH1fDzmkZw5Pz5zNM5r3p/P+RyLIAgCiIiIiGRYQ90AIiIi6r4YFIiIiEgRgwIREREpYlAgIiIiRQwKREREpIhBgYiIiBQxKBAREZEiBgUiIiJSxKBAREREihgUiIiISBGDAhERESliUCAiIiJFDApEQTRw4EDMnj27y/KcnBxMnDgxBC0iIlLHoEAUJP/5z39w/PhxZGZmSpY7nU7s2bMHY8aMCVHLiIiUMSgQBcmuXbsAoEtQ2LdvH9ra2jB69OhQNIuISBWDAlGQ7Nq1C1arFRdeeKFk+SeffAIADApE1C0xKBAFySeffIJhw4YhNjZWsry2thaRkZEYMWIEAKChoQETJ05Enz59MHz4cFRWVoaiuUREAICIUDeAqLf49NNPu3Q7AMBHH32E4cOHIzIyEgBQVFSElJQUNDQ04L333sNNN92EL774AgMGDAh2k4mIWFEgCgaXy4X9+/dj5MiRkuXHjh3Dtm3bxIGMzc3NeOONN/Dggw8iNjYW1113HUaPHo0333wzFM0mImJQIAoGp9OJjo4OtLa2iss6Ozvx61//Gp2dneL4hC+++AJxcXE499xzxfVGjx6NPXv2BL3NREQAux6IgiIyMhJjxozBc889h5iYGMTExGDdunWIiYkBcHYgY3NzM+Lj4yXbxsfH48SJE0FvMxERwKBAFDQvvfQSZs6ciSeeeAJDhw7FnDlzYLPZsH37djEoxMXFoampSbJdU1MT4uLiQtFkIiJYBEEQQt0IIjqjubkZAwYMwKFDh5CWlgYAuPLKKzF9+nTMmDEjxK0jot6IQYGom7nxxhuRkJCAp59+GpWVlSgsLORdD0QUMux6IOpmnn32WRQWFuKcc87Bueeei7Vr1zIkEFHIsKJAREREinTfHrl161ZMmjQJqampsFgseOONN3xuU1VVhYsuugh2ux3Dhg3Dyy+/bKCpREREFGy6g0JLSwsyMzOxfPlyTesfOnQIEydOxJVXXona2lrcc889uOOOO/DOO+/obiwREREFl19dDxaLBa+//jquv/56xXXmzZuHjRs3Yvfu3eKym2++GadOnUJFRYXRQxMREVEQBHwwY3V1NfLz8yXLCgoKcM899yhu43A44HA4xJ9dLhdOnjyJc845BxaLJVBNJSKiABAEAd9//z1SU1NhtQZuQuC2tja0t7ebsq+oqChER0ebsq+eLuBBoa6uDsnJyZJlycnJaGpqwunTp8WZ6TyVlpbiwQcfDHTTiIgoiI4ePSqZntxMbW1tyBjcB/XHXKbsLyUlBYcOHWJYQDe9PXLBggUoLi4Wf25sbMSPfvQjXBZ5PSIskSFsGRER6dUpdGBrxxvo27dvwI7R3t6O+mMu7P+/VPTt61/V4vvvXRie8x+0t7czKCAIQSElJQX19fWSZfX19YiPj5etJgCA3W6H3W7vsjzCEsmgQETUQwWj67hvXyvi/QwKJBXws5mXl4fKykrJss2bNyMvLy/QhyYiIiI/6Q4Kzc3NqK2tRW1tLYAztz/W1tbiyJEjAM50G0yfPl1c/84778TBgwdx7733Yt++fXj22Wfxt7/9DXPnzjXnExAREVHA6A4K//d//4exY8di7NixAIDi4mKMHTsWixYtAgB8++23YmgAgPPOOw8bN27E5s2bkZmZiT/96U948cUXUVBQYNJHICIiokDRPUbhiiuugNrUC3KzLl5xxRX4+OOP9R6KiIiIQowjPoiIiEgRgwIREREpYlAgIiIiRQwKREREpIhBgYiIiBR1yymciYiIjDjubIfD6ecUzk5znhcRLlhRICIiIkUMCkRERKSIQYGIiIgUMSgQERGRIgYFIiIiUsSgQERERIoYFIiIiEgRgwIREREpYlAgIiIiRQwKREREpIhBgYiIiBTxWQ9ERBQ2jrvsaHP59x242cVnPXhiRYGIiIgUMSgQERGRIgYFIiIiUsSgQERERIoYFIiIiEgRgwIREREpYlAgIiIiRQwKREREflq+fDkyMjIQHR2N3Nxc7NixQ3Hdjo4OPPTQQxg6dCiio6ORmZmJiooK3fv89a9/jaFDhyImJgYDBw7E5MmTsW/fPvH9Tz75BFOnTkV6ejpiYmIwcuRI/PnPf9b92RgUiIiI/LB27VoUFxejpKQEO3fuRGZmJgoKCnDs2DHZ9e+//348//zzePrpp/HZZ5/hzjvvxC9/+Ut8/PHHuvaZnZ2Nl156CXv37sU777wDQRAwYcIEOJ1OAEBNTQ2SkpKwevVq7NmzBwsXLsSCBQvwzDPP6Pp8FkEQBAPnJaiampqQkJCAq6JuRIQlMtTNISIiHTqFDmxpX4fGxkbEx8cH5Bju68SW3emI6+vnzIzfu3DVhUdx9OhRSXvtdjvsdnuX9XNzc/GTn/xEvAC7XC6kp6fjN7/5DebPn99l/dTUVCxcuBBFRUXishtuuAExMTFYvXq1oX0CwK5du5CZmYkDBw5g6NChsusUFRVh79692LJli8azwYoCERGFkRPOPmhwxvn1OuHsAwBIT09HQkKC+CotLe1yvPb2dtTU1CA/P19cZrVakZ+fj+rqatk2OhwOREdHS5bFxMRg27ZthvfZ0tKCl156Ceeddx7S09MVz09jYyMGDBig+L4cPuuBiIhIhlxFwdvx48fhdDqRnJwsWZ6cnCwZL+CpoKAAS5cuxWWXXYahQ4eisrISGzZsELsM9Ozz2Wefxb333ouWlhYMHz4cmzdvRlRUlOxxt2/fjrVr12Ljxo2+P7wHVhSIiIhkxMfHS15yQcGIP//5zzj//PMxYsQIREVFYfbs2ZgxYwasVv2X5GnTpuHjjz/GBx98gAsuuAA33XQT2trauqy3e/duTJ48GSUlJZgwYYKuYzAoEBERGZSYmAibzYb6+nrJ8vr6eqSkpMhuM3DgQLzxxhtoaWnBV199hX379iEuLg5DhgzRvc+EhAScf/75uOyyy7B+/Xrs27cPr7/+umSdzz77DFdffTVmzZqF+++/X/dnZFAgIiIyKCoqCtnZ2aisrBSXuVwuVFZWIi8vT3Xb6OhopKWlobOzE3//+98xefJkv/YpCAIEQYDD4RCX7dmzB1deeSUKCwvxyCOPGPqMHKNARETkh+LiYhQWFiInJwfjxo3DsmXL0NLSghkzZgAApk+fjrS0NHEw5IcffohvvvkGWVlZ+Oabb7B48WK4XC7ce++9mvd58OBBrF27FhMmTMDAgQPx9ddfY8mSJYiJicG1114L4Ex3w1VXXYWCggIUFxejrq4OAGCz2TBw4EDNn49BgYiIyA9TpkxBQ0MDFi1ahLq6OmRlZaGiokIcjHjkyBHJ+IO2tjbcf//9OHjwIOLi4nDttdfilVdeQb9+/TTvMzo6Gv/85z+xbNkyfPfdd0hOTsZll12G7du3IykpCQCwfv16NDQ0YPXq1eJtlwAwePBgHD58WPPn4zwKREQUUMGcR2HdJyMQ29fm175av3fixsx9AW1vT8IxCkRERKSIQYGIiIgUMSgQERGRIgYFIiIiUsS7HoiIKGzUdyYgpsO/S9vpzk6TWhMeWFEgIiIiRQwKREREpIhBgYiIiBQxKBAREZEiBgUiIiJSxKBAREREihgUiIiISBGDAhERESliUCAiIiJFDApERESkiEGBiIiIFPFZD0REFDZOdPZFdKd/l7Y2PutBghUFIiIiUsSgQERERIoYFIiIiEgRgwIREREpYlAgIiIiRQwKREREpIhBgYiIiBQxKBAREZEiBgUiIiJSxKBAREREigwFheXLlyMjIwPR0dHIzc3Fjh07VNdftmwZhg8fjpiYGKSnp2Pu3Lloa2sz1GAiIiIKHt0TYq9duxbFxcUoKytDbm4uli1bhoKCAuzfvx9JSUld1n/ttdcwf/58lJeXY/z48fj8889x2223wWKxYOnSpaZ8CCIiIgA40dEH9o5Iv/bh6OgwqTXhQXdFYenSpZg5cyZmzJiBUaNGoaysDLGxsSgvL5ddf/v27bj44otxyy23ICMjAxMmTMDUqVN9ViGIiIgo9HQFhfb2dtTU1CA/P//sDqxW5Ofno7q6Wnab8ePHo6amRgwGBw8exKZNm3DttdcqHsfhcKCpqUnyIiIiouDT1fVw/PhxOJ1OJCcnS5YnJydj3759stvccsstOH78OC655BIIgoDOzk7ceeeduO+++xSPU1paigcffFBP04iIiCgAAn7XQ1VVFR599FE8++yz2LlzJzZs2ICNGzfi4YcfVtxmwYIFaGxsFF9Hjx4NdDOJiIhIhq6KQmJiImw2G+rr6yXL6+vrkZKSIrvNAw88gFtvvRV33HEHAGD06NFoaWnBrFmzsHDhQlitXbOK3W6H3W7X0zQiIiIKAF0VhaioKGRnZ6OyslJc5nK5UFlZiby8PNltWltbu4QBm80GABAEQW97iYiIKIh03x5ZXFyMwsJC5OTkYNy4cVi2bBlaWlowY8YMAMD06dORlpaG0tJSAMCkSZOwdOlSjB07Frm5uThw4AAeeOABTJo0SQwMRERE1D3pHqMwZcoUPPnkk1i0aBGysrJQW1uLiooKcYDjkSNH8O2334rr33///fjd736H+++/H6NGjcLtt9+OgoICPP/88+Z9CiIiohDSMxHhFVdcAYvF0uU1ceJEcR259y0WC5544oku+3M4HMjKyoLFYkFtba24vKqqCpMnT8agQYPQp08fZGVl4dVXX9X92XRXFABg9uzZmD17tux7VVVV0gNERKCkpAQlJSVGDkVERNSt6Z2IcMOGDWhvbxd/PnHiBDIzM3HjjTeKyzy/cAPAP/7xD9x+++244YYbuuzv3nvvRWpqKj755BPJ8u3bt2PMmDGYN28ekpOT8fbbb2P69OlISEjAL37xC82fz1BQICIiojM8JyIEgLKyMmzcuBHl5eWYP39+l/UHDBgg+XnNmjWIjY2VBAXvGwTefPNNXHnllRgyZIhk+T/+8Q+8++67+Pvf/45//OMfkve8pyGYM2cO3n33XWzYsEFXUOBDoYiIiGR4T/zncDi6rGNkIkJvK1euxM0334w+ffrIvl9fX4+NGzfi9ttv77J85syZeOWVVxAbG6vpWI2NjV2Cii+sKBARUdj4riMWUR1Rfu2jveNMt0B6erpkeUlJCRYvXixZZmQiQk87duzA7t27sXLlSsV1Vq1ahb59++K//uu/xGWCIOC2227DnXfeiZycHBw+fNjnsf72t7/ho48+0j1GkEGBiIhIxtGjRxEfHy/+HIj5fVauXInRo0dj3LhxiuuUl5dj2rRpiI6OFpc9/fTT+P7777FgwQJNx3n//fcxY8YMrFixAj/+8Y91tZFdD0RERDLi4+MlL7mgYGQiQreWlhasWbOmS5eCp3/+85/Yv3+/OGmh25YtW1BdXQ273Y6IiAgMGzYMAJCTk4PCwkLJuh988AEmTZqE//mf/8H06dNV2ySHQYGIiMggIxMRuq1btw4OhwP/7//9P8V1Vq5ciezsbGRmZkqWP/XUU/jkk09QW1uL2tpabNq0CcCZOzAeeeQRcb2qqipMnDgRjz32GGbNmmXkI7LrgYjCiyVOfkCYEUJzi2n7ovCldyJCt5UrV+L666/HOeecI7vfpqYmrFu3Dn/605+6vPejH/1I8nNcXBwAYOjQoTj33HMBnOlu+MUvfoE5c+bghhtuQF1dHYAz4UbPgEYGBSLqccwMA/4ch0GCgDMTETY0NGDRokWoq6tDVlZWl4kIvR9lsH//fmzbtg3vvvuu4n7XrFkDQRAwdepUQ+1atWoVWltbUVpaKgkpl19+eZc5j9RYhB7wwIWmpiYkJCTgqqgbEWGJDHVziCiIghUKzMLw0FWn0IEt7evQ2NgoGRxoJvd14tb3pyIqzs+7Hprb8cqVfw1oe3sSVhSIqFvpacHAm3f7GRyop2NQIKKQ6+nhQI3nZ2NooJ6IQYGIQiKcw4EShgbqiRgUiChoemM4UMLQQD0FgwIRBRTDgW/uc8TAQN0RgwIRBQQDgn6sMvjvZHsfRDr8u+uho51313liUCAiUzEgmINVBuouGBSIyBQ9ISBYFB7FK7S2Brkl2jEwUKgxKBCRX0IdEJQu/oHaR6hCBQMDhQqDAhEZEoqAYEYoCEQbghkeGBgo2BgUiEiXYAaE7hAMtPBuZzCCgyWuD8MCBQWDAhFpFuiQ0FOCgS/BCg6sLlAwMCgQkU+BDAjhEg7UeH7GQIQGBgYKJKvvVYioNwtUSLDExvaKkOAtkJ871ANLKTyxokBEsgJx0QlaMOgTY85+Wk6bsx8ZgaoysLpAZmNQIKIuzA4JAQsIZgUCvfs3OUC4z4/ZgYFhgczAoEBEom4dEAIdCvSQa4sJ4cHswMCwQGZgUCAiAOaGBFMCQncKBlp4ttfP0GBmYOhtXRGNDjsiIux+7aPTYTGpNeGBQYGITAsJfgeEnhYOlJgUGswODL0lLJC5GBSIerFwDwiuPsa+WVpbHOY1woTQYImNZVigkGFQIOqlQh4STAgHRoOAP/v1K0T4ERrMqi4wLJBeDApEvZAZISEUASFQwcDfNhgKD+7zEILA0NvGLZB/GBSIepmQhQQDAaE7BAMtPNupOzT4ERhYXaBgYFAg6kX8DQnBCAiBCAfO2EhN69laO/w+luHQYCAwmFVdYFggNQwKRL1E0ENCEAOC1iDg7370BglDocFgYGBYoEBhUCDqBYIaEnQEBCPhwKxQYITcsbWGB/dn1RUYGBaoG2BQIApz/oSEQFUR9AaEUIYDXzzbpiU06Koy6KwuMCxQIDAoEIWxoIWEAAQEf8OBM8a/X2+20536j2kwNJgZGPwdt8CwQN4YFIjCVHcKCVoDgt5w4G8YMLJvrQFCT2hw9bFr65LQ0R3hT3WBYYE8MSgQhaGghAQTqwhaA0Igg4FW3m3QEhzcn08tMOiqLjAsKGppt8Pm57MenO0mNSZMhP7/OiIyVXcJCWYFBH/DQWeMVdf6EaddutbXExxMCww6uyJ6W1ggczEoEIWR7hASzAgIesOB3jBgdF9aQoS77WYEBrOqC2Y9K4J6JwYFItIWEkyoIpgVEMwMBnp4H1ctOHh+FqXQ4CswaK4uBDAssKpADApEYcJoNcGMkBDogGA0GHTGGtsuolVb94Nnu7SEBn8CgxldEQwLZASDAlEY6K4hIVgBwWgg0Ls/tQChJTRoCQyBri4wLJBeDApEPVxPDAlqAUFLODA7GGjlfVyl4OArNKgFBlOqCwwLZCIGBaJeKFQhwZ+AoCccdEZbNK+rJqJNUD+OR5t8hQYjgaE7hgXqfUITy4nIFEaqCf6GBFcfu2JIcMZGyoYEZ0yEYkjojLEqhoTOWKv4UtIZbenyMouefftqq9rnVDo3SucT0HB3iYbBp0aeBmrGY8rD0fLly5GRkYHo6Gjk5uZix44dquufOnUKRUVFGDRoEOx2Oy644AJs2rRJdt0lS5bAYrHgnnvukSy/4oorYLFYJK8777xTfP+TTz7B1KlTkZ6ejpiYGIwcORJ//vOfdX82VhSIeqhQhQQleqsIahUEX8HACF/b+aoeKO1Hbjt3++WqDEoVBiPVBZ/jFgJUWWAXhNTatWtRXFyMsrIy5ObmYtmyZSgoKMD+/fuRlJTUZf329nb87Gc/Q1JSEtavX4+0tDR89dVX6NevX5d1P/roIzz//PMYM2aM7LFnzpyJhx56SPw51uP/8ZqaGiQlJWH16tVIT0/H9u3bMWvWLNhsNsyePVvz52NQIOqBenJIUKseKNESDvytJKhtrxYiPLfzXs9XYFDqjjC1K4LdEAG3dOlSzJw5EzNmzAAAlJWVYePGjSgvL8f8+fO7rF9eXo6TJ09i+/btiIw88/9ORkZGl/Wam5sxbdo0rFixAn/84x9ljx0bG4uUlBTZ9371q19Jfh4yZAiqq6uxYcMGXUGBXQ9EvUCgQoLerga5kKBarlcp9wequ8GfYym9r/QZlbojlM6f2gBR1a4IHY/+1qo3dEE0NTVJXg5H1zDW3t6Ompoa5Ofni8usVivy8/NRXV0tu9+33noLeXl5KCoqQnJyMi688EI8+uijcDqdkvWKioowceJEyb69vfrqq0hMTMSFF16IBQsWoNVH4GtsbMSAAQNU1/HGigJRDxOQX9AGQ4LschOqCGrhQAtntKbVfLK1yS/31f3gfl9rhUGtO8K7uqB2V4Q/lYVw6YJocUTBZovyax9Ox5m/t/T0dMnykpISLF68WLLs+PHjcDqdSE5OlixPTk7Gvn37ZPd/8OBBbNmyBdOmTcOmTZtw4MAB3H333ejo6EBJSQkAYM2aNdi5cyc++ugjxXbecsstGDx4MFJTU7Fr1y7MmzcP+/fvx4YNG2TX3759O9auXYuNGzeqfn5vDApEPUhAuhxCEBLMDAhmhQKt+5YLD4rBwEBg0BIWAPVxC5qeRCmDXRBSR48eRXx8vPiz3e7fw6bcXC4XkpKS8MILL8BmsyE7OxvffPMNnnjiCZSUlODo0aOYM2cONm/ejOho5X/gs2bNEv88evRoDBo0CFdffTW+/PJLDB06VLLu7t27MXnyZJSUlGDChAm62sugQBTGghUS/K0iyJbrTQgHnQZDRIRCJcH72N6hQWm8glpgCGpYCMB4he5YVTBLfHy8JCjISUxMhM1mQ319vWR5fX294tiBQYMGITIyEjabTVw2cuRI1NXViV0Zx44dw0UXXSS+73Q6sXXrVjzzzDNwOBySbd1yc3MBAAcOHJAEhc8++wxXX301Zs2ahfvvv9/3B/fCMQpEPYTeaoKRW9/cAhES5Prp5fr0lfr5ndFnX3I6o7u+jNK6L7U2af1ssudFZuyC2i2UunC8gqmioqKQnZ2NyspKcZnL5UJlZSXy8vJkt7n44otx4MABuFxnA+Hnn3+OQYMGISoqCldffTU+/fRT1NbWiq+cnBxMmzYNtbW1siEBAGprawGcCSJue/bswZVXXonCwkI88sgjhj4jKwpEvZXCBSNQIUHys44Kglow0EprBUJpTILc8byrDu5jyFUZ5LokjFQXlG6hlKssBHu8Qm9WXFyMwsJC5OTkYNy4cVi2bBlaWlrEuyCmT5+OtLQ0lJaWAgDuuusuPPPMM5gzZw5+85vf4IsvvsCjjz6K3/72twCAvn374sILL5Qco0+fPjjnnHPE5V9++SVee+01XHvttTjnnHOwa9cuzJ07F5dddpl4K+Xu3btx1VVXoaCgAMXFxairqwMA2Gw2DBw4UPPnY1Ag6gFMrybo/FZpNCQY7WYwGg78Ha+gtL38uISzf/YMDXKBQa7rQXaZH10RoQ4L4dwF4cuUKVPQ0NCARYsWoa6uDllZWaioqBAHOB45cgRW69n/F9LT0/HOO+9g7ty5GDNmDNLS0jBnzhzMmzdP8zGjoqLw3nvviaEkPT0dN9xwg6RrYf369WhoaMDq1auxevVqcfngwYNx+PBhzceyCIKgbZaREGpqakJCQgKuiroRERadZTaiHi6YIUGumhDIkKDUxdB1u67LlNZV4lQZh2bTOfZPqfIgN7ZBbl3vaoLcPA3egUFuzgXvsKA0z4JiWPAxXkFvVUEpKHQKHdjSvg6NjY0++/yNcl8nLnh1Pmyx/g06dLY68Pm0JQFtb0/CigIRAQhsSDBaRZALCGrhQC0MqNEbIpQGNLrbq6XCoLe6oKWyYPbgRlYVCGBQIOrWglVN8PncgB+YERLMDgh6w4ErRnqxtZ5WH9PtvX/v4CAfBM78Vy0wKHVH6A0LXdqrMoOjLI5XIB8YFIh6CxPGJXgzOyRoDQhq4cA7CPjia33vIOF5bM/QoCcw+Kou6AkLpoxXMBGrCuGHt0cSdVPBuh3SaJeDPyHB+5ZCuVsQvddx2s++JO2PcUleWrjsAlx2bcOz1PYv1x652yXlPpv0ffVAJXf7pGR/Gqd7Vqwc+QiR/txqSz0fKwpEYcDMLgezQ4LeKoL3+3LVA7VAoDUAaFnX6ug6tsJ9bM9Kg7uNahUG7+qCXFeE2ZUFOf5MxqQVqwrhhUGBqBvqThPYBCsk+BMQ9IQDAED0Dw/faZOfuEZpv57BwbMt7tCgFBi8uyOUuiLMDgu6xyuo6CljFdrbImG1+vesB1ebvu6rcGeo62H58uXIyMhAdHQ0cnNzsWPHDtX1T506haKiIgwaNAh2ux0XXHABNm3aZKjBRCQVyGpCoEKCd1eDr5AgV/Z3dx/IhoRop/pL73o+jundPu8uCbmuFsnnlbynrxvCFzO7IPToTmGX/KO7orB27VoUFxejrKwMubm5WLZsGQoKCrB//34kJSV1Wb+9vR0/+9nPkJSUhPXr1yMtLQ1fffUV+vXrZ0b7iUiNn10OavwJCUrvyQUEyc9KlQOZi7oSm126rtOhUlXw3q9HBcLdFrkqg1KFQVo9OPNfz64IrZUFT8Ec3NhTqgpkLt1BYenSpZg5c6Y4NWVZWRk2btyI8vJyzJ8/v8v65eXlOHnyJLZv347IyDO/iDIyMvxrNVEY0/NNLJCDzHxVEyTvGQgJalUETQFBJRx4hwE1aut2CRGex/whNCgFBu8xDHJhAZB2RWgNC2aNV5DFsQrkRVcNy/1Uq/z8/LM7sFqRn5+P6upq2W3eeust5OXloaioCMnJybjwwgvx6KOPwulU/p/T4XCgqalJ8iIinYLU5RD0kCDTLWCzOyUvJfbo9i4vNar79WqHd5eEXHeE+GeVrgit3RB674TQ1QWhgndA9D66gsLx48fhdDrF+avdkpOTxYdNeDt48CDWr18Pp9OJTZs24YEHHsCf/vQn/PGPf1Q8TmlpKRISEsRXenq6nmYS9VihqCb46nLQGhK67FfDhdC7L9/zwtplLIBKQPAmFwiUQoHSunLrGw0MSp/XSFjwpHe8gmYcq0AeAj6PgsvlQlJSEl544QVkZ2djypQpWLhwIcrKyhS3WbBgARobG8XX0aNHA91MovDixwyMSo8zBtQvTN7zJJxdDtnl3gHBOySIvC7Ect/wtVYJ+tjbZV9KlPYpW2WQCQyen8+TP2FBNTiwqkABoGuMQmJiImw2G+rr6yXL6+vrkZKSIrvNoEGDEBkZKXl+9siRI1FXV4f29nZERXW9jcVut8Nu9++hHkQ9TSi+eentcpC8p3DxMhISxD/LdTP8QKlyIEft4q9n/RbH2d9PnsdytJ1dbrM7pWMZop2y4xfkBjoqDXL0NWZBz3gFb5pvmTRxrAL1bLoqClFRUcjOzkZlZaW4zOVyobKyEnl5ebLbXHzxxThw4ABcrrP/cD///HMMGjRINiQQkW+q3+j8fJ6DEi3jEkwLCTJVBE9y3/K1VAji7A7ZlxKlfXofX7a64EGtK0L8s8qdIXLUxit4UqsQiW0KcFXB0ocViJ5Md9dDcXExVqxYgVWrVmHv3r2466670NLSIt4FMX36dCxYsEBc/6677sLJkycxZ84cfP7559i4cSMeffRRFBUVmfcpiEg3PdUEvX3hfoeEHyh1MbgpXcj1BgIt68sdSzUwaOyKUAoLcsvUuh08GemCkGXiWAXquXTfHjllyhQ0NDRg0aJFqKurQ1ZWFioqKsQBjkeOHIHVevYfaXp6Ot555x3MnTsXY8aMQVpaGubMmYN58+aZ9ymIerhAD2LU+43R3y4HuWVGQoKbXPXAm1oQcOtnb5Ndfsoh/xXec5/NjrMfwH18d9eEu33uLglJd4RXV4T7NkrPWyg9uyHc/O2C0CtYD42insciCILOuU+Dr6mpCQkJCbgq6kZEWPRNCkPUE5gWFDR2OxitJhjtcjArJGgNCEqBQA+l8OAZGADpOAZAOn5BMnbBY7ImzzkX3GHBMyh4zrPgOd2ze7nn/AreEzF5hgXPsQpy8yp4j1VQDAoqYxW0TMDU6WpH5Xer0NjYiPj4eJ/rG+G+TmSsfADWWA19NypcrW04fPvDAW1vT8JnPRCFOTOrCT63VZkrATA3JHgHBK3h4Bz72QmATjiUA5rn/jxDg/u47sAgV10wUlnQMrjRTamqoEbLJEzhUFVwttsg2NSf4eGLq92/7cMNgwJRD2KkmuDNVzXBk95qgtwydzXBV0jQ2tWgNSB4BgIlaut4hgi50CAXGIyGBTe1GRy9l2npgvB1BwSRFgGfR4GI1AXytshAVxN8jdSXfQKkQiXBTW6woptnSOhnb+sSEs6xt4gvfynty/u4nm3ybK/nQEelOyLc50JpcKOblrsgtAjEoEbOqRD+WFEg6sX8rSacXeaxT5VxCb66G5S6GnxVEZSCwYBIY4HhZIc0vLn3711l0FNdUKosuHk/HwIITVUhHLofyFwMCkQ9RCC6Hfzhq8vBzXtWQgCGQ4KWgGA0HCjtwzM0eAcGd3s8A4NcWHCTCwu+xiu4Kd0FoZf3WAXNEzBRr8WgQBRC3bXbwWg1wU21y8GEkOAdENTCQWJks+J7co53xMnu2zswqFUXvMOC4piFH6iNVzBjYKMpYxVUZmrk46fDG8coEPVSWmbsU9xWZzVB7jHRclMymxUSEiObxZdeStsOiGyRHMt7/ILauAUAXSZmAiA7RkOuAqP1fHvy54FR/s7iSeGFFQWins6EbgelQYz+VBN8dTm4uS+gekOCd0DwFQoSI75Xff94Z9+u2/ywT88qg3eFwbO6oFRZcPOsLIhkuiDcfHVBnF2m7RZJcb/sfiAdWFEg6gGCObLc1zdRvd9u1aoJcg91MiskJEZ8L3n5ora+UoVBrm1ylQXvaZ8B+YqK3F0Qbr4qCG5Kz4DwZ34MEad07pUYFIjCkK/Ssd5uBzOrCWpdDnpDgtwFXGsw8EVLYNASFtzkZpVU64JwkxvvIb5n0m2TcvR2P/A2yfDFrgeiEAnmY6X97XZQ3beWaoLGLgc3rSHBk1I4SIk8pdxAGXUd/WT369k1kRjZLHZHDIhsUe2G8DW40Zu7C0Ludkk3dj9QMLGiQEQiI90O4nsavoBq6XLQGxLkvvmnRJ4SX3opbStXXZBrm9wtm2oPrNJTVdDa7RPQ7gfqdVhRIOrm9M6fYPaIda3dDuLxve90ULkAynU5uGkNCZ60BIOBNmklosEZp7Dmmf15Vhi8qwu+KguegxvdAllVCAqV2yS7hTYbYPHzWQ1tfNaDJ8ZLol5Gy/gEM7odvKlVE9zk+vb9DQkDbc2Sl973fVUXlCoLbu7P5G9VwU3tnGv9exP35ePfAscpEMCgQBT2tI5PMKPbQXYQowIt1QSjIUHtwu+L0rZaw4KbWheE3NwKvugd1KgUGtj9QHrxXwxRCARzIKPZtDygyLvbQU81Qe4CqzckmEVPWHBTqypo8sM5U7tVUi+tky+ZOcU3hQ8GBSJSpFbK9ufWPLVqgpuWZzZoDQkDrQ5NL9ltfYQFN7kuCLWnWHpXFeRuG/VFLrTp7X4g8oVBgainMjD5jZnjE8R9Guh28GakmuB5wVYcf6ASAOQora8WFrR2QWgZq+CL2t0PRIHCoEAURgI1R7+eC5ORbgdvss9vUAgJcvQGBC3baw0LbmY8xVJP94OecQqS7Yw884MzNPYqDApE3Zi/o8jNGsgormfCN1nvbge1/nstD3XyvoD7ExC67NtHWJCjpc1Kgxr13P0QLHxAFDEoEJFugSh9y93p4KbW5eBJLSQMtEWpvhS3UwkLvqoKbt7dD2bSGt60Trzk74DG3nqL5PLly5GRkYHo6Gjk5uZix44diuu+/PLLsFgskld0tPQvcsOGDZgwYQLOOeccWCwW1NbWyu6ruroaV111Ffr06YP4+HhcdtllOH367DwXjzzyCMaPH4/Y2Fj069fP0GdjUCAiWZrK1gH8sqn3EdHKgxHVg4De9fQwo/tBCccpdB9r165FcXExSkpKsHPnTmRmZqKgoADHjh1T3CY+Ph7ffvut+Prqq68k77e0tOCSSy7BY489priP6upqXHPNNZgwYQJ27NiBjz76CLNnz4bVevbS3t7ejhtvvBF33XWX4c/HmRmJyG9KszF6j0+Qe66DHmrVBDlGLvwDbVFocErbOdDqQIPL7rFOszijo/fsjYB0xkazyT162pve5z6Qf5YuXYqZM2dixowZAICysjJs3LgR5eXlmD9/vuw2FosFKSkpivu89dZbAQCHDx9WXGfu3Ln47W9/KznG8OHDJes8+OCDAM5UMYxiRYGIgs57fILabYRangQpf6eC8eqA3LZaxj7oeWql1nEKZs6nQPo0NTVJXg5H138D7e3tqKmpQX5+vrjMarUiPz8f1dXVivtubm7G4MGDkZ6ejsmTJ2PPnj262nbs2DF8+OGHSEpKwvjx45GcnIzLL78c27Zt07UfLRgUiEhVMEvcZpXqze5CMEMgxynQWdZ2C6wOP1/tZ7rd0tPTkZCQIL5KS0u7HO/48eNwOp1ITk6WLE9OTkZdXZ1sG4cPH47y8nK8+eabWL16NVwuF8aPH4+vv/5a8+c8ePAgAGDx4sWYOXMmKioqcNFFF+Hqq6/GF198oXk/WrDrgYhE3XmyHr3dDv6S64KQvt+s+kApN8+HRVHPcvToUcTHx4s/2+3mDMrJy8tDXl6e+PP48eMxcuRIPP/883j44Yc17cPlOlNh+vWvfy12eYwdOxaVlZUoLy+XDTVGMSgQ9RKG7pcPET0DGbvelRC4aoL3WAU3uXEKoeCMBmwsWJgmPj5eEhTkJCYmwmazob6+XrK8vr5edQyCp8jISIwdOxYHDhzQ3LZBgwYBAEaNGiVZPnLkSBw5ckTzfrRg1wMRkQJ/QofeuzYCpTtXicJBVFQUsrOzUVlZKS5zuVyorKyUVA3UOJ1OfPrpp+LFX4uMjAykpqZi//79kuWff/45Bg8erHk/WvScrxhERETdUHFxMQoLC5GTk4Nx48Zh2bJlaGlpEbsEpk+fjrS0NLE74KGHHsJPf/pTDBs2DKdOncITTzyBr776CnfccYe4z5MnT+LIkSP4z3/+AwBiIEhJSUFKSgosFgv+8Ic/oKSkBJmZmcjKysKqVauwb98+rF+/XtzPkSNHxH05nU5xPoZhw4YhLk7bnTkMCkSkiRmzMhKFoylTpqChoQGLFi1CXV0dsrKyUFFRIQ5wPHLkiGRug++++w4zZ85EXV0d+vfvj+zsbGzfvl3SjfDWW2+JQQMAbr75ZgBASUkJFi9eDAC455570NbWhrlz5+LkyZPIzMzE5s2bMXToUHG7RYsWYdWqVeLPY8eOBQC8//77uOKKKzR9PosgCN3+ZtumpiYkJCTgqqgbEWHhY1Cp59P6mGnVWe5k5tv3nm7Xc5Y97zEKclM4e5ap3X/2vOvBHRbcy7wfCKV1HgWl2yPddz3IPQhKaTBjoMcoeA9o9Byj4DmY0T1G4XhnX3GZey4F92DGE44z/z3lOHMCmx1n9tXiONNmR9uZ/zodtjM7aDvzX6vjzN+F9fSZvyf3PAru8QgRHuMSzi47+6vd/eeI1rO3WEacPvtn2+lOyWe0tXZIfra2yNwa2nK667IfCK2tkp87Xe2o/G4VGhsbffb5G+W+Tgx+7I+wRvuXal1tbfhq3v0BbW9PwjEKREREpIhBgYhIgdrtkb4EamZGvThDI/mLQYGol/AuL3dnei6y3rcr+nNx13sst+5wayTAWyMpMBgUiEjUnb99el6MtUx05C9fgUNrGzjZEvV0vOuBiFTZ2oI3jfPJjj6mTOPc4GzvdtM4ew9kpMCwnrbCKvj5HbiN36E98WwQUcC5R/O7R/e7R/u7L5rui6gcz7sIlMh1CfjTBSG3rVK3gyctbXUzescDUbDxXx4R+c19EXNf1NwXOfGiZxK93Q9GwoKWkCB3W6SnQA5k9PWIaaB7dyFRz8OgQESytFxstFy0jNJ7sVX6xt/gbNcUGLSup0cgxydw4CIFC4MCEekWiIuUu/tB7uLqWdJXqyqodQ+4g4DSS3E7jdUEtW6HQI5PiND4d6E02ZI378mW9PKebIl6PgYFom7M31+6ar/0PS8WnhcRNVovSmqUxinI0VJV0BMW9FILCUq0tFnr+ITuQHZWRupVGBSIwkigfqnrqSAojVNwXwy10FNVkNPgsvsVGOS29w4JvqoJ3tM2G6FnIKPc35Gm7iMj82uoTN9M4YdBgainMvDLWstFQe9AOPc4BX9G5avd/eD5DV2tC0Lu277ewKC0vtaQoFZN8H6+gxHez3ggCgbOo0AUAkJzi+YHQ4VSRJsgeVCUJyPzKzjaomCPbkeLIwp97O1odtjFB0R50zKnQl1HP8nDohqccZIHRonL/akuqIQET54hQUs1QbHbQQe5riDe8UBmY0WBiHTRMk7ByG2SRqoKQNcLt1mzNspVKbyPpdbl4EnXIMYAzJ8QrIGMFJ4YFIjCnFkDGtXK3Ua6H7wHNXqSuwNCb1hQ6o7wRWk7tZAg1+UgF3iUBjFqoXYrqtHxCURaMCgQhRlfAxrNHKegqa9cx6BGuW/eRsOCm2do6HKHhNd7SgFBa0hQqyaojU3Qc7eD2jnXPb7Ex78FvYNjeWtkeGJQIOrmVH/5BmH0udzFR637ocssjSq0VBUAfWHB5x0ROqoNcgFBS0hQmzdBSzVBS7eDGbeqGtLN73iwOs5UX/x5WXlHqASDAhGJzOh+UKNWVZCbV0FrWFAKDEYe/6y0rfcxfIUET/5WE9TudvDV7aB1fAKREt71QBQiwbzzwdbaAWdspOx7Eadd6Izp+p1B7Y4Hyb5V7n6wOixw2YUzF8Fop+Q97zsgPJ1yRKOf/cwV8ISjD86xn7n7wfNOiOMdcUiMPHuHg/tCnhjxvWRfRsKCJ7kBi1pCglyXQyCrCf52O3AgIylhRYEoDJkxTsGT1u4H1UGNKndAyHVBaK0seA8klKswGCG3H+/j+QoJntyf0ZOeaoLsewHsfuD4BHJjUCDqAYI5TsFI94PcMrWxClq6IABtYQFQDwxag4Pa+nL71xISvO9yANTnTVCrJrjPsa+xCQHtdujm4xMoMNj1QNRL+Nv9INcVEdEGdHp9ebY5AKdSl7zGLgjPiZjUuiEASCZlcl/MPbskxPcMVBnkbnv0Dim+QoIn2S4Hr3kTPGl9Oie7HSiQWFEg6qUMzfHv3tafqoJMF4T7Aur5zVtLZQE4c+FWqjDofVS12rbexznh6KMpJMiNS1DrclCrJvha5knrg77k8EFQ5IkVBaIQCuSARmuLA64+2qcu9qwqRLS60Bn7w59NqiqIAxs9OB022OzSCoPWygIAsboAyFcYAG1Pc1QjNzeCd1gxFBI8qFUTPAOBr0GMSpWFQHc7cHxCeGNFgaiHMGOcgneJOZBVBbmBjd5TOwPy4xXUKgve1QWlCoPcBV4rpX3IVRF8hQRPkpAg0+XgPldyXQ6eIcGfQYzsdiC9WFEgCmN6qwqejFYV5G6XtJ62whXj9a3WY7yCu7LgHq8AdK0sAJCtLgDyFQZAvhpghNzcCGrjEXxWEkzqclCqJvgziJHdDuSNFQWiEBOa1Z+QGEx6Lyq+yuKe34zVxisAypUFrdUF4Ow3frWnNmqltC9fVQQ9IUHuLgdf1QR/GK4msNuhV2NFgagHEVpbYYmNlX+z5TTQJ8bnPrzvfrCd7oQzRv5XgZaqglwFwXOZz/EKCpUFAJqrC25yVQY57sqD1kAhNy+CUhUB0BcSPHmGhFBUE4jkMCgQhTl/BjXqpacLwldYANClKwKAYmAAlEODNy0BQenR0GoBwd1mNy0hQW5cghkDGI0Ih24HmwPw/WgtH3r+aTAVgwJRN6Dn7odgVBX03gHhGQrkwoJnVcGfsACgy5TPcoEBUL7QewcIpfW8yc2LoFRFAPSFBE9KIUHLAEa1agK7HcgoBgWiXsCfQY3e9IYFN3/DAgDF6gIgvZB7hwZPWoOB9z49aa4iAJpCgp67HLR0OXjTcndLOFQTKDAYFIjCTYCrCnopjVfQGhYA+AwMgHJo8OYdItTW9SR3u6PWKgKgLSToGZegxtfYBDOqCdR7GPo/f/ny5cjIyEB0dDRyc3OxY8cOTdutWbMGFosF119/vZHDEoU1PXc/GCn5avnGqPbNU1LWVvhWq6VsLncnBCBzN4TXHRHeMzl6X6TddxzIXdA9NTvskpcapX16H9+7fd5VBL0hQctdDqGuJuj5Nyi0sIuiJ9MdFNauXYvi4mKUlJRg586dyMzMREFBAY4dO6a63eHDh/H73/8el156qeHGEpG5fH2zVPtmGqiwoHT7JNC1rO++YKuFBu+XN63ryh1LNiCodDXoDQlGuhxYTSCz6Q4KS5cuxcyZMzFjxgyMGjUKZWVliI2NRXl5ueI2TqcT06ZNw4MPPoghQ4b4PIbD4UBTU5PkRURSRmZqNFJV8LzwqH1z9Tcs6K0uaA0N3nwFBy37lDu+1q4GwLyQ4M3IAMZAVxOo59MVFNrb21FTU4P8/PyzO7BakZ+fj+rqasXtHnroISQlJeH222/XdJzS0lIkJCSIr/T0dD3NJOqxQjH5kt4pfJW6ILx/1hoWdFUXZCoMcs9O8LzAy730rqd6PJkqglpXgz8hwZs/D35SZWI1oTtNKEbG6AoKx48fh9PpRHJysmR5cnIy6urqZLfZtm0bVq5ciRUrVmg+zoIFC9DY2Ci+jh49qqeZRL2GWVUFX8+A8P6mqhYWJPvReKufr+qC1sCgFBy8aak8+NyvhoCg1tVgJCTo6XJgNSG49Izd27BhA3JyctCvXz/06dMHWVlZeOWVVyTrNDc3Y/bs2Tj33HMRExMjVvDdDh8+DIvFIvtat26duN5vf/tbZGdnw263Iysry9BnC+hdD99//z1uvfVWrFixAomJiZq3s9vtsNvNuZWLqKcx9YmSGu+AkKN2F4Q3tWdBqM2xACjP4Oj9fAj3RVhyd4RbtPQJlL7Cgs3u1BQoupB7RoPMDItauxqA4IQEXVhN0M09dq+srAy5ublYtmwZCgoKsH//fiQlJXVZf8CAAVi4cCFGjBiBqKgovP3225gxYwaSkpJQUFAAACguLsaWLVuwevVqZGRk4N1338Xdd9+N1NRUXHfddUhPT8e3334r2e8LL7yAJ554Aj//+c8ly3/1q1/hww8/xK5duwx9Pl1BITExETabDfX19ZLl9fX1SElJ6bL+l19+icOHD2PSpEniMpfrzD/oiIgI7N+/H0OHDjXSbiLyg9y8Ct63S/rifcukkbDg/Z77gup5CyUA9cAAdL2AewUHb5pDgkww8G6HZJlKQADU72zQGhK8aQkJrCYElufYPQAoKyvDxo0bUV5ejvnz53dZ/4orrpD8PGfOHKxatQrbtm0Tg8L27dtRWFgorjtr1iw8//zz2LFjB6677jrYbLYu193XX38dN910E+Lizj5a/amnngIANDQ0GA4KuroeoqKikJ2djcrKSnGZy+VCZWUl8vLyuqw/YsQIfPrpp6itrRVf1113Ha688krU1tZy7AGRCXz+8g5SFwSgPujO+yLpqytCrTsCOFvql7tgi90C/r68KB1TrptBa1dD13OjHhL0jkvQFRJYTZDwHlTvcHQ9b0bH7rkJgoDKykrs378fl112mbh8/PjxeOutt/DNN99AEAS8//77+PzzzzFhwgTZ/dTU1KC2tlbzWEA9dHc9FBcXo7CwEDk5ORg3bhyWLVuGlpYWMUlNnz4daWlpKC0tRXR0NC688ELJ9v369QOALsuJ6Cy93Q+q0zr7yVcXhK/KAgDFB0ipdUUA8t0RbnJVBvE9u7aJiXyRDSEybXGTm2FRravB+329IcG0aZo16CnVhIg2wObnX7/lh79H7y+zJSUlWLx4sWSZ2ti9ffv2KR6jsbERaWlpcDgcsNlsePbZZ/Gzn/1MfP/pp5/GrFmzcO655yIiIgJWqxUrVqyQhAlPK1euxMiRIzF+/Hgdn1Qb3UFhypQpaGhowKJFi1BXV4esrCxUVFSIJ+nIkSOwWvn0aqJuRWGsgtYuCH/CgvfP3oHAfeFUG7vgJhcaPAOD+J7KBd4fcuEA8D8gnHnf3JCghNUE7Y4ePYr4+HjxZzPHzvXt2xe1tbVobm5GZWUliouLMWTIELGr4emnn8a///1vvPXWWxg8eDC2bt2KoqIipKamSqoXAHD69Gm89tpreOCBB0xrnydDgxlnz56N2bNny75XVVWluu3LL79s5JBEvU6wqgqhCAuAenXB/T7QtcIAKFcZAPng4A+lcODZHsky2ac9qq8TiJBgZpdDT6kmmC0+Pl4SFOToHbvnZrVaMWzYMABAVlYW9u7di9LSUlxxxRU4ffo07rvvPrz++uuYOHEiAGDMmDGora3Fk08+2SUorF+/Hq2trZg+fbqRj+kTv/oT9RY6vzFqKVvrGbMg97Pct265i6rSGAa5C7V7vIDay8i6vo4t1065sRjdLiSYLFyqCVrpHbunxOVyiWMgOjo60NHR0aU6b7PZxBsCPK1cuRLXXXcdBg4caPBTqONDoYi6MdOrCjq6IICulQXvqgJgrLIAQLEr4sw6+GEdj7bIrAd0vWA7NVSH1aoE3uTCiOR9DRUEufXk7mQIVEhQxWqC3/SM3QPOTCqYk5ODoUOHwuFwYNOmTXjllVfw3HPPAThTybj88svxhz/8ATExMRg8eDA++OAD/OUvf8HSpUslxz5w4AC2bt2KTZs2ybbtwIEDaG5uRl1dHU6fPo3a2loAwKhRoxAV5XsuEYBBgaj30RkWvBkNCwB8dkUA2gOD97ri+yoXdqUQ4SsMdFlf4aFNWgLCmfV83/6o9xkOgHJICFaXQ2+rJrjpHbvX0tKCu+++G19//TViYmIwYsQIrF69GlOmTBHXWbNmDRYsWIBp06bh5MmTGDx4MB555BHceeedkmOXl5fj3HPPVbwb4o477sAHH3wg/jx27FgAwKFDh5CRkaHp81kEQTBneHAANTU1ISEhAVdF3YgIi/b7vInChd4JmHyOVVCZhEkuLMjNr+AdFgDITsjk/Xhqz3CgtkwuBHTKLPO1jRmUgoGb0YAgt0zu9ke5kGBKl0OQgkKn0IEt7evQ2Njos8/fKPd1YtTdj8Jm9+8fgtPRhs+evS+g7e1JWFEg6gGC1QUBGB/cCMjP3mikugCoVxjObN+17UoXdC0BwlcY8Kb0KGijAQEIz5BAPR+DAlGY8icsyDEzLAC+xy6Ix1AYm+B9oVarNugNAXKUgoGvY5gZEpRugQx1SKDwxqBA1EOY+gwINz8HNwJnL17et04C6DJuAYCm6oL3MsD32AS5C7mvrgolvkKBXJu67iPwVQTA/MGLRrCaEN4YFIjCmD8zNuoJC4D/1QVAPjB4Lwd8h4az+1B+zyi94UB1eYBCgj+TKrHLgbxxHgWiHsTIL2Wjz4IAlC84ShcouQua3IUvotUlf5FsExS/iStdbN1zE3i+zKJ133rbrfj5AxkSNGCXA8lhRYGohzHSBWH24EZAf2UB6HpXhFx3BKDc/eB90ZW7WwIwNyzIUXuao9r7Sg90Urr1Ue88CcEel9Adqwk2h//PekC7KU0JGwwKRHSGwbAAdL19Um7cAiDfFQHoDwze77sprecvX8HA1zpmBAQgcCHBiO4YEigwGBSIeqCAVBUAQ2EBMKe6APgODIB6GPB1QdcaOPQwEhCA7hUS2OVAahgUiHqoUIUFQH5SJrWwAMhXFwB9gQHwr4rgTyDQsw8zAwLgx3iEAIUEVhN6FwYFoh4sFGEB0N8VAchXFwBtgQGQDw2A/IXbrC4IPcFCLRwA6lMwm15FABgSyDQMCkS9UCDDAqC/ugCoBwZAW2gQ1zWhcqCFr3AAGAsIQPcMCdQ7MSgQ9XBGJ2IKRlgAlKsLgHpgALSFBsB3cDCLlmAA+H6Ak9GAAIQ2JLCa0DsxKBCFgVCHBUB+3AKgXF0A1AMDoC00AOoXcL0hQmsY6LKdlqc7qgQEIPBVBIAhgfRjUCAKEwEPC0BAqguA78AAaA8NXbYzeOHXtG8N4QDwLyAADAkUWgwKRGEkoGEB8KsrAtAeGADtoQHQFxz8oTUYAL7DAWBCQAAYEijgGBSICIC5YQFQ7ooAfAcGQFuVwU3tAq43ROgJA960hANA24OcukNIIAIYFIjCjj9PmTQrLADmBwY3LcHBkz8Xfi20hgPAxIAABCUksJpAAIMCUVgKWlgANAUGtbAAaAsM4rp+Bgd/6QkG4jYhCAhA7wwJEW0CbC7/bo+1tAfn9tqegkGBKEwFJSwAplUXAOkFVUtoAJQv3P4GCCOBQLK9hnDgxpBA3RmDAlEYC2pYAEwLDICx0CDZ3s8LvRF6wgEQmIAAMCSQuRgUiMKcv2EBgKnVBUB6gdQbGgBjwSEQ9AYDQEc4AIIaEACGBJLHoEDUC/gTFoDAVBfc9FQZ3EIVHIwEAzddAQFgSKBug0GBqJcIalgADAcGQF9oALRdwH2FCX9CgBLd4QDQHRAAhgQKLAYFol7EjLAA6OiKAHQHBsC/0KAkEEFAjqFwAIQkIAAMCeQbgwJRL+NvWAAMVBcAQ4EBCExoMJvhcAAYCggAQwIFD4MCUS9kVlgAdFYXAMOBAZC/IAc7PPgVCjwxIFAPwaBA1Eu5LxYhDwyAodDgpnbhNhoiTAsD3gyGA8C8aZgZEkgvBgWiXs6M6gJgsDvCzY8qg5qAXfD18CMcuDEkUCgxKBCRqWEBMFBdcPO+qJocHILGhHAAmPswJ4YEMopBgYgAmBcWABMCg5tJ3RNBYVI4ABgQ/GFzCIjw81kP6OCzHjwxKBCRyKxxC+L+zAoMgPyFOFThwcRQ4Mnsx0H3tpBAgcGgQERdmFldAEwODJ7ULtj+hogAhQE5ZgcEgCGBzMOgQESyzK4uANILoumhwVsQL/RGBCIcAAwIZD4GBSJSZXZ1QdxvoKoM3VigwoG4f4YECgAGBSLyKRDVBXHfwawyhECgwwHAgECBxaBARJoFqrog7t/rotpTg0MwwgHAgEDBwaBARLoEsrrQ5Vg9JDgEKxhIjsmQQEFiDXUDiKhnEppbgn6xElpbu7yCLdRtCMV5J9+WL1+OjIwMREdHIzc3Fzt27FBcd8WKFbj00kvRv39/9O/fH/n5+V3Wv+2222CxWCSva665RrLO559/jsmTJyMxMRHx8fG45JJL8P7770vW+e1vf4vs7GzY7XZkZWUZ+mwMCkTkl1BfuOQu3L5eZmwb9M/JgNBtrV27FsXFxSgpKcHOnTuRmZmJgoICHDt2THb9qqoqTJ06Fe+//z6qq6uRnp6OCRMm4JtvvpGsd8011+Dbb78VX3/9618l7//iF79AZ2cntmzZgpqaGmRmZuIXv/gF6urqJOv96le/wpQpUwx/PgYFIjJFT7qQhfqir0dPOq+91dKlSzFz5kzMmDEDo0aNQllZGWJjY1FeXi67/quvvoq7774bWVlZGDFiBF588UW4XC5UVlZK1rPb7UhJSRFf/fv3F987fvw4vvjiC8yfPx9jxozB+eefjyVLlqC1tRW7d+8W13vqqadQVFSEIUOGGP58DApEZCpe2MzB8xh6TU1NkpfD0fUhY+3t7aipqUF+fr64zGq1Ij8/H9XV1ZqO09raio6ODgwYMECyvKqqCklJSRg+fDjuuusunDhxQnzvnHPOwfDhw/GXv/wFLS0t6OzsxPPPP4+kpCRkZ2cb/MTyOJiRiALC8yIXjIGP4YDBwH8RrS5ERLr820nHme3T09Mli0tKSrB48WLJsuPHj8PpdCI5OVmyPDk5Gfv27dN0uHnz5iE1NVUSNq655hr813/9F8477zx8+eWXuO+++/Dzn/8c1dXVsNlssFgseO+993D99dejb9++sFqtSEpKQkVFhaTyYAYGBSIKuGDeKdETMSB0T0ePHkV8fLz4s91uN/0YS5YswZo1a1BVVYXo6Ghx+c033yz+efTo0RgzZgyGDh2KqqoqXH311RAEAUVFRUhKSsI///lPxMTE4MUXX8SkSZPw0UcfYdCgQaa1kV0PRBQ07nI6L4w8Fz1BfHy85CUXFBITE2Gz2VBfXy9ZXl9fj5SUFNX9P/nkk1iyZAneffddjBkzRnXdIUOGIDExEQcOHAAAbNmyBW+//TbWrFmDiy++GBdddBGeffZZxMTEYNWqVTo/qToGBSIKid54oeyNnzncRUVFITs7WzIQ0T0wMS8vT3G7xx9/HA8//DAqKiqQk5Pj8zhff/01Tpw4IVYKWn8YjGu1Si/jVqsVLpefXS9e2PVARCEXzuMZGArCX3FxMQoLC5GTk4Nx48Zh2bJlaGlpwYwZMwAA06dPR1paGkpLSwEAjz32GBYtWoTXXnsNGRkZ4u2McXFxiIuLQ3NzMx588EHccMMNSElJwZdffol7770Xw4YNQ0FBAQAgLy8P/fv3R2FhIRYtWoSYmBisWLEChw4dwsSJE8W2HThwAM3Nzairq8Pp06dRW1sLABg1ahSioqI0fT4GBSLqVrwvrD0tODAY9D5TpkxBQ0MDFi1ahLq6OmRlZaGiokIc4HjkyBHJN//nnnsO7e3t+O///m/JftyDJW02G3bt2oVVq1bh1KlTSE1NxYQJE/Dwww+L3R+JiYmoqKjAwoULcdVVV6GjowM//vGP8eabbyIzM1Pc5x133IEPPvhA/Hns2LEAgEOHDiEjI0PT57MIgiAYOjNB1NTUhISEBFwVdSMiLJGhbg4RdQOhDhAMBNp1Ch3Y0r4OjY2NksGBZnJfJ3567UOIiIz2vYGKzo42/HvTooC2tydhRYGIeiS1C7VZIYJhgIhBgYjCEC/wRObhXQ9ERESkiEGBiIiIFDEoEBERkSKOUSAiorAR0eZCRKefEw75u32YYUWBiIiIFDEoEBERkSIGBSIiIlLEoEBERESKGBSIiIhIkaGgsHz5cmRkZCA6Ohq5ubnYsWOH4rorVqzApZdeiv79+6N///7Iz89XXZ+IiIi6D91BYe3atSguLkZJSQl27tyJzMxMFBQU4NixY7LrV1VVYerUqXj//fdRXV2N9PR0TJgwAd98843fjSciIqLA0h0Uli5dipkzZ2LGjBkYNWoUysrKEBsbi/Lyctn1X331Vdx9993IysrCiBEj8OKLL8LlcqGyslLxGA6HA01NTZIXERERBZ+uoNDe3o6amhrk5+ef3YHVivz8fFRXV2vaR2trKzo6OjBgwADFdUpLS5GQkCC+0tPT9TSTiIiITKIrKBw/fhxOpxPJycmS5cnJyairq9O0j3nz5iE1NVUSNrwtWLAAjY2N4uvo0aN6mklEREQmCeoUzkuWLMGaNWtQVVWF6OhoxfXsdjvsdnsQW0ZERERydAWFxMRE2Gw21NfXS5bX19cjJSVFddsnn3wSS5YswXvvvYcxY8bobykREZEPttOdsEV0+rUPodO/7cONrq6HqKgoZGdnSwYiugcm5uXlKW73+OOP4+GHH0ZFRQVycnKMt5aIiIiCSnfXQ3FxMQoLC5GTk4Nx48Zh2bJlaGlpwYwZMwAA06dPR1paGkpLSwEAjz32GBYtWoTXXnsNGRkZ4liGuLg4xMXFmfhRiIiIyGy6g8KUKVPQ0NCARYsWoa6uDllZWaioqBAHOB45cgRW69lCxXPPPYf29nb893//t2Q/JSUlWLx4sX+tJyIiooAyNJhx9uzZmD17tux7VVVVkp8PHz5s5BBERETUDfBZD0RERKSIQYGIiIgUMSgQERGRIgYFIiIiUsSgQERERIoYFIiIiEgRgwIREREpCupDoYiIiALJdroTNluHX/sQnHzWgydWFIiIiEgRgwIREREpYlAgIiIiRQwKREREpIhBgYiIiBQxKBAREZEiBgUiIiJSxKBARETkp+XLlyMjIwPR0dHIzc3Fjh07FNfds2cPbrjhBmRkZMBisWDZsmVd1iktLcVPfvIT9O3bF0lJSbj++uuxf/9+yTpffvklfvnLX2LgwIGIj4/HTTfdhPr6esk6J0+exLRp0xAfH49+/frh9ttvR3Nzs67PxqBARETkh7Vr16K4uBglJSXYuXMnMjMzUVBQgGPHjsmu39raiiFDhmDJkiVISUmRXeeDDz5AUVER/v3vf2Pz5s3o6OjAhAkT0NLSAgBoaWnBhAkTYLFYsGXLFvzrX/9Ce3s7Jk2aBJfLJe5n2rRp2LNnDzZv3oy3334bW7duxaxZs3R9PosgCIKuLUKgqakJCQkJuCrqRkRYIkPdHCIi0qFT6MCW9nVobGxEfHx8QI4hXiey5iPCZvdrX51OB7bULtHc3tzcXPzkJz/BM888AwBwuVxIT0/Hb37zG8yfP19124yMDNxzzz245557VNdraGhAUlISPvjgA1x22WV499138fOf/xzfffed2MbGxkb0798f7777LvLz87F3716MGjUKH330EXJycgAAFRUVuPbaa/H1118jNTVVw9lgRYGIiEhWU1OT5OVwOLqs097ejpqaGuTn54vLrFYr8vPzUV1dbVpbGhsbAQADBgwAADgcDlgsFtjtZ0NRdHQ0rFYrtm3bBgCorq5Gv379xJAAAPn5+bBarfjwww81H5tBgYiIwoa1xWHKCwDS09ORkJAgvkpLS7sc7/jx43A6nUhOTpYsT05ORl1dnSmfyeVy4Z577sHFF1+MCy+8EADw05/+FH369MG8efPQ2tqKlpYW/P73v4fT6cS3334LAKirq0NSUpJkXxERERgwYICutjEoEBERyTh69CgaGxvF14IFC0LSjqKiIuzevRtr1qwRlw0cOBDr1q3D//7v/yIuLg4JCQk4deoULrroIlit5l7a+fRIIiIiGfHx8T7HKCQmJsJms3W526C+vl5xoKIes2fPFgchnnvuuZL3JkyYgC+//BLHjx9HREQE+vXrh5SUFAwZMgQAkJKS0mVAZWdnJ06ePKmrbawoEBERGRQVFYXs7GxUVlaKy1wuFyorK5GXl2d4v4IgYPbs2Xj99dexZcsWnHfeeYrrJiYmol+/ftiyZQuOHTuG6667DgCQl5eHU6dOoaamRlx3y5YtcLlcyM3N1dwWVhSIiIj8UFxcjMLCQuTk5GDcuHFYtmwZWlpaMGPGDADA9OnTkZaWJo5xaG9vx2effSb++ZtvvkFtbS3i4uIwbNgwAGe6G1577TW8+eab6Nu3rzimICEhATExMQCAl156CSNHjsTAgQNRXV2NOXPmYO7cuRg+fDgAYOTIkbjmmmswc+ZMlJWVoaOjA7Nnz8bNN9+s+Y4HgEGBiIjIL1OmTEFDQwMWLVqEuro6ZGVloaKiQhzgeOTIEcm4gf/85z8YO3as+POTTz6JJ598EpdffjmqqqoAAM899xwA4IorrpAc66WXXsJtt90GANi/fz8WLFiAkydPIiMjAwsXLsTcuXMl67/66quYPXs2rr76alitVtxwww146qmndH0+zqNAREQBFcx5FPLPn2vKPArvffE/AW1vT8IxCkRERKSIQYGIiIgUMSgQERGRIgYFIiIiUsSgQERERIp4eyQREYWP1tOA1eV7PTWurg9/6s1YUSAiIiJFDApERESkiEGBiIiIFDEoEBERkSIGBSIiIlLEoEBERESKGBSIiIhIEYMCERERKWJQICIiIkUMCkRERKSIUzgTEVHYEFpPQ7A6/duHq92k1oQHVhSIiIhIEYMCERERKWJQICIiIkUMCkRERKSIQYGIiIgUMSgQERGRIgYFIiIiUsSgQERERIoYFIiIiEgRgwIREREpYlAgIiIiRXzWAxERhQ2hpRWCpcO/fQj+bR9uWFEgIiIiRQwKREREpIhBgYiIiBQxKBAREZEiBgUiIiJSxKBAREREihgUiIiISJGhoLB8+XJkZGQgOjoaubm52LFjh+r669atw4gRIxAdHY3Ro0dj06ZNhhpLRETUHZl9XRQEAYsWLcKgQYMQExOD/Px8fPHFF5J1Tp48iWnTpiE+Ph79+vXD7bffjubmZsk6u3btwqWXXoro6Gikp6fj8ccf1/3ZdAeFtWvXori4GCUlJdi5cycyMzNRUFCAY8eOya6/fft2TJ06Fbfffjs+/vhjXH/99bj++uuxe/du3Y0lIiLqbgJxXXz88cfx1FNPoaysDB9++CH69OmDgoICtLW1ietMmzYNe/bswebNm/H2229j69atmDVrlvh+U1MTJkyYgMGDB6OmpgZPPPEEFi9ejBdeeEHX57MIgiDo2SA3Nxc/+clP8MwzzwAAXC4X0tPT8Zvf/Abz58/vsv6UKVPQ0tKCt99+W1z205/+FFlZWSgrK9N0zKamJiQkJOCqqBsRYYnU01wiIgqxTqEDW9rXobGxEfHx8QE5hpnXCb3tNfu6KAgCUlNT8bvf/Q6///3vAQCNjY1ITk7Gyy+/jJtvvhl79+7FqFGj8NFHHyEnJwcAUFFRgWuvvRZff/01UlNT8dxzz2HhwoWoq6tDVFQUAGD+/Pl44403sG/fPs3nQ9cUzu3t7aipqcGCBQvEZVarFfn5+aiurpbdprq6GsXFxZJlBQUFeOONNxSP43A44HA4xJ8bGxsBnPnLIyKinsX9u1vn91Jjx0IH4OdhOnGmvU1NTZLldrsddrtdsiwQ18VDhw6hrq4O+fn54vsJCQnIzc1FdXU1br75ZlRXV6Nfv35iSACA/Px8WK1WfPjhh/jlL3+J6upqXHbZZWJIcB/nsccew3fffYf+/ftrOh+6gsLx48fhdDqRnJwsWZ6cnKyYTurq6mTXr6urUzxOaWkpHnzwwS7Lt3a8oae5RETUjZw4cQIJCQkB2XdUVBRSUlKwte4NU/YXFxeH9PR0ybKSkhIsXrxYsiwQ10X3f32tk5SUJHk/IiICAwYMkKxz3nnnddmH+72ABIVgWbBggSRtnTp1CoMHD8aRI0cC9o8sHDQ1NSE9PR1Hjx4NWHkvHPA8+cZzpA3PkzaNjY340Y9+hAEDBgTsGNHR0Th06BDa29tN2Z8gCLBYLJJl3tWE3kJXUEhMTITNZkN9fb1keX19PVJSUmS3SUlJ0bU+IF/eAc6UXvg/o2/x8fE8TxrwPPnGc6QNz5M2Vmtg78iPjo5GdHR0QI/hLRDXRfd/6+vrMWjQIMk6WVlZ4jregyU7Oztx8uRJyX7kjuN5DC10/a1FRUUhOzsblZWV4jKXy4XKykrk5eXJbpOXlydZHwA2b96suD4REVFPEYjr4nnnnYeUlBTJOk1NTfjwww/FdfLy8nDq1CnU1NSI62zZsgUulwu5ubniOlu3bkVHR4fkOMOHD9fc7QAAEHRas2aNYLfbhZdffln47LPPhFmzZgn9+vUT6urqBEEQhFtvvVWYP3++uP6//vUvISIiQnjyySeFvXv3CiUlJUJkZKTw6aefaj5mY2OjAEBobGzU29xehedJG54n33iOtOF50ibcz1MgrotLliwR+vXrJ7z55pvCrl27hMmTJwvnnXeecPr0aXGda665Rhg7dqzw4YcfCtu2bRPOP/98YerUqeL7p06dEpKTk4Vbb71V2L17t7BmzRohNjZWeP7553V9Pt1BQRAE4emnnxZ+9KMfCVFRUcK4ceOEf//73+J7l19+uVBYWChZ/29/+5twwQUXCFFRUcKPf/xjYePGjbqO19bWJpSUlAhtbW1Gmttr8Dxpw/PkG8+RNjxP2vSG82T2ddHlcgkPPPCAkJycLNjtduHqq68W9u/fL1nnxIkTwtSpU4W4uDghPj5emDFjhvD9999L1vnkk0+ESy65RLDb7UJaWpqwZMkS3Z9N9zwKRERE1HvwWQ9ERESkiEGBiIiIFDEoEBERkSIGBSIiIlLEoEBERESKuk1QMPtZ3uFKz3lasWIFLr30UvTv3x/9+/dHfn6+z/MaDvT+W3Jbs2YNLBYLrr/++sA2sJvQe55OnTqFoqIiDBo0CHa7HRdccEGv+P9O73latmwZhg8fjpiYGKSnp2Pu3LmSRwOHo61bt2LSpElITU2FxWJRfeifW1VVFS666CLY7XYMGzYML7/8csDbSQbpvqEyANasWSNERUUJ5eXlwp49e4SZM2cK/fr1E+rr62XX/9e//iXYbDbh8ccfFz777DPh/vvv1z2JU0+k9zzdcsstwvLly4WPP/5Y2Lt3r3DbbbcJCQkJwtdffx3klgeP3nPkdujQISEtLU249NJLhcmTJwensSGk9zw5HA4hJydHuPbaa4Vt27YJhw4dEqqqqoTa2togtzy49J6nV199VbDb7cKrr74qHDp0SHjnnXeEQYMGCXPnzg1yy4Nr06ZNwsKFC4UNGzYIAITXX39ddf2DBw8KsbGxQnFxsfDZZ58JTz/9tGCz2YSKiorgNJh06RZBYdy4cUJRUZH4s9PpFFJTU4XS0lLZ9W+66SZh4sSJkmW5ubnCr3/964C2M9T0nidvnZ2dQt++fYVVq1YFqokhZ+QcdXZ2CuPHjxdefPFFobCwsFcEBb3n6bnnnhOGDBkitLe3B6uJ3YLe81RUVCRcddVVkmXFxcXCxRdfHNB2didagsK9994r/PjHP5YsmzJlilBQUBDAlpFRIe96cD/L2/O521qe5e25PnDmGdtK64cDI+fJW2trKzo6OgL6BLdQMnqOHnroISQlJeH2228PRjNDzsh5euutt5CXl4eioiIkJyfjwgsvxKOPPgqn0xmsZgedkfM0fvx41NTUiN0TBw8exKZNm3DttdcGpc09RW/8Hd6Thfwx04F4lnc4MnKevM2bNw+pqald/gcNF0bO0bZt27By5UrU1tYGoYXdg5HzdPDgQWzZsgXTpk3Dpk2bcODAAdx9993o6OhASUlJMJoddEbO0y233ILjx4/jkksugSAI6OzsxJ133on77rsvGE3uMZR+hzc1NeH06dOIiYkJUctITsgrChQcS5YswZo1a/D6668H/TGs3dX333+PW2+9FStWrEBiYmKom9OtuVwuJCUl4YUXXkB2djamTJmChQsXoqysLNRN61aqqqrw6KOP4tlnn8XOnTuxYcMGbNy4EQ8//HCom0ZkWMgrCoF4lnc4MnKe3J588kksWbIE7733HsaMGRPIZoaU3nP05Zdf4vDhw5g0aZK4zOVyAQAiIiKwf/9+DB06NLCNDgEj/5YGDRqEyMhI2Gw2cdnIkSNRV1eH9vZ2REVFBbTNoWDkPD3wwAO49dZbcccddwAARo8ejZaWFsyaNQsLFy6E1crvZoDy7/D4+HhWE7qhkP+rDcSzvMORkfMEAI8//jgefvhhVFRUICcnJxhNDRm952jEiBH49NNPUVtbK76uu+46XHnllaitrUV6enowmx80Rv4tXXzxxThw4IAYpADg888/x6BBg8IyJADGzlNra2uXMOAOVwKfvyfqjb/De7RQj6YUhMA8yzsc6T1PS5YsEaKiooT169cL3377rfjyfgxpONF7jrz1lrse9J6nI0eOCH379hVmz54t7N+/X3j77beFpKQk4Y9//GOoPkJQ6D1PJSUlQt++fYW//vWvwsGDB4V3331XGDp0qHDTTTeF6iMExffffy98/PHHwscffywAEJYuXSp8/PHHwldffSUIgiDMnz9fuPXWW8X13bdH/uEPfxD27t0rLF++nLdHdmPdIigIgvnP8g5Xes7T4MGDBQBdXiUlJcFveBDp/bfkqbcEBUHQf562b98u5ObmCna7XRgyZIjwyCOPCJ2dnUFudfDpOU8dHR3C4sWLhaFDhwrR0dFCenq6cPfddwvfffdd8BseRO+//77s7xr3uSksLBQuv/zyLttkZWUJUVFRwpAhQ4SXXnop6O0mbSyCwHoYERERyQv5GAUiIiLqvhgUiIiISBGDAhERESliUCAiIiJFDApERESkiEGBiIiIFDEoEBERkSIGBSIiIlLEoEBERESKGBSIiIhIEYMCERERKfr/TTmtzBeBn3QAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgoAAAGyCAYAAACFsOzQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABetUlEQVR4nO3de3hU1b038O/MhEzCJQkQcgEDEVAu5RJMJAYvVYhEpQrv6wWQA5QitgoeNactUDABrYCCNG1BUYRiiwoHj1UP8MZiIg9aUsBgFBCiCBhEE4iYBBJym9nvH3SGuez77D0zSb6f55lH2bMvazYh+zu/tfbaFkEQBBARERGJsIa6AURERBS+GBSIiIhIEoMCERERSWJQICIiIkkMCkRERCSJQYGIiIgkMSgQERGRJAYFIiIiksSgQERERJIYFIiIiEgSgwIRERFJYlAgIiIiSQwKREHUq1cvzJs3z295RkYGJkyYEIIWERHJY1AgCpLvvvsO1dXVGDlypNdyh8OBI0eOYMSIESFqGRGRNAYFoiD5/PPPAcAvKBw7dgyNjY0YPnx4KJpFRCSLQYEoSD7//HNYrVYMGzbMa/lnn30GAAwKRBSWGBSIguSzzz7DwIED0blzZ6/lZWVl6NSpEwYPHhyilhERSWNQIAqSQ4cO+XU7AMCBAwcwaNAgdOrUKQStIiKSx6BAFAROpxPl5eUYMmSI1/KzZ8/i448/9hrI+MILL2DKlCmYPn06YmNjMXToUHz99dfBbjIREQAGBaKgcDgcaGlpQUNDg3tZa2srfvnLX6K1tdVrfMKhQ4ewd+9ezJs3D+fPn8ewYcOwcePGUDSbiAgRoW4AUUfQqVMnjBgxAi+99BKio6MRHR2Nbdu2ITo6GgD8gsKSJUuQmZkJALjmmmsgCEJI2k1ExIoCUZD85S9/weDBg7Fy5Uq8/vrreOSRRzB79mwAV4KCw+HAF198gTvvvNO93ZEjRzjQkYhCxiLwqwpR2CgvL8ctt9yCqqoq97L+/fvj7bffRlpaWugaRkQdFisKRGHE986ICxcu4MyZMxg6dGgIW0VEHRmDAlEY8Q0Khw8fxrXXXovIyMgQtoqIOjJ2PRAREZEkzRWFPXv24O6770bv3r1hsVjwzjvvKG6ze/duXHfddbDb7Rg4cCA2bdqko6lEREQUbJqDQn19PUaOHIm1a9eqWv/kyZOYMGECbrvtNpSVleGJJ57AQw89hPfff19zY4mIiCi4Aup6sFgs+Pvf/45JkyZJrjN//nzs2LEDhw8fdi+bMmUKampqUFhYqPfQREREFASmT7hUUlKC7Oxsr2U5OTl44oknJLdpampCU1OT+89OpxPnz59Hz549YbFYzGoqERGZQBAEXLhwAb1794bVat4Y+sbGRjQ3Nxuyr8jISERFRRmyr7bO9KBQWVmJxMREr2WJiYmoq6vDpUuX3DPTeVq+fDmWLl1qdtOIiCiITp8+jauuusqUfTc2NiK1XxdUnXUasr+kpCScPHmSYQFhOoXzwoULkZub6/5zbW0t+vbti1s6TUKEhU/YIyJqS1qFFuxpeQfdunUz7RjNzc2oOutE+Se90a1bYFWLCxecGJTxHZqbmxkUEISgkJSU5DXLHABUVVUhJiZGtJoAAHa7HXa73W95hKUTgwIRURsVjK7jbt2siAkwKJA3089mVlYWioqKvJbt2rULWVlZZh+aiIiIAqQ5KFy8eBFlZWUoKysDcPn2x7KyMlRUVAC43G0wY8YM9/q/+tWvcOLECfz2t7/FsWPH8OKLL+K///u/8eSTTxrzCYiIiMg0moPCJ598glGjRmHUqFEAgNzcXIwaNQp5eXkAgO+//94dGgDg6quvxo4dO7Br1y6MHDkSL7zwAl599VXk5OQY9BGIiIjILJrHKNx6662Qm3pBbNbFW2+9FZ9++qnWQxEREVGIccQHERERSWJQICIiIkkMCkRERCSJQYGIiIgkMSgQERGRpLCcwpmIiEiPakczmhwBTuHsMOZ5Ee0FKwpEREQkiUGBiIiIJDEoEBERkSQGBSIiIpLEoEBERESSGBSIiIhIEoMCERERSWJQICIiIkkMCkRERCSJQYGIiIgkMSgQERGRJD7rgYiI2o1qpx2NzsC+A1908lkPnlhRICIiIkkMCkRERCSJQYGIiIgkMSgQERGRJAYFIiIiksSgQERERJIYFIiIiEgSgwIREVGA1q5di9TUVERFRSEzMxP79++XXPfWW2+FxWLxe02YMMG9zs9//nO/9++44w6v/Zw/fx7Tpk1DTEwM4uLiMHv2bFy8eFH0mMePH0e3bt0QFxen+bMxKBAREQVg69atyM3NRX5+Pg4ePIiRI0ciJycHZ8+eFV3/7bffxvfff+9+HT58GDabDffff7/XenfccYfXem+++abX+9OmTcORI0ewa9cubN++HXv27MHDDz/sd7yWlhZMnToVN998s67Px6BAREQkoq6uzuvV1NQkut7q1asxZ84czJo1C0OHDsW6devQuXNnbNy4UXT9Hj16ICkpyf3atWsXOnfu7BcU7Ha713rdu3d3v3f06FEUFhbi1VdfRWZmJm666Sb8+c9/xpYtW/Ddd9957Wfx4sUYPHgwHnjgAV3ngUGBiIjajR8cXXDO0TWg1w+OLgCAlJQUxMbGul/Lly/3O15zczNKS0uRnZ3tXma1WpGdnY2SkhJVbd6wYQOmTJmCLl26eC3fvXs3EhISMGjQIDzyyCP44Ycf3O+VlJQgLi4OGRkZ7mXZ2dmwWq3Yt2+fe1lxcTG2bduGtWvXqjuBIvisByIiIhGnT59GTEyM+892u91vnerqajgcDiQmJnotT0xMxLFjxxSPsX//fhw+fBgbNmzwWn7HHXfg//7f/4urr74aX3/9NX73u9/hzjvvRElJCWw2GyorK5GQkOC1TUREBHr06IHKykoAwA8//ICf//zn2Lx5s9fn0IpBgYjCjqVrF+WVgky4WB/qJlCQxcTEBHSBVWPDhg0YPnw4Ro8e7bV8ypQp7v8fPnw4RowYgQEDBmD37t0YN26cqn3PmTMHDz74IG655ZaA2sigQESmCccLvl56PwsDRvsWHx8Pm82Gqqoqr+VVVVVISkqS3ba+vh5btmzB008/rXic/v37Iz4+HsePH8e4ceOQlJTkN1iytbUV58+fdx+3uLgY7733HlatWgUAEAQBTqcTEREReOWVV/CLX/xC1WdkUCAi3dpTEDCLmnPEMNF2RUZGIj09HUVFRZg0aRIAwOl0oqioCPPmzZPddtu2bWhqasJ//Md/KB7n22+/xQ8//IDk5GQAQFZWFmpqalBaWor09HQAl4OB0+lEZmYmgMvjGBwOh3sf7777Lp577jns3bsXffr0Uf0ZGRSISBbDgPmUzjGDRHjLzc3FzJkzkZGRgdGjR6OgoAD19fWYNWsWAGDGjBno06eP32DIDRs2YNKkSejZs6fX8osXL2Lp0qW49957kZSUhK+//hq//e1vMXDgQOTk5AAAhgwZgjvuuANz5szBunXr0NLSgnnz5mHKlCno3bu3ex1Pn3zyCaxWK4YNG6bp8zEoEBEABoJwJvV3wwARHiZPnoxz584hLy8PlZWVSEtLQ2FhoXuAY0VFBaxW75sMy8vL8fHHH+Mf//iH3/5sNhs+//xzvPbaa6ipqUHv3r0xfvx4PPPMM14DKl9//XXMmzcP48aNg9Vqxb333os//elPhn8+iyAIguF7NVhdXR1iY2MxNvJ+RFg6hbo5RG0eQ0H7Fm4BolVoQXHzNtTW1po2ONB1ndj22WB07mYLaF8NFxy4f+QxU9vblrCiQNTOMRR0PGJ/5+EWHqjtYFAgamfaQzCwdO4c6iZ4ERoaQt2EgDE8kF4MCkTtQFsIB+F28ddCS9vbUqhgeCA1GBSI2qBwCwZtOQQYTc25COcwwfBAvhgUiNqIcAgHDATGkDqP4Rog2tJdF1WtsYhuCezSdqm11aDWtA8MCkRhLlQBgaEg+NpLgPBbz9kMNJvcGDINgwJRGAp2OAhpKOgSHbpj61F/KeiHFPv7CdfwQO0PgwJRmAhmOAhaMGhrIUANNZ8pCGHC9++QwYHMwqBAFGLBCAimBoP2GAYCJXdOTAoRDA5kFgYFohAxOyCYEg5CEAqcXezKKxnAWt8UlOOInkMTwoPn3z9DAwWCQYEoyMwMCIaGAxNDQbAu/lpobZOhwcLk8MBqAwWCQYEoSMwKCIaFAwODQTgGAaPJfUZDQoTv34dJwYGhgZQwKBCZzIyAYEg4MCAYBCMQODqb/yA4W0OLofuTOi8BBQiTggOrDaSEQYHIJGEZEAIIB0aGgmBc/LXQ0p5AQoXYOdQdHjz/LlltIBMxKBCZwMiQEKpwEGgwCLcwYBS5z6UnRPieZ13BgaGBTMSgQGSgsAkIOsKB3mBgZCBwRIf+V5Ltkv7pe8XOhdbwEHBwYGggg4X+XyVROxAWASEI4SCQUBAOIUANNe3UEiYCDQ+ef0fhGBqA8AoOP7R2Q1RrYD9rjXzWg5e28S+XKIwZFRKCERCCEQyMDgSt0VZD9+cSccmpe1upz6g2QPieV7XBwZDQYPKcDZ7CKUCQfgwKRDqFNCBorB6oDQhag4HeUGDWxd+MNmgJFGLnQ0148DzvWkNDuFQZxLh+ti1OG/CjqYciEzEoEOkQspBgQvVASzjQGgzMCAStnbXtM6JBf+UAkP8MakKE7zlTCg5aQ0O4Vhmo/WBQINLIiJBgVkAwOhyoDQZ6A4HWi36wjqE2XIh9bqXw4HlOzQoNDAxkJAYFIg0CDQmhDAhqwoEZwcCMMNAaZVFcJ6JR0L9/mTYrhQjfcyMXHPSEBgYGCjYGBSKVghoSghgQ1IQDtcFATyhQc9HXQ89+1YQLsc8oFx48z50RoUFLlYGBgYzAoECkIByrCEoBIdBwoCYYqA0FRgYBrfvSWlWQ27/cvnzPhVRw0BoajKoyMDBQIBgUiGSEWxXBzICgFA7UBAOtF3Kzqgl69y8bBkT2JbW+57kKJDRorTIwMJAZGBSIJAQtJJgcEAKpHiiFAzUX4kDDgCMqoM0BALZGdetJtVUyEPisL7ae0aFBKTCoHcOge6pohoUOh0GBSEQgIcHIKoJZAUFvOFC66GsJBUYEACOPJRcm1AQC3/UCDQ1ygYHVBQomXcOR165di9TUVERFRSEzMxP79++XXb+goACDBg1CdHQ0UlJS8OSTT6KxUWXEJwqyoISELtEBhQRH506SIcERHSEZElqjraIhobWz1f3yey/K4n7JvSf2viNK+iWlNcq8lxwt7VT63ErnDYDk+Xa/L/F3Bcj/HbvXUXkLrO6Hf6n4Gab2QXNFYevWrcjNzcW6deuQmZmJgoIC5OTkoLy8HAkJCX7rv/HGG1iwYAE2btyIMWPG4Msvv8TPf/5zWCwWrF692pAPQWSUoIUEGXqrCHLhQIrUhUru4ifbNhXf3JUu2GZSOnaEyPcX388kVnlQrCTIvK9UZZCrMCh1R2ipLgT8uOswqTD80NIF9pbAHlTW1KL/UeLtkeagsHr1asyZMwezZs0CAKxbtw47duzAxo0bsWDBAr/19+7dixtvvBEPPvggACA1NRVTp07Fvn37Amw6kbHCPSQYFRCMDAdKwUBtKAhmN4R8F4P/Mt/woBQc1IYGqa4JuS6JQLojTOuKcAmzwEDG0RQUmpubUVpaioULF7qXWa1WZGdno6SkRHSbMWPGYPPmzdi/fz9Gjx6NEydOYOfOnZg+fbrkcZqamtDUdOWHta6uTksziYLKiJAQbgFBKhwodRnIMSIMOBSq5DYV1zi5dohXC7z/LBccpEKDlsDg+nsyo7pg6kBHlyA+S4KCQ1NQqK6uhsPhQGJiotfyxMREHDt2THSbBx98ENXV1bjpppsgCAJaW1vxq1/9Cr/73e8kj7N8+XIsXbpUS9OIAqK3mqAqJASpiqAlIGipHkj304svl9vGbz2d3eNG7E8sVKjrZrjy/1KhQUuVQW9g0FNdCFpYcHH97DtC/xAw0s/0v73du3dj2bJlePHFF3Hw4EG8/fbb2LFjB5555hnJbRYuXIja2lr36/Tp02Y3kzqwcAwJUoMVpQaxyQ1Q9FomMrhOatCd2GA+qYGBSgMAHXbxlxhntNOwlxw17VH6XGrOh/822rp4JCtBMmNP5AY6mj7IkdodTRWF+Ph42Gw2VFVVeS2vqqpCUlKS6DZPPfUUpk+fjoceeggAMHz4cNTX1+Phhx/GokWLYLX6/7Db7XbY7fwhJfOFa0gQXa4hIPgtU9m9IHUxFG2P1HKFf7pKF3AjKR3Lesn7XPm23bfqIN/NcPm/aqoMklUEmepCm64sUJumqaIQGRmJ9PR0FBUVuZc5nU4UFRUhKytLdJuGhga/MGCz2QAAgqD/oS1EgQpVSHB2sWsKCWqrCForCF7HkKkeiK3nu67Ut3K13/KddsHUlxSltqmpNviSuhVTqrqgNsRJ3r5qcmWBSPNdD7m5uZg5cyYyMjIwevRoFBQUoL6+3n0XxIwZM9CnTx8sX74cAHD33Xdj9erVGDVqFDIzM3H8+HE89dRTuPvuu92BgaitMCIkiDG6iqC3gqD2AifXbSBF7oItK8qhbr1G6d8nUse2NnmfE9/2e1YcPD+zZ6VBqsogVmGQG8MgVl1Qe2eEmZUFVhVI8xiFyZMnY9WqVcjLy0NaWhrKyspQWFjoHuBYUVGB77//3r3+4sWL8V//9V9YvHgxhg4ditmzZyMnJwcvv/yycZ+CSCM91YRwDAlSVQTfPytVEOSqB+4/q6gaeC1X+lYf5VD3UkvHvpTaKPXZtFQZ1Jzby+upv/tEa2VBjprKAscrKNM6EWFNTQ3mzp2L5ORk2O12XHvttdi5c6fouitWrIDFYsETTzzhtfyXv/wlBgwYgOjoaPTq1QsTJ070urHgs88+w9SpU5GSkoLo6GgMGTIEf/zjHzV/Nl1TOM+bNw/z5s0TfW/37t3eB4iIQH5+PvLz8/Ucishw4R4SzKwiKFUQ1FYPxCoHshUDFRd8m11DKFDB0SRSYRBrh0clwvMzSFUbxKoMYmMZxMYwiI1fUBq7oGXcglRlQWmeBTVYWZCmdSLC5uZm3H777UhISMBbb72FPn364JtvvkFcXJzfugcOHMDLL7+MESNG+L2Xnp6OadOmoW/fvjh//jyWLFmC8ePH4+TJk7DZbCgtLUVCQgI2b96MlJQU7N27Fw8//DBsNpvkNVyMRWgDAwXq6uoQGxuLsZH3I8LCfjUKjClBIcxCQqABIeBwIBMMjA4EWokGCBeJ7gvf0AD4D4QExG+59A0MYrM/it2G6RsMxLohpCZnkuqGkAsLagY36g0KrY4mfPDVH1BbW4uYmBhd+1Diuk7858cTYe8a4MyMF1vwp5veVd3ezMxMXH/99VizZg2Ay2P3UlJS8Nhjj4lORLhu3TqsXLkSx44dQ6dO0m29ePEirrvuOrz44ov4/e9/j7S0NBQUFEiu//nnn2PkyJE4fvw4BgwYILrO3LlzcfToURQXFyt+Lhfe3EodimnVBAlGhwTfrgbfbgXfPyt1MUh1L3h9Bp/Su2ipXqLEb7M7/F6e7FHNpr7EyLVHqptCrGtCbZeE2q4IX0rhD5C+ddIsHa0Loq6uzuvlORGgi2siwuzsbPcypYkI33vvPWRlZWHu3LlITEzEsGHDsGzZMjgc3j93c+fOxYQJE7z2LaW+vh5/+ctfcPXVVyMlJUVyvdraWvTo0UNxf5749EgiGYF0OZgRErz+bGAVQSwc+PKrHohUDeSqBVIXbjFd7OrXBYD6pkjVx2xq9F7Xs81e1QbX5xPpmvCsMEh1SYgNePQd7Og70FHpcdiigx4N6oJQe8tkuPuxpTMiW8R/HtRqbrn8c+N7wc3Pz8eSJUu8lumZiPDEiRMoLi7GtGnTsHPnThw/fhyPPvooWlpa3N30W7ZswcGDB3HgwAHZtr744ov47W9/i/r6egwaNAi7du1CZKT459+7dy+2bt2KHTt2yO7TF4MCdRiBPMtBksan5wUjJATSzSA2INGPT0CQCgdywUBrEFAitz/fEOHbLs/g4PosegOD0vgF/zEJ8mFBy90QvuTuhCB1Tp8+7dX1YNT8Pk6nEwkJCXjllVdgs9mQnp6OM2fOYOXKlcjPz8fp06fx+OOPY9euXYiKkp/mdNq0abj99tvx/fffY9WqVXjggQfwz3/+02+7w4cPY+LEicjPz8f48eM1tZdBgTqEcBiXYEZI0FJFUBqH4Nu94EVF9UAqGKgJBV3txgySu9jkf959jy8VHMQCA+ARGlQEBrXVhUDDgi+5h0n5CnRgY0ca1BgTE6M4RkHPRITJycno1KmT1xQBQ4YMQWVlpbsr4+zZs7juuuvc7zscDuzZswdr1qxBU1OTe9vY2FjExsbimmuuwQ033IDu3bvj73//O6ZOnere9osvvsC4cePw8MMPY/HixZrPA4MCkYi2HhL0djMoBQSx6oFYQJAKB0YFAilS+/cMEJ5t8wwNnp/DNzSoqTAoVRfkwoIS37CgtqqgVXvpfggmz4kIJ02aBODKRIRSdxbceOONeOONN+B0Ot0TEn755ZdITk5GZGQkxo0bh0OHDnltM2vWLAwePBjz58+XnINIEAQIguA1luLIkSMYO3YsZs6ciWeffVbXZ2RQoHbPlC4HCWoHewUrJOjuZlAICGrDgVIwiLNruFqqVNPkX6r1bIeW0OAKDJJdEgGEBU96xiv47UPjRExkHK0TET7yyCNYs2YNHn/8cTz22GP46quvsGzZMvznf/4nAKBbt24YNmyY1zG6dOmCnj17upefOHECW7duxfjx49GrVy98++23WLFiBaKjo3HXXXcBuNzdMHbsWOTk5CA3NxeVlZUALs+O3KtXL9Wfj0GByEcg1QQxaia0MTsk6K0iKAUELeFASyjoaa+Xff+HJunwJ3Ycz/CgFBp8A4NidUFnWAi0CyKQqoIR8yrQFZMnT8a5c+eQl5eHyspKpKWl+U1E6Pkog5SUFLz//vt48sknMWLECPTp0wePP/445s+fr/qYUVFR+Oijj1BQUIAff/wRiYmJuOWWW7B371733A1vvfUWzp07h82bN2Pz5s3ubfv164dTp06pPhbnUaB2LZhjE/R2OegNCWrHI+ipImgNCGLhQC4YKAWBQMiFCEC84iA2tsF3LIPv3RJegcFn/gXfeRc8w4JvVcEzDPjOseA/B4P2uRW0zqlgxnwKwZxHYfqHUxHZNcC7Hi4242+3vWlqe9sSVhSIPHSUkKC2m0FrQJAKB2qDQY9O6gPE+RbxQOB7LN/g4GqjWJXBt8KgurqgUFnwZGQXhKqBjux+oAAxKBD9m9FdDr7CPSQYHRCkwoGWMCBHaj++AcKzHZ6hQSowmBEWtHRByDFrECORHAYFareCNYhRbTVBjpYxCd7reRzThJCgNSAEEg7iO11UXMdTdUtXxWNJhQa5wOBbXfAdu6AlLHjyDQtSxJ4JQRRKDApEapjc5eD1noEDF10hIdAqgp6AIBUOtAYCKWL78Q0PUqFBKjCorS74hgUvHmFBrgvCk1xVQc8dEERGYlCgdklrNSGQ5zko0dLlIFc9CFZICCQgiIUDtcEgPuKCqvUAoLq1m/g+PI4lFRp8A4MRYcHvbggPUl0QWsYqeL8nf/eDlsmXiNRgUCBSEmA1QY7cQ32kxiWEIiQYGRC0BAIpYvvwDQ+u44sFBrnqglJY8CQZFmS6INq6cJ+V8XxzF3SSePaHWi3NvLvOE4MCdXihrCZ4vadi8GKoQ4JSQAg0HCR1qpF8r7IlTnZb1zHUBAal6oJcWPAd4BgIvYMaiYKJQYHaHUMHMQapmiDX5SBG7TwJAEwJCWoCglw4kAsEWrfxDRCex/UMDVKBQW1Y8KSqC0JirILaQY2eGCIolBgUqEMLl2qC13oqqgnu95XubtAQEtR2NSiFBKmAoCccqOHar1jFIT7igmiFQS4sePIMC3JdEETtGYMCkQGUqglqBzBq7XJwkZ2SGfpDglxXg5qAoBQOetm03wFxziF+W6RUYBDrkpALC74DHOmKcB+fQOZgUCCSoqHbwZfYQ5+MIPf8BkB8XIIRIUGuiqAlIOgJBkr78A0OcoFBLix4kuqCkBqr0BEHNVLHoa2jjCjMBfNJkXoFUk1wv69hXIKL3ERKgLqQEN/pomxISOpUIxoSetkuGhISxEjtW6wdcuMmjJoxUopYpSfc8BHTJIYVBeqw9IxPUDOIUe/YBCWquxxkHu4EiA9cVBsSPImFBF9K4aCXVVsp+5xTuprTy3ZRtLogd7dEoFWFcKRmDgU9T45kt0PHxaBAJCbA5zpIMbqa4KJ2XILcZEqAvpCgJSBoDQZy24uFBtdxPQODb1iQ64KQG9goRu+tknrufCAKFf6kEpkoWNUEN4UuBxe5OxwA40NCL2tTwCEhkH36ttOISZ98iZ3nYFB6SFTInxxZfwlouBTaNlBAGBSo3dAyPiFY3Q6e5GZhdG9vQjXBRcu4BClyIUFsrIDWgNDLFoleNm3f0MX2H8h4CM+QpPbx2GYLxhwKcuMTdHU71F+6/KI2j10PRCGg9umQgLnVBBc1XQ5KIcGXUkCQCwRi751zSH9j72Vt8uuK8B2zIDdeQW6sgovcBEzkgwGhXWFQIPJl0PiEQLodxKoJYoyqJrgEIyRorRh4bqc1LMgRm4wpWAIZn6DU1aBE60BGzdWEEIeE2iY7IiICG2zaquKJnx0Jux6IdNI6ZbPi/hTCgVi3g5tB1QSttIQEPd0Kge7DrFsyw5XvHQ9qxycYdlskKwntEoMCkUHUjk8wpdvh34JdTfAlFxKMpHd/Zk0jrYbVhG+pgVYXlGiqJjAktFsMCtQuhGIgo1kC6XaQY3Y1IVghQY7Rd1Zo4ZqZMZzpmT9BFYaEdo1BgcgERt0W6WJEt4OLWdUEMcEMCWYzeyCj550NEY3iy005rkS3g+pqAkNCu8egQOTJpImWlCiNT3CR63YQI9btYAQ11QSzQ0KbCCEKz3mwaSyAyHU16B2fEBCGhA6BQYFIBy0DGaXmT9AzPkGO2PgEFz3dDmZMStTWKD1FUs+sjEbNyKh1fIJYtwOf7UBqMCgQGUDt0yKVwoFaZj1gyHcWRpdQDgKUInerpBS55z64aJnCWYz7yZEezBjIaCZV3Q6sJnQYDApE7UyophJuazznUFCabMmT2AOhAh3IqGZ8Qlh1OzAkdCgMCkRhRO20zXqJDWRsT3wnXPJ9mqReugYyGjw+wZOZ3Q6K1QSGhA6HQYFIQTBvjQwFzzse2gqzuh086RmfINbt4CkYT4wM+UOgqN1hUCCidkHL9M1S3Q5K4xP0dDsojU8wutvBb/+sJlCA+KwHog7ufEsXU6oK5xzNptzCqLaaYGa3g+LdDiLdDp7VhGB2OximjYSE+mY7bAE+60FHwapdY0WBiNoMqZCgNDbBs9tBTTVBrNtBrJrgKdC7HfRMsqQ0iNHQagJ1WAwKRB2I3tkFlfr3pcr+esYSSB5DZUjwpXVsgiel8yXa7aBhEKOaQOBZQWA1gUKBQYGI3Dy/Ycs9gllLWT/QsHDO0awpJMi1zYhqQrgMYtRzS2SoqglCA4NGW8YxCkTtTFNjpKq5FH5o6hLQQ6F8nXPaJR/K5HmhVzNuQU24UBMSpLocpHiGhECrCWLdDlLVBDWDGI24JVIXndUEoaHBmONTyDEoEIU5W1PgcylcbLKLTuOspLq1m3sq58qWOK8ZGs85uvo980EuLFzZLvDuCK0hwZfeOx3CpZrgy7Rqgo6QwIDQ/rDrgSiMRKgcxCb2bVXpgiXH82IpN0uh78VXrMx/zmnXdKuiFlL7VgoJWrsclO50CHY1wZeeWyKDgSGhfWJQIFLQlkeDi30D9rwIKk0qBCiX7KXGBBgZFuQCgpkhQWneBKU7HYLxAKhwqSYwJLRfDApEYUpsRLzohUdhlL2L0u19eqsKgHxY0FthUNpW7JhqQ4IaSl0OXnTe6RDKaoKRAZghAVi7di1SU1MRFRWFzMxM7N+/X3LdTZs2wWKxeL2ioryrWEuWLMHgwYPRpUsXdO/eHdnZ2di3b5/fvnbs2IHMzExER0eje/fumDRpktf7FRUVmDBhAjp37oyEhAT85je/QWurtooTxygQmSyiwel+1HREo+B+gqTn/xtNaUBjTVOU4nMfqlu6up8m6TlWAfAfrwBcuXD7jltwv29QhUFNQADkQ4IRXQ5tvZogS0M1gSEB2Lp1K3Jzc7Fu3TpkZmaioKAAOTk5KC8vR0JCgug2MTExKC8vd//ZYvH+XXDttddizZo16N+/Py5duoQ//OEPGD9+PI4fP45evXoBAP7nf/4Hc+bMwbJlyzB27Fi0trbi8OHD7n04HA5MmDABSUlJ2Lt3L77//nvMmDEDnTp1wrJly1R/PosgCCG6MVe9uro6xMbGYmzk/YiwdAp1cygMWbqqfzSwpXNn6Te7RIsuFnveg6Oz98+i56OmW6O9LxKuoAB4P2ra8/8dUa5l8FsGXBnQ6Iy+cnFwP246yuFeZrNf/n/PoNDFfvn/PQc0egYFz7sfPGdp9H3stGdYAOQfPy0VGPSSqlrIVREAfSFBqprgCgpeIUFkbILULIx6qgledz4YNMGSUV0OakNCq7MZRT++htraWsTExKjevxau68Sot3Jh6xzgzIwNTfj0vtWq25uZmYnrr78ea9asAQA4nU6kpKTgsccew4IFC/zW37RpE5544gnU1NSobpPr833wwQcYN24cWltbkZqaiqVLl2L27Nmi2/y///f/8LOf/QzfffcdEhMTAQDr1q3D/Pnzce7cOURGqps5lV0PRGFG7YBGLyKlb7EBd54XQKmxCnJdEL4X4cqWOMm7C8TGD2jl2odUFSEUIcGLwgBGT2pCgq+QVBNU6giVhLq6Oq9XU5N/wGpubkZpaSmys7Pdy6xWK7Kzs1FSUiK574sXL6Jfv35ISUnBxIkTceTIEcl1m5ub8corryA2NhYjR44EABw8eBBnzpyB1WrFqFGjkJycjDvvvNOrolBSUoLhw4e7QwIA5OTkoK6uTvZ4vtj1QOSp/pJkVUGLiEtOv6qCHrZG76oCcPnbqmdVwZejyeauKrjUN0W6qwpqbpX0fP6DZxcE4N8NAVz5Vi9WYTDqmQu+x/IVrJCgpctB6zMd5OZNCMrYBJXVhHAOCfVNkbAF+IwRR9Plc5+SkuK1PD8/H0uWLPFaVl1dDYfD4XUxBoDExEQcO3ZMdP+DBg3Cxo0bMWLECNTW1mLVqlUYM2YMjhw5gquuusq93vbt2zFlyhQ0NDQgOTkZu3btQnx8PADgxIkTAC6PZVi9ejVSU1Pxwgsv4NZbb8WXX36JHj16oLKyUrRdAFBZWan6fDAoEAWZ3nEKYvMpWJssV7ofRGgZq+A7AZNSWAD8uyLkAkOgjAgIgIEhQcODn/RUEzwZOQsjqXf69Gmvrge73ZhxNllZWcjKynL/ecyYMRgyZAhefvllPPPMM+7lt912G8rKylBdXY3169fjgQcewL59+5CQkACn8/LPwKJFi3DvvfcCAP7yl7/gqquuwrZt2/DLX/7SkLYC7HqgdkK4qH6GwVB8G1L6NghIX0zEKN39IPat1/Pip6YLAvDvhhDrihC7fdLVLRDIcxY89yE1WDEcQoKWBz/JCcYsjO29mmC0mJgYr5dYUIiPj4fNZkNVVZXX8qqqKiQlJak6TqdOnTBq1CgcP37ca3mXLl0wcOBA3HDDDdiwYQMiIiKwYcMGAEBycjIAYOjQoe717XY7+vfvj4qKCgBAUlKSaLtc76nFoECkgppbycya5EbpW6fShcrzYhdoWADEbzOUCgyA/wVf7UuM6zhiASHUIcHIAYyejHxCpCSGBN0iIyORnp6OoqIi9zKn04mioiKvqoEch8OBQ4cOuS/+UpxOp3ucRHp6Oux2u9edEy0tLTh16hT69esH4HLl4tChQzh79qx7nV27diEmJsYrYChh1wORTraGFr87HzypHaegt/tBdKxCo819B4TnWAXPLgip8QpquyGAK2HB964Izwu4b7eEXnITPomFFrmAAKgLCZJMCAm+zHpCZKDzJjAkSMvNzcXMmTORkZGB0aNHo6CgAPX19Zg1axYAYMaMGejTpw+WL18OAHj66adxww03YODAgaipqcHKlSvxzTff4KGHHgIA1NfX49lnn8U999yD5ORkVFdXY+3atThz5gzuv/9+AJerHb/61a+Qn5+PlJQU9OvXDytXrgQA9zrjx4/H0KFDMX36dDz//POorKzE4sWLMXfuXE3dKAwKRCEgFQ48By9GNF65VVJsUKMnr7EKHmFBK9+wAFy5ddJ1AVYTGADxC7xSeFDz4CbP43oSe2aD3pCgNHhRitbBi4C2AYyhrCaQtMmTJ+PcuXPIy8tDZWUl0tLSUFhY6B44WFFRAav1Spj88ccfMWfOHFRWVqJ79+5IT0/H3r173d/ybTYbjh07htdeew3V1dXo2bMnrr/+enz00Uf4yU9+4t7PypUrERERgenTp+PSpUvIzMxEcXExunfv7t7P9u3b8cgjjyArKwtdunTBzJkz8fTTT2v6fJxHgdoNw+ZSAETvfNA6lwIQ2HwKl5dDdLnnoEa18yoA4nMruEjNsQBA9CmTnoHBk1hoMILUzIpaAwKg/hbIYHc5aJkzAfAPCmZN1RxoNSGY8yhc+/oCQ+ZR+HLaClPb25awokCkkrW+yS8s+HY/2C61eoUFue4Hqbsf1FQVFJ8oqbELApDuhgD8qwuAeIUB8L+g6w0OSlMuqwkIgPyzG0IZEnyF6wBGdjkQgwK1G8LFetVVBaGhQbmqYALP6ZyN4jlWQaoLQm9YAOAXGHyrC1KBwUXrMxbkSD0SWikgAKEPCb4CGcAohrdDklkYFIgM5ltVUCuQqkKgYQGA5IRMUtUFQLzC4CIVHLSQCgZibXHREhAAhfEIAYQEX2q7HNRgNYGCiUGBSIzEDI1quh98+XY/SD0kykhawwLgfzcEANnqAiDeJeGidJHXS+rR2GIPdApmSPClt8shXAYwMiSQC4MCdVih6n6QY1RVwY+OsABIBwbAv0vCk1hw0EsqGLgEGhAA+RkX1YYEteMSArnLQQsjHyNNHRuDApEJlAY1qq0qGNYFAciGBQCSXRGAf2AApKsMgPLF3RUklNaTIhYOPNvpKZQhwVegcyQE6+mQbbma0NzYCVZrYM96cDbqD2jtka5RVWvXrkVqaiqioqKQmZmJ/fv3y65fU1ODuXPnIjk5GXa7Hddeey127typq8FEcrRM5axI4heq2C/hQAeSqZ2+V+qbqueFy/OC5jdro8w0z74X0PqmSL+L7MUmu9/FuKYpyuulxg9NXTSFBLljuNokVkXwHbDo29VgdkjQMi4hKF0OKhgdEoSL9RDq227wIB0Vha1btyI3Nxfr1q1DZmYmCgoKkJOTg/LyciQkJPit39zcjNtvvx0JCQl466230KdPH3zzzTeIi4szov1EYUtLVcGXVBeE3zFUVhYASI5ZACBZXQDkKwwA/J5EKRcWxKoParbzJVY5cFGqIADqxyMA4REStDDieQ5GMDS0U0hpDgqrV6/GnDlz3FNTrlu3Djt27MDGjRuxYMECv/U3btyI8+fPY+/evejU6fKAr9TU1MBaTWQQxXEKBg5qVKKnC8JvPZkxC3KzN/o+mlptYADkQ4MvLWHAl5ZwAGgPCIA5IcGX1sdHA21rACMDQvujqeuhubkZpaWlyM7OvrIDqxXZ2dkoKSkR3ea9995DVlYW5s6di8TERAwbNgzLli2DwyE9xWxTUxPq6uq8XkRqhdMvKt9f8L7fFP2+Scp0QagdLCfVDQH4XAgbbX5dEWLdEVJdEmIXZ89uAKkuATlS20uNPRBrh1ibxT6bUleDUSFB67iEQLocQj2AMZz+7ZFxNFUUqqur4XA43PNXuyQmJuLYsWOi25w4cQLFxcWYNm0adu7ciePHj+PRRx9FS0sL8vPzRbdZvnw5li5dqqVpRLoZefeDmqqC3i4I//eknwXhW1kAID/IEZDsjgDEKwyA/zd532qDi5awIEXpoU2qKgiApioCYGxICOcuh0CrCQwJ7Zfpj5l2Op1ISEjAK6+8gvT0dEyePBmLFi3CunXrJLdZuHAhamtr3a/Tp0+b3UwiaRoGNYrR+vhpuYuL/4UIku/5XuB8BznKDXQE5CsMYhdlwPtbvlTVQYnYPqT2I9UeyQpCGwsJbaXLgSGhfdNUUYiPj4fNZkNVVZXX8qqqKiQlJYluk5ycjE6dOsFmu/IPdMiQIaisrERzczMiI/1/Adjtdk2PwCTypWU6Z6OJVRW0Dmz0rSTIDW70rSwA0s+EEBu3APgMdAT8xi+4P4dIlcHFt9rgoicsSJEKKIDMUx4b/ZcHEhAA40OCmLbS5cCQ0P5pqihERkYiPT0dRUVF7mVOpxNFRUXIysoS3ebGG2/E8ePH4XRe+Yfx5ZdfIjk5WTQkEIWC4jeqAKsKYrSMV/Cl5cJla/IftyA7dgEQ/fYNXPmmLnZR9vx2r1R5UKJ2X3LtkaoghGNIUDMuQUoouxwYEjoGzV0Pubm5WL9+PV577TUcPXoUjzzyCOrr6913QcyYMQMLFy50r//II4/g/PnzePzxx/Hll19ix44dWLZsGebOnWvcpyAKIbXzKqj55a93cOPl9yH7vlhXhKruCB2hwZPURV/uJUdVOFAZEHwHLIZDSJASbg99YkjoODTfHjl58mScO3cOeXl5qKysRFpaGgoLC90DHCsqKmC1XvnHl5KSgvfffx9PPvkkRowYgT59+uDxxx/H/PnzjfsURCK0dj/ovVVSip4uCCW+XRJy3RBi77suhGq6IwCID3oEvLomAOmyv2dXhVZKAcRNJMS4+AUf+FcQAOWAAAQeEsQEOi4hVNUEhoSORdcUzvPmzcO8efNE39u9e7ffsqysLPzrX//ScyiiNkFsXgW19IxXAOA1ZgHwnmfh8vsQfR8QH7sAwO9ZEX5jGFx8L85R4oFA9cVeK43hAFAXEADlKoL4OsohwejBi6G+y4E6Dj7rgchDKKoKgPawILZMTXUB8B7oCIgHBgDqqgwuYhduifCgiUwg8CQVDgD9AQFQN5GSmSEhHIV7NcHRbINgCyygOptNCrhtFIMCtWvBvPtBqqoQ6rAAaO+OAJSrDC6iwQFQfZHXS2s4AMQDAqCvinB5PWNCgpRwqyaEe0ggczAoEPkIpKoQjLAAQPLWSUC8q0GpugB4X0TVVBnc74tcsCXDgw5ygcBrPYlwAKgPCIC+rgbJZSpugwTaRpcDQ0LHxaBA7Z4pVQWNXRBSxMKCL7GZG+XmWXDvW2V1AfB/6JRSlcFFLDgA6i/ugZILB4DxAeHyevpDgmmTKhGZiEGBSEQg0zprqSqIEbsTQm1YAKCqunB5PciuB0hXGVzELtRS4SFQSqHARSocAOoDgvS65ocEOawmUCgwKFCHEOyqghFdEAD8uiEAaB63APhXFy6vd/m/YoEBkA8NgHhwANRf0I0iFwwA6Sc5mhEQgMBDArscKNwE9180URui6herzC9oqV/sWiZjErvAiE3KJDaoTmyCJqlv1FJld6ltgCsTFIlNVGQmNceVa7vS5/VfX30VISQhgchkrChQh6GnqhDokyUDrSwA+rsiXMsA5e6Iy+te+f9W32qCTKXBvY6Ka5lUFULLPiS3lQg0LloqCJfXl6gYaBi0aERIUMRqApmMQYEoUDoHNkqFBQCKd0MA0mEBgOruCED8wi/WLeG7nSep8OC3rYFfjJWCASAdDuS21xIQAG3TMusJCYF2OejFkEAuDArUoZhWVdAxXgGQHuCo5tZJQHzcAqC+ugDIVwt8L7RiwcF3H57UBggpasKAL7lwoLTP9hgSOAMjBYpBgUiFcAoLAAKuLrhoCQ2Xt/X+s1RwENufWZSCgZp2GBEQgCCGBJXY5UBGYFCgDsfU2RqDEBYA7dUFwD8UKL2nZlyC2EVaKTwEQk0ocNEbDhTfC4eQYGKXA5EvBgUilQId2AgYHxYA8eoCYFxgANSFhiv7kn/fTOrGLQQnIAChDQkdtprQaAMsAU4fbvL0420NgwJ1SHqrCoF2QQD6wwLgP8gREK8uAMqBAdDWLeFuSwCDGY2kpVtD6ZHPegICwJBAHQODAnVYbS0sANqrC4B0YADUVRlcpIIDoHzR1hskAhnjoBQO1KyjJyTIzbbIqZmpLWJQIDJLgGEBgK7qAqA/MADSgUBLcPAVnEGN6h9EZUZAAAILCawmULhiUKAOzdSqAqAqLAAwtLoASHdHAPKBAVAXGnzXU7O+kbSEAi3b6A0IAEMCtV8MCtThhTosAIF1RQDaqwuA90Ux0NAgtn6oqW2LmkdB660iAOEfEoiUMCgQBSCYYQEQ74oA1AUGQDk0SAUGILQVBDW0hpRAAwIQPiEhEKwmkBIGBSIENrdCsMICEFhgALRVGQD54ABIX5zNDhB6KxdqwgFgfkAAjA0JrCaQmfj0SKJ/C+Sblepf1PWXFC8A1vomxYuI0oXIdqlV9mIWccnpfslxPRFR7QXWvd2/n15p1ktTWzR8BjXnxIgqgpFPgwwkJLCaYJy1a9ciNTUVUVFRyMzMxP79+1Vtt2XLFlgsFkyaNMm9rKWlBfPnz8fw4cPRpUsX9O7dGzNmzMB3333nte2XX36JiRMnIj4+HjExMbjpppvw4Ycfeq1z4MABjBs3DnFxcejevTtycnLw2WefafpsDApEHoISFgBV3xbVhIVAAwOg7uIIeF9w9YSHYNDTRrWhSc25DLirwSUI4xIYEoyzdetW5ObmIj8/HwcPHsTIkSORk5ODs2fPym536tQp/PrXv8bNN9/stbyhoQEHDx7EU089hYMHD+Ltt99GeXk57rnnHq/1fvazn6G1tRXFxcUoLS3FyJEj8bOf/QyVlZUAgIsXL+KOO+5A3759sW/fPnz88cfo1q0bcnJy0NKi/lZdiyAI4TP6SEJdXR1iY2MxNvJ+RFjES65ERgpkimdNszeqfOqkXHeEi1R3hNc6El0SYqS6J1Rtq9BlYYRAgoqaYOSiFA4Ag7oaXILU5RDMoNAqtKC4eRtqa2sRExNjyjFc14mUPzwDa3RgM4A5LzXi9JNPqW5vZmYmrr/+eqxZs+by9k4nUlJS8Nhjj2HBggWi2zgcDtxyyy34xS9+gY8++gg1NTV45513JI9x4MABjB49Gt988w369u2L6upq9OrVC3v27HEHjQsXLiAmJga7du1CdnY2PvnkE1x//fWoqKhASkoKAODQoUMYMWIEvvrqKwwcOFDV+WBFgchgRlcWAHUXGS0VBjUXP7XftEW3Fflmb/RLc5s0fh61FQSGhParrq7O69XU5P/32NzcjNLSUmRnZ7uXWa1WZGdno6SkRHLfTz/9NBISEjB79mxVbamtrYXFYkFcXBwAoGfPnhg0aBD++te/or6+Hq2trXj55ZeRkJCA9PR0AMCgQYPQs2dPbNiwAc3Nzbh06RI2bNiAIUOGIDU1VfV54GBGIhGBPjhK03MhXBcHFQMdAeXqgtKAR/d6CgMfPYldXAOpOASDnoCjJkC511U5yyJDQnBZmy2wWgMcTNt8eXvXt3CX/Px8LFmyxGtZdXU1HA4HEhMTvZYnJibi2LFjorv/+OOPsWHDBpSVlalqTmNjI+bPn4+pU6e6KxwWiwUffPABJk2ahG7dusFqtSIhIQGFhYXo3r07AKBbt27YvXs3Jk2ahGeeeQYAcM011+D9999HRIT6yz+DApGEoIYFQNVdEYB5gQHQ1jURTuFBTyjwFNKAAPAOhzB1+vRpr64Hu125C1DJhQsXMH36dKxfvx7x8fGK67e0tOCBBx6AIAh46aWX3MsFQcDcuXORkJCAjz76CNHR0Xj11Vdx991348CBA0hOTsalS5cwe/Zs3HjjjXjzzTfhcDiwatUqTJgwAQcOHEB0tLquTwYFIhkhCQtASAID4H/B1BIcAO0XbLlgEejFX4mWcABoe05DuIaE9l5NMFpMTIziGIX4+HjYbDZUVVV5La+qqkJSUpLf+l9//TVOnTqFu+++273M6bz8sx4REYHy8nIMGDAAwJWQ8M0336C4uNirLcXFxdi+fTt+/PFH9/IXX3wRu3btwmuvvYYFCxbgjTfewKlTp1BSUgKr9fK/tTfeeAPdu3fHu+++iylTpqg6DwwKRAqMCAuAxkGOKqsLgPLcCy6eFzo1oQEIPDgoMTsMeNIaDNzbhUFAABgSwlVkZCTS09NRVFTkvsXR6XSiqKgI8+bN81t/8ODBOHTokNeyxYsX48KFC/jjH//o7u5whYSvvvoKH374IXr27Om1TcO/fx5cAcDFarW6g0dDQwOsVissFovX+xaLxb2OGgwKRCoEGhaA4FQXAHV3SOgJDYD4xdbo8GAUvcHAvb3GJz2aGRIovOXm5mLmzJnIyMjA6NGjUVBQgPr6esyaNQsAMGPGDPTp0wfLly9HVFQUhg0b5rW9a4Cia3lLSwvuu+8+HDx4ENu3b4fD4XDf8tijRw9ERkYiKysL3bt3x8yZM5GXl4fo6GisX78eJ0+exIQJEwAAt99+O37zm99g7ty5eOyxx+B0OrFixQpERETgtttuU/35wvNfOFEYCklYADRVFwD1XRIuekODe3uZC3IwQkSggcBrXzoeA6158iQdIYHVhPA2efJknDt3Dnl5eaisrERaWhoKCwvdAxwrKir8vvnLOXPmDN577z0AQFpamtd7H374IW699VbEx8ejsLAQixYtwtixY9HS0oKf/OQnePfddzFy5EgAl6sX//u//4ulS5ciKysLVqsVo0aNQmFhIZKTk1W3h/MoEGkUaFhw70drYAA0BQYXtYFBjJ7g0NboCQeAjoAAdNiQEMx5FPo993tYowKcR6GxEd/MX2xqe9sSVhSINDKisgAEUF0AdFUYAO2hwfci2l6Cg95wAAQvIAC8w4HCA4MCkQ4hDQuArsAAaO+W8CV2gQ338BBIKPCk+/kMIQwJ4VBNoLaPQYFIJyPDAqCzKyLAwAAE1jUBSF+IQxEgjAoFnoIdEACGBAovDApEATAqLAChCQyA/4Uw0ODgYsZFO1gCfrojQwK1IwwKRAEyMiwAAXRHAN4XKB2hATAvOIQ7Qx79HOBtjxyTEDjrJSusQoAzhDaG9/TkwcagQGQA1ze4sKguuARQZfDUnoODIeEACKuAwGoCGY1BgchAZlQXAIMCAxBwaADEL65tITwYFgo8GTBxEkMChTsGBSKDGR0WAIMCA2B4aHBRuggHM0iYEgh8GTSzIkMCtQUMCkQmMLorwr1fowIDYFpoEBOUi3cwGDj1MscjUFvBoEBkIjOqC4DBgQHwvwCaHBzaFIOfy2BGQGA1gczEoEBkMrOqC4AJgcGlowcHkx7axJBAbRGDAlGQmFVdALwvQIaHBqBjBAcTn+hoVjcDQwIFA4MCURCZWV1wH8OsKoMnqYtqWwkQQXzMM0MCtXUMCkQhYGZ1wX0Ms6sMYpQuwMEOEkEMBL7MHKzIkEDBxKBAFCLBqC64jxWK0CAmhBfuYDH7bgaGBAo2BgWiEAtmYADCKDS0M8G43ZEhgUKBQYEoTAQ7MAD+FzcGB+2CNR8CQ4I61ibAZglwJ+1k2g+jMCgQhZlQBAb3sVltUCXYkyUxJFAoMSgQhalQBgZA/GLYkcNDqGZSZEigUGNQIApzoQ4MnjpSeAiHKZYZEigcMCgQtRHhFBg8tZfwEA7BwBNDAoULBgWiNiZcA4MnpYtuKINEuAUCMQwJFE4YFIjaKM+LSTiHBjFt4WIdCgwIFI6soW4AEQVOuFjPi0wbx78/ClesKBC1I225ytCRMSRQOGNQIGqnGBrCHwMCtQUMCkQdAENDeGFAoLaEQYGog/G9SDE4BA8DArVFDApEHRyrDcHBkBActibAFuhO+KwHLwwKROTGaoOxGA6oPWBQICJJDA76MCBQe8KgQESqiV0AGR4uYzig9krXhEtr165FamoqoqKikJmZif3796vabsuWLbBYLJg0aZKewxJRGHJN9uT76gg62ueljklzUNi6dStyc3ORn5+PgwcPYuTIkcjJycHZs2dltzt16hR+/etf4+abb9bdWCJqO9pbgGhPn4VIC81BYfXq1ZgzZw5mzZqFoUOHYt26dejcuTM2btwouY3D4cC0adOwdOlS9O/fX/EYTU1NqKur83oRUfsgFSDC6QIcjm0iChVNYxSam5tRWlqKhQsXupdZrVZkZ2ejpKREcrunn34aCQkJmD17Nj766CPF4yxfvhxLly7V0jQiakd4YSYKH5oqCtXV1XA4HEhMTPRanpiYiMrKStFtPv74Y2zYsAHr169XfZyFCxeitrbW/Tp9+rSWZhIREQWVlrF7b7/9NjIyMhAXF4cuXbogLS0Nf/vb39zvt7S0YP78+Rg+fDi6dOmC3r17Y8aMGfjuu++89vPss89izJgx6Ny5M+Li4iSPt2nTJowYMQJRUVFISEjA3LlzNX02U+96uHDhAqZPn47169cjPj5e9XZ2ux12u93ElhERERnDNXZv3bp1yMzMREFBAXJyclBeXo6EhAS/9Xv06IFFixZh8ODBiIyMxPbt2zFr1iwkJCQgJycHDQ0NOHjwIJ566imMHDkSP/74Ix5//HHcc889+OSTT9z7aW5uxv3334+srCxs2LBBtG2rV6/GCy+8gJUrVyIzMxP19fU4deqUps9nEQRBULtyc3MzOnfujLfeesvrzoWZM2eipqYG7777rtf6ZWVlGDVqFGy2K/NkOZ1OAJe7LMrLyzFgwADF49bV1SE2NhZjI+9HhKWT2uYSEVEYaBVaUNy8DbW1tYiJiTHlGK7rxLW5y2CzRwW0L0dTI75c/TvV7c3MzMT111+PNWvWALh8nUtJScFjjz2GBQsWqDrmddddhwkTJuCZZ54Rff/AgQMYPXo0vvnmG/Tt29frvU2bNuGJJ55ATU2N1/Iff/wRffr0wf/+7/9i3LhxqtohRlPXQ2RkJNLT01FUVORe5nQ6UVRUhKysLL/1Bw8ejEOHDqGsrMz9uueee3DbbbehrKwMKSkpuhtORERkJt9B9U1N/nM7u8buZWdnu5epGbvnIggCioqKUF5ejltuuUVyvdraWlgsFtkuBl+7du2C0+nEmTNnMGTIEFx11VV44IEHNHfna+56yM3NxcyZM5GRkYHRo0ejoKAA9fX1mDVrFgBgxowZ6NOnD5YvX46oqCgMGzbMa3vXh/RdTkREFKiIRsCmuk4uzvLvPOD7ZTY/Px9LlizxWiY3du/YsWOSx6itrUWfPn3Q1NQEm82GF198Ebfffrvouo2NjZg/fz6mTp2qqSJz4sQJOJ1OLFu2DH/84x8RGxuLxYsX4/bbb8fnn3+OyMhIVfvRHBQmT56Mc+fOIS8vD5WVlUhLS0NhYaH7JFVUVMBq1TWPExERUdg4ffq014XZyLFz3bp1Q1lZGS5evIiioiLk5uaif//+uPXWW73Wa2lpwQMPPABBEPDSSy9pOobT6URLSwv+9Kc/Yfz48QCAN998E0lJSfjwww+Rk5Ojaj+6BjPOmzcP8+bNE31v9+7dsttu2rRJzyGJiIiCKiYmRvEbfHx8PGw2G6qqqryWV1VVISkpSXI7q9WKgQMHAgDS0tJw9OhRLF++3CsouELCN998g+LiYs3jO5KTkwEAQ4cOdS/r1asX4uPjUVFRoXo//OpPRESkk9axe1KcTqfXGAhXSPjqq6/wwQcfoGfPnprbduONNwIAysvL3cvOnz+P6upq9OvXT/V++FAoIiKiAGgZuwdcnlQwIyMDAwYMQFNTE3bu3Im//e1v7q6FlpYW3HfffTh48CC2b98Oh8PhnquoR48e7rEFFRUVOH/+PCoqKuBwOFBWVgYAGDhwILp27Yprr70WEydOxOOPP45XXnkFMTExWLhwIQYPHozbbrtN9edjUCAiIgqA1rF79fX1ePTRR/Htt98iOjoagwcPxubNmzF58mQAwJkzZ/Dee+8BuNwt4enDDz90d0/k5eXhtddec783atQov3X++te/4sknn8SECRNgtVrx05/+FIWFhejUSf1UA5rmUQgVzqNARNR2BXMehaGPGjOPwhcvqp9Hob3jGAUiIiKSxKBAREREkhgUiIiISBKDAhEREUliUCAiIiJJvD2SiIjaDVtT4M96QLMhTWk3WFEgIiIiSQwKREREJIlBgYiIiCQxKBAREZEkBgUiIiKSxKBAREREkhgUiIiISBKDAhEREUliUCAiIiJJDApEREQkiUGBiIiIJPFZD0RE1G5ENAqwOQN72IOlOdCHRbQvrCgQERGRJAYFIiIiksSgQERERJIYFIiIiEgSgwIRERFJYlAgIiIiSQwKREREJIlBgYiIiCQxKBAREZEkBgUiIiKSxKBAREREkvisByIiajdsTQIiAnzWA1r4rAdPrCgQERGRJAYFIiIiksSgQERERJIYFIiIiEgSgwIRERFJYlAgIiIiSQwKREREJIlBgYiIKEBr165FamoqoqKikJmZif3790uuu379etx8883o3r07unfvjuzsbL/13377bYwfPx49e/aExWJBWVmZ5P4EQcCdd94Ji8WCd955R3SdH374AVdddRUsFgtqamo0fTYGBSIiogBs3boVubm5yM/Px8GDBzFy5Ejk5OTg7Nmzouvv3r0bU6dOxYcffoiSkhKkpKRg/PjxOHPmjHud+vp63HTTTXjuuecUj19QUACLxSK7zuzZszFixAhtH+zfGBSIiIgCsHr1asyZMwezZs3C0KFDsW7dOnTu3BkbN24UXf/111/Ho48+irS0NAwePBivvvoqnE4nioqK3OtMnz4deXl5yM7Olj12WVkZXnjhBcljAcBLL72Empoa/PrXv9b1+RgUiIiIRNTV1Xm9mpqa/NZpbm5GaWmp1wXdarUiOzsbJSUlqo7T0NCAlpYW9OjRQ1P7Ghoa8OCDD2Lt2rVISkoSXeeLL77A008/jb/+9a+wWvVd8hkUiIio3YhocBryAoCUlBTExsa6X8uXL/c7XnV1NRwOBxITE72WJyYmorKyUlWb58+fj969eytWD3w9+eSTGDNmDCZOnCj6flNTE6ZOnYqVK1eib9++mvbtiQ+FIiIiEnH69GnExMS4/2y32w0/xooVK7Blyxbs3r0bUVFRqrd77733UFxcjE8//VRynYULF2LIkCH4j//4j4DayIoCERGRiJiYGK+XWFCIj4+HzWZDVVWV1/KqqirJ7gCXVatWYcWKFfjHP/6heaBhcXExvv76a8TFxSEiIgIREZe/999777249dZb3ets27bN/f64cePcbc7Pz1d9LFYUiIiIdIqMjER6ejqKioowadIkAHAPTJw3b57kds8//zyeffZZvP/++8jIyNB83AULFuChhx7yWjZ8+HD84Q9/wN133w0A+J//+R9cunTJ/f6BAwfwi1/8Ah999BEGDBig+lgMCkRERAHIzc3FzJkzkZGRgdGjR6OgoAD19fWYNWsWAGDGjBno06ePe4zDc889h7y8PLzxxhtITU11j2Xo2rUrunbtCgA4f/48Kioq8N133wEAysvLAQBJSUleL199+/bF1VdfDQB+YaC6uhoAMGTIEMTFxan+fAwKREREAZg8eTLOnTuHvLw8VFZWIi0tDYWFhe4BjhUVFV53HLz00ktobm7Gfffd57Wf/Px8LFmyBMDlMQiuoAEAU6ZM8VsnWCyCIAhBPaIOdXV1iI2NxdjI+xFh6RTq5hARkQatQguKm7ehtrbWa3CgkVzXiRvuehoRndQPChTT2tKIf+3MM7W9bQkHMxIREZEkBgUiIiKSxKBAREREkhgUiIiISBKDAhEREUni7ZFERNRuRDQ6EdHqDGwngW7fzrCiQERERJIYFIiIiEgSgwIRERFJYlAgIiIiSQwKREREJElXUFi7di1SU1MRFRWFzMxM7N+/X3Ld9evX4+abb0b37t3RvXt3ZGdny65PRERE4UNzUNi6dStyc3ORn5+PgwcPYuTIkcjJycHZs2dF19+9ezemTp2KDz/8ECUlJUhJScH48eNx5syZgBtPRERE5tIcFFavXo05c+Zg1qxZGDp0KNatW4fOnTtj48aNouu//vrrePTRR5GWlobBgwfj1VdfhdPpRFFRkeQxmpqaUFdX5/UiIiKi4NMUFJqbm1FaWors7OwrO7BakZ2djZKSElX7aGhoQEtLC3r06CG5zvLlyxEbG+t+paSkaGkmERERGURTUKiurobD4UBiYqLX8sTERFRWVqrax/z589G7d2+vsOFr4cKFqK2tdb9Onz6tpZlERERkkKBO4bxixQps2bIFu3fvRlRUlOR6drsddrs9iC0jIiIiMZqCQnx8PGw2G6qqqryWV1VVISkpSXbbVatWYcWKFfjggw8wYsQI7S0lIiJSYLvUCltEa0D7EFoD27690dT1EBkZifT0dK+BiK6BiVlZWZLbPf/883jmmWdQWFiIjIwM/a0lIiKioNLc9ZCbm4uZM2ciIyMDo0ePRkFBAerr6zFr1iwAwIwZM9CnTx8sX74cAPDcc88hLy8Pb7zxBlJTU91jGbp27YquXbsa+FGIiIjIaJqDwuTJk3Hu3Dnk5eWhsrISaWlpKCwsdA9wrKiogNV6pVDx0ksvobm5Gffdd5/XfvLz87FkyZLAWk9ERESm0jWYcd68eZg3b57oe7t37/b686lTp/QcgoiIiMIAn/VAREREkhgUiIiISBKDAhEREUliUCAiIiJJDApEREQkiUGBiIiIJDEoEBERkaSgPhSKiIjITLZLrbDZWgLah+Dgsx48saJAREREkhgUiIiISBKDAhEREUliUCAiIiJJDApEREQkiUGBiIiIJDEoEBERkSQGBSIiogCtXbsWqampiIqKQmZmJvbv3y+57pEjR3DvvfciNTUVFosFBQUFmvd56tQpWCwW0de2bdsAAJ999hmmTp2KlJQUREdHY8iQIfjjH/+o+bMxKBAREQVg69atyM3NRX5+Pg4ePIiRI0ciJycHZ8+eFV2/oaEB/fv3x4oVK5CUlKRrnykpKfj++++9XkuXLkXXrl1x5513AgBKS0uRkJCAzZs348iRI1i0aBEWLlyINWvWaPp8FkEQBE1bhEBdXR1iY2MxNvJ+RFg6hbo5RESkQavQguLmbaitrUVMTIwpx3BfJ9IWIMJmD2hfrY4mFJetUN3ezMxMXH/99e4LsNPpREpKCh577DEsWLBAdtvU1FQ88cQTeOKJJwLe56hRo3Dddddhw4YNksebO3cujh49iuLiYsXP5cKKAhERkYi6ujqvV1NTk986zc3NKC0tRXZ2tnuZ1WpFdnY2SkpKdB1Xzz5LS0tRVlaG2bNny+67trYWPXr00NQePuuBiIjaDWt9E6y2APfhuBwIUlJSvJbn5+djyZIlXsuqq6vhcDiQmJjotTwxMRHHjh3TdXw9+9ywYQOGDBmCMWPGSO5379692Lp1K3bs2KGpPQwKREREIk6fPu3V9WC3B9alYZZLly7hjTfewFNPPSW5zuHDhzFx4kTk5+dj/PjxmvbPoEBERCQiJiZGcYxCfHw8bDYbqqqqvJZXVVVJDlRUonWfb731FhoaGjBjxgzR/X3xxRcYN24cHn74YSxevFhzezhGgYiISKfIyEikp6ejqKjIvczpdKKoqAhZWVlB2eeGDRtwzz33oFevXn7vHTlyBLfddhtmzpyJZ599Vld7WFEgIiIKQG5uLmbOnImMjAyMHj0aBQUFqK+vx6xZswAAM2bMQJ8+fbB8+XIAlwcrfvHFF+7/P3PmDMrKytC1a1cMHDhQ1T5djh8/jj179mDnzp1+7Tp8+DDGjh2LnJwc5ObmorKyEgBgs9lEQ4UUBgUiIqIATJ48GefOnUNeXh4qKyuRlpaGwsJC92DEiooKWK1XCvjfffcdRo0a5f7zqlWrsGrVKvz0pz/F7t27Ve3TZePGjbjqqqtExx289dZbOHfuHDZv3ozNmze7l/fr1w+nTp1S/fk4jwIREZkqmPMoZF/zpCHzKHzw1R9MbW9bwjEKREREJIlBgYiIiCQxKBAREZEkBgUiIiKSxKBAREREknh7JBERtR8NlwCrM7B9OP0f/tSRsaJAREREkhgUiIiISBKDAhEREUliUCAiIiJJDApEREQkiUGBiIiIJDEoEBERkSQGBSIiIpLEoEBERESSGBSIiIhIEqdwJiKidkNouATB6ghsH85mg1rTPrCiQERERJIYFIiIiEgSgwIRERFJYlAgIiIiSQwKREREJIlBgYiIiCQxKBAREZEkBgUiIiKSxKBAREREkhgUiIiISBKDAhEREUnisx6IiKjdEOobIFhaAtuHENj27Q0rCkRERCSJQYGIiIgkMSgQERGRJAYFIiIiksSgQERERJIYFIiIiEgSgwIRERFJ0hUU1q5di9TUVERFRSEzMxP79++XXX/btm0YPHgwoqKiMHz4cOzcuVNXY4mIiMKR0ddFQRCQl5eH5ORkREdHIzs7G1999ZXXOufPn8e0adMQExODuLg4zJ49GxcvXvRa5/PPP8fNN9+MqKgopKSk4Pnnn9f82TQHha1btyI3Nxf5+fk4ePAgRo4ciZycHJw9e1Z0/b1792Lq1KmYPXs2Pv30U0yaNAmTJk3C4cOHNTeWiIgo3JhxXXz++efxpz/9CevWrcO+ffvQpUsX5OTkoLGx0b3OtGnTcOTIEezatQvbt2/Hnj178PDDD7vfr6urw/jx49GvXz+UlpZi5cqVWLJkCV555RVNn88iCIKgZYPMzExcf/31WLNmDQDA6XQiJSUFjz32GBYsWOC3/uTJk1FfX4/t27e7l91www1IS0vDunXrVB2zrq4OsbGxGBt5PyIsnbQ0l4iIQqxVaEFx8zbU1tYiJibGlGMYeZ3Q2l6jr4uCIKB37974r//6L/z6178GANTW1iIxMRGbNm3ClClTcPToUQwdOhQHDhxARkYGAKCwsBB33XUXvv32W/Tu3RsvvfQSFi1ahMrKSkRGRgIAFixYgHfeeQfHjh1TfT40TeHc3NyM0tJSLFy40L3MarUiOzsbJSUlotuUlJQgNzfXa1lOTg7eeecdyeM0NTWhqanJ/efa2loAl//yiIiobXH97tb4vVTfsdACBHiYVlxub11dnddyu90Ou93utcyM6+LJkydRWVmJ7Oxs9/uxsbHIzMxESUkJpkyZgpKSEsTFxblDAgBkZ2fDarVi3759+D//5/+gpKQEt9xyizskuI7z3HPP4ccff0T37t1VnQ9NQaG6uhoOhwOJiYleyxMTEyXTSWVlpej6lZWVksdZvnw5li5d6rd8T8s7WppLRERh5IcffkBsbKwp+46MjERSUhL2VL5jyP66du2KlJQUr2X5+flYsmSJ1zIzrouu/yqtk5CQ4PV+REQEevTo4bXO1Vdf7bcP13umBIVgWbhwoVfaqqmpQb9+/VBRUWHaD1l7UFdXh5SUFJw+fdq08l57wPOkjOdIHZ4ndWpra9G3b1/06NHDtGNERUXh5MmTaG5uNmR/giDAYrF4LfOtJnQUmoJCfHw8bDYbqqqqvJZXVVUhKSlJdJukpCRN6wPi5R3gcumF/xiVxcTE8DypwPOkjOdIHZ4ndaxWc+/Ij4qKQlRUlKnH8GXGddH136qqKiQnJ3utk5aW5l7Hd7Bka2srzp8/77UfseN4HkMNTX9rkZGRSE9PR1FRkXuZ0+lEUVERsrKyRLfJysryWh8Adu3aJbk+ERFRW2HGdfHqq69GUlKS1zp1dXXYt2+fe52srCzU1NSgtLTUvU5xcTGcTicyMzPd6+zZswctLS1exxk0aJDqbgcAgKDRli1bBLvdLmzatEn44osvhIcffliIi4sTKisrBUEQhOnTpwsLFixwr//Pf/5TiIiIEFatWiUcPXpUyM/PFzp16iQcOnRI9TFra2sFAEJtba3W5nYoPE/q8Dwp4zlSh+dJnfZ+nsy4Lq5YsUKIi4sT3n33XeHzzz8XJk6cKFx99dXCpUuX3OvccccdwqhRo4R9+/YJH3/8sXDNNdcIU6dOdb9fU1MjJCYmCtOnTxcOHz4sbNmyRejcubPw8ssva/p8moOCIAjCn//8Z6Fv375CZGSkMHr0aOFf//qX+72f/vSnwsyZM73W/+///m/h2muvFSIjI4Wf/OQnwo4dOzQdr7GxUcjPzxcaGxv1NLfD4HlSh+dJGc+ROjxP6nSE82T0ddHpdApPPfWUkJiYKNjtdmHcuHFCeXm51zo//PCDMHXqVKFr165CTEyMMGvWLOHChQte63z22WfCTTfdJNjtdqFPnz7CihUrNH82zfMoEBERUcfBZz0QERGRJAYFIiIiksSgQERERJIYFIiIiEgSgwIRERFJCpugYPSzvNsrLedp/fr1uPnmm9G9e3d0794d2dnZiue1PdD6s+SyZcsWWCwWTJo0ydwGhgmt56mmpgZz585FcnIy7HY7rr322g7x707reSooKMCgQYMQHR2NlJQUPPnkk16PBm6P9uzZg7vvvhu9e/eGxWKRfeify+7du3HdddfBbrdj4MCB2LRpk+ntJJ0031Bpgi1btgiRkZHCxo0bhSNHjghz5swR4uLihKqqKtH1//nPfwo2m014/vnnhS+++EJYvHix5kmc2iKt5+nBBx8U1q5dK3z66afC0aNHhZ///OdCbGys8O233wa55cGj9Ry5nDx5UujTp49w8803CxMnTgxOY0NI63lqamoSMjIyhLvuukv4+OOPhZMnTwq7d+8WysrKgtzy4NJ6nl5//XXBbrcLr7/+unDy5Enh/fffF5KTk4Unn3wyyC0Prp07dwqLFi0S3n77bQGA8Pe//112/RMnTgidO3cWcnNzhS+++EL485//LNhsNqGwsDA4DSZNwiIojB49Wpg7d677zw6HQ+jdu7ewfPly0fUfeOABYcKECV7LMjMzhV/+8pemtjPUtJ4nX62trUK3bt2E1157zawmhpyec9Ta2iqMGTNGePXVV4WZM2d2iKCg9Ty99NJLQv/+/YXm5uZgNTEsaD1Pc+fOFcaOHeu1LDc3V7jxxhtNbWc4URMUfvvb3wo/+clPvJZNnjxZyMnJMbFlpFfIux5cz/L2fO62mmd5e64PXH7GttT67YGe8+SroaEBLS0tpj7BLZT0nqOnn34aCQkJmD17djCaGXJ6ztN7772HrKwszJ07F4mJiRg2bBiWLVsGh8MRrGYHnZ7zNGbMGJSWlrq7J06cOIGdO3firrvuCkqb24qO+Du8LQv5Y6bNeJZ3e6TnPPmaP38+evfu7fcPtL3Qc44+/vhjbNiwAWVlZUFoYXjQc55OnDiB4uJiTJs2DTt37sTx48fx6KOPoqWlBfn5+cFodtDpOU8PPvggqqurcdNNN0EQBLS2tuJXv/oVfve73wWjyW2G1O/wuro6XLp0CdHR0SFqGYkJeUWBgmPFihXYsmUL/v73vwf9Mazh6sKFC5g+fTrWr1+P+Pj4UDcnrDmdTiQkJOCVV15Beno6Jk+ejEWLFmHdunWhblpY2b17N5YtW4YXX3wRBw8exNtvv40dO3bgmWeeCXXTiHQLeUXBjGd5t0d6zpPLqlWrsGLFCnzwwQcYMWKEmc0MKa3n6Ouvv8apU6dw9913u5c5nU4AQEREBMrLyzFgwABzGx0Cen6WkpOT0alTJ9hsNveyIUOGoLKyEs3NzYiMjDS1zaGg5zw99dRTmD59Oh566CEAwPDhw1FfX4+HH34YixYtgtXK72aA9O/wmJgYVhPCUMh/as14lnd7pOc8AcDzzz+PZ555BoWFhcjIyAhGU0NG6zkaPHgwDh06hLKyMvfrnnvuwW233YaysjKkpKQEs/lBo+dn6cYbb8Tx48fdQQoAvvzySyQnJ7fLkADoO08NDQ1+YcAVrgQ+f8+tI/4Ob9NCPZpSEMx5lnd7pPU8rVixQoiMjBTeeust4fvvv3e/fB9D2p5oPUe+OspdD1rPU0VFhdCtWzdh3rx5Qnl5ubB9+3YhISFB+P3vfx+qjxAUWs9Tfn6+0K1bN+HNN98UTpw4IfzjH/8QBgwYIDzwwAOh+ghBceHCBeHTTz8VPv30UwGAsHr1auHTTz8VvvnmG0EQBGHBggXC9OnT3eu7bo/8zW9+Ixw9elRYu3Ytb48MY2ERFATB+Gd5t1dazlO/fv0EAH6v/Pz84Dc8iLT+LHnqKEFBELSfp7179wqZmZmC3W4X+vfvLzz77LNCa2trkFsdfFrOU0tLi7BkyRJhwIABQlRUlJCSkiI8+uijwo8//hj8hgfRhx9+KPq7xnVuZs6cKfz0pz/12yYtLU2IjIwU+vfvL/zlL38JertJHYsgsB5GRERE4kI+RoGIiIjCF4MCERERSWJQICIiIkkMCkRERCSJQYGIiIgkMSgQERGRJAYFIiIiksSgQERERJIYFIiIiEgSgwIRERFJYlAgIiIiSf8fbPvvfPbI/KQAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\")\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01)\n", + "\n", + "u_0 = Function(space, name=\"u_0\")\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "u_n = Function(space, name=\"u_n\")\n", + "u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + "u_h = 0.5 * (u_n + trial)\n", + "F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + "lhs, rhs = system(F)\n", + "\n", + "problem = LinearVariationalProblem(\n", + " lhs, rhs, u_np1,\n", + " constant_jacobian=True)\n", + "solver = LinearVariationalSolver(\n", + " problem, solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + "u_n.assign(u_0)\n", + "for n in range(N):\n", + " solver.solve()\n", + " u_n.assign(u_np1)\n", + "\n", + "J = assemble(inner(u_n, u_n) * dx)\n", + "\n", + "\n", + "def plot_output(u, title):\n", + " r = (u.dat.data_ro.min(), u.dat.data_ro.max())\n", + " eps = (r[1] - r[0]) * 1.0e-12\n", + " p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))\n", + " plt.gca().set_title(title)\n", + " plt.colorbar(p)\n", + " plt.gca().set_aspect(1.0)\n", + "\n", + "\n", + "plot_output(u_0, title=\"$u_0$\")\n", + "plot_output(u_n, title=\"$u_n$\")" + ] + }, + { + "cell_type": "markdown", + "id": "e2b78d66", + "metadata": {}, + "source": [ + "## Adding tlm_adjoint\n", + "\n", + "We first modify the code so that tlm_adjoint processes the calculations:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "fd78c6fa", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:45:22.041687Z", + "iopub.status.busy": "2023-12-20T16:45:22.041394Z", + "iopub.status.idle": "2023-12-20T16:45:28.169223Z", + "shell.execute_reply": "2023-12-20T16:45:28.168342Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(True, True)" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "reset_manager(\"memory\", {})\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\")\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01)\n", + "\n", + "u_0 = Function(space, name=\"u_0\")\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "\n", + "def forward(u_0, psi):\n", + " u_n = Function(space, name=\"u_n\")\n", + " u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + " u_h = 0.5 * (u_n + trial)\n", + " F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + " lhs, rhs = system(F)\n", + "\n", + " problem = LinearVariationalProblem(\n", + " lhs, rhs, u_np1,\n", + " constant_jacobian=True)\n", + " solver = LinearVariationalSolver(\n", + " problem, solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + " u_n.assign(u_0)\n", + " for n in range(N):\n", + " solver.solve()\n", + " u_n.assign(u_np1)\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u_n, u_n) * dx)\n", + " return J\n", + "\n", + "\n", + "start_manager()\n", + "J = forward(u_0, psi)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "721fd3a1", + "metadata": {}, + "source": [ + "Later we will configure a checkpointing schedule. Resetting the manager resets the record of forward equations but does not reset the checkpointing configuration, and so in this example whenever we reset the manager we also return it to the default checkpointing configuration with `reset_manager(\"memory\", {})`.\n", + "\n", + "## Computing derivatives using an adjoint\n", + "\n", + "The `compute_gradient` function can be used to compute derivatives using the adjoint method. Here we compute the derivative of the square of the $L^2$-norm of the final timestep solution, considered a function of the control defined by the initial condition `u_0` and stream function `psi`, with respect to this control:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "0e5b3bba", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:45:28.173427Z", + "iopub.status.busy": "2023-12-20T16:45:28.172784Z", + "iopub.status.idle": "2023-12-20T16:45:35.452344Z", + "shell.execute_reply": "2023-12-20T16:45:35.451681Z" + } + }, + "outputs": [], + "source": [ + "dJ_du_0, dJ_dpsi = compute_gradient(J, (u_0, psi))" + ] + }, + { + "cell_type": "markdown", + "id": "98132cf9", + "metadata": {}, + "source": [ + "As a simple check of the result, note that the solution to the (discretized) partial differential equation is unchanged by the addition of a constant to the stream function. Hence we expect the directional derivative with respect to the stream function, with direction equal to the unity valued function, to be zero. This is indeed found to be the case (except for roundoff errors):" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "b5069278", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:45:35.455414Z", + "iopub.status.busy": "2023-12-20T16:45:35.455184Z", + "iopub.status.idle": "2023-12-20T16:45:35.484120Z", + "shell.execute_reply": "2023-12-20T16:45:35.483483Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "dJ_dpsi_one=1.8775546878197248e-18\n" + ] + } + ], + "source": [ + "one = Function(space, name=\"one\")\n", + "one.interpolate(Constant(1.0))\n", + "\n", + "dJ_dpsi_one = var_inner(one, dJ_dpsi)\n", + "\n", + "print(f\"{dJ_dpsi_one=}\")\n", + "\n", + "assert abs(dJ_dpsi_one) < 1.0e-17" + ] + }, + { + "cell_type": "markdown", + "id": "d0ecd1a6", + "metadata": {}, + "source": [ + "## Computing Hessian information using an adjoint of a tangent-linear\n", + "\n", + "We next compute a Hessian action. Although the following calculation does work, it is inefficient – you may wish to skip forward to the optimized calculations.\n", + "\n", + "Here we compute a 'mixed' Hessian action, by defining a directional derivative with respect to the stream function, and then differentiating this with respect to the initial condition:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "95483f48", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:45:35.486483Z", + "iopub.status.busy": "2023-12-20T16:45:35.486283Z", + "iopub.status.idle": "2023-12-20T16:46:20.045261Z", + "shell.execute_reply": "2023-12-20T16:46:20.044557Z" + } + }, + "outputs": [], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "reset_manager(\"memory\", {})\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\")\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01)\n", + "\n", + "u_0 = Function(space, name=\"u_0\")\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "\n", + "def forward(u_0, psi):\n", + " u_n = Function(space, name=\"u_n\")\n", + " u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + " u_h = 0.5 * (u_n + trial)\n", + " F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + " lhs, rhs = system(F)\n", + "\n", + " problem = LinearVariationalProblem(\n", + " lhs, rhs, u_np1,\n", + " constant_jacobian=True)\n", + " solver = LinearVariationalSolver(\n", + " problem, solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + " u_n.assign(u_0)\n", + " for n in range(N):\n", + " solver.solve()\n", + " u_n.assign(u_np1)\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u_n, u_n) * dx)\n", + " return J\n", + "\n", + "\n", + "zeta = Function(space, name=\"zeta\")\n", + "zeta.assign(psi)\n", + "configure_tlm((psi, zeta))\n", + "\n", + "start_manager()\n", + "J = forward(u_0, psi)\n", + "stop_manager()\n", + "\n", + "dJ_dpsi_zeta = var_tlm(J, (psi, zeta))\n", + "\n", + "d2J_dpsi_zeta_du_0 = compute_gradient(dJ_dpsi_zeta, u_0)" + ] + }, + { + "cell_type": "markdown", + "id": "ddad9733", + "metadata": {}, + "source": [ + "## Optimization\n", + "\n", + "In the above we have successfully built a record of calculations, and used this to compute derivative information. However there are two issues:\n", + "\n", + "1. Building the record has a noticable cost – the forward calculation has slowed down. In the second order calculation overheads associated with the tangent-linear lead to substantial additional costs.\n", + "2. tlm_adjoint records the solution of the partial differential equation on all time levels. The memory usage here is manageable. However memory limits will be exceeded for larger problems with more fields, spatial degrees of freedom, or timesteps.\n", + "\n", + "Let's fix these issues in order.\n", + "\n", + "### Optimizing the annotation\n", + "\n", + "In the above code tlm_adjoint builds a new record for each finite element variational problem it encounters. Even though only one `LinearVariationalSolver` is instantiated, an `EquationSolver` record is instantiated on each call to the `solve` method. Building the record is sufficiently expensive that the forward calculation noticeably slows down, and this also leads to significant extra processing in the derivative calculations.\n", + "\n", + "Instead we can instantiate an `EquationSolver` directly, and reuse it. However if we do only that then the code will still be inefficient. A single `EquationSolver` will be used, but new linear solver data will be constructed each time its `solve` method is called. We need to also apply an optimization analogous to the `constant_jacobian=True` argument supplied to `LinearVariationalProblem`.\n", + "\n", + "A simple fix is to add `cache_jacobian=True` when instantiating the `EquationSolver`:\n", + "\n", + "```\n", + "eq = EquationSolver(\n", + " lhs == rhs, u_np1,\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"},\n", + " cache_jacobian=True)\n", + "```\n", + "\n", + "This works, but we can instead let tlm_adjoint detect that linear solver data can be cached. We can do that by adding `static=True` when instantiating variables whose value is unchanged throughout the forward calculation:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "2856cf07", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:46:20.048098Z", + "iopub.status.busy": "2023-12-20T16:46:20.047852Z", + "iopub.status.idle": "2023-12-20T16:46:21.596864Z", + "shell.execute_reply": "2023-12-20T16:46:21.596188Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(True, True)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "reset_manager(\"memory\", {})\n", + "clear_caches()\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N, static=True)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\", static=True)\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01, static=True)\n", + "\n", + "u_0 = Function(space, name=\"u_0\", static=True)\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "\n", + "def forward(u_0, psi):\n", + " u_n = Function(space, name=\"u_n\")\n", + " u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + " u_h = 0.5 * (u_n + trial)\n", + " F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + " lhs, rhs = system(F)\n", + "\n", + " eq = EquationSolver(\n", + " lhs == rhs, u_np1,\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + " u_n.assign(u_0)\n", + " for n in range(N):\n", + " eq.solve()\n", + " u_n.assign(u_np1)\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u_n, u_n) * dx)\n", + " return J\n", + "\n", + "\n", + "start_manager()\n", + "J = forward(u_0, psi)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "6fe47e03", + "metadata": {}, + "source": [ + "If we now query the relevant tlm_adjoint caches:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "eb3638f2", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:46:21.599650Z", + "iopub.status.busy": "2023-12-20T16:46:21.599442Z", + "iopub.status.idle": "2023-12-20T16:46:21.603252Z", + "shell.execute_reply": "2023-12-20T16:46:21.602586Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "len(assembly_cache())=2\n", + "len(linear_solver_cache())=1\n" + ] + } + ], + "source": [ + "print(f\"{len(assembly_cache())=}\")\n", + "print(f\"{len(linear_solver_cache())=}\")\n", + "\n", + "assert len(assembly_cache()) == 2\n", + "assert len(linear_solver_cache()) == 1" + ] + }, + { + "cell_type": "markdown", + "id": "fee57222", + "metadata": {}, + "source": [ + "we find that linear solver data associated with a single matrix has been cached. We also find that two assembled objects have been cached – it turns out that there are two cached matrices. As well as caching the matrix associated with the left-hand-side of the discrete problem, a matrix associated with the *right-hand-side* has been assembled and cached. Assembly of the right-hand-side has been converted into a matrix multiply. If we wished we could disable right-hand-side optimizations by adding `cache_rhs_assembly=False`:\n", + "\n", + "```\n", + "eq = EquationSolver(\n", + " lhs == rhs, u_np1,\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"},\n", + " cache_rhs_assembly=False)\n", + "```\n", + "\n", + "### Using a checkpointing schedule\n", + "\n", + "To address the storage issue we enable checkpointing. Here we enable binomial checkpointing with storage of a maximum of $10$ forward restart checkpoints in memory:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "20b7fb3f", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:46:21.605530Z", + "iopub.status.busy": "2023-12-20T16:46:21.605333Z", + "iopub.status.idle": "2023-12-20T16:46:25.281960Z", + "shell.execute_reply": "2023-12-20T16:46:25.281301Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 9223372036854775807\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 21\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 0 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 40\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 21 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 57\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 40 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 72\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 57 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 79\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 72 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 85\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 79 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 90\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 85 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 94\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 90 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 97\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 94 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 99\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 97 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 100\n" + ] + }, + { + "data": { + "text/plain": [ + "(True, True)" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import logging\n", + "\n", + "logger = logging.getLogger(\"tlm_adjoint\")\n", + "logger.setLevel(logging.DEBUG)\n", + "root_logger = logging.getLogger()\n", + "if len(logger.handlers) == 1:\n", + " if len(root_logger.handlers) == 1:\n", + " root_logger.handlers.pop()\n", + " root_logger.addHandler(logger.handlers.pop())\n", + "\n", + "reset_manager(\"memory\", {})\n", + "clear_caches()\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N, static=True)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\", static=True)\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01, static=True)\n", + "\n", + "u_0 = Function(space, name=\"u_0\", static=True)\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "\n", + "def forward(u_0, psi):\n", + " u_n = Function(space, name=\"u_n\")\n", + " u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + " u_h = 0.5 * (u_n + trial)\n", + " F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + " lhs, rhs = system(F)\n", + "\n", + " eq = EquationSolver(\n", + " lhs == rhs, u_np1,\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + " u_n.assign(u_0)\n", + " for n in range(N):\n", + " eq.solve()\n", + " u_n.assign(u_np1)\n", + " if n < N - 1:\n", + " new_block()\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u_n, u_n) * dx)\n", + " return J\n", + "\n", + "\n", + "configure_checkpointing(\"multistage\", {\"snaps_in_ram\": 10, \"blocks\": N})\n", + "start_manager()\n", + "J = forward(u_0, psi)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "c20a3dbb", + "metadata": {}, + "source": [ + "The key changes here are:\n", + "\n", + "- Configuration of a checkpointing schedule using `configure_checkpointing`. Here binomial checkpointing is applied, with a maximum of $10$ forward restart checkpoints stored in memory, indicated using the `\"snaps_in_ram\"` parameter. The total number of steps is indicated using the `\"blocks\"` parameter.\n", + "- The indication of the steps using `new_block()`.\n", + "\n", + "Extra logging output is also enabled so that we can see the details of the checkpointing schedule.\n", + "\n", + "### Computing derivatives\n", + "\n", + "We are now ready to compute derivatives. However a key restriction is that we can, with this checkpointing schedule, only perform the adjoint calculation *once* per forward calculation. We cannot call `compute_gradient` a second time, without first rerunning the entire forward calculation.\n", + "\n", + "In the following we compute both first and second derivative information using a single adjoint calculation:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "33534595", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:46:25.284712Z", + "iopub.status.busy": "2023-12-20T16:46:25.284498Z", + "iopub.status.idle": "2023-12-20T16:46:33.222130Z", + "shell.execute_reply": "2023-12-20T16:46:33.221508Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 9223372036854775807\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 21\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 0 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 40\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 21 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 57\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 40 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 72\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 57 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 79\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 72 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 85\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 79 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 90\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 85 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 94\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 90 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 97\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 94 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 99\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: save checkpoint data at 97 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "forward: forward advance to 100\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 99\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 97 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 98\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 99\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 98\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 97 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 98\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 97\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 94 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 95\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 96\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 95 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 97\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 96\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 95 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 96\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 95\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 94 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 95\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 94\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 90 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 91\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 92\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 91 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 93\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 92 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 94\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 93\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 92 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 93\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 92\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 91 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 92\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 91\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 90 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 91\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 90\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 85 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 86\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 87\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 86 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 88\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 87 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 89\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 88 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 90\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 89\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 88 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 89\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 88\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 87 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 88\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 87\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 86 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 87\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 86\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 85 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 86\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 85\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 79 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 80\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 81\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 80 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 82\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 81 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 83\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 82 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 84\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 83 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 85\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 84\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 83 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 84\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 83\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 82 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 83\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 82\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 81 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 82\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 81\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 80 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 81\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 80\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 79 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 80\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 79\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 72 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 73\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 74\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 73 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 75\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 74 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 76\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 75 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 77\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 76 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 78\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 77 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 79\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 78\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 77 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 78\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 77\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 76 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 77\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 76\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 75 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 76\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 75\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 74 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 75\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 74\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 73 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 74\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 73\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 72 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 73\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 72\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 57 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 59\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 61\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 59 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 63\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 61 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 65\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 63 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 67\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 65 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 69\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 67 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 71\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 69 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 72\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 71\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 69 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 70\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 71\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 70\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 69 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 70\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 69\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 67 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 68\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 69\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 68\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 67 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 68\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 67\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 65 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 66\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 67\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 66\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 65 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 66\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 65\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 63 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 64\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 65\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 64\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 63 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 64\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 63\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 61 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 62\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 63\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 62\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 61 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 62\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 61\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 59 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 60\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 61\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 60\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 59 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 60\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 59\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 57 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 58\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 59\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 58\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 57 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 58\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 57\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 40 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 42\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 44\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 42 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 46\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 44 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 48\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 46 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 50\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 48 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 52\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 50 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 54\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 52 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 56\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 54 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 57\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 56\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 54 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 55\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 56\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 55\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 54 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 55\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 54\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 52 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 53\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 54\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 53\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 52 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 53\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 52\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 50 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 51\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 52\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 51\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 50 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 51\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 50\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 48 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 49\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 50\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 49\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 48 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 49\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 48\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 46 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 47\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 48\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 47\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 46 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 47\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 46\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 44 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 45\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 46\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 45\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 44 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 45\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 44\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 42 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 43\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 44\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 43\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 42 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 43\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 42\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 40 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 41\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 42\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 41\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 40 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 41\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 40\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 21 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 23\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 25\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 23 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 27\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 25 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 29\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 27 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 31\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 29 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 33\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 31 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 35\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 33 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 37\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 35 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 39\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 37 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 40\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 39\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 37 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 38\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 39\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 38\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 37 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 38\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 37\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 35 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 36\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 37\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 36\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 35 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 36\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 35\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 33 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 34\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 35\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 34\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 33 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 34\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 33\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 31 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 32\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 33\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 32\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 31 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 32\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 31\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 29 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 30\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 31\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 30\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 29 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 30\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 29\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 27 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 28\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 29\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 28\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 27 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 28\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 27\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 25 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 26\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 27\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 26\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 25 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 26\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 25\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 23 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 24\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 25\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 24\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 23 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 24\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 23\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 21 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 22\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 23\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 22\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 21 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 22\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 21\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 0 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 2\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 4\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 2 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 6\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 4 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 8\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 6 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 10\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 8 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 12\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 10 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 14\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 12 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 16\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 14 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 18\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 16 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 20\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: save checkpoint data at 18 in RAM\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 21\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 20\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 18 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 19\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 20\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 19\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 18 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 19\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 18\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 16 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 17\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 18\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 17\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 16 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 17\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 16\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 14 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 15\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 16\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 15\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 14 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 15\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 14\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 12 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 13\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 14\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 13\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 12 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 13\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 12\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 10 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 11\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 12\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 11\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 10 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 11\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 10\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 8 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 9\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 10\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 9\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 8 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 9\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 8\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 6 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 7\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 8\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 7\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 6 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 7\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 6\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 4 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 5\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 6\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 5\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 4 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 5\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 4\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 2 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 3\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 4\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 3\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 2 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 3\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 2\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 0 from RAM and keep\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 1\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 2\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 1\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: load checkpoint data at 0 from RAM and delete\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: forward advance to 1\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "reverse: adjoint step back to 0\n" + ] + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import logging\n", + "\n", + "logger = logging.getLogger(\"tlm_adjoint\")\n", + "logger.setLevel(logging.DEBUG)\n", + "root_logger = logging.getLogger()\n", + "if len(logger.handlers) == 1:\n", + " if len(root_logger.handlers) == 1:\n", + " root_logger.handlers.pop()\n", + " root_logger.addHandler(logger.handlers.pop())\n", + "\n", + "reset_manager(\"memory\", {})\n", + "clear_caches()\n", + "\n", + "T = 0.1\n", + "N = 100\n", + "dt = Constant(T / N, static=True)\n", + "\n", + "mesh = UnitSquareMesh(128, 128)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test = TestFunction(space)\n", + "trial = TrialFunction(space)\n", + "\n", + "psi = Function(space, name=\"psi\", static=True)\n", + "psi.interpolate(-sin(pi * X[0]) * sin(pi * X[1]))\n", + "\n", + "kappa = Constant(0.01, static=True)\n", + "\n", + "u_0 = Function(space, name=\"u_0\", static=True)\n", + "u_0.interpolate(exp(-50.0 * ((X[0] - 0.75) ** 2 + (X[1] - 0.5) ** 2)))\n", + "\n", + "\n", + "def forward(u_0, psi):\n", + " u_n = Function(space, name=\"u_n\")\n", + " u_np1 = Function(space, name=\"u_np1\")\n", + "\n", + " u_h = 0.5 * (u_n + trial)\n", + " F = (inner(trial - u_n, test) * dx\n", + " + dt * inner(psi.dx(0) * u_h.dx(1) - psi.dx(1) * u_h.dx(0), test) * dx\n", + " + dt * inner(kappa * grad(u_h), grad(test)) * dx)\n", + " lhs, rhs = system(F)\n", + "\n", + " eq = EquationSolver(\n", + " lhs == rhs, u_np1,\n", + " solver_parameters={\"ksp_type\": \"preonly\",\n", + " \"pc_type\": \"lu\"})\n", + "\n", + " u_n.assign(u_0)\n", + " for n in range(N):\n", + " eq.solve()\n", + " u_n.assign(u_np1)\n", + " if n < N - 1:\n", + " new_block()\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(inner(u_n, u_n) * dx)\n", + " return J\n", + "\n", + "\n", + "zeta_u_0 = ZeroFunction(space, name=\"zeta_u_0\")\n", + "zeta_psi = Function(space, name=\"zeta_psi\", static=True)\n", + "zeta_psi.assign(psi)\n", + "configure_tlm(((u_0, psi), (zeta_u_0, zeta_psi)))\n", + "\n", + "configure_checkpointing(\"multistage\", {\"snaps_in_ram\": 10, \"blocks\": N})\n", + "start_manager()\n", + "J = forward(u_0, psi)\n", + "stop_manager()\n", + "\n", + "dJ_dpsi_zeta = var_tlm(J, ((u_0, psi), (zeta_u_0, zeta_psi)))\n", + "\n", + "dJ_du_0, dJ_dpsi, d2J_dpsi_zeta_du_0 = compute_gradient(\n", + " dJ_dpsi_zeta, (zeta_u_0, zeta_psi, u_0))" + ] + }, + { + "cell_type": "markdown", + "id": "283156e2", + "metadata": {}, + "source": [ + "The derivative calculation now alternates between forward + tangent-linear calculations, and adjoint calculations.\n", + "\n", + "If we wished we could perform higher order adjoint calculations, using a binomial checkpointing schedule, by supplying a higher order tangent-linear configuration and differentiating the result." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/4_riesz_maps.html b/examples/4_riesz_maps.html new file mode 100644 index 0000000..c526675 --- /dev/null +++ b/examples/4_riesz_maps.html @@ -0,0 +1,312 @@ + + + + + + + Visualizing derivatives — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Visualizing derivatives

+

When we differentiate a functional with respect to a finite element discretized function in some discrete function space \(V\), we do not obtain a function in \(V\), but instead obtain an element of the associated dual space \(V^*\). This notebook describes how a Riesz map can be used to construct a function, associated with the derivative, which can be visualized.

+

This example makes use of tlm_adjoint with the Firedrake backend, and we assume real spaces and a real build of Firedrake throughout.

+
+

Forward problem

+

We consider the solution \(u \in V\) of a discretization of the Poisson equation subject to homogeneous Dirichlet boundary conditions,

+
+\[\forall \zeta \in V_0 \qquad \int_\Omega \nabla \zeta \cdot \nabla u = -\int_\Omega \zeta m,\]
+

where \(V\) is a real \(P_1\) continuous finite element space defining functions on the domain \(\Omega = \left( 0, 1 \right)^2\), with \(m \in V\), and where \(V_0\) consists of the functions in \(V\) which have zero trace. We define a functional

+
+\[J \left( u \right) = \int_\Omega \left( 1 - x \right)^4 u^2,\]
+

where \(x\) and \(y\) denote Cartesian coordinates in \(\mathbb{R}^2\).

+

We first solve the forward problem for

+
+\[m = \mathcal{I} \left[ \sin \left( \pi x \right) \sin \left( 2 \pi y \right) \right],\]
+

where \(\mathcal{I}\) maps to an element of \(V\) through interpolation at mesh vertices.

+
+
[1]:
+
+
+
%matplotlib inline
+
+from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import matplotlib.pyplot as plt
+import numpy as np
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test, trial = TestFunction(space), TrialFunction(space)
+m = Function(space, name="m").interpolate(sin(pi * X[0]) * sin(2 * pi * X[1]))
+
+
+def forward(m):
+    u = Function(space, name="u")
+    solve(inner(grad(trial), grad(test)) * dx == -inner(m, test) * dx,
+          u, DirichletBC(space, 0.0, "on_boundary"))
+
+    J = Functional(name="J")
+    J.assign(((1.0 - X[0]) ** 4) * u * u * dx)
+    return u, J
+
+
+u, J = forward(m)
+
+print(f"{J.value=}")
+
+
+def plot_output(u, title):
+    r = (u.dat.data_ro.min(), u.dat.data_ro.max())
+    eps = (r[1] - r[0]) * 1.0e-12
+    p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))
+    plt.gca().set_title(title)
+    plt.colorbar(p)
+    plt.gca().set_aspect(1.0)
+
+
+plot_output(u, title="u")
+
+
+
+
+
+
+
+
+J.value=9.711060188691798e-06
+
+
+
+
+
+
+../_images/examples_4_riesz_maps_1_1.png +
+
+
+
+

First order adjoint

+

We can differentiate a functional respect to \(m\) using compute_gradient. Specifically we compute the derivative of \(\hat{J} = J \circ \hat{u}\), where \(\hat{u}\) is the function which maps from the control \(m\) to the solution to the discretized Poisson problem.

+

When we compute a derivative of a functional with respect to a finite element discretized function using the adjoint method the result is not a function, but is instead a member of a dual space. Here we have \(m \in V\), and the derivative of \(\hat{J}\) with respect to \(m\) is a member of the associated dual space \(V^*\). In order to visualize this derivative we first need to map it to \(V\). We can do this using a Riesz map. This is not uniquely defined, but here we +choose to define a Riesz map using the \(L^2\) inner product.

+

Being precise the function we visualize, \(g^\sharp \in V\), is defined such that

+
+\[\forall \zeta \in V \qquad \left. \frac{d \hat{J} \left( m + \alpha \zeta \right)}{d \alpha} \right|_{\alpha = 0} = \int_\Omega \zeta g^\sharp,\]
+

where \(\alpha\) is a scalar.

+
+
[2]:
+
+
+
reset_manager()
+
+start_manager()
+u, J = forward(m)
+stop_manager()
+
+print(f"{J.value=}")
+plot_output(u, title="u")
+
+
+dJdm = compute_gradient(J, m)
+plot_output(dJdm.riesz_representation("L2"), title=r"$g^\sharp$")
+
+
+
+
+
+
+
+
+J.value=9.711060188691798e-06
+
+
+
+
+
+
+../_images/examples_4_riesz_maps_3_1.png +
+
+
+
+
+
+../_images/examples_4_riesz_maps_3_2.png +
+
+
+
+

Hessian action

+

Next we seek to differentiate \(\hat{J}\) twice with respect to \(m\). However the second derivative defines a bilinear operator. This can be represented as a matrix – a Hessian matrix – but the number of elements in this matrix is equal to the square of the number of degrees of freedom for \(m\).

+

Instead of computing the full second derivative we can compute its action on a given direction \(\zeta \in V\). The degrees of freedom associated with the result define the action of the Hessian matrix on a vector – specifically the action of the Hessian matrix on the vector consisting of the degrees of freedom for \(\zeta\).

+

We do this in two stages. First we compute a directional derivative,

+
+\[\left. \frac{d \hat{J} \left( m + \alpha \zeta \right)}{d \alpha} \right|_{\alpha=0},\]
+

computed using the tangent-linear method. We consider the case where \(\zeta = 1\).

+
+
[3]:
+
+
+
reset_manager()
+
+zeta = Function(space).interpolate(Constant(1.0))
+configure_tlm((m, zeta))
+
+start_manager()
+u, J = forward(m)
+stop_manager()
+
+print(f"{J.value=}")
+plot_output(u, title="u")
+
+dJdm_zeta = var_tlm(J, (m, zeta))
+print(f"{dJdm_zeta.value=}")
+
+
+
+
+
+
+
+
+J.value=9.711060188691798e-06
+dJdm_zeta.value=1.1580693564843862e-06
+
+
+
+
+
+
+../_images/examples_4_riesz_maps_5_1.png +
+
+

Next we compute the derivative of this derivative using the adjoint method.

+

Again we need to remember that the result is a member of the dual space \(V^*\), and is not a function, so we again use a Riesz map to visualize it. Here we use the same Riesz map as before, defined using the \(L^2\) inner product.

+

Being precise the function we visualize, \(h^\sharp \in V\), is defined such that

+
+\[\forall \chi \in V \qquad \left. \frac{\partial^2 \hat{J} \left( m + \alpha \zeta + \beta \chi \right)}{\partial \beta \partial \alpha} \right|_{\alpha = 0, \beta = 0} = \int_\Omega \chi h^\sharp,\]
+

where \(\zeta \in V\) defines the direction on which the action is computed, and \(\alpha\) and \(\beta\) are scalars.

+
+
[4]:
+
+
+
d2Jdm2_zeta = compute_gradient(dJdm_zeta, m)
+plot_output(d2Jdm2_zeta.riesz_representation("L2"), title=r"$h^\sharp$")
+
+
+
+
+
+
+
+../_images/examples_4_riesz_maps_7_0.png +
+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/examples/4_riesz_maps.ipynb b/examples/4_riesz_maps.ipynb new file mode 100644 index 0000000..8febe7d --- /dev/null +++ b/examples/4_riesz_maps.ipynb @@ -0,0 +1,315 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "58eda45b-089d-4258-88fa-6ac36c24ed78", + "metadata": {}, + "source": [ + "# Visualizing derivatives\n", + "\n", + "When we differentiate a functional with respect to a finite element discretized function in some discrete function space $V$, we do not obtain a function in $V$, but instead obtain an element of the associated dual space $V^*$. This notebook describes how a Riesz map can be used to construct a function, associated with the derivative, which can be visualized.\n", + "\n", + "This example makes use of tlm_adjoint with the [Firedrake](https://firedrakeproject.org/) backend, and we assume real spaces and a real build of Firedrake throughout.\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the solution $u \\in V$ of a discretization of the Poisson equation subject to homogeneous Dirichlet boundary conditions,\n", + "\n", + "$$\\forall \\zeta \\in V_0 \\qquad \\int_\\Omega \\nabla \\zeta \\cdot \\nabla u = -\\int_\\Omega \\zeta m,$$\n", + "\n", + "where $V$ is a real $P_1$ continuous finite element space defining functions on the domain $\\Omega = \\left( 0, 1 \\right)^2$, with $m \\in V$, and where $V_0$ consists of the functions in $V$ which have zero trace. We define a functional\n", + "\n", + "$$J \\left( u \\right) = \\int_\\Omega \\left( 1 - x \\right)^4 u^2,$$\n", + "\n", + "where $x$ and $y$ denote Cartesian coordinates in $\\mathbb{R}^2$.\n", + "\n", + "We first solve the forward problem for\n", + "\n", + "$$m = \\mathcal{I} \\left[ \\sin \\left( \\pi x \\right) \\sin \\left( 2 \\pi y \\right) \\right],$$\n", + "\n", + "where $\\mathcal{I}$ maps to an element of $V$ through interpolation at mesh vertices." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "f977ffe4-03f5-426b-a437-aa012b4a125a", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:46:36.535205Z", + "iopub.status.busy": "2023-12-20T16:46:36.534472Z", + "iopub.status.idle": "2023-12-20T16:46:41.546166Z", + "shell.execute_reply": "2023-12-20T16:46:41.545387Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J.value=9.711060188691798e-06\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGzCAYAAACPa3XZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABz/0lEQVR4nO2de1xUdf7/X8wAA6JAKhdx8doFLRPDxDH7uiZF6Vpu9g0v621NaxMraUstEy9tZmvmVhRbadamafY1t8wfZZi5JWki7JYp5XrB0kHNBQRlgJnz+8OdkYG5nMvnnPM5M+/n4zGP8sznnPOZw5nzec77/bmECYIggCAIgiAIQiNMeleAIAiCIIjQguSDIAiCIAhNIfkgCIIgCEJTSD4IgiAIgtAUkg+CIAiCIDSF5IMgCIIgCE0h+SAIgiAIQlNIPgiCIAiC0BSSD4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgiAIgiA0heSDIHRi6tSp6NGjR5vtixYtQlhYmPYVIgiC0AiSD4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgtAJX51KHQ6HxjUhCILQFpIPgtCJK664AtXV1W22Hz9+XPvKEARBaAjJB0HoRO/evVFTU4N//etf7m2nTp3CBx98oGOtCIIg1CdMEARB70oQRCjyyy+/oHv37khKSsJDDz2ECxcu4NVXX0VCQgL2798P+moSBBGsUOSDIHSiU6dO+OCDD9CuXTs8/vjjeOutt7Bs2TKMHj1a76oRBEGoCkU+CIIgCILQFIp8EARBEAShKSQfBEEQBEFoCskHQRAEQRCaIlk+du3ahdGjRyMlJQVhYWHYsmVLwH127tyJG264ARaLBVdeeSXWrl0ro6oEQRAEQQQDkuWjvr4e/fv3R0FBgajyR48exahRozB8+HCUl5fjkUcewX333YdPPvlEcmUJgiAIgjA+ika7hIWF4YMPPsCYMWN8lpk7dy4+/vhjfPfdd+5t48aNQ3V1NYqKiuSemiAIgiAIgxKu9glKSkqQlZXlsS07OxuPPPKIz33sdjvsdrv7306nE+fOnUOnTp18rodBEARB8IkgCDh//jxSUlJgMqnX1bChoQGNjY1MjhUZGYmoqCgmxyLaorp82Gw2JCUleWxLSkpCbW0tLl68iOjo6Db7LFu2DIsXL1a7agRBEISGnDhxAr/61a9UOXZDQwN6dI9B1Wknk+MlJyfj6NGjJCAqobp8yGH+/PnIy8tz/7umpgbdunVD6uIFMNGNYDickTSPndExNVLEkZCPs6EBJ/KfRocOHVQ7R2NjI6pOO1GxLwUdOiiLrpw/78Q1A0+isbGR5EMlVJeP5ORkVFVVeWyrqqpCbGys16gHAFgsFlgsljbbTVFRXMqH00KNqz9oPHcQ4P2rqjsmO0mRkdAibd6hgwmxCuWDUB/V5cNqtWLbtm0e27Zv3w6r1ar2qTWBxIMg9IO37x/JEEGIQ7Ie1tXVoby8HOXl5QAuDaUtLy9HZWUlgEspk8mTJ7vLP/DAAzhy5Agef/xxHDp0CK+88gree+89zJkzh80n0BHeHnwEQeiL0yLQc4EgRCA58rFv3z4MHz7c/W9X34wpU6Zg7dq1OHXqlFtEAKBnz574+OOPMWfOHPzlL3/Br371K7zxxhvIzs5WXHn6khuUKIfeNSCk0GDWuwaGw/VsokgIQXhHsnz8+te/hr+pQbzNXvrrX/8aZWVlUk9FsIQafEIuvN07BpIhvX8gkfwQvMLlaBeCEbw1GgTBgtb3tYFkRGucFoEEhOASko9gg4SDCDVa3vMkIm0gASF4hORDTUgERGG20HXiGYfdQA26Xt85zqWHBITgDZIPtSDxaANJhjHx9XczlJSojQGiLyQgBE+QfKiBAcSDRIBQip73ENfi4/r+cyghJCAEL5B8sIZj8SDhIIKFlvcytyLCqYQEu4CcdTTC7lA4vbqDzfowhG9IPljCoXgYUTgsUWxWpSSUY2+I1LsKAWl9j3MnIxxKSLALCME/JB+s4Ew8jCIdJBp84+/vw6uYcCsjnEkICQihJyQfLJAhHkaRA1aQZAQfvv6mvEkJdykajiSEBITQC5IPJciMdvAoHiQHBCu0upfkSI6a3z3JYsOJhJCAEHoQevKhc3qEJ/EwknDEWIxT12Ci3s5XFKMlLe9fHqItru+2ESWEBITQmtCTDx3hQTx4Fw6SDL7w9ffgTUpa39d6yohiCWGFRJkhASG0hORDI/QWD96kgyTD2Pj7+/EgJjxERWRLCCuiHCQgBLeQfGiAXuLBSjhIFPzT3mJnerw6u4Xp8bSGt2iJ3lERXTu8koAQnELyoTJai0eoCAfrBp8nWH42nkSG9T0lV2b0lBFdoiEkIASHkHyohFzp0DM9wqNwBLNkaIFa148HqWl9v7KSETlIFRjNJYQEhOAMkg8VMJJ48CIcJBnGwtffS08pYSUjcnB9d7mWEBIQgiNIPhhjFPHQUzqCTTTiLQ2qHLfaHqXKcdXE399WazFpeY9rJSLcS0gICMhZpwUNTmVru9Q5aW0XtSH5YAjv4sFKOHiUB7UEQE9Yfya9ZUZPMdE6KsK1hISAgBD8Q/LBCF7FIxiFIxhFQwtYXTc1JEbu/SVXWrSKinArISQghM6QfDCAR/FgIR16CwdJBp/4+7toHV1pfY/KkREtoiJcSggJCKEjJB8KkSMecqVDi34aeghHMEhGJ0u9qsf/xR6j6vFZobeYtLx/WURFALYywp2EkIAQOkHyIROtox1qi4cW0sGTZKgtC6xhWV+9RMbX318tKWERFQHEf/ekSApXEkICQugAyUcAWE4Sxpt4sBIOvaXCaCKhN0qvF2t5kXr/yJUVFlERf7i+p3IkBJAmIswlhASE0BiSD43gRTyMLBwkGXzg7++gRVSl9b0nR0ZYRUW8IUdCAHnREH8/jmQtbEcCQmiEssHQhCh4EI/2Frti8Yi3NLhfatLJUu/1xTsdI+o1efGMHn87Fvel6/vB4nviIsbSKOs7bIlqZNIZXVbUVsbKuk6LIP08QUhBQQF69OiBqKgoZGZmYu/evX7Lb9q0CWlpaYiKikK/fv2wbds2j/c3b96M2267DZ06dUJYWBjKy8t9HksQBNxxxx0ICwvDli1b3NvXrl2LsLAwr6/Tp08DAL788kvcdNNN6NSpE6Kjo5GWloYXXnhB9nUQC0U+VEZP8WDxEFVTNPQWCt4bcl+wrPe5Jm36f2gVLWERFQHYpmi0jIS0xmxxUAREAzZu3Ii8vDwUFhYiMzMTq1atQnZ2NioqKpCYmNim/O7duzF+/HgsW7YMv/nNb7B+/XqMGTMG+/fvx3XXXQcAqK+vx9ChQ3HvvfdixowZfs+/atUqhIW1vf45OTm4/fbbPbZNnToVDQ0N7nrFxMQgNzcX119/PWJiYvDll1/i/vvvR0xMDGbOnCn3kgQkTBAE7rW1trYWcXFx6L78aZiiLj9MZBm3RLNX0udDjngolQ4e0ypaS4ZRpcIoaCUsaqRwWHRuVSojckfPKJEQWX1DJAoIAJ8C4mxowPG5C1BTU4PY2FjpdRGBq53Y8V0q2ndQOMPpeSduue4ETpw44VFfi8UCi6Xt3z8zMxM33ngjXn75ZQCA0+lEamoqZs+ejXnz5rUpn5OTg/r6emzdutW9bfDgwUhPT0dhYaFH2WPHjqFnz54oKytDenp6m2OVl5fjN7/5Dfbt24cuXbrggw8+wJgxY7x+rjNnzqBr165YvXo1Jk2a5PPz33333YiJicHf/vY3n2WUQpEPFdA62sGbcGglGyQZ+uDrurOWEjWiJS3vcb2iInpEQkIpAvKLIwYXHco64l5wXPrRmZqa6rE9Pz8fixYt8tjW2NiI0tJSzJ8/373NZDIhKysLJSUlXo9fUlKCvLw8j23Z2dkeKRNR9bxwARMmTEBBQQGSk5MDln/77bfRrl073HPPPT7LlJWVYffu3Xj66acl1UUqoSUfMvKZUjGaeBhFOIwoGp0j6lQ57tmm9qocVylaSQngeb8ZVUS0lhCtBCSY8Bb5aM3Zs2fhcDiQlJTksT0pKQmHDh3yelybzea1vM1mk1S/OXPmYMiQIbjrrrtElV+9ejUmTJiA6OjoNu/96le/wpkzZ9Dc3IxFixbhvvvuk1QXqYSWfEhEasrFSOLBu3TwJBtqSYRcpNSHB1Fp+bdUS0SUpmhc3wclaZn2Fruuq/oS7ImNjVUtTaSUDz/8EDt27EBZWZmo8iUlJTh48KDPVMo//vEP1NXV4euvv8a8efNw5ZVXYvz48Syr7IFh5UNyfw+V+3poKR7BKB1aygZvMqEmUj+r2rLS+u/MSkZc9yMLCdF6ivgYS6OsfiCWqEZF/UAINnTu3BlmsxlVVVUe26uqqnymQpKTkyWV98aOHTvw73//G/Hx8R7bx44di5tvvhk7d+702P7GG28gPT0dGRkZXo/Xs2dPAEC/fv1QVVWFRYsWqSofhhxqy9vQLrkdS7UUDxZDZFkNndR6+GjniDqPF+Gb1tdK7evF+h5gcX8q+a7I/X7KjX5qtSI24ZvIyEhkZGSguLjYvc3pdKK4uBhWq9XrPlar1aM8AGzfvt1neW/MmzcP//rXv1BeXu5+AcALL7yAN99806NsXV0d3nvvPUyfPl3UsZ1OJ+x2dWe9NmzkQxIqRj20HNEi9cHGW5RDq+gGCQZbWl5PNSMjLKMieqZi5KZf5EZACP3Jy8vDlClTMHDgQAwaNAirVq1CfX09pk2bBgCYPHkyunbtimXLlgEAHn74YQwbNgzPP/88Ro0ahQ0bNmDfvn147bXX3Mc8d+4cKisrcfLkSQBARUUFgEtRk5av1nTr1s0dxXCxceNGNDc343e/+12b8gUFBejWrRvS0tIAALt27cKKFSvw0EMPMbgyvgl++dCgk6kUjCQeLKRDC+Eg2dAOrUQEUN5XRM9UjJb9Pyj9oj85OTk4c+YMFi5cCJvNhvT0dBQVFbk7lVZWVsJkupxoGDJkCNavX48FCxbgiSeewFVXXYUtW7a45/gALvXpcMkLAIwbNw6A9xE3gVi9ejXuvvvuNika4FKUY/78+Th69CjCw8PRu3dvLF++HPfff7+kc0jFkPN8SEq7cNLXQ8tOpSzSK0pRUzqMKBudw8/L2u9scwfGNVEPrTq3yo2IsJg3RKqEyBEQNecB0WK+D29DbbWc52PTP9PQroPCobbnHfjf/odUrW+oE9yRDxIP0fCaWuFRNOSKhJrn4kFStE7PSJUQXkbFBELNDqiyhtsShAoEt3xwAO/iwZt06CUbWgqFGkitv9qy0vrvqIaMdIyolyUggHapGBp+SxDeCV754CDqofVoFinwlFrRUjiMLhms0DqiolZURI6AAGwkRGwURI6AUPSDCHaCVz4MiBbiwYt0aCEcJBrKaXkN1RARQLmMyE3DAOxSMUYTEILQm+CUDwNGPYwgHrxLB8mGuriuL+uUDauoiN5REJ4EJJSpao5DdJOypu1iczOj2hC+CE75MBha9O9QIh68SodRZSM5ojpgGVtTvOr1kIsa0RD3sRVGRZRGQQBla8VoPTuqLwJFPyj1QuhNyMuHnlEPnvt38Nifg0fZECMSahyXFzlRKxriPr7MqIjcKAigLBUTSEAo+kEQlwhp+ZAqHizhVTx4kg49ZUMtqWCF2PppJSlqRkPc55AoIkoFBJAXBeFFQKjvB8EzIS0fUmG1jgKP4sFLakVL4eBdMFgg5TOyEhWtRESsgADyJyZjNTS3NTwMwaXUC6EnISsfvE0o5g+15+9QKh5GkI5QEA2l+LpGSqREi/4hakdBAOmpGDU6oFL0gwgmQlY+9EKLNVq0Eg+epYNkgx0tryUrEQHYDt3lMQrCi4D4g6IfhF6EpHwYJerBq3jwKB1Glo0E8+XrecahzfoocnFdZxYpGpZREbECArCJggDiJISnETAEwRMhJx9G6WTKm3iwGrXCQjp4FY2WEqHWMXiRE1bREBcsoiJapmEA8akY1h1QpUY/KPVC8EjIyYdUtI56aLEwnBTxIOm4DAu5ULsOesgJaxEBlEVFtErDAOw6pOqZfgmYeolySF7ZliACQfLhB1ajW8QSjOKhVDq0FA4e5EIpYj6DmoLCMi3jQk5UhLcoiN79Pyj6QfAGyYdGyB1eGwgexYP3KEcwSIYSpHx+uaKiRjTERefw86KjIVpHQbQWEFZQx1NCa0g+fCAl6sEi5cLT5GFyxYNH6Qh10VAKi86wakZDxEZBpHRGBZRJiD9Yd0DVbPZTA6VefmnugKhmZU1bA63tojokHxrAOurBm3jwJB0kG+rhurY8RUPERkGkpGEAZYvUBer/oVcHVEq9EDxB8uEFraMeUuBJPHiRDh6FI8HEVjjPOPWdDbMlvEVD1EjDAPKjIHoICAuo4ymhJSQfHCAm5aL24nBSxIMH6eBBOFgLhpJz6SUnLKMhgHwZkZqGAaRFQQBpEqJkcToXUgSEFp8jjAbJRytYj3BhkXIJFvEwYpRDS8FQgr96aiEmrCZKU5qaUSsKAkhPxWjdAVWMgARKvVDHU0IrSD4UoEdHUzHwJh48S4dR5EIJYj4jS0FRGg1xITcqolZnVIB9h1Q9BEQRlHohGEHyoSJ6RD14Eg/epCMUREMuakROWE8bL7WfiFqdUQHxEsKi/wdrqOMpwQMkHzLhMerBi3jwIh0kG2xoeR15EBEpEqJmFAQQl4rRugOq0ugHdTwltIDkI0jgQTx4kA6ehSPBLL1BOOPQdjRVIFzXV0mahmVahpcoCAsBCQRX6Rc/OC0CTPYwXc5NGAeSDxmIiXpomXIJBvEIFumQIxhKjqeXnPASDZEaBdGzMypPM6BSx1NCb0g+dIJFykXNlWm1EA8jplZYy4VSeJATliKiREL0TsPwJiCqRj8o9UIoxCRnp4KCAvTo0QNRUVHIzMzE3r17/ZZftWoVrrnmGkRHRyM1NRVz5sxBQwP7UR5awFvUQyw8iUeCuY5JpEMt8UgwR/p8GQ2tP4vSv4vr3pBzf0i5F8Xe450j6iR9d6T+IPCGGiPgvKHmwplOi6DasYngQHLkY+PGjcjLy0NhYSEyMzOxatUqZGdno6KiAomJiW3Kr1+/HvPmzcOaNWswZMgQ/PDDD5g6dSrCwsKwcuVKJh/CaGj1cHHBi3jwFOkwokiwRsw1kBs50SsawkNn1EAREKPMgGrUjqe/NMXA0hSh6Bj2piZGtSF8ITnysXLlSsyYMQPTpk1D3759UVhYiHbt2mHNmjVey+/evRs33XQTJkyYgB49euC2227D+PHjA0ZLjIpaq9e2RuwvLB7Eg4dIh9EjGHrB4nrpEQ3ROwoS6PspJrIZ6EeKmGeN1ss/tISiH4Q/JMlHY2MjSktLkZWVdfkAJhOysrJQUlLidZ8hQ4agtLTULRtHjhzBtm3bMHLkSJ/nsdvtqK2t9XiFEqxSLryIhxJYpFdINpTDg4Rcqoe+AgLIX/W5NSy+50p/7KiZeiEIf0hKu5w9exYOhwNJSUke25OSknDo0CGv+0yYMAFnz57F0KFDIQgCmpub8cADD+CJJ57weZ5ly5Zh8eLFUqpGEARBEIRBkNXhVAo7d+7EM888g1deeQX79+/H5s2b8fHHH2Pp0qU+95k/fz5qamrcrxMnTjCrDw0fIwiCIAh9kRT56Ny5M8xmM6qqqjy2V1VVITk52es+Tz31FCZNmoT77rsPANCvXz/U19dj5syZePLJJ2EytfUfi8UCi4WfJcQJguCHBHMdk+naCYLQD0mRj8jISGRkZKC4uNi9zel0ori4GFar1es+Fy5caCMYZvOl6IMgUIckgiAIggg1JA+1zcvLw5QpUzBw4EAMGjQIq1atQn19PaZNmwYAmDx5Mrp27Yply5YBAEaPHo2VK1diwIAByMzMxOHDh/HUU09h9OjRbgkhCIIgCCJ0kNznIycnBytWrMDChQuRnp6O8vJyFBUVuTuhVlZW4tSpU+7yCxYswKOPPooFCxagb9++mD59OrKzs/HXv/6V3acgvCJ1kSyCIAhCHlIn39y0aRPS0tIQFRWFfv36Ydu2bR7vb968Gbfddhs6deqEsLAwlJeXtzlGQ0MDZs2ahU6dOqF9+/YYO3Zsm24RlZWVGDVqFNq1a4fExEQ89thjaG5u9jjPrbfeioSEBMTGxsJqteKTTz6RfyFEIqvDaW5uLo4fPw673Y49e/YgMzPT/d7OnTuxdu1a97/Dw8ORn5+Pw4cP4+LFi6isrERBQQHi4+OV1j1oUboAFUEQBKEdrsk38/PzsX//fvTv3x/Z2dk4ffq01/K7d+/G+PHjMX36dJSVlWHMmDEYM2YMvvvuO3eZ+vp6DB06FMuXL/d53jlz5uCjjz7Cpk2b8MUXX+DkyZO4++673e87HA6MGjUKjY2N2L17N9566y2sXbsWCxcudJfZtWsXbr31Vmzbtg2lpaUYPnw4Ro8ejbKyMgZXxjdhggE6XtTW1iIuLg7dlz8NU1RU4Mlrohx+3zZbfL8faNx7oEl7xIy7FzPDaaA5AFhPMsbrPB+A8llNaZ4PdrBYK0bJiriX6yEuqid21VtA3Eyn7rIiooqB1nkBAv/QCLTOC4CAM50GWt/F3wJzAUcHBpjh1LW6rbOhAcfnLkBNTQ1iY2P9H1MmrnbioS/vgqW9whlO65rw4tC/i65vZmYmbrzxRrz88ssALvWFTE1NxezZszFv3rw25XNyclBfX4+tW7e6tw0ePBjp6ekoLCz0KHvs2DH07NkTZWVlSE9Pd2+vqalBQkIC1q9fj3vuuQcAcOjQIfTp0wclJSUYPHgw/t//+3/4zW9+g5MnT7qzE4WFhZg7dy7OnDmDyEjvf/trr70WOTk5HpLCGtWH2hIEQeiF0tWWidCm9WSXdnvbH0JyJt8sKSnxKA8A2dnZPst7o7S0FE1NTR7HSUtLQ7du3dzHKSkpQb9+/Tzm5srOzkZtbS0OHDjg9bhOpxPnz59Hx44dRddFDrSqLUEQokkwR2qyUi5ByOU/Te0Q2aQs2tnYdOkeT01N9dien5+PRYsWeWyTM/mmzWbzWt5ms4muo81mQ2RkZJsuDC2P4+s8rve8sWLFCtTV1eHee+8VXRc5kHwQBEEQhBdOnDjhkXYJ9vmn1q9fj8WLF+Pvf/+714ViWUJpF8aovdKkWkjJdxOEElitTEwQahMbG+vx8iYfcibfTE5OllTe1zEaGxtRXV3t8zi+zuN6ryUbNmzAfffdh/fee69NSkgNSD4IgiAIQiZyJt+0Wq0e5QFg+/btPst7IyMjAxERER7HqaioQGVlpfs4VqsV3377rceom+3btyM2NhZ9+/Z1b3v33Xcxbdo0vPvuuxg1apToOiiB0i4EQRCcEm9pEDXihdAXqZNvPvzwwxg2bBief/55jBo1Chs2bMC+ffvw2muvuY957tw5VFZW4uTJkwAuiQVwKWKRnJyMuLg4TJ8+HXl5eejYsSNiY2Mxe/ZsWK1WDB48GABw2223oW/fvpg0aRKee+452Gw2LFiwALNmzXJHcdavX48pU6bgL3/5CzIzM919QaKjoxEXF6faNaPIBxH0UAdJQgxShpuLQexweML4SJ18c8iQIVi/fj1ee+019O/fH++//z62bNmC6667zl3mww8/xIABA9yRiHHjxmHAgAEeQ3FfeOEF/OY3v8HYsWPxP//zP0hOTsbmzZvd75vNZmzduhVmsxlWqxW/+93vMHnyZCxZssRd5rXXXkNzczNmzZqFLl26uF8PP/ywatcLoHk+vOJvro9A83wAgef64HGeD0D8w9do83xcqgfN9cEKHub6kLKwnBpzfYidPTjQXB9iJhQMFPmgeT4u4WonJn0+HpHtFY52qWvE34a/q2p9Qx2KfBAEQRAEoSkkH4RkpPySJAiCIIjWkHwQBEEQXgmUoiYIuZB8GBQx60bwgJTcPEEQBBEakHwQBEFIREpHboIg2kLzfBAEQRBBw7nGGEQEGOETiKZGZaviEoGhyAdBEARBEJpC8kEQBEEQhKaQfBAEQQQxYiZGJAitIfkgCIIgCEJTSD50gBaKIgiCIEIZkg+CIAiCIDSF5EMigRZpIgiCIAjCPyQfBEEQBEFoCslHkCN26W+CILSnk6Ve7yoQhC6QfBDcc8Zp0bsKBMEt7S12vatAEJIh+SAIwnAkmGltFYIwMrS2C0EQBBE01NgtCA9XFi1ttocxqg3hC4p8EARBEAShKSQfBEEQBEFoCskHQRAEQRCaQvKhAnV25aMzfrHHMKiJNM42dxBd1tYUL7rsGUd7nHEoG/KrdMTLGQctrsUCFteRxeglpfcTL7D4nrN43hCE1hiyw6nJHganRfBdoMEMRDl8vu2wm2G2+H5fC6rtUYi3NPgt84s9xu88AOeaYtAxIvA8Aa65PjpHBB4h4BKQzuHnA5a1NcUjOaI6YDkXrgZD7kiFlo1Wgkn68EJvDWeCmWasFYNRpUOKJEuRb5o/hyCUYUj5UBt7QyQsUb4ftvX2yIDLVNfZLQHH32spIMClB6YYAQEuPYjFCggAWRICKBcRORLiWRfff8dQFRPWUSK9Ih16i8e5JuVRDVqEkghWDCsfSqMfWsBKQAIhVUAA8VEQMQICSI+CuNA7GuL32EEqJlqloPRMr+gtHmLQKuVC61ERPGJY+VBKoNQLi+gHwEZAAkU/AGkCAoiPgkhNwwDSoiAueIqGiDoXp2LCQ98Wvft08CAeFPUgCP+ErHywgKWABEItAQH4iYK44DkaIur8AQRAiZzwIBfeYDUFvtKOpEYRD4p6EKGOoeXDSB1PAwkIi/4fwOUHHw9RECUCAhgvGiIWXgVCDnpHOVpiFPEgCMLg8qE2gVIvgPjohxhYCQggXULU7IwKyEvFtMTo0ZBggpcohwsp0gEYQzxYpFwo6kHwTPDLhwYdT7Xs/wGIFxBA3c6ogLgoCMBORFhGQ3xBcuIJ61WFWUiHVOFwwYN40Nwe6lLfaIFZ4douQRSc5Jbgl48AKO146kJrAZECD1GQlrRuOOTKiNJoiM/j+mlsg1VMWAuG13PoKB0AH+IhBop6EKFAyMuHHmg1AqY1UiRErc6o3lAySgZgEw0RfS6DRk20kAuf59ZZOgB+xIOiHgRxidCQD4UdT1lHPwDtRsB4Q42JyaSmYbzBIjWjVjRE9Pl1iJroKRb+4EE6gOASD4IIFkJDPjSEpYCw7v/REl6jIC6UioiW0RCxBBITXiVCCnp1IvWGFOkA+BAPrVMu9gZKzxD6EDrywcGMp97QU0AAfqMgLWGVluFFQrxhVPFgucAbC+EApEsHwId4iIFSLkSwEDryEQBWqReA7fBbQBsBAcRHQdTsjOoPltEQFzwLCY+osZqsntIB8CMeFPUgQgmSD5Vg3f9DbQEB1F0lF2AXCQGUR0NciG1MQ1FS1F62Xo/USpv9OR5O2xqKehDBhEnvCmhKg9nv2w67//el/lKQ8itEzINFzC8jpQ89KQ9YqYtsnW3u4PFiga0p3v1SkzOO9n5fRkSvz8Ti78XiHjKSeLCGoh7sKSgoQI8ePRAVFYXMzEzs3bvXb/lNmzYhLS0NUVFR6NevH7Zt2+bxviAIWLhwIbp06YLo6GhkZWXhxx9/9CjTo0cPhIWFebyeffZZ9/sVFRUYPnw4kpKSEBUVhV69emHBggVoamqSVBc1CC354ByWAqLkAXiuKUb0w/ZsU3vZK32qJSJ6wKOY8CZMvEgHwJ94aJlyIfFgz8aNG5GXl4f8/Hzs378f/fv3R3Z2Nk6fPu21/O7duzF+/HhMnz4dZWVlGDNmDMaMGYPvvvvOXea5557Diy++iMLCQuzZswcxMTHIzs5GQ4NnBHzJkiU4deqU+zV79mz3exEREZg8eTI+/fRTVFRUYNWqVXj99deRn58vqS5qECYIgp/FUfigtrYWcXFx6L78aZiiPL+kftd28UWAjqf++n6I7ffREqn9PwKlYKROQKYkFSNljRixfUECHodhekZpWkZtAqVzjBpVAdj15QCUp1cA6ZE6QH3xECsdLBeRkyoffiPCAaLJJnsYAMDZ0IDjcxegpqYGsbGxks4vFlc7MeD9PJjbKZzh9IIdZfesFF3fzMxM3HjjjXj55ZcBAE6nE6mpqZg9ezbmzZvXpnxOTg7q6+uxdetW97bBgwcjPT0dhYWFEAQBKSkpePTRR/HHP/4RAFBTU4OkpCSsXbsW48aNA3Ap8vHII4/gkUceEf3Z8vLy8M033+Af//iHqLqoBUU+JCLnV4PU2QYDPWik/krSMgrCApbpGT2jIWLgLTqhhJYpMJadSFmkV4wsHmLQRTxCgNraWo+X3d72h2FjYyNKS0uRlZXl3mYymZCVlYWSkhKvxy0pKfEoDwDZ2dnu8kePHoXNZvMoExcXh8zMzDbHfPbZZ9GpUycMGDAAf/7zn9Hc3Ozz8xw+fBhFRUUYNmyY6LqoBXU41Qg9RsC0xPVwVHtIrpTOqGJh0WmV5SJ3BNuoRmtYpeGUyDBP4kEdTaVRb4+E2awsteSwX4qop6amemzPz8/HokWLPLadPXsWDocDSUlJHtuTkpJw6NAhr8e32Wxey9tsNvf7rm2+ygDAQw89hBtuuAEdO3bE7t27MX/+fJw6dQorV6702G/IkCHYv38/7HY7Zs6ciSVLloiui1qQfHiB5bBbuai1BowSCVFrSK4UWIuIN0hO2qJF9IgH6QD4Eg+WUNRDOidOnPBIu1gsfIlgXl6e+/+vv/56REZG4v7778eyZcs86rpx40acP38e//znP/HYY49hxYoVePzxx/WoshuSDw2RGv1QcxE6LSYma9kIqCkiLPuIAOIb2mCUFD1SVHr15/CGmuKhlnSosYgcicclYmNjA/b56Ny5M8xmM6qqqjy2V1VVITk52es+ycnJfsu7/ltVVYUuXbp4lElPT/dZl8zMTDQ3N+PYsWO45ppr3NtdEZy+ffvC4XBg5syZePTRR2E2mwPWRS2oz4cPWA+7dcG6/wcg/6GmZFSM1MmWXHl3Vo2Ex7EZj5oRS+s+Dmr0eWANL3Vm9TdjdU9J6dsEaCcerFIuNMJFPSIjI5GRkYHi4mL3NqfTieLiYlitVq/7WK1Wj/IAsH37dnf5nj17Ijk52aNMbW0t9uzZ4/OYAFBeXg6TyYTExESfZZxOJ5qamuB0OkXVRS0o8qEDakRAlCA3FSMlDdMSNSMiakVD5OCvMZcbNeFVagLBWgz1iHS40EI8xEoHRT34IC8vD1OmTMHAgQMxaNAgrFq1CvX19Zg2bRoAYPLkyejatSuWLVsGAHj44YcxbNgwPP/88xg1ahQ2bNiAffv24bXXXgMAhIWF4ZFHHsHTTz+Nq666Cj179sRTTz2FlJQUjBkzBsCljqJ79uzB8OHD0aFDB5SUlGDOnDn43e9+hyuuuAIAsG7dOkRERKBfv36wWCzYt28f5s+fj5ycHERERIiqi1qQfChAi74fLlisARMILVbJbU3rRoSVjKg1syorjCoRYlErCsUqyiEXnsRDLBT1UJ+cnBycOXMGCxcuhM1mQ3p6OoqKitwdOSsrK2EyXU40DBkyBOvXr8eCBQvwxBNP4KqrrsKWLVtw3XXXucs8/vjjqK+vx8yZM1FdXY2hQ4eiqKgIUf+dbsJisWDDhg1YtGgR7HY7evbsiTlz5nj0AwkPD8fy5cvxww8/QBAEdO/eHbm5uZgzZ46kuqgBzfMRAH8dT13IFRDW83+4UCohgPxRMXIlxBusoyI8SkgwoHa6S88oR0u0WKdFqniIiXxIkQ/JUQ8O5/m4et08JvN8/DDxWVXrG+pQ5IMBciMgaqVfXA8+JRKiJBXDSkBYp2e8NZIkJNLRqm8NL9IBhIZ4SCaAeBCEP0g+AhBo2K1S1Oz/oVcqRm5fEH9okZ7xR6hKitadeAHjSgegf8fSltDQWoJnSD4YwVP/j5YESxSkNWoP421zviCWFD0Ew+P8DEdAsRAOQLv5O+RKhxodTSVBUQ9CISQfDNEq/QJIHwGjl4S0bAyCRUT8IaYhZykoeouDHNQYbq2ndAD8iQdFPQjeIfkQgZTUC88CAug/KsaFFiIC6C8j3jCiMMhFDdFoid7SAWgnHrpHO1xQ1INgAMlHCKJnKsZF60ZDbRnhUUKCEbVlw4Ue/Tm8EQziEWxRj8aGCJhMyq6Xs8HJqDaEL2TNcFpQUIAePXogKioKmZmZ2Lt3r9/y1dXVmDVrFrp06QKLxYKrr74a27Ztk1VhvZDyBdVq9lNAWUe1anuU4imflcyS2hLXDJOsfsm2Rs0ZVkOZltdViygHi3uE1T0r9btTZ7doIh56Dq0lCLFIjnxs3LgReXl5KCwsRGZmJlatWoXs7GxUVFR4ndK1sbERt956KxITE/H++++ja9euOH78OOLj41nUn1t4T7+0RM9UjDfUTs8EaiQpSuIdPcSNpYyyEA6Av/4dvOKa44MgvCFZPlauXIkZM2a4p40tLCzExx9/jDVr1mDevHltyq9Zswbnzp3D7t273dO59ujRQ1mtDYLRBATQNxXjDa3SMy0R28gGq6ToGR1SI/KlV3rFhZZpFop6EEZBknw0NjaitLQU8+fPd28zmUzIyspCSUmJ130+/PBDWK1WzJo1C3//+9+RkJCACRMmYO7cuTCbvd/MdrsddvvlhrS2tlZKNUMW10MuGCXEhRadVsUippHWWlCMllZSK82mZ5SjJbyKh2QkigdFPYhASJKPs2fPwuFwuOerd5GUlIRDhw553efIkSPYsWMHJk6ciG3btuHw4cN48MEH0dTUhPz8fK/7LFu2DIsXLxZVJ5M9TPoU6w1mSVOsK8H1QJAaAXE9gKRGQAD+JMQFaxnx1nDpLSStMZoMqIVaktEaXqQDCCLxIAgVUH20i9PpRGJiIl577TWYzWZkZGTg559/xp///Gef8jF//nyPxXFqa2uRmprq8xxaCYiS2U71lBAXcmSk9UOYpYy4UDNN44I3KQlWtJKMlvAkHID2/TtIPAgjIkk+OnfuDLPZjKqqKo/tVVVVSE5O9rpPly5dEBER4ZFi6dOnD2w2GxobGxEZ2faLY7FYYLFI+wIbQUAAfSTERcuHotyoCEsZcaGnlPiDhMU3RpaMlrASDhdaigdJB2FkJMlHZGQkMjIyUFxcjDFjxgC4FNkoLi5Gbm6u131uuukmrF+/Hk6n072k8A8//IAuXbp4FQ8laCkgrZEqJHpKCMAmKgJ4PrxZiEhLtJASfyhtYHmTFz2EQQ5qSEZr9EqreIPEgwhFJKdd8vLyMGXKFAwcOBCDBg3CqlWrUF9f7x79MnnyZHTt2hXLli0DAPzhD3/Ayy+/jIcffhizZ8/Gjz/+iGeeeQYPPfQQ20/yX/TqA+ISEq0lBFAuIgB/KRp/+GuctBITMRilsdcLLSSjJXqnVbxB4kGEKpLlIycnB2fOnMHChQths9mQnp6OoqIidyfUyspKd4QDAFJTU/HJJ59gzpw5uP7669G1a1c8/PDDmDt3LrtP0Qo9O6G2jIpIEZGWDxS9oiEtUSNF4wuWkqJ3tIRoi9aS0RIehcMFiQcRysjqcJqbm+szzbJz584226xWK77++ms5p5IND6Ng9IyG+EKvjqv+8NVAaCElYiF5CYyektEantIq3iDxIEKdoF7bhQcBAbSXEH+wSNeoLSMutJASsQSLvPAkCFJg3THUH2pKB0DioTaORjMEH3NIicXZSBOqqU1QywegQECAoJYQoO1DkIWMqCUiLeFJSsRi1EZfa7SUjJaoLRwutBSPYFswjggugl4+AJkCAqg2GRlvEuKChYxoFRXxhr+Gi2cxCUX0koyW6DlaRSwkHkSwEhLyAfAnIIByCQHUExFAnRSNFFiKC6vGjiRGGjxIRmtYSIcWC77JEQ+SDsIohIx8AHwKCCB/hAygfjTEBasUjRR8NRJaRlNao7Qx5UVeeJQCNTFClKMlJB5EsBNS8gFcXvCIh46o3mARDZGCXGlhPc+IFHiUErGEWqPPAq36Y/hC62XtSTyIUCDk5MMFLyNhfCFXQqTS+kEnR0b0iIp4w8hSQugvGS3RWjhckHgQoULIygfAv4AA2kmICzVkxB96pm+UQEIjH54kozV6SQdA4kGEFiEtHwBDAWmQ+BCQuZ6MVhLiQu3OrXIe9npFVlqitAHlRV54FgGt0FM4XJB4EKFGyMsHoHAuELnInEtELwkB2ERFWOCrseBBSsRCjb6+sBIOvSb/IvEgjA7Jx3+RPRJGKQolBNBHRADthvyKJRikhPCEh6hEa/SebZTEgwgGSD5aoJuAAJ6RFANFQ1zwEhXxhpoNGImNcngUDG/oLR0AiQcRPJB8tEJXAXHBIBrCCrlCo9aDmiepAdg0nDwIjNoCEGNpNIxktIQH4XChm3goTTFrTYMZCGOUFidUg+TDC1wICKDaGjNSaP3A0zO6AvhvDHgTE7EYsVGWipE+I0/C4YKZeFCjSnACyYcPuBEQgAsJccGbjLTEV6NhVCkh2iJFDKT+3XmUDkCmeJBkEJxD8uEHrgQE4EpCXPDQ8TUQoZICCiZY/M20lAmu+mKQeBAGgOQjANwJCMClhAB8R0XUINikhsXnCZZogxi4Eg4XJB6EQSD5EIHs9WDURsEIGS3Q8uEcTKJj5AbZyHUXA5fC4YLEgzAQJB8S4DIK4kLLB4/BRCeYxITQB66lAyDxIAwHyYdEuBYQrWj9oONQRlrCsuEgkdEP7gVAL0g8CANi0rsCRsSVhmF5vEAvrmkwX34FOQ67mYsXD59R6/MTXgiB75xRKCgoQI8ePRAVFYXMzEzs3bvXb/lNmzYhLS0NUVFR6NevH7Zt2+bxviAIWLhwIbp06YLo6GhkZWXhxx9/9Chz7tw5TJw4EbGxsYiPj8f06dNRV1fnfv/YsWMICwtr8/r666/dZZqamrBkyRL07t0bUVFR6N+/P4qKihhcEf+QfMhEjDCIfUk5H/e0FBF6MKoGD2Kj9/lDHvp+ccPGjRuRl5eH/Px87N+/H/3790d2djZOnz7ttfzu3bsxfvx4TJ8+HWVlZRgzZgzGjBmD7777zl3mueeew4svvojCwkLs2bMHMTExyM7ORkNDg7vMxIkTceDAAWzfvh1bt27Frl27MHPmzDbn++yzz3Dq1Cn3KyMjw/3eggUL8Ne//hUvvfQSvv/+ezzwwAP47W9/i7KyMoZXqC1hgiBwn0Oora1FXFwcui9/GqaoKL2rwwUhn/oRC+cpISIIIAloQ+sfSs6GBhyfuwA1NTWIjY1V5ZyudiL1haUwRStrJ5wXG3BizlOi65uZmYkbb7wRL7/88qX9nU6kpqZi9uzZmDdvXpvyOTk5qK+vx9atW93bBg8ejPT0dBQWFkIQBKSkpODRRx/FH//4RwBATU0NkpKSsHbtWowbNw4HDx5E37598c0332DgwIEAgKKiIowcORI//fQTUlJScOzYMfTs2RNlZWVIT0/3WveUlBQ8+eSTmDVrlnvb2LFjER0djXfeeUf0NZMKRT4MimFSMnrTOhIj90UQLaF7IySora31eNnt9jZlGhsbUVpaiqysLPc2k8mErKwslJSUeD1uSUmJR3kAyM7Odpc/evQobDabR5m4uDhkZma6y5SUlCA+Pt4tHgCQlZUFk8mEPXv2eBz7zjvvRGJiIoYOHYoPP/zQ4z273Y6oVj/qo6Oj8eWXX/q8LiygDqdBALdDgYMJIzYySqM+PA3lNuL1DxF4+wFkagyDyaSwTo2X9k9NTfXYnJ+fj0WLFnlsO3v2LBwOB5KSkjy2JyUl4dChQ14Pb7PZvJa32Wzu913b/JVJTEz0eD88PBwdO3Z0l2nfvj2ef/553HTTTTCZTPi///s/jBkzBlu2bMGdd94J4JL0rFy5Ev/zP/+D3r17o7i4GJs3b4bDoe53nuQjiODlIUASxAksG2xq/EMGXp4jPHDixAmPtIvFYtGxNtLp3Lkz8vLy3P++8cYbcfLkSfz5z392y8df/vIXzJgxA2lpaQgLC0Pv3r0xbdo0rFmzRtW6UdqFYI5anW4JgpAGfReVERsb6/HyJh+dO3eG2WxGVVWVx/aqqiokJyd7PW5ycrLf8q7/BirTukNrc3Mzzp075/O8wKX+KYcPH3b/OyEhAVu2bEF9fT2OHz+OQ4cOoX379ujVq5fPY7CA5IPgApajh7QchUQQPEL3sHZERkYiIyMDxcXF7m1OpxPFxcWwWq1e97FarR7lAWD79u3u8j179kRycrJHmdraWuzZs8ddxmq1orq6GqWlpe4yO3bsgNPpRGZmps/6lpeXo0uXLm22R0VFoWvXrmhubsb//d//4a677hLx6eVDaReC8EEoPLwpRaYfoXB/hQp5eXmYMmUKBg4ciEGDBmHVqlWor6/HtGnTAACTJ09G165dsWzZMgDAww8/jGHDhuH555/HqFGjsGHDBuzbtw+vvfYaACAsLAyPPPIInn76aVx11VXo2bMnnnrqKaSkpGDMmDEAgD59+uD222/HjBkzUFhYiKamJuTm5mLcuHFISUkBALz11luIjIzEgAEDAACbN2/GmjVr8MYbb7jrvmfPHvz8889IT0/Hzz//jEWLFsHpdOLxxx9X9ZqRfBBECEMNIEEoJycnB2fOnMHChQths9mQnp6OoqIid4fRyspKmEyXEw1DhgzB+vXrsWDBAjzxxBO46qqrsGXLFlx33XXuMo8//jjq6+sxc+ZMVFdXY+jQoSgqKvIYmbJu3Trk5uZixIgRMJlMGDt2LF588UWPui1duhTHjx9HeHg40tLSsHHjRtxzzz3u9xsaGrBgwQIcOXIE7du3x8iRI/G3v/0N8fHxKl2tS9A8HwRBEISqaDnPB4t2Qov6hjoU+eAA00XqesMrzmin3lUgCIIIOkg+FELiENyEyt+XJIsgCC0h+ZBBqDRIROjA4z1NQkQQwUvIywePD12CIAJ/N0lOCMK4BJ18kEzwi7ntsghBg8NYEx8GBTx910mE+MF00QSToPDeaODn3gpWDC8fPD2AeCSYG3yeMOJ1JmFih+s5RBJCEOIwvHwQbTFiQ0hoj9z7hKTFNyQhBCEOkg8VIQkgghG972sjyA9JCEH4h+RDJfR+QBsFc4M253F4mXNIq3P7qwMhnZbfLd5FRKyEsE4fk/QQvEPyoQIkHm3RuqHn7fy81CHYBKj1d41XGdG6bxpFXgjeIflgDO/iwUMDSOiH0r8/7/JiFBnRCtNFEwkIwSUkHwzhVTxIOAhW+LuXeBQTLb+TvIoOCQjBIyQfjOBRPIwqHeEK690ssRFUej5WSK03b2h5v/EsOjxKCAkIwRskHwzgSTyMJBxqNfq8yIRU1Kq30aXGGy3vc95EhFcJIQEheILkQyFyxINXQTBqo034h+XflUeRaf194kVGeJQQEhCCF0g+AsA6qsGbeJBwEFLwdb/wJCW8RUV4kxASEIIHQk4+9EyR8CQeRpMOX9eOVePC09+GhwZTKv7uJz3FhKeoCE/zkwSzgJjsgDlM4UE4SqUHKyEnH3rBQ+NmBOGQep14uK6sYfGZeBIYnvqyqHW/SL3ePERDgllACP4h+dAAPRtIXoUjGKWBJ3hpZNWk5b2td9rHdb2NJiEkIIRekHyojF6NLAvpIEEgWqN2+ksure93vWTEiBJCAkLoAcmHishpvHmIVISydIQ3CHpXwU1zlNLEtXbwJiV6R0W0khBffdikSgwJCKE1JB8qYTTx4Fk4eBICLVH6uXmQFx6mc9czKqJUQmSf1y5PQABaD4bQBpIPFTCKePAmHKEqGWrB+nrqITNqTOeuh4zoMfxXjoAAFAUhtIHkgzFGEA+9pYMkw5j4+7vxICYsZERLEdFCQkhACF4h+WCE3AZdK/FgJRyhIg68fU4eUij+8HW9tKw3CxlR8n2UKi5aSQgJCMEjJB8M4FU8glk4eKyTmhi1/4eeUqJ1qsP1feZRQuSOpiEBIdSC5EMhPIoHC+ngoXHnoQ7BAotryVIYlNRHTj20nOmUdwkhASF4gORDAVqKh1b9NPRo8EkyjAEPqZXW9ZB7bi1khFcJIQEheIDkww9qNPg8iodWjb8RJSP8Al8P3OZ2Jr2r0AY9O6K2PjcLGWHd6CuVEIB9nYJZQMx2wKz0ILS2i+qQfGgIT+LBSgR4EQreJEEtlH5OreVF62gJCxlRKyqiZFSNGtGQYBYQgn9IPjSCF/FgIQt6CkeoSIZaKLl+LMVFKylRI0XjDalSwEtKhgSE0AuSD5WR27GUpXgYMcpBksEfvv4mWkhJm3PKEAlWKRpvyJUCHiSEBITQA1lPjYKCAvTo0QNRUVHIzMzE3r17Re23YcMGhIWFYcyYMXJOazj0FI/wBsH94uE4Po9/wen1xRPhF53cvXhCj78hi/uy5TFY3d/mBvkTDcqNjrJ4XsiZzt01HTtByEFy5GPjxo3Iy8tDYWEhMjMzsWrVKmRnZ6OiogKJiYk+9zt27Bj++Mc/4uabb1ZUYaOgpXiwFgM1REMPoeCtkWaJ0s/WHK1+w+Hvb65mtERuRINFisaFXpGQlshZT4bWgyG0QvITYOXKlZgxYwamTZuGvn37orCwEO3atcOaNWt87uNwODBx4kQsXrwYvXr1CngOu92O2tpaj5eRMKp4MImU6PEr2ADRAd7QO9qi5n3CIqLBKiqidSSk9bkl7yNzlAdFQQipSLpjGhsbUVpaiqysrMsHMJmQlZWFkpISn/stWbIEiYmJmD59uqjzLFu2DHFxce5XamqqlGrqitHEg1l6hiQj6DGqlLBO0chBqYRo+VwhASG0QNLdcvbsWTgcDiQlJXlsT0pKgs1m87rPl19+idWrV+P1118XfZ758+ejpqbG/Tpx4oSUavpE6TLVgTCSeLCOcrDGKJJhvtis+Ys3tPg7sZIRvUVESR8NLRegVPtZGcqcO3cOEydORGxsLOLj4zF9+nTU1dX53aehoQGzZs1Cp06d0L59e4wdOxZVVVUeZSorKzFq1Ci0a9cOiYmJeOyxx9Dc7P158dVXXyE8PBzp6eke23ft2oXRo0cjJSUFYWFh2LJli8f7TU1NmDt3Lvr164eYmBikpKRg8uTJOHnypOTroKqqnj9/HpMmTcLrr7+Ozp07i97PYrEgNjbW4+UNnkxby85iLELJSlBDOPQQDaOKAK/1cqGVjCg+ho7Dzo0gIIQ6TJw4EQcOHMD27duxdetW7Nq1CzNnzvS7z5w5c/DRRx9h06ZN+OKLL3Dy5Encfffd7vcdDgdGjRqFxsZG7N69G2+99RbWrl2LhQsXtjlWdXU1Jk+ejBEjRrR5r76+Hv3790dBQYHXely4cAH79+/HU089hf3792Pz5s2oqKjAnXfeKfEqSOxw2rlzZ5jN5jbGVVVVheTk5Dbl//3vf+PYsWMYPXq0e5vTeemhER4ejoqKCvTu3VtypeUg1eSlPBy0nMNDzsOO2VBbVjl5HaIYejfIWiP28zqi1R9t3/LvzbKjq+t+VNJ51fXdUNLBNLxB4H7VYYIPDh48iKKiInzzzTcYOHAgAOCll17CyJEjsWLFCqSkpLTZp6amBqtXr8b69etxyy23AADefPNN9OnTB19//TUGDx6MTz/9FN9//z0+++wzJCUlIT09HUuXLsXcuXOxaNEiREZGuo/3wAMPYMKECTCbzW0iG3fccQfuuOMOn/WPi4vD9u3bPba9/PLLGDRoECorK9GtWzfR10LStzYyMhIZGRkoLi52b3M6nSguLobVam1TPi0tDd9++y3Ky8vdrzvvvBPDhw9HeXm5or4cakY91J7OXG60Q6pEsBpCyCTvrnNUg/CO1hEUNe4Dlv1ClOwvFYp+8E/rgQ92u7J8VElJCeLj493iAQBZWVkwmUzYs2eP131KS0vR1NTk0dcyLS0N3bp1c/e1LCkpQb9+/Ty6RGRnZ6O2thYHDhxwb3vzzTdx5MgR5OfnK/ocLampqUFYWBji4+Ml7Sf5Z09eXh6mTJmCgQMHYtCgQVi1ahXq6+sxbdo0AMDkyZPRtWtXLFu2DFFRUbjuuus89ndVsPV2NVEzfyn1QaBVmoVFaoUFWsgGyYV6uK6tGhGS1veG0qhI+AWn4iG8SqIYcvY1N8ibKCy8QfxwXLnnMCrhDYBZ4W+usP+2Ga1/IOfn52PRokWyj2uz2dpMSREeHo6OHTv67Ddps9kQGRnZpnFv2dfSZrN57Yvpeg8AfvzxR8ybNw//+Mc/EB7O5vvc0NCAuXPnYvz48T67R/hCcg1ycnJw5swZLFy4EDabDenp6SgqKnJ/0MrKSphM/PTFkIpaUQ+jpFmMIB0kG9rT8pqrlaphkZ7ROxWjZQpGioAQ8jhx4oRHo2qxeJ8IZd68eVi+fLnfYx08eJBp3aTgcDgwYcIELF68GFdffTWTYzY1NeHee++FIAh49dVXJe8v6ymSm5uL3Nxcr+/t3LnT775r166Vc0rZ8BD1MIJ48CwdRpMN84UmZsdytItgdixWaC0igHQZ0VtCpKBFZCLUoh+s8DfgoSWPPvoopk6d6rdMr169kJycjNOnT3tsb25uxrlz57z2mwSA5ORkNDY2orq62iP60bKvZXJycpuZxl19M5OTk3H+/Hns27cPZWVl7rbb6XRCEASEh4fj008/dfcnEYNLPI4fP44dO3ZIjnoAtLaLB2pEPXgXDyajBhgKh96iwVIcWCCmPnoKihYiAly+x+RIiNapGF7TL4R6JCQkICEhIWA5q9WK6upqlJaWIiMjAwCwY8cOOJ1OZGZmet0nIyMDERERKC4uxtixYwEAFRUVqKysdPe1tFqt+NOf/oTTp0+70zrbt29HbGws+vbti4iICHz77bcex33llVewY8cOvP/+++jZs6foz+oSjx9//BGff/45OnXqJHrflgS1fOgd9dB6GK3ospxFObQWDt4EQyliP4/aktL676hmPxEpEqJHFIRHAaHoh/706dMHt99+O2bMmIHCwkI0NTUhNzcX48aNc490+fnnnzFixAi8/fbbGDRoEOLi4jB9+nTk5eWhY8eOiI2NxezZs2G1WjF48GAAwG233Ya+ffti0qRJeO6552Cz2bBgwQLMmjXLnSpq3c8yMTGxTb/Muro6HD582P3vo0ePory8HB07dkS3bt3Q1NSEe+65B/v378fWrVvhcDjcfUo6duzoMaomEEErH2oOrVULo4gHC+nQUjiCTTbkorWkqBkVCb/o5D4VQ/0/CG+sW7cOubm5GDFiBEwmE8aOHYsXX3zR/X5TUxMqKipw4cIF97YXXnjBXdZutyM7OxuvvPKK+32z2YytW7fiD3/4A6xWK2JiYjBlyhQsWbJEUt327duH4cOHu/+dl5cHAJgyZQrWrl2Ln3/+GR9++CEAtJmg7PPPP8evf/1r0ecKEwRBu3XSZVJbW4u4uDh0X/40TFGXvmGBhtrqPa+HVJnRIs3CYrisUtSWDhIN9qgRMWEpI3I6p7JY2E6sWEgVELnRCTHyIeXYUheZA3wvMOdsaMDxuQtQU1Mjq3+AGFztRN8Hn4HZoszEHPYGfP/KE6rWN9QJ2siHFPSOevAsHrynVkg21Md1jVlKCMuoCO9REKkREDXTL5R6IXiB5EMiLCf40SLNAsgTDx6lw6iiYaqX3vnIGSPjZ6fKtLz+vImIHAEB2EkI6/QKdUAlgp2Qlw+9RrjwKh48SQdPsiFHINQ8n95yooWIANJkRO6IGED5qJhAAsJT/w+KfhA8ENLyIb1fBpvz8jiMlqf+HFpLh9ZiwQKe5ESNtIz72DKiInpFQVgLCEkCEcyEtHyogRqRFN7FwyjSYUTJkIvYz8pSUtSKhriPL2Gqd6VREIBNp9Q2x+ao/wdB6EnIyodRoh48iwfvqZVQkg25qBVBUTsaonYUBJCXihEjFzxEQII5qmK2K1/bBY1MqkL4IWTlwwjwKh68SgfJBntc11SphLhgPYcIj1EQNQSEIIKNkJQPtaIeLFMuvIkHb6kVQ4tG/UX/78dEa1MPCbS83kpSNaxTM1pGQQDxEsJaQOREKqjjKcEzISkfUtB6aK0WC8NJEQ+epIM74QgkEWoclwMxURoNccFKRKRGQeQKCCAtFcM6uqG1LJjt8iYaIwgxkHwwgkXUIxjFw9DSoZZcKIEjMWEVDQHYpGfERkGUpGEAaVEQvUfAUMdTgldIPvzAw4Rifo+p4hweSsTDEKkVHsVCKTqmc1hFQ1zIjYpoHQUBlI+K0bP/RyCZoegHoRYkHwaFR/HgOsoRjLIhFbHXQIGksIyGuJAjIlpFQYDAqRge+n8QBG+QfPhAStRD65QLb+LBnXSQaCij9fWTKSOsoyGAtCG8WnVGFXV8HQVEacdTin4QakDyoQEsUy48iQc30mFw2RD+u3R2WLt2OtfEB67rq1BCALbRELECAmgzJDdQ+oWG4BLEZUg+vMBj1EPtxeGkiAdJh29cIsF6Xy7EpOX15iAaIlZAAG2iIKwERCwsox8EoTUkHyrDIuoRTOJhdOlQIhdqnVcXMVEoIqznDeElCsJCQPTo/0GpF0JrSD6CEB7FwyjSoZdcKEF3MeEgLcNTFETpCrkAdUAlgh+SDwXoObeHz+NxJB7MOpEylg4jCoZcAn1WpnLCMC0DSBcRNTujAtKiIFqPgBFVpxDpeBreIMDsVPZcDWtkPzUC4QnJh4ponXLhRTx4inKEkmjIQTU5URgNAdreR2JlRI3OqIB0CdFSQCj6QRgNkg+Z8Bb14EE8eJEOEg52KB6JwyAa4kJKZ1W1oiAAm7lB3MfSWEBovReCF0g+VELLqEdQiEcQS4dQVy9737D2MQxrIp+W11bPaAhw6V7TOwoCiOsPwvMIGFnHD5B6MV00wRktb7VsIrQg+ZABy9VrlWJ48QgC6VAiF0qOrZeY8BANkRoFUaMzKqCdgND8H0SwQfIhEVbiwSQyYmTxMJB0qCkXSghUL7XlhIdoiNgoiFpDcgF2AhLwPIzSL2p3PKXoByEGkg8V0GtuD19wIx6cdiLlVS6UomXURM9oCA9REBYCQjOgEqEEyYcOaBn14EI8OIpyBKtoSEUtMdEzGqJWFMSIAqJ29IMglELywRieoh7BIB4spIOEQxqtr5dcGdEjGqJ3Z1QWC9QZIQJCqRdCKeot40gQHK69QkhHqbwxiVpJuJekyDGLdYqkwjKlShBGheSDIAiCIAhNIfkgCIIgCEJTqM8HY5qjwpiv12JYYqIp9UIQhKaY7QLCFa7tgiZ6hqsNRT44ROl8AARBEATBM9TKEQShOnrPQssCKaPHCILwD8kHQRBEEMLTMhAE0RqSD4IgCIIgNIXkgyAIgjOo0zoR7JB8EARBEAShKSQfBEEQBEFoCsmHRGixJYIgCEIu586dw8SJExEbG4v4+HhMnz4ddXV1fvdpaGjArFmz0KlTJ7Rv3x5jx45FVVWVR5nKykqMGjUK7dq1Q2JiIh577DE0N3uO0Fq3bh369++Pdu3aoUuXLvj973+PX375xaPMqlWrcM011yA6OhqpqamYM2cOGhq8915+9tlnERYWhkceeUTydSD5IAiCIAiNmDhxIg4cOIDt27dj69at2LVrF2bOnOl3nzlz5uCjjz7Cpk2b8MUXX+DkyZO4++673e87HA6MGjUKjY2N2L17N9566y2sXbsWCxcudJf56quvMHnyZEyfPh0HDhzApk2bsHfvXsyYMcNdZv369Zg3bx7y8/Nx8OBBrF69Ghs3bsQTTzzRpk7ffPMN/vrXv+L666+XdR1IPgiCIAhCAw4ePIiioiK88cYbyMzMxNChQ/HSSy9hw4YNOHnypNd9ampqsHr1aqxcuRK33HILMjIy8Oabb2L37t34+uuvAQCffvopvv/+e7zzzjtIT0/HHXfcgaVLl6KgoACNjY0AgJKSEvTo0QMPPfQQevbsiaFDh+L+++/H3r173efavXs3brrpJkyYMAE9evTAbbfdhvHjx3uUAYC6ujpMnDgRr7/+Oq644gpZ14LkgyAIgiC8UFtb6/Gy28WvmOyNkpISxMfHY+DAge5tWVlZMJlM2LNnj9d9SktL0dTUhKysLPe2tLQ0dOvWDSUlJe7j9uvXD0lJSe4y2dnZqK2txYEDBwAAVqsVJ06cwLZt2yAIAqqqqvD+++9j5MiR7n2GDBmC0tJSt2wcOXIE27Zt8ygDALNmzcKoUaM86iQVWtsliHFEh9OsjAShMeEXnWiOpt91ehF+wYnwCKeygzRd2j81NdVjc35+PhYtWiT7sDabDYmJiR7bwsPD0bFjR9hsNp/7REZGIj4+3mN7UlKSex+bzeYhHq73Xe8BwE033YR169YhJycHDQ0NaG5uxujRo1FQUODeZ8KECTh79iyGDh0KQRDQ3NyMBx54wCPtsmHDBuzfvx/ffPONvIvwX0LyG0KdRgnCgNAihYTGnDhxAjU1Ne7X/PnzvZabN28ewsLC/L4OHTqkce09+f777/Hwww9j4cKFKC0tRVFREY4dO4YHHnjAXWbnzp145pln8Morr2D//v3YvHkzPv74YyxduhTApevx8MMPY926dYiKUtaQUuSDIAiCILwQGxuL2NjYgOUeffRRTJ061W+ZXr16ITk5GadPn/bY3tzcjHPnziE5OdnrfsnJyWhsbER1dbVH9KOqqsq9T3Jycpt+Ga7RMK4yy5Ytw0033YTHHnsMAHD99dcjJiYGN998M55++ml06dIFTz31FCZNmoT77rsPANCvXz/U19dj5syZePLJJ1FaWorTp0/jhhtucJ/H4XBg165dePnll2G322E2mwNcrUuQfBAEEXSY6u1wxlj0rgYRIiQkJCAhISFgOavViurqapSWliIjIwMAsGPHDjidTmRmZnrdJyMjAxERESguLsbYsWMBABUVFaisrITVanUf909/+hNOnz7tTuts374dsbGx6Nu3LwDgwoULCA/3bPJdoiAIgruMyWTyWWbEiBH49ttvPd6fNm0a0tLSMHfuXNHiAZB8EARBEIQm9OnTB7fffjtmzJiBwsJCNDU1ITc3F+PGjUNKSgoA4Oeff8aIESPw9ttvY9CgQYiLi8P06dORl5eHjh07IjY2FrNnz4bVasXgwYMBALfddhv69u2LSZMm4bnnnoPNZsOCBQswa9YsWCyXJHz06NGYMWMGXn31VWRnZ+PUqVN45JFHMGjQIPe5R48ejZUrV2LAgAHIzMzE4cOH8dRTT2H06NEwm83o0KEDrrvuOo/PFBMTg06dOrXZHgiSD4IgiBAkvAFopv5vmrNu3Trk5uZixIgRMJlMGDt2LF588UX3+01NTaioqMCFCxfc21544QV3WbvdjuzsbLzyyivu981mM7Zu3Yo//OEPsFqtiImJwZQpU7BkyRJ3malTp+L8+fN4+eWX8eijjyI+Ph633HILli9f7i6zYMEChIWFYcGCBfj555+RkJCA0aNH409/+hPz6xAmuOItHFNbW4u4uDh0X/40TP/t5GK66L+vrDnAiCh/y02HB1iKOtBS1WIWhQpUJvxC4N7a4RcDl5Ey2sV8oUlUOVO9hOFmCjsJCi2+gLKPUVev+BgEENY+Rtn+7dopr0RMtOiiYtMujnYRoo/piBb3ey3QaJfmdoH7+jdHhSl6Hwjcud6ffATa1xHg8jqjLz+fnA0NOD53AWpqakT1oZCDq50YPHIJwiOUWVVzUwO+3rZQ1fqGOiE52oUgCIIgCP0g+SAIIqQRG/HjDTERVoLgFZIPgmuYhOoJgiAIriD5IIIepX0VCIIgCLaQfBAEQRAEoSk01JaQjDPGIm3EC0EQhEaENzgR3qxwbRel+xMBocgHQRCawGLYNEEQwQHJB0EQhMaImceHIIIZkg+CIAiRSJm0jyAI35B8EARBEAShKSQfBEEQBEFoiiz5KCgoQI8ePRAVFYXMzEzs3bvXZ9nXX38dN998M6644gpcccUVyMrK8lueCDIkrMVBEAQ/BFrDiiCUIFk+Nm7ciLy8POTn52P//v3o378/srOzcfr0aa/ld+7cifHjx+Pzzz9HSUkJUlNTcdttt+Hnn39WXHmCIAiCIIyHZPlYuXIlZsyYgWnTpqFv374oLCxEu3btsGbNGq/l161bhwcffBDp6elIS0vDG2+8AafTieLiYp/nsNvtqK2t9XgRBEEQBBEcSJKPxsZGlJaWIisr6/IBTCZkZWWhpKRE1DEuXLiApqYmdOzY0WeZZcuWIS4uzv1KTU2VUk3dEbPUdcBjiFhyWwxilwAHpC0tLna5cgCKUy9h7dopXuOFplhXhtLrx+JvqFYKT8p9L+X7RBCEbyS1cGfPnoXD4UBSUpLH9qSkJNhsNlHHmDt3LlJSUjwEpjXz589HTU2N+3XixIk2ZZzR6o2Tb47y/74jwPuXjuFfQMQISiABaY4W9+dzRIeLfmiqKiA6S0hY+xj3iwgMi+vFTDok3jti7029xIPFjwsWP3IIQi801fhnn30WGzZswM6dOxEV5bsFt1gssFgkNGxecFgAs58ZwB1R+neoao4KC7gsdnM7k98JiZqjTQi/KE7EHNHhouYpcD2QxSw1Lnmq9ZaNSP1F8fu1oGVjJnfWTDENqlBXL+vYRkFNCWOyGrFMWdVbPMT+KPB7DBILIsiRJB+dO3eG2WxGVVWVx/aqqiokJyf73XfFihV49tln8dlnn+H666+XXlONaY4CwhXKiSi5EFEm4HlUEBDg0sNZrIAAkL7eC0MRUWPqbn+Ns1HEROsoj57SAegvHmLQKuohJkIbjJgvNsMcrmwyOKGZJpNTG0nfgsjISGRkZHh0FnV1HrVarT73e+6557B06VIUFRVh4MCB8mvbikCpF4ey4In/Y4v8YotKrwRK0Yh4WEn5tSU1DSP2QS0pDdMahWkZV3ifSeMn5nwtUhLeXlrCQz30Sq+0hAfxoKgHQYhDstLn5eVhypQpGDhwIAYNGoRVq1ahvr4e06ZNAwBMnjwZXbt2xbJlywAAy5cvx8KFC7F+/Xr06NHD3Tekffv2aN++PcOPIh2tUi9MohsB0i+AtAgIwFkUxAUnaRmlBGr4pUROeO6jonekw4Xe4iFWOlh1JA9EqEY9COMgWT5ycnJw5swZLFy4EDabDenp6SgqKnJ3Qq2srITJdPkL9uqrr6KxsRH33HOPx3Hy8/OxaNEiZbXHpeiH6aI6X+hAqRcp8hJIQFj0/wDkCQggbs0KsQICyOgL4g1XoyRTQgB10zJK4FkoxMDDyBWpkTa9xYMVFBkhggFZyczc3Fzk5uZ6fW/nzp0e/z527JicUzBD7Y6nwSAggHqdUQEFURAXQRINMSpMU1k6SAdgHPFgIRYU9SCMAA1aDwCLjqfSzqc8RQNcfijqnYYBGEoIwDQa4o1QFxPV+szoJB0AP+JBw2sJ4jIhIR+Boh+Kj88w+iGmjJjoh7usRAmRmoYBxEVBgLaNhyIZYRAN8UagxtfocqJVh1w3OkoHwI94iDomiQURQoSEfASCRcdTngUE4KMzamtaNio8iog3jCAnmguGN3SWDoAv8aCOpgThCcmHCMSmXlgLSMBjyBAQQL0oiBwBccFcRFSWEF9oJSdcCIY3FEqHUuFwwYt4sJQO1pGRQDM5E4SahIx88DjjqVYdUNvsI0NC1EjD+IKJiGgYDZECt9IgF0brrbCQDinC4d6HE/HQuqMpiQehNyEjH0pRI/oh7rzqCAggfWZUQJsoSEuYighHEmJYGC/uppd0APyIh6jjUX8QIsgIKflQu+Op+zwa9/9QAu9RkJbwMIFZyKHSSrJ6SgfAl3hQ1IMIRUJKPgKh12JzenRAbbO/BAnRojOqP5inZQIR7KKikmB4Q2/pANQTD606lRL+MV9shtms7JkjOGhtF7Uh+ZCAlDk/pIqMHh1QvR5DZCpGzpBcF1xFQ8QgpnHmWVA0lAtfaDlyxe9xOBMPFms/STsfs0MRhCJCTj60Sr0A7AVEbP8PAIqjIGqskuvep0VDwlXfECUEauDVlBMO5MIbPEQ53MdRcYE4NcVDDDS8ljAiIScfgQgkDGrPeMpCQADt0zCAuChIm31ViIpoEg2Rihg54VQixMJqmCzARjqkCocLtcVDinToGfUgqSHUhORDZdToRyJFQAC+oyBtjsEwKqJ7NEQKBhMPlqLREj2lA+BLPMQgVhAo3ULwBsmHF1gLgxr9P6T0EVEqIWp1Rg14LIZREUOJCIeoJRsu9JYOgD/x0HN4LUU9CLUh+ZCBnNSL3gICsEnFqJ2G8XtcRlERXw0pSckl1BaNlvAgHQB/4sESSrcQPELywTFqzO/BIgqiZRrG57FV7rQaCKOLipaC4Q0th8v6g9c5PMTsR5JAGBkamO6DQF9sOTlUOQ+LQA8h2Q+3dib5vfSjTaIf2o7ocGYNhc9ztItgNipCLM4Yi6iX1vBYL9ffp+VL9rEY3U9S7mH3PhyJh/hjSStPQqM+586dw8SJExEbG4v4+HhMnz4ddXV1fvdpaGjArFmz0KlTJ7Rv3x5jx45FVVWVR5mHHnoIGRkZsFgsSE9Pb3OMiooKDB8+HElJSYiKikKvXr2wYMECNDVd/vG2du1ahIWFebyiojxviqlTp7Ypc/vtt0u+DhT50Bg9O6B63VdBKkbO9Owu1IiIqBENUUqght5fBEXv6IRc1BBBvSId7v04Ew8xkkCdTPlk4sSJOHXqFLZv346mpiZMmzYNM2fOxPr1633uM2fOHHz88cfYtGkT4uLikJubi7vvvhtfffWVR7nf//732LNnD/71r3+1OUZERAQmT56MG264AfHx8fjnP/+JGTNmwOl04plnnnGXi42NRUVFhfvfYWFt78nbb78db775pvvfFov0ZxXJhx/0HnZ7+TzqTsGuJBUjdXp2Fy0bk1AREW8YVTBaombUSev+HF7312CBOB7WbqGoh/ocPHgQRUVF+OabbzBw4EAAwEsvvYSRI0dixYoVSElJabNPTU0NVq9ejfXr1+OWW24BALz55pvo06cPvv76awwePBgA8OKLLwIAzpw541U+evXqhV69ern/3b17d+zcuRP/+Mc/PMqFhYUhOTnZ7+ewWCwBywSC0i46oEb6RWwZv/srmB5ayQPeFUpXK0WjR1ommGGVQvF7Dgb3gpzUSptjGFQ8KN3ChtraWo+X3a6sr1dJSQni4+Pd4gEAWVlZMJlM2LNnj9d9SktL0dTUhKysLPe2tLQ0dOvWDSUlJbLrcvjwYRQVFWHYsGEe2+vq6tC9e3ekpqbirrvuwoEDB9rsu3PnTiQmJuKaa67BH/7wB/zyyy+Sz0/yoRC5oU2eBUSLviD+UEtE1G4wgxUtZMN9Lo6kg1fx4EEUHBwH7Ez1diYvAEhNTUVcXJz7tWzZMkV1s9lsSExM9NgWHh6Ojh07wmaz+dwnMjIS8fHxHtuTkpJ87uOPIUOGICoqCldddRVuvvlmLFmyxP3eNddcgzVr1uDvf/873nnnHTidTgwZMgQ//fSTu8ztt9+Ot99+G8XFxVi+fDm++OIL3HHHHXA4HJLqQWmXAIjpoyE3/SKn/4fYFAwARSNllKZipKZhfKFWXxGxjSjPKRuW6CVkLAWThfhqNYxWrqyo0ddDqszwLB6sOXHiBGJjY93/9tW3Yd68eVi+fLnfYx08eJBp3eSyceNGnD9/Hv/85z/x2GOPYcWKFXj88ccBAFarFVar1V12yJAh6NOnD/76179i6dKlAIBx48a53+/Xrx+uv/569O7dGzt37sSIESNE14Pkw4CInuFUx8XqWApIS9TuK9LmfBIbZZ5khccID+toFgvhALRdn0VN8SDYEhsb6yEfvnj00UcxdepUv2V69eqF5ORknD592mN7c3Mzzp0757MPRXJyMhobG1FdXe0R/aiqqpLV7yI1NRUA0LdvXzgcDsycOROPPvoozGZzm7IREREYMGAADh8+7Pdzde7cGYcPHyb5YA1v0Q8p6BkFkdsZVSxajKCRiloRFR5FQgxqDbPWWzoAPsWDoh76kJCQgISEhIDlrFYrqqurUVpaioyMDADAjh074HQ6kZmZ6XWfjIwMREREoLi4GGPHjgVwadhsZWWlR5RCDk6nE01NTXA6nV7lw+Fw4Ntvv8XIkSN9HuOnn37CL7/8gi5dukg6N8kHQ3hLv7QuD+grIYB6IgJoHxVRglFlwh9qz+cC6JdaaXMMjcSDoh3BRZ8+fXD77bdjxowZKCwsRFNTE3JzczFu3Dj3SJeff/4ZI0aMwNtvv41BgwYhLi4O06dPR15eHjp27IjY2FjMnj0bVqvVPdIFuNSBtK6uDjabDRcvXkR5eTmASxGOyMhIrFu3DhEREejXrx8sFgv27duH+fPnIycnBxERl55HS5YsweDBg3HllVeiuroaf/7zn3H8+HHcd999AC51Rl28eDHGjh2L5ORk/Pvf/8bjjz+OK6+8EtnZ2ZKuBckHY7QafnvpXNLTKnqmYgB9RATgX0aMiBay4YIX6QC0W6NFbfGgqIc+rFu3Drm5uRgxYgRMJhPGjh3rHiYLAE1NTaioqMCFCxfc21544QV3WbvdjuzsbLzyyisex73vvvvwxRdfuP89YMAAAMDRo0fRo0cPhIeHY/ny5fjhhx8gCAK6d++O3NxczJkzx73Pf/7zH8yYMQM2mw1XXHEFMjIysHv3bvTt2xcAYDab8a9//QtvvfUWqqurkZKSgttuuw1Lly6VPNdHmCAIbOfvVoHa2lrExcWh+/KnYYpq+40xXZT2UDHLGC0lJTIhVz7kpl/kygSLqduVrBXT5lgqyogLkhD5aCkbAB+pFY/jcJhm8TyXuseXKh/O6MvfZ2dDA47PXYCamhpRfSjk4Gonsq6ag3CzMlNqdtjx2Y8vqFrfUIciHyqgdf8PudEMVlEQgI2EaBEV8dWAkpS0xaiy4T6eTukVpfuqLR5yUCIeBOENkg+V4LUDamtY9AUBlK+Y2+Z4rRoitaMiUhtaI8qK1jLhD9ai4T6ujlEOpfvLTbNQJ1PCiPDzNCIUoTSKoffcIAGPrVFfEbFIachZiwpPEiEWtWTDfXwDSwegnXhoAUU9CDEY7ynGAIdFer8PeSNSLv1XagTE9SCSMwLm0vmUS4iS47RsCNQWkZbwICXeMKIsKEFt0fA4l86pFRbH0VI8aPQMwQtB8VR0RjsldzrVSkAA/STEhWyJYCwirWEtJkaTEqOjpWQA7KIbgP7C4SKYIh4EIYWgkA+AfwEBtJeQy+dVLiOshMbjmD4aE62kxB+hLCxaS0Wb8zOUjDbHZiAdrMSFx46lQcGFi4BJ4ffXqWwBOSIwQSMfgDEEBNBPQi6fn0FEQwUZcR9bw2iJzzowbIDVFhm9ZUEuakqGx3k4iXK4oGgHQQSZfADGERBAfwm5VAd+UjSizqNRtIQlRpUDVmglGR7n5Ew4XJB4EMQlgk4+AG0FxBdSxIAHCblcF76jIj7PyUG0JNTRQzLa1IGjtEprSDwI4jJBKR+AdgLi81gyxIAnCblUH/ZRETGwlhWljWIoygsPIuFCLRnQ8jxaiweNaiF4J2jlA9BfQABlEgJIExGxDxw9O67KOY/a5wsEy4ZYlaHHHImCErSSDK3PSeJBEG0JavkA+BAQQMmw2Uv/ZblYXeuHEwsZ0SS14qeh0EtMpBIsoqAUPUSD9bnVbORpDg8i2Al6+QD4ERCALwlxwUJG9Ojn4e/8etWD8ERPyWiJEYTDBYkHEQqEhHwAfAkIwKeEuGj5IGOVohEL8/4ejBu/UJIZXsRBCSw+g1YNO6VZiFAiZOQD4E9AAL4lBGCXohEL76kVlg0y07lRgkAUvKHn59KyUSfxIEKNkJIPgE8BAfiXEBcsoiJyCbbUSrAKgxx4uRZ6NOYkHkQoEnLyAfArIIByCZGDXHHROiriC96jJcRleJGMlujZiJN4sEe4cBGCyaHsGM5GRrUhfBGS8gHwLSCAenN3eKP1A9DoMtIStRo7khr/8CgZLWHVeOsxARiJBxEMhKx8APwLCKCthLhQS0aUwIPItITHjqy8N/gs4KHh1XPGUR4+P0GwIKTlAzCGgAD6SIgLuZOesYTVVPa8EgriIAUeG1mSDoJgR8jLB2AcAQH07fAJsIuKsCTYxSSY4b1R5WFdFd6vEUHIgeTjv8gVEJbIXdhOzwaWRxlpiRYPbhKcwBipAeVBOFzIvW6sn00EwRqSjxbIERCWuB4YRpQQFzykaLTGSA0r4R2ehMMFiQcRzJB8tEJvAQGUS4gayJtynX09vBEqkhPK8CgHakLiQQQ7JB9e4EFAAPkSogZ69zXxh6+GiaTEmBhFNHiLeJF4EEaC5MMHvAgIwJeEAHzO5+ENf40YiYn+GEUyWsKbcAAkHYQxIfnwA08CAvAnIS54jor4wogNH6EPPAqHCxIPwqiQfASANwEB+JUQwDhREYLwB8/C4YLEgzAyJB8i4FFAAL4lxIWeD3ESn+DACCKgNSQevhHqL0AIa1J2DEHZ/kRgSD5EwquAAMaQED3w1WiRlPAJSYY4SDyIYIDkQwI8Cwig30PJaNJDjRzBChIBgpAHyYdEeBcQPWj9ADaajBCEFEg4CEI5JB8yIAHxD8kIEWyQcBAEW0g+ZOKMdup6fiPJD28PbpIhY8DbfUMQBDtIPgyKS36MJCG8QI0aEczo/cOIIMRALZfBcUY76WFDEAQAEg/COMiSj4KCAvTo0QNRUVHIzMzE3r17/ZbftGkT0tLSEBUVhX79+mHbtm2yKkv4xiUhvLwIgtAW+t4Zg3PnzmHixImIjY1FfHw8pk+fjrq6Or/7NDQ0YNasWejUqRPat2+PsWPHoqqqyqPMQw89hIyMDFgsFqSnp3s9zieffILBgwejQ4cOSEhIwNixY3Hs2DH3+5s3b8att96KhIQExMbGwmq14pNPPvE4Ro8ePRAWFtbmNWvWLEnXQbJ8bNy4EXl5ecjPz8f+/fvRv39/ZGdn4/Tp017L7969G+PHj8f06dNRVlaGMWPGYMyYMfjuu++knpowEHrLD0lRcKL3/cPzizAGEydOxIEDB7B9+3Zs3boVu3btwsyZM/3uM2fOHHz00UfYtGkTvvjiC5w8eRJ33313m3K///3vkZOT4/UYR48exV133YVbbrkF5eXl+OSTT3D27FmP4+zatQu33nortm3bhtLSUgwfPhyjR49GWVmZu8w333yDU6dOuV/bt28HAPzv//6vpOsQJgiCIGWHzMxM3HjjjXj55ZcBAE6nE6mpqZg9ezbmzZvXpnxOTg7q6+uxdetW97bBgwcjPT0dhYWFos5ZW1uLuLg4dF/+NExRNEkDQRCEkXA2NOD43AWoqalBbGysKudwtRO3RP4vwsMiFB2rWWjCjsZNzOt78OBB9O3bF9988w0GDhwIACgqKsLIkSPx008/ISUlpc0+NTU1SEhIwPr163HPPfcAAA4dOoQ+ffqgpKQEgwcP9ii/aNEibNmyBeXl5R7b33//fYwfPx52ux0m06W4w0cffYS77roLdrsdERHer9m1116LnJwcLFy40Ov7jzzyCLZu3Yoff/wRYWFhoq+FpA6njY2NKC0txfz5893bTCYTsrKyUFJS4nWfkpIS5OXleWzLzs7Gli1bfJ7HbrfDbr88JKGmpgbApRuYIAiCMBauZ7fE37qyaEYToPA0zbg0vXptba3HdovFAotFfo/1kpISxMfHu8UDALKysmAymbBnzx789re/bbNPaWkpmpqakJWV5d6WlpaGbt26eZUPX2RkZMBkMuHNN9/E1KlTUVdXh7/97W/IysryKR5OpxPnz59Hx44dvb7f2NiId955B3l5eZLEA5AoH2fPnoXD4UBSUpLH9qSkJBw6dMjrPjabzWt5m83m8zzLli3D4sWL22w/kf+0lOoSBEEQHPHLL78gLi5OlWNHRkYiOTkZu2xbmByvffv2SE1N9diWn5+PRYsWyT6mzWZDYmKix7bw8HB07NjRZ5tos9kQGRmJ+Ph4j+2B2tHW9OzZE59++inuvfde3H///XA4HLBarX77YK5YsQJ1dXW49957vb6/ZcsWVFdXY+rUqaLr4YLLobbz58/3iJZUV1eje/fuqKysVO3GDQZqa2uRmpqKEydOqBbaDAboOgWGrpE46DqJo6amBt26dfP5C5oFUVFROHr0KBobG5kcTxCENr/mfUU95s2bh+XLl/s93sGDB5nUSy42mw0zZszAlClTMH78eJw/fx4LFy7EPffcg+3bt7f5rOvXr8fixYvx97//vY0wuVi9ejXuuOMOr+miQEiSj86dO8NsNrfpZVtVVYXk5GSv+yQnJ0sqD/gObcXFxdEXXASxsbF0nURA1ykwdI3EQddJHK6+BmoRFRWFKB36BT766KMBf/336tULycnJbQZnNDc349y5c37b0MbGRlRXV3tEPwK1o60pKChAXFwcnnvuOfe2d955B6mpqdizZ49H+mbDhg247777sGnTJo90T0uOHz+Ozz77DJs3bxZdh5ZIuhMiIyORkZGB4uJi9zan04ni4mJYrVav+1itVo/yALB9+3af5QmCIAjCSCQkJCAtLc3vKzIyElarFdXV1SgtLXXvu2PHDjidTmRmZno9dkZGBiIiIjza0YqKClRWVkpqRy9cuNBG/sxmM4BL7biLd999F9OmTcO7776LUaNG+Tzem2++icTERL9l/CJIZMOGDYLFYhHWrl0rfP/998LMmTOF+Ph4wWazCYIgCJMmTRLmzZvnLv/VV18J4eHhwooVK4SDBw8K+fn5QkREhPDtt9+KPmdNTY0AQKipqZFa3ZCCrpM46DoFhq6ROOg6iYOu02Vuv/12YcCAAcKePXuEL7/8UrjqqquE8ePHu9//6aefhGuuuUbYs2ePe9sDDzwgdOvWTdixY4ewb98+wWq1Clar1eO4P/74o1BWVibcf//9wtVXXy2UlZUJZWVlgt1uFwRBEIqLi4WwsDBh8eLFwg8//CCUlpYK2dnZQvfu3YULFy4IgiAI69atE8LDw4WCggLh1KlT7ld1dbXHuRwOh9CtWzdh7ty5sq+DZPkQBEF46aWXhG7dugmRkZHCoEGDhK+//tr93rBhw4QpU6Z4lH/vvfeEq6++WoiMjBSuvfZa4eOPP5Z0voaGBiE/P19oaGiQU92Qga6TOOg6BYaukTjoOomDrtNlfvnlF2H8+PFC+/bthdjYWGHatGnC+fPn3e8fPXpUACB8/vnn7m0XL14UHnzwQeGKK64Q2rVrJ/z2t78VTp065XHcYcOGCbg0zsfjdfToUXeZd999VxgwYIAQExMjJCQkCHfeeadw8ODBgMdo3aZ/8sknAgChoqJC9nWQPM8HQRAEQRCEEmhtF4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgiAIgiA0hRv5KCgoQI8ePRAVFYXMzEzs3bvXb/lNmzYhLS0NUVFR6Nevn9/56YMJKdfp9ddfx80334wrrrgCV1xxBbKysgJe12BA6r3kYsOGDQgLC8OYMWPUrSAnSL1O1dXVmDVrFrp06QKLxYKrr746JL53Uq/TqlWrcM011yA6OhqpqamYM2cOGoJ8Ucxdu3Zh9OjRSElJQVhYmN+FQ13s3LkTN9xwAywWC6688kqsXbtW9XoSHCF7kC5DNmzYIERGRgpr1qwRDhw4IMyYMUOIj48XqqqqvJb/6quvBLPZLDz33HPC999/LyxYsEDyxGVGROp1mjBhglBQUCCUlZUJBw8eFKZOnSrExcUJP/30k8Y11w6p18jF0aNHha5duwo333yzcNddd2lTWR2Rep3sdrswcOBAYeTIkcKXX34pHD16VNi5c6dQXl6ucc21Rep1WrdunWCxWIR169YJR48eFT755BOhS5cuwpw5czSuubZs27ZNePLJJ4XNmzcLAIQPPvjAb/kjR44I7dq1E/Ly8oTvv/9eeOmllwSz2SwUFRVpU2FCd7iQj0GDBgmzZs1y/9vhcAgpKSnCsmXLvJa/9957hVGjRnlsy8zMFO6//35V66k3Uq9Ta5qbm4UOHToIb731llpV1B0516i5uVkYMmSI8MYbbwhTpkwJCfmQep1effVVoVevXkJjY6NWVeQCqddp1qxZwi233OKxLS8vT7jppptUrSdPiJGPxx9/XLj22ms9tuXk5AjZ2dkq1ozgCd3TLo2NjSgtLfVYvMZkMiErKwslJSVe9ykpKWmz2E12drbP8sGAnOvUmgsXLqCpqUnVlSX1RO41WrJkCRITEzF9+nQtqqk7cq7Thx9+CKvVilmzZiEpKQnXXXcdnnnmGTgcDq2qrTlyrtOQIUNQWlrqTs0cOXIE27Ztw8iRIzWps1EIxWc44YmkVW3V4OzZs3A4HEhKSvLYnpSUhEOHDnndx2azeS1vs9lUq6feyLlOrZk7dy5SUlJ8rlJodORcoy+//BKrV69GeXm5BjXkAznX6ciRI9ixYwcmTpyIbdu24fDhw3jwwQfR1NSE/Px8LaqtOXKu04QJE3D27FkMHToUgiCgubkZDzzwAJ544gktqmwYfD3Da2trcfHiRURHR+tUM0IrdI98ENrw7LPPYsOGDfjggw90WXKaR86fP49Jkybh9ddfR+fOnfWuDtc4nU4kJibitddeQ0ZGBnJycvDkk0+isLBQ76pxxc6dO/HMM8/glVdewf79+7F582Z8/PHHWLp0qd5VIwiu0D3y0blzZ5jNZlRVVXlsr6qqQnJystd9kpOTJZUPBuRcJxcrVqzAs88+i88++wzXX3+9mtXUFanX6N///jeOHTuG0aNHu7e5lpYODw9HRUUFevfurW6ldUDOvdSlSxdERES4l+AGgD59+sBms6GxsRGRkZGq1lkP5Fynp556CpMmTcJ9990HAOjXrx/q6+sxc+ZMPPnkk22WNA9VfD3DY2NjKeoRIuj+TYiMjERGRgaKi4vd25xOJ4qLi2G1Wr3uY7VaPcoDwPbt232WDwbkXCcAeO6557B06VIUFRVh4MCBWlRVN6Reo7S0NHz77bcoLy93v+68804MHz4c5eXlSE1N1bL6miHnXrrppptw+PBht5wBwA8//IAuXboEpXgA8q7ThQsX2giGS9gEWsPTTSg+w4lW6N3jVRAuDWezWCzC2rVrhe+//16YOXOmEB8fL9hsNkEQBGHSpEnCvHnz3OW/+uorITw8XFixYoVw8OBBIT8/P2SG2kq5Ts8++6wQGRkpvP/++8KpU6fcr5bLNwcbUq9Ra0JltIvU61RZWSl06NBByM3NFSoqKoStW7cKiYmJwtNPP63XR9AEqdcpPz9f6NChg/Duu+8KR44cET799FOhd+/ewr333qvXR9CE8+fPC2VlZUJZWZkAQFi5cqVQVlYmHD9+XBAEQZg3b54wadIkd3nXUNvHHntMOHjwoFBQUEBDbUMMLuRDEAThpZdeErp16yZERkYKgwYNEr7++mv3e8OGDROmTJniUf69994Trr76aiEyMlK49tprhY8//ljjGuuDlOvUvXt3AUCbV35+vvYV1xCp91JLQkU+BEH6ddq9e7eQmZkpWCwWoVevXsKf/vQnobm5WeNaa4+U69TU1CQsWrRI6N27txAVFSWkpqYKDz74oPCf//xH+4pryOeff+71WeO6NlOmTBGGDRvWZp/09HQhMjJS6NWrl/Dmm29qXm9CP8IEgWKBBEEQBEFoh+59PgiCIAiCCC1IPgiCIAiC0BSSD4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgiAIgiA0heSDIAiCIAhNIfkgCIIgCEJTSD4IgiAIgtAUkg+CIAiCIDTl/wNgUBO+PXphBgAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test, trial = TestFunction(space), TrialFunction(space)\n", + "m = Function(space, name=\"m\").interpolate(sin(pi * X[0]) * sin(2 * pi * X[1]))\n", + "\n", + "\n", + "def forward(m):\n", + " u = Function(space, name=\"u\")\n", + " solve(inner(grad(trial), grad(test)) * dx == -inner(m, test) * dx,\n", + " u, DirichletBC(space, 0.0, \"on_boundary\"))\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(((1.0 - X[0]) ** 4) * u * u * dx)\n", + " return u, J\n", + "\n", + "\n", + "u, J = forward(m)\n", + "\n", + "print(f\"{J.value=}\")\n", + "\n", + "\n", + "def plot_output(u, title):\n", + " r = (u.dat.data_ro.min(), u.dat.data_ro.max())\n", + " eps = (r[1] - r[0]) * 1.0e-12\n", + " p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))\n", + " plt.gca().set_title(title)\n", + " plt.colorbar(p)\n", + " plt.gca().set_aspect(1.0)\n", + "\n", + "\n", + "plot_output(u, title=\"u\")" + ] + }, + { + "cell_type": "markdown", + "id": "be99c89b-29c5-492e-b3bd-6792d3c3f4b2", + "metadata": {}, + "source": [ + "## First order adjoint\n", + "\n", + "We can differentiate a functional respect to $m$ using `compute_gradient`. Specifically we compute the derivative of $\\hat{J} = J \\circ \\hat{u}$, where $\\hat{u}$ is the function which maps from the control $m$ to the solution to the discretized Poisson problem.\n", + "\n", + "When we compute a derivative of a functional with respect to a finite element discretized function using the adjoint method the result is not a function, but is instead a member of a dual space. Here we have $m \\in V$, and the derivative of $\\hat{J}$ with respect to $m$ is a member of the associated dual space $V^*$. In order to visualize this derivative we first need to map it to $V$. We can do this using a Riesz map. This is not uniquely defined, but here we choose to define a Riesz map using the $L^2$ inner product.\n", + "\n", + "Being precise the function we visualize, $g^\\sharp \\in V$, is defined such that\n", + "\n", + "$$\\forall \\zeta \\in V \\qquad \\left. \\frac{d \\hat{J} \\left( m + \\alpha \\zeta \\right)}{d \\alpha} \\right|_{\\alpha = 0} = \\int_\\Omega \\zeta g^\\sharp,$$\n", + "\n", + "where $\\alpha$ is a scalar." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "54f6e366-2cda-41f4-aa9c-d6c37d5bc26a", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:46:41.548873Z", + "iopub.status.busy": "2023-12-20T16:46:41.548521Z", + "iopub.status.idle": "2023-12-20T16:46:44.831991Z", + "shell.execute_reply": "2023-12-20T16:46:44.831146Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J.value=9.711060188691798e-06\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGzCAYAAACPa3XZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABz/0lEQVR4nO2de1xUdf7/X8wAA6JAKhdx8doFLRPDxDH7uiZF6Vpu9g0v621NaxMraUstEy9tZmvmVhRbadamafY1t8wfZZi5JWki7JYp5XrB0kHNBQRlgJnz+8OdkYG5nMvnnPM5M+/n4zGP8sznnPOZw5nzec77/bmECYIggCAIgiAIQiNMeleAIAiCIIjQguSDIAiCIAhNIfkgCIIgCEJTSD4IgiAIgtAUkg+CIAiCIDSF5IMgCIIgCE0h+SAIgiAIQlNIPgiCIAiC0BSSD4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgiAIgiA0heSDIHRi6tSp6NGjR5vtixYtQlhYmPYVIgiC0AiSD4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgtAJX51KHQ6HxjUhCILQFpIPgtCJK664AtXV1W22Hz9+XPvKEARBaAjJB0HoRO/evVFTU4N//etf7m2nTp3CBx98oGOtCIIg1CdMEARB70oQRCjyyy+/oHv37khKSsJDDz2ECxcu4NVXX0VCQgL2798P+moSBBGsUOSDIHSiU6dO+OCDD9CuXTs8/vjjeOutt7Bs2TKMHj1a76oRBEGoCkU+CIIgCILQFIp8EARBEAShKSQfBEEQBEFoCskHQRAEQRCaIlk+du3ahdGjRyMlJQVhYWHYsmVLwH127tyJG264ARaLBVdeeSXWrl0ro6oEQRAEQQQDkuWjvr4e/fv3R0FBgajyR48exahRozB8+HCUl5fjkUcewX333YdPPvlEcmUJgiAIgjA+ika7hIWF4YMPPsCYMWN8lpk7dy4+/vhjfPfdd+5t48aNQ3V1NYqKiuSemiAIgiAIgxKu9glKSkqQlZXlsS07OxuPPPKIz33sdjvsdrv7306nE+fOnUOnTp18rodBEARB8IkgCDh//jxSUlJgMqnX1bChoQGNjY1MjhUZGYmoqCgmxyLaorp82Gw2JCUleWxLSkpCbW0tLl68iOjo6Db7LFu2DIsXL1a7agRBEISGnDhxAr/61a9UOXZDQwN6dI9B1Wknk+MlJyfj6NGjJCAqobp8yGH+/PnIy8tz/7umpgbdunVD6uIFMNGNYDickTSPndExNVLEkZCPs6EBJ/KfRocOHVQ7R2NjI6pOO1GxLwUdOiiLrpw/78Q1A0+isbGR5EMlVJeP5ORkVFVVeWyrqqpCbGys16gHAFgsFlgsljbbTVFRXMqH00KNqz9oPHcQ4P2rqjsmO0mRkdAibd6hgwmxCuWDUB/V5cNqtWLbtm0e27Zv3w6r1ar2qTWBxIMg9IO37x/JEEGIQ7Ie1tXVoby8HOXl5QAuDaUtLy9HZWUlgEspk8mTJ7vLP/DAAzhy5Agef/xxHDp0CK+88gree+89zJkzh80n0BHeHnwEQeiL0yLQc4EgRCA58rFv3z4MHz7c/W9X34wpU6Zg7dq1OHXqlFtEAKBnz574+OOPMWfOHPzlL3/Br371K7zxxhvIzs5WXHn6khuUKIfeNSCk0GDWuwaGw/VsokgIQXhHsnz8+te/hr+pQbzNXvrrX/8aZWVlUk9FsIQafEIuvN07BpIhvX8gkfwQvMLlaBeCEbw1GgTBgtb3tYFkRGucFoEEhOASko9gg4SDCDVa3vMkIm0gASF4hORDTUgERGG20HXiGYfdQA26Xt85zqWHBITgDZIPtSDxaANJhjHx9XczlJSojQGiLyQgBE+QfKiBAcSDRIBQip73ENfi4/r+cyghJCAEL5B8sIZj8SDhIIKFlvcytyLCqYQEu4CcdTTC7lA4vbqDzfowhG9IPljCoXgYUTgsUWxWpSSUY2+I1LsKAWl9j3MnIxxKSLALCME/JB+s4Ew8jCIdJBp84+/vw6uYcCsjnEkICQihJyQfLJAhHkaRA1aQZAQfvv6mvEkJdykajiSEBITQC5IPJciMdvAoHiQHBCu0upfkSI6a3z3JYsOJhJCAEHoQevKhc3qEJ/EwknDEWIxT12Ci3s5XFKMlLe9fHqItru+2ESWEBITQmtCTDx3hQTx4Fw6SDL7w9ffgTUpa39d6yohiCWGFRJkhASG0hORDI/QWD96kgyTD2Pj7+/EgJjxERWRLCCuiHCQgBLeQfGiAXuLBSjhIFPzT3mJnerw6u4Xp8bSGt2iJ3lERXTu8koAQnELyoTJai0eoCAfrBp8nWH42nkSG9T0lV2b0lBFdoiEkIASHkHyohFzp0DM9wqNwBLNkaIFa148HqWl9v7KSETlIFRjNJYQEhOAMkg8VMJJ48CIcJBnGwtffS08pYSUjcnB9d7mWEBIQgiNIPhhjFPHQUzqCTTTiLQ2qHLfaHqXKcdXE399WazFpeY9rJSLcS0gICMhZpwUNTmVru9Q5aW0XtSH5YAjv4sFKOHiUB7UEQE9Yfya9ZUZPMdE6KsK1hISAgBD8Q/LBCF7FIxiFIxhFQwtYXTc1JEbu/SVXWrSKinArISQghM6QfDCAR/FgIR16CwdJBp/4+7toHV1pfY/KkREtoiJcSggJCKEjJB8KkSMecqVDi34aeghHMEhGJ0u9qsf/xR6j6vFZobeYtLx/WURFALYywp2EkIAQOkHyIROtox1qi4cW0sGTZKgtC6xhWV+9RMbX318tKWERFQHEf/ekSApXEkICQugAyUcAWE4Sxpt4sBIOvaXCaCKhN0qvF2t5kXr/yJUVFlERf7i+p3IkBJAmIswlhASE0BiSD43gRTyMLBwkGXzg7++gRVSl9b0nR0ZYRUW8IUdCAHnREH8/jmQtbEcCQmiEssHQhCh4EI/2Frti8Yi3NLhfatLJUu/1xTsdI+o1efGMHn87Fvel6/vB4nviIsbSKOs7bIlqZNIZXVbUVsbKuk6LIP08QUhBQQF69OiBqKgoZGZmYu/evX7Lb9q0CWlpaYiKikK/fv2wbds2j/c3b96M2267DZ06dUJYWBjKy8t9HksQBNxxxx0ICwvDli1b3NvXrl2LsLAwr6/Tp08DAL788kvcdNNN6NSpE6Kjo5GWloYXXnhB9nUQC0U+VEZP8WDxEFVTNPQWCt4bcl+wrPe5Jm36f2gVLWERFQHYpmi0jIS0xmxxUAREAzZu3Ii8vDwUFhYiMzMTq1atQnZ2NioqKpCYmNim/O7duzF+/HgsW7YMv/nNb7B+/XqMGTMG+/fvx3XXXQcAqK+vx9ChQ3HvvfdixowZfs+/atUqhIW1vf45OTm4/fbbPbZNnToVDQ0N7nrFxMQgNzcX119/PWJiYvDll1/i/vvvR0xMDGbOnCn3kgQkTBAE7rW1trYWcXFx6L78aZiiLj9MZBm3RLNX0udDjngolQ4e0ypaS4ZRpcIoaCUsaqRwWHRuVSojckfPKJEQWX1DJAoIAJ8C4mxowPG5C1BTU4PY2FjpdRGBq53Y8V0q2ndQOMPpeSduue4ETpw44VFfi8UCi6Xt3z8zMxM33ngjXn75ZQCA0+lEamoqZs+ejXnz5rUpn5OTg/r6emzdutW9bfDgwUhPT0dhYaFH2WPHjqFnz54oKytDenp6m2OVl5fjN7/5Dfbt24cuXbrggw8+wJgxY7x+rjNnzqBr165YvXo1Jk2a5PPz33333YiJicHf/vY3n2WUQpEPFdA62sGbcGglGyQZ+uDrurOWEjWiJS3vcb2iInpEQkIpAvKLIwYXHco64l5wXPrRmZqa6rE9Pz8fixYt8tjW2NiI0tJSzJ8/373NZDIhKysLJSUlXo9fUlKCvLw8j23Z2dkeKRNR9bxwARMmTEBBQQGSk5MDln/77bfRrl073HPPPT7LlJWVYffu3Xj66acl1UUqoSUfMvKZUjGaeBhFOIwoGp0j6lQ57tmm9qocVylaSQngeb8ZVUS0lhCtBCSY8Bb5aM3Zs2fhcDiQlJTksT0pKQmHDh3yelybzea1vM1mk1S/OXPmYMiQIbjrrrtElV+9ejUmTJiA6OjoNu/96le/wpkzZ9Dc3IxFixbhvvvuk1QXqYSWfEhEasrFSOLBu3TwJBtqSYRcpNSHB1Fp+bdUS0SUpmhc3wclaZn2Fruuq/oS7ImNjVUtTaSUDz/8EDt27EBZWZmo8iUlJTh48KDPVMo//vEP1NXV4euvv8a8efNw5ZVXYvz48Syr7IFh5UNyfw+V+3poKR7BKB1aygZvMqEmUj+r2rLS+u/MSkZc9yMLCdF6ivgYS6OsfiCWqEZF/UAINnTu3BlmsxlVVVUe26uqqnymQpKTkyWV98aOHTvw73//G/Hx8R7bx44di5tvvhk7d+702P7GG28gPT0dGRkZXo/Xs2dPAEC/fv1QVVWFRYsWqSofhhxqy9vQLrkdS7UUDxZDZFkNndR6+GjniDqPF+Gb1tdK7evF+h5gcX8q+a7I/X7KjX5qtSI24ZvIyEhkZGSguLjYvc3pdKK4uBhWq9XrPlar1aM8AGzfvt1neW/MmzcP//rXv1BeXu5+AcALL7yAN99806NsXV0d3nvvPUyfPl3UsZ1OJ+x2dWe9NmzkQxIqRj20HNEi9cHGW5RDq+gGCQZbWl5PNSMjLKMieqZi5KZf5EZACP3Jy8vDlClTMHDgQAwaNAirVq1CfX09pk2bBgCYPHkyunbtimXLlgEAHn74YQwbNgzPP/88Ro0ahQ0bNmDfvn147bXX3Mc8d+4cKisrcfLkSQBARUUFgEtRk5av1nTr1s0dxXCxceNGNDc343e/+12b8gUFBejWrRvS0tIAALt27cKKFSvw0EMPMbgyvgl++dCgk6kUjCQeLKRDC+Eg2dAOrUQEUN5XRM9UjJb9Pyj9oj85OTk4c+YMFi5cCJvNhvT0dBQVFbk7lVZWVsJkupxoGDJkCNavX48FCxbgiSeewFVXXYUtW7a45/gALvXpcMkLAIwbNw6A9xE3gVi9ejXuvvvuNika4FKUY/78+Th69CjCw8PRu3dvLF++HPfff7+kc0jFkPN8SEq7cNLXQ8tOpSzSK0pRUzqMKBudw8/L2u9scwfGNVEPrTq3yo2IsJg3RKqEyBEQNecB0WK+D29DbbWc52PTP9PQroPCobbnHfjf/odUrW+oE9yRDxIP0fCaWuFRNOSKhJrn4kFStE7PSJUQXkbFBELNDqiyhtsShAoEt3xwAO/iwZt06CUbWgqFGkitv9qy0vrvqIaMdIyolyUggHapGBp+SxDeCV754CDqofVoFinwlFrRUjiMLhms0DqiolZURI6AAGwkRGwURI6AUPSDCHaCVz4MiBbiwYt0aCEcJBrKaXkN1RARQLmMyE3DAOxSMUYTEILQm+CUDwNGPYwgHrxLB8mGuriuL+uUDauoiN5REJ4EJJSpao5DdJOypu1iczOj2hC+CE75MBha9O9QIh68SodRZSM5ojpgGVtTvOr1kIsa0RD3sRVGRZRGQQBla8VoPTuqLwJFPyj1QuhNyMuHnlEPnvt38Nifg0fZECMSahyXFzlRKxriPr7MqIjcKAigLBUTSEAo+kEQlwhp+ZAqHizhVTx4kg49ZUMtqWCF2PppJSlqRkPc55AoIkoFBJAXBeFFQKjvB8EzIS0fUmG1jgKP4sFLakVL4eBdMFgg5TOyEhWtRESsgADyJyZjNTS3NTwMwaXUC6EnISsfvE0o5g+15+9QKh5GkI5QEA2l+LpGSqREi/4hakdBAOmpGDU6oFL0gwgmQlY+9EKLNVq0Eg+epYNkgx0tryUrEQHYDt3lMQrCi4D4g6IfhF6EpHwYJerBq3jwKB1Glo0E8+XrecahzfoocnFdZxYpGpZREbECArCJggDiJISnETAEwRMhJx9G6WTKm3iwGrXCQjp4FY2WEqHWMXiRE1bREBcsoiJapmEA8akY1h1QpUY/KPVC8EjIyYdUtI56aLEwnBTxIOm4DAu5ULsOesgJaxEBlEVFtErDAOw6pOqZfgmYeolySF7ZliACQfLhB1ajW8QSjOKhVDq0FA4e5EIpYj6DmoLCMi3jQk5UhLcoiN79Pyj6QfAGyYdGyB1eGwgexYP3KEcwSIYSpHx+uaKiRjTERefw86KjIVpHQbQWEFZQx1NCa0g+fCAl6sEi5cLT5GFyxYNH6Qh10VAKi86wakZDxEZBpHRGBZRJiD9Yd0DVbPZTA6VefmnugKhmZU1bA63tojokHxrAOurBm3jwJB0kG+rhurY8RUPERkGkpGEAZYvUBer/oVcHVEq9EDxB8uEFraMeUuBJPHiRDh6FI8HEVjjPOPWdDbMlvEVD1EjDAPKjIHoICAuo4ymhJSQfHCAm5aL24nBSxIMH6eBBOFgLhpJz6SUnLKMhgHwZkZqGAaRFQQBpEqJkcToXUgSEFp8jjAbJRytYj3BhkXIJFvEwYpRDS8FQgr96aiEmrCZKU5qaUSsKAkhPxWjdAVWMgARKvVDHU0IrSD4UoEdHUzHwJh48S4dR5EIJYj4jS0FRGg1xITcqolZnVIB9h1Q9BEQRlHohGEHyoSJ6RD14Eg/epCMUREMuakROWE8bL7WfiFqdUQHxEsKi/wdrqOMpwQMkHzLhMerBi3jwIh0kG2xoeR15EBEpEqJmFAQQl4rRugOq0ugHdTwltIDkI0jgQTx4kA6ehSPBLL1BOOPQdjRVIFzXV0mahmVahpcoCAsBCQRX6Rc/OC0CTPYwXc5NGAeSDxmIiXpomXIJBvEIFumQIxhKjqeXnPASDZEaBdGzMypPM6BSx1NCb0g+dIJFykXNlWm1EA8jplZYy4VSeJATliKiREL0TsPwJiCqRj8o9UIoxCRnp4KCAvTo0QNRUVHIzMzE3r17/ZZftWoVrrnmGkRHRyM1NRVz5sxBQwP7UR5awFvUQyw8iUeCuY5JpEMt8UgwR/p8GQ2tP4vSv4vr3pBzf0i5F8Xe450j6iR9d6T+IPCGGiPgvKHmwplOi6DasYngQHLkY+PGjcjLy0NhYSEyMzOxatUqZGdno6KiAomJiW3Kr1+/HvPmzcOaNWswZMgQ/PDDD5g6dSrCwsKwcuVKJh/CaGj1cHHBi3jwFOkwokiwRsw1kBs50SsawkNn1EAREKPMgGrUjqe/NMXA0hSh6Bj2piZGtSF8ITnysXLlSsyYMQPTpk1D3759UVhYiHbt2mHNmjVey+/evRs33XQTJkyYgB49euC2227D+PHjA0ZLjIpaq9e2RuwvLB7Eg4dIh9EjGHrB4nrpEQ3ROwoS6PspJrIZ6EeKmGeN1ss/tISiH4Q/JMlHY2MjSktLkZWVdfkAJhOysrJQUlLidZ8hQ4agtLTULRtHjhzBtm3bMHLkSJ/nsdvtqK2t9XiFEqxSLryIhxJYpFdINpTDg4Rcqoe+AgLIX/W5NSy+50p/7KiZeiEIf0hKu5w9exYOhwNJSUke25OSknDo0CGv+0yYMAFnz57F0KFDIQgCmpub8cADD+CJJ57weZ5ly5Zh8eLFUqpGEARBEIRBkNXhVAo7d+7EM888g1deeQX79+/H5s2b8fHHH2Pp0qU+95k/fz5qamrcrxMnTjCrDw0fIwiCIAh9kRT56Ny5M8xmM6qqqjy2V1VVITk52es+Tz31FCZNmoT77rsPANCvXz/U19dj5syZePLJJ2EytfUfi8UCi4WfJcQJguCHBHMdk+naCYLQD0mRj8jISGRkZKC4uNi9zel0ori4GFar1es+Fy5caCMYZvOl6IMgUIckgiAIggg1JA+1zcvLw5QpUzBw4EAMGjQIq1atQn19PaZNmwYAmDx5Mrp27Yply5YBAEaPHo2VK1diwIAByMzMxOHDh/HUU09h9OjRbgkhCIIgCCJ0kNznIycnBytWrMDChQuRnp6O8vJyFBUVuTuhVlZW4tSpU+7yCxYswKOPPooFCxagb9++mD59OrKzs/HXv/6V3acgvCJ1kSyCIAhCHlIn39y0aRPS0tIQFRWFfv36Ydu2bR7vb968Gbfddhs6deqEsLAwlJeXtzlGQ0MDZs2ahU6dOqF9+/YYO3Zsm24RlZWVGDVqFNq1a4fExEQ89thjaG5u9jjPrbfeioSEBMTGxsJqteKTTz6RfyFEIqvDaW5uLo4fPw673Y49e/YgMzPT/d7OnTuxdu1a97/Dw8ORn5+Pw4cP4+LFi6isrERBQQHi4+OV1j1oUboAFUEQBKEdrsk38/PzsX//fvTv3x/Z2dk4ffq01/K7d+/G+PHjMX36dJSVlWHMmDEYM2YMvvvuO3eZ+vp6DB06FMuXL/d53jlz5uCjjz7Cpk2b8MUXX+DkyZO4++673e87HA6MGjUKjY2N2L17N9566y2sXbsWCxcudJfZtWsXbr31Vmzbtg2lpaUYPnw4Ro8ejbKyMgZXxjdhggE6XtTW1iIuLg7dlz8NU1RU4Mlrohx+3zZbfL8faNx7oEl7xIy7FzPDaaA5AFhPMsbrPB+A8llNaZ4PdrBYK0bJiriX6yEuqid21VtA3Eyn7rIiooqB1nkBAv/QCLTOC4CAM50GWt/F3wJzAUcHBpjh1LW6rbOhAcfnLkBNTQ1iY2P9H1MmrnbioS/vgqW9whlO65rw4tC/i65vZmYmbrzxRrz88ssALvWFTE1NxezZszFv3rw25XNyclBfX4+tW7e6tw0ePBjp6ekoLCz0KHvs2DH07NkTZWVlSE9Pd2+vqalBQkIC1q9fj3vuuQcAcOjQIfTp0wclJSUYPHgw/t//+3/4zW9+g5MnT7qzE4WFhZg7dy7OnDmDyEjvf/trr70WOTk5HpLCGtWH2hIEQeiF0tWWidCm9WSXdnvbH0JyJt8sKSnxKA8A2dnZPst7o7S0FE1NTR7HSUtLQ7du3dzHKSkpQb9+/Tzm5srOzkZtbS0OHDjg9bhOpxPnz59Hx44dRddFDrSqLUEQokkwR2qyUi5ByOU/Te0Q2aQs2tnYdOkeT01N9dien5+PRYsWeWyTM/mmzWbzWt5ms4muo81mQ2RkZJsuDC2P4+s8rve8sWLFCtTV1eHee+8VXRc5kHwQBEEQhBdOnDjhkXYJ9vmn1q9fj8WLF+Pvf/+714ViWUJpF8aovdKkWkjJdxOEElitTEwQahMbG+vx8iYfcibfTE5OllTe1zEaGxtRXV3t8zi+zuN6ryUbNmzAfffdh/fee69NSkgNSD4IgiAIQiZyJt+0Wq0e5QFg+/btPst7IyMjAxERER7HqaioQGVlpfs4VqsV3377rceom+3btyM2NhZ9+/Z1b3v33Xcxbdo0vPvuuxg1apToOiiB0i4EQRCcEm9pEDXihdAXqZNvPvzwwxg2bBief/55jBo1Chs2bMC+ffvw2muvuY957tw5VFZW4uTJkwAuiQVwKWKRnJyMuLg4TJ8+HXl5eejYsSNiY2Mxe/ZsWK1WDB48GABw2223oW/fvpg0aRKee+452Gw2LFiwALNmzXJHcdavX48pU6bgL3/5CzIzM919QaKjoxEXF6faNaPIBxH0UAdJQgxShpuLQexweML4SJ18c8iQIVi/fj1ee+019O/fH++//z62bNmC6667zl3mww8/xIABA9yRiHHjxmHAgAEeQ3FfeOEF/OY3v8HYsWPxP//zP0hOTsbmzZvd75vNZmzduhVmsxlWqxW/+93vMHnyZCxZssRd5rXXXkNzczNmzZqFLl26uF8PP/ywatcLoHk+vOJvro9A83wAgef64HGeD0D8w9do83xcqgfN9cEKHub6kLKwnBpzfYidPTjQXB9iJhQMFPmgeT4u4WonJn0+HpHtFY52qWvE34a/q2p9Qx2KfBAEQRAEoSkkH4RkpPySJAiCIIjWkHwQBEEQXgmUoiYIuZB8GBQx60bwgJTcPEEQBBEakHwQBEFIREpHboIg2kLzfBAEQRBBw7nGGEQEGOETiKZGZaviEoGhyAdBEARBEJpC8kEQBEEQhKaQfBAEQQQxYiZGJAitIfkgCIIgCEJTSD50gBaKIgiCIEIZkg+CIAiCIDSF5EMigRZpIgiCIAjCPyQfBEEQBEFoCslHkCN26W+CILSnk6Ve7yoQhC6QfBDcc8Zp0bsKBMEt7S12vatAEJIh+SAIwnAkmGltFYIwMrS2C0EQBBE01NgtCA9XFi1ttocxqg3hC4p8EARBEAShKSQfBEEQBEFoCskHQRAEQRCaQvKhAnV25aMzfrHHMKiJNM42dxBd1tYUL7rsGUd7nHEoG/KrdMTLGQctrsUCFteRxeglpfcTL7D4nrN43hCE1hiyw6nJHganRfBdoMEMRDl8vu2wm2G2+H5fC6rtUYi3NPgt84s9xu88AOeaYtAxIvA8Aa65PjpHBB4h4BKQzuHnA5a1NcUjOaI6YDkXrgZD7kiFlo1Wgkn68EJvDWeCmWasFYNRpUOKJEuRb5o/hyCUYUj5UBt7QyQsUb4ftvX2yIDLVNfZLQHH32spIMClB6YYAQEuPYjFCggAWRICKBcRORLiWRfff8dQFRPWUSK9Ih16i8e5JuVRDVqEkghWDCsfSqMfWsBKQAIhVUAA8VEQMQICSI+CuNA7GuL32EEqJlqloPRMr+gtHmLQKuVC61ERPGJY+VBKoNQLi+gHwEZAAkU/AGkCAoiPgkhNwwDSoiAueIqGiDoXp2LCQ98Wvft08CAeFPUgCP+ErHywgKWABEItAQH4iYK44DkaIur8AQRAiZzwIBfeYDUFvtKOpEYRD4p6EKGOoeXDSB1PAwkIi/4fwOUHHw9RECUCAhgvGiIWXgVCDnpHOVpiFPEgCMLg8qE2gVIvgPjohxhYCQggXULU7IwKyEvFtMTo0ZBggpcohwsp0gEYQzxYpFwo6kHwTPDLhwYdT7Xs/wGIFxBA3c6ogLgoCMBORFhGQ3xBcuIJ61WFWUiHVOFwwYN40Nwe6lLfaIFZ4douQRSc5Jbgl48AKO146kJrAZECD1GQlrRuOOTKiNJoiM/j+mlsg1VMWAuG13PoKB0AH+IhBop6EKFAyMuHHmg1AqY1UiRErc6o3lAySgZgEw0RfS6DRk20kAuf59ZZOgB+xIOiHgRxidCQD4UdT1lHPwDtRsB4Q42JyaSmYbzBIjWjVjRE9Pl1iJroKRb+4EE6gOASD4IIFkJDPjSEpYCw7v/REl6jIC6UioiW0RCxBBITXiVCCnp1IvWGFOkA+BAPrVMu9gZKzxD6EDrywcGMp97QU0AAfqMgLWGVluFFQrxhVPFgucAbC+EApEsHwId4iIFSLkSwEDryEQBWqReA7fBbQBsBAcRHQdTsjOoPltEQFzwLCY+osZqsntIB8CMeFPUgQgmSD5Vg3f9DbQEB1F0lF2AXCQGUR0NciG1MQ1FS1F62Xo/USpv9OR5O2xqKehDBhEnvCmhKg9nv2w67//el/lKQ8itEzINFzC8jpQ89KQ9YqYtsnW3u4PFiga0p3v1SkzOO9n5fRkSvz8Ti78XiHjKSeLCGoh7sKSgoQI8ePRAVFYXMzEzs3bvXb/lNmzYhLS0NUVFR6NevH7Zt2+bxviAIWLhwIbp06YLo6GhkZWXhxx9/9CjTo0cPhIWFebyeffZZ9/sVFRUYPnw4kpKSEBUVhV69emHBggVoamqSVBc1CC354ByWAqLkAXiuKUb0w/ZsU3vZK32qJSJ6wKOY8CZMvEgHwJ94aJlyIfFgz8aNG5GXl4f8/Hzs378f/fv3R3Z2Nk6fPu21/O7duzF+/HhMnz4dZWVlGDNmDMaMGYPvvvvOXea5557Diy++iMLCQuzZswcxMTHIzs5GQ4NnBHzJkiU4deqU+zV79mz3exEREZg8eTI+/fRTVFRUYNWqVXj99deRn58vqS5qECYIgp/FUfigtrYWcXFx6L78aZiiPL+kftd28UWAjqf++n6I7ffREqn9PwKlYKROQKYkFSNljRixfUECHodhekZpWkZtAqVzjBpVAdj15QCUp1cA6ZE6QH3xECsdLBeRkyoffiPCAaLJJnsYAMDZ0IDjcxegpqYGsbGxks4vFlc7MeD9PJjbKZzh9IIdZfesFF3fzMxM3HjjjXj55ZcBAE6nE6mpqZg9ezbmzZvXpnxOTg7q6+uxdetW97bBgwcjPT0dhYWFEAQBKSkpePTRR/HHP/4RAFBTU4OkpCSsXbsW48aNA3Ap8vHII4/gkUceEf3Z8vLy8M033+Af//iHqLqoBUU+JCLnV4PU2QYDPWik/krSMgrCApbpGT2jIWLgLTqhhJYpMJadSFmkV4wsHmLQRTxCgNraWo+X3d72h2FjYyNKS0uRlZXl3mYymZCVlYWSkhKvxy0pKfEoDwDZ2dnu8kePHoXNZvMoExcXh8zMzDbHfPbZZ9GpUycMGDAAf/7zn9Hc3Ozz8xw+fBhFRUUYNmyY6LqoBXU41Qg9RsC0xPVwVHtIrpTOqGJh0WmV5SJ3BNuoRmtYpeGUyDBP4kEdTaVRb4+E2awsteSwX4qop6amemzPz8/HokWLPLadPXsWDocDSUlJHtuTkpJw6NAhr8e32Wxey9tsNvf7rm2+ygDAQw89hBtuuAEdO3bE7t27MX/+fJw6dQorV6702G/IkCHYv38/7HY7Zs6ciSVLloiui1qQfHiB5bBbuai1BowSCVFrSK4UWIuIN0hO2qJF9IgH6QD4Eg+WUNRDOidOnPBIu1gsfIlgXl6e+/+vv/56REZG4v7778eyZcs86rpx40acP38e//znP/HYY49hxYoVePzxx/WoshuSDw2RGv1QcxE6LSYma9kIqCkiLPuIAOIb2mCUFD1SVHr15/CGmuKhlnSosYgcicclYmNjA/b56Ny5M8xmM6qqqjy2V1VVITk52es+ycnJfsu7/ltVVYUuXbp4lElPT/dZl8zMTDQ3N+PYsWO45ppr3NtdEZy+ffvC4XBg5syZePTRR2E2mwPWRS2oz4cPWA+7dcG6/wcg/6GmZFSM1MmWXHl3Vo2Ex7EZj5oRS+s+Dmr0eWANL3Vm9TdjdU9J6dsEaCcerFIuNMJFPSIjI5GRkYHi4mL3NqfTieLiYlitVq/7WK1Wj/IAsH37dnf5nj17Ijk52aNMbW0t9uzZ4/OYAFBeXg6TyYTExESfZZxOJ5qamuB0OkXVRS0o8qEDakRAlCA3FSMlDdMSNSMiakVD5OCvMZcbNeFVagLBWgz1iHS40EI8xEoHRT34IC8vD1OmTMHAgQMxaNAgrFq1CvX19Zg2bRoAYPLkyejatSuWLVsGAHj44YcxbNgwPP/88xg1ahQ2bNiAffv24bXXXgMAhIWF4ZFHHsHTTz+Nq666Cj179sRTTz2FlJQUjBkzBsCljqJ79uzB8OHD0aFDB5SUlGDOnDn43e9+hyuuuAIAsG7dOkRERKBfv36wWCzYt28f5s+fj5ycHERERIiqi1qQfChAi74fLlisARMILVbJbU3rRoSVjKg1syorjCoRYlErCsUqyiEXnsRDLBT1UJ+cnBycOXMGCxcuhM1mQ3p6OoqKitwdOSsrK2EyXU40DBkyBOvXr8eCBQvwxBNP4KqrrsKWLVtw3XXXucs8/vjjqK+vx8yZM1FdXY2hQ4eiqKgIUf+dbsJisWDDhg1YtGgR7HY7evbsiTlz5nj0AwkPD8fy5cvxww8/QBAEdO/eHbm5uZgzZ46kuqgBzfMRAH8dT13IFRDW83+4UCohgPxRMXIlxBusoyI8SkgwoHa6S88oR0u0WKdFqniIiXxIkQ/JUQ8O5/m4et08JvN8/DDxWVXrG+pQ5IMBciMgaqVfXA8+JRKiJBXDSkBYp2e8NZIkJNLRqm8NL9IBhIZ4SCaAeBCEP0g+AhBo2K1S1Oz/oVcqRm5fEH9okZ7xR6hKitadeAHjSgegf8fSltDQWoJnSD4YwVP/j5YESxSkNWoP421zviCWFD0Ew+P8DEdAsRAOQLv5O+RKhxodTSVBUQ9CISQfDNEq/QJIHwGjl4S0bAyCRUT8IaYhZykoeouDHNQYbq2ndAD8iQdFPQjeIfkQgZTUC88CAug/KsaFFiIC6C8j3jCiMMhFDdFoid7SAWgnHrpHO1xQ1INgAMlHCKJnKsZF60ZDbRnhUUKCEbVlw4Ue/Tm8EQziEWxRj8aGCJhMyq6Xs8HJqDaEL2TNcFpQUIAePXogKioKmZmZ2Lt3r9/y1dXVmDVrFrp06QKLxYKrr74a27Ztk1VhvZDyBdVq9lNAWUe1anuU4imflcyS2hLXDJOsfsm2Rs0ZVkOZltdViygHi3uE1T0r9btTZ7doIh56Dq0lCLFIjnxs3LgReXl5KCwsRGZmJlatWoXs7GxUVFR4ndK1sbERt956KxITE/H++++ja9euOH78OOLj41nUn1t4T7+0RM9UjDfUTs8EaiQpSuIdPcSNpYyyEA6Av/4dvOKa44MgvCFZPlauXIkZM2a4p40tLCzExx9/jDVr1mDevHltyq9Zswbnzp3D7t273dO59ujRQ1mtDYLRBATQNxXjDa3SMy0R28gGq6ToGR1SI/KlV3rFhZZpFop6EEZBknw0NjaitLQU8+fPd28zmUzIyspCSUmJ130+/PBDWK1WzJo1C3//+9+RkJCACRMmYO7cuTCbvd/MdrsddvvlhrS2tlZKNUMW10MuGCXEhRadVsUippHWWlCMllZSK82mZ5SjJbyKh2QkigdFPYhASJKPs2fPwuFwuOerd5GUlIRDhw553efIkSPYsWMHJk6ciG3btuHw4cN48MEH0dTUhPz8fK/7LFu2DIsXLxZVJ5M9TPoU6w1mSVOsK8H1QJAaAXE9gKRGQAD+JMQFaxnx1nDpLSStMZoMqIVaktEaXqQDCCLxIAgVUH20i9PpRGJiIl577TWYzWZkZGTg559/xp///Gef8jF//nyPxXFqa2uRmprq8xxaCYiS2U71lBAXcmSk9UOYpYy4UDNN44I3KQlWtJKMlvAkHID2/TtIPAgjIkk+OnfuDLPZjKqqKo/tVVVVSE5O9rpPly5dEBER4ZFi6dOnD2w2GxobGxEZ2faLY7FYYLFI+wIbQUAAfSTERcuHotyoCEsZcaGnlPiDhMU3RpaMlrASDhdaigdJB2FkJMlHZGQkMjIyUFxcjDFjxgC4FNkoLi5Gbm6u131uuukmrF+/Hk6n072k8A8//IAuXbp4FQ8laCkgrZEqJHpKCMAmKgJ4PrxZiEhLtJASfyhtYHmTFz2EQQ5qSEZr9EqreIPEgwhFJKdd8vLyMGXKFAwcOBCDBg3CqlWrUF9f7x79MnnyZHTt2hXLli0DAPzhD3/Ayy+/jIcffhizZ8/Gjz/+iGeeeQYPPfQQ20/yX/TqA+ISEq0lBFAuIgB/KRp/+GuctBITMRilsdcLLSSjJXqnVbxB4kGEKpLlIycnB2fOnMHChQths9mQnp6OoqIidyfUyspKd4QDAFJTU/HJJ59gzpw5uP7669G1a1c8/PDDmDt3LrtP0Qo9O6G2jIpIEZGWDxS9oiEtUSNF4wuWkqJ3tIRoi9aS0RIehcMFiQcRysjqcJqbm+szzbJz584226xWK77++ms5p5IND6Ng9IyG+EKvjqv+8NVAaCElYiF5CYyektEantIq3iDxIEKdoF7bhQcBAbSXEH+wSNeoLSMutJASsQSLvPAkCFJg3THUH2pKB0DioTaORjMEH3NIicXZSBOqqU1QywegQECAoJYQoO1DkIWMqCUiLeFJSsRi1EZfa7SUjJaoLRwutBSPYFswjggugl4+AJkCAqg2GRlvEuKChYxoFRXxhr+Gi2cxCUX0koyW6DlaRSwkHkSwEhLyAfAnIIByCQHUExFAnRSNFFiKC6vGjiRGGjxIRmtYSIcWC77JEQ+SDsIohIx8AHwKCCB/hAygfjTEBasUjRR8NRJaRlNao7Qx5UVeeJQCNTFClKMlJB5EsBNS8gFcXvCIh46o3mARDZGCXGlhPc+IFHiUErGEWqPPAq36Y/hC62XtSTyIUCDk5MMFLyNhfCFXQqTS+kEnR0b0iIp4w8hSQugvGS3RWjhckHgQoULIygfAv4AA2kmICzVkxB96pm+UQEIjH54kozV6SQdA4kGEFiEtHwBDAWmQ+BCQuZ6MVhLiQu3OrXIe9npFVlqitAHlRV54FgGt0FM4XJB4EKFGyMsHoHAuELnInEtELwkB2ERFWOCrseBBSsRCjb6+sBIOvSb/IvEgjA7Jx3+RPRJGKQolBNBHRADthvyKJRikhPCEh6hEa/SebZTEgwgGSD5aoJuAAJ6RFANFQ1zwEhXxhpoNGImNcngUDG/oLR0AiQcRPJB8tEJXAXHBIBrCCrlCo9aDmiepAdg0nDwIjNoCEGNpNIxktIQH4XChm3goTTFrTYMZCGOUFidUg+TDC1wICKDaGjNSaP3A0zO6AvhvDHgTE7EYsVGWipE+I0/C4YKZeFCjSnACyYcPuBEQgAsJccGbjLTEV6NhVCkh2iJFDKT+3XmUDkCmeJBkEJxD8uEHrgQE4EpCXPDQ8TUQoZICCiZY/M20lAmu+mKQeBAGgOQjANwJCMClhAB8R0XUINikhsXnCZZogxi4Eg4XJB6EQSD5EIHs9WDURsEIGS3Q8uEcTKJj5AbZyHUXA5fC4YLEgzAQJB8S4DIK4kLLB4/BRCeYxITQB66lAyDxIAwHyYdEuBYQrWj9oONQRlrCsuEgkdEP7gVAL0g8CANi0rsCRsSVhmF5vEAvrmkwX34FOQ67mYsXD59R6/MTXgiB75xRKCgoQI8ePRAVFYXMzEzs3bvXb/lNmzYhLS0NUVFR6NevH7Zt2+bxviAIWLhwIbp06YLo6GhkZWXhxx9/9Chz7tw5TJw4EbGxsYiPj8f06dNRV1fnfv/YsWMICwtr8/r666/dZZqamrBkyRL07t0bUVFR6N+/P4qKihhcEf+QfMhEjDCIfUk5H/e0FBF6MKoGD2Kj9/lDHvp+ccPGjRuRl5eH/Px87N+/H/3790d2djZOnz7ttfzu3bsxfvx4TJ8+HWVlZRgzZgzGjBmD7777zl3mueeew4svvojCwkLs2bMHMTExyM7ORkNDg7vMxIkTceDAAWzfvh1bt27Frl27MHPmzDbn++yzz3Dq1Cn3KyMjw/3eggUL8Ne//hUvvfQSvv/+ezzwwAP47W9/i7KyMoZXqC1hgiBwn0Oora1FXFwcui9/GqaoKL2rwwUhn/oRC+cpISIIIAloQ+sfSs6GBhyfuwA1NTWIjY1V5ZyudiL1haUwRStrJ5wXG3BizlOi65uZmYkbb7wRL7/88qX9nU6kpqZi9uzZmDdvXpvyOTk5qK+vx9atW93bBg8ejPT0dBQWFkIQBKSkpODRRx/FH//4RwBATU0NkpKSsHbtWowbNw4HDx5E37598c0332DgwIEAgKKiIowcORI//fQTUlJScOzYMfTs2RNlZWVIT0/3WveUlBQ8+eSTmDVrlnvb2LFjER0djXfeeUf0NZMKRT4MimFSMnrTOhIj90UQLaF7IySora31eNnt9jZlGhsbUVpaiqysLPc2k8mErKwslJSUeD1uSUmJR3kAyM7Odpc/evQobDabR5m4uDhkZma6y5SUlCA+Pt4tHgCQlZUFk8mEPXv2eBz7zjvvRGJiIoYOHYoPP/zQ4z273Y6oVj/qo6Oj8eWXX/q8LiygDqdBALdDgYMJIzYySqM+PA3lNuL1DxF4+wFkagyDyaSwTo2X9k9NTfXYnJ+fj0WLFnlsO3v2LBwOB5KSkjy2JyUl4dChQ14Pb7PZvJa32Wzu913b/JVJTEz0eD88PBwdO3Z0l2nfvj2ef/553HTTTTCZTPi///s/jBkzBlu2bMGdd94J4JL0rFy5Ev/zP/+D3r17o7i4GJs3b4bDoe53nuQjiODlIUASxAksG2xq/EMGXp4jPHDixAmPtIvFYtGxNtLp3Lkz8vLy3P++8cYbcfLkSfz5z392y8df/vIXzJgxA2lpaQgLC0Pv3r0xbdo0rFmzRtW6UdqFYI5anW4JgpAGfReVERsb6/HyJh+dO3eG2WxGVVWVx/aqqiokJyd7PW5ycrLf8q7/BirTukNrc3Mzzp075/O8wKX+KYcPH3b/OyEhAVu2bEF9fT2OHz+OQ4cOoX379ujVq5fPY7CA5IPgApajh7QchUQQPEL3sHZERkYiIyMDxcXF7m1OpxPFxcWwWq1e97FarR7lAWD79u3u8j179kRycrJHmdraWuzZs8ddxmq1orq6GqWlpe4yO3bsgNPpRGZmps/6lpeXo0uXLm22R0VFoWvXrmhubsb//d//4a677hLx6eVDaReC8EEoPLwpRaYfoXB/hQp5eXmYMmUKBg4ciEGDBmHVqlWor6/HtGnTAACTJ09G165dsWzZMgDAww8/jGHDhuH555/HqFGjsGHDBuzbtw+vvfYaACAsLAyPPPIInn76aVx11VXo2bMnnnrqKaSkpGDMmDEAgD59+uD222/HjBkzUFhYiKamJuTm5mLcuHFISUkBALz11luIjIzEgAEDAACbN2/GmjVr8MYbb7jrvmfPHvz8889IT0/Hzz//jEWLFsHpdOLxxx9X9ZqRfBBECEMNIEEoJycnB2fOnMHChQths9mQnp6OoqIid4fRyspKmEyXEw1DhgzB+vXrsWDBAjzxxBO46qqrsGXLFlx33XXuMo8//jjq6+sxc+ZMVFdXY+jQoSgqKvIYmbJu3Trk5uZixIgRMJlMGDt2LF588UWPui1duhTHjx9HeHg40tLSsHHjRtxzzz3u9xsaGrBgwQIcOXIE7du3x8iRI/G3v/0N8fHxKl2tS9A8HwRBEISqaDnPB4t2Qov6hjoU+eAA00XqesMrzmin3lUgCIIIOkg+FELiENyEyt+XJIsgCC0h+ZBBqDRIROjA4z1NQkQQwUvIywePD12CIAJ/N0lOCMK4BJ18kEzwi7ntsghBg8NYEx8GBTx910mE+MF00QSToPDeaODn3gpWDC8fPD2AeCSYG3yeMOJ1JmFih+s5RBJCEOIwvHwQbTFiQ0hoj9z7hKTFNyQhBCEOkg8VIQkgghG972sjyA9JCEH4h+RDJfR+QBsFc4M253F4mXNIq3P7qwMhnZbfLd5FRKyEsE4fk/QQvEPyoQIkHm3RuqHn7fy81CHYBKj1d41XGdG6bxpFXgjeIflgDO/iwUMDSOiH0r8/7/JiFBnRCtNFEwkIwSUkHwzhVTxIOAhW+LuXeBQTLb+TvIoOCQjBIyQfjOBRPIwqHeEK690ssRFUej5WSK03b2h5v/EsOjxKCAkIwRskHwzgSTyMJBxqNfq8yIRU1Kq30aXGGy3vc95EhFcJIQEheILkQyFyxINXQTBqo034h+XflUeRaf194kVGeJQQEhCCF0g+AsA6qsGbeJBwEFLwdb/wJCW8RUV4kxASEIIHQk4+9EyR8CQeRpMOX9eOVePC09+GhwZTKv7uJz3FhKeoCE/zkwSzgJjsgDlM4UE4SqUHKyEnH3rBQ+NmBOGQep14uK6sYfGZeBIYnvqyqHW/SL3ePERDgllACP4h+dAAPRtIXoUjGKWBJ3hpZNWk5b2td9rHdb2NJiEkIIRekHyojF6NLAvpIEEgWqN2+ksure93vWTEiBJCAkLoAcmHishpvHmIVISydIQ3CHpXwU1zlNLEtXbwJiV6R0W0khBffdikSgwJCKE1JB8qYTTx4Fk4eBICLVH6uXmQFx6mc9czKqJUQmSf1y5PQABaD4bQBpIPFTCKePAmHKEqGWrB+nrqITNqTOeuh4zoMfxXjoAAFAUhtIHkgzFGEA+9pYMkw5j4+7vxICYsZERLEdFCQkhACF4h+WCE3AZdK/FgJRyhIg68fU4eUij+8HW9tKw3CxlR8n2UKi5aSQgJCMEjJB8M4FU8glk4eKyTmhi1/4eeUqJ1qsP1feZRQuSOpiEBIdSC5EMhPIoHC+ngoXHnoQ7BAotryVIYlNRHTj20nOmUdwkhASF4gORDAVqKh1b9NPRo8EkyjAEPqZXW9ZB7bi1khFcJIQEheIDkww9qNPg8iodWjb8RJSP8Al8P3OZ2Jr2r0AY9O6K2PjcLGWHd6CuVEIB9nYJZQMx2wKz0ILS2i+qQfGgIT+LBSgR4EQreJEEtlH5OreVF62gJCxlRKyqiZFSNGtGQYBYQgn9IPjSCF/FgIQt6CkeoSIZaKLl+LMVFKylRI0XjDalSwEtKhgSE0AuSD5WR27GUpXgYMcpBksEfvv4mWkhJm3PKEAlWKRpvyJUCHiSEBITQA1lPjYKCAvTo0QNRUVHIzMzE3r17Re23YcMGhIWFYcyYMXJOazj0FI/wBsH94uE4Po9/wen1xRPhF53cvXhCj78hi/uy5TFY3d/mBvkTDcqNjrJ4XsiZzt01HTtByEFy5GPjxo3Iy8tDYWEhMjMzsWrVKmRnZ6OiogKJiYk+9zt27Bj++Mc/4uabb1ZUYaOgpXiwFgM1REMPoeCtkWaJ0s/WHK1+w+Hvb65mtERuRINFisaFXpGQlshZT4bWgyG0QvITYOXKlZgxYwamTZuGvn37orCwEO3atcOaNWt87uNwODBx4kQsXrwYvXr1CngOu92O2tpaj5eRMKp4MImU6PEr2ADRAd7QO9qi5n3CIqLBKiqidSSk9bkl7yNzlAdFQQipSLpjGhsbUVpaiqysrMsHMJmQlZWFkpISn/stWbIEiYmJmD59uqjzLFu2DHFxce5XamqqlGrqitHEg1l6hiQj6DGqlLBO0chBqYRo+VwhASG0QNLdcvbsWTgcDiQlJXlsT0pKgs1m87rPl19+idWrV+P1118XfZ758+ejpqbG/Tpx4oSUavpE6TLVgTCSeLCOcrDGKJJhvtis+Ys3tPg7sZIRvUVESR8NLRegVPtZGcqcO3cOEydORGxsLOLj4zF9+nTU1dX53aehoQGzZs1Cp06d0L59e4wdOxZVVVUeZSorKzFq1Ci0a9cOiYmJeOyxx9Dc7P158dVXXyE8PBzp6eke23ft2oXRo0cjJSUFYWFh2LJli8f7TU1NmDt3Lvr164eYmBikpKRg8uTJOHnypOTroKqqnj9/HpMmTcLrr7+Ozp07i97PYrEgNjbW4+UNnkxby85iLELJSlBDOPQQDaOKAK/1cqGVjCg+ho7Dzo0gIIQ6TJw4EQcOHMD27duxdetW7Nq1CzNnzvS7z5w5c/DRRx9h06ZN+OKLL3Dy5Encfffd7vcdDgdGjRqFxsZG7N69G2+99RbWrl2LhQsXtjlWdXU1Jk+ejBEjRrR5r76+Hv3790dBQYHXely4cAH79+/HU089hf3792Pz5s2oqKjAnXfeKfEqSOxw2rlzZ5jN5jbGVVVVheTk5Dbl//3vf+PYsWMYPXq0e5vTeemhER4ejoqKCvTu3VtypeUg1eSlPBy0nMNDzsOO2VBbVjl5HaIYejfIWiP28zqi1R9t3/LvzbKjq+t+VNJ51fXdUNLBNLxB4H7VYYIPDh48iKKiInzzzTcYOHAgAOCll17CyJEjsWLFCqSkpLTZp6amBqtXr8b69etxyy23AADefPNN9OnTB19//TUGDx6MTz/9FN9//z0+++wzJCUlIT09HUuXLsXcuXOxaNEiREZGuo/3wAMPYMKECTCbzW0iG3fccQfuuOMOn/WPi4vD9u3bPba9/PLLGDRoECorK9GtWzfR10LStzYyMhIZGRkoLi52b3M6nSguLobVam1TPi0tDd9++y3Ky8vdrzvvvBPDhw9HeXm5or4cakY91J7OXG60Q6pEsBpCyCTvrnNUg/CO1hEUNe4Dlv1ClOwvFYp+8E/rgQ92u7J8VElJCeLj493iAQBZWVkwmUzYs2eP131KS0vR1NTk0dcyLS0N3bp1c/e1LCkpQb9+/Ty6RGRnZ6O2thYHDhxwb3vzzTdx5MgR5OfnK/ocLampqUFYWBji4+Ml7Sf5Z09eXh6mTJmCgQMHYtCgQVi1ahXq6+sxbdo0AMDkyZPRtWtXLFu2DFFRUbjuuus89ndVsPV2NVEzfyn1QaBVmoVFaoUFWsgGyYV6uK6tGhGS1veG0qhI+AWn4iG8SqIYcvY1N8ibKCy8QfxwXLnnMCrhDYBZ4W+usP+2Ga1/IOfn52PRokWyj2uz2dpMSREeHo6OHTv67Ddps9kQGRnZpnFv2dfSZrN57Yvpeg8AfvzxR8ybNw//+Mc/EB7O5vvc0NCAuXPnYvz48T67R/hCcg1ycnJw5swZLFy4EDabDenp6SgqKnJ/0MrKSphM/PTFkIpaUQ+jpFmMIB0kG9rT8pqrlaphkZ7ROxWjZQpGioAQ8jhx4oRHo2qxeJ8IZd68eVi+fLnfYx08eJBp3aTgcDgwYcIELF68GFdffTWTYzY1NeHee++FIAh49dVXJe8v6ymSm5uL3Nxcr+/t3LnT775r166Vc0rZ8BD1MIJ48CwdRpMN84UmZsdytItgdixWaC0igHQZ0VtCpKBFZCLUoh+s8DfgoSWPPvoopk6d6rdMr169kJycjNOnT3tsb25uxrlz57z2mwSA5ORkNDY2orq62iP60bKvZXJycpuZxl19M5OTk3H+/Hns27cPZWVl7rbb6XRCEASEh4fj008/dfcnEYNLPI4fP44dO3ZIjnoAtLaLB2pEPXgXDyajBhgKh96iwVIcWCCmPnoKihYiAly+x+RIiNapGF7TL4R6JCQkICEhIWA5q9WK6upqlJaWIiMjAwCwY8cOOJ1OZGZmet0nIyMDERERKC4uxtixYwEAFRUVqKysdPe1tFqt+NOf/oTTp0+70zrbt29HbGws+vbti4iICHz77bcex33llVewY8cOvP/+++jZs6foz+oSjx9//BGff/45OnXqJHrflgS1fOgd9dB6GK3ospxFObQWDt4EQyliP4/aktL676hmPxEpEqJHFIRHAaHoh/706dMHt99+O2bMmIHCwkI0NTUhNzcX48aNc490+fnnnzFixAi8/fbbGDRoEOLi4jB9+nTk5eWhY8eOiI2NxezZs2G1WjF48GAAwG233Ya+ffti0qRJeO6552Cz2bBgwQLMmjXLnSpq3c8yMTGxTb/Muro6HD582P3vo0ePory8HB07dkS3bt3Q1NSEe+65B/v378fWrVvhcDjcfUo6duzoMaomEEErH2oOrVULo4gHC+nQUjiCTTbkorWkqBkVCb/o5D4VQ/0/CG+sW7cOubm5GDFiBEwmE8aOHYsXX3zR/X5TUxMqKipw4cIF97YXXnjBXdZutyM7OxuvvPKK+32z2YytW7fiD3/4A6xWK2JiYjBlyhQsWbJEUt327duH4cOHu/+dl5cHAJgyZQrWrl2Ln3/+GR9++CEAtJmg7PPPP8evf/1r0ecKEwRBu3XSZVJbW4u4uDh0X/40TFGXvmGBhtrqPa+HVJnRIs3CYrisUtSWDhIN9qgRMWEpI3I6p7JY2E6sWEgVELnRCTHyIeXYUheZA3wvMOdsaMDxuQtQU1Mjq3+AGFztRN8Hn4HZoszEHPYGfP/KE6rWN9QJ2siHFPSOevAsHrynVkg21Md1jVlKCMuoCO9REKkREDXTL5R6IXiB5EMiLCf40SLNAsgTDx6lw6iiYaqX3vnIGSPjZ6fKtLz+vImIHAEB2EkI6/QKdUAlgp2Qlw+9RrjwKh48SQdPsiFHINQ8n95yooWIANJkRO6IGED5qJhAAsJT/w+KfhA8ENLyIb1fBpvz8jiMlqf+HFpLh9ZiwQKe5ESNtIz72DKiInpFQVgLCEkCEcyEtHyogRqRFN7FwyjSYUTJkIvYz8pSUtSKhriPL2Gqd6VREIBNp9Q2x+ao/wdB6EnIyodRoh48iwfvqZVQkg25qBVBUTsaonYUBJCXihEjFzxEQII5qmK2K1/bBY1MqkL4IWTlwwjwKh68SgfJBntc11SphLhgPYcIj1EQNQSEIIKNkJQPtaIeLFMuvIkHb6kVQ4tG/UX/78dEa1MPCbS83kpSNaxTM1pGQQDxEsJaQOREKqjjKcEzISkfUtB6aK0WC8NJEQ+epIM74QgkEWoclwMxURoNccFKRKRGQeQKCCAtFcM6uqG1LJjt8iYaIwgxkHwwgkXUIxjFw9DSoZZcKIEjMWEVDQHYpGfERkGUpGEAaVEQvUfAUMdTgldIPvzAw4Rifo+p4hweSsTDEKkVHsVCKTqmc1hFQ1zIjYpoHQUBlI+K0bP/RyCZoegHoRYkHwaFR/HgOsoRjLIhFbHXQIGksIyGuJAjIlpFQYDAqRge+n8QBG+QfPhAStRD65QLb+LBnXSQaCij9fWTKSOsoyGAtCG8WnVGFXV8HQVEacdTin4QakDyoQEsUy48iQc30mFw2RD+u3R2WLt2OtfEB67rq1BCALbRELECAmgzJDdQ+oWG4BLEZUg+vMBj1EPtxeGkiAdJh29cIsF6Xy7EpOX15iAaIlZAAG2iIKwERCwsox8EoTUkHyrDIuoRTOJhdOlQIhdqnVcXMVEoIqznDeElCsJCQPTo/0GpF0JrSD6CEB7FwyjSoZdcKEF3MeEgLcNTFETpCrkAdUAlgh+SDwXoObeHz+NxJB7MOpEylg4jCoZcAn1WpnLCMC0DSBcRNTujAtKiIFqPgBFVpxDpeBreIMDsVPZcDWtkPzUC4QnJh4ponXLhRTx4inKEkmjIQTU5URgNAdreR2JlRI3OqIB0CdFSQCj6QRgNkg+Z8Bb14EE8eJEOEg52KB6JwyAa4kJKZ1W1oiAAm7lB3MfSWEBovReCF0g+VELLqEdQiEcQS4dQVy9737D2MQxrIp+W11bPaAhw6V7TOwoCiOsPwvMIGFnHD5B6MV00wRktb7VsIrQg+ZABy9VrlWJ48QgC6VAiF0qOrZeY8BANkRoFUaMzKqCdgND8H0SwQfIhEVbiwSQyYmTxMJB0qCkXSghUL7XlhIdoiNgoiFpDcgF2AhLwPIzSL2p3PKXoByEGkg8V0GtuD19wIx6cdiLlVS6UomXURM9oCA9REBYCQjOgEqEEyYcOaBn14EI8OIpyBKtoSEUtMdEzGqJWFMSIAqJ29IMglELywRieoh7BIB4spIOEQxqtr5dcGdEjGqJ3Z1QWC9QZIQJCqRdCKeot40gQHK69QkhHqbwxiVpJuJekyDGLdYqkwjKlShBGheSDIAiCIAhNIfkgCIIgCEJTqM8HY5qjwpiv12JYYqIp9UIQhKaY7QLCFa7tgiZ6hqsNRT44ROl8AARBEATBM9TKEQShOnrPQssCKaPHCILwD8kHQRBEEMLTMhAE0RqSD4IgCIIgNIXkgyAIgjOo0zoR7JB8EARBEAShKSQfBEEQBEFoCsmHRGixJYIgCEIu586dw8SJExEbG4v4+HhMnz4ddXV1fvdpaGjArFmz0KlTJ7Rv3x5jx45FVVWVR5nKykqMGjUK7dq1Q2JiIh577DE0N3uO0Fq3bh369++Pdu3aoUuXLvj973+PX375xaPMqlWrcM011yA6OhqpqamYM2cOGhq8915+9tlnERYWhkceeUTydSD5IAiCIAiNmDhxIg4cOIDt27dj69at2LVrF2bOnOl3nzlz5uCjjz7Cpk2b8MUXX+DkyZO4++673e87HA6MGjUKjY2N2L17N9566y2sXbsWCxcudJf56quvMHnyZEyfPh0HDhzApk2bsHfvXsyYMcNdZv369Zg3bx7y8/Nx8OBBrF69Ghs3bsQTTzzRpk7ffPMN/vrXv+L666+XdR1IPgiCIAhCAw4ePIiioiK88cYbyMzMxNChQ/HSSy9hw4YNOHnypNd9ampqsHr1aqxcuRK33HILMjIy8Oabb2L37t34+uuvAQCffvopvv/+e7zzzjtIT0/HHXfcgaVLl6KgoACNjY0AgJKSEvTo0QMPPfQQevbsiaFDh+L+++/H3r173efavXs3brrpJkyYMAE9evTAbbfdhvHjx3uUAYC6ujpMnDgRr7/+Oq644gpZ14LkgyAIgiC8UFtb6/Gy28WvmOyNkpISxMfHY+DAge5tWVlZMJlM2LNnj9d9SktL0dTUhKysLPe2tLQ0dOvWDSUlJe7j9uvXD0lJSe4y2dnZqK2txYEDBwAAVqsVJ06cwLZt2yAIAqqqqvD+++9j5MiR7n2GDBmC0tJSt2wcOXIE27Zt8ygDALNmzcKoUaM86iQVWtsliHFEh9OsjAShMeEXnWiOpt91ehF+wYnwCKeygzRd2j81NdVjc35+PhYtWiT7sDabDYmJiR7bwsPD0bFjR9hsNp/7REZGIj4+3mN7UlKSex+bzeYhHq73Xe8BwE033YR169YhJycHDQ0NaG5uxujRo1FQUODeZ8KECTh79iyGDh0KQRDQ3NyMBx54wCPtsmHDBuzfvx/ffPONvIvwX0LyG0KdRgnCgNAihYTGnDhxAjU1Ne7X/PnzvZabN28ewsLC/L4OHTqkce09+f777/Hwww9j4cKFKC0tRVFREY4dO4YHHnjAXWbnzp145pln8Morr2D//v3YvHkzPv74YyxduhTApevx8MMPY926dYiKUtaQUuSDIAiCILwQGxuL2NjYgOUeffRRTJ061W+ZXr16ITk5GadPn/bY3tzcjHPnziE5OdnrfsnJyWhsbER1dbVH9KOqqsq9T3Jycpt+Ga7RMK4yy5Ytw0033YTHHnsMAHD99dcjJiYGN998M55++ml06dIFTz31FCZNmoT77rsPANCvXz/U19dj5syZePLJJ1FaWorTp0/jhhtucJ/H4XBg165dePnll2G322E2mwNcrUuQfBAEEXSY6u1wxlj0rgYRIiQkJCAhISFgOavViurqapSWliIjIwMAsGPHDjidTmRmZnrdJyMjAxERESguLsbYsWMBABUVFaisrITVanUf909/+hNOnz7tTuts374dsbGx6Nu3LwDgwoULCA/3bPJdoiAIgruMyWTyWWbEiBH49ttvPd6fNm0a0tLSMHfuXNHiAZB8EARBEIQm9OnTB7fffjtmzJiBwsJCNDU1ITc3F+PGjUNKSgoA4Oeff8aIESPw9ttvY9CgQYiLi8P06dORl5eHjh07IjY2FrNnz4bVasXgwYMBALfddhv69u2LSZMm4bnnnoPNZsOCBQswa9YsWCyXJHz06NGYMWMGXn31VWRnZ+PUqVN45JFHMGjQIPe5R48ejZUrV2LAgAHIzMzE4cOH8dRTT2H06NEwm83o0KEDrrvuOo/PFBMTg06dOrXZHgiSD4IgiBAkvAFopv5vmrNu3Trk5uZixIgRMJlMGDt2LF588UX3+01NTaioqMCFCxfc21544QV3WbvdjuzsbLzyyivu981mM7Zu3Yo//OEPsFqtiImJwZQpU7BkyRJ3malTp+L8+fN4+eWX8eijjyI+Ph633HILli9f7i6zYMEChIWFYcGCBfj555+RkJCA0aNH409/+hPz6xAmuOItHFNbW4u4uDh0X/40TP/t5GK66L+vrDnAiCh/y02HB1iKOtBS1WIWhQpUJvxC4N7a4RcDl5Ey2sV8oUlUOVO9hOFmCjsJCi2+gLKPUVev+BgEENY+Rtn+7dopr0RMtOiiYtMujnYRoo/piBb3ey3QaJfmdoH7+jdHhSl6Hwjcud6ffATa1xHg8jqjLz+fnA0NOD53AWpqakT1oZCDq50YPHIJwiOUWVVzUwO+3rZQ1fqGOiE52oUgCIIgCP0g+SAIIqQRG/HjDTERVoLgFZIPgmuYhOoJgiAIriD5IIIepX0VCIIgCLaQfBAEQRAEoSk01JaQjDPGIm3EC0EQhEaENzgR3qxwbRel+xMBocgHQRCawGLYNEEQwQHJB0EQhMaImceHIIIZkg+CIAiRSJm0jyAI35B8EARBEAShKSQfBEEQBEFoiiz5KCgoQI8ePRAVFYXMzEzs3bvXZ9nXX38dN998M6644gpcccUVyMrK8lueCDIkrMVBEAQ/BFrDiiCUIFk+Nm7ciLy8POTn52P//v3o378/srOzcfr0aa/ld+7cifHjx+Pzzz9HSUkJUlNTcdttt+Hnn39WXHmCIAiCIIyHZPlYuXIlZsyYgWnTpqFv374oLCxEu3btsGbNGq/l161bhwcffBDp6elIS0vDG2+8AafTieLiYp/nsNvtqK2t9XgRBEEQBBEcSJKPxsZGlJaWIisr6/IBTCZkZWWhpKRE1DEuXLiApqYmdOzY0WeZZcuWIS4uzv1KTU2VUk3dEbPUdcBjiFhyWwxilwAHpC0tLna5cgCKUy9h7dopXuOFplhXhtLrx+JvqFYKT8p9L+X7RBCEbyS1cGfPnoXD4UBSUpLH9qSkJNhsNlHHmDt3LlJSUjwEpjXz589HTU2N+3XixIk2ZZzR6o2Tb47y/74jwPuXjuFfQMQISiABaY4W9+dzRIeLfmiqKiA6S0hY+xj3iwgMi+vFTDok3jti7029xIPFjwsWP3IIQi801fhnn30WGzZswM6dOxEV5bsFt1gssFgkNGxecFgAs58ZwB1R+neoao4KC7gsdnM7k98JiZqjTQi/KE7EHNHhouYpcD2QxSw1Lnmq9ZaNSP1F8fu1oGVjJnfWTDENqlBXL+vYRkFNCWOyGrFMWdVbPMT+KPB7DBILIsiRJB+dO3eG2WxGVVWVx/aqqiokJyf73XfFihV49tln8dlnn+H666+XXlONaY4CwhXKiSi5EFEm4HlUEBDg0sNZrIAAkL7eC0MRUWPqbn+Ns1HEROsoj57SAegvHmLQKuohJkIbjJgvNsMcrmwyOKGZJpNTG0nfgsjISGRkZHh0FnV1HrVarT73e+6557B06VIUFRVh4MCB8mvbikCpF4ey4In/Y4v8YotKrwRK0Yh4WEn5tSU1DSP2QS0pDdMahWkZV3ifSeMn5nwtUhLeXlrCQz30Sq+0hAfxoKgHQYhDstLn5eVhypQpGDhwIAYNGoRVq1ahvr4e06ZNAwBMnjwZXbt2xbJlywAAy5cvx8KFC7F+/Xr06NHD3Tekffv2aN++PcOPIh2tUi9MohsB0i+AtAgIwFkUxAUnaRmlBGr4pUROeO6jonekw4Xe4iFWOlh1JA9EqEY9COMgWT5ycnJw5swZLFy4EDabDenp6SgqKnJ3Qq2srITJdPkL9uqrr6KxsRH33HOPx3Hy8/OxaNEiZbXHpeiH6aI6X+hAqRcp8hJIQFj0/wDkCQggbs0KsQICyOgL4g1XoyRTQgB10zJK4FkoxMDDyBWpkTa9xYMVFBkhggFZyczc3Fzk5uZ6fW/nzp0e/z527JicUzBD7Y6nwSAggHqdUQEFURAXQRINMSpMU1k6SAdgHPFgIRYU9SCMAA1aDwCLjqfSzqc8RQNcfijqnYYBGEoIwDQa4o1QFxPV+szoJB0AP+JBw2sJ4jIhIR+Boh+Kj88w+iGmjJjoh7usRAmRmoYBxEVBgLaNhyIZYRAN8UagxtfocqJVh1w3OkoHwI94iDomiQURQoSEfASCRcdTngUE4KMzamtaNio8iog3jCAnmguGN3SWDoAv8aCOpgThCcmHCMSmXlgLSMBjyBAQQL0oiBwBccFcRFSWEF9oJSdcCIY3FEqHUuFwwYt4sJQO1pGRQDM5E4SahIx88DjjqVYdUNvsI0NC1EjD+IKJiGgYDZECt9IgF0brrbCQDinC4d6HE/HQuqMpiQehNyEjH0pRI/oh7rzqCAggfWZUQJsoSEuYighHEmJYGC/uppd0APyIh6jjUX8QIsgIKflQu+Op+zwa9/9QAu9RkJbwMIFZyKHSSrJ6SgfAl3hQ1IMIRUJKPgKh12JzenRAbbO/BAnRojOqP5inZQIR7KKikmB4Q2/pANQTD606lRL+MV9shtms7JkjOGhtF7Uh+ZCAlDk/pIqMHh1QvR5DZCpGzpBcF1xFQ8QgpnHmWVA0lAtfaDlyxe9xOBMPFms/STsfs0MRhCJCTj60Sr0A7AVEbP8PAIqjIGqskuvep0VDwlXfECUEauDVlBMO5MIbPEQ53MdRcYE4NcVDDDS8ljAiIScfgQgkDGrPeMpCQADt0zCAuChIm31ViIpoEg2Rihg54VQixMJqmCzARjqkCocLtcVDinToGfUgqSHUhORDZdToRyJFQAC+oyBtjsEwKqJ7NEQKBhMPlqLREj2lA+BLPMQgVhAo3ULwBsmHF1gLgxr9P6T0EVEqIWp1Rg14LIZREUOJCIeoJRsu9JYOgD/x0HN4LUU9CLUh+ZCBnNSL3gICsEnFqJ2G8XtcRlERXw0pSckl1BaNlvAgHQB/4sESSrcQPELywTFqzO/BIgqiZRrG57FV7rQaCKOLipaC4Q0th8v6g9c5PMTsR5JAGBkamO6DQF9sOTlUOQ+LQA8h2Q+3dib5vfSjTaIf2o7ocGYNhc9ztItgNipCLM4Yi6iX1vBYL9ffp+VL9rEY3U9S7mH3PhyJh/hjSStPQqM+586dw8SJExEbG4v4+HhMnz4ddXV1fvdpaGjArFmz0KlTJ7Rv3x5jx45FVVWVR5mHHnoIGRkZsFgsSE9Pb3OMiooKDB8+HElJSYiKikKvXr2wYMECNDVd/vG2du1ahIWFebyiojxviqlTp7Ypc/vtt0u+DhT50Bg9O6B63VdBKkbO9Owu1IiIqBENUUqght5fBEXv6IRc1BBBvSId7v04Ew8xkkCdTPlk4sSJOHXqFLZv346mpiZMmzYNM2fOxPr1633uM2fOHHz88cfYtGkT4uLikJubi7vvvhtfffWVR7nf//732LNnD/71r3+1OUZERAQmT56MG264AfHx8fjnP/+JGTNmwOl04plnnnGXi42NRUVFhfvfYWFt78nbb78db775pvvfFov0ZxXJhx/0HnZ7+TzqTsGuJBUjdXp2Fy0bk1AREW8YVTBaombUSev+HF7312CBOB7WbqGoh/ocPHgQRUVF+OabbzBw4EAAwEsvvYSRI0dixYoVSElJabNPTU0NVq9ejfXr1+OWW24BALz55pvo06cPvv76awwePBgA8OKLLwIAzpw541U+evXqhV69ern/3b17d+zcuRP/+Mc/PMqFhYUhOTnZ7+ewWCwBywSC0i46oEb6RWwZv/srmB5ayQPeFUpXK0WjR1ommGGVQvF7Dgb3gpzUSptjGFQ8KN3ChtraWo+X3a6sr1dJSQni4+Pd4gEAWVlZMJlM2LNnj9d9SktL0dTUhKysLPe2tLQ0dOvWDSUlJbLrcvjwYRQVFWHYsGEe2+vq6tC9e3ekpqbirrvuwoEDB9rsu3PnTiQmJuKaa67BH/7wB/zyyy+Sz0/yoRC5oU2eBUSLviD+UEtE1G4wgxUtZMN9Lo6kg1fx4EEUHBwH7Ez1diYvAEhNTUVcXJz7tWzZMkV1s9lsSExM9NgWHh6Ojh07wmaz+dwnMjIS8fHxHtuTkpJ87uOPIUOGICoqCldddRVuvvlmLFmyxP3eNddcgzVr1uDvf/873nnnHTidTgwZMgQ//fSTu8ztt9+Ot99+G8XFxVi+fDm++OIL3HHHHXA4HJLqQWmXAIjpoyE3/SKn/4fYFAwARSNllKZipKZhfKFWXxGxjSjPKRuW6CVkLAWThfhqNYxWrqyo0ddDqszwLB6sOXHiBGJjY93/9tW3Yd68eVi+fLnfYx08eJBp3eSyceNGnD9/Hv/85z/x2GOPYcWKFXj88ccBAFarFVar1V12yJAh6NOnD/76179i6dKlAIBx48a53+/Xrx+uv/569O7dGzt37sSIESNE14Pkw4CInuFUx8XqWApIS9TuK9LmfBIbZZ5khccID+toFgvhALRdn0VN8SDYEhsb6yEfvnj00UcxdepUv2V69eqF5ORknD592mN7c3Mzzp0757MPRXJyMhobG1FdXe0R/aiqqpLV7yI1NRUA0LdvXzgcDsycOROPPvoozGZzm7IREREYMGAADh8+7Pdzde7cGYcPHyb5YA1v0Q8p6BkFkdsZVSxajKCRiloRFR5FQgxqDbPWWzoAPsWDoh76kJCQgISEhIDlrFYrqqurUVpaioyMDADAjh074HQ6kZmZ6XWfjIwMREREoLi4GGPHjgVwadhsZWWlR5RCDk6nE01NTXA6nV7lw+Fw4Ntvv8XIkSN9HuOnn37CL7/8gi5dukg6N8kHQ3hLv7QuD+grIYB6IgJoHxVRglFlwh9qz+cC6JdaaXMMjcSDoh3BRZ8+fXD77bdjxowZKCwsRFNTE3JzczFu3Dj3SJeff/4ZI0aMwNtvv41BgwYhLi4O06dPR15eHjp27IjY2FjMnj0bVqvVPdIFuNSBtK6uDjabDRcvXkR5eTmASxGOyMhIrFu3DhEREejXrx8sFgv27duH+fPnIycnBxERl55HS5YsweDBg3HllVeiuroaf/7zn3H8+HHcd999AC51Rl28eDHGjh2L5ORk/Pvf/8bjjz+OK6+8EtnZ2ZKuBckHY7QafnvpXNLTKnqmYgB9RATgX0aMiBay4YIX6QC0W6NFbfGgqIc+rFu3Drm5uRgxYgRMJhPGjh3rHiYLAE1NTaioqMCFCxfc21544QV3WbvdjuzsbLzyyisex73vvvvwxRdfuP89YMAAAMDRo0fRo0cPhIeHY/ny5fjhhx8gCAK6d++O3NxczJkzx73Pf/7zH8yYMQM2mw1XXHEFMjIysHv3bvTt2xcAYDab8a9//QtvvfUWqqurkZKSgttuuw1Lly6VPNdHmCAIbOfvVoHa2lrExcWh+/KnYYpq+40xXZT2UDHLGC0lJTIhVz7kpl/kygSLqduVrBXT5lgqyogLkhD5aCkbAB+pFY/jcJhm8TyXuseXKh/O6MvfZ2dDA47PXYCamhpRfSjk4Gonsq6ag3CzMlNqdtjx2Y8vqFrfUIciHyqgdf8PudEMVlEQgI2EaBEV8dWAkpS0xaiy4T6eTukVpfuqLR5yUCIeBOENkg+V4LUDamtY9AUBlK+Y2+Z4rRoitaMiUhtaI8qK1jLhD9ai4T6ujlEOpfvLTbNQJ1PCiPDzNCIUoTSKoffcIAGPrVFfEbFIachZiwpPEiEWtWTDfXwDSwegnXhoAUU9CDEY7ynGAIdFer8PeSNSLv1XagTE9SCSMwLm0vmUS4iS47RsCNQWkZbwICXeMKIsKEFt0fA4l86pFRbH0VI8aPQMwQtB8VR0RjsldzrVSkAA/STEhWyJYCwirWEtJkaTEqOjpWQA7KIbgP7C4SKYIh4EIYWgkA+AfwEBtJeQy+dVLiOshMbjmD4aE62kxB+hLCxaS0Wb8zOUjDbHZiAdrMSFx46lQcGFi4BJ4ffXqWwBOSIwQSMfgDEEBNBPQi6fn0FEQwUZcR9bw2iJzzowbIDVFhm9ZUEuakqGx3k4iXK4oGgHQQSZfADGERBAfwm5VAd+UjSizqNRtIQlRpUDVmglGR7n5Ew4XJB4EMQlgk4+AG0FxBdSxIAHCblcF76jIj7PyUG0JNTRQzLa1IGjtEprSDwI4jJBKR+AdgLi81gyxIAnCblUH/ZRETGwlhWljWIoygsPIuFCLRnQ8jxaiweNaiF4J2jlA9BfQABlEgJIExGxDxw9O67KOY/a5wsEy4ZYlaHHHImCErSSDK3PSeJBEG0JavkA+BAQQMmw2Uv/ZblYXeuHEwsZ0SS14qeh0EtMpBIsoqAUPUSD9bnVbORpDg8i2Al6+QD4ERCALwlxwUJG9Ojn4e/8etWD8ERPyWiJEYTDBYkHEQqEhHwAfAkIwKeEuGj5IGOVohEL8/4ejBu/UJIZXsRBCSw+g1YNO6VZiFAiZOQD4E9AAL4lBGCXohEL76kVlg0y07lRgkAUvKHn59KyUSfxIEKNkJIPgE8BAfiXEBcsoiJyCbbUSrAKgxx4uRZ6NOYkHkQoEnLyAfArIIByCZGDXHHROiriC96jJcRleJGMlujZiJN4sEe4cBGCyaHsGM5GRrUhfBGS8gHwLSCAenN3eKP1A9DoMtIStRo7khr/8CgZLWHVeOsxARiJBxEMhKx8APwLCKCthLhQS0aUwIPItITHjqy8N/gs4KHh1XPGUR4+P0GwIKTlAzCGgAD6SIgLuZOesYTVVPa8EgriIAUeG1mSDoJgR8jLB2AcAQH07fAJsIuKsCTYxSSY4b1R5WFdFd6vEUHIgeTjv8gVEJbIXdhOzwaWRxlpiRYPbhKcwBipAeVBOFzIvW6sn00EwRqSjxbIERCWuB4YRpQQFzykaLTGSA0r4R2ehMMFiQcRzJB8tEJvAQGUS4gayJtynX09vBEqkhPK8CgHakLiQQQ7JB9e4EFAAPkSogZ69zXxh6+GiaTEmBhFNHiLeJF4EEaC5MMHvAgIwJeEAHzO5+ENf40YiYn+GEUyWsKbcAAkHYQxIfnwA08CAvAnIS54jor4wogNH6EPPAqHCxIPwqiQfASANwEB+JUQwDhREYLwB8/C4YLEgzAyJB8i4FFAAL4lxIWeD3ESn+DACCKgNSQevhHqL0AIa1J2DEHZ/kRgSD5EwquAAMaQED3w1WiRlPAJSYY4SDyIYIDkQwI8Cwig30PJaNJDjRzBChIBgpAHyYdEeBcQPWj9ADaajBCEFEg4CEI5JB8yIAHxD8kIEWyQcBAEW0g+ZOKMdup6fiPJD28PbpIhY8DbfUMQBDtIPgyKS36MJCG8QI0aEczo/cOIIMRALZfBcUY76WFDEAQAEg/COMiSj4KCAvTo0QNRUVHIzMzE3r17/ZbftGkT0tLSEBUVhX79+mHbtm2yKkv4xiUhvLwIgtAW+t4Zg3PnzmHixImIjY1FfHw8pk+fjrq6Or/7NDQ0YNasWejUqRPat2+PsWPHoqqqyqPMQw89hIyMDFgsFqSnp3s9zieffILBgwejQ4cOSEhIwNixY3Hs2DH3+5s3b8att96KhIQExMbGwmq14pNPPvE4Ro8ePRAWFtbmNWvWLEnXQbJ8bNy4EXl5ecjPz8f+/fvRv39/ZGdn4/Tp017L7969G+PHj8f06dNRVlaGMWPGYMyYMfjuu++knpowEHrLD0lRcKL3/cPzizAGEydOxIEDB7B9+3Zs3boVu3btwsyZM/3uM2fOHHz00UfYtGkTvvjiC5w8eRJ33313m3K///3vkZOT4/UYR48exV133YVbbrkF5eXl+OSTT3D27FmP4+zatQu33nortm3bhtLSUgwfPhyjR49GWVmZu8w333yDU6dOuV/bt28HAPzv//6vpOsQJgiCIGWHzMxM3HjjjXj55ZcBAE6nE6mpqZg9ezbmzZvXpnxOTg7q6+uxdetW97bBgwcjPT0dhYWFos5ZW1uLuLg4dF/+NExRNEkDQRCEkXA2NOD43AWoqalBbGysKudwtRO3RP4vwsMiFB2rWWjCjsZNzOt78OBB9O3bF9988w0GDhwIACgqKsLIkSPx008/ISUlpc0+NTU1SEhIwPr163HPPfcAAA4dOoQ+ffqgpKQEgwcP9ii/aNEibNmyBeXl5R7b33//fYwfPx52ux0m06W4w0cffYS77roLdrsdERHer9m1116LnJwcLFy40Ov7jzzyCLZu3Yoff/wRYWFhoq+FpA6njY2NKC0txfz5893bTCYTsrKyUFJS4nWfkpIS5OXleWzLzs7Gli1bfJ7HbrfDbr88JKGmpgbApRuYIAiCMBauZ7fE37qyaEYToPA0zbg0vXptba3HdovFAotFfo/1kpISxMfHu8UDALKysmAymbBnzx789re/bbNPaWkpmpqakJWV5d6WlpaGbt26eZUPX2RkZMBkMuHNN9/E1KlTUVdXh7/97W/IysryKR5OpxPnz59Hx44dvb7f2NiId955B3l5eZLEA5AoH2fPnoXD4UBSUpLH9qSkJBw6dMjrPjabzWt5m83m8zzLli3D4sWL22w/kf+0lOoSBEEQHPHLL78gLi5OlWNHRkYiOTkZu2xbmByvffv2SE1N9diWn5+PRYsWyT6mzWZDYmKix7bw8HB07NjRZ5tos9kQGRmJ+Ph4j+2B2tHW9OzZE59++inuvfde3H///XA4HLBarX77YK5YsQJ1dXW49957vb6/ZcsWVFdXY+rUqaLr4YLLobbz58/3iJZUV1eje/fuqKysVO3GDQZqa2uRmpqKEydOqBbaDAboOgWGrpE46DqJo6amBt26dfP5C5oFUVFROHr0KBobG5kcTxCENr/mfUU95s2bh+XLl/s93sGDB5nUSy42mw0zZszAlClTMH78eJw/fx4LFy7EPffcg+3bt7f5rOvXr8fixYvx97//vY0wuVi9ejXuuOMOr+miQEiSj86dO8NsNrfpZVtVVYXk5GSv+yQnJ0sqD/gObcXFxdEXXASxsbF0nURA1ykwdI3EQddJHK6+BmoRFRWFKB36BT766KMBf/336tULycnJbQZnNDc349y5c37b0MbGRlRXV3tEPwK1o60pKChAXFwcnnvuOfe2d955B6mpqdizZ49H+mbDhg247777sGnTJo90T0uOHz+Ozz77DJs3bxZdh5ZIuhMiIyORkZGB4uJi9zan04ni4mJYrVav+1itVo/yALB9+3af5QmCIAjCSCQkJCAtLc3vKzIyElarFdXV1SgtLXXvu2PHDjidTmRmZno9dkZGBiIiIjza0YqKClRWVkpqRy9cuNBG/sxmM4BL7biLd999F9OmTcO7776LUaNG+Tzem2++icTERL9l/CJIZMOGDYLFYhHWrl0rfP/998LMmTOF+Ph4wWazCYIgCJMmTRLmzZvnLv/VV18J4eHhwooVK4SDBw8K+fn5QkREhPDtt9+KPmdNTY0AQKipqZFa3ZCCrpM46DoFhq6ROOg6iYOu02Vuv/12YcCAAcKePXuEL7/8UrjqqquE8ePHu9//6aefhGuuuUbYs2ePe9sDDzwgdOvWTdixY4ewb98+wWq1Clar1eO4P/74o1BWVibcf//9wtVXXy2UlZUJZWVlgt1uFwRBEIqLi4WwsDBh8eLFwg8//CCUlpYK2dnZQvfu3YULFy4IgiAI69atE8LDw4WCggLh1KlT7ld1dbXHuRwOh9CtWzdh7ty5sq+DZPkQBEF46aWXhG7dugmRkZHCoEGDhK+//tr93rBhw4QpU6Z4lH/vvfeEq6++WoiMjBSuvfZa4eOPP5Z0voaGBiE/P19oaGiQU92Qga6TOOg6BYaukTjoOomDrtNlfvnlF2H8+PFC+/bthdjYWGHatGnC+fPn3e8fPXpUACB8/vnn7m0XL14UHnzwQeGKK64Q2rVrJ/z2t78VTp065XHcYcOGCbg0zsfjdfToUXeZd999VxgwYIAQExMjJCQkCHfeeadw8ODBgMdo3aZ/8sknAgChoqJC9nWQPM8HQRAEQRCEEmhtF4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgiAIgiA0hRv5KCgoQI8ePRAVFYXMzEzs3bvXb/lNmzYhLS0NUVFR6Nevn9/56YMJKdfp9ddfx80334wrrrgCV1xxBbKysgJe12BA6r3kYsOGDQgLC8OYMWPUrSAnSL1O1dXVmDVrFrp06QKLxYKrr746JL53Uq/TqlWrcM011yA6OhqpqamYM2cOGoJ8Ucxdu3Zh9OjRSElJQVhYmN+FQ13s3LkTN9xwAywWC6688kqsXbtW9XoSHCF7kC5DNmzYIERGRgpr1qwRDhw4IMyYMUOIj48XqqqqvJb/6quvBLPZLDz33HPC999/LyxYsEDyxGVGROp1mjBhglBQUCCUlZUJBw8eFKZOnSrExcUJP/30k8Y11w6p18jF0aNHha5duwo333yzcNddd2lTWR2Rep3sdrswcOBAYeTIkcKXX34pHD16VNi5c6dQXl6ucc21Rep1WrdunWCxWIR169YJR48eFT755BOhS5cuwpw5czSuubZs27ZNePLJJ4XNmzcLAIQPPvjAb/kjR44I7dq1E/Ly8oTvv/9eeOmllwSz2SwUFRVpU2FCd7iQj0GDBgmzZs1y/9vhcAgpKSnCsmXLvJa/9957hVGjRnlsy8zMFO6//35V66k3Uq9Ta5qbm4UOHToIb731llpV1B0516i5uVkYMmSI8MYbbwhTpkwJCfmQep1effVVoVevXkJjY6NWVeQCqddp1qxZwi233OKxLS8vT7jppptUrSdPiJGPxx9/XLj22ms9tuXk5AjZ2dkq1ozgCd3TLo2NjSgtLfVYvMZkMiErKwslJSVe9ykpKWmz2E12drbP8sGAnOvUmgsXLqCpqUnVlSX1RO41WrJkCRITEzF9+nQtqqk7cq7Thx9+CKvVilmzZiEpKQnXXXcdnnnmGTgcDq2qrTlyrtOQIUNQWlrqTs0cOXIE27Ztw8iRIzWps1EIxWc44YmkVW3V4OzZs3A4HEhKSvLYnpSUhEOHDnndx2azeS1vs9lUq6feyLlOrZk7dy5SUlJ8rlJodORcoy+//BKrV69GeXm5BjXkAznX6ciRI9ixYwcmTpyIbdu24fDhw3jwwQfR1NSE/Px8LaqtOXKu04QJE3D27FkMHToUgiCgubkZDzzwAJ544gktqmwYfD3Da2trcfHiRURHR+tUM0IrdI98ENrw7LPPYsOGDfjggw90WXKaR86fP49Jkybh9ddfR+fOnfWuDtc4nU4kJibitddeQ0ZGBnJycvDkk0+isLBQ76pxxc6dO/HMM8/glVdewf79+7F582Z8/PHHWLp0qd5VIwiu0D3y0blzZ5jNZlRVVXlsr6qqQnJystd9kpOTJZUPBuRcJxcrVqzAs88+i88++wzXX3+9mtXUFanX6N///jeOHTuG0aNHu7e5lpYODw9HRUUFevfurW6ldUDOvdSlSxdERES4l+AGgD59+sBms6GxsRGRkZGq1lkP5Fynp556CpMmTcJ9990HAOjXrx/q6+sxc+ZMPPnkk22WNA9VfD3DY2NjKeoRIuj+TYiMjERGRgaKi4vd25xOJ4qLi2G1Wr3uY7VaPcoDwPbt232WDwbkXCcAeO6557B06VIUFRVh4MCBWlRVN6Reo7S0NHz77bcoLy93v+68804MHz4c5eXlSE1N1bL6miHnXrrppptw+PBht5wBwA8//IAuXboEpXgA8q7ThQsX2giGS9gEWsPTTSg+w4lW6N3jVRAuDWezWCzC2rVrhe+//16YOXOmEB8fL9hsNkEQBGHSpEnCvHnz3OW/+uorITw8XFixYoVw8OBBIT8/P2SG2kq5Ts8++6wQGRkpvP/++8KpU6fcr5bLNwcbUq9Ra0JltIvU61RZWSl06NBByM3NFSoqKoStW7cKiYmJwtNPP63XR9AEqdcpPz9f6NChg/Duu+8KR44cET799FOhd+/ewr333qvXR9CE8+fPC2VlZUJZWZkAQFi5cqVQVlYmHD9+XBAEQZg3b54wadIkd3nXUNvHHntMOHjwoFBQUEBDbUMMLuRDEAThpZdeErp16yZERkYKgwYNEr7++mv3e8OGDROmTJniUf69994Trr76aiEyMlK49tprhY8//ljjGuuDlOvUvXt3AUCbV35+vvYV1xCp91JLQkU+BEH6ddq9e7eQmZkpWCwWoVevXsKf/vQnobm5WeNaa4+U69TU1CQsWrRI6N27txAVFSWkpqYKDz74oPCf//xH+4pryOeff+71WeO6NlOmTBGGDRvWZp/09HQhMjJS6NWrl/Dmm29qXm9CP8IEgWKBBEEQBEFoh+59PgiCIAiCCC1IPgiCIAiC0BSSD4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgiAIgiA0heSDIAiCIAhNIfkgCIIgCEJTSD4IgiAIgtAUkg+CIAiCIDTl/wNgUBO+PXphBgAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAG3CAYAAACqrG+SAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB610lEQVR4nO3de3hT9f0H8HeTkraAbeXSm1aKohaUixYpxdsclTIqk4kTpONSC6hrFaiTiyJl6kDxhgjaiSD4DAbilDHAulpEpnSARX4CAspAQSAFVttKodec3x8sx6ZN0pxrzkner+fJs5l8k3xzWnLe/XwvJ0QQBAFEREREJmLxdweIiIiIpGKAISIiItNhgCEiIiLTYYAhIiIi02GAISIiItNhgCEiIiLTYYAhIiIi02GAISIiItNhgCEiIiLTYYAhIiIi02GAITK47du3+7sLRESGwwBDZFAHDx4EAFRUVODkyZN+7g0RkbEwwBAZ1HfffYdPP/0U8+fPx3vvvefv7hARGUoIr0ZNZDyCIODWW2/FlVdeiRtuuAHTpk3zd5eIiAyFFRgigxo/fjzeeecdXH311fjpp5/83R0iIkNhgCEyoJCQECQmJmL//v1YtGgR/vznP/u7S0REhsIAQ+QHK1euRJ8+fRAREYGUlBSUlpbivvvuQ9++fQFcHEJavHgxnnzySfzyl7/EH/7wBz/3mIjIWBhgiHQ2f/58TJgwAT169MCiRYswcOBA/PrXv8bu3bvRu3dvsd3NN9+M9evX4xe/+IX/OktEZFCh/u4AUTDZvXs3nnrqKcyaNQvz5s0T76+rq8OyZcswceJEABeHkG688UYAwMCBA/3SVyIiI2MFhkhH8+bNQ3x8PP74xz+63H/rrbcCgEsFJiMjQ9e+ERGZCQMMkU7q6urw4YcfYtSoUWjXrp3LY+fOnQPgGmCIiMgzBhginfznP//B+fPnxYm6zR05cgSRkZG44oor/NAzIiLzYYAh0smFCxcAABaL6z+7uro6/PWvf8X111/vj24REZkSAwyRTrp16wYA+Oyzz1zuf+6553Dq1CkGGCIiCbgKiUgnXbp0wdChQ/Hmm28iNDQUffr0wccff4yysjIAnP9CRCQFr4VEpKPTp0/jwQcfxJYtW9ChQwfcd999+MUvfoHf/OY3+Pe//43U1FR/d5GIyBQYYIj8LC8vD2vXrsWpU6cQGsqiKBGRLzgHhkgndXV1aPn3wrZt2/DnP/8ZDz30EMMLEZEErMAQ6WTDhg14+umnce+99yI6Ohq7d+/G22+/jRtuuAGffvopIiIi/N1FIiLT4J98RDrp1KkTbDYbnnvuOZw/fx5JSUmYMWMGZs2axfBCRCQRKzBERERkOpLnwGzbtg3Dhw9HQkICQkJCsH79+jafs3XrVtx4440ICwtDjx49sGLFChldJSIiMqYlS5YgKSkJ4eHhSE1Nxc6dO722X7duHZKTkxEeHo7evXtj8+bNLo8LgoA5c+YgPj4eERERSE9Px7fffuvSpqKiAllZWYiMjER0dDRycnLEy5IAQG1tLSZMmIDevXsjNDQUI0aMaNWPU6dOYcyYMbjmmmtgsVgwdepUt/2trKxEbm4u4uPjERYWhmuuucalz0lJSQgJCWl1y83NbePIySc5wNTU1KBv375YsmSJT+2PHj2KzMxM3HHHHdizZw+mTp2KiRMn4qOPPpLcWSIiIqNZu3Yt8vPzUVBQgN27d6Nv377IyMjA6dOn3bbfvn077r//fuTk5ODLL7/EiBEjMGLECOzbt09ss2DBAixatAiFhYXYsWMHOnTogIyMDNTW1optsrKysH//fhQXF2Pjxo3Ytm0bJk+eLD7e1NSEiIgIPProo0hPT3fbl7q6OnTt2hWzZ892e5kTAKivr8edd96J7777Du+99x4OHTqEpUuX4rLLLhPb7Nq1C6dOnRJvxcXFAIDf/va3vh9IqQQFAAgffPCB1zbTp08XrrvuOpf7Ro0aJWRkZCh5ayIiIkMYMGCAkJubK/53U1OTkJCQIMyfP99t+/vuu0/IzMx0uS81NVV48MEHBUEQBIfDIcTFxQkvvPCC+HhlZaUQFhYm/PWvfxUEQRC+/vprAYCwa9cusc2HH34ohISECCdOnGj1nuPHjxfuvvtur5/j9ttvF6ZMmdLq/jfeeEO48sorhfr6eq/Pb27KlCnCVVddJTgcDp+fI5Xmk3hLS0tbJb+MjAyPZSrgYiKsq6sT/9vhcKCiogKdO3dGSEiIVl0lIiINCIKAn376CQkJCa2uBaam2tpa1NfXq/JagiC0Ot+EhYUhLCzM5b76+nqUlZVh1qxZ4n0WiwXp6ekoLS11+9qlpaXIz893uS8jI0OcknH06FHY7XaXc2dUVBRSU1NRWlqK0aNHo7S0FNHR0ejfv7/YJj09HRaLBTt27MBvfvMbWZ/bnQ0bNiAtLQ25ubn4+9//jq5du2LMmDGYMWMGrFZrq/b19fX4y1/+gvz8fE3P2ZoHGLvdjtjYWJf7YmNjUV1djQsXLrhdfTF//nz88Y9/1LprRESko+PHj+Pyyy/X5LVra2uR1K0Dyk87VHm9jh07uswnAYCCggLMnTvX5b6zZ8+iqanJ7Xnu4MGDbl/b03nRbreLjzvv89YmJibG5fHQ0FB06tRJbKOWI0eOYMuWLcjKysLmzZtx+PBh/P73v0dDQwMKCgpatV+/fj0qKysxYcIEVfvRkiGXUc+aNcslnVZVVeGKK65At5lzYAkP92PPiLTjiFDni5fIaBy1tThe8CwuueQSzd6jvr4e5acdOPRFAi65RFmV56efHLi2/0kcP34ckZGR4v0tqy/BwuFwICYmBm+++SasVitSUlJw4sQJvPDCC24DzLJly/CrX/0KCQkJmvZL8wATFxeH8vJyl/vKy8sRGRnpce8Ld2U6ALCEhzPAUMCyGHBDA4YqUpMeUwAuucSCSIUBxikyMtIlwLjTpUsXWK1Wt+e5uLg4t8/xdF50tnf+b3l5OeLj413a9OvXT2zTcpJwY2MjKioqPL6vXPHx8WjXrp3LcFHPnj1ht9tRX18Pm80m3v/999/j448/xvvvv69qH9zR/FICaWlpKCkpcbmvuLgYaWlpWr81ESlkuWAx1I3IaGw2G1JSUlzOcw6HAyUlJR7Pc22dF7t37464uDiXNtXV1dixY4fYJi0tDZWVleLV7AFgy5YtcDgcql8U9uabb8bhw4fhcPz8B80333yD+Ph4l/ACAG+//TZiYmKQmZmpah/ckfyNcO7cOezZswd79uwBcHGy0Z49e3Ds2DEAF4d/xo0bJ7Z/6KGHcOTIEUyfPh0HDx7E66+/jnfffRfTpk1T5xMQUdBgkCEjys/Px9KlS7Fy5UocOHAADz/8MGpqapCdnQ0AGDdunMsk3ylTpqCoqAgvvfQSDh48iLlz5+KLL75AXl4egIuVqqlTp+LZZ5/Fhg0bsHfvXowbNw4JCQniXi49e/bE0KFDMWnSJOzcuROff/458vLyMHr0aJehm6+//hp79uxBRUUFqqqqXM7fTs77zp07hzNnzmDPnj34+uuvxccffvhhVFRUYMqUKfjmm2+wadMmzJs3r9UeLw6HA2+//TbGjx+vy7XdJL/DF198gTvuuEP8b+dclfHjx2PFihU4deqUGGaAi0ly06ZNmDZtGl599VVcfvnleOutt5CRkaFC943DWtd2GwKagnMImVTmDDEc4iIjGDVqFM6cOYM5c+bAbrejX79+KCoqEifhHjt2zGX11aBBg7B69WrMnj0bTzzxBK6++mqsX78e119/vdhm+vTpqKmpweTJk1FZWYlbbrkFRUVFCG82jWLVqlXIy8vD4MGDYbFYMHLkSCxatMilb8OGDcP3338v/vcNN9wAAC4XlnXeBwBlZWVYvXo1unXrhu+++w4AkJiYiI8++gjTpk1Dnz59cNlll2HKlCmYMWOGy3t9/PHHOHbsGB544AG5h1ISU1xKoLq6GlFRUeg+d57h5sAwuJgbAxUpxRDVNkdtLb6fMRtVVVVtzimRy3meOHnocsVzYKp/ciDh2h807S8pZ8hVSFIxRJBczX93GGZIDlaDiPwjIAIMkRr8HYQZoMyNQYZIXwwwGrHWtt0mWDUZaxTQMFgNCgyWCxaGGCIdMMCojMGlbe6OEUONK2eYYZAxJ1ZjiLTHAKMSBhdl/HX8jB6cWJUxNwYZIu0wwCjA0GJ+zX+GDDOkFQYZ/Zxtqkddk8JLCTTx52QGAR1gjBIwQg3SD6NoNGhQaPn7YuRAo9eEYwYldXF+DJF6AjrA+BuDi3vujosRQ42ZqjNaYdVHfazGEKmDAUYDDC7S6XnM5IQlPap5Rg9JnFisLq0vicCARIGOAUZFUk/CRhni0pIRT8otf05Gqf6YpeLDIGMOHK6iQMcAowIGF888fVYjnaCb//wYZnzH4SXjY4ihQBaUAcZfQzzBFFzaYtS9YIwcZoxwfDxhVca4GGIoUAVlgNEbg4tvjBZqjDbUZKaqDIOMsTDEUCBigNGQ1OASWmv4C4PL0hgeIvu5WoU/OQHASNUZo4cZBhnjYYihQMMAowEGF1fuPp+SUKMGpQHAiGHGyEEGYJgxAoYYCiQMMCpicPGdkUKN0g3s1JxTpSQMsSpDvmCIoUDBAKMCBhd1eDouegcbfwYBZxhSWtUxcphRYxdhhiBlGGIoEDDA/I8eE22lBpdACTpKA4g/qzX+CgJqDlEp+d02WvhxYjVHuUANMWcdYah1KNsk8Jwj8I5LIGKA0UGwBhcnLQKI0mMk5/39da0kf863MfL8GoBBRqlADTEUHBhgNCLnBBtowcUbfw8XtXx/pYFG7zDjryADGDPMMMjIxxBDZsUAo7Jgr7Yo5a/houbva4Yww6qMewwy8jDEkBkxwKhE6+ASet7cXy6N7eWPSesdapRWZ/QeavJXmGGQCSwMMWQ2DDAKMbj4xt3nUDvUANoEGzWrM21RGgb8McRk5OElBhlpGGLITBhg2qDWEE+wBhdv1A41gLTjLCeMKA0zbVGrqsGqjCsGGd8xxJBZMMBojMFFGk+fX2mwcfteCsOIGhOBPVGzquGPMMMgY24MMWQGDDAa0Tq4hF4w15dLY4TCyooG1RqX11chjGhVndEizOgdZABjhRkGmbYxxJDRMcCojMHFPXf9NnKoUbM6o0WYYVVGHbxWk3cMMWRkDDAqYXCRztNnUhJstJ5XY5ShJjMPMRkxyACsynjCEENGxQCjEIOL+tSu1ng75lLDjRGHmrQKM4C2gYbDS+ZhphDz36YOuNBkVfQa55uaVOoNaYkBRiajBRfrhUZJ7f2hKUL+r5sWQ1CA689FTqXGaNUZtasbes2ZMWJVhkHGlZlCDAWHgA0wnvbeaPkXptYYXH7Wsq9KAg2gbaXGH2Gm+WswyGj7PlJY6xhiiIxI/bWpOms+CU/2a3gMO+6rLL5UX0LPOySFl9ALDknhxXqh0VThxR0tPoPU4+jxdRQuZ1e6f5Ba+w9Za9W90rpefwDocXV4KdT4niEidQVkBUbL6ktbJxZWXKRr/pmUVmWcmh9XuVWZ0PMOVXYLVlKNUWsFk7VW3WqM3tdfMgJWYoiMxdQBxh/VF08YXNShZZiRE2ScP1elQYYhRjo1+0tEgcf0Q0gt+bP64vPrMLz4RO0hJufwkpwhJjWGlOT+/qh5xXKjDc20xWj95VASkXGYugLjK0/hRe/qi9bBxXq+QVJ7PTW1byf7uVpVZaRWZPxZjTFiJUavoSSjVWI4lERkDAFVgVHjrzWpE3d9CS9aT9C1nm8wdHgB1Ouj89ioUZmRO9nXXxN8jViJCdZJvUTkfwEVYNyRWn1R9b0ZXFpRs89qBBklIUZJkJE7pMQQYwwcSqKWlixZgqSkJISHhyM1NRU7d+702n7dunVITk5GeHg4evfujc2bN7s8LggC5syZg/j4eERERCA9PR3ffvutS5uKigpkZWUhMjIS0dHRyMnJwblz51zafPXVV7j11lsRHh6OxMRELFiwoFVfFi5ciGuvvRYRERFITEzEtGnTUFv78z+4pKQkhISEtLrl5uaK/XjkkUfE17jiiivw6KOPoqqqStIxlCpgAozRqi8MLt5pEWTkhhklS6/9UY0J5hBjJAwx5LR27Vrk5+ejoKAAu3fvRt++fZGRkYHTp0+7bb99+3bcf//9yMnJwZdffokRI0ZgxIgR2Ldvn9hmwYIFWLRoEQoLC7Fjxw506NABGRkZLsEiKysL+/fvR3FxMTZu3Iht27Zh8uTJ4uPV1dUYMmQIunXrhrKyMrzwwguYO3cu3nzzTbHN6tWrMXPmTBQUFODAgQNYtmwZ1q5diyeeeEJss2vXLpw6dUq8FRcXAwB++9vfAgBOnjyJkydP4sUXX8S+ffuwYsUKFBUVIScnR50D7EGIIAjqfRtqpLq6GlFRUeg+dx4s4T8Phjf/AnH3RazW3Bd396sRXuSccM0eXDxRMkfG42vKmC+jZCM8pddckjPHRa15MWrNMdFjToyR5sME21wYuTvxOmpr8f2M2aiqqkJkZKTKvbrIeZ5Y93/JaH+JwksJ/NSE3/Y9iOPHj7v0NywsDGFhrX/oqampuOmmm7B48WIAgMPhQGJiIh555BHMnDmzVftRo0ahpqYGGzduFO8bOHAg+vXrh8LCQgiCgISEBDz22GP4wx/+AACoqqpCbGwsVqxYgdGjR+PAgQPo1asXdu3ahf79+wMAioqKMGzYMPzwww9ISEjAG2+8gSeffBJ2ux02mw0AMHPmTKxfvx4HDx4EAOTl5eHAgQMoKSkR+/LYY49hx44d+Oyzz9wen6lTp2Ljxo349ttvERLi/jto3bp1+N3vfoeamhqEhmoz3TZgKjC+0mLTOln9kDFBN1DDC/Dz51PzM8qpyijZBM/M1RgjDc+0xUh9ZRXGeMobo2BviFZ0K2+MAgAkJiYiKipKvM2fP7/V+9XX16OsrAzp6enifRaLBenp6SgtLXXbx9LSUpf2AJCRkSG2P3r0KOx2u0ubqKgopKamim1KS0sRHR0thhcASE9Ph8ViwY4dO8Q2t912mxhenO9z6NAh/PjjjwCAQYMGoaysTBzyOnLkCDZv3oxhw4a57Xt9fT3+8pe/4IEHHvAYXgCIYVWr8AIE8CokLZdNK6m+aL2yyFJjvG9URwdpf6Y6P7NaVRmpq5j8uW+MnNVGaq1QUmO1TzCuTOKqpMDlrgLT0tmzZ9HU1ITY2FiX+2NjY8UqR0t2u91te7vdLj7uvM9bm5iYGJfHQ0ND0alTJ5c23bt3b/UazscuvfRSjBkzBmfPnsUtt9wCQRDQ2NiIhx56yGUIqbn169ejsrISEyZMcPs4cPGYPPPMMy7DWVoIqgqMWsumPVEzvEitRlhq6gwZXgD5ffN3VcZf1RizV2KCcVIvBabIyEiXm7sAY3Zbt27FvHnz8Prrr2P37t14//33sWnTJjzzzDNu2y9btgy/+tWvkJCQ4Pbx6upqZGZmolevXpg7d66GPQ/gCoyvtFg27Y6vJ81AqLh44uyr1IoM4Hpc1KjMWC80+lyNUXIpAkD+hSFZiTEPVmGCV5cuXWC1WlFeXu5yf3l5OeLi4tw+Jy4uzmt75/+Wl5cjPj7epU2/fv3ENi0nCTc2NqKiosLlddy9T/P3eOqppzB27FhMnDgRANC7d2/U1NRg8uTJePLJJ2Gx/Pz99f333+Pjjz/G+++/7/Zz/fTTTxg6dCguueQSfPDBB2jXTv25jc0FTQVGlb8sZQ4d+RJe5FQazBRemlNaLVJzTxlfKL04pNzQy0pM24xUheF8mOBks9mQkpLiMgnW4XCgpKQEaWlpbp+Tlpbm0h4AiouLxfbdu3dHXFycS5vq6mrs2LFDbJOWlobKykqUlZWJbbZs2QKHw4HU1FSxzbZt29DQ0ODyPtdeey0uvfRSAMD58+ddQgoAWK0XJ0G3XOPz9ttvIyYmBpmZma0+k3PFk81mw4YNGxAerv1fMEETYNxRa+KuGuFFCiMPF0mhVpBREmZ8HVJSepVrhhjtGCnEUHDKz8/H0qVLsXLlShw4cAAPP/wwampqkJ2dDQAYN24cZs2aJbafMmUKioqK8NJLL+HgwYOYO3cuvvjiC+Tl5QEAQkJCMHXqVDz77LPYsGED9u7di3HjxiEhIQEjRowAAPTs2RNDhw7FpEmTsHPnTnz++efIy8vD6NGjxeGdMWPGwGazIScnB/v378fatWvx6quvIj8/X+zL8OHD8cYbb2DNmjU4evQoiouL8dRTT2H48OFikAEuhrK3334b48ePbzUx1xleampqsGzZMlRXV8Nut8Nut6OpqUmTYw5wCMlnnk5ASk5qwVJxaYuSoSUnpRN/jTykZPbhJD0YpZ8cSgpOo0aNwpkzZzBnzhzY7Xb069cPRUVF4oTZY8eOuVQ5Bg0ahNWrV2P27Nl44okncPXVV2P9+vW4/vrrxTbTp08Xh3IqKytxyy23oKioyKWysWrVKuTl5WHw4MGwWCwYOXIkFi1aJD4eFRWFf/7zn8jNzUVKSgq6dOmCOXPmuEyunT17NkJCQjB79mycOHECXbt2xfDhw/GnP/3J5TN+/PHHOHbsGB544IFWn3/37t3iyqcePXq4PHb06FEkJSXJOKptC9h9YFr+5df6cXXmvsitvkidoBtMlASZ5uSEGV/3jlGyXwwgb16MmfeJ0Ws+jBFCDBC4IcYM+8AsLktFREdlf5tfONeIvJQdmvaXlGMFphm9wouvJAeXmguK31MzHSJ8bqpGRQaQN/HX+XNrK8goWWoNyKvGmLkSE2yTeolIe0E5B0aNOQJaDh1JnhtSc8HY4QWQ1Uc15/pInStj1Am+Zp4TE0zzYTihl0h7QRlg3FFz2bSSoaOAqrq4Y6IgIyXEKJ3gK+X3zN/XT1KCIYaI1MIAI4NWQ0cBV3XxxgBBxqd2En6eelZj/BViuDKJiIyCc2CgXvWlrZOdt5OmzydmM4cWd5yfR8YcGUCflUu+zosB1Jkb4+u8GH/NiTHKip+2GKGfXJWkv/82XoLwRmWnttpG5XMYSXuswHig9rJpxeHF7BWXtsj8fGpUZYxWjQmGSoxelxswgkAaSrJc4CmDjIMVGJWotWS6FQ1Ci3D+vOqv6U1I+/a+N5ZRkQGUV2Ws5xt8Wqnk634xgPI9YwK9EqPHyiQjVGGISBsMMG5Irb5oNu9FYnjRO5j4ytkvPYIMID/MGG1IiSFGHUYIMRxKIlIf64EaU2XeSxuE8+cNG16ak9VPhUNncoaYjDSkxOEkdRhhUm+gDCVxGImMgr+JPpJTfdF66MgswaUlfwYZX8OMViFGTpBhiFGHEUIMEamHAcYHWoQXJUNHZg0uLfkjyAC+V760CDGAvGpMMISYYMAqDJF6+FvoB0rDS6Ax8tCSrxvf+XpVayc51ZhADzHBUoUJlBBD5G8MMG1Qu/qiZN5LIIaX5vwVZHxhlGqMGUKMEsESYohIOa5CksEf8158PbEL52rkv7+GQjp28LmtolVLgKwl2L6sVtJiqTUgfaWSHquTAPlXsubKJB/ePwBWJVkuWGRfnZpIDazA6EjNJdMtCedqDBteAHn9kz3XR+ZlCnyh1ZASIK0ao3UlRilO6vXh/TmURKSIrACzZMkSJCUlITw8HKmpqdi5c6fX9gsXLsS1116LiIgIJCYmYtq0aaitNWcd12hDR0YPLi0ZOcj4e5USYKwQozT4mCHEEJF5SR5CWrt2LfLz81FYWIjU1FQsXLgQGRkZOHToEGJiYlq1X716NWbOnInly5dj0KBB+OabbzBhwgSEhITg5ZdfVuVDGIG/h47MxhliNB9aAiRvimeEISUjDScp2ezO6MNJHEpSxojDSP9t6ICwhrb/XXpT16Dg+5x0I7kC8/LLL2PSpEnIzs5Gr169UFhYiPbt22P58uVu22/fvh0333wzxowZg6SkJAwZMgT3339/m1UbM9FqyXRb4cVMlRdPjFqRYSVG2XOaM3olhkNJROYkKcDU19ejrKwM6enpP7+AxYL09HSUlpa6fc6gQYNQVlYmBpYjR45g8+bNGDZsmMf3qaurQ3V1tcvNrLSa9xII4aU5JUFGq5VLRlhq7XNbhhhF/B1izIx7wpC/SPrNO3v2LJqamhAbG+tyf2xsLOx2u9vnjBkzBk8//TRuueUWtGvXDldddRV+8Ytf4IknnvD4PvPnz0dUVJR4S0xMlNJNXSkaOqJW5AazQB1qk8LoIYY8YxWGSDrNo/PWrVsxb948vP7669i9ezfef/99bNq0Cc8884zH58yaNQtVVVXi7fjx41p3UzZf5kAQERGRuiRN4u3SpQusVivKy8td7i8vL0dcXJzb5zz11FMYO3YsJk6cCADo3bs3ampqMHnyZDz55JOwWFpnqLCwMISFGWNmW2OERfaF+IiIiEgbkiowNpsNKSkpKCkpEe9zOBwoKSlBWlqa2+ecP3++VUixWq0AAEFgSZqIiIikk7yMOj8/H+PHj0f//v0xYMAALFy4EDU1NcjOzgYAjBs3Dpdddhnmz58PABg+fDhefvll3HDDDUhNTcXhw4fx1FNPYfjw4WKQ0VtjeAjH84mIiExMcoAZNWoUzpw5gzlz5sBut6Nfv34oKioSJ/YeO3bMpeIye/ZshISEYPbs2Thx4gS6du2K4cOH409/+pN6n8KsOkQo3oGXLpK8NwwREZmarGsh5eXlIS8vz+1jW7dudX2D0FAUFBSgoKBAzlv5rCmcSyGJiIiCBRfwExERkekwwBAREZHpyBpCCmZNEaGSt4UnIiJ9/NjQHrYGm6LXqG+oV6k3pCVWYDTmy0UB5ZByEUQiIqJAwwBDREREpsMAg4v7whAREZF5MMCogNdDIiIi0hcDjIFxczYiIiL3GGCIiIjIdBhgiIiIyHQYYIiIiBRasmQJkpKSEB4ejtTUVOzcudNr+3Xr1iE5ORnh4eHo3bs3Nm/e7PK4IAiYM2cO4uPjERERgfT0dHz77bcubSoqKpCVlYXIyEhER0cjJycH586dc2nz1Vdf4dZbb0V4eDgSExOxYMECl8dXrFiBkJAQl1t4eLhLm5aPO28vvPACgIuXEPLUZteuXZKOoxQMMERERAqsXbsW+fn5KCgowO7du9G3b19kZGTg9OnTbttv374d999/P3JycvDll19ixIgRGDFiBPbt2ye2WbBgARYtWoTCwkLs2LEDHTp0QEZGBmprf77oX1ZWFvbv34/i4mJs3LgR27Ztw+TJk8XHq6urMWTIEHTr1g1lZWV44YUXMHfuXLz55psu/YmMjMSpU6fE2/fff+/yePPHTp06heXLlyMkJAQjR44EAAwaNKhVm4kTJ6J79+7o37+/4uPrCQOMG43tdTwsHSL0ey8T4AZ9RGQ2L7/8MiZNmoTs7Gz06tULhYWFaN++PZYvX+62/auvvoqhQ4fi8ccfR8+ePfHMM8/gxhtvxOLFiwFcrL4sXLgQs2fPxt13340+ffrgnXfewcmTJ7F+/XoAwIEDB1BUVIS33noLqampuOWWW/Daa69hzZo1OHnyJABg1apVqK+vx/Lly3Hddddh9OjRePTRR/Hyyy+79CckJARxcXHiLTY21uXx5o/FxcXh73//O+644w5ceeWVAACbzebyeOfOnfH3v/8d2dnZCAnRbpsSBhgiIiI3qqurXW51dXWt2tTX16OsrAzp6enifRaLBenp6SgtLXX7uqWlpS7tASAjI0Nsf/ToUdjtdpc2UVFRSE1NFduUlpYiOjrapcKRnp4Oi8WCHTt2iG1uu+022Gw2l/c5dOgQfvzxR/G+c+fOoVu3bkhMTMTdd9+N/fv3ezwm5eXl2LRpE3Jycjy22bBhA/773/8iOzvbYxs18FpIREQUMCrqO6BdnbJrITXUX9zbKzEx0eX+goICzJ071+W+s2fPoqmpqVXVIjY2FgcPHnT7+na73W17u90uPu68z1ubmJgYl8dDQ0PRqVMnlzbdu3dv9RrOxy699FJce+21WL58Ofr06YOqqiq8+OKLGDRoEPbv34/LL7+8Vd9XrlyJSy65BPfcc4/bzwYAy5YtQ0ZGhtvnq4kBRgeODmGw1LRO7kREZFzHjx9HZGSk+N9hYdpc286f0tLSkJaWJv73oEGD0LNnT/z5z3/GM88806r98uXLkZWV1Wqir9MPP/yAjz76CO+++65mfXbiEJKJcb4IEZF2IiMjXW7uAkyXLl1gtVpRXl7ucn95eTni4uLcvm5cXJzX9s7/batNy0nCjY2NqKiocGnj7jWav0dL7dq1ww033IDDhw+3euxf//oXDh06hIkTJ7p9LgC8/fbb6Ny5M3796197bKMWBhgiIiKZbDYbUlJSUFJSIt7ncDhQUlLiUtloLi0tzaU9ABQXF4vtu3fvjri4OJc21dXV2LFjh9gmLS0NlZWVKCsrE9ts2bIFDocDqampYptt27ahoaHB5X2uvfZaXHrppW771tTUhL179yI+Pr7VY8uWLUNKSgr69u3r9rmCIODtt9/GuHHj0K6d9pfYYYAhIiJSID8/H0uXLsXKlStx4MABPPzww6ipqREnsY4bNw6zZs0S20+ZMgVFRUV46aWXcPDgQcydOxdffPEF8vLyAFxcFTR16lQ8++yz2LBhA/bu3Ytx48YhISEBI0aMAAD07NkTQ4cOxaRJk7Bz5058/vnnyMvLw+jRo5GQkAAAGDNmDGw2G3JycrB//36sXbsWr776KvLz88W+PP300/jnP/+JI0eOYPfu3fjd736H77//vlWVpbq6GuvWrfNafdmyZQuOHj3qtY2aOAfGB40RFoRecHht09S+HaznG7y2ISKiwDNq1CicOXMGc+bMgd1uR79+/VBUVCROmD127Bgslp/rBYMGDcLq1asxe/ZsPPHEE7j66quxfv16XH/99WKb6dOno6amBpMnT0ZlZSVuueUWFBUVucw9WbVqFfLy8jB48GBYLBaMHDkSixYtEh+PiorCP//5T+Tm5iIlJQVdunTBnDlzXPaK+fHHHzFp0iRxUm9KSgq2b9+OXr16uXzGNWvWQBAE3H///R6Pw7JlyzBo0CAkJyfLP5gShAiCIOjyTgpUV1cjKioK3efOg6XZD8/aYl6s9ef9fRBa6/mxi48L3v/7vGtgaR5grBca3fbTW4DxOom35oLHh4Tz5z0/D4Bwrsbr42Yjd16PrAtf+rgHj6ODbxP3fL0qeVOE7383NEZIK5JK2cOoMVz6/gxynuPU5H7On4T3Vvb8tijtn+L3N/H8UEeE9z/wHLW1+H7GbFRVVblMilWT8zyR+dFEtOugcBVSTT02ZbylaX9JOQ4hGRyvSE1ERNQaAwwRERGZDgMMERERmQ4DjBHwekiKcJiNiCj4MMCQYXBjPiIi8hWXURMRUcCoqgtDaKiyJV2NddpdQZnUwwqMDFKWwRIREZH6GGB04uteIkSBquVeTERESgRsgNF60yt3fN3ITCpvk1QDZd6IrhvYAapvYmcEUjax8wd/bxRHRIHF2N94Kmr55dlyR1ElO4y6vI+XEOP1ZNghwutJNZBDjFHDixRG2IVXCr134SXvzLwLL5G/mDrAtPxHr+ZfeC3/mm15MvF2IpIdYoCgCjEhHTvI6ndI+/byLx0gIbz48xICUgV69cUfFVUiMraAno3aGO56TaSmcNdx+MbwkFbXQFKLt4s7Ok+MHq+P1CHC4/WRQtq393h9pLbCgFGum2SGiou/w4uU6ovU8MLqS2sc3iIyH9MHmKaw1hd1lKtloGlsb3G5qGPLq1I3RYR6vLAj8PPJzVuQUTvEeKNFlcbXUKTkvfUeKjJTeJFKahBRI7iw+kJEWjB9gGmpdZXFexVG8fu1EWKAtqsxckMM0PbVqrWm5dCVP+a4mOnK04C2Q0eBXnVx8nf1hfNfiOQx9sC5DtqazOvLCaIpIrTNE5U/JvealV5zXJpzdAgL+PAiJZCoNqmd1Rci0khABJi2JvOq+SXo7USjNMS0GWQ8CKQQ44/hIilLpY0SXqQKlmqKVP6uvhCRfAERYKTSckm1khAD+FCN8cDsIUbvqovU4AIYK7yYYeiI1Ze2cfiISL6AmwPjpOZcl7Ym87Z67/+dwDzNjdFycq8n/p4r446i0KXxBN2WtNikUI/KC+CfoSMif6mpD4NV4bWQmupV6gxpKmADTEv+WFLtyyoltUOMJ2pXaKQGItXeX+fgAkgLL75WX5SEFynVF38FEjNUXzh8RGRuAR1glFRhpC6p9tgHLUMMIDnIqEX3ISs/BBfAWOFF683qjFJ9CYahIyJSLqjmwLT8YmxrLoxa2lqlpNXk3oCg4xyX5pratzN9ePHX0JEZKhtG6CPnvxApE1QBRqq2llRLPRnJDTGA/Mm9pmaC4AJoG14a21s0Dy9GwuoLEfkqoIeQAO03tnOelHwZTgK8DylpNbnXNBQGMX9M0NUqvOh5bSOjrDoKJqy+ECkX8AFGqlZzX9qYCyPe7+OcGMC3VUqK5sW4Y8Rwo1LlyF8ri4waXsy4YR2gX/XF30GL4YVIHUEZYFpWYVR7XQkhBmi7GiMrxHii1jCTkiCk8lCXP5dEaxFe1Ki6+GPoyN+BwEwYXojUE5QBpqW2llRLWWKt9pCS7Ctaa8UA823MEFwA/cOLVErDjprBJRiqLwwvROpigJHB0zCSSxuJQ0qazIsJMGYJLkDgV17MVnXxd38ZXojUF7SrkPRYUi3lJKbZxSBNzrmqSM5nlLOiyOX5/1v+HojhRe7vd1O4+mFA6+oLwwtRYGIFxke+TuZt9TwJQ0qaTe41EbXCmB4ri9zRM7zIDSFKwovZ+LvPDC9E2gnqAKP2kmqv76XikFJb82JaMmqw0aJy5K/gIpWS8BJIw0VaVl8YXoJTTZ0NVqtN0Ws01al7WRnSRlAHmLbIXVLt8fV0CDHuqBEU1AhBWg51yQ0uaoUWPSovqgxrSnwNf4cAOYzQZ4YXIu0xwCjkPBn5GmSkDinJndyrNqPOszFTcAEYXprTovrC8EIUPBhgWpC7pFqraoySi0EGIr02n/PGqNczUpMRgoBURugzwwuRfhhgVKRliAHkTe41OzWWQAP+DS6A9PCiZnAJ9MqLEYILwPBCpDcGGDfaqsJ4488hpUAIMWoFFvH1/BxcAGnhxZ/BRQ8ML0SkFgYYGXwJNP4YUvJ08jdysFE7sLi8tk7Lob2+jsnCi1aBIFDnuwAML0T+wgDjgRpLqv01L6ZVe4UhQc0ApGVgEd+DwUUWhhfpGF6I/Cdod+KVquWJwdcTRWN7i7STWYTFp5OonB1i5XLuaKvGTZP+NdsxV+4x8fW4e32N//2s/XVdI6MNGTWGazNkZJTwQhc5Inz/Iy2QLVmyBElJSQgPD0dqaip27tzptf26deuQnJyM8PBw9O7dG5s3b3Z5XBAEzJkzB/Hx8YiIiEB6ejq+/fZblzYVFRXIyspCZGQkoqOjkZOTg3Pnzrm0+eqrr3DrrbciPDwciYmJWLBggcvjS5cuxa233opLL70Ul156KdLT01v1/dy5c8jLy8Pll1+OiIgI9OrVC4WFhS79eOSRR3DttdciIiICV1xxBR599FFUVVX5fPzkYIDRieRJnD6eTPXcfM0o1AgsTmoGF1nPVRg61AouZrs8ABkDw8tFa9euRX5+PgoKCrB792707dsXGRkZOH36tNv227dvx/3334+cnBx8+eWXGDFiBEaMGIF9+/aJbRYsWIBFixahsLAQO3bsQIcOHZCRkYHa2p+HBrKysrB//34UFxdj48aN2LZtGyZPniw+Xl1djSFDhqBbt24oKyvDCy+8gLlz5+LNN98U22zduhX3338/PvnkE5SWliIxMRFDhgzBiRMnxDb5+fkoKirCX/7yFxw4cABTp05FXl4eNmzYAAA4efIkTp48iRdffBH79u3DihUrUFRUhJycHNWOsTshgiAYfsvB6upqREVFofvcebCEu/9mtHrZZ83bUFCol8fcPc/d3BdfJ/iK7SUMK/m68Z2UISWzUTuk+WOYyO3zVQgvalErwATyzrotcfjI9wDjqK3F9zNmo6qqCpGRkZr0xXmeuGbVTFjbK/vhNJ2vwzdZz/nc39TUVNx0001YvHgxAMDhcCAxMRGPPPIIZs6c2ar9qFGjUFNTg40bN4r3DRw4EP369UNhYSEEQUBCQgIee+wx/OEPfwAAVFVVITY2FitWrMDo0aNx4MAB9OrVC7t27UL//v0BAEVFRRg2bBh++OEHJCQk4I033sCTTz4Ju90Om+3i7sQzZ87E+vXrcfDgQfefvakJl156KRYvXoxx48YBAK6//nqMGjUKTz31lNguJSUFv/rVr/Dss8+6fZ1169bhd7/7HWpqahAaqs0f2qzAqEDqiUSrIaVA0LK6oubnMtIwkZLwoeZwkZpDMqy6BJdgqL5UV1e73OrqWv+lXF9fj7KyMqSnp4v3WSwWpKeno7S01O3rlpaWurQHgIyMDLH90aNHYbfbXdpERUUhNTVVbFNaWoro6GgxvABAeno6LBYLduzYIba57bbbxPDifJ9Dhw7hxx9/dNu38+fPo6GhAZ06dRLvGzRoEDZs2IATJ05AEAR88skn+OabbzBkyBD3Bw4Qw59W4QXgJF7JPG5kJ2GpNaDNBF93J3ujV2b0Cl56LoX2+joKg4ta1Kxm6BFcWH0xFiOHl/radrBYlF0LyVF78fMlJia63F9QUIC5c+e63Hf27Fk0NTUhNjbW5f7Y2FiPVQ673e62vd1uFx933uetTUxMjMvjoaGh6NSpk0ub7t27t3oN52OXXnppq77NmDEDCQkJLuHptddew+TJk3H55ZcjNDQUFosFS5cuxW233eb28509exbPPPOMy3CWFmR9K0udrFRZWYnc3FzEx8cjLCwM11xzTasJS4FA6l/Gcib4SuWuoiH1piYtX9vJWWlpfpP9WipVW5RUTYxacQFYdaHAdvz4cVRVVYm3WbNm+btLmnruueewZs0afPDBBwhvNl3jtddew7///W9s2LABZWVleOmll5Cbm4uPP/641WtUV1cjMzMTvXr1ahX21Cb5DOKcrFRYWIjU1FQsXLhQLEm1TIPAxfLanXfeiZiYGLz33nu47LLL8P333yM6OlqN/vtFW9UWLasxUja+U4vRh6fUmtPi8poqVFz8cf0ib8w4UZeVF+MxcvVFbZGRkW3OgenSpQusVivKy8td7i8vL0dcXJzb58TFxXlt7/zf8vJyxMfHu7Tp16+f2KblJOHGxkZUVFS4vI6792n+Hk4vvvginnvuOXz88cfo06ePeP+FCxfwxBNP4IMPPkBmZiYAoE+fPtizZw9efPFFl0rNTz/9hKFDh+KSSy7BBx98gHbttN02Q/K39Msvv4xJkyYhOztbXErVvn17LF++3G375cuXo6KiAuvXr8fNN9+MpKQk3H777ejbt6/izhuZlvNiAG1O2mahVnWl1euqOL/FKKuLAG2qLgwvRBfZbDakpKSgpKREvM/hcKCkpARpaWlun5OWlubSHgCKi4vF9t27d0dcXJxLm+rqauzYsUNsk5aWhsrKSpSVlYlttmzZAofDgdTUVLHNtm3b0NDQ4PI+1157rcvw0YIFC/DMM8+gqKjIZU4NADQ0NKChoQEWi+v3otVqhcPxc5h1rniy2WzYsGGDSwVHK5K+qeVMVtqwYQPS0tKQm5uL2NhYXH/99Zg3bx6ampo8vk9dXV2ryVOA9+Tv7cvFX1+EegwpBXqQUXM4yON7GGCYqOXrqEGLPVP0Ci4ML8YUTNUXKfLz87F06VKsXLkSBw4cwMMPP4yamhpkZ2cDAMaNG+cy/DRlyhQUFRXhpZdewsGDBzF37lx88cUXyMvLAwCEhIRg6tSpePbZZ7Fhwwbs3bsX48aNQ0JCAkaMGAEA6NmzJ4YOHYpJkyZh586d+Pzzz5GXl4fRo0cjISEBADBmzBjYbDbk5ORg//79WLt2LV599VXk5+eLfXn++efx1FNPYfny5UhKSoLdbofdbhf3k4mMjMTtt9+Oxx9/HFu3bsXRo0exYsUKvPPOO/jNb34D4OfwUlNTg2XLlqG6ulp8HW/neqUkjQ3Imax05MgRbNmyBVlZWdi8eTMOHz6M3//+92hoaEBBQYHb58yfPx9//OMf3T7miHDAcsH9yaYpzPNyak876zq/kN0tp/a2G2/zk0yblxVocULy5TIEYlsfhpZantT1HF5Skx5hTO1N5ow2TARoc/LXa66L0YILwPBCbRs1ahTOnDmDOXPmwG63o1+/figqKhLPlceOHXOpYAwaNAirV6/G7Nmz8cQTT+Dqq6/G+vXrcf3114ttpk+fjpqaGkyePBmVlZW45ZZbUFRU5FLZWLVqFfLy8jB48GBYLBaMHDkSixYtEh+PiorCP//5T+Tm5iIlJQVdunTBnDlzXCbXvvHGG6ivr8e9997r8pmaT1hes2YNZs2ahaysLFRUVKBbt27405/+hIceeggAsHv3bnHlU48ePVxe5+jRo0hKSlJwdD2TtA/MyZMncdlll2H79u0upbHp06fj008/FT9Ac9dccw1qa2tx9OhRWK1WABeHoV544QWcOnXK7fvU1dW5LFerrq5GYmIiuj3/rLgPjKcQA+i3J4z715C4J4yE9lJWLelBSVAyY1gRX1elwKHFzrlmnqDL8GJ8cisweu4Dk7TsKVjaK/tlcpyvxXc5z2jaX1JOUgVGzmSl+Ph4tGvXTgwvwMXSl91uR319vcv6dKewsDCEhXn/5lC7EgNc/ML2FGJ8vTaSlMqM1PZSKzNaM9LwlR7b9xux2uLE4KI+hhciY5P0rS9nstLNN9+Mw4cPu0z2+eabbxAfH+82vEjhiHB4/ItA7pwYb1/eUr9opc6LkNLen9fd8afmn1uPY6DG3Ba15se4o/ZGdAwvF787GF6IjE/yN7/UyUoPP/wwKioqMGXKFHzzzTfYtGkT5s2bh9zcXNU+hLcQ4+mLSK8Q8/Nr6hNm9Dqx68Ffn8eIE3JbMntwAYwbXojIHCRv8CF1slJiYiI++ugjTJs2DX369MFll12GKVOmYMaMGep9CsgbUnJ+gXqa3Kt0OMkTLYeZWj1XxxCjZFjLLENARngPMw8XAcYMLgDDC5HZyNqhLC8vT1zu1dLWrVtb3ZeWloZ///vfct5KEr1XKLXF33Nm9Gakio8eQULv9zN7cAEYXohIPcbeYlUGvSf3etP8yzrYwoxe9A4qar+33id0BpfWGF4CS1O9FUKzRSNyOOqVPZ/0EXABBjBWiGn+2k5ahxlvzBx0/BlW1O5DsAQXgOGFiLQRkAEG+Hlir7sg468Q0/w9nLQIM76+ltaU9NUIYaU5M1ZbnBhe3GN4ITK3gA0wTp6qMc4vL0+Te7UOMc3fy0nvMKM1I4UQf/aFwcV4GF6IzC/gAwyg7wolJQI5zOjFKKHJnydwfwYXgOGFiPQRFAEG0G6FklxtBSAlYUZaP8wbfIwSVpoLlOBi9BAiF8MLUeAImgADaDO5V67mJxu1w4y0fugTAsw0d0cqI5zs1QovRvgs3vgSQDz+O2Z4IQooQRVgAGOFGCejhBktGTmAyKHmLrhGEQjhRUo7IjK3oAswgDYrlNQiN8wA5go0ejHiSdlIoQUw5jFqjoGEiNwJygDjpPYKJbVJCTOAOiciM4cgo5+IGVykY3ghIk+COsAA6q9Q0orUMCOXnic1ucfPDCdeJ6OFFsA8x4/hhYi8CfoAAyibF6M231Yc/fz/tQwzWjPLiVQqI4YWJzMccwYXIvIFA8z/yA0xapO+fPrn/2/mMKMXX8OFnGPJ4KIcwwspVmsFQhRey6iW10IyAwaYZuRO7tWKkjCjBrMHIiXHQ+8wotbVzeW8rlEwvBCRFAwwbsiZ3Ks1fyyf1vskLjcwGbny4Y3UcGGmMCIFg4v6nH+MEQUyBhgPjDKk5Pb9TboXTFvMGkSkCtQgIgfDi/oYXihYMMB4YeQQI/YjQMNMoDFbaGGwMCeGFwomDDBtaCvE6KmtwKT1SdLsAUnvVWNmCy0Ag4uZMbxQsGGA8YG3EKOn5icXf8/D0YMZ9okxY0hxh8GFiMyGAcZH3lYo+YO/w4weAiUcGB3Di/mx+kLBiAFGIqNUY5oLhjATyPQIEG53lGZwCQgMLxSsGGBkMGKIcdLzpMSwpIyePyuGlcDE8ELBjAFGJn9/cRghQPnjpCgnNPn75N28z/7uC6lP6XeBEf4tE5kRA4xJNf/SDKYvQDMGADP2mXyjxh8y/v5jiMisGGACQLCGGSJ/YvAwJkt9CCyWEGUvUq/w+aQLBhgfOcIEAIClzti/2P76UmVwIj1o/fvt6+8xwwuR/zHASOQIEwwfYvyBwYm0psfvGIMJkXkwwMjAEGMcPOEQEQUnBhiZnENKRsaQRUREgYoBxgszhBRv/Nl/hiciItJS0AcYs4cUo9LquDIYEREREEQBhkElMKj5c2QYIiIyr4BawuEIEzzeNBfeZOwbteLX3xciCkoVFRXIyspCZGQkoqOjkZOTg3Pnznl9Tm1tLXJzc9G5c2d07NgRI0eORHl5uUubY8eOITMzE+3bt0dMTAwef/xxNDY2urTZunUrbrzxRoSFhaFHjx5YsWJFq/dasmQJkpKSEB4ejtTUVOzcuVNyXx599FGkpKQgLCwM/fr1a/UeW7duxd133434+Hh06NAB/fr1w6pVq7weA3dMG2AMc9IxS0BgeJLEW7hR80ZEwSUrKwv79+9HcXExNm7ciG3btmHy5MlenzNt2jT84x//wLp16/Dpp5/i5MmTuOeee8THm5qakJmZifr6emzfvh0rV67EihUrMGfOHLHN0aNHkZmZiTvuuAN79uzB1KlTMXHiRHz00Udim7Vr1yI/Px8FBQXYvXs3+vbti4yMDJw+fdrnvjg98MADGDVqlNvPs337dvTp0wd/+9vf8NVXXyE7Oxvjxo3Dxo0bfT6OABAiCILhv0Wrq6sRFRWFbs8/C0t4OAADDAmZ9MRsGLVWf/fAsDi0RYHGUVuL72fMRlVVFSIjIzV5D3fnCbmc/T1+/LhLf8PCwhAWJv/aIAcOHECvXr2wa9cu9O/fHwBQVFSEYcOG4YcffkBCQkKr51RVVaFr165YvXo17r33XgDAwYMH0bNnT5SWlmLgwIH48MMPcdddd+HkyZOIjY0FABQWFmLGjBk4c+YMbDYbZsyYgU2bNmHfvn3ia48ePRqVlZUoKioCAKSmpuKmm27C4sWLLx4HhwOJiYl45JFHMHPmTJ/60tzcuXOxfv167Nmzp81jk5mZidjYWCxfvtzn42naCowuAqyqYChBXuHxhhUaIvksFyyq3AAgMTERUVFR4m3+/PmK+lZaWoro6GgxvABAeno6LBYLduzY4fY5ZWVlaGhoQHp6unhfcnIyrrjiCpSWloqv27t3bzG8AEBGRgaqq6uxf/9+sU3z13C2cb5GfX09ysrKXNpYLBakp6eLbXzpi1xVVVXo1KmTpOcEzSRerzQ4CVrDjHtibaoLsOqH3J+fCapAZrmEBVEgcleBUcJutyMmJsblvtDQUHTq1Al2u93jc2w2G6Kjo13uj42NFZ9jt9tdwovzcedj3tpUV1fjwoUL+PHHH9HU1OS2zcGDB33uixzvvvsudu3ahT//+c+Snhc8AUanv9SNHFyc9O6jYQNTy98JAwcaf1VjGJwomEVGRvo05DVz5kw8//zzXtscOHBArW4FlE8++QTZ2dlYunQprrvuOknPDbwA46chBTMEF39xd2wMGWqa/+4YOMzoiRUgorY99thjmDBhgtc2V155JeLi4lwmxAJAY2MjKioqEBcX5/Z5cXFxqK+vR2VlpUvlo7y8XHxOXFxcq9VCzpVBzdu0XC1UXl6OyMhIREREwGq1wmq1um3T/DXa6osUn376KYYPH45XXnkF48aNk/z8wJoD44fwYg1rYniRwXncmt8MJYjm3PiC83KIPOvatSuSk5O93mw2G9LS0lBZWYmysjLxuVu2bIHD4UBqaqrb105JSUG7du1QUlIi3nfo0CEcO3YMaWlpAIC0tDTs3bvXJRwVFxcjMjISvXr1Ets0fw1nG+dr2Gw2pKSkuLRxOBwoKSkR2/jSF19t3boVmZmZeP7559tcheVJ4FVgdCL1hBsWXq9RT5Spq7X5uwsitUOMZlUerUKMSao+rMoQydOzZ08MHToUkyZNQmFhIRoaGpCXl4fRo0eLK5BOnDiBwYMH45133sGAAQMQFRWFnJwc5Ofno1OnToiMjMQjjzyCtLQ0cdXPkCFD0KtXL4wdOxYLFiyA3W7H7NmzkZubK87beeihh7B48WJMnz4dDzzwALZs2YJ3330XmzZtEvuXn5+P8ePHo3///hgwYAAWLlyImpoaZGdnA4BPfQGAw4cP49y5c7Db7bhw4YK4CqlXr16w2Wz45JNPcNddd2HKlCkYOXKkOH/GZrNJmsjLANMGpSdVowYXJ73654+g5MvPzlBDWSYbwmKQIZJu1apVyMvLw+DBg2GxWDBy5EgsWrRIfLyhoQGHDh3C+fPnxfteeeUVsW1dXR0yMjLw+uuvi49brVZs3LgRDz/8MNLS0tChQweMHz8eTz/9tNime/fu2LRpE6ZNm4ZXX30Vl19+Od566y1kZGSIbUaNGoUzZ85gzpw5sNvt6NevH4qKilwm9rbVFwCYOHEiPv30U/G/b7jhBgAX96JJSkrCypUrcf78ecyfP99lZdftt9+OrVu3+nwsA2sfGJl/GWsxfGH04OJvRqr8OBkqzDiZIMg0xzBD7ui5D0z3ufNU2Qfm6NwnNO0vKRdUFRg95lkwuPjG3XHyd6hp/vthmDDDqgwRkVsBH2D0mhwqNbh0CDNO0KmpM0Y1xEihpuXvjSECjYnCjJaTfRmOiAgIggCjNTMHFyc9+iQ3JGlR0ZITigxXnXGGGYMHGS2wykNEAAOMbIEQXPTk7vP7q/LT8mcnNdB4q+rpHm5MVJVRW/MqD8MMUfBhgGmD0gpAsAcXb4wSapr/jJUOWSkdslQUgFiVYZAhWOoAq9JfgzpVukIaY4D5H7WHKqQEl45h/v/Xcq5O2TU+1NLyuOkdaNQMM3KoMlSl9j41JgpEDDJEwSMoA4yWK4WkVlyMEF4AffohJyT5s0pjlDDj9zk3JqzsMMgQBb6gDDBaMGtw0VPLzyy36qPWsJyUIOTPMGOYCcQMMkRkIAwwCjG4yOfuWOg5lNX8Zyc3zAD6BhpDVGVMOHGYQYYo8DDAKKDVPJfosFo53VGssk7Z7pVq8FeokRtmAP9UZ1iVkYdBhihwMMD4QMmQhRmCi17vLzcgqTX05Cu1wkxLWoUbQ1VlGGSISCcMMM2oueRZ6lCRv8OLHtx9RjmhRs8qjZIw05Kvk8flBh1DVGVMNrzEIENkXgwwKmNwkabl51erSuMLqaFHzTDjjTPoKKnYGCrMMMgQkQYYYFSiZXDpHFYjtTuy/beug27v5Y5aVRpfNP+ZGTHMqDW3xu9DTAwyRKQBBhgVaDXPRc/gosd7yg1HeoQaJfNs9Awzpq7KMMgQkYoYYBTQcoKuP8KL1tx9JqOGGrnVGa13Eg6IqoyJ5skwyBAZFwOMj+Tu38Lg4p2WoUatQGPUoSY1wowv127SNOSYpCrT/MKRnjDkGIO1DlD828TtukyBAaYNRgwundppG3IqGvw7D0atUOPLz0BqyFErzDSnRrBRY4jJE12GnkwSZLxhtYZIXwwwGtBqnovWwUWP95EbjloeJ7UmGzf/WekZZppTc9hJ6031NB96MtHwkicMMkT6YIBRkdmDix7cfRY5oUbNoScnI4QZQL1hJz2qMhxe8swRJjDEEGnIIudJS5YsQVJSEsLDw5GamoqdO3f69Lw1a9YgJCQEI0aMkPO2hhUdVutzeOkcVhO04cWTTu1qXG5yOY9t85tczp+pnH16OobViTelOoTVK95gMSy8XrypzRrWJN40E97U+mYSjjDBp/kzRCSd5ArM2rVrkZ+fj8LCQqSmpmLhwoXIyMjAoUOHEBMT4/F53333Hf7whz/g1ltvVdRhI9FySbSUE3mXduckvbYvzjZ0VP01faVWlQZQZ+hJyeRgLYaZgroq42Sy6gyHlYjUJznAvPzyy5g0aRKys7MBAIWFhdi0aROWL1+OmTNnun1OU1MTsrKy8Mc//hH/+te/UFlZ6fU96urqUFf38xd/dXW11G5qKpCDi9avLTcYGWnoSe5Qk1rXdFIjzGg5V4ZBxjMGGSL1SBpCqq+vR1lZGdLT039+AYsF6enpKC0t9fi8p59+GjExMcjJyfHpfebPn4+oqCjxlpiY6PK4FiVZNa+D5CR1qMjX8NKl3TlNw4uWnH1X4zOoMewESB/Wa07uMBMAVYaZ1Pi91Xp4SRcmGlYCtPkOIwo2kgLM2bNn0dTUhNjYWJf7Y2NjYbfb3T7ns88+w7Jly7B06VKf32fWrFmoqqoSb8ePH2/7SW6+wJR+eco9uUg5IUo9AZs1uHiiVhhTK8jIpeSaVkYIMYDvF5s0LJOFGCJSRtNVSD/99BPGjh2LpUuXokuXLj4/LywsDGFh7svrSv9ycfclLeUE4O1EZdbhIiNwfj6lc2+aH1MlQ0xK5srIvcK20jkyau0no8WQkm47/oY3mWY4iYiUkRRgunTpAqvVivLycpf7y8vLERcX16r9f/7zH3z33XcYPny4eJ/D4bj4xqGhOHToEK666io5/XZlgOqLViuLAj24tNT886oVZuQGGSU7AjPEuNI1xJgEl1kTKSMpwNhsNqSkpKCkpERcCu1wOFBSUoK8vLxW7ZOTk7F3716X+2bPno2ffvoJr776aqu5LVrTqvoiZbhICinhpUvoT5JeWy9nGy+R/Vx/V2UYYkwaYliFIQoKkoeQ8vPzMX78ePTv3x8DBgzAwoULUVNTI65KGjduHC677DLMnz8f4eHhuP76612eHx0dDQCt7pfNz9UXLcJLIAQXp+b9kxtm/FmV8ceQEkOMCkwSYliFUV9oLWBVOEc6hNdCMgXJAWbUqFE4c+YM5syZA7vdjn79+qGoqEic2Hvs2DFYLLL2x9OUlnNfvNFyuMjo4aUlZ3+NUpUxcjWGIYaIyDtZk3jz8vLcDhkBwNatW70+d8WKFXLe0mdGqb4E43CRr4wSZOSEGEB+NcasIca0WIUhCmjGK5VoQIvqi1rhRcoy4i6hP5k+vDSnxudRuqeMnOXXeu4ZY4Ql1lrtEaMLLq0mClgBFWD8te+LHFJPuoEUXFpSK5gpDTJS6LlnDENM4OPGdkTSBfzVqJV+8WpRfdEyuMS1q5TUviV7Q7Si5yuhxtASIH94Se8hJcD3Cb5GGE4y7XwYkwwlEZE0AR9g3HH3F6ka1Ze2wouRg4var9OSlGCkdpABfA8zUkMMoN8EX4YYBUwQYjgXhkiagBpCaknv6os3WoWXuHaVmoUONcnpp3NoSe/hJT3nxQDShpQCdTiJiEiqgA4w7vij+qLFJF2zBJeW5PbbX0FGCqUXhfRVIIYYXebDcEIvGUBFRQWysrIQGRmJ6Oho5OTk4Nw5799JtbW1yM3NRefOndGxY0eMHDmy1Y74x44dQ2ZmJtq3b4+YmBg8/vjjaGxsdGmzdetW3HjjjQgLC0OPHj3crgpesmQJkpKSEB4ejtTUVOzcudPl8QcffBBXXXUVIiIi0LVrV9x99904ePCgpL5MmDABISEhrW7XXXedL4dQFHQBxldSqi9Kw4vUk7MZg0tLRggyvpBzgUiGGHkYYjiZNxhkZWVh//79KC4uxsaNG7Ft2zZMnjzZ63OmTZuGf/zjH1i3bh0+/fRTnDx5Evfcc4/4eFNTEzIzM1FfX4/t27dj5cqVWLFiBebMmSO2OXr0KDIzM3HHHXdgz549mDp1KiZOnIiPPvpIbLN27Vrk5+ejoKAAu3fvRt++fZGRkYHTp0+LbVJSUvD222/jwIED+OijjyAIAoYMGYKmpiaf+/Lqq6/i1KlT4u348ePo1KkTfvvb30o6liGCIBj+X0x1dTWioqLQ7flngagWcwCafSG1/AJs+QUrpfqiR4Dx1zwXI1IyeVjpXBlf5sfIuZ6S3HkxUvaLUTInBoAq+8SoPSdG8/kwBp8LE4jzYBy1tfh+xmxUVVUhMjJSk/dwnid6/X4erGHSL+PRXFNdLb5+/QnV+3vgwAH06tULu3btQv/+/QEARUVFGDZsGH744QckJCS0ek5VVRW6du2K1atX49577wUAHDx4ED179kRpaSkGDhyIDz/8EHfddRdOnjwpbipbWFiIGTNm4MyZM7DZbJgxYwY2bdqEffv2ia89evRoVFZWoqioCACQmpqKm266CYsXLwZw8VJBiYmJeOSRRzBz5ky3n+mrr75C3759cfjwYVx11VU+9aWl9evX45577sHRo0fRrVs3n48nKzBuGC28mHW4SAoln1FpVcaXaozceTFyqjFmq8SoTfNKDKsw5KPq6mqXW12dsn9vpaWliI6OFsMLAKSnp8NisWDHjh1un1NWVoaGhgakp6eL9yUnJ+OKK65AaWmp+Lq9e/cWAwMAZGRkoLq6Gvv37xfbNH8NZxvna9TX16OsrMyljcViQXp6utimpZqaGrz99tvo3r27eG1DX/rS0rJly5Ceni4pvABBtApJi7kvWocXqSf0rlZjXrn6TJPvy5mdn1lORUbJdZe6tDvnUyVGr1VKZlqdpMXKJM2ZYFUSyWOtU34tJPzvdNHygsMFBQWYO3eu7Je12+2IiYlxuS80NBSdOnWC3W73+BybzSZeR9ApNjZWfI7dbncJDM7HnY95a1NdXY0LFy7gxx9/RFNTk9s2Lee4vP7665g+fTpqampw7bXXori4WKys+NKX5k6ePIkPP/wQq1evdvv5vWEFpgU1Vx4pISW8dLWeM2x4AeT1T2nFSU5VRut5MVJ/j8xUiTHlfBgDYxXGGI4fP46qqirxNmvWLLftZs6c6XZSavNbyxBgZllZWfjyyy/x6aef4pprrsF9992H2lp51wtcuXIloqOjMWLECMnPDZoKTEtGrb5IDS5m4uyvrxUZJdUYJ6l7yvi6CZ6cSgwgvRoTzJUYzfeHYRWG2hAZGenTHJjHHnsMEyZM8NrmyiuvRFxcnMuEWABobGxERUUF4uLi3D4vLi4O9fX1qKysdKnClJeXi8+Ji4trtVrIuUqpeZuWK5fKy8sRGRmJiIgIWK1WWK1Wt21a9i0qKgpRUVG4+uqrMXDgQFx66aX44IMPcP/99/vUFydBELB8+XKMHTvW7dyYtrAC04wa1ReGl7ZJrcioMQdI6p4yWs2LAaT/TrESoyEDz4dhFcY8unbtiuTkZK83m82GtLQ0VFZWoqysTHzuli1b4HA4kJqa6va1U1JS0K5dO5SUlIj3HTp0CMeOHUNaWhoAIC0tDXv37nUJR8XFxYiMjESvXr3ENs1fw9nG+Ro2mw0pKSkubRwOB0pKSsQ27giCAEEQxPlBvvTF6dNPP8Xhw4eRk5Pj8fW9YYCRQc5Jqy2+nqCNPlwkhT+CDOD78JKRhpQYYjRk4BBDgaVnz54YOnQoJk2ahJ07d+Lzzz9HXl4eRo8eLa5AOnHiBJKTk8UqRlRUFHJycpCfn49PPvkEZWVlyM7ORlpaGgYOHAgAGDJkCHr16oWxY8fi//7v//DRRx9h9uzZyM3NRVjYxarsQw89hCNHjmD69Ok4ePAgXn/9dbz77ruYNm2a2L/8/HwsXboUK1euxIEDB/Dwww+jpqYG2dnZAIAjR45g/vz5KCsrw7Fjx7B9+3b89re/RUREBIYNG+ZzX5yWLVuG1NRUXH/99bKOJwPM//iz+uLLSTmQgktL/gwybbbRMMQA0n7HGGI0xBBDOlm1ahWSk5MxePBgDBs2DLfccgvefPNN8fGGhgYcOnQI58+fF+975ZVXcNddd2HkyJG47bbbEBcXh/fff1983Gq1YuPGjbBarUhLS8Pvfvc7jBs3Dk8//bTYpnv37ti0aROKi4vRt29fvPTSS3jrrbeQkZEhthk1ahRefPFFzJkzB/369cOePXtQVFQkTsINDw/Hv/71LwwbNgw9evTAqFGjcMkll2D79u3i5GRf+gJcXB7+t7/9TXb1BQiifWBafiG3/IL3NcB4OlFpOXQUqMHFEymrlgDlF6D0dX6MVvvFANL2jDHTPjFqzokJ1v1hAmFfGD33gemdMw9Wm8J9YOprsXeZ+vvAkLqCdhJvc/5ceeQtvEgJLl0tyi+HoKUzDt9PpHIn+wLyl1/7EmJ8WWqtZHIv4FuQCdaJvcE6qZcXeSRyj0NIEqhdfQmW8AJc7KPUfsqpPCm5RIFP7TSc3Av4HpzNNJykJg4lEZETA4wbUqoval4qQCo5ocDfpPZZ7twfJVe/brOdQebFmCXEcD6MclyRRNQaA4yP1F55pKT6Ysbg0pLeQUZKmPE1xPhajZGDIcY7hhgiYoBRwB9DR2YPLi3pFWQAaVUZtYeU5GCIISLyjAHGB+5OQL4OI6gp0MJLc0YMMkaYF8MQ4xmrMOROaK2gyo2Mj6uQNKBF9UXr8NLVqt3F+M40+X7Scn5OX1ctSV2x1Fxcu8o2Vy35eikCI1wM0iyrk7gySRmuSiK6iAGmDVKrL/4KL1oGEKWcfTNikPElxAC+LbXW8jpKRgwxSjDEEJFSHEIyua5Wm6HDS3Ny+ipn6bWcK1+bYUjJaMNJRtqtN9iuXs2hJCIGGMmMVH0xS3BpSWqQkbuHjJwg0xY1QwwgfYJvoIUY0+B8GCLDYYBRCcOLdEYMMr6GGLX3i5ESZAIpxJiqCmPAEEMUzBhgyO+MFmTUXGotZbUaQ4xywRZiWIWhYMYAowJWX9RhpCCjdojRYkiJIcY9hhii4MAAQ4aj9URfX6m96Z0/6blPjFzc6I6IpGCAISIiItNhgCEiIiLTYYAhIiIi0+FOvGQ4UnbsJSJqzlonINShcGJzAydGmwErMG2QuuU7ERERaY8BBvD52jFERERkDEETYOReOZeIiIiMJ6ACjKZXpCUiIiLDCKgAQ0RERMGBAYaIiIhMhwGGiIiITIcBxoS4TwoREQU7Bhg/OtPU0fNjjjAde0JERGQuDDAqONt4icfH7A3R+nWEiIgoSDDAEBERkenwWkhERBQwQs87ENrOoexFGhQ+n3TBCgwRERGZDgOMRGcbPE+8JSIiIn0wwLjx3zpegZqIiMjIgjbAnKvjMmUiIiKzCtoAQ0RERObFAENERESmwwBDREREpsMAQ4Yi5zpPvOwCEZlFRUUFsrKyEBkZiejoaOTk5ODcuXNen1NbW4vc3Fx07twZHTt2xMiRI1FeXu7S5tixY8jMzET79u0RExODxx9/HI2NjS5ttm7dihtvvBFhYWHo0aMHVqxY0eq9lixZgqSkJISHhyM1NRU7d+5s1aa0tBS//OUv0aFDB0RGRuK2227DhQsXfP6MW7duxd133434+Hh06NAB/fr1w6pVq3w5fC4YYIiIiHSSlZWF/fv3o7i4GBs3bsS2bdswefJkr8+ZNm0a/vGPf2DdunX49NNPcfLkSdxzzz3i401NTcjMzER9fT22b9+OlStXYsWKFZgzZ47Y5ujRo8jMzMQdd9yBPXv2YOrUqZg4cSI++ugjsc3atWuRn5+PgoIC7N69G3379kVGRgZOnz4ttiktLcXQoUMxZMgQ7Ny5E7t27UJeXh4slp/jRFufcfv27ejTpw/+9re/4auvvkJ2djbGjRuHjRs3SjqWIYIgCJKe4QfV1dWIiopCt+efBaJa/LUd3uTyn9awn/87LNz1r/kOYa7/3TGsTvz/0WG1Lo91DqsR/3+ndjUuj3Vp1zotdwn9yWP/49pVenysq9Vz8u5qqfP8mNXm8TEz06sC4+1Cms35ei0rb9fDEtv4uIdQRYPvy/ilLPmvrAv3ua3cVXo1dcp+L+tq1fm9bqqzqvI6HtVq/PoSWepC/N0Frxy1tfh+xmxUVVUhMjJSk/dwnicGDnsaoe18/113p7GhFv/ePEf1/h44cAC9evXCrl270L9/fwBAUVERhg0bhh9++AEJCQmtnlNVVYWuXbti9erVuPfeewEABw8eRM+ePVFaWoqBAwfiww8/xF133YWTJ08iNjYWAFBYWIgZM2bgzJkzsNlsmDFjBjZt2oR9+/aJrz169GhUVlaiqKgIAJCamoqbbroJixcvBgA4HA4kJibikUcewcyZMwEAAwcOxJ133olnnnlGtc8IAJmZmYiNjcXy5ct9Pp6swKhE7gUd5V6RWs6JnrThS3ghIvOprq52udXVef6j0helpaWIjo4WT+wAkJ6eDovFgh07drh9TllZGRoaGpCeni7el5ycjCuuuAKlpaXi6/bu3VsMLwCQkZGB6upq7N+/X2zT/DWcbZyvUV9fj7KyMpc2FosF6enpYpvTp09jx44diImJwaBBgxAbG4vbb78dn332maLPCFwMap06dfL4uDsMMEREFDBCax0IvaDwVnvxWkiJiYmIiooSb/Pnz1fUN7vdjpiYGNf+hoaiU6dOsNvtHp9js9kQHR3tcn9sbKz4HLvd7hJenI87H/PWprq6GhcuXMDZs2fR1NTkto3zNY4cOQIAmDt3LiZNmoSioiLceOONGDx4ML799lvZn/Hdd9/Frl27kJ2d7fZxTxhgiIiI3Dh+/DiqqqrE26xZs9y2mzlzJkJCQrzeDh48qHPv1edwXAx2Dz74ILKzs3HDDTfglVdewbXXXitp6Ke5Tz75BNnZ2Vi6dCmuu+46Sc/l1aiJiIjciIyM9GkOzGOPPYYJEyZ4bXPllVciLi7OZUIsADQ2NqKiogJxcXFunxcXF4f6+npUVla6VGHKy8vF58TFxbVaLeRcpdS8TcuVS+Xl5YiMjERERASsViusVqvbNs7XiI+PBwD06tXLpU3Pnj1x7Ngx8X18/Yyffvophg8fjldeeQXjxo1z+/m9YQXGB1ImVRIRUXDp2rUrkpOTvd5sNhvS0tJQWVmJsrIy8blbtmyBw+FAamqq29dOSUlBu3btUFJSIt536NAhHDt2DGlpaQCAtLQ07N271yU4FBcXIzIyUgwbaWlpLq/hbON8DZvNhpSUFJc2DocDJSUlYpukpCQkJCTg0KFDLq/zzTffoFu3buL7+PIZt27diszMTDz//PNtrsLyhAGGiFTTcqUfEf2sZ8+eGDp0KCZNmoSdO3fi888/R15eHkaPHi2uzjlx4gSSk5PFikpUVBRycnKQn5+PTz75BGVlZcjOzkZaWhoGDhwIABgyZAh69eqFsWPH4v/+7//w0UcfYfbs2cjNzUVY2MXFIA899BCOHDmC6dOn4+DBg3j99dfx7rvvYtq0aWL/8vPzsXTpUqxcuRIHDhzAww8/jJqaGnFuSkhICB5//HEsWrQI7733Hg4fPoynnnoKBw8eRE5Ojs+f8ZNPPkFmZiYeffRRjBw5Ena7HXa7HRUVFZKOJ4eQiIiIdLJq1Srk5eVh8ODBsFgsGDlyJBYtWiQ+3tDQgEOHDuH8+fPifa+88orYtq6uDhkZGXj99dfFx61WKzZu3IiHH34YaWlp6NChA8aPH4+nn35abNO9e3ds2rQJ06ZNw6uvvorLL78cb731FjIyMsQ2o0aNwpkzZzBnzhzY7Xb069cPRUVFLhN7p06ditraWkybNg0VFRXo27cviouLcdVVV/n8GVeuXInz589j/vz5LhOjb7/9dmzdutXnYylrH5glS5bghRdegN1uR9++ffHaa69hwIABbtsuXboU77zzjrj2PCUlBfPmzfPY3h1/7wMDuO4F424fGIB7wajBjPvA+LqMOhj2gQGU7QXDfWDk4T4wP58nbvnlXISGKtwHprEWn22Zq2l/STnJQ0i+7NTX3NatW3H//ffjk08+QWlpKRITEzFkyBCcOHFCceeJiIgoOEkOMC+//DImTZqE7Oxs9OrVC4WFhWjfvr3HJVSrVq3C73//e/Tr1w/Jycl46623xIlBntTV1bXaQIiIiIjISVKA8WWnvracP38eDQ0NXnfcmz9/vsvmQYmJiVK6SSbF3YWJiMhXkgKMLzv1tWXGjBlISEhotaVxc7NmzXLZPOj48eM/d1jCWK+U8XQp8wM8zWXg5QTkkzv3Rer8lzNNHVW/DpIvfJ3/YnZGmP9CRMFB12XUzz33HNasWYMPPvgA4eGeA0NYWJi4gVCbGwlJmEwn5ctVyuTI5hhipDPaxF1A3Ys4SmH2CbxBwWATeImClaRl1F26dGlzpz5PXnzxRTz33HP4+OOP0adPH+k99VFTndVlJZI35+rCXFYieVPR0MFlJdLZho4eVyOdbbzE44oke0O0xxVJZ5o6elyRdMYR5nFF0pmmelOuSDJicAHUDy++VF+kbpZoxPBilOqL5iuQDMboK5D0Zr3QCGtoo6LXEBqVPZ/0IakC48tOfe4sWLAAzzzzDIqKilyuUCmXEYaRAO8nprONl3g8wdkboj2eJL0NcXgbMjnTVO/1ZiRy+qT1cJGTGcILERHJ2MguPz8f48ePR//+/TFgwAAsXLjQZae+cePG4bLLLhM3p3n++ecxZ84crF69GklJSeJcmY4dO6JjR5XmBdRaXfaD8VaFqamzuewH460K89+6Dq32g2nJWyUG0L8a44keIcZbFUju++tRcXHyR3iRg9UXz4Jt/xeiYCY5wLS1U9+xY8dgsfxc2HnjjTdQX1+Pe++91+V1CgoKMHfuXGW991Fdra3VpnaeVNaFt9rUzqnlMJKTWUKM1tQOSXqFFymTdf25aR1gzPBC/sPhIwpmsi4lkJeXh7y8PLePtdwG+LvvvpPzFtK1qMJ407IK440vVRiAIUZNRgwugP/Di1EZpfqiOVZfiAwlYC/mKKWULOWvT28nnbMNHducF+OJFiuUzEbPeS5mDC+svnjWVGcNusm7RMEuYANMS1pO5m2JIUYaOcEF0KfqAjC8eGOE6guDC1FwCqyrUftxMm9L/lhm3ZKRh5eUhC69ggvg//Aidz8irSkJLmrSLbwYcPiI818o2AVWgGmD1pN5W2orxADur2DtPNm6CzLOk7e3K1iLbXWozPgaktToi57BBTBneNGj+qJGeFGj+sLKC1FwC6oA05IWk3lbcp7c9KzG6EmPkGTU4AJoE16MWnUBjFF50T24GLD6QkRBMAdGy8m8Uk5Kes+LCRR6DxepfWkAwPfw8t+6DorCi5bVl5o6m2rhxVQrjwyKw0dEQRBgWlJ7Mi9DjPqcK4v0WF0EyA8uau6yq7TqonV4MQoOGxGRU1AMIWk9mdfXOTGA/pN7zUJJENNyZZHH56u4y24wDRmZrvrC4SPTsV5ohNXaoOg1hCZeC8kMgq4CA2izpNoIlRgzVWOaV1nMFF7a2uunOV9+J9SovGhVfTFS5QVg9YWIXAVFBaYtak3mdZ6w1FihJKcSA+gzpCS32qNm3/wRXKTQOrzI2avI3+GF1Rd1cP4L0UVBE2C8DSO15G0YyRdSllkD7lcoyV1mrQd/VnqMHlwA7S8RoHSjRTNi9YWIWgrKISRA+515/T2kFEick3PlbP8vN7xIGSpqTstJu1KHi5pj9UUig1ZfiOhnQRtgWmr5pe3tC9/Xkw9DjHxyQ4uT3sEF0C68KAkugPnDC6svRORO0AwhuaPWzrze+HuFklmoFcSUBBcltAwvZmW6qosJcP4L0c+COsC0JHUyLwCfduc1wuReI1KzemT04AJod0FGb/xVfTH1hRo5fERkCgwwXvgymVfKJQbUuoaStxCjFTXCkRb980dw0fJq0v4ILmpi1YWI9BL0AUaNYSQjhRitGGmujd6ripzkri4yQ3gxygUam2P1xRWHj4hcBX2AaUnKzrzNaRViAOnLrAORP3fOVbIsWs/wYvarSzfH8EJEbWGAkcjbZF6pIQbQdl6Mmal1UUV/BRdAv/Dir+EiIICGjAweXlh9IWqNAcYNKZN5W5IyuRcw75CSFtS+ErTWy6E90XOyrtLwoqT6olV40b36wvASUCw1dbAo/JFamuRvZEr6YYDxgZydeYNhXoxSagcW8XVNEFwAZeFFjaqL3PDCqot+GF6IPGOA8cBbFcbXPWH8EWLUpEUg0iK06LkU2h25u+rKodZwkVHDi27VF4YXItNjgPGR3OsjaTEvxluIUZNWFRI1KA0tgH+CC2DO8KJH1YXh5WcML0RtY4CRScrOvFrMi9ErxBiFGoHFSY9VRZ5ofRVpb4xadQF0Ci8mCC4AwwuRrxhgFHCejKQEGbVDDOB+mXUgUDO0AAwucnDISF8ML0S+Y4CRwNMwktRqjJ7zYsxE7cDiZLbgAjC8qIrhhSggMcCohCFGHi1Ci9K5LUBwBxet6bpMmuGFKGBZ/N0Bs/F2gpFy0vpvXQefT5IVDR18Oimfbeio6KYHrd7TeYzUmJirJLxU1oX7NbzU1NkMubOuk65VF4YXMqCKigpkZWUhMjIS0dHRyMnJwblz3v/4rK2tRW5uLjp37oyOHTti5MiRKC8vd2lz7NgxZGZmon379oiJicHjjz+OxsZGlzZbt27FjTfeiLCwMPTo0QMrVqxo9V5LlixBUlISwsPDkZqaip07d7o8brfbMXbsWMTFxaFDhw648cYb8be//c2lze7du3HnnXciOjoanTt3xuTJk91+xhUrVqBPnz4IDw9HTEwMcnNzvR6HlliBUZm/58UooVeIUYMaVZbm/FVxcfLnvi7NmT64AKYJLgDDSzDKysrCqVOnUFxcjIaGBmRnZ2Py5MlYvXq1x+dMmzYNmzZtwrp16xAVFYW8vDzcc889+PzzzwEATU1NyMzMRFxcHLZv345Tp05h3LhxaNeuHebNmwcAOHr0KDIzM/HQQw9h1apVKCkpwcSJExEfH4+MjAwAwNq1a5Gfn4/CwkKkpqZi4cKFyMjIwKFDhxATEwMAGDduHCorK7FhwwZ06dIFq1evxn333YcvvvgCN9xwA06ePIn09HSMGjUKixcvRnV1NaZOnYoJEybgvffeEz/Tyy+/jJdeegkvvPACUlNTUVNTg++++07SsQwRBEGQ9Aw/qK6uRlRUFLo9/yws4RdPFI4wD90Ob/L4OtYw9495upijp31gfF1O7WuIAXxfoQT4dvmBQKR2aAEYXJpjeNFXMIUXR20tvp8xG1VVVYiMjNTkPZznifSrpyHUquzfVWNTHT7+9hXV+3vgwAH06tULu3btQv/+/QEARUVFGDZsGH744QckJCS0ek5VVRW6du2K1atX49577wUAHDx4ED179kRpaSkGDhyIDz/8EHfddRdOnjyJ2NhYAEBhYSFmzJiBM2fOwGazYcaMGdi0aRP27dsnvvbo0aNRWVmJoqIiAEBqaipuuukmLF68GADgcDiQmJiIRx55BDNnzgQAdOzYEW+88QbGjh0rvk7nzp3x/PPPY+LEiXjzzTfx1FNP4dSpU7BYLg7y7N27F3369MG3336LHj164Mcff8Rll12Gf/zjHxg8eLDs42naISSP//hV/ALzdGLw9aQjZThBiyElM2s+JKTm53UeZyVDRc6fq5KhIudNCTWHixhe9BVM4cXMqqurXW51dcouMVBaWoro6GgxvABAeno6LBYLduzY4fY5ZWVlaGhoQHp6unhfcnIyrrjiCpSWloqv27t3bzG8AEBGRgaqq6uxf/9+sU3z13C2cb5GfX09ysrKXNpYLBakp6eLbQBg0KBBWLt2LSoqKuBwOLBmzRrU1tbiF7/4BQCgrq4ONptNDC8AEBERAQD47LPPAADFxcVwOBw4ceIEevbsicsvvxz33Xcfjh8/7uOR/F//JLU2GKkhxtOXqbcvb28hxteTkJQTnpSTq9ond3/SIqwA6gQWJyWhBYAqocXJDMGF4cU9hheNnb8A1Ci8nb8AAEhMTERUVJR4mz9/vqKu2e12cSjGKTQ0FJ06dYLdbvf4HJvNhujoaJf7Y2NjxefY7XaX8OJ83PmYtzbV1dW4cOECzp49i6amJrdtmvft3XffRUNDAzp37oywsDA8+OCD+OCDD9CjRw8AwC9/+UvY7Xa88MILqK+vx48//ihWb06dOgUAOHLkCBwOB+bNm4eFCxfivffeQ0VFBe68807U1/t+HULTz4Gx1IW4H06qtbodTmqqs7odSqqrtXkcSmrr4o7NT0ptDS81PwG2NcTU/ITb1hCTP0KM3KEsrfqqNKB4YoRhIiejDxcBvBgjBY7jx4+7DCGFhbn/tzxz5kw8//zzXl/rwIEDqvbNX5566ilUVlbi448/RpcuXbB+/Xrcd999+Ne//oXevXvjuuuuw8qVK5Gfn49Zs2bBarXi0UcfRWxsrFiVcTgcaGhowKJFizBkyBAAwF//+lfExcXhk08+EefktMX0AQYwRohxMkKY0Ys/Kz9ahRUnI4UWwBzBBdAhvJg8rLD6Yi6RkZE+zYF57LHHMGHCBK9trrzySsTFxeH06dMu9zc2NqKiogJxcXFunxcXF4f6+npUVla6VGHKy8vF58TFxbVaLeRcpdS8TcuVS+Xl5YiMjERERASsViusVqvbNs7X+M9//oPFixdj3759uO666wAAffv2xb/+9S8sWbIEhYWFAIAxY8ZgzJgxKC8vR4cOHRASEoKXX34ZV155JQAgPj4eANCrVy/xfbp27YouXbrg2LFj3g6jC1MPITXnz+EkT6QMGciZL6P1SdxI1BwK8sZIw0RORh8uAnQaMmJ4IYPq2rUrkpOTvd5sNhvS0tJQWVmJsrIy8blbtmyBw+FAamqq29dOSUlBu3btUFJSIt536NAhHDt2DGlpaQCAtLQ07N271yUcFRcXIzIyUgwJaWlpLq/hbON8DZvNhpSUFJc2DocDJSUlYpvz588DgMv8FgCwWq1wOByt+h4bG4uOHTti7dq1CA8Px5133gkAuPnmm8XP4VRRUYGzZ8+iW7dubo+DOwFRgXEyUiWmOa2qMoD06yyZgVGHgjxRYzKulkxfdQEYXigg9OzZE0OHDsWkSZNQWFiIhoYG5OXlYfTo0eIKpBMnTmDw4MF45513MGDAAERFRSEnJwf5+fno1KkTIiMj8cgjjyAtLQ0DBw4EAAwZMgS9evXC2LFjsWDBAtjtdsyePRu5ubnisNdDDz2ExYsXY/r06XjggQewZcsWvPvuu9i0aZPYv/z8fIwfPx79+/fHgAEDsHDhQtTU1CA7OxvAxcnDPXr0wIMPPogXX3wRnTt3xvr161FcXIyNGzeKr7N48WIMGjQIHTt2RHFxMR5//HE899xzYgXpmmuuwd13340pU6bgzTffRGRkJGbNmoXk5GTccccdPh/PgAowgHFDjJMeQ0z+JiVMmS2sNBfswQXgkBGRVKtWrUJeXh4GDx4Mi8WCkSNHYtGiReLjDQ0NOHTokFjtAIBXXnlFbFtXV4eMjAy8/vrr4uNWqxUbN27Eww8/jLS0NHTo0AHjx4/H008/Lbbp3r07Nm3ahGnTpuHVV1/F5ZdfjrfeestlvsmoUaNw5swZzJkzB3a7Hf369UNRUZE4sbddu3bYvHkzZs6cieHDh+PcuXPo0aMHVq5ciWHDhomvs3PnThQUFODcuXNITk7Gn//8Z5dl1wDwzjvvYNq0acjMzITFYsHtt9+OoqIitGvXzudjadp9YNoidZ8YqXvE+EJKyPF1bxlA2v4ywUCPsOJk9NACqBdcdJ+M25JJwgurK23TdR+Yyx5CqEXhPjCOOnx8olDT/pJyAVeBcdKjEtMW58nKn5N/A5GegcXJDMEFUCe8+D24AAwvRNSmgA0wgDFCDOB68pISZnypygR6mPFHWHEy0i65bWHVRX8ML0T+FdABBjBOiHGSEmakVGUA/57sjUztVUFt0fNK0AETXACGFyKSJOADDGC8EOOkZZgJZnoHFielwUWPSbctGSK4AAwvRCRZUAQYwLghxolhRh5/hRUnM4YWJ0OEFxMFF4DhxQyE8xcgWDxf1Nen13Co/x1P6guaAAOoG2Lk8iX8MMy45++w0hyDiwoYXohIgaAKMIB6IUau5icuKWFG6komck/P+Skt+TO0AAYKLgDDCxEpFnQBBvB/iHGSEmakrmSii/wZWJwYXJoxWXABGF6IjCooAwxgnBDjxDCjnBHCSnP+Di4Aw4tSDC9ExhW0AQYwXohxYphpm9HCipMRQgugYnAxYehQC8MLkbEFdYABjBtinJwnRKmTf0lfSoOLoSolQFAHF4DhhcgMgj7AAPJCjNraCkVSJ/9Sa0apjjgZLrQAQR9cAIYXIrNggPkfqSFGbc1PZgwzyhktrDRnyOACBH14YXAhMhcGmGb8HWKcGGakMXJYaY7BRR0MGkQEMMC0YpQQ4yQ3zJAxGDa0AAwuRGRqFn93wIg8flH6+Qu/qc5q7BMiiQz/s2J4ISKTYwXGA6NVYpqTUpUhg1dB9Gay4AIwvJA0Qs15CCENyl5DUPZ80gcDjBdGDjFODDOtmS6wmDBU6IHBhYi8YYBpg9cQo7c2QpPpTtzBjKHFK4YXImoLA4wPPIYYvTU/6RmkAkQSMbi0ieGFiHzBAOMjw4QYJ+eJkEFGGgYInzFIEJGRMcBIYLgQA7Aq0xYGFskYXIjIDBhgJDJkiHHiyZoUYnghIrNggJHB0CGGTInBgYhIGgYYmYx0wmGYMi8j/R4REZmJrJ14lyxZgqSkJISHhyM1NRU7d+702n7dunVITk5GeHg4evfujc2bN8vqLLlnqQsx3M2I/H1MzHKciIjMQHKAWbt2LfLz81FQUIDdu3ejb9++yMjIwOnTp9223759O+6//37k5OTgyy+/xIgRIzBixAjs27dPcefJuPwdDBgWiIgCW4ggCJLGH1JTU3HTTTdh8eLFAACHw4HExEQ88sgjmDlzZqv2o0aNQk1NDTZu3CjeN3DgQPTr1w+FhYVu36Ourg51dXXif1dVVeGKK65A4h9nwxIeLqW7RETkZ47aWhwveBaVlZWIiorS5D2qq6sRFRWF22wjEIp2il6rEQ3YVr8eVVVViIyMVKmHpDZJc2Dq6+tRVlaGWbNmifdZLBakp6ejtLTU7XNKS0uRn5/vcl9GRgbWr1/v8X3mz5+PP/7xj63uP17wrJTuEhGRgfz3v//VLMDYbDbExcVhm329Kq8XFxcHm82mymuRNiQFmLNnz6KpqQmxsbEu98fGxuLgwYNun2O32922t9vtHt9n1qxZLqGnsrIS3bp1w7FjxzT75Q8E1dXVSExMxPHjx/lXgxc8Tm3jMfINj5NvnFX0Tp06afYe4eHhOHr0KOrr61V5PZvNhnBW/A3NkKuQwsLCEBYW1ur+qKgofkn4IDIyksfJBzxObeMx8g2Pk28sFlnrRnwWHh7O0BFEJP02denSBVarFeXl5S73l5eXIy4uzu1z4uLiJLUnIiIiaoukAGOz2ZCSkoKSkhLxPofDgZKSEqSlpbl9Tlpamkt7ACguLvbYnoiIiKgtkoeQ8vPzMX78ePTv3x8DBgzAwoULUVNTg+zsbADAuHHjcNlll2H+/PkAgClTpuD222/HSy+9hMzMTKxZswZffPEF3nzzTZ/fMywsDAUFBW6HlehnPE6+4XFqG4+Rb3icfMPjRFqQvIwaABYvXowXXngBdrsd/fr1w6JFi5CamgoA+MUvfoGkpCSsWLFCbL9u3TrMnj0b3333Ha6++mosWLAAw4YNU+1DEBERUXCRFWCIiIiI/EnbKeFEREREGmCAISIiItNhgCEiIiLTYYAhIiIi0zFMgFmyZAmSkpIQHh6O1NRU7Ny502v7devWITk5GeHh4ejduzc2b96sU0/9S8pxWrp0KW699VZceumluPTSS5Gent7mcQ0EUn+XnNasWYOQkBCMGDFC2w4ahNTjVFlZidzcXMTHxyMsLAzXXHNNUPy7k3qcFi5ciGuvvRYRERFITEzEtGnTUFtbq1Nv/WPbtm0YPnw4EhISEBIS4vVad05bt27FjTfeiLCwMPTo0cNl5SqRTwQDWLNmjWCz2YTly5cL+/fvFyZNmiRER0cL5eXlbtt//vnngtVqFRYsWCB8/fXXwuzZs4V27doJe/fu1bnn+pJ6nMaMGSMsWbJE+PLLL4UDBw4IEyZMEKKiooQffvhB557rR+oxcjp69Khw2WWXCbfeeqtw991369NZP5J6nOrq6oT+/fsLw4YNEz777DPh6NGjwtatW4U9e/bo3HN9ST1Oq1atEsLCwoRVq1YJR48eFT766CMhPj5emDZtms4919fmzZuFJ598Unj//fcFAMIHH3zgtf2RI0eE9u3bC/n5+cLXX38tvPbaa4LVahWKior06TAFBEMEmAEDBgi5ubnifzc1NQkJCQnC/Pnz3ba/7777hMzMTJf7UlNThQcffFDTfvqb1OPUUmNjo3DJJZcIK1eu1KqLfifnGDU2NgqDBg0S3nrrLWH8+PFBEWCkHqc33nhDuPLKK4X6+nq9umgIUo9Tbm6u8Mtf/tLlvvz8fOHmm2/WtJ9G4kuAmT59unDddde53Ddq1CghIyNDw55RoPH7EFJ9fT3KysqQnp4u3mexWJCeno7S0lK3zyktLXVpDwAZGRke2wcCOceppfPnz6OhoUHTK8L6k9xj9PTTTyMmJgY5OTl6dNPv5BynDRs2IC0tDbm5uYiNjcX111+PefPmoampSa9u607OcRo0aBDKysrEYaYjR45g8+bN3LizhWD8Dif1+f1q1GfPnkVTUxNiY2Nd7o+NjcXBgwfdPsdut7ttb7fbNeunv8k5Ti3NmDEDCQkJrb44AoWcY/TZZ59h2bJl2LNnjw49NAY5x+nIkSPYsmULsrKysHnzZhw+fBi///3v0dDQgIKCAj26rTs5x2nMmDE4e/YsbrnlFgiCgMbGRjz00EN44okn9OiyaXj6Dq+ursaFCxcQERHhp56Rmfi9AkP6eO6557BmzRp88MEHvNz8//z0008YO3Ysli5dii5duvi7O4bmcDgQExODN998EykpKRg1ahSefPJJFBYW+rtrhrJ161bMmzcPr7/+Onbv3o33338fmzZtwjPPPOPvrhEFHL9XYLp06QKr1Yry8nKX+8vLyxEXF+f2OXFxcZLaBwI5x8npxRdfxHPPPYePP/4Yffr00bKbfiX1GP3nP//Bd999h+HDh4v3ORwOAEBoaCgOHTqEq666SttO+4Gc36X4+Hi0a9cOVqtVvK9nz56w2+2or6+HzWbTtM/+IOc4PfXUUxg7diwmTpwIAOjduzdqamowefJkPPnkk7BY+Dcj4Pk7PDIyktUX8pnf/zXZbDakpKSgpKREvM/hcKCkpARpaWlun5OWlubSHgCKi4s9tg8Eco4TACxYsADPPPMMioqK0L9/fz266jdSj1FycjL27t2LPXv2iLdf//rXuOOOO7Bnzx4kJibq2X3dyPlduvnmm3H48GEx4AHAN998g/j4+IAML4C843T+/PlWIcUZ+gRedk4UjN/hpAF/zyIWhItLFcPCwoQVK1YIX3/9tTB58mQhOjpasNvtgiAIwtixY4WZM2eK7T///HMhNDRUePHFF4UDBw4IBQUFQbOMWspxeu655wSbzSa89957wqlTp8TbTz/95K+PoDmpx6ilYFmFJPU4HTt2TLjkkkuEvLw84dChQ8LGjRuFmJgY4dlnn/XXR9CF1ONUUFAgXHLJJcJf//pX4ciRI8I///lP4aqrrhLuu+8+f30EXfz000/Cl19+KXz55ZcCAOHll18WvvzyS+H7778XBEEQZs6cKYwdO1Zs71xG/fjjjwsHDhwQlixZwmXUJJkhAowgCMJrr70mXHHFFYLNZhMGDBgg/Pvf/xYfu/3224Xx48e7tH/33XeFa665RrDZbMJ1110nbNq0Sece+4eU49StWzcBQKtbQUGB/h3XkdTfpeaCJcAIgvTjtH37diE1NVUICwsTrrzySuFPf/qT0NjYqHOv9SflODU0NAhz584VrrrqKiE8PFxITEwUfv/73ws//vij/h3X0SeffOL2u8Z5bMaPHy/cfvvtrZ7Tr18/wWazCVdeeaXw9ttv695vMrcQQWBdk4iIiMzF73NgiIiIiKRigCEiIiLTYYAhIiIi02GAISIiItNhgCEiIiLTYYAhIiIi02GAISIiItNhgCEiIiLTYYAhIiIi02GAISIiItNhgCEiIiLT+X/LAQsoZVTSvQAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "reset_manager()\n", + "\n", + "start_manager()\n", + "u, J = forward(m)\n", + "stop_manager()\n", + "\n", + "print(f\"{J.value=}\")\n", + "plot_output(u, title=\"u\")\n", + "\n", + "\n", + "dJdm = compute_gradient(J, m)\n", + "plot_output(dJdm.riesz_representation(\"L2\"), title=r\"$g^\\sharp$\")" + ] + }, + { + "cell_type": "markdown", + "id": "16946cc4-64e1-438a-9f42-6a170decc5dc", + "metadata": {}, + "source": [ + "## Hessian action\n", + "\n", + "Next we seek to differentiate $\\hat{J}$ twice with respect to $m$. However the second derivative defines a *bilinear* operator. This can be represented as a matrix – a Hessian matrix – but the number of elements in this matrix is equal to the *square* of the number of degrees of freedom for $m$.\n", + "\n", + "Instead of computing the full second derivative we can compute its action on a given direction $\\zeta \\in V$. The degrees of freedom associated with the result define the action of the Hessian matrix on a vector – specifically the action of the Hessian matrix on the vector consisting of the degrees of freedom for $\\zeta$.\n", + "\n", + "We do this in two stages. First we compute a directional derivative,\n", + "\n", + "$$\\left. \\frac{d \\hat{J} \\left( m + \\alpha \\zeta \\right)}{d \\alpha} \\right|_{\\alpha=0},$$\n", + "\n", + "computed using the tangent-linear method. We consider the case where $\\zeta = 1$." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "8e641946-4aeb-49e0-9415-6b86a822613e", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:46:44.834695Z", + "iopub.status.busy": "2023-12-20T16:46:44.834151Z", + "iopub.status.idle": "2023-12-20T16:46:46.210382Z", + "shell.execute_reply": "2023-12-20T16:46:46.209633Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J.value=9.711060188691798e-06\n", + "dJdm_zeta.value=1.1580693564843862e-06\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGzCAYAAACPa3XZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABz/0lEQVR4nO2de1xUdf7/X8wAA6JAKhdx8doFLRPDxDH7uiZF6Vpu9g0v621NaxMraUstEy9tZmvmVhRbadamafY1t8wfZZi5JWki7JYp5XrB0kHNBQRlgJnz+8OdkYG5nMvnnPM5M+/n4zGP8sznnPOZw5nzec77/bmECYIggCAIgiAIQiNMeleAIAiCIIjQguSDIAiCIAhNIfkgCIIgCEJTSD4IgiAIgtAUkg+CIAiCIDSF5IMgCIIgCE0h+SAIgiAIQlNIPgiCIAiC0BSSD4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgiAIgiA0heSDIHRi6tSp6NGjR5vtixYtQlhYmPYVIgiC0AiSD4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgtAJX51KHQ6HxjUhCILQFpIPgtCJK664AtXV1W22Hz9+XPvKEARBaAjJB0HoRO/evVFTU4N//etf7m2nTp3CBx98oGOtCIIg1CdMEARB70oQRCjyyy+/oHv37khKSsJDDz2ECxcu4NVXX0VCQgL2798P+moSBBGsUOSDIHSiU6dO+OCDD9CuXTs8/vjjeOutt7Bs2TKMHj1a76oRBEGoCkU+CIIgCILQFIp8EARBEAShKSQfBEEQBEFoCskHQRAEQRCaIlk+du3ahdGjRyMlJQVhYWHYsmVLwH127tyJG264ARaLBVdeeSXWrl0ro6oEQRAEQQQDkuWjvr4e/fv3R0FBgajyR48exahRozB8+HCUl5fjkUcewX333YdPPvlEcmUJgiAIgjA+ika7hIWF4YMPPsCYMWN8lpk7dy4+/vhjfPfdd+5t48aNQ3V1NYqKiuSemiAIgiAIgxKu9glKSkqQlZXlsS07OxuPPPKIz33sdjvsdrv7306nE+fOnUOnTp18rodBEARB8IkgCDh//jxSUlJgMqnX1bChoQGNjY1MjhUZGYmoqCgmxyLaorp82Gw2JCUleWxLSkpCbW0tLl68iOjo6Db7LFu2DIsXL1a7agRBEISGnDhxAr/61a9UOXZDQwN6dI9B1Wknk+MlJyfj6NGjJCAqobp8yGH+/PnIy8tz/7umpgbdunVD6uIFMNGNYDickTSPndExNVLEkZCPs6EBJ/KfRocOHVQ7R2NjI6pOO1GxLwUdOiiLrpw/78Q1A0+isbGR5EMlVJeP5ORkVFVVeWyrqqpCbGys16gHAFgsFlgsljbbTVFRXMqH00KNqz9oPHcQ4P2rqjsmO0mRkdAibd6hgwmxCuWDUB/V5cNqtWLbtm0e27Zv3w6r1ar2qTWBxIMg9IO37x/JEEGIQ7Ie1tXVoby8HOXl5QAuDaUtLy9HZWUlgEspk8mTJ7vLP/DAAzhy5Agef/xxHDp0CK+88gree+89zJkzh80n0BHeHnwEQeiL0yLQc4EgRCA58rFv3z4MHz7c/W9X34wpU6Zg7dq1OHXqlFtEAKBnz574+OOPMWfOHPzlL3/Br371K7zxxhvIzs5WXHn6khuUKIfeNSCk0GDWuwaGw/VsokgIQXhHsnz8+te/hr+pQbzNXvrrX/8aZWVlUk9FsIQafEIuvN07BpIhvX8gkfwQvMLlaBeCEbw1GgTBgtb3tYFkRGucFoEEhOASko9gg4SDCDVa3vMkIm0gASF4hORDTUgERGG20HXiGYfdQA26Xt85zqWHBITgDZIPtSDxaANJhjHx9XczlJSojQGiLyQgBE+QfKiBAcSDRIBQip73ENfi4/r+cyghJCAEL5B8sIZj8SDhIIKFlvcytyLCqYQEu4CcdTTC7lA4vbqDzfowhG9IPljCoXgYUTgsUWxWpSSUY2+I1LsKAWl9j3MnIxxKSLALCME/JB+s4Ew8jCIdJBp84+/vw6uYcCsjnEkICQihJyQfLJAhHkaRA1aQZAQfvv6mvEkJdykajiSEBITQC5IPJciMdvAoHiQHBCu0upfkSI6a3z3JYsOJhJCAEHoQevKhc3qEJ/EwknDEWIxT12Ci3s5XFKMlLe9fHqItru+2ESWEBITQmtCTDx3hQTx4Fw6SDL7w9ffgTUpa39d6yohiCWGFRJkhASG0hORDI/QWD96kgyTD2Pj7+/EgJjxERWRLCCuiHCQgBLeQfGiAXuLBSjhIFPzT3mJnerw6u4Xp8bSGt2iJ3lERXTu8koAQnELyoTJai0eoCAfrBp8nWH42nkSG9T0lV2b0lBFdoiEkIASHkHyohFzp0DM9wqNwBLNkaIFa148HqWl9v7KSETlIFRjNJYQEhOAMkg8VMJJ48CIcJBnGwtffS08pYSUjcnB9d7mWEBIQgiNIPhhjFPHQUzqCTTTiLQ2qHLfaHqXKcdXE399WazFpeY9rJSLcS0gICMhZpwUNTmVru9Q5aW0XtSH5YAjv4sFKOHiUB7UEQE9Yfya9ZUZPMdE6KsK1hISAgBD8Q/LBCF7FIxiFIxhFQwtYXTc1JEbu/SVXWrSKinArISQghM6QfDCAR/FgIR16CwdJBp/4+7toHV1pfY/KkREtoiJcSggJCKEjJB8KkSMecqVDi34aeghHMEhGJ0u9qsf/xR6j6vFZobeYtLx/WURFALYywp2EkIAQOkHyIROtox1qi4cW0sGTZKgtC6xhWV+9RMbX318tKWERFQHEf/ekSApXEkICQugAyUcAWE4Sxpt4sBIOvaXCaCKhN0qvF2t5kXr/yJUVFlERf7i+p3IkBJAmIswlhASE0BiSD43gRTyMLBwkGXzg7++gRVSl9b0nR0ZYRUW8IUdCAHnREH8/jmQtbEcCQmiEssHQhCh4EI/2Frti8Yi3NLhfatLJUu/1xTsdI+o1efGMHn87Fvel6/vB4nviIsbSKOs7bIlqZNIZXVbUVsbKuk6LIP08QUhBQQF69OiBqKgoZGZmYu/evX7Lb9q0CWlpaYiKikK/fv2wbds2j/c3b96M2267DZ06dUJYWBjKy8t9HksQBNxxxx0ICwvDli1b3NvXrl2LsLAwr6/Tp08DAL788kvcdNNN6NSpE6Kjo5GWloYXXnhB9nUQC0U+VEZP8WDxEFVTNPQWCt4bcl+wrPe5Jm36f2gVLWERFQHYpmi0jIS0xmxxUAREAzZu3Ii8vDwUFhYiMzMTq1atQnZ2NioqKpCYmNim/O7duzF+/HgsW7YMv/nNb7B+/XqMGTMG+/fvx3XXXQcAqK+vx9ChQ3HvvfdixowZfs+/atUqhIW1vf45OTm4/fbbPbZNnToVDQ0N7nrFxMQgNzcX119/PWJiYvDll1/i/vvvR0xMDGbOnCn3kgQkTBAE7rW1trYWcXFx6L78aZiiLj9MZBm3RLNX0udDjngolQ4e0ypaS4ZRpcIoaCUsaqRwWHRuVSojckfPKJEQWX1DJAoIAJ8C4mxowPG5C1BTU4PY2FjpdRGBq53Y8V0q2ndQOMPpeSduue4ETpw44VFfi8UCi6Xt3z8zMxM33ngjXn75ZQCA0+lEamoqZs+ejXnz5rUpn5OTg/r6emzdutW9bfDgwUhPT0dhYaFH2WPHjqFnz54oKytDenp6m2OVl5fjN7/5Dfbt24cuXbrggw8+wJgxY7x+rjNnzqBr165YvXo1Jk2a5PPz33333YiJicHf/vY3n2WUQpEPFdA62sGbcGglGyQZ+uDrurOWEjWiJS3vcb2iInpEQkIpAvKLIwYXHco64l5wXPrRmZqa6rE9Pz8fixYt8tjW2NiI0tJSzJ8/373NZDIhKysLJSUlXo9fUlKCvLw8j23Z2dkeKRNR9bxwARMmTEBBQQGSk5MDln/77bfRrl073HPPPT7LlJWVYffu3Xj66acl1UUqoSUfMvKZUjGaeBhFOIwoGp0j6lQ57tmm9qocVylaSQngeb8ZVUS0lhCtBCSY8Bb5aM3Zs2fhcDiQlJTksT0pKQmHDh3yelybzea1vM1mk1S/OXPmYMiQIbjrrrtElV+9ejUmTJiA6OjoNu/96le/wpkzZ9Dc3IxFixbhvvvuk1QXqYSWfEhEasrFSOLBu3TwJBtqSYRcpNSHB1Fp+bdUS0SUpmhc3wclaZn2Fruuq/oS7ImNjVUtTaSUDz/8EDt27EBZWZmo8iUlJTh48KDPVMo//vEP1NXV4euvv8a8efNw5ZVXYvz48Syr7IFh5UNyfw+V+3poKR7BKB1aygZvMqEmUj+r2rLS+u/MSkZc9yMLCdF6ivgYS6OsfiCWqEZF/UAINnTu3BlmsxlVVVUe26uqqnymQpKTkyWV98aOHTvw73//G/Hx8R7bx44di5tvvhk7d+702P7GG28gPT0dGRkZXo/Xs2dPAEC/fv1QVVWFRYsWqSofhhxqy9vQLrkdS7UUDxZDZFkNndR6+GjniDqPF+Gb1tdK7evF+h5gcX8q+a7I/X7KjX5qtSI24ZvIyEhkZGSguLjYvc3pdKK4uBhWq9XrPlar1aM8AGzfvt1neW/MmzcP//rXv1BeXu5+AcALL7yAN99806NsXV0d3nvvPUyfPl3UsZ1OJ+x2dWe9NmzkQxIqRj20HNEi9cHGW5RDq+gGCQZbWl5PNSMjLKMieqZi5KZf5EZACP3Jy8vDlClTMHDgQAwaNAirVq1CfX09pk2bBgCYPHkyunbtimXLlgEAHn74YQwbNgzPP/88Ro0ahQ0bNmDfvn147bXX3Mc8d+4cKisrcfLkSQBARUUFgEtRk5av1nTr1s0dxXCxceNGNDc343e/+12b8gUFBejWrRvS0tIAALt27cKKFSvw0EMPMbgyvgl++dCgk6kUjCQeLKRDC+Eg2dAOrUQEUN5XRM9UjJb9Pyj9oj85OTk4c+YMFi5cCJvNhvT0dBQVFbk7lVZWVsJkupxoGDJkCNavX48FCxbgiSeewFVXXYUtW7a45/gALvXpcMkLAIwbNw6A9xE3gVi9ejXuvvvuNika4FKUY/78+Th69CjCw8PRu3dvLF++HPfff7+kc0jFkPN8SEq7cNLXQ8tOpSzSK0pRUzqMKBudw8/L2u9scwfGNVEPrTq3yo2IsJg3RKqEyBEQNecB0WK+D29DbbWc52PTP9PQroPCobbnHfjf/odUrW+oE9yRDxIP0fCaWuFRNOSKhJrn4kFStE7PSJUQXkbFBELNDqiyhtsShAoEt3xwAO/iwZt06CUbWgqFGkitv9qy0vrvqIaMdIyolyUggHapGBp+SxDeCV754CDqofVoFinwlFrRUjiMLhms0DqiolZURI6AAGwkRGwURI6AUPSDCHaCVz4MiBbiwYt0aCEcJBrKaXkN1RARQLmMyE3DAOxSMUYTEILQm+CUDwNGPYwgHrxLB8mGuriuL+uUDauoiN5REJ4EJJSpao5DdJOypu1iczOj2hC+CE75MBha9O9QIh68SodRZSM5ojpgGVtTvOr1kIsa0RD3sRVGRZRGQQBla8VoPTuqLwJFPyj1QuhNyMuHnlEPnvt38Nifg0fZECMSahyXFzlRKxriPr7MqIjcKAigLBUTSEAo+kEQlwhp+ZAqHizhVTx4kg49ZUMtqWCF2PppJSlqRkPc55AoIkoFBJAXBeFFQKjvB8EzIS0fUmG1jgKP4sFLakVL4eBdMFgg5TOyEhWtRESsgADyJyZjNTS3NTwMwaXUC6EnISsfvE0o5g+15+9QKh5GkI5QEA2l+LpGSqREi/4hakdBAOmpGDU6oFL0gwgmQlY+9EKLNVq0Eg+epYNkgx0tryUrEQHYDt3lMQrCi4D4g6IfhF6EpHwYJerBq3jwKB1Glo0E8+XrecahzfoocnFdZxYpGpZREbECArCJggDiJISnETAEwRMhJx9G6WTKm3iwGrXCQjp4FY2WEqHWMXiRE1bREBcsoiJapmEA8akY1h1QpUY/KPVC8EjIyYdUtI56aLEwnBTxIOm4DAu5ULsOesgJaxEBlEVFtErDAOw6pOqZfgmYeolySF7ZliACQfLhB1ajW8QSjOKhVDq0FA4e5EIpYj6DmoLCMi3jQk5UhLcoiN79Pyj6QfAGyYdGyB1eGwgexYP3KEcwSIYSpHx+uaKiRjTERefw86KjIVpHQbQWEFZQx1NCa0g+fCAl6sEi5cLT5GFyxYNH6Qh10VAKi86wakZDxEZBpHRGBZRJiD9Yd0DVbPZTA6VefmnugKhmZU1bA63tojokHxrAOurBm3jwJB0kG+rhurY8RUPERkGkpGEAZYvUBer/oVcHVEq9EDxB8uEFraMeUuBJPHiRDh6FI8HEVjjPOPWdDbMlvEVD1EjDAPKjIHoICAuo4ymhJSQfHCAm5aL24nBSxIMH6eBBOFgLhpJz6SUnLKMhgHwZkZqGAaRFQQBpEqJkcToXUgSEFp8jjAbJRytYj3BhkXIJFvEwYpRDS8FQgr96aiEmrCZKU5qaUSsKAkhPxWjdAVWMgARKvVDHU0IrSD4UoEdHUzHwJh48S4dR5EIJYj4jS0FRGg1xITcqolZnVIB9h1Q9BEQRlHohGEHyoSJ6RD14Eg/epCMUREMuakROWE8bL7WfiFqdUQHxEsKi/wdrqOMpwQMkHzLhMerBi3jwIh0kG2xoeR15EBEpEqJmFAQQl4rRugOq0ugHdTwltIDkI0jgQTx4kA6ehSPBLL1BOOPQdjRVIFzXV0mahmVahpcoCAsBCQRX6Rc/OC0CTPYwXc5NGAeSDxmIiXpomXIJBvEIFumQIxhKjqeXnPASDZEaBdGzMypPM6BSx1NCb0g+dIJFykXNlWm1EA8jplZYy4VSeJATliKiREL0TsPwJiCqRj8o9UIoxCRnp4KCAvTo0QNRUVHIzMzE3r17/ZZftWoVrrnmGkRHRyM1NRVz5sxBQwP7UR5awFvUQyw8iUeCuY5JpEMt8UgwR/p8GQ2tP4vSv4vr3pBzf0i5F8Xe450j6iR9d6T+IPCGGiPgvKHmwplOi6DasYngQHLkY+PGjcjLy0NhYSEyMzOxatUqZGdno6KiAomJiW3Kr1+/HvPmzcOaNWswZMgQ/PDDD5g6dSrCwsKwcuVKJh/CaGj1cHHBi3jwFOkwokiwRsw1kBs50SsawkNn1EAREKPMgGrUjqe/NMXA0hSh6Bj2piZGtSF8ITnysXLlSsyYMQPTpk1D3759UVhYiHbt2mHNmjVey+/evRs33XQTJkyYgB49euC2227D+PHjA0ZLjIpaq9e2RuwvLB7Eg4dIh9EjGHrB4nrpEQ3ROwoS6PspJrIZ6EeKmGeN1ss/tISiH4Q/JMlHY2MjSktLkZWVdfkAJhOysrJQUlLidZ8hQ4agtLTULRtHjhzBtm3bMHLkSJ/nsdvtqK2t9XiFEqxSLryIhxJYpFdINpTDg4Rcqoe+AgLIX/W5NSy+50p/7KiZeiEIf0hKu5w9exYOhwNJSUke25OSknDo0CGv+0yYMAFnz57F0KFDIQgCmpub8cADD+CJJ57weZ5ly5Zh8eLFUqpGEARBEIRBkNXhVAo7d+7EM888g1deeQX79+/H5s2b8fHHH2Pp0qU+95k/fz5qamrcrxMnTjCrDw0fIwiCIAh9kRT56Ny5M8xmM6qqqjy2V1VVITk52es+Tz31FCZNmoT77rsPANCvXz/U19dj5syZePLJJ2EytfUfi8UCi4WfJcQJguCHBHMdk+naCYLQD0mRj8jISGRkZKC4uNi9zel0ori4GFar1es+Fy5caCMYZvOl6IMgUIckgiAIggg1JA+1zcvLw5QpUzBw4EAMGjQIq1atQn19PaZNmwYAmDx5Mrp27Yply5YBAEaPHo2VK1diwIAByMzMxOHDh/HUU09h9OjRbgkhCIIgCCJ0kNznIycnBytWrMDChQuRnp6O8vJyFBUVuTuhVlZW4tSpU+7yCxYswKOPPooFCxagb9++mD59OrKzs/HXv/6V3acgvCJ1kSyCIAhCHlIn39y0aRPS0tIQFRWFfv36Ydu2bR7vb968Gbfddhs6deqEsLAwlJeXtzlGQ0MDZs2ahU6dOqF9+/YYO3Zsm24RlZWVGDVqFNq1a4fExEQ89thjaG5u9jjPrbfeioSEBMTGxsJqteKTTz6RfyFEIqvDaW5uLo4fPw673Y49e/YgMzPT/d7OnTuxdu1a97/Dw8ORn5+Pw4cP4+LFi6isrERBQQHi4+OV1j1oUboAFUEQBKEdrsk38/PzsX//fvTv3x/Z2dk4ffq01/K7d+/G+PHjMX36dJSVlWHMmDEYM2YMvvvuO3eZ+vp6DB06FMuXL/d53jlz5uCjjz7Cpk2b8MUXX+DkyZO4++673e87HA6MGjUKjY2N2L17N9566y2sXbsWCxcudJfZtWsXbr31Vmzbtg2lpaUYPnw4Ro8ejbKyMgZXxjdhggE6XtTW1iIuLg7dlz8NU1RU4Mlrohx+3zZbfL8faNx7oEl7xIy7FzPDaaA5AFhPMsbrPB+A8llNaZ4PdrBYK0bJiriX6yEuqid21VtA3Eyn7rIiooqB1nkBAv/QCLTOC4CAM50GWt/F3wJzAUcHBpjh1LW6rbOhAcfnLkBNTQ1iY2P9H1MmrnbioS/vgqW9whlO65rw4tC/i65vZmYmbrzxRrz88ssALvWFTE1NxezZszFv3rw25XNyclBfX4+tW7e6tw0ePBjp6ekoLCz0KHvs2DH07NkTZWVlSE9Pd2+vqalBQkIC1q9fj3vuuQcAcOjQIfTp0wclJSUYPHgw/t//+3/4zW9+g5MnT7qzE4WFhZg7dy7OnDmDyEjvf/trr70WOTk5HpLCGtWH2hIEQeiF0tWWidCm9WSXdnvbH0JyJt8sKSnxKA8A2dnZPst7o7S0FE1NTR7HSUtLQ7du3dzHKSkpQb9+/Tzm5srOzkZtbS0OHDjg9bhOpxPnz59Hx44dRddFDrSqLUEQokkwR2qyUi5ByOU/Te0Q2aQs2tnYdOkeT01N9dien5+PRYsWeWyTM/mmzWbzWt5ms4muo81mQ2RkZJsuDC2P4+s8rve8sWLFCtTV1eHee+8VXRc5kHwQBEEQhBdOnDjhkXYJ9vmn1q9fj8WLF+Pvf/+714ViWUJpF8aovdKkWkjJdxOEElitTEwQahMbG+vx8iYfcibfTE5OllTe1zEaGxtRXV3t8zi+zuN6ryUbNmzAfffdh/fee69NSkgNSD4IgiAIQiZyJt+0Wq0e5QFg+/btPst7IyMjAxERER7HqaioQGVlpfs4VqsV3377rceom+3btyM2NhZ9+/Z1b3v33Xcxbdo0vPvuuxg1apToOiiB0i4EQRCcEm9pEDXihdAXqZNvPvzwwxg2bBief/55jBo1Chs2bMC+ffvw2muvuY957tw5VFZW4uTJkwAuiQVwKWKRnJyMuLg4TJ8+HXl5eejYsSNiY2Mxe/ZsWK1WDB48GABw2223oW/fvpg0aRKee+452Gw2LFiwALNmzXJHcdavX48pU6bgL3/5CzIzM919QaKjoxEXF6faNaPIBxH0UAdJQgxShpuLQexweML4SJ18c8iQIVi/fj1ee+019O/fH++//z62bNmC6667zl3mww8/xIABA9yRiHHjxmHAgAEeQ3FfeOEF/OY3v8HYsWPxP//zP0hOTsbmzZvd75vNZmzduhVmsxlWqxW/+93vMHnyZCxZssRd5rXXXkNzczNmzZqFLl26uF8PP/ywatcLoHk+vOJvro9A83wAgef64HGeD0D8w9do83xcqgfN9cEKHub6kLKwnBpzfYidPTjQXB9iJhQMFPmgeT4u4WonJn0+HpHtFY52qWvE34a/q2p9Qx2KfBAEQRAEoSkkH4RkpPySJAiCIIjWkHwQBEEQXgmUoiYIuZB8GBQx60bwgJTcPEEQBBEakHwQBEFIREpHboIg2kLzfBAEQRBBw7nGGEQEGOETiKZGZaviEoGhyAdBEARBEJpC8kEQBEEQhKaQfBAEQQQxYiZGJAitIfkgCIIgCEJTSD50gBaKIgiCIEIZkg+CIAiCIDSF5EMigRZpIgiCIAjCPyQfBEEQBEFoCslHkCN26W+CILSnk6Ve7yoQhC6QfBDcc8Zp0bsKBMEt7S12vatAEJIh+SAIwnAkmGltFYIwMrS2C0EQBBE01NgtCA9XFi1ttocxqg3hC4p8EARBEAShKSQfBEEQBEFoCskHQRAEQRCaQvKhAnV25aMzfrHHMKiJNM42dxBd1tYUL7rsGUd7nHEoG/KrdMTLGQctrsUCFteRxeglpfcTL7D4nrN43hCE1hiyw6nJHganRfBdoMEMRDl8vu2wm2G2+H5fC6rtUYi3NPgt84s9xu88AOeaYtAxIvA8Aa65PjpHBB4h4BKQzuHnA5a1NcUjOaI6YDkXrgZD7kiFlo1Wgkn68EJvDWeCmWasFYNRpUOKJEuRb5o/hyCUYUj5UBt7QyQsUb4ftvX2yIDLVNfZLQHH32spIMClB6YYAQEuPYjFCggAWRICKBcRORLiWRfff8dQFRPWUSK9Ih16i8e5JuVRDVqEkghWDCsfSqMfWsBKQAIhVUAA8VEQMQICSI+CuNA7GuL32EEqJlqloPRMr+gtHmLQKuVC61ERPGJY+VBKoNQLi+gHwEZAAkU/AGkCAoiPgkhNwwDSoiAueIqGiDoXp2LCQ98Wvft08CAeFPUgCP+ErHywgKWABEItAQH4iYK44DkaIur8AQRAiZzwIBfeYDUFvtKOpEYRD4p6EKGOoeXDSB1PAwkIi/4fwOUHHw9RECUCAhgvGiIWXgVCDnpHOVpiFPEgCMLg8qE2gVIvgPjohxhYCQggXULU7IwKyEvFtMTo0ZBggpcohwsp0gEYQzxYpFwo6kHwTPDLhwYdT7Xs/wGIFxBA3c6ogLgoCMBORFhGQ3xBcuIJ61WFWUiHVOFwwYN40Nwe6lLfaIFZ4douQRSc5Jbgl48AKO146kJrAZECD1GQlrRuOOTKiNJoiM/j+mlsg1VMWAuG13PoKB0AH+IhBop6EKFAyMuHHmg1AqY1UiRErc6o3lAySgZgEw0RfS6DRk20kAuf59ZZOgB+xIOiHgRxidCQD4UdT1lHPwDtRsB4Q42JyaSmYbzBIjWjVjRE9Pl1iJroKRb+4EE6gOASD4IIFkJDPjSEpYCw7v/REl6jIC6UioiW0RCxBBITXiVCCnp1IvWGFOkA+BAPrVMu9gZKzxD6EDrywcGMp97QU0AAfqMgLWGVluFFQrxhVPFgucAbC+EApEsHwId4iIFSLkSwEDryEQBWqReA7fBbQBsBAcRHQdTsjOoPltEQFzwLCY+osZqsntIB8CMeFPUgQgmSD5Vg3f9DbQEB1F0lF2AXCQGUR0NciG1MQ1FS1F62Xo/USpv9OR5O2xqKehDBhEnvCmhKg9nv2w67//el/lKQ8itEzINFzC8jpQ89KQ9YqYtsnW3u4PFiga0p3v1SkzOO9n5fRkSvz8Ti78XiHjKSeLCGoh7sKSgoQI8ePRAVFYXMzEzs3bvXb/lNmzYhLS0NUVFR6NevH7Zt2+bxviAIWLhwIbp06YLo6GhkZWXhxx9/9CjTo0cPhIWFebyeffZZ9/sVFRUYPnw4kpKSEBUVhV69emHBggVoamqSVBc1CC354ByWAqLkAXiuKUb0w/ZsU3vZK32qJSJ6wKOY8CZMvEgHwJ94aJlyIfFgz8aNG5GXl4f8/Hzs378f/fv3R3Z2Nk6fPu21/O7duzF+/HhMnz4dZWVlGDNmDMaMGYPvvvvOXea5557Diy++iMLCQuzZswcxMTHIzs5GQ4NnBHzJkiU4deqU+zV79mz3exEREZg8eTI+/fRTVFRUYNWqVXj99deRn58vqS5qECYIgp/FUfigtrYWcXFx6L78aZiiPL+kftd28UWAjqf++n6I7ffREqn9PwKlYKROQKYkFSNljRixfUECHodhekZpWkZtAqVzjBpVAdj15QCUp1cA6ZE6QH3xECsdLBeRkyoffiPCAaLJJnsYAMDZ0IDjcxegpqYGsbGxks4vFlc7MeD9PJjbKZzh9IIdZfesFF3fzMxM3HjjjXj55ZcBAE6nE6mpqZg9ezbmzZvXpnxOTg7q6+uxdetW97bBgwcjPT0dhYWFEAQBKSkpePTRR/HHP/4RAFBTU4OkpCSsXbsW48aNA3Ap8vHII4/gkUceEf3Z8vLy8M033+Af//iHqLqoBUU+JCLnV4PU2QYDPWik/krSMgrCApbpGT2jIWLgLTqhhJYpMJadSFmkV4wsHmLQRTxCgNraWo+X3d72h2FjYyNKS0uRlZXl3mYymZCVlYWSkhKvxy0pKfEoDwDZ2dnu8kePHoXNZvMoExcXh8zMzDbHfPbZZ9GpUycMGDAAf/7zn9Hc3Ozz8xw+fBhFRUUYNmyY6LqoBXU41Qg9RsC0xPVwVHtIrpTOqGJh0WmV5SJ3BNuoRmtYpeGUyDBP4kEdTaVRb4+E2awsteSwX4qop6amemzPz8/HokWLPLadPXsWDocDSUlJHtuTkpJw6NAhr8e32Wxey9tsNvf7rm2+ygDAQw89hBtuuAEdO3bE7t27MX/+fJw6dQorV6702G/IkCHYv38/7HY7Zs6ciSVLloiui1qQfHiB5bBbuai1BowSCVFrSK4UWIuIN0hO2qJF9IgH6QD4Eg+WUNRDOidOnPBIu1gsfIlgXl6e+/+vv/56REZG4v7778eyZcs86rpx40acP38e//znP/HYY49hxYoVePzxx/WoshuSDw2RGv1QcxE6LSYma9kIqCkiLPuIAOIb2mCUFD1SVHr15/CGmuKhlnSosYgcicclYmNjA/b56Ny5M8xmM6qqqjy2V1VVITk52es+ycnJfsu7/ltVVYUuXbp4lElPT/dZl8zMTDQ3N+PYsWO45ppr3NtdEZy+ffvC4XBg5syZePTRR2E2mwPWRS2oz4cPWA+7dcG6/wcg/6GmZFSM1MmWXHl3Vo2Ex7EZj5oRS+s+Dmr0eWANL3Vm9TdjdU9J6dsEaCcerFIuNMJFPSIjI5GRkYHi4mL3NqfTieLiYlitVq/7WK1Wj/IAsH37dnf5nj17Ijk52aNMbW0t9uzZ4/OYAFBeXg6TyYTExESfZZxOJ5qamuB0OkXVRS0o8qEDakRAlCA3FSMlDdMSNSMiakVD5OCvMZcbNeFVagLBWgz1iHS40EI8xEoHRT34IC8vD1OmTMHAgQMxaNAgrFq1CvX19Zg2bRoAYPLkyejatSuWLVsGAHj44YcxbNgwPP/88xg1ahQ2bNiAffv24bXXXgMAhIWF4ZFHHsHTTz+Nq666Cj179sRTTz2FlJQUjBkzBsCljqJ79uzB8OHD0aFDB5SUlGDOnDn43e9+hyuuuAIAsG7dOkRERKBfv36wWCzYt28f5s+fj5ycHERERIiqi1qQfChAi74fLlisARMILVbJbU3rRoSVjKg1syorjCoRYlErCsUqyiEXnsRDLBT1UJ+cnBycOXMGCxcuhM1mQ3p6OoqKitwdOSsrK2EyXU40DBkyBOvXr8eCBQvwxBNP4KqrrsKWLVtw3XXXucs8/vjjqK+vx8yZM1FdXY2hQ4eiqKgIUf+dbsJisWDDhg1YtGgR7HY7evbsiTlz5nj0AwkPD8fy5cvxww8/QBAEdO/eHbm5uZgzZ46kuqgBzfMRAH8dT13IFRDW83+4UCohgPxRMXIlxBusoyI8SkgwoHa6S88oR0u0WKdFqniIiXxIkQ/JUQ8O5/m4et08JvN8/DDxWVXrG+pQ5IMBciMgaqVfXA8+JRKiJBXDSkBYp2e8NZIkJNLRqm8NL9IBhIZ4SCaAeBCEP0g+AhBo2K1S1Oz/oVcqRm5fEH9okZ7xR6hKitadeAHjSgegf8fSltDQWoJnSD4YwVP/j5YESxSkNWoP421zviCWFD0Ew+P8DEdAsRAOQLv5O+RKhxodTSVBUQ9CISQfDNEq/QJIHwGjl4S0bAyCRUT8IaYhZykoeouDHNQYbq2ndAD8iQdFPQjeIfkQgZTUC88CAug/KsaFFiIC6C8j3jCiMMhFDdFoid7SAWgnHrpHO1xQ1INgAMlHCKJnKsZF60ZDbRnhUUKCEbVlw4Ue/Tm8EQziEWxRj8aGCJhMyq6Xs8HJqDaEL2TNcFpQUIAePXogKioKmZmZ2Lt3r9/y1dXVmDVrFrp06QKLxYKrr74a27Ztk1VhvZDyBdVq9lNAWUe1anuU4imflcyS2hLXDJOsfsm2Rs0ZVkOZltdViygHi3uE1T0r9btTZ7doIh56Dq0lCLFIjnxs3LgReXl5KCwsRGZmJlatWoXs7GxUVFR4ndK1sbERt956KxITE/H++++ja9euOH78OOLj41nUn1t4T7+0RM9UjDfUTs8EaiQpSuIdPcSNpYyyEA6Av/4dvOKa44MgvCFZPlauXIkZM2a4p40tLCzExx9/jDVr1mDevHltyq9Zswbnzp3D7t273dO59ujRQ1mtDYLRBATQNxXjDa3SMy0R28gGq6ToGR1SI/KlV3rFhZZpFop6EEZBknw0NjaitLQU8+fPd28zmUzIyspCSUmJ130+/PBDWK1WzJo1C3//+9+RkJCACRMmYO7cuTCbvd/MdrsddvvlhrS2tlZKNUMW10MuGCXEhRadVsUippHWWlCMllZSK82mZ5SjJbyKh2QkigdFPYhASJKPs2fPwuFwuOerd5GUlIRDhw553efIkSPYsWMHJk6ciG3btuHw4cN48MEH0dTUhPz8fK/7LFu2DIsXLxZVJ5M9TPoU6w1mSVOsK8H1QJAaAXE9gKRGQAD+JMQFaxnx1nDpLSStMZoMqIVaktEaXqQDCCLxIAgVUH20i9PpRGJiIl577TWYzWZkZGTg559/xp///Gef8jF//nyPxXFqa2uRmprq8xxaCYiS2U71lBAXcmSk9UOYpYy4UDNN44I3KQlWtJKMlvAkHID2/TtIPAgjIkk+OnfuDLPZjKqqKo/tVVVVSE5O9rpPly5dEBER4ZFi6dOnD2w2GxobGxEZ2faLY7FYYLFI+wIbQUAAfSTERcuHotyoCEsZcaGnlPiDhMU3RpaMlrASDhdaigdJB2FkJMlHZGQkMjIyUFxcjDFjxgC4FNkoLi5Gbm6u131uuukmrF+/Hk6n072k8A8//IAuXbp4FQ8laCkgrZEqJHpKCMAmKgJ4PrxZiEhLtJASfyhtYHmTFz2EQQ5qSEZr9EqreIPEgwhFJKdd8vLyMGXKFAwcOBCDBg3CqlWrUF9f7x79MnnyZHTt2hXLli0DAPzhD3/Ayy+/jIcffhizZ8/Gjz/+iGeeeQYPPfQQ20/yX/TqA+ISEq0lBFAuIgB/KRp/+GuctBITMRilsdcLLSSjJXqnVbxB4kGEKpLlIycnB2fOnMHChQths9mQnp6OoqIidyfUyspKd4QDAFJTU/HJJ59gzpw5uP7669G1a1c8/PDDmDt3LrtP0Qo9O6G2jIpIEZGWDxS9oiEtUSNF4wuWkqJ3tIRoi9aS0RIehcMFiQcRysjqcJqbm+szzbJz584226xWK77++ms5p5IND6Ng9IyG+EKvjqv+8NVAaCElYiF5CYyektEantIq3iDxIEKdoF7bhQcBAbSXEH+wSNeoLSMutJASsQSLvPAkCFJg3THUH2pKB0DioTaORjMEH3NIicXZSBOqqU1QywegQECAoJYQoO1DkIWMqCUiLeFJSsRi1EZfa7SUjJaoLRwutBSPYFswjggugl4+AJkCAqg2GRlvEuKChYxoFRXxhr+Gi2cxCUX0koyW6DlaRSwkHkSwEhLyAfAnIIByCQHUExFAnRSNFFiKC6vGjiRGGjxIRmtYSIcWC77JEQ+SDsIohIx8AHwKCCB/hAygfjTEBasUjRR8NRJaRlNao7Qx5UVeeJQCNTFClKMlJB5EsBNS8gFcXvCIh46o3mARDZGCXGlhPc+IFHiUErGEWqPPAq36Y/hC62XtSTyIUCDk5MMFLyNhfCFXQqTS+kEnR0b0iIp4w8hSQugvGS3RWjhckHgQoULIygfAv4AA2kmICzVkxB96pm+UQEIjH54kozV6SQdA4kGEFiEtHwBDAWmQ+BCQuZ6MVhLiQu3OrXIe9npFVlqitAHlRV54FgGt0FM4XJB4EKFGyMsHoHAuELnInEtELwkB2ERFWOCrseBBSsRCjb6+sBIOvSb/IvEgjA7Jx3+RPRJGKQolBNBHRADthvyKJRikhPCEh6hEa/SebZTEgwgGSD5aoJuAAJ6RFANFQ1zwEhXxhpoNGImNcngUDG/oLR0AiQcRPJB8tEJXAXHBIBrCCrlCo9aDmiepAdg0nDwIjNoCEGNpNIxktIQH4XChm3goTTFrTYMZCGOUFidUg+TDC1wICKDaGjNSaP3A0zO6AvhvDHgTE7EYsVGWipE+I0/C4YKZeFCjSnACyYcPuBEQgAsJccGbjLTEV6NhVCkh2iJFDKT+3XmUDkCmeJBkEJxD8uEHrgQE4EpCXPDQ8TUQoZICCiZY/M20lAmu+mKQeBAGgOQjANwJCMClhAB8R0XUINikhsXnCZZogxi4Eg4XJB6EQSD5EIHs9WDURsEIGS3Q8uEcTKJj5AbZyHUXA5fC4YLEgzAQJB8S4DIK4kLLB4/BRCeYxITQB66lAyDxIAwHyYdEuBYQrWj9oONQRlrCsuEgkdEP7gVAL0g8CANi0rsCRsSVhmF5vEAvrmkwX34FOQ67mYsXD59R6/MTXgiB75xRKCgoQI8ePRAVFYXMzEzs3bvXb/lNmzYhLS0NUVFR6NevH7Zt2+bxviAIWLhwIbp06YLo6GhkZWXhxx9/9Chz7tw5TJw4EbGxsYiPj8f06dNRV1fnfv/YsWMICwtr8/r666/dZZqamrBkyRL07t0bUVFR6N+/P4qKihhcEf+QfMhEjDCIfUk5H/e0FBF6MKoGD2Kj9/lDHvp+ccPGjRuRl5eH/Px87N+/H/3790d2djZOnz7ttfzu3bsxfvx4TJ8+HWVlZRgzZgzGjBmD7777zl3mueeew4svvojCwkLs2bMHMTExyM7ORkNDg7vMxIkTceDAAWzfvh1bt27Frl27MHPmzDbn++yzz3Dq1Cn3KyMjw/3eggUL8Ne//hUvvfQSvv/+ezzwwAP47W9/i7KyMoZXqC1hgiBwn0Oora1FXFwcui9/GqaoKL2rwwUhn/oRC+cpISIIIAloQ+sfSs6GBhyfuwA1NTWIjY1V5ZyudiL1haUwRStrJ5wXG3BizlOi65uZmYkbb7wRL7/88qX9nU6kpqZi9uzZmDdvXpvyOTk5qK+vx9atW93bBg8ejPT0dBQWFkIQBKSkpODRRx/FH//4RwBATU0NkpKSsHbtWowbNw4HDx5E37598c0332DgwIEAgKKiIowcORI//fQTUlJScOzYMfTs2RNlZWVIT0/3WveUlBQ8+eSTmDVrlnvb2LFjER0djXfeeUf0NZMKRT4MimFSMnrTOhIj90UQLaF7IySora31eNnt9jZlGhsbUVpaiqysLPc2k8mErKwslJSUeD1uSUmJR3kAyM7Odpc/evQobDabR5m4uDhkZma6y5SUlCA+Pt4tHgCQlZUFk8mEPXv2eBz7zjvvRGJiIoYOHYoPP/zQ4z273Y6oVj/qo6Oj8eWXX/q8LiygDqdBALdDgYMJIzYySqM+PA3lNuL1DxF4+wFkagyDyaSwTo2X9k9NTfXYnJ+fj0WLFnlsO3v2LBwOB5KSkjy2JyUl4dChQ14Pb7PZvJa32Wzu913b/JVJTEz0eD88PBwdO3Z0l2nfvj2ef/553HTTTTCZTPi///s/jBkzBlu2bMGdd94J4JL0rFy5Ev/zP/+D3r17o7i4GJs3b4bDoe53nuQjiODlIUASxAksG2xq/EMGXp4jPHDixAmPtIvFYtGxNtLp3Lkz8vLy3P++8cYbcfLkSfz5z392y8df/vIXzJgxA2lpaQgLC0Pv3r0xbdo0rFmzRtW6UdqFYI5anW4JgpAGfReVERsb6/HyJh+dO3eG2WxGVVWVx/aqqiokJyd7PW5ycrLf8q7/BirTukNrc3Mzzp075/O8wKX+KYcPH3b/OyEhAVu2bEF9fT2OHz+OQ4cOoX379ujVq5fPY7CA5IPgApajh7QchUQQPEL3sHZERkYiIyMDxcXF7m1OpxPFxcWwWq1e97FarR7lAWD79u3u8j179kRycrJHmdraWuzZs8ddxmq1orq6GqWlpe4yO3bsgNPpRGZmps/6lpeXo0uXLm22R0VFoWvXrmhubsb//d//4a677hLx6eVDaReC8EEoPLwpRaYfoXB/hQp5eXmYMmUKBg4ciEGDBmHVqlWor6/HtGnTAACTJ09G165dsWzZMgDAww8/jGHDhuH555/HqFGjsGHDBuzbtw+vvfYaACAsLAyPPPIInn76aVx11VXo2bMnnnrqKaSkpGDMmDEAgD59+uD222/HjBkzUFhYiKamJuTm5mLcuHFISUkBALz11luIjIzEgAEDAACbN2/GmjVr8MYbb7jrvmfPHvz8889IT0/Hzz//jEWLFsHpdOLxxx9X9ZqRfBBECEMNIEEoJycnB2fOnMHChQths9mQnp6OoqIid4fRyspKmEyXEw1DhgzB+vXrsWDBAjzxxBO46qqrsGXLFlx33XXuMo8//jjq6+sxc+ZMVFdXY+jQoSgqKvIYmbJu3Trk5uZixIgRMJlMGDt2LF588UWPui1duhTHjx9HeHg40tLSsHHjRtxzzz3u9xsaGrBgwQIcOXIE7du3x8iRI/G3v/0N8fHxKl2tS9A8HwRBEISqaDnPB4t2Qov6hjoU+eAA00XqesMrzmin3lUgCIIIOkg+FELiENyEyt+XJIsgCC0h+ZBBqDRIROjA4z1NQkQQwUvIywePD12CIAJ/N0lOCMK4BJ18kEzwi7ntsghBg8NYEx8GBTx910mE+MF00QSToPDeaODn3gpWDC8fPD2AeCSYG3yeMOJ1JmFih+s5RBJCEOIwvHwQbTFiQ0hoj9z7hKTFNyQhBCEOkg8VIQkgghG972sjyA9JCEH4h+RDJfR+QBsFc4M253F4mXNIq3P7qwMhnZbfLd5FRKyEsE4fk/QQvEPyoQIkHm3RuqHn7fy81CHYBKj1d41XGdG6bxpFXgjeIflgDO/iwUMDSOiH0r8/7/JiFBnRCtNFEwkIwSUkHwzhVTxIOAhW+LuXeBQTLb+TvIoOCQjBIyQfjOBRPIwqHeEK690ssRFUej5WSK03b2h5v/EsOjxKCAkIwRskHwzgSTyMJBxqNfq8yIRU1Kq30aXGGy3vc95EhFcJIQEheILkQyFyxINXQTBqo034h+XflUeRaf194kVGeJQQEhCCF0g+AsA6qsGbeJBwEFLwdb/wJCW8RUV4kxASEIIHQk4+9EyR8CQeRpMOX9eOVePC09+GhwZTKv7uJz3FhKeoCE/zkwSzgJjsgDlM4UE4SqUHKyEnH3rBQ+NmBOGQep14uK6sYfGZeBIYnvqyqHW/SL3ePERDgllACP4h+dAAPRtIXoUjGKWBJ3hpZNWk5b2td9rHdb2NJiEkIIRekHyojF6NLAvpIEEgWqN2+ksure93vWTEiBJCAkLoAcmHishpvHmIVISydIQ3CHpXwU1zlNLEtXbwJiV6R0W0khBffdikSgwJCKE1JB8qYTTx4Fk4eBICLVH6uXmQFx6mc9czKqJUQmSf1y5PQABaD4bQBpIPFTCKePAmHKEqGWrB+nrqITNqTOeuh4zoMfxXjoAAFAUhtIHkgzFGEA+9pYMkw5j4+7vxICYsZERLEdFCQkhACF4h+WCE3AZdK/FgJRyhIg68fU4eUij+8HW9tKw3CxlR8n2UKi5aSQgJCMEjJB8M4FU8glk4eKyTmhi1/4eeUqJ1qsP1feZRQuSOpiEBIdSC5EMhPIoHC+ngoXHnoQ7BAotryVIYlNRHTj20nOmUdwkhASF4gORDAVqKh1b9NPRo8EkyjAEPqZXW9ZB7bi1khFcJIQEheIDkww9qNPg8iodWjb8RJSP8Al8P3OZ2Jr2r0AY9O6K2PjcLGWHd6CuVEIB9nYJZQMx2wKz0ILS2i+qQfGgIT+LBSgR4EQreJEEtlH5OreVF62gJCxlRKyqiZFSNGtGQYBYQgn9IPjSCF/FgIQt6CkeoSIZaKLl+LMVFKylRI0XjDalSwEtKhgSE0AuSD5WR27GUpXgYMcpBksEfvv4mWkhJm3PKEAlWKRpvyJUCHiSEBITQA1lPjYKCAvTo0QNRUVHIzMzE3r17Re23YcMGhIWFYcyYMXJOazj0FI/wBsH94uE4Po9/wen1xRPhF53cvXhCj78hi/uy5TFY3d/mBvkTDcqNjrJ4XsiZzt01HTtByEFy5GPjxo3Iy8tDYWEhMjMzsWrVKmRnZ6OiogKJiYk+9zt27Bj++Mc/4uabb1ZUYaOgpXiwFgM1REMPoeCtkWaJ0s/WHK1+w+Hvb65mtERuRINFisaFXpGQlshZT4bWgyG0QvITYOXKlZgxYwamTZuGvn37orCwEO3atcOaNWt87uNwODBx4kQsXrwYvXr1CngOu92O2tpaj5eRMKp4MImU6PEr2ADRAd7QO9qi5n3CIqLBKiqidSSk9bkl7yNzlAdFQQipSLpjGhsbUVpaiqysrMsHMJmQlZWFkpISn/stWbIEiYmJmD59uqjzLFu2DHFxce5XamqqlGrqitHEg1l6hiQj6DGqlLBO0chBqYRo+VwhASG0QNLdcvbsWTgcDiQlJXlsT0pKgs1m87rPl19+idWrV+P1118XfZ758+ejpqbG/Tpx4oSUavpE6TLVgTCSeLCOcrDGKJJhvtis+Ys3tPg7sZIRvUVESR8NLRegVPtZGcqcO3cOEydORGxsLOLj4zF9+nTU1dX53aehoQGzZs1Cp06d0L59e4wdOxZVVVUeZSorKzFq1Ci0a9cOiYmJeOyxx9Dc7P158dVXXyE8PBzp6eke23ft2oXRo0cjJSUFYWFh2LJli8f7TU1NmDt3Lvr164eYmBikpKRg8uTJOHnypOTroKqqnj9/HpMmTcLrr7+Ozp07i97PYrEgNjbW4+UNnkxby85iLELJSlBDOPQQDaOKAK/1cqGVjCg+ho7Dzo0gIIQ6TJw4EQcOHMD27duxdetW7Nq1CzNnzvS7z5w5c/DRRx9h06ZN+OKLL3Dy5Encfffd7vcdDgdGjRqFxsZG7N69G2+99RbWrl2LhQsXtjlWdXU1Jk+ejBEjRrR5r76+Hv3790dBQYHXely4cAH79+/HU089hf3792Pz5s2oqKjAnXfeKfEqSOxw2rlzZ5jN5jbGVVVVheTk5Dbl//3vf+PYsWMYPXq0e5vTeemhER4ejoqKCvTu3VtypeUg1eSlPBy0nMNDzsOO2VBbVjl5HaIYejfIWiP28zqi1R9t3/LvzbKjq+t+VNJ51fXdUNLBNLxB4H7VYYIPDh48iKKiInzzzTcYOHAgAOCll17CyJEjsWLFCqSkpLTZp6amBqtXr8b69etxyy23AADefPNN9OnTB19//TUGDx6MTz/9FN9//z0+++wzJCUlIT09HUuXLsXcuXOxaNEiREZGuo/3wAMPYMKECTCbzW0iG3fccQfuuOMOn/WPi4vD9u3bPba9/PLLGDRoECorK9GtWzfR10LStzYyMhIZGRkoLi52b3M6nSguLobVam1TPi0tDd9++y3Ky8vdrzvvvBPDhw9HeXm5or4cakY91J7OXG60Q6pEsBpCyCTvrnNUg/CO1hEUNe4Dlv1ClOwvFYp+8E/rgQ92u7J8VElJCeLj493iAQBZWVkwmUzYs2eP131KS0vR1NTk0dcyLS0N3bp1c/e1LCkpQb9+/Ty6RGRnZ6O2thYHDhxwb3vzzTdx5MgR5OfnK/ocLampqUFYWBji4+Ml7Sf5Z09eXh6mTJmCgQMHYtCgQVi1ahXq6+sxbdo0AMDkyZPRtWtXLFu2DFFRUbjuuus89ndVsPV2NVEzfyn1QaBVmoVFaoUFWsgGyYV6uK6tGhGS1veG0qhI+AWn4iG8SqIYcvY1N8ibKCy8QfxwXLnnMCrhDYBZ4W+usP+2Ga1/IOfn52PRokWyj2uz2dpMSREeHo6OHTv67Ddps9kQGRnZpnFv2dfSZrN57Yvpeg8AfvzxR8ybNw//+Mc/EB7O5vvc0NCAuXPnYvz48T67R/hCcg1ycnJw5swZLFy4EDabDenp6SgqKnJ/0MrKSphM/PTFkIpaUQ+jpFmMIB0kG9rT8pqrlaphkZ7ROxWjZQpGioAQ8jhx4oRHo2qxeJ8IZd68eVi+fLnfYx08eJBp3aTgcDgwYcIELF68GFdffTWTYzY1NeHee++FIAh49dVXJe8v6ymSm5uL3Nxcr+/t3LnT775r166Vc0rZ8BD1MIJ48CwdRpMN84UmZsdytItgdixWaC0igHQZ0VtCpKBFZCLUoh+s8DfgoSWPPvoopk6d6rdMr169kJycjNOnT3tsb25uxrlz57z2mwSA5ORkNDY2orq62iP60bKvZXJycpuZxl19M5OTk3H+/Hns27cPZWVl7rbb6XRCEASEh4fj008/dfcnEYNLPI4fP44dO3ZIjnoAtLaLB2pEPXgXDyajBhgKh96iwVIcWCCmPnoKihYiAly+x+RIiNapGF7TL4R6JCQkICEhIWA5q9WK6upqlJaWIiMjAwCwY8cOOJ1OZGZmet0nIyMDERERKC4uxtixYwEAFRUVqKysdPe1tFqt+NOf/oTTp0+70zrbt29HbGws+vbti4iICHz77bcex33llVewY8cOvP/+++jZs6foz+oSjx9//BGff/45OnXqJHrflgS1fOgd9dB6GK3ospxFObQWDt4EQyliP4/aktL676hmPxEpEqJHFIRHAaHoh/706dMHt99+O2bMmIHCwkI0NTUhNzcX48aNc490+fnnnzFixAi8/fbbGDRoEOLi4jB9+nTk5eWhY8eOiI2NxezZs2G1WjF48GAAwG233Ya+ffti0qRJeO6552Cz2bBgwQLMmjXLnSpq3c8yMTGxTb/Muro6HD582P3vo0ePory8HB07dkS3bt3Q1NSEe+65B/v378fWrVvhcDjcfUo6duzoMaomEEErH2oOrVULo4gHC+nQUjiCTTbkorWkqBkVCb/o5D4VQ/0/CG+sW7cOubm5GDFiBEwmE8aOHYsXX3zR/X5TUxMqKipw4cIF97YXXnjBXdZutyM7OxuvvPKK+32z2YytW7fiD3/4A6xWK2JiYjBlyhQsWbJEUt327duH4cOHu/+dl5cHAJgyZQrWrl2Ln3/+GR9++CEAtJmg7PPPP8evf/1r0ecKEwRBu3XSZVJbW4u4uDh0X/40TFGXvmGBhtrqPa+HVJnRIs3CYrisUtSWDhIN9qgRMWEpI3I6p7JY2E6sWEgVELnRCTHyIeXYUheZA3wvMOdsaMDxuQtQU1Mjq3+AGFztRN8Hn4HZoszEHPYGfP/KE6rWN9QJ2siHFPSOevAsHrynVkg21Md1jVlKCMuoCO9REKkREDXTL5R6IXiB5EMiLCf40SLNAsgTDx6lw6iiYaqX3vnIGSPjZ6fKtLz+vImIHAEB2EkI6/QKdUAlgp2Qlw+9RrjwKh48SQdPsiFHINQ8n95yooWIANJkRO6IGED5qJhAAsJT/w+KfhA8ENLyIb1fBpvz8jiMlqf+HFpLh9ZiwQKe5ESNtIz72DKiInpFQVgLCEkCEcyEtHyogRqRFN7FwyjSYUTJkIvYz8pSUtSKhriPL2Gqd6VREIBNp9Q2x+ao/wdB6EnIyodRoh48iwfvqZVQkg25qBVBUTsaonYUBJCXihEjFzxEQII5qmK2K1/bBY1MqkL4IWTlwwjwKh68SgfJBntc11SphLhgPYcIj1EQNQSEIIKNkJQPtaIeLFMuvIkHb6kVQ4tG/UX/78dEa1MPCbS83kpSNaxTM1pGQQDxEsJaQOREKqjjKcEzISkfUtB6aK0WC8NJEQ+epIM74QgkEWoclwMxURoNccFKRKRGQeQKCCAtFcM6uqG1LJjt8iYaIwgxkHwwgkXUIxjFw9DSoZZcKIEjMWEVDQHYpGfERkGUpGEAaVEQvUfAUMdTgldIPvzAw4Rifo+p4hweSsTDEKkVHsVCKTqmc1hFQ1zIjYpoHQUBlI+K0bP/RyCZoegHoRYkHwaFR/HgOsoRjLIhFbHXQIGksIyGuJAjIlpFQYDAqRge+n8QBG+QfPhAStRD65QLb+LBnXSQaCij9fWTKSOsoyGAtCG8WnVGFXV8HQVEacdTin4QakDyoQEsUy48iQc30mFw2RD+u3R2WLt2OtfEB67rq1BCALbRELECAmgzJDdQ+oWG4BLEZUg+vMBj1EPtxeGkiAdJh29cIsF6Xy7EpOX15iAaIlZAAG2iIKwERCwsox8EoTUkHyrDIuoRTOJhdOlQIhdqnVcXMVEoIqznDeElCsJCQPTo/0GpF0JrSD6CEB7FwyjSoZdcKEF3MeEgLcNTFETpCrkAdUAlgh+SDwXoObeHz+NxJB7MOpEylg4jCoZcAn1WpnLCMC0DSBcRNTujAtKiIFqPgBFVpxDpeBreIMDsVPZcDWtkPzUC4QnJh4ponXLhRTx4inKEkmjIQTU5URgNAdreR2JlRI3OqIB0CdFSQCj6QRgNkg+Z8Bb14EE8eJEOEg52KB6JwyAa4kJKZ1W1oiAAm7lB3MfSWEBovReCF0g+VELLqEdQiEcQS4dQVy9737D2MQxrIp+W11bPaAhw6V7TOwoCiOsPwvMIGFnHD5B6MV00wRktb7VsIrQg+ZABy9VrlWJ48QgC6VAiF0qOrZeY8BANkRoFUaMzKqCdgND8H0SwQfIhEVbiwSQyYmTxMJB0qCkXSghUL7XlhIdoiNgoiFpDcgF2AhLwPIzSL2p3PKXoByEGkg8V0GtuD19wIx6cdiLlVS6UomXURM9oCA9REBYCQjOgEqEEyYcOaBn14EI8OIpyBKtoSEUtMdEzGqJWFMSIAqJ29IMglELywRieoh7BIB4spIOEQxqtr5dcGdEjGqJ3Z1QWC9QZIQJCqRdCKeot40gQHK69QkhHqbwxiVpJuJekyDGLdYqkwjKlShBGheSDIAiCIAhNIfkgCIIgCEJTqM8HY5qjwpiv12JYYqIp9UIQhKaY7QLCFa7tgiZ6hqsNRT44ROl8AARBEATBM9TKEQShOnrPQssCKaPHCILwD8kHQRBEEMLTMhAE0RqSD4IgCIIgNIXkgyAIgjOo0zoR7JB8EARBEAShKSQfBEEQBEFoCsmHRGixJYIgCEIu586dw8SJExEbG4v4+HhMnz4ddXV1fvdpaGjArFmz0KlTJ7Rv3x5jx45FVVWVR5nKykqMGjUK7dq1Q2JiIh577DE0N3uO0Fq3bh369++Pdu3aoUuXLvj973+PX375xaPMqlWrcM011yA6OhqpqamYM2cOGhq8915+9tlnERYWhkceeUTydSD5IAiCIAiNmDhxIg4cOIDt27dj69at2LVrF2bOnOl3nzlz5uCjjz7Cpk2b8MUXX+DkyZO4++673e87HA6MGjUKjY2N2L17N9566y2sXbsWCxcudJf56quvMHnyZEyfPh0HDhzApk2bsHfvXsyYMcNdZv369Zg3bx7y8/Nx8OBBrF69Ghs3bsQTTzzRpk7ffPMN/vrXv+L666+XdR1IPgiCIAhCAw4ePIiioiK88cYbyMzMxNChQ/HSSy9hw4YNOHnypNd9ampqsHr1aqxcuRK33HILMjIy8Oabb2L37t34+uuvAQCffvopvv/+e7zzzjtIT0/HHXfcgaVLl6KgoACNjY0AgJKSEvTo0QMPPfQQevbsiaFDh+L+++/H3r173efavXs3brrpJkyYMAE9evTAbbfdhvHjx3uUAYC6ujpMnDgRr7/+Oq644gpZ14LkgyAIgiC8UFtb6/Gy28WvmOyNkpISxMfHY+DAge5tWVlZMJlM2LNnj9d9SktL0dTUhKysLPe2tLQ0dOvWDSUlJe7j9uvXD0lJSe4y2dnZqK2txYEDBwAAVqsVJ06cwLZt2yAIAqqqqvD+++9j5MiR7n2GDBmC0tJSt2wcOXIE27Zt8ygDALNmzcKoUaM86iQVWtsliHFEh9OsjAShMeEXnWiOpt91ehF+wYnwCKeygzRd2j81NdVjc35+PhYtWiT7sDabDYmJiR7bwsPD0bFjR9hsNp/7REZGIj4+3mN7UlKSex+bzeYhHq73Xe8BwE033YR169YhJycHDQ0NaG5uxujRo1FQUODeZ8KECTh79iyGDh0KQRDQ3NyMBx54wCPtsmHDBuzfvx/ffPONvIvwX0LyG0KdRgnCgNAihYTGnDhxAjU1Ne7X/PnzvZabN28ewsLC/L4OHTqkce09+f777/Hwww9j4cKFKC0tRVFREY4dO4YHHnjAXWbnzp145pln8Morr2D//v3YvHkzPv74YyxduhTApevx8MMPY926dYiKUtaQUuSDIAiCILwQGxuL2NjYgOUeffRRTJ061W+ZXr16ITk5GadPn/bY3tzcjHPnziE5OdnrfsnJyWhsbER1dbVH9KOqqsq9T3Jycpt+Ga7RMK4yy5Ytw0033YTHHnsMAHD99dcjJiYGN998M55++ml06dIFTz31FCZNmoT77rsPANCvXz/U19dj5syZePLJJ1FaWorTp0/jhhtucJ/H4XBg165dePnll2G322E2mwNcrUuQfBAEEXSY6u1wxlj0rgYRIiQkJCAhISFgOavViurqapSWliIjIwMAsGPHDjidTmRmZnrdJyMjAxERESguLsbYsWMBABUVFaisrITVanUf909/+hNOnz7tTuts374dsbGx6Nu3LwDgwoULCA/3bPJdoiAIgruMyWTyWWbEiBH49ttvPd6fNm0a0tLSMHfuXNHiAZB8EARBEIQm9OnTB7fffjtmzJiBwsJCNDU1ITc3F+PGjUNKSgoA4Oeff8aIESPw9ttvY9CgQYiLi8P06dORl5eHjh07IjY2FrNnz4bVasXgwYMBALfddhv69u2LSZMm4bnnnoPNZsOCBQswa9YsWCyXJHz06NGYMWMGXn31VWRnZ+PUqVN45JFHMGjQIPe5R48ejZUrV2LAgAHIzMzE4cOH8dRTT2H06NEwm83o0KEDrrvuOo/PFBMTg06dOrXZHgiSD4IgiBAkvAFopv5vmrNu3Trk5uZixIgRMJlMGDt2LF588UX3+01NTaioqMCFCxfc21544QV3WbvdjuzsbLzyyivu981mM7Zu3Yo//OEPsFqtiImJwZQpU7BkyRJ3malTp+L8+fN4+eWX8eijjyI+Ph633HILli9f7i6zYMEChIWFYcGCBfj555+RkJCA0aNH409/+hPz6xAmuOItHFNbW4u4uDh0X/40TP/t5GK66L+vrDnAiCh/y02HB1iKOtBS1WIWhQpUJvxC4N7a4RcDl5Ey2sV8oUlUOVO9hOFmCjsJCi2+gLKPUVev+BgEENY+Rtn+7dopr0RMtOiiYtMujnYRoo/piBb3ey3QaJfmdoH7+jdHhSl6Hwjcud6ffATa1xHg8jqjLz+fnA0NOD53AWpqakT1oZCDq50YPHIJwiOUWVVzUwO+3rZQ1fqGOiE52oUgCIIgCP0g+SAIIqQRG/HjDTERVoLgFZIPgmuYhOoJgiAIriD5IIIepX0VCIIgCLaQfBAEQRAEoSk01JaQjDPGIm3EC0EQhEaENzgR3qxwbRel+xMBocgHQRCawGLYNEEQwQHJB0EQhMaImceHIIIZkg+CIAiRSJm0jyAI35B8EARBEAShKSQfBEEQBEFoiiz5KCgoQI8ePRAVFYXMzEzs3bvXZ9nXX38dN998M6644gpcccUVyMrK8lueCDIkrMVBEAQ/BFrDiiCUIFk+Nm7ciLy8POTn52P//v3o378/srOzcfr0aa/ld+7cifHjx+Pzzz9HSUkJUlNTcdttt+Hnn39WXHmCIAiCIIyHZPlYuXIlZsyYgWnTpqFv374oLCxEu3btsGbNGq/l161bhwcffBDp6elIS0vDG2+8AafTieLiYp/nsNvtqK2t9XgRBEEQBBEcSJKPxsZGlJaWIisr6/IBTCZkZWWhpKRE1DEuXLiApqYmdOzY0WeZZcuWIS4uzv1KTU2VUk3dEbPUdcBjiFhyWwxilwAHpC0tLna5cgCKUy9h7dopXuOFplhXhtLrx+JvqFYKT8p9L+X7RBCEbyS1cGfPnoXD4UBSUpLH9qSkJNhsNlHHmDt3LlJSUjwEpjXz589HTU2N+3XixIk2ZZzR6o2Tb47y/74jwPuXjuFfQMQISiABaY4W9+dzRIeLfmiqKiA6S0hY+xj3iwgMi+vFTDok3jti7029xIPFjwsWP3IIQi801fhnn30WGzZswM6dOxEV5bsFt1gssFgkNGxecFgAs58ZwB1R+neoao4KC7gsdnM7k98JiZqjTQi/KE7EHNHhouYpcD2QxSw1Lnmq9ZaNSP1F8fu1oGVjJnfWTDENqlBXL+vYRkFNCWOyGrFMWdVbPMT+KPB7DBILIsiRJB+dO3eG2WxGVVWVx/aqqiokJyf73XfFihV49tln8dlnn+H666+XXlONaY4CwhXKiSi5EFEm4HlUEBDg0sNZrIAAkL7eC0MRUWPqbn+Ns1HEROsoj57SAegvHmLQKuohJkIbjJgvNsMcrmwyOKGZJpNTG0nfgsjISGRkZHh0FnV1HrVarT73e+6557B06VIUFRVh4MCB8mvbikCpF4ey4In/Y4v8YotKrwRK0Yh4WEn5tSU1DSP2QS0pDdMahWkZV3ifSeMn5nwtUhLeXlrCQz30Sq+0hAfxoKgHQYhDstLn5eVhypQpGDhwIAYNGoRVq1ahvr4e06ZNAwBMnjwZXbt2xbJlywAAy5cvx8KFC7F+/Xr06NHD3Tekffv2aN++PcOPIh2tUi9MohsB0i+AtAgIwFkUxAUnaRmlBGr4pUROeO6jonekw4Xe4iFWOlh1JA9EqEY9COMgWT5ycnJw5swZLFy4EDabDenp6SgqKnJ3Qq2srITJdPkL9uqrr6KxsRH33HOPx3Hy8/OxaNEiZbXHpeiH6aI6X+hAqRcp8hJIQFj0/wDkCQggbs0KsQICyOgL4g1XoyRTQgB10zJK4FkoxMDDyBWpkTa9xYMVFBkhggFZyczc3Fzk5uZ6fW/nzp0e/z527JicUzBD7Y6nwSAggHqdUQEFURAXQRINMSpMU1k6SAdgHPFgIRYU9SCMAA1aDwCLjqfSzqc8RQNcfijqnYYBGEoIwDQa4o1QFxPV+szoJB0AP+JBw2sJ4jIhIR+Boh+Kj88w+iGmjJjoh7usRAmRmoYBxEVBgLaNhyIZYRAN8UagxtfocqJVh1w3OkoHwI94iDomiQURQoSEfASCRcdTngUE4KMzamtaNio8iog3jCAnmguGN3SWDoAv8aCOpgThCcmHCMSmXlgLSMBjyBAQQL0oiBwBccFcRFSWEF9oJSdcCIY3FEqHUuFwwYt4sJQO1pGRQDM5E4SahIx88DjjqVYdUNvsI0NC1EjD+IKJiGgYDZECt9IgF0brrbCQDinC4d6HE/HQuqMpiQehNyEjH0pRI/oh7rzqCAggfWZUQJsoSEuYighHEmJYGC/uppd0APyIh6jjUX8QIsgIKflQu+Op+zwa9/9QAu9RkJbwMIFZyKHSSrJ6SgfAl3hQ1IMIRUJKPgKh12JzenRAbbO/BAnRojOqP5inZQIR7KKikmB4Q2/pANQTD606lRL+MV9shtms7JkjOGhtF7Uh+ZCAlDk/pIqMHh1QvR5DZCpGzpBcF1xFQ8QgpnHmWVA0lAtfaDlyxe9xOBMPFms/STsfs0MRhCJCTj60Sr0A7AVEbP8PAIqjIGqskuvep0VDwlXfECUEauDVlBMO5MIbPEQ53MdRcYE4NcVDDDS8ljAiIScfgQgkDGrPeMpCQADt0zCAuChIm31ViIpoEg2Rihg54VQixMJqmCzARjqkCocLtcVDinToGfUgqSHUhORDZdToRyJFQAC+oyBtjsEwKqJ7NEQKBhMPlqLREj2lA+BLPMQgVhAo3ULwBsmHF1gLgxr9P6T0EVEqIWp1Rg14LIZREUOJCIeoJRsu9JYOgD/x0HN4LUU9CLUh+ZCBnNSL3gICsEnFqJ2G8XtcRlERXw0pSckl1BaNlvAgHQB/4sESSrcQPELywTFqzO/BIgqiZRrG57FV7rQaCKOLipaC4Q0th8v6g9c5PMTsR5JAGBkamO6DQF9sOTlUOQ+LQA8h2Q+3dib5vfSjTaIf2o7ocGYNhc9ztItgNipCLM4Yi6iX1vBYL9ffp+VL9rEY3U9S7mH3PhyJh/hjSStPQqM+586dw8SJExEbG4v4+HhMnz4ddXV1fvdpaGjArFmz0KlTJ7Rv3x5jx45FVVWVR5mHHnoIGRkZsFgsSE9Pb3OMiooKDB8+HElJSYiKikKvXr2wYMECNDVd/vG2du1ahIWFebyiojxviqlTp7Ypc/vtt0u+DhT50Bg9O6B63VdBKkbO9Owu1IiIqBENUUqght5fBEXv6IRc1BBBvSId7v04Ew8xkkCdTPlk4sSJOHXqFLZv346mpiZMmzYNM2fOxPr1633uM2fOHHz88cfYtGkT4uLikJubi7vvvhtfffWVR7nf//732LNnD/71r3+1OUZERAQmT56MG264AfHx8fjnP/+JGTNmwOl04plnnnGXi42NRUVFhfvfYWFt78nbb78db775pvvfFov0ZxXJhx/0HnZ7+TzqTsGuJBUjdXp2Fy0bk1AREW8YVTBaombUSev+HF7312CBOB7WbqGoh/ocPHgQRUVF+OabbzBw4EAAwEsvvYSRI0dixYoVSElJabNPTU0NVq9ejfXr1+OWW24BALz55pvo06cPvv76awwePBgA8OKLLwIAzpw541U+evXqhV69ern/3b17d+zcuRP/+Mc/PMqFhYUhOTnZ7+ewWCwBywSC0i46oEb6RWwZv/srmB5ayQPeFUpXK0WjR1ommGGVQvF7Dgb3gpzUSptjGFQ8KN3ChtraWo+X3a6sr1dJSQni4+Pd4gEAWVlZMJlM2LNnj9d9SktL0dTUhKysLPe2tLQ0dOvWDSUlJbLrcvjwYRQVFWHYsGEe2+vq6tC9e3ekpqbirrvuwoEDB9rsu3PnTiQmJuKaa67BH/7wB/zyyy+Sz0/yoRC5oU2eBUSLviD+UEtE1G4wgxUtZMN9Lo6kg1fx4EEUHBwH7Ez1diYvAEhNTUVcXJz7tWzZMkV1s9lsSExM9NgWHh6Ojh07wmaz+dwnMjIS8fHxHtuTkpJ87uOPIUOGICoqCldddRVuvvlmLFmyxP3eNddcgzVr1uDvf/873nnnHTidTgwZMgQ//fSTu8ztt9+Ot99+G8XFxVi+fDm++OIL3HHHHXA4HJLqQWmXAIjpoyE3/SKn/4fYFAwARSNllKZipKZhfKFWXxGxjSjPKRuW6CVkLAWThfhqNYxWrqyo0ddDqszwLB6sOXHiBGJjY93/9tW3Yd68eVi+fLnfYx08eJBp3eSyceNGnD9/Hv/85z/x2GOPYcWKFXj88ccBAFarFVar1V12yJAh6NOnD/76179i6dKlAIBx48a53+/Xrx+uv/569O7dGzt37sSIESNE14Pkw4CInuFUx8XqWApIS9TuK9LmfBIbZZ5khccID+toFgvhALRdn0VN8SDYEhsb6yEfvnj00UcxdepUv2V69eqF5ORknD592mN7c3Mzzp0757MPRXJyMhobG1FdXe0R/aiqqpLV7yI1NRUA0LdvXzgcDsycOROPPvoozGZzm7IREREYMGAADh8+7Pdzde7cGYcPHyb5YA1v0Q8p6BkFkdsZVSxajKCRiloRFR5FQgxqDbPWWzoAPsWDoh76kJCQgISEhIDlrFYrqqurUVpaioyMDADAjh074HQ6kZmZ6XWfjIwMREREoLi4GGPHjgVwadhsZWWlR5RCDk6nE01NTXA6nV7lw+Fw4Ntvv8XIkSN9HuOnn37CL7/8gi5dukg6N8kHQ3hLv7QuD+grIYB6IgJoHxVRglFlwh9qz+cC6JdaaXMMjcSDoh3BRZ8+fXD77bdjxowZKCwsRFNTE3JzczFu3Dj3SJeff/4ZI0aMwNtvv41BgwYhLi4O06dPR15eHjp27IjY2FjMnj0bVqvVPdIFuNSBtK6uDjabDRcvXkR5eTmASxGOyMhIrFu3DhEREejXrx8sFgv27duH+fPnIycnBxERl55HS5YsweDBg3HllVeiuroaf/7zn3H8+HHcd999AC51Rl28eDHGjh2L5ORk/Pvf/8bjjz+OK6+8EtnZ2ZKuBckHY7QafnvpXNLTKnqmYgB9RATgX0aMiBay4YIX6QC0W6NFbfGgqIc+rFu3Drm5uRgxYgRMJhPGjh3rHiYLAE1NTaioqMCFCxfc21544QV3WbvdjuzsbLzyyisex73vvvvwxRdfuP89YMAAAMDRo0fRo0cPhIeHY/ny5fjhhx8gCAK6d++O3NxczJkzx73Pf/7zH8yYMQM2mw1XXHEFMjIysHv3bvTt2xcAYDab8a9//QtvvfUWqqurkZKSgttuuw1Lly6VPNdHmCAIbOfvVoHa2lrExcWh+/KnYYpq+40xXZT2UDHLGC0lJTIhVz7kpl/kygSLqduVrBXT5lgqyogLkhD5aCkbAB+pFY/jcJhm8TyXuseXKh/O6MvfZ2dDA47PXYCamhpRfSjk4Gonsq6ag3CzMlNqdtjx2Y8vqFrfUIciHyqgdf8PudEMVlEQgI2EaBEV8dWAkpS0xaiy4T6eTukVpfuqLR5yUCIeBOENkg+V4LUDamtY9AUBlK+Y2+Z4rRoitaMiUhtaI8qK1jLhD9ai4T6ujlEOpfvLTbNQJ1PCiPDzNCIUoTSKoffcIAGPrVFfEbFIachZiwpPEiEWtWTDfXwDSwegnXhoAUU9CDEY7ynGAIdFer8PeSNSLv1XagTE9SCSMwLm0vmUS4iS47RsCNQWkZbwICXeMKIsKEFt0fA4l86pFRbH0VI8aPQMwQtB8VR0RjsldzrVSkAA/STEhWyJYCwirWEtJkaTEqOjpWQA7KIbgP7C4SKYIh4EIYWgkA+AfwEBtJeQy+dVLiOshMbjmD4aE62kxB+hLCxaS0Wb8zOUjDbHZiAdrMSFx46lQcGFi4BJ4ffXqWwBOSIwQSMfgDEEBNBPQi6fn0FEQwUZcR9bw2iJzzowbIDVFhm9ZUEuakqGx3k4iXK4oGgHQQSZfADGERBAfwm5VAd+UjSizqNRtIQlRpUDVmglGR7n5Ew4XJB4EMQlgk4+AG0FxBdSxIAHCblcF76jIj7PyUG0JNTRQzLa1IGjtEprSDwI4jJBKR+AdgLi81gyxIAnCblUH/ZRETGwlhWljWIoygsPIuFCLRnQ8jxaiweNaiF4J2jlA9BfQABlEgJIExGxDxw9O67KOY/a5wsEy4ZYlaHHHImCErSSDK3PSeJBEG0JavkA+BAQQMmw2Uv/ZblYXeuHEwsZ0SS14qeh0EtMpBIsoqAUPUSD9bnVbORpDg8i2Al6+QD4ERCALwlxwUJG9Ojn4e/8etWD8ERPyWiJEYTDBYkHEQqEhHwAfAkIwKeEuGj5IGOVohEL8/4ejBu/UJIZXsRBCSw+g1YNO6VZiFAiZOQD4E9AAL4lBGCXohEL76kVlg0y07lRgkAUvKHn59KyUSfxIEKNkJIPgE8BAfiXEBcsoiJyCbbUSrAKgxx4uRZ6NOYkHkQoEnLyAfArIIByCZGDXHHROiriC96jJcRleJGMlujZiJN4sEe4cBGCyaHsGM5GRrUhfBGS8gHwLSCAenN3eKP1A9DoMtIStRo7khr/8CgZLWHVeOsxARiJBxEMhKx8APwLCKCthLhQS0aUwIPItITHjqy8N/gs4KHh1XPGUR4+P0GwIKTlAzCGgAD6SIgLuZOesYTVVPa8EgriIAUeG1mSDoJgR8jLB2AcAQH07fAJsIuKsCTYxSSY4b1R5WFdFd6vEUHIgeTjv8gVEJbIXdhOzwaWRxlpiRYPbhKcwBipAeVBOFzIvW6sn00EwRqSjxbIERCWuB4YRpQQFzykaLTGSA0r4R2ehMMFiQcRzJB8tEJvAQGUS4gayJtynX09vBEqkhPK8CgHakLiQQQ7JB9e4EFAAPkSogZ69zXxh6+GiaTEmBhFNHiLeJF4EEaC5MMHvAgIwJeEAHzO5+ENf40YiYn+GEUyWsKbcAAkHYQxIfnwA08CAvAnIS54jor4wogNH6EPPAqHCxIPwqiQfASANwEB+JUQwDhREYLwB8/C4YLEgzAyJB8i4FFAAL4lxIWeD3ESn+DACCKgNSQevhHqL0AIa1J2DEHZ/kRgSD5EwquAAMaQED3w1WiRlPAJSYY4SDyIYIDkQwI8Cwig30PJaNJDjRzBChIBgpAHyYdEeBcQPWj9ADaajBCEFEg4CEI5JB8yIAHxD8kIEWyQcBAEW0g+ZOKMdup6fiPJD28PbpIhY8DbfUMQBDtIPgyKS36MJCG8QI0aEczo/cOIIMRALZfBcUY76WFDEAQAEg/COMiSj4KCAvTo0QNRUVHIzMzE3r17/ZbftGkT0tLSEBUVhX79+mHbtm2yKkv4xiUhvLwIgtAW+t4Zg3PnzmHixImIjY1FfHw8pk+fjrq6Or/7NDQ0YNasWejUqRPat2+PsWPHoqqqyqPMQw89hIyMDFgsFqSnp3s9zieffILBgwejQ4cOSEhIwNixY3Hs2DH3+5s3b8att96KhIQExMbGwmq14pNPPvE4Ro8ePRAWFtbmNWvWLEnXQbJ8bNy4EXl5ecjPz8f+/fvRv39/ZGdn4/Tp017L7969G+PHj8f06dNRVlaGMWPGYMyYMfjuu++knpowEHrLD0lRcKL3/cPzizAGEydOxIEDB7B9+3Zs3boVu3btwsyZM/3uM2fOHHz00UfYtGkTvvjiC5w8eRJ33313m3K///3vkZOT4/UYR48exV133YVbbrkF5eXl+OSTT3D27FmP4+zatQu33nortm3bhtLSUgwfPhyjR49GWVmZu8w333yDU6dOuV/bt28HAPzv//6vpOsQJgiCIGWHzMxM3HjjjXj55ZcBAE6nE6mpqZg9ezbmzZvXpnxOTg7q6+uxdetW97bBgwcjPT0dhYWFos5ZW1uLuLg4dF/+NExRNEkDQRCEkXA2NOD43AWoqalBbGysKudwtRO3RP4vwsMiFB2rWWjCjsZNzOt78OBB9O3bF9988w0GDhwIACgqKsLIkSPx008/ISUlpc0+NTU1SEhIwPr163HPPfcAAA4dOoQ+ffqgpKQEgwcP9ii/aNEibNmyBeXl5R7b33//fYwfPx52ux0m06W4w0cffYS77roLdrsdERHer9m1116LnJwcLFy40Ov7jzzyCLZu3Yoff/wRYWFhoq+FpA6njY2NKC0txfz5893bTCYTsrKyUFJS4nWfkpIS5OXleWzLzs7Gli1bfJ7HbrfDbr88JKGmpgbApRuYIAiCMBauZ7fE37qyaEYToPA0zbg0vXptba3HdovFAotFfo/1kpISxMfHu8UDALKysmAymbBnzx789re/bbNPaWkpmpqakJWV5d6WlpaGbt26eZUPX2RkZMBkMuHNN9/E1KlTUVdXh7/97W/IysryKR5OpxPnz59Hx44dvb7f2NiId955B3l5eZLEA5AoH2fPnoXD4UBSUpLH9qSkJBw6dMjrPjabzWt5m83m8zzLli3D4sWL22w/kf+0lOoSBEEQHPHLL78gLi5OlWNHRkYiOTkZu2xbmByvffv2SE1N9diWn5+PRYsWyT6mzWZDYmKix7bw8HB07NjRZ5tos9kQGRmJ+Ph4j+2B2tHW9OzZE59++inuvfde3H///XA4HLBarX77YK5YsQJ1dXW49957vb6/ZcsWVFdXY+rUqaLr4YLLobbz58/3iJZUV1eje/fuqKysVO3GDQZqa2uRmpqKEydOqBbaDAboOgWGrpE46DqJo6amBt26dfP5C5oFUVFROHr0KBobG5kcTxCENr/mfUU95s2bh+XLl/s93sGDB5nUSy42mw0zZszAlClTMH78eJw/fx4LFy7EPffcg+3bt7f5rOvXr8fixYvx97//vY0wuVi9ejXuuOMOr+miQEiSj86dO8NsNrfpZVtVVYXk5GSv+yQnJ0sqD/gObcXFxdEXXASxsbF0nURA1ykwdI3EQddJHK6+BmoRFRWFKB36BT766KMBf/336tULycnJbQZnNDc349y5c37b0MbGRlRXV3tEPwK1o60pKChAXFwcnnvuOfe2d955B6mpqdizZ49H+mbDhg247777sGnTJo90T0uOHz+Ozz77DJs3bxZdh5ZIuhMiIyORkZGB4uJi9zan04ni4mJYrVav+1itVo/yALB9+3af5QmCIAjCSCQkJCAtLc3vKzIyElarFdXV1SgtLXXvu2PHDjidTmRmZno9dkZGBiIiIjza0YqKClRWVkpqRy9cuNBG/sxmM4BL7biLd999F9OmTcO7776LUaNG+Tzem2++icTERL9l/CJIZMOGDYLFYhHWrl0rfP/998LMmTOF+Ph4wWazCYIgCJMmTRLmzZvnLv/VV18J4eHhwooVK4SDBw8K+fn5QkREhPDtt9+KPmdNTY0AQKipqZFa3ZCCrpM46DoFhq6ROOg6iYOu02Vuv/12YcCAAcKePXuEL7/8UrjqqquE8ePHu9//6aefhGuuuUbYs2ePe9sDDzwgdOvWTdixY4ewb98+wWq1Clar1eO4P/74o1BWVibcf//9wtVXXy2UlZUJZWVlgt1uFwRBEIqLi4WwsDBh8eLFwg8//CCUlpYK2dnZQvfu3YULFy4IgiAI69atE8LDw4WCggLh1KlT7ld1dbXHuRwOh9CtWzdh7ty5sq+DZPkQBEF46aWXhG7dugmRkZHCoEGDhK+//tr93rBhw4QpU6Z4lH/vvfeEq6++WoiMjBSuvfZa4eOPP5Z0voaGBiE/P19oaGiQU92Qga6TOOg6BYaukTjoOomDrtNlfvnlF2H8+PFC+/bthdjYWGHatGnC+fPn3e8fPXpUACB8/vnn7m0XL14UHnzwQeGKK64Q2rVrJ/z2t78VTp065XHcYcOGCbg0zsfjdfToUXeZd999VxgwYIAQExMjJCQkCHfeeadw8ODBgMdo3aZ/8sknAgChoqJC9nWQPM8HQRAEQRCEEmhtF4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgiAIgiA0hRv5KCgoQI8ePRAVFYXMzEzs3bvXb/lNmzYhLS0NUVFR6Nevn9/56YMJKdfp9ddfx80334wrrrgCV1xxBbKysgJe12BA6r3kYsOGDQgLC8OYMWPUrSAnSL1O1dXVmDVrFrp06QKLxYKrr746JL53Uq/TqlWrcM011yA6OhqpqamYM2cOGoJ8Ucxdu3Zh9OjRSElJQVhYmN+FQ13s3LkTN9xwAywWC6688kqsXbtW9XoSHCF7kC5DNmzYIERGRgpr1qwRDhw4IMyYMUOIj48XqqqqvJb/6quvBLPZLDz33HPC999/LyxYsEDyxGVGROp1mjBhglBQUCCUlZUJBw8eFKZOnSrExcUJP/30k8Y11w6p18jF0aNHha5duwo333yzcNddd2lTWR2Rep3sdrswcOBAYeTIkcKXX34pHD16VNi5c6dQXl6ucc21Rep1WrdunWCxWIR169YJR48eFT755BOhS5cuwpw5czSuubZs27ZNePLJJ4XNmzcLAIQPPvjAb/kjR44I7dq1E/Ly8oTvv/9eeOmllwSz2SwUFRVpU2FCd7iQj0GDBgmzZs1y/9vhcAgpKSnCsmXLvJa/9957hVGjRnlsy8zMFO6//35V66k3Uq9Ta5qbm4UOHToIb731llpV1B0516i5uVkYMmSI8MYbbwhTpkwJCfmQep1effVVoVevXkJjY6NWVeQCqddp1qxZwi233OKxLS8vT7jppptUrSdPiJGPxx9/XLj22ms9tuXk5AjZ2dkq1ozgCd3TLo2NjSgtLfVYvMZkMiErKwslJSVe9ykpKWmz2E12drbP8sGAnOvUmgsXLqCpqUnVlSX1RO41WrJkCRITEzF9+nQtqqk7cq7Thx9+CKvVilmzZiEpKQnXXXcdnnnmGTgcDq2qrTlyrtOQIUNQWlrqTs0cOXIE27Ztw8iRIzWps1EIxWc44YmkVW3V4OzZs3A4HEhKSvLYnpSUhEOHDnndx2azeS1vs9lUq6feyLlOrZk7dy5SUlJ8rlJodORcoy+//BKrV69GeXm5BjXkAznX6ciRI9ixYwcmTpyIbdu24fDhw3jwwQfR1NSE/Px8LaqtOXKu04QJE3D27FkMHToUgiCgubkZDzzwAJ544gktqmwYfD3Da2trcfHiRURHR+tUM0IrdI98ENrw7LPPYsOGDfjggw90WXKaR86fP49Jkybh9ddfR+fOnfWuDtc4nU4kJibitddeQ0ZGBnJycvDkk0+isLBQ76pxxc6dO/HMM8/glVdewf79+7F582Z8/PHHWLp0qd5VIwiu0D3y0blzZ5jNZlRVVXlsr6qqQnJystd9kpOTJZUPBuRcJxcrVqzAs88+i88++wzXX3+9mtXUFanX6N///jeOHTuG0aNHu7e5lpYODw9HRUUFevfurW6ldUDOvdSlSxdERES4l+AGgD59+sBms6GxsRGRkZGq1lkP5Fynp556CpMmTcJ9990HAOjXrx/q6+sxc+ZMPPnkk22WNA9VfD3DY2NjKeoRIuj+TYiMjERGRgaKi4vd25xOJ4qLi2G1Wr3uY7VaPcoDwPbt232WDwbkXCcAeO6557B06VIUFRVh4MCBWlRVN6Reo7S0NHz77bcoLy93v+68804MHz4c5eXlSE1N1bL6miHnXrrppptw+PBht5wBwA8//IAuXboEpXgA8q7ThQsX2giGS9gEWsPTTSg+w4lW6N3jVRAuDWezWCzC2rVrhe+//16YOXOmEB8fL9hsNkEQBGHSpEnCvHnz3OW/+uorITw8XFixYoVw8OBBIT8/P2SG2kq5Ts8++6wQGRkpvP/++8KpU6fcr5bLNwcbUq9Ra0JltIvU61RZWSl06NBByM3NFSoqKoStW7cKiYmJwtNPP63XR9AEqdcpPz9f6NChg/Duu+8KR44cET799FOhd+/ewr333qvXR9CE8+fPC2VlZUJZWZkAQFi5cqVQVlYmHD9+XBAEQZg3b54wadIkd3nXUNvHHntMOHjwoFBQUEBDbUMMLuRDEAThpZdeErp16yZERkYKgwYNEr7++mv3e8OGDROmTJniUf69994Trr76aiEyMlK49tprhY8//ljjGuuDlOvUvXt3AUCbV35+vvYV1xCp91JLQkU+BEH6ddq9e7eQmZkpWCwWoVevXsKf/vQnobm5WeNaa4+U69TU1CQsWrRI6N27txAVFSWkpqYKDz74oPCf//xH+4pryOeff+71WeO6NlOmTBGGDRvWZp/09HQhMjJS6NWrl/Dmm29qXm9CP8IEgWKBBEEQBEFoh+59PgiCIAiCCC1IPgiCIAiC0BSSD4IgCIIgNIXkgyAIgiAITSH5IAiCIAhCU0g+CIIgCILQFJIPgiAIgiA0heSDIAiCIAhNIfkgCIIgCEJTSD4IgiAIgtAUkg+CIAiCIDTl/wNgUBO+PXphBgAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "reset_manager()\n", + "\n", + "zeta = Function(space).interpolate(Constant(1.0))\n", + "configure_tlm((m, zeta))\n", + "\n", + "start_manager()\n", + "u, J = forward(m)\n", + "stop_manager()\n", + "\n", + "print(f\"{J.value=}\")\n", + "plot_output(u, title=\"u\")\n", + "\n", + "dJdm_zeta = var_tlm(J, (m, zeta))\n", + "print(f\"{dJdm_zeta.value=}\")" + ] + }, + { + "cell_type": "markdown", + "id": "74f20f32-c109-4506-8fbf-0db3e4cdffa1", + "metadata": {}, + "source": [ + "Next we compute the derivative of *this* derivative using the adjoint method.\n", + "\n", + "Again we need to remember that the result is a member of the dual space $V^*$, and is not a function, so we again use a Riesz map to visualize it. Here we use the same Riesz map as before, defined using the $L^2$ inner product.\n", + "\n", + "Being precise the function we visualize, $h^\\sharp \\in V$, is defined such that\n", + "\n", + "$$\\forall \\chi \\in V \\qquad \\left. \\frac{\\partial^2 \\hat{J} \\left( m + \\alpha \\zeta + \\beta \\chi \\right)}{\\partial \\beta \\partial \\alpha} \\right|_{\\alpha = 0, \\beta = 0} = \\int_\\Omega \\chi h^\\sharp,$$\n", + "\n", + "where $\\zeta \\in V$ defines the direction on which the action is computed, and $\\alpha$ and $\\beta$ are scalars." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "6107d209-7731-42c3-8d84-f61b92d30232", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:46:46.213205Z", + "iopub.status.busy": "2023-12-20T16:46:46.212986Z", + "iopub.status.idle": "2023-12-20T16:46:46.531377Z", + "shell.execute_reply": "2023-12-20T16:46:46.530260Z" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhwAAAG1CAYAAACyBS3HAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAACFZklEQVR4nO2dfXgU1d33v9ksySaBJCKSF+5YkKKAAlHQGESxmhorpaYPtoBcgJFCrcSC8aWIkFhBUW7lRl5qikrRCgWxlNsHeWJpkHIradBAKiAqKgq+JIC5QyCQt915/sAddjf7MmfmnJkzu7/Pde2l2T0zczJkZ77z/b2cOEVRFBAEQRAEQQjEYfUECIIgCIKIfkhwEARBEAQhHBIcBEEQBEEIhwQHQRAEQRDCIcFBEARBEIRwSHAQBEEQBCEcEhwEQRAEQQiHBAdBEARBEMIhwUEQErNr1y6rp0AQBMEFEhwEISEfffQRAKCxsRHffPONxbMhCIIwDgkOgjCR06dPw+FwYMmSJWHHffHFF/jnP/+JRYsW4fXXXzdpdgRBEOJwWj0Bgogl9u/fD0VRcPnll4ccoygKFi5ciEsuuQR33HEHfvvb35o4Q4IgCDGQw0EQJrJv3z4AwODBg8OOmzp1Kl555RUMGDAAp06dMmNqBEEQQiHBQRAmsm/fPqSmpiInJyfkmLi4OOTk5ODAgQNYtmwZ/vjHP5o4Q4IgCDGQ4CAIE9m3bx8GDRqEPXv24Cc/+Ql69OiBPn364LnnnlPHKIqCFStW4NFHH8VNN92EBx980MIZEwRB8IEEB0GYyL59+3Dq1Cn89Kc/xVVXXYVnnnkGWVlZuP/++9VwCwBcd9112Lx5M2688UbrJksQBMERSholCJP49ttv8d133yEuLg579uxRwyo33HADBg8ejL1792LIkCGIi4vDVVddBQC49tprrZwyQRAEN8jhIAiT+OCDDwAAjz/+uF8OR7du3QAACQkJ6nuFhYXmTo4gCEIwJDgIwiS8IZOf//znfu97m3xddtllps+JIAjCLEhwEIRJ7Nu3D3369EFmZqbf+//+97/hdDojlsoSBEHYGRIcBGES+/btw9ChQ7u8/8EHH+DSSy9FYmKiBbMiCIIwBxIcBGECbrcbBw8exLBhw7p89u9//zuoECEIgogmSHAQhAkcOnQIra2tXYTF2bNn8emnn5LgIAgi6iHBQRAm4E0YDRQW+/fvh9vtJsFBEETUE6coimL1JAiCIAiCiG7I4SAIgiAIQjgkOAiCIAiCEA4JDoIgCIIghEOCgyAIgiAI4TALjp07d2Ls2LHIzs5GXFwcNm/eHHGbHTt24KqrrkJiYiJ++MMfYs2aNTqmShAEQRCEXWEWHC0tLRg2bBhWrlypafzhw4cxZswY/OhHP0JdXR1mz56NX/3qV3jrrbeYJ0sQBEEQhD0xVBYbFxeHv/3tbygqKgo55ne/+x3efPNN7N+/X31vwoQJaGpqQmVlpd5DEwRBEARhI5yiD1BdXY2CggK/9woLCzF79uyQ27S1taGtrU392ePxoLGxERdeeCHi4uJETZUgCIIQgKIoOHXqFLKzs+FwiEsdbG1tRXt7O5d9JSQkwOVycdkXcQ7hgqO+vh4ZGRl+72VkZKC5uRlnz55FUlJSl20WLVqE3//+96KnRhAEQZjI0aNH8R//8R9C9t3a2oq+P0hBwzEPl/1lZmbi8OHDJDo4Ilxw6OGRRx5BaWmp+vPJkydx8cUX44ZuRXDGdbNwZgRBEAQrnUoHdnZsRo8ePYQdo729HQ3HPPj4/Wz06GHMRTl1yoPLRnyD9vZ2EhwcES44MjMz0dDQ4PdeQ0MDUlNTg7obAJCYmBh0qW5nXDcSHARBEDbFjJB4jx4OpBoUHIQYhAuO/Px8bN261e+9bdu2IT8/X/ShTSOue4rVUyB0oJxusXoKBEEQMQOz4Dh9+jQ+/fRT9efDhw+jrq4OPXv2xMUXX4xHHnkEX3/9NV555RUAwD333IMVK1bg4Ycfxt13343t27fjtddew5tvvsnvt7AIEhr2RsZ/PxJBBEFEK8yC4/3338ePfvQj9WdvrsXUqVOxZs0afPvttzhy5Ij6eb9+/fDmm2/i/vvvx3PPPYf/+I//wIsvvojCwkLDk5fxhkEQRgj8myYBQhBEtMAsOG688UaEa90RrIvojTfeiL1797IeiuBEXHKy1VOAcuaM1VOwJTKIahI9BEHwQMoqFbsjww1eNmQ8JySCtOErekh8EAShFxIcHJHxpkqEJti/F4mQ8JD4IAhCLyQ4OEBCI3qw8t/SbmLHqnAPCR2CsCckOAwgtdBICd7jRApazlo9AykJ/HuymwAxC3JZCMKeRKXgkEoIyHzjtwoZzokNRA8JkMiQ+CAI+xCVgkMKZLipEqEJ9u8juQghARIer/gg4RHbnHC3o81tsLW5m896LIQ/JDh4Q0LDvthMhJjl5NlN2JDrQRByQoKDF4xCw5PSda2YWMTR0mb1FMJjpoCUVNzY2Vkh8UEQ8kCCwygkNAxh1fmQUugE/i3ZQIDYVXzwhIQMQWiDBIcRGMQGCQ25CPbvIZ0IsYEAsav44Am5KAShjdgSHBbkV9hFaLiTuwk/RvyZDuHHMIL0IsT375fEh5RQ4ipBhCa2BIeJsAgNM272MqDl95RNlEgrQiR3P2JdfJDwIIiukODgDAkNY9hBlEgpQiQWILyraewkYOK6p5DoIIjvIcHBCRIa5iHq/BkRMrxDZ4YFDO/woaQCxg7ig9wOgjgHCQ6D2FlouJOs++ePP9tp2bFDEezfxyo3JfDvSioHRULxQcKDIOSHBAfEJ3ayCg0rhYBZaPkdZRAlsogQqQSIhOLDTq4HCQ8iVon+O1sERIoNEhrGkFWUBP67xrQAIfGhGxIeRKxBdzhBsIgNEhr64XXujAgXGVwQKQSIxOKDhEfscMKTiFaPsbVUTntoLRUR0J2OM7ILjc4kY19EUTjPWvsFD/ZvwVuEsGBUsPgKEMvFB2C5ALGD60HCg4h2SHBwQpTQkFUg8EbL72m2KOEtQpiOzTFsQ+6HP7KLDxIeRLRCgsMgJDTMQwZRYpUIESlAtMBVpEgoPkh4EIR4SHCEgGcJazQIjc5kPvNynhErCPSePyNCxQoR4vv3aUa+iLAQjSTiQ2bXg4QHES2Q4BCILEKDl1jggZa5iBYlwQh2/u0iQsyumjFFfIRDsDCR1fUg4UHYHRIcAhApNGQSD6KQRZSYIUJCwTNhVaQAsSQ51SRXRFbXg9qlE3aFBAdHSGiYh1WihLcICYXv35JRZ8QsAWJJcqpXfJjkegByiA9yOwg7QncxDriTnMLERmeywzKx0emKE/oSPv/vz12kl+HjJDm6vHji/fti/TsLub/kbn4vUXhSEtWXcFKSzr8EE5ecHPRlBXHdU1TxQVjLypUr0bdvX7hcLuTl5WH37t1hx2/cuBEDBw6Ey+XCkCFDsHXrVr/PFUVBWVkZsrKykJSUhIKCAhw6dMhvzBNPPIGRI0ciOTkZ6enpQY9z5MgRjBkzBsnJyejduzceeughdHaef4jZtGkTfvzjH+Oiiy5Camoq8vPz8dZbb/ntw+12Y/78+ejXrx+SkpLQv39/LFiwAIqiMJwhEhyG0CM0tN6MjN4MbSEIJJgDEFyYGN6nQBEiUoCIwhLxYTIyCA/fF2EeGzZsQGlpKcrLy7Fnzx4MGzYMhYWFOHbsWNDxu3btwsSJEzFt2jTs3bsXRUVFKCoqwv79+9UxixcvxrJly1BRUYGamhqkpKSgsLAQra2t6pj29nb84he/wG9+85ugx3G73RgzZgza29uxa9cuvPzyy1izZg3KysrUMTt37sSPf/xjbN26FbW1tfjRj36EsWPHYu/eveqYp59+Gs8//zxWrFiBgwcP4umnn8bixYuxfPlypvMUp7BKFAtobm5GWloabkr4BZxx5y+Kob5UIb/0QS5CoS6AoS6+ei7wrI6G5rEm3ZBlx9lqzp+wiBCNiHCMiORUUWEY0/I+LKiAkSH0IkvIpVPpwPb2jTh58iRSU1OFHMN7n9i+PwfdexjsNHrKg5uuOIqjR4/6zTcxMRGJiV3vGXl5ebj66quxYsUKAIDH40FOTg7uu+8+zJkzp8v48ePHo6WlBVu2bFHfu/baa5Gbm4uKigooioLs7Gw88MADePDBBwEAJ0+eREZGBtasWYMJEyb47W/NmjWYPXs2mpqa/N7/f//v/+GnP/0pvvnmG2RkZAAAKioq8Lvf/Q7Hjx9HQkJC0N//8ssvx/jx41Vh8tOf/hQZGRl46aWX1DHjxo1DUlISXn311ZDnMRByOAQiytEw8+lfL2Y6FWY5JXZxQng7IIA4F8Q058PEkIsXKx0PdQ4x6HR8507BcXd3Q6/v3OfOW05ODtLS0tTXokWLuhyvvb0dtbW1KCgoUN9zOBwoKChAdXV10DlWV1f7jQeAwsJCdfzhw4dRX1/vNyYtLQ15eXkh9xnqOEOGDFHFhvc4zc3NOHDgQNBtPB4PTp06hZ49e6rvjRw5ElVVVfjkk08AAP/+97/xzjvv4Cc/+YnmuQCxlDRq4oXG7o6GiONE2qdolyLc8Y0cO9i/n1EnJPDvx6gLwjMBFRDXA8S0iheTEk29xCUnW+p2UFWLfoI5HIGcOHECbrfb76YOABkZGfjoo4+C7re+vj7o+Pr6evVz73uhxmgh1HF8jxHIM888g9OnT+OXv/yl+t6cOXPQ3NyMgQMHIj4+Hm63G0888QQmTZqkeS6AjQUHczglCDzDKaJEBsAuAGR3P4IhShCwHpvHsXz/fXmEYbx/WzzCL96/YV5hF+93hXfIxfvdFC48YkR0EPpITU0VFgKSkXXr1uH3v/89/vu//xu9e/dW33/ttdewdu1arFu3Dpdffjnq6uowe/ZsZGdnY+rUqZr3b0vBwSN3IxSsYkMGN8OOAoMFM8VI4LGM7j/w39yIAPH9W5PV9bCd8DDR7SDREZ306tUL8fHxaGho8Hu/oaEBmZmZQbfJzMwMO97734aGBmRlZfmNyc3N1Ty3zMzMLtUy3uMGzm39+vX41a9+hY0bN3YJ9zz00EOYM2eOmjsyZMgQfPnll1i0aBGT4LBdDgcvscESMzYiNljj/Sw5B3bI5XC7xO5feN6GwDwQQ/vhWPkiIteDN8LzPEwKuVqd00HwJyEhAcOHD0dVVZX6nsfjQVVVFfLz84Nuk5+f7zceALZt26aO79evHzIzM/3GNDc3o6amJuQ+Qx1n3759ftUy27ZtQ2pqKgYPHqy+95e//AXFxcX4y1/+gjFjxnTZz5kzZ+Bw+F9v4uPj4fGwPfjY0uEIhJfYYLlQRrrYR4ubwUMwBNtHfGvX93jD2xmR0f2Q1fWwpeNhYhMxcjqii9LSUkydOhUjRozANddcg6VLl6KlpQXFxcUAgClTpqBPnz5q0umsWbMwevRoPPvssxgzZgzWr1+P999/H6tWrQIAxMXFYfbs2Vi4cCEGDBiAfv36Yf78+cjOzkZRUZF63CNHjqCxsRFHjhyB2+1GXV0dAOCHP/whunfvjltuuQWDBw/G5MmTsXjxYtTX12PevHmYOXOmmo+ybt06TJ06Fc899xzy8vLU3I6kpCSkpaUBAMaOHYsnnngCF198MS6//HLs3bsXS5Yswd133810nqJCcASFk9hgffKTTWiIdhj0Em5eZooRQwmjgnI/9IZdZMz1ECk8hIZZLF7FljeUOCqW8ePH4/jx4ygrK0N9fT1yc3NRWVmpJmgeOXLEzyEYOXIk1q1bh3nz5mHu3LkYMGAANm/ejCuuuEId8/DDD6OlpQUzZsxAU1MTRo0ahcrKSrhc5y+eZWVlePnll9Wfr7zySgDA22+/jRtvvBHx8fHYsmULfvOb3yA/Px8pKSmYOnUqHn/8cXWbVatWobOzEzNnzsTMmTPV96dOnYo1a9YAAJYvX4758+fj3nvvxbFjx5CdnY1f//rXfv08tGC7PhzdeqT7fWZFz41Q7oYWsSFSZMgqLngiUozwzAfhsS8eCae8+nzwSjLlLTyE9/EQKDzMdjmsFBxm9uHY+O+BSO4Rb2hfZ0658YthHwmdbyxiuxwOTViQJBpJbIjKzXC7zr9iAd/fN/BlFNlyP3j2+TCKrDkepvTvEITZ+Ryx2JODkIvoC6lYkCQa6sYQbW5GJ6djOgW5FLzCNNxDJQZyP4yGWdT9cMj14FlSyzPUYudKFsrnIGKJ6BIcFiSJGhUbLEKDz1O88X3wnoMoAeKL3sRV3omigfvUuj9ewgMwnuvhTnJK2cdDaF4HICy3w0zRQbkchJVEl+AIgsgkUSNiQ6TQkEFUsBBqvqKFiPe8yuJ+aNmfCOEBsIsPWRuIkeggCHmJasEhOkk06NgwQkKUyLCbwNBKuN+LpxjxPdd6xQfAoUyWoXJGlm6mPN0OgI/wMEV0ALatZIl2l6OhMw1JHcZubWc7+S+ASES54AiGmRUpfuMZkkC175NpClzRG97hVWUiSozocT288HI/WEt2rXY9eLsdgHHhIVx0ANzdDnI5iGgnagWHTEmikcSGlSLD7OTTwOOJKHPlIUaMCI9zczDufugVHoA1rgdvtwMwJjxMW48F4CY8zBId0e5yEHIStYIjGFYkiYYTG2YJDZlLZs3uQsqasKo33NL1uPrdDz1Nyni7Hla6HcC5726suB3kdBDRSswIDiuSREO9L0poyCwsWDDDBfHiPb9anA+jrsf5Y+pzPzpdcexihZPrEQ1uB4kOgrCWmBAcViSJGhEbdhMZbo3Rq3id13sz2qDrER68js/ifhhpyc7D9bCz22E30SEaCqsQZhMTgiMYIpNE9YoNq4WGVuHAa/96BYjfPjmLEd9/A1bxofeY/sfXJih4CA/A2IJxdkwqNSWvgxPkchDRRkwKDis6iYa6MZotMkSLChaCzYWHCFH3b1CMsLgewY5pNOdDU3mswUXojLgeLG4HIE54SOd22Ci0Qi4HYSYxJzisrkjRA6vQkElUsCLCBQl6HAZhoEd4BB5Dy3G6HpehLwcn4QGwiQ89/TtENA0j0UEQ8hNzgiMYosWGHnfDziLDk9T15uM4q28BMdEuCKA9GZQ13BLqOFqO5X9cNuFhuAGZDteD1e0A+K/NIl3prE1EB7kchFmQ4AiBVWKDRWiIFhnBhAOvfekVIEDo39uoEGGpQtHregQeywvLui4i8zv89sMoPIx0KwWMCw8pq1hsIjoIwgxiXnBYUZFiVGzwFBo8RYXR4xoRIUD488IiRvSEWwDzOpxaITxEux0AX+ERraJDFORyEGYQ84IjGCIrUvSKDSMiwypRwQpPFyQQvaEZM10P9uOZk1gKmOd2AHyEh3QhFk6ig1yOyHzX2QOuTmO3tlZaS0UI/K7oUQKPipSQ+9YhNtyJ2sWGJ8kT9GUlnkT9NznRvwvLuXW7tDtQna7zL91z03i8Tlec5r9N71gjf8vMawgxOIiBuJOcTI35umzP0Fk4EJalETTjbYNukLjkZC776bLf7ilC9ksQXsjh8MGKTqKhxrKIDDMwJByCbOto03fTE+GC+J7rSK4Ha+Inayv1UMeLHNphXHfFxAZiRtwOwJjjId1aLOR0EDEMCY4ImFGRogdRQsOIsDByHF4CBOCTkMo73OJFb86H28U3vyNwPMs26rY6hIde0QEYFx7SrMUiseigXA5CJCQ4wmBVkmgod4OXyDBLVLDAS4AA4c+TVjEiWngA+heSEyE89G4DsCWVGnU7AP3rtBgVHYA9OpQShKyQ4GDELmJDRlHBAs8wjN9+GUMyIsMtgWhNOhWRWBq4jRfNTokFbofZogPg6HaQy0HEICQ4QiCybbkIsWG6wHC59W/bGq9rM54uiLrP78+tFufDDNcDOPd3w3P1WkP5GoyCxUy3Q2+IxUheB8DR7ZBYdBCECEhwMMAiQswQG8JFhhFRwbJfTgIE4JMLotX1MCPcIoPwYM4LIbdDO5KKDnI5CBGQ4NAIj4qUULCKDa5CQ5SoMDIHnQIE4OOCaHU9zAi3sAoPUfkd3u1kdjusEh2AQbdDUtFBELwhwaEBK9qWcxcbMggLrQSbK0cXBNAmRESHWwDt4kNUfse5fYpLLDXT7bBKdAAc3A4JRQe5HARvSHDoRHTehi4sFBXxidqO7W7T6V5wdEEAfyESSXzoCbcAYsSHTImlMrodVuV1cMEGLdAJwgi6GhasXLkSffv2hcvlQl5eHnbv3h12/NKlS3HZZZchKSkJOTk5uP/++9HaaqD/s8VYlSQa0t1wuU0RG/GJ7pAvvfvQjfd39n3pxJOoaHaOWDqesnQyBdi7mfLcp54OpKzbdCY72JYNMNilVNd2Vncn5dCNVFQnUoIwCvO3csOGDSgtLUVFRQXy8vKwdOlSFBYW4uOPP0bv3r27jF+3bh3mzJmD1atXY+TIkfjkk09w1113IS4uDkuWLOHyS8iApWKDI4ZEAKfjWeWCeM+xleEWgCUJ9Nx/rU4sldntsGWTMA5OB6/Qih3DKt91pCCxQ79wBIC2DgudriiG+RFiyZIlmD59OoqLizF48GBUVFQgOTkZq1evDjp+165duO6663DnnXeib9++uOWWWzBx4sSIroisiGxbbpbY4OFUiMRqF8TreGhxPVjWefE6HrzXb2FZt0W7g8LoXkjqdsSy00EQssH0LW5vb0dtbS0KCgrO78DhQEFBAaqrq4NuM3LkSNTW1qoC4/PPP8fWrVtx2223hTxOW1sbmpub/V4yw9K23EyxYaWoSHS1+72MwHX+OsWHprGCwi2si8Zp2R+L8GCBeTyj6NAjPGwrOgzCK7RCi7oRvGD6Jp44cQJutxsZGRl+72dkZOCjjz4Kus2dd96JEydOYNSoUVAUBZ2dnbjnnnswd+7ckMdZtGgRfv/730ecj98XSscTge/Fi3UVTHU7hlCKaLFhhUOhVUz4jmtrTTB0TN/fU3f4BTh3HjWGXTyJiuYSW0+Sh6mFutbkUpaKFt5ltLKEWPRiZQWLbiiJlIgyhC9Pv2PHDjz55JP4wx/+gD179mDTpk148803sWDBgpDbPPLIIzh58qT6Onr0KNMxjT5daF/uO/j7QstfLRQbga6FXueCl/MBwLjzYSO3g+XvSkSIhQXW8Uz7Njm8ohcZXA6CkAmmb2CvXr0QHx+PhoYGv/cbGhqQmZkZdJv58+dj8uTJ+NWvfgUAGDJkCFpaWjBjxgw8+uijcDi6XjwSExORmMjny+pri+q54PheOLVcnFnyNmQVGzyEgJ5jWep82MTtiFang9XlMNqdlAVLXQ6CiCKYHhUSEhIwfPhwVFVVqe95PB5UVVUhPz8/6DZnzpzpIiri489d2BVFf8dMnuEUPQR7grSj2ODlWvBAVN6HZiRxOyKOsZHTwYLesCYLVuRzGMJg8iiVyBIywfztKy0txdSpUzFixAhcc801WLp0KVpaWlBcXAwAmDJlCvr06YNFixYBAMaOHYslS5bgyiuvRF5eHj799FPMnz8fY8eOVYUHT/TYmL4XOr0XVB6NvUSKDSuFhF54uR/MzofFbofsTgcLoktmzXI59MJtdVmCiAKYHynGjx+PZ555BmVlZcjNzUVdXR0qKyvVRNIjR47g22+/VcfPmzcPDzzwAObNm4fBgwdj2rRpKCwsxB//+Ed+v0UIzA6ndDk+Q5Iob7FhlWuRktge8mUE050PgW6HFmR2OmSrXGHFdi4HYQtYG2Ju3LgRAwcOhMvlwpAhQ7B161a/zxVFQVlZGbKyspCUlISCggIcOnTIb0xjYyMmTZqE1NRUpKenY9q0aTh9+rTfmLfeegvXXnstevTogYsuugjjxo3DF1984Tdm7dq1GDZsGJKTk5GVlYW7774b3333nfr5gQMHMG7cOPTt2xdxcXFYunQp+wmCzqTRkpISfPnll2hra0NNTQ3y8vLUz3bs2IE1a9aoPzudTpSXl+PTTz/F2bNnceTIEaxcuRLp6em6JswD3uEUHhUpQWEUG2YKDD2iQkbxERGGXie8QywkOjSOjXbRQT05pMfbELO8vBx79uzBsGHDUFhYiGPHjgUdv2vXLkycOBHTpk3D3r17UVRUhKKiIuzfv18ds3jxYixbtgwVFRWoqalBSkoKCgsL/bp0T5o0CQcOHMC2bduwZcsW7Ny5EzNmzFA/P3z4MG6//XbcdNNNqKurw1tvvYUTJ07g//yf/6OOeffddzFlyhRMmzYNBw4cwMaNG7F7925Mnz5dHXPmzBlccskleOqpp0Lma2ohTjGSSGESzc3NSEtLw00Jv0C3HukAfGKTPl/GwHBKKIcjVDls4EUxlMMRSXBwKX/VITZEYFQcRKKlzViiqC96wy6awiwMXUtZVqjVEmLRUjbLEgrREl7Ruk/WNVhYxrMkkeoJregpkwX0rbdiOKxisDzWaNdRo91GO5UObG/fiJMnTyI1NdXQvkLhvU/89p3bkdjdYKfR0x1YNuq/Nc83Ly8PV199NVasWAHgXG5jTk4O7rvvPsyZM6fL+PHjx6OlpQVbtmxR37v22muRm5uLiooKKIqC7OxsPPDAA3jwwQcBACdPnkRGRgbWrFmDCRMm4ODBgxg8eDDee+89jBgxAgBQWVmJ2267DV999RWys7Px+uuvY+LEiWhra1NzKf/v//2/uP3229HW1oZu3brhmWeewfPPP4/PPvtMncvy5cvx9NNP46uvvuoy9759+2L27NmYPXu29hP6PeKztCyCZziFB3YQG7zDIKzHNIpe58PqEEskt0Or0yFiHZbI+5IjiTTqXQ7CEgIbULa1dRWOehpiVldX+40HgMLCQnX84cOHUV9f7zcmLS0NeXl56pjq6mqkp6erYgMACgoK4HA4UFNTAwAYPnw4HA4H/vSnP8HtduPkyZP485//jIKCAnTrdu7vOD8/H0ePHsXWrVuhKAoaGhrw+uuvh23OqZeoFRyh0HVhMuhu8FhqnrfYsEJcsMzJKHrEh1UhFiByqE1EV1Je+2Ntgc6CWYu9icZwTw4Kq2jmfzuS0diRYuj1vx3nHPScnBykpaWpL28xhC/hGmLW19cHnWN9fX3Y8d7/RhoTuH6Z0+lEz5491TH9+vXD3//+d8ydOxeJiYlIT0/HV199hddee03d5rrrrsPatWsxfvx4JCQkIDMzE2lpaVi5cmX4E60Deb+hJsCjOiUSPMpfeYgNK8RF98Q29cWKVeJDs+iwIKGURAd/YsXloPJYfRw9etSvCeUjjzxi9ZSYqK+vx/Tp0zF16lS89957+Oc//4mEhATccccdaluKDz/8ELNmzUJZWRlqa2tRWVmJL774Avfccw/3+Zjbes8kjF4MtF4Q9VSuAHwWY9Ny8zRTWGgdc7qN/UnP9/cwmvPhPW/h8j28oiNiboeA8tlIpbO8W6FbVTIrqlxWT6msWW3PqUTWfqSmpkbM4dDTEDMzMzPseO9/GxoakJWV5TcmNzdXHROYlNrZ2YnGxkZ1+5UrVyItLQ2LFy9Wx7z66qvIyclBTU0Nrr32WixatAjXXXcdHnroIQDA0KFDkZKSguuvvx4LFy70O75Rot7hCJUsqnn7MOGULmMD7qWiyl/DiQ3RLoava6HHvTDiegD8K13CYZXbYVenQ5b25zKHVgxBYRUp0dMQMz8/3288AGzbtk0d369fP2RmZvqNaW5uRk1NjTomPz8fTU1NqK2tVcds374dHo9HrRwN13jT4/FEHMO7piRKv5mRMRpO0eNuiBQbvEVGMGGhVyREOoZeeIgPLqID4J7bQaIjyFiB+RxmhVZofZXopLS0FC+88AJefvllHDx4EL/5zW+6NMT0DcfMmjULlZWVePbZZ/HRRx/hsccew/vvv4+SkhIAQFxcHGbPno2FCxfijTfewL59+zBlyhRkZ2ejqKgIADBo0CDceuutmD59Onbv3o13330XJSUlmDBhArKzswEAY8aMwXvvvYfHH38chw4dwp49e1BcXIwf/OAHuPLKKwGca865adMmPP/88/j888/x7rvv4re//S2uueYadT/t7e2oq6tDXV0d2tvb8fXXX6Ourg6ffvop03mKypCKEXR3GtV7HeHQRdSo0OAtJIwcX0/IBTAWdkl0tfMLsQCawixe0REuzCJreCXyftjCJUz7NmFl2WglLjnZcHksEZzx48fj+PHjKCsrQ319PXJzc7s0xPR1EUaOHIl169Zh3rx5mDt3LgYMGIDNmzfjiiuuUMc8/PDD6rpjTU1NGDVqFCorK+FynVf9a9euRUlJCW6++WY4HA6MGzcOy5YtUz+/6aabsG7dOixevBiLFy9GcnIy8vPzUVlZiaSkc47ZXXfdhVOnTmHFihV44IEHkJ6ejptuuglPP/20up9vvvlGFSgA8Mwzz+CZZ57B6NGjsWPHDs3nKSr7cAQriQ184gnlcPDqvRH4hCqq/FWv2LBaZGhBr/jwhVV8ROrloXlROI59OyL16tAiOgDtORhaRAfvHh3M/Tw0ig7WfA6zenMYyuUw0JNDr+CwUx+OyW9PREJ3Y7le7afb8ecf/UXofGORmHQ4jGa9a230FRYLxIZZIiM9MfjdqKmNLQ5lJNHUC6vzocXt0LweC8DF7ZDR6dCyL6bVYgW5IqxJpHoTSAmCiExMCg5fQrkbRmBqX+4D714bogVGKGERabxe4QGYIz64hVgAbpUsJDoCxkoWWqGKFYKITMwLjlBorU7R4m50Cadw6LURyt0QJTJYxYXWfVktPkIJD63lsyLcjnCiAwgdYiHREWJctLkcKUm6wyqUx0FYScxWqeglUpa+HndDNrGRntja5SUKI/vnUT0TKSzFrYoFYCqfDft5mL8xK6pX9Paj4YHW8KgZpbJUsUIQ4YlpwSEinBIIS3trX8wWG2aIC5HHN9rVNByRenZoWovFi8a+HdEmOkR2ItW8XwbRQWWyBMEfCqkEQeuFmEeyKGuvjWDoFRtmiIsLE1vwXVsK0zZGQi6AvrBLpBALwDGhFNCU2xGpO2m4vA4ZS2atDq2wInVoxeSwSlz3FMOVKmbR2J6CbgY7Ene026t1vV0gwREB36c71nCKHndDpNgQKTAuTAx9MfL9TK/40CM8AHbxkZLYTqIjBJFERzTmc+iBEkgJIjgkOKCz0yjHUthIGBEbIkRGOHGhdVuzXQ9Au/jQIjqA0AmlzFUsQFjhQaKDb7ksi+iQ2uUgCJsR0zkcwdC9IBuHZFGWXhvhxAbvfIwLE1v8Xrz3yQqP3y2SWNPSMt3MhFKZcjp47EdEnobIVWVZMSWXg9ZWIWyGPN9QCWEJpwSipRQ2EqxiQ5TIEI1R4WE00TQcPEQHr/VYZBEdvCpXtIoOEeutmJFAKjN6lquP687mShJEICQ4dKBlVdhIMD39+hBObBjFTJHB+/hGxAcP0WHWInBGRYcW4WGm6NCKHUUHVawQhD/RJ90lgEeyaLCbHG+xIVJY9Ozmv+/GDranIx6JpoD2fI/uiW0R8zoAkzqURkgmNZLTAWjL69CWi2G8MZjV+Ry2x0C1CkGYDQmOEGgNp2hyNzSUwkaCl9jgLTIChYWWcXrFB6vwANjEh5a1W4xWsQAaK1lIdBgaK6JqRU8CabRVrNipPJaQDxIcjESypHmWwkZCq9iwSmRE2t5M1wPQLj60uB1GqlgAjW5HFImOSAipRpGoVFYoOl0OanNOmA3lcIiGY7KoL+HEhoiqkp7dWtQXL4zs0+jvFkmsmVHFAmhwuwTmdADm5XRYlUSqFa35HJTLQRD6IYfDAHoafWkphQ0k8OYX6mbJ08ngKSxYj8fifIhsKmZmiAUI43ZEidPBs0cH79AKC9SbgyD0QQ4HA8zNvgS5G8HgITZEuBhG5sGKXleHh9sRDi1VLEAEtyPCGiyyOB089sHb6ZBhgTfhLofOnhxUHkuYCTkcOjGrFFZLN1EjYoO3uOjV7XTIz050dGfal5mJpumJrRHzOoDQbgePKhZAQ0JpGLdDBqeD15L2vOGdz0Euh7ycbEuE02ksLNUZ5ntE6IccDk6IKoUNJPBpnFVs+LoYvMRGr26n1RePccEww/XQ0sdDdKMwwFhehxlORyTMXF1W1MqyWmDN55DV5SAIsyDBIQqBpbCsiAiVGBEPvtuzYkaiqQwhFplFh1mNwaIxtEIQsQx9szhgViksS88NGUUG732KFB4yVLHo7UYL2Ed0RN6HNQ6GqKoVGV0OyuMgzMJWgiMuhf2LEQiPBZ4i5m9wSBbVm7shIumTt8gIdxw9GBEe4TAqOrSgtwcLAMMlszLAq/05b5eDIAj+RPW3j8eiSzwuiHpKYQPR4m7I7mZoPa5erKiuMRpeiYQRlyMSZrgcPLAyT0ML0bi4G0GIIGoEB49mOXouoCJXhQ2HqHVQrBAZPOfAKjqMuhxa4JFIGhLJXQ4eYRWroFwOguCLLb9RemKOgZj11GRWsqjRp3vuuRnOU34vvXPSA2uIRfbQSrS7HGY5JV4orEIQ1hAT3zweTyo8ygUBa0phwyFKaGh9P+L+THQ7wiG6csUQMeByaHlAkD30QhCxTtQJDtYs8GDovkBaVArLemMVUm2iUVDI6naIClH5YiS0YqXLYQZmuxy8YcnjkLFShSDMIOoEBytGnopYnxxFl8JGQkR+hh7nQla3Q4bQim4EuhyxnDxKeRwEwY+Y+jaZUhLLAI9kUS03UpH5GUb3o2s7gW6HGf05whGtLodZyaNUHksQ8hK19VxWl8SKXBWWBd5Cgze9nKdworMH+3bf/16s67MA54QH69osLHRPbDO0wqxuDK4qGw4t66wYxYo1Vojoo6U9EfEG11JxC0y5imVI5n+PYVuYsRxWdCksT1eDh5shav8iQixmlMqGw44uB4VVqB8HQUQiJgWHjNnsPJNF7SI0gh1P13YGWqOHQubQSlgszOWIBIVVCCK2ifpvnKiSWJYLN49kUS3uBg9Xg1d+htHj69qWc0Ks0coVUaWy0exy2L1aRStUqULEIlEvOFjh8RQW6YagZ92UQLivlyJAZGR2a1JfeuajB1bRZeQ8iu5CSi5HqH3I25ODwioEEZqYFhwyhlYAfTcy3cvEcxYaoUSGXtFhhtthZWhFL+RymAuVxxKEcWLmWyS0JJYhYVSqdVMECA2jY4JhhtthRHREwkhoJRpdDh7weligPA6CMA/6toXArKcsPaWwgTdHVneDp6uhR0TI7HbowYwF3oJhV5dDtuRRq+DRFZkg7AQJDogVF4ZWAgXndVMsFho8thfpdsgaWtFdJqtj5WKtiHY57BxWsWMeB48FMWOdlStXom/fvnC5XMjLy8Pu3bvDjt+4cSMGDhwIl8uFIUOGYOvWrX6fK4qCsrIyZGVlISkpCQUFBTh06JDfmMbGRkyaNAmpqalIT0/HtGnTcPr0+evcY489hri4uC6vlJTzfYg2bdqEESNGID09HSkpKcjNzcWf//znkPO+5557EBcXh6VLlzKcnXPEnODg8dQTzo4W+dQJ6E9y5CE0jCSBhtsnKyLdDplLZXkTKaxidS5HJHglj9olrMJcqUKYxoYNG1BaWory8nLs2bMHw4YNQ2FhIY4dOxZ0/K5duzBx4kRMmzYNe/fuRVFREYqKirB//351zOLFi7Fs2TJUVFSgpqYGKSkpKCwsRGvr+evMpEmTcODAAWzbtg1btmzBzp07MWPGDPXzBx98EN9++63fa/DgwfjFL36hjunZsyceffRRVFdX44MPPkBxcTGKi4vx1ltvdZn33/72N/zrX/9Cdna2rvNkj2+ahUR8kjPwFMmjs2ikGygPV4O3yOC1fytCLGYs8haMaHM5zAqryI7QsAqVxprGkiVLMH36dBQXF2Pw4MGoqKhAcnIyVq9eHXT8c889h1tvvRUPPfQQBg0ahAULFuCqq67CihUrAJxzN5YuXYp58+bh9ttvx9ChQ/HKK6/gm2++webNmwEABw8eRGVlJV588UXk5eVh1KhRWL58OdavX49vvvkGANC9e3dkZmaqr4aGBnz44YeYNm2aOpcbb7wRP//5zzFo0CD0798fs2bNwtChQ/HOO+/4zfnrr7/Gfffdh7Vr16JbN31/tyQ4fBB9EWR9gtWzboovsguNYMdjRYToEFkqa0kCaRhEuhxG0eKSyJqnYcewCtGV5uZmv1dbW9fvb3t7O2pra1FQUKC+53A4UFBQgOrq6qD7ra6u9hsPAIWFher4w4cPo76+3m9MWloa8vLy1DHV1dVIT0/HiBEj1DEFBQVwOByoqakJetwXX3wRl156Ka6//vqgnyuKgqqqKnz88ce44YYb1Pc9Hg8mT56Mhx56CJdffnnQbbUQs4LDjAuViBuEL6FumkZdDbOFRuCxWTGy+mwoZM3nCIWMLkc0Jo9Seaz8tLQlcHkBQE5ODtLS0tTXokWLuhzvxIkTcLvdyMjI8Hs/IyMD9fX1QedYX18fdrz3v5HG9O7d2+9zp9OJnj17Bj1ua2sr1q5d6+dueDl58iS6d++OhIQEjBkzBsuXL8ePf/xj9fOnn34aTqcTv/3tb4P+PlohGa4RLU96RvI39HQWDYRXngYvLoo/f0M/7ta+yJp3DvUd6UzH07sQXCjCLfJ2YWILvmvTvwBcuAXewi3uluhqR1sr34XfIi3q5knywHHWmhutWQu6dSY74DxjnZtDyMnRo0eRmpqq/pyYaN88mr/97W84deoUpk6d2uWzHj16oK6uDqdPn0ZVVRVKS0txySWX4MYbb0RtbS2ee+457NmzB3FxxsQ7yfUgmJHwxvqUa7QUNhI8XY2L4k/7iQ3ve3rmxAqr2yEqn8NoqSz3MlmBLkc4ZEkelRkqj5WX1NRUv1cwwdGrVy/Ex8ejoaHB7/2GhgZkZmYG3a83nyLUeO9/I40JTErt7OxEY2Nj0OO++OKL+OlPf9rFNQHOhYB++MMfIjc3Fw888ADuuOMO1c35n//5Hxw7dgwXX3wxnE4nnE4nvvzySzzwwAPo27dv0N8vFCQ4jCC44RcLRkMoRvGKjHDCItLnwTAjoVTWfI5QiFhN1kguhx2SR62oVrFbHgeVxuojISEBw4cPR1VVlfqex+NBVVUV8vPzg26Tn5/vNx4Atm3bpo7v168fMjMz/cY0NzejpqZGHZOfn4+mpibU1taqY7Zv3w6Px4O8vDy/fR8+fBhvv/120HBKMDwej5qvMnnyZHzwwQeoq6tTX9nZ2XjooYeCVrKEw17fCAkw0qFRK0aTRbXCS2jo2YYlxAKcm6ueEAsATWGWXt1O40RH8DnJGFrRhcsNtMbz259GjIZFzAqraKUzyQHnWWvCL56URDhaGIRqShLQclbchAgAQGlpKaZOnYoRI0bgmmuuwdKlS9HS0oLi4mIAwJQpU9CnTx/VNZg1axZGjx6NZ599FmPGjMH69evx/vvvY9WqVQCAuLg4zJ49GwsXLsSAAQPQr18/zJ8/H9nZ2SgqKgIADBo0CLfeeiumT5+OiooKdHR0oKSkBBMmTOhStrp69WpkZWXhJz/5SZe5L1q0CCNGjED//v3R1taGrVu34s9//jOef/55AMCFF16ICy+80G+bbt26ITMzE5dddhnTeSLBEQYjT24sCaOs9rvRcIpVQiPY9nbJ7dArOtITW9HUxv8xPlwuR3yiG+42dmERKZcjHO5EIF5nLmynC3AaFBSdrjg4W8M/DGgZQxB6GD9+PI4fP46ysjLU19cjNzcXlZWVavjiyJEjcDjOO2gjR47EunXrMG/ePMydOxcDBgzA5s2bccUVV6hjHn74YbS0tGDGjBloamrCqFGjUFlZCZfr/PVk7dq1KCkpwc033wyHw4Fx48Zh2bJlfnPzeDxYs2YN7rrrLsTHd70utLS04N5778VXX32FpKQkDBw4EK+++irGjx/P+zQhTlEU6b+Bzc3NSEtLw80XTIXTkeBv/X1fa+5tiuONh3rtTG9Wudcu9VqrvharN87stXe9P/sKDq+l7Odw+IRUAq3sQMERboXYSAmjkfI3WMIHRsWGUaERDFa3A2AXHV4iCY9QLgeAkIIDQESXI5LoCOVyAAjrcoQSHWEFRxiXI5LgCJc8GklwhHMptAiOSC6HFjGhaYyG5FEWhyP+bKe2cWc6NI1jcjgAJodDOXNG+9jT2p3XTqUD29s34uTJk35JmDzx3icuXTsH8cnGkjvdZ9rwyaSnhM43FqEcDr1wbPgVDl7JojySQkWIDb371fu7RBJnokplI8G7VFZULkc4rM7lMDN5lMpjCYId+tZowGjzI5aEUdYblxZ3g4fQECU2jBxDVEKpCNFhpGqFezMwA2JZ9nbnkbB7RQtALc4J+0KCQwCiG35pxairYYbQCHZMVszsUBoJI6IjnMthZpmsrC6HWYLFqrVV7FYeG9ddf7I0EZuQ4LAYlidfrbkbsoRPLnK0qS/WY5vhdoQTHVaVyurBbIErs8vBa0E3ntipPJZKYwmRkOBgINSTH0u8XERba19kCJ8EExmsosM7F1ZkEB1G8jli3eWQBS0uh23yOGgRN0IS7CO9bYqRhl+sT9JWuxqRRMVFjjYc97DdjfT27AD0V7JoRVSpbLjeHKEQUSYbDiPtzsP11YhUIqulJ0cslL8y9+OIIdpbu8HhMNa/xtNKbe5FoOuKsXLlSvTt2xculwt5eXnYvXt32PFNTU2YOXMmsrKykJiYiEsvvRRbt27VNeFoxsiTsdW5GlodDLNCLID2c6LX5YiEiCRS7h1rY9TlkDWsYrc8DoJggVlwbNiwAaWlpSgvL8eePXswbNgwFBYWdunp7qW9vR0//vGP8cUXX+D111/Hxx9/jBdeeAF9+vQxPHnLCHORFtnwyxceSY+iwidat2PeRkLRISqfQ0/oTUTL83CIWrreTsmjtgmrEIQEMH9blixZgunTp6O4uBiDBw9GRUUFkpOTsXr16qDjV69ejcbGRmzevBnXXXcd+vbti9GjR2PYsGGGJy8DIi7kweC9WJtVQsPoPswsn/XFqv4cwTCzTFaUyyFD8qjdkaE8lipVCBaYBEd7eztqa2tRUFBwfgcOBwoKClBdXR10mzfeeAP5+fmYOXMmMjIycMUVV+DJJ5+E2x36ItfW1obm5ma/VzRgpOEXT8wKn4jcnwi3Q7ZSWVmagVmBGYu6ySpKKKxCRCtMguPEiRNwu91dlrfNyMhAfX190G0+//xzvP7663C73di6dSvmz5+PZ599FgsXLgx5nEWLFiEtLU195eTksEyTCRmaERmF5end7L4asYJIgSgzosIqsmBVTw6CiEaEf5s8Hg969+6NVatWYfjw4Rg/fjweffRRVFRUhNzmkUcewcmTJ9XX0aNHgw+UvNxL9JL0rPDqrSECWVyOcPAOaxnBzORRgiAIHjCVxfbq1Qvx8fFoaGjwe7+hoQGZmZlBt8nKykK3bt38VqkbNGgQ6uvr0d7ejoSEruVLiYmJSEy0Pj5pJiJi/bwRJTZkgsfKstGGkVVkw2Fk2XnZlqwnCCIyTA5HQkIChg8fjqqqKvU9j8eDqqoq5OfnB93muuuuw6effgqP57z1+sknnyArKyuo2LAKHjFjszo+6sk1sEMoxQ4uhwh4dx+VpbV+rECVKgShDeZvSmlpKV544QW8/PLLOHjwIH7zm9+gpaUFxcXFAIApU6bgkUceUcf/5je/QWNjI2bNmoVPPvkEb775Jp588knMnDmT2y8hQ7Y2KyJaXIskFtwNkdjBwSLEYqcW5wQhAuZvwPjx43H8+HGUlZWhvr4eubm5qKysVBNJjxw5AofjvI7JycnBW2+9hfvvvx9Dhw5Fnz59MGvWLPzud7/j91tEOUZzB+zgbkQD4TqP6kVP19FwiOg6Gg53IhBPWpUgCOhsbV5SUoKSkpKgn+3YsaPLe/n5+fjXv/6l51CmEw1VK7wx090ws/25ntbnvbqdxokOtmMR1hALLc4Jwk5Elccnc/266EXbQiFbzw07QImjchBpXRWCCIa7PR5KvDEXz9NungsYS1C2k4TEak8HL2Ylj8qEnpweKo0lNCN5CwEiNiDBEQIti0/J0LkxXMWFDO7GRfHyVCJZjZ0TR8O1OLdz8y9Zu40SRDQSlYJDhmxwlqdPGW9EPMWGHtFhV5dDFnfKLqWxRnKmKN+KIOxFVAqOaEZrDw6rb752cDbCuUOilqzXg1X5PwRBEDwhwREBO9vFehGRKBpLLofsyBAKtBO0ngpB8CGqv0mydgAU3fTL6puuHdwNGbFbMziCIAgW5LwjEypm2/ciy2DNEiJWC65w8M7XkW2BwGBoScC2O7I+3BCETNC3RCPhsvRlwsjNVsaqFNF9QPTmcYRDlsRRXYQpjbVjpQpVobATl5xs9RSIKIUEh2TY+WalRWxES7iFEkfNgceiitGMHdeRImIXEhycsKIMMfDpXOZQghEoeVQfdimNNYLdSmNlKNknCKsgwcGKjm6Mdng6NRq6YHEuosXl0Eu4PA5KHCUIIlohwWExMjb9khGRLkcs5nHIVBprN5eCkJzWeD4vgjskOGyE3ptfJMx0N4xsIxtm53GEwg6VKjIjY2KpzAtREoReSHAQhjBTOFAuh0kIWMQtFkpjCYIIDwkOiTHjCToWl5+3KzLkAtmxNJYgCDkgwRElWPEkb9TdkKndebg8DhGYlbsTC5UqkZAxZEIQsQgJDp3IlHSnF3I3tKF3ITe9iaNUqSIftJ4KQRiHvkUCiIUkPl65G2a5HNFONP/NRWr+JUuVC7U3J4jw0DfEZHg9vRoNARi5aduxwoSSR7sSDS4dQRD2gQSHRMjeu0EUsrgcZudx6EGGxFERyOJSEIReVq5cib59+8LlciEvLw+7d+8OO37jxo0YOHAgXC4XhgwZgq1bt/p9rigKysrKkJWVhaSkJBQUFODQoUN+YxobGzFp0iSkpqYiPT0d06ZNw+nT5x+uvvjiC8TFxXV5/etf//Lbz9KlS3HZZZchKSkJOTk5uP/++9Haev7h2O12Y/78+ejXrx+SkpLQv39/LFiwAIrCtsYYCY4YJNbcDS9GXA4ReRxSJ44KWMRN5tJYMxNLLWtvnpJkzXFjgA0bNqC0tBTl5eXYs2cPhg0bhsLCQhw7dizo+F27dmHixImYNm0a9u7di6KiIhQVFWH//v3qmMWLF2PZsmWoqKhATU0NUlJSUFhY6CcEJk2ahAMHDmDbtm3YsmULdu7ciRkzZnQ53j/+8Q98++236mv48OHqZ+vWrcOcOXNQXl6OgwcP4qWXXsKGDRswd+5cdczTTz+N559/HitWrMDBgwfx9NNPY/HixVi+fDnTeSLBYRNENf2SBTsLGRFQ4ihB2IclS5Zg+vTpKC4uxuDBg1FRUYHk5GSsXr066PjnnnsOt956Kx566CEMGjQICxYswFVXXYUVK1YAOOduLF26FPPmzcPtt9+OoUOH4pVXXsE333yDzZs3AwAOHjyIyspKvPjii8jLy8OoUaOwfPlyrF+/Ht98843f8S688EJkZmaqr27dzjeW27VrF6677jrceeed6Nu3L2655RZMnDjRz6HZtWsXbr/9dowZMwZ9+/bFHXfcgVtuuSWiixMICY4YI5rcDWoERhCESJqbm/1ebW1drznt7e2ora1FQUGB+p7D4UBBQQGqq6uD7re6utpvPAAUFhaq4w8fPoz6+nq/MWlpacjLy1PHVFdXIz09HSNGjFDHFBQUwOFwoKamxm/fP/vZz9C7d2+MGjUKb7zxht9nI0eORG1trSoePv/8c2zduhW33Xab35iqqip88sknAIB///vfeOedd/CTn/wkxJkLDi1dSEjDRfEJOO6O3moL0aQktqOlTS5RKAudrjg4W9nizVbjTu6G+DMdVk/Ddjja4+BwGAyRtZ/bPicnx+/t8vJyPPbYY37vnThxAm63GxkZGX7vZ2Rk4KOPPgq6+/r6+qDj6+vr1c+974Ub07t3b7/PnU4nevbsqY7p3r07nn32WVx33XVwOBz461//iqKiImzevBk/+9nPAAB33nknTpw4gVGjRkFRFHR2duKee+7xC6nMmTMHzc3NGDhwIOLj4+F2u/HEE09g0qRJQX+/UJDg4IDVzZW0PrVHYznpRY42HPewJQdcFH8ax93dg36W2a0J9R3pQT/r5TyFE509gn/W7TROdATfp8zEJ7rhbqOFqggiGEePHkVqaqr6c2KixIlIQejVqxdKS0vVn6+++mp88803+M///E9VcOzYsQNPPvkk/vCHPyAvLw+ffvopZs2ahQULFmD+/PkAgNdeew1r167FunXrcPnll6Ourg6zZ89GdnY2pk6dqnk+JDgEEy1VBWaFU6LR5ejZrQWNHSlBP7swsQXftQX/LBTdE9twus1eFz6jdLoAZ5i0FrcLiBec9tKZ7IDzDLVvjyVSU1P9BEcwevXqhfj4eDQ0NPi939DQgMzMzKDbZGZmhh3v/W9DQwOysrL8xuTm5qpjApNSOzs70djYGPK4AJCXl4dt27apP8+fPx+TJ0/Gr371KwDAkCFD0NLSghkzZuDRRx+Fw+HAQw89hDlz5mDChAnqmC+//BKLFi1iEhyUw2EhZi5NH43uhpdo/t14YaYLp7dShUpj9eFJiS3xKRsJCQkYPnw4qqqq1Pc8Hg+qqqqQn58fdJv8/Hy/8QCwbds2dXy/fv2QmZnpN6a5uRk1NTXqmPz8fDQ1NaG2tlYds337dng8HuTl5YWcb11dnZ+IOXPmDBwOfykQH3/O9fSWvYYa4/GwCXByOIiImJ0saobLES6sIgvpia1oarPwLuxyA63Bwy2eRAWONlqjJJDOJAecZ8kFiTVKS0sxdepUjBgxAtdccw2WLl2KlpYWFBcXAwCmTJmCPn36YNGiRQCAWbNmYfTo0Xj22WcxZswYrF+/Hu+//z5WrVoFAIiLi8Ps2bOxcOFCDBgwAP369cP8+fORnZ2NoqIiAMCgQYNw6623Yvr06aioqEBHRwdKSkowYcIEZGdnAwBefvllJCQk4MorrwQAbNq0CatXr8aLL76ozn3s2LFYsmQJrrzySjWkMn/+fIwdO1YVHmPHjsUTTzyBiy++GJdffjn27t2LJUuW4O6772Y6TyQ4JIXnSrHkALARLo9DdihxVD92TCwVRVxyMpQzZ6yehm0YP348jh8/jrKyMtTX1yM3NxeVlZVq0ueRI0f8HIKRI0di3bp1mDdvHubOnYsBAwZg8+bNuOKKK9QxDz/8sBraaGpqwqhRo1BZWQmX6/xDyNq1a1FSUoKbb74ZDocD48aNw7Jly/zmtmDBAnz55ZdwOp0YOHAgNmzYgDvuuEP9fN68eYiLi8O8efPw9ddf46KLLlIFhpfly5dj/vz5uPfee3Hs2DFkZ2fj17/+NcrKypjOU5zC2irMApqbm5GWloabL5gKpyMBccnJ5z74vpGN11J0J5+rLfZtrONd38C7+JK3wY/3v14b17teg/dnr/XrtYfVZkffN0TybQsdaFf7rmsRmMPh218hMKQS2CTKV3QE9uHw7YqpJWnUbuWwrA4Ha+LouWMEdzjCiY1QSaMAwiaNhsrhABA2hyOUwxEqhyOc2GhrDf5Z2KTREA4HgLAOh+Ns6GhtfJg/xXB5GOFyOCJte2778Jc6LWJDSw6HFocj/mxnxDEANFepOFo0fr9bzmobB2gWHMrpyKHhTqUD29s34uTJkxFzIvTivU/84OmFcLiMOYOe1lZ8+bt5Qucbi0RlDofWL7MZhEvuY00WtAqzkzjNOJ7s4RQpCCM2iOBQOIUgQhMVgsOr7q2qWQ/15CgTehwAuxCtv5ul+RsR0OtuEAQRu9j7ysBgD0YzZjytm+VyRFtJrCzIIor1hlMIgrA/9hYcMUS43AGtRKMTwDN3QxR68zeI80TK3zBlDtSDgyAMQVUqJtPU5gq5MFdjR4rwJeqPexJ1J5Aed7cLTSCVwd0QkTDKm1hr+qWFWHZHNCeMxgiOsw44FIPP0q30LC4COqsCiOayRBlEgREoWdQfamtOEIRZkOCIQaIltBItvwcvolnoEgRhf0hwSEw4m97KxlQiXA4qhfVH5goVgiAIPZDgsBirkgbt7g7YZf62TRiNsh4c1EGUIKyHBIdOWGLf0Zjkx9ORkCUvxA4Jo3rgXRJLa6gQBKEHEhwxjF1cgkDsUAorimgUrwRBxAYkOGxEuKdsvTdUI6KDhzMhi7tB8EHvGipEV7h3TqZGiYTFkOCwgHAJgeFi/jJiB8FgN3fDrIRRmUpijfTR4NGDg0eOB62jQhDhIcHBSohkOllaR+vBqtCKHrEiaq5mV/1InTAqGTJ0GSUIwjgkOCLgtYgpUS40MrscPNwNEUvS84Z6cNgHmVazJggzIcERAjvEm3k+lZvtcsjkbhABhCmJJeEdGyhnzlg9BSIKobVUAohvBdwcQugtbQlISdR2U/2uLQUXJgZfQ+VER3f06nba+IQEI3qdFYIqVKyEFm6zD442IN6oLrbBA6cdIYfje6IhTmw0fGCWg2CWu6H1fFjZtVU0ds4t4gU1/SIIOYh5wRHLq0zyRuZcDr2Ey9/QS7iEUTu3NA9XEksQBBGzV4hofeqR3eWQzd0wAiWMEgRBaCdmBYfZsMTfw92stDxxWyk6otHliEbM7sEhKgmbHEqCsA8kOAwgU+MkmQglOmR0N+ySv0EJowRB2B0SHFGK7KEVOyAif8POiCqJDedS2CWZm7qMEkRkSHDYHFmf0APdjGjruyFihVjeCaO6KlSibFl6LURrPhdByAYJDotgubnovbnFosshw7op4XJworWlOVWoaIO6jBKxDF0lOEI9D/zxuhqyJpLK6g6xQBUq4SH3giDkgQRHlGO1y6FXbFhdCkv5G4SVcF+aniAkgAQHQTAiIn8jHLwrVKKlJJYAHC10cgn7QGupSEK49VQCOdHZA72cpzTv+7i7Oy6K178ey3FPIi5ymHdhs9rdMILehl+ydxiVcdE26sFBBCO+DTAsqUnHCYEcDoFQfF1erMjfMDNh1C75RHYQDbRwG0HwgQRHjGB1LofI4/B2N+ySv8Fd0MZgSSxBEOZBgkNSRKzFYRfRITNm52/YAVElsWY0/aIqFoIwD11XipUrV6Jv375wuVzIy8vD7t27NW23fv16xMXFoaioSM9hbY+o9tTRUN4JmOduiDpfIkSi3RNGCUloOWv1DAiCXXBs2LABpaWlKC8vx549ezBs2DAUFhbi2LFjYbf74osv8OCDD+L666/XPVnCOORyWEO0NvyKhJUVKuReEIRcMAuOJUuWYPr06SguLsbgwYNRUVGB5ORkrF69OuQ2brcbkyZNwu9//3tccsklEY/R1taG5uZmv5es0BMjH2TI3QCsyd+QoqU5oRtaR4UgtMEkONrb21FbW4uCgoLzO3A4UFBQgOrq6pDbPf744+jduzemTZum6TiLFi1CWlqa+srJyWGZZlTCM3cgll0OI+GUWM3fsGLRNpHbWoVd2porZ85YPQUiSmESHCdOnIDb7UZGRobf+xkZGaivrw+6zTvvvIOXXnoJL7zwgubjPPLIIzh58qT6Onr0qN/nRr4QXGzWMNn8LE+XVvZekEl02D13AzB//RRZKlSifQ0VKokltMKa27hx40YMHDgQLpcLQ4YMwdatW/0+VxQFZWVlyMrKQlJSEgoKCnDo0CG/MY2NjZg0aRJSU1ORnp6OadOm4fTp4D2XPv30U/To0QPp6ekh5xQqz3LTpk245ZZbcOGFFyIuLg51dXVhf7dQCL1anDp1CpMnT8YLL7yAXr16ad4uMTERqampfi8AUFoMCA0OFw49T3osNwaWG1Og9a/nZnrc3d2Q8DjuSTQkPPRuL0pshAuniHI3wolOPQmj4QSvncJ/sVyhQm3N7QdrbuOuXbswceJETJs2DXv37kVRURGKioqwf/9+dczixYuxbNkyVFRUoKamBikpKSgsLERr6/kvx6RJk3DgwAFs27YNW7Zswc6dOzFjxowux+vo6MDEiRPD5lCGy7NsaWnBqFGj8PTTT7Ocli4wCY5evXohPj4eDQ0Nfu83NDQgMzOzy/jPPvsMX3zxBcaOHQun0wmn04lXXnkFb7zxBpxOJz777DNDk9cCj/hqpKc4lgs5y00k8KlZ1E3PCuGhV2iwzrO+I114FY9ed0Ovw2Wmu2EknEItzcVCbc3lgTW38bnnnsOtt96Khx56CIMGDcKCBQtw1VVXYcWKFQDOuRtLly7FvHnzcPvtt2Po0KF45ZVX8M0332Dz5s0AgIMHD6KyshIvvvgi8vLyMGrUKCxfvhzr16/HN99843e8efPmYeDAgfjlL38ZdD6R8iwnT56MsrIyv3QKPTAJjoSEBAwfPhxVVVXqex6PB1VVVcjPz+8yfuDAgdi3bx/q6urU189+9jP86Ec/Ql1dndDcDB7xUr0XTCNhFbNdDl/MCrPI5Gp40etuiCiFBaK/HFZ0/oZZ7gUljEY3gcULbW1dbwp6churq6u73LwLCwvV8YcPH0Z9fb3fmLS0NOTl5aljqqurkZ6ejhEjRqhjCgoK4HA4UFNTo763fft2bNy4EStXrgz5e7LmWeqFeS2V0tJSTJ06FSNGjMA111yDpUuXoqWlBcXFxQCAKVOmoE+fPli0aBFcLheuuOIKv+298aPA93kRf6YD7uRuxvbRCrg5ple0tCUgJVHfqqmNHSno2U3bGivA+ZtsZrcmXcfz3tj1rr3iFRPB1l7RG36xUmwYQYS7EQ7e1SmR3A07N/zSAuVv2BNnKxBvUG/GfX/5CnwoLi8vx2OPPeb3Xrjcxo8++ijo/uvr68PmQnr/G2lM7969/T53Op3o2bOnOua7777DXXfdhVdffVVNTQjEm2epNy+DBWbBMX78eBw/fhxlZWWor69Hbm4uKisr1RNz5MgROBzmJ5I5WtrgSTH2VOhsBTp13gfcbfGIT3RrGnu6LRHdE8/fkJvaXEhP1HaVPdHRHb26RRYDMgkPWYUGEFls6HU3jCSKhnM39IRTwrobgtqZiwqn2LE6hbAvR48e9btRJybaq0Jv+vTpuPPOO3HDDTcE/VxvnqVedK0WW1JSgpKSkqCf7dixI+y2a9as0XPI0LScBVKSDO1Cl6PRGg+4QguMttYEJLrOuxosLgfPlWPrO9J1iw6An/DQe1wWzBQbRuCdKAqY724YQQbRIGvCKCEXvkULoWDNbQSAzMzMsOO9/21oaEBWVpbfmNzcXHVMYFJqZ2cnGhsb1e23b9+ON954A8888wyAc7khHo8HTqcTq1atwlVXXaXmWXrxeM45e06nEx9//DH69+8f9vdnIbpr2iJg5UXHSPIoKzwSJ81c/l22ktdgmJ0oCshTCgvIHU6RMX/DLj04CHZYcxsBID8/3288AGzbtk0d369fP2RmZvqNaW5uRk1NjTomPz8fTU1NqK2tVcds374dHo8HeXl5AM7lefjmUD7++OPo0aMH6urq8POf/9z0PEtdDofdcbYq6HTpe4JztMXBk6jtghbocoRDRFglEKvDLFr3z4JeoSEqlGIEEWvtiEgWjSQ2KJyiHyElsbSOinBYchsBYNasWRg9ejSeffZZjBkzBuvXr8f777+PVatWAQDi4uIwe/ZsLFy4EAMGDEC/fv0wf/58ZGdnqz0yBg0ahFtvvRXTp09HRUUFOjo6UFJSggkTJiA7O1sd48v7778Ph8Phl0OpJc+ysbERR44cUatfPv74YwDnXJZQLk4wYkpwOM940Jms78nMcdYBT1L4J5pIeRxWhVUCkVF4mCU2tCSIGgml6HU3IomNcO6GrnCKoFLYSNhFNFDCKMECa27jyJEjsW7dOsybNw9z587FgAEDsHnzZr+b/MMPP4yWlhbMmDEDTU1NGDVqFCorK+Fynb+OrF27FiUlJbj55pvhcDgwbtw4LFu2jPvv98Ybb6jiCQAmTJgAIHgSbTjiFEWRPpjZ3NyMtLQ03JTwC3TrkQ4AiEtOPj/g+xwOb9Kot0rFnXReT3UmnfvH9goOr8Ph/a83h8ObNOqb0+H+/l7gFRx+DkdAHkeg4Ah0OAIFh2/yaKDDESg4fKtVAh0OFsERiJEcD8CY8NAbqrFKbBhJFNUrOCKFUkIJDr3JouEEh5ZQSjiHI5zgiBRO4VUOq2mMBsEhIqSi1eFg6sHB6HCwdnJWTkd+MOpUOrC9fSNOnjwZMSdCL977xOB7n0R8orEqMHdbKz78w1yh841FojqHw0jc1IonMZb4fuCN0UhppxX5HXqbjenNR7FiUTZfRCSKAvIli1I4pSuUv0EQ54hKwWEkDsq7B0DgDcFIq3NRuQOA8cRSFgFhtquhVWyIcjekShQFLFs3RbRoMDMJnBp+EQQ7USk49MDrYsiapGfk6Zany+FFpPAw0kJdtKthRGwYQZS7oSdZ1Ep3Q5ZmXwDlbxCEKGJecBh+KuLcOMlIq3OA7Yk+HLyFhxGhIbvYsCJRVDc2XBWWZytzs0vhLQ+nUIUKIRExKzjMvPBECquI6MnBK2eBR36HrCEUwHhzLxFLzwP6E0UBa9ZNieRu2DkHgyAIPsRUWSxgrDTWS6ReHCxtzoPBqyeH98ZrpIIFMF5Gq/d4rLCKLC1iw0goxS6JooDxyhQjGA2nmO1aiMrfoGXp+RDfZnwtFehb+oqIQMw6HHoIeeEVtB6FF6NP0bKEWbQeQw8iqlBEhVIiYSSUEk3rpvCGVzmsVkSFU2hZesKukODQgIgLqhlhlUBkFh5GcjX0/E4i8zYiIaLnhhFEuxuil6InCMIekOAIA8+LHeuNwkhPjojjOQoPHvsw09U40dFduNiwKpRiR3fDrHAKLdZGENZDgiMIPEr0eCfuiUhO5CE8jAgGI4JFhKsBiO11EgkhVSkGsNrdkBGt+Rus4RRZ1lBh7TJKECyQ4PDB0MWRwxOkkbCKkWoLs4WHUVdDlNjQgmzty72IaGMeCdG5G5G+j7zdDeq/QRBiibkqlWDoWT2WZdXYUERazI2lWoUHPKpawlW0mO1oAGxCQ2QoRSRWrAirhWhzNwiCMAY5HCZiNOEvUqtzXk/xvByPcD+zzkfXdhKJDcsSRSV2N4yELmXOyZAinEIQEkIOByNalqn3oqcfx+m2RL8VZFldjnB9OVgx6ngYTSo1InrMFBuRsCxRVBBaxIZRd4OXO8IznELrpxCEMcjh0IimJzodeRysiYJabn5aKjFY4FXVYsbxWH93HmJDyp4bgNAl6I0Sre6GaKgHB2FnSHBEINyTlh5LmncfhXA3TBHCQzRmuRoAn4oUI6GUSIgKpRjBDHeDF1Yki0oTTjGpQkU53cK8DRG7UEglBM5WoNOaHMAuGE0e9d6IeYRaeLVLD7VfXdvqEFVaxIbIvA0gvLshMpQi0t3QttCa8X0QBGE/SHAIhkceRyDftaXgwkT2JwtZhYcZSaG+mCE2ImFVKMUIVieKntvemlJYyt+wD85WBfEeY2G3uPbYDduJhEIqvDEhjyMQ1tAAz1CLoXwLg7kaejBLbMi2OJsXq92NaEZ0OIXyNwi7Qw6HQfT042hrTUCiy/rlCK1yPMwOn3jhITa0YGkoxebuBg/REstJpQQhM+RwmACP0sXAp+pIPTlY4O14GPk87LYGXA1eYkNk3oZhIogNI303IsFDKPDqLKoVCqcQhLmQw6GDiL04WuMBF1veRiCR8jhEwMvxCOZ2yOxqaEVkRQpgXaIoED6cIkPuhuzYPZxCFSqEGZDgYCC+DXAHuaeY0ebcTHgLD6Pz0AOr0BBdkQLImygqQ+6GjI2+AJu6GzpKYgnCDCikYhE8EgODhVV4PtHz7uPBemy9mC02tGBV+3LA3i3Mz++D8jIIwu6Q4DAJEXkcoRAhPMzCiMjR83ubUZEidCXYCBgJpWghmt0NUcgWTiEIs6CQiigkyOPw3nx7djMea+VZ0RJu/3rQK66srkgBrK1KiSQ2Yt3d0BpOYc3fIIhYJWYcjmAXDyMXssCLNY8251pi/JGqVYLhffLn4Xp4HQhuVS0mOxpeZKhIMRxWsziUQu4GQRAs2NbhUM6cQVxyst97jpY2eFKMVQrEtwJuV+T3gtEleTTA5dDSdTQweVSLy8HSedT3Bm3U+QgUCizuh1WVJ7wWYzNDbNg9lELuRpDxosMplDBKSIxtBYdW4s92wp3k/2s6z3jQmex/wXW2Kuh0sT0RalqqPoLo0NIEzOiS9aEIvHHzFCDBxIfMIsOLbcSGzpVgAXlCKWa7GwRBWIvtBIdyugVx3QNuIC1ngZQk7scKtoBbqNJYL1pKZCOJDi0lsoGiQ+/6Kr6IdD/0YpbQAKJDbESCh9gwK5RCjb4IPcS3KXAaXEsFHSRiRWA7waGV+DMdcCd383vPedaDzqTIzobWEAoQ3OWIFFoJRiTRoTW04oWn+AD4JJ7qPTYLehNBzeizwaUaRfISWMDcUIqV7gYlixIEG7YWHKLyOIIRToTwCK0EI5LoCBdaCbzxyuR+aDkGK6KEBhA9YsNo3gZgz0RRwJ7uBuVvENGGrQWHVkTmcYSCR2hFC1rzOWR2P2QVGoBEYiMCovM2tIoEs9wNq/M29LgbrAmjBBFtRI/gCJLHESysworWPA6ZQiuRsNr9MFqea6R/BkvXUB4LsXETGxbmbfAUG1asBkulsAQhB9EjODQiIo+DCZNDK1oQ5X74ig8ePUDMEhqARGJDg9AQmbfBK/yhdV9WOhfU6IsgxBKVgsOKPA5Ao8sRBJlKZXm6H3YTGQC/RdhkERtG8jZYxEashFL0Qv03CCJKBUcweOdxRCqP9UVEaCUY3hssD+Hhhaf7ofe4rIgSGoCJDb1MEBvh3A3eYsOKUArAP1mUcjcIQj9R39qcx5ed5ekt1EW+y80h4Iai5SYUeLMLdZNsanN1efHgu7YU9SUCI/vX+7uebkuUS2y0xhvK1/BiRnMvgI+zcW4//J0Lyt3QhnLmjNVTiApWrlyJvn37wuVyIS8vD7t37w47fuPGjRg4cCBcLheGDBmCrVu3+n2uKArKysqQlZWFpKQkFBQU4NChQ35jGhsbMWnSJKSmpiI9PR3Tpk3D6dPnmy7u2LEDt99+O7KyspCSkoLc3FysXbvWbx8dHR14/PHH0b9/f7hcLgwbNgyVlZUh5/3UU08hLi4Os2fP1nhmzhP1giMYWtdVifRUFuqirdnGjiA6tNzAtN4seQsQX3GgV4Dw2IeR30fruQPkaujlaIuTIm9Dq9iQPVFUZCksuRuxwYYNG1BaWory8nLs2bMHw4YNQ2FhIY4dOxZ0/K5duzBx4kRMmzYNe/fuRVFREYqKirB//351zOLFi7Fs2TJUVFSgpqYGKSkpKCwsRGvr+S/UpEmTcODAAWzbtg1btmzBzp07MWPGDL/jDB06FH/961/xwQcfoLi4GFOmTMGWLVvUMfPmzcMf//hHLF++HB9++CHuuece/PznP8fevXu7zPu9997DH//4RwwdOlTXeYpTFEX6oGhzczPS0tJwU8Iv4Izr5tdp1K8Ph0+Vim8Oh7dSxTek4k0c9Q2peEMpviEVb86Gb6WKbx5HqLBKqL4cXfI5goRWApNIA/M5AkMrRqtWAL5hmHChFx7uiBHBxCIyALnyNQBtCaLhBK+MYoN37garsyHLuilezM7f0OtwKKe1h1g7lQ5sb9+IkydPIjU1VdfxIuG9T4y4YyGc3Yw9VHV2tOL91+dpnm9eXh6uvvpqrFixAgDg8XiQk5OD++67D3PmzOkyfvz48WhpafG78V977bXIzc1FRUUFFEVBdnY2HnjgATz44IMAgJMnTyIjIwNr1qzBhAkTcPDgQQwePBjvvfceRowYAQCorKzEbbfdhq+++grZ2dlB5zpmzBhkZGRg9erVAIDs7Gw8+uijmDlzpjpm3LhxSEpKwquvvqq+d/r0aVx11VX4wx/+gIULFyI3NxdLly6NeG58iSmHI9gFI9jFKdjFLdQFltXliBRaCUakVWVZb6LB4OmABDoXvEIxRt0MVkdDJrGh1dUwI0mUVxjl3L6sFRuxDoVTwtPc3Oz3amvresFvb29HbW0tCgoK1PccDgcKCgpQXV0ddL/V1dV+4wGgsLBQHX/48GHU19f7jUlLS0NeXp46prq6Gunp6arYAICCggI4HA7U1NSE/J1OnjyJnj17qj+3tbXB5fK/riYlJeGdd97xe2/mzJkYM2ZMl3mzEBNJozz6cYTdP0MCaVB0lMoGwqM/hy++N3ae7oceZHM0vMhSiaKOk2RRNvV4NlgtVjZ3gzCO84wHzm4GhWfHue1zcnL83i4vL8djjz3m996JEyfgdruRkZHh935GRgY++uijoLuvr68POr6+vl793PteuDG9e/f2+9zpdKJnz57qmEBee+01NSzipbCwEEuWLMENN9yA/v37o6qqCps2bYLbff4etH79euzZswfvvfde0P1qJSYERzDM6McRquW5qAXevDdXnsIDMF98GHVY9Dg+LEID4CA2OIZQAPHNvVhdDZkXZxONXcQGuRuROXr0qF9IJTGRf7sFs3j77bdRXFyMF154AZdffrn6/nPPPYfp06dj4MCBiIuLQ//+/VFcXKyGXI4ePYpZs2Zh27ZtXZwQVmIqpMKLYBdT3gmkwYgUWvHiDR+whhG0IKL6JXC/etH7+7K6GmaIDZbEUDuKDS2IDqXI2OiL+m/IRWpqqt8rmODo1asX4uPj0dDQ4Pd+Q0MDMjMzg+43MzMz7HjvfyONCUxK7ezsRGNjY5fj/vOf/8TYsWPxX//1X5gyZYrfZxdddBE2b96MlpYWfPnll/joo4/QvXt3XHLJJQCA2tpaHDt2DFdddRWcTiecTif++c9/YtmyZXA6nX5OSCRiTnCIyOPQQ9CbCYdS2WAEChCeIkSvUOAlXIz8TlrzNABtQgPgJza04DjrMLwgm1Vig6e7IVJssGIXd4PgR0JCAoYPH46qqir1PY/Hg6qqKuTn5wfdJj8/3288AGzbtk0d369fP2RmZvqNaW5uRk1NjTomPz8fTU1NqK2tVcds374dHo8HeXl56ns7duzAmDFj8PTTT/tVsATicrnQp08fdHZ24q9//Stuv/12AMDNN9+Mffv2oa6uTn2NGDECkyZNQl1dHeLjtZfxx2xIRQShcjlEhlb0EniD5hGGCRQOgeEXno6IEdHEM0/DF7PFhhaM5G1YKTY0L08vOIzC4m4YFRvM7oYBKJzCl9LSUkydOhUjRozANddcg6VLl6KlpQXFxcUAgClTpqBPnz5YtGgRAGDWrFkYPXo0nn32WYwZMwbr16/H+++/j1WrVgGA2udi4cKFGDBgAPr164f58+cjOzsbRUVFAIBBgwbh1ltvxfTp01FRUYGOjg6UlJRgwoQJaoXK22+/jZ/+9KeYNWsWxo0bp+Z2JCQkqImjNTU1+Prrr5Gbm4uvv/4ajz32GDweDx5++GEAQI8ePXDFFVf4/b4pKSm48MILu7wfiZgWHEbyOELldvBOIA2Gni6kkQh2AzcqQngKDMA8kQFwFhqA4aXl/cZyEhu8Qh8811sBxIsNEe4GORuxzfjx43H8+HGUlZWhvr4eubm5qKysVJM+jxw5Aofj/Pd25MiRWLduHebNm4e5c+diwIAB2Lx5s98N/OGHH0ZLSwtmzJiBpqYmjBo1CpWVlX55FGvXrkVJSQluvvlmOBwOjBs3DsuWLVM/f/nll3HmzBksWrRIFTsAMHr0aOzYsQMA0Nrainnz5uHzzz9H9+7dcdttt+HPf/4z0tPTuZ+nmOjDAcCvSkVEPw6/YxntzQF0ER2RenMAXftz8IR3IqoWeIR+WIUGIK/YYAmfmJW3wSo2eLkbZokNrQ6HJe6GgfwNHg6HrH04rr3tcS59OP61tUzofGORmHY4fNG6rorvcvWsLkeo0EpQIpTKBgut8HA6QiEiBKPlOHrQIzIAAUIDMF1saAmhkNjQhtRiwwAUTiGsggSHQXgsY68llwPQLjoCESFCfIWBEfHBM4FVKqEBcBEbPF0NQG6xoWk/EoVRCIJggwRHEHj04+CSQKohn0MLgTdi3gKE1f2QQWQA1gkNgK/YMHshNraVZBlKWwU1+dIjNsjdiLAfhnAKQXghwcEBYQmkgK7QSiREuyCB7gfvXiBGRAZgrdAArBEbViSI8hYbMpW/8sCQ2KD+G4QNIcHhg548Dj0YKZMFxJTKinJBrMzJCIRFaADWiA3eIRRAa9dP4/vw35/1XUT1ig0zGn2Z7WwQhAyQ4OCEmQmkweDVn8OL6DAM6/H1wioyADFCA+AnNlhCKLKLDZnCKIC5fTfMJlbCKc5WD5ydBp0to9sTQSHBEQI9eRxCE0g1LPDGW3T4IjoMw0tgeLGT0ACsC6HYQWyIWm4+ELNamBt2NyicQtgUEhwmIKIDKRBadPgiSoAAxl0QniJDj8DwYqXQALSJDd6uBmAPscGKWWJDr7tBoRQiliHBoQGWPA4zE0i1EOxGbIYLEkx8yOBi+GK10AD4ig22hE7tYyPvi008iFqUTXZngws63Y1YCacQckOCwwBGk0cBMS5HJMxwQXiLC8C4wPDCJDQAy1wNwPwQCvv+Yk9sWOZuUCiFsDkkOBjRkqdhdgKpHtHhi5lhGFZ4iQxADqEByBtCYd9f7IkNvURDKIXcDcIouta2XrlyJfr27QuXy4W8vDzs3r075NgXXngB119/PS644AJccMEFKCgoCDveboS6iPPoexDyJsZwI9SDdyl235dZ8D6uuy1efWmmNZ6pzJX3cvIsrgaJDXPFhmWVKRavm0IQPGAWHBs2bEBpaSnKy8uxZ88eDBs2DIWFhTh27FjQ8Tt27MDEiRPx9ttvo7q6Gjk5Objlllvw9ddfG568HQl1M2HpvxAM5qd3RkQKEFECQ5ejYaHQAMSEUKwUG85WJebFBoVSCOIczHe5JUuWYPr06SguLsbgwYNRUVGB5ORkrF69Ouj4tWvX4t5770Vubi4GDhyIF198ER6PB1VVVSGP0dbWhubmZr+XzLC6HCx2OYvLEXizFSlCjLggUrgYvggSGgBbroaV+RqixAYLJDaCYFBsULIoIRNMgqO9vR21tbUoKCg4vwOHAwUFBaiurta0jzNnzqCjowM9e/YMOWbRokVIS0tTXzk5OSzTFE6wizOPrP9QNyeWG1wgZgkQILyQkMbFCERgnoYIV8PKEAqJDTaiIW+DIHjCJDhOnDgBt9uNjIwMv/czMjJQX1+vaR+/+93vkJ2d7SdaAnnkkUdw8uRJ9XX06FGWaQLQ/mXXevHStrR21/dYXQ4m0aEjlyOYCyJKiPASGdzn6nU0JAifRFsIRdf4KBQbXKBQChFlmFql8tRTT2H9+vXYsWMHXK7QpR6JiYlITNTWtEI5cwZxycnnfmg5C6QkBR0Xf6YD7uRu5/7/bCfcSZF/9WD9N9T9hahECVYq672wB47ntqIsYHhV2VA3ciPVL3rhLoB0JtmKCJ0AYipQAOtXe9U1nsSGMChZlJANJsHRq1cvxMfHo6Ghwe/9hoYGZGZmht32mWeewVNPPYV//OMfGDp0KPtMfVBOtyCue0rEcY6WNnhSIgsX51kPOpPO3TCCLeAWChbREWq89+YTKDyY+3NwWso+kHA3f15iRBaB4UUGoQFYKzb0dAKVUWxYhdW5G7FM/NlOxDuNlTwrnTZqBmcjmEIqCQkJGD58uF/CpzcBND8/P+R2ixcvxoIFC1BZWYkRI0bon20I/JS8xi+q1qce34tosAtqqLg6j0TSUPZ8SIvfN1QQ+BKA3vAM9xwMTr+rDKETgD1Xg8RGaMjdIAh5YA6plJaWYurUqRgxYgSuueYaLF26FC0tLSguLgYATJkyBX369MGiRYsAAE8//TTKysqwbt069O3bV8316N69O7p376574npcDt+wSiB6XQ4vwdwL740gWIiFR2Mwrd1IAYS+EZvsihiGs3hiTciVIXTiRYYQCus2Zi3EBlgjNihRlCBCwyw4xo8fj+PHj6OsrAz19fXIzc1FZWWlmkh65MgROBznL8rPP/882tvbcccdd/jtp7y8HI899pix2YciTC6HL3pyOczI6wA4hFi0Eu4GLkCMMGOxwFC3k0hoAHK4GqzbRbPY4Co0JCmFJQje6EoaLSkpQUlJSdDPduzY4ffzF198oecQzPgljwag1eXwRY/LAYQWEjwWe/Pe9AKFh2HREQoTXZGIx9SBkXJidR82FRosx4g2saEXKcQGQUQx0buWig6XwzesEo5wLoe6X5NDLMFurkJECMDPFZHEvQi6L0mSQX2RIYSiK+Ristiw1eqvgVCyKBHFRJXgCOdy+KLH5dAiMrocx4LSWb8xEW7AproiAuApMAB97eVlydNgPYbIJl5+29ogjAJEj7tB4RRCZqJKcHTBx+UIVyKr1+UAoMvpOLe9uNJZrYS7YQtzRwxglYMRDKt7aRg5jlYBYURoAOaJDaOOhjRig9wNIsqJOsFhtsuhNbwCmJvXYZRQN3czhYhMAsOLnYXGueNEh9jgFTaRRmxwgNwNQnaiTnB0QaPL4YtWl0Mdb8DtEF06yxsRQoR3aATgJzC8yCg0WI5lVggFECs2ZMjPkFFsEIQdiH7BEQat7c615nLoTSY9ty2fvI5IiBIlIkQD0/EtFBjqNhIKjXPHMsfVAOwlNqRq7kWlsEQMEHOCQ5TLoW6nwe0QmdcRiUg3ZjNcEh7wFBh6xIXf9iYIDf2VLeLFBqvIULezodggd4Mg9BMbgsPERd20jhGZ12GEcDdyq8SIDO5F0P0IFhp6Rca544kNoegVGer2JDbOQ8miXIk/24n4eGPuleK2PnQXjcSG4AhA9KJu6vY63Q5eeR28CXXj5yFEeIsKX3gJDHV/AoWGEZFx/phiXA2jIkPdj81yNgC5xQaFUwi7EDuCQ0C7cyByoqh3LK8QS6SW6F7MECBetAgRkYLCFyvFhS8iOoJGPqYYV4OX0ACsExtS5WsA5GoQMUnsCI4AeC3qprUhWCSBwiOvw+9zhhuvKHFihsiQRWB4MVtoiFxsjafQAKwRGzyEhsz9NsjdIOxEbAkOzi6HF15uB6+8DlYi3bTNdEsiIZvA8GJm6ETGtU/C7suCHhu8HA2ZFmUjCLsTW4IjAN6Luol0O8LldQB8hEcowt3keYsR3oLCb9+cxIUvZggNLuWqJrsaVnQN5R024SY2BAkNs9wN5XSLKcchop+YFhzh0NPuHODrdvAMsYgilEAIJURECooux+IsMMzqBnruWPw6upolNqxadE26/AxfbO5qkNggeEKCwweeS9fzcDv0hFjCYaYgMVNYqMeUSGB4EblyK899mdFLIxx6xIZIoWHY3bC50ABIbBD8Mad0QGLCXVh8L4J6LqzOVoVLiWKwm5azVZ+dr/UlO7zn7D2fvi8e89N2bO1/J1r2w5qroUdsOM96LBMb8Wc65BUbLWdNERuULCofK1euRN++feFyuZCXl4fdu3eHHb9x40YMHDgQLpcLQ4YMwdatW/0+VxQFZWVlyMrKQlJSEgoKCnDo0CG/MY2NjZg0aRJSU1ORnp6OadOm4fTp0+rnra2tuOuuuzBkyBA4nU4UFRWFndO7774Lp9OJ3Nxcv/f79u2LuLi4Lq+ZM2dGPjE+xLzgCETrhYzlIs0iOkKNDXXz4vEUHup4VgoS0cfnKS6CzVfbHIwJDV+BYUbzLq/I4CU0AH1iQySGxUaUQO4GGxs2bEBpaSnKy8uxZ88eDBs2DIWFhTh27FjQ8bt27cLEiRMxbdo07N27F0VFRSgqKsL+/fvVMYsXL8ayZctQUVGBmpoapKSkoLCwEK2t5y8wkyZNwoEDB7Bt2zZs2bIFO3fuxIwZM9TP3W43kpKS8Nvf/hYFBQVhf4empiZMmTIFN998c5fP3nvvPXz77bfqa9u2bQCAX/ziF0znKU5RFPnWIQ+gubkZaWlpuCnhF3DGnQ95xHVPCTo+7GqxIapUfBuB+YZVAqtVguVysDQE0xJmCTcuVFgkWIjFKiKFbqxwUHgLMyuqTIxu67cfjWKDp7jwJapCKCYLDdHuhgix0al0YHv7Rpw8eRKpqanc9w/43Cdy58AZbyybvdPdhu11T2meb15eHq6++mqsWLECAODxeJCTk4P77rsPc+bM6TJ+/PjxaGlpwZYtW9T3rr32WuTm5qKiogKKoiA7OxsPPPAAHnzwQQDAyZMnkZGRgTVr1mDChAk4ePAgBg8ejPfeew8jRowAAFRWVuK2227DV199hezsbL9j3nXXXWhqasLmzZuD/g4TJkzAgAEDEB8fj82bN6Ouri7k7zt79mxs2bIFhw4dQlyc9nW0yOEIgu+FTcuF0Uy3I9QTNK8ndR5Y7Y4AcjgYXeek04kw4GJ02RdDCIW32Ig/26m+mLcVIDYcLW3qizgHORv+NDc3+73a2rr+rbS3t6O2ttbPQXA4HCgoKEB1dXXQ/VZXV3dxHAoLC9Xxhw8fRn19vd+YtLQ05OXlqWOqq6uRnp6uig0AKCgogMPhQE1NDdPv+ac//Qmff/45ysvLI45tb2/Hq6++irvvvptJbACUNMpMqIoV70Vci9vBo5IlXBULCzI5I3oQIbL4J58yJnByTB4F9IVOeCFTTw0vdu6tIdLdiBax4WhpgyPe4D7c5/5GcnJy/N4vLy/HY4895vfeiRMn4Ha7kZGR4fd+RkYGPvroo6D7r6+vDzq+vr5e/dz7XrgxvXv39vvc6XSiZ8+e6hgtHDp0CHPmzMH//M//wOmMLAk2b96MpqYm3HXXXZqPoc6PeYsYQc+ibgDbmisslSwsooMFlhu22eLELMfGaoGhd5uQ+zJQ3spDbMjaklzmjqFaILFhPkePHvULqSQmStQFkQNutxt33nknfv/73+PSSy/VtM1LL72En/zkJ11CNlogwaGDSH05RLgdocaZ2X8jkgBgFSRWhYBkEBg8tvXbD4eGXUbEhiwrugZDSMjERLFhx5yNaCE1NTViDkevXr0QHx+PhoYGv/cbGhqQmZkZdJvMzMyw473/bWhoQFZWlt8YbwVJZmZml6TUzs5ONDY2hjxuIKdOncL777+PvXv3oqSkBMC5/BNFUeB0OvH3v/8dN910kzr+yy+/xD/+8Q9s2rRJ0/4DoRwOjei5oIrK7QhEllLWYKWl4V5mIKbCxVh1iO9+dG/7fS6G3rJWv30ZqDzRm5MRdp+c1j+xe36GcuYMiQ0bkJCQgOHDh6Oqqkp9z+PxoKqqCvn5+UG3yc/P9xsPANu2bVPH9+vXD5mZmX5jmpubUVNTo47Jz89HU1MTamtr1THbt2+Hx+NBXl6eprmnpqZi3759qKurU1/33HMPLrvsMtTV1XXZz5/+9Cf07t0bY8aM0bT/QMjhCIPWRd3CIcrtCNcoLBhWdCE1E1GCi3c+hZF9Wrm+iS8iloyXcu2TUAh2N6hluf0oLS3F1KlTMWLECFxzzTVYunQpWlpaUFxcDACYMmUK+vTpg0WLFgEAZs2ahdGjR+PZZ5/FmDFjsH79erz//vtYtWoVACAuLg6zZ8/GwoULMWDAAPTr1w/z589Hdna22ktj0KBBuPXWWzF9+nRUVFSgo6MDJSUlmDBhgl+448MPP0R7ezsaGxtx6tQptfokNzcXDocDV1xxhd/v0rt3b7hcri7vezwe/OlPf8LUqVM15XoEgwQHA4G5HN6LtlbhwTO3Q2v+hxeWG7Js4sRM94Z7wqbB/UWzyABsJjSAqBEbBF/Gjx+P48ePo6ysDPX19cjNzUVlZaWa9HnkyBE4HOev/yNHjsS6deswb948zJ07FwMGDMDmzZv9bvIPP/wwWlpaMGPGDDQ1NWHUqFGorKyEy3X+Ar127VqUlJTg5ptvhsPhwLhx47Bs2TK/ud1222348ssv1Z+vvPJKAOcai7Hwj3/8A0eOHMHdd9/NtJ0v1Ifje3z7cPgS6HCESh7VutYKwL9vB4vw4A2X1WotCgfJ5F6o21u4JHwwYt7NAKKyc6jZ7oaZfTgKBtzPpQ/HPw79l9D5xiLkcDASqmLFTm4HT2TIHdGKbO6F374sXqU1EBnzMryQ0DB4TAqlEBZBgoMzVuV2WCk6ZEGEYyFy/7KJDEDeklYv0ZCj4YXEBhFrkODQQaS+HFa5Haw3RLsJFNGCQvTxZMnHCER2kQGYIDSiqMw15HFJbBAWQ4JDIFZWsmiB5YZqhjgxW1CIPrasAgOQO2TiJZrcDMDahFASG4QM2FpwKKdbgiaOKmfOhE8cDYKjpS1o4mio0ljvBTtSB1LfG0Uk8WGV8NCC3huyd35WiolAZBcX6j5jUGQA0Sc0AKo+IQjA5oID0CE6Ws6GrFRhFR2A/0Vcq/jQKjyAyOLD9+YpY4jE7mGQkMexgcAA5K4yCSSakkF9IbFhMmfOAg6D3yWPfRvGyYztBQdgvehQxwh0PQA28aEVGUVKOKxySuwiMAD5e2YEEq1CAyCxQRC+RIXgAOQRHYA+1wPgG3LRipEbOG+xIlXYxUYCA7BH4qcvprYct0BoAJSzQRCBRI3gAOQSHep4AeKDxfUQiUwCgQURYiLocQQKDMA+ORm+mL62SRSXuIaCxAYhK1ElOAA5RYe6ncX5HrGCWYKiy3EFCwwvdnMzABIaZkFig5CZqBMcgHmiIxxW53sYRVbxYpWYCIZZAgOwp8gALBAaAIkNgpCUqBQcgDmiIxzei7pW4QHwFR9G0XtjNyJUZBITwZC9TDXssUwUGYAgoWFRLkYgsgkNgrALUSs4AOtFB+B/oRchPkQLD1ZkFw1ezHQnvJgpMADzRQZAQsMqyN0g7EBUCw5ADtHhRYT4MNP1sAtWiIlgmC0wAP4iw5KQiBdJhAZAYoMgeBD1ggOQS3R40SM+WEIuIpBB0MgiJoJhhcBQj23H1VhDQUJDMyQ2CDsRE4IDkFN0eBGR7yECmW/2VmClwACiTGQAJDQYIbFB2I2YERwAf9HBSiSRIirfg+CDHftehEMKkQFIJTQAEht2RzlzForDbWwfnnZOsyF8ibk7FU/RwYrvBV6U+NBLNIkWq52HUFgtMAASGeGwg9AASGwQ9iV67jIMWCk6vIgSH3oxcpM2W6zIKigCkUFgeCGhER4SGwQhnpgUHIAcosOLHvEhSnjowS4CQDQyCQxAIpEBkNDgAIkNwu7ErOAA5BIdXrw3CRlcDyI8sgkMgLPIkFQk8MBOQgMgsUFEBzEtOAA5RQegP+TCG7uKGRnFQCBSORBeJBYZdhMJvCCxQUQLMS84AJ2igycRBAyL+OCN3hu3SKFiBzERDCkFhhcSGgRBCIYEx/cwiw6e+F7sJRYfLNhVFPBEaoEBSC0yABIaALkbRHRBgsMHS0WHFx3iQ2bhEUtILzAA6UUGQELDC4kNItogwRGAFKLDi/fmwOB6mIEdBI4tbv5mYQORAdhTaJAoIAjtkOAIglSiA2ByPcyAbuacsIkQEI0dhQZAYoMgWCHBEQLpRIcXycQHwYDNBIZdhYBoSGgQhD5IcIRBWtHhhcSH3NhMYAAkMiJBYkN+lJYzUOKMJa0rCiW9i4AERwTCiQ4ziShwZLi5ySR6ZDgfNoFERmRIaBCEcUhwaCCU6DB1Dt/fFKRwVkJBN3nbQCJDGyQ0CIIfJDg0IoPoAPxvFFKLD0ITdOPXDt38CcLekOBgQBbR4YXEh/0ggcEOCQ2CiA5IcDAim+jwIuuNzEohJOs5IbRBQoMgogsSHDqQVXTICN307Qnd7AmC4A0JDp3IcEEm0UPwRIa/aYIgohcSHDbG9wZB4sOe0E2eIIhYwaFno5UrV6Jv375wuVzIy8vD7t27w47fuHEjBg4cCJfLhSFDhmDr1q26JkuERjndQi8bvgiCiA543xcVRUFZWRmysrKQlJSEgoICHDp0yG9MY2MjJk2ahNTUVKSnp2PatGk4ffq035gPPvgA119/PVwuF3JycrB48WIhc9ECs+DYsGEDSktLUV5ejj179mDYsGEoLCzEsWPHgo7ftWsXJk6ciGnTpmHv3r0oKipCUVER9u/fzzxZgiAIgpANEffFxYsXY9myZaioqEBNTQ1SUlJQWFiI1tZWdcykSZNw4MABbNu2DVu2bMHOnTsxY8YM9fPm5mbccsst+MEPfoDa2lr853/+Jx577DGsWrWK+1y0EKcoisKyQV5eHq6++mqsWLECAODxeJCTk4P77rsPc+bM6TJ+/PjxaGlpwZYtW9T3rr32WuTm5qKioiLoMdra2tDWdn6BsJMnT+Liiy/GDd2K4IzrxjJdgiAIwmI6lQ7s7NiMpqYmpKWlCTlGc3Mz0tLScENCEZwwdp/oRAd2tm/G0aNHkZqaqr6fmJiIxMSuq2Xzvi8qioLs7Gw88MADePDBBwGcuw9mZGRgzZo1mDBhAg4ePIjBgwfjvffew4gRIwAAlZWVuO222/DVV18hOzsbzz//PB599FHU19cjISEBADBnzhxs3rwZH330Ebe5aEZhoK2tTYmPj1f+9re/+b0/ZcoU5Wc/+1nQbXJycpT/+q//8nuvrKxMGTp0aMjjlJeXKwDoRS960YteUfT67LPPWG45TJw9e1bJzMzkNtfu3bt3ea+8vLzLcUXcFz/77DMFgLJ3716/MTfccIPy29/+VlEURXnppZeU9PR0v887OjqU+Ph4ZdOmTYqiKMrkyZOV22+/3W/M9u3bFQBKY2Mjt7lohSlp9MSJE3C73cjIyPB7PyMjQ1VLgdTX1wcdX19fH/I4jzzyCEpLS9Wfm5qa8IMf/ABHjhwRpo6jgebmZuTk5HRR5YQ/dJ4iQ+dIG3SetOF1qXv27CnsGC6XC4cPH0Z7ezuX/SmKgri4OL/3grkbIu6L3v9GGtO7d2+/z51OJ3r27Ok3pl+/fl324f3sggsu4DIXrUhZpRLKtkpLS6MvtQZSU1PpPGmAzlNk6Bxpg86TNhwOXXUKmnG5XHC5XEKPQeiH6V+/V69eiI+PR0NDg9/7DQ0NyMzMDLpNZmYm03iCIAiCsAsi7ove/0YaE5iU2tnZicbGRr8xwfbhewwec9EKk+BISEjA8OHDUVVVpb7n8XhQVVWF/Pz8oNvk5+f7jQeAbdu2hRxPEARBEHZBxH2xX79+yMzM9BvT3NyMmpoadUx+fj6amppQW1urjtm+fTs8Hg/y8vLUMTt37kRHR4ffcS677DJccMEF3OaiGaaMD0VR1q9fryQmJipr1qxRPvzwQ2XGjBlKenq6Ul9fryjKuSSVOXPmqOPfffddxel0Ks8884xy8OBBpby8XOnWrZuyb98+zcdsbW1VysvLldbWVtbpxhR0nrRB5ykydI60QedJG9F+nkTcF5966iklPT1d+e///m/lgw8+UG6//XalX79+ytmzZ9Uxt956q3LllVcqNTU1yjvvvKMMGDBAmThxovp5U1OTkpGRoUyePFnZv3+/sn79eiU5OVn54x//yH0uWmAWHIqiKMuXL1cuvvhiJSEhQbnmmmuUf/3rX+pno0ePVqZOneo3/rXXXlMuvfRSJSEhQbn88suVN998U89hCYIgCEJKeN8XPR6PMn/+fCUjI0NJTExUbr75ZuXjjz/2G/Pdd98pEydOVLp3766kpqYqxcXFyqlTp/zG/Pvf/1ZGjRqlJCYmKn369FGeeuqpLnPnMRctMPfhIAiCIAiCYEVsyjBBEARBEARIcBAEQRAEYQIkOAiCIAiCEA4JDoIgCIIghCON4KAl77XBcp5eeOEFXH/99bjgggtwwQUXoKCgIOJ5jQZY/5a8rF+/HnFxcSgqKhI7QUlgPU9NTU2YOXMmsrKykJiYiEsvvTQmvnes52np0qW47LLLkJSUhJycHNx///3Mq2rajZ07d2Ls2LHIzs5GXFwcNm/eHHGbHTt24KqrrkJiYiJ++MMfYs2aNcLnSVgMc12LANavX68kJCQoq1evVg4cOKBMnz5dSU9PVxoaGoKOf/fdd5X4+Hhl8eLFyocffqjMmzePubeHHWE9T3feeaeycuVKZe/evcrBgweVu+66S0lLS1O++uork2duHqznyMvhw4eVPn36KNdff32XxY6iEdbz1NbWpowYMUK57bbblHfeeUc5fPiwsmPHDqWurs7kmZsL63lau3atkpiYqKxdu1Y5fPiw8tZbbylZWVnK/fffb/LMzWXr1q3Ko48+qmzatEkB0GUhs0A+//xzJTk5WSktLVU+/PBDZfny5Up8fLxSWVlpzoQJS5BCcFxzzTXKzJkz1Z/dbreSnZ2tLFq0KOj4X/7yl8qYMWP83svLy1N+/etfC52n1bCep0A6OzuVHj16KC+//LKoKVqOnnPU2dmpjBw5UnnxxReVqVOnxoTgYD1Pzz//vHLJJZco7e3tZk1RCljP08yZM5WbbrrJ773S0lLluuuuEzpPmdAiOB5++GHl8ssv93tv/PjxSmFhocCZEVZjeUilvb0dtbW1KCgoUN9zOBwoKChAdXV10G2qq6v9xgNAYWFhyPHRgJ7zFMiZM2fQ0dEhdMVGK9F7jh5//HH07t0b06ZNM2OalqPnPL3xxhvIz8/HzJkzkZGRgSuuuAJPPvkk3G63WdM2HT3naeTIkaitrVXDLp9//jm2bt2K2267zZQ524VYvIYTEqwWa9aS93ZHz3kK5He/+x2ys7O7fNGjBT3n6J133sFLL72Euro6E2YoB3rO0+eff47t27dj0qRJ2Lp1Kz799FPce++96OjoQHl5uRnTNh095+nOO+/EiRMnMGrUKCiKgs7OTtxzzz2YO3euGVO2DaGu4c3NzTh79iySkpIsmhkhEssdDsIcnnrqKaxfvx5/+9vfaPnm7zl16hQmT56MF154Ab169bJ6OlLj8XjQu3dvrFq1CsOHD8f48ePx6KOPoqKiwuqpScWOHTvw5JNP4g9/+AP27NmDTZs24c0338SCBQusnhpBWI7lDgctea8NPefJyzPPPIOnnnoK//jHPzB06FCR07QU1nP02Wef4YsvvsDYsWPV9zweDwDA6XTi448/Rv/+/cVO2gL0/C1lZWWhW7duiI+PV98bNGgQ6uvr0d7ejoSEBKFztgI952n+/PmYPHkyfvWrXwEAhgwZgpaWFsyYMQOPPvooHA56xgNCX8NTU1PJ3YhiLP/rpyXvtaHnPAHA4sWLsWDBAlRWVmLEiBFmTNUyWM/RwIEDsW/fPtTV1amvn/3sZ/jRj36Euro65OTkmDl909Dzt3Tdddfh008/VQUZAHzyySfIysqKSrEB6DtPZ86c6SIqvCJNoWWrVGLxGk5AnrJYs5e8tyOs5+mpp55SEhISlNdff1359ttv1VfgaoLRBOs5CiRWqlRYz9ORI0eUHj16KCUlJcrHH3+sbNmyRendu7eycOFCq34FU2A9T+Xl5UqPHj2Uv/zlL8rnn3+u/P3vf1f69++v/PKXv7TqVzCFU6dOKXv37lX27t2rAFCWLFmi7N27V/nyyy8VRVGUOXPmKJMnT1bHe8tiH3roIeXgwYPKypUrqSw2BpBCcCgKLXmvFZbz9IMf/EAB0OVVXl5u/sRNhPVvyZdYERyKwn6edu3apeTl5SmJiYnKJZdcojzxxBNKZ2enybM2H5bz1NHRoTz22GNK//79FZfLpeTk5Cj33nuv8r//+7/mT9xE3n777aDXGu+5mTp1qjJ69Ogu2+Tm5ioJCQnKJZdcovzpT38yfd6EudDy9ARBEARBCMfyHA6CIAiCIKIfEhwEQRAEQQiHBAdBEARBEMIhwUEQBEEQhHBIcBAEQRAEIRwSHARBEARBCIcEB0EQBEEQwiHBQRAEQRCEcEhwEARBEAQhHBIcBEEQBEEIhwQHQRAEQRDC+f/ZthKkitL/gAAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "d2Jdm2_zeta = compute_gradient(dJdm_zeta, m)\n", + "plot_output(d2Jdm2_zeta.riesz_representation(\"L2\"), title=r\"$h^\\sharp$\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/5_optimization.html b/examples/5_optimization.html new file mode 100644 index 0000000..6a9288c --- /dev/null +++ b/examples/5_optimization.html @@ -0,0 +1,317 @@ + + + + + + + Functional minimization — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Functional minimization

+

Gradient-based optimization algorithms attempt to use derivative information, for example as computed using the adjoint method, to accelerate the solution of optimization problems. A typical example is to seek the minimum of an objective cost functional, constrained using one or more partial differential equations. This notebook describes how to solve such partial differential equation constrained optimization problems using tlm_adjoint.

+

This example makes use of tlm_adjoint with the Firedrake backend, and we assume real spaces and a real build of Firedrake throughout. The minimization problem is solved using the Toolkit for Advanced Optimization (TAO).

+
+

Forward problem

+

We first construct a partial differential equation constrained optimization problem where we know the answer – where we know the value of the control which minimizes the objective functional subject to the constraint that the forward problem is solved. To do this we consider the solution of the modified Helmholtz equation in the unit square domain,

+
+\[\alpha^2 \nabla^2 u - u = m \qquad \text{on} ~ \Omega = \left( 0, 1 \right)^2,\]
+

where \(\alpha \in \mathbb{R}\), subject to doubly periodic boundary conditions. We define an objective cost functional

+
+\[J \left( u, m \right) = \frac{1}{2} \int_\Omega \left( u - \tilde{u} \right)^2 + \frac{1}{2} \beta^2 \int_\Omega \left( m - \tilde{m} \right)^2,\]
+

where \(\tilde{u}\) and \(\tilde{m}\) are given functions and \(\beta \ne 0\) is a real scalar. If we set

+
+\[\tilde{m} \left( x, y \right) = -\sin \left( 2 \pi x \right) \sin \left( 2 \pi y \right),\]
+
+\[\tilde{u} \left( x, y \right) = -\frac{1}{1 + 8 \pi^2 \alpha^2} m \left( x, y \right),\]
+

where \(x\) and \(y\) are Cartesian coordinates in \(\mathbb{R}^2\), then \(u = \tilde{u}\) and \(m = \tilde{m}\) will be the minimum of \(J\) where the modified Helmholtz problem is solved.

+

We consider a continuous Galerkin finite element discretization, seeking \(u \in V\) such that

+
+\[\forall \zeta \in V \qquad \alpha^2 \int_\Omega \nabla \zeta \cdot \nabla u + \int_\Omega \zeta u = -\int_\Omega \zeta m,\]
+

where \(V\) is a real continuous \(P_1\) finite element space whose elements satisfy the doubly periodic boundary conditions. We now define \(\tilde{m} \in V\) and \(\tilde{u} \in V\) via interpolation, at mesh vertices, of the functions given above.

+

We first use Firedrake to solve the forward problem. We consider \(\alpha = 0.1\), \(\beta = 0.1\), and an ‘initial guess’ where \(m = -1\).

+
+
[1]:
+
+
+
%matplotlib inline
+
+from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import matplotlib.pyplot as plt
+import numpy as np
+
+alpha = Constant(0.1)
+beta = Constant(0.1)
+
+
+def forward(m):
+    space = m.function_space()
+    X = SpatialCoordinate(space.mesh())
+    test, trial = TestFunction(space), TrialFunction(space)
+
+    m_tilde = Function(space, name="m_tilde").interpolate(
+        -sin(2 * pi * X[0]) * sin(2 * pi * X[1]))
+    u_tilde = Function(space, name="u_tilde").assign(
+        -(1.0 / (1.0 + 8.0 * pi * pi * alpha * alpha)) * m_tilde)
+
+    u = Function(space, name="u")
+    solve(alpha * alpha * inner(grad(trial), grad(test)) * dx + inner(trial, test) * dx
+          == -inner(m, test) * dx,
+          u, solver_parameters={"ksp_type": "cg",
+                                "pc_type": "sor",
+                                "ksp_atol": 1.0e-32,
+                                "ksp_rtol": 1.0e-12})
+
+    J = Functional(name="J")
+    J.addto(0.5 * inner(u - u_tilde, u - u_tilde) * dx)
+    J.addto(0.5 * beta * beta * inner(m - m_tilde, m - m_tilde) * dx)
+    return m_tilde, u, J
+
+
+mesh = PeriodicSquareMesh(20, 20, 1.0)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+
+m_0 = Function(space, name="m_0").interpolate(Constant(-1.0))
+m_tilde, u, J = forward(m_0)
+
+
+
+
+
+

Inverse problem

+

We now seek to solve the inverse problem: to find the \(m \in V\) which minimizes \(J\), subject to the discretized modified Helmholtz problem being solved.

+

In the following we use the Toolkit for Advanced Optimization (TAO) to solve the inverse problem. We use the Limited-Memory Variable Metric (LMVM) approach with an absolute tolerance of \(10^{-10}\). Noting that the adjoint computed derivative is an element of the dual space \(V^*\), we need to define an appropriate dual space inner product. Here we define an inner product using the inverse mass matrix.

+
+
[2]:
+
+
+
def forward_J(m):
+    _, _, J = forward(m)
+    return J
+
+
+M_solver = LinearSolver(assemble(inner(TrialFunction(space), TestFunction(space)) * dx),
+                        solver_parameters={"ksp_type": "cg",
+                                           "pc_type": "sor",
+                                           "ksp_atol": 1.0e-32,
+                                           "ksp_rtol": 1.0e-12})
+
+
+def M_inv_action(x):
+    y = Function(space)
+    M_solver.solve(y, x.copy(deepcopy=True))
+    return y
+
+
+def post_callback(tao):
+    print(f"{tao.getIterationNumber()=}")
+
+
+m = minimize_tao(forward_J, m_0, gatol=1.0e-10, grtol=0.0,
+                 M_inv_action=M_inv_action, post_callback=post_callback)
+m.rename(name="m")
+m_tilde, u, J = forward(m)
+
+
+def plot_output(u, title):
+    r = (u.dat.data_ro.min(), u.dat.data_ro.max())
+    eps = (r[1] - r[0]) * 1.0e-12
+    p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))
+    plt.gca().set_title(title)
+    plt.colorbar(p)
+    plt.gca().set_aspect(1.0)
+
+
+plot_output(u, title="u")
+plot_output(m, title="m")
+
+
+
+
+
+
+
+
+tao.getIterationNumber()=54
+
+
+
+
+
+
+../_images/examples_5_optimization_3_1.png +
+
+
+
+
+
+../_images/examples_5_optimization_3_2.png +
+
+

We now test the inverse procedure by checking that it converges to the expected result, considering meshes with decreasing element size. We compute the \(L^2\) error in each case, and estimate the order of convergence by a power law fit between the error norms computed using subsequent pairs of meshes.

+
+
[3]:
+
+
+
Ns = np.array([20 * (2 ** p) for p in range(4)], dtype=int)
+error_norms = []
+for N in Ns:
+    mesh = PeriodicSquareMesh(N, N, 1.0)
+    X = SpatialCoordinate(mesh)
+    space = FunctionSpace(mesh, "Lagrange", 1)
+
+    m_0 = Function(space, name="m_0").interpolate(Constant(-1.0))
+
+    M_solver = LinearSolver(assemble(inner(TrialFunction(space), TestFunction(space)) * dx),
+                            solver_parameters={"ksp_type": "cg",
+                                               "pc_type": "sor",
+                                               "ksp_atol": 1.0e-32,
+                                               "ksp_rtol": 1.0e-12})
+
+    def M_inv_action(x):
+        y = Function(space)
+        M_solver.solve(y, x.copy(deepcopy=True))
+        return y
+
+    m = minimize_tao(forward_J, m_0, gatol=1.0e-10, grtol=0.0,
+                     M_inv_action=M_inv_action, post_callback=post_callback)
+    m_tilde, u, J = forward(m)
+
+    m_error_norm = sqrt(abs(assemble(inner(m - m_tilde, m - m_tilde) * dx)))
+    print(f"{N=} {m_error_norm=}")
+    error_norms.append(m_error_norm)
+error_norms = np.array(error_norms, dtype=float)
+
+orders = -np.log(error_norms[1:] / error_norms[:-1]) / np.log(Ns[1:] / Ns[:-1])
+print(f"{orders=}")
+
+assert (orders > 1.99).all()
+
+
+
+
+
+
+
+
+tao.getIterationNumber()=54
+N=20 m_error_norm=0.006304152630049985
+tao.getIterationNumber()=28
+N=40 m_error_norm=0.0015825703318738235
+tao.getIterationNumber()=23
+N=80 m_error_norm=0.00039604503914227284
+tao.getIterationNumber()=21
+N=160 m_error_norm=9.903640999431026e-05
+orders=array([1.99403285, 1.99853321, 1.99963358])
+
+
+

We find that we have close to second order convergence.

+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/examples/5_optimization.ipynb b/examples/5_optimization.ipynb new file mode 100644 index 0000000..db1bc95 --- /dev/null +++ b/examples/5_optimization.ipynb @@ -0,0 +1,320 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "258cd78c-55e3-40e6-97cb-057064281a61", + "metadata": {}, + "source": [ + "# Functional minimization\n", + "\n", + "Gradient-based optimization algorithms attempt to use derivative information, for example as computed using the adjoint method, to accelerate the solution of optimization problems. A typical example is to seek the minimum of an objective cost functional, constrained using one or more partial differential equations. This notebook describes how to solve such partial differential equation constrained optimization problems using tlm_adjoint.\n", + "\n", + "This example makes use of tlm_adjoint with the [Firedrake](https://firedrakeproject.org/) backend, and we assume real spaces and a real build of Firedrake throughout. The minimization problem is solved using the [Toolkit for Advanced Optimization (TAO)](https://petsc.org/release/manual/tao/).\n", + "\n", + "## Forward problem\n", + "\n", + "We first construct a partial differential equation constrained optimization problem where we know the answer – where we know the value of the control which minimizes the objective functional subject to the constraint that the forward problem is solved. To do this we consider the solution of the modified Helmholtz equation in the unit square domain,\n", + "\n", + "$$\\alpha^2 \\nabla^2 u - u = m \\qquad \\text{on} ~ \\Omega = \\left( 0, 1 \\right)^2,$$\n", + "\n", + "where $\\alpha \\in \\mathbb{R}$, subject to doubly periodic boundary conditions. We define an objective cost functional\n", + "\n", + "$$J \\left( u, m \\right) = \\frac{1}{2} \\int_\\Omega \\left( u - \\tilde{u} \\right)^2 + \\frac{1}{2} \\beta^2 \\int_\\Omega \\left( m - \\tilde{m} \\right)^2,$$\n", + "\n", + "where $\\tilde{u}$ and $\\tilde{m}$ are given functions and $\\beta \\ne 0$ is a real scalar. If we set\n", + "\n", + "$$\\tilde{m} \\left( x, y \\right) = -\\sin \\left( 2 \\pi x \\right) \\sin \\left( 2 \\pi y \\right),$$\n", + "$$\\tilde{u} \\left( x, y \\right) = -\\frac{1}{1 + 8 \\pi^2 \\alpha^2} m \\left( x, y \\right),$$\n", + "\n", + "where $x$ and $y$ are Cartesian coordinates in $\\mathbb{R}^2$, then $u = \\tilde{u}$ and $m = \\tilde{m}$ will be the minimum of $J$ where the modified Helmholtz problem is solved.\n", + "\n", + "We consider a continuous Galerkin finite element discretization, seeking $u \\in V$ such that\n", + "\n", + "$$\\forall \\zeta \\in V \\qquad \\alpha^2 \\int_\\Omega \\nabla \\zeta \\cdot \\nabla u + \\int_\\Omega \\zeta u = -\\int_\\Omega \\zeta m,$$\n", + "\n", + "where $V$ is a real continuous $P_1$ finite element space whose elements satisfy the doubly periodic boundary conditions. We now define $\\tilde{m} \\in V$ and $\\tilde{u} \\in V$ via interpolation, at mesh vertices, of the functions given above.\n", + "\n", + "We first use Firedrake to solve the forward problem. We consider $\\alpha = 0.1$, $\\beta = 0.1$, and an 'initial guess' where $m = -1$." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "f41c40c7-d0c3-438e-b874-e35cfeee7d40", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:46:48.938196Z", + "iopub.status.busy": "2023-12-20T16:46:48.935772Z", + "iopub.status.idle": "2023-12-20T16:46:54.013715Z", + "shell.execute_reply": "2023-12-20T16:46:54.012976Z" + } + }, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "alpha = Constant(0.1)\n", + "beta = Constant(0.1)\n", + "\n", + "\n", + "def forward(m):\n", + " space = m.function_space()\n", + " X = SpatialCoordinate(space.mesh())\n", + " test, trial = TestFunction(space), TrialFunction(space)\n", + "\n", + " m_tilde = Function(space, name=\"m_tilde\").interpolate(\n", + " -sin(2 * pi * X[0]) * sin(2 * pi * X[1]))\n", + " u_tilde = Function(space, name=\"u_tilde\").assign(\n", + " -(1.0 / (1.0 + 8.0 * pi * pi * alpha * alpha)) * m_tilde)\n", + "\n", + " u = Function(space, name=\"u\")\n", + " solve(alpha * alpha * inner(grad(trial), grad(test)) * dx + inner(trial, test) * dx\n", + " == -inner(m, test) * dx,\n", + " u, solver_parameters={\"ksp_type\": \"cg\",\n", + " \"pc_type\": \"sor\",\n", + " \"ksp_atol\": 1.0e-32,\n", + " \"ksp_rtol\": 1.0e-12})\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.addto(0.5 * inner(u - u_tilde, u - u_tilde) * dx)\n", + " J.addto(0.5 * beta * beta * inner(m - m_tilde, m - m_tilde) * dx)\n", + " return m_tilde, u, J\n", + "\n", + "\n", + "mesh = PeriodicSquareMesh(20, 20, 1.0)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "\n", + "m_0 = Function(space, name=\"m_0\").interpolate(Constant(-1.0))\n", + "m_tilde, u, J = forward(m_0)" + ] + }, + { + "cell_type": "markdown", + "id": "84b81c75-ceba-4e7e-a5ed-c3cbb521617c", + "metadata": {}, + "source": [ + "## Inverse problem\n", + "\n", + "We now seek to solve the inverse problem: to find the $m \\in V$ which minimizes $J$, subject to the discretized modified Helmholtz problem being solved.\n", + "\n", + "In the following we use the Toolkit for Advanced Optimization (TAO) to solve the inverse problem. We use the Limited-Memory Variable Metric (LMVM) approach with an absolute tolerance of $10^{-10}$. Noting that the adjoint computed derivative is an element of the dual space $V^*$, we need to define an appropriate dual space inner product. Here we define an inner product using the inverse mass matrix." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "d7d2768b-5ff1-4f58-9370-1833e01cb612", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:46:54.016633Z", + "iopub.status.busy": "2023-12-20T16:46:54.016317Z", + "iopub.status.idle": "2023-12-20T16:47:01.740432Z", + "shell.execute_reply": "2023-12-20T16:47:01.739569Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tao.getIterationNumber()=54\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAg0AAAGzCAYAAACsMCQMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAACFn0lEQVR4nO39e3xU1b3/j78yCZmESwjXBDRyrYIVjUJJo1WxpIXq4dTWVlSqQBFrlVaJPQoVxVvFC3poLZV6wcupHi+n6scKh5ZC+bbWVCrIr1oRD4JC0USQQiAhk8vs3x90xpnJ3nvW5b32Zeb9fDzmoZnZlzV7hlnP/X6/11oFlmVZYBiGYRiGyULE7wYwDMMwDBMOWBoYhmEYhhGCpYFhGIZhGCFYGhiGYRiGEYKlgWEYhmEYIVgaGIZhGIYRgqWBYRiGYRghWBoYhmEYhhGCpYFhGIZhGCFYGhiGYRiGEYKlgWEYhmEYIVgaGIZhGIYRgqWBYRiGYRghWBoYxidmzZqF4cOHd3v+lltuQUFBgfcNYhiGyQJLA8MwDMMwQrA0MAzDMAwjBEsDwzAMwzBCsDQwDMMwDCMESwPD+IRTsWNXV5fHLWEYhhGDpYFhfKJfv344cOBAt+c//PBD7xvDMAwjAEsDw/jEqFGjcPDgQfztb39LPvfxxx/jxRdf9LFVDMMwzhRYlmX53QiGyUc+/fRTDBs2DBUVFfjhD3+I1tZWPPjggxg0aBA2b94M/qfJMEzQ4EgDw/jEgAED8OKLL6Jnz564/vrr8cQTT2DJkiWYNm2a301jGIaxhSMNDMMwDMMIwZEGhmEYhmGEYGlgGIZhGEYIlgaGYRiGYYSQloY//vGPmDZtGoYOHYqCggK89NJLWffZsGEDTjvtNESjUYwePRqPP/64QlMZhmEYhvETaWloaWnBKaecguXLlwttv3PnTpx33nk455xzsGXLFlx77bW4/PLL8dvf/la6sQzDMAzD+IfW6ImCggK8+OKLOP/88x23ueGGG7Bq1Sq8/fbbyecuuugiHDhwAGvWrFE9NcMwDMMwHlNk+gQNDQ2oq6tLe27KlCm49tprHfeJxWKIxWLJv+PxOPbv348BAwY4ztfPMAzDBBPLsnDo0CEMHToUkYi5Urq2tja0t7eTHKu4uBglJSUkx8oljEtDY2MjKioq0p6rqKhAc3Mzjhw5gtLS0m77LFmyBLfeeqvppjEMwzAesnv3bhx77LFGjt3W1obhw3qh6ZM4yfEqKyuxc+dOFocMjEuDCgsXLkR9fX3y74MHD+K4445D1a2LEOEPkGFCR7w4P+aQi7RzJNSOeFsbdi++A3369DF2jvb2djR9Ese2N4aiTx+9aMahQ3GcMOEjtLe3szRkYFwaKisr0dTUlPZcU1MTysrKbKMMABCNRhGNRrs9HykpyVlpiBzJ7dGv8VIa+2fkiEeD0Vnn9rc7BfufNF+IxIInMF6kl/v0iaBMUxoYZ4xLQ21tLVavXp323Nq1a1FbW2v61ELkemcdFIJ0nfNFYIIiDIw/JD7/IMoDE16kf8kPHz6MLVu2YMuWLQCODqncsmULdu3aBeBoauGyyy5Lbn/llVdix44duP766/Huu+/iF7/4BZ577jnMnz+f5h1oEKSOjPGOyJFIzn/2LAxMgnjU4u8DQ4Z0pOGNN97AOeeck/w7UXswc+ZMPP744/j444+TAgEAI0aMwKpVqzB//nz89Kc/xbHHHotHHnkEU6ZMIWi+OrneaTDZSXwHghx5yNkf+5Iuv1tAT1uh3y1wRfW7xJEKJhVpaZg0aRLcpnawm+1x0qRJePPNN2VPJUSudv6FsezbhImu7iUqgUHnOxRk4TBKLnb6ulBdk4DJRzxqsTgwSQI5eiIXyLVOXxeK6xFE8YgcieSPOLAoeEPqdQ6IQLA4MAlYGghhUTBL5vUNikTktDiwKPhL5vX3USJYHBiApUEbFgX/CJJEhE4cAigDhdHgtUmGrpgHHbrs50YsGSwODEuDA0GUgcI2v1ugTpcH02vIfGYmBCM04uCzMIRdDpxwe1+eCIUdic+aUB5YHPIblgYb/BaGMMuBE07vyQuZsCPxGVPLQ+DFwZAw5KoIUCF7fcglg1geWBzyF5aGDEwJQy6KAAWy14VaMkzIQ2DFgVgYWBTMkXptSQWCUB6CKg77utoR69KcRrorgP9+AwJLQwrUwsCiQE/mNaWSCGp5CJQ4hFQWoiU0qxV6Qayt2NixM683iUQQyUNQxYExB0sDwisLRSGRkk6DKYjUa00hEKnfBV2BCIQ4EAmDSVEIkxw44fYeqIWCVCII5IHFIb/Ie2mgEgaTohAWOXDCrf2UQkEdhaCIPvgqDprCYEIUckEQZDEtFCSpDE15YHHIH/JaGnSFgVoUwi4HKpgUCiqJ0JUHX8RBURioRCEf5UAFp+ukKhN2n5+USGjIA4tDfpC30qAqDFSikI+CIIvTNVKVCV2J0JEHT8XBJ2HwQhR6RYMpIy0x2hRE5rXUiUgkPlcleQCkBILFIffJS2nwSxhMi0JQCy+pRzxkXkddifBKHjwRBwVh0JEFU6IQVDlwwq69lCKRep11ohBK6YuSLhYHJkneSYOKMOh0xiZEIahy4IRdeylFInGNdeRBpT2FMbOzUEqvSigpDKqyQCUKYRMDWZzen65MJK6/ijwoiwPD/ItQS4Ps6oSywqDaOVOJQtjkQAYTIpF63WUFQlUcTGFSGFRkgUIUcl0SRKGKSqhGH1gcGB1CKw0mhUGls6YQhVyWBBEoRUIl+qAiDiaiDaaEwQ9ZMC0KvaPeTN96OGZ2YZPU66QjEKLyIC0OkikKJncJpTSYEgY/ZMG0KBS1SXZACnSWmMtf6hYvyspD0CIOWREQBi/TECYkwSsxoGgDhVwkrqFpeeCIA6NCKKVBBhFh8DINQT9M07wUULSBSixUO3WZ1IXsOUzXNniNrCxQiEIQxICC1PehKxBeyIOUOHC0gUEeSIMJZGWBQhSCIAc6pLZfVyBURz181hZ6cfAFA1EGUWHQFYVckQQ3Mt+jqkT0irYrF09GS9ppxSEE7ItH0RbXW3vicDwgU8AHkJyWBuoog4ws6A/PDLckuGH33lREQkceqMXB82gD8XoSMtEFHWHwQhbKo+r/+A7EzJmi3XsXFQndqAOZOHC0Ie/JWWmgXk/CK2EwLQs6xzdZu6AjEqryENqIA2HhoxeyQCkKOkJg8viqsiErEqrykI8RB8YMOSkNlIWPXsgClSgEVThUZSPzfNmOoyIPlOLgSbSBKMJgsm5BVxJMi4EJUtusG63oHY1ljUCojLYgEweONuQ1OScNVBEGk3ULup172FIXVPUMieNQy0NoIw4uUC42JSIM+SgKTlAIROJ6iqQvZKIPHHFgdMk5aRCFchSDWMQiv0TBCQqBkJUHIHuHTyUOYRhJQZ2SUBUGL0VhQLTF9fVPY72MnDfzPcpKhIo8AO4CQSIOHG3IW/JWGtygTkmodvheikJRq321cGdPvSpk13NqCkRRm0Va95ArEQe3KAPl6AgVWaAWhWwyQHkcCrFQjULIyAOQfcQFRxwYVfJSGiiiDKZkgVoUnGSAen9duVAVCNGoQ4Js8kAhDmGINjghWrsgKww6skAlBrpQi4XdNckmEiL1DgmMiwNHG/ISc7eRISVblKGwzYwwFLVZapLRGnd9eAVlOxLXIvMhso8obp+hSKRJVTxlZzOVRSfKQCkM5dG2tIcMA6ItaY8wodtukevVOxoTlrZsn6lI5EmlPkZ6KvQcYPny5Rg+fDhKSkpQU1ODjRs3Cu33zDPPoKCgAOeff37a87NmzUJBQUHaY+rUqWnb7N+/HzNmzEBZWRnKy8sxZ84cHD58mOot2ZJ30qATZRDdV6TzkukQu+3rgxToQtFmmesqgl/iYAqd4kdqYVAhjJLgho5AUMoDhTg4QjxfSFh59tlnUV9fj8WLF2Pz5s045ZRTMGXKFHzyySeu+33wwQf40Y9+hDPPPNP29alTp+Ljjz9OPv77v/877fUZM2bg73//O9auXYtXXnkFf/zjH3HFFVeQvS878io9ke1H3q2joBYGWcIkCNnIfC8yqQ3RdIRovYNbqkEkVeF4XJcUReRIBPFS2s8zmzDoLjwlemerElWgpH8PeunY36Ffy5D6PmVTGCIpCyD7/A46qQqV+oZ41EIkZm5uFy9obm5O+zsajSIa7X6d77//fsydOxezZ88GAKxYsQKrVq3CypUrsWDBAttjd3V1YcaMGbj11lvxpz/9CQcOHOi2TTQaRWVlpe3+W7duxZo1a/DXv/4VEyZMAAA88MADOPfcc7F06VIMHTpU5q0Kk1fS4IbuwlOiIqASVaCk6Ajd8TpLaQJVqe9RVCBE5EFmlIWqOIShMDIb2e5EqaMLOqJgQgoozikjFrICkbi2uvUOFDUOtgSstuHTrl440qXXntauoxJeVVWV9vzixYtxyy23pD3X3t6OTZs2YeHChcnnIpEI6urq0NDQ4HiO2267DYMHD8acOXPwpz/9yXabDRs2YPDgwejXrx++/OUv44477sCAAQMAAA0NDSgvL08KAwDU1dUhEong9ddfxze+8Q2p9ywKS4MA2SMUtNEFVVGgFALK88nIhaxAUMmDiYhDUAoidaIMlMKgKgt+iIIsqW00JRAi8pAt6qCzjoUKYY827N69G2VlZcm/7aIM+/btQ1dXFyoqKtKer6iowLvvvmt73FdffRWPPvootmzZ4njuqVOn4pvf/CZGjBiB999/Hz/+8Y/xta99DQ0NDSgsLERjYyMGDx6ctk9RURH69++PxsZGiXcpR95Ig27+2nlfGmHQyvV7LAuyJNonG5mQEQiRdITMEM3u+zqLg0q0QSlFoZA/1il+pBIGFVkIgyg4kWi7bFojcZ1E5EEn6uAmDsrRhhylrKwsTRooOHToEC699FI8/PDDGDhwoON2F110UfL/x40bh5NPPhmjRo3Chg0bMHnyZNI2yZBT0kC93gSQTTb8E4agi4IdqvIAHL1GpsUhCKkGL6vOdYQhzLIwsEd6dfm+jt5GzqMjDxTiwHjDwIEDUVhYiKamprTnm5qabOsR3n//fXzwwQeYNm1a8rn4v1bVLCoqwrZt2zBq1Khu+40cORIDBw7E9u3bMXnyZFRWVnYrtOzs7MT+/fsd6yAoyClpUEG1+FFkCGDWcwdQFgqPdCb/v6vUzNcj9T3Ipi78jDjoFEaaxKkI0i3KECRhoBaFTCnQ2Z5CKFTkgUIcZOZ0SMUp2sCTPdlTXFyM8ePHY926dclhk/F4HOvWrcO8efO6bT9mzBi89dZbac8tWrQIhw4dwk9/+tNudRQJ/vGPf+DTTz/FkCFDAAC1tbU4cOAANm3ahPHjxwMA1q9fj3g8jpqaGsJ3mE5eSINT5x8WYaAWhVQxUN2OSihkow9U4uBEEKINthAObdNZ2ppSGHRkQVYMTJxHVihk5cGkOHhd25Dr1NfXY+bMmZgwYQImTpyIZcuWoaWlJTma4rLLLsMxxxyDJUuWoKSkBCeddFLa/uXl5QCQfP7w4cO49dZbccEFF6CyshLvv/8+rr/+eowePRpTpkwBAIwdOxZTp07F3LlzsWLFCnR0dGDevHm46KKLjI2cAHJIGihTE6aXthYVBh1ZEBUDU8eXlQoZeaAQBx2xCBKUC1MB7lGGbMJgUha8kgQZVNMcMkWTfqUquLZBjunTp2Pv3r24+eab0djYiOrqaqxZsyZZHLlr1y5EIuJR1cLCQvztb3/DE088gQMHDmDo0KH46le/ittvvz2tGPOpp57CvHnzMHnyZEQiEVxwwQX42c9+Rv7+UimwLCvwU3c1Nzejb9++GHb3HYiUlNjOqucmDbKRBtUoQ9YIhEFZMC0JOqhEJUTkQWR0hZsYuL3mFm1wSlHY7eM2esKuENKxpsEh0iCbmlBNS1AIQ67IQjZUUhrZ5EFkaKabODilKdyiDU7SYJuicBlyGYkVIN7Whg9vWISDBw+SFxYmSPQTz///xqBnH80hl4e68O1T3jXa3rCS8zNChkkYio7EpYWh8EhnoIUBUGujyLUQuqYan1lgIEpNhEUYBvY4nHyEEZW29+/R4nqNRK5xLi0vzgSXnEhPUKUm/BYGUYIuCU4k2i0TeciWttBNVaikKYJSEEmVmlBd2tqELFAzsOiQ1Pb7OvvQnbvHYaW6B6eog06qQqUoklMUjB05IQ1O+L02QBhkobC1Q3jbrp49aM6pKA+mxMG2jUEtiBRAd8roVHTuXkWFgUIWZOVA9jiqMpF4bzLyYEoc7OCCSEaWnJYGJ2QnczIV4hYVBl1ZkBEDleOoyoSsPJgSB6poQ5BFwyk1oZqWyBZlMCUMVHIgi9t5RYRCVh5MiANVtIGHXuY3eSkNdjjXPqgJQ7Yog4gwqMgClSBQnVNEKGTkwcuIA4UEmJpKmiI1YUIYck0WREhtWzaBkJEHXXGww04cSKINAVp/oqmzL0o79Lq2I53hTAF7Qc4WQlKkJkwIg2ixo6wwFLZ2+CIM2ZBpl2jBpNv1Uy2ODE1RZBYoUxN26AqDTJHgwKJDyUdYEG2r6HXQKY7kwkjGBHkXadBdzTLr8bMIQzZkZCGIkuBEoq1UkQfdiIMoppbO9hLZyZxUOhtRYRDBlCRU9jjg+FpjRznZeRLtp0pbUEccZKMNXBDJpJJ30mCHbFRC5a6UUhhMyUKkxT5cHe9FF1+nlAcdcbBLU1BM+GQnGUFZ7TIVp9SETh2DEzKpCAphcJMD2X10ZIJSHlTFwVRRJNc15C8sDQ44iYFOHYMTXkcXnORAZntdkShs7RAuoHSTB+qIg504BLnAMRXd1ISJOgbT0QUVQdA9vqxIyMqDF+Kgui6FKwGqa2DMkVfSoJua0KljcMJ0dEFWEHSPKyMTMlEH4Oi1ohQHishC0FMUOutMJAiKMJgWBFFURUJUHkyIgyg8BJPJRs4WQnpJUIQh0hLr9vAalTbIFkvaoVocKVoUKZPC8nt+kGyoTuaUia4wiBY5VvY4EBhhcEKmjSLv2+36qRRH2kWRZL4HdlEs6nVPmHCQ99Jg8gdeVxhEO1O/BEEUanmgFgdRdL4rlAuqmcApNaFSx5BNGGRGRJiShUGFhzGokH4GSll5cH2dWBzssBMHregU4WqsTDDJe2mwQ6bQUaUzyiYMMnfeQZaFTETbKvL+KcXBryGYdguvBQmK+RgykaldoBSGhCRkyoLdcxT4LQ528BBMhoLQ1TT49UNr27FoDq+0Q6Z2gVQYWo64v96rlOQ0kZaYcN2DTLFkKl4NxwwSduFjuztGitSEalrC6+iCrAhkbr+3S361ylQqexwQrnWgXPMCoKlvYBg7QicNlJhKTaimJTyNLmSTBLftNQWCShycCiNNEoSRFNS5ZKo7UL+FgTpakHo8VYGgEAfVwkgduCCScSJvpEF05ITJULXvwiArCqLHUZAI0xEHt2iD8DEER1cEfQSFCk6pCZW0hElhMFGXIHIeGYnwM+LAMNTkpDR4Ub3umB9XSEsYEwYqSZA5j4RAUIiDSrQhVCkKwsIy3dQExfBKJ2SEwStREG2DiEDoioNKtEE0RSE6Z0NYZob8tLMPSjr1urY2XnvCkZD8cjpDWZVusvhNd6VKKWFoOeKdMDidW7ANJgs5ZQQuV9aeSCBaAU+RmtBNS4gKg6kRD7qItouqOFIHLoZkdAm9NKgiGo2QGklhKMog3LH6KQtOEIqD07XSFTImHdXpojOhFoagQykOduhGcxiGgryVBh1kh1nqdGpSwhBUfIw4OImczrwNQZu8yfTKloBzakKnI8slYUhAJQ6y0Qanz0dHAClmE2Vyj7yQBtMrWwLOnZNO8WNOCEMCInEwGW0Iw5LZqiMn7OoZ7ELVXkUZ/BKGQZGY7YP0HAYjDhTRBk5RMDrkhTSI4HWHEVRhsFpb0x6k+BRxUJ0zg3FHtQMT6TCp6hdE5YBaICjEgSraIALV1OJM7pNz0mA6dCwT1la9+xXqOInqF7JJArlAEIgDVbSBYmrp5LEyvndBS2HoINsZuXV2osKgg64AUAmEqYiDrKxRRY/s4PUn8o+ckwY/kL2TdYsyCAuDBqoiQCYQBsXBiXyKNojkonVTE04dl1/CYCzVoCsgmuLgZbRBBC/qZ5hgw9LggOy00Zk43fX6JQzUEQPt4xlKVejWNoimpIISSTD9I266E0pFetpnQ6JAfT4TEQfd2gaua2BUyUtpoPzB9/QOVlIYjNUmuJxH6lya4kARbRARwaAVQwYV3SiDCF6Lgkg7RNqiIw5O19VJHOxETySKRL7iJZOT5Lw0eDFyQhTtKIMAXokCWRsMRBx43oZ0RIrcKFITTlClJfwWBTdMiwPDBAUlaVi+fDmGDx+OkpIS1NTUYOPGja7bL1u2DCeccAJKS0tRVVWF+fPno60tQL25ACYLIKnSEn6KghOmxcFEbQNHFo4ik5pwuhvOB2FIQCUOdlBEGzLhFAWjgvQE3c8++yzq6+uxYsUK1NTUYNmyZZgyZQq2bduGwYMHd9v+6aefxoIFC7By5UqcfvrpeO+99zBr1iwUFBTg/vvvJ3kTuqh2EhSpiVwWhgRWaysKevZ036jlSNZ1K6TWqpBYk0JkLQrRxavS2pCxImZhDOgSa35goZ6V0A9hGFSYvn7C3i66EPygSAx74+4f8qDCw67rVTitU6G7oJXIWhSi61DIEI9agId+8mlHL0Q75Ba4yyTWIXczkk9IRxruv/9+zJ07F7Nnz8aJJ56IFStWoGfPnli5cqXt9q+99hrOOOMMXHLJJRg+fDi++tWv4uKLL84anQgrKgWQrvggDNbhFliHaQvgqCIOdvg1vXRmbUwQUmGqIycyoSiAzBZl8FIYBhUWJx8yrymdy2DEwQ6/p5fuNuyScJE1JnhISUN7ezs2bdqEurq6zw4QiaCurg4NDQ22+5x++unYtGlTUhJ27NiB1atX49xzz3U8TywWQ3Nzc9qDCqoiSE+iDB4KQ0IUUmXB7jmtcxCIA0XtB6coaFFZYMkrYZCVASp50G07RW0DVYoiU0R52GV+IyUN+/btQ1dXFyoqKtKer6ioQGNjo+0+l1xyCW677TZ86UtfQo8ePTBq1ChMmjQJP/7xjx3Ps2TJEvTt2zf5qKqqkmmmNtQdhnKUIQsUwiAqBVTyYCriYOoaM5/h9x2tLDqdP1XUwf0c8tfT5AqYDCOC8dETGzZswJ133olf/OIX2Lx5M1544QWsWrUKt99+u+M+CxcuxMGDB5OP3bt3m25m8PBgPQkVCaBOW5iGR1LkJyTRAg/EgWHChlQh5MCBA1FYWIimpqa055uamlBZWWm7z0033YRLL70Ul19+OQBg3LhxaGlpwRVXXIEbb7wRkUh3b4lGo4hGw1cxFqYOSqfztw63oKC3e0EVEwyoh1uK4hZeN52aoOzsBxUWkxZKmmJgj8PY1+FcXMkwVEhFGoqLizF+/HisW7cu+Vw8Hse6detQW1tru09ra2s3MSgsLAQAWBbnjW0xXMtAkmbQkQ7NFIWRRa14kqecwER0QPWYXhdEMowXSA+5rK+vx8yZMzFhwgRMnDgRy5YtQ0tLC2bPng0AuOyyy3DMMcdgyZIlAIBp06bh/vvvx6mnnoqamhps374dN910E6ZNm5aUBz8R6QgoFzZKYKLjE4EyvcARByZIOXaT6QQ/Ig5OQy8Zxk+kaxqmT5+OpUuX4uabb0Z1dTW2bNmCNWvWJIsjd+3ahY8//ji5/aJFi3Dddddh0aJFOPHEEzFnzhxMmTIFv/zlL+neBSOEiXqEsNU4MMFHJTXhTeEi1zgwzshMevjCCy9gwoQJKC8vR69evVBdXY3/+q//StvGsizcfPPNGDJkCEpLS1FXV4f/+7//63asVatWoaamBqWlpejXrx/OP/986reWhnSkAQDmzZuHefPm2b62YcOG9BMUFWHx4sVYvHixyqlCj4mqfpXUhMnOXSXiIDThkySFrR3o6qk3qUuu49UcDU6YCMd72ZnLRhxEJnvyEpEJnhh5ZCc97N+/P2688UaMGTMGxcXFeOWVVzB79mwMHjwYU6ZMAQDcc889+NnPfoYnnngCI0aMwE033YQpU6bgnXfeQUnJ0Vnjfv3rX2Pu3Lm488478eUvfxmdnZ14++23jb7XnF97InQYGDXhRTTAyDkMjiDJp6WyKQjqcEs/7v6pzykrUqZTQiLFs0w6spMeTpo0Cd/4xjcwduxYjBo1Ctdccw1OPvlkvPrqqwCORhmWLVuGRYsW4etf/zpOPvlkPPnkk/joo4/w0ksvAQA6OztxzTXX4N5778WVV16J448/HieeeCIuvPBCo++VpUGBMHU4XqYPgpqqCNOollxAZ2IimdSEn+mCsKQqvFzaPBfJnGQwFuv+/VSZ9DAVy7Kwbt06bNu2DWeddRYAYOfOnWhsbEw7Zt++fVFTU5M85ubNm7Fnzx5EIhGceuqpGDJkCL72ta9xpMFrVCvkZTomnSJImdSEH524zDl1RoD4VUjKqEOZmghCpy3aBp3hozICFtRokNf8s6Mn9nf00nr8s+No6rSqqiptosFEgX8qKpMeAsDBgwfRu3dvFBcX47zzzsMDDzyAr3zlKwCQ3M/tmDt27AAA3HLLLVi0aBFeeeUV9OvXD5MmTcL+/fsVr152lGoamODj510/6agKgYWsGH/wa+REEIQhAdWoimyLWDH+sHv3bpSVlSX/ppw/qE+fPtiyZQsOHz6MdevWob6+HiNHjsSkSZOE9o/Hj0a8b7zxRlxwwQUAgMceewzHHnssnn/+eXzve98ja2sqLA1BgiiHH4Q0QS4Mx1RZ2TJsmJjYSRWxeQ2CIwwJwjIBFCNPWVlZmjTYoTLpIXA0hTF69GgAQHV1NbZu3YolS5Zg0qRJyf2ampowZMiQtGNWV1cDQPL5E088Mfl6NBrFyJEjsWvXLvE3KQmnJwxCPXJCJJwfBGFIILSmBfEKnbwGBRNEqJf3ZoKDyqSHdsTj8WTNxIgRI1BZWZl2zObmZrz++uvJY44fPx7RaBTbtm1LbtPR0YEPPvgAw4YN031bjnCkgTFKLkQcGHGccvAU9QxBjDIk8DraMLDoEPZ19iE5Vnm0DQdiJSTHyldkJz1csmQJJkyYgFGjRiEWi2H16tX4r//6Lzz44IMAgIKCAlx77bW444478LnPfS455HLo0KHJeRjKyspw5ZVXYvHixaiqqsKwYcNw7733AgC+/e1vG3uvLA1ZMDEbpC0EqYkgRRlI8aiuoag1js6eHHzzA74T57qGMDN9+nTs3bsXN998MxobG1FdXd1t0sPU5RRaWlpw1VVX4R//+AdKS0sxZswY/OpXv8L06dOT21x//fXJdZoOHDiAL33pS1izZk1yjgYAuPfee1FUVIRLL70UR44cQU1NDdavX49+/foZe68FVggWgGhubkbfvn0x7O47UGSlTwhUmPFbU5gxd01RW7bXLfe/baTBbsil3egJu1C5Y9U/wXoTQZUGkUhD1omeHKQh3qt7YZLdBE9dpd39uLO0uyBkSkNmTYNdjUNXSerrzq8BQFdGc+Ol6d+leDTjn2NJV9qfhdH0v6Mln93d9op2v9PNHHOfObmTXU1D5jA9p6p8u0JI1UhDWOsZUhGJNGSb6MlJGuymk7aLNNgtWrW/o/u/v8wJnjIjDYdj6e1siaVf+1hb+t9dsYwlAdoy/j4Yw4c3LMLBgwez1giokugnLv3DxSjurfddaT/cjv8657+Ntjes8G1VSKDO/ecbPFeDGCbG9ed6aiJBENrIwy4Z07A0EBCEDimoUQaAqG0GZocM0yRdTH7Aq14yQYelgWEIyUyH5SpBWt2SYRjvYGkgwC5f7jU5P0KBJ3gKFFSV+0w6XAjJBB2WhixQV9PbFe6JQL0ipJfkvNDkEHZFc7pQdIRhmDwpCG20K4RkGEr8v0Vm8h4vhEh09ESYaYkV246g0GVfR2/jBXZ749GcH3YZpCWyc5n97b3QI6ZXlNrR3n0EFnOU3PrVDBh2Q/8cIQi/8x2992QOqQwbmUPvdLEbGsjowdeUCRIsDRkEea2BMKYoSERGYo4Gv8mcp4E5Sq6nKCja5lU9A7UoMvkFS0OOEaRoQ5DawtBDXQzJ4XuGCT4sDUGCaIRAmDrrIEVP8mEK6bCvMRDEaAPFTJCy8OgVxi9y/1fSI2SGXeqE1UU7Wb/Fgez8kiIlVUfChJIgiQNVW3ioJRMWWBoUCEvVvV/i4LewMMGEsmMMgjgEoQ0M4zXh6P1CjN93vl534DLn00lNBLEIMpeRGf+vU+0vE8b3s9OWObdOakLmWqrO0RD2lBXjLXkvDYEbLSEQjpftbL0Sh6BGGIIwY2eYMDHBkyn8EIcwRRjC9Fky4SDvpUGEXCiQM92hGzm+wamjRVJMgRPKACJbkGcid+9lJ27iXFzPwISJ8PeGIcWP8LopcVA5bpBGTTDBQiWc74U4qJzDxDBSHjnB+AlLQxAxeIdNLQ5BSkn4XT8SRA7HsndauTLZj0lxCFNKgmFMwsleQrpKi1B4pNOTcxX07AmrtVVt3969YB1u0W+DSWHgVS1DTWNHOSp7HLB9bW9XbwwqNLOWxd6udgwq1Ft3wO6YpnBLTZiYPlpEEEVEM8gcjEVRVKT3HjpjnJp0giMNeYpuh6+zv25qwq+RE1zj4B06YX3KTl7nWDzDJZOLsDQEFQ/utJVqEXr3ClRKIhuiIydyodjVNGFadplmLYjwpCTC9Nkw4SbvfilFViUUuaOUmeDJVK6dophQRgIoZEGozZyaYAjQihJ4UljJHT0TPnJOGjKlwOtVB0mnkzYwZ4Pjcf4lD5li4PS8MbK8Z6drJiNmYZnRMyyoVPOLdJgU4f29Xe1yEzFJbu94HEOpCZlrLTJHg8jETi0x2hoRJtyE7tczXhpP+7uL4N+mUGRBM3zt1KkFSRySxzMkCl4Ps9SZ1EnkOyEStfIakR94kY7CqcOhnBnSK3E4eq52RyFwe03pXAJtzvbeTRRBMgwFoZOGIOF0x0o+A6EP4kANRVqCIsoQOtoK0/7siqX/HWvLLgleDLtUnTvAS3H47Jzt5KKQPDaBMDjhdI25noHxEpYGB0xUyitHG4DQikNBz56BqmPg1IQ4XkQbAH/EwQRUwkARZbD77PJhuCVjHv4F/ReqKQqqaAOlOAh31AaRaoPA+5IdZilz/UVST7k23NLEIkVOd8L5IA5+CQNHGRivyUtp8DMX7RZKpxKHBF4LhNL5NIUhSKkJr4tuZaBOUch2VpTikHj4DXVb3K5RkKaOFkmJMblLXkiD6o+56N2lbLTBS3FIYEogtI5rSBicrnu+pCYoq91lV0l069yoxCG5rQ8CoXpOL4dXin5mPHKCUSEnf0VNRhK8mATIlDgk0BUIEgExkJJQwTblFNKRE7Lopiicog1eikNyH4MCoXts3bQERQFkrqwvwvgPrz2RQmdJAYraLNJjOq1H0dWzBwpbOxz3i/eKItIScz94r1Kg5YhW+1I7fqe1LMjTGwTCIJuWyOUoQ6ytGNESc5MR7e/ohf499NcqSeC2LkUCnfUpMjv3QZEs/44EjqGKKWFwQjYylIu0tEdRqLn2RIgmA/Wc3P0lJUI3RQGoD8E0HXHIJDWCYKwegqC9bsJAXQAZRDKHXVIie0eqEm0AzEUcbI8jGCmgjFbs7eptdKSEiSiD6sgJk99HJniE81eTAJ3wMlVnI3K37LU4GEWwnappCYr5MewkUXXkRNBSGHadgmiKwsQdrJfikDxeihhkPsjOIdjmbO/fZPGjidEzTH6Qt9JgAlMh8NCLQ69SMmFQGS3hRWoiyCMnTKMabQD8EQeTeCEMTtfbi9QEj5xgWBoEoBijrzKSIpXQioNEm3QKH2WjDGFNTYiiU/XuFM421SnlijhQCYMbskNddQogeeQEY0du/3Km4NWdoMm7WmFxkLizN4JkG+K9okLvzXSUQVQOg5Z2CAI60QYg/OJAKQwqaQkZoePUBKMDj57IQHQERWfPCIpa41m3S0V1JEUqQqMqEmR22pojLaTOJYhodIGq+FEWypkgKRZXM8GBWAnKo21C21KPpEhFdFRFKqojLCiQlRjd6aFNzv5IOn10GxdG5jJ5E2mww9Qdo0q0QeYuWjmMTxmFSD1WQIXB6XNQnZshyIjmmkU7B5Wwtm60AZDvWBOjFLyIQqSey5QwUEYZvJibgUdO5B95LQ26qOTF3To5T8QhgUqnTygdFMLgBkWaSEckcjmFoVLbYFIcEpgQCIpjUggDVZTBLjUhKpJcBOnO8uXLMXz4cJSUlKCmpgYbN2503Pbvf/87LrjgAgwfPhwFBQVYtmyZ7XZ79uzBd77zHQwYMAClpaUYN24c3njjDQBAR0cHbrjhBowbNw69evXC0KFDcdlll+Gjjz4y8faSsDQIItuBqHZanopDKnZCQBBNsINKGFTSEroFkHYyENSRE34Usrl1bl6IQwLVqIBONCGTxo5y48JgKsrARZByPPvss6ivr8fixYuxefNmnHLKKZgyZQo++eQT2+1bW1sxcuRI3HXXXaisrLTd5p///CfOOOMM9OjRA//7v/+Ld955B/fddx/69euXPMbmzZtx0003YfPmzXjhhRewbds2/Pu//7ux9wnkSE1DVxQolJ/0zRGZmSHdahs6SyMoOtL9NafahuTr/+osReocpGocRDFURCkjOTrCICtsYU9NmOLTWC8MiNrXL6jWNuzr7IOBRYeEthWpcRAlVQAy6yBMpDZkpEd1PgbZiI9fBZCRWAHkqr/Cx/3334+5c+di9uzZAIAVK1Zg1apVWLlyJRYsWNBt+y984Qv4whe+AAC2rwPA3XffjaqqKjz22GPJ50aMGJH8/759+2Lt2rVp+/z85z/HxIkTsWvXLhx33HHa78uOnI00iIaHZcLITp2L292rztLZXT17CE8A5cU6DarItE/0PTvhJgwyUQa/RCISM39emUme3O5YnTqtbKH0fZ19SEdVyEIZTbBDJrqQ7TqopCVkogykBZA5SHNzc9ojFut+g9be3o5Nmzahrq4u+VwkEkFdXR0aGhqUz/3yyy9jwoQJ+Pa3v43Bgwfj1FNPxcMPP+y6z8GDB1FQUIDy8nLl82YjZ6XBDqcwsn3IWTIdYUgcAHl5SH34gUobZGRBZRVLinkZfKtTsKlGFy1AkwkzeykOgLg8yIT5/YQqHZHcRiEtQUHYUxMtsWKSBwBUVVWhb9++yceSJUu6nW/fvn3o6upCRUVF2vMVFRVobGxUfh87duzAgw8+iM997nP47W9/i+9///v44Q9/iCeeeMJ2+7a2Ntxwww24+OKLUVZWpnzebIQyPREvjSNyxItZ/rqnKdxSFyZSFWnbSgzNTGDXaVOmMyjERFcWsuEqdDmUmnBauKolVoxe0fTnD8ei6B0V/x6opCr2dfTGwB7Zh0QmOtBsaQu7DpkqfaGCisiIRlhUhYFXs6Rl9+7daR1wNOrdjVg8HseECRNw5513AgBOPfVUvP3221ixYgVmzpyZtm1HRwcuvPBCWJaFBx980Gi7QikNdujWNXSVAIViQ9VdURUHAELyIFPv4IRTR59NJqgjFzJpiGzC4NcqlkEtglTFbc4Gk+IAiMtDKl6IBEWUQ6ZuwYQw6NYz5OvIibKysqx37QMHDkRhYSGamprSnm9qanIschRhyJAhOPHEE9OeGzt2LH7961+nPZcQhg8//BDr1683GmUAcjw9QVHprrKAkUqqAjjaMVKnLGSwS2+YSnXIRBd0hEElymD3vO9DKDUnzLELOavks1VTFTK5ed2FmhJpgsyH6r66yEQXVIdWqgiD3ecf9tSEHxQXF2P8+PFYt25d8rl4PI5169ahtrZW+bhnnHEGtm3blvbce++9h2HDhiX/TgjD//3f/+H3v/89BgwYoHw+UXIm0kCBTLQh2wgLlYhDsh0eRx68QlZyRASKWhhk8DvK0BUrRGG0S+sYdmmKbDNEqo6qSHSIlCkLGbyui6CKLiRQqWPgKaO9ob6+HjNnzsSECRMwceJELFu2DC0tLcnRFJdddhmOOeaYZE1Ee3s73nnnneT/79mzB1u2bEHv3r0xevRoAMD8+fNx+umn484778SFF16IjRs34qGHHsJDDz0E4KgwfOtb38LmzZvxyiuvoKurK1lD0b9/fxQXmxHAnJIGiqGXduIgMwQzbT8NcQByRx5MyAJgRhjCXuPgVNcA2Nc2qKIzHNNveTANtSwA/tYx5GtqQobp06dj7969uPnmm9HY2Ijq6mqsWbMmWRy5a9cuRCKf/SZ99NFHOPXUU5N/L126FEuXLsXZZ5+NDRs2ADg6LPPFF1/EwoULcdttt2HEiBFYtmwZZsyYAeDoxE8vv/wyAKC6ujqtPX/4wx8wadIkI++1wLIs+d7QY5qbm9G3b18Mu/sOREqOmrNTIaSdNNhFD4pcIgr229tfJhGZcFujIps4pLVLsGDSdl+PZEInZUIhC8ltHKQha2rJ4XWZVJdTKsNu7Yl4affPPx51+E6VdI8q2EUanKQBgKM02BVFiqxH4SQOAITncRCteQCCKw8qKRWK6IJqHYNTasouPeEkDY4jeDJSaZFYAeJtbfjwhkU4ePCgsZx7op84/qkFKOypl07tao3hvRl3GW1vWMmpSINJnKINIlEI3YhDApnIQ7d9XTpzFaGgrqfwQhiy7ichDL7QVmgrDplQRRtEFrKimABKJfJgh1dCoVtzQRFdALwRBobJJG+lobPEOdrgVNsQBHEA9OTB9njEAiB8XskhlLrCQJmW8LueIQFFXQPgPATTK3EA5OTBdn+HzlxVJnTloNvxJIocdeZhoKxj4NQEk0neSgM1FOIAyKUrqOXBK7yWBcC7OobARCVckK1t8FIcAH156HY8m85/YNEhcilwPD+xLOjUMFDNAMmrW+YvOT3kMoHKD7nTPrqdT9bOrTQiPeeAzFBNv0i00Q9hcN03y2dGJQF29QwmUblDdOtQRO5esw3HlL171hmCmPXYhoUh0XavhUElLQEQpiY0hwYzwScvpMEJEyFmCnEA9OQhKAKh0x7R9y90LRUjCc7iKLe9aVTu+pw6CZPiAARPHqhRaavoNfFaGKQLIJm8IBi9S8jIOkeDZqoibTuFtAXgngKgTmdQSYqMJIlGF1QjQ2FIM2TDrSBSFd1URYJEJ6mStgDoUhcU6AgNVTpCVRhykfa2HohE9CIn8bZcX5dTHaVIw/LlyzF8+HCUlJSgpqYGGzdudN3+wIEDuPrqqzFkyBBEo1Ecf/zxWL16tVKDqXGLNuh0HFQRh+S2CpEHJ1IjAJkP2e0phEH2vYlGF4IiDF6nJnTJ1slQRBwSqEQegPTwf+bDBJTnS7xnL4QhG6QjJjg1kRdI/+I/++yzqK+vx4oVK1BTU4Nly5ZhypQp2LZtGwYPHtxt+/b2dnzlK1/B4MGD8T//8z845phj8OGHHxpdutMOqrUlEghFEwgjDsntFSMPoniV2lAVIIp0hE7hI2Vqwm6OBmpMTfZEFXFIoBJ5cMKtI3eKUJhOf6iIEYUwqNYxcGqCcUK6h7j//vsxd+7c5PSYK1aswKpVq7By5UosWLCg2/YrV67E/v378dprr6FHj6ND+4YPH67Xag9xkw2/xAFI73RNCYQJ/JQFEdw6f5UamLBFGRKIrIJJLQ4ArTzY4XVthF+yAHhU+MjkHVK/4O3t7di0aRPq6uo+O0Akgrq6OjQ0NNju8/LLL6O2thZXX301KioqcNJJJ+HOO+9EV5fz2PJYLIbm5ua0h0l0CiKF0hCCqYrEQ7oNhKkLEyTap9JG0Wsi+jmopiWCMi8DNW6dh0gunDJVkYpMCD9o6LQ9CMKgNDcDpybyBqlIw759+9DV1ZWcTztBRUUF3n33Xdt9duzYgfXr12PGjBlYvXo1tm/fjquuugodHR1YvHix7T5LlizBrbfeKtO0JKrrT6hM9vTZvkc7omzFkdm2SW6b0kmqpC6cMBmRoJQWWXGiEregEIkVOE8l7YDbJE86BZGUEYcEMpEHwHz0gQpVwZGRKl1hyIabMHBqggE8GD0Rj8cxePBgPPTQQygsLMT48eOxZ88e3HvvvY7SsHDhQtTX1yf/bm5uRlVVVfpxS+OO6084kb3zVxeHo/vTygOgLhC2xxLs2FPlwqsIBvXUz7Lb6EQZPB1pITiVdCY6tQ2JTshNHhKdmch6FaoCIdIpmxQLyqiHbPRFtNgxmzBwWoLRRUoaBg4ciMLCQjQ1NaU939TUhMrKStt9hgwZgh49eqCw8DNLHTt2LBobG9He3m67fGc0GkU0qm7LOqtd6orD0WOI1ToA4vIA0AqE63kCLgpAcKILpoSBOtoA6BdFUssDoBeBsENFLLxMgaikakxHFxLwlNGMCFK/2sXFxRg/fjzWrVuXfC4ej2PdunWora213eeMM87A9u3bEY9/1sm99957GDJkiPZ63yrV5yI/8hTDMLPlz1O3UyG1BkKn8/UD5dqNf11T6usauloGjfyxW8cgehcqWusgOxTw01ivtIcpUmsOvBAGnfdEKQwqoyUScGqCSSD9y11fX4+HH34YTzzxBLZu3Yrvf//7aGlpSY6muOyyy7Bw4cLk9t///vexf/9+XHPNNXjvvfewatUq3Hnnnbj66qvp3oUNblXrXonD0eNk7+RSO0PRTrHbMTIkwm+ZcGqPSr2C7DUR3barRC9SYDotEYnJfw9EftypxIGqUNIJLwTCFLqiICpdFMKQDRYGJhXpmobp06dj7969uPnmm9HY2Ijq6mqsWbMmWRy5a9cuRCKfdQxVVVX47W9/i/nz5+Pkk0/GMcccg2uuuQY33HAD3btQQKxGwT1VAYjP/SBdy5DR8cmkMdKOI9hJSxVceiAjyhEYzfqF9GMpNcEbXGobRFa+pJq/QSZlkYpo+iKBW+dLkdZQgVJmVORKRBhEZIHTEowMSoWQ8+bNw7x582xf27BhQ7fnamtr8Ze//EXlVFlxK4jMVtugKw6JYwBy8qAiAKmdoapAuB4/ACkOvUmXxKMLFAR9mmkKcQBAKg+pZHaSshKRimjnLSMXXkQ3VKMwVLIgAkcZmEzyfu0JCnFIHCcBxSgLkf11jhEUvBAFQL6TNzW9uJfoigOgJg+i4pBKageqIxBuBCHNoZOuAeiFIRejDF3thbAK9WQn3s6y5ETOS4PISAoqcUg9HmBeHlKPoXsc01CNZFA5DqUwBAqB4ZcU4gCkd0QUoyzc8EIgvMQLUUhAOaSSowyMHTkvDaJQi0PimIC4PCSgiECIQiEapoc26hxfrmiV9nhBmUKaShwSiEYfdOUBCJdA6MpBKipDKH2LMPBskHkFS0MKJsQhcVxAvmgyFVNRhKDOlGh6HYn0c2mfqvu5vRIGwcmeqMUBkJeHBBQRCFEoRYNSCuxQnWtBNrogIgzCUQYWhrwjL6RBZrInU+KQODagttomVTQiyHgpCkfPZ+bYQYkwZGJCHAD1oslUdKIRbpju6HXRnZTJ1/oFFoa8JC+kQRaT4pA4PqC3VHeYJYIyuqFSlGiyZsEXYZCYWtqUOADy8pCKlyLhJxQzN5qILKQiFGVgYchb8kYaZKeWFhUHQE8edMQhvS32HbFfMmEy7aEzekFpeeuQRxgyMSkOgJ48pJLawYZNICjkIBWVAsdcHBnB+E/eSIMK4mtNfPb/qvUOAJ1ApBLUmgVZKIY5mkpHAAEQBsmFrETFIYFq5EFXHBK4dcJ+CQW1GGSiMxJCRRi4joERIa+kQWUhK9logE70wa6TMiESYYBqLgSTkYXk9gp9h8q6KdSIiEOCRCekWu8A6EcenDDdeXsFxXBJo7IASAuDylToTLDJK2lQRSWNoBN9yDx3JrkkEiYmSlKtWVBpS6CEQWHZbBlxANTlAbDvFE2JRBigmlNBNQ0hPQ8DCwODPJQG1WWzdeoPdGsf7NqSSdBFwuQsihSFjaEXBg1kxQHQT10kcOo4c0kmKCdcSqBTr6A0aRMLA/Mv8k4agM9+7FVSFYC+PCSgkgiArlO2e29BnTbZL1kAAlDD4IRCtAFQE4cEOtEHJ8IYlTAhB6noFjYqz/DINQxMCjkhDW6LVrnhlzwkcOr0KGVCllwWhEzCIAyRWAHiUckRMIkfeYVUBQBteUhAKREATafsJB6mO3xVqEZA5JUwtBUCBZrtDuP79oickAZAXRwAfXnIJBdlwiuMzqFAEZnwIcKgJA6Ab/KQwKnDo5YJGYIqB4C5IZJ5JQyMcXJGGgA9cQDU5aHbcQwNo3TrUMMmFF4tCkUZOfEzJaEsDoC2PGRiSiYAf4XCK0zOn0C2yBQLA+NATkkDoC8OAJ08AN07LVMFi6FZmdEDTKRYglDDoCUOgLI8ZJLaMekKRCZBjE7o4NUES6QrUrIwMC7knDQANOIApHcUFAIBeCcR+YipeowgCEMCbXEAyOQB6N5ZUUtEAp7dsDvkS1ezLDAC5KQ0AHTikIAy+pB2XImOLh8Fw+/CzCAJQwIScQDSOwkCgQC8k4h8hVwUErAwMILkrDQA9OIAmJMHoXO7dKC5IBR+C0ImQRSGBGTikIAw+pCKTCeXz4JhTAZEYGFgJMhpaQDMiAMg16F4IRhB63CDSpBFQBZycQCMyYMIbh1nrgiFr3JgBwsDIwl9bxpA/J6Fryua/mC8J1evvbGZ99oKxR8e0BUrzImHZwTos8sXli9fjuHDh6OkpAQ1NTXYuHGj6/bPP/88xowZg5KSEowbNw6rV69Oe/2FF17AV7/6VQwYMAAFBQXYsmVLt2M0Njbi0ksvRWVlJXr16oXTTjsNv/71rynfVjfyQhoA/8UhFRYIb8iX6+z7lL3cEQUD/gx849lnn0V9fT0WL16MzZs345RTTsGUKVPwySef2G7/2muv4eKLL8acOXPw5ptv4vzzz8f555+Pt99+O7lNS0sLvvSlL+Huu+92PO9ll12Gbdu24eWXX8Zbb72Fb37zm7jwwgvx5ptvkr/HBAWWZRHHN+lpbm5G3759MezuOxAp0YvDm0hV+I0f9RVekIudvUl5JU9VUOBDmiOvCKggZIpsvK0NH96wCAcPHkRZWZmRcyb6iar/vB2RUr1+In6kDbvn3yTc3pqaGnzhC1/Az3/+86P7x+OoqqrCD37wAyxYsKDb9tOnT0dLSwteeeWV5HNf/OIXUV1djRUrVqRt+8EHH2DEiBF48803UV1dnfZa79698eCDD+LSSy9NPjdgwADcfffduPzyy2XesjA5X9OQieqPdpBlIxc71zAQpOgVYKjGQReqTi1X5SOgnT5zlObm5rS/o9EootH0H9z29nZs2rQJCxcuTD4XiURQV1eHhoYG2+M2NDSgvr4+7bkpU6bgpZdekmrf6aefjmeffRbnnXceysvL8dxzz6GtrQ2TJk2SOo4MeScNqqR2EEEWCMYbgiYMCXRSFYETjlS4c/UF31NfCkTaCxCJaLa7/ej+VVVVaU8vXrwYt9xyS9pz+/btQ1dXFyoqKtKer6iowLvvvmt7+MbGRtvtGxsbpZr53HPPYfr06RgwYACKiorQs2dPvPjiixg9erTUcWRgaVAg0WGwPOQnQRUGXRIdRKDlgfGMMAoDNbt3705LT2RGGfzmpptuwoEDB/D73/8eAwcOxEsvvYQLL7wQf/rTnzBu3Dgj52Rp0ICq82D58IZc7eypYXlgWBiOUlZWlrWmYeDAgSgsLERTU1Pa801NTaisrLTdp7KyUmp7O95//338/Oc/x9tvv43Pf/7zAIBTTjkFf/rTn7B8+fJutRFUsDQEAO7MmCBC1XGwfHgLd/jeUlxcjPHjx2PdunU4//zzARwthFy3bh3mzZtnu09tbS3WrVuHa6+9Nvnc2rVrUVtbK3ze1tZWAEfrJ1IpLCxEPG6uT2FpYBjGKNyJMblOfX09Zs6ciQkTJmDixIlYtmwZWlpaMHv2bABHh0Yec8wxWLJkCQDgmmuuwdlnn4377rsP5513Hp555hm88cYbeOihh5LH3L9/P3bt2oWPPvoIALBt2zYAR6MUlZWVGDNmDEaPHo3vfe97WLp0KQYMGICXXnoJa9euTRuVQQ1LA8MwDMNoMH36dOzduxc333wzGhsbUV1djTVr1iSLHXft2pUWETj99NPx9NNPY9GiRfjxj3+Mz33uc3jppZdw0kknJbd5+eWXk9IBABdddBGAz4oxe/TogdWrV2PBggWYNm0aDh8+jNGjR+OJJ57Aueeea+y95t08DQzDMIy3eDlPA0U/4UV7wwpHGjThfK23cKibYRjGP/JaGrjDDx9UnxnLhxw8wscbuCiaCTp5Kw0sDPlN6uefywLBnX24oPq8WD4YU+SlNLAwMKkkvg+5Jg8sDPlL6mfPAsFQEmppyMnOP1fn2E8Q4OmAVb9PQZQNFgYmQeK7kC/yEDkSQcTS/P638b8fJ0ItDYEk1zt9XSiuT8DEIx61jIpDLgoAr8zqParfo3yRDUYMlgYqWBa8I/VaB0QgTItD0MjVTl8XiusSNPGIHImwODBJWBp0YFHwn8zPwEeJyHVxYFHwhszrHASJYHFgErA02BEwGSiMBqs9qnTFPOjQZT47A4KRi+LAsuAvqdffT4FgcWAAlobu+CgMuSIHTri9P0+EIpPEZ00sD2EQh6CJQGGb3y2gocvwhLUyn5sJwWBxYFgaUjEkDLkuAxTIXiNSyTBQIxFkcfBTGHJFDpxwen+mZcKOxOdMLQ8sDvkNS0MCYmEIkyhES9ptn4+1FXvcEnFSr68RgSCQhyCKgylhyHUZ0EX2+lBKhon0BotD/sLSQCgLXomCUycf9POYkpDM604iEUTyECRxoBaGMIlCkU1bOwO89l3qtTUhEBTywOKQn+S3NBAJgylZ8EoOvMLt/VAKBalEEMhDEMSBUhi8kgW7jj7IxzclIZnXm0IiqOSBxSH/yF9pUBQGFgQzmEyR2H1m0iKhKQ9+igOFMJgUBdNy4BVu74NSKCglgkIeWBzyi/yUBp+FIYiC0Cvq3KaWmH+1DaZkIvFZeikPfoiDqjCYkIRckQMVTAqF3WclKxK68sDikD/knzT4JAymRcGt0w/KsSnlI/N6qkqE1/LgpTj4KQxBFAS39+XH6IYETtdKRyYS79VLeQiKOERiQKHuP7GADUkOEvklDQrCoCMLJkTBpByYxq7tVCKRuNY68qBU+1DSFZiprFNREYagy4LJFAnFsanFI/VaqgqE1/IQFHFgzBFaaZBekVBSGFRkgUoSwiwGslCLROpnICsQyuJgEJVFhmSFQafDNCEJYRqVkQpFmsAJXYEobFNrS2EsGNNYM8EhlNJgUhj8kIV8kgQRqERCJfqgJA6Gog1BFQYqUQirHMhgQiQS119WHlTFgWFSCZ00mBIGr2XBpCj0jvqbkDsco781Sb1esgIhG30IQsTBtDDIdtgUopAPkiAClUioyIOKOHC0gUkldNIgRcCEgVoU/JYDJ5zaRSUTvaLtyikM0eiDtDj4XNsgKgxeRxbMDtWUvIEgpLOEtphVZzIn2dQFRxwYHXJbGgSQFQZZWdAVhaCKgQqZ70VHIhLXVUceyMXBJ0SEQbXzVhEGSlHwUwzccGoXhUzodOqi0QfZc3C0gUmQu9IgEGWQEQYZWWBREINCInTkgVwcfIg2mFpLQkYWaEZdBFMOZEl9HzoCoTrq4bN20IsDwwC5Kg2E60l4FVkwLQrlUbrbvwMxM780OmkNVXkIc8TBREpCVBZ0RSFXJMENu/coKxI68kAtDhxtYIBclAbCOgbT0QVKUaCUAhPn0hGN1OuUTSBU5IFUHDyKNlBHGLyKLJiUBcpjU9csJFAVCVV54IgDQ01uSQNRhMFkdEFHFLwUA2pS204hEKLyAIgJRJgiDpSjJGRrFtQKKWk6cy+jE7Ln0pEMmZSGijxQigNHG5jckgZBKBedEhGGfBUFJygEQlQeAPHoA5k4BHSWyExMCYNu5x7G1AVVLUPiOKLyAIh19hxxYKjIS2lwgzoloSoMpmVhQLRFep9PY73I26ErEL2jMeECShF5CFPEIRtunbyJdIROZx9GUXCCQiBE5QEQjz7kizgUxgDtf535UYuuRN5JA0WUwZQsUImCihBQHldVLjLfv6hEyEQdgOzyQCIOIYk2ZENEGFQ7fC9EoahVfB2Ezp7yE2plPX/Ge5SViKI2S3gfEXmgEAdOUeQ3eScNbmSLMojWLsgIg44omJIDXZzaJSsTdtfGTSRU5MGoODjgxUqXulEGk9EFClmQkQETx1QVDJUohEzUAcguD/kScWDMkFfSoBNloBSGXBQFETLbrhKRKI+2ZY1AyNY7+CEOJtEZ3SA3PFO881eORhiQAwqc2iUjE7IjKVTkwZQ4cLQhf1HS5eXLl2P48OEoKSlBTU0NNm7cKLTfM888g4KCApx//vkqp9UimzC4RRn8FoYB0ZbkwyT9e7R0e5hE9X2VR9uErqNoxMft89VajMxlNI/0GipEuEUZqIWhqM1KPmQoao0nH2FDt+0i10vmmnoVcUpFZd0UJjxIRxqeffZZ1NfXY8WKFaipqcGyZcswZcoUbNu2DYMHD3bc74MPPsCPfvQjnHnmmVoNNoHuKpWinZOsMFBKgqoAiO63v0OvSDL1vYpGIBLXkyJloRNxCFK0wfSiUKKyoHRsDySh6Ej6OTpLzXVwqe9HNp0hUssgWu+gG3FwPC5HG/ISaWm4//77MXfuXMyePRsAsGLFCqxatQorV67EggULbPfp6urCjBkzcOutt+JPf/oTDhw44HqOWCyGWOyzjri5uVm2mWRkizJQRxdURcF0VED3/DJSISsQVPKgm6qwxcOCSJ05GagKHlWiClRkCgHlPrpyoSIQIukImSGaquKgUt8QORJBvDR8kSImO1L/Etrb27Fp0ybU1dV9doBIBHV1dWhoaHDc77bbbsPgwYMxZ84cofMsWbIEffv2TT6qqqpkmtkNt9SETpSBShhUw/RepREoUG2rzLWhTFnY4fZdUamX8StFoQKlMOiE8IuOxB0fJnE7r+y5Zd8/VcqCaghu2jF5aGLeISUN+/btQ1dXFyoqKtKer6ioQGNjo+0+r776Kh599FE8/PDDwudZuHAhDh48mHzs3r1bppnC6IyWoBQGWcIiCk6ott8LcaBevjwoqEYZRPPnQtsQiEJQ8UIgTKaGju6rvKstXNtgz/79+zFjxgyUlZWhvLwcc+bMweHDh133eeihhzBp0iSUlZWhoKCgW7T+gw8+wJw5czBixAiUlpZi1KhRWLx4Mdrb03/PLMvC0qVLcfzxxyMajeKYY47BT37yE6n2Gx09cejQIVx66aV4+OGHMXDgQOH9otEoolHJZBnhIlWAnjCYkAXTkjCwh/OXdl9HbyPnTLwn2dRFtpSF6AgLlVU1ldMURKjc2VEVP9ofW0wWpI8bYEHIRqLtsimNotZ41tSFbq0DD6X0nxkzZuDjjz/G2rVr0dHRgdmzZ+OKK67A008/7bhPa2srpk6diqlTp2LhwoXdXn/33XcRj8fxy1/+EqNHj8bbb7+NuXPnoqWlBUuXLk1ud8011+B3v/sdli5dinHjxmH//v3Yv3+/VPulpGHgwIEoLCxEU1NT2vNNTU2orKzstv3777+PDz74ANOmTUs+F48f/QdVVFSEbdu2YdSoUVINlsUpbKw6WsJrYaCUBTcxUN2PQihk5YFKHJxwq23wApU7NKfOX+fuUSQknvUYksJgWhYKj3R2e66r1My9k4o8eCEOKrBs0LB161asWbMGf/3rXzFhwgQAwAMPPIBzzz0XS5cuxdChQ233u/baawEAGzZssH09IRQJRo4ciW3btuHBBx9MSsPWrVvx4IMP4u2338YJJ5wAABgxYoT0e5D611JcXIzx48dj3bp1yWGT8Xgc69atw7x587ptP2bMGLz11ltpzy1atAiHDh3CT3/6U+1ahSSEUQad8DSVMOiIgqoYmDqfjFTIyAOFOKhGG/yCOn+cLS2h8lradqI5e0JRsJMC3X10pUJWHqjEwQlToylykcwifKUoeAoNDQ0oLy9PCgMA1NXVIRKJ4PXXX8c3vvEN5WNncvDgQfTv3z/5929+8xuMHDkSr7zyCqZOnQrLslBXV4d77rknbbtsSP9rqK+vx8yZMzFhwgRMnDgRy5YtQ0tLS3I0xWWXXYZjjjkGS5YsQUlJCU466aS0/cvLywGg2/NhwC3KkE0YRKMLKsLgtSjIkGibCXkwGXFQGUkRpKGX2TApDF7IgoocmDiXjFDIyAOFOFBHG8JCURtQqFljXPCvn/rMG9vFixfjlltuUT5uY2Njt6kJioqK0L9/f8e6QBW2b9+OBx54IC01sWPHDnz44Yd4/vnn8eSTT6Krqwvz58/Ht771Laxfv1742NLSMH36dOzduxc333wzGhsbUV1djTVr1iSLI3ft2oVIJBgFMLKpCdW0BIUw5JosZGJKHnTFIWzRBjtkUxOqdQxUwqAqC16KgiipbRIVCFF5MB1xcD4mRxsS7N69G2VlZcm/naIMCxYswN133+16rK1bt5K2zYk9e/Zg6tSp+Pa3v425c+cmn4/H44jFYnjyySdx/PHHAwAeffRRjB8/Htu2bUumLLKhFHebN2+ebToCcM65JHj88cdVTukMUWoiLMLghSgMLDoEANjX2Yf+2BryANgLhB/i4HVBJFVqIvtcDmYnZpIVhiCKghOFRzrJIw+64uD0mkqNQj7WNZSVlaVJgxPXXXcdZs2a5brNyJEjUVlZiU8++STt+c7OTuzfv9+2LlCWjz76COeccw5OP/10PPTQQ2mvDRkyBEVFRUlhAICxY8cCOHqzb1QawgDFapY6BF0WEmIguw2VSKjIA3D0mpkQBzv8Loj0GtW0hInoQphkIZVEuynlwZQ4uB+Pow0yDBo0CIMGDcq6XW1tLQ4cOIBNmzZh/PjxAID169cjHo+jpqZGqw179uzBOeecg/Hjx+Oxxx7rFvE/44wz0NnZiffffz85AOG9994DAAwbNkz4PDkrDU7ITuakE2VwQ1QYdGVBRA6ojqciFKqRB2pxoEpTeF3XQDVqIgjCQC0Kha0dSvt19eyhf25FeTAlDrZtzMPIgd+MHTsWU6dOxdy5c7FixQp0dHRg3rx5uOiii5IjJ/bs2YPJkyfjySefxMSJEwEcrYVobGzE9u3bAQBvvfUW+vTpg+OOOw79+/fHnj17MGnSJAwbNgxLly7F3r17k+dMRDDq6upw2mmn4bvf/S6WLVuGeDyOq6++Gl/5ylfSog/ZyDtpsMMpNaEqDNmiDCLCoCIL1IJAdX4RmZCVB68iDiRTS3s4lbQbzoJhLiXhhTCoyoHs8VRkQlYeTIlDvhZFBpGnnnoK8+bNw+TJkxGJRHDBBRfgZz/7WfL1jo4ObNu2Da2trcnnVqxYgVtvvTX591lnnQUAeOyxxzBr1iysXbsW27dvx/bt23Hsscemnc+yjv77jkQi+M1vfoMf/OAHOOuss9CrVy987Wtfw3333SfV/gIrccQA09zcjL59+2LY3XcAfTPuBm1qGmSnjbaTBhPCYCK64LcoyCATiRCVB6ciSZE1K+zEwS7a4JaisJMG20iDgzREYt1/yO3mabCraZCdFlhFGlSjDKZkgVoQKJCRCVF5cKtzEFm7wk4QnKTBLdpgl6Kw295p4arE+hPxtjZ8eMMiHDx4UKhGQIVEP3HiVXeiMKoXQumKteGdX/zYaHvDSjCGOeQIusIwsMdhYWEYWHQoVMIAyLVZ9Fo4XVfKFUKNLZ0dAPwShsIjnVLCUNjaEUhhAOTaJvq+3a6h6pTTOlNMM0yCvEpPUPzAq9QxUKYjTIhCZY8DWbdp7CgnO5/M6AyRtIVqqsIuTUFR2+D3fA0UawiYFgYRgioJTiTaKxJ5EElb6KYqROHJnhgZ8koa7JCZAVKnjsEJL1MRInIgu6+OTFDKg06NQyZ24qA9ksLnugaZuRlMCYNsZIGSSIv7mNV4L7q5OijlQUcc7OoYKGobuIAyv8kbaZCJMtjVM5ioYzAZXdARBN3zyIrEwKJDwvUOA3scJhUHnTUqgohMlIEyXE0lDBSykE0QZPbRkQkqeaCOONiJg64IFMac6xqY3CJvpMEUJoVBRha8kgQRVERCNupgOuIgE22wG0Xhd4pCFd3hlZmYjC6oyIEsFDJR2NohXCzpNEGUqjjwqAmGmryWBp3FqbLhlTAESRbcSLSTSh4oxYFqCGZWfEpR6C5/DailJUxEF7wQBREy25FNImSjDqYjDjLRhrDVNRTG9NeeQLjrm43CoycyyLb0dSoqdQzZhEFmhIEJYRhUeDjtQU1ljwNC7Ra5Dm7XUnZUhV36Sea7oEo86k9Fu0xqgmJopR0yow4iLbHACIMdou0Tfc9OwqUyqkLmsxYVTAoRZcJJqKQhXuzPD6zsiAnVpa39Ske4SYIpgRBtv4nRIpTDMXMJ2RoHtw4sW5TBd1loOXL0QYxoW0XkwbQ4UA/BpF66nQkmeZue0E1NOHU8qmkJL6MLqgKQud/eLrl1IzKp7HFAqGjSrVDSLU1hiqCuSSEzoZPwMQ3UMcgIAxlOgpD6fK9SklNFWmLCNQ8y9Q6peDUck2EyyYtvlujICdlREzL4LQwmIgYUxzQZcVCN+KTiRYrCayjuMJWXt/ZSGBLRBNGIQur2mlEImfa7XZOwLtzF5C45F2nwYnVLlSiDE6aEwUQ9gsz5ZKIQuhEHlWiDyvwNbni+VDZxTtk2fO0U6lZMSxgXBup0g2YUwnTEwS3aoAPPw8C4kReRBhFM3lHqrlQpKgwmCxhlkW2LqVEgMiJHFVWiJiy5Yl+EgSgyIHUeiXNRRBxUog2iaSWeWpqRJdzSYLNYFSUynYipKIOMMAQVSnFwuma6YpaL2EUjdAviVNISIsIgVfDolSgQtcHkqA+d0StS5wmmTzM+EG5pUESnCFK28l6nM8sFYUjgZ8TBxKJWJuf4CAKyBZA6uXepTtVPUXCCUBwoow2ZcFSBoSAvpYEap05Jp/gxl4QhAZU4cLShO0G9E8wWZZCOLgQVHyMOTtEGlZEvDJONnJcG1ZUtTea3gygMgyIx1wfZeXyKOIimj/ya5EkG1SJIU6kJ1btgKWEIA0TiYDLaIApP3sQ4kVPSoDpyQrRTkAlnq971inSWVMWOMlJAKRAU4kAVbaCc6ClTUL0YyWMCmTtUiuJHV4iFwWpttX2Q4VPEQbS2wYtJnpjcJueGXHqNbAGkW5RBVBhUoYoYpB5nb1xtabtBhYeFhmW6DceUWR0TcF6TgsmObMEdSVqCQBhEhSB1u4KePfVO2nIk6xDNbMMxnYZgOq1LwXxGUZuFwrieCBW0s0g5kVORBpNQ3JH6JQzUKQa7Y6sc31SqQre2QTQ1FeRiSJHwsl93mKaFQTeCQBKB8CHiYCd2lHUNmXUznMLIT/JOGkR+6EU7DZUCSFVkhcFEPYKJc+qKg+xMkXafmYgQBq2uwRQUqQm3KIMpYTCSatA9rqY4mKpt4HQEo0NOS4NqEaQJdKMMIvghChRtMRFxyKeRFKZHTsikJrwUBlOiQHo+Q+LgBRxJYOzIaWnwA9nOiiotEQRRcIJSHOygiDZkEtTZIVURubukKoC0g1IYvBSFbG0QaoeBVIXMCpginytHHxhR8l4aRMLOduFrisWQ8kEYElCJQ+ijDYZnMaXGqxkHRTrWIMiCHabFwUS0gSWBUSXvpcELVFZoBPwRhkGFxWkP0mMbjDioXuMEqnUNmTUyuTLsUgWnzi3rXbSgMAQZPyIOMtEGhqGCpYEQmTvbbHfMXgqDmyRQCwSFOFBEG3I5RWF65ITXyzWTFzcebkk+SI9LJA526EYbvEpRZC6uFjnCXUyukVefKNXICZnUhO4dsBM0kyzJyQCVPJiKOJi61vkKRQEkRZSBAidRSH2eQiIoxMGraIOIJGQKKA+7ZPJKGsKCF+tJ6HT+1GkL0wSytoHxLC0hIwNBEQc7/BxJwTAJWBqICFPHRBItCJk4MDR4nZrQQTV6QJ22YJhcgqUhhOikJig7e61oheFhmMxnUA+39AqdKINuxx8mcQiTyDHhhycxzwLlgkYJ3Ar5THaUJqIDgwqLsbfL+0m03NakUGVAtAWfxtzXpugdjeFwTG29jXzDNTdvsJaBqsO3DregoLfaWiVWa6v+GhaMEoUxC0Waa0+gg4ekOpHXkYZ8mRoYMJtO4FRFfhOkXDv5iAiTEQcXcTKxEibDUJDX0mAaE9X8agtDme/UVc5hYlIqHkGRG6ikJkx18GFKVTCMaVgaAoSJ1ISXUQAz6Q9z6RqKWT3DiFCNg8kJggykJkx37LkgDkGsW2HCB0tDDuNH2oBTFblLUAvuvOrQZc9jYuil6VQQTy/NZIOlQRK7u1PKmSDdkFty2r/OW2rCKI0URejXochhdHLyMqkJryMAuRBxYBgdWBpSCOu0wZkE4W6fdmgnd/hMd/zqwMMiDrwGBWMCloaAQNUxBkEYEgSpLYwZpMPlRPUMfnfcoufXmWuCYjpphqGGpcEQ1FX8YpMhBa+TFmlTkJf2zpXokxtBKpAL+mqWqZCJi0frbzDBYP/+/ZgxYwbKyspQXl6OOXPm4PBh55vG/fv34wc/+AFOOOEElJaW4rjjjsMPf/hDHDx4MLnN448/joKCAtvHJ598ktxuw4YNOO200xCNRjF69Gg8/vjj0u1naWByDh52mdv4HWVIJUhtYcLBjBkz8Pe//x1r167FK6+8gj/+8Y+44oorHLf/6KOP8NFHH2Hp0qV4++238fjjj2PNmjWYM2dOcpvp06fj448/TntMmTIFZ599NgYPHgwA2LlzJ8477zycc8452LJlC6699lpcfvnl+O1vfyvVfp4RMgBQpCaCGGVIQDFr5KDCw9jb1ZuoRc6IzArJuOMYVie4ow5jJ009O2Rhawe6evYgOVZRm4XOkgKSYzHZ2bp1K9asWYO//vWvmDBhAgDggQcewLnnnoulS5di6NCh3fY56aST8Otf/zr596hRo/CTn/wE3/nOd9DZ2YmioiKUlpaitLQ0uc3evXuxfv16PProo8nnVqxYgREjRuC+++4DAIwdOxavvvoq/vM//xNTpkwRfg8caXCBegppnZETuY5qioKvKeM3JCLDKYpA0tzcnPaIxfRSqQ0NDSgvL08KAwDU1dUhEong9ddfFz7OwYMHUVZWhqIi+/v+J598Ej179sS3vvWttHPX1dWlbTdlyhQ0NDRIvQeWhhAQ5Jx/mDA17LJX1Pu1N5woNFSC4UWhXZjqGbyCp5OWp6g1TvIAgKqqKvTt2zf5WLJkiVbbGhsbk+mCZHuLitC/f380NjYKHWPfvn24/fbbXVMajz76KC655JK06ENjYyMqKirStquoqEBzczOOHBGXVpYGTYIw/j/IqYkENMtx01/rfJ0V0o2gDtULY2oiQVCEKEhFr2Fg9+7dOHjwYPKxcOFC2+0WLFjgWIiYeLz77rva7WlubsZ5552HE088EbfccovtNg0NDdi6dWtazQMlXNPgMzwHgRkGFh3Cvs4+vpw7WtKOWFvwRc5Tcjz8rrMiJhWFRzrRVco/6ZSUlZWhrKws63bXXXcdZs2a5brNyJEjUVlZmTaaAQA6Ozuxf/9+VFZWuu5/6NAhTJ06FX369MGLL76IHj3s61oeeeQRVFdXY/z48WnPV1ZWoqmpKe25pqYmlJWVpUUkssHfMJ/Z29WbxYFhGCbEDBo0CIMGDcq6XW1tLQ4cOIBNmzYlO/X169cjHo+jpqbGcb/m5mZMmTIF0WgUL7/8MkpKSmy3O3z4MJ577jnbNEptbS1Wr16d9tzatWtRW1ubtd2pcHpCk30d5iv6s6E7MiEX8SvKkAt0lhr4WeglfifjhN938m4EuW1McBg7diymTp2KuXPnYuPGjfjzn/+MefPm4aKLLkqOnNizZw/GjBmDjRs3AjgqDF/96lfR0tKCRx99FM3NzWhsbERjYyO6urrSjv/ss8+is7MT3/nOd7qd+8orr8SOHTtw/fXX491338UvfvELPPfcc5g/f77Ue+BIgwufxnqRj6DIVyjExoshl2Gnq8RMMWRXaRHPOqgB5ZBLJzg1EQ6eeuopzJs3D5MnT0YkEsEFF1yAn/3sZ8nXOzo6sG3bNrT+qw5m8+bNyZEVo0ePTjvWzp07MXz48OTfjz76KL75zW+ivLy823lHjBiBVatWYf78+fjpT3+KY489Fo888ojUcEuApcFTGjvKlYYI7o1HeQSFIfZ38B0iNfFeUaWq/4KePQNTMBgU4r2ifjeBIaZ///54+umnHV8fPnw4LOuz1UYnTZqU9rcbr732muvrkyZNwptvvinWUAc4PREAKO6gw56i2Bs3/+NoKpXUEgt30WNnT49+BnI0RUHSJoJrI4JnnzWTs/A3KIUDMfviEhU4p/4ZJlMTjR3l2semJp9GTlDNTMgwTDhgaQgJXtyJM/lBkKcNFsn9BynaINIW6noGFjXGT1gacoggpihE2sRCxMgSBHEga4NHqQmGoYClgQCZXLlTOJ1qZECQxMGvtuikhnixKsMQdpB+ikMQpIVh/IBHT0iyv6OXb1MPi46i2NvV7vvU0pTCwEMtg4HMsEvVERSA3CiKgt69PJ9eWkYYdFITMiMneLjlZxS1xVHUqTldtu7+OQxHGnIUPyMOQYp2MLmPl3f9YYowiEzSFeT6FiaY5LU0HI6ZzaXLhsmp76j96Lxlz6lTzyAzciIIM3eGCSOzQiYwkMP3ojM3cg6uZ2BCRl5LgwhBy3HLdrJeioOJc5lMTYhM7EQ5DDdXMVHNrxLWNykOKsf2YhZIhvEaloY8wAtxUDkHj5pgqDEhDkFKSfBwS8ZvWBp8wC2sburO2qQ4BKmGgSfV6o5I3trETIF+TYFM2cn7JQwmrp3IZ8w1Dkw2lH4pli9fjuHDh6OkpAQ1NTXJ1bjsePjhh3HmmWeiX79+6NevH+rq6ly3Z7KjeoduJn0QHGFgQoRALl8nvE/R2escQ6jtXM/AhBBpaXj22WdRX1+PxYsXY/PmzTjllFMwZcoUfPLJJ7bbb9iwARdffDH+8Ic/oKGhAVVVVfjqV7+KPXv2aDc+SISl0I52KKT6sUTExy3qEsTpo5lgodXpByglkQ0ebsl4ibQ03H///Zg7dy5mz56NE088EStWrEDPnj2xcuVK2+2feuopXHXVVaiursaYMWPwyCOPIB6PY926dY7niMViaG5uTnswdNCsBZF7EQaRolfTI26CTtg6KKUCRi+EgaMMTEiRkob29nZs2rQJdXV1nx0gEkFdXR0aGhqEjtHa2oqOjg7079/fcZslS5agb9++yUdVVZVMMwOFSo7di8mMtKIEIROGsESBwoxSgZ7hFEXyGL17ia0RIbhd1uMYGjXBRZBMEJCShn379qGrqwsVFRVpz1dUVKCxsVHoGDfccAOGDh2aJh6ZLFy4EAcPHkw+du/eLdPMJJmrDYosYSwyxE5kqF42dMPrFCMP9na1Jx8y22ufVzM14QQXQYojVBzpMFeDTLQha0GfR+IAfCYFqWJg95zWOQhqGSiKIKkmduriEcdMBp6OnrjrrrvwzDPP4MUXX0RJifO3MRqNoqysLO0BAJH2jC95W2Han12x9L9FEAk3i87V4HRHayraQDlk0UkgZMUi63kIhIGinoFqjgYREaUm84e8s8T99aPbmB9BoXwn7KE4JI9HKArJYxosfnS6tmFLFzHhR+obN3DgQBQWFqKpqSnt+aamJlRWVrruu3TpUtx11134/e9/j5NPPlm+pTlIY0c5KnsccHx9b1dvDCo87HoM0fUoZDCVfjAVYXAjF1ITXVGgkPYjFqazNIKiI93n4Sdfh6JXKdByxHUTmTUpvIZKGLyKMuQyhUc6UVgk9t10wurU2z+Xkfp2FRcXY/z48WlFjImixtraWsf97rnnHtx+++1Ys2YNJkyYoN5aj9BJUVBGGwDvIw6moBIGpyiDzPWlSC/lAibG5DvdEQt1hj5EHCjwSxhkogw8RwNDhbSS1tfX4+GHH8YTTzyBrVu34vvf/z5aWlowe/ZsAMBll12GhQsXJre/++67cdNNN2HlypUYPnw4Ghsb0djYiMOH3e+gwwzFUtmpiIpDUOXBtDA4IfM58MgJeWRD45TikHj4BXUb3K5NkAogM1NhTP4hnRCbPn069u7di5tvvhmNjY2orq7GmjVrksWRu3btQiTymYs8+OCDaG9vx7e+9a204yxevBi33HKLXusVaIkVo1c0Pfx+OBZF76h7+PTTWC8MiKYvwSu7TPa+zj4YWHSo2/PZ0hSAWKoCSO+gqdMWMlALjJswcAGkGJ0lBShqs9y36RlBUWt6OsIpReFEV88eKGztsH2NKlWRILPTNpm+UBYED4dX5ntqgjGPUhXNvHnzMG/ePNvXNmzYkPb3Bx98oHIK3zkQK0F5tE15/30dvTGwh3g0hVIcktt7LBDqM1V6V3dgOjWROWLHL7pKgEKBr6+ISLieR6K2IQG1OKSS2rFTCIR2JEEzLUFRAGmXmuCRE4wqOaelmSMoTP+Iy3ZCbnfFVKkK2/3+lb6gjgDoHlc3LeF0PXVTE0EdOeEXsnew2ULqVKkKN1RTCGSpB0PC4ARHGRgv4G+ZBLLLZKtU7psUh+T+uh09kYCYEoZ8INuwSx1kh1463fUGQRwSuIlA6mskNQq9So2OlAjSMEuORuQfLA3/wq7ITeRuE6CNNgDeiEPyOAICkLoNzaRSvY0UPiZwkjUeNWEflhatmjdxJ+ulOCQgl4RUBNua7X1TFT+KpiZ45AQjSl5Kg5dhZdV5ArwUh+TxMuTASDpDsM3Z3j9VlEE0eqQ6ckJlwrEwohptAPwRByN4IAxO19mL1ASPnGCAPJUGHZw6GepoA+CPOJiEShjcoIgyiEaYMglKEWQCndCxU4rCVOcUenEgEgZKdGf4ZBg7+FvlAToTPuWKOFAKg9dRBkYMnWgDICEOQZIHifaIvD/TUQbRNATXKjBOsDRkQeau01TOXFQcgigPMu3SFQaKKaNVowxhRyanbTIULnwnnuis/RAIhXPrRhhMFj9S1jN08fxnOU9wynANEmsrRrQk+3oKIpM8AfYTPWXDad4GpwmfMhGZxwHofkcvM68DFSryorsIlZsw6MqcXT1DEIdbdpYARYRzM9hN9JQNp3kb3CZ8ykRoHodUUjtvhbkdpM8hiagwqBQ/OgmcF6mJoEYjCo90orBQ7LvmhNXFa084kRfSYIfdzJB+QS0OqaR24CYFQifKISoMKmkJJ2HwLTXRlhuFkbKzRAKGxSFBZueuKhFEEQwKYaCKMmiNnAmoIDDek7fSYAq3qaXdZok0KQ4JKAWCKhVCIQxUK1lSpyaCMHJCdGZI0nO6zBLpiTikIhOFIEx1yKQjVIXBVJooqBEEJhiwNCiikqLIhhfikEAljUFZMyGTjlAVBooaE9GhlkEbOUGFW4pCJdoA+CAOCTyqf6ASBjfchEF12mgTxEvlvx9MsOFCSAFk70DdOqtsd8WiIXjdGoBMEgWLqYWLds9RQCUMbrh9BrqpCdP1DF780MqEqt3y406dV7aQelfPHlKjKrwcqqiDTDoi2/tXSUvI1DLwhE6MCuGXBpscsWhY2O7HX2YSH7fOR1ccRIdjUstDAlMjMWTSEdmugUpawukzC/qoCZ2qdJlws5fiAMjLQ1ARFRvR9+tHWgLg1ASTnfBLgyAy4WOZKaVNiQMgF3UwJQ9UiLZRVJhMpyUA9VkgvcLuB96pYM1+W7k7TVPiAIh3ponOOfPhJarnF5UjVWHgyZwYLwhdTUMkVoB4VH0p30woRlG41TeoFkYmt/lXBypa65CJbu2DCsprRghKkqowUIyYCOJQS2rchmSq1DjILJ+d6FhFax4S2HXcFLUQFEIiU7dgQhh00xA8coJJJTfU1MAwNtkFrExGHADxO/BMEnf4mQ8qKI4t895MCINuasKrIkjdiXMoog2AesRBJkdPsWCTTFTCVARDJrqgOrRSRRi4nsE/9u/fjxkzZqCsrAzl5eWYM2cODh8WG81mWRa+9rWvoaCgAC+99FLaaz/84Q8xfvx4RKNRVFdXux5n+/bt6NOnD8rLy6XbnxvSYIPMcDeZu0dT4mBaHjKR7exNyYeMLKgOrVQRhqCnJhJQ5KBVxu+riAMg1znK1DvI4EV6Q6btItdDpY5BVgy4nsEbZsyYgb///e9Yu3YtXnnlFfzxj3/EFVdcIbTvsmXLUFDg/Ll+97vfxfTp012P0dHRgYsvvhhnnnmmVLsThC49oYPbzJB2aQrRGSJTUU1VAJ/dRWdLWQByaQtRvKyLkBEfEVlQqWOQjTCEJTXhNjMk5bwNOsMxEx2lSNpCNWXhNbKCIypPftYxcGqClq1bt2LNmjX461//igkTJgAAHnjgAZx77rlYunQphg4d6rjvli1bcN999+GNN97AkCFDur3+s5/9DACwd+9e/O1vf3M8zqJFizBmzBhMnjwZr732mvR7yNlIAxWyaQpAPeKQwI/Igxck2kodXTBdx+BGrszPoDpboGrEIUEi8kA90sJLZNsl+n47SyNG6hgoUhP5EpVobm5Oe8RienUyDQ0NKC8vTwoDANTV1SESieD111933K+1tRWXXHIJli9fjsrKSuXzr1+/Hs8//zyWL1+ufIzciTS0FQIlXVqHoJxaWifikMDvyAMFKkIjKkzZBEy1joEiNWFqNsiuKFAo8LtFFW0QWaeCagIo0YLJbB20qYgEhbBQRBcAemHIJQmItMQQ0fznF+k6+o+sqqoq7fnFixfjlltuUT5uY2MjBg8enPZcUVER+vfvj8bGRsf95s+fj9NPPx1f//rXlc/96aefYtasWfjVr36FsrIy5ePkjjTY0BUrRGE0XSREF69KxS5NcSBWgvKo+y8vhTgAavJgh2mhoIh4BFUYgpyaoEo5OAmC1+IAiKUtHI/h0LmLyoSJaIZMkaPOPAyUEQZOTQC7d+9O62CjUfvfhwULFuDuu+92PdbWrVuV2vDyyy9j/fr1ePPNN5X2TzB37lxccsklOOuss7SOk9PSoIJMtMFLcQDk5MF2f4dOXVYmTKRDZIocdeZhoJzEKcypCVnR8FIcABp56HZMH1Ib1LIQhLkYcikqkY2ysjKhu/LrrrsOs2bNct1m5MiRqKysxCeffJL2fGdnJ/bv3++Ydli/fj3ef//9biMdLrjgApx55pnYsGFD1vYljvPyyy9j6dKlAI6OxIjH4ygqKsJDDz2E7373u0LHyUtpoIo2AN6LA6AvD92O51NNhOxoCBFZ0KlhoBoxEYSFqgDxpbLT93GZo4FAHAD4Lg+mUZr+mUAYVKMMVBKgOyQ4zAwaNAiDBg3Kul1tbS0OHDiATZs2Yfz48QCOdubxeBw1NTW2+yxYsACXX3552nPjxo3Df/7nf2LatGnCbWxoaEBX12eR9//3//4f7r77brz22ms45phjhI+T89Jgl6KghkIcACjJA5U4eInK0EkKYVCtYwhyaiKBSorCaR+T4gDkrjyYkgXAe2GQmVmUEWfs2LGYOnUq5s6dixUrVqCjowPz5s3DRRddlBw5sWfPHkyePBlPPvkkJk6ciMrKStsoxHHHHYcRI0Yk/96+fTsOHz6MxsZGHDlyBFu2bAEAnHjiiSguLsbYsWPT9n/jjTcQiURw0kknSb2HnJcGJ2SHXwLuQzB1xQFI7xhN1Dv4ieocC6KpCK+FQSk1YWASsmyoRBuyH1NfHAA9eUjgp0SoTsYE0MkCYH6kBEPLU089hXnz5mHy5MmIRCK44IILksMlgaPzKGzbtg2tra1Sx7388svx//1//1/y71NPPRUAsHPnTgwfPpyk7UAeS4MqpsUhgWz0IbNTDoJEqIoCQCcLgPmREqkEJTWhSzYxoBIHQE0eErh13BRCoSMGdsgUOeoKQzYoowb5nJqQpX///nj66acdXx8+fDgsy/3flt3rorUNCWbNmpW1DsMOlgYHVIdfUooDoJe6cIOsHkJDDOyQKXLUFYZskKYlfIgyiKAz8oJSHID0DlVFIDIREQpqKbBDaTZHAmFQrWPg1ATjRl5Ig1Ndg0pBJJB9pkhqcQDU5cEJt84+VSiopcAO2dEQosWO2YRBtY7BKTUR1CiDaopCSAqIxSG5j0b0QYRcloVs26gIA8MkyAtp8AMT4gDQy4MdXogCoDZ0kiq6EJTCx0jM/5yzW7SBUhwABE4eqFGdY0F0KKVJYVCFUxP5Rd5Lg0pBJCC2LoWoOAAIpDyYQHWOBaroAqAnDF4VQMZL44gckeuA3Dt/9YJIUXEAYFwegOAJhNZkTBLzLugKQ/Z9nV/j1ASTIO+lIRvZxAFA1lQFAGF5AOQEQmXEhdfoTMYkM++CrjBkIyiTOYlOJZ2JztTSwlIgIBhA985SRiJEOmlKsdCRAtvjSU7QJCoC2bbjjp+hILekwWX9Cbf5GrLVNmQriqSUB4BGINyglAsdIXBCdoIm0WLHbMIQtPkYqKMNgP6aFCLyICoYafukdKSyUQjb4xF39LqozuRoOrrw2TG0DxEcWo8AEc3vUFxvYapcJpTSEIkVIB4V/0ESQVccALqURSqqAuGGiY5eF9WZHE1HFxJkizJ4XQSpGm0AaBazMiUPAL1A+InOtM+UwqATZeAIBZNKKKXBFY3VLqnEAaCLOqRiQiD8xqQsAOLCoBNlCNqoCbGIgYerYCrKAxBOgTAZVVDZPlunz7UMjAyhlQaVaIPIlNIU4gCYlQege2cbFonQWRtCds4FKmFwizKYFga3FIVbtMFrcQDE6h0yUU1h2OGVVFAvGuWXLBw9ltSpGSa80uCKYm1DAipxAOTkIRWdKIQdXkqFjhjY4ZcshB0KcQDo5cFuH9n9bI8l2Jk7Lqjl4QqSqnUIIvuRLUDFQsHYEGppUK1t8FocALF6h1QyO0qVaEQqoh25k1xQi0A2VGdyFBEGGVnwM8pAga44JI4BmJWH1P1U9pU6j0/LS+sWLFILA0cZGBVCLQ2uZKltoBKHBFQpCydSO1FdgXDDazlIRWfKZ8BbYQgKIgWRFOKQOE4C0WJJ1c7fK4EwjReikIB0LQkWCsaB3JUGASjEIUGiMzItD4B3AmEaLyQhFcp0RBiiDKlQiUPq8QCaIZrZyOw4gyYRlCtJqh7LrwgDzwaZf+S2NAiMpKAUB0Au+iCbsrAjDAKhKwepqA6f9C3C4MFCVaLDL6nFIXHMBCKTQyWglAgRKCIepvBqpUpRWRA9JgtDfpLb0iAItTgkEIk+2HWCFBEIGVRlg1IG3NCZZ8GELAhHGQK4sqUJcZA59mfnsO8oTUURvOj8ZdFtk2jnbqJ2gYUhf8l9aRCct8GUOAByqQuAViRE8Krzl0V3UiZRYTBSu+CxMMhM9mRaHAC95bZTCVoqQgcKcZGtNZAVBpHjszDkN7kvDRKYFAdAfsRFKpkdqEmJ8AOKWRsB+boFWWEQijIEMMKQiUlxSBwfUJeHz9oQroJI6oiGakFiPo+MsFqPwIqoTfCXPEZc7Xc6H8gPaZCYJVJUHAB4EnVwIqwSQSUHCVSLG1WiC0EvfpSdWlpUHBL4EXlIb4tYh2xaLkynOnRGLqjKAtcxMKLkhzRIIiIOQDDkIYFIZ+yFWFBLgR06oyBUUxFhrmNwQ64O4eh/deQBoBEIN4JYv5ANiiGOKsIgc15ZYYiXhmPab0aO/JEGyTUpRMUBSO+IVIslE1BJhB1edOimoBguaTy6ICkMkRh956aykJWMOAB00YdUTItE0KCaB8F0ZCG5PQsD8y/yRxoUkBGHBDrRB8C+czQpEkGEcj4F45GFBAEQBh1kxSGBTvQh8/yZhF0kTE2QpFOvoNImFgYmlfySBoUVMFXEAdCXh1RyVSRMrf2gMxpCqW4hYMKgumy2qjgA+tEHp/bYETSZ8GL2RN3CRtU2cg0Dk0l+SQPgqTgAeqkLN8IkEqYXhqIYMqlc5OhBDYPbSpdOJH7sVVIVgF7HbEIgUsmXKY6pRkCwMDCUhF4alBatSvzQS9Y4JKAQCIBWIoDcX7UxE6r5FYIsDAlUxAHQl4cEFBGITEwIRRgxMvmSbmSChYFxIPTSAKivdqkiD0D3ToZKIlKhFoowY2LyJe3hkz6MklAVB0BdHpL7E0lEKvkoFCbnTyBbEpuFgXEhJ6QB0BAHQFkeElBJRCpOHWUuy4TpVSXJ5lnwcViljjgA6R2CqkAAZiQiFafONSwy4dXkStSpGhYGJhs5Iw2ApjgA2vKQgCKV4USuRCe8WnaafEKmAMzDoCsOCXSjD2nH8mgehnye6TAVlgXGL3JKGgACcQDI5AEwE4VwwquOOAzkoiykQiUOAK08AOajEPmKqQJQFgZGhpyTBoBIHABSeUjgpUTkG8ameQ6YMCSgFAeAXh6Sx5Xs7PJJMvweCZKLwmC1tMIq6NA7hqW3fy6Tk9IAEIoDYEQeEsh2dPkmGb6v9xBQYUhALQ6AOXkQPr9LRxpWofBbDuzIRWFgzJOz0gAQiwNgVB5EcepEwy4TvsuBHQEXhgQmxAGQ71S8kIwgdr5Bg2WAMUlOSwNgQBwA+c7EA8kIZKcbREIiArKYEgcZqEZmMPKwKDBekfPSABgSBxkyOyofIxV5S47KQipBEIcEmZ0YSwQ9LAqMH+SFNAABEIdUUjswFghz5IEoZKKyWJAXokHRweWaeHCnz4SRvJEGQG2hIOOiQdWx5Zp85GGH7xepohGUSIUd3Ml6C69WydiRV9KgQkI0AhOlcII7Wc8J2vLWFCQ6iiDLA2MeFgbGCaVfhuXLl2P48OEoKSlBTU0NNm7c6Lr9888/jzFjxqCkpATjxo3D6tWrlRrrJ5FYQU52Eowauf5diJfGtR9MOOHPziz79+/HjBkzUFZWhvLycsyZMweHDx923WfSpEkoKChIe1x55ZVp22S+XlBQgGeeeSZtm+XLl2Ps2LEoLS3FCSecgCeffFK6/dKRhmeffRb19fVYsWIFampqsGzZMkyZMgXbtm3D4MGDu23/2muv4eKLL8aSJUvwb//2b3j66adx/vnnY/PmzTjppJOkG+w3VJ1F4CMXjCO5LgxUUHc+uR794M46P5gxYwY+/vhjrF27Fh0dHZg9ezauuOIKPP300677zZ07F7fddlvy7549e3bb5rHHHsPUqVOTf5eXlyf//8EHH8TChQvx8MMP4wtf+AI2btyIuXPnol+/fpg2bZpw+wssy5LqvWpqavCFL3wBP//5zwEA8XgcVVVV+MEPfoAFCxZ023769OloaWnBK6+8knzui1/8Iqqrq7FixQqhczY3N6Nv374YdvcdiJTwQG078kFCuLNmmHASb2vDhzcswsGDB1FWVmbkHIl+4svF30ZRQQ+tY3VaHVjf/jx5e7du3YoTTzwRf/3rXzFhwgQAwJo1a3DuuefiH//4B4YOHWq736RJk1BdXY1ly5Y5HrugoAAvvvgizj//fNvXTz/9dJxxxhm49957k89dd911eP311/Hqq68KvwepSEN7ezs2bdqEhQsXJp+LRCKoq6tDQ0OD7T4NDQ2or69Pe27KlCl46aWXHM8Ti8UQi31WKn3w4EEAR794jAN5cGn4Poxhwknit1vyHlWJTnQAmqfpxNFppJubm9Oej0ajiEbVK3IbGhpQXl6eFAYAqKurQyQSweuvv45vfOMbjvs+9dRT+NWvfoXKykpMmzYNN910U7dow9VXX43LL78cI0eOxJVXXonZs2ejoODozVYsFkNJxk13aWkpNm7ciI6ODvToISZaUtKwb98+dHV1oaKiIu35iooKvPvuu7b7NDY22m7f2NjoeJ4lS5bg1ltv7fb87sV3yDSXYRiGCRCffvop+vbta+TYxcXFqKysxB8bXyI5Xu/evVFVVZX23OLFi3HLLbcoH7OxsbFbGr+oqAj9+/d37RMvueQSDBs2DEOHDsXf/vY33HDDDdi2bRteeOGF5Da33XYbvvzlL6Nnz5743e9+h6uuugqHDx/GD3/4QwBHb9YfeeQRnH/++TjttNOwadMmPPLII+jo6MC+ffswZMgQofcQyNETCxcuTItOHDhwAMOGDcOuXbuMfeFygebmZlRVVWH37t3GQoC5AF+n7PA1EoOvkxgHDx7Ecccdh/79+xs7R0lJCXbu3In29naS41mWlbxLT+AUZViwYAHuvvtu1+Nt3bpVuS1XXHFF8v/HjRuHIUOGYPLkyXj//fcxatQoAMBNN92U3ObUU09FS0sL7r333qQ03HTTTWhsbMQXv/hFWJaFiooKzJw5E/fccw8iEfF6ISlpGDhwIAoLC9HU1JT2fFNTEyorK233qayslNoecA4B9e3bl/9hClBWVsbXSQC+TtnhayQGXycxZDonFUpKSrqF4L3guuuuw6xZs1y3GTlyJCorK/HJJ5+kPd/Z2Yn9+/e79omZ1NTUAAC2b9+elAa7bW6//XbEYjFEo1GUlpZi5cqV+OUvf4mmpiYMGTIEDz30EPr06YNBgwYJn1tKGoqLizF+/HisW7cuWWwRj8exbt06zJs3z3af2tparFu3Dtdee23yubVr16K2tlbm1AzDMAwTSAYNGiTU8dbW1uLAgQPYtGkTxo8fDwBYv3494vF4UgRE2LJlCwC4phS2bNmCfv36dbsB79GjB4499lgAwDPPPIN/+7d/MxdpAID6+nrMnDkTEyZMwMSJE7Fs2TK0tLRg9uzZAIDLLrsMxxxzDJYsWQIAuOaaa3D22Wfjvvvuw3nnnYdnnnkGb7zxBh566CHZUzMMwzBMaBk7diymTp2KuXPnYsWKFejo6MC8efNw0UUXJUdO7NmzB5MnT8aTTz6JiRMn4v3338fTTz+Nc889FwMGDMDf/vY3zJ8/H2eddRZOPvlkAMBvfvMbNDU14Ytf/CJKSkqwdu1a3HnnnfjRj36UPPd7772HjRs3oqamBv/85z9x//334+2338YTTzwh9yYsBR544AHruOOOs4qLi62JEydaf/nLX5KvnX322dbMmTPTtn/uuees448/3iouLrY+//nPW6tWrZI6X1tbm7V48WKrra1Npbl5A18nMfg6ZYevkRh8ncTg6/QZn376qXXxxRdbvXv3tsrKyqzZs2dbhw4dSr6+c+dOC4D1hz/8wbIsy9q1a5d11llnWf3797ei0ag1evRo6z/+4z+sgwcPJvf53//9X6u6utrq3bu31atXL+uUU06xVqxYYXV1dSW3eeedd6zq6mqrtLTUKisrs77+9a9b7777rnT7pedpYBiGYRgmP8ntKdYYhmEYhiGDpYFhGIZhGCFYGhiGYRiGEYKlgWEYhmEYIVgaGIZhGIYRIjDSsHz5cgwfPhwlJSWoqanBxo0bXbd//vnnMWbMGJSUlGDcuHFYvXq1Ry31F5nr9PDDD+PMM89Ev3790K9fP9TV1WW9rrmA7HcpwTPPPIOCggLHVeJyDdnrdODAAVx99dUYMmQIotEojj/++Lz4dyd7nZYtW4YTTjgBpaWlqKqqwvz589GW44vt/fGPf8S0adMwdOhQFBQUuC5ImGDDhg047bTTEI1GMXr0aDz++OPG28kQoDrWlJJnnnnGKi4utlauXGn9/e9/t+bOnWuVl5dbTU1Nttv/+c9/tgoLC6177rnHeuedd6xFixZZPXr0sN566y2PW+4tstfpkksusZYvX269+eab1tatW61Zs2ZZffv2tf7xj3943HLvkL1GCXbu3Gkdc8wx1plnnml9/etf96axPiJ7nWKxmDVhwgTr3HPPtV599VVr586d1oYNG6wtW7Z43HJvkb1OTz31lBWNRq2nnnrK2rlzp/Xb3/7WGjJkiDV//nyPW+4tq1evtm688UbrhRdesABYL774ouv2O3bssHr27GnV19db77zzjvXAAw9YhYWF1po1a7xpMKNMIKRh4sSJ1tVXX538u6uryxo6dKi1ZMkS2+0vvPBC67zzzkt7rqamxvre975ntJ1+I3udMuns7LT69OljPfHEE6aa6Dsq16izs9M6/fTTrUceecSaOXNmXkiD7HV68MEHrZEjR1rt7e1eNTEQyF6nq6++2vryl7+c9lx9fb11xhlnGG1nkBCRhuuvv976/Oc/n/bc9OnTrSlTphhsGUOB7+mJ9vZ2bNq0CXV1dcnnIpEI6urq0NDQYLtPQ0ND2vbA0WU/nbbPBVSuUyatra3o6OgwutKcn6heo9tuuw2DBw/GnDlzvGim76hcp5dffhm1tbW4+uqrUVFRgZNOOgl33nknurq6vGq256hcp9NPPx2bNm1KpjB27NiB1atX49xzz/WkzWEhH3/DcwXfl8bet28furq6UFFRkfZ8RUUF3n33Xdt9Ghsbbbd3W4887Khcp0xuuOEGDB06tNs/1lxB5Rq9+uqrePTRR5MLwOQDKtdpx44dWL9+PWbMmIHVq1dj+/btuOqqq9DR0YHFixd70WzPUblOl1xyCfbt24cvfelLsCwLnZ2duPLKK/HjH//YiyaHBqff8ObmZhw5cgSlpaU+tYzJhu+RBsYb7rrrLjzzzDN48cUXfVk6NogcOnQIl156KR5++GEMHDjQ7+YEmng8jsGDB+Ohhx7C+PHjMX36dNx4441YsWKF300LFBs2bMCdd96JX/ziF9i8eTNeeOEFrFq1CrfffrvfTWMYEnyPNAwcOBCFhYVoampKe76pqclxffHKykqp7XMBleuUYOnSpbjrrrvw+9//PrkqWi4ie43ef/99fPDBB5g2bVryuXg8DgAoKirCtm3bHNeqDzMq36UhQ4agR48eKCwsTD43duxYNDY2or29HcXFxUbb7Acq1+mmm27CpZdeissvvxwAMG7cOLS0tOCKK67AjTfeKLUEcS7j9BteVlbGUYaA4/s3uLi4GOPHj8e6deuSz8Xjcaxbtw61tbW2+9TW1qZtDwBr16513D4XULlOAHDPPffg9ttvx5o1azBhwgQvmuobstdozJgxeOutt7Bly5bk49///d9xzjnnYMuWLaiqqvKy+Z6h8l0644wzsH379qRUAUeX2h0yZEhOCgOgdp1aW1u7iUFCtCxeGzBJPv6G5wx+V2Ja1tFhTdFo1Hr88cetd955x7riiius8vJyq7Gx0bIsy7r00kutBQsWJLf/85//bBUVFVlLly61tm7dai1evDhvhlzKXKe77rrLKi4utv7nf/7H+vjjj5OP1GVYcw3Za5RJvoyekL1Ou3btsvr06WPNmzfP2rZtm/XKK69YgwcPtu644w6/3oInyF6nxYsXW3369LH++7//29qxY4f1u9/9zho1apR14YUX+vUWPOHQoUPWm2++ab355psWAOv++++33nzzTevDDz+0LMuyFixYYF166aXJ7RNDLv/jP/7D2rp1q7V8+XIechkSAiENlmVZDzzwgHXcccdZxcXF1sSJE62//OUvydfOPvtsa+bMmWnbP/fcc9bxxx9vFRcXW5///OetVatWedxif5C5TsOGDbMAdHssXrzY+4Z7iOx3KZV8kQbLkr9Or732mlVTU2NFo1Fr5MiR1k9+8hOrs7PT41Z7j8x16ujosG655RZr1KhRVklJiVVVVWVdddVV1j//+U/vG+4hf/jDH2x/axLXZubMmdbZZ5/dbZ/q6mqruLjYGjlypPXYY4953m5GngLL4pgZwzAMwzDZ8b2mgWEYhmGYcMDSwDAMwzCMECwNDMMwDMMIwdLAMAzDMIwQLA0MwzAMwwjB0sAwDMMwjBAsDQzDMAzDCMHSwDAMwzCMECwNDMMwDMMIwdLAMAzDMIwQLA0MwzAMwwjx/weklMdpkZzKRAAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAg0AAAGzCAYAAACsMCQMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAACFeElEQVR4nO29e3xU1bn//8lMyCRckggBAjZKkaNoRahQUrRWW3KEg6X11B5RqSA/C7WSXohtAUXBS0UrpRwtLadUvHwPFmu/am3hS4sgxxv1AtLaFvGIFyiSCKYQSMjkMvP7g844M9l7z7o8a9/meb9e89JM9mXtPWHWez/Ps9YqSiaTSTAMwzAMw+Qh4nUDGIZhGIYJBiwNDMMwDMMIwdLAMAzDMIwQLA0MwzAMwwjB0sAwDMMwjBAsDQzDMAzDCMHSwDAMwzCMECwNDMMwDMMIwdLAMAzDMIwQLA0MwzAMwwjB0sAwDMMwjBAsDQzDMAzDCMHSwDAMwzCMECwNDOMiS5YsQVFREd5880189atfRUVFBQYOHIibb74ZyWQS+/btw5e+9CWUl5ejuroaP/rRj7xuMsMwTBqWBobxgGnTpiGRSOCuu+5CbW0t7rjjDqxYsQL/+q//ipNPPhl33303RowYge9+97t49tlnvW4uwzAMAKDY6wYwTCEyfvx4/Nd//RcAYM6cORg2bBhuuOEGLF26FPPnzwcAXHnllRg6dCjWrFmDz372s142l2EYBgBHGhjGE772ta+l/z8ajWLcuHFIJpO49tpr0+9XVlbijDPOwNtvv+1FExmGYXrA0sAwHnDKKadk/VxRUYHS0lJUVVX1eP8f//iHm01jGIaxhaWBYTwgGo0KvQcAyWTSdHMYhmGEYGlgGIZhGEYIlgaGYRiGYYRgaWAYhmEYRgiWBoZhGIZhhChKcpUVwzAMwzACcKSBYRiGYRghWBoYhmEYhhGCpYFhGIZhGCGkpeHZZ5/F1KlTMXToUBQVFeHJJ5/Mu8/WrVtx7rnnIhaLYcSIEXjwwQcVmsowDMMw/sNEvyhyzMcffxwXX3wxBgwYgKKiIuzcuZPkepyQlobW1laMHj0aK1euFNr+nXfewSWXXILPfe5z2LlzJ77zne/ga1/7Gn7/+99LN5ZhGIZh/IaJflHkmK2trfjMZz6Du+++W/saRNEaPVFUVIQnnngCl156qe028+fPx/r16/GXv/wl/d4VV1yBw4cPY+PGjaqnZhiGYRjfYaJfzHfMd999Fx//+Mfx2muvYcyYMZpX4IzxpbG3bduGurq6rPcmTZqE73znO7b7xONxxOPx9M+JRALNzc3pEAzDMAwTHJLJJI4ePYqhQ4ciEjFXStfe3o6Ojg6SYyWTyR79TSwWQywW0z62Sr/oF4xLQ2NjIwYPHpz13uDBg9HS0oLjx4+jrKysxz5Lly7FrbfearppDMMwjIvs27cPH/vYx4wcu729HcNO7YOmDxIkx+vbty+OHTuW9d7ixYuxZMkS7WOr9It+wbg0qLBw4UI0NDSkfz5y5AhOOeUU1Ny6CJHSUg9bZpZECc+zlY9IB0eaGCZoJNrbsW/xHejXr5+xc3R0dKDpgwR2vzoU/frpRTOOHk3gjHHvY9++fSgvL0+/TxFlCDrGpaG6uhpNTU1Z7zU1NaG8vNzWpuxCQJHSUt9JQyJG19Hz+FcBCAU8EmcBcZvIcf4rdyJRRvOU7FfcSC/36xdBuaY0pCgvL8+SBipU+kW/YFwaJkyYgA0bNmS9t2nTJkyYMMH0qfNC2eEzwSP1+bM8OMMdvXtQ3uuwC0iQ8XO/mA9paTh27Bjeeuut9M/vvPMOdu7cif79++OUU07BwoULsX//fjz88MMAgOuuuw4/+clP8P3vfx//3//3/2HLli341a9+hfXr19NdhQIsDEwK3b8Fv0sHd/qFSepzZ3kwj4l+Md8xAaC5uRl79+7F+++/DwDYvXs3gBORjOrqaiPXKi0Nr776Kj73uc+lf07VHsycORMPPvggDhw4gL1796Z///GPfxzr16/HvHnz8J//+Z/42Mc+hl/84heYNGkSQfPVCIwwlHZ73QJa2qNet8AIiVjSt+LAwsDo/g2wdOTHRL+Y75gA8NRTT2HWrFnpba644goAdAWbVgRilcuWlhZUVFTg1Lvv0K5p8K0whE0QTOJT+fCbOARBGKLx/NsEie6Q1snpikOivR3vzV+EI0eOGKkRAD7qJ97f/THtmoaWowkMPePvRtsbVHw5ekIWX4kAd/7myb3HPpEIP0Uc/CgMYRMEKyiu0Y/iETke4YgDAyAk0uA5LAre4iQRdp+NIdEwKQ5+EYFC6Py9JPf++kUiWBwYgKVBD49kIRoLpqR0x12KCIh8LqltDMiDnyIOlLAseIOTRNh9JqZEg8WBYWmwwgeRg6CKgRNO1+SaUORiSB7CIg5eiEK03f1zUtDt0hQyIp9JahsT8sDiUNiwNORiSBjCKAGUyN4fcskwIA9+FwevIwdBlQM7nK7HLaHIxZQ8sDgULiwNmRgQBpYFM+TeVzKJyPwbIBAIv4qDKWEImwhQIXtfqCXDhDz4VRwOdXcg3q05jXS3/67LL7A0pCAUBi9FIVZKs8KbCeLtJcaOnXnPyQVCUx78Jg7UwsCiQE/uPaWSiMzPnkIg/CoOjDlYGgAyYTApC36WAVGcroFSKMijEATy4BdxoBQGr2Sh2KeS0mUwBWFCIqiiDywOhQVLg6YwUItCGORABZNCYfcZScuEpjx4LQ4UwmBSFPwqA6I4tZ9aKDI/B12BoJAHFofCobClQUEYqCShUOVABbt7RSUTbsqDF+LgR1kIuiDIYlIo7D4bWZnQlQcWh8KgcKXBI2FwQxb6xPwnJK1x+nqG3HupKhFuy4Ob4qAqDFSSUGhyoILdPaKSCTflgcUh/AReGqSnkFZMR+gIgwlR8KMYOGHXXkqZSN1nHXlQqn8o7fbNVNaZeCUMboiCH4svqUc85N5HVYlwWx5YHMJNoKXBjTUnVGWBShSCJgeyWF2frkhk3ntZgVAWB8PITiGtIgw6HbEJUfCjGDhh1V5KkUjdYx158Gq+CCY8BFYalIRBIsogKwsUkhB2QRCFMiqhEn3wmziYFgbVzplKFIImBzKYEInM+y4rECriEI37Z/0LxnsCKQ0mhcFtWXBDFPrG3Jn671jc7DdL6l7pyAMgJhDS4mAoRWFSGFQ6a11RCLMgiEJVuAioRR844sDoEEhpkEZAGFTSEKrCQC0KbklBPvK1g0oqdOQBOPG5GREHYkwJQxhlobjdfKoSALpKzRWwqtYeAPLRB1lx4GgDkyL80uATYaAQBb/IgSq57deViD6xDn+Jg08LIjNxSxjoh2i6IwX5cGoHlVDoyANw4vMyIQ4MA4RdGgwIg6gs6EpC0AVBBKtrlBUJ3ZRFECIOIuSLMrghCxSi4Bc5UCG37boSodOpmxCHoEQbDiViaE/orT1xLMGjP+wIrzQQLz4lE1nQEQY3ZKEypv7tfjhu9tFEVSQy77mMQJCLgwfRBuq1JGRkQX94ZnAlIR9W1yYrEropC444MNSEUxoIix7dkAVKUdARApPH15GNzPsjIxCi8hDkiIOIMMh07KLCoDc807woqJ7DZM0CoC4SqtNGU4tDUKINjDnCJw1EEQbTNQu6omBaDqjJbC+FQFDLA6k4uBRtoIwwmI4uUIqCSenwQjYyzykjEKLywBEHhpLwSYMgTlEGU8KgIwpBkwQnKARCRR4AZ4EIUsSBaqSEyboF3c49SKkL2Y4/33Go5YFSHDjaUNgUrDRQYFoW3BSFAbFW2999GO9j7Ly6AiEjD0D+6AOZOBiMNlBFGExEF3Q6+iBJghMUAqEiD4Bzh88RB4aCgpQGiiiDiDCoyAK1KDjJAPUxdOUi99plJKJvLC418sJpuGaQIg5OUA17FDmOaofvpigUt1lXxHf11qu0dzyn5ogKGXkA8kcfWBwYXQpSGnQwFV3QkQUKMaAgXztkpcLqnjiJhErUwag4+HjeBsqCR5WOn7TGwUYGTOyvKxiqElHcnpQSDqdOn0IcOEVRuBScNOhEGaiFQVUU/CIJsuS2WyUykbpnVPJAIQ4qmF4e26mzFxEGE+kIrdSFphhQ4dQOFaGQkQiVqANHHBhqCkoadJa3phQGFVkIqig4kXlNKlGIfOkLUXnQFQe/pylkoJaFMIiCKLnt1ZEIKnmgEAfbY3O0oSApGGnIJwxOUQaq+gU/yEL/XrTHa+6kKZJUEQiRqAMgVu9gLOLgxWRPmlGGfIiIgHKNQ8BEwYnMa5EVCJF0hKg86IqDSrQhcjyCRFl4PkvmIwpGGkxCLQw6okAtBVTnk5ELWYGgSlnoiINKtMFEikKn+JGqfkFWGEyIQvFxumN2lekXSqoIhKgUiAiGqYiDH6MNH3b3wfFuPVFv66adUThMsDRAL8pAJQyqouC2JKiS2U5TAkGRsjAScfBJQaRulIFaGFRlgVIIqM4nIxayAkGVstARB442MCnCJQ0Ks0GqLm8N5BcG0eiCijAERRasSLVdNrUxINZKIg46mCyO9BLnlAadLARFFGTJbJ8pgaBIWZiIOPgx2sCYI3DSkIjJ50rt6hl0RktQCIMfZaGq17Gsnw919jV2LhV5oBIHpzoHnSW3qYgclw+Jq9YyuCEMKrLgd1GwI9Vu2bRGcVuCRBx00C2OZMJP4KTBFuJVLf0kDNSikCsGqttSCYWsPLghDn5FZTZIVWEQOza9MJiWhejxrvT/d5eZ+wpUkQcqcXDahodRMjqERxokUa1jcBIGP8mCjBiYOoesVMjIg+lUhUptgx+HXurUMTgJgd9lIVMMdLajkApZefBDxIFh7Ai9NMjOzaC6vDWlMKjIghuSIEtmm2QEQlQeKMQhiNEGK1QiBqppCUph0BEFUTHQwekcskIhIw8U4qAqFpyiYJwIhzQQpybs0FmlUkQYwiILVqTaqSIPgL1AeFUc6UVBJOVS2F4Lg6osuCEKoqimOUTlwaQ4qKQo7PbhIsjCIhzSYANllEE1LWFCFoIiClaoyAPgHH3QFQe7aIMfCiJFkI0ymBIGU9EFP4mCHak2qsgDYC8QXqUqONrA2GFueTcfY1XPEARhqOp1jFwYqoqPCr9Iz6t4LXb3S+ReU68gavV3pDNVOTUUsz8Kn0swuiAjDNHjXYEQhkxU2+x0b4TurYLsUa2Cypxg5cqVGDZsGEpLS1FbW4uXX37ZdtuLLroIRUVFPV6XXHJJeptrrrmmx+8nT56cdZzm5mZMnz4d5eXlqKysxLXXXotjx8w+VIY60kCBH4RBVxSoOnyn4xzq6qd2TMW0BXXEIejRBlFMRBko0xEmJCHa1im8bXfvXjTnVIg8ACfuk1XUwS/FkTzywppHH30UDQ0NWLVqFWpra7FixQpMmjQJu3fvxqBBg3ps//jjj6Oj46OHjg8//BCjR4/Gf/zHf2RtN3nyZDzwwAPpn2Ox7O+o6dOn48CBA9i0aRM6Ozsxa9YszJkzB4888gjxFX5EaKXByye+fMJgUhaoIwIU5xURCll5cFMc/AzV06IpYXBTFmTkQPYYqjKhmragFgeV2gbhlTBDPLlTS0tL1s+xWKxHxw0Ay5cvx+zZszFr1iwAwKpVq7B+/XqsWbMGCxYs6LF9//79s35et24devfu3UMaYrEYqqurLdu2a9cubNy4Ea+88grGjRsHALjvvvswZcoULFu2DEOHDhW/UAkKLj0hMwOk7vBKK0wIg6kUAhUybZNJW/ghVaEzo6hJZFITQRSGaFtnj5dJrM4nc17ZtIWpVEUh0NRVgcbOSq1XU1cFAKCmpgYVFRXp19KlS3ucr6OjA9u3b0ddXV36vUgkgrq6Omzbtk2ozffffz+uuOIK9OmT/cCzdetWDBo0CGeccQa+8Y1v4MMPP0z/btu2baisrEwLAwDU1dUhEongpZdekrpnMoQ20iCD7DBL1bSEiDDIykKQSLWXMvKgE3GwwiraIJui8ON8DQBtDptCGGRlwbQYqJJql0g0QibyQB1xUElfFHpB5L59+1BeXp7+2SrKcOjQIXR3d2Pw4MFZ7w8ePBhvvPFG3nO8/PLL+Mtf/oL7778/6/3Jkyfjy1/+Mj7+8Y9jz549uPHGG/Fv//Zv2LZtG6LRKBobG3ukPoqLi9G/f380NjbKXKYUoZQGitSEXZTBlDB4LQvVvQ7b/q6xs5L0XNTyoCoOMkMxtWsbDCxaRSEBusMre+xDHF3wqyhYYUIedMTB8ngW4iBbp1BIdQ3l5eVZ0mCC+++/H6NGjcL48eOz3r/iiivS/z9q1Cicc845OO2007B161ZMnDjRaJucCKU02GEylOyGMFDJgpMgyGyvKxOU8kApDkGsbciEYtSESlqCMrpgShYirT0fBhJ9aD9ranlQFQc3Z40Mc11DPqqqqhCNRtHU1JT1flNTk209QorW1lasW7cOt912W97zDB8+HFVVVXjrrbcwceJEVFdX44MPPsjapqurC83NzXnPq0PB1TTkIpOaUMmDey0M1b0O93hRYXVslePLXJvT/dKpcRDB7m/FjaGXuhM7yUQldEdKWJ5fIqdPIQyR1rjlS3dbGShrHnRqHHrsY/H58vBLdUpKSjB27Fhs3rw5/V4ikcDmzZsxYcIEx30fe+wxxONxfPWrX817nr///e/48MMPMWTIEADAhAkTcPjwYWzfvj29zZYtW5BIJFBbW6t4NfkpqEiDKLIzP6p0SiZkgVIIdLBqR76ohGzUgTLi4Fq0wUCKQgfZ9INTB+UUZTAtCxQdvMxxZSMT0bZO4dEX0eNdtlEHlYiDTLTBLu1Q6HUNIjQ0NGDmzJkYN24cxo8fjxUrVqC1tTU9mmLGjBk4+eSTexRS3n///bj00ksxYMCArPePHTuGW2+9FZdddhmqq6uxZ88efP/738eIESMwadIkAMCZZ56JyZMnY/bs2Vi1ahU6OztRX1+PK664wtjICaCApEE3NWEXZdAtfHRCVBj8IgtOpNpIJQ/U4mAFRVFkPhKxJCJxuhCybmqCuo6BWhhMCYIMKikO2ZSFaXHQTV0UUl2DCNOmTcPBgwdxyy23oLGxEWPGjMHGjRvTxZF79+5FJJL9+ezevRvPP/88/vCHP/Q4XjQaxZ///Gc89NBDOHz4MIYOHYqLL74Yt99+e1Yx5tq1a1FfX4+JEyciEongsssuw7333mv0WouSyaTvx+e0tLSgoqICp959B1Bh8Y8zZ+0Jq9CwzCyQVpEGWWlwEoZ8UQavZWFg9ET7DnbTLH2di0wtRD55cCqQtFuvwk4crIoiraINdtKQuxaF5QgKi0iDlTREjmd/wVilJ6xCylbSYBd6thIEO2lQiTKICEOQZEEE0SiEiDw41TnYTTttJw5WgmAnDVYyYBVpEFmHIlF24m8j0d6O9+YvwpEjR4wVFqb6iZ9sr0VZX73n4ePHulA/9iWj7Q0qBV/TkItMasJEWsIrYRgYPZZ+Wb2X+b4uMrUPJkaK6NY4qK6E6idkIgosDOKItlWk3oGyxkFGECnJlV8m+AQ/PeHCCpeyBZB2UQY/CYOKBGTuQxGFqO51WCjqUFV81DbioJKmkCHoIylkkK5xUF2pUqJ2gVQYWo87/75PGclpUm0WiTrI1DuIojocMwWnHhgnAiUNiZIkWWhE92lRJS3hhBvCQBktyD2WqkRQiIMKqpM/uQHl05luVbxKdb7TE7Lr0YV8omC3LYFARFrj2uKgUt/AMCbhv7gMZGoZZHGKMpgSBlPphXznkkU3VaEyDNMK6lUw3Ua0CJIiLK0SZXBFGFqPZ788Po5MusILRP8W3FwxlfE3gYo0qEI9qRNllEFEGGRkwbQcqLRBJAohGnFgTkA9rt4y560wiZPuwlNKwqAjByrnkIxC6EYcnKINdlilKNyc7MlLPuzqh9Iuva6tvStYS7K7SegiDaqT6sjOzSCDztLWosLgRjRBFdF2iVyrSrTBBGEohqRGNy0hJQwU0QRVFKIQpoo5VetKGEaV0EkDJTLhapXix3xRBhlh8DuU4iCL7kyRJoVSBN3ZIDPRraI3lZYQ7lS9EgUnCMXB7l5RLB3OMBQUpDToPCVSTUlcSMKQgkocTEYbgl7XkIlOCkO2AFKnU5MSBr/iYcSBcorpFDytNGNHQUoDNapDLO3wShgGRuKWL9JzGBYHO3Rn5xQht3aGev0Jr1GZkyFflCEUwpCCSBxMRhu8mq+BCQ+hlwbVIkirJ063ogwiUAiDqBxQC4TJVIWsqFF9pl6jWt1ussPwozAk29p6vEjxKOLAtQ2MW4ReGkQwkbO267wo0hI6wqArAFQCQSEOpqINoikKr4ohVUPHwsPrJMLaqk+/Qp0mUf1CPkEgFwgCcaCKNuikKHocK+fvjlMYhUmwpcGF2SDzQRn2NiUMxlINugJiKOLg9kiKQkD2SdYpyiAsDBqoigCZQBgUBzs42sC4QbClwUVkwtgqUQYTwmBCFPKdS/acuuJAEW0Q+Wy9HkEhi8hToMzcDJbnsHnq9UoYqCMG2sczlKrQrW0QjThxJIGxouCkQSSkLBqepooyUAqDqaiCLDLtMBFx4GgDHa4+wUoKg7HaBIfzSJ1LUxwoog0iIsjFkIwoBScNXmH3REwlDH4QBTsoxcEKE7UNxoZeGkip+WmKX+0ogwBuiQJZGwyM/OB5GxivUJKGlStXYtiwYSgtLUVtbS1efvllx+1XrFiBM844A2VlZaipqcG8efPQ3u6jbzpCqJ9wRYXB71CJg6loA1WKwq/DLkWeJE0WQFKlJbwUBTuoxMFktKHHNhxZYBSRnqD70UcfRUNDA1atWoXa2lqsWLECkyZNwu7duzFo0KAe2z/yyCNYsGAB1qxZg/POOw9vvvkmrrnmGhQVFWH58uUkF6GDSEdg1aHIpCZUowxeCMPAaEnWzwe76UYIDIzEcTDhPAf/wOgxpRUz7VbBdFo6m4I+sQ60xkvybxhQKFITYRaGFMm2NhT17u28UevxvOtWiK5TAaitSWGaaBzo9ngl+Q87+yDWqbfceLzTmwXEgoB0pGH58uWYPXs2Zs2ahbPOOgurVq1C7969sWbNGsvtX3zxRZx//vm46qqrMGzYMFx88cW48sor80YngghllMFNYRgYLUm/ZH6ndC6CiAPFVNOepigCjkoBpCMeCEPyWCuSx2jn6DCZqtBdBVO1riG3GJKHXTJS0tDR0YHt27ejrq7uowNEIqirq8O2bdss9znvvPOwffv2tCS8/fbb2LBhA6ZMmWJ7nng8jpaWlqyXCH4JBYtgYo0FWWRlgEoeTNU4UEyaFQZUR0702MaNKIOLwpAShUxZsHpP6xwE4iBT+2EncJyiYEwhJQ2HDh1Cd3c3Bg8enPX+4MGD0djYaLnPVVddhdtuuw2f+cxn0KtXL5x22mm46KKLcOONN9qeZ+nSpaioqEi/ampqZJqZRmU2SJGnS4rUhBNuRBl0On+qqINb8EgK8yg9CbskDKJSQCUPpiIOutEGhqHA+OiJrVu34s4778RPf/pT7NixA48//jjWr1+P22+/3XafhQsX4siRI+nXvn37SNrCyxmfgCRaEDBxECGM8zUUOioSQJ22YJgwIVVFU1VVhWg0iqampqz3m5qaUF1dbbnPzTffjKuvvhpf+9rXAACjRo1Ca2sr5syZg5tuugmRSE9vicViiMU8rqaRJChPs5Sd/cBoiXKhpG5RZHWvw2jsrFQ6N6NOkIb66XT+yWOtKOrbh7A15vBjQSQTXqQiDSUlJRg7diw2b96cfi+RSGDz5s2YMGGC5T5tbW09xCAajQIAkknOqeViMjVhIjoQxohD2KAebimKY27ecGqCJM2gIx0+HunBMDpI62lDQwNmzpyJcePGYfz48VixYgVaW1sxa9YsAMCMGTNw8sknY+nSpQCAqVOnYvny5fjkJz+J2tpavPXWW7j55psxderUtDz4GROrIHpRBGmyc9eJODAMNZTpBaMRB4chmDJDLxnGTaSlYdq0aTh48CBuueUWNDY2YsyYMdi4cWO6OHLv3r1ZkYVFixahqKgIixYtwv79+zFw4EBMnToVP/jBD+iuooBQiTK4EQ1QEQeRFIUsdvM1MObwU4GeiXqEIKUqnChuS6Crt3Nwubg9ia7SIpdaxAQRpURYfX096uvrLX+3devW7BMUF2Px4sVYvHixyqmMYrqozQ9DAN1MH5iIOKhO9iRC/16taO4MfmcQRlTC+yYLGMMiDgyjC6894SN01l+wPp779QZ+rXEISqGqG7g1R4MtBtZicGPEg+w5TAy99FNUhylMWBoChNyS09513lITRmnMN+GHCbIKCZmREzqLU8lGGdwcIhmk4Ziurk7KFAw8TkcSq4mdZJ5i3ejo/PC0T5mqMJmiYIKNF524H1MVPOzyI/7R2RslnXrfgR2dXNhtB0caMvByrQGq1IQfhCGFn9qiCq8/QQxhasLLp37Rc+sMvaRaSpxhKGFpCBF+7KRF2hSEpb0Za7zKsfshTUDWBgM1HgxjCpYGQ1CPnOCOVRw/jFoJEiYmdlJF5MncD8KQwk9tYRg3YGkICX6MMqSgWevCndEPJibzYgob6tkhKaM7vNIlIwtLgw9wq0MMI34YQcELoWVjm4snCMP78cmepE2comACAkuDA9RPnaodXCGkJty4xkKYqyFqqG7TjYWqeL0GhvE/LA2a+KEj8nNqIoVfUxRWQ2gLHb+O7/djlEEUVSGiHkHhp/oVJpiwNHgMpybMwMWQPiPk4Xc/CE2Qli0PIytXrsSwYcNQWlqK2tpavPzyy0L7rVu3DkVFRbj00kuz3j927Bjq6+vxsY99DGVlZTjrrLOwatWqrG327NmDf//3f8fAgQNRXl6Oyy+/HE1NTVSXZAlLA8MQEyvlGgeGKSQeffRRNDQ0YPHixdixYwdGjx6NSZMm4YMPPnDc791338V3v/tdXHDBBT1+19DQgI0bN+K///u/sWvXLnznO99BfX09nnrqKQBAa2srLr74YhQVFWHLli144YUX0NHRgalTpyKRMBdRYmnwGJ7p0Ay80qU6XWUGvhZsloAOC36bIZJxl+XLl2P27NmYNWtWOiLQu3dvrFmzxnaf7u5uTJ8+HbfeeiuGDx/e4/cvvvgiZs6ciYsuugjDhg3DnDlzMHr06HQE44UXXsC7776LBx98EKNGjcKoUaPw0EMP4dVXX8WWLVuMXStLgwMfxvN/ERzq9L7Tp15Z0q8ERbDi7f6vMZHFD1MUc8fsjB8+o7DR0tKS9YrHe9aYdHR0YPv27airq0u/F4lEUFdXh23bttke+7bbbsOgQYNw7bXXWv7+vPPOw1NPPYX9+/cjmUzimWeewZtvvomLL74YABCPx1FUVIRYLJbep7S0FJFIBM8//7zqJeeF/8pcpLGzUmkExcFELPAjKPwqNmFbGru71NwICtMU9e4d2hEURb17K+2X6BPLvxGTRXNHH/SK64l7Z0cvAEBNTU3W+4sXL8aSJUuy3jt06BC6u7sxePDgrPcHDx6MN954w/L4zz//PO6//37s3LnTtg333Xcf5syZg4997GMoLi5GJBLB6tWr8dnPfhYA8OlPfxp9+vTB/PnzceeddyKZTGLBggXo7u7GgQMHJK9YHI40ML7gYML8l6OpqFCr5heU13T1pv0aKLSOjiQC4lL6hvqzDjv79u3DkSNH0q+FCxdqH/Po0aO4+uqrsXr1alRVVdlud9999+GPf/wjnnrqKWzfvh0/+tGPMHfuXDz99NMAgIEDB+Kxxx7Db3/7W/Tt2xcVFRU4fPgwzj33XEQi5j5njjRkcDheSrZA0aGufsIV/Ae7+2qPojjY3eHboZcUUQa71ERjZ6X2sRmX6FOmPYqiqG8fX4xUYAqD8vJylJeXO25TVVWFaDTaY9RCU1MTqqure2y/Z88evPvuu5g6dWr6vVThYnFxMXbv3o2hQ4fixhtvxBNPPIFLLrkEAHDOOedg586dWLZsWToVcvHFF2PPnj04dOgQiouLUVlZierqassaCSpYOwOCG0/iDCNLd+9epMcTCeP7qbZBpC2qqQk7qO85o0dJSQnGjh2LzZs3p99LJBLYvHkzJkyY0GP7kSNH4vXXX8fOnTvTry9+8Yv43Oc+h507d6KmpgadnZ3o7OzsETGIRqOWIyOqqqpQWVmJLVu24IMPPsAXv/hF+gv9JxxpCBF+jDaIRBn8JEQixa9Bp6u0KPBrDvgh4kAmLyEfWVIINDQ0YObMmRg3bhzGjx+PFStWoLW1FbNmzQIAzJgxAyeffDKWLl2K0tJSnH322Vn7V1ZWAkD6/ZKSElx44YX43ve+h7KyMpx66qn4n//5Hzz88MNYvnx5er8HHngAZ555JgYOHIht27bh29/+NubNm4czzjjD2LWyNPgEihTFieP4Rxy8Kn7k4ZY+hiBFkcJLcfBTtEOHrtIir5sQCqZNm4aDBw/illtuQWNjI8aMGYONGzemiyP37t0rXWewbt06LFy4ENOnT0dzczNOPfVU/OAHP8B1112X3mb37t1YuHAhmpubMWzYMNx0002YN28e6bXlUpRMJn3/yNHS0oKKigrU/Ph2RMpKP/pFaXfWdtFY9s+5k+zkLizUN9ZzREJuTUPu+hN20w5bTSdtV9NgN4JCRBpER1F4LQ4ywpAv0uA01NKqpsFOGqwKIXNHT1hFGg7HS7N+PhbPbm9uIWTukMvueDT7gO3ZP0fi2V/ckeM5Icmcjzx3dERxe77f9/wnnvte7vTCVlNJ2804aLfqos7CVTKjKNwWBxlhEEpN2EQa7ApKrdITVsMtrebbyC2EtJKG3Pe6S3N/D8ffd+c0O1GWQKK9He/NX4QjR47krRFQJdVPXPL7r6FXH83RE60dWD/pF0bbG1S4pkGSoAzR83KIo1vn5iJId/HrPABuPvWHJcLAMKqwNBjE6zC5F+Ige0636hn8MAkXYw43OnMj5+B6BiZgFLQ05IaZvUZkxkPZTtZNcTBxLpOzQIpEjXJTE4WCzFTS0tX8Ah2lyogDk+KgcmzqURMAj5xgvKegpUGEMFTTuyEOKufw06gJJhyYEIdCSklwYSSTD5YGD/AiF29SHPw0RbTXKSHGeyg7eaPC4BBxMTGrJs8GyVDgz8qmAoZq6KX1semHY/pJGBh/kegTsx9BQTj00gqK4Zg6wmAiNcGIcSQeQ3GxnnR1xTniYgerZwDRCetTdvI6xxK5BtmhlrqIpKL8VgcjgkjI2Y9Pobodr1anH6CUhF9HtTDhxH/fFAElSNX5NGtBBCfCEKTPhqFFqYDRDWFQGDXhRRFk7hwMDMPSUKBoRQkCJAwMLaRPtYZGUfQ4hqAEFPXtQyIMfkhNiIx+USl6zJ3YiSk8WBoMo1KYZ2LopfV5OuRmbpTc3vY4mqkJO7gIUhyhlAXBsMu8BX0uioOdFFDJAqA3A2QKmSJITk0wbhNqacidzjd3ul8rRMbl243vlwmD58vJuyUOJ87VYSsETr9TOheBMFDUM1DN0SDyNxU27Doq5fC5S+KQPlaGQJCOtCAQBjt4fgbGLwRbGnLm8O8xx78AIoVtunM12D0B+0kcPjpnB7kopI9tKMIAyK03ETbyrQVwYhvviiGFnpxdFgdqqISBYqilyJoTlvvxHA2MAMGWBg+hiDaI4IU4mIBKGNyKMviJ3AWA3CxOs0tRkEcbgMCKg1fCwKkJxgtYGiwwMXWwarQBCL44eCUMMgIX1uGWTrj5ZCncIQqKg9fykGoDVTuc7o+MiMnUoTCMCvwXBvoUhWy0gVIcUi+voW6L0z3yUwFkbh2Nn1BNUVBFGyjFAcjuuN2QCOVzGViUSibKYPmZCvwt8HBLxgqWBg1kw9xOnRuVOKS39UAgVM9pclGqXChTE4VYBJmLbIicWhwyMSUQWsfUTEtwASTjNwpOGkS/6HVTFHbRBjfFIb1PRmdOXjipeVzdtARFAaRVFCnoq1uKPiWKpihkw95OnZ1JcUihIxBkEQyXhcGL1ARHIwqP0FfSxNtLECs1NxlRc2cf9O+lN8d9Jo2dlajuddhxG931KTI7+IERm7UBBPfXwZQw2BG0Asig0F1WjOjxLql9HNelyIRgjYrMjj/Z1pZ3G20EZUd1pIRuasJyuxCNnGjtiCGqufYEz19nT+giDSrDLkWRHXqpEm0AzEUcLI8jGCmgjFQc7O5rdKQEdQEkoF4EafLvMRPKmfqohl7me1J2I+KQS24UgTydQSQMJqMMqoLAs0EyQAilQRWrTkE0RG3iCdZNcUgfLyeNYSKlIdrmfNdvsvhRNTXhZhEkZViYIkWhOvzPC3EwhgvCwMMsGa9haTCMarQB8EYcTOKGMNjdb05NnMDtMLTIE3MoxMFwSgKQFwbVURMA1yow9hSkNOhUvduFs011SmERByphcEJ2qKvOTJ9BGDmh88Vvl6IwEW0AAi4OhMKgMlpCtwCSUiRzJyJjwkdBSoPb6EQbgGCLg2j9AiB2nSppCRmhs0pNkE7q1O5OjYOfEO0IpcQh9fIKyTboCgNFlIEajkYUJiwNeZDJb5sMgYuKg0wnbRLZdugKQyGsMeEXVJ5sycUhhVsCkXkeyXPprifhJAxuFECe2Fd5VyZkFIQ0UBeoqYS1daMNgFzoPlMg3JAInfPprifhJAx2IufG3AxujZygwKpDUXladergjIlDCo2OPe/xFBG9Fi/SEkC4hloy7lAQ0iAKRRhaJdpgShwyMSEQFMcUvR7TaQk7RP8m/Dx9dC4UHYVqh2VcHDKR7fQJpSPRJ0YiDCp1IrqpCcq0Q6IsQXcwxhcU7Pid1ngJ+sTcncHjUGdfVPWynpTpUFc/VBUfFTqOyARQTuR28qITRVEKh4z8UKcldJc6pyyCjMTD96SXb7Kn7t69EG3rzHsc4QmgRMiVgNSEUQbSGlTRBaq0BOBORIGLIAsDjjQootvxWOFGxMEKp4iBiQiFTHRBdT4G2ShD0KeNBqzzzjJPjbIpCp2RFDIRB5KoQy6G6iBMpiNSON13mSiD1efNxY1MPgIlDZEO87YsM8mTyvDLfE/GMh0lpTikMF0LQZmOUJmTQUb2vFoKmzqka9URyDx5mhQHz+WBCNH2iV6zF2kJhhEh+OmJ9ihQ2q20K0WK4sN4HwyI9Vx7wmlNCqc0RXqbf3aa+VIWqU5YJ13hBtS1C16kJQDv5mfojgFRoki9E12lRShuT/Z8v3cExW3WMtNVFkHxcevfiaxLkepERVMWmZClLyRRERhRQaJMSwD6qYmgjZxojZcgGtX7d9od7/lvgDlB8KXBgu54FNGYmkgAJ54w+8ayv4wOx0tRGWu33N6UOADy8pCJVyKhvGYEgTCoFD/qpia8KoLsLgWi1n+SQtvaCYIdJsUBkJOHFHadN6VMaA+ZlEhFqAqDbJSBR00wqoRSGqywW+1SJtrglTgAcoWSKdwQCYoUiUzdgqowyEYZvEpN6NJVChQLisSJ7XuKg5NMmBYHQE0eclGRCRPpD4roAqAuDDJywPUMjAgFIw1OWImDVbQhH6riAIA06uCEXScvIhMmaihMpiNSOAmDTJQhCFNHOyETmXBCRxwAuCoPubhVF0ElC/mgEgb7Y2gfggkZ4ZAGjboGWZyiDYCaOADuy0MuJoTACWpZkJnEKQXFiAlXV7Y0VNcgG20A1MUByO4oTaUtvEB2RISoMFDXMVDIBEclCpfQlttazcbn9AVv9RSpGqLWWdTqUGdf4Y5SZ0iiF6Ta65YwOOEkDBSpCbdmg7QeGSG3vR35OhfVURVZ7SkrFu48dYYpmkRmFAggfs1dZRHSOgbHNrEEMIKEI9JgENmiyBSqEYcUXkceKJEVGypZoJxLI+ipiXzIFkWm99OIOGQimrpw6pzdiEToigtVdEE1LSE11JZFgrEgPNJAkKKgKopMoSsOgJo85OKmTOhGPtwQBqooQ5Cmjs5EprZBRCaoxAGQr3vI2temQ1eRCeqoBmUqwg1hcIKjEoVNaNMTgHyKwg67jkQkJ66TqshEJm3RY9+MtEDuSxXy4wleX3NnH18IgxO2qQlDy2LLpijssOtURDobilRFJroFglnH+mf6QOZFdm6J9ItXwsASQMPKlSsxbNgwlJaWora2Fi+//LLttqtXr8YFF1yAk046CSeddBLq6uqytu/s7MT8+fMxatQo9OnTB0OHDsWMGTPw/vvvZx3nzTffxJe+9CVUVVWhvLwcn/nMZ/DMM88Yu0Yg5NLgBm6KA6AnD5bHc+j8TchGj/NLXI/I/TIhDGFPTWRiUhxk5UGmw/UTqXbLyIKuMFBjJ50sGNY8+uijaGhowOLFi7Fjxw6MHj0akyZNwgcffGC5/datW3HllVfimWeewbZt21BTU4OLL74Y+/fvBwC0tbVhx44duPnmm7Fjxw48/vjj2L17N774xS9mHecLX/gCurq6sGXLFmzfvh2jR4/GF77wBTQ2Nhq71qJkMun7qa9aWlpQUVGBU+++A6hweBq0SE9YTfJkNV9DJnYpCqchmPlSFQAsUxUpRNMVuYjO8+AnZKVHVK7y1TBQSoNdxMqxANIi0mC3YFXkuHUHYTd6wi7d4DRng/0+9l8JInUPdqmKrG0kUhYpVFIWbqI09TORLFBHGVSkwWrBqtSU6In2drw3fxGOHDmC8vJy+4NokOonTl+7ANHemhNytcXx5vS7hNtbW1uLT33qU/jJT34CAEgkEqipqcE3v/lNLFiwIP/5urtx0kkn4Sc/+QlmzJhhuc0rr7yC8ePH47333sMpp5yCQ4cOYeDAgXj22WdxwQUXAACOHj2K8vJybNq0CXV1dRJXLE5BRhpM5KJ1Ig6AWNjdCurIg0lU2ioaXXBTGMKObl5cqJMLQeQhM6Ig2y7K6IIf0hJhXeGypaUl6xWP97T2jo4ObN++PauTjkQiqKurw7Zt24TO09bWhs7OTvTv3992myNHjqCoqAiVlZUAgAEDBuCMM87Aww8/jNbWVnR1deG//uu/MGjQIIwdO1buQiXwz79AQ6hMKW1XEJlvwied4sgUmZ2kTPRBpmDSLXRkhiq6APhEGAzVM6RQmbhJdbIn3eLIrO3+2XGqFEvaQR2RoBQVGVHyShiCnproaO+FSETv326i/cTfY01NTdb7ixcvxpIlS7LeO3ToELq7uzF48OCs9wcPHow33nhD6Hzz58/H0KFDbaMD7e3tmD9/Pq688sp05KOoqAhPP/00Lr30UvTr1w+RSASDBg3Cxo0bcdJJJwmdVwWlSINMwQcAHD58GHPnzsWQIUMQi8Vw+umnY8OGDUoNpkI12pCvWE434pCJSvQh9TRv9zIB5blkrtlvwuDW3AwyqA6byztHA1HEIb3tP5+8VYomc8mNAIhEBGS3l0H22rp6R/Leu67SIldGSogQ1igDAOzbtw9HjhxJvxYuXEh+jrvuugvr1q3DE088gdLSnv9gOzs7cfnllyOZTOJnP/tZ+v1kMom5c+di0KBBeO655/Dyyy/j0ksvxdSpU3HgwAHydqaQ/heRKvhYtWoVamtrsWLFCkyaNAm7d+/GoEGDemzf0dGBf/3Xf8WgQYPw61//GieffDLee++9dIgljFBEHDJJdaKqdQ+ZOHXmdlEK0+kPWTHSlQVAb6REUIdaWqEztTRlxCFrH4XogwxupTiUZnLUjC6IEOYoAzXl5eV5axqqqqoQjUbR1NSU9X5TUxOqq6sd9122bBnuuusuPP300zjnnHN6/D4lDO+99x62bNmS1ZYtW7bgd7/7Hf7xj3+k3//pT3+KTZs24aGHHhKqpVBB+q96+fLlmD17NmbNmoWzzjoLq1atQu/evbFmzRrL7desWYPm5mY8+eSTOP/88zFs2DBceOGFGD16tHbjdZGdITKFSIdDGXFIoVr3IIqb0QlA7XoohCEfTp+9kjAYTk3kw1S0QXgbxap/qsiDm6hGTESiC4D+Z6IiDKqkiiDDTklJCcaOHYvNmzen30skEti8eTMmTJhgu98Pf/hD3H777di4cSPGjRvX4/cpYfjf//1fPP300xgwYEDW79va2gCcqJ/IJBKJIJEwd++l/rJVCj6eeuopTJgwAXPnzsXgwYNx9tln484770R3t32dQTwe71GAEkRMiANgXh5Mkmq7iixQCYNqWsJJGFRSE3YjJ9zGuSMR66REppwW7Rh77EuYujCFavtkZEEkZeT2fAxhTk3I0NDQgNWrV+Ohhx7Crl278I1vfAOtra2YNWsWAGDGjBlZqY27774bN998M9asWYNhw4ahsbERjY2NOHbsRLS3s7MTX/nKV/Dqq69i7dq16O7uTm/T0XGi3m7ChAk46aSTMHPmTPzpT3/Cm2++ie9973t45513cMkllxi7Vqk4nUrBx9tvv40tW7Zg+vTp2LBhA9566y1cf/316OzsxOLFiy33Wbp0KW699VaZpiljt2Q24DxDZKrjybcSpmiqIoVoygJQL5r0AlXJEZUqXVkAgjFSwmnRKqdUQ74ls533PdER5U1FCE5FndlJqqYurDCVzsh3XuljSYoTVcTHeX+t3QueadOm4eDBg7jlllvQ2NiIMWPGYOPGjem+cu/evVkRgZ/97Gfo6OjAV77ylazjpAot9+/fj6eeegoAMGbMmKxtnnnmGVx00UWoqqrCxo0bcdNNN+Hzn/88Ojs78YlPfAK/+c1vjEbyjSf3EokEBg0ahJ///OeIRqMYO3Ys9u/fj3vuucdWGhYuXIiGhob0zy0tLekq1ki8CImY3NQS+UZQqIoDICYPqQ5NZC4HCoGwwqRUUEc9ZKMvFMKQD7frGBJlCdu5GlTREYcT++eXB1HBSG+vIRA9jiXRsWcKhhsRDOUUDZEw6EQZCrWeQZb6+nrU19db/m7r1q1ZP7/77ruOxxo2bBhEplAaN24cfv/734s2kQQpaVAp+BgyZAh69eqFaPSj8O2ZZ56ZDrOUlPT8Mo7FYojFFL7kNdaf0BEHIP9wTEBOHgB1gbBCpmPPFAy30iCqi0tRCYNOlMHtURM6S2TrisOJY9DLA9CzY9WVCMdzuZTqMCkLMts5H8P+dywMTC5Sf9EqBR/nn38+3nrrrazCjDfffBNDhgyxFAahRivkgkW+2FULI1Mci8fIiiRzSeX0KVdutEO17kAF1Ws6HC91JcIA0NcyiKJSSCbyJZ8vFC3aUVDVO9juq1EH4SWq7U7dK8r72l3qbvEjE36k/zXKFnx84xvfQHNzM7797W/jzTffxPr163HnnXdi7ty5dFeRiWaluq44AGLyINrpWeGmQJhAtf2peyYqC6IRhiDUMuSiW4BGKQ4ynRyFQPhBJKzaY1oUMvfJRz5ZEIGjDIwV0jUNsgUfNTU1+P3vf4958+bhnHPOwcknn4xvf/vbmD9/Pt1VCCI6O6RuqiKFTL1DCtHURQq7jlc3naELldCoiBVlOsKrKIMOonMvUKQqPjqWeDrCqtOTSWOkjyPZQYumO9wQEmV5Iqxf4CgDo4JSIaRMwQdwYmjIH//4R5VTqeFQ2+C2OABi9Q4pdCUihWynLSMZbkQ4VKMwbgmDH1AdSZGJiDgAZuTBaj+VfYXP4XV0QqP+QHRfqugARxkYO0K/9oQVVOIA2K+ImYvoEM1cqCQiH35IdehMyiRau0CVivBrlCETKnFIHQuQkwfVzt8NgXALN0QBkO/kwxxl6O6IIhnV+/eZ6PD/v2+vCK805BlJQSEOgHvykCKzYzUlEG6hO3OjTJGjrCyQRRlML1SVZyQFpTikjgeIHlMt6mB1jBR+lQiqtR5UjiMjDCKyIHo8ntipMAmvNAhAJQ5AdqckIhC68gAESyB0BSGFymgIGWEQkQXhKIPH00enoBaH1DFTiA7RBPQ7fdlOlUIyTC/+pHN8amEQPi8LQ8FS0NIA0IpDCpnoA4U8AGqdMpVoUAmBHTrDJqmFQRgXhUFk3gYT4pB5bEAu+pCJyeiBm6s9ymBy0ame56I9LgtDYRNuaRCc7MmEOABy0Qe7jlFXJpww3dnroDu/gilZEIoy+CTCkItJcUgdH5BfNdOuA/VrKkIFCnkxXbfAwsCIEG5pkMCUOKSQrX1IYdV5mhQJr6CYhMmzuoVMPBIG0VkiTYtD6hyA+pLbH7UhmAWRlNENlVEMpoocWRgYoBCkQWJqadPiAMgP17Qis4MNmkBQyEEmKqMhVGUhCCMmRJARB8B7eTjRFnfTGvkwmfbQGe6oIgw8vJKRIfzSIIlb4gDIRx2s8CKtkQ9qMchFZ9ikijAEpfBRZk0KuYmbTvxXRx4oxCEXv9YrqEDRcZtIR6S3lfwnrTIFOhMMCkMaJBeykhGHFG6mLEQw3XG7je78CkZlAZAWBpX1U6iR7cwzOyXVYslcTMiE36F6sldNQ8ien4WByaQwpEEBUXFIkeqUdOQhExMiERSoJmByLQ0RQGFIoRoF0I0+ZJ4/l7CIhImwv069gkp7WBiYXApHGhSWzZYVB0A/+pDCruMMk0yYWChKp7hRqWbBZ8Kgsmy2TvpAJ/rg1J5c/C4SJusCdAsbVdvGwsBYUTjSoIiKOKTQiT7YEcSohOlVJHVHQSgXOLogDImyBCLH5dZMUBUHQK9zNiEQKag6Zbvr81sxINUICLeEgSkcAisNkXgREjHJ6unUl7xCxAGAtjykoJQIgK5TtpMPPy4dTTVc0i1h0EFVHFLICITMTI9O5HZ61BKhit/kAKAfIqm9JHbQhaE9ChRp/vv06VwrfiCw0gAoigPgmTykMC0RqvhRDgD6+RS0h0568IWiIg4pUp2AavQhE4o0hh1+EQtTmFwkikKGAi8LjCsEWhoADXEAtOUhEx2REOkU/SIWJjG5FDXZHAsePoHoiAOgLg9Zx8jpnChrDZw61aAJhVurSJIthc3CwAgSeGkANMUBUJaHTHI7Jd1oRC5OHWoQhcKkIKQgnYzJJ+FKXXEAaOQhfSyidEY+wryUsyzUKRYWBkaGUEgDQCAOAIk8pDAtEZm40QEHCfKZG30iDCkoxAGglQfAbBSi0DFVi8HCwMgSGmkAiMQBIJWHFG5KRCFibIpnnwlDCipxAOjlIX1ciY6uUAXDy8JMFgZGhVBJA0AoDkB2h0EoEIBcJ1eoguH5Wg8+FYYUlOIAmJMHoXM7dJ5hEAq/jdpgYWBUCZ00AMTikMJA9EEUp84zLELhuSDk4nNhSEEtDoC38mCF3zpcP8MywJgmlNIAGBIHwFN5sMJ3na2fCYgIyGJCHAC1DsgvolFIsCgwbhJaaQAMigOg1gH5RDQKjpDKQiamxEGW3A6MJUIPr4WAp4Zmcgm1NACGxUEWgzUSBYdPRMBPi0/5RRwyoej0wigeXsuACCwMjBWhlwbAZ+KQgqrTC6t8+EQKnPCTMKTwozjoEoQONmywMDB2FIQ0AD4VBwoC0LmGET8KQwqdL/ywCQcjT9CFIdJRhEhE899nh3//fXtNwUgDoP9FH0rpYKTxszDokuowWB6CS9A7fcbfFJQ06JLqLFgegk2YO30qKDseFpD8cEfPBAWWBgWoOh2WD3G4ow8u3CEyTHhgafAQ7ggZhmGYIMFxQ4ZhGIZhhGBpYBiGYRhGCE5PMAzjClzDIw6nLhm/wpEGnxA5HuGXwItxl0QsSfZixOF7Hl6am5sxffp0lJeXo7KyEtdeey2OHTvmuE97ezvmzp2LAQMGoG/fvrjsssvQ1NSU/v2HH36IyZMnY+jQoYjFYqipqUF9fT1aWlrS2zz//PM4//zzMWDAAJSVlWHkyJH48Y9/LN1+jjRowh2Zu1De70Ko6ueOo7DJ/Pw5euEPpk+fjgMHDmDTpk3o7OzErFmzMGfOHDzyyCO2+8ybNw/r16/HY489hoqKCtTX1+PLX/4yXnjhBQBAJBLBl770Jdxxxx0YOHAg3nrrLcydOxfNzc3p4/bp0wf19fU455xz0KdPHzz//PP4+te/jj59+mDOnDnC7S9KJpO+/1ZpaWlBRUUFTr37DkRKadfJ5U6fCas8sDAwVGhPjNfejvfmL8KRI0dQXl5O1KpsKPuJVHv37duX1d5YLIZYTH1e8127duGss87CK6+8gnHjxgEANm7ciClTpuDvf/87hg4d2mOfI0eOYODAgXjkkUfwla98BQDwxhtv4Mwzz8S2bdvw6U9/2vJc9957L+655x7s27fPtj1f/vKX0adPH/yf//N/hK+hoCMNLAwMoPd34FfhCIQwhHXdFCB007snYsnARCoixyOIJDW/29tP7F9TU5P19uLFi7FkyRLlw27btg2VlZVpYQCAuro6RCIRvPTSS/j3f//3Hvts374dnZ2dqKurS783cuRInHLKKbbS8P777+Pxxx/HhRdeaNuW1157DS+++CLuuOMOqWsoWGkIgjDw6n7+J3I84jtxcFUYwtzx60BxX3wmHkESByqsIg06NDY2YtCgQVnvFRcXo3///mhsbLTdp6SkBJWVlVnvDx48uMc+V155JX7zm9/g+PHjmDp1Kn7xi1/0ON7HPvYxHDx4EF1dXViyZAm+9rWvSV2D/3tOYtwsqIvG9V5hRPee+PG+uPH35Jvit9Lu7BdjDh/e50BEsAgpLy/PetlJw4IFC1BUVOT4euONN4y398c//jF27NiB3/zmN9izZw8aGhp6bPPcc8/h1VdfxapVq7BixQr88pe/lDpHaCINfokc+LFTCxu599gP0Qs/RhzI8FGnVbCofAaGIhWFGHHIxw033IBrrrnGcZvhw4ejuroaH3zwQdb7XV1daG5uRnV1teV+1dXV6OjowOHDh7OiDU1NTT32qa6uRnV1NUaOHIn+/fvjggsuwM0334whQ4akt/n4xz8OABg1ahSampqwZMkSXHnllcLXGhpp8BIWBW9Ruf8mRCNw4uAzGYjG/NUeVbrjPkkrpD5fA/LA4pDNwIEDMXDgwLzbTZgwAYcPH8b27dsxduxYAMCWLVuQSCRQW1truc/YsWPRq1cvbN68GZdddhkAYPfu3di7dy8mTJhge65E4sR3UTxu/wWZSCQcf28FS4MDfpOBaLvXLdCnm3bwizKpz5ZaHgIjDh4KQ1jkwA6n6/NEKDI/a0KBYHGQ58wzz8TkyZMxe/ZsrFq1Cp2dnaivr8cVV1yRHjmxf/9+TJw4EQ8//DDGjx+PiooKXHvttWhoaED//v1RXl6Ob37zm5gwYUK6CHLDhg1oamrCpz71KfTt2xd//etf8b3vfQ/nn38+hg0bBgBYuXIlTjnlFIwcORIA8Oyzz2LZsmX41re+JXUNLA02eCkMYZADO5yuzQuhMCEPvhcHQ8IQdhmgQPYekUsGcfSBxUGetWvXor6+HhMnTkQkEsFll12Ge++9N/37zs5O7N69G21tben3fvzjH6e3jcfjmDRpEn7605+mf19WVobVq1dj3rx5iMfjqKmpwZe//GUsWLAgvU0ikcDChQvxzjvvoLi4GKeddhruvvtufP3rX5dqf2jmaaCsaTAlDGGWAbcxJRiU8kApDmQFaMTCEERRiJV2AADi7SUet0QOI1EKInnIJw5uztPw8SV3kszT8M6SG422N6hwpCEHamEIkigU57S1yyepBCty7yuVRFBGHnwXcSASBrdEIdW5B+34pmQk976TSARR5IEjDoUDS0MGVMLglijkdvJBOL4pEaGWCCp58IU4BEAWTAuCmzhdC6VQkEoEgTywOBQGLA3wvyyYlgM3cboWSqHI/Cx0BIJCHjwVB01hoBaFMMmBCiLXryoWVp+VtEhoygOLQ/gpeGnQFQZqUQiTIMiS79pVpcLuM5KRicy/ExWB8EQcFIWBShT8Jgh9Ys7taY37o8Yh877pRiZSn6WyPADSAsHiEG4KWhpUhYFKFPwmCH4b2ZALdc1F6nplr01VIFwVB4+EwaQo5Ov0/XB8avHIvZ+6UQilFEZpd6DEIRIHorqn9tlwez9RsNLglTCYFAWTtRQUx6YWDyqJUJUH4MTfkWlxkB45oSAMOrJgQhRMC4Ip7NpNJRO6UYhorNs/k08xgSQU0iA73FJFGHQ6TWpRCNKIjEys2k0pEqn7rCMPfoioZGJaGFRlgUoUgioHslhdp65IqA4dVRIHhWgDE04CLw2mhUG1g6YQhaDKgQwU9Qa56MiDn8TBpDDIygKFJBSKIIhCJRIq8sARB0aVQEuDSWGQ7bBZEmihiEpkfiYyAiErDrIpChFMCYNKZEFHGEyKQt+Y94nnY3HaDz51v3TkARATCGlx4GgDgwBLgylhUOm4dYTBpCgUt3s72WdXKW0hlE7tgWz0wU8Rh7wICIObaQhKUfCDGDhh1T4KkdCRB+DE52ZEHJiCJ7DSQI2baQhKUfBaDJzIbRuVROh06DLyEChxcMCN6AKFKPhdEETJvA5dgdCNPJCLA0cbCp6CkAYTa0nIyoL+qAv/yoEolBKhE3U40RZacTCRohCCeC0JGVnQFYWwSIITdtcoKxN9Yh3+EQemoAm9NIgIg0yHLioLLAn5sbpGWZHQTVkEOuJAmJZwSxZMikJljDbXdzhu7kNXSWtk3ncZgSAXB442FDShlgZKYZCJLOgNzzQnC1THpq5VyERVJFTlgVIcXI02EEYYTAsDlShQSwH1+XQlQyatIZu24IgDQ0VopYEqJeFGGoKqM3crOqFyHh3RyDxfvuOorDkRuIgD0UgJk3ULOqLgthxQkdluCoEQSWHIRB9IxYGjDQVLKKWBaqSESWHQ6eCDmLqQ6fhFjkMdfaASB+PRBqIIg4noQiGKgh2516MiEan7KVr/IBJ94IgDo0sopYECE+mIQhMFOygEQkUeAOcOP3ARBweoFp0SEQZVWXBDFAbEWqW2/zDex0g7dKIQ1PIQdnGIxgHtVoe/RleZgpUGqmGPIsdR7fBNi0Jxm9waCF295ebGEGqDpkAUtyel9svX4VOIg2cjKQQRjTKYEgYqWZAVAhPHVJEM1SiEijwYFQdOURQkBSsNTlCOkFDp+MlqHCSlgPp4spKhWgQpE3UA3BEHL3GKMogIg4l0hKoomBADSqzaJysSshIhWu8AuCAOTMFRkNKgG2UwkY5QjkYQiwEluW1TiVTIRCJkUxYU4mB7fJ9HG6ygloWwikI+MtuvE4lwkgeZqAOFONjC0YaCQynevHLlSgwbNgylpaWora3Fyy+/LLTfunXrUFRUhEsvvVTltK7gFGWItotHF0QkILWdrDAUtyXSryCR2W6VtsvcVxEoCmFVBFR2CnQZdKMM+RARhspYu5IwDIi1GheG/r1ae7xMkromlesSuY99Y3Ghz8RJCEX+LlRqZKTXT2ECgXSk4dFHH0VDQwNWrVqF2tparFixApMmTcLu3bsxaNAg2/3effddfPe738UFF1yg1WBdnL7kKRadouzUeuzngiQUH8+JDpSZ6+Ayr0cmCiESURCNOpiMOLgdbdApfqSqX5CVBUpJUBUA0f2aO/WKJFUjEJWxdqGUBeAceTAWceBoQ0EhLQ3Lly/H7NmzMWvWLADAqlWrsH79eqxZswYLFiyw3Ke7uxvTp0/Hrbfeiueeew6HDx92PEc8Hkc8/tEXVEtLi2wzLdFJS1DVL6hEFajIFQLKfXTlQkUgqORBVxxUahsixyNIlLkXKdKNMlAKg6oomI4K6J5fRipy70E+iRBJWQD56x10xEGltiERSyISNzcZHOM+Ut/0HR0d2L59O+rq6j46QCSCuro6bNu2zXa/2267DYMGDcK1114rdJ6lS5eioqIi/aqpqZFpphK6UQZKYVAN4RcfTzi+TEJ5btnrl0kF2UE9Z0f6uAEYuuUUZRAJf4umI1TD9G6kESjQSXmI3huR+5zvM9NNVVhCvAYK41+kpOHQoUPo7u7G4MGDs94fPHgwGhsbLfd5/vnncf/992P16tXC51m4cCGOHDmSfu3bt094XxNf0s4pjfx5duFcvGqu3wUp0EW1jaICQVHvYHKZchlUcsGqtQz5hCEfpmTBrZoDU+jIQz5Mz29BUfvChBejoyeOHj2Kq6++GqtXr0ZVVZXwfrFYDLGYfYiNupAsX/Gj/X400QVVUQgiqXarpDOK2xJ5Uxci8zY4beOUatAdUeEFqsIggqgwyGBSEqp6HbP93aHOvkbOmboe2dSFSLpCJ1Whumomw0hJQ1VVFaLRKJqamrLeb2pqQnV1dY/t9+zZg3fffRdTp05Nv5dInOg0iouLsXv3bpx22mkq7ZbGrvNXFQYRTAiDSVmIHu+yfL+7jN4tM69DRiCoxCGQuBgCdooyUMsCpSg4iYHOfrpSISsPVOLAMNRI9QYlJSUYO3YsNm/enB42mUgksHnzZtTX1/fYfuTIkXj99dez3lu0aBGOHj2K//zP/ySvVXAzf+wkBNSyQCkKdmKgug+FUMhGHyjEQVUs/BhtsEtNmEhLUAqDjiyoyoGJ88kIhYw8UIiDzERQmWjP38CEFulv/IaGBsycORPjxo3D+PHjsWLFCrS2tqZHU8yYMQMnn3wyli5ditLSUpx99tlZ+1dWVgJAj/dNQh1lcEsYdGRBRQ5MnEdGKmTkwWTEQWU0hPHZIQmjDF4Lg4osuC0JMmS2TVQgMu+Bk0CYFAeVFIXtCAofDbksbgeimlNEFAWggNkrpKVh2rRpOHjwIG655RY0NjZizJgx2LhxY7o4cu/evYhEzI3tt4MyyuC1MKjIgluSIEuqXSbkQVccQpvGcMCUMJiKLvhZFqyo6nVMOpWRL/rgVaqCow2MFUqx5fr6est0BABs3brVcd8HH3xQ5ZSuYUoYTEUX/CoLuZiSB1MRh6AURKqkJqihji64IQpVxUcBAIe6+tEf+5/tp5QHXXGgjDYwhU3o155QSU1QQx1dCIooWGFCHnTEgTLa4NcFrFKYiDJQCgO1LKTEQHYbKpHQkQcT4sAwFIReGmQwEWWgFAZqWYi2dUrv0927F825FeXBL+Lgh2iDzrTRmZgSBrdkQUQOqI6nIhQq8mBCHFSiDVYpCsu6Bp5KumAItTRQTdZjShjckgUVOVA5lopQRI93+UIcLNvm88iBHTKpiaAJA7UgyGJ3fhGZkJUHjjgwfiTU0mCHTGoiaMJAKQiy2J07n0zIRh10xcFyv4BGG0TRncgpEydhCKss5EOmRkJGHqjFQWUIJhdEMpm4P8zBp1BOIewkDKLTKUePdwkLQ7StM/3yI6JtS12zyHU73UORKadFkf27cGsqaorUhO7wylyohaGq+Gj6FRRk2lvV65jQvbC7r6oLf1l97rJSSZUaY4JHaKWB4stbd3hlj30MyUJQkGmryH2gFgfVJcvz4eakYxSjJlTSEiLCINpJmhCF6l6H874oUZEHp3ujKg6m16nIghetEqK5uRnTp09HeXk5Kisrce211+LYMfvPvrm5Gd/85jdxxhlnoKysDKeccgq+9a1v4ciRI1nbbd68Geeddx769euH6upqzJ8/H11dH32H7t69G5/73OcwePBglJaWYvjw4Vi0aBE6O+X6kIJLT1CMmtAtfLRDRhYoibTm79USfeRnlbMi1XbR+od8qQvqVIVVmiKotQ0pZJ4idUdKWCGTiqCQBVUBsNuvsbNSuS2yQzud5nnQSVXkYpWm4OGX7jB9+nQcOHAAmzZtQmdnJ2bNmoU5c+bgkUcesdz+/fffx/vvv49ly5bhrLPOwnvvvYfrrrsO77//Pn79618DAP70pz9hypQpuOmmm/Dwww9j//79uO6669Dd3Y1ly5YBAHr16oUZM2bg3HPPRWVlJf70pz9h9uzZSCQSuPPOO4XbX5RMJs08XhHS0tKCiooKnHr3HYiUllouWJX7NCcz1NJ+W7mn0XxpCTtk6hZ0hUFEEGTQlQmZ4kmnmgenuRycxMGqjsHqPSdpyK1rsNq22+I2Jcqs/yZ6rHJp8QQnMz+DlTTYpSbspEG1jkEmFSELdXRAFhWZEJUHp3oHu0mgnMTBqijSrrbBThxy6xpEZ4aMxIuQaG/He/MX4ciRIygvL7dtpw6pfuKs6+9ENKZn+d3xdvztpzeSt3fXrl0466yz8Morr2DcuHEAgI0bN2LKlCn4+9//jqFDhwod57HHHsNXv/pVtLa2ori4GDfeeCM2bdqEV155Jb3Nb3/7W1x++eX44IMP0K+f9d9dQ0MDXnnlFTz33HPC1xDa9IQusqFq08KgkoqItMZ7vKjRPYfMdTndK51URY/tLT57mXSXV0ts66YmqOsYqIXBZDpBBZX2yKQs7FBJVVh9tiJLnztRCHUNLS0tWa94XO+ebdu2DZWVlWlhAIC6ujpEIhG89NJLwsdJyUxx8YkHqXg8jtLSbFEqKytDe3s7tm/fbnmMt956Cxs3bsSFF14odQ0FJQ26qQnqOgbqdIRpQRBFpR0yxZJ2qIiDyaLIrH09+jgoRk2opCUoaxf8Ign5EG2n6HWbqHEQQetvxgd1DdH4iX+rWq9//nutqalBRUVF+rV06VKttjU2NmLQoEFZ7xUXF6N///5obGwUOsahQ4dw++23Y86cOen3Jk2ahBdffBG//OUv0d3djf379+O2224DABw4cCBr//POOw+lpaX4l3/5F1xwwQXp7UQpKGkwhUodg4gwiHakXktCPqjlwY2Ig6miyEys0mxuIPOEqZKWyCcMMkWCJmRhYPRY1osaSnmgFAcT0QYReqTbAsS+fftw5MiR9GvhwoWW2y1YsABFRUWOrzfeeEO7PS0tLbjkkktw1llnYcmSJen3L774Ytxzzz247rrrEIvFcPrpp2PKlCkA0GMtqEcffRQ7duzAI488gvXr16drHkQpuELIXHRDyappiXzIRBfIaT1+4r99ykgPm2qrSB1EtK3Tsd7BaVIo2eJI3QmfgjRfgx2yqQnVZa1lahcohcFJDjJ/d7BbbspnJ6p7HRaqe6gqPkq+BoZKcWQmXBR5gvLycqGahhtuuAHXXHON4zbDhw9HdXU1Pvjgg6z3u7q60NzcjOrqasf9jx49ismTJ6Nfv3544okn0KtX9vdjQ0MD5s2bhwMHDuCkk07Cu+++i4ULF2L48OFZ29XU1AAAzjrrLHR3d2POnDm44YYbEI2KzegZCmmgDv1SDMdTTUvI1C2QCkNKFJzeI5KISGvcuDgEFcqnMt3UBHVaws3ogmoEIXc/XYmgEAeVERWMuwwcOBADBw7Mu92ECRNw+PBhbN++HWPHjgUAbNmyBYlEArW1tbb7tbS0YNKkSYjFYnjqqad61C+kKCoqShdT/vKXv0RNTQ3OPfdc2+MmEgl0dnYikUiEVxooQ7omowy25yQQBhJZsJIEmX00BYJKHOxwijYIHyNAS2eLFkFShKNVogxuCIOJVANFFMLLiIMIorNE8syQ+px55pmYPHkyZs+ejVWrVqGzsxP19fW44oor0p39/v37MXHiRDz88MMYP348WlpacPHFF6OtrQ3//d//nS7KBE7ISqqzv+eeezB58mREIhE8/vjjuOuuu/CrX/0q/fu1a9eiV69eGDVqFGKxGF599VUsXLgQ06ZN6xG1cCJw0mAaily2SlrCFWFQEQXRYylIBIU4qEQbdFMUYcMqNaEyiZPuwlOywmBCEmTOJyMRuuLgFG2wQzdFwZhh7dq1qK+vx8SJExGJRHDZZZfh3nvvTf++s7MTu3fvRltbGwBgx44d6ZEVI0aMyDrWO++8g2HDhgEA/t//+3/4wQ9+gHg8jtGjR+M3v/kN/u3f/i29bXFxMe6++268+eabSCaTOPXUU1FfX4958+ZJtb9gpEFn5ISMSFDNx2CFtDBQSoLMuSQEwmTEgSLa4Ef8ONRNNy0hKgxui4ITslEIUXGQRSZFwQtZeU///v1tJ3ICgGHDhiFz+qSLLroIItMpbdmyxfH306ZNw7Rp08QbakP4vlHhzjh52dSEblpCWBhaj3/08grJNsiMrLB8n3jJ8LztcHMeBsIhbFapCZkCSFNpiSAKQy6ibRMdVWH5vmYkh2EoCKU0qKKbmjCVlpASBr9BLA6y2H0mOtN9BwGdIkjZAkidziwMwpCCUhxkMTFvA+XKqEx4YGnIg+VICsIoQz4CLQwpCMXB7QW63JivIQiozMmQL8oQJmFIQSUObkcb3JivgQkHBSsNri1hrJGWCIUwpDAccVCd8ClrOw1ByK2ZcTOFoTp9tMkVEP0oDAMjcdsXJV5FHETTR66ufMmEjoKVBhGoCiCt8KMwJNvaLF9kEIkDVbQhiCkK1SJI0SdJmXC26lOvSGdJNVujqBhQSwSFOFBFGyimlk6RK6h+LMplzBL40RMiEzuJjJwQFQSZjkY1LSEkDASyICoEmdsV9e6td9LW40KjK0RHVWSiOksk44xsAaRTlEFUGFShihpkHudgQm0l14HRY9qjKmTnbuDJnk58l0cTemnFog5OS9rB36IEUEYZTAuDbgSBJAJBEHEwNZJCVB69WsnSJBRRBq+EwUSaIffYKsc3larQrW0QTVFwMSSTC0uDy3glDOSpBmgKhMejKlKIRI6CVgwp8kUv2mmorjGhgqwwmKpJoD6nrjjIrNcBWH9mIkLIxZCMCIFPT6gg8pSoO2rCyNwBksJALQky5xJKY2imKuwmfLJLU4QxRaFaBGkC3SiDCG4KQj5kUhiiqQoZVGaJZBhdwvUN6gEyqQntKIMARgoYNdqRF59EHDIJWmSBApMFkFRpCT8JQy4ibRO5RpPRhlx4FAWjAkuDAWSjDFRpCa9FwQpKcbBCtrZBZ7nyfOQbdkm9GqsoqmFnitREIQhDCtPiYAXPEsm4DUsDxJ4srVITFB1QmIUhBZU42N0r3SGYqnUNYSyGzIdKAaQTbgvDwGhJjxclVOJgheo9TkFV15B32CXh1OeM/2BpcAnljs1lYUgea816kR3XcMTBCp1oQ5hTFCJhaTeiDG4Kg5MgUAsEhThQRBuoUhQ8goLJJHTS4PXTn0xqIm+UwUVhsJMESoGgEAdT0Yaw4MUXvO4TsB00kyzJyQCVPJiKOJi61wwjSuikIReVJbGF0hUGc+OiUAiDjBBQyIPbEQe3V8BkxHBjPQmdzp86bcEwYSH00pCLF5EIpadgF9aTUBUAyrSFXwjjfA3UBKnojiRaECBxCNJnwwSbgpMG5gTaEQON/f1ctFmIUK5NIILpWgbKzl4rWhGAER8MI0tBTu5kCjdD4TodL1WkIHmsFUV9Dc1z7zDxk8q6FIWKV7P8Ua/gKIqJ6MDAaAkOdpupFXGa9MlpTQrGnmg8iWLNtSfQWdhRRSc40uARjkWQBlMT1KmFMKYqGHH8VJhnMp0QpFQFw5iEpUGAIC6hbIWpDt4v4sAjKMKB2sJQ5jv1MIgDr0HB6FLw0mC6uM1ER6aSmjDdscse34t5GzLxw+iXQsTEqAk3O3PZc5kYeumn6A5TeBS8NPgOAx2lW5EAv0QcGHtUJ3bya3W+F0//YYg4MIwqLA0hx+2O3K3zySxgxXM1uItOEaTcktPedd5BEQc3lzZnCgOWBgWsQtukM0E6IJOa8OrJX3iyKB56WVBQpib80GmLtkFn6CUvXsX4DZaGHMIygY/XqQKy87swyRWjhle5dT8IQwqy9SpcmCGTYShgafATRB2k18KQwi/tUCUsAumE2xM7OSFWNOgfYUjhxzYxjClYGgxCPXJCJJwftI6aOkXBwy4ZP+Ln2SFFimMZJgVLA2MUEolxKUURlvk4vMQuB08RfvfzEz3NWhfi94iHXTJewdLgFwg6xqBFGXTRKSilwuul2MOCn5/EGYb5CF57Ig/UT5+qHV2YRxok29pQ1Lu30XNEj3ehuyzcf+59YmbWR+CqfBoGRuI4mJBfM4V6DYoBsVZ8GDe0ZowPKG5LoLiX5vd2J0cd7eBIA2Mcv0ZAeFbInpgY1x/21EQKP7SRBY8xDUsDAX6YPMivHTMZPPTSV3BO3Qw89JLxOywNDENIcYHUOBzq6ud1ExiG8QCWBgL8kCsv6hveHCUAoE+Z1y0oCJo7Q/535HMOdvf1ugkM4whLQx66etPeokQf+UIoAMYLBU3iV6HpKuM/fzeg6AgPdpsp8qTED2081MnSwZiFvzUZz3FDiExFg7pLs3/uKrXezg1a42YK8dzoiFRGFQSNQrhGJvywNBiku3cv8Y0Jwu9+fKInaROnJoxBPfSOcmhgoWEXkaG+p2EebsmYh6Uhh67SIq+bYEuQUxQmUE31MO4T9hSFn9vG+Ivm5mZMnz4d5eXlqKysxLXXXotjx5xHzVx00UUoKirKel133XVZ2+T+vqioCOvWrbM83gsvvIDi4mKMGTNGuv3eV/AxpBT17eOb4Zd+jHwwdBzq6kc69PJgIsYzQwrCo1eCy/Tp03HgwAFs2rQJnZ2dmDVrFubMmYNHHnnEcb/Zs2fjtttuS//c2+Ih8oEHHsDkyZPTP1dWVvbY5vDhw5gxYwYmTpyIpqYm6fazNPiJPmUk8xH4QRxEhYE6eiKVEsqBuujVjxyOlwZ6gaKD3R2+mEQpE5EoA9czMACwa9cubNy4Ea+88grGjRsHALjvvvswZcoULFu2DEOHDrXdt3fv3qiurnY8fmVlZd5trrvuOlx11VWIRqN48sknpa8h/N+SLiFTaKcTVhftZL18yic9N9czMDn4KRVA1RYeaulPWlpasl7xuF4kbNu2baisrEwLAwDU1dUhEongpZdectx37dq1qKqqwtlnn42FCxeizWJpgblz56Kqqgrjx4/HmjVrkEwms37/wAMP4O2338bixYuVr4EjDQp0lUUCMQWxFxEHTkkwdhzs7pt3xkPRFIUfIg5uyItMEaTqKJfDcQ+H/BiguD2B4i7N7+d/7l9TU5P19uLFi7FkyRLlwzY2NmLQoEFZ7xUXF6N///5obGy03e+qq67CqaeeiqFDh+LPf/4z5s+fj927d+Pxxx9Pb3Pbbbfh85//PHr37o0//OEPuP7663Hs2DF861vfAgD87//+LxYsWIDnnnsOxcXqXT9LQ8hxUxxkhUEnNSETrfHD5FtB51BnX+F1DRo7K22XyKbES3GQEQYvUxM8WZce+/btQ3l5efrnWMz6s1ywYAHuvvtux2Pt2rVLuR1z5sxJ//+oUaMwZMgQTJw4EXv27MFpp50GALj55pvT23zyk59Ea2sr7rnnHnzrW99Cd3c3rrrqKtx66604/fTTldsBsDQYp7t3L0TbOsV3EKhrKOrdW2rVSzfEwUiEwWBqQmRiJz+PpGFO4IU4UEcYODXhX8rLy7OkwY4bbrgB11xzjeM2w4cPR3V1NT744IOs97u6utDc3Jy3FiGT2tpaAMBbb72VlgarbW6//XbE43EcP34cr776Kl577TXU19cDABKJBJLJJIqLi/GHP/wBn//854XOXfDS0FVahOL2pPM2vSPkS2S7jUlxUBEGHj7qX5o7+xhZ7RIQS1HIH9M9cfBTPQXjHwYOHIiBAwfm3W7ChAk4fPgwtm/fjrFjxwIAtmzZgkQikRYBEXbu3AkAGDJkiOM2J510EmKxGHr16oXXX3896/c//elPsWXLFvz617/Gxz/+ceFzF7w0FBImxMFPNQw6IycYNaiHXQJqQy/dEAcVYTCRmjA93PJYnEd6mOLMM8/E5MmTMXv2bKxatQqdnZ2or6/HFVdckR45sX//fkycOBEPP/wwxo8fjz179uCRRx7BlClTMGDAAPz5z3/GvHnz8NnPfhbnnHMOAOC3v/0tmpqa8OlPfxqlpaXYtGkT7rzzTnz3u98FAEQiEZx99tlZbRk0aBBKS0t7vJ8PlgaPSPSJIdJq88VINPTSCkpx8JMwMIxJcfAiwmBidk2eDdJ71q5di/r6ekycOBGRSASXXXYZ7r333vTvOzs7sXv37vToiJKSEjz99NNYsWIFWltbUVNTg8suuwyLFi1K79OrVy+sXLkS8+bNQzKZxIgRI7B8+XLMnj2bvP1K0rBy5Urcc889aGxsxOjRo3Hfffdh/PjxltuuXr0aDz/8MP7yl78AAMaOHYs777zTdntGDNm6hqx9CcRBRxiEUhMO9QwmZoIUmaOBaxzEcasYMhcT4qAqDCJRBq5nKDz69+/vOJHTsGHDsoZK1tTU4H/+538cjzl58uSsSZ1EWLJkidJIEOl5Gh599FE0NDRg8eLF2LFjB0aPHo1Jkyb1KO5IsXXrVlx55ZV45plnsG3bNtTU1ODiiy/G/v37pRvrd4JUpa/V6QcowhCkz8QEIqFmt58+RTpKnbA+ZVQgKDUMvLol4xbS0pAKecyaNQtnnXUWVq1ahd69e2PNmjWW269duxbXX389xowZg5EjR+IXv/gFEokENm/ebHuOeDzeY1INhh6lAsYACQNTuFB09kERBtOYWj2VCSZS0tDR0YHt27ejrq7uowNEIqirq8O2bduEjtHW1obOzk7079/fdpulS5eioqIi/cqdYCNoKBXoCQw3pBiBUNS3j5AIiG6X9ziaqQk7uAjSPEF7mtXp9HWFwVRqQqYIUmSOBpWJneLtLBGFjJQ0HDp0CN3d3Rg8eHDW+4MHD3aczSqT+fPnY+jQoVnikcvChQtx5MiR9Gvfvn0yzXSkW+DfiEju2m6cP+l00i6JA/CRFGSKgdV7WucgEAaKegaqORpE/pYKAZVqftMpio/O0yE3EZPk9qbgJcYZv+Lq2hN33XUX1q1bhyeeeAKlpfbfuLFYLD2pRr7JNbpzvldyv8i7qCRBczEj5SdhF8UhfTxCUUgf01CEwYlCr2ewQ+Tp0u4pVSbakK/jc0scTpyrw1YInH6ndC6XCyCDFgFigo3Ut2pVVRWi0WiP5TSbmpryzma1bNky3HXXXXj66afTY0uDjt0aFN1lxYge7xI6huPQyxQGZol0EyphsIsyyAiZSJQhyLTGS9An5v6Tsup8DZTrUYif09z9oRAGO9nSnZ9BpOA1DHM0RI93IVos9v1rR7JLb/8wI/UNWlJSgrFjx2YVMaaKGidMmGC73w9/+EPcfvvt2LhxY9bqXn7GxPA6u85NKOTuQcSBAtPCYIdMlMHXwy3bo1k/dsezfxbJL1OPoKCMNgDuRhxM4sUQS7vPgtecYEwh/djV0NCA1atX46GHHsKuXbvwjW98A62trZg1axYAYMaMGVi4cGF6+7vvvhs333wz1qxZg2HDhqGxsRGNjY04dox2KlldqFMUsqFxSnFIvbyCug1O98ZPBZAiqTC/oJOisMPuSTjs4nAwESMTBlNRBoahQjrpO23aNBw8eBC33HILGhsbMWbMGGzcuDFdHLl3715EIh91rj/72c/Q0dGBr3zlK1nH0V1i1C/ILpPttIAVVaoiRW6nbTJ9oSwIBhelyiXsqQk3kVn1EhCb7Ek0VZGCMmUhi6zA6AiDLKKyJyKOPNySyUWpUqy+vj69UlYuW7duzfr53XffVTmFUbpLgWh7/u1EFrNyPI9NbYOb4pBJZsdOIRDakQTNtIRdlEE3NRHGkRPH4jH0jTn/XX0Y74MBseyZQmUXr3KqbaASh/S2OR23GxKhEu3QFQa7KINMmsjkBF65KTMm3PCjFwHUT7BUqQonVFMIZKkHQ8JgB0cZeqIyRj8Tlap9qlSF5X7/TBNQpzF0jutmhAHgWgbGPKH8JlUZdimK7NBLu6fefJ2eG+KQwkkEMn9HUqPQp8xI4WMK6gJIHbyKRlCHlKlqG1KYFIf0/jodfca+etNZ6wsDRZRBlDCMnGDME0ppoES0at7Ek6yb4pCCXBIyEWxrvus2Wfxo9XkX0kJVsmFsu87LD+KQPo6AAFBHKSjaTlX8aPWZqkaZeDZIhqUhA1Odg2q0AfBGHIzggjDY3Wc3UhNujpxQGXapi4mwt5vikD5eThTBSDpDsM2qaQkeZsl4ScFKg07o2C6sbapzCrw4EAmDE7JDXHVSE0ErgkxhFX7WrWsA1KMNgDfiYBIqYTAZZRCFR04wVhSsNLiNTrQBCKg4CNYvAGLXp5KWkBE502mI3CnP/YpdR2PqSTYs4uCGMFBEGShE0ZF2Hk0RZlgaBJDpTEyGwoXFIfPlNgrn1hUGk2tMBLWewe2nRJ1oAxBscTjY3dd4SgIwu8aEahEkD7csPHhFH0W6ekdQ3CY+qROgNm9DLkLzOGSS2XkrzO0gfQ5JKFattMNO4FTnZpAhqCkMLxGdxyET0TkdqJEVGFFZUElL2EUZqOdmCEoRZPR4F6JRse9TO5LdvPaEHQUTaRAtVPPqyVIm9K7c0VJFIYiOI3odXhU/Co+cCbggyISrncLgutEGQP5JPPWUbzoKkXkeL4SBKspg9VnzUEtGBo40GMBpammnFTCNRhyskIlCEKY6ZKSHOi2hvcR5wAVBFKvZIXWQWQVTJOJgBXUUgkJETEYYAHN1JpTprUi8CHIxWcbPFLQ0iE4nbYdKiiIfrotDCpfqHyiiC/mQjTJ4FV1KlLn/VSoynXQ+nKaWdlqTwg1xyCSz0xcRCOpoBZUwqBQ/Us7NIAUXQYaewKUn3PiilZngR2X4Zb4n5O7evWhHVfgESmFQSUvIRBmCVgBplW/WfVo0sV6ByVSFE3bpBVPpDdPC4ITM58apCUaWwEmDFTpD2SjCzSbEARCXh0SfmK/lQbR9otfrRVoCCE9qQma+BpXhl/k6ukNd/aRGVVDKQwpTdRCi7RW5B0730eRETjw/A+NEKKTBCt0veNnphE2JAyAvD5kvt1Ftg2hkxemeqRQ/6kYUqIsgI3HaCAdFB2BCHAB5efAzMnKjO3GTbFoCcCk1wRQEBVXT0FUKFFvUMMjUNjgtl21X46BaGNlj2392rKI1D4B9SoCiFoJCSihkAaBLSwA+TE20R4HS7qy3uuNRRGPdNjvkx6q24XC8FJUx638IdoWRqvUNWdv9sxPNV+9g1ynr1j7Iojz9s6AsqI6UkE0nyaQmgjLckjFPQUmDE1bi4CQIdqiKAwApeZARBytkZMJUxIJKGJxwEgYZOQhCaiLeXoJYaUeP91vjJegT6/m+FV6JAyAuD7lYdeJUIkEV4aASBpW0hEyUgVMTTD5CIw3dMSBKNJAgH/lkQkUcADl5UIk6iOBGSoNaFmQmcUr/jiCaEKT5GazEQWUkhao4AJCSB1lxyEU2KmEq/SGTitARBhNFqyk4ysBkEhppsEJ2SCVVtAFQFwfAH/JAjewQShFhUJ3AybE2hUAmjEQlLFIUpnCKNgBq4gDIyYNq1CEfbtVGUMoCoC4MdlEGHjXBqBLaQkg7VJ4OZYsi09torobZXVZMXizpJqk2ybRL9Jrz3UOK0RIp/JqakJ333yr0rNp56Cxqdaizr3DeXqZY0g/ItFf0PlALgx2yqQled6IwCXWkQQWqosj0NhoRh3SbFCIPVpiMRlAIi246Iv17xbSE1MJkPhUJu7oGJ2SLIlOoRhxS+CHyQIGK1IhKk+rQSidhCHsBZKQ1joimz0S6Xcp1B5BQSYNVXYN1ysF6FIUTdoLgljgA8gWTPfYnEAoT0QwqWQDcEQYn/BqVUMENcQDU5MEOk1JBEfGglAWTdQz54ChD4RIqafAKN8UB0JcHy2O6nNqQHRERBGFwQmcCMlmcog12IynsiiLdEgdAvmDS8hg2HbuMTJhIh8gMo9QVBqooA8NYwdJggV2KwnGOBk1xAOALeTCNCVkAaGsYMrGKHPgpNaE7X4MIbooDQCMPPY7pUV0EtSwAZoTBqp4hiKkJxjwFKw0qKYr8x1QXByC7g3QzbWEa1XkWdKML6W0UowyyqYagpCZkow2AvjikcDvy4AUqkzNRpCMohcEJTk0UNgUxekLly9xun3yhbJ1RFVnblEWkhxSmRh7oTIZEQWY7lNaJELx2PwmDE26mJlKYeEoUqcbP17E1d/aRLu6TGW3hJSrtFL0fOvULrqUkeIXLgqBgIw2AerQh7+ROmhGHrO00Uxd2UEQlKOVEejlrnwlDUKIM+cg34ZNOxCGTVEepEnkA/BF90BEZGXESEQbVtSU4ysDIUtDSYBJKcQDU5cEOpw4/JRRuRCzclgWR3zvvq7yrMezqGlQKIgH3xAFQkwcgf4dNJRXUEQ63ZUE1LcH1DIwdBZGeUMXp6VEoDUGUqsjaXiFtIYvpFEfqGrwQhnw4f+YKxwtpsTpFqiITlbSFE6lUgdVLZ1sVUtcWFGFQglMTwjQ3N2P69OkoLy9HZWUlrr32Whw75iy5jY2NuPrqq1FdXY0+ffrg3HPPxf/9v/83a5sf/OAHOO+889C7d29UVlZaHqeoqKjHa926dVLtD5U0qKw9YfrJ0YQ4AOodr5eotrerd4RMGFTTEk5/JyqpiUQZTcRIB6eOQyQPTi0OAL08WGFCDKxQuZYP4308Fwa7KINKaoJ6qfcwMH36dPz1r3/Fpk2b8Lvf/Q7PPvss5syZ47jPjBkzsHv3bjz11FN4/fXX8eUvfxmXX345XnvttfQ2HR0d+I//+A984xvfcDzWAw88gAMHDqRfl156qVT7Q5WeUF20yqm2wWmGyFQHlDcNIZiqSCGaskjvS5y6oER5fQgJkaKI+oSlJiEXlRkiZaBMVWSiOuLCa1SFR1SuRERNp/BRKS3BUQZhdu3ahY0bN+KVV17BuHHjAAD33XcfpkyZgmXLlmHo0KGW+7344ov42c9+hvHjxwMAFi1ahB//+MfYvn07PvnJTwIAbr31VgDAgw8+6NiGyspKVFdXK19DcB5TM1B5SsvXKeg8SXaVFgnl0UVD56kna9XURb4XFSbOJXvdVGki5/21dqdD8cvZqSPIF22gjDiIPkXnohLedxOd9sncEwph4DoGOVpaWrJe8bje9NLbtm1DZWVlWhgAoK6uDpFIBC+99JLtfueddx4effRRNDc3I5FIYN26dWhvb8dFF10k3Ya5c+eiqqoK48ePx5o1a5BMyi3IGKpIA6C3RLZqxOGj/QUiCoLRifT2GhEI22P6LKWhOjETlTDoRBlMRCgi8SIkYnL/kPNN8qRaFAl81BHlK44EkDfqAGQ/VetEIOygjEyYEBUVcTIdYQgVbceBiOZ3ZeLE33pNTU3W24sXL8aSJUuUD9vY2IhBgwZlvVdcXIz+/fujsbHRdr9f/epXmDZtGgYMGIDi4mL07t0bTzzxBEaMGCF1/ttuuw2f//zn0bt3b/zhD3/A9ddfj2PHjuFb3/qW8DECKw2JsgQixyXnMRDq+PXFAaBJWfTYJ6dzpZIIL9CZwVE4YkMwPTR1LYM2Gktk64gDkH9UBSAnD4CeQNjh14iE17KgE2Uo1KGW+/btQ3l5efrnWMz6Xi9YsAB3332347F27dql3I6bb74Zhw8fxtNPP42qqio8+eSTuPzyy/Hcc89h1KhRUsdJ8clPfhKtra245557CkManNCJNgD64nDiGPnlQTbq0GN/A1EIk7ghCjLb+rmOwUS0AaARB8A56gCI1TrkktupUkmEl6hOyiQ67wKFMOSjUIUBAMrLy7OkwY4bbrgB11xzjeM2w4cPR3V1NT744IOs97u6utDc3GxbZ7Bnzx785Cc/wV/+8hd84hOfAACMHj0azz33HFauXIlVq1aJXYwFtbW1uP322xGPx22FKJdQSoMT4p2+vjicOI64PGSiG4XocTwXpIJ6/QeVSAFlOsJ3UYYUGtEGQF8cAPqUhRUmohCm0Zm5UXaCJiphcIoyFLIwyDBw4EAMHDgw73YTJkzA4cOHsX37dowdOxYAsGXLFiQSCdTW1lru09bWBgCIRLK/X6PRKBIJve/1nTt34qSTThIWBiDE0uAUbXBbHE4cSy4dkdv5qUYj0scT7NAtF9QytBiUFXoTL7kjDH5GdAErCnEA5OQhE90oRC5uSgX1ktQqszmKCAP5XAyMNmeeeSYmT56M2bNnY9WqVejs7ER9fT2uuOKK9MiJ/fv3Y+LEiXj44Ycxfvx4jBw5EiNGjMDXv/51LFu2DAMGDMCTTz6ZHrKZYu/evWhubsbevXvR3d2NnTt3AgBGjBiBvn374re//S2amprw6U9/GqWlpdi0aRPuvPNOfPe735W6htBKQz6oxAGgjTrk21d1f+HzuCgIWec1LAsAXXTAz2kNwH1xAMTqHTLJ7ShVoxEpZDpyK8GgFoF8qE77TC0LHGVwn7Vr16K+vh4TJ05EJBLBZZddhnvvvTf9+87OTuzevTsdYejVqxc2bNiABQsWYOrUqTh27BhGjBiBhx56CFOmTEnvd8stt+Chhx5K/5waivnMM8/goosuQq9evbBy5UrMmzcPyWQSI0aMwPLlyzF79myp9hclZcdbeEBLSwsqKipw6t13IFL60T82kULIfLUNoh2+6BoVMtEHis7fpECYRH8YpPj+sp18viiD6PGcZoMUGTact6YhT4pCdMnsfPM4yMgDkL/eIR+6AuFXVEUBMFe7QCINDkOBU5M7Jdrb8d78RThy5IhQjYAKqX6i7uTrUBzRG0nSlYjj6f2rjLY3qBRspCEFRcQh93gAXbGk6DF0j2MKihEMOseREQaRlASFMLgFRcQByO6IqOodnMjsXIMsEDqSAMgPofREGJiCo+ClAaAXh9QxU4gO0wToBEIG3XSJSXTO41X9gmvCIFAQSSUOKVIdkxvyAARDIHTlIBPVuRaoUhLS8GyQBUfopUF0+KUJccg8NiAXfcjEdATBLQEQxeSiUz3PRXtcP0QYcqEWB0Au+pDZEVIJhCwqwkEpA07oTMpkQhYo0hJMeAm9NMhgUhxSxwfk6h5OnM+6E/VjOkIWKmGhrltQaoMXwiA4/NKEOKRQiT7kolsHkQ+3BEAG3RkcRYXByPTQLAwFS0FIg8xkT6bFQeYc+dvg73qGXKgjGiqjGJSWtw5ohCEXk+IAyMlDLlYdqGmRcBOKKZ4paxbsEIoy+FwYkm3HkYyoz2MCAMmEuUXegk5BSIMsbokDQCMPJ9ri3CGHIcWhO9TRlDB4jsRkT6bFAZAfrmkHVVrDTajXf1Cda6FQhYExT8FIg+zU0jLikMLNlIUsfqtbkMHvsiAbZVBZpZUSGXEA8g/JtEIn6mCFSGdsWizcWhBKd1ImWWHgGgZGhoKRBhXkZ3088V8decjFtEz4CcqnetWaBdk2+EYYJKeWFhUHgEYeADqBsCPIqzxSzN5oTBYAJWFIzdHAhIuCkgaVhaxU6g90ow+5588lDCJhIuyvU9yo0h7fCIMiMuIA6MkDYN0xmhYJP0I1vbNqgaP0HAwsDEwGBSUNqugULupEH5zaY4XfZMKNegDdURCqbfRl4aPCQlay4gBkd1aqApEizCJhYu0HnZEQbk3YxMIQbgItDYmyhNBU0pmkvuxVIg6AvjykoJSIFIEo2iOAarikm8LgtyhDJirikIJSIFLYdbZ+lAk3FoWiGDKpJAwcYWAsCLQ0AGriAHgnDynsOj4TMhFUyGdqJDiemxGGSLwo//oTuaS+6BUiDoD4ehVWmBCITApp1UbKuRXcEgamMAi8NADq4gDoy0MupmQCCK9QmF6GOmiykImSOACeygPg3OmZEIqgYmLiJe00BAsD40AopAHQEwdAXR56HCeng6KsMwiaUJiWAScoUzVe1y8oiwOgLQ+56MoEULhCYWRmxn9CVq/AwsDkITTSAOiLA5DdQegKBCC3cJUOXnbQfoK6rsNrYUihJQ6Asjzkkts5UUhEJkEUCpMykA/S4kYWBkaAUEkDQCMOKaiiD+njGYxCFDKmCkD9IgwptMUByO4YNAUCyO60qAUiFy87Zz9hZBQECwMjSOikAaAVB4BeHtLHlezsCkkyvB4J4jdhSEEiDimIog8pTEchChmjwyVDJgzJ1jYkizr1jpHU2z/MhFIaAHpxAMzJg/D5HTpSPwuF1wKQi1+FQBRScQDI5SGFbEdXaJLh1rwJjoRMGBjzhFYaADPiAGTLQ74OyC3B8FvH7DeCLgq5kIsDYEweRHHqRP0uFL4QgExYBhhDhFoaAHPiAIh1RLnbeBWlKFTCJguZGBEHQL7DcUEyfNcp+xWWBcYwoZcGwKw4yMISoYYfO38/zPJoTBxkIC6uZCRhUWBcpCCkAfCXOGRC1RkGWT78KAT58IMwpPCFOKTI7cBYIsTxaefPU0MzmRSMNAD+FQcKgtjxBhU/CUMKX4lDJlQdYZDlw6cyIAILA5NLQUkDoP+FH1bpYMTwozCk8K04UBDgjjeosDAwVij1gCtXrsSwYcNQWlqK2tpavPzyy47bP/bYYxg5ciRKS0sxatQobNiwQamxfiBRlvB1x8E4k/r8VF9+JxIv0noxDMDCwNgjHWl49NFH0dDQgFWrVqG2thYrVqzApEmTsHv3bgwaNKjH9i+++CKuvPJKLF26FF/4whfwyCOP4NJLL8WOHTtw9tlnk1yEF1B2IBy9cCYInXVYSHUWoY1YFADc4TMmKUomk1LfDrW1tfjUpz6Fn/zkJwCARCKBmpoafPOb38SCBQt6bD9t2jS0trbid7/7Xfq9T3/60xgzZgxWrVoldM6WlhZUVFTg1LvvQKSUJyRgmCDBAuJMIXTyifZ2vDd/EY4cOYLy8nIj50j1E58v+Q8UF/XSOlZXshNbOh4z2t6gIhVp6OjowPbt27Fw4cL0e5FIBHV1ddi2bZvlPtu2bUNDQ0PWe5MmTcKTTz5pe554PI54/KPhAEeOHAFw4g+PYZiAwf9sHSmEOFrqu1vyGVWJLnQCmqfpAk8jbYeUNBw6dAjd3d0YPHhw1vuDBw/GG2+8YblPY2Oj5faNjY2251m6dCluvfXWHu/vW3yHTHMZhmEYH/Hhhx+ioqLCyLFLSkpQXV2NZxufJDledXU1Skp4kbRcfDl6YuHChVnRicOHD+PUU0/F3r17jf3BhYGWlhbU1NRg3759HFJzgO9TfvgeicH3SYwjR47glFNOQf/+/Y2do7S0FO+88w46OmiWUC8pKUEpp8N7ICUNVVVViEajaGpqynq/qakJ1dXVlvtUV1dLbQ8AsVgMsVjPiQcqKir4H6YA5eXlfJ8E4PuUH75HYvB9EiMSMVv0XVpayh29YaQ+wZKSEowdOxabN29Ov5dIJLB582ZMmDDBcp8JEyZkbQ8AmzZtst2eYRiGYRh/Ip2eaGhowMyZMzFu3DiMHz8eK1asQGtrK2bNmgUAmDFjBk4++WQsXboUAPDtb38bF154IX70ox/hkksuwbp16/Dqq6/i5z//Oe2VMAzDMAxjFGlpmDZtGg4ePIhbbrkFjY2NGDNmDDZu3Jgudty7d29WCOq8887DI488gkWLFuHGG2/Ev/zLv+DJJ5+UmqMhFoth8eLFlikL5iP4PonB9yk/fI/E4PskBt+n8CA9TwPDMAzDMIUJT0XIMAzDMIwQLA0MwzAMwwjB0sAwDMMwjBAsDQzDMAzDCMHSwDAMwzCMEL6RhpUrV2LYsGEoLS1FbW0tXn75ZcftH3vsMYwcORKlpaUYNWoUNmzY4FJLvUXmPq1evRoXXHABTjrpJJx00kmoq6vLe1/DgOzfUop169ahqKgIl156qdkG+gTZ+3T48GHMnTsXQ4YMQSwWw+mnn14Q/+5k79OKFStwxhlnoKysDDU1NZg3bx7aQ77Y3rPPPoupU6di6NChKCoqclyQMMXWrVtx7rnnIhaLYcSIEXjwwQeNt5MhIOkD1q1blywpKUmuWbMm+de//jU5e/bsZGVlZbKpqcly+xdeeCEZjUaTP/zhD5N/+9vfkosWLUr26tUr+frrr7vccneRvU9XXXVVcuXKlcnXXnstuWvXruQ111yTrKioSP797393ueXuIXuPUrzzzjvJk08+OXnBBRckv/SlL7nTWA+RvU/xeDw5bty45JQpU5LPP/988p133klu3bo1uXPnTpdb7i6y92nt2rXJWCyWXLt2bfKdd95J/v73v08OGTIkOW/ePJdb7i4bNmxI3nTTTcnHH388CSD5xBNPOG7/9ttvJ3v37p1saGhI/u1vf0ved999yWg0mty4caM7DWaU8YU0jB8/Pjl37tz0z93d3cmhQ4cmly5darn95Zdfnrzkkkuy3qutrU1+/etfN9pOr5G9T7l0dXUl+/Xrl3zooYdMNdFzVO5RV1dX8rzzzkv+4he/SM6cObMgpEH2Pv3sZz9LDh8+PNnR0eFWE32B7H2aO3du8vOf/3zWew0NDcnzzz/faDv9hIg0fP/7309+4hOfyHpv2rRpyUmTJhlsGUOB5+mJjo4ObN++HXV1den3IpEI6urqsG3bNst9tm3blrU9AEyaNMl2+zCgcp9yaWtrQ2dnp9GV5rxE9R7ddtttGDRoEK699lo3muk5KvfpqaeewoQJEzB37lwMHjwYZ599Nu688050d3e71WzXUblP5513HrZv355OYbz99tvYsGEDpkyZ4kqbg0IhfoeHBc+Xxj506BC6u7vT01CnGDx4MN544w3LfRobGy23b2xsNNZOr1G5T7nMnz8fQ4cO7fGPNSyo3KPnn38e999/P3bu3OlCC/2Byn16++23sWXLFkyfPh0bNmzAW2+9heuvvx6dnZ1YvHixG812HZX7dNVVV+HQoUP4zGc+g2Qyia6uLlx33XW48cYb3WhyYLD7Dm9pacHx48dRVlbmUcuYfHgeaWDc4a677sK6devwxBNP8NKx/+To0aO4+uqrsXr1alRVVXndHF+TSCQwaNAg/PznP8fYsWMxbdo03HTTTVi1apXXTfMVW7duxZ133omf/vSn2LFjBx5//HGsX78et99+u9dNYxgSPI80VFVVIRqNoqmpKev9pqYmVFdXW+5TXV0ttX0YULlPKZYtW4a77roLTz/9NM455xyTzfQU2Xu0Z88evPvuu5g6dWr6vUQiAQAoLi7G7t27cdppp5lttAeo/C0NGTIEvXr1QjQaTb935plnorGxER0dHSgpKTHaZi9QuU8333wzrr76anzta18DAIwaNQqtra2YM2cObrrppqzF/AoZu+/w8vJyjjL4HM//gktKSjB27Fhs3rw5/V4ikcDmzZsxYcIEy30mTJiQtT0AbNq0yXb7MKBynwDghz/8IW6//XZs3LgR48aNc6OpniF7j0aOHInXX38dO3fuTL+++MUv4nOf+xx27tyJmpoaN5vvGip/S+effz7eeuuttFQBwJtvvokhQ4aEUhgAtfvU1tbWQwxSopXktQHTFOJ3eGjwuhIzmTwxrCkWiyUffPDB5N/+9rfknDlzkpWVlcnGxsZkMplMXn311ckFCxakt3/hhReSxcXFyWXLliV37dqVXLx4ccEMuZS5T3fddVeypKQk+etf/zp54MCB9Ovo0aNeXYJxZO9RLoUyekL2Pu3duzfZr1+/ZH19fXL37t3J3/3ud8lBgwYl77jjDq8uwRVk79PixYuT/fr1S/7yl79Mvv3228k//OEPydNOOy15+eWXe3UJrnD06NHka6+9lnzttdeSAJLLly9Pvvbaa8n33nsvmUwmkwsWLEheffXV6e1TQy6/973vJXft2pVcuXIlD7kMCL6QhmQymbzvvvuSp5xySrKkpCQ5fvz45B//+Mf07y688MLkzJkzs7b/1a9+lTz99NOTJSUlyU984hPJ9evXu9xib5C5T6eeemoSQI/X4sWL3W+4i8j+LWVSKNKQTMrfpxdffDFZW1ubjMViyeHDhyd/8IMfJLu6ulxutfvI3KfOzs7kkiVLkqeddlqytLQ0WVNTk7z++uuT//jHP9xvuIs888wzlt81qXszc+bM5IUXXthjnzFjxiRLSkqSw4cPTz7wwAOut5uRpyiZ5JgZwzAMwzD58bymgWEYhmGYYMDSwDAMwzCMECwNDMMwDMMIwdLAMAzDMIwQLA0MwzAMwwjB0sAwDMMwjBAsDQzDMAzDCMHSwDAMwzCMECwNDMMwDMMIwdLAMAzDMIwQLA0MwzAMwwjx/wMngytSJb4tNwAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "def forward_J(m):\n", + " _, _, J = forward(m)\n", + " return J\n", + "\n", + "\n", + "M_solver = LinearSolver(assemble(inner(TrialFunction(space), TestFunction(space)) * dx),\n", + " solver_parameters={\"ksp_type\": \"cg\",\n", + " \"pc_type\": \"sor\",\n", + " \"ksp_atol\": 1.0e-32,\n", + " \"ksp_rtol\": 1.0e-12})\n", + "\n", + "\n", + "def M_inv_action(x):\n", + " y = Function(space)\n", + " M_solver.solve(y, x.copy(deepcopy=True))\n", + " return y\n", + "\n", + "\n", + "def post_callback(tao):\n", + " print(f\"{tao.getIterationNumber()=}\")\n", + "\n", + "\n", + "m = minimize_tao(forward_J, m_0, gatol=1.0e-10, grtol=0.0,\n", + " M_inv_action=M_inv_action, post_callback=post_callback)\n", + "m.rename(name=\"m\")\n", + "m_tilde, u, J = forward(m)\n", + "\n", + "\n", + "def plot_output(u, title):\n", + " r = (u.dat.data_ro.min(), u.dat.data_ro.max())\n", + " eps = (r[1] - r[0]) * 1.0e-12\n", + " p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))\n", + " plt.gca().set_title(title)\n", + " plt.colorbar(p)\n", + " plt.gca().set_aspect(1.0)\n", + "\n", + "\n", + "plot_output(u, title=\"u\")\n", + "plot_output(m, title=\"m\")" + ] + }, + { + "cell_type": "markdown", + "id": "d2bf05e9-0e6e-4a4c-8b93-9e6019f689ee", + "metadata": {}, + "source": [ + "We now test the inverse procedure by checking that it converges to the expected result, considering meshes with decreasing element size. We compute the $L^2$ error in each case, and estimate the order of convergence by a power law fit between the error norms computed using subsequent pairs of meshes." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "ebfbaff7-a877-4e9b-bac7-b7bdad621e5c", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:47:01.743057Z", + "iopub.status.busy": "2023-12-20T16:47:01.742842Z", + "iopub.status.idle": "2023-12-20T16:47:27.159254Z", + "shell.execute_reply": "2023-12-20T16:47:27.158521Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tao.getIterationNumber()=54\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "N=20 m_error_norm=0.006304152630049985\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tao.getIterationNumber()=28\n", + "N=40 m_error_norm=0.0015825703318738235\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tao.getIterationNumber()=23\n", + "N=80 m_error_norm=0.00039604503914227284\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tao.getIterationNumber()=21\n", + "N=160 m_error_norm=9.903640999431026e-05\n", + "orders=array([1.99403285, 1.99853321, 1.99963358])\n" + ] + } + ], + "source": [ + "Ns = np.array([20 * (2 ** p) for p in range(4)], dtype=int)\n", + "error_norms = []\n", + "for N in Ns:\n", + " mesh = PeriodicSquareMesh(N, N, 1.0)\n", + " X = SpatialCoordinate(mesh)\n", + " space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "\n", + " m_0 = Function(space, name=\"m_0\").interpolate(Constant(-1.0))\n", + "\n", + " M_solver = LinearSolver(assemble(inner(TrialFunction(space), TestFunction(space)) * dx),\n", + " solver_parameters={\"ksp_type\": \"cg\",\n", + " \"pc_type\": \"sor\",\n", + " \"ksp_atol\": 1.0e-32,\n", + " \"ksp_rtol\": 1.0e-12})\n", + "\n", + " def M_inv_action(x):\n", + " y = Function(space)\n", + " M_solver.solve(y, x.copy(deepcopy=True))\n", + " return y\n", + "\n", + " m = minimize_tao(forward_J, m_0, gatol=1.0e-10, grtol=0.0,\n", + " M_inv_action=M_inv_action, post_callback=post_callback)\n", + " m_tilde, u, J = forward(m)\n", + "\n", + " m_error_norm = sqrt(abs(assemble(inner(m - m_tilde, m - m_tilde) * dx)))\n", + " print(f\"{N=} {m_error_norm=}\")\n", + " error_norms.append(m_error_norm)\n", + "error_norms = np.array(error_norms, dtype=float)\n", + "\n", + "orders = -np.log(error_norms[1:] / error_norms[:-1]) / np.log(Ns[1:] / Ns[:-1])\n", + "print(f\"{orders=}\")\n", + "\n", + "assert (orders > 1.99).all()" + ] + }, + { + "cell_type": "markdown", + "id": "3a913c18-c76b-4119-aab5-db6176ddac04", + "metadata": {}, + "source": [ + "We find that we have close to second order convergence." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/6_custom_operations.html b/examples/6_custom_operations.html new file mode 100644 index 0000000..79a1d1c --- /dev/null +++ b/examples/6_custom_operations.html @@ -0,0 +1,716 @@ + + + + + + + Defining custom operations — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Defining custom operations

+

This notebook describes the tlm_adjoint ‘escape hatch’ mechanism, which allows custom operations to be defined, recorded by the internal manager, and used in higher order derivative calculations.

+

The approach used by tlm_adjoint to define custom operations is described in

+
    +

  • James R. Maddison, Daniel N. Goldberg, and Benjamin D. Goddard, ‘Automated calculation of higher order partial differential equation constrained derivative information’, SIAM Journal on Scientific Computing, 41(5), pp. C417–C445, 2019, doi: 10.1137/18M1209465

    • +
+

We assume real spaces and a real build of Firedrake throughout.

+
+

Forward problem

+

We consider the Poisson equation in the unit square domain

+
+\[\nabla^2 u = m \quad \text{on} ~ \Omega = \left( 0, 1 \right)^2,\]
+

with

+
+\[\int_\Omega m = 0,\]
+

subject to boundary conditions

+
+\[\nabla u \cdot \hat{n} = 0,\]
+

where \(\hat{n}\) is an outward unit normal on the boundary \(\partial \Omega\) of the domain \(\Omega\).

+

We consider a continuous Galerkin discretization, now seeking \(u \in V\) such that

+
+\[\forall \zeta \in V \qquad \int_\Omega \nabla \zeta \cdot \nabla u = -\int_\Omega \zeta m,\]
+

where \(V\) is a real \(P_1\) continuous finite element space defining functions on the domain \(\Omega = \left( 0, 1 \right)^2\). \(m \in V\) is now defined via

+
+\[m = \mathcal{I} \left[ \cos \left( \pi x \right) \cos \left( \pi y \right) \right] - k,\]
+

where \(\mathcal{I}\) maps to an element of \(V\) through interpolation at mesh vertices, and where \(k\) is defined so that \(\int_\Omega m = 0\).

+

As stated the discrete problem does not have a unique solution – given any solution \(u_0\) to the problem we can define a new solution \(u_1 = u_0 + c_1\) for any scalar \(c_1\). We need to identify a solution by supplying an appropriate constraint. We will use the constraint

+
+\[\sum_i \tilde{u}_i = 0,\]
+

where the \(\tilde{u}_i\) are the degrees of freedom associated with \(u \in V\).

+

An implementation using Firedrake, solving for \(u\) and then computing \(\frac{1}{2} \int_\Omega u^2\), takes the form

+
+
[1]:
+
+
+
%matplotlib inline
+
+from firedrake import *
+
+import matplotlib.pyplot as plt
+import numpy as np
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test, trial = TestFunction(space), TrialFunction(space)
+
+
+def forward(m):
+    k = Constant(assemble(m * dx) / assemble(Constant(1.0) * dx(mesh)))
+    m_tilde = Function(m.function_space(), name="m_tilde").interpolate(m - k)
+
+    u = Function(space, name="u")
+    nullspace = VectorSpaceBasis(comm=u.comm, constant=True)
+
+    solve(inner(grad(trial), grad(test)) * dx == -inner(m_tilde, test) * dx,
+          u, nullspace=nullspace, transpose_nullspace=nullspace,
+          solver_parameters={"ksp_type": "cg", "pc_type": "hypre",
+                             "pc_hypre_type": "boomeramg",
+                             "ksp_rtol": 1.0e-10, "ksp_atol": 1.0e-16})
+
+    J = assemble(0.5 * inner(u, u) * dx)
+    return u, J
+
+
+m = Function(space, name="m").interpolate(cos(pi * X[0]) * cos(pi * X[1]))
+
+u, J = forward(m)
+
+
+def plot_output(u, title):
+    r = (u.dat.data_ro.min(), u.dat.data_ro.max())
+    eps = (r[1] - r[0]) * 1.0e-12
+    p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))
+    plt.gca().set_title(title)
+    plt.colorbar(p)
+    plt.gca().set_aspect(1.0)
+
+
+plot_output(u, title="u")
+
+
+
+
+
+
+
+../_images/examples_6_custom_operations_1_0.png +
+
+
+
+

Adding tlm_adjoint

+

Next we add the use of tlm_adjoint, which requires some minor modification of the forward code.

+

compute_gradient is used to compute the derivative of \(\frac{1}{2} \int_\Omega u^2\) with respect to \(m\), subject to the contraint that the forward problem is solved. The derivative is visualized using a Riesz map defined by the \(L^2\) inner product. A Taylor remainder convergence test is used to verify the derivative.

+

Note that, given an arbitrary \(m \in V\), we define

+
+\[\tilde{m} = m - \frac{\int_\Omega m}{\int_\Omega 1},\]
+

so that \(\int_\Omega \tilde{m} = 0\), and use \(\tilde{m}\) in place of \(m\) when solving the discrete Poisson problem.

+
+
[2]:
+
+
+
%matplotlib inline
+
+from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import matplotlib.pyplot as plt
+import numpy as np
+
+np.random.seed(54151610)
+reset_manager()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test, trial = TestFunction(space), TrialFunction(space)
+
+
+def forward(m):
+    v = Constant(name="v")
+    Assembly(v, Constant(1.0) * dx(mesh)).solve()
+    k = Constant(name="k")
+    Assembly(k, (m / v) * dx).solve()
+    m_tilde = Function(m.function_space(), name="m_tilde").interpolate(m - k)
+
+    u = Function(space, name="u")
+    nullspace = VectorSpaceBasis(comm=u.comm, constant=True)
+
+    solve(inner(grad(trial), grad(test)) * dx == -inner(m_tilde, test) * dx,
+          u, nullspace=nullspace, transpose_nullspace=nullspace,
+          solver_parameters={"ksp_type": "cg", "pc_type": "hypre",
+                             "pc_hypre_type": "boomeramg",
+                             "ksp_rtol": 1.0e-10, "ksp_atol": 1.0e-16})
+
+    J = Functional(name="J")
+    J.assign(0.5 * inner(u, u) * dx)
+    return u, J
+
+
+m = Function(space, name="m").interpolate(cos(pi * X[0]) * cos(pi * X[1]))
+
+start_manager()
+u, J = forward(m)
+stop_manager()
+
+dJ = compute_gradient(J, m)
+
+
+def plot_output(u, title):
+    r = (u.dat.data_ro.min(), u.dat.data_ro.max())
+    eps = (r[1] - r[0]) * 1.0e-12
+    p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))
+    plt.gca().set_title(title)
+    plt.colorbar(p)
+    plt.gca().set_aspect(1.0)
+
+
+plot_output(u, title="u")
+plot_output(dJ.riesz_representation("L2"), title=r"$g^\sharp$")
+
+
+def forward_J(m):
+    _, J = forward(m)
+    return J
+
+
+min_order = taylor_test(forward_J, m, J_val=J.value, dJ=dJ)
+assert min_order > 1.99
+
+
+
+
+
+
+
+
+Error norms, no adjoint   = [6.16667546e-08 3.06309860e-08 1.52648952e-08 7.61979814e-09
+ 3.80673670e-09]
+Orders,      no adjoint   = [1.00950111 1.00477412 1.002393   1.00119799]
+Error norms, with adjoint = [8.09565105e-10 2.02391273e-10 5.05978163e-11 1.26494529e-11
+ 3.16236247e-12]
+Orders,      with adjoint = [2.00000003 2.00000005 2.00000013 2.00000034]
+
+
+
+
+
+
+../_images/examples_6_custom_operations_3_1.png +
+
+
+
+
+
+../_images/examples_6_custom_operations_3_2.png +
+
+
+
+

Defining a custom Equation

+

Since the \(u\) we have computed should satisfy

+
+\[\sum_i \tilde{u}_i = 0,\]
+

we should have that the derivative of this quantity with respect to \(m\), subject to the forward problem being solved, is zero. We will compute this derivative by defining a custom operation which sums the degrees of freedom of a finite element discretized function. We can do this by inheriting from the Equation class and implementing appropriate methods.

+
+
[3]:
+
+
+
from tlm_adjoint import Equation
+
+
+class Sum(Equation):
+    def __init__(self, x, y):
+        super().__init__(x, deps=(x, y), nl_deps=(), ic=False, adj_ic=False,
+                         adj_type="conjugate_dual")
+
+    def forward_solve(self, x, deps=None):
+        if deps is None:
+            deps = self.dependencies()
+        _, y = deps
+        with y.dat.vec_ro as y_v:
+            y_sum = y_v.sum()
+        x.assign(y_sum)
+
+    def adjoint_jacobian_solve(self, adj_x, nl_deps, b):
+       return b
+
+    def adjoint_derivative_action(self, nl_deps, dep_index, adj_x):
+        if dep_index == 0:
+            return adj_x
+        elif dep_index == 1:
+            _, y = self.dependencies()
+            b = Cofunction(var_space(y).dual())
+            adj_x = float(adj_x)
+            b.dat.data[:] = -adj_x
+            return b
+        else:
+            raise ValueError("dep_index out of range")
+
+    def tangent_linear(self, M, dM, tlm_map):
+        tau_x, tau_y = (tlm_map[dep] for dep in self.dependencies())
+        if tau_y is None:
+            return ZeroAssignment(tau_x)
+        else:
+            return Sum(tau_x, tau_y)
+
+
+
+

We’ll investgate how this is put together shortly.

+

First, we check that the Sum class works. In the following we check that we can compute the sum of the degrees of freedom of a function, and then use Taylor remainder convergence tests to verify a first order tangent-linear calculation, a first order adjoint calculation, and a reverse-over-forward adjoint calculation of a Hessian action.

+
+
[4]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+np.random.seed(13561700)
+reset_manager()
+
+
+def forward(m):
+    m_sum = Float(name="m_sum")
+    Sum(m_sum, m).solve()
+
+    J = m_sum ** 3
+
+    return m_sum, J
+
+
+def forward_J(m):
+    _, J = forward(m)
+    return J
+
+
+mesh = UnitIntervalMesh(10)
+x, = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Discontinuous Lagrange", 0)
+
+m = Function(space, name="m")
+m.interpolate(exp(-x) * sin(pi * x))
+with m.dat.vec_ro as m_v:
+    m_sum_ref = m_v.sum()
+
+start_manager()
+m_sum, J = forward(m)
+stop_manager()
+
+# Verify the forward calculation
+
+print(f"{float(m_sum)=}")
+print(f"{m_sum_ref=}")
+assert abs(float(m_sum) - m_sum_ref) == 0.0
+
+# Verify tangent-linear and adjoint calculations
+
+min_order = taylor_test_tlm(forward_J, m, tlm_order=1)
+assert min_order > 2.00
+
+min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=1)
+assert min_order > 2.00
+
+min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=2)
+assert min_order > 1.99
+
+
+
+
+
+
+
+
+float(m_sum)=3.97146153200585
+m_sum_ref=3.97146153200585
+Error norms, no tangent-linear   = [2.32844986 1.15714317 0.57680653 0.28796267 0.14387127]
+Orders,      no tangent-linear   = [1.00880244 1.00440797 1.00220568 1.00110326]
+Error norms, with tangent-linear = [0.02826961 0.00705304 0.00176147 0.00044014 0.00011001]
+Orders,      with tangent-linear = [2.00293418 2.00146933 2.00073523 2.00036775]
+Error norms, no adjoint   = [2.68976716 1.33545989 0.66538227 0.33210525 0.16590628]
+Orders,      no adjoint   = [1.01014465 1.0050813  1.0025429  1.00127201]
+Error norms, with adjoint = [0.03760673 0.00937967 0.00234217 0.0005852  0.00014626]
+Orders,      with adjoint = [2.00338159 2.00169377 2.00084763 2.000424  ]
+Error norms, no adjoint   = [5.89746632 2.94164753 1.46905236 0.73408333 0.36693095]
+Orders,      no adjoint   = [1.00347088 1.00173858 1.00087008 1.00043523]
+Error norms, with adjoint = [0.02834252 0.00708563 0.00177141 0.00044285 0.00011071]
+Orders,      with adjoint = [2. 2. 2. 2.]
+
+
+

We can now use Sum with the Poisson solver. We find, as expected, that the derivative of the sum of the degrees of freedom for \(u\) with respect to \(m\), subject to the forward problem being solved, is zero.

+
+
[5]:
+
+
+
%matplotlib inline
+
+from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import matplotlib.pyplot as plt
+import numpy as np
+
+reset_manager()
+
+mesh = UnitSquareMesh(10, 10)
+X = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Lagrange", 1)
+test, trial = TestFunction(space), TrialFunction(space)
+
+
+def forward(m):
+    v = Constant(name="v")
+    Assembly(v, Constant(1.0) * dx(mesh)).solve()
+    k = Constant(name="k")
+    Assembly(k, (m / v) * dx).solve()
+    m_tilde = Function(m.function_space(), name="m_tilde").interpolate(m - k)
+
+    u = Function(space, name="u")
+    nullspace = VectorSpaceBasis(comm=u.comm, constant=True)
+
+    solve(inner(grad(trial), grad(test)) * dx == -inner(m_tilde, test) * dx,
+          u, nullspace=nullspace, transpose_nullspace=nullspace,
+          solver_parameters={"ksp_type": "cg", "pc_type": "hypre",
+                             "pc_hypre_type": "boomeramg",
+                             "ksp_rtol": 1.0e-10, "ksp_atol": 1.0e-16})
+
+    J = Functional(name="J")
+    J.assign(0.5 * inner(u, u) * dx)
+
+    u_sum = Constant(name="u_sum")
+    Sum(u_sum, u).solve()
+
+    K = Functional(name="K")
+    K.assign(u_sum)
+
+    return u, J, K
+
+
+m = Function(space, name="m").interpolate(cos(pi * X[0]) * cos(pi * X[1]))
+
+start_manager()
+u, J, K = forward(m)
+stop_manager()
+
+dJ, dK = compute_gradient((J, K), m)
+
+dK_norm = abs(dK.dat.data_ro).max()
+print(f"{dK_norm=}")
+
+assert dK_norm == 0.0
+
+
+
+
+
+
+
+
+dK_norm=0.0
+
+
+
+
+

Equation methods

+

We now return to the definition of the Sum class, and consider each method in turn.

+
+

__init__

+
def __init__(self, x, y):
+    super().__init__(x, deps=(x, y), nl_deps=(), ic=False, adj_ic=False,
+                     adj_type="conjugate_dual")
+
+
+

The arguments passed to the base class constructor are:

+
    +

  • The output of the forward operation, x.

    • +

  • deps: Defines all inputs and outputs of the forward operation.

    • +

  • nl_deps: Defines elements of deps on which the associated adjoint calculations can depend.

    • +

  • ic: Whether the forward operation accepts a non-zero ‘initial guess’.

    • +

  • adj_ic: Whether an adjoint solve accepts a non-zero ‘initial guess’.

    • +

  • adj_type: Either "primal" or "conjugate_dual" defining the space for an associated adjoint variable.

    • +
+

For this operation there is one output x and one input y. The operation is defined by the forward residual function

+
+\[F \left( x, y \right) = x - \sum_i \tilde{u}_i.\]
+

Given a value for \(y\) the output of the operation is defined to be the \(x\) for which \(F \left( x, y \right) = 0\). \(F\) depends linearly on both \(x\) and \(y\), and so associated adjoint calculations are independent of both \(x\) and \(y\). Hence we set deps=(x, y) and nl_deps=().

+

ic and adj_ic are used to indicate whether non-zero initial guesses should be supplied to forward and adjoint iterative solvers respectively. Here we do not use iterative solvers, and so no non-zero initial guesses are needed. Hence we set ic=False and adj_ic=False.

+

adj_type indicates whether an adjoint variable associated with the operation is:

+
    +

  • In the same space as x. This is the case where \(F\) maps to an element in the antidual space associated with \(x\), and is indicated with adj_x_type="primal". A typical example of this case is an operation defined via the solution of a finite element variational problem.

    • +

  • In the antidual space associated with the space for x. This is the case where \(F\) maps to an element in same space as \(x\), and is indiciated with adj_x_type="conjugate_dual".

    • +
+
+
+

forward_solve

+
def forward_solve(self, x, deps=None):
+    if deps is None:
+        deps = self.dependencies()
+    _, y = deps
+    with y.dat.vec_ro as y_v:
+        y_sum = y_v.sum()
+    x.assign(y_sum)
+
+
+

This method computes the output of the forward operation, storing the result in x. deps is used in rerunning the forward calculation when making use of a checkpointing schedule. If supplied, deps contains values associated with the dependencies as defined in the constructor (defined by the deps argument passed to the base class constructor). Otherwise the values contained in self.dependencies() are used.

+
+
+

adjoint_jacobian_solve

+
def adjoint_jacobian_solve(self, adj_x, nl_deps, b):
+   return b
+
+
+

Solves a linear problem

+
+\[\frac{\partial F}{\partial x}^T \lambda = b,\]
+

for the adjoint solution \(\lambda\), returning the result. The right-hand-side \(b\) is defined by the argument b. In this example \(\partial F / \partial x\) is simply the identity, and so \(\lambda = b\).

+

adj_x can contain an initial guess for an iterative solver used to compute \(\lambda\). nl_deps contains values associated with the forward dependencies of the adjoint, as defined in the constructor (defined by the nl_deps argument passed to the base class constructor).

+
+
+

adjoint_derivative_action

+
def adjoint_derivative_action(self, nl_deps, dep_index, adj_x):
+    if dep_index == 0:
+        return adj_x
+    elif dep_index == 1:
+        _, y = self.dependencies()
+        b = Cofunction(var_space(y).dual())
+        adj_x = float(adj_x)
+        b.dat.data[:] = -adj_x
+        return b
+    else:
+        raise ValueError("dep_index out of range")
+
+
+

Computes

+
+\[\frac{\partial F}{\partial v}^T \lambda,\]
+

where \(v\) is the dep_indexth dependency (for the dependencies defined by the deps argument passed to the base class constructor). Again nl_deps contains values associated with the forward dependencies of the adjoint, as defined in the constructor.

+
+
+

tangent_linear

+
def tangent_linear(self, M, dM, tlm_map):
+    tau_x, tau_y = (tlm_map[dep] for dep in self.dependencies())
+    if tau_y is None:
+        return ZeroAssignment(tau_x)
+    else:
+        return Sum(tau_x, tau_y)
+
+
+

Constructs a tangent-linear operation, for a tangent-linear computing directional derivatives with respect to the control defined by M with direction defined by dM. tlm_map is used to access tangent-linear variables.

+

The new operation is defined by the residual function

+
+\[F_\tau \left( \tau_x, \tau_y \right) = \frac{\partial F}{\partial x} \tau_x + \frac{\partial F}{\partial y} \tau_y,\]
+

where \(\tau_x\) and \(\tau_y\) are the tangent-linear variables associated with \(x\) and \(y\) respectively. Given a value for \(\tau_y\) the output of the operation is defined to be the \(\tau_x\) for which \(F_\tau \left( \tau_x, \tau_y \right) = 0\).

+

Note that this method returns an Equation instance – here either a ZeroAssignment, for the case where the tangent-linear operation sets a tangent-linear variable equal to zero, or another Sum. This new operation can then be recorded by the internal manager, so that reverse-over-forward algorithmic differentiation can be applied.

+
+
+

Reference dropping

+

An important method, not defined in this example, is the drop_references method. This method is used to drop references to forward data. For example if an Equation subclass defines a Function attribute _function, then the drop_references method might look like

+
def drop_references(self):
+    super().drop_references()
+    self._function = var_replacement(self._function)
+
+
+

Here var_replacement returns a ‘symbolic only’ version of self._function, which can for example be used in UFL expressions, but which has no associated value.

+
+
+
+

Defining custom operations using JAX

+

In some cases it may be more convenient to directly implement an operation using lower level code. tlm_adjoint integrates with JAX to allow this. For example a Firedrake Function can be converted to a JAX array using

+
v = to_jax(x)
+
+
+

and the result converted back to a Firedrake Function using

+
x = to_firedrake(v, space)
+
+
+

Here we use JAX to compute the sum of the degrees of freedom for a Firedrake function (assuming a serial calculation).

+
+
[6]:
+
+
+
from firedrake import *
+from tlm_adjoint.firedrake import *
+
+import numpy as np
+
+np.random.seed(81976463)
+reset_manager()
+
+
+def forward(m):
+    m_sum = new_jax_float(name="m_sum")
+    call_jax(m_sum, to_jax(m), lambda v: v.sum())
+
+    J = m_sum ** 3
+
+    return m_sum, J
+
+
+def forward_J(m):
+    _, J = forward(m)
+    return J
+
+
+mesh = UnitIntervalMesh(10)
+x, = SpatialCoordinate(mesh)
+space = FunctionSpace(mesh, "Discontinuous Lagrange", 0)
+
+m = Function(space, name="m")
+m.interpolate(exp(-x) * sin(pi * x))
+with m.dat.vec_ro as m_v:
+    m_sum_ref = m_v.sum()
+
+start_manager()
+m_sum, J = forward(m)
+stop_manager()
+
+# Verify the forward calculation
+
+print(f"{float(m_sum)=}")
+print(f"{m_sum_ref=}")
+assert abs(float(m_sum) - m_sum_ref) < 1.0e-15
+
+# Verify tangent-linear and adjoint calculations
+
+min_order = taylor_test_tlm(forward_J, m, tlm_order=1)
+assert min_order > 2.00
+
+min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=1)
+assert min_order > 2.00
+
+min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=2)
+assert min_order > 1.99
+
+
+
+
+
+
+
+
+float(m_sum)=3.9714615320058506
+m_sum_ref=3.97146153200585
+Error norms, no tangent-linear   = [3.04885302 1.51235222 0.75316949 0.37583459 0.18772994]
+Orders,      no tangent-linear   = [1.01147243 1.00574771 1.00287673 1.00143908]
+Error norms, with tangent-linear = [0.04816965 0.01201053 0.00299865 0.00074916 0.00018723]
+Orders,      with tangent-linear = [2.0038242  2.00191591 2.00095891 2.00047969]
+Error norms, no adjoint   = [2.15029451 1.06909935 0.53304195 0.26614458 0.13297826]
+Orders,      no adjoint   = [1.00813834 1.00407494 1.00203891 1.00101982]
+Error norms, with adjoint = [2.41462429e-02 6.02522041e-03 1.50488756e-03 3.76044699e-04
+ 9.39890257e-05]
+Orders,      with adjoint = [2.0027128  2.00135832 2.00067964 2.00033994]
+Error norms, no adjoint   = [7.11786236 3.54650747 1.77014781 0.88429742 0.44195459]
+Orders,      no adjoint   = [1.00504505 1.00252916 1.00126624 1.00063354]
+Error norms, with adjoint = [0.04969483 0.01242371 0.00310593 0.00077648 0.00019412]
+Orders,      with adjoint = [2. 2. 2. 2.]
+
+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/examples/6_custom_operations.ipynb b/examples/6_custom_operations.ipynb new file mode 100644 index 0000000..9f7dbab --- /dev/null +++ b/examples/6_custom_operations.ipynb @@ -0,0 +1,921 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "82be80f5-acad-4947-8f9a-c6a7a87a7ace", + "metadata": {}, + "source": [ + "# Defining custom operations\n", + "\n", + "This notebook describes the tlm_adjoint 'escape hatch' mechanism, which allows custom operations to be defined, recorded by the internal manager, and used in higher order derivative calculations.\n", + "\n", + "The approach used by tlm_adjoint to define custom operations is described in\n", + "\n", + "- James R. Maddison, Daniel N. Goldberg, and Benjamin D. Goddard, 'Automated calculation of higher order partial differential equation constrained derivative information', SIAM Journal on Scientific Computing, 41(5), pp. C417–C445, 2019, doi: 10.1137/18M1209465\n", + "\n", + "We assume real spaces and a real build of Firedrake throughout.\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the Poisson equation in the unit square domain\n", + "\n", + "$$\\nabla^2 u = m \\quad \\text{on} ~ \\Omega = \\left( 0, 1 \\right)^2,$$\n", + "\n", + "with\n", + "\n", + "$$\\int_\\Omega m = 0,$$\n", + "\n", + "subject to boundary conditions\n", + "\n", + "$$\\nabla u \\cdot \\hat{n} = 0,$$\n", + "\n", + "where $\\hat{n}$ is an outward unit normal on the boundary $\\partial \\Omega$ of the domain $\\Omega$.\n", + "\n", + "We consider a continuous Galerkin discretization, now seeking $u \\in V$ such that\n", + "\n", + "$$\\forall \\zeta \\in V \\qquad \\int_\\Omega \\nabla \\zeta \\cdot \\nabla u = -\\int_\\Omega \\zeta m,$$\n", + "\n", + "where $V$ is a real $P_1$ continuous finite element space defining functions on the domain $\\Omega = \\left( 0, 1 \\right)^2$. $m \\in V$ is now defined via\n", + "\n", + "$$m = \\mathcal{I} \\left[ \\cos \\left( \\pi x \\right) \\cos \\left( \\pi y \\right) \\right] - k,$$\n", + "\n", + "where $\\mathcal{I}$ maps to an element of $V$ through interpolation at mesh vertices, and where $k$ is defined so that $\\int_\\Omega m = 0$.\n", + "\n", + "As stated the discrete problem does not have a unique solution – given any solution $u_0$ to the problem we can define a new solution $u_1 = u_0 + c_1$ for any scalar $c_1$. We need to identify a solution by supplying an appropriate constraint. We will use the constraint\n", + "\n", + "$$\\sum_i \\tilde{u}_i = 0,$$\n", + "\n", + "where the $\\tilde{u}_i$ are the degrees of freedom associated with $u \\in V$.\n", + "\n", + "An implementation using Firedrake, solving for $u$ and then computing $\\frac{1}{2} \\int_\\Omega u^2$, takes the form" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "17707b1c-9237-46a3-8c70-f1e700c8ad00", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:47:29.911083Z", + "iopub.status.busy": "2023-12-20T16:47:29.910402Z", + "iopub.status.idle": "2023-12-20T16:47:35.094214Z", + "shell.execute_reply": "2023-12-20T16:47:35.093474Z" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGzCAYAAACPa3XZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABy00lEQVR4nO3de1iUdf4//iczwAAq4IljJB4ytEwMEtFaK1mx/Nj6zcrT5gl1KymTNlMztZNkq2YHzUsztU1XszU/pkY/w6gtSROlzfLw8VCYOnhaQUGGYeb+/eHOHQNzumfuw/t9z+txXXNdec/7vufmDrifvN6HO0QQBAGEEEIIISoxaH0ChBBCCAkuFD4IIYQQoioKH4QQQghRFYUPQgghhKiKwgchhBBCVEXhgxBCCCGqovBBCCGEEFVR+CCEEEKIqih8EEIIIURVFD4IIYQQoioKH4QQQghRFYUPQgghhKiKwgchhBBCVEXhgxCNjBs3Dqmpqc22z5s3DyEhIeqfECGEqITCByGEEEJUReGDEEIIIaqi8EEIIYQQVVH4IIQQQoiqKHwQohF3g0ptNpvKZ0IIIeqi8EGIRlq3bo3Lly832/7rr7+qfzKEEKIiCh+EaKRz586oqqrCv//9b3Hb2bNn8cknn2h4VoQQorwQQRAErU+CkGB08eJFdOjQAfHx8XjqqadQW1uLd999F+3bt8f+/ftBP5qEEL2iygchGmnbti0++eQTREVFYfr06Vi7di0KCwsxZMgQrU+NEEIURZUPQgghhKiKKh+EEEIIURWFD0IIIYSoisIHIYQQQlQlOXx8/fXXGDJkCJKSkhASEoItW7Z43aekpAS33347TCYTunTpgjVr1vhxqoQQQgibli5ditTUVERERCArKwt79+712H7Tpk1IS0tDREQEevTogR07dji9P27cOISEhDi9Bg0a5NTm0qVLGD16NKKjoxEbG4u8vDxcvXrV5ecdO3YMrVq1QmxsrNP2lStX4q677kLr1q3RunVr5OTkeD13OUgOHzU1NejZsyeWLl3qU/uTJ09i8ODBuOeee1BeXo6nn34aEydOxOeffy75ZAkhhBDWbNy4EQUFBZg7dy7279+Pnj17Ijc3F+fOnXPZfvfu3Rg5ciTy8vJw4MABDB06FEOHDsXBgwed2g0aNAhnz54VX//4xz+c3h89ejR++ukn7Ny5E9u2bcPXX3+NyZMnN/s8q9WKkSNH4q677mr2XklJCUaOHIkvv/wSpaWlSElJwcCBA3H69OkArogPhAAAED755BOPbaZPny7ccsstTtuGDx8u5ObmBvLRhBBCCBN69+4tTJkyRfy3zWYTkpKShMLCQpftH3nkEWHw4MFO27KysoS//OUv4r/Hjh0r/OlPf3L7mT///LMAQPj+++/FbZ999pkQEhIinD592qnt9OnThT//+c/C6tWrhZiYGI9fS0NDg9CqVSth7dq1HtsFKlTZaAOUlpYiJyfHaVtubi6efvppt/tYLBZYLBbx33a7HZcuXULbtm3dPg+DEEIImwRBwJUrV5CUlASDQbmhhnV1daivr5flWIIgNLvfmEwmmEwmp2319fUoKyvDzJkzxW0GgwE5OTkoLS11eezS0lIUFBQ4bcvNzW02jKGkpARxcXFo3bo17r33Xrzyyito27ateIzY2FhkZmaK7XNycmAwGLBnzx78v//3/wAAu3btwqZNm1BeXo7Nmzd7/bpra2thtVrRpk0br20DoXj4MJvNiI+Pd9oWHx+P6upqXLt2DZGRkc32KSwsxIsvvqj0qRFCCFHRqVOncMMNNyhy7Lq6OqR2aIHKc3ZZjteyZctm4yfmzp2LefPmOW27cOECbDaby/vc4cOHXR7b3X3RbDaL/x40aBAefPBBdOzYEcePH8esWbNw3333obS0FEajEWazGXFxcU7HCA0NRZs2bcTjXLx4EePGjcOHH36I6Ohon77u5557DklJSc2KBnJTPHz4Y+bMmU6psKqqCjfeeCOOlCWhVUs2JuhcsAWeri/YTd4beXDR1kJS+8qGGN+P3dDK97ZW387jP9Yoj+9fqvd+nCqL52tWU+/9mtZYwj2+X18X5vY9W73R88HrPL9vqPdcuTNcc//9bbC4fQsAYPTyfmhdoPt7Xo/QaPG+XmForecbQ2id9xuH8VqD1zb+tDXUeLkAjdVe871tE0IA+zodp6ZWluOooUGw4mvrFrRq5fvvFanq6+tRec6OI/uS0KpVYPeJK1fsuDnzDE6dOuV0025a9VDSiBEjxP/u0aMHbrvtNnTu3BklJSUYMGCAT8eYNGkSRo0ahT/84Q8+tX/ttdewYcMGlJSUICIiwq/z9pXi4SMhIQGVlZVO2yorKxEdHe2y6gG4Lm0BQKuWBkQH+E0lh/O2erSSYZZynT2wY1yzebkRNhFp9f1/d0SDb20vWFvC15/HcKvnm36Yl1AAAKGhnj/M6OV9ADAaPX+OweD+fcHo5ZqHeAkfBi/hQ3D/PWH00uPo7bvB6CUbeH3f7rlBqJf3ASA0zEu4CANCr3luYwyVED6MVp/bGqT8OBkC+Ou6pQlCbeDBQQjx/WtjhRrd5q1ayXefiI6O9loxaNeuHYxGo8v7XEJCgst93N0X3bUHgE6dOqFdu3Y4duwYBgwYgISEhGYDWhsaGnDp0iXxOLt27cLWrVuxcOFCANe7kux2O0JDQ7FixQpMmDBB3HfhwoV47bXX8MUXX+C2227z+DXLQfE7eXZ2NoqLi5227dy5E9nZ2Up/tCLOy1DxAIDzAVY9zttaynIehBA+hbSUVvkkyggPD0dGRobTfc5ut6O4uNjtfc6f++Jvv/2GixcvIjExUTzG5cuXUVZWJrbZtWsX7HY7srKyAFwfF1JeXi6+XnrpJbRq1Qrl5eXimBAAeP311/Hyyy+jqKjIaQyJkiRXPq5evYpjx46J/z558iTKy8vRpk0b3HjjjZg5cyZOnz6NDz74AADw2GOP4Z133sH06dMxYcIE7Nq1Cx999BG2b98u31dBSBCwmbx3jWipISLEa9eMHGyRoT53p9iiwmCs9a1CYG9h8r3rpUUkUCNP9wnhX0FBAcaOHYvMzEz07t0bS5YsQU1NDcaPHw8AGDNmDJKTk1FYWAgAmDp1Kvr3749FixZh8ODB2LBhA/bt24cVK1YAuH6fffHFFzFs2DAkJCTg+PHjmD59Orp06YLc3FwAQLdu3TBo0CBMmjQJy5cvh9VqRX5+PkaMGIGkpCSxTWP79u2DwWDArbfeKm5bsGAB5syZg/Xr1yM1NVUcL9KyZUu0bKncH7mSKx/79u1Dr1690KtXLwDXL3qvXr0wZ84cAMDZs2dRUVEhtu/YsSO2b9+OnTt3omfPnli0aBHee+898QLyRK6qB+suSBjv4atLPo4LISQYhER5Hv9E+DJ8+HAsXLgQc+bMQXp6OsrLy1FUVCQOKq2oqMDZs2fF9n379sX69euxYsUK9OzZEx9//DG2bNkihgKj0Yh///vfeOCBB9C1a1fk5eUhIyMD//rXv5yGJKxbtw5paWkYMGAA7r//ftx5551igPHVu+++i/r6ejz00ENITEwUX46uGqVw8VTb6upqxMTE4MyRGzQb8yFn8Ai0ywXwr9vFbI31qZ2U8HHB6tt5+BI+Llq8t7ls8TwI6qqXAamA9wGnljr379ssAQ44tfg/4BTwXPkwehlQ6nXAqdf9fRjT4aWNtwGngPcxH4DEQac+Vj4AiYNOA6x8yDHuAwCEqzWyHEdJDYIVu+o3oaqqyudZF1LJeZ+ovmJH0s2/KXq+wU770ZscYK3iQeM9tGM02bQ+Ba41RNGvHEIIhY+g4WvVg/DLpuzMOCIz6nohwYzChxesVT2CmbcuF19463Ih+mOLcr9uS1P2FhK6RFu4XipAbTTrhfCIwocHSgQPPU2x9XW8ByGEENIYhQ83qOJBiDYaIr3/WrJFMrk4s2TU9UKClT5+ggkhXjVEeJ/xQvgU0rIFF7Ne1HDBVg+LLcDl1W3yPB+GuEeVDxf0VvWgwaZEDg0R3pfG9qUNIYRQ+GhCyeDBw3gPJRYY44mnNT4IvxQbdCoD6nohwYjCRyN6q3iwQq4FxghhDiMzXgjhDYUPQoiqaKExZdCUW8IT+i3wX0pXPeRYUp1oz+vy6hzjbZEyvcx4ISQYUfgAH90tLK3vQQiRF437IMEm6MOHGsFDy6qHUjNdaIExoiRf1vogzVHXC+EF/YQTUbDPdCHEgcdl1gnhSdCGj/O2ei66Wwgh8pAy3VYL1PVCgklQhg81Q4ccXS403sM3Vy00qJcQ6nohPAi68EHVDkIIIURbQTVXLdiCBy2rTvTOFhkK47UGrU/j+riPmmsBHyYkKgpCba0MJxS8LthNqLMH9nf1VTs920VpQVH50Gp8B63tId/qppctnC1CQbin9jLrcqKuF8I63YcP3qsdao33oJkuRC4+PYCOVjlVHAUQwjJd/wbgPXiwitb4CF68P7VW0RkvMk25lXPWCwUQwipdhg8WptFSlwshgaGFxuRBAYSwSHc/3VqHDjnRFFtCiBwogBDW6Cp86Cl4BIpmupBgwdQD5hjsehGPSQGEMEQX4YOFbpbGeOtyocGmhLjG84wXVyiAEFZwHz5YCh1yoi4XogUbzWjWPQoghAVchw8WgwdvVQ+paKYLIfyjAEK0xmX4YK2bRW56qXr4ssAYIWpT/AFzDI/7cDo+BRCiIYZGavlGz6GD+M+Xh8rVWMJVOBNC+BHSsgWEqzVan4asLtpa4JrNGNAxam02mc6GuMNV5eMC48GDlS4Xnma60NLqRA5MzXgBuKl+AFQBIdrgKnwEA7W7XGimi754GzDaIEOO82n5dJmWWFdqoTEpXS9+z3jhLIA4XoSogcKHTOSoerA+1kPKYFMa76EMGxvFNSYpWf0IhgAifhYFEaICCh8yYKW7BeCry4UQLSk+8NSBwwAifiYFEaIQCh8Bkit4yFH1kBo8pHS50BRb0hirXS9MVj8ArgOI+NmNggiFERIoCh9+Om83cR08lERdLoQXqlU/AF0EkMYoiJBAUPiQwBE45Oxm0Sp4BNtAU5pm6zveVzlltvoB6C6AOFAQIVJR+PCB3IFDPC7jA0wdlOpykWuarS9rfAQTNWa8XD8Om10vUqla/QB0G0AcKIgQX1D4cEOJKofT8WUKHix1twB8d7lY6viojtCMF++Yrn4Aug8gDmIQacH2ecph6dKlSE1NRUREBLKysrB3716P7Tdt2oS0tDRERESgR48e2LFjh9P78+bNQ1paGlq0aIHWrVsjJycHe/bsEd8vKSlBSEiIy9f3338vthMEAQsXLkTXrl1hMpmQnJyMV199VXx/3LhxLo9xyy23yHRlXKPw0YjSgUNu/gaPYOtyIcFJSgBRvfoBBE0ACQYbN25EQUEB5s6di/3796Nnz57Izc3FuXPnXLbfvXs3Ro4ciby8PBw4cABDhw7F0KFDcfDgQbFN165d8c477+DHH3/EN998g9TUVAwcOBDnz58HAPTt2xdnz551ek2cOBEdO3ZEZmameJypU6fivffew8KFC3H48GFs3boVvXv3Ft9/8803nY5x6tQptGnTBg8//LBCV+u6EEEQBEU/QQbV1dWIiYnBv3+OQ6tW8uYlLYKG1gNMlZrlIqXq4UuXCyBft4svYz58qXzYLF6Wba7z/L7B4rkbwnDNt+9vo8XL+3We3w/18r63/X8/jm+/PnxpF1pr997mmvc2jRmvNUhrX2v1ua2hxsv/BF/VXJPlMEJtrSzHUUKDvR7F/1mLqqoqREdHK/IZjvvEph/SENUqwOXVr9jwcM/DOHXqlNP5mkwmmEzNf99kZWXhjjvuwDvvvAMAsNvtSElJwZNPPokZM2Y0az98+HDU1NRg27Zt4rY+ffogPT0dy5cv9/j1ffHFFxgwYECz961WK5KTk/Hkk0/ihRdeAAAcOnQIt912Gw4ePIibb77Zp699y5YtePDBB3Hy5El06NDBp338EZSVDy0rHDwFD0J4x3z3C0AVEJlVNsTAbI0N6FXZEAMASElJQUxMjPgqLCxs9nn19fUoKytDTk6OuM1gMCAnJwelpaUuz7G0tNSpPQDk5ua6bV9fX48VK1YgJiYGPXv2dNlm69atuHjxIsaPHy9u+/TTT9GpUyds27YNHTt2RGpqKiZOnIhLly65vX6rVq1CTk6OosED4PDBcoHQujuFt+BBa3sEL1uEb9WPhogQn6sfPLBFhUmqfsimRaQsFZCQqCimKyC8cVX5aOrChQuw2WyIj4932h4fH4/Dhw+7PK7ZbHbZ3mw2O23btm0bRowYgdraWiQmJmLnzp1o166dy2OuWrUKubm5uOGGG8RtJ06cwK+//opNmzbhgw8+gM1mw7Rp0/DQQw9h165dzY5x5swZfPbZZ1i/fr3Lz5CT7sOH1oHDgZeZLf7ivcuFFfZIu89dL4FoiPDe9SLv53kPKQ1RBq9dLw2RBsldL7bIUMndL76ytzDJ1/1CAYQ50dHRinUT+eKee+5BeXk5Lly4gJUrV+KRRx7Bnj17EBcX59Tut99+w+eff46PPvrIabvdbofFYsEHH3yArl27ArgeUjIyMnDkyJFmXTFr165FbGwshg4dqujXBeiw26Vxl4reggd1tyiHl5kuvuJ9rQ4taTL41IG6YLjTrl07GI1GVFZWOm2vrKxEQkKCy30SEhJ8at+iRQt06dIFffr0wapVqxAaGopVq1Y1O97q1avRtm1bPPDAA07bExMTERoaKgYPAOjWrRsAoKKiwqmtIAh4//338eijjyI8XPnfh9yHDxbDRmMsBA9/0EPkiJ5wMfbDQcYAQiFEeeHh4cjIyEBxcbG4zW63o7i4GNnZ2S73yc7OdmoPADt37nTbvvFxLRbnSpsgCFi9ejXGjBmDsDDn4NyvXz80NDTg+PHj4rajR48CQLMxHV999RWOHTuGvLw8j+cgF+66XVgMGK7I2c0SaPBgqerha5cL4Yev4z607HqRSurYD1m7XwDZumCA5lUQ6pKRX0FBAcaOHYvMzEz07t0bS5YsQU1NjTj4c8yYMUhOThYHrE6dOhX9+/fHokWLMHjwYGzYsAH79u3DihUrAAA1NTV49dVX8cADDyAxMREXLlzA0qVLcfr06WZTYHft2oWTJ09i4sSJzc4rJycHt99+OyZMmIAlS5bAbrdjypQp+OMf/+hUDQGud8dkZWXh1ltvVeISNcNV+LhgN4GHkRO8Bw8WBpr6Mt6DECmUHPsBsB1AGmscRiiIyGP48OE4f/485syZA7PZjPT0dBQVFYmDSisqKmAw/N7R0LdvX6xfvx6zZ8/GrFmzcNNNN2HLli3ijd9oNOLw4cNYu3YtLly4gLZt2+KOO+7Av/71r2aLf61atQp9+/ZFWlpas/MyGAz49NNP8eSTT+IPf/gDWrRogfvuuw+LFi1yaldVVYV//vOfePPNN+W+NG5xtc7HroMpaCnzOh9yYyl4AMqHDyUGmgK+hw+m1vgAAl7nA/BtrQ9v63wAga/14csxfj+Wuut9ANLX/ACUXffDQdYAAigSQFxRMoiouc7HO2VZiGwZ2N/V1642ID9jj6LnG+zYvpNzhrUZLbxWPYg65HrGi7TPlOdZL/6SOvbDn8GnrI4B8cYxRoTGiRA1UPiQwXlbS9mDh57GeUildtVDr/Q+40XJh80FitcA4kBBhCiN3Z9eTihR7dDqYXFSqx5KdbmoTW/TbLXgS0WDFWpUPwD+A4hD4yBCYYTIhcJHAFgNHtTdQnjna9cLy9UPQD8BpDEKIkQOXM12YYGS4zq0Ch7+UGptD5rlon8sLMkudeZLIMuu8zILxh8szpy52NAKEQ2B3drqGpSbFUWuo/DhAzUGkmrV1QIoX/VQosvFl/EecvJpposO+fqMFy2pse5HoPQcQBxYDCKEXRQ+POAtdPA8yJRoQ+1nvPAs0IfOBUMAcaAgQrxhu8NUA46ZK8ESPPypevDS5RLMM120IPegUylTbv0Z+6Hkkuvu6HEMiDfXx4ewf55EXUFf+dBqbQ5eg4dUrHe5yDrTxcsCY4R/gVY/gOCqgBDiTtBVPhpXNoI5ePiLl6oHkY8S64UoMTVXreqHHE+9DcYKCCGN6b7ywdqqo1oOLG1Mje4Wltf2IHzw5UFzvKIKCAlmugsfrIUNByVCB8vdLUpRe5YLELwzXXjkz8wXfx44J0f3C0ABhAQv7sMHq2GjMZaCh7+U6m4BqMtFT1hYw4M3FEBIMOIufPAQNhpjLXioNbtF6y4XmumiH1K7XnirfgAUQEjw4WrA6UUbX2MI9BA8WCJ3lws908V3vAw65Zkig1BpICphlF/hY+nSpUhNTUVERASysrKwd+9ej+2XLFmCm2++GZGRkUhJScG0adNQV6fvlY3kDh4XGlppsoiY0lUPJbpcNKl60DRbpvA086Ux2QMI8HsIoTBCGCL5p23jxo0oKCjA8uXLkZWVhSVLliA3NxdHjhxBXFxcs/br16/HjBkz8P7776Nv3744evQoxo0bh5CQECxevFiWL4IlrFU7xGMwtJgY0Tclxn3oedZLU7J3wTTVOIDosGvmorUFTNbAQqHFKk93GnFP8p8HixcvxqRJkzB+/Hh0794dy5cvR1RUFN5//32X7Xfv3o1+/fph1KhRSE1NxcCBAzFy5Eiv1RIe6Sl4+EupsR5azHIBaKZLsGGh+gEoVAFxhSoiRCOSwkd9fT3KysqQk5Pz+wEMBuTk5KC0tNTlPn379kVZWZkYNk6cOIEdO3bg/vvvd/s5FosF1dXVTi/WsRo8/KVG1UM3XS4+MljkG+Ngk/He1ODj/wbexn340/XiL64DiAN1zxAVSYr5Fy5cgM1mQ3x8vNP2+Ph4HD582OU+o0aNwoULF3DnnXdCEAQ0NDTgsccew6xZs9x+TmFhIV588UUpp6YpJcZ3yHYsqnoQDbDS9aLWzBfg9wAi1wwY4PcAomg3jDuuAogOu2mINhT/06CkpATz58/HsmXLsH//fmzevBnbt2/Hyy+/7HafmTNnoqqqSnydOnVK6dOUxGyNdXrJiYXgwePUWqlYnulij5RvbIPc1Qreqh+A+g+ds0WFKTIQVfVKiCtNqyNUJSF+kvQT1q5dOxiNRlRWVjptr6ysREJCgst9XnjhBTz66KOYOHEiAKBHjx6oqanB5MmT8fzzz8NgaP6LwWQywWRi4AcN6i2HrmUXS2NqBY9g63IJRqxUPwB1KyDi/gpVQjSpgnhDVRIikaQ/CcLDw5GRkYHi4mJxm91uR3FxMbKzs13uU1tb2yxgGI3XB/EJAjsrITatZihR1XBH7uBxwdqSZrcEAS3GfQDaVz8aovwr2KpdARGPIXMlhJkqiDeNKyNRVB0hziT/ZBUUFGDs2LHIzMxE7969sWTJEtTU1GD8+PEAgDFjxiA5ORmFhYUAgCFDhmDx4sXo1asXsrKycOzYMbzwwgsYMmSIGELUxszD3RSodqi9kJjS3S1KjPdguctFb6RUPyS15agCIh5HxhVRAY3HgxASIMnhY/jw4Th//jzmzJkDs9mM9PR0FBUViYNQKyoqnCods2fPRkhICGbPno3Tp0+jffv2GDJkCF599VX5vgo3WAkZrihR7QiEmuM86Fku6rJFAEaZ1/STckylnvfCawABdDQolRA/hQgs9X24UV1djZiYGGz6IQ1RrVxXS1gOGo2xWO3wt7tFjbEevlY+pIz3kFL58HmdDx9WOJUy1dZwzbcuAqOP9xtfg0KoxJDi+3F9/zUjNaj4u/iY1AACQJYA4nQ8GUOIA4shpMFmwRf/9waqqqoQHR2tyGc47hNPffMnmFoGuMjYVSveuvN/FT3fYMfVg+UqG2IQaeXqlEUsho5AsBQ8gpnN5HsA0ZKST7vlsQIiHi+YBqUS0ghXD5bjFcvBIxim1TYWrOM9lBgkqtRx/Zl6y9sg1GbHVGhQatMXIazgs4zACZZDB6CfcR40xZYPSlY/AL4rIOJxZR6U2pS7AKKnSsl/rFEItwb2O6HeWi/T2RB3KHzITMn1OrQOHv6iAabKkrPrpSFC+rgPacf3LYD4G1T0EkAAZcaDuOMqlOgpkBD2UPiQCS+hA2B7gKkDTbHlhxKzaYDgDiCANiGksWCokhDtUPgIgBqrkrJS7eB5nAcr7CZB1ofLSaVUSJBK6e4XQP0AAsg/E0Y8vsYhpCmqkhA5UPjwgRZLn7NS7QiEGlUPJcd7+DzNVmf86XphrfoBqBtAgOALIY1RICFSUfhoQutnrCgxfTbQ4MHKs1uIe7xMuXVQo/oBqB9AAHVCCIsBpKnGgcRu0/BECJOCNnxoHTJcYbHaoXbwUGptDxrvcZ2SXS8sVj8AbQIIoGwIYbkKQogvdB8+WAwZTbFY7QDUH+chNXhQlwvflHrui8v9NQogAIUQQlzhepGxCw2tvL5Y5u/TZ73R8um0LM1uIf6T8pTbwD5HvQG4ai5E5ootMlSRBcoA+RcpI9ItXboUqampiIiIQFZWFvbu3eux/aZNm5CWloaIiAj06NEDO3bsEN+zWq147rnn0KNHD7Ro0QJJSUkYM2YMzpw54/JYFosF6enpCAkJQXl5udN7H330EdLT0xEVFYUOHTrgb3/7m9P7Z8+exahRo9C1a1cYDAY8/fTTfn39UnEVPi5yFCw8UTJ0yBU8eJ9W25jULheqevhPqZVUAXmCitYBBPg9hPCwUirxzcaNG1FQUIC5c+di//796NmzJ3Jzc3Hu3DmX7Xfv3o2RI0ciLy8PBw4cwNChQzF06FAcPHgQAFBbW4v9+/fjhRdewP79+7F582YcOXIEDzzwgMvjTZ8+HUlJSc22f/bZZxg9ejQee+wxHDx4EMuWLcMbb7yBd955R2xjsVjQvn17zJ49Gz179pThaviGqwfLvbgnBxEt+ewpUvI5LHJWOgLpalE7fPja7aJK+PDhwXKA7w+X8/XBco1JGXDqy9gMfxcb82fch69dKnINUlXzYXS+UGpwKsBGl0yDzYJd5a+p8mC5R78cifCWAa5werUef7/nHz6fb1ZWFu644w7xpm6325GSkoInn3wSM2bMaNZ++PDhqKmpwbZt28Rtffr0QXp6OpYvX+7yM77//nv07t0bv/76K2688UZx+2effYaCggL885//xC233IIDBw4gPT0dADBq1ChYrVZs2rRJbP/222/j9ddfR0VFBUJCnH8f3X333UhPT8eSJUu8fs2B4qrywRtHhYOCh3tKBw/iP3+7XlivfgBsVEAaU6NLxtWLeFZdXe30sliaJ/z6+nqUlZUhJydH3GYwGJCTk4PS0lKXxy0tLXVqDwC5ublu2wNAVVUVQkJCEBsbK26rrKzEpEmT8Pe//x1RUVHN9rFYLIiIcP6BjIyMxG+//YZff/3V7WepgcKHApQOHIC8XSwAX8FDSUpWPYh7UkKFXgMIoGwIcfl5Ogwll+pb4KIlsNel+uu/D1NSUhATEyO+CgsLm33ehQsXYLPZEB8f77Q9Pj4eZrPZ5TmazWZJ7evq6vDcc89h5MiRYiVGEASMGzcOjz32GDIzM13ul5ubi82bN6O4uBh2ux1Hjx7FokWLAFwf66ElPvswGKTm4+1ZCR2ANsFDStWDpti6pvRqp/4cX83ZL+JxNJwF44nSa4V4/XwPAYSFbhy1nDp1yqnbxWRS/w8mq9WKRx55BIIg4N133xW3v/3227hy5Qpmzpzpdt9Jkybh+PHj+J//+R9YrVZER0dj6tSpmDdvHgwGbWsPVPkIkBpVDgeWqh2BUCt4kMCpNevFH3qugDioXQnxhd4qJZ5ER0c7vVyFj3bt2sFoNKKystJpe2VlJRISElweNyEhwaf2juDx66+/YufOnU5BaNeuXSgtLYXJZEJoaCi6dOkCAMjMzMTYsWMBACEhIViwYAGuXr2KX3/9FWazGb179wYAdOrUSeLVkBeFDwkaj+FQM3QA8lc75Ageaq9iqnTwoC4X7UkNFMEQQAA2Q0hjwRRImgoPD0dGRgaKi4vFbXa7HcXFxcjOzna5T3Z2tlN7ANi5c6dTe0fw+L//+z988cUXaNu2rVP7t956Cz/88APKy8tRXl4uTtXduHEjXn31Vae2RqMRycnJCA8Pxz/+8Q9kZ2ejffv2AX3dgWL3u1lDaoYKb+Res0OuagcP4zyoy0Vbaj3ITu9dMI1p3R0jReMAYmvQ9/rqBQUFGDt2LDIzM9G7d28sWbIENTU1GD9+PABgzJgxSE5OFseMTJ06Ff3798eiRYswePBgbNiwAfv27cOKFSsAXA8eDz30EPbv349t27bBZrOJ40HatGmD8PBwpxkvANCy5fX7VufOnXHDDTcAuD4e5eOPP8bdd9+Nuro6rF69Gps2bcJXX33ltK9jbZCrV6/i/PnzKC8vR3h4OLp3767MBUOQhg+WwoUrSi0SxnPwoO4W+fkaDvx50Fwg/AkTLAQQBzWCiC0ylIsAEiyGDx+O8+fPY86cOTCbzUhPT0dRUZE4qLSiosJpjEXfvn2xfv16zJ49G7NmzcJNN92ELVu24NZbbwUAnD59Glu3bgUAcdqsw5dffom7777b53Nbu3Yt/vrXv0IQBGRnZ6OkpETsenHo1auX+N9lZWVYv349OnTogF9++UXCVZBGd+t8sB4sPGE9dAB8BQ8plQ+1ulxYWedD3MfHUBFI+FBy3Q+59mt2HD/XAXE6hkrVEB5CSENDHb4qfUWVdT4Gfz4RYS0C+2PFWlOP7bnvKXq+wY6rysdFawuYrPrqS1R6KXStqx2BUiN4EO94qH4Esl+z4/hZAXE6xn+rIUqHEJ66Yghx4Cp86IUaz15hodrhwPpCYrScurL8HfuhhwACUAghxBWa7aIiuafKuqOH4BEIVaoeCna5sIrlabdNaT0LxuWxIg1BOz2XkKboO1Rhaj9hlqVuFtbHeRD1qF39CHRfp+PIVAERj6dyJaQpqowQFlD4UAivoQPQbnyHg5ozW6jLRd9YDSCAeiGkKQolhAUUPmSkduAA2A0dWozzUK3qEYRdLg5qDzy9/pmBBQiWAwigXQhpylNXDU/BpMpiQmhoYN29DTr5eWUZhY8AaRE4APmXRqfgQZSm1qJjrrAeQAB2QogrVC0hcqPwIYFWQcNBqWexyBU8tHpKrb/Bg5ZT9w+P1Q+5jgEoG0AAtkNIU3qplhD1Ufj4L62DhSeshw4g8OCh5xVMWe5yUbsaoWX1A+AngADqr5oqt8bBxNZAtxriLCi+I1gOFu4o/cRZvQQPvXa32CPtklc5tZn8W+XUH4FUP7SY+dL0OEDgq6E6puEqHUIA/oMIIU1xGz54DBTeqPWIe967WRwCCR40y+V3/ocBbQIIIM8y6nKHEICCCCG+4ip8/McahXCrvv7SVStwAGxVOxz03N0SKNarH4A2AeT658obQuR6JgwFEUJ8w1X40As1A4cDi8EjEFT1YIdWAeT6Z8tUvZAxzIjHpCBCiFsUPlSkRegA2O1m0aLqwVvwUKv6EXgI0C6AXP98dkMIoG0QcYcCCtEShQ+F8R44HFiodjjodZAp77QOINfPQb5BqXIHEPHYTZ4Xo0YYcXkeFFCIhih8KEAvgcNBDxUPgL+qRyC0qH4A7AQQgN0qSLPPUbkqIoW3gELhhPiLwoeftAoYTSn9HBY5g4ccoYOqHuxjIYBcPw++QgjAdhBxhaonxF8UPhphJVB4o8aD31gLHYEKpqqHg1bVD4CdAHL9XPgLIQB/QcSdhkgDGqzSxi0FoqbeBGOAz3ax1ct0MsStoAgfvIQKT9R80iyrwYOqHnxhKYBcPx/2x4O4/UydBBFCHLgKH5fqWyCMgb+i1aLFo+31uH5HMFY9HLSsfgBsBhCAvyqI02dTECE6wFX4CBZahA6A7eBBVQ9+sRZAAH2EEICCCOEXhQ9GaBU4AHa7WeQQzFUPB62rHwCbAQSQ/3kxDlQRIcQzCh8a0jJwAPxMoaWqhz6wHEAAmVc3jWj+JGM1A0nTtUQ8oaBCtEDhQ0V6CxsOrFU7HKjq8TsWqh8AuwEEUL4LRetA4o6vQYVCCpEThQ+ZaR0wmlJ6ZVKlgwdTVY+64AszwRZAAJXX9WCgu8ZXvoQUCijEVxQ+/MBawGhMzWXQlQwegYYO2aseOgge/j7xlsUAAqgTQlSfUtsojLAcRNxxF1DUXOeD8IHCRyMshwpPtHjuSlCN79BB8AgUawEE+D2EAHyNB5H62Vp9PiFK0k344DU4BIJCh2uyVj10Fjz8rX4AbAYQB6WrIZpPqaUgQnSGq/BRZTEhNMBlc3mn5dNlg279Dp0FDzmwHEAAdUKI1jd/CiJED7gKH8GKhcfZyx08lAodslU9dBw8Aql+AOwHEEDZEKJ1FaQxCiLN1VjCYTQGOmaMrqXSKHwwSo+BA2C80uGg4+AhFx4CCKDsuBCWQghAQYTwhcIHI1gIGwAfYzrckaXqwWDwsEfaYbgm72yBQKsfAD8BxEGpagiLN31Xa4q4w8o5k+BC4UNFrASMpoJqrQ4iK94CCKBOl4wDDzd2KUHFgYevi7CNwofMWA0Yjam1IqmaoUOvVQ8lyVH9APgMIIC6U3Ub08ON25fAooevkyiHwocfeAgYTam9BDoFj+DCawBxUGvhMkC/gaSpxl+nzSC9ukL0LajDB48hwldaPm9FreBBM1vYwnsAAdQNIY0FSyAhxIGr8FFTb4IxyNf58ETrB7xxFzqAoA8ecnW9iMfTQQAB1OmS8YbH8SOE+IoW3OdcjSVcfGlJrdksvAUPgyX4ys2Nb9xyadBwAWNbhDJfk1QNESHNXoQdS5cuRWpqKiIiIpCVlYW9e/d6bL9p0yakpaUhIiICPXr0wI4dO5ze37x5MwYOHIi2bdsiJCQE5eXlTu9funQJTz75JG6++WZERkbixhtvxFNPPYWqqiqndk899RQyMjJgMpmQnp7u8lwEQcDChQvRtWtXmEwmJCcn49VXX5V8DaSg8MEpFgIHcD10KB08ZA8dAAWPRmwKFBP1FkAANgJIUxRE2LBx40YUFBRg7ty52L9/P3r27Inc3FycO3fOZfvdu3dj5MiRyMvLw4EDBzB06FAMHToUBw8eFNvU1NTgzjvvxIIFC1we48yZMzhz5gwWLlyIgwcPYs2aNSgqKkJeXl6zthMmTMDw4cPdnv/UqVPx3nvvYeHChTh8+DC2bt2K3r17S7wK0oQIgsB8La+6uhoxMTHo9XEBjFHB1e3CQsBoisvulaYUDh9yBw+51/loSs6uF6fjKtRloXY3TFNadcX4irUuGlt9HQ6sex5VVVWIjo5W5DMc94mu62YEfJ+w1VpwdPRrPp9vVlYW7rjjDrzzzjsAALvdjpSUFDz55JOYMWNGs/bDhw9HTU0Ntm3bJm7r06cP0tPTsXz5cqe2v/zyCzp27IgDBw64rVw4bNq0CX/+859RU1OD0FDnURXz5s3Dli1bmlVQDh06hNtuuw0HDx7EzTff7PVrlQtVPhjQuOuk6YsFjuqGGlUOQKFKR2NBPs7DFSWqH4By1YKGCOqK8cRVF427F3Gvurra6WWxNE/p9fX1KCsrQ05OjrjNYDAgJycHpaWlLo9bWlrq1B4AcnNz3bb3lSMsNQ0ennz66afo1KkTtm3bho4dOyI1NRUTJ07EpUuXAjoXb7gacMo7VsKEL7RYGEzRwKEiXrpb1KLEIFSHxgFEi2qIVrNj5KS3Rcbq68JgMAT2+8teZwcApKSkOG2fO3cu5s2b57TtwoULsNlsiI+Pd9oeHx+Pw4cPuzy+2Wx22d5sNvt9zhcuXMDLL7+MyZMnS9rvxIkT+PXXX7Fp0yZ88MEHsNlsmDZtGh566CHs2rXL7/Pxxq/Kh9SBNZcvX8aUKVOQmJgIk8mErl27NhtcoxesVzE8UbO60ZjilY7GOOtu0Qs1qgRaVkMclRDWKyJykVJZ4bnCcurUKVRVVYmvmTNnan1KLlVXV2Pw4MHo3r17s3Dkjd1uh8ViwQcffIC77roLd999N1atWoUvv/wSR44cUeaE4UflwzGwZvny5cjKysKSJUuQm5uLI0eOIC4urln7+vp6/PGPf0RcXBw+/vhjJCcn49dff0VsbKwc5686HkKEr7Re9lwvlQ4HCh6eKVkBacwRQLQcF+IqgPBcHQlW0dHRXsd8tGvXDkajEZWVlU7bKysrkZCQ4HKfhIQESe09uXLlCgYNGoRWrVrhk08+QVhYmKT9ExMTERoaiq5du4rbunXrBgCoqKhQbByI5MrH4sWLMWnSJIwfPx7du3fH8uXLERUVhffff99l+/fffx+XLl3Cli1b0K9fP6SmpqJ///7o2bNnwCffmKeKg5wv3mlV3WhM1UqHQ51R0aoHBQ/fqFkVcFRCtJ4l4xBs1ZFgER4ejoyMDBQXF4vb7HY7iouLkZ2d7XKf7Oxsp/YAsHPnTrft3amursbAgQMRHh6OrVu3IiJC+jdWv3790NDQgOPHj4vbjh49CgDo0KGD5OP5SlLlwzGwpnHpydvAmq1btyI7OxtTpkzB//7v/6J9+/YYNWoUnnvuORiNrm8GFovFaWBPdXU1gOsBw2jkPwCoSevqRmOaVDpoSi1z1KqANKb12BBX3AUQqpDwp6CgAGPHjkVmZiZ69+6NJUuWoKamBuPHjwcAjBkzBsnJySgsLARwfWpr//79sWjRIgwePBgbNmzAvn37sGLFCvGYly5dQkVFBc6cOQMAYhdIQkICEhISxOBRW1uLDz/8UBwUCwDt27cX76/Hjh3D1atXYTabce3aNXG2S/fu3REeHo6cnBzcfvvtmDBhApYsWQK73Y4pU6bgj3/8o1M1RG6Swoc/A2tOnDiBXbt2YfTo0dixYweOHTuGJ554AlarFXPnznW5T2FhIV588UUpp0YaYSlwOGhS6dARe6Rd8em2cq926vGzNByoyWIQaYxCCX+GDx+O8+fPY86cOTCbzUhPT0dRUZF4r6yoqIDB8PvPb9++fbF+/XrMnj0bs2bNwk033YQtW7bg1ltvFdts3bpVDC8AMGLECAC/D3rdv38/9uzZAwDo0qWL0/mcPHkSqampAICJEyfiq6++Et/r1auXUxuDwYBPP/0UTz75JP7whz+gRYsWuO+++7Bo0SIZr1Bzktb5OHPmDJKTk7F7926n8tD06dPx1VdfiReisa5du6Kurg4nT54Uk9jixYvxt7/9DWfPnnX5Oa4qHykpKbLM39YTFkNGY5qN6VA5eKhZ+VA6gADqBRCnz2TkxspiGPGGlWvnia2+Dj+umqXKOh+pq16AISqwfi17bR1+yXtZ0fMNdpIqH/4MrElMTERYWJhTF0u3bt1gNptRX1+P8PDmN1CTyQSTiUKGA+shozHNB5HqrOLRlN4qIOJnMjJllfWqiCu+jh/R+toS0pik32L+DKzp168fjh07BrvdLm47evQoEhMTXQaPYNV0IS8WBob6yjGAlIKHOuyRdu+NAqTUomNeP5ehwZiNB6yyMmg1EE0HvEp5ESI3yX9CFRQUYOXKlVi7di0OHTqExx9/vNnAmsYDUh9//HFcunQJU6dOxdGjR7F9+3bMnz8fU6ZMke+r4AivAcMVJgKHQ5AEDwe1AoirlxpYvOnpKYhIFUhwYe3/I2GD5HU+pA6sSUlJweeff45p06bhtttuQ3JyMqZOnYrnnntOvq+CQbwGCm+YCRuNBVnwcFCjC8YVRwBRo2uGhUfbu9I0gPDSRaMVrSpphF1+La+en5+P/Px8l++VlJQ025adnY3vvvvOn48KiF4DgNqYDBxA0IaOxrQKIIC6IQRgZ1yIK1KrIRRWSLDj6tkucqzZT3zDbOBwoOAh0jKAAM5/1apdDfGExZDi4C6sUCgJnK3eCMHNGlK+stfT7xelcRU+iLKYDxwOFDya0TqAOKhdDfGkaUhhOYw4UCghwYLCB+EndAAUPDxgJYAAbIUQB1bHj/iCQgnRGwofOsFVgPAXBQ+vWAogAJshBOCzKuIKhRLCKwofnAmKkOEKBQ+fsRZAAHZDiAPPVRFX1JgOTAGHBILCB6OCNmS4QsFDMhYDCMB+CAH0UxVRmpSAY6NnL5ImKHxojEKGFxQ8/MZqAAH4CCEOequKEMICrsKHL1OojCab633pJq8+Cg6aYzmAAHyFEICqIoTIhavw4QsKGRqjwMEc1gMIwF8IcZBz6XAKMiSY6C58EI1Q6GAaDwEE4DeEyMFdkKFQQvSIwgfxHwUOrvASQIDgDiFNeaquUDAhvKLwQaShwME1ngIIQCHEG6qWEF5R+CDe6SxwGCzBPe+PtwACUAiRSovH2DMTeOqMQEiAv7N09juPRRQ+eEY/IJIFe/Bw4DGAAMo+mp2CTWA8BR7XcxBJMKPwwQMKGbKg4OGM1wCilKbBhsIIIcrhK3zIUU4jQYmCh2sUQNxrHEYoiBAiL77CByF+oODhGQUQ76gqQoi8KHwQ3aLQ4TsKINJQGCEkMBQ+iC5R8JCOAoj/lBwI6w4FHsIzCh9Edyh4+I8CCD/cBR4KJYQHFD6IoigI8IcCCN+0qMK4QiGIeELhg8iOAgf/KICQQDUOQXZBu/MgbKLwoSOB3vTtJv9/Q1Dg0B97pN3vfSm4EEI84Sp8GOpDYDDQTU4pFCCIXAIJLo1RiCFEn7gKH4SQ4OIIMRRCiK9k+SO1nv4QUxr9RBNCmGePtMtWTSGEaI8qH4QQbrAWQKgiQ4h/KHwQQoif1ApDFHKI3lD4IIQQxrFW8ZHKHsL3+RP5UZwmhBBCiKq4Ch9UeiSEEEL4x123i14CCO9lVEIIIcRf+riTc8hwzaDoixBCCHsuXbqE0aNHIzo6GrGxscjLy8PVq1c97lNXV4cpU6agbdu2aNmyJYYNG4bKykqnNhUVFRg8eDCioqIQFxeHZ599Fg0NDU5tSkpKcPvtt8NkMqFLly5Ys2ZNs886ffo0/vznP6Nt27aIjIxEjx49sG/fPvF9QRAwZ84cJCYmIjIyEjk5Ofi///s/ydeB7lI6pXS4oSBECCHSjR49Gj/99BN27tyJbdu24euvv8bkyZM97jNt2jR8+umn2LRpE7766iucOXMGDz74oPi+zWbD4MGDUV9fj927d2Pt2rVYs2YN5syZI7Y5efIkBg8ejHvuuQfl5eV4+umnMXHiRHz++edim//85z/o168fwsLC8Nlnn+Hnn3/GokWL0Lp1a7HN66+/jrfeegvLly/Hnj170KJFC+Tm5qKurk7SdQgRBIH5R/5UV1cjJiYGHefNhyEiQuvTIX6iriZCgpO9rg6/PjcbVVVViI6OVuQzHPeJDgteCfg+4TjfU6dOOZ2vyWSCyeT/Y4MPHTqE7t274/vvv0dmZiYAoKioCPfffz9+++03JCUlNdunqqoK7du3x/r16/HQQw8BAA4fPoxu3bqhtLQUffr0wWeffYb/+Z//wZkzZxAfHw8AWL58OZ577jmcP38e4eHheO6557B9+3YcPHhQPPaIESNw+fJlFBUVAQBmzJiBb7/9Fv/6179cnr8gCEhKSsIzzzyDv/71r+L5xcfHY82aNRgxYoTP14L+LCWqoUoIIURpclZsU1JSEBMTI74KCwsDOrfS0lLExsaKwQMAcnJyYDAYsGfPHpf7lJWVwWq1IicnR9yWlpaGG2+8EaWlpeJxe/ToIQYPAMjNzUV1dTV++uknsU3jYzjaOI4BAFu3bkVmZiYefvhhxMXFoVevXli5cqX4/smTJ2E2m52OExMTg6ysLKfj+IK7AaeEf3IHEKqoECCwpzI3JfUhi3J+th7Z7XxeH1eVj0CYzWbExcU5bQsNDUWbNm1gNpvd7hMeHo7Y2Fin7fHx8eI+ZrPZKXg43ne856lNdXU1rl27hsjISJw4cQLvvvsuCgoKMGvWLHz//fd46qmnEB4ejrFjx4rHcnUcd+fvDoUPwj1/wgwFFn1Q6qZPYYIAQHR0tE/dRDNmzMCCBQs8tjl06JBcp6UYu92OzMxMzJ8/HwDQq1cvHDx4EMuXL8fYsWNl/SyuwofBAhgb/UFiCyyEkiBmuGagAMIpCgaENc888wzGjRvnsU2nTp2QkJCAc+fOOW1vaGjApUuXkJCQ4HK/hIQE1NfX4/Lly07Vj8rKSnGfhIQE7N2712k/x2yYxm2azpCprKxEdHQ0IiMjAQCJiYno3r27U5tu3brhn//8p9OxKisrkZiY6HSc9PR0j19/U1yFj6aMFq3PgAIQzyiABI6CACFA+/bt0b59e6/tsrOzcfnyZZSVlSEjIwMAsGvXLtjtdmRlZbncJyMjA2FhYSguLsawYcMAAEeOHEFFRQWys7PF47766qs4d+6c2K2zc+dOREdHi2EiOzsbO3bscDr2zp07xWMAQL9+/XDkyBGnNkePHkWHDh0AAB07dkRCQgKKi4vFsFFdXY09e/bg8ccf9/r1N0aj/wJktLh+ET7QAFj/UfAgRJpu3bph0KBBmDRpEvbu3Ytvv/0W+fn5GDFihDjT5fTp00hLSxMrGTExMcjLy0NBQQG+/PJLlJWVYfz48cjOzkafPn0AAAMHDkT37t3x6KOP4ocffsDnn3+O2bNnY8qUKeI4lcceewwnTpzA9OnTcfjwYSxbtgwfffQRpk2bJp7ftGnT8N1332H+/Pk4duwY1q9fjxUrVmDKlCkAgJCQEDz99NN45ZVXsHXrVvz4448YM2YMkpKSMHToUEnXguvKB8u0CiBUiZGOKiDSUfAgxD/r1q1Dfn4+BgwYAIPBgGHDhuGtt94S37darThy5Ahqa2vFbW+88YbY1mKxIDc3F8uWLRPfNxqN2LZtGx5//HFkZ2ejRYsWGDt2LF566SWxTceOHbF9+3ZMmzYNb775Jm644Qa89957yM3NFdvccccd+OSTTzBz5ky89NJL6NixI5YsWYLRo0eLbaZPn46amhpMnjwZly9fxp133omioiJESJzezNU6H51nzoeR1vnwGQURaSiAeEehg/jDfq0Op6a9oMo6H3KsB2Wvq8PJebMUPd9gR5UPHWtafaEw4hlVQDyj4EEIkQuFjyDSOIxQEHGNAohrFDxkEmHT+gy0IQTp103covARpKgq4h4FEGcUPCQK1oBBiARchQ+jBTB6eN9Gw0H8puQAWR6DDQWQ63QXPCgYEMIErsKHN0ZpD9WTBQUe7xzBhrcQwmMA0V1YkAMFjqDSdDFKv9ByCYrTVfjQgrvAQ6GkOR5DCI8BhIACByGMo/ChEAol7vEWQiiAcIICByHcoPChMgolv+MphFAAYRQFDkK4ROGDEXKOV+EtyPASQiiAMIRCh6qMJt+ut83iaUoAIb+j8KFDTYMML2GEhxBCAUQGFBw88vVGzyJ35x5i4/drIsqg8BEEGocRHoII6yGEAkgAKHiIeA4ZhASKq/ARWgcYJcwkbODgRqs2nqoiLIcQCiB+CNLgQSGDkOa4Ch9ShSq87ocewg0PVRFWQwgFEAmCIHhQyCDEd7oOH0pzF254DSWsV0VYDCEUQHzAUfCgAEGIOih8KEAvoYTVqghrD8ijAOIBB8GDAgch6qPwoSKeQwmrVRFWqiEUQFxgPHhQ6CBEOxQ+GOBpbAqrwUTKuiRqBBUWQggFkEYYDR4UOAJniqiXvI/NblXgTFzz9gBSn9CzXRRH4YNxUgbNsh5UgiGEBH0AYTB06DVw+BMCCGEFV+HDaAHAQB8/q5oGFdbCiBYhxEHNMBK0AYSh4KGnwEEhg+gRV+EDCGwZclbGKaiF1TCiZggRP1PlMBJ0AYSR4MFr6KCAQYKNwZ+dli5ditTUVERERCArKwt79+71ab8NGzYgJCQEQ4cO9edjA2asC/zFs9C6318s0PKaNg0jSjBc8+vHiz8aBw+jySa+WGeKqHf5IiTYSP7tuHHjRhQUFGDu3LnYv38/evbsidzcXJw7d87jfr/88gv++te/4q677vL7ZFmgl3DSOIhoHUa0unYUQGSgUfBgPXBQyCDEM8m/GRcvXoxJkyZh/Pjx6N69O5YvX46oqCi8//77bvex2WwYPXo0XnzxRXTq1MnrZ1gsFlRXVzu9eMFrKGEhiGhxrSiABEDl4MFi4KCQQYh/JP1WrK+vR1lZGXJycn4/gMGAnJwclJaWut3vpZdeQlxcHPLy8nz6nMLCQsTExIivlJQUKafJJJ6qJVpXRdS+LhRAJIqwqRo8WA4chBD/SBpweuHCBdhsNsTHxzttj4+Px+HDh13u880332DVqlUoLy/3+XNmzpyJgoIC8d/V1dW6CCDuuLrRsjQ41hFA1B6waqxj6zqwzm6S8NRFf2jQxcJC6KCQ4V0Lk+drZLPRNSTOFJ3tcuXKFTz66KNYuXIl2rVr5/N+JpMJJlNwz6llMZA0roKoFUQogPhG0eARhKFDL4HDWyggRCuSwke7du1gNBpRWVnptL2yshIJCQnN2h8/fhy//PILhgwZIm6z269PPwwNDcWRI0fQuXNnf847KLG0xLma1RAKIJ4pXvFQkVahg+ewQQGD8EhS+AgPD0dGRgaKi4vF6bJ2ux3FxcXIz89v1j4tLQ0//vij07bZs2fjypUrePPNNyV3pYTWCTDapf2ibYgIkdSeJyxUR9QKIUoHEKNF++fD+EMvXS1ahA6eAgcFDKI3krtdCgoKMHbsWGRmZqJ3795YsmQJampqMH78eADAmDFjkJycjMLCQkRERODWW2912j82NhYAmm1XSmid9F/OPAcWraoEanTJUAXEmR4qHmqGDtbDBgUMeYTWAcYAfzRC6NkuipMcPoYPH47z589jzpw5MJvNSE9PR1FRkTgItaKiAgYD3yP7fQ0srIYULVYQbUzJaggFEBUpWPVQK3SwGDgoZBDi54DT/Px8l90sAFBSUuJx3zVr1vjzkUxqHFJYDCJ6DSEUQPiteqgROlgJHBQyCHGPu2e7sIrlIKL1zTq0jo8Awsu4D1WCh8xVDwodhJDGKHwowFW3jdaBhIUqCA8BhHW8BY9g6V6hwEGINBQ+VMJKIGk8Q0aLmTGsBxBeqh+KkSl4BEPo0GPgaGlSZqRlQwON4CTOKHxoqGkgUTuMaFEN4SGABC0ZgoeeZ6/wGDaUChOEBIqr8GG0CAj97zofWndjKEGr6ojaIYT1ABKU1Q8KHi6xHjgoXBBecTsnNrRO8PrSAzW/LjUfdqfEA+tYfEhfY8w+XI6j4KH0A91amOqdXqxoabK4fBH+XLp0CaNHj0Z0dDRiY2ORl5eHq1evetynrq4OU6ZMQdu2bdGyZUsMGzas2UrjTz31FDIyMmAymZCent7sGEeOHME999yD+Ph4REREoFOnTpg9ezasVqvYZvPmzcjMzERsbCxatGiB9PR0/P3vf3c6zrhx4xASEuL0GjRokOTrwFXlQyp/b9SsV1XUqJCoUQ1huQISlNUPxikVOlgJGRQmgsPo0aNx9uxZ7Ny5E1arFePHj8fkyZOxfv16t/tMmzYN27dvx6ZNmxATE4P8/Hw8+OCD+Pbbb53aTZgwAXv27MG///3vZscICwvDmDFjcPvttyM2NhY//PADJk2aBLvdjvnz5wMA2rRpg+effx5paWkIDw/Htm3bMH78eMTFxSE3N1c81qBBg7B69Wrx3/48i03X4cNfPC4y5jhn3kIIywEkKHBQ9ZA7dFDYIFo5dOgQioqK8P333yMzMxMA8Pbbb+P+++/HwoULkZSU1GyfqqoqrFq1CuvXr8e9994LAFi9ejW6deuG7777Dn369AEAvPXWWwCA8+fPuwwfnTp1QqdOncR/d+jQASUlJfjXv/4lbrv77rud9pk6dSrWrl2Lb775xil8mEwml89zk4LROjAfWOziUep8lOySCcYuGCZo8LRaKeTuYmGhK4W6TPhSXV3t9LJYAvv/VlpaitjYWDF4AEBOTg4MBgP27Nnjcp+ysjJYrVbk5OSI29LS0nDjjTeitLTU73M5duwYioqK0L9/f5fvC4KA4uJiHDlyBH/4wx+c3ispKUFcXBxuvvlmPP7447h48aLkz6fKh0xYW2RMqUoIoEw1hMUKiK67XhieUqtE94qWoYOChrqMlsCf7YL/frs0ffjp3LlzMW/ePL8PazabERcX57QtNDQUbdq0gdlsdrtPeHi4+Fw0h/j4eLf7eNK3b1/s378fFosFkydPxksvveT0flVVFZKTk2GxWGA0GrFs2TL88Y9/FN8fNGgQHnzwQXTs2BHHjx/HrFmzcN9996G0tBRGo9Hn86DwoQCWgkhonaDYOcgdQlgMILrEaMVDT2M6gi1wxJo8lxqtDWx0dUl16tQpREdHi/92N7ZhxowZWLBggcdjHTp0SNZz89fGjRtx5coV/PDDD3j22WexcOFCTJ8+XXy/VatWKC8vx9WrV1FcXIyCggJ06tRJ7JIZMWKE2LZHjx647bbb0LlzZ5SUlGDAgAE+nweFD4VpvZZH43NQOoQAgd/oWQsguqt+MLh6qV5CB8+Bw1t4CFbR0dFO4cOdZ555BuPGjfPYplOnTkhISMC5c+ectjc0NODSpUtux1AkJCSgvr4ely9fdqp+VFZW+jXuwlHN6d69O2w2GyZPnoxnnnlGrFoYDAZ06dIFAJCeno5Dhw6hsLCw2XiQxl9Xu3btcOzYMQofLNOyKqJ0CAHkqYawFkB0g7GKh9JTZtXCauigQKGe9u3bo3379l7bZWdn4/LlyygrK0NGRgYAYNeuXbDb7cjKynK5T0ZGBsLCwlBcXIxhw4YBuD5ttqKiAtnZ2QGdt91uh9Vqhd1ud9tlYrfbPY51+e2333Dx4kUkJiZK+myuwkdorR2hYXaX7zVE8Td2VqsgwkMIYSmA6KL6wdCD4pReIEzp4MFS2KCAwZdu3bph0KBBmDRpEpYvXw6r1Yr8/HyMGDFCnOly+vRpDBgwAB988AF69+6NmJgY5OXloaCgAG3atEF0dDSefPJJZGdnizNdgOsDSK9evQqz2Yxr166hvLwcwPUKR3h4ONatW4ewsDD06NEDJpMJ+/btw8yZMzF8+HCEhYUBAAoLC5GZmYnOnTvDYrFgx44d+Pvf/453330XAHD16lW8+OKLGDZsGBISEnD8+HFMnz4dXbp0cZoN4wuuwocnobWuQ4kD6+FEiyCi5HgQh0AqDiwFEK4xVPHgNXhoHTgoZOjHunXrkJ+fjwEDBsBgMGDYsGHiNFkAsFqtOHLkCGpra8Vtb7zxhtjWYrEgNzcXy5YtczruxIkT8dVXX4n/7tWrFwDg5MmTSE1NRWhoKBYsWICjR49CEAR06NAB+fn5mDZtmrhPTU0NnnjiCfz222+IjIxEWloaPvzwQwwfPhwAYDQa8e9//xtr167F5cuXkZSUhIEDB+Lll1+WvNZHiCAI7MwTdaO6uhoxMTHoc/9LCA1T/s7BYlBROiQoffxAbvhyBxDAv/ORo/phj/Qckr3u789TbYOg6iF36KCwIS9rTT22576Hqqoqn8ZQ+MNxn+iRNx/G8MB+adjq6/DjqlmKnm+w003lQ07eqigOaoYUpVc1VWNQKksVEKI+HoKHlqFDb4GDEE8ofASgcUjRolqixEwapafmshJA/DkXXYz9CJCaD4/zhVzBQ6vQQYGDBCsKHzLROogA8lUvlF6gjJUAQtTD8sJhagcPChyEUPhQhNZBhPUQwkoAoeqHNKxUPXjtZtFD6GhrqvFrv3orn4uMEeVQ+FCYlkGE5RDCSgAhypP7+SxyUSN48BI4/A0VhPiLq/ARWmcHwrQ+C/81HciqVhiRaxyH3ONBWAggVP3wjdZVD55CByuBI1gDRWidAKM9sEmcIfXMTwLlHlfhAwBCr3mfidIQyd5UWVfUrIqwWgVhIYAQ5QRa9eBlCq1WgSNYAwbhH3fhwxc8BhS1ggiLIUTrAELVD8+0qHoosViY3MGDAgch/tNl+PCFt4CiZThRI4iw1hWjdQAh8lN6NVMp5AoeFDgIkUfQhg9vWKmeKBlEWKuCaBlA1Kp+GK4ZAl7lVE28Vz3krHaoHTwocBA9o/ARgMYBhecgwlIIoQqIPrBQ9eA1eFDoIMGAwodM9BBEWAkhWgUQqn4447nqwVs3i94CR5sw56/HEmbV6EwIqyh8KID3ICLneBDAvxBCFRBtsfjwOF/wVO3gMXA0DRWE+IvCh8K0CiKBhhA5Z7P4G2a0CCBU/QhMIMEj0KoHD8GD1cBBoYKojavwYbzWAGNoAwDAFsnVqQNQN4iE1tqZqoL4K5AAojd2k34XPmIleOgxdFCwICzi7w7+X8ZrDW7f4yGYOIKIkiGEpQCiRYih7hcAEdK7T/ztctGiu4XlaodWgYPCBuEB+3dpP7gLJiyGEqVDCEsBxF9U/WCfFt0trAYPLUIHBQ7CG/buxgpiuVqiZAhhJYBQ9cN/LHe58Bw8eA4dFDhcM1oEhAb4bBdY2f1504ugCh+esFItUSqEsBJA/EXVDz+o2OXiDyWWUJdCruBBoYMQ6Sh8eKFVKFEihLAQQLQewErk5W/VI5DgEWjVg8dqBwUOojdsPV2NI8ZrDR67ceQSes3u01LvPh+vVp5jOabi8iCUjSec+82vLhc/qh5S8TjAlLfg0SashoIH0SWqfATIEUB4qoRoXQHRYt0P4p1aXS5ajfPgpZtFb2GjXdhV1IUp/4ca4QuFD5moHUKAwIIIrwHEX3oZeMoKLbpb/MVDtYPXwNEu7KrWp0A4ReFDZmqFECDwaojWAcQfVP3wAaMDTbUY58F6tYP10EHhgiiFwodCjNcaVJspE0gI0TKAsFr9oIAjPwoev2MpcFC4IFrhKnwYrzXAaLz+dERbVJjGZ+OdmlUQ4HoI4S2A+IPCgbbUHGiqp+ChZeigkEFYw1X4aMxY2/wRzawGErW7YngKIDT1VmYMdrmoPc5DjuDBc7WDggbhAbfhwxVXgQRgJ5SoOSiVAoi7z6KBp2rScj0Pf/Fa7aDQQXiiq/DhDmuhRI0QwlsAkUrprhfq2gkcb+M8eKx2UOAgvAqK8OGOu1ACqBNMlA4hPAUQqn7IQIUuFzXGe/AePJQOHRQ4PAuttSM0LMDFFK3yLexIXAvq8OGJmtUSJUMITwFEKqpO6A8FD/codBA9ofAhUeNQIncQabxcu5xBROsA4vPnMTb4lMKN/7R+aJwveAgdFDiIXlH4CIAjiPBQDdEygFD1g02sdbmoWfWQK3hQ6HCtXegVp39fC6Xl1YkzCh8y4CWE8FABUav6odtxH5zSej0PfygRPHgIHU2DBSH+oPAhIx5CiFYBhNXBp0R+Urtc1J5SK0fVQ+7gwWLooJBBlEThQwGshxAeAogUSna9sNCtY7CEwG4SZD0mK10ugQQPLbpb9FjtoJBBtMBV+DDUWGAwXv9vewuTtifjA5ZDCOtdMGpUP+TsejFaABv735K6oYfgoUXooKBBWKHeVAaZGWoszV6s8rSeSMDHvtbgNEtGCscD6STvV+v/HPjQOnn/gncwatf1zx0ll1OX0uXCyziPNmE1sgaPdmFXFQ8e7UKvuHwR7V26dAmjR49GdHQ0YmNjkZeXh6tXPX8/1NXVYcqUKWjbti1atmyJYcOGobKyUnz/hx9+wMiRI5GSkoLIyEh069YNb775ZrPjWCwWPP/88+jQoQNMJhNSU1Px/vvvi+//9NNPGDZsGFJTUxESEoIlS5Y0O8bXX3+NIUOGICkpCSEhIdiyZYtf14Gryoc37gIIC1USJasggP+VEJYrIKyM/WCh64UExt+qB0/VDgoXfBg9ejTOnj2LnTt3wmq1Yvz48Zg8eTLWr1/vdp9p06Zh+/bt2LRpE2JiYpCfn48HH3wQ3377LQCgrKwMcXFx+PDDD5GSkoLdu3dj8uTJMBqNyM/PF4/zyCOPoLKyEqtWrUKXLl1w9uxZ2O2//zFZW1uLTp064eGHH8a0adNcnktNTQ169uyJCRMm4MEHH/T7OugqfLjjKpRoFUjUCCGsBxClBp9SSFCGEuM9eJhWK3e1QykUOvhx6NAhFBUV4fvvv0dmZiYA4O2338b999+PhQsXIikpqdk+VVVVWLVqFdavX497770XALB69Wp069YN3333Hfr06YMJEyY47dOpUyeUlpZi8+bNYvgoKirCV199hRMnTqBNmzYAgNTUVKf97rjjDtxxxx0AgBkzZrj8Gu677z7cd999/l+E/+K22yVQWnfbGGutinXH+NMNo3YXjFLdL1KFyljBN7Lb86cLana3sB48qCtFHdXV1U4viyWwH/LS0lLExsaKwQMAcnJyYDAYsGfPHpf7lJWVwWq1IicnR9yWlpaGG2+8EaWlpW4/q6qqSgwZALB161ZkZmbi9ddfR3JyMrp27Yq//vWvuHbtWkBfk7+CovLhK08BRKlKiVKVEB4qID4fn5HuF+KZkqua+hs8/Kl6yBU8lAodxLPQOjtCGwJ8Nst/909JSXHaPHfuXMybN8/vw5rNZsTFxTltCw0NRZs2bWA2m93uEx4ejtjYWKft8fHxbvfZvXs3Nm7ciO3bt4vbTpw4gW+++QYRERH45JNPcOHCBTzxxBO4ePEiVq9e7ffX5C8KHz5SuuvGWGsNugCiRKhQquslWLt01HiQnFLkfkqtFBQ89OHUqVOIjo4W/20yuf6dP2PGDCxYsMDjsQ4dOiTrublz8OBB/OlPf8LcuXMxcOBAcbvdbkdISAjWrVuHmJgYAMDixYvx0EMPYdmyZYiMjFTl/BwofATAEUjkCiFKVEFYDyA+H1vB6gdNuVWX1PEePI3zCObQkRB22e17taHKzbBSUnR0tFP4cOeZZ57BuHHjPLbp1KkTEhIScO7cOaftDQ0NuHTpEhISElzul5CQgPr6ely+fNmp+lFZWdlsn59//hkDBgzA5MmTMXv2bKf3EhMTkZycLAYPAOjWrRsEQcBvv/2Gm266yevXKScKHzIw1Fhkr4IA8oUQlgMIT9UP4p5SXS48jfOQO3iwHDo8BY1g1L59e7Rv395ru+zsbFy+fBllZWXIyMgAAOzatQt2ux1ZWVku98nIyEBYWBiKi4sxbNgwAMCRI0dQUVGB7Oxssd1PP/2Ee++9F2PHjsWrr77a7Dj9+vXDpk2bcPXqVbRs2RIAcPToURgMBtxwww2Sv+ZABe2AU7kpMWhVzkGpLA9C9XXwqZRBqkqs+0FriaiLl3EeSqzbwUrwSAi77PJF/NOtWzcMGjQIkyZNwt69e/Htt98iPz8fI0aMEGe6nD59Gmlpadi7dy8AICYmBnl5eSgoKMCXX36JsrIyjB8/HtnZ2ejTpw+A610t99xzDwYOHIiCggKYzWaYzWacP39e/OxRo0ahbdu2GD9+PH7++Wd8/fXXePbZZzFhwgSxy6W+vh7l5eUoLy9HfX09Tp8+jfLychw7dkw8ztWrV8U2AHDy5EmUl5ejoqJC0rWgyofM5O6KAeSrhLBcAfH5uDT4VDUsTbH1ldrjPPQSOihQqGfdunXIz8/HgAEDYDAYMGzYMLz11lvi+1arFUeOHEFtba247Y033hDbWiwW5ObmYtmyZeL7H3/8Mc6fP48PP/wQH374obi9Q4cO+OWXXwAALVu2xM6dO/Hkk08iMzMTbdu2xSOPPIJXXnlFbH/mzBn06tVL/PfChQuxcOFC9O/fHyUlJQCAffv24Z577hHbFBQUAADGjh2LNWvW+HwdQgRBkDzncenSpfjb3/4Gs9mMnj174u2330bv3r1dtl25ciU++OADHDx4EMD1EtL8+fPdtneluroaMTExyEl+DKGG/97UW6g7OMZfcs+SkaMrxp8l2f0JIAAkBRBfQ4WU8CGl+8WXcR++Hs+XcR/2SN8rRD492yXCt351X1c4lRI+fO12kRI+1Brr4U/Vg+exHVoEjdorNjzc8zCqqqp8GkPhD8d94s575yE0NLB+14aGOnyza56i5xvsJN9RNm7ciIKCAsydOxf79+9Hz549kZub22wQjUNJSQlGjhyJL7/8EqWlpUhJScHAgQNx+vTpwM685prrF2Pk7o6RoyuG5S4Yn47JyBohgTJc00evpxLjPVgOHnJTar0O6jIhLJP822/x4sWYNGkSxo8fj+7du2P58uWIiopyWh++sXXr1uGJJ55Aeno60tLS8N5778Fut6O4uNjtZ1gslmaLu/iM0UDC2ngQNQOIz8dXIFTIPU4jWMZ98DbFVs3gIWfVQ+7QQSGD8EJS+Kivr0dZWZnTSmsGgwE5OTkeV1prrLa2Flar1WnltaYKCwsRExMjvpou9CIZI4GEtUGpagUQrasfWgQGWu20OaXGe6g5zkOu4KFEtYMCB+GJpPBx4cIF2Gw2xMfHO233tNJaU8899xySkpKcAkxTM2fORFVVlfg6deqUlNP0jYZhRIml3JV8cm5TSlZAtOxSkXOpdTn5NN5DI6x0uUilZXeLUl0shPBE1dkur732GjZs2ICSkhJERLgfEGQymdyuJKeYxgFEhcGscq8NAvi3Sqo/M2BYQTNfpPN1sCkveKt6UPAg5DpJd5127drBaDSisrLSaburldaaWrhwIV577TV88cUXuO2226SfqZqaVkIUCiNKBBC1SJ2Cq/TzXwjblJ5iK4VWs1uom0UdxmsNMIZK71JuTGgIbH/inaS7QXh4ODIyMpwGizoGjzZeaa2p119/HS+//DKKioqcnubHDQW7Z5QYAyJ5Hz/GfhCidZcLL7NbaHwHIc1JrrcXFBRg7NixyMzMRO/evbFkyRLU1NRg/PjxAIAxY8YgOTkZhYWFAIAFCxZgzpw5WL9+PVJTU8WxIS1bthSXeOWKI4DIWA3htQLi7wJkXo9LS65L5+MaH76Qe6YLS1UPfwRS9Qj2bpb2xuvX7qpR2ZlyhD+Sw8fw4cNx/vx5zJkzB2azGenp6SgqKhIHoVZUVMBg+P2G9O6776K+vh4PPfSQ03ECfTSx5mquMRtAWB37QV0v+qHUs1x8xcNiYsEYPBxhgxBv/Lrb5OfnIz8/3+V7jiVYHRxLu+oSwwEkWNCgU/1QcpaL2t0twRQ8KHAQf/A5zYEljAYQtaofSnW9EPlpMdOF5y4Xf6seeh/fQWGDyIHChxx0FECUpmXXi+7HfQQJqV0urDy7xR8sBA8KG0QJFD7kwmgAkYqV6gd1pwQXpbpceO5u0Sp4UNggaqDwIScGAwiL1Q8iM0ZnuijR5aLGomIsdLeoGTwobBAtcNVZL9Sy8ZA4j2ReC0SJpdi9UfqZL3I/60XOJdlZXWKdJVrPdPGVmt0tvAWP9sar4osQLXBX+RBqa8X/DomK0vBMPGCsAkLVD8IyNZ7loiS5gofSoYOCBmEJV5WPpoTaWvHFHM4rIGo98VZtWjzhlgVqz3ThsctFy0GmSgYPqnAQFnFX+XCHyYoIQxUQ1qofvsx6oUGn+sf7QFM5qh5KBI9gDhvGaw0wGgN7yrdgo0dOKI3ryoc7jSsimldFOK+ASMVD9YOQpvypelDwIMR/ugwfTWkeRBgJIFIfOsfTA+fkHHRK2COly0Vq1UOr7ha5gwd1rxCeBEX4aEyzIMJpAPGHr9UPX2a9sBgq9DpuRK5ptr6O9+B5oGmgVQ8lggchPAm68NGY6kGEkQAiBU/VD195Cw803dY9XqbZ+kqr7hY5UfAgPArq8NGYakGEgQDCUvWDKE+LZ7rISckuFy3IWfWg4EF4ReHDBcUHrDIQQKRQsvohZ9cLi100xDUWuly0qHoEW/Bob7CgvcGCdga2B8oT9elmqq2SFJnGq/E0XDWm3tITb4mesBI8WA8d7SloEB9Q+JDIEURkCSEMrQPijT8PnCPeGa4ZYI+kLip3lOpyUXuGi56DB4UN4g+6m/hJqK1lMoBIwdrCY56wtuCYsQ6wRWh9FvKR84FyehRI1UNvwYPCBpED1cQDINt4EBnHgCg5/kOpJdflftCcHGjGS3O+zHTxZZqtr+M99DbQNBBaBw/H2A0KHkQuFD4CxHsAUWPmi5p8HXSq17U6iHdSu1y0rnpoGTwocBClULeLDHjvgpHS/UJjP0gwCcbgwXvYMNRYYDAGeAwb39eAB1T5kAlrFRCWngEjV9cLTaUNHrwPNOUteFC3ClEbhQ8ZsRZApJDS/aLHVU8JacrfqocSD4tTAgUOoiWuwodQUwvhKtsDy1gKIKwNPtUbb+NGjJz8Tud5pgtrA01Zn9lCgYOwgqvw4SBcrRFfLGIpgEih5OBTNbte5OqeoRkv8vNlpouULhcptHp6rVRKBA8KHIQ13I8cbBpAQlq20OhMnLEyCJUWHnNPb2t1KEmuaba88KfLhbVxHhQ2CMu4rHx4wlJVhMcKiN6m3hL9YqnLhaXgQVUOwgPdhY/GWAgiLAQQVsZ+8Nj1QvRBSpeL1KoHa8GDEB7oOnw0pmUQ4S2AUPWDEN+wNLOFggcfLl26hNGjRyM6OhqxsbHIy8vD1auew2ddXR2mTJmCtm3bomXLlhg2bBgqKyvF9y9evIhBgwYhKSkJJpMJKSkpyM/PR3V1tdNx1q1bh549eyIqKgqJiYmYMGECLl686NRmyZIluPnmmxEZGYmUlBRMmzYNdXXOY7VOnz6NP//5z2jbti0iIyPRo0cP7Nu3T9J1CJrw0ZgWQYSFAKKUYJ75opcZL7xhcW0Pf8lR9aDgwY/Ro0fjp59+ws6dO7Ft2zZ8/fXXmDx5ssd9pk2bhk8//RSbNm3CV199hTNnzuDBBx8U3zcYDPjTn/6ErVu34ujRo1izZg2++OILPPbYY2Kbb7/9FmPGjEFeXh5++uknbNq0CXv37sWkSZPENuvXr8eMGTMwd+5cHDp0CKtWrcLGjRsxa9Yssc1//vMf9OvXD2FhYfjss8/w888/Y9GiRWjdurWk68DvaEGZNA4gSg9W1XoQqpTBp0o9dC70mh0NkZ4zb2itHQ1RXtr48KA5OR5GF1oHNNCg1KARyIqm/tD6mS1qaWcM1/oUmHDo0CEUFRXh+++/R2ZmJgDg7bffxv3334+FCxciKSmp2T5VVVVYtWoV1q9fj3vvvRcAsHr1anTr1g3fffcd+vTpg9atW+Pxxx8X9+nQoQOeeOIJ/O1vfxO3lZaWIjU1FU899RQAoGPHjvjLX/6CBQsWiG12796Nfv36YdSoUQCA1NRUjBw5Env27BHbLFiwACkpKVi9erW4rWPHjpKvRVBWPtxRoxqi1wpIMFc/iHyUmmarZ6xXPdpzHDyqq6udXhZLYNe6tLQUsbGxYvAAgJycHBgMBqcbfGNlZWWwWq3IyckRt6WlpeHGG29EaWmpy33OnDmDzZs3o3///uK27OxsnDp1Cjt27IAgCKisrMTHH3+M+++/X2zTt29flJWVYe/evQCAEydOYMeOHU5ttm7diszMTDz88MOIi4tDr169sHLlSsnXgsKHCyzMlCGEJ3I+zZYXLI33YJUmwaP22vU/zgJ51V7/4y4lJQUxMTHiq7CwMKBTM5vNiIuLc9oWGhqKNm3awGw2u90nPDwcsbGxTtvj4+Ob7TNy5EhERUUhOTkZ0dHReO+998T3+vXrh3Xr1mH48OEIDw9HQkICYmJisHTpUrHNqFGj8NJLL+HOO+9EWFgYOnfujLvvvtup2+XEiRN49913cdNNN+Hzzz/H448/jqeeegpr166VdC0ofLihZADhpfoRDANP6em23vmyxgeRTs9dLu2N4VxXPBxOnTqFqqoq8TVz5kyX7WbMmIGQkBCPr8OHDyt+vm+88Qb279+P//3f/8Xx48dRUFAgvvfzzz9j6tSpmDNnDsrKylBUVIRffvnFaVxISUkJ5s+fj2XLlmH//v3YvHkztm/fjpdffllsY7fbcfvtt2P+/Pno1asXJk+ejEmTJmH58uWSzjXox3x4IlytYWbRMkLIdXoabBooFrtc9BA6HKKjoxEdHe213TPPPINx48Z5bNOpUyckJCTg3LlzTtsbGhpw6dIlJCQkuNwvISEB9fX1uHz5slP1o7Kystk+CQkJSEhIQFpaGtq0aYO77roLL7zwAhITE1FYWIh+/frh2WefBQDcdtttaNGiBe666y688sorSExMxAsvvIBHH30UEydOBAD06NEDNTU1mDx5Mp5//nkYDAYkJiaie/fuTp/brVs3/POf//R6nRqj8OGF3gKIkiueEkKCm56ChxTt27dH+/btvbbLzs7G5cuXUVZWhoyMDADArl27YLfbkZWV5XKfjIwMhIWFobi4GMOGDQMAHDlyBBUVFcjOznb7WXb79TWTHONUamtrERrqfMs3Go0AAEEQxDYGg8Fjm379+uHIkSNObY4ePYoOHTp4+eqdUfjwgRIBROuZL4QQZ2rOdNHb9NpgDR1SdevWDYMGDRK7KaxWK/Lz8zFixAhxpsvp06cxYMAAfPDBB+jduzdiYmKQl5eHgoICtGnTBtHR0XjyySeRnZ2NPn36AAB27NiByspK3HHHHWjZsiV++uknPPvss+jXrx9SU1MBAEOGDMGkSZPw7rvvIjc3F2fPnsXTTz+N3r17i589ZMgQLF68GL169UJWVhaOHTuGF154AUOGDBFDyLRp09C3b1/Mnz8fjzzyCPbu3YsVK1ZgxYoVkq4FhQ8f6a0CogTen/VCiK9osOnvKHhIs27dOuTn52PAgAEwGAwYNmwY3nrrLfF9q9WKI0eOoLbR2MA33nhDbGuxWJCbm4tly5aJ70dGRmLlypWYNm0aLBYLUlJS8OCDD2LGjBlim3HjxuHKlSt455138MwzzyA2Nhb33nuv01Tb2bNnIyQkBLNnz8bp06fRvn17DBkyBK+++qrY5o477sAnn3yCmTNn4qWXXkLHjh2xZMkSjB49WtJ1CBEctRSGVVdXIyYmBveGP4zQEPnXnpBCzgAiS+UDkFz5kNLtInWtD1/Ch7d1PgB4XecDgE9rePjSxtvD5byt8+Ftf5uXy22P9LykvN3k5Uc0wubxbaPJ8/umCM8DSuV6qJxcT7RVasyHlMpHIOFDT0up+xo8qq/YkXTzb6iqqvJpDIU/HPeJnOTHEGoIrGu5wW7BF6eXK3q+wY5mu0gUbNNwlZjx4sszXggh7mkdPPQym4Voh8KHH+QKILxMudUCPWCOBErvM120QqGDyIHCByEk6Kk12JT3tT0oeBC5UPjwE8/dL1KecEuIHOQa70G06XKhbhYiN5qaoDG9TbmlGS9ESVIGmyol2Ga68BY6hNprEAyeB1l7PYadVvVVGlU+CCGEuMRb8CD8oD9RA0BrfxBCfMXTwmIUOojSqPLBANlmvSgkGB4wR/znyxofhB8UPIgaKHzoiYQpt1oPOqW1Pggr1FxWnRByHYWPAPE864UQvVJqjQ8tB5uq0eVCVQ+iFgofjGC964UQ4j8e1veg4EHUROGDyM54rUHrU5DE6H0JCkI0pXTVg4IHURuFDxlQ14syfFlinXjm7aFyhBCiBQofDJGl60WhQac044UoSe6n2RLfUdWDaIHCB9E9ergccUeNmS6BjvdQssuFggfRCoUPmVDXizbUCBahOh4TYoqgZaR9pbdl1Sl4EC3RCqeEEEJ0Q6iphRASWDexIFA3s9Ko8sEYvUy59WXGCy00Rog2qOpBtEbhQ0bMdL1IGHRK9M1mMWp9CqpTaoExf7A43oOCB2EBhY8gRzNeAkfrhBBeUPAgrKDwwSC9dL3Igdb60FYLU+ADUmNN/KczvQ02JURrFD4IISQIUNWDsITCh8yYGfdBCNEUS+M9KHgQ1lD4YBR1veiHUfmHkRI/qLHAGAsoeBAW+RU+li5ditTUVERERCArKwt79+712H7Tpk1IS0tDREQEevTogR07dvh1skQCBpZZV+sBc7SCqf7R0uqE6Ivk8LFx40YUFBRg7ty52L9/P3r27Inc3FycO3fOZfvdu3dj5MiRyMvLw4EDBzB06FAMHToUBw8eDPjkWUVdL4QQFlDVg7BKcvhYvHgxJk2ahPHjx6N79+5Yvnw5oqKi8P7777ts/+abb2LQoEF49tln0a1bN7z88su4/fbb8c477wR88noXDF0vrCw0RtNliZxYGO9BwYOwTNLy6vX19SgrK8PMmTPFbQaDATk5OSgtLXW5T2lpKQoKCpy25ebmYsuWLW4/x2KxwGL5/YevqqoKANDA0ZK3IXZ5npkRYg9gkSib79nSLuHJ67YG3xvbGrx/izVYPZ+nt/cBwGYI8fh+SD3QEOG5jc3D257eAwBvV8TuqWeoDrBHug9hdo87AxA8f3qIzf37Nrvnnymbzfv3cUOD5xultcH7MeqtnttYwnz72W8bVoM6H+/b10J97xasDZXwAwLgqjGwUB1hCGz/dsZwVIONYA8AV65ePxdBUL6LtAFWIMCPaQA/9xpeSQofFy5cgM1mQ3x8vNP2+Ph4HD582OU+ZrPZZXuz2ez2cwoLC/Hiiy822/61dYuU09WWXM/r+o9MxyGEEI1dvHgRMTExihw7PDwcCQkJ+Nq8RZbjJSQkIDycqkdKYfLBcjNnznSqlly+fBkdOnRARUWFYt+4elBdXY2UlBScOnUK0dHRWp8Os+g6eUfXyDd0nXxTVVWFG2+8EW3atFHsMyIiInDy5EnU18vzl194eDgiIiJkORZpTlL4aNeuHYxGIyorK522V1ZWIiEhweU+CQkJktoDgMlkgslkarY9JiaGfsB9EB0dTdfJB3SdvKNr5Bu6Tr4xGJRd3SEiIoICAyckfSeEh4cjIyMDxcXF4ja73Y7i4mJkZ2e73Cc7O9upPQDs3LnTbXtCCCGE6JvkbpeCggKMHTsWmZmZ6N27N5YsWYKamhqMHz8eADBmzBgkJyejsLAQADB16lT0798fixYtwuDBg7Fhwwbs27cPK1askPcrIYQQQggXJIeP4cOH4/z585gzZw7MZjPS09NRVFQkDiqtqKhwKq317dsX69evx+zZszFr1izcdNNN2LJlC2699VafP9NkMmHu3Lkuu2LI7+g6+Yauk3d0jXxD18k3dJ1IUyGCGnOfCCGEEEL+i57tQgghhBBVUfgghBBCiKoofBBCCCFEVRQ+CCGEEKIqCh+EEEIIURUz4WPp0qVITU1FREQEsrKysHfvXo/tN23ahLS0NERERKBHjx7YsWOHSmeqLSnXaeXKlbjrrrvQunVrtG7dGjk5OV6vqx5I/V5y2LBhA0JCQjB06FBlT5ARUq/T5cuXMWXKFCQmJsJkMqFr165B8XMn9TotWbIEN998MyIjI5GSkoJp06ahrk7fj03++uuvMWTIECQlJSEkJMTjg0MdSkpKcPvtt8NkMqFLly5Ys2aN4udJGCIwYMOGDUJ4eLjw/vvvCz/99JMwadIkITY2VqisrHTZ/ttvvxWMRqPw+uuvCz///LMwe/ZsISwsTPjxxx9VPnN1Sb1Oo0aNEpYuXSocOHBAOHTokDBu3DghJiZG+O2331Q+c/VIvUYOJ0+eFJKTk4W77rpL+NOf/qTOyWpI6nWyWCxCZmamcP/99wvffPONcPLkSaGkpEQoLy9X+czVJfU6rVu3TjCZTMK6deuEkydPCp9//rmQmJgoTJs2TeUzV9eOHTuE559/Xti8ebMAQPjkk088tj9x4oQQFRUlFBQUCD///LPw9ttvC0ajUSgqKlLnhInmmAgfvXv3FqZMmSL+22azCUlJSUJhYaHL9o888ogwePBgp21ZWVnCX/7yF0XPU2tSr1NTDQ0NQqtWrYS1a9cqdYqa8+caNTQ0CH379hXee+89YezYsUERPqRep3fffVfo1KmTUF9fr9YpMkHqdZoyZYpw7733Om0rKCgQ+vXrp+h5ssSX8DF9+nThlltucdo2fPhwITc3V8EzIyzRvNulvr4eZWVlyMnJEbcZDAbk5OSgtLTU5T6lpaVO7QEgNzfXbXs98Oc6NVVbWwur1arokyW15O81eumllxAXF4e8vDw1TlNz/lynrVu3Ijs7G1OmTEF8fDxuvfVWzJ8/HzabTa3TVp0/16lv374oKysTu2ZOnDiBHTt24P7771flnHkRjL/DiTPJy6vL7cKFC7DZbOLy7A7x8fE4fPiwy33MZrPL9mazWbHz1Jo/16mp5557DklJSc1+6PXCn2v0zTffYNWqVSgvL1fhDNngz3U6ceIEdu3ahdGjR2PHjh04duwYnnjiCVitVsydO1eN01adP9dp1KhRuHDhAu68804IgoCGhgY89thjmDVrlhqnzA13v8Orq6tx7do1REZGanRmRC2aVz6IOl577TVs2LABn3zyCT1y+r+uXLmCRx99FCtXrkS7du20Ph2m2e12xMXFYcWKFcjIyMDw4cPx/PPPY/ny5VqfGlNKSkowf/58LFu2DPv378fmzZuxfft2vPzyy1qfGiFM0bzy0a5dOxiNRlRWVjptr6ysREJCgst9EhISJLXXA3+uk8PChQvx2muv4YsvvsBtt92m5GlqSuo1On78OH755RcMGTJE3Ga32wEAoaGhOHLkCDp37qzsSWvAn++lxMREhIWFwWg0itu6desGs9mM+vp6hIeHK3rOWvDnOr3wwgt49NFHMXHiRABAjx49UFNTg8mTJ+P55593euhmMHP3Ozw6OpqqHkFC85+E8PBwZGRkoLi4WNxmt9tRXFyM7Oxsl/tkZ2c7tQeAnTt3um2vB/5cJwB4/fXX8fLLL6OoqAiZmZlqnKpmpF6jtLQ0/PjjjygvLxdfDzzwAO655x6Ul5cjJSVFzdNXjT/fS/369cOxY8fEcAYAR48eRWJioi6DB+DfdaqtrW0WMByBTaBneIqC8Xc4aULrEa+CcH06m8lkEtasWSP8/PPPwuTJk4XY2FjBbDYLgiAIjz76qDBjxgyx/bfffiuEhoYKCxcuFA4dOiTMnTs3aKbaSrlOr732mhAeHi58/PHHwtmzZ8XXlStXtPoSFCf1GjUVLLNdpF6niooKoVWrVkJ+fr5w5MgRYdu2bUJcXJzwyiuvaPUlqELqdZo7d67QqlUr4R//+Idw4sQJ4f/7//4/oXPnzsIjjzyi1ZegiitXrggHDhwQDhw4IAAQFi9eLBw4cED49ddfBUEQhBkzZgiPPvqo2N4x1fbZZ58VDh06JCxdupSm2gYZJsKHIAjC22+/Ldx4441CeHi40Lt3b+G7774T3+vfv78wduxYp/YfffSR0LVrVyE8PFy45ZZbhO3bt6t8xtqQcp06dOggAGj2mjt3rvonriKp30uNBUv4EATp12n37t1CVlaWYDKZhE6dOgmvvvqq0NDQoPJZq0/KdbJarcK8efOEzp07CxEREUJKSorwxBNPCP/5z3/UP3EVffnlly5/1ziuzdixY4X+/fs32yc9PV0IDw8XOnXqJKxevVr18ybaCREEqgUSQgghRD2aj/kghBBCSHCh8EEIIYQQVVH4IIQQQoiqKHwQQgghRFUUPgghhBCiKgofhBBCCFEVhQ9CCCGEqIrCByGEEEJUReGDEEIIIaqi8EEIIYQQVVH4IIQQQoiq/n9PR+QKZDZpBAAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test, trial = TestFunction(space), TrialFunction(space)\n", + "\n", + "\n", + "def forward(m):\n", + " k = Constant(assemble(m * dx) / assemble(Constant(1.0) * dx(mesh)))\n", + " m_tilde = Function(m.function_space(), name=\"m_tilde\").interpolate(m - k)\n", + " \n", + " u = Function(space, name=\"u\")\n", + " nullspace = VectorSpaceBasis(comm=u.comm, constant=True)\n", + "\n", + " solve(inner(grad(trial), grad(test)) * dx == -inner(m_tilde, test) * dx,\n", + " u, nullspace=nullspace, transpose_nullspace=nullspace,\n", + " solver_parameters={\"ksp_type\": \"cg\", \"pc_type\": \"hypre\",\n", + " \"pc_hypre_type\": \"boomeramg\",\n", + " \"ksp_rtol\": 1.0e-10, \"ksp_atol\": 1.0e-16})\n", + "\n", + " J = assemble(0.5 * inner(u, u) * dx)\n", + " return u, J\n", + "\n", + "\n", + "m = Function(space, name=\"m\").interpolate(cos(pi * X[0]) * cos(pi * X[1]))\n", + "\n", + "u, J = forward(m)\n", + "\n", + "\n", + "def plot_output(u, title):\n", + " r = (u.dat.data_ro.min(), u.dat.data_ro.max())\n", + " eps = (r[1] - r[0]) * 1.0e-12\n", + " p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))\n", + " plt.gca().set_title(title)\n", + " plt.colorbar(p)\n", + " plt.gca().set_aspect(1.0)\n", + "\n", + "\n", + "plot_output(u, title=\"u\")" + ] + }, + { + "cell_type": "markdown", + "id": "cc5e0b3d-426c-4b75-8b74-d46037c5e822", + "metadata": {}, + "source": [ + "## Adding tlm_adjoint\n", + "\n", + "Next we add the use of tlm_adjoint, which requires some minor modification of the forward code.\n", + "\n", + "`compute_gradient` is used to compute the derivative of $\\frac{1}{2} \\int_\\Omega u^2$ with respect to $m$, subject to the contraint that the forward problem is solved. The derivative is visualized using a Riesz map defined by the $L^2$ inner product. A Taylor remainder convergence test is used to verify the derivative.\n", + "\n", + "Note that, given an arbitrary $m \\in V$, we define\n", + "\n", + "$$\\tilde{m} = m - \\frac{\\int_\\Omega m}{\\int_\\Omega 1},$$\n", + "\n", + "so that $\\int_\\Omega \\tilde{m} = 0$, and use $\\tilde{m}$ in place of $m$ when solving the discrete Poisson problem." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "fa48c271-64ab-4104-8d8d-2ebc46adb7f0", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:47:35.096907Z", + "iopub.status.busy": "2023-12-20T16:47:35.096611Z", + "iopub.status.idle": "2023-12-20T16:47:38.455972Z", + "shell.execute_reply": "2023-12-20T16:47:38.455241Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [6.16667546e-08 3.06309860e-08 1.52648952e-08 7.61979814e-09\n", + " 3.80673670e-09]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [1.00950111 1.00477412 1.002393 1.00119799]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [8.09565105e-10 2.02391273e-10 5.05978163e-11 1.26494529e-11\n", + " 3.16236247e-12]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [2.00000003 2.00000005 2.00000013 2.00000034]\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGzCAYAAACPa3XZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABy00lEQVR4nO3de1iUdf4//iczwAAq4IljJB4ytEwMEtFaK1mx/Nj6zcrT5gl1KymTNlMztZNkq2YHzUsztU1XszU/pkY/w6gtSROlzfLw8VCYOnhaQUGGYeb+/eHOHQNzumfuw/t9z+txXXNdec/7vufmDrifvN6HO0QQBAGEEEIIISoxaH0ChBBCCAkuFD4IIYQQoioKH4QQQghRFYUPQgghhKiKwgchhBBCVEXhgxBCCCGqovBBCCGEEFVR+CCEEEKIqih8EEIIIURVFD4IIYQQoioKH4QQQghRFYUPQgghhKiKwgchhBBCVEXhgxCNjBs3Dqmpqc22z5s3DyEhIeqfECGEqITCByGEEEJUReGDEEIIIaqi8EEIIYQQVVH4IIQQQoiqKHwQohF3g0ptNpvKZ0IIIeqi8EGIRlq3bo3Lly832/7rr7+qfzKEEKIiCh+EaKRz586oqqrCv//9b3Hb2bNn8cknn2h4VoQQorwQQRAErU+CkGB08eJFdOjQAfHx8XjqqadQW1uLd999F+3bt8f+/ftBP5qEEL2iygchGmnbti0++eQTREVFYfr06Vi7di0KCwsxZMgQrU+NEEIURZUPQgghhKiKKh+EEEIIURWFD0IIIYSoisIHIYQQQlQlOXx8/fXXGDJkCJKSkhASEoItW7Z43aekpAS33347TCYTunTpgjVr1vhxqoQQQgibli5ditTUVERERCArKwt79+712H7Tpk1IS0tDREQEevTogR07dji9P27cOISEhDi9Bg0a5NTm0qVLGD16NKKjoxEbG4u8vDxcvXrV5ecdO3YMrVq1QmxsrNP2lStX4q677kLr1q3RunVr5OTkeD13OUgOHzU1NejZsyeWLl3qU/uTJ09i8ODBuOeee1BeXo6nn34aEydOxOeffy75ZAkhhBDWbNy4EQUFBZg7dy7279+Pnj17Ijc3F+fOnXPZfvfu3Rg5ciTy8vJw4MABDB06FEOHDsXBgwed2g0aNAhnz54VX//4xz+c3h89ejR++ukn7Ny5E9u2bcPXX3+NyZMnN/s8q9WKkSNH4q677mr2XklJCUaOHIkvv/wSpaWlSElJwcCBA3H69OkArogPhAAAED755BOPbaZPny7ccsstTtuGDx8u5ObmBvLRhBBCCBN69+4tTJkyRfy3zWYTkpKShMLCQpftH3nkEWHw4MFO27KysoS//OUv4r/Hjh0r/OlPf3L7mT///LMAQPj+++/FbZ999pkQEhIinD592qnt9OnThT//+c/C6tWrhZiYGI9fS0NDg9CqVSth7dq1HtsFKlTZaAOUlpYiJyfHaVtubi6efvppt/tYLBZYLBbx33a7HZcuXULbtm3dPg+DEEIImwRBwJUrV5CUlASDQbmhhnV1daivr5flWIIgNLvfmEwmmEwmp2319fUoKyvDzJkzxW0GgwE5OTkoLS11eezS0lIUFBQ4bcvNzW02jKGkpARxcXFo3bo17r33Xrzyyito27ateIzY2FhkZmaK7XNycmAwGLBnzx78v//3/wAAu3btwqZNm1BeXo7Nmzd7/bpra2thtVrRpk0br20DoXj4MJvNiI+Pd9oWHx+P6upqXLt2DZGRkc32KSwsxIsvvqj0qRFCCFHRqVOncMMNNyhy7Lq6OqR2aIHKc3ZZjteyZctm4yfmzp2LefPmOW27cOECbDaby/vc4cOHXR7b3X3RbDaL/x40aBAefPBBdOzYEcePH8esWbNw3333obS0FEajEWazGXFxcU7HCA0NRZs2bcTjXLx4EePGjcOHH36I6Ohon77u5557DklJSc2KBnJTPHz4Y+bMmU6psKqqCjfeeCOOlCWhVUs2JuhcsAWeri/YTd4beXDR1kJS+8qGGN+P3dDK97ZW387jP9Yoj+9fqvd+nCqL52tWU+/9mtZYwj2+X18X5vY9W73R88HrPL9vqPdcuTNcc//9bbC4fQsAYPTyfmhdoPt7Xo/QaPG+XmForecbQ2id9xuH8VqD1zb+tDXUeLkAjdVe871tE0IA+zodp6ZWluOooUGw4mvrFrRq5fvvFanq6+tRec6OI/uS0KpVYPeJK1fsuDnzDE6dOuV0025a9VDSiBEjxP/u0aMHbrvtNnTu3BklJSUYMGCAT8eYNGkSRo0ahT/84Q8+tX/ttdewYcMGlJSUICIiwq/z9pXi4SMhIQGVlZVO2yorKxEdHe2y6gG4Lm0BQKuWBkQH+E0lh/O2erSSYZZynT2wY1yzebkRNhFp9f1/d0SDb20vWFvC15/HcKvnm36Yl1AAAKGhnj/M6OV9ADAaPX+OweD+fcHo5ZqHeAkfBi/hQ3D/PWH00uPo7bvB6CUbeH3f7rlBqJf3ASA0zEu4CANCr3luYwyVED6MVp/bGqT8OBkC+Ou6pQlCbeDBQQjx/WtjhRrd5q1ayXefiI6O9loxaNeuHYxGo8v7XEJCgst93N0X3bUHgE6dOqFdu3Y4duwYBgwYgISEhGYDWhsaGnDp0iXxOLt27cLWrVuxcOFCANe7kux2O0JDQ7FixQpMmDBB3HfhwoV47bXX8MUXX+C2227z+DXLQfE7eXZ2NoqLi5227dy5E9nZ2Up/tCLOy1DxAIDzAVY9zttaynIehBA+hbSUVvkkyggPD0dGRobTfc5ut6O4uNjtfc6f++Jvv/2GixcvIjExUTzG5cuXUVZWJrbZtWsX7HY7srKyAFwfF1JeXi6+XnrpJbRq1Qrl5eXimBAAeP311/Hyyy+jqKjIaQyJkiRXPq5evYpjx46J/z558iTKy8vRpk0b3HjjjZg5cyZOnz6NDz74AADw2GOP4Z133sH06dMxYcIE7Nq1Cx999BG2b98u31dBSBCwmbx3jWipISLEa9eMHGyRoT53p9iiwmCs9a1CYG9h8r3rpUUkUCNP9wnhX0FBAcaOHYvMzEz07t0bS5YsQU1NDcaPHw8AGDNmDJKTk1FYWAgAmDp1Kvr3749FixZh8ODB2LBhA/bt24cVK1YAuH6fffHFFzFs2DAkJCTg+PHjmD59Orp06YLc3FwAQLdu3TBo0CBMmjQJy5cvh9VqRX5+PkaMGIGkpCSxTWP79u2DwWDArbfeKm5bsGAB5syZg/Xr1yM1NVUcL9KyZUu0bKncH7mSKx/79u1Dr1690KtXLwDXL3qvXr0wZ84cAMDZs2dRUVEhtu/YsSO2b9+OnTt3omfPnli0aBHee+898QLyRK6qB+suSBjv4atLPo4LISQYhER5Hv9E+DJ8+HAsXLgQc+bMQXp6OsrLy1FUVCQOKq2oqMDZs2fF9n379sX69euxYsUK9OzZEx9//DG2bNkihgKj0Yh///vfeOCBB9C1a1fk5eUhIyMD//rXv5yGJKxbtw5paWkYMGAA7r//ftx5551igPHVu+++i/r6ejz00ENITEwUX46uGqVw8VTb6upqxMTE4MyRGzQb8yFn8Ai0ywXwr9vFbI31qZ2U8HHB6tt5+BI+Llq8t7ls8TwI6qqXAamA9wGnljr379ssAQ44tfg/4BTwXPkwehlQ6nXAqdf9fRjT4aWNtwGngPcxH4DEQac+Vj4AiYNOA6x8yDHuAwCEqzWyHEdJDYIVu+o3oaqqyudZF1LJeZ+ovmJH0s2/KXq+wU770ZscYK3iQeM9tGM02bQ+Ba41RNGvHEIIhY+g4WvVg/DLpuzMOCIz6nohwYzChxesVT2CmbcuF19463Ih+mOLcr9uS1P2FhK6RFu4XipAbTTrhfCIwocHSgQPPU2x9XW8ByGEENIYhQ83qOJBiDYaIr3/WrJFMrk4s2TU9UKClT5+ggkhXjVEeJ/xQvgU0rIFF7Ne1HDBVg+LLcDl1W3yPB+GuEeVDxf0VvWgwaZEDg0R3pfG9qUNIYRQ+GhCyeDBw3gPJRYY44mnNT4IvxQbdCoD6nohwYjCRyN6q3iwQq4FxghhDiMzXgjhDYUPQoiqaKExZdCUW8IT+i3wX0pXPeRYUp1oz+vy6hzjbZEyvcx4ISQYUfgAH90tLK3vQQiRF437IMEm6MOHGsFDy6qHUjNdaIExoiRf1vogzVHXC+EF/YQTUbDPdCHEgcdl1gnhSdCGj/O2ei66Wwgh8pAy3VYL1PVCgklQhg81Q4ccXS403sM3Vy00qJcQ6nohPAi68EHVDkIIIURbQTVXLdiCBy2rTvTOFhkK47UGrU/j+riPmmsBHyYkKgpCba0MJxS8LthNqLMH9nf1VTs920VpQVH50Gp8B63tId/qppctnC1CQbin9jLrcqKuF8I63YcP3qsdao33oJkuRC4+PYCOVjlVHAUQwjJd/wbgPXiwitb4CF68P7VW0RkvMk25lXPWCwUQwipdhg8WptFSlwshgaGFxuRBAYSwSHc/3VqHDjnRFFtCiBwogBDW6Cp86Cl4BIpmupBgwdQD5hjsehGPSQGEMEQX4YOFbpbGeOtyocGmhLjG84wXVyiAEFZwHz5YCh1yoi4XogUbzWjWPQoghAVchw8WgwdvVQ+paKYLIfyjAEK0xmX4YK2bRW56qXr4ssAYIWpT/AFzDI/7cDo+BRCiIYZGavlGz6GD+M+Xh8rVWMJVOBNC+BHSsgWEqzVan4asLtpa4JrNGNAxam02mc6GuMNV5eMC48GDlS4Xnma60NLqRA5MzXgBuKl+AFQBIdrgKnwEA7W7XGimi754GzDaIEOO82n5dJmWWFdqoTEpXS9+z3jhLIA4XoSogcKHTOSoerA+1kPKYFMa76EMGxvFNSYpWf0IhgAifhYFEaICCh8yYKW7BeCry4UQLSk+8NSBwwAifiYFEaIQCh8Bkit4yFH1kBo8pHS50BRb0hirXS9MVj8ArgOI+NmNggiFERIoCh9+Om83cR08lERdLoQXqlU/AF0EkMYoiJBAUPiQwBE45Oxm0Sp4BNtAU5pm6zveVzlltvoB6C6AOFAQIVJR+PCB3IFDPC7jA0wdlOpykWuarS9rfAQTNWa8XD8Om10vUqla/QB0G0AcKIgQX1D4cEOJKofT8WUKHix1twB8d7lY6viojtCMF++Yrn4Aug8gDmIQacH2ecph6dKlSE1NRUREBLKysrB3716P7Tdt2oS0tDRERESgR48e2LFjh9P78+bNQ1paGlq0aIHWrVsjJycHe/bsEd8vKSlBSEiIy9f3338vthMEAQsXLkTXrl1hMpmQnJyMV199VXx/3LhxLo9xyy23yHRlXKPw0YjSgUNu/gaPYOtyIcFJSgBRvfoBBE0ACQYbN25EQUEB5s6di/3796Nnz57Izc3FuXPnXLbfvXs3Ro4ciby8PBw4cABDhw7F0KFDcfDgQbFN165d8c477+DHH3/EN998g9TUVAwcOBDnz58HAPTt2xdnz551ek2cOBEdO3ZEZmameJypU6fivffew8KFC3H48GFs3boVvXv3Ft9/8803nY5x6tQptGnTBg8//LBCV+u6EEEQBEU/QQbV1dWIiYnBv3+OQ6tW8uYlLYKG1gNMlZrlIqXq4UuXCyBft4svYz58qXzYLF6Wba7z/L7B4rkbwnDNt+9vo8XL+3We3w/18r63/X8/jm+/PnxpF1pr997mmvc2jRmvNUhrX2v1ua2hxsv/BF/VXJPlMEJtrSzHUUKDvR7F/1mLqqoqREdHK/IZjvvEph/SENUqwOXVr9jwcM/DOHXqlNP5mkwmmEzNf99kZWXhjjvuwDvvvAMAsNvtSElJwZNPPokZM2Y0az98+HDU1NRg27Zt4rY+ffogPT0dy5cv9/j1ffHFFxgwYECz961WK5KTk/Hkk0/ihRdeAAAcOnQIt912Gw4ePIibb77Zp699y5YtePDBB3Hy5El06NDBp338EZSVDy0rHDwFD0J4x3z3C0AVEJlVNsTAbI0N6FXZEAMASElJQUxMjPgqLCxs9nn19fUoKytDTk6OuM1gMCAnJwelpaUuz7G0tNSpPQDk5ua6bV9fX48VK1YgJiYGPXv2dNlm69atuHjxIsaPHy9u+/TTT9GpUyds27YNHTt2RGpqKiZOnIhLly65vX6rVq1CTk6OosED4PDBcoHQujuFt+BBa3sEL1uEb9WPhogQn6sfPLBFhUmqfsimRaQsFZCQqCimKyC8cVX5aOrChQuw2WyIj4932h4fH4/Dhw+7PK7ZbHbZ3mw2O23btm0bRowYgdraWiQmJmLnzp1o166dy2OuWrUKubm5uOGGG8RtJ06cwK+//opNmzbhgw8+gM1mw7Rp0/DQQw9h165dzY5x5swZfPbZZ1i/fr3Lz5CT7sOH1oHDgZeZLf7ivcuFFfZIu89dL4FoiPDe9SLv53kPKQ1RBq9dLw2RBsldL7bIUMndL76ytzDJ1/1CAYQ50dHRinUT+eKee+5BeXk5Lly4gJUrV+KRRx7Bnj17EBcX59Tut99+w+eff46PPvrIabvdbofFYsEHH3yArl27ArgeUjIyMnDkyJFmXTFr165FbGwshg4dqujXBeiw26Vxl4reggd1tyiHl5kuvuJ9rQ4taTL41IG6YLjTrl07GI1GVFZWOm2vrKxEQkKCy30SEhJ8at+iRQt06dIFffr0wapVqxAaGopVq1Y1O97q1avRtm1bPPDAA07bExMTERoaKgYPAOjWrRsAoKKiwqmtIAh4//338eijjyI8XPnfh9yHDxbDRmMsBA9/0EPkiJ5wMfbDQcYAQiFEeeHh4cjIyEBxcbG4zW63o7i4GNnZ2S73yc7OdmoPADt37nTbvvFxLRbnSpsgCFi9ejXGjBmDsDDn4NyvXz80NDTg+PHj4rajR48CQLMxHV999RWOHTuGvLw8j+cgF+66XVgMGK7I2c0SaPBgqerha5cL4Yev4z607HqRSurYD1m7XwDZumCA5lUQ6pKRX0FBAcaOHYvMzEz07t0bS5YsQU1NjTj4c8yYMUhOThYHrE6dOhX9+/fHokWLMHjwYGzYsAH79u3DihUrAAA1NTV49dVX8cADDyAxMREXLlzA0qVLcfr06WZTYHft2oWTJ09i4sSJzc4rJycHt99+OyZMmIAlS5bAbrdjypQp+OMf/+hUDQGud8dkZWXh1ltvVeISNcNV+LhgN4GHkRO8Bw8WBpr6Mt6DECmUHPsBsB1AGmscRiiIyGP48OE4f/485syZA7PZjPT0dBQVFYmDSisqKmAw/N7R0LdvX6xfvx6zZ8/GrFmzcNNNN2HLli3ijd9oNOLw4cNYu3YtLly4gLZt2+KOO+7Av/71r2aLf61atQp9+/ZFWlpas/MyGAz49NNP8eSTT+IPf/gDWrRogfvuuw+LFi1yaldVVYV//vOfePPNN+W+NG5xtc7HroMpaCnzOh9yYyl4AMqHDyUGmgK+hw+m1vgAAl7nA/BtrQ9v63wAga/14csxfj+Wuut9ANLX/ACUXffDQdYAAigSQFxRMoiouc7HO2VZiGwZ2N/V1642ID9jj6LnG+zYvpNzhrUZLbxWPYg65HrGi7TPlOdZL/6SOvbDn8GnrI4B8cYxRoTGiRA1UPiQwXlbS9mDh57GeUildtVDr/Q+40XJh80FitcA4kBBhCiN3Z9eTihR7dDqYXFSqx5KdbmoTW/TbLXgS0WDFWpUPwD+A4hD4yBCYYTIhcJHAFgNHtTdQnjna9cLy9UPQD8BpDEKIkQOXM12YYGS4zq0Ch7+UGptD5rlon8sLMkudeZLIMuu8zILxh8szpy52NAKEQ2B3drqGpSbFUWuo/DhAzUGkmrV1QIoX/VQosvFl/EecvJpposO+fqMFy2pse5HoPQcQBxYDCKEXRQ+POAtdPA8yJRoQ+1nvPAs0IfOBUMAcaAgQrxhu8NUA46ZK8ESPPypevDS5RLMM120IPegUylTbv0Z+6Hkkuvu6HEMiDfXx4ewf55EXUFf+dBqbQ5eg4dUrHe5yDrTxcsCY4R/gVY/gOCqgBDiTtBVPhpXNoI5ePiLl6oHkY8S64UoMTVXreqHHE+9DcYKCCGN6b7ywdqqo1oOLG1Mje4Wltf2IHzw5UFzvKIKCAlmugsfrIUNByVCB8vdLUpRe5YLELwzXXjkz8wXfx44J0f3C0ABhAQv7sMHq2GjMZaCh7+U6m4BqMtFT1hYw4M3FEBIMOIufPAQNhpjLXioNbtF6y4XmumiH1K7XnirfgAUQEjw4WrA6UUbX2MI9BA8WCJ3lws908V3vAw65Zkig1BpICphlF/hY+nSpUhNTUVERASysrKwd+9ej+2XLFmCm2++GZGRkUhJScG0adNQV6fvlY3kDh4XGlppsoiY0lUPJbpcNKl60DRbpvA086Ux2QMI8HsIoTBCGCL5p23jxo0oKCjA8uXLkZWVhSVLliA3NxdHjhxBXFxcs/br16/HjBkz8P7776Nv3744evQoxo0bh5CQECxevFiWL4IlrFU7xGMwtJgY0Tclxn3oedZLU7J3wTTVOIDosGvmorUFTNbAQqHFKk93GnFP8p8HixcvxqRJkzB+/Hh0794dy5cvR1RUFN5//32X7Xfv3o1+/fph1KhRSE1NxcCBAzFy5Eiv1RIe6Sl4+EupsR5azHIBaKZLsGGh+gEoVAFxhSoiRCOSwkd9fT3KysqQk5Pz+wEMBuTk5KC0tNTlPn379kVZWZkYNk6cOIEdO3bg/vvvd/s5FosF1dXVTi/WsRo8/KVG1UM3XS4+MljkG+Ngk/He1ODj/wbexn340/XiL64DiAN1zxAVSYr5Fy5cgM1mQ3x8vNP2+Ph4HD582OU+o0aNwoULF3DnnXdCEAQ0NDTgsccew6xZs9x+TmFhIV588UUpp6YpJcZ3yHYsqnoQDbDS9aLWzBfg9wAi1wwY4PcAomg3jDuuAogOu2mINhT/06CkpATz58/HsmXLsH//fmzevBnbt2/Hyy+/7HafmTNnoqqqSnydOnVK6dOUxGyNdXrJiYXgwePUWqlYnulij5RvbIPc1Qreqh+A+g+ds0WFKTIQVfVKiCtNqyNUJSF+kvQT1q5dOxiNRlRWVjptr6ysREJCgst9XnjhBTz66KOYOHEiAKBHjx6oqanB5MmT8fzzz8NgaP6LwWQywWRi4AcN6i2HrmUXS2NqBY9g63IJRqxUPwB1KyDi/gpVQjSpgnhDVRIikaQ/CcLDw5GRkYHi4mJxm91uR3FxMbKzs13uU1tb2yxgGI3XB/EJAjsrITatZihR1XBH7uBxwdqSZrcEAS3GfQDaVz8aovwr2KpdARGPIXMlhJkqiDeNKyNRVB0hziT/ZBUUFGDs2LHIzMxE7969sWTJEtTU1GD8+PEAgDFjxiA5ORmFhYUAgCFDhmDx4sXo1asXsrKycOzYMbzwwgsYMmSIGELUxszD3RSodqi9kJjS3S1KjPdguctFb6RUPyS15agCIh5HxhVRAY3HgxASIMnhY/jw4Th//jzmzJkDs9mM9PR0FBUViYNQKyoqnCods2fPRkhICGbPno3Tp0+jffv2GDJkCF599VX5vgo3WAkZrihR7QiEmuM86Fku6rJFAEaZ1/STckylnvfCawABdDQolRA/hQgs9X24UV1djZiYGGz6IQ1RrVxXS1gOGo2xWO3wt7tFjbEevlY+pIz3kFL58HmdDx9WOJUy1dZwzbcuAqOP9xtfg0KoxJDi+3F9/zUjNaj4u/iY1AACQJYA4nQ8GUOIA4shpMFmwRf/9waqqqoQHR2tyGc47hNPffMnmFoGuMjYVSveuvN/FT3fYMfVg+UqG2IQaeXqlEUsho5AsBQ8gpnN5HsA0ZKST7vlsQIiHi+YBqUS0ghXD5bjFcvBIxim1TYWrOM9lBgkqtRx/Zl6y9sg1GbHVGhQatMXIazgs4zACZZDB6CfcR40xZYPSlY/AL4rIOJxZR6U2pS7AKKnSsl/rFEItwb2O6HeWi/T2RB3KHzITMn1OrQOHv6iAabKkrPrpSFC+rgPacf3LYD4G1T0EkAAZcaDuOMqlOgpkBD2UPiQCS+hA2B7gKkDTbHlhxKzaYDgDiCANiGksWCokhDtUPgIgBqrkrJS7eB5nAcr7CZB1ofLSaVUSJBK6e4XQP0AAsg/E0Y8vsYhpCmqkhA5UPjwgRZLn7NS7QiEGlUPJcd7+DzNVmf86XphrfoBqBtAgOALIY1RICFSUfhoQutnrCgxfTbQ4MHKs1uIe7xMuXVQo/oBqB9AAHVCCIsBpKnGgcRu0/BECJOCNnxoHTJcYbHaoXbwUGptDxrvcZ2SXS8sVj8AbQIIoGwIYbkKQogvdB8+WAwZTbFY7QDUH+chNXhQlwvflHrui8v9NQogAIUQQlzhepGxCw2tvL5Y5u/TZ73R8um0LM1uIf6T8pTbwD5HvQG4ai5E5ootMlSRBcoA+RcpI9ItXboUqampiIiIQFZWFvbu3eux/aZNm5CWloaIiAj06NEDO3bsEN+zWq147rnn0KNHD7Ro0QJJSUkYM2YMzpw54/JYFosF6enpCAkJQXl5udN7H330EdLT0xEVFYUOHTrgb3/7m9P7Z8+exahRo9C1a1cYDAY8/fTTfn39UnEVPi5yFCw8UTJ0yBU8eJ9W25jULheqevhPqZVUAXmCitYBBPg9hPCwUirxzcaNG1FQUIC5c+di//796NmzJ3Jzc3Hu3DmX7Xfv3o2RI0ciLy8PBw4cwNChQzF06FAcPHgQAFBbW4v9+/fjhRdewP79+7F582YcOXIEDzzwgMvjTZ8+HUlJSc22f/bZZxg9ejQee+wxHDx4EMuWLcMbb7yBd955R2xjsVjQvn17zJ49Gz179pThaviGqwfLvbgnBxEt+ewpUvI5LHJWOgLpalE7fPja7aJK+PDhwXKA7w+X8/XBco1JGXDqy9gMfxcb82fch69dKnINUlXzYXS+UGpwKsBGl0yDzYJd5a+p8mC5R78cifCWAa5werUef7/nHz6fb1ZWFu644w7xpm6325GSkoInn3wSM2bMaNZ++PDhqKmpwbZt28Rtffr0QXp6OpYvX+7yM77//nv07t0bv/76K2688UZx+2effYaCggL885//xC233IIDBw4gPT0dADBq1ChYrVZs2rRJbP/222/j9ddfR0VFBUJCnH8f3X333UhPT8eSJUu8fs2B4qrywRtHhYOCh3tKBw/iP3+7XlivfgBsVEAaU6NLxtWLeFZdXe30sliaJ/z6+nqUlZUhJydH3GYwGJCTk4PS0lKXxy0tLXVqDwC5ublu2wNAVVUVQkJCEBsbK26rrKzEpEmT8Pe//x1RUVHN9rFYLIiIcP6BjIyMxG+//YZff/3V7WepgcKHApQOHIC8XSwAX8FDSUpWPYh7UkKFXgMIoGwIcfl5Ogwll+pb4KIlsNel+uu/D1NSUhATEyO+CgsLm33ehQsXYLPZEB8f77Q9Pj4eZrPZ5TmazWZJ7evq6vDcc89h5MiRYiVGEASMGzcOjz32GDIzM13ul5ubi82bN6O4uBh2ux1Hjx7FokWLAFwf66ElPvswGKTm4+1ZCR2ANsFDStWDpti6pvRqp/4cX83ZL+JxNJwF44nSa4V4/XwPAYSFbhy1nDp1yqnbxWRS/w8mq9WKRx55BIIg4N133xW3v/3227hy5Qpmzpzpdt9Jkybh+PHj+J//+R9YrVZER0dj6tSpmDdvHgwGbWsPVPkIkBpVDgeWqh2BUCt4kMCpNevFH3qugDioXQnxhd4qJZ5ER0c7vVyFj3bt2sFoNKKystJpe2VlJRISElweNyEhwaf2juDx66+/YufOnU5BaNeuXSgtLYXJZEJoaCi6dOkCAMjMzMTYsWMBACEhIViwYAGuXr2KX3/9FWazGb179wYAdOrUSeLVkBeFDwkaj+FQM3QA8lc75Ageaq9iqnTwoC4X7UkNFMEQQAA2Q0hjwRRImgoPD0dGRgaKi4vFbXa7HcXFxcjOzna5T3Z2tlN7ANi5c6dTe0fw+L//+z988cUXaNu2rVP7t956Cz/88APKy8tRXl4uTtXduHEjXn31Vae2RqMRycnJCA8Pxz/+8Q9kZ2ejffv2AX3dgWL3u1lDaoYKb+Res0OuagcP4zyoy0Vbaj3ITu9dMI1p3R0jReMAYmvQ9/rqBQUFGDt2LDIzM9G7d28sWbIENTU1GD9+PABgzJgxSE5OFseMTJ06Ff3798eiRYswePBgbNiwAfv27cOKFSsAXA8eDz30EPbv349t27bBZrOJ40HatGmD8PBwpxkvANCy5fX7VufOnXHDDTcAuD4e5eOPP8bdd9+Nuro6rF69Gps2bcJXX33ltK9jbZCrV6/i/PnzKC8vR3h4OLp3767MBUOQhg+WwoUrSi0SxnPwoO4W+fkaDvx50Fwg/AkTLAQQBzWCiC0ylIsAEiyGDx+O8+fPY86cOTCbzUhPT0dRUZE4qLSiosJpjEXfvn2xfv16zJ49G7NmzcJNN92ELVu24NZbbwUAnD59Glu3bgUAcdqsw5dffom7777b53Nbu3Yt/vrXv0IQBGRnZ6OkpETsenHo1auX+N9lZWVYv349OnTogF9++UXCVZBGd+t8sB4sPGE9dAB8BQ8plQ+1ulxYWedD3MfHUBFI+FBy3Q+59mt2HD/XAXE6hkrVEB5CSENDHb4qfUWVdT4Gfz4RYS0C+2PFWlOP7bnvKXq+wY6rysdFawuYrPrqS1R6KXStqx2BUiN4EO94qH4Esl+z4/hZAXE6xn+rIUqHEJ66Yghx4Cp86IUaz15hodrhwPpCYrScurL8HfuhhwACUAghxBWa7aIiuafKuqOH4BEIVaoeCna5sIrlabdNaT0LxuWxIg1BOz2XkKboO1Rhaj9hlqVuFtbHeRD1qF39CHRfp+PIVAERj6dyJaQpqowQFlD4UAivoQPQbnyHg5ozW6jLRd9YDSCAeiGkKQolhAUUPmSkduAA2A0dWozzUK3qEYRdLg5qDzy9/pmBBQiWAwigXQhpylNXDU/BpMpiQmhoYN29DTr5eWUZhY8AaRE4APmXRqfgQZSm1qJjrrAeQAB2QogrVC0hcqPwIYFWQcNBqWexyBU8tHpKrb/Bg5ZT9w+P1Q+5jgEoG0AAtkNIU3qplhD1Ufj4L62DhSeshw4g8OCh5xVMWe5yUbsaoWX1A+AngADqr5oqt8bBxNZAtxriLCi+I1gOFu4o/cRZvQQPvXa32CPtklc5tZn8W+XUH4FUP7SY+dL0OEDgq6E6puEqHUIA/oMIIU1xGz54DBTeqPWIe967WRwCCR40y+V3/ocBbQIIIM8y6nKHEICCCCG+4ip8/McahXCrvv7SVStwAGxVOxz03N0SKNarH4A2AeT658obQuR6JgwFEUJ8w1X40As1A4cDi8EjEFT1YIdWAeT6Z8tUvZAxzIjHpCBCiFsUPlSkRegA2O1m0aLqwVvwUKv6EXgI0C6AXP98dkMIoG0QcYcCCtEShQ+F8R44HFiodjjodZAp77QOINfPQb5BqXIHEPHYTZ4Xo0YYcXkeFFCIhih8KEAvgcNBDxUPgL+qRyC0qH4A7AQQgN0qSLPPUbkqIoW3gELhhPiLwoeftAoYTSn9HBY5g4ccoYOqHuxjIYBcPw++QgjAdhBxhaonxF8UPhphJVB4o8aD31gLHYEKpqqHg1bVD4CdAHL9XPgLIQB/QcSdhkgDGqzSxi0FoqbeBGOAz3ax1ct0MsStoAgfvIQKT9R80iyrwYOqHnxhKYBcPx/2x4O4/UydBBFCHLgKH5fqWyCMgb+i1aLFo+31uH5HMFY9HLSsfgBsBhCAvyqI02dTECE6wFX4CBZahA6A7eBBVQ9+sRZAAH2EEICCCOEXhQ9GaBU4AHa7WeQQzFUPB62rHwCbAQSQ/3kxDlQRIcQzCh8a0jJwAPxMoaWqhz6wHEAAmVc3jWj+JGM1A0nTtUQ8oaBCtEDhQ0V6CxsOrFU7HKjq8TsWqh8AuwEEUL4LRetA4o6vQYVCCpEThQ+ZaR0wmlJ6ZVKlgwdTVY+64AszwRZAAJXX9WCgu8ZXvoQUCijEVxQ+/MBawGhMzWXQlQwegYYO2aseOgge/j7xlsUAAqgTQlSfUtsojLAcRNxxF1DUXOeD8IHCRyMshwpPtHjuSlCN79BB8AgUawEE+D2EAHyNB5H62Vp9PiFK0k344DU4BIJCh2uyVj10Fjz8rX4AbAYQB6WrIZpPqaUgQnSGq/BRZTEhNMBlc3mn5dNlg279Dp0FDzmwHEAAdUKI1jd/CiJED7gKH8GKhcfZyx08lAodslU9dBw8Aql+AOwHEEDZEKJ1FaQxCiLN1VjCYTQGOmaMrqXSKHwwSo+BA2C80uGg4+AhFx4CCKDsuBCWQghAQYTwhcIHI1gIGwAfYzrckaXqwWDwsEfaYbgm72yBQKsfAD8BxEGpagiLN31Xa4q4w8o5k+BC4UNFrASMpoJqrQ4iK94CCKBOl4wDDzd2KUHFgYevi7CNwofMWA0Yjam1IqmaoUOvVQ8lyVH9APgMIIC6U3Ub08ON25fAooevkyiHwocfeAgYTam9BDoFj+DCawBxUGvhMkC/gaSpxl+nzSC9ukL0LajDB48hwldaPm9FreBBM1vYwnsAAdQNIY0FSyAhxIGr8FFTb4IxyNf58ETrB7xxFzqAoA8ecnW9iMfTQQAB1OmS8YbH8SOE+IoW3OdcjSVcfGlJrdksvAUPgyX4ys2Nb9xyadBwAWNbhDJfk1QNESHNXoQdS5cuRWpqKiIiIpCVlYW9e/d6bL9p0yakpaUhIiICPXr0wI4dO5ze37x5MwYOHIi2bdsiJCQE5eXlTu9funQJTz75JG6++WZERkbixhtvxFNPPYWqqiqndk899RQyMjJgMpmQnp7u8lwEQcDChQvRtWtXmEwmJCcn49VXX5V8DaSg8MEpFgIHcD10KB08ZA8dAAWPRmwKFBP1FkAANgJIUxRE2LBx40YUFBRg7ty52L9/P3r27Inc3FycO3fOZfvdu3dj5MiRyMvLw4EDBzB06FAMHToUBw8eFNvU1NTgzjvvxIIFC1we48yZMzhz5gwWLlyIgwcPYs2aNSgqKkJeXl6zthMmTMDw4cPdnv/UqVPx3nvvYeHChTh8+DC2bt2K3r17S7wK0oQIgsB8La+6uhoxMTHo9XEBjFHB1e3CQsBoisvulaYUDh9yBw+51/loSs6uF6fjKtRloXY3TFNadcX4irUuGlt9HQ6sex5VVVWIjo5W5DMc94mu62YEfJ+w1VpwdPRrPp9vVlYW7rjjDrzzzjsAALvdjpSUFDz55JOYMWNGs/bDhw9HTU0Ntm3bJm7r06cP0tPTsXz5cqe2v/zyCzp27IgDBw64rVw4bNq0CX/+859RU1OD0FDnURXz5s3Dli1bmlVQDh06hNtuuw0HDx7EzTff7PVrlQtVPhjQuOuk6YsFjuqGGlUOQKFKR2NBPs7DFSWqH4By1YKGCOqK8cRVF427F3Gvurra6WWxNE/p9fX1KCsrQ05OjrjNYDAgJycHpaWlLo9bWlrq1B4AcnNz3bb3lSMsNQ0ennz66afo1KkTtm3bho4dOyI1NRUTJ07EpUuXAjoXb7gacMo7VsKEL7RYGEzRwKEiXrpb1KLEIFSHxgFEi2qIVrNj5KS3Rcbq68JgMAT2+8teZwcApKSkOG2fO3cu5s2b57TtwoULsNlsiI+Pd9oeHx+Pw4cPuzy+2Wx22d5sNvt9zhcuXMDLL7+MyZMnS9rvxIkT+PXXX7Fp0yZ88MEHsNlsmDZtGh566CHs2rXL7/Pxxq/Kh9SBNZcvX8aUKVOQmJgIk8mErl27NhtcoxesVzE8UbO60ZjilY7GOOtu0Qs1qgRaVkMclRDWKyJykVJZ4bnCcurUKVRVVYmvmTNnan1KLlVXV2Pw4MHo3r17s3Dkjd1uh8ViwQcffIC77roLd999N1atWoUvv/wSR44cUeaE4UflwzGwZvny5cjKysKSJUuQm5uLI0eOIC4urln7+vp6/PGPf0RcXBw+/vhjJCcn49dff0VsbKwc5686HkKEr7Re9lwvlQ4HCh6eKVkBacwRQLQcF+IqgPBcHQlW0dHRXsd8tGvXDkajEZWVlU7bKysrkZCQ4HKfhIQESe09uXLlCgYNGoRWrVrhk08+QVhYmKT9ExMTERoaiq5du4rbunXrBgCoqKhQbByI5MrH4sWLMWnSJIwfPx7du3fH8uXLERUVhffff99l+/fffx+XLl3Cli1b0K9fP6SmpqJ///7o2bNnwCffmKeKg5wv3mlV3WhM1UqHQ51R0aoHBQ/fqFkVcFRCtJ4l4xBs1ZFgER4ejoyMDBQXF4vb7HY7iouLkZ2d7XKf7Oxsp/YAsHPnTrft3amursbAgQMRHh6OrVu3IiJC+jdWv3790NDQgOPHj4vbjh49CgDo0KGD5OP5SlLlwzGwpnHpydvAmq1btyI7OxtTpkzB//7v/6J9+/YYNWoUnnvuORiNrm8GFovFaWBPdXU1gOsBw2jkPwCoSevqRmOaVDpoSi1z1KqANKb12BBX3AUQqpDwp6CgAGPHjkVmZiZ69+6NJUuWoKamBuPHjwcAjBkzBsnJySgsLARwfWpr//79sWjRIgwePBgbNmzAvn37sGLFCvGYly5dQkVFBc6cOQMAYhdIQkICEhISxOBRW1uLDz/8UBwUCwDt27cX76/Hjh3D1atXYTabce3aNXG2S/fu3REeHo6cnBzcfvvtmDBhApYsWQK73Y4pU6bgj3/8o1M1RG6Swoc/A2tOnDiBXbt2YfTo0dixYweOHTuGJ554AlarFXPnznW5T2FhIV588UUpp0YaYSlwOGhS6dARe6Rd8em2cq926vGzNByoyWIQaYxCCX+GDx+O8+fPY86cOTCbzUhPT0dRUZF4r6yoqIDB8PvPb9++fbF+/XrMnj0bs2bNwk033YQtW7bg1ltvFdts3bpVDC8AMGLECAC/D3rdv38/9uzZAwDo0qWL0/mcPHkSqampAICJEyfiq6++Et/r1auXUxuDwYBPP/0UTz75JP7whz+gRYsWuO+++7Bo0SIZr1Bzktb5OHPmDJKTk7F7926n8tD06dPx1VdfiReisa5du6Kurg4nT54Uk9jixYvxt7/9DWfPnnX5Oa4qHykpKbLM39YTFkNGY5qN6VA5eKhZ+VA6gADqBRCnz2TkxspiGPGGlWvnia2+Dj+umqXKOh+pq16AISqwfi17bR1+yXtZ0fMNdpIqH/4MrElMTERYWJhTF0u3bt1gNptRX1+P8PDmN1CTyQSTiUKGA+shozHNB5HqrOLRlN4qIOJnMjJllfWqiCu+jh/R+toS0pik32L+DKzp168fjh07BrvdLm47evQoEhMTXQaPYNV0IS8WBob6yjGAlIKHOuyRdu+NAqTUomNeP5ehwZiNB6yyMmg1EE0HvEp5ESI3yX9CFRQUYOXKlVi7di0OHTqExx9/vNnAmsYDUh9//HFcunQJU6dOxdGjR7F9+3bMnz8fU6ZMke+r4AivAcMVJgKHQ5AEDwe1AoirlxpYvOnpKYhIFUhwYe3/I2GD5HU+pA6sSUlJweeff45p06bhtttuQ3JyMqZOnYrnnntOvq+CQbwGCm+YCRuNBVnwcFCjC8YVRwBRo2uGhUfbu9I0gPDSRaMVrSpphF1+La+en5+P/Px8l++VlJQ025adnY3vvvvOn48KiF4DgNqYDBxA0IaOxrQKIIC6IQRgZ1yIK1KrIRRWSLDj6tkucqzZT3zDbOBwoOAh0jKAAM5/1apdDfGExZDi4C6sUCgJnK3eCMHNGlK+stfT7xelcRU+iLKYDxwOFDya0TqAOKhdDfGkaUhhOYw4UCghwYLCB+EndAAUPDxgJYAAbIUQB1bHj/iCQgnRGwofOsFVgPAXBQ+vWAogAJshBOCzKuIKhRLCKwofnAmKkOEKBQ+fsRZAAHZDiAPPVRFX1JgOTAGHBILCB6OCNmS4QsFDMhYDCMB+CAH0UxVRmpSAY6NnL5ImKHxojEKGFxQ8/MZqAAH4CCEOequKEMICrsKHL1OojCab633pJq8+Cg6aYzmAAHyFEICqIoTIhavw4QsKGRqjwMEc1gMIwF8IcZBz6XAKMiSY6C58EI1Q6GAaDwEE4DeEyMFdkKFQQvSIwgfxHwUOrvASQIDgDiFNeaquUDAhvKLwQaShwME1ngIIQCHEG6qWEF5R+CDe6SxwGCzBPe+PtwACUAiRSovH2DMTeOqMQEiAv7N09juPRRQ+eEY/IJIFe/Bw4DGAAMo+mp2CTWA8BR7XcxBJMKPwwQMKGbKg4OGM1wCilKbBhsIIIcrhK3zIUU4jQYmCh2sUQNxrHEYoiBAiL77CByF+oODhGQUQ76gqQoi8KHwQ3aLQ4TsKINJQGCEkMBQ+iC5R8JCOAoj/lBwI6w4FHsIzCh9Edyh4+I8CCD/cBR4KJYQHFD6IoigI8IcCCN+0qMK4QiGIeELhg8iOAgf/KICQQDUOQXZBu/MgbKLwoSOB3vTtJv9/Q1Dg0B97pN3vfSm4EEI84Sp8GOpDYDDQTU4pFCCIXAIJLo1RiCFEn7gKH4SQ4OIIMRRCiK9k+SO1nv4QUxr9RBNCmGePtMtWTSGEaI8qH4QQbrAWQKgiQ4h/KHwQQoif1ApDFHKI3lD4IIQQxrFW8ZHKHsL3+RP5UZwmhBBCiKq4Ch9UeiSEEEL4x123i14CCO9lVEIIIcRf+riTc8hwzaDoixBCCHsuXbqE0aNHIzo6GrGxscjLy8PVq1c97lNXV4cpU6agbdu2aNmyJYYNG4bKykqnNhUVFRg8eDCioqIQFxeHZ599Fg0NDU5tSkpKcPvtt8NkMqFLly5Ys2ZNs886ffo0/vznP6Nt27aIjIxEjx49sG/fPvF9QRAwZ84cJCYmIjIyEjk5Ofi///s/ydeB7lI6pXS4oSBECCHSjR49Gj/99BN27tyJbdu24euvv8bkyZM97jNt2jR8+umn2LRpE7766iucOXMGDz74oPi+zWbD4MGDUV9fj927d2Pt2rVYs2YN5syZI7Y5efIkBg8ejHvuuQfl5eV4+umnMXHiRHz++edim//85z/o168fwsLC8Nlnn+Hnn3/GokWL0Lp1a7HN66+/jrfeegvLly/Hnj170KJFC+Tm5qKurk7SdQgRBIH5R/5UV1cjJiYGHefNhyEiQuvTIX6iriZCgpO9rg6/PjcbVVVViI6OVuQzHPeJDgteCfg+4TjfU6dOOZ2vyWSCyeT/Y4MPHTqE7t274/vvv0dmZiYAoKioCPfffz9+++03JCUlNdunqqoK7du3x/r16/HQQw8BAA4fPoxu3bqhtLQUffr0wWeffYb/+Z//wZkzZxAfHw8AWL58OZ577jmcP38e4eHheO6557B9+3YcPHhQPPaIESNw+fJlFBUVAQBmzJiBb7/9Fv/6179cnr8gCEhKSsIzzzyDv/71r+L5xcfHY82aNRgxYoTP14L+LCWqoUoIIURpclZsU1JSEBMTI74KCwsDOrfS0lLExsaKwQMAcnJyYDAYsGfPHpf7lJWVwWq1IicnR9yWlpaGG2+8EaWlpeJxe/ToIQYPAMjNzUV1dTV++uknsU3jYzjaOI4BAFu3bkVmZiYefvhhxMXFoVevXli5cqX4/smTJ2E2m52OExMTg6ysLKfj+IK7AaeEf3IHEKqoECCwpzI3JfUhi3J+th7Z7XxeH1eVj0CYzWbExcU5bQsNDUWbNm1gNpvd7hMeHo7Y2Fin7fHx8eI+ZrPZKXg43ne856lNdXU1rl27hsjISJw4cQLvvvsuCgoKMGvWLHz//fd46qmnEB4ejrFjx4rHcnUcd+fvDoUPwj1/wgwFFn1Q6qZPYYIAQHR0tE/dRDNmzMCCBQs8tjl06JBcp6UYu92OzMxMzJ8/HwDQq1cvHDx4EMuXL8fYsWNl/SyuwofBAhgb/UFiCyyEkiBmuGagAMIpCgaENc888wzGjRvnsU2nTp2QkJCAc+fOOW1vaGjApUuXkJCQ4HK/hIQE1NfX4/Lly07Vj8rKSnGfhIQE7N2712k/x2yYxm2azpCprKxEdHQ0IiMjAQCJiYno3r27U5tu3brhn//8p9OxKisrkZiY6HSc9PR0j19/U1yFj6aMFq3PgAIQzyiABI6CACFA+/bt0b59e6/tsrOzcfnyZZSVlSEjIwMAsGvXLtjtdmRlZbncJyMjA2FhYSguLsawYcMAAEeOHEFFRQWys7PF47766qs4d+6c2K2zc+dOREdHi2EiOzsbO3bscDr2zp07xWMAQL9+/XDkyBGnNkePHkWHDh0AAB07dkRCQgKKi4vFsFFdXY09e/bg8ccf9/r1N0aj/wJktLh+ET7QAFj/UfAgRJpu3bph0KBBmDRpEvbu3Ytvv/0W+fn5GDFihDjT5fTp00hLSxMrGTExMcjLy0NBQQG+/PJLlJWVYfz48cjOzkafPn0AAAMHDkT37t3x6KOP4ocffsDnn3+O2bNnY8qUKeI4lcceewwnTpzA9OnTcfjwYSxbtgwfffQRpk2bJp7ftGnT8N1332H+/Pk4duwY1q9fjxUrVmDKlCkAgJCQEDz99NN45ZVXsHXrVvz4448YM2YMkpKSMHToUEnXguvKB8u0CiBUiZGOKiDSUfAgxD/r1q1Dfn4+BgwYAIPBgGHDhuGtt94S37darThy5Ahqa2vFbW+88YbY1mKxIDc3F8uWLRPfNxqN2LZtGx5//HFkZ2ejRYsWGDt2LF566SWxTceOHbF9+3ZMmzYNb775Jm644Qa89957yM3NFdvccccd+OSTTzBz5ky89NJL6NixI5YsWYLRo0eLbaZPn46amhpMnjwZly9fxp133omioiJESJzezNU6H51nzoeR1vnwGQURaSiAeEehg/jDfq0Op6a9oMo6H3KsB2Wvq8PJebMUPd9gR5UPHWtafaEw4hlVQDyj4EEIkQuFjyDSOIxQEHGNAohrFDxkEmHT+gy0IQTp103covARpKgq4h4FEGcUPCQK1oBBiARchQ+jBTB6eN9Gw0H8puQAWR6DDQWQ63QXPCgYEMIErsKHN0ZpD9WTBQUe7xzBhrcQwmMA0V1YkAMFjqDSdDFKv9ByCYrTVfjQgrvAQ6GkOR5DCI8BhIACByGMo/ChEAol7vEWQiiAcIICByHcoPChMgolv+MphFAAYRQFDkK4ROGDEXKOV+EtyPASQiiAMIRCh6qMJt+ut83iaUoAIb+j8KFDTYMML2GEhxBCAUQGFBw88vVGzyJ35x5i4/drIsqg8BEEGocRHoII6yGEAkgAKHiIeA4ZhASKq/ARWgcYJcwkbODgRqs2nqoiLIcQCiB+CNLgQSGDkOa4Ch9ShSq87ocewg0PVRFWQwgFEAmCIHhQyCDEd7oOH0pzF254DSWsV0VYDCEUQHzAUfCgAEGIOih8KEAvoYTVqghrD8ijAOIBB8GDAgch6qPwoSKeQwmrVRFWqiEUQFxgPHhQ6CBEOxQ+GOBpbAqrwUTKuiRqBBUWQggFkEYYDR4UOAJniqiXvI/NblXgTFzz9gBSn9CzXRRH4YNxUgbNsh5UgiGEBH0AYTB06DVw+BMCCGEFV+HDaAHAQB8/q5oGFdbCiBYhxEHNMBK0AYSh4KGnwEEhg+gRV+EDCGwZclbGKaiF1TCiZggRP1PlMBJ0AYSR4MFr6KCAQYKNwZ+dli5ditTUVERERCArKwt79+71ab8NGzYgJCQEQ4cO9edjA2asC/zFs9C6318s0PKaNg0jSjBc8+vHiz8aBw+jySa+WGeKqHf5IiTYSP7tuHHjRhQUFGDu3LnYv38/evbsidzcXJw7d87jfr/88gv++te/4q677vL7ZFmgl3DSOIhoHUa0unYUQGSgUfBgPXBQyCDEM8m/GRcvXoxJkyZh/Pjx6N69O5YvX46oqCi8//77bvex2WwYPXo0XnzxRXTq1MnrZ1gsFlRXVzu9eMFrKGEhiGhxrSiABEDl4MFi4KCQQYh/JP1WrK+vR1lZGXJycn4/gMGAnJwclJaWut3vpZdeQlxcHPLy8nz6nMLCQsTExIivlJQUKafJJJ6qJVpXRdS+LhRAJIqwqRo8WA4chBD/SBpweuHCBdhsNsTHxzttj4+Px+HDh13u880332DVqlUoLy/3+XNmzpyJgoIC8d/V1dW6CCDuuLrRsjQ41hFA1B6waqxj6zqwzm6S8NRFf2jQxcJC6KCQ4V0Lk+drZLPRNSTOFJ3tcuXKFTz66KNYuXIl2rVr5/N+JpMJJlNwz6llMZA0roKoFUQogPhG0eARhKFDL4HDWyggRCuSwke7du1gNBpRWVnptL2yshIJCQnN2h8/fhy//PILhgwZIm6z269PPwwNDcWRI0fQuXNnf847KLG0xLma1RAKIJ4pXvFQkVahg+ewQQGD8EhS+AgPD0dGRgaKi4vF6bJ2ux3FxcXIz89v1j4tLQ0//vij07bZs2fjypUrePPNNyV3pYTWCTDapf2ibYgIkdSeJyxUR9QKIUoHEKNF++fD+EMvXS1ahA6eAgcFDKI3krtdCgoKMHbsWGRmZqJ3795YsmQJampqMH78eADAmDFjkJycjMLCQkRERODWW2912j82NhYAmm1XSmid9F/OPAcWraoEanTJUAXEmR4qHmqGDtbDBgUMeYTWAcYAfzRC6NkuipMcPoYPH47z589jzpw5MJvNSE9PR1FRkTgItaKiAgYD3yP7fQ0srIYULVYQbUzJaggFEBUpWPVQK3SwGDgoZBDi54DT/Px8l90sAFBSUuJx3zVr1vjzkUxqHFJYDCJ6DSEUQPiteqgROlgJHBQyCHGPu2e7sIrlIKL1zTq0jo8Awsu4D1WCh8xVDwodhJDGKHwowFW3jdaBhIUqCA8BhHW8BY9g6V6hwEGINBQ+VMJKIGk8Q0aLmTGsBxBeqh+KkSl4BEPo0GPgaGlSZqRlQwON4CTOKHxoqGkgUTuMaFEN4SGABC0ZgoeeZ6/wGDaUChOEBIqr8GG0CAj97zofWndjKEGr6ojaIYT1ABKU1Q8KHi6xHjgoXBBecTsnNrRO8PrSAzW/LjUfdqfEA+tYfEhfY8w+XI6j4KH0A91amOqdXqxoabK4fBH+XLp0CaNHj0Z0dDRiY2ORl5eHq1evetynrq4OU6ZMQdu2bdGyZUsMGzas2UrjTz31FDIyMmAymZCent7sGEeOHME999yD+Ph4REREoFOnTpg9ezasVqvYZvPmzcjMzERsbCxatGiB9PR0/P3vf3c6zrhx4xASEuL0GjRokOTrwFXlQyp/b9SsV1XUqJCoUQ1huQISlNUPxikVOlgJGRQmgsPo0aNx9uxZ7Ny5E1arFePHj8fkyZOxfv16t/tMmzYN27dvx6ZNmxATE4P8/Hw8+OCD+Pbbb53aTZgwAXv27MG///3vZscICwvDmDFjcPvttyM2NhY//PADJk2aBLvdjvnz5wMA2rRpg+effx5paWkIDw/Htm3bMH78eMTFxSE3N1c81qBBg7B69Wrx3/48i03X4cNfPC4y5jhn3kIIywEkKHBQ9ZA7dFDYIFo5dOgQioqK8P333yMzMxMA8Pbbb+P+++/HwoULkZSU1GyfqqoqrFq1CuvXr8e9994LAFi9ejW6deuG7777Dn369AEAvPXWWwCA8+fPuwwfnTp1QqdOncR/d+jQASUlJfjXv/4lbrv77rud9pk6dSrWrl2Lb775xil8mEwml89zk4LROjAfWOziUep8lOySCcYuGCZo8LRaKeTuYmGhK4W6TPhSXV3t9LJYAvv/VlpaitjYWDF4AEBOTg4MBgP27Nnjcp+ysjJYrVbk5OSI29LS0nDjjTeitLTU73M5duwYioqK0L9/f5fvC4KA4uJiHDlyBH/4wx+c3ispKUFcXBxuvvlmPP7447h48aLkz6fKh0xYW2RMqUoIoEw1hMUKiK67XhieUqtE94qWoYOChrqMlsCf7YL/frs0ffjp3LlzMW/ePL8PazabERcX57QtNDQUbdq0gdlsdrtPeHi4+Fw0h/j4eLf7eNK3b1/s378fFosFkydPxksvveT0flVVFZKTk2GxWGA0GrFs2TL88Y9/FN8fNGgQHnzwQXTs2BHHjx/HrFmzcN9996G0tBRGo9Hn86DwoQCWgkhonaDYOcgdQlgMILrEaMVDT2M6gi1wxJo8lxqtDWx0dUl16tQpREdHi/92N7ZhxowZWLBggcdjHTp0SNZz89fGjRtx5coV/PDDD3j22WexcOFCTJ8+XXy/VatWKC8vx9WrV1FcXIyCggJ06tRJ7JIZMWKE2LZHjx647bbb0LlzZ5SUlGDAgAE+nweFD4VpvZZH43NQOoQAgd/oWQsguqt+MLh6qV5CB8+Bw1t4CFbR0dFO4cOdZ555BuPGjfPYplOnTkhISMC5c+ectjc0NODSpUtux1AkJCSgvr4ely9fdqp+VFZW+jXuwlHN6d69O2w2GyZPnoxnnnlGrFoYDAZ06dIFAJCeno5Dhw6hsLCw2XiQxl9Xu3btcOzYMQofLNOyKqJ0CAHkqYawFkB0g7GKh9JTZtXCauigQKGe9u3bo3379l7bZWdn4/LlyygrK0NGRgYAYNeuXbDb7cjKynK5T0ZGBsLCwlBcXIxhw4YBuD5ttqKiAtnZ2QGdt91uh9Vqhd1ud9tlYrfbPY51+e2333Dx4kUkJiZK+myuwkdorR2hYXaX7zVE8Td2VqsgwkMIYSmA6KL6wdCD4pReIEzp4MFS2KCAwZdu3bph0KBBmDRpEpYvXw6r1Yr8/HyMGDFCnOly+vRpDBgwAB988AF69+6NmJgY5OXloaCgAG3atEF0dDSefPJJZGdnizNdgOsDSK9evQqz2Yxr166hvLwcwPUKR3h4ONatW4ewsDD06NEDJpMJ+/btw8yZMzF8+HCEhYUBAAoLC5GZmYnOnTvDYrFgx44d+Pvf/453330XAHD16lW8+OKLGDZsGBISEnD8+HFMnz4dXbp0cZoN4wuuwocnobWuQ4kD6+FEiyCi5HgQh0AqDiwFEK4xVPHgNXhoHTgoZOjHunXrkJ+fjwEDBsBgMGDYsGHiNFkAsFqtOHLkCGpra8Vtb7zxhtjWYrEgNzcXy5YtczruxIkT8dVXX4n/7tWrFwDg5MmTSE1NRWhoKBYsWICjR49CEAR06NAB+fn5mDZtmrhPTU0NnnjiCfz222+IjIxEWloaPvzwQwwfPhwAYDQa8e9//xtr167F5cuXkZSUhIEDB+Lll1+WvNZHiCAI7MwTdaO6uhoxMTHoc/9LCA1T/s7BYlBROiQoffxAbvhyBxDAv/ORo/phj/Qckr3u789TbYOg6iF36KCwIS9rTT22576Hqqoqn8ZQ+MNxn+iRNx/G8MB+adjq6/DjqlmKnm+w003lQ07eqigOaoYUpVc1VWNQKksVEKI+HoKHlqFDb4GDEE8ofASgcUjRolqixEwapafmshJA/DkXXYz9CJCaD4/zhVzBQ6vQQYGDBCsKHzLROogA8lUvlF6gjJUAQtTD8sJhagcPChyEUPhQhNZBhPUQwkoAoeqHNKxUPXjtZtFD6GhrqvFrv3orn4uMEeVQ+FCYlkGE5RDCSgAhypP7+SxyUSN48BI4/A0VhPiLq/ARWmcHwrQ+C/81HciqVhiRaxyH3ONBWAggVP3wjdZVD55CByuBI1gDRWidAKM9sEmcIfXMTwLlHlfhAwBCr3mfidIQyd5UWVfUrIqwWgVhIYAQ5QRa9eBlCq1WgSNYAwbhH3fhwxc8BhS1ggiLIUTrAELVD8+0qHoosViY3MGDAgch/tNl+PCFt4CiZThRI4iw1hWjdQAh8lN6NVMp5AoeFDgIkUfQhg9vWKmeKBlEWKuCaBlA1Kp+GK4ZAl7lVE28Vz3krHaoHTwocBA9o/ARgMYBhecgwlIIoQqIPrBQ9eA1eFDoIMGAwodM9BBEWAkhWgUQqn4447nqwVs3i94CR5sw56/HEmbV6EwIqyh8KID3ICLneBDAvxBCFRBtsfjwOF/wVO3gMXA0DRWE+IvCh8K0CiKBhhA5Z7P4G2a0CCBU/QhMIMEj0KoHD8GD1cBBoYKojavwYbzWAGNoAwDAFsnVqQNQN4iE1tqZqoL4K5AAojd2k34XPmIleOgxdFCwICzi7w7+X8ZrDW7f4yGYOIKIkiGEpQCiRYih7hcAEdK7T/ztctGiu4XlaodWgYPCBuEB+3dpP7gLJiyGEqVDCEsBxF9U/WCfFt0trAYPLUIHBQ7CG/buxgpiuVqiZAhhJYBQ9cN/LHe58Bw8eA4dFDhcM1oEhAb4bBdY2f1504ugCh+esFItUSqEsBJA/EXVDz+o2OXiDyWWUJdCruBBoYMQ6Sh8eKFVKFEihLAQQLQewErk5W/VI5DgEWjVg8dqBwUOojdsPV2NI8ZrDR67ceQSes3u01LvPh+vVp5jOabi8iCUjSec+82vLhc/qh5S8TjAlLfg0SashoIH0SWqfATIEUB4qoRoXQHRYt0P4p1aXS5ajfPgpZtFb2GjXdhV1IUp/4ca4QuFD5moHUKAwIIIrwHEX3oZeMoKLbpb/MVDtYPXwNEu7KrWp0A4ReFDZmqFECDwaojWAcQfVP3wAaMDTbUY58F6tYP10EHhgiiFwodCjNcaVJspE0gI0TKAsFr9oIAjPwoev2MpcFC4IFrhKnwYrzXAaLz+dERbVJjGZ+OdmlUQ4HoI4S2A+IPCgbbUHGiqp+ChZeigkEFYw1X4aMxY2/wRzawGErW7YngKIDT1VmYMdrmoPc5DjuDBc7WDggbhAbfhwxVXgQRgJ5SoOSiVAoi7z6KBp2rScj0Pf/Fa7aDQQXiiq/DhDmuhRI0QwlsAkUrprhfq2gkcb+M8eKx2UOAgvAqK8OGOu1ACqBNMlA4hPAUQqn7IQIUuFzXGe/AePJQOHRQ4PAuttSM0LMDFFK3yLexIXAvq8OGJmtUSJUMITwFEKqpO6A8FD/codBA9ofAhUeNQIncQabxcu5xBROsA4vPnMTb4lMKN/7R+aJwveAgdFDiIXlH4CIAjiPBQDdEygFD1g02sdbmoWfWQK3hQ6HCtXegVp39fC6Xl1YkzCh8y4CWE8FABUav6odtxH5zSej0PfygRPHgIHU2DBSH+oPAhIx5CiFYBhNXBp0R+Urtc1J5SK0fVQ+7gwWLooJBBlEThQwGshxAeAogUSna9sNCtY7CEwG4SZD0mK10ugQQPLbpb9FjtoJBBtMBV+DDUWGAwXv9vewuTtifjA5ZDCOtdMGpUP+TsejFaABv735K6oYfgoUXooKBBWKHeVAaZGWoszV6s8rSeSMDHvtbgNEtGCscD6STvV+v/HPjQOnn/gncwatf1zx0ll1OX0uXCyziPNmE1sgaPdmFXFQ8e7UKvuHwR7V26dAmjR49GdHQ0YmNjkZeXh6tXPX8/1NXVYcqUKWjbti1atmyJYcOGobKyUnz/hx9+wMiRI5GSkoLIyEh069YNb775ZrPjWCwWPP/88+jQoQNMJhNSU1Px/vvvi+//9NNPGDZsGFJTUxESEoIlS5Y0O8bXX3+NIUOGICkpCSEhIdiyZYtf14Gryoc37gIIC1USJasggP+VEJYrIKyM/WCh64UExt+qB0/VDgoXfBg9ejTOnj2LnTt3wmq1Yvz48Zg8eTLWr1/vdp9p06Zh+/bt2LRpE2JiYpCfn48HH3wQ3377LQCgrKwMcXFx+PDDD5GSkoLdu3dj8uTJMBqNyM/PF4/zyCOPoLKyEqtWrUKXLl1w9uxZ2O2//zFZW1uLTp064eGHH8a0adNcnktNTQ169uyJCRMm4MEHH/T7OugqfLjjKpRoFUjUCCGsBxClBp9SSFCGEuM9eJhWK3e1QykUOvhx6NAhFBUV4fvvv0dmZiYA4O2338b999+PhQsXIikpqdk+VVVVWLVqFdavX497770XALB69Wp069YN3333Hfr06YMJEyY47dOpUyeUlpZi8+bNYvgoKirCV199hRMnTqBNmzYAgNTUVKf97rjjDtxxxx0AgBkzZrj8Gu677z7cd999/l+E/+K22yVQWnfbGGutinXH+NMNo3YXjFLdL1KFyljBN7Lb86cLana3sB48qCtFHdXV1U4viyWwH/LS0lLExsaKwQMAcnJyYDAYsGfPHpf7lJWVwWq1IicnR9yWlpaGG2+8EaWlpW4/q6qqSgwZALB161ZkZmbi9ddfR3JyMrp27Yq//vWvuHbtWkBfk7+CovLhK08BRKlKiVKVEB4qID4fn5HuF+KZkqua+hs8/Kl6yBU8lAodxLPQOjtCGwJ8Nst/909JSXHaPHfuXMybN8/vw5rNZsTFxTltCw0NRZs2bWA2m93uEx4ejtjYWKft8fHxbvfZvXs3Nm7ciO3bt4vbTpw4gW+++QYRERH45JNPcOHCBTzxxBO4ePEiVq9e7ffX5C8KHz5SuuvGWGsNugCiRKhQquslWLt01HiQnFLkfkqtFBQ89OHUqVOIjo4W/20yuf6dP2PGDCxYsMDjsQ4dOiTrublz8OBB/OlPf8LcuXMxcOBAcbvdbkdISAjWrVuHmJgYAMDixYvx0EMPYdmyZYiMjFTl/BwofATAEUjkCiFKVEFYDyA+H1vB6gdNuVWX1PEePI3zCObQkRB22e17taHKzbBSUnR0tFP4cOeZZ57BuHHjPLbp1KkTEhIScO7cOaftDQ0NuHTpEhISElzul5CQgPr6ely+fNmp+lFZWdlsn59//hkDBgzA5MmTMXv2bKf3EhMTkZycLAYPAOjWrRsEQcBvv/2Gm266yevXKScKHzIw1Fhkr4IA8oUQlgMIT9UP4p5SXS48jfOQO3iwHDo8BY1g1L59e7Rv395ru+zsbFy+fBllZWXIyMgAAOzatQt2ux1ZWVku98nIyEBYWBiKi4sxbNgwAMCRI0dQUVGB7Oxssd1PP/2Ee++9F2PHjsWrr77a7Dj9+vXDpk2bcPXqVbRs2RIAcPToURgMBtxwww2Sv+ZABe2AU7kpMWhVzkGpLA9C9XXwqZRBqkqs+0FriaiLl3EeSqzbwUrwSAi77PJF/NOtWzcMGjQIkyZNwt69e/Htt98iPz8fI0aMEGe6nD59Gmlpadi7dy8AICYmBnl5eSgoKMCXX36JsrIyjB8/HtnZ2ejTpw+A610t99xzDwYOHIiCggKYzWaYzWacP39e/OxRo0ahbdu2GD9+PH7++Wd8/fXXePbZZzFhwgSxy6W+vh7l5eUoLy9HfX09Tp8+jfLychw7dkw8ztWrV8U2AHDy5EmUl5ejoqJC0rWgyofM5O6KAeSrhLBcAfH5uDT4VDUsTbH1ldrjPPQSOihQqGfdunXIz8/HgAEDYDAYMGzYMLz11lvi+1arFUeOHEFtba247Y033hDbWiwW5ObmYtmyZeL7H3/8Mc6fP48PP/wQH374obi9Q4cO+OWXXwAALVu2xM6dO/Hkk08iMzMTbdu2xSOPPIJXXnlFbH/mzBn06tVL/PfChQuxcOFC9O/fHyUlJQCAffv24Z577hHbFBQUAADGjh2LNWvW+HwdQgRBkDzncenSpfjb3/4Gs9mMnj174u2330bv3r1dtl25ciU++OADHDx4EMD1EtL8+fPdtneluroaMTExyEl+DKGG/97UW6g7OMZfcs+SkaMrxp8l2f0JIAAkBRBfQ4WU8CGl+8WXcR++Hs+XcR/2SN8rRD492yXCt351X1c4lRI+fO12kRI+1Brr4U/Vg+exHVoEjdorNjzc8zCqqqp8GkPhD8d94s575yE0NLB+14aGOnyza56i5xvsJN9RNm7ciIKCAsydOxf79+9Hz549kZub22wQjUNJSQlGjhyJL7/8EqWlpUhJScHAgQNx+vTpwM685prrF2Pk7o6RoyuG5S4Yn47JyBohgTJc00evpxLjPVgOHnJTar0O6jIhLJP822/x4sWYNGkSxo8fj+7du2P58uWIiopyWh++sXXr1uGJJ55Aeno60tLS8N5778Fut6O4uNjtZ1gslmaLu/iM0UDC2ngQNQOIz8dXIFTIPU4jWMZ98DbFVs3gIWfVQ+7QQSGD8EJS+Kivr0dZWZnTSmsGgwE5OTkeV1prrLa2Flar1WnltaYKCwsRExMjvpou9CIZI4GEtUGpagUQrasfWgQGWu20OaXGe6g5zkOu4KFEtYMCB+GJpPBx4cIF2Gw2xMfHO233tNJaU8899xySkpKcAkxTM2fORFVVlfg6deqUlNP0jYZhRIml3JV8cm5TSlZAtOxSkXOpdTn5NN5DI6x0uUilZXeLUl0shPBE1dkur732GjZs2ICSkhJERLgfEGQymdyuJKeYxgFEhcGscq8NAvi3Sqo/M2BYQTNfpPN1sCkveKt6UPAg5DpJd5127drBaDSisrLSaburldaaWrhwIV577TV88cUXuO2226SfqZqaVkIUCiNKBBC1SJ2Cq/TzXwjblJ5iK4VWs1uom0UdxmsNMIZK71JuTGgIbH/inaS7QXh4ODIyMpwGizoGjzZeaa2p119/HS+//DKKioqcnubHDQW7Z5QYAyJ5Hz/GfhCidZcLL7NbaHwHIc1JrrcXFBRg7NixyMzMRO/evbFkyRLU1NRg/PjxAIAxY8YgOTkZhYWFAIAFCxZgzpw5WL9+PVJTU8WxIS1bthSXeOWKI4DIWA3htQLi7wJkXo9LS65L5+MaH76Qe6YLS1UPfwRS9Qj2bpb2xuvX7qpR2ZlyhD+Sw8fw4cNx/vx5zJkzB2azGenp6SgqKhIHoVZUVMBg+P2G9O6776K+vh4PPfSQ03ECfTSx5mquMRtAWB37QV0v+qHUs1x8xcNiYsEYPBxhgxBv/Lrb5OfnIz8/3+V7jiVYHRxLu+oSwwEkWNCgU/1QcpaL2t0twRQ8KHAQf/A5zYEljAYQtaofSnW9EPlpMdOF5y4Xf6seeh/fQWGDyIHChxx0FECUpmXXi+7HfQQJqV0urDy7xR8sBA8KG0QJFD7kwmgAkYqV6gd1pwQXpbpceO5u0Sp4UNggaqDwIScGAwiL1Q8iM0ZnuijR5aLGomIsdLeoGTwobBAtcNVZL9Sy8ZA4j2ReC0SJpdi9UfqZL3I/60XOJdlZXWKdJVrPdPGVmt0tvAWP9sar4osQLXBX+RBqa8X/DomK0vBMPGCsAkLVD8IyNZ7loiS5gofSoYOCBmEJV5WPpoTaWvHFHM4rIGo98VZtWjzhlgVqz3ThsctFy0GmSgYPqnAQFnFX+XCHyYoIQxUQ1qofvsx6oUGn+sf7QFM5qh5KBI9gDhvGaw0wGgN7yrdgo0dOKI3ryoc7jSsimldFOK+ASMVD9YOQpvypelDwIMR/ugwfTWkeRBgJIFIfOsfTA+fkHHRK2COly0Vq1UOr7ha5gwd1rxCeBEX4aEyzIMJpAPGHr9UPX2a9sBgq9DpuRK5ptr6O9+B5oGmgVQ8lggchPAm68NGY6kGEkQAiBU/VD195Cw803dY9XqbZ+kqr7hY5UfAgPArq8NGYakGEgQDCUvWDKE+LZ7rISckuFy3IWfWg4EF4ReHDBcUHrDIQQKRQsvohZ9cLi100xDUWuly0qHoEW/Bob7CgvcGCdga2B8oT9elmqq2SFJnGq/E0XDWm3tITb4mesBI8WA8d7SloEB9Q+JDIEURkCSEMrQPijT8PnCPeGa4ZYI+kLip3lOpyUXuGi56DB4UN4g+6m/hJqK1lMoBIwdrCY56wtuCYsQ6wRWh9FvKR84FyehRI1UNvwYPCBpED1cQDINt4EBnHgCg5/kOpJdflftCcHGjGS3O+zHTxZZqtr+M99DbQNBBaBw/H2A0KHkQuFD4CxHsAUWPmi5p8HXSq17U6iHdSu1y0rnpoGTwocBClULeLDHjvgpHS/UJjP0gwCcbgwXvYMNRYYDAGeAwb39eAB1T5kAlrFRCWngEjV9cLTaUNHrwPNOUteFC3ClEbhQ8ZsRZApJDS/aLHVU8JacrfqocSD4tTAgUOoiWuwodQUwvhKtsDy1gKIKwNPtUbb+NGjJz8Tud5pgtrA01Zn9lCgYOwgqvw4SBcrRFfLGIpgEih5OBTNbte5OqeoRkv8vNlpouULhcptHp6rVRKBA8KHIQ13I8cbBpAQlq20OhMnLEyCJUWHnNPb2t1KEmuaba88KfLhbVxHhQ2CMu4rHx4wlJVhMcKiN6m3hL9YqnLhaXgQVUOwgPdhY/GWAgiLAQQVsZ+8Nj1QvRBSpeL1KoHa8GDEB7oOnw0pmUQ4S2AUPWDEN+wNLOFggcfLl26hNGjRyM6OhqxsbHIy8vD1auew2ddXR2mTJmCtm3bomXLlhg2bBgqKyvF9y9evIhBgwYhKSkJJpMJKSkpyM/PR3V1tdNx1q1bh549eyIqKgqJiYmYMGECLl686NRmyZIluPnmmxEZGYmUlBRMmzYNdXXOY7VOnz6NP//5z2jbti0iIyPRo0cP7Nu3T9J1CJrw0ZgWQYSFAKKUYJ75opcZL7xhcW0Pf8lR9aDgwY/Ro0fjp59+ws6dO7Ft2zZ8/fXXmDx5ssd9pk2bhk8//RSbNm3CV199hTNnzuDBBx8U3zcYDPjTn/6ErVu34ujRo1izZg2++OILPPbYY2Kbb7/9FmPGjEFeXh5++uknbNq0CXv37sWkSZPENuvXr8eMGTMwd+5cHDp0CKtWrcLGjRsxa9Yssc1//vMf9OvXD2FhYfjss8/w888/Y9GiRWjdurWk68DvaEGZNA4gSg9W1XoQqpTBp0o9dC70mh0NkZ4zb2itHQ1RXtr48KA5OR5GF1oHNNCg1KARyIqm/tD6mS1qaWcM1/oUmHDo0CEUFRXh+++/R2ZmJgDg7bffxv3334+FCxciKSmp2T5VVVVYtWoV1q9fj3vvvRcAsHr1anTr1g3fffcd+vTpg9atW+Pxxx8X9+nQoQOeeOIJ/O1vfxO3lZaWIjU1FU899RQAoGPHjvjLX/6CBQsWiG12796Nfv36YdSoUQCA1NRUjBw5Env27BHbLFiwACkpKVi9erW4rWPHjpKvRVBWPtxRoxqi1wpIMFc/iHyUmmarZ6xXPdpzHDyqq6udXhZLYNe6tLQUsbGxYvAAgJycHBgMBqcbfGNlZWWwWq3IyckRt6WlpeHGG29EaWmpy33OnDmDzZs3o3///uK27OxsnDp1Cjt27IAgCKisrMTHH3+M+++/X2zTt29flJWVYe/evQCAEydOYMeOHU5ttm7diszMTDz88MOIi4tDr169sHLlSsnXgsKHCyzMlCGEJ3I+zZYXLI33YJUmwaP22vU/zgJ51V7/4y4lJQUxMTHiq7CwMKBTM5vNiIuLc9oWGhqKNm3awGw2u90nPDwcsbGxTtvj4+Ob7TNy5EhERUUhOTkZ0dHReO+998T3+vXrh3Xr1mH48OEIDw9HQkICYmJisHTpUrHNqFGj8NJLL+HOO+9EWFgYOnfujLvvvtup2+XEiRN49913cdNNN+Hzzz/H448/jqeeegpr166VdC0ofLihZADhpfoRDANP6em23vmyxgeRTs9dLu2N4VxXPBxOnTqFqqoq8TVz5kyX7WbMmIGQkBCPr8OHDyt+vm+88Qb279+P//3f/8Xx48dRUFAgvvfzzz9j6tSpmDNnDsrKylBUVIRffvnFaVxISUkJ5s+fj2XLlmH//v3YvHkztm/fjpdffllsY7fbcfvtt2P+/Pno1asXJk+ejEmTJmH58uWSzjXox3x4IlytYWbRMkLIdXoabBooFrtc9BA6HKKjoxEdHe213TPPPINx48Z5bNOpUyckJCTg3LlzTtsbGhpw6dIlJCQkuNwvISEB9fX1uHz5slP1o7Kystk+CQkJSEhIQFpaGtq0aYO77roLL7zwAhITE1FYWIh+/frh2WefBQDcdtttaNGiBe666y688sorSExMxAsvvIBHH30UEydOBAD06NEDNTU1mDx5Mp5//nkYDAYkJiaie/fuTp/brVs3/POf//R6nRqj8OGF3gKIkiueEkKCm56ChxTt27dH+/btvbbLzs7G5cuXUVZWhoyMDADArl27YLfbkZWV5XKfjIwMhIWFobi4GMOGDQMAHDlyBBUVFcjOznb7WXb79TWTHONUamtrERrqfMs3Go0AAEEQxDYGg8Fjm379+uHIkSNObY4ePYoOHTp4+eqdUfjwgRIBROuZL4QQZ2rOdNHb9NpgDR1SdevWDYMGDRK7KaxWK/Lz8zFixAhxpsvp06cxYMAAfPDBB+jduzdiYmKQl5eHgoICtGnTBtHR0XjyySeRnZ2NPn36AAB27NiByspK3HHHHWjZsiV++uknPPvss+jXrx9SU1MBAEOGDMGkSZPw7rvvIjc3F2fPnsXTTz+N3r17i589ZMgQLF68GL169UJWVhaOHTuGF154AUOGDBFDyLRp09C3b1/Mnz8fjzzyCPbu3YsVK1ZgxYoVkq4FhQ8f6a0CogTen/VCiK9osOnvKHhIs27dOuTn52PAgAEwGAwYNmwY3nrrLfF9q9WKI0eOoLbR2MA33nhDbGuxWJCbm4tly5aJ70dGRmLlypWYNm0aLBYLUlJS8OCDD2LGjBlim3HjxuHKlSt455138MwzzyA2Nhb33nuv01Tb2bNnIyQkBLNnz8bp06fRvn17DBkyBK+++qrY5o477sAnn3yCmTNn4qWXXkLHjh2xZMkSjB49WtJ1CBEctRSGVVdXIyYmBveGP4zQEPnXnpBCzgAiS+UDkFz5kNLtInWtD1/Ch7d1PgB4XecDgE9rePjSxtvD5byt8+Ftf5uXy22P9LykvN3k5Uc0wubxbaPJ8/umCM8DSuV6qJxcT7RVasyHlMpHIOFDT0up+xo8qq/YkXTzb6iqqvJpDIU/HPeJnOTHEGoIrGu5wW7BF6eXK3q+wY5mu0gUbNNwlZjx4sszXggh7mkdPPQym4Voh8KHH+QKILxMudUCPWCOBErvM120QqGDyIHCByEk6Kk12JT3tT0oeBC5UPjwE8/dL1KecEuIHOQa70G06XKhbhYiN5qaoDG9TbmlGS9ESVIGmyol2Ga68BY6hNprEAyeB1l7PYadVvVVGlU+CCGEuMRb8CD8oD9RA0BrfxBCfMXTwmIUOojSqPLBANlmvSgkGB4wR/znyxofhB8UPIgaKHzoiYQpt1oPOqW1Pggr1FxWnRByHYWPAPE864UQvVJqjQ8tB5uq0eVCVQ+iFgofjGC964UQ4j8e1veg4EHUROGDyM54rUHrU5DE6H0JCkI0pXTVg4IHURuFDxlQ14syfFlinXjm7aFyhBCiBQofDJGl60WhQac044UoSe6n2RLfUdWDaIHCB9E9ergccUeNmS6BjvdQssuFggfRCoUPmVDXizbUCBahOh4TYoqgZaR9pbdl1Sl4EC3RCqeEEEJ0Q6iphRASWDexIFA3s9Ko8sEYvUy59WXGCy00Rog2qOpBtEbhQ0bMdL1IGHRK9M1mMWp9CqpTaoExf7A43oOCB2EBhY8gRzNeAkfrhBBeUPAgrKDwwSC9dL3Igdb60FYLU+ADUmNN/KczvQ02JURrFD4IISQIUNWDsITCh8yYGfdBCNEUS+M9KHgQ1lD4YBR1veiHUfmHkRI/qLHAGAsoeBAW+RU+li5ditTUVERERCArKwt79+712H7Tpk1IS0tDREQEevTogR07dvh1skQCBpZZV+sBc7SCqf7R0uqE6Ivk8LFx40YUFBRg7ty52L9/P3r27Inc3FycO3fOZfvdu3dj5MiRyMvLw4EDBzB06FAMHToUBw8eDPjkWUVdL4QQFlDVg7BKcvhYvHgxJk2ahPHjx6N79+5Yvnw5oqKi8P7777ts/+abb2LQoEF49tln0a1bN7z88su4/fbb8c477wR88noXDF0vrCw0RtNliZxYGO9BwYOwTNLy6vX19SgrK8PMmTPFbQaDATk5OSgtLXW5T2lpKQoKCpy25ebmYsuWLW4/x2KxwGL5/YevqqoKANDA0ZK3IXZ5npkRYg9gkSib79nSLuHJ67YG3xvbGrx/izVYPZ+nt/cBwGYI8fh+SD3QEOG5jc3D257eAwBvV8TuqWeoDrBHug9hdo87AxA8f3qIzf37Nrvnnymbzfv3cUOD5xultcH7MeqtnttYwnz72W8bVoM6H+/b10J97xasDZXwAwLgqjGwUB1hCGz/dsZwVIONYA8AV65ePxdBUL6LtAFWIMCPaQA/9xpeSQofFy5cgM1mQ3x8vNP2+Ph4HD582OU+ZrPZZXuz2ez2cwoLC/Hiiy822/61dYuU09WWXM/r+o9MxyGEEI1dvHgRMTExihw7PDwcCQkJ+Nq8RZbjJSQkIDycqkdKYfLBcjNnznSqlly+fBkdOnRARUWFYt+4elBdXY2UlBScOnUK0dHRWp8Os+g6eUfXyDd0nXxTVVWFG2+8EW3atFHsMyIiInDy5EnU18vzl194eDgiIiJkORZpTlL4aNeuHYxGIyorK522V1ZWIiEhweU+CQkJktoDgMlkgslkarY9JiaGfsB9EB0dTdfJB3SdvKNr5Bu6Tr4xGJRd3SEiIoICAyckfSeEh4cjIyMDxcXF4ja73Y7i4mJkZ2e73Cc7O9upPQDs3LnTbXtCCCGE6JvkbpeCggKMHTsWmZmZ6N27N5YsWYKamhqMHz8eADBmzBgkJyejsLAQADB16lT0798fixYtwuDBg7Fhwwbs27cPK1askPcrIYQQQggXJIeP4cOH4/z585gzZw7MZjPS09NRVFQkDiqtqKhwKq317dsX69evx+zZszFr1izcdNNN2LJlC2699VafP9NkMmHu3Lkuu2LI7+g6+Yauk3d0jXxD18k3dJ1IUyGCGnOfCCGEEEL+i57tQgghhBBVUfgghBBCiKoofBBCCCFEVRQ+CCGEEKIqCh+EEEIIURUz4WPp0qVITU1FREQEsrKysHfvXo/tN23ahLS0NERERKBHjx7YsWOHSmeqLSnXaeXKlbjrrrvQunVrtG7dGjk5OV6vqx5I/V5y2LBhA0JCQjB06FBlT5ARUq/T5cuXMWXKFCQmJsJkMqFr165B8XMn9TotWbIEN998MyIjI5GSkoJp06ahrk7fj03++uuvMWTIECQlJSEkJMTjg0MdSkpKcPvtt8NkMqFLly5Ys2aN4udJGCIwYMOGDUJ4eLjw/vvvCz/99JMwadIkITY2VqisrHTZ/ttvvxWMRqPw+uuvCz///LMwe/ZsISwsTPjxxx9VPnN1Sb1Oo0aNEpYuXSocOHBAOHTokDBu3DghJiZG+O2331Q+c/VIvUYOJ0+eFJKTk4W77rpL+NOf/qTOyWpI6nWyWCxCZmamcP/99wvffPONcPLkSaGkpEQoLy9X+czVJfU6rVu3TjCZTMK6deuEkydPCp9//rmQmJgoTJs2TeUzV9eOHTuE559/Xti8ebMAQPjkk088tj9x4oQQFRUlFBQUCD///LPw9ttvC0ajUSgqKlLnhInmmAgfvXv3FqZMmSL+22azCUlJSUJhYaHL9o888ogwePBgp21ZWVnCX/7yF0XPU2tSr1NTDQ0NQqtWrYS1a9cqdYqa8+caNTQ0CH379hXee+89YezYsUERPqRep3fffVfo1KmTUF9fr9YpMkHqdZoyZYpw7733Om0rKCgQ+vXrp+h5ssSX8DF9+nThlltucdo2fPhwITc3V8EzIyzRvNulvr4eZWVlyMnJEbcZDAbk5OSgtLTU5T6lpaVO7QEgNzfXbXs98Oc6NVVbWwur1arokyW15O81eumllxAXF4e8vDw1TlNz/lynrVu3Ijs7G1OmTEF8fDxuvfVWzJ8/HzabTa3TVp0/16lv374oKysTu2ZOnDiBHTt24P7771flnHkRjL/DiTPJy6vL7cKFC7DZbOLy7A7x8fE4fPiwy33MZrPL9mazWbHz1Jo/16mp5557DklJSc1+6PXCn2v0zTffYNWqVSgvL1fhDNngz3U6ceIEdu3ahdGjR2PHjh04duwYnnjiCVitVsydO1eN01adP9dp1KhRuHDhAu68804IgoCGhgY89thjmDVrlhqnzA13v8Orq6tx7do1REZGanRmRC2aVz6IOl577TVs2LABn3zyCT1y+r+uXLmCRx99FCtXrkS7du20Ph2m2e12xMXFYcWKFcjIyMDw4cPx/PPPY/ny5VqfGlNKSkowf/58LFu2DPv378fmzZuxfft2vPzyy1qfGiFM0bzy0a5dOxiNRlRWVjptr6ysREJCgst9EhISJLXXA3+uk8PChQvx2muv4YsvvsBtt92m5GlqSuo1On78OH755RcMGTJE3Ga32wEAoaGhOHLkCDp37qzsSWvAn++lxMREhIWFwWg0itu6desGs9mM+vp6hIeHK3rOWvDnOr3wwgt49NFHMXHiRABAjx49UFNTg8mTJ+P55593euhmMHP3Ozw6OpqqHkFC85+E8PBwZGRkoLi4WNxmt9tRXFyM7Oxsl/tkZ2c7tQeAnTt3um2vB/5cJwB4/fXX8fLLL6OoqAiZmZlqnKpmpF6jtLQ0/PjjjygvLxdfDzzwAO655x6Ul5cjJSVFzdNXjT/fS/369cOxY8fEcAYAR48eRWJioi6DB+DfdaqtrW0WMByBTaBneIqC8Xc4aULrEa+CcH06m8lkEtasWSP8/PPPwuTJk4XY2FjBbDYLgiAIjz76qDBjxgyx/bfffiuEhoYKCxcuFA4dOiTMnTs3aKbaSrlOr732mhAeHi58/PHHwtmzZ8XXlStXtPoSFCf1GjUVLLNdpF6niooKoVWrVkJ+fr5w5MgRYdu2bUJcXJzwyiuvaPUlqELqdZo7d67QqlUr4R//+Idw4sQJ4f/7//4/oXPnzsIjjzyi1ZegiitXrggHDhwQDhw4IAAQFi9eLBw4cED49ddfBUEQhBkzZgiPPvqo2N4x1fbZZ58VDh06JCxdupSm2gYZJsKHIAjC22+/Ldx4441CeHi40Lt3b+G7774T3+vfv78wduxYp/YfffSR0LVrVyE8PFy45ZZbhO3bt6t8xtqQcp06dOggAGj2mjt3rvonriKp30uNBUv4EATp12n37t1CVlaWYDKZhE6dOgmvvvqq0NDQoPJZq0/KdbJarcK8efOEzp07CxEREUJKSorwxBNPCP/5z3/UP3EVffnlly5/1ziuzdixY4X+/fs32yc9PV0IDw8XOnXqJKxevVr18ybaCREEqgUSQgghRD2aj/kghBBCSHCh8EEIIYQQVVH4IIQQQoiqKHwQQgghRFUUPgghhBCiKgofhBBCCFEVhQ9CCCGEqIrCByGEEEJUReGDEEIIIaqi8EEIIYQQVVH4IIQQQoiq/n9PR+QKZDZpBAAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAG3CAYAAABv3vUDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB6MklEQVR4nO3de1xUdf4/8BczMDNcBFQQxFC8huYFxUIs0zYS07W1bL3kQ801XUv6pXRRU8GummvmmhabXbRdTXPXWlMXM9TcErVQ9lumlLcwFZRYQFCGYeb8/nDnOAPDzJyZc/mcM+/n4zGPauYz53w46cyL9+dygjiO40AIIYQQwgid0h0ghBBCCHFE4YQQQgghTKFwQgghhBCmUDghhBBCCFMonBBCCCGEKRROCCGEEMIUCieEEEIIYQqFE0IIIYQwhcIJIYQQQphC4YQQQgghTKFwQgjjDh48qHQXCCFEVhROCGHUyZMnAQCVlZW4ePGiwr0hhBD5UDghhFHnzp3Dl19+iaVLl+Lvf/+70t0hhBDZBNFdiQlhD8dxGDJkCLp06YL+/ftj7ty5SneJEEJkQ5UTQhg1depUfPjhh+jevTuuXr2qdHcIIUQ2FE4IYVBQUBASExNx/PhxrF69Gn/5y1+U7hIhhMiGwgkhCtiwYQP69u2L0NBQpKamorCwEOPGjUO/fv0A3BjWWbNmDRYuXIjf/OY3eOaZZxTuMSGEyIfCCSEyW7p0KR599FF069YNq1evxqBBg/DAAw/g6NGj6NOnD9/uzjvvxKeffophw4Yp11lCCFFAsNIdICSQHD16FIsXL8aCBQvw6quv8s+bzWa89957eOyxxwDcGNYZMGAAAGDQoEGK9JUQQpRClRNCZPTqq6+iffv2eOGFF5yeHzJkCAA4VU4yMzNl7RshhLCCwgkhMjGbzfjXv/6F8ePHIyQkxOm12tpaAM7hhBBCAhWFE0Jkcvr0aVy7do2f9OrozJkziIyMRMeOHRXoGSGEsIXCCSEyuX79OgBAp3P+a2c2m/HRRx+hd+/eSnSLEEKYQ+GEEJl06tQJAPDVV185Pb9s2TJcunSJwgkhhPwPrdYhRCYxMTEYMWIE3nnnHQQHB6Nv37744osvUFRUBIDmmxBCiB3dW4cQGV2+fBl//OMfsXfvXoSHh2PcuHEYNmwYHnzwQRw6dAhpaWlKd5EQQhRH4YQQhWVlZWHLli24dOkSgoOpmEkIITTnhBCZmM1mNP1d4MCBA/jLX/6CWbNmUTAhhJD/ocoJITLZvn07XnzxRTz88MOIjo7G0aNH8cEHH6B///748ssvERoaqnQXCSGECfSrGiEyadOmDQwGA5YtW4Zr164hKSkJ8+bNw4IFCyiYEEKIA6qcEEIIIYQpguecHDhwAKNHj0ZCQgKCgoLw6aefenzP/v37MWDAABiNRnTr1g3r16/3oauEEEIICQSCw0ldXR369euHtWvXetX+7NmzGDVqFO655x4UFxdjzpw5eOyxx7B7927BnSWEEEKI9vk1rBMUFIRPPvkEY8aMabHNvHnzsHPnTnz//ff8cxMmTEBVVRXy8/N9PTUhhBBCNEryCbGFhYXIyMhwei4zMxNz5sxp8T1msxlms5n/b5vNhsrKSrRt2xZBQUFSdZUQQogEOI7D1atXkZCQ0OzeUmKqr69HQ0ODKMcyGAwwmUyiHIsIJ3k4KSsrQ1xcnNNzcXFxqKmpwfXr112uUli6dCleeOEFqbtGCCFERufPn8ctt9wiybHr6+uR1Ckc5ZdtohwvPj4eZ8+epYCiECaXEi9YsADZ2dn8f1dXV6Njx464O2QMgoNCFOwZEBQe5v8xwvxcNurD+23hRq/bWkO9+2PhbTsAaDS5/22pMczzb1NWo/uqWaPJc1XN6uEyNLr5HPL0Xpun10Nb/tC0GTyMrpqsbl/WG9y/bjBZWnwt3Oj+N81wg9nt6wAQZfTcpo2hzu3rrUOueTxG2xD3x3BqG3zV67ZxwdXeH1fvfR+aitF5vk5eHUdvEOU4crlaa8OtqRfRqlUryc7R0NCA8ss2lHybgFat/KvOXL1qw60DL6KhoYHCiUIkDyfx8fEoLy93eq68vByRkZEt7u1gNBphNDb/pA8OClE0nARFhItzHJ33QcElvfD32wS8JyjYu2sc5OWOpo2hOs9/0EK8+DAxuA8fnIfXAQAeLgPn7nVPn1GeLrHJzW90Rv/Cic7oIby4CYd6D6GvHkZEeAgfIZ76D8BgbDkgAYAxxP3rAGAK8f4jK1TAjrthIXqv20boff/iayXSkEakH31QkhzD8q1a6RDpZzghypP8/2B6ejoKCgqcntuzZw/S09OlPjXxgTVM2coUIVp2xVN5jRACwIdwUltbi+LiYhQXFwO4sVS4uLgYpaWlAG4MyUyZMoVvP2vWLJw5cwbPPfccTp48ibfeegsff/wx5s6dK85PIBOxqibEN94M2XhipeqsS3VmNoYIKi30d8xbV6ziTPokhFWCw8m3336L/v37o3///gCA7Oxs9O/fHzk5OQCAS5cu8UEFADp37oydO3diz5496NevH15//XW8++67yMzMFOlHkJ6YwSQozM85K+G0zTkh3qho9H5+Q5kl2uu2V6wRPvSGECKE4Dknw4YNa3ZnVUeudn8dNmwYjh07JvRUTNBCxUTIZFhCfFFlNiHaWC/5eSosEYgJqZX8PFK6YjMiVqSJsYRoFc0aIkQN6r2fsEkCAw3tEC2jcOKGFqomUhGyjJgQQggRgsJJC6QIJjTfhG3u9jiRms6s/Z2PfzVT2LejVTuEuEfhxAWqmBCiDVqfFEtDO0SrKJwQogFWM81JIYRoB00caEJrVRNB29bTBmxEYZWWcLTxsEW9FlbsALRqRyoV1gaYrX5uX28V5/48xHdUOXEgZTAJ1PkmjaGe/4h5c18dQohrNLRDtIi+Ff5HaxUTQgghRK0onBASAMz1/m1RX2v2PDxYZab7AwhFq3YIcY3CCaSvmvg9pEMI8ZnWV+wANLRDtCfgw4kqhnMYm29CG7ARX9FeJ4QQbwR0OJEjmChZNQmke+qIcddiQgghbAjocEJuomXEga3O7N+cFLlVWNgZUvGXWPNOaGiHaEnAhhNVDOcQQgghASjgwklQRLhswUSUIR3G5psQIrVKi7K/OKh1UiwhWhJQ4YSqJYQEJiErdpRAQzuEOAuYcBJowYSFybBi7Q5Lk10JISSwaH5NqFKhRE17m9BkWEKIVlTYjKi3+fd7d62N7q2jNE1XTlRfLaH5JoS0iJUVO2LNO6GhHUJu0mw4UX0wYRRtwEYCgZBJsYQQ8WkunMi5GqfFPqhoSIcQoj1UPSFqp6lwonQoERUN6RDGiHXzPy1uYc/a0A5AAYWom2bCiaaCiZ9YWKlDiD+k2OuE9eXEUqCAQtRK9eGEhWEcR2rbeI1W6hCibRRQiBqpOpywFEoCAU2GJaxhZcUOwObQDn9MCihEZVQbTlgMJjQR9iZvNmCTk9XzVAhCnGhtxQ4FFGmtXbsWSUlJMJlMSEtLw5EjR9y237p1K5KTk2EymdCnTx/s2rXL6XWO45CTk4P27dsjNDQUGRkZ+Omnn5zavPLKKxg8eDDCwsIQHR3t8jwFBQUYPHgwWrVqhfj4eMybNw+NjY386yUlJbjnnnsQFxcHk8mELl26YNGiRbBYLHyb48ePY+zYsUhKSkJQUBBWrVol7OL4gK1vEC+wNowjOpoISwiRCAUUaWzZsgXZ2dnIzc3F0aNH0a9fP2RmZuLy5csu2x88eBATJ07E9OnTcezYMYwZMwZjxozB999/z7dZvnw5Vq9ejby8PBw+fBjh4eHIzMxEfX0936ahoQG///3v8fjjj7s8z3/+8x+MHDkSI0aMwLFjx7BlyxZs374d8+fP59uEhIRgypQp+Pzzz1FSUoJVq1Zh3bp1yM3N5dtcu3YNXbp0wbJlyxAfH+/v5fJKEMdxnCxn8kNNTQ2ioqJwb+upCNaxe2t3FuabCJ0MK2TOiZBhHTm3rvemjTeVk0YPbTwdw+rh0ttC3e86aTN6+Ktosrp9WW90/7rR5P6LKdzo+Ysrwmj22CbaWO+xTVtjncc2bUI8twGAmJBar9oBQEzwVa/bAkB8SJWg9rF67/vi8Vg6z9fa52Pr5f0crblqQ8Ktv6C6uhqRkZHSnON/3xN7v09ERCs/d4i9asNvep/H+fPnnfprNBphNDb/i56Wlobbb78da9asAQDYbDYkJibiySefdAoCduPHj0ddXR127NjBPzdo0CCkpKQgLy8PHMchISEBTz/9NJ555hkAQHV1NeLi4rB+/XpMmDDB6Xjr16/HnDlzUFVV5fT8888/jz179uCbb77hn/vss88wbtw4XL58Ga1auZ4knp2djW+++Qb//ve/m72WlJSEOXPmYM6cOS7fKxbVVU5YRUM6hLBP6hU7arlLsZYrKL9aw3HFGuHX41frjep8YmIioqKi+MfSpUubna+hoQFFRUXIyMjgn9PpdMjIyEBhYaHLPhYWFjq1B4DMzEy+/dmzZ1FWVubUJioqCmlpaS0e0xWz2QyTyfm3qtDQUNTX16OoqMjle06dOoX8/HwMHTrU6/NIgcIJS2Qe0qGVOsRRnZndqiQRn5YDiljOnz+P6upq/rFgwYJmbSoqKmC1WhEXF+f0fFxcHMrKylwet6yszG17+z+FHNOVzMxMHDx4EB999BGsVisuXLiAF198EQBw6dIlp7aDBw+GyWRC9+7dMWTIEL6dUiiciICF4RypiT2kQ4hYpFyx48ukWJZX7TQ7h7WBQoobkZGRTg9XQzosGz58OP70pz9h1qxZMBqN6NGjB0aOHAngRnXH0ZYtW3D06FFs2rQJO3fuxIoVK5ToMo++RfzE0nAObb4W2KxmvdvXzfX+V0bk3CVWio3YAHk2Y1NTQAFuhhQKKsLFxMRAr9ejvLzc6fny8vIWJ4/Gx8e7bW//p5BjtiQ7OxtVVVUoLS1FRUUFfve73wEAunTp4tQuMTERvXr1wsSJE7Fs2TIsWbIEVqv7eWxSonDiB9GCiQhVE1YmwhIJ1bsPH4GMteoJoL6Awp+PgoogBoMBqampKCgo4J+z2WwoKChAenq6y/ekp6c7tQeAPXv28O07d+6M+Ph4pzY1NTU4fPhwi8d0JygoCAkJCQgNDcVHH32ExMREDBgwoMX2NpsNFosFNpv7SfxSom8dH4haLVEgmEiJhnSIGlQ0thK8cscXV6wRoqzguWIzSrp6p8XzOgQUuVf4qEl2djamTp2KgQMH4o477sCqVatQV1eHadOmAQCmTJmCDh068BNqn3rqKQwdOhSvv/46Ro0ahc2bN+Pbb7/FO++8A+BGmJgzZw5efvlldO/eHZ07d8bixYuRkJCAMWPG8OctLS1FZWUlSktLYbVaUVxcDADo1q0bIiJuhOM//elPGDFiBHQ6HbZt24Zly5bh448/hl5/45edjRs3IiQkBH369IHRaMS3336LBQsWYPz48QgJufFLbENDA3744Qf+3y9cuIDi4mJERESgW7duklxTCidekmT4RqFgoqWJsGItIybeqzUbPS4prjKbPC4p/tUc7nFJcaUl3OslxRWWCEHLioUos0QLXlZsp/aAwp+fgkqLxo8fjytXriAnJwdlZWVISUlBfn4+P6G1tLTUaY7H4MGDsWnTJixatAjPP/88unfvjk8//RS9e/fm2zz33HOoq6vDzJkzUVVVhbvuugv5+flOq29ycnKwYcMG/r/79+8PANi3bx+GDRsGAPjXv/6FV155BWazGf369cM///lP3H///fx7goOD8dprr+HHH38Ex3Ho1KkTsrKyMHfuXL7NxYsX+WMDwIoVK7BixQoMHToU+/fvF+ciNkH7nHgg2ZwSkSbASh1OhA7peFs5YW2PkxvH8u84ku9zAjCx1wnA5n4ngLA9TwBh+574Gk7sxNoDRcmA4oo3QUXOfU62/icZYa38GwK9dtWK3/c7KWl/iXtUg3chKCyMf0hCwWAiJTUP6XgKJoFC7cuJWZx7YqfWOSie0BwVIgX1fpuITPJAIjJfg4mWhnRIc3Ks2AHYXrUjJKDIsXLHkVYDih0FFSKWgA4nigQSFU2AlWpvE2+GdEgLaMWO6IQEFDFuBqj1gGJHQYX4I+C+JRStkCgcTKhqQgIFy8M7QOAEFLsKCihEoIBYrcPEUI2KKiYA7W1CxKHEqh1fyLW02JFWVvGwprwxCqEW/z6/rjc2itQb4itNV06YmUPCwNb0UlZNpBjSkXulDhHOm3knSmK9egIEXgWFEG9pKpw4DtkwEUoAJlbm0HAOEULtK3Z8JffkWDsKKIQ0p+pwwmQYccRAMPFFoA3paG0ZsVwrdryl5L121FA9ASigENKUqsJJUFgo22HELjyUmWAiddWEVumIR2f2PExFpKVU9QSggEKII/q2EJuI80tYDyZS8WYuCSHeUkv1BBA3oNgfhKgRhRMxMRRMfEV7m7hGE2td83ZSrJJDO0IJrZ6wGlD441FQISqk7m8MljCwIseRL1WTQJtrQgKH0OqJ1gIKf1wKKkQlKJyIQeRgEqjDOYD3Qzq0jFhaLK/YkaN64gu1BBT++BRUCMMonPhDxImvdmoYzgG0M6SjtZU6dqyt2AHEG9rxldTVE0B9AYU/DwUVwhh2vzVYJ8EwjhjBRM1VE0K8xWr1BFBvQOHPR0GFMIDCiRD2SgmjwcRXUlZNBB2XVumoEus7xTqSo3oCqD+g8Od1CCoUVoicaAakJzJMdBUrmLA4CZblIR1v0dwVeXlzrx1A+vvt+KvMEo34kCrRjifWvXj86oNDQGH1fj6/NraCqdG/z7V6ureO4tT/zSEViSokjmzhRkWDiVpRhUUeUkyK9WbeidTkqp4A2qmguEIVFSIlCieOJBy2aUrMYRxfg4kvVRMa0iEsYXnuiZ2WA4odBRUitsAOJ45hRMZ9StQaTISSYkjH2xDjzVCMVlfqeEvMFTtizzuRctUOIG/1BAiMgGJHQYWIIbDCiUJhxBELwcRXUlVNtMBKn8OKkat6QgFFOHtIqaCgQgTS9rcNA2HEkZIrchzJMZwjpGpCQzqBhYV5J4C099xpSSAGFEJ8oa1wwlgYsRNz4qsdy8M5UqEQIz8ld4oVMrSjluoJQAGFEG+oO5wwGkYcSVEt0dJwDuuBg5YRa5sS1ROAAgohnqgrnISxH0YcsRZM5KqaKL23CQUKZSi9GZuaqicABRRC3FFXOFERrQQTVibBsl5hIdKQetWO0iigEOIaG988GiPF/BIlNlmTOpgoGTjEXEZMK3WEk2pSrC/VE1+GdsSqngAUUAhxxadvn7Vr1yIpKQkmkwlpaWk4cuSI2/arVq3CrbfeitDQUCQmJmLu3Lmor6/3qcMsY2niq9MxZJwES0M66uLNXidKTooNFFIEFAopRM0Ef2tt2bIF2dnZyMvLQ1paGlatWoXMzEyUlJSgXbt2zdpv2rQJ8+fPx/vvv4/Bgwfjxx9/xKOPPoqgoCCsXLlSlB+CBawN4/jLl6qJVMGEhnQCm7f32rHz5Z47FZYIxIQIu29NRWMrxARfFfQed8S+Fw/QvIqi9L155PCrJRxGi3+fnWaLRaTeEF8J/jZZuXIlZsyYgWnTpqFXr17Iy8tDWFgY3n//fZftDx48iDvvvBOPPPIIkpKSMHz4cEycONFjtUVNWA4mLG1R73QOFQQOqsIIp/SkWLmJObwDiF9BacpeUaGqCmGdoG+hhoYGFBUVISMj4+YBdDpkZGSgsLDQ5XsGDx6MoqIiPoycOXMGu3btwsiRI1s8j9lsRk1NjdODVSzPLwmk4RyA7TBhC7WJd7B6vXjH0ii55p5IQeqAYkdBhbBM0LdXRUUFrFYr4uLinJ6Pi4vDyZMnXb7nkUceQUVFBe666y5wHIfGxkbMmjULzz//fIvnWbp0KV544QUhXZONlLu8snBnYZaGcwB1VFgCQZ3ZgHBjg6jHrDKbEG1ka+6Zr8M7AJgf4nEnEId/CNsk/3V3//79ePXVV/HWW2/h6NGj2LZtG3bu3ImXXnqpxfcsWLAA1dXV/OP8+fNSd9Ml+wRXx4cUpFiNw+pwDqB84Aj0G/5JTYqhHV+WFPu674mvFRQphnjkqqI05VhVoeoKUYKgb7CYmBjo9XqUl5c7PV9eXo74+HiX71m8eDEmT56Mxx57DADQp08f1NXVYebMmVi4cCF0uuZfiEajEUajfGPXSt7zRopqSaAN5wBsD+kIoTMHwWbkRDmW1ayH3mgV5ViA8tUToRNjAd8mxwK+VVAA6aooAGStpLjSUkChKguRgqBvFoPBgNTUVBQUFPDP2Ww2FBQUID093eV7rl271iyA6PU3xsw5TpwPYW+5qoQoFUykqpbIudma1MFE6QoL00Scd+LNcmKhhFRPpL4RoNwVFOBGSNFSJcUdqrIQKQj+JsvOzsbUqVMxcOBA3HHHHVi1ahXq6uowbdo0AMCUKVPQoUMHLF26FAAwevRorFy5Ev3790daWhpOnTqFxYsXY/To0XxIkQIrdwB2hbVqiZzBROnAodSQju66TtxJsQoRUj2pNRsRYTSLen5fqieA/BUU/v0amI/ii6YB5ZpVvAoeCQyCv2HGjx+PFStWICcnBykpKSguLkZ+fj4/Sba0tBSXLl3i2y9atAhPP/00Fi1ahF69emH69OnIzMzEX/7yF1F+AJaqIZ6wVi0B5N2eXqpgopUhHSlYzeL/AiDFpmxCqie+bmmvRAWFP4bIlRRWqyiBTOjmpFu3bkVycjJMJhP69OmDXbt2Ob3OcRxycnLQvn17hIaGIiMjAz/99JNTm1deeQWDBw9GWFgYoqOjm53j119/xYgRI5CQkACj0YjExERkZWU1WwG7f/9+DBgwAEajEd26dcP69ev9/vn85dM3U1ZWFn7++WeYzWYcPnwYaWlp/Gv79+93+sGCg4ORm5uLU6dO4fr16ygtLcXatWtdXkhP1BJCXGEtlPiDlXkmhH1S7XuixoACaGvSLLnJvjlpbm4ujh49in79+iEzMxOXL1922f7gwYOYOHEipk+fjmPHjmHMmDEYM2YMvv/+e77N8uXLsXr1auTl5eHw4cMIDw9HZmam0+7qDQ0N+P3vf4/HH3/c5Xl0Oh1+97vfYfv27fjxxx+xfv16fPHFF5g1axbf5uzZsxg1ahTuueceFBcXY86cOXjsscewe/dun38+MQRxck/88EFNTQ2ioqLwm5T5CNarJ5AA7A3hOGJ5OEdIeyGVEyHDOt4e19t763g7rCNoQqzJc7nc20mxRpOwya7eDu8IGdoRurTYlyEeAD4N8QDwa4in2bFEHOqxY3W459pVK37f7ySqq6sRGRkpyTns3xP/76vfwRjh5w6xtRasvuufXvc3LS0Nt99+O9asWQPgxlzMxMREPPnkk5g/f36z9uPHj0ddXR127NjBPzdo0CCkpKQgLy8PHMchISEBTz/9NJ555hkAQHV1NeLi4rB+/XpMmDDB6Xjr16/HnDlzUFVV5bGvq1evxp/+9Cd+Fey8efOwc+dOp2A0YcIEVFVVIT8/36efTwz0K7BEWBzCcaSF4RyhAnUJsRRDO0JIOTlWrRUUILAmzapV081AzebmQduXzUkLCwud2gNAZmYm3/7s2bMoKytzahMVFYW0tLQWj+mNixcvYtu2bRg6dKjXffHl5xMDhRMR2QMJy9USwPdg4kvVROpgIuV8Ez1b+4MxR8jcEym3tVdzQAGkDSmOj0DxX0sYKi3hfj3+awkDACQmJiIqKop/2Bd6OHK3OWlZWZnLPpaVlbltb/+nkGO6M3HiRISFhaFDhw6IjIzEu+++67EvNTU1uH79uk8/nxgonIhAqkACiD+3RM5g4itWKi2qmGjL+JJiX/iytFjtAQWQJqQ4chVYAim0+OL8+fNOG4IuWLBA6S755I033sDRo0fxz3/+E6dPn0Z2drbSXfJImRmVGiDHVvNiT3iVO5iwEjKAwB3SkZrSS4sdqW2ZcYvHFflux564Ciiszl2RW2RkpMc5J75sThofH++2vf2f5eXlaN++vVOblJQUoT8G4uPjER8fj+TkZLRp0wZDhgzB4sWL0b59+xb7EhkZidDQUOj1esE/nxiocuKB41CNlMM2TueUoFoi5xwTQJ5goorKBiOUnncilK8bs2mhggJIX0XxhCos3vNlc9L09HSn9gCwZ88evn3nzp0RHx/v1KampgaHDx9u8ZjestluTMy3z5/x1Bdffj4xUOXkf1i46R7ATrWEf7/M80xYqbZQ8PEeS9UTQDsVFECaTdx81VJAoSqL8M1Jn3rqKQwdOhSvv/46Ro0ahc2bN+Pbb7/FO++8AwAICgrCnDlz8PLLL6N79+7o3LkzFi9ejISEBIwZM4Y/b2lpKSorK1FaWgqr1Yri4mIAQLdu3RAREYFdu3ahvLwct99+OyIiInD8+HE8++yzuPPOO5GUlAQAmDVrFtasWYPnnnsOf/jDH7B37158/PHH2Llzp9c/nxQCMpywEkQcsRZKAPn3M5EqmGh+SKde79WSYm+Y6w2ClxQL5W1A8eeuxUoEFEDcpcZOx2copDRFw0I3lgZfuXIFOTk5KCsrQ0pKSrPNSR1v4zJ48GBs2rQJixYtwvPPP4/u3bvj008/Re/evfk2zz33HH8fuqqqKtx1113Iz8+HyXTzAy0nJwcbNmzg/7t///4AgH379mHYsGEIDQ3FunXrMHfuXJjNZiQmJuKhhx5yWv7buXNn7Ny5E3PnzsWf//xn3HLLLXj33XeRmZnp9c8nBc3uc8JiAHFFio3UxBrCkbNq4sv7vK1uCA0nQqsmiu5zYudlOPFmzxNfwonQGwJKufeJI7n3QbGTKqTwx2cwpLhzvbYRWamHZdnnZPK+iTBE+De5u6G2AX+95yNJ+0vcU13lRC2hwxOpdnelYEKUIPSOxXIM7wDyV1DsHOeiaGHSLCFyU9WEWKW2axeTVNvOiznplbanD2ysLCm28/euxXJPkm2qwhKhyqXHhCiJvoVkYA8krFdLAPmXDUs9AVbqIR1yk9AbAkq5MVtTSgcUgEIKIUJQOJGQ1DfnE7taopZgQgHCd2pbUmznb/UEYCOgABRSCPEGhRORSV0lsWOhWgKwWzEh/vF1aEfK6omWAgogfUihoELUTP2TOBgg51wYVkKJUqS6A7HQY/PvUddNspkk1+RYO6UmybZEymXILQUULU+mrWwIR4jAkNyUpUEbCy/UjMKJH9QaSgBxggkN56iT1az3akmxr4Su3BHCn71PHLEWUADp90pxOpebqoqWgwtRDwonXlB6lRAFE6IEOTZksxNSPdFyQAHkDSkuzx+A1RbCHgonTSgdRByxGEr8Or8fwUTqqgkN6aiL1gMKoHxIaYpCC5ETO9/EMmApeLREqhv0iRlMfAkZcgcTzW9ZzzhfhnaEzj0JhIACsBdSmnIVWiiwEH+x/23tJTUED3e0Gkr8RfNMpOHtvBM5h3YAdQYUwP/t7r3Bekhx1DSw1Dc2KtQTolaq+ka3hgYjKFhVXXZLqkDCH5+RYML6PBMa0pGGlBNjHSkdUAAKKYSITX1rSVXOvnGaWoJJoylIsWDia9WEhnTEJfd29r7sHCvGHiiA7/ug2FVawiXZE8UVKfZIIYQVFE5kIkcgAfzb6bXZsfyseLA8AVbu87BIjt1ihW7K5g9WAgogX0iRaiM3QpRG4URCclVJ+PMxMozjL38CgxxVE1+GdGyhNvE74qhe2qChhuoJwFZAASikEOIrCicSkDOQAOJXS8QIJkpstEbDOQRgL6AAFFIIEUo7s0sVJmcYcTovg9USJaouvgSTQB7SkZOvE2P92daehUmyrsg1cbalgEKTaIlaUDjxgxYCCSBumFDDPBN/MDmkIxO5lxUD2gwogLyrexwFQmipNhsRHOzfcrpGM9srDAMBhRMvKRVE+PNLtLurFoKJ1oZzdOYg2Iyc0t0QlVzLiptiOaAAyoWUpgIhtBB1CfhwonTo8ETKLedZGcZRIpjQ3ia+UVv1BGA/oADshJSm3M1foeBCpKTpcMJ68GiJ1PfAYaVa4v+5FTu117QypCMGpaongDoCCsBuSHGFqi1ESqoLJ2oNHJ7IdVM+1oKJGuaZEGdqrJ4AN1fx+BtS7Kt45AgpgDqCiiNXocVssSjQE6JmqvqmbzSpqrse2ZcAqy2YiLXc2L8++P5eCkQ3+boRm6/7nvizKZuve580xeJSY3fsy5Dl2nmWEBaornKiBXKFEf58jFVL7JQICWoJJj5Niq3XAybPN/MTi68VFH+Gd+wBhbUqip2U1RRA3RUVQoSgcCITuQMJf15Gg4k/1DDXJFAoEVAA9kKKnWNYoaBCiO+0NU7CGLmHbfjz/m/YheVgQlUTz3S+7LXgwzb2/t5jR4khHjvWhnoc/WoO5x9So6EfojVUORGZFiokUh83kPY00V3XBcSKHaUqKAC7VRRHSlVUWkKVFsI6CicCKRU+XJF6mEUL1RIWzu0PueaeWM166I3+zVdRMqAA6ggpgLxBpSWeAgyFF6I0CidgK3B4Ite8DxYnvqqtahKIlA4ogHpCCsBGUHGFqi9EaZoNJ2oKHN4I5FDiL1b6ISuFqicAGwEFEGdvFEC8Ddw8YTWotITV6ktdgxF6P++tY1Vmr0DiQFXhpDFMB4RoK3S4I/fqGJaDCVVN1IWlgAKoo4riSG1BxRXH8NJgoW97IkzgfNOrhBQrbbw9r1hYqlSw1BfZKbByx5GSq3iaqjUbRVnZU2U2SbKyxx05V/0QwgoKJwxQKpA4nl8MVpM0YYCqJurFUkAB2F567A0KKiRQqGpYR0uU3tBMSytxWsJin2Sn4NwTO1aGeOzUOtTTlBaGfghpCYUTmSgdRqTsg5QhgKom2sBaQAHEDymO5A4sFFSI1lA4ERELAcSRHP1htTrBar8UwUD1BGAzoADihRRHSgYWb4d8KMQQllE4EYC18OGKXH2U+suftYqJVZypCqoSSAEFEG/pcUuaBhalhoPshMxboSBD5EYTYuE8IdXdg1Vy91HqYRx/g4nY/dNEMPFh5Q4g7uodO38myUo1UdZOrFU93rCv/FFiBZBQjhNxXT0IsHbtWiQlJcFkMiEtLQ1Hjhxx237r1q1ITk6GyWRCnz59sGvXLqfXOY5DTk4O2rdvj9DQUGRkZOCnn35yalNZWYlJkyYhMjIS0dHRmD59Ompra5sdZ8WKFejRoweMRiM6dOiAV155xamN2WzGwoUL0alTJxiNRiQlJeH999/nX7dYLHjxxRfRtWtXmEwm9OvXD/n5+b5cJq+psnLCclCQi1LXQKpgwlqlxE4TwcRPLFVQgJsredQ21OOJY0BRuqoilKeAYmkIkaknytiyZQuys7ORl5eHtLQ0rFq1CpmZmSgpKUG7du2atT948CAmTpyIpUuX4re//S02bdqEMWPG4OjRo+jduzcAYPny5Vi9ejU2bNiAzp07Y/HixcjMzMQPP/wAk+nGn5VJkybh0qVL2LNnDywWC6ZNm4aZM2di06ZN/LmeeuopfP7551ixYgX69OmDyspKVFZWOvVn3LhxKC8vx3vvvYdu3brh0qVLsNlu3hNs0aJF+Nvf/oZ169YhOTkZu3fvxoMPPoiDBw+if//+UlxSBHEcJ/DGHfKrqalBVFQU+k96BXoDo99iMlAylKkllIjZT7GCib83/hN8bx13BM49cSR2QAHgc0BxJPVwj5wBxRW1BRVXLHUN2Jn5LqqrqxEZGSnJOfjvib9nQx/m5w6x18w49vBKr/ublpaG22+/HWvWrAEA2Gw2JCYm4sknn8T8+fObtR8/fjzq6uqwY8cO/rlBgwYhJSUFeXl54DgOCQkJePrpp/HMM88AAKqrqxEXF4f169djwoQJOHHiBHr16oVvvvkGAwcOBADk5+dj5MiR+OWXX5CQkIATJ06gb9+++P7773Hrrbe67Ht+fj4mTJiAM2fOoE2bNi7bJCQkYOHChZg9ezb/3NixYxEaGoq//e1vHq+PL2hYh3EsDCtJtXcJq9USgComrrA0xOPIPtwj5d4ocg31uKKWoR8tqqmpcXqYzc2DakNDA4qKipCRkcE/p9PpkJGRgcLCQpfHLSwsdGoPAJmZmXz7s2fPoqyszKlNVFQU0tLS+DaFhYWIjo7mgwkAZGRkQKfT4fDhwwCAzz77DF26dMGOHTvQuXNnJCUl4bHHHnOqnGzfvh0DBw7E8uXL0aFDB/To0QPPPPMMrl+/zrcxm818tcYuNDQUX331lfsL6AdVDusEAhaGrtS2oZpY/dV0MPFh5Y4j1oZ4mpJyyEeJoZ6m1Dz0I5c6swF6vX9B1Wq+Ua1MTEx0ej43NxdLlixxeq6iogJWqxVxcXFOz8fFxeHkyZMuj19WVuayfVlZGf+6/Tl3bZoOGQUHB6NNmzZ8mzNnzuDnn3/G1q1b8eGHH8JqtWLu3Ll4+OGHsXfvXr7NV199BZPJhE8++QQVFRV44okn8Ouvv+KDDz4AcCM4rVy5EnfffTe6du2KgoICbNu2DVar+NVU/meR7MjEaywEETu1DN9IRdPBRCSsBxRA+pCi9FAPQEFFDufPn3ca1jEa1fUBYbPZYDab8eGHH6JHjx4AgPfeew+pqakoKSnBrbfeCpvNhqCgIGzcuBFRUVEAgJUrV+Lhhx/GW2+9hdDQUPz5z3/GjBkzkJycjKCgIHTt2hXTpk1zmjQrNhrWkQmrq4DsW85LufW8HMGE9jURwMeVO45YHeJpSqohH6WHepqioR9pREZGOj1chZOYmBjo9XqUl5c7PV9eXo74+HiXx42Pj3fb3v5PT20uX77s9HpjYyMqKyv5Nu3bt0dwcDAfTACgZ8+eAIDS0lK+TYcOHfhgYm/DcRx++eUXAEBsbCw+/fRT1NXV4eeff8bJkycRERGBLl26uPz5xEDhRESsBpCmpAwjjtRSLbGjqokwagkodlKGFFaDirsHEYfBYEBqaioKCgr452w2GwoKCpCenu7yPenp6U7tAWDPnj18+86dOyM+Pt6pTU1NDQ4fPsy3SU9PR1VVFYqKivg2e/fuhc1mQ1paGgDgzjvvRGNjI06fPs23+fHHHwEAnTp14ttcvHjRaQnyjz/+CJ1Oh1tuucWpjyaTCR06dEBjYyP+8Y9/4He/+52XV0k4GtYRgLWQ4S0lqgpyBhMxfr6ACyZ+zj2xU8MQT1NSDfm4CigsDP+0xJeAQsNHrmVnZ2Pq1KkYOHAg7rjjDqxatQp1dXWYNm0aAGDKlCno0KEDli5dCuDG8t6hQ4fi9ddfx6hRo7B582Z8++23eOeddwAAQUFBmDNnDl5++WV0796dX0qckJCAMWPGALhR3RgxYgRmzJiBvLw8WCwWZGVlYcKECUhISABwY4LsgAED8Ic//AGrVq2CzWbD7Nmzcd999/HVlEceeQQvvfQSpk2bhhdeeAEVFRV49tln8Yc//AGhoaEAgMOHD+PChQtISUnBhQsXsGTJEthsNjz33HOSXdOACidqDRe+UGqYQ22hBAjAYCIyNQYUQN79UhyxHFg88SbQBGKAGT9+PK5cuYKcnByUlZUhJSUF+fn5/ITW0tJS6HQ3ByoGDx6MTZs2YdGiRXj++efRvXt3fPrpp/weJwDw3HPPoa6uDjNnzkRVVRXuuusu5OfnO62a2bhxI7KysnDvvfdCp9Nh7NixWL16Nf+6TqfDZ599hieffBJ33303wsPDcf/99+P111/n20RERGDPnj148sknMXDgQLRt2xbjxo3Dyy+/zLepr6/HokWLcObMGURERGDkyJH461//iujoaCkuJwCV7XPSZ/qrAb3PiSdKz7uQK5iwuJeJO0ztc9KUCNUTO1b3QRFC6j1TXFFzWPFWY50ZXz2wVpZ9TnpsnC/KPic/TlomaX+JewFVOdEipQOJnRonvaohmACAzhwkbUARiVorKI7kqKY0pbXqCiFioHCiQqwEEkCdoQRQTzCRnEhzT+y0EFAAZUKKIwosJNBROGEYSyHEkRqHb5yOS3NMJKWVgAIoH1IcNQ0sFFaIllE4URirAcSR3EuC1XBNPFFF1cRO5OoJoK2AArAVUuwcwwoFFaI1FE5korYvXCX2KJHjGtFwjnykCiiA/BNl7VgMKQAFFaI9FE5EpLYA4kjJDdPUfN2aomDiTIqAArARUlgLKHbebgin1RDTUB8Cnc6/zfZs9fT3WGk+7RC7du1aJCUlwWQyIS0tDUeOHHHbvqqqCrNnz0b79u1hNBrRo0cP7Nq1y6cOK6HpFu8tPdTGvrW8UsFE7usmddWEgolrUuwka2euN/APuUl5J2Q5OO5u6+2DELkIrpxs2bIF2dnZyMvLQ1paGlatWoXMzEyUlJQ0u0MicON20vfddx/atWuHv//97+jQoQN+/vlnvzZvUWMQYAEr28kr8f+PgomypKqgOFKqmtI0oLBaURGDLwFFqxUaIi3B4WTlypWYMWMGvy1vXl4edu7ciffffx/z589v1v79999HZWUlDh48iJCQEABAUlKST521GgFQeBeElUACaDOUABoJJhJMim1KjoACsDHk05SWA4sntWYjrIH74xMfCRrWaWhoQFFRETIyMm4eQKdDRkYGCgsLXb5n+/btSE9Px+zZsxEXF4fevXvj1VdfhdXa8oeU2WxGTU2N04N4T+nhGlfkDiZWIy0ZZpGUQzxNKTnk05Tj3ZHVPhxEiBwEVU4qKipgtVr5+wXYxcXF4eTJky7fc+bMGezduxeTJk3Crl27cOrUKTzxxBOwWCzIzc11+Z6lS5fihRdeENK1gMRS+HBFqeE3CiVsk6uC4sgxoChVUWmKKiyEtEzy1To2mw3t2rXDO++8A71ej9TUVFy4cAF/+tOfWgwnCxYsQHZ2Nv/fNTU1SExMlLqrTGI9gDSl9HwgrQYTWbawl2Fox85eQZE7pABsBhW7lioqFFpIoBEUTmJiYqDX61FeXu70fHl5OeLj412+p3379ggJCYFef7Oc27NnT5SVlaGhoQEGQ/O/jEajEUajRr9lWqC2EOJI6UBip9VgYqe1gAIoG1IAtoOKIwotJNAImnNiMBiQmpqKgoIC/jmbzYaCggKkp6e7fM+dd96JU6dOwWa7OWnwxx9/RPv27V0GEy1znAvS9KE2rC2h1nowsdOZg6Q/Sb1880LsrGa9rPNRXHGco8LCPBVvuJrLQnNaiBYIHtbJzs7G1KlTMXDgQNxxxx1YtWoV6urq+NU7U6ZMQYcOHbB06VIAwOOPP441a9bgqaeewpNPPomffvoJr776Kv7f//t/4v4kDFFj2PAWK2HEUaAEEzvZKih2AVRJcaSWqkpLxAgoVJkhShEcTsaPH48rV64gJycHZWVlSElJQX5+Pj9JtrS0FDrdzYJMYmIidu/ejblz56Jv377o0KEDnnrqKcybN09wZxtNAGcEgutdv0akwWIgsQu0YGInS0Cxc1VJkWHZMcBGSAHQrJKixrDiC38CDgUb4g+fJsRmZWUhKyvL5Wv79+9v9lx6ejoOHTrky6lcoiAiPZYDCRC4ocSRrAGlKXtgkSmkAOwEFaB5WHEUKMHFE8dgYzUr9OeUqBbdW4fwWA8kdhRMblI0oACyhRTAuz1SWAgwnuarUHiRlrVBD07v3/wlW4Oy858IhZOAp5ZAYkfBpDnFAwoga0hxh9VKi6OWwguFFkJuonCiEWoLGb6gYNIyJgIKwExIAZpXWlgNK3YUWgi5icKJygRCCHGFgolnzAQUgKmQYqeGqoorFFpIIKJwwqBADSAtoWDiPaYCCsBkSAHUV1VxhUIL0TIKJwqiEOIZBRPhmAsoALMhxU6tVRVXpNxAjoIPkYuqwonVCIC+0FWDgoVymAwoAPMhBdBGVUUqvgYfW73NcyNCHKgqnBB1oFDCBmYDCqCKkGKnpaoKIWpB4YSIggIJm5gOKICqQgpAQYUQuVA4IT6jQKIOzAcUQHUhBfBuUzghKOwQchOFEyIIBRJ1UkVAAVQZUsTSUtih0EICEYUT4hEFEm1QTUABAjqkNEWhhQQiCicqRqFBOFtoYK8aUFVAASikuEGhpQX1eiDIzyE3V3fiJrKicKICFELEEejBxE51AQXw7suCAgwA8efCuBPwQYhIRlXhxGYEQF/UxAcUTJypMqB44u9vuxRuBPM2CNFdfolQqgonhPiCgolrmgwo/mgabiisEKIYCidE0yiYuEcBxQ3HsEJBhRBZUTghmkXBxDsUULxAVRVCZEXhhGgOhRLhKKAIRGGFEElROCGS8iYo6K7rZD0fcY0Cih/kXnpKYYhonHjfCoT8jy3Uxj+Etvf3QfyjMwcp3QXijXq96wdRzNq1a5GUlASTyYS0tDQcOXLEbfutW7ciOTkZJpMJffr0wa5du5xe5zgOOTk5aN++PUJDQ5GRkYGffvrJqU1lZSUmTZqEyMhIREdHY/r06aitrXVq83//938YMmQITCYTEhMTsXz5cqfXt23bhoEDByI6Ohrh4eFISUnBX//6V6c2tbW1yMrKwi233ILQ0FD06tULeXl5Qi+RIBRONETJL20KCNpBAUXFWgotcj0C1JYtW5CdnY3c3FwcPXoU/fr1Q2ZmJi5fvuyy/cGDBzFx4kRMnz4dx44dw5gxYzBmzBh8//33fJvly5dj9erVyMvLw+HDhxEeHo7MzEzU19fzbSZNmoTjx49jz5492LFjBw4cOICZM2fyr9fU1GD48OHo1KkTioqK8Kc//QlLlizBO++8w7dp06YNFi5ciMLCQvzf//0fpk2bhmnTpmH37t18m+zsbOTn5+Nvf/sbTpw4gTlz5iArKwvbt28X8zI6CeI4jvk6bk1NDaKiotDptZehM5mU7g4hpAkaDiLu2K7X4/zcxaiurkZkZKQk57B/TyS+8RJ0of59Twjtb1paGm6//XasWbPmxvttNiQmJuLJJ5/E/Pnzm7UfP3486urqsGPHDv65QYMGISUlBXl5eeA4DgkJCXj66afxzDPPAACqq6sRFxeH9evXY8KECThx4gR69eqFb775BgMHDgQA5OfnY+TIkfjll1+QkJCAt99+GwsXLkRZWRkMBgMAYP78+fj0009x8uTJFn+eAQMGYNSoUXjppZcAAL1798b48eOxePFivk1qairuv/9+vPzyyx6vjy+ockII8ZvOHCTqgxAW1NTUOD3MZnOzNg0NDSgqKkJGRgb/nE6nQ0ZGBgoLC10et7Cw0Kk9AGRmZvLtz549i7KyMqc2UVFRSEtL49sUFhYiOjqaDyYAkJGRAZ1Oh8OHD/Nt7r77bj6Y2M9TUlKC//73v836xXEcCgoKUFJSgrvvvpt/fvDgwdi+fTsuXLgAjuOwb98+/Pjjjxg+fHjLF89PFE4IIcyhkEJ8pWsQISA33Pizl5iYiKioKP6xdOnSZuerqKiA1WpFXFyc0/NxcXEoKytz2ceysjK37e3/9NSmXbt2Tq8HBwejTZs2Tm1cHcPxHMCNqkxERAQMBgNGjRqFN998E/fddx//+ptvvolevXrhlltugcFgwIgRI7B27VqnACM2Wq1DCGGWPaDQsBFRwvnz552GdYxGbd4/pVWrViguLkZtbS0KCgqQnZ2NLl26YNiwYQBuhJNDhw5h+/bt6NSpEw4cOIDZs2cjISGhWQVILBROCCHMc1VFocBCpBYZGelxzklMTAz0ej3Ky8udni8vL0d8fLzL98THx7ttb/9neXk52rdv79QmJSWFb9N0wm1jYyMqKyudjuPqPI7nAG4MQ3Xr1g0AkJKSghMnTmDp0qUYNmwYrl+/jueffx6ffPIJRo0aBQDo27cviouLsWLFCsnCCQ3rEEJUSex5LjQvhvjCYDAgNTUVBQUF/HM2mw0FBQVIT093+Z709HSn9gCwZ88evn3nzp0RHx/v1KampgaHDx/m26Snp6OqqgpFRUV8m71798JmsyEtLY1vc+DAAVgsFqfz3HrrrWjdunWLP5PNZuPn11gsFlgsFuh0znFBr9fDZpNudSZVTgghxAdyBBSqDqlDdnY2pk6dioEDB+KOO+7AqlWrUFdXh2nTpgEApkyZgg4dOvBzVp566ikMHToUr7/+OkaNGoXNmzfj22+/5Zf4BgUFYc6cOXj55ZfRvXt3dO7cGYsXL0ZCQgLGjBkDAOjZsydGjBiBGTNmIC8vDxaLBVlZWZgwYQISEhIAAI888gheeOEFTJ8+HfPmzcP333+PP//5z3jjjTf4vi9duhQDBw5E165dYTabsWvXLvz1r3/F22+/DeBG9Wjo0KF49tlnERoaik6dOuHLL7/Ehx9+iJUrV0p2TVUVTmwGDvDyLyv9ZkMIUTu1fI4FeogaP348rly5gpycHJSVlSElJQX5+fn85NPS0lKnysPgwYOxadMmLFq0CM8//zy6d++OTz/9FL179+bbPPfcc6irq8PMmTNRVVWFu+66C/n5+TA5bKexceNGZGVl4d5774VOp8PYsWOxevVq/vWoqCh8/vnnmD17NlJTUxETE4OcnBynvVDq6urwxBNP4JdffkFoaCiSk5Pxt7/9DePHj+fbbN68GQsWLMCkSZNQWVmJTp064ZVXXsGsWbMkuZ6AyvY5EWP9OkvU8sFDCCH+sNXX4+d5i2TZ50SM/bDk6C9xT1WVE62R8rcNCj6EEELUisKJRrFSZqWQRAghRCgKJ0RSjiGJggohhBBv0FJiIhubkWOmokMIIYRdVDkhsrMHFKqkEELEpruug47z8/fuevq9XWkUTohipKqiUOghhBB1U1c4MVlvPFyp18vbF8Isb0MPhRht0V2X57ddW2jzXTHlOrdqUSWCCKSucOJOS6FFDhSMVMlm5CigqJwSoYCCCCHS0044URJVc1SLAor/pPqydlWhkPJ8hBB2UDiRkhLVHApEglFA8Z2UQYFCCCGBi8KJ1jQNRBRWvEIBRRgKDoQQKVE40TrHsEJBxS0KKN6hYEIIkRqFk0BCVRWPKKC4R8GEECIHCieBjKoqLlFAcY2CCSFELqoKJ3qDFTqjd5NMrWb6shVErMm7Ggk5FFCcUTARh96sdA8UEqg/N/GZqsKJEHovQ4wYKAg5cBdyVBZcKKDcQMFEuIANIYSIRLPhRE4tBSEKLU3Yg4uKQoraAgoFCfcoNGifzgzo/f0rS39OFEfhREIUWlqgspCitoBCnFEgIUR9KJwogELL/6gopFBAURcKJISoG4UThog5T0ZVQUclIYUCCtsokBCiHRRONMox6KgmqKggpFBAYQsFEkK0icJJAGhakWE+rDAeUiigKI9CCSHapqpwYjBZoDf5thrBXG8QuTfqpZqqCsMhhQKK7yhYeKavV7oHIqP/50QgVYUTfxhNDZIcV+2hRxVVFUZDCgUU4SiYONNcCCFEJAETTqTSUuhRa2hhuqrCYEihgOK9QA0mFEAIEY7CiUS0EFqYraowFlIooHimxmCirwesJmHtCSHioHAiMzWHFuaqKgyFFAooLVNTMGkaMChwEKIMCieMUFtoYSqoMBJSKKA0p5ZgQiGEELZQOGGctxN5lQwx3m4eJ3mIcbzpoEJBhQLKTawHEwok2qQ3A37/7Wf8z24gUFU4CTc2QG+8+cFfZ2azqqAEVyGGtaqLPcTIUmlRsJoS6AGF5VBCgUQawR6uaxDDfyYIm1QVTpoKN/q+PDgQgk3TwMJKWFEkpDQlcWgJ1IDCYjDRciDxFAoIUStVhxN/+BNs7NQWcBzDCgtBRdaQ0pTJSgFFZCwFEy0FEgogJBD5tN3q2rVrkZSUBJPJhLS0NBw5csSr923evBlBQUEYM2aML6dlTrixweVDDYymBqeHkvRGq6g3PfRaS1UVEdmMnOTnYAErwURfr95gElzv+kFIIBIcTrZs2YLs7Gzk5ubi6NGj6NevHzIzM3H58mW37zt37hyeeeYZDBkyxOfOqoUaQwsLQUWRkEIBxW9KBxN7IFFLKKEQQohngsPJypUrMWPGDEybNg29evVCXl4ewsLC8P7777f4HqvVikmTJuGFF15Aly5dPJ7DbDajpqbG6aEFagktSldVZA8pFFB8plQwYT2QtBRAKIQQ4h1B4aShoQFFRUXIyMi4eQCdDhkZGSgsLGzxfS+++CLatWuH6dOne3WepUuXIioqin8kJiYK6abqsB5alAoqWgsoWiN3MGExkFAAIUQagsJJRUUFrFYr4uLinJ6Pi4tDWVmZy/d89dVXeO+997Bu3Tqvz7NgwQJUV1fzj/PnzwvppmawGFbkDimKzEUhbunNygQTVlAIIf6orKzEpEmTEBkZiejoaEyfPh21tbVu31NfX4/Zs2ejbdu2iIiIwNixY1FeXu7UprS0FKNGjUJYWBjatWuHZ599Fo2NjU5t9u/fjwEDBsBoNKJbt25Yv3690+tLlixBUFCQ0yM5OdmpzenTp/Hggw8iNjYWkZGRGDduXLO+PPDAA+jYsSNMJhPat2+PyZMn4+LFi4Kuk08TYr119epVTJ48GevWrUNMTIzX7zMajYiMjHR6AEC4wYwIIyMz7xTAUnVFzpBCAcU7uuuS/nVWLJQoHUyoKuIbx0qX0vOSWDJp0iQcP34ce/bswY4dO3DgwAHMnDnT7Xvmzp2Lzz77DFu3bsWXX36Jixcv4qGHHuJft1qtGDVqFBoaGnDw4EFs2LAB69evR05ODt/m7NmzGDVqFO655x4UFxdjzpw5eOyxx7B7926nc9122224dOkS//jqq6/41+rq6jB8+HAEBQVh7969+Prrr9HQ0IDRo0fDZrPx7e655x58/PHHKCkpwT/+8Q+cPn0aDz/8sKDrJGgpcUxMDPR6fbOUVF5ejvj4+GbtT58+jXPnzmH06NH8c/YfIDg4GCUlJejataugDgPwKaDUmo2C36MGTQOK3Mub7QFF6qXJeqNV+W3yGSZHMJETC4FEa5S+pgQ4ceIE8vPz8c0332DgwIEAgDfffBMjR47EihUrkJCQ0Ow91dXVeO+997Bp0yb85je/AQB88MEH6NmzJw4dOoRBgwbh888/xw8//IAvvvgCcXFxSElJwUsvvYR58+ZhyZIlMBgMyMvLQ+fOnfH6668DAHr27ImvvvoKb7zxBjIzM/nzBQcHu/w+B4Cvv/4a586dw7Fjx/iiwYYNG9C6dWvs3buXn/Ixd+5c/j2dOnXC/PnzMWbMGFgsFoSEhHh1rQR9ohkMBqSmpqKgoIB/zmazoaCgAOnp6c3aJycn47vvvkNxcTH/eOCBB/jkJudckgijWfBDjZSqqMgxL4UqKK5pKZgoWSlRc3XEqUrRwoMI13Rhhtns31+GwsJCREdH88EEADIyMqDT6XD48GGX7ykqKoLFYnGa65mcnIyOHTvycz0LCwvRp08fpykXmZmZqKmpwfHjx/k2jsewt2k6X/Snn35CQkICunTpgkmTJqG0tJR/zWw2IygoCEbjzV/2TSYTdDqdU4XFUWVlJTZu3IjBgwd7HUwAH4Z1srOzsW7dOmzYsAEnTpzA448/jrq6OkybNg0AMGXKFCxYsIDvdO/evZ0e0dHRaNWqFXr37g2DQfmNwNxRc4BRcthHypBCAcWZ1MFELkp8gaptuIaCh3fcrZQS8gCAxMREp8UZS5cu9atvZWVlaNeunXN/g4PRpk2bFudtlpWVwWAwIDo62ul5x7meZWVlLueC2l9z16ampgbXr18HAKSlpWH9+vXIz8/H22+/jbNnz2LIkCG4evUqAGDQoEEIDw/HvHnzcO3aNdTV1eGZZ56B1WrFpUuXnI49b948hIeHo23btigtLcU///lPby8TAB/Cyfjx47FixQrk5OQgJSUFxcXFyM/P53/o0tLSZp3UMtaDihbnpUgWUFS2YkeOYCJ11UTuL1fWwwgFELacP3/eaXGG/RfvpubPn99sImnTx8mTJ2XuvXD3338/fv/736Nv377IzMzErl27UFVVhY8//hgAEBsbi61bt+Kzzz5DREQEoqKiUFVVhQEDBkCnc/48evbZZ3Hs2DF8/vnn0Ov1mDJlCjjO+y0VfNq+PisrC1lZWS5f279/v9v3Np0drCWOAYWlOS72gKLUdvtGU4Poc1ICfQ6K2ismcn3ZshhCKGioh+OCDHeefvppPProo27bdOnSBfHx8c02LG1sbERlZWWL8zzi4+PR0NCAqqoqp+qJ41zP+Pj4Zju12+eGOrZxNV80MjISoaGhLs8dHR2NHj164NSpU/xzw4cPx+nTp1FRUYHg4GBER0cjPj6+2R5mMTExiImJQY8ePdCzZ08kJibi0KFDLqeAuBKw99aRmqtKitKBRcmQopqAIsM9d9RCiqqJHF/MLAUSCiKBITY2FrGxsR7bpaeno6qqCkVFRUhNTQUA7N27FzabDWlpaS7fk5qaipCQEBQUFGDs2LEAgJKSEpSWlvJf9Onp6XjllVdw+fJlfthoz549iIyMRK9evfg2u3btcjr2nj173IaF2tpanD59GpMnT272mn0F7t69e3H58mU88MADLR7HvhBGyJwddf/6pTKszFdRctKs2NQ0B0WsXWLVOJwjx9AEK8M1NBRDWtKzZ0+MGDECM2bMwJEjR/D1118jKysLEyZM4FfqXLhwAcnJyXwlJCoqCtOnT0d2djb27duHoqIiTJs2Denp6Rg0aBCAG9WMXr16YfLkyfjPf/6D3bt3Y9GiRZg9ezY/eXXWrFk4c+YMnnvuOZw8eRJvvfUWPv74Y6eVNc888wy+/PJLnDt3DgcPHsSDDz4IvV6PiRMn8m0++OADHDp0CKdPn8bf/vY3/P73v8fcuXNx6623AgAOHz6MNWvWoLi4GD///DP27t2LiRMnomvXrl5XTQCqnChK6eqKEpUU1VRQGKW2YBIowzcURIi3Nm7ciKysLNx7773Q6XQYO3YsVq9ezb9usVhQUlKCa9eu8c+98cYbfFuz2YzMzEy89dZb/Ot6vR47duzA448/jvT0dISHh2Pq1Kl48cUX+TadO3fGzp07MXfuXPz5z3/GLbfcgnfffddpGfEvv/yCiRMn4tdff0VsbCzuuusuHDp0yKkqVFJSggULFqCyshJJSUlYuHChU8AJCwvDtm3bkJubi7q6OrRv3x4jRozAokWLnFb5eBLECZmhopCamhpERUVh1O7HEBLu/MVWZTYp1Cv5yBlY5AoqUuyLImpAkWhoR2cO8u/9Klk2LPckVyUESiAJrvf/K8LaUI9jGxeiurraqzkcvrB/T/R64lXojf59L1jN9fjhrecl7S9xT/WVk2ijd58Qag4xTSssUoYVuaopVEFhjxjBROuhRI2BRIxwQYjcVB9OvOVtiHHEaqCRYzhIjpDCdEAJsImxFExcU0sYoQBCtCZgwokvvAk0rAQYqZYxO06clSKoMB1QAoTa7nsidTBhNZBQACGBhMKJnxwDDGtBRS3VFGYDSoBVT/yh5iXCLIURCiCE3EDhRESsBZUIo1mS+SlShBRmA4rGqaFqIkUoYSGQUBAhpGUUTiTCSlCRqooCiB9SmAwoGq6esL4yh0IJ8YXeDOj9vcTK3fWD/A+FExmwEFTkCCmA/0GFyYCiQSxXTLQ4fEOBhBBhKJzITOmgItVQj50Y1RTmAorGqiesbrKmtVAS6IHE6ee3BPa1IMKpKpy0MdTBYLS0+Pqv5nAZe+M/pYKKlFUUO39DCnMBRSNYrJhIufpG7mCi9kCi9v4T7VBVOPGkrbHOYxtWA0zTZctyhBXWQwpTAUUD1RMp7pfjD60sCWb9C531/hHiiqbCiTe8CTCA8iHG1R4rUgUWuUKK6gOKirFUMdFCKGHpC5+lvhAiloALJ95iMcRIXV2RYz6KqgOKBqonYmFh1Ysr0q0cUj4AsNAHQuRC4cRPjiFG7mqLFNUVqasoLAUUIj81zS9ROgwofX5ClEThRERKBhU7e2BhOaSwElCoeuIbFqsmYvVJ6UCg9PkJYYW0918PYG2NdfxDCb7c6NAVVzcZFIPj3ihCGE20O5JaSbWpmtqDSXA9xz8IITdQOJGBUkEl2lgvSkiJMJolCSksBBS90Sr8TSYf3qMRLFVNxN1jRd5gQIGEEPdUNazTOuQajCE39jmptLC5JNgTJYZ+WB7qYWWIh0hL7KqJGqslWgoiwddswt5gEdieBDxVhRNHbUI8VyFYDzByBxVWQ4rSAUVNc09soTboritT8GShaqLGaomaQong0MGg4HoOept/1zyoQT3/z7RKteHEG54CDEvhRc6gEm2sF2UZsphLj5UOKEQ6YlRN1LYSh6VAooXAQQKPpsOJJ6yGFzmCCotVFCUDilzVE5uRg84cJOw8DFCqaiLFeaUKDopNqKXwQTQooMOJJywMHUkdVFgLKVRB0RZ/qiZqqJZQICFEGrRax09tQur4h9SkXPXD0tJjpVbxyLVyx2Zkp+TvDSWqJiwHE6VW2gRfs/EPQrSOKicicgwoclZUAHGqKixVUaiCon5S30PHW2IHEzlRECGBisKJROQMKoC4wz+shBQlAgrNPXGm9qqJWGGCQgkh8qJhHRnIOfQDQLShHxaGeljYqC1Q+RMSlK6aiDnsItuSY40O2wRftyG4Xls/E5GeqionbUPqYApx7nKFJUKh3vhG7qEflqooUt7x2BUa4lEfMaomahrGUWsQCb6uzn4T9VBVOHElJqS2xddYDy72oCJlSBEjoADi7Y3iC1+Hd7RIqQ3YhFCyaqKGagnLgYRCB2GF6sOJOy0FF9ZCi9QhhZWAQtUTZ3LNO9H7MKrmawVDqaXDaqiWKB1KKHgQNdF0OGkJq6FFypDCSkDxFVVP2EfBxMXxFAokFESI2gVkOGkJK0NEUoUUFgIKVU9858uQji9VE7mxEExEDTgUSBSlN3MI9vPeOrCoay8iLaJw4iVXwUXqwCJFSFFzQGG6eqLQjQCl4EtYUGKeCQUTCiREu9ifXcewmJBat9UWsYi9DFmsHWbFWmpM1E2J4RwxAoWoy41lXgIcfN1GwYRoGoUTEagxpCgdUMTY5l4Ite97wuqQDkv3zhFCjdUSeyChUEICAYUTEaktpCgdUHzh66ZsQvh0jx2NkCswKDnPRE3VEgokJFBROJGAmkKKkgGFqifq5mvVRKlgIvYwjpQokJBAR+FEQnIEFMD/kKK2gCJH9UTt1LBKRyh/g4lo/ZAomKi9SqK/3uj2QYgQqlqt0zb4KkKDb3a5orGVgr3xjj2gyLEU2Z/VPSys4pGDt8uKfboBoMrJMaQj9zwT1kOJWoIIhQsiN1VXTmKCrzZ7sEquoR4APldRlKqgUPVEfXwZ0pF7OIflYMJShcRTxYOCibgqKysxadIkREZGIjo6GtOnT0dtrfvvhvr6esyePRtt27ZFREQExo4di/Lycqc2paWlGDVqFMLCwtCuXTs8++yzaGy8+f/u0qVLeOSRR9CjRw/odDrMmTOn2XnWrVuHIUOGoHXr1mjdujUyMjJw5MgR/nWLxYJ58+ahT58+CA8PR0JCAqZMmYKLFy+67LfZbEZKSgqCgoJQXFzs/UWCysOJK64CC0uhRc75KL5Q4yRZobQ+90TO7erlOL5cdwV2eW6RJ70qGUooeLBh0qRJOH78OPbs2YMdO3bgwIEDmDlzptv3zJ07F5999hm2bt2KL7/8EhcvXsRDDz3Ev261WjFq1Cg0NDTg4MGD2LBhA9avX4+cnBy+jdlsRmxsLBYtWoR+/fq5PM/+/fsxceJE7Nu3D4WFhUhMTMTw4cNx4cIFAMC1a9dw9OhRLF68GEePHsW2bdtQUlKCBx54wOXxnnvuOSQkJAi9RACAII7jmN8Kr6amBlFRUVhTlIbQCPFHopQcHpJ6uMfXDdzEGOIBhN3N2JeN2XzdlM3bHWMFDe34sAmbt/fWEbKUWI5wIrRyIvd+JqLsgyJBtUQurIWOxsZ6fFn4MqqrqxEZGSnJOezfEwMffhnBIf4NLTda6vHt3xeJ3t8TJ06gV69e+OabbzBw4EAAQH5+PkaOHIlffvnF5Rd5dXU1YmNjsWnTJjz88MMAgJMnT6Jnz54oLCzEoEGD8K9//Qu//e1vcfHiRcTFxQEA8vLyMG/ePFy5cgUGg/Pn3bBhw5CSkoJVq1a57a/VakXr1q2xZs0aTJkyxWWbb775BnfccQd+/vlndOzYkX/+X//6F7Kzs/GPf/wDt912G44dO4aUlBRvL5X2Kie+ULLaInUlhSooRGmBHkzkrJZQNURcNTU1Tg+z2b+Z5oWFhYiOjuaDCQBkZGRAp9Ph8OHDLt9TVFQEi8WCjIwM/rnk5GR07NgRhYWF/HH79OnDBxMAyMzMRE1NDY4fP+5zf69duwaLxYI2bdq02Ka6uhpBQUGIjo7mnysvL8eMGTPw17/+FWFhYT6dm8KJG3KGFilDitIBxVtyLi3W6tAOa1UTJTda84WYwzhyhRIaonFm/3/o7wMAEhMTERUVxT+WLl3qV9/KysrQrl075/4GB6NNmzYoKytr8T0Gg8Hpyx8A4uLi+PeUlZU5BRP76/bXfDVv3jwkJCQ4BSNH9fX1mDdvHiZOnMhXmDiOw6OPPopZs2Y5hTChVLVahxWOAUXsISGpVve0CalTbBWPkBU8Qu+7w/T9dojPlKiaiF0tkRIFEXmcP3/eaVjHaHT92TR//ny89tprbo914sQJUfsmtWXLlmHz5s3Yv38/TKbmn98WiwXjxo0Dx3F4++23+efffPNNXL16FQsWLPDr/BRO/GQPKmoIKWoJKCwJxCXFYlHLcA6FEtKSyMhIr+acPP3003j00UfdtunSpQvi4+Nx+fJlp+cbGxtRWVmJ+Ph4l++Lj49HQ0MDqqqqmg2d2N8THx/vtKrG/rr9NaFWrFiBZcuW4YsvvkDfvn2bvW4PJj///DP27t3rdI327t2LwsLCZkFu4MCBmDRpEjZs2OBVHyiciETqkAKIE1SUDCjeouqJfKSeCCsUBZPmKJCwLzY2FrGxsR7bpaeno6qqCkVFRUhNTQVw48vcZrMhLS3N5XtSU1MREhKCgoICjB07FgBQUlKC0tJSpKen88d95ZVXcPnyZX7YaM+ePYiMjESvXr0E/SzLly/HK6+8gt27d7sclrEHk59++gn79u1D27ZtnV5fvXo1Xn75Zf6/L168iMzMTGzZsqXFn9EVCicikyqkAOJVU5QKKKxVT7zdkI1ok5hzS8SmlUCiv2YBAHBWbfw8/urZsydGjBiBGTNmIC8vDxaLBVlZWZgwYQK/UufChQu499578eGHH+KOO+5AVFQUpk+fjuzsbLRp0waRkZF48sknkZ6ejkGDBgEAhg8fjl69emHy5MlYvnw5ysrKsGjRIsyePdupgmHfa6S2thZXrlxBcXExDAYDH2Bee+015OTkYNOmTUhKSuLnq0RERCAiIgIWiwUPP/wwjh49ih07dsBqtfJt2rRpA4PB4LRix/5eAOjatStuueUWr68VhROJsB5SWK+gsFQ9oaEd4XwZ0pGzakLBxH/24EGE2bhxI7KysnDvvfdCp9Nh7NixWL16Nf+6xWJBSUkJrl27xj/3xhtv8G3NZjMyMzPx1ltv8a/r9Xrs2LEDjz/+ONLT0xEeHo6pU6fixRdfdDp3//79+X8vKirCpk2b0KlTJ5w7dw4A8Pbbb6OhoYFfsmyXm5uLJUuW4MKFC9i+fTsANFsWvG/fPgwbNsyfS+NEVfucbP1PMsJa3fySKLNEK9cpAaTeR8WfkKLEPihS7X3iSzgRdb8TBvY5EbJax7cAIc2x1TacEyihRKwA0mg1Y2/xMln2ORk08kVR9jk5tCtH0v4S91RdOYkPqXL5PGuhRcoqCuBfJUWJCgpLwzuiDu2YrD4FFLWQar6J3DvAshZMWAglVAUhrFF1OGmJq9DCQmCRI6SoJaB4S8jwDk2MVR+57zRMwYSCCFEHTYYTV1gKLFLPR1FDQGGpeuINrc07kXJzNDk2XpO92qLCUEIhhKhZwIQTV5QOLFIuP1ZDQPEWVU+UJ/USYiHknmeipmBCgYRoRUCHE1damscCSBdcYoKvBmRAYaV6QvNOlCFn9YOFYEKhhBDvUTgRQMpKixRVFC0FFBaqJ1ob2pGC1EM6cs4zYT2YUCBxLbjehuBGP//f+ft+4je68Z+f3FVafCH2zQV9vZkgizcLlPPGgIQ9slZaRAomUtyQT3/NQsGEaB6FExHEh1QxHVJYDyjRRvF/3Q43CrvrsKh3KTZZxTsWA6SYbyI0aMg5z0SMYEKhhBD/UDgREcshhfWA4q1ArZ7YQqnMLJSY980RItCHcHR1ZpcPQoSgcCIBqUKK38dgOKCwUD3xht6oraqImKSab6KW4ZxAq5ZQCCFSUlU4aauvQ6y+ln+wTuyQIkYVheWA4i0h1RMpAopXNDK0o/QSYrUM52i5WkIhhCjBp3Cydu1aJCUlwWQyIS0tDUeOHGmx7bp16zBkyBC0bt0arVu3RkZGhtv2QjgGFZZDC2tDPawGFCHVEymGd0Sdd0JaJHUlRM5gorVqCYUQwgrB4WTLli3Izs5Gbm4ujh49in79+iEzMxOXL1922X7//v2YOHEi9u3bh8LCQiQmJmL48OG4cOGC351vCYuBhbX5KHIHFG+pYXhHzKEdm5H5+256RY5dYb0h5zwTNVdLqBpCWCc4nKxcuRIzZszAtGnT0KtXL+Tl5SEsLAzvv/++y/YbN27EE088gZSUFCQnJ+Pdd9+FzWZDQUFBi+cwm82oqalxeviLlcDCUkiRM6AoPbxDhFPrkI7g8/hQNVFTtYRCCFEjQeGkoaEBRUVFyMjIuHkAnQ4ZGRkoLCz06hjXrl2DxWJBmzZtWmyzdOlSREVF8Y/ExEQh3fSakmFF7IAC+DZp1teAIiU1VE+8opF5J2KSMnAotWzYH1KEEgoiRAsEhZOKigpYrVbExcU5PR8XF4eysjKvjjFv3jwkJCQ4BZymFixYgOrqav5x/vx5Id30mdxBRc0BRerhHeJMd53duessDOmocThHikoJhRGiFbJuX79s2TJs3rwZ+/fvh8nU8pblRqMRRqN3W5VLpWlAuWIVvg28N+JDqhS7O7LcpLhBoJBt7b0h6n12GMVCmPBEjiEdJYdzKJgQ4p6gcBITEwO9Xo/y8nKn58vLyxEfH+/2vStWrMCyZcvwxRdfoG/fvsJ7qjDHsCJ2UBE7oPhyI0Ff7sPj6z14SGCSKnCobTiHgom09NcboQ/2L0RyjdLdpJF4R1Ct2GAwIDU11Wkyq31ya3p6eovvW758OV566SXk5+dj4MCBvveWEVIM/UgxxEOcKbbniYYpXYVR23COVPNLCNEawQPZ2dnZWLduHTZs2IATJ07g8ccfR11dHaZNmwYAmDJlChYsWMC3f+2117B48WK8//77SEpKQllZGcrKylBby95ETKFYDihqn3sixaRYrdOL+B2l5EodyfdBUaBqItXEV0K0SvCck/Hjx+PKlSvIyclBWVkZUlJSkJ+fz0+SLS0thU53M/O8/fbbaGhowMMPP+x0nNzcXCxZssS/3jMgVl8r6jCP1uegSDHvhLBPzi3o3fFnszVf0TAOgGvXle4BURmfJsRmZWUhKyvL5Wv79+93+u9z5875cgpVYTWgBMrcE7EnxRLvBMqQDgUTH9RRGCH+YXd9osqwPMRDiJ3SgcITSfdBkXk4J6CCSd115wchfqJwIiIWAwqLc0+kuNeOt2hSrPxYGdLxha9VE81PfKUwQiRG4URkWgkohMhNSIgROqTDwhb1vmIilFAYITJTVTiJ0ZkRq2PgL6oHLAYUOSi5a2yg32dHzJU6WqPm4RzFggmFEaIwVYUTu9j/hRSWgwprAUVo9UTqe+5IcSNAsRhNgT30I+YyYimGdOSYCMvCcI6swYTCCGGMKsOJI5aDCmsBRQ5quOcOzTvxn9gTayXbPVbG4RzVBRMKI4Rhqg8njlgMKiwFFNaqJ96gzdjUS80TYYUSK5jIMvGVAglRAVlv/CenpgHlik25fTBY3QdFKt7ue0IbssmL9WXE3pB6SEfJSbCShpIACiP6643Q6/0Li5yVjcnQgUxTlRN3lK6qsFJBUWP1xFuBPilWrVgZ0lFyOEeSYELDNkTFAiacOFIqqLASUOSghrkn/tAbrUp3gTCA6WBCiIoFZDhxJHdQYSGgsFQ98WbVjhLzTgJ1xY6SN/zzRMiQjlxVE3+JHkyoUkI0IuDDiSO5gooaA4ovWK6e0Iod33kzd8XboRo1T5r1t2oiSTAhRCMonLRA6qDCQkARQk1zTwgRwpeqiRT3zfELBROiMRROvOAYVMQMK0oHFFaqJ2IO7ahxUqzNKH31QO0rdaQc0lGCqFUTCiZEgyic+EDMkKJ0QBGCqidETCwM6ShRNQnEYMJdU0c/CTsonPiB1YAiBN0U0He0YkcecmxX7y1mggnjE1+5a9ecHuSmyspKTJo0CZGRkYiOjsb06dNRW+v+O6C+vh6zZ89G27ZtERERgbFjx6K8vNypTWlpKUaNGoWwsDC0a9cOzz77LBobb4bvS5cu4ZFHHkGPHj2g0+kwZ84cl+datWoVbr31VoSGhiIxMRFz585Fff3N0uuSJUsQFBTk9EhOTnY6xrBhw5q1mTVrlqDrROHETywGFNaqJ3IP7YglUFfsqJmQIR25V+iIGkwYQ2HEe5MmTcLx48exZ88e7NixAwcOHMDMmTPdvmfu3Ln47LPPsHXrVnz55Ze4ePEiHnroIf51q9WKUaNGoaGhAQcPHsSGDRuwfv165OTk8G3MZjNiY2OxaNEi9OvXz+V5Nm3ahPnz5yM3NxcnTpzAe++9hy1btuD55593anfbbbfh0qVL/OOrr75qdqwZM2Y4tVm+fLmQy6TdHWLlFKszi7IDrdg7yXorJvgqKhpbyX5eqUQYzag1e/7/EW5sQJ3ZIEOP1IHlZcQs8KdqorVgQgHENydOnEB+fj6++eYbDBw4EADw5ptvYuTIkVixYgUSEhKavae6uhrvvfceNm3ahN/85jcAgA8++AA9e/bEoUOHMGjQIHz++ef44Ycf8MUXXyAuLg4pKSl46aWXMG/ePCxZsgQGgwFJSUn485//DAB4//33Xfbv4MGDuPPOO/HII48AAJKSkjBx4kQcPnzYqV1wcDDi4+Pd/qxhYWEe27hDlRORsFZBYa16QrRBrGXEXi81ZmQiLBOrcxQOJoFYGampqXF6mM3+fc4XFhYiOjqaDyYAkJGRAZ1O1ywA2BUVFcFisSAjI4N/Ljk5GR07dkRhYSF/3D59+iAuLo5vk5mZiZqaGhw/ftzr/g0ePBhFRUU4cuQIAODMmTPYtWsXRo4c6dTup59+QkJCArp06YJJkyahtLS02bE2btyImJgY9O7dGwsWLMA1gX9uqHIiItYqKELuwSN19cSb++14c6+daGM9qswmMbtGApCcQzqiVE0UDCZqCyO6OjN0ej+PYb3x/ywxMdHp+dzcXCxZssTn45aVlaFdu3ZOzwUHB6NNmzYoKytr8T0GgwHR0dFOz8fFxfHvKSsrcwom9tftr3nrkUceQUVFBe666y5wHIfGxkbMmjXLaVgnLS0N69evx6233opLly7hhRdewJAhQ/D999+jVatW/HE6deqEhIQE/N///R/mzZuHkpISbNu2zeu+UDgRGWsBRQghASUmpBYVFvmHoAiRm+LDOQoEE7UFEqmcP38ekZGR/H8bja4/2+fPn4/XXnvN7bFOnDghat+ksH//frz66qt46623kJaWhlOnTuGpp57CSy+9hMWLFwMA7r//fr593759kZaWhk6dOuHjjz/G9OnTAcBpDk2fPn3Qvn173HvvvTh9+jS6du3qVV9UFU5i9AZE6m+MRF2xsjtZkaWAIuUdjCmg3FixYzW7+TXNZAXq/fw1LkBJNaQjV9VEbcGEAklzkZGRTuGkJU8//TQeffRRt226dOmC+Ph4XL582en5xsZGVFZWtjg/Iz4+Hg0NDaiqqnKqnpSXl/PviY+P54diHF+3v+atxYsXY/LkyXjssccA3AgWdXV1mDlzJhYuXAidrvlMkOjoaPTo0QOnTp1q8bhpaWkAgFOnTnkdTlQ75yRWb+AfLGJtDoq3pFxaLOeqHbE2Y6MVO4HN16qJJDfzk0AgziORQmxsLJKTk90+DAYD0tPTUVVVhaKiIv69e/fuhc1m47/Am0pNTUVISAgKCgr450pKSlBaWor09HQAQHp6Or777jun4LNnzx5ERkaiV69eXv8c165daxZA9Pobv1xxnOt5YrW1tTh9+jTat2/f4nGLi4sBwG2bplQbThyxGlRYCSg0ObZldI+dG9S8Uoe1HWFZX5lDgUQ5PXv2xIgRIzBjxgwcOXIEX3/9NbKysjBhwgR+pc6FCxeQnJzMV0KioqIwffp0ZGdnY9++fSgqKsK0adOQnp6OQYMGAQCGDx+OXr16YfLkyfjPf/6D3bt3Y9GiRZg9e7bTUFRxcTGKi4tRW1uLK1euoLi4GD/88AP/+ujRo/H2229j8+bNOHv2LPbs2YPFixdj9OjRfEh55pln8OWXX+LcuXM4ePAgHnzwQej1ekycOBEAcPr0abz00ksoKirCuXPnsH37dkyZMgV33303+vbt6/W1UtWwjjccAwoLQz8sDfF4S2tLi4n6SLXxmlJ3HxZMgmBCYYQNGzduRFZWFu69917odDqMHTsWq1ev5l+3WCwoKSlxWt3yxhtv8G3NZjMyMzPx1ltv8a/r9Xrs2LEDjz/+ONLT0xEeHo6pU6fixRdfdDp3//79+X8vKirCpk2b0KlTJ5w7dw4AsGjRIgQFBWHRokW4cOECYmNjMXr0aLzyyiv8+3755RdMnDgRv/76K2JjY3HXXXfh0KFDiI2NBQAYDAZ88cUXWLVqFerq6pCYmIixY8di0aJFgq5TENdSrYYhNTU1iIqKwsWSWxDZyrdij9JBRYyAAsCvgCJk7onQcCJk7omnVTsAPK7aAeDVqh1v9jvxtNeJud79627nnABezTnRmYPcv37d/Z97vYdf1j0tAfamciLnMmIW5pv4MqTD2jwTVgJJo60BBf/dgOrqaq/mcPjC/j2R0X0ugvX+fd42Ws344qc3JO0vcU8TwzrecBz6UWL4h5UhHm/RtvZEDVga0mEpmNCwDVG7gAknTSkRVJQOKDT3RBoe77FjonvwsEDKqglrwYQQtQvYcOJIzqCipoAiVfVEbat2aMUOcYellTkUTIhWUDhpIhACilTUWj2hFTvqxOLeJj4ToWpCwYRoCYUTF1hbkkwIkY/s99Fh5GZ+hLBEc0uJxRKrN0i6wkctS4xpWTEh7GO9asLVydi/a9cBnZ8TpW3sDNUFKqqcuEEVFELkJ9UeJ1Jgab4Jq7haz/PHCGmKwokHWgwoUq7aIYFLrD1OAoqGh3S42joKJsRnFE68wHpAYWlirFonxRJ1YWl/E6WxOKRDoYT4i8KJl6QIKGKt2iFEat7sDssq5lfqaAwFEyIGCicCsF5BIYT4R9aVOhpbPkzDOERMFE4ECsSAIsVmbN5sxOYNuTZiI6Qpmgx7E4USIjYKJz4QM6BodUM2MXizSywhqqShibAUTIgUKJz4KBArKIQQdig9pEPDOERKFE78oOaAQsuJSSCgybDSoFBCpEbhhAFaG9qh5cSEuKHyIR0KJkQOtH09IYRAgXvq+EGJIR21hBLu2nVwOqt/x7DRzUCVRpUTP6l5aEcIKVbskMDi1Q6yKtm6PtBW6qglmBDtoHDCCNqQjRDv0O6w8qJgQpRA4URjhMw7UXpSrFh7nRCiGiraeI1W4xAlUTgRQaAM7WhZuJHGmAmxo1BClEbhhCFaGtqhFTtETdQ0GZaQQEDhhDCNdokl5CY5hnSoakJYQOGEEOI3b1biyE2qDdh8Xqmjgv1NKJgQVlA4EYlY807EGNqRalIsq8uJ5bj5n9FEc1KIsqSumlAwISyhcEIIIYQQplA4IYSQAEdVE8IaCicioiXFzrxZsUN7nRAhVLsBm5/zTaQc0qFgQlhE99ZhUKzOjCs2o9LdIALojVZYzXqf328zctCZg0TskfootXW9kGXEWtu2XovBhKu7Bi7Iv6XhHEdLy5VGlRONUtNOsYQQQogjCici0/rQDqsrdgghwmixakK0g8IJo7S0W6y/aCM2omoMzjehYEJYR+GEEDmYrEr3oEXBnreJUR2pNmDTAgomRA0onEhA60M7hBBCiJQonDBMC0M7ct0A0JtdYgnxhxZW6lDVhKgFhRMNC7Rt7P0VbqQt6onIGJpvQsGEqAmFE4nQ0I73aCM2QqRFwYSojU/hZO3atUhKSoLJZEJaWhqOHDnitv3WrVuRnJwMk8mEPn36YNeuXT51NhBpYWiHFf7e/E+rGk3uX9eLMGLG4l2LAWEbsBEihsrKSkyaNAmRkZGIjo7G9OnTUVvrvspdX1+P2bNno23btoiIiMDYsWNRXl7u1Ka0tBSjRo1CWFgY2rVrh2effRaNjTcnhm/btg333XcfYmNjERkZifT0dOzevbvFcy5btgxBQUGYM2eO0/NlZWWYPHky4uPjER4ejgEDBuAf//hHs/fv3LkTaWlpCA0NRevWrTFmzBjPF8eB4HCyZcsWZGdnIzc3F0ePHkW/fv2QmZmJy5cvu2x/8OBBTJw4EdOnT8exY8cwZswYjBkzBt9//73QUxNCAphqt65XGFVN2DJp0iQcP34ce/bswY4dO3DgwAHMnDnT7Xvmzp2Lzz77DFu3bsWXX36Jixcv4qGHHuJft1qtGDVqFBoaGnDw4EFs2LAB69evR05ODt/mwIEDuO+++7Br1y4UFRXhnnvuwejRo3Hs2LFm5/vmm2/wl7/8BX379m322pQpU1BSUoLt27fju+++w0MPPYRx48Y5Hecf//gHJk+ejGnTpuE///kPvv76azzyyCOCrlMQx3GCfqVJS0vD7bffjjVr1gAAbDYbEhMT8eSTT2L+/PnN2o8fPx51dXXYsWMH/9ygQYOQkpKCvLw8l+cwm80wm2/+lltdXY2OHTuipCgBrSLUNRJVYfV/HkOFH1vZ/2oN97pteWOU98dtbOV9W4vnPvzXEuaxTWWD++NUmz1fp7oG923qzC0PxzXUh7h9r7XBw/b19e5f1zW0vH297rr7P/eeCmx6D6+7W07s6b033u/+Y0Rv9vwx42n7+uB678OJt0uJhSw5Fjwh9pq/8038ez9wYyt3FjRyFhywfIqqqipERXn/OSNETU0NoqKicLdhDILh/u+qJ42w4EDDpzh//jwiIyP5541GI4xG3z+PT5w4gV69euGbb77BwIEDAQD5+fkYOXIkfvnlFyQkJDR7T3V1NWJjY7Fp0yY8/PDDAICTJ0+iZ8+eKCwsxKBBg/Cvf/0Lv/3tb3Hx4kXExcUBAPLy8jBv3jxcuXIFBoPrz7XbbrsN48ePdwoxtbW1GDBgAN566y28/PLLSElJwapVq/jXIyIi8Pbbb2Py5Mn8c23btsVrr72Gxx57DI2NjUhKSsILL7yA6dOn+3ytwAlgNps5vV7PffLJJ07PT5kyhXvggQdcvicxMZF74403nJ7Lycnh+vbt2+J5cnNzOQD0oAc96EEPDT1Onz4t5CtHkOvXr3Px8fGi9TUiIqLZc7m5uX718b333uOio6OdnrNYLJxer+e2bdvm8j0FBQUcAO6///2v0/MdO3bkVq5cyXEcxy1evJjr16+f0+tnzpzhAHBHjx51eVyr1colJiZyb775ptPzU6ZM4ebMmcNxHMcNHTqUe+qpp5xev++++7hRo0Zxv/76K2e1WrmPPvqICwsL43766SeO4zju8OHDHADu/fff51JSUrj4+HhuxIgR3HfffdfidXFF0I3/KioqYLVa+WRmFxcXh5MnT7p8T1lZmcv2ZWVlLZ5nwYIFyM7O5v+7qqoKnTp1QmlpqWSpWwtqamqQmJjYLO0TZ3SdPKNr5B26Tt6xV7/btGkj2TlMJhPOnj2LhgZxVt1xHIegIOdqpj9VE+DG92G7du2cngsODkabNm1a/E4sKyuDwWBAdHS00/OO36Mtfc/aX3NlxYoVqK2txbhx4/jnNm/ejKNHj+Kbb75p8Wf4+OOPMX78eLRt2xbBwcEICwvDJ598gm7dugEAzpw5AwBYsmQJVq5ciaSkJLz++usYNmwYfvzxR6//DDB5V+KWSmdRUVH0AeCFyMhIuk5eoOvkGV0j79B18o5OJ+2wvMlkgsnkYYa3BObPn4/XXnvNbZsTJ07I1BvPNm3ahBdeeAH//Oc/+bB0/vx5PPXUU9izZ4/ba7h48WJUVVXhiy++QExMDD799FOMGzcO//73v9GnTx/YbDeGXxcuXIixY8cCAD744APccsst2Lp1K/74xz961UdB4SQmJgZ6vb7ZLOHy8nLEx8e7fE98fLyg9oQQQoiaPP3003j00UfdtunSpQvi4+ObLR5pbGxEZWWl2+/QhoYGVFVVOVVPHL9H4+Pjm62atX/vNj3u5s2b8dhjj2Hr1q3IyMjgny8qKsLly5cxYMAA/jmr1YoDBw5gzZo1MJvNOHfuHNasWYPvv/8et912GwCgX79++Pe//421a9ciLy8P7du3BwD06tWLP47RaESXLl1QWlrq9ho5EhRjDQYDUlNTUVBQwD9ns9lQUFCA9PR0l+9JT093ag8Ae/bsabE9IYQQoiaxsbFITk52+zAYDEhPT0dVVRWKior49+7duxc2mw1paWkuj52amoqQkBCn79GSkhKUlpby36Pp6en47rvvnILPnj17EBkZ6RQSPvroI0ybNg0fffQRRo0a5XSee++9F9999x2Ki4v5x8CBAzFp0iQUFxdDr9fj2v82BWxaAdPr9XzFJDU1FUajESUlJfzrFosF586dQ6dOnby/qIJmqHAct3nzZs5oNHLr16/nfvjhB27mzJlcdHQ0V1ZWxnEcx02ePJmbP38+3/7rr7/mgoODuRUrVnAnTpzgcnNzuZCQEEGTY+rr67nc3Fyuvr5eaHcDCl0n79B18oyukXfoOnmHrtNNI0aM4Pr3788dPnyY++qrr7ju3btzEydO5F//5ZdfuFtvvZU7fPgw/9ysWbO4jh07cnv37uW+/fZbLj09nUtPT+dfb2xs5Hr37s0NHz6cKy4u5vLz87nY2FhuwYIFfJuNGzdywcHB3Nq1a7lLly7xj6qqqhb72nRCbENDA9etWzduyJAh3OHDh7lTp05xK1as4IKCgridO3fy7Z566imuQ4cO3O7du7mTJ09y06dP59q1a8dVVlZ6fZ0EhxOO47g333yT69ixI2cwGLg77riDO3TokNMPM3XqVKf2H3/8MdejRw/OYDBwt912m9MPQQghhASKX3/9lZs4cSIXERHBRUZGctOmTeOuXr3Kv3727FkOALdv3z7+uevXr3NPPPEE17p1ay4sLIx78MEHuUuXLjkd99y5c9z999/PhYaGcjExMdzTTz/NWSwW/vWhQ4e6XJXU9PvakavVOj/++CP30EMPce3atePCwsK4vn37ch9++KFTm4aGBu7pp5/m2rVrx7Vq1YrLyMjgvv/+e0HXSfA+J4QQQgghUlLXjmaEEEII0TwKJ4QQQghhCoUTQgghhDCFwgkhhBBCmMJMOFm7di2SkpJgMpmQlpbWbEOZprZu3Yrk5GSYTCb06dMHu3btkqmnyhJyndatW4chQ4agdevWaN26NTIyMjxeVy0Q+mfJbvPmzQgKChJ8a2+1EnqdqqqqMHv2bLRv3x5GoxE9evQIiL93Qq/TqlWrcOuttyI0NBSJiYmYO3cu6uvd3FlRAw4cOIDRo0cjISEBQUFB+PTTTz2+Z//+/RgwYACMRiO6deuG9evXS95PoiKC1vZIZPPmzZzBYODef/997vjx49yMGTO46Ohorry83GX7r7/+mtPr9dzy5cu5H374gVu0aJHgvVPUSOh1euSRR7i1a9dyx44d406cOME9+uijXFRUFPfLL7/I3HP5CL1GdmfPnuU6dOjADRkyhPvd734nT2cVJPQ6mc1mbuDAgdzIkSO5r776ijt79iy3f/9+rri4WOaey0voddq4cSNnNBq5jRs3cmfPnuV2797NtW/fnps7d67MPZfXrl27uIULF3Lbtm3jADS7OWxTZ86c4cLCwrjs7Gzuhx9+4N58801Or9dz+fn58nSYMI+JcHLHHXdws2fP5v/barVyCQkJ3NKlS122HzduHDdq1Cin59LS0rg//vGPkvZTaUKvU1ONjY1cq1atuA0bNkjVRcX5co0aGxu5wYMHc++++y43derUgAgnQq/T22+/zXXp0oVraGiQq4tMEHqdZs+ezf3mN79xei47O5u78847Je0nS7wJJ8899xx32223OT03fvx4LjMzU8KeETVRfFinoaEBRUVFTnv863Q6ZGRkoLCw0OV7CgsLndoDQGZmZovttcCX69TUtWvXYLFYJL0zqJJ8vUYvvvgi2rVrh+nTp8vRTcX5cp22b9+O9PR0zJ49G3FxcejduzdeffVVWK1WubotO1+u0+DBg1FUVMQP/Zw5cwa7du3CyJEjZemzWgTiZzgRRvG7EldUVMBqtbq83fPJkyddvqel20O3dGtoLfDlOjU1b948JCQkNPtQ0ApfrtFXX32F9957D8XFxTL0kA2+XKczZ85g7969mDRpEnbt2oVTp07hiSeegMViQW5urhzdlp0v1+mRRx5BRUUF7rrrLnAch8bGRsyaNQvPP/+8HF1WjZY+w2tqanD9+nWEhoYq1DPCCsUrJ0Qey5Ytw+bNm/HJJ58ocktxFl29ehWTJ0/GunXrEBMTo3R3mGaz2dCuXTu88847SE1Nxfjx47Fw4ULk5eUp3TWm7N+/H6+++ireeustHD16FNu2bcPOnTvx0ksvKd01QlRF8cpJTEwM9Ho9f3tnO8fbQTcVHx8vqL0W+HKd7FasWIFly5bhiy++QN++faXspqKEXqPTp0/j3LlzGD16NP+c/c6awcHBKCkpQdeuXaXttAJ8+bPUvn17hISEQK/X88/17NkTZWVlaGhogMFgkLTPSvDlOi1evBiTJ0/GY489BgDo06cP6urqMHPmTCxcuLDZ3VwDVUuf4ZGRkVQ1IQAYqJwYDAakpqY63Q7aZrOhoKCAvx10U+np6U7tgRu3h26pvRb4cp0AYPny5XjppZeQn5+PgQMHytFVxQi9RsnJyc1uEf7AAw/gnnvuQXFxMRITE+Xsvmx8+bN055134tSpU3x4A4Aff/wR7du312QwAXy7TteuXXN5O3kA4Og2ZrxA/AwnAik9I5fjbizXMxqN3Pr167kffviBmzlzJhcdHc2VlZVxHMdxkydP5ubPn8+3//rrr7ng4GBuxYoV3IkTJ7jc3NyAWUos5DotW7aMMxgM3N///nenW2Q73gFTa4Reo6YCZbWO0OtUWlrKtWrVisvKyuJKSkq4HTt2cO3ateNefvllpX4EWQi9Trm5uVyrVq24jz76iDtz5gz3+eefc127duXGjRun1I8gi6tXr3LHjh3jjh07xgHgVq5cyR07doz7+eefOY7juPnz53OTJ0/m29uXEj/77LPciRMnuLVr19JSYuKEiXDCcRz35ptvch07duQMBgN3xx13cIcOHeJfGzp0aLPbOn/88cdcjx49OIPBwN12223czp07Ze6xMoRcp06dOrm8RXZubq78HZeR0D9LjgIlnHCc8Ot08OBBLi0tjTMajVyXLl24V155hWtsbJS51/ITcp0sFgu3ZMkSrmvXrpzJZOISExO5J554gvvvf/8rf8dltG/fPpefNfZrM3XqVG7o0KHN3pOSksIZDAauS5cu3AcffCB7vwm7gjiOao2EEEIIYYfic04IIYQQQhxROCGEEEIIUyicEEIIIYQpFE4IIYQQwhQKJ4QQQghhCoUTQgghhDCFwgkhhBBCmELhhBBCCCFMoXBCCCGEEKZQOCGEEEIIUyicEEIIIYQp/x/WD4BYzeOYPQAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "np.random.seed(54151610)\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test, trial = TestFunction(space), TrialFunction(space)\n", + "\n", + "\n", + "def forward(m):\n", + " v = Constant(name=\"v\")\n", + " Assembly(v, Constant(1.0) * dx(mesh)).solve()\n", + " k = Constant(name=\"k\")\n", + " Assembly(k, (m / v) * dx).solve()\n", + " m_tilde = Function(m.function_space(), name=\"m_tilde\").interpolate(m - k)\n", + " \n", + " u = Function(space, name=\"u\")\n", + " nullspace = VectorSpaceBasis(comm=u.comm, constant=True)\n", + "\n", + " solve(inner(grad(trial), grad(test)) * dx == -inner(m_tilde, test) * dx,\n", + " u, nullspace=nullspace, transpose_nullspace=nullspace,\n", + " solver_parameters={\"ksp_type\": \"cg\", \"pc_type\": \"hypre\",\n", + " \"pc_hypre_type\": \"boomeramg\",\n", + " \"ksp_rtol\": 1.0e-10, \"ksp_atol\": 1.0e-16})\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(0.5 * inner(u, u) * dx)\n", + " return u, J\n", + "\n", + "\n", + "m = Function(space, name=\"m\").interpolate(cos(pi * X[0]) * cos(pi * X[1]))\n", + "\n", + "start_manager()\n", + "u, J = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ = compute_gradient(J, m)\n", + "\n", + "\n", + "def plot_output(u, title):\n", + " r = (u.dat.data_ro.min(), u.dat.data_ro.max())\n", + " eps = (r[1] - r[0]) * 1.0e-12\n", + " p = tricontourf(u, np.linspace(r[0] - eps, r[1] + eps, 32))\n", + " plt.gca().set_title(title)\n", + " plt.colorbar(p)\n", + " plt.gca().set_aspect(1.0)\n", + "\n", + "\n", + "plot_output(u, title=\"u\")\n", + "plot_output(dJ.riesz_representation(\"L2\"), title=r\"$g^\\sharp$\")\n", + "\n", + "\n", + "def forward_J(m):\n", + " _, J = forward(m)\n", + " return J\n", + "\n", + "\n", + "min_order = taylor_test(forward_J, m, J_val=J.value, dJ=dJ)\n", + "assert min_order > 1.99" + ] + }, + { + "cell_type": "markdown", + "id": "23af6882-f085-4fa5-ae75-3ff04b367e3c", + "metadata": {}, + "source": [ + "## Defining a custom `Equation`\n", + "\n", + "Since the $u$ we have computed should satisfy\n", + "\n", + "$$\\sum_i \\tilde{u}_i = 0,$$\n", + "\n", + "we should have that the derivative of this quantity with respect to $m$, subject to the forward problem being solved, is zero. We will compute this derivative by defining a custom operation which sums the degrees of freedom of a finite element discretized function. We can do this by inheriting from the `Equation` class and implementing appropriate methods." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "1d3bdc61-be2a-4e94-82f7-d8f6d5faa92d", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:47:38.458585Z", + "iopub.status.busy": "2023-12-20T16:47:38.458276Z", + "iopub.status.idle": "2023-12-20T16:47:38.464655Z", + "shell.execute_reply": "2023-12-20T16:47:38.464080Z" + } + }, + "outputs": [], + "source": [ + "from tlm_adjoint import Equation\n", + "\n", + "\n", + "class Sum(Equation):\n", + " def __init__(self, x, y):\n", + " super().__init__(x, deps=(x, y), nl_deps=(), ic=False, adj_ic=False,\n", + " adj_type=\"conjugate_dual\")\n", + "\n", + " def forward_solve(self, x, deps=None):\n", + " if deps is None:\n", + " deps = self.dependencies()\n", + " _, y = deps\n", + " with y.dat.vec_ro as y_v:\n", + " y_sum = y_v.sum()\n", + " x.assign(y_sum)\n", + "\n", + " def adjoint_jacobian_solve(self, adj_x, nl_deps, b):\n", + " return b\n", + "\n", + " def adjoint_derivative_action(self, nl_deps, dep_index, adj_x):\n", + " if dep_index == 0:\n", + " return adj_x\n", + " elif dep_index == 1:\n", + " _, y = self.dependencies()\n", + " b = Cofunction(var_space(y).dual())\n", + " adj_x = float(adj_x)\n", + " b.dat.data[:] = -adj_x\n", + " return b\n", + " else:\n", + " raise ValueError(\"dep_index out of range\")\n", + "\n", + " def tangent_linear(self, M, dM, tlm_map):\n", + " tau_x, tau_y = (tlm_map[dep] for dep in self.dependencies())\n", + " if tau_y is None:\n", + " return ZeroAssignment(tau_x)\n", + " else:\n", + " return Sum(tau_x, tau_y)" + ] + }, + { + "cell_type": "markdown", + "id": "854d0055-7138-4728-a775-38cc8fecabf5", + "metadata": {}, + "source": [ + "We'll investgate how this is put together shortly.\n", + "\n", + "First, we check that the `Sum` class works. In the following we check that we can compute the sum of the degrees of freedom of a function, and then use Taylor remainder convergence tests to verify a first order tangent-linear calculation, a first order adjoint calculation, and a reverse-over-forward adjoint calculation of a Hessian action." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "0ed98c56-fa77-499c-8243-8f83727c2aaf", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:47:38.467003Z", + "iopub.status.busy": "2023-12-20T16:47:38.466805Z", + "iopub.status.idle": "2023-12-20T16:47:38.982591Z", + "shell.execute_reply": "2023-12-20T16:47:38.982006Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "float(m_sum)=3.97146153200585\n", + "m_sum_ref=3.97146153200585\n", + "Error norms, no tangent-linear = [2.32844986 1.15714317 0.57680653 0.28796267 0.14387127]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no tangent-linear = [1.00880244 1.00440797 1.00220568 1.00110326]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with tangent-linear = [0.02826961 0.00705304 0.00176147 0.00044014 0.00011001]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with tangent-linear = [2.00293418 2.00146933 2.00073523 2.00036775]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [2.68976716 1.33545989 0.66538227 0.33210525 0.16590628]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [1.01014465 1.0050813 1.0025429 1.00127201]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [0.03760673 0.00937967 0.00234217 0.0005852 0.00014626]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [2.00338159 2.00169377 2.00084763 2.000424 ]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [5.89746632 2.94164753 1.46905236 0.73408333 0.36693095]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [1.00347088 1.00173858 1.00087008 1.00043523]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [0.02834252 0.00708563 0.00177141 0.00044285 0.00011071]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [2. 2. 2. 2.]\n" + ] + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "np.random.seed(13561700)\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(m):\n", + " m_sum = Float(name=\"m_sum\")\n", + " Sum(m_sum, m).solve()\n", + "\n", + " J = m_sum ** 3\n", + " \n", + " return m_sum, J\n", + "\n", + "\n", + "def forward_J(m):\n", + " _, J = forward(m)\n", + " return J\n", + "\n", + "\n", + "mesh = UnitIntervalMesh(10)\n", + "x, = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Discontinuous Lagrange\", 0)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(exp(-x) * sin(pi * x))\n", + "with m.dat.vec_ro as m_v:\n", + " m_sum_ref = m_v.sum()\n", + "\n", + "start_manager()\n", + "m_sum, J = forward(m)\n", + "stop_manager()\n", + "\n", + "# Verify the forward calculation\n", + "\n", + "print(f\"{float(m_sum)=}\")\n", + "print(f\"{m_sum_ref=}\")\n", + "assert abs(float(m_sum) - m_sum_ref) == 0.0\n", + "\n", + "# Verify tangent-linear and adjoint calculations\n", + "\n", + "min_order = taylor_test_tlm(forward_J, m, tlm_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=2)\n", + "assert min_order > 1.99" + ] + }, + { + "cell_type": "markdown", + "id": "f6236cab-0eb6-4fad-ab87-e71e5ab071a2", + "metadata": {}, + "source": [ + "We can now use `Sum` with the Poisson solver. We find, as expected, that the derivative of the sum of the degrees of freedom for $u$ with respect to $m$, subject to the forward problem being solved, is zero." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "3fa84206-fdc1-4672-83df-190fc974fa43", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:47:38.985170Z", + "iopub.status.busy": "2023-12-20T16:47:38.984964Z", + "iopub.status.idle": "2023-12-20T16:47:39.089358Z", + "shell.execute_reply": "2023-12-20T16:47:39.088580Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "dK_norm=0.0\n" + ] + } + ], + "source": [ + "%matplotlib inline\n", + "\n", + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "reset_manager()\n", + "\n", + "mesh = UnitSquareMesh(10, 10)\n", + "X = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Lagrange\", 1)\n", + "test, trial = TestFunction(space), TrialFunction(space)\n", + "\n", + "\n", + "def forward(m):\n", + " v = Constant(name=\"v\")\n", + " Assembly(v, Constant(1.0) * dx(mesh)).solve()\n", + " k = Constant(name=\"k\")\n", + " Assembly(k, (m / v) * dx).solve()\n", + " m_tilde = Function(m.function_space(), name=\"m_tilde\").interpolate(m - k)\n", + " \n", + " u = Function(space, name=\"u\")\n", + " nullspace = VectorSpaceBasis(comm=u.comm, constant=True)\n", + "\n", + " solve(inner(grad(trial), grad(test)) * dx == -inner(m_tilde, test) * dx,\n", + " u, nullspace=nullspace, transpose_nullspace=nullspace,\n", + " solver_parameters={\"ksp_type\": \"cg\", \"pc_type\": \"hypre\",\n", + " \"pc_hypre_type\": \"boomeramg\",\n", + " \"ksp_rtol\": 1.0e-10, \"ksp_atol\": 1.0e-16})\n", + "\n", + " J = Functional(name=\"J\")\n", + " J.assign(0.5 * inner(u, u) * dx)\n", + " \n", + " u_sum = Constant(name=\"u_sum\")\n", + " Sum(u_sum, u).solve()\n", + "\n", + " K = Functional(name=\"K\")\n", + " K.assign(u_sum)\n", + "\n", + " return u, J, K\n", + "\n", + "\n", + "m = Function(space, name=\"m\").interpolate(cos(pi * X[0]) * cos(pi * X[1]))\n", + "\n", + "start_manager()\n", + "u, J, K = forward(m)\n", + "stop_manager()\n", + "\n", + "dJ, dK = compute_gradient((J, K), m)\n", + "\n", + "dK_norm = abs(dK.dat.data_ro).max()\n", + "print(f\"{dK_norm=}\")\n", + "\n", + "assert dK_norm == 0.0" + ] + }, + { + "cell_type": "markdown", + "id": "d7929c30-35c5-4ac7-bf43-3761685fd03b", + "metadata": {}, + "source": [ + "## `Equation` methods\n", + "\n", + "We now return to the definition of the `Sum` class, and consider each method in turn.\n", + "\n", + "### `__init__`\n", + "\n", + "```\n", + "def __init__(self, x, y):\n", + " super().__init__(x, deps=(x, y), nl_deps=(), ic=False, adj_ic=False,\n", + " adj_type=\"conjugate_dual\")\n", + "```\n", + "\n", + "The arguments passed to the base class constructor are:\n", + "\n", + "- The output of the forward operation, `x`.\n", + "- `deps`: Defines all inputs and outputs of the forward operation.\n", + "- `nl_deps`: Defines elements of `deps` on which the associated adjoint calculations can depend.\n", + "- `ic`: Whether the forward operation accepts a non-zero 'initial guess'.\n", + "- `adj_ic`: Whether an adjoint solve accepts a non-zero 'initial guess'.\n", + "- `adj_type`: Either `\"primal\"` or `\"conjugate_dual\"` defining the space for an associated adjoint variable.\n", + "\n", + "For this operation there is one output `x` and one input `y`. The operation is defined by the forward residual function\n", + "\n", + "$$F \\left( x, y \\right) = x - \\sum_i \\tilde{u}_i.$$\n", + "\n", + "Given a value for $y$ the output of the operation is defined to be the $x$ for which $F \\left( x, y \\right) = 0$. $F$ depends linearly on both $x$ and $y$, and so associated adjoint calculations are independent of both $x$ and $y$. Hence we set `deps=(x, y)` and `nl_deps=()`.\n", + "\n", + "`ic` and `adj_ic` are used to indicate whether non-zero initial guesses should be supplied to forward and adjoint iterative solvers respectively. Here we do not use iterative solvers, and so no non-zero initial guesses are needed. Hence we set `ic=False` and `adj_ic=False`.\n", + "\n", + "`adj_type` indicates whether an adjoint variable associated with the operation is:\n", + "\n", + "- In the same space as `x`. This is the case where $F$ maps to an element in the antidual space associated with $x$, and is indicated with `adj_x_type=\"primal\"`. A typical example of this case is an operation defined via the solution of a finite element variational problem.\n", + "- In the antidual space associated with the space for `x`. This is the case where $F$ maps to an element in same space as $x$, and is indiciated with `adj_x_type=\"conjugate_dual\"`.\n", + "\n", + "### `forward_solve`\n", + "\n", + "```\n", + "def forward_solve(self, x, deps=None):\n", + " if deps is None:\n", + " deps = self.dependencies()\n", + " _, y = deps\n", + " with y.dat.vec_ro as y_v:\n", + " y_sum = y_v.sum()\n", + " x.assign(y_sum)\n", + "```\n", + "\n", + "This method computes the output of the forward operation, storing the result in `x`. `deps` is used in rerunning the forward calculation when making use of a checkpointing schedule. If supplied, `deps` contains values associated with the dependencies as defined in the constructor (defined by the `deps` argument passed to the base class constructor). Otherwise the values contained in `self.dependencies()` are used.\n", + "\n", + "### `adjoint_jacobian_solve`\n", + "\n", + "```\n", + "def adjoint_jacobian_solve(self, adj_x, nl_deps, b):\n", + " return b\n", + "```\n", + "\n", + "Solves a linear problem\n", + "\n", + "$$\\frac{\\partial F}{\\partial x}^T \\lambda = b,$$\n", + "\n", + "for the adjoint solution $\\lambda$, returning the result. The right-hand-side $b$ is defined by the argument `b`. In this example $\\partial F / \\partial x$ is simply the identity, and so $\\lambda = b$.\n", + "\n", + "`adj_x` can contain an initial guess for an iterative solver used to compute $\\lambda$. `nl_deps` contains values associated with the forward dependencies of the adjoint, as defined in the constructor (defined by the `nl_deps` argument passed to the base class constructor).\n", + "\n", + "### `adjoint_derivative_action`\n", + "\n", + "```\n", + "def adjoint_derivative_action(self, nl_deps, dep_index, adj_x):\n", + " if dep_index == 0:\n", + " return adj_x\n", + " elif dep_index == 1:\n", + " _, y = self.dependencies()\n", + " b = Cofunction(var_space(y).dual())\n", + " adj_x = float(adj_x)\n", + " b.dat.data[:] = -adj_x\n", + " return b\n", + " else:\n", + " raise ValueError(\"dep_index out of range\")\n", + "```\n", + "\n", + "Computes\n", + "\n", + "$$\\frac{\\partial F}{\\partial v}^T \\lambda,$$\n", + "\n", + "where $v$ is the `dep_index`th dependency (for the dependencies defined by the `deps` argument passed to the base class constructor). Again `nl_deps` contains values associated with the forward dependencies of the adjoint, as defined in the constructor.\n", + "\n", + "### `tangent_linear`\n", + "\n", + "```\n", + "def tangent_linear(self, M, dM, tlm_map):\n", + " tau_x, tau_y = (tlm_map[dep] for dep in self.dependencies())\n", + " if tau_y is None:\n", + " return ZeroAssignment(tau_x)\n", + " else:\n", + " return Sum(tau_x, tau_y)\n", + "```\n", + "\n", + "Constructs a tangent-linear operation, for a tangent-linear computing directional derivatives with respect to the control defined by `M` with direction defined by `dM`. `tlm_map` is used to access tangent-linear variables.\n", + "\n", + "The new operation is defined by the residual function\n", + "\n", + "$$F_\\tau \\left( \\tau_x, \\tau_y \\right) = \\frac{\\partial F}{\\partial x} \\tau_x + \\frac{\\partial F}{\\partial y} \\tau_y,$$\n", + "\n", + "where $\\tau_x$ and $\\tau_y$ are the tangent-linear variables associated with $x$ and $y$ respectively. Given a value for $\\tau_y$ the output of the operation is defined to be the $\\tau_x$ for which $F_\\tau \\left( \\tau_x, \\tau_y \\right) = 0$.\n", + "\n", + "Note that this method returns an `Equation` instance – here either a `ZeroAssignment`, for the case where the tangent-linear operation sets a tangent-linear variable equal to zero, or another `Sum`. This new operation can then be recorded by the internal manager, so that reverse-over-forward algorithmic differentiation can be applied.\n", + "\n", + "### Reference dropping\n", + "\n", + "An important method, not defined in this example, is the `drop_references` method. This method is used to drop references to forward data. For example if an `Equation` subclass defines a `Function` attribute `_function`, then the `drop_references` method might look like\n", + "\n", + "```\n", + "def drop_references(self):\n", + " super().drop_references()\n", + " self._function = var_replacement(self._function)\n", + "```\n", + "\n", + "Here `var_replacement` returns a 'symbolic only' version of `self._function`, which can for example be used in UFL expressions, but which has no associated value.\n", + "\n", + "## Defining custom operations using JAX\n", + "\n", + "In some cases it may be more convenient to directly implement an operation using lower level code. tlm_adjoint integrates with [JAX](https://jax.readthedocs.io) to allow this. For example a Firedrake `Function` can be converted to a JAX array using\n", + "\n", + "```\n", + "v = to_jax(x)\n", + "```\n", + "\n", + "and the result converted back to a Firedrake `Function` using\n", + "\n", + "```\n", + "x = to_firedrake(v, space)\n", + "```\n", + "\n", + "Here we use JAX to compute the sum of the degrees of freedom for a Firedrake function (assuming a serial calculation)." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "c98042cd-c9b5-4ac8-abd5-8fbb1a5717ff", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:47:39.091849Z", + "iopub.status.busy": "2023-12-20T16:47:39.091646Z", + "iopub.status.idle": "2023-12-20T16:47:39.721345Z", + "shell.execute_reply": "2023-12-20T16:47:39.720708Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "float(m_sum)=3.9714615320058506\n", + "m_sum_ref=3.97146153200585\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no tangent-linear = [3.04885302 1.51235222 0.75316949 0.37583459 0.18772994]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no tangent-linear = [1.01147243 1.00574771 1.00287673 1.00143908]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with tangent-linear = [0.04816965 0.01201053 0.00299865 0.00074916 0.00018723]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with tangent-linear = [2.0038242 2.00191591 2.00095891 2.00047969]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [2.15029451 1.06909935 0.53304195 0.26614458 0.13297826]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [1.00813834 1.00407494 1.00203891 1.00101982]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [2.41462429e-02 6.02522041e-03 1.50488756e-03 3.76044699e-04\n", + " 9.39890257e-05]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [2.0027128 2.00135832 2.00067964 2.00033994]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [7.11786236 3.54650747 1.77014781 0.88429742 0.44195459]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [1.00504505 1.00252916 1.00126624 1.00063354]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [0.04969483 0.01242371 0.00310593 0.00077648 0.00019412]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [2. 2. 2. 2.]\n" + ] + } + ], + "source": [ + "from firedrake import *\n", + "from tlm_adjoint.firedrake import *\n", + "\n", + "import numpy as np\n", + "\n", + "np.random.seed(81976463)\n", + "reset_manager()\n", + "\n", + "\n", + "def forward(m):\n", + " m_sum = new_jax_float(name=\"m_sum\")\n", + " call_jax(m_sum, to_jax(m), lambda v: v.sum())\n", + "\n", + " J = m_sum ** 3\n", + " \n", + " return m_sum, J\n", + "\n", + "\n", + "def forward_J(m):\n", + " _, J = forward(m)\n", + " return J\n", + "\n", + "\n", + "mesh = UnitIntervalMesh(10)\n", + "x, = SpatialCoordinate(mesh)\n", + "space = FunctionSpace(mesh, \"Discontinuous Lagrange\", 0)\n", + "\n", + "m = Function(space, name=\"m\")\n", + "m.interpolate(exp(-x) * sin(pi * x))\n", + "with m.dat.vec_ro as m_v:\n", + " m_sum_ref = m_v.sum()\n", + "\n", + "start_manager()\n", + "m_sum, J = forward(m)\n", + "stop_manager()\n", + "\n", + "# Verify the forward calculation\n", + "\n", + "print(f\"{float(m_sum)=}\")\n", + "print(f\"{m_sum_ref=}\")\n", + "assert abs(float(m_sum) - m_sum_ref) < 1.0e-15\n", + "\n", + "# Verify tangent-linear and adjoint calculations\n", + "\n", + "min_order = taylor_test_tlm(forward_J, m, tlm_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(forward_J, m, adjoint_order=2)\n", + "assert min_order > 1.99" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/7_jax_integration.html b/examples/7_jax_integration.html new file mode 100644 index 0000000..04f4811 --- /dev/null +++ b/examples/7_jax_integration.html @@ -0,0 +1,353 @@ + + + + + + + JAX integration — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

JAX integration

+

This notebook introduces the use of JAX with tlm_adjoint.

+

JAX can be used e.g. with the Firedrake backend as an escape-hatch, allowing custom operations to be implemented using lower level code. However it can also be used independently, without a finite element code generator. Here JAX is used with tlm_adjoint to implement and differentiate a finite difference code.

+
+

Forward problem

+

We consider the diffusion equation in the unit square domain, subject to homogeneous Dirichlet boundary conditions,

+
+\[\partial_t u = \kappa \left( \partial_{xx} + \partial_{yy} \right) u + m \left( x, y \right) \qquad \text{on} ~ \left( x, y \right) \in \left( 0, 1 \right)^2,\]
+
+\[u = 0 \qquad \text{on} ~ \partial \Omega.\]
+

This is discretized using a basic finite difference scheme, using second order centered finite differencing for the \(x\) and \(y\) dimensions and forward Euler for the \(t\) dimension.

+

Here we implement the discretization using JAX, for \(\kappa = 0.01\), \(t \in \left[ 0, 0.25 \right]\), a time step size of \(0.0025\), and a uniform grid with a grid spacing of \(0.02\). We consider the case where \(m \left( x, y \right) = 0\), and compute the \(L^4\) norm of the \(t = T\) solution (with the \(L^4\) norm defined via nearest neighbour interpolation of the finite difference solution, ignoring boundary points where the solution is zero).

+
+
[1]:
+
+
+
%matplotlib inline
+
+import matplotlib.pyplot as plt
+import jax
+
+jax.config.update("jax_enable_x64", True)
+
+N = 50
+kappa = 0.01
+T = 0.25
+N_t = 100
+
+dx = 1.0 / N
+dt = T / N_t
+
+
+x = jax.numpy.linspace(0.0, 1.0, N + 1)
+y = jax.numpy.linspace(0.0, 1.0, N + 1)
+X, Y = jax.numpy.meshgrid(x, y, indexing="ij")
+
+
+def timestep(x_n, m):
+    x_np1 = jax.numpy.zeros_like(x_n)
+    x_np1 = x_np1.at[1:-1, 1:-1].set(
+        x_n[1:-1, 1:-1]
+        + (kappa * dt / (dx * dx)) * (x_n[1:-1, 2:]
+                                      + x_n[:-2, 1:-1]
+                                      - 4.0 * x_n[1:-1, 1:-1]
+                                      + x_n[2:, 1:-1]
+                                      + x_n[1:-1, :-2])
+        + dt * m[1:-1, 1:-1])
+    return x_np1
+
+
+def functional(x_n):
+    x_n = x_n.reshape((N + 1, N + 1))
+    return (dx * dx * (x_n[1:-1, 1:-1] ** 4).sum()) ** 0.25
+
+
+def forward(x_0, m):
+    x_n = x_0.copy()
+    for _ in range(N_t):
+        x_n = timestep(x_n, m)
+    J = functional(x_n)
+    return x_n, J
+
+
+x_0 = jax.numpy.zeros((N + 1, N + 1), dtype=jax.numpy.double)
+x_0 = x_0.at[1:-1, 1:-1].set(jax.numpy.exp(-((X[1:-1, 1:-1] - 0.5) ** 2 + (Y[1:-1, 1:-1]- 0.5) ** 2) / (2.0 * (0.05 ** 2))))
+m = jax.numpy.zeros((N + 1, N + 1), dtype=jax.numpy.double)
+
+fig, ax = plt.subplots(1, 1)
+p = ax.contourf(x, y, x_0.T, 32)
+fig.colorbar(p)
+ax.set_aspect(1.0)
+ax.set_title("$x_0$")
+
+x_n, J = forward(x_0, m)
+print(f"{J=}")
+
+fig, ax = plt.subplots(1, 1)
+p = ax.contourf(x, y, x_n.T, 32)
+fig.colorbar(p)
+ax.set_aspect(1.0)
+ax.set_title("$x_{N_t}$")
+
+
+
+
+
+
+
+
+J=Array(0.11009761, dtype=float64)
+
+
+
+
[1]:
+
+
+
+
+Text(0.5, 1.0, '$x_{N_t}$')
+
+
+
+
+
+
+../_images/examples_7_jax_integration_1_2.png +
+
+
+
+
+
+../_images/examples_7_jax_integration_1_3.png +
+
+
+
+

Adding tlm_adjoint

+

The tlm_adjoint Vector class wraps ndim 1 JAX arrays. The following uses the tlm_adjoint call_jax function to record JAX operations on the internal tlm_adjoint manager. Note the use of new_block, indicating steps, and enabling use of a step-based checkpointing schedule.

+
+
[2]:
+
+
+
from tlm_adjoint import *
+
+import jax
+
+reset_manager()
+
+N = 50
+kappa = 0.01
+T = 0.25
+N_t = 100
+
+dx = 1.0 / N
+dt = T / N_t
+
+
+x = jax.numpy.linspace(0.0, 1.0, N + 1)
+y = jax.numpy.linspace(0.0, 1.0, N + 1)
+X, Y = jax.numpy.meshgrid(x, y, indexing="ij")
+
+
+def timestep(x_n, m):
+    x_n = x_n.reshape((N + 1, N + 1))
+    m = m.reshape((N + 1, N + 1))
+    x_np1 = jax.numpy.zeros_like(x_n)
+    x_np1 = x_np1.at[1:-1, 1:-1].set(
+        x_n[1:-1, 1:-1]
+        + (kappa * dt / (dx * dx)) * (x_n[1:-1, 2:]
+                                      + x_n[:-2, 1:-1]
+                                      - 4.0 * x_n[1:-1, 1:-1]
+                                      + x_n[2:, 1:-1]
+                                      + x_n[1:-1, :-2])
+        + dt * m[1:-1, 1:-1])
+    return x_np1.flatten()
+
+
+def functional(x_n):
+    x_n = x_n.reshape((N + 1, N + 1))
+    return (dx * dx * (x_n[1:-1, 1:-1] ** 4).sum()) ** 0.25
+
+
+def forward(x_0, m):
+    x_n = Vector((N + 1) ** 2)
+    x_np1 = Vector((N + 1) ** 2)
+    x_n.assign(x_0)
+    for n_t in range(N_t):
+        call_jax(x_np1, (x_n, m), timestep)
+        x_n.assign(x_np1)
+        if n_t < N_t - 1:
+            new_block()
+    J = new_jax_float()
+    call_jax(J, x_n, functional)
+    return J
+
+
+x_0 = jax.numpy.zeros((N + 1, N + 1), dtype=jax.numpy.double)
+x_0 = x_0.at[1:-1, 1:-1].set(jax.numpy.exp(-((X[1:-1, 1:-1] - 0.5) ** 2 + (Y[1:-1, 1:-1]- 0.5) ** 2) / (2.0 * (0.05 ** 2))))
+x_0 = Vector(x_0.flatten())
+m = Vector((N + 1) ** 2)
+
+start_manager()
+J = forward(x_0, m)
+stop_manager()
+
+
+
+
+
[2]:
+
+
+
+
+(True, True)
+
+
+

We can now verify first order tangent-linear and adjoint calculations, and a second order reverse-over-forward calculation, using Taylor remainder convergence tests. Here we consider derivatives with respect to \(m\).

+
+
[3]:
+
+
+
min_order = taylor_test_tlm(lambda m: forward(x_0, m), m, tlm_order=1)
+assert min_order > 2.00
+
+min_order = taylor_test_tlm_adjoint(lambda m: forward(x_0, m), m, adjoint_order=1)
+assert min_order > 2.00
+
+min_order = taylor_test_tlm_adjoint(lambda m: forward(x_0, m), m, adjoint_order=2)
+assert min_order > 2.00
+
+
+
+
+
+
+
+
+Error norms, no tangent-linear   = [5.42494549e-04 2.71112518e-04 1.35522677e-04 6.77529553e-05
+ 3.38743833e-05]
+Orders,      no tangent-linear   = [1.00071691 1.00035745 1.0001785  1.0000892 ]
+Error norms, with tangent-linear = [5.37911835e-07 1.34199793e-07 3.35180646e-08 8.37571371e-09
+ 2.09346468e-09]
+Orders,      with tangent-linear = [2.00298728 2.00137169 2.00065481 2.00031955]
+Error norms, no adjoint   = [5.44879307e-04 2.72292054e-04 1.36109209e-04 6.80454089e-05
+ 3.40204066e-05]
+Orders,      no adjoint   = [1.00078182 1.0003902  1.00019495 1.00009744]
+Error norms, with adjoint = [5.89556319e-07 1.47178961e-07 3.67713890e-08 9.19011302e-09
+ 2.29719812e-09]
+Orders,      with adjoint = [2.00205818 2.00091587 2.00042917 2.00020732]
+Error norms, no adjoint   = [1.03638959e-04 5.16630940e-05 2.57966044e-05 1.28900955e-05
+ 6.44306279e-06]
+Orders,      no adjoint   = [1.00436048 1.00195287 1.00091823 1.00044439]
+Error norms, with adjoint = [5.80998182e-07 1.34113654e-07 3.21142524e-08 7.85039270e-09
+ 1.94023890e-09]
+Orders,      with adjoint = [2.11507753 2.06217051 2.03237698 2.01653052]
+
+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/examples/7_jax_integration.ipynb b/examples/7_jax_integration.ipynb new file mode 100644 index 0000000..7dd6134 --- /dev/null +++ b/examples/7_jax_integration.ipynb @@ -0,0 +1,389 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "ea6ba877-3da2-4a5b-b878-8350e1c74b08", + "metadata": {}, + "source": [ + "# JAX integration\n", + "\n", + "This notebook introduces the use of [JAX](https://jax.readthedocs.io) with tlm_adjoint.\n", + "\n", + "JAX can be used e.g. with the Firedrake backend as an escape-hatch, allowing custom operations to be implemented using lower level code. However it can also be used independently, without a finite element code generator. Here JAX is used with tlm_adjoint to implement and differentiate a finite difference code.\n", + "\n", + "## Forward problem\n", + "\n", + "We consider the diffusion equation in the unit square domain, subject to homogeneous Dirichlet boundary conditions,\n", + "\n", + "$$\\partial_t u = \\kappa \\left( \\partial_{xx} + \\partial_{yy} \\right) u + m \\left( x, y \\right) \\qquad \\text{on} ~ \\left( x, y \\right) \\in \\left( 0, 1 \\right)^2,$$\n", + "$$u = 0 \\qquad \\text{on} ~ \\partial \\Omega.$$\n", + "\n", + "This is discretized using a basic finite difference scheme, using second order centered finite differencing for the $x$ and $y$ dimensions and forward Euler for the $t$ dimension.\n", + "\n", + "Here we implement the discretization using JAX, for $\\kappa = 0.01$, $t \\in \\left[ 0, 0.25 \\right]$, a time step size of $0.0025$, and a uniform grid with a grid spacing of $0.02$. We consider the case where $m \\left( x, y \\right) = 0$, and compute the $L^4$ norm of the $t = T$ solution (with the $L^4$ norm defined via nearest neighbour interpolation of the finite difference solution, ignoring boundary points where the solution is zero)." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "4d51cefb-f6e5-4178-bf9e-07469c6410ec", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:47:42.283256Z", + "iopub.status.busy": "2023-12-20T16:47:42.282495Z", + "iopub.status.idle": "2023-12-20T16:47:44.350553Z", + "shell.execute_reply": "2023-12-20T16:47:44.349907Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J=Array(0.11009761, dtype=float64)\n" + ] + }, + { + "data": { + "text/plain": [ + "Text(0.5, 1.0, '$x_{N_t}$')" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfkAAAGyCAYAAADnKb1LAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA92klEQVR4nO3de3QV5aH+8Sf3hEuCkJIQjERwKVKRIIH8EC/YpqZLFkpPW1Ap0NTisSZWyfECIoTLkVBrWfQASkWovdgDtgXKEU68RNFSY2kTs9QKeAEkpSbAsSYQct/z+4Puzd47sy+zd3YSJt/PWrMk735n5t2zhCfvOzPvG2UYhiEAAGA70T3dAAAAEBmEPAAANkXIAwBgU4Q8AAA2RcgDAGBThDwAADZFyAMAYFOEPAAANkXIAwBgU4Q8AAA2RcgDAGBThDwAADZFyAPd5Pjx40pMTNT3vvc9j/JXX31VcXFxWrBgQQ+1DIBdRbEKHdB9ioqK9Mwzz+ijjz7SiBEjdPDgQU2ePFnXX3+9du7cqehofu8G0HUIeaAbHT9+XKNGjdL3vvc9rVy5Urm5uRo4cKD27dun/v3793TzANhMbE83AOhLhg8frvnz52vTpk2qqqpSU1OT3njjDQIeQEQwNgh0swcffFAtLS169913tWvXLg0fPtzj85MnT2ratGnq37+/rrjiCpWXl/dQSwFc6OjJA93s8ccflyS1t7dr8ODBnT4vLCxUenq6Tp48qVdffVUzZ87URx99ZFoXAPyhJw90ox//+Md69tlntX79esXGxroC3+nMmTPauXOnli9frn79+unWW2/V2LFj9Yc//KGHWgzgQkbIA91k586dWrhwoVauXKnCwkLdfffd+uUvf6kjR4646nz00UcaMGCALr74YlfZ2LFj9be//a0nmgzgAkfIA92gsrJSs2fP1uzZs7V48WJJ0sMPP6zo6GiP3vyZM2eUnJzssW9ycrLOnDnTre0FYA+EPBBhf//73zV9+nSNHz9emzZtcpVnZGToe9/7nkdvfsCAAWpoaPDYv6GhQQMGDOjWNgOwB96TB3qRM2fOaPDgwTpy5IjrqfubbrpJc+fOVUFBQQ+3DsCFhpAHeplvf/vbSklJ0bp161ReXq558+bxdD2AkPAKHdDLPPXUU5o3b56GDBmiiy++WNu2bSPgAYSEnjwAADZl+cG7N998U9OnT1dGRoaioqK0c+fOgPvs3btX11xzjRISEnTZZZfpueeeC6GpAADACssh39jYqHHjxmnDhg1B1T9y5IimTZumm266SdXV1XrggQf0/e9/Xy+99JLlxgIAgOCFNVwfFRWlHTt2aMaMGT7rPPLII9q9e7fef/99V9ntt9+uL774QmVlZaGeGgAABBDxB+8qKiqUl5fnUZafn68HHnjA5z4tLS1qaWlx/exwOPT5559ryJAhioqKilRTAQARYBiGTp8+rYyMDEVHR256lubmZrW2tnbJseLj45WYmNglx+pJEQ/52tpapaWleZSlpaWpoaFBTU1NSkpK6rRPaWmpli9fHummAQC6UU1NjceUzV2publZWSP6q+6Eo0uOl56eriNHjlzwQd8rX6FbtGiRiouLXT/X19frkksu0Q3x31BsVFwPtgwAYFW70aY3W3do4MCBETtHa2ur6k449MFfhmngwPBGC06fdmjMxM/U2tpKyAeSnp6uuro6j7K6ujolJyeb9uIlKSEhQQkJCZ3KY6PiFBsVH5F2AgAiqztutw4cGK3kMEPeTiJ+JSZPnqzy8nKPsldeeUWTJ0+O9KkBAOjTLIf8mTNnVF1drerqaknnXpGrrq7WsWPHJJ0bap87d66r/j333KPDhw/r4Ycf1sGDB/XUU0/phRde0IIFC7rmGwAAAFOWQ/6vf/2rxo8fr/Hjx0uSiouLNX78eC1dulSS9Nlnn7kCX5IuvfRS7d69W6+88orGjRunn/zkJ3r22WeVn5/fRV8BAACYsXxPfurUqfL3ar3ZbHZTp07VO++8Y/VUAAAgDDydAACATRHyAADYFCEPAIBNEfIAANgUIQ8AQJg2bNigrKwsJSYmKjc3V/v37/dZt62tTStWrNCoUaOUmJiocePGmS7Ydvz4cX3nO9/RkCFDlJSUpLFjx+qvf/2rpXYR8gAAhGHbtm0qLi5WSUmJqqqqNG7cOOXn5+vEiROm9R977DH97Gc/07p16/TBBx/onnvu0Te+8Q2Pt9D++c9/asqUKYqLi9P//u//6oMPPtBPfvITXXTRRZbaFtZSs92loaFBKSkp+krCTKa1BYALTLvRqtdaXlB9fb2Sk5Mjcg5nTtQcHB72tLYNpx3KHH1cNTU1Hu31NeV6bm6uJk6cqPXr10s6t3JqZmam7rvvPi1cuLBT/YyMDC1evFiFhYWusm9+85tKSkrSr3/9a0nSwoUL9ac//Ul//OMfw/ou9OQBALZR19Guz8Lc6jraJUmZmZlKSUlxbaWlpZ3O19raqsrKSo8l1aOjo5WXl6eKigrTNra0tHRa+CYpKUn79u1z/bxr1y7l5OTo29/+toYOHarx48dr06ZNlq9Hr1yFDgCAnmbWk/d26tQpdXR0mC6pfvDgQdPj5ufna82aNbrhhhs0atQolZeXa/v27ero6HDVOXz4sJ5++mkVFxfr0Ucf1V/+8hf98Ic/VHx8vObNmxf0dyDkAQAwkZycHJHbCz/96U81f/58jR49WlFRURo1apQKCgq0ZcsWVx2Hw6GcnBytWrVKkjR+/Hi9//772rhxo6WQZ7geAIAQpaamKiYmxnRJ9fT0dNN9vvSlL2nnzp1qbGzUp59+qoMHD2rAgAEaOXKkq86wYcM0ZswYj/2uvPJKj7VhgkHIAwAQovj4eE2YMMFjSXWHw6Hy8vKAS6onJiZq+PDham9v1+9//3vddtttrs+mTJmiQ4cOedT/8MMPNWLECEvtY7geAIAwFBcXa968ecrJydGkSZO0du1aNTY2qqCgQJI0d+5cDR8+3PXg3p///GcdP35c2dnZOn78uJYtWyaHw6GHH37YdcwFCxbo2muv1apVqzRz5kzt379fzzzzjJ555hlLbSPkAQAIw6xZs3Ty5EktXbpUtbW1ys7OVllZmethvGPHjik6+vzAeXNzsx577DEdPnxYAwYM0C233KJf/epXGjRokKvOxIkTtWPHDi1atEgrVqzQpZdeqrVr12r27NmW2sZ78gCAiOrO9+T/+rc0DQjzPfkzpx3K+XJdRNvbXbgnDwCATRHyAADYFCEPAIBNEfIAANgUIQ8AgE0R8gAA2BQhDwCATTEZDgDANv7RMUD9O2LCOkZjR4ekuoD1LgT05AEAsClCHgAAmyLkAQCwKUIeAACbIuQBALApQh4AAJsi5AEAsClCHgAAmyLkAQCwKUIeAACbIuQBALApQh4AAJsi5AEAsClCHgAAm2KpWQCAbXzWPkhJbeFFW1N7exe1pufRkwcAwKYIeQAAbIqQBwDApgh5AABsipAHAMCmCHkAAGyKkAcAwKYIeQAAbIqQBwDApgh5AABsipAHACBMGzZsUFZWlhITE5Wbm6v9+/f7rb927VpdccUVSkpKUmZmphYsWKDm5mbTuqtXr1ZUVJQeeOABy+0i5AEACMO2bdtUXFyskpISVVVVady4ccrPz9eJEydM6//mN7/RwoULVVJSogMHDmjz5s3atm2bHn300U51//KXv+hnP/uZrr766pDaRsgDABCGNWvWaP78+SooKNCYMWO0ceNG9evXT1u2bDGt/9Zbb2nKlCm68847lZWVpZtvvll33HFHp97/mTNnNHv2bG3atEkXXXRRSG0j5AEAMNHQ0OCxtbS0dKrT2tqqyspK5eXlucqio6OVl5eniooK0+Nee+21qqysdIX64cOHtWfPHt1yyy0e9QoLCzVt2jSPY1vFUrMAANs43naREtviwjpGc1ubJCkzM9OjvKSkRMuWLfMoO3XqlDo6OpSWluZRnpaWpoMHD5oe/84779SpU6d03XXXyTAMtbe365577vEYrt+6dauqqqr0l7/8JazvQsgDAGCipqZGycnJrp8TEhK65Lh79+7VqlWr9NRTTyk3N1cff/yx7r//fq1cuVJLlixRTU2N7r//fr3yyitKTEwM61yEPAAAJpKTkz1C3kxqaqpiYmJUV1fnUV5XV6f09HTTfZYsWaI5c+bo+9//viRp7Nixamxs1N13363FixersrJSJ06c0DXXXOPap6OjQ2+++abWr1+vlpYWxcTEBPUduCcPAECI4uPjNWHCBJWXl7vKHA6HysvLNXnyZNN9zp49q+hoz/h1hrZhGPrqV7+q9957T9XV1a4tJydHs2fPVnV1ddABL9GTBwAgLMXFxZo3b55ycnI0adIkrV27Vo2NjSooKJAkzZ07V8OHD1dpaakkafr06VqzZo3Gjx/vGq5fsmSJpk+frpiYGA0cOFBXXXWVxzn69++vIUOGdCoPhJAHACAMs2bN0smTJ7V06VLV1tYqOztbZWVlrofxjh075tFzf+yxxxQVFaXHHntMx48f15e+9CVNnz5djz/+eJe3LcowDKPLj9rFGhoalJKSoq8kzFRsVHxPNwcAYEG70arXWl5QfX19wHvcoXLmxNI/5ylxQJhP159p04rcVyPa3u7CPXkAAGyKkAcAwKYIeQAAbIqQBwDApkIK+UguqQcAALqG5ZCP5JJ6AACg61gO+UgtqQcAALqWpZCP5JJ67lpaWjot8QcAAKyxNONdpJbU81ZaWqrly5dbaRoAAKprSVFCXHiT4bS0tHVRa3pexJ+ud19Sr6qqStu3b9fu3bu1cuVKn/ssWrRI9fX1rq2mpibSzQQAwHYs9eQjsaSe90o80rk1e7tq3V4AAPoqSz35SCypBwAAIsPyKnRdvaQeAACIDMsh35uX1AMAAOex1CwAIKK6c6nZf3/zm0oIc6nZljNt+tkNv2epWQAA0HsR8gAA2BQhDwCATRHyAADYFCEPAIBNEfIAANgUIQ8AgE1ZngwHAIDe6lTrAMW1hDefSltraxe1pufRkwcAwKYIeQAAbIqQBwDApgh5AABsipAHAMCmCHkAAGyKkAcAwKYIeQAAbIqQBwDApgh5AABsipAHACBMGzZsUFZWlhITE5Wbm6v9+/f7rDt16lRFRUV12qZNmyZJamtr0yOPPKKxY8eqf//+ysjI0Ny5c/WPf/zDcrsIeQAAwrBt2zYVFxerpKREVVVVGjdunPLz83XixAnT+tu3b9dnn33m2t5//33FxMTo29/+tiTp7Nmzqqqq0pIlS1RVVaXt27fr0KFDuvXWWy23jQVqAAAIw5o1azR//nwVFBRIkjZu3Kjdu3dry5YtWrhwYaf6gwcP9vh569at6tevnyvkU1JS9Morr3jUWb9+vSZNmqRjx47pkksuCbpt9OQBADDR0NDgsbW0tHSq09raqsrKSuXl5bnKoqOjlZeXp4qKiqDOs3nzZt1+++3q37+/zzr19fWKiorSoEGDLH0HevIAANv4v+Z+io1JCOsY7c3nojEzM9OjvKSkRMuWLfMoO3XqlDo6OpSWluZRnpaWpoMHDwY81/79+/X+++9r8+bNPus0NzfrkUce0R133KHk5OQgv8U5hDwAACZqamo8QjUhIbxfHsxs3rxZY8eO1aRJk0w/b2tr08yZM2UYhp5++mnLxyfkAQAwkZycHLDnnJqaqpiYGNXV1XmU19XVKT093e++jY2N2rp1q1asWGH6uTPgP/30U7322muWe/ES9+QBAAhZfHy8JkyYoPLycleZw+FQeXm5Jk+e7Hff3/72t2ppadF3vvOdTp85A/6jjz7Sq6++qiFDhoTUPnryAACEobi4WPPmzVNOTo4mTZqktWvXqrGx0fW0/dy5czV8+HCVlpZ67Ld582bNmDGjU4C3tbXpW9/6lqqqqvTiiy+qo6NDtbW1ks49mR8fHx902wh5AADCMGvWLJ08eVJLly5VbW2tsrOzVVZW5noY79ixY4qO9hw4P3TokPbt26eXX3650/GOHz+uXbt2SZKys7M9Pnv99dc1derUoNtGyAMAEKaioiIVFRWZfrZ3795OZVdccYUMwzCtn5WV5fMzq7gnDwCATRHyAADYFCEPAIBNEfIAANgUIQ8AgE0R8gAA2BQhDwCATRHyAADYFJPhAABs45/N/RQTHd5qcR3NMV3Ump5HTx4AAJsi5AEAsClCHgAAmyLkAQCwKUIeAACbIuQBALApQh4AAJsi5AEAsClCHgAAmyLkAQCwKUIeAACbIuQBALApQh4AAJsi5AEAsCmWmgUA2MaZpnjFRIW51GyT0UWt6Xn05AEAsClCHgAAm2K4Huhh0SkDXX921J/uwZYAsBtCHogw9xAPty6/BACwgpAHIsBKsId6XAIfQCCEPNBFIhXswZyPwAdghpAHwtDdwe4LgQ/ADCEPhKC3hLsZZ9sIewCEPBCksIN94IDQ9z19xvIu9O4BEPJAEEIK+HBCPdCxLIZ+dMpAgh7og0KaDGfDhg3KyspSYmKicnNztX//fr/1v/jiCxUWFmrYsGFKSEjQ5Zdfrj179oTUYKC7WQr4gQPOb5EUwnl68y0G4EIXiVy0ekwzlkN+27ZtKi4uVklJiaqqqjRu3Djl5+frxIkTpvVbW1v1ta99TUePHtXvfvc7HTp0SJs2bdLw4cMtNxboTtEpA4MPxu4I9i44t6XvBCAokchFq8f0JcowDEsz8efm5mrixIlav369JMnhcCgzM1P33XefFi5c2Kn+xo0b9eMf/1gHDx5UXFycpcY5NTQ0KCUlRV9JmKnYqPiQjgFYYSncg+QYkBRia6ToM03BVw5yKJ/he3SXdqNVr7W8oPr6eiUnJ0fkHM6cGPWrRYrplxjWsTrONuuTOaVBtzcSuWj1mL5Y6sm3traqsrJSeXl55w8QHa28vDxVVFSY7rNr1y5NnjxZhYWFSktL01VXXaVVq1apo6PD53laWlrU0NDgsQHdJaiAD7L37BiQ5NrCYek4QbaNXj3gn3cOtbS0dKoTiVwM5Zi+WHrw7tSpU+ro6FBaWppHeVpamg4ePGi6z+HDh/Xaa69p9uzZ2rNnjz7++GPde++9amtrU0lJiek+paWlWr58uZWmAV0iYOgFCE+rYd6R3HlJzJiGzv+Q+DuHz16+s60BevY8lAc7aWuKU4dCGzV2cjSdC9vMzEyP8pKSEi1btsyjLBK5GMoxfYn40/UOh0NDhw7VM888o5iYGE2YMEHHjx/Xj3/8Y58hv2jRIhUXF7t+bmho6HSxga4WTsD7C3ezIPfHX32zXwCc5/Yb9gQ9YFlNTY3HcH1CQnjr1DuFkouhshTyqampiomJUV1dnUd5XV2d0tPTTfcZNmyY4uLiFBMT4yq78sorVVtbq9bWVsXHd77HnpCQ0GUXEwhGqAHvK9ytBHtb//N/DeMa2/3WdT+ud+D7DXuCHrAsOTk54D35SORiKMf0xdI9+fj4eE2YMEHl5eWuMofDofLyck2ePNl0nylTpujjjz+Ww+FwlX344YcaNmyYacAD3c1vwPu4v212f7wjOcG1eWvrH+tzC6Wev/P5vHcfxL167tED1kQiF0M5pi+WX6ErLi7Wpk2b9Itf/EIHDhzQD37wAzU2NqqgoECSNHfuXC1atMhV/wc/+IE+//xz3X///frwww+1e/durVq1SoWFhVZPDXS5gAHvxVeA+uq5+wpob+39YwLW8Xcss/P7DXs/CHrAmkjkYqBjBsvyPflZs2bp5MmTWrp0qWpra5Wdna2ysjLXAwLHjh1TdPT53x0yMzP10ksvacGCBbr66qs1fPhw3X///XrkkUesnhroUqEEvDdfvXYzgYLc3+exjeffRvE1vO9si9kwfqch/ADD9wzdA8GLRC4GOmawLL8n3xN4Tx5dLRIBbxbuwfTQrXIPfCfve/lmD+iZ3qsPcJ+eoEdX6M735C95Zqmik8J7T97R1Kxjd6+IaHu7C3PXo88JN+BDCfe2/lEWWmgurtHodGxn4Dvb4Ax7s159KD16ABc2Qh59SiQD3qzX7ivcgw19Z7C77+Ne5jyne9h7D+GHE/QM2wMXNkIefUZXBrx379094L0DPFCgt7udJtYrf733jWs0PMrce/f+evUEPdA3EfJAGAEfbLi3+54rx4O/erFNnXvz7j/769WHG/QALkyEPPoEK6+FhRvwgcK9vV/QTVHsWbf9/nUsX2Hvq1dvKeh9oDcPXJhCWk8euJBYGaa3GvBt/aNcgev+5/Ykz4Bv73d+c+es5755fG6yn3s9n+f3+AXkfLvNJs7x4Ocdet6fBy48hDxsLdIBf/5z/+Hu/pmvQHft46NeoLDv1Jb+MW5tJeiBvojhevRNEQp4Vz2vMPY4dn+HghXTGO1xjNim88d2DuW3J3Uewjcbvrc0dM/9eVygjKZYGWFGm9Fkn2i0zzcBukg4Ae8r3M2C3ZHkP+yjm6I99otpjPYZ9s6gd7bJO+jdv4+voAdgP4Q8bMvn0HKAXrxTqAFvFu6+Aj0qqfOqc85ehK99zMI+UND7ehjPXbC9eR7CAy4chDzgxmwu+mAD3lfP3RnWZoEuSfH92lx/bj0b5zP4HUkOj969e9hL1oLe/fvSmwfsi5BH3+KnF282TB9OwJuFu3ugmzH73D34Hf/6K+sr7IMNel/D9vTmAXsh5GFL4TwF7m9pWH8B76/37h3eA5IC957PNCV47OsMe2evXjof9v6C3vd3MR+2B2AfhDz6jiB78U7evfhgAt5XuJuF+uCks53KnD5v6tdpH2d/uvVf//UewvcV9FaH7XnSHrAPQh62Y7UXH8wwfVcEvFmopyY2dio71dy/U11n6J9pSlB8v7ZOvfpQgz6cp+0Zsgd6PybDQd8Q5BP1wTCbltZKwKcmNro2p6EJp/1+7tzfeTzn8V336v91fvdbBmZP+wfLygQ5AHovevLAv3jfi/fVi3f/s3uo+gt4sx67e7B7/3yi5dxohPd+wfTone0zW9EuUG8egL0Q8rCVcIbqnczWhXd95taLdx+mDzbgvYNdktIT6l1/rm1J6VTPO/D9Bb2zXa6Z8kzeofeHIXvAXgh52F8YQ83+evGS54Q1/gLeO9zdg91XuXfgn2gZ6Dfone0JpjcfEh7AAy44hDz6HLP78f7ei3cy68VL54bpgwl472C/OO5zn238e9vgToE/NOG0/6CXgu7NBzNkb2UpWgC9Ew/ewTa6YqjeXTC9ePeJboIJ+IvjPndtTsPj/unanLzrOfd3Hi81sbHTw3jSuV843EcXzB66M/sFxl2g6+KNlemA3ouePPo8Xw/cufPuxXsP0wcKeO9eu3ug+yo/3naRJM8ev98eveQxbC8pqN68+3XgATzAXgh52JuFV+fMHrjz1YuXOs9Fb/YevNm9d++AHx5z/n798Y6BHvWcQe9+POe9evfzft50/rcQ94fwnO3299Cd2eQ4TkyMgwtNdHO0osMdpG62zyC3fb4J0IuYPUUv+Q9405+9hvADHT/Q3PgA+hZCHrDAe11471A1ex/eGc6BAt5Xua+hfV/nc2c2OQ6AvoOQB0y435cPFI6+FptxH6oPFPDDYmL9fu6+v6/X77zb4Ws9einww3cA7IGQBywyC0/3+/HuQ+lmr8n5CvhAQe99PPfz+FvsBkDfRcgDQQj00J0UeOhcCtyDDybogzmvd/uszl9v9TU6AL0TIQ9bCPUdeX9rxwcKRn8PuZkN1QcKeF/lzv2CGbIP1C7J//fydz384V15oHci5IEe4Cvgg/28q/mbrx/AhYuQB3rAZx3+J50J9HlX8/WePIALGyEPW7C6EppzpTV/M7wFWrXNObuc06nm/q4/u09Y45zQxn2iG8l3kHuXO/dznxjHe0Icf+3y5u97hTrjHSvRAb0TIQ8EwTsY3WeUsyJQ0PsKeH/cf7lw8m5fMMvMurOy3CyA3ouQByxyLuXqzn1aWef679K51eS8+Qr6YALe/Xju53E/v792AoiMDRs2KCsrS4mJicrNzdX+/fuD2m/r1q2KiorSjBkzPMrPnDmjoqIiXXzxxUpKStKYMWO0ceNGy+3iXwHAhPvCLbEBXkE/02T+upnZkL3r5wA9eu/Pgxmq99UOMyGvKQ+gk23btqm4uFglJSWqqqrSuHHjlJ+frxMnTvjd7+jRo3rwwQd1/fXXd/qsuLhYZWVl+vWvf60DBw7ogQceUFFRkXbt2mWpbYQ8YIFzVTcnf/flnZy970BB76vce79A53Pn3t5Av6wACM2aNWs0f/58FRQUuHrc/fr105YtW3zu09HRodmzZ2v58uUaOXJkp8/feustzZs3T1OnTlVWVpbuvvtujRs3LugRAidCHogA96F0d4GC3l/A+xqqdxfooTsAwWtoaPDYWlo6P6vS2tqqyspK5eXlucqio6OVl5eniooKn8desWKFhg4dqrvuusv082uvvVa7du3S8ePHZRiGXn/9dX344Ye6+eabLX0HQh725rUsqseyqV7MXiNzf2At0MN3ZvfFzYbWzYLeufmr5+t43uf199Cd2QN4/l6f63S9WGYWvVxMY3SXbJKUmZmplJQU11ZaWtrpfKdOnVJHR4fS0tI8ytPS0lRbW2vaxn379mnz5s3atGmTz++xbt06jRkzRhdffLHi4+P19a9/XRs2bNANN9xg6Xqwnjz6vLjGdo+Z3uIajU4LuMSePb9QjfMfAOcc9q1n43RG5xaIcQZuamKjTrQM1NCE052C+eK4zz0C3H0mO+9gd++9O4/j7MWfau7vOt+ZpgSPXnx0U3TAoXrv+/Khvj4H2FVNTY2Sk5NdPyckhD/d8+nTpzVnzhxt2rRJqampPuutW7dOb7/9tnbt2qURI0bozTffVGFhoTIyMjxGDQIh5GEbjvrTlqZXjWlo8TtHuzPsY5vOTwXr/ufopmg5FOuaJ/5MU4LfoHdOResMbudiM2Y9du+n8gMFvJPRFOvxVL1Zzz3QQ3dWX5/jHXnYVXJyskfIm0lNTVVMTIzq6uo8yuvq6pSent6p/ieffKKjR49q+vTprjKH41yHITY2VocOHVJGRoYeffRR7dixQ9OmTZMkXX311aqurtaTTz5JyAP+RJ9pkmOA5wTuzt58bGOH2vvHBNWbd67VbjTFqlXn5owPFPRO6Qn1pq/XufMeAfAX8K1n4zyG6c168e6B7wx651C9WS/e360NAOfEx8drwoQJKi8vd70G53A4VF5erqKiok71R48erffee8+j7LHHHtPp06f105/+VJmZmWpublZbW5uioz3vqMfExLh+IQgWIQ/7O31GGjggpF2D6s27Ddv7C3rp/PKw3oHvZHbP3f0hO18B7xRuL94v7scDpoqLizVv3jzl5ORo0qRJWrt2rRobG1VQUCBJmjt3roYPH67S0lIlJibqqquu8th/0KBBkuQqj4+P14033qiHHnpISUlJGjFihN544w398pe/1Jo1ayy1jZCHrXTFkL2zN2/GrDfvPmzvL+glz8A2C3wns6fn/QW8+zB9oF68+/f0xlA9YN2sWbN08uRJLV26VLW1tcrOzlZZWZnrYbxjx4516pUHsnXrVi1atEizZ8/W559/rhEjRujxxx/XPffcY+k4UYZh9PpZMRoaGpSSkqKvJMxUbFR8TzcHvZxpyHv15N2H691D3vkAnjPknUP2zv+2J50PeWdvvqO/w9Wbd96fdy73OiDpXGgOTjr/5Fsw6847ub8H7y/gJc+H7Zyh7h7yzl68v6F695AP5sl6Qh7BaDda9VrLC6qvrw94jztUzpwYuXSVohMTwzqWo7lZh1c8GtH2dhdeoUPfYOFVukDMnlR39qKdgesMYGcgu7/mdqq5v8fmzeyzYALeVzutzlsv8eocYBcM18N2whmy9/UAnve9eWeAtid5DdsnOWQ0+R+6d3L27s2C3uyd+0AB796L9w54q734YNCLB3o/Qh59h9cDeO5P2Xdl0EuS419/tVr/dS7vfrBZ6HvznoveX+89lIB3Z3WYHsCFgZCHLVntzQcrUNBL8tmrd3L27oPhPU2tr9675D/gzb8Lk98Adsc9efQtfu7Nu/dmnQF4fljbcPvsXz1it4fb3O97O4M3uila0U3RMppiPd5hbz0b57G58/WZ8xjOY7qfx9kWfwHflQ/bSQzVAxcKevKAG3+v1LlPkOOrRy/579V78+7lS53nnnfyFe6StYD3/r4A7IuQh235HLL3c2/enfuc9sEGfXu/80HrHvam7fMR/JL50/Jm4S5ZD3hfw/T04gH7IeQBBX4IT7LWo/cX9s7pcM2C3Jv3Lwlm4e5ebiXg6cUD9kfIo28KMNVtqEEv+Q97yX/v3pdA4e5sh/efgw14nqiHXcQ0SzFhTvHWYaPff3nwDrbmd2g5wAQ5Zg/iSf4fxpP+9RCc+0N5Zz3L3TczvuqZHcvs/JEMeIbqgQsLPXnYnt/X6QLcnw+mRy/J9R6988+STHv2Ts6pcf3NRmc2s553fZ+/aBDwAETIo4+w8t681aA/V+45fC+Zh72TWYD74y/cvX8OOeD9IOCBCxMhD5jcn7cS9JI69erP1TMPe28e4R8gd4MNd2c73dvvzjTguQ8P2A4hjz7DyrC9FDjoJfns1Tu5D+O7cwa/5DvYfc1WZ1bu7xW5cAOeXjxw4SLk0ad0ZdBLnmHvHrTege8e6u7lVnjvYza5DQEPwB0hjz6nK4JeUqewd/bqJc9h/HOfh/lOj5tQwl0i4IG+iFfo0CdZebVOMg9I7yCNa2zvFLaxjR2uLRy+jmN2TgIegBM9efRZofToJQXVq3cy692Hy9e0tL5msCPggb6Lnjz6NKs9eim4Xr2TWU87VP6O5av3TsADfVtIIb9hwwZlZWUpMTFRubm52r9/f1D7bd26VVFRUZoxY0YopwUiImDQ+xi+N5shz7l5cwa0ry2Yet58nc9vuBPwQJ9iOeS3bdum4uJilZSUqKqqSuPGjVN+fr5OnDjhd7+jR4/qwQcf1PXXXx9yY4FICRhwfnr1vnr2vgLfTLA9fn/H9RnuUsB34Al4wJ4sh/yaNWs0f/58FRQUaMyYMdq4caP69eunLVu2+Nyno6NDs2fP1vLlyzVy5MiA52hpaVFDQ4PHBkRaqEEv+Q9Y92A220KtG8y5A7VbIuABO7MU8q2traqsrFReXt75A0RHKy8vTxUVFT73W7FihYYOHaq77rorqPOUlpYqJSXFtWVmZlppJhCyoII+iLAPGLxurPb6gz5HgLZKBDzsx7mQU7ibXVgK+VOnTqmjo0NpaWke5WlpaaqtrTXdZ9++fdq8ebM2bdoU9HkWLVqk+vp611ZTU2OlmUBYggq+IAJUkuXA75LjBNk2Ah6wv4i+Qnf69GnNmTNHmzZtUmpqatD7JSQkKCEhIXBFIEKCXtDGGaZ+1qZ3Cjfog25LEAh4oG+wFPKpqamKiYlRXV2dR3ldXZ3S09M71f/kk0909OhRTZ8+3VXmcDjOnTg2VocOHdKoUaNCaTcQcc4g7Oqw73KEOwAfLA3Xx8fHa8KECSovL3eVORwOlZeXa/LkyZ3qjx49Wu+9956qq6td26233qqbbrpJ1dXV3GvHBcFSMDqHyiO9olsI5yHggb7H8nB9cXGx5s2bp5ycHE2aNElr165VY2OjCgoKJElz587V8OHDVVpaqsTERF111VUe+w8aNEiSOpUDvZmV9ehdvAM4nF5+mL80EPBA32Q55GfNmqWTJ09q6dKlqq2tVXZ2tsrKylwP4x07dkzR0UykB/txD0rLgS91+3rtBDuAkB68KyoqUlFRkelne/fu9bvvc889F8opgV4l7MCPEIIdgDsWqAHCZOkBvQi3AQDcEfJAF+nu3j3BDiAQQh6IgEgFPsEOwAqekAMizFF/2u8Wal0AvYeV1Vm3b9+unJwcDRo0SP3791d2drZ+9atfdap34MAB3XrrrUpJSVH//v01ceJEHTt2zFK7CHmghxHiwIXN6uqsgwcP1uLFi1VRUaF3331XBQUFKigo0EsvveSq88knn+i6667T6NGjtXfvXr377rtasmSJEhMTLbUtyjAMI6xv1w0aGhqUkpKiryTMVGxUfE83BwBgQbvRqtdaXlB9fb2Sk5Mjcg5nTlxZtEoxCdaC0FtHS7MOrH806Pbm5uZq4sSJWr9+vaRzk8RlZmbqvvvu08KFC4M65zXXXKNp06Zp5cqVkqTbb79dcXFxpj18K+jJAwBgwnvJ85aWzitFhro6q5NhGCovL9ehQ4d0ww03SDr3S8Lu3bt1+eWXKz8/X0OHDlVubq527txp+TsQ8gAA24htlmKbwtyazx0rMzPTY9nz0tLSTucLZXVWSaqvr9eAAQMUHx+vadOmad26dfra174mSTpx4oTOnDmj1atX6+tf/7pefvllfeMb39C//du/6Y033rB2PSzVBgCgj6ipqfEYru/K1VEHDhyo6upqnTlzRuXl5SouLtbIkSM1depU10Jut912mxYsWCBJys7O1ltvvaWNGzfqxhtvDPo8hDwAACaSk5MD3pO3ujqrU3R0tC677DJJ5wL8wIEDKi0t1dSpU5WamqrY2FiNGTPGY58rr7xS+/bts/QdGK4HACBEVldn9cXhcLju+cfHx2vixIk6dOiQR50PP/xQI0aMsNQ+evIAAITByuqsklRaWqqcnByNGjVKLS0t2rNnj371q1/p6aefdh3zoYce0qxZs3TDDTfopptuUllZmf7nf/4n4Pow3gh5AADCYHV11sbGRt177736+9//rqSkJI0ePVq//vWvNWvWLFedb3zjG9q4caNKS0v1wx/+UFdccYV+//vf67rrrrPUNt6TBwBEVHe+Jz/2+6sUEx/me/KtzXrv2eDfk+/NuCcPAIBNEfIAANgUIQ8AgE0R8gAA2BQhDwCATRHyAADYFCEPAIBNEfIAANgUM94BAGwjrtFQTFt4c7xFt/b6OeKCRk8eAACbIuQBALApQh4AAJsi5AEAsClCHgAAmyLkAQCwKUIeAACbIuQBALApQh4AAJsi5AEAsClCHgAAmyLkAQCwKUIeAACbIuQBALAplpoFANhG7FlDsWEuNRsV5v69CT15AABsipAHAMCmCHkAAGyKkAcAwKYIeQAAbIqQBwDApgh5AABsipAHAMCmCHkAAGyKkAcAwKYIeQAAwrRhwwZlZWUpMTFRubm52r9/v8+6mzZt0vXXX6+LLrpIF110kfLy8vzWv+eeexQVFaW1a9dabhchDwBAGLZt26bi4mKVlJSoqqpK48aNU35+vk6cOGFaf+/evbrjjjv0+uuvq6KiQpmZmbr55pt1/PjxTnV37Niht99+WxkZGSG1jZAHACAMa9as0fz581VQUKAxY8Zo48aN6tevn7Zs2WJa//nnn9e9996r7OxsjR49Ws8++6wcDofKy8s96h0/flz33Xefnn/+ecXFxYXUNkIeAAATDQ0NHltLS0unOq2traqsrFReXp6rLDo6Wnl5eaqoqAjqPGfPnlVbW5sGDx7sKnM4HJozZ44eeughffnLXw75OxDyAADbiG3s6JJNkjIzM5WSkuLaSktLO53v1KlT6ujoUFpamkd5Wlqaamtrg2rzI488ooyMDI9fFH70ox8pNjZWP/zhD8O4GqwnDwCAqZqaGiUnJ7t+TkhI6PJzrF69Wlu3btXevXuVmJgoSaqsrNRPf/pTVVVVKSoqKqzj05MHAMBEcnKyx2YW8qmpqYqJiVFdXZ1HeV1dndLT0/0e/8knn9Tq1av18ssv6+qrr3aV//GPf9SJEyd0ySWXKDY2VrGxsfr000/1H//xH8rKyrL0HQh5AABCFB8frwkTJng8NOd8iG7y5Mk+93viiSe0cuVKlZWVKScnx+OzOXPm6N1331V1dbVry8jI0EMPPaSXXnrJUvsYrgcAIAzFxcWaN2+ecnJyNGnSJK1du1aNjY0qKCiQJM2dO1fDhw933dP/0Y9+pKVLl+o3v/mNsrKyXPfuBwwYoAEDBmjIkCEaMmSIxzni4uKUnp6uK664wlLbCHkAAMIwa9YsnTx5UkuXLlVtba2ys7NVVlbmehjv2LFjio4+P3D+9NNPq7W1Vd/61rc8jlNSUqJly5Z1adsIeQAAwlRUVKSioiLTz/bu3evx89GjRy0fP5R9JO7JAwBgW4Q8AAA2RcgDAGBThDwAADYVUshHckk9AADQNSyHfCSX1AMAAF3HcshHakk9dy0tLZ1W/wEAANZYCvlILannrbS01GPln8zMTCvNBAD0UXFn2xXXGOZ2tr2nv0aXsRTykVpSz9uiRYtUX1/v2mpqaqw0EwAAqJtnvDNbUs9MQkJCRJb0AwCgL7EU8l2xpN6rr77qsaQeAACIDEvD9ZFYUg8AAESG5eH6rl5SDwAARIblkO/NS+oBAIDzQnrwLtJL6gEAgPAxdz0AADZFyAMAYFOEPAAANkXIAwBgU4Q8AAA2RcgDAGBThDwAADbVrQvUAAAQSTGnWxUTExXWMYyO1i5qTc+jJw8AgE0R8gAA2BQhDwCATRHyAADYFCEPAIBNEfIAANgUIQ8AgE0R8gAA2BQhDwCATRHyAADYFCEPAECYNmzYoKysLCUmJio3N1f79+/3Wfdvf/ubvvnNbyorK0tRUVFau3ZtpzqlpaWaOHGiBg4cqKFDh2rGjBk6dOiQ5XYR8gAAhGHbtm0qLi5WSUmJqqqqNG7cOOXn5+vEiROm9c+ePauRI0dq9erVSk9PN63zxhtvqLCwUG+//bZeeeUVtbW16eabb1ZjY6OltrFADQAAYVizZo3mz5+vgoICSdLGjRu1e/dubdmyRQsXLuxUf+LEiZo4caIkmX4uSWVlZR4/P/fccxo6dKgqKyt1ww03BN02evIAAJhoaGjw2FpaWjrVaW1tVWVlpfLy8lxl0dHRysvLU0VFRZe1pb6+XpI0ePBgS/vRkwcA2EZ0Y5Oiox3hHcNxLswzMzM9yktKSrRs2TKPslOnTqmjo0NpaWke5WlpaTp48GBY7XByOBx64IEHNGXKFF111VWW9iXkAQAwUVNTo+TkZNfPCQkJPdKOwsJCvf/++9q3b5/lfQl5AABMJCcne4S8mdTUVMXExKiurs6jvK6uzudDdVYUFRXpxRdf1JtvvqmLL77Y8v7ckwcAIETx8fGaMGGCysvLXWUOh0Pl5eWaPHlyyMc1DENFRUXasWOHXnvtNV166aUhHYeePAAAYSguLta8efOUk5OjSZMmae3atWpsbHQ9bT937lwNHz5cpaWlks49rPfBBx+4/nz8+HFVV1drwIABuuyyyySdG6L/zW9+oz/84Q8aOHCgamtrJUkpKSlKSkoKum2EPAAAYZg1a5ZOnjyppUuXqra2VtnZ2SorK3M9jHfs2DFFR58fOP/HP/6h8ePHu35+8skn9eSTT+rGG2/U3r17JUlPP/20JGnq1Kke5/r5z3+u7373u0G3jZAHACBMRUVFKioqMv3MGdxOWVlZMgzD7/ECfR4s7skDAGBThDwAADZFyAMAYFOEPAAANkXIAwBgU4Q8AAA2RcgDAGBThDwAADbFZDgAAPs43ShFt4V3DEdr17SlF6AnDwCATRHyAADYFCEPAIBNEfIAANgUIQ8AgE0R8gAA2BQhDwCATRHyAADYFCEPAIBNEfIAANgUIQ8AgE0R8gAA2BQhDwCATbEKHQDANhwNp+WIig/vGAar0AEAgF6OkAcAwKYIeQAAbIqQBwDApgh5AABsipAHAMCmCHkAAGyKkAcAwKYIeQAAbIqQBwDApkIK+Q0bNigrK0uJiYnKzc3V/v37/db/7W9/q9GjRysxMVFjx47Vnj17QmosAAC9UVfnomEYWrp0qYYNG6akpCTl5eXpo48+stwuyyG/bds2FRcXq6SkRFVVVRo3bpzy8/N14sQJ0/pvvfWW7rjjDt1111165513NGPGDM2YMUPvv/++5cYCANDbRCIXn3jiCf3Xf/2XNm7cqD//+c/q37+/8vPz1dzcbKltUYZhGFZ2yM3N1cSJE7V+/XpJksPhUGZmpu677z4tXLiwU/1Zs2apsbFRL774oqvs//2//6fs7Gxt3LgxqHM2NDQoJSVFX0mYqdgwFx4AAHSvdqNVr7W8oPr6eiUnJ0fkHF2ZE1bb29W5aBiGMjIy9B//8R968MEHJUn19fVKS0vTc889p9tvvz3o72JpFbrW1lZVVlZq0aJFrrLo6Gjl5eWpoqLCdJ+KigoVFxd7lOXn52vnzp0+z9PS0qKWlhbXz/X19ZKkdqPNSnMBAL2A899ui33K0M6lNinM07TrXHsbGho8yhMSEpSQkOBRFolcPHLkiGpra5WXl+f6PCUlRbm5uaqoqIhcyJ86dUodHR1KS0vzKE9LS9PBgwdN96mtrTWtX1tb6/M8paWlWr58eafyN1t3WGkuAKAX+b//+z+lpKRE5Njx8fFKT0/Xm7VdkxMDBgxQZmamR1lJSYmWLVvmURaJXHT+12p2mumV68kvWrTI47ecL774QiNGjNCxY8ci9j+IHTQ0NCgzM1M1NTURGxKzA65TYFyj4HCdglNfX69LLrlEgwcPjtg5EhMTdeTIEbW2ds1a8IZhKCoqyqPMuxd/IbAU8qmpqYqJiVFdXZ1HeV1dndLT0033SU9Pt1RfMh8Skc4NV/AXKbDk5GSuUxC4ToFxjYLDdQpOdHRk39pOTExUYmJiRM/hLRK56PxvXV2dhg0b5lEnOzvbUvssXfH4+HhNmDBB5eXlrjKHw6Hy8nJNnjzZdJ/Jkyd71JekV155xWd9AAAuFJHIxUsvvVTp6ekedRoaGvTnP//ZenYaFm3dutVISEgwnnvuOeODDz4w7r77bmPQoEFGbW2tYRiGMWfOHGPhwoWu+n/605+M2NhY48knnzQOHDhglJSUGHFxccZ7770X9Dnr6+sNSUZ9fb3V5vYpXKfgcJ0C4xoFh+sUHLtfp0jk4urVq41BgwYZf/jDH4x3333XuO2224xLL73UaGpqstQ2yyFvGIaxbt0645JLLjHi4+ONSZMmGW+//bbrsxtvvNGYN2+eR/0XXnjBuPzyy434+Hjjy1/+srF7925L52tubjZKSkqM5ubmUJrbZ3CdgsN1CoxrFByuU3D6wnXq6lx0OBzGkiVLjLS0NCMhIcH46le/ahw6dMhyuyy/Jw8AAC4MzF0PAIBNEfIAANgUIQ8AgE0R8gAA2BQhDwCATfWakGeN+uBYuU6bNm3S9ddfr4suukgXXXSR8vLyAl5XO7D6/5LT1q1bFRUVpRkzZkS2gb2E1ev0xRdfqLCwUMOGDVNCQoIuv/zyPvH3zup1Wrt2ra644golJSUpMzNTCxYssLw86IXmzTff1PTp05WRkaGoqCi/C5A57d27V9dcc40SEhJ02WWX6bnnnot4O/ukEF4H7HJbt2414uPjjS1bthh/+9vfjPnz5xuDBg0y6urqTOv/6U9/MmJiYownnnjC+OCDD4zHHnvM8gQ7FyKr1+nOO+80NmzYYLzzzjvGgQMHjO9+97tGSkqK8fe//72bW959rF4jpyNHjhjDhw83rr/+euO2227rnsb2IKvXqaWlxcjJyTFuueUWY9++fcaRI0eMvXv3GtXV1d3c8u5l9To9//zzRkJCgvH8888bR44cMV566SVj2LBhxoIFC7q55d1rz549xuLFi43t27cbkowdO3b4rX/48GGjX79+RnFxsfHBBx8Y69atM2JiYoyysrLuaXAf0itCftKkSUZhYaHr546ODiMjI8MoLS01rT9z5kxj2rRpHmW5ubnGv//7v0e0nT3N6nXy1t7ebgwcOND4xS9+Eakm9rhQrlF7e7tx7bXXGs8++6wxb968PhHyVq/T008/bYwcOdJobW3trib2ClavU2FhofGVr3zFo6y4uNiYMmVKRNvZmwQT8g8//LDx5S9/2aNs1qxZRn5+fgRb1jf1+HC9cy1e93Vzg1mL172+dG4tXl/17SCU6+Tt7Nmzamtri+hKUD0p1Gu0YsUKDR06VHfddVd3NLPHhXKddu3apcmTJ6uwsFBpaWm66qqrtGrVKnV0dHRXs7tdKNfp2muvVWVlpWtI//Dhw9qzZ49uueWWbmnzhaIv/hveU3p8qdnuWqP+QhfKdfL2yCOPKCMjo9NfLrsI5Rrt27dPmzdvVnV1dTe0sHcI5TodPnxYr732mmbPnq09e/bo448/1r333qu2tjaVlJR0R7O7XSjX6c4779SpU6d03XXXyTAMtbe365577tGjjz7aHU2+YPj6N7yhoUFNTU1KSkrqoZbZT4/35NE9Vq9era1bt2rHjh3dvhRjb3X69GnNmTNHmzZtUmpqak83p1dzOBwaOnSonnnmGU2YMEGzZs3S4sWLtXHjxp5uWq+yd+9erVq1Sk899ZSqqqq0fft27d69WytXruzppqGP6vGefHetUX+hC+U6OT355JNavXq1Xn31VV199dWRbGaPsnqNPvnkEx09elTTp093lTkcDklSbGysDh06pFGjRkW20T0glP+Xhg0bpri4OMXExLjKrrzyStXW1qq1tVXx8fERbXNPCOU6LVmyRHPmzNH3v/99SdLYsWPV2Niou+++W4sXL474euoXCl//hicnJ9OL72I9/n8ca9QHJ5TrJElPPPGEVq5cqbKyMuXk5HRHU3uM1Ws0evRovffee6qurnZtt956q2666SZVV1crMzOzO5vfbUL5f2nKlCn6+OOPXb8ESdKHH36oYcOG2TLgpdCu09mzZzsFufMXI4O1wFz64r/hPaann/wzjJ5Zo/5CZPU6rV692oiPjzd+97vfGZ999plrO336dE99hYizeo289ZWn661ep2PHjhkDBw40ioqKjEOHDhkvvviiMXToUOM///M/e+ordAur16mkpMQYOHCg8d///d/G4cOHjZdfftkYNWqUMXPmzJ76Ct3i9OnTxjvvvGO88847hiRjzZo1xjvvvGN8+umnhmEYxsKFC405c+a46jtfoXvooYeMAwcOGBs2bOAVugjpFSFvGN2/Rv2Fysp1GjFihCGp01ZSUtL9De9GVv9fctdXQt4wrF+nt956y8jNzTUSEhKMkSNHGo8//rjR3t7eza3uflauU1tbm7Fs2TJj1KhRRmJiopGZmWnce++9xj//+c/ub3g3ev31103/rXFem3nz5hk33nhjp32ys7ON+Ph4Y+TIkcbPf/7zbm93X8B68gAA2FSP35MHAACRQcgDAGBThDwAADZFyAMAYFOEPAAANkXIAwBgU4Q8AAA2RcgDAGBThDwAADZFyAMAYFOEPAAANvX/AZSOiwbOtYxRAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgEAAAGxCAYAAAD7xGWJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABOHklEQVR4nO3de3wU5b0/8M/uJrsh5MIlkluDAbQiQhJMJA1qvW2N4otKiy8jUkIjglawlD1ViEKCUklA5MQKkopytD1wAD1qrXDCD6LRUiLUhIgi4AUwEckCRbKQkN0kO78/7C67m9ndmb0n83m/XvvSTJ6deXZFns98n2dmVIIgCCAiIiLFUYe7A0RERBQeDAFEREQKxRBARESkUAwBRERECsUQQEREpFAMAURERArFEEBERKRQDAFEREQKxRBARESkUAwBRERECsUQQBREJ06cQExMDB544AGn7bt27UJ0dDQWLFgQpp4REQEqPjuAKLjmzZuHl156CV9++SUuv/xyHD58GAUFBbjxxhvx9ttvQ61mFiei8GAIIAqyEydOYNSoUXjggQewbNky5OfnIz4+Hrt378bAgQPD3T0iUrCocHeAqL9LT0/H7NmzsX79ejQ2NuLixYv44IMPGACIKOxYhyQKgd///vcwm804cOAA3nnnHaSnpzv9/vjx41CpVLj33nudto8dOxbvvvtuKLtKRArCEEAUAs888wwAoLu7G0OGDOn1+08++QQjR47Ezp070dXVBQAwm804cuQIcnJyQtlVIlIQhgCiIHv22Wfx8ssvY82aNYiKirIHAkeffPIJbr75Zlx99dX44IMPAAAHDx5EQkICfvSjH4W6y0SkEAwBREH09ttvY9GiRVi2bBnmzp2LOXPm4M9//jOOHTvm1O6TTz5BdnY2Jk+ejL/97W9O24iIgoUhgChIGhoaMH36dEyfPh1PPvkkAODxxx+HWq3uVQ345JNPkJWV5TYE7NixA0ajMbQfgIj6PYYAoiD49ttvMXnyZIwfPx7r16+3b09LS8MDDzzgVA24cOECjh49iqysLIwdOxYA8NlnnzmFgJdffhk6nS70H4SI+jXeJ4AozPbs2YOioiK0tLQAAObPn4/k5GSsWrUKtbW1OH78OGbNmoWsrCwsXLgQd955Z5h7TET9Be8TQBRmtqkAm8mTJ+PBBx/E+fPnMWbMGIwbNw4TJ07kpYJEFHCcDiAKM9cQcNNNN+H777/H6NGjodPpcPjwYVx99dVh7CER9VecDiCKcFu3boXJZMKDDz4Y7q4QUT/DSgBRhLvqqqtQVVWF8vLycHeFiPoZ2SHgww8/xOTJk5GWlgaVSoW3337b63vq6upw7bXXQqfT4YorrsCrr77qQ1eJlCk7OxufffYZnnrqqXB3hYj6GdkhoL29HdnZ2Vi7dq2k9seOHcNdd92FW265BU1NTfjd736HBx98EDt27JDdWSIiIgocv9YEqFQqvPXWW5gyZYrbNgsXLsS2bdvw2Wef2bfdd999OHfuHGpqanw9NBEREfkp6JcI1tfXQ6/XO20rLCzE7373O7fvMZvNMJvN9p+tVivOnj2LoUOHQqVSBaurREQUBIIg4Pz580hLS4NaHbylaJ2dnbBYLAHZl1arRUxMTED2FcmCHgJaW1uRnJzstC05ORkmkwkXL17EgAEDer2noqKC859ERP1MS0tL0B6I1dnZiczLB8J4yhqQ/aWkpODYsWP9PghE5M2CSktLYTAY7D+3tbVh+PDh+Kn2F4hSRYexZ0REJFe30IUPLW8hPj4+aMewWCwwnrLi83+mIj7ev2rD+fNWjLnuJCwWC0OAv1JSUno9+MRoNCIhIUG0CgAAOp1O9D7pUapoRKm0QeknEREFVyimc+Pj1UjwMwQoSdC/qYKCAtTW1jpt27lzJwoKCoJ9aCIiIvJAdgi4cOECmpqa0NTUBOCHSwCbmprQ3NwM4IdSfnFxsb39ww8/jKNHj+Lxxx/H4cOH8eKLL2Lr1q1YsGBBYD4BERER+UR2CPj4448xfvx4jB8/HgBgMBgwfvx4lJWVAQBOnjxpDwQAMGLECGzbtg07d+5EdnY2nnvuObz88ssoLCwM0EcgIiIiX8heE3DzzTfD060FxO4GePPNN2P//v1yD0VERERBxNUTRERECsUQQEREpFAMAURERArFEEBERKRQDAFEREQKxRBARESkUAwBRERECsUQQEREpFAMAURERAoVkY8SJiIi8sV3PT0w9bi/q60UF3qsAepN5GMlgIiISKEYAoiIiBSKIYCIiEihGAKIiIgUiiGAiIhIoRgCiIiIFIohgIiISKEYAoiIiBSKIYCIiEihGAKIiIgUiiGAiIhIoRgCiIiI/LR27VpkZmYiJiYG+fn52Ldvn9u2b775JvLy8jBo0CAMHDgQOTk5+Mtf/tKrze23346hQ4dCpVKhqamp135uvvlmqFQqp9fDDz8sq98MAURERH7YsmULDAYDysvL0djYiOzsbBQWFuLUqVOi7YcMGYInn3wS9fX1OHDgAEpKSlBSUoIdO3bY27S3t+OGG27AihUrPB579uzZOHnypP21cuVKWX3nUwSJiIj8sHr1asyePRslJSUAgOrqamzbtg0bNmzAokWLerW/+eabnX6eP38+XnvtNezevRuFhYUAgBkzZgAAjh8/7vHYsbGxSElJ8bnvrAQQERGJMJlMTi+z2dyrjcViQUNDA/R6vX2bWq2GXq9HfX2912MIgoDa2locOXIEP/3pT2X3cePGjUhKSsLYsWNRWlqKjo4OWe9nJYCIiPqNb7sTMLBb49c+2rt7ALQiIyPDaXt5eTmWLl3qtO3MmTPo6elBcnKy0/bk5GQcPnzY7THa2tqQnp4Os9kMjUaDF198ET/72c9k9fP+++/H5ZdfjrS0NBw4cAALFy7EkSNH8Oabb0reB0MAERGRiJaWFiQkJNh/1ul0Adt3fHw8mpqacOHCBdTW1sJgMGDkyJG9pgo8mTNnjv3fx40bh9TUVNx22234+uuvMWrUKEn7YAggIiISkZCQ4BQCxCQlJUGj0cBoNDptNxqNHufq1Wo1rrjiCgBATk4ODh06hIqKClkhwFV+fj4A4KuvvpIcArgmgIiIyEdarRa5ubmora21b7NaraitrUVBQYHk/VitVtE1B3LYLiNMTU2V/B5WAoiIiPxgMBgwc+ZM5OXlYcKECaiqqkJ7e7v9aoHi4mKkp6ejoqICAFBRUYG8vDyMGjUKZrMZ27dvx1/+8hesW7fOvs+zZ8+iubkZ3333HQDgyJEjAICUlBSkpKTg66+/xqZNmzBp0iQMHToUBw4cwIIFC/DTn/4UWVlZkvvOEEBEROSHoqIinD59GmVlZWhtbUVOTg5qamrsiwWbm5uhVl8qvLe3t+ORRx7Bt99+iwEDBmD06NH47//+bxQVFdnbvPPOO/YQAQD33XcfgEuLE7VaLXbt2mUPHBkZGZg6dSoWL14sq+8qQRAEfz58KJhMJiQmJuJW3b2IUmnD3R0iIpKhW7DgPfNWtLW1eZ1j95VtnPjfT36MgfF+Xh1wvgdTs78Ian8jBdcEEBERKRRDABERkUIxBBARESkUQwAREZFCMQQQEREpFEMAERGRQjEEEBERKRRDABERkUIxBBARESkUbxtMRET9xonuIRjQ5d/QdrG7O0C9iXysBBARESkUQwAREZFCMQQQEREpFEMAERGRQjEEEBERKRRDABERkUIxBBARESkUQwAREZFCMQQQEREpFEMAERGRQjEEEBERKRRDABERkUIxBBARESkUQwAREZFCMQQQEREplH8PXSYiIoogLeah0EVH+7UPs7krQL2JfKwEEBERKRRDABERkUIxBBARESkUQwAREZFCMQQQEREplE8hYO3atcjMzERMTAzy8/Oxb98+j+2rqqpw1VVXYcCAAcjIyMCCBQvQ2dnpU4eJiIgoMGSHgC1btsBgMKC8vByNjY3Izs5GYWEhTp06Jdp+06ZNWLRoEcrLy3Ho0CG88sor2LJlC5544gm/O09ERES+kx0CVq9ejdmzZ6OkpARjxoxBdXU1YmNjsWHDBtH2e/bswfXXX4/7778fmZmZuP322zFt2jSv1QMiIiIKLlkhwGKxoKGhAXq9/tIO1Gro9XrU19eLvmfixIloaGiwD/pHjx7F9u3bMWnSJLfHMZvNMJlMTi8iIiIKLFl3DDxz5gx6enqQnJzstD05ORmHDx8Wfc/999+PM2fO4IYbboAgCOju7sbDDz/scTqgoqICTz31lJyuERERkUxBvzqgrq4Oy5cvx4svvojGxka8+eab2LZtG5YtW+b2PaWlpWhra7O/Wlpagt1NIiIixZEVApKSkqDRaGA0Gp22G41GpKSkiL5nyZIlmDFjBh588EGMGzcOv/jFL7B8+XJUVFTAarWKvken0yEhIcHpRUREFKnkXDW3fv163HjjjRg8eDAGDx4MvV7fq71KpRJ9Pfvss/Y2Z8+exfTp05GQkIBBgwZh1qxZuHDhgqx+ywoBWq0Wubm5qK2ttW+zWq2ora1FQUGB6Hs6OjqgVjsfRqPRAAAEQZDVWSIiokgj96q5uro6TJs2De+//z7q6+uRkZGB22+/HSdOnLC3OXnypNNrw4YNUKlUmDp1qr3N9OnTcfDgQezcuRPvvvsuPvzwQ8yZM0dW31WCzJF4y5YtmDlzJv70pz9hwoQJqKqqwtatW3H48GEkJyejuLgY6enpqKioAAAsXboUq1evxksvvYT8/Hx89dVX+M1vfoPc3Fxs2bJF0jFNJhMSExNxq+5eRKm0sj4gERGFV7dgwXvmrWhrawtaZdc2Tjy25y7o4vx8iuCFLjw7cZvk/ubn5+O6667DmjVrAPxwcpyRkYFHH30UixYt8vr+np4eDB48GGvWrEFxcbFomylTpuD8+fP2k/BDhw5hzJgx+Oc//4m8vDwAQE1NDSZNmoRvv/0WaWlpkj6r7EcJFxUV4fTp0ygrK0NraytycnJQU1NjXyzY3NzsdOa/ePFiqFQqLF68GCdOnMBll12GyZMn45lnnpF7aCIiopBxvTJNp9NBp9M5bbNdNVdaWmrf5u2qOVcdHR3o6urCkCFDRH9vNBqxbds2vPbaa/Zt9fX1GDRokD0AAIBer4darcbevXvxi1/8QtKxZYcAAJg3bx7mzZsn+ru6ujrnA0RFoby8HOXl5b4cioiISLJWcwK00f5VjC1mCwAgIyPDaXt5eTmWLl3qtM2Xq+ZcLVy4EGlpaU6X3zt67bXXEB8fj1/+8pf2ba2trRg2bJhTu6ioKAwZMgStra2Sjgv4GAKIiIj6u5aWFqfpANcqQCBUVlZi8+bNqKurQ0xMjGibDRs2YPr06W5/7w+GACIiIhFSrk7z5ao5m1WrVqGyshK7du1CVlaWaJu///3vOHLkSK81dCkpKb0WHnZ3d+Ps2bNej+uITxEkIiLykS9XzQHAypUrsWzZMtTU1DjN67t65ZVXkJubi+zsbKftBQUFOHfuHBoaGuzb3nvvPVitVuTn50vuPysBREREfjAYDJg5cyby8vLsV821t7ejpKQEAHpdNbdixQqUlZVh06ZNyMzMtM/hx8XFIS4uzr5fk8mE119/Hc8991yvY1599dW44447MHv2bFRXV6Orqwvz5s3DfffdJ/nKAIAhgIiIyC9yr5pbt24dLBYL7rnnHqf9uC483Lx5MwRBwLRp00SPu3HjRsybNw+33XYb1Go1pk6dij/+8Y+y+i77PgHhwPsEEBH1XaG8T8CM96dBG+fn1QEXLPjLLf8T1P5GCq4JICIiUiiGACIiIoViCCAiIlIohgAiIiKFYgggIiJSKIYAIiIihWIIICIiUiiGACIiIoViCCAiIlIo3jaYiIj6jVOd8YjW+HfHwK5OS4B6E/lYCSAiIlIohgAiIiKFYgggIiJSKIYAIiIihWIIICIiUiiGACIiIoViCCAiIlIo3ieAKIjUgxICvk/rOVPA90lEysQQQBQAwRjs5R6L4YCI5GIIIPJBKAd9qVz7xFBARN4wBBBJFIkDvyeO/WUgICIxDAFEHvS1gd8dBgIiEsMQQCSivwz+YmyfjWGAiBgCiP6tPw/8YlgdICKGAFI8pQ3+YlgdIFImhgBSLA7+vTEMUF/3r4uxiFLr/NpH90XlDI3K+aRE/xYxg39CvPvfmc6Hrh8iOFVApAwMAaQYIR38PQ3wgXp/iIKCelACgwBRP8UQQP1eSAZ/fwf9QBwziKGA0wRE/RNDAPVrQQ0A4Rj4PXHsT5ACAasCRP0LQwD1S0Eb/CNt4HcniIGAVQGi/oMhgPqdgAeAvjLwuxOkQMCqAFHfxxBA/UrAAkBfH/jdCXAgYBAg6tsYAqhfiNTB35owIKD7AwC16WJgdmT7rH6GAU4PEPVdDAHU5wUkAARg8A/GgC/1OH4FgwCGAQYBor6FIYD6NL8DgJ+Df6gGfm8c++FzIAhAGGAQIOpbGAKoTwrn2X+kDPzu+B0IEuL9DgIApweI+gJ1uDtAJFdAzv59CADWhAERHwBc2fosu98+fkeOIub2zEQhsHbtWmRmZiImJgb5+fnYt2+f27br16/HjTfeiMGDB2Pw4MHQ6/VO7bu6urBw4UKMGzcOAwcORFpaGoqLi/Hdd9857SczMxMqlcrpVVlZKavfrARQnxLq8n8gBv2uBP8eZuIo2mT2+b22zyKrOuDnFAGnB0gJtmzZAoPBgOrqauTn56OqqgqFhYU4cuQIhg0b1qt9XV0dpk2bhokTJyImJgYrVqzA7bffjoMHDyI9PR0dHR1obGzEkiVLkJ2dje+//x7z58/Hz3/+c3z88cdO+3r66acxe/Zs+8/x8fL+jmMIoD7DrwAQgsE/kIO93GPICQc+hwEGASJRq1evxuzZs1FSUgIAqK6uxrZt27BhwwYsWrSoV/uNGzc6/fzyyy/jf//3f1FbW4vi4mIkJiZi586dTm3WrFmDCRMmoLm5GcOHD7dvj4+PR0pKis9953QA9QmhCgByS+ddCTr7K5x86YfsaQI/pgc4NUB9kclkcnqZzb3DtsViQUNDA/R6vX2bWq2GXq9HfX29pON0dHSgq6sLQ4YMcdumra0NKpUKgwYNctpeWVmJoUOHYvz48Xj22WfR3d0t7cP9GysBFPF8HkBkDv5SBWPA74rXIPp8T2D25dA/KRUCWZUBP6YHWBGgUDBdjIFG5d//oz0XVQCAjIwMp+3l5eVYunSp07YzZ86gp6cHycnJTtuTk5Nx+PBhScdbuHAh0tLSnIKEo87OTixcuBDTpk1DQsKlvw9/+9vf4tprr8WQIUOwZ88elJaW4uTJk1i9erWk4wIMARThIikA+DL4d8VrAt5WTliw9VlqGJA8ReDj9ACDAPUlLS0tToOuThf4E4DKykps3rwZdXV1iImJ6fX7rq4u3HvvvRAEAevWrXP6ncFgsP97VlYWtFotHnroIVRUVEjuK0MARaxgB4BgDP5yBn1fuR5DSiiQGgZkVwUYBKgfS0hIcAoBYpKSkqDRaGA0Gp22G41Gr3P1q1atQmVlJXbt2oWsrKxev7cFgG+++Qbvvfee177k5+eju7sbx48fx1VXXeWxrQ3XBFBEioQAIHWOvSteY3+Fg5zjS/1MkqdHfLyUkGsEqL/QarXIzc1FbW2tfZvVakVtbS0KCgrcvm/lypVYtmwZampqkJeX1+v3tgDw5ZdfYteuXRg6dKjXvjQ1NUGtVotekeAOKwEUcXwaIAI8+Htt4+eA3xXn/f3RF+SvEXDsl6cKgZTKQLCrAqwIUH9hMBgwc+ZM5OXlYcKECaiqqkJ7e7v9aoHi4mKkp6ejoqICALBixQqUlZVh06ZNyMzMRGtrKwAgLi4OcXFx6Orqwj333IPGxka8++676OnpsbcZMmQItFot6uvrsXfvXtxyyy2Ij49HfX09FixYgF/96lcYPHiw5L4zBFDfF6AAEOjBX8pA7+v7pQQEW18DEQaCOT1A1NcVFRXh9OnTKCsrQ2trK3JyclBTU2NfLNjc3Ay1+lLhfd26dbBYLLjnnnuc9mNbeHjixAm88847AICcnBynNu+//z5uvvlm6HQ6bN68GUuXLoXZbMaIESOwYMECp3UCUqgEQRB8+MwhZTKZkJiYiFt19yJKpQ13dyiIZFcBQhQAJJXa/Rz0/SElFHhbO+BtvYDkRYMygwCrAf1ft2DBe+ataGtr8zqv7SvbOPHjjYugifXz6oAOM76YXhnU/kYKVgIoYoQjAPg7+Ps68FsGqry20bZLz+eO/XAXCLxVBrxVBYJVEeC0AFH4MARQRAhGAPDn7D9Qg7+UwV7ue72FA2+BQEoY8BQEAAlVAQYBoj6BIYD6JV/P/j0N/lIGfn8Gfalcj+EpFNj6LDcMBKQqwDUCRBGPIYDCLtBVgFAHAF8G/u6B3ttEtUvbl+Px3QWCrjiNx2kCX6sCPj2q2A1WA4hCjyGAwioSAkAwB38pg73c93oKB7Y+iYUBf6oCPgcBTgsQRTSGAAqbSA4A7gZ/bwO/P4O+VK7HEAsFnqoD3sIAgwCRcjAEUN8QwQFA6sDfHSutnZioDg/7dTi+u0DgrjIQqUGAiEKDIYDCIlS3jZUTAAI9+Psz6Hvbl7tQYOuTaxhwN03grirgSxAIJFYDiEKDzw6gkAvVNECwAkD3QPEA0B3r/JLDl/aejuWuj+4Cjdjnd1spcfO9er0ls8xnDPD5AkTB51MIWLt2LTIzMxETE4P8/Hzs27fPY/tz585h7ty5SE1NhU6nw49//GNs377dpw6TwoQpAFgGqnoNmN4Gf09cB22xQdxbG0/HkBMGxD4bEJlBgEguc0c0Oju0fr3MHdHh/hghIzsEbNmyBQaDAeXl5WhsbER2djYKCwtx6tQp0fYWiwU/+9nPcPz4cbzxxhs4cuQI1q9fj/T0dL87T31PIM/ughEA3A2Qcgd/f6oCnnjbp6cw4Erss0ZaEGA1gCi4ZK8JWL16NWbPnm1/OlJ1dTW2bduGDRs2YNGiRb3ab9iwAWfPnsWePXsQHf1DusrMzPSv16QMHgaLYAUAV+4GfzFSB/vuOKu0hgCiLrjP6Y7Hc10jYPud43ZP6wUc1wqILRiUu0YgkPcQ4PoAouCRVQmwWCxoaGiAXq+/tAO1Gnq9HvX19aLveeedd1BQUIC5c+ciOTkZY8eOxfLly9HT4/5hJmazGSaTyelFfZ+sszofysZSA0BXnCagAcDjmXmctddLDqnvd1sBENkuZa2A2HcktyLgEacFiCKCrBBw5swZ9PT02B+PaJOcnGx/1rGro0eP4o033kBPTw+2b9+OJUuW4LnnnsMf/vAHt8epqKhAYmKi/ZWRkSGnm9TPeS03O5Dy9D8pc/+ig6m7gdeHAb8n1oqeWGntPe3fUxhw+tnNWgFv5DxKWc5/JyIKj6BfHWC1WjFs2DC89NJLyM3NRVFREZ588klUV1e7fU9paSna2trsr5aWlmB3k4IsUFUAOdMAvkwBSDn79zb4i7EN8u5eUtqJcRcI3IWWXu/3EgSkrhHwaX0A1wYQhZ2sNQFJSUnQaDQwGo1O241GI1JSUkTfk5qaiujoaGg0l/7iuPrqq9Ha2gqLxQKtVtvrPTqdDjqdf8+Dpj4qQNMAwQwAvdp4GPgDyXF/mo7e+b07ztprDUF3rMu6gFiR9QMDndcJ+LpGwKd7CPAmQkRhJasSoNVqkZubi9raWvs2q9WK2tpaFBQUiL7n+uuvx1dffQWr9dJfYF988QVSU1NFAwD1P4E6i5NaXg5EAHA9kxY9sxY5A/d25q6J7ZL9EuPuOFKqAlLWCfhz1YCrQE0LsBpAFHiypwMMBgPWr1+P1157DYcOHcJvfvMbtLe3268WKC4uRmlpqb39b37zG5w9exbz58/HF198gW3btmH58uWYO3du4D4F9Q8BvBrAqY0PAcDjzx4Gf1dSBnRvvO1DThjw+LMPQcBVsKcFiCiwZF8iWFRUhNOnT6OsrAytra3IyclBTU2NfbFgc3Mz1OpL2SIjIwM7duzAggULkJWVhfT0dMyfPx8LFy4M3KegiBXMszcp0wDBCACOPJ3xSxETa+m1rbPDc4XMtu8elxua2PriOFXgOkXgbXpA7tRAwKYFJOLlgkSB5dOzA+bNm4d58+aJ/q6urq7XtoKCAnz00Ue+HIqUwocqgCtv5Wl/AoDYvL9rAPA08IsN9nLbuoYDx+M5BgLXMGDruy0M+BsEXLm7h4Arj/cO4NoAorDgswOoT/JlGsCRPwFArPwuFgBiYi32VyB42p+7aQJHjp9D7tSAI3+mBYgosvApghQ0kqcCglAFkHIzIBu5AcCRu8HfncTYTre/c6etI8btMRyrA2LTBD2xVrfTA1KuHLDxZVpATCCqAZwSIAochgDqc/w9y/R0lutPAHA3+Psy8Lt7v2sgcBcGfA0CjlynBeQK1WOHich3DAEUFIFYEBiMKoCnaQCpAUDK4C9l4E8a4H6EPXNRPKm4CwSuYcC1KiA1CHhaHxCSaoBErAYQBQZDAIWXzMvD5FYBpE4DBCoAuBv8PQ34Utu7BgPbsVzDgLuqgK9BwJG3RYKuZFcDuECQKKQYAqhP81YFcORYBQh0AAjU4O+JbV/ewkCgg4CnaYFwVgOIxPRcjIaAaO8NPbBe9P5nuL9gCKCAC9ZUgLcqgJxpACncBQApZ//uBv/kAfLPco0Xnasljvt2DASJsZ1OQQBwnh5wFwTccRcEpNxW2FGw1gZwSoDIf7xEkMLHzzvFyXminSMpVQBfA0DSgPZeASB5wHn7yxee3u96LNf+OPbX6b4CDp/P06WD7niaZvH1v4sd7yBIFDIMARRxfKkCuHJXBQhWABAb/AFpZ/6pujb7yxuxMOB67EAGAad/9+PeAWL//fioYaLw43QABVSoHvIiZy1AMLgGAFfuBn9vA7273580J/bav9g0gW16wHFqwBOpUwNySV0b4C9OCRD5h5UACo8glnyDXQXwFADEztTlnOm7I/Z+d1UBsX66qwY4klsN8DQl4DdOCRCFBEMARZRALAj0VSACgCtvA//lujO9Xp6I7c/fIODuIUhycUqAqO/hdAD1OZ4Wnrk7O5V6SaArXwOAu8Hf2yDvrs035qRe+3acInCdHnA3NeB6+aAruZcMerpvQKimBIjId6wEUMAE4lkBgSTnskBP9wMApAcAsbK91LN8T8Te73osqRUBGymLBMNK4p+TUK1DIeqPGAIoYoRiKsBbFcDT1QBiXAOAKykDf0b0v+wvb8T25ykIiJGyPsDGXQXFHU4JEPUtDAHUp0idCnC3IFCMtzlxd1UAfwKAu4Hfcbu7UCAnCHirBjiSszZA6gJBv+8ZQERBxRBAiiRnLYAvxAZqOWf8ru+Rsn93xC5h9Pb5ImZKgIiCiiGAQiuMl355qwjIWQvgqQrgLgD4Q0oQkDIt4G1tgBipdxEMCl4qSBRUDAHUZ0lZD+DLVIA/VQBfA0BGlPd7CMitCHhbHyD2Ob1NCQRiXQBRf7R27VpkZmYiJiYG+fn52Ldvn9u2Bw8exNSpU5GZmQmVSoWqqqpebXp6erBkyRKMGDECAwYMwKhRo7Bs2TIIwqWrcQRBQFlZGVJTUzFgwADo9Xp8+eWXsvrNEEARwd/FYXIeFuRtQaAYKVUAuQEgI6rN/hL7WfQ9XoKAu0sTvT3NUOzze5sSCOS6AH//+/MKAQqnLVu2wGAwoLy8HI2NjcjOzkZhYSFOnTol2r6jowMjR45EZWUlUlJSRNusWLEC69atw5o1a3Do0CGsWLECK1euxAsvvGBvs3LlSvzxj39EdXU19u7di4EDB6KwsBCdnZ7X/zhiCKCACMZfwq4ryUO9yEysdO5vAJAy0HtrJycIiFUDvC0QDDa5z4EgCheTyeT0MpvFn4a5evVqzJ49GyUlJRgzZgyqq6sRGxuLDRs2iLa/7rrr8Oyzz+K+++6DTif+/8OePXtw991346677kJmZibuuece3H777fYKgyAIqKqqwuLFi3H33XcjKysLf/7zn/Hdd9/h7bfflvwZGQJIEcTK2FKnArydRYvptepfwsDvdl8i7/VljYHUBYJi30tY1wUQyaC5qIamw8/XxR+GxoyMDCQmJtpfFRUVvY5nsVjQ0NAAvV5v36ZWq6HX61FfX+/z55g4cSJqa2vxxRdfAAA++eQT7N69G3feeScA4NixY2htbXU6bmJiIvLz82Udl3cMJAowfxcByjlOS9dQ0d+l6tp6PXQoUBzvHkjUn7W0tCAh4VKVU+ys/cyZM+jp6UFycrLT9uTkZBw+fNjnYy9atAgmkwmjR4+GRqNBT08PnnnmGUyfPh0A0Nraaj+O63Ftv5OClQDqt6SevXq7KsBGzlSAI18rAHL3464fUqcEvK2LIFKahIQEp5e70n0wbN26FRs3bsSmTZvQ2NiI1157DatWrcJrr70W0OOwEkDUh3mqBgSC47MEiKi3pKQkaDQaGI1Gp+1Go9Htoj8pHnvsMSxatAj33XcfAGDcuHH45ptvUFFRgZkzZ9r3bTQakZqa6nTcnJwcycfh/93UJ/l6GVqwb4Lj61TAjzR987I6d1cI8DJBUgqtVovc3FzU1tbat1mtVtTW1qKgoMDn/XZ0dECtdh6iNRoNrNYf/g4bMWIEUlJSnI5rMpmwd+9eWcdlJYBCJ0g3fpFzeaBcviwKdCRlKsAWAH6k0eDbHs9P3cuIakNLt+9z/Y5PF+wzEuIBk/dnIhCFi8FgwMyZM5GXl4cJEyagqqoK7e3tKCkpAQAUFxcjPT3dvrDQYrHg888/t//7iRMn0NTUhLi4OFxxxRUAgMmTJ+OZZ57B8OHDcc0112D//v1YvXo1HnjgAQCASqXC7373O/zhD3/AlVdeiREjRmDJkiVIS0vDlClTJPedIYDo3/y9VbAvXCsAUoKAJ5frzjg9dlgKb48XJiLPioqKcPr0aZSVlaG1tRU5OTmoqamxL9prbm52Oqv/7rvvMH78ePvPq1atwqpVq3DTTTehrq4OAPDCCy9gyZIleOSRR3Dq1CmkpaXhoYceQllZmf19jz/+ONrb2zFnzhycO3cON9xwA2pqahATEyO57yrB8fZDEcpkMiExMRG36u5FlIp/WUUiSfcJ8FAJkPIEQcf7BDiWm6U8OMj2747TAbZL4WwL4hxDgG3hnNhNgtwtCnR3XwB3PE0BeAoCYpUAx3UBjiHAdoWA8eKl795WCWjruPQXhS0E9HRE27dpOn74S8u2JsDxigCnf3colmjbL/11En3h0meIPu/8eaJNva+3Vpsu9tpmJ6ESYD1n8tqGwqNbsOA981a0tbU5rbYPJNs4cfmKP0AtYxAUY+3sxDcLFwe1v5GCawKo35N6j4BA8zUAePt9oK42kMufewXwaYJEkYkhgPo9sWvabWe5weRp7t5byV9uJSDQxL4ff+4N4FoJIKLIwBBAJJPjTXgcS+9yL9VzN9DLXRPg7rjBulkQEfUfDAFE/xaOxXGuA74/iwIByF4UCITncxNRZGAIoNAJ0mVejovRovy7oq8Xfy+nk1K6tw38UgKAv1MBfe7yQICXBxIFEUMA9UmOK8/lCPbd73y9e5+vFYBg3i1QCilXBhBR5GIIICK3eMtgov6N/4dTvyV1NbvjtfE2jtfQ2zhea+9ucaCYQK3m97YfsfsDAM79thH7fGLfAxH1bwwBRAEWqhJ9uKYC+Bhhov6Dtw0mRYjq6H2zG02HutdNgzo7tL1uH3zm4kDZzxBo6RrqdAdB21m8Lzf68XaHQKnEFgWKXRkQ6HsEEIWS5oIamm7/zm97OpVzfqycT0pBFYxbtrreWjaYN5wJxpSA2EDd0p1of3niqZ3Yfv2dCgglsVsGE1F4MARQRPB433gJ5Fwm6MtiN8ezaH+DgP13LgO9lIDgawDwdmmgWAjy9j25uzLAlZTw5u9/fz43gMg3DAHUZ0m5DM1bGVus9O3PzXPkBgF7GwmLB70FALmkTgU4kjItwMsDifoOhgAKrTDe+MXbAOZtSkBKNQDwPQh4IiUASKkC+HJVQFjXA/BGQURBxYWBpEhRF9ROjxV2JbZA0JOT5kSnRwx/Y05yesww4DyQiz122JWn4OBPBQDwXu3g/QGIlIH/p1Of4ml+2d26AF+mBBxJqQaI8TRQt3QNtb+kbPe2X1+rAI7kPFkxkOsBiCh8GAIoYogtDvO2klzu/LNjIBA723Usjbs7W/Y0LSBlasCVlIHf3f5cj+ktmADOn8vbgkCn7ysA6wHE/nv6uyiQiHzHEEABI3mFdojmeeU8TMjxLFjK5YKeKgJiQcD28oW797sex7UfctYCyKkChIzEPye8MoDId1wTQH1O9PkedMVrRH+nbRdgGajqtd3xZkFO/y5jbUBbRwwSYzvtv3O8iZDxYjySB1watGwDtOM6AUC8MuC4dkBKUHAd/G3Hd+QuAMhZC+CuCsCpAKL+IwLjPylZKKYE3HFXDXAcOOVUBADxAduVnEpBoAJAMKoAnAog6nsYAig8gjglIGWBoLu1AYEIAmLTA2LrBaRy936xY/kTAAJZBfAbLw0kCglOB1BAWc+ZoB6UEPTjuE4JRF/oQVec+BRBMHiaGgAunZ07ThEA4mfyjlMGUoKCu8V/cq4ECDTXKkCopgK4HoDIP6wEUMTxZUrAVbCrAYDnioCNlNX6cioFYvs7c3GgxwAQjCqAXJwKIIpMDAEUPn6WfH092wx2EHANA7ayvZRAIMbd+8WOFawA4CqoCwI5FUAUMpwOoIALxJSA2nQR1oQBTtuiTWZ0Jejcvsd1SsDxSoGodqDb8zN0enF81HBPRzQ0sV0ALg2s7q4aAC5VBVwfQexrEBDbtytPVwH4uxDQ01qAcC0I5FQAkf8YAqhPk7M2wDEISL1k0F0QAHpfPghAchjwhZTB39YvR54CgL/TAOFaC0DkTtRFQOP+ql9JVAp62jVDAIWX6TyQIP3s2Fs1wJW7+wYAgQ8CgPcw4MhTMPD26F/HYzkKdABwJfeKALlrOTgVQBRaDAEUFMGaEhDjrRrgaVrA1yAAwO30AOA8QLsGAhspA70rd6v+PQ3+gO8BQM40gNQqAKcCiCIHQwD1OXKrAa48rQ+QGgQA8aoAgF5PH5QSCDzxdLmf2B0AfQ0Arvy5GgDwoQpARCHHEEBBI7ka4GFKIBjVAFeOA7/rz1KCAABJYQAI3PX7UgZ/QF4AkDMNEJQqAJ8VQBRyvESQ+iQpZ5meVq27nuV6GhBdnzao6VD3GlzFBuDODq39FQie9id29u9PAJCzGFC0DasARH0CKwEUGYJQDXDlWg3wtD7A9WfbACqlKgA4VwYA9w/uEasWSA0NYsHD1i9HriFGbgDwthgwlFUAIgosVgIoqIJZuhU723QdkFzPWl0HNDkVAUBaVQD4YYC2vTxxPLuXWjVwt19vZ/+A/wFAyjRAMKsAnAogCiyGAIocHs4G3Z1FhisISA0DgPRA4I7j++UM/sEOAGLcBQBWAYgik08hYO3atcjMzERMTAzy8/Oxb98+Se/bvHkzVCoVpkyZ4sthqY8K1NmbP5eW+RIEXC+dkxMGpAQCqS8x7o7jbvD3+ll8CAChvCQQYBWAKBhkh4AtW7bAYDCgvLwcjY2NyM7ORmFhIU6dOuXxfcePH8fvf/973HjjjT53lhTAh7NCKdUAQH4QAKStoBcbeAHvgUAuT/tz1wdJ/Q9QAPBpGoBVAKKwkv230+rVqzF79myUlJRgzJgxqK6uRmxsLDZs2OD2PT09PZg+fTqeeuopjBw50usxzGYzTCaT04v6NllncUGaFgACFwTkhAHAeQAXe0ltI/WYomf7YQgAgZoGYBWAKDhkhQCLxYKGhgbo9fpLO1CrodfrUV9f7/Z9Tz/9NIYNG4ZZs2ZJOk5FRQUSExPtr4yMDDndpH4uFEHA2/SAu23ApYHZXSAQI6da4Gn/Uvsp9hml3BJYzrMB+KhgUhI50+QHDx7E1KlTkZmZCZVKhaqqql5t1q1bh6ysLCQkJCAhIQEFBQX4v//7P6c2N998M1QqldPr4YcfltVvWSHgzJkz6OnpQXJystP25ORktLa2ir5n9+7deOWVV7B+/XrJxyktLUVbW5v91dLSIqebFKECVQ2Qy5cgAEirCti2ubvRjuOALTccyHmv20Ai4ewfEK8ASL0hULCnAVgFoEgnd5q8o6MDI0eORGVlJVJSUkTb/OhHP0JlZSUaGhrw8ccf49Zbb8Xdd9+NgwcPOrWbPXs2Tp48aX+tXLlSVt+Dep+A8+fPY8aMGVi/fj2SkpIkv0+n00Gn8/22sNRP+HDvAHe3FBa7h4CUuwraBkzX+wkAzvcUcNwu9jtXcqoEbvfh6Za/Egd/wL9FgLwagMh5mhwAqqursW3bNmzYsAGLFi3q1f66667DddddBwCivweAyZMnO/38zDPPYN26dfjoo49wzTXX2LfHxsa6DRJSyPqbKCkpCRqNBkaj0Wm70WgU7cTXX3+N48ePY/LkyYiKikJUVBT+/Oc/45133kFUVBS+/vprnztOfVMgz+rkTAsA0isCcqoCbisAHc6vQJCyT7cVATdn/yEPADKxCkDh5Lo2zWzu/Wfe12lyOXp6erB582a0t7ejoKDA6XcbN25EUlISxo4di9LSUnR0yPsLR1YlQKvVIjc3F7W1tfbL/KxWK2prazFv3rxe7UePHo1PP/3UadvixYtx/vx5PP/885zrVyhZTxj08qjhYFQEgEtnx653GAR6P3xISgXAWxDojvU9LMh9CJC7uf+QBABWASjIojoAjfSlK6JU//7j7TpGlZeXY+nSpU7bPE2THz582K9+fPrppygoKEBnZyfi4uLw1ltvYcyYMfbf33///bj88suRlpaGAwcOYOHChThy5AjefPNNyceQPR1gMBgwc+ZM5OXlYcKECaiqqkJ7e7u9DFJcXIz09HRUVFQgJiYGY8eOdXr/oEGDAKDXdiK3whQEAOlTBPbfyZgScPe+QLSXWvq3icQAwCoAhVtLSwsSEi6dsIR6mvqqq65CU1MT2tra8MYbb2DmzJn44IMP7EFgzpw59rbjxo1DamoqbrvtNnz99dcYNWqUpGPIDgFFRUU4ffo0ysrK0NraipycHNTU1NhTUHNzM9Rq3oiQPJNVDfCD3CAAQFJVAHAeaL0FAns7GcHA035E23l46I+cwR8I8CJAHzAAUCSwrcz3RO40uRxarRZXXHEFACA3Nxf//Oc/8fzzz+NPf/qTaPv8/HwAwFdffRW8EAAA8+bNEy3/A0BdXZ3H97766qu+HJKUzsdqACAvCACeqwJA7zAAeK4OOLUL0PoAsWOL8XTZXyADAKcBSKnkTpP7w2q1iq5LsGlqagIApKamSt4nnyJIYSO7GhCkIABAclUAkBYGbLyFAl94GvRtAjn4A6ENAKwCUF8jZ5oc+GEx4eeff27/9xMnTqCpqQlxcXH2M//S0lLceeedGD58OM6fP49Nmzahrq4OO3bsAPDDwvtNmzZh0qRJGDp0KA4cOIAFCxbgpz/9KbKysiT3nSGAwioSggDguSoAyA8DNp4GbE8BQcpAL8aXwR9gACDyh9xp8u+++w7jx4+3/7xq1SqsWrUKN910k72afurUKRQXF+PkyZNITExEVlYWduzYgZ/97GcAfqhA7Nq1yx44MjIyMHXqVCxevFhW31WCIHi/TViYmUwmJCYm4lbdvYhSSXvWOvUtstcHeAgCANwGARt3YUAsCNh/JxIEXHkKBMEi6U5/DAAURt2CBe+Zt6Ktrc3rHLuvbOPEmEeWQ6OL8WtfPeZOfP7iE0Htb6RgJYD6JU8VAUD+9ADgPJC6CwSuA3IwQoGUQR/w/uhfXwd/gLcEJuovGAIoIgR6WgC4NFD5Oj0AuK8MeJomcORpwPYUEKQO9GL8GfwBP8/+bVgFIOoTGAIoYgQjCADe1wkA7qcHpIYBQNp0gSN/BnpP/XDbxo/BH2AAIOqPGAIoooQjCACeqwKA9zAA9B6I5YYCOaQM+va2Ep78xwBApEwMAdT3BTAIAO6rAoC0MGBv62Wg9hYS5Az0ou8PwOAPBC8AEFH4MQRQxPHpboK2AcjPdQKAvDAASAsEovvwc5AX3aeEgR+IjMGfVQCi8GMIoIjk822FA1QVAKSFASAwgcAfUgd+QPptfxkAiJSBIYAiViiCAOD9ngJSwwAgPiAHMhjIGfCd3hfowR9gACDqBxgCKKIFOwgA0qoCgLww4PQ+HwfuQJDzwB8GACLlYQigiOdXEAACWhUAnAdWuYEgFOQ+6S8Ugz/AAEChEdUBaLr924fKEpi+9AUMAdQn+PXoYZlVAUBaGAAiJxD48ohf2Xf9YwAg6ncYAqjP8DsIAEELA4D7gTiQ4cCXwd5VKAd/gAGAKJIxBFCf4lcQAGRVBQDnAVNOIHAUiIHbXz7f658BgKhfYwigPicgQQCQFQYA36oD4RauwR9gACDqCxgCqE+yDTChrArYBKI6EEx+P+GPZ/9EisEQQH1auKoCNpESCALyaF+e/RMpDkMA9Xl+BwHA7zAAuB+IAx0OAjLgOwrQPf8ZAIj6HoYA6hcCMj0ABCQMuAr4oB0oHPyJFI8hgPqVgFQFAOcBMoCBIOwC/KQ/BgCivo0hgPqdgAUBm74eCIL0iF8GAKK+jyGA+qWATQ+46iuBIEgDP8DBn6g/YQigfi1oYQCIvEAQxIHfhgGAqH9hCCBFCPgUgSvXATgUoSAEg74NB3+i/okhgBQjqFUBV1IHaHdhIYQDvCcc/In6N4YAUpyQhgFvImSwd8XBn0gZGAJIsSIqDEQIDv7U10V3CIjqEvzaR7ef7+9LGAJI8RgGOPgTKRVDANG/OQ6ESggEHPiJiCGASER/rg5w8CciG4YAIg/6S3WAAz8RiWEIIJKorwUCDvxE5A1DAJEPXAfYSAgFHPSJSC6GAKIA8DQABzIgcKAnokBiCCAKMg7cRBSp1OHuABEREYUHQwAREZFCMQQQEREpFEMAERGRn9auXYvMzEzExMQgPz8f+/bt89j+9ddfx+jRoxETE4Nx48Zh+/btTr83Go349a9/jbS0NMTGxuKOO+7Al19+6dSms7MTc+fOxdChQxEXF4epU6fCaDTK6jdDABERkR+2bNkCg8GA8vJyNDY2Ijs7G4WFhTh16pRo+z179mDatGmYNWsW9u/fjylTpmDKlCn47LPPAACCIGDKlCk4evQo/vrXv2L//v24/PLLodfr0d7ebt/PggUL8Le//Q2vv/46PvjgA3z33Xf45S9/KavvKkEQIv5xSSaTCYmJibhVdy+iVNpwd4eIiGToFix4z7wVbW1tSEgIzj01bONE7r3PICo6xq99dXd1omHrk5L7m5+fj+uuuw5r1qwBAFitVmRkZODRRx/FokWLerUvKipCe3s73n33Xfu2n/zkJ8jJyUF1dTW++OILXHXVVfjss89wzTXX2PeZkpKC5cuX48EHH0RbWxsuu+wybNq0Cffccw8A4PDhw7j66qtRX1+Pn/zkJ5I+KysBREREIkwmk9PLbDb3amOxWNDQ0AC9Xm/fplarodfrUV9fL7rf+vp6p/YAUFhYaG9vO05MzKUwo1arodPpsHv3bgBAQ0MDurq6nPYzevRoDB8+3O1xxTAEEBFRvxF9oScgLwDIyMhAYmKi/VVRUdHreGfOnEFPTw+Sk5OdticnJ6O1tVW0j62trR7b2wbz0tJSfP/997BYLFixYgW+/fZbnDx50r4PrVaLQYMGST6uGN4siIiISERLS4vTdIBOpwvJcaOjo/Hmm29i1qxZGDJkCDQaDfR6Pe68804EegafIYCIiEhEQkKC1zUBSUlJ0Gg0vVblG41GpKSkiL4nJSXFa/vc3Fw0NTWhra0NFosFl112GfLz85GXl2ffh8Viwblz55yqAZ6OK4bTAURERD7SarXIzc1FbW2tfZvVakVtbS0KCgpE31NQUODUHgB27twp2j4xMRGXXXYZvvzyS3z88ce4++67AfwQEqKjo532c+TIETQ3N7s9rhhWAoiIiPxgMBgwc+ZM5OXlYcKECaiqqkJ7eztKSkoAAMXFxUhPT7evKZg/fz5uuukmPPfcc7jrrruwefNmfPzxx3jppZfs+3z99ddx2WWXYfjw4fj0008xf/58TJkyBbfffjuAH8LBrFmzYDAYMGTIECQkJODRRx9FQUGB5CsDAIYAIiIivxQVFeH06dMoKytDa2srcnJyUFNTY1/819zcDLX6UuF94sSJ2LRpExYvXownnngCV155Jd5++22MHTvW3ubkyZMwGAwwGo1ITU1FcXExlixZ4nTc//zP/4RarcbUqVNhNptRWFiIF198UVbfeZ8AIiIKqlDeJ+Ank54OyH0CPtpeFtT+RgquCSAiIlIohgAiIiKFYgggIiJSKIYAIiIihWIIICIiUiiGACIiIoViCCAiIlIohgAiIiKFYgggIiJSKN42mIiI+o3oCz2Iiurxax+qbv/e35ewEkBERKRQDAFEREQK5VMIWLt2LTIzMxETE4P8/Hzs27fPbdv169fjxhtvxODBgzF48GDo9XqP7YmIiCg0ZIeALVu2wGAwoLy8HI2NjcjOzkZhYSFOnTol2r6urg7Tpk3D+++/j/r6emRkZOD222/HiRMn/O48ERER+U52CFi9ejVmz56NkpISjBkzBtXV1YiNjcWGDRtE22/cuBGPPPIIcnJyMHr0aLz88suwWq2ora11ewyz2QyTyeT0IiIiosCSFQIsFgsaGhqg1+sv7UCthl6vR319vaR9dHR0oKurC0OGDHHbpqKiAomJifZXRkaGnG4SERGRBLJCwJkzZ9DT04Pk5GSn7cnJyWhtbZW0j4ULFyItLc0pSLgqLS1FW1ub/dXS0iKnm0RERCRBSO8TUFlZic2bN6Ourg4xMTFu2+l0Ouh0uhD2jIiISHlkhYCkpCRoNBoYjUan7UajESkpKR7fu2rVKlRWVmLXrl3IysqS31MiIiIKKFnTAVqtFrm5uU6L+myL/AoKCty+b+XKlVi2bBlqamqQl5fne2+JiIgoYGRPBxgMBsycORN5eXmYMGECqqqq0N7ejpKSEgBAcXEx0tPTUVFRAQBYsWIFysrKsGnTJmRmZtrXDsTFxSEuLi6AH4WIiIjkkB0CioqKcPr0aZSVlaG1tRU5OTmoqamxLxZsbm6GWn2pwLBu3TpYLBbcc889TvspLy/H0qVL/es9ERER+cynhYHz5s3DvHnzRH9XV1fn9PPx48d9OQQREREFGZ8dQEREpFAMAURERAoV0vsEEBERBVP0eTOiNCq/9qHqMQeoN5GPlQAiIiKFYgggIiJSKIYAIiIihWIIICIiUiiGACIiIoViCCAiIlIohgAiIiKFYgggIiJSKIYAIiIihWIIICIiUiiGACIiIoViCCAiIvLT2rVrkZmZiZiYGOTn52Pfvn0e27/++usYPXo0YmJiMG7cOGzfvt1t24cffhgqlQpVVVVO2zMzM6FSqZxelZWVsvrNEEBEROSHLVu2wGAwoLy8HI2NjcjOzkZhYSFOnTol2n7Pnj2YNm0aZs2ahf3792PKlCmYMmUKPvvss15t33rrLXz00UdIS0sT3dfTTz+NkydP2l+PPvqorL4zBBAREflh9erVmD17NkpKSjBmzBhUV1cjNjYWGzZsEG3//PPP44477sBjjz2Gq6++GsuWLcO1116LNWvWOLU7ceIEHn30UWzcuBHR0dGi+4qPj0dKSor9NXDgQFl9ZwggIiISYTKZnF5mc+9HDFssFjQ0NECv19u3qdVq6PV61NfXi+63vr7eqT0AFBYWOrW3Wq2YMWMGHnvsMVxzzTVu+1hZWYmhQ4di/PjxePbZZ9Hd3S3rM0bJak1ERBTB1KZOqDWCf/vo+WGwz8jIcNpeXl6OpUuXOm07c+YMenp6kJyc7LQ9OTkZhw8fFt1/a2uraPvW1lb7zytWrEBUVBR++9vfuu3nb3/7W1x77bUYMmQI9uzZg9LSUpw8eRKrV6/2+hltGAKIiIhEtLS0ICEhwf6zTqcLyXEbGhrw/PPPo7GxESqVym07g8Fg//esrCxotVo89NBDqKiokNxXTgcQERGJSEhIcHqJDaxJSUnQaDQwGo1O241GI1JSUkT3m5KS4rH93//+d5w6dQrDhw9HVFQUoqKi8M033+A//uM/kJmZ6ba/+fn56O7uxvHjxyV/RoYAIiIiH2m1WuTm5qK2tta+zWq1ora2FgUFBaLvKSgocGoPADt37rS3nzFjBg4cOICmpib7Ky0tDY899hh27Njhti9NTU1Qq9UYNmyY5P5zOoCIiMgPBoMBM2fORF5eHiZMmICqqiq0t7ejpKQEAFBcXIz09HRUVFQAAObPn4+bbroJzz33HO666y5s3rwZH3/8MV566SUAwNChQzF06FCnY0RHRyMlJQVXXXUVgB8WF+7duxe33HIL4uPjUV9fjwULFuBXv/oVBg8eLLnvDAFERER+KCoqwunTp1FWVobW1lbk5OSgpqbGvvivubkZavWlwvvEiROxadMmLF68GE888QSuvPJKvP322xg7dqzkY+p0OmzevBlLly6F2WzGiBEjsGDBAqd1AlKoBEHwbxllCJhMJiQmJuJW3b2IUmnD3R0iIpKhW7DgPfNWtLW1OS20CyTbOKEf9TtEafxbwNfdY8aur6uC2t9IwTUBRERECsUQQEREpFAMAURERArFEEBERKRQDAFEREQKxRBARESkUAwBRERECsUQQEREpFAMAURERArF2wYTEVH/cf4CoLb4tw+rn+/vQ1gJICIiUiiGACIiIoViCCAiIlIohgAiIiKFYgggIiJSKIYAIiIihWIIICIiUiiGACIiIoViCCAiIlIohgAiIiKFYgggIiJSKIYAIiIihWIIICIiUiiGACIiIoViCCAiIlKoqHB3gIiIKFCsbSZYVVr/9iFYAtSbyMdKABERkUIxBBARESkUQwAREZFCMQQQEREpFEMAERGRQjEEEBERKRRDABERkUIxBBARESkUQwAREZFCMQQQEREpFEMAERGRQvkUAtauXYvMzEzExMQgPz8f+/bt89j+9ddfx+jRoxETE4Nx48Zh+/btPnWWiIgoEgV6XBQEAWVlZUhNTcWAAQOg1+vx5ZdfOrU5e/Yspk+fjoSEBAwaNAizZs3ChQsXZPVbdgjYsmULDAYDysvL0djYiOzsbBQWFuLUqVOi7ffs2YNp06Zh1qxZ2L9/P6ZMmYIpU6bgs88+k3toIiKiiBOMcXHlypX44x//iOrqauzduxcDBw5EYWEhOjs77W2mT5+OgwcPYufOnXj33Xfx4YcfYs6cObL6rhIEQZDzhvz8fFx33XVYs2YNAMBqtSIjIwOPPvooFi1a1Kt9UVER2tvb8e6779q3/eQnP0FOTg6qq6slHdNkMiExMRG36u5FlJ9PhyIiotDqFix4z7wVbW1tSEhICMoxAjlOyO1voMdFQRCQlpaG//iP/8Dvf/97AEBbWxuSk5Px6quv4r777sOhQ4cwZswY/POf/0ReXh4AoKamBpMmTcK3336LtLQ0SZ9V1qOELRYLGhoaUFpaat+mVquh1+tRX18v+p76+noYDAanbYWFhXj77bfdHsdsNsNsNtt/bmtrAwB0C11yuktERBHA9ne3zHNO346FLsDPw3Tjh/6aTCan7TqdDjqdzmlbMMbFY8eOobW1FXq93v77xMRE5Ofno76+Hvfddx/q6+sxaNAgewAAAL1eD7Vajb179+IXv/iFpM8qKwScOXMGPT09SE5OdtqenJyMw4cPi76ntbVVtH1ra6vb41RUVOCpp57qtf1Dy1tyuktERBHkX//6FxITE4Oyb61Wi5SUFHzYGphxIi4uDhkZGU7bysvLsXTpUqdtwRgXbf/01mbYsGFOv4+KisKQIUM8jq+uZIWAUCktLXVKSefOncPll1+O5ubmoP0B6g9MJhMyMjLQ0tIStJJbf8DvyTt+R9Lwe5Kmra0Nw4cPx5AhQ4J2jJiYGBw7dgwWiyUg+xMEASqVymmbaxWgP5AVApKSkqDRaGA0Gp22G41GpKSkiL4nJSVFVntAvOQC/FAO4f9o3iUkJPB7koDfk3f8jqTh9ySNWh3cq9JjYmIQExMT1GO4Csa4aPun0WhEamqqU5ucnBx7G9eFh93d3Th79qzH8dWVrP8iWq0Wubm5qK2ttW+zWq2ora1FQUGB6HsKCgqc2gPAzp073bYnIiLqK4IxLo4YMQIpKSlObUwmE/bu3WtvU1BQgHPnzqGhocHe5r333oPVakV+fr70DyDItHnzZkGn0wmvvvqq8Pnnnwtz5swRBg0aJLS2tgqCIAgzZswQFi1aZG//j3/8Q4iKihJWrVolHDp0SCgvLxeio6OFTz/9VPIx29raBABCW1ub3O4qCr8nafg9ecfvSBp+T9L09+8pGONiZWWlMGjQIOGvf/2rcODAAeHuu+8WRowYIVy8eNHe5o477hDGjx8v7N27V9i9e7dw5ZVXCtOmTZPVd9khQBAE4YUXXhCGDx8uaLVaYcKECcJHH31k/91NN90kzJw506n91q1bhR//+MeCVqsVrrnmGmHbtm2yjtfZ2SmUl5cLnZ2dvnRXMfg9ScPvyTt+R9Lwe5JGCd9ToMdFq9UqLFmyREhOThZ0Op1w2223CUeOHHFq869//UuYNm2aEBcXJyQkJAglJSXC+fPnZfVb9n0CiIiIqH/gswOIiIgUiiGAiIhIoRgCiIiIFIohgIiISKEYAoiIiBQqYkJAoJ/F3F/J+Z7Wr1+PG2+8EYMHD8bgwYOh1+u9fq/9gdw/SzabN2+GSqXClClTgtvBCCH3ezp37hzmzp2L1NRU6HQ6/PjHP1bE/3dyv6eqqipcddVVGDBgADIyMrBgwQKnx7/2Rx9++CEmT56MtLQ0qFQqjw+Is6mrq8O1114LnU6HK664Aq+++mrQ+0kiZF1QGCSbN28WtFqtsGHDBuHgwYPC7NmzhUGDBglGo1G0/T/+8Q9Bo9EIK1euFD7//HNh8eLFsm9A1BfJ/Z7uv/9+Ye3atcL+/fuFQ4cOCb/+9a+FxMRE4dtvvw1xz0NH7ndkc+zYMSE9PV248cYbhbvvvjs0nQ0jud+T2WwW8vLyhEmTJgm7d+8Wjh07JtTV1QlNTU0h7nloyf2eNm7cKOh0OmHjxo3CsWPHhB07dgipqanCggULQtzz0Nq+fbvw5JNPCm+++aYAQHjrrbc8tj969KgQGxsrGAwG4fPPPxdeeOEFQaPRCDU1NaHpMNlFRAiYMGGCMHfuXPvPPT09QlpamlBRUSHa/t577xXuuusup235+fnCQw89FNR+hpvc78lVd3e3EB8fL7z22mvB6mLY+fIddXd3CxMnThRefvllYebMmYoIAXK/p3Xr1gkjR44ULBZLqLoYEeR+T3PnzhVuvfVWp20Gg0G4/vrrg9rPSCIlBDz++OPCNddc47StqKhIKCwsDGLPSEzYpwNsz2J2fG6ylGcxO7YHfngWs7v2/YEv35Orjo4OdHV1BfVJXuHk63f09NNPY9iwYZg1a1Youhl2vnxP77zzDgoKCjB37lwkJydj7NixWL58OXp6ekLV7ZDz5XuaOHEiGhoa7FMGR48exfbt2zFp0qSQ9LmvUOLf4ZEq7I8SDsazmPsjX74nVwsXLkRaWlqv//n6C1++o927d+OVV15BU1NTCHoYGXz5no4ePYr33nsP06dPx/bt2/HVV1/hkUceQVdXF8rLy0PR7ZDz5Xu6//77cebMGdxwww0QBAHd3d14+OGH8cQTT4Siy32Gu7/DTSYTLl68iAEDBoSpZ8oT9koAhUZlZSU2b96Mt956K+SP2oxU58+fx4wZM7B+/XokJSWFuzsRzWq1YtiwYXjppZeQm5uLoqIiPPnkk6iurg531yJKXV0dli9fjhdffBGNjY148803sW3bNixbtizcXSMSFfZKQDCexdwf+fI92axatQqVlZXYtWsXsrKygtnNsJL7HX399dc4fvw4Jk+ebN9mtVoBAFFRUThy5AhGjRoV3E6HgS9/llJTUxEdHQ2NRmPfdvXVV6O1tRUWiwVarTaofQ4HX76nJUuWYMaMGXjwwQcBAOPGjUN7ezvmzJmDJ598Emo1z7sA93+HJyQksAoQYmH/ExmMZzH3R758TwCwcuVKLFu2DDU1NcjLywtFV8NG7nc0evRofPrpp2hqarK/fv7zn+OWW25BU1MTMjIyQtn9kPHlz9L111+Pr776yh6SAOCLL75AampqvwwAgG/fU0dHR6+B3hacBD6rzU6Jf4dHrHCvTBSE4DyLuT+S+z1VVlYKWq1WeOONN4STJ0/aX3IfNdmXyP2OXCnl6gC531Nzc7MQHx8vzJs3Tzhy5Ijw7rvvCsOGDRP+8Ic/hOsjhITc76m8vFyIj48X/ud//kc4evSo8P/+3/8TRo0aJdx7773h+gghcf78eWH//v3C/v37BQDC6tWrhf379wvffPONIAiCsGjRImHGjBn29rZLBB977DHh0KFDwtq1a3mJYJhERAgQhMA/i7m/kvM9XX755QKAXq/y8vLQdzyE5P5ZcqSUECAI8r+nPXv2CPn5+YJOpxNGjhwpPPPMM0J3d3eIex16cr6nrq4uYenSpcKoUaOEmJgYISMjQ3jkkUeE77//PvQdD6H3339f9O8a23czc+ZM4aabbur1npycHEGr1QojR44U/uu//ivk/SZBUAkCa1RERERKFPY1AURERBQeDAFEREQKxRBARESkUAwBRERECsUQQEREpFAMAURERArFEEBERKRQDAFEREQKxRBARESkUAwBRERECsUQQEREpFD/H8CRwf/p+YjhAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%matplotlib inline\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import jax\n", + "\n", + "jax.config.update(\"jax_enable_x64\", True)\n", + "\n", + "N = 50\n", + "kappa = 0.01\n", + "T = 0.25\n", + "N_t = 100\n", + "\n", + "dx = 1.0 / N\n", + "dt = T / N_t\n", + " \n", + " \n", + "x = jax.numpy.linspace(0.0, 1.0, N + 1)\n", + "y = jax.numpy.linspace(0.0, 1.0, N + 1)\n", + "X, Y = jax.numpy.meshgrid(x, y, indexing=\"ij\")\n", + "\n", + "\n", + "def timestep(x_n, m):\n", + " x_np1 = jax.numpy.zeros_like(x_n)\n", + " x_np1 = x_np1.at[1:-1, 1:-1].set(\n", + " x_n[1:-1, 1:-1]\n", + " + (kappa * dt / (dx * dx)) * (x_n[1:-1, 2:]\n", + " + x_n[:-2, 1:-1]\n", + " - 4.0 * x_n[1:-1, 1:-1]\n", + " + x_n[2:, 1:-1]\n", + " + x_n[1:-1, :-2])\n", + " + dt * m[1:-1, 1:-1])\n", + " return x_np1\n", + "\n", + "\n", + "def functional(x_n):\n", + " x_n = x_n.reshape((N + 1, N + 1))\n", + " return (dx * dx * (x_n[1:-1, 1:-1] ** 4).sum()) ** 0.25\n", + "\n", + "\n", + "def forward(x_0, m):\n", + " x_n = x_0.copy()\n", + " for _ in range(N_t):\n", + " x_n = timestep(x_n, m)\n", + " J = functional(x_n)\n", + " return x_n, J\n", + "\n", + "\n", + "x_0 = jax.numpy.zeros((N + 1, N + 1), dtype=jax.numpy.double)\n", + "x_0 = x_0.at[1:-1, 1:-1].set(jax.numpy.exp(-((X[1:-1, 1:-1] - 0.5) ** 2 + (Y[1:-1, 1:-1]- 0.5) ** 2) / (2.0 * (0.05 ** 2))))\n", + "m = jax.numpy.zeros((N + 1, N + 1), dtype=jax.numpy.double)\n", + "\n", + "fig, ax = plt.subplots(1, 1)\n", + "p = ax.contourf(x, y, x_0.T, 32)\n", + "fig.colorbar(p)\n", + "ax.set_aspect(1.0)\n", + "ax.set_title(\"$x_0$\")\n", + "\n", + "x_n, J = forward(x_0, m)\n", + "print(f\"{J=}\")\n", + "\n", + "fig, ax = plt.subplots(1, 1)\n", + "p = ax.contourf(x, y, x_n.T, 32)\n", + "fig.colorbar(p)\n", + "ax.set_aspect(1.0)\n", + "ax.set_title(\"$x_{N_t}$\")" + ] + }, + { + "cell_type": "markdown", + "id": "3ff06fb2-5469-492f-95ce-7ce741375e30", + "metadata": {}, + "source": [ + "## Adding tlm_adjoint\n", + "\n", + "The tlm_adjoint `Vector` class wraps ndim 1 JAX arrays. The following uses the tlm_adjoint `call_jax` function to record JAX operations on the internal tlm_adjoint manager. Note the use of `new_block`, indicating steps, and enabling use of a step-based checkpointing schedule." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "55f03da3", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:47:44.353240Z", + "iopub.status.busy": "2023-12-20T16:47:44.352897Z", + "iopub.status.idle": "2023-12-20T16:47:46.718352Z", + "shell.execute_reply": "2023-12-20T16:47:46.717295Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(True, True)" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from tlm_adjoint import *\n", + "\n", + "import jax\n", + "\n", + "reset_manager()\n", + "\n", + "N = 50\n", + "kappa = 0.01\n", + "T = 0.25\n", + "N_t = 100\n", + "\n", + "dx = 1.0 / N\n", + "dt = T / N_t\n", + " \n", + " \n", + "x = jax.numpy.linspace(0.0, 1.0, N + 1)\n", + "y = jax.numpy.linspace(0.0, 1.0, N + 1)\n", + "X, Y = jax.numpy.meshgrid(x, y, indexing=\"ij\")\n", + "\n", + "\n", + "def timestep(x_n, m):\n", + " x_n = x_n.reshape((N + 1, N + 1))\n", + " m = m.reshape((N + 1, N + 1))\n", + " x_np1 = jax.numpy.zeros_like(x_n)\n", + " x_np1 = x_np1.at[1:-1, 1:-1].set(\n", + " x_n[1:-1, 1:-1]\n", + " + (kappa * dt / (dx * dx)) * (x_n[1:-1, 2:]\n", + " + x_n[:-2, 1:-1]\n", + " - 4.0 * x_n[1:-1, 1:-1]\n", + " + x_n[2:, 1:-1]\n", + " + x_n[1:-1, :-2])\n", + " + dt * m[1:-1, 1:-1])\n", + " return x_np1.flatten()\n", + "\n", + "\n", + "def functional(x_n):\n", + " x_n = x_n.reshape((N + 1, N + 1))\n", + " return (dx * dx * (x_n[1:-1, 1:-1] ** 4).sum()) ** 0.25\n", + "\n", + "\n", + "def forward(x_0, m):\n", + " x_n = Vector((N + 1) ** 2)\n", + " x_np1 = Vector((N + 1) ** 2)\n", + " x_n.assign(x_0)\n", + " for n_t in range(N_t):\n", + " call_jax(x_np1, (x_n, m), timestep)\n", + " x_n.assign(x_np1)\n", + " if n_t < N_t - 1:\n", + " new_block()\n", + " J = new_jax_float()\n", + " call_jax(J, x_n, functional)\n", + " return J\n", + "\n", + "\n", + "x_0 = jax.numpy.zeros((N + 1, N + 1), dtype=jax.numpy.double)\n", + "x_0 = x_0.at[1:-1, 1:-1].set(jax.numpy.exp(-((X[1:-1, 1:-1] - 0.5) ** 2 + (Y[1:-1, 1:-1]- 0.5) ** 2) / (2.0 * (0.05 ** 2))))\n", + "x_0 = Vector(x_0.flatten())\n", + "m = Vector((N + 1) ** 2)\n", + "\n", + "start_manager()\n", + "J = forward(x_0, m)\n", + "stop_manager()" + ] + }, + { + "cell_type": "markdown", + "id": "5df893e2-47b5-4ba1-81c8-04542c344448", + "metadata": {}, + "source": [ + "We can now verify first order tangent-linear and adjoint calculations, and a second order reverse-over-forward calculation, using Taylor remainder convergence tests. Here we consider derivatives with respect to $m$." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "33eb96c6-8934-4c2c-b356-7b29962842e6", + "metadata": { + "execution": { + "iopub.execute_input": "2023-12-20T16:47:46.722240Z", + "iopub.status.busy": "2023-12-20T16:47:46.721795Z", + "iopub.status.idle": "2023-12-20T16:48:31.887203Z", + "shell.execute_reply": "2023-12-20T16:48:31.886469Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no tangent-linear = [5.42494549e-04 2.71112518e-04 1.35522677e-04 6.77529553e-05\n", + " 3.38743833e-05]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no tangent-linear = [1.00071691 1.00035745 1.0001785 1.0000892 ]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with tangent-linear = [5.37911835e-07 1.34199793e-07 3.35180646e-08 8.37571371e-09\n", + " 2.09346468e-09]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with tangent-linear = [2.00298728 2.00137169 2.00065481 2.00031955]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [5.44879307e-04 2.72292054e-04 1.36109209e-04 6.80454089e-05\n", + " 3.40204066e-05]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [1.00078182 1.0003902 1.00019495 1.00009744]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [5.89556319e-07 1.47178961e-07 3.67713890e-08 9.19011302e-09\n", + " 2.29719812e-09]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [2.00205818 2.00091587 2.00042917 2.00020732]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, no adjoint = [1.03638959e-04 5.16630940e-05 2.57966044e-05 1.28900955e-05\n", + " 6.44306279e-06]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, no adjoint = [1.00436048 1.00195287 1.00091823 1.00044439]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error norms, with adjoint = [5.80998182e-07 1.34113654e-07 3.21142524e-08 7.85039270e-09\n", + " 1.94023890e-09]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Orders, with adjoint = [2.11507753 2.06217051 2.03237698 2.01653052]\n" + ] + } + ], + "source": [ + "min_order = taylor_test_tlm(lambda m: forward(x_0, m), m, tlm_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(lambda m: forward(x_0, m), m, adjoint_order=1)\n", + "assert min_order > 2.00\n", + "\n", + "min_order = taylor_test_tlm_adjoint(lambda m: forward(x_0, m), m, adjoint_order=2)\n", + "assert min_order > 2.00" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/genindex.html b/genindex.html new file mode 100644 index 0000000..df8d151 --- /dev/null +++ b/genindex.html @@ -0,0 +1,2118 @@ + + + + + + Index — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+
    +
    • +
  • Index
    • +
  • +
    • +
+
+
+
+
+ + +

Index

+ +
+ A + | B + | C + | D + | E + | F + | G + | H + | I + | K + | L + | M + | N + | O + | P + | R + | S + | T + | U + | V + | W + | X + | Z + +
+

A

+ + + +
+ +

B

+ + + +
+ +

C

+ + + +
+ +

D

+ + + +
+ +

E

+ + + +
+ +

F

+ + + +
+ +

G

+ + + +
+ +

H

+ + + +
+ +

I

+ + + +
+ +

K

+ + +
+ +

L

+ + + +
+ +

M

+ + + +
+ +

N

+ + + +
+ +

O

+ + + +
+ +

P

+ + + +
+ +

R

+ + + +
+ +

S

+ + + +
+ +

T

+ + + +
+ +

U

+ + +
+ +

V

+ + + +
+ +

W

+ + + +
+ +

X

+ + + +
+ +

Z

+ + + +
+ + + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/index.html b/index.html new file mode 100644 index 0000000..e41408b --- /dev/null +++ b/index.html @@ -0,0 +1,186 @@ + + + + + + + tlm_adjoint — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

tlm_adjoint

+

tlm_adjoint is a high-level algorithmic differentiation tool, principally for +use with Firedrake.

+

The primary aim of tlm_adjoint is to enable higher order adjoint calculations +– and in particular to compute Hessian information – while also using adjoint +checkpointing schedules, and allowing for caching of assembled finite element +data, and caching of linear solver data.

+
+

Features

+
+
    +

  • Integrates with the Firedrake automated code generation library, and +applies a high-level algorithmic differentiation approach to enable +tangent-linear and adjoint calculations.

    • +

  • Applies a reverse-over-multiple-forward approach for higher order +adjoint calculations. For example a Hessian action on some given +direction is computed by deriving an adjoint associated with a combined +forward/tangent-linear calculation.

    • +

  • Implements adjoint checkpointing schedules to enable large problems to +be tackled.

    • +

  • Can apply automated finite element assembly and linear solver caching. +Cached data can be reused across the forward, tangent-linear, and adjoint +calculations.

    • +

  • Provides a simple ‘escape-hatch’ to enable parts of the problem, not +otherwise supported by tlm_adjoint or the backend library, to be added, +with adjoint or tangent-linear information defined manually.

    • +
+
+
+
+

Examples

+

The following Jupyter notebooks introduce derivative calculations using +tlm_adjoint.

+ +
+
+

Source and license

+

The source code is available from the +tlm_adjoint GitHub repository. +tlm_adjoint is licensed under the GNU LGPL version 3.

+
+
+

Indices

+ +
+
+
+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/objects.inv b/objects.inv new file mode 100644 index 0000000..11b7af9 Binary files /dev/null and b/objects.inv differ diff --git a/py-modindex.html b/py-modindex.html new file mode 100644 index 0000000..0aad560 --- /dev/null +++ b/py-modindex.html @@ -0,0 +1,338 @@ + + + + + + Python Module Index — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+
    +
    • +
  • Python Module Index
    • +
  • +
    • +
+
+
+
+
+ + +

Python Module Index

+ +
+ t +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
 
+ t
+ tlm_adjoint +
    + tlm_adjoint.adjoint +
    + tlm_adjoint.alias +
    + tlm_adjoint.cached_hessian +
    + tlm_adjoint.caches +
    + tlm_adjoint.checkpoint_schedules.binomial +
    + tlm_adjoint.checkpoint_schedules.h_revolve +
    + tlm_adjoint.checkpoint_schedules.memory +
    + tlm_adjoint.checkpoint_schedules.mixed +
    + tlm_adjoint.checkpoint_schedules.none +
    + tlm_adjoint.checkpoint_schedules.periodic +
    + tlm_adjoint.checkpoint_schedules.schedule +
    + tlm_adjoint.checkpointing +
    + tlm_adjoint.eigendecomposition +
    + tlm_adjoint.equation +
    + tlm_adjoint.equations +
    + tlm_adjoint.fenics.backend_interface +
    + tlm_adjoint.fenics.caches +
    + tlm_adjoint.fenics.equations +
    + tlm_adjoint.fenics.fenics_equations +
    + tlm_adjoint.fenics.functions +
    + tlm_adjoint.firedrake.backend_interface +
    + tlm_adjoint.firedrake.block_system +
    + tlm_adjoint.firedrake.caches +
    + tlm_adjoint.firedrake.equations +
    + tlm_adjoint.firedrake.firedrake_equations +
    + tlm_adjoint.firedrake.functions +
    + tlm_adjoint.firedrake.hessian_system +
    + tlm_adjoint.fixed_point +
    + tlm_adjoint.functional +
    + tlm_adjoint.hessian +
    + tlm_adjoint.instructions +
    + tlm_adjoint.interface +
    + tlm_adjoint.jax +
    + tlm_adjoint.linear_equation +
    + tlm_adjoint.manager +
    + tlm_adjoint.markers +
    + tlm_adjoint.optimization +
    + tlm_adjoint.overloaded_float +
    + tlm_adjoint.storage +
    + tlm_adjoint.tangent_linear +
    + tlm_adjoint.tlm_adjoint +
    + tlm_adjoint.verification +
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/search.html b/search.html new file mode 100644 index 0000000..d76f55a --- /dev/null +++ b/search.html @@ -0,0 +1,128 @@ + + + + + + Search — tlm_adjoint documentation + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+
    +
    • +
  • Search
    • +
  • +
    • +
+
+
+
+
+ + + + +
+ +
+ +
+
+ +
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/searchindex.js b/searchindex.js new file mode 100644 index 0000000..ab8d378 --- /dev/null +++ b/searchindex.js @@ -0,0 +1 @@ +Search.setIndex({"docnames": ["acknowledgements", "autoapi/tlm_adjoint/adjoint/index", "autoapi/tlm_adjoint/alias/index", "autoapi/tlm_adjoint/cached_hessian/index", "autoapi/tlm_adjoint/caches/index", "autoapi/tlm_adjoint/checkpoint_schedules/binomial/index", "autoapi/tlm_adjoint/checkpoint_schedules/h_revolve/index", "autoapi/tlm_adjoint/checkpoint_schedules/memory/index", "autoapi/tlm_adjoint/checkpoint_schedules/mixed/index", "autoapi/tlm_adjoint/checkpoint_schedules/none/index", "autoapi/tlm_adjoint/checkpoint_schedules/periodic/index", "autoapi/tlm_adjoint/checkpoint_schedules/schedule/index", "autoapi/tlm_adjoint/checkpointing/index", "autoapi/tlm_adjoint/eigendecomposition/index", "autoapi/tlm_adjoint/equation/index", "autoapi/tlm_adjoint/equations/index", "autoapi/tlm_adjoint/fenics/backend_interface/index", "autoapi/tlm_adjoint/fenics/caches/index", "autoapi/tlm_adjoint/fenics/equations/index", "autoapi/tlm_adjoint/fenics/fenics_equations/index", "autoapi/tlm_adjoint/fenics/functions/index", "autoapi/tlm_adjoint/firedrake/backend_interface/index", "autoapi/tlm_adjoint/firedrake/block_system/index", "autoapi/tlm_adjoint/firedrake/caches/index", "autoapi/tlm_adjoint/firedrake/equations/index", "autoapi/tlm_adjoint/firedrake/firedrake_equations/index", "autoapi/tlm_adjoint/firedrake/functions/index", "autoapi/tlm_adjoint/firedrake/hessian_system/index", "autoapi/tlm_adjoint/fixed_point/index", "autoapi/tlm_adjoint/functional/index", "autoapi/tlm_adjoint/hessian/index", "autoapi/tlm_adjoint/index", "autoapi/tlm_adjoint/instructions/index", "autoapi/tlm_adjoint/interface/index", "autoapi/tlm_adjoint/jax/index", "autoapi/tlm_adjoint/linear_equation/index", "autoapi/tlm_adjoint/manager/index", "autoapi/tlm_adjoint/markers/index", "autoapi/tlm_adjoint/optimization/index", "autoapi/tlm_adjoint/overloaded_float/index", "autoapi/tlm_adjoint/storage/index", "autoapi/tlm_adjoint/tangent_linear/index", "autoapi/tlm_adjoint/tlm_adjoint/index", "autoapi/tlm_adjoint/verification/index", "dependencies", "examples/0_getting_started", "examples/1_time_independent", "examples/2_verification", "examples/3_time_dependent", "examples/4_riesz_maps", "examples/5_optimization", "examples/6_custom_operations", "examples/7_jax_integration", "index"], "filenames": ["acknowledgements.rst", "autoapi/tlm_adjoint/adjoint/index.rst", "autoapi/tlm_adjoint/alias/index.rst", "autoapi/tlm_adjoint/cached_hessian/index.rst", "autoapi/tlm_adjoint/caches/index.rst", "autoapi/tlm_adjoint/checkpoint_schedules/binomial/index.rst", "autoapi/tlm_adjoint/checkpoint_schedules/h_revolve/index.rst", "autoapi/tlm_adjoint/checkpoint_schedules/memory/index.rst", "autoapi/tlm_adjoint/checkpoint_schedules/mixed/index.rst", "autoapi/tlm_adjoint/checkpoint_schedules/none/index.rst", "autoapi/tlm_adjoint/checkpoint_schedules/periodic/index.rst", "autoapi/tlm_adjoint/checkpoint_schedules/schedule/index.rst", "autoapi/tlm_adjoint/checkpointing/index.rst", "autoapi/tlm_adjoint/eigendecomposition/index.rst", "autoapi/tlm_adjoint/equation/index.rst", "autoapi/tlm_adjoint/equations/index.rst", "autoapi/tlm_adjoint/fenics/backend_interface/index.rst", "autoapi/tlm_adjoint/fenics/caches/index.rst", "autoapi/tlm_adjoint/fenics/equations/index.rst", "autoapi/tlm_adjoint/fenics/fenics_equations/index.rst", "autoapi/tlm_adjoint/fenics/functions/index.rst", "autoapi/tlm_adjoint/firedrake/backend_interface/index.rst", "autoapi/tlm_adjoint/firedrake/block_system/index.rst", "autoapi/tlm_adjoint/firedrake/caches/index.rst", "autoapi/tlm_adjoint/firedrake/equations/index.rst", "autoapi/tlm_adjoint/firedrake/firedrake_equations/index.rst", "autoapi/tlm_adjoint/firedrake/functions/index.rst", "autoapi/tlm_adjoint/firedrake/hessian_system/index.rst", "autoapi/tlm_adjoint/fixed_point/index.rst", "autoapi/tlm_adjoint/functional/index.rst", "autoapi/tlm_adjoint/hessian/index.rst", "autoapi/tlm_adjoint/index.rst", "autoapi/tlm_adjoint/instructions/index.rst", "autoapi/tlm_adjoint/interface/index.rst", "autoapi/tlm_adjoint/jax/index.rst", "autoapi/tlm_adjoint/linear_equation/index.rst", "autoapi/tlm_adjoint/manager/index.rst", "autoapi/tlm_adjoint/markers/index.rst", "autoapi/tlm_adjoint/optimization/index.rst", "autoapi/tlm_adjoint/overloaded_float/index.rst", "autoapi/tlm_adjoint/storage/index.rst", "autoapi/tlm_adjoint/tangent_linear/index.rst", "autoapi/tlm_adjoint/tlm_adjoint/index.rst", "autoapi/tlm_adjoint/verification/index.rst", "dependencies.rst", "examples/0_getting_started.ipynb", "examples/1_time_independent.ipynb", "examples/2_verification.ipynb", "examples/3_time_dependent.ipynb", "examples/4_riesz_maps.ipynb", "examples/5_optimization.ipynb", "examples/6_custom_operations.ipynb", "examples/7_jax_integration.ipynb", "index.rst"], "titles": ["References and acknowledgements", "tlm_adjoint.adjoint", "tlm_adjoint.alias", "tlm_adjoint.cached_hessian", "tlm_adjoint.caches", "tlm_adjoint.checkpoint_schedules.binomial", "tlm_adjoint.checkpoint_schedules.h_revolve", "tlm_adjoint.checkpoint_schedules.memory", "tlm_adjoint.checkpoint_schedules.mixed", "tlm_adjoint.checkpoint_schedules.none", "tlm_adjoint.checkpoint_schedules.periodic", "tlm_adjoint.checkpoint_schedules.schedule", "tlm_adjoint.checkpointing", "tlm_adjoint.eigendecomposition", "tlm_adjoint.equation", "tlm_adjoint.equations", "tlm_adjoint.fenics.backend_interface", "tlm_adjoint.fenics.caches", "tlm_adjoint.fenics.equations", "tlm_adjoint.fenics.fenics_equations", "tlm_adjoint.fenics.functions", "tlm_adjoint.firedrake.backend_interface", "tlm_adjoint.firedrake.block_system", "tlm_adjoint.firedrake.caches", "tlm_adjoint.firedrake.equations", "tlm_adjoint.firedrake.firedrake_equations", "tlm_adjoint.firedrake.functions", "tlm_adjoint.firedrake.hessian_system", "tlm_adjoint.fixed_point", "tlm_adjoint.functional", "tlm_adjoint.hessian", "tlm_adjoint", "tlm_adjoint.instructions", "tlm_adjoint.interface", "tlm_adjoint.jax", "tlm_adjoint.linear_equation", "tlm_adjoint.manager", "tlm_adjoint.markers", "tlm_adjoint.optimization", "tlm_adjoint.overloaded_float", "tlm_adjoint.storage", "tlm_adjoint.tangent_linear", "tlm_adjoint.tlm_adjoint", "tlm_adjoint.verification", "Dependencies", "Getting started with tlm_adjoint", "Time-independent example", "Verifying derivative calculations", "Time-dependent example", "Visualizing derivatives", "Functional minimization", "Defining custom operations", "JAX integration", "tlm_adjoint"], "terms": {"i": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53], "describ": [0, 5, 8, 18, 24, 27, 28, 38, 42, 43, 45, 46, 47, 48, 49, 50, 51], "jame": [0, 8, 11, 45, 51], "r": [0, 8, 11, 18, 24, 27, 45, 46, 48, 49, 50, 51], "maddison": [0, 8, 11, 18, 24, 45, 46, 51], "daniel": [0, 5, 45, 51], "n": [0, 1, 5, 11, 12, 19, 22, 25, 34, 42, 45, 47, 48, 50, 51, 52], "goldberg": [0, 5, 45, 51], "benjamin": [0, 45, 51], "d": [0, 3, 5, 11, 12, 19, 25, 30, 40, 43, 45, 46, 47, 49, 51], "goddard": [0, 45, 51], "autom": [0, 43, 45, 46, 47, 51, 53], "calcul": [0, 3, 5, 6, 7, 8, 9, 10, 11, 12, 14, 18, 20, 24, 26, 32, 35, 39, 42, 43, 45, 46, 48, 51, 52, 53], "higher": [0, 42, 43, 48, 51, 53], "order": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 22, 28, 42, 43, 46, 48, 50, 51, 52, 53], "partial": [0, 15, 18, 19, 24, 25, 27, 35, 45, 46, 47, 48, 49, 50, 51, 52, 53], "equat": [0, 1, 5, 12, 19, 25, 27, 28, 29, 32, 33, 34, 35, 36, 38, 39, 40, 42, 45, 46, 47, 48, 49, 50, 52, 53], "constrain": [0, 38, 45, 50, 51, 53], "deriv": [0, 3, 5, 6, 7, 8, 9, 10, 11, 14, 15, 18, 19, 24, 25, 28, 30, 32, 34, 35, 36, 37, 39, 40, 41, 42, 43, 50, 51, 52, 53], "inform": [0, 5, 13, 14, 18, 24, 28, 34, 35, 39, 40, 42, 45, 47, 50, 51, 53], "siam": [0, 5, 38, 42, 43, 45, 46, 47, 51], "journal": [0, 5, 27, 38, 42, 43, 45, 46, 47, 51], "scientif": [0, 5, 38, 42, 43, 45, 46, 47, 51], "comput": [0, 3, 5, 12, 13, 14, 15, 18, 19, 22, 24, 25, 27, 28, 30, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 47, 49, 50, 51, 52, 53], "41": [0, 45, 48, 51], "5": [0, 38, 43, 45, 46, 47, 48, 50, 51, 52], "pp": [0, 5, 27, 28, 42, 43, 45, 46, 47, 48, 51], "c417": [0, 45, 51], "c445": [0, 45, 51], "2019": [0, 45, 51], "doi": [0, 5, 18, 24, 27, 28, 38, 42, 43, 45, 46, 47, 48, 51], "10": [0, 5, 18, 24, 27, 28, 38, 42, 43, 45, 46, 47, 48, 49, 50, 51], "1137": [0, 5, 38, 42, 43, 45, 46, 47, 51], "18m1209465": [0, 45, 51], "The": [0, 1, 3, 4, 5, 6, 8, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 45, 47, 48, 49, 50, 51, 52, 53], "assembli": [0, 17, 18, 22, 23, 24, 48, 51, 53], "linear": [0, 3, 6, 7, 8, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 39, 40, 41, 42, 43, 47, 49, 51, 52, 53], "solver": [0, 17, 18, 19, 22, 23, 24, 25, 28, 43, 47, 48, 51, 53], "cach": [0, 3, 12, 14, 16, 18, 19, 20, 21, 24, 25, 26, 33, 34, 39, 42, 48, 53], "appli": [0, 12, 17, 18, 20, 22, 23, 24, 26, 27, 38, 42, 45, 46, 47, 48, 51, 53], "base": [0, 12, 14, 18, 22, 24, 27, 28, 30, 33, 35, 38, 40, 42, 46, 48, 50, 51, 52], "approach": [0, 5, 18, 24, 28, 30, 38, 42, 43, 45, 46, 47, 50, 51, 53], "j": [0, 3, 5, 18, 22, 24, 27, 30, 36, 37, 38, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52], "p": [0, 1, 5, 18, 19, 24, 25, 38, 43, 46, 47, 48, 49, 50, 51, 52], "e": [0, 1, 3, 5, 6, 11, 12, 18, 22, 24, 30, 33, 35, 41, 42, 43, 46, 47, 52], "farrel": [0, 18, 24, 43, 46, 47], "rapid": [0, 18, 24, 46], "develop": [0, 13, 18, 24, 46], "adjoin": [0, 18, 24, 46], "transient": [0, 18, 24, 43, 46, 47], "finit": [0, 17, 18, 19, 22, 23, 24, 25, 33, 39, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53], "element": [0, 4, 6, 11, 12, 14, 15, 17, 18, 19, 22, 23, 24, 25, 28, 32, 33, 34, 35, 37, 38, 39, 40, 43, 46, 47, 48, 49, 50, 51, 52, 53], "model": [0, 5, 18, 24, 39, 46], "method": [0, 1, 2, 4, 11, 12, 13, 14, 15, 18, 19, 22, 24, 25, 27, 28, 32, 33, 34, 35, 38, 39, 40, 42, 45, 46, 47, 48, 49, 50, 53], "mechan": [0, 18, 24, 46, 51], "engin": [0, 18, 24, 46], "276": [0, 18, 24, 46], "95": [0, 18, 24, 46, 48], "121": [0, 18, 24, 46], "2014": [0, 18, 24, 46], "1016": [0, 18, 24, 27, 46], "cma": [0, 18, 24, 46], "03": [0, 18, 24, 46, 51], "010": [0, 18, 24, 46], "mix": [0, 5, 22, 48], "forward": [0, 1, 3, 5, 6, 7, 8, 10, 11, 12, 14, 15, 18, 19, 24, 25, 28, 30, 32, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 53], "restart": [0, 5, 6, 7, 8, 10, 11, 12, 22, 42, 48], "non": [0, 4, 6, 7, 8, 10, 11, 12, 13, 14, 15, 18, 19, 22, 24, 25, 28, 32, 33, 34, 35, 37, 38, 39, 40, 42, 45, 46, 47, 51], "depend": [0, 3, 4, 6, 7, 8, 10, 11, 12, 14, 15, 18, 19, 24, 25, 28, 30, 32, 33, 34, 35, 37, 38, 39, 40, 42, 45, 46, 47, 51, 53], "data": [0, 6, 7, 8, 10, 11, 12, 13, 17, 18, 22, 23, 24, 25, 27, 33, 34, 39, 40, 42, 48, 51, 53], "schedul": [0, 5, 6, 7, 8, 9, 10, 42, 51, 52, 53], "defin": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 52, 53], "code": [0, 13, 45, 46, 48, 51, 52, 53], "checkpoint_schedul": 0, "py": [0, 13], "ar": [0, 1, 4, 5, 6, 7, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 29, 33, 34, 35, 38, 39, 42, 43, 45, 46, 47, 48, 49, 50, 51], "On": [0, 8, 11, 42], "implement": [0, 5, 8, 11, 14, 17, 18, 22, 23, 24, 28, 33, 34, 35, 38, 39, 40, 42, 43, 46, 47, 48, 51, 52, 53], "high": [0, 5, 8, 11, 43, 45, 46, 47, 53], "level": [0, 5, 6, 8, 11, 39, 43, 45, 46, 47, 48, 51, 52, 53], "algorithm": [0, 5, 8, 11, 13, 27, 38, 39, 42, 45, 46, 48, 50, 51, 53], "http": [0, 8, 11], "arxiv": [0, 8, 11], "org": [0, 8, 11], "ab": [0, 8, 11, 48, 50, 51], "2305": [0, 8, 11], "09568v1": [0, 8, 11], "2023": [0, 8, 11], "us": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 30, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 44, 47, 49, 50, 52, 53], "an": [0, 1, 2, 3, 4, 5, 6, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 42, 43, 47, 49, 50, 51, 52, 53], "A": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 46, 47, 48, 50, 51, 53], "ham": [0, 43, 46, 47], "": [0, 5, 22, 38, 43, 45, 46, 47, 48], "w": [0, 22, 43, 46, 47], "funk": [0, 43, 46, 47], "m": [0, 1, 3, 5, 12, 14, 15, 18, 19, 22, 24, 25, 27, 28, 30, 32, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 49, 50, 51, 52], "rogn": [0, 43, 46, 47], "program": [0, 43, 46, 47], "35": [0, 43, 45, 46, 47, 48], "4": [0, 5, 28, 38, 43, 45, 46, 47, 48, 49, 50, 51, 52], "c369": [0, 43, 46, 47], "c393": [0, 43, 46, 47], "2013": [0, 43, 46, 47], "120873558": [0, 43, 46, 47], "wa": [0, 12, 33, 42], "from": [0, 1, 4, 5, 6, 10, 11, 12, 14, 15, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 33, 34, 35, 39, 40, 41, 42, 45, 46, 47, 48, 49, 50, 51, 52, 53], "custom": [0, 22, 38, 52, 53], "extens": 0, "function": [0, 3, 5, 6, 11, 14, 16, 18, 19, 21, 22, 24, 25, 27, 30, 33, 36, 37, 38, 39, 42, 43, 45, 46, 47, 48, 49, 51, 52, 53], "verif": [0, 31, 47], "eigendecompos": [0, 13, 27], "eigendecomposit": [0, 27], "origin": [0, 2, 4, 5, 6, 13, 22, 33, 42, 45, 46], "loos": [0, 13], "follow": [0, 13, 42, 46, 47, 48, 50, 51, 52, 53], "slepc4pi": [0, 13, 27], "3": [0, 5, 13, 28, 38, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53], "6": [0, 13, 38, 45, 46, 47, 48, 51, 52], "0": [0, 5, 6, 12, 13, 14, 15, 18, 19, 22, 24, 28, 35, 38, 39, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52], "demo": [0, 13], "ex3": [0, 13], "licens": [0, 13], "python": [0, 2, 13, 22, 32], "author": [0, 13], "lisandro": [0, 13], "dalcin": [0, 13], "contact": [0, 13], "dalcinl": [0, 13], "gmail": [0, 13], "com": [0, 13], "copyright": [0, 13], "c": [0, 5, 13, 22], "2015": [0, 13, 27], "all": [0, 1, 4, 7, 11, 13, 22, 27, 28, 33, 34, 38, 42, 45, 46, 47, 48, 50, 51], "right": [0, 1, 3, 13, 14, 15, 17, 18, 19, 22, 23, 24, 25, 27, 28, 30, 32, 33, 34, 35, 37, 38, 39, 40, 43, 45, 46, 47, 48, 49, 50, 51, 52], "reserv": [0, 13], "redistribut": [0, 13], "sourc": [0, 13], "binari": [0, 13], "form": [0, 13, 17, 18, 19, 20, 22, 23, 24, 25, 26, 38, 42, 46, 47, 48, 51], "without": [0, 13, 33, 48, 52], "modif": [0, 13, 51], "permit": [0, 5, 6, 7, 8, 9, 10, 11, 13, 19, 28], "provid": [0, 5, 11, 12, 13, 14, 22, 25, 32, 35, 36, 38, 40, 42, 45, 53], "condit": [0, 12, 13, 14, 17, 18, 20, 22, 23, 24, 26, 38, 42, 45, 46, 47, 48, 49, 50, 51, 52], "met": [0, 13], "must": [0, 3, 13, 14, 15, 46], "retain": [0, 13, 38, 42], "abov": [0, 13, 18, 24, 38, 46, 47, 48, 50], "notic": [0, 13, 48], "thi": [0, 3, 4, 5, 11, 12, 13, 14, 15, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52], "list": [0, 13], "disclaim": [0, 13], "reproduc": [0, 13, 47], "document": [0, 13, 38, 39], "other": [0, 13, 14, 18, 21, 22, 24, 25, 27, 28, 34, 35, 38, 39, 42, 43], "materi": [0, 13], "distribut": [0, 5, 13, 42], "softwar": [0, 5, 13, 28, 42, 48], "BY": [0, 13], "THE": [0, 13], "holder": [0, 13], "AND": [0, 13], "contributor": [0, 13], "AS": [0, 13], "ani": [0, 1, 13, 33, 40, 42, 45, 46, 47, 51], "express": [0, 13, 18, 19, 24, 25, 39, 51], "OR": [0, 13], "impli": [0, 13], "warranti": [0, 13], "includ": [0, 12, 13, 19, 20, 25, 26, 35, 42, 43, 46, 47, 53], "BUT": [0, 13], "NOT": [0, 13], "limit": [0, 13, 22, 38, 48, 50], "TO": [0, 13], "OF": [0, 13], "merchant": [0, 13], "fit": [0, 13, 43, 50], "FOR": [0, 13], "particular": [0, 13, 18, 24, 27, 33, 39, 46, 53], "purpos": [0, 13], "IN": [0, 13], "NO": [0, 13], "event": [0, 13], "shall": [0, 13], "BE": [0, 13], "liabl": [0, 13], "direct": [0, 13, 14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 38, 39, 40, 41, 42, 43, 45, 47, 48, 49, 51, 53], "indirect": [0, 13], "incident": [0, 13], "special": [0, 13], "exemplari": [0, 13], "consequenti": [0, 13], "damag": [0, 13], "procur": [0, 13], "substitut": [0, 13, 47], "good": [0, 13], "servic": [0, 13], "loss": [0, 13], "profit": [0, 13], "busi": [0, 13], "interrupt": [0, 13], "howev": [0, 13, 45, 46, 48, 49, 52], "caus": [0, 13], "ON": [0, 13], "theori": [0, 13], "liabil": [0, 13], "whether": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 20, 21, 22, 24, 26, 27, 28, 29, 32, 33, 34, 35, 36, 38, 39, 40, 42, 51], "contract": [0, 13], "strict": [0, 13], "tort": [0, 13], "neglig": [0, 13], "otherwis": [0, 1, 4, 12, 13, 14, 15, 18, 19, 24, 25, 28, 32, 33, 34, 35, 37, 39, 40, 41, 42, 51, 53], "aris": [0, 13], "wai": [0, 13, 43], "out": [0, 13, 46, 48, 51], "even": [0, 13, 48], "IF": [0, 13], "advis": [0, 13], "possibl": [0, 4, 5, 13, 33, 46], "SUCH": [0, 13], "fixedpointsolv": [0, 28], "class": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 46, 51, 52], "fixed_point": 0, "tangent": [0, 3, 14, 15, 16, 18, 19, 20, 21, 24, 25, 26, 28, 29, 30, 32, 33, 34, 35, 36, 39, 40, 41, 42, 43, 47, 49, 51, 52, 53], "jean": [0, 28], "charl": [0, 28], "gilbert": [0, 28], "automat": [0, 1, 3, 12, 28, 33, 42, 46], "iter": [0, 5, 6, 7, 8, 9, 10, 11, 14, 15, 18, 19, 22, 24, 25, 28, 32, 33, 34, 35, 37, 38, 39, 40, 43, 47, 51], "process": [0, 12, 19, 28, 33, 34, 36, 38, 39, 40, 42, 45, 46, 48], "optim": [0, 5, 14, 18, 24, 28, 34, 39, 42, 50, 53], "1": [0, 5, 6, 15, 18, 22, 24, 27, 28, 30, 34, 38, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52], "13": [0, 28, 45, 46, 47, 48], "21": [0, 28, 48, 50], "1992": [0, 28], "1080": [0, 28], "10556789208805503": [0, 28], "bruce": [0, 28], "christianson": [0, 28], "revers": [0, 5, 11, 28, 42, 48, 51, 52, 53], "accumul": [0, 28], "attract": [0, 28], "311": [0, 28], "326": [0, 28], "1994": [0, 28], "10556789408805572": [0, 28], "multistagecheckpointschedul": [0, 5], "strategi": [0, 42], "andrea": [0, 5, 42, 48], "griewank": [0, 5, 42, 48], "walther": [0, 5, 42, 48], "799": [0, 5, 42, 48], "revolv": [0, 5, 6, 42, 44, 48], "mode": [0, 5, 42, 48], "acm": [0, 5, 42, 48], "transact": [0, 5, 42, 48], "mathemat": [0, 5, 42, 48], "26": [0, 5, 42, 48], "19": [0, 5, 42, 48], "45": [0, 5, 42, 48], "2000": [0, 5, 42, 48], "1145": [0, 5, 42, 48], "347837": [0, 5, 42, 48], "347846": [0, 5, 42, 48], "determin": [0, 5, 20, 22, 26, 42], "memori": [0, 3, 5, 6, 38, 40, 42, 45, 46, 48, 50], "disk": [0, 5, 6, 7, 8, 9, 10, 11, 12, 40, 42], "storag": [0, 5, 6, 7, 8, 9, 10, 11, 12, 42, 45, 46, 48], "initi": [0, 1, 4, 5, 6, 8, 10, 11, 12, 14, 15, 18, 19, 20, 22, 24, 25, 26, 28, 32, 34, 35, 37, 38, 39, 40, 42, 45, 46, 48, 50, 51], "run": [0, 5, 10, 11, 30, 42, 43, 46], "lead": [0, 33, 39, 42, 46, 47, 48], "equival": [0, 5, 40, 42], "philipp": [0, 5, 42], "stumm": [0, 5, 42], "multistag": [0, 5, 42, 48], "offlin": [0, 5, 6, 8, 11, 42], "31": [0, 5, 42, 45, 48], "1946": [0, 5, 42], "1967": [0, 5, 42], "2009": [0, 5, 42], "080718036": [0, 5, 42], "twolevelcheckpointschedul": [0, 5], "two": [0, 5, 6, 14, 15, 17, 18, 19, 22, 23, 24, 25, 27, 33, 35, 37, 40, 45, 46, 47, 48, 49], "period": [0, 5, 42, 50], "gavin": [0, 5], "pringl": [0, 5], "jone": [0, 5], "sudipta": [0, 5], "goswami": [0, 5], "sri": [0, 5], "hari": [0, 5], "krishna": [0, 5], "narayanan": [0, 5], "archer": [0, 5], "commun": [0, 5, 12, 20, 22, 26, 27, 32, 33, 34, 38, 39, 42], "tool": [0, 5, 53], "perform": [0, 5, 9, 14, 17, 18, 19, 23, 24, 25, 32, 33, 43, 45, 46, 47, 48], "oceanograph": [0, 5], "cryospher": [0, 5], "version": [0, 5, 6, 22, 42, 51, 53], "epcc": [0, 5], "2016": [0, 5], "support": [0, 5, 53], "t": [0, 3, 5, 15, 27, 30, 45, 46, 48, 51, 52], "smith": [0, 5], "h": [0, 3, 5, 6, 27, 30, 38, 40, 44, 45, 46, 47, 49], "k": [0, 1, 5, 43, 47, 51], "heimbach": [0, 5], "morlighem": [0, 5], "bathymetr": [0, 5], "influenc": [0, 5], "antarct": [0, 5, 27], "ic": [0, 5, 12, 14, 27, 35, 51], "shelf": [0, 5], "melt": [0, 5], "rate": [0, 5, 43], "geophys": [0, 5], "research": [0, 5], "ocean": [0, 5], "125": [0, 5], "11": [0, 5, 45, 46, 47, 48, 51], "e2020jc016370": [0, 5], "2020": [0, 5], "1029": [0, 5], "2020jc016370": [0, 5], "file": [0, 12, 40], "jorg": [0, 38], "noced": [0, 38], "stephen": [0, 38], "wright": [0, 38], "numer": [0, 38], "springer": [0, 38], "new": [0, 4, 5, 14, 15, 17, 19, 22, 23, 25, 30, 33, 34, 36, 38, 39, 41, 42, 43, 45, 46, 47, 48, 51], "york": [0, 38], "ny": [0, 38], "2006": [0, 38], "second": [0, 6, 33, 38, 43, 46, 48, 49, 50, 52], "edit": [0, 38], "1007": [0, 38], "978": [0, 38], "387": [0, 38], "40065": [0, 38], "richard": [0, 38], "byrd": [0, 38], "peihuang": [0, 38], "lu": [0, 19, 38, 48], "ciyou": [0, 38], "zhu": [0, 38], "bound": [0, 33, 38], "16": [0, 38, 48, 51], "1190": [0, 38], "1208": [0, 38], "1995": [0, 38], "0916069": [0, 38], "earli": 0, "work": [0, 48, 51], "conduct": 0, "part": [0, 13, 33, 43, 53], "u": [0, 22, 27, 46, 47, 48, 49, 50, 51, 52], "natur": [0, 47], "environ": 0, "council": 0, "project": [0, 18, 19, 24, 25, 38], "ne": [0, 50], "l005166": 0, "further": [0, 22, 45, 46], "ha": [0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 45, 46, 48, 51], "been": [0, 1, 11, 12, 13, 22, 33, 38, 41, 42, 46, 48], "physic": [0, 27], "scienc": 0, "ep": [0, 13, 27, 45, 46, 48, 49, 50, 51], "r021600": 0, "t001607": 0, "adjointrh": [1, 14, 18, 24, 28, 34, 39], "x": [1, 3, 4, 12, 14, 15, 18, 19, 20, 21, 22, 24, 25, 26, 28, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 45, 46, 47, 48, 49, 50, 51, 52], "hand": [1, 14, 15, 17, 18, 19, 22, 23, 24, 25, 27, 28, 32, 33, 34, 35, 37, 39, 40, 43, 45, 46, 47, 48, 51], "side": [1, 14, 15, 17, 18, 19, 22, 23, 24, 25, 27, 28, 32, 33, 34, 35, 37, 39, 40, 43, 45, 46, 47, 48, 51], "variabl": [1, 3, 4, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24, 25, 26, 28, 30, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 45, 46, 48, 50, 51], "associ": [1, 3, 4, 6, 11, 12, 15, 16, 19, 20, 21, 22, 26, 28, 30, 33, 34, 35, 39, 40, 41, 42, 43, 45, 46, 48, 49, 51, 53], "solv": [1, 5, 6, 12, 13, 14, 15, 18, 19, 22, 24, 25, 27, 28, 29, 32, 34, 35, 37, 38, 39, 40, 42, 45, 46, 47, 48, 49, 50, 51], "paramet": [1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 48], "b": [1, 13, 14, 15, 18, 19, 22, 24, 25, 27, 28, 30, 32, 34, 35, 37, 39, 40, 51], "copi": [1, 12, 22, 33, 50, 52], "fals": [1, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 20, 21, 26, 33, 34, 35, 36, 39, 40, 42, 48, 51], "return": [1, 2, 3, 4, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52], "If": [1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 18, 20, 22, 24, 25, 26, 32, 33, 35, 38, 39, 40, 41, 42, 43, 46, 47, 48, 50, 51], "true": [1, 3, 11, 12, 14, 16, 20, 21, 22, 26, 27, 28, 32, 33, 34, 35, 36, 38, 40, 41, 42, 45, 46, 48, 50, 51, 52], "intern": [1, 12, 19, 30, 38, 42, 43, 51, 52], "store": [1, 3, 4, 5, 6, 7, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 30, 32, 33, 34, 35, 37, 38, 39, 40, 42, 45, 46, 48, 51], "valu": [1, 3, 4, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 32, 33, 34, 35, 37, 38, 39, 40, 42, 43, 45, 46, 47, 48, 49, 50, 51], "itself": [1, 22, 33, 46], "alloc": 1, "typic": [1, 14, 15, 18, 19, 22, 24, 25, 33, 34, 35, 36, 37, 39, 40, 42, 43, 50, 51], "need": [1, 12, 14, 33, 35, 40, 41, 42, 45, 46, 48, 49, 50, 51], "call": [1, 2, 4, 11, 13, 22, 27, 30, 33, 36, 38, 42, 46, 48], "manual": [1, 12, 13, 22, 27, 38, 42, 53], "sub": [1, 22, 35], "subtract": [1, 14, 15, 18, 24, 28, 33, 34, 35, 39], "term": [1, 14, 15, 17, 18, 19, 22, 23, 24, 25, 27, 28, 30, 34, 35, 37, 39, 40, 43, 46, 47], "subtract_adjoint_derivative_act": [1, 14, 15, 18, 19, 24, 25, 28, 33, 34, 35, 37, 39, 40], "is_empti": 1, "empti": [1, 4, 22], "mean": [1, 16, 20, 21, 26, 33, 34, 39, 43, 45], "adjointequationrh": 1, "eq": [1, 12, 18, 24, 28, 42, 46, 48], "multipl": [1, 4, 5, 6, 7, 8, 9, 10, 11, 13, 18, 22, 24, 42, 45, 53], "object": [1, 2, 4, 12, 15, 20, 22, 26, 27, 28, 33, 34, 35, 39, 42, 45, 46, 48, 50], "mai": [1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 32, 33, 34, 35, 37, 38, 39, 40, 42, 46, 47, 48, 51], "access": [1, 11, 12, 22, 33, 34, 36, 39, 45, 51], "index": [1, 12, 42, 52, 53], "g": [1, 5, 6, 11, 12, 18, 20, 22, 24, 26, 33, 35, 39, 41, 42, 43, 46, 49, 51, 52], "adj_eq_rh": 1, "adj_rh": 1, "For": [1, 11, 13, 17, 23, 33, 34, 39, 42, 43, 45, 46, 51, 53], "case": [1, 6, 9, 19, 22, 25, 27, 33, 43, 45, 47, 48, 49, 50, 51, 52], "where": [1, 4, 7, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 22, 23, 24, 25, 27, 30, 33, 34, 35, 36, 37, 38, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52], "singl": [1, 11, 13, 14, 15, 18, 19, 22, 24, 25, 27, 28, 30, 32, 34, 35, 37, 38, 39, 40, 42, 45, 47, 48], "themselv": 1, "tupl": [1, 3, 4, 6, 12, 13, 14, 15, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 30, 32, 33, 34, 35, 37, 38, 39, 40, 42], "adjointblockrh": 1, "block": [1, 12, 19, 22, 25, 42, 45, 46, 48], "adj_block_rh": 1, "sequenc": [1, 3, 4, 12, 14, 15, 18, 19, 22, 24, 25, 27, 28, 30, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43], "pop": [1, 47, 48], "remov": [1, 4, 12, 42], "last": [1, 11, 33], "adjointmodelrh": 1, "adj_model_rh": 1, "contain": [1, 4, 12, 13, 22, 33, 41, 45, 46, 51], "each": [1, 13, 14, 18, 22, 24, 28, 34, 39, 42, 43, 46, 47, 48, 50, 51], "map": [1, 12, 14, 17, 18, 19, 22, 23, 24, 25, 28, 33, 34, 39, 41, 42, 49, 51, 53], "item": [1, 12, 14, 18, 22, 24, 28, 34, 39, 42], "int": [1, 4, 12, 13, 14, 15, 18, 19, 20, 24, 25, 26, 28, 33, 34, 35, 37, 40, 41, 42, 43, 50], "In": [1, 11, 18, 24, 30, 33, 43, 45, 46, 48, 49, 50, 51], "latter": [1, 46], "gc_disabl": 2, "fn": [2, 4, 33, 34, 36, 39], "decor": [2, 4, 33, 36, 39], "disabl": [2, 3, 12, 33, 36, 39, 42, 45, 48], "garbag": [2, 32, 33], "collector": [2, 33], "callabl": [2, 4, 13, 22, 27, 28, 30, 33, 34, 36, 38, 39, 42, 43], "which": [2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 30, 32, 33, 34, 35, 36, 38, 39, 40, 42, 43, 46, 47, 49, 50, 51], "should": [2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 17, 18, 19, 20, 22, 23, 24, 25, 26, 28, 29, 32, 33, 34, 35, 37, 38, 39, 40, 42, 43, 46, 47, 51], "obj": [2, 33], "hold": [2, 42], "refer": [2, 3, 4, 5, 12, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24, 25, 26, 28, 33, 34, 35, 39, 42, 43], "weakalia": [2, 42], "doe": [2, 14, 18, 22, 24, 35, 40, 41, 46, 48, 51], "intend": [2, 14, 39, 45], "combin": [2, 53], "weakref": 2, "final": [2, 11, 42, 47, 48], "so": [2, 11, 14, 18, 19, 22, 24, 25, 33, 35, 38, 40, 42, 45, 46, 48, 49, 51], "attribut": [2, 33, 51], "updat": [2, 4, 12, 14, 15, 18, 22, 24, 27, 28, 33, 34, 35, 38, 39, 46, 52], "when": [2, 5, 10, 12, 18, 22, 24, 33, 40, 42, 45, 46, 48, 49, 51], "destroi": [2, 33, 42], "still": [2, 45, 46, 48], "after": [2, 4, 11, 13, 17, 22, 23, 27, 38, 39, 42, 46], "cachedhessian": [3, 46, 47], "manag": [3, 14, 30, 31, 33, 34, 38, 39, 42, 43, 45, 46, 48, 51, 52], "none": [3, 4, 7, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 51], "cache_adjoint": 3, "repres": [3, 14, 15, 18, 19, 20, 21, 22, 24, 25, 26, 30, 33, 35, 37, 39, 46, 48, 49], "hessian": [3, 27, 38, 43, 45, 47, 51, 53], "given": [3, 4, 19, 22, 25, 28, 30, 33, 38, 45, 46, 47, 49, 50, 51, 53], "equationmanag": [3, 14, 15, 18, 19, 24, 25, 28, 29, 30, 32, 34, 35, 36, 38, 39, 40, 42, 43], "record": [3, 14, 15, 18, 24, 29, 32, 34, 39, 42, 45, 46, 48, 51, 52], "have": [3, 15, 16, 18, 20, 21, 24, 26, 33, 42, 43, 45, 46, 47, 48, 49, 50, 51], "checkpoint": [3, 5, 6, 7, 8, 9, 10, 11, 16, 20, 21, 26, 33, 34, 39, 42, 45, 46, 51, 52, 53], "drop": [3, 14, 15, 18, 24, 25, 28, 35, 42], "suppli": [3, 4, 11, 12, 13, 14, 15, 18, 19, 20, 22, 24, 25, 26, 28, 30, 32, 33, 34, 35, 38, 39, 40, 43, 45, 47, 48, 51], "first": [3, 6, 12, 22, 33, 38, 40, 42, 43, 45, 46, 48, 50, 51, 52], "adjoint": [3, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 18, 19, 24, 25, 28, 30, 32, 33, 34, 35, 37, 39, 40, 42, 43, 47, 50, 51, 52, 53], "compute_gradi": [3, 30, 36, 42, 45, 46, 47, 48, 49, 51], "m0": [3, 30, 38, 43], "conjug": [3, 13, 16, 20, 21, 26, 27, 30, 33, 34, 38, 39, 42], "respect": [3, 13, 14, 15, 18, 19, 22, 24, 25, 28, 30, 32, 34, 35, 37, 38, 39, 40, 41, 42, 43, 45, 46, 48, 49, 51, 52, 53], "control": [3, 11, 14, 15, 18, 19, 24, 25, 27, 28, 30, 32, 34, 35, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51], "type": [3, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 30, 33, 34, 35, 37, 38, 39, 40, 42, 45], "action": [3, 11, 14, 15, 18, 19, 22, 24, 25, 27, 30, 35, 37, 38, 40, 43, 45, 46, 47, 48, 51, 53], "dm": [3, 14, 15, 18, 19, 24, 25, 28, 30, 32, 34, 35, 39, 40, 41, 43, 45, 47, 51], "some": [3, 5, 6, 7, 8, 9, 10, 11, 30, 33, 38, 39, 43, 44, 46, 47, 49, 51, 53], "zeta": [3, 30, 43, 45, 46, 47, 48, 49, 50, 51], "consid": [3, 30, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52], "row": [3, 30, 45], "vector": [3, 15, 18, 22, 28, 30, 33, 34, 38, 43, 45, 49, 52], "left": [3, 13, 14, 15, 22, 27, 30, 35, 37, 38, 39, 40, 43, 45, 46, 47, 48, 49, 50, 51, 52], "frac": [3, 15, 30, 35, 38, 45, 47, 49, 50, 51], "mathcal": [3, 14, 15, 18, 19, 24, 25, 30, 35, 37, 39, 40, 49, 51], "dj": [3, 30, 43, 47, 51], "ddj": [3, 30, 43, 47], "result": [3, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 30, 33, 34, 35, 38, 39, 40, 45, 46, 47, 48, 49, 50, 51, 53], "cachedgaussnewton": 3, "r_inv_act": [3, 30], "b_inv_act": [3, 27, 30], "gauss": [3, 30], "newton": [3, 30], "approxim": [3, 18, 24, 27, 30, 38, 45, 46], "state": [3, 4, 30, 33, 42, 45, 46, 51], "see": [3, 5, 6, 11, 12, 13, 18, 24, 27, 28, 30, 33, 36, 39, 42, 43, 45, 46, 47, 48], "gaussnewton": [3, 30], "cacheref": [4, 17, 19, 23, 25], "entri": [4, 36, 46], "can": [4, 12, 14, 15, 18, 19, 20, 22, 24, 25, 26, 28, 30, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 51, 52, 53], "later": [4, 11, 27, 38, 48], "clear": [4, 11, 12, 42, 45, 46], "referenc": [4, 42, 45, 46], "indic": [4, 11, 12, 14, 15, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 45, 48, 51, 52], "clear_cach": [4, 46, 48], "dep": [4, 12, 14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 39, 40, 51], "onli": [4, 12, 14, 15, 18, 19, 24, 25, 27, 28, 32, 33, 34, 35, 39, 40, 42, 46, 47, 48, 51], "local_cach": 4, "befor": [4, 5, 13, 22, 27, 38, 45, 46, 49], "properti": [4, 5, 6, 7, 8, 9, 10, 11, 12, 22, 34, 39, 40, 41], "id": [4, 12, 33, 41, 42, 45, 46], "uniqu": [4, 33, 41, 49, 51], "add": [4, 12, 15, 22, 29, 33, 34, 35, 38, 39, 42, 48, 51], "kei": [4, 12, 33, 40, 45, 46, 47, 48], "take": [4, 38, 43, 46, 47, 48, 51], "argument": [4, 5, 6, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 29, 30, 32, 34, 35, 37, 38, 39, 40, 42, 43, 45, 47, 48, 51], "exist": 4, "value_ref": [4, 17, 19, 23, 25], "get": [4, 53], "arg": [4, 11, 15, 16, 18, 20, 21, 22, 24, 26, 29, 33, 36, 42], "default": [4, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 33, 34, 35, 36, 38, 39, 42, 46, 47, 48], "zero": [4, 9, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 32, 33, 34, 35, 37, 39, 40, 41, 42, 43, 46, 47, 48, 49, 51, 52], "one": [4, 5, 6, 8, 10, 11, 14, 17, 22, 23, 28, 30, 33, 34, 38, 42, 43, 46, 48, 50, 51], "except": [4, 33, 38, 48], "rais": [4, 11, 33, 51], "also": [4, 12, 33, 34, 39, 42, 43, 45, 46, 47, 48, 52, 53], "chang": [4, 20, 26, 28, 33, 38, 45, 46, 47, 48], "either": [4, 5, 8, 11, 12, 14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 37, 39, 40, 42, 47, 51], "current": [4, 11, 14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 36, 39, 40, 42], "invalid": [4, 14, 33, 46], "check": [4, 21, 27, 33, 45, 48, 50, 51], "potenti": [4, 22], "max_n": [5, 6, 7, 8, 11], "snapshots_in_ram": [5, 6], "snapshots_on_disk": [5, 6], "trajectori": 5, "maximum": [5, 6, 19, 22, 27, 28, 38, 42, 48], "differenti": [5, 8, 11, 14, 15, 18, 19, 24, 25, 28, 34, 35, 37, 39, 40, 42, 45, 46, 47, 48, 49, 50, 51, 52, 53], "hereaft": 5, "gw2000": 5, "between": [5, 15, 19, 22, 33, 42, 43, 46, 47, 48, 50], "ram": [5, 6, 8, 11, 48], "number": [5, 6, 8, 11, 13, 14, 15, 18, 19, 22, 24, 25, 28, 33, 34, 35, 37, 38, 39, 40, 42, 43, 47, 48, 49], "step": [5, 6, 8, 10, 11, 38, 42, 46, 48, 52], "advanc": [5, 6, 8, 10, 11, 12, 48, 50], "unit": [5, 8, 11, 47, 50, 51, 52], "avail": [5, 8, 14, 33, 44, 46, 53], "gener": [5, 6, 7, 8, 9, 10, 11, 13, 27, 32, 38, 45, 46, 47, 52, 53], "solut": [5, 12, 13, 14, 15, 18, 19, 22, 24, 25, 27, 28, 29, 32, 34, 35, 36, 37, 38, 39, 40, 42, 46, 47, 48, 49, 50, 51, 52, 53], "problem": [5, 18, 19, 22, 24, 25, 27, 53], "fig": [5, 52], "select": 5, "standard": 5, "specifi": 5, "bottom": 5, "34": [5, 48], "correspond": [5, 6, 13, 14, 15, 18, 19, 22, 24, 25, 28, 30, 32, 33, 34, 35, 38, 39, 40, 42, 43, 45, 46, 47], "size": [5, 8, 43, 50, 52], "compat": [5, 22, 33], "region": 5, "name": [5, 6, 16, 20, 21, 22, 26, 33, 34, 39, 42, 43, 45, 46, 47, 48, 49, 50, 51], "snaps_in_ram": [5, 6, 42, 48], "snaps_on_disk": [5, 6, 42], "dolfin_adjoint": [5, 6, 42], "adj_checkpoint": [5, 6, 42], "dolfin": [5, 6, 16, 18, 19, 20, 24, 42, 43, 44], "2017": [5, 6, 18, 24, 28, 42, 43], "is_exhaust": [5, 6, 7, 8, 9, 10, 11], "conclud": [5, 6, 7, 8, 9, 10, 11], "note": [5, 6, 7, 8, 9, 10, 11, 12, 30, 33, 35, 38, 45, 48, 50, 51, 52], "never": [5, 6, 7, 8, 9, 10, 11, 46], "uses_disk_storag": [5, 6, 7, 8, 9, 10, 11], "requir": [5, 6, 7, 8, 9, 10, 11, 14, 17, 22, 23, 27, 28, 35, 38, 40, 42, 43, 46, 47, 51], "overridden": [5, 6, 7, 8, 9, 10, 11, 14, 18, 24, 28, 34, 39, 42], "binomial_snapshot": 5, "binomial_storag": 5, "binomial_trajectori": 5, "onlin": [5, 7, 9, 10, 11], "unlimit": [5, 7, 10], "everi": [5, 10], "addit": [5, 15, 19, 25, 33, 34, 35, 39, 48], "constructor": [5, 16, 18, 19, 20, 21, 24, 25, 26, 34, 39, 51], "hrevolvecheckpointschedul": 6, "wvect": 6, "rvect": 6, "uf": 6, "ub": 6, "2": [6, 28, 32, 43, 45, 46, 47, 48, 49, 50, 51, 52], "kwarg": [6, 16, 18, 20, 21, 22, 24, 26, 27, 29, 33, 38], "convert": [6, 15, 16, 18, 21, 24, 34, 39, 46, 48, 51], "librari": [6, 12, 33, 40, 42, 53], "write": [6, 11, 12], "cost": [6, 48, 50], "save": [6, 11, 40, 48], "read": [6, 11, 12], "load": [6, 11, 12, 40, 48], "bf": 6, "over": [6, 8, 11, 51, 52, 53], "remain": [6, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 34, 38], "keyword": [6, 27, 38, 42], "pass": [6, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 34, 35, 37, 38, 40, 42, 51], "hrevolv": 6, "memorycheckpointschedul": 7, "mixedcheckpointschedul": 8, "snapshot": 8, "assum": [8, 14, 15, 18, 19, 24, 25, 27, 28, 32, 34, 35, 39, 40, 46, 47, 48, 49, 50, 51], "same": [8, 15, 22, 33, 34, 39, 42, 45, 46, 47, 49, 51], "locat": [8, 11, 19], "nonecheckpointschedul": 9, "periodicdiskcheckpointschedul": 10, "recomput": 10, "re": [10, 30, 43, 46], "greater": [10, 47], "than": [10, 43, 46, 47], "checkpointact": 11, "clear_ic": [11, 12], "clear_data": [11, 12], "intermedi": 11, "configur": [11, 12, 13, 22, 27, 38, 42, 45, 46, 48], "store_": [11, 12], "store_data": [11, 12], "n0": [11, 12, 34], "n1": [11, 12, 34], "start": [11, 36, 42, 53], "delet": [11, 12, 48], "endforward": 11, "end": [11, 42], "endrevers": 11, "exhaust": 11, "checkpointschedul": [11, 42], "dispatch": 11, "functool": 11, "singledispatch": 11, "def": [11, 22, 34, 36, 38, 42, 45, 46, 47, 48, 49, 50, 51, 52], "cp_action": 11, "typeerror": 11, "f": [11, 14, 15, 18, 19, 24, 25, 33, 35, 37, 38, 39, 40, 45, 46, 48, 49, 50, 51, 52], "unexpect": [11, 33], "regist": 11, "action_forward": 11, "logger": [11, 43, 47, 48], "debug": [11, 48], "cp_schedul": 11, "isinst": 11, "break": 11, "buffer": [11, 12], "immedi": 11, "detail": [11, 13, 22, 27, 33, 38, 42, 48], "known": [11, 46], "instanti": [11, 33, 39, 41, 42, 46, 48], "unknown": 11, "abstract": [11, 12, 14, 22, 30, 35, 40], "execut": 11, "far": 11, "yet": 11, "is_run": 11, "least": 11, "checkpointstorag": 12, "three": [12, 45, 46], "var_is_stat": [12, 33], "These": [12, 45], "overlap": 12, "exampl": [12, 22, 33, 39, 43, 49, 50, 51], "within": 12, "via": [12, 13, 14, 22, 27, 30, 33, 38, 41, 43, 45, 46, 47, 50, 51, 52], "nl_dep": [12, 14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 37, 39, 40, 51], "cp": 12, "here": [12, 22, 36, 43, 45, 46, 47, 48, 49, 50, 51, 52], "nonlinear_depend": [12, 14, 35], "enabl": [12, 18, 24, 33, 39, 42, 45, 48, 52, 53], "clear_ref": 12, "initial_condit": 12, "ref": 12, "variablestatelockdictionari": [12, 33], "x_id": 12, "x_valu": 12, "add_initial_condit": [12, 42], "update_kei": 12, "keep": [12, 48], "those": [12, 30, 33, 42], "allow": [12, 22, 39, 42, 46, 51, 52, 53], "add_equ": [12, 42], "extract": 12, "add_equation_data": 12, "As": [12, 22, 45, 48, 51], "checkpoint_data": 12, "prior": [12, 14, 36, 42], "dict": [12, 40], "x_indic": 12, "whose": [12, 14, 15, 18, 20, 22, 24, 26, 28, 33, 34, 39, 40, 42, 43, 46, 48, 50], "compon": [12, 14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 37, 39, 40, 41, 42, 45], "being": [12, 38, 43, 45, 46, 50, 51], "replaystorag": 12, "transpose_dep": 12, "replay_storag": 12, "set": [12, 13, 14, 15, 17, 18, 19, 22, 23, 24, 25, 27, 28, 32, 33, 34, 35, 36, 38, 39, 40, 43, 46, 50, 51, 52], "test": [12, 22, 27, 28, 41, 43, 46, 48, 49, 50, 51, 52, 53], "rang": [12, 34, 47, 48, 50, 51, 52], "transposecomputationalgraph": 12, "activ": [12, 14, 15, 18, 19, 24, 25, 28, 32, 33, 34, 35, 39, 40, 42], "analysi": [12, 42], "is_act": 12, "popleft": 12, "dealloc": 12, "longer": 12, "progress": 12, "written": 12, "ic_id": 12, "picklecheckpoint": 12, "prefix": [12, 33], "comm": [12, 20, 22, 26, 27, 32, 33, 34, 38, 39, 42, 51], "pickl": [12, 42], "_": [12, 38, 43, 46, 49, 50, 51, 52], "root_pid": 12, "root_py2f": 12, "rank": [12, 22, 27], "root": [12, 38, 42, 46], "fortran": 12, "mpi": [12, 33], "hdf5checkpoint": 12, "h5py": [12, 40, 42, 44], "hdf5": [12, 42], "space": [13, 14, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 29, 33, 34, 35, 38, 39, 46, 47, 48, 49, 50, 51, 52], "a_act": 13, "b_action": [13, 27], "arg_space_typ": 13, "primal": [13, 14, 16, 20, 21, 26, 33, 34, 35, 38, 39, 51], "action_space_typ": 13, "n_eigenvalu": 13, "solver_typ": [13, 19], "problem_typ": [13, 27], "toler": [13, 19, 22, 25, 28, 38, 43, 47, 50], "1e": 13, "12": [13, 45, 46, 47, 48, 49, 50, 51], "pre_callback": [13, 22, 27, 38], "post_callback": [13, 22, 38, 50], "interfac": [13, 18, 22, 24, 27, 31, 34, 36, 38], "slepc": [13, 27, 44], "matrix": [13, 15, 17, 18, 19, 22, 23, 24, 25, 27, 35, 38, 46, 48, 49, 50], "free": [13, 27], "eigenproblem": [13, 27], "v": [13, 20, 22, 26, 27, 34, 46, 47, 49, 50, 51], "lambda": [13, 27, 35, 45, 46, 51, 52], "eigenvector": [13, 27], "accept": [13, 22, 27, 28, 30, 38, 42, 43, 51], "input": [13, 22, 30, 33, 34, 37, 38, 51], "dual": [13, 16, 20, 26, 33, 34, 38, 39, 46, 49, 50, 51], "conjugate_du": [13, 14, 16, 20, 21, 26, 33, 34, 35, 39, 51], "eigenvalu": 13, "attempt": [13, 50], "dimens": [13, 19, 25, 48, 52], "problemtyp": [13, 27], "gnhep": [13, 27], "nhep": 13, "largest_magnitud": 13, "converg": [13, 22, 28, 38, 43, 50, 51, 52, 53], "By": [13, 33, 38, 42, 47], "criterion": [13, 38], "conv": 13, "rel": [13, 14, 22, 28, 33, 35, 38], "option": [13, 22, 27, 38, 42], "setup": [13, 22, 27], "lam": [13, 27], "numpi": [13, 19, 25, 33, 34, 39, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52], "ndarrai": [13, 19, 25, 33], "hermitian": [13, 22, 38], "real": [13, 27, 34, 39, 43, 46, 47, 48, 49, 50, 51], "build": [13, 19, 46, 47, 48, 49, 50, 51], "petsc": [13, 22, 32, 33, 38, 44], "v_r": 13, "v_i": 13, "complex": [13, 14, 15, 18, 19, 24, 25, 27, 33, 34, 35, 37, 39, 40, 43], "ic_dep": 14, "adj_ic_dep": 14, "adj_ic": [14, 35, 36, 42, 51], "adj_typ": [14, 35, 51], "core": [14, 18, 19, 20, 24, 25, 26, 39, 42], "oper": [14, 15, 18, 19, 24, 25, 27, 28, 30, 32, 33, 34, 35, 38, 39, 40, 49, 52, 53], "tape": [14, 15, 32, 42], "residu": [14, 15, 18, 19, 24, 25, 35, 37, 39, 40, 51], "implicitli": 14, "y_0": 14, "y_1": [14, 15, 35, 39], "ldot": [14, 15, 35, 39, 43], "y_i": [14, 15, 35], "overload": [14, 35, 39, 40], "inherit": [14, 33, 35, 40, 51], "abc": [14, 35, 40], "superset": 14, "subset": [14, 25], "guess": [14, 15, 18, 19, 22, 24, 25, 28, 32, 34, 35, 37, 38, 39, 40, 50, 51], "equal": [14, 21, 35, 38, 42, 43, 46, 48, 49, 51], "drop_refer": [14, 15, 18, 24, 25, 28, 35, 42, 51], "th": [14, 15, 22, 35, 42, 43, 51], "initial_condition_depend": 14, "adjoint_initial_condition_depend": 14, "adj_x_typ": [14, 51], "exactli": [14, 22], "One": [14, 22, 28, 33], "new_adj_x": 14, "suitabl": [14, 27], "annot": [14, 18, 24, 29, 34, 36, 39, 42, 45, 46], "tlm": [14, 18, 24, 29, 34, 36, 39, 42], "wrap": [14, 52], "forward_solv": [14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 39, 40], "handl": [14, 33, 42, 45, 46], "paus": [14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 36, 39, 40, 42], "subclass": [14, 15, 18, 19, 24, 25, 28, 32, 33, 34, 35, 37, 39, 40, 42, 51], "replac": [14, 15, 18, 19, 20, 24, 25, 26, 28, 32, 33, 34, 35, 37, 39, 40, 42, 47], "modifi": [14, 15, 18, 19, 22, 24, 25, 28, 32, 34, 35, 37, 38, 39, 40, 45, 46, 48, 50], "self": [14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 37, 39, 40, 46, 51], "adj_x": [14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 37, 39, 40, 42, 51], "dep_b": [14, 18, 24, 28, 34, 39], "dep_index": [14, 15, 18, 19, 24, 25, 28, 34, 35, 37, 39, 40, 51], "adjoint_cach": 14, "adjoint_derivative_act": [14, 15, 18, 19, 24, 25, 28, 33, 34, 35, 37, 39, 40], "neg": [14, 15, 18, 19, 24, 25, 35, 37, 40], "Will": [14, 15, 18, 19, 24, 25, 35, 37, 40], "valid": [14, 15, 18, 19, 22, 24, 25, 33, 35, 37, 40], "upon": [14, 15, 18, 19, 24, 25, 33, 35, 37, 40], "alpha": [14, 15, 18, 19, 22, 24, 25, 33, 35, 37, 40, 47, 49, 50], "product": [14, 15, 18, 19, 24, 25, 33, 35, 37, 38, 40, 49, 50, 51], "adjoint_jacobian_solv": [14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 37, 39, 40], "tangent_linear": [14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 39, 40], "tlm_map": [14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 39, 40, 41, 51], "tangentlinearmap": [14, 15, 18, 19, 24, 25, 28, 32, 34, 35, 39, 40, 41], "zeroassign": [14, 51], "assign": [14, 15, 18, 24, 25, 29, 33, 34, 35, 39, 40, 46, 47, 48, 49, 50, 51, 52], "nullsolv": 14, "emptyequ": 15, "y": [15, 16, 18, 19, 21, 22, 24, 25, 29, 33, 34, 38, 39, 45, 46, 47, 48, 49, 50, 51, 52], "convers": 15, "degre": [15, 19, 22, 25, 28, 33, 34, 38, 42, 43, 46, 47, 48, 49, 51], "freedom": [15, 19, 22, 25, 28, 33, 34, 38, 43, 46, 48, 49, 51], "tild": [15, 22, 50, 51], "differ": [15, 22, 33, 43, 46, 47, 52, 53], "linearcombin": 15, "sum_i": [15, 35, 51], "alpha_i": 15, "y_2": [15, 35, 39], "consist": [15, 18, 22, 24, 27, 28, 38, 46, 49], "scalar": [15, 19, 22, 25, 33, 34, 39, 43, 47, 49, 50, 51], "axpi": 15, "y_new": 15, "y_old": 15, "y_": 15, "text": [15, 30, 37, 40, 43, 46, 47, 50, 51, 52], "old": 15, "dotproduct": 15, "z": [15, 45], "innerproduct": 15, "linear_equ": 15, "ident": [15, 18, 19, 22, 24, 25, 35, 38, 51], "matrixactionrh": 15, "add_forward": [15, 35], "rh": [15, 18, 19, 24, 25, 35, 48], "deps_index": [15, 35], "tangent_linear_rh": [15, 35], "construct": [15, 17, 19, 22, 23, 27, 33, 34, 35, 39, 42, 48, 49, 50, 51], "obtain": [15, 35, 39, 49], "That": [15, 35, 45, 47], "tau_": [15, 35], "dotproductrh": 15, "innerproductrh": 15, "space_typ": [16, 20, 21, 26, 33, 34, 39], "static": [16, 20, 21, 26, 33, 34, 37, 39, 41, 48], "extend": [16, 20, 21, 26], "replai": [16, 20, 21, 26, 33, 34, 39, 42], "involv": [16, 20, 21, 26, 27, 33, 34, 39, 45, 46], "zerofunct": [16, 21, 48], "flag": [16, 20, 21, 26, 33], "togeth": [16, 20, 21, 26, 51], "to_fen": 16, "str": [16, 20, 21, 26, 33, 34, 39, 40, 42], "assemblycach": [17, 23], "assembl": [17, 18, 22, 23, 24, 25, 46, 47, 48, 50, 51, 53], "bc": [17, 18, 22, 23, 24, 26], "form_compiler_paramet": [17, 18, 19, 23, 24, 25], "linear_solver_paramet": [17, 23], "replace_map": [17, 19, 23, 25], "previous": [17, 19, 23, 25, 33, 45, 46], "ufl": [17, 18, 19, 20, 22, 23, 24, 25, 26, 44, 51], "dirichlet": [17, 18, 20, 22, 23, 24, 26, 46, 49, 52], "boundari": [17, 18, 20, 22, 23, 24, 26, 46, 47, 48, 49, 50, 51, 52], "compil": [17, 18, 23, 24, 25], "appear": [17, 23, 33, 43, 45, 46], "symbol": [17, 18, 19, 20, 21, 23, 24, 25, 26, 33, 39, 42, 51], "ariti": [17, 18, 19, 23, 24, 25], "b_bc": [17, 23], "ad": [17, 22, 23, 33, 42, 53], "homogen": [17, 22, 23, 46, 49, 52], "linearsolvercach": [17, 23], "linear_solv": [17, 22, 23], "assembly_cach": [17, 23, 46, 48], "set_assembly_cach": [17, 23], "linear_solver_cach": [17, 23, 46, 48], "set_linear_solver_cach": [17, 23], "equationsolv": [18, 19, 24, 25, 48], "variat": [18, 19, 24, 25, 30, 46, 48, 51], "match_quadratur": [18, 19, 24, 25], "quadratur": [18, 24, 47], "solver_paramet": [18, 22, 24, 28, 46, 48, 50, 51], "adjoint_solver_paramet": [18, 24], "tlm_solver_paramet": [18, 24], "cache_jacobian": [18, 19, 24, 25, 48], "cache_adjoint_jacobian": [18, 24], "cache_tlm_jacobian": [18, 24], "cache_rhs_assembli": [18, 19, 24, 25, 48], "jacobian": [18, 24, 30], "autodetect": [18, 20, 24, 26], "divid": [18, 24], "expr_new_x": [18, 24], "expr": [18, 19, 20, 24, 25, 26, 39], "x_old": [18, 24], "linear_equation_new_x": [18, 24], "onto": [18, 19, 24, 25, 38], "dirichletbcappl": [18, 24], "applic": [18, 22, 24, 27], "specif": [18, 24, 43, 45, 46, 47, 48, 49], "dirichletbc": [18, 20, 22, 24, 26, 46, 49], "function_spac": [18, 24, 50, 51], "dirichet": [18, 24], "exprinterpol": [18, 24], "interpol": [18, 19, 24, 25, 46, 47, 48, 49, 50, 51, 52], "localsolvercach": [19, 25], "wise": [19, 25], "local": [19, 22, 25, 33, 34], "diagon": [19, 25, 27], "local_solv": [19, 25], "localsolv": 19, "solvertyp": 19, "local_solver_cach": [19, 25], "set_local_solver_cach": [19, 25], "localproject": [19, 25], "mass": [19, 22, 25, 50], "x_coord": [19, 25], "behav": 19, "correctli": 19, "edg": [19, 42], "own": [19, 33, 34], "node": [19, 22, 34, 42], "graph": [19, 42], "discret": [19, 33, 46, 47, 48, 49, 50, 51, 52, 53], "coordin": [19, 25, 49, 50], "shape": [19, 20, 25, 26], "geometr": [19, 25], "ignor": [19, 25, 34, 52], "scipi": [19, 38, 44], "spars": 19, "spmatrix": 19, "distanc": 19, "boundingboxtre": 19, "compute_closest_ent": 19, "point": [19, 25, 28, 38, 39, 43, 52], "cell": 19, "mesh": [19, 25, 46, 47, 48, 49, 50, 51], "pointinterpol": [19, 25], "interact": [20, 26, 33, 36, 42, 46], "constant": [20, 26, 46, 47, 48, 49, 50, 51], "domain": [20, 26, 46, 47, 48, 49, 50, 51, 52], "mixin": [20, 26, 33], "elimin": [20, 26], "zeroconst": [20, 26], "eliminate_zero": [20, 26], "sub_domain": [20, 26], "_homogen": [20, 26], "homogeneousdirichletbc": [20, 26, 46], "count": [20, 21, 26], "replacementconst": [20, 26], "replacementfunct": [20, 26], "replacementzeroconst": [20, 26], "replacementzerofunct": [20, 26], "cofunct": [21, 22, 25, 27, 51], "replacementcofunct": 21, "to_firedrak": [21, 51], "system": [22, 27, 35, 48], "structur": 22, "outer": [22, 45], "krylov": 22, "precondition": [22, 27], "pc_fn": [22, 27], "callback": [22, 36, 38, 42], "make": [22, 46, 49, 50, 51], "singular": [22, 27], "instead": [22, 33, 39, 43, 46, 48, 49], "full": [22, 49], "column": [22, 27], "span": 22, "nullspac": [22, 27, 51], "invert": 22, "matric": [22, 30, 46, 48], "posit": [22, 27, 38, 42], "definit": [22, 27, 38, 43, 51], "its": [22, 33, 42, 43, 45, 46, 48, 49], "transpos": 22, "primari": [22, 53], "superspac": 22, "trial": [22, 46, 47, 48, 49, 50, 51], "essenti": 22, "orthogon": [22, 27], "would": 22, "choos": [22, 47, 49], "seek": [22, 45, 46, 49, 50, 51], "subject": [22, 46, 47, 48, 49, 50, 51, 52], "constraint": [22, 50, 51], "similar": [22, 43], "similarli": [22, 46], "basic": [22, 39, 52], "tree": [22, 42], "depth": 22, "search": [22, 38], "henc": [22, 42, 46, 48, 51], "u_0": [22, 48, 51], "u_1": [22, 51], "u_2": 22, "both": [22, 33, 42, 45, 48, 51], "represent": 22, "mixedspac": 22, "split": 22, "leaf": 22, "flatten": [22, 52], "petsc4pi": [22, 33, 38], "vec": 22, "space_0": 22, "space_1": 22, "space_2": 22, "mixed_spac": 22, "to_petsc": 22, "u_petsc": 22, "from_petsc": 22, "split_spac": 22, "flattened_spac": 22, "local_s": [22, 34], "global_s": [22, 34], "global": [22, 33, 34], "new_split": 22, "ghost": 22, "apply_nullspace_transformation_lhs_right": 22, "transform": 22, "rightarrow": [22, 33, 47], "apply_nullspace_transformation_lhs_left": 22, "constraint_correct_lh": 22, "pc_constraint_correct_soln": 22, "correct_soln": 22, "correct": [22, 27, 43], "pre_mult_correct_lh": 22, "pre": 22, "post_mult_correct_lh": 22, "post": 22, "correct_rh": 22, "pc_pre_mult_correct": 22, "pc_post_mult_correct": 22, "nonenullspac": [22, 27], "constantnullspac": 22, "ones": [22, 45], "length": [22, 43], "unitynullspac": 22, "uniti": [22, 46, 48], "dirichletbcnullspac": 22, "per": [22, 48], "blocknullspac": 22, "arg_spac": 22, "action_spac": 22, "mult_add": 22, "petscmatrix": 22, "mat": 22, "form_matrix": 22, "sesquilinear": 22, "blockmatrix": 22, "correct_initial_guess": 22, "correct_solut": 22, "fenic": [22, 24, 28, 44], "fgmre": 22, "pc_side": 22, "overrid": 22, "precondit": 22, "relative_toler": [22, 28], "absolute_toler": [22, 28], "absolut": [22, 28, 38, 47, 50], "divergence_limit": 22, "diverg": 22, "maximum_iter": [22, 28], "1000": [22, 28, 38], "norm_typ": 22, "norm": [22, 28, 33, 38, 43, 46, 47, 48, 50, 51, 52], "nonzero_initial_guess": [22, 28], "gmres_restart": 22, "gmre": 22, "ksp": 22, "baseform": [24, 25, 26], "invers": [25, 27, 30, 38], "_interp": 25, "vertexonlymesh": 25, "exprassign": 25, "evalu": [25, 33, 35, 38, 43, 45], "pyop2": [25, 44], "hessiansystem": 27, "hessian_eigendecompos": 27, "correct_eigenvector": 27, "despit": 27, "notat": [27, 45], "appropri": [27, 50, 51], "subspac": 27, "ghep": 27, "b_inv_orthonormality_test": 27, "orthonorm": 27, "max_diagonal_error_norm": 27, "max_off_diagonal_error_norm": 27, "normal": [27, 47, 51], "error": [27, 43, 47, 48, 50, 51, 52], "magnitud": [27, 43], "hessian_eigendecomposition_pc": 27, "spectrum": 27, "symmetr": 27, "approx": 27, "low": [27, 45], "tobin": 27, "isaac": 27, "noemi": 27, "petra": 27, "georg": 27, "stadler": 27, "omar": 27, "ghatta": 27, "scalabl": 27, "effici": 27, "propag": 27, "uncertainti": 27, "through": [27, 49, 51], "infer": [27, 46], "predict": 27, "larg": [27, 43, 47, 53], "scale": [27, 38], "flow": 27, "sheet": 27, "296": 27, "348": 27, "368": 27, "jcp": 27, "04": [27, 47, 51, 52], "047": 27, "20": [27, 48, 50], "customnormsq": 28, "norm_sq": 28, "adj_norm_sq": 28, "squar": [28, 46, 47, 48, 49, 50, 51, 52], "total": [28, 33, 42, 48], "sum": [28, 33, 51, 52], "float": [28, 29, 34, 39, 43, 46, 50, 51], "l_2": [28, 33], "fix": [28, 48], "until": [28, 46], "reach": 28, "krylovsolv": 28, "bool": [28, 38, 42], "adjoint_nonzero_initial_guess": 28, "conveni": [29, 51], "addto": [29, 34, 39, 50], "action_fn": 30, "generalhessian": 30, "more": [30, 38, 42, 43, 44, 45, 46, 47, 48, 50, 51], "creat": [30, 34, 38, 43], "r_": 30, "ob": 30, "assimil": 30, "observ": [30, 43], "covari": 30, "background": 30, "generalgaussnewton": 30, "overloaded_float": 31, "dure": [32, 42], "garbagecollect": 32, "garbage_cleanup": [32, 33], "collect": [32, 42], "cleanup": 32, "backend": [33, 46, 47, 48, 49, 50, 52, 53], "runtim": 33, "bind": 33, "variableinterfac": 33, "spaceinterfac": 33, "extra": [33, 45, 46, 48], "relat": 33, "space_": 33, "var_": 33, "distinguish": 33, "member": [33, 49], "antidu": [33, 51], "antilinear": 33, "could": [33, 47, 48], "default_comm": 33, "mpi4pi": [33, 44], "comm_world": 33, "dummi": 33, "serial": [33, 38, 51], "serialcomm": 33, "comm_dup_cach": 33, "duplic": 33, "previou": [33, 38, 42, 45, 46], "freed": 33, "add_interfac": 33, "interface_cl": 33, "attr": 33, "attach": 33, "_tlm_adjoint__space_interface_attr": 33, "_tlm_adjoint__var_interface_attr": 33, "do": [33, 45, 46, 48, 49, 50, 51], "dynam": 33, "is_spac": 33, "had": 33, "arbitrari": [33, 51], "space_comm": 33, "space_dtyp": 33, "doubl": [33, 34, 39, 52], "cdoubl": [33, 34, 39], "space_id": 33, "space_new": [33, 39], "relative_space_typ": 33, "rel_space_typ": 33, "conjugate_space_typ": 33, "dual_space_typ": 33, "conjugate_dual_space_typ": 33, "no_space_type_check": 33, "paused_space_type_check": 33, "context": [33, 39, 42], "temporarili": [33, 39, 42], "spacetypeerror": 33, "encount": [33, 48], "check_space_typ": 33, "fail": [33, 43], "conjuguate_du": 33, "check_space_types_conjug": 33, "check_space_types_du": 33, "check_space_types_conjugate_du": 33, "is_var": 33, "var_comm": 33, "var_spac": [33, 51], "var_space_typ": 33, "var_dtyp": 33, "var_id": 33, "share": [33, 42], "thei": [33, 42], "var_nam": 33, "var_stat": 33, "counter": 33, "var_lock_st": 33, "lock": 33, "dictionari": 33, "like": [33, 46, 51], "releas": 33, "consequ": 33, "caution": 33, "destruct": 33, "determinist": 33, "var_update_st": 33, "ensur": [33, 46], "deleg": 33, "sinc": [33, 45, 46, 51], "again": [33, 43, 49, 51], "var_is_cach": 33, "var_cach": 33, "var_update_cach": 33, "var_zero": 33, "var_assign": 33, "var_axpi": 33, "place": [33, 34, 39, 51], "var_inn": [33, 46, 48], "inner": [33, 38, 45, 46, 47, 48, 49, 50, 51], "convent": 33, "var_linf_norm": 33, "l_": [33, 43], "infti": [33, 43], "var_local_s": 33, "var_global_s": 33, "across": [33, 42, 46, 53], "var_local_indic": 33, "yield": 33, "var_get_valu": 33, "var_set_valu": 33, "var_new": 33, "deprec": 33, "var_new_conjug": 33, "var_new_du": 33, "var_new_conjugate_du": 33, "var_copi": 33, "var_replac": [33, 51], "possibli": 33, "var_is_replac": 33, "var_is_scalar": 33, "var_scalar_valu": 33, "var_is_alia": 33, "alia": 33, "alias": 33, "contribut": 33, "set_default_jax_dtyp": 34, "dtype": [34, 39, 50, 52], "vectorspac": 34, "ownership_rang": 34, "slice": 34, "rdtype": [34, 39], "arrai": [34, 50, 51, 52], "ndim": [34, 52], "cast": [34, 39], "vectorequ": 34, "with_tlm": 34, "_forward_eq": 34, "y0": 34, "y1": 34, "x0": [34, 38], "x1": 34, "output": [34, 37, 48, 51], "call_jax": [34, 51, 52], "new_jax": 34, "to_jax": [34, 51], "new_jax_float": [34, 51, 52], "linearequ": 35, "b_i": 35, "has_initial_condit": 35, "adjoint_has_initial_condit": 35, "forward_act": 35, "adjoint_act": 35, "b_index": 35, "nl_dep_index": 35, "nl_deps_index": 35, "adjoint_solv": 35, "sign": 35, "tau_x": [35, 41, 51], "simpl": [36, 45, 46, 47, 48, 53], "set_manag": 36, "restore_manag": 36, "revert": 36, "restor": 36, "exit": 36, "configure_checkpoint": [36, 42, 48], "cp_method": [36, 42], "cp_paramet": [36, 42], "manager_info": [36, 45, 46], "info": [36, 42, 43, 47], "print": [36, 42, 45, 46, 48, 49, 50, 51, 52], "reset_manag": [36, 45, 46, 47, 48, 49, 51, 52], "reset": [36, 42, 45, 48], "annotation_en": [36, 42], "start_manag": [36, 45, 46, 47, 48, 49, 51, 52], "stop_manag": [36, 45, 46, 47, 48, 49, 51, 52], "stop": [36, 42, 45, 46], "paused_manag": 36, "manager_dis": 36, "configure_tlm": [36, 42, 45, 46, 48, 49], "tlm_enabl": [36, 42], "var_tlm": [36, 42, 45, 46, 48, 49], "prune_forward": [36, 42], "prune_adjoint": [36, 42], "prune_replai": [36, 42], "cache_adjoint_degre": [36, 42], "store_adjoint": [36, 42], "new_block": [36, 42, 48, 52], "controlsmark": 37, "m_": 37, "functionalmark": 37, "j_": 37, "minimize_scipi": 38, "minim": [38, 53], "gradient": [38, 50], "gather": 38, "return_valu": 38, "lbfgshessianapproxim": 38, "l": [38, 46, 47, 48, 49, 50, 51, 52], "bfg": 38, "pair": [38, 42, 43, 45, 47, 50], "append": [38, 50], "s_inner_i": 38, "separ": 38, "line": 38, "inverse_act": 38, "h_0_action": 38, "theta": 38, "7": [38, 45, 46, 47, 48, 51, 52], "l_bfg": 38, "fp": 38, "30": [38, 45, 48], "s_atol": 38, "g_atol": 38, "max_it": 38, "theta_scal": 38, "delta": 38, "m_action": 38, "m_inv_act": [38, 50], "c1": 38, "0001": 38, "c2": 38, "9": [38, 45, 46, 47, 48, 49, 50, 51, 52], "1204": 38, "discuss": [38, 48], "section": 38, "precis": [38, 45, 49], "sqrt": [38, 46, 50], "g_k": 38, "y_k": 38, "h_0": 38, "s_k": 38, "criteria": 38, "f_old": 38, "f_new": 38, "x_new": 38, "g_new": 38, "_m": 38, "armijo": 38, "c_1": [38, 51], "6a": 38, "curvatur": 38, "c_2": 38, "6b": 38, "f_call": 38, "fp_call": 38, "hessian_approx": 38, "minimize_l_bfg": 38, "minimize_tao": [38, 50], "gatol": [38, 50], "grtol": [38, 50], "gttol": 38, "tao": [38, 50], "lmvm": [38, 50], "small": [39, 43, 47], "import": [39, 45, 46, 47, 48, 49, 50, 51, 52], "np": [39, 45, 46, 47, 48, 49, 50, 51], "pi": [39, 45, 46, 47, 48, 49, 50, 51], "sin": [39, 45, 46, 47, 48, 49, 50, 51], "set_default_float_dtyp": 39, "symbolicfloat": 39, "floatspac": 39, "float_cl": 39, "no_float_overload": 39, "overloadedfloat": 39, "paused_float_overload": 39, "sympi": [39, 44], "els": [39, 51], "floatequ": [39, 45, 46], "to_float": 39, "With": [40, 44], "subsequ": [40, 50], "x_": [40, 52], "is_sav": 40, "memorystorag": 40, "hdf5storag": 40, "plai": 42, "role": 42, "No": 42, "periodic_disk": 42, "path": 42, "directori": 42, "format": 42, "interv": 42, "m_i": 42, "dm_i": 42, "identifi": [42, 51], "different": 42, "newli": 42, "cannot": [42, 48], "function_tlm": 42, "annotation_st": 42, "tlm_state": 42, "them": [42, 46], "begin": 42, "driver": 42, "diagnost": 42, "j_i": 42, "definin": 42, "due": 42, "travers": 42, "trace": [42, 46, 49], "reus": [42, 46, 48, 53], "up": [42, 43, 47], "taylor": [43, 51, 52, 53], "remaind": [43, 51, 52, 53], "suffici": [43, 46, 48], "regular": 43, "theorem": 43, "we": [43, 45, 46, 47, 48, 49, 50, 51, 52], "perturb": [43, 46, 47], "varepsilon": [43, 47], "o": [43, 47], "denot": [43, 45, 49], "quantiti": [43, 47, 51], "uncorrect": 43, "investig": 43, "success": [43, 47], "while": [43, 44, 53], "There": [43, 45, 46], "incorrect": 43, "design": [43, 45], "find": [43, 45, 46, 48, 50, 51], "too": [43, 47], "asymptot": [43, 47], "roundoff": [43, 47, 48], "prevent": 43, "expect": [43, 47, 48, 50, 51], "principl": 43, "expans": [43, 47], "practic": 43, "effect": [43, 45, 47], "problemat": 43, "verifi": [43, 51, 52, 53], "redefin": 43, "onc": [43, 46, 48], "log": [43, 47, 48, 50], "power": [43, 50], "law": [43, 50], "consecut": [43, 47], "sever": [43, 46], "minimum": [43, 47, 50], "relev": [43, 46, 48], "min_ord": [43, 47, 51, 52], "taylor_test_tlm": [43, 47, 51, 52], "assert": [43, 46, 47, 48, 50, 51, 52], "99": [43, 45, 47, 48, 50, 51], "taylor_test_tlm_adjoint": [43, 47, 51, 52], "taylor_test": [43, 47, 51], "j_val": [43, 47, 51], "seed": [43, 47, 51], "01": [43, 48, 52], "aim": [43, 53], "behaviour": 43, "repeatedli": [43, 47], "eta": 43, "max": [43, 47, 48, 49, 50, 51], "quad": [43, 46, 47, 51], "jm": 43, "djdm": [43, 49], "hjm": 43, "random": [43, 47, 51], "perturbation_direct": 43, "close": [43, 47, 50], "tlm_order": [43, 47, 51, 52], "adjoint_ord": [43, 47, 51, 52], "tlm_adjoint": [44, 47, 49, 50], "ffc": 44, "firedrak": [44, 46, 47, 48, 49, 50, 51, 52, 53], "featur": [44, 45], "jax": [44, 53], "itertool": 44, "numba": 44, "notebook": [45, 46, 47, 48, 49, 50, 51, 52, 53], "introduc": [45, 46, 47, 52, 53], "primarili": 45, "25": [45, 48, 52], "exp": [45, 47, 48, 51, 52], "idea": 45, "To": [45, 46, 47, 48, 50], "just": 45, "let": [45, 46, 47, 48], "displai": [45, 46, 47], "statu": [45, 46], "annotationst": [45, 46], "tangentlinearst": [45, 46], "f_2": 45, "f_4": 45, "f_6": 45, "f_8": 45, "8": [45, 46, 47, 48, 50, 51, 52], "ye": [45, 46], "four": [45, 46], "dz_dx": 45, "dz_dy": 45, "9885002159138745": 45, "592156957782821": 45, "compar": [45, 46], "differenc": [45, 46, 52], "center": [45, 52], "dj_dm": [45, 46, 47], "0e": [45, 46, 47, 48, 49, 50, 51], "9885002155707312": 45, "5921569579125929": 45, "dz": 45, "dx": [45, 46, 47, 48, 49, 50, 51, 52], "dy": 45, "most": 45, "told": 45, "what": 45, "ahead": 45, "zeta_x": 45, "zeta_i": 45, "dz_dm_zeta": 45, "2005295584792861": 45, "fact": 45, "happen": [45, 46], "now": [45, 46, 47, 48, 50, 51, 52], "f_31": 45, "28": [45, 48, 50], "f_31_tlm": 45, "33": [45, 48], "f_35": 45, "f_35_tlm": 45, "37": [45, 48], "f_39": 45, "39": [45, 48, 52], "f_39_tlm": 45, "f_43": 45, "43": [45, 48], "27": [45, 48], "f_43_tlm": 45, "46": [45, 48], "29": [45, 48], "eight": 45, "simpli": [45, 51], "d2z_dm_zeta_dx": 45, "d2z_dm_zeta_di": 45, "7764708733484629": 45, "49": [45, 48], "4290736694207": 45, "e_1": 45, "want": [45, 46], "e_1_x": 45, "e_1_i": 45, "deri": 45, "f_103": 45, "103": 45, "98": [45, 48], "f_103_tlm": 45, "105": 45, "100": [45, 48, 52], "107": 45, "102": 45, "_tlm": 45, "110": 45, "zeta_y_tlm": 45, "109": 45, "f_112": 45, "112": 45, "f_112_tlm": 45, "114": 45, "116": 45, "118": 45, "f_120": 45, "120": 45, "f_120_tlm": 45, "122": 45, "124": 45, "126": 45, "f_128": 45, "128": [45, 48], "97": [45, 48], "f_128_tlm": 45, "131": 45, "14": [45, 46, 47, 48], "134": 45, "101": 45, "15": [45, 46, 48, 51], "138": 45, "zeta_x_tlm": 45, "137": 45, "sixteen": 45, "constitut": 45, "d3z_dm_zeta_dx_dx": 45, "d3z_dm_zeta_dx_di": 45, "48": [45, 48], "244759753855064": 45, "increasingli": 45, "importantli": 46, "how": [46, 49, 50, 51], "facilit": 46, "linearli": [46, 51], "throughout": [46, 47, 48, 49, 50, 51], "v_0": [46, 49], "foral": [46, 47, 49, 50, 51], "qquad": [46, 47, 49, 50, 51, 52], "int_": [46, 47, 49, 50, 51], "omega": [46, 47, 49, 50, 51, 52], "nabla": [46, 47, 49, 50, 51], "cdot": [46, 47, 49, 50, 51], "p_1": [46, 47, 48, 49, 50, 51], "continu": [46, 47, 48, 49, 50, 51], "unitsquaremesh": [46, 47, 48, 49, 51], "spatialcoordin": [46, 47, 48, 49, 50, 51], "functionspac": [46, 47, 48, 49, 50, 51], "lagrang": [46, 47, 48, 49, 50, 51], "testfunct": [46, 47, 48, 49, 50, 51], "trialfunct": [46, 47, 48, 49, 50, 51], "grad": [46, 47, 48, 49, 50, 51], "on_boundari": [46, 49], "j_sq": 46, "f_11": 46, "dj_dm_one": 46, "02035145324320026": 46, "amplitud": [46, 47], "spatial": [46, 48], "020351453248329543": 46, "next": [46, 47, 48, 49, 51], "directli": [46, 48, 51], "dj_dm_zeta": 46, "d2j_dm_zeta_dm": 46, "zeta_0": [46, 47], "zeta_1": 46, "dj_dm_zeta_0": 46, "dj_dm_zeta_1": 46, "d2j_dm_zeta_0_dm": 46, "d2j_dm_zeta_1_dm": 46, "detect": [46, 48], "know": [46, 50], "sequenti": 46, "avoid": 46, "long": 46, "poisson": [46, 49, 51], "stiff": 46, "moreov": 46, "choleski": 46, "factor": 46, "seen": [46, 47], "look": [46, 51], "ksp_type": [46, 48, 50, 51], "preonli": [46, 48], "pc_type": [46, 48, 50, 51], "leak": 46, "tell": 46, "help": 46, "queri": [46, 48], "len": [46, 47, 48], "turn": [46, 48, 51], "former": 46, "slightli": 46, "introduct": 46, "issu": [46, 48], "resolv": 46, "multipli": [46, 48], "doesn": 46, "ineffici": [46, 48], "awar": 46, "time": [47, 52, 53], "independ": [47, 51, 52, 53], "hat": [47, 49, 51], "outward": [47, 51], "50": [47, 48, 52], "methodologi": 47, "wish": [47, 48], "repeat": 47, "pseudorandom": 47, "It": 47, "33582866": 47, "getlogg": [47, 48], "setlevel": [47, 48], "root_logg": [47, 48], "handler": [47, 48], "addhandl": [47, 48], "83675325e": 47, "18679773e": 47, "05": [47, 50, 51, 52], "59415781e": 47, "29726877e": 47, "14868187e": 47, "99952386": 47, "99976165": 47, "99988076": 47, "99994036": 47, "21373327e": 47, "07": [47, 52], "03720079e": 47, "08": [47, 51, 52], "59658259e": 47, "09": [47, 51, 52], "89959287e": 47, "74953355e": 47, "99863722": 47, "99931983": 47, "9996603": 47, "99983251": 47, "rerun": [47, 48, 51], "becom": 47, "improv": 47, "altern": [47, 48], "19986557": 47, "00": [47, 51, 52], "10559794e": 47, "05304003e": 47, "26580325e": 47, "63305245e": 47, "31656394e": 47, "9996697": 47, "99983476": 47, "99991736": 47, "99995867": 47, "17834456e": 47, "14548609e": 47, "43110530e": 47, "78548691e": 47, "19004148e": 47, "00227403": 47, "0007582": 47, "00274037": 47, "02728746": 47, "third": 47, "although": [47, 48], "suggest": 47, "affect": 47, "smallest": 47, "dk": [47, 51], "chosen": 47, "fourth": 47, "76149511": 47, "14011610e": 47, "07017190e": 47, "35114451e": 47, "67564355e": 47, "33783961e": 47, "99984651": 47, "99992316": 47, "99996156": 47, "99998077": 47, "55807682e": 47, "14053926e": 47, "85262144e": 47, "13314473e": 47, "78348619e": 47, "99870913": 47, "9993559": 47, "99967815": 47, "9998382": 47, "50091607e": 47, "25462282e": 47, "62835054e": 47, "14434796e": 47, "06": [47, 49, 52], "07282247e": 47, "99815267": 47, "99907905": 47, "9995402": 47, "99977027": 47, "66371223e": 47, "15377915e": 47, "03775510e": 47, "59352448e": 47, "48273139e": 47, "0019095": 47, "00095849": 47, "00048012": 47, "00024029": 47, "37224954e": 47, "85404163e": 47, "42520880e": 47, "71215009e": 47, "55961300e": 47, "001516": 47, "00076302": 47, "00038276": 47, "0001917": 47, "89362317e": 47, "26204003e": 47, "81900660e": 47, "55188642e": 47, "13851782e": 47, "99443027": 47, "99722411": 47, "99861431": 47, "99930769": 47, "tsfc": 47, "warn": 47, "estim": [47, 50], "tenfold": 47, "coeffici": 47, "86571468e": 47, "43890711e": 47, "20965028e": 47, "60860261e": 47, "80524552e": 47, "99392152": 47, "99697228": 47, "998489": 47, "99924522": 47, "41887274e": 47, "04459357e": 47, "51082379e": 47, "77665303e": 47, "44112417e": 47, "00061762": 47, "00030994": 47, "00015525": 47, "00007769": 47, "highest": 47, "74728054": 47, "96925244e": 47, "84831481e": 47, "92467146e": 47, "46246436e": 47, "23126435e": 47, "99969928": 47, "9998494": 47, "99992464": 47, "9999623": 47, "22009804e": 47, "05743897e": 47, "14661321e": 47, "28703002e": 47, "21803907e": 47, "99830597": 47, "99915437": 47, "99957766": 47, "99979196": 47, "06217536e": 47, "03568452e": 47, "51898950e": 47, "59781316e": 47, "79962268e": 47, "99781372": 47, "99890997": 47, "99945576": 47, "99972807": 47, "83663729e": 47, "58635021e": 47, "14593026e": 47, "86400265e": 47, "15897696e": 47, "00164832": 47, "00082728": 47, "00041451": 47, "00020749": 47, "77271516e": 47, "85216211e": 47, "42321703e": 47, "21089118e": 47, "10526609e": 47, "00185896": 47, "00093384": 47, "00046801": 47, "00023428": 47, "57678459e": 47, "14702543e": 47, "87109831e": 47, "18216298e": 47, "79609281e": 47, "99643702": 47, "99822274": 47, "99911243": 47, "99955649": 47, "68076588e": 47, "34645957e": 47, "74748172e": 47, "37753585e": 47, "68971655e": 47, "99347432": 47, "99674984": 47, "99837808": 47, "99918983": 47, "42982799e": 47, "07250783e": 47, "51786806e": 47, "79434579e": 47, "48545885e": 47, "00048984": 47, "00024606": 47, "00012332": 47, "00006169": 47, "overhead": 48, "binomi": 48, "advect": 48, "diffus": [48, 52], "partial_t": [48, 52], "partial_x": 48, "psi": 48, "partial_i": 48, "kappa": [48, 52], "partial_": [48, 52], "xx": [48, 52], "yy": [48, 52], "vanish": 48, "flux": 48, "tempor": 48, "75": 48, "stream": 48, "implicit": 48, "trapezoid": 48, "rule": 48, "matplotlib": [48, 49, 50, 51, 52], "inlin": [48, 49, 50, 51, 52], "pyplot": [48, 49, 50, 51, 52], "plt": [48, 49, 50, 51, 52], "dt": [48, 52], "u_n": 48, "u_np1": 48, "u_h": 48, "lh": 48, "linearvariationalproblem": 48, "constant_jacobian": 48, "linearvariationalsolv": 48, "plot_output": [48, 49, 50, 51], "titl": [48, 49, 50, 51], "dat": [48, 49, 50, 51], "data_ro": [48, 49, 50, 51], "min": [48, 49, 50, 51], "tricontourf": [48, 49, 50, 51], "linspac": [48, 49, 50, 51, 52], "32": [48, 49, 50, 51, 52], "gca": [48, 49, 50, 51], "set_titl": [48, 49, 50, 51, 52], "colorbar": [48, 49, 50, 51, 52], "set_aspect": [48, 49, 50, 51, 52], "whenev": 48, "timestep": [48, 52], "dj_du_0": 48, "dj_dpsi": 48, "unchang": 48, "inde": 48, "found": 48, "dj_dpsi_on": 48, "17": 48, "8775546878197248e": 48, "18": 48, "you": 48, "skip": 48, "dj_dpsi_zeta": 48, "d2j_dpsi_zeta_du_0": 48, "successfulli": 48, "built": 48, "slow": 48, "down": 48, "substanti": 48, "usag": 48, "exceed": 48, "larger": 48, "field": 48, "though": 48, "expens": 48, "signific": 48, "analog": 48, "well": 48, "address": 48, "9223372036854775807": 48, "40": [48, 50], "57": 48, "72": 48, "79": 48, "85": 48, "90": 48, "94": 48, "readi": 48, "restrict": 48, "entir": 48, "zeta_u_0": 48, "zeta_psi": 48, "back": [48, 51], "96": 48, "91": 48, "92": 48, "93": 48, "86": 48, "87": 48, "88": 48, "89": 48, "80": [48, 50], "81": 48, "82": 48, "83": 48, "84": 48, "73": 48, "74": 48, "76": 48, "77": 48, "78": 48, "59": 48, "61": 48, "63": 48, "65": 48, "67": 48, "69": 48, "71": 48, "70": 48, "68": 48, "66": 48, "64": 48, "62": 48, "60": 48, "58": 48, "42": 48, "44": 48, "52": 48, "54": [48, 50], "56": 48, "55": 48, "53": 48, "51": 48, "47": 48, "23": [48, 50], "38": 48, "36": 48, "24": 48, "22": 48, "riesz": [49, 51, 53], "cartesian": [49, 50], "mathbb": [49, 50], "vertic": [49, 50, 51], "711060188691798e": 49, "circ": 49, "Being": 49, "sharp": [49, 51], "riesz_represent": [49, 51], "l2": [49, 51], "twice": 49, "bilinear": 49, "stage": 49, "djdm_zeta": 49, "1580693564843862e": 49, "rememb": 49, "chi": 49, "beta": [49, 50], "d2jdm2_zeta": 49, "acceler": 50, "toolkit": 50, "answer": 50, "helmholtz": 50, "doubli": 50, "galerkin": [50, 51], "satisfi": [50, 51], "m_tild": [50, 51], "u_tild": 50, "cg": [50, 51], "sor": 50, "ksp_atol": [50, 51], "ksp_rtol": [50, 51], "periodicsquaremesh": 50, "m_0": 50, "metric": 50, "forward_j": [50, 51], "m_solver": 50, "linearsolv": 50, "deepcopi": 50, "getiterationnumb": 50, "renam": 50, "procedur": 50, "decreas": 50, "error_norm": 50, "m_error_norm": 50, "006304152630049985": 50, "0015825703318738235": 50, "00039604503914227284": 50, "160": 50, "903640999431026e": 50, "99403285": 50, "99853321": 50, "99963358": 50, "escap": [51, 52, 53], "hatch": [51, 52, 53], "co": 51, "_i": 51, "vectorspacebasi": 51, "transpose_nullspac": 51, "hypr": 51, "pc_hypre_typ": 51, "boomeramg": 51, "minor": 51, "contraint": 51, "visual": [51, 53], "54151610": 51, "16667546e": 51, "06309860e": 51, "52648952e": 51, "61979814e": 51, "80673670e": 51, "00950111": 51, "00477412": 51, "002393": 51, "00119799": 51, "09565105e": 51, "02391273e": 51, "05978163e": 51, "26494529e": 51, "16236247e": 51, "00000003": 51, "00000005": 51, "00000013": 51, "00000034": 51, "super": 51, "vec_ro": 51, "y_v": 51, "y_sum": 51, "elif": 51, "valueerror": 51, "tau_i": 51, "ll": 51, "investg": 51, "put": 51, "shortli": 51, "13561700": 51, "m_sum": 51, "unitintervalmesh": 51, "discontinu": 51, "m_v": 51, "m_sum_ref": 51, "97146153200585": 51, "32844986": 51, "15714317": 51, "57680653": 51, "28796267": 51, "14387127": 51, "00880244": 51, "00440797": 51, "00220568": 51, "00110326": 51, "02826961": 51, "00705304": 51, "00176147": 51, "00044014": 51, "00011001": 51, "00293418": 51, "00146933": 51, "00073523": 51, "00036775": 51, "68976716": 51, "33545989": 51, "66538227": 51, "33210525": 51, "16590628": 51, "01014465": 51, "0050813": 51, "0025429": 51, "00127201": 51, "03760673": 51, "00937967": 51, "00234217": 51, "0005852": 51, "00014626": 51, "00338159": 51, "00169377": 51, "00084763": 51, "000424": 51, "89746632": 51, "94164753": 51, "46905236": 51, "73408333": 51, "36693095": 51, "00347088": 51, "00173858": 51, "00087008": 51, "00043523": 51, "02834252": 51, "00708563": 51, "00177141": 51, "00044285": 51, "00011071": 51, "u_sum": 51, "dk_norm": 51, "indici": 51, "f_": 51, "tau": 51, "instanc": 51, "anoth": 51, "_function": 51, "might": 51, "lower": [51, 52], "integr": [51, 53], "81976463": 51, "9714615320058506": 51, "04885302": 51, "51235222": 51, "75316949": 51, "37583459": 51, "18772994": 51, "01147243": 51, "00574771": 51, "00287673": 51, "00143908": 51, "04816965": 51, "01201053": 51, "00299865": 51, "00074916": 51, "00018723": 51, "0038242": 51, "00191591": 51, "00095891": 51, "00047969": 51, "15029451": 51, "06909935": 51, "53304195": 51, "26614458": 51, "13297826": 51, "00813834": 51, "00407494": 51, "00203891": 51, "00101982": 51, "41462429e": 51, "02": [51, 52], "02522041e": 51, "50488756e": 51, "76044699e": 51, "39890257e": 51, "0027128": 51, "00135832": 51, "00067964": 51, "00033994": 51, "11786236": 51, "54650747": 51, "77014781": 51, "88429742": 51, "44195459": 51, "00504505": 51, "00252916": 51, "00126624": 51, "00063354": 51, "04969483": 51, "01242371": 51, "00310593": 51, "00077648": 51, "00019412": 51, "scheme": 52, "euler": 52, "0025": 52, "uniform": 52, "grid": 52, "nearest": 52, "neighbour": 52, "config": 52, "jax_enable_x64": 52, "n_t": 52, "meshgrid": 52, "ij": 52, "x_n": 52, "x_np1": 52, "zeros_lik": 52, "reshap": 52, "x_0": 52, "ax": 52, "subplot": 52, "contourf": 52, "11009761": 52, "float64": 52, "42494549e": 52, "71112518e": 52, "35522677e": 52, "77529553e": 52, "38743833e": 52, "00071691": 52, "00035745": 52, "0001785": 52, "0000892": 52, "37911835e": 52, "34199793e": 52, "35180646e": 52, "37571371e": 52, "09346468e": 52, "00298728": 52, "00137169": 52, "00065481": 52, "00031955": 52, "44879307e": 52, "72292054e": 52, "36109209e": 52, "80454089e": 52, "40204066e": 52, "00078182": 52, "0003902": 52, "00019495": 52, "00009744": 52, "89556319e": 52, "47178961e": 52, "67713890e": 52, "19011302e": 52, "29719812e": 52, "00205818": 52, "00091587": 52, "00042917": 52, "00020732": 52, "03638959e": 52, "16630940e": 52, "57966044e": 52, "28900955e": 52, "44306279e": 52, "00436048": 52, "00195287": 52, "00091823": 52, "00044439": 52, "80998182e": 52, "34113654e": 52, "21142524e": 52, "85039270e": 52, "94023890e": 52, "11507753": 52, "06217051": 52, "03237698": 52, "01653052": 52, "princip": 53, "tackl": 53, "jupyt": 53, "github": 53, "repositori": 53, "under": 53, "gnu": 53, "lgpl": 53, "modul": 53}, "objects": {"": [[31, 0, 0, "-", "tlm_adjoint"]], "tlm_adjoint": [[1, 0, 0, "-", "adjoint"], [2, 0, 0, "-", "alias"], [3, 0, 0, "-", "cached_hessian"], [4, 0, 0, "-", "caches"], [12, 0, 0, "-", "checkpointing"], [13, 0, 0, "-", "eigendecomposition"], [14, 0, 0, "-", "equation"], [15, 0, 0, "-", "equations"], [28, 0, 0, "-", "fixed_point"], [29, 0, 0, "-", "functional"], [30, 0, 0, "-", "hessian"], [32, 0, 0, "-", "instructions"], [33, 0, 0, "-", "interface"], [34, 0, 0, "-", "jax"], [35, 0, 0, "-", "linear_equation"], [36, 0, 0, "-", "manager"], [37, 0, 0, "-", "markers"], [38, 0, 0, "-", "optimization"], [39, 0, 0, "-", "overloaded_float"], [40, 0, 0, "-", "storage"], [41, 0, 0, "-", "tangent_linear"], [42, 0, 0, "-", "tlm_adjoint"], [43, 0, 0, "-", "verification"]], "tlm_adjoint.adjoint": [[1, 1, 1, "", "AdjointBlockRHS"], [1, 1, 1, "", "AdjointEquationRHS"], [1, 1, 1, "", "AdjointModelRHS"], [1, 1, 1, "", "AdjointRHS"]], "tlm_adjoint.adjoint.AdjointBlockRHS": [[1, 2, 1, "", "is_empty"], [1, 2, 1, "", "pop"]], "tlm_adjoint.adjoint.AdjointEquationRHS": [[1, 2, 1, "", "B"], [1, 2, 1, "", "b"], [1, 2, 1, "", "is_empty"]], "tlm_adjoint.adjoint.AdjointModelRHS": [[1, 2, 1, "", "is_empty"], [1, 2, 1, "", "pop"]], "tlm_adjoint.adjoint.AdjointRHS": [[1, 2, 1, "", "b"], [1, 2, 1, "", "initialize"], [1, 2, 1, "", "is_empty"], [1, 2, 1, "", "sub"]], "tlm_adjoint.alias": [[2, 1, 1, "", "Alias"], [2, 1, 1, "", "WeakAlias"], [2, 3, 1, "", "gc_disabled"]], "tlm_adjoint.cached_hessian": [[3, 1, 1, "", "CachedGaussNewton"], [3, 1, 1, "", "CachedHessian"]], "tlm_adjoint.cached_hessian.CachedGaussNewton": [[3, 2, 1, "", "action"]], "tlm_adjoint.cached_hessian.CachedHessian": [[3, 2, 1, "", "action"], [3, 2, 1, "", "compute_gradient"]], "tlm_adjoint.caches": [[4, 1, 1, "", "Cache"], [4, 1, 1, "", "CacheRef"], [4, 1, 1, "", "Caches"], [4, 3, 1, "", "clear_caches"], [4, 3, 1, "", "local_caches"]], "tlm_adjoint.caches.Cache": [[4, 2, 1, "", "add"], [4, 2, 1, "", "clear"], [4, 2, 1, "", "get"], [4, 4, 1, "", "id"]], "tlm_adjoint.caches.CacheRef": [[4, 2, 1, "", "clear"]], "tlm_adjoint.caches.Caches": [[4, 2, 1, "", "add"], [4, 2, 1, "", "clear"], [4, 2, 1, "", "remove"], [4, 2, 1, "", "update"]], "tlm_adjoint.checkpoint_schedules": [[5, 0, 0, "-", "binomial"], [6, 0, 0, "-", "h_revolve"], [7, 0, 0, "-", "memory"], [8, 0, 0, "-", "mixed"], [9, 0, 0, "-", "none"], [10, 0, 0, "-", "periodic"], [11, 0, 0, "-", "schedule"]], "tlm_adjoint.checkpoint_schedules.binomial": [[5, 1, 1, "", "MultistageCheckpointSchedule"], [5, 1, 1, "", "TwoLevelCheckpointSchedule"]], "tlm_adjoint.checkpoint_schedules.binomial.MultistageCheckpointSchedule": [[5, 4, 1, "", "is_exhausted"], [5, 2, 1, "", "iter"], [5, 4, 1, "", "uses_disk_storage"]], "tlm_adjoint.checkpoint_schedules.binomial.TwoLevelCheckpointSchedule": [[5, 4, 1, "", "is_exhausted"], [5, 2, 1, "", "iter"], [5, 4, 1, "", "uses_disk_storage"]], "tlm_adjoint.checkpoint_schedules.h_revolve": [[6, 1, 1, "", "HRevolveCheckpointSchedule"]], "tlm_adjoint.checkpoint_schedules.h_revolve.HRevolveCheckpointSchedule": [[6, 4, 1, "", "is_exhausted"], [6, 2, 1, "", "iter"], [6, 4, 1, "", "uses_disk_storage"]], "tlm_adjoint.checkpoint_schedules.memory": [[7, 1, 1, "", "MemoryCheckpointSchedule"]], "tlm_adjoint.checkpoint_schedules.memory.MemoryCheckpointSchedule": [[7, 4, 1, "", "is_exhausted"], [7, 2, 1, "", "iter"], [7, 4, 1, "", "uses_disk_storage"]], "tlm_adjoint.checkpoint_schedules.mixed": [[8, 1, 1, "", "MixedCheckpointSchedule"]], "tlm_adjoint.checkpoint_schedules.mixed.MixedCheckpointSchedule": [[8, 4, 1, "", "is_exhausted"], [8, 2, 1, "", "iter"], [8, 4, 1, "", "uses_disk_storage"]], "tlm_adjoint.checkpoint_schedules.none": [[9, 1, 1, "", "NoneCheckpointSchedule"]], "tlm_adjoint.checkpoint_schedules.none.NoneCheckpointSchedule": [[9, 4, 1, "", "is_exhausted"], [9, 2, 1, "", "iter"], [9, 4, 1, "", "uses_disk_storage"]], "tlm_adjoint.checkpoint_schedules.periodic": [[10, 1, 1, "", "PeriodicDiskCheckpointSchedule"]], "tlm_adjoint.checkpoint_schedules.periodic.PeriodicDiskCheckpointSchedule": [[10, 4, 1, "", "is_exhausted"], [10, 2, 1, "", "iter"], [10, 4, 1, "", "uses_disk_storage"]], "tlm_adjoint.checkpoint_schedules.schedule": [[11, 1, 1, "", "CheckpointAction"], [11, 1, 1, "", "CheckpointSchedule"], [11, 1, 1, "", "Clear"], [11, 1, 1, "", "Configure"], [11, 1, 1, "", "EndForward"], [11, 1, 1, "", "EndReverse"], [11, 1, 1, "", "Forward"], [11, 1, 1, "", "Read"], [11, 1, 1, "", "Reverse"], [11, 1, 1, "", "Write"]], "tlm_adjoint.checkpoint_schedules.schedule.CheckpointAction": [[11, 4, 1, "", "args"]], "tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule": [[11, 2, 1, "", "finalize"], [11, 4, 1, "", "is_exhausted"], [11, 4, 1, "", "is_running"], [11, 2, 1, "", "iter"], [11, 4, 1, "", "max_n"], [11, 4, 1, "", "n"], [11, 4, 1, "", "r"], [11, 4, 1, "", "uses_disk_storage"]], "tlm_adjoint.checkpoint_schedules.schedule.Clear": [[11, 4, 1, "", "clear_data"], [11, 4, 1, "", "clear_ics"]], "tlm_adjoint.checkpoint_schedules.schedule.Configure": [[11, 4, 1, "", "store_data"], [11, 4, 1, "", "store_ics"]], "tlm_adjoint.checkpoint_schedules.schedule.EndReverse": [[11, 4, 1, "", "exhausted"]], "tlm_adjoint.checkpoint_schedules.schedule.Forward": [[11, 4, 1, "", "n0"], [11, 4, 1, "", "n1"]], "tlm_adjoint.checkpoint_schedules.schedule.Read": [[11, 4, 1, "", "delete"], [11, 4, 1, "", "n"], [11, 4, 1, "", "storage"]], "tlm_adjoint.checkpoint_schedules.schedule.Reverse": [[11, 4, 1, "", "n0"], [11, 4, 1, "", "n1"]], "tlm_adjoint.checkpoint_schedules.schedule.Write": [[11, 4, 1, "", "n"], [11, 4, 1, "", "storage"]], "tlm_adjoint.checkpointing": [[12, 1, 1, "", "CheckpointStorage"], [12, 1, 1, "", "Checkpoints"], [12, 1, 1, "", "HDF5Checkpoints"], [12, 1, 1, "", "PickleCheckpoints"], [12, 1, 1, "", "ReplayStorage"]], "tlm_adjoint.checkpointing.CheckpointStorage": [[12, 2, 1, "", "add_equation"], [12, 2, 1, "", "add_equation_data"], [12, 2, 1, "", "add_initial_condition"], [12, 2, 1, "", "checkpoint_data"], [12, 2, 1, "", "clear"], [12, 2, 1, "", "configure"], [12, 2, 1, "", "initial_condition"], [12, 2, 1, "", "initial_conditions"], [12, 4, 1, "", "store_data"], [12, 4, 1, "", "store_ics"], [12, 2, 1, "", "update"], [12, 2, 1, "", "update_keys"]], "tlm_adjoint.checkpointing.Checkpoints": [[12, 2, 1, "", "delete"], [12, 2, 1, "", "read"], [12, 2, 1, "", "write"]], "tlm_adjoint.checkpointing.HDF5Checkpoints": [[12, 2, 1, "", "delete"], [12, 2, 1, "", "read"], [12, 2, 1, "", "write"]], "tlm_adjoint.checkpointing.PickleCheckpoints": [[12, 2, 1, "", "delete"], [12, 2, 1, "", "read"], [12, 2, 1, "", "write"]], "tlm_adjoint.checkpointing.ReplayStorage": [[12, 2, 1, "", "is_active"], [12, 2, 1, "", "popleft"], [12, 2, 1, "", "update"]], "tlm_adjoint.eigendecomposition": [[13, 3, 1, "", "eigendecompose"]], "tlm_adjoint.equation": [[14, 1, 1, "", "Equation"], [14, 1, 1, "", "NullSolver"], [14, 1, 1, "", "ZeroAssignment"]], "tlm_adjoint.equation.Equation": [[14, 2, 1, "", "X"], [14, 2, 1, "", "adj_X_type"], [14, 2, 1, "", "adj_x_type"], [14, 2, 1, "", "adjoint"], [14, 2, 1, "", "adjoint_cached"], [14, 2, 1, "", "adjoint_derivative_action"], [14, 2, 1, "", "adjoint_initial_condition_dependencies"], [14, 2, 1, "", "adjoint_jacobian_solve"], [14, 2, 1, "", "dependencies"], [14, 2, 1, "", "drop_references"], [14, 2, 1, "", "forward"], [14, 2, 1, "", "forward_solve"], [14, 2, 1, "", "initial_condition_dependencies"], [14, 2, 1, "", "new_adj_X"], [14, 2, 1, "", "new_adj_x"], [14, 2, 1, "", "nonlinear_dependencies"], [14, 2, 1, "", "solve"], [14, 2, 1, "", "subtract_adjoint_derivative_actions"], [14, 2, 1, "", "tangent_linear"], [14, 2, 1, "", "x"]], "tlm_adjoint.equation.ZeroAssignment": [[14, 2, 1, "", "adjoint_derivative_action"], [14, 2, 1, "", "adjoint_jacobian_solve"], [14, 2, 1, "", "forward_solve"], [14, 2, 1, "", "tangent_linear"]], "tlm_adjoint.equations": [[15, 1, 1, "", "Assignment"], [15, 1, 1, "", "Axpy"], [15, 1, 1, "", "Conversion"], [15, 1, 1, "", "DotProduct"], [15, 1, 1, "", "DotProductRHS"], [15, 1, 1, "", "EmptyEquation"], [15, 1, 1, "", "InnerProduct"], [15, 1, 1, "", "InnerProductRHS"], [15, 1, 1, "", "LinearCombination"], [15, 1, 1, "", "MatrixActionRHS"]], "tlm_adjoint.equations.Assignment": [[15, 2, 1, "", "adjoint_derivative_action"], [15, 2, 1, "", "adjoint_jacobian_solve"], [15, 2, 1, "", "forward_solve"], [15, 2, 1, "", "tangent_linear"]], "tlm_adjoint.equations.Conversion": [[15, 2, 1, "", "adjoint_derivative_action"], [15, 2, 1, "", "adjoint_jacobian_solve"], [15, 2, 1, "", "forward_solve"], [15, 2, 1, "", "tangent_linear"]], "tlm_adjoint.equations.DotProductRHS": [[15, 2, 1, "", "add_forward"], [15, 2, 1, "", "drop_references"], [15, 2, 1, "", "subtract_adjoint_derivative_action"], [15, 2, 1, "", "tangent_linear_rhs"]], "tlm_adjoint.equations.EmptyEquation": [[15, 2, 1, "", "forward_solve"]], "tlm_adjoint.equations.InnerProductRHS": [[15, 2, 1, "", "add_forward"], [15, 2, 1, "", "drop_references"], [15, 2, 1, "", "subtract_adjoint_derivative_action"], [15, 2, 1, "", "tangent_linear_rhs"]], "tlm_adjoint.equations.LinearCombination": [[15, 2, 1, "", "adjoint_derivative_action"], [15, 2, 1, "", "adjoint_jacobian_solve"], [15, 2, 1, "", "forward_solve"], [15, 2, 1, "", "tangent_linear"]], "tlm_adjoint.equations.MatrixActionRHS": [[15, 2, 1, "", "add_forward"], [15, 2, 1, "", "drop_references"], [15, 2, 1, "", "subtract_adjoint_derivative_action"], [15, 2, 1, "", "tangent_linear_rhs"]], "tlm_adjoint.fenics": [[16, 0, 0, "-", "backend_interface"], [17, 0, 0, "-", "caches"], [18, 0, 0, "-", "equations"], [19, 0, 0, "-", "fenics_equations"], [20, 0, 0, "-", "functions"]], "tlm_adjoint.fenics.backend_interface": [[16, 1, 1, "", "Function"], [16, 1, 1, "", "ZeroFunction"], [16, 3, 1, "", "to_fenics"]], "tlm_adjoint.fenics.caches": [[17, 1, 1, "", "AssemblyCache"], [17, 1, 1, "", "LinearSolverCache"], [17, 3, 1, "", "assembly_cache"], [17, 3, 1, "", "linear_solver_cache"], [17, 3, 1, "", "set_assembly_cache"], [17, 3, 1, "", "set_linear_solver_cache"]], "tlm_adjoint.fenics.caches.AssemblyCache": [[17, 2, 1, "", "assemble"]], "tlm_adjoint.fenics.caches.LinearSolverCache": [[17, 2, 1, "", "linear_solver"]], "tlm_adjoint.fenics.equations": [[18, 1, 1, "", "Assembly"], [18, 1, 1, "", "DirichletBCApplication"], [18, 1, 1, "", "EquationSolver"], [18, 1, 1, "", "ExprInterpolation"], [18, 1, 1, "", "Projection"], [18, 3, 1, "", "expr_new_x"], [18, 3, 1, "", "linear_equation_new_x"]], "tlm_adjoint.fenics.equations.Assembly": [[18, 2, 1, "", "adjoint_derivative_action"], [18, 2, 1, "", "adjoint_jacobian_solve"], [18, 2, 1, "", "drop_references"], [18, 2, 1, "", "forward_solve"], [18, 2, 1, "", "tangent_linear"]], "tlm_adjoint.fenics.equations.DirichletBCApplication": [[18, 2, 1, "", "adjoint_derivative_action"], [18, 2, 1, "", "adjoint_jacobian_solve"], [18, 2, 1, "", "forward_solve"], [18, 2, 1, "", "tangent_linear"]], "tlm_adjoint.fenics.equations.EquationSolver": [[18, 2, 1, "", "adjoint_jacobian_solve"], [18, 2, 1, "", "drop_references"], [18, 2, 1, "", "forward_solve"], [18, 2, 1, "", "subtract_adjoint_derivative_actions"], [18, 2, 1, "", "tangent_linear"]], "tlm_adjoint.fenics.equations.ExprInterpolation": [[18, 2, 1, "", "adjoint_derivative_action"], [18, 2, 1, "", "adjoint_jacobian_solve"], [18, 2, 1, "", "drop_references"], [18, 2, 1, "", "forward_solve"], [18, 2, 1, "", "tangent_linear"]], "tlm_adjoint.fenics.fenics_equations": [[19, 1, 1, "", "Interpolation"], [19, 1, 1, "", "LocalProjection"], [19, 1, 1, "", "LocalSolverCache"], [19, 1, 1, "", "PointInterpolation"], [19, 3, 1, "", "local_solver_cache"], [19, 3, 1, "", "set_local_solver_cache"]], "tlm_adjoint.fenics.fenics_equations.LocalProjection": [[19, 2, 1, "", "adjoint_jacobian_solve"], [19, 2, 1, "", "forward_solve"], [19, 2, 1, "", "tangent_linear"]], "tlm_adjoint.fenics.fenics_equations.LocalSolverCache": [[19, 2, 1, "", "local_solver"]], "tlm_adjoint.fenics.fenics_equations.PointInterpolation": [[19, 2, 1, "", "adjoint_derivative_action"], [19, 2, 1, "", "adjoint_jacobian_solve"], [19, 2, 1, "", "forward_solve"], [19, 2, 1, "", "tangent_linear"]], "tlm_adjoint.fenics.functions": [[20, 1, 1, "", "Constant"], [20, 1, 1, "", "DirichletBC"], [20, 1, 1, "", "HomogeneousDirichletBC"], [20, 1, 1, "", "Replacement"], [20, 1, 1, "", "ReplacementConstant"], [20, 1, 1, "", "ReplacementFunction"], [20, 1, 1, "", "ReplacementZeroConstant"], [20, 1, 1, "", "ReplacementZeroFunction"], [20, 1, 1, "", "Zero"], [20, 1, 1, "", "ZeroConstant"], [20, 3, 1, "", "eliminate_zeros"]], "tlm_adjoint.firedrake": [[21, 0, 0, "-", "backend_interface"], [22, 0, 0, "-", "block_system"], [23, 0, 0, "-", "caches"], [24, 0, 0, "-", "equations"], [25, 0, 0, "-", "firedrake_equations"], [26, 0, 0, "-", "functions"], [27, 0, 0, "-", "hessian_system"]], "tlm_adjoint.firedrake.backend_interface": [[21, 1, 1, "", "Cofunction"], [21, 1, 1, "", "Function"], [21, 1, 1, "", "ReplacementCofunction"], [21, 1, 1, "", "ZeroFunction"], [21, 3, 1, "", "to_firedrake"]], "tlm_adjoint.firedrake.backend_interface.Cofunction": [[21, 2, 1, "", "equals"]], "tlm_adjoint.firedrake.block_system": [[22, 1, 1, "", "BlockMatrix"], [22, 1, 1, "", "BlockNullspace"], [22, 1, 1, "", "ConstantNullspace"], [22, 1, 1, "", "DirichletBCNullspace"], [22, 1, 1, "", "Matrix"], [22, 1, 1, "", "MixedSpace"], [22, 1, 1, "", "NoneNullspace"], [22, 1, 1, "", "Nullspace"], [22, 1, 1, "", "PETScMatrix"], [22, 1, 1, "", "System"], [22, 1, 1, "", "UnityNullspace"], [22, 3, 1, "", "form_matrix"]], "tlm_adjoint.firedrake.block_system.BlockMatrix": [[22, 2, 1, "", "mult_add"]], "tlm_adjoint.firedrake.block_system.BlockNullspace": [[22, 2, 1, "", "apply_nullspace_transformation_lhs_left"], [22, 2, 1, "", "apply_nullspace_transformation_lhs_right"], [22, 2, 1, "", "constraint_correct_lhs"], [22, 2, 1, "", "pc_constraint_correct_soln"]], "tlm_adjoint.firedrake.block_system.ConstantNullspace": [[22, 2, 1, "", "apply_nullspace_transformation_lhs_left"], [22, 2, 1, "", "apply_nullspace_transformation_lhs_right"], [22, 2, 1, "", "constraint_correct_lhs"], [22, 2, 1, "", "pc_constraint_correct_soln"]], "tlm_adjoint.firedrake.block_system.DirichletBCNullspace": [[22, 2, 1, "", "apply_nullspace_transformation_lhs_left"], [22, 2, 1, "", "apply_nullspace_transformation_lhs_right"], [22, 2, 1, "", "constraint_correct_lhs"], [22, 2, 1, "", "pc_constraint_correct_soln"]], "tlm_adjoint.firedrake.block_system.Matrix": [[22, 4, 1, "", "action_space"], [22, 4, 1, "", "arg_space"], [22, 2, 1, "", "mult_add"]], "tlm_adjoint.firedrake.block_system.MixedSpace": [[22, 4, 1, "", "comm"], [22, 4, 1, "", "flattened_space"], [22, 2, 1, "", "from_petsc"], [22, 4, 1, "", "global_size"], [22, 4, 1, "", "local_size"], [22, 2, 1, "", "new_split"], [22, 4, 1, "", "split_space"], [22, 2, 1, "", "to_petsc"]], "tlm_adjoint.firedrake.block_system.NoneNullspace": [[22, 2, 1, "", "apply_nullspace_transformation_lhs_left"], [22, 2, 1, "", "apply_nullspace_transformation_lhs_right"], [22, 2, 1, "", "constraint_correct_lhs"], [22, 2, 1, "", "pc_constraint_correct_soln"]], "tlm_adjoint.firedrake.block_system.Nullspace": [[22, 2, 1, "", "apply_nullspace_transformation_lhs_left"], [22, 2, 1, "", "apply_nullspace_transformation_lhs_right"], [22, 2, 1, "", "constraint_correct_lhs"], [22, 2, 1, "", "correct_rhs"], [22, 2, 1, "", "correct_soln"], [22, 2, 1, "", "pc_constraint_correct_soln"], [22, 2, 1, "", "pc_post_mult_correct"], [22, 2, 1, "", "pc_pre_mult_correct"], [22, 2, 1, "", "post_mult_correct_lhs"], [22, 2, 1, "", "pre_mult_correct_lhs"]], "tlm_adjoint.firedrake.block_system.PETScMatrix": [[22, 2, 1, "", "mult_add"]], "tlm_adjoint.firedrake.block_system.System": [[22, 2, 1, "", "solve"]], "tlm_adjoint.firedrake.block_system.UnityNullspace": [[22, 2, 1, "", "apply_nullspace_transformation_lhs_left"], [22, 2, 1, "", "apply_nullspace_transformation_lhs_right"], [22, 2, 1, "", "constraint_correct_lhs"], [22, 2, 1, "", "pc_constraint_correct_soln"]], "tlm_adjoint.firedrake.caches": [[23, 1, 1, "", "AssemblyCache"], [23, 1, 1, "", "LinearSolverCache"], [23, 3, 1, "", "assembly_cache"], [23, 3, 1, "", "linear_solver_cache"], [23, 3, 1, "", "set_assembly_cache"], [23, 3, 1, "", "set_linear_solver_cache"]], "tlm_adjoint.firedrake.caches.AssemblyCache": [[23, 2, 1, "", "assemble"]], "tlm_adjoint.firedrake.caches.LinearSolverCache": [[23, 2, 1, "", "linear_solver"]], "tlm_adjoint.firedrake.equations": [[24, 1, 1, "", "Assembly"], [24, 1, 1, "", "DirichletBCApplication"], [24, 1, 1, "", "EquationSolver"], [24, 1, 1, "", "ExprInterpolation"], [24, 1, 1, "", "Projection"], [24, 3, 1, "", "expr_new_x"], [24, 3, 1, "", "linear_equation_new_x"]], "tlm_adjoint.firedrake.equations.Assembly": [[24, 2, 1, "", "adjoint_derivative_action"], [24, 2, 1, "", "adjoint_jacobian_solve"], [24, 2, 1, "", "drop_references"], [24, 2, 1, "", "forward_solve"], [24, 2, 1, "", "tangent_linear"]], "tlm_adjoint.firedrake.equations.DirichletBCApplication": [[24, 2, 1, "", "adjoint_derivative_action"], [24, 2, 1, "", "adjoint_jacobian_solve"], [24, 2, 1, "", "forward_solve"], [24, 2, 1, "", "tangent_linear"]], "tlm_adjoint.firedrake.equations.EquationSolver": [[24, 2, 1, "", "adjoint_jacobian_solve"], [24, 2, 1, "", "drop_references"], [24, 2, 1, "", "forward_solve"], [24, 2, 1, "", "subtract_adjoint_derivative_actions"], [24, 2, 1, "", "tangent_linear"]], "tlm_adjoint.firedrake.equations.ExprInterpolation": [[24, 2, 1, "", "adjoint_derivative_action"], [24, 2, 1, "", "adjoint_jacobian_solve"], [24, 2, 1, "", "drop_references"], [24, 2, 1, "", "forward_solve"], [24, 2, 1, "", "tangent_linear"]], "tlm_adjoint.firedrake.firedrake_equations": [[25, 1, 1, "", "ExprAssignment"], [25, 1, 1, "", "LocalProjection"], [25, 1, 1, "", "LocalSolverCache"], [25, 1, 1, "", "PointInterpolation"], [25, 3, 1, "", "local_solver_cache"], [25, 3, 1, "", "set_local_solver_cache"]], "tlm_adjoint.firedrake.firedrake_equations.ExprAssignment": [[25, 2, 1, "", "adjoint_derivative_action"], [25, 2, 1, "", "adjoint_jacobian_solve"], [25, 2, 1, "", "drop_references"], [25, 2, 1, "", "forward_solve"], [25, 2, 1, "", "tangent_linear"]], "tlm_adjoint.firedrake.firedrake_equations.LocalProjection": [[25, 2, 1, "", "adjoint_jacobian_solve"], [25, 2, 1, "", "forward_solve"], [25, 2, 1, "", "tangent_linear"]], "tlm_adjoint.firedrake.firedrake_equations.LocalSolverCache": [[25, 2, 1, "", "local_solver"]], "tlm_adjoint.firedrake.firedrake_equations.PointInterpolation": [[25, 2, 1, "", "adjoint_derivative_action"], [25, 2, 1, "", "adjoint_jacobian_solve"], [25, 2, 1, "", "forward_solve"], [25, 2, 1, "", "tangent_linear"]], "tlm_adjoint.firedrake.functions": [[26, 1, 1, "", "Constant"], [26, 1, 1, "", "DirichletBC"], [26, 1, 1, "", "HomogeneousDirichletBC"], [26, 1, 1, "", "Replacement"], [26, 1, 1, "", "ReplacementConstant"], [26, 1, 1, "", "ReplacementFunction"], [26, 1, 1, "", "ReplacementZeroConstant"], [26, 1, 1, "", "ReplacementZeroFunction"], [26, 1, 1, "", "Zero"], [26, 1, 1, "", "ZeroConstant"], [26, 3, 1, "", "eliminate_zeros"]], "tlm_adjoint.firedrake.hessian_system": [[27, 3, 1, "", "B_inv_orthonormality_test"], [27, 1, 1, "", "HessianSystem"], [27, 3, 1, "", "hessian_eigendecompose"], [27, 3, 1, "", "hessian_eigendecomposition_pc"]], "tlm_adjoint.firedrake.hessian_system.HessianSystem": [[27, 2, 1, "", "solve"]], "tlm_adjoint.fixed_point": [[28, 1, 1, "", "CustomNormSq"], [28, 1, 1, "", "FixedPointSolver"]], "tlm_adjoint.fixed_point.FixedPointSolver": [[28, 2, 1, "", "adjoint_jacobian_solve"], [28, 2, 1, "", "drop_references"], [28, 2, 1, "", "forward_solve"], [28, 2, 1, "", "subtract_adjoint_derivative_actions"], [28, 2, 1, "", "tangent_linear"]], "tlm_adjoint.functional": [[29, 1, 1, "", "Functional"]], "tlm_adjoint.functional.Functional": [[29, 2, 1, "", "addto"], [29, 2, 1, "", "assign"]], "tlm_adjoint.hessian": [[30, 1, 1, "", "GaussNewton"], [30, 1, 1, "", "GeneralGaussNewton"], [30, 1, 1, "", "GeneralHessian"], [30, 1, 1, "", "Hessian"]], "tlm_adjoint.hessian.GaussNewton": [[30, 2, 1, "", "action"], [30, 2, 1, "", "action_fn"]], "tlm_adjoint.hessian.GeneralHessian": [[30, 2, 1, "", "action"], [30, 2, 1, "", "compute_gradient"]], "tlm_adjoint.hessian.Hessian": [[30, 2, 1, "", "action"], [30, 2, 1, "", "action_fn"], [30, 2, 1, "", "compute_gradient"]], "tlm_adjoint.instructions": [[32, 1, 1, "", "GarbageCollection"], [32, 1, 1, "", "Instruction"]], "tlm_adjoint.instructions.GarbageCollection": [[32, 2, 1, "", "adjoint_jacobian_solve"], [32, 2, 1, "", "forward_solve"], [32, 2, 1, "", "tangent_linear"]], "tlm_adjoint.instructions.Instruction": [[32, 2, 1, "", "adjoint_jacobian_solve"]], "tlm_adjoint.interface": [[33, 1, 1, "", "SpaceInterface"], [33, 5, 1, "", "SpaceTypeError"], [33, 1, 1, "", "VariableInterface"], [33, 1, 1, "", "VariableStateLockDictionary"], [33, 3, 1, "", "add_interface"], [33, 3, 1, "", "check_space_type"], [33, 3, 1, "", "check_space_types"], [33, 3, 1, "", "check_space_types_conjugate"], [33, 3, 1, "", "check_space_types_conjugate_dual"], [33, 3, 1, "", "check_space_types_dual"], [33, 3, 1, "", "comm_dup_cached"], [33, 3, 1, "", "conjugate_dual_space_type"], [33, 3, 1, "", "conjugate_space_type"], [33, 3, 1, "", "dual_space_type"], [33, 3, 1, "", "garbage_cleanup"], [33, 3, 1, "", "is_space"], [33, 3, 1, "", "is_var"], [33, 3, 1, "", "no_space_type_checking"], [33, 3, 1, "", "paused_space_type_checking"], [33, 3, 1, "", "relative_space_type"], [33, 3, 1, "", "space_comm"], [33, 3, 1, "", "space_dtype"], [33, 3, 1, "", "space_id"], [33, 3, 1, "", "space_new"], [33, 3, 1, "", "subtract_adjoint_derivative_action"], [33, 3, 1, "", "var_assign"], [33, 3, 1, "", "var_axpy"], [33, 3, 1, "", "var_caches"], [33, 3, 1, "", "var_comm"], [33, 3, 1, "", "var_copy"], [33, 3, 1, "", "var_dtype"], [33, 3, 1, "", "var_get_values"], [33, 3, 1, "", "var_global_size"], [33, 3, 1, "", "var_id"], [33, 3, 1, "", "var_inner"], [33, 3, 1, "", "var_is_alias"], [33, 3, 1, "", "var_is_cached"], [33, 3, 1, "", "var_is_replacement"], [33, 3, 1, "", "var_is_scalar"], [33, 3, 1, "", "var_is_static"], [33, 3, 1, "", "var_linf_norm"], [33, 3, 1, "", "var_local_indices"], [33, 3, 1, "", "var_local_size"], [33, 3, 1, "", "var_lock_state"], [33, 3, 1, "", "var_name"], [33, 3, 1, "", "var_new"], [33, 3, 1, "", "var_new_conjugate"], [33, 3, 1, "", "var_new_conjugate_dual"], [33, 3, 1, "", "var_new_dual"], [33, 3, 1, "", "var_replacement"], [33, 3, 1, "", "var_scalar_value"], [33, 3, 1, "", "var_set_values"], [33, 3, 1, "", "var_space"], [33, 3, 1, "", "var_space_type"], [33, 3, 1, "", "var_state"], [33, 3, 1, "", "var_update_caches"], [33, 3, 1, "", "var_update_state"], [33, 3, 1, "", "var_zero"]], "tlm_adjoint.jax": [[34, 1, 1, "", "Vector"], [34, 1, 1, "", "VectorEquation"], [34, 1, 1, "", "VectorSpace"], [34, 3, 1, "", "call_jax"], [34, 3, 1, "", "new_jax"], [34, 3, 1, "", "new_jax_float"], [34, 3, 1, "", "set_default_jax_dtype"], [34, 3, 1, "", "to_jax"]], "tlm_adjoint.jax.Vector": [[34, 2, 1, "", "addto"], [34, 2, 1, "", "assign"], [34, 4, 1, "", "name"], [34, 2, 1, "", "new"], [34, 4, 1, "", "space"], [34, 4, 1, "", "space_type"], [34, 4, 1, "", "value"], [34, 4, 1, "", "vector"]], "tlm_adjoint.jax.VectorEquation": [[34, 2, 1, "", "adjoint_jacobian_solve"], [34, 2, 1, "", "forward_solve"], [34, 2, 1, "", "solve"], [34, 2, 1, "", "subtract_adjoint_derivative_actions"], [34, 2, 1, "", "tangent_linear"]], "tlm_adjoint.jax.VectorSpace": [[34, 4, 1, "", "comm"], [34, 4, 1, "", "dtype"], [34, 4, 1, "", "global_size"], [34, 4, 1, "", "local_size"], [34, 4, 1, "", "ownership_range"], [34, 2, 1, "", "rdtype"]], "tlm_adjoint.linear_equation": [[35, 1, 1, "", "LinearEquation"], [35, 1, 1, "", "Matrix"], [35, 1, 1, "", "RHS"]], "tlm_adjoint.linear_equation.LinearEquation": [[35, 2, 1, "", "adjoint_derivative_action"], [35, 2, 1, "", "adjoint_jacobian_solve"], [35, 2, 1, "", "drop_references"], [35, 2, 1, "", "forward_solve"], [35, 2, 1, "", "tangent_linear"]], "tlm_adjoint.linear_equation.Matrix": [[35, 2, 1, "", "adjoint_action"], [35, 2, 1, "", "adjoint_derivative_action"], [35, 2, 1, "", "adjoint_has_initial_condition"], [35, 2, 1, "", "adjoint_solve"], [35, 2, 1, "", "drop_references"], [35, 2, 1, "", "forward_action"], [35, 2, 1, "", "forward_solve"], [35, 2, 1, "", "has_initial_condition"], [35, 2, 1, "", "nonlinear_dependencies"], [35, 2, 1, "", "tangent_linear_rhs"]], "tlm_adjoint.linear_equation.RHS": [[35, 2, 1, "", "add_forward"], [35, 2, 1, "", "dependencies"], [35, 2, 1, "", "drop_references"], [35, 2, 1, "", "nonlinear_dependencies"], [35, 2, 1, "", "subtract_adjoint_derivative_action"], [35, 2, 1, "", "tangent_linear_rhs"]], "tlm_adjoint.manager": [[36, 3, 1, "", "annotation_enabled"], [36, 3, 1, "", "compute_gradient"], [36, 3, 1, "", "configure_checkpointing"], [36, 3, 1, "", "configure_tlm"], [36, 3, 1, "", "manager"], [36, 3, 1, "", "manager_disabled"], [36, 3, 1, "", "manager_info"], [36, 3, 1, "", "new_block"], [36, 3, 1, "", "paused_manager"], [36, 3, 1, "", "reset_manager"], [36, 3, 1, "", "restore_manager"], [36, 3, 1, "", "set_manager"], [36, 3, 1, "", "start_manager"], [36, 3, 1, "", "stop_manager"], [36, 3, 1, "", "tlm_enabled"], [36, 3, 1, "", "var_tlm"]], "tlm_adjoint.markers": [[37, 1, 1, "", "ControlsMarker"], [37, 1, 1, "", "FunctionalMarker"]], "tlm_adjoint.markers.ControlsMarker": [[37, 2, 1, "", "adjoint_jacobian_solve"]], "tlm_adjoint.markers.FunctionalMarker": [[37, 2, 1, "", "adjoint_derivative_action"], [37, 2, 1, "", "adjoint_jacobian_solve"]], "tlm_adjoint.optimization": [[38, 1, 1, "", "LBFGSHessianApproximation"], [38, 3, 1, "", "l_bfgs"], [38, 3, 1, "", "minimize_l_bfgs"], [38, 3, 1, "", "minimize_scipy"], [38, 3, 1, "", "minimize_tao"]], "tlm_adjoint.optimization.LBFGSHessianApproximation": [[38, 2, 1, "", "append"], [38, 2, 1, "", "inverse_action"]], "tlm_adjoint.overloaded_float": [[39, 1, 1, "", "Float"], [39, 1, 1, "", "FloatEquation"], [39, 1, 1, "", "FloatSpace"], [39, 1, 1, "", "OverloadedFloat"], [39, 1, 1, "", "SymbolicFloat"], [39, 3, 1, "", "no_float_overloading"], [39, 3, 1, "", "paused_float_overloading"], [39, 3, 1, "", "set_default_float_dtype"], [39, 3, 1, "", "to_float"]], "tlm_adjoint.overloaded_float.FloatEquation": [[39, 2, 1, "", "adjoint_jacobian_solve"], [39, 2, 1, "", "forward_solve"], [39, 2, 1, "", "subtract_adjoint_derivative_actions"], [39, 2, 1, "", "tangent_linear"]], "tlm_adjoint.overloaded_float.FloatSpace": [[39, 4, 1, "", "comm"], [39, 4, 1, "", "dtype"], [39, 4, 1, "", "float_cls"], [39, 2, 1, "", "rdtype"]], "tlm_adjoint.overloaded_float.SymbolicFloat": [[39, 2, 1, "", "addto"], [39, 2, 1, "", "assign"], [39, 2, 1, "", "new"], [39, 4, 1, "", "value"]], "tlm_adjoint.storage": [[40, 1, 1, "", "HDF5Storage"], [40, 1, 1, "", "MemoryStorage"], [40, 1, 1, "", "Storage"]], "tlm_adjoint.storage.HDF5Storage": [[40, 2, 1, "", "is_saved"], [40, 2, 1, "", "load"], [40, 2, 1, "", "save"]], "tlm_adjoint.storage.MemoryStorage": [[40, 2, 1, "", "is_saved"], [40, 2, 1, "", "load"], [40, 2, 1, "", "save"]], "tlm_adjoint.storage.Storage": [[40, 2, 1, "", "adjoint_derivative_action"], [40, 2, 1, "", "adjoint_jacobian_solve"], [40, 2, 1, "", "forward_solve"], [40, 2, 1, "", "is_saved"], [40, 4, 1, "", "key"], [40, 2, 1, "", "load"], [40, 2, 1, "", "save"], [40, 2, 1, "", "tangent_linear"]], "tlm_adjoint.tangent_linear": [[41, 1, 1, "", "TangentLinearMap"]], "tlm_adjoint.tangent_linear.TangentLinearMap": [[41, 4, 1, "", "id"]], "tlm_adjoint.tlm_adjoint": [[42, 1, 1, "", "EquationManager"]], "tlm_adjoint.tlm_adjoint.EquationManager": [[42, 2, 1, "", "add_equation"], [42, 2, 1, "", "add_initial_condition"], [42, 2, 1, "", "annotation_enabled"], [42, 2, 1, "", "comm"], [42, 2, 1, "", "compute_gradient"], [42, 2, 1, "", "configure_checkpointing"], [42, 2, 1, "", "configure_tlm"], [42, 2, 1, "", "drop_references"], [42, 2, 1, "", "finalize"], [42, 2, 1, "", "function_tlm"], [42, 2, 1, "", "info"], [42, 2, 1, "", "new"], [42, 2, 1, "", "new_block"], [42, 2, 1, "", "paused"], [42, 2, 1, "", "reset"], [42, 2, 1, "", "start"], [42, 2, 1, "", "stop"], [42, 2, 1, "", "tlm_enabled"], [42, 2, 1, "", "var_tlm"]], "tlm_adjoint.verification": [[43, 3, 1, "", "taylor_test"], [43, 3, 1, "", "taylor_test_tlm"], [43, 3, 1, "", "taylor_test_tlm_adjoint"]]}, "objtypes": {"0": "py:module", "1": "py:class", "2": "py:method", "3": "py:function", "4": "py:property", "5": "py:exception"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "class", "Python class"], "2": ["py", "method", "Python method"], "3": ["py", "function", "Python function"], "4": ["py", "property", "Python property"], "5": ["py", "exception", "Python exception"]}, "titleterms": {"refer": [0, 51], "acknowledg": 0, "cite": 0, "tlm_adjoint": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 48, 51, 52, 53], "dolfin": 0, "adjoint": [0, 1, 45, 46, 48, 49], "taylor": [0, 47], "remaind": [0, 47], "converg": [0, 47], "test": [0, 47], "solv": 0, "eigenproblem": 0, "slepc": 0, "differenti": 0, "fix": 0, "point": [0, 45], "problem": [0, 46, 47, 48, 49, 50, 51, 52], "binomi": [0, 5], "checkpoint": [0, 12, 48], "l": 0, "bfg": 0, "fund": 0, "modul": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43], "content": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43], "alia": 2, "cached_hessian": 3, "cach": [4, 17, 23, 46], "checkpoint_schedul": [5, 6, 7, 8, 9, 10, 11], "h_revolv": 6, "memori": 7, "mix": 8, "none": 9, "period": 10, "schedul": [11, 48], "eigendecomposit": 13, "equat": [14, 15, 18, 24, 51], "fenic": [16, 17, 18, 19, 20], "backend_interfac": [16, 21], "fenics_equ": 19, "function": [20, 26, 29, 50], "firedrak": [21, 22, 23, 24, 25, 26, 27], "block_system": 22, "firedrake_equ": 25, "hessian_system": 27, "fixed_point": 28, "hessian": [30, 46, 48, 49], "submodul": 31, "instruct": 32, "interfac": 33, "jax": [34, 51, 52], "linear_equ": 35, "manag": 36, "marker": 37, "optim": [38, 48], "overloaded_float": 39, "storag": 40, "tangent_linear": [41, 51], "verif": 43, "depend": [44, 48], "requir": 44, "backend": 44, "option": 44, "get": 45, "start": 45, "A": 45, "float": 45, "exampl": [45, 46, 48, 53], "ad": [45, 46, 48, 51, 52], "comput": [45, 46, 48], "deriv": [45, 46, 47, 48, 49], "us": [45, 46, 48, 51], "an": [45, 46, 48], "tangent": [45, 46, 48], "linear": [45, 46, 48], "second": [45, 47], "third": 45, "higher": [45, 47], "order": [45, 47, 49], "time": [46, 48], "independ": 46, "forward": [46, 47, 48, 49, 50, 51, 52], "inform": [46, 48], "singl": 46, "direct": 46, "multipl": 46, "assembli": 46, "solver": 46, "equationsolv": 46, "flag": 46, "data": 46, "The": 46, "static": 46, "verifi": 47, "calcul": 47, "first": [47, 49], "annot": 48, "visual": 49, "action": 49, "minim": 50, "invers": 50, "defin": 51, "custom": 51, "oper": 51, "method": 51, "__init__": 51, "forward_solv": 51, "adjoint_jacobian_solv": 51, "adjoint_derivative_act": 51, "drop": 51, "integr": 52, "featur": 53, "sourc": 53, "licens": 53, "indic": 53}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "nbsphinx": 4, "sphinx.ext.intersphinx": 1, "sphinx": 60}, "alltitles": {"References and acknowledgements": [[0, "references-and-acknowledgements"]], "Citing tlm_adjoint": [[0, "citing-tlm-adjoint"]], "References": [[0, "references"]], "dolfin-adjoint": [[0, "dolfin-adjoint"]], "Taylor remainder convergence testing": [[0, "taylor-remainder-convergence-testing"]], "Solving eigenproblems with SLEPc": [[0, "solving-eigenproblems-with-slepc"]], "Differentiating fixed-point problems": [[0, "differentiating-fixed-point-problems"]], "Binomial checkpointing": [[0, "binomial-checkpointing"]], "L-BFGS": [[0, "l-bfgs"]], "Funding": [[0, "funding"]], "tlm_adjoint.adjoint": [[1, "module-tlm_adjoint.adjoint"]], "Module Contents": [[1, "module-contents"], [2, "module-contents"], [3, "module-contents"], [4, "module-contents"], [5, "module-contents"], [6, "module-contents"], [7, "module-contents"], [8, "module-contents"], [9, "module-contents"], [10, "module-contents"], [11, "module-contents"], [12, "module-contents"], [13, "module-contents"], [14, "module-contents"], [15, "module-contents"], [16, "module-contents"], [17, "module-contents"], [18, "module-contents"], [19, "module-contents"], [20, "module-contents"], [21, "module-contents"], [22, "module-contents"], [23, "module-contents"], [24, "module-contents"], [25, "module-contents"], [26, "module-contents"], [27, "module-contents"], [28, "module-contents"], [29, "module-contents"], [30, "module-contents"], [32, "module-contents"], [33, "module-contents"], [34, "module-contents"], [35, "module-contents"], [36, "module-contents"], [37, "module-contents"], [38, "module-contents"], [39, "module-contents"], [40, "module-contents"], [41, "module-contents"], [42, "module-contents"], [43, "module-contents"]], "tlm_adjoint.alias": [[2, "module-tlm_adjoint.alias"]], "tlm_adjoint.cached_hessian": [[3, "module-tlm_adjoint.cached_hessian"]], "tlm_adjoint.caches": [[4, "module-tlm_adjoint.caches"]], "tlm_adjoint.checkpoint_schedules.binomial": [[5, "module-tlm_adjoint.checkpoint_schedules.binomial"]], "tlm_adjoint.checkpoint_schedules.h_revolve": [[6, "module-tlm_adjoint.checkpoint_schedules.h_revolve"]], "tlm_adjoint.checkpoint_schedules.memory": [[7, "module-tlm_adjoint.checkpoint_schedules.memory"]], "tlm_adjoint.checkpoint_schedules.mixed": [[8, "module-tlm_adjoint.checkpoint_schedules.mixed"]], "tlm_adjoint.checkpoint_schedules.none": [[9, "module-tlm_adjoint.checkpoint_schedules.none"]], "tlm_adjoint.checkpoint_schedules.periodic": [[10, "module-tlm_adjoint.checkpoint_schedules.periodic"]], "tlm_adjoint.checkpoint_schedules.schedule": [[11, "module-tlm_adjoint.checkpoint_schedules.schedule"]], "tlm_adjoint.checkpointing": [[12, "module-tlm_adjoint.checkpointing"]], "tlm_adjoint.eigendecomposition": [[13, "module-tlm_adjoint.eigendecomposition"]], "tlm_adjoint.equation": [[14, "module-tlm_adjoint.equation"]], "tlm_adjoint.equations": [[15, "module-tlm_adjoint.equations"]], "tlm_adjoint.fenics.backend_interface": [[16, "module-tlm_adjoint.fenics.backend_interface"]], "tlm_adjoint.fenics.caches": [[17, "module-tlm_adjoint.fenics.caches"]], "tlm_adjoint.fenics.equations": [[18, "module-tlm_adjoint.fenics.equations"]], "tlm_adjoint.fenics.fenics_equations": [[19, "module-tlm_adjoint.fenics.fenics_equations"]], "tlm_adjoint.fenics.functions": [[20, "module-tlm_adjoint.fenics.functions"]], "tlm_adjoint.firedrake.backend_interface": [[21, "module-tlm_adjoint.firedrake.backend_interface"]], "tlm_adjoint.firedrake.block_system": [[22, "module-tlm_adjoint.firedrake.block_system"]], "tlm_adjoint.firedrake.caches": [[23, "module-tlm_adjoint.firedrake.caches"]], "tlm_adjoint.firedrake.equations": [[24, "module-tlm_adjoint.firedrake.equations"]], "tlm_adjoint.firedrake.firedrake_equations": [[25, "module-tlm_adjoint.firedrake.firedrake_equations"]], "tlm_adjoint.firedrake.functions": [[26, "module-tlm_adjoint.firedrake.functions"]], "tlm_adjoint.firedrake.hessian_system": [[27, "module-tlm_adjoint.firedrake.hessian_system"]], "tlm_adjoint.fixed_point": [[28, "module-tlm_adjoint.fixed_point"]], "tlm_adjoint.functional": [[29, "module-tlm_adjoint.functional"]], "tlm_adjoint.hessian": [[30, "module-tlm_adjoint.hessian"]], "tlm_adjoint": [[31, "module-tlm_adjoint"], [53, "tlm-adjoint"]], "Submodules": [[31, "submodules"]], "tlm_adjoint.instructions": [[32, "module-tlm_adjoint.instructions"]], "tlm_adjoint.interface": [[33, "module-tlm_adjoint.interface"]], "tlm_adjoint.jax": [[34, "module-tlm_adjoint.jax"]], "tlm_adjoint.linear_equation": [[35, "module-tlm_adjoint.linear_equation"]], "tlm_adjoint.manager": [[36, "module-tlm_adjoint.manager"]], "tlm_adjoint.markers": [[37, "module-tlm_adjoint.markers"]], "tlm_adjoint.optimization": [[38, "module-tlm_adjoint.optimization"]], "tlm_adjoint.overloaded_float": [[39, "module-tlm_adjoint.overloaded_float"]], "tlm_adjoint.storage": [[40, "module-tlm_adjoint.storage"]], "tlm_adjoint.tangent_linear": [[41, "module-tlm_adjoint.tangent_linear"]], "tlm_adjoint.tlm_adjoint": [[42, "module-tlm_adjoint.tlm_adjoint"]], "tlm_adjoint.verification": [[43, "module-tlm_adjoint.verification"]], "Dependencies": [[44, "dependencies"]], "Required": [[44, "required"]], "Backend dependencies": [[44, "backend-dependencies"]], "Optional": [[44, "optional"]], "Getting started with tlm_adjoint": [[45, "Getting-started-with-tlm_adjoint"]], "A floating point example": [[45, "A-floating-point-example"]], "Adding tlm_adjoint": [[45, "Adding-tlm_adjoint"], [46, "Adding-tlm_adjoint"], [48, "Adding-tlm_adjoint"], [51, "Adding-tlm_adjoint"], [52, "Adding-tlm_adjoint"]], "Computing derivatives using an adjoint": [[45, "Computing-derivatives-using-an-adjoint"], [46, "Computing-derivatives-using-an-adjoint"], [48, "Computing-derivatives-using-an-adjoint"]], "Computing derivatives using a tangent-linear": [[45, "Computing-derivatives-using-a-tangent-linear"]], "Computing second derivatives using an adjoint of a tangent-linear": [[45, "Computing-second-derivatives-using-an-adjoint-of-a-tangent-linear"]], "Computing second derivatives using a tangent-linear of a tangent-linear": [[45, "Computing-second-derivatives-using-a-tangent-linear-of-a-tangent-linear"]], "Computing third derivatives using an adjoint of a tangent-linear of a tangent-linear": [[45, "Computing-third-derivatives-using-an-adjoint-of-a-tangent-linear-of-a-tangent-linear"]], "Higher order": [[45, "Higher-order"]], "Time-independent example": [[46, "Time-independent-example"]], "Forward problem": [[46, "Forward-problem"], [47, "Forward-problem"], [48, "Forward-problem"], [49, "Forward-problem"], [50, "Forward-problem"], [51, "Forward-problem"], [52, "Forward-problem"]], "Computing Hessian information using an adjoint of a tangent-linear": [[46, "Computing-Hessian-information-using-an-adjoint-of-a-tangent-linear"], [48, "Computing-Hessian-information-using-an-adjoint-of-a-tangent-linear"]], "Single direction": [[46, "Single-direction"]], "Multiple directions": [[46, "Multiple-directions"]], "Assembly and solver caching": [[46, "Assembly-and-solver-caching"]], "Using an EquationSolver": [[46, "Using-an-EquationSolver"]], "Flagging data for caching": [[46, "Flagging-data-for-caching"]], "The \u2018cache\u2019 flag": [[46, "The-'cache'-flag"]], "The \u2018static\u2019 flag": [[46, "The-'static'-flag"]], "Verifying derivative calculations": [[47, "Verifying-derivative-calculations"]], "Taylor remainder convergence testing: First order": [[47, "Taylor-remainder-convergence-testing:-First-order"]], "Taylor remainder convergence testing: Second order": [[47, "Taylor-remainder-convergence-testing:-Second-order"]], "Taylor remainder convergence testing: Higher order": [[47, "Taylor-remainder-convergence-testing:-Higher-order"]], "Time-dependent example": [[48, "Time-dependent-example"]], "Optimization": [[48, "Optimization"]], "Optimizing the annotation": [[48, "Optimizing-the-annotation"]], "Using a checkpointing schedule": [[48, "Using-a-checkpointing-schedule"]], "Computing derivatives": [[48, "Computing-derivatives"]], "Visualizing derivatives": [[49, "Visualizing-derivatives"]], "First order adjoint": [[49, "First-order-adjoint"]], "Hessian action": [[49, "Hessian-action"]], "Functional minimization": [[50, "Functional-minimization"]], "Inverse problem": [[50, "Inverse-problem"]], "Defining custom operations": [[51, "Defining-custom-operations"]], "Defining a custom Equation": [[51, "Defining-a-custom-Equation"]], "Equation methods": [[51, "Equation-methods"]], "__init__": [[51, "__init__"]], "forward_solve": [[51, "forward_solve"]], "adjoint_jacobian_solve": [[51, "adjoint_jacobian_solve"]], "adjoint_derivative_action": [[51, "adjoint_derivative_action"]], "tangent_linear": [[51, "tangent_linear"]], "Reference dropping": [[51, "Reference-dropping"]], "Defining custom operations using JAX": [[51, "Defining-custom-operations-using-JAX"]], "JAX integration": [[52, "JAX-integration"]], "Features": [[53, "features"]], "Examples": [[53, "examples"]], "Source and license": [[53, "source-and-license"]], "Indices": [[53, "indices"]]}, "indexentries": {"adjointblockrhs (class in tlm_adjoint.adjoint)": [[1, "tlm_adjoint.adjoint.AdjointBlockRHS"]], "adjointequationrhs (class in tlm_adjoint.adjoint)": [[1, "tlm_adjoint.adjoint.AdjointEquationRHS"]], "adjointmodelrhs (class in tlm_adjoint.adjoint)": [[1, "tlm_adjoint.adjoint.AdjointModelRHS"]], "adjointrhs (class in tlm_adjoint.adjoint)": [[1, "tlm_adjoint.adjoint.AdjointRHS"]], "b() (tlm_adjoint.adjoint.adjointequationrhs method)": [[1, "tlm_adjoint.adjoint.AdjointEquationRHS.B"], [1, "tlm_adjoint.adjoint.AdjointEquationRHS.b"]], "b() (tlm_adjoint.adjoint.adjointrhs method)": [[1, "tlm_adjoint.adjoint.AdjointRHS.b"]], "initialize() (tlm_adjoint.adjoint.adjointrhs method)": [[1, "tlm_adjoint.adjoint.AdjointRHS.initialize"]], "is_empty() (tlm_adjoint.adjoint.adjointblockrhs method)": [[1, "tlm_adjoint.adjoint.AdjointBlockRHS.is_empty"]], "is_empty() (tlm_adjoint.adjoint.adjointequationrhs method)": [[1, "tlm_adjoint.adjoint.AdjointEquationRHS.is_empty"]], "is_empty() (tlm_adjoint.adjoint.adjointmodelrhs method)": [[1, "tlm_adjoint.adjoint.AdjointModelRHS.is_empty"]], "is_empty() (tlm_adjoint.adjoint.adjointrhs method)": [[1, "tlm_adjoint.adjoint.AdjointRHS.is_empty"]], "module": [[1, "module-tlm_adjoint.adjoint"], [2, "module-tlm_adjoint.alias"], [3, "module-tlm_adjoint.cached_hessian"], [4, "module-tlm_adjoint.caches"], [5, "module-tlm_adjoint.checkpoint_schedules.binomial"], [6, "module-tlm_adjoint.checkpoint_schedules.h_revolve"], [7, "module-tlm_adjoint.checkpoint_schedules.memory"], [8, "module-tlm_adjoint.checkpoint_schedules.mixed"], [9, "module-tlm_adjoint.checkpoint_schedules.none"], [10, "module-tlm_adjoint.checkpoint_schedules.periodic"], [11, "module-tlm_adjoint.checkpoint_schedules.schedule"], [12, "module-tlm_adjoint.checkpointing"], [13, "module-tlm_adjoint.eigendecomposition"], [14, "module-tlm_adjoint.equation"], [15, "module-tlm_adjoint.equations"], [16, "module-tlm_adjoint.fenics.backend_interface"], [17, "module-tlm_adjoint.fenics.caches"], [18, "module-tlm_adjoint.fenics.equations"], [19, "module-tlm_adjoint.fenics.fenics_equations"], [20, "module-tlm_adjoint.fenics.functions"], [21, "module-tlm_adjoint.firedrake.backend_interface"], [22, "module-tlm_adjoint.firedrake.block_system"], [23, "module-tlm_adjoint.firedrake.caches"], [24, "module-tlm_adjoint.firedrake.equations"], [25, "module-tlm_adjoint.firedrake.firedrake_equations"], [26, "module-tlm_adjoint.firedrake.functions"], [27, "module-tlm_adjoint.firedrake.hessian_system"], [28, "module-tlm_adjoint.fixed_point"], [29, "module-tlm_adjoint.functional"], [30, "module-tlm_adjoint.hessian"], [31, "module-tlm_adjoint"], [32, "module-tlm_adjoint.instructions"], [33, "module-tlm_adjoint.interface"], [34, "module-tlm_adjoint.jax"], [35, "module-tlm_adjoint.linear_equation"], [36, "module-tlm_adjoint.manager"], [37, "module-tlm_adjoint.markers"], [38, "module-tlm_adjoint.optimization"], [39, "module-tlm_adjoint.overloaded_float"], [40, "module-tlm_adjoint.storage"], [41, "module-tlm_adjoint.tangent_linear"], [42, "module-tlm_adjoint.tlm_adjoint"], [43, "module-tlm_adjoint.verification"]], "pop() (tlm_adjoint.adjoint.adjointblockrhs method)": [[1, "tlm_adjoint.adjoint.AdjointBlockRHS.pop"]], "pop() (tlm_adjoint.adjoint.adjointmodelrhs method)": [[1, "tlm_adjoint.adjoint.AdjointModelRHS.pop"]], "sub() (tlm_adjoint.adjoint.adjointrhs method)": [[1, "tlm_adjoint.adjoint.AdjointRHS.sub"]], "tlm_adjoint.adjoint": [[1, "module-tlm_adjoint.adjoint"]], "alias (class in tlm_adjoint.alias)": [[2, "tlm_adjoint.alias.Alias"]], "weakalias (class in tlm_adjoint.alias)": [[2, "tlm_adjoint.alias.WeakAlias"]], "gc_disabled() (in module tlm_adjoint.alias)": [[2, "tlm_adjoint.alias.gc_disabled"]], "tlm_adjoint.alias": [[2, "module-tlm_adjoint.alias"]], "cachedgaussnewton (class in tlm_adjoint.cached_hessian)": [[3, "tlm_adjoint.cached_hessian.CachedGaussNewton"]], "cachedhessian (class in tlm_adjoint.cached_hessian)": [[3, "tlm_adjoint.cached_hessian.CachedHessian"]], "action() (tlm_adjoint.cached_hessian.cachedgaussnewton method)": [[3, "tlm_adjoint.cached_hessian.CachedGaussNewton.action"]], "action() (tlm_adjoint.cached_hessian.cachedhessian method)": [[3, "tlm_adjoint.cached_hessian.CachedHessian.action"]], "compute_gradient() (tlm_adjoint.cached_hessian.cachedhessian method)": [[3, "tlm_adjoint.cached_hessian.CachedHessian.compute_gradient"]], "tlm_adjoint.cached_hessian": [[3, "module-tlm_adjoint.cached_hessian"]], "cache (class in tlm_adjoint.caches)": [[4, "tlm_adjoint.caches.Cache"]], "cacheref (class in tlm_adjoint.caches)": [[4, "tlm_adjoint.caches.CacheRef"]], "caches (class in tlm_adjoint.caches)": [[4, "tlm_adjoint.caches.Caches"]], "add() (tlm_adjoint.caches.cache method)": [[4, "tlm_adjoint.caches.Cache.add"]], "add() (tlm_adjoint.caches.caches method)": [[4, "tlm_adjoint.caches.Caches.add"]], "clear() (tlm_adjoint.caches.cache method)": [[4, "tlm_adjoint.caches.Cache.clear"]], "clear() (tlm_adjoint.caches.cacheref method)": [[4, "tlm_adjoint.caches.CacheRef.clear"]], "clear() (tlm_adjoint.caches.caches method)": [[4, "tlm_adjoint.caches.Caches.clear"]], "clear_caches() (in module tlm_adjoint.caches)": [[4, "tlm_adjoint.caches.clear_caches"]], "get() (tlm_adjoint.caches.cache method)": [[4, "tlm_adjoint.caches.Cache.get"]], "id (tlm_adjoint.caches.cache property)": [[4, "tlm_adjoint.caches.Cache.id"]], "local_caches() (in module tlm_adjoint.caches)": [[4, "tlm_adjoint.caches.local_caches"]], "remove() (tlm_adjoint.caches.caches method)": [[4, "tlm_adjoint.caches.Caches.remove"]], "tlm_adjoint.caches": [[4, "module-tlm_adjoint.caches"]], "update() (tlm_adjoint.caches.caches method)": [[4, "tlm_adjoint.caches.Caches.update"]], "multistagecheckpointschedule (class in tlm_adjoint.checkpoint_schedules.binomial)": [[5, "tlm_adjoint.checkpoint_schedules.binomial.MultistageCheckpointSchedule"]], "twolevelcheckpointschedule (class in tlm_adjoint.checkpoint_schedules.binomial)": [[5, "tlm_adjoint.checkpoint_schedules.binomial.TwoLevelCheckpointSchedule"]], "is_exhausted (tlm_adjoint.checkpoint_schedules.binomial.multistagecheckpointschedule property)": [[5, "tlm_adjoint.checkpoint_schedules.binomial.MultistageCheckpointSchedule.is_exhausted"]], "is_exhausted (tlm_adjoint.checkpoint_schedules.binomial.twolevelcheckpointschedule property)": [[5, "tlm_adjoint.checkpoint_schedules.binomial.TwoLevelCheckpointSchedule.is_exhausted"]], "iter() (tlm_adjoint.checkpoint_schedules.binomial.multistagecheckpointschedule method)": [[5, "tlm_adjoint.checkpoint_schedules.binomial.MultistageCheckpointSchedule.iter"]], "iter() (tlm_adjoint.checkpoint_schedules.binomial.twolevelcheckpointschedule method)": [[5, "tlm_adjoint.checkpoint_schedules.binomial.TwoLevelCheckpointSchedule.iter"]], "tlm_adjoint.checkpoint_schedules.binomial": [[5, "module-tlm_adjoint.checkpoint_schedules.binomial"]], "uses_disk_storage (tlm_adjoint.checkpoint_schedules.binomial.multistagecheckpointschedule property)": [[5, "tlm_adjoint.checkpoint_schedules.binomial.MultistageCheckpointSchedule.uses_disk_storage"]], "uses_disk_storage (tlm_adjoint.checkpoint_schedules.binomial.twolevelcheckpointschedule property)": [[5, "tlm_adjoint.checkpoint_schedules.binomial.TwoLevelCheckpointSchedule.uses_disk_storage"]], "hrevolvecheckpointschedule (class in tlm_adjoint.checkpoint_schedules.h_revolve)": [[6, "tlm_adjoint.checkpoint_schedules.h_revolve.HRevolveCheckpointSchedule"]], "is_exhausted (tlm_adjoint.checkpoint_schedules.h_revolve.hrevolvecheckpointschedule property)": [[6, "tlm_adjoint.checkpoint_schedules.h_revolve.HRevolveCheckpointSchedule.is_exhausted"]], "iter() (tlm_adjoint.checkpoint_schedules.h_revolve.hrevolvecheckpointschedule method)": [[6, "tlm_adjoint.checkpoint_schedules.h_revolve.HRevolveCheckpointSchedule.iter"]], "tlm_adjoint.checkpoint_schedules.h_revolve": [[6, "module-tlm_adjoint.checkpoint_schedules.h_revolve"]], "uses_disk_storage (tlm_adjoint.checkpoint_schedules.h_revolve.hrevolvecheckpointschedule property)": [[6, "tlm_adjoint.checkpoint_schedules.h_revolve.HRevolveCheckpointSchedule.uses_disk_storage"]], "memorycheckpointschedule (class in tlm_adjoint.checkpoint_schedules.memory)": [[7, "tlm_adjoint.checkpoint_schedules.memory.MemoryCheckpointSchedule"]], "is_exhausted (tlm_adjoint.checkpoint_schedules.memory.memorycheckpointschedule property)": [[7, "tlm_adjoint.checkpoint_schedules.memory.MemoryCheckpointSchedule.is_exhausted"]], "iter() (tlm_adjoint.checkpoint_schedules.memory.memorycheckpointschedule method)": [[7, "tlm_adjoint.checkpoint_schedules.memory.MemoryCheckpointSchedule.iter"]], "tlm_adjoint.checkpoint_schedules.memory": [[7, "module-tlm_adjoint.checkpoint_schedules.memory"]], "uses_disk_storage (tlm_adjoint.checkpoint_schedules.memory.memorycheckpointschedule property)": [[7, "tlm_adjoint.checkpoint_schedules.memory.MemoryCheckpointSchedule.uses_disk_storage"]], "mixedcheckpointschedule (class in tlm_adjoint.checkpoint_schedules.mixed)": [[8, "tlm_adjoint.checkpoint_schedules.mixed.MixedCheckpointSchedule"]], "is_exhausted (tlm_adjoint.checkpoint_schedules.mixed.mixedcheckpointschedule property)": [[8, "tlm_adjoint.checkpoint_schedules.mixed.MixedCheckpointSchedule.is_exhausted"]], "iter() (tlm_adjoint.checkpoint_schedules.mixed.mixedcheckpointschedule method)": [[8, "tlm_adjoint.checkpoint_schedules.mixed.MixedCheckpointSchedule.iter"]], "tlm_adjoint.checkpoint_schedules.mixed": [[8, "module-tlm_adjoint.checkpoint_schedules.mixed"]], "uses_disk_storage (tlm_adjoint.checkpoint_schedules.mixed.mixedcheckpointschedule property)": [[8, "tlm_adjoint.checkpoint_schedules.mixed.MixedCheckpointSchedule.uses_disk_storage"]], "nonecheckpointschedule (class in tlm_adjoint.checkpoint_schedules.none)": [[9, "tlm_adjoint.checkpoint_schedules.none.NoneCheckpointSchedule"]], "is_exhausted (tlm_adjoint.checkpoint_schedules.none.nonecheckpointschedule property)": [[9, "tlm_adjoint.checkpoint_schedules.none.NoneCheckpointSchedule.is_exhausted"]], "iter() (tlm_adjoint.checkpoint_schedules.none.nonecheckpointschedule method)": [[9, "tlm_adjoint.checkpoint_schedules.none.NoneCheckpointSchedule.iter"]], "tlm_adjoint.checkpoint_schedules.none": [[9, "module-tlm_adjoint.checkpoint_schedules.none"]], "uses_disk_storage (tlm_adjoint.checkpoint_schedules.none.nonecheckpointschedule property)": [[9, "tlm_adjoint.checkpoint_schedules.none.NoneCheckpointSchedule.uses_disk_storage"]], "periodicdiskcheckpointschedule (class in tlm_adjoint.checkpoint_schedules.periodic)": [[10, "tlm_adjoint.checkpoint_schedules.periodic.PeriodicDiskCheckpointSchedule"]], "is_exhausted (tlm_adjoint.checkpoint_schedules.periodic.periodicdiskcheckpointschedule property)": [[10, "tlm_adjoint.checkpoint_schedules.periodic.PeriodicDiskCheckpointSchedule.is_exhausted"]], "iter() (tlm_adjoint.checkpoint_schedules.periodic.periodicdiskcheckpointschedule method)": [[10, "tlm_adjoint.checkpoint_schedules.periodic.PeriodicDiskCheckpointSchedule.iter"]], "tlm_adjoint.checkpoint_schedules.periodic": [[10, "module-tlm_adjoint.checkpoint_schedules.periodic"]], "uses_disk_storage (tlm_adjoint.checkpoint_schedules.periodic.periodicdiskcheckpointschedule property)": [[10, "tlm_adjoint.checkpoint_schedules.periodic.PeriodicDiskCheckpointSchedule.uses_disk_storage"]], "checkpointaction (class in tlm_adjoint.checkpoint_schedules.schedule)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointAction"]], "checkpointschedule (class in tlm_adjoint.checkpoint_schedules.schedule)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule"]], "clear (class in tlm_adjoint.checkpoint_schedules.schedule)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Clear"]], "configure (class in tlm_adjoint.checkpoint_schedules.schedule)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Configure"]], "endforward (class in tlm_adjoint.checkpoint_schedules.schedule)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.EndForward"]], "endreverse (class in tlm_adjoint.checkpoint_schedules.schedule)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.EndReverse"]], "forward (class in tlm_adjoint.checkpoint_schedules.schedule)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Forward"]], "read (class in tlm_adjoint.checkpoint_schedules.schedule)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Read"]], "reverse (class in tlm_adjoint.checkpoint_schedules.schedule)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Reverse"]], "write (class in tlm_adjoint.checkpoint_schedules.schedule)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Write"]], "args (tlm_adjoint.checkpoint_schedules.schedule.checkpointaction property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointAction.args"]], "clear_data (tlm_adjoint.checkpoint_schedules.schedule.clear property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Clear.clear_data"]], "clear_ics (tlm_adjoint.checkpoint_schedules.schedule.clear property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Clear.clear_ics"]], "delete (tlm_adjoint.checkpoint_schedules.schedule.read property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Read.delete"]], "exhausted (tlm_adjoint.checkpoint_schedules.schedule.endreverse property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.EndReverse.exhausted"]], "finalize() (tlm_adjoint.checkpoint_schedules.schedule.checkpointschedule method)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule.finalize"]], "is_exhausted (tlm_adjoint.checkpoint_schedules.schedule.checkpointschedule property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule.is_exhausted"]], "is_running (tlm_adjoint.checkpoint_schedules.schedule.checkpointschedule property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule.is_running"]], "iter() (tlm_adjoint.checkpoint_schedules.schedule.checkpointschedule method)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule.iter"]], "max_n (tlm_adjoint.checkpoint_schedules.schedule.checkpointschedule property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule.max_n"]], "n (tlm_adjoint.checkpoint_schedules.schedule.checkpointschedule property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule.n"]], "n (tlm_adjoint.checkpoint_schedules.schedule.read property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Read.n"]], "n (tlm_adjoint.checkpoint_schedules.schedule.write property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Write.n"]], "n0 (tlm_adjoint.checkpoint_schedules.schedule.forward property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Forward.n0"]], "n0 (tlm_adjoint.checkpoint_schedules.schedule.reverse property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Reverse.n0"]], "n1 (tlm_adjoint.checkpoint_schedules.schedule.forward property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Forward.n1"]], "n1 (tlm_adjoint.checkpoint_schedules.schedule.reverse property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Reverse.n1"]], "r (tlm_adjoint.checkpoint_schedules.schedule.checkpointschedule property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule.r"]], "storage (tlm_adjoint.checkpoint_schedules.schedule.read property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Read.storage"]], "storage (tlm_adjoint.checkpoint_schedules.schedule.write property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Write.storage"]], "store_data (tlm_adjoint.checkpoint_schedules.schedule.configure property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Configure.store_data"]], "store_ics (tlm_adjoint.checkpoint_schedules.schedule.configure property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.Configure.store_ics"]], "tlm_adjoint.checkpoint_schedules.schedule": [[11, "module-tlm_adjoint.checkpoint_schedules.schedule"]], "uses_disk_storage (tlm_adjoint.checkpoint_schedules.schedule.checkpointschedule property)": [[11, "tlm_adjoint.checkpoint_schedules.schedule.CheckpointSchedule.uses_disk_storage"]], "checkpointstorage (class in tlm_adjoint.checkpointing)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage"]], "checkpoints (class in tlm_adjoint.checkpointing)": [[12, "tlm_adjoint.checkpointing.Checkpoints"]], "hdf5checkpoints (class in tlm_adjoint.checkpointing)": [[12, "tlm_adjoint.checkpointing.HDF5Checkpoints"]], "picklecheckpoints (class in tlm_adjoint.checkpointing)": [[12, "tlm_adjoint.checkpointing.PickleCheckpoints"]], "replaystorage (class in tlm_adjoint.checkpointing)": [[12, "tlm_adjoint.checkpointing.ReplayStorage"]], "add_equation() (tlm_adjoint.checkpointing.checkpointstorage method)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.add_equation"]], "add_equation_data() (tlm_adjoint.checkpointing.checkpointstorage method)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.add_equation_data"]], "add_initial_condition() (tlm_adjoint.checkpointing.checkpointstorage method)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.add_initial_condition"]], "checkpoint_data() (tlm_adjoint.checkpointing.checkpointstorage method)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.checkpoint_data"]], "clear() (tlm_adjoint.checkpointing.checkpointstorage method)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.clear"]], "configure() (tlm_adjoint.checkpointing.checkpointstorage method)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.configure"]], "delete() (tlm_adjoint.checkpointing.checkpoints method)": [[12, "tlm_adjoint.checkpointing.Checkpoints.delete"]], "delete() (tlm_adjoint.checkpointing.hdf5checkpoints method)": [[12, "tlm_adjoint.checkpointing.HDF5Checkpoints.delete"]], "delete() (tlm_adjoint.checkpointing.picklecheckpoints method)": [[12, "tlm_adjoint.checkpointing.PickleCheckpoints.delete"]], "initial_condition() (tlm_adjoint.checkpointing.checkpointstorage method)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.initial_condition"]], "initial_conditions() (tlm_adjoint.checkpointing.checkpointstorage method)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.initial_conditions"]], "is_active() (tlm_adjoint.checkpointing.replaystorage method)": [[12, "tlm_adjoint.checkpointing.ReplayStorage.is_active"]], "popleft() (tlm_adjoint.checkpointing.replaystorage method)": [[12, "tlm_adjoint.checkpointing.ReplayStorage.popleft"]], "read() (tlm_adjoint.checkpointing.checkpoints method)": [[12, "tlm_adjoint.checkpointing.Checkpoints.read"]], "read() (tlm_adjoint.checkpointing.hdf5checkpoints method)": [[12, "tlm_adjoint.checkpointing.HDF5Checkpoints.read"]], "read() (tlm_adjoint.checkpointing.picklecheckpoints method)": [[12, "tlm_adjoint.checkpointing.PickleCheckpoints.read"]], "store_data (tlm_adjoint.checkpointing.checkpointstorage property)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.store_data"]], "store_ics (tlm_adjoint.checkpointing.checkpointstorage property)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.store_ics"]], "tlm_adjoint.checkpointing": [[12, "module-tlm_adjoint.checkpointing"]], "update() (tlm_adjoint.checkpointing.checkpointstorage method)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.update"]], "update() (tlm_adjoint.checkpointing.replaystorage method)": [[12, "tlm_adjoint.checkpointing.ReplayStorage.update"]], "update_keys() (tlm_adjoint.checkpointing.checkpointstorage method)": [[12, "tlm_adjoint.checkpointing.CheckpointStorage.update_keys"]], "write() (tlm_adjoint.checkpointing.checkpoints method)": [[12, "tlm_adjoint.checkpointing.Checkpoints.write"]], "write() (tlm_adjoint.checkpointing.hdf5checkpoints method)": [[12, "tlm_adjoint.checkpointing.HDF5Checkpoints.write"]], "write() (tlm_adjoint.checkpointing.picklecheckpoints method)": [[12, "tlm_adjoint.checkpointing.PickleCheckpoints.write"]], "eigendecompose() (in module tlm_adjoint.eigendecomposition)": [[13, "tlm_adjoint.eigendecomposition.eigendecompose"]], "tlm_adjoint.eigendecomposition": [[13, "module-tlm_adjoint.eigendecomposition"]], "equation (class in tlm_adjoint.equation)": [[14, "tlm_adjoint.equation.Equation"]], "nullsolver (class in tlm_adjoint.equation)": [[14, "tlm_adjoint.equation.NullSolver"]], "x() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.X"], [14, "tlm_adjoint.equation.Equation.x"]], "zeroassignment (class in tlm_adjoint.equation)": [[14, "tlm_adjoint.equation.ZeroAssignment"]], "adj_x_type() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.adj_X_type"], [14, "tlm_adjoint.equation.Equation.adj_x_type"]], "adjoint() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.adjoint"]], "adjoint_cached() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.adjoint_cached"]], "adjoint_derivative_action() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.adjoint_derivative_action"]], "adjoint_derivative_action() (tlm_adjoint.equation.zeroassignment method)": [[14, "tlm_adjoint.equation.ZeroAssignment.adjoint_derivative_action"]], "adjoint_initial_condition_dependencies() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.adjoint_initial_condition_dependencies"]], "adjoint_jacobian_solve() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.equation.zeroassignment method)": [[14, "tlm_adjoint.equation.ZeroAssignment.adjoint_jacobian_solve"]], "dependencies() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.dependencies"]], "drop_references() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.drop_references"]], "forward() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.forward"]], "forward_solve() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.forward_solve"]], "forward_solve() (tlm_adjoint.equation.zeroassignment method)": [[14, "tlm_adjoint.equation.ZeroAssignment.forward_solve"]], "initial_condition_dependencies() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.initial_condition_dependencies"]], "new_adj_x() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.new_adj_X"], [14, "tlm_adjoint.equation.Equation.new_adj_x"]], "nonlinear_dependencies() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.nonlinear_dependencies"]], "solve() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.solve"]], "subtract_adjoint_derivative_actions() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.subtract_adjoint_derivative_actions"]], "tangent_linear() (tlm_adjoint.equation.equation method)": [[14, "tlm_adjoint.equation.Equation.tangent_linear"]], "tangent_linear() (tlm_adjoint.equation.zeroassignment method)": [[14, "tlm_adjoint.equation.ZeroAssignment.tangent_linear"]], "tlm_adjoint.equation": [[14, "module-tlm_adjoint.equation"]], "assignment (class in tlm_adjoint.equations)": [[15, "tlm_adjoint.equations.Assignment"]], "axpy (class in tlm_adjoint.equations)": [[15, "tlm_adjoint.equations.Axpy"]], "conversion (class in tlm_adjoint.equations)": [[15, "tlm_adjoint.equations.Conversion"]], "dotproduct (class in tlm_adjoint.equations)": [[15, "tlm_adjoint.equations.DotProduct"]], "dotproductrhs (class in tlm_adjoint.equations)": [[15, "tlm_adjoint.equations.DotProductRHS"]], "emptyequation (class in tlm_adjoint.equations)": [[15, "tlm_adjoint.equations.EmptyEquation"]], "innerproduct (class in tlm_adjoint.equations)": [[15, "tlm_adjoint.equations.InnerProduct"]], "innerproductrhs (class in tlm_adjoint.equations)": [[15, "tlm_adjoint.equations.InnerProductRHS"]], "linearcombination (class in tlm_adjoint.equations)": [[15, "tlm_adjoint.equations.LinearCombination"]], "matrixactionrhs (class in tlm_adjoint.equations)": [[15, "tlm_adjoint.equations.MatrixActionRHS"]], "add_forward() (tlm_adjoint.equations.dotproductrhs method)": [[15, "tlm_adjoint.equations.DotProductRHS.add_forward"]], "add_forward() (tlm_adjoint.equations.innerproductrhs method)": [[15, "tlm_adjoint.equations.InnerProductRHS.add_forward"]], "add_forward() (tlm_adjoint.equations.matrixactionrhs method)": [[15, "tlm_adjoint.equations.MatrixActionRHS.add_forward"]], "adjoint_derivative_action() (tlm_adjoint.equations.assignment method)": [[15, "tlm_adjoint.equations.Assignment.adjoint_derivative_action"]], "adjoint_derivative_action() (tlm_adjoint.equations.conversion method)": [[15, "tlm_adjoint.equations.Conversion.adjoint_derivative_action"]], "adjoint_derivative_action() (tlm_adjoint.equations.linearcombination method)": [[15, "tlm_adjoint.equations.LinearCombination.adjoint_derivative_action"]], "adjoint_jacobian_solve() (tlm_adjoint.equations.assignment method)": [[15, "tlm_adjoint.equations.Assignment.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.equations.conversion method)": [[15, "tlm_adjoint.equations.Conversion.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.equations.linearcombination method)": [[15, "tlm_adjoint.equations.LinearCombination.adjoint_jacobian_solve"]], "drop_references() (tlm_adjoint.equations.dotproductrhs method)": [[15, "tlm_adjoint.equations.DotProductRHS.drop_references"]], "drop_references() (tlm_adjoint.equations.innerproductrhs method)": [[15, "tlm_adjoint.equations.InnerProductRHS.drop_references"]], "drop_references() (tlm_adjoint.equations.matrixactionrhs method)": [[15, "tlm_adjoint.equations.MatrixActionRHS.drop_references"]], "forward_solve() (tlm_adjoint.equations.assignment method)": [[15, "tlm_adjoint.equations.Assignment.forward_solve"]], "forward_solve() (tlm_adjoint.equations.conversion method)": [[15, "tlm_adjoint.equations.Conversion.forward_solve"]], "forward_solve() (tlm_adjoint.equations.emptyequation method)": [[15, "tlm_adjoint.equations.EmptyEquation.forward_solve"]], "forward_solve() (tlm_adjoint.equations.linearcombination method)": [[15, "tlm_adjoint.equations.LinearCombination.forward_solve"]], "subtract_adjoint_derivative_action() (tlm_adjoint.equations.dotproductrhs method)": [[15, "tlm_adjoint.equations.DotProductRHS.subtract_adjoint_derivative_action"]], "subtract_adjoint_derivative_action() (tlm_adjoint.equations.innerproductrhs method)": [[15, "tlm_adjoint.equations.InnerProductRHS.subtract_adjoint_derivative_action"]], "subtract_adjoint_derivative_action() (tlm_adjoint.equations.matrixactionrhs method)": [[15, "tlm_adjoint.equations.MatrixActionRHS.subtract_adjoint_derivative_action"]], "tangent_linear() (tlm_adjoint.equations.assignment method)": [[15, "tlm_adjoint.equations.Assignment.tangent_linear"]], "tangent_linear() (tlm_adjoint.equations.conversion method)": [[15, "tlm_adjoint.equations.Conversion.tangent_linear"]], "tangent_linear() (tlm_adjoint.equations.linearcombination method)": [[15, "tlm_adjoint.equations.LinearCombination.tangent_linear"]], "tangent_linear_rhs() (tlm_adjoint.equations.dotproductrhs method)": [[15, "tlm_adjoint.equations.DotProductRHS.tangent_linear_rhs"]], "tangent_linear_rhs() (tlm_adjoint.equations.innerproductrhs method)": [[15, "tlm_adjoint.equations.InnerProductRHS.tangent_linear_rhs"]], "tangent_linear_rhs() (tlm_adjoint.equations.matrixactionrhs method)": [[15, "tlm_adjoint.equations.MatrixActionRHS.tangent_linear_rhs"]], "tlm_adjoint.equations": [[15, "module-tlm_adjoint.equations"]], "function (class in tlm_adjoint.fenics.backend_interface)": [[16, "tlm_adjoint.fenics.backend_interface.Function"]], "zerofunction (class in tlm_adjoint.fenics.backend_interface)": [[16, "tlm_adjoint.fenics.backend_interface.ZeroFunction"]], "tlm_adjoint.fenics.backend_interface": [[16, "module-tlm_adjoint.fenics.backend_interface"]], "to_fenics() (in module tlm_adjoint.fenics.backend_interface)": [[16, "tlm_adjoint.fenics.backend_interface.to_fenics"]], "assemblycache (class in tlm_adjoint.fenics.caches)": [[17, "tlm_adjoint.fenics.caches.AssemblyCache"]], "linearsolvercache (class in tlm_adjoint.fenics.caches)": [[17, "tlm_adjoint.fenics.caches.LinearSolverCache"]], "assemble() (tlm_adjoint.fenics.caches.assemblycache method)": [[17, "tlm_adjoint.fenics.caches.AssemblyCache.assemble"]], "assembly_cache() (in module tlm_adjoint.fenics.caches)": [[17, "tlm_adjoint.fenics.caches.assembly_cache"]], "linear_solver() (tlm_adjoint.fenics.caches.linearsolvercache method)": [[17, "tlm_adjoint.fenics.caches.LinearSolverCache.linear_solver"]], "linear_solver_cache() (in module tlm_adjoint.fenics.caches)": [[17, "tlm_adjoint.fenics.caches.linear_solver_cache"]], "set_assembly_cache() (in module tlm_adjoint.fenics.caches)": [[17, "tlm_adjoint.fenics.caches.set_assembly_cache"]], "set_linear_solver_cache() (in module tlm_adjoint.fenics.caches)": [[17, "tlm_adjoint.fenics.caches.set_linear_solver_cache"]], "tlm_adjoint.fenics.caches": [[17, "module-tlm_adjoint.fenics.caches"]], "assembly (class in tlm_adjoint.fenics.equations)": [[18, "tlm_adjoint.fenics.equations.Assembly"]], "dirichletbcapplication (class in tlm_adjoint.fenics.equations)": [[18, "tlm_adjoint.fenics.equations.DirichletBCApplication"]], "equationsolver (class in tlm_adjoint.fenics.equations)": [[18, "tlm_adjoint.fenics.equations.EquationSolver"]], "exprinterpolation (class in tlm_adjoint.fenics.equations)": [[18, "tlm_adjoint.fenics.equations.ExprInterpolation"]], "projection (class in tlm_adjoint.fenics.equations)": [[18, "tlm_adjoint.fenics.equations.Projection"]], "adjoint_derivative_action() (tlm_adjoint.fenics.equations.assembly method)": [[18, "tlm_adjoint.fenics.equations.Assembly.adjoint_derivative_action"]], "adjoint_derivative_action() (tlm_adjoint.fenics.equations.dirichletbcapplication method)": [[18, "tlm_adjoint.fenics.equations.DirichletBCApplication.adjoint_derivative_action"]], "adjoint_derivative_action() (tlm_adjoint.fenics.equations.exprinterpolation method)": [[18, "tlm_adjoint.fenics.equations.ExprInterpolation.adjoint_derivative_action"]], "adjoint_jacobian_solve() (tlm_adjoint.fenics.equations.assembly method)": [[18, "tlm_adjoint.fenics.equations.Assembly.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.fenics.equations.dirichletbcapplication method)": [[18, "tlm_adjoint.fenics.equations.DirichletBCApplication.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.fenics.equations.equationsolver method)": [[18, "tlm_adjoint.fenics.equations.EquationSolver.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.fenics.equations.exprinterpolation method)": [[18, "tlm_adjoint.fenics.equations.ExprInterpolation.adjoint_jacobian_solve"]], "drop_references() (tlm_adjoint.fenics.equations.assembly method)": [[18, "tlm_adjoint.fenics.equations.Assembly.drop_references"]], "drop_references() (tlm_adjoint.fenics.equations.equationsolver method)": [[18, "tlm_adjoint.fenics.equations.EquationSolver.drop_references"]], "drop_references() (tlm_adjoint.fenics.equations.exprinterpolation method)": [[18, "tlm_adjoint.fenics.equations.ExprInterpolation.drop_references"]], "expr_new_x() (in module tlm_adjoint.fenics.equations)": [[18, "tlm_adjoint.fenics.equations.expr_new_x"]], "forward_solve() (tlm_adjoint.fenics.equations.assembly method)": [[18, "tlm_adjoint.fenics.equations.Assembly.forward_solve"]], "forward_solve() (tlm_adjoint.fenics.equations.dirichletbcapplication method)": [[18, "tlm_adjoint.fenics.equations.DirichletBCApplication.forward_solve"]], "forward_solve() (tlm_adjoint.fenics.equations.equationsolver method)": [[18, "tlm_adjoint.fenics.equations.EquationSolver.forward_solve"]], "forward_solve() (tlm_adjoint.fenics.equations.exprinterpolation method)": [[18, "tlm_adjoint.fenics.equations.ExprInterpolation.forward_solve"]], "linear_equation_new_x() (in module tlm_adjoint.fenics.equations)": [[18, "tlm_adjoint.fenics.equations.linear_equation_new_x"]], "subtract_adjoint_derivative_actions() (tlm_adjoint.fenics.equations.equationsolver method)": [[18, "tlm_adjoint.fenics.equations.EquationSolver.subtract_adjoint_derivative_actions"]], "tangent_linear() (tlm_adjoint.fenics.equations.assembly method)": [[18, "tlm_adjoint.fenics.equations.Assembly.tangent_linear"]], "tangent_linear() (tlm_adjoint.fenics.equations.dirichletbcapplication method)": [[18, "tlm_adjoint.fenics.equations.DirichletBCApplication.tangent_linear"]], "tangent_linear() (tlm_adjoint.fenics.equations.equationsolver method)": [[18, "tlm_adjoint.fenics.equations.EquationSolver.tangent_linear"]], "tangent_linear() (tlm_adjoint.fenics.equations.exprinterpolation method)": [[18, "tlm_adjoint.fenics.equations.ExprInterpolation.tangent_linear"]], "tlm_adjoint.fenics.equations": [[18, "module-tlm_adjoint.fenics.equations"]], "interpolation (class in tlm_adjoint.fenics.fenics_equations)": [[19, "tlm_adjoint.fenics.fenics_equations.Interpolation"]], "localprojection (class in tlm_adjoint.fenics.fenics_equations)": [[19, "tlm_adjoint.fenics.fenics_equations.LocalProjection"]], "localsolvercache (class in tlm_adjoint.fenics.fenics_equations)": [[19, "tlm_adjoint.fenics.fenics_equations.LocalSolverCache"]], "pointinterpolation (class in tlm_adjoint.fenics.fenics_equations)": [[19, "tlm_adjoint.fenics.fenics_equations.PointInterpolation"]], "adjoint_derivative_action() (tlm_adjoint.fenics.fenics_equations.pointinterpolation method)": [[19, "tlm_adjoint.fenics.fenics_equations.PointInterpolation.adjoint_derivative_action"]], "adjoint_jacobian_solve() (tlm_adjoint.fenics.fenics_equations.localprojection method)": [[19, "tlm_adjoint.fenics.fenics_equations.LocalProjection.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.fenics.fenics_equations.pointinterpolation method)": [[19, "tlm_adjoint.fenics.fenics_equations.PointInterpolation.adjoint_jacobian_solve"]], "forward_solve() (tlm_adjoint.fenics.fenics_equations.localprojection method)": [[19, "tlm_adjoint.fenics.fenics_equations.LocalProjection.forward_solve"]], "forward_solve() (tlm_adjoint.fenics.fenics_equations.pointinterpolation method)": [[19, "tlm_adjoint.fenics.fenics_equations.PointInterpolation.forward_solve"]], "local_solver() (tlm_adjoint.fenics.fenics_equations.localsolvercache method)": [[19, "tlm_adjoint.fenics.fenics_equations.LocalSolverCache.local_solver"]], "local_solver_cache() (in module tlm_adjoint.fenics.fenics_equations)": [[19, "tlm_adjoint.fenics.fenics_equations.local_solver_cache"]], "set_local_solver_cache() (in module tlm_adjoint.fenics.fenics_equations)": [[19, "tlm_adjoint.fenics.fenics_equations.set_local_solver_cache"]], "tangent_linear() (tlm_adjoint.fenics.fenics_equations.localprojection method)": [[19, "tlm_adjoint.fenics.fenics_equations.LocalProjection.tangent_linear"]], "tangent_linear() (tlm_adjoint.fenics.fenics_equations.pointinterpolation method)": [[19, "tlm_adjoint.fenics.fenics_equations.PointInterpolation.tangent_linear"]], "tlm_adjoint.fenics.fenics_equations": [[19, "module-tlm_adjoint.fenics.fenics_equations"]], "constant (class in tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.Constant"]], "dirichletbc (class in tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.DirichletBC"]], "homogeneousdirichletbc (class in tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.HomogeneousDirichletBC"]], "replacement (class in tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.Replacement"]], "replacementconstant (class in tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.ReplacementConstant"]], "replacementfunction (class in tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.ReplacementFunction"]], "replacementzeroconstant (class in tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.ReplacementZeroConstant"]], "replacementzerofunction (class in tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.ReplacementZeroFunction"]], "zero (class in tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.Zero"]], "zeroconstant (class in tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.ZeroConstant"]], "eliminate_zeros() (in module tlm_adjoint.fenics.functions)": [[20, "tlm_adjoint.fenics.functions.eliminate_zeros"]], "tlm_adjoint.fenics.functions": [[20, "module-tlm_adjoint.fenics.functions"]], "cofunction (class in tlm_adjoint.firedrake.backend_interface)": [[21, "tlm_adjoint.firedrake.backend_interface.Cofunction"]], "function (class in tlm_adjoint.firedrake.backend_interface)": [[21, "tlm_adjoint.firedrake.backend_interface.Function"]], "replacementcofunction (class in tlm_adjoint.firedrake.backend_interface)": [[21, "tlm_adjoint.firedrake.backend_interface.ReplacementCofunction"]], "zerofunction (class in tlm_adjoint.firedrake.backend_interface)": [[21, "tlm_adjoint.firedrake.backend_interface.ZeroFunction"]], "equals() (tlm_adjoint.firedrake.backend_interface.cofunction method)": [[21, "tlm_adjoint.firedrake.backend_interface.Cofunction.equals"]], "tlm_adjoint.firedrake.backend_interface": [[21, "module-tlm_adjoint.firedrake.backend_interface"]], "to_firedrake() (in module tlm_adjoint.firedrake.backend_interface)": [[21, "tlm_adjoint.firedrake.backend_interface.to_firedrake"]], "blockmatrix (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.BlockMatrix"]], "blocknullspace (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.BlockNullspace"]], "constantnullspace (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.ConstantNullspace"]], "dirichletbcnullspace (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.DirichletBCNullspace"]], "matrix (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.Matrix"]], "mixedspace (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.MixedSpace"]], "nonenullspace (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.NoneNullspace"]], "nullspace (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace"]], "petscmatrix (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.PETScMatrix"]], "system (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.System"]], "unitynullspace (class in tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.UnityNullspace"]], "action_space (tlm_adjoint.firedrake.block_system.matrix property)": [[22, "tlm_adjoint.firedrake.block_system.Matrix.action_space"]], "apply_nullspace_transformation_lhs_left() (tlm_adjoint.firedrake.block_system.blocknullspace method)": [[22, "tlm_adjoint.firedrake.block_system.BlockNullspace.apply_nullspace_transformation_lhs_left"]], "apply_nullspace_transformation_lhs_left() (tlm_adjoint.firedrake.block_system.constantnullspace method)": [[22, "tlm_adjoint.firedrake.block_system.ConstantNullspace.apply_nullspace_transformation_lhs_left"]], "apply_nullspace_transformation_lhs_left() (tlm_adjoint.firedrake.block_system.dirichletbcnullspace method)": [[22, "tlm_adjoint.firedrake.block_system.DirichletBCNullspace.apply_nullspace_transformation_lhs_left"]], "apply_nullspace_transformation_lhs_left() (tlm_adjoint.firedrake.block_system.nonenullspace method)": [[22, "tlm_adjoint.firedrake.block_system.NoneNullspace.apply_nullspace_transformation_lhs_left"]], "apply_nullspace_transformation_lhs_left() (tlm_adjoint.firedrake.block_system.nullspace method)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace.apply_nullspace_transformation_lhs_left"]], "apply_nullspace_transformation_lhs_left() (tlm_adjoint.firedrake.block_system.unitynullspace method)": [[22, "tlm_adjoint.firedrake.block_system.UnityNullspace.apply_nullspace_transformation_lhs_left"]], "apply_nullspace_transformation_lhs_right() (tlm_adjoint.firedrake.block_system.blocknullspace method)": [[22, "tlm_adjoint.firedrake.block_system.BlockNullspace.apply_nullspace_transformation_lhs_right"]], "apply_nullspace_transformation_lhs_right() (tlm_adjoint.firedrake.block_system.constantnullspace method)": [[22, "tlm_adjoint.firedrake.block_system.ConstantNullspace.apply_nullspace_transformation_lhs_right"]], "apply_nullspace_transformation_lhs_right() (tlm_adjoint.firedrake.block_system.dirichletbcnullspace method)": [[22, "tlm_adjoint.firedrake.block_system.DirichletBCNullspace.apply_nullspace_transformation_lhs_right"]], "apply_nullspace_transformation_lhs_right() (tlm_adjoint.firedrake.block_system.nonenullspace method)": [[22, "tlm_adjoint.firedrake.block_system.NoneNullspace.apply_nullspace_transformation_lhs_right"]], "apply_nullspace_transformation_lhs_right() (tlm_adjoint.firedrake.block_system.nullspace method)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace.apply_nullspace_transformation_lhs_right"]], "apply_nullspace_transformation_lhs_right() (tlm_adjoint.firedrake.block_system.unitynullspace method)": [[22, "tlm_adjoint.firedrake.block_system.UnityNullspace.apply_nullspace_transformation_lhs_right"]], "arg_space (tlm_adjoint.firedrake.block_system.matrix property)": [[22, "tlm_adjoint.firedrake.block_system.Matrix.arg_space"]], "comm (tlm_adjoint.firedrake.block_system.mixedspace property)": [[22, "tlm_adjoint.firedrake.block_system.MixedSpace.comm"]], "constraint_correct_lhs() (tlm_adjoint.firedrake.block_system.blocknullspace method)": [[22, "tlm_adjoint.firedrake.block_system.BlockNullspace.constraint_correct_lhs"]], "constraint_correct_lhs() (tlm_adjoint.firedrake.block_system.constantnullspace method)": [[22, "tlm_adjoint.firedrake.block_system.ConstantNullspace.constraint_correct_lhs"]], "constraint_correct_lhs() (tlm_adjoint.firedrake.block_system.dirichletbcnullspace method)": [[22, "tlm_adjoint.firedrake.block_system.DirichletBCNullspace.constraint_correct_lhs"]], "constraint_correct_lhs() (tlm_adjoint.firedrake.block_system.nonenullspace method)": [[22, "tlm_adjoint.firedrake.block_system.NoneNullspace.constraint_correct_lhs"]], "constraint_correct_lhs() (tlm_adjoint.firedrake.block_system.nullspace method)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace.constraint_correct_lhs"]], "constraint_correct_lhs() (tlm_adjoint.firedrake.block_system.unitynullspace method)": [[22, "tlm_adjoint.firedrake.block_system.UnityNullspace.constraint_correct_lhs"]], "correct_rhs() (tlm_adjoint.firedrake.block_system.nullspace method)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace.correct_rhs"]], "correct_soln() (tlm_adjoint.firedrake.block_system.nullspace method)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace.correct_soln"]], "flattened_space (tlm_adjoint.firedrake.block_system.mixedspace property)": [[22, "tlm_adjoint.firedrake.block_system.MixedSpace.flattened_space"]], "form_matrix() (in module tlm_adjoint.firedrake.block_system)": [[22, "tlm_adjoint.firedrake.block_system.form_matrix"]], "from_petsc() (tlm_adjoint.firedrake.block_system.mixedspace method)": [[22, "tlm_adjoint.firedrake.block_system.MixedSpace.from_petsc"]], "global_size (tlm_adjoint.firedrake.block_system.mixedspace property)": [[22, "tlm_adjoint.firedrake.block_system.MixedSpace.global_size"]], "local_size (tlm_adjoint.firedrake.block_system.mixedspace property)": [[22, "tlm_adjoint.firedrake.block_system.MixedSpace.local_size"]], "mult_add() (tlm_adjoint.firedrake.block_system.blockmatrix method)": [[22, "tlm_adjoint.firedrake.block_system.BlockMatrix.mult_add"]], "mult_add() (tlm_adjoint.firedrake.block_system.matrix method)": [[22, "tlm_adjoint.firedrake.block_system.Matrix.mult_add"]], "mult_add() (tlm_adjoint.firedrake.block_system.petscmatrix method)": [[22, "tlm_adjoint.firedrake.block_system.PETScMatrix.mult_add"]], "new_split() (tlm_adjoint.firedrake.block_system.mixedspace method)": [[22, "tlm_adjoint.firedrake.block_system.MixedSpace.new_split"]], "pc_constraint_correct_soln() (tlm_adjoint.firedrake.block_system.blocknullspace method)": [[22, "tlm_adjoint.firedrake.block_system.BlockNullspace.pc_constraint_correct_soln"]], "pc_constraint_correct_soln() (tlm_adjoint.firedrake.block_system.constantnullspace method)": [[22, "tlm_adjoint.firedrake.block_system.ConstantNullspace.pc_constraint_correct_soln"]], "pc_constraint_correct_soln() (tlm_adjoint.firedrake.block_system.dirichletbcnullspace method)": [[22, "tlm_adjoint.firedrake.block_system.DirichletBCNullspace.pc_constraint_correct_soln"]], "pc_constraint_correct_soln() (tlm_adjoint.firedrake.block_system.nonenullspace method)": [[22, "tlm_adjoint.firedrake.block_system.NoneNullspace.pc_constraint_correct_soln"]], "pc_constraint_correct_soln() (tlm_adjoint.firedrake.block_system.nullspace method)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace.pc_constraint_correct_soln"]], "pc_constraint_correct_soln() (tlm_adjoint.firedrake.block_system.unitynullspace method)": [[22, "tlm_adjoint.firedrake.block_system.UnityNullspace.pc_constraint_correct_soln"]], "pc_post_mult_correct() (tlm_adjoint.firedrake.block_system.nullspace method)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace.pc_post_mult_correct"]], "pc_pre_mult_correct() (tlm_adjoint.firedrake.block_system.nullspace method)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace.pc_pre_mult_correct"]], "post_mult_correct_lhs() (tlm_adjoint.firedrake.block_system.nullspace method)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace.post_mult_correct_lhs"]], "pre_mult_correct_lhs() (tlm_adjoint.firedrake.block_system.nullspace method)": [[22, "tlm_adjoint.firedrake.block_system.Nullspace.pre_mult_correct_lhs"]], "solve() (tlm_adjoint.firedrake.block_system.system method)": [[22, "tlm_adjoint.firedrake.block_system.System.solve"]], "split_space (tlm_adjoint.firedrake.block_system.mixedspace property)": [[22, "tlm_adjoint.firedrake.block_system.MixedSpace.split_space"]], "tlm_adjoint.firedrake.block_system": [[22, "module-tlm_adjoint.firedrake.block_system"]], "to_petsc() (tlm_adjoint.firedrake.block_system.mixedspace method)": [[22, "tlm_adjoint.firedrake.block_system.MixedSpace.to_petsc"]], "assemblycache (class in tlm_adjoint.firedrake.caches)": [[23, "tlm_adjoint.firedrake.caches.AssemblyCache"]], "linearsolvercache (class in tlm_adjoint.firedrake.caches)": [[23, "tlm_adjoint.firedrake.caches.LinearSolverCache"]], "assemble() (tlm_adjoint.firedrake.caches.assemblycache method)": [[23, "tlm_adjoint.firedrake.caches.AssemblyCache.assemble"]], "assembly_cache() (in module tlm_adjoint.firedrake.caches)": [[23, "tlm_adjoint.firedrake.caches.assembly_cache"]], "linear_solver() (tlm_adjoint.firedrake.caches.linearsolvercache method)": [[23, "tlm_adjoint.firedrake.caches.LinearSolverCache.linear_solver"]], "linear_solver_cache() (in module tlm_adjoint.firedrake.caches)": [[23, "tlm_adjoint.firedrake.caches.linear_solver_cache"]], "set_assembly_cache() (in module tlm_adjoint.firedrake.caches)": [[23, "tlm_adjoint.firedrake.caches.set_assembly_cache"]], "set_linear_solver_cache() (in module tlm_adjoint.firedrake.caches)": [[23, "tlm_adjoint.firedrake.caches.set_linear_solver_cache"]], "tlm_adjoint.firedrake.caches": [[23, "module-tlm_adjoint.firedrake.caches"]], "assembly (class in tlm_adjoint.firedrake.equations)": [[24, "tlm_adjoint.firedrake.equations.Assembly"]], "dirichletbcapplication (class in tlm_adjoint.firedrake.equations)": [[24, "tlm_adjoint.firedrake.equations.DirichletBCApplication"]], "equationsolver (class in tlm_adjoint.firedrake.equations)": [[24, "tlm_adjoint.firedrake.equations.EquationSolver"]], "exprinterpolation (class in tlm_adjoint.firedrake.equations)": [[24, "tlm_adjoint.firedrake.equations.ExprInterpolation"]], "projection (class in tlm_adjoint.firedrake.equations)": [[24, "tlm_adjoint.firedrake.equations.Projection"]], "adjoint_derivative_action() (tlm_adjoint.firedrake.equations.assembly method)": [[24, "tlm_adjoint.firedrake.equations.Assembly.adjoint_derivative_action"]], "adjoint_derivative_action() (tlm_adjoint.firedrake.equations.dirichletbcapplication method)": [[24, "tlm_adjoint.firedrake.equations.DirichletBCApplication.adjoint_derivative_action"]], "adjoint_derivative_action() (tlm_adjoint.firedrake.equations.exprinterpolation method)": [[24, "tlm_adjoint.firedrake.equations.ExprInterpolation.adjoint_derivative_action"]], "adjoint_jacobian_solve() (tlm_adjoint.firedrake.equations.assembly method)": [[24, "tlm_adjoint.firedrake.equations.Assembly.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.firedrake.equations.dirichletbcapplication method)": [[24, "tlm_adjoint.firedrake.equations.DirichletBCApplication.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.firedrake.equations.equationsolver method)": [[24, "tlm_adjoint.firedrake.equations.EquationSolver.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.firedrake.equations.exprinterpolation method)": [[24, "tlm_adjoint.firedrake.equations.ExprInterpolation.adjoint_jacobian_solve"]], "drop_references() (tlm_adjoint.firedrake.equations.assembly method)": [[24, "tlm_adjoint.firedrake.equations.Assembly.drop_references"]], "drop_references() (tlm_adjoint.firedrake.equations.equationsolver method)": [[24, "tlm_adjoint.firedrake.equations.EquationSolver.drop_references"]], "drop_references() (tlm_adjoint.firedrake.equations.exprinterpolation method)": [[24, "tlm_adjoint.firedrake.equations.ExprInterpolation.drop_references"]], "expr_new_x() (in module tlm_adjoint.firedrake.equations)": [[24, "tlm_adjoint.firedrake.equations.expr_new_x"]], "forward_solve() (tlm_adjoint.firedrake.equations.assembly method)": [[24, "tlm_adjoint.firedrake.equations.Assembly.forward_solve"]], "forward_solve() (tlm_adjoint.firedrake.equations.dirichletbcapplication method)": [[24, "tlm_adjoint.firedrake.equations.DirichletBCApplication.forward_solve"]], "forward_solve() (tlm_adjoint.firedrake.equations.equationsolver method)": [[24, "tlm_adjoint.firedrake.equations.EquationSolver.forward_solve"]], "forward_solve() (tlm_adjoint.firedrake.equations.exprinterpolation method)": [[24, "tlm_adjoint.firedrake.equations.ExprInterpolation.forward_solve"]], "linear_equation_new_x() (in module tlm_adjoint.firedrake.equations)": [[24, "tlm_adjoint.firedrake.equations.linear_equation_new_x"]], "subtract_adjoint_derivative_actions() (tlm_adjoint.firedrake.equations.equationsolver method)": [[24, "tlm_adjoint.firedrake.equations.EquationSolver.subtract_adjoint_derivative_actions"]], "tangent_linear() (tlm_adjoint.firedrake.equations.assembly method)": [[24, "tlm_adjoint.firedrake.equations.Assembly.tangent_linear"]], "tangent_linear() (tlm_adjoint.firedrake.equations.dirichletbcapplication method)": [[24, "tlm_adjoint.firedrake.equations.DirichletBCApplication.tangent_linear"]], "tangent_linear() (tlm_adjoint.firedrake.equations.equationsolver method)": [[24, "tlm_adjoint.firedrake.equations.EquationSolver.tangent_linear"]], "tangent_linear() (tlm_adjoint.firedrake.equations.exprinterpolation method)": [[24, "tlm_adjoint.firedrake.equations.ExprInterpolation.tangent_linear"]], "tlm_adjoint.firedrake.equations": [[24, "module-tlm_adjoint.firedrake.equations"]], "exprassignment (class in tlm_adjoint.firedrake.firedrake_equations)": [[25, "tlm_adjoint.firedrake.firedrake_equations.ExprAssignment"]], "localprojection (class in tlm_adjoint.firedrake.firedrake_equations)": [[25, "tlm_adjoint.firedrake.firedrake_equations.LocalProjection"]], "localsolvercache (class in tlm_adjoint.firedrake.firedrake_equations)": [[25, "tlm_adjoint.firedrake.firedrake_equations.LocalSolverCache"]], "pointinterpolation (class in tlm_adjoint.firedrake.firedrake_equations)": [[25, "tlm_adjoint.firedrake.firedrake_equations.PointInterpolation"]], "adjoint_derivative_action() (tlm_adjoint.firedrake.firedrake_equations.exprassignment method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.ExprAssignment.adjoint_derivative_action"]], "adjoint_derivative_action() (tlm_adjoint.firedrake.firedrake_equations.pointinterpolation method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.PointInterpolation.adjoint_derivative_action"]], "adjoint_jacobian_solve() (tlm_adjoint.firedrake.firedrake_equations.exprassignment method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.ExprAssignment.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.firedrake.firedrake_equations.localprojection method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.LocalProjection.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.firedrake.firedrake_equations.pointinterpolation method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.PointInterpolation.adjoint_jacobian_solve"]], "drop_references() (tlm_adjoint.firedrake.firedrake_equations.exprassignment method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.ExprAssignment.drop_references"]], "forward_solve() (tlm_adjoint.firedrake.firedrake_equations.exprassignment method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.ExprAssignment.forward_solve"]], "forward_solve() (tlm_adjoint.firedrake.firedrake_equations.localprojection method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.LocalProjection.forward_solve"]], "forward_solve() (tlm_adjoint.firedrake.firedrake_equations.pointinterpolation method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.PointInterpolation.forward_solve"]], "local_solver() (tlm_adjoint.firedrake.firedrake_equations.localsolvercache method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.LocalSolverCache.local_solver"]], "local_solver_cache() (in module tlm_adjoint.firedrake.firedrake_equations)": [[25, "tlm_adjoint.firedrake.firedrake_equations.local_solver_cache"]], "set_local_solver_cache() (in module tlm_adjoint.firedrake.firedrake_equations)": [[25, "tlm_adjoint.firedrake.firedrake_equations.set_local_solver_cache"]], "tangent_linear() (tlm_adjoint.firedrake.firedrake_equations.exprassignment method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.ExprAssignment.tangent_linear"]], "tangent_linear() (tlm_adjoint.firedrake.firedrake_equations.localprojection method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.LocalProjection.tangent_linear"]], "tangent_linear() (tlm_adjoint.firedrake.firedrake_equations.pointinterpolation method)": [[25, "tlm_adjoint.firedrake.firedrake_equations.PointInterpolation.tangent_linear"]], "tlm_adjoint.firedrake.firedrake_equations": [[25, "module-tlm_adjoint.firedrake.firedrake_equations"]], "constant (class in tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.Constant"]], "dirichletbc (class in tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.DirichletBC"]], "homogeneousdirichletbc (class in tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.HomogeneousDirichletBC"]], "replacement (class in tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.Replacement"]], "replacementconstant (class in tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.ReplacementConstant"]], "replacementfunction (class in tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.ReplacementFunction"]], "replacementzeroconstant (class in tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.ReplacementZeroConstant"]], "replacementzerofunction (class in tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.ReplacementZeroFunction"]], "zero (class in tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.Zero"]], "zeroconstant (class in tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.ZeroConstant"]], "eliminate_zeros() (in module tlm_adjoint.firedrake.functions)": [[26, "tlm_adjoint.firedrake.functions.eliminate_zeros"]], "tlm_adjoint.firedrake.functions": [[26, "module-tlm_adjoint.firedrake.functions"]], "b_inv_orthonormality_test() (in module tlm_adjoint.firedrake.hessian_system)": [[27, "tlm_adjoint.firedrake.hessian_system.B_inv_orthonormality_test"]], "hessiansystem (class in tlm_adjoint.firedrake.hessian_system)": [[27, "tlm_adjoint.firedrake.hessian_system.HessianSystem"]], "hessian_eigendecompose() (in module tlm_adjoint.firedrake.hessian_system)": [[27, "tlm_adjoint.firedrake.hessian_system.hessian_eigendecompose"]], "hessian_eigendecomposition_pc() (in module tlm_adjoint.firedrake.hessian_system)": [[27, "tlm_adjoint.firedrake.hessian_system.hessian_eigendecomposition_pc"]], "solve() (tlm_adjoint.firedrake.hessian_system.hessiansystem method)": [[27, "tlm_adjoint.firedrake.hessian_system.HessianSystem.solve"]], "tlm_adjoint.firedrake.hessian_system": [[27, "module-tlm_adjoint.firedrake.hessian_system"]], "customnormsq (class in tlm_adjoint.fixed_point)": [[28, "tlm_adjoint.fixed_point.CustomNormSq"]], "fixedpointsolver (class in tlm_adjoint.fixed_point)": [[28, "tlm_adjoint.fixed_point.FixedPointSolver"]], "adjoint_jacobian_solve() (tlm_adjoint.fixed_point.fixedpointsolver method)": [[28, "tlm_adjoint.fixed_point.FixedPointSolver.adjoint_jacobian_solve"]], "drop_references() (tlm_adjoint.fixed_point.fixedpointsolver method)": [[28, "tlm_adjoint.fixed_point.FixedPointSolver.drop_references"]], "forward_solve() (tlm_adjoint.fixed_point.fixedpointsolver method)": [[28, "tlm_adjoint.fixed_point.FixedPointSolver.forward_solve"]], "subtract_adjoint_derivative_actions() (tlm_adjoint.fixed_point.fixedpointsolver method)": [[28, "tlm_adjoint.fixed_point.FixedPointSolver.subtract_adjoint_derivative_actions"]], "tangent_linear() (tlm_adjoint.fixed_point.fixedpointsolver method)": [[28, "tlm_adjoint.fixed_point.FixedPointSolver.tangent_linear"]], "tlm_adjoint.fixed_point": [[28, "module-tlm_adjoint.fixed_point"]], "functional (class in tlm_adjoint.functional)": [[29, "tlm_adjoint.functional.Functional"]], "addto() (tlm_adjoint.functional.functional method)": [[29, "tlm_adjoint.functional.Functional.addto"]], "assign() (tlm_adjoint.functional.functional method)": [[29, "tlm_adjoint.functional.Functional.assign"]], "tlm_adjoint.functional": [[29, "module-tlm_adjoint.functional"]], "gaussnewton (class in tlm_adjoint.hessian)": [[30, "tlm_adjoint.hessian.GaussNewton"]], "generalgaussnewton (class in tlm_adjoint.hessian)": [[30, "tlm_adjoint.hessian.GeneralGaussNewton"]], "generalhessian (class in tlm_adjoint.hessian)": [[30, "tlm_adjoint.hessian.GeneralHessian"]], "hessian (class in tlm_adjoint.hessian)": [[30, "tlm_adjoint.hessian.Hessian"]], "action() (tlm_adjoint.hessian.gaussnewton method)": [[30, "tlm_adjoint.hessian.GaussNewton.action"]], "action() (tlm_adjoint.hessian.generalhessian method)": [[30, "tlm_adjoint.hessian.GeneralHessian.action"]], "action() (tlm_adjoint.hessian.hessian method)": [[30, "tlm_adjoint.hessian.Hessian.action"]], "action_fn() (tlm_adjoint.hessian.gaussnewton method)": [[30, "tlm_adjoint.hessian.GaussNewton.action_fn"]], "action_fn() (tlm_adjoint.hessian.hessian method)": [[30, "tlm_adjoint.hessian.Hessian.action_fn"]], "compute_gradient() (tlm_adjoint.hessian.generalhessian method)": [[30, "tlm_adjoint.hessian.GeneralHessian.compute_gradient"]], "compute_gradient() (tlm_adjoint.hessian.hessian method)": [[30, "tlm_adjoint.hessian.Hessian.compute_gradient"]], "tlm_adjoint.hessian": [[30, "module-tlm_adjoint.hessian"]], "tlm_adjoint": [[31, "module-tlm_adjoint"]], "garbagecollection (class in tlm_adjoint.instructions)": [[32, "tlm_adjoint.instructions.GarbageCollection"]], "instruction (class in tlm_adjoint.instructions)": [[32, "tlm_adjoint.instructions.Instruction"]], "adjoint_jacobian_solve() (tlm_adjoint.instructions.garbagecollection method)": [[32, "tlm_adjoint.instructions.GarbageCollection.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.instructions.instruction method)": [[32, "tlm_adjoint.instructions.Instruction.adjoint_jacobian_solve"]], "forward_solve() (tlm_adjoint.instructions.garbagecollection method)": [[32, "tlm_adjoint.instructions.GarbageCollection.forward_solve"]], "tangent_linear() (tlm_adjoint.instructions.garbagecollection method)": [[32, "tlm_adjoint.instructions.GarbageCollection.tangent_linear"]], "tlm_adjoint.instructions": [[32, "module-tlm_adjoint.instructions"]], "spaceinterface (class in tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.SpaceInterface"]], "spacetypeerror": [[33, "tlm_adjoint.interface.SpaceTypeError"]], "variableinterface (class in tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.VariableInterface"]], "variablestatelockdictionary (class in tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.VariableStateLockDictionary"]], "add_interface() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.add_interface"]], "check_space_type() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.check_space_type"]], "check_space_types() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.check_space_types"]], "check_space_types_conjugate() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.check_space_types_conjugate"]], "check_space_types_conjugate_dual() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.check_space_types_conjugate_dual"]], "check_space_types_dual() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.check_space_types_dual"]], "comm_dup_cached() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.comm_dup_cached"]], "conjugate_dual_space_type() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.conjugate_dual_space_type"]], "conjugate_space_type() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.conjugate_space_type"]], "dual_space_type() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.dual_space_type"]], "garbage_cleanup() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.garbage_cleanup"]], "is_space() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.is_space"]], "is_var() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.is_var"]], "no_space_type_checking() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.no_space_type_checking"]], "paused_space_type_checking() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.paused_space_type_checking"]], "relative_space_type() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.relative_space_type"]], "space_comm() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.space_comm"]], "space_dtype() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.space_dtype"]], "space_id() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.space_id"]], "space_new() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.space_new"]], "subtract_adjoint_derivative_action() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.subtract_adjoint_derivative_action"]], "tlm_adjoint.interface": [[33, "module-tlm_adjoint.interface"]], "var_assign() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_assign"]], "var_axpy() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_axpy"]], "var_caches() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_caches"]], "var_comm() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_comm"]], "var_copy() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_copy"]], "var_dtype() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_dtype"]], "var_get_values() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_get_values"]], "var_global_size() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_global_size"]], "var_id() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_id"]], "var_inner() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_inner"]], "var_is_alias() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_is_alias"]], "var_is_cached() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_is_cached"]], "var_is_replacement() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_is_replacement"]], "var_is_scalar() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_is_scalar"]], "var_is_static() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_is_static"]], "var_linf_norm() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_linf_norm"]], "var_local_indices() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_local_indices"]], "var_local_size() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_local_size"]], "var_lock_state() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_lock_state"]], "var_name() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_name"]], "var_new() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_new"]], "var_new_conjugate() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_new_conjugate"]], "var_new_conjugate_dual() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_new_conjugate_dual"]], "var_new_dual() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_new_dual"]], "var_replacement() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_replacement"]], "var_scalar_value() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_scalar_value"]], "var_set_values() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_set_values"]], "var_space() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_space"]], "var_space_type() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_space_type"]], "var_state() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_state"]], "var_update_caches() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_update_caches"]], "var_update_state() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_update_state"]], "var_zero() (in module tlm_adjoint.interface)": [[33, "tlm_adjoint.interface.var_zero"]], "vector (class in tlm_adjoint.jax)": [[34, "tlm_adjoint.jax.Vector"]], "vectorequation (class in tlm_adjoint.jax)": [[34, "tlm_adjoint.jax.VectorEquation"]], "vectorspace (class in tlm_adjoint.jax)": [[34, "tlm_adjoint.jax.VectorSpace"]], "addto() (tlm_adjoint.jax.vector method)": [[34, "tlm_adjoint.jax.Vector.addto"]], "adjoint_jacobian_solve() (tlm_adjoint.jax.vectorequation method)": [[34, "tlm_adjoint.jax.VectorEquation.adjoint_jacobian_solve"]], "assign() (tlm_adjoint.jax.vector method)": [[34, "tlm_adjoint.jax.Vector.assign"]], "call_jax() (in module tlm_adjoint.jax)": [[34, "tlm_adjoint.jax.call_jax"]], "comm (tlm_adjoint.jax.vectorspace property)": [[34, "tlm_adjoint.jax.VectorSpace.comm"]], "dtype (tlm_adjoint.jax.vectorspace property)": [[34, "tlm_adjoint.jax.VectorSpace.dtype"]], "forward_solve() (tlm_adjoint.jax.vectorequation method)": [[34, "tlm_adjoint.jax.VectorEquation.forward_solve"]], "global_size (tlm_adjoint.jax.vectorspace property)": [[34, "tlm_adjoint.jax.VectorSpace.global_size"]], "local_size (tlm_adjoint.jax.vectorspace property)": [[34, "tlm_adjoint.jax.VectorSpace.local_size"]], "name (tlm_adjoint.jax.vector property)": [[34, "tlm_adjoint.jax.Vector.name"]], "new() (tlm_adjoint.jax.vector method)": [[34, "tlm_adjoint.jax.Vector.new"]], "new_jax() (in module tlm_adjoint.jax)": [[34, "tlm_adjoint.jax.new_jax"]], "new_jax_float() (in module tlm_adjoint.jax)": [[34, "tlm_adjoint.jax.new_jax_float"]], "ownership_range (tlm_adjoint.jax.vectorspace property)": [[34, "tlm_adjoint.jax.VectorSpace.ownership_range"]], "rdtype() (tlm_adjoint.jax.vectorspace method)": [[34, "tlm_adjoint.jax.VectorSpace.rdtype"]], "set_default_jax_dtype() (in module tlm_adjoint.jax)": [[34, "tlm_adjoint.jax.set_default_jax_dtype"]], "solve() (tlm_adjoint.jax.vectorequation method)": [[34, "tlm_adjoint.jax.VectorEquation.solve"]], "space (tlm_adjoint.jax.vector property)": [[34, "tlm_adjoint.jax.Vector.space"]], "space_type (tlm_adjoint.jax.vector property)": [[34, "tlm_adjoint.jax.Vector.space_type"]], "subtract_adjoint_derivative_actions() (tlm_adjoint.jax.vectorequation method)": [[34, "tlm_adjoint.jax.VectorEquation.subtract_adjoint_derivative_actions"]], "tangent_linear() (tlm_adjoint.jax.vectorequation method)": [[34, "tlm_adjoint.jax.VectorEquation.tangent_linear"]], "tlm_adjoint.jax": [[34, "module-tlm_adjoint.jax"]], "to_jax() (in module tlm_adjoint.jax)": [[34, "tlm_adjoint.jax.to_jax"]], "value (tlm_adjoint.jax.vector property)": [[34, "tlm_adjoint.jax.Vector.value"]], "vector (tlm_adjoint.jax.vector property)": [[34, "tlm_adjoint.jax.Vector.vector"]], "linearequation (class in tlm_adjoint.linear_equation)": [[35, "tlm_adjoint.linear_equation.LinearEquation"]], "matrix (class in tlm_adjoint.linear_equation)": [[35, "tlm_adjoint.linear_equation.Matrix"]], "rhs (class in tlm_adjoint.linear_equation)": [[35, "tlm_adjoint.linear_equation.RHS"]], "add_forward() (tlm_adjoint.linear_equation.rhs method)": [[35, "tlm_adjoint.linear_equation.RHS.add_forward"]], "adjoint_action() (tlm_adjoint.linear_equation.matrix method)": [[35, "tlm_adjoint.linear_equation.Matrix.adjoint_action"]], "adjoint_derivative_action() (tlm_adjoint.linear_equation.linearequation method)": [[35, "tlm_adjoint.linear_equation.LinearEquation.adjoint_derivative_action"]], "adjoint_derivative_action() (tlm_adjoint.linear_equation.matrix method)": [[35, "tlm_adjoint.linear_equation.Matrix.adjoint_derivative_action"]], "adjoint_has_initial_condition() (tlm_adjoint.linear_equation.matrix method)": [[35, "tlm_adjoint.linear_equation.Matrix.adjoint_has_initial_condition"]], "adjoint_jacobian_solve() (tlm_adjoint.linear_equation.linearequation method)": [[35, "tlm_adjoint.linear_equation.LinearEquation.adjoint_jacobian_solve"]], "adjoint_solve() (tlm_adjoint.linear_equation.matrix method)": [[35, "tlm_adjoint.linear_equation.Matrix.adjoint_solve"]], "dependencies() (tlm_adjoint.linear_equation.rhs method)": [[35, "tlm_adjoint.linear_equation.RHS.dependencies"]], "drop_references() (tlm_adjoint.linear_equation.linearequation method)": [[35, "tlm_adjoint.linear_equation.LinearEquation.drop_references"]], "drop_references() (tlm_adjoint.linear_equation.matrix method)": [[35, "tlm_adjoint.linear_equation.Matrix.drop_references"]], "drop_references() (tlm_adjoint.linear_equation.rhs method)": [[35, "tlm_adjoint.linear_equation.RHS.drop_references"]], "forward_action() (tlm_adjoint.linear_equation.matrix method)": [[35, "tlm_adjoint.linear_equation.Matrix.forward_action"]], "forward_solve() (tlm_adjoint.linear_equation.linearequation method)": [[35, "tlm_adjoint.linear_equation.LinearEquation.forward_solve"]], "forward_solve() (tlm_adjoint.linear_equation.matrix method)": [[35, "tlm_adjoint.linear_equation.Matrix.forward_solve"]], "has_initial_condition() (tlm_adjoint.linear_equation.matrix method)": [[35, "tlm_adjoint.linear_equation.Matrix.has_initial_condition"]], "nonlinear_dependencies() (tlm_adjoint.linear_equation.matrix method)": [[35, "tlm_adjoint.linear_equation.Matrix.nonlinear_dependencies"]], "nonlinear_dependencies() (tlm_adjoint.linear_equation.rhs method)": [[35, "tlm_adjoint.linear_equation.RHS.nonlinear_dependencies"]], "subtract_adjoint_derivative_action() (tlm_adjoint.linear_equation.rhs method)": [[35, "tlm_adjoint.linear_equation.RHS.subtract_adjoint_derivative_action"]], "tangent_linear() (tlm_adjoint.linear_equation.linearequation method)": [[35, "tlm_adjoint.linear_equation.LinearEquation.tangent_linear"]], "tangent_linear_rhs() (tlm_adjoint.linear_equation.matrix method)": [[35, "tlm_adjoint.linear_equation.Matrix.tangent_linear_rhs"]], "tangent_linear_rhs() (tlm_adjoint.linear_equation.rhs method)": [[35, "tlm_adjoint.linear_equation.RHS.tangent_linear_rhs"]], "tlm_adjoint.linear_equation": [[35, "module-tlm_adjoint.linear_equation"]], "annotation_enabled() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.annotation_enabled"]], "compute_gradient() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.compute_gradient"]], "configure_checkpointing() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.configure_checkpointing"]], "configure_tlm() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.configure_tlm"]], "manager() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.manager"]], "manager_disabled() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.manager_disabled"]], "manager_info() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.manager_info"]], "new_block() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.new_block"]], "paused_manager() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.paused_manager"]], "reset_manager() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.reset_manager"]], "restore_manager() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.restore_manager"]], "set_manager() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.set_manager"]], "start_manager() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.start_manager"]], "stop_manager() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.stop_manager"]], "tlm_adjoint.manager": [[36, "module-tlm_adjoint.manager"]], "tlm_enabled() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.tlm_enabled"]], "var_tlm() (in module tlm_adjoint.manager)": [[36, "tlm_adjoint.manager.var_tlm"]], "controlsmarker (class in tlm_adjoint.markers)": [[37, "tlm_adjoint.markers.ControlsMarker"]], "functionalmarker (class in tlm_adjoint.markers)": [[37, "tlm_adjoint.markers.FunctionalMarker"]], "adjoint_derivative_action() (tlm_adjoint.markers.functionalmarker method)": [[37, "tlm_adjoint.markers.FunctionalMarker.adjoint_derivative_action"]], "adjoint_jacobian_solve() (tlm_adjoint.markers.controlsmarker method)": [[37, "tlm_adjoint.markers.ControlsMarker.adjoint_jacobian_solve"]], "adjoint_jacobian_solve() (tlm_adjoint.markers.functionalmarker method)": [[37, "tlm_adjoint.markers.FunctionalMarker.adjoint_jacobian_solve"]], "tlm_adjoint.markers": [[37, "module-tlm_adjoint.markers"]], "lbfgshessianapproximation (class in tlm_adjoint.optimization)": [[38, "tlm_adjoint.optimization.LBFGSHessianApproximation"]], "append() (tlm_adjoint.optimization.lbfgshessianapproximation method)": [[38, "tlm_adjoint.optimization.LBFGSHessianApproximation.append"]], "inverse_action() (tlm_adjoint.optimization.lbfgshessianapproximation method)": [[38, "tlm_adjoint.optimization.LBFGSHessianApproximation.inverse_action"]], "l_bfgs() (in module tlm_adjoint.optimization)": [[38, "tlm_adjoint.optimization.l_bfgs"]], "minimize_l_bfgs() (in module tlm_adjoint.optimization)": [[38, "tlm_adjoint.optimization.minimize_l_bfgs"]], "minimize_scipy() (in module tlm_adjoint.optimization)": [[38, "tlm_adjoint.optimization.minimize_scipy"]], "minimize_tao() (in module tlm_adjoint.optimization)": [[38, "tlm_adjoint.optimization.minimize_tao"]], "tlm_adjoint.optimization": [[38, "module-tlm_adjoint.optimization"]], "float (class in tlm_adjoint.overloaded_float)": [[39, "tlm_adjoint.overloaded_float.Float"]], "floatequation (class in tlm_adjoint.overloaded_float)": [[39, "tlm_adjoint.overloaded_float.FloatEquation"]], "floatspace (class in tlm_adjoint.overloaded_float)": [[39, "tlm_adjoint.overloaded_float.FloatSpace"]], "overloadedfloat (class in tlm_adjoint.overloaded_float)": [[39, "tlm_adjoint.overloaded_float.OverloadedFloat"]], "symbolicfloat (class in tlm_adjoint.overloaded_float)": [[39, "tlm_adjoint.overloaded_float.SymbolicFloat"]], "addto() (tlm_adjoint.overloaded_float.symbolicfloat method)": [[39, "tlm_adjoint.overloaded_float.SymbolicFloat.addto"]], "adjoint_jacobian_solve() (tlm_adjoint.overloaded_float.floatequation method)": [[39, "tlm_adjoint.overloaded_float.FloatEquation.adjoint_jacobian_solve"]], "assign() (tlm_adjoint.overloaded_float.symbolicfloat method)": [[39, "tlm_adjoint.overloaded_float.SymbolicFloat.assign"]], "comm (tlm_adjoint.overloaded_float.floatspace property)": [[39, "tlm_adjoint.overloaded_float.FloatSpace.comm"]], "dtype (tlm_adjoint.overloaded_float.floatspace property)": [[39, "tlm_adjoint.overloaded_float.FloatSpace.dtype"]], "float_cls (tlm_adjoint.overloaded_float.floatspace property)": [[39, "tlm_adjoint.overloaded_float.FloatSpace.float_cls"]], "forward_solve() (tlm_adjoint.overloaded_float.floatequation method)": [[39, "tlm_adjoint.overloaded_float.FloatEquation.forward_solve"]], "new() (tlm_adjoint.overloaded_float.symbolicfloat method)": [[39, "tlm_adjoint.overloaded_float.SymbolicFloat.new"]], "no_float_overloading() (in module tlm_adjoint.overloaded_float)": [[39, "tlm_adjoint.overloaded_float.no_float_overloading"]], "paused_float_overloading() (in module tlm_adjoint.overloaded_float)": [[39, "tlm_adjoint.overloaded_float.paused_float_overloading"]], "rdtype() (tlm_adjoint.overloaded_float.floatspace method)": [[39, "tlm_adjoint.overloaded_float.FloatSpace.rdtype"]], "set_default_float_dtype() (in module tlm_adjoint.overloaded_float)": [[39, "tlm_adjoint.overloaded_float.set_default_float_dtype"]], "subtract_adjoint_derivative_actions() (tlm_adjoint.overloaded_float.floatequation method)": [[39, "tlm_adjoint.overloaded_float.FloatEquation.subtract_adjoint_derivative_actions"]], "tangent_linear() (tlm_adjoint.overloaded_float.floatequation method)": [[39, "tlm_adjoint.overloaded_float.FloatEquation.tangent_linear"]], "tlm_adjoint.overloaded_float": [[39, "module-tlm_adjoint.overloaded_float"]], "to_float() (in module tlm_adjoint.overloaded_float)": [[39, "tlm_adjoint.overloaded_float.to_float"]], "value (tlm_adjoint.overloaded_float.symbolicfloat property)": [[39, "tlm_adjoint.overloaded_float.SymbolicFloat.value"]], "hdf5storage (class in tlm_adjoint.storage)": [[40, "tlm_adjoint.storage.HDF5Storage"]], "memorystorage (class in tlm_adjoint.storage)": [[40, "tlm_adjoint.storage.MemoryStorage"]], "storage (class in tlm_adjoint.storage)": [[40, "tlm_adjoint.storage.Storage"]], "adjoint_derivative_action() (tlm_adjoint.storage.storage method)": [[40, "tlm_adjoint.storage.Storage.adjoint_derivative_action"]], "adjoint_jacobian_solve() (tlm_adjoint.storage.storage method)": [[40, "tlm_adjoint.storage.Storage.adjoint_jacobian_solve"]], "forward_solve() (tlm_adjoint.storage.storage method)": [[40, "tlm_adjoint.storage.Storage.forward_solve"]], "is_saved() (tlm_adjoint.storage.hdf5storage method)": [[40, "tlm_adjoint.storage.HDF5Storage.is_saved"]], "is_saved() (tlm_adjoint.storage.memorystorage method)": [[40, "tlm_adjoint.storage.MemoryStorage.is_saved"]], "is_saved() (tlm_adjoint.storage.storage method)": [[40, "tlm_adjoint.storage.Storage.is_saved"]], "key (tlm_adjoint.storage.storage property)": [[40, "tlm_adjoint.storage.Storage.key"]], "load() (tlm_adjoint.storage.hdf5storage method)": [[40, "tlm_adjoint.storage.HDF5Storage.load"]], "load() (tlm_adjoint.storage.memorystorage method)": [[40, "tlm_adjoint.storage.MemoryStorage.load"]], "load() (tlm_adjoint.storage.storage method)": [[40, "tlm_adjoint.storage.Storage.load"]], "save() (tlm_adjoint.storage.hdf5storage method)": [[40, "tlm_adjoint.storage.HDF5Storage.save"]], "save() (tlm_adjoint.storage.memorystorage method)": [[40, "tlm_adjoint.storage.MemoryStorage.save"]], "save() (tlm_adjoint.storage.storage method)": [[40, "tlm_adjoint.storage.Storage.save"]], "tangent_linear() (tlm_adjoint.storage.storage method)": [[40, "tlm_adjoint.storage.Storage.tangent_linear"]], "tlm_adjoint.storage": [[40, "module-tlm_adjoint.storage"]], "tangentlinearmap (class in tlm_adjoint.tangent_linear)": [[41, "tlm_adjoint.tangent_linear.TangentLinearMap"]], "id (tlm_adjoint.tangent_linear.tangentlinearmap property)": [[41, "tlm_adjoint.tangent_linear.TangentLinearMap.id"]], "tlm_adjoint.tangent_linear": [[41, "module-tlm_adjoint.tangent_linear"]], "equationmanager (class in tlm_adjoint.tlm_adjoint)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager"]], "add_equation() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.add_equation"]], "add_initial_condition() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.add_initial_condition"]], "annotation_enabled() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.annotation_enabled"]], "comm() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.comm"]], "compute_gradient() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.compute_gradient"]], "configure_checkpointing() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.configure_checkpointing"]], "configure_tlm() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.configure_tlm"]], "drop_references() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.drop_references"]], "finalize() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.finalize"]], "function_tlm() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.function_tlm"]], "info() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.info"]], "new() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.new"]], "new_block() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.new_block"]], "paused() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.paused"]], "reset() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.reset"]], "start() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.start"]], "stop() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.stop"]], "tlm_adjoint.tlm_adjoint": [[42, "module-tlm_adjoint.tlm_adjoint"]], "tlm_enabled() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.tlm_enabled"]], "var_tlm() (tlm_adjoint.tlm_adjoint.equationmanager method)": [[42, "tlm_adjoint.tlm_adjoint.EquationManager.var_tlm"]], "taylor_test() (in module tlm_adjoint.verification)": [[43, "tlm_adjoint.verification.taylor_test"]], "taylor_test_tlm() (in module tlm_adjoint.verification)": [[43, "tlm_adjoint.verification.taylor_test_tlm"]], "taylor_test_tlm_adjoint() (in module tlm_adjoint.verification)": [[43, "tlm_adjoint.verification.taylor_test_tlm_adjoint"]], "tlm_adjoint.verification": [[43, "module-tlm_adjoint.verification"]]}}) \ No newline at end of file