Rework high level interfaces (#392)

* trigger all ambiguity errors.
* Move dispatch on p into subfunctions, that existed before anyways.
* rework the safeguards to dispatch internally.
* remove deprecated definitions.
* remove all deprecated parameters.
* rename set_manopt_parameter! to set_parameter and get_manopt_parameter to get_parameter
* unify stabilisation through projection keyword.
* remove `update_stopping_criterion` to update values in the stopping criterion and submerge it in the set_parameter! scheme
* unify state constructors of ALM and ARC
* Adapt EPM.
* Finish DoC and DCPPA
* Unify how p and X are passed to states overall in all states.
* Sketch a first idea of a factory.
* Fix scaling parameter in quasi newton. This resolvs #382
* Start changing the docs snippets to using a documentation glossary
* 📚Finish the glossary work
* Unify signatures, fix a few tex typos and improve english on the factory info/note.

---------

Co-authored-by: Hajg Jasa <[email protected]>
Co-authored-by: Bagaev Dmitry <[email protected]>
Co-authored-by: Mateusz Baran <[email protected]>

.codecov.yml

@@ -1,4 +1,3 @@
 ignore:
-  - "src/examples/.*"
-  - "src/tutorials/.*"
   - "src/helpers/exports/.*" # exclude exports, since there are no CI tests for Asy-exports
+  - "src/documentation_glossary.jl" # exclude this since it is just use to create the docs and code cov goes bogus on this.

.vale.ini

@@ -7,7 +7,6 @@ Packages = Google
 [formats]
 # code blocks with Julia in Markdown do not yet work well
 qmd = md
-jl = md
 
 [docs/src/*.md]
 BasedOnStyles = Vale, Google

Changelog.md

@@ -5,16 +5,105 @@ All notable Changes to the Julia package `Manopt.jl` will be documented in this
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [0.4.70] WIP
-
-### Added
-
+# [0.5.0] – August 29, 2024
+
+This breaking update is mainly concerned with improving a unified experience through all solvers
+and some usability improvements, such that for example the different gradient update rules are easier to specify.
+
+In general we introduce a few factories, that avoid having to pass the manifold to keyword arguments
+
+## Added
+
+* A `ManifoldDefaultsFactory` that postpones the creation/allocation of manifold-specific fields in for example direction updates, step sizes and stopping criteria. As a rule of thumb, internal structures, like a solver state should store the final type. Any high-level interface, like the functions to start solvers, should accept such a factory in the appropriate places and call the internal `_produce_type(factory, M)`, for example before passing something to the state.
+* a `documentation_glossary.jl` file containing a glossary of often used variables in fields, arguments, and keywords, to print them in a unified manner. The same for usual sections, tex, and math notation that is often used within the doc-strings.
+
+## Changed
+
+* Any `Stepsize` now hase a `Stepsize` struct used internally as the original `struct`s before. The newly exported terms aim to fit `stepsize=...` in naming and create a `ManifoldDefaultsFactory` instead, so that any stepsize can be created without explicitly specifying the manifold.
+  * `ConstantStepsize` is no longer exported, use `ConstantLength` instead. The length parameter is now a positional argument following the (optonal) manifold. Besides that `ConstantLength` works as before,just that omitting the manifold fills the one specified in the solver now.
+  * `DecreasingStepsize` is no longer exported, use `DecreasingLength` instead. `ConstantLength` works as before,just that omitting the manifold fills the one specified in the solver now.
+  * `ArmijoLinesearch` is now called `ArmijoLinesearchStepsize`. `ArmijoLinesearch` works as before,just that omitting the manifold fills the one specified in the solver now.
+  * `WolfePowellLinesearch` is now called `WolfePowellLinesearchStepsize`, its constant `c_1` is now unified with Armijo and called `sufficient_decrease`, `c_2` was renamed to `sufficient_curvature`. Besides that, `WolfePowellLinesearch` works as before, just that omitting the manifold fills the one specified in the solver now.
+  * `WolfePowellBinaryLinesearch` is now called `WolfePowellBinaryLinesearchStepsize`, its constant `c_1` is now unified with Armijo and called `sufficient_decrease`, `c_2` was renamed to `sufficient_curvature`. Besides that, `WolfePowellBinaryLinesearch` works as before, just that omitting the manifold fills the one specified in the solver now.
+  * `NonmonotoneLinesearch` is now called `NonmonotoneLinesearchStepsize`. `NonmonotoneLinesearch` works as before, just that omitting the manifold fills the one specified in the solver now.
+  * `AdaptiveWNGradient` is now called `AdaptiveWNGradientStepsize`. Its second positional argument, the gradient function was only evaluated once for the `gradient_bound` default, so it has been replaced by the keyword `X=` accepting a tangent vector. The last positional argument `p` has also been moved to a keyword argument. Besides that, `AdaptiveWNGradient` works as before, just that omitting the manifold fills the one specified in the solver now.
+* Any `DirectionUpdateRule` now has the `Rule` in its name, since the original name is used to create the `ManifoldDefaultsFactory` instead. The original constructor now no longer requires the manifold as a parameter, that is later done in the factory. The `Rule` is, however, also no longer exported.
+  * `AverageGradient` is now called `AverageGradientRule`. `AverageGradient` works as before, but the manifold as its first parameter is no longer necessary and `p` is now a keyword argument.
+  * The `IdentityUpdateRule` now accepts a manifold optionally for consistency, and you can use `Gradient()` for short as well as its factory. Hence `direction=Gradient()` is now available.
+  * `MomentumGradient` is now called `MomentumGradientRule`. `MomentumGradient` works as before, but the manifold as its first parameter is no longer necessary and `p` is now a keyword argument.
+  * `Nesterov` is now called `NesterovRule`. `Nesterov` works as before, but the manifold as its first parameter is no longer necessary and `p` is now a keyword argument.
+  * `ConjugateDescentCoefficient` is now called `ConjugateDescentCoefficientRule`. `ConjugateDescentCoefficient` works as before, but can now use the factory in between
+  * the `ConjugateGradientBealeRestart` is now called `ConjugateGradientBealeRestartRule`. For the `ConjugateGradientBealeRestart` the manifold is now a first parameter, that is not necessary and no longer the `manifold=` keyword.
+  * `DaiYuanCoefficient` is now called `DaiYuanCoefficientRule`. For the `DaiYuanCoefficient` the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the `vector_transport_method=` keyword.
+  * `FletcherReevesCoefficient` is now called `FletcherReevesCoefficientRule`. `FletcherReevesCoefficient` works as before, but can now use the factory in between
+  * `HagerZhangCoefficient` is now called `HagerZhangCoefficientRule`. For the `HagerZhangCoefficient` the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the `vector_transport_method=` keyword.
+  * `HestenesStiefelCoefficient` is now called `HestenesStiefelCoefficientRule`. For the `HestenesStiefelCoefficient` the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the `vector_transport_method=` keyword.
+  * `LiuStoreyCoefficient` is now called `LiuStoreyCoefficientRule`. For the `LiuStoreyCoefficient` the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the `vector_transport_method=` keyword.
+  * `PolakRibiereCoefficient` is now called `PolakRibiereCoefficientRule`. For the `PolakRibiereCoefficient` the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the `vector_transport_method=` keyword.
+  * the `SteepestDirectionUpdateRule` is now called `SteepestDescentCoefficientRule`. The `SteepestDescentCoefficient` is equivalent, but creates the new factory interims wise.
+  * `AbstractGradientGroupProcessor` is now called `AbstractGradientGroupDirectionRule`
+    * the `StochasticGradient` is now called `StochasticGradientRule`. The `StochasticGradient` is equivalent, but creates the new factory interims wise, so that the manifold is not longer necessary.
+  * the `AlternatingGradient` is now called `AlternatingGradientRule`.
+  The `AlternatingGradient` is equivalent, but creates the new factory interims wise, so that the manifold is not longer necessary.
+* `quasi_Newton` had a keyword `scale_initial_operator=` that was inconsistently declared (sometimes bool, sometimes real) and was unused.
+  It is now called `initial_scale=1.0` and scales the initial (diagonal, unit) matrix within the approximation of the Hessian additionally to the $\frac{1}{\lVert g_k\rVert}$ scaling with the norm of the oldest gradient for the limited memory variant. For the full matrix variant the initial identity matrix is now scaled with this parameter.
 * Unify doc strings and presentation of keyword arguments
   * general indexing, for example in a vector, uses `i`
   * index for inequality constraints is unified to `i` running from `1,...,m`
   * index for equality constraints is unified to `j` running from `1,...,n`
   * iterations are using now `k`
-* Doc strings unified and even reusing similar docstring snippets.
+* `get_manopt_parameter` has been renamed to `get_parameter` since it is internal,
+  so internally that is clear; accessing it from outside hence reads anyways `Manopt.get_parameter`
+* `set_manopt_parameter!` has been renamed to `set_parameter!` since it is internal,
+  so internally that is clear; accessing it from outside hence reads `Manopt.set_parameter!`
+* changed the `stabilize::Bool=` keyword in `quasi_Newton` to the more flexible `project!=`
+  keyword, this is also more in line with the other solvers. Internally the same is done
+  within the `QuasiNewtonLimitedMemoryDirectionUpdate`. To adapt,
+  * the previous `stabilize=true` is now set with `(project!)=embed_project!` in general,
+    and if the manifold is represented by points in the embedding, like the sphere, `(project!)=project!` suffices
+  * the new default is `(project!)=copyto!`, so by default no projection/stabilization is performed.
+* the positional argument `p` (usually the last or the third to last if subsolvers existed) has been moved to a keyword argument `p=` in all State constructors
+* in `NelderMeadState` the `population` moved from positional to keyword argument as well,
+* the way to initialise sub solvers in the solver states has been unified In the new variant
+  * the `sub_problem` is always a positional argument; namely the last one
+  * if the `sub_state` is given as a optional positional argument after the problem, it has to be a manopt solver state
+  * you can provide the new `ClosedFormSolverState(e::AbstractEvaluationType)` for the state
+    to indicate that the `sub_problem` is a closed form solution (function call) and how it
+    has to be called
+  * if you do not provide the `sub_state` as positional, the keyword `evaluation=` is used
+    to generate the state `ClosedFormSolverState`.
+  * when previously `p` and eventually `X` where positional arguments, they are now moved
+    to keyword arguments of the same name for start point and tangent vector.
+  * in detail
+    * `AdaptiveRegularizationState(M, sub_problem [, sub_state]; kwargs...)` replaces
+      the (anyways unused) variant to only provide the objective; both `X` and `p` moved to keyword arguments.
+    * `AugmentedLagrangianMethodState(M, objective, sub_problem; evaluation=...)` was added
+    * ``AugmentedLagrangianMethodState(M, objective, sub_problem, sub_state; evaluation=...)` now has `p=rand(M)` as keyword argument instead of being the second positional one
+    * `ExactPenaltyMethodState(M, sub_problem; evaluation=...)` was added and `ExactPenaltyMethodState(M, sub_problem, sub_state; evaluation=...)` now has `p=rand(M)` as keyword argument instead of being the second positional one
+    * `DifferenceOfConvexState(M, sub_problem; evaluation=...)` was added and `DifferenceOfConvexState(M, sub_problem, sub_state; evaluation=...)` now has `p=rand(M)` as keyword argument instead of being the second positional one
+    * `DifferenceOfConvexProximalState(M, sub_problem; evaluation=...)` was added and `DifferenceOfConvexProximalState(M, sub_problem, sub_state; evaluation=...)` now has `p=rand(M)` as keyword argument instead of being the second positional one
+  * bumped `Manifolds.jl`to version 0.10; this mainly means that any algorithm working on a productmanifold and requiring `ArrayPartition` now has to explicitly do `using RecursiveArrayTools`.
+### Fixed
+
+* the `AverageGradientRule` filled its internal vector of gradients wrongly – or mixed it up in parallel transport. This is now fixed.
+
+### Removed
+
+* the `convex_bundle_method` and its `ConvexBundleMethodState` no longer accept the keywords `k_size`, `p_estimate` nor `ϱ`, they are superseded by just providing `k_max`.
+* the `truncated_conjugate_gradient_descent(M, f, grad_f, hess_f)` has the Hessian now
+   a mandatory argument. To use the old variant,
+   provide `ApproxHessianFiniteDifference(M, copy(M, p), grad_f)` to `hess_f` directly.
+* all deprecated keyword arguments and a few function signatures were removed:
+  * `get_equality_constraints`, `get_equality_constraints!`, `get_inequality_constraints`, `get_inequality_constraints!` are removed. Use their singular forms and set the index to `:` instead.
+  * `StopWhenChangeLess(ε)` is removed, use ``StopWhenChangeLess(M, ε)` instead to fill for example the retraction properly used to determine the change
+ * In the `WolfePowellLinesearch` and  `WolfeBinaryLinesearch`the `linesearch_stopsize=` keyword is replaced by `stop_when_stepsize_less=`
+ * `DebugChange` and `RecordChange` had a `manifold=` and a `invretr` keyword that were replaced by the first positional argument `M` and `inverse_retraction_method=`, respectively
+ * in the `NonlinearLeastSquaresObjective` and `LevenbergMarquardt` the `jacB=` keyword is now called `jacobian_tangent_basis=`
+ * in `particle_swarm` the `n=` keyword is replaced by `swarm_size=`.
+ * `update_stopping_criterion!` has been removed and unified with `set_parameter!`. The code adaptions are
+   * to set a parameter of a stopping criterion, just replace `update_stopping_criterion!(sc, :Val, v)` with `set_parameter!(sc, :Val, v)`
+   * to update a stopping criterion in a solver state, replace the old `update_stopping_criterion!(state, :Val, v)` tat passed down to the stopping criterion by the explicit pass down with `set_parameter!(state, :StoppingCriterion, :Val, v)`
+
 
 ## [0.4.69] – August 3, 2024
 

Project.toml

@@ -1,7 +1,7 @@
 name = "Manopt"
 uuid = "0fc0a36d-df90-57f3-8f93-d78a9fc72bb5"
 authors = ["Ronny Bergmann <[email protected]>"]
-version = "0.4.70"
+version = "0.5.0"
 
 [deps]
 ColorSchemes = "35d6a980-a343-548e-a6ea-1d62b119f2f4"
@@ -27,13 +27,15 @@ LineSearches = "d3d80556-e9d4-5f37-9878-2ab0fcc64255"
 Manifolds = "1cead3c2-87b3-11e9-0ccd-23c62b72b94e"
 Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
 QuadraticModels = "f468eda6-eac5-11e8-05a5-ff9e497bcd19"
+RecursiveArrayTools = "731186ca-8d62-57ce-b412-fbd966d074cd"
 RipQP = "1e40b3f8-35eb-4cd8-8edd-3e515bb9de08"
 
 [extensions]
 ManoptJuMPExt = "JuMP"
 ManoptLRUCacheExt = "LRUCache"
 ManoptLineSearchesExt = "LineSearches"
 ManoptManifoldsExt = "Manifolds"
+ManoptRecursiveArrayToolsExt = "RecursiveArrayTools"
 ManoptRipQPQuadraticModelsExt = ["RipQP", "QuadraticModels"]
 
 [compat]
@@ -49,21 +51,22 @@ LRUCache = "1.4"
 LineSearches = "7.2.0"
 LinearAlgebra = "1.6"
 ManifoldDiff = "0.3.8"
-Manifolds = "0.9.11"
+Manifolds = "0.9.11, 0.10"
 ManifoldsBase = "0.15.10"
-ManoptExamples = "0.1.4"
+ManoptExamples = "0.1.8"
 Markdown = "1.6"
 Plots = "1.30"
 Preferences = "1.4"
 Printf = "1.6"
 QuadraticModels = "0.9"
 Random = "1.6"
+RecursiveArrayTools = "2, 3"
 Requires = "0.5, 1"
 RipQP = "0.6.4"
 SparseArrays = "1.6"
 Statistics = "1.6"
 Test = "1.6"
-julia = "1.6"
+julia = "1.8"
 
 [extras]
 Aqua = "4c88cf16-eb10-579e-8560-4a9242c79595"
@@ -76,8 +79,9 @@ Manifolds = "1cead3c2-87b3-11e9-0ccd-23c62b72b94e"
 ManoptExamples = "5b8d5e80-5788-45cb-83d6-5e8f1484217d"
 Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
 QuadraticModels = "f468eda6-eac5-11e8-05a5-ff9e497bcd19"
+RecursiveArrayTools = "731186ca-8d62-57ce-b412-fbd966d074cd"
 RipQP = "1e40b3f8-35eb-4cd8-8edd-3e515bb9de08"
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
 
 [targets]
-test = ["Test", "Aqua", "ForwardDiff", "JuMP", "Manifolds", "ManoptExamples", "ManifoldDiff", "Plots", "LineSearches", "LRUCache", "RipQP", "QuadraticModels"]
+test = ["Test", "Aqua", "ForwardDiff", "JuMP", "Manifolds", "ManoptExamples", "ManifoldDiff", "Plots", "LineSearches", "LRUCache", "RecursiveArrayTools", "RipQP", "QuadraticModels"]

docs/Project.toml

@@ -21,6 +21,7 @@ Manopt = "0fc0a36d-df90-57f3-8f93-d78a9fc72bb5"
 Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
 QuadraticModels = "f468eda6-eac5-11e8-05a5-ff9e497bcd19"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
+RecursiveArrayTools = "731186ca-8d62-57ce-b412-fbd966d074cd"
 RipQP = "1e40b3f8-35eb-4cd8-8edd-3e515bb9de08"
 
 [compat]
@@ -39,9 +40,10 @@ JuMP = "1"
 LRUCache = "1"
 LineSearches = "7"
 Literate = "2"
-Manifolds = "0.8.81, 0.9"
+Manifolds = "0.8.81, 0.9, 0.10"
 ManifoldsBase = "0.14.12, 0.15"
-Manopt = "0.4"
+Manopt = "0.4, 0.5"
 Plots = "1"
 QuadraticModels = "0.9.6"
+RecursiveArrayTools = "2, 3"
 RipQP = "0.6.4"

docs/make.jl

@@ -47,6 +47,8 @@ if "--quarto" ∈ ARGS
         tutorials_folder = (@__DIR__) * "/../tutorials"
         # instantiate the tutorials environment if necessary
         Pkg.activate(tutorials_folder)
+        # For a breaking release -> also set the tutorials folder to the most recent version
+        Pkg.develop(PackageSpec(; path=(@__DIR__) * "/../"))
         Pkg.resolve()
         Pkg.instantiate()
         Pkg.build("IJulia") # build `IJulia` to the right version.
@@ -72,7 +74,7 @@ end
 # (c) load necessary packages for the docs
 using Documenter
 using DocumenterCitations, DocumenterInterLinks
-using JuMP, LineSearches, LRUCache, Manopt, Manifolds, Plots
+using JuMP, LineSearches, LRUCache, Manopt, Manifolds, Plots, RecursiveArrayTools
 using RipQP, QuadraticModels
 
 # (d) add contributing.md to docs