Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the trust region subsolver exchangeable. #294

Merged
merged 51 commits into from
Nov 2, 2023

Conversation

kellertuer
Copy link
Member

@kellertuer kellertuer commented Sep 25, 2023

This should resolve #240 and unifies notation as well.

To keep track:

  • η the subsolver iterate is now called Y
  • η_Cauchy the Cauchy point to check for the randomised variant is now called Z
  • random will not only be true/false but will allow to set the amount of randomness
  • documentation updated as well

This is still an early stage, but I wanted to keep track here. Especially, setting initial values and checking the radius afterwards are still quite intertwined.

@kellertuer kellertuer changed the title Kellertuer/tr subsolver Make the trust region subsolver exchangeable. Sep 25, 2023
@kellertuer kellertuer marked this pull request as ready for review September 29, 2023 17:51
@codecov
Copy link

codecov bot commented Sep 29, 2023

Codecov Report

Merging #294 (7f1707e) into master (ff5cd7e) will decrease coverage by 0.01%.
The diff coverage is 99.50%.

@@            Coverage Diff             @@
##           master     #294      +/-   ##
==========================================
- Coverage   99.74%   99.73%   -0.01%     
==========================================
  Files          76       78       +2     
  Lines        7090     7206     +116     
==========================================
+ Hits         7072     7187     +115     
- Misses         18       19       +1     
Files Coverage Δ
src/Manopt.jl 87.50% <ø> (ø)
src/functions/gradients.jl 100.00% <100.00%> (ø)
src/functions/manifold_functions.jl 100.00% <ø> (ø)
.../plans/adabtive_regularization_with_cubics_plan.jl 100.00% <100.00%> (ø)
src/plans/debug.jl 100.00% <100.00%> (ø)
src/plans/gradient_plan.jl 100.00% <ø> (ø)
src/plans/hessian_plan.jl 100.00% <ø> (ø)
src/plans/higher_order_primal_dual_plan.jl 100.00% <ø> (ø)
src/plans/plan.jl 100.00% <ø> (ø)
src/plans/problem.jl 100.00% <100.00%> (ø)
... and 21 more

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@kellertuer
Copy link
Member Author

kellertuer commented Sep 30, 2023

While this rework per se is done (besides the idea above to add a second subsolver), we still miss a sub_problem in the trust region solver for full generality; since that is defined in a tangent space, I would like to wait for JuliaManifolds/ManifoldsBase.jl#169 to be merged, though this might still take a bit of time.

edit: The docs could probably be shortened a bit, the TR docs currently describe a lot of things where we usually referred to the corresponding papers. I will update that the next days.

# Conflicts:
#	docs/src/solvers/truncated_conjugate_gradient_descent.md
#	src/data/artificialDataFunctions.jl
#	src/solvers/trust_regions.jl
#	test/solvers/test_trust_regions.jl
# Conflicts:
#	docs/src/solvers/DouglasRachford.md
but we might really first need 0.15 registered and Manifolds.jl and ManifoldDiff.jl bumped on dependencies before continuing
# Conflicts:
#	Project.toml
#	docs/Project.toml
#	docs/src/solvers/ChambollePock.md
#	ext/ManoptManifoldsExt/ARC_CG.jl
#	src/data/artificialDataFunctions.jl
but there is still some work to do, to make the new state constructors nice.
@kellertuer
Copy link
Member Author

This still needs a bit of rework, for best of cases also for the other states with sub solvers to be unified in interface and even easy to use with default sub solvers.

@kellertuer
Copy link
Member Author

kellertuer commented Oct 30, 2023

Hm, in reworking Lanczos / The sub solvers Objective I seem to have introduced a bug that is really hard to narrow down where that comes from.

edit: What a side effect, a stopping criterion modified the state, which they never should do.

@kellertuer
Copy link
Member Author

@blegat since we discussed a possible tutorial about methods required for a solver... I did not yet write such a tutorial here, but since I was changing a few things in the docs anyways, I wrote a short section https://manoptjl.org/previews/PR294/solvers/gradient_descent.html#Technical-Details that we could add to every solver, mentioning, which functions are required. Would that help for the use case you described in JuliaManifolds/Manifolds.jl#658?

@kellertuer kellertuer added the Ready-for-Review A label for pull requests that are feature-ready label Nov 1, 2023
@kellertuer
Copy link
Member Author

Everything is working now, I might have to check for a missing line in coverage and whether that is fixable, but the rest is done and good to go – the TR subsolver is fully replaceable.

@blegat
Copy link
Contributor

blegat commented Nov 2, 2023

Would that help for the use case you described in JuliaManifolds/Manifolds.jl#658?

Yes, I think that is a very valuable addition

@kellertuer
Copy link
Member Author

Cool. I can add more on a next PR, when I write the tutorial then – that might also help further to read those technical details.

@kellertuer kellertuer merged commit f47e87e into master Nov 2, 2023
13 of 15 checks passed
@kellertuer kellertuer deleted the kellertuer/TR-subsolver branch November 20, 2023 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ready-for-Review A label for pull requests that are feature-ready
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make the sub solver of trust regions replaceable
3 participants