Introduce proxes.

[kellertuer]

Introduce the first proximal maps to this collection. I will surely add more docs and testing.

@mateuszbaran is it ok to also define proximal maps for basic functions (like the distance) here? Otherwise I can also defined them in ManoptExamples.jl, but I thought for such “elementary” functions like the distance it might be nice to have them here already.

This PR will be accompanied by two PRs over on ManoptExamples (JuliaManifolds/ManoptExamples.jl#12) and Manopt (JuliaManifolds/Manopt.jl#331) to clean up the later a bit.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7fb4aa7) 95.26% compared to head (f1523ba) 95.46%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #37      +/-   ##
==========================================
+ Coverage   95.26%   95.46%   +0.20%     
==========================================
  Files          20       21       +1     
  Lines         359      375      +16     
==========================================
+ Hits          342      358      +16     
  Misses         17       17              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mateuszbaran]

What is the benefit of having proximal maps here instead of Manopt.jl? Can they be used for anything other than optimization?

[kellertuer]

I am not sure – they should not be defined in Manopt.jl, since that package should focus on optimization and not corresponding functions /cost/grad/...) my alternative would be to defined them in ManopExamples.jl, but I feel the prox is as elementary as the gradient, especially for things like the distance.
According to https://en.wikipedia.org/wiki/Proximal_operator – it seems to mainly be used in optimization, that is true, but I would claim the same for the gradient https://en.wikipedia.org/wiki/Gradient

[mateuszbaran]

OK, but if we are going to standardize it maybe we should consider a bit more generic interface, maybe more similar to https://github.com/JuliaFirstOrder/ProximalOperators.jl ?

[kellertuer]

That would add a dependency here, but I think that sounds like a good idea.

[kellertuer]

Or do you think we should introduce our own generic interface here? Until now we do not do that for gradients of differentials or such, so why do it for proxes?

Until now we just have a naming convention, how to find gradients / differentials / ...

[mateuszbaran]

I wouldn't necessarily make that a dependency but an interface inspired by that package. We need manifold as an additional argument after all.

Or do you think we should introduce our own generic interface here? Until now we do not do that for gradients of differentials or such, so why do it for proxes?

Until now we just have a naming convention, how to find gradients / differentials / ...

To be honest, I think we should also have such interface for gradients and differentials. I don't like our current convention, I just didn't have time to work on something better.

[kellertuer]

But then maybe let's continue with introducing proxies also using the naming convention still,
but open an issue to have a nicer interface for all these functions (grad/diff/adjoint diff/Hessian/prox) later. We would keep the naming scheme and the implementations but mainly have an interface on top I think?

And sure I also am not that fond of the naming scheme – such an interface like ProximalOperators has would be nice, sure.

[mateuszbaran]

We would keep the naming scheme and the implementations but mainly have an interface on top I think?

OK, we can postpone it but I'd try implementing all proximal operators as methods of prox! directly.

[kellertuer]

Ah, but then the interface would be breaking (and similarly for grad/diff/...)? Sure that would be fine with me.

[mateuszbaran]

Yes, it would be a breaking change.

[kellertuer]

Ok, let me do this one without a break – but I can open an issue and write down the design we have in mind here.
Maybe I use that then as an excuse to still not continue with the test suite but do the breaking change once I am done with this group of PRs here.

[kellertuer]

I will need this first in the examples, so this PR has to go first and should be finished by now
(having finished the examples I will update Manopt accordingly).

[mateuszbaran]

I have a few small comments but I'd like to take a look at it again after I understand how exactly you define the proximal map here.

[mateuszbaran]

Last batch of comments :). Everything else looks OK.

[mateuszbaran]

Ref. JuliaFirstOrder/ProximalOperators.jl#117

[kellertuer]

Oh my search-and-replace f->p_data was not careful enough there. Thanks for always being precise here :)