[ITensors] [ENHANCEMENT] Expose ConvergenceInfo in calls to KrylovKit methods

[NuclearPowerNerd]

I'm be curious what others think of the value of the following. I was recently trying to debug some issues I was having with linsolve via ITensorTDVP (which I understand will be deprecated by ITensorMPS). linsolve is built on top of KrylovKits methods and in this particular case their implementation of GMRES.

Underlying this all, there is a series of iterations which are trying to minimize a cost function (usually the square of the L2 norm of some residual vector, for linsolve its the cost function formed by doing two-site overlapping updates up the MPS chain). I found myself wondering how I could see this residual or "cost" for each iteration of the GMRES call. I don't think it is currently exposed in ITensors. I know with outputlevel you can see the maxtruncerr but I don't think that is the same as the residual or cost from GMRES. If I have that wrong please set me straight. From KrylovKit's documentation, the residual information is contained in a struct called ConvergenceInfo.

Starting from my specific use case of linsolve here is what I found:

• linsolve calls alternating_update
• alternating_update calls sweep_update
• sweep_update calls sub_sweep_update

Within sub_sweep_update there is a call to region_update! which does appear to return ConvergenceInfo from KyrlovKit as info (see line 117 of sweep_update.jl). However, it never makes it outside the call to sweep_update. So it would be a matter of propagating info back up the chain of return values. Probably some type of container would need to be created to hold the ConvergenceInfo for each call to region_update!. This could be done with a NamedTuple or maybe something more complex if its warranted.

Thoughts on this?