`info_ctmrg` fields in `leading_boundary`

[ogauthe]

Hi!

When running CTMRG with leading_boundary, the final convergence accuracy η is an important information, especially if the loop stopped due to reaching alg.maxiter. It would be nice if η was included into info_ctmrg, currently it cannot be accessed from outside. I can make a PR if you like the idea.

I also realized that info_ctmrg contains U_full and V_full for a total memory cost $32\chi^2D^4$ on a 2x2 unit cell. This is a lot. I understand these tensors are used in AD, where the pullback is only computed on a fully converged environment. If I am correct, it is not used in the (many) converging steps before calling AD, nor when AD is not needed at all. Would it be possible to use a dedicated function in AD that would also return U_full and V_full , but keep them out of the regular leading_boundary? The memory gain is clear but I am unsure how much code duplication this would generate, there is a tradeoff to be made.
EDIT: I realize this is only for SimultaneousCTMRG: SequentialCTMRG does not return these tensors.

[Yue-Zhengyuan]

It would be nice if η was included into info_ctmrg

This is included in @leburgel's PR #264 (which has been delayed for quite some time). Though you can still first make a PR if you need this right now.

[leburgel]

In the meantime there were some other requests for additional convergence info, in particular returning the final convergence error as well as a dedicated convergence flag to easily check if a contraction actually converged without having to increase the verbosity. I'll split this off from #264 and file a dedicated PR which also contains the convergence flag, no reason to have to wait until that is merged for something that simple.

About the full decompositions that are being kept around, it's not as simple as dropping them from all but the final CTMRG iteration unfortunately. By default we use a fixed-point differentiation approach, in which case we indeed only use the full decompositions in a single iteration that is being differentiated at the fixed point. For this case, we can keep the full decomposition around only in the pullback, and completely remove it from the forward pass. However, we also support a naive AD mode where we really backpropagate through all of the iterations, in which case we do need the full decomposition in every iteration. But also then, we could only keep the full decomposition around in the pullback of the iterations, removing them from the forward pass.

The main reason this is not done right now is that we still handle backwards differentiation rules in PEPSKit.jl in quite a "manual" way, instead of directly relying on the MatrixAlgebraKit.jl rules where some of these issues might be solved by now. This is more a legacy effect (MatrixAlgebraKit.jl didn't exist when this was all first implemented here), and could likely be solved in a more elegant way in future updates.

So, I'll start with the info since that's easy, but we are aware of the issues that come with our handling of AD of truncated decompositions and are looking to resolve these in the near future.

[leburgel]

The leading_boundary info now contains the converegence error as well as a dedicated convergence flag, and no longer contains any info on the decompositions (#365). The full decompositions are still present in the info returned by ctmrg_iteration. This will be resolved by #364, I'll leave this open until that's merged.

[leburgel]

Full decompositions are now only ever used in the backwards pass, so I think this is resolved now.