Some basic svd forward rules and tests

[kshyatt]

Definitely not optimized...

[kshyatt]

Took another look at this. The Enzyme tests seem to be failing because of finite differences, for svd_full for example when size(A) == (17, 19), the finite differences result for dU is all over the place compared to that from the rule for the section of U which is only present in the full case. I'll try to think of a nice way to handle this, I think it is not occurring for Mooncake because Mooncake uses a different technique for checking against FD.

[github-actions]

Your PR no longer requires formatting changes. Thank you for your contribution!

[Jutho]

In the gauge fixing parts, we could use more views for the slicing ΔU₁[I] etc, but I did not want to push my luck on the GPU with views into views using CartesianIndices.

[Jutho]

Anyway, I will leave this aside now for a while and look at some other PRs. In principle, tests can now be expanded to cover the complex case and svd_compact of rectangular matrices as well (but as always, I would only consider double precision for finite difference comparisons).

[lkdvos]

Left some final small comments, otherwise good to go?

[lkdvos]

I think below only the sum and difference are actually used, we could use a kernel like

MatrixAlgebraKit.jl/src/implementations/polar.jl

Lines 209 to 220 in e271b4a

	function _avgdiff!(A::AbstractArray, B::AbstractArray)
	axes(A) == axes(B) || throw(DimensionMismatch())
	@simd for I in eachindex(A, B)
	@inbounds begin
	a = A[I]
	b = B[I]
	A[I] = (a + b) / 2
	B[I] = b - a
	end
	end
	return A, B
	end

to avoid the two extra allocations, but I'm also happy to just leave them as-is, it's hard to imagine this really making that huge of a difference

[lkdvos]

Probably best to either not define this or explicitly error()?

[kshyatt]

The GPU tests will probably fail until a new version is tagged at GPUArrays (JuliaGPU/GPUArrays.jl#738)