Many tests very sensitive to the RNG

[fredrikekre]

In Julia 1.7 the global rng changed and that caused many test failures in this package. I adjusted test-matrices a bit in cbb92e7 to get tests running, but it is a bit worrisome that tests are so sensitive to the RNG, I would have thought the methods would be more stable in many of the examples here. Could it indicate that something is wrong in some of the implementations? (See #316 for lobpcg specifivally). Would be good if someone more familiar with the implementation could check.

[jarlebring]

I think

IterativeSolvers.jl/test/gmres.jl

Line 30 in cbb92e7

@test norm(F \ (A * x - b)) / norm(b) ≤ reltol

should be

    @test norm(F \ (A * x - b)) / norm(F \ b) ≤ reltol

since termination is based on the preconditioned residual

IterativeSolvers.jl/src/gmres.jl

Line 18 in cbb92e7

current::resT # Current, absolute, preconditioned residual

That change seems to fix the test failure for T=Float32. In fact termination is based on the cheap matvec-free estimation of the residual norm so for other matrices this could still fail.

Also, the property that the residual is non-increasing is very prone to rounding errors so this test also seems sensitive to which problem it is applied to:

IterativeSolvers.jl/test/gmres.jl

Lines 22 to 25 in cbb92e7

	# Test optimality condition: residual should be non-increasing
	x, history = gmres(A, b, log=true, restart=3, maxiter=10, reltol=reltol);
	@test isa(history, ConvergenceHistory)
	@test all(diff(history[:resnorm]) .<= 0.0)

[jarlebring]

In general, I'm not sure Random should be used for test of iterative methods considering that the generated sequence is likely to change in future julia release. The convergence is a property very sensitive to the matrix and there is always a substantial risk it will not work for a given new random matrix. Changing the tolerance, etc are quick fixes which lessens the likelyhood of test failure but never removes it completely.

[fredrikekre]

Yea, would be good to use https://github.com/JuliaRandom/StableRNGs.jl