Skip to content

Unwrap operator mass matrices before findall in Differentiation - #3944

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
singhharsh1708:fix-2929-matrixoperator-massmatrix-sparsity
Aug 1, 2026
Merged

Unwrap operator mass matrices before findall in Differentiation#3944
ChrisRackauckas merged 1 commit into
SciML:masterfrom
singhharsh1708:fix-2929-matrixoperator-massmatrix-sparsity

Conversation

@singhharsh1708

@singhharsh1708 singhharsh1708 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Removes the MethodError crash sites on the path reported in #2929. #2929 itself stays open — see below.

Bug

Several places in OrdinaryDiffEqDifferentiation seed a sparsity pattern from the mass matrix with

idxs = findall(!iszero, f.mass_matrix)
@. @view(sparsity[idxs]) = @view(f.mass_matrix[idxs])

Both findall and getindex need a concrete matrix. A SciMLOperator mass matrix supports neither, so combining one with a jac_prototype throws MethodError: no method matching keys(::MatrixOperator{...}):

using OrdinaryDiffEqDifferentiation, SparseArrays, SciMLOperators, SciMLBase, ADTypes
M = sparse([1.0 2.0; 0.0 1.0])
mass_matrix = MatrixOperator(M; update_func = (A, u, p, t) -> M)
jac_prototype = sparse([1.0 0.0; 0.0 1.0])
f!(du, u, p, t) = (du .= u; nothing)
odef = ODEFunction(f!; mass_matrix, jac_prototype, sparsity = copy(jac_prototype))
prob = ODEProblem(odef, ones(2), (0.0, 1.0))
OrdinaryDiffEqDifferentiation.prepare_user_sparsity(AutoForwardDiff(), prob)
# MethodError: no method matching keys(::MatrixOperator{...})

Fix

One helper, used at all four sites that had the pattern — prepare_user_sparsity and its sparsity unwrap in alg_utils.jl, and build_jac_config and sparsity_colorvec in derivative_wrappers.jl:

concrete_mass_matrix(mm) = mm
concrete_mass_matrix(mm::AbstractSciMLOperator) = convert(AbstractMatrix, mm)

convert reads the operator's currently-cached array rather than re-evaluating it, so no update_coefficients! is involved. That is what is wanted here: only the structural pattern matters, and prepare_alg runs before init, so the cached array is the one the operator was constructed with. Non-operator mass matrices dispatch to the identity method, so plain-matrix behaviour is unchanged.

Besides MatrixOperator this also covers DiagonalOperator, IdentityOperator, ScaledOperator, AddedOperator, ComposedOperator and TensorProductOperator, which all define convert(AbstractMatrix, ·). ScalarOperator, FunctionOperator and AffineOperator have no such method and still error, but they were already failing earlier on keys and none is a plausible mass matrix here.

What this does not fix

#2929's reproducer calls solve, and that still fails — but for a different reason and further along. With these four sites fixed it gets past every keys error and reaches the sparse-W branch of jacobian2W! (derivative_utils.jl), which broadcasts the operator directly into a SparseMatrixCSC and stack-overflows. So #2929 needs at least one more change and should stay open; this PR is a strict improvement rather than a resolution.

Measured, same MWE: master fails with MethodError: keys(::MatrixOperator); this branch fails with StackOverflowError from jacobian2W!.

An earlier version of this description blamed a SciMLOperators setfield! bug for the remaining failure. That was SciML/SciMLOperators.jl#409, fixed by #416 and released in SciMLOperators v1.25.2, and is no longer relevant.

Tests

New prepare_user_sparsity_tests.jl: MatrixOperator and DiagonalOperator mass matrices, plus a case where sparsity is left to default to jac_prototype (how #2929 constructs the problem). Three of those error on master with the keys MethodError and pass here. Two further testsets pin the UniformScaling and plain-sparse paths as unchanged, and pass on both sides.

Branch: 9/9. OrdinaryDiffEqDifferentiation Core suite goes from 61 passes on master to 67 here.

AI Disclosure

Claude assisted with this work.

@singhharsh1708
singhharsh1708 force-pushed the fix-2929-matrixoperator-massmatrix-sparsity branch 2 times, most recently from dd6b78a to 9410f9b Compare July 21, 2026 14:31
@singhharsh1708
singhharsh1708 force-pushed the fix-2929-matrixoperator-massmatrix-sparsity branch 3 times, most recently from f5a6694 to ca27c50 Compare August 1, 2026 21:26
Seeding a sparsity pattern from the mass matrix uses findall and getindex,
which a SciMLOperator does not support, so a MatrixOperator mass matrix
combined with a jac_prototype threw MethodError: keys(::MatrixOperator).

The same pattern appeared at four places: prepare_user_sparsity and its
sparsity unwrap in alg_utils.jl, and build_jac_config and sparsity_colorvec
in derivative_wrappers.jl. Route all four through one concrete_mass_matrix
helper, which reads the operator's cached array rather than re-evaluating it,
since only the structural pattern matters and this runs before init.
@singhharsh1708
singhharsh1708 force-pushed the fix-2929-matrixoperator-massmatrix-sparsity branch from ca27c50 to f52f8e1 Compare August 1, 2026 22:16
@singhharsh1708 singhharsh1708 changed the title prepare_user_sparsity: unwrap operator mass matrices before findall (#2929) Unwrap operator mass matrices before findall in Differentiation Aug 1, 2026
@ChrisRackauckas
ChrisRackauckas merged commit abfc05f into SciML:master Aug 1, 2026
159 of 183 checks passed
@singhharsh1708
singhharsh1708 deleted the fix-2929-matrixoperator-massmatrix-sparsity branch August 1, 2026 22:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants