Skip to content

Make simplification deterministic w.r.t. hash/objectid ordering#4675

Draft
baggepinnen wants to merge 1 commit into
masterfrom
fbc/simplification-determinism
Draft

Make simplification deterministic w.r.t. hash/objectid ordering#4675
baggepinnen wants to merge 1 commit into
masterfrom
fbc/simplification-determinism

Conversation

@baggepinnen

Copy link
Copy Markdown
Contributor

Problem

mtkcompile can produce different results run-to-run, even within a single Julia session (observed ~9/100 on metadata/array/connector-heavy models). The structural symbol hash is actually stable (SymbolicUtils deliberately avoids objectid), so the leak is elsewhere:

Several pipeline stages order observable output by iterating Dict/Set collections keyed by types. Type iteration order follows objectid/hash(::DataType), which is not stable for types freshly created per compile (e.g. FunctionWrapper{R,Tuple{…}} / closure parameter types that multibody-style models route into nonnumeric parameter buffers). This permutes the parameter buffer layout and generated-function argument order — invisible in the equation list, which is why it's easy to miss. Builtin-typed parameters hash stably, which is why the symptom is intermittent.

Reproduced deterministically with parameters carrying fresh struct symtypes: 176/200 distinct buffer layouts over 200 in-session compiles (with GC.gc(true) between runs); 1/200 after this PR.

A second, independent source: IfLifting stored its condition→variable map in a plain Dict and named the generated condition variables with gensym (global counter) — both non-deterministic. That reproducer went 300/300 distinct → 1/300.

Changes

  • lib/ModelingToolkitBase/src/canonical_ordering.jl (new): a shared, hash/objectid-independent canonical_sort_key / canonical_name / canonical_sort, keyed on variable names (mirrors the approach already used in ModelingToolkitTearing).
  • index_cache.jl: order the nonnumeric/constant/discrete parameter buffers (and the symbols within each) by canonical_sort_key of the parameters, not by Dict{TypeT,Set} / Set{TypeT} iteration. This is the primary fix.
  • if_lifting.jl: conditions is now an OrderedDict, and condition variables get deterministic ifelse_cond_N names instead of gensym.
  • sccnonlinearproblem.jl: order SCC cache buffers/types canonically in build_caches!.
  • test/nondeterminism_simplification.jl (new, registered in group_interfaceii.jl): compiles representative models 150× with GC.gc(true) between runs and asserts identical compiled structure.

Verification

  • index_cache & if-lifting reproducers: distinct=1 (were 176/200 and 300/300).
  • Functional solve + symbolic parameter access correct after the buffer relayout.
  • Existing if_lifting.jl passes; scc_nonlinear_problem.jl core testsets pass.

Not included (follow-up)

  • ModelingToolkitTearing: the lexicographic equation sort (bounded_string in tearingstate.jl) keys on the printed form, whose Add/Mul operand order is in principle hash-dependent. It is currently stable (operand iteration verified stable), so this is latent and lives in a separate repo; suggested patch is to sort by canonical incidence instead:
    sortidxs = sortperm(
        [sort!([canonical_sort_key(v) for v in symbolic_incidence[i]]) for i in eachindex(eqs)];
        alg = Base.Sort.DEFAULT_STABLE)
    (may change which valid tearing arrangement is chosen — re-check test/structural_transformation/tearing.jl).
  • HashedRandom initial-guess seeding (seeded by hash(var)) left as-is.

Draft pending a full Pkg.test run; please watch parameter-layout- and tearing-sensitive tests.

🤖 Generated with Claude Code

Several pipeline stages ordered observable output by iterating Dict/Set
collections keyed by symbolic variables or by types. Type iteration order
follows objectid, which is not stable for types freshly created per compile
(e.g. FunctionWrapper/closure parameter types), so the parameter buffer layout
(and generated-function argument order) could differ run-to-run within a single
session - producing non-reproducible simplification/results. (Builtin-typed
params are stable, hence the bug is intermittent.)

- Add lib/ModelingToolkitBase/src/canonical_ordering.jl: a shared,
  hash/objectid-independent `canonical_sort_key`/`canonical_name`/`canonical_sort`
  keyed on variable names.
- index_cache.jl: order the nonnumeric/constant/discrete parameter buffers and
  their members by `canonical_sort_key` of the parameters rather than by the
  Dict{TypeT,Set}/Set{TypeT} iteration order.
- if_lifting.jl: store conditions in an OrderedDict and name the generated
  condition variables deterministically (`ifelse_cond_N`) instead of `gensym`
  (whose global counter was itself a source of non-determinism).
- sccnonlinearproblem.jl: order SCC cache buffers/types canonically in
  `build_caches!`.
- Add test/nondeterminism_simplification.jl (compiles models repeatedly under
  GC stress and asserts identical compiled structure) and register it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
canonical tie-break falls back to the original order among the colliding entries.
"""
function canonical_name(x::SymbolicT)
Moshi.Match.@match x begin

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Runic] reported by reviewdog 🐶

Suggested change
Moshi.Match.@match x begin
return Moshi.Match.@match x begin

Comment on lines +51 to +53
BSImpl.Div(;) => :/
BSImpl.ArrayOp(;) => Symbol("#arrayop")
BSImpl.Const(;) => Symbol("#const")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Runic] reported by reviewdog 🐶

Suggested change
BSImpl.Div(;) => :/
BSImpl.ArrayOp(;) => Symbol("#arrayop")
BSImpl.Const(;) => Symbol("#const")
BSImpl.Div() => :/
BSImpl.ArrayOp() => Symbol("#arrayop")
BSImpl.Const() => Symbol("#const")

Comment on lines +488 to +489
sort!(sorted_buffers; by = b -> canonical_sort_key(first(b[2])),
alg = Base.Sort.DEFAULT_STABLE)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Runic] reported by reviewdog 🐶

Suggested change
sort!(sorted_buffers; by = b -> canonical_sort_key(first(b[2])),
alg = Base.Sort.DEFAULT_STABLE)
sort!(
sorted_buffers; by = b -> canonical_sort_key(first(b[2])),
alg = Base.Sort.DEFAULT_STABLE
)

all_cacheexprs = reduce(vcat, values(cacheexprs); init = SymbolicT[])
sorted_cache_types = sort!(collect(keys(cachevars)); by = string)
all_cacheexprs = reduce(
vcat, (cacheexprs[T] for T in sorted_cache_types); init = SymbolicT[])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Runic] reported by reviewdog 🐶

Suggested change
vcat, (cacheexprs[T] for T in sorted_cache_types); init = SymbolicT[])
vcat, (cacheexprs[T] for T in sorted_cache_types); init = SymbolicT[]
)

Comment on lines +61 to +62
@variables x1(t)=0.1 x2(t)=0.2 x3(t)=0.3 x4(t)=0.4
@parameters a=1.0 b=2.0 c=3.0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Runic] reported by reviewdog 🐶

Suggested change
@variables x1(t)=0.1 x2(t)=0.2 x3(t)=0.3 x4(t)=0.4
@parameters a=1.0 b=2.0 c=3.0
@variables x1(t) = 0.1 x2(t) = 0.2 x3(t) = 0.3 x4(t) = 0.4
@parameters a = 1.0 b = 2.0 c = 3.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant