`Vector{Int}` indexing of `NamedArrayPartition` broken for v4+

[jlchan]

Describe the example

Indexing into a NamedArrayPartition with Vector{Int} does not work after the v4 update. This seems to be due to 05faa73

Minimal Reproducible Example 👇

using RecursiveArrayTools
x = NamedArrayPartition(a = ones(2), b = 2 * ones(3))
x[1:2] # fails with v4+

However, the above works for x = NamedArrayPartition(a = ones(1), b = 2 * ones(1)).

Error & Stacktrace ⚠️

julia> x[1:2] # fails with v4+
ERROR: MethodError: no method matching NamedArrayPartition(::Vector{Float64}, ::@NamedTuple{a::Int64, b::Int64})
The type `NamedArrayPartition` exists, but no method is defined for this combination of argument types when trying to construct it.

Closest candidates are:
  NamedArrayPartition(::A, ::NT) where {T, A<:(ArrayPartition{T}), NT<:NamedTuple}
   @ RecursiveArrayTools ~/.julia/packages/RecursiveArrayTools/GM23x/src/named_array_partition.jl:10
  NamedArrayPartition(::F, ::Any, ::Any) where F<:Function
   @ RecursiveArrayTools ~/.julia/packages/RecursiveArrayTools/GM23x/src/named_array_partition.jl:144
  NamedArrayPartition(; kwargs...)
   @ RecursiveArrayTools ~/.julia/packages/RecursiveArrayTools/GM23x/src/named_array_partition.jl:13
  ...

Stacktrace:
 [1] similar(A::NamedArrayPartition{Float64, ArrayPartition{…}, @NamedTuple{…}}, ::Type{Float64}, dims::Tuple{Int64})
   @ RecursiveArrayTools ~/.julia/packages/RecursiveArrayTools/GM23x/src/named_array_partition.jl:53
 [2] similar
   @ ./abstractarray.jl:822 [inlined]
 [3] _unsafe_getindex(::IndexCartesian, A::NamedArrayPartition{…}, I::UnitRange{…})
   @ Base ./multidimensional.jl:968
 [4] _getindex
   @ ./multidimensional.jl:956 [inlined]
 [5] getindex(A::NamedArrayPartition{Float64, ArrayPartition{…}, @NamedTuple{…}}, I::UnitRange{Int64})
   @ Base ./abstractarray.jl:1342
 [6] top-level scope
   @ REPL[4]:1
Some type information was truncated. Use `show(err)` to see complete types.

Not Working Environment (please complete the following information):

• Output of using Pkg; Pkg.status()

julia> using Pkg; Pkg.status()
Status `/private/var/folders/rq/3s_b2f257xs0blrhx7tx1xkc0000gq/T/jl_8EQxJI/Project.toml`
  [731186ca] RecursiveArrayTools v4.2.0

• Output of using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)

julia> using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)
Status `/private/var/folders/rq/3s_b2f257xs0blrhx7tx1xkc0000gq/T/jl_8EQxJI/Manifest.toml`
  [7d9f7c33] Accessors v0.1.44
  [79e6a3ab] Adapt v4.5.2
  [4fba245c] ArrayInterface v7.24.0
  [a33af91c] CompositionsBase v0.1.2
  [187b0558] ConstructionBase v1.6.0
  [ffbed154] DocStringExtensions v0.9.5
  [e2ba6199] ExprTools v0.1.10
  [46192b85] GPUArraysCore v0.2.0
  [3587e190] InverseFunctions v0.1.17
  [1914dd2f] MacroTools v0.5.16
  [aea7be01] PrecompileTools v1.3.3
  [21216c6a] Preferences v1.5.2
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v4.2.0
  [ae029012] Requires v1.3.1
  [7e49a35a] RuntimeGeneratedFunctions v0.5.18
  [1e83bf80] StaticArraysCore v1.4.4
  [2efcf032] SymbolicIndexingInterface v0.3.46
  [56f22d72] Artifacts v1.11.0
  [ade2ca70] Dates v1.11.0
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.12.0
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0
  [fa267f1f] TOML v1.0.3
  [cf7118a7] UUIDs v1.11.0
  [4ec0a83e] Unicode v1.11.0
  [e66e0078] CompilerSupportLibraries_jll v1.3.0+1
  [4536629a] OpenBLAS_jll v0.3.29+0
  [8e850b90] libblastrampoline_jll v5.15.0+0

• Output of versioninfo()

julia> versioninfo()
Julia Version 1.12.2
Commit ca9b6662be4 (2025-11-20 16:25 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 16 × Apple M4 Max
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, apple-m4)
  GC: Built with stock GC
Threads: 12 default, 1 interactive, 12 GC (on 12 virtual cores)
Environment:
  JULIA_EDITOR = code
  JULIA_VSCODE_REPL = 1
  JULIA_NUM_THREADS = 12

Working Environment (please complete the following information):

• Output of using Pkg; Pkg.status()

julia> using Pkg; Pkg.status()
Status `/private/var/folders/rq/3s_b2f257xs0blrhx7tx1xkc0000gq/T/jl_EtaLDd/Project.toml`
⌃ [731186ca] RecursiveArrayTools v3.8.1
Info Packages marked with ⌃ have new versions available and may be upgradable.

• Output of using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)

julia> using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)
Status `/private/var/folders/rq/3s_b2f257xs0blrhx7tx1xkc0000gq/T/jl_EtaLDd/Manifest.toml`
  [7d9f7c33] Accessors v0.1.44
  [79e6a3ab] Adapt v4.5.2
  [4fba245c] ArrayInterface v7.24.0
  [a33af91c] CompositionsBase v0.1.2
  [187b0558] ConstructionBase v1.6.0
  [9a962f9c] DataAPI v1.16.0
  [e2d170a0] DataValueInterfaces v1.0.0
  [ffbed154] DocStringExtensions v0.9.5
  [e2ba6199] ExprTools v0.1.10
⌅ [46192b85] GPUArraysCore v0.1.6
  [3587e190] InverseFunctions v0.1.17
  [82899510] IteratorInterfaceExtensions v1.0.0
  [1914dd2f] MacroTools v0.5.16
  [bac558e1] OrderedCollections v1.8.1
  [aea7be01] PrecompileTools v1.3.3
  [21216c6a] Preferences v1.5.2
  [3cdcf5f2] RecipesBase v1.3.4
⌃ [731186ca] RecursiveArrayTools v3.8.1
  [ae029012] Requires v1.3.1
  [7e49a35a] RuntimeGeneratedFunctions v0.5.18
  [1e83bf80] StaticArraysCore v1.4.4
  [10745b16] Statistics v1.11.1
  [2efcf032] SymbolicIndexingInterface v0.3.46
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.12.1
  [56f22d72] Artifacts v1.11.0
  [ade2ca70] Dates v1.11.0
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.12.0
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0
  [2f01184e] SparseArrays v1.12.0
  [fa267f1f] TOML v1.0.3
  [cf7118a7] UUIDs v1.11.0
  [4ec0a83e] Unicode v1.11.0
  [e66e0078] CompilerSupportLibraries_jll v1.3.0+1
  [4536629a] OpenBLAS_jll v0.3.29+0
  [bea87d4a] SuiteSparse_jll v7.8.3+2
  [8e850b90] libblastrampoline_jll v5.15.0+0
Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m`

• Output of versioninfo()

julia> versioninfo()
Julia Version 1.12.2
Commit ca9b6662be4 (2025-11-20 16:25 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 16 × Apple M4 Max
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, apple-m4)
  GC: Built with stock GC
Threads: 12 default, 1 interactive, 12 GC (on 12 virtual cores)
Environment:
  JULIA_EDITOR = code
  JULIA_VSCODE_REPL = 1
  JULIA_NUM_THREADS = 12

[ChrisRackauckas-Claude]

Investigated this and posted a fix in #584. Sharing the diagnosis here because the type-stability story is worth a wider eye.

What's actually broken

The invalidation cleanup in 05faa73 was correct — narrowing getindex(::NamedArrayPartition, ...) from a catch-all args... to just Int is what gets us to 0 invalidation trees. But it exposed a pre-existing latent bug in the similar overloads.

When dims != size(A), the underlying similar(::ArrayPartition, T, dims) already degrades to a plain Vector — a partition layout no longer fits the requested shape (src/array_partition.jl:72-78). The NamedArrayPartition overloads then try to wrap that Vector via the inner constructor

NamedArrayPartition{T, A<:ArrayPartition{T}, NT}(::A, ::NT)

which raises the MethodError reported in this issue. The pre-05faa730 catch-all args... getindex bypassed similar entirely (going straight to getindex(ArrayPartition(x), ...) which has its own _unsafe_getindex shortcut), so the bug stayed hidden.

The fix in #584

Mirror ArrayPartition's own degradation in the two similar overloads — wrap when the inner result is an ArrayPartition, otherwise propagate the fallback array as-is:

function Base.similar(A::NamedArrayPartition, ::Type{T}, dims::NTuple{N, Int}) where {T, N}
    inner = similar(getfield(A, :array_partition), T, dims)
    inner isa ArrayPartition || return inner
    return NamedArrayPartition(inner, getfield(A, :names_to_indices))
end

x[1:2] and x[[1, 4]] work again (returning Vector), x[1:end] still preserves the NamedArrayPartition type, no method shadows a wider signature than today, and SnoopCompile.invalidation_trees(@snoop_invalidations using RecursiveArrayTools) still reports 0 trees.

The type-stability nuance

The fix infers to a small union, and so does any path that goes through it:

similar(x, Float64, (2,)) :: Union{NamedArrayPartition{...}, Vector{Float64}}
x[1:2]    :: Union{NamedArrayPartition{...}, Vector{Float64}}
x[[1, 4]] :: Union{NamedArrayPartition{...}, Vector{Float64}}
x[1:end]  :: Union{NamedArrayPartition{...}, Vector{Float64}}
x[1]      :: Float64                                       (stable)

This is inherited from upstream ArrayPartition.similar — that one already infers to Union{ArrayPartition{...}, Vector{Float64}} for the same reason (the dims == size(A) branch is a runtime check). The reason ap[1:2] infers to a clean Vector{Float64} while x[1:2] doesn't is that ArrayPartition has an _unsafe_getindex shortcut at array_partition.jl:317 that always allocates a Vector and never routes through similar. NAP has no equivalent shortcut, so it goes through the generic AbstractArray path that hits similar.

Three options, mutually exclusive in pairs:

1. Current Fix NamedArrayPartition Vector{Int} / UnitRange indexing (#583) #584 — small union. x[1:end] :: NAP at runtime, x[1:2] :: Vector at runtime; both inferred as the union. Optimizer handles this via union-splitting (one runtime isa branch, not a boxed Any). Matches the post-05faa730 test typeof(x .+ x[1:end]) <: NamedArrayPartition.

2. Always-Vector slicing. Add an _unsafe_getindex shortcut for NAP mirroring ArrayPartition's. x[I] :: Vector{Float64} for every non-scalar I. Fully type-stable, no union. Cost: x[1:end] :: Vector again, regressing the post-05faa730 test back to v3 behavior.

3. Always-NAP slicing. Reject the dims-mismatch case in similar. Worse than today — re-breaks the issue MWE.

The semantic argument for (1): when the slice length equals length(x) the partition layout still applies and the names are still meaningful, so returning NAP is sensible; for partial slices it isn't, so returning Vector is sensible. (1) tracks ArrayPartition's own (un)stability rather than introducing new instability — and (2) would be a behavior regression for x[1:end] users on top of the bug fix.

If the maintainer preference is full type-stability over preserving x[1:end] :: NAP, happy to flip #584 to option (2) — the _unsafe_getindex shortcut is small. Otherwise (1) is what's in the PR.

[ChrisRackauckas]

@jlchan it seems like it would be good to just make this type stable, but that would make it always return a vector? I'd be curious what your preference is here.

[JoshuaLampert]

In the test of StartUpDG.jl, which fails for RecursiveArrayTools.jl v4 (see jlchan/StartUpDG.jl#219 (comment)) the indexing of the NamedArrayPartition already returned a vector under RecursiveArrayTools.jl v3. So AFAICT, always returning a vector should be fine if @jlchan does not object.

[jlchan]

Thank you @joshualambert - yes, that is fine with me. Thanks for the quick fix, @ChrisRackauckas!

…

On Tue, Apr 28, 2026 at 8:24 AM Joshua Lampert ***@***.***> wrote: *JoshuaLampert* left a comment (SciML/RecursiveArrayTools.jl#583) <#583 (comment)> In the test of StartUpDG.jl, which fails for RecursiveArrayTools.jl v4 (see jlchan/StartUpDG.jl#219 (comment) <jlchan/StartUpDG.jl#219 (comment)>) the indexing of the NamedArrayPartition already returned a vector under RecursiveArrayTools.jl v3. So AFAICT, always returning a vector should be fine if @jlchan <https://github.com/jlchan> does not object. — Reply to this email directly, view it on GitHub <#583 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI2HUDY2ZPR4PE54MJHYUT4YCWIVAVCNFSM6AAAAACYIAYWEOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DGMZVGYZTEMZVGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>