Skip to content

perf: update planning list variable elements by a cascade instead of the graph - #2548

Draft
fodzal wants to merge 3 commits into
TimefoldAI:mainfrom
fodzal:perf/list-element-cascade
Draft

perf: update planning list variable elements by a cascade instead of the graph#2548
fodzal wants to merge 3 commits into
TimefoldAI:mainfrom
fodzal:perf/list-element-cascade

Conversation

@fodzal

@fodzal fodzal commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Why

Follow-up to the performance discussion in #2357 and to #2506.

The arbitrary variable reference graph is often much slower than what hand-written
variable listeners achieve for the same model: it maintains a node per entity/variable
pair, dynamic edges and an incremental topological order on every move. #1659 closed that
gap for simple structures (EMPTY, NO_DYNAMIC_EDGES, SINGLE_DIRECTIONAL_PARENT), but
a model that falls outside them drops back to the arbitrary graph, and on large problems
that cost dominates solving. In planning list variable models this hits particularly
hard: the elements are typically counted in thousands while the other entities are
counted in hundreds, so the per-element part of the graph is usually the updater's
bottleneck.

The motivating use case is multi-trip vehicle routing that cannot be modeled by simulating
a depot return when capacity is exceeded, for example:

  • an ambulance that may return to the depot at any time to change crews;
  • two vehicles that must ride the same visits for part of the day (e.g. training) —
    unlike synchronized visits, where a single visit requires two technicians.

Such problems are modeled with one planning entity per trip: vehicle A becomes trips A1
and A2, and A2's first travel leg starts from A1's last visit. The trip's start must
depend on the previous trip's end, and the trip's end on its own visits — a list element
source (#2506) plus a cross-entity reference, which today classifies the whole model as
arbitrary:

@PlanningEntity
public class Vehicle {

    Vehicle previousTrip; // The same physical vehicle's previous trip, or null; a fact.

    @PlanningListVariable
    List<Visit> visits;

    @ShadowVariable(supplierName = "startTimeSupplier")
    LocalDateTime startTime;

    @ShadowSources("previousTrip.endTime")
    LocalDateTime startTimeSupplier() {
        return previousTrip == null ? SHIFT_START : previousTrip.getEndTime();
    }

    @ShadowVariable(supplierName = "endTimeSupplier")
    LocalDateTime endTime;

    @ShadowSources("visits[].endServiceTime")
    LocalDateTime endTimeSupplier() {
        return visits.isEmpty() ? startTime : visits.getLast().getEndServiceTime();
    }
}

@PlanningEntity
public class Visit {

    Duration serviceDuration;

    @InverseRelationShadowVariable(sourceVariableName = "visits")
    Vehicle vehicle;

    @PreviousElementShadowVariable(sourceVariableName = "visits")
    Visit previous;

    @ShadowVariable(supplierName = "endServiceTimeSupplier")
    LocalDateTime endServiceTime;

    @ShadowSources({ "previous.endServiceTime", "vehicle.startTime" })
    LocalDateTime endServiceTimeSupplier() {
        if (vehicle == null) {
            return null;
        }
        var departureTime = previous == null ? vehicle.getStartTime() : previous.getEndServiceTime();
        return departureTime.plus(travelTime()).plus(serviceDuration);
    }
}

In such a model the visits vastly outnumber the vehicles, so nearly all of the arbitrary
graph's nodes, edges and topological-order maintenance is spent on them — while their
dependencies are precisely the ones that need no graph: a visit only reads its chain and
its own vehicle.

What

A narrower fix would extend the SINGLE_DIRECTIONAL_PARENT detection to this shape and
add another specialized updater. This PR instead splits the model so that each half is
handled by machinery that is already good at it:

  • the planning list variable's elements are excluded from the variable reference
    graph and updated by a cascade that walks each dirty entity's list in chain order,
    the way a variable listener would;
  • the remaining entities keep the existing graph machinery, unchanged and with its
    full generality: any structure, dynamic edges, groups, several entity classes,
    dependency loops handled as inconsistency. Without per-element edges the graph is small
    and often has no dynamic edges left; the example above gets a fixed graph over the
    vehicles.

The detection (GraphStructure.determineListElementCascade) only inspects the element
class and the references toward it, and accepts the decomposition when:

  • the elements' declarative sources stay chain-local: previous/next (a single
    direction across the model), their own members, or — through their inverse — a
    declarative variable of their entity that does not itself depend on the list
    (pre-chain);
  • the other classes only reach the elements through visits[].x on their own list
    variable;
  • there are no alignment keys on the element or list entity class, and the element class
    is distinct from the list entity class.

Anything else falls through to the current classification, so unsupported models behave
exactly as they do today.

How

  • The list entity's declarative variables are classified as pre-chain (readable by
    the elements, computed before walking a chain) or post-chain (depending on
    visits[].x directly or transitively, computed after). Cross-entity references do not
    propagate post-chain status: "B starts where A ends" keeps startTime pre-chain of B.

  • ListElementCascadeVariableReferenceGraph wraps the inner graph and receives all
    events. Element source changes accumulate dirty elements; list change events mark the
    changed range dirty and the owner's post-chain variables changed, even for an empty
    range, since a removal changes the aggregate; a ChangedVariableNotifier wrapper
    observes pre-chain changes during updates and flags the owner for a whole-chain walk.

  • updateChanged() alternates the cascade and the inner graph until neither has work
    left. A walk starts at the earliest dirty element and may terminate early after the
    last dirty one once values stop changing. An owner whose pre-chain variables just
    changed is deferred while other chains still need walking, so every supplier is
    computed once per update; the number of passes is bounded by the depth of inter-entity
    dependencies that traverse the lists.

    On the example above, an update that touches A's route settles in this order:

    A.startTime --> [a1 -> a2 -> ... -> aN] --> A.endTime --> B.startTime --> [b1 -> ... -> bM] --> B.endTime --> ...
     pre-chain           cascade walk           post-chain     pre-chain          cascade walk       post-chain
    (inner graph)                              (inner graph) (inner graph)                          (inner graph)
    
  • When the inner graph marks an owner inconsistent (a dependency loop the solver may
    break later), the cascade marks its elements inconsistent instead of computing them,
    and recomputes them when consistency returns.

  • The factory changes are mechanical: the existing graph builders now take the list of
    descriptors to cover (defaulting to all of them), and under a cascade the per-element
    locator and edges are skipped while the "list changed -> target dirty" processor is
    kept.

Notes:

  • The chain walk itself is the same idea as @CascadingUpdateShadowVariable's listener:
    start at the first changed element, stop early once values settle. What differs is what
    triggers and surrounds it. A cascading update shadow variable only reacts to its
    element's own list position changes, so a dependency on another entity's variables is
    neither declared nor retriggered — the multi-trip model above is not expressible with
    it. Here the walk is embedded in the declarative dependency graph: a pre-chain change
    re-walks the chain, post-chain variables and downstream entities are ordered by the
    graph, and dependency loops are still detected as inconsistency.
  • A model where only the elements have declarative variables keeps using
    SINGLE_DIRECTIONAL_PARENT. With an EMPTY inner graph the wrapper is behaviorally
    equivalent to it, so a later unification is possible, but out of scope here.
  • The detection relies on the model having a single planning list variable, which
    SolutionDescriptor currently guarantees; a guard documents the assumption and falls
    back to the current behavior if that restriction is ever lifted.

Testing

  • Detection in GraphStructureTest: accepted shapes (fact-collection chain, plain-fact
    chain, next direction, a three-class model whose depot keeps its own graph) and a
    rejected shape (a fact collection of another vehicle's visits).
  • Session-level tests on dedicated domains: propagation across chained vehicles, a swap
    creating non-contiguous dirty elements on the same chain, and a pre-chain change
    reaching an element that reads it directly while its predecessors are unchanged.
  • Every domain solves with FULL_ASSERT, with constraints reading the shadow variables;
    after each scenario and each solve, the incrementally maintained values are compared
    against a from-scratch SolutionManager.updateShadowVariables recompute, which uses
    the arbitrary graph.
  • ListElementCascadeVariableReferenceGraphTest counts supplier calls to pin the
    single-computation guarantee, and documents the one residual extra evaluation (the
    post-chain variable of an entity whose chain is walked again).
  • A dependency loop between vehicles still hits the existing "fixed dependency loops"
    fail-fast.
  • Full core test suite passes.

Local search throughput at the motivating scale (579 vehicle trips, 7,000 visits, same
solver configuration and machine within each row):

Model main (arbitrary graph) This PR (cascade) Speedup
Production multi-trip VRPTW (real constraints) 39 moves/s 26,765 moves/s ~686x
Synthetic fact-chain test domain (one trivial constraint) 69 moves/s ~14,100 moves/s ~200x

The synthetic row reproduces the gap independently of the business model, on this PR's
own fact-chain test domain, from an initialized solution and with graph construction
excluded from the measurement. The unit test domains are far too small to exhibit this
(the gap grows with the element count), which is why they focus on correctness instead.

fodzal and others added 3 commits July 28, 2026 13:13
…the graph

When a model's list elements only read their chain and, through their
inverse, pre-chain declarative variables of their own list entity, and
the other entities only reach the elements through the list variable
itself, the elements are excluded from the variable reference graph.
The graph then only covers the remaining entities and is built by the
existing machinery, unchanged and with its full generality; without
per-element edges it often has no dynamic edges at all and stays fixed.

The elements are updated by a cascade that walks each dirty entity's
list from the earliest dirty element, marking the entity's post-chain
variables changed when an element changed. Updates alternate the
cascade and the graph until neither has work left; an entity whose
pre-chain variables just changed is deferred while other entities still
need walking, so every element settles in a single computation.

This removes the per-element graph nodes, edges and topological order
maintenance, which dominate the graph cost in vehicle routing models
where elements far outnumber the other entities.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@triceo

triceo commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

@fodzal Thanks for the PR! Looks interesting.

FYI This will take us a while to review. We first need to finish some other work in this area, which will likely cause conflicts for you. Once you resolve them, we should be able to review.

Please bear with us for a while; I'll let you know when we're ready to move forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants