Invert DataService delivery to pull-on-frame

[SimonHeybrock]

Closes the main architectural item from #1044 (see the evaluation comments there for the full analysis). Stacked on #1043; targets pause-inactive-delivery.

What

Inverts the DataService hot path from push-per-delta to pull-on-frame. Ingestion now only marks keys updated and notifies subscribers keys-only; the plot orchestrator marks affected layers dirty per grid, and flush_frames pulls each dirty viewed layer's input through its extractors from current buffer state, rebuilds, and commits the grid frame. The 0→1 viewer transition still rebuilds synchronously on the polling thread, as introduced in #1043.

Why

Extraction (dominated by window aggregation over buffered history for image-sized data) previously ran per subscriber per ingestion batch, on the notification path, and its results traveled through per-layer stashes — which is what the viewer gate in #1043 had to work around. Pulling at flush time makes "no viewers → no work" the default instead of a gated special case, and removes the machinery the gate required: the DataServiceSubscriber.active gate, the LayerStateMachine stash/ComputeTask plumbing, and the documented delivery races (out-of-order data delivery, in-flight delivery racing a pause) are all gone. Net −100 lines.

Because a pull always reads current buffer state, coalescing or deferring flushes is lossless — a future min-frame-interval throttle in flush_frames (to cap compute at display cadence under ingestion load) becomes a one-line policy choice rather than a correctness question. Deliberately not included here; cadence today is identical to before.

The frame-gated session flush from ADR 0005 is unchanged: per-grid commits, generation-gated session flushes, and the threading constraint all hold. The ADR gains an amendment describing the input-side change (frame = buffer state at flush, rather than the drained burst's payload).

Also fixes static overlays never rendering when created without a viewer token, a gap the gate in #1043 introduced: they are now built unconditionally at creation, since they have no data subscription to pull from later and the build is a one-off.

Test plan

Verified end-to-end against the fake backend via scripts/drive_dashboard.py: detector images fill on the visible tab and keep updating (peak moves between frames), tab-away/tab-back shows fresh data promptly (activation refresh), and rapid tab flipping does not wedge delivery.

Needs a check against a real timestamped backend:

• Live stream: plots update at the usual cadence, freshness pill sits near pipeline lag.
• Multiple visible plots repaint together (no stagger).
• Tab switch shows latest data promptly; hidden tabs cost no compute.

🤖 Generated with Claude Code

[SimonHeybrock]

Note from the perf follow-up (#1055, detailed proposal in #1046 comment), no change requested here: once delivery is pull-on-frame, flush_frames' per-grid commit becomes the single point where "a new displayable frame exists" is known — which makes it the natural site to wake sessions with a data-free doc.add_next_tick_callback tick, instead of sessions detecting the generation advance by 100 ms polling. Feasibility is verified (panel 1.9.3: woken tick runs in correct session context, hold/freeze work; see #1046). Two composition points worth keeping in mind while this branch evolves: the min-frame-interval throttle anticipated in the description and the wake compose cleanly (throttle caps compute cadence, wake removes display-side polling), and commit(grid_id) staying the last step of a grid's flush is what makes a wake-on-commit never observe a half-built frame — already the case here, just worth preserving.

---

Generated by Claude Code