feat(scale): chunked durable sync apply and large-library hardening (#1380)

* feat(library): chunk sync_apply_unit into durable per-chunk commits

* feat(scale): always-on sync time estimate and paged metadata cache init

* fix(artwork): apply shortcut icons live via SteamClient instead of clobbered vdf write

* docs: chunked apply adr, assigned-appid model corrections, sync estimate guide

* fix(library): guard inter-chunk cancel window and keyless rebind chunk routing

* fix(ui): stack library stats on separate lines in the qam

* docs: soften sync sleep guidance — suspend pauses and resumes safely

* perf(sync): serve covers from backend grid files instead of pushing base64 through cef

* fix(ui): keep fine sync progress and eta across qam remounts

* perf(sync): poll app overview after addshortcut and skip no-op launch-options confirm

* feat(ui): narrate fetch and cover prep and interpolate coarse sync progress

* fix(ui): spinner during silent sync phases and clean fine-line wrapping

* fix(library): full fetch when a platform has rows but no bound shortcuts

* fix(ui): wrap dynamic qam text instead of clipping

* feat(library): stamp completed platform units so partial runs skip them

* perf(library): fetch rom lists at page size 500 without unused server aggregations

* feat(ui): live sync eta countdown and last-attempt display

* fix(ui): keep applying stage across qam remount and reconcile aggregate-count docs

* fix(library): force full sync clears all run history so last sync isn't a stale cancel

* fix(ui): tear down sync ui on sync_complete not just the trailing done frame

* fix(artwork): nudge steam's asset cache so fresh covers appear in-session

* fix(library): clear platform stamps at apply start and on local removals

* fix(library): make the completion stamp the sole incremental-skip authority

* feat(library): record interrupted sync runs distinctly from user cancels

A heartbeat-timeout death (steamwebhelper crash/reload) and a backend
restart mid-run now terminate the SyncRun as 'interrupted' instead of
'cancelled'/'errored' — 'cancelled' is reserved for the user's own
Cancel. Migration 012 widens the sync_runs status CHECK via an in-place
table rebuild; get_latest_terminal includes the new status so it
surfaces as the QAM's last-attempt hint; the terminal frame's message
reads 'Sync interrupted: …' for the timeout case. A parametrized test
binds the domain status literal to the SQL CHECK list.

* fix(ui): hold live eta across fetch gaps and isolate sync-progress listeners

The live countdown's readiness gate re-arms ~5s after every inter-unit
fetch gap and the run tail's small units never re-arm it, so the display
kept reverting to the static 'up to ~X' seed. The measured ETA is now
held as a wall-clock deadline that keeps counting down through gaps and
is re-anchored by each fresh measurement.

The sync-progress store now notifies each listener inside its own
try/catch — a throwing subscriber could previously starve later
listeners and abort the emitting per-item apply loop, skipping that
game's shortcut creation — and MainPage's subscriber updates its local
mirror first, with all derived work fenced so it can never break the
re-render chain (the on-device QAM freeze class).

* feat(ui): resume sync label, preview-delta eta seed, force-full visibility

The sync button reads 'Resume Sync' when the last attempt ended
cancelled or interrupted — the stamp/chunk model makes every re-sync an
effective resume, so the button says what the run will do. Applying
from a preview seeds the ETA from the real delta (new + changed counts)
instead of the total_roms upper bound, so a small resume no longer
reads 'up to ~46 min'; the sync_plan bound now only seeds when no
tighter estimate is present. Force Full Sync shows whenever any
terminal run is recorded — previously it vanished exactly in the
resume situation, when a forced fresh start matters most — and flips
the main button back to 'Sync Library' via the existing stats refresh.

* fix(ui): price sync estimates at walk cost so resumes never undershoot

The preview 'Estimated time' row and the apply seed priced only the
delta (new + changed), but the apply re-walks every non-skipped item —
an un-stamped platform's unchanged ROMs still get cheap update touches.
A resume showed '~2 min' for 153 added while the walk took ~10 minutes.
Both surfaces now share one walk-cost helper (new at create rate, all
other visible items at the update rate): an honest upper bound that
stamp-skipped platforms only push downward, with the live countdown
correcting within seconds of applying.

* fix(artwork): stamp rt_custom_image_mtime so fresh covers appear in-session

ReportLibraryAssetCacheMiss is a no-op for non-erroring default tiles —
2,155 nudged appIds produced zero visible covers on-device. Steam
caches a fresh shortcut's asset resolution outside the JS context and
keys the tile URL on the overview's rt_custom_image_mtime (the field a
client restart normally stamps). After sync_complete the plugin now
stamps that field on each created appId's overview instead: the tile
picks its grid cover up on the next render, no restart, no base64
through the bridge. Verified on-device against a gray tile whose field
was unset.

* fix(ui): gate the resume label on actual bound shortcuts

An interrupted attempt followed by a mass removal left the button
reading 'Resume Sync' with nothing on disk to resume — the next run is
a full fresh import. The label now additionally requires a non-zero
bound-shortcut count (stats.roms, our own registry — foreign or manual
shortcuts never appear in it), so it flips back to 'Sync Library' the
moment no committed partial progress exists.

* fix(ui): recalibrate sync estimates and segment the live rate window

The static cost constants predated the overview-poll optimization —
measured post-poll rates are 0.30-0.36s per create and ~0.15s per
update, so 0.85/0.35 produced ceilings 3x over reality. Recalibrated to
0.45/0.20 with a flat 90s fetch allowance so small syncs price their
fetch-dominated phases honestly.

The live estimator no longer measures across inter-unit gaps: a >10s
silence between applying samples starts a fresh segment instead of
pairing samples across the gap, whose tiny item delta over a long span
briefly read as an absurd rate (the transient ~50min countdown spike);
the sticky deadline carries the display through each segment's re-arm.

* chore(db): renumber migration and adr around the #1369 collision

* feat(artwork): stamp covers per chunk so they appear during the run

Each chunk's freshly-created shortcuts get their rt_custom_image_mtime
stamped right after the chunk's ack returns — the backend has committed
the chunk and written the grid files by then, so the tile URL resolves
(stamping at create time would hit a missing file and risk the client's
negative cache). Covers now fill in progressively instead of all at
sync_complete, and a mid-run pause or interrupt leaves everything
committed so far visible. The end-of-run sweep stays as the safety net.

* refactor(library): extract the active-unit teardown into a box verb

The five-field in-flight-chunk reset was copy-pasted at four sites —
this PR itself had to add active_chunk_index to every one of them.
clear_active_unit() on the state box is now the single teardown; the
heartbeat-timeout branch deliberately keeps not calling it (staging
survives for the late ack).

* refactor(ui): let the estimator own the displayed eta

The sticky deadline lived as MainPage state while its rationale and
inputs lived in syncEta — the displayed number was smeared across two
modules. syncEta now holds the deadline on its run state and exposes
displayedEtaSeconds(now); MainPage renders it without owning any ETA
state. The two near-identical duration formatters collapse onto one
helper with both public names and output strings preserved.

* docs: tell the final story of chunked sync, estimates, and progressive covers

User guide covers the resume flow ('(interrupted)' last-attempt line,
the Resume Sync relabel, Force Full Sync), estimates as deliberate
ceilings replaced by the live countdown, and covers filling in during
the run with the small known residue. Architecture pages document the
per-chunk mtime stamp as the primary cover path and the estimate/live-
ETA design; ADR-0023 records the measured renderer session budget the
chunk model defends against.

* fix(artwork): re-sweep cover stamps after steam's catalog association settles

Steam name-matches some fresh shortcuts against its own catalog and
re-materializes their app overview seconds after creation, wiping the
JS-set rt_custom_image_mtime — census over a 368-create run: exactly 4
stamps vanished within ~30s of sync_complete and the set stayed stable
for 8 minutes. One delayed re-sweep (90s) over the run's created appIds
restores them; a fresh sync supersedes a pending re-sweep and unload
clears the timer.

* fix(artwork): batch cover stamps and heal wiped ones by polling

Stamping a chunk's covers mutates one MobX observable per shortcut;
a synchronous burst of up to a few hundred writes flushed Steam's
reactions hard enough to visibly flicker the QAM at platform
transitions. All stamp paths now share one helper that writes in
slices of 25 with a yield between slices.

The blind 90s re-sweep becomes a verify-and-heal poll: every 15s the
created appIds' overviews are read back and only wiped stamps are
re-written, exiting after two consecutive clean rounds (or a hard cap).
Steam's catalog-name association wipes ~1% of stamps once, early — the
poll heals them within seconds and still catches a late wipe.

* fix(artwork): stamp only acked creates and harden the heal poll

The end-of-run sweep and heal poll now operate on the acked-created
set — a cancelled run's uncommitted in-flight chunk was previously
stamped toward a grid file that was never written. Observable writes
go through the shared mobx state transaction (extracted from the
metadata patches) so they survive strict-actions builds; a throwing
heal round logs and keeps polling instead of dying; a zero-created
sync no longer aborts the previous run's heal window; and the icon
toast counts only icons that actually applied.

* fix(ui): keep the eta now-read out of render and drop dead narrowing

ESLint: displayedEtaSeconds(Date.now()) in the render body violates
render purity — the now-read moves into the store subscriber, which
every progress frame drives anyway, and render reads the mirrored
value. The resume-label guard's optional chain and nullish fallback
were dead after the preceding narrowing.

* build: raise the bundle size budget to 700 kB

The chunked-apply feature set adds ~40 kB of first-party frontend code
(chunk protocol, estimate/live-eta chain, cover stamping and heal,
resume UX, paged metadata init) and lands at 686 kB against the 650 kB
budget. 700 kB keeps the guard tight against accidental bloat while
fitting the deliberate growth.

* refactor: flatten the new code's sonar hotspots

Decomposes the chunk-apply loop, the collection page fetch, and the
artwork application into verb-named helpers under the cognitive
complexity gate; restructures the paged metadata-init loop so its
condition and incrementer agree; extracts the nested display ternaries;
tightens two test assertions. Behavior-preserving throughout — the
existing suites pass unchanged.

Author: danielcopper

.size-limit.json

@@ -3,6 +3,6 @@
     "name": "dist/index.js (raw)",
     "path": "dist/index.js",
     "brotli": false,
-    "limit": "650 KB"
+    "limit": "700 KB"
   }
 ]

CLAUDE.md

@@ -30,28 +30,39 @@ silent omission. The CI check enforces this.
 - **Frontend API**: `@decky/ui` + `@decky/api` (NOT deprecated `decky-frontend-lib`). Use `callable()` (NOT
   `ServerAPI.callPluginMethod()`).
 - **RomM API quirks**: Filter param is `platform_ids` (plural). Cover URLs have unencoded spaces (must URL-encode).
-  Paginated: `{"items": [...], "total": N}`.
-- **AddShortcut timing**: Must wait 300-500ms after `AddShortcut()` before setting properties. Use 50ms delay between
-  operations.
+  Paginated: `{"items": [...], "total": N}`. The `/api/roms` list calls page at `limit=500` (`lib/romm_paging.py`
+  `LIST_PAGE_SIZE`; the endpoint bounds `limit` at `le=10_000`, unchanged across 4.9.x) and append
+  `&with_char_index=false&with_filter_values=false` to skip the unused char-index + filter-values aggregations the
+  server computes on every list request — so a typical platform is one request, not dozens.
+- **AddShortcut timing**: After `AddShortcut()`, wait for the new app's overview before setting properties — poll
+  `appStore.GetAppOverviewByAppID(appId)` (`waitForAppOverview`, ~100ms cadence, 1000ms fallback that proceeds anyway)
+  instead of a blind fixed delay. On create, an empty `launch_options` (uninstalled ROM) skips both
+  `SetAppLaunchOptions` and the confirm poll — a fresh shortcut's launch options are already empty, and skipping avoids
+  the confirm poll's fat `AppDetails` cache hit. Use 50ms delay between operations in the apply loop.
 - **Large payloads**: Never send bulk base64 data through `decky.emit()` — WebSocket bridge has size limits. Use
-  per-item callables instead.
+  per-item callables instead. Bulk lists are chunked too: the library apply emits shortcuts ~200 at a time
+  (`_APPLY_CHUNK_SIZE`, ADR-0023) and the metadata cache is loaded page-by-page (`get_metadata_cache_page`), so a large
+  library never pushes a multi-MB frame in one response.
 - **User-Agent on outgoing HTTP**: SteamGridDB **and** RomM behind Cloudflare Tunnel reject the default `Python-urllib`
   UA with 403 (Bot Fight Mode at the edge). Every HTTP-talking adapter (`RommHttpAdapter`, `SteamGridDbAdapter`) takes a
   `user_agent: str` ctor param. Bootstrap reads `package.json` once via `PluginMetadataReader` and threads
   `decky-romm-sync/<version>` to both — single source of truth, no hardcoded version strings.
 - **AddShortcut ignores most params**: `SteamClient.Apps.AddShortcut(name, exe, startDir, launchOptions)` ignores
   startDir and launchOptions (confirmed by MoonDeck plugin). Must use `Set*` calls (`SetShortcutName`, `SetShortcutExe`,
-  `SetShortcutStartDir`, `SetAppLaunchOptions`) after a 500ms delay. Do NOT pass quoted exe paths — the API handles
-  quoting internally.
+  `SetShortcutStartDir`, `SetAppLaunchOptions`) once the new app's overview is registered (see AddShortcut timing
+  above). Do NOT pass quoted exe paths — the API handles quoting internally.
 - **BIsModOrShortcut bypass DROPPED**: Phase 5.6 removed the bypass counter entirely. Shortcuts return
   `BIsModOrShortcut() = true` (natural state). We own the entire game detail UI via RomMPlaySection + future
   RomMGameInfoPanel.
-- **Shortcut property updates**: A shortcut's appId is derived from `exe + appName` (CRC32), so
-  `launchOptions`/`startDir` changes are **appId-safe** (same shortcut, binding/artwork/collections survive) and
-  `SetAppLaunchOptions`-on-existing is **reliable** (hardware-validated in #827: in-session + restart + churn). Use the
-  fire-then-poll-`AppDetails` confirm (`setLaunchOptionsConfirmed`) since `Set*` returns void. Delete + recreate
-  (re-sync) is only needed for **exe/name** changes, which produce a different appId. The real hazard is removal-churn
-  corrupting Steam's in-memory shortcut state (a restart clears it).
+- **Shortcut property updates**: Steam **assigns** a shortcut's appId at creation and it is **stable for the shortcut's
+  lifetime** (the plugin never computes it — it records the assigned id in `roms.shortcut_app_id` and detects ownership
+  by the exe path; the old `CRC32(exe + appName)` derivation is disproven on current Steam, see
+  `docs/architecture/steam-non-steam-shortcuts.md` §App IDs). So `launchOptions`/`startDir` changes are **appId-safe**
+  (same shortcut, binding/artwork/collections survive) and `SetAppLaunchOptions`-on-existing is **reliable**
+  (hardware-validated in #827: in-session + restart + churn). Use the fire-then-poll-`AppDetails` confirm
+  (`setLaunchOptionsConfirmed`) since `Set*` returns void. Delete + recreate (re-sync) is still the path for
+  **exe/name** changes (a fresh `AddShortcut` yields a new appId). The real hazard is removal-churn corrupting Steam's
+  in-memory shortcut state (a restart clears it).
 - **Launcher + launch_options model**: `bin/rom-launcher` (renamed from `bin/romm-launcher`, #778) is a pure `exec "$@"`
   wrapper — no state, no path resolution, no emulator knowledge. The Steam shortcut's `launch_options` carries the FULL
   launch command (`flatpak run net.retrodeck.retrodeck "<rom-path>"`) for installed ROMs, or `""` (placeholder) for
@@ -262,11 +273,11 @@ purity.)
 
 **Aggregates** (CP chapters 1–7 scope — locked in #788, refined by
 [ADR-0003](docs/adr/0003-json-sqlite-persistence-boundary.md)). The aggregate roots, their tables, and the enforcement
-layers live in `docs/architecture/database-design.md` (canonical — 8 roots after ADR-0003). Persistence boundary:
-config-shaped toggles (`save_sync_enabled`, `sync_before_launch`, `sync_after_exit`, `default_slot`,
-`autocleanup_limit`, `device_name`, `enabled_platforms`) live in `settings.json`, not SQLite —
-`SyncSettings`/`Platform`/`Device` were considered as aggregates and dropped. The rules below apply to the relational
-state that _does_ live in SQLite:
+layers live in `docs/architecture/database-design.md` (canonical — 9 roots: ADR-0003's 8 plus ADR-0023's
+`PlatformSyncState`). Persistence boundary: config-shaped toggles (`save_sync_enabled`, `sync_before_launch`,
+`sync_after_exit`, `default_slot`, `autocleanup_limit`, `device_name`, `enabled_platforms`) live in `settings.json`, not
+SQLite — `SyncSettings`/`Platform`/`Device` were considered as aggregates and dropped. The rules below apply to the
+relational state that _does_ live in SQLite:
 
 - `[CP]` One Repository Protocol per aggregate root, not per table. Aggregate boundaries are domain-modeling decisions;
   table layout is downstream and may need multiple tables to back one aggregate.

docs/adr/0009-launcher-pure-exec-wrapper-baked-launch-options.md

@@ -32,6 +32,12 @@ its artwork, its collection membership, and its `roms.shortcut_app_id` binding.
 destructive (it yields a different `appId`, i.e. a different shortcut), and the launch command lives entirely in
 `launch_options` — never in `exe` or `appName` — so re-resolving a path never disturbs the binding.
 
+> **Errata (2026-07).** The `appId = CRC32(exe + appName)` derivation stated here and later in this ADR (inherited from
+> ADR-0005) does not hold on current Steam — the appId is **assigned at creation** and stable for the shortcut's
+> lifetime. This does not change the decision: a `launch_options` write is appId-safe because it does not change an
+> existing shortcut's assigned appId (not because it falls outside a hash). Corrected model:
+> [Steam Non-Steam Shortcuts — App IDs and Artwork](../architecture/steam-non-steam-shortcuts.md#app-ids-and-artwork).
+
 ## Decision
 
 The launcher becomes a **pure exec wrapper**. `bin/rom-launcher` (renamed from `bin/romm-launcher`) is:

docs/adr/0021-sibling-group-one-shortcut-binding-active-version.md

@@ -32,6 +32,12 @@ one-binding-per-appId rule unbinds whichever sibling rows lost the race, and the
 sibling nondeterministically while the remaining rows permanently read as unsynced. Siblings whose display name differs
 (a Japanese dump with its native title) instead become **duplicate shortcuts** for the same game.
 
+> **Errata (2026-07).** The `appId = CRC32(exe + appName)` premise in this Context is disproven on current Steam — the
+> appId is **assigned at creation**, not hashed. Without a hash, same-named siblings do not collide onto one appId; they
+> instead produce **duplicate shortcuts** (the same failure the differing-name case already describes). The decision
+> below is unaffected: one shortcut per sibling group prevents both manifestations. Corrected model:
+> [Steam Non-Steam Shortcuts — App IDs and Artwork](../architecture/steam-non-steam-shortcuts.md#app-ids-and-artwork).
+
 ## Decision
 
 ### 1. Group identity is computed client-side, mirroring RomM's key; the fetch stays ungrouped