Skip to content

Commit 5fe66c9

Browse files
Lykhoydaclaude
andauthored
feat(actions): durable node:sqlite action store — Phase 1 dual-write mirror (#359)
* docs(spec): action corpus persistence — node:sqlite index + history (design) Approved brainstorming design for durable, structured action storage: YAML stays the git-tracked source of truth; a derived, gitignored node:sqlite DB (.rn-agent/state/actions.db) holds the corpus index + run/repair history, replacing the per-action JSON sidecars. Two phases: P1 DB layer + migration + graceful degradation; P2 reconcile() loud diagnostics + #348/#357 resolution guard (subsumes #357). All source TypeScript; learned-actions.mjs migrated to TS. Non-goals: cross-project library, team/remote sync. Refs #357, #348, #300 (item 3); #356 unblocked as a separate spec. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(plan): action storage persistence Phase 1 — TDD implementation plan 9-task TDD plan for the node:sqlite store: action-db (open/schema/degrade, state load-save, sidecar migration), backend-selecting facade, wire run/repair/save-as-action call-sites, cdp_status.actionStore, supervisor --experimental-sqlite + engines>=22.5 + gitignore, learned-actions.mjs→TS, changeset+docs. YAML stays source of truth; DB derived/gitignored/rebuildable. Refs spec 2026-06-19-action-storage-persistence-design.md. Phase 2 (reconcile + #348/#357 resolution guard) is a separate plan. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(plan): amendments applied from the multi-LLM plan review Claude Opus + Codex review returned no-ship-as-is; applied findings to spec + plan before any code (CLAUDE.md workflow step 2): - A1/P0: ESM build → require('node:sqlite') ships dead; use createRequire(import.meta.url); bump @types/node ^20→^22.5/^24; CI asserts loadSqlite() non-null on Node 24. - A2/P0: real seam is domain/action-store.ts (loadAction/saveAction/saveActionWithCAS), not the 3 tool files — wiring only tools = split-brain (stale-sidecar reads, DB writes). - A3/P0: Phase 1 is additive — sidecars stay authoritative, DB is a mirror; flip authority in Phase 2 with reconcile() so deleting the gitignored DB can't silently lose history. - A4/P1: keep engines >=22 (not >=22.5); version-gated sqliteFlagForNode (22.5–23.5 only); verify RN_BRIDGE_SUPERVISOR=0 in-process path degrades. - A5/P1: append-and-trim in BEGIN IMMEDIATE + busy_timeout + WAL; explicit stats_json (don't recompute cumulative totalRuns from capped history); preserve failure_detail; COALESCE. - A6/P1: compile learned-actions to dist/ (strip-types needs 22.6 + won't resolve .ts specifiers). - A7/P2: read-only 3-way storeMode; ESM tests; side-effect-free workerSpawnArgs; dbCache close/reset lifecycle; drop redundant state/*.db gitignore (templates already ignore state/). Gemini was unavailable (Code Assist individual tier retired → use antigravity,codex next time). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(actions): node:sqlite store — open + schema + PRAGMA + graceful degradation (Task 1) Adds `scripts/cdp-bridge/src/domain/action-db.ts`: - `loadSqlite()` — ESM-safe dynamic loader via `createRequire(import.meta.url)`; returns `DatabaseSync` ctor or null (never throws) - `openActionDb(projectRoot, opts?)` — opens `.rn-agent/state/actions.db`, creates the three-table schema (actions_index + stats_json, run_records with failure_detail, repair_records), sets PRAGMA busy_timeout=5000 + journal_mode=WAL, and returns `{ db, close }` or null on any failure - Graceful degradation path tested via `opts.sqliteCtor: null` - CI-loud test asserts `loadSqlite()` is non-null on Node 24 so a future ESM-loader regression fails visibly - Bumps @types/node devDependency from ^20 to ^24 for Node 22.5+ API types TDD: 4/4 unit tests pass; full 2367-test suite remains green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(actions): T2 action-db persistence primitives — insertRunRecord/insertRepairRecord, upsertIndex (COALESCE), loadState (stats from stats_json), recentRepairCount Extends ActionDb handle with five read/write primitives: - insertRunRecord: IMMEDIATE tx INSERT + cap-trim to RUN_HISTORY_MAX=50 - insertRepairRecord: IMMEDIATE tx INSERT + cap-trim to REPAIR_HISTORY_MAX=25 - upsertIndex: INSERT…ON CONFLICT DO UPDATE with COALESCE so partial (stats-only) updates never null out app_id/path/content_hash/status - loadState: reconstructs ActionRuntimeState from DB; stats read from stored stats_json — NOT recomputed from capped row history (so totalRuns correctly exceeds runHistory.length) - recentRepairCount: COUNT repair_records WHERE ts >= sinceIso (repair budget gate) Tests (TDD, 8 new cases): round-trip, no-recompute proof (totalRuns=70 with 50 rows), trim-at-cap for both tables, COALESCE preservation, repair history round-trip, recentRepairCount windowing, transport field fidelity. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(actions): migrate legacy JSON sidecars into the db (idempotent) Adds migrateSidecars() to the ActionDb handle: scans <projectRoot>/.rn-agent/state/<id>.state.json files, skips any id already present in actions_index, imports runHistory/repairHistory via the Task-2 append primitives (preserves chronological order), upserts the index with revision/stats/mtimeBaseline. Skips corrupt JSON and schemaVersion !== 1 files without throwing. Returns { migrated: count }; second call returns 0 (idempotent by design). Extends the test suite with 4 new cases: happy-path import + idempotency, missing state dir, corrupt/wrong-version skip, and a multi-record roundtrip covering both runHistory and repairHistory. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(actions): backend-selecting state-store facade (dual-write mirror) Phase 1 additive dual-write facade between the action-store/tool layer and action-db.ts. Sidecars stay authoritative (read source); the SQLite DB is a best-effort populated mirror + read-only surface. - loadOrInitState: reads from the authoritative sidecar (DB read flips in Phase 2) - persist: sidecar-first (never swallowed), then best-effort DB mirror that appends only the new run/repair record(s) and COALESCE-upserts the index; a mirror failure never throws and never corrupts the sidecar - storeMode: READ-ONLY 3-way detector (sqlite | degraded:sqlite-unavailable | degraded:open-failed) that never opens/migrates the DB - dbFor: lazy per-root open + one-time migrateSidecars, cached with a failed-open marker - resetActionStore / closeActionStoresForTest lifecycle; __setSqliteCtorForTest seam Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(actions): stage rebuilt dist/action-db.js + lockfile (Tasks 1-3) dist/ is tracked; Tasks 1-3 committed action-db.ts source but not the rebuilt dist/domain/action-db.js artifact, and Task 1's @types/node bump left package-lock.json unstaged. Reconcile so a fresh checkout has the compiled module the facade imports. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(actions): make action persistence store-aware (Phase 1 dual-write) Wire the authoritative write paths to also mirror to the node:sqlite DB (A2/A3/A5). Sidecars stay authoritative + the read source; the DB is a best-effort, sidecar-less mirror that never throws into the authoritative path. - action-state-store: add sidecar-less mirrorToDb() (derives projectRoot from the .rn-agent/actions/<id>.yaml path, fails open for non-conventional paths); refactor persist() to compose saveSidecar + mirrorToDb (DRY, behavior intact). - action-store: saveAction + acknowledgeExternalEdit mirror the index/mtime AFTER the #101 atomic pair-write (no double sidecar write, no row). - run-action persistRun: append the RunRecord row only on saveActionWithCAS ok:true (never on the #117 CAS-conflict path). - repair-action: append the RepairRecord row after saveAction. - save-as-action: seed the DB index row after the initial pair-write. #101 atomic pair-write + #117 CAS semantics preserved (62 regression tests green; full suite 2390/2390). New TDD test pins the mirror, the path derivation, the never-throw guarantee, and the CAS conflict. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(status): surface actionStore backend in cdp_status payload Adds `actionStore` field to the cdp_status response — a read-only 3-way detector (`'sqlite'` / `'legacy-files'` / `'degraded:<reason>'`) that calls `storeMode()` without opening or migrating the DB. Also narrows `storeMode`'s return type from `string` to the discriminated `ActionStoreMode = 'sqlite' | 'legacy-files' | \`degraded:\${string}\`` so the 3-way contract is encoded in the type system. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(actions): version-gated --experimental-sqlite flag in worker spawn (Task 7) Add pure supervisor-args.ts with sqliteFlagForNode() and workerSpawnArgs(): flag is passed only for 22.5 ≤ node < 23.6 (where node:sqlite needs it); v < 22.5 degrades gracefully, v ≥ 23.6 already on by default. Wire supervisor.ts to use workerSpawnArgs() instead of inline array. Engines kept at >=22 (not bumped) so older Node degrades rather than rejects. gitignore unchanged — templates/rn-agent/.gitignore state/ already covers *.db. 12/12 new tests; 2405/2405 full suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(T8): migrate learned-actions.mjs → TypeScript compiled to dist/ Port scripts/learned-actions.mjs (480-line CLI inventory tool) to scripts/cdp-bridge/src/learned-actions.ts, compiled by tsc into dist/learned-actions.js via the existing tsconfig (rootDir=src, outDir=dist). Output is byte-identical to the golden baseline (verified diff empty). - Add explicit TypeScript types (Flags, MemoryItem, FlowItem, FlowMeta, ProducesMap, SkeletonItem, CommandItem, *Result interfaces) - Use `import type` where type-only - No logic changes — faithful port of all 4 sections (A/B/C/D), all flags, exit codes, and human/JSON output modes - Delete scripts/learned-actions.mjs (git rm) - Update all 7 invocation sites to scripts/cdp-bridge/dist/learned-actions.js: commands/list-learned-actions.md, commands/run-action.md, agents/rn-tester.md, agents/rn-debugger.md, skills/creating-actions/SKILL.md, skills/creating-actions/references/m7-header-reference.md, plus source comments in domain/reusable-action.ts and tools/test-recorder-generators.ts - Add parity test (learned-actions-parity.test.js, 8 assertions): fixture corpus → spawn compiled CLI → assert count/ids/fields/exit codes Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: changeset for action-storage persistence Phase 1 Adds .changeset/action-storage-phase1.md (minor bump for rn-dev-agent-cdp) describing the node:sqlite action history store, dual-write mirror, graceful degradation, version-gated --experimental-sqlite flag, and learned-actions TS migration (Tasks 1–8). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(action-store): idempotent DB-mirror append fixes I1 migration-boundary double-count The dual-write flow writes the authoritative sidecar FIRST, then mirrorToDb runs. The first mirror per project triggers dbFor() -> migrateSidecars(), which imports the entire current sidecar history (already including the just-appended record); mirrorToDb then appended that same record AGAIN, duplicating the newest run/repair row for any migrated action. Make insertRunRecord / insertRepairRecord idempotent: within the same BEGIN IMMEDIATE transaction, skip the INSERT when a row for (action_id, ts) already exists. This absorbs the migration-boundary overlap and any other accidental re-append while preserving full pre-migration history (A5 append semantics, best-effort mirror, authoritative sidecar/#101/#117 untouched). Tests: - I1 regression (action-store-mirror): seed a legacy sidecar with run1+run2 / repair1+repair2, drive a record-bearing mirror -> DB == authoritative count (2, not 3), no duplicate ts; a distinct run3 then appends to 3, each once. - Fresh-action control: no legacy sidecar -> appends work normally. - T2 (action-db): loadState round-trips a fail RunRecord with both failureCode AND failureDetail. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(action-db): codex-pair review fixes — migrateSidecars robustness, test cleanup, CAS assertion Finding 1 (F1, MED): migrateSidecars now validates Array.isArray(runHistory/repairHistory) before importing — a malformed sidecar is skipped cleanly with no index row written (retryable). History rows are inserted FIRST, upsertIndex LAST, so "index row present" reliably means "fully migrated"; any mid-import throw leaves no partial row. Finding 2 (F2, MED): all test bodies that open action-store/sqlite singletons or probe handles now wrap in try/finally so closeActionStoresForTest()/probe.close() run even when an assertion throws — preventing state leakage into later tests. Finding 3 (F3, MED): the CAS #117 test now captures and asserts ok===true on the conflict-creating write before asserting the stale bWriter gets EXTERNAL_WRITE, so the assertion cannot pass for the wrong reason. Regression test added: F1 regression — malformed sidecar (non-array histories) leaves no actions_index row; valid sibling migrates fully; idempotent re-run adds no duplicate rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(save-as-action): mirror new actions with the post-write mtime Codex PR review (P2): cdp_record_test_save_as_action seeded the DB mirror from the pre-write initialState (lastSeenMtimeMs=0), but pairWrite rewrites the sidecar with its finalMtimeMs. A DB-backed load (Phase 2) would then see mtime_baseline=0 and treat the just-written YAML as externally edited. Mirror a state carrying writeResult.finalMtimeMs, matching saveAction's existing handling. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 80c225e commit 5fe66c9

41 files changed

Lines changed: 4996 additions & 89 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"rn-dev-agent-cdp": minor
3+
---
4+
5+
Action corpus run/repair history now persists in a derived, gitignored node:sqlite store (.rn-agent/state/actions.db) alongside the per-action JSON sidecars (Phase 1 dual-write: sidecars stay authoritative, the DB is a rebuildable mirror), with graceful degradation to sidecar-only when node:sqlite is unavailable. The worker enables node:sqlite via a version-gated --experimental-sqlite flag (Node 22.5–23.5); the engines floor stays >=22. cdp_status now reports the active backend as `actionStore`. The learned-actions inventory script is migrated from JavaScript to TypeScript (compiled to dist/).

agents/rn-debugger.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ than a manual walk-through, and gives you a deterministic re-run after the
7777
fix. Codified in `feedback_execute_artifacts_before_manual.md`.
7878

7979
```bash
80-
node "${CLAUDE_PLUGIN_ROOT}/scripts/learned-actions.mjs" \
80+
node "${CLAUDE_PLUGIN_ROOT}/scripts/cdp-bridge/dist/learned-actions.js" \
8181
--json --section b --filter "<feature-or-screen-keyword>" \
8282
--workspace-root "$PWD" --memory-cwd "$PWD" \
8383
> /tmp/learned-actions.json

agents/rn-tester.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ Maestro replay. Codified in
7171
`feedback_execute_artifacts_before_manual.md`.
7272

7373
```bash
74-
node "${CLAUDE_PLUGIN_ROOT}/scripts/learned-actions.mjs" \
74+
node "${CLAUDE_PLUGIN_ROOT}/scripts/cdp-bridge/dist/learned-actions.js" \
7575
--json --section b --filter "<feature-keyword>" \
7676
--workspace-root "$PWD" --memory-cwd "$PWD" \
7777
> /tmp/learned-actions.json

commands/list-learned-actions.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
command: list-learned-actions
3-
description: List persisted "learned actions" — feedback memories, Maestro flows, UI skeletons, and plugin commands that should be consulted before composing new device_* primitives. Wraps scripts/learned-actions.mjs for programmatic discovery.
3+
description: List persisted "learned actions" — feedback memories, Maestro flows, UI skeletons, and plugin commands that should be consulted before composing new device_* primitives. Wraps scripts/cdp-bridge/dist/learned-actions.js for programmatic discovery.
44
argument-hint: [filter-keyword]
55
allowed-tools: Bash, Read, Glob
66
---
@@ -16,15 +16,15 @@ should look at BEFORE re-deriving anything from scratch:
1616
2. **Executable artifacts**: `.rn-agent/actions/*.yaml`, `.rn-agent/skeleton.yaml`
1717

1818
This command surfaces both lists in one place — and the underlying
19-
`scripts/learned-actions.mjs` is the **same script** invoked programmatically
19+
`scripts/cdp-bridge/src/learned-actions.ts` (compiled to `scripts/cdp-bridge/dist/learned-actions.js`) is the **same script** invoked programmatically
2020
by `/rn-dev-agent:test-feature` Step 0 and by the `rn-tester` / `rn-debugger`
2121
agents' artifact-scan steps. Keeping the discovery logic in one script means
2222
every consumer sees the same inventory.
2323

2424
## Run
2525

2626
```bash
27-
node "${CLAUDE_PLUGIN_ROOT}/scripts/learned-actions.mjs" \
27+
node "${CLAUDE_PLUGIN_ROOT}/scripts/cdp-bridge/dist/learned-actions.js" \
2828
--workspace-root "$PWD" \
2929
--memory-cwd "$PWD" \
3030
${ARGUMENTS:+--filter "$ARGUMENTS"}
@@ -46,7 +46,7 @@ decision-making. **Always use `--json` from a programmatic caller** — the
4646
human table format is not a stable contract.
4747

4848
```bash
49-
RESULT=$(node "${CLAUDE_PLUGIN_ROOT}/scripts/learned-actions.mjs" \
49+
RESULT=$(node "${CLAUDE_PLUGIN_ROOT}/scripts/cdp-bridge/dist/learned-actions.js" \
5050
--json --section b --filter "task creation" --workspace-root "$PWD" --memory-cwd "$PWD")
5151
echo "$RESULT" | jq '.sections.flows.items[0].path'
5252
```

commands/run-action.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
command: run-action
3-
description: Execute a learned Maestro flow ("action") by name with optional -e KEY=VALUE parameters. Looks the flow up via scripts/learned-actions.mjs (same inventory as /rn-dev-agent:list-learned-actions), then replays it via cdp_run_action — auto-repair-aware orchestration with structured RunRecords (GH #116). Counterpart to /list-learned-actions — list discovers, run executes.
3+
description: Execute a learned Maestro flow ("action") by name with optional -e KEY=VALUE parameters. Looks the flow up via scripts/cdp-bridge/dist/learned-actions.js (same inventory as /rn-dev-agent:list-learned-actions), then replays it via cdp_run_action — auto-repair-aware orchestration with structured RunRecords (GH #116). Counterpart to /list-learned-actions — list discovers, run executes.
44
argument-hint: <action-name> [-e KEY=VALUE ...] [--platform ios|android] [--no-auto-repair] [--dry-run]
55
allowed-tools: Bash, Read, Glob, mcp__plugin_rn-dev-agent_cdp__cdp_run_action
66
---
@@ -50,7 +50,7 @@ Example calls:
5050
`.rn-agent/actions/` directly):
5151
```bash
5252
ACTION_NAME="<first-arg>"
53-
RESULT=$(node "${CLAUDE_PLUGIN_ROOT}/scripts/learned-actions.mjs" \
53+
RESULT=$(node "${CLAUDE_PLUGIN_ROOT}/scripts/cdp-bridge/dist/learned-actions.js" \
5454
--json --section b \
5555
--workspace-root "$PWD" --memory-cwd "$PWD" \
5656
--filter "$ACTION_NAME")
@@ -218,5 +218,5 @@ reliably call other slash commands), but the contract is the same.
218218
— but most flows still cold-start.
219219

220220
See `commands/list-learned-actions.md` for the discovery counterpart and
221-
`scripts/learned-actions.mjs` for the underlying script (single source of
221+
`scripts/cdp-bridge/src/learned-actions.ts` (compiled to `scripts/cdp-bridge/dist/learned-actions.js`) for the underlying script (single source of
222222
truth for both commands).

0 commit comments

Comments
 (0)