Skip to content

Commit 5c4ca04

Browse files
Lykhoydaclaude
andauthored
feat: observability UI — watch the agent live (read-only timeline + device + state) (#195)
* docs(plan): observability UI implementation plan — 7 phases, TDD Turns the reviewed spec into bite-sized TDD tasks: events.ts (classify/summarize/ clipThenRedact), recorder.ts (ring buffer + seq + screenshot bytes), telemetry observer (record-once post-ghost), SSE server (atomic snapshot, Host-header + Sec-Fetch-Site, screenshot route, backpressure, keep-alive), observe tool + slash command, React/Vite single-file SPA (Layout A, virtualized), CI bundle-freshness guard, changeset. All 12 Codex+Gemini findings mapped to tasks. * chore(changeset): observability UI * feat(observability): AgentEvent type + classifyFamily tool classifier Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(observability): clipThenRedact — clip-before-redact, fail-closed payload sanitizer Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(observability): summarize + mapObservation + ToolObservation type Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(observability): Recorder core Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(observability): capture screenshot bytes at record time Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(observability): harden recorder guarantees (survive-deletion, oversize, FIFO evict, subscriber isolation, FAIL-skip) * feat(telemetry): tool observer hook (record once post-ghost) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(observability): register recorder as tool observer Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(observability): SSE server skeleton + port fallback Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(observability): Host-header + Sec-Fetch-Site guard Node undici strips a forged Host header on the client side, so the DNS-rebinding leg of the guard is exercised over a raw socket in the test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(observability): SSE stream + heartbeat + backpressure Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(observability): screenshot route (buffer-only) + SPA serve Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * build(observability): commit compiled dist for phases 1-3 (dist/ is the shipped artifact, not gitignored) * feat(observability): observe MCP tool * feat(observability): /observe slash command * feat(observability): Layout A SPA + single-file build Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(observability): unwrap MCP result envelope + serve bundle from dist (3 critical integration defects) The observability layer read o.result.data directly, but the real value is the raw MCP ToolResult envelope ({ content:[{ text: JSON.stringify({ ok, data }) }] }). Effects: screenshots never captured (404 -> blank pane), state tabs showed the escaped-JSON wrapper, and the SPA route reader looked at the wrong field. - events.ts: add exported unwrapResult(); mapObservation now uses the unwrapped data as the payload (defensive fallback to raw result when unwrap fails). - recorder.ts: screenshotPath() unwraps the envelope before reading path/message. - vite.config.ts: outDir -> dist/observability/web-dist so the compiled server actually serves the committed bundle; remove the stale src/observability/web-dist. - server.ts: comment documenting the dist bundle path + Phase 7 CI guard target. - main.tsx: routeOf() reads the real React-Navigation shape (routeName, nested); drop dead selectedEv / hidden div. - tests: real-envelope regression guards in observability-events and -recorder (fail-before/pass-after), plus an existsSync-guarded GET / bundle-serve test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: web-bundle freshness guard for the observability SPA (Gemini 4) Rebuilds the single-file SPA from src/observability/web/ with npm ci (lockfile-exact → deterministic) and fails the PR if the committed dist/observability/web-dist/index.html drifted. PR-only, matches the require-changeset job pattern. Corrects the plan's stale path (src/.../web-dist → dist/.../web-dist, the actual shipped artifact). Verified locally: GREEN on a clean tree, RED on a source/bundle drift. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: document /rn-dev-agent:observe (README + CLAUDE-MD template + security note) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(dx): add /end-session maintainer command (project-scoped, not shipped) A repo-local slash command (.claude/commands/) that wraps up a work session: appends ADRs to DECISIONS.md, defects to BUGS.md, and a dated narrative + findings to ROADMAP.md in the sibling workspace, continuing the Dxxxx/Bxxxx sequences, then runs a git + changeset hygiene check. Append-only, never commits. Un-ignores .claude/commands/ so it persists in the repo while keeping .claude/settings*.json and worktrees ignored. Not in plugin.json — invoked as bare /end-session, not shipped to plugin users. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(observability): register observe.md in plugin.json (Phase-5 gap) The /rn-dev-agent:observe command file shipped but was never added to the plugin.json commands manifest, so it wasn't actually exposed to users. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(observability): address final-review findings (redact errors, drain SSE on stop, fail-closed write, port parse) Final code review (FIX-FIRST) surfaced four real issues: 1. CRITICAL — error.message bypassed redaction. args/payload went through redact() but raw tool errors (which often interpolate 401 bodies / auth echoes) streamed verbatim. Now routed through redact({m}).m, fail-closed to [REDACTED:error]. TDD: token + email + Bearer in an error are scrubbed. 2. stop() hung forever with an open SSE tab (s.close() waits for drain). Now s.closeAllConnections() first. TDD: stop() resolves <2s with a live stream connection (previously hung the test process). 3. SSE data write was bare res.write(); a throw escaped the detach path and leaked the subscriber. Now fail-closed (try/catch → false), matching the heartbeat's existing pattern in the same function. 4. RN_AGENT_OBSERVE_PORT="abc" → Number()=NaN → listen(NaN). Extracted pure parsePinnedPort() (finite, 1..65535 else undefined). TDD unit test. Also trimmed a stale Phase-7 CI-NOTE comment in server.index(). Full suite 1620/1620 green; web-bundle guard fresh. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 2c82b18 commit 5c4ca04

37 files changed

Lines changed: 4769 additions & 3 deletions

.changeset/observability-ui.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
"rn-dev-agent-cdp": minor
3+
"rn-dev-agent-plugin": minor
4+
---
5+
6+
Add the read-only observability UI (D1226 "watch the agent live"): an in-process recorder + opt-in SSE server serving a React SPA (timeline | device | state). New `observe` MCP tool + `/rn-dev-agent:observe` slash command. Deep-redacted (args + payload, fail-closed), localhost-only with Host-header + Sec-Fetch-Site guards.

.claude-plugin/plugin.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,8 @@
5252
"./commands/doctor.md",
5353
"./commands/list-learned-actions.md",
5454
"./commands/run-action.md",
55-
"./commands/check-vercel-rules.md"
55+
"./commands/check-vercel-rules.md",
56+
"./commands/observe.md"
5657
],
5758
"mcpServers": {
5859
"cdp": {

.claude/commands/end-session.md

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
command: end-session
3+
description: Wrap up the work session — log decisions to DECISIONS.md, bugs to BUGS.md, findings + a dated narrative to ROADMAP.md (in the sibling rn-dev-agent-workspace), then run a git/changeset hygiene check. Appends only; never deletes or rewrites existing entries; never commits.
4+
argument-hint: [optional focus or extra notes to weave in]
5+
allowed-tools: Bash, Read, Edit, Grep, Glob
6+
---
7+
8+
# /end-session — session wrap-up
9+
10+
A maintainer ritual for **this repo** (`claude-react-native-dev-plugin`). It distills
11+
what happened this session into the project's living docs, then flags anything left
12+
undone. The docs live in the **sibling workspace repo**, per the project `CLAUDE.md`:
13+
14+
| Doc | Path | What goes here |
15+
|---|---|---|
16+
| Roadmap | `../rn-dev-agent-workspace/docs/ROADMAP.md` | A dated narrative: what shipped, what's in flight, what's next, key findings |
17+
| Decisions | `../rn-dev-agent-workspace/docs/DECISIONS.md` | Architectural decisions as `Dxxxx` ADRs |
18+
| Bugs | `../rn-dev-agent-workspace/docs/BUGS.md` | Defects discovered/fixed as `Bxxxx` entries |
19+
20+
`$ARGUMENTS` (if present) is an explicit focus or note from the maintainer — weave it in;
21+
it does not replace your own reconstruction of the session.
22+
23+
## Iron rules
24+
25+
1. **Append only.** Never delete, reorder, or rewrite existing entries (this honors the
26+
standing "never delete md files" instruction). You add new entries and, for a bug that
27+
got fixed this session, append its `Fix` block / move it to the Fixed section — you do
28+
not edit history.
29+
2. **Never fabricate.** If the session produced no new decisions, write nothing to
30+
DECISIONS.md. Same for bugs. An empty doc update is the correct output when nothing
31+
qualifies — say so in the summary. Only ROADMAP always gets an entry (the session
32+
itself is the news).
33+
3. **Never commit.** You write the files and report. The maintainer reviews and commits.
34+
(The workspace is a separate git repo; its commits are theirs to make.)
35+
4. **Continue the sequence.** IDs are monotonic. Read the current max `Dxxxx` / `Bxxxx`
36+
before writing and increment from there — do not reuse or guess.
37+
5. **Match the house format** of each doc exactly (see the templates below). Use today's
38+
date from `date +%F`, never a guessed date.
39+
40+
## Procedure
41+
42+
### Step 0 — Preconditions
43+
- `WS=../rn-dev-agent-workspace/docs`. Confirm `$WS/ROADMAP.md`, `$WS/DECISIONS.md`,
44+
`$WS/BUGS.md` all exist. If the workspace is missing, **stop** and tell the maintainer
45+
— do **not** create these files inside the plugin repo (that's the "two test-apps"
46+
class of mistake the project `CLAUDE.md` warns about).
47+
48+
### Step 1 — Gather evidence (don't rely on memory alone)
49+
Run these and read the output before writing anything:
50+
- `date +%F` → today's date for the entries.
51+
- This repo: `git -C . log --oneline -25` and `git -C . status --short` and
52+
`git -C . branch --show-current`.
53+
- Workspace: `git -C ../rn-dev-agent-workspace log --oneline -15` and
54+
`git -C ../rn-dev-agent-workspace status --short`.
55+
- Current max IDs:
56+
- `grep -oE '### D[0-9]+' "$WS/DECISIONS.md" | grep -oE '[0-9]+' | sort -n | tail -1`
57+
- `grep -oE '### B[0-9]+' "$WS/BUGS.md" | grep -oE '[0-9]+' | sort -n | tail -1`
58+
- Then reconstruct from THIS conversation: what was built/changed, which choices were
59+
architectural (→ ADR), which were defects (→ bug), which were noteworthy observations
60+
(→ findings in the roadmap entry).
61+
62+
### Step 2 — DECISIONS.md (only if there were architectural decisions)
63+
For each new decision, append a block at the end, continuing the `Dxxxx` sequence:
64+
65+
```markdown
66+
### D<next>: <imperative one-line title>
67+
68+
**Context.** <what forced the decision; the constraint or surprise>
69+
70+
**Decision.** <what was chosen; alternatives considered and why rejected>
71+
72+
**Lesson.** <the durable, transferable takeaway — optional but valued in this repo>
73+
74+
**Refs.** <GH #, PR #, commit SHAs, files touched, related Dxxxx/Bxxxx>
75+
```
76+
77+
### Step 3 — BUGS.md (only if defects were found or fixed)
78+
New defect → append under the relevant section continuing the `Bxxxx` sequence:
79+
80+
```markdown
81+
### B<next>: <one-line symptom> (<Open | Fixed — YYYY-MM-DD>)
82+
83+
**Observed.** <how it showed up; repro if known>
84+
85+
**Fix.** <what resolved it, or "Proposed fix." if still open>
86+
87+
**Refs.** <GH #, PR #, commit SHAs, files, related Dxxxx/Bxxxx>
88+
```
89+
If a previously-Open bug was fixed this session, append a `Fix.` block to it (don't rewrite
90+
the original Observed text).
91+
92+
### Step 4 — ROADMAP.md (always)
93+
Append a dated section at the end:
94+
95+
```markdown
96+
## <YYYY-MM-DD> — <session title>
97+
98+
<1–3 sentence framing: what the session was about and why it mattered.>
99+
100+
**Shipped.** <bullet or prose list of what landed — commits, files, behavior changes.>
101+
102+
**Findings.** <noteworthy observations that aren't ADRs or bugs — perf numbers,
103+
gotchas, things to watch. Omit the heading if there were none.>
104+
105+
**Process.** <how the work was done if notable — TDD, multi-agent review, brainstorm.>
106+
107+
**Forward.** <what's still open / next — carry forward unfinished items from the prior
108+
entry's Forward block that are still open.>
109+
110+
**Refs.** <Dxxxx/Bxxxx created this session, GH #, PR #.>
111+
```
112+
113+
### Step 5 — Hygiene check (report, don't fix)
114+
- Uncommitted work: surface `git status --short` for **both** repos (the plugin repo and
115+
the workspace, including the docs you just wrote).
116+
- Changeset gate: if `scripts/cdp-bridge/src/` changed this session (`git diff --name-only`
117+
against the branch base), confirm a `.changeset/*.md` exists. If shippable src changed
118+
with no changeset, **flag it** — the `require-changeset` CI job will fail the PR.
119+
- Branch: note the current branch and whether it's ahead of `main` / has an open PR.
120+
121+
### Step 6 — Summarize
122+
Print a compact report:
123+
- Which docs were updated and the new IDs assigned (e.g. "DECISIONS +D1229, D1230; BUGS
124+
+B162; ROADMAP +2026-05-31 entry").
125+
- The hygiene flags (uncommitted files, missing changeset, branch state).
126+
- An explicit reminder that **nothing was committed** — list the exact files the
127+
maintainer should review and commit, in both repos.
128+
129+
Keep the final report short; the value is in the doc updates, not the recap.

.github/workflows/ci.yml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,3 +72,16 @@ jobs:
7272
run: |
7373
git fetch origin "$BASE_REF_NAME"
7474
BASE_REF="origin/$BASE_REF_NAME" bash scripts/require-changeset.sh
75+
76+
web-bundle:
77+
name: Web bundle freshness
78+
runs-on: ubuntu-latest
79+
# PR-only: rebuilds the observability SPA and fails if the committed
80+
# single-file bundle drifted from source (npm ci → deterministic).
81+
if: github.event_name == 'pull_request'
82+
steps:
83+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 — Phase 134.5 SHA pin
84+
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 — Phase 134.5 SHA pin
85+
with:
86+
node-version: 22
87+
- run: bash scripts/check-web-bundle.sh

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
node_modules/
22

3-
.claude
3+
.claude/*
4+
!.claude/commands/
45
.agents
56
.idea
67
skills-lock.json

CLAUDE-MD-TEMPLATE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -522,4 +522,5 @@ After implementing any feature, in this order:
522522
| `/rn-dev-agent:check-env` | Verify Metro, CDP, simulator are ready before starting work |
523523
| `/rn-dev-agent:proof-capture` | Feature is done, need PR-ready video + screenshots + PR body |
524524
| `/rn-dev-agent:nav-graph` | Need to understand or query the app's navigation structure |
525+
| `/rn-dev-agent:observe` | Watch the agent live in a browser — read-only `127.0.0.1` UI with the tool-call timeline, latest device screenshot, and route/store/component-tree panels (deep-redacted, opt-in) |
525526
| `/rn-dev-agent:send-feedback` | Report a plugin bug or issue (creates sanitized GitHub issue) |

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ Claude runs an [8-phase pipeline](https://lykhoyda.github.io/rn-dev-agent/comman
7373
| `/rn-dev-agent:debug-screen` | Diagnose and fix a broken screen — gathers parallel evidence from CDP + native logs + component tree |
7474
| `/rn-dev-agent:build-and-test <desc>` | Build app (local or EAS), install on device, then test |
7575
| `/rn-dev-agent:proof-capture <desc>` | Rehearsal-gated video + screenshots + PR body |
76+
| `/rn-dev-agent:observe` | Start a read-only local web UI to **watch the agent live** — tool-call timeline, latest device screenshot, and route/store/component-tree panels. Prints a `127.0.0.1` URL to open in a browser. |
7677

7778
**Actions: replayable app flows + the LLM/pragmatic hybrid**
7879

@@ -217,6 +218,8 @@ Recommended usage:
217218

218219
The plugin makes no attempt to sandbox `cdp_evaluate` calls. If you need that, gate the agent's tool access through Claude Code's permission prompts rather than trusting the tool layer to enforce safety.
219220

221+
The **observability UI** (`/rn-dev-agent:observe`) is opt-in and read-only: it binds to `127.0.0.1` on a random port, rejects cross-origin requests via Host-header + `Sec-Fetch-Site` checks, and serves no mutation endpoints. Tool arguments and payloads are deep-redacted (fail-closed — an unredactable value is dropped, not leaked) before they ever reach the stream, so tokens, passwords, and PII render as `[REDACTED_*]`. The recorder keeps only a small bounded in-memory ring buffer (recent events, never written to disk); nothing is exposed until you explicitly start the server.
222+
220223
## Keeping up to date
221224

222225
Enable auto-update in the plugin manager (Marketplaces tab), or update manually:

commands/observe.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
command: observe
3+
description: Start the read-only observability web UI and print the URL to watch the agent live.
4+
---
5+
6+
Call the `observe` MCP tool with `action: "start"`. Print the returned `url` prominently and tell the user to open it in a browser to watch the live tool-call timeline, device screenshot, and app state. If it's already running, `observe status` returns the existing URL; to stop, call `observe` with `action: "stop"`.

0 commit comments

Comments
 (0)