|
1 | 1 | # Changelog |
2 | 2 |
|
3 | | -## [1.15.2.0] - 2026-04-26 |
| 3 | +## [1.16.1.0] - 2026-04-26 |
4 | 4 |
|
5 | 5 | ## **`browse screenshot` no longer bricks your session by capturing 3000px-tall PNGs.** |
6 | 6 |
|
@@ -39,6 +39,85 @@ Reported by @raffoz in #1214 with a clean reproducer and three proposed options. |
39 | 39 | - New `browse/test/fixtures/tall.html` (~3000px scroll) feeds the regression coverage. |
40 | 40 | - Five new test cases in `browse/test/commands.test.ts`: default-viewport regression for #1214, `--full-page` capture height verification, oversize warning emission, and the two mutually-exclusive flag combinations (`--full-page + --clip`, `--full-page + selector`). |
41 | 41 | - PNG dimensions are read from the IHDR chunk (zero deps, ~10 lines in `meta-commands.ts`). |
| 42 | +## [1.15.0.0] - 2026-04-26 |
| 43 | + |
| 44 | +## **Real-PTY test harness ships. 11 plan-mode E2E tests, 23 unit tests, and 50K fewer tokens per invocation.** |
| 45 | + |
| 46 | +Two big pieces of engineering in one release. The headline is a real-PTY test harness — 654 lines of TypeScript on top of `Bun.spawn({terminal:})` — that drives the actual `claude` binary and parses rendered terminal frames. Six new E2E tests on the harness cover behaviors that were structurally unreachable before: format compliance for every gstack `AskUserQuestion`, plan-design UI-scope detection (positive coverage), tool-budget regression vs prior runs, `/ship` end-to-end idempotency against a real git fixture, `/plan-ceo` answer-routing, and `/autoplan` phase sequencing. The branch nets ~11.6K lines smaller against `main` while adding ~1,450 lines of new TypeScript test code — preamble resolvers were rewritten to keep every semantic rule in less prose, and the test surface that catches AskUserQuestion drift expanded from zero to gate-tier on every PR. |
| 47 | + |
| 48 | +### The numbers that matter |
| 49 | + |
| 50 | +Branch totals come from `git diff --shortstat origin/main..HEAD`. Token-level reduction comes from regenerating every `SKILL.md` against the rewritten resolvers (`bun run gen:skill-docs --host all`). E2E numbers come from `EVALS=1 EVALS_TIER=gate bun test test/skill-e2e-*.test.ts` on a clean working tree. |
| 51 | + |
| 52 | +| Metric | Δ | |
| 53 | +|---|---| |
| 54 | +| Net branch size vs `main` | **−11,609 lines** (89 files, +7,240 / −18,849) | |
| 55 | +| New test files added | **8 files** (1 harness unit-test + 7 E2E tests) | |
| 56 | +| New test code shipped | **~1,453 lines** of TypeScript | |
| 57 | +| Real-PTY harness module | **654 lines** in `test/helpers/claude-pty-runner.ts` | |
| 58 | +| Per-invocation token savings | **−196K tokens (−25%)** on cold reads | |
| 59 | +| `plan-ceo-review` preamble | **−43%** (54 KB → 31 KB) | |
| 60 | +| Plan-mode E2E test count | **5 → 11** | |
| 61 | +| New gate-tier paid E2E tests | **+3** (format compliance, design-with-UI, budget regression) | |
| 62 | +| New periodic-tier paid E2E tests | **+3** (mode-routing, ship-idempotency, autoplan-chain) | |
| 63 | +| Helper unit test coverage | **+23 tests** for parser + budget primitives | |
| 64 | +| All free tests | **49 pass, 0 fail** | |
| 65 | + |
| 66 | +| Skill class | Per-invocation surface | Δ | |
| 67 | +|---|---|---| |
| 68 | +| Tier-≥3 plan reviews (full preamble) | ~50 KB → ~30 KB | −40% | |
| 69 | +| Tier-1 quick skills | ~12 KB → ~9 KB | −25% | |
| 70 | + |
| 71 | +Every gstack invocation now sends ~50K fewer tokens to the model on cold reads — that's roughly a quarter of a typical 200K context window freed up for actual work. Tier-≥3 plan reviews keep their full functional surface (Brain Sync, Context Recovery, Routing Injection) and still lose almost half the bytes. |
| 72 | + |
| 73 | +### What this means for builders |
| 74 | + |
| 75 | +Three new classes of regression that were previously impossible to catch now block every PR. **Format drift**: a missing `Recommendation:` line or absent Pros/Cons bullet on an `AskUserQuestion` is caught against the real rendered terminal — not the model's claim about what it would have shown. **Conditional skill paths**: `/plan-design-review` had to early-exit when there's no UI scope, but until this release nothing tested the *positive* path; a regression that flipped the detector to "early-exit always" could have shipped silently. **Tool-budget regressions**: a preamble change that makes any skill burn 2× its prior tool calls fails a free, branch-scoped assertion that runs on every `bun test`. |
| 76 | + |
| 77 | +The harness itself is a reusable primitive. `runPlanSkillObservation()` watches plan-mode terminal output and classifies outcomes as `asked` / `plan_ready` / `silent_write` / `exited` / `timeout`. Three periodic-tier tests built on top of it cover the heavier cases — multi-phase chain ordering, ship idempotency state-machine end-to-end, and answer routing through 8-12 sequential prompts — that don't fit a per-PR budget but run weekly. Pull, run `bun run gen:skill-docs --host all`, and every skill invocation is meaningfully smaller and meaningfully better-tested than the prior release. |
| 78 | + |
| 79 | +### Itemized changes |
| 80 | + |
| 81 | +#### Added |
| 82 | + |
| 83 | +- `test/helpers/claude-pty-runner.ts`: real-PTY test harness using `Bun.spawn({terminal:})` (Bun 1.3.10+ has built-in PTY — no `node-pty`, no native modules). Exposes `launchClaudePty()` for raw session control and `runPlanSkillObservation()` as the high-level contract for plan-mode skill tests. |
| 84 | +- `parseNumberedOptions(visible)` and `isPermissionDialogVisible(visible)` helpers in `claude-pty-runner.ts`. Tests can now look up an option index by its label without hard-coding positions, and auto-grant Claude Code's file-edit / workspace-trust / bash-permission dialogs that fire during preamble side-effects. |
| 85 | +- `findBudgetRegressions()` and `assertNoBudgetRegression()` in `test/helpers/eval-store.ts`. Pure functions returning tests that grew >2× in tools or turns vs the prior eval run, with floors at 5 prior tools / 3 prior turns to avoid noise. Env override `GSTACK_BUDGET_RATIO`. |
| 86 | +- 6 new real-PTY E2E tests on the harness: |
| 87 | + - `skill-e2e-ask-user-question-format-compliance.test.ts` (gate, ~$0.50/run): asserts every gstack `AskUserQuestion` rendering contains the 7 mandated format elements (ELI10, Recommendation, Pros/Cons with ✅/❌, Net, `(recommended)` label). |
| 88 | + - `skill-e2e-plan-design-with-ui.test.ts` (gate, ~$0.80/run): positive coverage for `/plan-design-review` UI-scope detection. Counterpart to the existing no-UI early-exit test — without it, a regression that flips the detector to "early-exit always" would ship undetected. |
| 89 | + - `skill-budget-regression.test.ts` (gate, free): branch-scoped library-only assertion that no skill burns >2× tools or turns vs its prior recorded run. |
| 90 | + - `skill-e2e-plan-ceo-mode-routing.test.ts` (periodic, ~$3/run): verifies AskUserQuestion answer routing — HOLD SCOPE picks routes to rigor language, SCOPE EXPANSION picks route to expansion language. |
| 91 | + - `skill-e2e-ship-idempotency.test.ts` (periodic, ~$3/run): runs `/ship` end-to-end against a real git fixture with `STATE: ALREADY_BUMPED` baked in; asserts no double-bump, no double-commit, no fixture mutation. |
| 92 | + - `skill-e2e-autoplan-chain.test.ts` (periodic, ~$8/run): asserts `/autoplan` phase ordering by tee'ing timestamps as each `**Phase N complete.**` marker appears. |
| 93 | +- `test/helpers-unit.test.ts`: 23 unit tests covering `parseNumberedOptions` edge cases (empty, partial paint, >9 options, stale-vs-fresh anchoring) and `findBudgetRegressions` (noise floor, env override, missing tool data). |
| 94 | +- `test/fixtures/plans/ui-heavy-feature.md`: planted plan with explicit UI scope keywords for the new design-with-UI test. |
| 95 | +- Auto-handling of the workspace-trust dialog so tests run in temp directories without manual intervention. |
| 96 | +- Outcome contract: `asked` | `plan_ready` | `silent_write` | `exited` | `timeout`. Tests pass on `asked` or `plan_ready`, fail on the rest. |
| 97 | + |
| 98 | +#### Changed |
| 99 | + |
| 100 | +- 18 preamble resolvers compressed: `generate-ask-user-format.ts`, `generate-brain-sync-block.ts`, `generate-completeness-section.ts`, `generate-completion-status.ts`, `generate-confusion-protocol.ts`, `generate-context-health.ts`, `generate-context-recovery.ts`, `generate-continuous-checkpoint.ts`, `generate-lake-intro.ts`, `generate-preamble-bash.ts`, `generate-proactive-prompt.ts`, `generate-routing-injection.ts`, `generate-telemetry-prompt.ts`, `generate-upgrade-check.ts`, `generate-vendoring-deprecation.ts`, `generate-voice-directive.ts`, `generate-writing-style-migration.ts`, `generate-writing-style.ts`. |
| 101 | +- All 47 generated `SKILL.md` files regenerated; 3 ship golden fixtures regenerated. |
| 102 | +- Plan-* skills retain full preamble surface (Brain Sync, Context Recovery, Routing Injection) — the early slim attempt that cut these was reverted after diagnosing them as load-bearing. |
| 103 | +- 5 existing plan-mode tests (`plan-ceo`, `plan-eng`, `plan-design`, `plan-devex`, `plan-mode-no-op`) rewritten onto the new harness with a 300s observation budget. All 5 verify-pass under `EVALS=1 EVALS_TIER=gate` against the real `claude` binary in 790s sequential. |
| 104 | +- `isNumberedOptionListVisible` regex tolerates whitespace collapse from TTY cursor-positioning escapes (`\x1b[40C`) which `stripAnsi` removes — `\b2\.` was failing on word-to-word transitions where stripped output read `text2.`. |
| 105 | + |
| 106 | +#### Fixed |
| 107 | + |
| 108 | +- `scripts/skill-check.ts`: new `isRepoRootSymlink()` helper so dev installs that mount the repo root at `host/skills/gstack` (e.g., codex's `.agents/skills/gstack`) get skipped instead of double-counted. |
| 109 | +- `test/skill-validation.test.ts`: known-large-fixture exemption keeps `browse/test/fixtures/security-bench-haiku-responses.json` (27 MB BrowseSafe-Bench replay fixture, intentional) out of the size warning. |
| 110 | + |
| 111 | +#### Removed |
| 112 | + |
| 113 | +- `test/helpers/plan-mode-helpers.ts`: superseded by `claude-pty-runner.ts`. Zero callers remained after the rewrite. |
| 114 | + |
| 115 | +#### For contributors |
| 116 | + |
| 117 | +- `test/helpers/touchfiles.ts`: 5 plan-mode test selections + e2e-harness-audit selection now point at `claude-pty-runner.ts` instead of the deleted helper. 6 new entries (`ask-user-question-format-pty`, `plan-ceo-mode-routing`, `plan-design-with-ui-scope`, `budget-regression-pty`, `ship-idempotency-pty`, `autoplan-chain-pty`) with tier classifications: 3 gate, 3 periodic. |
| 118 | +- `test/e2e-harness-audit.test.ts`: recognizes `runPlanSkillObservation` as a valid coverage path alongside the legacy `canUseTool` / `runPlanModeSkillTest` patterns. |
| 119 | +- New unit test: `test/gen-skill-docs.test.ts` asserts plan-review preambles stay under 33 KB and the slim Voice section preserves its load-bearing semantic contract (lead-with-the-point, name-the-file, user-outcome framing, no-corporate, no-AI-vocab, user-sovereignty). |
| 120 | +- `test/touchfiles.test.ts`: skill-specific change selection count updated 15 → 18 to match the 6 new touchfile entries that depend on `plan-ceo-review/**`. |
42 | 121 |
|
43 | 122 | ## [1.14.0.0] - 2026-04-25 |
44 | 123 |
|
@@ -66,6 +145,7 @@ The old chat queue is gone. `sidebar-agent.ts`, `/sidebar-command`, `/sidebar-ch |
66 | 145 | ### Itemized changes |
67 | 146 |
|
68 | 147 | #### Added |
| 148 | + |
69 | 149 | - **Interactive Terminal sidebar tab.** xterm.js + a non-compiled `terminal-agent.ts` Bun process that spawns claude with `Bun.spawn({terminal: {rows, cols, data}})`. Auto-connects when the side panel opens, no keypress needed. |
70 | 150 | - **`$B tab-each <command>`** — fan-out helper for multi-tab work. Returns `{command, args, total, results: [{tabId, url, title, status, output}]}`. Skips chrome:// pages, scope-checks the inner command before iterating, restores the original active tab in a `finally` block, never pulls focus away from the user's foreground app. |
71 | 151 | - **Live tab state files.** `<stateDir>/tabs.json` (full list with id, url, title, active, pinned, audible, windowId) and `<stateDir>/active-tab.json` (current active). Updated atomically on every `chrome.tabs` event (activated, created, removed, URL/title change). Claude reads on demand instead of running `$B tabs`. |
|
0 commit comments