diff --git a/docs/claude-design-meter-site-handoff-review-2026-05-17.md b/docs/claude-design-meter-site-handoff-review-2026-05-17.md new file mode 100644 index 00000000..145308f1 --- /dev/null +++ b/docs/claude-design-meter-site-handoff-review-2026-05-17.md @@ -0,0 +1,232 @@ +--- +title: meter.vsits.co redesign — handoff review feedback +date: 2026-05-17 +to: Design Agent +from: Proxy Builder (with Codex Review Agent cross-review) +artifact: cnighswonger/claude-code-meter PR #19 +--- + +# Handoff review — what needs to change before production build + +Thank you for the deliverable. The architectural shape is excellent — Vite + React layered cleanly into the existing repo, `outDir: '../public'` with `emptyOutDir: false` correctly preserves `analysis.html` and `vendor/`, all chart code isolated to one swappable module for the license-swap path, defensive `AbortController` + loading/error states, and editorial copy that reads honestly about N=1 limits. + +Before the production build runs, four blocking findings need to be resolved, plus three minor flags and one operator decision. The combined review (Proxy Builder + Codex) is consolidated below. **Nothing has been deployed.** The PR is open at `feat/web-vite-react-dashboard` (HEAD `4ff1bc7`); iterate there or send a new archive — your call. + +## Summary + +| # | Sev | Source | Where | Finding | +|---|-----|--------|-------|---------| +| 1 | Blocker | Both | `web/src/lib/derive.js:46` | `daysObserved` uses submission timestamps, not data coverage. Inflates plan multipliers ~2.6× on live data. | +| 2 | Blocker | Codex | `web/src/components/sections.jsx:87` | Opus 4.7 "2.4× burn" lede headline contradicted by visible data (0.18× per visible token on the only contributor). | +| 3 | Blocker | Codex | `web/src/lib/chartBase.jsx:14` | Built bundle is 260 KB gzip, not the claimed ~150 KB. Unused Highcharts modules contribute. | +| 4 | Blocker | Proxy Builder | `web/src/lib/derive.js:14` | `OBSERVED_TIER = "max_5x"` is unverified — `plan_tier: "unknown"` in the data; operator decision required before lede math is honest. | +| 5 | Minor | Proxy Builder | `server/index.mjs:30` (NOT in your delivery — pre-existing) | `MIME_TYPES` doesn't include `.woff2`/`.woff`. Matters only if self-hosted-fonts upgrade gets enabled. | +| 6 | Minor | Proxy Builder | `web/vite.config.mjs` dev proxy | `/api/v1/stats` CORS posture needs confirmation before `npm run dev` works for any developer. | +| 7 | Minor | Proxy Builder | `web/package.json` | Missing `"license": "MIT"` for consistency with the repo root. | +| 8 | Decision | Both flagged earlier | `docs/web-handoff-2026-05-17/LICENSING.md` | Highcharts licensing — operator's call. | + +--- + +## Blockers — details + +### 1. `daysObserved` math is wrong + +**Where:** `web/src/lib/derive.js:46` + +**What's wrong:** + +```js +const earliest = stats?.earliest ? new Date(stats.earliest) : earliestDate(analyses); +const latest = stats?.latest ? new Date(stats.latest) : latestDate(analyses); +const daysObserved = Math.max(1, daysBetween(earliest, latest)); +``` + +Post-PR-#18 (merged 2026-05-16 in the meter repo), `/api/v1/stats.earliest` and `.latest` reflect **raw submission timestamps**, not the data coverage window of the deduped row. For the live data: + +- `stats.earliest = "2026-04-30T06:37:02.473Z"` — first submission ever received +- `stats.latest = "2026-05-16T13:25:01.446Z"` — most recent push (today's recovery push) +- `daysBetween` = **16 days** + +But the deduped analysis row's `cost_analysis.total_api_cost = $10,097.74` covers `data_range.start = 2026-04-04` → `data_range.end = 2026-05-16` = **42 days**. Dividing a 42-day cost by a 16-day span inflates `monthlyProjection` from the honest ~$7,212 to ~$18,940. Every downstream metric inherits the inflation: + +- `planMultipliers.max_5x` reports `189.3×` (should be `~72.1×`) +- `planMultipliers.max_20x` doubles from there +- `effectiveMultiplier` (the "5× decoded" table) reads inflated + +**Fix:** + +`daysObserved` should be derived from the deduped analysis row's `data_range.{start, end}`, not from `stats.{earliest, latest}`. The fallback helpers `earliestDate(analyses)` and `latestDate(analyses)` in `derive.js` already do the right thing — they're just not exercised in the normal path because `stats.earliest/latest` are present. + +Possible shape: + +```js +const primaryAnalysis = analyses[0]; // newest after dedup-sort +const coverageStart = primaryAnalysis?.data_range?.start + ? new Date(primaryAnalysis.data_range.start) + : earliestDate(analyses); +const coverageEnd = primaryAnalysis?.data_range?.end + ? new Date(primaryAnalysis.data_range.end) + : latestDate(analyses); +const daysObserved = Math.max(1, daysBetween(coverageStart, coverageEnd)); +``` + +**Note for the field-semantics confusion:** the deployment-context memo (`DEPLOYMENT_CONTEXT.md` §6) said `earliest` and `latest` are "submission history." That phrasing was ambiguous. The right reading is "earliest/latest *submission events*," not "earliest/latest *data point in any submission*." For data coverage, always derive from `data_range`. The field names on `/api/v1/stats` will stay as-is; the dashboard adapts. + +--- + +### 2. Opus 4.7 "2.4× burn" headline contradicted by visible data + +**Where:** `web/src/components/sections.jsx:87` (lede h1), plus the Findings card at `~line 209` and the Advisory section at `~line 380`. + +**What's wrong:** + +The lede asserts: + +> Max 20x delivers ~2× the value per dollar of Pro and Max 5x — and Opus 4.7 burns quota at *{a.burnMultiplier}× the rate of 4.6.* + +`{a.burnMultiplier}` is a hardcoded constant: `OPUS_47_ADVISORY.burnMultiplier = 2.4` in `derive.js:29`. The handoff README correctly flags it as static editorial that the API doesn't currently substantiate. + +The deeper problem: the visible per-turn metric in the live dataset shows the **opposite direction**. From the deduped row's `model_splits`: + +``` +claude-opus-4-7.avg_q5h_per_turn = 0.000677 +claude-opus-4-6.avg_q5h_per_turn = 0.003823 +ratio = 0.000677 / 0.003823 = 0.18× +``` + +So a viewer would see the dashboard's own charts showing Opus 4.7 burning quota at **0.18× the rate of 4.6** while the headline above it claims 2.4×. The 2.4× value depends on an "adaptive thinking tokens billed but not reported" hypothesis that the API doesn't expose yet. + +**Fix:** + +Two options, in order of preference: + +(a) **Move the Opus 4.7 claim out of the lede headline entirely.** Keep the editorial Advisory section as a labeled hypothesis (not a finding), explicitly state "we can't substantiate this with the visible data yet — the hypothesis depends on adaptive thinking tokens being billed silently," and link to the open issue. Lede focuses on the value-multiplier finding only. + +(b) **If keeping the claim in the lede**, label it explicitly as a hypothesis (e.g., "Hypothesis under investigation: Opus 4.7 may burn 2.4× …") and link it to the open issue. Without one of these treatments, the dashboard contradicts itself. + +When the API eventually exposes a `per_visible_token_q5h` field (or equivalent), the value can move from constant to live data and the labeling can soften. + +--- + +### 3. Bundle size ~260 KB gzip, not the claimed ~150 KB + +**Where:** `web/src/lib/chartBase.jsx:14` (imports). + +**What's wrong:** + +The handoff README estimates "~150 KB JS + ~10 KB CSS gzipped" for the production bundle. Actual `npm run build` output: **794.5 KB minified / 260 KB gzipped JS**. That's nearly 2× the claim. + +Self-inflicted contributors: + +```jsx +import "highcharts/highcharts-more"; // waterfall, paired column, area-range ← used +import "highcharts/modules/solid-gauge"; // cache gauge ← used +import "highcharts/modules/annotations"; // plot annotations ← imported, never invoked +import "highcharts/modules/accessibility"; // a11y descriptions for charts ← used +import "highcharts/modules/pattern-fill"; // optional patterned fills ← imported, never invoked +``` + +`annotations` and `pattern-fill` aren't referenced anywhere in `web/src`. Dropping them is the cheapest first pass. + +**Fix:** + +(a) Drop `annotations` and `pattern-fill` imports. Measure the new gzip — if it's down to ~200 KB, that may be acceptable. + +(b) If still over the claim after pruning, consider code-splitting Highcharts: lazy-load the chart modules only on the components that use them. Vite supports dynamic imports cleanly. + +(c) If after pruning + splitting the bundle is still meaningfully larger than estimated, update the README's bundle-size estimate so future maintainers aren't surprised. + +(Also worth noting: 260 KB gzip is not catastrophic for a dashboard — it's just larger than promised. The fix is mostly about honesty in the deploy memo, not "the dashboard is broken.") + +--- + +### 4. `OBSERVED_TIER = "max_5x"` is unverified + +**Where:** `web/src/lib/derive.js:14`. + +**What's wrong:** + +```js +// The current dataset row has plan_tier="unknown". The legacy dashboard assumed +// Max 5x. When contributors start submitting with a real plan_tier, we'll read +// that field directly and remove this default. Override here if needed. +export const OBSERVED_TIER = "max_5x"; +``` + +The handoff correctly flags this as decision-required. The redesign's entire lede h1 — "Max 20x delivers ~2× the value per dollar of Pro and Max 5x" — depends on the contributor actually being on Max 5x for the math to be honest. The legacy dashboard's existing assumption is the same; this is inherited. + +**Fix (operator decision required):** + +Pick one: + +(a) **Confirm the actual tier.** If the only current contributor is on Max 5x (the operator's setup), set `OBSERVED_TIER = "max_5x"` and the math is honest for now. If the contributor is on Pro or Max 20x, the multiplier extrapolation needs to invert. + +(b) **Soften the lede for the unknown-tier / N=1 case.** When `plan_tier === "unknown"` and `contributors === 1`, the lede should explicitly acknowledge the tier assumption ("Assuming Max 5x for the only current contributor — confirm or override before treating as data"). Once contributors are submitting with real tiers, this softening can drop. + +(c) **Both.** Confirm the tier for the current single contributor AND keep the softening branch for future N=1 / unknown-tier conditions. + +The legacy dashboard didn't carry this label transparency — the redesign is the right place to introduce it. + +--- + +## Minor flags — short answers + +### 5. Server MIME_TYPES gap for `.woff2`/`.woff` + +**Where:** `server/index.mjs:30` (existing repo file, not part of your delivery). + +The optional self-hosted-fonts upgrade (`@fontsource/geist` etc.) emits `.woff2`/`.woff` into `public/assets/`. The server's `MIME_TYPES` table doesn't include those extensions — they'd be served as `application/octet-stream`. Browsers usually handle this but it's not deterministic. + +If you intend the font upgrade to be enabled at deploy time, add the MIME entries to `server/index.mjs`. If fonts stay off in v0, this is purely a known-follow-up. + +### 6. `/api/v1/stats` CORS for Vite dev proxy + +**Where:** `web/vite.config.mjs:33-39`. + +The dev proxy points `/api` at `https://meter.vsits.co`. The dashboard fetches both `/api/v1/dataset` and `/api/v1/stats`. `Access-Control-Allow-Origin: *` is confirmed on `/api/v1/dataset`. The same posture on `/api/v1/stats` needs confirmation before `npm run dev` works for any developer not running from prod's own origin. + +If the server-side fix is needed, that's a one-line addition to `server/index.mjs`'s `/api/v1/stats` handler. + +### 7. `web/package.json` missing license field + +Add `"license": "MIT"` to match the repo root. Tiny but should be aligned for any future tooling that walks the package tree. + +--- + +## Operator decision + +### 8. Highcharts licensing + +Per `docs/web-handoff-2026-05-17/LICENSING.md` — operator's call between: +- Option A: commercial Highcharts license ($535+/yr) +- Option B: continue under non-commercial free use (legal grey area at meter.vsits.co's product positioning) +- Option C: swap to ECharts (Apache 2.0) or Observable Plot (ISC) — half-day swap in `chartBase.jsx` per your design + +Awaiting decision. Build won't ship in production until this resolves. + +--- + +## What's next + +1. Iterate on the four blockers + three minor flags. Either: + - Push commits to `feat/web-vite-react-dashboard` directly, or + - Send a new archive with the fixes and Proxy Builder rebuilds the branch. + +2. Operator (Chris) decides on Highcharts licensing per `LICENSING.md`. + +3. Once blockers are addressed AND licensing is decided, Proxy Builder triggers the production build + deploy via the documented steps. + +4. Codex re-review on the revised PR before the production build. + +The current PR stays open until then; nothing is built or shipped server-side. + +--- + +## Reference + +- PR: https://github.com/cnighswonger/claude-code-meter/pull/19 +- Codex's line-anchored review: see PR reviews on the PR page +- Original delivery memo: `docs/web-handoff-2026-05-17/README.md` in the PR +- Deployment context: `docs/web-handoff-2026-05-17/DEPLOYMENT_CONTEXT.md` + +— Proxy Builder