Skip to content

Latest commit

 

History

History
1476 lines (1223 loc) · 370 KB

File metadata and controls

1476 lines (1223 loc) · 370 KB

Phase 1 — Launch Product

Goal: Ship the LLM Process Visualizer as defined in SPEC.md. All 9 vendors, persistent threads, real-time visualization, replay, export (JSON + GIF), opt-in thread sharing, deployed to DreamHost VPS.

Status: Not started. Source of truth for scope: SPEC.md §2.1 (in scope) and §2.2 (out of scope).

This document breaks Phase 1 into 15 milestones (M1–M15). Each milestone lists its purpose, concrete tasks, dependencies, exit criteria, and a rough effort estimate.

Note on estimates. Time estimates are in "engineer-days" assuming one person working full-time and familiar with the stack. They are planning aids, not commitments.


Milestone overview

# Milestone Depends on Estimate Status Vertical-slice gate?
M1 Foundation 3 days ✅ Complete
M2 Auth + Users M1 4 days ✅ Complete
M3 Model Registry M2 4 days ✅ Complete
M4 API Keys + Vendor Clients M2 12 days ✅ Complete
M5 Threads + Runs (data) M3, M4 4 days ✅ Complete
M6 Realtime + Streaming Pipeline M5 6 days ✅ Complete
M7 Frontend — Static UI M5 7 days ✅ Complete
M8 Frontend — Live Visualization M6, M7 12 days ✅ Complete (superseded by M13) ✅ End-of-M8 = vertical slice
M9 Replay + JSON Export M8 4 days ✅ Complete
M10 GIF Export M8 6 days ✅ Complete
M11 Thread Sharing M9 3 days ✅ Done
M12 Accessibility + Polish M11 5 days ✅ Done
M13 Cinematic Inference Visualization M12 12 days ✅ Done Replaces the M8 live viz with a 20-scene narrative
M14 Deployment M13 5 days ⚪ Not started
M15 Launch Prep M14 5 days ⚪ Not started
Total ~80 engineer-days

M1 — Foundation

Purpose: Empty repo → working Laravel + React + Vite dev environment with CI and linters.

Tasks

  • composer create-project laravel/laravel in repo root (Laravel 13.9.0, upgraded from initial Laravel 11 scaffold).
  • Switch DB driver to SQLite; create database/database.sqlite (SQLite is the default in Laravel 13; DB file is gitignored per-environment).
  • Add .editorconfig, .gitignore (Laravel + Node), .gitattributes (Laravel 13 defaults + project-specific extensions for SQLite/Sentry/OS files).
  • Install Inertia.js Laravel adapter (inertiajs/inertia-laravel ^3.1) + React preset (TypeScript): React 19, @inertiajs/react ^2, TypeScript, @types/react/@types/react-dom/@types/node, @vitejs/plugin-react ^5 (pinned to v5 for Vite 6 compat). Ziggy installed too (tightenco/ziggy ^2.6 + ziggy-js) for Laravel-named-routes-in-JS.
  • Install Tailwind CSS 3 + configure tailwind.config.js (Tailwind 3.4 already shipped in the skeleton; content paths updated to include .{ts,tsx} files).
  • Configure Vite for React + Inertia. vite.config.jsvite.config.ts; react() plugin added; input updated to resources/js/app.tsx; @/* alias added pointing to resources/js/.
  • Set up Laravel Pint config: pint.json with laravel preset + concat_space, ordered_imports, no_unused_imports, global_namespace_import. All existing files auto-formatted to match.
  • Set up ESLint + Prettier for JS/TS. ESLint 9 flat config (eslint.config.js) with typescript-eslint recommended + eslint-plugin-react + eslint-plugin-react-hooks + eslint-plugin-jsx-a11y (a11y rules support WCAG goals) + eslint-config-prettier to disable conflicting style rules. Prettier config in .prettierrc.json (4-space, single-quote, trailing commas, 100-col; YAML overridden to 2-space). Markdown files added to .prettierignore to avoid table-alignment churn on edits.
  • Install Husky + lint-staged for pre-commit hooks. .husky/pre-commit runs npx lint-staged (Pint on PHP, ESLint+Prettier on TS/JS/CSS/JSON/YAML/HTML) followed by npm run type-check (TS errors caught at commit time, not just CI).
  • Install Pest for PHP testing (Pest 4.7, replacing the default PHPUnit ExampleTests). New tests/Pest.php base config + tests/Feature/WelcomePageTest.php covering Inertia round-trip. PHPUnit downgraded to 12.5.24 for Pest compat. pest-plugin-laravel skipped — its Laravel 13 support hasn't shipped and most of its features are absorbed into Pest core anyway.
  • Extend .env.example with project-specific keys (Socialite Google/MS/FB, Pusher-protocol broadcast, OpenRouter, Sentry DSN, GIF renderer, repo URL for AGPL §13, default rate limit). All values blank or sensible defaults; production overrides happen in .env.
  • GitHub Actions workflow: .github/workflows/lint.yml (Pint, ESLint, Prettier, TypeScript) + .github/workflows/test.yml (Pest + Vitest + React Testing Library). Both pin PHP 8.4 + Node 20, cache Composer + npm, and use concurrency to cancel superseded runs. Initial PHP version was 8.3 per the SPEC, but Laravel 13 ships with Symfony 8.x which requires PHP 8.4+ — spec corrected accordingly. Vitest + RTL + jsdom installed for frontend testing; resources/js/Pages/__tests__/Welcome.test.tsx provides the first 3 passing tests. Cypress E2E deferred to a later milestone (no real UI to drive yet).
  • Install Sentry Laravel SDK + frontend SDK. Backend: sentry/sentry-laravel ^4.25 with config/sentry.php published; Integration::handles($exceptions) registered in bootstrap/app.php for unhandled-exception capture (Laravel 11+ requirement). Frontend: @sentry/react with Sentry.init() gated on VITE_SENTRY_DSN (no-op when blank — local dev stays silent and makes no network calls); Sentry.ErrorBoundary wraps the Inertia App with a <ErrorFallback /> UI. Vite env types extended in resources/js/types/global.d.ts so import.meta.env.VITE_SENTRY_* is type-checked. Verified: artisan boots, npm run build succeeds (~411 KB JS), Pest still passes.
  • Verify npm run dev and php artisan serve both work and Inertia round-trips between PHP and React. Verified end-to-end: npm run build produces a clean bundle; php artisan serve + curl / returns HTTP 200 with the Inertia data-page attribute embedded, component: "Welcome", page props serialized (Laravel 13.9.0, PHP 8.4.21), and Ziggy route data injected.

Exit criteria

  • php artisan serve + npm run dev boot cleanly. Verified during chunk 2.
  • A Welcome Inertia page renders React content. Tested by tests/Feature/WelcomePageTest.php (Pest, 2 tests / 6 assertions) and resources/js/Pages/__tests__/Welcome.test.tsx (Vitest, 3 tests).
  • CI passes on a no-op PR. Verified by the lint + test workflows added in chunk 5; first runs will exercise once a PR lands.
  • Pre-commit hook blocks unformatted code. Verified live during chunks 3 and 4 — every chunk-closing commit ran lint-staged + type-check before landing.

M1 closed: 2026-05-17. Chunks 1–5 all green. Estimate was 3 engineer-days; actual was substantially less (single afternoon) but the user supplied a working environment and clear answers throughout, so it's not a representative pace for the rest of Phase 1.

Things that needed follow-up patches after the initial chunk-5 push:

  • CI PHP version: spec said PHP 8.3+ but Laravel 13 ships with Symfony 8.x which actually requires PHP ^8.4. CI bumped to 8.4 and the spec corrected (SPEC.md, README.md updated). Local dev was already on 8.4 so no functional change.
  • tests/Unit/ lost from git when we deleted the only file in chunk 3 (git doesn't track empty dirs). Added a .gitkeep so the directory survives fresh checkouts; phpunit.xml still references both Unit and Feature testsuites.

Known follow-ups not blocking M1:

  • GitHub Actions Node 20 deprecation: actions/checkout@v4, actions/setup-node@v4, actions/cache@v4 all use Node 20, which GitHub flips to Node 24 default on 2026-06-02 and removes on 2026-09-16. Bump to v5 of each action when they're available (not yet at time of M1 close).

M2 — Auth + Users

Purpose: Users can sign in via Google/Microsoft/Facebook. Admin can promote users. Rate limits enforce.

Tasks

  • Migration: users (per SPEC §6, no password column, including max_runs_per_hour, store_prompts, avatar_url, role). Edited the Laravel-default migration (greenfield, no audit-trail concerns yet). password_reset_tokens table dropped from the migration (not needed under social-only auth). PHP 8.1 enum App\Enums\UserRole with User + Admin cases; Eloquent casts the role column to it. User model gains socialAccounts() HasMany + isAdmin() helper. UserFactory updated with admin() and privacyOpted() states.
  • Migration: social_accounts with (provider, provider_user_id) unique constraint, (user_id, provider) index, and cascadeOnDelete from users. SocialAccount model + SocialAccountFactory with google() / microsoft() / facebook() states.
  • Install Laravel Sanctum (laravel/sanctum ^4.3). Config published, personal_access_tokens migration applied. Not used in Phase 1 — kept in place for future API-token use cases per SPEC.
  • Install Laravel Socialite (^5.27) + socialiteproviders/microsoft (^4.9). Microsoft provider registered via Event::listen(SocialiteWasCalled::class, MicrosoftExtendSocialite::handle) in AppServiceProvider::boot(). Google + Facebook ship in Socialite core. config/services.php extended with all three providers (env-driven; blank vars = provider button shown but redirect would fail with clear error).
  • Implement OAuth flow for Google, Microsoft, Facebook with callback handler. App\Http\Controllers\Auth\SocialiteController exposes redirect() / callback() / logout(). Routes: GET /auth/{provider}/redirect, GET /auth/{provider}/callback, POST /logout. Provider allowlist (['google', 'microsoft', 'facebook']) gates both endpoints with 404 on unknown providers.
  • On callback: upsert user by provider+provider_user_id, log them in via session auth, redirect to dashboard. Email auto-link behavior (decided 2026-05-17): if no matching social_accounts row but the OAuth email matches an existing user, attach a new social_accounts row to that user instead of creating a duplicate account. email_verified_at set to now() on first creation (providers verify emails before issuing OAuth tokens). Avatar backfill: if the existing user has no avatar_url and the new provider supplies one, save it; never overwrite an existing avatar.
  • Login page (Inertia React): three large provider buttons. resources/js/Pages/Login.tsx is the dedicated sign-in page with branding, an error banner that reads errors.social from Inertia's auto-shared errors prop, and a "back to home" link. The OAuth callback failure path in SocialiteController::callback() now redirects to route('login') (instead of route('home')) so the error displays where the user came from. Vitest covers button presence, route wiring, back link, and no-error default state.
  • Logout endpoint + UI. POST /logout was wired in chunk 2; chunk 3 adds the visible UI in AppLayout.tsx (logout form button in the sidebar footer). Dashboard's standalone logout button removed in favor of the layout-level one.
  • Artisan command: php artisan user:promote {email}. Implemented as App\Console\Commands\PromoteUser using Laravel 13 attribute-based signature/description. Three behaviors: promotes a user-role user → admin, no-ops with clear message if already admin, fails with non-zero exit + hint ("the user must sign in at least once") if no user matches. 3 Pest tests cover all branches.
  • Rate-limit middleware: per-user, per-hour, backed by Laravel's RateLimiter facade. Registered as a named limiter runs in AppServiceProvider::boot() — the closure reads users.max_runs_per_hour live on every request so admin edits take effect immediately. Routes attach via middleware('throttle:runs'). The actual run-submission route lands in M5/M6; for now the limiter is verified via a test-only stub route. X-RateLimit-Limit and X-RateLimit-Remaining headers are set by Laravel's ThrottleRequests middleware automatically. Note: Laravel's Limit::perHour is a fixed-hour-bucket scheme, not a true sliding window — close enough for our use case; can swap to a true sliding implementation later if user-visible behavior on the bucket boundary becomes a problem.
  • Admin UI shell (gated by role = admin): user list with inline editor for max_runs_per_hour. App\Http\Controllers\Admin\UsersController@index paginates 20-per-page (User::orderByDesc('created_at')); update validates max_runs_per_hour as integer 0–10000 then redirects back with a rate-limit-updated:{user_id} flash status. Frontend at resources/js/Pages/Admin/Users.tsx renders a table with an inline <RateLimitForm> per row (number input + Save button + ✓ indicator on save). Routes gated by the existing admin alias from chunk 3.
  • Privacy toggle on user settings: store_prompts checkbox. App\Http\Controllers\SettingsController@show renders the page with the current value; update validates as boolean and redirects back with settings-saved flash status. Frontend uses Inertia's useForm with explicit Save button (not auto-save). Flash status read via the new flash.status shared prop (added to HandleInertiaRequests::share()).

Exit criteria

  • [~] Sign in works for all 3 providers (verified manually with real OAuth apps in dev mode). Verified at the mock level: tests/Feature/Auth/SocialiteCallbackTest.php (10 tests) exercises the full callback flow for each provider via Socialite's mockable contracts — user creation, existing-link login, email auto-link, avatar backfill, logout, guest gating. Real-credential verification deferred until the user provisions OAuth apps with Google/Microsoft/Facebook and populates .env. The .env.example keys are in place from M1 chunk 3.
  • Promoting a user via user:promote flips their role and unlocks /admin routes. Verified by tests/Feature/Console/PromoteUserCommandTest.php (3 tests, including the user-not-found error case) combined with tests/Feature/Auth/AuthGatedRoutesTest.php ("returns 403 when a non-admin hits /admin/users" + "renders the Admin Users page for an admin").
  • 31st request within an hour returns 429. Verified by tests/Feature/RateLimit/RunsRateLimiterTest.php ("returns 429 when the per-user limit is exceeded"). The actual test uses max_runs_per_hour = 3 for speed; the assertion is identical at any limit, including the default 30.

M2 closed: 2026-05-17. Chunks 1–5 all green. The single 🟠 marker on the manual-OAuth criterion is the only thing standing between M2 and a full ✅, and it's gated on the operator provisioning real OAuth credentials — not on any code work.

M2 retrospective notes:

  • Real value of eslint-plugin-jsx-a11y: caught a <label> without explicit htmlFor association on the Settings page during chunk 4. Cost ~30 seconds to fix; would otherwise have surfaced in the WCAG audit at M12.
  • Vite manifest gotcha (twice): adding new Inertia pages without running npm run build produces "Unable to locate file in Vite manifest" 500s in Pest tests. CI handles this (test workflow runs npm run build before Pest); local doesn't. Not adding a guard, but worth knowing.
  • Inertia page glob picked up a test file: in chunk 2 the path Pages/__tests__/Welcome.test.tsx got bundled into the production build via the page-resolver glob — 457 KB of Vitest leaked into prod. Moved tests to resources/js/__tests__/Pages/ (outside the glob). Worth a CI guard later if this happens again.
  • Auto-link by email decision (chunk 2): users with the same email across providers now share one account. Acceptable since Google/MS/FB all verify emails before issuing OAuth tokens. If we see a single phishing/takeover report, revisit by adding an explicit "link account" workflow.
  • Sliding-window vs fixed-bucket rate limiting (chunk 4): SPEC said sliding, used fixed-hour-bucket via Laravel's built-in. Documented; swap if bucket-boundary unfairness becomes a real complaint.

Stats: Pest 52 tests / 147 assertions, Vitest 8 tests. All CI green.


M3 — Model Registry

Purpose: Models table populated from OpenRouter weekly, with architecture metadata enriched from a local fixture. Admin can edit and override.

Tasks

  • Migration: models per SPEC §6 — plus manual_override (refresh-skip flag) and metadata_estimated (best-guess marker for closed-source layer counts). Architecture metadata columns (architecture_type, layers, hidden_dim, attention_heads, moe_experts, moe_active_experts, position_encoding) are all nullable since OpenRouter doesn't expose them — they're populated by the fixture or admin edits in chunks 2 and 4. PHP 8.1 enums App\Enums\ArchitectureType and App\Enums\PositionEncoding provide strong typing; Eloquent casts both columns. Eloquent model named App\Models\LlmModel (with explicit $table = 'models') to avoid shadowing Illuminate\Database\Eloquent\Model. Factory + states (moe(), vendor(), estimated(), manuallyOverridden()).
  • Migration: registry_meta (key/value; tracks last_successful_refresh_at). String-primary-key key/value table with JSON value column and updated_at only (no created_at — meta entries are upserted, not "created"). App\Models\RegistryMeta exposes getValue($key), setValue($key, $value), and forget($key) static helpers. setValue uses the Query Builder's updateOrInsert rather than Eloquent's updateOrCreate so identical-value writes still bump updated_at (otherwise the dirty-checking in save() would skip the write and break "last refresh time" tracking).
  • OpenRouterClient (Guzzle): fetchModels(): array. Implemented in App\Services\OpenRouter\OpenRouterClient using Laravel's Http facade (Guzzle under the hood). Normalizes each row: splits the vendor/model id, converts per-token pricing strings to per-million-token floats, extracts display_name and context_length. Skips rows with unparseable ids (missing slash, null/non-string id). Throws RuntimeException on HTTP failure or missing data array. Reads base URL from config('services.openrouter.base_url'), defaulting to https://openrouter.ai/api/v1. Constructor optional $baseUrl override for tests.
  • Fixture file: database/seeders/data/architecture_metadata.php. Keyed by models.name (post-slash portion of OpenRouter id). Each entry carries architecture_type, layers, hidden_dim, attention_heads, moe_experts/moe_active_experts (where applicable), position_encoding, display_name override, capability flags, and metadata_estimated (true for closed-source vendors, false for open-weights). All 10 Phase 1 launch-set models present per SPEC §7. Open-weights (Llama 3.1 70B/405B, Mixtral 8x22B) have real architecture data; closed-source (GPT-4o, Claude 3.5, Gemini 1.5 Pro, Grok 2, Mistral Large) are flagged estimated with rope position encoding as a sensible default.
  • Service: ModelRegistryRefreshService — joins OpenRouter + fixture, upserts, respects manual_override. Implemented in App\Services\ModelRegistry\RefreshService with a small RefreshResult value object. Logic: fetch upstream, load fixture, for each row check for existing match by name, skip if manual_override = true, else merge OpenRouter base with fixture entry (fixture wins on overlap), upsert. Wraps everything in a DB transaction so a catastrophic failure rolls back the partial state; per-row exceptions are caught and recorded in RefreshResult.errors so one bad row doesn't kill the whole refresh. On success, writes last_successful_refresh_at and last_refresh_summary to registry_meta. Constructor takes the client; refresh(?$fixturePath) accepts an override path for tests.
  • Artisan command: php artisan registry:refresh. Implemented as App\Console\Commands\RegistryRefresh using Laravel 13 attribute-based signature. Accepts --fixture= to override the architecture metadata path (for tests / staging). Outputs the RefreshResult::summary() string on success and any per-row errors. Exit code 0 on success, 1 on uncaught RuntimeException from the service.
  • Scheduled task (Laravel scheduler): weekly, with admin-email notification on failure. Registered in routes/console.php (Laravel 11+ pattern, replacing the old Kernel::schedule). Runs weeklyOn(1, '03:00') — Monday 03:00 UTC, intentionally off-peak. ->onFailure(...) callback queries User::where('role', UserRole::Admin) and sends App\Notifications\RegistryRefreshFailed to all admins via the mail channel. With MAIL_MAILER=log in dev, failure notifications appear in storage/logs/laravel.log rather than emailing anyone.
  • Seeder for initial registry: Database\Seeders\ModelRegistrySeeder calls RefreshService on php artisan migrate --seed. Resilient: catches network failures with a clear warning rather than aborting the seed. Short-circuits in the testing env so Http::fake() controls test fixtures without the seeder interfering. Wired into DatabaseSeeder::run().
  • Admin UI: model CRUD with manual_override toggle. App\Http\Controllers\Admin\ModelsController provides index / edit / update / destroy / refresh. Index has search (name + display_name), vendor filter, architecture filter, paginated at 25, with badge indicators for manual_override and metadata_estimated. Edit form covers all admin-editable fields grouped into Identity / Architecture / Capacity+pricing / Capabilities / Chat template / Refresh control sections (skipping supported_params JSON — too low-frequency to merit a UI for now). Validation: vendor required, architecture_type/position_encoding constrained to enum values, numeric ranges enforced on layers/hidden_dim/heads/MoE/context/pricing. Refresh button posts to admin.models.refresh which invokes RefreshService::refresh() and surfaces success summary or errors.refresh flash. Delete is a hard delete — M5's runs.model_id foreign key will need an ON DELETE strategy when that lands. The placeholder /models route now redirects authenticated admins to /admin/models and keeps a ComingSoon → M7 placeholder for non-admins (public model browser is an M7 deliverable). Sidebar split into Admin · Users and Admin · Models, both admin-only.
  • Staleness banner component: visible to all authed users when last_successful_refresh_at > 14 days ago (or never). Threshold (14 days) is a public constant HandleInertiaRequests::STALENESS_THRESHOLD_DAYS per SPEC §7.1. Shared with every Inertia page via a registry prop (is_stale, days_stale, last_refresh_at). Banner lives in resources/js/Components/RegistryStalenessBanner.tsx and renders inside AppLayout just above the page content — only when stale. Admins see "View registry" + "Refresh now" actions; non-admins see "Ask an admin to run the registry refresh". The SPEC said "model-selector page"; we put it in the layout so the warning is visible on every authed page since the registry feeds cost estimates app-wide. When M7 ships the public model selector, it can either rely on the layout banner or add a more prominent variant.

Exit criteria

  • [~] php artisan registry:refresh populates ≥ 50 models with vendor, pricing, context length. Verified at the mock level: RefreshServiceTest exercises the full create/update/skip cycle against Http::fake() responses. Real ≥ 50-model count deferred until first production run hits the live OpenRouter API (the upstream typically returns 100+ models). The fixture, refresh service, and command are otherwise complete and tested.
  • The launch-set 9+ models have full architecture metadata. database/seeders/data/architecture_metadata.php contains all 10 SPEC §7 launch-set entries. ArchitectureMetadataFixtureTest verifies presence + correctness of metadata_estimated flags + Mixtral 8x22B MoE structure.
  • Editing a model and setting manual_override = true survives the next refresh. Verified by RefreshServiceTest::it skips rows with manual_override = true — pre-existing row with manual_override = true and stale context_length is untouched after the refresh runs. The admin UI's Edit form (chunk 4) is the path admins use to set this flag.

M3 closed: 2026-05-17. Chunks 1–5 all green.

M3 retrospective notes:

  • The RegistryMeta::setValue Eloquent dirty-checking gotcha (chunk 1): updateOrCreate skips DB writes when no attribute is dirty, so repeated identical refreshes wouldn't bump updated_at. Switched to Query Builder's updateOrInsert. This is the kind of subtlety that would have shipped silently and only surfaced months later when "last refresh" data stopped moving.
  • Pages/__tests__/ glob leak revisited (chunk 4): moved 3 new test files (Admin/Models test, registry tests) outside resources/js/Pages/ to keep the M2 chunk-2 lesson holding. Worth adding a CI check that fails the build if any .test.tsx file appears inside Pages/.
  • expectsOutputToContain consumes substring (chunk 3): Laravel 13's PendingCommand consumes the matched substring from the captured buffer, so chained assertions on overlapping content silently fail. Switch to a single longer substring or split into separate tests.
  • $this->seed(SeederClass) wires a mocked OutputStyle (chunk 3): that doesn't expect SymfonyStyle's internal askQuestion() calls from $this->command->info(). Workaround: invoke the seeder directly via app(SeederClass)->run(). The ?-> null-safe operator skips the command-bound output.
  • Staleness banner in AppLayout, not just model-selector (chunk 5): SPEC said model selector (M7 territory). We put it layout-level since registry data feeds cost estimates everywhere. When M7 lands, can re-evaluate placement.

Stats: Pest 121 tests / 329 assertions, Vitest 8 tests. All CI green.


M4 — API Keys + Vendor Clients

Purpose: Per-user encrypted API key storage and a working LlmClientInterface implementation for all 9 vendors.

Tasks

  • Migration: api_keys (encrypted_key, vendor, label). Schema includes user_id (FK with cascadeOnDelete), vendor, nullable label, encrypted encrypted_key (text, via Eloquent's encrypted cast), denormalized last_four (cached plaintext suffix to avoid per-row decrypts on the list view), nullable last_used_at, timestamps. UNIQUE(user_id, vendor, label) per SPEC §6 so a user can hold multiple keys per vendor under different labels. App\Models\ApiKey with $casts['encrypted_key' => 'encrypted'] + booted() event that recomputes last_four on every save when the key is dirty. encrypted_key hidden from serialization. User::apiKeys() HasMany added. ApiKeyFactory with vendor() / withLabel(?string) / withKey() states.
  • API key management UI: list + add + delete; values shown masked except last 4 chars. App\Http\Controllers\ApiKeysController with index / store / destroy — authenticated only, plus a 403 check in destroy so users can't delete each other's keys (even admins are blocked — BYOK trust means admins shouldn't see other users' secrets). Vendor allowlist SUPPORTED_VENDORS (the same 9 from SPEC §3.2.2) gates the validation. Duplicate (vendor, label) returns a friendly label validation error instead of a DB exception. Page at resources/js/Pages/ApiKeys/Index.tsx replaces the chunk-3 ComingSoon placeholder. Form has vendor select / optional label / masked password input. Existing keys table shows masked display (••••XXXX), last_used date, and a Delete button. Flash banners for add/delete. Plaintext key value is never echoed back to the client after creation.
  • Define LlmClientInterface. Parked decision (docs/parked-decisions.md item 2) resolved 2026-05-17 in favor of hand-rolled — Laravel AI SDK doesn't expose logprobs (needed for SPEC §3.1.5 logits panel) and has no custom-vendor extension path; Prism is still pre-1.0. The interface is in app/Services/Llm/Contracts/LlmClientInterface.php with three methods: stream() returns Generator<int, LlmTokenChunk>, complete() returns LlmCompletion, plus a vendor() identifier for factory registration. Slight refinement from the SPEC: takes an ApiKey model + the model string explicitly (rather than relying on globals), and yields/returns typed value objects (LlmTokenChunk, LlmCompletion, LlmUsage) instead of bare arrays. Vendor-specific exceptions (InvalidApiKeyException, VendorRateLimitedException, generic LlmClientException) communicate user-actionable error categories.
  • Implement clients (one per vendor — each is its own sub-task with its own tests):
    • OpenAiClient (SSE streaming, supports logprobs). Lives at app/Services/Llm/Clients/OpenAiClient.php. Streams via POST /v1/chat/completions with stream: true, parses SSE through a reusable App\Services\Llm\Support\SseParser. Requests stream_options.include_usage = true so the final chunk carries token totals. Passes through temperature, top_p, max_tokens, seed; silently drops top_k (not an OpenAI param). When params.logprobs = true, sets top_logprobs = 5 (or override) so the logits panel can show the top-5 alternatives. HTTP error mapping: 401/403 → InvalidApiKeyException, 429 → VendorRateLimitedException (with Retry-After parsed), other non-2xx → generic LlmClientException. api_keys.last_used_at updated on success. Registered with LlmClientFactory in AppServiceProvider::boot(). Designed as the base class for chunk 4's 4 OpenAI-compatible vendors (xAI, Mistral, Groq, Together) — subclasses only override vendor() and defaultBaseUrl().
    • AnthropicClient (event-stream). Bespoke — x-api-key header + anthropic-version: 2023-06-01. System prompts extracted from history into a top-level system field. max_tokens mandatory on Anthropic; defaults to 4096 if caller didn't specify. No logprobs support. Reuses SseParser for the SSE shell; dispatches on the type field inside each event (message_start, content_block_delta, message_delta, message_stop). Tracks cumulative usage across the stream since Anthropic reports input_tokens in message_start and output_tokens in the trailing message_delta.
    • GoogleGeminiClient (streamGenerateContent). Bespoke. Auth via URL query string (?key=…), not a header. Endpoint: /v1beta/models/{model}:streamGenerateContent?alt=sse. Body uses contents+parts shape instead of messages; role assistant mapped to model; system messages extracted into top-level systemInstruction; param naming differs (topP/topK/maxOutputTokens under generationConfig). Reuses SseParser. No logprobs support.
    • XaiClient (OpenAI-compatible). Thin subclass of OpenAiClient — only overrides vendor(), defaultBaseUrl() (api.x.ai/v1), and extraHeaders() to drop the OpenAI-Organization header. All payload / streaming / error-mapping logic inherited.
    • MistralClient (OpenAI-compatible). Same pattern; base URL api.mistral.ai/v1. Mistral's optional safe_prompt parameter isn't exposed in our params shape; can be added if needed.
    • GroqClient (OpenAI-compatible, very fast). Same pattern; base URL api.groq.com/openai/v1 (note the /openai/ prefix — Groq exposes their OpenAI-compatible surface there).
    • TogetherClient (OpenAI-compatible). Same pattern; base URL api.together.xyz/v1. Will also back MetaViaTogetherClient in chunk 5.
    • HuggingFaceClient (Inference Endpoints). SPEC deviation: SPEC called for TGI native protocol; we use HF's OpenAI-compatible chat-completions surface instead because (a) it handles per-model chat templates server-side, (b) the wire format matches OpenAI exactly so SseParser and OpenAiClient's payload code reuse cleanly, (c) most managed HF Inference Endpoints expose this surface. Thin subclass of OpenAiClient. No global base URL — users configure services.huggingface.base_url per their Inference Endpoint URL; per-model api_base_url overrides wire in at M5/M6.
    • MetaViaTogetherClient — Llama models proxied through Together. Trivial subclass of TogetherClient overriding only vendor(). Exists so the model registry can list Llama models under vendor='meta' while still routing to Together's API. Key resolution: M5/M6 run-submission layer picks which ApiKey to pass (UX note: a fallback "use Together key if no Meta key exists" would be friendlier — deferred to M5).
  • Vendor client factory: maps models.vendor → concrete class. App\Services\Llm\LlmClientFactory is registration-based — concrete clients call $factory->register($client) and are looked up by vendor(). Registered as a singleton in AppServiceProvider::register() so registrations apply app-wide and tests can swap implementations via the same instance. clientFor() throws UnsupportedVendorException on unknown vendor. The 9 concrete client registrations land in chunks 3-5.
  • Per-vendor token counter (uses tiktoken-php for OpenAI; approximate BPE for others). yethee/tiktoken ^1.1 installed (PHP port of OpenAI's BPE tables). TokenCounterInterface exposes count() + isExact() (the UI uses the latter to decide whether to show a ~ prefix). OpenAiTokenCounter selects encoding by model name (o200k_base for GPT-4o family, cl100k_base for older). ApproximateTokenCounter uses chars/4 with a small whitespace adjustment (±20% accuracy in English prose). TokenCounterFactory returns the OpenAI counter for vendor=openai and the approximate counter for everything else. EncoderProvider cached as a singleton so BPE tables load once per process.
  • Integration tests: recorded HTTP fixtures (VCR-style) per vendor for streaming + non-streaming. Each client suite (OpenAiClientTest, AnthropicClientTest, GoogleGeminiClientTest, the OpenAI-compat cluster dataset, the HF/Meta subclass tests) uses Http::fake() with realistic vendor SSE bodies / non-streaming JSON payloads as inline PHP strings. Vendor-side error codes (401/403/429/500) are also fixture-driven. Inline-string fixtures over fixture files is a deliberate trade-off for now — extract to tests/Fixtures/Llm/*.{sse,json} when test files start sharing payloads.
  • Smoke-test artisan command: php artisan vendors:smoke-test. Implemented as App\Console\Commands\VendorsSmokeTest. Iterates LlmClientFactory::supportedVendors(), reads each vendor's test key from SMOKE_TEST_{VENDOR}_KEY env var (skips with a clear notice if unset), reads optional model override from SMOKE_TEST_{VENDOR}_MODEL, sends "Say 'ok' and nothing else." with max_tokens: 10, temperature: 0. Reports per-vendor status (✓ passed / ✗ failed / ○ skipped). Default behavior: stop on first failure; --keep-going to continue. --vendor=name filter (repeatable) for testing a single vendor. Returns 0 if all pass-or-skip, 1 if any fail. Safe to run with no env configured at all — exits 0 with all-skipped (intentional so it can be wired into a deploy hook before keys are provisioned). Uses transient (unsaved) ApiKey instances; ApiKey::touchUsed() was updated to no-op for unsaved models so the smoke test doesn't insert orphan rows.

Exit criteria

  • Each of the 9 clients passes its recorded-fixture tests. Verified: ~99 tests across the 9 client + factory + token-counter + value-object suites, all green.
  • [~] vendors:smoke-test succeeds against all 9 live APIs. Mock-level verified: command flow / error handling / exit codes / env-var skip / --vendor filter / --keep-going all tested with fake clients. Real-network verification deferred until operator provisions SMOKE_TEST_*_KEY env vars with CI-account keys.
  • Submitting a 50-token prompt to OpenAI via the client yields a stream of token chunks in PHP. Verified at the mock level by OpenAiClientTest. Real-network verification follows once keys are provisioned.

M4 closed: 2026-05-17. Chunks 1–6 all green.

M4 retrospective notes:

  • The parked Laravel-AI-SDK vs hand-rolled decision (chunk 2) was the most consequential M4 choice. Laravel AI SDK covers all 9 vendors but doesn't expose logprobs — SPEC §3.1.5's logits panel needs them. Going hand-rolled meant more code (chunks 3–5) but full control of raw signals. If the viz ends up not needing logprobs after all, the Laravel AI SDK becomes a viable refactor target — every concrete client could be replaced with a thin SDK wrapper behind LlmClientInterface.
  • The 9 vendors clustered better than expected. OpenAI-compatible (5) + bespoke (3) + wrapper (1). The base-class abstraction in OpenAiClient paid off twice: chunk 4's cluster (5 vendors × ~15 LOC each) and chunk 5's HuggingFace.
  • Pint's php_unit_method_casing rule trips on any class with a method starting with test — even non-test classes. Caught at chunk 6 when my private testVendor() got snake-cased to test_vendor (breaking the caller). Renamed to attemptVendor to avoid. Consider excluding the rule for app/Console/ if we hit this again.
  • ApiKey::touchUsed had to become defensive so the smoke-test command could use transient ApiKey instances without inserting orphan rows. One-line guard; never fires in normal production flow.
  • SPEC deviation on HuggingFace (chunk 5): SPEC said TGI native, we went OpenAI-compatible. Documented in HuggingFaceClient docblock + commit. Practical reasons: chat templates handled server-side, code reuse from OpenAiClient. Revisit if a user-deployed endpoint exposes only the TGI surface.
  • Inline-string SSE fixtures instead of fixture files: works fine at 9 vendors. Extract when payloads start being shared across tests.

Stats: Pest 244 tests / 578 assertions, Vitest 8 tests. All CI green.


M5 — Threads + Runs (data layer)

Purpose: The thread/run data model is fully implemented and accessible through Eloquent models, but not yet wired to streaming.

Tasks

  • Migration: threads per SPEC §6 (including share_token, share_enabled_at). Schema: user_id (cascade), nullable title (auto-filled to first 60 chars of first prompt at M5 chunk 4), nullable system_prompt, nullable default_model_id with nullOnDelete so a removed model doesn't break the thread, default_parameters JSON, archived bool default false, tags JSON, share_token unique (M11 sharing), share_enabled_at, last_activity_at, timestamps. Indexed by user_id / archived / last_activity_at.
  • Migration: runs per SPEC §6 (including thread_id, sequence_in_thread). Schema: thread_id (cascade), denormalized user_id (cascade), model_id with restrictOnDelete so admin-deleting a model with runs is blocked (matches M3 chunk 4's design note), sequence_in_thread (unique with thread_id), nullable prompt (privacy opt-out per SPEC §10.4), prompt_hash always required (deterministic replay seed), conversation_history / parameters / token_log all JSON nullable, output_text + numeric timing/cost columns, status (cast to App\Enums\RunStatus: Pending / Streaming / Complete / Error with isTerminal()), nullable error_message.
  • Eloquent models with relationships (User → Threads → Runs → Model). Thread::runs() orders by sequence_in_thread by default. Run::thread()/user()/model() BelongsTo. User::threads() and User::runs() HasMany. LlmModel::runs() HasMany (with explicit model_id FK because Factory::for() would otherwise auto-derive a llmModel() relation name from the class). Thread::isShared() helper; Run::isTerminal() delegating to the enum.
  • ThreadService: create(), archive(), rename(), tag(), delete(). Plus unarchive() for symmetry. App\Services\Threads\ThreadService is deliberately Eloquent-thin — authorization stays in the controller/policy layer above. create() accepts optional title / system_prompt / default_model_id / default_parameters / tags; title left null lets RunService::submit (chunk 4) auto-fill from the first prompt. rename() trims whitespace and rejects empty strings with InvalidArgumentException. tag() normalizes (trim per-tag, drop empty/whitespace-only, dedupe case-sensitively to preserve "AI" vs "ai" intent); empty array or null clears tags. delete() is a hard delete — FK cascade drops the thread's runs. None of these methods touch last_activity_at (only run-submission does in chunk 4).
  • RunService::submit($user, $thread, $model, $prompt, $params). Signature refined from SPEC: takes User + LlmModel objects (caller resolves them via Laravel route binding) instead of scalar IDs. Validates in order: user owns thread (ThreadOwnershipException), prompt non-empty (EmptyPromptException), params within SPEC §3.1.4 bounds (InvalidParamsException), user has API key for the model's vendor with Meta → Together fallback (NoApiKeyException), context budget not exceeded with max_tokens reserved for response (ContextOverflowException). Then in a DB transaction: auto-titles the thread (first 60 chars of first prompt, word-boundary truncated, ellipsis if truncated) on the first run only, bumps last_activity_at, computes next sequence_in_thread, applies privacy redaction per user.store_prompts (nulls prompt and conversation_history but always stores prompt_hash), snapshots model architecture into parameters.model_snapshot for replay determinism (SPEC §10.1), creates the Run in pending state. Returns the Run with thread/user/model relations pre-set so the M6 pipeline doesn't need to re-load. All five failure modes raise a RunSubmissionException subclass that the HTTP layer in M6 maps to status codes.
  • Conversation-history builder: produces [{role, content}, ...] from a thread's completed runs. App\Services\Threads\ConversationHistoryBuilder::build($thread) returns a vendor-agnostic array starting with {role: 'system', content: thread.system_prompt} (if non-empty), followed by alternating {role: 'user'} / {role: 'assistant'} pairs from prior completed runs ordered by sequence_in_thread. Pending/streaming/errored runs are skipped. Privacy-redacted runs (prompt=null or output_text=null) are skipped entirely — including a half-pair would give the model nonsensical context. Documented as a known limitation: users wanting continuity keep store_prompts=true. The caller's new prompt is NOT appended here — that's the vendor client's job (each protocol formats the current turn differently).
  • Context-budget calculator: App\Services\Threads\ContextBudgetCalculator::check($model, $history, $newPrompt, $reservedForResponse = 0). Returns a ContextBudgetResult value object with fits / totalTokens / budget / overBy so the caller can surface a useful "you're N tokens over a Y-token window" message. Routes to OpenAI's exact tiktoken counter when vendor=openai, else the approximate counter — same TokenCounterFactory as the prompt-preview UI. Models with null/zero context_length are treated as unlimited (fits=true) since we can't enforce a budget we don't know. Defense-in-depth check — the frontend already does a precise client-side check before submit (SPEC §3.5); this guards against a malicious or buggy client.
  • Tests: thread CRUD, run validation, context overflow rejection, history snapshot correctness. Covered across chunks 1–4: ThreadModelTest (11), RunModelTest (14), ThreadServiceTest (18), ConversationHistoryBuilderTest (10), ContextBudgetCalculatorTest (10), RunServiceTest (22) — 85 M5 tests in total.

Exit criteria

  • Threads/runs can be created and queried via Eloquent. Verified by ThreadModelTest + RunModelTest (25 tests covering persistence, casts, relations, FK behavior).
  • RunService::submit rejects invalid input with clear errors. Verified by RunServiceTest validation group (10 tests): wrong owner, empty prompt, no key, all four param bounds, context overflow with details — each raising a distinct RunSubmissionException subclass.
  • Submitting two runs in a thread results in the second run's conversation_history containing the first run's user+assistant turns. Verified by RunServiceTest::it stores the conversation history snapshot — creates a completed first run, submits a second, asserts the second run's snapshot is [{user: first q}, {assistant: first a}].

M5 closed: 2026-05-18. Chunks 1–5 all green.

M5 retrospective notes:

  • Migration timestamp collision. php artisan make:migration produced both create_threads_table and create_runs_table with the same timestamp; runs would have run alphabetically first despite its FK to threads. Bumped the runs filename timestamp by 1s. Worth knowing for any future multi-migration commit — generate them seconds apart or rename.
  • Factory::for($model) auto-derives the relation name from the related model's class name (camelCase). LlmModelllmModel(). We define the relation as model() for cleaner code, so test setups need Factory::for($model, 'model') explicitly.
  • Pest's toThrow(string) does substring matching, not isinstance when given a string. toThrow(\Throwable::class) reads as "the exception message contains the substring 'Throwable'" — never matches real exceptions. Use concrete exception classes (QueryException::class, etc.).
  • Test factories + unique constraints. Hardcoding 'name' => 'gpt-4o' in tests clashes with factories made in beforeEach. Drop the override; the factory generates unique names; OpenAI token counter falls back to o200k_base for unknown names so behavior is identical.
  • Pint's lambda_not_used_import rule auto-strips unused variables from closure use() lists. Useful but can be surprising if you expected a future-use placeholder to stick around — annotate intent in a comment if you really want to keep it.
  • The Meta-via-Together key fallback was closed here in RunService (chunk 4), addressing the UX gap noted at M4 chunk 5. When a user submits a vendor=meta model and has no meta API key but DOES have a together key, the service silently uses the Together key. No-op for users who explicitly added a Meta key.

Stats: Pest 329 tests / 756 assertions, Vitest 8 tests. All CI green.


M6 — Realtime + Streaming Pipeline

Purpose: End-to-end: prompt submitted → vendor stream → WebSocket events → frontend receives them. No visualization yet (M7/M8), but a debug page shows raw events.

Tasks

  • Install + configure Laravel Reverb on local dev. (SPEC originally said Soketi; swapped to Reverb at M6 chunk 1 — see docs/parked-decisions.md item 1.) composer require laravel/reverb ^1.10. config/reverb.php + routes/channels.php published. bootstrap/app.php extended with channels: __DIR__.'/../routes/channels.php' so channel-auth routes register at boot.
  • Configure Laravel broadcasting.php with the reverb driver. BROADCAST_CONNECTION=reverb set in .env.example. Old PUSHER_* env keys renamed to REVERB_* to match Laravel conventions. Vite frontend env mirrors (VITE_REVERB_APP_KEY etc.) added for the chunk 4 Echo wiring. resources/js/types/global.d.ts ImportMetaEnv augmentation updated accordingly.
  • Install Laravel Echo + pusher-js on frontend (chunk 4b). laravel-echo ^2.3 + pusher-js ^8.5 — pusher-js handles the Reverb-compatible WebSocket transport. resources/js/echo.ts initializes window.Echo from the VITE_REVERB_* env vars; init is conditional on VITE_REVERB_APP_KEY being set so environments without Reverb config don't bomb the bundle on load. Side-effect-imported from app.tsx before any page subscribes. npm audit flags 3 moderate transitive ws advisories via socket.io-client (one of laravel-echo's optional transports we don't use); the vulnerable code is unreachable in our pusher-js bundle. Future hardening (M14 deployment) can add a package.json overrides block if it matters.
  • Channel: private-runs.{run_id} (chunk 4a). Registered in routes/channels.php via Broadcast::channel('runs.{runId}', ...) — the callback resolves the Run by ID and authorizes only when run->user_id === user->id. Nonexistent runs and stranger requests both return false (and thus 403 from /broadcasting/auth) — the closure doesn't distinguish them to avoid leaking the existence of other users' runs. Auth flow: pusher-js POSTs socket_id + channel_name to /broadcasting/auth (the route Laravel auto-registers from bootstrap/app.php's withRouting(channels: ...)); on 200, Laravel signs the channel with the broadcasting secret and the client uses that signature to subscribe. 4 Pest tests cover owner/stranger/nonexistent/unauthenticated paths. Test-env wrinkle: BROADCAST_CONNECTION=pusher is set in phpunit.xml so the channel-auth endpoint actually runs the callback — the default log driver's auth() is a no-op that returns 403 for everything, which would make the tests pass trivially for the wrong reason. Fake PUSHER_APP_* values are fine since no network leaves the box.
  • Job: StreamRunJob — pulls run, picks vendor client, iterates stream(), broadcasts events.
  • Events: RunStarted (chunk 1), TokenReceived, LayerAdvanced, MoeRouted, RunCompleted, RunErrored. All in App\Events\Runs\, all implement ShouldBroadcastNow on private-runs.{run_id} (refactored from ShouldBroadcast in chunk 3 — per-token broadcasts fire from inside the queued StreamRunJob worker, so going through the queue again would cost an extra round-trip and risk out-of-order delivery; ShouldBroadcastNow keeps the broadcast in-process and ordered). Each broadcastWith() returns a JSON-safe payload (scalars + arrays only) so the frontend doesn't need to know about Eloquent shapes. MoeRouted is the only one that's vendor-conditional (emitted only for architecture_type='moe' model snapshots).
  • State machine: App\Services\Runs\RunEventEmitter translates each LlmTokenChunk into a deterministic event sequence: TokenReceived + LayerAdvanced, plus MoeRouted for MoE models. MoE expert selection is a function of (run.id, token_index) only, via a SHA-256-based stateless PRNG — replays produce identical animations per SPEC §10.1. Scores are synthetic descending probabilities renormalized to ~1.0 (no proprietary MoE vendor exposes real router logits, so this is illustrative-only). Also exposes completedEvent() (computes tokens-per-second from duration) and erroredEvent() for the StreamRunJob (chunk 3) to call once at terminal status.
  • Job: StreamRunJob (App\Jobs\StreamRunJob) — implements ShouldQueue, takes a Run via constructor. Per-run flow: refresh from DB (defensive against worker retries — bails immediately if status is no longer Pending); resolve user's ApiKey for the model's vendor with Meta → Together fallback mirroring RunService::resolveApiKey; resolve vendor client via LlmClientFactory; flip status to Streaming and broadcast RunStarted; iterate client->stream(...). Per chunk: compute t_ms from microtime(true), append {token, index, t_ms, logprobs} to in-memory token_log (only when chunk->text is non-empty), concatenate output_text, dispatch the events RunEventEmitter::eventsForChunk() returns, track the latest non-null chunk->usage. On clean finish: compute duration_ms, derive input_tokens/output_tokens (from latest usage or fall back to token count), compute tokens_per_second, compute estimated_cost from parameters.model_snapshot pricing (returns null when either price is absent — so a price change between submit + execute can't retroactively rewrite a recorded cost), persist final state, touch ApiKey::last_used_at, broadcast RunCompleted. On vendor exception: friendly-error mapping for LlmClientException subclasses (rate-limit, invalid-key, generic), persist partial output_text + token_log + duration_ms + output_tokens, broadcast RunErrored with the partial output. Pre-stream errors (no API key, no client registered) still broadcast RunErrored — the frontend's chunk-4 subscription needs the signal even if RunStarted never fired. 19 Pest tests cover happy path (Pending→Complete + event order + MoE + duration/TPS/cost computation + chunk-count fallback + logprobs preservation + last_used_at touch + cost-null when snapshot lacks pricing), error paths (mid-stream throw with partial preserved, pre-stream key/client failures, vanished-API-key, RunErrored dispatch), Meta→Together fallback (both directions), and idempotency (re-running a non-Pending Run is a silent no-op). Chunk 5a edit: added per-chunk incremental UPDATE of token_log + output_text so the SSE fallback (/runs/{run}/stream) can read in-flight progress from the persisted row without a message broker between processes. One additional test (#20) uses a custom observer client that captures DB snapshots after each yield, proving the row's token_log grows monotonically during the stream rather than only at terminal.
  • HTTP submission: POST /threads/{thread}/runs (chunk 4a). App\Http\Requests\SubmitRunRequest does field-level validation (model_id exists, prompt required + non-empty, parameters within SPEC §3.1.4 bounds) so the response carries proper 422 per-field errors before the service ever runs. App\Http\Controllers\RunController::store delegates orchestration to RunService::submit and translates each domain exception: ThreadOwnershipException → 403, EmptyPromptException → 422 on prompt, NoApiKeyException → 422 on model_id with vendor in message, InvalidParamsException → 422 on parameters.{field}, ContextOverflowException → 422 on prompt. On success: persists the Pending Run, dispatches StreamRunJob, returns 201 with {run: {...}, channel: 'private-runs.{id}'} so the frontend knows the channel to subscribe to. Route is throttled via the 'runs' named limiter from AppServiceProvider (live-reads users.max_runs_per_hour); X-RateLimit-* headers come along for free. 16 Pest tests cover auth (401), authz (403 for non-owners), validation (7 fields + bounds), service exception mapping (no-API-key with vendor name in error, context overflow), success (run persisted + Bus-faked StreamRunJob dispatch + response shape + auto-title on first prompt + last_activity_at bump), Meta→Together fallback at the HTTP layer.
  • SSE fallback route: GET /runs/{run}/stream (chunk 5a). App\Http\Controllers\StreamRunController returns Symfony\Component\HttpFoundation\StreamedResponse with Content-Type: text/event-stream, X-Accel-Buffering: no (defeats nginx response buffering), Cache-Control: no-cache. Architecture: polling-based, not pub/sub. Since the queue-worker process running StreamRunJob has no shared memory with the PHP-FPM process serving this request, chunk 5a teaches StreamRunJob to write token_log + output_text incrementally on every chunk (additive to the existing terminal write — no new tests broken); the SSE controller then polls the run row every ~150 ms and emits the delta since the last cursor as SSE frames. Tail latency ~150 ms; no message broker required. Hard cap of 4000 iterations (~10 min) guards against an FPM worker hanging on a stuck run. Heartbeat comment every 200 iterations (~30 s) keeps proxies from closing the conn. Frame names match the WebSocket broadcastAs() strings (run.started, token.received, layer.advanced, run.completed, run.errored) so the frontend SSE consumer (chunk 5b) shares the same RunEvent type. Owner-only — same authz invariant as runs.{runId} channel auth. PHP-FPM caveat: long-lived SSE requests tie up FPM workers; M14 deployment chunk should size pm.max_children with this in mind. 11 Pest tests cover authz (redirect/403/404), response headers, SSE wire format, emission against already-Complete + already-Error runs, layer-count from model snapshot, partial-output preservation; tests use streamedContent() (not getContent()) to capture the streamed body.
  • Frontend hook: useRunStream(runId) (chunks 4b + 5b) — prefers WebSocket via Laravel Echo, falls back to SSE on connection failure. Returns { events, status, transport, disabled } where transport: 'websocket' | 'sse' | 'none' exposes which path is live, disabled is the back-compat alias for transport === 'none'. Discriminated-union RunEvent type in resources/js/types/runs.ts mirrors each PHP event's broadcastWith() exactly. Two-effect shape: effect 1 resets state when (runId, transport) changes; effect 2 sets up the subscription. Splitting ensures events: [] lands before the new transport's first event regardless of React's scheduling. Transport selection: picks 'websocket' if window.Echo is set, else 'sse' if window.EventSource exists, else 'none'. Fallback trigger (chunk 5b): subscribes to pusher's connection.state_change and flips to SSE when state becomes 'failed' or 'unavailable'. Stays on SSE for the rest of the run even if WebSocket recovers — avoids transport thrashing and the dedup logic switching-back would require. Fallback UX (chunk 5b): clears events on transport change so the SSE controller's cursor-0 replay can repopulate without dedup. M8 can refine if the viz needs smoother UX. 21 Vitest tests cover WebSocket happy path, SSE-only path (Echo=null), WS→SSE fallback (failed/unavailable state, event reset, EventSource ctor URL, stay-on-SSE after WS recovery), transport='none' when neither is available, runId transitions. Mock EventSource is implemented as a class (not a plain vi.fn()) so new EventSource() works under jsdom (which doesn't ship EventSource).
  • Event names refactor (chunk 4b): added broadcastAs() to each of the 6 events so the frontend listens with short kebab-case names (.run.started, .token.received, .layer.advanced, .moe.routed, .run.completed, .run.errored) instead of full PHP class FQNs. Decouples the JS subscription from the PHP namespace — renaming/moving event classes won't silently break the frontend. 6 Pest tests in RunEventBroadcastAsTest.php lock the strings down so a future drift fails the test before it ships.
  • Debug page /runs/{id}/debug (chunks 4b + 5b): App\Http\Controllers\DebugRunController::show does the same-user-as-run check (mirrors the runs.{runId} channel auth invariant) and returns an Inertia render of Runs/Debug with the run's static metadata + the channel name to subscribe against. The TSX page renders the metadata header up top, a "subscribed channel + live status + active transport" line, and a chronological JSON event list below — append-only as useRunStream delivers events. Active transport label reads WebSocket, SSE (fallback), or unavailable; chunk 5b also rewrites the disabled-notice copy to "no realtime transport is available" since SSE is now a fallback. config/inertia.php added so the Inertia view-finder knows about .tsx page paths (defaults are Vue-only); without this, assertInertia(...)->component('...') fails with "page component file does not exist" on every test. 4 Pest tests cover authz (redirect/403/404 + 200 render with prop assertions); 8 Vitest tests cover render + event accumulation + status transitions + transport label (WebSocket / SSE / unavailable) + disabled-notice gate.
  • Reconnect logic (chunk 6): GET /runs/{run}/events?since=N JSON backfill endpoint (App\Http\Controllers\RunEventsController) returns the persisted token_log slice from index N onward, plus current status + completion/error blocks for terminal runs. Lightweight one-shot — no streaming, no FPM-worker hold. Owner-only authz (same invariant as channel auth, SSE, and debug page). useRunStream tracks maxSeenIndex + wasDisconnected refs; on pusher state_change transitioning from disconnected/connecting back to connected, fires the backfill with since=maxSeenIndex+1 and appends the returned token_log entries as if they arrived live. Also synthesizes run.completed / run.errored events from the response's completion/error blocks so the closing event isn't missed when the run terminates during the gap. Pusher-js auto-reconnects WS but does not replay — that's the gap this closes. The SSE fallback (chunk 5b) is for hard failures; this is the catch-up path for transient blips. 11 Pest tests on the endpoint (authz, slicing semantics, since clamping, terminal-state blocks) + 6 Vitest tests on the hook (initial-connect no-fetch, since=N+1 URL, run.completed/run.errored synthesis, double-reconnect dedup, error-tolerance).
  • Queue worker config (chunk 6): deployment/supervisor/laravel-queue.conf (2 worker processes, 600 s job timeout, 605 s stopwaitsecs so SIGTERM gives in-flight runs a clean exit) + deployment/supervisor/laravel-reverb.conf (single process — Reverb's in-memory pub/sub doesn't multi-process out of the box; horizontal scale needs the Redis backend, deferred to M14). Both .conf files have install instructions in the header comment. For dev: foreground via php artisan queue:work + php artisan reverb:start --host=0.0.0.0.

Exit criteria

  • ✅ Submitting a run results in tokens streaming to the debug page within ~200 ms of vendor delivery — covered by the chunk-3 StreamRunJob tests (broadcast dispatch on each chunk + microtime-based t_ms) plus the chunk-4b useRunStream / Debug page render tests. Manual verification recipe in docs/m6-exit-criteria.md §1.
  • ✅ Killing the Reverb process mid-stream causes the frontend to fall back to SSE without dropping tokens — covered by chunk-5b's WS → SSE fallback test group (transport flip on pusher failed/unavailable, event-list reset, SSE EventSource opened against /runs/{id}/stream) + chunk-5a's incremental token_log persistence (no events lost because the SSE controller replays from the persisted log, not from an in-memory queue). Manual recipe in docs/m6-exit-criteria.md §2.
  • ✅ A second tab subscribing to the same runs.{id} channel receives the same events — covered by Reverb's pubsub-layer semantics (multiple subscribers to a private channel fan out identically) + channel auth tests (chunk 4a) verifying the owner-only invariant. Manual recipe in docs/m6-exit-criteria.md §3.

M6 retrospective

Sized at 6 days; landed in 6 chunks across a similar elapsed window. Quality bar held — 432 Pest tests / 1030 assertions, 43 Vitest tests; Pint / ESLint / type-check / Vite build all green at every chunk boundary.

What worked

  • Splitting chunk 4 into 4a (backend HTTP + channel auth) + 4b (frontend Echo wiring + debug page) kept each commit reviewable. Same for chunk 5 (5a backend SSE / 5b frontend fallback).
  • ShouldBroadcastNow over ShouldBroadcast (chunk 3): per-token broadcasts fire in-process from the queue worker, preserving order without a second queue round-trip. Caught early because the chunk-1 RunStarted event went out as ShouldBroadcast and would have caused subtle reordering once token streaming kicked in.
  • broadcastAs() short kebab names (chunk 4b) decouple frontend listeners from PHP class FQNs. Renaming an event class no longer silently breaks the wire contract.
  • Polling-based SSE (chunk 5a) avoided introducing Redis as a hard runtime dep — the persisted token_log is the source of truth, both transports read from it. Tail latency is ~150 ms which is well under the "tokens stream in real time" perception threshold.

Surprises / footnotes

  • BROADCAST_CONNECTION=log (the default .env driver) no-ops on the channel-auth callback — needed BROADCAST_CONNECTION=pusher with fake test creds in phpunit.xml to actually exercise /broadcasting/auth in tests. Documented inline.
  • Inertia's view-finder defaults to .vue; needed config/inertia.php to teach it about .tsx pages or assertInertia(...)->component('...') fails on every test.
  • assertInertia requires the route to return a View (not the X-Inertia JSON form), so test HTTP calls don't pass the X-Inertia header.
  • jsdom doesn't ship EventSource; chunk 5b needed a real ES6 class mock (not a plain vi.fn()) so new EventSource() works.
  • Symfony\StreamedResponse::getContent() returns false; chunk-5a SSE tests use streamedContent() to actually capture the body.
  • npm audit reports 3 moderate transitive ws advisories via socket.io-client (an optional laravel-echo transport we don't use). Unreachable in our pusher-js bundle; deferred to M14.

Decisions parked or deferred to later milestones

  • Mid-stream WS↔SSE switch-back: once on SSE we stay there. Avoiding dedup complexity is worth the brief flicker. M8 can revisit if the viz needs smoother UX (chunk 5b documented this choice).
  • Real cross-process pub/sub (Redis) for sub-100ms SSE: not needed for our concurrency target; polling-based SSE is good enough for launch.
  • Horizontal Reverb scaling (Redis backend): single-process is fine for the single-VPS target; M14 deployment chunk will revisit if traffic warrants.
  • Browser-based E2E test harness (Playwright/Cypress): chosen against in chunk 6 (unit + integration + manual recipes is enough for M6); M12 accessibility/polish is the natural home.

Carry-forward into M7

  • The debug page proves the plumbing is alive end-to-end. M7's thread-detail page replaces it as the user-facing surface; the debug page stays for internal use.
  • useRunStream is the only frontend consumer of the streaming pipeline today. M7 will refactor it into the thread-page hierarchy; the hook's contract (events + status + transport + disabled) should be enough.

M7 — Frontend Static UI

Purpose: All non-animated UI is built and navigable: dashboard, thread list, thread detail, prompt input, model selector, parameter controls, API keys, settings, admin. The submit button works (creates a run) but the right pane just shows the debug log from M6.

Decisions (chunk 1):

  • Component library: shadcn/ui (Radix + Tailwind, copy-paste pattern). Frontloads accessibility via Radix internals; gives us a consistent visual language for M7 + M8 + M12. Initial deps: class-variance-authority, clsx, tailwind-merge, tailwindcss-animate, lucide-react, @radix-ui/react-slot. Components live in resources/js/Components/ui/. CSS-var-based theme tokens in resources/css/app.css (light + dark palettes); <html class="dark"> makes dark the app default. components.json is committed for future npx shadcn@latest add ... calls.
  • Token counter: server-side AJAX via the existing TokenCounter services (from M3). Debounced POST endpoint per chunk 5; no client-side WASM, no bundle bloat. Slight perceptible lag during fast typing is acceptable for the M7 scope.
  • Onboarding: empty-state CTAs on each page + a "Set up your first API key" nudge on the dashboard when none exists. No dedicated wizard route.
  • Split: 10 chunks (foundation → layout + 404 + responsive → dashboard → threads list → thread detail → prompt input → model selector → param controls → account/admin polish → closeout).

Tasks

  • Foundation (chunk 1): shadcn/ui deps installed; resources/js/lib/utils.ts with cn() helper; tailwind.config.js extended with theme tokens that read from CSS vars (bg-background, text-foreground, etc.); resources/css/app.css :root + .dark blocks define the light + dark palettes; tailwindcss-animate plugin registered for Radix animations. 5 baseline UI primitives in resources/js/Components/ui/: button.tsx (6 variants × 4 sizes via CVA, asChild Slot pattern), card.tsx (6 subcomponents), input.tsx, label.tsx, separator.tsx (decorative + a11y modes). 22 Vitest tests cover render, variant classes, cn() override behavior (tailwind-merge resolves conflicts so caller h-20 beats base h-10), ref forwarding, asChild Slot, and Separator a11y modes. Lint exception: Label primitive disables jsx-a11y/label-has-associated-control inline — design-system primitives can't statically prove association; that's the caller's job (via htmlFor / id).
  • Layout shell (chunk 2): AppLayout refactored onto shadcn primitives — sidebar uses Button + Separator, nav links use theme tokens (bg-accent / text-accent-foreground for active state) instead of hardcoded slate-*. Desktop ≥ md: 240px fixed sidebar. Sub-md: sidebar hides, top bar shows a hamburger Button that opens the sidebar in a left-anchored Sheet (Radix Dialog wrapper; new primitive in resources/js/Components/ui/sheet.tsx with side variant + 4 Vitest tests). Same NAV_ITEMS array drives both desktop + mobile — single source of truth. Optional <title> prop slot in the top bar; falls back to the active nav item's label. SheetHeader/SheetTitle/SheetDescription kept in sr-only wrapper to satisfy Radix's a11y requirement without visual noise.
  • Dashboard page (chunk 3): App\Http\Controllers\DashboardController::index aggregates the signed-in user's stats (total run count, sum of input + output tokens across complete runs, sum of estimated_cost across complete runs) + 5 most-recent threads ordered by last_activity_at desc with withCount('runs') + has_api_keys boolean. Per-user isolation enforced via where('user_id', $user->id) on every query — covered by a cross-user-leakage Pest test. Token / cost sums skip errored / streaming / pending runs (their counts may be incomplete, their cost is null). React side (resources/js/Pages/Dashboard.tsx) refactored onto shadcn Card primitives: 3 stat cards in an md:grid-cols-3 grid, recent-threads section with relative timestamps ("3h ago", "2d ago"), NoApiKeyCallout at the top when has_api_keys=false linking to /api-keys, EmptyThreads state in the recent section linking to either /api-keys or /threads depending on whether a key exists. Currency formatting via Intl.NumberFormat. Per-thread links deferred to chunk 5 — recent threads display as cards but don't link individually yet (the /threads/{id} route lands then). "View all" link in the section header points at /threads (still the chunk-4-bound placeholder). 10 Pest tests (auth, prop shape, aggregation, errored-run exclusion, cross-user isolation, recent-thread ordering + count + isolation, has_api_keys flag both states); 8 Vitest tests (3 stat cards present, number-format, no-API-key callout visibility + link target, empty-threads CTA + dependency on has_api_keys, recent threads render with relative time + archived hint). Test gotcha: JSON has no float-vs-int distinction; 0.0 round-trips as int 0, so the zero-cost assertion uses 0, not 0.0.
  • Threads list page (chunk 4): App\Http\Controllers\ThreadController::index returns a 20-per-page paginated list scoped to the signed-in user with withCount('runs'), ordered by last_activity_at desc. Filters: ?q= (case-insensitive title LIKE), ?archived=false|true|all (default false), ?tag= (single tag via whereJsonContains). Also returns available_tags — a sorted unique list derived from the user's threads — for the filter dropdown. store action creates an empty thread and redirects to threads.show (a ComingSoon placeholder until chunk 5). Archive/share/delete actions deferred to chunk 5 since they're per-thread concerns. React page (Pages/Threads/Index.tsx) uses shadcn Card/Button/Input + a native <select> for the tag filter. 300 ms debounced search via setTimeout + router.get(..., { preserveScroll, preserveState, replace }). Archive-toggle is a 3-button group (Active / All / Archived) with aria-selected. Two empty states (empty-all vs empty-filtered); Clear filters only renders when any filter is non-default. "New thread" button posts to /threads via useForm. Pagination is a Previous/Next bar with explicit hrefs preserving filters. Lint gotcha: the React Compiler / react-hooks/immutability rule needs the search handler wrapped in useCallback AND declared before the useEffect that references it. 17 Pest tests (auth, prop shape, ordering, isolation, run_count, search trimming, all 3 archive states, tag filter + isolation, available_tags sort/uniqueness, pagination math + ?page, store flow); 11 Vitest tests (both empty states, card rendering + per-thread link, debounced search via fake timers, archive toggle reflects + dispatches, tag dropdown shows only when tags exist, clear button gates correctly, create-thread POSTs, pagination only when needed). Vitest gotcha: vi.mock is hoisted above top-level consts, so the spy fns (routerGet, formPost) live inside vi.hoisted(...) to be available to the mock factory.
  • Thread detail page (chunk 5): App\Http\Controllers\ThreadController::show / update / destroy plus their routes. show returns the thread, its runs in sequence_in_thread order, usable_models (vendors the user has an API key for, with the Meta→Together fallback from RunService::resolveApiKey honored), and has_api_keys for the empty-state copy. update validates title (nullable, max 200) and archived (bool); destroy cascade-deletes runs via the FK. All three abort_unless on ownership. Pages/Threads/Show.tsx renders: header with inline-editable title + archive toggle + delete-with-confirm (Radix AlertDialog), read-only transcript of run cards (status badge + prompt + output + token/duration/cost stats; error message for errored runs), and a bare prompt input footer (Textarea + native <select> grouped by vendor + Submit posting to the existing /threads/{thread}/runs route). 2 empty-state branches: no API key → CTA to /api-keys; key but no usable models → registry-refresh hint. Chunks 7–8 enrich the prompt input footer with autosize / token-count / context-warning / fancy model picker / parameter controls. Chunk 6 wires the live-stream view (useRunStream) into a right pane so submitted tokens visibly stream into the page. New primitives: Textarea + AlertDialog (Radix-backed; 4 + 4 Vitest tests). 17 Pest tests (auth, ownership, render shape, run ordering, isolation, vendor filtering + Meta→Together, title update + null + max-length validation, archive toggle, delete + cascade verification); 16 Vitest tests (title + archive + delete header actions including confirm/cancel paths, transcript empty state + per-run rendering, footer state machine across all 3 branches, vendor-grouped optgroups, Submit gated by empty prompt). Lint exceptions: the title-edit Input uses autoFocus after an explicit click (legitimate UX), and react/no-unescaped-entities flagged a literal apostrophe in copy.
  • Prompt input panel (chunks 6a + 6b):
    • Multi-line textarea with autosize (chunk 6a) — useAutosizeTextarea hook drives style.height off scrollHeight on every change, clamped to 480px. No JS dependency.
    • Conversation history above (chunk 6a) — collapsible HistoryPreview panel renders the same OpenAI-shape array that RunService::submit will send (system + user/assistant pairs from prior complete runs). Closed by default.
    • [~] Chat-template preview pane (toggleable) — partial via the history preview. Vendor-specific template renderers (jinja2-ish strings stored in models.chat_template) defer to a future polish chunk; the history preview already shows what context the model will see.
    • Token-count preview (chunk 6a) — server-side via POST /threads/{thread}/preview (App\Http\Controllers\PromptPreviewController) returning {history, token_counts: {history, prompt, reserved, total}, budget, fits, over_by, model}. Counts use the same TokenCounterFactory that ContextBudgetCalculator uses (exact tiktoken for OpenAI, BPE-approximate elsewhere) so server + frontend display stay aligned. Debounced 400ms client-side via useDebouncedPreview with AbortController cancellation on rapid input changes. No client-side WASM, no bundle bloat (per the chunk-1 decision).
    • Approaching-context-limit warning at 80% (chunk 6a) — BudgetIndicator renders a horizontal bar that's primary at < 80%, amber at 80%–100%, destructive at > 100% (with the over_by count next to it). Submit button is gated off the same fits flag — you can't post a run that the server-side ContextBudgetCalculator will reject. Models with context_length=null get a plain "N tokens" readout (no bar). 16 Pest tests + 8 new Vitest tests.
    • Live debug pane (chunk 6b)LiveStreamPane component on Show.tsx subscribes to the currently-active run via useRunStream (from M6, WebSocket → SSE fallback included). findActiveRun picks the latest non-terminal run from the runs prop. Layout switches to a lg:grid-cols-3 two-column at desktop (transcript + form on cols 1–2, live pane sticky on col 3); single-column stack at mobile. Shows event JSON-per-line, status badge, active transport label. Auto-reloads the runs prop via router.reload({only: ['runs']}) 400ms after a terminal event so the transcript picks up the final output_text + token counts and the live pane returns to its empty state. Dual-respond fix on RunController::store: returns an Inertia redirect to threads.show when X-Inertia: true is present (so useForm.post triggers the page-reload that surfaces the new run); keeps the chunk-4a 201 + JSON shape for postJson() / future API callers. 2 new Pest tests cover both branches; 8 new Vitest tests cover empty state, active-run detection (incl. ignoring terminal runs), event rendering, transport label + unavailable notice, router.reload triggered for both complete + errored terminal states. Satisfies the M7 exit criterion "see tokens stream into the debug pane on the right." M8 will replace this debug-style pane with the real visualization.
  • Model selector — combobox (chunk 7). ThreadController::usableModels payload extended with architecture_type, position_encoding, layers, hidden_dim, attention_heads, moe_experts, moe_active_experts, pricing pair. Two new shadcn primitives: Popover (Radix) + Command (cmdk-backed; keyboard-first list with fuzzy match over display_name + name). ModelPicker component: trigger Button shows the currently-selected model name, opens a Popover with arch tabs (Dense / All / MoE), vendor multi-chip filter (hidden when only one vendor is on file), and a Command list grouped by vendor with a context-length chip on each row. ModelMetadataCard: two-column key/value grid showing the selected model's arch (with 8 experts (2 active) for MoE), context (128K tokens etc.), layers, hidden_dim, attention_heads, position encoding, pricing pair ($X.YZ / M tokens). Size filter dropped (chunk-7 decision) — we don't store parameter count and the name-regex heuristic is too brittle; arch + vendor + context-length sort + search covers the use case. Vitest infra additions: jsdom polyfills for ResizeObserver and Element.prototype.scrollIntoView in test/setup.ts since cmdk uses both. 1 new Pest test on the extended payload; 7 new Vitest tests on the picker (initial metadata card render, MoE-aware metadata format, popover open on click, arch + vendor filter chips visibility, MoE-filtered list, selection triggers form.setData('model_id', ...), vendor chips hidden when one vendor).
  • Inference parameter controls (chunk 8). ThreadController::usableModels payload extended with supported_params (the LlmModel JSON map of param-name → bool). New shadcn Slider primitive (Radix-backed; 3 Vitest tests cover ARIA role/value, disabled data-disabled marker, prop wiring). ParameterControls component: collapsible Card (closed by default; chevron toggle in header; primary-colored "custom" dot when any value is non-default). Five rows — temperature (0–2 step 0.05), top_p (0–1 step 0.05), top_k (0–500), max_tokens (1 to model's context_length), seed (null = random, input-only). Each numeric row pairs a Slider + small Input bound to the same value; input clamps to the slider's min/max on the way in. Unsupported params (per supported_params[key] === false) get a "(not supported)" hint and disabled Slider + Input. Missing/null supported_params treated as "all supported" (defensive against older registry rows). "Reset to defaults" link sends the full default object. Show.tsx PromptForm wires the controls into form.data.parameters; useDebouncedPreview now keys off max_tokens so the chunk-6a budget bar updates when the user adjusts the reserve. 1 new Pest assertion on the extended payload; 13 new Vitest tests on ParameterControls (collapsed default, expand on toggle, custom-dot toggling, input dispatch, clamp on out-of-range, unsupported hint + disabled state, Reset link payload, max_tokens ceiling honored, seed empty/Random/randomize paths, null supported_params fallback).
  • API Keys page (chunk 9a polish; original behavior shipped in M4 chunk 1). Refactored onto shadcn: Card/CardHeader/CardContent wrapper, Button/Input/Label primitives, theme tokens replace hardcoded slate-*. confirm() JS dialog swapped for the AlertDialog confirm flow (one page-level dialog driven by pendingDelete state, no per-row dialog instances). Existing M4 backend tests stay green (encrypted-key storage, vendor allowlist, masked display). 8 new Vitest tests cover render + empty state + per-key rows + Add-form POST + delete-with-confirm (open/confirm/cancel paths) + vendor dropdown options.
  • Settings page (chunk 9a polish). Refactored onto shadcn Card/Button/Label. Single store_prompts toggle (display name lands when the SPEC user-profile column gets added; not a blocker for M7 exit). Existing M3-era backend tests stay green. 4 new Vitest tests cover render (both checked + unchecked init), PATCH dispatch on submit, and checkbox toggling.
  • Admin pages (chunk 9b polish; original behavior shipped in M2/M3). Admin Users (Pages/Admin/Users.tsx): user list table + inline rate-limit editor refactored onto Card/Input/Button + theme tokens. Admin Models index (Pages/Admin/Models/Index.tsx): filter form + paginated registry table; refresh-from-OpenRouter and per-row delete now use AlertDialog confirms instead of native confirm(). Admin Models edit (Pages/Admin/Models/Edit.tsx): the heaviest single form (identity / architecture / capacity + pricing / capabilities / chat template / refresh control sections); all rows refactored onto Input/Label/Textarea/Button with Section/Field internal helpers preserved. Pagination styling moved to theme tokens (bg-accent for active, cursor-not-allowed on disabled links). Existing M3 admin-controller Pest tests stay green. 12 new Vitest tests (4 Users + 5 Models Index + 3 Models Edit) cover render shape, per-row dispatch, refresh + delete confirm flows, filter form submit, and the metadata_estimated notice.
  • 404 / error pages (chunk 2): 4 Inertia pages in resources/js/Pages/Errors/ (NotFound, Forbidden, Expired, ServerError) all built on a shared ErrorShell component. Standalone — NOT wrapped in AppLayout since unauthenticated users hit them too. usePage().props.auth.user determines whether the CTA reads "Back to dashboard" or "Sign in". bootstrap/app.php's withExceptions callback maps 403/404/419/500/503 to the matching component and renders via Inertia when APP_DEBUG=false; debug-mode dev keeps the stock Laravel whoops/error pages with stack traces. JSON requests bypass the Inertia render and get the default JSON error response (so XHR callers aren't broken). 6 Pest tests cover the 404/403/500 mappings, authed + unauthed paths, and the JSON-bypass.
  • Responsive: desktop + tablet breakpoints (chunk 2). The sidebar↔drawer breakpoint is md (768px); below that the layout collapses to a single column with the hamburger Sheet. Verified on a phone (192.168.0.205:8001) via the dev-login magic-link flow.

Exit criteria

  • ✅ All pages render and route correctly. Covered by Pest controller tests on every route (DashboardControllerTest, ThreadIndexTest / ThreadShowTest, PromptPreviewTest, ApiKeysControllerTest, SettingsControllerTest, ErrorPagesTest, Admin/ModelsControllerTest, Admin/UsersControllerTest, AuthGatedRoutesTest) + Vitest render tests on every page (Welcome, Dashboard, Threads/Index, Threads/Show, ApiKeys/Index, Settings, Admin/Users, Admin/Models/Index + Edit, Runs/Debug, the 4 Errors/* pages, plus the underlying Components/ui/* primitive tests). Manual recipe in docs/m7-exit-criteria.md §1.
  • ✅ A logged-in user can: create a thread, fill out a prompt, select a model, set parameters, submit → see tokens stream into the debug pane on the right. Covered by ThreadController::store test (creates Pending run + redirects to threads.show), the chunk-7 ModelPicker Vitest (selection updates form.data.model_id), the chunk-8 ParameterControls Vitest (slider + input dispatch into form.data.parameters), the chunk-6a PromptPreviewController + budget-indicator Vitest (token-count + over-budget submit lock), the chunk-6b RunController::store dual-respond (Inertia redirect path), and the chunk-6b LiveStreamPane Vitest (active-run detection + useRunStream subscription + auto-reload on terminal status). The chunk-6/M6 streaming pipeline (StreamRunJob → broadcast events → useRunStream) carries the actual token-by-token delivery. Manual recipe in docs/m7-exit-criteria.md §2.
  • ⚠️ Lighthouse score ≥ 90 on the dashboard. Not unit-testable from this repo — requires a real browser. Manual recipe in docs/m7-exit-criteria.md §3 documents the Chrome DevTools Lighthouse run. The dashboard is server-rendered Inertia with minimal client-side work; shadcn primitives carry Radix accessibility defaults (focus rings, ARIA roles, sr-only labels on the mobile-nav Sheet). Treat any sub-90 finding as M12 (Accessibility + Polish) work rather than re-opening M7.

M7 retrospective

Sized at 7 days; landed in 10 chunks (1–5, 7, 8, 10 single; chunks 6 + 9 split into a/b) across a similar elapsed window. Quality bar held at every chunk boundary — 512 Pest tests / 1512 assertions, 181 Vitest tests across 24 files; Pint / ESLint / type-check / Vite build all green at every commit.

What worked

  • shadcn/ui adoption (chunk 1) frontloaded accessibility. Every primitive (Button, Card, Input, Label, Separator, Sheet, AlertDialog, Popover, Command, Slider, Textarea) wraps a Radix internal that handles focus return, escape-to-close, scroll lock, and ARIA wiring. We didn't have to think about it page-by-page.
  • Theme tokens over hardcoded colors. Moving the palette to CSS vars (:root + .dark in app.css) means a single source of truth. Chunks 2 + 9 surfaced the cost of NOT doing this — every hardcoded slate-* in the M1–M5 pages had to be swept.
  • 10-chunk split with sub-chunks (6a/6b, 9a/9b). Each commit was reviewable in under ~1k lines of diff. Chunk 6's six features (autosize / history / template / token count / warning / live pane) would have been unreviewable as one.
  • Server-side AJAX token-count preview (chunk 6a). Skipping client-side tiktoken-js WASM saved ~500KB from the bundle. The 400ms debounce + reuse of TokenCounterFactory keeps server + client in lockstep so the UI can't show a fits/over-by reading that disagrees with what RunService::submit would reject.
  • Primitives compounded. AlertDialog from chunk 2 powered chunk-5's thread delete, chunk-9a's API-key delete, AND chunk-9b's two admin confirms (refresh + delete). Each later use was a 10-line copy of the pattern, not a new dialog implementation.
  • Component-library tests catch contract drift early. When chunk 7 extended usable_models with supported_params, the Vitest fixtures broke at the TypeScript level — caught at type-check, not at runtime.

Surprises / footnotes

  • Each new Inertia page needs a Vite rebuild before assertInertia works. Pages/Threads/Show.tsx, Pages/Errors/*, Pages/Threads/Index.tsx all hit it. Pattern: write the route + controller + tests, expect a fail, build the page TSX, run npm run build, re-run Pest. Could be automated; not worth it for the cadence we're at.
  • jsdom polyfill churn. Chunk 4b needed an EventSource class polyfill (the standard vi.fn() isn't constructable). Chunk 7 needed ResizeObserver + Element.prototype.scrollIntoView for cmdk. All live in resources/js/test/setup.ts now.
  • vi.mock hoisting + vi.hoisted(). Mock factories run before top-level consts. Mock-state objects (mockStreamState.value, spy fns shared across tests) must be hoisted with vi.hoisted(). Bit me on chunk 4b, chunk 7, AND chunk 6b before I internalized the pattern.
  • JSON int/float round-trip. PHP encodes 0.0 as "0"; Inertia ships JSON; assertions came back as int. Bit chunk 3 (dashboard cost) and chunk 7 (model pricing). Documented the workaround in both tests.
  • Inertia useForm.post wants a redirect, not 201+JSON. Chunk 4a built RunController::store as a JSON API endpoint; chunk 6b had to add a dual-respond shim (X-Inertia header → redirect; otherwise → JSON). Worth doing the dual-respond from day one on any new POST endpoint.
  • react-hooks/immutability flags legitimate setState-in-effect patterns (debounce hooks, transport-fallback in useRunStream, the parameter-controls reset). The documented workaround is // eslint-disable-next-line react-hooks/set-state-in-effect with a rationale comment. Used sparingly — every disable carries a why.
  • Inertia view-finder is Vue-by-default. Chunk 4b's assertInertia(...)->component('...') failed until config/inertia.php declared the .tsx page paths. Recurs from M6 — should have been a foundational change in chunk 1.
  • react/no-unknown-property flags cmdk's cmdk-input-wrapper="" sentinel (chunk 7). Opaque styling hook for cmdk's CSS, not a real DOM attr. Disable-with-rationale.
  • jsx-a11y/label-has-associated-control can't see runtime htmlFor association on design-system primitives (the Label component in ui/). Inline disable on the primitive definition; callers pair via htmlFor and the rule is happy.

Decisions parked / deferred

  • Size filter on the ModelPicker (chunk 7). Dropped: we don't store parameter count, and a name-regex heuristic is too brittle. Arch + vendor + context-length sort + cmdk fuzzy search covers the use case. A future parameter_count column on models could resurrect the filter — file under M12 polish or a registry-schema chunk.
  • Full vendor-specific chat-template preview (chunk 6a). Partial via the history preview (shows the role/content array the model sees). Jinja-style template rendering using the models.chat_template column is deferred — useful for Anthropic/HF tinkerers, low priority for OpenAI-protocol vendors that handle templates server-side.
  • Display name on Settings (chunk 9a). SPEC mentions it; the users table lacks a separate display_name column (it has name, which the OAuth flow populates). Adding one would be a one-line migration + form field, but not a blocker; left for M12 / M15 polish.
  • Real Lighthouse-in-CI gating. Out of scope for M7 — would need a headless Chrome + Lighthouse-CI npm package + a separate workflow step. M14 deployment is a more natural fit if we want a CI gate; M12 polish would handle one-off audits.
  • E2E browser test harness (Playwright/Cypress). Same call as M6's closeout — unit + integration + manual recipes is enough. M12 might add Playwright for a11y audits.

Carry-forward into M8

  • LiveStreamPane is the placeholder. M8 replaces its JSON-per-line <pre> block with the real visualization: 3D transformer stack (Three.js), attention heatmaps (D3.js), MoE routing graph, logits panel, embedding-space token flow. The events array from useRunStream is the data feed; the shape is stable (per-event discriminated union from resources/js/types/runs.ts).
  • router.reload({only: ['runs']}) after a terminal status (chunk 6b) refreshes the transcript with the final output. M8 might want to keep the live pane in some form (debug toggle?) since the chunk-7 + chunk-8 controls keep producing valuable streaming detail (per-token logprobs, MoE expert IDs).
  • useDebouncedPreview (chunk 6a) keys off (threadId, prompt, modelId, max_tokens). M8 might add other params if the viz needs context (e.g., a "preview the viz given these settings" panel). Today's deps are sufficient for the budget bar.
  • All page chrome is shadcn-themed. M8's viz-controls (play/pause/scrub, layer-focus, attention-head-select) should use the same primitives (Slider, Button, Card, Popover) so the visual language stays coherent.
  • Per-thread links from the dashboard's recent-threads section (chunk 3) still don't link to /threads/{id} — chunk 3 deferred this to chunk 5, but I never went back to wire it. Quick follow-up: thread cards on the dashboard should link to their detail page. TODO for M8 or a polish pass.

M7 closed: 2026-05-19. Chunks 1–10 all green. 512 Pest / 1512 assertions, 181 Vitest across 24 files; build at 489 KB / 157 KB gzipped.


M8 — Frontend Live Visualization

Purpose: The right pane shows the real visualization, not the debug log. This is the vertical-slice gate.

Tasks Decisions (chunks 1–9):

  • Lazy-load Three.js via React.lazy + Suspense — keeps the ~600KB Three.js bundle out of the main app chunk so users without an in-flight run never download it. Vite splits VizPane into its own ~521KB chunk (132KB gzipped) as observed on chunk-1 build output.
  • Keep the chunk-6b debug pane behind a viewer toggle. Viz is the default; "Debug" tab swaps in the chunk-6b LiveStreamPane. Useful while M8 lands one chunk at a time — if a viz piece regresses, the debug fallback confirms events ARE arriving.
  • Lift the useRunStream subscription up to ThreadShow. Both views share one Echo channel; the toggle just swaps the renderer. Single source of truth for events.
  • Split into 9 chunks (1: foundation; 2: transformer stack; 3: token-flow particles; 4: live text + cost/TPS + KV bar; 5: D3 attention + logits; 6: MoE routing; 7: embedding scatter; 8: playback controls; 9: vertical-slice E2E + closeout). Chunk 2 absorbed what was originally going to be chunks 2 + 3 (stack + cascade) because the cascade animation has no value without the stack and the stack has no animation without the cascade — splitting felt like a contrived seam.
  • Cascade waves overlap; no queuing. The SPEC calls for layers lighting up sequentially per token. Implemented as 600ms waves walking bottom → top with no per-wave back-pressure. At 30 tok/s peak, ~18 concurrent waves visible — reads as a scrolling band, which is more legible than a single bouncing head trying to keep up.
  • HTML overlay (not in-scene Three.js text) for layer detail. WebGL text is expensive and inaccessible to screen readers. The overlay is a positioned <div> with an aria-labeled region and a real close button. Trade-off: it visually breaks the 3D illusion slightly, but the readability + a11y win is worth it.
  • total_layers + architecture_type come from the run's model_snapshot, not the live LlmModel row. Per SPEC §10.1, model rows can be deleted / re-keyed; the snapshot is the canonical record at run time. The dual-source pattern matters once admins start refreshing the registry.
  • 'active' (indigo) is for cascade; 'selected' (amber) is for the focused layer. Disjoint colors so the selection state survives a wave passing through it visually — the cascade just temporarily over-paints, then the selection re-paints on the next state-change effect.
  • Particles are streaks, not glowing spheres. (chunk 3) User choice — streaks read as "lightning" / flowing electricity, more cinematic than dots. Implemented as elongated boxes with a vertex-color gradient + additive blending rather than per-instance shader work; comet-tail effect comes from geometry, not GPU code.
  • Burst per token (5–10 particles), not 1:1. (chunk 3) Looks like activation flowing rather than a single moving dot. Density picked to read at 30 tok/s without saturating the 256-particle pool (worst case ~1500 active particles if all bursts overlap → pool cap is the visual ceiling, not a correctness requirement).
  • Straight-column trajectory + random X/Z jitter, not helix. (chunk 3) Spirals occluded the slabs they were passing — important since the slabs are the primary signal and particles are secondary. Jitter (±0.6 on each axis) gives the burst dispersion without overlapping the layer geometry.
  • Fade via scale ramp, not custom shader for per-instance alpha. (chunk 3) InstancedMesh has no built-in per-instance opacity uniform. Shrinking scale 1.0 → 0.0 over the last 800ms achieves the same visual outcome without forking the shader. If we ever need true per-particle alpha (e.g. logprob-driven brightness), revisit with a custom ShaderMaterial.
  • Live text inline on the active run row, not a separate "Live" card. (chunk 4) User choice — keeps the conversation flow uninterrupted; the row transitions seamlessly from streaming-with-cursor to static-with-final-tokens via the 400ms terminal-reload that already exists. Implementation cost is equal either way, but the inline approach avoids a phantom card popping in/out between runs.
  • Cumulative TPS, no rolling window. (chunk 4) User choice — smoother number, no per-event timestamp buffer to maintain, derivation stays a pure function of (events, run, model) with no ticker required. The displayed value updates only on new tokens, which is fine because cumulative-by-definition doesn't change between events.
  • Derive TPS from t_ms of the latest token, not wall clock. (chunk 4) Wall-clock would need a 250ms setInterval re-render. Using t_ms (server-set per event, stable between renders) keeps the row pure and avoids extra render churn. Trade-off: if WebSocket dies mid-stream the TPS freezes rather than continuing to drift; we'd rather show a frozen-but-real number than a wall-clock that lies about throughput.
  • Pricing is per-field-nullable. (chunk 4) pricing_input_per_million and pricing_output_per_million are independently optional — the cost calculation skips the null side rather than refusing to compute. A model with only output pricing still shows an output-only cost (cheaper than nothing). The entire cost row hides when BOTH are null, which is the only case where "$0.0000" would mislead.
  • Context bar shares the M7 budget-bar visual language. (chunk 4) Same color thresholds (amber at 80%, destructive at 100%), same h-1 / bg-muted track. The bar is the same widget pre-submit (M7 chunk 6a BudgetIndicator) and during-stream (chunk 4 LiveMetricsStrip) — users only learn one chart.
  • Attention heatmap is synthetic, labeled "Illustrative", and tied to layer depth. (chunk 5a) Vendor APIs ship logprobs but not attention tensors, and we'd rather show a pedagogically honest causal-decay pattern with a "vendor APIs don't expose attention" caveat than fabricate something that pretends to be real. Decay constant widens with layerIndex / totalLayers so the heatmap visibly differs across layers — that's what makes click-to-zoom worth doing.
  • Heatmap lives inside the layer-detail overlay, not as a top-level card. (chunk 5a) User choice. Keeps "all per-layer detail in one place" — the sub-component list + heatmap together. Avoids a phantom card cluttering the layout when no layer is selected. Trade-off: invisible by default; user must click a slab to see attention. Acceptable because the slab cascade already invites click interaction.
  • Logits chart hides entirely when logprobs are null, no placeholder. (chunk 5b) User choice. The fact that a vendor doesn't expose alternatives is itself information; a "no logprobs from this vendor" card felt like clutter. If we ever need the disclosure for support reasons we can add a single inline footnote on the live row.
  • Logits live below the live text on the active run, not in the viz pane. (chunk 5b) User choice. The chart answers "what alternatives did the model consider for the latest token?" — most useful next to the token text itself. Embedding it in the viz pane would have required a third tab and reduced its conceptual link to the streaming text.
  • Logprob → probability via per-K normalization, not softmax over full vocab. (chunk 5b) The vendor only returns the top alternatives; renormalizing across the truncated set gives bars that sum to 100% and answer "weight among the reported alternatives" honestly. Pretending the un-returned tail is zero would over-state the chosen token's confidence.
  • Whitespace tokens display via JSON.stringify. (chunk 5b) Most top-K alternatives at the start of a response are " ", "\n", or "." — rendering them raw would either collapse them (CSS whitespace) or insert real line breaks. JSON.stringify shows the literal escape ("\n") so the user actually sees what the model considered.
  • MoE routing co-locates with logits on the live run row. (chunk 6) User choice. Both views answer "what did the model just do for this token?", so they belong next to each other. Layer-detail overlay would be technically purer (MoE lives inside FFN of each layer) but the user can't see it without clicking, and per-token signal demands persistent visibility.
  • MoE router scores renormalize to 100%, same as logits. (chunk 6) User choice. Mirrors the chunk-5b decision — the vendor's per-expert score scale varies (some report post-softmax [0, 1], some report raw logits), so per-K normalization gives a uniform reading across vendors. Raw scores are still available on the MoERoutingExpert.rawScore field in case a later view wants them.
  • Utilization is one bar per expert, not a heatmap or text list. (chunk 6) User choice. Bars give the long-tail shape at a glance (a sharp concentration looks like a spike, dispersed routing looks flat). Tested explicitly to handle pools up to 160 experts (DeepSeek-V2) via flex-distributed widths — the grid stretches to fill, bar widths shrink, the visual story stays legible.
  • Bar-height normalization uses the per-run max, not a fixed scale. (chunk 6) An expert pool with sharp concentration would otherwise show a single visible bar and many invisible ones at a fixed scale. Per-max normalization preserves the comparative shape. Trade-off: a uniform-routing pattern looks the same shape as a concentrated one until the user reads the activation count. Tooltips on hover expose the raw count.
  • MoE component is mounted by the parent, not self-gated. (chunk 6) The component still tolerates non-MoE event streams as a safety net (returns null when no moe.routed events) but the actual gate is model.architecture_type === 'moe' in LiveRunBody. Keeps the component reusable (could appear elsewhere with different gating) and explicit in the parent about when it appears.
  • Embedding scatter uses synthetic clusters, not real PCA. (chunk 7) User choice. The SPEC called for "PCA-reduced 3D scatter (precomputed per tokenizer)" but real PCA requires running offline against ~500MB-1GB of HuggingFace model weights per tokenizer family. Stylized synthetic clusters give the same visual story (token types form spatial groups) without the data wrangling. The illustrative honesty comes from cluster labels and the "synthetic vocab map" footer caption. Real PCA can be a future swap-in — only the vocab data file changes.
  • Embeddings get a third top-level tab, not a sub-mode. (chunk 7) User choice. Considered putting Embeddings as a stack/embeddings camera-preset switch inside the Viz tab, but the scene infrastructure is different enough (Points cloud + spotlight vs slabs + cascade + streaks) that conflating them would have meant rebuilding the scene on toggle. Separate tab keeps the scenes independent and lazy-loadable.
  • Reduced-motion only disables the Viz tab, not Embeddings. (chunk 7) The transformer-stack scene auto-rotates continuously and runs the cascade animation — clearly motion-heavy. The embedding scene only auto-orbits when no spotlight is active, and the trail update is per-event (not per-frame). User can still benefit from the spatial view without continuous motion.
  • Token-to-vocab-point resolution falls back exact → trim → lowercase → trim+lowercase. (chunk 7) Most vocab tokens come from a tokenizer with leading-space variants (" the" vs "the"); some streams emit ALL-CAPS or Title-Case. The fallback chain catches the common variants. Misses are silently dropped — the scatter visibly skips them rather than fabricating a position for unknown tokens.
  • Per-vertex alpha via custom ShaderMaterial, not per-mesh InstancedMesh. (chunk 7) THREE.Points with a custom ShaderMaterial allows per-vertex alpha through a vertex-shader attribute; 280 points + alpha attribute is a single draw call. InstancedMesh would have been heavier (28 transformations per instance vs one attribute write per visited token) and harder to update incrementally without rebuilding the matrices.
  • Vocab lookup is a module-level singleton. (chunk 7) buildEmbeddingLookup() is invoked once per page load and cached in a module-scoped let. The vocab data is static, so rebuilding on every render would be pure waste. The trade-off (a singleton holds memory for the whole page) is fine for ~280 entries (~10KB).
  • Playback engine is shared across live + (future) M9 replay. (chunk 8) The hook treats events as opaque — same code path works for a live useRunStream array AND a replay array hydrated from runs.token_log. M9 just hands it a different events source and gets the same controls for free.
  • 1× = LIVE = head-sync, not a fixed-rate dispenser. (chunk 8) User choice. Means viz keeps perfect real-time fidelity at 1× (no delayed-by-buffering drift) while still allowing artificial throttle at 0.5× and rapid drain at 2× / 4×. Trade-off: 1× and 2×+ have qualitatively different mechanisms — the hook special-cases speed===1 with a simple sync useEffect. Single-mechanism would be cleaner but produces noticeably-delayed playback at 1× which the user didn't want.
  • Pause/step/speed affect ALL consumers, including the live Assistant text on the run row. (chunk 8) The transcript's live text reads from playback.visibleEvents, not raw stream events. Pausing freezes the rendered text mid-stream so the user can read what's there without it scrolling under them. Some users might expect pause to only affect viz, but uniform behavior is the simpler mental model — pause = "freeze this moment" everywhere.
  • Step skips to next token.received, not next event. (chunk 8) User choice. SPEC says "advance one token" and most users care about token granularity, not the cascade/MoE intermediate events. Side effect: stepping near the end of a stream can become a no-op if the only remaining events are non-token (e.g., final run.completed). That's acceptable — jumpToLive covers the "I want to see the rest" case.
  • Stream-shrink auto-resumes LIVE. (chunk 8) When the events array shrinks (new run starts, or SSE replays from cursor 0), the hook detects via prevLengthRef and resets cursor + speed=1 + playing=true. Avoids the surprising "I paused on the previous run and now this run is invisible" case. A future "remember playback state per run" option could opt out but isn't worth the complexity now.

Tasks

  • Three.js setup: scene, camera, renderer, OrbitControls (chunk 1). resources/js/Components/Viz/VizPane.tsx (lazy-loaded via React.lazy from Threads/Show) mounts a WebGLRenderer on a canvas ref, sets up a PerspectiveCamera, two lights (ambient + directional key), OrbitControls with damping. A ResizeObserver keeps the renderer's drawing buffer in sync with the canvas's CSS size (Three.js doesn't auto-resize). Animation loop guarded by a mounted flag so React strict-mode remounts don't leak requestAnimationFrame callbacks. Cleanup disposes geometries, materials, controls, and the renderer itself. Placeholder content: a slow-spinning wireframe icosahedron + empty-state overlay ("Submit a prompt to see the visualization") — replaced wholesale by chunk 2's transformer stack. FpsCounter (dev-only) sits absolute-positioned top-right of the canvas; rolling 30-frame window updating ~4×/sec. useReducedMotion hook subscribes to the prefers-reduced-motion media query and fires the auto-fallback in RightPane: when the user has motion reduced, the Viz tab is aria-disabled and the page boots into Debug; toggling the OS preference mid-session auto-flips to Debug. New jsdom polyfills (window.matchMedia + ResizeObserver/scrollIntoView from chunk 7) in test/setup.ts. The chunk-6b auto-reload-on-terminal effect lifted from LiveStreamPane to RightPane so it fires from any view. 8 new Vitest tests (default Viz mount, toggle aria + click flow, Debug → Viz round-trip, reduced-motion forces Debug + disables Viz tab, lifted stream forwards to both panes, single-subscription invariant, hook returns matchMedia at mount). The 6 chunk-6b tests that asserted on LiveStreamPane DOM updated to flip the Debug tab first.
  • 3D Transformer Stack component (chunk 2):
    • Renders N layers from the run's model snapshot. parameters.model_snapshot.layers is plumbed onto each run row in ThreadController::show() (so the viz survives a model row being deleted / re-keyed), through the RunRow TS interface, into VizPane as totalLayers. Falls back to 12 layers when the snapshot is missing.
    • TransformerStack (resources/js/Components/Viz/TransformerStack.ts) is a pure-Three.js builder: a Group of N BoxGeometry slabs centered on y=0, height normalized to 4 world units so a 12-layer Mistral and an 80-layer Llama-3 fit the same camera frame. State machine: per-slab base | active | hover | selected with immediate color swaps. Layer index lives on mesh.userData.layerIndex for raycaster hit-testing. 12 Vitest tests cover construction, normalization, state-color swap, out-of-range guards.
    • Cascade animation driven by CascadeController (resources/js/Components/Viz/CascadeController.ts). Each layer.advanced event spawns a 600ms wave that walks bottom → top through the stack; waves overlap (no queuing), so a model generating 30 tok/s reads as a scrolling band rather than a bouncing head. Controller owns a monotonic clock advanced by the animation loop's deltaMs — tests advance it deterministically without performance.now(). The events watcher tracks consumedEventCount so re-renders don't re-fire waves; a shrinking events array (different run / SSE replay) calls reset(). 7 Vitest tests cover single wave, expiration, multi-wave overlap, reset, 1-layer and 80-layer edge cases.
    • Click-to-zoom + sub-component overlay. Raycaster on canvas click resolves userData.layerIndex → React selectedLayer state. Camera lerps to a 3.2-unit offset focus position (8% per frame ≈ ~5-frame settle); orbit target lerps to the slab. The selected slab paints 'selected' (amber) — chosen so it doesn't conflict with the cascade's 'active' (indigo). HTML overlay (LayerDetailOverlay) anchors bottom-left of the canvas with the 5-step transformer-block sequence (RMSNorm → Attention → Residual → FFN → Residual); subComponentsFor(architectureType) swaps in 'MoE Router → Experts' for architecture_type === 'moe' runs. Close button + selection-clear on totalLayers change. 3 Vitest tests on the sub-component helper.
    • Auto-rotation of the stack group pauses while a layer is focused — once you've drilled in, the view stays still. Resumes when you close the overlay.
  • Token-flow particles (chunk 3): GPU-instanced streak trails. ParticleSystem (resources/js/Components/Viz/ParticleSystem.ts) maintains a 256-particle pool rendered as one InstancedMesh of thin Y-aligned boxes. Each box's BoxGeometry carries a per-vertex color attribute interpolating from black at the bottom face to cyan-300 (0x67e8f9) at the top — combined with AdditiveBlending + depthWrite: false this paints a comet-tail without a custom shader. Each token.received event spawns a burst of 5–10 streaks (5 + floor(rand() * 6)) at Y=−2 with random X/Z jitter within ±0.6. Streaks rise at 3 world units/sec (full-stack traversal ~1.3s), live for 2000ms total, and shrink from scale=1 → 0 between 1200–2000ms so the fade lands above the top slab. Per-instance opacity-via-shader was rejected in favor of scale-ramp fade — InstancedMesh has no built-in per-instance alpha and a custom shader felt premature for chunk 3. Pool semantics: spawnBurst() walks for inactive slots; pool exhaustion silently drops excess (the renderer is the visual cap, not the prompt). RNG is injectable so spawn-position tests are deterministic. 13 Vitest tests cover construction, kinematics (Y velocity), fade window, lifetime cutoff, slot reuse, reset, jitter bounds, vertex-color gradient.
  • Embedding space view (chunk 7). Lazy-loaded third tab in the right-pane toggle (Viz / Embeddings / Debug). Renders ~280 vocab tokens as a THREE.Points cloud with per-vertex colors driven by 8 hand-curated semantic clusters (whitespace, punctuation, pronouns, common-English, long-words, numbers, code-keywords, code-symbols). Each cluster has a 3D center and tokens are jittered by a deterministic xorshift32 seeded by (clusterIndex, tokenIndex) so positions are stable across reloads + tests. Custom ShaderMaterial gives each point per-vertex alpha so visited tokens fade from base intensity (0.35) to full (1.0) and a glowing spotlight sphere snaps to the latest matched token. Auto-orbit pauses once a spotlight is active. lib/embeddingHighlight.ts resolves incoming token strings to vocab indices via exact → trim → lowercase → trim-AND-lowercase fallback chain; misses are silently dropped. Vocab data ships in resources/js/data/embeddingClusters.ts (TypeScript module so it bundles cleanly with the lazy chunk). 9 Vitest tests on the data (cluster count, color format, deterministic build, positions within bounded scatter, cluster index assignment) + 11 on the highlight lib (exact wins, trim, lowercase, trim+lowercase, order preserved, duplicates kept) + 3 integration in Show.test.tsx (tab present + clickable, EmbeddingScene mounts lazily, reduced-motion still allows tab).
  • D3 attention heatmap component (chunk 5a; per-layer; illustrative). lib/attentionPattern.ts synthesizes a per-(token, layer) N×N matrix: causal lower-triangular, exponential distance-decay, per-row normalized, deterministic via xorshift32 seeded from (layerIndex, tokenCount). Decay constant grows with layer depth so the receptive field widens toward later layers — early layers attend locally, late layers attend broadly (a property tested explicitly: meanDistLate > meanDistEarly). Synthetic on purpose: SPEC §3.1.x notes that none of the supported vendor APIs expose attention tensors. AttentionHeatmap.tsx renders the matrix as SVG rects with a d3-scale linear interpolation from slate-950 → cyan-300 (palette-matched to chunk-3 particles). Mounted inside LayerDetailOverlay (chunk-2 click-to-zoom), capped at 24 tokens for legibility in the ~256px overlay width. Hidden when tokenCount === 0 (nothing to attend to yet). 9 Vitest tests on the pattern (shape, causal mask, row-stochastic, determinism, depth-effect) + 7 on the heatmap render (cell count, color extremes, caption, illustrative note, size prop).
  • D3 logits distribution component (chunk 5b; top-10 bars, live updates). lib/logitsExtract.ts walks the event stream backwards, returns the latest token.received with non-null logprobs as { chosenToken, alternatives: [{token, probability}] }. Logprobs → probabilities (exp(logprob)), then top-K normalization so the chart represents "weight among the reported alternatives" rather than pretending we have the full vocabulary tail. Components/LogitsDistribution.tsx renders top-10 horizontal bars with JSON.stringify(token) labels so whitespace / newline tokens stay legible (a \n token reads as "\n", not a line break). Chosen-token bar paints primary (indigo), alternatives muted. Mounted inside LiveRunBody between the live text and the metrics strip. Per the chunk-5 decision, returns null when no token has logprobs — no placeholder, no fabrication. 9 Vitest tests on the extractor (latest-wins, fallback-when-latest-is-null, normalization, top-K trim, skips non-token events, empty array). 7 tests on the component (null on no-logprobs, bar count, chosen highlight, whitespace escape, sum-to-100, top-K cap). 2 integration tests in Show.test.tsx.
  • MoE routing component (chunk 6). Two views stacked: (1) latest-token router scores as normalized horizontal bars (matches chunk-5b logits-chart visual), (2) cumulative expert-utilization as a horizontal grid of mini vertical bars, one per expert. Scales from Mixtral's 8 experts to DeepSeek-V2's 160 by letting flex distribute bar widths. lib/moeMetrics.ts exposes extractLatestRouting(events) (walks backwards, renormalizes the K reported scores to sum to 1, sorts descending) and extractUtilization(events, totalExperts) (counts per-expert activations, falls back to inferred max-expert-id+1 sizing when the model snapshot lacks moe_experts, defensively drops out-of-range expert IDs). The component is mounted by LiveRunBody only when model.architecture_type === 'moe' — dense runs skip it entirely (no placeholder, mirrors the chunk-5b logits-when-null decision). Header reads "MoE routing · N experts · top-K". 14 Vitest tests on the pure helpers (latest-wins, renormalization, sort, degenerate-event skip, fallback sizing, defensive ID range, non-MoE-event ignore) + 8 tests on the component (null on no events, header, bar count, normalized %, utilization counts, fallback sizing, unused-expert marking) + 2 integration tests in Show.test.tsx (mounts on MoE run, skipped on dense run).
  • Live text + cost/TPS + context bar (chunk 4 — fused 4/9–11/9 items into one chunk because they all read from the same derived metrics). The Transcript card for a pending/streaming run swaps its empty Assistant block for LiveRunBody: live token text concatenated from token.received events with a blinking cursor (, animate-pulse), and a LiveMetricsStrip showing N out · X.X t/s · $0.0000 plus a context bar used / model.context_length. Cumulative TPS (output_tokens / last_t_ms × 1000) — no rolling buffer, no wall-clock ticker, derivation invariant between events. Cost uses the model's pricing_input_per_million + pricing_output_per_million and gracefully omits when either field or the whole pricing object is null. Context bar percent crosses to amber at 80% and destructive at 100% (matches the M7 prompt budget bar). All derivation lives in resources/js/lib/streamMetrics.ts — a pure function returning { liveText, outputTokens, elapsedMs, tps, costSoFar, contextUsed, contextBudget } so the math is unit-testable without React. 12 Vitest tests cover the pure helper (no events, ordering, partial pricing, divide-by-zero, ignored non-token events). 6 Vitest tests on <ThreadShow /> cover the live row + cursor + metrics strip + the hide-bar-when-no-context-length / hide-cost-when-no-pricing / static-row-during-terminal-flicker paths.
  • KV cache progress bar (rolled into the chunk-4 context bar above — same denominator, same widget).
  • Playback controls (chunk 8). useEventPlayback(events) is a single source of truth: returns { visibleEvents, cursor, totalEvents, playing, speed, isLive, play, pause, toggle, step, setSpeed, jumpToLive }. visibleEvents = events.slice(0, cursor) flows to every viz consumer (Transcript, VizPane, EmbeddingScene, LiveStreamPane) — pause/step/speed affect the live text, the cascade animation, the particles, the heatmap, the logits, MoE routing, AND the embedding spotlight uniformly. Semantics per chunk-8 decision: 1× = LIVE (cursor syncs to events.length on every render via a useEffect — no setInterval, no artificial delay); 0.5× = throttle (setInterval dispenses one event every ~67ms = 15 events/sec, buffer grows when source is faster); 2× / 4× = drain (faster dispenser to catch up after a pause, stops at the head until more events arrive). Step finds the next token.received event and lands cursor on it — skips intermediate layer.advanced / moe.routed so "advance one token" matches the user's mental model. Stream-shrink (new run / SSE replay) auto-resets to head + resumes LIVE so a fresh run never displays a blank viz. PlaybackControls is a compact toolbar above the right-pane tab toggle: Play/Pause icon button, Step icon (disabled at head), LIVE pill OR cursor/total counter (clickable to jump back to live), 0.5× / 1× / 2× / 4× segmented control. 11 Vitest tests on the hook (renderHook + fake timers; LIVE-sync, pause-freezes, step-skips-non-tokens, throttle-at-0.5×, drain-at-4×, shrink-resets, jump-to-live, toggle, slice-matches-cursor). 11 tests on the component (icons, click-dispatch, disabled-step, LIVE pill, jump-to-live counter, speed-segmented). 4 integration tests in Show.test.tsx.
  • Live token-stream text panel (left side of split layout) — landed in chunk 4 above; the Transcript's active-run row IS the live panel.
  • Cost / tokens-per-second readout — landed in chunk 4 above (LiveMetricsStrip).
  • FPS counter (dev-only) — landed in chunk 1 (Components/Viz/FpsCounter.tsx), absolute-positioned top-right of every Three.js canvas (VizPane, EmbeddingScene). Rolling 30-frame window, updates ~4×/sec, gated behind import.meta.env.DEV so it disappears in production builds.
  • Reduced-motion fallback — landed in chunk 1 via useReducedMotion + the RightPane auto-flip. When the OS prefers-reduced-motion media query is set: the Viz tab is aria-disabled with explanatory title, the user is auto-flipped to the Debug tab on mount, and toggling reduced-motion mid-session re-routes to Debug. The Embeddings tab (chunk 7) stays available because its auto-orbit pauses once a spotlight is active and the trail update fires per-event, not per-frame — so the cumulative motion footprint is small.
  • Vertical-slice end-to-end test (chunk 9). Two complementary coverages: (1) programmatic kitchen-sink integration test in Show.test.tsx that fires a realistic mixed event sequence — run.started + 3 token.received (each with logprobs) interspersed with layer.advanced + moe.routed — through <ThreadShow /> for a Mixtral-shaped model, then asserts every viz piece responds: live text + cursor (chunk 4), metrics strip with TPS / cost / context bar (chunk 4), logits chart with chosen-highlight (chunk 5b), MoE routing with utilization counts (chunk 6), viz pane receives all events (chunks 1–3), Embeddings tab is mountable (chunk 7), PlaybackControls LIVE pill flips to cursor counter on pause (chunk 8). Single test, ~120 lines, asserts ~20 invariants. (2) Manual recipe documented below for human verification with a real vendor call + the 30-FPS exit gate.

Vertical-slice manual recipe (chunk 9)

Prerequisites: an API key on file for the chosen vendor; a thread created; a recent build of npm run dev running.

  1. Submit a prompt that produces ~100 tokens — e.g. "Write a 100-word explanation of quaternions" against OpenAI gpt-4o-mini or Together mixtral-8x7b.
  2. Watch the Viz tab (default). The transformer stack should render as N slabs; a 600ms cascade wave should walk bottom → top per token (chunks 2–3). Streak particles should rise from the base of the stack on each token (chunk 3). The slow Y-axis rotation should pause when a layer is clicked.
  3. Click a layer slab. The detail overlay should appear bottom-left with the 5-step sub-component list (RMSNorm → Attention → Residual → FFN/MoE → Residual) and a 24×24 attention heatmap below (chunks 2 + 5a). For a Mixtral run the FFN row reads "MoE Router → Experts" and an MoE badge appears next to the layer number.
  4. Watch the active run row in the transcript. Live token text should stream in with a blinking cursor; the metrics strip below should update with N out · X.X t/s · $0.0000 plus a context-used bar (chunk 4). For a vendor that returns logprobs, the top-10 alternatives bar chart should render below the live text with the chosen token highlighted (chunk 5b). For Mixtral, the MoE routing panel should appear with the latest token's router scores and a per-expert utilization grid (chunk 6).
  5. Switch to the Embeddings tab. The vocab scatter cloud should render with 8 colored clusters; as the stream progresses, matched tokens should fade up to full intensity and a glowing spotlight sphere should snap to the latest matched token's position (chunk 7). Auto-orbit pauses once a spotlight is active.
  6. Verify FPS. In dev mode, the FPS counter sits top-right of every canvas. Expected: ≥30 FPS during the 100-token stream on a 2020-era machine. (The cascade + particles + spotlight stay well under the limit for that hardware.)
  7. Exercise playback controls. Click pause — live text, viz, logits, MoE all freeze on the current cursor. Click Step — cursor advances to the next token.received event, and the whole stack moves one token forward (chunk 8). Click 0.5× then play — the dispenser slows to ~15 events/sec. Click the cursor counter — jumps back to LIVE at 1×.
  8. Switch model to a dense one mid-thread. Submit another prompt with gpt-4o-mini or llama-3.1-8b. The MoE routing panel should not appear on the new run (chunk 6 gate). Sub-component overlay's FFN row reads "FFN" instead of "MoE Router → Experts" (chunk 5a / subComponentsFor).
  9. Verify reduced-motion fallback. Toggle your OS preference (macOS: System Settings → Accessibility → Display → Reduce motion). The Viz tab should grey out, the page should auto-flip to Debug. Embeddings tab stays available.

If any step fails, the failure isolates to that chunk — open the relevant Components/Viz/*.tsx or lib/*.ts and re-run its Vitest suite.

Exit criteria — vertical-slice gate

  • Submitting a prompt produces a recognizable, smooth animation that visibly tracks the token stream. (Manual recipe steps 1–4; programmatic kitchen-sink test asserts every viz piece responds.)
  • Switching to a Mixtral run shows the MoE routing graph. (Chunk 6 gate; manual recipe step 8.)
  • Switching to a Llama-3 run hides the MoE graph. (Chunk 6 gate; manual recipe step 8.)
  • 30 FPS sustained for a 100-token stream on a 2020 MacBook Air. (FpsCounter overlay visible in dev mode; manual recipe step 6. Per-token cost: cascade is ≤8 layer-state writes per frame, particles are one InstancedMesh draw call, embedding scene is one Points draw + one Mesh.)
  • The visualization is compelling and the milestone proceeds to M9 (Replay + JSON export). No re-evaluation triggered.

M8 retrospective

What worked

  • Pure-function-first design. Every viz piece had a lib/*.ts or data/*.ts helper (streamMetrics, attentionPattern, logitsExtract, moeMetrics, embeddingClusters, embeddingHighlight) that was independently testable. The hot integration tests in Show.test.tsx cover wiring; the dense pure-function tests cover semantics. Total Vitest count grew from 47 at M7 closeout to 351 at M8 closeout — and the pure tests are the bedrock: they catch math regressions before the integration tests get involved.
  • Lazy-load discipline kept the main app bundle small. VizPane and EmbeddingScene each React.lazy-import. Three.js + custom shaders + the vocab data file all stay out of /dashboard, /threads, /api-keys until the user actually opens a thread. Vite split the Three.js dep into a shared chunk between the two viz tabs (~526KB / 132KB gzipped) once both started using it — Rollup recognized the shared module automatically without manualChunks config.
  • Custom ShaderMaterial for per-vertex alpha (chunk 7) beat InstancedMesh. Per-instance opacity isn't a built-in Three.js feature; alternatives are (a) custom shader, (b) per-instance matrix manipulation. For ~280 points, the shader was cheaper to write and faster at runtime (single attribute write per visited token vs. 16 transform-matrix writes).
  • Playback engine shape carries cleanly into M9 replay. useEventPlayback(events) is the entire abstraction — it doesn't care if events came from a WebSocket or runs.token_log. The chunk-9 manual recipe step 7 (pause / step / 0.5× / cursor-jump-to-live) is the exact UX M9 replay will inherit for free.
  • Synthetic over real data (chunk 7) was the right scope call. Real PCA per tokenizer would have meant downloading ~500MB-1GB of HuggingFace weights, running PCA offline, and committing a generated JSON per tokenizer family. For Phase 1's "vertical slice = compelling animation" goal, the spatial story (token types form groups) is what matters; provenance of the positions is secondary. The decision is documented and the swap-in point (data/embeddingClusters.ts) is clear.
  • Decision questions up-front, per chunk. Every chunk in M8 opened with AskUserQuestion covering 2–4 consequential decisions (data approach, layout, threshold semantics, etc.). Captured each answer in the chunk's Decisions block. Total decisions documented in M8: ~30 across 9 chunks. The decision log is the most-useful artifact for anyone re-reading the code six months from now — git log shows what changed, the decision blocks explain why.

What we learned

  • Math.max(8, ...) for setInterval intervals matters in tests. Chunk 8's hook math: 1000 / (BASE_RATE * speed) produces an 8.3ms interval at 4× speed. Floor-clamping to 8ms keeps the timer well-defined; the clamp is also the only place fake timers + the throttle path could deadlock. (Caught during the chunk-8 test pass.)
  • Lazy-loaded components need await screen.findByTestId not getByTestId for the first activation. A vi.mock-stubbed component still goes through React.lazy's Suspense cycle; subsequent renders are synchronous. The chunk-7 Embeddings tab test had to switch to await findByTestId once. Worth documenting if M9+ adds more lazy chunks.
  • textContent regex matches can pick up inline-style values in jsdom under some setups. The chunk-6 router-bar test originally matched (\d+\.\d+)% against row textContent and got 400% instead of 100% — the bar's inline style="width: 50%" was leaking into the match. Fixed by scoping the regex to the trailing label span via class. Generalized lesson: if a regex is summing percentages from a component that also uses % in inline styles, scope the query precisely.

Parked / carry-forward to M9

  • Replay route reuses the entire M8 stack. Replay/Show.tsx will look like a cousin of Threads/Show.tsx minus the prompt footer and the WebSocket subscription. The events source becomes runs.token_log (already persisted incrementally by StreamRunJob per M6 chunk 5a). useEventPlayback takes that array verbatim. The chunk-8 decisions about LIVE = head-sync, throttle, step granularity all still apply (just without the head moving — once a replay loads, totalEvents is fixed).
  • Real PCA data is an opt-in upgrade. If a user wants accurate per-tokenizer vocab geometry, the swap-in point is data/embeddingClusters.ts. The format (token + [x, y, z] + cluster index + color) is generic enough that real PCA output drops in by re-running the build script against a different source.
  • Vertical-slice E2E test is deliberately not a full Playwright/Cypress test. Per chunk-9 decision, that's M11 Hardening territory. The kitchen-sink integration test catches the integration-regression case the unit tests miss; the manual recipe gives the human verification path. Once we're confident in stability, browser automation can layer on top.

M9 — Replay + JSON Export

Purpose: Saved runs can be replayed without hitting vendor APIs. Runs and threads export to JSON.

Tasks

  • Replay route: /threads/{thread}/runs/{run}/replay (chunk 1). ReplayController::show authorizes thread owner + run-in-thread + terminal-status (422 for streaming runs), then renders Inertia 'Runs/Replay' with { thread, run, events, model }. The event stream is synthesized by app/Services/Runs/RunReplayEventSynthesizer — given a Run, it walks token_log and produces the same wire-shape RunEvent[] the live broadcast pipeline emits: run.started first, per-token token.received + layer.advanced (+ moe.routed for MoE), then run.completed / run.errored. Reuses RunEventEmitter so MoE expert routing stays deterministic per (run.id, token_index) — replays are frame-identical to live (SPEC §10.1). 9 Pest tests on the synthesizer + 8 on the controller.
  • Replay page reuses M8 components but reads events from runs.token_log rather than the WebSocket (chunk 1). resources/js/Pages/Runs/Replay.tsx is a standalone Inertia page: thread breadcrumb + run header, a prompt + re-streaming assistant body driven by useEventPlayback({ mode: 'replay', initialPlaying: false }), and the M8 right pane (Viz/Embeddings/Debug) sharing the same lazy-loaded Three.js bundle. The viz consumers (LogitsDistribution, MoERouting, VizPane, EmbeddingScene) need zero changes — they read from playback.visibleEvents exactly like the live page. The Debug tab is a simpler standalone variant (no transport label since there's no WebSocket). 8 Vitest tests on the page; per the chunk-1 decision, the initial state is paused-at-cursor-0 and the user clicks Play to start.
  • Replay launch — Replay button on each terminal run row in the thread detail page transcript (chunk 1). Renders a small pill-shaped Link with the Play icon next to the status badge for complete and error runs. Hidden for pending/streaming runs. 3 Vitest assertions.
  • Deterministic seeding verified — same run replayed twice produces frame-identical animation (chunk 2). Per chunk-2 strict-determinism decision, every visual signal — including the previously-Math.random-driven particle X/Z jitter and burst size — is now a pure function of token.received.payload.index. New lib/particleBurst.ts exposes burstForToken(tokenIndex) → { count: 5–10, seed } derived via xorshift32; ParticleSystem.spawnBurst(count, seed?) switched to use the seed when provided (falls back to constructor random for back-compat with existing tests). VizPane's events-watcher now calls burstForToken(e.payload.index) per token. Backend determinism (already strong from chunk 1 — MoE expert routing is hash-seeded by (run.id, token_index) via RunEventEmitter) is now exhaustively covered by ReplayDeterminismTest: byte-equal across instances, byte-equal across repeated build() calls, identical MoE routing for the same run, different routing for different run IDs, stable event ordering, dense runs emit zero MoE events, logprobs survive verbatim. 6 Vitest tests on the burst helper + 4 new on the seeded ParticleSystem + 8 Pest determinism cases.
  • JSON export: GET /runs/{run}/export.json (chunk 3). Owner-only download. Schema 1.0 top-level: { schema_version, exported_at, thread: { id, title, tags }, run: { full metadata + parameters (with embedded model_snapshot) + conversation_history + token_log } }. RunExportSerializer (app/Services/Runs/RunExportSerializer.php) is the pure-function source of truth; RunExportController is a thin owner-auth wrapper that sets Content-Disposition: attachment; filename="run-{id}.json" and pretty-prints with JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES. Excluded by design: user_id, model_id, api_key_id — exports carry zero user identity so the same payload is safe to forward to a future M11 sharing flow. Privacy: users.store_prompts=false already nulls prompt + conversation_history at write time; the export surfaces those nulls naturally with no extra redaction. 10 Pest tests on the serializer + 10 on the controller (auth, headers, schema-structure assertion, round-trip parse, MoE + dense + errored runs).
  • Thread JSON export: GET /threads/{thread}/export.json (chunk 4). Owner-only download. Schema 1.0: { schema_version, exported_at, thread: { id, title, tags, archived, created_at, last_activity_at }, runs: [run sections ordered by sequence_in_thread asc] }. ThreadExportSerializer builds the document; the per-run shape is byte-equal to chunk-3 because both serializers share a single RunExportSerializer::runSection(Run) helper. Per chunk-4 decision, all run statuses are included (pending / streaming / complete / error) — matches the chunk-3 single-run endpoint's policy. Empty threads return runs: []. Content-Disposition: attachment; filename="thread-{id}.json". 8 Pest tests on the serializer + 12 on the controller (auth, headers, schema structure, ordering, all-status inclusion, empty thread, round-trip, identity-field exclusion).
  • Download UI: button on run detail + thread detail pages (chunk 5). Three placements: (1) thread header — new outlined Export button next to Title / Archive / Delete links to /threads/{id}/export.json with download="thread-{id}.json"; (2) terminal run rows in the Transcript — small icon-only JSON Download pill next to the existing Replay link, links to /runs/{id}/export.json; (3) Replay page header — outlined Download JSON button on the right of the header bar. All three use plain <a href download> so the browser handles Content-Disposition natively — no JS fetch. Per chunk-5 decision, per-run buttons are terminal-only (consistent with the chunk-1 Replay placement); in-flight runs hide both buttons. 5 new Vitest assertions (3 in Show.test for run-row visibility + thread header + URL; 1 in Replay.test for replay-page download; 1 for streaming-row hidden).

Decisions (chunks 1–5):

  • useEventPlayback gains a mode: 'live' | 'replay' option. In 'replay' mode, 1× becomes a throttled dispenser (BASE_RATE × 1 events/sec) instead of LIVE head-sync. isReplay is exposed so the toolbar can suppress the LIVE pill. The hook's other semantics (pause, step-to-next-token, 0.5× throttle, 2×/4× drain, setCursor for scrubbing) work identically across modes. 5 new Vitest cases cover the replay-mode paths. (Decision: extending the existing hook over building a parallel one — same buffer model, same controls UX, M9 inherits chunk-8's playback engine.)
  • RunReplayEventSynthesizer reuses RunEventEmitter rather than reimplementing. Constructs a LlmTokenChunk from each token_log entry and runs it through the same emitter the live pipeline uses, then maps each Laravel event to its broadcastAs() + broadcastWith() pair. Means MoE expert routing's (run.id, token_index) → expert IDs hash stays consistent with what the original live broadcast emitted — replays match per SPEC §10.1. (Decision: synthesizer reuses emitter; doesn't persist layer/MoE events. Keeps token_log schema tiny; replay is a pure function of (run, token_log).)
  • Initial state: paused at cursor 0. User clicks Play to start. Considered: auto-play at 1× (more immediate) vs starts-at-end (full state visible). Paused-at-0 is the cleanest "give me a moment to set up the view" UX. The PlaybackControls toolbar's Step button works from cursor 0 so users can also walk through token-by-token from the start.
  • Replay page is standalone, not a flag on Threads/Show. Considered conditional logic inside the live page. Standalone keeps the prompt-footer + thread-title-editing logic on the live page only and the replay logic is much smaller (~340 lines including the RightPane inline). Worth a tiny duplication of the RightPane structure (tab toggle + lazy viz/embeddings + debug pane) — only ~80 lines.
  • Replay button only shows on terminal runs (complete/error). Pending/streaming runs have no stable token_log to replay. The button on errored runs still renders because partial output is replayable; users can see how the run progressed before failure.
  • 422 for non-terminal runs in the controller, not 404. Conveys "this is a real run but not in a state that supports replay" rather than "no such run." Semantically clearer error.
  • Strict frame-identity, not pragmatic "meaningful signal only". (chunk 2) User choice. Considered leaving decorative particle jitter on Math.random() and documenting it as "intended visual variety". Chose strict because: (a) the SPEC §10.1 text is unambiguous, (b) the implementation cost was small (~50 lines), (c) it sets up M10 (GIF export) for byte-stable encoding without further fiddling, and (d) the parallel between "live deterministic visuals" and "replayable deterministic visuals" is a useful guarantee anywhere we care to invoke it. The cost is that consecutive live submissions of the same prompt to a vendor will produce identical particle bursts on every replay — but live runs never produce identical token streams anyway because the vendor's stochastic decoding varies.
  • Burst params derive from token.received.payload.index, not from event-array position. (chunk 2) Token index is the semantic "this is the Nth generated token" identifier — survives any future change to event ordering or interleaving (e.g., if we ever batch layer.advanced events). Same-run live and replay both produce the same token indices in the same order, so the seed is naturally stable across the two paths.
  • ParticleSystem.spawnBurst keeps seed optional, not required. (chunk 2) Constructor random still works when no seed is passed. Preserves the 13 existing chunk-3 tests + the chunk-3 fixedRng dependency-injection pattern. Strict-determinism is opt-in at the call site; the test suite explicitly verifies both paths.
  • JSON export schema gets a schema_version: "1.0" field. (chunk 3) User choice. Cost is negligible (one extra key); benefit is large the day we want to ship a v2 (e.g., adding attention tensors when a future vendor exposes them). Consumers can dispatch on the version; M11 shared-link rendering can refuse versions it doesn't know.
  • Export uses the run's parameters.model_snapshot, not the live LlmModel row. (chunk 3) The snapshot is the canonical record at run time per SPEC §10.1; the live model row could be deleted or re-keyed by an admin. Embedding the snapshot keeps the export self-describing and replay-able forever regardless of what happens to the registry. model_id (FK) is deliberately omitted.
  • Export carries no user identity (no user_id, api_key_id). (chunk 3) Future M11 sharing wants to hand someone else this exact JSON without leaking who ran it. Stripping identity at the serializer means we don't have to write a redaction step later — the export is portable by construction.
  • Round-trip is schema-stability only for chunk 3. (chunk 3) User choice. No POST /runs/import endpoint yet — that needs UI scope, model-resolution policy ("what if the source model isn't in our registry"), and validation rules. Defer until M11 or whenever a real "import this thread" UI lands. The schema is documented + stable enough that import can be added later without breaking exports.
  • Content-Disposition: attachment not inline. (chunk 3) The endpoint exists for download UX, not in-browser viewing. Browsers respect attachment + filename so a click yields a run-{id}.json file instead of pretty-printed JSON in a tab.
  • Thread export includes all runs regardless of status. (chunk 4) User choice. Mirrors the chunk-3 single-run endpoint which also has no status filter. Pending / streaming runs surface with partial fields — that's honest. Trade-off: an in-flight run is visible in an export with token_log: [...] partial state; future consumers may want to filter. Adding a ?status= query param is a one-line follow-up if needed.
  • Per-run shape is shared between single-run and thread exports. (chunk 4) Extracted RunExportSerializer::runSection(Run): array as a static helper that both RunExportSerializer::build() and ThreadExportSerializer::build() call. Means a consumer that understands one shape understands the other — and any future per-run field addition (e.g., a vendor's attention tensor) lands in one place. Trade-off: the static is a small ergonomic friction, but in practice the serializer never holds per-instance state worth carrying.
  • No NDJSON / streaming response for large threads. (chunk 4) A thread with hundreds of runs × full token_log could reach 10MB; FPM-worker memory is fine at that size. A streaming application/x-ndjson response would help if exports grow to gigabytes, but that's not a real shape today. If it becomes one, the swap-in point is the controller — the serializer can yield runs one-by-one already.
  • Per-run download button is terminal-only, matches the Replay button placement. (chunk 5) User choice. The export endpoint itself accepts any status (chunk 3 decision), but the UI hides the button on in-flight runs. Rationale: visual consistency with Replay; downloading a half-baked token_log is rarely what users want. Power users can still hit /runs/{id}/export.json directly if needed.
  • Plain <a href download> instead of JS fetch / blob construction. (chunk 5) Browsers handle Content-Disposition: attachment natively when the link is a regular anchor. JS fetch would require building a Blob URL and revoking it — more code, more failure modes, no real upside. The download attribute also gives the file a sensible default name in browsers that ignore the server header.
  • Thread Export button uses shadcn Button with asChild. (chunk 5) Keeps the visual language identical to the adjacent Title / Archive / Delete buttons. asChild renders the Button's styles onto the inner <a> so we get the right outlined look without nesting buttons-in-buttons or losing the anchor's download behavior.

Exit criteria

  • Replaying a 100-token run shows the same animation as the original generation. Verified at the data layer by ReplayDeterminismTest (byte-equal events across instances, identical MoE expert routing per (run.id, token_index), stable event ordering — chunk 2). Verified at the rendering layer by the chunk-2 strict-determinism path: every visual signal (cascade timing, MoE routing, attention pattern, logits, embedding spotlight, particle burst size + jitter positions) is a pure function of payload.index. The manual recipe step 1–4 below covers the human verification.
  • Exported JSON re-imports cleanly (round-trip test). Verified by RunExportSerializerTest::it('round-trip: encodes to JSON and decodes back to an identical array') and the controller's round-trip parsing assertion, plus the thread-export equivalents.

Replay + Export manual recipe (chunk 6)

Prerequisites: a Run that's reached complete or error status. Quickest path: submit any prompt against any vendor (gpt-4o-mini is cheap + fast).

Replay

  1. From the thread detail page (/threads/{id}), find the completed run's row in the transcript. Click the Replay pill next to the status badge.
  2. The replay page (/threads/{id}/runs/{id}/replay) opens. Header shows back-to-thread + Download JSON; below sits the prompt and an empty assistant body with a blinking cursor — replay starts paused at cursor 0 per the chunk-1 decision.
  3. Click Play in the PlaybackControls toolbar. The viz, the assistant text, the logits chart, the MoE routing (if MoE), and the embedding spotlight all advance through the saved event stream. At 1× the dispenser fires ~30 events/sec; 0.5× slows to ~15; 2×/4× drain faster.
  4. Click Step while paused to walk one token at a time. Click Embeddings in the right-pane toggle to watch the vocab cloud light up matched tokens.
  5. Reload the page. The replay starts paused at cursor 0 again. Click Play. The particle bursts, cascade waves, and MoE routing arrive at the same positions / experts as the previous viewing — frame-identical per SPEC §10.1.

Export

  1. From the thread detail page, click the Export outlined button in the header (next to Archive / Delete). The browser downloads thread-{id}.json.
  2. Open the file. Top-level fields: schema_version, exported_at, thread, runs (array, ordered by sequence_in_thread).
  3. Back on the thread page, on any terminal run row click the JSON pill next to Replay. The browser downloads run-{id}.json. Top-level fields: schema_version, exported_at, thread, run (single object, same shape as one entry in the thread export's runs[]).
  4. On the replay page header, the Download JSON button hits the same single-run endpoint.
  5. Verify the export shape contains no user_id, model_id, or api_key_id — exports are portable for future M11 sharing.
  6. Try a non-owner replay or export URL (sign in as a different user, paste the URL) — should return 403.

M9 retrospective

What worked

  • Reusing M8's playback engine via a mode option. useEventPlayback({ mode: 'replay', initialPlaying: false, initialCursor: 0 }) is the entire frontend-side adaptation. The hook keeps a single buffer model and a single set of controls; replay just changes the meaning of 1× (throttle vs head-sync) and the starting state. 5 new Vitest cases were enough to validate the replay path; the chunk-8 live-mode tests stayed green throughout.
  • Token-log is the canonical replay source — no second persistence path. StreamRunJob already writes token_log incrementally (M6 chunk 5a edit). M9 didn't have to touch the streaming pipeline at all. The synthesizer just walks that array and reuses RunEventEmitter for the MoE / layer events, which inherits the live pipeline's (run.id, token_index) hash for free.
  • Strict frame-identity was cheap. Chunk-2 worried that seeding every viz signal might be a slog; in practice it was ~50 lines (lib/particleBurst.ts + ParticleSystem.spawnBurst seed path + the VizPane events-watcher line). The hard work — the MoE/layer determinism — was already done back at M6. The remaining randomness in particle positions was the only loose end, and a pure xorshift32 keyed on payload.index cleared it.
  • Schema-version-from-day-one paid off conceptually. Even though there's no v2 yet, the schema_version: "1.0" field in both export shapes documents the intent: consumers dispatch on it, version bumps are explicit. M11 sharing-link rendering can read this single field and refuse versions it doesn't understand.
  • Shared runSection helper between single-run and thread exports. Extracted in chunk 4 once both endpoints existed. Means a consumer that parses one export shape parses the other. Adding a future field (e.g., a vendor's attention tensor) lands in one place.

What we learned

  • Inertia view-finder caching pinned a Pest test failure. Chunk-1's ReplayController tests failed with "Unable to locate file in Vite manifest" until the Runs/Replay.tsx file existed on disk — even though we're rendering through Inertia not directly through Vite. The PHP-side view layer requires the chunk to resolve from the manifest. Easy unblock once spotted; worth documenting because the error message points away from the real issue.
  • Lazy-loaded components need findByTestId, not getByTestId. Already learned at M8 chunk 7 with EmbeddingScene; M9 chunk 1's Replay page reused the same pattern.
  • Button asChild with an anchor + download attr is the right Tailwind/shadcn idiom for download buttons. Avoids fighting with Link (Inertia routing) for non-Inertia URLs. The same pattern landed in three places in chunk 5.

Parked / carry-forward to M10

  • M10 GIF export inherits the entire replay infrastructure. The most natural implementation: spin up Puppeteer / Playwright, navigate to /threads/{id}/runs/{id}/replay, click Play at the desired speed, capture frames from the Viz canvas, encode via ffmpeg. The replay is already frame-identical and deterministic so the captured GIF will look the same on every render.
  • Synthesizer can emit alternate event types if needed. If M10 ever wants finer-grained intermediate events (e.g., "particle spawn at this t_ms" for sub-frame capture), the synthesizer is the place to add them — keeps the existing live + replay paths in sync.
  • JSON import endpoint is deferred. Per chunk-3 decision, no POST /runs/import exists yet. M11 sharing might want it (paste-a-link-to-import-a-run); the schema's stable enough that adding it is a validate + create + write token_log controller, ~50 lines.
  • Real-browser smoke test is not automated. Per chunk-6 decision, the manual recipe above is the verification path. Playwright/Cypress E2E lands in M11 Hardening (same call as M8).

M10 — GIF Export

Purpose: Completed runs can be downloaded as animated GIFs.

Tasks

  • Job: ExportRunGif (queued) — chunk 1 landed the foundation skeleton. app/Jobs/ExportRunGif.php is a ShouldQueue job constructed with a $runId (kept primitive so the queue payload stays small + survives DB refreshes between dispatch and worker). handle(GifRenderer, ExportStorage) cache-hits early when both export files exist, fetches the Run, and calls the renderer with a RenderConfig built from config('gif_export.frame_rate') + max_duration_ms. Single try / 5-minute timeout matches the SPEC. The supporting cast also landed: RenderConfig (immutable, 30 FPS / 5-min default with withFrameRate / withMaxDurationMs builders), RenderResult (gif/mp4 paths + framesCount + durationMs surface fields), GifRenderer interface (chunks 2 + 4 implement), NullRenderer placeholder that logs + throws so misconfigurations land in failed_jobs, GifRendererFactory (driver dispatch on gif_export.renderer config), ExportStorage paths + cache helpers (hasGif/hasMp4/bothExist, treats partial renders as cache miss). Service binding in AppServiceProvider lets the job type-hint the GifRenderer interface and get the configured implementation via the factory. 23 Pest tests across the 4 new test files.
  • SVG renderer (default GIF_RENDERER=svg) — chunk 2 lands the PNG-sequence half of the pipeline. Architecture: a new FrameRenderer interface (renderFrames(Run, RenderConfig, outputDir): list<string>) is implemented by SvgFrameRenderer. For each frame timestamp sampled at config.frameRate from token_log's last t_ms (per chunk-2 decision), it walks the log, derives state (live text, token count, last logprobs, attention pattern, MoE state-if-MoE), and emits an SVG with the 4 panels in a 1280×720 canvas: assistant text + metrics line + 12×12 attention heatmap + top-K logits bars + MoE routing strip (when applicable). Footer reads "2D summary view · frame N / M". Rasterization shells out to ImageMagick's convert CLI (chunk-2 decision: avoids the ext-imagick PHP-extension dep — apt install imagemagick is enough). A separate VideoEncoder interface lands its NullVideoEncoder stub (logs + throws); chunk 3 swaps in the FfmpegEncoder. SvgRenderer is the GifRenderer orchestrator: creates a tmp dir under sys_get_temp_dir(), runs the frame renderer, calls the encoder with the storage-relative gif/mp4 paths, cleans up on success AND failure (via try { … } finally { cleanup }). GifRendererFactory now resolves 'svg' to SvgRenderer via the container (so chunk 3 swaps the encoder binding without touching the factory). 14 new Pest tests across the 3 new files (7 frame renderer + 6 orchestrator + 1 null encoder).
  • [~] Puppeteer renderer (opt-in GIF_RENDERER=puppeteer) — chunk 4 skeleton landed per the chunk-4 scope decision. PHP-side contract is complete; Node script + /render route + frontend record mode ship in a follow-up:
    • ChromiumDetector — singleton service that resolves a Chromium binary via CHROMIUM_PATH env or a list of standard Linux/macOS locations. Cached per-process so the boot-time check runs once. 7 Pest tests on the resolution paths.
    • PuppeteerFrameRenderer implements FrameRenderer — checks the detector, then the Node script's existence at config('gif_export.node_script_path'). Raises ChromiumUnavailableException (a new custom exception) when Chromium is missing so chunk 6's fallback path can catch it specifically and re-route to the SVG renderer; raises plain RuntimeException for everything else (missing Node script, Node non-zero exit, zero frames produced). The shell-out shape — node <script> --run --out --fps --max-ms --chromium — is defined now so the follow-up Node script implementation has a stable contract. 5 Pest tests.
    • Factory wires 'puppeteer' to the same SvgRenderer orchestrator the SVG path uses (the orchestrator is renderer-agnostic per the chunk-2 design), with PuppeteerFrameRenderer injected instead of SvgFrameRenderer. Encoder + storage are shared verbatim.
    • Deferred to a follow-up chunk (real puppeteer integration): the actual Node frame-capture script + /runs/{id}/render Laravel route + signed-URL auth bypass for the spawned process + frontend record mode on Replay.tsx (autoplay, hidden controls, fixed viewport, window.__exportComplete signal).
  • ffmpeg shell-out: PNG sequence → animated GIF and MP4 (H.264). Both formats produced from the same frame sequence (chunk 3). FfmpegEncoder (app/Services/Exports/FfmpegEncoder.php) implements VideoEncoder. Three ffmpeg invocations per export, per chunk-3 decision: (1) PNG sequence → MP4 with libx264 / yuv420p / +faststart / scale=trunc(iw/2)*2:trunc(ih/2)*2 for web-compat + even-dimension x264 requirement; (2) PNG sequence → palette via palettegen=stats_mode=full for the chunk-3 "palette-optimized GIF" decision; (3) PNG sequence + palette → GIF via paletteuse=dither=bayer:bayer_scale=3 -loop 0 for clean dithering on slate gradients + infinite playback in Slack/Discord. Atomicity by ordering: storage writes happen only after all three passes succeed; a mid-pipeline failure leaves the disk untouched + the next dispatch is a clean cache miss. AppServiceProvider binds VideoEncoderFfmpegEncoder by default (chunk-2's NullVideoEncoder stays in the codebase as the operator-error fallback chunk 6 will use). 10 Pest tests on the encoder (3 invocations, libx264/palettegen/paletteuse flags, frame_rate plumbing, storage writes, atomicity on each-pass failure, missing-output-file guard, error message has pass name + exit code).
  • Per-export timeout: 5 minutes (chunk 1). The ExportRunGif job class declares public int $timeout = 300 so the queue worker kills any run that exceeds 5 minutes. RenderConfig::maxDurationMs (chunk 1, default 300_000) is the renderer-internal frame-count cap; the job timeout is the wall-clock guard above that.
  • Results stored at storage/app/exports/{run_id}.gif and storage/app/exports/{run_id}.mp4 (chunks 1–3). The ExportStorage helper from chunk 1 defines the paths; the FfmpegEncoder from chunk 3 writes to them via the configured gif_export.storage_disk (default: local, can be S3 in prod).
  • WebSocket completion event surfaces both download URLs; download UI offers a chooser (chunk 5). Backend: ExportCompleted + ExportFailed events broadcast on the existing private-runs.{id} channel (broadcast names export.completed / export.failed), with { run_id, gif_url, mp4_url, frames_count, duration_ms } and { run_id, message } respectively. POST /runs/{id}/export (ExportTriggerController) is owner-only — cache hit returns 200 + URLs + re-broadcasts ExportCompleted so other open tabs flip state; cache miss dispatches ExportRunGif and 202s. GET /runs/{id}/exports/{format} (ExportDownloadController) is owner-only, format constrained to gif|mp4, serves via Storage::download with image/gif or video/mp4 content type. The job dispatches ExportCompleted on success (including cache hit) and ExportFailed (then rethrows) on renderer crash. Frontend: useExportTrigger hook POSTs to /runs/{id}/export, handles immediate-ready (cache hit) vs queued (subscribe + wait for the broadcast); state machine idle → rendering → ready | error. ExportDownloadMenu is a Popover-based chooser with three items: JSON (instant link to chunk-3 M9 endpoint), GIF + MP4 (trigger the hook + animate to download links on completion). Mounted on terminal run rows in the thread transcript + replay page header. Per chunk-5 decision, cache hits go straight to 'ready' (no flicker); WebSocket beats polling. 21 new Pest tests (6 events + 6 trigger controller + 6 download controller + 3 job event-dispatch) + 16 new Vitest (9 hook + 7 menu) + 2 integration assertions updated. Full suite: 660 pest + 398 vitest.
  • Renderer fallback: if puppeteer configured but Chromium unavailable at boot, log warning and fall back to SVG with a "fallback engaged" badge (chunk 6). Backend: GifRendererFactory::make('puppeteer') checks ChromiumDetector::isAvailable() before constructing. If the binary's missing, it logs a one-time warning (GIF_RENDERER=puppeteer configured but no Chromium binary found...) AND silently returns the SVG-bound renderer instead. The factory exposes fallbackEngaged() as a per-make() flag — ExportRunGif::handle() re-resolves the factory after rendering to read it, then includes fallback_engaged: bool in the ExportCompleted broadcast payload. ExportTriggerController does the same on the cache-hit branch so the response body + the re-broadcast both carry the flag. Frontend: useExportTrigger exposes a fallbackEngaged field; ExportDownloadMenu renders a small italic amber "(2D fallback)" badge next to the GIF and MP4 menu items when the flag is true — visible during both the 'rendering' and 'ready' states so the user knows what to expect AND knows what they're downloading. Hidden when the puppeteer driver isn't configured at all (just the default SVG path). 8 new Pest tests (3 factory + 1 event + 1 trigger controller + 3 chunk-6-specific) + 4 new Vitest (2 hook + 2 menu) — total chunk-6 additions: 12 tests. Full suite: 665 pest + 402 vitest.

Export manual recipe (chunk 6)

Prerequisites: a Run that's reached complete or error status (the JSON / GIF / MP4 trigger UI is terminal-only — same gate as Replay). Backend deps: imagemagick (the convert binary) + ffmpeg, both standard on Linux + macOS.

Trigger an export

  1. From the thread detail page or the replay page, find the completed run. Click the Download dropdown next to the status badge / on the replay header.
  2. The menu opens with three items: JSON (instant download — chunk-3 M9 endpoint), Animated GIF (.gif), MP4 video (.mp4).
  3. Click Animated GIF OR MP4 video — both share the same render. The menu item flips to a spinner with "(rendering…)" while the job runs.
  4. When ExportCompleted arrives on the WebSocket, both GIF and MP4 menu items become anchor links. Click either to download.

Verify cache hit

  1. Click Download again on the same run.
  2. The menu items are anchor links immediately — no spinner. The chunk-1 cache-hit short-circuit in ExportRunGif (and the controller's bothExist check) returns the URLs from disk without re-rendering.
  3. Verify storage/app/exports/{run_id}.gif and .mp4 exist + match the timestamps of the last render.

Verify multi-tab sync

  1. Open the same thread in two browser tabs.
  2. Tab A: click Download → GIF. Wait for the render.
  3. Tab B (still on idle): refresh the dropdown. JSON downloads instantly; GIF / MP4 either show immediately as anchors OR (if Tab B opened the menu before Tab A's render completed) the chunk-5 broadcast flipped them.

Verify the chunk-6 Chromium fallback

  1. With GIF_RENDERER=svg (the default), trigger an export — no fallback badge appears.
  2. Set GIF_RENDERER=puppeteer in .env, restart the workers, and trigger again on a host without Chromium installed. The menu items show "(2D fallback)" next to the format label.
  3. storage/logs/laravel.log shows: GIF_RENDERER=puppeteer configured but no Chromium binary found. Falling back to the SVG renderer for this process. Install Chromium .... The warning fires once per worker process (the chunk-6 latch); subsequent exports don't re-spam.
  4. Install Chromium (apt install chromium-browser or snap install chromium) and restart. The badge stops appearing — though the actual Puppeteer-driven path still throws "Node script not found" until the chunk-4-deferred follow-up lands.

M10 retrospective

What worked

  • Skeleton-first chunk 1 was the right call. Foundation (job + interface + storage + config + factory + DI bindings) landed in a single tight chunk; chunks 2–4 then swapped in concrete implementations without touching anything else. The NullRenderer/NullVideoEncoder stubs gave clear "not yet wired" failure modes during development.
  • Splitting FrameRenderer from VideoEncoder was load-bearing. The chunk-2 architecture decision meant chunk 3 (ffmpeg) was completely orthogonal to chunk 4 (Puppeteer), and chunk 6's fallback boils down to a single match-arm swap inside the factory. If we'd built one monolithic Renderer per format, each chunk would have rewritten the previous one.
  • Reusing the SvgRenderer orchestrator for the Puppeteer driver. The renderer-agnostic orchestrator + the FrameRenderer interface meant chunk 4's "puppeteer" wiring was ~10 lines in the factory. The chunk-6 fallback was the same lines, just with a different fallback path.
  • ChromiumUnavailableException as a distinct catchable signal. Chunk-4 decision that paid off in chunk 6: the fallback only triggers on this specific exception, leaving every other puppeteer-path failure (missing Node script, Node crashed, zero frames produced) propagating to failed_jobs for operator inspection.
  • Per-chunk decision documentation. Each chunk's decisions block tracks not just what we built but what we considered and rejected — and why. M10 alone documents 28 decisions; the rationale is now searchable and reviewable.
  • Process::fake closures over real binaries in tests. Letting the test suite mock convert + ffmpeg via the Process facade kept the build green on hosts that don't have ImageMagick (chunk 2) and on CI without the binaries (chunks 3, 5). The chunk-2 dev lesson "use string commands not arrays so Process::fake patterns + assertRanTimes closures work" carried through chunks 3 and 4.

What we learned

  • Pusher tries to broadcast even with ShouldBroadcastNow if Event::fake() isn't engaged. Chunk 5a's first integration tests against the trigger controller failed with "Pusher error: auth_key should be a valid app key" because ExportCompleted is ShouldBroadcastNow. Fix: Event::fake([ExportCompleted::class]) at the top of every controller / job test that exercises the broadcast path. Documented now for future broadcast-event tests.
  • Pint reformats files and breaks subsequent Edit calls. When the commit hook runs Pint, edits made within the same task window that don't match the new whitespace fail. The recovery pattern: re-Read the file, then re-do the Edit. Worth remembering when chaining many edits.
  • escapeshellarg flips array-form vs string-form Process::fake matching. Discovered in chunk 2; same pattern applies to ffmpeg in chunk 3. Always use string commands when you need to assert against them.

Parked / carry-forward to Phase 2 + future chunks

  • Real Puppeteer integration is deferred per chunk-4 decision. Skeleton + fallback-detection ships in Phase 1; the actual Node frame-capture script (node-scripts/puppeteer-export.cjs), /runs/{id}/render Laravel route, signed-URL auth bypass for the spawned process, and frontend record mode on Replay.tsx (autoplay + hidden controls + fixed viewport + window.__exportComplete signal) all need to land before Phase 1's "Optional Puppeteer install produces a 3D-accurate GIF + MP4" exit criterion is fully met. Phase 1 ships with the SVG renderer as the only working backend; 3D-accurate exports are a Phase 2 feature.
  • No S3 disk verification yet. gif_export.storage_disk defaults to local. Production deployments will likely want S3 (M14 territory). The ExportStorage + FfmpegEncoder already use Storage::disk(config('gif_export.storage_disk')) so switching is a .env change — but we haven't tested the S3 path end-to-end.
  • No browser-level smoke test of the actual GIF/MP4 output yet. The Pest suite mocks ffmpeg via Process::fake, so we haven't exercised the real encoding pipeline against a real PNG sequence. M11 Hardening should add a non-mocked integration test that uses the real binaries.
  • SvgRenderer is a misleading name once it also serves Puppeteer. Chunk-4 decision called this out; the rename to GifMp4Renderer (or PngSequenceRenderer) is pending. Not worth the churn yet — all consumers go through the factory — but worth doing alongside any future M11/M12 renderer additions.

Decisions (chunks 1–2):

  • Skeleton-only scope for chunk 1. User choice. No real rendering yet — both svg and puppeteer driver values currently resolve to NullRenderer which throws on render(). Trade-off: an export job dispatched today fails fast with a clear message. Benefit: the pipeline (job + renderer interface + storage + cache short-circuit + DI binding) is testable + reviewable before any pixel work lands. Chunks 2 (SVG) and 4 (Puppeteer) swap the resolution in GifRendererFactory::make() without touching the job, the storage helper, the config, or the binding.
  • Cache: return existing file if present. User choice. ExportStorage::bothExist(runId) short-circuits the job when both .gif and .mp4 are on disk. Repeated downloads cost a single filesystem stat; the renderer + ffmpeg cost is avoided. Partial renders (one file present, the other missing) are treated as a cache miss so the next dispatch re-renders cleanly — covers the "ffmpeg died mid-encode" edge case.
  • 30 FPS default. User choice. Matches the M8 animation target + the manual recipe steps. Configurable via GIF_FRAME_RATE env so per-deployment tuning stays out of code. The RenderConfig value object exposes maxFrames() for renderers that need a hard frame-count cap (frame_rate × max_duration_ms / 1000).
  • Run ID, not Run model, in the job constructor. Keeps the queue payload tiny (['runId' => 42]) and skips any SerializesModels stale-data concern between dispatch + worker. The worker re-fetches via Run::find($this->runId) and exits silently if the row vanished (delete-after-dispatch is the only realistic case).
  • GifRenderer interface, not abstract class. Two implementations land in different chunks (chunks 2 + 4) with very different shapes (PHP+Imagick vs Node+ffmpeg). The interface is the minimum shared API — render(Run, RenderConfig): RenderResult. Container binding via AppServiceProvider resolves the configured concrete; tests use app()->instance(GifRenderer::class, $fake) to swap.
  • Single tries=1 + failed_jobs over retries. A failed render is operator-actionable (Chromium missing, ffmpeg path wrong, run.token_log corrupted), not transient. Retry-blasting wastes CPU. The 5-minute timeout (matches SPEC) is the wall-clock guard; failures land in failed_jobs for inspection.
  • ImageMagick convert CLI shell-out instead of ext-imagick. (chunk 2) User choice. SPEC literal text says Imagick the PHP extension; in practice the dev environment has the convert binary but not the extension, and every Linux deployment already has ImageMagick installed. Switching to shell-out drops the apt install php-imagick step from every host and avoids the PHP version × Imagick version compatibility matrix. Cost is ~10ms per-frame process spawn — for a 90-frame run that's ~1s overhead; ffmpeg is shell-out anyway.
  • Frames sampled at frame_rate from token_log timestamps, not per-event. (chunk 2) User choice. 30 FPS over the last token's t_ms produces a fixed frame count regardless of layer.advanced / moe.routed density. Output duration matches the original run's wall-clock duration — the GIF plays at "real time." Edge case: zero tokens → one static frame at the empty-state.
  • FrameRenderer + VideoEncoder interfaces (renderer architecture split). (chunk 2) User choice. Separating "produces PNGs" from "encodes PNGs to GIF+MP4" means: (a) chunk 2 stands alone with a NullVideoEncoder stub that throws clearly until chunk 3 lands the real FfmpegEncoder, (b) chunk 4's PuppeteerFrameRenderer reuses the same encoder, (c) chunk 6's fallback path (Chromium unavailable → fall back to SVG) just swaps the FrameRenderer container binding. Trade-off: more files, but each one has a single concern.
  • SVG composed as raw string concatenation, not via an SVG library. (chunk 2) PHP doesn't ship an SVG builder; pulling one in for ~5 rect / text / line element types was over-scope. The frame composition is in SvgFrameRenderer::composeSvg as a single HEREDOC; any future template engine swap-in lives in that one method.
  • Process::run with a string command + escapeshellarg, not the array form. (chunk 2) Laravel's Process::fake matches patterns against the command string. Passing an array makes $process->command an array at assertion time, which broke Process::assertRanTimes closures with type errors during dev. String form + escapeshellarg is the simpler + more testable path.
  • Tmp dir cleanup in a try { … } finally { cleanup() }. (chunk 2) Encoder failures (chunk 3 ffmpeg crash, malformed PNG) must not leak the scratch dir. Test explicitly verifies cleanup on both success + throw paths.
  • GifRendererFactory takes a Container in its constructor. (chunk 2) Resolves SvgRenderer via $container->make() so its FrameRenderer + VideoEncoder + ExportStorage dependencies get auto-wired. The factory becomes a thin driver-name → service-resolution dispatcher; future drivers add a single match arm.
  • Three separate ffmpeg invocations, not a single -map graph. (chunk 3) User choice. Each pass has a clear concern (MP4, palette, GIF) and a clean failure surface — the exception message names which pass failed + the ffmpeg stderr. A -map multi-output graph would shave ~1 second per export but every filter-chain change has to consider both output streams together, and ffmpeg version differences in the multi-output path are a real maintenance hazard. Two separate processes is the operational hygiene choice.
  • Palette-optimized GIF (two-pass) over single-pass default. (chunk 3) User choice. palettegen stats_mode=full reads every pixel of every frame to build the optimal 256-color palette, then paletteuse re-encodes with dither=bayer:bayer_scale=3. ~2-3× slower than ffmpeg's default GIF path; the output looks dramatically better on the slate-dark backgrounds + cyan particle streaks we have to render. Single-pass would visibly band the gradients.
  • Atomicity by ordering: storage writes after every pass succeeds. (chunk 3) The encoder writes to the configured storage disk (gif_export.storage_disk) only after all three ffmpeg invocations have succeeded AND both tmp files exist on disk. A pass-1 failure leaves the storage untouched; a pass-2/3 failure also leaves it untouched (we don't write the MP4 until both the MP4 + GIF tmp files exist). ExportStorage::bothExist() then drives a clean cache-miss on the next dispatch — no half-baked artifacts get served from storage.
  • file_get_contents then Storage::put (full file in memory). (chunk 3) The artifact size class is ~5–50MB per file; PHP memory handles this easily. Streaming chunks would be more memory-friendly but adds complexity; if exports ever grow past a few hundred MB the swap-in point is single-method.
  • -y flag everywhere for overwrite-on-collision. (chunk 3) The tmp dir is fresh per export (chunk 2's try { … } finally cleanup) so collisions shouldn't happen, but a flaky cleanup + immediate re-dispatch could leave a stale output.mp4. -y makes ffmpeg overwrite silently rather than prompt-and-hang in stdin.
  • Puppeteer chunk 4 is skeleton + fallback-detection, not full integration. User choice. Real headless-Chrome rendering would add ~150MB Chromium download, a Node runtime requirement, an /render route with auth-bypass for the spawned process, a frontend record-mode flow, and a frame-capture script. The skeleton's value is establishing the boundary: ChromiumUnavailableException is the catchable signal chunk 6 needs; the Node shell-out command shape is stable so a follow-up chunk drops in node-scripts/puppeteer-export.cjs without re-architecting. Phase 1 ships with the SVG renderer as the only working backend; 3D-accurate exports land in Phase 2 or driven by user feedback.
  • ChromiumUnavailableException extends RuntimeException. (chunk 4) Distinct exception type so chunk 6's fallback can catch (ChromiumUnavailableException $e) specifically — every other puppeteer-side failure (missing Node script, Node crashed, zero frames) raises plain RuntimeException and propagates to failed_jobs. The fallback only kicks in for the "Chromium missing" case because that's the deployment-level config issue; everything else is an operator-actionable bug.
  • Reusing SvgRenderer as the orchestrator for 'puppeteer' too. (chunk 4) The orchestrator is renderer-agnostic — it takes any FrameRenderer, plus the encoder + storage. The factory's 'puppeteer' arm constructs an SvgRenderer with a PuppeteerFrameRenderer injected. The "Svg" in the class name is misleading; a future rename to GifMp4Renderer is an option but not yet worth the churn since all consumers go through the factory.
  • ChromiumDetector cached on a singleton. (chunk 4) The binary check runs once per process — isAvailable() is called from PuppeteerFrameRenderer::renderFrames which fires once per export job. Caching avoids a which shell-out + is_executable storm on hosts that don't have Chromium. reset() exists for test ergonomics.
  • WebSocket completion signal over polling. (chunk 5) User choice. Matches the M6 / M8 viz pipeline shape — same channel (private-runs.{id}), same event-name kebab convention (export.completed), same auth gate. Polling would have been simpler frontend code but added ongoing backend load whenever the menu was open. The WebSocket also enables the "other tabs flip state" behavior: a cache hit on one tab broadcasts on the channel so any other open tab subscribed to the same run sees the chooser update too.
  • Single chooser dropdown over multiple format-specific buttons. (chunk 5) User choice. Replaces the chunk-5 M9 single-format JSON pill with a Popover that lists JSON / GIF / MP4. Saves horizontal space on dense run rows, groups the three formats together cognitively, and lets the rendering state animate in one place rather than across multiple buttons. Trade-off: one extra click to download.
  • Cache hit → immediate download (no 'Rendering...' flicker). (chunk 5) User choice. Frontend hook flips directly to 'ready' when the trigger endpoint returns 200 with URLs; only the 202 path subscribes to the channel and shows the spinner. The cache-hit branch also re-broadcasts ExportCompleted so any other open tabs (on the same run's channel) flip too.
  • GIF + MP4 share a single trigger state. (chunk 5) The chunk-3 encoder produces both formats from one PNG sequence in one job invocation. The frontend mirrors this: one useExportTrigger instance per run; clicking either format triggers the same render; both download links light up together when the broadcast arrives. Avoids a redundant second job dispatch for the second format.
  • Cache hit re-broadcasts ExportCompleted from the controller. (chunk 5) The trigger controller fires the event AND returns the URLs in the response body. Belt-and-braces: the response handles the requesting tab; the broadcast handles every other tab. Without the controller-side broadcast, a cache hit on tab A would silently leave tab B's chooser still in 'idle'.
  • ExportFailed payload is { run_id, message } only. (chunk 5) No stack trace, no file paths, no PHP internals exposed to the browser. The full exception still lands in failed_jobs + Laravel logs for operator inspection; the broadcast is just a clean user-facing signal.

Exit criteria

  • Default install (SVG renderer) produces both GIF and MP4 for a 100-token run in < 30 seconds. Verified end-to-end via the chunk-3 FfmpegEncoder test path; real-binary wall-clock not measured (deferred to M11 Hardening's non-mocked integration test).
  • [~] Optional Puppeteer install produces a 3D-accurate GIF + MP4. Skeleton + fallback-detection done per chunk-4 decision; actual Node frame-capture script + /render route + record mode are Phase-2 / future-chunk work. Documented above.
  • Fallback path verified by intentionally removing Chromium. Chunk 6's it falls back to SvgRenderer + sets fallbackEngaged when Chromium is missing Pest test reproduces the exact scenario; the chunk-6 manual recipe walks through the operator-visible verification.
  • [~] MP4 plays in Chrome, Firefox, Safari; GIF renders inline in Slack and Discord. The libx264 / yuv420p / +faststart (chunk 3) + paletteuse + -loop 0 flags are the industry-standard set for these targets, but per-client smoke testing requires real artifacts in those clients. Verified via flag inspection in Pest; per-browser/per-client testing is M11 Hardening / M15 Launch Prep work.

M11 — Thread Sharing

Purpose: Users can share a thread read-only via a public URL.

Tasks

  • Share toggle UI on thread detail (chunk 3). New ShareMenu Popover button in the thread header next to Title / Archive / Export / Delete, per chunk-3 decision. State surface: when sharing is OFF, shows a single "Enable sharing" button that POSTs to /threads/{id}/share; when ON, shows the full share URL in a readonly input + a Copy button (uses navigator.clipboard.writeText with a 2-second "Copied!" checkmark; falls back to window.prompt when clipboard is blocked) + a single-click "Disable sharing" button that DELETEs the same route. A small green pulse-dot indicator on the trigger button signals "Sharing is ON" without opening the popover. Per chunk-3 decision the disable path has no confirmation dialog — re-enabling is one click away if the user changes their mind (just with a fresh token per the chunk-1 always-regenerate policy). ThreadController::show now plumbs share_token + share_enabled_at into the thread props; Pages/Threads/Show.tsx passes both to ShareMenu along with window.location.origin so the URL surfaces as a real absolute string. 9 Vitest tests on the menu + 4 integration tests on Show.test.tsx (button present, ON indicator, Enable fires router.post, Disable fires router.delete) + 2 Pest tests verifying the new thread props.
  • Endpoint: POST /threads/{thread}/share (chunk 1). Owner-only. Always generates a fresh token per chunk-1 decision — re-enabling after disable produces a new URL so any accidentally-leaked old link stays dead. Sets share_token (32 hex chars / 128 bits entropy from ShareTokenGenerator) + share_enabled_at = now(). Redirects to /threads/{id} so the toggle UI re-renders from updated Inertia props.
  • Endpoint: DELETE /threads/{thread}/share (chunk 1). Owner-only. Nulls both share_token + share_enabled_at. Idempotent: DELETE on an already-unshared thread is a no-op redirect, not 404.
  • Public route: /share/{token} — bypasses auth, IP rate-limited to 60/min (chunk 2). SharedThreadController::show($token) resolves via whereNotNull('share_token')->where('share_token', $token); 404 on miss. Rate-limited via a new share RateLimiter (Limit::perMinute(60)->by($request->ip())) applied to the Route::middleware('throttle:share')->prefix('share') group — same budget covers both the thread reader AND /share/{token}/runs/{run}/replay per chunk-2 decision.
  • Read-only thread view (chunk 2). New Pages/Share/Show.tsx standalone page per chunk-2 decision (no readOnly flag on Threads/Show.tsx). Custom inline SharedLayout replaces AppLayout so anonymous viewers don't see the authenticated sidebar; layout has a brand header + a footer "What is this?" link to the about page. Sub-requirements:
    • No prompt input ✅ (page has no PromptFooter)
    • No share toggle ✅ (controller only sends sanitized props; no header buttons)
    • Replay available per run ✅ — terminal-only Replay pill on each run row → /share/{token}/runs/{id}/replay
    • Prompts redacted if owner had store_prompts = false ✅ — SharedThreadController substitutes [prompt redacted by author] for any null prompts when the owner's store_prompts is false; the page also shows a top-of-thread redaction notice
  • /share/{token}/runs/{run}/replay — public replay (chunk 2). SharedReplayController mirrors the M9 owner replay but: (a) auth-bypassed; (b) cross-thread defense — abort_unless($run->thread_id === $thread->id, 404) so a valid token for thread A can't reach a run in private thread B; (c) 422 for non-terminal runs; (d) same prompt-redact policy. Reuses RunReplayEventSynthesizer. Pages/Share/Replay.tsx mirrors Pages/Runs/Replay.tsx but with SharedLayout and no download menu (M10 export endpoints are owner-only).
  • Copy-to-clipboard button for the share URL (chunk 3). Lives inside the ShareMenu popover's ON state alongside the URL input. Uses navigator.clipboard.writeText — modern path; the Check icon flashes for 2 seconds on success. Older browsers / non-HTTPS dev contexts fall back to window.prompt so the user can manually copy. Vitest covers both paths.
  • Documentation link on the share page: "What is this?" → about page (chunk 4). New public Route::get('about')Inertia::render('About'), no auth, no rate limit. Pages/About.tsx is a standalone page (no AppLayout — anonymous viewers landing from a share link shouldn't see authenticated chrome) with four sections: (1) what LLM-Viz is, (2) what a /share/{token} link is (read-only + rate-limited + revocable), (3) privacy posture (BYOK, prompt-storage opt-out, encrypted keys, [prompt redacted by author] substitution), (4) AGPL §13 source link to https://github.com/CyberSecDef/llm.trackr.live. Page has a sign-in CTA + a back-to-home link. SharedLayout footers (in Pages/Share/Show.tsx and Pages/Share/Replay.tsx) updated: "What is this?" now points at /about instead of /, and a new "Source" link sits next to it satisfying AGPL §13 from every public share page. Welcome page also gets a "What is this?" link next to the sign-in CTA for discoverability from the home route. 3 Pest tests for the route (anonymous 200, authed 200, route-name resolves). 8 Vitest tests for Pages/About.tsx (headline + four section testids + body source link + footer source link + sign-in CTA + back-link). Updated existing Share/Show, Share/Replay, and Welcome Vitest tests for the new footer + nav targets.

Decisions (chunks 1–2):

  • Always regenerate token on enable. (chunk 1) User choice. POST is intentionally non-idempotent: hitting it twice produces two different tokens; the first URL becomes dead the moment the second is generated. Trade-off: a user who saved a URL, toggled sharing off + back on, and revisits the saved URL gets a 404 — the right outcome for a security-sensitive operation.
  • Inertia redirect over JSON response. (chunk 1) Matches the existing ThreadController patterns (title edit, archive, delete). The chunk-3 share-toggle UI will use useForm.post / router.delete and let Inertia re-render the thread page with updated props.
  • ShareTokenGenerator as an injectable service. (chunk 1) Tests bind a FixedTokenGenerator subclass via $this->app->instance(ShareTokenGenerator::class, $fake) for deterministic regen assertions. Matches the M10 chunk-2 FrameRenderer-fake pattern.
  • 128-bit entropy (16 bytes / 32 hex chars). (chunk 1) Collision probability across the lifetime of the app is essentially zero. The UNIQUE constraint on threads.share_token (from the M5 migration) is the belt-and-braces guard.
  • Idempotent DELETE. (chunk 1) Calling DELETE on a thread that's already unshared returns the same redirect, not a 404 — a stale tab that re-issues the request after the user already toggled off shouldn't error.
  • New Pages/Share/Show.tsx page over a readOnly flag on Threads/Show.tsx. (chunk 2) User choice. Standalone page = no conditional logic on the live page = no risk of leaking a feature (prompt input, share toggle, title editor) into the public view by accident. Cost is a small amount of duplication for the run-row layout; the M8/M9 viz components are reused verbatim.
  • Separate /share/{token}/runs/{run}/replay route over inline modal. (chunk 2) User choice. Mirrors the M9 owner-replay shape. Reuses every viz component; only the controller + layout differ.
  • Single share rate limiter for both routes, not per-route. (chunk 2) User choice. 60 req/min per IP across the whole /share/* namespace. SPEC's literal "IP rate-limited to 60/min" reads as a budget, not a per-route quota.
  • Cross-thread defense in SharedReplayController. (chunk 2) abort_unless($run->thread_id === $thread->id, 404) guards against an attacker with a valid token for thread A trying to navigate to a run in private thread B. 404 (not 403) avoids acknowledging the existence of the private run.
  • Backend does the prompt redaction, not the frontend. (chunk 2) The controller substitutes [prompt redacted by author] for null prompts when user.store_prompts=false before the payload leaves the server. Frontend just renders whatever's in the field; also includes prompts_redacted: bool so the page can show a top-of-thread notice.
  • whereNotNull('share_token') in the lookup. (chunk 2) Defense-in-depth: even if some database state has share_token = null rows that match a falsy URL, the WHERE clause filters them out. The unique constraint on share_token is the primary guard; this is the runtime fallback.
  • No user_id / model_id / api_key_id in the public response. (chunk 2) Matches M9's JSON-export design — the public share payload is identity-free. The model_snapshot inside parameters is what the page uses for total_layers + architecture_type; the FK doesn't leak.
  • Replay link only on terminal runs. (chunk 2) Same gate the M9 replay button uses on the owner thread page. Pending/streaming runs don't have a stable token_log; absence is the signal.
  • Popover for the share toggle, not a modal or inline panel. (chunk 3) User choice. Matches the chunk-5 M10 ExportDownloadMenu pattern — compact trigger button in the header, click-to-open content panel. Inline would have taken vertical space even when sharing is off; a modal would have been heavier for a frequent toggle.
  • Single-click disable, no confirmation dialog. (chunk 3) User choice. Re-enabling produces a fresh token per the chunk-1 always-regenerate policy, so an accidental disable is recoverable with one click — the URL just changes. A confirmation dialog would add friction for the common case (intentional disable) without protecting against the rare case (accidental click) any better than the regen-on-re-enable invariant already does.
  • navigator.clipboard.writeText with a window.prompt fallback. (chunk 3) The modern Clipboard API is available in every HTTPS context — covers production. Dev on http://localhost works because Chromium treats localhost as a secure origin. The fallback covers the edge case where clipboard is blocked (e.g., non-HTTPS LAN IP testing) — the user still gets the URL in a prompt they can manually copy from.
  • Green ON-indicator dot on the trigger, no inline status text. (chunk 3) The button stays compact at "Share" with a tiny emerald pulse when sharing is on. The full state (URL, enable/disable buttons, explanatory text) lives inside the popover. Trades discoverability of the current URL (one click required) for header density (no expanded text taking horizontal space).
  • New /about Inertia page over a #what-is-this anchor on Welcome. (chunk 4) User choice. A dedicated route is a real destination — bookmarkable, shareable, has its own <title>, doesn't couple About content with sign-in content. Welcome.tsx is the "sign in or go away" page; About.tsx is the "here's what this is and how privacy works" page. Different audiences, different pages.
  • AGPL §13 source link landed now on About + SharedLayout, not deferred to M15. (chunk 4) User choice. AGPL §13 requires the source to be available from every page the app serves to a remote user. The SharedLayout and About are exactly the pages a remote (un-authed) user touches first, so landing the link with the chunk that builds those pages keeps the compliance story coherent. M15 still owns the cross-app audit (authed pages, login, error pages already had a Source link); this chunk just ensures the public-facing surface is covered up-front.
  • Welcome page also links to /about. (chunk 4) Discoverability — a signed-out user landing on / from the marketing URL needs a way to read about the project without first hitting a share link. Single underline link next to the sign-in CTA, not a full nav.
  • No throttle on /about. (chunk 4) Cheap static render, no DB queries, no rate-sensitive resources. The share throttle exists to protect the share-thread payload (which does DB work + prompt-redaction logic); the about page has neither.

Share manual recipe (chunk 5)

Prerequisites: a thread with at least one terminal (complete or error) run — the public Replay pill only shows on terminal runs (same gate as the M9 owner replay).

Enable + visit + replay (golden path)

  1. As the owner, open the thread detail page. Click Share in the header action row. The popover opens with "Sharing is OFF" + a single Enable sharing button.
  2. Click Enable sharing. The popover content flips to "Sharing is ON" + the share URL in a readonly input + a Copy icon button + a Disable sharing button. The trigger pill gets a small green pulse-dot indicator.
  3. Click the Copy icon. The icon flashes a green check for ~2 seconds. Paste somewhere visible to confirm — it should look like https://your.host/share/{32-hex-token}.
  4. Open a private/incognito window. Paste the URL. The shared thread page renders without any login prompt — title + tags + run rows visible, no authenticated sidebar nav. The top of the page shows the "Anonymous view · shared by the author" indicator.
  5. On any terminal run, click the Replay pill. The /share/{token}/runs/{id}/replay page loads with the same M8 viz stack the owner replay uses (PlaybackControls + Viz / Embeddings / Debug tabs). Press play; the run replays frame-identical to the original.
  6. Click Back to shared thread in the replay header — returns to /share/{token}.

Verify the "What is this?" link + AGPL §13 source link

  1. From the shared thread page (or shared replay page), scroll to the footer. Two links should be visible: What is this? and Source.
  2. Click What is this? — lands on /about (200, no auth required). All four sections render: "What it is", "What this share link is", "Privacy posture", "Open source".
  3. Click the body source link OR the footer Source link → both open https://github.com/CyberSecDef/llm.trackr.live in a new tab.
  4. Click Sign in to make your own → lands on /login.
  5. Click ← Back to home → lands on / (signed-out viewers see Welcome; signed-in viewers redirect to Dashboard).

Verify disable + token regeneration (chunk-1 invariant)

  1. Back as the owner: open the thread. Click Share → popover shows "Sharing is ON" with the URL (let this be URL-A).
  2. Click Disable sharing. The popover flips to "Sharing is OFF" immediately; the green pulse-dot disappears.
  3. In the incognito tab, refresh URL-A → 404 (the share token is now NULL in threads).
  4. Owner tab: click Enable sharing again. The popover shows a new URL (URL-B). Confirm URL-A !== URL-B — per the chunk-1 always-regenerate decision, the old URL stays dead even if you re-enable.
  5. (Optional) Open URL-A in incognito → still 404. Open URL-B → renders the thread.

Verify rate limit (chunk-2 share throttle)

  1. With a valid share URL handy, run a quick burst from one IP:
    for i in $(seq 1 70); do
      curl -s -o /dev/null -w '%{http_code}\n' https://your.host/share/{token}
    done | sort | uniq -c
  2. Expected output: ~60 lines of 200, then 429s for the rest of the burst. The share RateLimiter caps the whole /share/* namespace at 60 req/min/IP — /share/{token}/runs/{run}/replay shares the same budget.

Verify prompt redaction (chunk-2 store_prompts=false path)

  1. Sign in as a user whose Settings → Store prompts is OFF. Make sure that user has at least one thread with a terminal run.
  2. Enable sharing on that thread. Open the URL in an incognito window.
  3. The thread page shows a top-of-thread amber banner: "The author has prompts disabled — prompt text is shown as [prompt redacted by author]." Each run row substitutes [prompt redacted by author] for the prompt; assistant outputs are preserved verbatim. Replay also redacts the prompt the same way.

Verify cross-thread defense (chunk-2 invariant)

  1. Owner: create two threads (T1 and T2), each with one terminal run. Enable sharing on T1 only.
  2. Note T1's share token + run ID. Note T2's run ID.
  3. Try: GET /share/{T1-token}/runs/{T2-run-id}/replay → 404. The controller's abort_unless($run->thread_id === $thread->id, 404) blocks the cross-thread navigation without acknowledging that the private run exists.

M11 retrospective

What worked

  • Always-regenerate token on enable simplified the entire UX. The chunk-1 decision meant the chunk-3 disable affordance didn't need a confirmation dialog: re-enable is one click and produces a fresh URL, so an "accidental" disable has zero data-loss surface. The same invariant also produces the cleanest possible "I leaked the URL, kill it now" recovery story — toggle off, toggle back on, the old URL is permanently dead. One decision; downstream effects on UX, security model, and operator playbook.
  • Standalone Pages/Share/Show.tsx over a readOnly flag on the live thread page. Chunk-2 decision that paid off across the whole milestone. Zero risk of leaking the prompt input, the share toggle, the title editor, or the export menu into the public view via a missed conditional. The M8/M9 viz components are reused verbatim (no if (!readOnly) smell inside them); only the controller + page wrapper differ. The duplication cost is one run-row template, which is fine.
  • Backend-side prompt redaction. Chunk-2 decision. The controller substitutes [prompt redacted by author] for null prompts at the data layer; the frontend just renders strings. Means there's no frontend-code path that could ever leak an un-redacted prompt — prompts_redacted: bool on the props is just for the banner, not the redaction itself. Defensive in depth.
  • Single share rate limiter for the whole /share/* namespace. Chunk-2 decision. One Limit::perMinute(60)->by($request->ip()) covers both the reader route and the replay route. SPEC said "IP rate-limited to 60/min"; reading that as a namespace budget (not per-route) keeps the operator surface to one tunable.
  • ShareMenu reuses the M10 ExportDownloadMenu Popover pattern. Chunk-3 decision. The Radix Popover affordance was already in the user's mental model from the export dropdown on the run rows; reusing it for share kept the header compact + cognitively familiar. The trigger button stays "Share" + a tiny indicator dot whether sharing is on or off — full state lives inside the popover, header stays dense.
  • New /about page landed the AGPL §13 source link on the public surface. Chunk-4 decision. The SharedLayout + Welcome both link to /about now, and the About page + SharedLayout both have the GitHub source link. AGPL §13 compliance for the unauthenticated surface is done; M15 still owns the cross-app audit for authed pages, but the externally-visible surface ships compliant.
  • Cross-thread defense via abort_unless($run->thread_id === $thread->id, 404). Chunk-2 invariant. Cheap to write, but it's the difference between "valid token for thread A lets you navigate to a run in private thread B" and "404." 404 (not 403) keeps the existence of the private run unacknowledged.

What we learned

  • Vite manifest stale on new Pages/*.tsx. Hit this twice — chunk 2 (Share/Show.tsx + Share/Replay.tsx) and chunk 4 (About.tsx). Pest's Inertia view-finder uses the production Vite manifest; new page files don't appear until npx vite build regenerates it. Canonical fix: rebuild between adding a page file and running Pest tests that render the page. Worth a quick check-and-rebuild pattern in any milestone that introduces new Inertia page components.
  • jsdom doesn't ship navigator.clipboard. Chunk-3 dev lesson. The standard mock pattern is Object.assign(navigator, { clipboard: { writeText: vi.fn().mockResolvedValue(undefined) } }) in beforeEach. Failing-clipboard tests need mockRejectedValue(new Error('blocked')) to exercise the window.prompt fallback path.
  • vi.hoisted Inertia router mocks need every method the component calls. Chunk-3 ShareMenu added a router.post call on a mock that previously only had get/patch/delete (the Threads/Show.test.tsx fixture). Symptom was "router.post is not a function" in the integration test. Fix: extend the hoist block + the spread mock + the mockReset() in afterEach. Worth treating as a checklist item any time a component starts using a new router method.
  • ESLint react/no-unescaped-entities catches typographic punctuation in JSX text. Chunk-4 — the pre-commit hook rejected "bring your own key" and can't in About.tsx text. Mechanical fix: &apos; / &ldquo; / &rdquo;. Worth keeping in mind whenever I write user-facing prose directly in JSX rather than in a t() call.
  • Pint reformat still breaks chained Edits. Same lesson the M10 retrospective documented. Hit again in chunk 3. The pattern is: when a commit hook reformats a file, in-flight Edit handles from before the reformat have stale old_string content. Re-Read after Pint runs, then re-Edit.

Parked / carry-forward to M15

  • Authed pages don't link to /about yet. Per the chunk-4 scope, only Pages/Welcome.tsx, Pages/Share/Show.tsx, and Pages/Share/Replay.tsx link to it. The authenticated app (Dashboard, Threads, Settings, Admin) has no "About" / "What is this?" affordance. M15's AGPL §13 compliance audit task is the right owner — add a footer link to AppLayout there, audit every served page for the source link, and decide whether to surface About from the user menu.
  • No browser-level E2E for the enable → incognito → disable → 404 flow. Same call M6, M8, M9, and M10 closeouts all made: Playwright/Cypress lands in M15 Launch Prep, not now. The Pest + Vitest coverage validates the data plane + the component behavior; the manual recipe above is the human verification path until E2E ships.
  • Shared replay doesn't subscribe to new runs. A share URL is a snapshot at view-time — new runs the owner submits show up on refresh, not via WebSocket. Out of scope for M11 by design (anonymous WebSocket subscriptions on private-runs.{id} would require a separate channels.php auth route that doesn't gate on user identity). If someone asks for "live shared threads," it's Phase 2 / Phase 3 territory.
  • No JSON-import endpoint for shared payloads. M9 chunk 3 parked the import side; M11 doesn't add it either. A future "paste a /share/{token} URL into your own dashboard to clone the thread" UX would need both the JSON-import endpoint AND a /share/{token}/export.json companion to the existing owner-only export. Phase 2 candidate.

Exit criteria

  • Toggling share on a thread produces a /share/{token} URL that works in an incognito window. Covered programmatically by ThreadShareControllerTest (POST sets share_token + share_enabled_at + redirects) + SharedThreadControllerTest (the public /share/{token} route returns 200 + Inertia Share/Show for the resolved token, with sanitized props). Manual recipe above §"Enable + visit + replay" is the human verification path.
  • Replay works on the shared view. Covered programmatically by SharedReplayControllerTest (200 + Inertia Share/Replay for valid token + run + terminal status; 422 for non-terminal; 404 for cross-thread; cross-thread defense) + the Vitest Pages/Share/Replay.test.tsx (renders PlaybackControls + viz pane + back-link). Manual recipe above step 5 (incognito play) is the human verification.
  • Disabling sharing returns 404 for the old URL. Covered programmatically by ThreadShareControllerTest (DELETE nulls both columns + idempotent for already-unshared threads) + SharedThreadControllerTest's 404-on-missing-token assertion. The chunk-1 always-regenerate decision strengthens this: even re-enabling produces a fresh token, so the old URL stays 404 forever. Manual recipe above §"Verify disable + token regeneration" is the human verification.

Bonus invariants from the implementation:

  • Cross-thread token navigation returns 404, not 403. Chunk-2 invariant. Pest SharedReplayControllerTest::cross-thread defense covers it.
  • Prompts are redacted for store_prompts=false owners. Chunk-2 invariant. Pest covers both the redaction substitution and the prompts_redacted: bool flag on the page props.
  • Rate limit applies across the whole /share/* namespace. Chunk-2 invariant. AppServiceProvider share RateLimiter is registered + applied in routes/web.php; the manual recipe above exercises it from the operator side.
  • /about is reachable for both anonymous and authed users. Chunk-4. Pest AboutPageTest covers both auth states.

M12 — Accessibility + Polish

Purpose: WCAG 2.1 AA compliance + general UX polish.

Tasks

  • [~] Full WCAG 2.1 AA audit using axe-core in CI + manual screen-reader pass (NVDA + VoiceOver). Chunk 1 lands the CI half: jest-axe wired into Vitest via expect.extend(toHaveNoViolations) in resources/js/test/setup.ts; a shared runner at resources/js/test/axe.ts configures axe-core to run the full wcag2a / wcag2aa / wcag21a / wcag21aa rule set with two jsdom-incompatible rules disabled (color-contrast — jsdom returns rgba(0,0,0,0) for most computed colors, would false-positive everywhere; scrollable-region-focusable — same root cause, depends on computed overflow). A new resources/js/__tests__/a11y/ directory holds the page-level audits; chunk 1 ships audits for the four pages named in the SPEC exit criteria (Dashboard, Threads/Show, Share/Show, Share/Replay), each across 2–3 meaningful states (empty / loaded / archived / redacted / errored). 11 new Vitest tests; zero structural violations on the first run across all four pages. Hard-fail CI gate: any future regression breaks the build. Manual screen-reader pass + color-contrast verification deferred to the M12 cross-browser smoke chunk (matches the chunk-1 visual-a11y decision below).
  • Keyboard navigation: all viz controls operable without mouse; visible focus rings (chunk 2). Per chunk-2 audit, the codebase's reliance on shadcn/Radix primitives (Button, Popover, Dialog, Sheet, Command, Slider, Textarea, AlertDialog) already covered the keyboard-reachability + focus-trap + Escape-to-close story for ~95% of interactive elements — the M7 frontload-shadcn decision paid off again. Found 5 hand-rolled hot spots without focus-visible: rings: the right-pane tab toggles in Pages/Threads/Show.tsx, Pages/Runs/Replay.tsx, and Pages/Share/Replay.tsx (9 buttons total — Visualization / Embeddings / Debug); the PlaybackControls toolbar in Components/PlaybackControls.tsx (play/pause + step + cursor-jump-to-live + 4 speed buttons = 7 buttons); and the archive filter tablist in Pages/Threads/Index.tsx (3 buttons + missing type="button" + missing tablist aria-label). Applied the standard focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 focus-visible:ring-offset-background pattern to all 19 buttons. Added aria-label="Right pane view" to the Share/Replay tablist (the other two already had it) and aria-label="Archive filter" + type="button" to the Threads/Index tablist. 7 new Vitest keyboard tests in resources/js/__tests__/a11y/keyboard.test.tsx: PlaybackControls Tab order, Space toggles play, Enter steps, speed-button focus-visible coverage, ShareMenu Enter-opens-popover, ShareMenu Escape-closes-and-returns-focus (Radix-provided but verified to survive jsdom), focus-visible class assertions on the playback buttons. Manual keyboard-only vertical-slice recipe documented below — closes the second SPEC exit criterion.
  • ARIA labels on canvases (Three.js scene gets a textual description that updates on layer-advance) (chunk 3). Audit found: the SVG charts (AttentionHeatmap, MoERouting) already had role="img" + descriptive aria-labels from M8 chunks 5b/6; the Three.js canvases (VizPane, EmbeddingScene) had static aria-labels but no live-updating announcer. Chunk 3 added a hidden <span role="status" aria-live="polite" className="sr-only"> adjacent to each Three.js <canvas> whose text is derived by the new pure helper resources/js/lib/vizAnnouncement.ts. Per chunk-3 decision the announcer fires per-decile token milestones (10/20/30 tokens generated…), not per token/layer/event — aria-live="polite" queues but does not drop, so a 100-token stream would otherwise announce 100 lines back-to-back. State transitions covered: empty → "Run started." (Viz) / "Embedding scene loaded." (Embedding), every 10 tokens → "N tokens generated.", completion → "Run complete. N tokens generated.", error → "Run errored: {message}." Errors take precedence over completion + milestones. LogitsDistribution was inspected too — it's a <ul> with readable percentage text + already-accessible list semantics; no aria gap. 8 new Vitest tests in resources/js/__tests__/lib/vizAnnouncement.test.ts cover every transition + pluralization + error-precedence invariant.
  • prefers-reduced-motion honored (stepped static frames instead of continuous animation) (chunk 3). M8 chunk 9 already shipped the headline gate (right-pane Viz tab is disabled when useReducedMotion() returns true, so the Three.js scene never mounts; users land on the Debug tab fallback). Chunk 3 closed the remaining gaps for animation paths that didn't go through that gate: 6 animate-pulse / animate-spin Tailwind classes (PlaybackControls LIVE-pill Radio icon, ExportDownloadMenu Loader2 rendering spinner, streaming-cursor on Threads/Show, Runs/Replay, Share/Replay, and the streaming-status indicator dot in the thread transcript) all got the motion-safe: Tailwind variant prefix. The prefix compiles into @media (prefers-reduced-motion: no-preference) — animations run when the user hasn't opted out and stop running when they have. 2 new Vitest assertions in resources/js/__tests__/a11y/motion.test.tsx lock the pattern: regression-safe against a future contributor pasting plain animate-pulse back in.
  • Color-blind palette check on heatmaps + MoE bars (viridis / cividis as defaults) (chunk 4). New resources/js/lib/palettes.ts exports two canonical CB-safe palettes: VIRIDIS_STOPS (5-stop perceptually-uniform sequential, used for AttentionHeatmap) + OKABE_ITO (7-chromatic + white-neutral categorical, used for EmbeddingScene cluster identities). AttentionHeatmap: dropped the M8 slate-950 → cyan-300 single-hue ramp, switched to a 5-stop multi-stop scaleLinear over viridis; aria-label updated to "Attention heatmap (illustrative, viridis palette)" so screen-reader users know the palette they're inspecting. embeddingClusters.ts: rewrote all 8 cluster colors to map 1:1 against Okabe-Ito (the 8th cluster — code-symbols — uses the swapped-in white neutral instead of the original Okabe-Ito #000000, since pure black would blend into the slate-950 scene background). Two near-duplicate cyan shades + two near-duplicate amber shades from the old palette are gone — every cluster is now perceptually distinct under deuteranopia, protanopia, and tritanopia. MoERouting + LogitsDistribution audited and left as-is: their bars use single-hue magnitude encoding (length → value, color is decorative); WCAG 2.1 SC 1.4.1 "Use of Color" is satisfied because the magnitude is in the length, not the hue. Status pills (complete/streaming/error) similarly disambiguate via text labels next to the color. 8 new Vitest tests: palettes.test.ts × 5 (canonical stops, domain pairing, no duplicates, EmbeddingClusters all match Okabe-Ito, banned M8 hexes absent) + AttentionHeatmap.test.tsx × 1 new (palette + aria-label) + 2 existing tests updated for viridis low-stop assertion.
  • Empty states for: no threads, no API keys, no runs in a thread (chunk 5). Audit confirmed all three SPEC-named states already existed from M7 (the chunk-1 frontload-shadcn decision paid off again) — but two were noticeably terser than the rest: Threads/Show.empty-transcript (single line of muted text, no icon) and ApiKeys/Index.empty-keys (single line in a table cell, no icon). Both were polished to match the icon + title + body pattern the Dashboard already used: empty-transcript got a ChevronDown icon directing the eye to the prompt input below + a bolder "No prompts yet" title; empty-keys got a KeyRound icon + a "No API keys yet" title + a "Bring-your-own-key — we charge nothing" tagline. The full inventory of locked-in empty states is now: no API keysDashboard.no-api-key-callout (full-page amber callout + "Add a key" CTA), Threads/Show.no-api-key-footer (prompt-footer amber strip + "API Keys" CTA), ApiKeys/Index.empty-keys (table empty row + icon + body); no threadsDashboard.empty-threads (icon + title + dynamic CTA per hasApiKeys), Threads/Index.empty-all (icon + title + body, page header has the "New thread" CTA), Threads/Index.empty-filtered (icon + title + "Clear filters" button); no runs in a threadThreads/Show.empty-transcript (icon + title + "type your first one below" guidance), Share/Show.shared-empty (acknowledgment only — read-only viewer can't act). 7 new Vitest tests in resources/js/__tests__/a11y/empty-states.test.tsx lock the icon-present + title-text + CTA-href invariants across all of them; regression-safe against a future contributor downgrading any state back to bare text.
  • Loading states + skeletons (chunk 6). New Components/ui/skeleton.tsx (shadcn standard one-liner — motion-safe:animate-pulse rounded-md bg-muted, gated under prefers-reduced-motion per chunk-3) + new Components/VizSkeleton.tsx shared wrapper used by all 6 Three.js Suspense fallbacks (Pages/Threads/Show.tsx × 2, Pages/Runs/Replay.tsx × 2, Pages/Share/Replay.tsx × 2). The wrapper reserves an aspect-square slot matching the rendered canvas footprint — zero layout shift when VizPane/EmbeddingScene mounts — and carries role="status" + aria-label so screen-reader users hear "Loading visualization" or "Loading embedding scatter" without waiting for the WebGL canvas to actually render. 4 new Vitest tests on VizSkeleton (container + skeleton-child + caption text + motion-safe gating).
  • Toast notifications for errors and success (chunk 6). Installed sonner (the canonical toast lib shadcn ships); new Components/ui/sonner.tsx wraps it with our dark-slate theme (top-right position, richColors, closeButton, theme tokens). <Toaster /> mounts once at the root of Layouts/AppLayout.tsx. New hooks/useFlashToast.ts watches usePage().props.flash?.status and translates known Laravel-session statuses into the right toast variant (settings-saved → success, api-key-added → success, api-key-deleted:{vendor} → info, refresh-complete:{msg} → success, model-updated:{name} → success, model-deleted:{name} → info, rate-limit-updated:{userId} → success). AppLayout invokes the hook so every authed page gets toasts for free. Removed the inline emerald/amber flash blocks from Pages/ApiKeys/Index.tsx (api-key-added / api-key-deleted) — they would have double-announced alongside the toasts. Also added toast.success() / toast.info() calls inside ShareMenu for enable / disable / copy actions (previously silent — the popover state was the only feedback). 6 new Vitest tests on useFlashToast cover the 4 known status patterns + the no-flash + unknown-status graceful degradation paths.
  • 404 / 500 pages styled (chunk 7). Audit confirmed all 4 error pages existed from M7 (NotFound, ServerError, Expired, Forbidden) sharing an ErrorShell component. Chunk 7 polished them: (a) split 503 out of ServerError into a new dedicated Maintenance.tsx ("We'll be right back" — 503 has different semantics from a logged exception; users can't retry sooner, the situation clears itself), with bootstrap/app.php routing 503 → Errors/Maintenance separately; (b) added a lucide-react icon to each page (Compass / ShieldAlert / Clock / AlertTriangle / Wrench) via a new optional icon prop on ErrorShell; (c) added a "Go back" secondary CTA that calls window.history.back() (falls through to the primary CTA destination when history is empty — direct deep-link hits). Primary CTA still flips between "Sign in" / "Back to dashboard" depending on auth; AGPL §13 Source link remains. 2 new Pest tests (Errors/Maintenance for 503, Errors/Expired for 419 — previously only 404 / 403 / 500 had Pest coverage); 26 new Vitest tests in resources/js/__tests__/Pages/Errors/ErrorPages.test.tsx using describe.each over all 5 error pages (status line + headline + icon + Go-back CTA + Source link + Sign-in CTA × 5 pages = 25, plus 1 Go-back behavior test).
  • Cross-browser testing: Chrome, Firefox, Edge, Safari (latest two) (chunk 9). Documented as a manual recipe (below) — the four target browser families × latest two versions each = 8 build targets. Per the existing M6/M8/M9/M10/M11 closeout pattern, automated Playwright/Cypress E2E is parked for M15 launch prep; chunks 1–8 cover the structural a11y story via jsdom + axe + jest-axe, and this recipe gives the human verification path for the visual / runtime concerns that jsdom can't model (real focus rings, prefers-reduced-motion behavior, color-contrast ratios, actual WebGL 2 capability, fingerprinted clipboard/IntersectionObserver/etc.).
  • WebGL 2.0 detection + clear "unsupported browser" message with fallback to 2D-only viz (chunk 8). New resources/js/lib/webgl.ts is the canonical capability probe: creates a throwaway canvas, calls getContext('webgl2'), returns the boolean. Cached per-process — GPU capability doesn't change during a page lifetime. SSR-safe (returns false when document is undefined so the server-rendered HTML is the safe fallback; the client hook re-probes on mount and corrects upward). New resources/js/hooks/useWebGL2Support.ts wraps the probe in a stateful hook with an optimistic initial value of true (matches the supported-runtime majority — initial false would mount the fallback first and then yank the user back to the Three.js tab after hydration). The 3 viz pages (Pages/Threads/Show.tsx, Pages/Runs/Replay.tsx, Pages/Share/Replay.tsx) now compose the WebGL gate with the existing reduced-motion gate: Viz tab disabled when reducedMotion || !webgl2Supported; Embeddings tab disabled when !webgl2Supported (reduced motion alone keeps the embedding cloud usable because its auto-orbit pauses on spotlight per M8 chunk 7); default mode flips to Debug (the existing 2D text fallback from M6) when either is the case; an amber webgl-unsupported-notice banner explains the situation below the tablist when WebGL 2 is missing. 10 new Vitest tests (4 helper + 2 hook + 4 fallback integration on Threads/Show). One global test-suite shim: resources/js/test/setup.ts mocks useWebGL2Support to return true by default so existing page tests (which assume the Three.js stub mounts) keep working; the new webgl-fallback.test.tsx overrides the mock locally to exercise the unsupported path.

Decisions (chunk 1):

  • jest-axe over vitest-axe or a custom matcher. User choice. jest-axe is the mature path — large user base, axe-core 4.x under the hood, the toHaveNoViolations matcher is a plain {pass, message} shape that consumes via Vitest's expect.extend(...) API. The vitest-axe package is a smaller community wrapper around the same axe-core; rolling our own matcher would have been ~10 lines but added a dependency we'd own without payoff. The TypeScript surface comes from @types/jest-axe + import 'jest-axe/extend-expect' in resources/js/types/global.d.ts.
  • Hard-fail CI gate from day one, not report-only. User choice. Any axe violation in any a11y test fails Vitest, which fails CI. Forces fixes in-chunk rather than letting a backlog accumulate. The trade-off — false-positives could block a PR — is acceptable because the jsdom-safe rule config (color-contrast + scrollable-region-focusable disabled) eliminates the two rules that fire spuriously under jsdom; everything else axe checks (labels, ARIA roles, headings, landmarks, name/role/value, alt text, button-has-name, document-title) is deterministic and meaningful.
  • Dedicated __tests__/a11y/ directory + per-page test files, not assertions tacked onto existing tests. Existing page tests already carry complex fixture state (Inertia hoist mocks, fetch stubs, stream-state mocks). Co-locating axe assertions would tangle the a11y signal with the unit-test signal. Per-page files mean each a11y file owns the minimal fixtures needed to render the page in its meaningful states; the a11y file becomes a clean "is this page accessible?" signal independent of the unit-test surface.
  • Test multiple page states, not just the default render. Each a11y file covers 2–3 states: empty (no data) / loaded (realistic data) / variant (archived / errored / redacted). Empty states have their own a11y surface (different headings, different landmarks, empty-state CTAs); variants like "archived" introduce status badges that need accessible names. Testing only the loaded state would miss those code paths.
  • color-contrast + scrollable-region-focusable disabled under jsdom; visual a11y deferred. jsdom doesn't compute CSS; getComputedStyle returns rgba(0,0,0,0) for most elements, so contrast checks would false-positive every element with Tailwind classes. The visual a11y story (real contrast ratios, focus-ring visibility, prefers-reduced-motion behavior in real browsers) needs a Playwright + @axe-core/playwright pass or a manual cross-browser audit — both deferred to a later M12 chunk or M15. The structural a11y story (everything else axe-core checks) runs hot in jsdom and catches regressions just fine.
  • Run the full wcag2a / wcag2aa / wcag21a / wcag21aa tag set, not just wcag2aa. The SPEC says "WCAG 2.1 AA". Running A + AA at both 2.0 and 2.1 levels covers the full conformance target. Best-practice rules (best-practice tag) are intentionally NOT included — they catch UX-quality issues that aren't WCAG conformance failures, and we don't want a "page has heading levels skipping" best-practice rule (which axe flags) to gate CI when it isn't a WCAG violation.
  • Skeleton chunks: chunk 1 is axe + four pages only. SPEC's M12 task list has 11 items; bundling them all into one chunk would be too much surface area. Chunk 1 lands the harness + the SPEC exit-criteria pages. Subsequent chunks own the remaining tasks (keyboard nav + focus rings, ARIA on canvases, prefers-reduced-motion audit, color-blind palettes on heatmaps, empty states, loading/skeletons, toasts, error pages, cross-browser smoke, WebGL detection + 2D fallback).

Decisions (chunk 2):

  • Surgical focus-visible fixes, not a sweeping refactor. User choice. Audit found 5 hand-rolled hot spots (3 right-pane tablists + PlaybackControls toolbar + Threads/Index archive filter); fixed each in place by appending the standard focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 focus-visible:ring-offset-background class chain. Trade-off: a future contributor adding a new bare <button> outside the shadcn Button primitive could miss the convention. Mitigations: (a) jest-axe in CI catches the button-name / label failures; (b) the eslint-plugin-jsx-a11y rule set (already in devDependencies) catches a lot of this at lint time; (c) a follow-up M12-or-later chunk could define a .focus-ring utility class to make the pattern even shorter — out of scope for this surgical fix.
  • Kept the custom hand-rolled tablists; did NOT migrate to @radix-ui/react-tabs. The three right-pane tablists (Visualization / Embeddings / Debug) and the Threads/Index archive filter could be replaced with Radix Tabs to inherit arrow-key navigation between tabs (WAI-ARIA Authoring Practices guidance). WCAG 2.1 AA's "operable without mouse" is already satisfied because each tab is independently Tab-focusable + Enter-activatable; arrow-key navigation is best practice but not a conformance failure. Migrating would be ~150 lines of churn across 4 pages + new Vitest fixture updates — saved for a later polish chunk if anyone files it.
  • Vitest keyboard tests, not Playwright. Same call M6/M8/M9/M10/M11 closeouts all made: Playwright lands in M15, not now. @testing-library/user-event already in the dev deps + Vitest's jsdom environment is enough to verify Tab order + Space/Enter activation + Escape close + focus return. Visual focus-ring rendering is verified only via the class-name assertion (the manual recipe is the human verification path until real-browser E2E lands).
  • Object.defineProperty(navigator, 'clipboard', ...) over Object.assign(navigator, ...). Dev lesson: jsdom 29 protects navigator.clipboard with a getter when an earlier test file in the same run has already defined it. Object.assign then throws "Cannot set property clipboard of #<Navigator> which has only a getter." Object.defineProperty with configurable: true, writable: true bypasses the getter. The ShareMenu chunk-3 test got lucky because it ran first in its old test order; the chunk-2 keyboard test exposed the issue. Now documented so future a11y tests get the pattern right out of the gate.
  • Tablist aria-label audit landed alongside the focus-ring fix. Threads/Index's role="tablist" had no aria-label (axe doesn't flag this — the aria-allowed-attr rule only catches invalid attrs, not missing labels — but it's a real screen-reader gap). Added aria-label="Archive filter". Share/Replay's tablist had the same gap; added aria-label="Right pane view". Threads/Show + Runs/Replay already had it (M8 chunk 8). Now all four tablists are screen-reader-discoverable.

Decisions (chunk 3):

  • Per-decile milestone announcer, not per-event. User choice. aria-live="polite" queues but does NOT drop announcements — a 100-token stream would otherwise pile up 100 announcements in the AT queue and a screen reader would read them all in sequence. Per-decile milestones produce a useful narrative ("Run started. … 10 tokens generated. … 30 tokens generated. … Run complete.") without the flood. Trade-off: a 1–9-token run never moves off "Run started." until completion — acceptable, the completion announcement covers it.
  • Pure helper (deriveVizAnnouncement) over inline useMemo in each canvas component. The same announcement logic now serves VizPane AND EmbeddingScene (with a different startedLabel for each — "Run started." vs "Embedding scene loaded."). Extraction also makes the function testable without spinning up Three.js / WebGL stubs — 8 Vitest assertions hit the helper directly, covering every state transition + pluralization + error-precedence invariant.
  • Errors take precedence over completion or milestones. If an run.errored event is in the array, the announcement is the error message regardless of how many tokens streamed before the error or whether run.completed also appears (the latter shouldn't but defensive ordering matters). Screen reader users hear about the failure first; the token count is implicit in the prior milestones they already heard.
  • role="status" aria-live="polite" on a <span className="sr-only">, not <div>. Both are valid host elements for the role; <span> is inline-by-default and matches the surrounding canvas-wrapper's flow. The sr-only Tailwind utility positions it off-screen via position: absolute; width: 1px; height: 1px; clip: rect(0 0 0 0) — invisible to sighted users + readable to screen readers. role="status" is the implicit role for aria-live="polite" per WAI-ARIA; being explicit aids older AT implementations.
  • motion-safe: prefix, not useReducedMotion() runtime checks. User choice (implicit — surgical preference from chunk 2 carries forward). Tailwind's motion-safe: variant compiles to @media (prefers-reduced-motion: no-preference); the browser handles the media-query evaluation natively, no JS overhead, no re-render needed when the preference flips. The useReducedMotion hook is reserved for cases where a component must structurally change under reduced motion (M8's right-pane gate that swaps the Viz tab for Debug). Inline animations (pulse, spin) just shouldn't animate — CSS handles that cleanly.
  • LogitsDistribution left as-is — already accessible. It's a <ul> of percentage labels + token strings, not an <svg> or <canvas>. Screen readers read each list item + the percentage text natively. Adding role="img" would have reduced accessibility by treating the list as a single opaque image. Axe is happy with the current shape; M12 chunk 1 already verified this on Threads/Show.
  • Three.js <canvas> keeps its descriptive aria-label; we did NOT change it dynamically. Per the user's chunk-3 decision the live state goes in the adjacent aria-live region, not on the <canvas> itself. Rationale: some screen readers don't re-announce a changed aria-label; a dedicated role="status" region is the AT-stable pattern.

Decisions (chunk 4):

  • Viridis for sequential, Okabe-Ito for categorical. User choice (matches SPEC wording on the sequential side, picked separately for categorical). Viridis is the canonical perceptually-uniform sequential gradient (van der Walt + Smith, matplotlib 2.0); it's monotonic-in-luminance under sRGB AND under the three common color-vision deficiencies, so a darker cell is always "less" and a lighter cell is always "more" regardless of who's viewing. Cividis (the explicit CB-only variant) was considered; viridis won because it retains better contrast for non-CB users too. Okabe-Ito (2008) is the academic standard CB-safe categorical palette specifically designed so every pair of stops remains distinguishable under deuteranopia / protanopia / tritanopia.
  • Hardcoded 5-stop viridis array, no d3-scale-chromatic. Could have pulled in interpolateViridis from d3-scale-chromatic (a separate npm dep). Instead the 5 canonical stops are inlined as a readonly string[] constant; the existing d3-scale scaleLinear already handles multi-stop ranges. Trade-off: 5 stops instead of the full 256-color continuous viridis. Acceptable because the matrices we render are 32×32 max and 5 stops with d3's RGB interpolation between them is visually indistinguishable from the continuous version at that resolution.
  • 8th cluster slot uses white, not the Okabe-Ito #000000. The original Okabe-Ito set lists #000000 as the 8th color, but our EmbeddingScene background is slate-950 (#020617) — pure black would render invisible. White is on the lightness axis perpendicular to the 7 chromatic Okabe-Ito stops, so it remains distinguishable from all 7 under any CVD. Documented inline in palettes.ts so a future contributor reading the code understands why we diverged from the canonical Okabe-Ito 8th.
  • Did NOT change MoERouting / LogitsDistribution / status pills. These already satisfy WCAG 2.1 SC 1.4.1 ("Use of Color"): MoE + Logits bars encode magnitude in length, not hue (color is decorative); status pills pair their color with a text label. CB-safe by construction, no migration needed. Explicit decision recorded so a future audit doesn't sweep them up.
  • EmbeddingScene's EMBEDDING_CLUSTERS array order kept stable. Cluster names (whitespace, punctuation, pronouns, common-english, long-words, numbers, code-keywords, code-symbols) keep their positions; only the color: field changed. Order-stable migration so existing fixtures + integration tests that index into the array still resolve correctly (none do directly, but defensive choice).
  • Old hex test in AttentionHeatmap.test.tsx updated, not deleted. The M8 test that asserted rgb(2, 6, 23) for zero-weight cells now asserts rgb(68, 1, 84) (viridis low stop). Updating rather than deleting keeps the test's intent (causal-masked cells use the low stop) and only updates the constant. Plus an explicit "banned old hexes" assertion locks the migration: a contributor pasting the old cyan-300 back in would fail the test.
  • aria-label calls out the palette name. Updated to "Attention heatmap (illustrative, viridis palette)". Screen-reader users who care about palette choice (data-viz academics, color-blind users explicitly verifying the safe choice) hear it; everyone else gets a slightly longer but still-readable label. Trade-off: cluttering one extra word into the label is worth the explicit a11y signal.

Decisions (chunk 5):

  • Audit + polish only the gaps, not a sweeping redesign. Carries forward chunk-2's surgical-fix preference. All three SPEC-named states already existed; chunk 5 just upgraded the two terser ones (Threads/Show.empty-transcript, ApiKeys/Index.empty-keys) to the icon + title + body pattern the Dashboard versions already used. The other six existing states (no-api-key-callout, no-api-key-footer, empty-threads, empty-all, empty-filtered, shared-empty) were already on-pattern — no churn.
  • No CTA on Threads/Index.empty-all (the page-level header has one). Threads/Index already shows a "New thread" submit button in the page header. Adding a duplicate button to the empty-state card would feel redundant + give the user two near-identical actions to pick between. Dashboard's empty-threads does have its own button because Dashboard has no page-level CTA. Asymmetry by design; documented so a future reviewer doesn't try to "fix" it.
  • No CTA on Share/Show.shared-empty. Read-only public viewer has no action to take when a shared thread has no runs yet — the thread's owner is the only person who can change the state. The empty state just acknowledges the situation; the SharedLayout footer already provides the "What is this?" link.
  • Did NOT upgrade no-usable-models (the bonus state). Out of SPEC scope. The state fires when a user has an API key but the registry doesn't carry any model for their vendor — a registry-staleness edge case. The existing text mentions "an admin can refresh the registry from the Models page" but doesn't link there because non-admin users can't access that route anyway. Parking as a M15 launch-prep follow-up: surface a "ping an admin" affordance or auto-trigger the registry-refresh fetch when an authed user lands on this state.
  • ChevronDown icon on empty-transcript, not the more generic MessageSquare. The chevron is a directional affordance — it points downward toward the prompt input that's literally rendered immediately below the empty state card. Subtle but useful: a screen-reader user hears "type your first one in the input below" while a sighted user follows the icon's arrow to the input. Function over decoration.
  • Lock empty-state structure in tests, not just getByTestId presence. The existing M7 tests on the empty-state testIds only verified the element rendered. Chunk 5 adds assertions that there's an <svg> icon child AND a title text node AND (where applicable) a CTA <a href> with the expected target. Future regressions that delete the icon or downgrade the title pass the old tests but fail the new ones — strictly more coverage at marginal additional cost.

Decisions (chunk 6):

  • sonner over a hand-rolled toast. It's the toast library shadcn endorses, has Radix-style a11y defaults (role="status" + aria-live="polite"), supports richColors / closeButton / dark theme out of the box. Building our own would have been ~80 lines of context-aware portal management + an exit transition + accessibility wiring — buying that for one dep is the right trade. Same call M10's chunk-2 made when it picked imagemagick convert over hand-rolled SVG rasterization.
  • Single Toaster mounted in AppLayout, not per-page. The Toaster is a portal; multiple instances would duplicate every notification. AppLayout already wraps every authed page so it's the natural root. The Welcome / Login / Errors / Share pages don't use AppLayout — they don't currently have toast-worthy state changes either, so no gap.
  • useFlashToast hook over toast() calls in each controller's redirect handler. Flash data lands as a prop on the next Inertia visit; pages would otherwise have to wire their own useEffect to react. Centralizing the prop → toast mapping in one hook makes the vocabulary discoverable AND keeps each page's render pure (no side effects scattered across components). The hook uses a useRef to track the last-fired status so re-renders with unchanged flash don't fire duplicate toasts.
  • Removed inline flash blocks instead of keeping both. The chunk-6 audit found the inline emerald/amber blocks in ApiKeys/Index (and elsewhere) would double-announce with the new toasts. Kept the toasts; deleted the inline blocks. Cleaner UX (one source of feedback, not two competing) + smaller pages. Pages that I haven't touched in this chunk (Admin/Models/Index, Admin/Users) still have their inline blocks; they're parked for chunk-7 polish or a separate sweep — the useFlashToast hook is opt-in by virtue of AppLayout coverage, so toasts already fire there even without removing the blocks (so admins see both until we get to those pages).
  • VizSkeleton is its own component, not inline JSX in each Suspense fallback. Three pages × two fallbacks each = 6 spots. Six copies of the same inline JSX would have drifted out of sync. The single component lets the M8 viz pages share a unified loading shape — same aspect-square footprint, same caption strip, same a11y wiring.
  • role="status" aria-label on the loading container, not on the Skeleton element itself. The Skeleton primitive is decoration; the container is the semantic loading region. Putting the role on the container also means screen readers don't announce the pulsing skeleton's visual changes (which would happen if role="status" were on the Skeleton, since aria-live regions announce DOM mutations).
  • Skeleton inherits motion-safe:animate-pulse from chunk 3's pattern. Single source of truth: the chunk-3 reduced-motion decision said "all animate-* classes get motion-safe: prefixes." The Skeleton primitive that ships in chunk 6 already follows this. Future contributors who add a new <Skeleton> get the prefers-reduced-motion behavior for free.
  • Toasts for share enable / disable / copy. Previously these actions were silent — the popover content state was the only feedback, which sighted users could see but screen-reader users couldn't unambiguously confirm. Toasts give every user a definitive "the action succeeded" signal that ALSO doubles as a "you can close the popover now" affordance.
  • Did NOT add toasts on every form-submit success. Out of scope. The useFlashToast hook covers anything the backend marks with a known status; bolting toast() calls onto every router.post call would have duplicated the flash mechanism. Stick to one delivery path.

Decisions (chunk 7):

  • Split 503 into a dedicated Maintenance.tsx. Previously bootstrap/app.php routed both 500 and 503 to Errors/ServerError. 500 means "we logged an unexpected exception" — the user message is "try again in a moment" because retrying might succeed. 503 means "service intentionally unavailable" — deploy in progress, scheduled maintenance, or overload-shedding — and retrying immediately won't help. Different copy ("We'll be right back" vs "Something went wrong"), different icon (Wrench vs AlertTriangle). The bootstrap match arm now lists them separately so the messaging matches the actual semantics.
  • Optional icon prop on ErrorShell, default null. Backwards compatible — any caller that doesn't pass icon gets the existing M7 layout. Each chunk-7 page passes a lucide-react icon sized at h-12 w-12 for visual consistency. Picks: Compass (404 — you've wandered), ShieldAlert (403 — gated by auth), Clock (419 — session timed out), AlertTriangle (500 — unexpected error), Wrench (503 — we're working on it). Each is aria-hidden="true" because the headline + status line already announce the error verbally — the icon is decorative.
  • window.history.back() for "Go back", not Inertia's router.visit('-1'). Inertia doesn't have a back-navigation primitive; the browser's history API does. For a direct deep-link hit (history.length === 1) the back call would be a no-op, so we fall through to window.location.assign(goHomeHref) — same destination as the primary CTA. Trade-off: this leaves SPA-style soft navigation for the in-app case but uses a hard navigation for the deep-link fallback. Acceptable because the deep-link case is rare (a user lands on an error page from outside the app).
  • Three CTAs in a row: Sign in/Dashboard + Go back + Source. Visual order matches priority: primary action (escape to a useful page), secondary action (return to where they came from), tertiary (AGPL §13 compliance). Button variants default / outline / ghost carry the hierarchy. Source button moved from outlineghost so the new Go-back button takes the "secondary" visual slot.
  • describe.each over the 5 pages, not 5 separate test files. Single test file with a typed cases array; each per-page assertion runs against every component. ~30 lines of test code covers 26 assertions. Adding a future error page would just append to the array — no boilerplate.
  • Kept ServerError's text "An unexpected error occurred on our end. The error has been logged". Mentions logging without leaking a stack trace or event ID. A future M15 chunk can surface the Sentry event ID for support requests — parked for now because it needs a Sentry SDK call from the server-side render path.

Decisions (chunk 8):

  • Runtime probe over user-agent sniffing. User-agent strings lie (Chromium browsers spoof Safari; embedded WebViews report Chrome; Electron apps look like any UA they want). And even a browser that can support WebGL 2 can fail at runtime if the GPU driver is blocklisted (Linux/Mesa edge cases) or if the user disabled hardware acceleration. The probe answers the only question we care about: "can this user mount a WebGL 2 context right now?"
  • Optimistic initial state (true), pessimistic mount correction. useWebGL2Support returns true on first render. The mount useEffect re-probes and corrects downward to false if the runtime can't actually back the context. Trade-off: a user on an unsupported runtime sees the Three.js tab briefly during hydration before it flips to Debug. The opposite default (initial false) would flicker every supported user through the fallback first, which is a much worse experience because supported is the common case. The useReducedMotion hook makes the opposite call because reduced-motion users are a minority preference; here we're optimizing for the supported-runtime majority.
  • Composed gate over a single boolean. vizDisabled = reducedMotion || !webgl2Supported and embeddingsDisabled = !webgl2Supported. Two reasons to disable the Viz tab; only one reason to disable Embeddings. Reduced motion alone doesn't disable Embeddings because the cloud's auto-orbit pauses once a spotlight is active (M8 chunk 7 decision) — the page is still consumable. WebGL unavailability disables both because both rely on Three.js + a WebGL2 context.
  • Amber role="note" banner under the tablist, not a full-page block. The user can still consume the Debug tab + the run transcript on the left — we're not blocking the whole page, just two of three tabs. A small notice in the affected region is the right scope. Amber matches the existing pattern (no-api-key banner, registry-staleness banner, no-usable-models banner). role="note" is the semantic match for "additional context that doesn't carry the page's main message."
  • title attribute on the disabled tabs that names the cause. Sighted users hovering the Viz tab see "Visualization disabled because this browser does not support WebGL 2.0." Different from "...prefers-reduced-motion is set." Carries forward the M8 chunk 9 pattern but extends it for the new gate. Title precedence: WebGL message first (more fundamental), reduced-motion second.
  • No URL state, no preferences-persistence. The user can't override the gate ("yes I really want to try Viz anyway") — a runtime that can't mount WebGL 2 will throw inside Three.js's renderer constructor; honoring an override would just produce a crash. Same call M8 chunk 9 made for reduced motion.
  • Global test mock + opt-in local override. setup.ts mocks useWebGL2Support to return true by default so every existing page test that assumes the Three.js stub mounts keeps working. The new webgl-fallback.test.tsx vi.mock(...)s it back to false to exercise the unsupported branch. The alternative (per-test mocks scattered across 60+ test files) would have been a much wider blast radius for the same coverage.
  • HTMLCanvasElement.prototype.getContext mock pattern works in jsdom. Dev lesson: jsdom-29 logs "Not implemented: HTMLCanvasElement's getContext() method: without installing the canvas npm package" to stderr but the probe's try { ... } catch swallows the throw and returns false. Test suite green; the warning is jsdom's stderr noise, not a test failure. Documented now so a future contributor doesn't go install the canvas npm package thinking the warning needs fixing.

Cross-browser smoke recipe (chunk 9)

Use this to verify the SPEC's M12 cross-browser-testing line. Run against the latest two stable versions of each browser family — Chrome, Firefox, Microsoft Edge, Safari. macOS, Linux, and Windows have slightly different OS-level a11y behaviors (notably prefers-reduced-motion toggle locations + how Tab focuses link/button elements); cover at least one Apple platform (for Safari + reduced-motion macOS specifics) + one Chromium platform (Chrome or Edge, on Linux or Windows).

Per-browser feature checklist

For each browser × version, run through the steps below. Mark a row complete only when every step passes; a failing step is a blocker logged in the Known issues table.

# Step Expected behavior
1 Sign in via the configured OAuth provider Login → Dashboard transition is smooth; sidebar nav renders
2 Create a new thread Header CTA fires; lands on /threads/{id}
3 Submit a prompt against a small model Run row appears as streaming; right pane shows the Visualization tab by default
4 Watch the viz stream Three.js cascade animates layer-by-layer; particles spawn per token; ≥30 FPS sustained (FPS overlay in dev mode)
5 Toggle to Embeddings tab Vocab-cluster scatter loads (no blank canvas, no console errors)
6 Toggle to Debug tab Event stream renders as JSON; new events append in real time
7 Press Space on the play button Playback pauses; cursor → counter
8 Press Enter on a step button Cursor advances by one token
9 Use Tab/Enter/Space-only to navigate the whole page Every interactive element gets a visible focus ring (chunk-2 surgical fixes hold)
10 Press Esc inside the Share popover Popover closes; focus returns to the Share trigger
11 Copy the share URL Toast appears ("Share URL copied to clipboard.") in the top-right; clipboard contains the URL
12 Visit /share/{token} in a private window Public read-only view renders; "Anonymous view" indicator shows; About + Source links visible in footer
13 Click a terminal-run Replay link Replay page loads; PlaybackControls operate keyboard-only
14 Verify the AttentionHeatmap palette Viridis 5-stop gradient visible (dark purple → blue → green → yellow); no banding on adjacent cells
15 Verify the embedding scatter palette 8 distinct Okabe-Ito cluster colors visible against the slate-950 background; pronouns / numbers / code-keywords visually distinct
16 Trigger an export Download dropdown opens; GIF + MP4 items show a spinner during render; toast notifies on completion
17 Visit /this-does-not-exist NotFound page renders with the Compass icon + Sign in / Go back / Source CTAs
18 Open OS-level reduced-motion preference and toggle ON Reload thread page — Viz tab is disabled with title "...prefers-reduced-motion is set"; Embeddings still operational; Debug tab opens by default
19 Disable WebGL 2.0 (chrome://flags or about:config) Reload thread page — Viz + Embeddings both disabled with title "...does not support WebGL 2.0"; amber webgl-unsupported-notice banner visible; Debug tab opens by default
20 Open DevTools → console No uncaught errors, no React warnings, no missing-key warnings

Per-browser known-quirks reference

  • Safari (macOS / iOS): Tab key reaches buttons + links only when System Settings → Keyboard → Keyboard Navigation is on (or equivalent on iOS). Document this in the user-facing help if any future report says "I can't Tab to anything in Safari."
  • Safari (Webkit): WebGL 2 is gated behind the "WebGL via Metal" preference on some older macOS versions. Step 4 may fail on a Mac running macOS < 13 with WebGL 2 disabled in Develop menu.
  • Firefox: prefers-reduced-motion is toggled via about:preferences#general ("Browser zoom") on desktop or via OS settings. Step 18 should still flip correctly.
  • Edge (Chromium): behaves identically to Chrome for every test step. Listed separately because organizations may pin Edge for compliance reasons.
  • Chrome: the reference platform. Treat any Chrome failure as a structural bug, not a browser quirk.

Known issues

Browser Version Step Issue Status
empty until populated

The table starts empty by design — a fresh smoke run populates it. M15 launch-prep work tracks open issues to closure; non-blocking issues are documented as "won't fix" with a rationale (e.g., "Safari pre-15 lacks :has() selector — affects one cosmetic ring layer; ignore.").

M12 retrospective

What worked

  • Hard-fail axe gate from day 1 (chunk 1). Picking the strictest CI gate up-front meant chunks 2–8 couldn't accidentally regress structural a11y while landing other work. The cost (false positives blocking PRs) never materialized because the jsdom-safe rule config (color-contrast + scrollable-region-focusable disabled) eliminated the only two rules that fire spuriously in jsdom — everything else axe checks is deterministic.
  • The frontload-shadcn decision from M7 chunk 1 paid off across every M12 chunk. Chunk 1 found zero structural violations on its first run because Radix carries the ARIA story for every primitive. Chunk 2's keyboard audit found only 5 hand-rolled hot spots — Radix handled the other ~95%. Chunk 5's empty-state audit confirmed all 3 SPEC-named states already existed from M7. Chunk 6's Skeleton + Sonner integration slotted into the shadcn primitive tree without contortion. M12 would have been a much larger surface-area chunk if M7 had rolled its own button / popover / dialog.
  • Surgical fixes only, no sweeping refactors. Chunks 2 (focus-visible rings), 3 (motion-reduce gating), 5 (empty-state polish), and 7 (error-page icons) all stuck to "find what's missing, fix it in place, don't rewrite." Kept the chunk surface area predictable + the risk of regression near-zero. The opposite call would have been a "let's standardize on a focus-ring utility class" refactor across the entire codebase — much bigger blast radius with the same end-user benefit.
  • Per-decile aria-live throttling (chunk 3). Picking milestone announcements over per-event announcements avoided flooding the AT queue without losing the live-update signal. Screen-reader users hear "Run started… 10 tokens generated… Run complete." instead of 100 announcements back-to-back. A pure helper (deriveVizAnnouncement) made the logic testable without spinning up Three.js / WebGL stubs.
  • Color-blind palettes (chunk 4) closed the WCAG 2.1 SC 1.4.1 "Use of Color" target structurally. Viridis (heatmap) + Okabe-Ito (embedding clusters) + the existing single-hue magnitude bars (Logits + MoE) + text-paired status pills mean every information channel that could rely solely on color now doesn't. The two near-duplicate cyan / amber pairs in the old M8 cluster palette are gone.
  • useFlashToast hook centralized the inline-alert → toast migration (chunk 6). Single source of truth for the flash-status → toast-variant mapping. The hook is opt-in by virtue of mounting in AppLayout, so any page wrapped in AppLayout gets toasts for free — no per-page wiring. Removed two inline emerald/amber blocks from ApiKeys/Index to avoid double-announce; the rest of the inline blocks (Admin pages, etc.) keep working via the natural flash-prop flow until a future chunk cleans them up.
  • Splitting 503 into a dedicated Maintenance.tsx (chunk 7). Tiny code change, big semantic-correctness win. 500 ("we logged an error, try again") and 503 ("service intentionally unavailable, retrying won't help") deserve different copy + different icons. The old shared ServerError was a UX shortcut hiding two real states.
  • Composed gate (chunk 8) over a single supported-or-not boolean. Modeling vizDisabled = reducedMotion || !webgl2Supported vs embeddingsDisabled = !webgl2Supported separately let us preserve the M8 decision that the embedding cloud is consumable under reduced motion. A single boolean would have either disabled too much or too little. Each gate has its own reason; each tab carries its own title attribute explaining why it's disabled.

What we learned

  • jsdom hides three classes of a11y signal that need a real browser. (1) Computed CSS is mostly rgba(0,0,0,0), so color-contrast axe rules false-positive everywhere; we disabled the rule. (2) WebGL contexts can't be created, so the chunk-8 probe correctly returns false and triggers the fallback path — useful for one test, but it means setup.ts has to globally mock useWebGL2Support so the existing 60+ page tests don't all flip to Debug mode. (3) prefers-reduced-motion media-query evaluation requires a real window.matchMedia implementation; we already had a matchMedia stub from M8 chunk 9. Cross-browser real-browser smoke (this chunk's recipe) covers what jsdom misses.
  • Vite manifest stale on new Pages/*.tsx. Hit this twice in M12 (chunk 1 adding a11y/ test renders that touch Inertia view-finder; chunk 7 adding Pages/Errors/Maintenance.tsx). Same lesson as M11. The canonical fix: rebuild between adding a page file and running Pest tests that render it. Worth adding to the M15 release-checklist as a final-pass gate.
  • Object.defineProperty(navigator, 'clipboard', ...) over Object.assign(navigator, ...). jsdom 29 protects navigator.clipboard with a getter when an earlier test file in the same run has already defined it. Object.assign then throws; Object.defineProperty with configurable: true, writable: true bypasses the getter. The ShareMenu chunk-3 (M11) test got lucky because it ran first in its old order; the chunk-2 (M12) keyboard test exposed the issue. Pattern documented in keyboard.test.tsx.
  • ESLint's react-hooks/set-state-in-effect rule fires on legitimate sync-with-external-system effects. The useWebGL2Support hook needs to re-probe after mount to correct the optimistic true initial state down to the real false in unsupported runtimes — that's a synchronization-with-external-system effect, the exact case the React docs say is legitimate. We disable the rule per-line with an explicit // eslint-disable-next-line react-hooks/set-state-in-effect and a comment explaining why. The reduced-motion hook (M8) does the same pattern; both are documented inline.
  • Pint reformat still breaks chained Edits. Same lesson M10/M11 documented. Hit again in chunk 3 (rebuilding Threads/Show.tsx whitespace), chunk 6 (useFlashToast import), chunk 7 (bootstrap/app.php). The pattern is unchanged: re-Read after the hook runs, then re-Edit.
  • react/no-unescaped-entities catches typographic punctuation in JSX text. Chunk 6's toast hook + chunk 8's WebGL banner both tripped the rule. &apos; / &ldquo; / &rdquo; are the mechanical fixes. Worth keeping in mind any time user-facing prose lands directly in JSX.

Parked / carry-forward to M15

  • Real-browser cross-browser smoke pass. The recipe above is the manual verification path; the actual run produces the Known Issues table content. M15 Launch Prep owns the first full run + populating the table. Subsequent runs become release-gate work.
  • Playwright / Cypress E2E. Same call M6 / M8 / M9 / M10 / M11 closeouts all made. Unit + integration + manual recipes is enough for Phase 1 ship; browser-automated E2E is a Phase 2 / launch-readiness expansion. M12 chunk 1's jest-axe matcher + chunk 8's vi.mock pattern would adapt directly to @axe-core/playwright when E2E lands.
  • Color-contrast verification (real CSS, real browsers). The chunk-1 jsdom-disabled rules (color-contrast, scrollable-region-focusable) need real-browser verification before we can claim WCAG 2.1 AA conformance with confidence. The M15 cross-browser pass (step 14, 15) covers the viridis + Okabe-Ito visual verification; a parallel run of @axe-core/playwright in a future chunk could automate it.
  • Screen reader pass (NVDA / VoiceOver). The SPEC's M12 task line mentions "manual screen-reader pass" alongside the axe-core CI part. Chunk 1 landed the CI half; the manual SR pass is parked for M15 (or earlier if a contributor with an SR is available — the work scales linearly with viz / pages and stays under 1 hour for the vertical slice).
  • Authed pages link to /about. Per M11 chunk 4 + the M12 chunk-6 toast pattern decision, only the public surfaces (SharedLayout + Welcome) link to /about today. Dashboard, Threads, Settings, Admin don't surface an About / "What is this?" link. M15's AGPL §13 audit owns this — add a footer link to AppLayout.
  • Sentry event ID on 500 page. Chunk 7 decision recorded this. M15 surface for the ServerError page so support requests can quote a reference ID without exposing a stack trace.
  • No-usable-models empty state polish. Chunk-5 decision parked it. The state fires when a user has an API key but the registry doesn't carry a model for their vendor — a registry-staleness edge case. M15 could surface a "ping an admin" affordance or auto-trigger the registry-refresh fetch.
  • Three hand-rolled tablists could migrate to @radix-ui/react-tabs. Chunk-2 decision recorded this. Would inherit arrow-key navigation between tabs (WAI-ARIA Authoring Practices best practice, not WCAG conformance failure). ~150 lines of churn across 4 pages; saved for a polish chunk if anyone files it.
  • The 3 moderate-severity ws CVEs in socket.io-client transitives are still open. Documented in chunk 1's commit body. npm audit fix would update them; deferring to avoid a major-version bump on the WebSocket layer outside a release window.

Exit criteria

  • axe-core reports zero violations on dashboard, prompt-input, thread-detail, share-view pages. Mapped to the four pages in resources/js/__tests__/a11y/: Dashboard.a11y.test.tsx (Dashboard + empty + loaded), Threads.Show.a11y.test.tsx (prompt-input lives inside Threads/Show via PromptFooter; covered in empty + loaded + archived states), Share.Show.a11y.test.tsx (Share/Show in empty + loaded + redacted states), Share.Replay.a11y.test.tsx (Share/Replay in complete + errored + redacted states). 11 tests, hard-fail gate. Caveat: jsdom-disabled rules (color-contrast, scrollable-region-focusable) carry over to M15 real-browser pass.
  • Manual keyboard navigation can run a full vertical slice (sign in → new thread → submit → watch viz → replay) without a mouse. Manual recipe in §"Keyboard-only manual recipe (chunk 2)" above. Automated coverage: 7 Vitest keyboard tests in resources/js/__tests__/a11y/keyboard.test.tsx. Chunk-2 surgical focus-visible fixes (19 buttons across 5 hot spots) closed the last gaps.

Bonus M12 work beyond the SPEC's two exit criteria:

  • Three.js canvases announce state transitions via role="status" aria-live="polite" (chunk 3, per-decile milestones)
  • motion-safe: prefix on every animate-* class (chunk 3, prefers-reduced-motion compliance for inline animations)
  • CB-safe palettes everywhere color carries meaning (chunk 4, viridis + Okabe-Ito; magnitude bars + text-paired status pills are intrinsically safe)
  • 8 empty-state surfaces locked in tests (chunk 5, icon + title + body + CTA-href invariants)
  • Skeleton + Sonner toaster wired (chunk 6, both via shadcn primitives)
  • 5 error pages with icons + Go-back CTA + 503 split (chunk 7)
  • WebGL 2.0 fallback to the 2D Debug tab (chunk 8, composed with reduced-motion gate)
  • Cross-browser smoke recipe published for M15 verification (chunk 9)

M13 — Cinematic Inference Visualization

Purpose: Replace the M8 live-cascade visualization with a 20-scene cinematic narrative through the entire inference pipeline — prompt entry, byte-encoding, BPE tokenization, embedding lookup, positional encoding, every step inside a transformer block (layer norm, Q/K/V projection, multi-head attention, residual, FFN, second residual), the layer-stack loop, final logits, sampling, detokenization, and the autoregressive loop with KV cache. Design source: docs/visualization.md. The pipeline takes ~60-90s at explanatory speed; the real run streams in parallel and finishes faster, with its tokens accumulating in a chat-bubble overlay while the viz keeps playing.

Status of M8 viz components: M8 is locked at ✅ Complete. The historical M8 docs + decisions remain in this file as-is (the cascade + particles + EmbeddingScene shipped a usable feature for several milestones). M13 is a rip-and-replace per user direction at the chunk-pre-discussion: chunk 1 deletes the M8 viz components + their tests outright so chunks 2-14 don't have to code around dead weight. The git history preserves the M8 implementation for anyone who wants to spelunk; phase1.md keeps the documentation. No feature flag, no parallel mounts — clean slate from chunk 1.

Visual language (per docs/visualization.md):

  • Tokens = rounded rectangles, color from a hash of the token ID
  • Vectors = horizontal 1D heatmap strips (viridis ramp, mostly-implied long axis via "…")
  • Matrices = 2D heatmap grids (also viridis)
  • Weights / parameters = dimmed gray, "always there in the background"
  • Active computation = bright + animated, traveling particles or pulses
  • Camera = mostly fixed per scene, with a few zoom-out reveals (notably Scene 5 and Scene 12)

Persistent UI in the visualization region (per user direction on chunk-discussion: "make the visualization div larger to hold everything; these are sections in it"):

  • Vocabulary sidebar (left edge) — scrolling list + highlight on token lookup, populated during BPE in Scene 3
  • Chat bubble (bottom-right) — streaming output area; mirrors the real run's tokens as they arrive on the WebSocket
  • Layer counter HUD (top-right) — visible during the tower scene (12) and across all stage-3 scenes
  • Pipeline progress bar (top, 20 segments) — shows which scene is currently active + click-to-jump

Tasks

  • Chunk 1 — Rip M8 + scene architecture + page layout. Deletion-first chunk:

    • Delete M8 viz components: resources/js/Components/Viz/VizPane.tsx, EmbeddingScene.tsx, CascadeController.ts, ParticleSystem.ts, TransformerStack.ts, subComponents.ts, FpsCounter.tsx (keep — the new viz reuses it), AttentionHeatmap.tsx (keep — chunks 5/8 reuse it). Delete their Vitest files. Delete the VizSkeleton.tsx Suspense fallback from M12 chunk 6 and its test — the new viz won't need a Three.js lazy-import skeleton in the same shape.
    • Drop the right-pane Visualization / Embeddings tabs from Pages/Threads/Show.tsx, Pages/Runs/Replay.tsx, Pages/Share/Replay.tsx. The cinematic viz is a single mount, not a tabbed surface. Keep the Debug tab as the WebGL-2-unsupported fallback target (chunk 13 evolves it into the SVG fallback path).
    • Drop the M12 chunk-8 WebGL gate banner copy that says "the 3D visualization + embedding scenes are unavailable" — replace in chunk 13 with the new tri-state messaging.
    • Scene<I, O> interface (input, onComplete(output), durationMs, normalized progress t in [0,1]). SceneRunner state machine that walks scenes 0→20, threads each scene's output into the next scene's input, owns t advancement, exposes pause / resume / scrub / setScene / setSpeed.
    • Page-layout change: Pages/Threads/Show.tsx + Pages/Runs/Replay.tsx + Pages/Share/Replay.tsx widen the visualization region (target: viz takes 2/3 of the row, transcript+prompt take 1/3 — flip of the current 1/3 viz : 2/3 transcript split). Placeholder mounts for the four persistent UI sections (vocab sidebar, chat bubble, layer counter, pipeline progress bar) so chunks 2+ can drop content into them without touching layout.
    • Stub <CinematicViz events={...} model={...}/> mounted in place of the old VizPane. Renders an empty canvas + the four persistent UI placeholders + a "Scene 0 / 20" placeholder text so the visible layout is correct, even though no scenes are implemented yet.
    • Updates __tests__/a11y/webgl-fallback.test.tsx + __tests__/Pages/Threads/Show.test.tsx + Runs/Replay.test.tsx + Share/Replay.test.tsx for the new mount shape (no more view-viz / view-embeddings / view-debug tabs to assert on).
    • Full suite must stay green at the chunk-1 push (every M8 viz test deleted, every page test updated for the new mount). The cinematic-viz tests are placeholders that just assert "Scene 0 / 20" renders.
  • Chunk 2 — Visual-language primitive components. Pure presenters, no scene logic:

    • <TokenPill id={n} label={'token'} idHash={u32}/> — rounded rect, hue from idHash % 360, displays string + optional ID number
    • <VectorStrip values={number[]} visibleCells={128} totalLength={4096}/> — 1D heatmap, viridis ramp, trailing indicator
    • <MatrixGrid values={number[][]} dim={[rows, cols]} maxVisible={32}/> — 2D heatmap with same fade-out for off-screen extent
    • <WeightFog/> — dimmed gray background mesh that sits behind active components
    • <ParticleTrail from={Vector3} to={Vector3} colorFromHash={n}/> — single-line pulse animation
    • All TypeScript-typed, all Vitest-tested in isolation (no Three.js required for the SVG variants; Three.js for the ParticleTrail).
  • Chunk 3 — Scenes 0-4 (text → tokens). Scene 0: prompt-entry textbox in the center, types in (or pastes), glows once, characters detach + float. Scene 1: char bounce + 180° flip to byte values, staggered left-to-right (~30ms/char); emoji + non-Latin chars visibly split into 2-4 bytes ("bytes ≠ characters" teaching beat). Scene 2: chat-template wrap — system-prompt block slides in from above (purple tint), <|user|> / <|assistant|> markers (teal tint) bracket the row; special-token strings show as text then collapse to byte integers. Scene 3: BPE animation — brackets slide in around 1-4 adjacent bytes, snap-lock, fuse into a token pill labeled with the token string; concurrent line shoots to the vocabulary sidebar, scrolls + highlights the matching row. Common tokens fast (~200ms), rare words slow (~600ms). End with strings fading, IDs remaining. Scene 4: 0.5s breather — clean integer row + label "context length: N tokens." Tokenizer-pick constraint (per user direction): must work in every browser the M12 chunk-9 smoke recipe covers — Chrome, Firefox, Edge, Safari (latest two of each). Candidates: @dqbd/tiktoken (Wasm-backed; needs WebAssembly, which Safari has had since 11 + every modern Chromium/Firefox supports; faster steady-state, ~600KB bundle); js-tiktoken (pure-JS port, ~200KB, slower but zero runtime requirements beyond ES2020). Measure cold-start + per-call latency in chunk 3 against the SPEC's "explanatory speed" budget and pick whichever doesn't make Scene 3 feel sluggish on Safari, the slowest target. Document the choice + the measured numbers in the chunk 3 decisions block.

  • Chunk 4 — Scenes 5-7 (embedding → norm). Scene 5: the first "wow" moment. Camera zooms out, token row shrinks to the bottom, a massive wireframe matrix labeled "Embedding table: 128,000 × 4096" materializes (rendered as ~16×16 visible cells with extents, but the label communicates the real scale). Per-token vertical beams shoot up to specific matrix rows; the rows detach + slide down to replace the pills as horizontal vector strips. Scene 6: positional encoding / RoPE. Position counter (0, 1, 2…) appears above each strip; a subtle shear/wave or 3D rotation around the long axis with rotation amount proportional to position. Brief "θ_pos" annotation on the first few. Scene 7: layer norm — strips temporarily render as bar charts (heights = values), squish animation equalizes the distribution, snaps back to heatmap form.

  • Chunk 5 — Scene 8 (multi-head self-attention, the centerpiece). Three sub-beats:

    • 8a (1.5s): From each input strip, three new strips bud off color-coded — Q (blue), K (red), V (green) — arranging into three parallel rows
    • 8b (3s): Attention-matrix grid materializes (N×N, sequence length). For each (i, j) with j ≤ i (causal mask), brief arc/laser from Q_i to K_j; matrix cell lights with intensity proportional to the dot product. Cells above diagonal stay black. Row-by-row softmax animation rebalances each row to sum to 1. Multi-head reveal: the matrix is actually a stack of ~32 transparent layers (per model attention_heads); fan them out briefly to show parallelism, collapse back. Heads concatenate into one output via a quick "squeeze" animation
    • 8c (2s): For each output position, V vectors pull in with opacity matching attention weight, blend, form a new output strip. Detailed for positions 0-2, fast-forward the rest
    • Animation values seeded deterministically from (run.id, layer_index, token_index) so a replay is frame-identical.
  • Chunk 6 — Scenes 9-11 (residual + FFN + second residual). Scene 9 (1s): pre-attention vectors (kept ghosted in the background since Scene 6) slide forward + merge with attention output via a glowing + symbol. Scene 10 (3s): FFN — strips flow through a "pipe" that expands to ~4x width (hidden_dim → 4 × hidden_dim), passes through a non-linearity (wavy color shift representing SwiGLU/GELU per model.architecture_type), contracts back. Tiny "neuron firing" sparkles inside the expansion. Scene 11 (1s): pre-FFN ghosts merge with FFN output — mirror of Scene 9.

  • Chunk 7 — Scene 12 (layer-stack loop / tower view). The big camera move. Camera pulls way back revealing that scenes 5-11 happened on a single floor of a tall tower (one floor per layer, count from model.layers ?? 32). A glowing packet (the current vector sequence) sits on floor 1. As it moves up to floor 2, camera follows partway then accelerates — floors 3 through (N-2) blur past in ~3s with a layer counter HUD ticking up rapidly ("Layer 03… 17… 42… 78…"). Camera slows for floor N-1 and N, where we re-zoom for full detail on the final layer's attention + FFN. Auxiliary mini-map stays docked in the corner showing "Layer 42 / 80" with a progress bar throughout — populated even when not on Scene 12 so the user can orient at any time.

  • Chunk 8 — Scenes 13-17 (final norm → logits → sampling → emit). Scene 13 (0.5s): brief final layer norm squish. Scene 14 (3s): only the LAST vector strip matters for next-token prediction — others dim + fade. Final strip floats center, projects through "LM Head" matrix (4096 × 128000, mirror of Scene 5). Beams shoot through, emerge as a 128k-long mostly-cool-colored heatmap with a few hot spikes. Scene 15 (2s): logits row pivots 90° → horizontal bar chart sorted descending. Softmax wave rescales bars to probabilities summing to 1. Most collapse near-zero; handful at left dominate. Optional temperature-slider corner showing high vs low temp distributions. Scene 16 (1.5s): sampling. Mode-aware: greedy dart slams bar #1; top-k fades bars beyond k, dart wobbles among survivors; top-p sweeps a fill-line until cumulative-p reached. Chosen bar pulses; winning token string flashes above. Scene 17 (0.5s): winning token flies down, appends to "generated so far" tray + the chat-bubble UI in the corner.

  • Chunk 9 — Scenes 18-20 (autoregressive loop + KV cache + detokenization). Scene 18 (variable): meta-loop. Full sequence (input + generated so far) becomes new input. Compress Scenes 5-17 into ~2s for token #2, ~1.5s for #3, accelerating to ~200ms/token (5/sec) by token #10+. The real run drives token timing via the WebSocket; if the run finishes before the viz catches up, viz keeps playing at its accelerated pace while the chat bubble already has the full text. Scene 19 (2s, one-time reveal during first loop iteration): KV cache. Pause briefly; show K + V matrices from previous tokens appearing in a dimmed "cache drawer" with a small lock/disk icon. For the new token, only its row of K + V is fresh-computed + added. Attention matrix gains a single new bottom row per step instead of recomputing the whole grid. Subsequent loop iterations skip the explanatory beat; cache drawer just fills. Scene 20 (continuous during Scene 18): detokenization — each chosen token ID briefly transforms back to its string fragment via reverse-lookup highlight in the vocab sidebar, then flies into the chat bubble + concatenates. End-of-sequence token: chat bubble glows, full pipeline canvas dims, completion flourish.

  • Chunk 10 — Persistent UI completion. Wire the four sections to live data:

    • Vocabulary sidebar (left): scrolling list of (id, string) populated during Scene 3's tokenization. Persists through all 20 scenes. Highlight + scroll-into-view on lookup events from Scene 3 (forward) + Scene 20 (reverse).
    • Chat bubble (bottom-right): mock chat-message UI. Starts empty; grows as Scene 17 appends each token's string. Driven directly by the real token.received WebSocket event so the user sees the response coming in even if the visualization hasn't reached Scene 17 yet for that token.
    • Layer counter HUD (top-right): integer counter + tower progress mini-bar. Visible during scenes 5-12. Auto-hides during scenes 0-4 + 13-20.
    • Pipeline progress bar (top, 20 segments): full-width strip, each segment named ("Prompt", "Bytes", "Chat tpl", "BPE", "IDs", "Embed", "RoPE", "Norm", "Attention", "Residual", "FFN", "Residual", "Layers", "Norm", "LM head", "Softmax", "Sample", "Emit", "Loop", "Cache", "Detok"). Current scene highlighted. Click any segment → jump to that scene's start (uses SceneRunner's setScene(n)).
  • Chunk 11 — Playback controls + per-scene scrub. Adapt the M8 chunk-8 PlaybackControls to scene-level. Speed buttons: 0.25× / 1× / 4× (per docs/visualization.md production notes). Step: advance to next scene-boundary, not next event. Jump-to-live: catch up to the scene that matches the current token-stream position. Click any vector strip to expand it into a numerical-values panel (per the production note "let the user click any vector strip to expand it and see actual numerical values"). The expanded panel docks to the side, doesn't block the canvas.

  • Chunk 12 — Performance hardening (20-30 FPS target). Per user direction: aim for 20-30 FPS sustained. Simpler shapes are OK as long as the idea carries. Monitor FPS via the existing FpsCounter overlay (M8 chunk 4); add a degraded-mode automatic fallback when FPS drops below 18 for 2+ seconds:

    • Multi-head attention stack → single representative head shown
    • Particle trails → straight-line connectors without animation
    • WeightFog → static texture instead of animated noise
    • VectorStrip cell count clamped to 64 (down from 128) on the degraded path Document the degrade triggers + restoration condition (< 18 FPS for 2s → degrade; > 24 FPS for 5s → restore) inline + in the M13 retrospective.
  • Chunk 13 — Reduced-motion + WebGL gate adaptation. The M12 chunk-8 gate disabled the M8 Viz + Embeddings tabs entirely when WebGL 2 is missing or reduced-motion is set. The new cinematic viz can degrade more gracefully:

    • Reduced-motion: scenes still display, but as static frames. The user advances scene-by-scene via the chunk-11 Step button (or auto-advances on a slow 4-second-per-scene timer). No camera moves, no particle trails, no continuous animations. Equivalent of "PowerPoint mode."
    • WebGL 2 unavailable: fall back to the SVG variants of every primitive. The 2D Debug-tab fallback from M12 chunk 8 remains as the absolute-last-resort path. New webgl-unsupported-notice copy explains: "3D camera moves are unavailable; the visualization is rendering in 2D mode." Replaces the M12 chunk-8 binary disable/fallback with a tri-state (full / 2D-svg / debug-text).
  • Chunk 14 — Tests + closeout. Vitest assertions per scene (deterministic synth check via fixed seed; snapshot of the scene's output state given a known input state). Integration test: full-pipeline end-to-end with a mock event stream that fires through scenes 0-20 + asserts the chat bubble accumulates the right output. Manual recipe (matches M8 chunk-9 + M9 + M11 patterns): visual review per-scene, FPS spot-check, degraded-mode visual confirmation, reduced-motion visual confirmation, share-link verification (the cinematic viz needs to work for the public /share/{token}/runs/{run}/replay route too). Cross-browser smoke on the new viz (per M15 recipe step list — extend the table with cinematic-specific rows: every scene renders, scrubber jumps to each scene, FPS stays in band on each browser). M13 retrospective covering all 14 chunks. Status table line 25 (M8) stays ✅ Complete but gains a "(superseded by M13)" note. Flip M13 status to ✅ Done. (Deletion of M8 components happened in chunk 1; this chunk has nothing left to clean up.)

Exit criteria

  • The 20-scene narrative runs end-to-end on a fresh thread submit with a real OpenAI run, hitting all milestones from prompt-entry through end-of-sequence flourish in 60-90 seconds at 1× speed.
  • The real run's tokens stream into the chat-bubble overlay independent of (and typically faster than) the visualization — the user sees the response build in the corner while the viz keeps playing.
  • 20-30 FPS sustained on a 2020-class machine; automatic degrade triggers cleanly when FPS dips, restores when stable again.
  • Reduced-motion mode shows every scene as a static frame the user can step through.
  • WebGL 2 unavailable mode shows the SVG fallback; both modes still produce all 20 scenes (just rendered differently).
  • The persistent UI sections (vocab sidebar, chat bubble, layer counter, pipeline progress) are populated and click-targets work (sidebar highlights on lookup, progress-bar segments jump to scene).
  • The /share/{token}/runs/{run}/replay public route shows the same visualization as the owner view (no owner-only features in the viz layer).
  • M8 components retired: VizPane.tsx, EmbeddingScene.tsx, related controllers + tests removed from the codebase.

Architectural notes

  • Each scene is self-contained. Per the production note in docs/visualization.md: "Build each scene as a self-contained component that accepts its input state as props and emits its output state on completion. That lets you replay individual scenes for debugging and lets users jump to any point in the pipeline." This is the major architectural shift from M8's concurrent-renderers-driven-by-shared-event-stream model. The new SceneRunner walks scenes sequentially; each scene owns the canvas during its window.
  • Observable vs illustrative. The vendor API only exposes the input prompt, the streaming token IDs/strings, per-token logprobs (sometimes), and timing. Everything else is illustrative: BPE animation (we tokenize client-side via tiktoken), embedding lookup, RoPE rotation, layer norm, Q/K/V projections, attention scores, FFN expansion, LM-head projection — all rendered deterministically from (run.id, model_metadata, token_index, layer_index) seeds. The visualization is a pedagogical model of inference, not a measurement of it. Per user: "this whole site is an education tool."
  • Real run continues independently. The M6 streaming pipeline + the M8 useRunStream hook keep doing exactly what they do. The chat bubble subscribes to the same WebSocket events and displays tokens as they arrive. Scenes 18-20 use the same event timeline to drive the autoregressive-loop pacing, but the viz lagging behind is expected + acceptable.
  • WebSocket → Scene coupling. Scenes 5-17 fire once per generated token. The current RunEvent schema (M6) emits one token.received per token; the SceneRunner subscribes + advances. For the input tokens (the prompt), the runner enters Scenes 0-4 at submit time without waiting for the WebSocket.

Decisions (chunk 13): (Reduced-motion + WebGL gate tri-state.)

  • Reduced-motion = PowerPoint mode. Each scene renders at t = 1 (completion frame). Autoplay disabled, scene runner doesn't tick through t. 4-second setTimeout auto-advances to the next scene; Step button in PlaybackControls always works as a manual override. Once the runner reaches the last scene, no auto-advance (the timer guards sceneIndex >= totalScenes - 1).
  • t = 1 is the chosen pin. Each scene's last frame is its most informative static view — Scene 5 shows the embedding strips (post-detach), Scene 16 shows the chosen bar pulse, Scene 20 shows the EOS flourish. t = 0.5 was considered but most scenes have peak content mid-window only briefly; the completion frame stays maximally informative.
  • All reduced-motion logic lives in CinematicViz, not in useSceneRunner. The hook is untouched: we pass autoplay: false, override the rendered t, and run a sibling setTimeout for the 4s advance. Keeps the runner pure (it still drives t for non-reduced-motion paths), and the reduced-motion behaviour is one cohesive 15-line block in CinematicViz with clear comments.
  • WebGL 2 gate = informational only. The chunk-1 decisions block locked "All 5 primitives are SVG/HTML/CSS — no Three.js" — every M13 scene already renders via SVG. There's no 3D scene to fall back from. The notice exists because the spec calls for it, but functionally the viz behaves identically with WebGL2 missing. The copy explains this honestly: "3D camera moves are unavailable; the visualization is rendering in 2D mode."
  • Tri-state precedence: WebGL gate wins over reduced-motion when both apply. The WebGL notice is the more critical info (different rendering pipeline), reduced-motion is a comfort/accessibility mode. Only one notice surface; the more informative one shows. The reduced-motion behaviour (t = 1 pin + auto-advance) still applies underneath even when the WebGL notice covers the visible message.
  • data-gate-mode attribute on the notice (webgl / reduced-motion). Lets tests assert which gate fired without parsing the human-readable copy. Pattern matches the chunk-12 FpsCounter's data-degraded.
  • data-reduced-motion="true|false" on the canvas root. Sister assertion to the gate notice — confirms the t-pin behaviour is active regardless of which notice (if any) is visible. Used by the chunk-13 tests to verify the "both gates" case.
  • Gate-notice tests use dynamic import + vi.doMock. The global setup defaults useWebGL2Support to true and useReducedMotion to false (via the matchMedia stub). Chunk-13 suites override per-suite via vi.doMock + vi.resetModules() + dynamic await import('@/Components/Viz/CinematicViz'). The vi.resetModules() call in beforeEach is required — without it the first test in each suite inherits the already-loaded module from before the mock applied. Documented inline.
  • The chunk-1 a11y test (webgl-fallback.test.tsx) was updated to match the chunk-13 copy. Its comment block explicitly anticipated this change ("Chunk 13 evolves the gate into the tri-state… for now we just assert that the notice renders"). Updated to assert the new spec literals + data-gate-mode="webgl" rather than the chunk-1 "WebGL 2.0" placeholder copy.
  • No behavioural change for ParticleTrail/etc. ParticleTrail already gates its animation via motion-safe: (chunk 2) which respects prefers-reduced-motion natively. The chunk-12 degraded-mode hook handles FPS-driven trims. Chunk 13's contribution is the scene-runner-level t-pin + auto-advance, not per-component animation overrides — those existed already and continue to fire.
  • Auto-advance is per scene, not per token. Spec says "auto-advances on a slow 4-second-per-scene timer" — chunk 13 uses a fresh setTimeout(controls.nextScene, 4000) per scene-change, cleared on unmount or sceneIndex change. Any manual setScene() call (PipelineProgressBar click, PlaybackControls Step) cancels the pending advance and arms a new one for the new scene.

Decisions (chunk 12): (Performance hardening + degraded-mode state machine.)

  • State machine in useFpsTracker, transport via PerformanceModeContext. Hook drives the RAF loop + hysteresis; context flows { fps, degraded } to ~13 scene/component callsites. Same pattern as chunk 11b's VectorInspection. Mounting the hook in CinematicViz keeps a single global tracker; per-component subscribers stay declarative.
  • Hysteresis thresholds match the spec literal: < 18 FPS for 2s → degrade; > 24 FPS for 5s → restore. Asymmetric on purpose — degrade fast, restore slow. The 6-point gap between thresholds (18 vs 24) prevents oscillation when the actual FPS sits around the boundary. Exported as FPS_TRACKER_CONFIG for tests + decision-block citation.
  • Decisions made every UPDATE_INTERVAL_MS (250ms), not every frame. The displayed FPS already updates at this cadence; the state machine piggy-backs. A per-frame decision would consume CPU for no behavioral gain — hysteresis windows are measured in seconds.
  • WeightFog is already at the degraded target. Spec calls for "WeightFog → static texture instead of animated noise"; the chunk-2 implementation never animated (single SVG <pattern> with no <animate>). Degraded mode is a no-op for WeightFog. Documented inline so a future reader doesn't wonder why no opt-in code lives there.
  • ParticleTrail's degraded mode strips the CSS animation outright, not via motion-safe: gating. The chunk-2 component already freezes under prefers-reduced-motion via the motion-safe: variant. Degraded mode is independent (FPS-driven, not preference-driven) and stronger — it removes the animation class + style entirely, so even a non-reduced-motion user gets a static dot. The data-degraded="true" attribute makes the behavior testable.
  • VectorStrip degraded path clamps via Math.min(visibleCells, 64), not a fixed-64 override. Callers that already pass visibleCells < 64 (e.g., the chunk-5 AttentionScene Q/K/V rows at 32) stay at their requested count. Degraded never INCREASES the cell count.
  • AttentionScene shows headMatrices.slice(0, 1) in degraded mode. Drops from 6 representative heads to 1. Caption gains a " · degraded" suffix so the viewer can see the mode change. The full attention math is still computed for the head 0; only the multi-head fan is collapsed.
  • FpsCounter visible when import.meta.env.DEV || ?debug=fps query param. Per user direction — dev-only by default, but production users can append ?debug=fps to see the overlay without a rebuild. Useful for spotting field perf issues without baking the overlay into shipping pages. The state machine itself runs in all builds; only the overlay is gated.
  • No useReducedMotion bypass. Considered: skip the state machine + force degraded when prefers-reduced-motion is set. Rejected because reduced-motion is its own thing (chunk 13) with a different degradation profile (per-scene step-through, not just animation suppression). Mixing the two would conflate accessibility with perf.
  • Tests use a faux RAF + mocked performance.now. jsdom doesn't drive frames; the chunk-12 tests advance time + drain the RAF queue manually. Pattern: vi.spyOn(window, 'requestAnimationFrame') captures callbacks, advance(dt) flushes them at the chosen interval. Same approach scales to chunk 11's useSceneRunner if it ever needs deep RAF tests.
  • Two new contexts now wrap the canvas. <PerformanceModeProvider> sits outermost, <VectorInspectionProvider> nested inside. Outermost = read by the most consumers (every VectorStrip, ParticleTrail, AttentionScene); inner = scoped to the canvas region where strips can be clicked. Mirror of the React context "by frequency-of-use" pattern.

Decisions (chunk 11b): (Click-to-expand numerical-values panel — completing chunk 11.)

  • Opt-in via React context, not per-scene wiring. <VectorInspectionProvider> wraps the canvas; VectorStrip calls useVectorInspection() and auto-becomes clickable when inside the provider. Outside the provider the strip stays non-interactive (the chunk-2 behaviour). Means ~100 strip mounts across 20 scenes get the feature for free, no scene code changes.
  • Context's open / close are useCallback-stable. The first draft had an infinite render loop: tests' useEffect([inspection], () => inspection.open(...)) re-fired each time setActive recreated the context value. Fix is stable callbacks via useCallback (empty deps); tests now depend on open not inspection. Documented in the context source as a footgun warning.
  • VectorStrip becomes role="button" + tabIndex={0} only when inspectable. Outside the provider it stays role="img" (the chunk-2 default — non-focusable). Inside the provider, Enter + Space keyboard activation matches the M12 chunk-2 a11y pattern. focus-visible ring on cyan. Hover gives a subtle shadow.
  • NumericalValuesPanel docks as a right-overlay taking ~40% of the canvas, with a 60%-width backdrop on the left. Click backdrop OR the × button OR press Escape to dismiss. Empty-state (no active inspection) returns null — no DOM weight.
  • Cell list caps at 64 entries. Renders as a 2-column grid with index + value + viridis swatch. Big vectors (e.g., 4096-cell embeddings) show "First 64 cells (of 4096)" header. Stats use the FULL vector, not just the rendered 64 — dim/mean/std/min-max all reflect the complete data.
  • Stats grid is dim / mean / std / range, in a 4-column row. Range is "min…max" with em-dash separator. All values use formatNumber: 3 decimals for |v| < 100, fewer for larger magnitudes. Test pins the mean of [1,2,3,4] at 2.500.
  • Escape-to-dismiss via a global keydown listener mounted only while the panel is active. Attaches in useEffect([inspection]), removes on cleanup. Doesn't impair other keyboard handlers when the panel is closed.
  • inspectionLabel prop on VectorStrip falls through to caption then "Unnamed vector". Gives every scene a chance to set a meaningful label (e.g., "Layer 5 hidden state · token 3") without requiring it. Most existing scenes already use captions, so the fallback chain works without per-scene changes.
  • Empty vectors are explicitly NOT inspectable (data-inspectable="false"). Clicking a zero-cell strip would open a panel with nothing useful. Tests assert this.
  • VectorInspectionProvider mounts INSIDE the chat-bubble / layer-counter scope in CinematicViz, so the panel only overlays the scene canvas — not the chat bubble or vocab sidebar (those stay accessible). The backdrop blur is light (1px) to keep the underlying scene readable as context.
  • Chunk 11 complete with both halves. PlaybackControls (chunk 11a) + click-to-expand panel (chunk 11b). All four affordances from the spec line are now wired: speed selector, step, jump-to-live, click-strip-to-expand.

Decisions (chunk 11a): (PlaybackControls — chunk 11 is mid-flight; superseded by chunk 11b.)

  • useSceneRunner controls already covered the needs. Speed (PLAYBACK_SPEEDS already [0.25, 1, 4]), play/pause/toggle, prev/next scene (= "step to scene-boundary" per chunk-11 spec). No runner changes needed — chunk 11a is pure UI + jump-to-live derivation.
  • PlaybackControls is a pure render component taking state + controls + liveSceneIndex. CinematicViz computes liveSceneIndex from the events stream and passes it through. Same pattern as the chunk-1 PipelineProgressBar (state + onSelectScene). Keeps the chunk-11 component pure-render, easy to test.
  • Jump-to-live heuristic matches the chunk-11 design decision: no events → null → button disabled; any token.received → setScene(18); is_final → setScene(20). Documented in the inline comment + tested explicitly. The button is also disabled when the live target equals the current scene — "no jump needed" UX state.
  • Step = nextScene() / prevScene() only. Spec explicitly calls out scene-boundary step, not event-boundary step. The runner already exposes both; mapping them to ◂◂ / ▸▸ is straightforward. No event-stepping mode — would conflict with the synthetic-vs-real timing split.
  • Speed selector is role="radiogroup" with three role="radio" buttons. Per the M12 chunk-2 a11y pattern: the active speed is aria-checked="true"; the inactive two are aria-checked="false". Single button visually pressed via cyan background. focus-visible ring chain present.
  • Play/pause uses semantic colors (amber when playing → "pause to interrupt", emerald when paused → "play to resume"). Picked for color-blind safety (the verb is in the label, not just the color) and to differentiate from the cyan speed buttons.
  • Jump-to-live uses violet accent. Distinct from cyan (speed) / emerald (play) / amber (pause) — a fourth semantic slot for "fast-forward through the run." Disabled state uses the muted-foreground/30 pattern.
  • Scene-position label "Scene N / M · t = X.XX" mounted at the right of the toolbar. Read-only, informative. Helps debug-mode users see exactly where the runner is. Could be hidden behind a flag in chunk 13; for now visible.
  • PlaybackControls mounted between PipelineProgressBar and the canvas row in CinematicViz. Pipeline progress (overview) → playback controls (action) → canvas (content). Top-to-bottom visual hierarchy matches the spec's "PipelineProgressBar at the very top, PlaybackControls just below."
  • The click-to-expand affordance lands in chunk 11b. Chunk-11 checkbox stays unchecked until 11b ships the VectorInspectionProvider + NumericalValuesPanel.

Decisions (chunk 10b): (ChatBubble + WebSocket events wiring — completing chunk 10.)

  • ChatBubble accepts tokens: string[] + isFinal: boolean — the simplest contract that matches both data sources. CinematicViz does the events-vs-state selection upstream; ChatBubble itself doesn't know which source produced the array.
  • Events-primary, state-fallback selection in CinematicViz. When the events prop contains any token.received entries, the bubble reads from those. When empty (chunk-10 isolated testing, replay without a stream), it falls back to state.pipelineState.generatedTokens.map(t => t.string). This delivers the spec's "ahead-of-viz" property automatically: live events stream from the real run independent of which scene the viz is on; chunk-10 tests + replays without a stream still show something.
  • is_final derived from the last token event's payload, or scene-based fallback. When events stream, is_final is the tokenEvents[N-1].payload.is_final flag. When no events (state fallback), the bubble enters "complete" mode when Scene 20's flourish is active (sceneId === 'detokenize' && t >= 0.9). Mirrors the in-canvas "Inference complete" badge timing.
  • Blinking cursor only when !isFinal. Subtle pulse on a 1px-wide bar at the end of the text. CSS animate-pulse (Tailwind's keyframes). Hidden once the run is final — matches the "stop talking" beat. Aria-hidden because the cursor is purely decorative.
  • Token count rendered alongside "Response" header. "12 tokens" / "12 tokens · complete" — quick orientation for the viewer about how much output has streamed in. The chunk-9 in-canvas tray uses the same count format.
  • whitespace-pre-wrap break-words on the text. Real tokens include leading spaces (' the', ' a'); preserving them gives natural word-boundary rendering. break-words keeps unusually long single tokens from horizontally overflowing the 288px bubble.
  • CinematicViz's events prop defaults to []. Existing pages (Threads/Show, Runs/Replay, Share/Replay) already pass an events array from the M6 streaming wiring; the default is for chunk-10 isolated testing only. The page mounts continue to work without changes.
  • Type narrowing via Extract<RunEvent, { event: 'token.received' }>. TypeScript's discriminated-union narrowing inside the filter predicate gives us typed access to e.payload.token + e.payload.is_final without a cast. Documented in the inline comment.
  • No useRunStream import in CinematicViz. The hook stays at the page level (Threads/Show passes its events through to CinematicViz). Keeps CinematicViz a pure render component — no live data subscription, easier to test in isolation. Chunk 11 will keep the same separation when adding playback controls.
  • Chunk 10 complete with all four surfaces wired. PipelineProgressBar (chunk 1), VocabSidebar (chunks 3c + 10a), LayerCounterHud (chunk 10a), ChatBubble (chunk 10b). The off-canvas persistent UI now reflects scene + run state continuously.

Decisions (chunk 10a): (LayerCounterHud + VocabSidebar reverse-lookup.)

  • PipelineProgressBar was already wired in chunk 1. The 21-segment control + onSelectScene click-to-jump existed since the chunk-1 stub plus the chunk-3a integration. No work needed in chunk 10. Status table line in the chunk-10 task list now reads "✅ already wired (chunk 1)".
  • LayerCounterHud derives currentLayer from viz state, not real layer.advanced telemetry. Spec line says "visible during scenes 5-12" — the HUD reflects where the viz is, not raw run progress. During scenes 5-11 we show the representative layer (= 1) since those scenes visualize a single layer's worth of computation. During Scene 12 (tower view) the HUD's value matches Scene 12's in-canvas counter exactly via the same five-phase math. Real layer.advanced events stay out of scope; chunk 12 (perf) or chunk 14 may add a "real layer" indicator if needed.
  • Phase math for Scene 12 inlined in CinematicViz, not re-imported from lib/towerCamera. Considered importing counterValue from chunk 7's lib. Rejected because the HUD already mirrors the in-canvas counter exactly; re-importing would tightly couple the persistent HUD to the scene-internal math. The 10-line inline equivalent is its own contract — chunk 7's towerCamera is testable + the HUD is testable; both can evolve independently without circular dependency.
  • currentLayer is the only Scene-12-specific calculation in CinematicViz. Other scenes contribute a constant 1 to the HUD. Trade-off: makes the HUD "wrong" if some future scene visualizes multiple layers without being Scene 12. Acceptable because Scene 12 is the only multi-layer scene in M13's spec; chunks 11+ don't add new layer-wise scenes.
  • VocabSidebar gets an explicit highlightTokenIndex override, not a state-shape change. Two highlight modes now coexist: the chunk-3c "most-recently-revealed" implicit highlight, and the chunk-10 "reverse-lookup" explicit override. The override (when set) takes precedence and uses an emerald ring vs. the forward-mode cyan ring. Different palette = different meaning ("you're going forward through tokens" vs. "you're looking back at a generated one").
  • scrollIntoView uses block: 'nearest', not default behaviour. Default scrollIntoView can move the page-level scrollroot when the sidebar already has the row visible. nearest only scrolls the sidebar's own overflow-y-auto container. Smooth scrolling because the lookup beats are spaced ~250-400ms apart; instant scrolling would feel jarring at that cadence.
  • Reverse-lookup row matched by token string, not by vocab index. The Scene 20 generated tokens have synthetic vocab indices (chunk 8b's syntheticTokenString bank); the sidebar's vocab indices come from the real js-tiktoken tokenization. They won't match. String matching ("does any sidebar row have this string?") gives a sensible "the model picked this word, here's where it lives in vocab" demo even with synthetic tokens. Real run wiring in chunk 11 or beyond will pass real vocab IDs and the match becomes index-based.
  • Out-of-range highlightTokenIndex is a no-op, not an error. Test pins this — passing 5 when only 1 row is visible silently disables the override. Lets the chunk-10 wiring be optimistic without needing per-render bounds checks.
  • Test for scrollIntoView monkey-patches HTMLElement.prototype.scrollIntoView per-test. jsdom doesn't implement the API; the spy + try/finally restoration captures the call without polluting other tests. Pattern is documented inline; reusable if chunk 11's scrubber needs similar treatment.
  • Scene 20 reverse-lookup highlights during the REVEAL phase only (t < 0.75). After t=0.75, Scene 20 enters the EOS badge + flourish phases; the lookup widget is hidden, so the sidebar highlight should clear too. CinematicViz computes this gate explicitly to match the in-canvas widget's gate from chunk 9b.

Decisions (chunk 9b): (Scenes 19 + 20 — completing chunk 9. 20 of 20 cinematic scenes are now wired.)

  • Scene 19 = standalone explanatory scene, not overlay during Scene 18. Spec describes the KV cache reveal as a 2s overlay during Scene 18's first loop iteration. Implemented sequentially (matches the chunk-9a decision for the same reason). The drawer slides in from the right, K + V matrices fill row-by-row, then a "new-row" marker pulses at the bottom to communicate the savings.
  • K + V matrices rendered as separate small heatmaps (16×8 cells each). Real K and V are (N_tokens × head_dim); we render 16 cols × 8 rows side-by-side with viridis fill. The cached-vs-new split is conveyed via the row-fill animation (cached rows fade in viridis; the new-row position gets a separate fill color pulse).
  • Lock icon = inline SVG path, not lucide-react import. The icon is one-off (only used in Scene 19), so a 7-line inline SVG saves a tree-shake assertion + decouples the scene from lucide's API surface.
  • Scene 20 phase budget: 75% reveal / 15% EOS badge / 10% flourish. The reveal phase walks through every emitted token (capped at 8) showing the per-token reverse-lookup widget. EOS badge then appears as a transition. Final 10% applies the completion flourish (background dim + chat-bubble glow + "Inference complete" text overlay).
  • Reverse-lookup widget is in-canvas, not extending the off-canvas VocabSidebar. Per the chunk-9 design decision, the off-canvas sidebar wiring is chunk 10's. The in-canvas widget shows token #N → vocab[idx] → "string" per token, replacing once per iteration. Cleaner separation; chunk 10 can later add the off-canvas highlight without touching Scene 20.
  • Completion flourish via background-color + box-shadow on the chat bubble. Considered: full-canvas overlay with backdrop-filter blur. Rejected — Tailwind's backdrop-filter isn't loaded in chunk 9b's set, and the rgba dim + emerald glow on the bubble achieves the "everything dims, success bubble glows" beat with two style props per element.
  • Chat-token pills cap at 8. Same readability rationale as earlier chunks; long generations would overflow the 600px chat-bubble width. Render-only cap; generatedTokens retains all entries.
  • EOS badge uses amber accent (amber-500/15 + amber-300), not emerald. The completion is celebrated via the chat bubble glow; the EOS badge is a status marker mid-transition. Different palette = different meaning. Test asserts the badge presence at the right t-window.
  • Active-token highlight in chat: scale 1.1, opacity stays at 1. Reveal-phase tokens before the active index stay at full opacity (hasLanded flag); tokens after the active index dim to 0.25. The active token also scales up briefly. Visual: tokens light up as the reverse-lookup widget walks through them.
  • Both Scene 19 and Scene 20 use identity transform(). They're narrative/camera scenes over already-populated state (generatedTokens from Scenes 17 + 18). Returning identity by reference matches chunk 7's Scene 12 pattern and keeps Object.is equality stable for downstream consumers.
  • Chunk 9 close-out: 20 of 20 scenes are now in ALL_SCENES. The cinematic narrative now plays end-to-end from prompt entry through completion flourish. Remaining M13 chunks 10-14 are wiring + polish (persistent UI, playback controls, perf, accessibility, closeout). The 20-scene contract from SCENE_IDS in Scene.ts:27-49 is fully satisfied.

Decisions (chunk 9a): (Scene 18 only.)

  • Linear scenes 18 → 19 → 20, not overlay scenes. Spec describes Scene 19 as "during Scene 18's first iteration" and Scene 20 as "continuous during Scene 18". The existing SceneRunner walks scenes strictly sequentially; refactoring it to support overlays would touch every shipped scene for backwards compatibility. The pedagogical beats (loop → cache → detokenize → EOS) land in the same order; only the temporal overlap is forfeited. Architectural debt: a future polish pass could add Scene.overlay: boolean to the contract.
  • Loop iteration pacing extracted into lib/syntheticAutoregression.ts. LOOP_ITERATION_DURATIONS = [2000, 1500, 1100, 800, 600, 500, 400] matches the spec's "~2s for #2 → ~200ms by #10+". Sum = 6900ms; scene duration = LOOP_TOTAL_DURATION so any future tuning of the array stays in one place. Pure-function iterationAtTime(t, iters) maps the scene's t to current iter + local t — 7 of the 16 lib tests cover this.
  • Synthetic continuation = 7 iterations, not parameter-driven. Real runs will be variable-length (WebSocket-driven); for chunk 9a the canned 7 is enough to demonstrate the loop concept + decelerating pace. Chunk 10 swaps the synthesizer for a WebSocket-driven version with the same signature.
  • Per-iteration phases mirror chunk 5/6 patterns: 0..0.3 input flash, 0.3..0.7 compute beam, 0.7..1 new-token landing. No per-iter useMemo (the iteration list is memoized once at scene-start).
  • loopIterations field in PipelineState, type aliased. The autoregression module defines LoopIteration; Scene.ts defines LoopIterationState with the identical shape. Two names so Scene.ts doesn't import from the autoregression module (the module imports syntheticTokenString; circular if both were the canonical type). Documented at the field declaration.
  • Scene 18 transform() appends 7 tokens to generatedTokens AND populates loopIterations. The cumulative tray is reconstructed at render-time from both fields. Scrubs through Scene 18 see consistent counts.
  • In-canvas chat tray ("Chat bubble · N tokens") is the same surface as Scene 17's tray. Spec calls for both a chat tray and an off-canvas chat bubble; chunk 9a builds the in-canvas surface only. Chunk 10 wires the off-canvas ChatBubble to read from the same generatedTokens field.
  • Beam-through-layers icon is a 5-floor SVG sketch, not a re-render of Scene 12. Showing the actual tower view per iteration would be a 5-second nested animation × 7 iterations. The sketch icon (5 thin floor rects + green beam line during the compute phase) conveys "compute through layers" in ~20 lines of SVG.
  • Iteration counter format: "Iteration 4 / 7 · this token: 800ms". The duration label exposes the deceleration explicitly so the viewer can see "tokens land faster as the loop continues" without needing to time anything mentally. Test asserts the bounds.
  • Scene 19/20 deferred to chunk 9b. KV cache + detokenization + EOS flourish land in a separate commit so 9a's loop animation is reviewable in isolation. Chunk-9 checkbox stays unchecked until 9b lands.

Decisions (chunk 8b): (Scenes 15 + 16 + 17 — completing chunk 8.)

  • Softmax + sampling helpers extend syntheticLogits.ts, not a new syntheticSampling.ts. All five helpers (softmax / topPCutoffIndex / sampleByMode / SamplingMode type / syntheticTokenString) operate on logits-derived data. Keeping them colocated means one import line in each scene; one file to grep. Total file is ~280 lines — under any size threshold worth splitting.
  • softmax() is numerically stable via standard max-subtraction. exp(v - max) / Σ exp(v - max). Handles [1000, 999, 998] without Infinity blow-up; test pins this.
  • sampleByMode() returns the sorted-position index, not the original vocab index. Caller maps back via the indexed top-K list. Saved a parameter (no original-indices array to thread through) and matches how Scene 16's bar chart is rendered (bars at positions 0..K-1 even when their vocab indices are scattered across 128k).
  • Sampling defaults: greedy / k=40 / p=0.95 / T=1.0. Greedy is the safest default — deterministic output, no temperature surprises. K=40 / P=0.95 match common production defaults (OpenAI Chat API, Llama config). T=1.0 = no temperature scaling.
  • samplingMode / samplingK / samplingP / samplingTemperature added to PipelineState. Chunk 10 will wire from run.parameters (JSON column on runs). For chunk 8b, the values come from CinematicViz initial-state defaults. The scene also renders the active values in the mode label — switching modes through any future control would update the visual immediately.
  • Sampling render is dynamic per samplingMode. Greedy → dart hovers over bar 0 the whole scene. Top-K → bars beyond k fade during narrow phase, dart wobbles among survivors. Top-P → fill line sweeps L→R until topPCutoffIndex(p), bars past the line fade. All three modes share the same chosen-bar pulse + winning-token flash beats.
  • Dart wobble (computeDartIndex) uses linear progression + sine jitter. Could have been a deterministic random walk; rejected as too jumpy. Sine jitter on top of a linear sweep through [0, cutoff) reads as "the dart is exploring options" while landing on the actual sampled index by t=0.7.
  • Synthetic token-string bank: 16 plausible mid-sentence English tokens. Strings include leading-space variants (' the', ' a') so concatenation in the chat-bubble looks like natural text. Real tokenizer-grounded strings replace these in chunk 10 via the WebSocket-streamed winner. Tests assert non-empty + deterministic + wrap-around behaviour.
  • Scene 17 in-canvas tray, not the off-canvas ChatBubble. Per phase1.md:1032, the off-canvas ChatBubble wiring is chunk 10's job (it needs the live WebSocket stream too). Chunk 8b's in-canvas "Generated so far" tray builds the same semantic surface from generatedTokens in PipelineState — chunk 10 just adds the off-canvas mirror.
  • Scene 17's transform() is idempotent via last-entry check. If the tail of generatedTokens already matches sampledToken, the transform returns the same state reference. Required so a scrub through Scene 17 doesn't double-append. Test pins this with a transform(transform(state)) invariant.
  • All three scenes follow the chunk-5 React-component-with-useMemo pattern. Scene render: (t, state) => <SceneComponent t={t} state={state} />; the actual useMemo calls live inside the inner React component. Hooks can't be called in the bare render callback (initial draft did this; fixed before testing). Documented for chunk-9 onwards.
  • Probability bars cap at 16 (TOP_K_RENDER). Bigger K would push individual bar widths below ~37px on the 600px canvas region — labels and pulses become hard to see. 16 is also enough to make the "long tail collapses near-zero" beat visually clear: bar 1 dwarfs bars 6+.
  • Chunk 8 = 8a + 8b, single chunk-8 checkbox flipped only now. The phase1.md status row goes from "chunks 1-7 done" to "chunks 1-8 done" with this commit. 8a's decisions block stays in place; 8b's block sits above it.

Decisions (chunk 8a): (Scenes 13 + 14 only.)

  • Split chunk 8 into 8a / 8b at the logits boundary. Five scenes (13-17) is the largest scene group in M13; splitting at the logits computation makes each commit individually reviewable in ~700 lines. 8a covers the projection-out side (final-norm + LM-head matrix); 8b covers the probability/sampling side (softmax wave, mode-aware dart, emit).
  • Scene 13 is a separate file, not a LayerNormScene factory refactor. Spec says "same squish animation as Scene 7, briefer" — could have factored LayerNormScene like chunk-6 did with createResidualScene. Rejected because the chunk-4 Scene 7 implementation is already committed and a refactor would touch shipped code unnecessarily. Cost: ~100 lines of shared rendering grammar duplicated. Worth it for chunk-4 stays frozen simplicity. If a 3rd norm scene ever lands, refactor then.
  • vocab_size type plumbed through CinematicViz, but pages still pass null. The DB schema has no vocab_size column on runs. Added the optional field to the CinematicViz.model interface so the plumbing exists when either (a) a future migration adds runs.vocab_size, or (b) chunk 10 sources it from models.vocab_size (if added there). Scene 14 falls back to 128,000 (Llama-3 / GPT-4o class) when null. Documented inline in CinematicViz.tsx. The user choice was "Plumb model.vocab_size, default 128k if null" — this delivers the plumbing spirit without an unrelated DB migration.
  • 128k label is dynamic, formatted via formatVocabCount(n). Real vocab_size for known models ranges 32k (older LLaMA) through 256k (some Gemma variants). The label reads "4096 × 50,000" or "4096 × 128,000" depending on state.vocabSize. Tests cover both branches.
  • Logits stored at full vocab length; rendered at 1024 cells via max-pool downsample. Drawing 128,000 SVG rects at 30 FPS is a non-starter (12× the render budget of every other scene combined). downsampleLogits() max-pools to 1024 cells so the spike positions stay visible. Downstream Scene 15 still operates on the full-length array for bar-chart accuracy.
  • Synthesizer: 8 hot-spike positions injected over cool xorshift noise. Mirrors the real LLM logits sparsity pattern ("most positions near zero, a handful clearly dominate"). Spike heights decay from HOT_AMPLITUDE = 4 down to ~1.3 so top-1 dominates, top-2 second, etc. — the "natural shape" the spec calls for. Test asserts >90% of cells are cool (abs(v) < 0.5).
  • Logits seed = (seedKey, sumOfLastVector). Different last vectors produce different distributions (test enforces). The seedKey is a per-scene constant (0xc0ffee) so multiple renders within the same scene match. For chunk 10's run-determinism, the seedKey will incorporate runId. Documented in the helper.
  • LM Head matrix render mirrors Scene 5's chunk-4 implementation. WeightFog patch + 14 horizontal accent lines + center label card. Same approach kept the "this is a massive matrix" beat consistent between embed lookup (Scene 5) and unembedding (Scene 14) — they're the same operation in two directions. The label flips: "Embedding table" / "vocab × hidden dim" → "LM Head" / "hidden dim × vocab".
  • Beam = single green vertical line, opacity-gated to projection phase. Considered: multiple sweep lines, particle stream. Rejected as visually noisy for a 3s scene with a lot already going on (input strips + matrix materialization + logits emergence). Single beam reads clean.
  • Phase boundaries inline (not extracted to a lib). Considered: factor scene phases into a lib/scenePhases.ts helper. Rejected as YAGNI — the phase math is 4 lines per scene and the phase boundaries differ per scene by design. The chunk-7 towerCamera extraction was justified because it had non-trivial easing curves; chunk 8a's phases are linear ramps that read better inline.
  • Cap render at 6 input strips for Scene 14, 6 row strips for Scene 13. Same readability rationale as chunks 4-7 — long prompts can't fit more vertical/horizontal density without becoming a wall of strips. The transform() outputs cover every token regardless; the cap is render-only. The "last token" highlight uses Math.min(SAMPLE_LAST_INDEX, 5) so the last visible strip is the highlighted one even when the actual last index is beyond the cap.
  • Logits-heatmap normalization is (v - min) / (max - min), not abs/symmetric. Logits range from large positive to small negative; using viridis on the [min, max] range maps the spikes to the warm end of the palette (the spec's "few hot spikes") and the cool noise to the cold end. Symmetric normalization would put min at the cold end and 0 at the middle, washing out the spike effect.

Decisions (chunk 7):

  • Camera math extracted into lib/towerCamera.ts, pure functions only. packetFloor(t, N), cameraScale(t), counterValue(t, N), blurAmount(t), towerPhase(t), easeOutQuad(x). The scene component becomes thin glue: no easing math, no phase-boundary literals, no monotonicity guarantees inline. 27 of the chunk's 43 tests live in towerCamera.test.ts — the hardest behavior (phase boundaries + ease-out monotonicity + clamping at t ∈ {0, 1}) tested without React in the loop.
  • Five-phase animation breakdown (10s total): reveal 1s / follow 1s / blur 3s / slow 3s / rezoom 2s. Picked the middle of the spec range (8-15s) so the blur phase gets its full ~3s budget per phase1.md. Boundary fractions are 0.10 / 0.20 / 0.50 / 0.80 / 1.0 — the cleaner-than-it-looks math makes every phase's progress = (t - start) / (end - start) easy to reason about and easy to test.
  • SVG tower, not Three.js / CSS-3D. Spec doesn't require 3D; the "camera zoom" is achieved via a single CSS transform: scale() on the SVG container. Matches the rest of M13's SVG-first visual language (chunks 2-6 all use SVG). Cost: no real perspective effect; viewer reads it as "the camera pulled out" because the floor count went from 1 visible to 32 visible, not because of a perspective shift. Acceptable per visualization.md's "Option A (recommended)" framing.
  • packetFloor uses easeOutQuad only for the blur phase. Other phases are linear, including the slow phase (N-2 → N), because the deceleration in the slow phase comes from the physical layer count (just 2 floors) rather than from the easing curve. Eased blur produces the spec's "Layer 03… 17… 42… 78…" tick pattern naturally — early jumps are big, late jumps are small.
  • totalLayers added to PipelineState, populated by CinematicViz from model.layers. Same pattern as chunk 6's architectureType. Defaults to null; Scene 12 falls back to 32 (Llama-7B / GPT-3.5 class). model.layers was already plumbed from runs.total_layers → page → CinematicViz props, so this is one-line wiring.
  • Persistent off-canvas LayerCounterHud stays a stub. Per phase1.md:1033, wiring the persistent HUD (visible across scenes 5-12) is chunk 10's task list. Chunk 7's scope: the in-canvas tower view + its own counter, not the persistent one. The user-visible difference: the off-canvas pill still reads "Layer — / —" outside Scene 12; the in-canvas counter inside Scene 12 reads "Layer 17 / 32" etc. Chunk 10 unifies them.
  • Packet = horizontal row of dots, one per token (capped at 6). Spec says "glowing packet representing your current sequence"; one dot per token communicates "this is the sequence ascending the tower" more clearly than a single rect. 6-cap matches the visual budget — narrower towers can't fit more dots side-by-side. Glow halo (ellipse with low opacity) sits beneath the dots.
  • Motion-blur streak = single tall transparent rect. Considered: multiple semi-transparent packet ghosts. Rejected — DOM cost scales with blur intensity, and one rect with height = 60 * blurAmount(t) reads as "the packet is moving fast" just as well. Streak appears only when blurAmount(t) > 0 (i.e., during the blur window).
  • Counter format is String(layer).padStart(2, '0') + / ${String(N).padStart(2, '0')}. Two-digit zero-padding so the rapid tick doesn't visually jump width (Layer 9 / 32Layer 10 / 32 would shift right by one digit). Spec mention of "Layer 03…" implies the zero-pad explicitly.
  • Active-floor divider gets a brighter stroke. Floor lines render at #1e293b opacity 0.5; the one within ±1 of the current packet position gets #67e8f9 opacity 0.9. Reads as "the packet is on this floor" without needing a separate highlight rect. testIds use the absolute floor number (scene-12-floor-${floorNumber}), not the array index, so tests can assert "floor 17 is highlighted at the right t."
  • Final-layer detail panel is a compact strip stack, not a Scene-8/10 miniature. Considered: re-rendering attention matrix + FFN pipe in miniature inside the rezoom panel. Rejected — that would duplicate ~300 lines of scene logic with different sizing. The detail panel instead shows a viridis output strip per token from residualOutput2 (the layer-stack output), labelled "Final layer · attention + FFN" with a footnote pointing at Scene 14. Carries the "you finished the tower" beat without re-doing the per-layer math visually.
  • transform() is identity. Scene 12 doesn't produce new vectors — it's a narrative / camera scene over residualOutput2 (which Scene 11 already populated). Returning identity by reference keeps downstream Object.is equality checks happy and matches the chunk-5 idempotency pattern.
  • Phase label in the HUD ("Phase: blur") is intentional debug-y polish. Helps reviewers + future contributors orient when scrubbing. Could be hidden behind a debug flag later; for chunk 7 it ships visible to make the phase machinery obvious. Test asserts the text content matches the active phase string from towerPhase(t).

Decisions (chunk 6):

  • Scene 9 + Scene 11 share ResidualScene via a createResidualScene(config) factory. Spec explicitly says Scene 11 is the mirror of Scene 9 ("Same as Scene 9 — pre-FFN ghost streams merge with FFN output"). One component file, one render path, two registry entries — Scene 9 wires positionEncoded + attentionOutput → residualOutput; Scene 11 wires residualOutput + ffnOutput → residualOutput2. The config also carries the caption + ghost/main labels + testId prefix so tests can assert per-scene structure. Saved ~120 lines of duplication and a future divergence-risk.
  • Residual ghost source = positionEncoded, not layerNormed. Modern transformers use pre-LN: x + Attention(LayerNorm(x)), where x is the input to layer norm — i.e., positionEncoded (Scene 6's output). Spec phrasing "kept ghosted in the background since Scene 6" matches. Trade-off: doesn't match post-LN architectures (original Transformer, BERT) exactly, but pre-LN is the modern default and matches every model we'll see at runtime.
  • architectureType added to PipelineState, populated by CinematicViz from model.architecture_type. Drives the SwiGLU/GELU pick in Scene 10. Alternative was enlarging the Scene<I, O> contract to take a separate model handle — rejected as cross-cutting and best left for chunk 10's full WebSocket wiring. PipelineState already passes through every scene; architectureType rides along. Defaults to null (idle screen) → falls back to GELU.
  • pickNonlinearity() is a string-heuristic, not a vendor-aware lookup. Llama / Mistral / Qwen / Gemma / Phi / any-with-moe → SwiGLU. Everything else → GELU. Hand-rolled heuristic matches what the M3 OpenRouter registry actually returns for architecture_type strings without round-tripping through a vendor-specific table. If a new SwiGLU family ships (e.g., Apple's Foundation), we add a substring; if the existing heuristic wrongly classifies, the visual cost is the same wavy color shift with a different label — pedagogically resilient.
  • FFN expansion synthesizes 4× cells in the middle phase, contracts on exit. Spec: "the expanded middle section can briefly show many more cells, emphasizing where most parameters live." expandToFFNDim(v, factor=4, tokenIndex) interpolates adjacent cells with xorshift32-seeded ±5% jitter so the expanded view looks like a richer version of the input, not a 4-pixel duplicate. contractFromFFNDim() averages groups back down — inverse operation (jitter averages out, interpolation cancels).
  • The "non-linearity wave" is a single hot column sweeping L→R. Implementation choice: one amber <rect> of width ~2 cell-widths sweeps across [0.5, 0.85] of t. Considered: per-cell color phase shift propagating through. Rejected as visually noisy + ambiguous about what "pass through a non-linearity" means at a glance. The single sweeping column reads cleanly as "this stage is happening now."
  • Sparkles are 10 deterministic-seeded dots, opacity-gated to expansion peak. sparklePositions(tokenIndex, count=10) returns (x, y) ∈ [0,1]² per token; sparkleEnvelope(expansionAmount) returns 0 until expansionAmount > 0.6. Sparkles appear only during peak expansion, fade in/out cleanly, and reposition per token. Replay-deterministic per token-index seed.
  • FFN renders only the first 4 tokens (cap). Per-token pipe is 200-400px wide depending on expansion phase; 4 stacked rows fit the viz region comfortably with the caption + non-linearity label + output label. Tokens 5+ aren't part of the PipelineState rendering for Scene 10; their ffnOutput is still computed in transform() so chunk-7 onward sees the full sequence. The cap is render-only.
  • Residual scene renders the first 6 tokens (cap). Two-column layout (ghost left, main right) with a center + is more horizontally efficient than the FFN pipe rows, so 6 fits where FFN takes 4. Same render-only cap pattern.
  • Pipe outline + sparkle SVG share one <svg> per token. Considered: separate SVGs for pipe outline / strip cells / wave column / sparkles. Rejected as 4× the DOM nodes per token = 16× for the scene at peak (4 tokens × 4 elements). Single <svg> per row → one DOM node per token. The pipe-outline border is on the parent <div> via border-cyan-500/30 (CSS), not in SVG.
  • Phase-window opacities use the chunk-5 leading/middle/trailing pattern. Scene 9 / 11 use a single in-phase + out-phase render (the outputOpacity is the only crossfade); Scene 10 uses an envelope function expansionEnvelope(t) that holds at 1 between t=0.4 and t=0.6 (the peak hold). Both patterns avoid the t = 1 fade-to-zero bug documented in chunk 5.
  • applyFFN() returns same-length output, but internal compute touches 4× cells. Caller perspective: vector in, vector out, same dim. Internal: expand → activate → contract. Pedagogically the expansion is the point; the result-dim invariant lets downstream scenes (chunks 7+) consume ffnOutput without re-shape gymnastics. The 4× world only exists during the scene's render window.

Decisions (chunk 5):

  • Reuse M8 lib/attentionPattern.ts as the matrix generator. Chunk 1 kept the file specifically for this chunk. Same causal mask + distance-decay + xorshift32 noise + row-sum-to-1 invariant; chunk 5 wraps it in generateMultiHeadMatrices() which folds head index into the seed (seedLayer = (layerIndex × 1000 + h × 7) mod totalLayers) so different heads render as visibly different patterns. No re-implementation of M8 logic.
  • Defer run.id from the seed, plumb in chunk 10. Spec asks for (run.id, layer_index, token_index) determinism. For chunk 5 the seed is (tokenIndex, headIndex, layerIndex=0) only — runId plumbing through CinematicVizPipelineState is cross-cutting (touches Threads/Show, Runs/Replay, Share/Replay) and rightly belongs to chunk 10 where the WebSocket event stream wires up. Scene 8 is still frame-deterministic for a given prompt; replay determinism across runs of the same prompt holds trivially.
  • Representative 4–6 fanned heads, not literal 32. Spec says "stack of ~32 transparent layers; fan them out briefly to show parallelism, collapse back." Literal 32 N×N matrices is ~7,200 SVG rects at the peak of the fan (for N=15) — performs OK but visually overwhelming. Chunk-12's degraded-mode plan already calls for "single representative head shown" under FPS pressure. Default is 6; the real model.attention_heads count goes in the caption ("Attention · showing 6 of 32 heads"). Perceptual clarity over literal count.
  • Q / K / V are sign-flip projections of the same input embedding. Real transformer Q/K/V come from W_Q/W_K/W_V learned linear projections. The chunk-5 stand-in: xorshift32-seeded sign-flip mask per cell, keyed by (tokenIndex, headIndex, role_offset). Output magnitudes equal input magnitudes; only signs flip. Trade-off: doesn't model the projection dimension reduction (real K/V live in head_dim = hidden_dim / heads); the visualization shows Q/K/V at the same length as the input strip. Pedagogical clarity over mathematical fidelity, per the M13 architectural note.
  • splitQKV keys roles by distinct seed offsets (0x11111111 / 0x22222222 / 0x33333333). Adjacent role bits ensure Q/K/V mask different cells from each other for any non-trivial input. Test explicitly asserts "Q !== K !== V" for at least one cell across the triple.
  • Illustrative softmax beat, not faithful pre→post. The matrix from attentionPattern.ts is already row-normalized. Scene 8b's "softmax wave" plays as an amber horizontal line that sweeps top-down across tokenList.length rows during phase 8b — visually communicates "each row gets balanced" without requiring a separate pre-softmax helper. Matches the M13 "illustrative not measurement" principle.
  • Three sub-beat cross-fades via leading/middle/trailing opacity helpers, not a single phaseOpacity(). First iteration used one helper that fades in/out at both boundaries; broke at t = 1 because the trailing phase faded out into the void. Split into leadingPhaseOpacity (no fade-in), middlePhaseOpacity (both), trailingPhaseOpacity (no fade-out). Clear intent at the call site + correct behavior at the t-boundaries.
  • 8b multi-head fan: bell-curve open/close on a single normalized 8b clock. bell(tIn8b, center=0.5, width=0.25) peaks halfway through 8b and falls off symmetrically. No explicit "open then close" sequencing — the bell curve handles both. Falls off cleanly back to a single centered matrix at the 8b→8c boundary.
  • Softmax wave only animates on head 0. During the fan-out moment, all 6 heads are visible but the amber sweeping line is only drawn on the representative head (head 0). Drawing the wave on all 6 would compete with the multi-head reveal beat — separate visual ideas.
  • Scene 8 output strips reuse viridis via normalize() from M12 chunk 4. Post-blend V values aren't naturally in [0, 1] for color mapping; normalize stretches the row's min-max into the palette domain. Same helper M9's AttentionHeatmap already uses, so cross-scene color language stays coherent.
  • Q/K/V row cap at 8 tokens, output blend cap at 8 rows. Same readability rationale as chunk 4 Scene 7. Three roles × 8 tokens = 24 strips already pushes the viz region; more would be unreadable. The cap is on render only — the underlying qkv array in PipelineState contains entries for every token so chunk-6/9 scenes downstream see the full set.
  • transform() falls back through layerNormed → positionEncoded → embeddings → tokens to source the input. The chunk-3a SceneRunner doesn't yet auto-call transform() between scenes (chunk-10 wiring). Each scene must therefore be robust to the upstream fields being absent — Scene 8 derives whatever it can from the most-derived field available, all the way back to synthesizing from tokens.

Decisions (chunk 4):

  • syntheticEmbedding(tokenId, dim) is seeded by token ID alone, not (run.id, token_index). The M13 architectural note specifies seeds from (run.id, model_metadata, token_index, layer_index), but for Scenes 5-7 the "same token → same embedding row" invariant is the load-bearing pedagogical beat: a viewer should be able to recognize a recurring word by its color signature in the heatmap strip. Keying off run.id would break that recognition across replays of the same prompt. The fuller seed set is reserved for Scene 8+ where per-position activations diverge from per-token embeddings.
  • xorshift32 PRNG, same family as the chunk-2 tokenIdToHue. Same generator + same seed-mix constant (0x9e3779b9 golden ratio) so a debugger comparing pill color → embedding strip can reason about both from one mental model. Zero-state guard (if (s === 0) s = 1) covers tokenId = 0 — xorshift32 collapses to all-zeros without it.
  • VISIBLE_EMBEDDING_DIM = 128, label says 4096. Real model.hidden_dim is 4096-16384 depending on the model; rendering that many SVG <rect>s per token at 30 FPS would tank performance. VectorStrip's chunk-2 totalLength={4096} prop carries the implied extent via the "showing 128 of 4096 cells" aria-label, so screen-reader users + the matrix caption both communicate the real scale.
  • applyPositionRotation is illustrative RoPE, not faithful RoPE. Real RoPE uses dim-pair-specific frequencies (θ_i = 10000^(-2i/dim)); we use a single global (positionIndex × π / 16) angle so the visible color shift is monotone in position. The pedagogical beat is "position changes the vector"; mathematical fidelity is reserved for a future explanatory mode that doesn't exist yet.
  • layerNormalize adds ε = 1e-6 inside the sqrt. The constant-input edge case ([5,5,5,5]) has std=0; without ε it would divide by zero. Test pins the behavior. Matches PyTorch nn.LayerNorm default eps=1e-5 to within a small constant factor — both are within float32 precision for our value range.
  • Scene 5 "matrix" is 14 horizontal accent lines + WeightFog, not a rendered grid. The label "128,000 × 4096" sells the scale; drawing actual cells would be visually noisy and burn the GPU. The accent lines fire row-by-row as beamPhase advances so the "rows lighting up" beat reads even without a real matrix behind them.
  • Scene 5 staggered detach: leftmost token flips first. Per-token localPhase = detachPhase * 1.5 - i / N — the cascade is visible and matches the left-to-right reading order. Without staggering, all tokens would flip at the same instant and the transition would feel like a layout shift rather than a beat.
  • Scene 7 dual-render with absolute overlay, opacity crossfade. Both <BarChart> and <HeatStrip> mount in the same width × height absolute container; opacity props discriminate. Avoids layout shift mid-squish (a single component that swapped DOM shape would have caused a one-frame reflow on the parent flex row). Tests assert both can be null at t = 0 and the heat-strip is visible at t = 1.
  • Scene 7 caps render at 8 rows. Long prompts (50+ tokens after the chunk-3 tokenizer) would push the strip stack past the viz region's vertical bounds. Capping keeps the stack readable + the "squish" effect legible. Scenes 5 + 6 use horizontal flex-wrap so they self-manage; Scene 7's vertical stack needs the cap.
  • Each scene's transform() is idempotent + returns the same reference if already populated. Object.is equality for the no-op path lets React downstream-effect tests rely on referential stability. Asserted for all three scenes — transform(transform(state)) === transform(state).
  • Scenes have render() fallbacks that synthesize embeddings on the fly when state hasn't been transformed yet. The chunk-3a SceneRunner skeleton doesn't yet call transform() between scenes (that wiring lands in chunk 10). Until then, each scene synthesizes from state.tokens directly so isolation-tests with just a tokens array render correctly.

Decisions (chunk 3): (backfilled in chunk 4 — chunks 3a/3b/3c committed without an inline decisions block.)

  • js-tiktoken over @dqbd/tiktoken. The spec hedged: WebAssembly speed vs pure-JS portability. Pure JS won — ~200KB bundle (vs ~600KB for the Wasm variant), zero runtime requirements beyond ES2020, identical behavior across Chrome/Firefox/Edge/Safari without Wasm-loading edge cases. The steady-state per-call latency is slower than the Wasm path but Scene 3's 4500ms budget swallows it: tokenization runs once on prompt arrival and the result is cached for the whole pipeline.
  • cl100k_base as the universal vocab. GPT-3.5/4 tokenization is the most-recognizable modern frontier-model encoding; the M13 viz is pedagogical, not measurement, so a single representative vocab beats per-vendor splits (Llama, Mistral, Claude would each pull their own bundle). Vendor-specific tokenization can land as a chunk-14 polish if anyone asks.
  • Codepoint-per-token fallback when js-tiktoken fails to load. Network-flaky environments still get a degraded-but-functional viz: every codepoint becomes one pseudo-token. Scene 3 still animates; the BPE-merging beat just renders as "every char is its own token" which is the trivial-correct case. Better than a hard crash.
  • PipelineState is a wide-record (all-optional fields), not a typed I/O chain. Considered: Scene<I, O> generics enforcing that each scene's O is the next's I. Rejected for verbosity — every scene-pair would need explicit type plumbing, and the wide record matches how the viz actually thinks about state: at any point the full accumulated derivation is available for the current scene to reference. Inline comment in Scene.ts locks in the choice.
  • useSceneRunner pre-walks the registry once + caches pipelineInputs[i]. O(1) jump-to-scene for chunk 11's scrubber. Trade-off: every scene's transform() runs on registry change even if the user never visits that scene; acceptable because transforms are pure and cheap. Re-walks on initialState change (new prompt) or scenes change (developer flow).
  • useSceneRunner resets to scene 0 on initialState or scenes change. Both prompt-change (user submits a new prompt) and registry edit reset playback to the top. Could have preserved playback position; explicit reset matches the cinematic-narrative metaphor (a new prompt is a new movie) and avoids the "scene index out of bounds" footgun when the registry shrinks.
  • CinematicViz idle screen when prompt === null. Distinguishes "no run yet" from "run in progress" without a third state. Pages plumbing (Threads/Show, Runs/Replay, Share/Replay) explicitly pass the prompt; the viz sits at idle until one arrives instead of mounting Scene 0 with empty content.
  • Tokenizer warmed on CinematicViz mount via useEffect + loadTokenizer(). Scenes 0/1/2 = 1.8 + 2.4 + 2.5 = 6.7s combined; by the time Scene 3 plays, the tokenizer is cached. getCachedTokenizer() returns null on cold cache so Scene 3 renders a "Tokenizing…" placeholder + falls through to identity-transform until warm. Low-end devices still get a coherent viz.
  • Scene 1 multi-byte "N-byte" tag for emoji / CJK / accented chars. UTF-8 visibly expanding non-ASCII chars to 2-4 bytes is the most teach-able moment for "tokens emerge from bytes, not characters." The tag fades in after the char's flip animation so the bytes-count is the punchline, not the lead.
  • Scene 2 chatTemplateTints array carries forward to Scene 3. Per-byte tint group (system / user / assistant / user-prompt) decided in Scene 2 and passed through PipelineState. Scene 3's BPE colors each emitted token by the modal tint of the bytes it absorbed, so the system prompt stays purple all the way through to the final token row.
  • Scene 3 per-token duration scales 200ms (common, ≤ 2 chars) → 600ms (rare, ≥ 6 chars), linear between. Mirrors how a reader's eye dwells longer on unfamiliar words. Common tokens fly by; long/rare words get visual emphasis. Total Scene 3 budget (4500ms) accommodates ~10-20 tokens at average pace — adequate for typical short-prompt teaching examples.
  • VocabSidebar revealedCount derived from (sceneIndex, t), not stored in state. During Scene 3, ramps 0 → tokens.length. Outside Scene 3, fixed at 0 (before) or tokens.length (after). Deriving from the scene clock means scrubbing/replay shows the exact same sidebar progression — pure-function of the runner's clock, no state to manage.
  • VocabSidebar substitutes · for space and for newline. Whitespace tokens otherwise render as empty strings in the sidebar list — invisible. Glyph substitution affects display only; the underlying token.string is unchanged. Chunk-11's scrollIntoView() will use the substituted glyph for the highlight target.

Decisions (chunk 2):

  • All 5 primitives are SVG/HTML/CSS — no Three.js for chunk 2. User choice. The chunk-2 spec hedged on <ParticleTrail> being Three.js but in pre-chunk discussion the SVG-with-CSS-animation path won: standalone in the DOM, no scene context required, composes anywhere two anchor pixels are known, scales to 50+ simultaneous trails at 60 FPS. A future <GpuParticleSwarm> Three.js component can land when a scene needs many thousands of particles, but every named scene in docs/visualization.md is covered by SVG.
  • viridisAt(t) is a hand-rolled linear interpolator over VIRIDIS_STOPS, not a d3-scale call. Cinematic-viz primitives render hundreds of cells per scene at 30 FPS; calling into a d3 scale per cell wastes ~30 µs each. The inline 20-line interpolator reuses the existing M12 chunk-4 5-stop viridis palette so the visual language stays coherent with the M9 AttentionHeatmap. normalize() is colocated in the same lib/vizColors.ts so both primitives share the saturation-stretching logic.
  • tokenIdToHue is xorshift32-based, not a CSS hsl(var(--token-${id}-hue)) lookup. A pure function: deterministic per token ID, no CSS-variable plumbing through every consumer, no global stylesheet. Adjacent IDs land on visually distinct hues (the assertion in TokenPill.test.tsx checks ID 100 and 101 differ by >10° on the hue wheel). Trade-off: not strictly CB-safe (two different tokens can still hash to similar hues), but the label is always the source of truth — WCAG 2.1 SC 1.4.1 satisfied because every TokenPill displays its string inline.
  • Heatmap aria-labels spell out "showing N of M". When VectorStrip truncates a 4096-dim vector to 128 visible cells, the aria-label reads "Vector heatmap, showing 128 of 4096 cells, viridis palette" — same for MatrixGrid. Screen-reader users get the implied extent without needing the visual cue.
  • WeightFog is role="presentation" aria-hidden="true". It's background fabric — no semantic content. AT should skip it entirely. The pattern id is derived from width × height × density so multiple fog mounts on the same page don't collide on the <defs> id.
  • ParticleTrail inlines its @keyframes per-mount. The keyframe name embeds the from/to pixels so different trails don't share an animation declaration. Pulse uses motion-safe: so prefers-reduced-motion freezes it — carries the M12 chunk-3 pattern forward. Guide line is strokeDasharray="2 3" at 35% opacity so the data path is visible even under reduced motion.
  • data-testid on every cell. vector-strip-cell-N and matrix-grid-cell-R-C make the heatmap content directly assertable. Lets future scene tests verify "the Q vector for token 3 ended up bright at index 7" without reading the rendered SVG. The 46 chunk-2 tests use these heavily.
  • Empty inputs render valid (empty) SVGs. VectorStrip with values={[]} renders the bordered SVG container with zero cells. MatrixGrid with values={[]} does the same. Avoids NaN-cellWidth artifacts + lets the parent layout reserve space for the strip even when the data hasn't arrived yet.
  • vizColors.ts lives at lib/, not Components/Viz/. It's a pure helper, not a component. Other libs (existing M9 attentionPattern.ts, M8 streamMetrics.ts) follow the same pattern. Keeps Components/Viz/ as React components only.
  • Test run flakiness observed under heavy system load. The full Vitest suite under load avg 77 returned 6-8 spurious failures across runs, with per-test durations stretched 20× normal. Running the same files individually or with light system load shows 100% pass. Documented here so a future contributor doesn't waste time chasing a "regression" that's actually CPU starvation; the canonical fix is to retry after the host quiets down. All 46 new chunk-2 tests pass deterministically when the suite isn't fighting for CPU.

Decisions (chunk 1):

  • Build placeholders first, swap imports second, delete last. Pre-discussion plan held: created Scene.ts + useSceneRunner.ts + 4 persistent UI stubs + CinematicViz.tsx first, then swapped the three page mounts to use the new component, then deleted the M8 viz files. Build stayed green throughout instead of going red mid-chunk.
  • File location: resources/js/Components/Viz/ — user choice. Same path the deleted M8 files lived in; namespace was empty post-deletion so no collisions. Future imports stay @/Components/Viz/CinematicViz etc.
  • SceneRunner is a custom hook (useSceneRunner) — user choice. Matches the existing M8 useEventPlayback / useRunStream shape. Returns { state, controls }. RAF loop owns advancement via refs (so the per-frame work doesn't trigger React renders); state updates fire only on scene index / t / playing / speed changes. Tests can call the hook in isolation via renderHook.
  • Layout: viz takes 2/3 (lg:col-span-2), transcript+prompt take 1/3 (lg:col-span-1) — user choice. Flip of the M8-era split. Cinematic viz becomes the primary surface; transcript becomes a supporting column. Mobile collapses to single-column as before.
  • Deleted LiveStreamPane + ReplayRightPane + ReplayDebugPane outright. The M8 right-pane tab toggle (view-viz / view-embeddings / view-debug) is gone. The M11/M12 spec talked about keeping a "Debug tab last-resort fallback" for chunk 13's tri-state, but the cleanest path was to delete the M8 implementation now and let chunk 13 build a fresh fallback inside CinematicViz when WebGL 2 isn't available — preserving M8 code as "future dead-code reference" was net-negative.
  • SCENE_IDS as a const array tuple, not an enum. Gives us both the stable string identifier and ordinal-by-index for free, with TypeScript narrowing via (typeof SCENE_IDS)[number]. Enums would have boxed the integers into nominal types and made the pipeline progress bar's aria-label="Scene ${i}" awkward. The labeled order matters: it's the order the SceneRunner walks.
  • PipelineProgressBar fully implemented in chunk 1, not stubbed. The 21-segment control surface is static (always Scene 0 through Scene 20, labels locked in Scene.ts). Chunks 3-9 just change which segment is aria-current as scenes advance. Clicking any segment fires SceneRunner.setScene. Cheap to build now; saves refactoring later.
  • Gate notice copy is placeholder. Chunk 13's tri-state (full / 2D-svg / debug-text) hasn't landed yet, so the WebGL-2-unavailable + reduced-motion notices currently say "...once Chunk 13 lands." Chunk 13 will rewrite these strings + actually flip between rendering modes.
  • tests/Pest.php got a global RefreshDatabase opt-in. Tests like AboutPageTest and WelcomePageTest didn't call uses(RefreshDatabase::class) and were happily reading the dev sqlite db before the M12-followup :memory: switch. After the switch, they hit "no such table: registry_meta" because migrations didn't run. One-line fix: pest()->use(RefreshDatabase::class)->in('Feature') so the trait applies globally to every Feature test. Cheap in :memory: (~50ms per test for migrations); guarantees a clean per-test DB regardless of whether the test file remembered to opt in.
  • phpunit.xml got an APP_URL=http://localhost override. Pre-existing latent bug: tests inheriting APP_URL from .env failed when the dev server ran on a non-standard port (http://localhost:8001 in this session). The ExportTriggerControllerTest cache-hit assertion was the canary; pinning APP_URL deterministically in phpunit.xml makes URL-asserting tests env-independent.

Parked into M13 from earlier milestones

  • M8 chunk-9 vertical-slice "30 FPS sustained" gate stays — M13 just raises the bar to "20-30 FPS sustained with automatic degrade fallback."
  • M9 strict-determinism (replay = original) extends to M13: every scene's animation must be a pure function of (run.id, scene_index, token_index, layer_index) so a replay is frame-identical.
  • M10 GIF export needs to capture the new viz. The chunk-2 SVG renderer composes scene-by-scene; chunk 6's Puppeteer skeleton would also need to record the new viz once enabled. Park: M13 closeout doesn't refresh the SVG renderer's frame composition; that work lands in a follow-up M13 polish chunk or M14/M15 if it's required for launch.

M13 manual smoke recipe (chunk 14)

Per the chunk-14 spec line — matches the M8 chunk-9 / M9 / M11 / M12 patterns. Run on a 2020-class machine (the SPEC reference) and a fresh login.

1. Visual review per-scene (~5 min). 1.1 Sign in. Submit a fresh prompt (e.g., "Explain the transformer architecture in three sentences."). 1.2 Watch the canvas play through Scene 0 → Scene 20 at 1× speed. Each scene's caption + content should appear at the top of its window; no flicker, no missing labels. 1.3 Click PipelineProgressBar segments out of order — every scene jumps cleanly (no orphan render, no stale data from the previous scene). 1.4 Scrub backward (Step ◂◂) through each scene boundary — earlier scenes still render correctly when revisited.

2. FPS spot-check + degraded-mode visual (~3 min). 2.1 Open the page with ?debug=fps in the URL. The FpsCounter overlay appears top-right showing rolling FPS. 2.2 Open ~10 other browser tabs running anything CPU-heavy (other dev servers, YouTube, etc.) to push FPS below 18 sustained. 2.3 After 2 seconds of sustained low FPS, the counter turns amber, suffix " (degraded)" appears, and you should observe: multi-head attention fans only 1 head (Scene 8b caption reads "showing 1 of 32 heads · degraded"), particle trails stop animating, VectorStrips shorten to 64 cells where they used to show 128. 2.4 Close the heavy tabs. After ~5 seconds of FPS > 24, the counter returns to emerald, the suffix disappears, full-quality rendering resumes.

3. Reduced-motion visual (~2 min). 3.1 OS-level toggle: enable "Reduce motion" in your OS accessibility settings (System Settings → Accessibility → Display → Reduce motion on macOS; Settings → Ease of Access → Display → Show animations off on Windows; gsettings on GNOME). 3.2 Reload the viz page. An amber banner reads "Reduced-motion is set. Scenes display as static completion frames; advance via Step or wait 4 seconds." 3.3 The current scene shows its t = 1 (completion frame) — no animation. After 4 seconds it auto-advances to the next scene. 3.4 Press Step (▸▸) — the timer cancels + the next scene appears immediately. Auto-advance resumes from the new scene. 3.5 Disable reduce-motion. Reload. The banner disappears + normal animated playback resumes.

4. WebGL-2 fallback visual (~1 min). 4.1 In Chrome: open chrome://flags, set #disable-webgl2 to Disabled, restart. (Or use a browser without WebGL 2 — older Safari, headless without GPU.) 4.2 Reload the viz page. Banner reads "3D camera moves are unavailable; the visualization is rendering in 2D mode." 4.3 Confirm the viz still renders normally — every scene plays through (the M13 viz is SVG-only by design, so the WebGL gate is informational). 4.4 Re-enable WebGL 2 + reload. Banner disappears.

5. Share-link replay verification (~3 min). 5.1 Submit a prompt + let the run complete. 5.2 In the thread header, toggle Share on. Copy the /share/{token}/runs/{run}/replay URL. 5.3 Open the URL in a private/incognito window (verifies no-auth access). 5.4 Confirm: viz mounts, PlaybackControls visible, chat bubble shows the completed text, scene runner plays through 0→20 the same way. 5.5 Test click-to-inspect: click any VectorStrip in the public replay → NumericalValuesPanel opens with the same data the owner saw. 5.6 Test PipelineProgressBar click + Step + Speed change — all work in the public view.

6. Cross-browser smoke (~10 min, defer to M15). The matrix below extends the M12 chunk-9 cross-browser table with M13-specific rows. M15 launch prep owns the first full run; this recipe is the M13-side of the contract.

Browser M13-specific check Expected
Chrome latest Scenes 0-20 each render Every scene caption + content visible
Chrome latest PipelineProgressBar click-to-jump Each segment jumps to its scene cleanly
Chrome latest FPS in band on a fresh tab ≥ 30 FPS sustained at 1× speed
Firefox latest Same three checks Same
Edge latest Same three checks Same
Safari latest Same three checks Same
Chrome -1 Same three checks Same
Firefox -1 Same three checks Same
Edge -1 Same three checks Same
Safari -1 Same three checks Same

(Empty until M15 populates from a real cross-browser run. Per the M12 chunk-9 pattern, non-blocking issues are documented as "won't fix" with a rationale.)

M13 retrospective

What worked

  • Rip-and-replace from chunk 1 was the right call. Deleting the M8 viz files up front (VizPane.tsx, EmbeddingScene.tsx, CascadeController.ts, ParticleSystem.ts, TransformerStack.ts, etc.) kept chunks 2-14 from coding around dead weight. The git history preserves M8 for spelunking; phase1.md keeps the documentation. Zero feature-flag plumbing, zero parallel mounts, zero "what does this old code do?" tax across 14 chunks.
  • SVG-only primitive set (chunk 2's locked-in decision). "All 5 primitives are SVG/HTML/CSS — no Three.js" turned out to be the single largest scope-reducing call of M13. The chunk-13 WebGL gate became informational only (no actual rendering pipeline to switch), the chunk-12 degraded-mode opt-ins were cheap (clamp visibleCells, strip a CSS animation, slice the head matrix), and no scene needed a separate Three.js fallback path. The chunk-2 pre-discussion's hedge on <ParticleTrail> being Three.js + the SVG-with-CSS-animation winning meant every named scene in docs/visualization.md was covered by SVG out of the gate.
  • The PipelineState wide-record + per-scene transform() contract (chunk 3a) scaled to 21 scenes cleanly. Each scene reads what it needs, adds its derived field, returns. Idempotent transforms made scrubbing/replay trivial. The chunk-3a alternative — a typed Scene<I, O> chain with verbose generics — would have required 21 type mappings + would have made cross-scene fallbacks (the "if X missing, use Y" fallback chains in chunks 5-9) impossible without a per-scene migration story.
  • Pure-function libs paid off in test density + reuse. lib/towerCamera.ts (chunk 7), lib/syntheticEmbedding.ts (chunk 4), lib/syntheticAttention.ts (chunk 5), lib/syntheticLogits.ts (chunk 8), lib/syntheticAutoregression.ts (chunk 9), lib/syntheticFFN.ts (chunk 6), hooks/useFpsTracker.ts (chunk 12) — each carries the math/state-machine logic separately from the React render layer. Tests for those libs run without jsdom + mounting. ~150 of M13's ~930 Vitest assertions are pure-function lib tests.
  • React context for cross-cutting concerns (chunks 11b + 12). VectorInspectionContext made every existing VectorStrip click-to-inspectable without touching any scene. PerformanceModeContext made every VectorStrip + ParticleTrail + AttentionScene FPS-aware without touching any scene. Both contexts return safe defaults outside their providers, so per-component tests don't need the wrapper. The pattern beats threading props through 21 scenes by a wide margin.
  • Chunk-split decisions stayed flexible. Chunks 3, 8, 9, 10, 11 split into a/b sub-chunks when the work was natural to bisect at a state boundary (e.g., 8a at the logits computation, 9a at the loop/explainer line, 11a at the inspection-UI boundary). Chunks 4, 5, 6, 7, 12, 13 stayed single-commit when the work was one coherent feature. The split came up at the design-question step of each chunk; the user-direction-driven default usually held.
  • The chunk-1 placeholder PipelineProgressBar paid back through every subsequent chunk. Building the 21-segment control surface in chunk 1 (when nothing about scene content was decided) meant chunks 3-9 just toggled the aria-current segment as the runner advanced. Click-to-jump-to-scene was already wired in chunk 3a's controls.setScene. The alternative — stubbing the progress bar and building it after content settled — would have required a refactor at chunk 10 or 11 once playback controls landed.
  • The decisions block per chunk became a real reference. ~150 documented decisions across M13. When chunk 14's integration test needed to know "how does the chat bubble decide which scene to render?" the chunk-10b decisions block answered it without re-reading the source. Same for the chunk-9a "linear scenes, not overlays" choice (referenced in chunk-13 when planning the gate behavior) and the chunk-5 "defer runId" choice (referenced in chunk-10b's events-vs-state selector).

What we learned

  • vi.doMock + dynamic import requires vi.resetModules() in beforeEach. Hit during chunk-13 gate tests: the first test in each vi.doMock-guarded suite inherited the already-loaded CinematicViz module from before the mock applied, returning the default hook value. vi.resetModules() before the vi.doMock invalidates the cache so the next await import('@/...') is fresh. Documented inline; the pattern works for any per-suite hook override.
  • useCallback-stable context values prevent infinite-render footguns. Caught in chunk 11b: useEffect([inspection], () => inspection.open(...)) re-fired each time setActive recreated the context value (which happened on every open() call). Fix is useCallback on open + close with empty deps so the consumer can depend on them directly. Same pattern applied in chunk-12's PerformanceModeContext.
  • ESLint's react-hooks/set-state-in-effect rule fires on legitimate URL-param reads. Chunk-12's first draft of FpsCounter used useState + useEffect to read window.location.search. The rule (correctly) said no. Fix is useSyncExternalStore with a subscribe/getSnapshot pair — the idiomatic React 18+ way to bridge an external source. Reusable for any future query-param consumer.
  • Hooks-rule violations slip past tests but fail the prettier+eslint pre-commit. Initial drafts of SoftmaxScene (chunk 8b) called useMemo inside the Scene.render callback, which is a hooks-rule violation. Tests passed locally but the pre-commit hook caught it. Pattern from chunk 5 onwards: scene render: (t, state) => <SceneComponent t={t} state={state} /> with useMemo inside the inner React function component. Documented in chunk-8b decisions.
  • Per-scene component-test files vs unified scene-determinism.test.ts. Chunks 3-9 each shipped their scenes' own test files (scenes-3-4.test.tsx, scenes-5-7.test.tsx, etc.) covering scene-specific UI invariants. The chunk-14 scene-determinism.test.ts is the unified walk asserting determinism + output-shape for ALL 21 scenes. Both layers carry weight: per-scene files catch behavior regressions, the unified file catches cross-scene contract drift.
  • Stale vitest worker accumulation during debugging. When debugging chunk-11b's infinite-render loop, ~5 stalled vitest processes piled up (spawn-per-invocation with workers). Each one consumed 100% CPU pinning the next foreground vitest. Workflow lesson: pkill -9 -f vitest between investigations when test runs hang past their normal duration.
  • Inline duplication beats coupling for crossing-concern math. Chunk-7's tower camera math is in lib/towerCamera.ts; chunk-10a's persistent HUD currentLayer derivation inlines the same 5-phase math in CinematicViz rather than importing. Reason: the persistent HUD shouldn't depend on a scene-internal lib for behavior parity. The 10-line duplication is its own contract; both copies have their own tests; either can evolve independently. Trade-off documented in chunk-10a decisions.
  • Spec gaps surface during integration, not design. Chunk 10's spec line said "Driven directly by the real token.received WebSocket event" — the design question of how to handle the no-events case (replays, isolated tests) only surfaced at implementation. The events-primary / state-fallback selector emerged then; documented as a chunk-10b decision. Pattern: lean toward fallback-to-something-sensible at every spec-vs-reality gap rather than blocking the chunk on a spec amendment.
  • Linter reformats break in-flight Edit handles. Same lesson from M10/M11/M12. Hit in M13 chunks 4, 8b, 9b, 11b, 13 (every commit that goes through prettier/eslint pre-commit). Pattern unchanged: re-Read after the hook runs, then re-Edit. Worth automating the read-after-reformat into the workflow.

Parked / carry-forward to M14/M15

  • Real WebSocket-driven scene pacing. Chunk 9a's synthesizeAutoregressiveLoop is a canned 7-iteration timeline. Real runs vary in token count + per-token time. Chunk 10b wires the chat bubble to live events; chunk 11a's jump-to-live derives liveSceneIndex from events. But the scene timing inside Scene 18 still uses the synthetic durations. M14/M15 polish chunk should drive Scene 18's iteration timing from the actual token.received t_ms field.
  • runId in the deterministic-synth seed. Chunks 4, 5, 6, 7 use (tokenIndex, layerIndex=0) for synthesis; the spec calls for (run.id, layer_index, token_index). Plumbing runId from CinematicViz → PipelineState → every synth helper is cross-cutting; deferred to a chunk-10-style wiring pass. Replay determinism across the same prompt holds today; replay determinism across different runs of the same prompt doesn't yet.
  • vocab_size DB column. Chunk 8a plumbed the type but pages pass null because runs doesn't have a vocab_size column. Scene 14 falls back to 128k. A future migration adds the column + the page-source switches over without code changes to CinematicViz.
  • Off-canvas reverse-lookup highlight uses string-match, not vocab-index-match. Chunk-10a's Scene 20 reverse-lookup highlights sidebar rows whose token string matches the generated string. Real runs would use vocab-index matching once runId plumbing lands. The string match is fine for the chunk-9b synthetic tokens but loses precision when two different vocab entries share a string.
  • Persistent LayerCounterHud vs Scene 12's in-canvas counter. Two surfaces, two sources of truth (chunk 10a's HUD inlines the chunk-7 math; the in-canvas counter uses the lib). Drift risk if the math changes in either place without the other. M14/M15 polish could extract a shared useTowerCounter(state, sceneIndex) hook that both surfaces consume.
  • GIF export needs to capture the new viz. Per the parked-items list above. Chunk 2's SVG renderer composes scene-by-scene; chunk-6's M10 Puppeteer skeleton would also need to record the new viz once enabled. Lands as a polish chunk or M14/M15 if required for launch.
  • Three-commits-per-chunk pattern (3a/3b/3c) didn't recur after chunk 3. Most chunks split into 2 (a/b) when split at all. Pattern: 3 sub-commits is the sweet spot only when scenes have natural infra/scene-content/integration steps that each individually compile. Most chunks fit cleanly into 2 sub-commits at most.
  • Scene-overlap forfeited in chunk 9. Spec described Scenes 19+20 as overlaid on Scene 18; the runner is strict-sequential, so they're implemented as linear scenes. Closeout note: a future Scene contract extension (overlay: boolean) could land them as true overlays. Today's pedagogical beats land in the correct order; only the visual overlap is missing.
  • Chunk-12 degraded-mode "restoration condition" not yet observable in production. The < 18 FPS for 2s → degrade; > 24 FPS for 5s → restore state machine is testable via mocked RAF (chunk-12 tests). The actual production trigger requires a real low-end device + a real GPU starvation scenario. M15 launch prep should include a "drive the laptop to thermal throttling and confirm degrade-then-restore fires correctly" smoke step.
  • Cross-browser real-browser smoke. The matrix above is empty by design — M15 launch prep owns the first full run. Per the M12 chunk-9 cross-browser table pattern, non-blocking issues are documented as "won't fix" with rationale. Real-browser smoke is the only verification path for the runtime behaviors jsdom can't model: real focus rings, real prefers-reduced-motion, real GPU.

Exit criteria

  • The 20-scene narrative runs end-to-end on a fresh thread submit with a real OpenAI run, hitting all milestones from prompt-entry through end-of-sequence flourish. Covered by pipeline-integration.test.tsx (mock event stream + ChatBubble accumulation invariants) + per-scene Vitest in scenes-*.test.tsx files (87 determinism assertions in scene-determinism.test.ts walk the full registry). Manual recipe step 1 is the human verification path.
  • The real run's tokens stream into the chat-bubble overlay independent of the visualization — covered by chunk-10b's events-primary path + the integration-test "chat bubble accumulates ALL streamed tokens regardless of scene position" assertion. Manual recipe step 5.4-5.5.
  • 20-30 FPS sustained on a 2020-class machine; automatic degrade triggers cleanly. Chunk-12's useFpsTracker state machine (8 Vitest assertions with mocked RAF) + chunk-12 degraded-mode opt-ins (10 assertions across VectorStrip / ParticleTrail / AttentionScene). Manual recipe step 2 is the human verification path; the "drive the laptop to thermal throttling" smoke step is parked for M15.
  • Reduced-motion mode shows every scene as a static frame the user can step through. Chunk-13 t = 1 pin + 4s auto-advance + Step override. Tests in CinematicViz-gates.test.tsx. Manual recipe step 3.
  • WebGL 2 unavailable mode shows the SVG fallback; both modes still produce all 20 scenes (just rendered differently). Chunk-13 informational gate (the SVG fallback is the default since chunk 2). Manual recipe step 4.
  • The persistent UI sections (vocab sidebar, chat bubble, layer counter, pipeline progress) are populated and click-targets work. Chunk-10a (LayerCounterHud + VocabSidebar reverse-lookup) + chunk-10b (ChatBubble events-driven). Click-targets covered by chunk-3a's PipelineProgressBar setScene wiring + chunk-11a's PlaybackControls.
  • The /share/{token}/runs/{run}/replay public route shows the same visualization as the owner view. Pages mount <CinematicViz /> identically in Threads/Show, Runs/Replay, Share/Replay (chunk 1 + chunk 10b events plumbing). Manual recipe step 5.
  • M8 components retired: VizPane.tsx, EmbeddingScene.tsx, related controllers + tests removed from the codebase. Chunk 1 deletion-first commit (a0bdf6b). Git history preserves the M8 implementation; phase1.md keeps the documentation.

M14 — Deployment

Purpose: Application deploys to DreamHost VPS reliably and is observable.

Tasks

  • Provision a single DreamHost VPS that hosts both production and staging.
  • Install: PHP 8.4, Composer, Node 20, SQLite, supervisor, nginx, certbot, ffmpeg.
  • (Optional) install Chromium if GIF_RENDERER=puppeteer.
  • nginx vhost config: two server blocks on the same VPS.
    • llm.trackr.live — production. Document root: /var/www/llm-viz/current/public.
    • staging.llm.trackr.live — staging. Document root: /var/www/llm-viz-staging/current/public.
    • Both proxy WebSocket upgrades to the local Reverb instance on their own ports (e.g., 8080 prod, 8081 staging).
  • HTTPS via Let's Encrypt direct to the VPS (no Cloudflare in front). HSTS header enabled. Auto-renewal via certbot's systemd timer.
  • supervisor configs (with separate program names for prod and staging so they can be restarted independently):
    • llm-viz-queue / llm-viz-staging-queue — queue workers.
    • llm-viz-reverb / llm-viz-staging-reverbphp artisan reverb:start on different ports.
    • (Optional) Puppeteer is spawn-per-export — no supervisor entry needed.
  • Two isolated SQLite databases (/var/www/llm-viz/database/database.sqlite and likewise for staging) so staging data can't pollute production.
  • Deploy script: ./deploy.sh {env}git pull, composer install --no-dev, npm ci && npm run build, php artisan migrate --force, php artisan optimize, supervisorctl restart llm-viz-{env}-*.
  • GitHub Actions deploy workflow: on push to main → SSH and run ./deploy.sh staging. On v* tag → run ./deploy.sh production.
  • SQLite backup cron: nightly .dump of production only to off-VPS storage (e.g., S3 or Backblaze).
  • Sentry DSN configured in both production and staging .env with distinct SENTRY_ENVIRONMENT values.
  • Smoke test from production deploy: sign in, submit a run, watch viz. Document the procedure in docs/deployment.md.
  • Resource guardrails documented: at expected launch traffic the VPS handles prod + staging concurrently, but a load spike on staging could affect prod (shared CPU/RAM). Note in docs/deployment.md to throttle staging during load tests.

Exit criteria

  • A push to main deploys to staging.llm.trackr.live within 5 minutes.
  • A v0.9.0 tag deploys to llm.trackr.live.
  • Both domains return 200 over HTTPS, certificates valid, HSTS header present.
  • Sentry receives a deliberately-thrown test error from production with environment=production.
  • Backup verified by restoring production database.sqlite from latest backup into a scratch directory.
  • Restarting staging supervisor processes does not interrupt production WebSocket connections.

M15 — Launch Prep

Purpose: Run the full acceptance criteria checklist, fix anything that fails, write user-facing documentation, and seed the database for first users. No invite-only beta — launch goes straight to public access at v1.0.

Tasks

  • Acceptance criteria walkthrough (all 18 items from SPEC §9), each ticked off with evidence (screenshot/recording/test name).
  • Performance: load-test with k6 or Artillery → 100 concurrent simulated users hitting the streaming pipeline. Run from staging to avoid blowing up production during the test.
  • Security audit:
    • Verify API keys are encrypted at rest (look at raw SQLite).
    • Test share-token enumeration (60/min IP rate limit holds).
    • CSRF on all state-changing routes.
    • Composer + npm audit clean.
    • AGPL compliance review: ensure every page footer and the API responses include a link to the source repository (AGPL §13 — interaction-with-source obligation).
  • User guide (docs/user-guide.md): screenshots, walkthroughs.
  • Admin guide (docs/admin-guide.md): registry refresh, user promotion, rate limit adjustment, backup restore.
  • CHANGELOG.md initialized with v1.0.0 entry.
  • Registry seeded with the launch set (9+ models per SPEC §7).
  • Promote one real admin user via user:promote.
  • Soft-launch checklist (since public from day one, no waitlist):
    • Verify rate-limit defaults (30/hour) are reasonable for first-day curiosity traffic.
    • Confirm Sentry alerts route to a real notification channel (email or Slack webhook).
    • Pre-write a "we hit a snag" status page or pinned issue template.
    • Have an admin promotion ready in case an early user needs higher limits.
  • Tag v1.0.0 and deploy to production.
  • Announce (forum/social/HN, at the operator's discretion).

Exit criteria

  • All 18 SPEC §9 criteria pass.
  • llm.trackr.live is live, returning 200, and a fresh user can complete a sign-in → run → replay loop end-to-end.
  • AGPL §13 source-link is visible on every page served to a user.

Cross-cutting concerns (apply across all milestones)

  • Test coverage: Each milestone must leave overall backend coverage ≥ 70%. Failing this blocks the milestone from being marked done.
  • Documentation: When a milestone introduces a new concept (e.g., M3 introduces registry refresh), docs/architecture.md or docs/admin-guide.md is updated in the same PR.
  • Security review: Any PR touching auth, encryption, share-link routes, or admin endpoints requires a second-pass review before merge.
  • Performance budget: Every PR that touches the viz canvas runs a Chrome DevTools performance trace as part of review.