Goal: Ship the LLM Process Visualizer as defined in SPEC.md. All 9 vendors, persistent threads, real-time visualization, replay, export (JSON + GIF), opt-in thread sharing, deployed to DreamHost VPS.
Status: Not started.
Source of truth for scope: SPEC.md §2.1 (in scope) and §2.2 (out of scope).
This document breaks Phase 1 into 15 milestones (M1–M15). Each milestone lists its purpose, concrete tasks, dependencies, exit criteria, and a rough effort estimate.
Note on estimates. Time estimates are in "engineer-days" assuming one person working full-time and familiar with the stack. They are planning aids, not commitments.
| # | Milestone | Depends on | Estimate | Status | Vertical-slice gate? |
|---|---|---|---|---|---|
| M1 | Foundation | — | 3 days | ✅ Complete | |
| M2 | Auth + Users | M1 | 4 days | ✅ Complete | |
| M3 | Model Registry | M2 | 4 days | ✅ Complete | |
| M4 | API Keys + Vendor Clients | M2 | 12 days | ✅ Complete | |
| M5 | Threads + Runs (data) | M3, M4 | 4 days | ✅ Complete | |
| M6 | Realtime + Streaming Pipeline | M5 | 6 days | ✅ Complete | |
| M7 | Frontend — Static UI | M5 | 7 days | ✅ Complete | |
| M8 | Frontend — Live Visualization | M6, M7 | 12 days | ✅ Complete (superseded by M13) | ✅ End-of-M8 = vertical slice |
| M9 | Replay + JSON Export | M8 | 4 days | ✅ Complete | |
| M10 | GIF Export | M8 | 6 days | ✅ Complete | |
| M11 | Thread Sharing | M9 | 3 days | ✅ Done | |
| M12 | Accessibility + Polish | M11 | 5 days | ✅ Done | |
| M13 | Cinematic Inference Visualization | M12 | 12 days | ✅ Done | Replaces the M8 live viz with a 20-scene narrative |
| M14 | Deployment | M13 | 5 days | ⚪ Not started | |
| M15 | Launch Prep | M14 | 5 days | ⚪ Not started | |
| Total | ~80 engineer-days |
Purpose: Empty repo → working Laravel + React + Vite dev environment with CI and linters.
Tasks
-
composer create-project laravel/laravelin repo root (Laravel 13.9.0, upgraded from initial Laravel 11 scaffold). - Switch DB driver to SQLite; create
database/database.sqlite(SQLite is the default in Laravel 13; DB file is gitignored per-environment). - Add
.editorconfig,.gitignore(Laravel + Node),.gitattributes(Laravel 13 defaults + project-specific extensions for SQLite/Sentry/OS files). - Install Inertia.js Laravel adapter (
inertiajs/inertia-laravel^3.1) + React preset (TypeScript): React 19,@inertiajs/react^2, TypeScript,@types/react/@types/react-dom/@types/node,@vitejs/plugin-react^5 (pinned to v5 for Vite 6 compat). Ziggy installed too (tightenco/ziggy^2.6 +ziggy-js) for Laravel-named-routes-in-JS. - Install Tailwind CSS 3 + configure
tailwind.config.js(Tailwind 3.4 already shipped in the skeleton; content paths updated to include.{ts,tsx}files). - Configure Vite for React + Inertia.
vite.config.js→vite.config.ts;react()plugin added; input updated toresources/js/app.tsx;@/*alias added pointing toresources/js/. - Set up Laravel Pint config:
pint.jsonwithlaravelpreset +concat_space,ordered_imports,no_unused_imports,global_namespace_import. All existing files auto-formatted to match. - Set up ESLint + Prettier for JS/TS. ESLint 9 flat config (
eslint.config.js) withtypescript-eslintrecommended +eslint-plugin-react+eslint-plugin-react-hooks+eslint-plugin-jsx-a11y(a11y rules support WCAG goals) +eslint-config-prettierto disable conflicting style rules. Prettier config in.prettierrc.json(4-space, single-quote, trailing commas, 100-col; YAML overridden to 2-space). Markdown files added to.prettierignoreto avoid table-alignment churn on edits. - Install Husky + lint-staged for pre-commit hooks.
.husky/pre-commitrunsnpx lint-staged(Pint on PHP, ESLint+Prettier on TS/JS/CSS/JSON/YAML/HTML) followed bynpm run type-check(TS errors caught at commit time, not just CI). - Install Pest for PHP testing (Pest 4.7, replacing the default PHPUnit ExampleTests). New
tests/Pest.phpbase config +tests/Feature/WelcomePageTest.phpcovering Inertia round-trip. PHPUnit downgraded to 12.5.24 for Pest compat.pest-plugin-laravelskipped — its Laravel 13 support hasn't shipped and most of its features are absorbed into Pest core anyway. - Extend
.env.examplewith project-specific keys (Socialite Google/MS/FB, Pusher-protocol broadcast, OpenRouter, Sentry DSN, GIF renderer, repo URL for AGPL §13, default rate limit). All values blank or sensible defaults; production overrides happen in.env. - GitHub Actions workflow:
.github/workflows/lint.yml(Pint, ESLint, Prettier, TypeScript) +.github/workflows/test.yml(Pest + Vitest + React Testing Library). Both pin PHP 8.4 + Node 20, cache Composer + npm, and useconcurrencyto cancel superseded runs. Initial PHP version was 8.3 per the SPEC, but Laravel 13 ships with Symfony 8.x which requires PHP 8.4+ — spec corrected accordingly. Vitest + RTL + jsdom installed for frontend testing;resources/js/Pages/__tests__/Welcome.test.tsxprovides the first 3 passing tests. Cypress E2E deferred to a later milestone (no real UI to drive yet). - Install Sentry Laravel SDK + frontend SDK. Backend:
sentry/sentry-laravel^4.25 withconfig/sentry.phppublished;Integration::handles($exceptions)registered inbootstrap/app.phpfor unhandled-exception capture (Laravel 11+ requirement). Frontend:@sentry/reactwithSentry.init()gated onVITE_SENTRY_DSN(no-op when blank — local dev stays silent and makes no network calls);Sentry.ErrorBoundarywraps the Inertia App with a<ErrorFallback />UI. Vite env types extended inresources/js/types/global.d.tssoimport.meta.env.VITE_SENTRY_*is type-checked. Verified: artisan boots,npm run buildsucceeds (~411 KB JS), Pest still passes. - Verify
npm run devandphp artisan serveboth work and Inertia round-trips between PHP and React. Verified end-to-end:npm run buildproduces a clean bundle;php artisan serve+curl /returns HTTP 200 with the Inertiadata-pageattribute embedded,component: "Welcome", page props serialized (Laravel 13.9.0, PHP 8.4.21), and Ziggy route data injected.
Exit criteria
-
php artisan serve+npm run devboot cleanly. Verified during chunk 2. - A
WelcomeInertia page renders React content. Tested bytests/Feature/WelcomePageTest.php(Pest, 2 tests / 6 assertions) andresources/js/Pages/__tests__/Welcome.test.tsx(Vitest, 3 tests). - CI passes on a no-op PR. Verified by the lint + test workflows added in chunk 5; first runs will exercise once a PR lands.
- Pre-commit hook blocks unformatted code. Verified live during chunks 3 and 4 — every chunk-closing commit ran lint-staged + type-check before landing.
M1 closed: 2026-05-17. Chunks 1–5 all green. Estimate was 3 engineer-days; actual was substantially less (single afternoon) but the user supplied a working environment and clear answers throughout, so it's not a representative pace for the rest of Phase 1.
Things that needed follow-up patches after the initial chunk-5 push:
- CI PHP version: spec said PHP 8.3+ but Laravel 13 ships with Symfony 8.x which actually requires PHP ^8.4. CI bumped to 8.4 and the spec corrected (
SPEC.md,README.mdupdated). Local dev was already on 8.4 so no functional change. tests/Unit/lost from git when we deleted the only file in chunk 3 (git doesn't track empty dirs). Added a.gitkeepso the directory survives fresh checkouts; phpunit.xml still references both Unit and Feature testsuites.
Known follow-ups not blocking M1:
- GitHub Actions Node 20 deprecation:
actions/checkout@v4,actions/setup-node@v4,actions/cache@v4all use Node 20, which GitHub flips to Node 24 default on 2026-06-02 and removes on 2026-09-16. Bump to v5 of each action when they're available (not yet at time of M1 close).
Purpose: Users can sign in via Google/Microsoft/Facebook. Admin can promote users. Rate limits enforce.
Tasks
- Migration:
users(per SPEC §6, nopasswordcolumn, includingmax_runs_per_hour,store_prompts,avatar_url,role). Edited the Laravel-default migration (greenfield, no audit-trail concerns yet).password_reset_tokenstable dropped from the migration (not needed under social-only auth). PHP 8.1 enumApp\Enums\UserRolewithUser+Admincases; Eloquent casts therolecolumn to it.Usermodel gainssocialAccounts()HasMany +isAdmin()helper.UserFactoryupdated withadmin()andprivacyOpted()states. - Migration:
social_accountswith(provider, provider_user_id)unique constraint,(user_id, provider)index, andcascadeOnDeletefrom users.SocialAccountmodel +SocialAccountFactorywithgoogle()/microsoft()/facebook()states. - Install Laravel Sanctum (
laravel/sanctum^4.3). Config published,personal_access_tokensmigration applied. Not used in Phase 1 — kept in place for future API-token use cases per SPEC. - Install Laravel Socialite (^5.27) +
socialiteproviders/microsoft(^4.9). Microsoft provider registered viaEvent::listen(SocialiteWasCalled::class, MicrosoftExtendSocialite::handle)inAppServiceProvider::boot(). Google + Facebook ship in Socialite core.config/services.phpextended with all three providers (env-driven; blank vars = provider button shown but redirect would fail with clear error). - Implement OAuth flow for Google, Microsoft, Facebook with callback handler.
App\Http\Controllers\Auth\SocialiteControllerexposesredirect()/callback()/logout(). Routes:GET /auth/{provider}/redirect,GET /auth/{provider}/callback,POST /logout. Provider allowlist (['google', 'microsoft', 'facebook']) gates both endpoints with 404 on unknown providers. - On callback: upsert user by provider+provider_user_id, log them in via session auth, redirect to dashboard. Email auto-link behavior (decided 2026-05-17): if no matching
social_accountsrow but the OAuth email matches an existing user, attach a newsocial_accountsrow to that user instead of creating a duplicate account.email_verified_atset tonow()on first creation (providers verify emails before issuing OAuth tokens). Avatar backfill: if the existing user has noavatar_urland the new provider supplies one, save it; never overwrite an existing avatar. - Login page (Inertia React): three large provider buttons.
resources/js/Pages/Login.tsxis the dedicated sign-in page with branding, an error banner that readserrors.socialfrom Inertia's auto-shared errors prop, and a "back to home" link. The OAuth callback failure path inSocialiteController::callback()now redirects toroute('login')(instead ofroute('home')) so the error displays where the user came from. Vitest covers button presence, route wiring, back link, and no-error default state. - Logout endpoint + UI. POST
/logoutwas wired in chunk 2; chunk 3 adds the visible UI inAppLayout.tsx(logout form button in the sidebar footer). Dashboard's standalone logout button removed in favor of the layout-level one. - Artisan command:
php artisan user:promote {email}. Implemented asApp\Console\Commands\PromoteUserusing Laravel 13 attribute-based signature/description. Three behaviors: promotes a user-role user → admin, no-ops with clear message if already admin, fails with non-zero exit + hint ("the user must sign in at least once") if no user matches. 3 Pest tests cover all branches. - Rate-limit middleware: per-user, per-hour, backed by Laravel's
RateLimiterfacade. Registered as a named limiterrunsinAppServiceProvider::boot()— the closure readsusers.max_runs_per_hourlive on every request so admin edits take effect immediately. Routes attach viamiddleware('throttle:runs'). The actual run-submission route lands in M5/M6; for now the limiter is verified via a test-only stub route.X-RateLimit-LimitandX-RateLimit-Remainingheaders are set by Laravel'sThrottleRequestsmiddleware automatically. Note: Laravel'sLimit::perHouris a fixed-hour-bucket scheme, not a true sliding window — close enough for our use case; can swap to a true sliding implementation later if user-visible behavior on the bucket boundary becomes a problem. - Admin UI shell (gated by
role = admin): user list with inline editor formax_runs_per_hour.App\Http\Controllers\Admin\UsersController@indexpaginates 20-per-page (User::orderByDesc('created_at'));updatevalidatesmax_runs_per_houras integer 0–10000 then redirects back with arate-limit-updated:{user_id}flash status. Frontend atresources/js/Pages/Admin/Users.tsxrenders a table with an inline<RateLimitForm>per row (number input + Save button + ✓ indicator on save). Routes gated by the existingadminalias from chunk 3. - Privacy toggle on user settings:
store_promptscheckbox.App\Http\Controllers\SettingsController@showrenders the page with the current value;updatevalidates as boolean and redirects back withsettings-savedflash status. Frontend uses Inertia'suseFormwith explicit Save button (not auto-save). Flash status read via the newflash.statusshared prop (added toHandleInertiaRequests::share()).
Exit criteria
- [~] Sign in works for all 3 providers (verified manually with real OAuth apps in dev mode). Verified at the mock level:
tests/Feature/Auth/SocialiteCallbackTest.php(10 tests) exercises the full callback flow for each provider via Socialite's mockable contracts — user creation, existing-link login, email auto-link, avatar backfill, logout, guest gating. Real-credential verification deferred until the user provisions OAuth apps with Google/Microsoft/Facebook and populates.env. The.env.examplekeys are in place from M1 chunk 3. - Promoting a user via
user:promoteflips theirroleand unlocks/adminroutes. Verified bytests/Feature/Console/PromoteUserCommandTest.php(3 tests, including the user-not-found error case) combined withtests/Feature/Auth/AuthGatedRoutesTest.php("returns 403 when a non-admin hits /admin/users" + "renders the Admin Users page for an admin"). - 31st request within an hour returns 429. Verified by
tests/Feature/RateLimit/RunsRateLimiterTest.php("returns 429 when the per-user limit is exceeded"). The actual test usesmax_runs_per_hour = 3for speed; the assertion is identical at any limit, including the default 30.
M2 closed: 2026-05-17. Chunks 1–5 all green. The single 🟠 marker on the manual-OAuth criterion is the only thing standing between M2 and a full ✅, and it's gated on the operator provisioning real OAuth credentials — not on any code work.
M2 retrospective notes:
- Real value of
eslint-plugin-jsx-a11y: caught a<label>without explicithtmlForassociation on the Settings page during chunk 4. Cost ~30 seconds to fix; would otherwise have surfaced in the WCAG audit at M12. - Vite manifest gotcha (twice): adding new Inertia pages without running
npm run buildproduces "Unable to locate file in Vite manifest" 500s in Pest tests. CI handles this (test workflow runsnpm run buildbefore Pest); local doesn't. Not adding a guard, but worth knowing. - Inertia page glob picked up a test file: in chunk 2 the path
Pages/__tests__/Welcome.test.tsxgot bundled into the production build via the page-resolver glob — 457 KB of Vitest leaked into prod. Moved tests toresources/js/__tests__/Pages/(outside the glob). Worth a CI guard later if this happens again. - Auto-link by email decision (chunk 2): users with the same email across providers now share one account. Acceptable since Google/MS/FB all verify emails before issuing OAuth tokens. If we see a single phishing/takeover report, revisit by adding an explicit "link account" workflow.
- Sliding-window vs fixed-bucket rate limiting (chunk 4): SPEC said sliding, used fixed-hour-bucket via Laravel's built-in. Documented; swap if bucket-boundary unfairness becomes a real complaint.
Stats: Pest 52 tests / 147 assertions, Vitest 8 tests. All CI green.
Purpose: Models table populated from OpenRouter weekly, with architecture metadata enriched from a local fixture. Admin can edit and override.
Tasks
- Migration:
modelsper SPEC §6 — plusmanual_override(refresh-skip flag) andmetadata_estimated(best-guess marker for closed-source layer counts). Architecture metadata columns (architecture_type,layers,hidden_dim,attention_heads,moe_experts,moe_active_experts,position_encoding) are all nullable since OpenRouter doesn't expose them — they're populated by the fixture or admin edits in chunks 2 and 4. PHP 8.1 enumsApp\Enums\ArchitectureTypeandApp\Enums\PositionEncodingprovide strong typing; Eloquent casts both columns. Eloquent model namedApp\Models\LlmModel(with explicit$table = 'models') to avoid shadowingIlluminate\Database\Eloquent\Model. Factory + states (moe(),vendor(),estimated(),manuallyOverridden()). - Migration:
registry_meta(key/value; trackslast_successful_refresh_at). String-primary-key key/value table with JSON value column andupdated_atonly (no created_at — meta entries are upserted, not "created").App\Models\RegistryMetaexposesgetValue($key),setValue($key, $value), andforget($key)static helpers.setValueuses the Query Builder'supdateOrInsertrather than Eloquent'supdateOrCreateso identical-value writes still bumpupdated_at(otherwise the dirty-checking insave()would skip the write and break "last refresh time" tracking). -
OpenRouterClient(Guzzle):fetchModels(): array. Implemented inApp\Services\OpenRouter\OpenRouterClientusing Laravel'sHttpfacade (Guzzle under the hood). Normalizes each row: splits thevendor/modelid, converts per-token pricing strings to per-million-token floats, extracts display_name and context_length. Skips rows with unparseable ids (missing slash, null/non-string id). ThrowsRuntimeExceptionon HTTP failure or missingdataarray. Reads base URL fromconfig('services.openrouter.base_url'), defaulting tohttps://openrouter.ai/api/v1. Constructor optional$baseUrloverride for tests. - Fixture file:
database/seeders/data/architecture_metadata.php. Keyed bymodels.name(post-slash portion of OpenRouter id). Each entry carriesarchitecture_type,layers,hidden_dim,attention_heads,moe_experts/moe_active_experts(where applicable),position_encoding, display_name override, capability flags, andmetadata_estimated(true for closed-source vendors, false for open-weights). All 10 Phase 1 launch-set models present per SPEC §7. Open-weights (Llama 3.1 70B/405B, Mixtral 8x22B) have real architecture data; closed-source (GPT-4o, Claude 3.5, Gemini 1.5 Pro, Grok 2, Mistral Large) are flagged estimated with rope position encoding as a sensible default. - Service:
ModelRegistryRefreshService— joins OpenRouter + fixture, upserts, respectsmanual_override. Implemented inApp\Services\ModelRegistry\RefreshServicewith a smallRefreshResultvalue object. Logic: fetch upstream, load fixture, for each row check for existing match byname, skip ifmanual_override = true, else merge OpenRouter base with fixture entry (fixture wins on overlap), upsert. Wraps everything in a DB transaction so a catastrophic failure rolls back the partial state; per-row exceptions are caught and recorded inRefreshResult.errorsso one bad row doesn't kill the whole refresh. On success, writeslast_successful_refresh_atandlast_refresh_summarytoregistry_meta. Constructor takes the client;refresh(?$fixturePath)accepts an override path for tests. - Artisan command:
php artisan registry:refresh. Implemented asApp\Console\Commands\RegistryRefreshusing Laravel 13 attribute-based signature. Accepts--fixture=to override the architecture metadata path (for tests / staging). Outputs theRefreshResult::summary()string on success and any per-row errors. Exit code 0 on success, 1 on uncaughtRuntimeExceptionfrom the service. - Scheduled task (Laravel scheduler): weekly, with admin-email notification on failure. Registered in
routes/console.php(Laravel 11+ pattern, replacing the old Kernel::schedule). RunsweeklyOn(1, '03:00')— Monday 03:00 UTC, intentionally off-peak.->onFailure(...)callback queriesUser::where('role', UserRole::Admin)and sendsApp\Notifications\RegistryRefreshFailedto all admins via themailchannel. WithMAIL_MAILER=login dev, failure notifications appear instorage/logs/laravel.lograther than emailing anyone. - Seeder for initial registry:
Database\Seeders\ModelRegistrySeedercallsRefreshServiceonphp artisan migrate --seed. Resilient: catches network failures with a clear warning rather than aborting the seed. Short-circuits in thetestingenv so Http::fake() controls test fixtures without the seeder interfering. Wired intoDatabaseSeeder::run(). - Admin UI: model CRUD with
manual_overridetoggle.App\Http\Controllers\Admin\ModelsControllerprovidesindex/edit/update/destroy/refresh. Index has search (name + display_name), vendor filter, architecture filter, paginated at 25, with badge indicators formanual_overrideandmetadata_estimated. Edit form covers all admin-editable fields grouped into Identity / Architecture / Capacity+pricing / Capabilities / Chat template / Refresh control sections (skippingsupported_paramsJSON — too low-frequency to merit a UI for now). Validation: vendor required, architecture_type/position_encoding constrained to enum values, numeric ranges enforced on layers/hidden_dim/heads/MoE/context/pricing. Refresh button posts toadmin.models.refreshwhich invokesRefreshService::refresh()and surfaces success summary orerrors.refreshflash. Delete is a hard delete — M5'sruns.model_idforeign key will need an ON DELETE strategy when that lands. The placeholder/modelsroute now redirects authenticated admins to/admin/modelsand keeps aComingSoon → M7placeholder for non-admins (public model browser is an M7 deliverable). Sidebar split intoAdmin · UsersandAdmin · Models, both admin-only. - Staleness banner component: visible to all authed users when
last_successful_refresh_at > 14 days ago(or never). Threshold (14 days) is a public constantHandleInertiaRequests::STALENESS_THRESHOLD_DAYSper SPEC §7.1. Shared with every Inertia page via aregistryprop (is_stale,days_stale,last_refresh_at). Banner lives inresources/js/Components/RegistryStalenessBanner.tsxand renders insideAppLayoutjust above the page content — only when stale. Admins see "View registry" + "Refresh now" actions; non-admins see "Ask an admin to run the registry refresh". The SPEC said "model-selector page"; we put it in the layout so the warning is visible on every authed page since the registry feeds cost estimates app-wide. When M7 ships the public model selector, it can either rely on the layout banner or add a more prominent variant.
Exit criteria
- [~]
php artisan registry:refreshpopulates ≥ 50 models with vendor, pricing, context length. Verified at the mock level:RefreshServiceTestexercises the full create/update/skip cycle againstHttp::fake()responses. Real ≥ 50-model count deferred until first production run hits the live OpenRouter API (the upstream typically returns 100+ models). The fixture, refresh service, and command are otherwise complete and tested. - The launch-set 9+ models have full architecture metadata.
database/seeders/data/architecture_metadata.phpcontains all 10 SPEC §7 launch-set entries.ArchitectureMetadataFixtureTestverifies presence + correctness ofmetadata_estimatedflags + Mixtral 8x22B MoE structure. - Editing a model and setting
manual_override = truesurvives the next refresh. Verified byRefreshServiceTest::it skips rows with manual_override = true— pre-existing row withmanual_override = trueand stale context_length is untouched after the refresh runs. The admin UI's Edit form (chunk 4) is the path admins use to set this flag.
M3 closed: 2026-05-17. Chunks 1–5 all green.
M3 retrospective notes:
- The
RegistryMeta::setValueEloquent dirty-checking gotcha (chunk 1):updateOrCreateskips DB writes when no attribute is dirty, so repeated identical refreshes wouldn't bumpupdated_at. Switched to Query Builder'supdateOrInsert. This is the kind of subtlety that would have shipped silently and only surfaced months later when "last refresh" data stopped moving. Pages/__tests__/glob leak revisited (chunk 4): moved 3 new test files (Admin/Models test, registry tests) outsideresources/js/Pages/to keep the M2 chunk-2 lesson holding. Worth adding a CI check that fails the build if any.test.tsxfile appears insidePages/.expectsOutputToContainconsumes substring (chunk 3): Laravel 13's PendingCommand consumes the matched substring from the captured buffer, so chained assertions on overlapping content silently fail. Switch to a single longer substring or split into separate tests.$this->seed(SeederClass)wires a mocked OutputStyle (chunk 3): that doesn't expect SymfonyStyle's internalaskQuestion()calls from$this->command->info(). Workaround: invoke the seeder directly viaapp(SeederClass)->run(). The?->null-safe operator skips the command-bound output.- Staleness banner in AppLayout, not just model-selector (chunk 5): SPEC said model selector (M7 territory). We put it layout-level since registry data feeds cost estimates everywhere. When M7 lands, can re-evaluate placement.
Stats: Pest 121 tests / 329 assertions, Vitest 8 tests. All CI green.
Purpose: Per-user encrypted API key storage and a working LlmClientInterface implementation for all 9 vendors.
Tasks
- Migration:
api_keys(encrypted_key, vendor, label). Schema includesuser_id(FK with cascadeOnDelete),vendor, nullablelabel, encryptedencrypted_key(text, via Eloquent'sencryptedcast), denormalizedlast_four(cached plaintext suffix to avoid per-row decrypts on the list view), nullablelast_used_at, timestamps.UNIQUE(user_id, vendor, label)per SPEC §6 so a user can hold multiple keys per vendor under different labels.App\Models\ApiKeywith$casts['encrypted_key' => 'encrypted']+booted()event that recomputeslast_fouron every save when the key is dirty.encrypted_keyhidden from serialization.User::apiKeys()HasMany added. ApiKeyFactory withvendor()/withLabel(?string)/withKey()states. - API key management UI: list + add + delete; values shown masked except last 4 chars.
App\Http\Controllers\ApiKeysControllerwithindex/store/destroy— authenticated only, plus a 403 check indestroyso users can't delete each other's keys (even admins are blocked — BYOK trust means admins shouldn't see other users' secrets). Vendor allowlistSUPPORTED_VENDORS(the same 9 from SPEC §3.2.2) gates the validation. Duplicate (vendor, label) returns a friendlylabelvalidation error instead of a DB exception. Page atresources/js/Pages/ApiKeys/Index.tsxreplaces the chunk-3 ComingSoon placeholder. Form has vendor select / optional label / masked password input. Existing keys table shows masked display (••••XXXX), last_used date, and a Delete button. Flash banners for add/delete. Plaintext key value is never echoed back to the client after creation. - Define
LlmClientInterface. Parked decision (docs/parked-decisions.mditem 2) resolved 2026-05-17 in favor of hand-rolled — Laravel AI SDK doesn't expose logprobs (needed for SPEC §3.1.5 logits panel) and has no custom-vendor extension path; Prism is still pre-1.0. The interface is inapp/Services/Llm/Contracts/LlmClientInterface.phpwith three methods:stream()returnsGenerator<int, LlmTokenChunk>,complete()returnsLlmCompletion, plus avendor()identifier for factory registration. Slight refinement from the SPEC: takes anApiKeymodel + themodelstring explicitly (rather than relying on globals), and yields/returns typed value objects (LlmTokenChunk,LlmCompletion,LlmUsage) instead of bare arrays. Vendor-specific exceptions (InvalidApiKeyException,VendorRateLimitedException, genericLlmClientException) communicate user-actionable error categories. - Implement clients (one per vendor — each is its own sub-task with its own tests):
-
OpenAiClient(SSE streaming, supportslogprobs). Lives atapp/Services/Llm/Clients/OpenAiClient.php. Streams viaPOST /v1/chat/completionswithstream: true, parses SSE through a reusableApp\Services\Llm\Support\SseParser. Requestsstream_options.include_usage = trueso the final chunk carries token totals. Passes throughtemperature,top_p,max_tokens,seed; silently dropstop_k(not an OpenAI param). Whenparams.logprobs = true, setstop_logprobs = 5(or override) so the logits panel can show the top-5 alternatives. HTTP error mapping: 401/403 →InvalidApiKeyException, 429 →VendorRateLimitedException(withRetry-Afterparsed), other non-2xx → genericLlmClientException.api_keys.last_used_atupdated on success. Registered withLlmClientFactoryinAppServiceProvider::boot(). Designed as the base class for chunk 4's 4 OpenAI-compatible vendors (xAI, Mistral, Groq, Together) — subclasses only overridevendor()anddefaultBaseUrl(). -
AnthropicClient(event-stream). Bespoke —x-api-keyheader +anthropic-version: 2023-06-01. System prompts extracted from history into a top-levelsystemfield.max_tokensmandatory on Anthropic; defaults to 4096 if caller didn't specify. No logprobs support. ReusesSseParserfor the SSE shell; dispatches on thetypefield inside each event (message_start,content_block_delta,message_delta,message_stop). Tracks cumulative usage across the stream since Anthropic reportsinput_tokensinmessage_startandoutput_tokensin the trailingmessage_delta. -
GoogleGeminiClient(streamGenerateContent). Bespoke. Auth via URL query string (?key=…), not a header. Endpoint:/v1beta/models/{model}:streamGenerateContent?alt=sse. Body usescontents+partsshape instead ofmessages; roleassistantmapped tomodel; system messages extracted into top-levelsystemInstruction; param naming differs (topP/topK/maxOutputTokensundergenerationConfig). ReusesSseParser. No logprobs support. -
XaiClient(OpenAI-compatible). Thin subclass ofOpenAiClient— only overridesvendor(),defaultBaseUrl()(api.x.ai/v1), andextraHeaders()to drop the OpenAI-Organization header. All payload / streaming / error-mapping logic inherited. -
MistralClient(OpenAI-compatible). Same pattern; base URL api.mistral.ai/v1. Mistral's optionalsafe_promptparameter isn't exposed in our params shape; can be added if needed. -
GroqClient(OpenAI-compatible, very fast). Same pattern; base URL api.groq.com/openai/v1 (note the/openai/prefix — Groq exposes their OpenAI-compatible surface there). -
TogetherClient(OpenAI-compatible). Same pattern; base URL api.together.xyz/v1. Will also backMetaViaTogetherClientin chunk 5. -
HuggingFaceClient(Inference Endpoints). SPEC deviation: SPEC called for TGI native protocol; we use HF's OpenAI-compatible chat-completions surface instead because (a) it handles per-model chat templates server-side, (b) the wire format matches OpenAI exactly soSseParserandOpenAiClient's payload code reuse cleanly, (c) most managed HF Inference Endpoints expose this surface. Thin subclass ofOpenAiClient. No global base URL — users configureservices.huggingface.base_urlper their Inference Endpoint URL; per-modelapi_base_urloverrides wire in at M5/M6. -
MetaViaTogetherClient— Llama models proxied through Together. Trivial subclass ofTogetherClientoverriding onlyvendor(). Exists so the model registry can list Llama models undervendor='meta'while still routing to Together's API. Key resolution: M5/M6 run-submission layer picks whichApiKeyto pass (UX note: a fallback "use Together key if no Meta key exists" would be friendlier — deferred to M5).
-
- Vendor client factory: maps
models.vendor→ concrete class.App\Services\Llm\LlmClientFactoryis registration-based — concrete clients call$factory->register($client)and are looked up byvendor(). Registered as a singleton inAppServiceProvider::register()so registrations apply app-wide and tests can swap implementations via the same instance.clientFor()throwsUnsupportedVendorExceptionon unknown vendor. The 9 concrete client registrations land in chunks 3-5. - Per-vendor token counter (uses
tiktoken-phpfor OpenAI; approximate BPE for others).yethee/tiktoken^1.1 installed (PHP port of OpenAI's BPE tables).TokenCounterInterfaceexposescount()+isExact()(the UI uses the latter to decide whether to show a~prefix).OpenAiTokenCounterselects encoding by model name (o200k_base for GPT-4o family, cl100k_base for older).ApproximateTokenCounteruses chars/4 with a small whitespace adjustment (±20% accuracy in English prose).TokenCounterFactoryreturns the OpenAI counter forvendor=openaiand the approximate counter for everything else.EncoderProvidercached as a singleton so BPE tables load once per process. - Integration tests: recorded HTTP fixtures (VCR-style) per vendor for streaming + non-streaming. Each client suite (
OpenAiClientTest,AnthropicClientTest,GoogleGeminiClientTest, the OpenAI-compat cluster dataset, the HF/Meta subclass tests) usesHttp::fake()with realistic vendor SSE bodies / non-streaming JSON payloads as inline PHP strings. Vendor-side error codes (401/403/429/500) are also fixture-driven. Inline-string fixtures over fixture files is a deliberate trade-off for now — extract totests/Fixtures/Llm/*.{sse,json}when test files start sharing payloads. - Smoke-test artisan command:
php artisan vendors:smoke-test. Implemented asApp\Console\Commands\VendorsSmokeTest. IteratesLlmClientFactory::supportedVendors(), reads each vendor's test key fromSMOKE_TEST_{VENDOR}_KEYenv var (skips with a clear notice if unset), reads optional model override fromSMOKE_TEST_{VENDOR}_MODEL, sends"Say 'ok' and nothing else."withmax_tokens: 10, temperature: 0. Reports per-vendor status (✓ passed / ✗ failed / ○ skipped). Default behavior: stop on first failure;--keep-goingto continue.--vendor=namefilter (repeatable) for testing a single vendor. Returns 0 if all pass-or-skip, 1 if any fail. Safe to run with no env configured at all — exits 0 with all-skipped (intentional so it can be wired into a deploy hook before keys are provisioned). Uses transient (unsaved)ApiKeyinstances;ApiKey::touchUsed()was updated to no-op for unsaved models so the smoke test doesn't insert orphan rows.
Exit criteria
- Each of the 9 clients passes its recorded-fixture tests. Verified: ~99 tests across the 9 client + factory + token-counter + value-object suites, all green.
- [~]
vendors:smoke-testsucceeds against all 9 live APIs. Mock-level verified: command flow / error handling / exit codes / env-var skip /--vendorfilter /--keep-goingall tested with fake clients. Real-network verification deferred until operator provisionsSMOKE_TEST_*_KEYenv vars with CI-account keys. - Submitting a 50-token prompt to OpenAI via the client yields a stream of token chunks in PHP. Verified at the mock level by
OpenAiClientTest. Real-network verification follows once keys are provisioned.
M4 closed: 2026-05-17. Chunks 1–6 all green.
M4 retrospective notes:
- The parked Laravel-AI-SDK vs hand-rolled decision (chunk 2) was the most consequential M4 choice. Laravel AI SDK covers all 9 vendors but doesn't expose logprobs — SPEC §3.1.5's logits panel needs them. Going hand-rolled meant more code (chunks 3–5) but full control of raw signals. If the viz ends up not needing logprobs after all, the Laravel AI SDK becomes a viable refactor target — every concrete client could be replaced with a thin SDK wrapper behind
LlmClientInterface. - The 9 vendors clustered better than expected. OpenAI-compatible (5) + bespoke (3) + wrapper (1). The base-class abstraction in
OpenAiClientpaid off twice: chunk 4's cluster (5 vendors × ~15 LOC each) and chunk 5's HuggingFace. - Pint's
php_unit_method_casingrule trips on any class with a method starting withtest— even non-test classes. Caught at chunk 6 when my privatetestVendor()got snake-cased totest_vendor(breaking the caller). Renamed toattemptVendorto avoid. Consider excluding the rule forapp/Console/if we hit this again. ApiKey::touchUsedhad to become defensive so the smoke-test command could use transientApiKeyinstances without inserting orphan rows. One-line guard; never fires in normal production flow.- SPEC deviation on HuggingFace (chunk 5): SPEC said TGI native, we went OpenAI-compatible. Documented in
HuggingFaceClientdocblock + commit. Practical reasons: chat templates handled server-side, code reuse fromOpenAiClient. Revisit if a user-deployed endpoint exposes only the TGI surface. - Inline-string SSE fixtures instead of fixture files: works fine at 9 vendors. Extract when payloads start being shared across tests.
Stats: Pest 244 tests / 578 assertions, Vitest 8 tests. All CI green.
Purpose: The thread/run data model is fully implemented and accessible through Eloquent models, but not yet wired to streaming.
Tasks
- Migration:
threadsper SPEC §6 (includingshare_token,share_enabled_at). Schema: user_id (cascade), nullable title (auto-filled to first 60 chars of first prompt at M5 chunk 4), nullable system_prompt, nullable default_model_id withnullOnDeleteso a removed model doesn't break the thread, default_parameters JSON, archived bool default false, tags JSON, share_token unique (M11 sharing), share_enabled_at, last_activity_at, timestamps. Indexed by user_id / archived / last_activity_at. - Migration:
runsper SPEC §6 (includingthread_id,sequence_in_thread). Schema: thread_id (cascade), denormalized user_id (cascade), model_id with restrictOnDelete so admin-deleting a model with runs is blocked (matches M3 chunk 4's design note), sequence_in_thread (unique with thread_id), nullable prompt (privacy opt-out per SPEC §10.4), prompt_hash always required (deterministic replay seed), conversation_history / parameters / token_log all JSON nullable, output_text + numeric timing/cost columns, status (cast toApp\Enums\RunStatus: Pending / Streaming / Complete / Error withisTerminal()), nullable error_message. - Eloquent models with relationships (
User → Threads → Runs → Model).Thread::runs()orders by sequence_in_thread by default.Run::thread()/user()/model()BelongsTo.User::threads()andUser::runs()HasMany.LlmModel::runs()HasMany (with explicitmodel_idFK becauseFactory::for()would otherwise auto-derive allmModel()relation name from the class).Thread::isShared()helper;Run::isTerminal()delegating to the enum. -
ThreadService:create(),archive(),rename(),tag(),delete(). Plusunarchive()for symmetry.App\Services\Threads\ThreadServiceis deliberately Eloquent-thin — authorization stays in the controller/policy layer above.create()accepts optionaltitle/system_prompt/default_model_id/default_parameters/tags; title left null lets RunService::submit (chunk 4) auto-fill from the first prompt.rename()trims whitespace and rejects empty strings withInvalidArgumentException.tag()normalizes (trim per-tag, drop empty/whitespace-only, dedupe case-sensitively to preserve "AI" vs "ai" intent); empty array or null clears tags.delete()is a hard delete — FK cascade drops the thread's runs. None of these methods touchlast_activity_at(only run-submission does in chunk 4). -
RunService::submit($user, $thread, $model, $prompt, $params). Signature refined from SPEC: takesUser+LlmModelobjects (caller resolves them via Laravel route binding) instead of scalar IDs. Validates in order: user owns thread (ThreadOwnershipException), prompt non-empty (EmptyPromptException), params within SPEC §3.1.4 bounds (InvalidParamsException), user has API key for the model's vendor with Meta → Together fallback (NoApiKeyException), context budget not exceeded withmax_tokensreserved for response (ContextOverflowException). Then in a DB transaction: auto-titles the thread (first 60 chars of first prompt, word-boundary truncated, ellipsis if truncated) on the first run only, bumpslast_activity_at, computes nextsequence_in_thread, applies privacy redaction peruser.store_prompts(nullspromptandconversation_historybut always storesprompt_hash), snapshots model architecture intoparameters.model_snapshotfor replay determinism (SPEC §10.1), creates the Run inpendingstate. Returns the Run withthread/user/modelrelations pre-set so the M6 pipeline doesn't need to re-load. All five failure modes raise aRunSubmissionExceptionsubclass that the HTTP layer in M6 maps to status codes. - Conversation-history builder: produces
[{role, content}, ...]from a thread's completed runs.App\Services\Threads\ConversationHistoryBuilder::build($thread)returns a vendor-agnostic array starting with{role: 'system', content: thread.system_prompt}(if non-empty), followed by alternating{role: 'user'}/{role: 'assistant'}pairs from prior completed runs ordered bysequence_in_thread. Pending/streaming/errored runs are skipped. Privacy-redacted runs (prompt=null or output_text=null) are skipped entirely — including a half-pair would give the model nonsensical context. Documented as a known limitation: users wanting continuity keepstore_prompts=true. The caller's new prompt is NOT appended here — that's the vendor client's job (each protocol formats the current turn differently). - Context-budget calculator:
App\Services\Threads\ContextBudgetCalculator::check($model, $history, $newPrompt, $reservedForResponse = 0). Returns aContextBudgetResultvalue object withfits / totalTokens / budget / overByso the caller can surface a useful "you're N tokens over a Y-token window" message. Routes to OpenAI's exact tiktoken counter whenvendor=openai, else the approximate counter — sameTokenCounterFactoryas the prompt-preview UI. Models with null/zerocontext_lengthare treated as unlimited (fits=true) since we can't enforce a budget we don't know. Defense-in-depth check — the frontend already does a precise client-side check before submit (SPEC §3.5); this guards against a malicious or buggy client. - Tests: thread CRUD, run validation, context overflow rejection, history snapshot correctness. Covered across chunks 1–4:
ThreadModelTest(11),RunModelTest(14),ThreadServiceTest(18),ConversationHistoryBuilderTest(10),ContextBudgetCalculatorTest(10),RunServiceTest(22) — 85 M5 tests in total.
Exit criteria
- Threads/runs can be created and queried via Eloquent. Verified by
ThreadModelTest+RunModelTest(25 tests covering persistence, casts, relations, FK behavior). -
RunService::submitrejects invalid input with clear errors. Verified byRunServiceTestvalidation group (10 tests): wrong owner, empty prompt, no key, all four param bounds, context overflow with details — each raising a distinctRunSubmissionExceptionsubclass. - Submitting two runs in a thread results in the second run's
conversation_historycontaining the first run's user+assistant turns. Verified byRunServiceTest::it stores the conversation history snapshot— creates a completed first run, submits a second, asserts the second run's snapshot is[{user: first q}, {assistant: first a}].
M5 closed: 2026-05-18. Chunks 1–5 all green.
M5 retrospective notes:
- Migration timestamp collision.
php artisan make:migrationproduced bothcreate_threads_tableandcreate_runs_tablewith the same timestamp; runs would have run alphabetically first despite its FK to threads. Bumped the runs filename timestamp by 1s. Worth knowing for any future multi-migration commit — generate them seconds apart or rename. Factory::for($model)auto-derives the relation name from the related model's class name (camelCase).LlmModel→llmModel(). We define the relation asmodel()for cleaner code, so test setups needFactory::for($model, 'model')explicitly.- Pest's
toThrow(string)does substring matching, not isinstance when given a string.toThrow(\Throwable::class)reads as "the exception message contains the substring 'Throwable'" — never matches real exceptions. Use concrete exception classes (QueryException::class, etc.). - Test factories + unique constraints. Hardcoding
'name' => 'gpt-4o'in tests clashes with factories made inbeforeEach. Drop the override; the factory generates unique names; OpenAI token counter falls back too200k_basefor unknown names so behavior is identical. - Pint's
lambda_not_used_importrule auto-strips unused variables from closureuse()lists. Useful but can be surprising if you expected a future-use placeholder to stick around — annotate intent in a comment if you really want to keep it. - The Meta-via-Together key fallback was closed here in RunService (chunk 4), addressing the UX gap noted at M4 chunk 5. When a user submits a
vendor=metamodel and has nometaAPI key but DOES have atogetherkey, the service silently uses the Together key. No-op for users who explicitly added a Meta key.
Stats: Pest 329 tests / 756 assertions, Vitest 8 tests. All CI green.
Purpose: End-to-end: prompt submitted → vendor stream → WebSocket events → frontend receives them. No visualization yet (M7/M8), but a debug page shows raw events.
Tasks
- Install + configure Laravel Reverb on local dev. (SPEC originally said Soketi; swapped to Reverb at M6 chunk 1 — see
docs/parked-decisions.mditem 1.)composer require laravel/reverb^1.10.config/reverb.php+routes/channels.phppublished.bootstrap/app.phpextended withchannels: __DIR__.'/../routes/channels.php'so channel-auth routes register at boot. - Configure Laravel
broadcasting.phpwith thereverbdriver.BROADCAST_CONNECTION=reverbset in.env.example. Old PUSHER_* env keys renamed toREVERB_*to match Laravel conventions. Vite frontend env mirrors (VITE_REVERB_APP_KEYetc.) added for the chunk 4 Echo wiring.resources/js/types/global.d.tsImportMetaEnvaugmentation updated accordingly. - Install Laravel Echo +
pusher-json frontend (chunk 4b).laravel-echo^2.3 +pusher-js^8.5 — pusher-js handles the Reverb-compatible WebSocket transport.resources/js/echo.tsinitializeswindow.Echofrom theVITE_REVERB_*env vars; init is conditional onVITE_REVERB_APP_KEYbeing set so environments without Reverb config don't bomb the bundle on load. Side-effect-imported fromapp.tsxbefore any page subscribes.npm auditflags 3 moderate transitivewsadvisories via socket.io-client (one of laravel-echo's optional transports we don't use); the vulnerable code is unreachable in our pusher-js bundle. Future hardening (M14 deployment) can add a package.jsonoverridesblock if it matters. - Channel:
private-runs.{run_id}(chunk 4a). Registered inroutes/channels.phpviaBroadcast::channel('runs.{runId}', ...)— the callback resolves the Run by ID and authorizes only whenrun->user_id === user->id. Nonexistent runs and stranger requests both returnfalse(and thus 403 from/broadcasting/auth) — the closure doesn't distinguish them to avoid leaking the existence of other users' runs. Auth flow: pusher-js POSTssocket_id+channel_nameto/broadcasting/auth(the route Laravel auto-registers frombootstrap/app.php'swithRouting(channels: ...)); on 200, Laravel signs the channel with the broadcasting secret and the client uses that signature to subscribe. 4 Pest tests cover owner/stranger/nonexistent/unauthenticated paths. Test-env wrinkle:BROADCAST_CONNECTION=pusheris set inphpunit.xmlso the channel-auth endpoint actually runs the callback — the defaultlogdriver'sauth()is a no-op that returns 403 for everything, which would make the tests pass trivially for the wrong reason. FakePUSHER_APP_*values are fine since no network leaves the box. - Job:
StreamRunJob— pulls run, picks vendor client, iteratesstream(), broadcasts events. - Events:
RunStarted(chunk 1),TokenReceived,LayerAdvanced,MoeRouted,RunCompleted,RunErrored. All inApp\Events\Runs\, all implementShouldBroadcastNowonprivate-runs.{run_id}(refactored fromShouldBroadcastin chunk 3 — per-token broadcasts fire from inside the queuedStreamRunJobworker, so going through the queue again would cost an extra round-trip and risk out-of-order delivery;ShouldBroadcastNowkeeps the broadcast in-process and ordered). EachbroadcastWith()returns a JSON-safe payload (scalars + arrays only) so the frontend doesn't need to know about Eloquent shapes.MoeRoutedis the only one that's vendor-conditional (emitted only forarchitecture_type='moe'model snapshots). - State machine:
App\Services\Runs\RunEventEmittertranslates eachLlmTokenChunkinto a deterministic event sequence:TokenReceived+LayerAdvanced, plusMoeRoutedfor MoE models. MoE expert selection is a function of(run.id, token_index)only, via a SHA-256-based stateless PRNG — replays produce identical animations per SPEC §10.1. Scores are synthetic descending probabilities renormalized to ~1.0 (no proprietary MoE vendor exposes real router logits, so this is illustrative-only). Also exposescompletedEvent()(computes tokens-per-second from duration) anderroredEvent()for the StreamRunJob (chunk 3) to call once at terminal status. - Job:
StreamRunJob(App\Jobs\StreamRunJob) — implementsShouldQueue, takes aRunvia constructor. Per-run flow: refresh from DB (defensive against worker retries — bails immediately if status is no longerPending); resolve user'sApiKeyfor the model's vendor with Meta → Together fallback mirroringRunService::resolveApiKey; resolve vendor client viaLlmClientFactory; flip status toStreamingand broadcastRunStarted; iterateclient->stream(...). Per chunk: computet_msfrommicrotime(true), append{token, index, t_ms, logprobs}to in-memorytoken_log(only whenchunk->textis non-empty), concatenateoutput_text, dispatch the eventsRunEventEmitter::eventsForChunk()returns, track the latest non-nullchunk->usage. On clean finish: computeduration_ms, deriveinput_tokens/output_tokens(from latest usage or fall back to token count), computetokens_per_second, computeestimated_costfromparameters.model_snapshotpricing (returns null when either price is absent — so a price change between submit + execute can't retroactively rewrite a recorded cost), persist final state, touchApiKey::last_used_at, broadcastRunCompleted. On vendor exception: friendly-error mapping forLlmClientExceptionsubclasses (rate-limit, invalid-key, generic), persist partialoutput_text+token_log+duration_ms+output_tokens, broadcastRunErroredwith the partial output. Pre-stream errors (no API key, no client registered) still broadcastRunErrored— the frontend's chunk-4 subscription needs the signal even ifRunStartednever fired. 19 Pest tests cover happy path (Pending→Complete + event order + MoE + duration/TPS/cost computation + chunk-count fallback + logprobs preservation +last_used_attouch + cost-null when snapshot lacks pricing), error paths (mid-stream throw with partial preserved, pre-stream key/client failures, vanished-API-key,RunErroreddispatch), Meta→Together fallback (both directions), and idempotency (re-running a non-Pending Run is a silent no-op). Chunk 5a edit: added per-chunk incremental UPDATE oftoken_log+output_textso the SSE fallback (/runs/{run}/stream) can read in-flight progress from the persisted row without a message broker between processes. One additional test (#20) uses a custom observer client that captures DB snapshots after each yield, proving the row'stoken_loggrows monotonically during the stream rather than only at terminal. - HTTP submission:
POST /threads/{thread}/runs(chunk 4a).App\Http\Requests\SubmitRunRequestdoes field-level validation (model_id exists, prompt required + non-empty, parameters within SPEC §3.1.4 bounds) so the response carries proper 422 per-field errors before the service ever runs.App\Http\Controllers\RunController::storedelegates orchestration toRunService::submitand translates each domain exception:ThreadOwnershipException→ 403,EmptyPromptException→ 422 onprompt,NoApiKeyException→ 422 onmodel_idwith vendor in message,InvalidParamsException→ 422 onparameters.{field},ContextOverflowException→ 422 onprompt. On success: persists the Pending Run, dispatchesStreamRunJob, returns 201 with{run: {...}, channel: 'private-runs.{id}'}so the frontend knows the channel to subscribe to. Route is throttled via the'runs'named limiter fromAppServiceProvider(live-readsusers.max_runs_per_hour);X-RateLimit-*headers come along for free. 16 Pest tests cover auth (401), authz (403 for non-owners), validation (7 fields + bounds), service exception mapping (no-API-key with vendor name in error, context overflow), success (run persisted + Bus-faked StreamRunJob dispatch + response shape + auto-title on first prompt + last_activity_at bump), Meta→Together fallback at the HTTP layer. - SSE fallback route:
GET /runs/{run}/stream(chunk 5a).App\Http\Controllers\StreamRunControllerreturnsSymfony\Component\HttpFoundation\StreamedResponsewithContent-Type: text/event-stream,X-Accel-Buffering: no(defeats nginx response buffering),Cache-Control: no-cache. Architecture: polling-based, not pub/sub. Since the queue-worker process runningStreamRunJobhas no shared memory with the PHP-FPM process serving this request, chunk 5a teachesStreamRunJobto writetoken_log+output_textincrementally on every chunk (additive to the existing terminal write — no new tests broken); the SSE controller then polls the run row every ~150 ms and emits the delta since the last cursor as SSE frames. Tail latency ~150 ms; no message broker required. Hard cap of 4000 iterations (~10 min) guards against an FPM worker hanging on a stuck run. Heartbeat comment every 200 iterations (~30 s) keeps proxies from closing the conn. Frame names match the WebSocketbroadcastAs()strings (run.started,token.received,layer.advanced,run.completed,run.errored) so the frontend SSE consumer (chunk 5b) shares the sameRunEventtype. Owner-only — same authz invariant asruns.{runId}channel auth. PHP-FPM caveat: long-lived SSE requests tie up FPM workers; M14 deployment chunk should sizepm.max_childrenwith this in mind. 11 Pest tests cover authz (redirect/403/404), response headers, SSE wire format, emission against already-Complete + already-Error runs, layer-count from model snapshot, partial-output preservation; tests usestreamedContent()(notgetContent()) to capture the streamed body. - Frontend hook:
useRunStream(runId)(chunks 4b + 5b) — prefers WebSocket via Laravel Echo, falls back to SSE on connection failure. Returns{ events, status, transport, disabled }wheretransport: 'websocket' | 'sse' | 'none'exposes which path is live,disabledis the back-compat alias fortransport === 'none'. Discriminated-unionRunEventtype inresources/js/types/runs.tsmirrors each PHP event'sbroadcastWith()exactly. Two-effect shape: effect 1 resets state when (runId, transport) changes; effect 2 sets up the subscription. Splitting ensuresevents: []lands before the new transport's first event regardless of React's scheduling. Transport selection: picks'websocket'ifwindow.Echois set, else'sse'ifwindow.EventSourceexists, else'none'. Fallback trigger (chunk 5b): subscribes to pusher'sconnection.state_changeand flips to SSE when state becomes'failed'or'unavailable'. Stays on SSE for the rest of the run even if WebSocket recovers — avoids transport thrashing and the dedup logic switching-back would require. Fallback UX (chunk 5b): clears events on transport change so the SSE controller's cursor-0 replay can repopulate without dedup. M8 can refine if the viz needs smoother UX. 21 Vitest tests cover WebSocket happy path, SSE-only path (Echo=null), WS→SSE fallback (failed/unavailablestate, event reset, EventSource ctor URL, stay-on-SSE after WS recovery),transport='none'when neither is available, runId transitions. MockEventSourceis implemented as a class (not a plainvi.fn()) sonew EventSource()works under jsdom (which doesn't ship EventSource). - Event names refactor (chunk 4b): added
broadcastAs()to each of the 6 events so the frontend listens with short kebab-case names (.run.started,.token.received,.layer.advanced,.moe.routed,.run.completed,.run.errored) instead of full PHP class FQNs. Decouples the JS subscription from the PHP namespace — renaming/moving event classes won't silently break the frontend. 6 Pest tests inRunEventBroadcastAsTest.phplock the strings down so a future drift fails the test before it ships. - Debug page
/runs/{id}/debug(chunks 4b + 5b):App\Http\Controllers\DebugRunController::showdoes the same-user-as-run check (mirrors theruns.{runId}channel auth invariant) and returns an Inertia render ofRuns/Debugwith the run's static metadata + the channel name to subscribe against. The TSX page renders the metadata header up top, a "subscribed channel + live status + active transport" line, and a chronological JSON event list below — append-only asuseRunStreamdelivers events. Active transport label readsWebSocket,SSE (fallback), orunavailable; chunk 5b also rewrites the disabled-notice copy to "no realtime transport is available" since SSE is now a fallback.config/inertia.phpadded so the Inertia view-finder knows about.tsxpage paths (defaults are Vue-only); without this,assertInertia(...)->component('...')fails with "page component file does not exist" on every test. 4 Pest tests cover authz (redirect/403/404 + 200 render with prop assertions); 8 Vitest tests cover render + event accumulation + status transitions + transport label (WebSocket / SSE / unavailable) + disabled-notice gate. - Reconnect logic (chunk 6):
GET /runs/{run}/events?since=NJSON backfill endpoint (App\Http\Controllers\RunEventsController) returns the persistedtoken_logslice from index N onward, plus currentstatus+completion/errorblocks for terminal runs. Lightweight one-shot — no streaming, no FPM-worker hold. Owner-only authz (same invariant as channel auth, SSE, and debug page).useRunStreamtracksmaxSeenIndex+wasDisconnectedrefs; on pusherstate_changetransitioning fromdisconnected/connectingback toconnected, fires the backfill withsince=maxSeenIndex+1and appends the returnedtoken_logentries as if they arrived live. Also synthesizesrun.completed/run.erroredevents from the response'scompletion/errorblocks so the closing event isn't missed when the run terminates during the gap. Pusher-js auto-reconnects WS but does not replay — that's the gap this closes. The SSE fallback (chunk 5b) is for hard failures; this is the catch-up path for transient blips. 11 Pest tests on the endpoint (authz, slicing semantics, since clamping, terminal-state blocks) + 6 Vitest tests on the hook (initial-connect no-fetch, since=N+1 URL, run.completed/run.errored synthesis, double-reconnect dedup, error-tolerance). - Queue worker config (chunk 6):
deployment/supervisor/laravel-queue.conf(2 worker processes, 600 s job timeout, 605 s stopwaitsecs so SIGTERM gives in-flight runs a clean exit) +deployment/supervisor/laravel-reverb.conf(single process — Reverb's in-memory pub/sub doesn't multi-process out of the box; horizontal scale needs the Redis backend, deferred to M14). Both.conffiles have install instructions in the header comment. For dev: foreground viaphp artisan queue:work+php artisan reverb:start --host=0.0.0.0.
Exit criteria
- ✅ Submitting a run results in tokens streaming to the debug page within ~200 ms of vendor delivery — covered by the chunk-3 StreamRunJob tests (broadcast dispatch on each chunk + microtime-based t_ms) plus the chunk-4b useRunStream / Debug page render tests. Manual verification recipe in
docs/m6-exit-criteria.md§1. - ✅ Killing the Reverb process mid-stream causes the frontend to fall back to SSE without dropping tokens — covered by chunk-5b's
WS → SSE fallbacktest group (transport flip on pusherfailed/unavailable, event-list reset, SSE EventSource opened against/runs/{id}/stream) + chunk-5a's incrementaltoken_logpersistence (no events lost because the SSE controller replays from the persisted log, not from an in-memory queue). Manual recipe indocs/m6-exit-criteria.md§2. - ✅ A second tab subscribing to the same
runs.{id}channel receives the same events — covered by Reverb's pubsub-layer semantics (multiple subscribers to a private channel fan out identically) + channel auth tests (chunk 4a) verifying the owner-only invariant. Manual recipe indocs/m6-exit-criteria.md§3.
M6 retrospective
Sized at 6 days; landed in 6 chunks across a similar elapsed window. Quality bar held — 432 Pest tests / 1030 assertions, 43 Vitest tests; Pint / ESLint / type-check / Vite build all green at every chunk boundary.
What worked
- Splitting chunk 4 into 4a (backend HTTP + channel auth) + 4b (frontend Echo wiring + debug page) kept each commit reviewable. Same for chunk 5 (5a backend SSE / 5b frontend fallback).
ShouldBroadcastNowoverShouldBroadcast(chunk 3): per-token broadcasts fire in-process from the queue worker, preserving order without a second queue round-trip. Caught early because the chunk-1 RunStarted event went out asShouldBroadcastand would have caused subtle reordering once token streaming kicked in.broadcastAs()short kebab names (chunk 4b) decouple frontend listeners from PHP class FQNs. Renaming an event class no longer silently breaks the wire contract.- Polling-based SSE (chunk 5a) avoided introducing Redis as a hard runtime dep — the persisted
token_logis the source of truth, both transports read from it. Tail latency is ~150 ms which is well under the "tokens stream in real time" perception threshold.
Surprises / footnotes
BROADCAST_CONNECTION=log(the default.envdriver) no-ops on the channel-auth callback — neededBROADCAST_CONNECTION=pusherwith fake test creds inphpunit.xmlto actually exercise/broadcasting/authin tests. Documented inline.- Inertia's view-finder defaults to
.vue; neededconfig/inertia.phpto teach it about.tsxpages orassertInertia(...)->component('...')fails on every test. assertInertiarequires the route to return aView(not the X-Inertia JSON form), so test HTTP calls don't pass the X-Inertia header.- jsdom doesn't ship
EventSource; chunk 5b needed a real ES6 class mock (not a plainvi.fn()) sonew EventSource()works. Symfony\StreamedResponse::getContent()returnsfalse; chunk-5a SSE tests usestreamedContent()to actually capture the body.- npm audit reports 3 moderate transitive
wsadvisories via socket.io-client (an optional laravel-echo transport we don't use). Unreachable in our pusher-js bundle; deferred to M14.
Decisions parked or deferred to later milestones
- Mid-stream WS↔SSE switch-back: once on SSE we stay there. Avoiding dedup complexity is worth the brief flicker. M8 can revisit if the viz needs smoother UX (chunk 5b documented this choice).
- Real cross-process pub/sub (Redis) for sub-100ms SSE: not needed for our concurrency target; polling-based SSE is good enough for launch.
- Horizontal Reverb scaling (Redis backend): single-process is fine for the single-VPS target; M14 deployment chunk will revisit if traffic warrants.
- Browser-based E2E test harness (Playwright/Cypress): chosen against in chunk 6 (unit + integration + manual recipes is enough for M6); M12 accessibility/polish is the natural home.
Carry-forward into M7
- The debug page proves the plumbing is alive end-to-end. M7's thread-detail page replaces it as the user-facing surface; the debug page stays for internal use.
useRunStreamis the only frontend consumer of the streaming pipeline today. M7 will refactor it into the thread-page hierarchy; the hook's contract (events + status + transport + disabled) should be enough.
Purpose: All non-animated UI is built and navigable: dashboard, thread list, thread detail, prompt input, model selector, parameter controls, API keys, settings, admin. The submit button works (creates a run) but the right pane just shows the debug log from M6.
Decisions (chunk 1):
- Component library: shadcn/ui (Radix + Tailwind, copy-paste pattern). Frontloads accessibility via Radix internals; gives us a consistent visual language for M7 + M8 + M12. Initial deps:
class-variance-authority,clsx,tailwind-merge,tailwindcss-animate,lucide-react,@radix-ui/react-slot. Components live inresources/js/Components/ui/. CSS-var-based theme tokens inresources/css/app.css(light + dark palettes);<html class="dark">makes dark the app default.components.jsonis committed for futurenpx shadcn@latest add ...calls. - Token counter: server-side AJAX via the existing
TokenCounterservices (from M3). Debounced POST endpoint per chunk 5; no client-side WASM, no bundle bloat. Slight perceptible lag during fast typing is acceptable for the M7 scope. - Onboarding: empty-state CTAs on each page + a "Set up your first API key" nudge on the dashboard when none exists. No dedicated wizard route.
- Split: 10 chunks (foundation → layout + 404 + responsive → dashboard → threads list → thread detail → prompt input → model selector → param controls → account/admin polish → closeout).
Tasks
- Foundation (chunk 1): shadcn/ui deps installed;
resources/js/lib/utils.tswithcn()helper;tailwind.config.jsextended with theme tokens that read from CSS vars (bg-background,text-foreground, etc.);resources/css/app.css:root+.darkblocks define the light + dark palettes;tailwindcss-animateplugin registered for Radix animations. 5 baseline UI primitives inresources/js/Components/ui/:button.tsx(6 variants × 4 sizes via CVA,asChildSlot pattern),card.tsx(6 subcomponents),input.tsx,label.tsx,separator.tsx(decorative + a11y modes). 22 Vitest tests cover render, variant classes,cn()override behavior (tailwind-merge resolves conflicts so callerh-20beats baseh-10), ref forwarding,asChildSlot, and Separator a11y modes. Lint exception: Label primitive disablesjsx-a11y/label-has-associated-controlinline — design-system primitives can't statically prove association; that's the caller's job (viahtmlFor/id). - Layout shell (chunk 2):
AppLayoutrefactored onto shadcn primitives — sidebar uses Button + Separator, nav links use theme tokens (bg-accent/text-accent-foregroundfor active state) instead of hardcoded slate-*. Desktop ≥ md: 240px fixed sidebar. Sub-md: sidebar hides, top bar shows a hamburger Button that opens the sidebar in a left-anchored Sheet (Radix Dialog wrapper; new primitive inresources/js/Components/ui/sheet.tsxwithsidevariant + 4 Vitest tests). SameNAV_ITEMSarray drives both desktop + mobile — single source of truth. Optional<title>prop slot in the top bar; falls back to the active nav item's label. SheetHeader/SheetTitle/SheetDescription kept insr-onlywrapper to satisfy Radix's a11y requirement without visual noise. - Dashboard page (chunk 3):
App\Http\Controllers\DashboardController::indexaggregates the signed-in user's stats (total run count, sum of input + output tokens across complete runs, sum of estimated_cost across complete runs) + 5 most-recent threads ordered bylast_activity_atdesc withwithCount('runs')+has_api_keysboolean. Per-user isolation enforced viawhere('user_id', $user->id)on every query — covered by a cross-user-leakage Pest test. Token / cost sums skip errored / streaming / pending runs (their counts may be incomplete, their cost is null). React side (resources/js/Pages/Dashboard.tsx) refactored onto shadcn Card primitives: 3 stat cards in anmd:grid-cols-3grid, recent-threads section with relative timestamps ("3h ago", "2d ago"),NoApiKeyCalloutat the top whenhas_api_keys=falselinking to/api-keys,EmptyThreadsstate in the recent section linking to either/api-keysor/threadsdepending on whether a key exists. Currency formatting viaIntl.NumberFormat. Per-thread links deferred to chunk 5 — recent threads display as cards but don't link individually yet (the/threads/{id}route lands then). "View all" link in the section header points at/threads(still the chunk-4-bound placeholder). 10 Pest tests (auth, prop shape, aggregation, errored-run exclusion, cross-user isolation, recent-thread ordering + count + isolation, has_api_keys flag both states); 8 Vitest tests (3 stat cards present, number-format, no-API-key callout visibility + link target, empty-threads CTA + dependency on has_api_keys, recent threads render with relative time + archived hint). Test gotcha: JSON has no float-vs-int distinction;0.0round-trips as int0, so the zero-cost assertion uses0, not0.0. - Threads list page (chunk 4):
App\Http\Controllers\ThreadController::indexreturns a 20-per-page paginated list scoped to the signed-in user withwithCount('runs'), ordered bylast_activity_at desc. Filters:?q=(case-insensitive titleLIKE),?archived=false|true|all(defaultfalse),?tag=(single tag viawhereJsonContains). Also returnsavailable_tags— a sorted unique list derived from the user's threads — for the filter dropdown.storeaction creates an empty thread and redirects tothreads.show(a ComingSoon placeholder until chunk 5). Archive/share/delete actions deferred to chunk 5 since they're per-thread concerns. React page (Pages/Threads/Index.tsx) uses shadcn Card/Button/Input + a native<select>for the tag filter. 300 ms debounced search viasetTimeout+router.get(..., { preserveScroll, preserveState, replace }). Archive-toggle is a 3-button group (Active / All / Archived) witharia-selected. Two empty states (empty-allvsempty-filtered);Clear filtersonly renders when any filter is non-default. "New thread" button posts to/threadsviauseForm. Pagination is a Previous/Next bar with explicit hrefs preserving filters. Lint gotcha: the React Compiler /react-hooks/immutabilityrule needs the search handler wrapped inuseCallbackAND declared before theuseEffectthat references it. 17 Pest tests (auth, prop shape, ordering, isolation, run_count, search trimming, all 3 archive states, tag filter + isolation,available_tagssort/uniqueness, pagination math +?page, store flow); 11 Vitest tests (both empty states, card rendering + per-thread link, debounced search via fake timers, archive toggle reflects + dispatches, tag dropdown shows only when tags exist, clear button gates correctly, create-thread POSTs, pagination only when needed). Vitest gotcha:vi.mockis hoisted above top-level consts, so the spy fns (routerGet,formPost) live insidevi.hoisted(...)to be available to the mock factory. - Thread detail page (chunk 5):
App\Http\Controllers\ThreadController::show / update / destroyplus their routes.showreturns the thread, its runs insequence_in_threadorder,usable_models(vendors the user has an API key for, with the Meta→Together fallback fromRunService::resolveApiKeyhonored), andhas_api_keysfor the empty-state copy.updatevalidatestitle(nullable, max 200) andarchived(bool);destroycascade-deletes runs via the FK. All three abort_unless on ownership.Pages/Threads/Show.tsxrenders: header with inline-editable title + archive toggle + delete-with-confirm (Radix AlertDialog), read-only transcript of run cards (status badge + prompt + output + token/duration/cost stats; error message for errored runs), and a bare prompt input footer (Textarea + native<select>grouped by vendor + Submit posting to the existing/threads/{thread}/runsroute). 2 empty-state branches: no API key → CTA to /api-keys; key but no usable models → registry-refresh hint. Chunks 7–8 enrich the prompt input footer with autosize / token-count / context-warning / fancy model picker / parameter controls. Chunk 6 wires the live-stream view (useRunStream) into a right pane so submitted tokens visibly stream into the page. New primitives:Textarea+AlertDialog(Radix-backed; 4 + 4 Vitest tests). 17 Pest tests (auth, ownership, render shape, run ordering, isolation, vendor filtering + Meta→Together, title update + null + max-length validation, archive toggle, delete + cascade verification); 16 Vitest tests (title + archive + delete header actions including confirm/cancel paths, transcript empty state + per-run rendering, footer state machine across all 3 branches, vendor-grouped optgroups, Submit gated by empty prompt). Lint exceptions: the title-edit Input usesautoFocusafter an explicit click (legitimate UX), andreact/no-unescaped-entitiesflagged a literal apostrophe in copy. - Prompt input panel (chunks 6a + 6b):
- Multi-line textarea with autosize (chunk 6a) —
useAutosizeTextareahook drivesstyle.heightoffscrollHeighton every change, clamped to 480px. No JS dependency. - Conversation history above (chunk 6a) — collapsible
HistoryPreviewpanel renders the same OpenAI-shape array thatRunService::submitwill send (system + user/assistant pairs from prior complete runs). Closed by default. - [~] Chat-template preview pane (toggleable) — partial via the history preview. Vendor-specific template renderers (jinja2-ish strings stored in
models.chat_template) defer to a future polish chunk; the history preview already shows what context the model will see. - Token-count preview (chunk 6a) — server-side via
POST /threads/{thread}/preview(App\Http\Controllers\PromptPreviewController) returning{history, token_counts: {history, prompt, reserved, total}, budget, fits, over_by, model}. Counts use the sameTokenCounterFactorythatContextBudgetCalculatoruses (exact tiktoken for OpenAI, BPE-approximate elsewhere) so server + frontend display stay aligned. Debounced 400ms client-side viauseDebouncedPreviewwithAbortControllercancellation on rapid input changes. No client-side WASM, no bundle bloat (per the chunk-1 decision). - Approaching-context-limit warning at 80% (chunk 6a) —
BudgetIndicatorrenders a horizontal bar that's primary at < 80%, amber at 80%–100%, destructive at > 100% (with theover_bycount next to it). Submit button is gated off the samefitsflag — you can't post a run that the server-sideContextBudgetCalculatorwill reject. Models withcontext_length=nullget a plain "N tokens" readout (no bar). 16 Pest tests + 8 new Vitest tests. - Live debug pane (chunk 6b) —
LiveStreamPanecomponent onShow.tsxsubscribes to the currently-active run viauseRunStream(from M6, WebSocket → SSE fallback included).findActiveRunpicks the latest non-terminal run from therunsprop. Layout switches to alg:grid-cols-3two-column at desktop (transcript + form on cols 1–2, live pane sticky on col 3); single-column stack at mobile. Shows event JSON-per-line, status badge, active transport label. Auto-reloads therunsprop viarouter.reload({only: ['runs']})400ms after a terminal event so the transcript picks up the finaloutput_text+ token counts and the live pane returns to its empty state. Dual-respond fix onRunController::store: returns an Inertia redirect tothreads.showwhenX-Inertia: trueis present (souseForm.posttriggers the page-reload that surfaces the new run); keeps the chunk-4a 201 + JSON shape forpostJson()/ future API callers. 2 new Pest tests cover both branches; 8 new Vitest tests cover empty state, active-run detection (incl. ignoring terminal runs), event rendering, transport label + unavailable notice, router.reload triggered for both complete + errored terminal states. Satisfies the M7 exit criterion "see tokens stream into the debug pane on the right." M8 will replace this debug-style pane with the real visualization.
- Multi-line textarea with autosize (chunk 6a) —
- Model selector — combobox (chunk 7).
ThreadController::usableModelspayload extended witharchitecture_type,position_encoding,layers,hidden_dim,attention_heads,moe_experts,moe_active_experts, pricing pair. Two new shadcn primitives:Popover(Radix) +Command(cmdk-backed; keyboard-first list with fuzzy match overdisplay_name + name).ModelPickercomponent: trigger Button shows the currently-selected model name, opens a Popover with arch tabs (Dense / All / MoE), vendor multi-chip filter (hidden when only one vendor is on file), and a Command list grouped by vendor with a context-length chip on each row.ModelMetadataCard: two-column key/value grid showing the selected model's arch (with8 experts (2 active)for MoE), context (128K tokensetc.), layers, hidden_dim, attention_heads, position encoding, pricing pair ($X.YZ / M tokens). Size filter dropped (chunk-7 decision) — we don't store parameter count and the name-regex heuristic is too brittle; arch + vendor + context-length sort + search covers the use case. Vitest infra additions: jsdom polyfills forResizeObserverandElement.prototype.scrollIntoViewintest/setup.tssince cmdk uses both. 1 new Pest test on the extended payload; 7 new Vitest tests on the picker (initial metadata card render, MoE-aware metadata format, popover open on click, arch + vendor filter chips visibility, MoE-filtered list, selection triggersform.setData('model_id', ...), vendor chips hidden when one vendor). - Inference parameter controls (chunk 8).
ThreadController::usableModelspayload extended withsupported_params(the LlmModel JSON map of param-name → bool). New shadcnSliderprimitive (Radix-backed; 3 Vitest tests cover ARIA role/value, disableddata-disabledmarker, prop wiring).ParameterControlscomponent: collapsible Card (closed by default; chevron toggle in header; primary-colored "custom" dot when any value is non-default). Five rows —temperature(0–2 step 0.05),top_p(0–1 step 0.05),top_k(0–500),max_tokens(1 to model'scontext_length),seed(null = random, input-only). Each numeric row pairs a Slider + small Input bound to the same value; input clamps to the slider's min/max on the way in. Unsupported params (persupported_params[key] === false) get a "(not supported)" hint and disabled Slider + Input. Missing/nullsupported_paramstreated as "all supported" (defensive against older registry rows). "Reset to defaults" link sends the full default object.Show.tsxPromptForm wires the controls intoform.data.parameters;useDebouncedPreviewnow keys offmax_tokensso the chunk-6a budget bar updates when the user adjusts the reserve. 1 new Pest assertion on the extended payload; 13 new Vitest tests onParameterControls(collapsed default, expand on toggle, custom-dot toggling, input dispatch, clamp on out-of-range, unsupported hint + disabled state, Reset link payload,max_tokensceiling honored, seed empty/Random/randomize paths, nullsupported_paramsfallback). - API Keys page (chunk 9a polish; original behavior shipped in M4 chunk 1). Refactored onto shadcn:
Card/CardHeader/CardContentwrapper,Button/Input/Labelprimitives, theme tokens replace hardcoded slate-*.confirm()JS dialog swapped for theAlertDialogconfirm flow (one page-level dialog driven bypendingDeletestate, no per-row dialog instances). Existing M4 backend tests stay green (encrypted-key storage, vendor allowlist, masked display). 8 new Vitest tests cover render + empty state + per-key rows + Add-form POST + delete-with-confirm (open/confirm/cancel paths) + vendor dropdown options. - Settings page (chunk 9a polish). Refactored onto shadcn
Card/Button/Label. Singlestore_promptstoggle (display name lands when the SPEC user-profile column gets added; not a blocker for M7 exit). Existing M3-era backend tests stay green. 4 new Vitest tests cover render (both checked + unchecked init), PATCH dispatch on submit, and checkbox toggling. - Admin pages (chunk 9b polish; original behavior shipped in M2/M3). Admin Users (
Pages/Admin/Users.tsx): user list table + inline rate-limit editor refactored ontoCard/Input/Button+ theme tokens. Admin Models index (Pages/Admin/Models/Index.tsx): filter form + paginated registry table; refresh-from-OpenRouter and per-row delete now useAlertDialogconfirms instead of nativeconfirm(). Admin Models edit (Pages/Admin/Models/Edit.tsx): the heaviest single form (identity / architecture / capacity + pricing / capabilities / chat template / refresh control sections); all rows refactored ontoInput/Label/Textarea/ButtonwithSection/Fieldinternal helpers preserved. Pagination styling moved to theme tokens (bg-accentfor active,cursor-not-allowedon disabled links). Existing M3 admin-controller Pest tests stay green. 12 new Vitest tests (4 Users + 5 Models Index + 3 Models Edit) cover render shape, per-row dispatch, refresh + delete confirm flows, filter form submit, and themetadata_estimatednotice. - 404 / error pages (chunk 2): 4 Inertia pages in
resources/js/Pages/Errors/(NotFound, Forbidden, Expired, ServerError) all built on a sharedErrorShellcomponent. Standalone — NOT wrapped inAppLayoutsince unauthenticated users hit them too.usePage().props.auth.userdetermines whether the CTA reads "Back to dashboard" or "Sign in".bootstrap/app.php'swithExceptionscallback maps 403/404/419/500/503 to the matching component and renders via Inertia whenAPP_DEBUG=false; debug-mode dev keeps the stock Laravel whoops/error pages with stack traces. JSON requests bypass the Inertia render and get the default JSON error response (so XHR callers aren't broken). 6 Pest tests cover the 404/403/500 mappings, authed + unauthed paths, and the JSON-bypass. - Responsive: desktop + tablet breakpoints (chunk 2). The sidebar↔drawer breakpoint is
md(768px); below that the layout collapses to a single column with the hamburger Sheet. Verified on a phone (192.168.0.205:8001) via the dev-login magic-link flow.
Exit criteria
- ✅ All pages render and route correctly. Covered by Pest controller tests on every route (
DashboardControllerTest,ThreadIndexTest/ThreadShowTest,PromptPreviewTest,ApiKeysControllerTest,SettingsControllerTest,ErrorPagesTest,Admin/ModelsControllerTest,Admin/UsersControllerTest,AuthGatedRoutesTest) + Vitest render tests on every page (Welcome,Dashboard,Threads/Index,Threads/Show,ApiKeys/Index,Settings,Admin/Users,Admin/Models/Index+Edit,Runs/Debug, the 4Errors/*pages, plus the underlyingComponents/ui/*primitive tests). Manual recipe indocs/m7-exit-criteria.md§1. - ✅ A logged-in user can: create a thread, fill out a prompt, select a model, set parameters, submit → see tokens stream into the debug pane on the right. Covered by
ThreadController::storetest (creates Pending run + redirects tothreads.show), the chunk-7 ModelPicker Vitest (selection updatesform.data.model_id), the chunk-8 ParameterControls Vitest (slider + input dispatch intoform.data.parameters), the chunk-6a PromptPreviewController + budget-indicator Vitest (token-count + over-budget submit lock), the chunk-6bRunController::storedual-respond (Inertia redirect path), and the chunk-6b LiveStreamPane Vitest (active-run detection +useRunStreamsubscription + auto-reload on terminal status). The chunk-6/M6 streaming pipeline (StreamRunJob→ broadcast events → useRunStream) carries the actual token-by-token delivery. Manual recipe indocs/m7-exit-criteria.md§2. ⚠️ Lighthouse score ≥ 90 on the dashboard. Not unit-testable from this repo — requires a real browser. Manual recipe indocs/m7-exit-criteria.md§3 documents the Chrome DevTools Lighthouse run. The dashboard is server-rendered Inertia with minimal client-side work; shadcn primitives carry Radix accessibility defaults (focus rings, ARIA roles, sr-only labels on the mobile-nav Sheet). Treat any sub-90 finding as M12 (Accessibility + Polish) work rather than re-opening M7.
M7 retrospective
Sized at 7 days; landed in 10 chunks (1–5, 7, 8, 10 single; chunks 6 + 9 split into a/b) across a similar elapsed window. Quality bar held at every chunk boundary — 512 Pest tests / 1512 assertions, 181 Vitest tests across 24 files; Pint / ESLint / type-check / Vite build all green at every commit.
What worked
- shadcn/ui adoption (chunk 1) frontloaded accessibility. Every primitive (
Button,Card,Input,Label,Separator,Sheet,AlertDialog,Popover,Command,Slider,Textarea) wraps a Radix internal that handles focus return, escape-to-close, scroll lock, and ARIA wiring. We didn't have to think about it page-by-page. - Theme tokens over hardcoded colors. Moving the palette to CSS vars (
:root+.darkinapp.css) means a single source of truth. Chunks 2 + 9 surfaced the cost of NOT doing this — every hardcodedslate-*in the M1–M5 pages had to be swept. - 10-chunk split with sub-chunks (6a/6b, 9a/9b). Each commit was reviewable in under ~1k lines of diff. Chunk 6's six features (autosize / history / template / token count / warning / live pane) would have been unreviewable as one.
- Server-side AJAX token-count preview (chunk 6a). Skipping client-side
tiktoken-jsWASM saved ~500KB from the bundle. The 400ms debounce + reuse ofTokenCounterFactorykeeps server + client in lockstep so the UI can't show a fits/over-by reading that disagrees with whatRunService::submitwould reject. - Primitives compounded.
AlertDialogfrom chunk 2 powered chunk-5's thread delete, chunk-9a's API-key delete, AND chunk-9b's two admin confirms (refresh + delete). Each later use was a 10-line copy of the pattern, not a new dialog implementation. - Component-library tests catch contract drift early. When chunk 7 extended
usable_modelswithsupported_params, the Vitest fixtures broke at the TypeScript level — caught at type-check, not at runtime.
Surprises / footnotes
- Each new Inertia page needs a Vite rebuild before
assertInertiaworks.Pages/Threads/Show.tsx,Pages/Errors/*,Pages/Threads/Index.tsxall hit it. Pattern: write the route + controller + tests, expect a fail, build the page TSX, runnpm run build, re-run Pest. Could be automated; not worth it for the cadence we're at. - jsdom polyfill churn. Chunk 4b needed an
EventSourceclass polyfill (the standardvi.fn()isn't constructable). Chunk 7 neededResizeObserver+Element.prototype.scrollIntoViewfor cmdk. All live inresources/js/test/setup.tsnow. vi.mockhoisting +vi.hoisted(). Mock factories run before top-level consts. Mock-state objects (mockStreamState.value, spy fns shared across tests) must be hoisted withvi.hoisted(). Bit me on chunk 4b, chunk 7, AND chunk 6b before I internalized the pattern.- JSON int/float round-trip. PHP encodes
0.0as"0"; Inertia ships JSON; assertions came back as int. Bit chunk 3 (dashboard cost) and chunk 7 (model pricing). Documented the workaround in both tests. - Inertia
useForm.postwants a redirect, not 201+JSON. Chunk 4a builtRunController::storeas a JSON API endpoint; chunk 6b had to add a dual-respond shim (X-Inertia header → redirect; otherwise → JSON). Worth doing the dual-respond from day one on any new POST endpoint. react-hooks/immutabilityflags legitimate setState-in-effect patterns (debounce hooks, transport-fallback in useRunStream, the parameter-controls reset). The documented workaround is// eslint-disable-next-line react-hooks/set-state-in-effectwith a rationale comment. Used sparingly — every disable carries a why.- Inertia view-finder is Vue-by-default. Chunk 4b's
assertInertia(...)->component('...')failed untilconfig/inertia.phpdeclared the.tsxpage paths. Recurs from M6 — should have been a foundational change in chunk 1. react/no-unknown-propertyflags cmdk'scmdk-input-wrapper=""sentinel (chunk 7). Opaque styling hook for cmdk's CSS, not a real DOM attr. Disable-with-rationale.jsx-a11y/label-has-associated-controlcan't see runtimehtmlForassociation on design-system primitives (theLabelcomponent inui/). Inline disable on the primitive definition; callers pair viahtmlForand the rule is happy.
Decisions parked / deferred
- Size filter on the ModelPicker (chunk 7). Dropped: we don't store parameter count, and a name-regex heuristic is too brittle. Arch + vendor + context-length sort + cmdk fuzzy search covers the use case. A future
parameter_countcolumn onmodelscould resurrect the filter — file under M12 polish or a registry-schema chunk. - Full vendor-specific chat-template preview (chunk 6a). Partial via the history preview (shows the role/content array the model sees). Jinja-style template rendering using the
models.chat_templatecolumn is deferred — useful for Anthropic/HF tinkerers, low priority for OpenAI-protocol vendors that handle templates server-side. - Display name on Settings (chunk 9a). SPEC mentions it; the
userstable lacks a separatedisplay_namecolumn (it hasname, which the OAuth flow populates). Adding one would be a one-line migration + form field, but not a blocker; left for M12 / M15 polish. - Real Lighthouse-in-CI gating. Out of scope for M7 — would need a headless Chrome + Lighthouse-CI npm package + a separate workflow step. M14 deployment is a more natural fit if we want a CI gate; M12 polish would handle one-off audits.
- E2E browser test harness (Playwright/Cypress). Same call as M6's closeout — unit + integration + manual recipes is enough. M12 might add Playwright for a11y audits.
Carry-forward into M8
LiveStreamPaneis the placeholder. M8 replaces its JSON-per-line<pre>block with the real visualization: 3D transformer stack (Three.js), attention heatmaps (D3.js), MoE routing graph, logits panel, embedding-space token flow. Theeventsarray fromuseRunStreamis the data feed; the shape is stable (per-event discriminated union fromresources/js/types/runs.ts).router.reload({only: ['runs']})after a terminal status (chunk 6b) refreshes the transcript with the final output. M8 might want to keep the live pane in some form (debug toggle?) since the chunk-7 + chunk-8 controls keep producing valuable streaming detail (per-token logprobs, MoE expert IDs).useDebouncedPreview(chunk 6a) keys off(threadId, prompt, modelId, max_tokens). M8 might add other params if the viz needs context (e.g., a "preview the viz given these settings" panel). Today's deps are sufficient for the budget bar.- All page chrome is shadcn-themed. M8's viz-controls (play/pause/scrub, layer-focus, attention-head-select) should use the same primitives (
Slider,Button,Card,Popover) so the visual language stays coherent. - Per-thread links from the dashboard's recent-threads section (chunk 3) still don't link to
/threads/{id}— chunk 3 deferred this to chunk 5, but I never went back to wire it. Quick follow-up: thread cards on the dashboard should link to their detail page. TODO for M8 or a polish pass.
M7 closed: 2026-05-19. Chunks 1–10 all green. 512 Pest / 1512 assertions, 181 Vitest across 24 files; build at 489 KB / 157 KB gzipped.
Purpose: The right pane shows the real visualization, not the debug log. This is the vertical-slice gate.
Tasks Decisions (chunks 1–9):
- Lazy-load Three.js via
React.lazy+Suspense— keeps the ~600KB Three.js bundle out of the main app chunk so users without an in-flight run never download it. Vite splitsVizPaneinto its own ~521KB chunk (132KB gzipped) as observed on chunk-1 build output. - Keep the chunk-6b debug pane behind a viewer toggle. Viz is the default; "Debug" tab swaps in the chunk-6b LiveStreamPane. Useful while M8 lands one chunk at a time — if a viz piece regresses, the debug fallback confirms events ARE arriving.
- Lift the
useRunStreamsubscription up toThreadShow. Both views share one Echo channel; the toggle just swaps the renderer. Single source of truth for events. - Split into 9 chunks (1: foundation; 2: transformer stack; 3: token-flow particles; 4: live text + cost/TPS + KV bar; 5: D3 attention + logits; 6: MoE routing; 7: embedding scatter; 8: playback controls; 9: vertical-slice E2E + closeout). Chunk 2 absorbed what was originally going to be chunks 2 + 3 (stack + cascade) because the cascade animation has no value without the stack and the stack has no animation without the cascade — splitting felt like a contrived seam.
- Cascade waves overlap; no queuing. The SPEC calls for layers lighting up sequentially per token. Implemented as 600ms waves walking bottom → top with no per-wave back-pressure. At 30 tok/s peak, ~18 concurrent waves visible — reads as a scrolling band, which is more legible than a single bouncing head trying to keep up.
- HTML overlay (not in-scene Three.js text) for layer detail. WebGL text is expensive and inaccessible to screen readers. The overlay is a positioned
<div>with an aria-labeled region and a real close button. Trade-off: it visually breaks the 3D illusion slightly, but the readability + a11y win is worth it. total_layers+architecture_typecome from the run'smodel_snapshot, not the liveLlmModelrow. Per SPEC §10.1, model rows can be deleted / re-keyed; the snapshot is the canonical record at run time. The dual-source pattern matters once admins start refreshing the registry.'active'(indigo) is for cascade;'selected'(amber) is for the focused layer. Disjoint colors so the selection state survives a wave passing through it visually — the cascade just temporarily over-paints, then the selection re-paints on the next state-change effect.- Particles are streaks, not glowing spheres. (chunk 3) User choice — streaks read as "lightning" / flowing electricity, more cinematic than dots. Implemented as elongated boxes with a vertex-color gradient + additive blending rather than per-instance shader work; comet-tail effect comes from geometry, not GPU code.
- Burst per token (5–10 particles), not 1:1. (chunk 3) Looks like activation flowing rather than a single moving dot. Density picked to read at 30 tok/s without saturating the 256-particle pool (worst case ~1500 active particles if all bursts overlap → pool cap is the visual ceiling, not a correctness requirement).
- Straight-column trajectory + random X/Z jitter, not helix. (chunk 3) Spirals occluded the slabs they were passing — important since the slabs are the primary signal and particles are secondary. Jitter (±0.6 on each axis) gives the burst dispersion without overlapping the layer geometry.
- Fade via scale ramp, not custom shader for per-instance alpha. (chunk 3)
InstancedMeshhas no built-in per-instance opacity uniform. Shrinking scale 1.0 → 0.0 over the last 800ms achieves the same visual outcome without forking the shader. If we ever need true per-particle alpha (e.g. logprob-driven brightness), revisit with a custom ShaderMaterial. - Live text inline on the active run row, not a separate "Live" card. (chunk 4) User choice — keeps the conversation flow uninterrupted; the row transitions seamlessly from streaming-with-cursor to static-with-final-tokens via the 400ms terminal-reload that already exists. Implementation cost is equal either way, but the inline approach avoids a phantom card popping in/out between runs.
- Cumulative TPS, no rolling window. (chunk 4) User choice — smoother number, no per-event timestamp buffer to maintain, derivation stays a pure function of
(events, run, model)with no ticker required. The displayed value updates only on new tokens, which is fine because cumulative-by-definition doesn't change between events. - Derive TPS from
t_msof the latest token, not wall clock. (chunk 4) Wall-clock would need a 250mssetIntervalre-render. Usingt_ms(server-set per event, stable between renders) keeps the row pure and avoids extra render churn. Trade-off: if WebSocket dies mid-stream the TPS freezes rather than continuing to drift; we'd rather show a frozen-but-real number than a wall-clock that lies about throughput. - Pricing is per-field-nullable. (chunk 4)
pricing_input_per_millionandpricing_output_per_millionare independently optional — the cost calculation skips the null side rather than refusing to compute. A model with only output pricing still shows an output-only cost (cheaper than nothing). The entire cost row hides when BOTH are null, which is the only case where "$0.0000" would mislead. - Context bar shares the M7 budget-bar visual language. (chunk 4) Same color thresholds (amber at 80%, destructive at 100%), same h-1 /
bg-mutedtrack. The bar is the same widget pre-submit (M7 chunk 6aBudgetIndicator) and during-stream (chunk 4LiveMetricsStrip) — users only learn one chart. - Attention heatmap is synthetic, labeled "Illustrative", and tied to layer depth. (chunk 5a) Vendor APIs ship logprobs but not attention tensors, and we'd rather show a pedagogically honest causal-decay pattern with a "vendor APIs don't expose attention" caveat than fabricate something that pretends to be real. Decay constant widens with
layerIndex / totalLayersso the heatmap visibly differs across layers — that's what makes click-to-zoom worth doing. - Heatmap lives inside the layer-detail overlay, not as a top-level card. (chunk 5a) User choice. Keeps "all per-layer detail in one place" — the sub-component list + heatmap together. Avoids a phantom card cluttering the layout when no layer is selected. Trade-off: invisible by default; user must click a slab to see attention. Acceptable because the slab cascade already invites click interaction.
- Logits chart hides entirely when logprobs are null, no placeholder. (chunk 5b) User choice. The fact that a vendor doesn't expose alternatives is itself information; a "no logprobs from this vendor" card felt like clutter. If we ever need the disclosure for support reasons we can add a single inline footnote on the live row.
- Logits live below the live text on the active run, not in the viz pane. (chunk 5b) User choice. The chart answers "what alternatives did the model consider for the latest token?" — most useful next to the token text itself. Embedding it in the viz pane would have required a third tab and reduced its conceptual link to the streaming text.
- Logprob → probability via per-K normalization, not softmax over full vocab. (chunk 5b) The vendor only returns the top alternatives; renormalizing across the truncated set gives bars that sum to 100% and answer "weight among the reported alternatives" honestly. Pretending the un-returned tail is zero would over-state the chosen token's confidence.
- Whitespace tokens display via
JSON.stringify. (chunk 5b) Most top-K alternatives at the start of a response are" ","\n", or"."— rendering them raw would either collapse them (CSS whitespace) or insert real line breaks.JSON.stringifyshows the literal escape ("\n") so the user actually sees what the model considered. - MoE routing co-locates with logits on the live run row. (chunk 6) User choice. Both views answer "what did the model just do for this token?", so they belong next to each other. Layer-detail overlay would be technically purer (MoE lives inside FFN of each layer) but the user can't see it without clicking, and per-token signal demands persistent visibility.
- MoE router scores renormalize to 100%, same as logits. (chunk 6) User choice. Mirrors the chunk-5b decision — the vendor's per-expert score scale varies (some report post-softmax
[0, 1], some report raw logits), so per-K normalization gives a uniform reading across vendors. Raw scores are still available on theMoERoutingExpert.rawScorefield in case a later view wants them. - Utilization is one bar per expert, not a heatmap or text list. (chunk 6) User choice. Bars give the long-tail shape at a glance (a sharp concentration looks like a spike, dispersed routing looks flat). Tested explicitly to handle pools up to 160 experts (DeepSeek-V2) via flex-distributed widths — the grid stretches to fill, bar widths shrink, the visual story stays legible.
- Bar-height normalization uses the per-run max, not a fixed scale. (chunk 6) An expert pool with sharp concentration would otherwise show a single visible bar and many invisible ones at a fixed scale. Per-max normalization preserves the comparative shape. Trade-off: a uniform-routing pattern looks the same shape as a concentrated one until the user reads the activation count. Tooltips on hover expose the raw count.
- MoE component is mounted by the parent, not self-gated. (chunk 6) The component still tolerates non-MoE event streams as a safety net (returns null when no
moe.routedevents) but the actual gate ismodel.architecture_type === 'moe'inLiveRunBody. Keeps the component reusable (could appear elsewhere with different gating) and explicit in the parent about when it appears. - Embedding scatter uses synthetic clusters, not real PCA. (chunk 7) User choice. The SPEC called for "PCA-reduced 3D scatter (precomputed per tokenizer)" but real PCA requires running offline against ~500MB-1GB of HuggingFace model weights per tokenizer family. Stylized synthetic clusters give the same visual story (token types form spatial groups) without the data wrangling. The illustrative honesty comes from cluster labels and the "synthetic vocab map" footer caption. Real PCA can be a future swap-in — only the vocab data file changes.
- Embeddings get a third top-level tab, not a sub-mode. (chunk 7) User choice. Considered putting Embeddings as a stack/embeddings camera-preset switch inside the Viz tab, but the scene infrastructure is different enough (Points cloud + spotlight vs slabs + cascade + streaks) that conflating them would have meant rebuilding the scene on toggle. Separate tab keeps the scenes independent and lazy-loadable.
- Reduced-motion only disables the Viz tab, not Embeddings. (chunk 7) The transformer-stack scene auto-rotates continuously and runs the cascade animation — clearly motion-heavy. The embedding scene only auto-orbits when no spotlight is active, and the trail update is per-event (not per-frame). User can still benefit from the spatial view without continuous motion.
- Token-to-vocab-point resolution falls back exact → trim → lowercase → trim+lowercase. (chunk 7) Most vocab tokens come from a tokenizer with leading-space variants (
" the"vs"the"); some streams emit ALL-CAPS or Title-Case. The fallback chain catches the common variants. Misses are silently dropped — the scatter visibly skips them rather than fabricating a position for unknown tokens. - Per-vertex alpha via custom ShaderMaterial, not per-mesh InstancedMesh. (chunk 7)
THREE.Pointswith a customShaderMaterialallows per-vertex alpha through a vertex-shader attribute; 280 points + alpha attribute is a single draw call.InstancedMeshwould have been heavier (28 transformations per instance vs one attribute write per visited token) and harder to update incrementally without rebuilding the matrices. - Vocab lookup is a module-level singleton. (chunk 7)
buildEmbeddingLookup()is invoked once per page load and cached in a module-scopedlet. The vocab data is static, so rebuilding on every render would be pure waste. The trade-off (a singleton holds memory for the whole page) is fine for ~280 entries (~10KB). - Playback engine is shared across live + (future) M9 replay. (chunk 8) The hook treats
eventsas opaque — same code path works for a liveuseRunStreamarray AND a replay array hydrated fromruns.token_log. M9 just hands it a different events source and gets the same controls for free. - 1× = LIVE = head-sync, not a fixed-rate dispenser. (chunk 8) User choice. Means viz keeps perfect real-time fidelity at 1× (no delayed-by-buffering drift) while still allowing artificial throttle at 0.5× and rapid drain at 2× / 4×. Trade-off: 1× and 2×+ have qualitatively different mechanisms — the hook special-cases speed===1 with a simple sync useEffect. Single-mechanism would be cleaner but produces noticeably-delayed playback at 1× which the user didn't want.
- Pause/step/speed affect ALL consumers, including the live Assistant text on the run row. (chunk 8) The transcript's live text reads from
playback.visibleEvents, not raw stream events. Pausing freezes the rendered text mid-stream so the user can read what's there without it scrolling under them. Some users might expect pause to only affect viz, but uniform behavior is the simpler mental model — pause = "freeze this moment" everywhere. - Step skips to next
token.received, not next event. (chunk 8) User choice. SPEC says "advance one token" and most users care about token granularity, not the cascade/MoE intermediate events. Side effect: stepping near the end of a stream can become a no-op if the only remaining events are non-token (e.g., finalrun.completed). That's acceptable —jumpToLivecovers the "I want to see the rest" case. - Stream-shrink auto-resumes LIVE. (chunk 8) When the events array shrinks (new run starts, or SSE replays from cursor 0), the hook detects via
prevLengthRefand resets cursor + speed=1 + playing=true. Avoids the surprising "I paused on the previous run and now this run is invisible" case. A future "remember playback state per run" option could opt out but isn't worth the complexity now.
Tasks
- Three.js setup: scene, camera, renderer, OrbitControls (chunk 1).
resources/js/Components/Viz/VizPane.tsx(lazy-loaded viaReact.lazyfromThreads/Show) mounts aWebGLRendereron a canvas ref, sets up aPerspectiveCamera, two lights (ambient + directional key),OrbitControlswith damping. AResizeObserverkeeps the renderer's drawing buffer in sync with the canvas's CSS size (Three.js doesn't auto-resize). Animation loop guarded by amountedflag so React strict-mode remounts don't leakrequestAnimationFramecallbacks. Cleanup disposes geometries, materials, controls, and the renderer itself. Placeholder content: a slow-spinning wireframe icosahedron + empty-state overlay ("Submit a prompt to see the visualization") — replaced wholesale by chunk 2's transformer stack. FpsCounter (dev-only) sits absolute-positioned top-right of the canvas; rolling 30-frame window updating ~4×/sec.useReducedMotionhook subscribes to theprefers-reduced-motionmedia query and fires the auto-fallback inRightPane: when the user has motion reduced, the Viz tab isaria-disabledand the page boots into Debug; toggling the OS preference mid-session auto-flips to Debug. New jsdom polyfills (window.matchMedia+ResizeObserver/scrollIntoViewfrom chunk 7) intest/setup.ts. The chunk-6b auto-reload-on-terminal effect lifted fromLiveStreamPanetoRightPaneso it fires from any view. 8 new Vitest tests (default Viz mount, toggle aria + click flow, Debug → Viz round-trip, reduced-motion forces Debug + disables Viz tab, lifted stream forwards to both panes, single-subscription invariant, hook returns matchMedia at mount). The 6 chunk-6b tests that asserted on LiveStreamPane DOM updated to flip the Debug tab first. - 3D Transformer Stack component (chunk 2):
- Renders N layers from the run's model snapshot.
parameters.model_snapshot.layersis plumbed onto each run row inThreadController::show()(so the viz survives a model row being deleted / re-keyed), through theRunRowTS interface, intoVizPaneastotalLayers. Falls back to 12 layers when the snapshot is missing. TransformerStack(resources/js/Components/Viz/TransformerStack.ts) is a pure-Three.js builder: aGroupof NBoxGeometryslabs centered on y=0, height normalized to 4 world units so a 12-layer Mistral and an 80-layer Llama-3 fit the same camera frame. State machine: per-slabbase | active | hover | selectedwith immediate color swaps. Layer index lives onmesh.userData.layerIndexfor raycaster hit-testing. 12 Vitest tests cover construction, normalization, state-color swap, out-of-range guards.- Cascade animation driven by
CascadeController(resources/js/Components/Viz/CascadeController.ts). Eachlayer.advancedevent spawns a 600ms wave that walks bottom → top through the stack; waves overlap (no queuing), so a model generating 30 tok/s reads as a scrolling band rather than a bouncing head. Controller owns a monotonic clock advanced by the animation loop'sdeltaMs— tests advance it deterministically withoutperformance.now(). The events watcher tracksconsumedEventCountso re-renders don't re-fire waves; a shrinking events array (different run / SSE replay) callsreset(). 7 Vitest tests cover single wave, expiration, multi-wave overlap, reset, 1-layer and 80-layer edge cases. - Click-to-zoom + sub-component overlay. Raycaster on canvas click resolves
userData.layerIndex→ ReactselectedLayerstate. Camera lerps to a 3.2-unit offset focus position (8% per frame ≈ ~5-frame settle); orbit target lerps to the slab. The selected slab paints'selected'(amber) — chosen so it doesn't conflict with the cascade's'active'(indigo). HTML overlay (LayerDetailOverlay) anchors bottom-left of the canvas with the 5-step transformer-block sequence (RMSNorm → Attention → Residual → FFN → Residual);subComponentsFor(architectureType)swaps in'MoE Router → Experts'forarchitecture_type === 'moe'runs. Close button + selection-clear ontotalLayerschange. 3 Vitest tests on the sub-component helper. - Auto-rotation of the stack group pauses while a layer is focused — once you've drilled in, the view stays still. Resumes when you close the overlay.
- Renders N layers from the run's model snapshot.
- Token-flow particles (chunk 3): GPU-instanced streak trails.
ParticleSystem(resources/js/Components/Viz/ParticleSystem.ts) maintains a 256-particle pool rendered as oneInstancedMeshof thin Y-aligned boxes. Each box's BoxGeometry carries a per-vertex color attribute interpolating from black at the bottom face to cyan-300 (0x67e8f9) at the top — combined withAdditiveBlending+depthWrite: falsethis paints a comet-tail without a custom shader. Eachtoken.receivedevent spawns a burst of 5–10 streaks (5 + floor(rand() * 6)) at Y=−2 with random X/Z jitter within ±0.6. Streaks rise at 3 world units/sec (full-stack traversal ~1.3s), live for 2000ms total, and shrink from scale=1 → 0 between 1200–2000ms so the fade lands above the top slab. Per-instance opacity-via-shader was rejected in favor of scale-ramp fade — InstancedMesh has no built-in per-instance alpha and a custom shader felt premature for chunk 3. Pool semantics:spawnBurst()walks for inactive slots; pool exhaustion silently drops excess (the renderer is the visual cap, not the prompt). RNG is injectable so spawn-position tests are deterministic. 13 Vitest tests cover construction, kinematics (Y velocity), fade window, lifetime cutoff, slot reuse, reset, jitter bounds, vertex-color gradient. - Embedding space view (chunk 7). Lazy-loaded third tab in the right-pane toggle (Viz / Embeddings / Debug). Renders ~280 vocab tokens as a
THREE.Pointscloud with per-vertex colors driven by 8 hand-curated semantic clusters (whitespace, punctuation, pronouns, common-English, long-words, numbers, code-keywords, code-symbols). Each cluster has a 3D center and tokens are jittered by a deterministic xorshift32 seeded by(clusterIndex, tokenIndex)so positions are stable across reloads + tests. CustomShaderMaterialgives each point per-vertex alpha so visited tokens fade from base intensity (0.35) to full (1.0) and a glowing spotlight sphere snaps to the latest matched token. Auto-orbit pauses once a spotlight is active.lib/embeddingHighlight.tsresolves incoming token strings to vocab indices via exact → trim → lowercase → trim-AND-lowercase fallback chain; misses are silently dropped. Vocab data ships inresources/js/data/embeddingClusters.ts(TypeScript module so it bundles cleanly with the lazy chunk). 9 Vitest tests on the data (cluster count, color format, deterministic build, positions within bounded scatter, cluster index assignment) + 11 on the highlight lib (exact wins, trim, lowercase, trim+lowercase, order preserved, duplicates kept) + 3 integration in Show.test.tsx (tab present + clickable, EmbeddingScene mounts lazily, reduced-motion still allows tab). - D3 attention heatmap component (chunk 5a; per-layer; illustrative).
lib/attentionPattern.tssynthesizes a per-(token, layer) N×N matrix: causal lower-triangular, exponential distance-decay, per-row normalized, deterministic via xorshift32 seeded from(layerIndex, tokenCount). Decay constant grows with layer depth so the receptive field widens toward later layers — early layers attend locally, late layers attend broadly (a property tested explicitly:meanDistLate > meanDistEarly). Synthetic on purpose: SPEC §3.1.x notes that none of the supported vendor APIs expose attention tensors.AttentionHeatmap.tsxrenders the matrix as SVG rects with ad3-scalelinear interpolation from slate-950 → cyan-300 (palette-matched to chunk-3 particles). Mounted insideLayerDetailOverlay(chunk-2 click-to-zoom), capped at 24 tokens for legibility in the ~256px overlay width. Hidden whentokenCount === 0(nothing to attend to yet). 9 Vitest tests on the pattern (shape, causal mask, row-stochastic, determinism, depth-effect) + 7 on the heatmap render (cell count, color extremes, caption, illustrative note, size prop). - D3 logits distribution component (chunk 5b; top-10 bars, live updates).
lib/logitsExtract.tswalks the event stream backwards, returns the latesttoken.receivedwith non-null logprobs as{ chosenToken, alternatives: [{token, probability}] }. Logprobs → probabilities (exp(logprob)), then top-K normalization so the chart represents "weight among the reported alternatives" rather than pretending we have the full vocabulary tail.Components/LogitsDistribution.tsxrenders top-10 horizontal bars withJSON.stringify(token)labels so whitespace / newline tokens stay legible (a\ntoken reads as"\n", not a line break). Chosen-token bar paints primary (indigo), alternatives muted. Mounted insideLiveRunBodybetween the live text and the metrics strip. Per the chunk-5 decision, returnsnullwhen no token has logprobs — no placeholder, no fabrication. 9 Vitest tests on the extractor (latest-wins, fallback-when-latest-is-null, normalization, top-K trim, skips non-token events, empty array). 7 tests on the component (null on no-logprobs, bar count, chosen highlight, whitespace escape, sum-to-100, top-K cap). 2 integration tests inShow.test.tsx. - MoE routing component (chunk 6). Two views stacked: (1) latest-token router scores as normalized horizontal bars (matches chunk-5b logits-chart visual), (2) cumulative expert-utilization as a horizontal grid of mini vertical bars, one per expert. Scales from Mixtral's 8 experts to DeepSeek-V2's 160 by letting flex distribute bar widths.
lib/moeMetrics.tsexposesextractLatestRouting(events)(walks backwards, renormalizes the K reported scores to sum to 1, sorts descending) andextractUtilization(events, totalExperts)(counts per-expert activations, falls back to inferred max-expert-id+1 sizing when the model snapshot lacksmoe_experts, defensively drops out-of-range expert IDs). The component is mounted byLiveRunBodyonly whenmodel.architecture_type === 'moe'— dense runs skip it entirely (no placeholder, mirrors the chunk-5b logits-when-null decision). Header reads "MoE routing · N experts · top-K". 14 Vitest tests on the pure helpers (latest-wins, renormalization, sort, degenerate-event skip, fallback sizing, defensive ID range, non-MoE-event ignore) + 8 tests on the component (null on no events, header, bar count, normalized %, utilization counts, fallback sizing, unused-expert marking) + 2 integration tests in Show.test.tsx (mounts on MoE run, skipped on dense run). - Live text + cost/TPS + context bar (chunk 4 — fused 4/9–11/9 items into one chunk because they all read from the same derived metrics). The Transcript card for a pending/streaming run swaps its empty Assistant block for
LiveRunBody: live token text concatenated fromtoken.receivedevents with a blinking cursor (▍,animate-pulse), and aLiveMetricsStripshowingN out · X.X t/s · $0.0000plus a context barused / model.context_length. Cumulative TPS (output_tokens / last_t_ms × 1000) — no rolling buffer, no wall-clock ticker, derivation invariant between events. Cost uses the model'spricing_input_per_million+pricing_output_per_millionand gracefully omits when either field or the whole pricing object is null. Context bar percent crosses to amber at 80% and destructive at 100% (matches the M7 prompt budget bar). All derivation lives inresources/js/lib/streamMetrics.ts— a pure function returning{ liveText, outputTokens, elapsedMs, tps, costSoFar, contextUsed, contextBudget }so the math is unit-testable without React. 12 Vitest tests cover the pure helper (no events, ordering, partial pricing, divide-by-zero, ignored non-token events). 6 Vitest tests on<ThreadShow />cover the live row + cursor + metrics strip + the hide-bar-when-no-context-length / hide-cost-when-no-pricing / static-row-during-terminal-flicker paths. - KV cache progress bar (rolled into the chunk-4 context bar above — same denominator, same widget).
- Playback controls (chunk 8).
useEventPlayback(events)is a single source of truth: returns{ visibleEvents, cursor, totalEvents, playing, speed, isLive, play, pause, toggle, step, setSpeed, jumpToLive }.visibleEvents = events.slice(0, cursor)flows to every viz consumer (Transcript,VizPane,EmbeddingScene,LiveStreamPane) — pause/step/speed affect the live text, the cascade animation, the particles, the heatmap, the logits, MoE routing, AND the embedding spotlight uniformly. Semantics per chunk-8 decision: 1× = LIVE (cursor syncs to events.length on every render via a useEffect — no setInterval, no artificial delay); 0.5× = throttle (setInterval dispenses one event every ~67ms = 15 events/sec, buffer grows when source is faster); 2× / 4× = drain (faster dispenser to catch up after a pause, stops at the head until more events arrive). Step finds the nexttoken.receivedevent and lands cursor on it — skips intermediatelayer.advanced/moe.routedso "advance one token" matches the user's mental model. Stream-shrink (new run / SSE replay) auto-resets to head + resumes LIVE so a fresh run never displays a blank viz.PlaybackControlsis a compact toolbar above the right-pane tab toggle: Play/Pause icon button, Step icon (disabled at head), LIVE pill OR cursor/total counter (clickable to jump back to live), 0.5× / 1× / 2× / 4× segmented control. 11 Vitest tests on the hook (renderHook + fake timers; LIVE-sync, pause-freezes, step-skips-non-tokens, throttle-at-0.5×, drain-at-4×, shrink-resets, jump-to-live, toggle, slice-matches-cursor). 11 tests on the component (icons, click-dispatch, disabled-step, LIVE pill, jump-to-live counter, speed-segmented). 4 integration tests inShow.test.tsx. - Live token-stream text panel (left side of split layout) — landed in chunk 4 above; the Transcript's active-run row IS the live panel.
- Cost / tokens-per-second readout — landed in chunk 4 above (
LiveMetricsStrip). - FPS counter (dev-only) — landed in chunk 1 (
Components/Viz/FpsCounter.tsx), absolute-positioned top-right of every Three.js canvas (VizPane,EmbeddingScene). Rolling 30-frame window, updates ~4×/sec, gated behindimport.meta.env.DEVso it disappears in production builds. - Reduced-motion fallback — landed in chunk 1 via
useReducedMotion+ theRightPaneauto-flip. When the OSprefers-reduced-motionmedia query is set: the Viz tab isaria-disabledwith explanatorytitle, the user is auto-flipped to the Debug tab on mount, and toggling reduced-motion mid-session re-routes to Debug. The Embeddings tab (chunk 7) stays available because its auto-orbit pauses once a spotlight is active and the trail update fires per-event, not per-frame — so the cumulative motion footprint is small. - Vertical-slice end-to-end test (chunk 9). Two complementary coverages: (1) programmatic kitchen-sink integration test in
Show.test.tsxthat fires a realistic mixed event sequence —run.started+ 3token.received(each with logprobs) interspersed withlayer.advanced+moe.routed— through<ThreadShow />for a Mixtral-shaped model, then asserts every viz piece responds: live text + cursor (chunk 4), metrics strip with TPS / cost / context bar (chunk 4), logits chart with chosen-highlight (chunk 5b), MoE routing with utilization counts (chunk 6), viz pane receives all events (chunks 1–3), Embeddings tab is mountable (chunk 7),PlaybackControlsLIVE pill flips to cursor counter on pause (chunk 8). Single test, ~120 lines, asserts ~20 invariants. (2) Manual recipe documented below for human verification with a real vendor call + the 30-FPS exit gate.
Prerequisites: an API key on file for the chosen vendor; a thread created; a recent build of npm run dev running.
- Submit a prompt that produces ~100 tokens — e.g. "Write a 100-word explanation of quaternions" against OpenAI
gpt-4o-minior Togethermixtral-8x7b. - Watch the Viz tab (default). The transformer stack should render as N slabs; a 600ms cascade wave should walk bottom → top per token (chunks 2–3). Streak particles should rise from the base of the stack on each token (chunk 3). The slow Y-axis rotation should pause when a layer is clicked.
- Click a layer slab. The detail overlay should appear bottom-left with the 5-step sub-component list (RMSNorm → Attention → Residual → FFN/MoE → Residual) and a 24×24 attention heatmap below (chunks 2 + 5a). For a Mixtral run the FFN row reads "MoE Router → Experts" and an MoE badge appears next to the layer number.
- Watch the active run row in the transcript. Live token text should stream in with a blinking
▍cursor; the metrics strip below should update withN out · X.X t/s · $0.0000plus a context-used bar (chunk 4). For a vendor that returns logprobs, the top-10 alternatives bar chart should render below the live text with the chosen token highlighted (chunk 5b). For Mixtral, the MoE routing panel should appear with the latest token's router scores and a per-expert utilization grid (chunk 6). - Switch to the Embeddings tab. The vocab scatter cloud should render with 8 colored clusters; as the stream progresses, matched tokens should fade up to full intensity and a glowing spotlight sphere should snap to the latest matched token's position (chunk 7). Auto-orbit pauses once a spotlight is active.
- Verify FPS. In dev mode, the FPS counter sits top-right of every canvas. Expected: ≥30 FPS during the 100-token stream on a 2020-era machine. (The cascade + particles + spotlight stay well under the limit for that hardware.)
- Exercise playback controls. Click pause — live text, viz, logits, MoE all freeze on the current cursor. Click Step — cursor advances to the next
token.receivedevent, and the whole stack moves one token forward (chunk 8). Click0.5×then play — the dispenser slows to ~15 events/sec. Click the cursor counter — jumps back to LIVE at 1×. - Switch model to a dense one mid-thread. Submit another prompt with
gpt-4o-miniorllama-3.1-8b. The MoE routing panel should not appear on the new run (chunk 6 gate). Sub-component overlay's FFN row reads "FFN" instead of "MoE Router → Experts" (chunk 5a /subComponentsFor). - Verify reduced-motion fallback. Toggle your OS preference (macOS: System Settings → Accessibility → Display → Reduce motion). The Viz tab should grey out, the page should auto-flip to Debug. Embeddings tab stays available.
If any step fails, the failure isolates to that chunk — open the relevant Components/Viz/*.tsx or lib/*.ts and re-run its Vitest suite.
Exit criteria — vertical-slice gate
- Submitting a prompt produces a recognizable, smooth animation that visibly tracks the token stream. (Manual recipe steps 1–4; programmatic kitchen-sink test asserts every viz piece responds.)
- Switching to a Mixtral run shows the MoE routing graph. (Chunk 6 gate; manual recipe step 8.)
- Switching to a Llama-3 run hides the MoE graph. (Chunk 6 gate; manual recipe step 8.)
- 30 FPS sustained for a 100-token stream on a 2020 MacBook Air. (FpsCounter overlay visible in dev mode; manual recipe step 6. Per-token cost: cascade is ≤8 layer-state writes per frame, particles are one InstancedMesh draw call, embedding scene is one Points draw + one Mesh.)
- The visualization is compelling and the milestone proceeds to M9 (Replay + JSON export). No re-evaluation triggered.
What worked
- Pure-function-first design. Every viz piece had a
lib/*.tsordata/*.tshelper (streamMetrics, attentionPattern, logitsExtract, moeMetrics, embeddingClusters, embeddingHighlight) that was independently testable. The hot integration tests inShow.test.tsxcover wiring; the dense pure-function tests cover semantics. Total Vitest count grew from 47 at M7 closeout to 351 at M8 closeout — and the pure tests are the bedrock: they catch math regressions before the integration tests get involved. - Lazy-load discipline kept the main app bundle small.
VizPaneandEmbeddingSceneeachReact.lazy-import. Three.js + custom shaders + the vocab data file all stay out of/dashboard,/threads,/api-keysuntil the user actually opens a thread. Vite split the Three.js dep into a shared chunk between the two viz tabs (~526KB / 132KB gzipped) once both started using it — Rollup recognized the shared module automatically without manualChunks config. - Custom ShaderMaterial for per-vertex alpha (chunk 7) beat InstancedMesh. Per-instance opacity isn't a built-in Three.js feature; alternatives are (a) custom shader, (b) per-instance matrix manipulation. For ~280 points, the shader was cheaper to write and faster at runtime (single attribute write per visited token vs. 16 transform-matrix writes).
- Playback engine shape carries cleanly into M9 replay.
useEventPlayback(events)is the entire abstraction — it doesn't care ifeventscame from a WebSocket orruns.token_log. The chunk-9 manual recipe step 7 (pause / step / 0.5× / cursor-jump-to-live) is the exact UX M9 replay will inherit for free. - Synthetic over real data (chunk 7) was the right scope call. Real PCA per tokenizer would have meant downloading ~500MB-1GB of HuggingFace weights, running PCA offline, and committing a generated JSON per tokenizer family. For Phase 1's "vertical slice = compelling animation" goal, the spatial story (token types form groups) is what matters; provenance of the positions is secondary. The decision is documented and the swap-in point (
data/embeddingClusters.ts) is clear. - Decision questions up-front, per chunk. Every chunk in M8 opened with
AskUserQuestioncovering 2–4 consequential decisions (data approach, layout, threshold semantics, etc.). Captured each answer in the chunk'sDecisionsblock. Total decisions documented in M8: ~30 across 9 chunks. The decision log is the most-useful artifact for anyone re-reading the code six months from now —git logshows what changed, the decision blocks explain why.
What we learned
Math.max(8, ...)for setInterval intervals matters in tests. Chunk 8's hook math:1000 / (BASE_RATE * speed)produces an 8.3ms interval at 4× speed. Floor-clamping to 8ms keeps the timer well-defined; the clamp is also the only place fake timers + the throttle path could deadlock. (Caught during the chunk-8 test pass.)- Lazy-loaded components need
await screen.findByTestIdnotgetByTestIdfor the first activation. Avi.mock-stubbed component still goes through React.lazy's Suspense cycle; subsequent renders are synchronous. The chunk-7 Embeddings tab test had to switch toawait findByTestIdonce. Worth documenting if M9+ adds more lazy chunks. textContentregex matches can pick up inline-style values in jsdom under some setups. The chunk-6 router-bar test originally matched(\d+\.\d+)%against row textContent and got 400% instead of 100% — the bar's inlinestyle="width: 50%"was leaking into the match. Fixed by scoping the regex to the trailing label span via class. Generalized lesson: if a regex is summing percentages from a component that also uses%in inline styles, scope the query precisely.
Parked / carry-forward to M9
- Replay route reuses the entire M8 stack.
Replay/Show.tsxwill look like a cousin ofThreads/Show.tsxminus the prompt footer and the WebSocket subscription. The events source becomesruns.token_log(already persisted incrementally byStreamRunJobper M6 chunk 5a).useEventPlaybacktakes that array verbatim. The chunk-8 decisions about LIVE = head-sync, throttle, step granularity all still apply (just without the head moving — once a replay loads, totalEvents is fixed). - Real PCA data is an opt-in upgrade. If a user wants accurate per-tokenizer vocab geometry, the swap-in point is
data/embeddingClusters.ts. The format (token +[x, y, z]+ cluster index + color) is generic enough that real PCA output drops in by re-running the build script against a different source. - Vertical-slice E2E test is deliberately not a full Playwright/Cypress test. Per chunk-9 decision, that's M11 Hardening territory. The kitchen-sink integration test catches the integration-regression case the unit tests miss; the manual recipe gives the human verification path. Once we're confident in stability, browser automation can layer on top.
Purpose: Saved runs can be replayed without hitting vendor APIs. Runs and threads export to JSON.
Tasks
- Replay route:
/threads/{thread}/runs/{run}/replay(chunk 1).ReplayController::showauthorizes thread owner + run-in-thread + terminal-status (422 for streaming runs), then rendersInertia 'Runs/Replay'with{ thread, run, events, model }. The event stream is synthesized byapp/Services/Runs/RunReplayEventSynthesizer— given a Run, it walkstoken_logand produces the same wire-shapeRunEvent[]the live broadcast pipeline emits:run.startedfirst, per-tokentoken.received+layer.advanced(+moe.routedfor MoE), thenrun.completed/run.errored. ReusesRunEventEmitterso MoE expert routing stays deterministic per (run.id, token_index) — replays are frame-identical to live (SPEC §10.1). 9 Pest tests on the synthesizer + 8 on the controller. - Replay page reuses M8 components but reads events from
runs.token_lograther than the WebSocket (chunk 1).resources/js/Pages/Runs/Replay.tsxis a standalone Inertia page: thread breadcrumb + run header, a prompt + re-streaming assistant body driven byuseEventPlayback({ mode: 'replay', initialPlaying: false }), and the M8 right pane (Viz/Embeddings/Debug) sharing the same lazy-loaded Three.js bundle. The viz consumers (LogitsDistribution, MoERouting, VizPane, EmbeddingScene) need zero changes — they read fromplayback.visibleEventsexactly like the live page. The Debug tab is a simpler standalone variant (no transport label since there's no WebSocket). 8 Vitest tests on the page; per the chunk-1 decision, the initial state is paused-at-cursor-0 and the user clicks Play to start. - Replay launch — Replay button on each terminal run row in the thread detail page transcript (chunk 1). Renders a small pill-shaped Link with the Play icon next to the status badge for
completeanderrorruns. Hidden forpending/streamingruns. 3 Vitest assertions. - Deterministic seeding verified — same run replayed twice produces frame-identical animation (chunk 2). Per chunk-2 strict-determinism decision, every visual signal — including the previously-Math.random-driven particle X/Z jitter and burst size — is now a pure function of
token.received.payload.index. Newlib/particleBurst.tsexposesburstForToken(tokenIndex) → { count: 5–10, seed }derived via xorshift32;ParticleSystem.spawnBurst(count, seed?)switched to use the seed when provided (falls back to constructorrandomfor back-compat with existing tests).VizPane's events-watcher now callsburstForToken(e.payload.index)per token. Backend determinism (already strong from chunk 1 — MoE expert routing is hash-seeded by(run.id, token_index)viaRunEventEmitter) is now exhaustively covered byReplayDeterminismTest: byte-equal across instances, byte-equal across repeated build() calls, identical MoE routing for the same run, different routing for different run IDs, stable event ordering, dense runs emit zero MoE events, logprobs survive verbatim. 6 Vitest tests on the burst helper + 4 new on the seeded ParticleSystem + 8 Pest determinism cases. - JSON export:
GET /runs/{run}/export.json(chunk 3). Owner-only download. Schema 1.0 top-level:{ schema_version, exported_at, thread: { id, title, tags }, run: { full metadata + parameters (with embedded model_snapshot) + conversation_history + token_log } }.RunExportSerializer(app/Services/Runs/RunExportSerializer.php) is the pure-function source of truth;RunExportControlleris a thin owner-auth wrapper that setsContent-Disposition: attachment; filename="run-{id}.json"and pretty-prints withJSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES. Excluded by design:user_id,model_id,api_key_id— exports carry zero user identity so the same payload is safe to forward to a future M11 sharing flow. Privacy:users.store_prompts=falsealready nullsprompt+conversation_historyat write time; the export surfaces those nulls naturally with no extra redaction. 10 Pest tests on the serializer + 10 on the controller (auth, headers, schema-structure assertion, round-trip parse, MoE + dense + errored runs). - Thread JSON export:
GET /threads/{thread}/export.json(chunk 4). Owner-only download. Schema 1.0:{ schema_version, exported_at, thread: { id, title, tags, archived, created_at, last_activity_at }, runs: [run sections ordered by sequence_in_thread asc] }.ThreadExportSerializerbuilds the document; the per-run shape is byte-equal to chunk-3 because both serializers share a singleRunExportSerializer::runSection(Run)helper. Per chunk-4 decision, all run statuses are included (pending / streaming / complete / error) — matches the chunk-3 single-run endpoint's policy. Empty threads returnruns: []. Content-Disposition:attachment; filename="thread-{id}.json". 8 Pest tests on the serializer + 12 on the controller (auth, headers, schema structure, ordering, all-status inclusion, empty thread, round-trip, identity-field exclusion). - Download UI: button on run detail + thread detail pages (chunk 5). Three placements: (1) thread header — new outlined
Exportbutton next toTitle/Archive/Deletelinks to/threads/{id}/export.jsonwithdownload="thread-{id}.json"; (2) terminal run rows in the Transcript — small icon-onlyJSONDownload pill next to the existing Replay link, links to/runs/{id}/export.json; (3) Replay page header — outlinedDownload JSONbutton on the right of the header bar. All three use plain<a href download>so the browser handlesContent-Dispositionnatively — no JS fetch. Per chunk-5 decision, per-run buttons are terminal-only (consistent with the chunk-1 Replay placement); in-flight runs hide both buttons. 5 new Vitest assertions (3 in Show.test for run-row visibility + thread header + URL; 1 in Replay.test for replay-page download; 1 for streaming-row hidden).
Decisions (chunks 1–5):
useEventPlaybackgains amode: 'live' | 'replay'option. In'replay'mode, 1× becomes a throttled dispenser (BASE_RATE × 1 events/sec) instead of LIVE head-sync.isReplayis exposed so the toolbar can suppress the LIVE pill. The hook's other semantics (pause, step-to-next-token, 0.5× throttle, 2×/4× drain, setCursor for scrubbing) work identically across modes. 5 new Vitest cases cover the replay-mode paths. (Decision: extending the existing hook over building a parallel one — same buffer model, same controls UX, M9 inherits chunk-8's playback engine.)RunReplayEventSynthesizerreusesRunEventEmitterrather than reimplementing. Constructs aLlmTokenChunkfrom each token_log entry and runs it through the same emitter the live pipeline uses, then maps each Laravel event to itsbroadcastAs()+broadcastWith()pair. Means MoE expert routing's(run.id, token_index) → expert IDshash stays consistent with what the original live broadcast emitted — replays match per SPEC §10.1. (Decision: synthesizer reuses emitter; doesn't persist layer/MoE events. Keepstoken_logschema tiny; replay is a pure function of (run, token_log).)- Initial state: paused at cursor 0. User clicks Play to start. Considered: auto-play at 1× (more immediate) vs starts-at-end (full state visible). Paused-at-0 is the cleanest "give me a moment to set up the view" UX. The PlaybackControls toolbar's Step button works from cursor 0 so users can also walk through token-by-token from the start.
- Replay page is standalone, not a flag on
Threads/Show. Considered conditional logic inside the live page. Standalone keeps the prompt-footer + thread-title-editing logic on the live page only and the replay logic is much smaller (~340 lines including the RightPane inline). Worth a tiny duplication of the RightPane structure (tab toggle + lazy viz/embeddings + debug pane) — only ~80 lines. - Replay button only shows on terminal runs (complete/error). Pending/streaming runs have no stable token_log to replay. The button on errored runs still renders because partial output is replayable; users can see how the run progressed before failure.
- 422 for non-terminal runs in the controller, not 404. Conveys "this is a real run but not in a state that supports replay" rather than "no such run." Semantically clearer error.
- Strict frame-identity, not pragmatic "meaningful signal only". (chunk 2) User choice. Considered leaving decorative particle jitter on
Math.random()and documenting it as "intended visual variety". Chose strict because: (a) the SPEC §10.1 text is unambiguous, (b) the implementation cost was small (~50 lines), (c) it sets up M10 (GIF export) for byte-stable encoding without further fiddling, and (d) the parallel between "live deterministic visuals" and "replayable deterministic visuals" is a useful guarantee anywhere we care to invoke it. The cost is that consecutive live submissions of the same prompt to a vendor will produce identical particle bursts on every replay — but live runs never produce identical token streams anyway because the vendor's stochastic decoding varies. - Burst params derive from
token.received.payload.index, not from event-array position. (chunk 2) Token index is the semantic "this is the Nth generated token" identifier — survives any future change to event ordering or interleaving (e.g., if we ever batch layer.advanced events). Same-run live and replay both produce the same token indices in the same order, so the seed is naturally stable across the two paths. ParticleSystem.spawnBurstkeepsseedoptional, not required. (chunk 2) Constructorrandomstill works when no seed is passed. Preserves the 13 existing chunk-3 tests + the chunk-3 fixedRng dependency-injection pattern. Strict-determinism is opt-in at the call site; the test suite explicitly verifies both paths.- JSON export schema gets a
schema_version: "1.0"field. (chunk 3) User choice. Cost is negligible (one extra key); benefit is large the day we want to ship a v2 (e.g., adding attention tensors when a future vendor exposes them). Consumers can dispatch on the version; M11 shared-link rendering can refuse versions it doesn't know. - Export uses the run's
parameters.model_snapshot, not the liveLlmModelrow. (chunk 3) The snapshot is the canonical record at run time per SPEC §10.1; the live model row could be deleted or re-keyed by an admin. Embedding the snapshot keeps the export self-describing and replay-able forever regardless of what happens to the registry.model_id(FK) is deliberately omitted. - Export carries no user identity (no
user_id,api_key_id). (chunk 3) Future M11 sharing wants to hand someone else this exact JSON without leaking who ran it. Stripping identity at the serializer means we don't have to write a redaction step later — the export is portable by construction. - Round-trip is schema-stability only for chunk 3. (chunk 3) User choice. No
POST /runs/importendpoint yet — that needs UI scope, model-resolution policy ("what if the source model isn't in our registry"), and validation rules. Defer until M11 or whenever a real "import this thread" UI lands. The schema is documented + stable enough that import can be added later without breaking exports. Content-Disposition: attachmentnotinline. (chunk 3) The endpoint exists for download UX, not in-browser viewing. Browsers respect attachment + filename so a click yields arun-{id}.jsonfile instead of pretty-printed JSON in a tab.- Thread export includes all runs regardless of status. (chunk 4) User choice. Mirrors the chunk-3 single-run endpoint which also has no status filter. Pending / streaming runs surface with partial fields — that's honest. Trade-off: an in-flight run is visible in an export with
token_log: [...]partial state; future consumers may want to filter. Adding a?status=query param is a one-line follow-up if needed. - Per-run shape is shared between single-run and thread exports. (chunk 4) Extracted
RunExportSerializer::runSection(Run): arrayas a static helper that bothRunExportSerializer::build()andThreadExportSerializer::build()call. Means a consumer that understands one shape understands the other — and any future per-run field addition (e.g., a vendor's attention tensor) lands in one place. Trade-off: the static is a small ergonomic friction, but in practice the serializer never holds per-instance state worth carrying. - No NDJSON / streaming response for large threads. (chunk 4) A thread with hundreds of runs × full
token_logcould reach 10MB; FPM-worker memory is fine at that size. A streamingapplication/x-ndjsonresponse would help if exports grow to gigabytes, but that's not a real shape today. If it becomes one, the swap-in point is the controller — the serializer can yield runs one-by-one already. - Per-run download button is terminal-only, matches the Replay button placement. (chunk 5) User choice. The export endpoint itself accepts any status (chunk 3 decision), but the UI hides the button on in-flight runs. Rationale: visual consistency with Replay; downloading a half-baked
token_logis rarely what users want. Power users can still hit/runs/{id}/export.jsondirectly if needed. - Plain
<a href download>instead of JS fetch / blob construction. (chunk 5) Browsers handleContent-Disposition: attachmentnatively when the link is a regular anchor. JS fetch would require building a Blob URL and revoking it — more code, more failure modes, no real upside. The download attribute also gives the file a sensible default name in browsers that ignore the server header. - Thread Export button uses shadcn
ButtonwithasChild. (chunk 5) Keeps the visual language identical to the adjacentTitle/Archive/Deletebuttons.asChildrenders theButton's styles onto the inner<a>so we get the right outlined look without nesting buttons-in-buttons or losing the anchor's download behavior.
Exit criteria
- Replaying a 100-token run shows the same animation as the original generation. Verified at the data layer by
ReplayDeterminismTest(byte-equal events across instances, identical MoE expert routing per(run.id, token_index), stable event ordering — chunk 2). Verified at the rendering layer by the chunk-2 strict-determinism path: every visual signal (cascade timing, MoE routing, attention pattern, logits, embedding spotlight, particle burst size + jitter positions) is a pure function ofpayload.index. The manual recipe step 1–4 below covers the human verification. - Exported JSON re-imports cleanly (round-trip test). Verified by
RunExportSerializerTest::it('round-trip: encodes to JSON and decodes back to an identical array')and the controller's round-trip parsing assertion, plus the thread-export equivalents.
Prerequisites: a Run that's reached complete or error status. Quickest path: submit any prompt against any vendor (gpt-4o-mini is cheap + fast).
Replay
- From the thread detail page (
/threads/{id}), find the completed run's row in the transcript. Click the Replay pill next to the status badge. - The replay page (
/threads/{id}/runs/{id}/replay) opens. Header shows back-to-thread + Download JSON; below sits the prompt and an empty assistant body with a blinking cursor — replay starts paused at cursor 0 per the chunk-1 decision. - Click Play in the PlaybackControls toolbar. The viz, the assistant text, the logits chart, the MoE routing (if MoE), and the embedding spotlight all advance through the saved event stream. At 1× the dispenser fires ~30 events/sec; 0.5× slows to ~15; 2×/4× drain faster.
- Click Step while paused to walk one token at a time. Click Embeddings in the right-pane toggle to watch the vocab cloud light up matched tokens.
- Reload the page. The replay starts paused at cursor 0 again. Click Play. The particle bursts, cascade waves, and MoE routing arrive at the same positions / experts as the previous viewing — frame-identical per SPEC §10.1.
Export
- From the thread detail page, click the Export outlined button in the header (next to Archive / Delete). The browser downloads
thread-{id}.json. - Open the file. Top-level fields:
schema_version,exported_at,thread,runs(array, ordered bysequence_in_thread). - Back on the thread page, on any terminal run row click the JSON pill next to Replay. The browser downloads
run-{id}.json. Top-level fields:schema_version,exported_at,thread,run(single object, same shape as one entry in the thread export'sruns[]). - On the replay page header, the Download JSON button hits the same single-run endpoint.
- Verify the export shape contains no
user_id,model_id, orapi_key_id— exports are portable for future M11 sharing. - Try a non-owner replay or export URL (sign in as a different user, paste the URL) — should return 403.
What worked
- Reusing M8's playback engine via a
modeoption.useEventPlayback({ mode: 'replay', initialPlaying: false, initialCursor: 0 })is the entire frontend-side adaptation. The hook keeps a single buffer model and a single set of controls; replay just changes the meaning of 1× (throttle vs head-sync) and the starting state. 5 new Vitest cases were enough to validate the replay path; the chunk-8 live-mode tests stayed green throughout. - Token-log is the canonical replay source — no second persistence path.
StreamRunJobalready writestoken_logincrementally (M6 chunk 5a edit). M9 didn't have to touch the streaming pipeline at all. The synthesizer just walks that array and reusesRunEventEmitterfor the MoE / layer events, which inherits the live pipeline's(run.id, token_index)hash for free. - Strict frame-identity was cheap. Chunk-2 worried that seeding every viz signal might be a slog; in practice it was ~50 lines (
lib/particleBurst.ts+ParticleSystem.spawnBurstseed path + the VizPane events-watcher line). The hard work — the MoE/layer determinism — was already done back at M6. The remaining randomness in particle positions was the only loose end, and a purexorshift32keyed onpayload.indexcleared it. - Schema-version-from-day-one paid off conceptually. Even though there's no v2 yet, the
schema_version: "1.0"field in both export shapes documents the intent: consumers dispatch on it, version bumps are explicit. M11 sharing-link rendering can read this single field and refuse versions it doesn't understand. - Shared
runSectionhelper between single-run and thread exports. Extracted in chunk 4 once both endpoints existed. Means a consumer that parses one export shape parses the other. Adding a future field (e.g., a vendor's attention tensor) lands in one place.
What we learned
- Inertia view-finder caching pinned a Pest test failure. Chunk-1's ReplayController tests failed with "Unable to locate file in Vite manifest" until the
Runs/Replay.tsxfile existed on disk — even though we're rendering through Inertia not directly through Vite. The PHP-side view layer requires the chunk to resolve from the manifest. Easy unblock once spotted; worth documenting because the error message points away from the real issue. - Lazy-loaded components need
findByTestId, notgetByTestId. Already learned at M8 chunk 7 with EmbeddingScene; M9 chunk 1's Replay page reused the same pattern. Button asChildwith an anchor +downloadattr is the right Tailwind/shadcn idiom for download buttons. Avoids fighting withLink(Inertia routing) for non-Inertia URLs. The same pattern landed in three places in chunk 5.
Parked / carry-forward to M10
- M10 GIF export inherits the entire replay infrastructure. The most natural implementation: spin up Puppeteer / Playwright, navigate to
/threads/{id}/runs/{id}/replay, click Play at the desired speed, capture frames from the Viz canvas, encode viaffmpeg. The replay is already frame-identical and deterministic so the captured GIF will look the same on every render. - Synthesizer can emit alternate event types if needed. If M10 ever wants finer-grained intermediate events (e.g., "particle spawn at this t_ms" for sub-frame capture), the synthesizer is the place to add them — keeps the existing live + replay paths in sync.
- JSON import endpoint is deferred. Per chunk-3 decision, no
POST /runs/importexists yet. M11 sharing might want it (paste-a-link-to-import-a-run); the schema's stable enough that adding it is avalidate + create + write token_logcontroller, ~50 lines. - Real-browser smoke test is not automated. Per chunk-6 decision, the manual recipe above is the verification path. Playwright/Cypress E2E lands in M11 Hardening (same call as M8).
Purpose: Completed runs can be downloaded as animated GIFs.
Tasks
- Job:
ExportRunGif(queued) — chunk 1 landed the foundation skeleton.app/Jobs/ExportRunGif.phpis aShouldQueuejob constructed with a$runId(kept primitive so the queue payload stays small + survives DB refreshes between dispatch and worker).handle(GifRenderer, ExportStorage)cache-hits early when both export files exist, fetches the Run, and calls the renderer with aRenderConfigbuilt fromconfig('gif_export.frame_rate')+max_duration_ms. Single try / 5-minute timeout matches the SPEC. The supporting cast also landed:RenderConfig(immutable, 30 FPS / 5-min default withwithFrameRate/withMaxDurationMsbuilders),RenderResult(gif/mp4 paths + framesCount + durationMs surface fields),GifRendererinterface (chunks 2 + 4 implement),NullRendererplaceholder that logs + throws so misconfigurations land in failed_jobs,GifRendererFactory(driver dispatch ongif_export.rendererconfig),ExportStoragepaths + cache helpers (hasGif/hasMp4/bothExist, treats partial renders as cache miss). Service binding inAppServiceProviderlets the job type-hint theGifRendererinterface and get the configured implementation via the factory. 23 Pest tests across the 4 new test files. - SVG renderer (default
GIF_RENDERER=svg) — chunk 2 lands the PNG-sequence half of the pipeline. Architecture: a newFrameRendererinterface (renderFrames(Run, RenderConfig, outputDir): list<string>) is implemented bySvgFrameRenderer. For each frame timestamp sampled atconfig.frameRatefromtoken_log's lastt_ms(per chunk-2 decision), it walks the log, derives state (live text, token count, last logprobs, attention pattern, MoE state-if-MoE), and emits an SVG with the 4 panels in a 1280×720 canvas: assistant text + metrics line + 12×12 attention heatmap + top-K logits bars + MoE routing strip (when applicable). Footer reads "2D summary view · frame N / M". Rasterization shells out to ImageMagick'sconvertCLI (chunk-2 decision: avoids theext-imagickPHP-extension dep —apt install imagemagickis enough). A separateVideoEncoderinterface lands itsNullVideoEncoderstub (logs + throws); chunk 3 swaps in theFfmpegEncoder.SvgRendereris the GifRenderer orchestrator: creates a tmp dir undersys_get_temp_dir(), runs the frame renderer, calls the encoder with the storage-relative gif/mp4 paths, cleans up on success AND failure (viatry { … } finally { cleanup }).GifRendererFactorynow resolves'svg'toSvgRenderervia the container (so chunk 3 swaps the encoder binding without touching the factory). 14 new Pest tests across the 3 new files (7 frame renderer + 6 orchestrator + 1 null encoder). - [~] Puppeteer renderer (opt-in
GIF_RENDERER=puppeteer) — chunk 4 skeleton landed per the chunk-4 scope decision. PHP-side contract is complete; Node script +/renderroute + frontend record mode ship in a follow-up:ChromiumDetector— singleton service that resolves a Chromium binary viaCHROMIUM_PATHenv or a list of standard Linux/macOS locations. Cached per-process so the boot-time check runs once. 7 Pest tests on the resolution paths.PuppeteerFrameRendererimplements FrameRenderer — checks the detector, then the Node script's existence atconfig('gif_export.node_script_path'). RaisesChromiumUnavailableException(a new custom exception) when Chromium is missing so chunk 6's fallback path can catch it specifically and re-route to the SVG renderer; raises plainRuntimeExceptionfor everything else (missing Node script, Node non-zero exit, zero frames produced). The shell-out shape —node <script> --run --out --fps --max-ms --chromium— is defined now so the follow-up Node script implementation has a stable contract. 5 Pest tests.- Factory wires
'puppeteer'to the sameSvgRendererorchestrator the SVG path uses (the orchestrator is renderer-agnostic per the chunk-2 design), withPuppeteerFrameRendererinjected instead ofSvgFrameRenderer. Encoder + storage are shared verbatim. - Deferred to a follow-up chunk (real puppeteer integration): the actual Node frame-capture script +
/runs/{id}/renderLaravel route + signed-URL auth bypass for the spawned process + frontend record mode onReplay.tsx(autoplay, hidden controls, fixed viewport,window.__exportCompletesignal).
- ffmpeg shell-out: PNG sequence → animated GIF and MP4 (H.264). Both formats produced from the same frame sequence (chunk 3).
FfmpegEncoder(app/Services/Exports/FfmpegEncoder.php) implementsVideoEncoder. Three ffmpeg invocations per export, per chunk-3 decision: (1) PNG sequence → MP4 withlibx264 / yuv420p / +faststart / scale=trunc(iw/2)*2:trunc(ih/2)*2for web-compat + even-dimension x264 requirement; (2) PNG sequence → palette viapalettegen=stats_mode=fullfor the chunk-3 "palette-optimized GIF" decision; (3) PNG sequence + palette → GIF viapaletteuse=dither=bayer:bayer_scale=3 -loop 0for clean dithering on slate gradients + infinite playback in Slack/Discord. Atomicity by ordering: storage writes happen only after all three passes succeed; a mid-pipeline failure leaves the disk untouched + the next dispatch is a clean cache miss.AppServiceProviderbindsVideoEncoder→FfmpegEncoderby default (chunk-2'sNullVideoEncoderstays in the codebase as the operator-error fallback chunk 6 will use). 10 Pest tests on the encoder (3 invocations, libx264/palettegen/paletteuse flags, frame_rate plumbing, storage writes, atomicity on each-pass failure, missing-output-file guard, error message has pass name + exit code). - Per-export timeout: 5 minutes (chunk 1). The
ExportRunGifjob class declarespublic int $timeout = 300so the queue worker kills any run that exceeds 5 minutes.RenderConfig::maxDurationMs(chunk 1, default 300_000) is the renderer-internal frame-count cap; the job timeout is the wall-clock guard above that. - Results stored at
storage/app/exports/{run_id}.gifandstorage/app/exports/{run_id}.mp4(chunks 1–3). TheExportStoragehelper from chunk 1 defines the paths; theFfmpegEncoderfrom chunk 3 writes to them via the configuredgif_export.storage_disk(default: local, can be S3 in prod). - WebSocket completion event surfaces both download URLs; download UI offers a chooser (chunk 5). Backend:
ExportCompleted+ExportFailedevents broadcast on the existingprivate-runs.{id}channel (broadcast namesexport.completed/export.failed), with{ run_id, gif_url, mp4_url, frames_count, duration_ms }and{ run_id, message }respectively.POST /runs/{id}/export(ExportTriggerController) is owner-only — cache hit returns 200 + URLs + re-broadcasts ExportCompleted so other open tabs flip state; cache miss dispatchesExportRunGifand 202s.GET /runs/{id}/exports/{format}(ExportDownloadController) is owner-only, format constrained togif|mp4, serves viaStorage::downloadwithimage/giforvideo/mp4content type. The job dispatchesExportCompletedon success (including cache hit) andExportFailed(then rethrows) on renderer crash. Frontend:useExportTriggerhook POSTs to/runs/{id}/export, handles immediate-ready (cache hit) vs queued (subscribe + wait for the broadcast); state machineidle → rendering → ready | error.ExportDownloadMenuis a Popover-based chooser with three items: JSON (instant link to chunk-3 M9 endpoint), GIF + MP4 (trigger the hook + animate to download links on completion). Mounted on terminal run rows in the thread transcript + replay page header. Per chunk-5 decision, cache hits go straight to 'ready' (no flicker); WebSocket beats polling. 21 new Pest tests (6 events + 6 trigger controller + 6 download controller + 3 job event-dispatch) + 16 new Vitest (9 hook + 7 menu) + 2 integration assertions updated. Full suite: 660 pest + 398 vitest. - Renderer fallback: if
puppeteerconfigured but Chromium unavailable at boot, log warning and fall back to SVG with a "fallback engaged" badge (chunk 6). Backend:GifRendererFactory::make('puppeteer')checksChromiumDetector::isAvailable()before constructing. If the binary's missing, it logs a one-time warning (GIF_RENDERER=puppeteer configured but no Chromium binary found...) AND silently returns the SVG-bound renderer instead. The factory exposesfallbackEngaged()as a per-make()flag —ExportRunGif::handle()re-resolves the factory after rendering to read it, then includesfallback_engaged: boolin theExportCompletedbroadcast payload.ExportTriggerControllerdoes the same on the cache-hit branch so the response body + the re-broadcast both carry the flag. Frontend:useExportTriggerexposes afallbackEngagedfield;ExportDownloadMenurenders a small italic amber "(2D fallback)" badge next to the GIF and MP4 menu items when the flag is true — visible during both the 'rendering' and 'ready' states so the user knows what to expect AND knows what they're downloading. Hidden when the puppeteer driver isn't configured at all (just the default SVG path). 8 new Pest tests (3 factory + 1 event + 1 trigger controller + 3 chunk-6-specific) + 4 new Vitest (2 hook + 2 menu) — total chunk-6 additions: 12 tests. Full suite: 665 pest + 402 vitest.
Prerequisites: a Run that's reached complete or error status (the JSON / GIF / MP4 trigger UI is terminal-only — same gate as Replay). Backend deps: imagemagick (the convert binary) + ffmpeg, both standard on Linux + macOS.
Trigger an export
- From the thread detail page or the replay page, find the completed run. Click the Download dropdown next to the status badge / on the replay header.
- The menu opens with three items: JSON (instant download — chunk-3 M9 endpoint), Animated GIF (.gif), MP4 video (.mp4).
- Click Animated GIF OR MP4 video — both share the same render. The menu item flips to a spinner with "(rendering…)" while the job runs.
- When
ExportCompletedarrives on the WebSocket, both GIF and MP4 menu items become anchor links. Click either to download.
Verify cache hit
- Click Download again on the same run.
- The menu items are anchor links immediately — no spinner. The chunk-1 cache-hit short-circuit in
ExportRunGif(and the controller'sbothExistcheck) returns the URLs from disk without re-rendering. - Verify
storage/app/exports/{run_id}.gifand.mp4exist + match the timestamps of the last render.
Verify multi-tab sync
- Open the same thread in two browser tabs.
- Tab A: click Download → GIF. Wait for the render.
- Tab B (still on idle): refresh the dropdown. JSON downloads instantly; GIF / MP4 either show immediately as anchors OR (if Tab B opened the menu before Tab A's render completed) the chunk-5 broadcast flipped them.
Verify the chunk-6 Chromium fallback
- With
GIF_RENDERER=svg(the default), trigger an export — no fallback badge appears. - Set
GIF_RENDERER=puppeteerin.env, restart the workers, and trigger again on a host without Chromium installed. The menu items show "(2D fallback)" next to the format label. storage/logs/laravel.logshows:GIF_RENDERER=puppeteer configured but no Chromium binary found. Falling back to the SVG renderer for this process. Install Chromium .... The warning fires once per worker process (the chunk-6 latch); subsequent exports don't re-spam.- Install Chromium (
apt install chromium-browserorsnap install chromium) and restart. The badge stops appearing — though the actual Puppeteer-driven path still throws "Node script not found" until the chunk-4-deferred follow-up lands.
What worked
- Skeleton-first chunk 1 was the right call. Foundation (job + interface + storage + config + factory + DI bindings) landed in a single tight chunk; chunks 2–4 then swapped in concrete implementations without touching anything else. The NullRenderer/NullVideoEncoder stubs gave clear "not yet wired" failure modes during development.
- Splitting
FrameRendererfromVideoEncoderwas load-bearing. The chunk-2 architecture decision meant chunk 3 (ffmpeg) was completely orthogonal to chunk 4 (Puppeteer), and chunk 6's fallback boils down to a single match-arm swap inside the factory. If we'd built one monolithicRendererper format, each chunk would have rewritten the previous one. - Reusing the
SvgRendererorchestrator for the Puppeteer driver. The renderer-agnostic orchestrator + theFrameRendererinterface meant chunk 4's "puppeteer" wiring was ~10 lines in the factory. The chunk-6 fallback was the same lines, just with a different fallback path. ChromiumUnavailableExceptionas a distinct catchable signal. Chunk-4 decision that paid off in chunk 6: the fallback only triggers on this specific exception, leaving every other puppeteer-path failure (missing Node script, Node crashed, zero frames produced) propagating tofailed_jobsfor operator inspection.- Per-chunk decision documentation. Each chunk's decisions block tracks not just what we built but what we considered and rejected — and why. M10 alone documents 28 decisions; the rationale is now searchable and reviewable.
- Process::fake closures over real binaries in tests. Letting the test suite mock
convert+ffmpegvia the Process facade kept the build green on hosts that don't have ImageMagick (chunk 2) and on CI without the binaries (chunks 3, 5). The chunk-2 dev lesson "use string commands not arrays so Process::fake patterns + assertRanTimes closures work" carried through chunks 3 and 4.
What we learned
- Pusher tries to broadcast even with
ShouldBroadcastNowif Event::fake() isn't engaged. Chunk 5a's first integration tests against the trigger controller failed with "Pusher error: auth_key should be a valid app key" becauseExportCompletedisShouldBroadcastNow. Fix:Event::fake([ExportCompleted::class])at the top of every controller / job test that exercises the broadcast path. Documented now for future broadcast-event tests. - Pint reformats files and breaks subsequent
Editcalls. When the commit hook runs Pint, edits made within the same task window that don't match the new whitespace fail. The recovery pattern: re-Read the file, then re-do the Edit. Worth remembering when chaining many edits. escapeshellargflips array-form vs string-form Process::fake matching. Discovered in chunk 2; same pattern applies to ffmpeg in chunk 3. Always use string commands when you need to assert against them.
Parked / carry-forward to Phase 2 + future chunks
- Real Puppeteer integration is deferred per chunk-4 decision. Skeleton + fallback-detection ships in Phase 1; the actual Node frame-capture script (
node-scripts/puppeteer-export.cjs),/runs/{id}/renderLaravel route, signed-URL auth bypass for the spawned process, and frontend record mode onReplay.tsx(autoplay + hidden controls + fixed viewport +window.__exportCompletesignal) all need to land before Phase 1's "Optional Puppeteer install produces a 3D-accurate GIF + MP4" exit criterion is fully met. Phase 1 ships with the SVG renderer as the only working backend; 3D-accurate exports are a Phase 2 feature. - No S3 disk verification yet.
gif_export.storage_diskdefaults tolocal. Production deployments will likely want S3 (M14 territory). TheExportStorage+FfmpegEncoderalready useStorage::disk(config('gif_export.storage_disk'))so switching is a.envchange — but we haven't tested the S3 path end-to-end. - No browser-level smoke test of the actual GIF/MP4 output yet. The Pest suite mocks
ffmpegviaProcess::fake, so we haven't exercised the real encoding pipeline against a real PNG sequence. M11 Hardening should add a non-mocked integration test that uses the real binaries. SvgRendereris a misleading name once it also serves Puppeteer. Chunk-4 decision called this out; the rename toGifMp4Renderer(orPngSequenceRenderer) is pending. Not worth the churn yet — all consumers go through the factory — but worth doing alongside any future M11/M12 renderer additions.
Decisions (chunks 1–2):
- Skeleton-only scope for chunk 1. User choice. No real rendering yet — both
svgandpuppeteerdriver values currently resolve toNullRendererwhich throws onrender(). Trade-off: an export job dispatched today fails fast with a clear message. Benefit: the pipeline (job + renderer interface + storage + cache short-circuit + DI binding) is testable + reviewable before any pixel work lands. Chunks 2 (SVG) and 4 (Puppeteer) swap the resolution inGifRendererFactory::make()without touching the job, the storage helper, the config, or the binding. - Cache: return existing file if present. User choice.
ExportStorage::bothExist(runId)short-circuits the job when both.gifand.mp4are on disk. Repeated downloads cost a single filesystem stat; the renderer + ffmpeg cost is avoided. Partial renders (one file present, the other missing) are treated as a cache miss so the next dispatch re-renders cleanly — covers the "ffmpeg died mid-encode" edge case. - 30 FPS default. User choice. Matches the M8 animation target + the manual recipe steps. Configurable via
GIF_FRAME_RATEenv so per-deployment tuning stays out of code. TheRenderConfigvalue object exposesmaxFrames()for renderers that need a hard frame-count cap (frame_rate × max_duration_ms / 1000). - Run ID, not Run model, in the job constructor. Keeps the queue payload tiny (
['runId' => 42]) and skips any SerializesModels stale-data concern between dispatch + worker. The worker re-fetches viaRun::find($this->runId)and exits silently if the row vanished (delete-after-dispatch is the only realistic case). GifRendererinterface, not abstract class. Two implementations land in different chunks (chunks 2 + 4) with very different shapes (PHP+Imagick vs Node+ffmpeg). The interface is the minimum shared API —render(Run, RenderConfig): RenderResult. Container binding viaAppServiceProviderresolves the configured concrete; tests useapp()->instance(GifRenderer::class, $fake)to swap.- Single
tries=1+failed_jobsover retries. A failed render is operator-actionable (Chromium missing, ffmpeg path wrong, run.token_log corrupted), not transient. Retry-blasting wastes CPU. The 5-minute timeout (matches SPEC) is the wall-clock guard; failures land infailed_jobsfor inspection. - ImageMagick
convertCLI shell-out instead ofext-imagick. (chunk 2) User choice. SPEC literal text says Imagick the PHP extension; in practice the dev environment has theconvertbinary but not the extension, and every Linux deployment already has ImageMagick installed. Switching to shell-out drops theapt install php-imagickstep from every host and avoids the PHP version × Imagick version compatibility matrix. Cost is ~10ms per-frame process spawn — for a 90-frame run that's ~1s overhead; ffmpeg is shell-out anyway. - Frames sampled at frame_rate from
token_logtimestamps, not per-event. (chunk 2) User choice. 30 FPS over the last token'st_msproduces a fixed frame count regardless oflayer.advanced/moe.routeddensity. Output duration matches the original run's wall-clock duration — the GIF plays at "real time." Edge case: zero tokens → one static frame at the empty-state. FrameRenderer+VideoEncoderinterfaces (renderer architecture split). (chunk 2) User choice. Separating "produces PNGs" from "encodes PNGs to GIF+MP4" means: (a) chunk 2 stands alone with aNullVideoEncoderstub that throws clearly until chunk 3 lands the realFfmpegEncoder, (b) chunk 4'sPuppeteerFrameRendererreuses the same encoder, (c) chunk 6's fallback path (Chromium unavailable → fall back to SVG) just swaps theFrameRenderercontainer binding. Trade-off: more files, but each one has a single concern.- SVG composed as raw string concatenation, not via an SVG library. (chunk 2) PHP doesn't ship an SVG builder; pulling one in for ~5 rect / text / line element types was over-scope. The frame composition is in
SvgFrameRenderer::composeSvgas a single HEREDOC; any future template engine swap-in lives in that one method. Process::runwith a string command +escapeshellarg, not the array form. (chunk 2) Laravel'sProcess::fakematches patterns against the command string. Passing an array makes$process->commandan array at assertion time, which brokeProcess::assertRanTimesclosures with type errors during dev. String form + escapeshellarg is the simpler + more testable path.- Tmp dir cleanup in a
try { … } finally { cleanup() }. (chunk 2) Encoder failures (chunk 3 ffmpeg crash, malformed PNG) must not leak the scratch dir. Test explicitly verifies cleanup on both success + throw paths. GifRendererFactorytakes a Container in its constructor. (chunk 2) ResolvesSvgRenderervia$container->make()so itsFrameRenderer+VideoEncoder+ExportStoragedependencies get auto-wired. The factory becomes a thin driver-name → service-resolution dispatcher; future drivers add a single match arm.- Three separate ffmpeg invocations, not a single
-mapgraph. (chunk 3) User choice. Each pass has a clear concern (MP4, palette, GIF) and a clean failure surface — the exception message names which pass failed + the ffmpeg stderr. A-mapmulti-output graph would shave ~1 second per export but every filter-chain change has to consider both output streams together, and ffmpeg version differences in the multi-output path are a real maintenance hazard. Two separate processes is the operational hygiene choice. - Palette-optimized GIF (two-pass) over single-pass default. (chunk 3) User choice.
palettegen stats_mode=fullreads every pixel of every frame to build the optimal 256-color palette, thenpaletteusere-encodes withdither=bayer:bayer_scale=3. ~2-3× slower than ffmpeg's default GIF path; the output looks dramatically better on the slate-dark backgrounds + cyan particle streaks we have to render. Single-pass would visibly band the gradients. - Atomicity by ordering: storage writes after every pass succeeds. (chunk 3) The encoder writes to the configured storage disk (
gif_export.storage_disk) only after all three ffmpeg invocations have succeeded AND both tmp files exist on disk. A pass-1 failure leaves the storage untouched; a pass-2/3 failure also leaves it untouched (we don't write the MP4 until both the MP4 + GIF tmp files exist).ExportStorage::bothExist()then drives a clean cache-miss on the next dispatch — no half-baked artifacts get served from storage. file_get_contentsthenStorage::put(full file in memory). (chunk 3) The artifact size class is ~5–50MB per file; PHP memory handles this easily. Streaming chunks would be more memory-friendly but adds complexity; if exports ever grow past a few hundred MB the swap-in point is single-method.-yflag everywhere for overwrite-on-collision. (chunk 3) The tmp dir is fresh per export (chunk 2'stry { … } finallycleanup) so collisions shouldn't happen, but a flaky cleanup + immediate re-dispatch could leave a staleoutput.mp4.-ymakes ffmpeg overwrite silently rather than prompt-and-hang in stdin.- Puppeteer chunk 4 is skeleton + fallback-detection, not full integration. User choice. Real headless-Chrome rendering would add ~150MB Chromium download, a Node runtime requirement, an
/renderroute with auth-bypass for the spawned process, a frontend record-mode flow, and a frame-capture script. The skeleton's value is establishing the boundary:ChromiumUnavailableExceptionis the catchable signal chunk 6 needs; the Node shell-out command shape is stable so a follow-up chunk drops innode-scripts/puppeteer-export.cjswithout re-architecting. Phase 1 ships with the SVG renderer as the only working backend; 3D-accurate exports land in Phase 2 or driven by user feedback. ChromiumUnavailableExceptionextendsRuntimeException. (chunk 4) Distinct exception type so chunk 6's fallback cancatch (ChromiumUnavailableException $e)specifically — every other puppeteer-side failure (missing Node script, Node crashed, zero frames) raises plainRuntimeExceptionand propagates tofailed_jobs. The fallback only kicks in for the "Chromium missing" case because that's the deployment-level config issue; everything else is an operator-actionable bug.- Reusing
SvgRendereras the orchestrator for'puppeteer'too. (chunk 4) The orchestrator is renderer-agnostic — it takes anyFrameRenderer, plus the encoder + storage. The factory's'puppeteer'arm constructs anSvgRendererwith aPuppeteerFrameRendererinjected. The "Svg" in the class name is misleading; a future rename toGifMp4Rendereris an option but not yet worth the churn since all consumers go through the factory. ChromiumDetectorcached on a singleton. (chunk 4) The binary check runs once per process —isAvailable()is called fromPuppeteerFrameRenderer::renderFrameswhich fires once per export job. Caching avoids awhichshell-out +is_executablestorm on hosts that don't have Chromium.reset()exists for test ergonomics.- WebSocket completion signal over polling. (chunk 5) User choice. Matches the M6 / M8 viz pipeline shape — same channel (
private-runs.{id}), same event-name kebab convention (export.completed), same auth gate. Polling would have been simpler frontend code but added ongoing backend load whenever the menu was open. The WebSocket also enables the "other tabs flip state" behavior: a cache hit on one tab broadcasts on the channel so any other open tab subscribed to the same run sees the chooser update too. - Single chooser dropdown over multiple format-specific buttons. (chunk 5) User choice. Replaces the chunk-5 M9 single-format JSON pill with a Popover that lists JSON / GIF / MP4. Saves horizontal space on dense run rows, groups the three formats together cognitively, and lets the rendering state animate in one place rather than across multiple buttons. Trade-off: one extra click to download.
- Cache hit → immediate download (no 'Rendering...' flicker). (chunk 5) User choice. Frontend hook flips directly to 'ready' when the trigger endpoint returns 200 with URLs; only the 202 path subscribes to the channel and shows the spinner. The cache-hit branch also re-broadcasts ExportCompleted so any other open tabs (on the same run's channel) flip too.
- GIF + MP4 share a single trigger state. (chunk 5) The chunk-3 encoder produces both formats from one PNG sequence in one job invocation. The frontend mirrors this: one
useExportTriggerinstance per run; clicking either format triggers the same render; both download links light up together when the broadcast arrives. Avoids a redundant second job dispatch for the second format. - Cache hit re-broadcasts ExportCompleted from the controller. (chunk 5) The trigger controller fires the event AND returns the URLs in the response body. Belt-and-braces: the response handles the requesting tab; the broadcast handles every other tab. Without the controller-side broadcast, a cache hit on tab A would silently leave tab B's chooser still in 'idle'.
ExportFailedpayload is{ run_id, message }only. (chunk 5) No stack trace, no file paths, no PHP internals exposed to the browser. The full exception still lands infailed_jobs+ Laravel logs for operator inspection; the broadcast is just a clean user-facing signal.
Exit criteria
- Default install (SVG renderer) produces both GIF and MP4 for a 100-token run in < 30 seconds. Verified end-to-end via the chunk-3 FfmpegEncoder test path; real-binary wall-clock not measured (deferred to M11 Hardening's non-mocked integration test).
- [~] Optional Puppeteer install produces a 3D-accurate GIF + MP4. Skeleton + fallback-detection done per chunk-4 decision; actual Node frame-capture script +
/renderroute + record mode are Phase-2 / future-chunk work. Documented above. - Fallback path verified by intentionally removing Chromium. Chunk 6's
it falls back to SvgRenderer + sets fallbackEngaged when Chromium is missingPest test reproduces the exact scenario; the chunk-6 manual recipe walks through the operator-visible verification. - [~] MP4 plays in Chrome, Firefox, Safari; GIF renders inline in Slack and Discord. The
libx264 / yuv420p / +faststart(chunk 3) +paletteuse + -loop 0flags are the industry-standard set for these targets, but per-client smoke testing requires real artifacts in those clients. Verified via flag inspection in Pest; per-browser/per-client testing is M11 Hardening / M15 Launch Prep work.
Purpose: Users can share a thread read-only via a public URL.
Tasks
- Share toggle UI on thread detail (chunk 3). New
ShareMenuPopover button in the thread header next to Title / Archive / Export / Delete, per chunk-3 decision. State surface: when sharing is OFF, shows a single "Enable sharing" button that POSTs to/threads/{id}/share; when ON, shows the full share URL in a readonly input + a Copy button (usesnavigator.clipboard.writeTextwith a 2-second "Copied!" checkmark; falls back towindow.promptwhen clipboard is blocked) + a single-click "Disable sharing" button that DELETEs the same route. A small green pulse-dot indicator on the trigger button signals "Sharing is ON" without opening the popover. Per chunk-3 decision the disable path has no confirmation dialog — re-enabling is one click away if the user changes their mind (just with a fresh token per the chunk-1 always-regenerate policy).ThreadController::shownow plumbsshare_token+share_enabled_atinto the thread props;Pages/Threads/Show.tsxpasses both to ShareMenu along withwindow.location.originso the URL surfaces as a real absolute string. 9 Vitest tests on the menu + 4 integration tests onShow.test.tsx(button present, ON indicator, Enable fires router.post, Disable fires router.delete) + 2 Pest tests verifying the new thread props. - Endpoint:
POST /threads/{thread}/share(chunk 1). Owner-only. Always generates a fresh token per chunk-1 decision — re-enabling after disable produces a new URL so any accidentally-leaked old link stays dead. Setsshare_token(32 hex chars / 128 bits entropy fromShareTokenGenerator) +share_enabled_at = now(). Redirects to/threads/{id}so the toggle UI re-renders from updated Inertia props. - Endpoint:
DELETE /threads/{thread}/share(chunk 1). Owner-only. Nulls bothshare_token+share_enabled_at. Idempotent: DELETE on an already-unshared thread is a no-op redirect, not 404. - Public route:
/share/{token}— bypasses auth, IP rate-limited to 60/min (chunk 2).SharedThreadController::show($token)resolves viawhereNotNull('share_token')->where('share_token', $token); 404 on miss. Rate-limited via a newshareRateLimiter (Limit::perMinute(60)->by($request->ip())) applied to theRoute::middleware('throttle:share')->prefix('share')group — same budget covers both the thread reader AND/share/{token}/runs/{run}/replayper chunk-2 decision. - Read-only thread view (chunk 2). New
Pages/Share/Show.tsxstandalone page per chunk-2 decision (no readOnly flag on Threads/Show.tsx). Custom inlineSharedLayoutreplacesAppLayoutso anonymous viewers don't see the authenticated sidebar; layout has a brand header + a footer "What is this?" link to the about page. Sub-requirements:- No prompt input ✅ (page has no PromptFooter)
- No share toggle ✅ (controller only sends sanitized props; no header buttons)
- Replay available per run ✅ — terminal-only
Replaypill on each run row →/share/{token}/runs/{id}/replay - Prompts redacted if owner had
store_prompts = false✅ —SharedThreadControllersubstitutes[prompt redacted by author]for any null prompts when the owner'sstore_promptsis false; the page also shows a top-of-thread redaction notice
-
/share/{token}/runs/{run}/replay— public replay (chunk 2).SharedReplayControllermirrors the M9 owner replay but: (a) auth-bypassed; (b) cross-thread defense —abort_unless($run->thread_id === $thread->id, 404)so a valid token for thread A can't reach a run in private thread B; (c) 422 for non-terminal runs; (d) same prompt-redact policy. ReusesRunReplayEventSynthesizer.Pages/Share/Replay.tsxmirrorsPages/Runs/Replay.tsxbut withSharedLayoutand no download menu (M10 export endpoints are owner-only). - Copy-to-clipboard button for the share URL (chunk 3). Lives inside the
ShareMenupopover's ON state alongside the URL input. Usesnavigator.clipboard.writeText— modern path; theCheckicon flashes for 2 seconds on success. Older browsers / non-HTTPS dev contexts fall back towindow.promptso the user can manually copy. Vitest covers both paths. - Documentation link on the share page: "What is this?" → about page (chunk 4). New public
Route::get('about')→Inertia::render('About'), no auth, no rate limit.Pages/About.tsxis a standalone page (no AppLayout — anonymous viewers landing from a share link shouldn't see authenticated chrome) with four sections: (1) what LLM-Viz is, (2) what a/share/{token}link is (read-only + rate-limited + revocable), (3) privacy posture (BYOK, prompt-storage opt-out, encrypted keys,[prompt redacted by author]substitution), (4) AGPL §13 source link tohttps://github.com/CyberSecDef/llm.trackr.live. Page has a sign-in CTA + a back-to-home link. SharedLayout footers (inPages/Share/Show.tsxandPages/Share/Replay.tsx) updated: "What is this?" now points at/aboutinstead of/, and a new "Source" link sits next to it satisfying AGPL §13 from every public share page. Welcome page also gets a "What is this?" link next to the sign-in CTA for discoverability from the home route. 3 Pest tests for the route (anonymous 200, authed 200, route-name resolves). 8 Vitest tests forPages/About.tsx(headline + four section testids + body source link + footer source link + sign-in CTA + back-link). Updated existing Share/Show, Share/Replay, and Welcome Vitest tests for the new footer + nav targets.
Decisions (chunks 1–2):
- Always regenerate token on enable. (chunk 1) User choice. POST is intentionally non-idempotent: hitting it twice produces two different tokens; the first URL becomes dead the moment the second is generated. Trade-off: a user who saved a URL, toggled sharing off + back on, and revisits the saved URL gets a 404 — the right outcome for a security-sensitive operation.
- Inertia redirect over JSON response. (chunk 1) Matches the existing ThreadController patterns (title edit, archive, delete). The chunk-3 share-toggle UI will use
useForm.post/router.deleteand let Inertia re-render the thread page with updated props. ShareTokenGeneratoras an injectable service. (chunk 1) Tests bind aFixedTokenGeneratorsubclass via$this->app->instance(ShareTokenGenerator::class, $fake)for deterministic regen assertions. Matches the M10 chunk-2 FrameRenderer-fake pattern.- 128-bit entropy (16 bytes / 32 hex chars). (chunk 1) Collision probability across the lifetime of the app is essentially zero. The
UNIQUEconstraint onthreads.share_token(from the M5 migration) is the belt-and-braces guard. - Idempotent DELETE. (chunk 1) Calling DELETE on a thread that's already unshared returns the same redirect, not a 404 — a stale tab that re-issues the request after the user already toggled off shouldn't error.
- New
Pages/Share/Show.tsxpage over areadOnlyflag onThreads/Show.tsx. (chunk 2) User choice. Standalone page = no conditional logic on the live page = no risk of leaking a feature (prompt input, share toggle, title editor) into the public view by accident. Cost is a small amount of duplication for the run-row layout; the M8/M9 viz components are reused verbatim. - Separate
/share/{token}/runs/{run}/replayroute over inline modal. (chunk 2) User choice. Mirrors the M9 owner-replay shape. Reuses every viz component; only the controller + layout differ. - Single
sharerate limiter for both routes, not per-route. (chunk 2) User choice. 60 req/min per IP across the whole/share/*namespace. SPEC's literal "IP rate-limited to 60/min" reads as a budget, not a per-route quota. - Cross-thread defense in
SharedReplayController. (chunk 2)abort_unless($run->thread_id === $thread->id, 404)guards against an attacker with a valid token for thread A trying to navigate to a run in private thread B. 404 (not 403) avoids acknowledging the existence of the private run. - Backend does the prompt redaction, not the frontend. (chunk 2) The controller substitutes
[prompt redacted by author]for null prompts whenuser.store_prompts=falsebefore the payload leaves the server. Frontend just renders whatever's in the field; also includesprompts_redacted: boolso the page can show a top-of-thread notice. whereNotNull('share_token')in the lookup. (chunk 2) Defense-in-depth: even if some database state hasshare_token = nullrows that match a falsy URL, the WHERE clause filters them out. The unique constraint onshare_tokenis the primary guard; this is the runtime fallback.- No
user_id/model_id/api_key_idin the public response. (chunk 2) Matches M9's JSON-export design — the public share payload is identity-free. Themodel_snapshotinsideparametersis what the page uses fortotal_layers+architecture_type; the FK doesn't leak. Replaylink only on terminal runs. (chunk 2) Same gate the M9 replay button uses on the owner thread page. Pending/streaming runs don't have a stable token_log; absence is the signal.- Popover for the share toggle, not a modal or inline panel. (chunk 3) User choice. Matches the chunk-5 M10 ExportDownloadMenu pattern — compact trigger button in the header, click-to-open content panel. Inline would have taken vertical space even when sharing is off; a modal would have been heavier for a frequent toggle.
- Single-click disable, no confirmation dialog. (chunk 3) User choice. Re-enabling produces a fresh token per the chunk-1 always-regenerate policy, so an accidental disable is recoverable with one click — the URL just changes. A confirmation dialog would add friction for the common case (intentional disable) without protecting against the rare case (accidental click) any better than the regen-on-re-enable invariant already does.
navigator.clipboard.writeTextwith awindow.promptfallback. (chunk 3) The modern Clipboard API is available in every HTTPS context — covers production. Dev onhttp://localhostworks because Chromium treatslocalhostas a secure origin. The fallback covers the edge case where clipboard is blocked (e.g., non-HTTPS LAN IP testing) — the user still gets the URL in a prompt they can manually copy from.- Green ON-indicator dot on the trigger, no inline status text. (chunk 3) The button stays compact at "Share" with a tiny emerald pulse when sharing is on. The full state (URL, enable/disable buttons, explanatory text) lives inside the popover. Trades discoverability of the current URL (one click required) for header density (no expanded text taking horizontal space).
- New
/aboutInertia page over a#what-is-thisanchor on Welcome. (chunk 4) User choice. A dedicated route is a real destination — bookmarkable, shareable, has its own<title>, doesn't couple About content with sign-in content. Welcome.tsx is the "sign in or go away" page; About.tsx is the "here's what this is and how privacy works" page. Different audiences, different pages. - AGPL §13 source link landed now on About + SharedLayout, not deferred to M15. (chunk 4) User choice. AGPL §13 requires the source to be available from every page the app serves to a remote user. The SharedLayout and About are exactly the pages a remote (un-authed) user touches first, so landing the link with the chunk that builds those pages keeps the compliance story coherent. M15 still owns the cross-app audit (authed pages, login, error pages already had a
Sourcelink); this chunk just ensures the public-facing surface is covered up-front. - Welcome page also links to /about. (chunk 4) Discoverability — a signed-out user landing on
/from the marketing URL needs a way to read about the project without first hitting a share link. Single underline link next to the sign-in CTA, not a full nav. - No throttle on /about. (chunk 4) Cheap static render, no DB queries, no rate-sensitive resources. The
sharethrottle exists to protect the share-thread payload (which does DB work + prompt-redaction logic); the about page has neither.
Prerequisites: a thread with at least one terminal (complete or error) run — the public Replay pill only shows on terminal runs (same gate as the M9 owner replay).
Enable + visit + replay (golden path)
- As the owner, open the thread detail page. Click Share in the header action row. The popover opens with "Sharing is OFF" + a single Enable sharing button.
- Click Enable sharing. The popover content flips to "Sharing is ON" + the share URL in a readonly input + a Copy icon button + a Disable sharing button. The trigger pill gets a small green pulse-dot indicator.
- Click the Copy icon. The icon flashes a green check for ~2 seconds. Paste somewhere visible to confirm — it should look like
https://your.host/share/{32-hex-token}. - Open a private/incognito window. Paste the URL. The shared thread page renders without any login prompt — title + tags + run rows visible, no authenticated sidebar nav. The top of the page shows the "Anonymous view · shared by the author" indicator.
- On any terminal run, click the Replay pill. The
/share/{token}/runs/{id}/replaypage loads with the same M8 viz stack the owner replay uses (PlaybackControls + Viz / Embeddings / Debug tabs). Press play; the run replays frame-identical to the original. - Click Back to shared thread in the replay header — returns to
/share/{token}.
Verify the "What is this?" link + AGPL §13 source link
- From the shared thread page (or shared replay page), scroll to the footer. Two links should be visible: What is this? and Source.
- Click What is this? — lands on
/about(200, no auth required). All four sections render: "What it is", "What this share link is", "Privacy posture", "Open source". - Click the body source link OR the footer Source link → both open
https://github.com/CyberSecDef/llm.trackr.livein a new tab. - Click Sign in to make your own → lands on
/login. - Click ← Back to home → lands on
/(signed-out viewers see Welcome; signed-in viewers redirect to Dashboard).
Verify disable + token regeneration (chunk-1 invariant)
- Back as the owner: open the thread. Click Share → popover shows "Sharing is ON" with the URL (let this be URL-A).
- Click Disable sharing. The popover flips to "Sharing is OFF" immediately; the green pulse-dot disappears.
- In the incognito tab, refresh URL-A → 404 (the share token is now NULL in
threads). - Owner tab: click Enable sharing again. The popover shows a new URL (URL-B). Confirm
URL-A !== URL-B— per the chunk-1 always-regenerate decision, the old URL stays dead even if you re-enable. - (Optional) Open URL-A in incognito → still 404. Open URL-B → renders the thread.
Verify rate limit (chunk-2 share throttle)
- With a valid share URL handy, run a quick burst from one IP:
for i in $(seq 1 70); do curl -s -o /dev/null -w '%{http_code}\n' https://your.host/share/{token} done | sort | uniq -c
- Expected output: ~60 lines of
200, then429s for the rest of the burst. TheshareRateLimiter caps the whole/share/*namespace at 60 req/min/IP —/share/{token}/runs/{run}/replayshares the same budget.
Verify prompt redaction (chunk-2 store_prompts=false path)
- Sign in as a user whose Settings → Store prompts is OFF. Make sure that user has at least one thread with a terminal run.
- Enable sharing on that thread. Open the URL in an incognito window.
- The thread page shows a top-of-thread amber banner: "The author has prompts disabled — prompt text is shown as
[prompt redacted by author]." Each run row substitutes[prompt redacted by author]for the prompt; assistant outputs are preserved verbatim. Replay also redacts the prompt the same way.
Verify cross-thread defense (chunk-2 invariant)
- Owner: create two threads (T1 and T2), each with one terminal run. Enable sharing on T1 only.
- Note T1's share token + run ID. Note T2's run ID.
- Try:
GET /share/{T1-token}/runs/{T2-run-id}/replay→ 404. The controller'sabort_unless($run->thread_id === $thread->id, 404)blocks the cross-thread navigation without acknowledging that the private run exists.
What worked
- Always-regenerate token on enable simplified the entire UX. The chunk-1 decision meant the chunk-3 disable affordance didn't need a confirmation dialog: re-enable is one click and produces a fresh URL, so an "accidental" disable has zero data-loss surface. The same invariant also produces the cleanest possible "I leaked the URL, kill it now" recovery story — toggle off, toggle back on, the old URL is permanently dead. One decision; downstream effects on UX, security model, and operator playbook.
- Standalone
Pages/Share/Show.tsxover areadOnlyflag on the live thread page. Chunk-2 decision that paid off across the whole milestone. Zero risk of leaking the prompt input, the share toggle, the title editor, or the export menu into the public view via a missed conditional. The M8/M9 viz components are reused verbatim (noif (!readOnly)smell inside them); only the controller + page wrapper differ. The duplication cost is one run-row template, which is fine. - Backend-side prompt redaction. Chunk-2 decision. The controller substitutes
[prompt redacted by author]for null prompts at the data layer; the frontend just renders strings. Means there's no frontend-code path that could ever leak an un-redacted prompt —prompts_redacted: boolon the props is just for the banner, not the redaction itself. Defensive in depth. - Single
sharerate limiter for the whole/share/*namespace. Chunk-2 decision. OneLimit::perMinute(60)->by($request->ip())covers both the reader route and the replay route. SPEC said "IP rate-limited to 60/min"; reading that as a namespace budget (not per-route) keeps the operator surface to one tunable. - ShareMenu reuses the M10 ExportDownloadMenu Popover pattern. Chunk-3 decision. The Radix Popover affordance was already in the user's mental model from the export dropdown on the run rows; reusing it for share kept the header compact + cognitively familiar. The trigger button stays "Share" + a tiny indicator dot whether sharing is on or off — full state lives inside the popover, header stays dense.
- New
/aboutpage landed the AGPL §13 source link on the public surface. Chunk-4 decision. The SharedLayout + Welcome both link to/aboutnow, and the About page + SharedLayout both have the GitHub source link. AGPL §13 compliance for the unauthenticated surface is done; M15 still owns the cross-app audit for authed pages, but the externally-visible surface ships compliant. - Cross-thread defense via
abort_unless($run->thread_id === $thread->id, 404). Chunk-2 invariant. Cheap to write, but it's the difference between "valid token for thread A lets you navigate to a run in private thread B" and "404." 404 (not 403) keeps the existence of the private run unacknowledged.
What we learned
- Vite manifest stale on new
Pages/*.tsx. Hit this twice — chunk 2 (Share/Show.tsx + Share/Replay.tsx) and chunk 4 (About.tsx). Pest's Inertia view-finder uses the production Vite manifest; new page files don't appear untilnpx vite buildregenerates it. Canonical fix: rebuild between adding a page file and running Pest tests that render the page. Worth a quick check-and-rebuild pattern in any milestone that introduces new Inertia page components. - jsdom doesn't ship
navigator.clipboard. Chunk-3 dev lesson. The standard mock pattern isObject.assign(navigator, { clipboard: { writeText: vi.fn().mockResolvedValue(undefined) } })inbeforeEach. Failing-clipboard tests needmockRejectedValue(new Error('blocked'))to exercise thewindow.promptfallback path. vi.hoistedInertia router mocks need every method the component calls. Chunk-3 ShareMenu added arouter.postcall on a mock that previously only hadget/patch/delete(the Threads/Show.test.tsx fixture). Symptom was "router.post is not a function" in the integration test. Fix: extend the hoist block + the spread mock + themockReset()inafterEach. Worth treating as a checklist item any time a component starts using a new router method.- ESLint
react/no-unescaped-entitiescatches typographic punctuation in JSX text. Chunk-4 — the pre-commit hook rejected"bring your own key"andcan'tin About.tsx text. Mechanical fix:'/“/”. Worth keeping in mind whenever I write user-facing prose directly in JSX rather than in at()call. - Pint reformat still breaks chained Edits. Same lesson the M10 retrospective documented. Hit again in chunk 3. The pattern is: when a commit hook reformats a file, in-flight Edit handles from before the reformat have stale
old_stringcontent. Re-Read after Pint runs, then re-Edit.
Parked / carry-forward to M15
- Authed pages don't link to
/aboutyet. Per the chunk-4 scope, onlyPages/Welcome.tsx,Pages/Share/Show.tsx, andPages/Share/Replay.tsxlink to it. The authenticated app (Dashboard, Threads, Settings, Admin) has no "About" / "What is this?" affordance. M15's AGPL §13 compliance audit task is the right owner — add a footer link to AppLayout there, audit every served page for the source link, and decide whether to surface About from the user menu. - No browser-level E2E for the enable → incognito → disable → 404 flow. Same call M6, M8, M9, and M10 closeouts all made: Playwright/Cypress lands in M15 Launch Prep, not now. The Pest + Vitest coverage validates the data plane + the component behavior; the manual recipe above is the human verification path until E2E ships.
- Shared replay doesn't subscribe to new runs. A share URL is a snapshot at view-time — new runs the owner submits show up on refresh, not via WebSocket. Out of scope for M11 by design (anonymous WebSocket subscriptions on
private-runs.{id}would require a separatechannels.phpauth route that doesn't gate on user identity). If someone asks for "live shared threads," it's Phase 2 / Phase 3 territory. - No JSON-import endpoint for shared payloads. M9 chunk 3 parked the import side; M11 doesn't add it either. A future "paste a
/share/{token}URL into your own dashboard to clone the thread" UX would need both the JSON-import endpoint AND a/share/{token}/export.jsoncompanion to the existing owner-only export. Phase 2 candidate.
Exit criteria
- Toggling share on a thread produces a
/share/{token}URL that works in an incognito window. Covered programmatically byThreadShareControllerTest(POST setsshare_token+share_enabled_at+ redirects) +SharedThreadControllerTest(the public/share/{token}route returns 200 + InertiaShare/Showfor the resolved token, with sanitized props). Manual recipe above §"Enable + visit + replay" is the human verification path. - Replay works on the shared view. Covered programmatically by
SharedReplayControllerTest(200 + InertiaShare/Replayfor valid token + run + terminal status; 422 for non-terminal; 404 for cross-thread; cross-thread defense) + the VitestPages/Share/Replay.test.tsx(renders PlaybackControls + viz pane + back-link). Manual recipe above step 5 (incognito play) is the human verification. - Disabling sharing returns 404 for the old URL. Covered programmatically by
ThreadShareControllerTest(DELETE nulls both columns + idempotent for already-unshared threads) +SharedThreadControllerTest's 404-on-missing-token assertion. The chunk-1 always-regenerate decision strengthens this: even re-enabling produces a fresh token, so the old URL stays 404 forever. Manual recipe above §"Verify disable + token regeneration" is the human verification.
Bonus invariants from the implementation:
- Cross-thread token navigation returns 404, not 403. Chunk-2 invariant. Pest
SharedReplayControllerTest::cross-thread defensecovers it. - Prompts are redacted for
store_prompts=falseowners. Chunk-2 invariant. Pest covers both the redaction substitution and theprompts_redacted: boolflag on the page props. - Rate limit applies across the whole
/share/*namespace. Chunk-2 invariant.AppServiceProvidershareRateLimiter is registered + applied inroutes/web.php; the manual recipe above exercises it from the operator side. -
/aboutis reachable for both anonymous and authed users. Chunk-4. PestAboutPageTestcovers both auth states.
Purpose: WCAG 2.1 AA compliance + general UX polish.
Tasks
- [~] Full WCAG 2.1 AA audit using axe-core in CI + manual screen-reader pass (NVDA + VoiceOver). Chunk 1 lands the CI half:
jest-axewired into Vitest viaexpect.extend(toHaveNoViolations)inresources/js/test/setup.ts; a shared runner atresources/js/test/axe.tsconfigures axe-core to run the fullwcag2a / wcag2aa / wcag21a / wcag21aarule set with two jsdom-incompatible rules disabled (color-contrast— jsdom returnsrgba(0,0,0,0)for most computed colors, would false-positive everywhere;scrollable-region-focusable— same root cause, depends on computed overflow). A newresources/js/__tests__/a11y/directory holds the page-level audits; chunk 1 ships audits for the four pages named in the SPEC exit criteria (Dashboard,Threads/Show,Share/Show,Share/Replay), each across 2–3 meaningful states (empty / loaded / archived / redacted / errored). 11 new Vitest tests; zero structural violations on the first run across all four pages. Hard-fail CI gate: any future regression breaks the build. Manual screen-reader pass + color-contrast verification deferred to the M12 cross-browser smoke chunk (matches the chunk-1 visual-a11y decision below). - Keyboard navigation: all viz controls operable without mouse; visible focus rings (chunk 2). Per chunk-2 audit, the codebase's reliance on shadcn/Radix primitives (Button, Popover, Dialog, Sheet, Command, Slider, Textarea, AlertDialog) already covered the keyboard-reachability + focus-trap + Escape-to-close story for ~95% of interactive elements — the M7 frontload-shadcn decision paid off again. Found 5 hand-rolled hot spots without
focus-visible:rings: the right-pane tab toggles inPages/Threads/Show.tsx,Pages/Runs/Replay.tsx, andPages/Share/Replay.tsx(9 buttons total — Visualization / Embeddings / Debug); thePlaybackControlstoolbar inComponents/PlaybackControls.tsx(play/pause + step + cursor-jump-to-live + 4 speed buttons = 7 buttons); and the archive filter tablist inPages/Threads/Index.tsx(3 buttons + missingtype="button"+ missing tablistaria-label). Applied the standardfocus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 focus-visible:ring-offset-backgroundpattern to all 19 buttons. Addedaria-label="Right pane view"to the Share/Replay tablist (the other two already had it) andaria-label="Archive filter"+type="button"to the Threads/Index tablist. 7 new Vitest keyboard tests inresources/js/__tests__/a11y/keyboard.test.tsx: PlaybackControls Tab order, Space toggles play, Enter steps, speed-button focus-visible coverage, ShareMenu Enter-opens-popover, ShareMenu Escape-closes-and-returns-focus (Radix-provided but verified to survive jsdom), focus-visible class assertions on the playback buttons. Manual keyboard-only vertical-slice recipe documented below — closes the second SPEC exit criterion. - ARIA labels on canvases (Three.js scene gets a textual description that updates on layer-advance) (chunk 3). Audit found: the SVG charts (
AttentionHeatmap,MoERouting) already hadrole="img"+ descriptive aria-labels from M8 chunks 5b/6; the Three.js canvases (VizPane,EmbeddingScene) had static aria-labels but no live-updating announcer. Chunk 3 added a hidden<span role="status" aria-live="polite" className="sr-only">adjacent to each Three.js<canvas>whose text is derived by the new pure helperresources/js/lib/vizAnnouncement.ts. Per chunk-3 decision the announcer fires per-decile token milestones (10/20/30 tokens generated…), not per token/layer/event —aria-live="polite"queues but does not drop, so a 100-token stream would otherwise announce 100 lines back-to-back. State transitions covered: empty → "Run started." (Viz) / "Embedding scene loaded." (Embedding), every 10 tokens → "N tokens generated.", completion → "Run complete. N tokens generated.", error → "Run errored: {message}." Errors take precedence over completion + milestones.LogitsDistributionwas inspected too — it's a<ul>with readable percentage text + already-accessible list semantics; no aria gap. 8 new Vitest tests inresources/js/__tests__/lib/vizAnnouncement.test.tscover every transition + pluralization + error-precedence invariant. -
prefers-reduced-motionhonored (stepped static frames instead of continuous animation) (chunk 3). M8 chunk 9 already shipped the headline gate (right-pane Viz tab is disabled whenuseReducedMotion()returns true, so the Three.js scene never mounts; users land on the Debug tab fallback). Chunk 3 closed the remaining gaps for animation paths that didn't go through that gate: 6animate-pulse/animate-spinTailwind classes (PlaybackControlsLIVE-pillRadioicon,ExportDownloadMenuLoader2rendering spinner, streaming-cursor▍onThreads/Show,Runs/Replay,Share/Replay, and the streaming-status indicator dot in the thread transcript) all got themotion-safe:Tailwind variant prefix. The prefix compiles into@media (prefers-reduced-motion: no-preference)— animations run when the user hasn't opted out and stop running when they have. 2 new Vitest assertions inresources/js/__tests__/a11y/motion.test.tsxlock the pattern: regression-safe against a future contributor pasting plainanimate-pulseback in. - Color-blind palette check on heatmaps + MoE bars (viridis / cividis as defaults) (chunk 4). New
resources/js/lib/palettes.tsexports two canonical CB-safe palettes:VIRIDIS_STOPS(5-stop perceptually-uniform sequential, used forAttentionHeatmap) +OKABE_ITO(7-chromatic + white-neutral categorical, used forEmbeddingScenecluster identities).AttentionHeatmap: dropped the M8 slate-950 → cyan-300 single-hue ramp, switched to a 5-stop multi-stopscaleLinearover viridis; aria-label updated to "Attention heatmap (illustrative, viridis palette)" so screen-reader users know the palette they're inspecting.embeddingClusters.ts: rewrote all 8 cluster colors to map 1:1 against Okabe-Ito (the 8th cluster —code-symbols— uses the swapped-in white neutral instead of the original Okabe-Ito #000000, since pure black would blend into the slate-950 scene background). Two near-duplicate cyan shades + two near-duplicate amber shades from the old palette are gone — every cluster is now perceptually distinct under deuteranopia, protanopia, and tritanopia.MoERouting+LogitsDistributionaudited and left as-is: their bars use single-hue magnitude encoding (length → value, color is decorative); WCAG 2.1 SC 1.4.1 "Use of Color" is satisfied because the magnitude is in the length, not the hue. Status pills (complete/streaming/error) similarly disambiguate via text labels next to the color. 8 new Vitest tests:palettes.test.ts× 5 (canonical stops, domain pairing, no duplicates, EmbeddingClusters all match Okabe-Ito, banned M8 hexes absent) +AttentionHeatmap.test.tsx× 1 new (palette + aria-label) + 2 existing tests updated for viridis low-stop assertion. - Empty states for: no threads, no API keys, no runs in a thread (chunk 5). Audit confirmed all three SPEC-named states already existed from M7 (the chunk-1 frontload-shadcn decision paid off again) — but two were noticeably terser than the rest:
Threads/Show.empty-transcript(single line of muted text, no icon) andApiKeys/Index.empty-keys(single line in a table cell, no icon). Both were polished to match the icon + title + body pattern the Dashboard already used:empty-transcriptgot aChevronDownicon directing the eye to the prompt input below + a bolder "No prompts yet" title;empty-keysgot aKeyRoundicon + a "No API keys yet" title + a "Bring-your-own-key — we charge nothing" tagline. The full inventory of locked-in empty states is now: no API keys →Dashboard.no-api-key-callout(full-page amber callout + "Add a key" CTA),Threads/Show.no-api-key-footer(prompt-footer amber strip + "API Keys" CTA),ApiKeys/Index.empty-keys(table empty row + icon + body); no threads →Dashboard.empty-threads(icon + title + dynamic CTA perhasApiKeys),Threads/Index.empty-all(icon + title + body, page header has the "New thread" CTA),Threads/Index.empty-filtered(icon + title + "Clear filters" button); no runs in a thread →Threads/Show.empty-transcript(icon + title + "type your first one below" guidance),Share/Show.shared-empty(acknowledgment only — read-only viewer can't act). 7 new Vitest tests inresources/js/__tests__/a11y/empty-states.test.tsxlock the icon-present + title-text + CTA-href invariants across all of them; regression-safe against a future contributor downgrading any state back to bare text. - Loading states + skeletons (chunk 6). New
Components/ui/skeleton.tsx(shadcn standard one-liner —motion-safe:animate-pulse rounded-md bg-muted, gated under prefers-reduced-motion per chunk-3) + newComponents/VizSkeleton.tsxshared wrapper used by all 6 Three.js Suspense fallbacks (Pages/Threads/Show.tsx× 2,Pages/Runs/Replay.tsx× 2,Pages/Share/Replay.tsx× 2). The wrapper reserves anaspect-squareslot matching the rendered canvas footprint — zero layout shift when VizPane/EmbeddingScene mounts — and carriesrole="status"+aria-labelso screen-reader users hear "Loading visualization" or "Loading embedding scatter" without waiting for the WebGL canvas to actually render. 4 new Vitest tests onVizSkeleton(container + skeleton-child + caption text + motion-safe gating). - Toast notifications for errors and success (chunk 6). Installed
sonner(the canonical toast lib shadcn ships); newComponents/ui/sonner.tsxwraps it with our dark-slate theme (top-right position,richColors,closeButton, theme tokens).<Toaster />mounts once at the root ofLayouts/AppLayout.tsx. Newhooks/useFlashToast.tswatchesusePage().props.flash?.statusand translates known Laravel-session statuses into the right toast variant (settings-saved→ success,api-key-added→ success,api-key-deleted:{vendor}→ info,refresh-complete:{msg}→ success,model-updated:{name}→ success,model-deleted:{name}→ info,rate-limit-updated:{userId}→ success).AppLayoutinvokes the hook so every authed page gets toasts for free. Removed the inline emerald/amber flash blocks fromPages/ApiKeys/Index.tsx(api-key-added/api-key-deleted) — they would have double-announced alongside the toasts. Also addedtoast.success()/toast.info()calls insideShareMenufor enable / disable / copy actions (previously silent — the popover state was the only feedback). 6 new Vitest tests onuseFlashToastcover the 4 known status patterns + the no-flash + unknown-status graceful degradation paths. - 404 / 500 pages styled (chunk 7). Audit confirmed all 4 error pages existed from M7 (
NotFound,ServerError,Expired,Forbidden) sharing anErrorShellcomponent. Chunk 7 polished them: (a) split 503 out ofServerErrorinto a new dedicatedMaintenance.tsx("We'll be right back" — 503 has different semantics from a logged exception; users can't retry sooner, the situation clears itself), withbootstrap/app.phprouting 503 →Errors/Maintenanceseparately; (b) added alucide-reacticon to each page (Compass/ShieldAlert/Clock/AlertTriangle/Wrench) via a new optionaliconprop onErrorShell; (c) added a "Go back" secondary CTA that callswindow.history.back()(falls through to the primary CTA destination when history is empty — direct deep-link hits). Primary CTA still flips between "Sign in" / "Back to dashboard" depending on auth; AGPL §13 Source link remains. 2 new Pest tests (Errors/Maintenancefor 503,Errors/Expiredfor 419 — previously only 404 / 403 / 500 had Pest coverage); 26 new Vitest tests inresources/js/__tests__/Pages/Errors/ErrorPages.test.tsxusingdescribe.eachover all 5 error pages (status line + headline + icon + Go-back CTA + Source link + Sign-in CTA × 5 pages = 25, plus 1 Go-back behavior test). - Cross-browser testing: Chrome, Firefox, Edge, Safari (latest two) (chunk 9). Documented as a manual recipe (below) — the four target browser families × latest two versions each = 8 build targets. Per the existing M6/M8/M9/M10/M11 closeout pattern, automated Playwright/Cypress E2E is parked for M15 launch prep; chunks 1–8 cover the structural a11y story via jsdom + axe + jest-axe, and this recipe gives the human verification path for the visual / runtime concerns that jsdom can't model (real focus rings, prefers-reduced-motion behavior, color-contrast ratios, actual WebGL 2 capability, fingerprinted clipboard/IntersectionObserver/etc.).
- WebGL 2.0 detection + clear "unsupported browser" message with fallback to 2D-only viz (chunk 8). New
resources/js/lib/webgl.tsis the canonical capability probe: creates a throwaway canvas, callsgetContext('webgl2'), returns the boolean. Cached per-process — GPU capability doesn't change during a page lifetime. SSR-safe (returnsfalsewhendocumentis undefined so the server-rendered HTML is the safe fallback; the client hook re-probes on mount and corrects upward). Newresources/js/hooks/useWebGL2Support.tswraps the probe in a stateful hook with an optimistic initial value oftrue(matches the supported-runtime majority — initialfalsewould mount the fallback first and then yank the user back to the Three.js tab after hydration). The 3 viz pages (Pages/Threads/Show.tsx,Pages/Runs/Replay.tsx,Pages/Share/Replay.tsx) now compose the WebGL gate with the existing reduced-motion gate: Viz tab disabled whenreducedMotion || !webgl2Supported; Embeddings tab disabled when!webgl2Supported(reduced motion alone keeps the embedding cloud usable because its auto-orbit pauses on spotlight per M8 chunk 7); default mode flips toDebug(the existing 2D text fallback from M6) when either is the case; an amberwebgl-unsupported-noticebanner explains the situation below the tablist when WebGL 2 is missing. 10 new Vitest tests (4 helper + 2 hook + 4 fallback integration on Threads/Show). One global test-suite shim:resources/js/test/setup.tsmocksuseWebGL2Supportto returntrueby default so existing page tests (which assume the Three.js stub mounts) keep working; the newwebgl-fallback.test.tsxoverrides the mock locally to exercise the unsupported path.
Decisions (chunk 1):
jest-axeovervitest-axeor a custom matcher. User choice. jest-axe is the mature path — large user base, axe-core 4.x under the hood, thetoHaveNoViolationsmatcher is a plain{pass, message}shape that consumes via Vitest'sexpect.extend(...)API. Thevitest-axepackage is a smaller community wrapper around the same axe-core; rolling our own matcher would have been ~10 lines but added a dependency we'd own without payoff. The TypeScript surface comes from@types/jest-axe+import 'jest-axe/extend-expect'inresources/js/types/global.d.ts.- Hard-fail CI gate from day one, not report-only. User choice. Any axe violation in any a11y test fails Vitest, which fails CI. Forces fixes in-chunk rather than letting a backlog accumulate. The trade-off — false-positives could block a PR — is acceptable because the jsdom-safe rule config (
color-contrast+scrollable-region-focusabledisabled) eliminates the two rules that fire spuriously under jsdom; everything else axe checks (labels, ARIA roles, headings, landmarks, name/role/value, alt text, button-has-name, document-title) is deterministic and meaningful. - Dedicated
__tests__/a11y/directory + per-page test files, not assertions tacked onto existing tests. Existing page tests already carry complex fixture state (Inertia hoist mocks, fetch stubs, stream-state mocks). Co-locating axe assertions would tangle the a11y signal with the unit-test signal. Per-page files mean each a11y file owns the minimal fixtures needed to render the page in its meaningful states; the a11y file becomes a clean "is this page accessible?" signal independent of the unit-test surface. - Test multiple page states, not just the default render. Each a11y file covers 2–3 states: empty (no data) / loaded (realistic data) / variant (archived / errored / redacted). Empty states have their own a11y surface (different headings, different landmarks, empty-state CTAs); variants like "archived" introduce status badges that need accessible names. Testing only the loaded state would miss those code paths.
color-contrast+scrollable-region-focusabledisabled under jsdom; visual a11y deferred. jsdom doesn't compute CSS;getComputedStylereturnsrgba(0,0,0,0)for most elements, so contrast checks would false-positive every element with Tailwind classes. The visual a11y story (real contrast ratios, focus-ring visibility,prefers-reduced-motionbehavior in real browsers) needs a Playwright +@axe-core/playwrightpass or a manual cross-browser audit — both deferred to a later M12 chunk or M15. The structural a11y story (everything else axe-core checks) runs hot in jsdom and catches regressions just fine.- Run the full
wcag2a / wcag2aa / wcag21a / wcag21aatag set, not justwcag2aa. The SPEC says "WCAG 2.1 AA". Running A + AA at both 2.0 and 2.1 levels covers the full conformance target. Best-practice rules (best-practicetag) are intentionally NOT included — they catch UX-quality issues that aren't WCAG conformance failures, and we don't want a "page has heading levels skipping" best-practice rule (which axe flags) to gate CI when it isn't a WCAG violation. - Skeleton chunks: chunk 1 is axe + four pages only. SPEC's M12 task list has 11 items; bundling them all into one chunk would be too much surface area. Chunk 1 lands the harness + the SPEC exit-criteria pages. Subsequent chunks own the remaining tasks (keyboard nav + focus rings, ARIA on canvases,
prefers-reduced-motionaudit, color-blind palettes on heatmaps, empty states, loading/skeletons, toasts, error pages, cross-browser smoke, WebGL detection + 2D fallback).
Decisions (chunk 2):
- Surgical focus-visible fixes, not a sweeping refactor. User choice. Audit found 5 hand-rolled hot spots (3 right-pane tablists + PlaybackControls toolbar + Threads/Index archive filter); fixed each in place by appending the standard
focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 focus-visible:ring-offset-backgroundclass chain. Trade-off: a future contributor adding a new bare<button>outside the shadcnButtonprimitive could miss the convention. Mitigations: (a) jest-axe in CI catches thebutton-name/labelfailures; (b) theeslint-plugin-jsx-a11yrule set (already in devDependencies) catches a lot of this at lint time; (c) a follow-up M12-or-later chunk could define a.focus-ringutility class to make the pattern even shorter — out of scope for this surgical fix. - Kept the custom hand-rolled tablists; did NOT migrate to
@radix-ui/react-tabs. The three right-pane tablists (Visualization / Embeddings / Debug) and the Threads/Index archive filter could be replaced with Radix Tabs to inherit arrow-key navigation between tabs (WAI-ARIA Authoring Practices guidance). WCAG 2.1 AA's "operable without mouse" is already satisfied because each tab is independently Tab-focusable + Enter-activatable; arrow-key navigation is best practice but not a conformance failure. Migrating would be ~150 lines of churn across 4 pages + new Vitest fixture updates — saved for a later polish chunk if anyone files it. - Vitest keyboard tests, not Playwright. Same call M6/M8/M9/M10/M11 closeouts all made: Playwright lands in M15, not now.
@testing-library/user-eventalready in the dev deps + Vitest's jsdom environment is enough to verify Tab order + Space/Enter activation + Escape close + focus return. Visual focus-ring rendering is verified only via the class-name assertion (the manual recipe is the human verification path until real-browser E2E lands). Object.defineProperty(navigator, 'clipboard', ...)overObject.assign(navigator, ...). Dev lesson: jsdom 29 protectsnavigator.clipboardwith a getter when an earlier test file in the same run has already defined it.Object.assignthen throws "Cannot set property clipboard of #<Navigator> which has only a getter."Object.definePropertywithconfigurable: true, writable: truebypasses the getter. The ShareMenu chunk-3 test got lucky because it ran first in its old test order; the chunk-2 keyboard test exposed the issue. Now documented so future a11y tests get the pattern right out of the gate.- Tablist
aria-labelaudit landed alongside the focus-ring fix. Threads/Index'srole="tablist"had noaria-label(axe doesn't flag this — thearia-allowed-attrrule only catches invalid attrs, not missing labels — but it's a real screen-reader gap). Addedaria-label="Archive filter". Share/Replay's tablist had the same gap; addedaria-label="Right pane view". Threads/Show + Runs/Replay already had it (M8 chunk 8). Now all four tablists are screen-reader-discoverable.
Decisions (chunk 3):
- Per-decile milestone announcer, not per-event. User choice.
aria-live="polite"queues but does NOT drop announcements — a 100-token stream would otherwise pile up 100 announcements in the AT queue and a screen reader would read them all in sequence. Per-decile milestones produce a useful narrative ("Run started. … 10 tokens generated. … 30 tokens generated. … Run complete.") without the flood. Trade-off: a 1–9-token run never moves off "Run started." until completion — acceptable, the completion announcement covers it. - Pure helper (
deriveVizAnnouncement) over inlineuseMemoin each canvas component. The same announcement logic now servesVizPaneANDEmbeddingScene(with a differentstartedLabelfor each — "Run started." vs "Embedding scene loaded."). Extraction also makes the function testable without spinning up Three.js / WebGL stubs — 8 Vitest assertions hit the helper directly, covering every state transition + pluralization + error-precedence invariant. - Errors take precedence over completion or milestones. If an
run.erroredevent is in the array, the announcement is the error message regardless of how many tokens streamed before the error or whetherrun.completedalso appears (the latter shouldn't but defensive ordering matters). Screen reader users hear about the failure first; the token count is implicit in the prior milestones they already heard. role="status" aria-live="polite"on a<span className="sr-only">, not<div>. Both are valid host elements for the role;<span>is inline-by-default and matches the surrounding canvas-wrapper's flow. Thesr-onlyTailwind utility positions it off-screen viaposition: absolute; width: 1px; height: 1px; clip: rect(0 0 0 0)— invisible to sighted users + readable to screen readers.role="status"is the implicit role foraria-live="polite"per WAI-ARIA; being explicit aids older AT implementations.motion-safe:prefix, notuseReducedMotion()runtime checks. User choice (implicit — surgical preference from chunk 2 carries forward). Tailwind'smotion-safe:variant compiles to@media (prefers-reduced-motion: no-preference); the browser handles the media-query evaluation natively, no JS overhead, no re-render needed when the preference flips. TheuseReducedMotionhook is reserved for cases where a component must structurally change under reduced motion (M8's right-pane gate that swaps the Viz tab for Debug). Inline animations (pulse, spin) just shouldn't animate — CSS handles that cleanly.LogitsDistributionleft as-is — already accessible. It's a<ul>of percentage labels + token strings, not an<svg>or<canvas>. Screen readers read each list item + the percentage text natively. Addingrole="img"would have reduced accessibility by treating the list as a single opaque image. Axe is happy with the current shape; M12 chunk 1 already verified this on Threads/Show.- Three.js
<canvas>keeps its descriptive aria-label; we did NOT change it dynamically. Per the user's chunk-3 decision the live state goes in the adjacentaria-liveregion, not on the<canvas>itself. Rationale: some screen readers don't re-announce a changedaria-label; a dedicatedrole="status"region is the AT-stable pattern.
Decisions (chunk 4):
- Viridis for sequential, Okabe-Ito for categorical. User choice (matches SPEC wording on the sequential side, picked separately for categorical). Viridis is the canonical perceptually-uniform sequential gradient (van der Walt + Smith, matplotlib 2.0); it's monotonic-in-luminance under sRGB AND under the three common color-vision deficiencies, so a darker cell is always "less" and a lighter cell is always "more" regardless of who's viewing. Cividis (the explicit CB-only variant) was considered; viridis won because it retains better contrast for non-CB users too. Okabe-Ito (2008) is the academic standard CB-safe categorical palette specifically designed so every pair of stops remains distinguishable under deuteranopia / protanopia / tritanopia.
- Hardcoded 5-stop viridis array, no
d3-scale-chromatic. Could have pulled ininterpolateViridisfromd3-scale-chromatic(a separate npm dep). Instead the 5 canonical stops are inlined as areadonly string[]constant; the existingd3-scalescaleLinearalready handles multi-stop ranges. Trade-off: 5 stops instead of the full 256-color continuous viridis. Acceptable because the matrices we render are 32×32 max and 5 stops with d3's RGB interpolation between them is visually indistinguishable from the continuous version at that resolution. - 8th cluster slot uses white, not the Okabe-Ito #000000. The original Okabe-Ito set lists
#000000as the 8th color, but ourEmbeddingScenebackground isslate-950(#020617) — pure black would render invisible. White is on the lightness axis perpendicular to the 7 chromatic Okabe-Ito stops, so it remains distinguishable from all 7 under any CVD. Documented inline inpalettes.tsso a future contributor reading the code understands why we diverged from the canonical Okabe-Ito 8th. - Did NOT change MoERouting / LogitsDistribution / status pills. These already satisfy WCAG 2.1 SC 1.4.1 ("Use of Color"): MoE + Logits bars encode magnitude in length, not hue (color is decorative); status pills pair their color with a text label. CB-safe by construction, no migration needed. Explicit decision recorded so a future audit doesn't sweep them up.
- EmbeddingScene's
EMBEDDING_CLUSTERSarray order kept stable. Cluster names (whitespace, punctuation, pronouns, common-english, long-words, numbers, code-keywords, code-symbols) keep their positions; only thecolor:field changed. Order-stable migration so existing fixtures + integration tests that index into the array still resolve correctly (none do directly, but defensive choice). - Old hex test in
AttentionHeatmap.test.tsxupdated, not deleted. The M8 test that assertedrgb(2, 6, 23)for zero-weight cells now assertsrgb(68, 1, 84)(viridis low stop). Updating rather than deleting keeps the test's intent (causal-masked cells use the low stop) and only updates the constant. Plus an explicit "banned old hexes" assertion locks the migration: a contributor pasting the old cyan-300 back in would fail the test. aria-labelcalls out the palette name. Updated to "Attention heatmap (illustrative, viridis palette)". Screen-reader users who care about palette choice (data-viz academics, color-blind users explicitly verifying the safe choice) hear it; everyone else gets a slightly longer but still-readable label. Trade-off: cluttering one extra word into the label is worth the explicit a11y signal.
Decisions (chunk 5):
- Audit + polish only the gaps, not a sweeping redesign. Carries forward chunk-2's surgical-fix preference. All three SPEC-named states already existed; chunk 5 just upgraded the two terser ones (
Threads/Show.empty-transcript,ApiKeys/Index.empty-keys) to the icon + title + body pattern the Dashboard versions already used. The other six existing states (no-api-key-callout,no-api-key-footer,empty-threads,empty-all,empty-filtered,shared-empty) were already on-pattern — no churn. - No CTA on
Threads/Index.empty-all(the page-level header has one). Threads/Index already shows a "New thread" submit button in the page header. Adding a duplicate button to the empty-state card would feel redundant + give the user two near-identical actions to pick between. Dashboard'sempty-threadsdoes have its own button because Dashboard has no page-level CTA. Asymmetry by design; documented so a future reviewer doesn't try to "fix" it. - No CTA on
Share/Show.shared-empty. Read-only public viewer has no action to take when a shared thread has no runs yet — the thread's owner is the only person who can change the state. The empty state just acknowledges the situation; the SharedLayout footer already provides the "What is this?" link. - Did NOT upgrade
no-usable-models(the bonus state). Out of SPEC scope. The state fires when a user has an API key but the registry doesn't carry any model for their vendor — a registry-staleness edge case. The existing text mentions "an admin can refresh the registry from the Models page" but doesn't link there because non-admin users can't access that route anyway. Parking as a M15 launch-prep follow-up: surface a "ping an admin" affordance or auto-trigger the registry-refresh fetch when an authed user lands on this state. ChevronDownicon onempty-transcript, not the more genericMessageSquare. The chevron is a directional affordance — it points downward toward the prompt input that's literally rendered immediately below the empty state card. Subtle but useful: a screen-reader user hears "type your first one in the input below" while a sighted user follows the icon's arrow to the input. Function over decoration.- Lock empty-state structure in tests, not just
getByTestIdpresence. The existing M7 tests on the empty-state testIds only verified the element rendered. Chunk 5 adds assertions that there's an<svg>icon child AND a title text node AND (where applicable) a CTA<a href>with the expected target. Future regressions that delete the icon or downgrade the title pass the old tests but fail the new ones — strictly more coverage at marginal additional cost.
Decisions (chunk 6):
sonnerover a hand-rolled toast. It's the toast library shadcn endorses, has Radix-style a11y defaults (role="status" + aria-live="polite"), supportsrichColors/closeButton/darktheme out of the box. Building our own would have been ~80 lines of context-aware portal management + an exit transition + accessibility wiring — buying that for one dep is the right trade. Same call M10's chunk-2 made when it pickedimagemagick convertover hand-rolled SVG rasterization.- Single Toaster mounted in AppLayout, not per-page. The Toaster is a portal; multiple instances would duplicate every notification. AppLayout already wraps every authed page so it's the natural root. The Welcome / Login / Errors / Share pages don't use AppLayout — they don't currently have toast-worthy state changes either, so no gap.
useFlashToasthook overtoast()calls in each controller's redirect handler. Flash data lands as a prop on the next Inertia visit; pages would otherwise have to wire their ownuseEffectto react. Centralizing the prop → toast mapping in one hook makes the vocabulary discoverable AND keeps each page's render pure (no side effects scattered across components). The hook uses auseRefto track the last-fired status so re-renders with unchanged flash don't fire duplicate toasts.- Removed inline flash blocks instead of keeping both. The chunk-6 audit found the inline emerald/amber blocks in
ApiKeys/Index(and elsewhere) would double-announce with the new toasts. Kept the toasts; deleted the inline blocks. Cleaner UX (one source of feedback, not two competing) + smaller pages. Pages that I haven't touched in this chunk (Admin/Models/Index, Admin/Users) still have their inline blocks; they're parked for chunk-7 polish or a separate sweep — theuseFlashToasthook is opt-in by virtue of AppLayout coverage, so toasts already fire there even without removing the blocks (so admins see both until we get to those pages). VizSkeletonis its own component, not inline JSX in each Suspense fallback. Three pages × two fallbacks each = 6 spots. Six copies of the same inline JSX would have drifted out of sync. The single component lets the M8 viz pages share a unified loading shape — same aspect-square footprint, same caption strip, same a11y wiring.role="status" aria-labelon the loading container, not on the Skeleton element itself. The Skeleton primitive is decoration; the container is the semantic loading region. Putting the role on the container also means screen readers don't announce the pulsing skeleton's visual changes (which would happen ifrole="status"were on the Skeleton, since aria-live regions announce DOM mutations).- Skeleton inherits
motion-safe:animate-pulsefrom chunk 3's pattern. Single source of truth: the chunk-3 reduced-motion decision said "allanimate-*classes getmotion-safe:prefixes." The Skeleton primitive that ships in chunk 6 already follows this. Future contributors who add a new<Skeleton>get the prefers-reduced-motion behavior for free. - Toasts for share enable / disable / copy. Previously these actions were silent — the popover content state was the only feedback, which sighted users could see but screen-reader users couldn't unambiguously confirm. Toasts give every user a definitive "the action succeeded" signal that ALSO doubles as a "you can close the popover now" affordance.
- Did NOT add toasts on every form-submit success. Out of scope. The
useFlashToasthook covers anything the backend marks with a known status; boltingtoast()calls onto everyrouter.postcall would have duplicated the flash mechanism. Stick to one delivery path.
Decisions (chunk 7):
- Split 503 into a dedicated
Maintenance.tsx. Previouslybootstrap/app.phprouted both 500 and 503 toErrors/ServerError. 500 means "we logged an unexpected exception" — the user message is "try again in a moment" because retrying might succeed. 503 means "service intentionally unavailable" — deploy in progress, scheduled maintenance, or overload-shedding — and retrying immediately won't help. Different copy ("We'll be right back" vs "Something went wrong"), different icon (Wrench vs AlertTriangle). The bootstrap match arm now lists them separately so the messaging matches the actual semantics. - Optional
iconprop onErrorShell, default null. Backwards compatible — any caller that doesn't passicongets the existing M7 layout. Each chunk-7 page passes alucide-reacticon sized ath-12 w-12for visual consistency. Picks:Compass(404 — you've wandered),ShieldAlert(403 — gated by auth),Clock(419 — session timed out),AlertTriangle(500 — unexpected error),Wrench(503 — we're working on it). Each isaria-hidden="true"because the headline + status line already announce the error verbally — the icon is decorative. window.history.back()for "Go back", not Inertia'srouter.visit('-1'). Inertia doesn't have a back-navigation primitive; the browser's history API does. For a direct deep-link hit (history.length === 1) the back call would be a no-op, so we fall through towindow.location.assign(goHomeHref)— same destination as the primary CTA. Trade-off: this leaves SPA-style soft navigation for the in-app case but uses a hard navigation for the deep-link fallback. Acceptable because the deep-link case is rare (a user lands on an error page from outside the app).- Three CTAs in a row: Sign in/Dashboard + Go back + Source. Visual order matches priority: primary action (escape to a useful page), secondary action (return to where they came from), tertiary (AGPL §13 compliance).
Buttonvariantsdefault/outline/ghostcarry the hierarchy. Source button moved fromoutline→ghostso the new Go-back button takes the "secondary" visual slot. describe.eachover the 5 pages, not 5 separate test files. Single test file with a typedcasesarray; each per-page assertion runs against every component. ~30 lines of test code covers 26 assertions. Adding a future error page would just append to the array — no boilerplate.- Kept ServerError's text "An unexpected error occurred on our end. The error has been logged". Mentions logging without leaking a stack trace or event ID. A future M15 chunk can surface the Sentry event ID for support requests — parked for now because it needs a Sentry SDK call from the server-side render path.
Decisions (chunk 8):
- Runtime probe over user-agent sniffing. User-agent strings lie (Chromium browsers spoof Safari; embedded WebViews report Chrome; Electron apps look like any UA they want). And even a browser that can support WebGL 2 can fail at runtime if the GPU driver is blocklisted (Linux/Mesa edge cases) or if the user disabled hardware acceleration. The probe answers the only question we care about: "can this user mount a WebGL 2 context right now?"
- Optimistic initial state (
true), pessimistic mount correction.useWebGL2Supportreturnstrueon first render. The mountuseEffectre-probes and corrects downward tofalseif the runtime can't actually back the context. Trade-off: a user on an unsupported runtime sees the Three.js tab briefly during hydration before it flips to Debug. The opposite default (initialfalse) would flicker every supported user through the fallback first, which is a much worse experience because supported is the common case. TheuseReducedMotionhook makes the opposite call because reduced-motion users are a minority preference; here we're optimizing for the supported-runtime majority. - Composed gate over a single boolean.
vizDisabled = reducedMotion || !webgl2SupportedandembeddingsDisabled = !webgl2Supported. Two reasons to disable the Viz tab; only one reason to disable Embeddings. Reduced motion alone doesn't disable Embeddings because the cloud's auto-orbit pauses once a spotlight is active (M8 chunk 7 decision) — the page is still consumable. WebGL unavailability disables both because both rely on Three.js + a WebGL2 context. - Amber
role="note"banner under the tablist, not a full-page block. The user can still consume the Debug tab + the run transcript on the left — we're not blocking the whole page, just two of three tabs. A small notice in the affected region is the right scope. Amber matches the existing pattern (no-api-key banner, registry-staleness banner, no-usable-models banner).role="note"is the semantic match for "additional context that doesn't carry the page's main message." titleattribute on the disabled tabs that names the cause. Sighted users hovering the Viz tab see "Visualization disabled because this browser does not support WebGL 2.0." Different from "...prefers-reduced-motion is set." Carries forward the M8 chunk 9 pattern but extends it for the new gate. Title precedence: WebGL message first (more fundamental), reduced-motion second.- No URL state, no preferences-persistence. The user can't override the gate ("yes I really want to try Viz anyway") — a runtime that can't mount WebGL 2 will throw inside Three.js's renderer constructor; honoring an override would just produce a crash. Same call M8 chunk 9 made for reduced motion.
- Global test mock + opt-in local override.
setup.tsmocksuseWebGL2Supportto returntrueby default so every existing page test that assumes the Three.js stub mounts keeps working. The newwebgl-fallback.test.tsxvi.mock(...)s it back tofalseto exercise the unsupported branch. The alternative (per-test mocks scattered across 60+ test files) would have been a much wider blast radius for the same coverage. HTMLCanvasElement.prototype.getContextmock pattern works in jsdom. Dev lesson: jsdom-29 logs "Not implemented: HTMLCanvasElement's getContext() method: without installing the canvas npm package" to stderr but the probe'stry { ... } catchswallows the throw and returnsfalse. Test suite green; the warning is jsdom's stderr noise, not a test failure. Documented now so a future contributor doesn't go install thecanvasnpm package thinking the warning needs fixing.
Use this to verify the SPEC's M12 cross-browser-testing line. Run against the latest two stable versions of each browser family — Chrome, Firefox, Microsoft Edge, Safari. macOS, Linux, and Windows have slightly different OS-level a11y behaviors (notably prefers-reduced-motion toggle locations + how Tab focuses link/button elements); cover at least one Apple platform (for Safari + reduced-motion macOS specifics) + one Chromium platform (Chrome or Edge, on Linux or Windows).
Per-browser feature checklist
For each browser × version, run through the steps below. Mark a row complete only when every step passes; a failing step is a blocker logged in the Known issues table.
| # | Step | Expected behavior |
|---|---|---|
| 1 | Sign in via the configured OAuth provider | Login → Dashboard transition is smooth; sidebar nav renders |
| 2 | Create a new thread | Header CTA fires; lands on /threads/{id} |
| 3 | Submit a prompt against a small model | Run row appears as streaming; right pane shows the Visualization tab by default |
| 4 | Watch the viz stream | Three.js cascade animates layer-by-layer; particles spawn per token; ≥30 FPS sustained (FPS overlay in dev mode) |
| 5 | Toggle to Embeddings tab | Vocab-cluster scatter loads (no blank canvas, no console errors) |
| 6 | Toggle to Debug tab | Event stream renders as JSON; new events append in real time |
| 7 | Press Space on the play button | Playback pauses; cursor → counter |
| 8 | Press Enter on a step button | Cursor advances by one token |
| 9 | Use Tab/Enter/Space-only to navigate the whole page | Every interactive element gets a visible focus ring (chunk-2 surgical fixes hold) |
| 10 | Press Esc inside the Share popover | Popover closes; focus returns to the Share trigger |
| 11 | Copy the share URL | Toast appears ("Share URL copied to clipboard.") in the top-right; clipboard contains the URL |
| 12 | Visit /share/{token} in a private window |
Public read-only view renders; "Anonymous view" indicator shows; About + Source links visible in footer |
| 13 | Click a terminal-run Replay link | Replay page loads; PlaybackControls operate keyboard-only |
| 14 | Verify the AttentionHeatmap palette | Viridis 5-stop gradient visible (dark purple → blue → green → yellow); no banding on adjacent cells |
| 15 | Verify the embedding scatter palette | 8 distinct Okabe-Ito cluster colors visible against the slate-950 background; pronouns / numbers / code-keywords visually distinct |
| 16 | Trigger an export | Download dropdown opens; GIF + MP4 items show a spinner during render; toast notifies on completion |
| 17 | Visit /this-does-not-exist |
NotFound page renders with the Compass icon + Sign in / Go back / Source CTAs |
| 18 | Open OS-level reduced-motion preference and toggle ON | Reload thread page — Viz tab is disabled with title "...prefers-reduced-motion is set"; Embeddings still operational; Debug tab opens by default |
| 19 | Disable WebGL 2.0 (chrome://flags or about:config) | Reload thread page — Viz + Embeddings both disabled with title "...does not support WebGL 2.0"; amber webgl-unsupported-notice banner visible; Debug tab opens by default |
| 20 | Open DevTools → console | No uncaught errors, no React warnings, no missing-key warnings |
Per-browser known-quirks reference
- Safari (macOS / iOS): Tab key reaches buttons + links only when System Settings → Keyboard → Keyboard Navigation is on (or equivalent on iOS). Document this in the user-facing help if any future report says "I can't Tab to anything in Safari."
- Safari (Webkit): WebGL 2 is gated behind the "WebGL via Metal" preference on some older macOS versions. Step 4 may fail on a Mac running macOS < 13 with WebGL 2 disabled in Develop menu.
- Firefox:
prefers-reduced-motionis toggled viaabout:preferences#general("Browser zoom") on desktop or via OS settings. Step 18 should still flip correctly. - Edge (Chromium): behaves identically to Chrome for every test step. Listed separately because organizations may pin Edge for compliance reasons.
- Chrome: the reference platform. Treat any Chrome failure as a structural bug, not a browser quirk.
Known issues
| Browser | Version | Step | Issue | Status |
|---|---|---|---|---|
| empty until populated |
The table starts empty by design — a fresh smoke run populates it. M15 launch-prep work tracks open issues to closure; non-blocking issues are documented as "won't fix" with a rationale (e.g., "Safari pre-15 lacks :has() selector — affects one cosmetic ring layer; ignore.").
What worked
- Hard-fail axe gate from day 1 (chunk 1). Picking the strictest CI gate up-front meant chunks 2–8 couldn't accidentally regress structural a11y while landing other work. The cost (false positives blocking PRs) never materialized because the jsdom-safe rule config (
color-contrast+scrollable-region-focusabledisabled) eliminated the only two rules that fire spuriously in jsdom — everything else axe checks is deterministic. - The frontload-shadcn decision from M7 chunk 1 paid off across every M12 chunk. Chunk 1 found zero structural violations on its first run because Radix carries the ARIA story for every primitive. Chunk 2's keyboard audit found only 5 hand-rolled hot spots — Radix handled the other ~95%. Chunk 5's empty-state audit confirmed all 3 SPEC-named states already existed from M7. Chunk 6's Skeleton + Sonner integration slotted into the shadcn primitive tree without contortion. M12 would have been a much larger surface-area chunk if M7 had rolled its own button / popover / dialog.
- Surgical fixes only, no sweeping refactors. Chunks 2 (focus-visible rings), 3 (motion-reduce gating), 5 (empty-state polish), and 7 (error-page icons) all stuck to "find what's missing, fix it in place, don't rewrite." Kept the chunk surface area predictable + the risk of regression near-zero. The opposite call would have been a "let's standardize on a focus-ring utility class" refactor across the entire codebase — much bigger blast radius with the same end-user benefit.
- Per-decile aria-live throttling (chunk 3). Picking milestone announcements over per-event announcements avoided flooding the AT queue without losing the live-update signal. Screen-reader users hear "Run started… 10 tokens generated… Run complete." instead of 100 announcements back-to-back. A pure helper (
deriveVizAnnouncement) made the logic testable without spinning up Three.js / WebGL stubs. - Color-blind palettes (chunk 4) closed the WCAG 2.1 SC 1.4.1 "Use of Color" target structurally. Viridis (heatmap) + Okabe-Ito (embedding clusters) + the existing single-hue magnitude bars (Logits + MoE) + text-paired status pills mean every information channel that could rely solely on color now doesn't. The two near-duplicate cyan / amber pairs in the old M8 cluster palette are gone.
useFlashToasthook centralized the inline-alert → toast migration (chunk 6). Single source of truth for the flash-status → toast-variant mapping. The hook is opt-in by virtue of mounting inAppLayout, so any page wrapped in AppLayout gets toasts for free — no per-page wiring. Removed two inline emerald/amber blocks fromApiKeys/Indexto avoid double-announce; the rest of the inline blocks (Admin pages, etc.) keep working via the natural flash-prop flow until a future chunk cleans them up.- Splitting 503 into a dedicated
Maintenance.tsx(chunk 7). Tiny code change, big semantic-correctness win. 500 ("we logged an error, try again") and 503 ("service intentionally unavailable, retrying won't help") deserve different copy + different icons. The old sharedServerErrorwas a UX shortcut hiding two real states. - Composed gate (chunk 8) over a single supported-or-not boolean. Modeling
vizDisabled = reducedMotion || !webgl2SupportedvsembeddingsDisabled = !webgl2Supportedseparately let us preserve the M8 decision that the embedding cloud is consumable under reduced motion. A single boolean would have either disabled too much or too little. Each gate has its own reason; each tab carries its own title attribute explaining why it's disabled.
What we learned
- jsdom hides three classes of a11y signal that need a real browser. (1) Computed CSS is mostly
rgba(0,0,0,0), socolor-contrastaxe rules false-positive everywhere; we disabled the rule. (2) WebGL contexts can't be created, so the chunk-8 probe correctly returnsfalseand triggers the fallback path — useful for one test, but it means setup.ts has to globally mockuseWebGL2Supportso the existing 60+ page tests don't all flip to Debug mode. (3)prefers-reduced-motionmedia-query evaluation requires a realwindow.matchMediaimplementation; we already had amatchMediastub from M8 chunk 9. Cross-browser real-browser smoke (this chunk's recipe) covers what jsdom misses. - Vite manifest stale on new
Pages/*.tsx. Hit this twice in M12 (chunk 1 addinga11y/test renders that touch Inertia view-finder; chunk 7 addingPages/Errors/Maintenance.tsx). Same lesson as M11. The canonical fix: rebuild between adding a page file and running Pest tests that render it. Worth adding to the M15 release-checklist as a final-pass gate. Object.defineProperty(navigator, 'clipboard', ...)overObject.assign(navigator, ...). jsdom 29 protectsnavigator.clipboardwith a getter when an earlier test file in the same run has already defined it.Object.assignthen throws;Object.definePropertywithconfigurable: true, writable: truebypasses the getter. The ShareMenu chunk-3 (M11) test got lucky because it ran first in its old order; the chunk-2 (M12) keyboard test exposed the issue. Pattern documented inkeyboard.test.tsx.- ESLint's
react-hooks/set-state-in-effectrule fires on legitimate sync-with-external-system effects. TheuseWebGL2Supporthook needs to re-probe after mount to correct the optimistictrueinitial state down to the realfalsein unsupported runtimes — that's a synchronization-with-external-system effect, the exact case the React docs say is legitimate. We disable the rule per-line with an explicit// eslint-disable-next-line react-hooks/set-state-in-effectand a comment explaining why. The reduced-motion hook (M8) does the same pattern; both are documented inline. - Pint reformat still breaks chained Edits. Same lesson M10/M11 documented. Hit again in chunk 3 (rebuilding
Threads/Show.tsxwhitespace), chunk 6 (useFlashToastimport), chunk 7 (bootstrap/app.php). The pattern is unchanged: re-Read after the hook runs, then re-Edit. react/no-unescaped-entitiescatches typographic punctuation in JSX text. Chunk 6's toast hook + chunk 8's WebGL banner both tripped the rule.'/“/”are the mechanical fixes. Worth keeping in mind any time user-facing prose lands directly in JSX.
Parked / carry-forward to M15
- Real-browser cross-browser smoke pass. The recipe above is the manual verification path; the actual run produces the Known Issues table content. M15 Launch Prep owns the first full run + populating the table. Subsequent runs become release-gate work.
- Playwright / Cypress E2E. Same call M6 / M8 / M9 / M10 / M11 closeouts all made. Unit + integration + manual recipes is enough for Phase 1 ship; browser-automated E2E is a Phase 2 / launch-readiness expansion. M12 chunk 1's
jest-axematcher + chunk 8'svi.mockpattern would adapt directly to@axe-core/playwrightwhen E2E lands. - Color-contrast verification (real CSS, real browsers). The chunk-1 jsdom-disabled rules (
color-contrast,scrollable-region-focusable) need real-browser verification before we can claim WCAG 2.1 AA conformance with confidence. The M15 cross-browser pass (step 14, 15) covers the viridis + Okabe-Ito visual verification; a parallel run of@axe-core/playwrightin a future chunk could automate it. - Screen reader pass (NVDA / VoiceOver). The SPEC's M12 task line mentions "manual screen-reader pass" alongside the axe-core CI part. Chunk 1 landed the CI half; the manual SR pass is parked for M15 (or earlier if a contributor with an SR is available — the work scales linearly with viz / pages and stays under 1 hour for the vertical slice).
- Authed pages link to
/about. Per M11 chunk 4 + the M12 chunk-6 toast pattern decision, only the public surfaces (SharedLayout + Welcome) link to/abouttoday. Dashboard, Threads, Settings, Admin don't surface an About / "What is this?" link. M15's AGPL §13 audit owns this — add a footer link toAppLayout. - Sentry event ID on 500 page. Chunk 7 decision recorded this. M15 surface for the ServerError page so support requests can quote a reference ID without exposing a stack trace.
- No-usable-models empty state polish. Chunk-5 decision parked it. The state fires when a user has an API key but the registry doesn't carry a model for their vendor — a registry-staleness edge case. M15 could surface a "ping an admin" affordance or auto-trigger the registry-refresh fetch.
- Three hand-rolled tablists could migrate to
@radix-ui/react-tabs. Chunk-2 decision recorded this. Would inherit arrow-key navigation between tabs (WAI-ARIA Authoring Practices best practice, not WCAG conformance failure). ~150 lines of churn across 4 pages; saved for a polish chunk if anyone files it. - The 3 moderate-severity
wsCVEs in socket.io-client transitives are still open. Documented in chunk 1's commit body.npm audit fixwould update them; deferring to avoid a major-version bump on the WebSocket layer outside a release window.
Exit criteria
- axe-core reports zero violations on dashboard, prompt-input, thread-detail, share-view pages. Mapped to the four pages in
resources/js/__tests__/a11y/:Dashboard.a11y.test.tsx(Dashboard + empty + loaded),Threads.Show.a11y.test.tsx(prompt-input lives inside Threads/Show viaPromptFooter; covered in empty + loaded + archived states),Share.Show.a11y.test.tsx(Share/Show in empty + loaded + redacted states),Share.Replay.a11y.test.tsx(Share/Replay in complete + errored + redacted states). 11 tests, hard-fail gate. Caveat: jsdom-disabled rules (color-contrast,scrollable-region-focusable) carry over to M15 real-browser pass. - Manual keyboard navigation can run a full vertical slice (sign in → new thread → submit → watch viz → replay) without a mouse. Manual recipe in §"Keyboard-only manual recipe (chunk 2)" above. Automated coverage: 7 Vitest keyboard tests in
resources/js/__tests__/a11y/keyboard.test.tsx. Chunk-2 surgical focus-visible fixes (19 buttons across 5 hot spots) closed the last gaps.
Bonus M12 work beyond the SPEC's two exit criteria:
- Three.js canvases announce state transitions via
role="status" aria-live="polite"(chunk 3, per-decile milestones) -
motion-safe:prefix on everyanimate-*class (chunk 3, prefers-reduced-motion compliance for inline animations) - CB-safe palettes everywhere color carries meaning (chunk 4, viridis + Okabe-Ito; magnitude bars + text-paired status pills are intrinsically safe)
- 8 empty-state surfaces locked in tests (chunk 5, icon + title + body + CTA-href invariants)
- Skeleton + Sonner toaster wired (chunk 6, both via shadcn primitives)
- 5 error pages with icons + Go-back CTA + 503 split (chunk 7)
- WebGL 2.0 fallback to the 2D Debug tab (chunk 8, composed with reduced-motion gate)
- Cross-browser smoke recipe published for M15 verification (chunk 9)
Purpose: Replace the M8 live-cascade visualization with a 20-scene cinematic narrative through the entire inference pipeline — prompt entry, byte-encoding, BPE tokenization, embedding lookup, positional encoding, every step inside a transformer block (layer norm, Q/K/V projection, multi-head attention, residual, FFN, second residual), the layer-stack loop, final logits, sampling, detokenization, and the autoregressive loop with KV cache. Design source: docs/visualization.md. The pipeline takes ~60-90s at explanatory speed; the real run streams in parallel and finishes faster, with its tokens accumulating in a chat-bubble overlay while the viz keeps playing.
Status of M8 viz components: M8 is locked at ✅ Complete. The historical M8 docs + decisions remain in this file as-is (the cascade + particles + EmbeddingScene shipped a usable feature for several milestones). M13 is a rip-and-replace per user direction at the chunk-pre-discussion: chunk 1 deletes the M8 viz components + their tests outright so chunks 2-14 don't have to code around dead weight. The git history preserves the M8 implementation for anyone who wants to spelunk; phase1.md keeps the documentation. No feature flag, no parallel mounts — clean slate from chunk 1.
Visual language (per docs/visualization.md):
- Tokens = rounded rectangles, color from a hash of the token ID
- Vectors = horizontal 1D heatmap strips (viridis ramp, mostly-implied long axis via "…")
- Matrices = 2D heatmap grids (also viridis)
- Weights / parameters = dimmed gray, "always there in the background"
- Active computation = bright + animated, traveling particles or pulses
- Camera = mostly fixed per scene, with a few zoom-out reveals (notably Scene 5 and Scene 12)
Persistent UI in the visualization region (per user direction on chunk-discussion: "make the visualization div larger to hold everything; these are sections in it"):
- Vocabulary sidebar (left edge) — scrolling list + highlight on token lookup, populated during BPE in Scene 3
- Chat bubble (bottom-right) — streaming output area; mirrors the real run's tokens as they arrive on the WebSocket
- Layer counter HUD (top-right) — visible during the tower scene (12) and across all stage-3 scenes
- Pipeline progress bar (top, 20 segments) — shows which scene is currently active + click-to-jump
Tasks
-
Chunk 1 — Rip M8 + scene architecture + page layout. Deletion-first chunk:
- Delete M8 viz components:
resources/js/Components/Viz/VizPane.tsx,EmbeddingScene.tsx,CascadeController.ts,ParticleSystem.ts,TransformerStack.ts,subComponents.ts,FpsCounter.tsx(keep — the new viz reuses it),AttentionHeatmap.tsx(keep — chunks 5/8 reuse it). Delete their Vitest files. Delete theVizSkeleton.tsxSuspense fallback from M12 chunk 6 and its test — the new viz won't need a Three.js lazy-import skeleton in the same shape. - Drop the right-pane Visualization / Embeddings tabs from
Pages/Threads/Show.tsx,Pages/Runs/Replay.tsx,Pages/Share/Replay.tsx. The cinematic viz is a single mount, not a tabbed surface. Keep the Debug tab as the WebGL-2-unsupported fallback target (chunk 13 evolves it into the SVG fallback path). - Drop the M12 chunk-8 WebGL gate banner copy that says "the 3D visualization + embedding scenes are unavailable" — replace in chunk 13 with the new tri-state messaging.
Scene<I, O>interface (input,onComplete(output),durationMs, normalized progresstin [0,1]).SceneRunnerstate machine that walks scenes 0→20, threads each scene's output into the next scene's input, ownstadvancement, exposespause / resume / scrub / setScene / setSpeed.- Page-layout change:
Pages/Threads/Show.tsx+Pages/Runs/Replay.tsx+Pages/Share/Replay.tsxwiden the visualization region (target: viz takes 2/3 of the row, transcript+prompt take 1/3 — flip of the current 1/3 viz : 2/3 transcript split). Placeholder mounts for the four persistent UI sections (vocab sidebar, chat bubble, layer counter, pipeline progress bar) so chunks 2+ can drop content into them without touching layout. - Stub
<CinematicViz events={...} model={...}/>mounted in place of the old VizPane. Renders an empty canvas + the four persistent UI placeholders + a "Scene 0 / 20" placeholder text so the visible layout is correct, even though no scenes are implemented yet. - Updates
__tests__/a11y/webgl-fallback.test.tsx+__tests__/Pages/Threads/Show.test.tsx+Runs/Replay.test.tsx+Share/Replay.test.tsxfor the new mount shape (no moreview-viz/view-embeddings/view-debugtabs to assert on). - Full suite must stay green at the chunk-1 push (every M8 viz test deleted, every page test updated for the new mount). The cinematic-viz tests are placeholders that just assert "Scene 0 / 20" renders.
- Delete M8 viz components:
-
Chunk 2 — Visual-language primitive components. Pure presenters, no scene logic:
<TokenPill id={n} label={'token'} idHash={u32}/>— rounded rect, hue fromidHash % 360, displays string + optional ID number<VectorStrip values={number[]} visibleCells={128} totalLength={4096}/>— 1D heatmap, viridis ramp, trailing…indicator<MatrixGrid values={number[][]} dim={[rows, cols]} maxVisible={32}/>— 2D heatmap with same fade-out for off-screen extent<WeightFog/>— dimmed gray background mesh that sits behind active components<ParticleTrail from={Vector3} to={Vector3} colorFromHash={n}/>— single-line pulse animation- All TypeScript-typed, all Vitest-tested in isolation (no Three.js required for the SVG variants; Three.js for the ParticleTrail).
-
Chunk 3 — Scenes 0-4 (text → tokens). Scene 0: prompt-entry textbox in the center, types in (or pastes), glows once, characters detach + float. Scene 1: char bounce + 180° flip to byte values, staggered left-to-right (~30ms/char); emoji + non-Latin chars visibly split into 2-4 bytes ("bytes ≠ characters" teaching beat). Scene 2: chat-template wrap — system-prompt block slides in from above (purple tint),
<|user|>/<|assistant|>markers (teal tint) bracket the row; special-token strings show as text then collapse to byte integers. Scene 3: BPE animation — brackets slide in around 1-4 adjacent bytes, snap-lock, fuse into a token pill labeled with the token string; concurrent line shoots to the vocabulary sidebar, scrolls + highlights the matching row. Common tokens fast (~200ms), rare words slow (~600ms). End with strings fading, IDs remaining. Scene 4: 0.5s breather — clean integer row + label "context length: N tokens." Tokenizer-pick constraint (per user direction): must work in every browser the M12 chunk-9 smoke recipe covers — Chrome, Firefox, Edge, Safari (latest two of each). Candidates:@dqbd/tiktoken(Wasm-backed; needs WebAssembly, which Safari has had since 11 + every modern Chromium/Firefox supports; faster steady-state, ~600KB bundle);js-tiktoken(pure-JS port, ~200KB, slower but zero runtime requirements beyond ES2020). Measure cold-start + per-call latency in chunk 3 against the SPEC's "explanatory speed" budget and pick whichever doesn't make Scene 3 feel sluggish on Safari, the slowest target. Document the choice + the measured numbers in the chunk 3 decisions block. -
Chunk 4 — Scenes 5-7 (embedding → norm). Scene 5: the first "wow" moment. Camera zooms out, token row shrinks to the bottom, a massive wireframe matrix labeled "Embedding table: 128,000 × 4096" materializes (rendered as ~16×16 visible cells with
…extents, but the label communicates the real scale). Per-token vertical beams shoot up to specific matrix rows; the rows detach + slide down to replace the pills as horizontal vector strips. Scene 6: positional encoding / RoPE. Position counter (0, 1, 2…) appears above each strip; a subtle shear/wave or 3D rotation around the long axis with rotation amount proportional to position. Brief "θ_pos" annotation on the first few. Scene 7: layer norm — strips temporarily render as bar charts (heights = values), squish animation equalizes the distribution, snaps back to heatmap form. -
Chunk 5 — Scene 8 (multi-head self-attention, the centerpiece). Three sub-beats:
- 8a (1.5s): From each input strip, three new strips bud off color-coded — Q (blue), K (red), V (green) — arranging into three parallel rows
- 8b (3s): Attention-matrix grid materializes (N×N, sequence length). For each (i, j) with j ≤ i (causal mask), brief arc/laser from Q_i to K_j; matrix cell lights with intensity proportional to the dot product. Cells above diagonal stay black. Row-by-row softmax animation rebalances each row to sum to 1. Multi-head reveal: the matrix is actually a stack of ~32 transparent layers (per model
attention_heads); fan them out briefly to show parallelism, collapse back. Heads concatenate into one output via a quick "squeeze" animation - 8c (2s): For each output position, V vectors pull in with opacity matching attention weight, blend, form a new output strip. Detailed for positions 0-2, fast-forward the rest
- Animation values seeded deterministically from
(run.id, layer_index, token_index)so a replay is frame-identical.
-
Chunk 6 — Scenes 9-11 (residual + FFN + second residual). Scene 9 (1s): pre-attention vectors (kept ghosted in the background since Scene 6) slide forward + merge with attention output via a glowing
+symbol. Scene 10 (3s): FFN — strips flow through a "pipe" that expands to ~4x width (hidden_dim → 4 × hidden_dim), passes through a non-linearity (wavy color shift representing SwiGLU/GELU permodel.architecture_type), contracts back. Tiny "neuron firing" sparkles inside the expansion. Scene 11 (1s): pre-FFN ghosts merge with FFN output — mirror of Scene 9. -
Chunk 7 — Scene 12 (layer-stack loop / tower view). The big camera move. Camera pulls way back revealing that scenes 5-11 happened on a single floor of a tall tower (one floor per layer, count from
model.layers ?? 32). A glowing packet (the current vector sequence) sits on floor 1. As it moves up to floor 2, camera follows partway then accelerates — floors 3 through (N-2) blur past in ~3s with a layer counter HUD ticking up rapidly ("Layer 03… 17… 42… 78…"). Camera slows for floor N-1 and N, where we re-zoom for full detail on the final layer's attention + FFN. Auxiliary mini-map stays docked in the corner showing "Layer 42 / 80" with a progress bar throughout — populated even when not on Scene 12 so the user can orient at any time. -
Chunk 8 — Scenes 13-17 (final norm → logits → sampling → emit). Scene 13 (0.5s): brief final layer norm squish. Scene 14 (3s): only the LAST vector strip matters for next-token prediction — others dim + fade. Final strip floats center, projects through "LM Head" matrix (4096 × 128000, mirror of Scene 5). Beams shoot through, emerge as a 128k-long mostly-cool-colored heatmap with a few hot spikes. Scene 15 (2s): logits row pivots 90° → horizontal bar chart sorted descending. Softmax wave rescales bars to probabilities summing to 1. Most collapse near-zero; handful at left dominate. Optional temperature-slider corner showing high vs low temp distributions. Scene 16 (1.5s): sampling. Mode-aware: greedy dart slams bar #1; top-k fades bars beyond k, dart wobbles among survivors; top-p sweeps a fill-line until cumulative-p reached. Chosen bar pulses; winning token string flashes above. Scene 17 (0.5s): winning token flies down, appends to "generated so far" tray + the chat-bubble UI in the corner.
-
Chunk 9 — Scenes 18-20 (autoregressive loop + KV cache + detokenization). Scene 18 (variable): meta-loop. Full sequence (input + generated so far) becomes new input. Compress Scenes 5-17 into ~2s for token #2, ~1.5s for #3, accelerating to ~200ms/token (5/sec) by token #10+. The real run drives token timing via the WebSocket; if the run finishes before the viz catches up, viz keeps playing at its accelerated pace while the chat bubble already has the full text. Scene 19 (2s, one-time reveal during first loop iteration): KV cache. Pause briefly; show K + V matrices from previous tokens appearing in a dimmed "cache drawer" with a small lock/disk icon. For the new token, only its row of K + V is fresh-computed + added. Attention matrix gains a single new bottom row per step instead of recomputing the whole grid. Subsequent loop iterations skip the explanatory beat; cache drawer just fills. Scene 20 (continuous during Scene 18): detokenization — each chosen token ID briefly transforms back to its string fragment via reverse-lookup highlight in the vocab sidebar, then flies into the chat bubble + concatenates. End-of-sequence token: chat bubble glows, full pipeline canvas dims, completion flourish.
-
Chunk 10 — Persistent UI completion. Wire the four sections to live data:
- Vocabulary sidebar (left): scrolling list of
(id, string)populated during Scene 3's tokenization. Persists through all 20 scenes. Highlight + scroll-into-view on lookup events from Scene 3 (forward) + Scene 20 (reverse). - Chat bubble (bottom-right): mock chat-message UI. Starts empty; grows as Scene 17 appends each token's string. Driven directly by the real
token.receivedWebSocket event so the user sees the response coming in even if the visualization hasn't reached Scene 17 yet for that token. - Layer counter HUD (top-right): integer counter + tower progress mini-bar. Visible during scenes 5-12. Auto-hides during scenes 0-4 + 13-20.
- Pipeline progress bar (top, 20 segments): full-width strip, each segment named ("Prompt", "Bytes", "Chat tpl", "BPE", "IDs", "Embed", "RoPE", "Norm", "Attention", "Residual", "FFN", "Residual", "Layers", "Norm", "LM head", "Softmax", "Sample", "Emit", "Loop", "Cache", "Detok"). Current scene highlighted. Click any segment → jump to that scene's start (uses SceneRunner's
setScene(n)).
- Vocabulary sidebar (left): scrolling list of
-
Chunk 11 — Playback controls + per-scene scrub. Adapt the M8 chunk-8
PlaybackControlsto scene-level. Speed buttons: 0.25× / 1× / 4× (perdocs/visualization.mdproduction notes). Step: advance to next scene-boundary, not next event. Jump-to-live: catch up to the scene that matches the current token-stream position. Click any vector strip to expand it into a numerical-values panel (per the production note "let the user click any vector strip to expand it and see actual numerical values"). The expanded panel docks to the side, doesn't block the canvas. -
Chunk 12 — Performance hardening (20-30 FPS target). Per user direction: aim for 20-30 FPS sustained. Simpler shapes are OK as long as the idea carries. Monitor FPS via the existing
FpsCounteroverlay (M8 chunk 4); add a degraded-mode automatic fallback when FPS drops below 18 for 2+ seconds:- Multi-head attention stack → single representative head shown
- Particle trails → straight-line connectors without animation
- WeightFog → static texture instead of animated noise
- VectorStrip cell count clamped to 64 (down from 128) on the degraded path
Document the degrade triggers + restoration condition (
< 18 FPS for 2s → degrade; > 24 FPS for 5s → restore) inline + in the M13 retrospective.
-
Chunk 13 — Reduced-motion + WebGL gate adaptation. The M12 chunk-8 gate disabled the M8 Viz + Embeddings tabs entirely when WebGL 2 is missing or reduced-motion is set. The new cinematic viz can degrade more gracefully:
- Reduced-motion: scenes still display, but as static frames. The user advances scene-by-scene via the chunk-11 Step button (or auto-advances on a slow 4-second-per-scene timer). No camera moves, no particle trails, no continuous animations. Equivalent of "PowerPoint mode."
- WebGL 2 unavailable: fall back to the SVG variants of every primitive. The 2D Debug-tab fallback from M12 chunk 8 remains as the absolute-last-resort path. New
webgl-unsupported-noticecopy explains: "3D camera moves are unavailable; the visualization is rendering in 2D mode." Replaces the M12 chunk-8 binary disable/fallback with a tri-state (full / 2D-svg / debug-text).
-
Chunk 14 — Tests + closeout. Vitest assertions per scene (deterministic synth check via fixed seed; snapshot of the scene's output state given a known input state). Integration test: full-pipeline end-to-end with a mock event stream that fires through scenes 0-20 + asserts the chat bubble accumulates the right output. Manual recipe (matches M8 chunk-9 + M9 + M11 patterns): visual review per-scene, FPS spot-check, degraded-mode visual confirmation, reduced-motion visual confirmation, share-link verification (the cinematic viz needs to work for the public
/share/{token}/runs/{run}/replayroute too). Cross-browser smoke on the new viz (per M15 recipe step list — extend the table with cinematic-specific rows: every scene renders, scrubber jumps to each scene, FPS stays in band on each browser). M13 retrospective covering all 14 chunks. Status table line 25 (M8) stays ✅ Complete but gains a "(superseded by M13)" note. Flip M13 status to ✅ Done. (Deletion of M8 components happened in chunk 1; this chunk has nothing left to clean up.)
Exit criteria
- The 20-scene narrative runs end-to-end on a fresh thread submit with a real OpenAI run, hitting all milestones from prompt-entry through end-of-sequence flourish in 60-90 seconds at 1× speed.
- The real run's tokens stream into the chat-bubble overlay independent of (and typically faster than) the visualization — the user sees the response build in the corner while the viz keeps playing.
- 20-30 FPS sustained on a 2020-class machine; automatic degrade triggers cleanly when FPS dips, restores when stable again.
- Reduced-motion mode shows every scene as a static frame the user can step through.
- WebGL 2 unavailable mode shows the SVG fallback; both modes still produce all 20 scenes (just rendered differently).
- The persistent UI sections (vocab sidebar, chat bubble, layer counter, pipeline progress) are populated and click-targets work (sidebar highlights on lookup, progress-bar segments jump to scene).
- The
/share/{token}/runs/{run}/replaypublic route shows the same visualization as the owner view (no owner-only features in the viz layer). - M8 components retired:
VizPane.tsx,EmbeddingScene.tsx, related controllers + tests removed from the codebase.
Architectural notes
- Each scene is self-contained. Per the production note in
docs/visualization.md: "Build each scene as a self-contained component that accepts its input state as props and emits its output state on completion. That lets you replay individual scenes for debugging and lets users jump to any point in the pipeline." This is the major architectural shift from M8's concurrent-renderers-driven-by-shared-event-stream model. The new SceneRunner walks scenes sequentially; each scene owns the canvas during its window. - Observable vs illustrative. The vendor API only exposes the input prompt, the streaming token IDs/strings, per-token logprobs (sometimes), and timing. Everything else is illustrative: BPE animation (we tokenize client-side via
tiktoken), embedding lookup, RoPE rotation, layer norm, Q/K/V projections, attention scores, FFN expansion, LM-head projection — all rendered deterministically from(run.id, model_metadata, token_index, layer_index)seeds. The visualization is a pedagogical model of inference, not a measurement of it. Per user: "this whole site is an education tool." - Real run continues independently. The M6 streaming pipeline + the M8
useRunStreamhook keep doing exactly what they do. The chat bubble subscribes to the same WebSocket events and displays tokens as they arrive. Scenes 18-20 use the same event timeline to drive the autoregressive-loop pacing, but the viz lagging behind is expected + acceptable. - WebSocket → Scene coupling. Scenes 5-17 fire once per generated token. The current
RunEventschema (M6) emits onetoken.receivedper token; the SceneRunner subscribes + advances. For the input tokens (the prompt), the runner enters Scenes 0-4 at submit time without waiting for the WebSocket.
Decisions (chunk 13): (Reduced-motion + WebGL gate tri-state.)
- Reduced-motion = PowerPoint mode. Each scene renders at
t = 1(completion frame). Autoplay disabled, scene runner doesn't tick through t. 4-secondsetTimeoutauto-advances to the next scene; Step button in PlaybackControls always works as a manual override. Once the runner reaches the last scene, no auto-advance (the timer guardssceneIndex >= totalScenes - 1). t = 1is the chosen pin. Each scene's last frame is its most informative static view — Scene 5 shows the embedding strips (post-detach), Scene 16 shows the chosen bar pulse, Scene 20 shows the EOS flourish.t = 0.5was considered but most scenes have peak content mid-window only briefly; the completion frame stays maximally informative.- All reduced-motion logic lives in
CinematicViz, not inuseSceneRunner. The hook is untouched: we passautoplay: false, override the renderedt, and run a siblingsetTimeoutfor the 4s advance. Keeps the runner pure (it still drives t for non-reduced-motion paths), and the reduced-motion behaviour is one cohesive 15-line block in CinematicViz with clear comments. - WebGL 2 gate = informational only. The chunk-1 decisions block locked "All 5 primitives are SVG/HTML/CSS — no Three.js" — every M13 scene already renders via SVG. There's no 3D scene to fall back from. The notice exists because the spec calls for it, but functionally the viz behaves identically with WebGL2 missing. The copy explains this honestly: "3D camera moves are unavailable; the visualization is rendering in 2D mode."
- Tri-state precedence: WebGL gate wins over reduced-motion when both apply. The WebGL notice is the more critical info (different rendering pipeline), reduced-motion is a comfort/accessibility mode. Only one notice surface; the more informative one shows. The reduced-motion behaviour (t = 1 pin + auto-advance) still applies underneath even when the WebGL notice covers the visible message.
data-gate-modeattribute on the notice (webgl/reduced-motion). Lets tests assert which gate fired without parsing the human-readable copy. Pattern matches the chunk-12 FpsCounter'sdata-degraded.data-reduced-motion="true|false"on the canvas root. Sister assertion to the gate notice — confirms the t-pin behaviour is active regardless of which notice (if any) is visible. Used by the chunk-13 tests to verify the "both gates" case.- Gate-notice tests use dynamic import +
vi.doMock. The global setup defaultsuseWebGL2Supportto true anduseReducedMotionto false (via the matchMedia stub). Chunk-13 suites override per-suite viavi.doMock + vi.resetModules() + dynamic await import('@/Components/Viz/CinematicViz'). Thevi.resetModules()call in beforeEach is required — without it the first test in each suite inherits the already-loaded module from before the mock applied. Documented inline. - The chunk-1 a11y test (
webgl-fallback.test.tsx) was updated to match the chunk-13 copy. Its comment block explicitly anticipated this change ("Chunk 13 evolves the gate into the tri-state… for now we just assert that the notice renders"). Updated to assert the new spec literals +data-gate-mode="webgl"rather than the chunk-1 "WebGL 2.0" placeholder copy. - No behavioural change for ParticleTrail/etc. ParticleTrail already gates its animation via
motion-safe:(chunk 2) which respectsprefers-reduced-motionnatively. The chunk-12 degraded-mode hook handles FPS-driven trims. Chunk 13's contribution is the scene-runner-level t-pin + auto-advance, not per-component animation overrides — those existed already and continue to fire. - Auto-advance is per scene, not per token. Spec says "auto-advances on a slow 4-second-per-scene timer" — chunk 13 uses a fresh
setTimeout(controls.nextScene, 4000)per scene-change, cleared on unmount or sceneIndex change. Any manualsetScene()call (PipelineProgressBar click, PlaybackControls Step) cancels the pending advance and arms a new one for the new scene.
Decisions (chunk 12): (Performance hardening + degraded-mode state machine.)
- State machine in
useFpsTracker, transport viaPerformanceModeContext. Hook drives the RAF loop + hysteresis; context flows{ fps, degraded }to ~13 scene/component callsites. Same pattern as chunk 11b'sVectorInspection. Mounting the hook inCinematicVizkeeps a single global tracker; per-component subscribers stay declarative. - Hysteresis thresholds match the spec literal:
< 18 FPS for 2s → degrade; > 24 FPS for 5s → restore. Asymmetric on purpose — degrade fast, restore slow. The 6-point gap between thresholds (18 vs 24) prevents oscillation when the actual FPS sits around the boundary. Exported asFPS_TRACKER_CONFIGfor tests + decision-block citation. - Decisions made every UPDATE_INTERVAL_MS (250ms), not every frame. The displayed FPS already updates at this cadence; the state machine piggy-backs. A per-frame decision would consume CPU for no behavioral gain — hysteresis windows are measured in seconds.
- WeightFog is already at the degraded target. Spec calls for "WeightFog → static texture instead of animated noise"; the chunk-2 implementation never animated (single SVG
<pattern>with no<animate>). Degraded mode is a no-op for WeightFog. Documented inline so a future reader doesn't wonder why no opt-in code lives there. - ParticleTrail's degraded mode strips the CSS animation outright, not via
motion-safe:gating. The chunk-2 component already freezes underprefers-reduced-motionvia themotion-safe:variant. Degraded mode is independent (FPS-driven, not preference-driven) and stronger — it removes the animation class + style entirely, so even a non-reduced-motion user gets a static dot. Thedata-degraded="true"attribute makes the behavior testable. - VectorStrip degraded path clamps via
Math.min(visibleCells, 64), not a fixed-64 override. Callers that already passvisibleCells < 64(e.g., the chunk-5 AttentionScene Q/K/V rows at 32) stay at their requested count. Degraded never INCREASES the cell count. - AttentionScene shows
headMatrices.slice(0, 1)in degraded mode. Drops from 6 representative heads to 1. Caption gains a " · degraded" suffix so the viewer can see the mode change. The full attention math is still computed for the head 0; only the multi-head fan is collapsed. - FpsCounter visible when
import.meta.env.DEV || ?debug=fpsquery param. Per user direction — dev-only by default, but production users can append?debug=fpsto see the overlay without a rebuild. Useful for spotting field perf issues without baking the overlay into shipping pages. The state machine itself runs in all builds; only the overlay is gated. - No useReducedMotion bypass. Considered: skip the state machine + force degraded when
prefers-reduced-motionis set. Rejected because reduced-motion is its own thing (chunk 13) with a different degradation profile (per-scene step-through, not just animation suppression). Mixing the two would conflate accessibility with perf. - Tests use a faux RAF + mocked
performance.now. jsdom doesn't drive frames; the chunk-12 tests advance time + drain the RAF queue manually. Pattern:vi.spyOn(window, 'requestAnimationFrame')captures callbacks,advance(dt)flushes them at the chosen interval. Same approach scales to chunk 11'suseSceneRunnerif it ever needs deep RAF tests. - Two new contexts now wrap the canvas.
<PerformanceModeProvider>sits outermost,<VectorInspectionProvider>nested inside. Outermost = read by the most consumers (every VectorStrip, ParticleTrail, AttentionScene); inner = scoped to the canvas region where strips can be clicked. Mirror of the React context "by frequency-of-use" pattern.
Decisions (chunk 11b): (Click-to-expand numerical-values panel — completing chunk 11.)
- Opt-in via React context, not per-scene wiring.
<VectorInspectionProvider>wraps the canvas;VectorStripcallsuseVectorInspection()and auto-becomes clickable when inside the provider. Outside the provider the strip stays non-interactive (the chunk-2 behaviour). Means ~100 strip mounts across 20 scenes get the feature for free, no scene code changes. - Context's
open/closeareuseCallback-stable. The first draft had an infinite render loop: tests'useEffect([inspection], () => inspection.open(...))re-fired each timesetActiverecreated the context value. Fix is stable callbacks via useCallback (empty deps); tests now depend onopennotinspection. Documented in the context source as a footgun warning. - VectorStrip becomes
role="button"+tabIndex={0}only when inspectable. Outside the provider it staysrole="img"(the chunk-2 default — non-focusable). Inside the provider, Enter + Space keyboard activation matches the M12 chunk-2 a11y pattern. focus-visible ring on cyan. Hover gives a subtle shadow. - NumericalValuesPanel docks as a right-overlay taking ~40% of the canvas, with a 60%-width backdrop on the left. Click backdrop OR the × button OR press Escape to dismiss. Empty-state (no active inspection) returns null — no DOM weight.
- Cell list caps at 64 entries. Renders as a 2-column grid with index + value + viridis swatch. Big vectors (e.g., 4096-cell embeddings) show "First 64 cells (of 4096)" header. Stats use the FULL vector, not just the rendered 64 — dim/mean/std/min-max all reflect the complete data.
- Stats grid is dim / mean / std / range, in a 4-column row. Range is "min…max" with em-dash separator. All values use
formatNumber: 3 decimals for |v| < 100, fewer for larger magnitudes. Test pins the mean of[1,2,3,4]at2.500. - Escape-to-dismiss via a global
keydownlistener mounted only while the panel is active. Attaches inuseEffect([inspection]), removes on cleanup. Doesn't impair other keyboard handlers when the panel is closed. inspectionLabelprop on VectorStrip falls through tocaptionthen "Unnamed vector". Gives every scene a chance to set a meaningful label (e.g., "Layer 5 hidden state · token 3") without requiring it. Most existing scenes already use captions, so the fallback chain works without per-scene changes.- Empty vectors are explicitly NOT inspectable (
data-inspectable="false"). Clicking a zero-cell strip would open a panel with nothing useful. Tests assert this. - VectorInspectionProvider mounts INSIDE the chat-bubble / layer-counter scope in
CinematicViz, so the panel only overlays the scene canvas — not the chat bubble or vocab sidebar (those stay accessible). The backdrop blur is light (1px) to keep the underlying scene readable as context. - Chunk 11 complete with both halves. PlaybackControls (chunk 11a) + click-to-expand panel (chunk 11b). All four affordances from the spec line are now wired: speed selector, step, jump-to-live, click-strip-to-expand.
Decisions (chunk 11a): (PlaybackControls — chunk 11 is mid-flight; superseded by chunk 11b.)
useSceneRunnercontrols already covered the needs. Speed (PLAYBACK_SPEEDS already[0.25, 1, 4]), play/pause/toggle, prev/next scene (= "step to scene-boundary" per chunk-11 spec). No runner changes needed — chunk 11a is pure UI + jump-to-live derivation.PlaybackControlsis a pure render component takingstate + controls + liveSceneIndex. CinematicViz computesliveSceneIndexfrom the events stream and passes it through. Same pattern as the chunk-1PipelineProgressBar(state + onSelectScene). Keeps the chunk-11 component pure-render, easy to test.- Jump-to-live heuristic matches the chunk-11 design decision: no events → null → button disabled; any token.received → setScene(18); is_final → setScene(20). Documented in the inline comment + tested explicitly. The button is also disabled when the live target equals the current scene — "no jump needed" UX state.
- Step =
nextScene()/prevScene()only. Spec explicitly calls out scene-boundary step, not event-boundary step. The runner already exposes both; mapping them to◂◂/▸▸is straightforward. No event-stepping mode — would conflict with the synthetic-vs-real timing split. - Speed selector is
role="radiogroup"with threerole="radio"buttons. Per the M12 chunk-2 a11y pattern: the active speed isaria-checked="true"; the inactive two arearia-checked="false". Single button visually pressed via cyan background. focus-visible ring chain present. - Play/pause uses semantic colors (amber when playing → "pause to interrupt", emerald when paused → "play to resume"). Picked for color-blind safety (the verb is in the label, not just the color) and to differentiate from the cyan speed buttons.
- Jump-to-live uses violet accent. Distinct from cyan (speed) / emerald (play) / amber (pause) — a fourth semantic slot for "fast-forward through the run." Disabled state uses the muted-foreground/30 pattern.
- Scene-position label "Scene N / M · t = X.XX" mounted at the right of the toolbar. Read-only, informative. Helps debug-mode users see exactly where the runner is. Could be hidden behind a flag in chunk 13; for now visible.
- PlaybackControls mounted between PipelineProgressBar and the canvas row in CinematicViz. Pipeline progress (overview) → playback controls (action) → canvas (content). Top-to-bottom visual hierarchy matches the spec's "PipelineProgressBar at the very top, PlaybackControls just below."
- The click-to-expand affordance lands in chunk 11b. Chunk-11 checkbox stays unchecked until 11b ships the VectorInspectionProvider + NumericalValuesPanel.
Decisions (chunk 10b): (ChatBubble + WebSocket events wiring — completing chunk 10.)
- ChatBubble accepts
tokens: string[]+isFinal: boolean— the simplest contract that matches both data sources. CinematicViz does the events-vs-state selection upstream; ChatBubble itself doesn't know which source produced the array. - Events-primary, state-fallback selection in CinematicViz. When the
eventsprop contains anytoken.receivedentries, the bubble reads from those. When empty (chunk-10 isolated testing, replay without a stream), it falls back tostate.pipelineState.generatedTokens.map(t => t.string). This delivers the spec's "ahead-of-viz" property automatically: live events stream from the real run independent of which scene the viz is on; chunk-10 tests + replays without a stream still show something. is_finalderived from the last token event's payload, or scene-based fallback. When events stream,is_finalis thetokenEvents[N-1].payload.is_finalflag. When no events (state fallback), the bubble enters "complete" mode when Scene 20's flourish is active (sceneId === 'detokenize' && t >= 0.9). Mirrors the in-canvas "Inference complete" badge timing.- Blinking cursor only when
!isFinal. Subtle pulse on a 1px-wide bar at the end of the text. CSSanimate-pulse(Tailwind's keyframes). Hidden once the run is final — matches the "stop talking" beat. Aria-hidden because the cursor is purely decorative. - Token count rendered alongside "Response" header. "12 tokens" / "12 tokens · complete" — quick orientation for the viewer about how much output has streamed in. The chunk-9 in-canvas tray uses the same count format.
whitespace-pre-wrap break-wordson the text. Real tokens include leading spaces (' the',' a'); preserving them gives natural word-boundary rendering.break-wordskeeps unusually long single tokens from horizontally overflowing the 288px bubble.- CinematicViz's events prop defaults to
[]. Existing pages (Threads/Show, Runs/Replay, Share/Replay) already pass an events array from the M6 streaming wiring; the default is for chunk-10 isolated testing only. The page mounts continue to work without changes. - Type narrowing via
Extract<RunEvent, { event: 'token.received' }>. TypeScript's discriminated-union narrowing inside thefilterpredicate gives us typed access toe.payload.token+e.payload.is_finalwithout a cast. Documented in the inline comment. - No
useRunStreamimport inCinematicViz. The hook stays at the page level (Threads/Show passes its events through to CinematicViz). Keeps CinematicViz a pure render component — no live data subscription, easier to test in isolation. Chunk 11 will keep the same separation when adding playback controls. - Chunk 10 complete with all four surfaces wired. PipelineProgressBar (chunk 1), VocabSidebar (chunks 3c + 10a), LayerCounterHud (chunk 10a), ChatBubble (chunk 10b). The off-canvas persistent UI now reflects scene + run state continuously.
Decisions (chunk 10a): (LayerCounterHud + VocabSidebar reverse-lookup.)
- PipelineProgressBar was already wired in chunk 1. The 21-segment control +
onSelectSceneclick-to-jump existed since the chunk-1 stub plus the chunk-3a integration. No work needed in chunk 10. Status table line in the chunk-10 task list now reads "✅ already wired (chunk 1)". - LayerCounterHud derives
currentLayerfrom viz state, not reallayer.advancedtelemetry. Spec line says "visible during scenes 5-12" — the HUD reflects where the viz is, not raw run progress. During scenes 5-11 we show the representative layer (= 1) since those scenes visualize a single layer's worth of computation. During Scene 12 (tower view) the HUD's value matches Scene 12's in-canvas counter exactly via the same five-phase math. Reallayer.advancedevents stay out of scope; chunk 12 (perf) or chunk 14 may add a "real layer" indicator if needed. - Phase math for Scene 12 inlined in
CinematicViz, not re-imported fromlib/towerCamera. Considered importingcounterValuefrom chunk 7's lib. Rejected because the HUD already mirrors the in-canvas counter exactly; re-importing would tightly couple the persistent HUD to the scene-internal math. The 10-line inline equivalent is its own contract — chunk 7'stowerCamerais testable + the HUD is testable; both can evolve independently without circular dependency. currentLayeris the only Scene-12-specific calculation in CinematicViz. Other scenes contribute a constant1to the HUD. Trade-off: makes the HUD "wrong" if some future scene visualizes multiple layers without being Scene 12. Acceptable because Scene 12 is the only multi-layer scene in M13's spec; chunks 11+ don't add new layer-wise scenes.- VocabSidebar gets an explicit
highlightTokenIndexoverride, not a state-shape change. Two highlight modes now coexist: the chunk-3c "most-recently-revealed" implicit highlight, and the chunk-10 "reverse-lookup" explicit override. The override (when set) takes precedence and uses an emerald ring vs. the forward-mode cyan ring. Different palette = different meaning ("you're going forward through tokens" vs. "you're looking back at a generated one"). scrollIntoViewusesblock: 'nearest', not default behaviour. Default scrollIntoView can move the page-level scrollroot when the sidebar already has the row visible.nearestonly scrolls the sidebar's ownoverflow-y-autocontainer. Smooth scrolling because the lookup beats are spaced ~250-400ms apart; instant scrolling would feel jarring at that cadence.- Reverse-lookup row matched by token string, not by vocab index. The Scene 20 generated tokens have synthetic vocab indices (chunk 8b's
syntheticTokenStringbank); the sidebar's vocab indices come from the realjs-tiktokentokenization. They won't match. String matching ("does any sidebar row have this string?") gives a sensible "the model picked this word, here's where it lives in vocab" demo even with synthetic tokens. Real run wiring in chunk 11 or beyond will pass real vocab IDs and the match becomes index-based. - Out-of-range
highlightTokenIndexis a no-op, not an error. Test pins this — passing5when only 1 row is visible silently disables the override. Lets the chunk-10 wiring be optimistic without needing per-render bounds checks. - Test for
scrollIntoViewmonkey-patchesHTMLElement.prototype.scrollIntoViewper-test. jsdom doesn't implement the API; the spy + try/finally restoration captures the call without polluting other tests. Pattern is documented inline; reusable if chunk 11's scrubber needs similar treatment. - Scene 20 reverse-lookup highlights during the REVEAL phase only (t < 0.75). After t=0.75, Scene 20 enters the EOS badge + flourish phases; the lookup widget is hidden, so the sidebar highlight should clear too. CinematicViz computes this gate explicitly to match the in-canvas widget's gate from chunk 9b.
Decisions (chunk 9b): (Scenes 19 + 20 — completing chunk 9. 20 of 20 cinematic scenes are now wired.)
- Scene 19 = standalone explanatory scene, not overlay during Scene 18. Spec describes the KV cache reveal as a 2s overlay during Scene 18's first loop iteration. Implemented sequentially (matches the chunk-9a decision for the same reason). The drawer slides in from the right, K + V matrices fill row-by-row, then a "new-row" marker pulses at the bottom to communicate the savings.
- K + V matrices rendered as separate small heatmaps (16×8 cells each). Real K and V are
(N_tokens × head_dim); we render 16 cols × 8 rows side-by-side with viridis fill. The cached-vs-new split is conveyed via the row-fill animation (cached rows fade in viridis; the new-row position gets a separate fill color pulse). - Lock icon = inline SVG path, not
lucide-reactimport. The icon is one-off (only used in Scene 19), so a 7-line inline SVG saves a tree-shake assertion + decouples the scene from lucide's API surface. - Scene 20 phase budget: 75% reveal / 15% EOS badge / 10% flourish. The reveal phase walks through every emitted token (capped at 8) showing the per-token reverse-lookup widget. EOS badge then appears as a transition. Final 10% applies the completion flourish (background dim + chat-bubble glow + "Inference complete" text overlay).
- Reverse-lookup widget is in-canvas, not extending the off-canvas
VocabSidebar. Per the chunk-9 design decision, the off-canvas sidebar wiring is chunk 10's. The in-canvas widget showstoken #N → vocab[idx] → "string"per token, replacing once per iteration. Cleaner separation; chunk 10 can later add the off-canvas highlight without touching Scene 20. - Completion flourish via background-color + box-shadow on the chat bubble. Considered: full-canvas overlay with backdrop-filter blur. Rejected — Tailwind's
backdrop-filterisn't loaded in chunk 9b's set, and the rgba dim + emerald glow on the bubble achieves the "everything dims, success bubble glows" beat with two style props per element. - Chat-token pills cap at 8. Same readability rationale as earlier chunks; long generations would overflow the 600px chat-bubble width. Render-only cap;
generatedTokensretains all entries. - EOS badge uses amber accent (
amber-500/15+amber-300), not emerald. The completion is celebrated via the chat bubble glow; the EOS badge is a status marker mid-transition. Different palette = different meaning. Test asserts the badge presence at the right t-window. - Active-token highlight in chat: scale 1.1, opacity stays at 1. Reveal-phase tokens before the active index stay at full opacity (
hasLandedflag); tokens after the active index dim to 0.25. The active token also scales up briefly. Visual: tokens light up as the reverse-lookup widget walks through them. - Both Scene 19 and Scene 20 use identity
transform(). They're narrative/camera scenes over already-populated state (generatedTokensfrom Scenes 17 + 18). Returning identity by reference matches chunk 7's Scene 12 pattern and keepsObject.isequality stable for downstream consumers. - Chunk 9 close-out: 20 of 20 scenes are now in
ALL_SCENES. The cinematic narrative now plays end-to-end from prompt entry through completion flourish. Remaining M13 chunks 10-14 are wiring + polish (persistent UI, playback controls, perf, accessibility, closeout). The 20-scene contract fromSCENE_IDSinScene.ts:27-49is fully satisfied.
Decisions (chunk 9a): (Scene 18 only.)
- Linear scenes 18 → 19 → 20, not overlay scenes. Spec describes Scene 19 as "during Scene 18's first iteration" and Scene 20 as "continuous during Scene 18". The existing
SceneRunnerwalks scenes strictly sequentially; refactoring it to support overlays would touch every shipped scene for backwards compatibility. The pedagogical beats (loop → cache → detokenize → EOS) land in the same order; only the temporal overlap is forfeited. Architectural debt: a future polish pass could addScene.overlay: booleanto the contract. - Loop iteration pacing extracted into
lib/syntheticAutoregression.ts.LOOP_ITERATION_DURATIONS = [2000, 1500, 1100, 800, 600, 500, 400]matches the spec's "~2s for #2 → ~200ms by #10+". Sum = 6900ms; scene duration =LOOP_TOTAL_DURATIONso any future tuning of the array stays in one place. Pure-functioniterationAtTime(t, iters)maps the scene's t to current iter + local t — 7 of the 16 lib tests cover this. - Synthetic continuation = 7 iterations, not parameter-driven. Real runs will be variable-length (WebSocket-driven); for chunk 9a the canned 7 is enough to demonstrate the loop concept + decelerating pace. Chunk 10 swaps the synthesizer for a WebSocket-driven version with the same signature.
- Per-iteration phases mirror chunk 5/6 patterns: 0..0.3 input flash, 0.3..0.7 compute beam, 0.7..1 new-token landing. No per-iter useMemo (the iteration list is memoized once at scene-start).
loopIterationsfield inPipelineState, type aliased. The autoregression module definesLoopIteration;Scene.tsdefinesLoopIterationStatewith the identical shape. Two names soScene.tsdoesn't import from the autoregression module (the module importssyntheticTokenString; circular if both were the canonical type). Documented at the field declaration.- Scene 18
transform()appends 7 tokens togeneratedTokensAND populatesloopIterations. The cumulative tray is reconstructed at render-time from both fields. Scrubs through Scene 18 see consistent counts. - In-canvas chat tray ("Chat bubble · N tokens") is the same surface as Scene 17's tray. Spec calls for both a chat tray and an off-canvas chat bubble; chunk 9a builds the in-canvas surface only. Chunk 10 wires the off-canvas
ChatBubbleto read from the samegeneratedTokensfield. - Beam-through-layers icon is a 5-floor SVG sketch, not a re-render of Scene 12. Showing the actual tower view per iteration would be a 5-second nested animation × 7 iterations. The sketch icon (5 thin floor rects + green beam line during the compute phase) conveys "compute through layers" in ~20 lines of SVG.
- Iteration counter format: "Iteration 4 / 7 · this token: 800ms". The duration label exposes the deceleration explicitly so the viewer can see "tokens land faster as the loop continues" without needing to time anything mentally. Test asserts the bounds.
- Scene 19/20 deferred to chunk 9b. KV cache + detokenization + EOS flourish land in a separate commit so 9a's loop animation is reviewable in isolation. Chunk-9 checkbox stays unchecked until 9b lands.
Decisions (chunk 8b): (Scenes 15 + 16 + 17 — completing chunk 8.)
- Softmax + sampling helpers extend
syntheticLogits.ts, not a newsyntheticSampling.ts. All five helpers (softmax / topPCutoffIndex / sampleByMode / SamplingMode type / syntheticTokenString) operate on logits-derived data. Keeping them colocated means one import line in each scene; one file to grep. Total file is ~280 lines — under any size threshold worth splitting. softmax()is numerically stable via standard max-subtraction.exp(v - max) / Σ exp(v - max). Handles[1000, 999, 998]withoutInfinityblow-up; test pins this.sampleByMode()returns the sorted-position index, not the original vocab index. Caller maps back via the indexed top-K list. Saved a parameter (no original-indices array to thread through) and matches how Scene 16's bar chart is rendered (bars at positions 0..K-1 even when their vocab indices are scattered across 128k).- Sampling defaults: greedy / k=40 / p=0.95 / T=1.0. Greedy is the safest default — deterministic output, no temperature surprises. K=40 / P=0.95 match common production defaults (OpenAI Chat API, Llama config). T=1.0 = no temperature scaling.
samplingMode/samplingK/samplingP/samplingTemperatureadded to PipelineState. Chunk 10 will wire fromrun.parameters(JSON column onruns). For chunk 8b, the values come from CinematicViz initial-state defaults. The scene also renders the active values in the mode label — switching modes through any future control would update the visual immediately.- Sampling render is dynamic per
samplingMode. Greedy → dart hovers over bar 0 the whole scene. Top-K → bars beyond k fade during narrow phase, dart wobbles among survivors. Top-P → fill line sweeps L→R untiltopPCutoffIndex(p), bars past the line fade. All three modes share the same chosen-bar pulse + winning-token flash beats. - Dart wobble (
computeDartIndex) uses linear progression + sine jitter. Could have been a deterministic random walk; rejected as too jumpy. Sine jitter on top of a linear sweep through[0, cutoff)reads as "the dart is exploring options" while landing on the actual sampled index by t=0.7. - Synthetic token-string bank: 16 plausible mid-sentence English tokens. Strings include leading-space variants (
' the',' a') so concatenation in the chat-bubble looks like natural text. Real tokenizer-grounded strings replace these in chunk 10 via the WebSocket-streamed winner. Tests assert non-empty + deterministic + wrap-around behaviour. - Scene 17 in-canvas tray, not the off-canvas
ChatBubble. Per phase1.md:1032, the off-canvasChatBubblewiring is chunk 10's job (it needs the live WebSocket stream too). Chunk 8b's in-canvas "Generated so far" tray builds the same semantic surface fromgeneratedTokensin PipelineState — chunk 10 just adds the off-canvas mirror. - Scene 17's
transform()is idempotent via last-entry check. If the tail ofgeneratedTokensalready matchessampledToken, the transform returns the same state reference. Required so a scrub through Scene 17 doesn't double-append. Test pins this with atransform(transform(state))invariant. - All three scenes follow the chunk-5 React-component-with-
useMemopattern. Scenerender: (t, state) => <SceneComponent t={t} state={state} />; the actualuseMemocalls live inside the inner React component. Hooks can't be called in the bare render callback (initial draft did this; fixed before testing). Documented for chunk-9 onwards. - Probability bars cap at 16 (
TOP_K_RENDER). Bigger K would push individual bar widths below ~37px on the 600px canvas region — labels and pulses become hard to see. 16 is also enough to make the "long tail collapses near-zero" beat visually clear: bar 1 dwarfs bars 6+. - Chunk 8 = 8a + 8b, single chunk-8 checkbox flipped only now. The phase1.md status row goes from "chunks 1-7 done" to "chunks 1-8 done" with this commit. 8a's decisions block stays in place; 8b's block sits above it.
Decisions (chunk 8a): (Scenes 13 + 14 only.)
- Split chunk 8 into 8a / 8b at the logits boundary. Five scenes (13-17) is the largest scene group in M13; splitting at the logits computation makes each commit individually reviewable in ~700 lines. 8a covers the projection-out side (final-norm + LM-head matrix); 8b covers the probability/sampling side (softmax wave, mode-aware dart, emit).
- Scene 13 is a separate file, not a
LayerNormScenefactory refactor. Spec says "same squish animation as Scene 7, briefer" — could have factoredLayerNormScenelike chunk-6 did withcreateResidualScene. Rejected because the chunk-4 Scene 7 implementation is already committed and a refactor would touch shipped code unnecessarily. Cost: ~100 lines of shared rendering grammar duplicated. Worth it forchunk-4 stays frozensimplicity. If a 3rd norm scene ever lands, refactor then. vocab_sizetype plumbed through CinematicViz, but pages still passnull. The DB schema has novocab_sizecolumn onruns. Added the optional field to theCinematicViz.modelinterface so the plumbing exists when either (a) a future migration addsruns.vocab_size, or (b) chunk 10 sources it frommodels.vocab_size(if added there). Scene 14 falls back to 128,000 (Llama-3 / GPT-4o class) when null. Documented inline inCinematicViz.tsx. The user choice was "Plumb model.vocab_size, default 128k if null" — this delivers the plumbing spirit without an unrelated DB migration.- 128k label is dynamic, formatted via
formatVocabCount(n). Realvocab_sizefor known models ranges 32k (older LLaMA) through 256k (some Gemma variants). The label reads "4096 × 50,000" or "4096 × 128,000" depending onstate.vocabSize. Tests cover both branches. - Logits stored at full vocab length; rendered at 1024 cells via max-pool downsample. Drawing 128,000 SVG rects at 30 FPS is a non-starter (12× the render budget of every other scene combined).
downsampleLogits()max-pools to 1024 cells so the spike positions stay visible. Downstream Scene 15 still operates on the full-length array for bar-chart accuracy. - Synthesizer: 8 hot-spike positions injected over cool xorshift noise. Mirrors the real LLM logits sparsity pattern ("most positions near zero, a handful clearly dominate"). Spike heights decay from
HOT_AMPLITUDE = 4down to~1.3so top-1 dominates, top-2 second, etc. — the "natural shape" the spec calls for. Test asserts >90% of cells are cool (abs(v) < 0.5). - Logits seed =
(seedKey, sumOfLastVector). Different last vectors produce different distributions (test enforces). The seedKey is a per-scene constant (0xc0ffee) so multiple renders within the same scene match. For chunk 10's run-determinism, the seedKey will incorporaterunId. Documented in the helper. - LM Head matrix render mirrors Scene 5's chunk-4 implementation.
WeightFogpatch + 14 horizontal accent lines + center label card. Same approach kept the "this is a massive matrix" beat consistent between embed lookup (Scene 5) and unembedding (Scene 14) — they're the same operation in two directions. The label flips: "Embedding table" / "vocab × hidden dim" → "LM Head" / "hidden dim × vocab". - Beam = single green vertical line, opacity-gated to projection phase. Considered: multiple sweep lines, particle stream. Rejected as visually noisy for a 3s scene with a lot already going on (input strips + matrix materialization + logits emergence). Single beam reads clean.
- Phase boundaries inline (not extracted to a lib). Considered: factor scene phases into a
lib/scenePhases.tshelper. Rejected as YAGNI — the phase math is 4 lines per scene and the phase boundaries differ per scene by design. The chunk-7towerCameraextraction was justified because it had non-trivial easing curves; chunk 8a's phases are linear ramps that read better inline. - Cap render at 6 input strips for Scene 14, 6 row strips for Scene 13. Same readability rationale as chunks 4-7 — long prompts can't fit more vertical/horizontal density without becoming a wall of strips. The
transform()outputs cover every token regardless; the cap is render-only. The "last token" highlight usesMath.min(SAMPLE_LAST_INDEX, 5)so the last visible strip is the highlighted one even when the actual last index is beyond the cap. - Logits-heatmap normalization is
(v - min) / (max - min), not abs/symmetric. Logits range from large positive to small negative; using viridis on the[min, max]range maps the spikes to the warm end of the palette (the spec's "few hot spikes") and the cool noise to the cold end. Symmetric normalization would put min at the cold end and 0 at the middle, washing out the spike effect.
Decisions (chunk 7):
- Camera math extracted into
lib/towerCamera.ts, pure functions only.packetFloor(t, N),cameraScale(t),counterValue(t, N),blurAmount(t),towerPhase(t),easeOutQuad(x). The scene component becomes thin glue: no easing math, no phase-boundary literals, no monotonicity guarantees inline. 27 of the chunk's 43 tests live intowerCamera.test.ts— the hardest behavior (phase boundaries + ease-out monotonicity + clamping at t ∈ {0, 1}) tested without React in the loop. - Five-phase animation breakdown (10s total): reveal 1s / follow 1s / blur 3s / slow 3s / rezoom 2s. Picked the middle of the spec range (8-15s) so the blur phase gets its full ~3s budget per phase1.md. Boundary fractions are 0.10 / 0.20 / 0.50 / 0.80 / 1.0 — the cleaner-than-it-looks math makes every phase's
progress = (t - start) / (end - start)easy to reason about and easy to test. - SVG tower, not Three.js / CSS-3D. Spec doesn't require 3D; the "camera zoom" is achieved via a single CSS
transform: scale()on the SVG container. Matches the rest of M13's SVG-first visual language (chunks 2-6 all use SVG). Cost: no real perspective effect; viewer reads it as "the camera pulled out" because the floor count went from 1 visible to 32 visible, not because of a perspective shift. Acceptable per visualization.md's "Option A (recommended)" framing. packetFlooruseseaseOutQuadonly for the blur phase. Other phases are linear, including the slow phase (N-2 → N), because the deceleration in the slow phase comes from the physical layer count (just 2 floors) rather than from the easing curve. Eased blur produces the spec's "Layer 03… 17… 42… 78…" tick pattern naturally — early jumps are big, late jumps are small.totalLayersadded to PipelineState, populated byCinematicVizfrommodel.layers. Same pattern as chunk 6'sarchitectureType. Defaults tonull; Scene 12 falls back to32(Llama-7B / GPT-3.5 class).model.layerswas already plumbed fromruns.total_layers→ page → CinematicViz props, so this is one-line wiring.- Persistent off-canvas
LayerCounterHudstays a stub. Per phase1.md:1033, wiring the persistent HUD (visible across scenes 5-12) is chunk 10's task list. Chunk 7's scope: the in-canvas tower view + its own counter, not the persistent one. The user-visible difference: the off-canvas pill still reads "Layer — / —" outside Scene 12; the in-canvas counter inside Scene 12 reads "Layer 17 / 32" etc. Chunk 10 unifies them. - Packet = horizontal row of dots, one per token (capped at 6). Spec says "glowing packet representing your current sequence"; one dot per token communicates "this is the sequence ascending the tower" more clearly than a single rect. 6-cap matches the visual budget — narrower towers can't fit more dots side-by-side. Glow halo (ellipse with low opacity) sits beneath the dots.
- Motion-blur streak = single tall transparent rect. Considered: multiple semi-transparent packet ghosts. Rejected — DOM cost scales with blur intensity, and one rect with
height = 60 * blurAmount(t)reads as "the packet is moving fast" just as well. Streak appears only whenblurAmount(t) > 0(i.e., during the blur window). - Counter format is
String(layer).padStart(2, '0')+/ ${String(N).padStart(2, '0')}. Two-digit zero-padding so the rapid tick doesn't visually jump width (Layer 9 / 32→Layer 10 / 32would shift right by one digit). Spec mention of "Layer 03…" implies the zero-pad explicitly. - Active-floor divider gets a brighter stroke. Floor lines render at
#1e293bopacity 0.5; the one within ±1 of the current packet position gets#67e8f9opacity 0.9. Reads as "the packet is on this floor" without needing a separate highlight rect. testIds use the absolute floor number (scene-12-floor-${floorNumber}), not the array index, so tests can assert "floor 17 is highlighted at the right t." - Final-layer detail panel is a compact strip stack, not a Scene-8/10 miniature. Considered: re-rendering attention matrix + FFN pipe in miniature inside the rezoom panel. Rejected — that would duplicate ~300 lines of scene logic with different sizing. The detail panel instead shows a viridis output strip per token from
residualOutput2(the layer-stack output), labelled "Final layer · attention + FFN" with a footnote pointing at Scene 14. Carries the "you finished the tower" beat without re-doing the per-layer math visually. transform()is identity. Scene 12 doesn't produce new vectors — it's a narrative / camera scene overresidualOutput2(which Scene 11 already populated). Returning identity by reference keeps downstreamObject.isequality checks happy and matches the chunk-5 idempotency pattern.- Phase label in the HUD ("Phase: blur") is intentional debug-y polish. Helps reviewers + future contributors orient when scrubbing. Could be hidden behind a debug flag later; for chunk 7 it ships visible to make the phase machinery obvious. Test asserts the text content matches the active phase string from
towerPhase(t).
Decisions (chunk 6):
- Scene 9 + Scene 11 share
ResidualScenevia acreateResidualScene(config)factory. Spec explicitly says Scene 11 is the mirror of Scene 9 ("Same as Scene 9 — pre-FFN ghost streams merge with FFN output"). One component file, one render path, two registry entries — Scene 9 wirespositionEncoded + attentionOutput → residualOutput; Scene 11 wiresresidualOutput + ffnOutput → residualOutput2. The config also carries the caption + ghost/main labels + testId prefix so tests can assert per-scene structure. Saved ~120 lines of duplication and a future divergence-risk. - Residual ghost source =
positionEncoded, notlayerNormed. Modern transformers use pre-LN:x + Attention(LayerNorm(x)), wherexis the input to layer norm — i.e.,positionEncoded(Scene 6's output). Spec phrasing "kept ghosted in the background since Scene 6" matches. Trade-off: doesn't match post-LN architectures (original Transformer, BERT) exactly, but pre-LN is the modern default and matches every model we'll see at runtime. architectureTypeadded toPipelineState, populated byCinematicVizfrommodel.architecture_type. Drives the SwiGLU/GELU pick in Scene 10. Alternative was enlarging theScene<I, O>contract to take a separate model handle — rejected as cross-cutting and best left for chunk 10's full WebSocket wiring. PipelineState already passes through every scene;architectureTyperides along. Defaults tonull(idle screen) → falls back to GELU.pickNonlinearity()is a string-heuristic, not a vendor-aware lookup. Llama / Mistral / Qwen / Gemma / Phi / any-with-moe→ SwiGLU. Everything else → GELU. Hand-rolled heuristic matches what the M3 OpenRouter registry actually returns forarchitecture_typestrings without round-tripping through a vendor-specific table. If a new SwiGLU family ships (e.g., Apple's Foundation), we add a substring; if the existing heuristic wrongly classifies, the visual cost is the same wavy color shift with a different label — pedagogically resilient.- FFN expansion synthesizes 4× cells in the middle phase, contracts on exit. Spec: "the expanded middle section can briefly show many more cells, emphasizing where most parameters live."
expandToFFNDim(v, factor=4, tokenIndex)interpolates adjacent cells with xorshift32-seeded ±5% jitter so the expanded view looks like a richer version of the input, not a 4-pixel duplicate.contractFromFFNDim()averages groups back down — inverse operation (jitter averages out, interpolation cancels). - The "non-linearity wave" is a single hot column sweeping L→R. Implementation choice: one amber
<rect>of width ~2 cell-widths sweeps across[0.5, 0.85]of t. Considered: per-cell color phase shift propagating through. Rejected as visually noisy + ambiguous about what "pass through a non-linearity" means at a glance. The single sweeping column reads cleanly as "this stage is happening now." - Sparkles are 10 deterministic-seeded dots, opacity-gated to expansion peak.
sparklePositions(tokenIndex, count=10)returns(x, y) ∈ [0,1]²per token;sparkleEnvelope(expansionAmount)returns 0 untilexpansionAmount > 0.6. Sparkles appear only during peak expansion, fade in/out cleanly, and reposition per token. Replay-deterministic per token-index seed. - FFN renders only the first 4 tokens (cap). Per-token pipe is 200-400px wide depending on expansion phase; 4 stacked rows fit the viz region comfortably with the caption + non-linearity label + output label. Tokens 5+ aren't part of the PipelineState rendering for Scene 10; their
ffnOutputis still computed intransform()so chunk-7 onward sees the full sequence. The cap is render-only. - Residual scene renders the first 6 tokens (cap). Two-column layout (ghost left, main right) with a center
+is more horizontally efficient than the FFN pipe rows, so 6 fits where FFN takes 4. Same render-only cap pattern. - Pipe outline + sparkle SVG share one
<svg>per token. Considered: separate SVGs for pipe outline / strip cells / wave column / sparkles. Rejected as 4× the DOM nodes per token = 16× for the scene at peak (4 tokens × 4 elements). Single<svg>per row → one DOM node per token. The pipe-outline border is on the parent<div>viaborder-cyan-500/30(CSS), not in SVG. - Phase-window opacities use the chunk-5 leading/middle/trailing pattern. Scene 9 / 11 use a single in-phase + out-phase render (the
outputOpacityis the only crossfade); Scene 10 uses an envelope functionexpansionEnvelope(t)that holds at 1 betweent=0.4andt=0.6(the peak hold). Both patterns avoid thet = 1fade-to-zero bug documented in chunk 5. applyFFN()returns same-length output, but internal compute touches 4× cells. Caller perspective: vector in, vector out, same dim. Internal: expand → activate → contract. Pedagogically the expansion is the point; the result-dim invariant lets downstream scenes (chunks 7+) consumeffnOutputwithout re-shape gymnastics. The 4× world only exists during the scene's render window.
Decisions (chunk 5):
- Reuse M8
lib/attentionPattern.tsas the matrix generator. Chunk 1 kept the file specifically for this chunk. Same causal mask + distance-decay + xorshift32 noise + row-sum-to-1 invariant; chunk 5 wraps it ingenerateMultiHeadMatrices()which folds head index into the seed (seedLayer = (layerIndex × 1000 + h × 7) mod totalLayers) so different heads render as visibly different patterns. No re-implementation of M8 logic. - Defer
run.idfrom the seed, plumb in chunk 10. Spec asks for(run.id, layer_index, token_index)determinism. For chunk 5 the seed is(tokenIndex, headIndex, layerIndex=0)only —runIdplumbing throughCinematicViz→PipelineStateis cross-cutting (touches Threads/Show, Runs/Replay, Share/Replay) and rightly belongs to chunk 10 where the WebSocket event stream wires up. Scene 8 is still frame-deterministic for a given prompt; replay determinism across runs of the same prompt holds trivially. - Representative 4–6 fanned heads, not literal 32. Spec says "stack of ~32 transparent layers; fan them out briefly to show parallelism, collapse back." Literal 32 N×N matrices is ~7,200 SVG rects at the peak of the fan (for N=15) — performs OK but visually overwhelming. Chunk-12's degraded-mode plan already calls for "single representative head shown" under FPS pressure. Default is 6; the real
model.attention_headscount goes in the caption ("Attention · showing 6 of 32 heads"). Perceptual clarity over literal count. - Q / K / V are sign-flip projections of the same input embedding. Real transformer Q/K/V come from
W_Q/W_K/W_Vlearned linear projections. The chunk-5 stand-in: xorshift32-seeded sign-flip mask per cell, keyed by(tokenIndex, headIndex, role_offset). Output magnitudes equal input magnitudes; only signs flip. Trade-off: doesn't model the projection dimension reduction (real K/V live inhead_dim = hidden_dim / heads); the visualization shows Q/K/V at the same length as the input strip. Pedagogical clarity over mathematical fidelity, per the M13 architectural note. splitQKVkeys roles by distinct seed offsets (0x11111111/0x22222222/0x33333333). Adjacent role bits ensure Q/K/V mask different cells from each other for any non-trivial input. Test explicitly asserts "Q !== K !== V" for at least one cell across the triple.- Illustrative softmax beat, not faithful pre→post. The matrix from
attentionPattern.tsis already row-normalized. Scene 8b's "softmax wave" plays as an amber horizontal line that sweeps top-down acrosstokenList.lengthrows during phase 8b — visually communicates "each row gets balanced" without requiring a separate pre-softmax helper. Matches the M13 "illustrative not measurement" principle. - Three sub-beat cross-fades via leading/middle/trailing opacity helpers, not a single
phaseOpacity(). First iteration used one helper that fades in/out at both boundaries; broke att = 1because the trailing phase faded out into the void. Split intoleadingPhaseOpacity(no fade-in),middlePhaseOpacity(both),trailingPhaseOpacity(no fade-out). Clear intent at the call site + correct behavior at the t-boundaries. - 8b multi-head fan: bell-curve open/close on a single normalized 8b clock.
bell(tIn8b, center=0.5, width=0.25)peaks halfway through 8b and falls off symmetrically. No explicit "open then close" sequencing — the bell curve handles both. Falls off cleanly back to a single centered matrix at the 8b→8c boundary. - Softmax wave only animates on head 0. During the fan-out moment, all 6 heads are visible but the amber sweeping line is only drawn on the representative head (head 0). Drawing the wave on all 6 would compete with the multi-head reveal beat — separate visual ideas.
- Scene 8 output strips reuse
viridisvianormalize()from M12 chunk 4. Post-blend V values aren't naturally in[0, 1]for color mapping; normalize stretches the row's min-max into the palette domain. Same helper M9'sAttentionHeatmapalready uses, so cross-scene color language stays coherent. - Q/K/V row cap at 8 tokens, output blend cap at 8 rows. Same readability rationale as chunk 4 Scene 7. Three roles × 8 tokens = 24 strips already pushes the viz region; more would be unreadable. The cap is on render only — the underlying
qkvarray in PipelineState contains entries for every token so chunk-6/9 scenes downstream see the full set. transform()falls back throughlayerNormed → positionEncoded → embeddings → tokensto source the input. The chunk-3aSceneRunnerdoesn't yet auto-calltransform()between scenes (chunk-10 wiring). Each scene must therefore be robust to the upstream fields being absent — Scene 8 derives whatever it can from the most-derived field available, all the way back to synthesizing from tokens.
Decisions (chunk 4):
syntheticEmbedding(tokenId, dim)is seeded by token ID alone, not(run.id, token_index). The M13 architectural note specifies seeds from(run.id, model_metadata, token_index, layer_index), but for Scenes 5-7 the "same token → same embedding row" invariant is the load-bearing pedagogical beat: a viewer should be able to recognize a recurring word by its color signature in the heatmap strip. Keying offrun.idwould break that recognition across replays of the same prompt. The fuller seed set is reserved for Scene 8+ where per-position activations diverge from per-token embeddings.- xorshift32 PRNG, same family as the chunk-2
tokenIdToHue. Same generator + same seed-mix constant (0x9e3779b9golden ratio) so a debugger comparing pill color → embedding strip can reason about both from one mental model. Zero-state guard (if (s === 0) s = 1) coverstokenId = 0— xorshift32 collapses to all-zeros without it. VISIBLE_EMBEDDING_DIM = 128, label says 4096. Realmodel.hidden_dimis 4096-16384 depending on the model; rendering that many SVG<rect>s per token at 30 FPS would tank performance.VectorStrip's chunk-2totalLength={4096}prop carries the implied extent via the "showing 128 of 4096 cells" aria-label, so screen-reader users + the matrix caption both communicate the real scale.applyPositionRotationis illustrative RoPE, not faithful RoPE. Real RoPE uses dim-pair-specific frequencies (θ_i = 10000^(-2i/dim)); we use a single global(positionIndex × π / 16)angle so the visible color shift is monotone in position. The pedagogical beat is "position changes the vector"; mathematical fidelity is reserved for a future explanatory mode that doesn't exist yet.layerNormalizeaddsε = 1e-6inside the sqrt. The constant-input edge case ([5,5,5,5]) has std=0; without ε it would divide by zero. Test pins the behavior. Matches PyTorchnn.LayerNormdefaulteps=1e-5to within a small constant factor — both are within float32 precision for our value range.- Scene 5 "matrix" is 14 horizontal accent lines +
WeightFog, not a rendered grid. The label "128,000 × 4096" sells the scale; drawing actual cells would be visually noisy and burn the GPU. The accent lines fire row-by-row asbeamPhaseadvances so the "rows lighting up" beat reads even without a real matrix behind them. - Scene 5 staggered detach: leftmost token flips first. Per-token
localPhase = detachPhase * 1.5 - i / N— the cascade is visible and matches the left-to-right reading order. Without staggering, all tokens would flip at the same instant and the transition would feel like a layout shift rather than a beat. - Scene 7 dual-render with absolute overlay, opacity crossfade. Both
<BarChart>and<HeatStrip>mount in the samewidth × heightabsolute container; opacity props discriminate. Avoids layout shift mid-squish (a single component that swapped DOM shape would have caused a one-frame reflow on the parent flex row). Tests assert both can benullatt = 0and the heat-strip is visible att = 1. - Scene 7 caps render at 8 rows. Long prompts (50+ tokens after the chunk-3 tokenizer) would push the strip stack past the viz region's vertical bounds. Capping keeps the stack readable + the "squish" effect legible. Scenes 5 + 6 use horizontal flex-wrap so they self-manage; Scene 7's vertical stack needs the cap.
- Each scene's
transform()is idempotent + returns the same reference if already populated.Object.isequality for the no-op path lets React downstream-effect tests rely on referential stability. Asserted for all three scenes —transform(transform(state)) === transform(state). - Scenes have
render()fallbacks that synthesize embeddings on the fly when state hasn't been transformed yet. The chunk-3aSceneRunnerskeleton doesn't yet calltransform()between scenes (that wiring lands in chunk 10). Until then, each scene synthesizes fromstate.tokensdirectly so isolation-tests with just atokensarray render correctly.
Decisions (chunk 3): (backfilled in chunk 4 — chunks 3a/3b/3c committed without an inline decisions block.)
js-tiktokenover@dqbd/tiktoken. The spec hedged: WebAssembly speed vs pure-JS portability. Pure JS won — ~200KB bundle (vs ~600KB for the Wasm variant), zero runtime requirements beyond ES2020, identical behavior across Chrome/Firefox/Edge/Safari without Wasm-loading edge cases. The steady-state per-call latency is slower than the Wasm path but Scene 3's 4500ms budget swallows it: tokenization runs once on prompt arrival and the result is cached for the whole pipeline.cl100k_baseas the universal vocab. GPT-3.5/4 tokenization is the most-recognizable modern frontier-model encoding; the M13 viz is pedagogical, not measurement, so a single representative vocab beats per-vendor splits (Llama, Mistral, Claude would each pull their own bundle). Vendor-specific tokenization can land as a chunk-14 polish if anyone asks.- Codepoint-per-token fallback when js-tiktoken fails to load. Network-flaky environments still get a degraded-but-functional viz: every codepoint becomes one pseudo-token. Scene 3 still animates; the BPE-merging beat just renders as "every char is its own token" which is the trivial-correct case. Better than a hard crash.
PipelineStateis a wide-record (all-optional fields), not a typed I/O chain. Considered:Scene<I, O>generics enforcing that each scene'sOis the next'sI. Rejected for verbosity — every scene-pair would need explicit type plumbing, and the wide record matches how the viz actually thinks about state: at any point the full accumulated derivation is available for the current scene to reference. Inline comment inScene.tslocks in the choice.useSceneRunnerpre-walks the registry once + cachespipelineInputs[i]. O(1) jump-to-scene for chunk 11's scrubber. Trade-off: every scene'stransform()runs on registry change even if the user never visits that scene; acceptable because transforms are pure and cheap. Re-walks oninitialStatechange (new prompt) orsceneschange (developer flow).useSceneRunnerresets to scene 0 oninitialStateorsceneschange. Both prompt-change (user submits a new prompt) and registry edit reset playback to the top. Could have preserved playback position; explicit reset matches the cinematic-narrative metaphor (a new prompt is a new movie) and avoids the "scene index out of bounds" footgun when the registry shrinks.- CinematicViz idle screen when
prompt === null. Distinguishes "no run yet" from "run in progress" without a third state. Pages plumbing (Threads/Show, Runs/Replay, Share/Replay) explicitly pass the prompt; the viz sits at idle until one arrives instead of mounting Scene 0 with empty content. - Tokenizer warmed on CinematicViz mount via
useEffect+loadTokenizer(). Scenes 0/1/2 = 1.8 + 2.4 + 2.5 = 6.7s combined; by the time Scene 3 plays, the tokenizer is cached.getCachedTokenizer()returns null on cold cache so Scene 3 renders a "Tokenizing…" placeholder + falls through to identity-transform until warm. Low-end devices still get a coherent viz. - Scene 1 multi-byte "N-byte" tag for emoji / CJK / accented chars. UTF-8 visibly expanding non-ASCII chars to 2-4 bytes is the most teach-able moment for "tokens emerge from bytes, not characters." The tag fades in after the char's flip animation so the bytes-count is the punchline, not the lead.
- Scene 2
chatTemplateTintsarray carries forward to Scene 3. Per-byte tint group (system/user/assistant/user-prompt) decided in Scene 2 and passed throughPipelineState. Scene 3's BPE colors each emitted token by the modal tint of the bytes it absorbed, so the system prompt stays purple all the way through to the final token row. - Scene 3 per-token duration scales 200ms (common, ≤ 2 chars) → 600ms (rare, ≥ 6 chars), linear between. Mirrors how a reader's eye dwells longer on unfamiliar words. Common tokens fly by; long/rare words get visual emphasis. Total Scene 3 budget (4500ms) accommodates ~10-20 tokens at average pace — adequate for typical short-prompt teaching examples.
- VocabSidebar
revealedCountderived from(sceneIndex, t), not stored in state. During Scene 3, ramps 0 → tokens.length. Outside Scene 3, fixed at 0 (before) or tokens.length (after). Deriving from the scene clock means scrubbing/replay shows the exact same sidebar progression — pure-function of the runner's clock, no state to manage. - VocabSidebar substitutes
·for space and↵for newline. Whitespace tokens otherwise render as empty strings in the sidebar list — invisible. Glyph substitution affects display only; the underlyingtoken.stringis unchanged. Chunk-11's scrollIntoView() will use the substituted glyph for the highlight target.
Decisions (chunk 2):
- All 5 primitives are SVG/HTML/CSS — no Three.js for chunk 2. User choice. The chunk-2 spec hedged on
<ParticleTrail>being Three.js but in pre-chunk discussion the SVG-with-CSS-animation path won: standalone in the DOM, no scene context required, composes anywhere two anchor pixels are known, scales to 50+ simultaneous trails at 60 FPS. A future<GpuParticleSwarm>Three.js component can land when a scene needs many thousands of particles, but every named scene indocs/visualization.mdis covered by SVG. viridisAt(t)is a hand-rolled linear interpolator overVIRIDIS_STOPS, not ad3-scalecall. Cinematic-viz primitives render hundreds of cells per scene at 30 FPS; calling into a d3 scale per cell wastes ~30 µs each. The inline 20-line interpolator reuses the existing M12 chunk-4 5-stop viridis palette so the visual language stays coherent with the M9AttentionHeatmap.normalize()is colocated in the samelib/vizColors.tsso both primitives share the saturation-stretching logic.tokenIdToHueis xorshift32-based, not a CSShsl(var(--token-${id}-hue))lookup. A pure function: deterministic per token ID, no CSS-variable plumbing through every consumer, no global stylesheet. Adjacent IDs land on visually distinct hues (the assertion inTokenPill.test.tsxchecks ID100and101differ by>10°on the hue wheel). Trade-off: not strictly CB-safe (two different tokens can still hash to similar hues), but the label is always the source of truth — WCAG 2.1 SC 1.4.1 satisfied because every TokenPill displays its string inline.- Heatmap aria-labels spell out "showing N of M". When
VectorStriptruncates a 4096-dim vector to 128 visible cells, the aria-label reads "Vector heatmap, showing 128 of 4096 cells, viridis palette" — same forMatrixGrid. Screen-reader users get the implied extent without needing the visual…cue. WeightFogisrole="presentation" aria-hidden="true". It's background fabric — no semantic content. AT should skip it entirely. The pattern id is derived fromwidth × height × densityso multiple fog mounts on the same page don't collide on the<defs>id.ParticleTrailinlines its@keyframesper-mount. The keyframe name embeds the from/to pixels so different trails don't share an animation declaration. Pulse usesmotion-safe:so prefers-reduced-motion freezes it — carries the M12 chunk-3 pattern forward. Guide line isstrokeDasharray="2 3"at 35% opacity so the data path is visible even under reduced motion.data-testidon every cell.vector-strip-cell-Nandmatrix-grid-cell-R-Cmake the heatmap content directly assertable. Lets future scene tests verify "the Q vector for token 3 ended up bright at index 7" without reading the rendered SVG. The 46 chunk-2 tests use these heavily.- Empty inputs render valid (empty) SVGs.
VectorStripwithvalues={[]}renders the bordered SVG container with zero cells.MatrixGridwithvalues={[]}does the same. AvoidsNaN-cellWidth artifacts + lets the parent layout reserve space for the strip even when the data hasn't arrived yet. vizColors.tslives atlib/, notComponents/Viz/. It's a pure helper, not a component. Other libs (existing M9attentionPattern.ts, M8streamMetrics.ts) follow the same pattern. KeepsComponents/Viz/as React components only.- Test run flakiness observed under heavy system load. The full Vitest suite under load avg 77 returned 6-8 spurious failures across runs, with per-test durations stretched 20× normal. Running the same files individually or with light system load shows 100% pass. Documented here so a future contributor doesn't waste time chasing a "regression" that's actually CPU starvation; the canonical fix is to retry after the host quiets down. All 46 new chunk-2 tests pass deterministically when the suite isn't fighting for CPU.
Decisions (chunk 1):
- Build placeholders first, swap imports second, delete last. Pre-discussion plan held: created
Scene.ts+useSceneRunner.ts+ 4 persistent UI stubs +CinematicViz.tsxfirst, then swapped the three page mounts to use the new component, then deleted the M8 viz files. Build stayed green throughout instead of going red mid-chunk. - File location:
resources/js/Components/Viz/— user choice. Same path the deleted M8 files lived in; namespace was empty post-deletion so no collisions. Future imports stay@/Components/Viz/CinematicVizetc. - SceneRunner is a custom hook (
useSceneRunner) — user choice. Matches the existing M8useEventPlayback/useRunStreamshape. Returns{ state, controls }. RAF loop owns advancement via refs (so the per-frame work doesn't trigger React renders); state updates fire only on scene index /t/ playing / speed changes. Tests can call the hook in isolation viarenderHook. - Layout: viz takes 2/3 (
lg:col-span-2), transcript+prompt take 1/3 (lg:col-span-1) — user choice. Flip of the M8-era split. Cinematic viz becomes the primary surface; transcript becomes a supporting column. Mobile collapses to single-column as before. - Deleted
LiveStreamPane+ReplayRightPane+ReplayDebugPaneoutright. The M8 right-pane tab toggle (view-viz / view-embeddings / view-debug) is gone. The M11/M12 spec talked about keeping a "Debug tab last-resort fallback" for chunk 13's tri-state, but the cleanest path was to delete the M8 implementation now and let chunk 13 build a fresh fallback inside CinematicViz when WebGL 2 isn't available — preserving M8 code as "future dead-code reference" was net-negative. SCENE_IDSas aconstarray tuple, not an enum. Gives us both the stable string identifier and ordinal-by-index for free, with TypeScript narrowing via(typeof SCENE_IDS)[number]. Enums would have boxed the integers into nominal types and made the pipeline progress bar'saria-label="Scene ${i}"awkward. The labeled order matters: it's the order the SceneRunner walks.- PipelineProgressBar fully implemented in chunk 1, not stubbed. The 21-segment control surface is static (always Scene 0 through Scene 20, labels locked in
Scene.ts). Chunks 3-9 just change which segment isaria-currentas scenes advance. Clicking any segment firesSceneRunner.setScene. Cheap to build now; saves refactoring later. - Gate notice copy is placeholder. Chunk 13's tri-state (full / 2D-svg / debug-text) hasn't landed yet, so the WebGL-2-unavailable + reduced-motion notices currently say "...once Chunk 13 lands." Chunk 13 will rewrite these strings + actually flip between rendering modes.
tests/Pest.phpgot a globalRefreshDatabaseopt-in. Tests likeAboutPageTestandWelcomePageTestdidn't calluses(RefreshDatabase::class)and were happily reading the dev sqlite db before the M12-followup:memory:switch. After the switch, they hit "no such table: registry_meta" because migrations didn't run. One-line fix:pest()->use(RefreshDatabase::class)->in('Feature')so the trait applies globally to every Feature test. Cheap in :memory: (~50ms per test for migrations); guarantees a clean per-test DB regardless of whether the test file remembered to opt in.phpunit.xmlgot anAPP_URL=http://localhostoverride. Pre-existing latent bug: tests inheritingAPP_URLfrom.envfailed when the dev server ran on a non-standard port (http://localhost:8001in this session). TheExportTriggerControllerTestcache-hit assertion was the canary; pinning APP_URL deterministically in phpunit.xml makes URL-asserting tests env-independent.
Parked into M13 from earlier milestones
- M8 chunk-9 vertical-slice "30 FPS sustained" gate stays — M13 just raises the bar to "20-30 FPS sustained with automatic degrade fallback."
- M9 strict-determinism (replay = original) extends to M13: every scene's animation must be a pure function of
(run.id, scene_index, token_index, layer_index)so a replay is frame-identical. - M10 GIF export needs to capture the new viz. The chunk-2 SVG renderer composes scene-by-scene; chunk 6's Puppeteer skeleton would also need to record the new viz once enabled. Park: M13 closeout doesn't refresh the SVG renderer's frame composition; that work lands in a follow-up M13 polish chunk or M14/M15 if it's required for launch.
Per the chunk-14 spec line — matches the M8 chunk-9 / M9 / M11 / M12 patterns. Run on a 2020-class machine (the SPEC reference) and a fresh login.
1. Visual review per-scene (~5 min). 1.1 Sign in. Submit a fresh prompt (e.g., "Explain the transformer architecture in three sentences."). 1.2 Watch the canvas play through Scene 0 → Scene 20 at 1× speed. Each scene's caption + content should appear at the top of its window; no flicker, no missing labels. 1.3 Click PipelineProgressBar segments out of order — every scene jumps cleanly (no orphan render, no stale data from the previous scene). 1.4 Scrub backward (Step ◂◂) through each scene boundary — earlier scenes still render correctly when revisited.
2. FPS spot-check + degraded-mode visual (~3 min).
2.1 Open the page with ?debug=fps in the URL. The FpsCounter overlay appears top-right showing rolling FPS.
2.2 Open ~10 other browser tabs running anything CPU-heavy (other dev servers, YouTube, etc.) to push FPS below 18 sustained.
2.3 After 2 seconds of sustained low FPS, the counter turns amber, suffix " (degraded)" appears, and you should observe: multi-head attention fans only 1 head (Scene 8b caption reads "showing 1 of 32 heads · degraded"), particle trails stop animating, VectorStrips shorten to 64 cells where they used to show 128.
2.4 Close the heavy tabs. After ~5 seconds of FPS > 24, the counter returns to emerald, the suffix disappears, full-quality rendering resumes.
3. Reduced-motion visual (~2 min).
3.1 OS-level toggle: enable "Reduce motion" in your OS accessibility settings (System Settings → Accessibility → Display → Reduce motion on macOS; Settings → Ease of Access → Display → Show animations off on Windows; gsettings on GNOME).
3.2 Reload the viz page. An amber banner reads "Reduced-motion is set. Scenes display as static completion frames; advance via Step or wait 4 seconds."
3.3 The current scene shows its t = 1 (completion frame) — no animation. After 4 seconds it auto-advances to the next scene.
3.4 Press Step (▸▸) — the timer cancels + the next scene appears immediately. Auto-advance resumes from the new scene.
3.5 Disable reduce-motion. Reload. The banner disappears + normal animated playback resumes.
4. WebGL-2 fallback visual (~1 min).
4.1 In Chrome: open chrome://flags, set #disable-webgl2 to Disabled, restart. (Or use a browser without WebGL 2 — older Safari, headless without GPU.)
4.2 Reload the viz page. Banner reads "3D camera moves are unavailable; the visualization is rendering in 2D mode."
4.3 Confirm the viz still renders normally — every scene plays through (the M13 viz is SVG-only by design, so the WebGL gate is informational).
4.4 Re-enable WebGL 2 + reload. Banner disappears.
5. Share-link replay verification (~3 min).
5.1 Submit a prompt + let the run complete.
5.2 In the thread header, toggle Share on. Copy the /share/{token}/runs/{run}/replay URL.
5.3 Open the URL in a private/incognito window (verifies no-auth access).
5.4 Confirm: viz mounts, PlaybackControls visible, chat bubble shows the completed text, scene runner plays through 0→20 the same way.
5.5 Test click-to-inspect: click any VectorStrip in the public replay → NumericalValuesPanel opens with the same data the owner saw.
5.6 Test PipelineProgressBar click + Step + Speed change — all work in the public view.
6. Cross-browser smoke (~10 min, defer to M15). The matrix below extends the M12 chunk-9 cross-browser table with M13-specific rows. M15 launch prep owns the first full run; this recipe is the M13-side of the contract.
| Browser | M13-specific check | Expected |
|---|---|---|
| Chrome latest | Scenes 0-20 each render | Every scene caption + content visible |
| Chrome latest | PipelineProgressBar click-to-jump | Each segment jumps to its scene cleanly |
| Chrome latest | FPS in band on a fresh tab | ≥ 30 FPS sustained at 1× speed |
| Firefox latest | Same three checks | Same |
| Edge latest | Same three checks | Same |
| Safari latest | Same three checks | Same |
| Chrome -1 | Same three checks | Same |
| Firefox -1 | Same three checks | Same |
| Edge -1 | Same three checks | Same |
| Safari -1 | Same three checks | Same |
(Empty until M15 populates from a real cross-browser run. Per the M12 chunk-9 pattern, non-blocking issues are documented as "won't fix" with a rationale.)
What worked
- Rip-and-replace from chunk 1 was the right call. Deleting the M8 viz files up front (
VizPane.tsx,EmbeddingScene.tsx,CascadeController.ts,ParticleSystem.ts,TransformerStack.ts, etc.) kept chunks 2-14 from coding around dead weight. The git history preserves M8 for spelunking;phase1.mdkeeps the documentation. Zero feature-flag plumbing, zero parallel mounts, zero "what does this old code do?" tax across 14 chunks. - SVG-only primitive set (chunk 2's locked-in decision). "All 5 primitives are SVG/HTML/CSS — no Three.js" turned out to be the single largest scope-reducing call of M13. The chunk-13 WebGL gate became informational only (no actual rendering pipeline to switch), the chunk-12 degraded-mode opt-ins were cheap (clamp visibleCells, strip a CSS animation, slice the head matrix), and no scene needed a separate Three.js fallback path. The chunk-2 pre-discussion's hedge on
<ParticleTrail>being Three.js + the SVG-with-CSS-animation winning meant every named scene indocs/visualization.mdwas covered by SVG out of the gate. - The
PipelineStatewide-record + per-scenetransform()contract (chunk 3a) scaled to 21 scenes cleanly. Each scene reads what it needs, adds its derived field, returns. Idempotent transforms made scrubbing/replay trivial. The chunk-3a alternative — a typedScene<I, O>chain with verbose generics — would have required 21 type mappings + would have made cross-scene fallbacks (the "if X missing, use Y" fallback chains in chunks 5-9) impossible without a per-scene migration story. - Pure-function libs paid off in test density + reuse.
lib/towerCamera.ts(chunk 7),lib/syntheticEmbedding.ts(chunk 4),lib/syntheticAttention.ts(chunk 5),lib/syntheticLogits.ts(chunk 8),lib/syntheticAutoregression.ts(chunk 9),lib/syntheticFFN.ts(chunk 6),hooks/useFpsTracker.ts(chunk 12) — each carries the math/state-machine logic separately from the React render layer. Tests for those libs run without jsdom + mounting. ~150 of M13's ~930 Vitest assertions are pure-function lib tests. - React context for cross-cutting concerns (chunks 11b + 12).
VectorInspectionContextmade every existing VectorStrip click-to-inspectable without touching any scene.PerformanceModeContextmade every VectorStrip + ParticleTrail + AttentionScene FPS-aware without touching any scene. Both contexts return safe defaults outside their providers, so per-component tests don't need the wrapper. The pattern beats threading props through 21 scenes by a wide margin. - Chunk-split decisions stayed flexible. Chunks 3, 8, 9, 10, 11 split into a/b sub-chunks when the work was natural to bisect at a state boundary (e.g., 8a at the logits computation, 9a at the loop/explainer line, 11a at the inspection-UI boundary). Chunks 4, 5, 6, 7, 12, 13 stayed single-commit when the work was one coherent feature. The split came up at the design-question step of each chunk; the user-direction-driven default usually held.
- The chunk-1 placeholder PipelineProgressBar paid back through every subsequent chunk. Building the 21-segment control surface in chunk 1 (when nothing about scene content was decided) meant chunks 3-9 just toggled the
aria-currentsegment as the runner advanced. Click-to-jump-to-scene was already wired in chunk 3a'scontrols.setScene. The alternative — stubbing the progress bar and building it after content settled — would have required a refactor at chunk 10 or 11 once playback controls landed. - The decisions block per chunk became a real reference. ~150 documented decisions across M13. When chunk 14's integration test needed to know "how does the chat bubble decide which scene to render?" the chunk-10b decisions block answered it without re-reading the source. Same for the chunk-9a "linear scenes, not overlays" choice (referenced in chunk-13 when planning the gate behavior) and the chunk-5 "defer runId" choice (referenced in chunk-10b's events-vs-state selector).
What we learned
vi.doMock+ dynamic import requiresvi.resetModules()inbeforeEach. Hit during chunk-13 gate tests: the first test in eachvi.doMock-guarded suite inherited the already-loaded CinematicViz module from before the mock applied, returning the default hook value.vi.resetModules()before thevi.doMockinvalidates the cache so the nextawait import('@/...')is fresh. Documented inline; the pattern works for any per-suite hook override.useCallback-stable context values prevent infinite-render footguns. Caught in chunk 11b:useEffect([inspection], () => inspection.open(...))re-fired each timesetActiverecreated the context value (which happened on everyopen()call). Fix isuseCallbackonopen+closewith empty deps so the consumer can depend on them directly. Same pattern applied in chunk-12'sPerformanceModeContext.- ESLint's
react-hooks/set-state-in-effectrule fires on legitimate URL-param reads. Chunk-12's first draft of FpsCounter useduseState+useEffectto readwindow.location.search. The rule (correctly) said no. Fix isuseSyncExternalStorewith asubscribe/getSnapshotpair — the idiomatic React 18+ way to bridge an external source. Reusable for any future query-param consumer. - Hooks-rule violations slip past tests but fail the prettier+eslint pre-commit. Initial drafts of
SoftmaxScene(chunk 8b) calleduseMemoinside theScene.rendercallback, which is a hooks-rule violation. Tests passed locally but the pre-commit hook caught it. Pattern from chunk 5 onwards: scenerender: (t, state) => <SceneComponent t={t} state={state} />withuseMemoinside the inner React function component. Documented in chunk-8b decisions. - Per-scene component-test files vs unified scene-determinism.test.ts. Chunks 3-9 each shipped their scenes' own test files (
scenes-3-4.test.tsx,scenes-5-7.test.tsx, etc.) covering scene-specific UI invariants. The chunk-14scene-determinism.test.tsis the unified walk asserting determinism + output-shape for ALL 21 scenes. Both layers carry weight: per-scene files catch behavior regressions, the unified file catches cross-scene contract drift. - Stale vitest worker accumulation during debugging. When debugging chunk-11b's infinite-render loop, ~5 stalled vitest processes piled up (spawn-per-invocation with workers). Each one consumed 100% CPU pinning the next foreground vitest. Workflow lesson:
pkill -9 -f vitestbetween investigations when test runs hang past their normal duration. - Inline duplication beats coupling for crossing-concern math. Chunk-7's tower camera math is in
lib/towerCamera.ts; chunk-10a's persistent HUDcurrentLayerderivation inlines the same 5-phase math inCinematicVizrather than importing. Reason: the persistent HUD shouldn't depend on a scene-internal lib for behavior parity. The 10-line duplication is its own contract; both copies have their own tests; either can evolve independently. Trade-off documented in chunk-10a decisions. - Spec gaps surface during integration, not design. Chunk 10's spec line said "Driven directly by the real
token.receivedWebSocket event" — the design question of how to handle the no-events case (replays, isolated tests) only surfaced at implementation. The events-primary / state-fallback selector emerged then; documented as a chunk-10b decision. Pattern: lean toward fallback-to-something-sensible at every spec-vs-reality gap rather than blocking the chunk on a spec amendment. - Linter reformats break in-flight Edit handles. Same lesson from M10/M11/M12. Hit in M13 chunks 4, 8b, 9b, 11b, 13 (every commit that goes through prettier/eslint pre-commit). Pattern unchanged: re-Read after the hook runs, then re-Edit. Worth automating the read-after-reformat into the workflow.
Parked / carry-forward to M14/M15
- Real WebSocket-driven scene pacing. Chunk 9a's
synthesizeAutoregressiveLoopis a canned 7-iteration timeline. Real runs vary in token count + per-token time. Chunk 10b wires the chat bubble to live events; chunk 11a's jump-to-live derivesliveSceneIndexfrom events. But the scene timing inside Scene 18 still uses the synthetic durations. M14/M15 polish chunk should drive Scene 18's iteration timing from the actualtoken.receivedt_msfield. runIdin the deterministic-synth seed. Chunks 4, 5, 6, 7 use(tokenIndex, layerIndex=0)for synthesis; the spec calls for(run.id, layer_index, token_index). PlumbingrunIdfrom CinematicViz → PipelineState → every synth helper is cross-cutting; deferred to a chunk-10-style wiring pass. Replay determinism across the same prompt holds today; replay determinism across different runs of the same prompt doesn't yet.vocab_sizeDB column. Chunk 8a plumbed the type but pages pass null becauserunsdoesn't have avocab_sizecolumn. Scene 14 falls back to 128k. A future migration adds the column + the page-source switches over without code changes to CinematicViz.- Off-canvas reverse-lookup highlight uses string-match, not vocab-index-match. Chunk-10a's Scene 20 reverse-lookup highlights sidebar rows whose token string matches the generated string. Real runs would use vocab-index matching once
runIdplumbing lands. The string match is fine for the chunk-9b synthetic tokens but loses precision when two different vocab entries share a string. - Persistent
LayerCounterHudvs Scene 12's in-canvas counter. Two surfaces, two sources of truth (chunk 10a's HUD inlines the chunk-7 math; the in-canvas counter uses the lib). Drift risk if the math changes in either place without the other. M14/M15 polish could extract a shareduseTowerCounter(state, sceneIndex)hook that both surfaces consume. - GIF export needs to capture the new viz. Per the parked-items list above. Chunk 2's SVG renderer composes scene-by-scene; chunk-6's M10 Puppeteer skeleton would also need to record the new viz once enabled. Lands as a polish chunk or M14/M15 if required for launch.
- Three-commits-per-chunk pattern (3a/3b/3c) didn't recur after chunk 3. Most chunks split into 2 (a/b) when split at all. Pattern: 3 sub-commits is the sweet spot only when scenes have natural infra/scene-content/integration steps that each individually compile. Most chunks fit cleanly into 2 sub-commits at most.
- Scene-overlap forfeited in chunk 9. Spec described Scenes 19+20 as overlaid on Scene 18; the runner is strict-sequential, so they're implemented as linear scenes. Closeout note: a future Scene contract extension (
overlay: boolean) could land them as true overlays. Today's pedagogical beats land in the correct order; only the visual overlap is missing. - Chunk-12 degraded-mode "restoration condition" not yet observable in production. The
< 18 FPS for 2s → degrade; > 24 FPS for 5s → restorestate machine is testable via mocked RAF (chunk-12 tests). The actual production trigger requires a real low-end device + a real GPU starvation scenario. M15 launch prep should include a "drive the laptop to thermal throttling and confirm degrade-then-restore fires correctly" smoke step. - Cross-browser real-browser smoke. The matrix above is empty by design — M15 launch prep owns the first full run. Per the M12 chunk-9 cross-browser table pattern, non-blocking issues are documented as "won't fix" with rationale. Real-browser smoke is the only verification path for the runtime behaviors jsdom can't model: real focus rings, real
prefers-reduced-motion, real GPU.
Exit criteria
- The 20-scene narrative runs end-to-end on a fresh thread submit with a real OpenAI run, hitting all milestones from prompt-entry through end-of-sequence flourish. Covered by
pipeline-integration.test.tsx(mock event stream + ChatBubble accumulation invariants) + per-scene Vitest inscenes-*.test.tsxfiles (87 determinism assertions inscene-determinism.test.tswalk the full registry). Manual recipe step 1 is the human verification path. - The real run's tokens stream into the chat-bubble overlay independent of the visualization — covered by chunk-10b's events-primary path + the integration-test "chat bubble accumulates ALL streamed tokens regardless of scene position" assertion. Manual recipe step 5.4-5.5.
- 20-30 FPS sustained on a 2020-class machine; automatic degrade triggers cleanly. Chunk-12's
useFpsTrackerstate machine (8 Vitest assertions with mocked RAF) + chunk-12 degraded-mode opt-ins (10 assertions across VectorStrip / ParticleTrail / AttentionScene). Manual recipe step 2 is the human verification path; the "drive the laptop to thermal throttling" smoke step is parked for M15. - Reduced-motion mode shows every scene as a static frame the user can step through. Chunk-13
t = 1pin + 4s auto-advance + Step override. Tests inCinematicViz-gates.test.tsx. Manual recipe step 3. - WebGL 2 unavailable mode shows the SVG fallback; both modes still produce all 20 scenes (just rendered differently). Chunk-13 informational gate (the SVG fallback is the default since chunk 2). Manual recipe step 4.
- The persistent UI sections (vocab sidebar, chat bubble, layer counter, pipeline progress) are populated and click-targets work. Chunk-10a (LayerCounterHud + VocabSidebar reverse-lookup) + chunk-10b (ChatBubble events-driven). Click-targets covered by chunk-3a's PipelineProgressBar
setScenewiring + chunk-11a's PlaybackControls. - The
/share/{token}/runs/{run}/replaypublic route shows the same visualization as the owner view. Pages mount<CinematicViz />identically in Threads/Show, Runs/Replay, Share/Replay (chunk 1 + chunk 10b events plumbing). Manual recipe step 5. - M8 components retired:
VizPane.tsx,EmbeddingScene.tsx, related controllers + tests removed from the codebase. Chunk 1 deletion-first commit (a0bdf6b). Git history preserves the M8 implementation;phase1.mdkeeps the documentation.
Purpose: Application deploys to DreamHost VPS reliably and is observable.
Tasks
- Provision a single DreamHost VPS that hosts both production and staging.
- Install: PHP 8.4, Composer, Node 20, SQLite, supervisor, nginx, certbot, ffmpeg.
- (Optional) install Chromium if
GIF_RENDERER=puppeteer. - nginx vhost config: two server blocks on the same VPS.
llm.trackr.live— production. Document root:/var/www/llm-viz/current/public.staging.llm.trackr.live— staging. Document root:/var/www/llm-viz-staging/current/public.- Both proxy WebSocket upgrades to the local Reverb instance on their own ports (e.g., 8080 prod, 8081 staging).
- HTTPS via Let's Encrypt direct to the VPS (no Cloudflare in front). HSTS header enabled. Auto-renewal via certbot's systemd timer.
- supervisor configs (with separate program names for prod and staging so they can be restarted independently):
llm-viz-queue/llm-viz-staging-queue— queue workers.llm-viz-reverb/llm-viz-staging-reverb—php artisan reverb:starton different ports.- (Optional) Puppeteer is spawn-per-export — no supervisor entry needed.
- Two isolated SQLite databases (
/var/www/llm-viz/database/database.sqliteand likewise for staging) so staging data can't pollute production. - Deploy script:
./deploy.sh {env}—git pull,composer install --no-dev,npm ci && npm run build,php artisan migrate --force,php artisan optimize,supervisorctl restart llm-viz-{env}-*. - GitHub Actions deploy workflow: on push to
main→ SSH and run./deploy.sh staging. Onv*tag → run./deploy.sh production. - SQLite backup cron: nightly
.dumpof production only to off-VPS storage (e.g., S3 or Backblaze). - Sentry DSN configured in both production and staging
.envwith distinctSENTRY_ENVIRONMENTvalues. - Smoke test from production deploy: sign in, submit a run, watch viz. Document the procedure in
docs/deployment.md. - Resource guardrails documented: at expected launch traffic the VPS handles prod + staging concurrently, but a load spike on staging could affect prod (shared CPU/RAM). Note in
docs/deployment.mdto throttle staging during load tests.
Exit criteria
- A push to
maindeploys tostaging.llm.trackr.livewithin 5 minutes. - A
v0.9.0tag deploys tollm.trackr.live. - Both domains return 200 over HTTPS, certificates valid, HSTS header present.
- Sentry receives a deliberately-thrown test error from production with
environment=production. - Backup verified by restoring production
database.sqlitefrom latest backup into a scratch directory. - Restarting staging supervisor processes does not interrupt production WebSocket connections.
Purpose: Run the full acceptance criteria checklist, fix anything that fails, write user-facing documentation, and seed the database for first users. No invite-only beta — launch goes straight to public access at v1.0.
Tasks
- Acceptance criteria walkthrough (all 18 items from SPEC §9), each ticked off with evidence (screenshot/recording/test name).
- Performance: load-test with k6 or Artillery → 100 concurrent simulated users hitting the streaming pipeline. Run from staging to avoid blowing up production during the test.
- Security audit:
- Verify API keys are encrypted at rest (look at raw SQLite).
- Test share-token enumeration (60/min IP rate limit holds).
- CSRF on all state-changing routes.
- Composer + npm audit clean.
- AGPL compliance review: ensure every page footer and the API responses include a link to the source repository (AGPL §13 — interaction-with-source obligation).
- User guide (
docs/user-guide.md): screenshots, walkthroughs. - Admin guide (
docs/admin-guide.md): registry refresh, user promotion, rate limit adjustment, backup restore. -
CHANGELOG.mdinitialized withv1.0.0entry. - Registry seeded with the launch set (9+ models per SPEC §7).
- Promote one real admin user via
user:promote. - Soft-launch checklist (since public from day one, no waitlist):
- Verify rate-limit defaults (30/hour) are reasonable for first-day curiosity traffic.
- Confirm Sentry alerts route to a real notification channel (email or Slack webhook).
- Pre-write a "we hit a snag" status page or pinned issue template.
- Have an admin promotion ready in case an early user needs higher limits.
- Tag
v1.0.0and deploy to production. - Announce (forum/social/HN, at the operator's discretion).
Exit criteria
- All 18 SPEC §9 criteria pass.
llm.trackr.liveis live, returning 200, and a fresh user can complete a sign-in → run → replay loop end-to-end.- AGPL §13 source-link is visible on every page served to a user.
- Test coverage: Each milestone must leave overall backend coverage ≥ 70%. Failing this blocks the milestone from being marked done.
- Documentation: When a milestone introduces a new concept (e.g., M3 introduces registry refresh),
docs/architecture.mdordocs/admin-guide.mdis updated in the same PR. - Security review: Any PR touching auth, encryption, share-link routes, or admin endpoints requires a second-pass review before merge.
- Performance budget: Every PR that touches the viz canvas runs a Chrome DevTools performance trace as part of review.