architecture.md

Sketchi Architecture

Sketchi v2 should make diagram generation boring in the best way: inputs are validated, rendering is deterministic, and UI states are exercised outside the app shell.

At A Glance

Area	Source of truth	Proof
Diagram contract	packages/diagram/core	core tests and fixtures
Deterministic scene	packages/diagram/renderer	renderer tests and stories
Excalidraw output	packages/diagram/excalidraw	real-scene validation
Agentic generation	packages/diagram/generation, then diagram-agent	fixture/eval tests
Product surfaces	apps/*	Nx app builds and Worker deploys

flowchart LR
  Prompt["Prompt or revision"] --> Generation["diagram-generation<br/>model request + candidate"]
  Generation --> Core["diagram-core<br/>typed IR validation"]
  Core --> Renderer["diagram-renderer<br/>scene model"]
  Renderer --> Excalidraw["diagram-excalidraw<br/>persistable scene"]
  Excalidraw --> UI["apps + diagram-ui"]

Loading

Package Boundaries

• diagram-core owns the intermediate representation, semantic validation, and reusable fixtures.
• diagram-renderer converts validated diagrams into a deterministic scene model.
• diagram-excalidraw converts deterministic scenes into persisted Excalidraw elements and validates real-scene invariants.
• diagram-scenarios owns maintained prompts, assertions, fixture evaluation, and local command-provider runs.
• diagram-generation owns provider request/response mapping and candidate parsing.
• diagram-agent owns the canonical buildFlowchart request/result vertical, structured issues, quality assessment, render/export orchestration, and accepted artifact persistence used by Studio, HTTP, and MCP.
• diagram-ui renders the scene model and owns user-facing component states.
• studio-projects owns Studio project/diagram contracts, browser API helpers, anonymous and authenticated ownership, and object-bucket persistence. It has no dependency on app routes, Cloudflare bindings, or the Code Mode runtime.
• apps/eval-harness composes the packages in a TanStack Start internal eval harness for scenarios, candidate inspection, and regression review.
• apps/playground owns the public Playground worker boundary: ephemeral chat generation over the canonical build runtime and the HTTP/browser routes for the persisted Studio foundation. A thin server adapter translates the R2 binding and validated Code Mode source artifacts into studio-projects; the app never performs a second accepted-artifact write.
• apps/web owns the public home/docs surface and should stay free of diagram runtime dependencies unless docs become interactive.
• apps/excalidraw owns the internal Excalidraw rendering workspace and composes the diagram packages into a real canvas for isolated validation.
• apps/icons owns icons.sketchi.app and serves the copied pre-cleaned sketchi-icons/output tree from its app-local public assets.

See Agentic Generation for the intended Convex, Cloudflare Worker, MCP, AI SDK, Effect, and Nx boundaries.

Workspace Enforcement

The repository has one truthful project inventory:

• pnpm discovers package-backed projects through apps/*, packages/*, packages/*/*, and tools/*;
• Nx discovers the same package roots plus the generated-only native-conversion-storybook composition project;
• the root TypeScript solution references every composite app, buildable package, and generator library. It deliberately omits the Storybook-only composition root because that surface has no composite TypeScript project.

Every Nx project has a scope:* tag and an explicit type:* role. Packages cannot import apps, normal apps cannot import apps, runtime packages cannot import eval or app code, and persistence cannot import app, eval, or UI code. native-conversion-storybook is the sole scope:composition surface and the only project allowed to declare app dependencies. Buildable libraries may only depend on other buildable libraries.

The flat ESLint configuration intentionally enforces architecture rather than introducing a repository-wide style rewrite. Nx infers a lint target for all projects and applies the dependency rules to project source, configuration, test, and Storybook files. tools/project-graph.test.ts compares all three inventories to the explicit intended roots, so deleting a manifest, composite setting, or root reference cannot shrink both sides of the check. It also fails if tags, lint targets, boundary coverage, or app dependency ownership drift. Required CI builds all projects and then dry-runs Wrangler for every project in the canonical Worker map, using the generated per-app deployment configs.

Diagram Pipeline

1. The remote thread planner chooses a generated tool such as generateDiagram, restructureDiagram, or tweakDiagram.
2. Generation produces or edits a typed intermediate diagram. The first hard target is the flowchart contract.
3. diagram-core validates shape, references, and diagram-type invariants.
4. diagram-renderer converts the diagram into deterministic scene primitives.
5. diagram-excalidraw converts the scene into Excalidraw elements and validates real output constraints such as arrow bindings and bound text fit.
6. diagram-scenarios runs maintained assertions against fixture or one-shot model output.
7. UI packages display those primitives in a real Excalidraw canvas and expose stateful review workflows.

This makes defects local: invalid references fail in core tests, scene drift fails in renderer tests, Excalidraw syntax issues fail before persistence, visual UI states fail in Storybook/component tests, and app routing/deploy failures stay in the app boundary.

Flowchart First

The first high-reliability path is decision-heavy flowchart generation. The LLM should produce typed flowchart IR: start/process/decision/end nodes, labeled decision branches, and explicit edges. Code owns layout, real Excalidraw conversion, arrow bindings, and text wrapping. The canonical evaluation fixture is the pharma batch disposition flow.

Agentic Generation

The durable behavior is the generation runtime, not one chat surface. Keep the runtime importable by Convex actions, Cloudflare Workers, MCP routes, app routes, and local evals.

flowchart TB
  Convex["Convex<br/>managed threads"] --> Runtime["shared generation runtime"]
  Worker["Worker<br/>HTTP + MCP + AI Gateway"] --> Runtime
  CLI["CLI/evals"] --> Runtime
  Runtime --> IR["validated IR"]
  Runtime --> Artifact["rendered artifact"]

Loading

AI SDK remains a thin adapter for model invocation, streaming, and tool-call plumbing. Effect can live inside the shared generation packages for schemas, typed errors, and pipeline composition. Nx still owns the project graph, builds, and affected checks.

Eval Harness

The apps/eval-harness surface is not the public product Playground and does not include the Convex remote agent loop. It is an independently deployable internal testing ground for maintained scenarios, pasted model output, and deterministic IR-to-Excalidraw conversion.

The public Sketchi Playground direction is anonymous prompt-to-diagram generation with artifact handoff. That experience is implemented by apps/playground on the durable sketchi-studio Worker while authenticated Studio remains a separate future cutover.

The portable CLI path is:

pnpm nx scenario diagram-scenarios -- --scenario pharma-batch-disposition --fixture --out .memory/pharma-batch.excalidraw

For one-shot model runs through Nx, pass a local command in SKETCHI_GENERATOR_COMMAND. The command reads the scenario prompt from stdin and writes candidate IR JSON to stdout:

SKETCHI_GENERATOR_COMMAND="your-llm-command" pnpm nx scenario diagram-scenarios -- --scenario pharma-batch-disposition

The CLI also supports direct --generator-command usage when it is invoked outside Nx; put that flag last because the scenario CLI treats the rest of the argv as the provider command.

This keeps local/OpenCode-style integrations separate from Convex auth, thread orchestration, and persistence.

App Surfaces

The v2 workspace has five TanStack Start app surfaces:

• eval-harness: internal scenario evaluation and prompt-output inspection.
• playground: public Playground worker boundary with chat generation, artifact handoff, and the Cloudflare/HTTP adapter for the persisted Studio foundation.
• web: public home/docs for the Sketchi product direction.
• excalidraw: internal Excalidraw rendering workspace; Excalidraw is an implementation/editor capability, not a standalone public product route.
• icons: a standalone browser for the curated Sketchi icon outputs.

flowchart LR
  Workspace["Nx workspace"] --> EvalHarness["eval-harness"]
  Workspace --> Playground["playground"]
  Workspace --> Web["web"]
  Workspace --> Excalidraw["excalidraw"]
  Workspace --> Icons["icons"]

  EvalHarness --> Deploy["preview/prod Worker matrix"]
  Playground --> Deploy
  Web --> Deploy
  Excalidraw --> Deploy
  Icons --> Deploy

Loading

Keep app-specific UI in the app that owns it. Generate those components with @sketchi/generators:ui-component --projectRoot=apps/<app> so stories and tests stay close to the route shell without adding shared package dependencies.

Generator Contract

New UI components should be created with @sketchi/generators:ui-component. The generator expands EJS templates into a component folder with implementation, Vitest coverage, Storybook story, barrel export, and package export wiring.

New diagram types should be created with @sketchi/generators:diagram-type. The generator updates the core diagram type registry and expands EJS templates for a typed fixture, core contract test, renderer contract test, and Storybook story. This keeps each diagram type previewable and testable before it is connected to generation.

The registry is guarded by tests: every registered diagram type must have core, renderer, and Storybook coverage, and every reusable Studio component must have an implementation, test, story, local export, and package export.

Deployment Direction

Each app is scaffolded for Cloudflare Workers through Vite and Wrangler. The adopted production route map is:

• sketchi.app and www.sketchi.app -> web
• playground.sketchi.app -> playground, which owns the public Playground boundary
• icons.sketchi.app -> icons
• the eval harness and standalone Excalidraw workspace remain internal
• authenticated Studio remains deferred until it is deliberately launched

The manual domain attach target for playground.sketchi.app is the playground project. Its durable Cloudflare service remains sketchi-studio; authenticated Studio remains unexposed until deliberately launched. Custom-domain attachment is a post-merge operator action described in Production domain cutover. Until that cutover, use stable workers.dev URLs for deployment and runtime proof.

Preview deploys strip production routes and deploy app-specific Workers using the durable preview prefix declared for each app.

Today scripts/lib/worker-apps.mjs maps each explicit deploy project ID to its durable Worker name, input Wrangler config, and isolated dist/apps/<app> output for eval-harness, playground, web, excalidraw, and icons. eval-harness intentionally maps to the existing sketchi-playground Worker; workflows never derive Worker identities from Nx names. Custom-domain attachment remains an explicit manual workflow dispatch.

AI Gateway Observability

The live playground uses the Cloudflare AI Gateway Worker binding with collectLog: true and scenario metadata on each request. Cloudflare stores Gateway logs and can retain request/response payloads for prompt tuning when the gateway is configured to keep payloads.

Inspect and summarize logs through the Cloudflare API MCP instead of keeping repo-local Cloudflare API scripts. Configure Codex or another MCP client with Cloudflare's Code Mode server:

{
  "mcpServers": {
    "cloudflare-api": {
      "url": "https://mcp.cloudflare.com/mcp"
    }
  }
}

Use the repo-local $sketchi-log-analysis skill in .agents/skills/sketchi-log-analysis/SKILL.md for reusable analysis. The skill keeps the operational behavior read-only, asks for payload inspection only when retained by the Gateway, and turns model failures into scenario or prompt-tuning follow-up.

Code Mode Usage Capture

Studio records a server-side usage trail for the harness-facing Code Mode surface. The /api/v1/flowcharts/build, /api/v1/artifacts/:artifactId/patch, and MCP execute paths create a sketchi.codemode.usage.v1 event. Capture is scheduled with Cloudflare Workers waitUntil so the response is not blocked on telemetry storage. Capture is best-effort and must not fail artifact generation when telemetry storage is unavailable.

The raw usage event stores operation, surface, run/attempt/event IDs, client headers such as harness/model/reasoning level when supplied, status, duration, artifact refs, and bounded request and response JSON snapshots when they are serializable and below the current size cap.

Events are date-partitioned under the existing artifact bucket:

codemode/usage/YYYY/MM/DD/<run_id>/<attempt_id>/<event_id>/event.json

Studio also sends normalized rows to Cloudflare Pipeline streams. The CODEMODE_USAGE_EVENTS binding writes one flat row per Code Mode request and the CODEMODE_USAGE_ISSUES binding writes one row per structured issue:

Surface	Binding	Stream ID
Preview	CODEMODE_USAGE_EVENTS	e9fc3bcd35314fa39fc6a89018207acc
Preview	CODEMODE_USAGE_ISSUES	d95a1767edf246af8c637c5b9bf5a5c5
Production	CODEMODE_USAGE_EVENTS	d9044253316f4273a60298098f444a62
Production	CODEMODE_USAGE_ISSUES	f687dab6e7d742c1a76834089e709462

The committed stream schemas live in scripts/pipelines/. Keep Pipeline rows flat and aggregate-friendly; keep the richer request/response snapshots in the raw R2 event objects.

R2 Data Catalog and R2 SQL are the downstream aggregate-query path. The durable catalog topology is recorded in scripts/pipelines/codemode-r2-catalog.mjs:

Surface	Stream kind	Catalog bucket	Table
Preview	Events	sketchi-codemode-usage-analytics-preview-v4	sketchi_codemode.usage_events
Preview	Issues	sketchi-codemode-usage-analytics-preview-v4	sketchi_codemode.usage_issues
Production	Events	sketchi-codemode-usage-analytics-production-v4	sketchi_codemode.usage_events
Production	Issues	sketchi-codemode-usage-analytics-production-v4	sketchi_codemode.usage_issues

Keep the credential boundary explicit: a Wrangler OAuth token can list catalog metadata, but Data Catalog sinks and R2 SQL data scans require an R2 API token with Admin Read & Write permissions. The 2026-06-29 smoke run passed by querying the exact Pipeline-ingested row after the table metadata appeared, and the real production/preview Code Mode streams were then verified by querying both usage_events and usage_issues rows for live API run IDs through the v4 catalog buckets. Use pnpm r2sql:codemode:resources to print the durable resource map and pnpm r2sql:codemode:verify-run -- --production-run-id <id> --preview-run-id <id> to query R2 SQL for a real end-to-end run pair. The verifier polls R2 SQL because catalog ingestion is asynchronous. Event rows are required; issue rows are queried and reported, but they are only required when the probe is expected to emit issues and the command includes --require-issues or explicit --production-issue-run-id / --preview-issue-run-id values.