Brainstorm perf improvements for running c# emitter in playground

[jorgerangel-msft]

Speed up the TypeSpec playground regenerate-loop (caching + emitter optimizations)

Introduction

The TypeSpec playground now ships a @typespec/http-client-csharp emitter, letting users see the .NET code their spec would generate (live link). It is a great step forward, but the iterate-on-spec loop is slow.

Today, every regenerate — even after a one-character edit, an undo, or a page reload — pays the full pipeline cost end-to-end:

1. The browser recompiles the TypeSpec source via the playground's emitter (emitter/src/emit-generate.browser.ts).
2. The serialized code model is POSTed to a hosted ASP.NET service (csharp-playground-server.azurewebsites.net/generate).
3. That service spawns a fresh dotnet subprocess running Microsoft.TypeSpec.Generator.dll against a temp directory (playground-server/Program.cs).
4. The CLR boots, MEF discovers and loads plugins, Roslyn warms up its AdhocWorkspace, the full pipeline runs, files are emitted, and the process exits — discarding everything it built.
5. The response is sent back over HTTP and rendered in the playground.

There is no caching anywhere in this pipeline and no fast path for "input unchanged". Identical or near-identical requests — undo/redo, share-link replays, whitespace-only TypeSpec edits, copy-paste, demos hit by multiple users — repeat the entire loop from scratch. Users notice the lag, and the experience falls short of the "see your code immediately" promise the playground sets up.

This proposal is a pragmatic, playground-only speedup. It deliberately avoids touching the generator's internals (Roslyn post-processing, type emission, re-entrancy semantics) so that the production codegen path used by every Azure SDK and CI pipeline is untouched and bit-identical.

Constraints

• Production codegen path (Node.js emitter/src/emit-generate.ts) must remain bit-identical and untouched.
• No new external services or shared databases. Browser IndexedDB and container-local memory/disk only.
• Output must remain production-fidelity (Roslyn post-processing not skippable).

Out of scope (explicit non-goals)

• Persistent in-process generator worker (AssemblyLoadContext-based host that loads the generator DLL once and serves many requests). Forces the generator to be safely re-entrant; permanent audit tax on every contributor for every static, every singleton, every plugin static field. Not justified by playground-only UX gain.
• Incremental TypeProvider memoization (caching per-type generated text keyed on input hashes). Cross-cutting dependencies — referenced types' names/namespaces/accessibility, polymorphism, model factory aggregation, serialization decisions — make precise invalidation either too conservative (low real-world hit rate on tightly-coupled libraries like Azure.AI.Agents.Persistent) or too clever (silent cache-poisoning bug class). Permanent test surface for "every random edit must produce identical output to a clean run".
• Skipping Roslyn post-processing in playground mode. Output must match production.
• Any change to generator semantics or type emission.

Proposed work — single phase, four independent items

Each item is shippable on its own. Together they cover the most common iterative-edit patterns without touching generator internals.

Item 1 — Browser-side IndexedDB response cache

Hash (generatorName, codeModelJSON, configurationJSON, generatorVersion) with SHA-256, look up in IndexedDB before issuing the HTTP request. Hit returns the cached GenerateResponse instantly without a network round-trip. Helps undo/redo, share-link replays, page reloads, tutorial demos.

• New file: emitter/src/playground-cache.browser.ts (small native-IndexedDB wrapper, no new deps).
• Modify emitter/src/emit-generate.browser.ts: cache lookup before fetch, store after.
• LRU 50 entries / 64 MB; eviction by lastAccessed timestamp.
• Cache key includes server-reported generatorVersion so deploys auto-invalidate.

Item 2 — Emitter-side no-op suppression (in-memory)

In the same browser session, keep the last-sent (codeModelJSON, configurationJSON) in a closure. If the next regenerate produces identical bytes, short-circuit before fetch and reuse the previous response. Catches the very common "edit → undo → regenerate" cycle plus TypeSpec whitespace/comment-only edits that don't change the emitted code model.

• Modify emitter/src/emit-generate.browser.ts: one closure-scoped last = { hash, response } cell at module scope.
• Effort: ~1 hour. Complementary to Item 1 (Item 2 is in-memory and instant; Item 1 persists across reloads).

Item 3 — Server-side response cache (container-local)

Same hash key as Item 1, server-side. Helps cross-session replays — multiple users hitting the same demo spec, popular tutorial pages, retry storms.

• Tier 1: IMemoryCache (Microsoft.Extensions.Caching.Memory, transitively available in ASP.NET Core), capped at 256 MB.
• Tier 2: content-addressed file store under /var/cache/tsp-playground/{hash[0..1]}/{hash}.json, 2 GB cap, LRU by mtime. Plain files; no DB schema.
• Hash includes the running generator's assembly file version → deploys auto-invalidate.
• New header X-Cache: HIT|MISS for observability.
• New file: playground-server/GenerationCache.cs.
• Modify playground-server/Program.cs: wrap the /generate handler around IGenerationCache.GetOrAdd.
• Modify playground-server/Dockerfile: declare VOLUME /var/cache/tsp-playground so the cache survives restarts when the host provides persistent storage (and is harmless when it doesn't).

Item 4 — Emitter payload trim

Reduce time the browser spends preparing the request body and the bytes on the wire.

• In emitter/src/code-model-writer.ts: drop the prettierOutput re-parse pass for the browser build target; plain JSON.stringify without indent.
• In emitter/src/emit-generate.browser.ts: Content-Encoding: gzip on the request body.
• Verify gzip handling on the Kestrel side in playground-server/Program.cs.

Optional Item 5 — Per-stage timings in GenerateResponse

Observability that lets us decide whether any further optimization is warranted.

• Modify playground-server/Program.cs to capture LoggingHelpers.LogElapsedTime lines from the generator's stdout and forward them in a new timings field on GenerateResponse.
• Surface in the playground UI (or response devtools view) for power users.
• ~½ day. The generator already emits these markers in generator/Microsoft.TypeSpec.Generator/src/CSharpGen.cs and generator/Microsoft.TypeSpec.Generator/src/PostProcessing/GeneratedCodeWorkspace.cs.

Verification

1. Item 1: integration test with mocked fetch confirms a second identical generate call returns from IndexedDB without a network request.
2. Item 2: integration test confirms a second identical call within a session does not call the cache backend or issue fetch.
3. Item 3: unit tests for GenerationCache (both tiers, LRU eviction, version-keyed invalidation). Integration test asserts X-Cache: HIT on a duplicate POST.
4. Item 4: bundle-size check; before/after gzip comparison via curl.
5. Item 5: manual — confirm timings field populates.
6. Cross-cutting fidelity gate: cache-hit response bytes must equal a fresh-run response bytes for a fixture spec from debug/20260428/. CI job runs on every PR.

Cost summary

Item	Effort	Risk to production codegen
1 — IndexedDB cache	1–2 days	None (browser-only)
2 — in-memory no-op suppression	~1 hour	None
3 — server response cache	2–3 days	None (wraps /generate)
4 — payload trim	≤1 day	None (browser path only)
5 — timings (optional)	½ day	None
Total	~1 sprint	None

Open questions

1. Confirm the IndexedDB cache should key on a server-reported generator-version so deploys auto-invalidate stale entries (recommended yes).
2. Disk cache size cap (2 GB suggested) — does the hosted container have a writable volume of that size, or should we cap lower / fall back to memory-only?
3. Should we ship a ?nocache=1 query-param escape hatch for debugging?

Decision log

• Dropped: in-process worker. Permanent re-entrancy audit cost on the generator outweighs playground UX gain.
• Dropped: TypeProvider memoization. Silent-corruption bug class on cross-cutting dependencies; conservative invalidation collapses to low hit rates on real libraries.
• Kept: 99% production codegen path is unchanged. All four items live in the playground client + playground server, never in the generator.

Conclusion

The four items above target the specific traffic patterns the playground sees but the current pipeline doesn't optimize for: identical re-runs, near-identical sessions, and bytes-on-the-wire that nobody benefits from sending.

Concretely, after this work:

• Identical re-runs return instantly. Items 1 and 2 turn undo/redo, page reloads, share-link replays, and TypeSpec whitespace edits into ~zero-cost client-side renders. The user gets the result before they finish moving the mouse.
• Cross-user replays return fast. Item 3 means popular demo specs, tutorial pages, and retry storms hit a server-local cache instead of re-spawning dotnet. The first user to try a spec pays the full cost; everyone after pays an HTTP round-trip and a file read.
• The wire is leaner. Item 4 cuts the per-request bytes the browser ships and the parse work it does to prepare them — a small but free win on every request.
• We can measure further work. Item 5 surfaces existing per-stage generator timings, so any future investment beyond this scope is data-driven.

Critically, none of this changes the generator. The production codegen path used by Azure SDK CI, Spector, and every customer's tsp invocation runs the same code as today, with the same outputs, byte-for-byte. The risk surface is contained to the playground client and the playground server.

[github-actions]

🎯 Issue Triage

Feature request proposing a multi-item caching and payload optimization strategy to reduce latency in the TypeSpec playground's C# code generation loop. The proposal covers browser-side IndexedDB caching, in-memory no-op suppression, server-side response caching, and emitter payload compression — none of which touch the production codegen path.

Category: Feature Request
Area: emitter:client:csharp

🔍 Analysis

• The playground currently has no caching at any layer, meaning every regenerate (even undo/redo or whitespace edits) pays the full pipeline cost: TypeSpec compile → code model serialization → HTTP POST → dotnet subprocess → Roslyn processing → response.
• The proposal is well-scoped into 4–5 independent, shippable items with explicit non-goals (no in-process worker, no incremental TypeProvider memoization).
• Affected components: packages/http-client-csharp/emitter/src/emit-generate.browser.ts, emitter/src/code-model-writer.ts, playground-server/Program.cs, playground-server/Dockerfile.
• No risk to production codegen — all changes are playground-client and playground-server only.
• No duplicate issues found.

📝 Suggested next steps

• Confirm open questions: generator-version key strategy for auto-invalidation, disk cache size cap, ?nocache=1 escape hatch.
• Break into individual tracking issues or sub-issues per item (1–4) for easier assignment and review.
• Prioritize Item 2 (in-memory no-op suppression, ~1 hour effort) as a quick win.

Generated by Agentic Triage