Skip to content

[codex] Use compiler-backed schema previews#825

Merged
RhysSullivan merged 19 commits into
mainfrom
codex/json-schema-ts-compiler
May 16, 2026
Merged

[codex] Use compiler-backed schema previews#825
RhysSullivan merged 19 commits into
mainfrom
codex/json-schema-ts-compiler

Conversation

@RhysSullivan

@RhysSullivan RhysSullivan commented May 15, 2026

Copy link
Copy Markdown
Owner

Summary

  • Replace the hand-written JSON Schema to TypeScript preview renderer with compiler-backed generation.
  • Vendor the compiler into @executor-js/json-schema-to-typescript so Executor owns the schema compiler used by @executor-js/sdk.
  • Remove the compiler's Prettier dependency; the vendored formatter is now a no-op and callers normalize preview display themselves.
  • Fix vendored compiler type-only imports/exports so Bun runtime imports do not fail and collapse previews to unknown.
  • Normalize refs and nullable schemas before compilation so code-mode previews get valid TypeScript for dashed refs, quoted dashed properties, nullable enums/constants, recursive refs, and large schema graphs.
  • Return compact schema roots plus reachable shared definitions, instead of embedding full $defs into every root.
  • Keep the browser bundle off Node-only SDK/compiler code through @executor-js/sdk/shared.
  • Make the local UI schema tab load JSON schema only; TypeScript compilation now runs only when includeTypeScript=true / the TypeScript tab is opened.
  • Trim OpenAPI source metadata and tools-list responses so Cloudflare source pages do not download raw specs or all row descriptions up front.

Root Cause

The previous preview renderer built TypeScript strings by hand and used raw $ref tails as type names, so schema names such as foo-bar could produce invalid TypeScript. Adding a real compiler at the SDK root also exposed an existing package-boundary issue: React imported @executor-js/sdk for shared IDs/errors/policy helpers, which could pull Node-only compiler dependencies into the browser bundle.

The remaining Cloudflare load slowness had three separate causes: the source detail page fetched the full stored OpenAPI spec even though it only needed metadata, the default schema tab eagerly ran the TypeScript compiler despite rendering the raw schema view first, and the tool-list endpoints shipped descriptions/annotation text for every row before any tool was selected.

After vendoring, Bun's TypeScript runtime hit upstream-style type-only imports/exports emitted as runtime imports. The SDK caught that import failure and returned unknown for the whole preview. The vendored package now uses explicit import type / export type for those symbols.

Vendored Compiler Notes

  • @executor-js/sdk now depends on @executor-js/json-schema-to-typescript via workspace:*.
  • The vendored compiler is based on json-schema-to-typescript@15.0.4 source, without the upstream CLI entrypoint.
  • Runtime deps for the vendored compiler are only @apidevtools/json-schema-ref-parser, js-yaml, and lodash.
  • Prettier can still appear in the root lockfile through unrelated repo tooling and transitive dependencies, but it is no longer part of the Executor SDK schema compiler path.

Local Measurements

  • Cloudflare OpenAPI source metadata: ~11,031,966 B before, 317 B after omitting config.spec.
  • Cloudflare source tools list: ~1,160,557 B before, 258,518 B after compact rows.
  • Cloudflare global tools list: ~1,096,457 B before, 237,667 B after compact rows.
  • Cloudflare schema endpoint, default JSON/schema view warm runs: 71,222 B, ~0.06-0.14s.
  • Cloudflare schema endpoint with includeTypeScript=true after runtime import fix: Input { account_id: AccessIdentifier; body: AccessAppRequest; }, Output (AccessAppsComponentsSchemasSingleResponse & { result?: AccessAppResponse; }), 184 shared TypeScript definitions.

Validation

  • bun run --cwd packages/core/json-schema-to-typescript typecheck
  • bun run --cwd packages/core/json-schema-to-typescript build
  • bun run --cwd packages/core/sdk test src/schema-types.test.ts
  • Bun runtime repro for buildToolTypeScriptPreview returns { account_id: string; body: unknown; } instead of root unknown.
  • Live Cloudflare schema endpoint repro on a fresh local server returns { account_id: AccessIdentifier; body: AccessAppRequest; }.
  • bun run --cwd packages/core/sdk test -- src/schema-refs.test.ts
  • bun run --cwd packages/core/sdk test -- src/executor.test.ts
  • bun run --cwd packages/core/execution test -- src/tool-invoker.test.ts
  • bun run --cwd packages/core/sdk typecheck
  • bun run --cwd packages/core/api typecheck
  • bun run --cwd packages/react typecheck
  • bun run --cwd packages/plugins/openapi typecheck
  • bun run --cwd packages/plugins/mcp typecheck
  • bun run --cwd packages/plugins/graphql typecheck
  • bun run --cwd packages/plugins/google-discovery typecheck
  • bun run --cwd packages/plugins/onepassword typecheck
  • bun run --cwd packages/core/sdk build
  • bun run --cwd apps/local build
  • bunx oxfmt --check on changed files
  • git diff --check
  • oxlint --deny-warnings on changed TypeScript files

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented May 15, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
executor-cloud 4142102 May 16 2026, 08:34 AM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented May 15, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
executor-marketing 4142102 Commit Preview URL

Branch Preview URL
May 16 2026, 08:34 AM

@RhysSullivan RhysSullivan force-pushed the codex/json-schema-ts-compiler branch 4 times, most recently from d0c41b7 to 3bc6e67 Compare May 15, 2026 21:10
@RhysSullivan RhysSullivan force-pushed the codex/json-schema-ts-compiler branch from 3bc6e67 to eaae2d5 Compare May 15, 2026 22:19
@pkg-pr-new

pkg-pr-new Bot commented May 16, 2026

Copy link
Copy Markdown

Open in StackBlitz

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@825

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@825

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@825

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@825

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@825

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@825

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@825

@executor-js/plugin-google-discovery

npm i https://pkg.pr.new/@executor-js/plugin-google-discovery@825

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@825

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@825

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@825

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@825

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@825

executor

npm i https://pkg.pr.new/executor@825

commit: 4142102

@RhysSullivan RhysSullivan marked this pull request as ready for review May 16, 2026 07:09
Comment thread apps/cli/src/main.ts Outdated
Comment thread apps/cli/src/main.ts
Comment thread apps/cloud/src/services/sources-refresh.node.test.ts Outdated
Comment thread apps/cloud/src/services/sources-refresh.node.test.ts Outdated
Comment thread apps/cloud/src/services/tenant-isolation.node.test.ts
@RhysSullivan RhysSullivan merged commit 27bd9b4 into main May 16, 2026
11 checks passed
RhysSullivan added a commit that referenced this pull request May 31, 2026
* Use compiler-backed schema previews

* Keep schema compiler off browser path

* Vendor schema TypeScript compiler

* Fix vendored compiler runtime imports

* Fix schema compiler CI failures

* Fold schema compiler into SDK

* Document vendored schema compiler attribution

* Simplify tool schema TypeScript loading

* Always include schema TypeScript previews

* Use raw schema compiler previews

* Revert "Use raw schema compiler previews"

This reverts commit f95c650b17216ae7eac3e4f366eb5900ae6f27bf.

* Restore tool metadata responses

* Remove vendored compiler lodash dependency

* Clean up vendored schema compiler helpers

* Treat vendored schema compiler as internal code

* Add real OpenAPI schema preview baselines

* Resolve schema refs without external loading

* Fix schema compiler format and typecheck

* Remove vendored compiler debug logging
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant