List a connection's tools from a full OpenAPI spec on Workers without re-parsing it (#1084)

* openapi: stream compile + serve from content-addressed bindings (no re-parse)

The serve path (resolveOpenApiBackedTools, the 2nd OOM site) re-parsed the
whole spec on every tools/list. For the full Microsoft Graph spec (37MB /
16.5k ops) that rebuilt a ~300MB document tree and OOM'd the 128MB Workers
isolate even after the add-path parser swap.

Add a memory-safe compile + serve path that never re-parses:

- streamOperationBindings (extract.ts): two-pass streaming compile. Pass 1
  plans tool paths from schema-free metadata (planToolPaths, extracted from
  compileToolDefinitions); pass 2 builds invocation bindings in bounded
  chunks, flushing each before the next so a huge spec's bindings are never
  all co-resident with the parsed tree.
- Content-addressed defs blob (store.ts: putDefs/getDefs, keyed by specHash):
  the normalized #/$defs schemas, built once per spec at add time and shared
  across tenants. buildDefsJson serializes them one schema at a time so the
  normalized tree never co-resides with the parsed document.
- Persist a resolved description per operation so the serve path rebuilds each
  tool def from the binding alone.
- resolveOpenApiBackedTools fast path: getDefs(specHash) + listOperations +
  toolDefFromStoredOperation, falling back to a spec re-parse for legacy rows
  or a missing/corrupt blob.
- Route the openapi, microsoft (full-graph), and google add/update paths
  through the streaming persist + putDefs.

* e2e: guard full Graph catalog add + serve at real scale

Drives the public API to add every Microsoft Graph workload (the full
~37MB / 16.5k-operation spec) and then list the served catalog. Pins
both former OOM sites at real scale: the streaming add persists one
binding per operation, and tools/list rebuilds >5000 tools from the
persisted bindings plus the content-addressed defs blob, never
re-parsing the spec. Proven green on cloud and selfhost.

* openapi: keep the file-emit hint on the fast serve path

The content-addressed serve path rebuilds a tool def from the persisted
binding instead of re-parsing the spec. The re-parse path (kept as the
fallback) appends the ToolFile emit contract to a file-returning tool's
description at the projection step; the fast path bypassed that step and
served the bare description, so a file-returning operation lost the
contract whenever it was served from the binding (the common case once a
spec is persisted).

Apply the hint at the same projection step in the fast path, sourced from
the binding's response fileHint (already inspected for the output schema),
so a file tool carries the contract identically whether served fast or via
the fallback. No persist-path or schema change: existing rows pick it up on
the next list. Covered by the tool-descriptions e2e scenario.

Author: RhysSullivan

e2e/scenarios/microsoft-graph-full.test.ts

@@ -0,0 +1,122 @@
+import { randomBytes } from "node:crypto";
+
+import { expect } from "@effect/vitest";
+import { Effect } from "effect";
+import { composePluginApi } from "@executor-js/api/server";
+import {
+  MICROSOFT_AUTH_TEMPLATE_SLUG,
+  MICROSOFT_GRAPH_ALL_PRESET_IDS,
+  MICROSOFT_GRAPH_DELEGATED_DEFAULT_SCOPES,
+} from "@executor-js/plugin-microsoft";
+import { microsoftHttpPlugin } from "@executor-js/plugin-microsoft/api";
+import { AuthTemplateSlug, ConnectionName, IntegrationSlug } from "@executor-js/sdk/shared";
+
+import { scenario } from "../src/scenario";
+import { Api, Target } from "../src/services";
+
+const api = composePluginApi([microsoftHttpPlugin()] as const);
+
+type ToolView = {
+  readonly name: string;
+};
+
+const unique = (prefix: string) => `${prefix}_${randomBytes(4).toString("hex")}`;
+
+// Adding *every* Graph workload pulls the full Microsoft Graph OpenAPI document
+// (~37MB, ~16.5k operations) and persists a binding per operation. That whole-
+// document path used to 503 on the Cloudflare worker: parsing the spec, and
+// then re-parsing it on every tools/list, each rebuilt a ~300MB JS tree that
+// blew the 128MB isolate. This scenario is the regression guard for both sites
+// at real scale: the add streams the compile + persist, and tools/list serves
+// the catalog back from the persisted bindings (+ the content-addressed defs
+// blob) without ever re-parsing the spec. It drives only the public API, so a
+// green run is evidence the full catalog lands and serves end to end.
+scenario(
+  "Microsoft Graph: the full catalog adds and serves without re-parsing the spec",
+  { timeout: 300_000 },
+  Effect.gen(function* () {
+    const target = yield* Target;
+    const { client: makeApiClient } = yield* Api;
+    const identity = yield* target.newIdentity();
+    const client = yield* makeApiClient(api, identity);
+
+    const integration = unique("msgraph_full");
+    const connection = ConnectionName.make("main");
+
+    yield* Effect.ensuring(
+      Effect.gen(function* () {
+        // Add path (1st former OOM site): the full spec is fetched and
+        // stream-compiled into one persisted binding per operation.
+        const added = yield* client.microsoft.addGraph({
+          payload: {
+            presetIds: [...MICROSOFT_GRAPH_ALL_PRESET_IDS],
+            customScopes: [],
+            slug: integration,
+            name: "Microsoft Graph (full)",
+          },
+        });
+        expect(added.slug, "the full Graph source keeps the requested slug").toBe(integration);
+        expect(
+          added.toolCount,
+          "adding every Graph workload extracts the whole catalog (thousands of operations)",
+        ).toBeGreaterThan(5_000);
+
+        const config = yield* client.microsoft.getConfig({ params: { slug: integration } });
+        expect(config?.microsoftGraphPresetIds, "every Graph workload preset is persisted").toEqual(
+          [...MICROSOFT_GRAPH_ALL_PRESET_IDS],
+        );
+        expect(
+          config?.microsoftGraphCoversFullGraph,
+          "selecting every workload is recognized as full Graph",
+        ).toBe(true);
+        expect(
+          config?.microsoftGraphScopes,
+          "full Graph delegates the app-registration default scope set",
+        ).toEqual([...MICROSOFT_GRAPH_DELEGATED_DEFAULT_SCOPES]);
+
+        yield* client.connections.create({
+          payload: {
+            owner: "org",
+            name: connection,
+            integration: IntegrationSlug.make(integration),
+            template: AuthTemplateSlug.make(MICROSOFT_AUTH_TEMPLATE_SLUG),
+            value: "token-xyz",
+          },
+        });
+
+        // Serve path (2nd former OOM site): tools/list rebuilds the catalog from
+        // the persisted bindings. The whole catalog must come back, with real
+        // descriptions, and without re-parsing the 37MB spec.
+        const tools = yield* client.tools.list({
+          query: { integration: IntegrationSlug.make(integration), connection },
+        });
+        expect(
+          tools.length,
+          "the served catalog returns the whole set of operations, not a re-parse failure",
+        ).toBeGreaterThan(5_000);
+
+        const names = tools.map((tool: ToolView) => tool.name);
+        const messageTools = names.filter((name) => name.toLowerCase().includes("message"));
+        const siteTools = names.filter((name) => name.toLowerCase().includes("site"));
+        const userTools = names.filter((name) => name.toLowerCase().includes("user"));
+        expect(messageTools, "the served catalog spans mail operations").not.toEqual([]);
+        expect(siteTools, "the served catalog spans SharePoint site operations").not.toEqual([]);
+        expect(userTools, "the served catalog spans directory user operations").not.toEqual([]);
+      }),
+      Effect.gen(function* () {
+        yield* client.connections
+          .remove({
+            params: {
+              owner: "org",
+              integration: IntegrationSlug.make(integration),
+              name: connection,
+            },
+          })
+          .pipe(Effect.ignore);
+        yield* client.microsoft
+          .removeGraph({ params: { slug: IntegrationSlug.make(integration) } })
+          .pipe(Effect.ignore);
+      }),
+    );
+  }),
+);

packages/plugins/google/src/sdk/plugin.ts

@@ -143,6 +143,7 @@ const makeGooglePluginExtension = (
       };
 
       yield* ctx.storage.putSpec(specHash, conversion.specText);
+      yield* ctx.storage.putDefs(specHash, JSON.stringify(compiled.hoistedDefs));
 
       yield* ctx.transaction(
         Effect.gen(function* () {
@@ -183,6 +184,7 @@ const makeGooglePluginExtension = (
 
       const specHash = yield* sha256Hex(conversion.specText);
       yield* ctx.storage.putSpec(specHash, conversion.specText);
+      yield* ctx.storage.putDefs(specHash, JSON.stringify(compiled.hoistedDefs));
 
       const nextConfig: GoogleIntegrationConfig = {
         ...current,
@@ -298,7 +300,8 @@ export const googlePlugin = definePlugin((options?: GooglePluginOptions) => ({
   describeAuthMethods: describeGoogleAuthMethods,
   describeIntegrationDisplay: describeGoogleIntegrationDisplay,
 
-  resolveTools: ({ config, storage }) => resolveOpenApiBackedTools({ config, storage }),
+  resolveTools: ({ integration, config, storage }) =>
+    resolveOpenApiBackedTools({ integration, config, storage }),
 
   invokeTool: ({ ctx, toolRow, credential, args }) => {
     const httpClientLayer = options?.httpClientLayer ?? ctx.httpClientLayer;

packages/plugins/microsoft/src/sdk/plugin.test.ts

@@ -33,6 +33,7 @@ paths:
   /me:
     get:
       operationId: me.GetUser
+      summary: Get the signed-in user profile
       security:
         - azureAdDelegated:
             - User.Read
@@ -341,6 +342,45 @@ describe("Microsoft Graph provider", () => {
         expect(delegated?.kind === "oauth2" ? delegated.scopes : undefined).toEqual([
           ...MICROSOFT_GRAPH_DELEGATED_DEFAULT_SCOPES,
         ]);
+
+        // Full-graph add routes through the streaming `parsedDocument` persist
+        // branch (the path the real 37MB spec takes): it persists each op's
+        // binding plus a `description` and writes the content-addressed defs
+        // blob, never re-parsing on serve. Read the operations back through the
+        // live serve path to prove they landed in storage AND that the serve
+        // fast path rebuilds tools from the persisted bindings (no spec parse).
+        yield* executor.connections.create({
+          owner: "org",
+          name: ConnectionName.make("full"),
+          integration: IntegrationSlug.make("microsoft_graph_full"),
+          template: AuthTemplateSlug.make(MICROSOFT_AUTH_TEMPLATE_SLUG),
+          value: "token-xyz",
+        });
+
+        const tools = yield* executor.tools.list();
+        const toolNames = tools.map((tool) => String(tool.name));
+        expect(toolNames).toContain("me.getUser");
+        expect(toolNames).toContain("me.messagesListMessages");
+        expect(toolNames).toContain("sites.listSites");
+
+        // The serve fast path must rebuild every tool's description from the
+        // persisted operation, not drop it. Each graph tool carries a non-empty
+        // description.
+        for (const tool of tools) {
+          expect(tool.description.length).toBeGreaterThan(0);
+        }
+
+        // `me.getUser`'s spec summary survives the add -> persist -> serve
+        // round-trip. The bare `${METHOD} ${path}` fallback inside the serve
+        // path would be "GET /me", so matching the summary proves the persisted
+        // `description` field is what's served.
+        const getUser = tools.find((tool) => String(tool.name) === "me.getUser");
+        expect(getUser?.description).toBe("Get the signed-in user profile");
+
+        // An op without a spec summary falls back to `${METHOD} ${path}`, also
+        // sourced from the persisted binding on the serve fast path.
+        const listSites = tools.find((tool) => String(tool.name) === "sites.listSites");
+        expect(listSites?.description).toBe("GET /sites");
       }),
     ),
   );

packages/plugins/microsoft/src/sdk/plugin.ts

@@ -15,21 +15,25 @@ import {
   type IntegrationConfig,
   type IntegrationRecord,
   type PluginCtx,
+  type StorageFailure,
 } from "@executor-js/sdk/core";
 import { describeApiKeyAuthMethod } from "@executor-js/sdk/http-auth";
 import {
-  compileOpenApiDocument,
-  compileOpenApiSpec,
+  compileAndPersistOpenApiOperations,
+  compileAndPersistOpenApiSpec,
   decodeOpenApiIntegrationConfig,
   invokeOpenApiBackedTool,
   makeDefaultOpenapiStore,
   normalizeOpenApiAuthInputs,
-  openApiStoredOperationsFromCompiled,
+  OpenApiExtractionError,
+  OpenApiParseError,
   resolveOpenApiBackedAnnotations,
   resolveOpenApiBackedTools,
   type Authentication,
   type AuthenticationInput,
+  type OpenApiPersistResult,
   type OpenapiStore,
+  type ParsedDocument,
 } from "@executor-js/plugin-openapi";
 
 import {
@@ -122,13 +126,31 @@ const makeMicrosoftPluginExtension = (
   ctx: PluginCtx<OpenapiStore>,
   httpClientLayer: Layer.Layer<HttpClient.HttpClient, never, never>,
 ) => {
+  const persistGraphOperations = (
+    graph: { readonly parsedDocument?: ParsedDocument; readonly specText: string },
+    integration: string,
+    specHash: string,
+  ): Effect.Effect<
+    OpenApiPersistResult,
+    OpenApiExtractionError | OpenApiParseError | StorageFailure
+  > =>
+    graph.parsedDocument !== undefined
+      ? compileAndPersistOpenApiOperations({
+          doc: graph.parsedDocument,
+          integration,
+          storage: ctx.storage,
+          specHash,
+        })
+      : compileAndPersistOpenApiSpec({
+          specText: graph.specText,
+          integration,
+          storage: ctx.storage,
+          specHash,
+        });
+
   const addGraph = (config: MicrosoftGraphConfig) =>
     Effect.gen(function* () {
       const graph = yield* buildMicrosoftGraphOpenApiSpec(config, httpClientLayer);
-      const compiled =
-        graph.parsedDocument !== undefined
-          ? yield* compileOpenApiDocument(graph.parsedDocument)
-          : yield* compileOpenApiSpec(graph.specText);
       const slug = IntegrationSlug.make(config.slug?.trim() || DEFAULT_MICROSOFT_SLUG);
 
       const existing = yield* ctx.core.integrations.get(slug);
@@ -156,7 +178,7 @@ const makeMicrosoftPluginExtension = (
 
       yield* ctx.storage.putSpec(specHash, graph.specText);
 
-      yield* ctx.transaction(
+      const persisted = yield* ctx.transaction(
         Effect.gen(function* () {
           yield* ctx.core.integrations.register({
             slug,
@@ -167,14 +189,11 @@ const makeMicrosoftPluginExtension = (
             canRemove: true,
             canRefresh: true,
           });
-          yield* ctx.storage.putOperations(
-            String(slug),
-            openApiStoredOperationsFromCompiled(String(slug), compiled),
-          );
+          return yield* persistGraphOperations(graph, String(slug), specHash);
         }),
       );
 
-      return { slug, toolCount: compiled.definitions.length };
+      return { slug, toolCount: persisted.toolCount };
     });
 
   const updateGraph = (rawSlug: string, input?: MicrosoftUpdateInput) =>
@@ -199,14 +218,8 @@ const makeMicrosoftPluginExtension = (
         },
         httpClientLayer,
       );
-      const compiled =
-        graph.parsedDocument !== undefined
-          ? yield* compileOpenApiDocument(graph.parsedDocument)
-          : yield* compileOpenApiSpec(graph.specText);
-
       const previousOperations = yield* ctx.storage.listOperations(rawSlug);
       const previousNames = new Set(previousOperations.map((op) => op.toolName));
-      const nextNames = new Set(compiled.definitions.map((def) => def.toolPath));
 
       const specHash = yield* sha256Hex(graph.specText);
       yield* ctx.storage.putSpec(specHash, graph.specText);
@@ -229,17 +242,15 @@ const makeMicrosoftPluginExtension = (
         ...(input?.baseUrl ? { baseUrl: input.baseUrl } : {}),
       };
 
-      yield* ctx.transaction(
+      const persisted = yield* ctx.transaction(
         Effect.gen(function* () {
           yield* ctx.core.integrations.update(slug, {
             config: nextConfig satisfies MicrosoftGraphIntegrationConfig as IntegrationConfig,
           });
-          yield* ctx.storage.putOperations(
-            rawSlug,
-            openApiStoredOperationsFromCompiled(rawSlug, compiled),
-          );
+          return yield* persistGraphOperations(graph, rawSlug, specHash);
         }),
       );
+      const nextNames = new Set(persisted.toolNames);
 
       const connections = yield* ctx.connections.list({ integration: slug });
       yield* Effect.forEach(
@@ -257,7 +268,7 @@ const makeMicrosoftPluginExtension = (
 
       return {
         slug,
-        toolCount: compiled.definitions.length,
+        toolCount: persisted.toolCount,
         addedTools: [...nextNames].filter((name) => !previousNames.has(name)).sort(),
         removedTools: [...previousNames].filter((name) => !nextNames.has(name)).sort(),
       };
@@ -340,7 +351,8 @@ export const microsoftPlugin = definePlugin((options?: MicrosoftPluginOptions) =
   describeAuthMethods: describeMicrosoftAuthMethods,
   describeIntegrationDisplay: describeMicrosoftIntegrationDisplay,
 
-  resolveTools: ({ config, storage }) => resolveOpenApiBackedTools({ config, storage }),
+  resolveTools: ({ integration, config, storage }) =>
+    resolveOpenApiBackedTools({ integration, config, storage }),
 
   invokeTool: ({ ctx, toolRow, credential, args }) => {
     const httpClientLayer = options?.httpClientLayer ?? ctx.httpClientLayer;