fix(mcp): keep tool catalogs in sync with the server's live tool set (#1246)

* fix(mcp): keep tool catalogs in sync with the server's live tool set

A connection's tools were listed once at create time and only re-listed on
an executor-side config edit or a manual Refresh, so server-side catalog
changes (tools added, removed, renamed) silently broke invocations and
never surfaced new tools.

Spec-compliance and staleness fixes:

- Follow nextCursor pagination on tools/list: discovery previously kept
  only the first page of a paginated catalog.
- Handle notifications/tools/list_changed received during a call window:
  the connection's persisted catalog is marked stale (tools_synced_at
  cleared) and re-listed on the next tools read.
- Detect unknown-tool rejections on tools/call, in both wire shapes (the
  spec's -32602 protocol error and the reference TS SDK's isError
  envelope), anchored to the exact tool name. The call answers a typed
  mcp_tool_unknown failure telling the caller to re-list, and the catalog
  heals on the next read.
- Re-list remote catalogs on read once older than a freshness TTL
  (ExecutorConfig.toolsSyncTtlMs, default 15 minutes, null disables),
  covering servers that change tools with no notification. Plugins opt in
  via the new remoteToolCatalog flag; MCP sets it.
- Treat failed listings as non-authoritative (ResolveToolsResult
  .incomplete): keep the previously persisted catalog instead of wiping
  working tools over a transient outage, and stamp the sync time so a
  down server isn't re-dialed on every read.

Core additions: ctx.connections.markToolsStale (deferred refresh for
mid-invocation signals) and a bounded stale-scan in toolsList that only
widens with the TTL when a loaded plugin actually declares a remote
catalog.

* test(e2e): prove MCP tool-catalog sync end-to-end

Four cross-target scenarios (cloud + selfhost) driving only public surfaces
(typed API + sandbox executions) against real MCP servers whose catalogs
mutate mid-scenario:

- list_changed: a tool call renames the server's catalog mid-call; the
  notification rides the open connection and the very next tools read
  re-lists — no refresh click, no failing retry.
- unknown-tool self-heal: the catalog mutates outside any call window (no
  notification available); invoking the retired tool fails with the typed
  mcp_tool_unknown error telling the agent to re-list, and that failure
  alone heals the catalog for the next read.
- pagination: a server paging tools/list via nextCursor registers its whole
  catalog, not just the first page.
- outage resilience: a refresh against a dead server keeps the working
  catalog instead of wiping it, and converges to the server's new catalog
  once it recovers.

The mutable-catalog test fixture now also sends list_changed through the
tool handler's request-scoped notifier: the SDK's RegisteredTool.update
emits it with no relatedRequestId, which the streamable-http transport
routes to the standalone GET SSE stream — a stream a request-scoped client
may never hold open. Stamping the request id routes it onto the in-flight
call's response stream, matching how a real server notifies mid-call.

* test(e2e): film the Tools tab following an MCP server-side rename

A browser scenario (cloud, video + per-step screenshots) walking the whole
user journey: add a live MCP server through the add flow, connect with no
auth, see the v1 tool in the Tools tab, run the tool that renames the
server's catalog mid-call (the server pushes list_changed on the open
connection), then revisit Tools and watch the renamed catalog appear — no
Refresh click anywhere. The old behavior freezes this UI on the stale tool
forever.

The mutable-catalog fixture takes a per-run server name so the derived
integration namespace can't collide on shared-tenant targets.

Author: RhysSullivan

.changeset/mcp-tool-catalog-sync.md

@@ -0,0 +1,23 @@
+---
+"@executor-js/plugin-mcp": patch
+"@executor-js/sdk": patch
+---
+
+Keep MCP tool catalogs in sync with the server's live tool set. Previously a
+connection's tools were listed once at create time and never updated unless the
+integration's config changed or a user clicked Refresh, so server-side tool
+changes silently broke invocations.
+
+- `tools/list` discovery now follows `nextCursor` pagination per the MCP spec,
+  so servers with paginated catalogs list completely instead of first-page-only.
+- The client handles `notifications/tools/list_changed` received during a tool
+  call and marks the connection's persisted catalog stale; the next tools read
+  re-lists from the server.
+- An unknown-tool rejection from the server (protocol error or the reference
+  SDK's error envelope) returns a typed `mcp_tool_unknown` failure telling the
+  caller to re-list, and marks the catalog stale so it heals on the next read.
+- Remote catalogs now also refresh on read once older than a freshness TTL
+  (`ExecutorConfig.toolsSyncTtlMs`, default 15 minutes, `null` to disable),
+  covering servers that change tools without notifying.
+- A failed listing (server unreachable, auth not ready) no longer wipes the
+  previously persisted catalog; it is kept and retried after the TTL.

e2e/scenarios/mcp-catalog-sync-ui.test.ts

@@ -0,0 +1,111 @@
+// Cross-target (browser): MCP tool-catalog freshness, FILMED through the real
+// web UI. The session video + per-step screenshots are the artifact: a user
+// adds a live MCP server, sees its tool in the Tools tab, runs a tool that
+// renames the server's catalog mid-call (the server pushes
+// `notifications/tools/list_changed` on the open connection), and then WATCHES
+// the Tools tab serve the renamed catalog on the next visit — no Refresh
+// click anywhere in the journey. The old behavior freezes this UI on the
+// stale tool forever.
+import { randomBytes } from "node:crypto";
+
+import { expect } from "@effect/vitest";
+import { Effect } from "effect";
+import { makeMutableCatalogMcpServer, serveMcpServer } from "@executor-js/plugin-mcp/testing";
+
+import { scenario } from "../src/scenario";
+import { Browser, Target } from "../src/services";
+
+scenario(
+  "MCP catalog · the Tools tab follows a server-side rename after a list_changed notification",
+  {},
+  Effect.scoped(
+    Effect.gen(function* () {
+      const target = yield* Target;
+      const browser = yield* Browser;
+
+      // A real MCP server whose `rename_greet` tool renames `greet` →
+      // `greet_v2` mid-call and notifies the open connection. The server name
+      // is unique per run (it derives the integration namespace) so runs can't
+      // collide on targets whose identities share one tenant.
+      const mutable = makeMutableCatalogMcpServer({
+        name: `catalog-sync-${randomBytes(3).toString("hex")}`,
+      });
+      const server = yield* serveMcpServer(mutable.factory);
+      const identity = yield* target.newIdentity();
+
+      yield* browser.session(identity, async ({ page, step }) => {
+        await step("Open the add-MCP flow pointed at the live server", async () => {
+          await page.goto(`/integrations/add/mcp?url=${encodeURIComponent(server.endpoint)}`, {
+            waitUntil: "networkidle",
+          });
+          // The URL auto-probes (debounced); the method list appears once the
+          // probe lands — an open server seeds the detected no-auth method.
+          await page.getByText("How does this server authenticate?").waitFor();
+          await page.getByText("Method 1 · Detected").waitFor();
+        });
+
+        await step("Add the source", async () => {
+          await page.getByRole("button", { name: "Add source" }).click();
+          await page.waitForURL(/\/integrations\/(?!add\b)[^/?]+$/, { timeout: 30_000 });
+          await page.getByText("Connections").first().waitFor();
+        });
+
+        await step("Connect with no authentication", async () => {
+          await page.getByRole("button", { name: "Add connection" }).first().click();
+          await page.getByRole("tab", { name: "No authentication" }).waitFor();
+          await page.getByRole("button", { name: "Add connection" }).last().click();
+          // The connect flow dials the server and produces the tool catalog.
+          await page.getByText("Connection added").waitFor();
+        });
+
+        // Tools render as a collapsed dotted-name tree (namespace → leaf);
+        // typing in the filter box expands every match, so it is both the
+        // reveal mechanism and a search the video shows off.
+        const filterTools = async (query: string) => {
+          const filter = page.getByPlaceholder(/^Filter \d+ tools/);
+          await filter.waitFor();
+          await filter.fill(query);
+        };
+
+        // A tool's tree row is a button whose accessible name is the full
+        // leaf label — filter-highlight <mark> spans inside the label don't
+        // fragment it, so exact role queries can't false-match a substring
+        // (`greet` inside `greet_v2`).
+        const toolRow = (name: string) => page.getByRole("button", { name, exact: true }).first();
+
+        await step("The Tools tab lists the server's v1 catalog", async () => {
+          await page.getByRole("tab", { name: "Tools" }).click();
+          await filterTools("greet");
+          await toolRow(mutable.initialToolName).waitFor();
+          await toolRow("rename_greet").waitFor();
+        });
+
+        await step("Run rename_greet — the server renames its catalog mid-call", async () => {
+          await toolRow("rename_greet").click();
+          await page.getByRole("tab", { name: "Run" }).click();
+          await page.getByRole("button", { name: "Run", exact: true }).click();
+          // The result card proves the call reached the server (the payload
+          // renders inside a highlighted code block, so assert the badge);
+          // the server pushed list_changed on the same open connection.
+          await page.getByText("Result").first().waitFor();
+          await page.getByText("Success").first().waitFor();
+        });
+
+        await step("Revisit Tools — the catalog followed the rename by itself", async () => {
+          // Re-enter the page: a fresh tools read. The list_changed the server
+          // sent during the call marked the catalog stale, so THIS read
+          // re-lists — the renamed tool appears with no Refresh click.
+          await page.reload({ waitUntil: "networkidle" });
+          await page.getByRole("tab", { name: "Tools" }).click();
+          await filterTools("greet");
+          await toolRow(mutable.renamedToolName).waitFor({ timeout: 30_000 });
+        });
+
+        const staleToolStillListed = await toolRow(mutable.initialToolName)
+          .isVisible()
+          .catch(() => false);
+        expect(staleToolStillListed, "the retired tool left the catalog").toBe(false);
+      });
+    }),
+  ),
+);