feat(binding-mcp): mcp · proxy cache option (#1737)

[jfallows]

Closes #1737.

Summary

Adds options.cache to the mcp · proxy binding. When configured, the proxy hydrates a shared store with tools/list, resources/list, and prompts/list payloads aggregated across the binding's configured routes, and serves those list calls from cache without round-tripping downstream. Folds what was originally proposed as a separate mcp · cache kind into an option on the existing proxy — every original topology has an exact equivalent in the folded model, and the engine kind: cache prerequisite drops out.

Config

Binding options

options:
  cache:
    store: memory0           # required; references a configured store
    ttl: PT5M                # optional; refresh interval (default PT5M)
    authorization:           # optional; max one entry (schema-enforced)
      jwt0:
        credentials: "${env.SERVICE_TOKEN}"

Engine configuration

Property	Default	Purpose
zilla.binding.mcp.hydrate.filter	accepts every kind	Static Class::method reference returning an IntPredicate; restricts which list kinds the binding hydrates (mainly used by per-kind ITs to scope refresh scenarios)
zilla.binding.mcp.lease.ttl	PT30S	TTL on the cross-worker lifecycle and per-kind list leases
zilla.binding.mcp.lease.retry	PT0.1S	Poll interval for losing-worker lease retry

Runtime behaviour

• Hydrate. At binding attach, McpProxyCacheHydrater opens one lifecycle stream to the binding itself (originId == routedId == binding.id) and issues per-kind list BEGINs (gated by MCP_HYDRATE_FILTER) via McpProxyCacheListHydrater subclasses — one per kind. The binding's own dispatch (McpProxyListFactory.newStream) detects the self-loop (originId == routedId), skips the cache check, and creates an McpListServer that fans out across all matching routes via binding.resolveAll(...). Each kind's aggregated response body is written to the configured store under keys tools / resources / prompts.
• Lifecycle gating. Agent lifecycle BEGIN replies are deferred via signaler.signalNow (delivered to the agent's reply-direction stream — the engine registers binding consumers in throttles[replyId], where SignalFW is dispatched) until every configured kind has settled. By the time an agent sees lifecycle initialized, every list call hits a populated cache. The self-loop hydrate session bypasses this gating to avoid deadlocking against itself.
• Lifecycle persistence. One hydrate-side lifecycle is opened to each downstream route and reused for all three per-kind list hydrate streams — exercised by the cache.hydrate scenario, whose app1-side script asserts exactly one lifecycle BEGIN and three list BEGINs (not four).
• Multi-route aggregation. For a binding configured with multiple toolkit routes, the hydrate fans out to every matching route via binding.resolveAll(...). Each route returns its own list, McpListServer injects the per-route prefix (e.g., bluesky__get_weather, quartz+file:///doc.txt) and merges them into the single per-kind cache value — verified by cache.hydrate.toolkit against proxy.cache.toolkit.yaml.
• Authorization. When options.cache.authorization is set, attach resolves the single guard, calls guard.reauthorize(traceId, binding.id, 0L, credentials) once, and stamps the resulting token onto every outbound BEGIN/END/WINDOW on the hydrate session and its list streams. Guard and credentials live on McpLifecycleCache so the hydrater doesn't reach back into the binding config for them.
• Serve. Agent tools/list, resources/list, prompts/list requests hit McpCacheListServer, which reads the cached payload from the store and emits it as DATA + END without forwarding.
• Periodic refresh. After each kind's terminal write (cache.put completion, bodyLen==0 skip-put, or abort/reset), McpProxyCacheListHydrater.scheduleRefresh schedules signaler.signalAt(Instant.now().plus(ttl), kindSignalId, …) for that kind. On expiry, a new list stream is issued and routed through the same self-loop path; the existing lifecycle is reused via McpLifecycleClient.supplyClient. Aborts during refresh preserve the prior cached entry.
• Lease coordination across workers. A binding-wide lifecycle.lock (TTL = MCP_LEASE_TTL) arbitrates which worker opens the hydrate session (loser polls at MCP_LEASE_RETRY intervals); a per-kind .lock arbitrates which worker issues each list call on the wire. Winners release on hydrate complete (lifecycle lease) or after cache.put / abort terminal (per-kind lease). Settle paths split: cache hits funnel through markSettled so only the worker that actually did the list-stream work arms a refresh signal.
• Notification-driven refresh (notifications/{tools,resources,prompts}/list_changed) — deferred per the issue description.

Refactor

McpProxyFactory ballooned absorbing cache support. Extracted into four per-kind-family factories, each implementing BindingHandler and dispatched from McpProxyFactory.factories: Int2ObjectHashMap:

• McpProxyLifecycleFactory — KIND_LIFECYCLE. Detects self-loop in onServerBegin and bypasses the hydrate gating so the hydrater's own lifecycle reply doesn't deadlock.
• McpProxyItemFactory (abstract base + McpProxyToolsCallFactory / McpProxyPromptsGetFactory / McpProxyResourcesReadFactory) — KIND_TOOLS_CALL / KIND_PROMPTS_GET / KIND_RESOURCES_READ. Each subclass passes its kind to the base via the constructor and overrides injectInitialClientBeginEx.
• McpProxyListFactory (abstract base + McpProxyToolsListFactory / McpProxyPromptsListFactory / McpProxyResourcesListFactory) — KIND_TOOLS_LIST / KIND_PROMPTS_LIST / KIND_RESOURCES_LIST. newStream checks cache != null && originId != routedId — agent calls hit McpCacheListServer (serve-from-cache); hydrate self-loop falls through to McpListServer (passthrough/aggregation).

Hydrate coordination extracted into dedicated top-level classes:

• McpProxyCacheHydrater — multi-kind coordinator; owns the lifecycle stream via inner McpHydrateLifecycleStream. Self-targets binding.id (originId == routedId).
• McpProxyCacheListHydrater (abstract base + McpProxyCacheToolsListHydrater / McpProxyCacheResourcesListHydrater / McpProxyCachePromptsListHydrater) — per-kind list stream via inner McpListHydrateStream. Subclasses override signalId() and injectInitialBeginEx(); the inner stream uses ExpandableArrayBuffer for body accumulation. The initial stream is closed deferred — doListHydrateBegin marks closingInitial, and onListHydrateWindow issues doListHydrateEnd once the peer credit lands.

Stream construction follows the standard "ctor for state init, do*Begin / do*End post-construction" split — no behavioural work in stream constructors. Both stream classes handle the full close protocol (onListHydrateAbort → doListHydrateAbort on initial, onListHydrateReset → doListHydrateReset on reply).

McpProxyFactory shrinks from ~3631 lines to ~134 lines and now owns only the dispatch map and attach/detach. McpBindingConfig owns the hydrater (hydrater.start() on attach, cleanup on detach), the per-kind McpListCache instances, and the McpLifecycleCache (which carries its own guard + credentials). McpAuthorizationConfig is reused under both options.authorization (server/client kinds) and options.cache.authorization (proxy kind) — same shape, the latter additionally carries credentials.

Engine SPI change

runtime/engine: Signaler gains two signalAt(Instant, …) abstract overloads alongside the existing long-based methods. The single in-tree implementation (EngineWorker.EngineSignaler) converts via Instant.toEpochMilli(); KafkaClientConnectionPool.KafkaClientSignaler and TlsWorker.TlsSignaler (test bench) provide matching delegations. Used by binding-mcp for lease retry and cache refresh scheduling so callers don't have to mix currentTimeMillis() with Duration.toMillis() arithmetic.

Spec scripts

Folder	Verifies
cache.hydrate	hydrate session opens with lifecycle, persists across all three list kinds, cache populates for tools / resources / prompts
cache.hydrate.toolkit	hydrate fans out across two toolkit routes (bluesky / quartz); each downstream sees one lifecycle plus three list BEGINs; merged cache value carries route prefixes
cache.hydrate.error	downstream abort on tools/list during hydrate; resources & prompts still complete (cross-kind error isolation)
cache.hydrate.credentials	hydrate BEGIN carries the guard-issued auth token across lifecycle + all three list streams
cache.serve.initialize	agent observes lifecycle initialised once cache is ready
cache.serve.tools.list / .resources.list / .prompts.list	each list served from cache, no downstream call
cache.refresh.tools / .resources / .prompts	second list call after TTL on same hydrate session
cache.refresh.tools.error	refresh aborts; prior cached entry retained
cache.refresh.tools.contended	two workers, exactly one issues each list call on the wire (lifecycle reused for initial + refresh)

ITs

Peer-to-peer (specs/binding-mcp.spec/.../streams/cache/)	Engine-driven (runtime/binding-mcp/...)
ProxyCacheLifecycleIT (5 tests)	McpProxyCacheLifecycleIT (5 tests)
ProxyCacheToolsListIT (4 tests)	McpProxyCacheToolsListIT (3 tests)
ProxyCacheResourcesListIT (2 tests)	McpProxyCacheResourcesListIT (2 tests)
ProxyCachePromptsListIT (2 tests)	McpProxyCachePromptsListIT (2 tests)
—	McpProxyCacheContentionIT (1 test, ENGINE_WORKERS=2)

cache.hydrate (multi-kind single-route) is the canonical hydrate scenario; per-kind hydrate scripts dropped since they were strict subsets of the multi-kind one. cache.hydrate.toolkit covers the multi-route aggregation case — the peer-to-peer client connects directly to app1 and app2 (sequenced with read notify / connect await) to mirror the server's two accepts, since the proxy is absent in peer-to-peer mode. The per-kind ITs retain MCP_HYDRATE_FILTER configuration to scope the cache.refresh.* scenarios to one kind.

Test-only configs

• proxy.cache.yaml — empty test store, single worker, single downstream route; used by hydrate ITs.
• proxy.cache.toolkit.yaml — empty test store, two toolkit routes (bluesky → app1, quartz → app2); multi-route hydrate IT.
• proxy.cache.refresh.yaml — empty test store with ttl: PT1S; refresh + contention ITs.
• proxy.cache.seeded.yaml — test store pre-seeded with cached JSON; serve ITs.
• proxy.cache.credentials.yaml — test store + engine test guard with credentials; credentials IT.

TestStore enhancement (runtime/engine test sources)

TestStore now owns ConcurrentMap<String, Map<String, Entry>> keyed by storeId; workers attached to the same store share entries. options.entries seeds via putIfAbsent so the first attach wins. Previously the entries map was per-handler, so putIfAbsent-based leases were trivially won by every worker — required for McpProxyCacheContentionIT to observe a single winner. Transparent for single-worker scenarios. binding-mcp no longer test-depends on store-memory; all cache ITs run on type: test.

Test plan

• Peer-to-peer ITs (specs/binding-mcp.spec ProxyCache*IT) → 13 pass / 0 fail.
• Engine-driven binding-mcp ITs (runtime/binding-mcp) → 155 pass / 0 fail / 0 skipped.
• runtime/engine unit tests → 320 pass / 0 fail.
• specs/engine.spec → 39 pass / 0 fail after the TestStore sharing change.

🤖 Generated with Claude Code

---

Generated by Claude Code

[jfallows]

LGTM