Skip to content

Commit f3e7d47

Browse files
docs(examples): caching story asserts client-side honouring; README adds cacheMode + custom-store sections
The client now calls listTools() and readResource() twice each and asserts the second of each pair is cache-served — the server's resource handler counts how many times it ran and exposes that via a read-count tool, so the example verifies (server-side) that the cache hit never reached the wire. Demonstrates cacheMode:'refresh' and the post-refresh return to cache-serving. README drops the follow-up note (honouring is shipped), adds a §cacheMode section, and adds a §Custom store section showing the four-method ResponseCacheStore interface shape with the cachePartition guidance for shared stores.
1 parent b29d81c commit f3e7d47

3 files changed

Lines changed: 123 additions & 7 deletions

File tree

examples/caching/README.md

Lines changed: 45 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,52 @@
11
# caching
22

33
`CacheableResult` freshness hints (protocol revision 2026-07-28). The server declares hints at two layers — a per-registration `cacheHint` on the resource and server-level `ServerOptions.cacheHints` — and the SDK resolves most-specific-author-first (handler-return fields would
4-
take precedence over both) and stamps `ttlMs`/`cacheScope` on the wire toward modern clients only. The client reads the stamped values back.
5-
6-
> Full client-side cache **honouring** (re-using a still-fresh result instead of re-requesting) is a follow-up; this example reads what the server emits today.
4+
take precedence over both) and stamps `ttlMs`/`cacheScope` on the wire toward modern clients only. The client honours the stamped values: a still-fresh held entry is served without a round trip.
75

86
```bash
97
pnpm tsx examples/caching/client.ts
108
```
9+
10+
The client calls `listTools()` and `readResource()` twice each; the second of each pair is served from the response cache. The server exposes a `request-count` tool (how many `tools/list` requests reached it) and a `read-count` tool (how many times the resource handler ran), so the example asserts each counter is unchanged after the cache-served call and increments after `cacheMode: 'refresh'`.
11+
12+
## `cacheMode`
13+
14+
Per-call control on the cacheable verbs (`listTools()` / `listPrompts()` / `listResources()` / `listResourceTemplates()` / `readResource()`):
15+
16+
```ts
17+
await client.readResource({ uri: 'config://app' }); // 'use' (default): serve from cache if fresh
18+
await client.readResource({ uri: 'config://app' }, { cacheMode: 'refresh' }); // always fetch, then re-store
19+
await client.readResource({ uri: 'config://app' }, { cacheMode: 'bypass' }); // fetch; do not read or write the cache
20+
```
21+
22+
A `list_changed` notification still evicts immediately regardless of TTL.
23+
24+
## Custom store
25+
26+
The default per-client `InMemoryResponseCacheStore` (bounded at 512 entries by default) is enough for most hosts. To back the cache with something persistent (Redis, KV, IndexedDB), implement the five-method `ResponseCacheStore` interface — the store is a dumb keyed-value carrier; freshness and partitioning are the client's job:
27+
28+
```ts
29+
import type { CacheEntry, CacheKey, CacheScope, ResponseCacheStore } from '@modelcontextprotocol/client';
30+
31+
class MyStore implements ResponseCacheStore {
32+
async get(key: CacheKey): Promise<CacheEntry | undefined> {
33+
/* read {value, stamp, expiresAt, scope} from your backend */
34+
}
35+
async set(key: CacheKey, entry: { value: unknown; expiresAt?: number; scope?: CacheScope }): Promise<number> {
36+
/* write entry under key; return a monotonically-increasing stamp */
37+
}
38+
async delete(key: CacheKey): Promise<void> {
39+
/* drop the single entry under key (no-op if absent) */
40+
}
41+
async evict(method: string): Promise<void> {
42+
/* drop every entry whose key.method === method (across every partition) */
43+
}
44+
async clear(): Promise<void> {
45+
/* drop everything */
46+
}
47+
}
48+
49+
const client = new Client({ name: 'host', version: '1.0.0' }, { responseCacheStore: new MyStore(), cachePartition: principalId });
50+
```
51+
52+
The SDK scopes every entry by the connected server's identity automatically — you do not encode server identity into `cachePartition` or the store key yourself. When one store backs several principals against the same server, set `ClientOptions.cachePartition` to a stable identity of the authorization context (e.g. the auth subject) so `'private'`-scoped entries are isolated per principal; `'public'`-scoped entries are shared within the connected server's namespace automatically. Note `serverInfo` is self-reported, so a server that deliberately impersonates another's `name`/`version` shares its `'public'` slot; the per-principal isolation holds regardless.

examples/caching/client.ts

Lines changed: 40 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,7 @@
11
/**
22
* Reads the cache hints emitted on cacheable results (2026-07-28 connections
3-
* only) and asserts the configured values reached the wire. Full client-side
4-
* cache *honouring* (re-using a fresh result instead of re-requesting) is a
5-
* follow-up — see the SDK's tracking issue for client cache support.
3+
* only) and asserts the client honours them: a still-fresh cached entry is
4+
* served without a round trip.
65
*/
76
import { check, connectFromArgs, runClient } from '../harness.js';
87

@@ -11,22 +10,60 @@ interface Cacheable {
1110
cacheScope?: 'public' | 'private';
1211
}
1312

13+
async function callCount(client: Awaited<ReturnType<typeof connectFromArgs>>, name: 'read-count' | 'request-count'): Promise<number> {
14+
const r = await client.callTool({ name });
15+
return Number((r.content[0] as { text: string }).text);
16+
}
17+
1418
runClient('caching', async () => {
1519
// connectFromArgs picks transport (default: spawn ./server.ts over stdio; --http <url>) and era (--legacy) from argv. Your code would construct a Client and connect over your chosen transport directly.
1620
const client = await connectFromArgs(import.meta.dirname);
1721
check.equal(client.getNegotiatedProtocolVersion(), '2026-07-28');
1822

23+
// The server stamps `tools/list` with `ttlMs: 30_000, cacheScope: 'public'`.
1924
const tools = (await client.listTools()) as Cacheable & Awaited<ReturnType<typeof client.listTools>>;
2025
check.equal(tools.ttlMs, 30_000);
2126
check.equal(tools.cacheScope, 'public');
27+
// `request-count` proves the wire was reached exactly once.
28+
check.equal(await callCount(client, 'request-count'), 1);
29+
30+
// The second call is served from the response cache: the server-side
31+
// `tools/list` counter is unchanged, and the result is a fresh copy of the
32+
// held entry (so mutating it cannot reach the cache).
33+
const toolsAgain = await client.listTools();
34+
check.deepEqual(
35+
toolsAgain.tools.map(t => t.name),
36+
tools.tools.map(t => t.name)
37+
);
38+
check.equal(await callCount(client, 'request-count'), 1);
39+
40+
// `cacheMode: 'refresh'` always fetches and re-stores: the counter moves.
41+
await client.listTools(undefined, { cacheMode: 'refresh' });
42+
check.equal(await callCount(client, 'request-count'), 2);
2243

2344
const resources = (await client.listResources()) as Cacheable & Awaited<ReturnType<typeof client.listResources>>;
2445
check.equal(resources.ttlMs, 5000);
2546
check.equal(resources.cacheScope, 'public');
2647

48+
// `readResource`: the resource handler counts how many times it ran, and
49+
// the `read-count` tool exposes that counter.
2750
const read = (await client.readResource({ uri: 'config://app' })) as Cacheable & Awaited<ReturnType<typeof client.readResource>>;
2851
check.equal(read.ttlMs, 60_000);
2952
check.equal(read.cacheScope, 'private');
53+
check.equal(await callCount(client, 'read-count'), 1);
54+
55+
// Within TTL, default `cacheMode: 'use'` → served from cache; the server
56+
// handler does not run.
57+
await client.readResource({ uri: 'config://app' });
58+
check.equal(await callCount(client, 'read-count'), 1);
59+
60+
// `cacheMode: 'refresh'` always fetches and re-stores.
61+
await client.readResource({ uri: 'config://app' }, { cacheMode: 'refresh' });
62+
check.equal(await callCount(client, 'read-count'), 2);
63+
64+
// After the refresh the entry is fresh again — back to cache-served.
65+
await client.readResource({ uri: 'config://app' });
66+
check.equal(await callCount(client, 'read-count'), 2);
3067

3168
await client.close();
3269
});

examples/caching/server.ts

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,13 @@ import { McpServer } from '@modelcontextprotocol/server';
1717

1818
import { runServerFromArgs } from '../harness.js';
1919

20+
// Module-level (process-wide) counters so the values survive the harness's
21+
// stateless HTTP leg (fresh `buildServer()` per request) as well as stdio's
22+
// single per-connection instance. The client asserts against these to prove a
23+
// cache-served call never reached the server.
24+
let readCount = 0;
25+
let listCount = 0;
26+
2027
function buildServer(): McpServer {
2128
const server = new McpServer(
2229
{ name: 'caching-example', version: '1.0.0' },
@@ -40,12 +47,42 @@ function buildServer(): McpServer {
4047
description: 'Static application config (rarely changes)',
4148
cacheHint: { ttlMs: 60_000, cacheScope: 'private' }
4249
},
43-
async uri => ({ contents: [{ uri: uri.href, mimeType: 'application/json', text: '{"feature":true}' }] })
50+
async uri => {
51+
readCount++;
52+
return { contents: [{ uri: uri.href, mimeType: 'application/json', text: '{"feature":true}' }] };
53+
}
4454
);
4555

4656
// A tool, so tools/list has something to cache.
4757
server.registerTool('noop', { description: 'no-op' }, async () => ({ content: [{ type: 'text', text: 'ok' }] }));
4858

59+
// Exposes the server-side `resources/read` invocation count so the client
60+
// can assert that a cache-served call did not reach the wire.
61+
server.registerTool('read-count', { description: 'Number of resources/read calls that reached this server' }, async () => ({
62+
content: [{ type: 'text', text: String(readCount) }]
63+
}));
64+
65+
// Exposes the server-side `tools/list` invocation count.
66+
server.registerTool('request-count', { description: 'Number of tools/list requests that reached this server' }, async () => ({
67+
content: [{ type: 'text', text: String(listCount) }]
68+
}));
69+
70+
// Wrap the auto-generated `tools/list` handler so the example can prove a
71+
// cache-served `listTools()` never reached the wire. `McpServer` registers
72+
// the handler lazily on the first `registerTool()`; we re-seat it here so
73+
// every dispatch increments `listCount` before delegating to the original.
74+
// (Reaches the underlying request-handler map directly — there is no public
75+
// wrapper hook; acceptable for an instrumentation example.)
76+
const handlers = (server.server as unknown as { _requestHandlers: Map<string, (...a: unknown[]) => Promise<unknown>> })
77+
._requestHandlers;
78+
const original = handlers.get('tools/list');
79+
if (original) {
80+
handlers.set('tools/list', (...a) => {
81+
listCount++;
82+
return original(...a);
83+
});
84+
}
85+
4986
return server;
5087
}
5188

0 commit comments

Comments
 (0)