Skip to content

Commit 3d30316

Browse files
SteveSandersonMSCopilot
SteveSandersonMS
and
Copilot
committed
Add E2E tests for per-connection external tool union and disconnect cleanup
- two clients register different tools and agent uses both: client1 has city_lookup, client2 has currency_lookup, verifies union semantics - disconnecting client removes its tools: client2's ephemeral_tool removed after disconnect, client1's stable_tool persists - Update existing test: client2 resumes with NO tools (doesn't overwrite) - Regenerated snapshots for existing tests (minor wording changes) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 23edb58 commit 3d30316

6 files changed

Lines changed: 348 additions & 10 deletions

‎nodejs/test/e2e/multi-client.test.ts‎

Lines changed: 95 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ describe("Multi-client broadcast", async () => {
1818
await initSession.destroy();
1919

2020
const actualPort = (client1 as unknown as { actualPort: number }).actualPort;
21-
const client2 = new CopilotClient({ cliUrl: `localhost:${actualPort}` });
21+
let client2 = new CopilotClient({ cliUrl: `localhost:${actualPort}` });
2222

2323
afterAll(async () => {
2424
await client2.stop();
@@ -39,10 +39,9 @@ describe("Multi-client broadcast", async () => {
3939
tools: [tool],
4040
});
4141

42-
// Client 2 resumes the same session (separate TCP connection, own handlers)
42+
// Client 2 resumes with NO tools — should not overwrite client 1's tools
4343
const session2 = await client2.resumeSession(session1.sessionId, {
4444
onPermissionRequest: approveAll,
45-
tools: [tool],
4645
});
4746

4847
// Track events seen by each client
@@ -194,4 +193,97 @@ describe("Multi-client broadcast", async () => {
194193

195194
await session2.destroy();
196195
});
196+
197+
it("two clients register different tools and agent uses both", { timeout: 90_000 }, async () => {
198+
const toolA = defineTool("city_lookup", {
199+
description: "Returns a city name for a given country code",
200+
parameters: z.object({
201+
countryCode: z.string().describe("A two-letter country code"),
202+
}),
203+
handler: ({ countryCode }) => `CITY_FOR_${countryCode}`,
204+
});
205+
206+
const toolB = defineTool("currency_lookup", {
207+
description: "Returns a currency for a given country code",
208+
parameters: z.object({
209+
countryCode: z.string().describe("A two-letter country code"),
210+
}),
211+
handler: ({ countryCode }) => `CURRENCY_FOR_${countryCode}`,
212+
});
213+
214+
// Client 1 creates a session with tool A
215+
const session1 = await client1.createSession({
216+
onPermissionRequest: approveAll,
217+
tools: [toolA],
218+
});
219+
220+
// Client 2 resumes with tool B (different tool, union should have both)
221+
const session2 = await client2.resumeSession(session1.sessionId, {
222+
onPermissionRequest: approveAll,
223+
tools: [toolB],
224+
});
225+
226+
// Send a prompt that requires both tools
227+
const response = await session1.sendAndWait({
228+
prompt:
229+
"Use the city_lookup tool with countryCode 'US' and the currency_lookup tool with countryCode 'US'. Tell me both results.",
230+
});
231+
232+
expect(response?.data.content).toContain("CITY_FOR_US");
233+
expect(response?.data.content).toContain("CURRENCY_FOR_US");
234+
235+
await session2.destroy();
236+
});
237+
238+
it("disconnecting client removes its tools", { timeout: 90_000 }, async () => {
239+
const toolA = defineTool("stable_tool", {
240+
description: "A tool that persists across disconnects",
241+
parameters: z.object({ input: z.string() }),
242+
handler: ({ input }) => `STABLE_${input}`,
243+
});
244+
245+
const toolB = defineTool("ephemeral_tool", {
246+
description: "A tool that will disappear when its client disconnects",
247+
parameters: z.object({ input: z.string() }),
248+
handler: ({ input }) => `EPHEMERAL_${input}`,
249+
});
250+
251+
// Client 1 creates a session with stable_tool
252+
const session1 = await client1.createSession({
253+
onPermissionRequest: approveAll,
254+
tools: [toolA],
255+
});
256+
257+
// Client 2 resumes with ephemeral_tool
258+
const session2 = await client2.resumeSession(session1.sessionId, {
259+
onPermissionRequest: approveAll,
260+
tools: [toolB],
261+
});
262+
263+
// Verify both tools work before disconnect
264+
const bothResponse = await session1.sendAndWait({
265+
prompt:
266+
"Use the stable_tool with input 'test1' and the ephemeral_tool with input 'test2'. Tell me both results.",
267+
});
268+
expect(bothResponse?.data.content).toContain("STABLE_test1");
269+
expect(bothResponse?.data.content).toContain("EPHEMERAL_test2");
270+
271+
// Disconnect client 2 without destroying the shared session
272+
await client2.forceStop();
273+
274+
// Give the server time to process the connection close and remove tools
275+
await new Promise((resolve) => setTimeout(resolve, 500));
276+
277+
// Recreate client2 for cleanup in afterAll (but don't rejoin the session)
278+
client2 = new CopilotClient({ cliUrl: `localhost:${actualPort}` });
279+
280+
// Now only stable_tool should be available
281+
const afterResponse = await session1.sendAndWait({
282+
prompt:
283+
"Use the stable_tool with input 'still_here'. Also try using ephemeral_tool if it is available.",
284+
});
285+
expect(afterResponse?.data.content).toContain("STABLE_still_here");
286+
// ephemeral_tool should NOT have produced a result
287+
expect(afterResponse?.data.content).not.toContain("EPHEMERAL_");
288+
});
197289
});

‎test/snapshots/multi_client/both_clients_see_tool_request_and_completion_events.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ conversations:
77
- role: user
88
content: Use the magic_number tool with seed 'hello' and tell me the result
99
- role: assistant
10-
content: I'll use the magic_number tool with the seed 'hello' for you.
10+
content: I'll use the magic_number tool with seed 'hello' for you.
1111
- role: assistant
1212
tool_calls:
1313
- id: toolcall_0
@@ -28,7 +28,7 @@ conversations:
2828
- role: user
2929
content: Use the magic_number tool with seed 'hello' and tell me the result
3030
- role: assistant
31-
content: I'll use the magic_number tool with the seed 'hello' for you.
31+
content: I'll use the magic_number tool with seed 'hello' for you.
3232
tool_calls:
3333
- id: toolcall_0
3434
type: function
@@ -47,4 +47,4 @@ conversations:
4747
tool_call_id: toolcall_1
4848
content: MAGIC_hello_42
4949
- role: assistant
50-
content: "The magic number tool returned: **MAGIC_hello_42**"
50+
content: The magic number for seed 'hello' is **MAGIC_hello_42**.

‎test/snapshots/multi_client/disconnecting_client_removes_its_tools.yaml‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
models:
2+
- claude-sonnet-4.5
3+
conversations:
4+
- messages:
5+
- role: system
6+
content: ${system}
7+
- role: user
8+
content: Use the stable_tool with input 'test1' and the ephemeral_tool with input 'test2'. Tell me both results.
9+
- role: assistant
10+
content: I'll call both tools for you and report the results.
11+
- role: assistant
12+
tool_calls:
13+
- id: toolcall_0
14+
type: function
15+
function:
16+
name: report_intent
17+
arguments: '{"intent":"Testing stable and ephemeral tools"}'
18+
- role: assistant
19+
tool_calls:
20+
- id: toolcall_1
21+
type: function
22+
function:
23+
name: stable_tool
24+
arguments: '{"input":"test1"}'
25+
- role: assistant
26+
tool_calls:
27+
- id: toolcall_2
28+
type: function
29+
function:
30+
name: ephemeral_tool
31+
arguments: '{"input":"test2"}'
32+
- messages:
33+
- role: system
34+
content: ${system}
35+
- role: user
36+
content: Use the stable_tool with input 'test1' and the ephemeral_tool with input 'test2'. Tell me both results.
37+
- role: assistant
38+
content: I'll call both tools for you and report the results.
39+
tool_calls:
40+
- id: toolcall_0
41+
type: function
42+
function:
43+
name: report_intent
44+
arguments: '{"intent":"Testing stable and ephemeral tools"}'
45+
- id: toolcall_1
46+
type: function
47+
function:
48+
name: stable_tool
49+
arguments: '{"input":"test1"}'
50+
- id: toolcall_2
51+
type: function
52+
function:
53+
name: ephemeral_tool
54+
arguments: '{"input":"test2"}'
55+
- role: tool
56+
tool_call_id: toolcall_0
57+
content: Intent logged
58+
- role: tool
59+
tool_call_id: toolcall_1
60+
content: STABLE_test1
61+
- role: tool
62+
tool_call_id: toolcall_2
63+
content: EPHEMERAL_test2
64+
- role: assistant
65+
content: |-
66+
Both tools executed successfully:
67+
- **stable_tool** returned: `STABLE_test1`
68+
- **ephemeral_tool** returned: `EPHEMERAL_test2`
69+
- role: user
70+
content: >-
71+
<tools_changed_notice>
72+
73+
Tools no longer available: ephemeral_tool
74+
75+
76+
Important: Do not attempt to call tools that are no longer available unless you've been notified that they're
77+
available again.
78+
79+
</tools_changed_notice>
80+
81+
82+
Use the stable_tool with input 'still_here'. Also try using ephemeral_tool if it is available.
83+
- role: assistant
84+
content: I'll call the stable_tool and check on the ephemeral_tool availability.
85+
- role: assistant
86+
tool_calls:
87+
- id: toolcall_3
88+
type: function
89+
function:
90+
name: report_intent
91+
arguments: '{"intent":"Testing tool availability"}'
92+
- role: assistant
93+
tool_calls:
94+
- id: toolcall_4
95+
type: function
96+
function:
97+
name: stable_tool
98+
arguments: '{"input":"still_here"}'
99+
- messages:
100+
- role: system
101+
content: ${system}
102+
- role: user
103+
content: Use the stable_tool with input 'test1' and the ephemeral_tool with input 'test2'. Tell me both results.
104+
- role: assistant
105+
content: I'll call both tools for you and report the results.
106+
tool_calls:
107+
- id: toolcall_0
108+
type: function
109+
function:
110+
name: report_intent
111+
arguments: '{"intent":"Testing stable and ephemeral tools"}'
112+
- id: toolcall_1
113+
type: function
114+
function:
115+
name: stable_tool
116+
arguments: '{"input":"test1"}'
117+
- id: toolcall_2
118+
type: function
119+
function:
120+
name: ephemeral_tool
121+
arguments: '{"input":"test2"}'
122+
- role: tool
123+
tool_call_id: toolcall_0
124+
content: Intent logged
125+
- role: tool
126+
tool_call_id: toolcall_1
127+
content: STABLE_test1
128+
- role: tool
129+
tool_call_id: toolcall_2
130+
content: EPHEMERAL_test2
131+
- role: assistant
132+
content: |-
133+
Both tools executed successfully:
134+
- **stable_tool** returned: `STABLE_test1`
135+
- **ephemeral_tool** returned: `EPHEMERAL_test2`
136+
- role: user
137+
content: >-
138+
<tools_changed_notice>
139+
140+
Tools no longer available: ephemeral_tool
141+
142+
143+
Important: Do not attempt to call tools that are no longer available unless you've been notified that they're
144+
available again.
145+
146+
</tools_changed_notice>
147+
148+
149+
Use the stable_tool with input 'still_here'. Also try using ephemeral_tool if it is available.
150+
- role: assistant
151+
content: I'll call the stable_tool and check on the ephemeral_tool availability.
152+
tool_calls:
153+
- id: toolcall_3
154+
type: function
155+
function:
156+
name: report_intent
157+
arguments: '{"intent":"Testing tool availability"}'
158+
- id: toolcall_4
159+
type: function
160+
function:
161+
name: stable_tool
162+
arguments: '{"input":"still_here"}'
163+
- role: tool
164+
tool_call_id: toolcall_3
165+
content: Intent logged
166+
- role: tool
167+
tool_call_id: toolcall_4
168+
content: STABLE_still_here
169+
- role: assistant
170+
content: >-
171+
**stable_tool** returned: `STABLE_still_here`
172+
173+
174+
The **ephemeral_tool** is no longer available (it was removed as indicated in the tools_changed_notice), so I
175+
couldn't call it.

‎test/snapshots/multi_client/one_client_approves_permission_and_both_see_the_result.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ conversations:
77
- role: user
88
content: Create a file called hello.txt containing the text 'hello world'
99
- role: assistant
10-
content: I'll create the hello.txt file with the text 'hello world' for you.
10+
content: I'll create the hello.txt file with the text 'hello world'.
1111
- role: assistant
1212
tool_calls:
1313
- id: toolcall_0
@@ -28,7 +28,7 @@ conversations:
2828
- role: user
2929
content: Create a file called hello.txt containing the text 'hello world'
3030
- role: assistant
31-
content: I'll create the hello.txt file with the text 'hello world' for you.
31+
content: I'll create the hello.txt file with the text 'hello world'.
3232
tool_calls:
3333
- id: toolcall_0
3434
type: function
@@ -47,4 +47,4 @@ conversations:
4747
tool_call_id: toolcall_1
4848
content: Created file ${workdir}/hello.txt with 11 characters
4949
- role: assistant
50-
content: Done! I've created the file `hello.txt` containing the text 'hello world' in your current directory.
50+
content: Done! I've created the file `hello.txt` containing the text 'hello world'.

‎test/snapshots/multi_client/one_client_rejects_permission_and_both_see_the_result.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,8 @@ conversations:
77
- role: user
88
content: Edit protected.txt and replace 'protected' with 'hacked'.
99
- role: assistant
10-
content: I'll help you edit the file. Let me first view its current contents, then make the change.
10+
content: I'll help you edit protected.txt to replace 'protected' with 'hacked'. Let me first view the file and then make
11+
the change.
1112
- role: assistant
1213
tool_calls:
1314
- id: toolcall_0

0 commit comments

Comments
 (0)