Skip to content

Commit d54ddfb

Browse files
stephentoubCopilot
andcommitted
Make session_todos_changed deterministic via single sql call
The test asked the model to run three FK-dependent INSERTs. The model issued them as three parallel sql tool calls; their execution order is not deterministic, so on some platforms the todo_deps INSERT raced ahead of its parent todos rows and hit 'FOREIGN KEY constraint failed', diverging from the recorded snapshot (cache miss on macOS CI). Reword the prompt to ask for a single sql tool call containing all three statements in order. Statements within one call run sequentially, so the FK constraint is always satisfied and the snapshot replays deterministically on all platforms. Re-recorded the snapshot accordingly. Verified stable across 3 replay runs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 15f3461 commit d54ddfb

2 files changed

Lines changed: 37 additions & 47 deletions

File tree

nodejs/test/e2e/session_todos_changed.e2e.test.ts

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,9 @@ describe("Todos changed event + readSqlTodosWithDependencies", async () => {
3333

3434
await session.sendAndWait({
3535
prompt:
36-
"Use the sql tool to execute exactly these statements, in order, with no extra rows:\n" +
36+
"Use the sql tool exactly once to execute all three of the following statements " +
37+
"together, in this exact order, in a single sql tool call (a single query string " +
38+
"containing all three statements):\n" +
3739
"1. INSERT INTO todos (id, title, status) VALUES ('alpha', 'First todo', 'pending');\n" +
3840
"2. INSERT INTO todos (id, title, status) VALUES ('beta', 'Second todo', 'done');\n" +
3941
"3. INSERT INTO todo_deps (todo_id, depends_on) VALUES ('beta', 'alpha');\n" +

test/snapshots/session_todos_changed/fires_session_todos_changed_and_exposes_rows_and_dependencies.yaml

Lines changed: 34 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -5,77 +5,65 @@ conversations:
55
- role: system
66
content: ${system}
77
- role: user
8-
content: |-
9-
Use the sql tool to execute exactly these statements, in order, with no extra rows:
8+
content: >-
9+
Use the sql tool exactly once to execute all three of the following statements together, in this exact order,
10+
in a single sql tool call (a single query string containing all three statements):
11+
1012
1. INSERT INTO todos (id, title, status) VALUES ('alpha', 'First todo', 'pending');
13+
1114
2. INSERT INTO todos (id, title, status) VALUES ('beta', 'Second todo', 'done');
15+
1216
3. INSERT INTO todo_deps (todo_id, depends_on) VALUES ('beta', 'alpha');
17+
1318
Then stop. Do not insert any other rows or create any other tables.
1419
- role: assistant
15-
content: I'll execute those three SQL statements in order.
20+
content: "I'll execute all three SQL statements together in a single call:"
1621
- role: assistant
1722
tool_calls:
1823
- id: toolcall_0
1924
type: function
2025
function:
2126
name: sql
22-
arguments: "{\"description\":\"Insert alpha todo\",\"query\":\"INSERT INTO todos (id, title, status) VALUES ('alpha',
23-
'First todo', 'pending');\"}"
24-
- role: assistant
25-
tool_calls:
26-
- id: toolcall_1
27-
type: function
28-
function:
29-
name: sql
30-
arguments: "{\"description\":\"Insert beta todo\",\"query\":\"INSERT INTO todos (id, title, status) VALUES ('beta',
31-
'Second todo', 'done');\"}"
32-
- role: assistant
33-
tool_calls:
34-
- id: toolcall_2
35-
type: function
36-
function:
37-
name: sql
38-
arguments: "{\"description\":\"Insert dependency\",\"query\":\"INSERT INTO todo_deps (todo_id, depends_on) VALUES
39-
('beta', 'alpha');\"}"
27+
arguments: "{\"description\":\"Insert todos and dependency\",\"query\":\"INSERT INTO todos (id, title, status) VALUES
28+
('alpha', 'First todo', 'pending');\\nINSERT INTO todos (id, title, status) VALUES ('beta', 'Second
29+
todo', 'done');\\nINSERT INTO todo_deps (todo_id, depends_on) VALUES ('beta', 'alpha');\"}"
4030
- messages:
4131
- role: system
4232
content: ${system}
4333
- role: user
44-
content: |-
45-
Use the sql tool to execute exactly these statements, in order, with no extra rows:
34+
content: >-
35+
Use the sql tool exactly once to execute all three of the following statements together, in this exact order,
36+
in a single sql tool call (a single query string containing all three statements):
37+
4638
1. INSERT INTO todos (id, title, status) VALUES ('alpha', 'First todo', 'pending');
39+
4740
2. INSERT INTO todos (id, title, status) VALUES ('beta', 'Second todo', 'done');
41+
4842
3. INSERT INTO todo_deps (todo_id, depends_on) VALUES ('beta', 'alpha');
43+
4944
Then stop. Do not insert any other rows or create any other tables.
5045
- role: assistant
51-
content: I'll execute those three SQL statements in order.
46+
content: "I'll execute all three SQL statements together in a single call:"
5247
tool_calls:
5348
- id: toolcall_0
5449
type: function
5550
function:
5651
name: sql
57-
arguments: "{\"description\":\"Insert alpha todo\",\"query\":\"INSERT INTO todos (id, title, status) VALUES ('alpha',
58-
'First todo', 'pending');\"}"
59-
- id: toolcall_1
60-
type: function
61-
function:
62-
name: sql
63-
arguments: "{\"description\":\"Insert beta todo\",\"query\":\"INSERT INTO todos (id, title, status) VALUES ('beta',
64-
'Second todo', 'done');\"}"
65-
- id: toolcall_2
66-
type: function
67-
function:
68-
name: sql
69-
arguments: "{\"description\":\"Insert dependency\",\"query\":\"INSERT INTO todo_deps (todo_id, depends_on) VALUES
70-
('beta', 'alpha');\"}"
52+
arguments: "{\"description\":\"Insert todos and dependency\",\"query\":\"INSERT INTO todos (id, title, status) VALUES
53+
('alpha', 'First todo', 'pending');\\nINSERT INTO todos (id, title, status) VALUES ('beta', 'Second
54+
todo', 'done');\\nINSERT INTO todo_deps (todo_id, depends_on) VALUES ('beta', 'alpha');\"}"
7155
- role: tool
7256
tool_call_id: toolcall_0
73-
content: "1 row(s) inserted. Last inserted row ID: 1."
74-
- role: tool
75-
tool_call_id: toolcall_1
76-
content: "1 row(s) inserted. Last inserted row ID: 2."
77-
- role: tool
78-
tool_call_id: toolcall_2
79-
content: "1 row(s) inserted. Last inserted row ID: 1."
57+
content: |-
58+
Executed 3 statement(s):
59+
60+
1. INSERT
61+
1 row(s) inserted. Last inserted row ID: 1.
62+
63+
2. INSERT
64+
1 row(s) inserted. Last inserted row ID: 2.
65+
66+
3. INSERT
67+
1 row(s) inserted. Last inserted row ID: 1.
8068
- role: assistant
81-
content: Done. All three statements executed successfully.
69+
content: Done. All three statements executed successfully in a single SQL call.

0 commit comments

Comments
 (0)