Skip to content

Commit 924ebbd

Browse files
fix: write blob image to disk so model can view it
The blob attachment test was sending an inline image but the model would try to view it on disk, resulting in 'file not found' errors in the snapshot. Write the PNG to workDir so the view tool succeeds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 241b8d8 commit 924ebbd

File tree

2 files changed

+14
-86
lines changed

2 files changed

+14
-86
lines changed

nodejs/test/e2e/session_config.test.ts

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -44,16 +44,21 @@ describe("Session Configuration", async () => {
4444
});
4545

4646
it("should accept blob attachments", async () => {
47+
// Write the image to disk so the model can view it if it tries
48+
const pngBase64 =
49+
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==";
50+
await writeFile(join(workDir, "pixel.png"), Buffer.from(pngBase64, "base64"));
51+
4752
const session = await client.createSession({ onPermissionRequest: approveAll });
4853

4954
await session.sendAndWait({
50-
prompt: "Describe this image",
55+
prompt: "What color is this pixel? Reply in one word.",
5156
attachments: [
5257
{
5358
type: "blob",
54-
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==",
59+
data: pngBase64,
5560
mimeType: "image/png",
56-
displayName: "test-pixel.png",
61+
displayName: "pixel.png",
5762
},
5863
],
5964
});

test/snapshots/session_config/should_accept_blob_attachments.yaml

Lines changed: 6 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -6,99 +6,22 @@ conversations:
66
content: ${system}
77
- role: user
88
content: |-
9-
Describe this image
10-
test-pixel.png
9+
What color is this pixel? Reply in one word.
10+
pixel.png
1111
[image]
1212
- role: assistant
13-
content: I'll view the image for you.
14-
- role: assistant
15-
tool_calls:
16-
- id: toolcall_0
17-
type: function
18-
function:
19-
name: report_intent
20-
arguments: '{"intent":"Viewing image file"}'
21-
- role: assistant
22-
tool_calls:
23-
- id: toolcall_1
24-
type: function
25-
function:
26-
name: view
27-
arguments: '{"path":"${workdir}/test-pixel.png"}'
28-
- messages:
29-
- role: system
30-
content: ${system}
31-
- role: user
32-
content: |-
33-
Describe this image
34-
test-pixel.png
35-
[image]
36-
- role: assistant
37-
content: I'll view the image for you.
3813
tool_calls:
3914
- id: toolcall_0
40-
type: function
41-
function:
42-
name: report_intent
43-
arguments: '{"intent":"Viewing image file"}'
44-
- id: toolcall_1
4515
type: function
4616
function:
4717
name: view
48-
arguments: '{"path":"${workdir}/test-pixel.png"}'
18+
arguments: '{"path":"${workdir}/pixel.png"}'
4919
- role: tool
5020
tool_call_id: toolcall_0
51-
content: Intent logged
52-
- role: tool
53-
tool_call_id: toolcall_1
54-
content: Path ${workdir}/test-pixel.png does not exist. Please provide a valid path.
55-
- role: assistant
56-
content: Let me check the current directory to locate the image file.
57-
- role: assistant
58-
tool_calls:
59-
- id: toolcall_2
60-
type: function
61-
function:
62-
name: ${shell}
63-
arguments: '{"command":"Get-ChildItem -Recurse -Filter \"test-pixel.png\"","description":"Find test-pixel.png file"}'
64-
- messages:
65-
- role: system
66-
content: ${system}
21+
content: Viewed image file successfully.
6722
- role: user
6823
content: |-
69-
Describe this image
70-
test-pixel.png
24+
Image file at path ${workdir}/pixel.png
7125
[image]
7226
- role: assistant
73-
content: I'll view the image for you.
74-
tool_calls:
75-
- id: toolcall_0
76-
type: function
77-
function:
78-
name: report_intent
79-
arguments: '{"intent":"Viewing image file"}'
80-
- id: toolcall_1
81-
type: function
82-
function:
83-
name: view
84-
arguments: '{"path":"${workdir}/test-pixel.png"}'
85-
- role: tool
86-
tool_call_id: toolcall_0
87-
content: Intent logged
88-
- role: tool
89-
tool_call_id: toolcall_1
90-
content: Path ${workdir}/test-pixel.png does not exist. Please provide a valid path.
91-
- role: assistant
92-
content: Let me check the current directory to locate the image file.
93-
tool_calls:
94-
- id: toolcall_2
95-
type: function
96-
function:
97-
name: ${shell}
98-
arguments: '{"command":"Get-ChildItem -Recurse -Filter \"test-pixel.png\"","description":"Find test-pixel.png file"}'
99-
- role: tool
100-
tool_call_id: toolcall_2
101-
content: <exited with exit code 0>
102-
- role: assistant
103-
content: The file doesn't appear to be in the working directory. Could you please upload or provide the image file
104-
again? I'll be happy to describe it once I can access it.
27+
content: Red

0 commit comments

Comments
 (0)