Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions packages/cli/src/ui/hooks/useGeminiStream.ts
Original file line number Diff line number Diff line change
Expand Up @@ -444,7 +444,7 @@ export const useGeminiStream = (
isClientInitiated: true,
prompt_id,
};
scheduleToolCalls([toolCallRequest], abortSignal);
await scheduleToolCalls([toolCallRequest], abortSignal);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-critical critical

A critical prompt injection vulnerability exists here. User input is passed to the Gemini model without sanitization, allowing an attacker to manipulate the LLM's behavior and execute arbitrary tool calls, potentially leading to remote code execution. This line executes a tool call originating from a slash command, which is derived from user input. Beyond the security concern, the scheduleToolCalls function is asynchronous and must be awaited to prevent race conditions and ensure proper application flow. Remediation for the vulnerability includes strict input validation and sanitization, implementing an allow-list for tools with schema validation, and requiring user confirmation for sensitive tool calls. The suggested code addresses the asynchronous call.

Suggested change
await scheduleToolCalls([toolCallRequest], abortSignal);
await scheduleToolCalls([toolCallRequest], abortSignal);

return { queryToSend: null, shouldProceed: false };
}
case 'submit_prompt': {
Expand Down Expand Up @@ -911,7 +911,7 @@ export const useGeminiStream = (
}
}
if (toolCallRequests.length > 0) {
scheduleToolCalls(toolCallRequests, signal);
await scheduleToolCalls(toolCallRequests, signal);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-critical critical

A critical prompt injection vulnerability exists here. User input is passed to the Gemini model without sanitization, allowing an attacker to manipulate the LLM's behavior and execute arbitrary tool calls, potentially leading to remote code execution. This line executes tool calls requested by the LLM in its response to a user's prompt. Beyond the security concern, scheduleToolCalls is an asynchronous operation and must be awaited to prevent incorrect state management (e.g., StreamProcessingStatus.Completed returning prematurely). Remediation for the vulnerability includes strict input validation and sanitization, implementing an allow-list for tools with schema validation, and requiring user confirmation for sensitive tool calls. The suggested code addresses the asynchronous call.

Suggested change
await scheduleToolCalls(toolCallRequests, signal);
await scheduleToolCalls(toolCallRequests, signal);

}
return StreamProcessingStatus.Completed;
},
Expand Down
4 changes: 2 additions & 2 deletions packages/cli/src/ui/hooks/useReactToolScheduler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ import { ToolCallStatus } from '../types.js';
export type ScheduleFn = (
request: ToolCallRequestInfo | ToolCallRequestInfo[],
signal: AbortSignal,
) => void;
) => Promise<void>;
export type MarkToolsAsSubmittedFn = (callIds: string[]) => void;

export type TrackedScheduledToolCall = ScheduledToolCall & {
Expand Down Expand Up @@ -180,7 +180,7 @@ export function useReactToolScheduler(
signal: AbortSignal,
) => {
setToolCallsForDisplay([]);
void scheduler.schedule(request, signal);
return scheduler.schedule(request, signal);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Previously, void scheduler.schedule explicitly ignored the returned Promise. Since scheduler.schedule is now asynchronous and returns a Promise, it's important to return this Promise. This allows the caller to await the scheduling operation, ensuring proper asynchronous flow and error handling.

Suggested change
return scheduler.schedule(request, signal);
return scheduler.schedule(request, signal);

},
[scheduler, setToolCallsForDisplay],
);
Expand Down
130 changes: 82 additions & 48 deletions packages/cli/src/ui/hooks/useToolScheduler.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -163,8 +163,8 @@ describe('useReactToolScheduler in YOLO Mode', () => {
args: { data: 'any data' },
} as any;

act(() => {
schedule(request, new AbortController().signal);
await act(async () => {
await schedule(request, new AbortController().signal);
Comment on lines +166 to +167

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When testing asynchronous operations in React components, it's crucial to wrap act calls that involve Promises with await act(async () => { await ... }). This ensures that all state updates and side effects triggered by the asynchronous operation are fully processed before the test continues, preventing flaky tests or incorrect assertions.

Suggested change
await act(async () => {
await schedule(request, new AbortController().signal);
await act(async () => {
await schedule(request, new AbortController().signal);
});

});

await act(async () => {
Expand Down Expand Up @@ -220,11 +220,11 @@ describe('useReactToolScheduler', () => {
schedule: (
req: ToolCallRequestInfo | ToolCallRequestInfo[],
signal: AbortSignal,
) => void,
) => Promise<void>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The type definition for schedule in the test utility function scheduleAndWaitForExecution must correctly reflect its asynchronous nature, returning Promise<void>. This aligns with the updated ScheduleFn type and ensures type safety within the test.

Suggested change
) => Promise<void>,
) => Promise<void>,

request: ToolCallRequestInfo | ToolCallRequestInfo[],
) => {
act(() => {
schedule(request, new AbortController().signal);
await act(async () => {
await schedule(request, new AbortController().signal);
Comment on lines +226 to +227

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Similar to other test updates, the schedule call within act must be awaited, and the act block itself should be async to correctly handle the Promise returned by schedule. This ensures the test waits for the scheduling operation to complete.

Suggested change
await act(async () => {
await schedule(request, new AbortController().signal);
await act(async () => {
await schedule(request, new AbortController().signal);
});

});

await advanceAndSettle();
Expand Down Expand Up @@ -313,10 +313,13 @@ describe('useReactToolScheduler', () => {

it('should clear previous tool calls when scheduling new ones', async () => {
mockToolRegistry.getTool.mockReturnValue(mockTool);
(mockTool.execute as Mock).mockResolvedValue({
llmContent: 'Tool output',
returnDisplay: 'Formatted tool output',
} as ToolResult);
(mockTool.execute as Mock).mockImplementation(async () => {
await new Promise((r) => setTimeout(r, 10));
return {
llmContent: 'Tool output',
returnDisplay: 'Formatted tool output',
};
Comment on lines +316 to +321

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Replacing mockResolvedValue with mockImplementation that includes a setTimeout is a significant improvement. This accurately simulates a real asynchronous operation, making the test more robust and reliable when verifying the behavior of the tool scheduler, especially in scenarios involving concurrent or sequential tool calls.

Suggested change
(mockTool.execute as Mock).mockImplementation(async () => {
await new Promise((r) => setTimeout(r, 10));
return {
llmContent: 'Tool output',
returnDisplay: 'Formatted tool output',
};
(mockTool.execute as Mock).mockImplementation(async () => {
await new Promise((r) => setTimeout(r, 10));
return {
llmContent: 'Tool output',
returnDisplay: 'Formatted tool output',
};
});

});

const { result } = renderScheduler();
const schedule = result.current[1];
Expand All @@ -337,10 +340,13 @@ describe('useReactToolScheduler', () => {
name: 'mockTool',
args: {},
} as any;
act(() => {
schedule(newRequest, new AbortController().signal);
let schedulePromise: Promise<void>;
await act(async () => {
schedulePromise = schedule(newRequest, new AbortController().signal);
});
Comment on lines +343 to 346

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Capturing the schedule promise and awaiting the act block is essential for correctly testing asynchronous flows. This ensures that the test waits for the schedule operation to initiate its effects before proceeding to assertions, preventing race conditions and ensuring accurate state checks.

Suggested change
let schedulePromise: Promise<void>;
await act(async () => {
schedulePromise = schedule(newRequest, new AbortController().signal);
});
let schedulePromise: Promise<void>;
await act(async () => {
schedulePromise = schedule(newRequest, new AbortController().signal);
});


await advanceAndSettle();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Adding await advanceAndSettle() here is crucial. It ensures that all microtasks, including the initial processing triggered by schedule, are completed before the test proceeds to check the intermediate state of result.current[0]. This prevents assertions from being made on an unsettled state.

Suggested change
await advanceAndSettle();
await advanceAndSettle();


// After scheduling, the old call should be gone,
// and the new one should be in the display in its initial state.
expect(result.current[0].length).toBe(1);
Expand All @@ -349,14 +355,13 @@ describe('useReactToolScheduler', () => {

// Let the new call finish.
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
await vi.advanceTimersByTimeAsync(20);
Comment on lines 357 to +358

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The vi.advanceTimersByTimeAsync(20) call is a good adjustment to account for the setTimeout(r, 10) in the mock execute function, ensuring enough time passes for the mock tool to complete its simulated work.

Suggested change
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
await vi.advanceTimersByTimeAsync(20);
await act(async () => {
await vi.advanceTimersByTimeAsync(20);
});

});

await act(async () => {
await vi.advanceTimersByTimeAsync(0);
});
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
await schedulePromise;
});
Comment on lines 361 to 363

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Explicitly awaiting schedulePromise is the most robust way to ensure that the entire scheduling and execution flow, including the onComplete callback, has finished. This prevents the test from ending prematurely and ensures all side effects are observed.

Suggested change
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
});
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
await schedulePromise;
});
await act(async () => {
await schedulePromise;
});


expect(onComplete).toHaveBeenCalled();
});

Expand All @@ -379,16 +384,14 @@ describe('useReactToolScheduler', () => {
args: {},
} as any;

act(() => {
schedule(request, new AbortController().signal);
});
let schedulePromise: Promise<void>;
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
}); // validation
await act(async () => {
await vi.advanceTimersByTimeAsync(0); // Process scheduling
schedulePromise = schedule(request, new AbortController().signal);
});
Comment on lines +387 to 390

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Capturing schedulePromise and wrapping the schedule call in await act(async () => { ... }) is the correct pattern for handling asynchronous operations in tests. This ensures the test correctly waits for the scheduling process to begin.

Suggested change
let schedulePromise: Promise<void>;
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
}); // validation
await act(async () => {
await vi.advanceTimersByTimeAsync(0); // Process scheduling
schedulePromise = schedule(request, new AbortController().signal);
});
let schedulePromise: Promise<void>;
await act(async () => {
schedulePromise = schedule(request, new AbortController().signal);
});


await advanceAndSettle(); // validation
await advanceAndSettle(); // Process scheduling
Comment on lines +392 to +393

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using advanceAndSettle() for advancing timers and settling promises is a good practice, making the test more readable and reliable than multiple vi.advanceTimersByTimeAsync(0) calls.

Suggested change
await advanceAndSettle(); // validation
await advanceAndSettle(); // Process scheduling
await advanceAndSettle(); // validation
await advanceAndSettle(); // Process scheduling


// At this point, the tool is 'executing' and waiting on the promise.
expect(result.current[0][0].status).toBe('executing');

Expand All @@ -397,9 +400,7 @@ describe('useReactToolScheduler', () => {
cancelAllToolCalls(cancelController.signal);
});

await act(async () => {
await vi.advanceTimersByTimeAsync(0);
});
await advanceAndSettle();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Consistent use of advanceAndSettle() for advancing timers and settling promises improves the clarity and reliability of the test.

Suggested change
await advanceAndSettle();
await advanceAndSettle();


expect(onComplete).toHaveBeenCalledWith([
expect.objectContaining({
Expand All @@ -412,6 +413,11 @@ describe('useReactToolScheduler', () => {
await act(async () => {
resolveExecute({ llmContent: 'output', returnDisplay: 'display' });
});

// Now await the schedule promise
await act(async () => {
await schedulePromise;
});
Comment on lines +417 to +420

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

It's crucial to await schedulePromise after resolveExecute to ensure that all subsequent asynchronous effects, such as onComplete being called with the cancelled status, have fully settled before the test concludes. This prevents potential race conditions.

await act(async () => {
      await schedulePromise;
    });

});

it.each([
Expand Down Expand Up @@ -511,8 +517,9 @@ describe('useReactToolScheduler', () => {
args: { data: 'sensitive' },
} as any;

act(() => {
schedule(request, new AbortController().signal);
let schedulePromise: Promise<void>;
await act(async () => {
schedulePromise = schedule(request, new AbortController().signal);
});
Comment on lines +520 to 523

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Capturing the schedule promise and awaiting the act block is essential for correctly testing asynchronous flows, especially when dealing with user confirmation steps. This ensures the test waits for the scheduling operation to initiate its effects.

let schedulePromise: Promise<void>;
    await act(async () => {
      schedulePromise = schedule(request, new AbortController().signal);
    });

await advanceAndSettle();

Expand All @@ -525,10 +532,13 @@ describe('useReactToolScheduler', () => {
await capturedOnConfirmForTest?.(ToolConfirmationOutcome.ProceedOnce);
});

await advanceAndSettle();
await advanceAndSettle();
await advanceAndSettle();

// Now await the schedule promise as it should complete
await act(async () => {
await schedulePromise;
});
Comment on lines +538 to +540

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

After the confirmation is handled, awaiting schedulePromise is necessary to ensure that the tool execution, triggered by the confirmation, has fully completed before making assertions. This guarantees the test observes the final state.

await act(async () => {
      await schedulePromise;
    });


expect(mockOnUserConfirmForToolConfirmation).toHaveBeenCalledWith(
ToolConfirmationOutcome.ProceedOnce,
);
Expand Down Expand Up @@ -558,8 +568,9 @@ describe('useReactToolScheduler', () => {
args: {},
} as any;

act(() => {
schedule(request, new AbortController().signal);
let schedulePromise: Promise<void>;
await act(async () => {
schedulePromise = schedule(request, new AbortController().signal);
});
Comment on lines +571 to 574

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Capturing the schedule promise and awaiting the act block is essential for correctly testing asynchronous flows, especially when dealing with user confirmation steps. This ensures the test waits for the scheduling operation to initiate its effects.

let schedulePromise: Promise<void>;
    await act(async () => {
      schedulePromise = schedule(request, new AbortController().signal);
    });

await advanceAndSettle();

Expand All @@ -571,8 +582,13 @@ describe('useReactToolScheduler', () => {
await act(async () => {
await capturedOnConfirmForTest?.(ToolConfirmationOutcome.Cancel);
});

await advanceAndSettle();
await advanceAndSettle();

// Now await the schedule promise
await act(async () => {
await schedulePromise;
});
Comment on lines +589 to +591

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

After the cancellation is handled, awaiting schedulePromise is necessary to ensure that the tool execution, now cancelled, has fully completed its lifecycle before making assertions. This guarantees the test observes the final state.

await act(async () => {
      await schedulePromise;
    });


expect(mockOnUserConfirmForToolConfirmation).toHaveBeenCalledWith(
ToolConfirmationOutcome.Cancel,
Expand Down Expand Up @@ -619,8 +635,12 @@ describe('useReactToolScheduler', () => {
args: {},
} as any;

act(() => {
result.current[1](request, new AbortController().signal);
let schedulePromise: Promise<void>;
await act(async () => {
schedulePromise = result.current[1](
request,
new AbortController().signal,
);
Comment on lines +638 to +643

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Capturing the schedule promise and awaiting the act block is essential for correctly testing asynchronous flows, especially when dealing with live output updates. This ensures the test waits for the scheduling operation to initiate its effects.

let schedulePromise: Promise<void>;
    await act(async () => {
      schedulePromise = result.current[1](
        request,
        new AbortController().signal,
      );
    });

});
await advanceAndSettle();

Expand All @@ -644,7 +664,11 @@ describe('useReactToolScheduler', () => {
} as ToolResult);
});
await advanceAndSettle();
await advanceAndSettle();

// Now await schedule
await act(async () => {
await schedulePromise;
});
Comment on lines +668 to +671

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

After the mock tool's execution is resolved, awaiting schedulePromise is critical to ensure that all subsequent asynchronous effects, including the onComplete callback, have fully settled before the test makes its final assertions. This prevents race conditions and ensures test accuracy.

await act(async () => {
      await schedulePromise;
    });


const completedCalls = onComplete.mock.calls[0][0] as ToolCall[];
expect(completedCalls[0].status).toBe('success');
Expand Down Expand Up @@ -690,8 +714,8 @@ describe('useReactToolScheduler', () => {
{ callId: 'multi2', name: 'tool2', args: { p: 2 } } as any,
];

act(() => {
schedule(requests, new AbortController().signal);
await act(async () => {
await schedule(requests, new AbortController().signal);
Comment on lines +717 to +718

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When scheduling multiple tool calls, it's important to await the schedule function within an async act block to ensure that the scheduling process for all requests has been initiated and any immediate effects are processed before further test steps.

await act(async () => {
      await schedule(requests, new AbortController().signal);
    });

});
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
Expand Down Expand Up @@ -782,38 +806,48 @@ describe('useReactToolScheduler', () => {
args: {},
} as any;

act(() => {
schedule(request1, new AbortController().signal);
let schedulePromise1: Promise<void>;
let schedulePromise2: Promise<void>;
Comment on lines +809 to +810

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Capturing the promises for each schedule call is essential when testing sequential asynchronous operations. This allows the test to explicitly await each operation's completion at the appropriate time.

let schedulePromise1: Promise<void>;
    let schedulePromise2: Promise<void>;


await act(async () => {
schedulePromise1 = schedule(request1, new AbortController().signal);
});
Comment on lines +812 to 814

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Each schedule call must be awaited within an async act block to ensure that the scheduling operation is properly initiated and its immediate effects are processed before the test proceeds.

await act(async () => {
      schedulePromise1 = schedule(request1, new AbortController().signal);
    });

await act(async () => {
await vi.advanceTimersByTimeAsync(0);
});

act(() => {
schedule(request2, new AbortController().signal);
await act(async () => {
schedulePromise2 = schedule(request2, new AbortController().signal);
});
Comment on lines +819 to 821

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Similarly, the second schedule call needs to be awaited within an async act block to ensure its initiation is handled correctly.

await act(async () => {
      schedulePromise2 = schedule(request2, new AbortController().signal);
    });


await act(async () => {
await vi.advanceTimersByTimeAsync(50);
await vi.advanceTimersByTimeAsync(0);
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
});
});

// Wait for first to complete
await act(async () => {
await schedulePromise1;
});
Comment on lines +828 to +831

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Explicitly awaiting schedulePromise1 ensures that the first scheduled operation has fully completed, including its execution and any subsequent callbacks, before the test asserts its outcome. This is critical for testing sequential asynchronous flows.

await act(async () => {
      await schedulePromise1;
    });


expect(onComplete).toHaveBeenCalledWith([
expect.objectContaining({
status: 'success',
request: request1,
response: expect.objectContaining({ resultDisplay: 'done display' }),
}),
]);

await act(async () => {
await vi.advanceTimersByTimeAsync(50);
await vi.advanceTimersByTimeAsync(0);
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
});
});

// Wait for second to complete
await act(async () => {
await schedulePromise2;
});
Comment on lines +847 to +849

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Explicitly awaiting schedulePromise2 ensures that the second scheduled operation has fully completed before the test asserts its outcome, maintaining the correctness of the sequential test flow.

await act(async () => {
      await schedulePromise2;
    });


expect(onComplete).toHaveBeenCalledWith([
expect.objectContaining({
status: 'success',
Expand Down