Skip to content

Commit 698be5c

Browse files
committed
feat(realtime): render Manage Mode tool calls in the Talk transcript
Previously the realtime endpoint only emitted response.output_item.added for the FunctionCall item, and Talk.jsx's switch ignored the event — so server-side tool runs were invisible in the UI. The model would speak the result but the user had no way to see what tool was actually called. realtime.go: after executing an assistant tool inproc, emit a second output_item.added/.done pair for the FunctionCallOutput item. Mirrors the way the chat page displays tool_call + tool_result blocks. Talk.jsx: handle both response.output_item.added and .done. Render FunctionCall (with arguments) and FunctionCallOutput (pretty-printed JSON when possible) as two transcript entries — `tool_call` with the wrench icon, `tool_result` with the clipboard icon, both in mono-space secondary-colour. Resets streamingRef after the result so the next assistant text delta starts a fresh transcript entry instead of appending to the previous turn. Assisted-by: claude-code:claude-opus-4-7-1m [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com>
1 parent 9f9d207 commit 698be5c

2 files changed

Lines changed: 59 additions & 10 deletions

File tree

core/http/endpoints/openai/realtime.go

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1800,12 +1800,26 @@ func triggerResponse(ctx context.Context, session *Session, conv *Conversation,
18001800
conv.Lock.Lock()
18011801
conv.Items = append(conv.Items, &foItem)
18021802
conv.Lock.Unlock()
1803+
// First the call, then the output — gives the UI two distinct
1804+
// transcript entries (🔧 calling X / 📋 X returned …).
18031805
sendEvent(t, types.ResponseOutputItemDoneEvent{
18041806
ServerEventBase: types.ServerEventBase{},
18051807
ResponseID: responseID,
18061808
OutputIndex: outputIndex,
18071809
Item: fcItem,
18081810
})
1811+
sendEvent(t, types.ResponseOutputItemAddedEvent{
1812+
ServerEventBase: types.ServerEventBase{},
1813+
ResponseID: responseID,
1814+
OutputIndex: outputIndex,
1815+
Item: foItem,
1816+
})
1817+
sendEvent(t, types.ResponseOutputItemDoneEvent{
1818+
ServerEventBase: types.ServerEventBase{},
1819+
ResponseID: responseID,
1820+
OutputIndex: outputIndex,
1821+
Item: foItem,
1822+
})
18091823
executedAssistantTool = true
18101824
continue
18111825
}

core/http/react-ui/src/pages/Talk.jsx

Lines changed: 45 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -264,6 +264,29 @@ export default function Talk() {
264264
case 'response.output_audio.delta':
265265
updateStatus('speaking', 'Speaking...')
266266
break
267+
case 'response.output_item.added':
268+
case 'response.output_item.done': {
269+
// Server-executed tools (Manage Mode) surface as output items —
270+
// a FunctionCall when the model decides to invoke a tool, and a
271+
// FunctionCallOutput once the server has run it. We only render
272+
// 'done' for the call (so it shows up once with its arguments)
273+
// and 'added' for the output (rendered immediately on arrival).
274+
const item = event.item
275+
if (!item) break
276+
if (event.type === 'response.output_item.done' && item.FunctionCall) {
277+
setTranscript(prev => [...prev, {
278+
role: 'tool_call',
279+
text: `${item.FunctionCall.name}(${item.FunctionCall.arguments || ''})`,
280+
}])
281+
} else if (event.type === 'response.output_item.added' && item.FunctionCallOutput) {
282+
let preview = item.FunctionCallOutput.output || ''
283+
// Pretty-print JSON for readability; fall back to raw string.
284+
try { preview = JSON.stringify(JSON.parse(preview), null, 2) } catch (_) { /* keep raw */ }
285+
setTranscript(prev => [...prev, { role: 'tool_result', text: preview }])
286+
streamingRef.current = null // assistant turn is now fresh after the tool
287+
}
288+
break
289+
}
267290
case 'response.function_call_arguments.done':
268291
// Don't await — keep the event loop free; handleFunctionCall sends
269292
// conversation.item.create + response.create when it's done.
@@ -758,16 +781,28 @@ export default function Talk() {
758781
Conversation will appear here...
759782
</p>
760783
)}
761-
{transcript.map((entry, i) => (
762-
<div key={i} style={{ display: 'flex', alignItems: 'flex-start', gap: 'var(--spacing-xs)' }}>
763-
<i className={entry.role === 'user' ? 'fa-solid fa-user' : 'fa-solid fa-robot'}
764-
style={{
765-
color: entry.role === 'user' ? 'var(--color-primary)' : 'var(--color-accent)',
766-
marginTop: 3, flexShrink: 0, fontSize: '0.75rem',
767-
}} />
768-
<p style={{ margin: 0 }}>{entry.text}</p>
769-
</div>
770-
))}
784+
{transcript.map((entry, i) => {
785+
const isToolCall = entry.role === 'tool_call'
786+
const isToolResult = entry.role === 'tool_result'
787+
const isUser = entry.role === 'user'
788+
const iconClass = isToolCall ? 'fa-solid fa-screwdriver-wrench'
789+
: isToolResult ? 'fa-solid fa-clipboard-list'
790+
: isUser ? 'fa-solid fa-user' : 'fa-solid fa-robot'
791+
const iconColor = isToolCall || isToolResult ? 'var(--color-text-secondary)'
792+
: isUser ? 'var(--color-primary)' : 'var(--color-accent)'
793+
return (
794+
<div key={i} style={{ display: 'flex', alignItems: 'flex-start', gap: 'var(--spacing-xs)' }}>
795+
<i className={iconClass} style={{ color: iconColor, marginTop: 3, flexShrink: 0, fontSize: '0.75rem' }} />
796+
<p style={{
797+
margin: 0,
798+
fontFamily: (isToolCall || isToolResult) ? 'var(--font-mono)' : undefined,
799+
fontSize: (isToolCall || isToolResult) ? '0.8125rem' : undefined,
800+
color: (isToolCall || isToolResult) ? 'var(--color-text-secondary)' : undefined,
801+
whiteSpace: isToolResult ? 'pre-wrap' : undefined,
802+
}}>{entry.text}</p>
803+
</div>
804+
)
805+
})}
771806
<div ref={transcriptEndRef} />
772807
</div>
773808

0 commit comments

Comments
 (0)