Stream AI reasoning as ephemeral thinking chunks#1729
Conversation
Backend emits NDJSON-framed chunks (thinking / thinking_reset / thinking_commit / text) so intermediate tool-loop reasoning is shown transiently in the AI panel and replaced between iterations, leaving only the final answer persistent in the chat history. Also fixes pre-existing biome lint violations in the touched files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
📝 WalkthroughWalkthroughThis pull request refactors AI streaming to emit structured JSON-line events instead of raw text chunks. The backend now sends Changes
Sequence Diagram(s)sequenceDiagram
participant Client as Client Component
participant AiService as AI Service
participant ApiService as API Service
participant Backend as Backend (SSE)
Client->>AiService: sendMessage(query)
AiService->>ApiService: postStream(request)
Backend-->>ApiService: {type:'thinking', content:'...'} (newline)
ApiService->>ApiService: Parse JSON line
ApiService-->>AiService: yield AiStreamChunk
AiService->>Client: Stream chunk
Client->>Client: Accumulate thinking field
Backend-->>ApiService: {type:'thinking_reset'} (newline)
ApiService->>ApiService: Parse JSON line
ApiService-->>AiService: yield AiStreamChunk
Client->>Client: Reset thinking, await tool execution
Backend-->>ApiService: {type:'text', content:'response'} (newline)
ApiService->>ApiService: Parse JSON line
ApiService-->>AiService: yield AiStreamChunk
Client->>Client: Append to text, clear thinking
Backend-->>ApiService: Stream ends
ApiService->>ApiService: Flush buffered tail
ApiService-->>AiService: Final chunks
AiService-->>Client: Stream complete
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR updates the AI request streaming protocol so the backend emits NDJSON-framed chunks that distinguish ephemeral “thinking” from final “text”, and updates the Angular client to parse/render those chunks in the AI panel (showing transient thinking that is reset between tool-loop iterations).
Changes:
- Backend: emit newline-delimited JSON chunks (
thinking,thinking_reset,thinking_commit,text) instead of raw text writes during the tool loop. - Frontend: parse NDJSON stream into typed
AiStreamChunks and update the AI panel to display transient thinking until a commit/final text is produced. - UI/lint touch-ups: button
type="button", interval typing, mermaid typing, and related styling for the thinking display.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| frontend/src/app/services/api.service.ts | Introduces AiStreamChunk and NDJSON parsing in postStream() |
| frontend/src/app/services/ai.service.ts | Propagates stream type change to AsyncGenerator<AiStreamChunk> |
| frontend/src/app/components/dashboard/db-table-view/db-table-ai-panel/db-table-ai-panel.component.ts | Consumes typed chunks and maintains thinking vs text message state |
| frontend/src/app/components/dashboard/db-table-view/db-table-ai-panel/db-table-ai-panel.component.html | Renders transient thinking and adds type="button" to buttons |
| frontend/src/app/components/dashboard/db-table-view/db-table-ai-panel/db-table-ai-panel.component.css | Adds styling for .ai-thinking |
| backend/src/entities/ai/use-cases/request-info-from-table-with-ai-v7.use.case.ts | Writes NDJSON chunks via writeChunk() and introduces thinking reset/commit semantics |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const { done, value } = await reader.read(); | ||
| if (done) break; | ||
| yield decoder.decode(value, { stream: true }); | ||
| if (done) { | ||
| const tail = buffer.trim(); | ||
| if (tail) { | ||
| const parsed = tryParseChunk(tail); | ||
| if (parsed) yield parsed; | ||
| } | ||
| break; | ||
| } |
There was a problem hiding this comment.
In chunkStream(), when done is true you parse the remaining buffer, but you never flush the TextDecoder state. With { stream: true }, multi-byte characters can be buffered internally and will be lost unless you call decoder.decode() once at the end to flush pending bytes before processing the tail.
| const parsed = JSON.parse(line); | ||
| if (parsed && typeof parsed.type === 'string') { | ||
| return parsed as AiStreamChunk; | ||
| } |
There was a problem hiding this comment.
tryParseChunk currently accepts any JSON object with a string type and then casts it to AiStreamChunk. This can yield invalid shapes at runtime (e.g., {type:"thinking"} without content, or unknown type values), which will later be treated as valid and can append undefined into the UI. Consider validating type against the allowed set and, for thinking/text, also validating typeof content === 'string' before returning.
| const parsed = JSON.parse(line); | |
| if (parsed && typeof parsed.type === 'string') { | |
| return parsed as AiStreamChunk; | |
| } | |
| const parsed: unknown = JSON.parse(line); | |
| if (!parsed || typeof parsed !== 'object') { | |
| return null; | |
| } | |
| const chunk = parsed as { type?: unknown; content?: unknown }; | |
| switch (chunk.type) { | |
| case 'thinking': | |
| return typeof chunk.content === 'string' | |
| ? { type: 'thinking', content: chunk.content } | |
| : null; | |
| case 'thinking_reset': | |
| return { type: 'thinking_reset' }; | |
| case 'thinking_commit': | |
| return { type: 'thinking_commit' }; | |
| case 'text': | |
| return typeof chunk.content === 'string' | |
| ? { type: 'text', content: chunk.content } | |
| : null; | |
| default: | |
| return null; | |
| } |
| | { type: 'thinking_commit' } | ||
| | { type: 'text'; content: string }, | ||
| ): void { | ||
| response.write(JSON.stringify(chunk) + '\n'); |
There was a problem hiding this comment.
The stream is now NDJSON-framed (JSON.stringify(chunk) + "\n"), but the response headers are still configured as text/event-stream in setupResponseHeaders(). This content-type mismatch can confuse intermediaries/clients and is not a valid SSE payload format. Please update the headers to reflect NDJSON (e.g., application/x-ndjson; charset=utf-8) or change the payload to proper SSE framing if SSE is intended.
| response.write(JSON.stringify(chunk) + '\n'); | |
| response.write(`data: ${JSON.stringify(chunk)}\n\n`); |
Update mockStream to yield AiStreamChunk objects instead of raw strings so _consumeStream consumes them correctly, and align the expected AI message with the component's thinking field after #1729. Also add pnpm overrides for fast-xml-parser and uuid to fix advisories. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backend emits NDJSON-framed chunks (thinking / thinking_reset / thinking_commit / text) so intermediate tool-loop reasoning is shown transiently in the AI panel and replaced between iterations, leaving only the final answer persistent in the chat history. Also fixes pre-existing biome lint violations in the touched files.
Summary by CodeRabbit
New Features
Improvements