refactor: Streaming Chat API for Structured Responses and Citation Handling

[Pavan-Microsoft]

Purpose

This pull request refactors and improves the chart handling and streaming logic in the Chat.tsx component. The changes introduce more robust parsing for chart responses and citations, unify chart result handling, and clean up the streaming code to better support both automatic and manual chart generation. The update also modifies type definitions to support the new streaming format.

Chart response and streaming improvements:

• Added parseChartResponse and handleChartResult helper functions to centralize and streamline chart response parsing and result handling, including error management and UI updates for both automatic and manual chart generation. [1] [2] [3]
• Refactored chart API request logic (makeApiRequestForChart) to distinguish between automatic and manual chart generation, suppressing UI updates and error messages for automatic requests. [1] [2] [3] [4] [5] [6] [7]

Streaming and citation parsing enhancements:

• Updated streaming response parsing to accumulate assistant and tool delta chunks, properly handle citations, and improve message updates during streaming. [1] [2] [3]
• Improved citation parsing logic to handle arrays and objects, ensuring robust extraction of citation data from streamed tool messages.

Type definition updates:

• Modified ParsedChunk type in AppTypes.ts to support both messages and delta formats, reflecting the new streaming structure.
• Updated import statements to use the new Citation type instead of ToolMessageContent.

Minor backend cleanup:

• Removed an unused variable (first_chunk) from citation processing in chat_service.py.

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

[Copilot]

Pull request overview

This PR refactors the chat streaming contract to use structured delta chunks (role/content) and updates the frontend to parse streamed citations as dedicated tool messages, alongside consolidating chart parsing/handling logic in Chat.tsx.

Changes:

• Backend streaming now yields (role, content) tuples and stream_chat_request emits choices[0].delta chunks (assistant text + tool citations).
• Frontend streaming parser updated to accumulate assistant deltas and handle citations from role: "tool" deltas; chart parsing/result handling is centralized via helpers.
• Types updated to support both legacy messages and new delta formats; tests updated for the new streaming shape.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
src/tests/api/services/test_chat_service.py	Updates stream test expectations to validate delta chunks and tool citation deltas.
src/api/services/chat_service.py	Changes streaming output from embedded JSON-in-text to (role, content) tuples and emits JSON-lines delta chunks.
src/App/src/types/AppTypes.ts	Expands ParsedChunk typing to support delta-based streaming.
src/App/src/components/Chat/Chat.tsx	Refactors chart parsing/handling and updates streaming parsing to accumulate assistant deltas and process tool citation deltas.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Refactors the chat streaming contract to use structured delta chunks (including a dedicated tool chunk for citations) and updates the frontend streaming/chart logic to parse and display these responses more robustly.

Changes:

• Backend: stream_openai_text now yields (role, content) tuples and stream_chat_request emits JSON-lines chunks with choices[0].delta.
• Frontend: streaming parser now accumulates assistant deltas and captures citations from tool deltas; chart parsing/handling is centralized via helper functions.
• Types/tests: updates ParsedChunk to support both messages and delta formats and adjusts Python tests accordingly.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
src/App/src/components/Chat/Chat.tsx	Adds chart parsing/handling helpers, updates streaming parsing for delta + tool citations, and changes auto-chart behavior.
src/App/src/types/AppTypes.ts	Expands ParsedChunk typing to support the new streaming delta format (and legacy messages).
src/api/services/chat_service.py	Changes streaming generator to yield (role, content) and wraps chunks into choices[0].delta JSON-lines.
src/tests/api/services/test_chat_service.py	Updates tests to match tuple-based streaming and delta chunk structure.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

This PR refactors the chat streaming contract and frontend streaming parsers to support structured delta chunks with separate tool-delivered citations, while also consolidating chart parsing/handling logic in Chat.tsx.

Changes:

• Backend: changed stream_openai_text to yield (role, content) tuples and stream_chat_request to emit choices[0].delta json-lines, sending citations as a single role: "tool" delta.
• Frontend: updated streaming parsing to accumulate assistant deltas and process citations from tool deltas; refactored chart parsing/handling into helper functions and added “automatic chart” suppression behavior.
• Types/tests: updated ParsedChunk to support delta and updated Python tests to match the new streaming yield contract.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
src/tests/api/services/test_chat_service.py	Updates unit tests to validate (role, content) streaming and tool-based citation chunks.
src/api/services/chat_service.py	Refactors streaming generator to emit role/content tuples and outputs json-lines deltas with citations in a tool message.
src/App/src/types/AppTypes.ts	Updates streaming chunk typing to support delta payloads.
src/App/src/components/Chat/Chat.tsx	Refactors chart parsing/dispatching and updates stream parsing to handle delta roles and tool citation messages.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

This PR refactors the frontend chat/chart streaming logic and updates the backend streaming contract so the UI can process structured streaming deltas (choices[0].delta) and receive citations as a dedicated tool message.

Changes:

• Backend: change stream_openai_text to yield (role, content) tuples and emit citations as a single tool chunk (JSON array string).
• Frontend: update streaming parsing to accumulate assistant deltas, handle tool citation chunks, and centralize chart parsing/dispatch via parseChartResponse + handleChartResult.
• Types/tests: update ParsedChunk to support the new delta format and adjust API tests for the tuple-based streaming contract.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
src/api/services/chat_service.py	Switches streaming output to (role, content) tuples and emits citations via role: "tool" deltas.
src/tests/api/services/test_chat_service.py	Updates tests to validate the new tuple-based stream and delta output format.
src/App/src/types/AppTypes.ts	Expands ParsedChunk typing to allow choices[].delta parsing (and keeps legacy messages optional).
src/App/src/components/Chat/Chat.tsx	Refactors streaming parsing for deltas/tool citations and centralizes chart parsing/handling logic.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Refactors the chat streaming and chart-handling flows to support a structured JSON-lines streaming contract where incremental content is delivered via choices[0].delta, with citations delivered as a separate tool delta.

Changes:

• Backend: stream_openai_text now yields (role, content) tuples and stream_chat_request wraps them into choices[0].delta JSON-lines chunks.
• Frontend: streaming parser updated to accumulate assistant deltas, capture tool citation payloads, and centralize chart parsing/dispatch via helper functions.
• Types/tests updated to reflect the new delta-based streaming format and tool-based citation delivery.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
src/api/services/chat_service.py	Switches streaming to (role, content) tuples and emits delta JSON-lines chunks; citations emitted as a separate tool message.
src/tests/api/services/test_chat_service.py	Updates tests for tuple-yielding stream_openai_text and validates delta-based stream_chat_request output.
src/App/src/types/AppTypes.ts	Updates ParsedChunk typing to support delta chunks and tool-role citation delivery.
src/App/src/components/Chat/Chat.tsx	Refactors streaming parsing and adds helpers for chart response parsing and chart result handling; updates citation parsing for tool payloads.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

🎉 This PR is included in version 3.20.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀