Skip to content

Commit 4e7d0f8

Browse files
feat(core): optional ThinkingChunk StreamChunk variant (default-off, upstream-compatible; supersedes #39)
Adds a fourth, opt-in StreamChunk variant for streaming agent reasoning/"thinking" to chat platforms. The whole design is default-off: with no opt-in, the normalized stream and the posted message are byte-for-byte identical to upstream chat@4.31. - Additive type: ThinkingChunk(type="thinking", content=str) added to the StreamChunk union. The existing three variants are unaffected. - Opt-in emit: from_full_stream(stream, emit_thinking=False) and a thread-level emit_thinking config flag (threaded into the internal _from_full_stream) surface AI-SDK reasoning/reasoning-delta (and pydantic-ai part_kind=="thinking") parts as ThinkingChunk only when enabled. Default False drops reasoning exactly as upstream does. - Graceful consume: Thread._handle_stream never accumulates a ThinkingChunk into the posted-message text, and every adapter's stream handler skips it. Slack/Teams expose an optional render_thinking hook (via shared.adapter_utils.maybe_render_thinking); the text-accumulate adapters ignore it structurally — no crash, posted message unchanged. - No state pollution: ThinkingChunk is streaming-only. Message has no thinking field, to_json() is unchanged, and a round-tripped Message is byte-identical, so cross-SDK Redis/Postgres state stays compatible. - Docs: UPSTREAM_SYNC.md Known Non-Parity row added. Rationale: upstream's chat-platform SDK drops reasoning (leaves it to the AI-SDK web UI); chinchill streams thinking to Slack/Teams out-of-band today because there is no path. This gives the SDK a first-class, opt-in one without changing any default behavior. Gauntlet: ruff check + format, audit_test_quality (0 hard failures), verify_test_fidelity --strict (732/732, 0 missing), pyrefly src (0 errors), pytest (5121 passed, 4 pre-existing skips).
1 parent bf2963a commit 4e7d0f8

15 files changed

Lines changed: 900 additions & 19 deletions

docs/UPSTREAM_SYNC.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -684,6 +684,7 @@ stay explicit instead of being rediscovered in code review.
684684
| Teams `cards_input` empty-options default (vercel/chat#8c71411, chat@4.31) | `input_request_to_teams_adaptive_card` reads `options = request.get("options") or []` | Upstream `cards-primitives/input.ts` reads `const options = request.options ?? []` | Benign truthiness divergence. The only values that differ between `or []` and `?? []` are the falsy-but-present ones (`[]`, `None`); for an options list both produce the same empty list, so the rendered card is byte-identical. Documented (not "fixed" to `is not None`) because there is no observable behavior difference — a present empty `options` and an absent `options` both yield "no choices". |
685685
| Teams `graph` path-segment encoding (vercel/chat#8c71411, chat@4.31) | Path segments (`team_id` / `channel_id` / `chat_id` / `message_id`) are interpolated through `quote(segment, safe='')` | Upstream `graph/{channels,messages}.ts` use `encodeURIComponent(...)` | Benign over-encoding divergence. `quote(safe='')` percent-encodes a strictly larger set than `encodeURIComponent` (notably `!`, `'`, `(`, `)`, `*` — which `encodeURIComponent` leaves literal). Graph IDs (team/channel/chat/message GUIDs and thread tokens) never contain those characters, so the encoded path is identical in practice; where they did differ, the stricter `quote` is the safer choice (no URL injection via an unescaped sub-delimiter). Documented rather than narrowed to match `encodeURIComponent` exactly. |
686686
| Teams `graph` defensive shape coercion (vercel/chat#8c71411, chat@4.31) | `to_graph_message` / channel + message readers coerce unexpected Graph payload shapes with `isinstance` guards (`x if isinstance(x, Mapping) else {}`, `value if isinstance(value, list) else []`) before reading fields | Upstream `graph/messages.ts` reads `message.from?.user` / `message.body?.content ?? ""` etc. with optional chaining — a non-object where an object is expected throws at the property access | Benign defensive divergence. Upstream's optional chaining tolerates `null`/`undefined` but throws on a wrong-typed non-null (e.g. a string where an object is expected); our `isinstance` coercion fails closed to an empty mapping/list instead of raising. For well-formed Graph responses the behavior is identical; the divergence only manifests on malformed payloads, where returning an empty-shape result is more resilient than throwing. Mirrors the repo's general "more resilient than throw" stance (cf. the `renderPostable on unknown input` row). |
687+
| `ThinkingChunk` StreamChunk variant (Python-only, default-off; supersedes PR #39) | A fourth `StreamChunk` variant — `ThinkingChunk(type="thinking", content=str)` — surfaces AI-SDK `reasoning`/`reasoning-delta` (and pydantic-ai `part_kind == "thinking"`) parts. **OPT-IN, default-off**: emitted only when a caller passes `from_full_stream(stream, emit_thinking=True)` or sets the thread-level `emit_thinking=True` config; the internal `_from_full_stream` threads the same flag. With the default (`emit_thinking=False`) the normalized stream is **byte-for-byte identical** to upstream — reasoning parts are dropped and **no** `ThinkingChunk` is produced. Consumption is graceful: `Thread._handle_stream` never accumulates a `ThinkingChunk` into the posted-message text, and every adapter's stream handler skips it (Slack/Teams expose an optional `render_thinking` hook via `chat_sdk.shared.adapter_utils.maybe_render_thinking`; the text-accumulate adapters ignore it structurally). **Streaming-only — never persisted**: `Message` has no `thinking` field, `to_json()` is unchanged, and a round-tripped `Message` is byte-identical, so cross-SDK state (Redis/Postgres shared with the TS SDK) stays compatible. | Upstream `from-full-stream.ts` forwards only `text-delta` + `finish-step`; AI-SDK `reasoning`/`reasoning-delta` parts fall through and are discarded. `StreamChunk = MarkdownTextChunk | TaskUpdateChunk | PlanUpdateChunk` — no reasoning variant. Upstream leaves reasoning display to the AI-SDK web UI. | chinchill actively streams agent thinking to Slack/Teams but has to intercept the model stream out-of-band today because the chat-platform SDK has no path for it. This gives the SDK a first-class, opt-in one without changing any default behavior. The whole design constraint is that default-off == upstream: additive type, opt-in emit, graceful/skip consume, zero state pollution. Regression coverage: `tests/test_thinking_chunk.py`, `tests/test_from_full_stream.py::TestThinkingOptIn`, `tests/test_types.py::TestThinkingChunk`, plus per-adapter no-crash tests in `tests/test_slack_api.py`, `tests/test_teams_native_streaming.py`, `tests/test_twilio_adapter.py`, `tests/test_messenger_api.py`. |
687688
### Platform-specific gaps
688689

689690
| Area | Python | TS | Rationale |

src/chat_sdk/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -212,6 +212,7 @@
212212
StreamChunk,
213213
StreamOptions,
214214
TaskUpdateChunk,
215+
ThinkingChunk,
215216
Thread,
216217
ThreadInfo,
217218
ThreadSummary,
@@ -450,6 +451,7 @@
450451
"StreamChunk",
451452
"StreamOptions",
452453
"TaskUpdateChunk",
454+
"ThinkingChunk",
453455
"Thread",
454456
"ThreadInfo",
455457
"ThreadSummary",

src/chat_sdk/adapters/slack/adapter.py

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,12 @@
6464
from chat_sdk.emoji import emoji_to_slack, resolve_emoji_from_slack
6565
from chat_sdk.logger import ConsoleLogger, Logger
6666
from chat_sdk.modals import ModalElement, OptionsLoadGroup, SelectOptionElement
67-
from chat_sdk.shared.adapter_utils import extract_card, extract_files
67+
from chat_sdk.shared.adapter_utils import (
68+
extract_card,
69+
extract_files,
70+
is_thinking_chunk,
71+
maybe_render_thinking,
72+
)
6873
from chat_sdk.shared.errors import AdapterRateLimitError, AuthenticationError, ValidationError
6974
from chat_sdk.types import (
7075
ActionEvent,
@@ -4006,6 +4011,15 @@ async def push_text_and_flush(text: str) -> None:
40064011
await push_text_and_flush(text_value)
40074012
elif hasattr(chunk, "type") and chunk.type == "markdown_text": # type: ignore[union-attr]
40084013
await push_text_and_flush(chunk.text) # type: ignore[union-attr]
4014+
elif is_thinking_chunk(chunk):
4015+
# Python-only divergence: ``ThinkingChunk`` is streaming-only
4016+
# agent reasoning, NOT message content. By default it is
4017+
# skipped (no effect on the posted message). An adapter/consumer
4018+
# that wants to display thinking sets ``self.render_thinking``;
4019+
# only then is it invoked. Skipping (not routing to
4020+
# ``send_structured_chunk``) keeps the default posted message
4021+
# byte-identical and avoids disabling structured-chunk support.
4022+
await maybe_render_thinking(getattr(self, "render_thinking", None), chunk)
40094023
else:
40104024
await send_structured_chunk(chunk)
40114025

src/chat_sdk/adapters/teams/adapter.py

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,12 @@
3434
from chat_sdk.emoji import convert_emoji_placeholders
3535
from chat_sdk.errors import ChatNotImplementedError
3636
from chat_sdk.logger import ConsoleLogger, Logger
37-
from chat_sdk.shared.adapter_utils import extract_card, extract_files
37+
from chat_sdk.shared.adapter_utils import (
38+
extract_card,
39+
extract_files,
40+
is_thinking_chunk,
41+
maybe_render_thinking,
42+
)
3843
from chat_sdk.shared.buffer_utils import buffer_to_data_uri, to_buffer
3944
from chat_sdk.shared.errors import (
4045
AdapterPermissionError,
@@ -1724,6 +1729,12 @@ async def stream(
17241729
text = chunk
17251730
elif isinstance(chunk, dict) and chunk.get("type") == "markdown_text":
17261731
text = chunk.get("text", "")
1732+
elif is_thinking_chunk(chunk):
1733+
# Python-only divergence: streaming-only reasoning, not message
1734+
# content. Default-skip keeps the buffered post byte-identical;
1735+
# an opt-in ``render_thinking`` hook may display it.
1736+
await maybe_render_thinking(getattr(self, "render_thinking", None), chunk)
1737+
continue
17271738
if not text:
17281739
continue
17291740
accumulated += text
@@ -1796,6 +1807,13 @@ async def _on_chunk(activity: Any) -> None:
17961807
text = chunk
17971808
elif isinstance(chunk, dict) and chunk.get("type") == "markdown_text":
17981809
text = chunk.get("text", "")
1810+
elif is_thinking_chunk(chunk):
1811+
# Python-only divergence: streaming-only reasoning, not
1812+
# message content. Default-skip keeps emitted text
1813+
# byte-identical; an opt-in ``render_thinking`` hook may
1814+
# display it.
1815+
await maybe_render_thinking(getattr(self, "render_thinking", None), chunk)
1816+
continue
17991817
elif getattr(chunk, "type", None) == "markdown_text":
18001818
# Dataclass ``MarkdownTextChunk`` form — mirror
18011819
# ``Thread.stream``'s ``_wrapped_stream`` extraction so the

src/chat_sdk/from_full_stream.py

Lines changed: 52 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -11,23 +11,62 @@
1111
- **StreamChunk objects** (``task_update``, ``plan_update``,
1212
``markdown_text``) -- passed through as-is for adapters with native
1313
structured chunk support.
14+
15+
Python-only divergence (default-off): when ``emit_thinking=True``, AI-SDK
16+
``reasoning`` / ``reasoning-delta`` parts (and pydantic-ai
17+
``part_kind == "thinking"`` parts) are surfaced as
18+
:class:`~chat_sdk.types.ThinkingChunk` objects. With the default
19+
``emit_thinking=False`` the output is byte-for-byte identical to upstream
20+
chat@4.31 (reasoning is dropped, no ``ThinkingChunk`` is emitted). See
21+
``docs/UPSTREAM_SYNC.md`` (Known Non-Parity).
1422
"""
1523

1624
from __future__ import annotations
1725

1826
from collections.abc import AsyncIterable, AsyncIterator
1927

20-
from chat_sdk.types import StreamChunk
28+
from chat_sdk.types import StreamChunk, ThinkingChunk
2129

2230
_STREAM_CHUNK_TYPES = frozenset({"markdown_text", "task_update", "plan_update"})
2331

32+
# AI-SDK v5/v6 reasoning part types, plus pydantic-ai's ``thinking`` part kind.
33+
# These are only consulted when ``emit_thinking=True``; otherwise they fall
34+
# through and are dropped exactly as upstream does.
35+
_REASONING_TYPES = frozenset({"reasoning", "reasoning-delta", "thinking"})
36+
37+
_TEXT_KEYS = ("text", "delta", "textDelta", "text_delta")
38+
# Reasoning payloads carry the text under the same keys, with ``content`` added
39+
# for pydantic-ai's ``ThinkingPart`` shape (``part_kind == "thinking"``).
40+
_REASONING_KEYS = ("content", "text", "delta", "textDelta", "text_delta")
41+
42+
43+
def _pick(event: object, keys: tuple[str, ...]) -> object | None:
44+
"""Return the first non-``None`` value among ``keys`` on a dict/object."""
45+
if isinstance(event, dict):
46+
return next((v for k in keys if (v := event.get(k)) is not None), None)
47+
return next((v for k in keys if (v := getattr(event, k, None)) is not None), None)
48+
2449

2550
async def from_full_stream(
2651
stream: AsyncIterable[object],
52+
*,
53+
emit_thinking: bool = False,
2754
) -> AsyncIterator[str | StreamChunk]:
2855
"""Normalize an async iterable stream for use with ``thread.post()``.
2956
3057
Yields either plain ``str`` chunks or ``StreamChunk`` objects.
58+
59+
Args:
60+
stream: The source async iterable (text stream, full stream, or
61+
pre-built ``StreamChunk`` objects).
62+
emit_thinking: **Opt-in, default off.** When ``False`` (the default),
63+
behavior is byte-for-byte upstream: AI-SDK ``reasoning`` /
64+
``reasoning-delta`` parts are dropped and **no**
65+
:class:`~chat_sdk.types.ThinkingChunk` is emitted. When ``True``,
66+
such parts (and pydantic-ai ``thinking`` parts) are surfaced as
67+
``ThinkingChunk`` objects so a consumer/adapter can render agent
68+
reasoning. Thinking is never accumulated into the posted message
69+
text, so the posted message is unchanged either way.
3170
"""
3271
needs_separator = False
3372
has_emitted_text = False
@@ -52,23 +91,24 @@ async def from_full_stream(
5291
if not event_type:
5392
continue
5493

55-
# Pass through StreamChunk objects
94+
# Pass through StreamChunk objects (includes pre-built ThinkingChunk).
5695
if event_type in _STREAM_CHUNK_TYPES:
5796
yield event # type: ignore[misc]
5897
continue
5998

99+
# Opt-in reasoning surfacing. Default-off => this whole branch is
100+
# skipped and reasoning parts fall through (dropped) exactly as
101+
# upstream chat@4.31 does.
102+
if event_type in _REASONING_TYPES:
103+
if emit_thinking:
104+
content = _pick(event, _REASONING_KEYS)
105+
if isinstance(content, str) and content:
106+
yield ThinkingChunk(content=content)
107+
continue
108+
60109
# AI SDK v6 uses "text", v5 uses "textDelta"; also accept "delta"
61110
# Priority: text > delta > textDelta > text_delta (matches TS)
62-
if isinstance(event, dict):
63-
text_content = next(
64-
(v for k in ("text", "delta", "textDelta", "text_delta") if (v := event.get(k)) is not None),
65-
None,
66-
)
67-
else:
68-
text_content = next(
69-
(v for k in ("text", "delta", "textDelta", "text_delta") if (v := getattr(event, k, None)) is not None),
70-
None,
71-
)
111+
text_content = _pick(event, _TEXT_KEYS)
72112

73113
if event_type == "text-delta" and isinstance(text_content, str):
74114
if needs_separator and has_emitted_text:

src/chat_sdk/shared/adapter_utils.py

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,54 @@
22

33
from __future__ import annotations
44

5+
import inspect
6+
from collections.abc import Awaitable, Callable
7+
from typing import Any
8+
59
from chat_sdk.cards import CardElement, is_card_element
610
from chat_sdk.types import AdapterPostableMessage, Attachment, FileUpload
711

12+
# Optional opt-in hook signature an adapter/consumer may set on the adapter
13+
# instance as ``render_thinking`` to receive streaming ``ThinkingChunk``
14+
# content. May be sync or async; both are supported by
15+
# :func:`maybe_render_thinking`.
16+
RenderThinking = Callable[[str], Awaitable[None] | None]
17+
18+
19+
def is_thinking_chunk(chunk: Any) -> bool:
20+
"""Return True if a stream chunk is a Python-only ``ThinkingChunk``.
21+
22+
Matches both the dataclass form (``chunk.type == "thinking"``) and the
23+
dict-normalized form (``chunk["type"] == "thinking"``). Used by adapter
24+
stream loops to *skip* thinking when accumulating the posted message —
25+
thinking is streaming-only reasoning, not message content.
26+
"""
27+
if isinstance(chunk, dict):
28+
return chunk.get("type") == "thinking"
29+
return getattr(chunk, "type", None) == "thinking"
30+
31+
32+
def thinking_content(chunk: Any) -> str:
33+
"""Extract the reasoning text from a ``ThinkingChunk`` (dict or dataclass)."""
34+
value = chunk.get("content") if isinstance(chunk, dict) else getattr(chunk, "content", None)
35+
return value if isinstance(value, str) else ""
36+
37+
38+
async def maybe_render_thinking(hook: RenderThinking | None, chunk: Any) -> None:
39+
"""Invoke an opt-in ``render_thinking`` hook for a ``ThinkingChunk``, else skip.
40+
41+
Default behavior (``hook is None``) is a no-op: the thinking chunk is
42+
silently skipped so the posted message stays byte-identical to upstream and
43+
adapters that don't render thinking never crash. When a hook is provided it
44+
is called with the reasoning text; both sync and async hooks are supported.
45+
"""
46+
if hook is None:
47+
return
48+
content = thinking_content(chunk)
49+
result = hook(content)
50+
if inspect.isawaitable(result):
51+
await result
52+
853

954
def extract_card(message: AdapterPostableMessage) -> CardElement | None:
1055
"""Extract CardElement from an AdapterPostableMessage if present."""

0 commit comments

Comments
 (0)