feat(phase8 #92 D8.5-BE): canonical UIMessage chat history + runtime_kind discriminator (#1706)

Phase 8 task #92 (D8.5-BE) — first-cut backend migration of the
non-agent chat path to the canonical ``UIMessage`` shape, scoped per
architect msg=01918929 + Weston msg=df87fe24 + earayu2 msg=f20d5034
hard-cut acceptance:

The inventory revealed the production "non-agent chat path" the
original D8.5 design assumed has already converged on the agent
runtime (``chat_completion_service.openai_chat_completions`` already
delegates to ``runtime_manager.turn_service.create_or_get_turn`` and
``ChatService.create_chat`` rejects non-AGENT bots). So the actual
#92 work is A+B+C only — adding the discriminator column for future
non-agent paths and migrating the user-visible chat history shape to
canonical UIMessage. The translator extension (``chat.text.delta`` /
``chat.completed``) and the ``StoredChatMessagePart`` /
``RedisChatMessageHistory`` deletion are deferred per architect /
Weston canonical lock.

Changes:

A. ``runtime_kind`` discriminator on ``agent_message`` table
- ``aperag/domains/agent_runtime/db/models.py``: new
  ``runtime_kind: str`` ORM column with values
  ``agent_runtime`` / ``direct_chat`` / ``rag_chat`` (mutually
  exclusive enum); existing rows backfill via
  ``server_default="agent_runtime"``. ``role`` keeps speaker
  semantics independent of the runtime that produced the message.
- ``aperag/migration/versions/...c8f2d34a51e7_add_agent_message_runtime_kind.py``:
  additive migration; downgrade drops the column.

B. ``ChatService._build_v3_chat_history`` rewrite
- Returns ``list[AgentTurnSnapshot]`` (one snapshot per assistant
  turn) instead of the legacy ``list[list[ChatMessage]]`` shape.
- Reuses ``snapshot_assembler.assemble_parts_from_artifacts`` (the
  #90 D8.4d projection) so historical turns expose the same
  ``UIMessagePart`` shape the FE consumes from the live SSE stream
  (D8 §2 wire/at-rest byte-equal).
- ``error_text`` for FAILED / CANCELLED turns surfaces an
  ``error_summary`` artifact's message, falling back to
  ``turn.error_message`` — mirrors the snapshot endpoint contract.
- The turn's user query lives at ``input_text`` on the snapshot
  envelope (rather than as a separate ``role=human`` ChatMessage)
  so the FE renders user/assistant from a single object per turn.
- Legacy ``_extract_artifact_text`` / ``_extract_references`` /
  ``_map_reference_item`` / ``_artifact_type_value`` /
  ``_coerce_timestamp`` helpers are retired alongside the legacy
  shape.

C. ``ChatDetails.history`` schema
- ``aperag/domains/conversation/schemas.py``: ``history`` is now
  ``Optional[list[AgentTurnSnapshot]]`` with explicit description
  citing D8 §2 byte-equal canonical and the new shape.
- The ``conversation.schemas`` ↔ ``agent_runtime.uimessage``
  ↔ ``agent_runtime.schemas`` ↔ ``conversation.schemas`` cycle is
  broken via ``TYPE_CHECKING`` import + a module-level
  ``ChatDetails.model_rebuild()`` hook at the bottom of
  ``conversation/schemas.py``. Pydantic resolves the forward ref at
  load time so the OpenAPI schema is fully populated.
- ``aperag/domains/agent_runtime/uimessage.py``: ``AgentTurnSnapshot``
  gains ``runtime_kind: RuntimeKind`` (default ``"agent_runtime"``)
  and ``input_text: Optional[str]`` so historical turns can render
  the user query without a separate envelope round-trip.
- ``TurnService.get_turn_snapshot`` writes both new fields on the
  live snapshot endpoint so live and historical reload paths match.

D. (deferred) Translator extension for ``chat.text.delta`` /
``chat.completed`` and ``StoredChatMessagePart`` /
``RedisChatMessageHistory`` deletion stay out of #92 per
Weston msg=df87fe24 / PM msg=01918929. The non-agent live path the
extension would have served does not exist in the current
codebase; reintroducing it is a feature task, not a refactor.

Tests:
- ``tests/unit_test/chat/test_chat_service.py`` rewritten:
  * ``test_get_chat_returns_canonical_uimessage_history`` pins the
    new shape (snapshot per turn with text + source-url +
    data-citation parts, runtime_kind, input_text)
  * ``test_get_chat_history_surfaces_error_text_for_failed_turn``
    pins the error_text contract for FAILED turns
  * ``test_get_chat_history_does_not_expose_legacy_chatmessage_shape``
    regression-guard against revert to ``list[list[ChatMessage]]``
- ``tests/unit_test/agent_runtime/test_agent_runtime_v3.py`` updated
  to import ``AgentTurnSnapshot`` from ``agent_runtime.uimessage``
  (the back-compat re-export through ``agent_runtime.schemas`` was
  retired to break the new cycle).

Per D10 §G hard gate 1 (comprehensive grep sweep) ran across
``aperag/`` + ``tests/unit_test/`` + ``tests/e2e_http/hurl/`` +
``tests/e2e_http/scripts/``: only the FE
``web/src/components/chat/chat-messages.tsx`` reads ``chat.history``
in the old shape — that is the explicit hand-off seam for #93
huangheng (per architect msg=6e53a7c4).

Gates: full unit suite 833 / 29 skip / 0 fail; ruff check + format
clean.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

Author: earayu

aperag/domains/agent_runtime/api/routes.py

@@ -43,14 +43,14 @@
 from aperag.domains.agent_runtime.runtime import agent_runtime_manager as runtime_manager
 from aperag.domains.agent_runtime.schemas import (
     AgentArtifactEnvelope,
-    AgentTurnSnapshot,
     CancelTurnResponse,
     CreateTurnRequest,
     CreateTurnResponse,
 )
 from aperag.domains.agent_runtime.tools.consent import ConsentOwnershipError, ConsentService
 from aperag.domains.agent_runtime.tools.elicitation import ElicitationOwnershipError, ElicitationService
 from aperag.domains.agent_runtime.tools.lifecycle import translate_lifecycle_envelope
+from aperag.domains.agent_runtime.uimessage import AgentTurnSnapshot
 from aperag.domains.agent_runtime.wire import (
     StreamPart,
     TranslatorState,

aperag/domains/agent_runtime/db/models.py

@@ -153,7 +153,7 @@ class AgentArtifact(Base):
 
 
 class AgentMessage(Base):
-    """At-rest UIMessage envelope (Phase 8 D8.2 first-cut).
+    """At-rest UIMessage envelope (Phase 8 D8.2 first-cut, D8.5-BE refined).
 
     One row per assistant turn (1:1 with ``AgentTurn`` for now via
     ``turn_id``). ``parts`` holds the JSON-serialised
@@ -163,6 +163,13 @@ class AgentMessage(Base):
     is the runtime contract version tag so future renderer updates
     can branch without a separate negotiation.
 
+    ``runtime_kind`` (D8.5-BE / #92) tags the runtime that produced the
+    message — ``agent_runtime`` for the agent reasoning loop (D8.x
+    Phase A), and a forward-compat enum for direct LLM (``direct_chat``)
+    or RAG-only (``rag_chat``) paths. ``role`` retains its speaker
+    semantics independent of runtime origin per Weston msg=94dac98a /
+    architect canonical lock msg=e01e9b4b.
+
     The legacy ``AgentArtifact`` / ``AgentTimelineEvent`` tables are
     retained alongside this table during D8.x rollout — they will be
     dropped in D8.6 (#80) once D8.4 FE renderer is consuming
@@ -179,6 +186,7 @@ class AgentMessage(Base):
     turn_id = Column(String(24), nullable=False, index=True)
     chat_id = Column(String(24), nullable=False, index=True)
     role = Column(String(16), nullable=False)
+    runtime_kind = Column(String(24), nullable=False, default="agent_runtime", server_default="agent_runtime")
     schema_version = Column(String(64), nullable=False)
     parts = Column(JSON, default=lambda: [], nullable=False)
     gmt_created = Column(DateTime(timezone=True), default=utc_now, nullable=False)

aperag/domains/agent_runtime/schemas.py

@@ -44,7 +44,6 @@
 
 from pydantic import BaseModel, Field
 
-from aperag.domains.conversation.schemas import File
 from aperag.domains.knowledge_base.schemas import Collection
 from aperag.schema.common import ModelSpec
 
@@ -88,6 +87,18 @@ class UserActivityEnvelope(BaseModel):
     context: Optional[UserActivityContext] = None
 
 
+# ``File`` is imported lazily here to break the cycle introduced by D8.5-BE
+# (#92): ``conversation.schemas.ChatDetails.history`` now references
+# ``AgentTurnSnapshot`` from :mod:`aperag.domains.agent_runtime.uimessage`,
+# and ``uimessage`` in turn imports ``AGENT_RUNTIME_SCHEMA_VERSION`` and
+# ``UserActivityEnvelope`` from this module. Importing ``File`` at the
+# module top would close that cycle. By this point both symbols
+# ``uimessage`` needs are already defined, so importing ``File`` here is
+# safe and only the classes below (``CreateTurnRequest`` /
+# ``AgentMessage``) actually depend on it.
+from aperag.domains.conversation.schemas import File  # noqa: E402
+
+
 class AgentTurnEnvelope(BaseModel):
     schema_version: str = AGENT_RUNTIME_SCHEMA_VERSION
     turn_id: str
@@ -176,14 +187,14 @@ class CreateTurnResponse(BaseModel):
     stream_url: str
 
 
-# ``AgentTurnSnapshot`` is the canonical UIMessage at-rest envelope and
-# lives in :mod:`aperag.domains.agent_runtime.uimessage` next to the
-# other UIMessage classes. It is re-exported here so the existing
-# ``from aperag.domains.agent_runtime.schemas import AgentTurnSnapshot``
-# import sites continue to work without a domain-internal hop.
-from aperag.domains.agent_runtime.uimessage import (  # noqa: E402, F401
-    AgentTurnSnapshot,
-)
+# ``AgentTurnSnapshot`` lives in :mod:`aperag.domains.agent_runtime.uimessage`
+# next to the rest of the ``UIMessage`` family. The previous deferred
+# re-export from this module was retired in D8.5-BE (#92) because it
+# would close a fresh cycle between ``conversation.schemas`` (which
+# now imports ``AgentTurnSnapshot`` directly to type ``ChatDetails.history``)
+# and ``agent_runtime.schemas``. Existing call sites that still import
+# from this module are migrated to import from
+# ``aperag.domains.agent_runtime.uimessage`` directly.
 
 
 class CancelTurnResponse(BaseModel):
@@ -236,7 +247,6 @@ class AgentMessage(BaseModel):
     "AgentMessage",
     "AgentTimelineEventEnvelope",
     "AgentTurnEnvelope",
-    "AgentTurnSnapshot",
     "CancelTurnResponse",
     "CreateTurnRequest",
     "CreateTurnResponse",

aperag/domains/agent_runtime/services.py

@@ -24,7 +24,6 @@
     AgentArtifactEnvelope,
     AgentTimelineEventEnvelope,
     AgentTurnEnvelope,
-    AgentTurnSnapshot,
     CreateTurnRequest,
     ReferenceBundleItem,
     UserActivityContext,
@@ -36,6 +35,7 @@
     extract_error_text,
 )
 from aperag.domains.agent_runtime.storage import AgentRuntimeRedisStore
+from aperag.domains.agent_runtime.uimessage import AgentTurnSnapshot
 from aperag.domains.agent_runtime.uimessage_store import UIMessageStore
 from aperag.domains.conversation.db.models import BotType
 from aperag.domains.conversation.schemas import BotConfig
@@ -489,6 +489,7 @@ async def get_turn_snapshot(self, user: str, chat_id: str, turn_id: str) -> Agen
         return AgentTurnSnapshot(
             turn_id=turn.id,
             chat_id=turn.chat_id,
+            runtime_kind="agent_runtime",
             status=status_str,
             parts=parts,
             error_text=error_text,
@@ -497,6 +498,7 @@ async def get_turn_snapshot(self, user: str, chat_id: str, turn_id: str) -> Agen
             finished_at=turn.gmt_finished,
             created_at=turn.gmt_created,
             updated_at=turn.gmt_updated,
+            input_text=turn.input_text,
         )
 
     @staticmethod

aperag/domains/agent_runtime/uimessage.py

@@ -299,8 +299,11 @@ class UIMessage(BaseModel):
 # ---------------------------------------------------------------------
 
 
+RuntimeKind = Literal["agent_runtime", "direct_chat", "rag_chat"]
+
+
 class AgentTurnSnapshot(BaseModel):
-    """Canonical UIMessage at-rest snapshot for an agent turn (Phase 8 D8.4d / #90).
+    """Canonical UIMessage at-rest snapshot for an agent turn (Phase 8 D8.4d / #90, D8.5-BE / #92).
 
     Replaces the legacy ``{turn, timeline, artifacts}`` snapshot shape
     with the ``UIMessage``-aligned canonical that the FE renderer
@@ -310,6 +313,11 @@ class AgentTurnSnapshot(BaseModel):
     deserializes through the same ``UIMessagePart`` discriminated
     union the wire emits.
 
+    ``runtime_kind`` (D8.5-BE) tags the runtime that produced the
+    turn — ``agent_runtime`` for the agent reasoning loop,
+    ``direct_chat`` / ``rag_chat`` reserved for future paths. ``role``
+    keeps speaker semantics independent of runtime kind.
+
     ``parts`` is the persistable subset (transient ``data-activity``
     is stripped before write per :func:`persistable_parts`);
     ``status`` mirrors the runtime ``AgentTurnStatus`` enum value;
@@ -327,6 +335,8 @@ class AgentTurnSnapshot(BaseModel):
     schema_version: str = AGENT_RUNTIME_SCHEMA_VERSION
     turn_id: str
     chat_id: str
+    runtime_kind: RuntimeKind = "agent_runtime"
+    input_text: Optional[str] = None
     role: Literal["assistant"] = "assistant"
     status: str
     parts: list[UIMessagePart] = Field(default_factory=list)

aperag/domains/conversation/schemas.py

@@ -45,10 +45,19 @@
 from __future__ import annotations
 
 from datetime import datetime
-from typing import Any, Literal, Optional
+from typing import TYPE_CHECKING, Any, Literal, Optional
 
 from pydantic import BaseModel, Field, conint
 
+if TYPE_CHECKING:
+    # Type-only import to keep the OpenAPI schema for ``ChatDetails.history``
+    # populated without forming a runtime cycle: ``agent_runtime.uimessage``
+    # imports ``AGENT_RUNTIME_SCHEMA_VERSION`` / ``UserActivityEnvelope`` from
+    # ``agent_runtime.schemas``, which in turn imports ``File`` from this
+    # module. Pydantic resolves the forward reference via
+    # :func:`ChatDetails.model_rebuild` at the bottom of this file.
+    from aperag.domains.agent_runtime.uimessage import AgentTurnSnapshot
+
 from aperag.domains.knowledge_base.schemas import Collection as KBCollectionSchema
 from aperag.schema.common import ModelSpec, PageResult, PaginatedResponse
 
@@ -169,9 +178,16 @@ class ChatDetails(BaseModel):
     bot_id: Optional[str] = None
     peer_id: Optional[str] = None
     peer_type: Optional[Literal["system", "feishu", "weixin", "weixin_official", "web", "dingtalk"]] = None
-    history: Optional[list[list[ChatMessage]]] = Field(
+    history: Optional[list[AgentTurnSnapshot]] = Field(
         None,
-        description="Array of conversation turns, where each turn is an array of message parts",
+        description=(
+            "Phase 8 D8.5-BE (#92): historical conversation turns as canonical "
+            "``AgentTurnSnapshot`` envelopes — each turn carries the same "
+            "``UIMessagePart[]`` shape the FE consumes from the live SSE stream "
+            "(D8 §2 wire/at-rest byte-equal). Replaces the legacy "
+            "``list[list[ChatMessage]]`` shape; FE renders historical turns with "
+            "the same renderer used for live turns."
+        ),
     )
     status: Optional[Literal["active", "archived"]] = None
     created: Optional[datetime] = None
@@ -219,6 +235,21 @@ class Feedback(BaseModel):
 TurnFeedbackWrite = Feedback
 
 
+# Phase 8 D8.5-BE (#92): resolve the forward reference to
+# ``AgentTurnSnapshot`` after ``conversation.schemas`` finishes loading.
+# A direct top-level import would form a cycle through
+# ``agent_runtime.uimessage`` → ``agent_runtime.schemas`` → this module.
+# The TYPE_CHECKING block at the top declares the import for static
+# checkers / OpenAPI; this rebuild wires it up at runtime.
+def _rebuild_chat_details() -> None:
+    from aperag.domains.agent_runtime.uimessage import AgentTurnSnapshot  # noqa: F401
+
+    ChatDetails.model_rebuild()
+
+
+_rebuild_chat_details()
+
+
 __all__ = [
     "Agent",
     "Bot",