fix(chat): stop silent output-token truncation from stranding agent writes#151
Merged
Conversation
…rites Two staging conversations froze on a "…saving now:" preamble whose save never ran. Root cause: the tool loop called the model with a hardcoded `max_tokens: 4096`. A large write tool call (a full ui-strings dictionary, or several articles across locales in one turn) exceeded 4096 output tokens, so generation was cut off mid tool_use — no `tool_use_end` fired, the incomplete call never landed in `currentToolCalls`, and the loop's generic "not tool_use → end turn" check closed the turn silently. The user saw a frozen preamble with no error and nothing saved. Three complementary fixes: - Raise the output-token ceiling via the model catalog. `maxOutputTokens` is now a `ChatModelEntry` field (16K for every chat model, safely under each model's documented limit) plus `maxOutputTokensFor()` with an 8K fallback for Conversation-API / legacy model IDs. It is a cap, not a target — billed on actual output — so this is cost-neutral for normal turns and only prevents realistic bulk writes from truncating. Threaded through `ConversationConfig` and both callers (Studio chat + ee Conversation API). - Handle `stop_reason: max_tokens` explicitly in the loop instead of falling through to the silent end-of-turn: keep the partial text for the transcript, emit a typed `output_truncated` error, and close the turn. The client localizes it via `chat.output_truncated` (falls back to the generic send error for other error events). - Add the `rules.batch_writes` system-prompt rule steering the agent to split large writes across smaller save_content calls (a few entries per call, one locale per call) so it stops emitting oversized single tool calls in the first place. Adds a regression test asserting a `max_tokens` stop yields an `output_truncated` error and still closes the turn with the partial text.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two staging conversations (Lanista/collabers project) froze on a "…saving now:" preamble whose save never ran, with no error shown. Reproduced from the staging DB: the stuck assistant turns had
content_blocks = [text]only andtoken_count_output = 4096exactly.Root cause: the chat tool loop called the model with a hardcoded
max_tokens: 4096(conversation-engine.ts). A large write tool call — a fullui-stringsdictionary, or several articles across locales in one turn — exceeded 4096 output tokens, so generation was cut off midtool_use. Notool_use_endfires, the incomplete call never lands incurrentToolCalls, and the loop's generic "not tool_use → end turn" check closed the turn silently. The user saw a frozen preamble and nothing saved.Fix — three complementary changes
1. Raise the output-token ceiling (catalog-driven).
maxOutputTokensis now aChatModelEntryfield (16K for every chat model, safely under each model's documented limit) plusmaxOutputTokensFor()with an 8K fallback for Conversation-API / legacy model IDs. It's a cap, not a target — billed on actual output — so this is cost-neutral for normal turns and only prevents realistic bulk writes from truncating. Threaded throughConversationConfigand both callers (Studio chat + ee Conversation API).2. Handle
stop_reason: max_tokensexplicitly. Instead of falling through to the silent end-of-turn: keep the partial text for the transcript, emit a typedoutput_truncatederror, and close the turn. The client localizes it viachat.output_truncated(falls back to the generic send error for other error events).3.
rules.batch_writessystem-prompt rule. Steers the agent to split large writes across smallersave_contentcalls (a few collection entries per call) so it stops emitting oversized single tool calls. Reviewed against the write path for safety — the rule is scoped to the collection-entry axis and explicitly preserves id stability (same entry id across locales, existing id on updates), so batching cannot cause i18n id drift. Collection sibling/field merge, dictionary merge, document merge, and relation integrity are all preserved/enforced by existing code (save-content.ts, MCPplanContentSave, and the relation pre-flight, which returns a self-correcting error rather than silently breaking).Testing
max_tokensstop yields anoutput_truncatederror and still closes the turn with the partial text preserved.pnpm test(unit + integration + nuxt): 1049/1049 pass.pnpm typecheck, lint: clean.Follow-up
Merge-path change — worth a real save on staging after deploy (e.g. the same
ui-stringscapitalization edit) to confirm the large-write path end-to-end; mocked git tests don't exercise the provider truncation behavior.