fix(decopilot): stream user prompt to other thread viewers#4461
Merged
Conversation
The per-thread run stream carried only assistant/harness chunks, so a second viewer of a shared thread never saw the other user's prompt until a DB refetch. Publish the materialized request message onto the same JetStream subject as a transient `data-user-message` control chunk at POST time (before the run's assistant chunks), reusing publishRawChunk → serializeChunk which already fragments payloads over MAX_PUBLISH_BYTES and drops over MAX_CHUNKED_BYTES. The client intercepts the chunk before the assistant fold (the AI SDK reassembler hardcodes role:"assistant") and upserts it by id via applyLocalMessage, so the author's optimistic copy dedupes while other viewers get it live. The durable projection filters the chunk out; the DB copy remains the emitRequestMessage write.
decocms Bot
pushed a commit
that referenced
this pull request
Jul 11, 2026
PR: #4461 fix(decopilot): stream user prompt to other thread viewers Bump type: patch - decocms (apps/mesh/package.json): 4.11.3 -> 4.11.4 Deploy-Scope: both
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
We stream the LLM output server-side to multiple consumers (several users viewing the same thread). But the user prompt is never put on the stream — it's only written to the DB. So a second viewer of a shared thread doesn't see the other user's message until a DB refetch happens (reconnect or a
thread.statusevent), which feels laggy on a live thread.Fix
Publish the materialized request message onto the same per-thread NATS JetStream subject as a transient
data-user-messagecontrol chunk, at POST time, before the run's assistant chunks (so it sorts first).Key point — large prompts are handled for free: the publish goes through the existing
publishRawChunk → serializeChunkpath, which already fragments anything overMAX_PUBLISH_BYTES(768 KiB) and drops overMAX_CHUNKED_BYTES(32 MiB). File attachments are already offloaded to object-storage URLs byuploadFileParts, so the payload is essentially text. No new large-payload machinery — this reuses the harness stream codec.Server
user-message-stream.ts(new) —buildUserMessageChunk/isUserMessageControlChunk/publishUserMessage, mirroringrun-status-stage.ts. Publish is best-effort (never fails the POST).routes.ts— publish right afteremitRequestMessage, in the sameshouldPersistRequestMessagegate.project-chunks.ts— filter the chunk out of the durable assistant projection.fenceFilteralready drops it (it's fence-less), but the explicit type filter matches thedata-run-statusprecedent and future-proofs against it ever being published with a run fence.Client
thread-connection.ts— interceptdata-user-messageinhandleChunkbefore the AI-SDK fold (the reassembler hardcodesrole:"assistant"), and route it into the existingapplyLocalMessage→mergeAndSort/upsertByIdpath. Dedup is free: the author's optimistic row shares the same id, and a later DB refetch reconcilescreated_atviapreserveCreatedAt.Testing
user-message-stream.test.ts).project-chunks.test.tsto assert the chunk never leaks into projected assistant parts while normal assistant text still folds.bun test(affected),tsc --noEmit(0 errors),bun run lint(0 errors) all pass.Not covered / follow-up
Live two-viewer end-to-end wasn't driven headlessly here — worth a manual check (or an e2e in
packages/e2e) with two sessions on one thread to confirm the prompt renders live for the non-author.Summary by cubic
Mirror the user's prompt onto the per-thread run stream so other viewers see it instantly, fixing lag in shared threads. Uses a transient
data-user-messagecontrol chunk and keeps the durable projection unchanged.data-user-messagechunk at POST time (before assistant chunks) via existingpublishRawChunkfragmentation; filter it out of the assistant projection; gated with the sameshouldPersistRequestMessagecheck.data-user-messagebefore the assistant fold and upsert viaapplyLocalMessageso the author's optimistic message dedupes and other viewers see it live.Written for commit 4f24dff. Summary will update on new commits.