fix(sse): preserve broadcast order and de-race message history#483
Open
walcz-de wants to merge 1 commit into
Open
fix(sse): preserve broadcast order and de-race message history#483walcz-de wants to merge 1 commit into
walcz-de wants to merge 1 commit into
Conversation
The broadcast manager ran a pool of N competing delivery workers on a single channel. Each message is handed to exactly one worker, so with N>1 the delivery order to clients depends on goroutine scheduling: token-level SSE streams (agent chat deltas) arrive scrambled, which surfaces as interleaved/corrupted chat output in the React UI. Replace the pool with a single delivery goroutine consuming a buffered channel: FIFO order is preserved end-to-end, Send() stays non-blocking in practice, and slow clients still drop messages instead of stalling the broadcast (unchanged semantics). Also fix the message history: - Add() was called once per *delivered client* (duplicates with >1 client, nothing recorded with 0 clients) and raced unsynchronized across workers. It now records exactly once per message, under a mutex, delivery-independent. - Send() now snapshots under the lock and replays non-blocking, so a full client buffer cannot deadlock replay against the broadcaster. Adds order-preservation and history-dedup tests (pass with -race; the order test fails against the previous worker-pool implementation). Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
walcz-de
added a commit
to walcz-de/LocalAI
that referenced
this pull request
Jul 3, 2026
Agent chat streams arrived visibly corrupted in the React UI (tokens of one turn interleaved/scrambled, internal generations merged into one bubble). Two fixes: - Bump the walcz-de/LocalAGI override to ad5ef2e (fix/sse-broadcast- ordering, upstream PR mudler/LocalAGI#483): the SSE broadcast manager ran 5 competing delivery workers on one channel, so message order to clients was nondeterministic; history was recorded once per delivered client (duplicates on EventSource reconnect) and raced unsynchronized. Now a single FIFO delivery goroutine + mutexed, per-message history. - AgentChat.jsx: reset accumulated stream content on 'done' stream events. One agent turn runs several internal LLM generations (tool selection, reasoning, final answer) over the same SSE channel; without the reset the internal generations' text merged into the final answer's live bubble. Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The SSE broadcast manager runs a pool of N competing delivery workers (
NewManager(5)incore/state/pool.go) on a single broadcast channel. Each message is handed to exactly one worker, so with N>1 the delivery order to connected clients depends on goroutine scheduling.Why it matters
Token-level SSE streams (
stream_eventchat deltas emitted by the pool'sWithStreamCallback) are only meaningful in send order. Under real streaming rates the worker pool visibly scrambles them — in the React UI this surfaces as interleaved/corrupted agent chat output (tokens woven out of order, seemingly "multiple answers braided together").There are two additional defects in the same file:
messageHistory.Add()was called once per delivered client inside the per-client loop: with 2 clients each message was recorded twice (duplicated on replay after anEventSourcereconnect), and with 0 clients nothing was recorded at all. It also raced unsynchronized across the 5 workers (slice append without a mutex).history.Send()wrote to the client channel blocking, so a client with a full buffer could stall the connection handler.Fix
Send()stays non-blocking in practice, and slow clients still drop messages instead of stalling the broadcast (unchanged semantics).Tests
go test ./core/sse/... -race— adds order-preservation, history-dedup and history-without-clients tests. The order test fails against the previous worker-pool implementation.🤖 Generated with Claude Code
https://claude.ai/code/session_01KYp5FnAfsdGb6yjLmv8Mge