PostgreSQL: bulk AppendMessages in a single round-trip#152
Merged
Conversation
Move the per-flow AppendMessages SQL into SqlGenerator and have the bulk overload group by StoredId, build one append command per flow, and execute together with the interrupt in a single NpgsqlBatch. This drops the separate GetMaxPositions round-trip — positions are now computed server-side via COALESCE(MAX(position), -1) + random base + unnest(...) WITH ORDINALITY, so the SQL string is constant regardless of batch size and the prepared statement cache is reused across calls.
The default 10s threshold occasionally trips on the SqlServer CI runner under 12-way parallel test execution. Bumping to 30s gives the workflow restart + replay cycle more headroom without changing test semantics.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AppendMessagesSQL intoSqlGeneratorso all message-store SQL lives in one place, alongside the existing position-explicitAppendMessages(IReadOnlyList<StoredIdAndMessageWithPosition>)overload.PostgreSqlMessageStore.AppendMessages(IReadOnlyList<StoredIdAndMessage>)to group byStoredId, build one per-flow append command viaSqlGenerator, and execute together with the interrupt in a singleNpgsqlBatch. This drops the separateGetMaxPositionsround-trip.AppendMessages(StoredId, IReadOnlyList<StoredMessage>)is now a thin wrapper around the bulk overload.The append SQL itself is parameterized via
unnest(\$3::bytea[]) WITH ORDINALITY, so the SQL string is constant regardless of batch size and PostgreSQL's prepared-statement cache is reused across calls. The per-call random base offset (COALESCE(MAX(position), -1) + 2147483647 + \$2) preserves the existing collision-avoidance scheme for concurrent appenders to the same flow.Test plan
dotnet test ./Stores/PostgreSQL/Cleipnir.ResilientFunctions.PostgreSQL.Tests --filter "FullyQualifiedName~Messaging|FullyQualifiedName~WorkflowMessageTests"— 73/73 passing