Capture received messages as per-message child effects in QueueManager#187
Merged
Conversation
Each message a running flow receives is now durably captured as its own child effect (under a dedicated ReceivedMessagesRoot id) the moment it is staged in ProcessMessages, and deleted again when the message is delivered or idempotency-deduped. The write uses the new FlushlessCreateNextChild primitive and is flushless, so it costs no I/O for the running flow and dies with an equally-unflushed delivery - recovery stays store-backed and at-least-once. Initialize rebuilds the position->child map from GetChildren and re-stages the children a prior incarnation left behind; a child whose position was already delivered is pruned rather than re-delivered. A dedicated reserved id ([-1,2]) is used rather than PendingMessages.EffectId because FlushlessClear cascades to children, so sharing the parent would let the completed-flow blob's clear delete these children. The completed-flow inline blob path is left untouched; the two carriers coexist.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Iteration 1 of simplifying how messages get inlined into a flow. Each message a running flow receives is now durably captured as its own child effect the moment it is staged in
ProcessMessages, and removed again when the message is delivered or idempotency-deduped — the same patternIdempotencyKeysalready uses, built on the newEffect.FlushlessCreateNextChildprimitive (#186).What changed
ProcessMessages: on staging, creates a per-message child effect viaFlushlessCreateNextChild(ReceivedMessagesRoot, EncodeMessage(message)), tracked in a new_pendingMessageChildrenmap (position → child id). Flushless → no I/O for the running flow.PruneDeliveredPendingMessage: on delivery/dedup,FlushlessClear(childId)deletes just that child.Initialize: rebuilds the map fromGetChildren(ReceivedMessagesRoot)and re-stages children a prior incarnation left behind; a child whose position was already delivered is pruned rather than re-delivered (analogue of the existing delivered-positions store-row clear).PendingMessages.EncodeMessage/DecodeMessagemadepublicfor single-message round-trips.Design decision
The children hang off a dedicated reserved id
[-1,2](ReceivedMessagesRoot), notPendingMessages.EffectId([-1,1]).FlushlessCleardeletes withclearChildren: true, so the completed-flow blob's own clear of[-1,1]would cascade-delete these children if they shared the parent. Separate ids keep the carriers independent — the completed-flow inline blob path (TryInlinePendingMessages,ExistingMessages) is untouched. A position landing in both is pruned by both branches (idempotent, self-correcting).Sets up iteration 2
With the running flow capturing each message as a child, the completed-flow inline can move to the same representation next —
TryInlinePendingMessages's merge-write-verify loop collapses into "append a child per message", and once both writers use children the[-1,1]blob and its bimodal prune branch go away.Testing
CreateNextChild— all pass.