Skip to content

Capture received messages as per-message child effects in QueueManager#187

Merged
stidsborg merged 1 commit into
mainfrom
simplify-inline-pending-messages
Jul 5, 2026
Merged

Capture received messages as per-message child effects in QueueManager#187
stidsborg merged 1 commit into
mainfrom
simplify-inline-pending-messages

Conversation

@stidsborg

Copy link
Copy Markdown
Owner

Summary

Iteration 1 of simplifying how messages get inlined into a flow. Each message a running flow receives is now durably captured as its own child effect the moment it is staged in ProcessMessages, and removed again when the message is delivered or idempotency-deduped — the same pattern IdempotencyKeys already uses, built on the new Effect.FlushlessCreateNextChild primitive (#186).

What changed

  • ProcessMessages: on staging, creates a per-message child effect via FlushlessCreateNextChild(ReceivedMessagesRoot, EncodeMessage(message)), tracked in a new _pendingMessageChildren map (position → child id). Flushless → no I/O for the running flow.
  • PruneDeliveredPendingMessage: on delivery/dedup, FlushlessClear(childId) deletes just that child.
  • Initialize: rebuilds the map from GetChildren(ReceivedMessagesRoot) and re-stages children a prior incarnation left behind; a child whose position was already delivered is pruned rather than re-delivered (analogue of the existing delivered-positions store-row clear).
  • PendingMessages.EncodeMessage/DecodeMessage made public for single-message round-trips.

Design decision

The children hang off a dedicated reserved id [-1,2] (ReceivedMessagesRoot), not PendingMessages.EffectId ([-1,1]). FlushlessClear deletes with clearChildren: true, so the completed-flow blob's own clear of [-1,1] would cascade-delete these children if they shared the parent. Separate ids keep the carriers independent — the completed-flow inline blob path (TryInlinePendingMessages, ExistingMessages) is untouched. A position landing in both is pruned by both branches (idempotent, self-correcting).

Sets up iteration 2

With the running flow capturing each message as a child, the completed-flow inline can move to the same representation next — TryInlinePendingMessages's merge-write-verify loop collapses into "append a child per message", and once both writers use children the [-1,1] blob and its bimodal prune branch go away.

Testing

  • Core builds, 0 warnings.
  • In-memory: 226 RFunctionTests + 100 messaging/effect/CreateNextChild — all pass.
  • PostgreSQL: 95 messaging + the 11 completed-flow-inline & idempotency tests by name — all pass.

Each message a running flow receives is now durably captured as its own
child effect (under a dedicated ReceivedMessagesRoot id) the moment it is
staged in ProcessMessages, and deleted again when the message is delivered
or idempotency-deduped. The write uses the new FlushlessCreateNextChild
primitive and is flushless, so it costs no I/O for the running flow and
dies with an equally-unflushed delivery - recovery stays store-backed and
at-least-once.

Initialize rebuilds the position->child map from GetChildren and re-stages
the children a prior incarnation left behind; a child whose position was
already delivered is pruned rather than re-delivered.

A dedicated reserved id ([-1,2]) is used rather than PendingMessages.EffectId
because FlushlessClear cascades to children, so sharing the parent would let
the completed-flow blob's clear delete these children. The completed-flow
inline blob path is left untouched; the two carriers coexist.
@stidsborg stidsborg merged commit c834dba into main Jul 5, 2026
5 of 8 checks passed
@stidsborg stidsborg deleted the simplify-inline-pending-messages branch July 5, 2026 07:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant