Commit d0d184b
authored
backup: Phase 0b M5-3 impl - SQS partitioned-FIFO reverse encoder (#929)
## Summary
Phase 0b M5-3 implementation — SQS partitioned-FIFO reverse encoder.
The design doc landed via PR #914 (merged 2026-06-03). This PR carries
the implementation that the doc described, in 4 commits matching the
slice plan:
- **Slice A+B** (`235c13ba`): decoder partition plumbing —
`parseSQSMessageDataKey` widens to `(enc, partition, isPartitioned,
err)`,
`sqsMessageRecord.Partition *uint32`, `HandleMessageData` wires the
parsed partition.
- **Slice C** (`6640674a`): mirror the four partitioned key constructors
(`sqsPartitionedMsgDataKeyBytes`, `…Vis…`, `…ByAge…`, `…Dedup…`) plus
`effectivePartitionCount` and the `"perQueue"` constant. M3b-3
circular-dep pattern — `internal/backup/` cannot import `adapter/`.
- **Slice D** (`1037deba`): drop `ErrSQSEncodeUnsupportedPartitioned`,
add
four new sentinels (`ErrSQSEncodeMissingPartition`,
`ErrSQSEncodeOutOfRangePartition`,
`ErrSQSEncodePartitionRoutingMismatch`), add `validatePartitioning` +
`sortMessagesForPartitionedEmit`, thread `partition *uint32` through
`addMessage` and `addSideRecords`.
- **Slice E** (`7603bf78`): 10 tests — the 8 design-doc rows plus
`TestSQSEncodeGateUsesRawPartitionCount` (codex P2 v914 v7 regression)
and `TestSQSEncodePartitionedSortStableAcrossPartitions`.
## Validation gates
All four use **raw `meta.PartitionCount > 1`** as the partitioned-queue
predicate, never `effectivePartitionCount`. Codex P2 v914 v7
demonstrated
that effective-count gates would silently allow a `perQueue` +
`PartitionCount=2` + `Partition==nil` dump to slip past validation, then
`addMessage`'s `partition != nil` branch would emit classic-shape keys
against a partitioned-keyspace queue (silent data loss on first read).
| Gate | Predicate | Sentinel |
|---|---|---|
| missing partition | `PartitionCount > 1 && Partition == nil` |
`ErrSQSEncodeMissingPartition` |
| out-of-range | `PartitionCount > 1 && *Partition >= PartitionCount` |
`ErrSQSEncodeOutOfRangePartition` |
| classic + nonzero | `PartitionCount <= 1 && *Partition != 0` |
`ErrSQSEncodeInvalidMessage` |
| perQueue + nonzero | `PartitionCount > 1 && perQueue && *Partition !=
0` | `ErrSQSEncodePartitionRoutingMismatch` |
## 5-lens self-review
1. **Data loss** — All four gates enforce key-shape ↔ predicate
consistency. Classic-queue tests stay green
(`Partition==nil` + `PartitionCount<=1` takes legacy path).
2. **Concurrency** — Offline encoder; no new goroutines / locks /
shared state.
3. **Performance** — One extra O(N) pre-walk over `records`. No extra
allocations. `effectivePartitionCount` not on the hot path.
4. **Data consistency** — Raw `PartitionCount > 1` selects key shape in
both this encoder AND the live `adapter/sqs_keys_dispatch.go` — they
agree. Partitioned dedup key includes `<group-seg>+|+<dedup-seg>`
(CodeRabbit major PR #732 round 6).
5. **Test coverage** — 10 new tests; 4 are explicit gate regressions for
prior review findings.
## Test plan
- [x] `go test ./internal/backup/` passes
- [x] `golangci-lint run --config=.golangci.yaml ./internal/backup/` 0
issues
- [ ] Bot review cycle
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Full support for backing up and restoring partitioned SQS queues,
preserving partition attribution and stable partitioned ordering.
* **Behavior Changes**
* Refined partition validation with distinct, actionable validation
errors and stricter power-of-two checks.
* Encoder now pre-sorts and stages partitioned messages to ensure stable
FIFO sequence computation.
* **Tests**
* Extensive tests covering partitioned encoding, validation gates, dedup
collisions, and round-trip semantics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->6 files changed
Lines changed: 1376 additions & 61 deletions
File tree
- internal/backup
0 commit comments