Commit 76cb788
authored
feat(sqs): finish Milestone 1 - purge, batch, tags, redrive, FIFO, retention reaper (#638)
## Summary
Closes the remaining `NotImplemented` gaps on the SQS adapter so it can
serve real client workloads, not just the catalog and the single-message
hot path.
- **Queue ops**: PurgeQueue (with AWS's 60s cooldown persisted on the
meta record), TagQueue / UntagQueue / ListQueueTags. GetQueueAttributes
now returns ApproximateNumberOfMessages / NotVisible / Delayed,
QueueArn, CreatedTimestamp, LastModifiedTimestamp.
- **Message ops**: SendMessageBatch / DeleteMessageBatch /
ChangeMessageVisibilityBatch with per-entry validation.
MessageAttributes is fully wired: structured input, AWS-canonical MD5
(length-prefixed sorted encoding + type byte), filtered echo on Receive.
- **DLQ redrive**: RedrivePolicy is parsed, persisted, and enforced —
receive transactionally moves a message to the DLQ when it would exceed
maxReceiveCount.
- **FIFO**: per-queue monotonic SequenceNumber, 5-minute dedup window
(`!sqs|msg|dedup|`), and group lock (`!sqs|msg|group|`) that keeps the
head pinned across visibility expiries — released only on Delete / DLQ
redrive / retention.
- **Retention reaper**: new `!sqs|msg|byage|` send-age index plus a
leader-side goroutine that drops expired records (data + vis + byage +
optional group lock) under one OCC dispatch per record. Per-queue budget
prevents starvation; followers short-circuit so only the leader emits
Dispatches.
## Test plan
- [x] `go vet ./...`
- [x] `go test -run 'TestSQS' ./adapter/...`
- [x] 11 new integration tests covering purge cooldown, batch shape
errors, MD5 round-trip, malformed attributes, DLQ redrive, FIFO sequence
/ dedup / content-based dedup, FIFO group lock pinning across visibility
expiry, retention reaper.
- [x] `golangci-lint run --config=.golangci.yaml ./adapter/...` clean.
## Out of scope
- Long-poll cross-node notifier (still polls every 200ms; design §7.3
considers polling acceptable for Milestone 1).
- Operator console UI (`adapter/console.go`).
- Jepsen workload (`jepsen/sqs/`).
- Query-protocol XML compatibility (JSON-1.0 only).13 files changed
Lines changed: 5055 additions & 227 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
55 | | - | |
56 | 55 | | |
57 | 56 | | |
58 | 57 | | |
| |||
69 | 68 | | |
70 | 69 | | |
71 | 70 | | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
72 | 79 | | |
73 | 80 | | |
74 | 81 | | |
| |||
84 | 91 | | |
85 | 92 | | |
86 | 93 | | |
| 94 | + | |
87 | 95 | | |
88 | | - | |
89 | | - | |
90 | | - | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
91 | 101 | | |
92 | 102 | | |
93 | 103 | | |
| |||
96 | 106 | | |
97 | 107 | | |
98 | 108 | | |
99 | | - | |
| 109 | + | |
100 | 110 | | |
101 | | - | |
| 111 | + | |
102 | 112 | | |
103 | 113 | | |
104 | | - | |
| 114 | + | |
105 | 115 | | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
110 | 120 | | |
111 | 121 | | |
112 | 122 | | |
| |||
120 | 130 | | |
121 | 131 | | |
122 | 132 | | |
| 133 | + | |
123 | 134 | | |
124 | 135 | | |
125 | 136 | | |
126 | 137 | | |
127 | 138 | | |
128 | 139 | | |
129 | 140 | | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
130 | 144 | | |
131 | 145 | | |
132 | 146 | | |
| |||
237 | 251 | | |
238 | 252 | | |
239 | 253 | | |
240 | | - | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
245 | | - | |
246 | | - | |
247 | | - | |
248 | | - | |
249 | 254 | | |
250 | 255 | | |
251 | 256 | | |
| |||
0 commit comments