Skip to content

Commit b1b7a83

Browse files
authored
feat(sqs/admin): SigV4-bypass admin entrypoints + SPA queues pages (#670)
## Summary **Replaces #659**, which has unresolvable conflicts now that main has moved on (PR #649 squashed into main; PR #658 added the S3 admin endpoints; the Approximate counters implementation now lives directly in `adapter/sqs_catalog.go` via `scanApproxCounters`). Rather than a multi-day rebase, this PR re-applies the unique SQS admin code on a fresh branch off current main. What survived from #659: - `adapter/sqs_admin.go` — `SQSServer.AdminListQueues` / `AdminDescribeQueue` / `AdminDeleteQueue` (SigV4-bypass entrypoints, same shape as `AdminListTables` / `AdminListBuckets`). - `internal/admin/sqs_handler.go` — HTTP handler for `/admin/api/v1/sqs/queues{,/{name}}` with role re-evaluation on DELETE. - `web/admin/src/pages/SqsList.tsx` / `SqsDetail.tsx` — SPA pages for the queues view + delete confirmation. What changed during the re-apply: - `AdminQueueCounters` is now `int64` (matches `sqsApproxCounters` from main; bridge does no width conversion). - `AdminDescribeQueue` calls main's `scanApproxCounters` instead of the duplicate `computeApproxCounters` from the old branch — same numeric output, single implementation. - Dropped the `CountersTruncated` field; main's counter type doesn't expose truncation. SPA's "truncated" pill came out with it. - `apiRouteTable.dispatch` refactored to extract `resourceHandlerFor` so the dispatcher stays under cyclop=10 as new resources land. ## Backend - Re-evaluates the principal's role against the live `MapRoleStore` on every `DELETE` so a downgraded key cannot keep mutating with a still-valid JWT (Codex P1 pattern from earlier admin PRs). - `admin.QueuesSource` is **opt-in**: deployments without `--sqsAddress` leave `/admin/api/v1/sqs/*` off the wire; the SPA renders a soft "endpoint pending" notice on the 404, mirroring the Tables / Buckets `nil` contract. - The bridge in `main_admin.go` (`sqsQueuesBridge`, `convertAdminQueueSummary`, `translateAdminQueuesError`) keeps `internal/admin` free of the heavy adapter dependency tree, same architectural pattern as Dynamo and S3. ## Frontend - **/sqs** queue list with refresh + per-row link to detail. - **/sqs/:name** detail showing FIFO badge, counters card (Visible / In-flight / Delayed), raw attributes table, and a Delete confirmation `Modal` gated by `RequireFullAccess`. - `api/client.ts` gains `listQueues` / `describeQueue` / `deleteQueue` with the same `AbortSignal` pattern used for `cluster` / `dynamo` / `s3` reads. - Layout nav adds an SQS tab between DynamoDB and S3. ## Out of scope (recorded in `docs/design/2026_04_24_proposed_sqs_compatible_adapter.md` Section 14, deferred per the SQS partial doc §16.2) - **PurgeQueue from the SPA** — the underlying `purgeQueueWithRetry` adapter method is on main; the admin entrypoint is a trivial follow-up. - **Send / Peek / CreateQueue from the SPA** — each needs its own SigV4-bypass adapter entrypoint and form UX; deferred to keep this PR focused. ## Test plan - [x] `go build ./...` — clean - [x] `go test -race ./internal/admin/...` — passes - [x] `go test -race -run TestSQS ./adapter/` — passes - [x] `go test -run TestStartAdmin .` — passes - [x] `golangci-lint run ./adapter/... ./internal/admin/... ./...` — `0 issues.`, no `//nolint` - [x] `cd web/admin && npm run build` — 49 modules, 199 KB JS / 61 KB gzip + 14.7 KB CSS - [ ] Manual smoke (after PR lands): start a node with `--sqsAddress :4566 --adminEnabled --adminAllowInsecureDevCookie`, create a queue, send a few messages, hit `/admin/sqs/<name>` → counters match `GetQueueAttributes("All")`, Delete dialog returns to list. ## Self-review (5 lenses) 1. **Data loss** — Delete reuses the existing `deleteQueueWithRetry` OCC path; counters are read-only. No new write paths. 2. **Concurrency** — Per-request leader check on Delete; counters scan uses one snapshot read TS. 3. **Performance** — Counters bounded by main's existing `sqsApproxCounterScanLimit`; admin reads are cheap point lookups + one bounded scan. 4. **Data consistency** — `AdminDescribeQueue` and SigV4 `GetQueueAttributes` both call `scanApproxCounters` at a fresh `nextTxnReadTS`, so a single point in time produces the same counters via either surface. 5. **Test coverage** — Existing admin / SQS race suites stay green via the new `nil` Queues argument added to `startAdminServer` call sites; the new bridge is exercised by the cross-package build itself. ## Stacking This PR is **independent** — branched from current `main` (which has the merged versions of #649 / #658 / #650 / counter implementation). Closing #659 in favour of this clean rewrite.
2 parents 53c687d + 5ee0175 commit b1b7a83

14 files changed

Lines changed: 1346 additions & 35 deletions

File tree

adapter/sqs_admin.go

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
package adapter
2+
3+
import (
4+
"context"
5+
"strconv"
6+
"strings"
7+
"time"
8+
9+
"github.com/cockroachdb/errors"
10+
)
11+
12+
// AdminQueueSummary is the per-queue projection the admin dashboard
13+
// surfaces. It deliberately covers only the fields the SPA renders so
14+
// the package's wire-format types stay internal.
15+
//
16+
// Counters mirror the AWS Approximate* attribute set produced by
17+
// scanApproxCounters; they are best-effort by AWS contract and stop
18+
// counting once the catalog's per-call cap is reached (the SPA polls
19+
// continuously, so an unbounded scan would pin the leader).
20+
type AdminQueueSummary struct {
21+
Name string
22+
IsFIFO bool
23+
Generation uint64
24+
CreatedAt time.Time
25+
Attributes map[string]string
26+
Counters AdminQueueCounters
27+
}
28+
29+
// AdminQueueCounters matches sqsApproxCounters (int64) so the admin
30+
// bridge does not have to convert between widths. Visible /
31+
// NotVisible / Delayed are the AWS Approximate* triple.
32+
type AdminQueueCounters struct {
33+
Visible int64
34+
NotVisible int64
35+
Delayed int64
36+
}
37+
38+
// AdminListQueues returns every queue name this server knows about,
39+
// in the lexicographic order the queue catalog index produces. Read
40+
// path; runs on follower or leader and uses the same scanQueueNames
41+
// helper the SigV4 ListQueues handler does.
42+
func (s *SQSServer) AdminListQueues(ctx context.Context) ([]string, error) {
43+
return s.scanQueueNames(ctx) //nolint:wrapcheck // pure pass-through; the adapter owns the error context.
44+
}
45+
46+
// AdminDescribeQueue returns a snapshot of name's metadata plus the
47+
// approximate counters. The triple (result, present, error) lets
48+
// admin callers distinguish a missing queue from a storage error
49+
// without sniffing sentinels.
50+
//
51+
// Like AdminDescribeTable on the Dynamo side, this entrypoint runs
52+
// on either the leader or a follower (read-only); the counter scan
53+
// uses a fresh nextTxnReadTS so the result is consistent with what
54+
// SigV4 GetQueueAttributes would have returned at the same instant.
55+
func (s *SQSServer) AdminDescribeQueue(ctx context.Context, name string) (*AdminQueueSummary, bool, error) {
56+
if strings.TrimSpace(name) == "" {
57+
return nil, false, ErrAdminSQSValidation
58+
}
59+
readTS := s.nextTxnReadTS(ctx)
60+
meta, exists, err := s.loadQueueMetaAt(ctx, name, readTS)
61+
if err != nil {
62+
return nil, false, errors.WithStack(err)
63+
}
64+
if !exists {
65+
return nil, false, nil
66+
}
67+
counters, err := s.scanApproxCounters(ctx, name, meta.Generation, readTS)
68+
if err != nil {
69+
return nil, false, err
70+
}
71+
return adminQueueSummary(name, meta, counters, s.queueArn(name)), true, nil
72+
}
73+
74+
// adminQueueSummary projects a queue meta + counters into the
75+
// SPA-facing AdminQueueSummary. CreatedAt comes from the canonical
76+
// wall-clock CreatedAtMillis (not CreatedAtHLC, which the meta's own
77+
// comment calls "unsuitable for wall-clock display"); a zero millis
78+
// value yields a zero time.Time so the JSON omitempty drops the field
79+
// and the SPA renders "—" instead of an HLC-derived 1970 epoch.
80+
// queueArn is threaded in by the caller (AdminDescribeQueue) because
81+
// the server's region lives on *SQSServer and the helper itself is
82+
// kept method-free for unit-testability without a coordinator.
83+
// Pulled into a helper so the conversion is unit-testable without
84+
// standing up a full coordinator.
85+
func adminQueueSummary(name string, meta *sqsQueueMeta, counters sqsApproxCounters, queueArn string) *AdminQueueSummary {
86+
var createdAt time.Time
87+
if meta.CreatedAtMillis > 0 {
88+
createdAt = time.UnixMilli(meta.CreatedAtMillis).UTC()
89+
}
90+
return &AdminQueueSummary{
91+
Name: name,
92+
IsFIFO: meta.IsFIFO,
93+
Generation: meta.Generation,
94+
CreatedAt: createdAt,
95+
Attributes: metaAttributesForAdmin(meta, queueArn),
96+
Counters: AdminQueueCounters(counters),
97+
}
98+
}
99+
100+
// AdminDeleteQueue is the SigV4-bypass counterpart to deleteQueue.
101+
// Returns the same sentinel errors as AdminCreateTable on the Dynamo
102+
// side: ErrAdminForbidden on a read-only principal, ErrAdminNotLeader
103+
// on a follower, ErrAdminSQSNotFound when the queue is absent.
104+
func (s *SQSServer) AdminDeleteQueue(ctx context.Context, principal AdminPrincipal, name string) error {
105+
if !principal.Role.canWrite() {
106+
return ErrAdminForbidden
107+
}
108+
if !isVerifiedSQSLeader(s.coordinator) {
109+
return ErrAdminNotLeader
110+
}
111+
if strings.TrimSpace(name) == "" {
112+
return ErrAdminSQSValidation
113+
}
114+
if err := s.deleteQueueWithRetry(ctx, name); err != nil {
115+
// deleteQueueWithRetry returns sqsAPIError with
116+
// sqsErrQueueDoesNotExist when the queue is missing; map
117+
// to the structured ErrAdminSQSNotFound so the admin
118+
// handler can render 404 without sniffing the AWS code.
119+
if isSQSAdminQueueDoesNotExist(err) {
120+
return ErrAdminSQSNotFound
121+
}
122+
return errors.Wrap(err, "admin delete queue")
123+
}
124+
return nil
125+
}
126+
127+
// metaAttributesForAdmin renders the non-counter queue config
128+
// attributes. Mirrors queueMetaToAttributes("All") (sqs_catalog.go)
129+
// except for two deliberate omissions:
130+
//
131+
// - The Approximate* counters — the admin summary surfaces them as
132+
// the typed AdminQueueCounters struct alongside this map, so the
133+
// SPA can render them without round-tripping strings.
134+
// - CreatedTimestamp — surfaced as the typed AdminQueueSummary.CreatedAt
135+
// field for the same reason.
136+
//
137+
// LastModifiedTimestamp stays in the map (SetQueueAttributes updates
138+
// LastModifiedAtMillis and operators need it for change-tracking;
139+
// there is no dedicated typed field for it). QueueArn is included so
140+
// the SPA can show the AWS-shaped identifier without recomputing it
141+
// client-side.
142+
func metaAttributesForAdmin(meta *sqsQueueMeta, queueArn string) map[string]string {
143+
out := map[string]string{
144+
"QueueArn": queueArn,
145+
"VisibilityTimeout": strconv.FormatInt(meta.VisibilityTimeoutSeconds, 10),
146+
"MessageRetentionPeriod": strconv.FormatInt(meta.MessageRetentionSeconds, 10),
147+
"DelaySeconds": strconv.FormatInt(meta.DelaySeconds, 10),
148+
"ReceiveMessageWaitTimeSeconds": strconv.FormatInt(meta.ReceiveMessageWaitSeconds, 10),
149+
"MaximumMessageSize": strconv.FormatInt(meta.MaximumMessageSize, 10),
150+
"FifoQueue": strconv.FormatBool(meta.IsFIFO),
151+
"ContentBasedDeduplication": strconv.FormatBool(meta.ContentBasedDedup),
152+
}
153+
if mod := meta.LastModifiedAtMillis; mod > 0 {
154+
out["LastModifiedTimestamp"] = strconv.FormatInt(mod/sqsMillisPerSecond, 10)
155+
}
156+
if meta.RedrivePolicy != "" {
157+
out["RedrivePolicy"] = meta.RedrivePolicy
158+
}
159+
return out
160+
}
161+
162+
// ErrAdminSQSValidation is returned when an admin entrypoint receives
163+
// a request with a missing or syntactically-bad queue name. Maps to
164+
// 400 in the admin HTTP handler.
165+
var ErrAdminSQSValidation = errors.New("sqs admin: invalid queue name")
166+
167+
// ErrAdminSQSNotFound is returned by write entrypoints when the
168+
// target queue does not exist. Maps to 404. The describe path uses
169+
// the (nil, false, nil) tuple instead of this sentinel for the
170+
// not-found signal, mirroring AdminDescribeTable.
171+
var ErrAdminSQSNotFound = errors.New("sqs admin: queue not found")
172+
173+
// isSQSAdminQueueDoesNotExist matches the deleteQueueWithRetry path's
174+
// "queue does not exist" sqsAPIError so AdminDeleteQueue can normalise
175+
// it to ErrAdminSQSNotFound. Falls through to false on any unrelated
176+
// error, which AdminDeleteQueue then wraps and propagates.
177+
func isSQSAdminQueueDoesNotExist(err error) bool {
178+
var apiErr *sqsAPIError
179+
if !errors.As(err, &apiErr) || apiErr == nil {
180+
return false
181+
}
182+
return apiErr.errorType == sqsErrQueueDoesNotExist
183+
}

adapter/sqs_admin_test.go

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
package adapter
2+
3+
import (
4+
"strconv"
5+
"testing"
6+
"time"
7+
)
8+
9+
const testQueueArn = "arn:aws:sqs:us-east-1:000000000000:orders"
10+
11+
// TestAdminQueueSummary_CreatedAtUsesMillisNotHLC pins the
12+
// invariant that the admin AdminDescribeQueue path derives
13+
// CreatedAt from sqsQueueMeta.CreatedAtMillis (the canonical
14+
// wall-clock field), not from hlcToTime(CreatedAtHLC) — the meta
15+
// struct documents HLC as "unsuitable for wall-clock display" and
16+
// the SigV4 path (sqs_catalog.go:942) reads CreatedAtMillis. Two
17+
// failure modes the test pins:
18+
//
19+
// 1. CreatedAtMillis == 0 must yield a zero time.Time so the JSON
20+
// encoder's omitempty drops the field and the SPA renders "—"
21+
// rather than the HLC-derived 1970-01-01T00:00:00Z.
22+
// 2. CreatedAtMillis > 0 must round-trip through time.UnixMilli in
23+
// UTC.
24+
func TestAdminQueueSummary_CreatedAtUsesMillisNotHLC(t *testing.T) {
25+
t.Parallel()
26+
27+
t.Run("zero millis yields zero time even with HLC populated", func(t *testing.T) {
28+
t.Parallel()
29+
meta := sqsQueueMeta{
30+
Name: "orders",
31+
Generation: 1,
32+
CreatedAtHLC: 42 << s3HLCPhysicalShift, // would render as ~1970 epoch via hlcToTime
33+
// CreatedAtMillis intentionally zero
34+
}
35+
summary := adminQueueSummary("orders", &meta, sqsApproxCounters{}, testQueueArn)
36+
if !summary.CreatedAt.IsZero() {
37+
t.Fatalf("CreatedAt should be zero when CreatedAtMillis==0; got %v", summary.CreatedAt)
38+
}
39+
})
40+
41+
t.Run("positive millis round-trips via time.UnixMilli UTC", func(t *testing.T) {
42+
t.Parallel()
43+
const wantMillis int64 = 1_724_419_200_000 // 2024-08-23T12:00:00Z
44+
meta := sqsQueueMeta{
45+
Name: "orders",
46+
Generation: 2,
47+
CreatedAtMillis: wantMillis,
48+
CreatedAtHLC: 1, // must be ignored
49+
}
50+
summary := adminQueueSummary("orders", &meta, sqsApproxCounters{}, testQueueArn)
51+
want := time.UnixMilli(wantMillis).UTC()
52+
if !summary.CreatedAt.Equal(want) {
53+
t.Fatalf("CreatedAt=%v want=%v", summary.CreatedAt, want)
54+
}
55+
if summary.CreatedAt.Location() != time.UTC {
56+
t.Fatalf("CreatedAt location=%v want UTC", summary.CreatedAt.Location())
57+
}
58+
})
59+
}
60+
61+
// TestMetaAttributesForAdmin_IncludesQueueArnAndLastModified pins
62+
// the parity contract between metaAttributesForAdmin and
63+
// queueMetaToAttributes("All"): QueueArn (the AWS-shaped identifier
64+
// the SPA shows for change-tracking) and LastModifiedTimestamp
65+
// (updated on SetQueueAttributes — the only handle operators have
66+
// on "when did somebody last touch this queue's config") must both
67+
// be present.
68+
func TestMetaAttributesForAdmin_IncludesQueueArnAndLastModified(t *testing.T) {
69+
t.Parallel()
70+
71+
t.Run("QueueArn always present", func(t *testing.T) {
72+
t.Parallel()
73+
meta := sqsQueueMeta{Name: "orders", Generation: 1}
74+
attrs := metaAttributesForAdmin(&meta, testQueueArn)
75+
got, ok := attrs["QueueArn"]
76+
if !ok {
77+
t.Fatalf("QueueArn missing from attributes: %v", attrs)
78+
}
79+
if got != testQueueArn {
80+
t.Fatalf("QueueArn=%q want=%q", got, testQueueArn)
81+
}
82+
})
83+
84+
t.Run("LastModifiedTimestamp emitted in unix seconds when populated", func(t *testing.T) {
85+
t.Parallel()
86+
const wantMillis int64 = 1_724_419_200_000 // 2024-08-23T12:00:00Z
87+
meta := sqsQueueMeta{
88+
Name: "orders",
89+
Generation: 1,
90+
LastModifiedAtMillis: wantMillis,
91+
}
92+
attrs := metaAttributesForAdmin(&meta, testQueueArn)
93+
got, ok := attrs["LastModifiedTimestamp"]
94+
if !ok {
95+
t.Fatalf("LastModifiedTimestamp missing from attributes: %v", attrs)
96+
}
97+
want := strconv.FormatInt(wantMillis/sqsMillisPerSecond, 10)
98+
if got != want {
99+
t.Fatalf("LastModifiedTimestamp=%q want=%q (unix seconds)", got, want)
100+
}
101+
})
102+
103+
t.Run("LastModifiedTimestamp omitted when zero", func(t *testing.T) {
104+
t.Parallel()
105+
meta := sqsQueueMeta{Name: "orders", Generation: 1}
106+
attrs := metaAttributesForAdmin(&meta, testQueueArn)
107+
if _, ok := attrs["LastModifiedTimestamp"]; ok {
108+
t.Fatalf("LastModifiedTimestamp should be omitted when zero: got %q", attrs["LastModifiedTimestamp"])
109+
}
110+
})
111+
112+
t.Run("CreatedTimestamp deliberately not in map (typed field instead)", func(t *testing.T) {
113+
t.Parallel()
114+
meta := sqsQueueMeta{
115+
Name: "orders",
116+
Generation: 1,
117+
CreatedAtMillis: 1_724_419_200_000,
118+
}
119+
attrs := metaAttributesForAdmin(&meta, testQueueArn)
120+
if _, ok := attrs["CreatedTimestamp"]; ok {
121+
t.Fatalf("CreatedTimestamp must NOT be in attrs (it lives on AdminQueueSummary.CreatedAt): got %q", attrs["CreatedTimestamp"])
122+
}
123+
})
124+
}

0 commit comments

Comments
 (0)