Skip to content

Commit 56a547c

Browse files
committed
admin: ship S3 bucket write endpoints (P2 slice 2a)
Slice 2a of P2 (docs/design/2026_04_24_proposed_admin_dashboard.md Section 4.1): POST /admin/api/v1/s3/buckets, PUT /admin/api/v1/s3/buckets/{name}/acl, and DELETE /admin/api/v1/s3/buckets/{name} become reachable so the SPA's S3 write modals stop receiving 405. Slice 2b will plumb AdminForward so a follower can hand these writes off to the leader transparently. Adapter (`adapter/s3_admin.go`): - `AdminCreateBucket` performs the same atomic bucket-meta + ACL + generation-key txn the SigV4 createBucket path does, with three in-method guards: principal must be AdminRoleFull, the local node must be the verified S3 leader, and bucket-name + ACL must pass the existing validators. The owner field is now populated with the caller's access key so the audit-log breadcrumb survives storage. - `AdminPutBucketAcl` mutates only `meta.Acl` in place, preserving generation so existing object references remain valid. - `AdminDeleteBucket` rejects non-empty buckets with `ErrAdminBucketNotEmpty` (mirrors the SigV4 path's `BucketNotEmpty`); the dashboard cannot force a recursive delete by design. - New sentinels: `ErrAdminBucketAlreadyExists`, `ErrAdminBucketNotFound`, `ErrAdminBucketNotEmpty`, `ErrAdminInvalidBucketName`, `ErrAdminInvalidACL`. Reuses the shared `ErrAdminForbidden` and `ErrAdminNotLeader` from the Dynamo side. Admin package: - `BucketsSource` interface gains `AdminCreateBucket` / `AdminPutBucketAcl` / `AdminDeleteBucket`. New types `CreateBucketRequest` and `PutBucketACLRequest` carry the documented JSON shapes. - New sentinels `ErrBucketsNotEmpty`, `ErrBucketsAlreadyExists` (the former is new for slice 2a; the latter was reserved earlier and is now wired). - `S3Handler` now routes POST/PUT/DELETE through dedicated per-method handlers with `principalForWrite` re-validating the role on every request against the live `MapRoleStore`. The route switch was refactored into `serveCollection` + `servePerBucket` to keep the cyclomatic budget under control as the surface grew. - `handleCreate` / `handlePutACL` / `handleDelete` log a structured `admin_audit` line on success matching the Dynamo handler's shape. - `decodeAdminS3JSONBody` is the strict shared decoder (`DisallowUnknownFields`, NUL-byte rejection, trailing-token rejection, 64 KiB cap) used by both POST and PUT. - `writeBucketsError` translates the source-side sentinels into the design's HTTP statuses (403 / 503+Retry-After:1 / 404 / 409 / 400 invalid_request via `*ValidationError`). Bridge (`main_admin.go`): - `bucketsBridge` gains write methods that call the adapter's `Admin*` methods and run their errors through `translateAdminBucketsError` — the same pattern as `translateAdminTablesError`, with the leader-churn fallback routing kv-internal sentinels to `admin.ErrBucketsNotLeader` so the SPA's retry contract stays intact. Tests: - 19 admin-package tests covering: create happy path / read-only rejected / 409 already-exists / 503 not-leader / 400 invalid JSON shapes (5 sub-cases) / NUL-byte / 413 oversize / put_acl happy + read-only + missing field + 404 + non-PUT method-not-allowed / delete happy + read-only + 409 not-empty + 404 + 503 not-leader / cross-method missing-principal 401. - 10 adapter-level tests against the in-memory MVCC store covering the same surfaces from the storage side: happy path round-trips through describe, ACL default to private, role / leader / ACL / bucket-name validation rejections, AlreadyExists / NotFound / NotEmpty paths. - Existing `TestS3Handler_DescribeBucket_SubpathReturns404` was superseded by two more precise tests: `TestS3Handler_PerBucket_UnknownSubpathReturns404` (any non-/acl sub-path 404s) and `TestS3Handler_PerBucket_AclSubpathRejectsGet` (the now-valid /acl path returns 405 on GET).
1 parent ee521be commit 56a547c

7 files changed

Lines changed: 1286 additions & 63 deletions

File tree

adapter/s3_admin.go

Lines changed: 240 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,10 @@ import (
44
"bytes"
55
"context"
66
"sort"
7+
"strings"
78

89
"github.com/bootjp/elastickv/internal/s3keys"
10+
"github.com/bootjp/elastickv/kv"
911
"github.com/bootjp/elastickv/store"
1012
"github.com/cockroachdb/errors"
1113
)
@@ -175,3 +177,241 @@ func summaryFromBucketMeta(name string, meta *s3BucketMeta) AdminBucketSummary {
175177
Owner: meta.Owner,
176178
}
177179
}
180+
181+
// Sentinel errors the admin write methods return so the bridge in
182+
// main_admin.go can translate them into admin-package vocabulary
183+
// without sniffing strings. Named separately from
184+
// ErrAdminTableAlreadyExists / ErrAdminTableNotFound on the Dynamo
185+
// side so a future per-resource role / status divergence does not
186+
// require renaming both packages' callers.
187+
var (
188+
// ErrAdminBucketAlreadyExists signals that AdminCreateBucket
189+
// targeted a name already in use. Maps to 409 Conflict.
190+
ErrAdminBucketAlreadyExists = errors.New("s3 admin: bucket already exists")
191+
// ErrAdminBucketNotFound signals that AdminDeleteBucket /
192+
// AdminPutBucketAcl targeted a missing bucket. Maps to 404.
193+
ErrAdminBucketNotFound = errors.New("s3 admin: bucket not found")
194+
// ErrAdminBucketNotEmpty signals that AdminDeleteBucket targeted
195+
// a bucket that still has objects. Maps to 409 Conflict to match
196+
// the SigV4 path's BucketNotEmpty response (the dashboard cannot
197+
// force a recursive delete; the operator must clean up first).
198+
ErrAdminBucketNotEmpty = errors.New("s3 admin: bucket is not empty")
199+
// ErrAdminInvalidBucketName signals that AdminCreateBucket got
200+
// a name that does not satisfy validateS3BucketName. Maps to 400.
201+
ErrAdminInvalidBucketName = errors.New("s3 admin: invalid bucket name")
202+
// ErrAdminInvalidACL signals that the ACL string did not pass
203+
// validateS3CannedAcl. Maps to 400 (the SigV4 path returns 501
204+
// NotImplemented for unsupported canned ACLs, but the admin API
205+
// is documented as private/public-read only and rejecting other
206+
// values as invalid input is a more useful contract for the
207+
// dashboard).
208+
ErrAdminInvalidACL = errors.New("s3 admin: invalid ACL")
209+
)
210+
211+
// AdminCreateBucket creates a bucket on behalf of the admin
212+
// dashboard. The principal MUST be re-validated by the caller (the
213+
// admin HTTP handler does this against the live RoleStore); this
214+
// method enforces the authorisation invariant a second time so a
215+
// follower-forwarded call cannot smuggle a read-only principal past
216+
// the check on the leader side (Section 3.2 "認可の真実は常に
217+
// adapter 側").
218+
//
219+
// The transaction is atomic: bucket meta + generation + ACL all land
220+
// in a single OperationGroup, mirroring the SigV4 createBucket path.
221+
// On success returns the freshly-stored summary; on conflict returns
222+
// ErrAdminBucketAlreadyExists; on a non-leader / non-full-role / bad
223+
// input returns the corresponding sentinel.
224+
func (s *S3Server) AdminCreateBucket(ctx context.Context, principal AdminPrincipal, name, acl string) (*AdminBucketSummary, error) {
225+
if !principal.Role.canWrite() {
226+
return nil, ErrAdminForbidden
227+
}
228+
if !s.isVerifiedS3Leader() {
229+
return nil, ErrAdminNotLeader
230+
}
231+
if err := validateS3BucketName(name); err != nil {
232+
return nil, errors.Wrapf(ErrAdminInvalidBucketName, "%s", err.Error())
233+
}
234+
acl = adminCanonicalACL(acl)
235+
if err := validateS3CannedAcl(acl); err != nil {
236+
return nil, errors.Wrapf(ErrAdminInvalidACL, "%s", err.Error())
237+
}
238+
239+
var summary *AdminBucketSummary
240+
err := s.retryS3Mutation(ctx, func() error {
241+
out, err := s.adminCreateBucketTxn(ctx, principal, name, acl)
242+
if err != nil {
243+
return err
244+
}
245+
summary = out
246+
return nil
247+
})
248+
if err != nil {
249+
return nil, err //nolint:wrapcheck // sentinel errors propagate as-is; structured errors are already wrapped above.
250+
}
251+
return summary, nil
252+
}
253+
254+
// adminCreateBucketTxn is the per-attempt body retryS3Mutation
255+
// invokes. Pulled out so AdminCreateBucket stays under the
256+
// cyclomatic ceiling without hiding the bucket-existence /
257+
// generation / commit-ts dance — every step has a meaningful
258+
// error path that the wrapping retry harness needs to see.
259+
func (s *S3Server) adminCreateBucketTxn(ctx context.Context, principal AdminPrincipal, name, acl string) (*AdminBucketSummary, error) {
260+
readTS := s.readTS()
261+
startTS := s.txnStartTS(readTS)
262+
readPin := s.pinReadTS(readTS)
263+
defer readPin.Release()
264+
265+
existing, exists, err := s.loadBucketMetaAt(ctx, name, readTS)
266+
if err != nil {
267+
return nil, errors.WithStack(err)
268+
}
269+
if exists && existing != nil {
270+
return nil, ErrAdminBucketAlreadyExists
271+
}
272+
nextGeneration, err := s.nextBucketGenerationAt(ctx, name, readTS)
273+
if err != nil {
274+
return nil, errors.WithStack(err)
275+
}
276+
commitTS, err := s.nextTxnCommitTS(startTS)
277+
if err != nil {
278+
return nil, errors.WithStack(err)
279+
}
280+
meta := &s3BucketMeta{
281+
BucketName: name,
282+
Generation: nextGeneration,
283+
CreatedAtHLC: commitTS,
284+
Region: s.effectiveRegion(),
285+
Owner: principal.AccessKey,
286+
Acl: acl,
287+
}
288+
body, err := encodeS3BucketMeta(meta)
289+
if err != nil {
290+
return nil, errors.WithStack(err)
291+
}
292+
_, err = s.coordinator.Dispatch(ctx, &kv.OperationGroup[kv.OP]{
293+
IsTxn: true,
294+
StartTS: startTS,
295+
CommitTS: commitTS,
296+
Elems: []*kv.Elem[kv.OP]{
297+
{Op: kv.Put, Key: s3keys.BucketMetaKey(name), Value: body},
298+
{Op: kv.Put, Key: s3keys.BucketGenerationKey(name), Value: encodeS3Generation(nextGeneration)},
299+
},
300+
})
301+
if err != nil {
302+
return nil, errors.WithStack(err)
303+
}
304+
out := summaryFromBucketMeta(name, meta)
305+
return &out, nil
306+
}
307+
308+
// AdminPutBucketAcl swaps the canned ACL on an existing bucket.
309+
// Same authorisation contract as AdminCreateBucket. Mutates only
310+
// the meta.Acl field; generation is preserved so existing object
311+
// references stay valid.
312+
func (s *S3Server) AdminPutBucketAcl(ctx context.Context, principal AdminPrincipal, name, acl string) error {
313+
if !principal.Role.canWrite() {
314+
return ErrAdminForbidden
315+
}
316+
if !s.isVerifiedS3Leader() {
317+
return ErrAdminNotLeader
318+
}
319+
acl = adminCanonicalACL(acl)
320+
if err := validateS3CannedAcl(acl); err != nil {
321+
return errors.Wrapf(ErrAdminInvalidACL, "%s", err.Error())
322+
}
323+
324+
err := s.retryS3Mutation(ctx, func() error {
325+
readTS := s.readTS()
326+
startTS := s.txnStartTS(readTS)
327+
readPin := s.pinReadTS(readTS)
328+
defer readPin.Release()
329+
330+
meta, exists, err := s.loadBucketMetaAt(ctx, name, readTS)
331+
if err != nil {
332+
return errors.WithStack(err)
333+
}
334+
if !exists || meta == nil {
335+
return ErrAdminBucketNotFound
336+
}
337+
meta.Acl = acl
338+
body, err := encodeS3BucketMeta(meta)
339+
if err != nil {
340+
return errors.WithStack(err)
341+
}
342+
_, err = s.coordinator.Dispatch(ctx, &kv.OperationGroup[kv.OP]{
343+
IsTxn: true,
344+
StartTS: startTS,
345+
Elems: []*kv.Elem[kv.OP]{
346+
{Op: kv.Put, Key: s3keys.BucketMetaKey(name), Value: body},
347+
},
348+
})
349+
return errors.WithStack(err)
350+
})
351+
if err != nil {
352+
return err //nolint:wrapcheck // sentinel errors propagate as-is.
353+
}
354+
return nil
355+
}
356+
357+
// AdminDeleteBucket removes a bucket if it is empty. Same
358+
// authorisation contract as the other admin write methods. The
359+
// bucket-must-be-empty rule mirrors the SigV4 deleteBucket path —
360+
// the dashboard cannot force a recursive delete, by design.
361+
func (s *S3Server) AdminDeleteBucket(ctx context.Context, principal AdminPrincipal, name string) error {
362+
if !principal.Role.canWrite() {
363+
return ErrAdminForbidden
364+
}
365+
if !s.isVerifiedS3Leader() {
366+
return ErrAdminNotLeader
367+
}
368+
369+
err := s.retryS3Mutation(ctx, func() error {
370+
readTS := s.readTS()
371+
startTS := s.txnStartTS(readTS)
372+
readPin := s.pinReadTS(readTS)
373+
defer readPin.Release()
374+
375+
meta, exists, err := s.loadBucketMetaAt(ctx, name, readTS)
376+
if err != nil {
377+
return errors.WithStack(err)
378+
}
379+
if !exists || meta == nil {
380+
return ErrAdminBucketNotFound
381+
}
382+
start := s3keys.ObjectManifestPrefixForBucket(name, meta.Generation)
383+
kvs, err := s.store.ScanAt(ctx, start, prefixScanEnd(start), 1, readTS)
384+
if err != nil {
385+
return errors.WithStack(err)
386+
}
387+
if len(kvs) > 0 {
388+
return ErrAdminBucketNotEmpty
389+
}
390+
_, err = s.coordinator.Dispatch(ctx, &kv.OperationGroup[kv.OP]{
391+
IsTxn: true,
392+
StartTS: startTS,
393+
Elems: []*kv.Elem[kv.OP]{
394+
{Op: kv.Del, Key: s3keys.BucketMetaKey(name)},
395+
},
396+
})
397+
return errors.WithStack(err)
398+
})
399+
if err != nil {
400+
return err //nolint:wrapcheck // sentinel errors propagate as-is.
401+
}
402+
return nil
403+
}
404+
405+
// adminCanonicalACL normalises an empty input to the canned
406+
// "private" default. The SigV4 createBucket / putBucketAcl paths
407+
// apply the same default after trimming the x-amz-acl header.
408+
// Pulled out so the admin write methods do not silently accept a
409+
// blank string and create / mutate with whatever validateS3CannedAcl
410+
// happens to allow on its empty branch.
411+
func adminCanonicalACL(acl string) string {
412+
trimmed := strings.TrimSpace(acl)
413+
if trimmed == "" {
414+
return s3AclPrivate
415+
}
416+
return trimmed
417+
}

0 commit comments

Comments
 (0)