Commit 63bfe8d
committed
backup: DynamoDB encoder for tables and items (Phase 0a)
Builds on PR #714. Adds the DynamoDB encoder for the Phase 0
logical-backup decoder.
Snapshot prefixes handled:
- !ddb|meta|table|<base64url(table)> -> dynamodb/<table>/_schema.json
(DynamoTableSchema proto -> DescribeTable-shaped JSON; cluster-
internal fields like key_encoding_version and migrating_from_generation
stripped from the public projection because they're not user-visible).
- !ddb|item|<base64url(table)>|<gen>|<rest> -> per-item JSON files
under dynamodb/<table>/items/. Hash-only tables emit
items/<pk>.json; composite-key tables emit items/<pk>/<sk>.json.
- !ddb|gsi|... ignored (derivable from items + schema; replaying GSI
rows on restore would conflict with the destination's own index
maintenance).
- !ddb|meta|gen|... ignored (operational counter, not user state).
Implementation choices:
- Lex order ('i' < 'm') means items arrive before the table schema.
Encoder buffers per encoded-table-segment and emits at Finalize
once the schema is known, parallel to the SQS encoder's strategy.
- Table-segment parsing is unambiguous: base64url alphabet contains
no '|', so the first '|' after the prefix is the table/gen
separator. No heuristic boundary detection needed.
- Item filename derivation reads the hash and range key NAMES from
the schema, then looks them up in the item's attributes map. A
missing required-key attribute on an item is structural error
(it could never have been GetItem-able) and surfaces as
ErrDDBInvalidItem on Finalize.
- B-attribute (binary) primary keys take EncodeBinarySegment so they
cannot collide with hex-shaped string keys; matches the design's
"binary keys take b64.<base64url>" rule.
- All 10 documented attribute kinds (S, N, B, BOOL, NULL, SS, NS,
BS, L, M) are translated to their AWS-DynamoDB-JSON shapes.
Empty oneof (a malformed proto value) surfaces as NULL=true so
the dump remains deserialisable.
- attributeValueToPublic is split into scalar / set / composite
helpers so the cyclomatic complexity stays under the package cap.
- Bundle mode (--dynamodb-bundle-mode jsonl) is left as a stub:
WithBundleJSONL(true) makes Finalize return a clear "not
implemented in this PR" error so the master pipeline can
surface the capability gap. Per-item layout is the documented
default.
Tests cover hash-only and composite-key round-trips, binary primary
key rendering, orphan items without schema, value-magic rejection
on schema and item paths, missing-key-attribute rejection at
Finalize, GSI rows ignored, all 10 attribute kinds round-trip
through JSON, empty-oneof -> NULL, bundle-mode stub, malformed
table-segment key rejection.1 parent 796a42f commit 63bfe8d
2 files changed
Lines changed: 825 additions & 0 deletions
0 commit comments