Skip to content

Commit 07b2fd9

Browse files
committed
docs(encryption): address PR707 round-21 codex P1+P2 (cutover-bypass + epoch wire type)
P1 (codex on 5e27f2d line 1804): §7.1 Phase 2 step 7 quiescence barrier blocked engine.Propose globally before submitting the cutover entry, but step 3 of the same sequence is itself an engine.Propose call -- so the literal implementation would reject its own cutover proposal and leave the cluster stuck in a write-rejecting barrier forever. Specify a per-call source tag: USER proposals from coordinators are blocked, but the encryption-admin path (cutover proposals + RegisterEncryptionWriter triggered by ConfChangeAddLearner mid-cutover) bypasses with source = "encryption_admin". P2 (codex on 5e27f2d line 1317): §6.1 RPC schema used uint32 for local_epoch but the §4.1 nonce field reserves only 16 bits. Without explicit decode-side validation, a corrupted/malicious response carrying a value > 0xFFFF would silently truncate when copied into the nonce field, breaking the monotonicity invariant the round-18 ErrLocalEpochRollback was added to enforce. Added explicit "value <= 0xFFFF or ErrLocalEpochOutOfRange" rule applied symmetrically to CapabilityReport.local_epoch and every value in SidecarStateReport.writer_registry_for_caller. New ErrLocalEpochOutOfRange added to §9.1 startup refusal list (and applies on every RPC decode site).
1 parent 5e27f2d commit 07b2fd9

1 file changed

Lines changed: 47 additions & 7 deletions

File tree

docs/design/2026_04_29_proposed_data_at_rest_encryption.md

Lines changed: 47 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1314,7 +1314,7 @@ pre-conditions and idempotency rules are different.
13141314
string build_sha = 2;
13151315
bool sidecar_present = 3;
13161316
uint64 full_node_id = 4; // for §5.6 batch registry
1317-
uint32 local_epoch = 5; // for §5.6 batch registry
1317+
uint32 local_epoch = 5; // 16-bit value -- see decode rule
13181318
}
13191319
message SidecarStateReport {
13201320
map<uint32, bytes> wrapped_deks_by_id = 1; // every unretired DEK
@@ -1323,12 +1323,26 @@ pre-conditions and idempotency rules are different.
13231323
bool storage_envelope_active = 4;
13241324
uint64 raft_envelope_cutover_index = 5;
13251325
uint64 latest_applied_index = 6;
1326-
map<uint32, uint32> writer_registry_for_caller = 7; // dek_id → last_seen_local_epoch for caller's uint16(node_id), used by §5.5 to forbid local_epoch rollback after resync
1326+
map<uint32, uint32> writer_registry_for_caller = 7; // dek_id → last_seen_local_epoch for caller's uint16(node_id), 16-bit value
13271327
}
13281328
rpc GetCapability(Empty) returns (CapabilityReport);
13291329
rpc GetSidecarState(Empty) returns (SidecarStateReport);
13301330
```
13311331

1332+
**`local_epoch` wire-type rule.** The §4.1 nonce field
1333+
reserves 16 bits for `local_epoch`; protobuf has no native
1334+
`uint16`, so the wire type is `uint32` to keep the schema
1335+
language-portable. Every decode site MUST validate `value
1336+
<= 0xFFFF` and return `ErrLocalEpochOutOfRange` otherwise.
1337+
Without this validation, a corrupted or maliciously-crafted
1338+
RPC response carrying a value above `0xFFFF` would silently
1339+
truncate when copied into the 16-bit nonce field, breaking
1340+
the §4.1 monotonicity invariant and re-issuing previously-
1341+
used nonces — exactly the rollback failure mode that
1342+
`ErrLocalEpochRollback` was added to prevent. The rule
1343+
applies symmetrically to `CapabilityReport.local_epoch` and
1344+
to every value in `SidecarStateReport.writer_registry_for_caller`.
1345+
13321346
`GetSidecarState` is the §5.5 compaction-fallback RPC: any
13331347
node can request the leader's full encryption state to
13341348
rebuild a sidecar that fell behind a Raft-log compaction
@@ -1797,18 +1811,36 @@ flag becomes a *capability* assertion, not a *behaviour* trigger.
17971811

17981812
```text
17991813
leader_enable_raft_envelope():
1800-
1. block new proposal intake at the engine.Propose entrypoint
1801-
(return ErrEnvelopeCutoverInProgress to coordinator)
1814+
1. block new USER proposal intake at engine.Propose
1815+
(return ErrEnvelopeCutoverInProgress to client coordinators).
1816+
The block is keyed on a per-call "source" tag the
1817+
coordinator already passes; the encryption-admin path that
1818+
proposes the cutover entry below uses source = "encryption_admin"
1819+
and bypasses the gate. Without this exemption, step 3 below
1820+
would reject its own cutover proposal and the cluster would
1821+
stay stuck in a write-rejecting barrier forever.
18021822
2. wait for the in-flight proposal queue to drain (all
18031823
previously-accepted proposals committed and applied)
1804-
3. propose the enable-raft-envelope entry (raftEncodeEncryptionRotation
1805-
= 0x05, NOT raft-DEK-wrapped — see below)
1824+
3. encryption-admin path proposes the enable-raft-envelope
1825+
entry (raftEncodeEncryptionRotation = 0x05, NOT
1826+
raft-DEK-wrapped — see below). Because this call carries
1827+
source = "encryption_admin" the step-1 gate lets it
1828+
through.
18061829
4. wait for that entry to commit AND for the local FSM
18071830
apply to set raft_envelope_cutover_index in the sidecar
18081831
5. flip the leader's "wrap on Propose" switch to true
1809-
6. unblock proposal intake
1832+
6. unblock USER proposal intake
18101833
```
18111834

1835+
The narrower "user vs encryption-admin" gate (rather than a
1836+
global Propose mutex) is the load-bearing detail: a global
1837+
gate would deadlock the cluster on its own cutover proposal.
1838+
The same source-tag exemption applies to any future internal
1839+
proposal that must be issued during a quiesced cutover —
1840+
for example, a `RegisterEncryptionWriter` triggered by
1841+
`ConfChangeAddLearner` mid-cutover (§4.1 fourth path) also
1842+
uses `source = "encryption_admin"` and bypasses the gate.
1843+
18121844
Without the barrier, in a busy leader cleartext proposals
18131845
accepted between step 3 (proposing the cutover entry) and
18141846
step 5 (flipping the wrap-on-propose switch) could be
@@ -2152,6 +2184,14 @@ The process refuses to start if any of the following hold:
21522184
already been used under the same DEK. Recovery is either a
21532185
manual `local_epoch` bump above the registry record or a
21542186
DEK rotation (which retires the registry slice).
2187+
- Any `local_epoch` value received over the
2188+
`EncryptionAdmin.GetCapability` or
2189+
`EncryptionAdmin.GetSidecarState` RPCs exceeds `0xFFFF`
2190+
(`ErrLocalEpochOutOfRange`). The wire type is `uint32` for
2191+
language portability per §6.1, but the §4.1 nonce field
2192+
reserves only 16 bits; silently truncating a larger value
2193+
would break the monotonic-epoch invariant that prevents
2194+
nonce reuse.
21552195
- The local sidecar's `raft_envelope_cutover_index` disagrees
21562196
with the value carried in the most-recent ingested snapshot
21572197
header (`ErrEnvelopeCutoverDivergence`). This catches a node

0 commit comments

Comments
 (0)