Commit 1fd63ef
authored
test: fix flaky sentinel_status_slash by asserting the fault on the checkpoint slot (#23483)
## Summary
Root-cause fix for the `sentinel_status_slash.parallel.test.ts` flake
(added in #23286) instead of skipping it. The two malicious-proposer
tests now discover the slot at which the fault is actually recorded
rather than asserting it on the pre-warped block-proposer slot.
(This PR previously carried a `.test_patterns.yml` skip on `next`; per
maintainer request the skip is dropped and the PR now carries the fix on
`merge-train/spartan`, where the test lives.)
## Root cause
The failing merge-train run (test log
http://ci.aztec-labs.com/83589a3b1ce4a8b7) shows every honest observer
recording the target proposer as `checkpoint-missed` at the warped slot
(15), with zero `checkpoint-invalid` / `checkpoint-unvalidated` /
`checkpoint-valid` for it anywhere — the malicious node didn't even
self-record `checkpoint-valid` for that slot.
`checkpoint-missed` means *block proposals were seen for the slot, but
no checkpoint-proposal re-execution outcome was recorded for it*
(`sentinel.ts` `getSlotActivity`). The re-execution outcome that drives
`checkpoint-invalid` / `checkpoint-unvalidated` is keyed by
`proposal.slotNumber` — the slot at which a node *closes a checkpoint*
(`proposal_handler.ts:887`). A proposer self-records `checkpoint-valid`
only when it actually builds a checkpoint proposal.
`warpToSlotBeforeTargetProposer` warps to the target's *block-proposer*
slot and the test then asserts the fault at exactly that slot. But a
proposer emits a checkpoint proposal only when its slot closes a
checkpoint, and that does not reliably coincide with the warped
block-proposer slot. When the warped slot is mid-checkpoint, no outcome
is ever keyed to it → `checkpoint-missed`. Whether the proposer slot
lines up with a checkpoint close shifts with pipelining/wall-clock
timing, so it flakes (it fails on every retry, so the
`e2e-p2p-epoch-flakes` group does not absorb it).
## Fix
Stop pinning the pre-warped slot. New helper `findObservedStatusSlot`
polls an observer for the slot at which it records the expected fault
status (`checkpoint-unvalidated` / `checkpoint-invalid`); the test then
requires every honest observer to agree at that slot and the malicious
node to self-record `checkpoint-valid` for the same slot. The warp is
kept purely to cut wall-clock by advancing near the proposer region.
This stays faithful to what the test verifies: it still fails (times
out) if the malicious proposal is never detected, so it does not mask a
genuine propagation/detection bug — it only removes the brittle
assumption that the fault lands on one specific pre-chosen slot.
## Verification
- Type/usage review only: `findObservedStatusSlot` reuses the same
`getValidatorsStats()` → `history` shape and `retryUntil` pattern as the
existing helpers; `SlotNumber` / `ValidatorStatusInSlot` / `retryUntil`
are already imported.
- **Not run locally**: this is an intermittent e2e p2p flake; a single
local run would not establish flake-freedom and a full build + repeated
e2e grind is impractical to do reliably in-session. CI on this draft
exercises the test; a rerun grind is the real confirmation bar.
## Related
- Replaces the skip approach (the prior content of this PR and the
now-closed #23443).
- Adjacent to `palla/fix-sentinel-inactivity-want-to-slash` (changes the
sentinel inactivity computation), which is independent of this
test-timing fix.1 parent 2a2f49e commit 1fd63ef
1 file changed
Lines changed: 43 additions & 11 deletions
Lines changed: 43 additions & 11 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
140 | 140 | | |
141 | 141 | | |
142 | 142 | | |
143 | | - | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
144 | 149 | | |
145 | 150 | | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
151 | 157 | | |
152 | 158 | | |
153 | 159 | | |
| |||
156 | 162 | | |
157 | 163 | | |
158 | 164 | | |
159 | | - | |
| 165 | + | |
160 | 166 | | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | | - | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
165 | 172 | | |
166 | 173 | | |
167 | 174 | | |
| |||
297 | 304 | | |
298 | 305 | | |
299 | 306 | | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
300 | 332 | | |
301 | 333 | | |
302 | 334 | | |
| |||
0 commit comments