test: deflake epochs_optimistic_proving reorg-during-proving gate (#24308)

## Problem

`e2e_epochs/epochs_optimistic_proving.parallel.test.ts` › "handles a
reorg arriving while the top of the epoch is proving" flakes (e.g. CI
runs `a8d5f346e269c668`, `5039f193fa054832`), timing out at:

```
TimeoutError: Timeout awaiting prover-node sees the prune and recreates session with fewer provers
```

## Root cause

The test installs a `beforeTopTreeProve` gate meant to pause top-tree
proving so the L1 reorg lands at a deterministic point (session parked
mid-proof, all sub-trees done). The gate never actually engaged:

- The hook fires from `TopTreeJob.run()` at the `beforeProve` boundary
(`top-tree-job.ts:198`), which runs *after* `EpochSession` has already
flipped the session state from `awaiting-checkpoints` to `awaiting-root`
(`epoch-session.ts:426-431`).
- The gate predicate looked for a job in `awaiting-checkpoints`, so at
hook time it matched nothing and returned early — never blocking.
(`Top-tree proving gated` appears 0 times in both the failed and passed
CI logs.)

With the gate disabled, the reorg raced real, pipelined sub-tree proving
(`top-tree-orchestrator.ts:85-87`), and the outcome depended on epoch
size:

- **Small gated epoch** (epoch 0, 2 checkpoints, near-instant proving):
a sub-tree was still mid-execution when the prune removed its block →
`world-state ... Unable to get meta data for block 2` → the
`EpochSession` caught the error and went terminal `failed`
(`epoch-session.ts:211-218`) → the prune-reconcile
(`recreateInvalidSessions`, `session-manager.ts:266-269`) found it
already terminal and **deleted it without recreating** → the test's wait
for a trimmed session timed out.
- **Large gated epoch** (4 checkpoints): all sub-trees finished before
the prune, so the session was still non-terminal (`awaiting-root`) and
the prune took the clean cancel-then-recreate path — the test passed.

The flake is the small-epoch case, which is exactly the `2 → 1` survivor
transition the test is meant to cover.

## Fix (test only)


`yarn-project/end-to-end/src/e2e_epochs/epochs_optimistic_proving.parallel.test.ts`:

- Query the session for a job in `awaiting-root` (the state it is
actually in when `beforeProve` fires), keeping the `>= 2` checkpoint
condition.
- Resolve a `gateEntered` signal from *inside* the hook, right before
awaiting the gate, so the gate genuinely blocks and the test learns the
gated epoch only once the session is parked.
- Fire the reorg only after `await executeTimeout(() => gateEntered,
...)`, instead of acting on a transient `awaiting-checkpoints`
observation.

This makes the gate do what its comment always claimed:
deterministically park the session at the top-tree boundary (sub-trees
proven, root prove not yet started) before the reorg. That removes the
sub-tree-vs-prune race — when the reorg fires the session is
non-terminal, so the prune always takes the
cancel-and-recreate-with-survivors path the test verifies. No behavioral
assertion is relaxed; the `>= 2` gating and the final "proven up to the
surviving checkpoint" assertions are unchanged. Not a skip, not a
`.test_patterns.yml` entry.

## Verification

- `yarn build`: exit 0; `yarn lint end-to-end`: clean.
- Local run (`ANVIL_PORT=8600`) passed and reproduced the exact
previously-failing condition — it gated epoch 0 with 2 checkpoints, the
gate engaged this time (`Top-tree proving gated for epoch 0`, absent in
both CI logs), and the recreate path ran (`Prover-node trimmed in-flight
session: 2 → 1 tracked checkpoints`), proving up to the surviving
checkpoint.

## Also in this PR

- Refactored the two proving-gate deferred promises to
`promiseWithResolvers` from `@aztec/foundation/promise`, replacing `new
Promise(resolve => { outerVar = resolve })` with escaped `let`
placeholders.
- Added a TypeScript style note to `yarn-project/CLAUDE.md` preferring
`promiseWithResolvers` for promises settled from outside the executor.

## Note

This flake incidentally exposed a separate product edge, tracked
separately and intentionally not bundled into this test fix: if a reorg
prunes a block out from under an in-flight sub-tree, the `EpochSession`
can go terminal `failed`, and `recreateInvalidSessions` drops terminal
sessions without recreating them (unlike the non-terminal
cancel→recreate path).

This is **not a liveness risk**: the network self-heals (a different
prover node proves the epoch, and publishing dedups against the proven
chain), and the affected node itself usually recovers on the next
checkpoint event for the epoch (a path not gated by `lastTickEpoch`) or
on a process restart. It is durably stuck on a single node only in the
narrow case where no further checkpoint event ever arrives for the
epoch, leaving recovery to the periodic tick — which `lastTickEpoch`
blocks. The likely fix is to classify a prune-induced sub-tree read
fault as a cancellation so the session recreates via the existing tested
path, while keeping the `lastTickEpoch` anti-resubmission guard intact.

Author: spalladino

yarn-project/CLAUDE.md

@@ -198,6 +198,8 @@ When working with `AztecAsyncKVStore`, wrap related reads and writes in `store.t
 <general_style>
 Prefer `const` over `let`. Prefer `async`/`await` over `.then()`/`.catch()` callbacks. Named exports only (no default exports). Explicit return types on public API methods; inferred types acceptable on private/internal methods. Only export types needed by external consumers. Avoid `const self = this`; use arrow functions.
 
+When you need a promise whose `resolve`/`reject` are called from outside the executor (deferred gates, signals, manual settlement), use `promiseWithResolvers` from `@aztec/foundation/promise` instead of `new Promise(resolve => { outerVar = resolve })` with an escaped `let`. The helper returns `{ promise, resolve, reject }` directly, avoiding the mutable placeholder.
+
 Prefer high-level collection functions (`find`, `filter`, `map`, helpers from `foundation/src/collection/`) over imperative loops, but prefer imperative loops over `forEach` and complex `reduce`. Prefer `sum(items.map(item => item.value))` over `reduce(...)` for addition.
 
 Simplify function arguments to single expressions where possible. Use expression bodies instead of block bodies when the block only contains a `return`. Block bodies are appropriate when the callback has multiple statements.

yarn-project/end-to-end/src/e2e_epochs/epochs_optimistic_proving.parallel.test.ts

@@ -2,7 +2,9 @@ import type { Logger } from '@aztec/aztec.js/log';
 import { RollupContract } from '@aztec/ethereum/contracts';
 import { BlockNumber, CheckpointNumber, EpochNumber, SlotNumber } from '@aztec/foundation/branded-types';
 import { Fr } from '@aztec/foundation/curves/bn254';
+import { promiseWithResolvers } from '@aztec/foundation/promise';
 import { retryUntil } from '@aztec/foundation/retry';
+import { executeTimeout } from '@aztec/foundation/timer';
 import type { TestProverNode } from '@aztec/prover-node/test';
 import { getEpochAtSlot, getSlotRangeForEpoch } from '@aztec/stdlib/epoch-helpers';
 import type { AztecNode } from '@aztec/stdlib/interfaces/server';
@@ -669,46 +671,50 @@ describe('e2e_epochs/epochs_optimistic_proving', () => {
 
     it('handles a reorg arriving while the top of the epoch is proving', async () => {
       // Gate top-tree proving so it deterministically blocks until we release it. This
-      // gives us a window where the session is mid-proof, and we can fire the reorg
+      // gives us a window where the session is parked at the top-tree boundary (all
+      // sub-trees proven, root prove not yet started), and we can fire the reorg
       // precisely during that window. We use the session's `beforeTopTreeProve` hook
       // rather than monkey-patching the orchestrator factory.
       const proverNode = test.proverNodes[0].getProverNode() as TestProverNode;
-      let releaseProvingGate: () => void = () => {};
-      const provingGate = new Promise<void>(resolve => {
-        releaseProvingGate = resolve;
-      });
-      // Only gate sessions with at least 2 checkpoints — reorging the last checkpoint
-      // of a single-checkpoint epoch leaves nothing to prove, the session is cancelled
-      // without replacement, and the test's "wait for fewer checkpoints" check never
-      // converges. Sessions with one checkpoint just pass through.
-      const sessionHasMultipleCheckpoints = async () => {
-        const job = (await proverNode.getJobs()).find(j => j.status === 'awaiting-checkpoints');
+      const provingGate = promiseWithResolvers<void>();
+      // Resolves with the gated epoch once a session is actually parked inside the gate.
+      // Firing the reorg only after this settles guarantees the session is blocked at the
+      // top-tree boundary — not racing real proving — so the prune deterministically takes
+      // the cancel-and-recreate path instead of failing a half-run sub-tree.
+      const gateEntered = promiseWithResolvers<EpochNumber>();
+      // The hook fires from inside `beforeProve`, by which point the session has already
+      // transitioned from `awaiting-checkpoints` to `awaiting-root`; query that state to
+      // find the gateable session. Only gate sessions with at least 2 checkpoints —
+      // reorging the last checkpoint of a single-checkpoint epoch leaves nothing to prove,
+      // the session is cancelled without replacement, and the test's "wait for fewer
+      // checkpoints" check never converges. Sessions with one checkpoint just pass through.
+      const findGateableEpoch = () => {
+        const job = proverNode.sessionManager.getJobs().find(j => j.status === 'awaiting-root');
         const session = job && proverNode.sessionManager.getFullSession(job.epochNumber);
-        return !!session && session.getCheckpoints().length >= 2;
+        return session && session.getCheckpoints().length >= 2 ? job!.epochNumber : undefined;
       };
       proverNode.setSessionHooks({
         beforeTopTreeProve: async () => {
-          if (!(await sessionHasMultipleCheckpoints())) {
+          const epoch = findGateableEpoch();
+          if (epoch === undefined) {
             return;
           }
-          logger.warn('Top-tree proving gated — waiting for test to release');
-          await provingGate;
+          logger.warn(`Top-tree proving gated for epoch ${epoch} — waiting for test to release`);
+          gateEntered.resolve(epoch);
+          await provingGate.promise;
           logger.warn('Proving gate released');
         },
       });
 
-      // Wait for a session with at least 2 checkpoints to hit the gate. The session
-      // manager opens one full session at a time, starting with the lowest unproven
-      // epoch; small epochs pass through (see hook above) and we keep polling until a
-      // gateable epoch lands. `getJobs()` tells us which epoch is actually blocked.
-      await retryUntil(
-        sessionHasMultipleCheckpoints,
+      // Wait until a session with at least 2 checkpoints is actually parked inside the
+      // gate. The session manager opens one full session at a time, starting with the
+      // lowest unproven epoch; small epochs pass through (see hook above) and we keep
+      // waiting until a gateable epoch lands.
+      const gatedEpoch = await executeTimeout(
+        () => gateEntered.promise,
+        L2_SLOT_DURATION_IN_S * 12 * 1000,
         'gateable session blocks at proving gate',
-        L2_SLOT_DURATION_IN_S * 12,
-        0.5,
       );
-      const gatedJob = (await proverNode.getJobs()).find(j => j.status === 'awaiting-checkpoints')!;
-      const gatedEpoch = gatedJob.epochNumber;
       logger.info(`Job for epoch ${gatedEpoch} is blocked inside proving — firing reorg now`);
 
       // Capture the in-flight session and the last checkpoint of the gated epoch —
@@ -771,7 +777,7 @@ describe('e2e_epochs/epochs_optimistic_proving', () => {
       // Release the gate. The cancelled top tree #1 short-circuits with
       // TopTreeCancelledError, the finalize loop restarts with the surviving sub-trees,
       // and a fresh top tree submits a valid proof for checkpoints 1..afterReorgCheckpoint.
-      releaseProvingGate();
+      provingGate.resolve();
 
       // The in-flight epoch should now be proven on L1
       await test.waitUntilProvenCheckpointNumber(afterReorgCheckpoint, 240);