|
| 1 | +# Handoff: `private_kernel_init_3` is wired end-to-end |
| 2 | + |
| 3 | +`private_kernel_init_3` — the batched first-iteration kernel that verifies three app |
| 4 | +calls in a single circuit — is now plumbed end-to-end behind a `PXE_USE_INIT_3` |
| 5 | +flag. The new path captures, proves, and verifies on real client flows, with |
| 6 | +measured savings in the expected shape (saving 2 prev-kernel HN verifications + |
| 7 | +2 Oink proves per tx). |
| 8 | + |
| 9 | +This branch is `lde/integrate-init-3`, based on `merge-train/barretenberg`. |
| 10 | + |
| 11 | +## What's done — 7 commits |
| 12 | + |
| 13 | +``` |
| 14 | +69362fbe test(end-to-end): reproducer for the PXE_USE_INIT_3 canary |
| 15 | +397224be feat(pxe): orchestrator init_3 path behind PXE_USE_INIT_3 flag |
| 16 | +43f203c1 feat(bb-prover): implement generateInit3Output / simulateInit3 |
| 17 | +3bcad728 feat(noir-protocol-circuits-types): wire private_kernel_init_3 artifact and VK |
| 18 | +7de74078 feat(stdlib): add PrivateKernelInit3CircuitPrivateInputs and prover interface methods |
| 19 | +0dc31e27 docs(barretenberg/cpp): note that find-bb defaults to bb-avm |
| 20 | +94b52e31 feat(protocol-circuits): allocate PRIVATE_KERNEL_INIT_3_VK_INDEX and accept it as a previous kernel |
| 21 | +``` |
| 22 | + |
| 23 | +Layered top-down, each commit corresponds to one layer of the stack: |
| 24 | + |
| 25 | +| Commit | Layer | What changed | |
| 26 | +|---|---|---| |
| 27 | +| `94b52e31` | Noir protocol circuits | `PRIVATE_KERNEL_INIT_3_VK_INDEX = 65`, extends `ALLOWED_PREVIOUS_CIRCUITS` for inner / reset / tail / tail_to_public | |
| 28 | +| `7de74078` | TS stdlib | New `PrivateKernelInit3CircuitPrivateInputs`, new `generateInit3Output` / `simulateInit3` on `PrivateKernelProver` | |
| 29 | +| `3bcad728` | TS noir-protocol-circuits-types | New `PrivateKernelInit3Artifact` in the union, bundle, vks/client, vks/server. Simulated lookup reuses the constrained artifact (no `private-kernel-init-3-simulated` crate exists) | |
| 30 | +| `43f203c1` | TS bb-prover | New witness-map converters + `BBPrivateKernelProver` methods | |
| 31 | +| `397224be` | TS PXE orchestrator | `useInit3 = process.env.PXE_USE_INIT_3 === '1' && totalAppCalls >= 3`. Falls back to standard init when fewer than 3 apps. Refactored top-of-loop into `consumeNextApp` helper | |
| 32 | +| `0dc31e27` | Docs | `find-bb` defaults to `bb-avm` — flagged because we lost a session debugging the wrong binary | |
| 33 | +| `69362fbe` | Reproducer | `yarn-project/end-to-end/scripts/reproduce_init3_canary.sh`, plus a `SKIP_STEP_COUNT_CHECK=1` opt-out in `captureProfile` | |
| 34 | + |
| 35 | +## Verify it works |
| 36 | + |
| 37 | +After `./bootstrap.sh` from the repo root, run the reproducer: |
| 38 | + |
| 39 | +```bash |
| 40 | +yarn-project/end-to-end/scripts/reproduce_init3_canary.sh |
| 41 | +``` |
| 42 | + |
| 43 | +This runs the `deploy_ecdsar1+sponsored_fpc` client flow with `PXE_USE_INIT_3=1`, |
| 44 | +captures the IVC inputs, proves with `bb prove --scheme chonk`, and verifies the |
| 45 | +proof. Asserts the capture references `PrivateKernelInit3Artifact` so a |
| 46 | +regression in flag plumbing is caught at capture time. Total runtime ~2–3 min. |
| 47 | + |
| 48 | +Final output on success: |
| 49 | + |
| 50 | +``` |
| 51 | +init_3 canary verified end-to-end. |
| 52 | + Captured inputs: /tmp/init3-canary/captures/deploy_ecdsar1+sponsored_fpc/ivc-inputs.msgpack |
| 53 | + Proof artifacts: /tmp/init3-canary/proof/ |
| 54 | +``` |
| 55 | + |
| 56 | +To verify a different flow, set the right test name + flow path: |
| 57 | + |
| 58 | +```bash |
| 59 | +# transfer_0 (3 apps total — init_3 absorbs all of them) |
| 60 | +PXE_USE_INIT_3=1 \ |
| 61 | +CAPTURE_IVC_FOLDER=/tmp/init3-out \ |
| 62 | +SKIP_STEP_COUNT_CHECK=1 \ |
| 63 | +BENCHMARK_CONFIG=key_flows \ |
| 64 | +LOG_LEVEL=warn \ |
| 65 | +node --experimental-vm-modules yarn-project/node_modules/.bin/jest \ |
| 66 | + --testTimeout=300000 --no-cache --runInBand \ |
| 67 | + --testNamePattern "ecdsar1.*0 recursions.*sponsored_fpc" \ |
| 68 | + --rootDir yarn-project/end-to-end/src \ |
| 69 | + client_flows/transfers |
| 70 | +# then bb prove + bb verify against /tmp/init3-out/<flow>/ivc-inputs.msgpack |
| 71 | +``` |
| 72 | + |
| 73 | +## Measured benefit (single run, remote bench machine, HARDWARE_CONCURRENCY=16) |
| 74 | + |
| 75 | +| Flow | Apps | Baseline | init_3 | Δ | |
| 76 | +|---|---:|---:|---:|---:| |
| 77 | +| `deploy_ecdsar1+sponsored_fpc` | 6 | 6.29 s | 5.99 s | **-0.30 s (-4.8%)** | |
| 78 | +| `ecdsar1+transfer_0_recursions+sponsored_fpc` | 3 | 5.09 s | 4.65 s | **-0.44 s (-8.6%)** | |
| 79 | + |
| 80 | +Savings come exactly where the design predicted: -2 `HypernovaFoldingProver::fold` |
| 81 | +calls, -2 `OinkProver::prove`, -2 `Chonk::complete_kernel_circuit_logic`. ECCVM |
| 82 | +shrinks ~23% in trace rows but only ~7% in time (fixed sumcheck overhead). |
| 83 | +Translator stays constant at 8192 rows by design (proof is constant-size). |
| 84 | + |
| 85 | +## What was deferred and why |
| 86 | + |
| 87 | +**Phase 6 (audit one-app-per-kernel assumptions) — folded into Phase 7.** The |
| 88 | +canary proving + verifying real client flows on real bb is a stronger integration |
| 89 | +gate than a code audit. If a one-app assumption had been violated, prove would |
| 90 | +have crashed. |
| 91 | + |
| 92 | +**Phase 8 (VK pin update) — not needed yet.** The pin script |
| 93 | +(`barretenberg/cpp/scripts/test_chonk_standalone_vks_havent_changed.sh`) compares |
| 94 | +VKs derived from *pinned bytecode* (frozen msgpack from S3) against pinned VKs in |
| 95 | +the same msgpack. It catches C++-side bb regressions, not Noir-side circuit |
| 96 | +changes. Our changes are Noir-side, so the pin test still passes on this branch |
| 97 | +without intervention. The pin will need updating when init_3 stops being |
| 98 | +flag-gated and becomes the default — at that point newly captured pins will |
| 99 | +include init_3 steps, and the reference msgpack on S3 should be rotated. |
| 100 | + |
| 101 | +## Follow-up opportunities |
| 102 | + |
| 103 | +In rough order of payoff: |
| 104 | + |
| 105 | +1. **`inner_3`** — the next obvious win. The deployment flow has 6 apps; init_3 |
| 106 | + collapses the first 3 but the remaining 3 still go through 3 inner kernels. |
| 107 | + An inner that batches 3 apps would collapse those, saving another ~2 prev-kernel |
| 108 | + HN verifications. Same shape as init_3 but for the middle of the chain. |
| 109 | + |
| 110 | +2. **Skip inner kernels entirely when init_3 absorbs everything.** Marked with |
| 111 | + `WORKTODO(luke)` at the bottom of the init_3 branch in |
| 112 | + `private_kernel_execution_prover.ts`. If `executionStack.length === 0` after |
| 113 | + init_3, the chain can go directly to reset+tail. The protocol-circuit |
| 114 | + ALLOWED_PREVIOUS lists already accept init_3 as a predecessor of |
| 115 | + reset/tail/tail_to_public, so this is purely an orchestrator change. |
| 116 | + Relevant for short flows like transfer_0. |
| 117 | + |
| 118 | +3. **`init_2` / `inner_2` / `init_1` / `inner_1`.** Round out the variant set so |
| 119 | + the orchestrator can pick the largest batch that fits. Today the dispatch is |
| 120 | + binary (init_3 if ≥3 apps, else init); a graceful fallback through init_2 |
| 121 | + would help any flow with exactly 2 apps. Probably low priority — most real |
| 122 | + flows have 3+. |
| 123 | + |
| 124 | +4. **Generic `PrivateKernelInitNCircuitPrivateInputs<N>`.** The TS class is |
| 125 | + currently concrete (three explicit `privateCallN` fields). When a second |
| 126 | + variant lands, generalize. |
| 127 | + |
| 128 | +5. **Remove the `PXE_USE_INIT_3` flag.** Once a few weeks of green CI confirm |
| 129 | + stability, the flag and its `totalAppCalls >= 3` gate can collapse into |
| 130 | + unconditional use. Coordinate with Phase 8 (pin update) at that point. |
| 131 | + |
| 132 | +## Known papercuts |
| 133 | + |
| 134 | +- **`find-bb` defaults to `bb-avm`** (`AVM=1`). Rebuilding only the non-AVM `bb` |
| 135 | + target leaves a stale `bb-avm` and downstream bootstraps keep failing with the |
| 136 | + same error after the supposed fix. This is now documented in |
| 137 | + `barretenberg/cpp/CLAUDE.md`. |
| 138 | +- **`SKIP_STEP_COUNT_CHECK=1`** is required when running client_flow tests with |
| 139 | + `PXE_USE_INIT_3=1` because the `captureProfile` `expectedSteps` assertion is |
| 140 | + hard-coded for the standard init+inner chain. The reproducer script sets it; |
| 141 | + if you write new test paths that exercise init_3, set it explicitly. |
| 142 | +- **Pre-existing stale gitignored derivatives** in |
| 143 | + `aztec.js/src/contract/protocol_contracts/` and `noir-contracts.js/src/` may |
| 144 | + exist on a tree that hasn't been bootstrapped recently. Symptom: `aztecVersion |
| 145 | + is missing in type ContractArtifact` build errors. Fix: run `./bootstrap.sh` |
| 146 | + from the repo root. |
| 147 | + |
| 148 | +## Out of scope of this work |
| 149 | + |
| 150 | +- Any change to the C++ Chonk recursive-verifier internals. The existing |
| 151 | + `bool is_init_kernel = front().type == OINK` (`chonk.cpp:314`) and the rest |
| 152 | + of `complete_kernel_circuit_logic` already correctly handle a 3-entry queue |
| 153 | + `[OINK, HN, HN]` — confirmed by the canary running real proofs. No bb code |
| 154 | + changes were needed. |
| 155 | +- Changes to reset / tail config dimensions. init_3 produces the same |
| 156 | + `PrivateKernelCircuitPublicInputs` shape as init, so downstream kernels see |
| 157 | + no new shape. |
| 158 | + |
| 159 | +## Files of interest |
| 160 | + |
| 161 | +- `noir-projects/noir-protocol-circuits/crates/private-kernel-lib/src/private_kernel_init_n.nr` |
| 162 | + — the lib that init_3's binary instantiates with N=3. |
| 163 | +- `noir-projects/noir-protocol-circuits/crates/private-kernel-init-3/src/main.nr` |
| 164 | + — the concrete crate. |
| 165 | +- `yarn-project/pxe/src/private_kernel/private_kernel_execution_prover.ts` |
| 166 | + — the orchestrator. The init_3 dispatch is in the `firstIteration` branch. |
| 167 | +- `yarn-project/end-to-end/scripts/reproduce_init3_canary.sh` — the reproducer. |
0 commit comments