feat: merge-train/spartan (#22352)

BEGIN_COMMIT_OVERRIDE
chore: fix mempool limit test (#22332)
fix(bot): bot fee juice funding (#21949)
fix(foundation): flush current batch on BatchQueue.stop() (#22341)
chore: (A-750) read JSON body then parse to avoid double stream
consumption on error message (#22247)
chore: bump log level in stg-public (#22354)
chore: fix main.tf syntax (#22356)
chore: wire up spartan checks to make (#22358)
fix(p2p): reduce flakiness in proposal tx collector benchmark (#22240)
fix: disable sponsored fpc and test accounts for devnet (#22331)
chore: add v4-devnet-3 to tf network ingress (#22327)
chore: remove unused env var (#22365)
chore: add pdb (#22364)
chore: dispatch CB on failed deployments (#22367)
chore: (A-749) single character url join (#22269)
feat: support different docker image for HA validator nodes (#22371)
chore: fix the daily healthchecks (#22373)
chore: remove v4-devnet-2 references (#22372)
fix: rename #team-alpha → #e-team-alpha slack channel (#22374)
chore(pipeline): timetable adjustments under pipelining (#21076)
feat(pipeline): handle pipeline prunes (#21250)
fix: handle error types serialization errors (#22379)
feat(spartan): configurable HA validator replica count (#22384)
fix(e2e): increase prune timeout in epochs_mbps_pipeline test (#22392)
fix(epoch-cache): use TTL-based caching with finalization tracking and
correct lag (#22204)
chore: deflake e2e ha sync test (#22403)
chore(ci): skip prunes-uncheckpointed test in epochs_mbps_pipeline
(#22401)
refactor(slasher): remove empire slasher model (#21830)
fix: use strict equality in world-state ops queue (#22398)
fix: remove unused BLOCK reqresp sub-protocol (#22407)
refactor(sequencer): sign last block before archiver sync (#22117)
feat(world-state): add genesis timestamp support and GenesisData type
(#22359)
fix: use Int64Value instead of Uint32Value for 64-bit map sizes (#22400)
chore: Reduce logging verbosity (#22423)
fix(p2p): include values in tx validation error messages (#22422)
END_COMMIT_OVERRIDE

Author: alexghr

.claude/claudebox/deploy-investigation.md

@@ -0,0 +1,143 @@
+# ClaudeBox Deploy Investigation
+
+Instructions for ClaudeBox when investigating a deployment failure.
+This is triggered by `deploy-network.yml` when a deployment fails.
+
+## Context
+
+You will receive a prompt like:
+> Deployment of NETWORK (version SEMVER) failed.
+> Follow .claude/claudebox/deploy-investigation.md to investigate.
+> GitHub Actions run: RUN_URL. Network: NETWORK. Version: SEMVER.
+> Docker image: IMAGE_TAG. Git ref: REF. Namespace: NAMESPACE.
+> Deploy contracts: true|false.
+
+Extract these variables from the prompt:
+- `NETWORK`: the network name (e.g., `testnet`, `staging-public`, `next-net`)
+- `SEMVER`: the version being deployed
+- `RUN_URL`: the GitHub Actions run URL
+- `RUN_ID`: the numeric ID at the end of `RUN_URL`
+- `NAMESPACE`: the Kubernetes namespace (usually same as NETWORK)
+- `IMAGE_TAG`: the Docker image tag
+- `DEPLOY_CONTRACTS`: whether fresh contract deployment was requested
+
+## Constraints
+
+You are running inside ClaudeBox. You do **not** have `gh` CLI or `git push`.
+Use MCP tools instead: `github_api`, `respond_to_user`.
+
+## Workflow
+
+### 1. Fetch the Failed Job
+
+```
+github_api(method="GET", path="repos/AztecProtocol/aztec-packages/actions/runs/<RUN_ID>/jobs")
+```
+
+Find the job with `conclusion: "failure"`. Extract its `id` and note which step failed
+(look at the `steps` array for the step with `conclusion: "failure"`).
+
+### 2. Download GitHub Actions Job Logs
+
+```
+github_api(method="GET", path="repos/AztecProtocol/aztec-packages/actions/jobs/<JOB_ID>/logs")
+```
+
+The GitHub Actions logs are a wrapper. The **actual detailed logs** are on
+ci.aztec-labs.com. Look for lines like:
+
+```
+Executing: <command> (http://ci.aztec-labs.com/<HASH>)
+   0 . failed (Xs) (http://ci.aztec-labs.com/<HASH>)
+```
+
+Extract the `ci.aztec-labs.com/<HASH>` URL(s) from the logs. The hash after the
+failed command is the one you need.
+
+### 3. Download and Analyze CI Logs
+
+Use `yarn ci dlog` to download the actual failure logs:
+
+```bash
+cd yarn-project && yarn ci dlog <HASH> > /tmp/<HASH>.log 2>&1
+```
+
+Read the downloaded log to find the root cause. These logs contain the real
+Terraform output, Helm errors, script failures, etc.
+
+If the log references further nested ci.aztec-labs.com URLs, follow them the same
+way to get to the deepest failure.
+
+Look for:
+- Terraform errors (plan/apply failures, state locks, quota limits)
+- Helm/Kubernetes errors (pod timeouts, image pull failures)
+- Contract deployment errors (L1 tx reverts, gas issues)
+- Script errors (missing env vars, bad config)
+
+### 4. Categorize the Failure
+
+**Infrastructure failures:**
+- Terraform plan/apply errors (resource conflicts, quota limits, state locks)
+- GCP authentication issues
+- GKE cluster connectivity problems
+- Helm release failures (timeout waiting for pods)
+
+**Application failures:**
+- Container crash loops (OOMKilled, startup probe failures)
+- Contract deployment failures (L1 transaction reverts, gas issues)
+- Configuration errors (missing env vars, invalid addresses)
+
+**Network/external failures:**
+- L1 RPC endpoint unreachable
+- Docker image not found or pull rate limited
+- DNS resolution failures
+
+### 5. Query Network Logs (conditional)
+
+If the deployment got far enough that pods were created but the network failed to
+start (e.g., "waiting for network to be ready" timeout, Helm succeeded but health
+checks failed), query network logs for application-level errors.
+
+Read `.claude/skills/network-logs/SKILL.md` for instructions, then use the
+`network-logs` agent to query:
+- Namespace: `<NAMESPACE>`
+- Freshness: 30 minutes
+- Focus: startup errors, crash loops, contract deployment failures
+
+**Skip this step** if the failure was clearly infrastructure-level (Terraform error,
+image pull failure, GCP auth issue, etc.).
+
+### 6. Report Findings
+
+Use `respond_to_user()` to reply to the Slack alert thread with a concise summary.
+
+Format for Slack mrkdwn:
+
+```
+:red_circle: *Deploy Investigation: <NETWORK> v<SEMVER>*
+
+*Root cause*: <one-line summary of what went wrong>
+
+*Failed step*: "<step name>"
+
+*Error*:
+> <key error message, 1-3 lines>
+
+*Category*: <infrastructure | application | external>
+
+*CI log*: http://ci.aztec-labs.com/<HASH>
+
+<If applicable>
+*Suggested fix*: <what to do next — retry, fix config, increase quota, etc.>
+
+<If network logs were queried>
+*Network logs*: <relevant findings>
+
+<RUN_URL|Full workflow logs>
+```
+
+Keep it concise and actionable — operators need to act quickly on deploy failures.
+
+If you cannot determine the root cause, say so and provide what you found.
+
+**Do NOT attempt to fix the deployment or re-run it. Investigation only.**

.claude/skills/merge-trains/SKILL.md

@@ -18,7 +18,7 @@ A merge train is an automated batching system (inspired by [Rust rollups](https:
 | `merge-train/ci` | CI infrastructure / ci3 | `#help-ci` |
 | `merge-train/docs` | Documentation | `#dev-rels` |
 | `merge-train/fairies` | aztec-nr | `#team-fairies` |
-| `merge-train/spartan` | Spartan / infra / yarn-project sequencer and prover orchestration | `#team-alpha` |
+| `merge-train/spartan` | Spartan / infra / yarn-project sequencer and prover orchestration | `#e-team-alpha` |
 
 ## How to Use a Merge Train
 

.github/workflows/deploy-network.yml

@@ -30,6 +30,10 @@ on:
         required: false
         type: boolean
         default: false
+      ha_docker_image:
+        description: "Full docker image for HA validator nodes (optional, defaults to aztec docker image)"
+        required: false
+        type: string
       source_tag:
         description: "Source tag that triggered this deploy"
         required: false
@@ -62,6 +66,10 @@ on:
         required: false
         type: boolean
         default: false
+      ha_docker_image:
+        description: "Full docker image for HA validator nodes (optional, defaults to aztec docker image)"
+        required: false
+        type: string
       source_tag:
         description: "Source tag that triggered this deploy"
         required: false
@@ -172,6 +180,7 @@ jobs:
           AZTEC_DOCKER_IMAGE: "aztecprotocol/aztec:${{ inputs.docker_image_tag || inputs.semver }}"
           CREATE_ROLLUP_CONTRACTS: ${{ inputs.deploy_contracts == true && 'true' || '' }}
           PROVER_AGENT_DOCKER_IMAGE: "aztecprotocol/aztec-prover-agent:${{ inputs.docker_image_tag || inputs.semver }}"
+          VALIDATOR_HA_DOCKER_IMAGE: ${{ inputs.ha_docker_image || '' }}
         run: |
           echo "Deploying network: ${{ inputs.network }}"
           echo "Using image: $AZTEC_DOCKER_IMAGE"
@@ -207,24 +216,51 @@ jobs:
             fi
           } >> "$GITHUB_STEP_SUMMARY"
 
-      - name: Notify Slack on failure
+      - name: Notify Slack and dispatch ClaudeBox on failure
         if: failure()
         env:
           SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+          GH_TOKEN: ${{ secrets.AZTEC_BOT_GITHUB_TOKEN }}
         run: |
-          if [ -n "${SLACK_BOT_TOKEN}" ]; then
-            read -r -d '' data <<EOF || true
-            {
-              "channel": "#alerts-${{ inputs.network }}",
-              "text": "Deploy Network workflow FAILED for *${{ inputs.network }}* (version ${{ inputs.semver }}): <https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Run>"
-            }
-          EOF
-            curl -X POST https://slack.com/api/chat.postMessage \
-              -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
-              -H "Content-type: application/json" \
-              --data "$data"
+          if [ -z "${SLACK_BOT_TOKEN:-}" ]; then
+            echo "No SLACK_BOT_TOKEN, skipping notification"
+            exit 0
           fi
 
+          CHANNEL="#alerts-${{ inputs.network }}"
+          RUN_URL="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+          TEXT="Deploy Network workflow FAILED for *${{ inputs.network }}* (version ${{ inputs.semver }}): <${RUN_URL}|View Run> (🤖)"
+
+          # Post to Slack and capture timestamp for permalink
+          RESP=$(curl -sS -X POST https://slack.com/api/chat.postMessage \
+            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
+            -H "Content-type: application/json" \
+            -d "$(jq -n --arg c "$CHANNEL" --arg t "$TEXT" '{channel:$c, text:$t}')")
+          echo "Slack response: $RESP"
+
+          TS=$(echo "$RESP" | jq -r '.ts // empty')
+          CHANNEL_ID=$(echo "$RESP" | jq -r '.channel // empty')
+
+          LINK=""
+          if [[ -n "$TS" && -n "$CHANNEL_ID" ]]; then
+            LINK="https://aztecprotocol.slack.com/archives/$CHANNEL_ID/p${TS//./}"
+          fi
+
+          # Dispatch ClaudeBox to investigate the failure
+          PROMPT="Deployment of ${{ inputs.network }} (version ${{ inputs.semver }}) failed. \
+          Follow .claude/claudebox/deploy-investigation.md to investigate. \
+          GitHub Actions run: ${RUN_URL}. \
+          Network: ${{ inputs.network }}. Version: ${{ inputs.semver }}. \
+          Docker image: ${{ inputs.docker_image_tag || inputs.semver }}. \
+          Git ref: ${{ steps.checkout-ref.outputs.ref }}. \
+          Namespace: ${{ inputs.namespace || inputs.network }}. \
+          Deploy contracts: ${{ inputs.deploy_contracts }}."
+
+          gh workflow run claudebox.yml \
+            -f prompt="$PROMPT" \
+            -f link="${LINK:-$RUN_URL}" \
+            -f target_ref="${{ steps.checkout-ref.outputs.ref }}" || true
+
   update-irm:
     needs: deploy-network
     if: inputs.network == 'testnet' || inputs.network == 'mainnet'

.github/workflows/network-healthcheck.yml

@@ -22,4 +22,28 @@ jobs:
         env:
           SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
           GH_TOKEN: ${{ secrets.AZTEC_BOT_GITHUB_TOKEN }}
-        run: ./ci3/network_healthcheck "${{ inputs.networks || 'v4-devnet-2,testnet,mainnet,staging-public,next-net' }}"
+          CI: "1"
+        run: |
+          NETWORKS="${{ inputs.networks || 'next-net,staging-public,testnet,mainnet' }}"
+
+          PROMPT="Run a network healthcheck for: ${NETWORKS}.
+
+          For each network, query Cloud Logging to report:
+          1. Components running
+          2. Latest L2 block and slot numbers
+          3. Peer counts
+          4. Block production cadence (last ~10 checkpoints)
+          5. Any errors (level >= 50) or warnings (level 40) in the last 8 hours
+          6. Bot status if applicable
+
+          Create a gist with the full healthcheck report. Then post a concise summary to the #e-team-alpha channel via respond_to_user. Flag anything that needs attention (stopped bots, missed slots, errors, low peer counts).
+
+          Format the respond_to_user message as a brief network status overview, e.g.:
+          - testnet: healthy, block 5570, 100 peers
+          - mainnet: healthy, block 1234, 50 peers
+          - devnet: WARNING - bot stopped (insufficient balance)
+          Link to the gist for full details."
+
+          ./ci3/slack_notify_with_claudebox_kickoff "#e-team-alpha" \
+            "Starting network healthcheck for: ${NETWORKS}" \
+            "$PROMPT"

.test_patterns.yml

@@ -253,6 +253,13 @@ tests:
     owners:
       - *phil
 
+  # Consistently times out — prune detection timing too tight for CI resources
+  # See https://github.com/AztecProtocol/aztec-packages/pull/22392
+  - regex: "epochs_mbps.pipeline.parallel.test.ts.*prunes uncheckpointed"
+    skip: true
+    owners:
+      - *sean
+
   # Blanket flake patterns for unstable p2p, epoch, and l1 tx utils tests
   # This is a temporary measure while team bandwidth is constrained
   # Replaced many specific patterns - see https://github.com/AztecProtocol/aztec-packages/pull/17962 for historical context

Makefile

@@ -47,15 +47,15 @@ endef
 # PHONY TARGETS - List every target that has a file/dir of the same name.
 #==============================================================================
 
-.PHONY: noir barretenberg noir-projects l1-contracts release-image boxes playground docs aztec-up
+.PHONY: noir barretenberg noir-projects l1-contracts release-image boxes playground docs aztec-up spartan
 
 #==============================================================================
 # BOOTSTRAP TARGETS
 #==============================================================================
 
 # Fast bootstrap.
 fast: release-image barretenberg boxes playground docs aztec-up \
-		  bb-tests l1-contracts-tests yarn-project-tests boxes-tests playground-tests aztec-up-tests docs-tests noir-protocol-circuits-tests release-image-tests
+		  bb-tests l1-contracts-tests yarn-project-tests boxes-tests playground-tests aztec-up-tests docs-tests noir-protocol-circuits-tests release-image-tests spartan
 
 # Full bootstrap.
 full: fast bb-full-tests bb-cpp-full yarn-project-benches
@@ -364,3 +364,6 @@ aztec-up: yarn-project
 
 aztec-up-tests: aztec-up
 	$(call test,$@,aztec-up)
+
+spartan:
+	$(call build,$@,spartan)

barretenberg/cpp/src/barretenberg/nodejs_module/lmdb_store/lmdb_store_wrapper.cpp

@@ -35,7 +35,12 @@ LMDBStoreWrapper::LMDBStoreWrapper(const Napi::CallbackInfo& info)
     uint64_t map_size = DEFAULT_MAP_SIZE;
     if (info.Length() > map_size_index) {
         if (info[map_size_index].IsNumber()) {
-            map_size = info[map_size_index].As<Napi::Number>().Uint32Value();
+            // Int64Value is the widest integer accessor in N-API (no Uint64Value exists)
+            int64_t val = info[map_size_index].As<Napi::Number>().Int64Value();
+            if (val <= 0) {
+                throw Napi::TypeError::New(env, "Map size must be a positive number");
+            }
+            map_size = static_cast<uint64_t>(val);
         } else {
             throw Napi::TypeError::New(env, "Map size must be a number or an object");
         }

barretenberg/cpp/src/barretenberg/nodejs_module/world_state/world_state.cpp

@@ -119,19 +119,39 @@ WorldStateWrapper::WorldStateWrapper(const Napi::CallbackInfo& info)
         throw Napi::TypeError::New(env, "Header generator point needs to be a number");
     }
 
+    uint64_t genesis_timestamp = 0;
+    size_t genesis_timestamp_index = 5;
+    if (info.Length() > genesis_timestamp_index) {
+        if (info[genesis_timestamp_index].IsNumber()) {
+            genesis_timestamp = static_cast<uint64_t>(info[genesis_timestamp_index].As<Napi::Number>().Int64Value());
+        } else {
+            throw Napi::TypeError::New(env, "Genesis timestamp needs to be a number");
+        }
+    }
+
     // optional parameters
-    size_t map_size_index = 5;
+    size_t map_size_index = 6;
     if (info.Length() > map_size_index) {
         if (info[map_size_index].IsObject()) {
             Napi::Object obj = info[map_size_index].As<Napi::Object>();
 
             for (auto tree_id : tree_ids) {
                 if (obj.Has(tree_id)) {
-                    map_size[tree_id] = obj.Get(tree_id).As<Napi::Number>().Uint32Value();
+                    // Int64Value is the widest integer accessor in N-API (no Uint64Value exists)
+                    int64_t val = obj.Get(tree_id).As<Napi::Number>().Int64Value();
+                    if (val <= 0) {
+                        throw Napi::TypeError::New(env, "Map size must be a positive number");
+                    }
+                    map_size[tree_id] = static_cast<uint64_t>(val);
                 }
             }
         } else if (info[map_size_index].IsNumber()) {
-            uint64_t size = info[map_size_index].As<Napi::Number>().Uint32Value();
+            // Int64Value is the widest integer accessor in N-API (no Uint64Value exists)
+            int64_t val = info[map_size_index].As<Napi::Number>().Int64Value();
+            if (val <= 0) {
+                throw Napi::TypeError::New(env, "Map size must be a positive number");
+            }
+            uint64_t size = static_cast<uint64_t>(val);
             for (auto tree_id : tree_ids) {
                 map_size[tree_id] = size;
             }
@@ -140,7 +160,7 @@ WorldStateWrapper::WorldStateWrapper(const Napi::CallbackInfo& info)
         }
     }
 
-    size_t thread_pool_size_index = 6;
+    size_t thread_pool_size_index = 7;
     if (info.Length() > thread_pool_size_index) {
         if (!info[thread_pool_size_index].IsNumber()) {
             throw Napi::TypeError::New(env, "Thread pool size must be a number");
@@ -155,7 +175,8 @@ WorldStateWrapper::WorldStateWrapper(const Napi::CallbackInfo& info)
                                        tree_height,
                                        tree_prefill,
                                        prefilled_public_data,
-                                       initial_header_generator_point);
+                                       initial_header_generator_point,
+                                       genesis_timestamp);
 
     _dispatcher.register_target(
         WorldStateMessageType::GET_TREE_INFO,