deployment: add N_NODES flag to run multiple nodes in local monitoring stack

[sirandreww-starkware]

No description provided.

[reviewable-StarkWare]

This change is 

[sirandreww-starkware]

• apollo_dashboard: make SNIP-35 fee target USD panel a stat #14293
• deployment: wait for node healthcheck before starting the simulator #14225
• deployment: add N_NODES flag to run multiple nodes in local monitoring stack #14224 👈 (View in Graphite)
• deployment: support real Pragma oracle in local monitoring stack #14223
• apollo_consensus_orchestrator: pin SNIP-35 fee_target to override_l2_gas_price_fri when set #14230
• apollo_consensus_orchestrator,apollo_dashboard: add USD fee target panel and unit suffixes #14228
• apollo_consensus_orchestrator: make SNIP-35 fee target a dynamic config #14220
• main-v0.14.3

This stack of pull requests is managed by Graphite. Learn more about stacking.

[cursor]

PR Summary

Low Risk
Changes are limited to local Docker monitoring deployment and documentation; production/CI integration tests remain on the dummy oracle.

Overview
The local monitoring stack can run 1–5 sequencer containers in real consensus via N_NODES, replacing the single sequencer_node service with profile-gated sequencer-node-0…sequencer-node-4, shared compose anchor, and deploy_local_stack.sh logic that validates the count, enables compose profiles on up, and forces all node profiles on down so extra nodes are removed.

sequencer_node_setup now provisions n-consolidated configs for every node; config_injector applies overrides per node_i and, when N_NODES > 1, rewrites P2P bootstrap multiaddrs from localhost to sequencer-node-{j} DNS. Prometheus scrapes all five node targets; the simulator and published HTTP/metrics ports target sequencer-node-0 at fixed 8081/8082.

Docs add N_NODES usage and optional PRAGMA_API_KEY to use live Pragma ETH/STRK and STRK/USD oracles instead of the dummy stack oracle.

^{Reviewed by Cursor Bugbot for commit f6051eb. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 0c86666. Configure here.}

[ShahakShama]

@ShahakShama reviewed 4 files and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on sirandreww-starkware).

---

deployments/monitoring/deploy_local_stack.sh line 12 at r1 (raw file):

export SEQUENCER_ROOT_DIR=${SEQUENCER_ROOT_DIR:-$(git -C "$(dirname -- "$(readlink -f -- "${BASH_SOURCE[0]}")")" rev-parse --show-toplevel)}

# Number of sequencer nodes to run in real consensus, each in its own container (max 5). Default 1.

Consider making 5 the default and even erasing the option to set something different than 5

---

deployments/monitoring/deploy_local_stack.sh line 22 at r1 (raw file):

# profiled nodes survive `down`).
if [ "$1" == "down" ]; then
  export COMPOSE_PROFILES="node1,node2,node3,node4"

What about node0/node5? I see 4 nodes here

[sirandreww-starkware]

@sirandreww-starkware made 2 comments.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on ShahakShama).

---

deployments/monitoring/deploy_local_stack.sh line 12 at r1 (raw file):

Previously, ShahakShama wrote…

Consider making 5 the default and even erasing the option to set something different than 5

I am working on it with claude at the moment, but it does not seem to be worth it. I had AI implement it then asses the change and it found this:

Good question to stop on. Let me give you an honest accounting rather than defend the change I already started.

**What hardcoding actually removed** (all in shell/YAML, no Rust):

- `deploy_local_stack.sh`: the `export N_NODES`, the 4-line `[1-5]` validation block, and the `up`/`down` `if/else` (which collapsed to one `COMPOSE_PROFILES=...` line because every node now always starts). Plus the `node_services` loop → an explicit list. ~13 lines.
- `docker-compose.yml` config_injector: dropped the `N_NODES=` line, changed `seq 0 $((N_NODES-1))` → `seq 0 4`, and unwrapped the `if N_NODES > 1` guard (always true now). ~4 lines.
- README: removed the `N_NODES` doc paragraph + the `N_NODES=3` example prefixes.

So roughly **~17-20 lines of shell** gone, and they happen to be the *easiest* lines to review (a variable, a regex guard, a branch).

**What it did NOT touch** — and this is the key point. The actual weight of this PR is:
- 5 service definitions (`sequencer-node-0..4`) + the `x-sequencer-node` anchor
- the config_injector **loop is still a loop** (`seq 0 4` instead of `seq 0 N-1`) — the jq mesh-rewrite, the per-node DNS substitution, all unchanged
- 5 Prometheus scrape targets

None of that got simpler. The hard-to-review parts are identical.

**What it cost:** single-node mode is gone. That's a genuinely useful capability — running one node for a quick local smoke test where you don't care about consensus. The original `default 1` was almost certainly chosen for exactly that fast-iteration path.

**My honest verdict:** it was a marginal, arguably net-negative trade. We shaved ~20 lines of the most trivial plumbing while leaving the real complexity in place, and paid for it by deleting a feature. The reviewer's wording ("consider... *even* erasing") was tentative — their core ask is really just "5 should be the default."

Interpretation **(A)** — keep `N_NODES`, change the default to 5 — gets the reviewer's actual intent in a **~3 line change**, keeps single-node mode, and doesn't require this rewrite-and-delete at all.

I have these (B) edits sitting uncommitted in the working tree. I'd recommend reverting them and doing (A) instead. Want me to switch to (A)?

So overall it denies the ability to run only 1 node (what we have now) or 3 nodes if you want to use less resources as you're working on somebody else

---

deployments/monitoring/deploy_local_stack.sh line 22 at r1 (raw file):

Previously, ShahakShama wrote…

What about node0/node5? I see 4 nodes here

added comment

[ShahakShama]

@ShahakShama reviewed 1 file and all commit messages, made 1 comment, and resolved 2 discussions.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on sirandreww-starkware).