docs: high availability sequencer guide (#3293)

* docs: ev-node high availability

* docs: node placement

* docs(ha): address PR review feedback

Critical fixes:
- Fix snapshot_threshold math: 5000 ÷ 10 = 500s ≈ 8.3 min (not 83s)
- Fix trailing_logs math: 18000 ÷ 10 = 1800s = 30 min (not 5 min)

Medium fixes:
- Fix heartbeat_timeout description: it is a follower-side election trigger,
  not the interval at which the leader sends heartbeats
- Add explicit restart instruction after Step 5 data copy in single-to-ha.md
  so the chain keeps producing blocks during preparation (Steps 6-8)
- Replace priv_validator_key.json with signer.json in single-to-ha.md
  to match cluster-setup.md and the E2E tests

Minor fixes:
- Exclude self from raft.peers in all examples (cluster-setup.md node-1
  yaml/CLI/systemd, single-to-ha.md node-1 and node-2)
- Add "exclude local node" note to raft.peers description in overview.md
- Fix P2P port in overview.md Interaction with P2P section (7676 → 26656)
- Add text language tag to all bare fenced blocks (MD040): multiaddr
  example, RTT equations, and all log snippets

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(ha): absorb raft_production.md into ha/overview.md

raft_production.md had no sidebar entry and its content was fully
superseded by the new ha/ guides. Extract the three pieces that were
unique to it — bootstrap flag docs, auto-detection startup mode
explanation, and static-membership limitation note — into
ha/overview.md, then delete the file.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(ha): use EnvironmentFile for signer passphrase

Passing --evnode.signer.passphrase inline exposes the secret in
ps aux, journalctl, and shell history.

- Add EnvironmentFile=/etc/ev-node/env (chmod 600) to the systemd
  unit in cluster-setup.md with setup instructions
- Replace all inline <YOUR_PASSPHRASE> occurrences with
  $EV_SIGNER_PASSPHRASE sourced from /etc/ev-node/env in every
  evm start / evm init snippet across both guides

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(ha): explicit node-2 peers and action-based rolling restart

- Replace "peers list is identical" stub in node-2 config with an
  explicit peers list that excludes node-2 itself, and add a note
  that each node must omit itself from raft.peers
- Replace "Wait ~30 seconds" in rolling restart with journalctl
  one-liners that exit as soon as the node logs follower/leader state,
  giving a deterministic signal instead of an arbitrary timeout

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(ha): fix raft.peers self-inclusion startup bug

The abbreviated node-2 snippet with "# peers list is identical" caused
a startup failure: with raft_addr=0.0.0.0:5001 the bootstrap code's
literal address comparison does not recognise node-2@10.0.0.2:5001 as
self, so node-2 is appended twice and deduplicateServers returns
"duplicate peers found in config".

- Fix intro text: "only raft.node_id and raft_addr differ" →
  "raft.node_id is unique; raft.peers and p2p.peers must exclude self"
- Expand node-2 snippet to a full evnode.yaml with the correct peers
  list (node-1, node-3, node-4, node-5 — no node-2) and an inline
  explanation of the wildcard address pitfall
- Align overview.md trailing_logs example to 1 block/s (matching
  block_time: "1s" used throughout) and note the 10 block/s rate too

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(ha): fix passphrase flag and failover kill cardinality check

Replace non-existent --evnode.signer.passphrase with the actual
--evnode.signer.passphrase_file flag throughout cluster-setup and
single-to-ha guides. Update passphrase setup to create a chmod 600
file at /etc/ev-node/passphrase referenced directly by the flag.

Add mapfile-based cardinality check in the failover test fallback
kill command to guard against killing the wrong process.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(ha): fix RPC endpoints, init ordering, and snap_count CLI flag

Replace incorrect CometBFT RPC calls (port 26657/status) with the
actual ev-node HTTP API (port 7331 /health/ready, /raft/node) and
EVM execution layer (cast block latest) throughout both guides.

Align single-to-ha Step 2 init ordering with cluster-setup: create
passphrase file before evm init so the signer key is encrypted from
the start, and pass --evnode.node.aggregator and passphrase_file flags.

Fix Step 9a fallback kill in single-to-ha to use mapfile cardinality
check, matching the pattern already applied in cluster-setup.

Add --evnode.raft.snap_count=3 to the CLI start example to match
the YAML config block.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: 2 people

docs/.vitepress/config.ts

@@ -279,6 +279,24 @@ function sidebarHome() {
             },
           ],
         },
+        {
+          text: "High Availability",
+          collapsed: true,
+          items: [
+            {
+              text: "Overview & Configuration",
+              link: "/guides/ha/overview",
+            },
+            {
+              text: "Bootstrap a 5-Node Cluster",
+              link: "/guides/ha/cluster-setup",
+            },
+            {
+              text: "Migrate Single → HA",
+              link: "/guides/ha/single-to-ha",
+            },
+          ],
+        },
         {
           text: "Run a Full Node",
           link: "/guides/full-node",