Commit 87b5084
authored
feat: cross-enclave sync via
## Summary
- Lets one Kurtosis enclave sync from another. Source enclave A runs
`apache` and publishes the canonical bal-devnet metadata files
(`/enodes.txt`, `/bootstrap_nodes.txt`, `/bootstrap_nodes.yaml`,
`/network-config.tar`) at the URL root.
- Target enclave B sets `network: kt-<A-host>:<A-apache-port>`, fetches
the tar, and bootstraps off A's enodes/ENRs via the existing devnet code
paths in the EL/CL launchers — no client launcher changes needed.
- Validator counts auto-zero on the target (existing logic for
non-`kurtosis`/`shadowfork` networks), so target is observer-mode by
default.
## Changes
- `src/apache/apache_launcher.star`, `src/nginx/nginx_launcher.star`
- Rename file constants to canonical bal-devnet names (`boot_enr.yaml` →
`bootstrap_nodes.yaml`, `bootnode.txt` → `enodes.txt`;
`bootstrap_nodes.txt` already matched).
- Fix `cp -R /network-configs /htdocs/` → `cp -R /network-configs/.
/htdocs/` so files land at the URL root, not nested under
`/network-configs/`.
- `src/apache/apache_launcher.star`
- Print the exact `kt-<host>:<port>` string the operator should use on a
sync target.
- Warn when `port_publisher.el.nat_exit_ip` is unset (otherwise
published enodes advertise unreachable docker IPs).
- `src/network_launcher/remote_enclave.star` (new)
- Parses `kt-<host>:<port>` (also accepts `kt-http://...` /
`kt-https://...`).
- Curls `<base>/network-config.tar`, extracts to the
`el_cl_genesis_data` artifact, reads timestamp + network_id from the
embedded `config.yaml`.
- `src/package_io/constants.star`
- Adds `NETWORK_NAME.remote_enclave = "kt-"` (used as a prefix).
- `src/participant_network.star`
- New `elif network_params.network.startswith("kt-")` branch dispatches
to the new launcher, placed before the public/devnet fallbacks.
## Breaking change
The apache/nginx services no longer serve files under the legacy names
`boot_enr.yaml` and `bootnode.txt`. Anyone scraping those URLs needs to
switch to `bootstrap_nodes.yaml` and `enodes.txt`.
## Caveat
Geth's snap-sync wedges with `missing trie node ... layer stale` when
joining a fresh kurtosis devnet mid-flight. Not specific to
cross-enclave sync — same issue affects any kurtosis-to-snapshot path.
Mitigations:
- Use `el_extra_params: [--syncmode=full]` on the target (proven to work
in testing), or
- Wait until the source enclave has finalized at least one epoch before
launching the target.
## Test plan
- [x] Local 2-enclave test on one host (10.100.116.15)
- A: 2 nodes (geth+lighthouse, reth+teku) + apache on port 9128,
`port_publisher.{el,cl}.enabled: true`, `nat_exit_ip: 10.100.116.15`.
- B: 1 node (geth+lighthouse) with `network: kt-10.100.116.15:9128` and
`--syncmode=full`.
- Result: A EL block `0x38` == B EL block `0x38`, B CL `sync_distance:
0`, B's geth importing chain segments. EL+CL peering established
cross-enclave (2 EL peers, 2 CL peers).
- [x] Verified canonical files served at root and that enodes advertise
the public NAT IP, not docker-internal:
- `/enodes.txt` contains `enode://...@10.100.116.15:32000`
- `/bootstrap_nodes.txt`, `/bootstrap_nodes.yaml`, `/network-config.tar`
all 200 OK
- [ ] Remote-machine test (two physical hosts) — pending.
---------
Signed-off-by: Barnabas Busa <barnabas.busa@ethereum.org>kt-<host>:<port> network (#1378)1 parent 3f9aa2f commit 87b5084
40 files changed
Lines changed: 240 additions & 395 deletions
File tree
- .github/tests
- examples
- src
- apache
- mempool_bridge
- mev/flashbots
- mev_boost
- mev_relay
- network_launcher
- nginx
- package_io
- static_files
- static_files
- apache-config
- mev/helix
- nginx-config
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
0 commit comments