refactor: use upload_files for URL fetches instead of run_sh+curl

[barnabasbusa]

Summary

• Replace run_sh+curl fetches with native plan.upload_files(src=<https-url>, ...) where the URL is known at interpretation time
• Remaining run_sh calls along these paths now only parse the mounted artifact (jq/yq/sed) — no more network egress from those steps
• In mempool_bridge_launcher.star, the enode list URL is now fetched once instead of MAX_ENODES_TO_FETCH (= 5) times

Files touched:

• src/network_launcher/public_network.star — eth-clients genesis.json
• src/network_launcher/shadowfork.star — ephemery config.yaml
• src/network_launcher/remote_enclave.star — remote enclave network-config.tar
• src/mempool_bridge/mempool_bridge_launcher.star — eth-clients enodes.yaml

Requirements

Kurtosis >= 1.18.3 is required. This PR depends on kurtosis-tech/kurtosis#3111 ("allow http(s) URLs in upload_files src"), which shipped in Kurtosis 1.18.3. Earlier Kurtosis versions will fail at interpretation time when they hit the new upload_files(src="https://...") calls.

Test plan

• kurtosis lint --format . clean (already verified locally)
• Run a public network (e.g. hoodi) end-to-end and confirm el_cl_genesis_data artifact is populated and osakaTime is read correctly
• Run an ephemery shadowfork and confirm the chain id is read from the uploaded config
• Run mempool-bridge against a public network and confirm source enodes are populated
• Run a remote-enclave (kt-...) network and confirm the tar is fetched + extracted into el_cl_genesis_data

[qu0b-reviewer]

🤖 qu0b-reviewer

Summary

The PR replaces plan.run_sh with embedded curl for fetching remote files with plan.upload_files(src=URL), which is a cleaner approach. Two of the four changed files have initialization-order correctness bugs introduced by hoisting the artifact creation above the code that previously validated whether those lines would execute. The other two files (mempool_bridge_launcher.star, shadowfork.star) are clean.

Issues

• 🔴 blocker src/network_launcher/public_network.star:125 — genesis_artifact is created unconditionally, but in the old code the fetch_osaka_time run_sh was inside the if network_params.force_snapshot_sync block. If force_snapshot_sync is true, the artifact is created and passed to new_el_cl_genesis_data (which likely breaks since the genesis.json structure differs for snapshot nodes), but osaka_time is never read. The snapshot-sync path needs to skip genesis fetching entirely, and the non-snapshot path should lazily create the artifact only when needed.

• 🔴 blocker src/network_launcher/remote_enclave.star:27 — Same pattern. remote_tar_artifact is now created unconditionally before the el_cl_genesis_data_uuid run_sh, but in the original code, the curl was nested inside the run_sh and thus only executed when the function ran. The artifact creation has no guard — it runs even on parse_remote_enclave_url failure (which calls fail() before reaching this line) or any other early-exit path that previously would have never executed the run_sh.

Suggestions

• src/mempool_bridge/mempool_bridge_launcher.star:83 and src/network_launcher/shadowfork.star:25 — Both look fine, but unlike the previous curl which failed loudly with curl -fsSL, upload_files with a bad URL will also fail at plan-interpretation time. Worth confirming this is the desired failure mode (vs. the silence from || echo "genesis.json not found" in the old public_network code — which was itself buggy and swallowed failures, so the new behavior is actually safer here).

---

_{Reviewed @ bdd80fe6
"If your only tool is a hammer, every problem looks like a microservice."}