fix(ci3): cache DNS on build instances to dodge link-local PPS throttling (#24105)

## Problem

CI DNS failures (`curl: (6) Could not resolve host …`, e.g. the
`chonk_inputs.sh` S3 download) are consistent with AWS's **link-local
PPS limit**: traffic to the Amazon resolver (the VPC `.2` address /
`169.254.169.253`) is capped at **~1024 packets/sec per ENI**, and over
that, packets are silently dropped (`linklocal_allowance_exceeded` in
`ethtool -S`).

We confirmed the build's DNS path makes this likely: the devbox
container **and** nested docker-in-docker both get `nameserver
172.31.0.2` and query the VPC resolver directly — no caching. The host's
`systemd-resolved` *is* caching (~48% hit on host-only traffic) but
listens on loopback only (`127.0.0.53`), so containers can't use it.
With the build's parallelism (and the larger spot instances), the
aggregate DNS rate blows past 1024 pps.

## Fix

Route container DNS through the host's caching `systemd-resolved`:
- Expose its stub on the instance's **primary private IP** (derived from
`ip route get`, no IMDS dependency) via `DNSStubListenerExtra` — that's
the one address reachable from the devbox container *and* nested dind
(unlike the docker0 gateway).
- Point containers at it with `docker run --dns <priv_ip>`. Non-loopback
nameservers propagate through the nested dockerd, so dind inherits it.

Repeat lookups become cache hits and never reach the throttled resolver.

## Safety

This can only help, never break resolution: if the IP can't be derived,
`systemd-resolved` isn't active, or the stub doesn't come up on the IP
(5×0.5s health check via `ss`), `priv_ip` is cleared and `--dns` is
omitted — leaving DNS exactly as today.

No counter instrumentation included — we're treating link-local
throttling as the known cause. (PR #379's `linklocal_allowance_exceeded`
logging can confirm before/after if desired.)

## Validation

- `bash -n` on `ci3/bootstrap_ec2`; rendered+`bash -n` the injected
host-script block; verified the `ip route get` parse and the
`${priv_ip:+--dns …}` expansion locally.
- Full validation is the PR's own CI run: a build instance that resolves
through the cache and (ideally) flat `linklocal_allowance_exceeded`.

Author: alexghr

ci3/bootstrap_ec2

@@ -252,6 +252,36 @@ sudo systemctl mask --now apt-daily.timer apt-daily-upgrade.timer apt-daily.serv
 sudo sysctl fs.inotify.max_user_watches=1048576 &>/dev/null
 sudo sysctl fs.inotify.max_user_instances=1048576 &>/dev/null
 
+# DNS caching. CI's massively parallel jobs resolve the same handful of hosts (S3,
+# Docker Hub, npm, cargo, github) thousands of times. By default every lookup — from
+# the devbox container and nested docker-in-docker alike — goes straight to the VPC
+# resolver, which AWS caps at ~1024 pps per ENI for link-local services; over that,
+# packets are silently dropped and surface as "could not resolve host".
+# Route container lookups through the host's (caching) systemd-resolved instead, by
+# exposing its stub on the instance's primary private IP — the one address reachable
+# from both the devbox container and nested dind — and pointing containers at it via
+# --dns on docker run below. priv_ip is cleared (and --dns omitted, leaving DNS
+# unchanged) if any step fails, so this can only help, never break resolution.
+priv_ip=\$(ip -4 route get 169.254.169.253 2>/dev/null | awk '{for(i=1;i<=NF;i++) if(\$i=="src"){print \$(i+1); exit}}' || true)
+if [ -n "\$priv_ip" ] && systemctl is-active --quiet systemd-resolved \
+   && echo "DNSStubListenerExtra=\$priv_ip" | sudo tee -a /etc/systemd/resolved.conf >/dev/null \
+   && sudo systemctl restart systemd-resolved; then
+  # Only route containers to the cache once it's actually listening on priv_ip.
+  for _ in 1 2 3 4 5; do
+    sudo ss -lnu "sport = :53" 2>/dev/null | grep -qF "\$priv_ip:53" && { dns_ready=1; break; }
+    sleep 0.5
+  done
+  if [ "\${dns_ready:-0}" = 1 ]; then
+    echo "HOST: DNS cache active on \$priv_ip (systemd-resolved)."
+  else
+    echo "HOST: DNS cache failed to bind \$priv_ip; using default resolver."
+    priv_ip=
+  fi
+else
+  echo "HOST: DNS cache not configured; using default resolver."
+  priv_ip=
+fi
+
 # Pin host processes to top CPU cores to keep benchmark cores clean.
 # CPU layout: physical cores 0..N/2-1, hyperthreads N/2..N-1.
 # OS gets top 8 physical cores + their hyperthread siblings.
@@ -302,7 +332,7 @@ start_build() {
   local_uid=\$(id -u)
   local_gid=\$(id -g)
 
-  docker run --privileged --rm \${docker_args:-} \
+  docker run --privileged --rm \${docker_args:-} \${priv_ip:+--dns \$priv_ip} \
     --name aztec_build \
     --hostname $docker_hostname \
     -v bootstrap_ci_local_docker:/var/lib/docker \