feat(dstack-ingress): delegate every record, so DNS is configured once

[kvinwang]

Follows #107. Breaking: ACME_CHALLENGE_ALIAS becomes DELEGATION_ZONE, and the
records the operator creates change shape.

Delegation exists so the DNS token in the enclave never touches the served
domain's zone. It got the token out; it did not get the operator out. This
finishes the job.

Before

Setting up svc.example.com, with a delegation zone and a token scoped to it:

1. Create three records in the production zone:

   svc.example.com                      CNAME  <GATEWAY_DOMAIN>
   _dstack-app-address.svc.example.com  TXT    "<app-id>:443"
   _acme-challenge.svc.example.com      CNAME  _acme-challenge.svc.example.com.deleg.example.net
   

   The app id has to be looked up, and the container is not running yet to print
   it for you.

2. Deploy, and wait for the first certificate.

3. Come back and create a fourth record, which the container prints only now
   because it names an ACME account that did not exist until step 2:

   svc.example.com  CAA  0 issue "letsencrypt.org;validationmethods=dns-01;accounturi=…"
   

4. Later, whenever the app id moves — it defaults to the hash of
   app-compose.json, so adding an environment variable is enough — go back and
   update the TXT from step 1. Nothing alerts you: the certificate keeps
   renewing, only traffic stops.

Wildcards additionally kept their CAA in the production zone permanently, since
*.example.com is evaluated at example.com.

After

1. Create three CNAMEs in the production zone, before deploying anything.
   Nothing here needs a value you have to look up:

   svc.example.com                      CNAME  svc.example.com.deleg.example.net
   _dstack-app-address.svc.example.com  CNAME  _dstack-app-address.svc.example.com.deleg.example.net
   _acme-challenge.svc.example.com      CNAME  _acme-challenge.svc.example.com.deleg.example.net
   

2. Deploy.

That is the whole of it. There is no step 3, and no later. The container
publishes everything those names point at — the gateway pointer, the app-address
TXT, the challenge, and the accounturi CAA — into the zone it holds a token for,
and keeps them current. An app id that moves, an ACME account that is recreated,
a gateway that changes address: none of them reach the production zone any more.

A wildcard takes one more CNAME, aliasing its base, because RFC 8659
evaluates *.app.example.com at app.example.com:

*.app.example.com                            CNAME  app.example.com.deleg.example.net
app.example.com                              CNAME  app.example.com.deleg.example.net
_dstack-app-address-wildcard.app.example.com CNAME  _dstack-app-address-wildcard.app.example.com.deleg.example.net
_acme-challenge.app.example.com              CNAME  _acme-challenge.app.example.com.deleg.example.net

With that in place the CAA is published for wildcards too. The base has to be a
name you can alias, which a zone apex is not, so a wildcard served straight off
its own zone is not a fit for delegation.

Setting it up

Same as before, minus the second visit:

- CHALLENGE_TYPE=dns-01
- DOMAIN=svc.example.com
- GATEWAY_DOMAIN=_.dstack-prod5.phala.network
- TARGET_ENDPOINT=app:80
- DELEGATION_ZONE=deleg.example.net          # was ACME_CHALLENGE_ALIAS
- CLOUDFLARE_API_TOKEN=<scoped to deleg.example.net only>

Keep /etc/letsencrypt on a volume. The ACME account lives there and the CAA
names it, so losing it means the published CAA no longer matches and issuance
stops.

Evidence for the DNS and CA behaviour this rests on — which records the CA reads
through a CNAME, and why the gateway pointer is a CNAME on Cloudflare and an
address record elsewhere — is in the comments below.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[kvinwang]

What the CA actually does with CAA behind a CNAME

The design rests on this, so each row is a real issuance against Let's Encrypt
staging with a CAA that forbids the CA, checking whether it is refused.

Layout	Verdict
CAA directly on the name (control)	refused
Name is a CNAME; CAA on the target	refused — the CA follows and honours it
Name is a CNAME; target is also a CNAME; CAA at the delegation zone apex	issued — the CA does not climb past the target
Wildcard; CAA on the base's CNAME target, issuewild	refused — While processing CAA for *.wb.kvin.wang: CAA record for wb.kvin.wang prevents issuance

So the CAA has to sit on the name the alias points at, beside the gateway
pointer. That puts a CNAME and a CAA on one name, which RFC 1034 does not allow.
Cloudflare allows it anyway and the CA reads it.

Providers that enforce the standard reject the pair, so the record type follows
the provider:

Provider	Gateway pointer	Gateway that moves
cloudflare	CNAME	followed by DNS, immediately
everything else	A, re-resolved each pass	picked up within RENEW_INTERVAL

Only Cloudflare has been tested, so only Cloudflare gets the optimistic default.
DELEGATION_GATEWAY_RECORD overrides it either way.

Two smaller fixes that came out of testing

Renewal is no longer blocked on records it does not need. Under dns-01 the CA
reads a TXT record and never connects here, so the gateway CNAME and the
app-address TXT are about serving, not issuance. Blocking a renewal on them turns
a routing problem — fixable at any time — into an expired certificate, which is
not. Later passes verify only the challenge alias; the first pass still checks
everything, because that is setup.

The delegated challenge's propagation wait goes from 30s to 120s. It has to
outlast the record's own 60s TTL, not just the write: on the second of the two
issuance attempts a resolver still serving the previous challenge value fails
Let's Encrypt's multi-perspective check with During secondary validation: Incorrect TXT record found. The dns-01 plugin path already waited 120s.

Verification

End to end against staging with real Cloudflare credentials, delegation zone
starting empty, three CNAMEs created by hand:

Successfully set alias record for cc3.kvin.wang.cz3.kvin.wang
Successfully set TXT record for _dstack-app-address.cc3.kvin.wang.cz3.kvin.wang

[dns-check #1] all records verified:
    OK   CNAME cc3.kvin.wang: ok (CNAME)
    OK   CNAME _dstack-app-address.cc3.kvin.wang: ok (CNAME)
    OK   CNAME _acme-challenge.cc3.kvin.wang: ok (CNAME)

✓ Certificate obtained successfully for cc3.kvin.wang
Successfully set CAA record for cc3.kvin.wang.cz3.kvin.wang

Renewal works with the published CAA in place — the case that had never been
exercised before, since the record is only written after a successful issuance
and the second attempt always reports nothing to renew:

# restart, account unchanged: …/acme/acct/318178554
    OK   CAA cc3.kvin.wang: ok (from cc3.kvin.wang)
✓ Certificate renewal completed

A wildcard issues, and every subdomain resolves through one delegated name —
a wildcard CNAME copies its target verbatim rather than expanding it (RFC 4592):

subject=CN = *.wc.kvin.wang,  SAN: DNS:*.wc.kvin.wang

a.wc.kvin.wang.   CNAME  wc.kvin.wang.wz.kvin.wang.
xyz.wc.kvin.wang. CNAME  wc.kvin.wang.wz.kvin.wang.
wc.kvin.wang.wz.kvin.wang. CNAME gwz.kvin.wang. → A 72.52.116.133

App id churn is gone. Changing the TXT in the delegation zone is immediately
visible from the production name with no edit there:

_dstack-app-address.fd1.kvin.wang  →  ['NEWAPPID9999:443']

29 unit tests, the sanitizer suite, bash -n and ast.parse across all scripts.

Still not covered

• A token scoped to only the delegation zone. Testing used a name under a zone
  the token already owned, which exercises every line of the mechanism but not
  the scoping property it exists for.
• A gateway that actually changes address mid-deployment.
• Any provider other than Cloudflare.