fix(deploy): let the cluster own the Gateway, keep only the route (#518)

This repo cannot know the cluster it lands in, and gateway-listeners.yaml was
trying to. Between them, the ListenerSet and its routes hardcoded four facts
about cfp-live-cluster — the shared Gateway's name and namespace, the
ClusterIssuer, the TLS Secret name, and the ListenerSet API itself — and three
were wrong there:

  - cert-manager.io/cluster-issuer: letsencrypt-prod is dead. It solves ACME
    over ingress-nginx, which requires PROXY protocol on every connection and
    is therefore unreachable from inside the cluster. No certificate has issued
    through it since May.
  - certificateRefs: balancer-tls is the legacy Ingress Secret. Gateway certs
    are <app>-gw-tls.
  - ListenerSet is not reconciled by Envoy Gateway v1.7.3. It watches
    XListenerSet and logs "XListenerSet CRD not found, skipping XListenerSet
    watch". The object applies cleanly and is then ignored — the app just has
    no listener, with nothing in any log to say why.

None of that is a mistake anyone could have avoided from this repo. It isn't
knowable here. So stop trying to know it.

Split on what each side actually knows:

  this repo -> HTTPRoute: paths, backend Services, ports. App facts. They
               change when the app changes, in the same commit.
  cluster   -> Gateway: listeners, hostnames, TLS, issuers. Cluster facts. They
               differ per environment and are shared with every other app.

The route now attaches to a Gateway the cluster provides, named after the app,
and declares no hostnames — a route that omits hostnames inherits them from the
listeners it attaches to. So one file serves balancerproject.org in live and
balancer.sandbox.k8s.phl.io in sandbox, with no patches and no placeholders.
The HOSTNAME_PLACEHOLDER substitutions and the overlay routing patches are gone.

Drops balancer-http-redirect too: the cluster already redirects every hostname
reaching it on port 80. Worse than redundant — a route claiming an exact
hostname on the shared HTTP listener outranks the catch-all redirect and can
shadow cert-manager's ACME solver route, which is what produced the apex 404
that #517 set out to fix. That 404 was a stuck certificate, and it cleared the
moment the cert issued.

Adds a README stating the contract, so the next person doesn't have to infer it.

Refs CodeForPhilly/cfp-live-cluster#166, #168

Author: themightychris

deploy/manifests/balancer/README.md

@@ -0,0 +1,60 @@
+# balancer deployment manifests
+
+Consumed by the Code for Philly cluster repos
+([cfp-live-cluster](https://github.com/CodeForPhilly/cfp-live-cluster),
+[cfp-sandbox-cluster](https://github.com/CodeForPhilly/cfp-sandbox-cluster)), which project
+`base/` into their GitOps trees. Also usable directly against a local `kind` cluster
+(`deploy/kind-config.yaml`).
+
+## Who owns what
+
+Routing is split along a single line: **this repo declares what the app serves; the
+cluster declares where it's reachable.**
+
+| This repo owns | The cluster owns |
+| --- | --- |
+| `HTTPRoute` — paths, backend Services, ports | `Gateway` — listeners, hostnames, TLS |
+| Deployment, Service, app config | Certificates, ClusterIssuers, `GatewayClass` |
+
+Paths and ports are facts about the application: they change when the app changes, and
+they belong in the same commit as that change. Hostnames, TLS, and issuers are facts
+about the cluster: they differ between live and sandbox, they're negotiated across every
+app sharing the ingress, and they change when the *cluster* changes.
+
+## The contract
+
+**The cluster provides a `Gateway` named after the app, in the app's namespace.** That
+sentence is the whole interface.
+
+`base/httproute.yaml` attaches to it by name and declares no hostnames:
+
+```yaml
+spec:
+  parentRefs:
+    - name: balancer     # the Gateway the cluster provides
+  # no hostnames — inherited from the Gateway's listeners
+```
+
+A route that omits `hostnames` inherits them from the listeners it attaches to. So the
+same file serves `balancerproject.org` in live and `balancer.sandbox.k8s.phl.io` in
+sandbox, with no patching and no placeholders.
+
+## What must NOT live in this repo
+
+Anything that names a cluster implementation detail:
+
+- **`Gateway` or `ListenerSet` resources.** Cluster-owned.
+- **`cert-manager.io/cluster-issuer` annotations**, or any ClusterIssuer name.
+- **TLS `certificateRefs` / cert Secret names.**
+- **`gatewayClassName`**, or the name/namespace of any shared Gateway.
+- **An HTTP→HTTPS redirect.** The cluster already redirects every hostname reaching it
+  on port 80. An app-level redirect is redundant, and a route claiming an exact hostname
+  on the shared HTTP listener can shadow cert-manager's ACME solver route.
+
+None of these are knowable from this repo, and getting one wrong fails in ways that are
+hard to see. A `ListenerSet`, for instance, applies cleanly to the API server and is then
+silently ignored by Envoy Gateway v1.7.3 — the app simply has no listener, with nothing
+in any log to say why.
+
+If you need something from the cluster that isn't here, open an issue on the cluster repo
+rather than guessing at its internals.

deploy/manifests/balancer/base/gateway-listeners.yaml

@@ -2,16 +2,20 @@ apiVersion: gateway.networking.k8s.io/v1
 kind: HTTPRoute
 metadata:
   name: balancer
-  annotations:
-    hostname: HOSTNAME_PLACEHOLDER
 spec:
+  # Attaches to the Gateway the cluster provides for this app. The cluster names
+  # it after the app, in the app's namespace — that convention is the entire
+  # contract. See ../README.md.
   parentRefs:
-    - name: balancer-listeners
-      kind: ListenerSet
-      group: gateway.networking.k8s.io
-      sectionName: https
-  hostnames:
-    - HOSTNAME_PLACEHOLDER
+    - name: balancer
+
+  # No `hostnames` on purpose.
+  #
+  # A route that omits hostnames inherits the hostnames of the listeners it
+  # attaches to, so the cluster's Gateway decides where this app is reachable —
+  # balancerproject.org in live, balancer.sandbox.k8s.phl.io in sandbox — and
+  # this file stays identical in every environment.
+
   rules:
     - matches:
         - path:
@@ -20,24 +24,3 @@ spec:
       backendRefs:
         - name: balancer
           port: 80
----
-apiVersion: gateway.networking.k8s.io/v1
-kind: HTTPRoute
-metadata:
-  name: balancer-http-redirect
-  annotations:
-    hostname: HOSTNAME_PLACEHOLDER
-spec:
-  parentRefs:
-    - name: balancer-listeners
-      kind: ListenerSet
-      group: gateway.networking.k8s.io
-      sectionName: http
-  hostnames:
-    - HOSTNAME_PLACEHOLDER
-  rules:
-    - filters:
-        - type: RequestRedirect
-          requestRedirect:
-            scheme: https
-            statusCode: 301

deploy/manifests/balancer/base/httproute.yaml

@@ -5,7 +5,6 @@ resources:
   - namespace.yaml
   - deployment.yaml
   - service.yaml
-  - gateway-listeners.yaml
   - httproute.yaml
 
 labels:

deploy/manifests/balancer/base/kustomization.yaml

@@ -24,28 +24,9 @@ labels:
       environment: production
       app.kubernetes.io/instance: balancer-production
 
-patches:
-  - target:
-      kind: ListenerSet
-      name: balancer-listeners
-    patch: |
-      - op: replace
-        path: /spec/listeners/0/hostname
-        value: balancerproject.org
-      - op: replace
-        path: /spec/listeners/1/hostname
-        value: balancerproject.org
-  - target:
-      kind: HTTPRoute
-      name: balancer
-    patch: |
-      - op: replace
-        path: /spec/hostnames/0
-        value: balancerproject.org
-  - target:
-      kind: HTTPRoute
-      name: balancer-http-redirect
-    patch: |
-      - op: replace
-        path: /spec/hostnames/0
-        value: balancerproject.org
+# No routing patches. The cluster's Gateway carries the hostname and TLS, and
+# the HTTPRoute inherits it — see ../../README.md.
+#
+# HOSTNAME above is still set here because the *application* needs to know its
+# own public name (Django ALLOWED_HOSTS / CSRF_TRUSTED_ORIGINS). That's app
+# config, not routing.

deploy/manifests/balancer/overlays/production/kustomization.yaml

@@ -25,31 +25,9 @@ labels:
       environment: sandbox
       app.kubernetes.io/instance: balancer-sandbox
 
+# No routing patches — the cluster's Gateway carries the hostname and TLS, and
+# the HTTPRoute inherits it. See ../../README.md.
 patches:
-  - target:
-      kind: ListenerSet
-      name: balancer-listeners
-    patch: |
-      - op: replace
-        path: /spec/listeners/0/hostname
-        value: sandbox.balancerproject.org
-      - op: replace
-        path: /spec/listeners/1/hostname
-        value: sandbox.balancerproject.org
-  - target:
-      kind: HTTPRoute
-      name: balancer
-    patch: |
-      - op: replace
-        path: /spec/hostnames/0
-        value: sandbox.balancerproject.org
-  - target:
-      kind: HTTPRoute
-      name: balancer-http-redirect
-    patch: |
-      - op: replace
-        path: /spec/hostnames/0
-        value: sandbox.balancerproject.org
   - target:
       kind: Deployment
       name: balancer