chart(0.3.28): sync from internal source + Hetzner nodeSelector fix

Catches the public chart up to the internal source of truth at
appVersion 0.10.21 (was 0.10.10):

- per-pod TCPRoute listeners (`pgwire.publicTcpPortBase`) — chart was
  missing the 0.10.17 work, so customers on the public chart had no
  way to expose the L4-passthrough port range that DBeaver / Tableau /
  libpq <17 / DuckDB's bundled postgres need.
- configmap.yaml: __POD_TCP_PORT__ placeholder + sed substitution
- gateway.yaml: per-pod TCPRoute listeners
- statefulset.yaml + superadmin-secret.yaml: minor sync

values-hetzner-example.yaml:
- image tag bumped to aarch64-generic-linux-0.10.21
- **nodeSelector switched to `karpenter.sh/nodepool: boilstream-arm64-hel`**.
  The CFKE-template label `boilstream.com/nodepool: arm64-hel` is in
  the NodePool's `template.metadata.labels` but not in
  `spec.requirements`, and CFKE forbids customer modifications to
  NodePool requirements — so freshly-provisioned Karpenter nodes were
  failing to satisfy the pod's nodeSelector. Karpenter-managed labels
  are auto-applied to every node Karpenter provisions; switching to
  that path makes the chart durable across NodePool reconciliations.
- bucket name is now a placeholder `<your-hetzner-bucket-name>` with
  the prior staging-specific value as a documentation example.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: dforsber

CHANGELOG.md

@@ -11,6 +11,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - **PGWire stability for clients that auto-name prepared statements.** Connection-pool reuse no longer leaks server-side prepared statements between successive PGWire clients, which previously surfaced as intermittent `prepared statement "sN" already exists` errors against any libpq / tokio-postgres / pgjdbc workload that allocates statement names automatically (DBeaver, Tableau ODBC, Power BI, JDBC apps, psycopg, DuckDB's bundled `postgres` extension). 31 binary-parameter / parameter-types / boolean-encoding / prepared-statement test cases now pass deterministically against staging where they previously flaked. No client-side changes required.
 
+### Chart 0.3.28 — Hetzner / CloudFleet survivability
+
+- **`values-hetzner-example.yaml` now selects nodes via `karpenter.sh/nodepool: boilstream-arm64-hel`** instead of the CFKE-template label `boilstream.com/nodepool: arm64-hel`. The Karpenter-managed label is auto-applied to every node Karpenter provisions, so freshly-spun-up nodes are picked up without any extra requirement on the NodePool spec — and CFKE forbids customers from modifying NodePool requirements directly.
+- **Chart catches up to the per-pod TCPRoute work shipped in 0.10.17 / 0.10.18**: `pgwire.publicTcpPortBase` (default `15432`), the `__POD_TCP_PORT__` substitution in the rendered config, and the per-pod `TCPRoute` listeners are now on the public chart. Customers running the public chart no longer need to pin to a pre-0.10.17 image.
+
 ### Notes
 
 - Chart version **0.3.27** tracks appVersion `0.10.21`.

charts/boilstream/Chart.yaml

@@ -6,8 +6,8 @@ description: |
   support. Each pod participates in S3-based leader election and serves
   per-user catalogs; failed pods are recovered from S3 backups.
 type: application
-version: 0.3.16
-appVersion: "0.10.10"
+version: 0.3.28
+appVersion: "0.10.21"
 kubeVersion: ">=1.27.0"
 keywords:
   - streaming

charts/boilstream/templates/configmap.yaml

@@ -76,11 +76,21 @@ data:
     #                   clients can connect back to the correct pod from outside
     #                   the cluster. Kept in sync with the per-pod TLSRoute in
     #                   gateway.yaml: boilstream-N.{{ .Values.domain }}
+    # public_tcp_port = per-pod external TCP port baked into vended DuckLake
+    #                   credentials. Pairs with the per-pod `TCPRoute` listener
+    #                   in gateway.yaml at `publicTcpPortBase + pod_index`.
+    #                   Envoy forwards that port L4-plain straight to the
+    #                   pod's :5432, so remote clients speak traditional
+    #                   Postgres SSL (plaintext SSLRequest → TLS upgrade)
+    #                   with the pod directly — DuckDB's bundled libpq <17
+    #                   works with just `sslmode=require`. __POD_TCP_PORT__
+    #                   is resolved by the shell below once POD_NAME is known.
     cluster_block = '''cluster_mode:
       enabled: true
       node_id_path: "/data/node-id"
       advertised_host: "__POD_NAME__.boilstream-headless.__NS__.svc.cluster.local"
       public_host: "__POD_NAME__.{{ .Values.domain }}"
+      public_tcp_port: __POD_TCP_PORT__
       internal_api_port: 8444
       leader_heartbeat_interval_secs: 15
       leader_stale_threshold_secs: 60
@@ -189,8 +199,14 @@ data:
     with open(p, 'w') as f: f.write(c)
     PY
 
-    # Substitute pod-specific placeholders in the final config
-    sed -i "s/__POD_NAME__/${POD_NAME}/g; s/__NS__/${NS}/g" /cfg/config.yaml
+    # Substitute pod-specific placeholders in the final config.
+    # POD_TCP_PORT = publicTcpPortBase + pod ordinal. StatefulSet names pods
+    # as <statefulset>-0, <statefulset>-1, ... so the trailing integer is
+    # the ordinal. Adding it to the base (15432) gives each pod a unique
+    # external TCP port matching the gateway's per-pod TCPRoute listener.
+    POD_ORDINAL="${POD_NAME##*-}"
+    POD_TCP_PORT=$(( {{ int .Values.pgwire.publicTcpPortBase }} + POD_ORDINAL ))
+    sed -i "s/__POD_NAME__/${POD_NAME}/g; s/__NS__/${NS}/g; s/__POD_TCP_PORT__/${POD_TCP_PORT}/g" /cfg/config.yaml
 
     # Generate (once per pod) a self-signed loopback cert covering
     # 127.0.0.1 / ::1 / localhost. Auth server serves this cert via SNI for

charts/boilstream/templates/gateway.yaml

@@ -67,6 +67,25 @@ spec:
       allowedRoutes:
         namespaces: { from: Same }
         kinds: [ { kind: TLSRoute } ]
+    {{- /*
+      Per-pod PgWire TCP passthrough (TCPRoute, NOT TLSRoute). Each listener
+      is a pure L4 pipe from its LB port straight into one specific pod's
+      :5432. Envoy doesn't parse TLS for these — the plaintext Postgres
+      SSLRequest preamble reaches the pod unmolested, so libpq ≥9 (incl.
+      DuckDB's bundled libpq <17) can connect with plain `sslmode=require`.
+      The port we pick is `pgwire.publicTcpPortBase + pod_index` so clients
+      compute it as `15432 + N` without coordination, matching
+      `boilstream-N.{{ .Values.domain }}`.
+    */}}
+    {{- $tcpBase := int .Values.pgwire.publicTcpPortBase }}
+    {{- range $i, $_ := until (int .Values.replicas) }}
+    - name: pgwire-pod-{{ $i }}
+      protocol: TCP
+      port: {{ add $tcpBase $i }}
+      allowedRoutes:
+        namespaces: { from: Same }
+        kinds: [ { kind: TCPRoute } ]
+    {{- end }}
 ---
 {{- $vals := . -}}
 {{- /*
@@ -128,5 +147,29 @@ spec:
         {{- end }}
 ---
 {{- end }}
+
+{{- /*
+  Per-pod PgWire TCPRoutes. One route per pod, each attached to its own
+  dedicated TCP listener. No SNI, no hostname filtering — the listener
+  itself has exactly one backend, so Envoy is a straight L4 proxy. Clients
+  use `host=boilstream-N.{{ .Values.domain }} port={{ .Values.pgwire.publicTcpPortBase }}+N`
+  and talk plain Postgres + SSLRequest upgrade to the pod.
+*/}}
+{{- $tcpBase := int .Values.pgwire.publicTcpPortBase }}
+{{- range $i, $_ := until (int .Values.replicas) }}
+apiVersion: gateway.networking.k8s.io/v1alpha2
+kind: TCPRoute
+metadata:
+  name: boilstream-{{ $i }}-pgwire-tcp
+  namespace: {{ $vals.Values.namespace }}
+spec:
+  parentRefs:
+    - { name: boilstream, sectionName: pgwire-pod-{{ $i }} }
+  rules:
+    - backendRefs:
+        - name: boilstream-{{ $i }}
+          port: 5432
+---
+{{- end }}
 {{- end }}
 

charts/boilstream/templates/statefulset.yaml

@@ -1,5 +1,6 @@
 {{- $tlsSecret := default "boilstream-wildcard-tls" .Values.tls.existingSecret -}}
 {{- $superSecret := default "boilstream-superadmin" .Values.superadmin.existingSecret -}}
+{{- $superMfaSecret := default "boilstream-superadmin-mfa" .Values.superadmin.existingMfaSecret -}}
 {{- $clusterTlsSecret := default "boilstream-cluster-mtls" .Values.clusterTls.existingSecret -}}
 apiVersion: apps/v1
 kind: StatefulSet
@@ -81,13 +82,11 @@ spec:
                   name: {{ .Values.s3.existingCredsSecret }}
                   key: secret_key
             {{- end }}
-            {{- if .Values.superadmin.existingMfaSecret }}
             - name: SUPERADMIN_MFA_SECRET
               valueFrom:
                 secretKeyRef:
-                  name: {{ .Values.superadmin.existingMfaSecret }}
+                  name: {{ $superMfaSecret }}
                   key: mfa_secret
-            {{- end }}
           volumeMounts:
             - { name: config-tpl, mountPath: /tpl }
             - { name: config,     mountPath: /cfg }

charts/boilstream/templates/superadmin-secret.yaml

@@ -1,3 +1,25 @@
+{{- /*
+Superadmin Secrets.
+
+Two secrets:
+  - boilstream-superadmin        — password (always required; operator
+                                   supplies .Values.superadmin.password
+                                   OR points at an existingSecret).
+  - boilstream-superadmin-mfa    — base32 TOTP secret. Optional for the
+                                   operator: if they pass
+                                   .Values.superadmin.existingMfaSecret
+                                   we defer to that; if they pass a
+                                   literal .Values.superadmin.mfaSecret
+                                   we use it; otherwise we generate a
+                                   fresh 20-byte random secret and
+                                   persist it so pod restarts don't
+                                   require re-enrolment of the operator's
+                                   authenticator app.
+
+The auto-generated path uses lookup() to preserve the same secret across
+helm upgrades — regenerating on every upgrade would invalidate every
+superadmin TOTP.
+*/ -}}
 {{- if not .Values.superadmin.existingSecret }}
 apiVersion: v1
 kind: Secret
@@ -10,3 +32,29 @@ type: Opaque
 stringData:
   password: {{ required ".Values.superadmin.password is required when superadmin.existingSecret is empty" .Values.superadmin.password | quote }}
 {{- end }}
+
+{{- if not .Values.superadmin.existingMfaSecret }}
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: boilstream-superadmin-mfa
+  namespace: {{ .Values.namespace }}
+  labels:
+    {{- include "boilstream.labels" . | nindent 4 }}
+  annotations:
+    "helm.sh/resource-policy": keep
+type: Opaque
+stringData:
+  {{- /* Priority:
+           1. explicit .Values.superadmin.mfaSecret (operator-supplied)
+           2. existing secret in cluster (re-reads on helm upgrade)
+           3. freshly generated 20-byte random, base32-encoded per RFC 4648 §6 */}}
+  mfa_secret: {{- with .Values.superadmin.mfaSecret }} {{ . | quote }}
+  {{- else }}
+  {{- $existing := (lookup "v1" "Secret" .Values.namespace "boilstream-superadmin-mfa") }}
+  {{- if and $existing $existing.data $existing.data.mfa_secret }} {{ $existing.data.mfa_secret | b64dec | quote }}
+  {{- else }} {{ randAlphaNum 32 | b32enc | trimSuffix "=" | quote }}
+  {{- end }}
+  {{- end }}
+{{- end }}

charts/boilstream/values-hetzner-example.yaml

@@ -12,7 +12,7 @@
 #   helm install boilstream ./charts/boilstream \
 #     -f charts/boilstream/values-hetzner-example.yaml \
 #     --set image.repository=<your-cfcr-tenant>.europe.registry.cloudfleet.dev/boilstream \
-#     --set image.tag=0.10.10-pro-1.5.2 \
+#     --set image.tag=aarch64-generic-linux-0.10.21 \
 #     --set superadmin.existingSecret=boilstream-superadmin
 
 replicas: 2                              # 2 nodes, 1 pod each
@@ -25,7 +25,7 @@ image:
   # Ampere A1 / other non-Graviton-3+ ARM64 hosts need to avoid SIGILL on
   # SHA3 / SHA512 paths.
   repository: docker.io/boilinginsights/boilstream
-  tag: aarch64-generic-linux-0.10.10
+  tag: aarch64-generic-linux-0.10.21
   pullPolicy: IfNotPresent
   pullSecrets: []                        # Docker Hub is public; no pull secret needed
 
@@ -34,7 +34,7 @@ domain: app.boilstream.com                  # local-test wildcard while we verif
 # Hetzner Object Storage in Helsinki (S3-compatible).
 s3:
   endpoint: https://hel1.your-objectstorage.com
-  bucket: boilstream-cluster-state-hel1
+  bucket: <your-hetzner-bucket-name>     # e.g. boilstream-cluster-state-hel1
   region: hel1
   prefix: cluster/
   # Path-style ({endpoint}/{bucket}/{key}) — current BoilStream/object_store
@@ -56,9 +56,17 @@ resources:
   requests: { cpu: "1",   memory: "2Gi" }
   limits:   { cpu: "2",   memory: "4Gi" }
 
-# Pin to the boilstream-arm64-hel NodePool (arm64 + hel1).
+# Pin to the Karpenter NodePool that fronts your Hetzner ARM64 fleet.
+# Use the Karpenter-managed `karpenter.sh/nodepool` label rather than the
+# CFKE-managed `boilstream.com/nodepool: arm64-hel` template label, because
+# CFKE forbids customer modifications to NodePool requirements — and the
+# `karpenter.sh/nodepool` label is auto-applied by Karpenter to every node
+# it provisions, so freshly-spun nodes are picked up without an extra
+# requirement on the NodePool spec. The value must match your actual
+# NodePool's `metadata.name` (default in the CloudFleet bring-up flow is
+# `boilstream-arm64-hel`).
 nodeSelector:
-  boilstream.com/nodepool: arm64-hel
+  karpenter.sh/nodepool: boilstream-arm64-hel
 
 # Soft anti-affinity — pods prefer separate nodes, but tolerate co-location
 # during capacity churn. Hard `requiredDuringScheduling` caused a 3-way

charts/boilstream/values.yaml

@@ -22,6 +22,16 @@ image:
 # <pod-name>.<domain>:<port> via SNI through Envoy Gateway.
 domain: boilstream.test
 
+# PgWire external exposure.
+pgwire:
+  # Base TCP port for per-pod direct listeners. Each pod gets its own Envoy
+  # TCPRoute on `publicTcpPortBase + pod_index` (e.g. 15432, 15433, ...).
+  # Traffic is forwarded L4-plain to the pod's :5432, so clients speak the
+  # traditional Postgres SSL flow (plaintext SSLRequest → TLS upgrade)
+  # directly with the pod — no SNI, no `sslnegotiation=direct`, works with
+  # DuckDB's bundled libpq <17.
+  publicTcpPortBase: 15432
+
 # S3 / object storage for cluster coordination + per-user catalog backups.
 s3:
   endpoint: http://host.docker.internal:9000