fix(supervisor): KUBERNETES_WORKER_POD_ANNOTATIONS values must be strings

K8s annotations are 'map[string]string' per API spec. The default
JsonStringMap validator on JsonObjectEnv treats each value as a
nested Record<string,string>, which silently accepts object values
like {} and then crashes at the K8s apiserver:

  Pod in version v1 cannot be handled as a Pod: json: cannot unmarshal
  object into Go struct field ObjectMeta.metadata.annotations of type
  string

Real production failure on FedStart with the Rubix pod-cert annotation
(`{"com.palantir.rubix.service/pod-cert": {}}`) — pre-fix the env
var passed zod validation, then kubernetes.ts spread the object into
the Pod spec, and the apiserver rejected with the above. Worker pods
never got created → no task runs ever scheduled.

Fix: override the valueValidator to z.string() for annotations.
Schema now matches K8s annotation reality. Object values are rejected
at zod parse time with a clear error before they ever reach the
apiserver. Security context env vars stay on JsonAny (their values
genuinely ARE nested objects).

Tests: add a 'flat string values, K8s-annotation shape' suite covering:
 - the Rubix pod-cert string value passes
 - multiple flat string annotations pass
 - the exact pre-fix failure shape (object value) is now rejected
 - other non-string values (numeric/boolean/nested map) rejected
Regression case is the literal FedStart input that caused the prod
crash, so anyone changing this validator in the future has to face it.

Co-authored-by: Cursor <cursoragent@cursor.com>

Author: ConProgramming

apps/supervisor/src/env.ts

@@ -95,8 +95,23 @@ const Env = z
     KUBERNETES_WORKER_SERVICE_ACCOUNT: z.string().optional(), // Service account for worker pods
     KUBERNETES_WORKER_AUTOMOUNT_SERVICE_ACCOUNT_TOKEN: BoolEnv.default(false), // Whether to mount SA token
     // Extra annotations to apply to every worker pod (e.g. for service mesh
-    // sidecar injection, certificate injection, scheduling hints).
-    KUBERNETES_WORKER_POD_ANNOTATIONS: JsonObjectEnv("KUBERNETES_WORKER_POD_ANNOTATIONS"),
+    // sidecar injection, certificate injection, scheduling hints). K8s
+    // annotation values are always strings (`map[string]string` per the API
+    // spec) — and `kubernetes.ts` spreads this parsed object straight into
+    // the pod's `metadata.annotations` without any conversion. So each
+    // value here must be a string. The default validator on JsonObjectEnv
+    // is JsonStringMap (nested string→string map), which is the right
+    // shape for security contexts but the wrong shape for annotations:
+    // accepting nested objects here would silently pass zod validation
+    // and then get rejected by the K8s API with
+    //   "Pod in version v1 cannot be handled as a Pod: json: cannot
+    //    unmarshal object into Go struct field
+    //    ObjectMeta.metadata.annotations of type string"
+    // (Real failure mode hit on FedStart with
+    //  `{"com.palantir.rubix.service/pod-cert": {}}`.)
+    KUBERNETES_WORKER_POD_ANNOTATIONS: JsonObjectEnv("KUBERNETES_WORKER_POD_ANNOTATIONS", {
+      valueValidator: z.string(),
+    }),
     // Pod-level securityContext applied to every worker pod (V1PodSecurityContext shape).
     // Default is empty `{}`, preserving the upstream behavior of not setting
     // a pod-level securityContext. Provide a JSON object to enforce e.g.

apps/supervisor/src/envUtil.test.ts

@@ -154,3 +154,65 @@ describe("JsonObjectEnv (arbitrary-value)", () => {
     expect(() => named.parse("{notjson")).toThrowError(/KUBERNETES_WORKER_POD_SECURITY_CONTEXT/);
   });
 });
+
+describe("JsonObjectEnv (flat string values, K8s-annotation shape)", () => {
+  // Regression test for a real production incident on FedStart: passing
+  // KUBERNETES_WORKER_POD_ANNOTATIONS with a nested-map value silently
+  // passed zod validation under the default JsonStringMap validator
+  // (because `{}` is a valid empty `Record<string,string>`), but then
+  // got spread directly into a K8s Pod's metadata.annotations and
+  // rejected by the apiserver:
+  //
+  //   "Pod in version v1 cannot be handled as a Pod: json: cannot
+  //    unmarshal object into Go struct field
+  //    ObjectMeta.metadata.annotations of type string"
+  //
+  // K8s annotations are flat `map[string]string` per the API spec, so
+  // KUBERNETES_WORKER_POD_ANNOTATIONS must validate values as `z.string()`,
+  // not as a nested string map.
+  const schema = JsonObjectEnv("KUBERNETES_WORKER_POD_ANNOTATIONS", {
+    valueValidator: z.string(),
+  });
+
+  it("accepts the Rubix pod-cert annotation as a string value", () => {
+    // Rubix's `com.palantir.rubix.service/pod-cert` annotation expects
+    // its VALUE to be the 2-char string "{}" (which Rubix internally
+    // parses as JSON to determine cert minting config). The env var
+    // input is itself JSON-stringified, so the value here is the
+    // 4-char string `"{}"` — two quotes around two braces.
+    expect(schema.parse('{"com.palantir.rubix.service/pod-cert":"{}"}')).toEqual({
+      "com.palantir.rubix.service/pod-cert": "{}",
+    });
+  });
+
+  it("accepts multiple flat string annotations", () => {
+    expect(
+      schema.parse(
+        '{"com.palantir.rubix.service/pod-cert":"{}","trigger.dev/owner":"supervisor"}'
+      )
+    ).toEqual({
+      "com.palantir.rubix.service/pod-cert": "{}",
+      "trigger.dev/owner": "supervisor",
+    });
+  });
+
+  it("rejects object values (annotations must be flat strings)", () => {
+    // The pre-fix bug: this would have passed zod with the default
+    // JsonStringMap validator, then crashed at K8s apiserver. Schema
+    // must reject before the value ever reaches K8s.
+    expect(() => schema.parse('{"com.palantir.rubix.service/pod-cert":{}}')).toThrowError(
+      /has invalid value/
+    );
+  });
+
+  it("rejects nested string maps as values", () => {
+    expect(() => schema.parse('{"some/annotation":{"nested":"value"}}')).toThrowError(
+      /has invalid value/
+    );
+  });
+
+  it("rejects numeric and boolean values", () => {
+    expect(() => schema.parse('{"some/annotation":42}')).toThrowError(/has invalid value/);
+    expect(() => schema.parse('{"some/annotation":true}')).toThrowError(/has invalid value/);
+  });
+});

hosting/k8s/helm/Chart.yaml

@@ -2,7 +2,7 @@ apiVersion: v2
 name: trigger
 description: The official Trigger.dev Helm chart
 type: application
-version: 4.5.0-rc.4-plt663.6
+version: 4.5.0-rc.4-plt663.7
 appVersion: v4.5.0-rc.4
 home: https://trigger.dev
 sources: