chore(infra): retire self-hosted MinIO — DO Spaces is canonical (supersedes #4) (#16)

DO Spaces (nyc3.digitaloceanspaces.com, bucket instant-shared) has
been the active object-store backend in production since 2026-05-11.
The self-hosted MinIO Deployment in instant-data is no longer in the
request path for /storage/new — verified via live cluster:

  OBJECT_STORE_BACKEND=shared-key  (alias → do-spaces)
  OBJECT_STORE_ENDPOINT=nyc3.digitaloceanspaces.com
  OBJECT_STORE_BUCKET=instant-shared

This commit retires the local MinIO manifests and replaces every
MINIO_* env injection on production k8s deployments with OBJECT_STORE_*
sourced from instant-secrets / instant-infra-secrets.

Supersedes PR #4 (2026-05-11), which had drifted from current master
after the broad manifest reconciliation work landed.

Manifests deleted:
  - k8s/data/minio.yaml             (Deployment + PVC + ClusterIP + NodePort)
  - k8s/data/minio-bucket-init.yaml (one-shot Job creating instant-shared)
  - k8s/data/minio-secret.yaml      (local-dev MinIO root creds)

Env-var injection migrated MINIO_* → OBJECT_STORE_* in:
  - k8s/app.yaml             (instant-api) — removed MINIO_ROOT_USER/PASSWORD
                              optional refs; OBJECT_STORE_* keys already
                              wired via prior storage-abstraction work.
  - k8s/worker/deployment.yaml (instant-worker) — storage_bytes scanner
                              now reads OBJECT_STORE_* from instant-infra-secrets.
  - k8s/provisioner/deployment.yaml (instant-provisioner) — storage_bytes
                              scanner block migrated. Also removes the
                              dangling reference to a 'minio-secrets'
                              Secret in the instant-infra namespace that
                              was never defined in-tree (the in-repo
                              Secret lived in instant-data).
  - k8s/configmap.yaml — removed MINIO_ENDPOINT / MINIO_BUCKET_NAME.
  - k8s/secrets.yaml (template) — removed MINIO_ROOT_USER/PASSWORD
                              template keys; added OBJECT_STORE_*
                              template keys for DO Spaces.

Docs:
  - k8s/APPLY-CHECKLIST.md — new 'MinIO retirement' section with the
    post-merge operator cleanup commands (kubectl delete deploy/minio
    pvc/minio-data svc/{minio,minio-external} job/minio-bucket-init
    secret/minio-secrets in instant-data). Per CLAUDE.md rule 15
    (infra has no auto-apply), the operator runs this manually after
    merge.

Verification:
  - kubectl apply --dry-run=server -f k8s/{app,configmap,secrets,
    provisioner/deployment,worker/deployment}.yaml — all clean (no
    schema errors; pre-existing E2E_TEST_TOKEN hides-previous warning
    is unrelated to this change).
  - Live confirmation prod is on DO Spaces: instant-secrets has
    OBJECT_STORE_BACKEND=shared-key (alias to do-spaces),
    OBJECT_STORE_ENDPOINT=nyc3.digitaloceanspaces.com.

Coverage block:
  Symptom:       Self-hosted MinIO still deployed in instant-data while
                 prod traffic is on DO Spaces
  Enumeration:   rg -n -i 'minio' k8s/ --type yaml
  Sites found:   8 (3 manifest files + 5 env-injection blocks across
                 app/configmap/worker/provisioner/secrets)
  Sites touched: 8 (3 deleted + 5 migrated to OBJECT_STORE_*)
  Coverage test: kubectl apply --dry-run=server clean across all
                 mutated manifests (no resolveError on missing minio-
                 secrets, no env-var redefinition warnings beyond the
                 pre-existing E2E_TEST_TOKEN one)
  Live verified: Operator runs APPLY-CHECKLIST.md 'MinIO retirement'
                 section commands post-merge; prod request path
                 verified pre-PR via OBJECT_STORE_BACKEND env on live
                 instant-secrets (DO Spaces, nyc3, instant-shared).

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: mastermanas805

k8s/APPLY-CHECKLIST.md

@@ -170,6 +170,57 @@ of silently regressing.
 
 ---
 
+## MinIO retirement (2026-05-20)
+
+The self-hosted MinIO Deployment in `instant-data` was retired in
+`chore/retire-self-hosted-minio-2026-05-20` (supersedes the stale PR #4
+from 2026-05-11). DO Spaces (`nyc3.digitaloceanspaces.com`, bucket
+`instant-shared`) is the canonical production object-store backend, selected
+by `OBJECT_STORE_BACKEND=do-spaces` in `instant-secrets` (and mirrored to
+`instant-infra-secrets` for worker + provisioner storage_bytes scanners).
+
+After this PR merges, run the following on the prod cluster (and on any
+local Rancher Desktop clusters that still have the legacy MinIO workload
+deployed):
+
+```bash
+# 1. Confirm context (do NOT run against the wrong cluster)
+kubectl config current-context
+
+# 2. Inventory what's actually there before deleting anything
+kubectl get deploy,pvc,svc,job,secret -n instant-data -l app=minio
+kubectl get secret -n instant-data minio-secrets 2>/dev/null || echo "no minio-secrets"
+
+# 3. Remove the MinIO workload + storage + services + bootstrap Job + Secret
+kubectl delete -n instant-data deploy/minio --ignore-not-found
+kubectl delete -n instant-data pvc/minio-data --ignore-not-found
+kubectl delete -n instant-data svc/minio svc/minio-external --ignore-not-found
+kubectl delete -n instant-data job/minio-bucket-init --ignore-not-found
+kubectl delete -n instant-data secret/minio-secrets --ignore-not-found
+
+# 4. Verify nothing left
+kubectl get pods -n instant-data | grep -i minio   # should print nothing
+
+# 5. (Optional) If the legacy `s3.instanode.dev` Ingress still points at
+#    the minio Service, delete or repoint it to DO Spaces. The Ingress
+#    object was untracked in this repo — confirm what's live:
+kubectl get ingress -A | grep -i minio
+
+# 6. Sanity-check the storage hot path is unaffected
+curl -sS https://api.instanode.dev/healthz | jq .
+# /storage/new responses should reference *.digitaloceanspaces.com,
+# not minio.instant-data.svc.cluster.local.
+```
+
+Rollback: revert the merge commit on master, re-apply the deleted
+manifests from history (`git show <revert-sha>~1 -- k8s/data/minio*.yaml |
+kubectl apply -f -`), and flip `OBJECT_STORE_BACKEND` back to `minio` in
+`instant-secrets`. Storage data isn't lost in either direction — the PVC
+was on local-path in Rancher Desktop only; DO Spaces holds the real
+production object bytes.
+
+---
+
 ## Related files
 
 - `README.md` — secrets clobber warning (the same class of bug, but for

k8s/app.yaml

@@ -194,18 +194,9 @@ spec:
                   name: instant-secrets
                   key: R2_BUCKET_NAME
                   optional: true
-            - name: MINIO_ROOT_USER
-              valueFrom:
-                secretKeyRef:
-                  name: instant-secrets
-                  key: MINIO_ROOT_USER
-                  optional: true
-            - name: MINIO_ROOT_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: instant-secrets
-                  key: MINIO_ROOT_PASSWORD
-                  optional: true
+            # MINIO_ROOT_USER/MINIO_ROOT_PASSWORD env refs retired 2026-05-20
+            # (chore/retire-self-hosted-minio-2026-05-20). The api reads
+            # OBJECT_STORE_* below for the canonical DO Spaces backend.
             # ── Razorpay annual plan IDs (codified 2026-05-20) ────────────────
             # Live since the annual-pricing toggle shipped. Required for any
             # *_yearly plan checkout to succeed. NOT optional in prod, but
@@ -262,8 +253,10 @@ spec:
                   optional: true
             # ── Object-store backend (storage abstraction, 2026-05-20) ────────
             # OBJECT_STORE_BACKEND selects the impl: do-spaces (prod today),
-            # r2, s3, or minio. The rest are backend-specific. See
-            # common/storageprovider/factory.go.
+            # r2, or s3. The rest are backend-specific. See
+            # common/storageprovider/factory.go. The self-hosted MinIO backend
+            # was retired 2026-05-20 — DO Spaces (`nyc3.digitaloceanspaces.com`,
+            # bucket `instant-shared`) is the canonical production backend.
             - name: OBJECT_STORE_BACKEND
               valueFrom:
                 secretKeyRef:

k8s/configmap.yaml

@@ -36,8 +36,9 @@ data:
   COMPUTE_PROVIDER: "k8s"
   STACK_EXPOSE_VIA: "nodeport"
   KUBE_NAMESPACE_APPS: "instant-apps"
-  MINIO_ENDPOINT: "minio.instant-data.svc.cluster.local:9000"
-  MINIO_BUCKET_NAME: "instant-shared"
+  # MINIO_ENDPOINT / MINIO_BUCKET_NAME retired 2026-05-20 (self-hosted MinIO
+  # retirement). Object storage now lives behind OBJECT_STORE_* in
+  # instant-secrets (DO Spaces, nyc3, bucket instant-shared).
   # Deploy / stack ingress wiring. DEPLOY_DOMAIN is the wildcard zone the
   # api builds per-deploy hostnames under (<id>.deployment.instanode.dev);
   # without it the code falls back to the hardcoded default "instant.dev".

k8s/data/minio-bucket-init.yaml

@@ -201,21 +201,55 @@ spec:
               value: "1"
             - name: K8S_MONGO_STORAGE_GI
               value: "1"
-            # ── MinIO storage bytes queries ───────────────────────────────────
-            - name: MINIO_ENDPOINT
-              value: "minio.instant-data.svc.cluster.local:9000"
-            - name: MINIO_BUCKET_NAME
-              value: "instant-shared"
-            - name: MINIO_ROOT_USER
+            # ── Object-storage bytes queries ──────────────────────────────────
+            # Migrated 2026-05-20 from MINIO_* to OBJECT_STORE_* as part of
+            # chore/retire-self-hosted-minio-2026-05-20 (supersedes PR #4).
+            # Self-hosted MinIO retired; DO Spaces is the canonical backend.
+            # Also removes a previously dangling reference to a `minio-secrets`
+            # Secret in the instant-infra namespace that was never defined in
+            # the tree (the in-repo Secret lived in instant-data).
+            - name: OBJECT_STORE_BACKEND
               valueFrom:
                 secretKeyRef:
-                  name: minio-secrets
-                  key: MINIO_ROOT_USER
-            - name: MINIO_ROOT_PASSWORD
+                  name: instant-infra-secrets
+                  key: OBJECT_STORE_BACKEND
+                  optional: true
+            - name: OBJECT_STORE_ENDPOINT
+              valueFrom:
+                secretKeyRef:
+                  name: instant-infra-secrets
+                  key: OBJECT_STORE_ENDPOINT
+                  optional: true
+            - name: OBJECT_STORE_REGION
+              valueFrom:
+                secretKeyRef:
+                  name: instant-infra-secrets
+                  key: OBJECT_STORE_REGION
+                  optional: true
+            - name: OBJECT_STORE_ACCESS_KEY
+              valueFrom:
+                secretKeyRef:
+                  name: instant-infra-secrets
+                  key: OBJECT_STORE_ACCESS_KEY
+                  optional: true
+            - name: OBJECT_STORE_SECRET_KEY
+              valueFrom:
+                secretKeyRef:
+                  name: instant-infra-secrets
+                  key: OBJECT_STORE_SECRET_KEY
+                  optional: true
+            - name: OBJECT_STORE_BUCKET
               valueFrom:
                 secretKeyRef:
-                  name: minio-secrets
-                  key: MINIO_ROOT_PASSWORD
+                  name: instant-infra-secrets
+                  key: OBJECT_STORE_BUCKET
+                  optional: true
+            - name: OBJECT_STORE_SECURE
+              valueFrom:
+                secretKeyRef:
+                  name: instant-infra-secrets
+                  key: OBJECT_STORE_SECURE
+                  optional: true
             # ── OTLP headers (NR license key) — CODIFIED INTENT ───────────────
             # The live deployment has this as an inline `value:` literal (NR
             # license key embedded). Codified intent is to pull from

k8s/data/minio-secret.yaml

@@ -51,11 +51,18 @@ stringData:
   R2_API_TOKEN: "CHANGE_ME"
   R2_ENDPOINT: "CHANGE_ME"
   R2_BUCKET_NAME: "CHANGE_ME"
-  # MinIO credentials (same values as in infra/k8s/data/minio-secret.yaml).
-  # Used only by the legacy self-hosted MinIO Deployment, which was retired
-  # 2026-05-20 in favour of DigitalOcean Spaces. Operators MUST replace.
-  MINIO_ROOT_USER: "CHANGE_ME"
-  MINIO_ROOT_PASSWORD: "CHANGE_ME"
+  # Object-storage backend (canonical 2026-05-20 onward — self-hosted MinIO retired).
+  # OBJECT_STORE_BACKEND selects the impl: do-spaces (prod), r2, or s3.
+  # The remaining keys are backend-specific credentials. See
+  # common/storageprovider/factory.go.
+  OBJECT_STORE_BACKEND: "do-spaces"
+  OBJECT_STORE_ENDPOINT: "CHANGE_ME"
+  OBJECT_STORE_PUBLIC_URL: "CHANGE_ME"
+  OBJECT_STORE_REGION: "CHANGE_ME"
+  OBJECT_STORE_ACCESS_KEY: "CHANGE_ME"
+  OBJECT_STORE_SECRET_KEY: "CHANGE_ME"
+  OBJECT_STORE_BUCKET: "instant-shared"
+  OBJECT_STORE_SECURE: "true"
   # Comma-separated list of admin user emails (case-insensitive). Empty
   # / unset → admin endpoints reject every caller (closed-by-default).
   # Read by api/internal/middleware/admin.go on every request, no app