fix(rbac): grant ci-deployer watch on deployments (unblock rollout status)#58
Merged
Merged
Conversation
…atus)
The api/worker/provisioner Deploy workflows authenticate as
system:serviceaccount:kube-system:ci-deployer (via KUBECONFIG_B64) and run
`kubectl set image` + `kubectl rollout status`. The live ci-deployer-rollout
ClusterRole granted get/list/patch/update on deployments but NOT `watch`, so
`set image` succeeded while `rollout status` (which WATCHes the Deployment)
was denied in instant-infra (worker/provisioner) and instant (api):
User "system:serviceaccount:kube-system:ci-deployer" cannot watch
resource "deployments" in API group "apps" in the namespace "instant-infra"
(worker run 27053035376). The image still landed; only the status step red.
The ClusterRole + ClusterRoleBinding had been applied by hand on 2026-05-15
and lived only in the cluster — never in this repo. This adds the canonical
manifest (k8s/ci-deployer-rbac.yaml) capturing the live shape and ADDING the
missing `watch` on deployments (+ list/watch on deployments/status that
rollout status also reads). Least-privilege: rollout/read of existing
Deployments only; no create/delete, no secret/namespace access.
Verified live (impersonating the CI SA):
kubectl rollout status deployment/{instant-worker,instant-api,instant-provisioner}
--as=system:serviceaccount:kube-system:ci-deployer → all "successfully rolled out".
Applied to do-nyc3-instant-prod (additive RBAC, no-op diff except the new
verbs). infra has no auto-apply (rule 15); manifest now matches live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The api/worker/provisioner Deploy workflows authenticate to prod as
system:serviceaccount:kube-system:ci-deployer(via theKUBECONFIG_B64repo secret) and run:The live
ci-deployer-rolloutClusterRole grantedget,list,patch,updateondeploymentsbut notwatch.kubectl set image(patch/update) succeeded, butkubectl rollout statusperforms a WATCH and was denied:(worker run
27053035376.) The image still landed — only the rollout-status step red, so deploys half-succeeded.Root cause
The
ci-deployerSA +ci-deployer-rolloutClusterRole + binding were applied by hand on 2026-05-15 and lived only in the cluster — there was no manifest in this repo, so the missing-watchdrift was invisible and un-reviewable.Fix
Adds the canonical manifest
k8s/ci-deployer-rbac.yamlcapturing the live shape and adding the missingwatchondeployments(+list/watchondeployments/status, whichrollout statusalso reads). Least-privilege: rollout/read of existing Deployments only — no create/delete, no secret/namespace/configmap access. Applies to bothinstant-infra(worker/provisioner) andinstant(api), since it's a cluster-scoped binding driving all three pipelines.Verified live (impersonating the exact CI SA)
Worker is already running the latest master SHA (
master-eb427f4) — the prior failure only red the status step, theset imagehad landed.Operator note
infra has no auto-apply (CLAUDE.md rule 15). I already applied this live to
do-nyc3-instant-prod(additive RBAC, no-op diff except the new verbs) so CI deploys unblock immediately. The manifest now matches live; a futurekubectl apply -f k8s/ci-deployer-rbac.yamlis a clean no-op.🤖 Generated with Claude Code