Skip to content

envoy: server-side apply, skip unchanged applies, and ConfigMap checksum for pod rollouts#246

Open
HarshithaMS005 wants to merge 2 commits into
kubernetes-sigs:mainfrom
HarshithaMS005:fix/envoy-proxy-infra-reconcile
Open

envoy: server-side apply, skip unchanged applies, and ConfigMap checksum for pod rollouts#246
HarshithaMS005 wants to merge 2 commits into
kubernetes-sigs:mainfrom
HarshithaMS005:fix/envoy-proxy-infra-reconcile

Conversation

@HarshithaMS005
Copy link
Copy Markdown
Contributor

@HarshithaMS005 HarshithaMS005 commented Apr 20, 2026

What type of PR is this?
/kind feature

**What this PR does **:
The Envoy ResourceManager used to only create the proxy ServiceAccount, ConfigMap, Deployment, and Service when they were missing. This PR reconciles those objects on every sync so changes (Envoy image, bootstrap/SDS content, manual edits) converge without deleting the Gateway.
Infra updates use server-side apply (docs) with a single field manager and Force, instead of field-by-field comparison in reconcile.go. reconcile.go now only holds configMapDataChecksum for the pod-template annotation. Because updating a mounted ConfigMap alone does not restart pods, the Deployment pod template keeps a checksum annotation derived from rendered ConfigMap data so bootstrap/SDS changes still trigger a rollout. The controller does not wait for the Envoy Deployment Available condition, so it does not block reconciliation of other Gateways (#198).
Proxy resources use the standard Gateway label gateway.networking.k8s.io/gateway-name (GEP-1762).

Which issue(s) this PR fixes:
Fixes #238

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Apr 20, 2026
@netlify
Copy link
Copy Markdown

netlify Bot commented Apr 20, 2026

Deploy Preview for kube-agentic-networking ready!

Name Link
🔨 Latest commit 2847436
🔍 Latest deploy log https://app.netlify.com/projects/kube-agentic-networking/deploys/6a0ef60d06fb0e0008568641
😎 Deploy Preview https://deploy-preview-246--kube-agentic-networking.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Apr 20, 2026
@k8s-ci-robot k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Apr 20, 2026
@HarshithaMS005 HarshithaMS005 force-pushed the fix/envoy-proxy-infra-reconcile branch from 492fdad to 717ab2f Compare April 20, 2026 19:07
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 22, 2026
@HarshithaMS005 HarshithaMS005 force-pushed the fix/envoy-proxy-infra-reconcile branch from 717ab2f to 142f746 Compare April 24, 2026 09:48
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 24, 2026
Comment thread pkg/infra/envoy/reconcile.go Outdated
Comment thread pkg/infra/envoy/reconcile_test.go Outdated
Comment thread pkg/infra/envoy/proxy.go
Comment thread pkg/infra/envoy/proxy.go Outdated
@HarshithaMS005 HarshithaMS005 force-pushed the fix/envoy-proxy-infra-reconcile branch 2 times, most recently from b3d3d1b to 0fd05f0 Compare April 29, 2026 17:25
@HarshithaMS005
Copy link
Copy Markdown
Contributor Author

/test pull-kube-agentic-networking-verify

@bowei
Copy link
Copy Markdown
Contributor

bowei commented Apr 29, 2026

I think since we are adding this new server-side apply logic, it would be good to send a PR for one resource for review to avoid having a quite large review. Once that review is done, we can have a follow up for the rest of the resources as they will be boilerplate in a single follow-up PR.

In addition, it would be good to discuss explicitly which fields you chose to be managed vs unmanaged. There are quite a lot of choices here and it's not clear that the choices are obvious?

Finally, the checksum on the ConfigMap. I would like this separate out into its own PR.

Comment thread pkg/infra/envoy/proxy.go
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 4, 2026
@HarshithaMS005 HarshithaMS005 force-pushed the fix/envoy-proxy-infra-reconcile branch from 0fd05f0 to d7844d0 Compare May 11, 2026 09:16
@HarshithaMS005 HarshithaMS005 changed the title fix(envoy): reconcile proxy infra(deploy, CM, SA, svc) envoy: skip redundant ServiceAccount server-side apply when unchanged May 11, 2026
@HarshithaMS005 HarshithaMS005 force-pushed the fix/envoy-proxy-infra-reconcile branch 2 times, most recently from 4437b9a to 5f53341 Compare May 11, 2026 10:30
@HarshithaMS005 HarshithaMS005 changed the title envoy: skip redundant ServiceAccount server-side apply when unchanged envoy: server-side apply, skip unchanged applies, and ConfigMap checksum for pod rollouts May 11, 2026
@HarshithaMS005 HarshithaMS005 force-pushed the fix/envoy-proxy-infra-reconcile branch from 5f53341 to b4a6c11 Compare May 11, 2026 11:41
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 11, 2026
@HarshithaMS005
Copy link
Copy Markdown
Contributor Author

HarshithaMS005 commented May 11, 2026

@bowei , Thanks for the review — agree on all three points.

  • In this branch i ended up with stacked commits (ServiceAccount skip first, then ConfigMap / Deployment / Service skip) rather than separate GitHub PRs; happy to mirror that in the PR description or split into follow-up PRs if you’d rather merge SA-only first.
  • Managed vs unmanaged: Documented in serviceAccountDesiredMatchesExisting and in proxy_desired_match.go for ConfigMap, Deployment, and Service (configMapDesiredMatchesExisting, deploymentDesiredMatchesExisting, serviceDesiredMatchesExisting), aligned with what each serviceAccountApply/configMapApply/deploymentApply/
    serviceApply sends on SSA.
  • Checksum/rollout: Understood on isolating that story. Today it ships with the SSA infra work on this branch; we can either call that out explicitly in the description/release notes, or I can follow up with a small PR that only moves/splits checksum-related files/commits if you want a distinct review for “hash ConfigMap data → pod rollout.” Say what you prefer.

@haiyanmeng
Copy link
Copy Markdown
Contributor

/lgtm

/hold for @LiorLieberman to approve

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 11, 2026
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 11, 2026
Comment thread pkg/infra/envoy/proxy.go Outdated
}

// ensureSA ensures that the ServiceAccount for the Envoy proxy exists.
// ensureSA applies the ServiceAccount for the Envoy proxy (server-side apply).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some resources we should just consider "owned completely" by the controller. In these cases, it's not out of the question to do the simple thing and overwrite all fields. Is Service Account one of these cases?

I'm trying to think through which controllers/users will "collaborate" on the definition of the service account.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes — the Envoy proxy ServiceAccount is fully owned by this controller. It has a deterministic hashed name, a Gateway ownerReference, is only referenced by the proxy Deployment, and is deleted with the Gateway. We don’t expect users or other controllers to co-manage this object directly; the only supported customization path is Gateway.spec.infrastructure labels/annotations propagated to proxy infra.
We still shouldn’t do a full-object replace that clears .secrets (populated by the token controllers). SSA via serviceAccountApply only sets labels, annotations, and ownerReferences with our field manager + Force, which is the simple “overwrite what we own” model without touching apiserver-owned fields.

I removed the GET/compare skip-apply logic for the ServiceAccount — that optimization made sense for shared/complex resources but was unnecessary here. ensureSA now always Applies. ConfigMap/Deployment/Service still use skip-apply where the field surface is larger.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 16, 2026
…ollouts.

ServiceAccount: GET before Apply; if labels, annotations, and
ownerReferences match the rendered desired object, skip Apply to
avoid redundant writes.
Document managed vs unmanaged fields in godoc on
serviceAccountDesiredMatchesExisting for review clarity.
Server-side apply for Envoy infra (apply.go) and ServiceAccount get/compare/skip.
ConfigMap digest on Deployment pod template (annotation + reconcile helper + tests)
so bootstrap/SDS ConfigMap updates roll pods.

ConfigMap/Deployment/Service skip-apply comes in the following commit.

Signed-off-by: Harshitha MS <harshitha.ms@ibm.com>
GET before Apply for bootstrap ConfigMap, Envoy Deployment, and proxy
Service; skip Apply when desired state matches live objects per
configMapApply / deploymentApply / serviceApply.

Signed-off-by: Harshitha MS <harshitha.ms@ibm.com>
@HarshithaMS005 HarshithaMS005 force-pushed the fix/envoy-proxy-infra-reconcile branch from b4a6c11 to 2847436 Compare May 21, 2026 12:09
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 21, 2026
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

New changes are detected. LGTM label has been removed.

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 21, 2026
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: HarshithaMS005
Once this PR has been reviewed and has the lgtm label, please ask for approval from haiyanmeng. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. kind/feature Categorizes issue or PR as related to a new feature. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement] Reconcile Envoy Proxy Infrastructure Changes

4 participants