fix: make ListenerSet and GRPCRoute watches optional - #9583
Conversation
The Kubernetes provider watches, indexes and Lists ListenerSet and GRPCRoute unconditionally. When either CRD is absent, the manager's cache sync fails and Envoy Gateway crash-loops at startup with: error watching resources: no matches for kind "ListenerSet" in version "gateway.networking.k8s.io/v1" Both kinds are in the Gateway API standard channel, but some managed Kubernetes offerings install only a curated subset of it and don't let users add the missing CRDs. GKE's managed gateway-api-crds addon ships Gateway API v1.5 without listenersets and grpcroutes, because the GKE Gateway controller doesn't implement them; installing them out-of-band means a second owner for addon-managed CRDs. Guard both watches, their indexers, the reconcile-time processing and the GRPCRoute predicate Lists behind a CRD existence check, restoring the pre-envoyproxy#8365 behavior for these two kinds only. TLSRoute and BackendTLSPolicy stay unconditional. When the CRDs are present, behavior is unchanged; when one is absent, the controller starts and simply doesn't serve that kind instead of taking down the whole controller. The offline (file) provider keeps assuming all CRDs are present. Existence is probed once at startup, so a CRD installed later is not watched until the controller restarts, the same trade-off already accepted for EnvoyProxy, TCPRoute and UDPRoute. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
✅ Deploy Preview for cerulean-figolla-1f9435 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
@codex review |
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Codecov Report❌ Patch coverage is
❌ Your patch check has failed because the patch coverage (52.17%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #9583 +/- ##
==========================================
- Coverage 75.65% 75.64% -0.01%
==========================================
Files 253 254 +1
Lines 41879 41942 +63
==========================================
+ Hits 31684 31729 +45
- Misses 8064 8072 +8
- Partials 2131 2141 +10 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
TLSRoute graduated to the Gateway API standard channel in v1.5, so it is subject to the same problem as ListenerSet and GRPCRoute: a cluster may run a bundle that predates the graduation, or a managed offering may ship only a curated subset of the standard channel, and the unconditional watch then fails the manager's cache sync and crash-loops Envoy Gateway at startup. Guard the TLSRoute watch, its indexers, the reconcile-time processing and the TLSRoute predicate List behind a CRD existence check, matching what the previous commit does for ListenerSet and GRPCRoute. Also correct the rationale comment: TCPRoute and UDPRoute are standard channel as of Gateway API v1.6, they are not experimental-only. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
|
/retest |
|
LGTM, thanks! |
|
@zhaohuabing Thanks! |
envoyproxy#9583 landed the ListenerSet, GRPCRoute and TLSRoute guards this PR originally proposed. What remains unguarded is BackendTLSPolicy, which is the next kind the controller fails on for the same reason, so this PR is now scoped to that alone.
* fix: make ListenerSet and GRPCRoute watches optional (envoyproxy#8991) The Kubernetes provider watches, indexes and Lists ListenerSet and GRPCRoute unconditionally. When either CRD is absent, the manager's cache sync fails and Envoy Gateway crash-loops at startup with: error watching resources: no matches for kind "ListenerSet" in version "gateway.networking.k8s.io/v1" Both kinds are in the Gateway API standard channel, but some managed Kubernetes offerings install only a curated subset of it and don't let users add the missing CRDs. GKE's managed gateway-api-crds addon ships Gateway API v1.5 without listenersets and grpcroutes, because the GKE Gateway controller doesn't implement them; installing them out-of-band means a second owner for addon-managed CRDs. Guard both watches, their indexers, the reconcile-time processing and the GRPCRoute predicate Lists behind a CRD existence check, restoring the pre-envoyproxy#8365 behavior for these two kinds only. TLSRoute and BackendTLSPolicy stay unconditional. When the CRDs are present, behavior is unchanged; when one is absent, the controller starts and simply doesn't serve that kind instead of taking down the whole controller. The offline (file) provider keeps assuming all CRDs are present. Existence is probed once at startup, so a CRD installed later is not watched until the controller restarts, the same trade-off already accepted for EnvoyProxy, TCPRoute and UDPRoute. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix: make the TLSRoute watch optional too TLSRoute graduated to the Gateway API standard channel in v1.5, so it is subject to the same problem as ListenerSet and GRPCRoute: a cluster may run a bundle that predates the graduation, or a managed offering may ship only a curated subset of the standard channel, and the unconditional watch then fails the manager's cache sync and crash-loops Envoy Gateway at startup. Guard the TLSRoute watch, its indexers, the reconcile-time processing and the TLSRoute predicate List behind a CRD existence check, matching what the previous commit does for ListenerSet and GRPCRoute. Also correct the rationale comment: TCPRoute and UDPRoute are standard channel as of Gateway API v1.6, they are not experimental-only. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
What this PR does / why we need it:
The Kubernetes provider watches, indexes and Lists
ListenerSetandGRPCRouteunconditionally. When either CRD is absent, the manager's cache sync fails and Envoy Gateway crash-loops at startup:Both kinds are in the Gateway API standard channel, but some managed Kubernetes offerings install only a curated subset of it and don't let users add the missing CRDs. As reported in #9517, GKE's managed
gateway-api-crdsaddon (1.5.0-gke.3503) ships Gateway API v1.5 withoutlistenersetsandgrpcroutes, because the GKE Gateway controller doesn't implement them; installing them out-of-band means a second owner for addon-lifecycle-managed CRDs and a guaranteed conflict later. The absent kind can't be used on those clusters anyway, so its absence should disable the feature, not take down the whole controller.This is a rescope of #9517 to the scope agreed in its review thread; credit to @electricjesus for the original patch and to @dragon3 for the GKE data points.