-
Notifications
You must be signed in to change notification settings - Fork 815
feat: add MergeBackends option for backend cluster deduplication #9477
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
muwaqar-cflt
wants to merge
127
commits into
envoyproxy:main
Choose a base branch
from
muwaqar:backend-cluster-dedup-rebase
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
127 commits
Select commit
Hold shift + click to select a range
f24d27b
ir: add BackendCluster and BackendClusterRef types
muwaqar b0a3b87
gatewayapi: populate BackendClusterRefs alongside Settings at all con…
muwaqar 4710fe3
ir: move Has* methods to BackendCluster, update NeedsClusterPerSetting
muwaqar f741b67
xds: switch cluster building to use BackendCluster
muwaqar 4a702d4
ir: add ToBackendWeights to BackendCluster, enforce multi-ref invaria…
muwaqar af3bdbc
xds: switch route action builder to use BackendClusters
muwaqar 990a7ac
xds: switch ext service and auth/logging readers to use BackendClusters
muwaqar 616668e
gatewayapi: switch remaining Settings readers to use BackendClusters
muwaqar afc6b85
ir/gatewayapi: eliminate remaining Settings-only usage, reuse Backend…
muwaqar 3a68e32
ir: remove unused Xds.Backends field
muwaqar c199136
gatewayapi: dedupe route BackendClusters across all route types
muwaqar a169a8b
ir: add Xds.Backends registry and BackendClusterRef.Name (additive)
muwaqar 2ffdc47
gatewayapi: register route-kind BackendClusters into Xds.Backends
muwaqar 1de1f2d
gatewayapi: register remaining non-route BackendClusters into Xds.Bac…
muwaqar 6785366
gatewayapi: resolve OIDC provider TLS from constructed cluster, not G…
muwaqar 2a1f682
xds/translator: add name-keyed BackendCluster registry index
muwaqar b3b9aa2
xds/translator: resolve BackendClusters via registry index instead of…
muwaqar 242bd41
ir: remove BackendCluster-embedding methods now that consumers resolv…
muwaqar 3ee2979
ir: remove BackendClusterRef.Backend — routes now hold only a name in…
muwaqar 0a7f98c
gatewayapi: drop always-nil weight/filters params from registerBacken…
muwaqar 9d797b0
xds/translator: fix ext-service JWT/OIDC cluster guard defeated by Ge…
muwaqar f99b63a
gatewayapi: exclude route-level BackendTrafficPolicy cluster settings…
muwaqar d3c35df
gatewayapi: never merge dynamic-resolver or custom backends in MergeB…
muwaqar 61bbc25
gatewayapi: exclude SessionPersistence multi-backend HTTP rules from …
muwaqar f3a90a3
api: expose MergeBackends via EnvoyProxy CRD
muwaqar 79d9479
gatewayapi: scope route-scoped BackendClusterKey by gateway, verify X…
muwaqar 0fc6adb
gatewayapi: remove stale Backends ignore from TestTranslateWithExtens…
muwaqar 5bb98ec
ir: denormalize IsDynamicResolver and ResolvedMetricSink.Backends
muwaqar 4f64d96
gatewayapi: regenerate golden files for IsDynamicResolver/Backends fi…
muwaqar 8300ad5
ir: enable BackendClusterRefs json/yaml serialization
muwaqar 9d224b5
xds/translator: check BackendClusterRefs instead of Settings in jwt/o…
muwaqar 4191087
gatewayapi: dedupe BackendClusterRefs per rule, stop setting removed …
muwaqar 9bce423
ir: remove RouteDestination.Settings, BackendClusterRefs is now the s…
muwaqar 38fa146
gatewayapi: remove dead destSettings accumulation in TLS/UDP/TCP rout…
muwaqar 7da7da7
gatewayapi,egctl: remove stale BackendClusterRefs comparison ignores
muwaqar 88e4f0a
gatewayapi: fix stale yaml:"-" tag comment in merge_backend_test.go
muwaqar 918206d
gatewayapi: scope route-level BackendClusterKey by parentRef listener
muwaqar 95f2a48
gatewayapi: give malformed duplicate-named httproute test fixtures un…
muwaqar a930524
gatewayapi: move merge-backend tests to their function's test file
muwaqar 1bc3933
testdata: regenerate golden files for BackendClusterRefs migration
muwaqar a25c760
testdata: migrate new post-rebase fixtures, regenerate golden files
muwaqar 869a999
gatewayapi,ir,xds: trim over-detailed comments from the backend-clust…
muwaqar 6329669
xds/translator: move cluster-per-setting/weight helpers out of backen…
muwaqar 67773fc
docs,helm: regenerate API reference and CRD test fixtures for MergeBa…
muwaqar 4b8a847
gatewayapi,infra,test: fix lint issues (hugeParam, unconvert, gofumpt…
muwaqar 1942af3
docs: add release note for MergeBackends (#9477)
muwaqar bdfda86
gatewayapi: scope merged BackendClusterKey by resolved protocol
muwaqar c7bfcbf
gatewayapi: sync merged setting name and add cross-protocol test cove…
muwaqar c1e7fc6
gatewayapi: exclude ConsistentHash-routed multi-backend rules from Me…
muwaqar c71751a
gatewayapi: respect BTP precedence overrides in BTPLoadBalancerIndex
muwaqar 52c5a00
gatewayapi: consolidate merge-eligibility and naming logic into shoul…
muwaqar 8e80c16
gatewayapi: exclude fallback-backend rules from MergeBackends
muwaqar afaac2b
gatewayapi: extract per-rule mergeIncompatible computation into a sha…
muwaqar d64d1cf
gatewayapi: exclude CredentialInjection-filtered backendRefs from Mer…
muwaqar 695ec13
gatewayapi: move merge-check functions together and clarify shouldMer…
muwaqar 562059e
ci: add merge-backends conformance profile
muwaqar d27c556
gatewayapi: fix golangci-lint findings in route_test.go
muwaqar c98a455
gatewayapi: exclude UseClientProtocol-configured routes from MergeBac…
muwaqar ce581b7
ir,gatewayapi,xds/translator: stop leaking an arbitrary route's hostn…
muwaqar 213c27f
xds/translator: reference the resolved cluster name in HTTP RouteActions
muwaqar e305e28
xds/translator,gatewayapi: fix TCP/UDP cluster names, guard TCP/UDP/T…
muwaqar d3ab553
gatewayapi,ir: exclude any per-backendRef Filters, not just Credentia…
muwaqar 845c966
gatewayapi: add golden fixture for RoutingType-divergence demerge
muwaqar 00c34da
ir,xds/translator: remove dead BackendWeights.Name field
muwaqar ae3b069
xds/translator,gatewayapi: add TLS/GRPC/weighted-path xds-layer cover…
muwaqar 0d93ee0
gatewayapi,ir,xds/translator: naming and comment cleanup from code re…
muwaqar c6acc30
xds/translator: migrate http-route-https-mtls-allow-expired-cert fixt…
muwaqar a99c8f7
gatewayapi: add unit test coverage for IsMergeBackendsEnabled
muwaqar c0357f0
xds/translator: stop leaking a route's stat name into a merged cluster
muwaqar 24095d3
gatewayapi: include AdmissionControl in cluster-scoped BTP field check
muwaqar fb31053
xds/translator: resolve weighted-route weight per-route for merged cl…
muwaqar 367ac61
gatewayapi: key BTP cluster-settings/load-balancer indexes by target …
muwaqar eb0e668
gatewayapi: honor a Gateway-level EnvoyProxy's MergeBackends override
muwaqar 4dbe1a7
test/e2e: add MergeBackends conformance test
muwaqar 6b57b37
ir: fix staticcheck ST1016 receiver name mismatch on BackendWeights
muwaqar 4660e84
api: drop MergeBackendsConfig.Mode from the initial MergeBackends API
muwaqar fce36fd
gatewayapi: explain why HasRouteLevelClusterSettings skips gateway level
muwaqar 7360d1a
xds/translator: dedupe client-certificate secrets by name
muwaqar be7dd73
gatewayapi: use the backend's own identity for a merged cluster's Met…
muwaqar fad3198
xds/translator: restore dropped route metadata on geoip-authorization…
muwaqar a91c822
gatewayapi: simplify BTPLoadBalancerIndex to gateway-level only
muwaqar 1942609
gatewayapi: extract multi-backendRef merge checks into a named method
muwaqar 1f84761
gatewayapi: key merged backend clusters by the owning Gateway's own i…
muwaqar 0cf3762
test/e2e: add MergeGatewaysBackends conformance test
muwaqar fff4d8f
testdata: regenerate golden fixtures for tests merged in from main
muwaqar 4ac0e28
ir: restore RouteDestination.Settings, make BackendCluster.Setting si…
muwaqar-cflt fb70d52
gatewayapi: bifurcate HTTP and GRPC rule processing on merge eligibility
muwaqar-cflt a195cf6
gatewayapi: track weight validity for every backendRef, not just non-…
muwaqar-cflt 162f6d8
gatewayapi: bifurcate TLS, UDP, and TCP rule processing on merge elig…
muwaqar-cflt 37ba0bc
gatewayapi: revert RequestMirror filter to pre-PR (mirror backends ne…
muwaqar-cflt 95bee06
gatewayapi: revert telemetry sink backends (access log, tracing, metr…
muwaqar-cflt ff5bf69
gatewayapi: revert ext-service and global-resources backends to pre-PR
muwaqar-cflt 96de30d
gatewayapi: simplify BackendClusterKey, lazily init BackendClusterMap…
muwaqar-cflt b60db70
xds/translator: replace unified backend resolution with merge-only re…
muwaqar-cflt e31b664
xds/translator: revert cluster building to pre-PR signatures, add add…
muwaqar-cflt 15c4b5b
xds/translator: revert weighted-route-action building to pre-PR, add …
muwaqar-cflt 38df198
gatewayapi,xds/translator: fix tests for the merge/non-merge bifurcation
muwaqar-cflt f1a2bce
xds/translator: revert incomplete Settings/BackendClusterRefs reverts…
muwaqar-cflt 07c2af3
xds/translator: restore applyTraffic for merged HTTP clusters, droppe…
muwaqar-cflt 13a4026
testdata: regenerate golden fixtures for the merge/non-merge bifurcation
muwaqar-cflt fbf08dd
gatewayapi,xds/translator: remove dead parameters and functions left …
muwaqar-cflt c84063d
gatewayapi: move BackendClusterMap above the BTP*Index fields in Tran…
muwaqar-cflt df01723
gatewayapi: skip building merge-eligibility BTP indexes when MergeBac…
muwaqar-cflt 97ecb6d
ir,gatewayapi: remove RouteDestination.IsDynamicResolver and BackendC…
muwaqar-cflt fb72f29
ir: restore RouteDestination.Settings validation test coverage
muwaqar-cflt 2783ae6
xds/types,ir: remove dead ResourceVersionTable.GetBackendClusters/Bac…
muwaqar-cflt 467b7c6
xds/runner: revert test fixtures to construct destinations via Settings
muwaqar-cflt 113524b
xds/translator: revert cosmetic style deviations in cluster.go/cluste…
muwaqar-cflt e283119
xds/translator: build merged backend clusters once via xdsIR.Backends…
muwaqar-cflt b9ada81
xds/translator: stop threading *Translator through functions that onl…
muwaqar-cflt 3f8ac16
xds/translator: minimize diff vs main and drop dead/unnecessary test …
muwaqar-cflt 68375fa
xds/translator: simplify buildXdsRouteAction and drop merge/non-merge…
muwaqar-cflt 389a094
gatewayapi: minor comment/style cleanup in non-route.go MergeBackends…
muwaqar-cflt e1f5eb0
gatewayapi: consolidate BTP index building, address PR perf/naming fe…
muwaqar-cflt 1bd087a
gatewayapi: route.go cleanup pass - drop backendClusterRefBuilder, fi…
muwaqar-cflt 06b3510
gatewayapi: collapse ResolvedBackendCluster, defer merge-eligibility …
muwaqar-cflt c43029c
gatewayapi: move misplaced udpRoute single-rule comment to its var block
muwaqar-cflt bf1372b
infrastructure: fix gocritic rangeValCopy lint finding in ConvertReso…
muwaqar-cflt a16123c
gatewayapi: fix TLSRoute multi-rule merge eligibility to consider all…
muwaqar-cflt be123c1
gatewayapi: add CTPClusterSettingsIndex tracking listener-level HTTP1…
muwaqar-cflt 8ff4e90
gatewayapi: build CTPClusterSettingsIndex once per Translate() call
muwaqar-cflt 355ef9d
testdata: add fixture reproducing the CTP HTTP1 MergeBackends leak
muwaqar-cflt 45feeae
gatewayapi: disqualify merging when a listener has a ClientTrafficPol…
muwaqar-cflt 10fe778
testdata: regenerate golden fixture confirming CTP HTTP1 demerge fix
muwaqar-cflt 2ac31c2
gatewayapi: add reflection-based exhaustiveness guard for CTP/BTP mer…
muwaqar-cflt 6722048
gatewayapi,test: remove MergeGateways+MergeBackends combination support
muwaqar-cflt 7c0a69e
api: enforce mergeGateways and mergeBackends are mutually exclusive v…
muwaqar-cflt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We may want to default this to
trueeventually. The fallback approach won't break current user-facing routing behaviors, and it helps with the Envoy memory issue.It may be too risky to turn this on by default in 1.9, as it does break EPPs and extension servers, but we could keep it opt-in for the first release and consider flipping the default in 1.10 after we have more coverage and user feedback.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
resolving because not seeing an action item for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's keep it unresolved for now, so other reviewer can see and discuss on this. cc @arkodg @guydc
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this flag needed ? mergeBackends: {} should imply we are turning on the feature
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, not necessarily. A bool flag is more explicit than
mergeBackends: {}, I'm fine with both.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i like an explicit flag,
mergeBackends: {}looks odd