Skip to content

feat: cross ns policy attachment #8676

Merged
zhaohuabing merged 46 commits intoenvoyproxy:mainfrom
zhaohuabing:cross-ns-attachment
Apr 30, 2026
Merged

feat: cross ns policy attachment #8676
zhaohuabing merged 46 commits intoenvoyproxy:mainfrom
zhaohuabing:cross-ns-attachment

Conversation

@zhaohuabing
Copy link
Copy Markdown
Member

@zhaohuabing zhaohuabing commented Apr 5, 2026

Picking up #5145 and continuing the work from here. Thanks, @guydc, for the earlier work on this.

This PR adds cross-namespace policy attachment support for ClientTrafficPolicy, BackendTrafficPolicy, EnvoyExtensionPolicy, and SecurityPolicy.

Extension Server policies aren’t included yet since the issue is mostly around CTP, BTP, and EEP, and EG doesn’t have proper tests for Extension Server policies yet.

We can add support for Extension Server policies later in a follow-up PR if needed.

Implements: #5145

guydc added 4 commits February 2, 2025 11:16
Signed-off-by: Guy Daich <guy.daich@sap.com>
Signed-off-by: Guy Daich <guy.daich@sap.com>
Signed-off-by: Guy Daich <guy.daich@sap.com>
@zhaohuabing zhaohuabing requested a review from a team as a code owner April 5, 2026 14:17
@zhaohuabing zhaohuabing marked this pull request as draft April 5, 2026 14:18
@netlify
Copy link
Copy Markdown

netlify Bot commented Apr 5, 2026

Deploy Preview for cerulean-figolla-1f9435 ready!

Name Link
🔨 Latest commit 33fd9ab
🔍 Latest deploy log https://app.netlify.com/projects/cerulean-figolla-1f9435/deploys/69f1eada03a842000804ba98
😎 Deploy Preview https://deploy-preview-8676--cerulean-figolla-1f9435.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 5, 2026

Codecov Report

❌ Patch coverage is 87.07753% with 65 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.65%. Comparing base (d48292b) to head (33fd9ab).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
internal/gatewayapi/helpers.go 88.83% 18 Missing and 5 partials ⚠️
internal/gatewayapi/envoyextensionpolicy.go 68.88% 12 Missing and 2 partials ⚠️
internal/gatewayapi/securitypolicy.go 74.54% 12 Missing and 2 partials ⚠️
internal/gatewayapi/clienttrafficpolicy.go 75.00% 6 Missing and 1 partial ⚠️
internal/provider/kubernetes/controller.go 91.95% 5 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8676      +/-   ##
==========================================
+ Coverage   74.59%   74.65%   +0.05%     
==========================================
  Files         250      250              
  Lines       39751    40164     +413     
==========================================
+ Hits        29654    29983     +329     
- Misses       8048     8118      +70     
- Partials     2049     2063      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
// Selector selects namespaces when From is set to Selector.
//
// +optional
Selector *metav1.LabelSelector `json:"selector,omitempty"`
Copy link
Copy Markdown
Member Author

@zhaohuabing zhaohuabing Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m a bit hesitant to add Selector support, as we don’t have a similar flexible ReferenceGrant API for it.

Or EG probably shouldn't require ReferenceGrant for cross-ns policy attachment at all. In the Gateway API semantics, ReferenceGrant is used for cross-namespace references to foreign objects, not for attachment relationships.

There is already precedent for this distinction in the Gateway API:

  • cross-namespace Route-to-Gateway attachment is governed by allowedRoutes, no ReferenceGrant required.
  • cross-namespace ListenerSet-to-Gateway attachment is governed by allowedListeners, no ReferenceGrant required.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern with allowing this without ReferenceGrant is that app teams could affect cluster-wide resources without the infra team’s knowledge or consent.

In Gateway API model, the affected resource (e.g. Gateway managed by the infra team) has the authority to control who can attach to it. In other words, consent comes from the target side.

By contrast, policy resources can be created by app teams, and a policy in one team’s namespace could target resources owned by other teams, without giving those teams any way to opt in or opt out.

That said, from the infra team’s perspective, this feature is very useful because a single policy can apply across resources in all namespaces.

So I think it's important to be clear about who should have that authority.
Would it make sense to add a toggle in EnvoyGateway to enable or disable this feature, or to allow it only for specific namespaces?

Copy link
Copy Markdown
Member Author

@zhaohuabing zhaohuabing Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @kkk777-7 That's a good point! I agree the key question is who should have the authority to enable cross-namespace attachment.

One option is to require ReferenceGrant, but this might not be the right semantic fit, since Gateway API generally uses ReferenceGrant for cross-namespace object references rather than attachment-style relationships. A more Gateway API-native model would be target-side consent, for example something like an AllowedPolicies field on Gateways or Routes, though that would likely require upstream API work.

We've got a few options here:

  1. Use ReferenceGrant to explicitly authorize cross-namespace policy attachment.
  2. Disable cross-namespace policy attachment by default, and let operators enable it only for specific namespaces.
type CrossNamespacePolicyAttachment struct {
	Enabled bool `json:"enabled,omitempty"`
	AllowedPolicyNamespaces []string `json:"allowedPolicyNamespaces,omitempty"`
}
  1. Pursue a more target-side consent model, though that likely needs upstream API support. This is not feasible if we want to land it in EG in the near future as it requires some time to discuss this upstream.

Option 1 provides more granular control, while option 2 is simpler and would allow a smoother transition to a more Gateway API-native target-side consent model, such as an AllowedPolicies attachment mechanism in the future.

If we go with option 1, we should also start with a simpler namespace allowlist first rather than a fully flexible Selector, since the latter makes the authorization and UX harder to reason about.

type TargetSelectorNamespaces struct {
	// +kubebuilder:validation:Enum=Same;All;Names
	From TargetNamespaceFrom `json:"from"`

	MatchNames []string `json:"matchNames,omitempty"`
}

In the meantime, we could pursue a more target-side consent model in the upstream Gateway API.

cc @envoyproxy/gateway-maintainers could you please weigh in on the options?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @zhaohuabing :)
I agree that option3 needs some times to discuss this upstream, so prefer option2.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for option1, since there isnt a direct parent child relationship for policy and target, similar to gateway-route or gateway-listenerSet
take a case where app team in namespace A tries to influence routes in namespace B using a policy A, we dont want this to be the default, and want team B to opt in

I do think we need to think of a way to opt out of explicit ReferenceGrants, to reduce the number of resources needed to author intent, that can be a future follow up

Copy link
Copy Markdown
Member Author

@zhaohuabing zhaohuabing Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Raised kubernetes-sigs/gateway-api#4799 in Gateway API to explore an “allow-from-target” approach, similar to Gateway.allowedListeners.

Comment thread api/v1alpha1/policy_helpers.go
@zhaohuabing zhaohuabing marked this pull request as ready for review April 13, 2026 00:32
@zhaohuabing zhaohuabing marked this pull request as draft April 14, 2026 07:37
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
@zhaohuabing zhaohuabing force-pushed the cross-ns-attachment branch from 479741a to 7d277a7 Compare April 15, 2026 10:42
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
@zhaohuabing zhaohuabing marked this pull request as ready for review April 21, 2026 04:13
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e507c223d1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/gatewayapi/extensionserverpolicy.go Outdated
@zhaohuabing zhaohuabing marked this pull request as draft April 21, 2026 04:33
@zhaohuabing zhaohuabing requested a review from kkk777-7 April 25, 2026 01:26
kkk777-7
kkk777-7 previously approved these changes Apr 25, 2026
@kkk777-7
Copy link
Copy Markdown
Member

/retest

Comment thread site/content/en/v1.4/api/extension_types.md Outdated
@zhaohuabing zhaohuabing requested a review from jukie April 25, 2026 05:33
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
@zhaohuabing
Copy link
Copy Markdown
Member Author

/retest

kind: Gateway
matchLabels:
policy: selected
namespaces:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this is the selector case, should we skip targets with no refgrants or should we flag it as partially accepted in the status ?

Copy link
Copy Markdown
Member Author

@zhaohuabing zhaohuabing Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If two routes under the same Gateway are selected, and one is permitted by a ReferenceGrant while another is not, the status for that Gateway is set to Accepted=False in this PR:

status:
  ancestors:
  - ancestorRef:
      group: gateway.networking.k8s.io
      kind: Gateway
      name: mixed-gateway
      namespace: gateway-ns
      sectionName: http
    conditions:
    - lastTransitionTime: null
      message: Target HTTPRoute policy-target-b/denied-route is not permitted by
        any ReferenceGrant.
      reason: RefNotPermitted
      status: "False"
      type: Accepted

And the targets without RefrenceGrant are skipped during the policy stranslation.

We probably shouldn't silently skip targets without ReferenceGrants though, since users would otherwise have no way to understand why the policy is not taking effect on some routes.

A “partially accepted” status would be better, but right now the Accepted condition only supports True or False, and adding an addtional ParitallyAccepted condition for this feels like too much.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

  • If an ancestor has at least one effective target: Accepted=True,

  • if some selector-matched targets under that same ancestor were skipped due to missing ReferenceGrant: add Warning=True, reason RefNotPermitted.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
@zhaohuabing
Copy link
Copy Markdown
Member Author

@codex review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33fd9ab20c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +321 to +328
if len(denied) > 0 {
policy, found := handledPolicies[policyName]
if !found {
policy = policyCopies[i]
handledPolicies[policyName] = policy
res = append(res, policy)
}
setPolicyTargetRefNotPermittedStatus(&policy.Status, denied, t.GatewayControllerName, policy.Generation)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Defer RefNotPermitted status until all targets are processed

Setting RefNotPermitted immediately after route selector evaluation can leave an ancestor permanently Accepted=False even when a later gateway target for the same ancestor is valid. In this flow, route denials are recorded first, then gateway processing runs afterward; because SetAcceptedForPolicyAncestor only sets Accepted when it is unset, the later successful attachment cannot flip the ancestor back to accepted. This produces incorrect status for mixed-grant policies (e.g., route denied but gateway allowed under the same parent).

Useful? React with 👍 / 👎.

Comment on lines +1074 to +1077
"Target %s %s/%s is not permitted by any ReferenceGrant.",
deniedMatch.Target.GetObjectKind().GroupVersionKind().Kind,
deniedMatch.Target.GetNamespace(),
deniedMatch.Target.GetName(),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid exposing cross-namespace target identities in denial status

This message includes the concrete kind/namespace/name of denied cross-namespace targets discovered via selectors. That leaks object existence/details across namespace boundaries to a policy author without a matching ReferenceGrant, which is exactly the case being denied. For multi-tenant clusters, this becomes an information disclosure vector whenever broad selectors (for example namespaces.from: All) are used.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

@arkodg arkodg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks
Approving so this makes it into rc and the the current limitations can hopefully be addressed before the release

@arkodg arkodg requested review from a team April 29, 2026 15:03
Copy link
Copy Markdown
Contributor

@guydc guydc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks, this is super useful.

Some docs clarifying RefGrant requirements would be great in a follow-up PR.

@zhaohuabing
Copy link
Copy Markdown
Member Author

/retest

@zhaohuabing zhaohuabing merged commit bef5ed1 into envoyproxy:main Apr 30, 2026
89 of 94 checks passed
@zhaohuabing zhaohuabing deleted the cross-ns-attachment branch April 30, 2026 01:32
kkk777-7 pushed a commit to kkk777-7/gateway that referenced this pull request Apr 30, 2026
* api: cross-namespace policy target selectors

Signed-off-by: Guy Daich <guy.daich@sap.com>

* use gw-api style for namespace selector

Signed-off-by: Guy Daich <guy.daich@sap.com>

* fix json tag

Signed-off-by: Guy Daich <guy.daich@sap.com>

* update API

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* update API

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* implmentation

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add e2e tests for cross ns policy attachment

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* update test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix lint

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* update test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add referencegrants to the provider

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* simplify code

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* address comments

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* simplify code

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* simplify code

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* rename

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix race

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* remove unnecessary change

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* simplify code

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* rename

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* refactor

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* address comments

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* address comments

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add referencegrants for extension server policies

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* Revert "add referencegrants for extension server policies"

This reverts commit a0fbe68.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* remove cross-ns policy attachment for extension server

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix gen

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix lint

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* address comments

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* remove unnecessary change

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix cel validation

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add warning for partially accepted targets

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Guy Daich <guy.daich@sap.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Co-authored-by: Guy Daich <guy.daich@sap.com>
skos-ninja pushed a commit to skos-ninja/envoy-gateway that referenced this pull request May 1, 2026
* api: cross-namespace policy target selectors

Signed-off-by: Guy Daich <guy.daich@sap.com>

* use gw-api style for namespace selector

Signed-off-by: Guy Daich <guy.daich@sap.com>

* fix json tag

Signed-off-by: Guy Daich <guy.daich@sap.com>

* update API

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* update API

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* implmentation

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add e2e tests for cross ns policy attachment

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* update test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix lint

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* update test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add referencegrants to the provider

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* simplify code

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* address comments

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* simplify code

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* simplify code

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* rename

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix race

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* remove unnecessary change

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* simplify code

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* rename

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* refactor

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* address comments

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* address comments

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add referencegrants for extension server policies

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* Revert "add referencegrants for extension server policies"

This reverts commit a0fbe68.

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* remove cross-ns policy attachment for extension server

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix gen

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix lint

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix test

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* address comments

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* remove unnecessary change

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* fix cel validation

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

* add warning for partially accepted targets

Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>

---------

Signed-off-by: Guy Daich <guy.daich@sap.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Co-authored-by: Guy Daich <guy.daich@sap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants