Update NPEP-133 with Admin-tier restriction, DNS security guidance, and v1alpha2 API

The PR updates the details around domainNames
as discussed in KubeCon Atlanta:
https://docs.google.com/document/d/1AtWQy2fNa4qXRag9cCp5_HsefD7bxKe3ea2RPn8jnSs/edit?tab=t.k47ujuef4zxk#bookmark=id.hl0pbdvwfotd

- Restrict domainNames to Admin tier only with CEL validation
  (Baseline tier breaks the NetworkPolicy override model)
- Add CEL validation for Accept-only action restriction
- Update API examples to v1alpha2 ClusterNetworkPolicy format
- Add Expected Behavior points for DNS reachability and resolv.conf
- Add Recommended Behavior section covering DNS security concerns,
  grace period, and DNSSEC best practice
- Add Connection Lifecycle section for policy add/remove semantics

Author: tssurya

npeps/npep-133-fqdn-egress-selector.md

@@ -16,33 +16,35 @@ Names](https://www.wikipedia.org/wiki/Fully_qualified_domain_name) (FQDNs).
   (for example `kubernetes.io`).
 * Support basic wildcard matching capabilities when specifying FQDNs (for
   example `*.cloud-provider.io`)
-* Currently only `ALLOW` type rules are proposed.
+* Currently only `ACCEPT` type rules are proposed.
   * Safely enforcing `DENY` rules based on FQDN selectors is difficult as there
     is no guarantee a Network Policy plugin is aware of all IPs backing a FQDN
     policy. If a Network Policy plugin has incomplete information, it may
     accidentally allow traffic to an IP belonging to a denied domain. This would
     constitute a security breach.
 
-    By contrast, `ALLOW` rules, which may also have an incomplete list of IPs,
+    By contrast, `ACCEPT` rules, which may also have an incomplete list of IPs,
     would not create a security breach. In case of incomplete information, valid
     traffic would be dropped as the plugin believes the destination IP does not
     belong to the domain. While this is definitely undesirable, it is at least
     not an unsafe failure.
 
-* Currently only AdminNetworkPolicy is the intended scope for this proposal.
-  * Since Kubernetes NetworkPolicy does not have a FQDN selector, adding this
-    capability to BaselineAdminNetworkPolicy could result in writing baseline
-    rules that can't be replicated by an overriding NetworkPolicy. For example,
-    if BANP allows traffic to `example.io`, but the namespace admin installs a
-    Kubernetes Network Policy, the namespace admin has no way to replicate the
-    `example.io` selector using just Kubernetes Network Policies.
+* DomainNames is restricted to the Admin tier of ClusterNetworkPolicy only.
+  * Since Kubernetes NetworkPolicy does not have a FQDN selector, using
+    domainNames in the Baseline tier would allow writing baseline rules that
+    can't be replicated by an overriding NetworkPolicy. For example, if a
+    Baseline-tier ClusterNetworkPolicy allows traffic to `example.io`, but
+    the namespace admin installs a Kubernetes NetworkPolicy, the namespace
+    admin has no way to replicate the `example.io` selector using just
+    Kubernetes NetworkPolicies. This breaks the fundamental tier override
+    model where NetworkPolicy can always override Baseline-tier rules.
 
 ## Non-Goals
 
 * This enhancement does not include a FQDN selector for allowing ingress
   traffic.
 * This enhancement only describes enhancements to the existing L4 filtering as
-  provided by AdminNetworkPolicy. It does not propose any new L7 matching or
+  provided by ClusterNetworkPolicy. It does not propose any new L7 matching or
   filtering capabilities, like matching HTTP traffic or URL paths.
   * This selector should not control what DNS records are resolvable from a
     particular workload.
@@ -92,15 +94,29 @@ goal in this case is to ensure we do not make these unimplementable down the
 line.
 
 * As a cluster admin, I want to switch the default disposition of the cluster to
-  be default deny. This is enforced using a `BaselineAdminNetworkPolicy`. I also
-  want individual namespace owners to be able to specify their egress peers.
-  Namespace admins would then use a FQDN selector in the Kubernetes
-  `NetworkPolicy` objects to allow `my-service.com`.
+  be default deny. This is enforced using a Baseline-tier
+  `ClusterNetworkPolicy`. I also want individual namespace owners to be able to
+  specify their egress peers. Namespace admins would then use a FQDN selector
+  in the Kubernetes `NetworkPolicy` objects to allow `my-service.com`.
 
 ## API
 
-This NPEP proposes adding a new type of `AdminNetworkPolicyEgressPeer` called
-`FQDNPeerSelector` which allows specifying domain names.
+This NPEP proposes adding a `DomainNames` field to
+`ClusterNetworkPolicyEgressPeer` which allows specifying domain names as
+egress peers. DomainNames is only available with Accept rules in the Admin
+tier of ClusterNetworkPolicy.
+
+These restrictions are enforced via CEL validation on
+`ClusterNetworkPolicySpec` (Baseline tier) and
+`ClusterNetworkPolicyEgressRule` (Accept-only):
+
+```go
+// +kubebuilder:validation:XValidation:rule="self.tier == 'Baseline' ? !self.egress.exists(rule, rule.to.exists(peer, has(peer.domainNames))) : true",message="domainNames cannot be used in Baseline tier as NetworkPolicy cannot override FQDN rules"
+type ClusterNetworkPolicySpec struct { ... }
+
+// +kubebuilder:validation:XValidation:rule="self.to.exists(peer, has(peer.domainNames)) ? self.action == 'Accept' : true",message="domainNames may only be used with Accept action"
+type ClusterNetworkPolicyEgressRule struct { ... }
+```
 
 ```golang
 
@@ -126,22 +142,27 @@ This NPEP proposes adding a new type of `AdminNetworkPolicyEgressPeer` called
 // +kubebuilder:validation:Pattern=`^(\*\.)?([a-zA-z0-9]([-a-zA-Z0-9_]*[a-zA-Z0-9])?\.)+[a-zA-z0-9]([-a-zA-Z0-9_]*[a-zA-Z0-9])?\.?$`
 type DomainName string
 
-type AdminNetworkPolicyEgressPeer struct {
+type ClusterNetworkPolicyEgressPeer struct {
     <snipped>
     // DomainNames provides a way to specify domain names as peers.
-    // 
-    // DomainNames is only supported for Allow rules. In order to control
-    // access, DomainNames Allow rules should be used with a lower priority
-    // egress deny -- this allows the admin to maintain an explicit "allowlist"
-    // of reachable domains.
+    //
+    // DomainNames is only supported for Accept rules in the Admin tier.
+    // In order to control access, DomainNames Accept rules should be used
+    // with a lower precedence egress deny -- this allows the admin to
+    // maintain an explicit "allowlist" of reachable domains.
+    //
+    // DomainNames cannot be used in the Baseline tier because Kubernetes
+    // NetworkPolicy has no FQDN selector, so a Baseline FQDN rule cannot
+    // be overridden by a NetworkPolicy.
     // 
     // Support: Extended
     //
     // <network-policy-api:experimental>
     // +optional
     // +listType=set
     // +kubebuilder:validation:MinItems=1
-    DomainNames []Domain `json:"domainNames,omitempty"`
+    // +kubebuilder:validation:MaxItems=25
+    DomainNames []DomainName `json:"domainNames,omitempty"`
 }
 ```
 
@@ -150,108 +171,116 @@ type AdminNetworkPolicyEgressPeer struct {
 #### Pods in `monitoring` namespace can talk to `my-service.com` and `*.cloud-provider.io`
 
 ```yaml
-apiVersion: policy.networking.k8s.io/v1alpha1
-kind: AdminNetworkPolicy
+apiVersion: policy.networking.k8s.io/v1alpha2
+kind: ClusterNetworkPolicy
 metadata:
   name: allow-my-service-egress
 spec:
+  tier: Admin
   priority: 55
   subject:
     namespaces:
       matchLabels:
         kubernetes.io/metadata.name: "monitoring"
   egress:
   - name: "allow-to-my-service"
-    action: "Allow"
+    action: "Accept"
     to:
     - domainNames:
       - "my-service.com"
       - "*.cloud-provider.io"
-    ports:
-    - portNumber:
-        protocol: TCP
-        port: 443
+    protocols:
+    - tcp:
+        destinationPort:
+          number: 443
 ```
 
 #### Maintaining an allowlist of domains
 
 There are a couple ways to maintain an allowlist:
 
-This example, includes the DENY rule in the same ANP object. It's also possible
-to use another ANP object with a lower priority (e.g. `100` in this example):
+This example includes the Deny rule in the same ClusterNetworkPolicy object.
+It's also possible to use another ClusterNetworkPolicy object with a lower
+priority (e.g. `100` in this example):
 ```yaml
-apiVersion: policy.networking.k8s.io/v1alpha1
-kind: AdminNetworkPolicy
+apiVersion: policy.networking.k8s.io/v1alpha2
+kind: ClusterNetworkPolicy
 metadata:
   name: allow-my-service-egress
 spec:
+  tier: Admin
   priority: 55
   subject:
     namespaces:
       matchLabels:
         kubernetes.io/metadata.name: "monitoring"
   egress:
   - name: "allow-to-my-service"
-    action: "Allow"
+    action: "Accept"
     to:
     - domainNames:
       - "my-service.com"
       - "*.cloud-provider.io"
-    ports:
-    - portNumber:
-        protocol: TCP
-        port: 443
+    protocols:
+    - tcp:
+        destinationPort:
+          number: 443
   - name: "default-deny"
     action: "Deny"
     to:
     - networks:
       - "0.0.0.0/0"
 ```
 
-This example uses a default-deny BaselineAdminNetworkPolicy to create the
-allowlist:
+This example uses a Baseline-tier default-deny ClusterNetworkPolicy to create
+the allowlist:
 ```yaml
-apiVersion: policy.networking.k8s.io/v1alpha1
-kind: AdminNetworkPolicy
+apiVersion: policy.networking.k8s.io/v1alpha2
+kind: ClusterNetworkPolicy
 metadata:
   name: allow-my-service-egress
 spec:
+  tier: Admin
   priority: 55
   subject:
     namespaces:
       matchLabels:
         kubernetes.io/metadata.name: "monitoring"
   egress:
   - name: "allow-to-my-service"
-    action: "Allow"
+    action: "Accept"
     to:
     - domainNames:
       - "my-service.com"
       - "*.cloud-provider.io"
-    ports:
-    - portNumber:
-        protocol: TCP
-        port: 443
+    protocols:
+    - tcp:
+        destinationPort:
+          number: 443
 ---
-apiVersion: policy.networking.k8s.io/v1alpha1
-kind: BaselineAdminNetworkPolicy
+apiVersion: policy.networking.k8s.io/v1alpha2
+kind: ClusterNetworkPolicy
 metadata:
-  name: default
+  name: default-deny
 spec:
+  tier: Baseline
+  priority: 0
   subject:
     namespaces: {}
-  ingress:
-    - action: Deny
-      to:
-      - networks:
-        - "0.0.0.0/0"
+  egress:
+  - name: "default-deny"
+    action: "Deny"
+    to:
+    - networks:
+      - "0.0.0.0/0"
 ```
 
 ### Expected Behavior
 
 1. A FQDN egress policy does not grant the workload permission to communicate
    with any in-cluster DNS services (like `kube-dns`). A separate rule needs to
-   be configured to allow traffic to any DNS servers.
+   be configured to allow traffic to any DNS servers. FQDN policies are not
+   expected to work if the pod cannot reach DNS.
 1. FQDN policies should not affect the ability of workloads to resolve domains,
    only their ability to communicate with the IP backing them. Put another way,
    FQDN policies should not result in any form of DNS filtering.
@@ -263,6 +292,10 @@ spec:
    considered authoritative for resolving domain names. This could be the
    `kube-dns` Service or potentially some other DNS provider specified in the
    implementation's configuration.
+1. Pods are expected to use the DNS configuration provided via `resolv.conf`
+   (i.e. the canonical cluster DNS server). If a pod uses a different DNS
+   server (e.g. hardcoded `8.8.8.8`), FQDN rule processing is not guaranteed
+   to work.
 1. DNS record querying and lifetimes:
    *  Pods are expected to make a DNS query for a domain before sending traffic
       to it. If the Pod fails to send a DNS request and instead just sends
@@ -272,9 +305,12 @@ spec:
       establish new connection using DNS records that are expired is not
       guaranteed to work.
    *  When the TTL for a DNS record expires, the implementor should stop
-      allowing new connections to that IP. Existing connection will still be
+      allowing new connections to that IP. Existing connections will still be
       allowed (that's consistent with NetworkPolicy behavior on long-running
-      connections). 
+      connections).
+   *  If a DNS record is refreshed before the TTL expires (e.g., due to a new
+      DNS query from the workload), the TTL timer should be reset based on the
+      new response.
 1. Implementations must support at least 100 unique IPs (either IPv4 or IPv6)
    for each domain. This is true for both explicitly specified domains, as well
    as for each domain selected by a wild-card rule. For example, the rule
@@ -342,6 +378,67 @@ spec:
         The implementer can still deny traffic to `1.2.3.4` because no single
         response contained the full chain required to resolve the domain.
 
+### Recommended Behavior
+
+The following recommendations are based on operational experience from existing
+implementations and feedback gathered during KubeCon Atlanta 2025.
+
+1. Allowing pods to reach arbitrary DNS servers is a security concern: an
+   untrusted DNS server could return malicious IP mappings, potentially
+   bypassing FQDN policy intent. The administrator should create a DNS allow
+   rule that restricts egress DNS traffic to only the trusted canonical DNS
+   server (e.g. `kube-dns` in `kube-system`), rather than allowing DNS to all
+   destinations. For example:
+   ```yaml
+   apiVersion: policy.networking.k8s.io/v1alpha2
+   kind: ClusterNetworkPolicy
+   metadata:
+     name: allow-dns
+   spec:
+     tier: Admin
+     priority: 50
+     subject:
+       namespaces:
+         matchLabels:
+           requires-fqdn-policy: "true"
+     egress:
+     - name: "allow-dns"
+       action: "Accept"
+       to:
+       - namespaces:
+           matchLabels:
+             kubernetes.io/metadata.name: "kube-system"
+       protocols:
+       - udp:
+           destinationPort:
+             number: 53
+       - tcp:
+           destinationPort:
+             number: 53
+   ```
+1. Implementations SHOULD NOT trust DNS responses that do not originate from
+   the canonical DNS server, as these could be spoofed or manipulated.
+1. Implementations MAY provide a configurable grace period beyond the TTL to
+   accommodate DNS propagation delays and client-side caching.
+1. Although securing DNS resolution is a non-goal of this NPEP, implementations
+   are recommended to consider mitigations for DNS cache poisoning (e.g.,
+   DNSSEC validation by the canonical DNS server) when documenting their trust
+   model.
+
+## Connection Lifecycle on Policy Updates
+
+When FQDN policies change, implementations must handle in-flight connections
+gracefully. See the [Expected Behavior](#expected-behavior) section for DNS
+record updates.
+
+* **Policy addition**: New connections matching the FQDN are allowed once DNS
+  resolution for the domain has been observed. Connections to IPs that haven't
+  been observed via DNS are not guaranteed to be allowed.
+* **Policy removal**: Existing established connections SHOULD be allowed to
+  complete gracefully. New connections SHOULD be denied after the policy is
+  removed. This is consistent with the general NetworkPolicy behavior for
+  long-running connections.
+
 ## Alternatives
 
 ### IP Block Selector