Skip to content

Commit 12e0a46

Browse files
scotwellsclaude
scotwells
and
claude
committed
test: move the TPP Enforce regression into the federated edge suite
The issue #243 regression (a TrafficProtectionPolicy in Enforce mode must not take the gateway offline) needs the real WAF data plane — extension server + Coraza on the datum-downstream-gateway class. It lived in test/e2e, which the single-stack workflow auto-runs and cannot exclude, so it failed there against an environment with no WAF (the Gateway never reaches Programmed). Relocate it to test/e2e-edge, adapted to the two-cluster topology and the shared fixtures/templates, and add it to the federated scenario set. The single-stack suite keeps the fast operator-only tests; edge/WAF/federation tests belong here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JbCy8vy66RdNYzGSgqH6P6
1 parent abcf604 commit 12e0a46

3 files changed

Lines changed: 292 additions & 165 deletions

File tree

‎Taskfile.test-infra.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -419,7 +419,7 @@ tasks:
419419
vars:
420420
# Scenarios authored against this env's cluster names and downstream path.
421421
# Older fixtures targeting the previous cluster names are excluded here.
422-
DEFAULT_SCENARIOS: extension-server-smoke waf-enforcement branded-error-page connector-offline-503 atomic-reject-isolation
422+
DEFAULT_SCENARIOS: extension-server-smoke waf-enforcement trafficprotectionpolicy-enforce branded-error-page connector-offline-503 atomic-reject-isolation
423423
# CLI_ARGS (after --) wins; else SCENARIOS env; else the default set.
424424
SELECTED: '{{.CLI_ARGS | default .SCENARIOS | default .DEFAULT_SCENARIOS}}'
425425
deps:

‎test/e2e-edge/trafficprotectionpolicy-enforce/chainsaw-test.yaml‎

Lines changed: 291 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,291 @@
1+
# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json
2+
#
3+
# TrafficProtectionPolicy Enforce mode must not take the gateway offline.
4+
#
5+
# Regression for issue #243: the branded error page NSO injects into Envoy's
6+
# local_reply_config is a SubstitutionFormatString, so an unescaped '%' in the
7+
# page (e.g. CSS "height: 100%") makes Envoy reject the whole listener and, with
8+
# failOpen:false, drops the xDS update — every request then fails with an empty
9+
# reply instead of the backend's 200. This drives a real benign request through
10+
# a WAF gateway carrying an Enforce-mode policy and requires a 200, asserted on
11+
# the wire rather than on Gateway.status (EG reports Programmed from its own
12+
# translation, independent of Envoy's xDS ACK).
13+
#
14+
# This lives in the edge suite, not test/e2e, because it needs the real WAF data
15+
# plane (extension server + Coraza on the datum-downstream-gateway class) the
16+
# single-stack environment does not have. The two cluster names are written
17+
# inline (nso-upstream / nso-downstream) because chainsaw cannot bind a cluster
18+
# name. Runs alone (concurrent: false) because it shares cluster-wide downstream
19+
# config with the other data-plane tests.
20+
apiVersion: chainsaw.kyverno.io/v1alpha1
21+
kind: Test
22+
metadata:
23+
name: trafficprotectionpolicy-enforce-serves-traffic
24+
spec:
25+
concurrent: false
26+
bindings:
27+
- name: clusterIssuerName
28+
value: (join('-', ['e2e', $namespace]))
29+
- name: gatewayClassName
30+
value: (join('-', ['e2e', $namespace]))
31+
# The downstream gateway class wired to the extension server.
32+
- name: downstreamGatewayClass
33+
value: datum-downstream-gateway-e2e
34+
- name: hostname
35+
value: tpp-enforce.e2e.env.datum.net
36+
cluster: nso-upstream
37+
steps:
38+
- name: Create CA on the downstream cluster
39+
try:
40+
- create:
41+
cluster: nso-downstream
42+
resource:
43+
apiVersion: cert-manager.io/v1
44+
kind: ClusterIssuer
45+
metadata:
46+
name: (join('-', [$clusterIssuerName, 'issuer']))
47+
spec:
48+
selfSigned: {}
49+
- create:
50+
cluster: nso-downstream
51+
resource:
52+
apiVersion: cert-manager.io/v1
53+
kind: Certificate
54+
metadata:
55+
name: (join('-', [$clusterIssuerName, 'ca']))
56+
namespace: cert-manager
57+
spec:
58+
isCA: true
59+
commonName: (join('-', [$clusterIssuerName, 'ca']))
60+
secretName: ($clusterIssuerName)
61+
privateKey:
62+
algorithm: ECDSA
63+
size: 256
64+
issuerRef:
65+
name: (join('-', [$clusterIssuerName, 'issuer']))
66+
kind: ClusterIssuer
67+
group: cert-manager.io
68+
- create:
69+
cluster: nso-downstream
70+
resource:
71+
apiVersion: cert-manager.io/v1
72+
kind: ClusterIssuer
73+
metadata:
74+
name: ($clusterIssuerName)
75+
spec:
76+
ca:
77+
secretName: ($clusterIssuerName)
78+
- assert:
79+
cluster: nso-downstream
80+
resource:
81+
apiVersion: cert-manager.io/v1
82+
kind: Certificate
83+
metadata:
84+
name: (join('-', [$clusterIssuerName, 'ca']))
85+
namespace: cert-manager
86+
status:
87+
conditions:
88+
- type: Ready
89+
status: "True"
90+
91+
- name: Create the upstream GatewayClass (downstream class/EnvoyProxy are pre-provisioned)
92+
try:
93+
- create:
94+
cluster: nso-upstream
95+
resource:
96+
apiVersion: gateway.networking.k8s.io/v1
97+
kind: GatewayClass
98+
metadata:
99+
name: ($gatewayClassName)
100+
spec:
101+
controllerName: gateway.networking.datumapis.com/external-global-proxy-controller
102+
- apply:
103+
cluster: nso-downstream
104+
resource:
105+
apiVersion: v1
106+
kind: Namespace
107+
metadata:
108+
name: datum-downstream-gateway-hostnames
109+
110+
- name: Create the echo backend
111+
try:
112+
- apply:
113+
cluster: nso-downstream
114+
file: ../_fixtures/echo-backend.yaml
115+
116+
- name: Provision and verify the Domain
117+
try:
118+
- create:
119+
cluster: nso-upstream
120+
resource:
121+
apiVersion: networking.datumapis.com/v1alpha
122+
kind: Domain
123+
metadata:
124+
name: test-domain
125+
spec:
126+
domainName: e2e.env.datum.net
127+
- description: Stand in for the Domain controller's HTTP/DNS verification.
128+
script:
129+
content: |
130+
kubectl -n $NAMESPACE patch domain test-domain \
131+
--subresource=status --type=merge \
132+
-p '{"status":{"conditions":[{"type": "Verified", "status": "True", "reason": "Test", "message": "test", "lastTransitionTime": "2025-02-24T23:59:09Z"}]}}'
133+
134+
- name: Provision a Gateway, route, and an Enforce-mode TrafficProtectionPolicy
135+
try:
136+
- create:
137+
cluster: nso-upstream
138+
resource:
139+
apiVersion: gateway.networking.k8s.io/v1
140+
kind: Gateway
141+
metadata:
142+
name: test-gateway
143+
spec:
144+
gatewayClassName: ($gatewayClassName)
145+
listeners:
146+
- protocol: HTTPS
147+
port: 443
148+
name: https-waf
149+
allowedRoutes:
150+
namespaces:
151+
from: Same
152+
hostname: ($hostname)
153+
tls:
154+
mode: Terminate
155+
options:
156+
gateway.networking.datumapis.com/certificate-issuer: ($clusterIssuerName)
157+
- create:
158+
cluster: nso-upstream
159+
resource:
160+
apiVersion: discovery.k8s.io/v1
161+
kind: EndpointSlice
162+
metadata:
163+
name: test-slice-1
164+
addressType: FQDN
165+
endpoints:
166+
- addresses:
167+
- echo-backend.default.svc.cluster.local
168+
conditions:
169+
ready: true
170+
serving: true
171+
terminating: false
172+
ports:
173+
- name: http
174+
appProtocol: http
175+
port: 8080
176+
- create:
177+
cluster: nso-upstream
178+
resource:
179+
apiVersion: gateway.networking.k8s.io/v1
180+
kind: HTTPRoute
181+
metadata:
182+
name: test-route
183+
spec:
184+
parentRefs:
185+
- name: test-gateway
186+
kind: Gateway
187+
rules:
188+
- matches:
189+
- path:
190+
type: PathPrefix
191+
value: /
192+
backendRefs:
193+
- group: discovery.k8s.io
194+
kind: EndpointSlice
195+
name: test-slice-1
196+
port: 8080
197+
# Enforce mode: the policy is live and can block. The regression is that
198+
# it must not, by itself, break the listener for benign traffic.
199+
- create:
200+
cluster: nso-upstream
201+
resource:
202+
apiVersion: networking.datumapis.com/v1alpha
203+
kind: TrafficProtectionPolicy
204+
metadata:
205+
name: test-waf
206+
spec:
207+
targetRefs:
208+
- group: gateway.networking.k8s.io
209+
kind: Gateway
210+
name: test-gateway
211+
mode: Enforce
212+
ruleSets:
213+
- type: OWASPCoreRuleSet
214+
owaspCoreRuleSet: {}
215+
- assert:
216+
timeout: 120s
217+
cluster: nso-upstream
218+
resource:
219+
apiVersion: gateway.networking.k8s.io/v1
220+
kind: Gateway
221+
metadata:
222+
name: test-gateway
223+
status:
224+
(addresses[?type == 'Hostname'] | [0]):
225+
type: Hostname
226+
catch:
227+
- script:
228+
cluster: nso-upstream
229+
content: |
230+
kubectl -n network-services-operator-system logs -l app.kubernetes.io/name=network-services-operator --tail=200
231+
kubectl -n $NAMESPACE get gateway,httproute,trafficprotectionpolicy -o yaml
232+
233+
- name: Stand up the long-lived conntest client
234+
try:
235+
- assert:
236+
cluster: nso-downstream
237+
resource:
238+
apiVersion: v1
239+
kind: Pod
240+
metadata:
241+
name: echo-backend
242+
namespace: default
243+
status:
244+
phase: Running
245+
- create:
246+
cluster: nso-downstream
247+
resource:
248+
apiVersion: v1
249+
kind: Pod
250+
metadata:
251+
name: conntest
252+
namespace: default
253+
spec:
254+
containers:
255+
- name: curl
256+
image: curlimages/curl:8.11.1
257+
command: ["sleep", "infinity"]
258+
resources:
259+
requests:
260+
cpu: 50m
261+
memory: 64Mi
262+
limits:
263+
cpu: 100m
264+
memory: 128Mi
265+
terminationGracePeriodSeconds: 0
266+
- assert:
267+
cluster: nso-downstream
268+
resource:
269+
apiVersion: v1
270+
kind: Pod
271+
metadata:
272+
name: conntest
273+
namespace: default
274+
status:
275+
phase: Running
276+
277+
- name: Benign traffic still returns 200 under an Enforce-mode policy (issue #243)
278+
description: >
279+
The core regression check: with an Enforce-mode policy attached and the
280+
branded error page in the proxy's local_reply_config, a benign request
281+
must still reach the backend. Before the fix the listener is rejected and
282+
this returns 000 / empty reply; after the fix it returns the backend's
283+
200.
284+
use:
285+
template: ../_steps/assert-http-response.yaml
286+
with:
287+
bindings:
288+
- { name: host, value: tpp-enforce.e2e.env.datum.net }
289+
- { name: path, value: /get }
290+
- { name: scheme, value: https }
291+
- { name: expectCode, value: "200" }

0 commit comments

Comments
 (0)