feat: support Filesystem Wasm code source via EnvoyProxy allowlist#9530
feat: support Filesystem Wasm code source via EnvoyProxy allowlist#9530daanvinken wants to merge 4 commits into
Conversation
✅ Deploy Preview for cerulean-figolla-1f9435 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
@zhaohuabing if you have some time 🙏 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 53e53febd3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| wasms []ir.Wasm | ||
| wasmFailOpen bool | ||
| ) | ||
| if wasms, wasmError, wasmFailOpen = t.buildWasms(policy, resources, gtwCtx.envoyProxy); wasmError != nil { |
There was a problem hiding this comment.
Scope filesystem Wasm errors per ancestor
When an HTTPRoute is attached to multiple managed Gateways, this call now resolves the filesystem module against each Gateway's EnvoyProxy but appends any failure to one errs value returned to processEnvoyExtensionPolicyForRoute, which then sets the translation error on all ancestor refs. If one Gateway allowlists the module and another does not, the successful ancestor is still marked translation-failed even though its routes get the filter; please report/apply the error per parentRef instead of aggregating it across all ancestors.
Useful? React with 👍 / 👎.
53e53fe to
089e532
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #9530 +/- ##
=======================================
Coverage 75.55% 75.56%
=======================================
Files 252 252
Lines 41661 41755 +94
=======================================
+ Hits 31479 31552 +73
- Misses 8057 8071 +14
- Partials 2125 2132 +7 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
f4fe92e to
fddff27
Compare
containsWasm (internal/xds/translator/globalresources.go:144) returns true for any Wasm, so createWasmHTTPServiceCluster still emits the wasm_cluster (the EG HTTP serving cluster, plus its upstream TLS transportSocket) even when every Wasm is a Filesystem source that never contacts it. This is visible in the PR's own fixture wasm-filesystem.clusters.yaml, which contains a full wasm_cluster that no filter references. Suggest gating on the presence of a remote source: This avoids shipping a dead cluster (and its client-cert secret dependency) to proxies that only use filesystem modules — which is precisely the "skip the control-plane path" benefit the PR advertises.
In translateEnvoyExtensionPolicyForRoute, buildWasms moved from once-per-policy to once-per-parentRef (necessary, since the filesystem path resolves against gtwCtx.envoyProxy, which can differ per Gateway). This matches the existing buildLuas/buildDynamicModules placement, so structurally it's fine. The wrinkle unique to Wasm: for HTTP/Image sources this now calls t.WasmCache.Get() N times for a route with N parentRefs, where it previously ran once. For IfNotPresent this is a cheap cache hit, but with pullPolicy: Always it can trigger repeated fetches. Not a correctness bug, but a behavioral change worth a comment or a second look — a small memoization keyed by resolved config would eliminate it if it matters.
The field-level annotation changed from Enum=HTTP;Image;ConfigMap to Enum=HTTP;Image;Filesystem (wasm_types.go:41). ConfigMap was never implemented (no code path, and the type alias only listed HTTP;Image), so this is stale cleanup rather than a real removal. Still, it's technically a narrowing of the accepted API surface — worth an explicit mention in the PR description/release note in case anyone had it set (it would have been non-functional regardless). |
Register Wasm modules on EnvoyProxy.spec.wasmModules and reference them by name from EnvoyExtensionPolicy with type Filesystem. Envoy loads the local file via AsyncDataSource.local and skips the control-plane fetch/cache path used by HTTP and Image sources. Includes generated CRD/helm fixtures, API docs, and the jwt DnsCluster fixture update needed for gen-check. Fixes envoyproxy#9448 Signed-off-by: daanvinken <daanvinken@tythus.com>
fddff27 to
3e206c5
Compare
Only create the control-plane wasm HTTP service cluster (and gate the client certificate) when a route uses a remote HTTP/Image code source. Filesystem modules load a local path and never use that cluster. Signed-off-by: daanvinken <daanvinken@tythus.com>
Filesystem resolution is Gateway-scoped, so buildWasms runs per parentRef like Lua and DynamicModules. HTTP/Image may call WasmCache.Get once per parent; IfNotPresent hits the cache, Always may re-fetch. Signed-off-by: daanvinken <daanvinken@tythus.com>
ConfigMap was never implemented; the code source enum is now HTTP, Image, and Filesystem only. Signed-off-by: daanvinken <daanvinken@tythus.com>
|
Super helpful, thanks @zhaohuabing I've addressed your comments. I'll see when I can carve out some time to deploy this on an actual cluster and test. |
|
Should we mirror |
You should be able to add an e2e test for this, following the dynamic module example. You can use this WASM file. for the test. |
I agree with the suggested shape for filesystem WASM modules but long-term handling of image/http sources might get awkward so we should discuss that as well. |
Description
Adds a Filesystem WASM code source so Envoy can load modules from a local path
without the control-plane fetch/cache path used by HTTP and Image sources.
Design matches the discussion in #9448 (DynamicModule-style allowlist):
EnvoyProxy.spec.wasmModules(name+ absolutepath)EnvoyExtensionPolicywithcode.type: Filesystemandfilesystem.nameonly (no path on the policy)AsyncDataSource.local; HTTP/Image behavior is unchangedFixes #9448
API note
The Wasm
code.typeenum is nowHTTP,Image, andFilesystem. Theprevious
ConfigMapenum value was never implemented and has been removed.Testing
(registered module resolves to a path; unknown module name is rejected)
code.local.filenamefor a Filesystem moduleIf desired, I can test it on a real cluster, need to upgrade Envoy Gateway first...
Generated with Grok Build 4.5.