Add comprehensive gVisor firewall comparison workflow#5236
Conversation
- Compare Squid vs Envoy proxy approaches - Test under both runc and gVisor runtimes - Verify iptables DNAT/redirect compatibility with gVisor - Benchmark performance (latency comparison) - Generate summary report with recommendations Tests answer key questions: 1. Does gVisor support iptables DNAT for traffic redirection? 2. Which proxy approach works better with gVisor? 3. Can AWF keep current Squid architecture or need Envoy? Related to issue #3264 (gVisor compatibility investigation) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (1 files)
Coverage comparison generated by |
There was a problem hiding this comment.
Pull request overview
Adds a new manual GitHub Actions workflow to empirically compare AWF’s current Squid+iptables approach versus an Envoy-based proxy approach under both the default Docker runtime (runc) and gVisor (runsc), including a simple latency benchmark and a final aggregated report.
Changes:
- Introduces a multi-job workflow that runs Squid+iptables tests under runc and gVisor.
- Adds Envoy+iptables test jobs under runc and gVisor.
- Adds a basic “100 requests” latency benchmark and a final summary report job.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/test-gvisor-firewall-comparison.yml | New workflow to compare proxy/firewall behaviors (Squid vs Envoy) across runc and gVisor, plus a latency benchmark and summary output. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/1 changed files
- Comments generated: 19
| sudo apt-get install -y apt-transport-https gnupg2 curl lsb-release | ||
| curl -sL 'https://deb.dl.getenvoy.io/public/gpg.8115BA8E629CC074.key' | sudo gpg --dearmor -o /usr/share/keyrings/getenvoy-keyring.gpg | ||
| echo "deb [arch=amd64 signed-by=/usr/share/keyrings/getenvoy-keyring-gpg] https://deb.dl.getenvoy.io/public/deb/ubuntu $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/getenvoy.list | ||
| sudo apt-get update -qq |
| if [ "$HTTP_CODE" = "403" ] || [ "$HTTP_CODE" = "000" ]; then | ||
| echo "✅ PASS: google.com blocked (HTTP $HTTP_CODE)" | ||
| else | ||
| echo "❌ FAIL: google.com accessible (HTTP $HTTP_CODE)" | ||
| fi |
| # Test if DNAT actually redirects or just accepts the rule | ||
| echo "" | ||
| echo "=== Verifying DNAT actually redirects traffic ===" | ||
| if curl -v --max-time 5 https://github.com 2>&1 | grep -q "Connected to 172.30.0.10"; then | ||
| echo "✅ CONFIRMED: DNAT redirected traffic to proxy" | ||
| else | ||
| echo "⚠️ DNAT rule exists but may not redirect traffic" | ||
| fi |
| run: | | ||
| set -x | ||
| echo "## Performance Test: Squid" | ||
|
|
| run: | | ||
| set -x | ||
| echo "## Performance Test: Envoy" | ||
|
|
| - name: Checkout repository | ||
| uses: actions/checkout@v4 |
| - name: "Test: Envoy under gVisor" | ||
| run: | | ||
| set -x | ||
| echo "## Testing Envoy + iptables under gVisor" | ||
| echo "" | ||
|
|
Fixes all 19 review comments: 1. Remove ineffective setup job (ran on different runner) 2. Pin actions/checkout by SHA for supply-chain hardening 3. Add set -euo pipefail and EXIT traps to all test steps 4. Make test assertions fail with exit 1 instead of just logging 5. Add DNAT fallback tests with proxy env disabled 6. Fix benchmark outputs to write to $GITHUB_OUTPUT 7. Fix benchmark to use explicit proxy (-x flag) 8. Fix Envoy gVisor config to match runc (add dynamic_forward_proxy) 9. Clarify HTTPS expectations for Envoy (known limitation) 10. Add job outputs for performance comparison 11. Add header note explaining defense-in-depth test approach Key changes: - All test jobs now properly propagate failures - DNAT verification tests actually check enforcement (not just rule acceptance) - Performance benchmarks capture and output latency correctly - Cleanup happens reliably via EXIT traps - Envoy configs consistent between runc and gVisor tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
✅ Copilot review passed with no inline comments. @lpcox Add the |
✅ All 19 review comments addressedThanks for the thorough review! I've addressed all feedback in commit d97caca. Key fixes:Architecture changes:
Reliability improvements: DNAT verification: Performance benchmarks: Envoy configuration: Documentation: All jobs now reliably report pass/fail, the summary report will show actual performance numbers, and the tests validate actual enforcement (not just configuration acceptance). |
Summary
This PR adds a comprehensive test workflow that compares Squid vs Envoy proxy approaches under both runc and gVisor runtimes.
Test Coverage
1. Squid + iptables DNAT (Current AWF Approach)
2. Envoy + iptables redirect (Alternative Approach)
3. Performance Comparison
Key Questions Answered
Does gVisor's userspace network stack support iptables DNAT?
Which approach works better with gVisor?
Performance implications?
Can AWF keep its current architecture with gVisor?
Running the Tests
The workflow generates a summary report showing pass/fail status, performance metrics, and recommendations.
Related Issues
Addresses questions raised in #3264 (gVisor compatibility investigation)
Files Changed
.github/workflows/test-gvisor-firewall-comparison.yml- New comprehensive test workflow