Skip to content

Commit 95a3de3

Browse files
Mossakaclaude
Mossaka
and
claude
authored
fix: restructure smoke-chroot workflow to avoid Docker-in-Docker (#508)
* feat: add build test workflows for external test repos Add 7 agentic workflows that clone and test external repositories to validate firewall network rules work correctly for different language ecosystems: - Node.js: clsx, execa, p-limit (npm install/test) - Go: color, env, uuid (go mod download/test) - Rust: fd, zoxide (cargo build/test) - Java: gson, caffeine (mvn compile/test) - C++: fmt, json (cmake/make) - Deno: oak, std (deno test) - Bun: elysia, hono (bun install/test) Each workflow: - Triggers on pull_request and workflow_dispatch - Uses appropriate network allowlists (defaults, github, language-specific) - Reports results via PR comments with pass/fail tables - Adds language-specific labels on success Also updates postprocess-smoke-workflows.ts to include the new workflows for local build patching in CI. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: restructure smoke-chroot workflow to avoid Docker-in-Docker The smoke-chroot workflow had a design incompatibility where it attempted to run `awf --enable-chroot` commands from within an AWF sandbox container. This creates a nested Docker-in-Docker scenario that is not supported (Docker-in-Docker was removed in AWF v0.9.1, PR #205). This fix restructures the workflow to: 1. Run all chroot tests in the frontmatter `steps:` section, which executes directly on the GitHub Actions runner BEFORE the agent sandbox starts 2. Save test results to files in /tmp/gh-aw/chroot-test/ 3. Have the agent (running inside AWF) simply read the results and post PR comments/labels The workflow now: - Sets up Go (for consistent version testing) - Captures host versions (Python, Node, Go) - Builds local containers - Runs chroot version tests via `awf --enable-chroot` - Compares versions and saves results - Agent analyzes results and posts to PR This approach avoids the Docker-in-Docker problem while still validating the chroot feature end-to-end. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: merge main and regenerate smoke-chroot workflow Regenerate smoke-chroot.lock.yml with gh-aw v0.42.0 after merging origin/main to resolve conflicts. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add awf build steps before chroot version tests The smoke-chroot workflow was trying to run `sudo -E awf` commands before the awf CLI was built and installed. This adds the necessary npm ci, npm run build, and awf binary installation steps before the chroot version tests. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: improve version extraction in chroot smoke tests - Quote values in host-versions.env to prevent bash interpretation - Extract version patterns directly with grep instead of using tail -1 (which captured awf's exit message instead of the version) - Simplify comparison logic since chroot versions are already extracted Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 42bc5c7 commit 95a3de3

2 files changed

Lines changed: 165 additions & 63 deletions

File tree

‎.github/workflows/smoke-chroot.lock.yml‎

Lines changed: 42 additions & 33 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎.github/workflows/smoke-chroot.md‎

Lines changed: 123 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
description: Smoke test workflow that validates the --enable-chroot feature by testing host binary access, network firewall, and security boundaries
2+
description: Smoke test workflow that validates the --enable-chroot feature by testing host binary access and comparing versions
33
on:
44
workflow_dispatch:
55
pull_request:
@@ -44,50 +44,143 @@ safe-outputs:
4444
run-failure: "**Chroot tests failed** [{workflow_name}]({run_url}) {status} - See logs for details."
4545
timeout-minutes: 20
4646
steps:
47+
- name: Setup Go
48+
uses: actions/setup-go@0aaccfd150d50ccaeb58ebd88d36e91967a5f35b
49+
with:
50+
go-version: '1.22'
4751
- name: Capture host versions for verification
4852
run: |
4953
echo "=== Capturing host versions for post-verification ==="
50-
echo "HOST_PYTHON_VERSION=$(python3 --version 2>&1 | head -1)" >> /tmp/host-versions.env
51-
echo "HOST_NODE_VERSION=$(node --version 2>&1 | head -1)" >> /tmp/host-versions.env
52-
echo "HOST_GO_VERSION=$(go version 2>&1 | head -1)" >> /tmp/host-versions.env
53-
cat /tmp/host-versions.env
54+
mkdir -p /tmp/gh-aw/chroot-test
55+
{
56+
echo "HOST_PYTHON_VERSION='$(python3 --version 2>&1 | head -1)'"
57+
echo "HOST_NODE_VERSION='$(node --version 2>&1 | head -1)'"
58+
echo "HOST_GO_VERSION='$(go version 2>&1 | head -1)'"
59+
} > /tmp/gh-aw/chroot-test/host-versions.env
60+
cat /tmp/gh-aw/chroot-test/host-versions.env
61+
- name: Install awf dependencies
62+
run: npm ci
63+
- name: Build awf
64+
run: npm run build
65+
- name: Install awf binary (local)
66+
run: |
67+
WORKSPACE_PATH="${GITHUB_WORKSPACE:-$(pwd)}"
68+
NODE_BIN="$(command -v node)"
69+
sudo tee /usr/local/bin/awf > /dev/null <<EOF
70+
#!/bin/bash
71+
exec "${NODE_BIN}" "${WORKSPACE_PATH}/dist/cli.js" "\$@"
72+
EOF
73+
sudo chmod +x /usr/local/bin/awf
74+
- name: Build local containers
75+
run: |
76+
echo "=== Building local containers ==="
77+
docker build -t ghcr.io/github/gh-aw-firewall/squid:latest containers/squid/
78+
docker build -t ghcr.io/github/gh-aw-firewall/agent:latest containers/agent/
79+
- name: Run chroot version tests
80+
run: |
81+
echo "=== Running chroot version tests ==="
82+
83+
# Capture GOROOT for chroot tests
84+
export GOROOT=$(go env GOROOT)
85+
86+
# Test Python version in chroot
87+
echo "Testing Python..."
88+
CHROOT_PYTHON=$(sudo -E awf --enable-chroot --skip-pull --allow-domains localhost -- python3 --version 2>&1 | grep -oP 'Python \d+\.\d+\.\d+' | head -1) || CHROOT_PYTHON="FAILED"
89+
90+
# Test Node version in chroot
91+
echo "Testing Node..."
92+
CHROOT_NODE=$(sudo -E awf --enable-chroot --skip-pull --allow-domains localhost -- node --version 2>&1 | grep -oP 'v\d+\.\d+\.\d+' | head -1) || CHROOT_NODE="FAILED"
93+
94+
# Test Go version in chroot
95+
echo "Testing Go..."
96+
CHROOT_GO=$(sudo -E awf --enable-chroot --skip-pull --allow-domains localhost -- go version 2>&1 | grep -oP 'go\d+\.\d+(\.\d+)?' | head -1) || CHROOT_GO="FAILED"
97+
98+
# Save chroot versions
99+
{
100+
echo "CHROOT_PYTHON_VERSION=$CHROOT_PYTHON"
101+
echo "CHROOT_NODE_VERSION=$CHROOT_NODE"
102+
echo "CHROOT_GO_VERSION=$CHROOT_GO"
103+
} > /tmp/gh-aw/chroot-test/chroot-versions.env
104+
105+
cat /tmp/gh-aw/chroot-test/chroot-versions.env
106+
107+
# Compare versions and create results
108+
source /tmp/gh-aw/chroot-test/host-versions.env
109+
110+
PYTHON_MATCH="NO"
111+
NODE_MATCH="NO"
112+
GO_MATCH="NO"
113+
114+
# Compare Python (extract version number - chroot already extracted as "Python X.Y.Z")
115+
HOST_PY_NUM=$(echo "$HOST_PYTHON_VERSION" | grep -oP 'Python \d+\.\d+\.\d+' || echo "")
116+
CHROOT_PY_NUM="$CHROOT_PYTHON"
117+
[ "$HOST_PY_NUM" = "$CHROOT_PY_NUM" ] && [ -n "$HOST_PY_NUM" ] && PYTHON_MATCH="YES"
118+
119+
# Compare Node (extract version number - already extracted as v\d+.\d+.\d+)
120+
HOST_NODE_NUM=$(echo "$HOST_NODE_VERSION" | grep -oP 'v\d+\.\d+\.\d+' || echo "")
121+
CHROOT_NODE_NUM="$CHROOT_NODE"
122+
[ "$HOST_NODE_NUM" = "$CHROOT_NODE_NUM" ] && [ -n "$HOST_NODE_NUM" ] && NODE_MATCH="YES"
123+
124+
# Compare Go (extract version number - chroot already extracted as "goX.Y.Z")
125+
HOST_GO_NUM=$(echo "$HOST_GO_VERSION" | grep -oP 'go\d+\.\d+(\.\d+)?' || echo "")
126+
CHROOT_GO_NUM="$CHROOT_GO"
127+
[ "$HOST_GO_NUM" = "$CHROOT_GO_NUM" ] && [ -n "$HOST_GO_NUM" ] && GO_MATCH="YES"
128+
129+
# Create results summary
130+
{
131+
echo "PYTHON_MATCH=$PYTHON_MATCH"
132+
echo "NODE_MATCH=$NODE_MATCH"
133+
echo "GO_MATCH=$GO_MATCH"
134+
echo "HOST_PY_NUM=$HOST_PY_NUM"
135+
echo "CHROOT_PY_NUM=$CHROOT_PY_NUM"
136+
echo "HOST_NODE_NUM=$HOST_NODE_NUM"
137+
echo "CHROOT_NODE_NUM=$CHROOT_NODE_NUM"
138+
echo "HOST_GO_NUM=$HOST_GO_NUM"
139+
echo "CHROOT_GO_NUM=$CHROOT_GO_NUM"
140+
} > /tmp/gh-aw/chroot-test/results.env
141+
142+
cat /tmp/gh-aw/chroot-test/results.env
143+
144+
# Determine overall result
145+
if [ "$PYTHON_MATCH" = "YES" ] && [ "$NODE_MATCH" = "YES" ] && [ "$GO_MATCH" = "YES" ]; then
146+
echo "ALL_TESTS_PASSED=true" >> /tmp/gh-aw/chroot-test/results.env
147+
echo "=== ALL CHROOT TESTS PASSED ==="
148+
else
149+
echo "ALL_TESTS_PASSED=false" >> /tmp/gh-aw/chroot-test/results.env
150+
echo "=== SOME CHROOT TESTS FAILED ==="
151+
fi
152+
- name: Cleanup test containers
153+
if: always()
154+
run: |
155+
./scripts/ci/cleanup.sh || true
54156
---
55157

56-
# Verify Language Runtimes Match Host
57-
58-
This smoke test validates that `--enable-chroot` provides transparent access to host binaries by comparing versions.
59-
60-
## Step 1: Read Host Versions
158+
# Analyze Chroot Test Results
61159

62-
First, read the host versions that were captured in the setup step:
160+
The chroot version comparison tests have already been executed in the setup steps. Your job is to analyze the results and report them.
63161

64-
```bash
65-
cat /tmp/host-versions.env
66-
```
67-
68-
## Step 2: Run Tests via AWF Chroot
162+
## Step 1: Read Test Results
69163

70-
Run the same version commands through `awf --enable-chroot` and verify they match:
164+
Read the test results from the files created during setup:
71165

72166
```bash
73-
# Test Python version matches host
74-
sudo -E awf --enable-chroot --allow-domains localhost -- python3 --version
75-
76-
# Test Node version matches host
77-
sudo -E awf --enable-chroot --allow-domains localhost -- node --version
78-
79-
# Test Go version matches host
80-
sudo -E awf --enable-chroot --allow-domains localhost -- go version
167+
cat /tmp/gh-aw/chroot-test/host-versions.env
168+
cat /tmp/gh-aw/chroot-test/chroot-versions.env
169+
cat /tmp/gh-aw/chroot-test/results.env
81170
```
82171

83-
## Step 3: Verify Versions Match
172+
## Step 2: Create Summary Comment
84173

85-
Compare the versions from chroot with the host versions from `/tmp/host-versions.env`.
174+
Based on the results, add a comment to the PR with a comparison table:
86175

87-
Create a summary table showing:
88176
| Runtime | Host Version | Chroot Version | Match? |
89177
|---------|--------------|----------------|--------|
178+
| Python | (from HOST_PY_NUM) | (from CHROOT_PY_NUM) | (PYTHON_MATCH) |
179+
| Node.js | (from HOST_NODE_NUM) | (from CHROOT_NODE_NUM) | (NODE_MATCH) |
180+
| Go | (from HOST_GO_NUM) | (from CHROOT_GO_NUM) | (GO_MATCH) |
181+
182+
## Step 3: Add Label if Passed
90183

91-
If ALL versions match, the test passes. Add a comment to the PR with the comparison table.
184+
If ALL_TESTS_PASSED is true, add the `smoke-chroot` label to the PR.
92185

93-
If all runtimes match, add the label `smoke-chroot`.
186+
Keep your comment brief and focused on the results.

0 commit comments

Comments
 (0)