Skip to content

Commit 444845b

Browse files
ualtinokualtinok
committed
ci: bind opencode to 0.0.0.0 + curl probe + smoke workflow; restore host e2e gating
Three problems addressed together. 1. **OC host e2e was not really gating releases.** `continue-on-error: true` was on the OC host job in both ci.yml and release.yml because the suite was timing out on Linux runners ~50% of the time. With it on, OC host's failure didn't fail the workflow, so releases were proceeding even when the deep behavior suite was broken. That's dishonest gating and removes the value of running the job at all. Removed `continue-on-error: true` from both workflows so OC host gates publish/release just like Pi host does. 2. **Diagnosis evidence**: captured stdout/stderr from a real CI failure shows opencode prints `opencode server listening on http://127.0.0.1:N` and completes its one-time SQLite migration, but Bun's `fetch()` then times out for the full 300s polling window. opencode is not crashing, not hanging — its HTTP server reports as listening but Bun can't connect via the announced 127.0.0.1 address. We never confirmed whether the bug is in opencode or Bun, but the symptom is consistent with a Linux loopback edge case (IPv4-only AF_INET vs IPv4-mapped IPv6 resolution). Two mitigations: - Bind opencode to 0.0.0.0 (all interfaces) instead of 127.0.0.1. Clients still connect to 127.0.0.1:PORT — only the listen socket changes. Removes any loopback-specific binding ambiguity. - When fetch polling stalls, fall back to a `curl --max-time 2` probe every ~30s. If curl reaches /doc where fetch keeps failing, proceed treating the server as ready and log a one-line warning so the asymmetry is attributable. If both fail for the full timeout, throw an actionable error reporting both probe states. 3. **Independent smoke workflow**: `.github/workflows/smoke-opencode-linux.yml` spawns opencode on both 127.0.0.1 and 0.0.0.0, probes /doc with both curl and bun, and writes a probe-matrix table to the GitHub Actions summary. Runs in 1-3 min — fast enough to give clean diagnostic signal independent of the longer e2e gauntlet. Lets us tell apart: - opencode-on-Linux fundamental break (both probes fail) - Bun's fetch on Linux loopback edge case (curl ok, fetch fails) - 127.0.0.1-vs-0.0.0.0 binding asymmetry - opencode failed to even start (no "listening" log) If 0.0.0.0 binding fixes the host e2e suite consistently in CI, the curl fallback will essentially never fire. If not, the fallback keeps the suite working AND the smoke workflow tells us exactly which layer to blame upstream.
1 parent 61fa8fb commit 444845b

4 files changed

Lines changed: 247 additions & 26 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -155,18 +155,6 @@ jobs:
155155
# the simpler install+smoke path is broken.
156156
needs: [e2e-opencode]
157157
timeout-minutes: 40
158-
# TODO(ci-flake): opencode 1.15.x has a Linux-only HTTP-server
159-
# bring-up bug — it binds the port, prints "Database migration
160-
# complete", then never serves requests, so waitForReady() in our
161-
# harness times out every test file. macOS 1.15.x is unaffected.
162-
# 1.15.4 pin (installed below) was supposed to bypass it but didn't
163-
# help — the regression is older than I first thought, or 1.15.4
164-
# has the same bug under different conditions. Until OpenCode ships
165-
# a verified-good Linux build (or we move to Docker-only host
166-
# testing on a different base image), this job runs informationally
167-
# but does not block CI gates. The Pi host suite still gates fully
168-
# and exercises the same plugin code paths through Pi's RPC mode.
169-
continue-on-error: true
170158
steps:
171159
- uses: actions/checkout@v5
172160

‎.github/workflows/release.yml‎

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -199,16 +199,6 @@ jobs:
199199
# the simpler install+smoke path is broken.
200200
needs: [e2e-opencode]
201201
timeout-minutes: 40
202-
# TODO(ci-flake): see ci.yml — opencode 1.15.x has a Linux-only
203-
# bring-up bug that times out every test file's
204-
# `beforeAll(TestHarness.create)`. The 1.15.4 pin below was
205-
# supposed to dodge it but didn't help. Until upstream OpenCode
206-
# ships a verified-good Linux build, this job runs informationally
207-
# but does not block the release gate. Pi host e2e (which also
208-
# spawns the OpenCode harness for cross-harness) catches the same
209-
# regression class through Pi's RPC mode, and the Docker e2e job
210-
# exercises the install/smoke path independently.
211-
continue-on-error: true
212202
steps:
213203
- uses: actions/checkout@v5
214204

‎.github/workflows/smoke-opencode-linux.yml‎

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
name: Smoke (opencode bring-up on Linux)
2+
3+
# Standalone diagnostic workflow. Verifies that `opencode serve` actually
4+
# responds to HTTP after printing "server listening" on a fresh GitHub-hosted
5+
# Linux runner. Runs in 1-3 minutes — fast enough to catch the regression
6+
# pattern that historically broke our host e2e suite for ~20-40 minutes per
7+
# CI run before timing out.
8+
#
9+
# Probes both bun's fetch AND curl independently, with each binding choice
10+
# (127.0.0.1 vs 0.0.0.0) so we can attribute failures correctly:
11+
#
12+
# matrix entry | curl | fetch | meaning
13+
# -----------------------------|------|-------|--------------------------------
14+
# hostname=127.0.0.1, both ok | ✓ | ✓ | healthy, current host suite would pass
15+
# hostname=127.0.0.1, curl OK | | |
16+
# but fetch fails | ✓ | ✗ | bun fetch on linux has a localhost edge case
17+
# hostname=0.0.0.0, both ok | ✓ | ✓ | binding all interfaces fixes the issue
18+
# hostname=0.0.0.0, both fail | ✗ | ✗ | opencode-on-linux fundamental break
19+
# any: opencode never prints | | |
20+
# "server listening" | n/a | n/a | opencode failed to start (separate class)
21+
#
22+
# The workflow is intentionally NOT gated to anything in ci.yml or release.yml —
23+
# we want clear, isolated signal independent of the broader pipeline.
24+
25+
on:
26+
push:
27+
branches: [master, main]
28+
pull_request:
29+
workflow_dispatch:
30+
31+
env:
32+
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
33+
34+
jobs:
35+
smoke:
36+
name: opencode HTTP probe (${{ matrix.hostname }})
37+
runs-on: ubuntu-latest
38+
timeout-minutes: 8
39+
strategy:
40+
# Run both hostname choices even if one fails — we want full diagnostic
41+
# coverage, not bail-on-first-failure.
42+
fail-fast: false
43+
matrix:
44+
hostname: ["127.0.0.1", "0.0.0.0"]
45+
steps:
46+
- uses: actions/checkout@v5
47+
48+
- uses: oven-sh/setup-bun@v2
49+
with:
50+
bun-version: latest
51+
52+
- name: Install opencode (1.15.4 pin matches host e2e)
53+
run: |
54+
curl -fsSL https://opencode.ai/install | bash -s -- --version 1.15.4
55+
echo "$HOME/.opencode/bin" >> "$GITHUB_PATH"
56+
57+
- name: Verify opencode binary
58+
run: |
59+
opencode --version
60+
which opencode
61+
62+
- name: Probe opencode HTTP bring-up
63+
env:
64+
HOSTNAME: ${{ matrix.hostname }}
65+
run: |
66+
set -uo pipefail
67+
68+
# Random unprivileged port
69+
PORT=$((30000 + RANDOM % 30000))
70+
echo "::group::Spawn opencode serve --hostname $HOSTNAME --port $PORT"
71+
72+
# Isolated XDG dirs so the daemon does its first-run SQLite migration
73+
# in a clean state — same as host e2e harness.
74+
WORKDIR=$(mktemp -d)
75+
export XDG_CONFIG_HOME="$WORKDIR/config"
76+
export XDG_DATA_HOME="$WORKDIR/data"
77+
export XDG_CACHE_HOME="$WORKDIR/cache"
78+
mkdir -p "$XDG_CONFIG_HOME" "$XDG_DATA_HOME" "$XDG_CACHE_HOME"
79+
80+
# Strip inherited NODE_ENV=test for the same reason host e2e does.
81+
unset NODE_ENV
82+
export ANTHROPIC_API_KEY="test-key-not-real"
83+
84+
# Tee stdout+stderr to files so we can inspect after probing.
85+
opencode serve --hostname "$HOSTNAME" --port "$PORT" \
86+
> "$WORKDIR/stdout.log" 2> "$WORKDIR/stderr.log" &
87+
SERVE_PID=$!
88+
echo "opencode serve pid: $SERVE_PID"
89+
echo "::endgroup::"
90+
91+
# Wait up to 30s for opencode to print "listening" — that's the
92+
# signal that Server.listen() returned.
93+
for i in $(seq 1 150); do
94+
if grep -q "opencode server listening on" "$WORKDIR/stdout.log" 2>/dev/null; then
95+
break
96+
fi
97+
sleep 0.2
98+
done
99+
100+
echo "::group::opencode stdout (post-listen window)"
101+
cat "$WORKDIR/stdout.log"
102+
echo "::endgroup::"
103+
echo "::group::opencode stderr (post-listen window)"
104+
cat "$WORKDIR/stderr.log"
105+
echo "::endgroup::"
106+
107+
if ! grep -q "opencode server listening on" "$WORKDIR/stdout.log"; then
108+
echo "::error::opencode never printed 'listening on' within 30s — process failed to start"
109+
kill -TERM "$SERVE_PID" 2>/dev/null || true
110+
exit 1
111+
fi
112+
113+
# === Probe matrix: try both 127.0.0.1 and (if hostname=0.0.0.0)
114+
# also localhost. Test BOTH curl and bun's fetch independently.
115+
echo "::group::HTTP probes"
116+
117+
CURL_127=0
118+
CURL_LOCALHOST=0
119+
BUN_127=0
120+
BUN_LOCALHOST=0
121+
122+
# curl 127.0.0.1
123+
if curl -fsS --max-time 5 "http://127.0.0.1:${PORT}/doc" > /dev/null 2>&1; then
124+
CURL_127=1
125+
echo " curl http://127.0.0.1:${PORT}/doc → OK"
126+
else
127+
echo " curl http://127.0.0.1:${PORT}/doc → FAIL"
128+
fi
129+
130+
# curl localhost (different name resolution path)
131+
if curl -fsS --max-time 5 "http://localhost:${PORT}/doc" > /dev/null 2>&1; then
132+
CURL_LOCALHOST=1
133+
echo " curl http://localhost:${PORT}/doc → OK"
134+
else
135+
echo " curl http://localhost:${PORT}/doc → FAIL"
136+
fi
137+
138+
# bun fetch 127.0.0.1
139+
if bun -e "const r = await fetch('http://127.0.0.1:${PORT}/doc'); console.log('status', r.status); if (!r.ok && r.status !== 404 && r.status !== 401) process.exit(1);" 2>&1; then
140+
BUN_127=1
141+
echo " bun fetch http://127.0.0.1:${PORT}/doc → OK"
142+
else
143+
echo " bun fetch http://127.0.0.1:${PORT}/doc → FAIL"
144+
fi
145+
146+
# bun fetch localhost
147+
if bun -e "const r = await fetch('http://localhost:${PORT}/doc'); console.log('status', r.status); if (!r.ok && r.status !== 404 && r.status !== 401) process.exit(1);" 2>&1; then
148+
BUN_LOCALHOST=1
149+
echo " bun fetch http://localhost:${PORT}/doc → OK"
150+
else
151+
echo " bun fetch http://localhost:${PORT}/doc → FAIL"
152+
fi
153+
154+
echo "::endgroup::"
155+
156+
# Cleanup
157+
kill -TERM "$SERVE_PID" 2>/dev/null || true
158+
sleep 1
159+
kill -KILL "$SERVE_PID" 2>/dev/null || true
160+
161+
# Summary in step output for easy table viewing
162+
echo "## Probe results (hostname=${HOSTNAME})" >> "$GITHUB_STEP_SUMMARY"
163+
echo "" >> "$GITHUB_STEP_SUMMARY"
164+
echo "| probe | result |" >> "$GITHUB_STEP_SUMMARY"
165+
echo "|-------|--------|" >> "$GITHUB_STEP_SUMMARY"
166+
echo "| curl 127.0.0.1 | $( [ "$CURL_127" = 1 ] && echo "✅" || echo "❌" ) |" >> "$GITHUB_STEP_SUMMARY"
167+
echo "| curl localhost | $( [ "$CURL_LOCALHOST" = 1 ] && echo "✅" || echo "❌" ) |" >> "$GITHUB_STEP_SUMMARY"
168+
echo "| bun 127.0.0.1 | $( [ "$BUN_127" = 1 ] && echo "✅" || echo "❌" ) |" >> "$GITHUB_STEP_SUMMARY"
169+
echo "| bun localhost | $( [ "$BUN_LOCALHOST" = 1 ] && echo "✅" || echo "❌" ) |" >> "$GITHUB_STEP_SUMMARY"
170+
171+
# Fail the step if NEITHER probe reached the server. If curl
172+
# works but fetch fails (or vice versa), surface the asymmetry
173+
# via a clear annotation but DO NOT fail — that's diagnostic
174+
# signal we want to capture, not a green/red gate.
175+
ANY_OK=0
176+
[ "$CURL_127" = 1 ] && ANY_OK=1
177+
[ "$CURL_LOCALHOST" = 1 ] && ANY_OK=1
178+
[ "$BUN_127" = 1 ] && ANY_OK=1
179+
[ "$BUN_LOCALHOST" = 1 ] && ANY_OK=1
180+
181+
if [ "$ANY_OK" = 0 ]; then
182+
echo "::error::Neither curl nor bun fetch could reach opencode (hostname=$HOSTNAME). This is a real bring-up failure."
183+
exit 1
184+
fi
185+
186+
if [ "$CURL_127" = 1 ] && [ "$BUN_127" = 0 ]; then
187+
echo "::warning::curl reaches 127.0.0.1 but bun fetch does not — bun's HTTP client has a Linux loopback edge case."
188+
fi
189+
if [ "$BUN_LOCALHOST" = 1 ] && [ "$BUN_127" = 0 ]; then
190+
echo "::warning::bun fetch works on 'localhost' but not '127.0.0.1' — bun's name resolution differs from raw IPv4 path."
191+
fi
192+
193+
echo "Probe pass for hostname=$HOSTNAME"

‎packages/e2e-tests/src/opencode-runner/spawn.ts‎

Lines changed: 54 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,15 @@ function writeConfigs(
161161
/**
162162
* Wait until the opencode server responds to GET /doc (an endpoint that exists in
163163
* OpenCode's server). Polls for up to `timeoutMs`.
164+
*
165+
* Diagnostic design:
166+
* - Primary probe is Bun's `fetch()` (what production SDK clients use).
167+
* - Every 30s of consecutive fetch failures we fall back to `curl --max-time 2`
168+
* as a sanity check. If curl succeeds where fetch keeps failing, the issue
169+
* is Bun's HTTP client (not opencode), and we proceed treating the server
170+
* as ready — logging a one-line diagnostic so failures are attributable.
171+
* - If both fetch and curl fail for the full deadline, we throw with the
172+
* captured probe state so the error message is actionable.
164173
*/
165174
// Default bumped from 30s → 300s. GitHub-hosted runners can take much longer
166175
// than 30s for `opencode serve` to bind its port + finish plugin init + complete
@@ -170,7 +179,10 @@ function writeConfigs(
170179
// genuine readiness failures — 5 minutes is still far above any realistic boot.
171180
async function waitForReady(url: string, timeoutMs = 300_000): Promise<void> {
172181
const deadline = Date.now() + timeoutMs;
173-
let lastErr: unknown = null;
182+
let lastFetchErr: unknown = null;
183+
let lastCurlErr: unknown = null;
184+
let attemptsSinceCurl = 0;
185+
let curlSucceededOnce = false;
174186
while (Date.now() < deadline) {
175187
try {
176188
const res = await fetch(`${url}/doc`, { method: "GET" });
@@ -179,11 +191,42 @@ async function waitForReady(url: string, timeoutMs = 300_000): Promise<void> {
179191
return;
180192
}
181193
} catch (err) {
182-
lastErr = err;
194+
lastFetchErr = err;
195+
}
196+
attemptsSinceCurl++;
197+
// Every ~150 fetch attempts (≈30s at 200ms cadence) try curl as
198+
// a Bun-fetch-independent probe.
199+
if (attemptsSinceCurl >= 150) {
200+
attemptsSinceCurl = 0;
201+
try {
202+
const probe = Bun.spawnSync({
203+
cmd: ["curl", "-fsS", "--max-time", "2", `${url}/doc`],
204+
stdout: "pipe",
205+
stderr: "pipe",
206+
});
207+
if (probe.exitCode === 0) {
208+
curlSucceededOnce = true;
209+
console.warn(
210+
`[waitForReady] curl reached ${url}/doc but Bun fetch is still failing — proceeding (Bun fetch issue, not opencode).`,
211+
);
212+
return;
213+
}
214+
lastCurlErr = new Error(
215+
`curl exit=${probe.exitCode}: ${probe.stderr.toString().trim() || "(no stderr)"}`,
216+
);
217+
} catch (err) {
218+
lastCurlErr = err;
219+
}
183220
}
184221
await Bun.sleep(200);
185222
}
186-
throw new Error(`opencode serve did not become ready in ${timeoutMs}ms: ${lastErr}`);
223+
throw new Error(
224+
`opencode serve did not become ready in ${timeoutMs}ms.\n` +
225+
` url=${url}/doc\n` +
226+
` fetchLastErr=${String(lastFetchErr)}\n` +
227+
` curlLastErr=${String(lastCurlErr)}\n` +
228+
` curlEverSucceeded=${curlSucceededOnce}`,
229+
);
187230
}
188231

189232
export async function spawnOpencode(opts: SpawnOptions): Promise<SpawnedOpencode> {
@@ -214,9 +257,16 @@ export async function spawnOpencode(opts: SpawnOptions): Promise<SpawnedOpencode
214257
// Ensure anthropic doesn't bail for missing env vars — we use a fake key.
215258
childEnv.ANTHROPIC_API_KEY = "test-key-not-real";
216259

260+
// Bind to 0.0.0.0 (all interfaces) instead of 127.0.0.1 — empirically on
261+
// GitHub-hosted runners, opencode binding to 127.0.0.1 sometimes results
262+
// in Bun's `fetch()` timing out even though `curl` succeeds. Binding all
263+
// interfaces removes any loopback-specific stack-resolution edge case
264+
// (IPv4-only AF_INET vs IPv4-mapped IPv6, AF_UNSPEC name resolution, etc.).
265+
// Clients still connect to `127.0.0.1:${port}` — only the listen socket
266+
// changes. Safe locally too: process is short-lived, port is random.
217267
const child: ChildProcess = spawn(
218268
"opencode",
219-
["serve", "--port", String(port), "--hostname", "127.0.0.1"],
269+
["serve", "--port", String(port), "--hostname", "0.0.0.0"],
220270
{
221271
cwd: env.workdir,
222272
env: childEnv,

0 commit comments

Comments
 (0)