|
| 1 | +--- |
| 2 | +title: Scenario stale-port cleanup must not kill unrelated listeners |
| 3 | +date: 2026-04-25 |
| 4 | +category: test-failures |
| 5 | +module: scenarios |
| 6 | +problem_type: test_failure |
| 7 | +component: tooling |
| 8 | +symptoms: |
| 9 | + - "`bun run scenario:test -- all` dies with SIGKILL after the first scenario" |
| 10 | + - "A single scenario like `bun run scenario:test -- expo` passes" |
| 11 | + - "Running `expo` then `expo-auth` in one process dies between scenarios" |
| 12 | +root_cause: logic_error |
| 13 | +resolution_type: code_fix |
| 14 | +severity: high |
| 15 | +tags: |
| 16 | + - scenarios |
| 17 | + - runtime |
| 18 | + - cleanup |
| 19 | + - lsof |
| 20 | + - sigkill |
| 21 | +--- |
| 22 | + |
| 23 | +# Scenario stale-port cleanup must not kill unrelated listeners |
| 24 | + |
| 25 | +## Problem |
| 26 | + |
| 27 | +The aggregate scenario runtime gate can kill itself between scenarios when |
| 28 | +stale prepared apps point at shared local ports. The bug hides in cleanup, so |
| 29 | +single-scenario proof can pass while `scenario:test -- all` dies. |
| 30 | + |
| 31 | +## Symptoms |
| 32 | + |
| 33 | +- `bun run scenario:test -- all` exits with SIGKILL immediately after the |
| 34 | + first scenario reaches ready. |
| 35 | +- `bun run scenario:test -- expo` and `bun run scenario:test -- expo-auth` |
| 36 | + both pass on their own. |
| 37 | +- A two-scenario repro logs `AFTER expo`, then dies before `expo-auth` starts. |
| 38 | + |
| 39 | +## What Didn't Work |
| 40 | + |
| 41 | +- Treating the failure as memory pressure was too vague. The machine had other |
| 42 | + stale processes, but the first scenario passed reliably on its own. |
| 43 | +- Rerunning the full gate without isolating the transition only repeated the |
| 44 | + SIGKILL. |
| 45 | +- Looking only at dev server shutdown missed the stale prepared-app cleanup |
| 46 | + that runs before the next scenario is prepared. |
| 47 | + |
| 48 | +## Solution |
| 49 | + |
| 50 | +Constrain stale-port cleanup to processes that are actually owned by the |
| 51 | +prepared scenario project. |
| 52 | + |
| 53 | +Before this fix, `stopLocalConvexBackendForProject()` read a port from the old |
| 54 | +project's `.env.local`, ran `lsof -ti tcp:<port>`, and sent `kill -9` to every |
| 55 | +listener. That was too broad because scenario ports are shared across prepared |
| 56 | +apps. |
| 57 | + |
| 58 | +The fixed flow checks each candidate listener's cwd and only kills it when the |
| 59 | +process cwd is inside the target scenario project: |
| 60 | + |
| 61 | +```ts |
| 62 | +export const isProcessOwnedByProject = (pid: string, projectDir: string) => { |
| 63 | + const result = Bun.spawnSync({ |
| 64 | + cmd: ["lsof", "-a", "-p", pid, "-d", "cwd", "-Fn"], |
| 65 | + stdin: "ignore", |
| 66 | + stdout: "pipe", |
| 67 | + stderr: "ignore", |
| 68 | + }); |
| 69 | + |
| 70 | + const resolvedProjectDir = path.resolve(projectDir); |
| 71 | + return result.stdout |
| 72 | + .toString() |
| 73 | + .split(/\r?\n/) |
| 74 | + .filter((line) => line.startsWith("n")) |
| 75 | + .some((line) => { |
| 76 | + const cwd = path.resolve(line.slice(1)); |
| 77 | + return ( |
| 78 | + cwd === resolvedProjectDir || |
| 79 | + cwd.startsWith(`${resolvedProjectDir}${path.sep}`) |
| 80 | + ); |
| 81 | + }); |
| 82 | +}; |
| 83 | +``` |
| 84 | + |
| 85 | +Add a regression with an unrelated child listener on the stale port. Cleanup |
| 86 | +must leave that process alive. |
| 87 | + |
| 88 | +## Why This Works |
| 89 | + |
| 90 | +The cleanup intent is to stop stale processes from the prepared scenario app, |
| 91 | +not to own the whole machine's port table. Filtering by cwd preserves that |
| 92 | +intent while avoiding a broad `kill -9` against any process that happens to use |
| 93 | +the same localhost port. |
| 94 | + |
| 95 | +## Prevention |
| 96 | + |
| 97 | +- Never kill by port alone in scenario tooling. |
| 98 | +- For temp-app cleanup, prove both sides: stale project-owned listeners are |
| 99 | + eligible, unrelated listeners are not. |
| 100 | +- When `scenario:test -- all` fails but individual scenarios pass, debug the |
| 101 | + transition between scenarios before blaming the scenario itself. |
| 102 | + |
| 103 | +## Related Issues |
| 104 | + |
| 105 | +- [Scenario dev needed a Vite frontend split and React 18-safe client build](../integration-issues/scenario-vite-dev-split-and-react18-runtime-20260322.md) |
0 commit comments