test(connect): black-box bats regression suite for connect*.sh + CI

[juan-malbeclabs]

What

Adds behavioral regression tests for the scripts/connect*.sh installers, plus a CI workflow to run them.

The tests drive the byte-identical shipped installers end-to-end through a stub-first PATH (fake docker/sudo/ss/curl/python3/...) and assert on what each script tried to do (the argv handed to the docker stub, its exit status). No source guard, refactor, or test-only seam is added to the files served over the CDN via curl | bash.

Why

The installers carry non-trivial logic and ship verbatim to users. #70 was a silent regression: under set -e, a bare port_in_use "$p" returning non-zero (the common case — the WS port is free) tripped errexit and aborted the whole installer before it started the container. Nothing caught it (no shell tests; shellcheck doesn't reliably flag that set -e footgun).

Contents

• tests/scripts/_helpers.bash — stub-PATH harness: docker argv capture, free/busy ss toggle, hermetic offline stubs, and helpers that iterate all three installers so per-script drift is caught.
• tests/scripts/set_e_regression.bats — pins the fix(connect): WS port preflight aborts under set -e when the port is free #70 fix: with the WS port free (and busy, non-interactive) the installer must survive preflight and reach docker run. Fails against the pre-fix(connect): WS port preflight aborts under set -e when the port is free #70 code, passes against the fix.
• .github/workflows/shell-tests.yml — installs bats and runs bats tests/scripts/ on PRs and main/hotfix pushes. Runs unconditionally (not paths-filtered) so it is safe as a required check, mirroring rust.yml and the reasoning documented in actionlint.yml.

Verification

• bats tests/scripts/ → both tests pass across all three installers (connect, connect-testnet, connect-devnet).
• Reintroducing the pre-fix(connect): WS port preflight aborts under set -e when the port is free #70 bug into a copy makes the suite fail, confirming it catches the regression.
• Iterating all three surfaced real drift now handled by the harness (e.g. connect-devnet.sh requires DZ_GHCR_TOKEN where the others don't).

Follow-ups (not in this PR)

The remaining coverage buckets from the design doc — ws_port.bats (preflight decision table + build_env_args via docker-argv capture) and accesspass.bats (embedded Python checker with the real python3 + mock RPC). The workflow globs tests/scripts/*.bats, so they're picked up automatically when added.

[ben-dz]

Test-only + CI + docs PR adding black-box bats regression tests for the connect*.sh installers. Verified locally: the suite passes across all three scripts and genuinely catches the #70 regression (reintroducing the pre-#70 bug makes it fail). The harness is hermetic and the CI workflow is least-privilege and safe. No critical/high issues; no merge-safety concern. NOTE: a two-dot git diff main HEAD in a shallow checkout makes it look like the PR reverts #71 (cee8d00) by removing ~33 lines from each installer — this is a diff-base artifact; the PR's own commits touch no installer script and a 3-way merge preserves #71. The PR description's 'no runtime code changed' is accurate. Findings are Medium/Low: (M1) the committed design doc describes a source-guard+refactor approach and ~3x the coverage that shipped, contradicting the black-box approach actually taken; (M2) the busy-port test is a near-duplicate of the free-port test and can't fail for its stated reason (doesn't assert port detection fired); plus low-severity polish (first-failure short-circuit, unquoted heredoc in ss_reports_busy, naming).

• M1 (medium) — docs/superpowers/specs/2026-07-06-connect-script-unit-tests-design.md contradicts the implementation. The committed 'approved' design mandates a source guard inside each script plus function extraction (dz_token_to_json, build_env_args) — i.e. editing the shipped installers — and lists six test files / four coverage buckets. The PR deliberately did the opposite (black-box, byte-identical scripts, no guard/refactor) and shipped only set_e_regression.bats + _helpers.bash. A committed spec that contradicts its own implementation misleads the next maintainer and invites re-introducing the CDN-served source guard the PR intentionally avoided. Add a 'Superseded — implemented as black-box, see PR #73' note or update the doc.
• L5 (low) — Naming: set_e_regression.bats's second test is a general preflight-continues case, not a set -e case; if the suite stays black-box, preflight_ws_port.bats would read more accurately. Also the .bats/.bash files are mode 0664 (non-executable) despite the bats shebang — harmless, just inconsistent.

[juan-malbeclabs]

Thanks for the review. Addressed all findings in the latest push (rebased on main):

M1 — docs/superpowers/specs/2026-07-06-connect-script-unit-tests-design.md is now marked Status: SUPERSEDED, with a note up top that the shipped suite is black-box by design (byte-identical installers, no source guard / no dz_token_to_json/build_env_args extraction) and an explicit "do not reintroduce the CDN-served source guard" warning. The old proposal is kept only for history.

M2 — the busy-port test now captures stderr and asserts the already in use warn, which is emitted only from the conflict branch. It no longer passes merely because docker run is reached (both paths reach it under DZ_ASSUME_YES=1); it pins that preflight actually detected the bound port. Verified it bites: with ss reporting the port free while it's treated as busy, the warn is absent and the test fails.

L5 — renamed set_e_regression.bats -> preflight_ws_port.bats (it covers the whole WS-port preflight path, not just the set -e case), and chmod +x'd the .bats/.bash files to match their shebang.

Polish — the ss_reports_busy heredoc is now quoted (port passed to the stub via an exported env var at runtime), and both tests collect failures across all three scripts instead of short-circuiting on the first.

Good call on the two-dot git diff diff-base artifact — confirmed the PR's own commits touch no installer script.