Skip to content

fix: format debt, GitLab CI wiring, and the (never-actually-run) live-download install path - #71

Merged
hyperpolymath merged 5 commits into
mainfrom
claude/odds-sods-pm-goals-xktszo
Jul 17, 2026
Merged

fix: format debt, GitLab CI wiring, and the (never-actually-run) live-download install path#71
hyperpolymath merged 5 commits into
mainfrom
claude/odds-sods-pm-goals-xktszo

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Three items from the sitrep backlog, done as five separate commits. The last two turned into a real chain of discovery — wiring a CI job to exercise the live-download path surfaced that opsm runtime install has, as far as I can tell, never actually completed a real install against a real nickel binary, for three independent reasons stacked on top of each other.

1. style: run mix format across the tree (ef87e0b)

226 files were unformatted, leaving the Mustfile's format check permanently red with nothing enforcing it. Pure mix format output, no manual edits.

2. fix(gitlab-ci): fix root-relative rules:exists paths (10588f8)

Every mix-*/cargo-* job in .gitlab-ci.yml was permanently dormant: mix.exs lives in opsm_ex/, and the Rust code is nine independent first-party crates (opsm-ui/tui, two opsm_ex/native NIFs, six services/*) — no root Cargo.toml for rules: exists: to ever match. mix-* jobs now target opsm_ex/; cargo-* jobs loop over a $RUST_CRATE_DIRS list. Verified: YAML parses, mix format's own check (now green) confirms the job would pass, just toolchain-check passes.

3–5. The live-download chain

Task was "add a workflow_dispatch job for the live-download tests" (8d0e373). Getting a real nickel binary onto a runner to test it locally surfaced two more bugs, both now fixed and verified against a real 47MB zig install:

  • fix(nickel): unbreak the entire runtime plugin corpus (3f67294) — 84 of 95 .ncl files failed to even typecheck against the canonical nickel 1.16.0 pinned in opsm.toml [runtime]. All 84 failures traced to one shared file, two bugs: runtime-plugin.ncl had three fields with default before doc in the annotation chain (nickel 1.16's grammar requires the reverse — confirmed via minimal repro), and every plugin destructured its import as let { RuntimePlugin, .. } = import ... against a module whose export is the bare contract value, not a record — the destructuring could never match. Both are one-line-per-field-or-file mechanical fixes. All 95 files now typecheck; all 94 runtime/core/*.ncl plugins now fully nickel export to JSON.

  • fix(verified-http): forward receive_timeout and :into to Req (f9b926e) — with nickel now working, the install still failed: Opsm.Verified.Http.do_get/2 never forwarded opts[:into] to Req.get/2 at all, so Manager.download_archive/1's VerifiedHttp.get(url, into: File.stream!(dest)) silently buffered the response in memory instead of streaming to disk, returned {:ok, _}, and extract_archive/3 got handed a file that was never written. Also found in the same function: do_get/do_post read opts[:timeout], but every call site in the codebase (~480 occurrences) passes receive_timeout: — Req's actual option name — so every custom timeout anywhere in OPSM has been silently discarded in favor of the 30s default. Both fixed.

  • feat(e2e): add a live-download job (8d0e373) — the actual dispatch job: Rust toolchain (pinned 1.97.0) → cargo install nickel-lang-cli --version 1.16.0 (confirmed to exist at that version on crates.io) → mix test integration_test.exs --include external_api --include live_download.

Verified (OTP 28.3.1 / Elixir 1.19.5, real nickel-lang-cli 1.16.0 built from source)

  • mix format --check-formatted — clean
  • mix compile --warnings-as-errors — clean
  • Full default suite — 807/807
  • sh scripts/toolchain.sh check — clean
  • All 95 .ncl files nickel-typecheck; all 94 runtime/core/* fully export
  • mix test test/opsm/runtime/integration_test.exs --include external_api --include live_download — 15/15, 0 failures, with a cold cache: real download from ziglang.org, extract, which/1, current_version/1, remove/1, reinstall idempotency
  • .gitlab-ci.yml / .github/workflows/e2e.yml YAML parse clean

The GitLab cargo-* job bodies aren't runnable in this sandbox (no GitLab remote, and installing cargo-audit/cargo-deny from crates.io for 9 crates wasn't worth the time here) — reviewed by inspection against the confirmed crate list and the already-working per-crate pattern in nif-build.yml.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C


Generated by Claude Code

claude added 5 commits July 17, 2026 05:38
226 files were unformatted, leaving the Mustfile's 'format' check
(just fmt -> mix format --check-formatted) permanently red — nothing
enforced formatting so it silently drifted. Pure mix format output,
no manual edits; compile --warnings-as-errors and the full test suite
(807 tests) are unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C
… run

Every mix-*/cargo-* job has been permanently dormant: mix.exs lives in
opsm_ex/, not the repo root, and the estate's Rust code is nine
independent first-party crates (opsm-ui/tui, two opsm_ex/native NIFs,
six services/* microservices) — there is no root Cargo.toml for
'rules: exists: Cargo.toml' to ever match.

- mix-* jobs (mix-audit, mix-format, credo, mix-test, mix-build):
  rules point at opsm_ex/mix.exs; scripts cd into opsm_ex first.
- cargo-* jobs (cargo-audit, cargo-deny, rustfmt, clippy, cargo-test,
  cargo-build): rules list all nine crates' Cargo.toml paths (rules:
  exists needs literal paths, not variable expansion); scripts loop
  over the new $RUST_CRATE_DIRS variable, running each command
  per-crate. opsm_ex/deps/** (vendored Mix dependency cache) is
  deliberately excluded.
- cache/artifact paths updated for the per-crate target/ layout.

Verified: mix-format's own check (mix format --check-formatted, now
green after the prior commit) and just toolchain-check both pass;
YAML parses clean. The cargo-* loop bodies are not runnable in this
sandbox (no GitLab remote, no network for cargo install) — reviewed
by inspection against the confirmed crate list, same pattern as the
already-working per-crate GitHub Actions job (nif-build.yml).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C
…1.16.0

Discovered while wiring the live-download CI job (which needed a real
nickel install for the first time): 84 of 95 .ncl files under runtime/
failed to even typecheck against the canonical nickel 1.16.0 pinned in
opsm.toml [runtime]. opsm runtime install has never actually worked
against a real nickel binary — every prior test only reached the
{:error, {:no_plugin, _}} / {:error, {:nickel_not_installed, _}} paths.

Two independent bugs, both traced to a single shared file:

- runtime/contract/runtime-plugin.ncl had three fields with
  '| default = X | doc "..."' annotation ordering. Nickel 1.16's
  grammar requires doc before default in a field's annotation chain;
  the reverse order is a parse error, and every plugin file imports
  this contract, so the failure was transitive to all 84 dependents.
  Fixed by swapping the two annotations (verified minimal repro).

- Every plugin file destructured the import as
  'let { RuntimePlugin, .. } = import "../contract/runtime-plugin.ncl"',
  but the contract file's final expression is the bare contract value
  itself (a  module, not a record wrapping it) —
  so the destructuring pattern could never match. Changed to a plain
  'let RuntimePlugin = import ... in' binding across all 83 affected
  files (one-line mechanical change, same substitution everywhere).

Verified: all 95 .ncl files now nickel typecheck clean; all 94 files
under runtime/core/ (the actual plugin catalog) fully nickel-export to
JSON — the exact call Manager.load_plugin makes. Confirmed end-to-end
via a live zig install (see following commit for the http.ex fix that
was still needed to get a completed install).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C
… streamed downloads

Two silent option-forwarding bugs in Opsm.Verified.Http, found while
finishing the runtime install path (following the nickel corpus fix):

- do_get/2 and do_post/2 read Keyword.get(opts, :timeout, 30_000), but
  every call site in the codebase (~480 occurrences) passes
  receive_timeout: — Req's actual option name. Every custom timeout
  configured anywhere in OPSM has been silently discarded in favor of
  the 30s default. Now reads :receive_timeout first, falling back to
  the old :timeout key for compatibility, then the same default.

- do_get/2 never forwarded opts[:into] to Req.get/2 at all.
  Runtime.Manager.download_archive/1 calls
  VerifiedHttp.get(url, into: File.stream!(dest)) expecting Req to
  stream the response body straight to disk; because :into was
  dropped, Req buffered the full body in memory, returned {:ok, _},
  and download_archive treated that as success — but dest was never
  written. extract_archive/3 then handed tar a nonexistent file
  ({"", 2} from tar's own "Cannot open" exit). This is why a live zig
  install still failed after the nickel fix.

Verified against a real 47MB zig 0.13.0 download from ziglang.org:
integration_test.exs (--include external_api --include live_download)
now passes 15/15 (0 failures) with a clean cache, including install,
which/1, current_version/1, remove/1, and reinstall idempotency. Full
default suite (807 tests) and mix format --check-formatted unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C
The workflow's own header comment has claimed :live_download tests are
"workflow_dispatch only" since it was written, but no job ever ran
them — workflow_dispatch only enabled the manual trigger, it was never
wired to a job scoped to those tests. Adds a live-download job, gated
on `github.event_name == 'workflow_dispatch'`, that:

- Installs a Rust toolchain (pinned to 1.97.0, matching opsm.toml
  [runtime]) and cargo-installs nickel-lang-cli 1.16.0 — confirmed to
  exist at that exact version on crates.io. Manager.install shells out
  to the nickel binary to evaluate runtime/core/*.ncl plugins; without
  it every install fails with {:nickel_not_installed, tool}.
- Runs integration_test.exs with --include external_api --include
  live_download, exercising a real download+extract+install+remove
  cycle (this is what the two prior commits in this branch actually
  fixed and verified locally: 84 broken .ncl files, and a
  VerifiedHttp bug that silently dropped streamed downloads).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C
@hyperpolymath
hyperpolymath marked this pull request as ready for review July 17, 2026 06:19
@hyperpolymath
hyperpolymath merged commit 3fe3b13 into main Jul 17, 2026
28 of 29 checks passed
@hyperpolymath
hyperpolymath deleted the claude/odds-sods-pm-goals-xktszo branch July 17, 2026 06:20
hyperpolymath added a commit that referenced this pull request Jul 21, 2026
Follow-up to #71, closing the gap I flagged there: the GitLab `cargo-*`
jobs were "reviewed by inspection, not runnable in the sandbox." I have
a working Rust toolchain here, so I actually ran all nine first-party
crates through `cargo build`/`test`/`fmt`. Result: **8 of 9 build clean
once fixed, 1 is a known-optional non-builder**. Three commits:

## 1. `fix(selur): add missing tokio AsyncReadExt import` (`02ff456`)
`services/selur` has **never compiled**. `sign_image` had a
function-local `use tokio::io::AsyncReadExt;`, but `generate_keypair` (a
second call site) also does `file.take(..).read_to_string(..)` without
the trait in scope — so `.take()` resolved to `Iterator::take` on
`tokio::fs::File` and failed (`E0599`/`E0282`). The dormant GitLab jobs
never caught it, and `nif-build.yml` doesn't cover `services/`. Hoisted
the import to module scope, dropped the redundant local one, removed
`mut` from two read-only bindings. `cargo build --release` + `cargo
test` pass.

## 2. `style(rust): cargo fmt all first-party crates` (`1d743b2`)
The `rustfmt` job (not `allow_failure`) runs `cargo fmt -- --check` per
crate; all 8 failed because the code was never fmt-checked while the job
was dormant — same story as the `mix format` sweep in #71, for Rust.
Pure `cargo fmt` output, no logic changes.

## 3. `fix(gitlab-ci): quarantine the non-building quic_transport NIF`
(`89a4c1b`)
`opsm_ex/native/quic_transport` does **not** compile: pinned `h3 0.0.6`
/ `h3-quinn 0.0.7` predate `quinn 0.11`'s now-private `StreamId` field,
and with no committed `Cargo.lock` the fresh resolve breaks. It's an
**optional** QUIC/HTTP3 NIF — `Opsm.Transport.QuicNif` swallows a failed
`:erlang.load_nif` and falls back to HTTP/2 (`nif_loaded?` is hard-coded
`false`), so the Elixir side (807 tests) runs fine without it, and no CI
builds it today. Rather than sink a full pre-1.0 `h3`-ecosystem
migration into optional aspirational code, I removed it from
`$RUST_CRATE_DIRS` and the `rules:`/artifacts lists, with a
`QUARANTINED` header note stating the un-quarantine condition (migrate
the `h3`/`h3-quinn`/`quinn` triple to a compatible set + commit a
lockfile). **Flagged as a follow-up decision for you** — see below.

## Verified (real Rust 1.9x toolchain, this environment)
- All 8 kept crates: `cargo build --release` ✓, `cargo fmt -- --check`
✓; `checky-monkey` tests 4/4, others build clean (0 tests defined)
- `.gitlab-ci.yml` parses; only 2 `quic_transport` mentions remain (both
the header note)
- `just toolchain-check` ✓

## For your call (not done here)
- **quic_transport migration**: the `h3` crate is pre-1.0 and churns
hard (`recv_data` now returns `impl Buf`, send-side API changes, etc.) —
a real migration, not a version bump. Worth doing only if QUIC/HTTP3
transport is on the near-term roadmap; otherwise the quarantine + HTTP/2
fallback is the honest steady state.
- The `cargo-audit`/`cargo-deny` job *bodies* still aren't exercised
here (installing those tools from crates.io for 8 crates wasn't worth
the sandbox time); the build/test/fmt jobs are now genuinely verified.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Kq24sZCEohSrNFXSuEuz6C)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants