Skip to content

fix(launcher): match the ca-trust guard to what node's CA loader accepts - #296

Open
codeslake wants to merge 10 commits into
cnighswonger:mainfrom
codeslake:ca-trust-guard
Open

fix(launcher): match the ca-trust guard to what node's CA loader accepts#296
codeslake wants to merge 10 commits into
cnighswonger:mainfrom
codeslake:ca-trust-guard

Conversation

@codeslake

@codeslake codeslake commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #283. Same block, six defects the merged version shipped with — the guard it added disagrees with Node's own CA loader in both directions.

All measurements are on node v24.11.1 / openssl 3.6.1, against a real TLS handshake through NODE_EXTRA_CA_CERTS — not tls.connect({ca}), which behaves differently (see below).

Correction, second time, and the pattern is the finding. This description has now claimed a clean sweep twice and been wrong twice.

The first version said "0 false accepts across 36 shapes." A review found three. I corrected it here and re-measured — and that re-measurement is the sentence immediately below the sweep, which has since been falsified by seven more false accepts found across three further review rounds (6cd4d24, 8f64855, ee31cb3, 8ed796a): the END marker not required to end its own line, base64 accepted by alphabet rather than by whole quanta, an over-restricted label grammar, Unicode whitespace stripped where node accepts only ASCII, and a malformed BEGIN line skipped instead of rejected.

So the honest statement is not a count. Every clean-sweep number in this description has been falsified by the next reviewer, and I have no reason to believe the current one is different in kind. What the shape table can support is a floor, not a ceiling: these shapes are checked, these are the ones a regression would catch. It cannot say the set is complete, because five successive rounds have shown my shape set is the thing that is wrong, not the guard's logic.

Leaving both corrections visible rather than editing the claim away — AGENTS.md requires a load-bearing claim not survive on plausibility, and the track record of this particular claim is itself evidence about how far to trust the next one.

The guard vs. a real handshake

False rejects — a healthy bundle refused. Refusing is not the safe direction: the fallback drops every sibling and corporate CA for the whole session, which is the failure this contract exists to prevent, while printing ignoring <ca-trust.pem> (torn block) and blaming the builder for a file the runtime loads happily.

bundle shape #283 node
CRL block before/after ours reject AUTHORIZED
PUBLIC KEY block before ours reject AUTHORIZED
PRIVATE KEY block after ours reject AUTHORIZED
# see -----BEGIN CERTIFICATE----- then our CA reject AUTHORIZED
mid-line -----BEGIN , then our CA reject AUTHORIZED
torn / corrupt block after ours reject AUTHORIZED

False accepts — a bundle node cannot load, waved through. The dangerous direction: claude then distrusts the very proxy it is routed through and every request fails TLS.

bundle shape #283 after 8f46b89 node
our CA relabelled TRUSTED CERTIFICATE accept reject no
corrupt PUBLIC KEY ahead of our CA reject accept no
corrupt X509 CRL ahead of our CA reject accept no
corrupt block, BEGIN line has a trailing space reject accept no

The relabelled case is the subtlest: X509Certificate ignores the PEM label and decodes the body, so our CA relabelled TRUSTED CERTIFICATE yields byte-identical DER and the guard says "carries our CA" — while node's loader skips any block not labelled exactly CERTIFICATE.

DER identical to our real CA      : true
NODE_EXTRA_CA_CERTS=<relabelled>  : {"authorized":false,"err":"UNABLE_TO_VERIFY_LEAF_SIGNATURE"}
NODE_EXTRA_CA_CERTS=<original>    : {"authorized":true,"err":null}

The other three are one mistake in three places: deciding a block is safe without proving it decodes.

  • Non-certificate blocks were skipped outright. "Node ignores non-cert blocks" holds only for well-formed ones — node aborts the whole extras load on any block it cannot decode, whatever the label. What "decodes" means differs by label, and both halves were measured: a CERTIFICATE must parse as X509 (base64 validity is not enough — a well-formed base64 body that is not a certificate still kills the load), everything else needs only valid base64 armor. Demanding more would re-reject the CRLs and key blocks a real corporate bundle legitimately carries.
  • The marker was $-anchored, so -----BEGIN CERTIFICATE----- (one trailing space) was invisible to the guard while openssl still reacted to it.
  • The END search ran to end-of-file, so a torn block could borrow the terminator of a later one; the unterminated check never fired and the slice spanned two entries. Now bounded at the next BEGIN.

Current shape table: 26 rows, 9 accept / 17 reject, every accept row cross-checked against a real NODE_EXTRA_CA_CERTS handshake. Read that as coverage, not as proof of absence — see the correction above. The reject direction is the one the guard is allowed to take: where it cannot tell (a block damaged after ours, whose truncated body may or may not still decode, since openssl's base64 reader treats the next - as end-of-data rather than an error) it refuses. Refusing costs one session's sibling CAs; accepting costs the session entirely.

Why the tests did not catch any of this

Two independent gaps, both measured:

  1. The guard was hand-duplicated. bundleIsUsable in the test file was a copy of the launcher's inline decision under a "change one, change both" comment. Mutating the launcher's copy to accept everything left the whole suite green. Moved to bin/ca-trust.mjs, which the launcher imports and the test imports. Two call sites is below this repo's bar for a new module; the justification is not reuse, it is that a test cannot import a top-level script and a copy is not the thing that ships.

  2. The oracle was the wrong mechanism. The table cross-checked against tls.connect({ca: ...}). That option accepts the relabelled bundle that NODE_EXTRA_CA_CERTS rejects — so the test was certifying the guard against a code path the launcher does not use. The helper now spawns a child with the variable set from birth (node reads it once at startup, so setting it in-process after boot tests nothing).

Three more in the same block

  • The orphan reaper shared the publish try, so renameSync throwing skipped it. On exactly the hosts where publishing is persistently broken (a root-owned ccf.pem, a read-only mount, ENOSPC) each launch abandoned one full-CA temp and collected none — unbounded growth in the directory a builder globs. Measured with a directory at the publish target: both the new temp and a pre-seeded 2-hour-old orphan survived.

  • A corrupt ca.pem was blamed on the bundle, and then handed to claude anyway. The X509 parse sat inside the bundle try, so an unparseable ca.pem printed ignoring <ca-trust.pem> (no start line) — naming a file that may be perfectly healthy. It is now parsed in its own step and named in its own message, and NODE_EXTRA_CA_CERTS is left unset rather than pointed at the file that just failed to parse (measured before the fix: CA=/tmp/.../ca.pem, the unparseable one; after: CA=UNSET). Node falls back to its built-in store, which is the honest state — we have no usable CA to add. Reachable because the proxy's reuse guard keys on existsSync(ca.pem) && existsSync(ca.key), so a corrupt pem with its key beside it is reused, not regenerated.

  • The proxy's export NODE_EXTRA_CA_CERTS=<our ca.pem> recipe reached the operator immediately after the launcher had wired claude via ca-trust.d — telling them to undo it. The server now prints the recipe only when the operator is the one wiring: process.channel is set exactly when our launcher fork()ed it, and the launcher is the only fork() site (the server subcommand uses spawn; a service manager runs it bare). The mode line prints either way. Standalone, the recipe carries a same-host-MITM caveat, as do the README's manual-wiring recipes.

Coverage

Every clause is mutation-verified — each of these was caught by exactly one test:

mutation result
label check removed 12 pass / 1 fail
carriesUs always true 12 / 1
unterminated-block check → skip 12 / 1
line anchor dropped 12 / 1
undecodable CERTIFICATE check → skip 12 / 1
END search unbounded again 12 / 1
non-cert blocks skipped again 12 / 1
trailing-space tolerance removed 12 / 1
reaper moved back inside the publish try 20 / 1
own-CA parse folded back into the bundle try 3 / 18
corrupt ca.pem handed to claude again 22 / 1
process.channel gate forced false 21 / 1

Size

+67 production code, +126 comment, +233 test (3.5x), 1 new file, 0 new env vars, 0 new on-disk paths. Comparable to the calibration rows in AGENTS.md (#261: 23 code, 8.4x). Under 300 production LOC, so no ## Non-Functional Requirements section — but load-bearing: yes (TLS trust path), so this wants human review before merge.

Test run

1507 pass / 0 fail on the full suite; 1499 / 0 at the merge base (23346ac). The 8 added tests are this PR's.

🤖 Generated with Claude Code

codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 1, 2026
codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 1, 2026
codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 1, 2026
codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 1, 2026
@codeslake
codeslake marked this pull request as ready for review August 1, 2026 20:37
codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 1, 2026
codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 1, 2026
codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 1, 2026
codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 1, 2026
codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 1, 2026
codeslake and others added 9 commits August 1, 2026 21:42
The guard shipped in cnighswonger#283 disagreed with a real handshake on 8 of 20
measured bundle shapes (node v24.11.1 / openssl 3.6.1).

Seven were needless refusals of healthy bundles. Every PEM block was
parsed as a certificate, so any non-certificate block a merged bundle
legitimately carries — a CRL, a public key, key material — threw and
voided the whole file; and the torn-block check counted raw occurrences
of "-----BEGIN ", so a provenance comment that merely mentioned the
marker made a healthy bundle look torn. Refusing is not the safe
direction: the fallback drops every sibling and corporate CA for that
session, which is the failure this contract exists to prevent.

The eighth was the dangerous direction. Our own CA relabelled
TRUSTED CERTIFICATE parses to byte-identical DER, so the guard reported
"carries our CA" while node's loader skipped the block entirely, leaving
the session trusting nothing and failing every request with
UNABLE_TO_VERIFY_LEAF_SIGNATURE. The guard's own comment promised it was
"allowed to be conservative, never permissive" — this was permissive.

Markers are now anchored to line starts, non-CERTIFICATE blocks are
skipped the way node skips them, and the DER match must land on a
CERTIFICATE block. Where the guard cannot tell — a block damaged AFTER
ours, whose truncated body may or may not still decode, since openssl's
base64 reader treats the next '-' as end-of-data rather than an error —
it refuses.

The decision moved to bin/ca-trust.mjs so the tests drive the shipped
code. It was inline in a top-level script with a hand-copied twin in the
test file under a "change one, change both" comment; measured, mutating
the real one left the entire suite green. The test file's oracle was
also wrong: it verified through tls.connect({ca}), which ACCEPTS the
relabelled bundle that NODE_EXTRA_CA_CERTS rejects, so it was certifying
the guard against a mechanism the launcher does not use.

Three related defects in the same block:

- The orphan reaper shared the publish try, so rename() throwing skipped
  it. On exactly the hosts where publishing is persistently broken (a
  root-owned ccf.pem, a read-only mount, ENOSPC) each launch abandoned
  one full-CA temp and collected none.

- Our own CA was parsed inside the bundle try, so an unparseable ca.pem
  was reported as `ignoring <ca-trust.pem> (...)` — naming a file that
  may be healthy — and then fell back to the file that had just failed
  to parse.

- The spawned proxy's `export NODE_EXTRA_CA_CERTS=<our ca.pem>` recipe
  was relayed to the operator immediately after the launcher had wired
  claude via ca-trust.d, telling them to undo it. The launcher now drops
  those lines from the stderr it relays; standalone the recipe carries a
  same-host-MITM caveat.

Every clause is covered in both directions: five mutations of the guard
and one of each fix above were each caught by exactly one test.

prod +58 code / +101 comment, tests +199 (3.4x), 1 new file, 0 new env
vars. Full suite 1501 pass / 2 fail, both EMFILE from an fs.watch test
that fails identically at the merge base (inotify max_user_instances=128
on this host, unrelated to this change).

Co-Authored-By: Claude <noreply@anthropic.com>
…stderr

Two simplifications, both measured before applying.

The launcher was line-buffering the proxy's stderr and stripping the
wiring recipe with a regex — 20 lines to remove text it had caused. The
server can tell directly: process.channel is set exactly when fork()
created the process, and the launcher is the only fork() site (the
`server` subcommand uses spawn, and a service manager runs it bare).
Measured: fork -> channel set, standalone -> undefined. So the recipe is
now gated at the source and the relay is a plain pass-through again.

And bundleCarriesOurCA no longer normalizes CRLF. `$` in a /m regex
matches before a `\r`, and the END search is anchored on the leading
`\n`, so both halves already read a CRLF file the same as an LF one.
Measured across 102 shapes (34 bundle layouts x LF/CRLF/mixed):
identical verdicts with and without the replace, and 0 false accepts
against a real handshake.

No behavior change: standalone still prints the full recipe, the
launcher still prints the mode line and not the recipe, and the guard's
verdicts are unchanged. Mutating the new gate to false fails exactly one
test, and all five guard-clause mutations are still caught.

-16 net production lines. Full suite 1503 pass / 0 fail.

Co-Authored-By: Claude <noreply@anthropic.com>
…a.pem

A review pass found three false accepts the previous commit still had, and
one claim in its own CHANGELOG that was not true. All four reproduced
against a real NODE_EXTRA_CA_CERTS handshake before fixing.

Two were the same mistake: the guard decided a block was safe without
proving it decodes.

- Non-CERTIFICATE blocks were skipped outright. "Node ignores non-cert
  blocks" holds only for WELL-FORMED ones — node's reader aborts the
  whole extras load on any block it cannot decode, whatever the label.
  Measured: a corrupt PUBLIC KEY and a corrupt X509 CRL each ahead of a
  healthy CA gave guard=accept, handshake=UNABLE_TO_VERIFY_LEAF_SIGNATURE.
  Every block must now decode; what that means differs by label, measured
  per label: a CERTIFICATE must parse as X509 (base64 validity is not
  enough — a well-formed base64 body that is not a certificate still kills
  the load), everything else only needs valid base64 armor. Demanding more
  would re-reject the CRLs and key blocks a real corporate bundle carries.

- The marker was anchored with a bare `$`, so `-----BEGIN CERTIFICATE----- `
  (one trailing space) was invisible to the guard while openssl still
  reacted to it. A corrupt block wearing a trailing space rode through.

The third was the END search: `indexOf` scanned to end-of-file, so a torn
block with no END of its own could borrow the END line of a later block.
The unterminated check never fired and the slice spanned two entries. Now
bounded at the next BEGIN.

The fourth was a false claim, not a code defect I had introduced — the
CHANGELOG said a corrupt ca.pem no longer "fell back to the very file that
had just failed to parse". Only the message had been fixed; caForClaude
still defaulted to it. Measured: `CA=/tmp/.../ca.pem`, the unparseable
file. NODE_EXTRA_CA_CERTS is now left unset in that case, so node falls
back to its built-in store — the honest state, since we have no usable CA
to add. CHANGELOG and README corrected to match what the code does; the
README's "non-certificate blocks are ignored" line was overbroad for the
same reason as the second bug above.

Coverage: five new rows in the guard table (both corrupt non-cert labels,
a well-formed one that must still pass, the trailing-space case, the
borrowed-END case) plus a wrapper test asserting CA=UNSET. Each of the
four fixes is mutation-verified — reverting it fails exactly one test.

Re-measured after: 0 false accepts across 36 handshake-checked shapes,
5 conservative rejects (all damaged-bundle cases, the allowed direction).
Full suite 1502 pass / 2 fail, both EMFILE from an fs.watch test that
fails identically at the merge base (inotify max_user_instances=128 here).

Co-Authored-By: Claude <noreply@anthropic.com>
…ignal

A Codex review pass found three more defects. All three reproduced against
a real NODE_EXTRA_CA_CERTS handshake before fixing; the two P1s were
false accepts of the same class the previous commits were fixing.

- Base64 was checked as an ALPHABET, not as whole quanta. Measured: a
  PUBLIC KEY body of `A` ahead of our CA gave guard=accept while node
  reported `bad base64 decode` and loaded zero extra CAs. Padding is
  positional too — `AAA=` and `AA==` load, `A===`, `=AAA` and `AA=A` do
  not. Now length%4==0 plus trailing-only padding: 16/16 agreement with
  a real handshake on the body shapes measured.

- The label pattern was [A-Z0-9 ], so every other legal PEM label was
  invisible while openssl still treated the block as real. Measured: a
  malformed `X-FOO` block gave guard=accept, node loaded zero CAs. Every
  label tried behaved as a real block (hyphenated, lowercase, underscored,
  dotted, punctuated, empty), so the label now decides only WHICH check a
  block gets, never whether it is one. Note `[^-]*` does NOT fix this —
  `-` is legal inside a label, so the stop condition is the `-----` run.

- The banner suppression keyed on `process.channel`, which only proves
  SOME parent opened an IPC descriptor. Measured: a plain fork() of
  server.mjs (which this suite itself does, and any supervisor may) got
  the suppressed banner plus the false claim that a launcher had wired
  the client — leaving an operator with no wiring instructions at all.
  Now an explicit CACHE_FIX_WIRED_BY_LAUNCHER the launcher sets. This is
  an internal handshake between the two files, not an operator knob, and
  is deliberately undocumented as one.

Also fixes the test-suite temp-dir leak reported in the first review and
skipped then. Measured: one run of proxy-wrapper.test.mjs left 38 dirs
behind, and a /tmp that had accumulated 1954 of them held 432 ca.key /
leaf.key files — forward mode mints an RSA CA and leaf per config dir, so
the leak is private key material, not empty directories. Registered
centrally with one after() hook rather than per-test rmSync, because a
failing test throws before its own cleanup and every future test would
have to remember. A leak is invisible to assertions (measured: suite
still reported 23 pass / 0 fail while leaking 39 dirs), so the guard is a
source-level check that nothing bypasses the registrar.

Coverage: 164 measured shapes across four sweeps, 0 false accepts. Each
of the three fixes plus the registrar is mutation-verified — reverting it
fails exactly one test. Full suite 1504 pass / 2 fail, both EMFILE from
an fs.watch test that fails identically at the merge base.

Co-Authored-By: Claude <noreply@anthropic.com>
A review pass died mid-response, but its last line named the gap: the
guard never validated what followed the END marker. Measured, and it was
two more false accepts.

`indexOf("\n-----END <label>-----")` matches a prefix, so it treated
`-----END CERTIFICATE-----garbage` and `-----END CERTIFICATE-------` as
terminators. Both make openssl reject the block: guard=accept while node
loaded zero extra CAs, on a bundle whose remaining entries were healthy.

Only whitespace may follow — 13/13 agreement with a real handshake on
what a tail may contain (space, tab, nothing: loads; any other character,
including a further dash run: does not). The END search now skips
candidates whose line does not end there, rather than taking the first
textual match.

Three rows added, including the positive one: a trailing space must keep
being ACCEPTED, or the fix trades two false accepts for a false reject.
Mutation-verified — reverting to the bare indexOf fails exactly one test.

Re-measured across all four sweeps at 164 shapes: 0 false accepts, no
regression in either direction.

Co-Authored-By: Claude <noreply@anthropic.com>
Three cuts, no behaviour change, plus one coverage hole they exposed.

isBase64Body took (block, endMarker) and re-derived the body by slicing
between the first newline and the last END marker — arithmetic the caller
had already done to build the block. It now takes the body itself, which
the caller has in hand as text.slice(m.index + m[0].length, end). Verified
equivalent under both LF and CRLF before applying: the BEGIN match
excludes the \r, so the two slices normalize to the same bytes.

The two `if (remoteControl)` lines merged into one block, and a comment
restating the line below it dropped.

The hole: mutating away the `length % 4` check left the suite GREEN. No
fixture had an alphabet-valid body of the wrong length, so a clause my
previous commit message claimed was covered was not. Two rows added — a
one-character body and `A===` — and both base64 clauses now fail exactly
one test when removed.

175 measured shapes across six sweeps, 0 false accepts, identical to
before the cuts. Full suite 1504 pass / 2 fail (EMFILE, same at the merge
base).

net: -8 lines.

Co-Authored-By: Claude <noreply@anthropic.com>
…ability

The paragraph said a reader "has no previous state to compare against",
which reads as a limitation — and a limitation is an invitation. Someone
adds the previous bundle as state, believes they have lifted it, and adds
a cert-count floor.

The floor would still be wrong. A shrink is legitimate whenever a root is
retired or a component is uninstalled, and only the builder knows which
happened, so a reader holding BOTH bundles still cannot tell a regression
from a fact. Measured across two machines here: a legitimate bundle is 5
certs on one and 168 on the other, so any floor that catches narrowing on
one host rejects a healthy bundle on the next.

Surfaced by a peer session that had the mirror-image wording in its own
comment and changed it after the same argument.

Co-Authored-By: Claude <noreply@anthropic.com>
…an unparseable CA

Two false-accept paths found by Codex review, both reproduced here before
being agreed with.

STRIP ASCII WHITESPACE ONLY. isBase64Body stripped with /\s+/, which is the
Unicode whitespace set. Node's PEM reader accepts space, tab, CR and LF and
nothing else. Measured one character at a time against a real
NODE_EXTRA_CA_CERTS load: those four load 1, while U+00A0 U+2003 U+2028 U+2029
U+FEFF U+1680 U+205F U+3000 and ASCII VTAB and FORMFEED each load 0 with
`bad base64 decode`. All ten are stripped by \s, so a body damaged by any of
them read as clean and the guard accepted a bundle that costs the session every
extra root. A NBSP is what a paste through a rich-text field leaves behind.

PARSE BEFORE PUBLISHING. The copy into ca-trust.d/ccf.pem happened before the
X509 parse, so a corrupt ca.pem was handed to every OTHER component. Our own
session degrades fine (it falls back to node's built-in store), but the builder
concatenates sort(*.pem) and "ccf" sorts first — the same fatal leading
position the torn-write guard already protects, reached by a different cause.
Atomicity guarantees whole bytes, never loadable ones. Now the parse throws
into the existing catch, which warns and leaves any previous good ccf.pem for
siblings to keep trusting.

Both TDD: each test fails on the pre-fix code and passes after. Both
mutation-checked: reverting [ \t\r\n] to \s fails proxy-forward-ca, removing
the pre-publish parse fails proxy-wrapper.

Suite 1505/1507. The 2 failures are the inotify EMFILE (max_user_instances=128
on this host) and fail identically at the merge base.

Four of the six review findings were against upstream code outside this PR's
diff — session-budget-breaker and tier-advisor — and are not touched here.

Co-Authored-By: Claude <noreply@anthropic.com>
Codex review against the real merge base, P1 and the only finding.

The marker pattern described a WELL-FORMED opener, so an over-dashed one
(`-----BEGIN CERTIFICATE-------`) matched nothing at all and the block became
invisible to the guard: nothing was checked, and our CA later in the file
carried the verdict. openssl does not skip it — it consumes the line as an
opener and then fails the ENTIRE extras load on the END it cannot match.
Measured, node v24.11.1: guard=accept, loader=0 CAs, `bad end line`.

A trailing `.*` makes the line match, which is all the fix needs: the block is
then seen and the existing per-block check rejects it as an undecodable
CERTIFICATE. Being SEEN is what a guard needs; skipping is what lets a bad
block through.

This is the same defect already fixed on the END side, in its mirror position.
The lesson: a shape fixed at one marker is a shape to go and check at the other.

Two rows: the malformed opener rejects, and a BEGIN wearing one trailing space
still ACCEPTS, so the fix cannot drift into the over-strict guard this PR set
out to remove.

Mutation-checked: reverting to the strict pattern fails the new row.

An earlier attempt added a separate pre-scan loop and an `undefined` label
branch. The branch was dead — `(?!-----)` still captures `CERTIFICATE` from an
over-dashed line — and the mutation SURVIVED, which is what exposed it. Removed
rather than kept as defence for a case that cannot happen.

Suite 1505/1507, the 2 being the inotify EMFILE that fails identically at the
merge base.

Co-Authored-By: Claude <noreply@anthropic.com>
codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 2, 2026
codeslake added a commit to codeslake/claude-code-cache-fix that referenced this pull request Aug 2, 2026
No "cnighswonger#296": GitHub reads a #N in any pushed commit message as an issue
reference and posts it to that PR timeline. This branch is our deploy artifact
and is rebuilt on every upstream move, so each rebuild was appending a
"referenced" line to a maintainer PR that has nothing to do with it — 10 of
them on 2026-08-01 alone. Plain "PR 296" says the same thing to a human and
links nothing.
@codeslake

codeslake commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Codex review round, applied. Head is now 8ed796a, rebased onto c8f7bb8.

Three false accepts, each reproduced before being agreed with

1. A malformed BEGIN line was skipped instead of rejectedbin/ca-trust.mjs

The marker pattern described a well-formed opener, so an over-dashed one matched nothing and the block became invisible to the guard: nothing was checked, and our CA later in the file carried the verdict. openssl does not skip it — it consumes the line as an opener and then fails the entire extras load on the END it cannot match.

bundle:  -----BEGIN CERTIFICATE-------  +  valid CCF CA
guard:   {ok: true}
node v24.11.1:  extra CAs loaded = 0,  PEM routines::bad end line

A trailing .* is the whole fix: the block is then seen, and the existing per-block check rejects it as an undecodable CERTIFICATE. Being seen is what a guard needs; skipping is what lets a bad block through.

This is the same defect already fixed on the END side in this branch, in its mirror position. The lesson worth keeping: a shape fixed at one marker is a shape to go and check at the other.

2. Unicode whitespace was stripped where node accepts only ASCIIbin/ca-trust.mjs

isBase64Body stripped with /\s+/. Measured one character at a time against a real NODE_EXTRA_CA_CERTS load:

space, tab, CR, LF                                 -> loads 1
U+00A0 U+2003 U+2028 U+2029 U+FEFF U+1680 U+205F   -> loads 0, bad base64 decode
U+3000, ASCII VTAB (\x0b), ASCII FORMFEED (\x0c)   -> loads 0, bad base64 decode

All ten are stripped by \s, so a body damaged by any of them read as clean and the guard accepted a bundle that costs the session every extra root. A NBSP is what a paste through a rich-text field leaves behind — this is a shape bundles really acquire.

3. A corrupt ca.pem was published before it was parsedbin/claude-via-proxy.mjs

The copy into ca-trust.d/ccf.pem happened before the X509 parse. Our own session degrades fine (it falls back to node's built-in store), but the builder concatenates sort(*.pem) and ccf sorts first — the same fatal leading position the torn-write guard already protects, reached by a different cause. Atomicity guarantees whole bytes, never loadable ones. The parse now throws into the existing catch, which warns and leaves any previous good ccf.pem for siblings to keep trusting.

Method

Each fix is TDD: the row fails on the pre-fix code and passes after. Each is mutation-checked — remove the guard, watch a test die, restore:

guard mutation result
trailing .* on the BEGIN marker revert to the strict pattern proxy-forward-ca fails
[ \t\r\n] instead of \s revert to \s proxy-forward-ca fails
pre-publish X509 parse delete the line proxy-wrapper fails

Every reject-direction row is paired with an accept-direction one (a BEGIN wearing one trailing space, a tab inside a body, a well-formed PUBLIC KEY) so the fixes cannot drift into the over-strict guard this PR set out to remove.

One mutation survived and that is worth reporting: an earlier attempt at finding 1 added a separate pre-scan loop plus an undefined label branch. Deleting the branch left the suite green. Investigating why showed (?!-----) still captures CERTIFICATE from an over-dashed line, so the branch was unreachable — dead code wearing the shape of a guard. Removed rather than kept as defence for a case that cannot happen.

Verification

Suite 1505/1507 on the rebased tree. The 2 failures are fs.inotify.max_user_instances (128 on this host) in test/proxy-server.test.mjs and fail identically at the merge base (1497/1499, same two test names) — environmental, not from this branch. Happy to re-run anywhere that has a larger inotify budget.

Unrelated, unverified — four leads in already-merged code

An earlier review run resolved its base to a stale main and covered 13 commits of yours as well as this branch. Four findings landed there. I did not reproduce these — they are outside this PR's diff and I am reporting them as leads, not findings, for whoever owns that code:

  • proxy/extensions/session-budget-breaker.mjs:350 — streamed accrual reads ctx.headers; the stream context built at proxy/stream.mjs:63 has no headers, so the lookup would always return null and normal streaming usage would never reach a ceiling. The non-streaming path reads ctx.meta._sbbSessionId.
  • tools/tier-advisor.mjs:537 — the recommendation ignores planRes.plan, so a Max 20x user can be told to upgrade to Max 20x.
  • tools/tier-advisor.mjs:522 — on the first run after a weekly reset, the new week's utilization is stored as q7d_actual_at_reset for the week that just ended.
  • tools/tier-advisor.mjs:535countConsecutiveWeeksOver is passed downgradeThreshold where the documented field consecutive_weeks_over_upgrade_threshold means the upgrade one.

Worth a separate issue if they hold up; the session-budget-breaker one looked load-bearing.

Still out of scope, deliberately

Unchanged from the PR description: the ccf.pem basename collision when CACHE_FIX_CA_DIR points somewhere other than the config dir, and the fact that a CA rotated mid-tick cannot match a bundle read in the same tick. Both are real; neither is what this PR is fixing.

— codeslake (CCF contributor)

Correction: my earlier comments on this PR were signed "— Proxy Builder". That is this repo's own review agent, not me. Copied from my fork's role naming by mistake; a contributor signing as the reviewer inverts the audit trail. Fixed on all of them.

🤖 Generated with Claude Code

ponytail-review over the production diff. bin/ca-trust.mjs was 34 lines of code
under 121 lines of comment (3.6:1), most of it retelling how each false accept
was discovered — six incidents at roughly six lines each.

Every constraint survives, in the form that stops someone tightening it back:
what the clause defends and that it was measured. What went is the narrative of
finding it, which git log already holds verbatim and in more detail than a
source comment can carry.

Same pass on proxy/server.mjs: the process.channel history is dead (nothing
reads it now), so it keeps only the live reason the env var is the signal.

  bin/ca-trust.mjs   157 -> 111 lines, comments 121 -> 75, code unchanged at 34
  proxy/server.mjs   -2

Re-verified after cutting, since a comment pass can still break code: suite
1505/1507 (the 2 being the inotify EMFILE that fails identically at the merge
base), and all three guards still mutation-lethal — reverting the whitespace
class, dropping the trailing `.*`, and dropping the next-BEGIN bound each fail
proxy-forward-ca.

CC_WRAPPER_SKIP_TESTS=1: the cross-component suite fails on a LIVE check,
"every published component CA is in the bundle — missing: cswap-pin.pem".
Measured it is not ours rather than assuming: the same check fails identically
(passed=12 failed=1) with these edits stashed, and cswap-pin.pem was published
21:09 against a bundle last built 20:41 — a rebuild the builder has not run yet,
on another session's component.

Co-Authored-By: Claude <noreply@anthropic.com>
@codeslake

codeslake commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Correcting two things in my previous comment.

The ccf.pem basename is not contract-bound, and I implied it was. I listed it as out of scope without saying why, which reads as "this needs coordination". It does not. Measured who actually consumes the name:

cachefix-ensure:303   for _pem in "$ca_trust_d"/*.pem     # glob, not a name
cswap-pin proxy.py    CA_TRUST_DIR = "ca-trust.d"         # directory only

The builder globs and sorts; the sibling component knows the directory and nothing else. The contract is "each component drops one file naming itself" — the name is the publisher's to choose. So this is a one-line change in bin/claude-via-proxy.mjs that no other component has to agree to.

Deferring it anyway, now on the real grounds rather than an implied constraint:

  • The collision needs CACHE_FIX_CA_DIR diverging from the config dir and a shared config dir and two live instances. The README recommends that variable in exactly one place (Docker, for a writable path).
  • The obvious fix — fingerprinting the CA into the filename — leaves a stale ccf-<old>.pem in the directory on every rotation, so it needs a second reaper. That is more moving parts than the case it covers.
  • This PR narrows the same blast radius from the other side: an unparseable CA is no longer published at all, so an overwrite can no longer put garbage in front of everyone else's roots.

Happy to take it as a follow-up if you would rather have it closed.

"Unchanged from the PR description" was wrong — the description has no out-of-scope section, so there was nothing to be unchanged from. Both items were carried in my working notes, not in this PR's text. Stating them here is the first time they appear.

The other deferral stands as written: a CA rotated mid-tick cannot match a bundle read in the same tick.

Nothing in the code or the verification changes; this is a correction to the framing only.

— codeslake (CCF contributor)

Correction: my earlier comments on this PR were signed "— Proxy Builder". That is this repo's own review agent, not me. Copied from my fork's role naming by mistake; a contributor signing as the reviewer inverts the audit trail. Fixed on all of them.

🤖 Generated with Claude Code

@codeslake

codeslake commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Self-review pass against this repo's own review history, before asking anyone else to spend time on it. I went back through the comments on #246, #251 and #283 and ran each recurring finding as a check against this branch. One hit, and it is the same shape as a #283 blocker — "the PR body states X; as written it isn't".

The description has claimed a clean sweep twice and been wrong twice. The body is edited; flagging it here because nobody re-reads a description.

The first version said "0 false accepts across 36 shapes" — a review found three, and I corrected it inline. But the correction's own re-measurement has since been falsified by seven more, across three further rounds:

commit false accepts
6cd4d24 END marker not required to end its own line (2)
8f64855 base64 by alphabet rather than whole quanta; over-restricted label grammar (3)
ee31cb3 Unicode whitespace stripped where node accepts only ASCII (1)
8ed796a malformed BEGIN line skipped instead of rejected (1)

The shape count was also simply wrong: the body said 36, the table has 26 rows.

So the description no longer makes a count claim. It now says what the table can actually support — 26 rows, 9 accept / 17 reject, every accept row cross-checked against a real NODE_EXTRA_CA_CERTS handshake — read as coverage, not as proof of absence. Five successive rounds have shown the thing that is wrong is my shape set, not the guard's logic, so a sweep number from me is worth less than the track record of sweep numbers from me.

The other recurring findings, checked rather than assumed:

One thing I got wrong while checking, worth recording since it is a method note: I first flagged CACHE_FIX_WIRED_BY_LAUNCHER as untested because grepping the test directory for the constant returned nothing. Mutating the launcher to stop setting it fails proxy-wrapper.test.mjs:551. The test asserts the behaviour — that --remote-control does not print the banner that would undo its own coexistence — which is the right thing to pin. My grep was the wrong instrument.

Expecting, from #283's precedent: this is the same TLS trust path, so schema-change and needs-sim-validation presumably apply, and live multi-MITM validation is not reproducible on your side. The handshake results here are my measurements, not independently confirmed — stating that rather than letting it be implied.

— codeslake (CCF contributor)

Correction: my earlier comments on this PR were signed "— Proxy Builder". That is this repo's own review agent, not me. Copied from my fork's role naming by mistake; a contributor signing as the reviewer inverts the audit trail. Fixed on all of them.

🤖 Generated with Claude Code

@vsits-codex-review-agent vsits-codex-review-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: PR #296

Date: 2026-08-03
Reviewed: PR head 306090cebb663632e70d134ca79a759b8b781186 merged onto current origin/main
Round: 1
Label applied: changes-requested

What Is Correct

  • Measured: the two defects called out in the dispatch note are fixed at this head. A relabelled TRUSTED CERTIFICATE block now yields bundleCarriesOurCA() -> { ok:false, reason:"bundle does not carry our CA" } and a fresh child process with NODE_EXTRA_CA_CERTS=<bundle> fails UNABLE_TO_VERIFY_LEAF_SIGNATURE; a valid X509 CRL block ahead of our CA now yields bundleCarriesOurCA() -> { ok:true } and the same handshake authorizes.
  • Measured: the #283 round-1 blockers called out in the prompt are not reintroduced in the merged head. node --test test/proxy-forward-ca.test.mjs test/proxy-wrapper.test.mjs passed 39/39, including the publish-dir override, the orphan-temp reaper on publish failure, the no-bundle fallback, the CLAUDE_CONFIG_DIR path contract, and the banner-suppression cases.
  • Measured: the full merged-head suite passed 1507/1507 via npm test.
  • Measured: the new bin/ca-trust.mjs extraction is justified and is not bloat. On the original merged 23346ac9, I mutated the inline launcher guard in bin/claude-via-proxy.mjs to accept the merged bundle unconditionally; node --test test/proxy-forward-ca.test.mjs still passed 12/12. That proves the pre-PR test file was exercising its hand-copied twin, not the shipped launcher code.
  • Read + Measured: the proxy/server.mjs change belongs in this PR. The launcher now sets CACHE_FIX_WIRED_BY_LAUNCHER at bin/claude-via-proxy.mjs, and the server consumes it at proxy/server.mjs to suppress the standalone NODE_EXTRA_CA_CERTS=<our ca.pem> recipe only when the launcher already wired claude through ca-trust.d. The paired tests --remote-control does not print the wiring banner... and a plain fork of the server still gets the wiring recipe both passed.

Blockers

  • Measured + Read: two more guard/loader disagreements remain in bin/ca-trust.mjs, so the trust-path claim is still not true. First, a malformed overlapping opener ahead of our CA is still a false accept:
    -----BEGIN PUBLIC KEY----------BEGIN CERTIFICATE-----\nAAAA\n-----END PUBLIC KEY-----\n<our CA>
    bundleCarriesOurCA() returns { ok:true }, but a fresh child process with NODE_EXTRA_CA_CERTS=<bundle> fails UNABLE_TO_VERIFY_LEAF_SIGNATURE and warns PEM routines::bad end line. The skip happens because the guard only reasons over blocks matched by bin/ca-trust.mjs; this malformed opener is not rejected, it is ignored, and our later CA carries the verdict. Any accept path must be measured-loadable, so this remains blocking.
  • Measured + Read: the same function still has a false reject in the other direction. With our CA first, followed by a non-certificate block whose body contains a line-start -----BEGIN marker, bundleCarriesOurCA() returns { ok:false, reason:"unterminated PUBLIC KEY block" }, while a fresh NODE_EXTRA_CA_CERTS handshake authorizes. The culprit is the unconditional nextBegin = text.indexOf("\n-----BEGIN ", ...) / end > nextBegin => end = -1 logic at bin/ca-trust.mjs and bin/ca-trust.mjs: it treats a line-start marker-looking payload line inside a non-certificate block as the start of a new PEM entry. Refusing is not the safe direction here; it drops every sibling CA for the session.

What Needs Attention

  • Measured: the contributor's "floor, not ceiling" framing was the right one. I did find additional shapes beyond the table. At minimum the suite needs rows for the two shapes above before this can be called fixed.
  • Read: no schema surface was added. I do not see a schema-change label case here.

Bloat / Non-Functional

  • None. Production surface is proportionate to the defect: 3 production files changed (bin/ca-trust.mjs, bin/claude-via-proxy.mjs, proxy/server.mjs), 1 new production file, 0 new env vars, 0 new on-disk paths. The new module is justified by the measured test-gap above rather than by speculative reuse.

Recommendations

  • Add the two new measured shapes as regression rows in test/proxy-forward-ca.test.mjs, using the same NODE_EXTRA_CA_CERTS child-process oracle the PR already adopted.
  • Tighten the BEGIN-line handling so a malformed opener is rejected rather than skipped, and so a line-start -----BEGIN inside a non-certificate payload does not automatically terminate the surrounding block.
  • After that, re-run the existing full suite and a real --remote-control / forward-proxy session against live Claude traffic before merge; this path is load-bearing even though the local handshake harness is now much better than before.

Bottom Line

Revise. The PR fixes the two defects called out in the dispatch and correctly closes the old test-gap, but the central trust-path claim is still not true: I measured one remaining false accept and one remaining false reject in bundleCarriesOurCA(). Because this code decides whether to hand Claude a merged trust bundle on a live TLS path, those remaining disagreements are merge-blocking.

— Codex review

@vsits-codex-review-agent vsits-codex-review-agent Bot added changes-requested Blocking review findings are outstanding needs-sim-validation Requires integration testing with live CC traffic labels Aug 3, 2026
@codeslake

Copy link
Copy Markdown
Contributor Author

Both blockers reproduce, and fixing them the way the review suggests would have been round 6 of a losing argument. Changed approach instead — see #300, which names this exact pattern and parks the design question with @cnighswonger and me.

Both blockers confirmed

Reproduced before agreeing, fixtures built in Python so the byte sequences survive the shell:

shape guard NODE_EXTRA_CA_CERTS loads
-----BEGIN PUBLIC KEY----------BEGIN CERTIFICATE----- ahead of our CA ok 0
non-cert block whose body has a line-start -----BEGIN unusable 1

False accept and false reject, exactly as reported.

Why the recommended fix is not the one I made

The review recommends tightening the BEGIN-line handling. That is the right fix for this predicate and it is the fifth time we have made it — the round count is in #300: five rounds on one function, three parties, each finding shapes the last missed.

The rule the predicate is reaching for turns out not to be expressible from outside. An identical tear is recovered or fatal depending only on whether its truncated body happens to be complete DER, which is a question about bytes no parser can answer:

same tear, first in the file, body = a complete cert body  ->  loader reads 1
same tear, first in the file, body = junk base64           ->  loader reads 0

So bundleCarriesOurCA is gone. The launcher now asks node: a child with NODE_EXTRA_CA_CERTS set from birth stands up a TLS server holding our leaf and connects to it. Only a bundle from which the loader really loaded our CA completes that handshake.

Deliberately a handshake and not tls.getCACertificates — that API lands in v22.15 / v23.10 while this package declares engines: >=18, so on node 18/20/≤22.14 an API probe answers "cannot tell" for every input, which is not a guard. Measured: v20.19.0 undefined, v22.14.0 undefined, v22.15.0 function.

Cost, 100 interleaved runs: bare spawn median 16.4 ms, oracle 20.4 ms — 4.0 ms, once per launcher start, on a path that already forks node at bin/claude-via-proxy.mjs:144.

Both blocker shapes, and the whole 26-row table, now agree with the loader

The shape table survives as a regression table; the oracle just answers it correctly by construction. Running it found one pre-existing row that was wrong: torn block borrowing a later END was recorded as unusable, and the loader reads 2 certs with a passing handshake. The row had recorded the predicate's behaviour as the expectation, and five rounds re-certified it because every round compared the code to the table and none compared the table to node.

Three outcomes, never two

ok / not ok / unknown — the last meaning the probe could not run. A guard that answers "unusable" when it could not ask drops every corporate root on a machine whose bundle was fine. This was a real defect in my first draft (a healthy bundle came back "unusable" purely because node was unreachable), caught by an independent reviewer.

A refused merge no longer costs the other publishers their CAs

Previously a refusal meant "use our own CA alone", which drops every other component's CA for the session. The damage lives in the merge, not in the files that fed it, so the launcher now rebuilds from the ca-trust.d/ publishers that still work. Measured on a three-publisher host: 1 certificate under the old fallback, 2 under the rebuild.

What the review asked for that I did not do

"Add the two new shapes as regression rows." Done, plus rows for a context-dependent tear and a Buffer-typed CA. But the honest note is that the rows are no longer what makes the guard correct — the oracle is. The table now guards against regressing to a predicate, which is a different and smaller job.

"Re-run a real --remote-control session against live Claude traffic before merge." Not done, and I am not claiming it. I ran the shipped launcher end-to-end against the real 132-certificate bundle on this host (accepted, 132 extras loaded, our CA present) and against a damaged merge (refused, rebuilt, peer CA preserved), but that is a local proxy, not live API traffic. Flagging it as still owed rather than quietly satisfied.

One defect the 43 green tests did not catch

Worth reporting because it is the reason I now distrust a green suite as evidence. Every test built the CA with readFileSync(path, "utf8") — a string. The launcher uses readFileSync(path) — a Buffer. t.endsWith is not a function, swallowed by the outer catch, surfaced as could not evaluate ca-trust.pem, which reads as a bundle problem rather than a type error in my code:

before:  extras 1   ours true   peer FALSE
after:   extras 3   ours true   peer TRUE

Coverage of a function is not coverage of its caller. Found by running the real entry point, fixed test-first.

Verification

Linux  node v24.11.1   1512 tests  1512 pass  0 fail
macOS  node v26.5.1    1512 tests  1511 pass  0 fail  1 skip (linux-only)
macOS  node v25.8.0    1512 tests  1511 pass  0 fail  1 skip (linux-only)

Three node majors, which matters more for an oracle than for a predicate: a predicate imitates one openssl's behaviour and needs re-verifying per version; the oracle asks whichever loader is installed.

Not pushed yet — an internal review round is still open on this diff and I would rather not update the head twice.

— codeslake (CCF contributor)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changes-requested Blocking review findings are outstanding needs-sim-validation Requires integration testing with live CC traffic

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant