Benchmark validity: close the 3 audit-flagged holes (sense-leak + 2 factsCore gaming surfaces)#121
Draft
anantham wants to merge 18 commits into
Draft
Benchmark validity: close the 3 audit-flagged holes (sense-leak + 2 factsCore gaming surfaces)#121anantham wants to merge 18 commits into
anantham wants to merge 18 commits into
Conversation
Agent git worktrees live under .claude/worktrees/ INSIDE the repo. Vitest run from the main checkout therefore discovered every worktree's duplicate test copies — inflating test counts and coverage (a reviewer's "five-file verification" ran ten files / 91 tests because it also picked up a stale copy under .claude/worktrees/opus-p1-tech-debt). Added '**/.claude/**' to both test.exclude and coverage.exclude. Vitest matches exclude globs relative to the config root, so this is safe when run from inside a worktree (its own tests are at tests/… with no .claude segment) while excluding the worktrees when run from the main checkout. Verified: a normal test file still runs from inside the worktree with the exclude active. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
A model occasionally returns two prompts for the SAME [ILLUSTRATION-N] marker. Only one image can render at a marker (the image cache key is chapterId:placementMarker), so the duplicate is dead weight — and worse, it costs a second PAID generation whose result overwrites the first. The validator did not catch it: when the shared marker is already present in the text, the `jsonMarkers > textMarkers` branch filters "unmatched" prompts by presence-in-text, so both duplicates pass through. My P1.1/P1.2 consolidation routed all three providers through this validator, widening the exposure of that latent gap. Two fixes (approved: deduplicate keep-first, not fail-and-regenerate): 1. Validator (responseValidators.ts) — dedupe suggestedIllustrations by placementMarker, keep-first, BEFORE the count-based reconciliation. A surviving duplicate would also inflate jsonMarkers and defeat the text-vs-json balance checks, so this both stops the double-bill and keeps the reconciliation honest. This is the primary fix: duplicates never leave validation. 2. Image generator (imageGenerationService.ts) — defense-in-depth at the point money is actually spent. The generation loop keys every call by chapterId:marker; two entries sharing a marker issued two paid generateImage calls and the post-generation findIndex(...) wrote both results into the FIRST entry. The needs-generation list is now deduped by marker (with a logged drop) so it can't double-fire regardless of what reaches it. Tests: both verified RED against the pre-fix code — the validator returns 2 entries where it should return 1, and the generator issues 2 paid calls where it should issue 1. A distinct-marker control confirms non-duplicates still each generate. Part of the P1/P2 review-fix batch (PR #114). #5 (worktree exclude) already landed; #2 (budget ledger), #3 (Claude/Gemini abort), #4 (pending leak) still to come. tsc: 17 pre-existing errors, none in touched files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
handleTranslate claims the chapter in pendingTranslations before the budget checks so two rapid triggers can't both slip past the dedup guard during the awaits (P1.6). But releasePending() was only wired to the two graceful early-returns. If a budget await rejected — the dynamic import, hasKnownPricing (which may hit the network for '/' models), or getNovelTranslationCost's IndexedDB read — the throw left the claim set, and the guard at the top of handleTranslate then blocked EVERY future attempt on that chapter until a full reload. Fix: wrap the budget block so any throw releases the claim, then rethrows (fail-closed — no paid translation runs on a budget we couldn't verify). The claim still has to be set BEFORE the budget awaits, so moving it later (which would avoid the leak) is not an option — that reopens the P1.6 double-fire race. Test: verified RED against the pre-fix slice — the chapter stays in pendingTranslations after the pricing check throws, and a second attempt is silently blocked by the leaked claim (hasKnownPricing called once, not twice). Part of the P1/P2 review-fix batch (PR #114). #5, #1 already landed; #2 (budget ledger) and #3 (Claude/Gemini abort) next. tsc: 17 pre-existing errors, none in touched files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
P1.4 recorded failed/truncated translation calls in api_metrics, but the budget GATE
never read them — getNovelTranslationCost summed only successfully-persisted translation
versions. So failed, truncated, and failed-retry calls were billed by the provider yet
invisible to the cap. The docstring even admitted it ("the api_metrics ledger is the
eventual complete source"); this wires that source in.
Design — a hybrid of two DISJOINT sources, so nothing is double-counted:
1. Persisted translation versions (estimatedCost) — every successful paid version.
2. FAILED translation calls from api_metrics — billed but never became a version.
A call is either a persisted success or a recorded failure, never both, so adding the two
is exact. The join is by chapterId: verified that the value adapters record as api_metrics
chapterId is the same stableId the budget gate keys its chapters on (translationService
passes it through "for API metrics tracking"; all three adapters record it). New
apiMetricsService.getFailedTranslationCostForChapters(stableIds) reads the persisted store
and sums the failed-translation subset for those chapters; it degrades to 0 on a read error
so the version sum (the primary signal) always stands.
Still imperfect and documented as such: a SUCCESSFUL version the user deleted escapes both
sums (its metric is a success, its version is gone) — a much narrower gap than before, when
every failed call was invisible.
Also (the second half of #2): an EMPTY OpenAI response threw before the metric block, so
that billed call recorded nothing. The empty check now runs after the cost/metric helpers
and records a failed metric before throwing — same treatment as truncation and parse
failure.
Tests: both verified RED against pre-fix — the budget sum is 0.08 (versions only) where it
should be 0.20 with failed spend, and the adapter records no metric on an empty response.
Part of the P1/P2 review-fix batch (PR #114). #5, #1, #4 already landed; #3 (Claude/Gemini
abort) next.
tsc: 17 pre-existing errors, none in touched files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
P1.3 made the Translator abort a per-attempt signal, but only OpenAIAdapter forwarded it
to its SDK. Claude took NO signal at all, and Gemini only checked the signal AFTER
generateContent returned — so a timeout started a retry while the original Claude/Gemini
request was still running and billing. The P1.3 test only asserted the signal became
aborted, never that a provider received it.
Both SDKs support native cancellation; we just weren't using it:
- Claude: translateWithClaude now takes an abortSignal and passes it as the Anthropic
SDK's request options — claude.messages.create(payload, { signal }). ClaudeAdapter
forwards request.abortSignal (translate path) and input.abortSignal (chatJSON path).
- Gemini: model.generateContent(request, { signal }) — @google/generative-ai ≥0.24
supports it via SingleRequestOptions.signal. The "Gemini doesn't support native abort"
comment was outdated; the after-the-fact check is kept as a belt-and-braces fallback.
Tests: a new test captures each SDK call and asserts the signal is threaded in — verified
RED against the pre-fix providers (they call the SDK with a single arg, no signal). Three
pre-existing ClaudeAdapter assertions were updated for the new second argument (a
deliberate behavior change).
Completes the P1/P2 review-fix batch (PR #114): #5, #1, #4, #2, #3 all landed.
tsc: 17 pre-existing errors, none in touched files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
…ance The published leaderboard hardcoded `closedBook: true` with the note "Models receive ONLY the raw Pāli phrase — no dictionary or retrieval context." That contradicted reality AND its own source list: the benchmark feeds DPD attestations (roots/POS/attested senses) to both the Anatomist and Lexicographer passes (the DPD source is even listed right below as "factual authority"). The P2.1 anatomist grounding sharpened the contradiction — the benchmark was never closed-book (the lexicographer was always DPD-grounded). Set `closedBook: false` and rewrote the note to state accurately what models receive: raw Pāli PLUS DPD attestations at the Anatomist/Lexicographer passes, while still withholding the SuttaCentral dictionary, retrieval context, and prior-phase window that production adds (the SUTTA-014 parity gap) — so benchmark output is LESS grounded than production, not un-grounded. This is the code that stamps the field on the next generated board; the current published JSON is a stale July-3 preview that the v2.2 rerun replaces. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
…budget cap #2 — held-out leakage. The 30-phase ranked set claimed to be "honest, uncontaminated," but phase-ad, phase-ag and phase-aj each grade the exact `ātāpī sampajāno satimā` sequence that phase-aa teaches the model as a worked example (config/suttaStudioExamples) — the satipaṭṭhāna refrain recurs verbatim, so those three phases were the answer key in the prompt. Removed them from phasesToTest (now 27); re-goldening can't help, the sequence IS the example, and the refrain is still exercised once as phase-aa. Added tests/scripts/sutta-studio/phase-contract.test.ts, which derives the example sequences from the actual prompt example objects and fails if any ranked phase grades one — verified RED against the pre-fix config (it lists ad/ag/aj) and green after removal. Nothing hardcodes a 30-phase count; the survivorship logic reads phasesToTest.length. #6 — no spend cap. The runner estimated cost after each response, let an unknown price become null (counting as $0), and had no cumulative stop, so a mispriced model or runaway loop could spend unbounded. New SpendGuard (scripts/sutta-studio/spend-guard.ts) tracks cumulative spend; every LLM call accrues, and the run ABORTS at the next loop boundary once BENCHMARK_CONFIG.maxSpendUsd (default $50) is reached. Enforcement is at the model- and phase-loop tops, NOT inside a pass runner — a pass runner's try/catch would swallow the throw into a per-phase failure and let the run keep spending; a loop-top throw propagates to runBenchmark().catch and aborts. A null-cost call is charged a conservative $0.25 estimate so unpriced spend drives toward the cap instead of slipping through as $0. Unit-tested. NOT addressed here (still gates a paid run, per the review): the roster is still 6 models not the intended 12, the 40/30/30 formula decision, and the full 30-phase golden-contract repair (#1) — those are operator-directed and larger. tsc: 17 pre-existing errors, none in touched files. Full sutta-studio suite green (81). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
…ph + vocab
The proposed facts score (advisory today, ranked under the v2.2 40/30/30 proposal) had two
confirmed defects the review flagged:
1. A bogus assertion could RAISE the macro. Morph was graded per-word, gated on the model
having asserted *something*, and fitsReading passed VACUOUSLY when the DPD reading was
silent on the asserted key. So `{note: "nonsense"}` made the word "assert morph", the
junk key vacuously "fit", and morph appeared as a free 1.0 category — the reviewer
measured a weak macro rise 0.500 → 0.667.
2. A correct assertion could score WRONG on vocabulary. `case: "ins"` vs DPD's `instr` are
the same case, but the exact-string compare called it a miss (macro → 0.333).
Fix:
- Grade morph per AUTHORITY-KNOWN key, not per model assertion. For each eligible word,
the gradeable keys are the gender/case/number a reading actually specifies; the model's
value on such a key is scored (correct iff it matches some reading), and keys no reading
covers — like an invented `note` — are IGNORED. The denominator is authority-set, so extra
assertions can't game it; omission of a known key stays uncharged (silence ≠ wrong), but an
unconfirmable assertion is no longer free.
- Canonicalise morph vocabulary before comparison (`ins`≡`instr`, `m`≡`masc`, `sg`≡`sing`, …),
so equivalent abbreviations match.
- `pos` is excluded from morph (open-ended vocabulary; the word-class check covers it) and its
docstring now says honestly that the `pos` breakdown is a word-class agreement check, not
morphological part-of-speech — matching the reviewer's accepted "root/word-class macro".
Tests: two new guards reproduce the reviewer's exact repros, verified RED against the pre-fix
scorer (junk inflates morph to 1/1; `ins` scores 0/1). One existing morph test updated for
the deliberate per-word → per-key change (kāye 2/2, kurūnaṁ 0/2). Facts + quality scorer
suites green (28).
NOT addressed (still gates a paid run, operator-directed): the 12-model roster, the 40/30/30
formula decision, and the full 30-phase golden-contract repair (#1).
tsc: 17 pre-existing errors, none in touched files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
…hanical fixes + gate The board scores each phase by aligning the model's analysis to a golden. For that to be fair the golden must cover the SAME Pāli word sequence the prompt shows the model. It didn't: a mechanical audit confirmed only 6/30 ranked phases matched. The dominant cause — 16 phases — was a golden that is a correct contiguous SLICE of a shared segment, but with no `wordRange` to slice the PROMPT to match, so the model was prompted with the whole segment yet graded on a fraction (100% "coverage" while ignoring most prompted words). This is the shared-segment problem ADR SUTTA-003 opened and left unfinished beyond 7 phases. Part 1 (this commit, mechanical — no golden CONTENT changed): - Added `_phases[].wordRange` to the 16 SLICE phases, slicing each prompt to exactly its existing golden span. MATCH goes 6/30 → 22/30. The golden word sequences were already correct; only the prompt derivation was missing the slice. - New gate tests/scripts/sutta-studio/golden-prompt-contract.test.ts derives the prompt sequence exactly as the runner does and compares it (normalised) to the golden. Verified RED without the wordRange additions (all 16 SLICE phases listed as violations), green after. A second test keeps the pending list honest — it fails if a listed phase is repaired but not removed from the list. Part 2 (drafted, NEEDS OPERATOR DECISION — docs/roadmaps/GOLDEN-CONTRACT-REPAIR.md): - 8 phases where the golden SPLITS a joined token the prompt presents as one whitespace token (sandhi `etadavoca`→`etad`+`avoca`; the `'ti` quotative in the breathing section) or OMITS a word. `wordRange` can't split a token, so these need golden CONTENT edits. The doc gives each phase's exact mismatch and the recommended resolution: make the golden follow the Anatomist's one-word-per-whitespace-token rule (sandhi moves to morpheme SEGMENTS, not word boundaries), and add the omitted words. Tracked in the test's KNOWN_SANDHI_PENDING allowlist. I did NOT edit golden content unilaterally — the sandhi-tokenization policy is a scholarly call. So: the mechanical majority is repaired and the contract is now enforceable; the remaining 8 are documented with a proposed direction, pending your sign-off on the sandhi policy. tsc: 17 pre-existing errors, none in touched files. Sutta-studio suite green (43). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
…word-per-token) Operator approved the one-word-per-whitespace-token policy. This reconciles the 8 remaining phases so all 30 ranked phases satisfy the golden/prompt contract (30/30). Anatomist golden: - Merged each split pair into ONE word matching the prompt's whitespace token; the sandhi split is preserved in that word's morpheme SEGMENTS (etad·avoca; assasāmī·ti), which still reconstruct the surface (verified). The CONTENT word's id survives the merge (so its senses stay valid), e.g. `etad`(fn)+`avoca`(content) → the merged `etadavoca` keeps avoca's id. Relations retargeted to the survivor; intra-word relations dropped. - Added the two omitted function words: `vā` in phase-an (with wordRange [0,8)); for phase-aq, merged `sato`+`va`→`satova` and added the second `satova` the prompt has. Downstream goldens (the merge changed anatomist word ids, so these had to follow or they'd dangle — verified none dangle after): - Lexicographer: dropped the removed function words' sense entries (the merged word is scored on the surviving content word's senses); added a sense entry for the inserted 2nd satova. - Weaver: retargeted linkedPaliId to the surviving word. - Typesetter: remapped layoutBlocks to the surviving ids and de-duplicated. Gate: tests/scripts/sutta-studio/golden-prompt-contract.test.ts now normalises on the Pāli LETTER sequence (quotes/punctuation dropped — the golden writes `'ti`, the source prints a curly ‘…’) and its KNOWN_SANDHI_PENDING list is EMPTY. Verified RED against the pre-Part-2 golden (all 8 listed as violations), green after. Validated: all 4 goldens internally consistent (segment reconstruction, zero dangling word/segment refs). tsc 17 (baseline). Sutta-studio suite green (43). Diffs localised to the 8 phases; the golden-repair doc updated to DONE. With this, review finding #1 is closed — the last mechanical benchmark-validity blocker. What remains before the paid retake is operator-directed: the 40/30/30 formula and the 12 model IDs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
…re weights) Operator resolved the review's finding #3: the 40/30/30 weights are the FINAL score, not the fidelity-internal weights nested inside the old 60/25/15. So the ranked score becomes: overall = gateFactor × (0.40·segmentationF1 + 0.30·factsCore + 0.30·senseF1) quality-scorer.ts (RUBRIC_VERSION 2.1 → 2.2; leaderboard RANKED_RUBRIC_VERSION → 2.2): - segmentationF1 — morpheme-boundary micro-F1 vs golden (unchanged). - factsCore — root + word-class macro from scoreFactsDetail. Morph is EXCLUDED (advisory this cycle per the reviewer); its gaming + vocabulary were fixed separately (review #4) so it can be promoted later. Scored against the golden's DPD-verified √tooltips (no external DPD needed). - senseF1 — strict sense-english micro-F1 (scoreSenseFidelityDetail, SUTTA-012 drop-penalised). - Usability and richness are RETIRED from rank (still computed for display); alignment, the LLM judge and the reader probe stay advisory. - Components a phase's golden doesn't provide (e.g. a function-only phase with no senses) renormalise over the present weights rather than being charged 0. - fidelityScore stays the v2.1 value, still used as the leaderboard's "has a golden → rankable" gate. The board's methodology string now discloses the v2.2 formula; ADR SUTTA-013 amended with the decision + implementation. Note the facts-scorer import creates a call-time cycle with quality-scorer (which exports alignWords/tokenize back) — safe because both sides use the other's exports only inside functions. Tests: 4 new guards (the 40/30/30 weights, the retirement of usability/richness, renormalisation), verified RED against the v2.1 scorer (it returns 0 where v2.2 returns 0.55 — it never read factsCore/senseF1). Full sutta-studio + scorer suite green. tsc 17 (baseline). This is a rubric bump — v2.1/v2.2 scores are not comparable (version gate enforces it), so it needs a FLEET RE-RUN before the board is republished. Remaining follow-ups: board columns for factsCore/senseF1, and the twelve model IDs for the run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sd5oCcYiQBnGAiNT4SAo8L
…nto fix/opus-benchmark-validity
…to fix/opus-benchmark-validity
…ix/opus-benchmark-validity
… fix/opus-benchmark-validity
…nse axis The lexicographer PROMPT teaches `viharati → "dwells"` in its ripple worked example, and 8 ranked phases (z/af/ai/az/ba/bc/bd/bg) grade exactly that sense — so a model copying the example earns free senseF1 credit (30% of the ranked score). Same leak class the benchmark removed phase-ad/ag/aj for, but on the sense axis instead of segmentation; the phase-contract guard (anatomist-only) misses it. This contradicts benchmark-config's "honest, uncontaminated ranking set". Reglossed the example to `gacchati → goes/going/went` (verified absent from every ranked phase's graded surfaces AND senses), preserving the tense-ripple demonstration. Added lexico-example-leak.test.ts: no gloss taught in either lexico worked example may be a graded sense in any ranked phase — RED against the old example (lists viharati:dwells across 8 phases), green after. Durable: any future example teaching a ranked gloss fails CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…root spray-cap) factsCore is 30% of the ranked score (root + word-class macro). Two goodhart holes found by the adversarial audit: A1 — POS (word-class) was scored over golden CONTENT words only, so a model labelling every word `content` collected a free 1.0 (its function-word errors were never checked). Now scored over EVERY golden word. A phase with no content word stays null (no facts to grade), so the "returns null" edge is preserved. A2 — root was recall-only / precision-blind: any asserted √root matching the authority counted as correct, no penalty for the wrong roots also sprayed, so hedging many roots to guarantee a hit was rewarded (#117 hardened MORPH's denominator against exactly this but left ROOT open). A hit buried in > authority + ROOT_SPRAY_SLACK(2) roots is now `sprayed`, not correct; a focused answer (right root + a little slack) still counts. 2 new tests (A1 penalty, A2 spray-vs-focused) + updated existing assertions, all verified RED against the pre-fix scorer. tsc at baseline, full sutta suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
report-facts-layer.ts rendered only fab/sil/drop, so the new 'sprayed' root failure category (audit A2) was invisible and the displayed breakdown no longer summed to root failures. Aggregate and display root.sprayed in the root column. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the three validity items surfaced by the adversarial audit — the ones you said were yours to call. All three are free to do now and expensive later: they change
factsCore/senseF1, so once a paid v2.2 board exists they'd force a re-run to re-score; done before the first paid run, they cost nothing.The fixes genuinely depend on the merged state: A1/A2 need #117's
facts-scorer.ts; the C1 leak guard reads #116's 27-phase set and #118's goldens. So this branch contains the merge of #114/#116/#117/#118/#119 plus my two fix commits. Merge those five first — then this PR's diff collapses to just the two commits below (or ping me and I'll re-cut it cleanly against the new main; if you squash-merge the five, I'll rebase). Review my two commits, not the whole diff.C1 — de-leak the lexicographer worked example (commit 1)
The lexicographer prompt teaches
viharati → "dwells", and 8 ranked phases (z/af/ai/az/ba/bc/bd/bg) grade exactly that sense → free senseF1 credit (30% of rank). Same leak class you removed phase-ad/ag/aj for, on the sense axis; the anatomist-onlyphase-contractguard misses it. This is what dinged #116's "honest, uncontaminated ranking set."gacchati → goes/going/went, verified absent from every ranked phase's graded surfaces and senses; the tense-ripple lesson is preserved.lexico-example-leak.test.ts— no gloss taught in either lexico worked example may be graded in any ranked phase. RED against the old example (printsviharati → "dwells"in 8 phases), green after. Durable against future recurrence.A1 — POS scored over ALL golden words (commit 2)
Word-class was scored over content words only, so labelling every word
contentearned a free 1.0 (function-word errors never checked). Now scored over every golden word; a phase with no content word staysnull(no facts to grade), so the "returns null" edge is preserved.A2 — root spray-cap (commit 2)
Root was recall-only: any matching √root counted as correct with no penalty for the wrong roots also sprayed — rewarding hedging. (#117 hardened MORPH's denominator against exactly this but left ROOT open.) A hit buried in
> authority + ROOT_SPRAY_SLACK(2)roots is nowsprayed, not correct; a focused answer (right root + a little slack) still counts.Verification
tscat the 17-error baseline; full sutta-studio suite green (60).With this + #120 (the wasted-spend fix), all audit findings that affect the paid run are closed. The only remaining gate is the twelve model IDs.
🤖 Generated with Claude Code