Skip to content

Commit 209f867

Browse files
vsits-proxy-builder[bot]vsits-proxy-builder[bot]claude
authored
docs(agents): require an evidence class on every review finding (#289)
The global baseline says to distinguish confirmed from assumed. That is self-assessed: a reviewer who read a code path and found it plausible files it honestly under "confirmed." Findings now carry Measured (name the command, paste the result) / Read (cite file:line, say it is a read) / Reported (name the source, say you did not reproduce). A load-bearing claim from a PR body may not be repeated as fact without independent measurement. Binds every reviewer including our own maintainer comments, not just Codex. Motivating case is recorded in the section: on #270 a second model family endorsed an agent-id claim that measurement destroyed. Both figures cited are dated, and the current one was re-measured against today's log (still 0 populated rows, now of 184,976) rather than quoted from the older review. Claude-Session: https://claude.ai/code/session_01GvZKP1JeXgHFCovTaAPT5B Co-authored-by: vsits-proxy-builder[bot] <223447982+vsits-proxy-builder[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 0817302 commit 209f867

1 file changed

Lines changed: 49 additions & 0 deletions

File tree

AGENTS.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,55 @@ transform pipeline sitting in front of `api.anthropic.com`.
101101
unit / integration tests alone can't prove behavior under real
102102
traffic.
103103

104+
## Evidence Class (every finding, every reviewer)
105+
106+
The global baseline asks you to distinguish what is **confirmed correct**
107+
from what is **assumed or hypothesized**. That is a disposition, and a
108+
disposition is self-assessed: a reviewer who read a code path and found
109+
it plausible will file it honestly under "confirmed."
110+
111+
So state *how* you know, not just that you know. Tag every finding —
112+
blocking or not — with one of three classes:
113+
114+
- **Measured** — you ran something. Name the command and paste the
115+
result. `node --test test/proxy-read-dedupe.test.mjs`
116+
`actual: 'insertion-normalization', expected: 'cache-control-normalize'`.
117+
- **Read** — you read the code. Cite `file:line`, and say plainly that
118+
it is a code read. Reading proves a path *exists*, not what it does on
119+
real traffic.
120+
- **Reported** — the author's claim, or another agent's. Name the source
121+
and say you did not reproduce it. Never restate it as fact.
122+
123+
**The hard rule:** a load-bearing claim from a PR body may not be
124+
repeated as fact in a review without independent measurement. If you
125+
cannot measure it, mark it *Reported* and say what would settle it.
126+
127+
This binds every reviewer — Codex, the implementation agent's own
128+
maintainer comments, and any third model added later. A finding relayed
129+
without its class is not usable by the next reader.
130+
131+
### Why this rule exists
132+
133+
On PR #270 the reviewer **endorsed** a claim about agent-id availability
134+
that measurement later destroyed. Measured 2026-06 during that review:
135+
the canonical header was present on 38 of 121,685 requests (0.03%), and
136+
0 of 176,344 usage rows were populated. Re-measured 2026-07-31 against a
137+
now-larger log: still **0 of 184,976**. The claim was plausible, the code
138+
path existed, and a second model family agreed with it. Cross-family
139+
review did not catch it. One query did.
140+
141+
Two consequences worth internalizing:
142+
143+
- **A reviewer agreeing is not verification.** Correlated plausibility
144+
is what agreement measures.
145+
- **Measurement is usually cheap.** The two queries that settled #270's
146+
design were one command each; run on day one they would have prevented
147+
most of five review rounds.
148+
149+
When several reviewers are on one PR, the classes are what make their
150+
findings cheap to reconcile — *Measured* from one and *Read* from
151+
another on the same point is a signal, not a contradiction.
152+
104153
## Anti-Bloat Lens (no-directive PRs)
105154

106155
The global baseline's bloat bar is *"larger than the directive's

0 commit comments

Comments
 (0)