week_1-3: Module C — real 319-row golden dataset from OpenCRE DB

[PRAteek-singHWY]

What this PR is

The third and final PR of Module C, Week 1. It replaces the 10-row seed dataset from week_1-2 with the real golden dataset — 319 rows derived from standards_cache.sqlite (the OpenCRE graph the team already curated). This is the actual ruler every later C PR will be measured against in CI.

Stacked on gsocmodule_C_week_1_2. When PR 2 merges, GitHub re-targets this to main.

Why this PR is the most important one in Week 1

W1 PR 1 nailed the contracts. W1 PR 2 nailed the harness scaffolding. But neither has a real measurement. The 10-row seed in PR 2 only proved the wiring; with this PR landed, every later C PR is measured against 319 real rows with real DB-grounded ground truth, and an accuracy drop > 2pts blocks the PR. That's the W1 promise from your golden-dataset-plan.md: "It makes a mapping change behave like a code change: versioned, measured, and revertible."

What's in this PR, step by step

1. The build script — scripts/build_golden_dataset.py

A deterministic derivation from standards_cache.sqlite:

• Joins node → cre_node_links → cre to pull ground-truth labels. The OpenCRE graph is the ground truth — per golden-dataset-plan.md §3, "Ground truth = the existing OpenCRE DB mappings."
• Every query has an explicit ORDER BY so output is byte-stable.
• --check mode re-derives and exits non-zero if the committed JSON has drifted from what the DB now produces. CI runs this to guarantee the JSON tracks live mappings.
• All curated rows (explicit / update) pin a real ASVS section_id and fetch the real CRE external_id at build time — text is synthesized but ground truth stays real.

2. The dataset — application/tests/librarian/fixtures/golden_dataset.json

319 rows total, by slice:

Slice	Count	Source
positive (1:1)	277	All 277 ASVS requirements + their single mapped CRE (DB-grounded)
positive (multi-link)	15	OWASP Top 10 2021 + CWE nodes with 2–4 CRE mappings — exercises the Q-D rule with real data
hard_negative	12	ASVS reqs whose text contains do not / does not / shall not / should not, with real DB CRE (cross-encoder must beat cosine without losing the right CRE)
explicit	5	Synthesized text that literally cites a real cre.external_id
update	5	Synthesized before/after rewordings of real ASVS reqs, real CRE id
ambiguous	5	Broad SDLC / governance / culture statements; decision=review, reason_code=BELOW_THRESHOLD / NO_CANDIDATES

Every row carries an honest provenance.ground_truth_source recording whether the ground truth came from the DB (most rows) or was manually synthesized (the broad ambiguous slice).

3. The validation + determinism tests — application/tests/librarian/dataset_test.py

What CI now guarantees, on every PR:

• Every row validates against both GoldenDatasetRow (Pydantic) and golden_dataset.schema.json (jsonschema).
• At least 277 positive rows are present (the master-guide §9.3 hard floor).
• All 5 slices are present and each has ≥5 rows so the harness can stratify.
• At least one multi-link positive row exists — otherwise the Q-D rule has nothing to operate on and is dead code.
• Every row carries provenance.ground_truth_source.
• All row ids are unique.
• Determinism: build_golden_dataset.py --check against the committed JSON exits 0. (Auto-skipped if the DB file is absent.)

4. Multi-link rule consistency

Per your proposal's Q-D rule (scoring.py from week_1-2):

Jaccard(expected, predicted) ≥ 0.5 AND top-1 in expected.

The dataset honours this by including 15 genuine multi-link positive rows from OWASP Top 10 and CWE, where the expected cre_ids list has 2–4 entries pulled straight from the DB. The hard floor is set so that this rule has real data to exercise, not just a single hand-crafted multi-link example.

How to verify locally

# all librarian tests pass on the real 319-row dataset
python3 -m unittest discover -s application/tests/librarian -p '*_test.py' -t .
# expected: 48 passed

# determinism — re-derive and prove no drift
python3 scripts/build_golden_dataset.py --check
# expected: "OK: golden dataset matches derivation (319 rows)"

# harness runs end-to-end on the real dataset
python3 scripts/evaluate_librarian.py --dataset application/tests/librarian/fixtures/golden_dataset.json
# expected:
#   loaded 319 golden rows from .../golden_dataset.json
#     ambiguous      5
#     explicit       5
#     hard_negative  12
#     positive       292
#     update         5
#   hub-firewall: ON; stripped 319 leaking hub entries
#   correct (W1 stub, no predictions): 5/319

# regenerate the dataset (idempotent against the same DB)
python3 scripts/build_golden_dataset.py

The 319 stripped leaking hub entries is exactly the point of the firewall — the W1 stub hub is seeded from the rows' own text, so without the firewall every row would leak. 5/319 correct is the W1 baseline: the 5 ambiguous rows have no expected cre_ids so empty predictions match; the 314 linked-decision rows correctly show as not-yet-predicted (real predictions arrive when the pipeline lands W3–W6).

What this PR does NOT claim

• Does not re-validate or republish the locked pre-code +33.3% / +28.6% negative top-1 numbers. Those are your separate locked experiment; this is the ongoing CI ruler.
• Does not include any retriever / ranker / embedding / linking code — that's W3 onwards.
• Does not modify any RFC-side schema. Internal GoldenDatasetRow was nailed down in week_1-1.

Why some slices are smaller than others

• WSTG and most other standards have empty description fields in the DB — no usable text. That's why multi-link rows come from OWASP Top 10 + CWE (the standards with both multi-CRE mappings and real text), not WSTG.
• ASVS uses positive phrasing ("Verify that X is not Y") rather than imperative negation ("Do not Y"), so the 12 negation hits found are exactly the rows that legitimately matched. Expanding this slice synthetically would dilute the signal — better 12 real rows than 30 fabricated ones.

---

That closes Week 1: contracts (PR 1) → harness (PR 2) → real ruler (PR 3). Mentor confirmation of Q-D at the Friday call unblocks merging PR 2 and the multi-link reasoning in this PR. After that, week 2 starts the deterministic stage — the SectionNormalizer (RFC contract → internal Section, no ML yet).