week_1-1: Module C — Pydantic contracts + config loader

[PRAteek-singHWY]

What this PR is

This is the first of three planned PRs for Module C, Week 1 (May 25–31). It lands the boundary layer for the Librarian — the data contracts the module reads and writes, plus the test scaffolding that will keep us honest as the pipeline grows. There is no linking logic, no embeddings, no decisions in this PR; that comes in week_1-2 (eval harness + hub-firewall + scoring) and week_1-3 (real 277-row golden dataset). Per Spyros's guidance, W1 is contracts and tests before any pipeline code, so this PR is deliberately small and behaviour-neutral for OpenCRE proper.

Why this PR exists at all

Module C consumes a JSON envelope from Module B (Manshu) and emits two JSON envelopes — one to the OpenCRE graph and one to Module D's review queue. Those shapes are defined in RFC OWASP#734 (docs/owasp-graph/apis/schemas/*.json). If we start writing the C pipeline before pinning down the contracts in Python types, every later PR risks rework when Spyros's RFC settles. So W1's job is just:

1. make those JSON shapes real Pydantic types,
2. prove they actually match the RFC (not what a summary doc says), and
3. wire up the CRE_LIBRARIAN_* config the later weeks will read.

That is exactly what this PR does.

What's in this PR, step by step

1. The librarian package skeleton

application/utils/librarian/ is a new module. Its __init__.py documents the three contract directions (B → C, C → graph, C → D) and pins the upstream RFC commit we're tracking (upstream/owasp-graph @ 2b143798) with the resync command, so any contributor can re-pull the canonical schemas if Spyros updates PR OWASP#734.

2. The v0.2.0 RFC contracts in schemas.py

Modeled directly from PR OWASP#734's JSON schemas, not from the master-guide summary:

• `KnowledgeItem` — what Module B emits for one chunk. Full RFC envelope: `event_id`, `filtered_at`, `status ∈ {accepted, rejected, deferred}`, `source`, `locator`, a `content` block (`text`, `title_hint`, `keywords`, `language`), and a `filter` block whose `stages[]` records the regex/LLM pipeline plus scores. Conditional rules from the RFC's `allOf` block — `status=accepted` requires `content`, `status=rejected` requires `rejection` — are enforced as Pydantic `model_validator`s.
• `LinkProposal` — C's auto-link output. `status` is fixed to `"linked"` (`Literal` const), `pipeline_run_id` and `update_detection` are required (the RFC says so), `links` must be non-empty, `additionalProperties: false`.
• `ReviewItem` — C's human-review output. `status` fixed to `"review_required"`; `reason_code` is the 4-value enum from the RFC (`BELOW_THRESHOLD`, `NO_CANDIDATES`, `ADVERSARIAL_FLAG`, `UPDATE_AMBIGUOUS`).
• Shared sub-models — `SourceRef` (with the conditional `type=github → repo + commit_sha`), `Locator` (with `kind=repo_path → path`, `kind=url|feed_item → url`), `KnowledgeSnapshot`, `RetrievalAudit`, `CreCandidate` (including `score_hybrid` for any future BM25 hybrid), `ProposedLink`, `UpdateDetection`.

Every required field, conditional rule, enum, and `additionalProperties: false` constraint from the canonical schemas is enforced.

3. Internal-only models (NOT part of the RFC)

• `KnowledgeQueueItem` — a Pydantic mirror of Module B's `knowledge_queue` Postgres row, per master guide §1.2. At W8, C will read these rows and synthesize the RFC `KnowledgeItem` envelope from them. This is just an internal contract between B's SQL and C's Python.
• `GoldenDatasetRow` — Pydantic model of the regression-harness row defined in `gsoc-notes/bonding/golden_dataset.schema.json`. The harness itself lands in `week_1-2`.

4. The drift guard — vendored canonical schemas + a round-trip test

The single biggest risk: my Pydantic models drift from Spyros's RFC and we don't notice until weeks later. The fix:

• The 6 canonical RFC schemas are vendored verbatim under `application/utils/librarian/_rfc_schemas/`, pinned to a specific upstream SHA.
• `schemas_test.py` dumps each Pydantic contract to JSON and validates it against the canonical schema using `jsonschema.Draft202012Validator` + a `referencing.Registry` that resolves `$ref`s by `$id`. If a Pydantic model ever produces something the RFC schema rejects (or vice versa), this test fails the build.
• It also re-validates the literal `ReviewItem` example from `docs/owasp-graph/apis/module-c-librarian.md`, so we catch drift in the prose example too.

This is the guardrail that protects every later C PR from silently diverging.

5. The config loader

`config_loader.py` reads the 7 `CRE_LIBRARIAN_*` env vars into a frozen dataclass:

Env var	Default
`CRE_LIBRARIAN_CROSSENCODER_MODEL`	`cross-encoder/ms-marco-MiniLM-L-6-v2`
`CRE_LIBRARIAN_TOP_K_RETRIEVAL`	`20`
`CRE_LIBRARIAN_TOP_K_RERANK`	`5`
`CRE_LIBRARIAN_LINK_THRESHOLD`	`0.8`
`CRE_LIBRARIAN_BATCH_SIZE`	`32`
`CRE_LIBRARIAN_ECE_TARGET`	`0.10`
`CRE_LIBRARIAN_CONFORMAL_ALPHA`	`0.10`

Tests cover defaults, env overrides, frozen behaviour, and invalid-int handling.

6. Real B-SQL-row fixture

`fixtures/sample_knowledge_queue.jsonl` has 3 mock rows in the actual `knowledge_queue` column shape (`id`, `source_repo`, `source_path`, `source_commit_sha`, `text`, `confidence`, `llm_label`, `llm_reasoning`, `created_at`, `consumed_at`) — so future tests work against realistic input rather than a guess.

7. Pydantic pin

One-line change to `requirements.txt`: `pydantic>=2,<3` (was unpinned). This PR relies on Pydantic v2's `model_validator` and `ConfigDict`.

What is deliberately NOT in this PR

• Hub-firewall, eval harness, scoring, knowledge-source stub → land in `week_1-2`.
• The real 277-row `golden_dataset.json` → lands in `week_1-3` (derived from `standards_cache.sqlite`'s `cre_node_links`).
• No edits to `config.py`, `cre.py`, `cre_main.py`, `db.py`, or any migration. Those are W3 (CLI wiring) and W8 (DB models / live B→C integration) per master-guide §8.2.
• No retriever / embeddings / cross-encoder code. W3 onwards.

How to verify locally

```bash
python3 -m unittest discover -s application/tests/librarian -p '*_test.py' -t .
```
Expected: 31 passed. The suite covers the RFC round-trips (`LinkProposal`, `ReviewItem`, `KnowledgeItem` accepted + rejected, and the `module-c-librarian.md` example), all the conditional rules, `additionalProperties: false`, config defaults + overrides, and the internal `KnowledgeQueueItem` + `GoldenDatasetRow`.

Black-clean. No existing repo test affected (no production file is touched beyond the one-line pydantic pin).

Open question for mentors (does NOT block this PR)

Q-D — multi-link scoring rule. When a chunk maps to several CREs, what counts as "correct" in the regression harness? My proposed default (which lands in `week_1-2`): Jaccard ≥ 0.5 of the expected CRE set AND top-1 in the set. Confirmation appreciated at the Friday call — this blocks `week_1-2`'s merge, not this PR (there's no scoring code here yet).

Resyncing if the RFC changes

The vendored schemas are pinned to `upstream/owasp-graph @ 2b14379`. If Spyros pushes new commits to PR OWASP#734:
```bash
git fetch upstream owasp-graph
for f in link-proposal review-item knowledge-item proposed-link source-ref locator; do
git show upstream/owasp-graph:docs/owasp-graph/apis/schemas/$f.json \
> application/utils/librarian/_rfc_schemas/$f.json
done

update the SHA in application/utils/librarian/init.py, then re-run the tests

```
The drift test will surface any change that needs a Pydantic-side update.

---

Following the Slack-agreed convention: branch `gsocmodule_C_week_1_1` off `main` of the fork, PR title `week_1-N: ` with numbering for multiple PRs in a week. `week_1-2` (harness) and `week_1-3` (golden set) will follow.