Add test suite: 92% coverage on the core library

[shaunpatterson]

Brings the importable core library from ~19% to 92% test coverage, and adds a coverage config that scopes the target sensibly.

Note: this branch is stacked on the four fix PRs (#2–#5) — it merges them so the new tests assert the fixed behavior. Once those merge, this rebases to just the test additions + config.

What's added

Pure-logic and mockable tests (no heavy ML deps, no network, no model downloads):

• splitters.py (99%) — both merge modes, backward order, overlap re-splitting, regex separators, empty-string binary split, validation errors, group_chunks/group_pages/combine_blocks/regex_splitter.
• postprocessing.py (99%) — page/title info, gap check/repair, oversized-split & small-chunk merges, and the *_from_df drivers with tiny parquet fixtures.
• metrics.py (77%) — block integrity, missing-ref errors, cohesion/coherence/dissimilarity with a deterministic fake embedding model, sentence_transformers mocked via sys.modules, real scikit-learn for the lexical metric, and the pure coref helpers.
• jina_embedder.py (80%) — happy-path + normalization with httpx mocked.
• pipeline.py (94%) — chunk_files end-to-end with a fake parser.
• compute_metrics / split_documents / extract_mentions (99–100%) — directory drivers with fake splitters/models and parquet/JSON fixtures.
• chunking_utils.py (90%).

Coverage config ([tool.coverage] in pyproject)

Scopes measurement to the library and omits:

• paper/* — one-off research replication scripts, not part of the API.
• parsing.py — thin adapters over Azure DI / Docling / PyMuPDF, which are heavy optional dependencies needing model downloads (still exercised by test_parsing.py, just not measured).

The remaining uncovered gap is the spaCy/maverick coreference internals in metrics.py (extract_entity_pronoun_pairs, CoreferenceSolver.__init__/find_mentions), which require real models to run.

Result

pytest: 211 passed, 1 skipped; 92% on the scoped target.

🤖 Generated with Claude Code

[Avo-k]

The four fix PRs (#2–#5) are now merged into main — thanks again for the whole series, this is exactly the kind of contribution we hope for. Could you rebase this branch on main so it shows just the test additions + coverage config? We're also setting up CI shortly, so this suite will become the merge gate for the repo. We'll review right after the rebase.

[Avo-k]

Security pass + full review done — this is a great suite. The claims check out exactly on our side: 92% coverage measured, 211 tests green in ~7s on a local merge with main, everything runs without the heavy ML deps. The mock hygiene (hermetic httpx / sentence-transformers fakes, everything under tmp_path) made the audit easy.

Two small changes before merge — and if you prefer, we're happy to apply both ourselves at merge time, just say the word:

1. test_gpu_memory_stats_raises_importerror_without_torch fails when torch is installed (it asserts ImportError, but a full dev env has torch). A conditional skip would fix it, e.g. @pytest.mark.skipif(importlib.util.find_spec("torch") is not None, reason="torch installed").
2. _inject_fake_sentence_transformers (test_metrics_extra.py) assigns sys.modules["sentence_transformers"] directly and never restores it, so the fake can leak into any later test that imports the real package. monkeypatch.setitem(sys.modules, "sentence_transformers", st) keeps it scoped per-test.

Two notes, no action needed in this PR:

• Your test_merge_with_existing_parquet tests correctly characterize a real duplication quirk in the *_from_df merge path — nice catch. Opened #8 to fix it source-side; once that lands, those assertions can tighten.
• Heads-up for the rebase (still needed — the branch predates today's main): main now has a CI workflow (.github/workflows/ci.yml) and a [test] extra in pyproject, so you'll see small conflicts there. After the rebase, this PR's CI runs should be green out of the box except the two points above.