Skip to content

Stdlib-parity compatibility suite, migration guide, and honest benchmark table #103

Description

@kurok

Problem

The fastest parser wins nothing if migrating to it is a research project. Today, moving a codebase from email (stdlib) or mail-parser/mailparser to fast_mail_parser means reverse-engineering semantic differences by trial: How do header names compare (case)? What does it do with RFC 2047 in display names vs stdlib? Which of the flattening quirks differ? The benchmark README sells the speed; nothing sells the safety of switching. A compatibility test suite + migration guide converts "interesting library" into "drop-in decision made in an afternoon" — and doubles as the correctness oracle that catches our own regressions against the ecosystem's reference behavior.

Proposal

1. Differential compatibility suite (tests/test_stdlib_parity.py)

  • For the full fixture + RFC corpus, parse each message with both fast_mail_parser and stdlib email (policy=default), and assert agreement on the comparable surface: subject (post-subject/date are derived from the lossy header map #28), body text sets, attachment (filename, mimetype, content) sets, header multiset (post-Duplicate headers silently dropped by HashMap<String,String> collapse #23), address parsing (with the typed-API issue), date interpretation.
  • Divergences become one of: a bug here (fix), a bug/limitation in stdlib (document as intentional divergence with fixture + rationale), or an RFC gray zone (document the choice). The output artifact is a docs/compatibility.md table — every known divergence from stdlib, stated, with a fixture proving it. Unknown-divergence count: zero, enforced by CI (new divergence = failing test until classified).

2. Migration guide

  • docs/migrating.md: side-by-side snippets for the common operations (stdlib email, mailparser (Python), and this library): read subject/from/to, walk parts, extract attachments, handle encoding errors. Honest limitations section (what stdlib does that this library doesn't — e.g. mutation, generation).

3. Benchmark expansion (the sales sheet, kept honest)

  • Extend the benchmark suite to include stdlib email and mailparser on the same corpus (currently only mail-parser is compared), publish the table in the README with corpus + methodology stated, and keep it regenerable via a make target so numbers never rot into fiction.

Acceptance criteria

  • Parity suite runs green over the full corpus with every divergence classified (fix / documented-intentional / gray-zone) — zero unclassified divergences, enforced as a test failure mode.
  • docs/compatibility.md generated/maintained with the divergence table, each row backed by a named fixture (spot-checkable).
  • Parity dimensions expand automatically as roadmap items land (typed addresses/date compare against stdlib's email.utils parsing when that issue merges — sequencing hooks noted in both issues).
  • docs/migrating.md covers the listed operations for all three source libraries with runnable snippets (doctest or CI-executed examples — snippets that rot are worse than none).
  • Benchmark table regenerated by make bench-table (or equivalent) with stdlib + mailparser columns; README updated; methodology (corpus, machine class, versions) stated alongside.
  • Suite marked appropriately so ordinary CI time stays sane (parity on push, full-matrix nightly if runtime demands splitting).
  • ruff/mypy --strict clean (pure Python addition); CHANGELOG entry.

Out of scope

  • API-compatibility shims (email.message.Message adapter) — document differences, don't emulate; an adapter is a separate library's job.
  • Windows-specific mail formats (TNEF/winmail.dat) — noted in the epic as a demand-driven candidate.

Part of the roadmap (see tracking issue). No dependencies to start; grows richer as #23/#28 and the typed-API issue land. Doubles as the regression oracle for the entire correctness backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or requestpriority: mediumMedium priorityroadmapRoadmap itemtestingTest coverage & quality

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions