Skip to content

parse_email_tree: full MIME part-tree API with nested message/rfc822 support and stdlib-parity walk() #99

Description

@kurok

Problem

PyMail is a flattened projection — bodies here, attachments there — and the open classification bugs (#22 phantom parts, #25 ignored Content-Disposition) are symptoms of a deeper limitation: any flattening policy is wrong for someone. Forensics tooling needs the exact MIME topology; nested message/rfc822 (bounced/forwarded mail — ubiquitous in abuse pipelines) is currently smashed into the parent's attachment list with its structure lost; multipart/alternative vs multipart/mixed distinctions (which HTML body corresponds to which plain-text sibling) are unrecoverable. Python's stdlib email has walk(); the fast library should not force a fallback to the slow one for structure.

Proposal

parse_email_tree — the structural API (additive; parse_email unchanged)

node = parse_email_tree(payload)     # PyMimePart

# PyMimePart:
#   content_type: str            # "multipart/alternative", "text/plain", ...
#   headers: ...                 # this part's headers (same header semantics as PyMail post-#23)
#   disposition / filename / content_id
#   children: list[PyMimePart]   # empty for leaves
#   content: bytes | None        # decoded leaf content (None for multipart containers)
#   is_message: bool             # message/rfc822 — children contain the parsed inner message's root

Acceptance criteria

  • Tree structure matches stdlib email.message_from_bytes(...).walk() topology (content types in order) for the full test corpus + RFC corpus (parity test — the strongest correctness oracle available).
  • message/rfc822 fixtures (single and doubly-nested bounce) expose inner subject/headers/bodies via the subtree; recursion cap fires at the documented depth with a clear ParseError (extends test_dos_limits.py).
  • Leaf content equals the corresponding PyMail body/attachment bytes where both APIs expose the part (cross-API consistency test).
  • multipart/alternative fixture: plain and HTML siblings are children of the same node (the relationship PyMail loses — asserted).
  • walk() order stdlib-identical (fixture test); generator is lazy (no full materialization of a deferred-mode tree on iteration).
  • parse_email outputs across the corpus unchanged (pure addition).
  • Stubs, README (a "structure vs convenience: which API when" section), CHANGELOG; mypy --strict/ruff/clippy/fmt clean; benchmark showing tree parse overhead ≤ 10% vs flat parse on the corpus.

Out of scope

Part of the roadmap (see tracking issue). Coordinates with #21 (shared recursion caps), #22/#25 (gives their fix a precise vocabulary), and the lazy-parsing issue (shared modes).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions