Improve full markdown output around complex table regions

[StevenVincentOne]

Problem

On some table-heavy PDFs, the default markdown output in full mode can degrade nearby reading order around complex tables.

Using DeepSeek_R1.pdf as a repro, the markdown around Table 4 contains a mix of:

• fake footnote bullets like - 1https://example.com
• flattened benchmark header/value runs
• a dangling narrative fragment before the next real subsection heading

This means the issue is not only table fidelity. The default markdown output can also disrupt surrounding prose/section structure in the table region.

Reproduction

Source PDF:

• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• DeepSeek-AI et al.

Observed full-mode markdown shape around Table 4:

##### 3.1. DeepSeek-R1 Evaluation

- 1https://example.com
- 2https://example.com
- 3https://example.com

Claude-3.5- GPT-4o DeepSeek OpenAI OpenAI DeepSeek
...
Table 4 | Comparison between DeepSeek-R1 and other representative models.
...
performance of DeepSeek-R1 will improve in the next version...

##### 3.2. Distilled Model Evaluation

Expected behavior

In full mode, ODL should preserve surrounding reading order while still attempting to emit rich table markdown.

That means table-adjacent artifacts like the fake footnote bullets and dangling prose fragment should not leak between valid subsection boundaries.

Scope

This is intentionally separate from the new markdown-table-output PR that adds caption_only / off modes.

That PR provides a safe alternative for consumers who want to suppress tables.
This issue is about improving the default full markdown behavior itself.

Related

• PR: Add markdown table output modes #317 Add markdown table output modes
• Related issue: Recover flattened benchmark tables emitted as paragraph runs #259 Recover flattened benchmark tables emitted as paragraph runs

[bundolee]

🤖 AI Issue Fix — status: triaged

Analysis: this issue requires large-scale development beyond automated fix scope.
Root cause is in TableBorderProcessor.addContentToTableBorder() — bounding box intersection logic causes footnotes/prose to contaminate table regions. Fixing this requires reworking the coordinate-based content assignment pipeline.
Related to #259 (flattened benchmark tables) but distinct in scope — #259 targets table structure recovery, #318 targets surrounding prose corruption.
Label enhancement applied. Requires human implementation.

---

Thanks for the detailed report and reproduction steps. I analyzed the markdown output pipeline and traced the issue to how content gets assigned to table regions based on bounding box intersection in the TableBorderProcessor. When footnote URLs and nearby prose overlap the table's coordinate boundaries, they get incorrectly pulled into the table region, which disrupts the reading order around the table.

This is a significant improvement that requires reworking how the processor assigns content to table vs. non-table regions — it goes beyond what can be addressed as a quick fix. I've labeled this as an enhancement for planned development. It's related to but distinct from #259, which focuses on recovering the table structure itself.