Skip to content

feat(staging): render list bullets and title heading depth in markdown element_to_md() function #4384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aaronsteers wants to merge 1 commit into
base: main
Choose a base branch
from
+24 −2
Open

feat(staging): render list bullets and title heading depth in markdown element_to_md() function #4384

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions test_unstructured/staging/test_base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -592,6 +592,19 @@ def test_elements_to_md_conversion(json_filename: str, expected_md_filename: str
("element", "expected_markdown", "exclude_binary"),
[
(Title("Test Title"), "# Test Title", False),
(
Title("Second Level", metadata=ElementMetadata(category_depth=2)),
"## Second Level",
False,
),
(
Title("Third Level", metadata=ElementMetadata(category_depth=3)),
"### Third Level",
False,
),
(Title("No Depth"), "# No Depth", False),
(ListItem("Buy milk"), "- Buy milk", False),
(ListItem("Another item"), "- Another item", False),
(NarrativeText("This is some narrative text."), "This is some narrative text.", False),
(Formula(r"\int_a^b x^2 dx"), "$$\n\\int_a^b x^2 dx\n$$", False),
(
Expand Down
13 changes: 11 additions & 2 deletions unstructured/staging/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
ElementMetadata,
Formula,
Image,
ListItem,
Table,
Title,
)
Expand Down Expand Up @@ -278,8 +279,16 @@ def element_to_md(
formula_markdown_style: str = FORMULA_MARKDOWN_AUTO,
) -> str:
match element:
case Title(text=text):
return f"# {text}"
case Title(text=text, metadata=metadata):
depth = metadata.category_depth if metadata.category_depth else 1

@cubic-dev-ai cubic-dev-ai Bot Jul 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: The new Markdown heading logic is off-by-one because metadata.category_depth is zero-indexed in this codebase, but the code treats it as the literal Markdown heading level. For example, an h2 (category_depth=1) renders as # instead of ##, and an h3 (category_depth=2) renders as ## instead of ###. This undermines the PR's goal of adding L2/L3 header support.

Also, the converted depth is not clamped to Markdown's valid 1-6 ATX heading range, so edge-case values like 0, negatives, or values greater than 5 can produce invalid or oversized heading markers. Consider computing the Markdown depth as max(1, min(category_depth + 1, 6)) and treating None as the top-level default (0).

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At unstructured/staging/base.py, line 283:

<comment>The new Markdown heading logic is off-by-one because `metadata.category_depth` is zero-indexed in this codebase, but the code treats it as the literal Markdown heading level. For example, an h2 (category_depth=1) renders as `#` instead of `##`, and an h3 (category_depth=2) renders as `##` instead of `###`. This undermines the PR's goal of adding L2/L3 header support.

Also, the converted depth is not clamped to Markdown's valid 1-6 ATX heading range, so edge-case values like `0`, negatives, or values greater than 5 can produce invalid or oversized heading markers. Consider computing the Markdown depth as `max(1, min(category_depth + 1, 6))` and treating `None` as the top-level default (0).</comment>

<file context>
@@ -278,8 +279,16 @@ def element_to_md(
-        case Title(text=text):
-            return f"# {text}"
+        case Title(text=text, metadata=metadata):
+            depth = metadata.category_depth if metadata.category_depth else 1
+            if not isinstance(depth, int):
+                try:
</file context>

if not isinstance(depth, int):
try:
depth = int(depth) if isinstance(depth, (str, float)) else 1
except (TypeError, ValueError):
depth = 1
return f"{'#' * depth} {text}"
case ListItem(text=text):

@cubic-dev-ai cubic-dev-ai Bot Jul 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The new ListItem branch renders a Markdown list item by simply prepending - to the text. If ListItem.text contains an embedded newline, the generated Markdown becomes - first line\nsecond line, where the continuation line starts at column 0 and is parsed as a separate paragraph rather than part of the list item. Since ListItem is a plain text element and the serializer does not normalize incoming text, a multiline value from a partitioner or deserialization will corrupt the Markdown output. Consider collapsing embedded newlines to spaces or indenting continuation lines so they stay inside the list item.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At unstructured/staging/base.py, line 290:

<comment>The new `ListItem` branch renders a Markdown list item by simply prepending `- ` to the text. If `ListItem.text` contains an embedded newline, the generated Markdown becomes `- first line\nsecond line`, where the continuation line starts at column 0 and is parsed as a separate paragraph rather than part of the list item. Since `ListItem` is a plain text element and the serializer does not normalize incoming text, a multiline value from a partitioner or deserialization will corrupt the Markdown output. Consider collapsing embedded newlines to spaces or indenting continuation lines so they stay inside the list item.</comment>

<file context>
@@ -278,8 +279,16 @@ def element_to_md(
+                except (TypeError, ValueError):
+                    depth = 1
+            return f"{'#' * depth} {text}"
+        case ListItem(text=text):
+            return f"- {text}"
         case Formula(text=text):
</file context>

return f"- {text}"
case Formula(text=text):
return _emit_formula_markdown(
text,
Expand Down