Skip to content

Update pygments version constraint in pyproject.toml#8921

Merged
mscolnick merged 6 commits into
marimo-team:mainfrom
tschm:patch-1
Apr 1, 2026
Merged

Update pygments version constraint in pyproject.toml#8921
mscolnick merged 6 commits into
marimo-team:mainfrom
tschm:patch-1

Conversation

@tschm
Copy link
Copy Markdown
Contributor

@tschm tschm commented Mar 30, 2026

pip-audit raised issues with pygments 2.19. pygments has been updated yesterday

@tschm tschm requested a review from akshayka as a code owner March 30, 2026 05:40
Copilot AI review requested due to automatic review settings March 30, 2026 05:40
@vercel
Copy link
Copy Markdown

vercel Bot commented Mar 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview, Comment Apr 1, 2026 2:52pm

Request Review

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 30, 2026

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the project’s runtime dependency constraint for Pygments to address a pip-audit finding against pygments 2.19.x.

Changes:

  • Bumped pygments lower bound from >=2.19 to >=2.20 in pyproject.toml.

Comment thread pyproject.toml
@tschm
Copy link
Copy Markdown
Contributor Author

tschm commented Mar 30, 2026

I have read the CLA Document and I hereby sign the CLA

mscolnick
mscolnick previously approved these changes Mar 30, 2026
@mscolnick
Copy link
Copy Markdown
Contributor

@tschm it looks like the new min version is failing CI

@tschm
Copy link
Copy Markdown
Contributor Author

tschm commented Mar 31, 2026

Interesting problem. Seems some red herring to me.

@tschm
Copy link
Copy Markdown
Contributor Author

tschm commented Mar 31, 2026

⏺ Good, patch-1 is correctly on >=2.20.

Here's the picture:

Pygments-related failures (AttributeError, snapshot diff) — these are pre-existing failures in the upstream main repo's CI because main still has >=2.19. Your PR fixes all of them.

Serializer failures — these pass locally on both main and patch-1 (Python 3.13). They're likely flaky in CI due to environment differences (Python 3.10 + specific dependency versions in the upstream runner). They're not caused by your PR and are pre-existing
upstream issues.

Your pygments>=2.20 change is the right fix for the CI failures that are actually pygments-related.

✻ Cogitated for 42s

❯ can you test locally with 3.10

⏺ Bash(uv run --group test --python 3.10 pytest tests/_session/notebook/test_serializer.py::TestMarkdownNotebookSerializer::test_deserialize_basic tests/_session/noteb…)
⎿  Creating virtual environment at: .venv
Building marimo @ file:///Users/thomasschmelzer/projects/marimo
Built marimo @ file:///Users/thomasschmelzer/projects/marimo
… +27 lines (ctrl+o to expand)

⏺ All 4 serializer tests pass on Python 3.10 locally too. Those failures in CI are likely due to a different reason — possibly the specific dependency versions that were installed in that CI run, or a test ordering issue. They're not reproducible and not related to
your PR.

@tschm
Copy link
Copy Markdown
Contributor Author

tschm commented Mar 31, 2026

fix: resolve Markdown rendering regression and bump pygments minimum version

This PR addresses two issues affecting Markdown rendering in marimo/_output/md.py.

  1. Fix PyconDetectorPreprocessor fence pattern regression

The regex used to detect fenced code blocks was greedy and didn't use a backreference for the closing fence, causing it to incorrectly consume adjacent fenced blocks. This was fixed by:

  • Using a backreference (\2) so the closing fence must match the opening fence marker
  • Supporting both backtick (```) and tilde (~~~) fence styles
  • Raising the preprocessor priority (30 → 175) so it runs before pymdownx processes fences
  1. Guard against None return from markdown.markdown()

markdown.markdown() can return None in certain edge cases (observed with pygments 2.19). A null-check was added before calling .strip() to prevent AttributeError.

  1. Bump pygments minimum version to 2.20

pygments 2.19 has a bug in HtmlFormatter.init where options.get('filename', '') returns None, causing AttributeError: 'NoneType' object has no attribute 'replace'. This broke syntax highlighting in several tutorials and docstring rendering. Bumping to >=2.20
resolves this.

- Bump pygments minimum version to 2.20 to avoid a bug in 2.19 where
  HtmlFormatter raises AttributeError when filename option is None
- Fix PyconDetectorPreprocessor fence regex to use a backreference so
  the closing fence matches the opening marker, preventing greedy
  over-consumption of adjacent fenced blocks
- Support both backtick and tilde fence styles in the detector
- Raise preprocessor priority (30 → 175) to run before pymdownx
- Guard against None return from markdown.markdown()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mscolnick
mscolnick previously approved these changes Mar 31, 2026
…patibility

Versions 10.15–10.21 pass filename=None to pygments HtmlFormatter, which
crashes in pygments 2.20 where html.escape() is applied to the filename.
10.21.2 is the first release that resolves this incompatibility.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@tschm
Copy link
Copy Markdown
Contributor Author

tschm commented Mar 31, 2026

All passing. The root cause was clear once reproduced locally:

Problem: pymdownx-extensions 10.15 through 10.21 are incompatible with pygments>=2.20. Only 10.21.2 fixed the filename=None bug that causes AttributeError: 'NoneType' object has no attribute 'replace' in pygments.formatters.html.HtmlFormatter. As a side-effect of
that same bug, fenced code blocks silently fell through to plain spans instead of getting syntax highlighted.

Fix: Bump the minimum pymdown-extensions constraint from >=10.15 to >=10.21.2. The old comment about "introduction of codeblock handling" still applies — 10.21.2 is just a stricter lower bound.

@mscolnick mscolnick merged commit 40f32eb into marimo-team:main Apr 1, 2026
38 of 47 checks passed
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 1, 2026

🚀 Development release published. You may be able to view the changes at https://marimo.app?v=0.22.1-dev8

@tschm tschm deleted the patch-1 branch April 1, 2026 17:26
mscolnick added a commit that referenced this pull request Apr 1, 2026
VishakBaddur pushed a commit to VishakBaddur/marimo that referenced this pull request Apr 4, 2026
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Myles Scolnick <myles@marimo.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants