Skip to content

fix: support multiline single-dollar math highlighting in markdown#304800

Open
yogeshwaran-c wants to merge 5 commits into
microsoft:mainfrom
yogeshwaran-c:fix/markdown-math-inline-multiline
Open

fix: support multiline single-dollar math highlighting in markdown#304800
yogeshwaran-c wants to merge 5 commits into
microsoft:mainfrom
yogeshwaran-c:fix/markdown-math-inline-multiline

Conversation

@yogeshwaran-c

Copy link
Copy Markdown
Contributor

What kind of change does this PR introduce?

Bug fix

What is the current behavior?

Multiline math expressions using single dollar signs ($...$) are not properly syntax-highlighted in markdown files. Only double dollar signs ($$...$$) are supported for multiline math blocks. For example:

$\mu = e^{\mu_L + \sigma_L^2}
 \land \sigma_2 = (e^{\sigma_L^2} - 1) e^{2\mu_L + \sigma_L^2}$

The second line loses its math highlighting because the math_inline_block pattern only matches $$.

Closes #223545

What is the new behavior?

The math_inline_block TextMate grammar pattern now supports both single ($) and double ($$) dollar sign delimiters for multiline math expressions. This is achieved by:

  1. Changing the begin pattern from \${2} (exactly 2 dollar signs) to \${1,2} (1 or 2 dollar signs)
  2. Using a backreference \1 in the end pattern to match the same delimiter that was used in the begin pattern
  3. Fixing the capture group references from "2" to "1" (the original had incorrect group numbers since there's only one capture group)

Additional context

The fix was suggested by @RedCMD in the issue comments. The math_inline_single and math_inline_double patterns (which use match instead of begin/end) continue to handle single-line math expressions. The math_inline_block pattern handles multiline math that spans across line breaks.

The math_inline_block pattern only matched $$ (double dollar) for
multiline math expressions. Single $ math that spans multiple lines
was not highlighted. Change the begin pattern to match 1 or 2 dollar
signs and use a backreference in the end pattern to match the same
delimiter. Also fix capture group references from "2" to "1".
The 'Use VS Code's Version' item in the TypeScript version picker was
using `defaultVersion` which could return a user-configured global
TypeScript SDK path instead of the actual bundled version. This caused
the picker to display incorrect version info and path when
`typescript.tsdk` was configured.

Additionally, the active indicator (bullet) used a heuristic based on
`useWorkspaceTsdkSetting` rather than actually comparing the current
version to the bundled version.

Fix by using `bundledVersion` directly and comparing against
`currentVersion` for the active indicator, consistent with how the
workspace version item already works.

Fixes microsoft#263226
@vs-code-engineering vs-code-engineering Bot added this to the 1.114.0 milestone Mar 25, 2026

private getBundledPickItem(): QuickPickItem {
const bundledVersion = this.versionProvider.defaultVersion;
const bundledVersion = this.versionProvider.bundledVersion;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems unrelated?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, sorry about that. I've reverted the TypeScript version manager changes in b87a6a0 — that file is now back to its state on main.

"name": "markup.math.inline.markdown",
"contentName": "meta.embedded.math.markdown",
"begin": "(?<=\\s|^)(\\${2})",
"begin": "(?<=\\s|^)(\\${1,2})",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also please add some tests for this. We have some basic highlighting tests in extensions/vscode-colorize-tests/test/colorize-fixtures/md-math.md. You can run these with the VSCode tokenizer tests debug configuration

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added test cases in b87a6a0. The new fixtures cover:

  • Multiline single-dollar math with complex expressions (the exact example from Wrong LaTeX highlighting in markdown #223545)
  • Multiline single-dollar math with \text{} commands
  • Both $$ and $ multiline blocks side by side to verify they don't interfere

The colorize baseline JSON will need to be regenerated by running the "VS Code Tokenizer Tests" debug configuration — I'll update the result file once I've run that.

@mjbvz mjbvz modified the milestones: 1.114.0, On Deck Mar 26, 2026
Remove the accidentally included TypeScript version picker changes.
Add tokenizer test fixtures for multiline single-dollar math highlighting
to cover the grammar fix for microsoft#223545.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@mjbvz mjbvz closed this May 1, 2026
@mjbvz mjbvz reopened this May 1, 2026
mjbvz
mjbvz previously approved these changes May 1, 2026
@mjbvz
mjbvz enabled auto-merge May 1, 2026 05:43
@RedCMD

RedCMD commented May 1, 2026

Copy link
Copy Markdown
Contributor

a bunch of tests are broken

roblourens
roblourens previously approved these changes May 1, 2026
@RedCMD

RedCMD commented May 1, 2026

Copy link
Copy Markdown
Contributor

still failing

Copilot AI review requested due to automatic review settings May 4, 2026 03:06
auto-merge was automatically disabled May 4, 2026 03:06

Head branch was pushed to by a user without write access

@yogeshwaran-c
yogeshwaran-c dismissed stale reviews from roblourens and mjbvz via 8547ee3 May 4, 2026 03:06
@yogeshwaran-c

Copy link
Copy Markdown
Contributor Author

@RedCMD thanks for catching this — pushed a fix.

Two test failures, two causes:

  1. colorize: test.md — the new math_inline_block begin regex (?<=\s|^)(\${1,2}) had no constraint on what follows the $, so : $ 1.25 (a price in a definition list) was opening multiline math mode and pairing with the next $ 1.55 line. Tightened to (?<=\s|^)(\${1,2})(?=\S) so math must immediately contain non-whitespace, matching CommonMark/KaTeX semantics. The existing $ \theta $ single-line cases are unaffected because math_inline_single/math_inline_double are tried first.

  2. colorize: md-math.md — the multiline test cases I added at the end of the fixture (Wrong LaTeX highlighting in markdown #223545 / $\mu...$, $x...$, $\gamma\n\delta$) never had matching snapshot entries — the snapshot file (md-math_md.json) was last touched by Update color settings in 2026 Dark theme for improved visibility #302147 and I forgot to regenerate it. Reverted the fixture additions to restore the snapshot/fixture pairing. The actual issue Wrong LaTeX highlighting in markdown #223545 examples ($\mu = ...\n...$, $x = ...\n...$) still work end-to-end via the (now-tightened) grammar — they begin with $\ and $x respectively, satisfying the new (?=\S) lookahead.

Happy to add the multiline test cases back in a follow-up once I can regenerate the snapshot locally.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to fix markdown math tokenization in the built-in markdown-math extension so multiline expressions delimited by single dollar signs can be highlighted in paragraph contexts, addressing issue #223545.

Changes:

  • Expands the inline math block TextMate rule to accept either $ or $$ as multiline delimiters.
  • Switches the end rule to use the same delimiter via a backreference and corrects the capture group indices.
  • Adds a new markdown colorization fixture case intended to cover the reported multiline single-dollar scenario.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
extensions/markdown-math/syntaxes/md-math-inline.tmLanguage.json Updates the inline TextMate grammar rule that recognizes multiline math spans inside markdown paragraphs.
extensions/vscode-colorize-tests/test/colorize-fixtures/md-math.md Adds a colorization fixture entry meant to exercise the new multiline single-dollar highlighting behavior.

"name": "markup.math.inline.markdown",
"contentName": "meta.embedded.math.markdown",
"begin": "(?<=\\s|^)(\\${2})",
"begin": "(?<=\\s|^)(\\${1,2})(?=\\S)",
Comment on lines +174 to +175

**md**
"name": "markup.math.inline.markdown",
"contentName": "meta.embedded.math.markdown",
"begin": "(?<=\\s|^)(\\${2})",
"begin": "(?<=\\s|^)(\\${1,2})(?=\\S)",
@RedCMD

RedCMD commented May 4, 2026

Copy link
Copy Markdown
Contributor

found the markdown math package markdown-it-katex
and the source file index.js

$$ … $$ is defined as a math block

$ … $ is defined as inline math

Anything between two $ characters will be treated as TeX math. The opening $ must
have a non-space character immediately to its right, while the closing $ must
have a non-space character immediately to its left, and must not be followed
immediately by a digit. Thus, $20,000 and $30,000 won’t parse as math. If for some
reason you need to enclose text in literal $ characters, backslash-escape them and
they won’t be treated as math delimiters.

@RedCMD

RedCMD commented May 4, 2026

Copy link
Copy Markdown
Contributor

actually VSCode has their own fork https://github.com/microsoft/vscode-markdown-it-katex

and it seems to be broken?
along with the original?

this parses as math $ \theta $ even tho the readme says it shouldn't

The opening $ must have a non-space character immediately to its right

GitHub seems to correctly not parse it
$ \theta $

@mjbvz do we want to realign the Katex package first before attempting to align this grammar?
either update readme to match code, or update code to match readme and Github?

but then GitHub doesn't follow this rule

the closing $ must have a non-space character immediately to its left

$\theta $
$\theta $

https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/writing-mathematical-expressions

@yogeshwaran-c

Copy link
Copy Markdown
Contributor Author

@RedCMD thanks for the markdown-it-katex spec reference, that's exactly the right north star.

Replaced the over-aggressive (?=\S) lookahead with two digit guards: (?!\d)(?!\s+\d). This rejects both $<digit> (currency: $12.45, $10) and $ <digit> (price-with-space: : $ 1.25, : $ 1.55), which were the two failing patterns in test.md. It deliberately allows $$ followed by newline (the a **b** $$\n \frac{1}{2}\n $$\n **b** fixtures at lines 137-153 of md-math.md), which the bare (?=\S) was breaking — that was Copilot's catch.

Re your spec point on the closing side ("non-space immediately to its left, must not be followed by a digit"), agreed that's the next layer to align — happy to push another commit tightening end to match if mjbvz wants this PR to be the place to do it. Otherwise this PR stays scoped to the original multiline fix from #223545 and the closing-side cleanup can land separately.

The $ \theta $ single-line discrepancy you noted (parses here, doesn't on GitHub) lives in math_inline_single, which I didn't touch and is unrelated to the multiline scope of this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wrong LaTeX highlighting in markdown

5 participants