Complex error in PDF build--- parse-latex filter, pandoc, Quarto

[friendly]

Description

For nearly two weeks I've been struggling to re-build the HTML & PDF versions of my book, Visualizing Multivariare Data and Models in R. I get the same, persistent error every time. Each build takes ~18 min, so this is extremely frustrating.

With the help of Claude, I think I tracked this down to two problems, one related to the tarleb/parse-latex extension and the other to the Quarto freeze/cache mechanism, described below. I'll also post this as an issue for tarleb/parse-latex, but the behavior I saw leads me to ask for help here.

I've documented the attempts at solving this problem at Two Weeks in Quarto / Pandoc / LaTeX Hell

Problem

The tarleb/parse-latex extension, when listed under format: html: filters: in _quarto.yml, still expands user-defined LaTeX macros during the PDF pandoc pass. This is seems to be because the filter calls pandoc.read(raw.text,'latex') on raw LaTeX blocks, which populates pandoc's internal macro table. Subsequent pandoc.read() calls on raw inline content then expand those macros — using whatever multi-line \providecommand or \newcommand definition was registered — and the expanded text contains % characters from the definition body, causing Xelatex to fail!

When a Markdown footnote reference (.[^fn]) follows the expanded inline in the source, the last % comments out the .\footnote{ opener, producing:

! Extra }, or forgotten \endgroup

A compounding factor: Quarto's freeze cache does not track changes to {{< include >}}'d files, so stale macro definitions can persist in cached chapter output even after the included file is corrected.

---

Setup

• Project: Quarto book, HTML + PDF (XeLaTeX / documentclass: krantz)
• Quarto version: 1.8.25 / 1.9.x (error reproduced on both)
• Extension: tarleb/parse-latex
• _quarto.yml:

My _quarto.yml includes:

format:
  html:
    filters:
      - quarto
      - line-highlight
      - parse-latex

The book uses {{< include latex/latex-commands.qmd >}} at the top of every chapter. latex-commands.qmd defines fallback versions of custom indexing macros for HTML rendering contexts:

\providecommand{\ixd}[1]{%
  \index{#1@\texttt{#1} data}%
  \index{datasets!#1@\texttt{#1}}%
}
\providecommand{\ixp}[1]{%
   \index{#1@\texttt{#1} package}%
   \index{packages!#1@\texttt{#1}}%
}
\providecommand{\ixfunc}[2]{%
  \index{#1@\texttt{#2}}%
}

These same macros are also defined (authoritatively) in latex/preamble.tex via
include-in-header, which is loaded before the document body for PDF output.
\providecommand is used in latex-commands.qmd so that the preamble definitions
take precedence for PDF.

Inline R functions (dataset(), pkg(), func() in R/common.R) emit these macros
as raw LaTeX inlines:

# e.g., dataset("Prestige") returns (for PDF output):
"\texttt{Prestige}\ixd{Prestige}"

The .qmd source then has a Markdown footnote immediately after:

recorded in `r dataset("Prestige")`.[^prestige-src]

[^prestige-src]: The dataset was collected by...[@Blishen-etal-1987].

---

The Error

Every PDF build failed at the same location:

! Extra }, or forgotten \endgroup.
l.4193 ...citeproc{ref-Blishen-etal-1987}{1987}).}

The generated index.tex (Quarto's internal tex file, compiled by xelatex) contained:

\texttt{Prestige}%
  \index{Prestige@\texttt{Prestige} data}%
  \index{datasets!Prestige@\texttt{Prestige}}%.\footnote{The dataset was
  collected by...(\citeproc{ref-Blishen-etal-1987}{1987}).}

The % at the end of the last \index{} line comments out .\footnote{. LaTeX
never opens the footnote, encounters orphaned text, and when it hits the closing }
of the footnote, reports Extra }.

---

Investigation: Two Weeks of Failed Fixes

Fix 1 — Remove \n from inline R output

The R functions originally emitted:

ref <- paste0(ref, "\n\\index{...}\n\\index{...}\n")

The explicit \n caused pandoc to add % at each line break. We switched to a
custom LaTeX macro call:

ref <- paste0(ref, "\\ixd{", dname, "}")   # single-line, no \n

Result: Error persisted.

Fix 2 — Delete all freeze caches

Suspected the .quarto/_freeze/*/execute-results/tex.json files were stale and serving old multi-line output. Deleted all 15+ tex.json files.

Result: Error persisted. Caches were regenerated with new \ixd{} format —
but the expansion was still happening downstream.

Fix 3 — Move parse-latex to format: html: filters: only

parse-latex was originally in the global filters: list. Moved it to format: html: filters: to prevent it running during the PDF pandoc pass.

Result: Error persisted.

Fix 4 — Change parse-latex.lua FORMAT guard

The filter's built-in guard was:

if FORMAT:match 'latex' then
  return {}
end

Hypothesis: Quarto's PDF pandoc pass uses a FORMAT value that doesn't contain "latex" (e.g. "pdf" or a Quarto-internal format). Changed to:

if not FORMAT:match 'html' then
  return {}
end

Result: Error persisted. Most \ixd{} calls in index.tex were still expanded
(785 \index{} occurrences vs. only 4 unexpanded \ixd{}).

Fix 5 — Make preamble.tex macro definitions single-line

Hypothesis: parse-latex expands \ixd{} using its definition from preamble.tex, and the multi-line definition body (with % at line ends) causes the problem. Changed to single-line:

\newcommand{\ixd}[1]{\index{#1@\texttt{#1} data}\index{datasets!#1@\texttt{#1}}}

Result: Error persisted. The content of index.tex was identical — unchanged by the preamble fix. This ruled out preamble.tex as the source of the definition being used.

---

Root Cause

Inspecting the freeze cache (tex.json) revealed that it contained not just the knitr R output, but the full resolved chapter content including the {{< include latex/latex-commands.qmd >}} content. Crucially, tex.json stored the old multi-line \providecommand{\ixd} definition:

"\\providecommand{\\ixd}[1]{%\n  \\index{#1@\\texttt{#1} data}%\n  \\index{datasets!#1@\\texttt{#1}}%\n}"

The mechanism:

1. Quarto reads tex.json from the freeze cache (which embeds the latex-commands.qmd
   content with multi-line \providecommand{\ixd})
2. During the pandoc pass, parse-latex's RawBlock handler calls
   pandoc.read(raw.text, 'latex') on the raw LaTeX block containing \providecommand{\ixd}
3. This registers \ixd in pandoc's internal macro table with the multi-line body
4. Later, parse-latex's RawInline handler processes \ixd{Prestige} and calls
   pandoc.read("\ixd{Prestige}", 'latex')
5. Pandoc now knows \ixd's definition and expands it — producing the multi-line
   body with % literally included
6. When written back to the LaTeX output, the % from the definition body comments
   out the .\footnote{ that follows on the same line in the source

This happens even though parse-latex is in format: html: filters: only. The FORMAT
guard changes had no effect because — regardless of format — pandoc.read() calls
within a Lua filter share pandoc's global macro table.

Changing preamble.tex had no effect because parse-latex never reads preamble.tex
via pandoc.read() — it only processes raw LaTeX blocks that appear in the document
body, which come from the freeze cache, not from include-in-header.

Compounding factor: freeze cache doesn't track {{< include >}} dependencies

After fixing latex-commands.qmd to use single-line definitions, the tex.json freeze caches still contained the old multi-line definitions — because Quarto's freeze invalidation only tracks changes to the main .qmd file, not to {{< include >}}'d files. The fix required:

1. Updating latex-commands.qmd to single-line definitions
2. Manually deleting all tex.json freeze caches to force regeneration

---

The Fix

latex/latex-commands.qmd — single-line \providecommand bodies

\providecommand{\ixd}[1]{\index{#1@\texttt{#1} data}\index{datasets!#1@\texttt{#1}}}
\providecommand{\ixp}[1]{\index{#1@\texttt{#1} package}\index{packages!#1@\texttt{#1}}}
\providecommand{\ixfunc}[2]{\index{#1@\texttt{#2}}}

When parse-latex expands these via pandoc.read(), the expansion body contains no
% characters. The .\footnote{ that follows in the source is no longer commented out.

Delete all tex.json freeze caches

find .quarto/_freeze -name "tex.json" -delete

Required because Quarto does not invalidate freeze caches when included files change.

---

Questions / Requests

1. Is parse-latex's pandoc.read() behavior — registering macro definitions in
   pandoc's global macro table and then expanding them in subsequent calls — expected
   or a side effect? The filter is designed for HTML output only, but its
   pandoc.read() calls have PDF-build side effects because the macro table is global
   within a pandoc process.

2. Should format: html: filters: actually prevent a filter from running during the
   PDF pandoc pass? Our experience suggests it does not — the filter appears to run
   regardless of format placement. The FORMAT guard (if not FORMAT:match 'html' then return {} end) also did not prevent the macro expansion side effects.

3. Should the freeze cache track {{< include >}} file dependencies? Currently,
   changing an included file requires manual cache deletion. This is a significant
   problem when included files define LaTeX macros that are then embedded in cached
   chapter output.

---

Environment

Quarto: 1.8.25 (also reproduced on 1.9.x)
Platform: Windows 11
R: 4.5.2
pandoc: (Quarto-bundled)
Extension: tarleb/parse-latex (from _extensions/)

[friendly]

Also filed as an issue for parse-latex: tarleb/parse-latex#7

[mcanouil]

Is parse-latex's pandoc.read() behavior — registering macro definitions in
pandoc's global macro table and then expanding them in subsequent calls — expected
or a side effect? The filter is designed for HTML output only, but its
pandoc.read() calls have PDF-build side effects because the macro table is global
within a pandoc process.

That's a question for the developer of the filter.

---

Should format: html: filters: actually prevent a filter from running during the
PDF pandoc pass? Our experience suggests it does not — the filter appears to run
regardless of format placement. The FORMAT guard (if not FORMAT:match 'html' then return {} end) also did not prevent the macro expansion side effects.

It does, take a simple example or ask Claude to use simple example when testing/investigating.

• Quarto document

---
title: Title of the document
format:
  pdf:
    filters:
      - format-latex.lua
  html: default
---

## Overview

Body text goes here.

• format-latex.lua

--- format-latex.lua
--- A simple Pandoc Lua filter that logs messages when the output format is LaTeX.

function Pandoc(doc)
  if FORMAT:match("latex") then
    io.stderr:write("[format-latex] Output format is LaTeX.\n")
    io.stderr:write("[format-latex] Number of blocks: " .. #doc.blocks .. "\n")
    io.stderr:write("[format-latex] LaTeX filter processing complete.\n")
  end
  return doc
end

HTML

PDF (LaTeX)

❯ quarto render demo.qmd --to html pandoc to: html output-file: demo.html standalone: true section-divs: true html-math-method: mathjax wrap: none default-image-extension: png variables: {} metadata document-css: false link-citations: true date-format: long lang: en title: v*md Output created: demo.html

❯ quarto render demo.qmd --to pdf pandoc to: latex output-file: demo.tex standalone: true pdf-engine: lualatex variables: graphics: true tables: true default-image-extension: pdf metadata documentclass: scrartcl classoption: - DIV=11 - numbers=noendperiod papersize: letter header-includes: - \KOMAoption{captions}{tableheading} block-headings: true title: v*md [format-latex] Output format is LaTeX. [format-latex] Number of blocks: 3 [format-latex] LaTeX filter processing complete. Rendering PDF running lualatex - 1 This is LuaHBTeX, Version 1.22.0 (TeX Live 2025) restricted system commands enabled. running lualatex - 2 This is LuaHBTeX, Version 1.22.0 (TeX Live 2025) restricted system commands enabled. Output created: demo.pdf

---

Should the freeze cache track {{< include >}} file dependencies? Currently,
changing an included file requires manual cache deletion. This is a significant
problem when included files define LaTeX macros that are then embedded in cached
chapter output.

Unclear what you are asking. Freeze and Cache are two different features. The former is handled by Quarto to save the text result/output after computation, the latter is handled by the engine (knitr?) and saves the code cells objects (data.frame, ggplot2 object, etc.).

---

Since you are using Claude, ask it to create a small and simple reproducible example.
This will help everyone, including Claude figuring out the real issue.

[friendly]

Thank you @mcanouil — your suggestion to create a minimal reproducible example was
exactly right and led to a much clearer diagnosis.

The Test

I created a minimal repro document with three tests, available in the repo:

• issues/quarto-pandoc-test.qmd
• issues/quarto-pandoc-test-macros.qmd
• Generated: issues/quarto-pandoc-test.tex

The document is configured with parse-latex in format: html: filters: only (matching
the book setup), and keep-tex: true so the generated .tex can be inspected.

Each test defines a multi-line \providecommand with % at line-ends (the standard
LaTeX style for suppressing whitespace in the expanded output), uses the macro in a raw
inline, and places a Markdown footnote immediately after — the pattern that caused
! Extra }, or forgotten \endgroup in the book.

Test 1 — \providecommand defined directly in a {=latex} raw block in the document:

\providecommand{\myone}[1]{%
  \textit{T1: #1}%
}

followed by `\myone{alpha}`{=tex}.[^fn]

Test 2 — Same definition, but via {{< include quarto-pandoc-test-macros.qmd >}}
(simulating how the book includes latex/latex-commands.qmd in every chapter).

Test 3 — The macro emitted by inline R via cat("\\mythree{gamma}") followed by
.[^fn] on the next line.

Results (from the generated .tex)

Tests 1 and 2 — macros pass through unexpanded:

% Test 1:
\providecommand{\myone}[1]{%
  \textit{T1: #1}%
}
...
Static inline then footnote: \myone{alpha}.\footnote{Test 1 footnote...}

% Test 2 (from {{< include >}}):
\providecommand{\mytwo}[1]{%
  \textit{T2: #1}%
}
...
\mytwo{beta}.\footnote{Test 2 footnote...}

Both macros appear unexpanded in the .tex file, and \myone{alpha}.\footnote{...}
is on a single line — exactly what we want. XeLaTeX then expands \myone{alpha} natively;
the % in the definition body acts as a TeX comment, so the expansion is just
\textit{T1: alpha} with no literal %. Both tests compiled successfully.

This confirms your point: format: html: filters: correctly prevents parse-latex
from running during PDF builds. It is not the direct cause of the expansion.

Test 3 — different issue:

\mythree{gamma}

.\footnote{Test 3 footnote...}

A knitr code chunk adds \n\n after its output, splitting the macro call and the
.[^fn] footnote reference into separate paragraphs. This is a formatting problem
(dangling . in its own paragraph) but not the Extra } error.

What Was Actually Happening in the Book

The real cause was a combination of two things:

1. Stale freeze cache with old macro definitions.

The book's latex/latex-commands.qmd (included in every chapter via {{< include >}})
originally defined the index macros with multi-line bodies:

\providecommand{\ixd}[1]{%
  \index{#1@\texttt{#1} data}%
  \index{datasets!#1@\texttt{#1}}%
}

When parse-latex was still in the global filters: list (before we moved it to
format: html: filters:), it ran for both HTML and PDF pandoc passes. During the
PDF pass, parse-latex's RawBlock handler called pandoc.read(raw.text, 'latex') on
the block containing \providecommand{\ixd}. This registered \ixd in pandoc's macro
table. When \ixd{Prestige} was subsequently encountered as a raw inline and
pandoc.read("\ixd{Prestige}", 'latex') was called, pandoc expanded it using the
registered multi-line definition — inserting the literal % characters from the body
into the output. The % then commented out the .\footnote{ that followed.

2. {{< include >}} changes do not invalidate the freeze cache.

After we moved parse-latex to format: html: filters: (Fix 3 in the original post),
the filter should have stopped running for PDF — and the simple repro confirms it does.
But the error persisted in the book. Why?

Quarto's freeze cache (tex.json) stores the fully resolved chapter content,
including content from {{< include >}}'d files. When we subsequently updated
latex/latex-commands.qmd to use single-line definitions, the existing tex.json
files were not invalidated — because Quarto only tracks changes to the main .qmd
file, not to files it includes. The stale caches still embedded the old multi-line
\providecommand{\ixd} definitions.

The fix that finally worked required both:

1. Updating latex-commands.qmd to single-line definitions
2. Manually deleting all tex.json freeze caches:

   find .quarto/_freeze -name "tex.json" -delete

Remaining Question

Given the above, would it be feasible for Quarto to track {{< include >}} file
dependencies when deciding whether to invalidate freeze cache entries? Currently,
any change to an included file is invisible to the freeze mechanism, requiring manual
cache deletion to take effect. This is a significant footgun, especially when included
files define content (like LaTeX macro definitions) that gets embedded in cached output.

I'll also note that the parse-latex filter — when it was running for PDF (in global
filters) — had an unexpected side effect: pandoc.read() calls within the filter share
pandoc's global macro table across invocations within the same pandoc process. A
\newcommand/\providecommand encountered in one RawBlock call is available for
expansion in subsequent RawInline calls. Whether this is a pandoc behavior worth
documenting (or guarding against in the filter) is a question for @tarleb.

[cscheid]

would it be feasible for Quarto to track {{< include >}} file dependencies when deciding whether to invalidate freeze cache entries?

We could improve the include situation specifically, but that will still not be a precise tracking in some situations. I confess I still don't exactly understand the freeze + include problem, and would still benefit from a minimal reproduction of the problem that doesn't involve any other Quarto subsystems.

[friendly]

Thanks for the reply. I'm not sure what you mean by "doesn't involve any other Quarto subsystems".
I described the problem as well as I could understand, but it seems to involve both Quarto and pandoc.

[mcanouil]

He meant, without the Lua filter you are using.

The Lua filter is old even for Pandoc and can cause various side effect on Quarto's own Lua processing.