>>`. The pipeline is
+ `?Send`, so `Rc` is correct on wasm32 and on native pollster.
+ Provider is shared across all pages a renderer touches —
+ future multi-page render modes work without a second migration.
+ Costs a one-line rewrite at every existing
+ `Some(Box::new(provider))` call site (~half-dozen sites + tests).
+
+2. **`JsUserGrammars` `Clone` derive — skip.** Not needed under
+ model (B): the `Rc` shares the underlying value without cloning
+ `JsUserGrammars` itself. If a later use case needs it, derive
+ then.
+
+3. **Test infrastructure — both.**
+ - **Rust unit test** under `crates/quarto-core/tests/` driving
+ `RenderToHtmlRenderer` directly with a stub
+ `UserGrammarProvider` returning canned `data-hl-spans`. Fast,
+ deterministic, catches the wiring bug independent of any real
+ tree-sitter grammar. Lives in CI's default test suite.
+ - **E2E**: the existing
+ `highlighting/03-user-grammar-toml.qmd` smoke fixture (already
+ failing today via `cargo xtask verify --e2e`) doubles as the
+ integration check. Caveat: `--e2e` is not in default CI, so
+ the unit test is the load-bearing one for regression
+ prevention.
+
+4. **Priority — left as P3 by default.** Open question: does the
+ user want it bumped now that the hub-client routes all
+ `_quarto.yml`-rooted documents through the project path? Not
+ blocking design; can be revisited at implementation time.
+
+## Phase 4 end-to-end verification artifact
+
+Re-ran the e2e suite for the `highlighting/03-user-grammar-toml.qmd`
+fixture from the worktree on `beads/bd-izfv-thread-user-grammars`
+(rebased onto `chore/e2e-ci` to pick up the smoke-all e2e
+infrastructure). After landing both this branch's Rust changes
+*and* the test-harness improvements documented in Phase 5:
+
+Invocation:
+
+```sh
+cd hub-client
+npx playwright test --grep "user-grammar" smoke-all.spec.ts
+```
+
+Result: `1 passed`. Inspecting the iframe innerHTML inside the
+live preview at assertion time confirms the user grammar is
+applied:
+
+```html
+name = "example"
+count = 42
+```
+
+All four `ensureFileRegexMatches` patterns (`"example"`,
+`hl-number">42`, `hl-operator">=`) match.
+
+Two test-harness gaps surfaced during this phase, fixed in the
+same branch:
+
+- `smoke-all.spec.ts` previously read every fixture file via
+ `readFileSync(path, 'utf-8')` and forwarded it with
+ `contentType: 'text' as const`, which corrupts binary grammar
+ `.wasm` payloads before they reach Automerge → VFS. Discovery
+ (`smokeAllDiscovery.ts`) now detects binary extensions
+ (`.wasm`, common image and font types), reads them as bytes,
+ encodes base64, and tags `contentType: 'binary'` with a
+ matching `mimeType`. `smoke-all.spec.ts` forwards both fields
+ to `createProjectOnServer`. Output was inspected via the
+ iframe innerHTML before the fix (no `sourceCode toml` class,
+ no `hl-*` spans) and after the fix (the snippet above).
+- `previewExtraction.ts::getPreviewHtml` did a re-render via
+ `renderToHtml({ documentPath })` without forwarding a
+ `userGrammars` context, so user-grammar fixtures got
+ unhighlighted output from the helper even when the live
+ preview rendered correctly. The helper now derives a
+ `userGrammars` context from VFS state — stripping the
+ `/project/` prefix `vfs_list_files` returns to match
+ `discoverUserGrammars`'s project-relative shape, and wiring
+ binary/text resolvers back through `vfsReadBinaryFile` /
+ `vfsReadFile`.
+
+## Risks / tradeoffs
+
+- **Public API ripple.** Changing the type of
+ `RenderContext::user_grammar_provider` from `Option>` to
+ `Option>>` is a breaking change for any out-of-tree
+ consumer of `RenderContext`. There are none today, but the field
+ is `pub`. The compiler will catch every internal call site at
+ rename time.
+- **No CI coverage for `--e2e`.** Even after the fix, the e2e
+ fixture itself won't be caught by default CI — the Phase-0 Rust
+ unit test is what guards against regression. Worth a separate
+ follow-up beads to either gate a smoke subset of `--e2e` on PR
+ builds, or to expand the Rust integration tests further. Out of
+ scope for bd-izfv proper.
+- **`Rc>` borrow discipline.** The walker that calls
+ `provider.contains(...)` / `provider.highlight(...)` must not
+ hold a borrow across reentrant calls. The trait's `&mut self` on
+ `highlight` already pushes implementations toward short-lived
+ borrows, but watch for any code that grabs the provider once and
+ iterates — convert to per-call borrow there.
diff --git a/claude-notes/plans/thread-user-grammars-renderer-investigation/repro-and-codepath.md b/claude-notes/plans/thread-user-grammars-renderer-investigation/repro-and-codepath.md
new file mode 100644
index 000000000..64cbf40ee
--- /dev/null
+++ b/claude-notes/plans/thread-user-grammars-renderer-investigation/repro-and-codepath.md
@@ -0,0 +1,112 @@
+# bd-izfv investigation — repro + code path notes
+
+## Symptom (reproducible at HEAD)
+
+Smoke-all e2e fixture
+`crates/quarto/tests/smoke-all/highlighting/03-user-grammar/03-user-grammar-toml.qmd`
+fails its `ensureFileRegexMatches` assertions when rendered through the
+hub-client preview (Playwright `cargo xtask verify --e2e`).
+
+Expected (frontmatter):
+- `"example"`
+- `hl-number">42`
+- `hl-operator">=`
+
+Observed (from `hub-client/playwright-report/data/063e6...md`, the
+error-context page snapshot for the failing run):
+
+```
+- code [ref=f1e25]: name = "example" count = 42
+```
+
+A bare `` block — no `` wrapper, no
+`hl-*` spans. The user-supplied tree-sitter grammar at
+`_quarto/grammars/toml/` was never invoked.
+
+## Why it's the project-render branch specifically
+
+The fixture has `_quarto.yml` next to it, so on the hub-client side
+`renderPageInProject(path, grammarsHandle)` (hub-client/src/services/wasmRenderer.ts:944)
+takes the project branch — Pass-1 builds the index, Pass-2 runs
+`RenderToHtmlRenderer::render`.
+
+Single-file fall-through (no `_quarto.yml`) goes through
+`render_single_doc_to_response` (crates/wasm-quarto-hub-client/src/lib.rs:1341)
+which threads the provider correctly:
+
+```rust
+let mut ctx = RenderContext::new(project, &doc, &format, &binaries).with_options(options);
+if let Some(provider) = user_grammars {
+ ctx.user_grammar_provider = Some(Box::new(provider));
+}
+```
+
+The project branch is `render_project_active_page_to_response`
+(crates/wasm-quarto-hub-client/src/lib.rs:1416) and contains the
+explicit drop:
+
+```rust
+// Note: `user_grammars` is currently dropped on the orchestrator
+// path because the renderer constructs its own RenderContext
+// per page. Threading user grammars through the renderer is a
+// sub-phase 9.4 follow-up — file as bd-XXXX on close-out.
+let _ = user_grammars;
+```
+
+(That "file as bd-XXXX" was filed as bd-izfv.)
+
+## Where the wiring needs to happen
+
+`RenderToHtmlRenderer::render` constructs the per-page `RenderContext`
+itself (crates/quarto-core/src/project/pass2_renderer.rs:336):
+
+```rust
+let mut ctx = RenderContext::new(project, doc_info, format, &binaries).with_options(options);
+ctx.project_index = Some(index);
+ctx.resource_resolver = Some(resolver.clone());
+```
+
+The provider would have to be installed here — analogous to the
+single-doc branch's `ctx.user_grammar_provider = Some(Box::new(provider))`.
+
+## Lifetime constraints
+
+`RenderContext::user_grammar_provider` is typed
+`Option>`
+(crates/quarto-core/src/render.rs:164). The trait's `highlight`
+takes `&mut self` (crates/quarto-highlight/src/provider.rs:49), and
+the WASM impl `JsUserGrammars` holds a `HashMap`
+(crates/wasm-quarto-hub-client/src/lib.rs:188).
+
+`js_sys::Function` is `Clone` (it's a JS handle), so deriving `Clone`
+on `JsUserGrammars` is mechanical. **However**, the trait object
+(`Box`) is not `Clone` and the field can't
+ergonomically support multi-page renders without one of:
+
+- (A) **Single-render consume.** Hub-client uses `RenderMode::ActivePage`
+ which renders exactly one page per call. Store
+ `Option>` on `RenderToHtmlRenderer` and
+ `take()` it on the first/only `render()` call. Simple, but if a
+ future caller drives `RenderToHtmlRenderer` over multi-page modes,
+ pages 2..N silently lose user grammars.
+- (B) **Shared interior-mutable provider.** Change the field on
+ `RenderContext` to `Option>>`,
+ and adjust the existing call sites. Works for any RenderMode and
+ matches the WASM single-thread model. Costs a small public-API
+ ripple on `RenderContext` (and on every test that constructs one).
+- (C) **`Arc>`.** Same as B but `Send`/`Sync`. The pipeline
+ is `?Send` already; this would over-restrict.
+
+The bd-izfv description hints at (B): *"Arc>>
+(or similar)"* — though `Mutex` is wrong for the WASM single-thread
+case, `Arc>` would be a closer fit if we want sharability
+on wasm32. `Rc>` is also a candidate.
+
+## What the smoke-all CLI does (already passes)
+
+The native render path of the same fixture (`cargo xtask verify`,
+no `--e2e`) passes — see `crates/quarto-highlight/tests/user_grammar_toml.rs`
+and the smoke-all native runner. So the bug is strictly:
+"Phase-9 hub-client project-render path doesn't thread user grammars
+into Pass-2's per-page RenderContext."
diff --git a/crates/quarto-core/src/pipeline.rs b/crates/quarto-core/src/pipeline.rs
index d4ccee9a3..84e1f860b 100644
--- a/crates/quarto-core/src/pipeline.rs
+++ b/crates/quarto-core/src/pipeline.rs
@@ -527,7 +527,11 @@ pub async fn run_pipeline(
stage_ctx.artifacts = std::mem::take(&mut ctx.artifacts);
// Transfer user-grammar provider (browser path sets this; native CLI
// leaves it None and falls back to `CodeHighlightStage`'s disk scan).
- stage_ctx.user_grammar_provider = ctx.user_grammar_provider.take();
+ // Cloning the `Rc` is cheap and keeps the provider shared across
+ // every page the renderer touches (bd-izfv: the project-render path
+ // calls `run_pipeline` once per page through a single
+ // `RenderToHtmlRenderer`).
+ stage_ctx.user_grammar_provider = ctx.user_grammar_provider.clone();
// Transfer the project index (set by ProjectPipeline::pass_two).
// Cloning the `Arc` is cheap and keeps the RenderContext usable
// after the stage context is built.
diff --git a/crates/quarto-core/src/project/pass2_renderer.rs b/crates/quarto-core/src/project/pass2_renderer.rs
index 050e96352..24ab37cd2 100644
--- a/crates/quarto-core/src/project/pass2_renderer.rs
+++ b/crates/quarto-core/src/project/pass2_renderer.rs
@@ -33,7 +33,9 @@
//!
//! [`ProjectPipeline`]: crate::project::orchestrator::ProjectPipeline
+use std::cell::RefCell;
use std::path::Path;
+use std::rc::Rc;
use std::sync::Arc;
use async_trait::async_trait;
@@ -281,6 +283,14 @@ pub struct RenderToHtmlRenderer {
/// project scope alike) lives in WASM. `RenderResolverContext::vfs_root`
/// will be constructed with this root.
vfs_root: std::path::PathBuf,
+
+ /// Optional user-grammar provider attached by the caller. Shared
+ /// across every page the renderer touches (one
+ /// `RenderToHtmlRenderer` may produce many pages in `ActivePage`
+ /// mode plus future multi-page modes). The pipeline is `?Send`
+ /// so `Rc>` is correct on both wasm32 and on the
+ /// native single-task executor used by tests. (bd-izfv)
+ user_grammars: Option>>,
}
impl RenderToHtmlRenderer {
@@ -289,8 +299,21 @@ impl RenderToHtmlRenderer {
pub fn new(vfs_root: impl Into) -> Self {
Self {
vfs_root: vfs_root.into(),
+ user_grammars: None,
}
}
+
+ /// Attach a user-grammar provider. The renderer installs it on
+ /// every per-page [`crate::render::RenderContext`] before
+ /// running the pipeline, so `CodeHighlightStage` consults it
+ /// in preference to the native disk loader. (bd-izfv)
+ pub fn with_user_grammars(
+ mut self,
+ provider: Rc>,
+ ) -> Self {
+ self.user_grammars = Some(provider);
+ self
+ }
}
#[async_trait(?Send)]
@@ -337,6 +360,11 @@ impl Pass2Renderer for RenderToHtmlRenderer {
RenderContext::new(project, doc_info, format, &binaries).with_options(options);
ctx.project_index = Some(index);
ctx.resource_resolver = Some(resolver.clone());
+ // bd-izfv: forward the renderer-attached user-grammar provider
+ // (if any) to the per-page context. `run_pipeline` clones the
+ // `Rc` into the inner `StageContext`, so the same handle is
+ // shared across every page this renderer renders.
+ ctx.user_grammar_provider = self.user_grammars.clone();
let config = HtmlRenderConfig::with_resolver(resolver.clone());
let source_name = doc_info.input.to_string_lossy().to_string();
diff --git a/crates/quarto-core/src/render.rs b/crates/quarto-core/src/render.rs
index 42042861e..1bf79d273 100644
--- a/crates/quarto-core/src/render.rs
+++ b/crates/quarto-core/src/render.rs
@@ -12,7 +12,9 @@
//! - Transforms can access project configuration and format settings
//! - Writers use the context to determine output paths
+use std::cell::RefCell;
use std::path::PathBuf;
+use std::rc::Rc;
use std::sync::Arc;
use quarto_analysis::AnalysisContext;
@@ -161,7 +163,7 @@ pub struct RenderContext<'a> {
/// browser hub-client sets this to a `JsUserGrammars` (Phase 4.3 of
/// the syntax-highlighting plan) so JS-backed user grammars flow
/// through the same `CodeHighlightStage` code path.
- pub user_grammar_provider: Option>,
+ pub user_grammar_provider: Option>>,
/// Per-document resource report (`bd-o8pr`). Mirrors
/// [`crate::stage::StageContext::resource_report`]; `run_pipeline`
diff --git a/crates/quarto-core/src/stage/context.rs b/crates/quarto-core/src/stage/context.rs
index 7408f7024..495a15f0d 100644
--- a/crates/quarto-core/src/stage/context.rs
+++ b/crates/quarto-core/src/stage/context.rs
@@ -16,7 +16,9 @@
//! - Supports potential task spawning for parallelization
//! - Allows stages to clone what they need without lifetime constraints
+use std::cell::RefCell;
use std::path::PathBuf;
+use std::rc::Rc;
use std::sync::Arc;
use quarto_error_reporting::DiagnosticMessage;
@@ -148,7 +150,7 @@ pub struct StageContext {
/// - **Browser (hub-client)**: set to a `JsUserGrammars` handle that
/// forwards highlight requests to JS callbacks backed by
/// `web-tree-sitter`. See `crates/wasm-quarto-hub-client/src/lib.rs`.
- pub user_grammar_provider: Option>,
+ pub user_grammar_provider: Option>>,
}
impl StageContext {
diff --git a/crates/quarto-core/src/stage/stages/code_highlight.rs b/crates/quarto-core/src/stage/stages/code_highlight.rs
index 74db34cf6..942d0e046 100644
--- a/crates/quarto-core/src/stage/stages/code_highlight.rs
+++ b/crates/quarto-core/src/stage/stages/code_highlight.rs
@@ -78,17 +78,15 @@ impl PipelineStage for CodeHighlightStage {
// over the native disk-loader path. This lets both targets share
// one code path without cfg-gating at the call site.
//
- // Manual `&mut **b` rather than `as_deref_mut()` because the
- // latter preserves the trait-object's implicit `'static` bound,
- // which then fights the async-trait `'static` future-lifetime
- // check. `&mut **b` produces a fresh borrow with the natural
- // (method-body) lifetime.
- let result = if ctx.user_grammar_provider.is_some() {
- let user = ctx
- .user_grammar_provider
- .as_mut()
- .map(|b| &mut **b as &mut dyn quarto_highlight::UserGrammarProvider);
- quarto_highlight::annotate_pandoc(&mut doc.ast, user)
+ // The provider lives in an `Rc>` (bd-izfv) so a single
+ // handle can be shared across every page a renderer touches. We
+ // hold a `RefMut` only for the duration of the `annotate_pandoc`
+ // call; the trait's `&mut self` methods on `highlight` don't
+ // re-enter the provider, so the borrow is straight-line.
+ let result = if let Some(rc) = ctx.user_grammar_provider.as_ref() {
+ let mut guard = rc.borrow_mut();
+ let user_ref = Some(&mut *guard as &mut dyn quarto_highlight::UserGrammarProvider);
+ quarto_highlight::annotate_pandoc(&mut doc.ast, user_ref)
} else {
#[cfg(not(target_arch = "wasm32"))]
{
diff --git a/crates/quarto-core/tests/render_to_html_user_grammars.rs b/crates/quarto-core/tests/render_to_html_user_grammars.rs
new file mode 100644
index 000000000..b93fe6135
--- /dev/null
+++ b/crates/quarto-core/tests/render_to_html_user_grammars.rs
@@ -0,0 +1,159 @@
+/*
+ * tests/render_to_html_user_grammars.rs
+ * Copyright (c) 2026 Posit, PBC
+ *
+ * bd-izfv: regression test that `RenderToHtmlRenderer` forwards a
+ * caller-supplied user-grammar provider to the per-page
+ * `RenderContext`. Without this wiring the orchestrator's project-
+ * render path silently drops grammars and any qmd that lives under
+ * a `_quarto.yml` ancestor renders code blocks unhighlighted.
+ *
+ * The test installs a stub provider that emits a known `data-hl-spans`
+ * triple-array for one custom class, drives `RenderToHtmlRenderer`
+ * via `ProjectPipeline<…>::with_renderer` exactly like the WASM
+ * entry point does, and asserts the rendered HTML carries the
+ * matching `` wrapper. The pampa HTML writer
+ * turns `data-hl-spans` into the nested span markup (see
+ * `crates/pampa/src/writers/html.rs`).
+ */
+
+use std::cell::RefCell;
+use std::path::{Path, PathBuf};
+use std::rc::Rc;
+use std::sync::Arc;
+
+use tempfile::TempDir;
+
+use quarto_core::format::Format;
+use quarto_core::project::ProjectContext;
+use quarto_core::project::orchestrator::{ProjectPipeline, RenderMode, project_type_for};
+use quarto_core::project::pass2_renderer::{RenderToHtmlRenderer, WasmPassTwoOutput};
+use quarto_highlight::{HighlightError, UserGrammarProvider};
+use quarto_system_runtime::{NativeRuntime, SystemRuntime};
+
+/// Stub provider that owns one class and emits a fixed triple-array
+/// JSON covering the entire source. Used to verify the wiring without
+/// pulling in real tree-sitter grammars.
+struct StubProvider {
+ class: String,
+ capture: String,
+}
+
+impl UserGrammarProvider for StubProvider {
+ fn contains(&self, class: &str) -> bool {
+ class == self.class
+ }
+
+ fn highlight(&mut self, _class: &str, source: &str) -> Result, HighlightError> {
+ let end = source.len();
+ let json = format!(
+ r#"[[0,{end},"{capture}"]]"#,
+ end = end,
+ capture = self.capture
+ );
+ Ok(Some(json))
+ }
+}
+
+fn write(path: &Path, contents: &str) {
+ if let Some(parent) = path.parent() {
+ std::fs::create_dir_all(parent).unwrap();
+ }
+ std::fs::write(path, contents).unwrap();
+}
+
+fn canonical(path: &Path) -> PathBuf {
+ path.canonicalize().unwrap_or_else(|_| path.to_path_buf())
+}
+
+fn render_with_provider(
+ active: &Path,
+ provider: Rc>,
+) -> WasmPassTwoOutput {
+ let runtime: Arc = Arc::new(NativeRuntime::new());
+ let mut project = ProjectContext::discover(active, runtime.as_ref()).unwrap();
+ if !project.is_single_file {
+ project = ProjectContext::discover(&project.dir, runtime.as_ref()).unwrap();
+ }
+
+ let project_type = project_type_for(&project);
+ let vfs_root = project.dir.join(".quarto/project-artifacts");
+ let renderer = RenderToHtmlRenderer::new(&vfs_root).with_user_grammars(provider);
+
+ let mut pipeline = ProjectPipeline::with_renderer(
+ &mut project,
+ project_type,
+ Format::html(),
+ "html",
+ runtime.clone(),
+ renderer,
+ )
+ .with_mode(RenderMode::ActivePage(active.to_path_buf()));
+
+ let summary = pollster::block_on(pipeline.run()).expect("pipeline run");
+ assert!(
+ summary.pass1_failures.is_empty(),
+ "unexpected pass-1 failures: {:?}",
+ summary.pass1_failures,
+ );
+ assert!(
+ summary.pass2_failures.is_empty(),
+ "unexpected pass-2 failures: {:?}",
+ summary.pass2_failures,
+ );
+ summary
+ .outputs
+ .into_iter()
+ .next()
+ .expect("ActivePage mode should produce one output")
+}
+
+/// bd-izfv: when a project-rooted document is rendered via
+/// `RenderToHtmlRenderer`, a user-grammar provider attached with
+/// `with_user_grammars` must flow into the per-page `RenderContext`
+/// so `CodeHighlightStage` consults it. The stub provider claims the
+/// custom class `mygrammar` and emits one triple covering the whole
+/// source; the pampa HTML writer should turn that into a single
+/// `…` wrapper inside the rendered
+/// code block.
+#[test]
+fn project_render_threads_user_grammar_provider() {
+ let temp = TempDir::new().unwrap();
+ let project_dir = canonical(temp.path());
+
+ write(
+ &project_dir.join("_quarto.yml"),
+ "project:\n type: default\n",
+ );
+ write(
+ &project_dir.join("index.qmd"),
+ "---\ntitle: Stub grammar\n---\n\n```mygrammar\nhello world\n```\n",
+ );
+
+ let active = canonical(&project_dir.join("index.qmd"));
+ let provider: Rc> = Rc::new(RefCell::new(StubProvider {
+ class: "mygrammar".to_string(),
+ capture: "marker".to_string(),
+ }));
+
+ let output = render_with_provider(&active, provider);
+
+ assert!(
+ output.html.contains(" String {
+ if s.len() <= 400 {
+ s.to_string()
+ } else {
+ format!("{}…", &s[..400])
+ }
+}
diff --git a/crates/wasm-quarto-hub-client/src/lib.rs b/crates/wasm-quarto-hub-client/src/lib.rs
index 265b6e620..fa935f7b5 100644
--- a/crates/wasm-quarto-hub-client/src/lib.rs
+++ b/crates/wasm-quarto-hub-client/src/lib.rs
@@ -24,7 +24,9 @@ pub mod c_shim;
/// so they do not spam `console.error` with stack traces.
pub struct LuaThrow;
+use std::cell::RefCell;
use std::path::Path;
+use std::rc::Rc;
use std::sync::{Arc, OnceLock};
use quarto_core::{
@@ -1066,7 +1068,7 @@ pub async fn render_qmd(path: &str, user_grammars: Option) -> St
let mut ctx = RenderContext::new(&project, &doc, &format, &binaries).with_options(options);
if let Some(provider) = user_grammars {
- ctx.user_grammar_provider = Some(Box::new(provider));
+ ctx.user_grammar_provider = Some(Rc::new(RefCell::new(provider)));
}
// Use the unified async pipeline (same as CLI). Phase 5:
@@ -1187,7 +1189,7 @@ pub async fn render_qmd_content(
let mut ctx = RenderContext::new(&project, &doc, &format, &binaries).with_options(options);
if let Some(provider) = user_grammars {
- ctx.user_grammar_provider = Some(Box::new(provider));
+ ctx.user_grammar_provider = Some(Rc::new(RefCell::new(provider)));
}
// Use the unified async pipeline (same as CLI). Phase 5:
@@ -1363,7 +1365,7 @@ async fn render_single_doc_to_response(
let mut ctx = RenderContext::new(project, &doc, &format, &binaries).with_options(options);
if let Some(provider) = user_grammars {
- ctx.user_grammar_provider = Some(Box::new(provider));
+ ctx.user_grammar_provider = Some(Rc::new(RefCell::new(provider)));
}
// Phase 5 VFS-root resolver — every artifact resolves under
@@ -1436,14 +1438,11 @@ async fn render_project_active_page_to_response(
Err(e) => return error_response(e),
};
- // Note: `user_grammars` is currently dropped on the orchestrator
- // path because the renderer constructs its own RenderContext
- // per page. Threading user grammars through the renderer is a
- // sub-phase 9.4 follow-up — file as bd-XXXX on close-out.
- let _ = user_grammars;
-
let project_type = project_type_for(&project);
- let renderer = RenderToHtmlRenderer::new("/.quarto/project-artifacts");
+ let mut renderer = RenderToHtmlRenderer::new("/.quarto/project-artifacts");
+ if let Some(provider) = user_grammars {
+ renderer = renderer.with_user_grammars(Rc::new(RefCell::new(provider)));
+ }
// Canonicalize the active path so it matches the form
// `project.files` was filled with (Pass-2's `RenderMode::ActivePage`
diff --git a/hub-client/changelog.md b/hub-client/changelog.md
index b3cdc0981..cf7c82f33 100644
--- a/hub-client/changelog.md
+++ b/hub-client/changelog.md
@@ -13,6 +13,10 @@ be in reverse chronological order (latest first).
-->
+### 2026-05-10
+
+- [`979982bd`](https://github.com/quarto-dev/q2/commits/979982bd): Custom syntax highlighting now works in Quarto Hub projects (bd-izfv) — user-supplied tree-sitter grammars under `_quarto/grammars//` now apply to code blocks even when the qmd file lives under a `_quarto.yml` ancestor, matching the single-file render path.
+
### 2026-05-05
- [`5ecdfe48`](https://github.com/quarto-dev/q2/commits/5ecdfe48): Surface doctemplate diagnostics (e.g. `Q-10-2 Undefined variable`) through `quarto render` and the hub-client preview (bd-xdnk). Custom templates referencing undefined variables now produce ariadne-rendered warnings with accurate source locations instead of being silently dropped. Also fixes a separate pre-existing bug where the `template:` YAML key was ignored under `quarto render` because the lookup didn't handle `PandocInlines`-shaped scalars.
diff --git a/hub-client/e2e/helpers/previewExtraction.ts b/hub-client/e2e/helpers/previewExtraction.ts
index 5d0f5c7fd..6548fc145 100644
--- a/hub-client/e2e/helpers/previewExtraction.ts
+++ b/hub-client/e2e/helpers/previewExtraction.ts
@@ -105,7 +105,53 @@ export async function getPreviewHtml(
await window.__quartoTestReady;
const hooks = window.__quartoTest;
if (!hooks) throw new Error('__quartoTest missing — rebuild with VITE_E2E=1');
- const result = await hooks.wasmRenderer.renderToHtml({ documentPath: docPath });
+ const renderer = hooks.wasmRenderer;
+
+ // Discover user grammars from the VFS so the re-render matches the
+ // in-Preview render (Preview.tsx forwards a project-file list +
+ // resolvers from automergeSync; bd-izfv made the project-render
+ // path actually honor that handle). Without this, fixtures that
+ // depend on `_quarto/grammars//` render unhighlighted here
+ // even though the live iframe renders them correctly.
+ //
+ // `vfs_list_files` returns the VFS-absolute form (`/project/...`);
+ // `discoverUserGrammars` expects project-relative paths with no
+ // leading slash, and the resolver callbacks below mirror that
+ // shape (matches `Preview.tsx` → `automergeSync` wiring).
+ const VFS_PROJECT_PREFIX = '/project/';
+ const stripPrefix = (vfsPath: string): string | null =>
+ vfsPath.startsWith(VFS_PROJECT_PREFIX)
+ ? vfsPath.slice(VFS_PROJECT_PREFIX.length)
+ : null;
+ const toVfsPath = (relPath: string): string =>
+ relPath.startsWith('/') ? relPath : `${VFS_PROJECT_PREFIX}${relPath}`;
+
+ const listing = renderer.vfsListFiles();
+ const projectFilePaths: string[] = listing.success
+ ? (listing.files ?? [])
+ .map(stripPrefix)
+ .filter((p): p is string => p !== null)
+ : [];
+
+ const result = await renderer.renderToHtml({
+ documentPath: docPath,
+ userGrammars: {
+ files: projectFilePaths,
+ getBinaryContent: async (path: string) => {
+ const r = renderer.vfsReadBinaryFile(toVfsPath(path));
+ if (!r.success || typeof r.content !== 'string') return null;
+ // Decode base64 → Uint8Array on the page (Buffer isn't available).
+ const binary = atob(r.content);
+ const bytes = new Uint8Array(binary.length);
+ for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i);
+ return bytes;
+ },
+ getTextContent: async (path: string) => {
+ const r = renderer.vfsReadFile(toVfsPath(path));
+ return r.success && typeof r.content === 'string' ? r.content : null;
+ },
+ },
+ });
return result.html ?? '';
}, documentPath);
}
diff --git a/hub-client/e2e/helpers/projectFactory.ts b/hub-client/e2e/helpers/projectFactory.ts
index 1f3c3d3d9..ac5089c96 100644
--- a/hub-client/e2e/helpers/projectFactory.ts
+++ b/hub-client/e2e/helpers/projectFactory.ts
@@ -23,8 +23,11 @@ import type {} from './testHooks';
export interface ProjectFile {
path: string;
+ /** UTF-8 text for `'text'`, base64-encoded bytes for `'binary'`. */
content: string;
contentType: 'text' | 'binary';
+ /** Required by `createNewProject` for binary files. */
+ mimeType?: string;
}
/**
diff --git a/hub-client/e2e/helpers/smokeAllDiscovery.ts b/hub-client/e2e/helpers/smokeAllDiscovery.ts
index 38a15b658..a597a2d3d 100644
--- a/hub-client/e2e/helpers/smokeAllDiscovery.ts
+++ b/hub-client/e2e/helpers/smokeAllDiscovery.ts
@@ -22,13 +22,10 @@ const SKIP_PRINTS_MESSAGE: Set = new Set([
// Tests that fail in the hub-client preview because of a known gap in
// the WASM render pipeline. The native CLI runner handles these. Skip
-// in the browser until the gap is closed.
-const SKIP_WASM_UNSUPPORTED: Map = new Map([
- [
- 'highlighting/03-user-grammar/03-user-grammar-toml.qmd',
- 'bd-izfv: RenderToHtmlRenderer drops user_grammars on the project-render path',
- ],
-]);
+// in the browser until the gap is closed. (bd-izfv's user-grammar entry
+// was removed here once the project-render path started threading
+// user_grammars through; leave the slot in place for future WASM gaps.)
+const SKIP_WASM_UNSUPPORTED: Map = new Map([]);
// ---------------------------------------------------------------------------
// Types
@@ -72,8 +69,20 @@ export interface DiscoveredTest {
relPath: string;
/** Absolute path to the project root (containing _quarto.yml) */
projectRoot: string;
- /** All project files as { relativePath, content } */
- projectFiles: { path: string; content: string }[];
+ /**
+ * All project files. Text files have UTF-8 `content`; binary fixtures
+ * (e.g. `_quarto/grammars//*.wasm`, images) have base64-encoded
+ * `content` with `contentType: 'binary'` and an appropriate
+ * `mimeType`. The Playwright runner forwards both flavors to
+ * `createProjectOnServer`, which routes them to Automerge text vs.
+ * binary documents.
+ */
+ projectFiles: {
+ path: string;
+ content: string;
+ contentType: 'text' | 'binary';
+ mimeType?: string;
+ }[];
/** Which file to render (relative to project root) */
renderPath: string;
/** Run config from frontmatter */
@@ -297,9 +306,52 @@ function findProjectRoot(qmdDir: string): string {
return qmdDir;
}
-/** Recursively read all files in a directory. */
-function readAllFiles(dir: string): { path: string; content: string }[] {
- const files: { path: string; content: string }[] = [];
+/**
+ * Extension → MIME mapping for binary fixture files. The set is
+ * narrow: only formats the smoke-all suite actually carries (user
+ * grammars and a couple of image fixtures). Add new entries here
+ * when a future fixture needs a new binary type.
+ */
+const BINARY_EXTENSIONS: Record = {
+ '.wasm': 'application/wasm',
+ '.png': 'image/png',
+ '.jpg': 'image/jpeg',
+ '.jpeg': 'image/jpeg',
+ '.gif': 'image/gif',
+ '.webp': 'image/webp',
+ '.woff': 'font/woff',
+ '.woff2': 'font/woff2',
+ '.ttf': 'font/ttf',
+ '.otf': 'font/otf',
+};
+
+function binaryMimeFor(path: string): string | null {
+ const dot = path.lastIndexOf('.');
+ if (dot < 0) return null;
+ return BINARY_EXTENSIONS[path.slice(dot).toLowerCase()] ?? null;
+}
+
+/**
+ * Recursively read all files in a directory. Text fixtures land as
+ * `contentType: 'text'` with UTF-8 content; binary fixtures (see
+ * {@link BINARY_EXTENSIONS}) land as `contentType: 'binary'` with the
+ * raw bytes base64-encoded — matching the shape
+ * `createProjectOnServer` expects, which forwards binary content into
+ * Automerge as a binary document instead of corrupting it through
+ * UTF-8 decoding.
+ */
+function readAllFiles(dir: string): {
+ path: string;
+ content: string;
+ contentType: 'text' | 'binary';
+ mimeType?: string;
+}[] {
+ const files: {
+ path: string;
+ content: string;
+ contentType: 'text' | 'binary';
+ mimeType?: string;
+ }[] = [];
function walk(d: string) {
const entries = readdirSync(d, { withFileTypes: true });
@@ -308,8 +360,22 @@ function readAllFiles(dir: string): { path: string; content: string }[] {
if (entry.isDirectory()) {
walk(full);
} else if (entry.isFile()) {
- const content = readFileSync(full, 'utf-8');
- files.push({ path: full, content });
+ const mimeType = binaryMimeFor(full);
+ if (mimeType) {
+ const bytes = readFileSync(full);
+ files.push({
+ path: full,
+ content: bytes.toString('base64'),
+ contentType: 'binary',
+ mimeType,
+ });
+ } else {
+ files.push({
+ path: full,
+ content: readFileSync(full, 'utf-8'),
+ contentType: 'text',
+ });
+ }
}
}
}
@@ -352,6 +418,8 @@ export function discoverSmokeAllTests(): DiscoveredTest[] {
const projectFiles = allFiles.map((f) => ({
path: relative(projectRoot, f.path),
content: f.content,
+ contentType: f.contentType,
+ mimeType: f.mimeType,
}));
tests.push({
diff --git a/hub-client/e2e/smoke-all.spec.ts b/hub-client/e2e/smoke-all.spec.ts
index 83d3ca851..c29034a3c 100644
--- a/hub-client/e2e/smoke-all.spec.ts
+++ b/hub-client/e2e/smoke-all.spec.ts
@@ -85,7 +85,8 @@ test.describe('smoke-all E2E tests', () => {
sortedFiles.map((f) => ({
path: f.path,
content: f.content,
- contentType: 'text' as const,
+ contentType: f.contentType,
+ mimeType: f.mimeType,
})),
);