Skip to content

Commit 5fc58cd

Browse files
cdervcderv
authored
Fix transient .quarto_ipynb files accumulating during preview
During Jupyter-based preview, each re-render created a new numbered .quarto_ipynb variant (test.quarto_ipynb_1, _2, etc.) that persisted on disk even with keep-ipynb: false. Regression from v1.8.27 to v1.9 caused by the collision-avoidance loop from #13804 interacting with cache invalidation that deleted the cache entry but not the file. - Add invalidateForFile() to FileInformationCacheMap that deletes the transient notebook file from disk before removing the cache entry - Pass ProjectContext from cmd.ts to preview() via pProject parameter, eliminating redundant context creation that orphaned a .quarto_ipynb at startup - Add unit tests for cache invalidation and manual preview test fixtures Fixes #14281
1 parent ad06e01 commit 5fc58cd

File tree

19 files changed

+744
-20
lines changed

19 files changed

+744
-20
lines changed

‎.claude/commands/quarto-preview-test/SKILL.md‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
---
2+
name: quarto-preview-test
3+
description: Use when testing preview functionality, verifying live reload, or validating preview fixes. Covers starting preview with port/logging, browser verification via /agent-browser, and checking logs/filesystem for artifacts.
4+
---
5+
6+
# Quarto Preview Test
7+
8+
Interactive testing of `quarto preview` with automated browser verification.
9+
10+
## Tools
11+
12+
| Tool | When to use |
13+
|------|-------------|
14+
| `/agent-browser` | **Preferred.** Token-efficient browser automation. Navigate, verify content, screenshot. |
15+
| Chrome DevTools MCP | Deep debugging: console messages, network requests, DOM inspection. |
16+
| `jq` / `grep` | Parse debug log output. |
17+
18+
## Prerequisites
19+
20+
- Quarto dev version built (`./configure.sh` or `./configure.cmd`)
21+
- Test environment configured (`tests/configure-test-env.sh` or `.ps1`)
22+
- `/agent-browser` CLI installed (preferred), OR Chrome + Chrome DevTools MCP connected
23+
24+
## Starting Preview
25+
26+
Preview needs the test venv for Jupyter tests. Activate it first (`tests/.venv`), matching how `run-tests.sh` / `run-tests.ps1` do it.
27+
28+
```bash
29+
# Linux/macOS
30+
source tests/.venv/bin/activate
31+
./package/dist/bin/quarto preview <file-or-dir> --no-browser --port 4444
32+
33+
# Windows (Git Bash)
34+
source tests/.venv/Scripts/activate
35+
./package/dist/bin/quarto.cmd preview <file-or-dir> --no-browser --port 4444
36+
```
37+
38+
Use `--no-browser` to control browser connection. Use `--port` for a predictable URL.
39+
40+
### With debug logging
41+
42+
```bash
43+
./package/dist/bin/quarto preview <file> --no-browser --port 4444 --log-level debug 2>&1 | tee preview.log
44+
```
45+
46+
### In background
47+
48+
```bash
49+
# Linux/macOS (after venv activation)
50+
./package/dist/bin/quarto preview <file> --no-browser --port 4444 &
51+
PREVIEW_PID=$!
52+
# ... run verification ...
53+
kill $PREVIEW_PID
54+
55+
# Windows (Git Bash, after venv activation)
56+
./package/dist/bin/quarto.cmd preview <file> --no-browser --port 4444 &
57+
PREVIEW_PID=$!
58+
# ... run verification ...
59+
kill $PREVIEW_PID
60+
```
61+
62+
## Edit-Verify Cycle
63+
64+
The core test pattern:
65+
66+
1. Start preview with `--no-browser --port 4444`
67+
2. Use `/agent-browser` to navigate to `http://localhost:4444/` and verify content
68+
3. Edit source file, wait 3-5 seconds for re-render
69+
4. Verify content updated in browser
70+
5. Check filesystem for unexpected artifacts
71+
6. Stop preview, verify cleanup
72+
73+
## What to Verify
74+
75+
**In browser** (via `/agent-browser`): Page loads, content matches source, updates reflect edits.
76+
77+
**In terminal/logs**: No `BadResource` errors, no crashes, preview stays responsive.
78+
79+
**On filesystem**: No orphaned temp files, cleanup happens on exit.
80+
81+
## Windows Limitations
82+
83+
On Windows, `kill` from Git Bash does not trigger Quarto's `onCleanup` handler (SIGINT doesn't propagate to Windows processes the same way). Cleanup-on-exit verification requires an interactive terminal with Ctrl+C. For automated testing, verify artifacts *during* preview instead.
84+
85+
## Context Types
86+
87+
Preview behaves differently depending on input:
88+
89+
| Input | Code path |
90+
|-------|-----------|
91+
| Single file (no project) | `preview()` -> `renderForPreview()` |
92+
| File within a project | May redirect to project preview via `serveProject()` |
93+
| Project directory | `serveProject()` -> `watchProject()` |
94+
95+
See `llm-docs/preview-architecture.md` for the full architecture.
96+
97+
## When NOT to Use
98+
99+
- Automated smoke tests — use `tests/smoke/` instead
100+
- Testing render output only (no live preview needed) — use `quarto render`
101+
- CI environments without browser access
102+
103+
## Test Fixtures and Cases
104+
105+
Test fixtures live in `tests/docs/manual/preview/`. The full test matrix is in `tests/docs/manual/preview/README.md`.
106+
107+
## Baseline Comparison
108+
109+
Compare dev build against installed release to distinguish regressions:
110+
111+
```bash
112+
quarto --version # installed
113+
./package/dist/bin/quarto --version # dev
114+
```
115+
116+
If both show the same issue, it's pre-existing.

‎.claude/rules/testing/typescript-tests.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,3 +92,20 @@ const fixtureDir = docs("my-fixture"); // → tests/docs/my-fixture/
9292
- `tests/docs/<feature>/` - Test fixtures
9393

9494
**Details:** `llm-docs/testing-patterns.md` for comprehensive patterns and examples.
95+
96+
## Common Test Utilities
97+
98+
**Constructing `MappedString` values:**
99+
```typescript
100+
import { asMappedString } from "../../../src/core/lib/mapped-text.ts";
101+
102+
// Use asMappedString("") instead of casting or constructing MappedString manually
103+
const markdown = asMappedString("");
104+
const markdownWithContent = asMappedString("# Title\nSome content");
105+
```
106+
107+
**Mock ProjectContext:**
108+
```typescript
109+
import { createMockProjectContext } from "./utils.ts"; // tests/unit/project/utils.ts
110+
const project = createMockProjectContext(); // Creates temp dir + FileInformationCacheMap
111+
```

‎llm-docs/preview-architecture.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
---
2+
main_commit: 94ebb7f79
3+
analyzed_date: 2026-03-31
4+
key_files:
5+
- src/command/preview/cmd.ts
6+
- src/command/preview/preview.ts
7+
- src/project/serve/serve.ts
8+
- src/project/serve/watch.ts
9+
- src/project/project-shared.ts
10+
- src/execute/jupyter/jupyter.ts
11+
- src/execute/engine.ts
12+
---
13+
14+
# Preview Architecture
15+
16+
How `quarto preview` works, from CLI entry through rendering and file watching.
17+
18+
## Entry Points
19+
20+
- `src/command/preview/cmd.ts` — CLI command handler, routing logic
21+
- `src/command/preview/preview.ts` — Single-file preview lifecycle
22+
- `src/project/serve/serve.ts` — Project preview via `serveProject()`
23+
- `src/project/serve/watch.ts` — Project file watcher
24+
25+
## cmd.ts Branching (5 Paths)
26+
27+
The command handler in `cmd.ts` determines which preview mode to use. The key variables `file` and `projectTarget` are mutated as a state machine to route between paths.
28+
29+
```
30+
quarto preview [input]
31+
│
32+
▼
33+
isFile(input)?
34+
├── YES ──► Create ProjectContext, detect format
35+
│ ├── Shiny? ──► previewShiny() / serve() ──► EXIT (Path C)
36+
│ ├── Serveable project, file NOT in inputs?
37+
│ │ └── .md + external previewer ──► file = project.dir (Path B1)
38+
│ ├── Serveable project, file IN inputs?
39+
│ │ └── HTML/serve output ──► renderProject() then file = project.dir (Path B2)
40+
│ └── None of above ──► file stays as-is (Path A)
41+
│
42+
└── NO (directory) ──► straight to isDirectory check (Path D)
43+
│
44+
▼
45+
isDirectory(file)? ← file may have been mutated above
46+
├── YES ──► serveProject(projectTarget, ...) ← projectTarget may be ProjectContext
47+
└── NO ──► preview(file, ..., project) ← single-file preview
48+
```
49+
50+
### Path details
51+
52+
| Path | Input | Condition | `file` mutated? | Terminal action |
53+
|------|-------|-----------|-----------------|-----------------|
54+
| A | file | Not in serveable project | No | `preview()` |
55+
| B1 | file | `.md` not in project inputs + external previewer | `file = project.dir` | `serveProject()` |
56+
| B2 | file | In project inputs, HTML/serve output | `file = project.dir` | `serveProject()` (after pre-render) |
57+
| C | file | Shiny document | N/A (exits early) | `previewShiny()`/`serve()` |
58+
| D | directory | User passed directory or cwd | N/A (isFile skipped) | `serveProject()` |
59+
60+
The `file` mutation pattern (`file = project.dir`) is intentional design by JJ Allaire (2022, commit `5508ace5bd`). It converts a single-file preview into a project preview when the file lives in a serveable project, so the browser gets full project navigation.
61+
62+
`projectTarget` (`string | ProjectContext`) carries the context to `serveProject()`, which accepts both types. When it receives a string, it resolves the project itself.
63+
64+
## Single-File Preview Lifecycle (Path A)
65+
66+
### Context creation
67+
68+
`cmd.ts` creates a `ProjectContext` for format detection (routing decisions). This context is passed to `preview()` via the `pProject` parameter to avoid creating a duplicate.
69+
70+
```typescript
71+
// cmd.ts creates context for routing
72+
project = (await projectContext(dirname(file), nbContext)) ||
73+
(await singleFileProjectContext(file, nbContext));
74+
75+
// preview() reuses it
76+
export async function preview(
77+
file, flags, pandocArgs, options,
78+
pProject?: ProjectContext, // reused from cmd.ts
79+
)
80+
```
81+
82+
This mirrors `render()`'s `pContext` pattern in `render-shared.ts`.
83+
84+
### Startup sequence
85+
86+
1. `preview()` receives or creates `ProjectContext`
87+
2. `previewFormat()` determines output format (calls `renderFormats()` if `--to` not specified)
88+
3. `renderForPreview()` does the initial render
89+
4. `createChangeHandler()` sets up file watchers
90+
5. HTTP dev server starts
91+
92+
### Re-render on file change
93+
94+
When the watched source file changes:
95+
96+
1. `createChangeHandler` triggers the `render` closure
97+
2. `renderForPreview()` is called with the **same** `project` from the closure
98+
3. `invalidateForFile(file)` cleans up the transient notebook and removes the cache entry
99+
4. `render()` runs with the project context, which creates a fresh target/notebook
100+
5. Browser reloads
101+
102+
The project context persists across all re-renders. Only the per-file cache entry is invalidated.
103+
104+
## FileInformationCache and invalidateForFile
105+
106+
`FileInformationCacheMap` stores per-file cached data:
107+
108+
| Field | Content | Cost of re-computation |
109+
|-------|---------|----------------------|
110+
| `fullMarkdown` | Expanded markdown with includes | Re-reads file, re-expands includes |
111+
| `includeMap` | Include source→target mappings | Recomputed with markdown |
112+
| `codeCells` | Parsed code cells | Recomputed from markdown |
113+
| `engine` | Execution engine instance | Re-determined |
114+
| `target` | Execution target (includes `.quarto_ipynb` path) | Re-created by `target()` |
115+
| `metadata` | YAML front matter | Recomputed from markdown |
116+
| `brand` | Resolved `_brand.yml` data | Re-loaded from disk |
117+
118+
### invalidateForFile() (added for #14281)
119+
120+
Before each preview re-render, the cache entry for the changed file must be invalidated so fresh content is picked up. `invalidateForFile()` does two things:
121+
122+
1. Deletes any transient `.quarto_ipynb` file from disk (if the cached target is transient)
123+
2. Removes the cache entry
124+
125+
Without step 1, the Jupyter engine's `target()` function sees the old file on disk and its collision-avoidance loop creates numbered variants (`_1`, `_2`, etc.) that accumulate.
126+
127+
### cleanupFileInformationCache()
128+
129+
Called at project cleanup (preview exit). Delegates to `invalidateForFile()` for each cache entry, removing all transient files and clearing the cache. This is the final cleanup — `invalidateForFile()` handles per-render cleanup for individual files.
130+
131+
## Transient Notebook Lifecycle (.quarto_ipynb)
132+
133+
When rendering a `.qmd` with a Jupyter kernel, the engine creates a transient `.ipynb` notebook:
134+
135+
1. `target()` in `jupyter.ts` generates the path: `{stem}.quarto_ipynb`
136+
2. If the file already exists, a collision-avoidance loop appends `_1`, `_2`, etc.
137+
3. The target is marked `data: { transient: true }`
138+
4. `execute()` runs the notebook through Jupyter
139+
5. `cleanupNotebook()` flips `transient = false` if `keep-ipynb: true`
140+
6. At preview exit, `cleanupFileInformationCache()` deletes files where `transient = true`
141+
142+
## Context Computation Count (Summary)
143+
144+
| Scenario | Startup computations | Per-change |
145+
|----------|---------------------|------------|
146+
| Single file, no project | 1 (cmd.ts, passed to preview) | 0 (cached project reused) |
147+
| Single file in serveable project | 1 (cmd.ts, passed to serveProject) | See project rows |
148+
| Project directory | 1 (serve.ts) | See project rows |
149+
| Project: single input changed | — | 1 (render() without pContext) |
150+
| Project: multiple inputs changed | — | 0 (renderProject reuses cached) |
151+
| Project: config file changed (HTML) | — | 1 (refreshProjectConfig) |
152+
153+
## Key Files
154+
155+
| File | Purpose |
156+
|------|---------|
157+
| `src/command/preview/cmd.ts` | CLI handler, routing state machine |
158+
| `src/command/preview/preview.ts` | Single-file preview lifecycle, `renderForPreview()`, `previewFormat()` |
159+
| `src/project/serve/serve.ts` | `serveProject()` — project preview server |
160+
| `src/project/serve/watch.ts` | `watchProject()` — file watcher, `refreshProjectConfig()` |
161+
| `src/command/render/render-shared.ts` | `render()` — accepts optional `pContext` |
162+
| `src/command/render/render-contexts.ts` | `renderContexts()`, `renderFormats()` — calls `fileExecutionEngineAndTarget()` |
163+
| `src/execute/engine.ts` | `fileExecutionEngineAndTarget()` — caching wrapper |
164+
| `src/execute/jupyter/jupyter.ts` | `target()` — creates `.quarto_ipynb`, collision-avoidance loop |
165+
| `src/project/project-shared.ts` | `FileInformationCacheMap`, `invalidateForFile()`, `cleanupFileInformationCache()` |
166+
| `src/project/types/single-file/single-file.ts` | `singleFileProjectContext()` — creates minimal context |

‎news/changelog-1.10.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ All changes included in 1.10:
33
## Regression fixes
44

55
- ([#14267](https://github.com/quarto-dev/quarto-cli/issues/14267)): Fix Windows paths with accented characters (e.g., `C:\Users\Sébastien\`) breaking dart-sass compilation.
6+
- ([#14281](https://github.com/quarto-dev/quarto-cli/issues/14281)): Fix transient `.quarto_ipynb` files accumulating during `quarto preview` with Jupyter engine.
67

78
## Formats
89

@@ -12,6 +13,10 @@ All changes included in 1.10:
1213

1314
## Commands
1415

16+
### `quarto preview`
17+
18+
- ([#14281](https://github.com/quarto-dev/quarto-cli/issues/14281)): Avoid creating a duplicate `.quarto_ipynb` file on preview startup for single-file Jupyter documents.
19+
1520
### `quarto create`
1621

1722
- ([#14250](https://github.com/quarto-dev/quarto-cli/issues/14250)): Fix `quarto create` producing read-only files when Quarto is installed via system packages (e.g., `.deb`). Files copied from installed resources now have user-write permission ensured.

‎src/command/preview/cmd.ts‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -274,10 +274,11 @@ export const previewCommand = new Command()
274274
// and convert the render to a project one
275275
let touchPath: string | undefined;
276276
let projectTarget: string | ProjectContext = file;
277+
let project: ProjectContext | undefined;
277278
if (Deno.statSync(file).isFile) {
278279
// get project and preview format
279280
const nbContext = notebookContext();
280-
const project = (await projectContext(dirname(file), nbContext)) ||
281+
project = (await projectContext(dirname(file), nbContext)) ||
281282
(await singleFileProjectContext(file, nbContext));
282283
const formats = await (async () => {
283284
const services = renderServices(nbContext);
@@ -431,6 +432,6 @@ export const previewCommand = new Command()
431432
[kProjectWatchInputs]: options.watchInputs,
432433
timeout: options.timeout,
433434
presentation: options.presentation,
434-
});
435+
}, project);
435436
}
436437
});

‎src/command/preview/preview.ts‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -148,10 +148,13 @@ export async function preview(
148148
flags: RenderFlags,
149149
pandocArgs: string[],
150150
options: PreviewOptions,
151+
pProject?: ProjectContext,
151152
) {
152-
const nbContext = notebookContext();
153-
// see if this is project file
154-
const project = (await projectContext(file, nbContext)) ||
153+
// Reuse the project context from cmd.ts if provided, avoiding redundant
154+
// context creation and transient notebook file duplication (#14281).
155+
const nbContext = pProject?.notebookContext ?? notebookContext();
156+
const project = pProject ??
157+
(await projectContext(file, nbContext)) ??
155158
(await singleFileProjectContext(file, nbContext));
156159
onCleanup(() => {
157160
project.cleanup();
@@ -427,9 +430,10 @@ export async function renderForPreview(
427430
// Invalidate file cache for the file being rendered so changes are picked up.
428431
// The project context persists across re-renders in preview mode, but the
429432
// fileInformationCache contains file content that needs to be refreshed.
430-
// TODO(#13955): Consider adding a dedicated invalidateForFile() method on ProjectContext
433+
// Uses invalidateForFile() to also clean up transient notebook files
434+
// (.quarto_ipynb) from disk before removing the cache entry (#14281).
431435
if (project?.fileInformationCache) {
432-
project.fileInformationCache.delete(file);
436+
project.fileInformationCache.invalidateForFile(file);
433437
}
434438

435439
// render

0 commit comments

Comments
 (0)