|
| 1 | +# SuperDoc |
| 2 | + |
| 3 | +A document editing and rendering library for the web. |
| 4 | + |
| 5 | +## Architecture: Rendering |
| 6 | + |
| 7 | +SuperDoc uses its own rendering pipeline — **ProseMirror is NOT used for visual output**. |
| 8 | + |
| 9 | +``` |
| 10 | +PM Doc (hidden) → pm-adapter → FlowBlock[] → layout-engine → Layout[] → DomPainter → DOM |
| 11 | +``` |
| 12 | + |
| 13 | +- `PresentationEditor` wraps a hidden ProseMirror `Editor` instance for document state and editing commands |
| 14 | +- The hidden Editor's contenteditable DOM is never shown to the user |
| 15 | +- **DomPainter** (`layout-engine/painters/dom/`) owns all visual rendering |
| 16 | +- Style-resolved properties (backgrounds, fonts, borders, etc.) must flow through `pm-adapter` → DomPainter, not through PM decorations |
| 17 | + |
| 18 | +### Where visual changes go |
| 19 | + |
| 20 | +| Change | Where | |
| 21 | +|--------|-------| |
| 22 | +| How something looks | `pm-adapter/` (data) + `painters/dom/` (rendering) | |
| 23 | +| Style resolution | `style-engine/` | |
| 24 | +| Editing behavior | `super-editor/src/extensions/` | |
| 25 | + |
| 26 | +**Do NOT** add ProseMirror decoration plugins for visual styling — DomPainter handles rendering. |
| 27 | + |
| 28 | +### State Communication |
| 29 | + |
| 30 | +State flows from super-editor → Layout Engine via: |
| 31 | +- `PresentationEditor.ts` listens to editor events (`super-editor/src/core/presentation-editor/`) |
| 32 | +- Calls DomPainter methods to update state |
| 33 | +- DomPainter re-renders with new state |
| 34 | + |
| 35 | +## Project Structure |
| 36 | + |
| 37 | +``` |
| 38 | +packages/ |
| 39 | + superdoc/ Main entry point (npm: superdoc) |
| 40 | + react/ React wrapper (@superdoc-dev/react) |
| 41 | + super-editor/ ProseMirror editor (@superdoc/super-editor) |
| 42 | + layout-engine/ Layout & pagination pipeline |
| 43 | + contracts/ - Shared type definitions |
| 44 | + pm-adapter/ - ProseMirror → Layout bridge |
| 45 | + layout-engine/ - Pagination algorithms |
| 46 | + layout-bridge/ - Pipeline orchestration |
| 47 | + painters/dom/ - DOM rendering |
| 48 | + style-engine/ - OOXML style resolution |
| 49 | + ai/ AI integration |
| 50 | + collaboration-yjs/ Collaboration server |
| 51 | +shared/ Internal utilities |
| 52 | +e2e-tests/ Playwright tests |
| 53 | +tests/visual/ Visual regression tests (Playwright + R2 baselines) |
| 54 | +``` |
| 55 | + |
| 56 | +## Where to Look |
| 57 | + |
| 58 | +| Task | Location | |
| 59 | +|------|----------| |
| 60 | +| React integration | `packages/react/src/SuperDocEditor.tsx` | |
| 61 | +| Editing features | `super-editor/src/extensions/` | |
| 62 | +| Presentation mode visuals | `layout-engine/painters/dom/src/features/feature-registry.ts` → feature module | |
| 63 | +| Rendering orchestration | `layout-engine/painters/dom/src/renderer.ts` | |
| 64 | +| DOCX import/export | `super-editor/src/core/super-converter/` | |
| 65 | +| Style resolution | `layout-engine/style-engine/` | |
| 66 | +| Main entry point (Vue) | `superdoc/src/SuperDoc.vue` | |
| 67 | +| Visual regression tests | `tests/visual/` (see its CLAUDE.md) | |
| 68 | +| Document API contract | `packages/document-api/src/contract/operation-definitions.ts` | |
| 69 | +| Adding a doc-api operation | See `packages/document-api/README.md` § "Adding a new operation" | |
| 70 | +| Theming (`createTheme()`) | `packages/superdoc/src/core/theme/create-theme.js` | |
| 71 | +| CSS variable defaults | `packages/superdoc/src/assets/styles/helpers/variables.css` | |
| 72 | +| Preset themes | `packages/superdoc/src/assets/styles/helpers/themes.css` | |
| 73 | +| Consumer-facing agent guide | `packages/superdoc/AGENTS.md` (ships with npm package) | |
| 74 | + |
| 75 | +## Style Resolution Boundary |
| 76 | + |
| 77 | +**The importer stores raw OOXML properties. The style-engine resolves them at render time.** |
| 78 | + |
| 79 | +- The converter (`super-converter/`) should only parse and store what is explicitly in the XML (inline properties, style references). It must NOT resolve style cascades, conditional formatting, or inherited properties. |
| 80 | +- The style-engine (`layout-engine/style-engine/`) is the single source of truth for cascade logic. All style resolution (defaults → table style → conditional formatting → inline overrides) happens here. |
| 81 | +- Both rendering systems call the style-engine to compute final visual properties. |
| 82 | + |
| 83 | +**Why**: Resolving styles during import bakes them into node attributes as inline properties. On export, these get written as direct formatting instead of style references, losing the original document intent. |
| 84 | + |
| 85 | +## When to Modify Which System |
| 86 | + |
| 87 | +- **Visual rendering**: Check `painters/dom/src/features/feature-registry.ts` to find the feature module, then modify it. If no module exists yet, create one (see layout-engine CLAUDE.md). Feed data via `pm-adapter/` |
| 88 | +- **Style resolution**: Modify `style-engine/` — called by pm-adapter during conversion |
| 89 | +- **Editing commands/behavior**: Modify `super-editor/src/extensions/` |
| 90 | +- **State bridging**: Modify `PresentationEditor.ts` |
| 91 | + |
| 92 | +## Document API Contract |
| 93 | + |
| 94 | +The `packages/document-api/` package uses a contract-first pattern with a single source of truth. |
| 95 | + |
| 96 | +- **`operation-definitions.ts`** — canonical object defining every operation's key, metadata, member path, reference doc path, and group. All downstream maps are projected from this file automatically. |
| 97 | +- **`operation-registry.ts`** — type-level registry mapping each operation to its `input`, `options`, and `output` types. |
| 98 | +- **`invoke.ts`** — `TypedDispatchTable` validates dispatch wiring against the registry at compile time. |
| 99 | + |
| 100 | +Adding a new operation touches 4 files: `operation-definitions.ts`, `operation-registry.ts`, `invoke.ts` (dispatch table), and the implementation. See `packages/document-api/README.md` for the full guide. |
| 101 | + |
| 102 | +Do NOT hand-edit `COMMAND_CATALOG`, `OPERATION_MEMBER_PATH_MAP`, `OPERATION_REFERENCE_DOC_PATH_MAP`, or `REFERENCE_OPERATION_GROUPS` — they are derived from `OPERATION_DEFINITIONS`. |
| 103 | + |
| 104 | +## JSDoc types |
| 105 | + |
| 106 | +Many packages use `.js` files with JSDoc `@typedef` for type definitions (e.g., `packages/superdoc/src/core/types/index.js`). These typedefs ARE the published type declarations — `vite-plugin-dts` generates `.d.ts` files from them. |
| 107 | + |
| 108 | +- **Keep JSDoc typedefs in sync with code.** If a function destructures `{ a, b, c }`, the `@typedef` must include all three properties. Missing properties become type errors for consumers. |
| 109 | +- **Verify types after adding parameters.** When adding a parameter to a function, update its `@typedef` or `@param` JSDoc. Build with `pnpm run --filter superdoc build:es` and check the generated `.d.ts` in `dist/`. |
| 110 | +- **Workspace packages don't publish types.** `@superdoc/common`, `@superdoc/contracts`, etc. are private. If a public API references their types, those types must be inlined or resolved through path aliases — consumers can't resolve workspace packages. |
| 111 | + |
| 112 | +## Commands |
| 113 | + |
| 114 | +- `pnpm build` - Build all packages |
| 115 | +- `pnpm test` - Run tests |
| 116 | +- `pnpm dev` - Start dev server (from examples/) |
| 117 | +- `pnpm run generate:all` - Generate all derived artifacts (schemas, SDK clients, tool catalogs, reference docs) |
| 118 | + |
| 119 | +## AI Eval Suite |
| 120 | + |
| 121 | +The `evals/` directory contains a Promptfoo-based evaluation suite for validating AI tool call quality. |
| 122 | + |
| 123 | +| Command | What it does | Cost | |
| 124 | +|---------|-------------|------| |
| 125 | +| `pnpm --filter @superdoc-testing/evals run eval` | Run deterministic evals (reading + argument tests) | ~$0.30 | |
| 126 | +| `pnpm --filter @superdoc-testing/evals run eval:reading` | Run reading tool tests only | ~$0.15 | |
| 127 | +| `pnpm --filter @superdoc-testing/evals run eval:gdpval` | Run GDPval benchmark (Model+SuperDoc vs Model-Only) | ~$1-2 | |
| 128 | +| `pnpm --filter @superdoc-testing/evals run eval:view` | Open Promptfoo web UI with results | Free | |
| 129 | +| `pnpm --filter @superdoc-testing/evals run baseline:save <label>` | Save versioned results snapshot | Free | |
| 130 | + |
| 131 | +Tool definitions are extracted from `packages/sdk/tools/` via `evals/tools/extract.mjs`. Run `pnpm run generate:all` first if SDK artifacts are missing. |
| 132 | + |
| 133 | +Test files are YAML in `evals/tests/`. Each test has a `vars.task` prompt and JavaScript assertions that check tool call structure (Level 1: tool selection + argument accuracy, not execution). |
| 134 | + |
| 135 | +The system prompt at `evals/prompts/agent.txt` is a copy of the proven prompt from `examples/eval-demo/lib/agent.ts`. Update both when changing the prompt. |
| 136 | + |
| 137 | +## Generated Artifacts |
| 138 | + |
| 139 | +These directories are produced by `pnpm run generate:all`: |
| 140 | + |
| 141 | +| Directory | In git? | What it contains | |
| 142 | +|-----------|---------|-----------------| |
| 143 | +| `packages/document-api/generated/` | No (gitignored) | Agent artifacts, JSON schemas | |
| 144 | +| `apps/cli/generated/` | No (gitignored) | SDK contract JSON exported from CLI metadata | |
| 145 | +| `packages/sdk/langs/node/src/generated/` | No (gitignored) | Node SDK generated client code | |
| 146 | +| `packages/sdk/langs/python/superdoc/generated/` | No (gitignored) | Python SDK generated client code | |
| 147 | +| `packages/sdk/tools/*.json` | No (gitignored) | Tool catalogs for all providers (catalog.json, tools.openai.json, etc.) | |
| 148 | +| `apps/docs/document-api/reference/` | Yes (Mintlify deploys from git) | Reference doc pages generated from contract | |
| 149 | + |
| 150 | +After a fresh clone, run `pnpm run generate:all` before working on SDK, CLI, or doc-api code. |
| 151 | + |
| 152 | +Note: `packages/sdk/tools/__init__.py` is a manual file (Python package marker) and stays committed. |
| 153 | + |
| 154 | +## Testing |
| 155 | + |
| 156 | +| What to verify | Command | Speed | |
| 157 | +|---|---|---| |
| 158 | +| Logic works? | `pnpm test` | seconds | |
| 159 | +| Editing works? | `pnpm test:behavior` | minutes | |
| 160 | +| Layout regressed? | `pnpm test:layout` | ~10 min | |
| 161 | +| Pixel diff? | `pnpm test:visual` | ~5 min | |
| 162 | + |
| 163 | +### Unit Tests (Vitest) |
| 164 | + |
| 165 | +Co-located with source code as `feature.test.ts` next to `feature.ts`. Test pure logic, data transformations, and utilities in isolation. |
| 166 | + |
| 167 | +- Framework: **Vitest** (config at `vitest.config.mjs`) |
| 168 | +- Most coverage in `packages/super-editor/` (526 files) and `packages/layout-engine/` (150 files) |
| 169 | +- Run a single package: `pnpm --filter <package> test` |
| 170 | + |
| 171 | +### Behavior Tests (Playwright) |
| 172 | + |
| 173 | +End-to-end tests that exercise editing features through the browser. Located in `tests/behavior/`. |
| 174 | + |
| 175 | +- Framework: **Playwright** (Chromium, Firefox, WebKit) |
| 176 | +- Tests editing commands, formatting, tables, comments, tracked changes, lists, toolbar |
| 177 | +- Asserts on document state, not pixels — see `tests/behavior/README.md` |
| 178 | + |
| 179 | +### Layout Comparison (`pnpm test:layout`) |
| 180 | + |
| 181 | +Compares layout engine output (JSON structure) across ~382 test documents against a published npm version. This is the primary tool for catching rendering regressions. |
| 182 | + |
| 183 | +- Run: `pnpm test:layout` (interactive — prompts for reference version) |
| 184 | +- Flags: `--reference <version>`, `--match <pattern>`, `--limit <n>` |
| 185 | +- Handles auth, corpus download, build, and comparison automatically |
| 186 | +- Reports written to `tests/layout/reports/` |
| 187 | +- Lower-level access: `pnpm layout:compare` (same engine, no interactive UX) |
| 188 | +- One-time setup: `npx wrangler login` (for corpus download from R2) |
| 189 | + |
| 190 | +### Visual Comparison (`pnpm test:visual`) |
| 191 | + |
| 192 | +Pixel-level before/after comparison for documents that failed layout comparison. Reads the latest layout report and generates an HTML diff report. |
| 193 | + |
| 194 | +- Run `pnpm test:layout` first to generate a comparison report |
| 195 | +- Then `pnpm test:visual` to see pixel differences for changed docs |
| 196 | +- HTML report output in `devtools/visual-testing/results/` |
| 197 | + |
| 198 | +### Uploading Test Documents to Corpus |
| 199 | + |
| 200 | +Test documents for layout and visual tests are stored in R2. Rendering tests auto-discover all `.docx` files in the corpus — just upload a file and it becomes a test case. |
| 201 | + |
| 202 | +**Interactive** (prompts for issue ID and description): |
| 203 | +```bash |
| 204 | +pnpm corpus:upload ~/Downloads/my-file.docx |
| 205 | +``` |
| 206 | + |
| 207 | +**Non-interactive** (for scripts and agents): |
| 208 | +```bash |
| 209 | +pnpm corpus:upload ~/Downloads/my-file.docx --issue SD-1234 --description short-kebab-desc |
| 210 | +``` |
| 211 | + |
| 212 | +Files are uploaded to `rendering/<issue-id>-<description>.docx`. After uploading: |
| 213 | +```bash |
| 214 | +pnpm corpus:pull # sync the new file locally |
| 215 | +pnpm test:visual # verify it renders |
| 216 | +``` |
| 217 | + |
| 218 | +One-time setup: `npx wrangler login` (for R2 access). If the token expires, run it again. Note: wrangler may write to `~/.wrangler/config/` while the corpus scripts read from `~/Library/Preferences/.wrangler/config/` — copy the token if you get auth errors after a fresh login. |
| 219 | + |
| 220 | +## Brand & Design System |
| 221 | + |
| 222 | +Brand guidelines, voice, and design tokens live in `brand/`. |
| 223 | +Token contract source is `packages/superdoc/src/assets/styles/helpers/variables.css` (`:root` defaults). |
| 224 | +Preset theme overrides are defined in `packages/superdoc/src/assets/styles/helpers/themes.css`. |
| 225 | + |
| 226 | +**When creating or modifying UI components:** |
| 227 | +- Use `--sd-*` CSS custom properties — never hardcode hex values. |
| 228 | +- Treat `variables.css` as the canonical token contract; add new tokens there. |
| 229 | +- Keep preset themes in `themes.css` (`.sd-theme-*`) and override only the tokens that need theme-specific values. |
| 230 | +- Tokens are organized by layers: primitive (`--sd-color-blue-500`) → UI/document tokens (`--sd-ui-*`, `--sd-comments-*`, etc.) → component usage. |
| 231 | +- Expose UI component-specific variables as `--sd-ui-{component}-*` so consumers can customize via CSS. |
| 232 | + |
| 233 | +**When writing copy or content:** see `brand.md` for the full brand identity — strategy, voice, and visual guidelines. Product name is always **SuperDoc** (capital S, capital D). |
0 commit comments