Skip to content

Commit e0f7193

Browse files
authored
Enable pdf support (#51)
* feat: add PDF document generation (ha:pdf, ha:pdf-charts, ha:doc-core) PDF 1.7 document generation with flow layout, auto-pagination, and professional output quality. Includes E2E test infrastructure with automated visual validation. Modules: - ha:doc-core: extracted format-agnostic themes, colours, validation from ha:ooxml-core (shared by PPTX and PDF) - ha:pdf: core PDF generation (~3500 LOC) - Flow layout engine with auto-pagination - Text elements: paragraph, heading, bulletList, numberedList, richText, codeBlock, quote - Tables: table (with columnAlign), kvTable (right-aligned values), comparisonTable, with 5 style presets - Images: JPEG (DCTDecode) and PNG (FlateDecode) embedding - Page templates: titlePage, contentPage, twoColumnPage, quotePage - Document furniture: addPageNumbers, addFooter - Layout: twoColumn inline flow element, spacer, pageBreak, rule - Serialization: serializeDocument / restoreDocument - Cursor tracking for mixed low-level + flow content - Standard 14 PDF fonts with glyph width metrics - ha:pdf-charts: bar, line, pie, combo charts as PDF drawing ops Skill & patterns: - skills/pdf-expert/SKILL.md: comprehensive LLM guidance - Updated patterns/file-generation with PDF support - Updated docs/MODULES.md, skills tables Agent improvements (benefit ALL modules, not just PDF): - module_info now returns typeDefinitions showing full interface shapes from .d.ts files — LLM can discover ALL parameter options (columnAlign, style, colWidths, etc.) without guessing - System prompt instructs LLM to READ typeDefinitions - Validation error hint clarifies handler was NOT registered - Interface extraction via brace-counting parser in format-exports.ts - Fixed pre-existing type errors in docgen-modules.test.ts E2E test infrastructure: - scripts/run-pdf-prompts.sh: automated prompt runner - Requires qpdf + pdftoppm (fails loudly if missing) - Renders PDF pages to PNG for visual inspection - Structural validation (header, xref, EOF, qpdf --check) - Content checks from YAML expected_content - Generic feedback agent (_feedback-instructions.txt) - 10 test prompt YAMLs (invoice, report, resume, letter, etc.) Tests: 2027 total (241 PDF-specific), 33 test files, 0 failures Zero type errors (tsc --noEmit clean) * fix PDF zlib compliance and add content extraction tests * feat: addContent returns lastY, drawText accepts align option From LLM feedback: - addContent() now returns { lastY: number } so callers can continue drawing below flow content (critical for hybrid layouts like letters) - drawText accepts align: 'left' | 'center' | 'right' — no more manual measureText + X calculation for centering/right-aligning * fix: richText accepts segments alias + feedback-driven fixes * fix: module_info ALWAYS returns typeDefinitions in markdown format Root cause: typeDefinitions was only included when source was omitted (>10KB modules). Now: - Source code is NEVER included in module_info responses - typeDefinitions ALWAYS included as markdown with headers - Formatted with ## Parameter Types header and ### per interface - System prompt tells LLM to READ typeDefinitions section - Removes dead includeSource logic This fixes the #1 feedback issue across all prompt runs: LLM couldn't discover parameter names (columnAlign, spaceBefore, entries, etc.) because typeDefinitions was buried or missing from the response. * feat: track module_info calls and warn on uninspected imports Add modulesInspected Set to AgentState to track which ha:* modules the LLM has called module_info() on. When register_handler detects imports from uninspected modules, it adds an apiDiscoveryWarning to the result, nudging the LLM to read typeDefinitions before guessing. * fix: chart rendering — filled polygon pie wedges, right-aligned Y-axis labels, round markers BREAKING: ChartDrawOp now supports 'polygon' type with filled paths. Pie chart: replaces wireframe lines + rect indicators with proper filled polygon wedges (center → arc points → close). White borders between slices. Donut variant draws a white circle polygon over center. Line/combo chart: data point markers are now circular polygons instead of square rects. Y-axis labels right-aligned against the axis line. PDF renderer: new polygonOp() for PDF path fill/stroke, polygon case in chart draw op replay. Tests: 3 new regression tests (filled wedges, no rect indicators, donut hole, right-aligned labels). Total: 2041 passing. * feat: add document quality standards to pdf-expert skill Charts MUST have headings and interpretation paragraphs — no more naked charts stranded on empty pages like Robinson Crusoe. Added mandatory quality section: structure rules, content rules, quality checklist, good vs bad examples. Updated data-dashboard prompt to demand narrative context, executive summary, chart interpretation, and key takeaways. * fix: heading orphan prevention + chart text baseline alignment Headings no longer get orphaned from their following content. The orphan check now peeks at the next element and includes its estimated height — so a heading + 200pt chart won't split across pages. Chart text ops now apply the same fontSize baseline offset as flow layout renderLines — fixes X-axis labels overlapping with bars and chart titles rendering at wrong positions. X-axis label gap increased from 4pt to 8pt for cleaner separation. Loop changed from for-of to indexed for element lookahead. * fix: stronger heading orphan prevention — h1 looks 2 elements ahead Section titles (h1) now peek TWO elements ahead for the orphan check, not just one. Prevents h1 → h2 → chart sequences from stranding the h1 at the bottom of a page with massive whitespace. All headings now require at least 100pt of following content space, so even h2-h6 are less likely to orphan above large content blocks. * fix: remove aggressive 100pt orphan guard + adaptive Y-axis tick count The 100pt minimum orphan guard caused the LLM to spend 6 iterations fighting the layout engine trying to fit content on 2 pages. Removed the arbitrary minimum — the element lookahead (h1 peeks 2 ahead, others peek 1 ahead) already handles real orphan cases. Y-axis tick count now adapts to chart height. Small charts (100pt) get fewer ticks to prevent overlapping labels. Each tick needs ~18pt (8pt font + 10pt gap), so a 100pt chart gets max 5 ticks instead of 8+ that would overlap. Based on LLM feedback: the #1 pain point is height estimation — no way to predict element heights before rendering. estimateHeight() is the next priority. * feat: estimateHeight() + layout docs for page fitting Add estimateHeight(elements, opts?) that returns total vertical space (in points) an element array would consume — without rendering. Works for all element types including charts, tables, twoColumn (recursive). Based on LLM feedback: height estimation was the #1 pain point, requiring 6 iterations of trial-and-error to fit content on 2 pages. Also: - Fix heading JSDoc: spaceBefore defaults were wrong (said 24, actual 16) - Add font sizes and approximate heights per heading level to JSDoc - Document chart height as TOTAL (including axes/legend), not plot area - Add Layout Budget section to SKILL.md with height reference table and estimateHeight() usage example * feat: bullet encoding fix, metricCard, chart subtitle, compact tables, comparisonTable text values 5 features from LLM feedback: 1. Bullet char encoding: Unicode→WinAnsiEncoding mapping for standard 14 fonts. Bullet (•), em-dash, smart quotes, euro sign, etc. now render correctly instead of showing as double-quote characters. 2. comparisonTable: values can now be boolean OR string. Boolean renders as Y/N, string renders as-is. Enables metric comparisons, not just feature matrices. 3. metricCard(): new element — large value + label in a coloured box. For dashboard KPI display. Use inside twoColumn() for side-by-side. 4. Chart subtitle: all chart types accept subtitle option, rendered in smaller text below the title (e.g. 'Values in $M'). 5. Compact table mode: table({ compact: true }) reduces row padding from 2.2x to 1.6x fontSize for information-dense layouts. Also: prompt no longer demands 2 pages (was causing LLM to fight the layout engine). Uses 'Section' instead of 'Page', mentions metricCard, expects 3+ pages. * feat: metricCard change indicator + addContent page docs metricCard() now accepts an optional 'change' parameter (e.g. '+14%', '-2.3%') that renders as a coloured trend indicator: green for +, red for -, grey for neutral. Added clear documentation in SKILL.md about addContent() page behaviour: starts on current page, auto-creates new pages on overflow, multiple calls continue where the last left off. * fix: dark theme bg, describeThemes API, twoColumn richText, SKILL cleanup P0: addContent now fills page background with theme.bg on dark themes. Both initial page creation and ensureSpace page breaks fill the bg. Cursor reset after bg fill so content starts at margins.top, not page bottom. API: describeThemes() returns markdown table of all themes with colours, style notes, and recommended use cases. SKILL.md now says 'call describeThemes() or module_info("doc-core")' instead of hardcoding a theme table that goes stale. twoColumn: column renderer now handles richText and metricCard elements (were silently dropped via default:break — caused resume company names to disappear). SKILL.md: removed entries alias mention for kvTable (just use items), added strong guidance that light-clean should be used for documents, dark themes only for title pages. Quarterly-report prompt: changed from corporate-blue to light-clean. * feat: PDF runtime validation — overlap, bounds, whitespace detection Like PPTX's bounds/overflow validation, PDF now catches layout garbage at render time so the LLM gets an error instead of producing shit. Three checks run on exportToFile(): 1. TEXT OVERLAP: detects text boxes overlapping >30% — catches the 'SOLUTIONINVOICE' problem where headings render on top of each other. Error includes both texts, Y positions, and fix guidance. 2. CONTENT CLIPPED: detects text outside page bounds (negative X, beyond page width/height). Catches text pushed off-page edges. 3. EXCESSIVE WHITESPACE: interior pages with <15% content coverage and <3 text elements trigger a warning to merge with adjacent pages. Implementation: drawText records TextBox {x,y,w,h,text} on each page. validateDocument() runs O(n²) pair comparison per page. exportToFile throws on validation failure — LLM sees the error and can fix the layout. 4 new tests: overlap detection, bounds clipping, clean pass, export blocking. * fix: table cell text overflow — truncate with ellipsis Table text that's wider than its column now gets truncated with '...' instead of silently overflowing into adjacent cells. Affects both header and data rows in renderTable. Uses binary search to find the maximum text length that fits with ellipsis within the column width minus padding. Prevents 'GRAND TOTAL' from bleeding into the value column. * refactor: strip SKILLs to guidance only, DRY type resolution in module_info SKILLS: - pdf-expert: 386 → 155 lines (60% reduction) - pptx-expert: 470 → 124 lines (74% reduction) - Removed ALL API reference duplication — module_info typeDefinitions is now the single source of truth for parameters/types - SKILLs now contain ONLY: discovery instructions, quality rules, anti-patterns, layout guidelines, common mistakes, setup sequence module_info: - resolveTypeReferences() inlines cross-referenced types so the LLM sees TableStyle, ChartSeries, etc. without needing follow-up queries - Works for ALL modules with .d.ts files (17 modules) - Import added to index.ts, called in module_info handler table rendering: - fitCellText() truncates text with ellipsis when wider than column - Prevents 'GRAND TOTAL' from overflowing into adjacent cells * fix: PPTX image rel IDs now use standard rId* format Image relationships used rIdImage15 format instead of the OOXML-standard rId1/rId2/... sequential format. PowerPoint accepted them but flagged for repair, stripping associated notes slides as collateral damage. Root cause: embedImage() used a global image counter (rIdImage${idx}) while slide layout/notes/chart rels used per-slide relIdCounter (rId${n}). The non-standard IDs triggered PPT's repair dialog. Fix: buildPptxZip now remaps rIdImage* to rId${relIdCounter++} in both the slide rels AND the slide XML (r:embed attributes). The embed-time API is unchanged — remapping happens at build time. Also: modulesInspected warning now skips trivial modules (shared-state, xml-escape, base64, etc.) to reduce noise. * feat: maxPages option + remove ooxml-core re-exports P0: Removed all re-exports from ooxml-core — themes, validation, colours now live ONLY in doc-core. Tests updated to import from doc-core directly. No backwards compat needed — LLM discovers APIs fresh via module_info. P1: addContent({ maxPages: N }) auto-scales spacing to fit content within N pages. Estimates total height, calculates scale factor, applies it to spaceBefore/spaceAfter/lineHeight on paragraphs, headings, lists, richText, quotes. Minimum scale 0.4 to prevent unreadable compression. Font sizes are NOT reduced. This eliminates the #1 LLM pain point — letter prompt was doing 30 tool calls (15 edit_handler) for page-fitting iterations. * feat: underline text decoration + textBlock() for addresses TextRun now accepts 'underline: true' — draws a 0.5pt line under each line of text in richText paragraphs. Correctly handles multi-line wrapping (underline per line, not one long line). textBlock({ lines: [...] }) renders an array of strings with tight spacing (1.2x line height, 0pt spaceBefore) — ideal for addresses, contact info, and other compact multi-line text that would otherwise need 5+ separate paragraph() calls with spaceAfter:2 each. * fix(pdf): fix 23 wrong JSDoc, columns(n) layout, TableOptions ColumnDef, kvTable fixes - Fix 23 @param opts JSDoc descriptions shifted off-by-one (every function had the previous function's description) - Fix addContent JSDoc to mention maxPages (was hidden as 'Optional margins') - Fix TableOptions.columns inline type mangling extractInterfaces parser (use named ColumnDef) - Add columns(n) layout element (2-6 cols with independent widths) - Hoist renderColumn out of twoColumn, shared by both twoColumn and columns - kvTable: remove hardcoded Key/Value headers, add per-item bold, auto-widen key column - Increase type truncation limit 80->200 chars in extractInterfaces - Remove entries alias from KvTableOptions .d.ts (runtime alias still works) * fix(security): edit_handler now runs validator before committing edit_handler previously bypassed the static analysis validator entirely, allowing the LLM to register a minimal handler then edit_handler to inject unvalidated code. Now edit_handler pre-validates the edited code using the same validateJavaScriptGuest() flow as register_handler. If validation fails, the edit is rejected and the handler is NOT modified. * fix(validator): skip property access checks inside string literals The property access validator was doing raw byte scanning for .identifier patterns without tracking string literal boundaries. This caused false positives when string content contained dots (e.g. 'docs.rust-lang.org' or data containing '.rust'). Now tracks single/double/template literal state and skips dots inside strings. Also skips single-line comments. * fix(pdf): maxPages font scaling when spacing-only compression insufficient maxPages:1 was clamped at 0.4x spacing scale, which couldn't fit content exceeding 2.5x available space. Now: spacing scale lowered to 0.3 min, and font sizes reduced slightly (max 0.8x) when spacing alone can't fit. This ensures single-page docs like letters actually stay on one page. * fix(pdf): titlePage auto-wraps long subtitles instead of rendering off-page Long subtitle text was rendered at x=(pageWidth-textWidth)/2 which went negative when text exceeded page width, causing TEXT CLIPPED validation errors (x=-49). Now wraps subtitle to 75% page width and centers each line independently. * fix(agent): module_info(fn) now includes relevant typeDefinitions When calling module_info with specific function names, the response now includes typeDefinitions for the parameter types used by those functions. Previously type defs were only returned in the full module view, meaning the LLM had to make 2 calls to discover param shapes. Now a single module_info('pdf', 'paragraph') shows ParagraphOptions inline. * feat(pdf): add calloutBox element for highlights, warnings, info boxes Colored background box with left accent border, optional bold title, and wrapped body text. Requested by 4/10 prompts. Supports custom colors for bg, border, title, and body text. Integrates with flow layout via addContent() and estimateHeight(). * feat(pdf): textBlock firstLineBold option for address blocks Adds firstLineBold option to textBlock() so the first line (e.g. name) renders bold while remaining lines are regular weight. Uses richText internally when firstLineBold is set. Requested by 3/10 prompts. * feat(pdf): titlePage theme override, sectionHeading, table footerRow, textBlock firstLineBold - titlePage: add theme param to override document theme per-page - sectionHeading(): convenience wrapper returning heading+rule array - table: add footerRow option for bold totals/summary row with thicker border - textBlock: add firstLineBold for address blocks (name bold, rest normal) All requested by multiple prompts in the 10-prompt test suite. * feat(pdf): signatureLine, quote italic option, kvTable separator, overlap fix hints, table style names - signatureLine(): formal signature element with line, name, and title - quote: add italic option (default true, can set false for non-italic) - kvTable items: add separator option for thicker border above totals row - table/kvTable/comparisonTable: style JSDoc now lists all preset names - overlap validation error now suggests specific fix (increase spaceAfter) * feat(pdf): metricCard trend dot indicator, paragraph bold JSDoc visibility - metricCard: colored dot indicator (green/red) drawn before change text - Already had bold/italic in ParagraphOptions type — just LLM discoverability * fix(agent): --skill flag no longer sends /skill-name as user message The --skill flag was sending '/pdf-expert' as a literal user message to the LLM, which confused it ('The user is asking about a /pdf-expert command'). The skill content is already injected via preLoadedSkills → runSuggestApproach into the system message. The extra processMessage call was wasteful and confusing. Now --skill just logs the preloaded skills and lets the prompt processing handle skill injection. * fix(pdf): P0-P2 from round 3 review — sectionHeading, auto-flatten, title wrap, inline bold P0 (blocking quality — 8/10 prompts crashed): - sectionHeading() now returns a single PdfElement (was array, broke 8 prompts) - addContent() auto-flattens nested arrays (legacy safety net) - addContent error messages show element index - titlePage title auto-wraps long titles (was rendering off-page) P1 (API clarity): - ComparisonTableOptions: extract ComparisonOption named type (parser fix) P2 (enhancements): - paragraph() supports **bold** markdown markers inline - paragraph() align option already existed but was undiscoverable Tests: 19 new tests covering all changes (2064 total, all pass) * feat(pdf): P3 batch — watermark, hyperlinks, TOC, jobEntry, letterhead, verticalCenter P3-2: addWatermark() — diagonal semi-transparent text with ExtGState transparency support added to PDF serializer P3-3: jobEntry() — convenience for resume experience entries (title+company left, dates right, bullets below) P3-4: letterhead() — company header with name, address, contact, rule P3-5: link() — clickable hyperlinks via PDF Link annotations Added /Annots support to page objects in serializer P3-6: tableOfContents() — TOC with title/page entries using twoColumn P3-7: verticalCenter option on addContent() — centers content on page Infrastructure: - PageData: added extGStates (Map), links (array) fields - buildPdfBytes: ExtGState + Annots in page resource/object dicts - 8 new tests (2072 total, all pass) * fix(pdf): emoji chars corrupting PDF streams — qpdf EOF root cause Root cause: Characters outside WinAnsiEncoding (emoji, CJK, etc.) were passed through escapeTextString unchanged, then encodeText truncated them to their low byte via charCodeAt() & 0xFF. This produced raw control characters in PDF string literals: ✨ (U+2728) → low byte 0x28 → unescaped '(' in PDF string 🚀 (U+1F680) → low byte 0x80 → raw € byte 🐢 (U+1F422) → low byte 0x22 → raw '"' byte This caused unbalanced parentheses in Tj operators, which qpdf reported as 'EOF while reading token'. The bug was triggered by live GitHub API data containing emoji in repo descriptions. Fix: escapeTextString now strips characters with codepoints > 0xFF that aren't in the UNICODE_TO_WINANSI mapping. Surrogate pairs (emoji above U+FFFF) are also handled by skipping the low surrogate. 3 new tests reproduce the exact scenario with the Node.js repo description 'Node.js JavaScript runtime ✨🐢🚀✨'. * feat(pdf): Phase 11 — custom TrueType font embedding with Unicode support Full TrueType font embedding pipeline: 11a: TTF Binary Parser (~250 LOC) - Parses head, hhea, hmtx, maxp, cmap (format 4), name, post, OS/2 tables - Extracts per-glyph advance widths, Unicode→glyph mapping, font metrics - Handles both platform 0 (Unicode) and platform 3 (Windows) cmap subtables 11a: PDF Embedding (~200 LOC) - Type0 composite font with CIDFontType2 descendant - FontDescriptor with metrics (ascent, descent, bbox, stemV, flags) - FontFile2 stream (embedded TTF with FlateDecode compression) - ToUnicode CMap for text extraction (copy/paste, search) - /W array with per-glyph widths for used codepoints 11a: API - registerCustomFont(doc, { name, data: Uint8Array }) - measureText() auto-detects custom fonts and uses hmtx widths - drawText/paragraph/etc use hex glyph IDs for custom fonts 11b: Subsetting (stub) - Tracks used codepoints per font during rendering - subsetTTF() framework in place (returns full font for now) - TODO: Proper glyf/loca table rebuilding for CJK size reduction Unicode support: Cyrillic, Extended Latin, CJK (with appropriate font), and any Unicode codepoint the font supports. No more WinAnsi limitation. 15 new tests covering parser, registration, rendering, Unicode, PDF structure, and mixed standard+custom fonts (2090 total, all pass) * fix(pdf): custom font compression — skip when deflate output >= original Font data like DejaVu Sans (757KB) is already efficiently packed. deflate() output can be >= input size, making the zlib-wrapped stream larger than the original. PDF viewers couldn't decompress the 'compressed' font data. Now: only apply FlateDecode when compression actually saves space, otherwise embed uncompressed with no /Filter. Also: multilingual-report.yaml prompt for E2E testing custom fonts, pdftotext-based extraction test to verify ToUnicode CMap works. * feat(pdf): Phase 11b — real TTF font subsetting Replaced the stub subsetTTF with actual implementation: - Zeros out glyf data for unused glyphs (keeping original glyph IDs) - Scans composite glyph references recursively to include dependencies - Updates loca table to mark unused glyphs as zero-length - Zeroed regions compress extremely well with deflate Result: DejaVu Sans 757KB → <200KB PDF for typical documents (4-char test). Previously 764KB PDFs are now dramatically smaller. 3 new tests: null run detection, compressed size assertion, codepoint tracking. 2094 tests total, all pass. * fix(pdf): 11b font subsetting — physically compact glyf table The previous approach zeroed unused glyph data but kept the font at full size (757KB), relying on deflate compression to shrink it. But ha:ziplib deflate inside the sandbox doesn't handle 757KB inputs well. Now: physically remove the trailing glyf data and shift subsequent tables, producing a genuinely smaller font file. Also added unicode-showcase.yaml prompt for testing symbols (✔✘★❤☺☃▶☂) with custom fonts. * fix(pdf): custom font support across all elements, loca corruption, Unicode validation Bug fixes: - subsetTTF loca table corruption: removing loca update that shifted used glyph start offsets, causing characters to vanish (E, :, W, r) when preceded by unused glyphs in the font ordering New features: - font parameter on table(), kvTable(), calloutBox(), comparisonTable() (previously silently ignored, now properly wires through to TableStyle) - Unicode validation: standard fonts now throw a clear error when text contains characters outside WinAnsiEncoding, with a message pointing to registerCustomFont() as the fix Tests updated: emoji tests now expect the validation error instead of silent stripping. 2095 tests total, all pass. * fix(pdf): ToUnicode CMap compression — skip when deflate fails The CMap stream had /Filter /FlateDecode but contained raw ASCII text instead of zlib-compressed data. deflate() from ha:ziplib returns raw input for small strings, making wrapZlib produce invalid zlib (valid header + raw body). PDF viewers couldn't decompress the CMap, so text extraction returned garbled output (glyph IDs treated as char codes). Fix: same pattern as FontFile2 — only add /Filter when compression actually produces smaller output. Uncompressed CMap works correctly for text extraction (copy/paste, search, accessibility). * fix(pdf): calloutBox dark theme readability CalloutBox default bgColor 'EEF2FF' (light blue) was unreadable on dark themes — white text on light background. Now detects dark themes and switches to dark blue-grey background ('2A3040') with light text colors. Explicit bgColor/textColor still override. * feat(pdf): Phase 8 — visual regression tests with pixelmatch golden baselines 6 fixture generators covering: text rendering, table styles, two-column layout, callout box on dark theme, title page, signature line. Pipeline: generate deterministic PDF → pdftoppm → PNG → pixelmatch against golden baselines in tests/golden/pdf/. Threshold: 50 pixels at 0.1 colour distance tolerance. Justfile: just test-pdf-visual, just update-pdf-golden Deps: pixelmatch, pngjs (dev) 2101 tests total (37 files), all pass. * docs: Phase 10 quality loop CI design plan Config-driven improvement loop: run prompts → validate → deduplicate findings into GitHub Issues → assign top 3 to Copilot Workspace → fix → merge → loop. Generic across modules (PDF, future DOCX, etc.) via watch_paths in module config. * Fix ci and tests Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * dont install validation tools on mariner Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * fix: address PR review comments for PDF support - Add skip guards for PDF test dependencies (pdftoppm, qpdf, pdftotext, fonts-dejavu-core) on Windows and when tools are missing, with warnings on Linux - Fix Rust template literal scanner to scan ${...} interpolation expressions for property accesses while still skipping literal text - Add backward-compat re-exports from ha:ooxml-core for theme/validation symbols moved to ha:doc-core (prevents breaking existing handlers) - Remove Object.assign(core, docCore) test hacks (no longer needed) - Fix require() -> await import() in pdf-visual.test.ts (ESM compliance) - Add @module headers to doc-core, pdf, pdf-charts builtin modules - Fix xref-verify.test.ts: unique tmp path + cleanup + remove unused import - Install poppler-utils/qpdf/fonts-dejavu-core on KVM CI runners - Add @types/pngjs dev dependency Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * fix: skip PDF tool tests on Windows (process.platform check) hasCommand() now returns false immediately on win32 instead of trying to run 'which' (which doesn't exist on Windows). Also adds pdfinfo to the skip check in pdf-content.test.ts and warns on Linux when tools are missing. Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> --------- Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
1 parent 7af9373 commit e0f7193

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

65 files changed

+18088
-1227
lines changed

.github/instructions/skills.instructions.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ Detailed instructions for the LLM when this skill is active.
4040
| Skill | Purpose |
4141
|-------|---------|
4242
| `pptx-expert` | PowerPoint presentation building |
43+
| `pdf-expert` | PDF document building |
4344
| `research-synthesiser` | Research and synthesis |
4445
| `data-processor` | Data processing workflows |
4546
| `web-scraper` | Web scraping |

.github/workflows/pr-validate.yml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,11 @@ jobs:
7373
env:
7474
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
7575

76+
- name: Install PDF test dependencies
77+
run: |
78+
sudo apt-get update -qq
79+
sudo apt-get install -y -qq poppler-utils qpdf fonts-dejavu-core
80+
7681
- name: Setup
7782
run: just setup
7883

@@ -137,6 +142,12 @@ jobs:
137142
env:
138143
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
139144

145+
- name: Install PDF test dependencies
146+
if: matrix.hypervisor == 'kvm'
147+
run: |
148+
sudo apt-get update -qq
149+
sudo apt-get install -y -qq poppler-utils qpdf fonts-dejavu-core
150+
140151
- name: Setup
141152
run: just setup
142153

Justfile

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,14 @@ fmt-all: fmt fmt-analysis-guest fmt-runtime
256256
test-all: test test-analysis-guest
257257
@echo "✅ All tests passed"
258258

259+
# PDF visual regression tests
260+
test-pdf-visual:
261+
npx vitest run tests/pdf-visual.test.ts
262+
263+
# Update PDF golden baselines (run after intentional visual changes)
264+
update-pdf-golden:
265+
UPDATE_GOLDEN=1 npx vitest run tests/pdf-visual.test.ts
266+
259267
# ── OOXML Validation ─────────────────────────────────────────────────
260268

261269
# Validate a PPTX file against the OpenXML SDK schema.

builtin-modules/doc-core.json

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{
2+
"name": "doc-core",
3+
"description": "Format-agnostic document infrastructure — themes, colour validation, contrast utilities, input guards. Used by ha:pptx, ha:pdf, and other format modules.",
4+
"author": "system",
5+
"mutable": false,
6+
"sourceHash": "sha256:b9119a600839812d",
7+
"dtsHash": "sha256:1f311b99f56fdcbb",
8+
"importStyle": "named",
9+
"hints": {
10+
"overview": "Shared utilities for all document formats. Provides themes, colour validation (WCAG AA contrast), and input guards. You rarely import this directly — ha:pptx and ha:pdf re-use it internally.",
11+
"relatedModules": [
12+
"ha:ooxml-core",
13+
"ha:pdf",
14+
"ha:pdf-charts"
15+
],
16+
"criticalRules": [
17+
"All colours must be 6-char hex (no # prefix, no named colours, no rgb())",
18+
"Theme properties: bg, fg, accent1, accent2, accent3, accent4, subtle, titleFont, bodyFont, isDark — NOT 'accent', 'primary', 'secondary'",
19+
"Use requireHex() at API boundaries to validate colour inputs",
20+
"Use autoTextColor(bgHex) to pick readable text colour for any background",
21+
"Theme palette colours always pass contrast checks — use them when possible"
22+
],
23+
"antiPatterns": [
24+
"Don't use this module directly for document generation — use ha:pptx or ha:pdf",
25+
"Don't hardcode colour values — use theme colours via getTheme()"
26+
]
27+
}
28+
}

builtin-modules/ooxml-core.json

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,13 @@
33
"description": "Shared OOXML infrastructure - units, colors, themes, Content_Types, relationships",
44
"author": "system",
55
"mutable": false,
6-
"sourceHash": "sha256:b5f017fe2d4e2ed3",
7-
"dtsHash": "sha256:6aac85502082bf89",
6+
"sourceHash": "sha256:d7a3b33a14dda68f",
7+
"dtsHash": "sha256:649e1f850d1a391c",
88
"importStyle": "named",
99
"hints": {
1010
"overview": "Low-level OOXML infrastructure. Most users should use ha:pptx instead.",
1111
"relatedModules": [
12+
"ha:doc-core",
1213
"ha:pptx",
1314
"ha:xml-escape"
1415
],

builtin-modules/pdf-charts.json

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
{
2+
"name": "pdf-charts",
3+
"description": "Chart rendering for PDF documents — bar, line, pie, combo charts as PDF drawing operations.",
4+
"author": "system",
5+
"mutable": false,
6+
"sourceHash": "sha256:e86a394653c9c1a1",
7+
"dtsHash": "sha256:b61044c815cba0ea",
8+
"importStyle": "named",
9+
"hints": {
10+
"overview": "Render bar, line, pie, and combo charts in PDF documents. Charts are returned as PdfElement objects for use with addContent(). All chart values must be finite numbers.",
11+
"relatedModules": [
12+
"ha:pdf",
13+
"ha:doc-core"
14+
],
15+
"criticalRules": [
16+
"series.name is REQUIRED for all chart data series",
17+
"All values must be finite numbers — not null, undefined, NaN, or strings",
18+
"series.values.length must equal categories.length for bar/line/combo charts",
19+
"pieChart: labels.length must equal values.length",
20+
"Maximum 24 series per chart, 100 categories, 100 pie slices"
21+
],
22+
"antiPatterns": [
23+
"Don't pass empty categories or series arrays",
24+
"Don't mix up pieChart (labels + values) with barChart/lineChart (categories + series)"
25+
],
26+
"commonPatterns": [
27+
"import { barChart } from 'ha:pdf-charts'; addContent(doc, [barChart({ categories: ['Q1','Q2'], series: [{ name: 'Rev', values: [100,200] }] })]);"
28+
]
29+
}
30+
}

builtin-modules/pdf.json

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"name": "pdf",
3+
"description": "PDF 1.7 document generation — text, graphics, metadata, standard fonts. Flow-based layout for auto-paginating documents.",
4+
"author": "system",
5+
"mutable": false,
6+
"sourceHash": "sha256:202d5c76da3d341a",
7+
"dtsHash": "sha256:38f8a8a62174a4f7",
8+
"importStyle": "named",
9+
"hints": {
10+
"overview": "Generate PDF documents with text, shapes, and metadata. Uses PDF's 14 standard fonts (no embedding required). Coordinates are in points (72 points = 1 inch), with top-left origin.",
11+
"relatedModules": [
12+
"ha:doc-core",
13+
"ha:pdf-charts"
14+
],
15+
"requiredPlugins": [
16+
"fs-write"
17+
],
18+
"criticalRules": [
19+
"Call doc.addPage() before any drawing operations",
20+
"All coordinates are in POINTS (72 points = 1 inch) from TOP-LEFT corner",
21+
"Colours must be 6-char hex without # (e.g. '2196F3' not '#2196F3')",
22+
"Use exportToFile(doc, path, fsWrite) to save — requires host:fs-write plugin",
23+
"Use doc.buildPdf() to get raw bytes as Uint8Array",
24+
"TABLE_STYLES available: 'default', 'dark', 'minimal', 'corporate', 'emerald'",
25+
"Use columnAlign on table() to right-align numeric columns: columnAlign: ['left','center','right','right']",
26+
"kvTable values are RIGHT-ALIGNED by default — good for financial data"
27+
],
28+
"antiPatterns": [
29+
"Don't write raw PDF operators — use doc.drawText(), doc.drawRect(), etc.",
30+
"Don't assume bottom-left origin — the API uses top-left (like screens)",
31+
"Don't embed custom fonts (not yet supported) — use standard 14 fonts"
32+
],
33+
"commonPatterns": [
34+
"const doc = createDocument({ theme: 'light-clean', pageSize: 'a4' }); doc.addPage(); doc.drawText('Hello', 72, 72); exportToFile(doc, 'out.pdf', fsWrite);"
35+
]
36+
}
37+
}

builtin-modules/pptx-charts.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
"description": "OOXML DrawingML chart generation - bar, pie, line charts for PPTX presentations",
44
"author": "system",
55
"mutable": false,
6-
"sourceHash": "sha256:4174b6f03be2e0fb",
6+
"sourceHash": "sha256:11b86c34942c316e",
77
"dtsHash": "sha256:4353b8263dc99405",
88
"importStyle": "named",
99
"hints": {

builtin-modules/pptx-tables.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
"description": "Styled tables for PPTX presentations - headers, borders, alternating rows",
44
"author": "system",
55
"mutable": false,
6-
"sourceHash": "sha256:2d58934ed7df9fe1",
7-
"dtsHash": "sha256:3ba75bbc44353467",
6+
"sourceHash": "sha256:2fd1b0318ee87ab2",
7+
"dtsHash": "sha256:130d021921083af6",
88
"importStyle": "named",
99
"hints": {
1010
"overview": "Table generation for PPTX. Always used with ha:pptx.",

builtin-modules/pptx.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
"description": "PowerPoint PPTX presentation builder - slides, text, shapes, themes, layouts",
44
"author": "system",
55
"mutable": false,
6-
"sourceHash": "sha256:23569540a0f8622f",
7-
"dtsHash": "sha256:27520514e4401465",
6+
"sourceHash": "sha256:0139928e7286a4f7",
7+
"dtsHash": "sha256:0e08c6d6b51b36e8",
88
"importStyle": "named",
99
"hints": {
1010
"overview": "Core PPTX slide building. Charts in ha:pptx-charts, tables in ha:pptx-tables.",

0 commit comments

Comments
 (0)