Skip to content

Commit 36ad2b8

Browse files
authored
feat(testing): visual-regression public docs + dogfood + APPROVE_PROPERTY (Track N N2-N4, @SInCE 1.6.9) (#127)
Completes the public visual-regression surface introduced in #126: - expose PdfVisualRegression.APPROVE_PROPERTY (@SInCE 1.6.9) so consumers toggle approve mode without hard-coding the system-property string (mirrors LayoutSnapshotAssertions.UPDATE_PROPERTY) - add docs/operations/visual-regression-testing.md (pixel vs semantic, API, approve mode, cross-platform tolerance calibration) - README 'Which API should I use?' gains a pixel-level visual-regression row - add PublicVisualApiDogfoodTest: composes a document and pixel-tests it end-to-end through the public testing.visual surface Full suite 1060/1060 green.
1 parent 1de0cff commit 36ad2b8

5 files changed

Lines changed: 252 additions & 4 deletions

File tree

CHANGELOG.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,17 @@ Housekeeping cycle plus the public pixel-level visual-regression API (Track N).
1717
consumers can now run the same render-PDF → diff-PNG baseline gate against
1818
their own presets and templates instead of copying the harness. Behaviour is
1919
unchanged; the PDF→image step is inlined on PDFBox's `PDFRenderer`.
20+
- Exposed `PdfVisualRegression.APPROVE_PROPERTY` (`@since 1.6.9`) — the
21+
`graphcompose.visual.approve` system-property name — so consumers can toggle
22+
baseline-approve mode without hard-coding the string (mirrors
23+
`LayoutSnapshotAssertions.UPDATE_PROPERTY`).
24+
25+
### Documentation
26+
27+
- Added [`docs/operations/visual-regression-testing.md`](docs/operations/visual-regression-testing.md):
28+
pixel-vs-semantic guidance, the `PdfVisualRegression` API, approve mode,
29+
baseline layout, and cross-platform tolerance calibration.
30+
- README "Which API should I use?" gains a pixel-level visual-regression row.
2031

2132
## v1.6.8 — 2026-06-01
2233

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ Sits between **iText** (low-level page primitives) and **JasperReports** (XML-te
5959

6060
- **Server-side PDF generation in Java** — invoices, CVs, reports, proposals, statements, schedules.
6161
- **Templated documents from data** — themed presets (`ModernProfessional`, `InvoiceTemplateV2`, …) you parameterise instead of re-styling every time.
62-
- **Regression-tested layouts** — `DocumentSession#layoutSnapshot()` makes layout changes visible in PRs before any byte ships.
62+
- **Regression-tested layouts** — `DocumentSession#layoutSnapshot()` makes layout changes visible in PRs before any byte ships; `PdfVisualRegression` adds a pixel-level gate for font and colour fidelity.
6363
- **Streaming PDFs from web backends** — Spring Boot `@RestController` writing straight to the response ([`HttpStreamingExample`](./examples/src/main/java/com/demcha/examples/features/streaming/HttpStreamingExample.java)).
6464
- **Higher-level than PDFBox, lighter than JasperReports** — Java DSL describes semantics; no XML templates, no manual coordinates.
6565

@@ -90,6 +90,7 @@ GraphCompose uses PDFBox under the hood as the rendering backend — the com
9090
| Generate a CV / cover letter from data | Layered templates | `ModernProfessional.create().compose(session, cvDocument)` — see [layered templates](./docs/templates/v2-layered/README.md) |
9191
| Add a custom visual primitive | Engine extension | `NodeDefinition` + `PdfFragmentRenderHandler` — see [extension guide](./docs/contributing/extension-guide.md) |
9292
| Regression-test generated layouts | Layout snapshots | `DocumentSession#layoutSnapshot()` — quickstart at [Testing your document](./docs/operations/test-your-document.md); full reference at [snapshot testing](./docs/operations/layout-snapshot-testing.md) |
93+
| Pixel-test the rendered PDF (fonts, colours, anti-aliasing) | Visual regression | `PdfVisualRegression.standard()…assertMatchesBaseline(...)` — see [visual regression testing](./docs/operations/visual-regression-testing.md) |
9394
| See the live playground / gallery | Next.js showcase site | [Showcase](https://DemchaAV.github.io/GraphCompose/) — source under [`site/`](./site), built with `next build` and deployed via the [Pages workflow](./.github/workflows/deploy-site.yml) |
9495

9596
## Installation
Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# Visual Regression Testing
2+
3+
Pixel-level visual regression is the outermost geometry-fidelity layer in GraphCompose. It renders a PDF to one PNG per page and diffs each page against a committed baseline.
4+
5+
It complements — does not replace — [layout snapshot testing](./layout-snapshot-testing.md):
6+
7+
1. unit tests validate isolated layout math
8+
2. **layout snapshots** validate the resolved document tree (coordinates, page spans, layer/order) — structural geometry
9+
3. **visual regression** validates the rendered pixels — font shape, colour, anti-aliasing, glyph fallback
10+
4. human inspection of the PDF remains the final eye
11+
12+
Reach for visual regression when the failure you care about is *pixel-level* rather than *geometry-level*: the layout snapshot still matches but the PDF looks wrong (wrong font, wrong colour, missing glyph, anti-aliasing drift).
13+
14+
## Pixel vs semantic — which layer?
15+
16+
| You want to catch… | Use |
17+
|---|---|
18+
| A node moved / page break shifted / sibling order changed | layout snapshot (semantic) |
19+
| The PDF looks identical pixel-for-pixel — fonts, colours, glyphs | **visual regression (pixel)** |
20+
| A specific layout-math rule | a focused unit test |
21+
22+
The semantic layer is cheap, deterministic, and cross-platform stable. The pixel layer is precise but sensitive to platform font rendering (see [Cross-platform tolerance](#cross-platform-tolerance) below). A flagship template or a preset you publish to others deserves both.
23+
24+
## Public API
25+
26+
The harness is `com.demcha.compose.testing.visual.PdfVisualRegression` (`@since 1.6.9`), a sibling to the semantic `com.demcha.compose.testing.layout.*` helpers. It ships in the main artifact, so library consumers use the exact helpers GraphCompose uses in its own tests.
27+
28+
### Assert a committed baseline
29+
30+
```java
31+
import com.demcha.compose.GraphCompose;
32+
import com.demcha.compose.document.api.DocumentPageSize;
33+
import com.demcha.compose.document.api.DocumentSession;
34+
import com.demcha.compose.testing.visual.PdfVisualRegression;
35+
import org.junit.jupiter.api.Test;
36+
37+
class InvoiceVisualParityTest {
38+
39+
@Test
40+
void invoiceRendersPixelIdentical() throws Exception {
41+
byte[] pdfBytes;
42+
try (DocumentSession document = GraphCompose.document()
43+
.pageSize(DocumentPageSize.A4)
44+
.margin(22, 22, 22, 22)
45+
.create()) {
46+
template.compose(document, spec);
47+
pdfBytes = document.toPdfBytes();
48+
}
49+
50+
PdfVisualRegression.standard()
51+
.assertMatchesBaseline("invoice_standard", pdfBytes);
52+
}
53+
}
54+
```
55+
56+
`assertMatchesBaseline(name, pdfBytes)` renders every page, compares against `<name>-page-N.png` under the baseline root, and throws `AssertionError` if any page exceeds the configured budget. On failure it writes `<name>-page-N.actual.png` and `<name>-page-N.diff.png` next to the baseline for inspection.
57+
58+
### Configure the harness
59+
60+
`PdfVisualRegression` is immutable; every setter returns a copy.
61+
62+
| Setter | Default | Meaning |
63+
|---|---|---|
64+
| `baselineRoot(Path)` | `src/test/resources/visual-baselines` | where baselines and diff sidecars live |
65+
| `renderScale(float)` | `1.0` | render scale multiplier (`2.0` = retina); must be `> 0` |
66+
| `perPixelTolerance(int)` | `6` | allowed per-channel delta (`0..255`) before a pixel counts as mismatched |
67+
| `mismatchedPixelBudget(long)` | `0` | mismatched pixels tolerated per page before the assertion fails |
68+
69+
### Diff images directly
70+
71+
For ad-hoc comparison, render pages and call `ImageDiff` yourself:
72+
73+
```java
74+
List<BufferedImage> pages = PdfVisualRegression.standard().renderPages(pdfBytes);
75+
ImageDiff.Result diff = ImageDiff.compare(expectedPng, pages.get(0), 6);
76+
assertThat(diff.withinBudget(0)).isTrue();
77+
```
78+
79+
## Approve mode (blessing baselines)
80+
81+
There is no baseline the first time. Run with the approve flag to write the current renders as the baseline:
82+
83+
```bash
84+
./mvnw test -Dtest=InvoiceVisualParityTest -Dgraphcompose.visual.approve=true
85+
```
86+
87+
The system-property name is exposed as `PdfVisualRegression.APPROVE_PROPERTY`; the environment variable `GRAPHCOMPOSE_VISUAL_APPROVE=true` works as a fallback. In approve mode the harness writes baselines and skips the diff assertion — so **never enable it in CI verification**, only when you have reviewed the new render and intend to re-bless.
88+
89+
## Where files live
90+
91+
- committed baselines: `<baselineRoot>/<name>-page-N.png`
92+
- mismatch artifacts (normal runs): `<baselineRoot>/<name>-page-N.actual.png` and `<name>-page-N.diff.png` (mismatched pixels red, matching pixels greyscale)
93+
94+
Use a flat `name`, or pre-create nested baseline directories — the harness creates the baseline root but not intermediate folders.
95+
96+
## Cross-platform tolerance
97+
98+
PDFBox font rasterization drifts slightly across platforms (different system fonts, different rasterizer). A baseline recorded on Windows will not match Linux CI pixel-for-pixel.
99+
100+
The `standard()` defaults are strict (tolerance `6`, budget `0`) — good for same-platform, deterministic renders. For baselines that must survive a Windows-author → Linux-CI round trip, loosen both. GraphCompose's own CV / cover-letter parity tests calibrate to:
101+
102+
```java
103+
PdfVisualRegression.standard()
104+
.perPixelTolerance(8) // absorb sub-pixel anti-aliasing drift
105+
.mismatchedPixelBudget(50_000) // ~glyph edges across a full A4 page
106+
.assertMatchesBaseline(slug, pdfBytes);
107+
```
108+
109+
Tune these to your fonts and page density: too tight and CI flakes on anti-aliasing noise; too loose and a real regression slips through. Start from the values above and tighten until CI is stable.
110+
111+
## Using visual regression in downstream projects
112+
113+
Library consumers use the same published helpers:
114+
115+
```java
116+
import com.demcha.compose.testing.visual.PdfVisualRegression;
117+
118+
PdfVisualRegression.standard()
119+
.baselineRoot(Path.of("src", "test", "resources", "pdf-baselines"))
120+
.perPixelTolerance(8)
121+
.mismatchedPixelBudget(50_000)
122+
.assertMatchesBaseline("reports/monthly_invoice", pdfBytes);
123+
```
124+
125+
`PublicVisualApiDogfoodTest` in this repository drives exactly this consumer workflow end-to-end and proves the published surface is sufficient without any package-private access.
126+
127+
## When not to use pixel regression
128+
129+
- when structural geometry is what you care about → use a [layout snapshot](./layout-snapshot-testing.md) (cheaper, cross-platform stable)
130+
- when a small unit test proves the same rule more directly
131+
- as the *only* gate on a CI that runs on a different OS than where baselines were recorded — pair it with semantic snapshots and a sensible tolerance budget
132+
133+
## Examples in this repository
134+
135+
- `CvV2VisualParityTest`, `CoverLetterV2VisualParityTest` — preset parity with Windows-baseline / Linux-CI calibration
136+
- `ShapeContainerVisualRegressionTest` — engine primitive fidelity
137+
- `TableRowSpanDemoTest` — table rendering
138+
- `PdfVisualRegressionTest` — the harness's own unit tests
139+
- `PublicVisualApiDogfoodTest` — consumer-surface dogfood

src/main/java/com/demcha/compose/testing/visual/PdfVisualRegression.java

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,16 @@
4040
*/
4141
public final class PdfVisualRegression {
4242

43-
private static final String APPROVE_SYS_PROP = "graphcompose.visual.approve";
43+
/**
44+
* System property that switches the harness into approve mode
45+
* ({@code -Dgraphcompose.visual.approve=true}): instead of asserting, it
46+
* (re)writes the current renders to the baseline location and skips the
47+
* diff. The environment variable {@code GRAPHCOMPOSE_VISUAL_APPROVE=true}
48+
* works as a fallback.
49+
*
50+
* @since 1.6.9
51+
*/
52+
public static final String APPROVE_PROPERTY = "graphcompose.visual.approve";
4453
private static final String APPROVE_ENV_VAR = "GRAPHCOMPOSE_VISUAL_APPROVE";
4554

4655
private final Path baselineRoot;
@@ -155,7 +164,7 @@ public void assertMatchesBaseline(String baselineName, byte[] pdfBytes) throws I
155164
Path actualOut = sidecarPath(baselineName, page, "actual");
156165
ImageIO.write(rendered.get(page), "png", actualOut.toFile());
157166
failures.add("Missing baseline " + baseline + " — wrote rendered output to " + actualOut
158-
+ ". Re-run with -D" + APPROVE_SYS_PROP + "=true to approve.");
167+
+ ". Re-run with -D" + APPROVE_PROPERTY + "=true to approve.");
159168
continue;
160169
}
161170
BufferedImage expected = ImageIO.read(baseline.toFile());
@@ -206,7 +215,7 @@ private Path sidecarPath(String baselineName, int pageIndex, String suffix) {
206215
}
207216

208217
private static boolean approveMode() {
209-
String prop = System.getProperty(APPROVE_SYS_PROP);
218+
String prop = System.getProperty(APPROVE_PROPERTY);
210219
if (prop != null) {
211220
return Boolean.parseBoolean(prop);
212221
}
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
package com.demcha.testing.visual;
2+
3+
import com.demcha.compose.GraphCompose;
4+
import com.demcha.compose.document.api.DocumentPageSize;
5+
import com.demcha.compose.document.api.DocumentSession;
6+
import com.demcha.compose.document.style.DocumentTextStyle;
7+
import com.demcha.compose.testing.visual.PdfVisualRegression;
8+
import org.junit.jupiter.api.Test;
9+
import org.junit.jupiter.api.io.TempDir;
10+
11+
import java.nio.file.Path;
12+
13+
import static org.assertj.core.api.Assertions.assertThat;
14+
15+
/**
16+
* Drives a consumer-style "compose a document, then pixel-test the rendered
17+
* PDF" workflow entirely through the public {@code
18+
* com.demcha.compose.testing.visual} surface — no package-private access.
19+
*
20+
* <p>Sibling to {@code LayoutSnapshotPublicApiDogfoodTest} on the semantic
21+
* layer; it proves the published harness is sufficient for downstream adoption
22+
* and guards against accidental private-to-public regressions in CI.</p>
23+
*/
24+
class PublicVisualApiDogfoodTest {
25+
26+
@TempDir
27+
Path tempDir;
28+
29+
@Test
30+
void shouldSupportConsumerPixelRegressionThroughPublicApi() throws Exception {
31+
Path baselineRoot = tempDir.resolve("pdf-baselines");
32+
String baselineName = "consumer_simple_document";
33+
byte[] pdfBytes = renderConsumerDocument();
34+
35+
PdfVisualRegression harness = PdfVisualRegression.standard().baselineRoot(baselineRoot);
36+
37+
// 1. Consumer blesses a fresh baseline via the public approve flag.
38+
String previous = System.getProperty(PdfVisualRegression.APPROVE_PROPERTY);
39+
try {
40+
System.setProperty(PdfVisualRegression.APPROVE_PROPERTY, "true");
41+
harness.assertMatchesBaseline(baselineName, pdfBytes);
42+
} finally {
43+
restoreProperty(PdfVisualRegression.APPROVE_PROPERTY, previous);
44+
}
45+
46+
assertThat(baselineRoot.resolve(baselineName + "-page-0.png"))
47+
.as("approve mode should write a page-0 baseline")
48+
.exists()
49+
.isRegularFile();
50+
51+
// 2. A second render of the same document matches the just-written baseline.
52+
harness.assertMatchesBaseline(baselineName, pdfBytes);
53+
54+
// 3. A matching run must not leave a mismatch sidecar.
55+
assertThat(baselineRoot.resolve(baselineName + "-page-0.actual.png"))
56+
.as("a matching run must not leave a mismatch sidecar")
57+
.doesNotExist();
58+
}
59+
60+
private byte[] renderConsumerDocument() throws Exception {
61+
try (DocumentSession document = GraphCompose.document()
62+
.pageSize(DocumentPageSize.A4)
63+
.margin(18, 18, 18, 18)
64+
.create()) {
65+
document.dsl()
66+
.pageFlow()
67+
.name("ConsumerRoot")
68+
.spacing(8)
69+
.addParagraph(paragraph -> paragraph
70+
.name("Title")
71+
.text("Consumer pixel baseline")
72+
.textStyle(DocumentTextStyle.DEFAULT))
73+
.addShape(shape -> shape
74+
.name("AccentBox")
75+
.size(120, 32))
76+
.build();
77+
return document.toPdfBytes();
78+
}
79+
}
80+
81+
private void restoreProperty(String key, String previous) {
82+
if (previous == null) {
83+
System.clearProperty(key);
84+
} else {
85+
System.setProperty(key, previous);
86+
}
87+
}
88+
}

0 commit comments

Comments
 (0)