Skip to content

Commit fbc23cf

Browse files
DemchaAVDemchaAV
authored
chore(benchmarks): re-measure the committed datasets and the deck that renders them (#466)
* chore(benchmarks): re-measure the committed datasets and the deck that renders them Both committed result files predated the 2.x line. The comparative snapshot the flagship deck renders had one commit in its entire history, taken on the 1.7.1 codebase, and the perf baseline was captured a week before that — while the deck PDFs built from them were re-committed at every release, so freshly dated assets carried figures from a codebase two minor versions behind. Re-measured on one quiet machine: five current-speed runs aggregated to a median for the baseline, and one comparative run for the deck. The ratios moved, and unlike absolute timings they compare across machines because both libraries are measured in the same run. At 1000 rows the advantage over iText 9 reads 4.54x on time and 3.56x on allocation, where the published snapshot said 5.38x and 4.19x; part of that is iText moving 9.6.0 -> 9.7.0 after the snapshot was taken. Against JasperReports, whose version is unchanged, the gap at 1000 rows is now 1.1%. The deck's own footnotes are corrected while they are being regenerated: one still asked for a refresh "on the final 2.0 commit" two releases after 2.0 shipped, and the summary line still spoke for 2.0 alone. Both now say what the data is — one machine, one run, ratios that travel and timings that do not. Layout snapshot re-recorded for the reworded footnotes; the figures alone did not move it, because every value kept its digit count and the auto-width columns kept their widths. * docs(examples): correct the deck's class javadoc alongside its slides The class javadoc carried the same stale instruction the slides did - refresh data and provenance "on the release commit" for a 2.0 asset, two releases after 2.0 shipped - and it also described the scaling iteration metadata as missing from a "historical file" rather than as covering only one tier, which is what the format actually does. It now says what the slides say: read the ratios rather than the timings, because both libraries are measured in the same run and their proportions survive a change of machine while the milliseconds do not. * fix(examples): the deck called the PPTX backend a manifest skeleton render-pptx ships two backends: PptxSemanticBackend, the slide-safe skeleton, and PptxFixedLayoutBackend, which implements FixedLayoutRenderer and is discovered through FixedLayoutBackendProvider exactly as the PDF backend is. The module table listed render-pptx as "manifest only" and the architecture proof card said PPTX was a skeleton, contradicting the deck's own first page, which already labels render-pptx "fixed layout", and the capability matrix, which gives PPTX (fixed) a column beside PDF. The deck is republished by this change, so it was about to carry that understatement of the engine's own capability into the release assets again.
1 parent e7b357c commit fbc23cf

8 files changed

Lines changed: 143 additions & 106 deletions

File tree

‎assets/readme/examples/engine-deck-v2.pdf‎

68 Bytes
Binary file not shown.

‎assets/readme/examples/engine-deck.pdf‎

80 Bytes
Binary file not shown.

‎baselines/current-speed-full.json‎

Lines changed: 87 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -1,98 +1,133 @@
11
{
2-
"timestamp" : "2026-06-09 17:19:39",
2+
"timestamp" : "2026-07-26 22:59:52",
33
"profile" : "full",
44
"warmupIterations" : 12,
55
"measurementIterations" : 40,
66
"docsPerThread" : 12,
7-
"threadCounts" : [ 1, 2, 4, 8 ],
7+
"threadCounts" : [ 1, 2, 4, 8, 16 ],
88
"latency" : [ {
99
"scenario" : "cv-template",
1010
"description" : "Compose-first CV template",
11-
"avgMillis" : 2.45,
12-
"p50Millis" : 2.24,
13-
"p95Millis" : 3.53,
14-
"maxMillis" : 4.27,
15-
"docsPerSecond" : 408.15,
16-
"avgKilobytes" : 2.22,
17-
"peakHeapMb" : 20.0
11+
"avgMillis" : 4.81,
12+
"p50Millis" : 4.52,
13+
"p95Millis" : 7.09,
14+
"maxMillis" : 7.56,
15+
"docsPerSecond" : 208.05,
16+
"avgKilobytes" : 2.46,
17+
"peakHeapMb" : 27.48
1818
}, {
1919
"scenario" : "engine-simple",
2020
"description" : "One-page engine composition",
21-
"avgMillis" : 2.0,
22-
"p50Millis" : 1.8,
23-
"p95Millis" : 2.88,
24-
"maxMillis" : 3.97,
25-
"docsPerSecond" : 498.91,
21+
"avgMillis" : 3.36,
22+
"p50Millis" : 3.08,
23+
"p95Millis" : 5.12,
24+
"maxMillis" : 6.84,
25+
"docsPerSecond" : 297.83,
2626
"avgKilobytes" : 1.08,
27-
"peakHeapMb" : 8.0
27+
"peakHeapMb" : 10.0
2828
}, {
2929
"scenario" : "feature-rich",
3030
"description" : "QR, barcode, watermark, header/footer, page break",
31-
"avgMillis" : 31.99,
32-
"p50Millis" : 31.32,
33-
"p95Millis" : 36.35,
34-
"maxMillis" : 40.65,
35-
"docsPerSecond" : 31.26,
31+
"avgMillis" : 54.16,
32+
"p50Millis" : 52.0,
33+
"p95Millis" : 62.22,
34+
"maxMillis" : 83.58,
35+
"docsPerSecond" : 18.47,
3636
"avgKilobytes" : 6.33,
37-
"peakHeapMb" : 58.89
37+
"peakHeapMb" : 171.2
3838
}, {
3939
"scenario" : "invoice-template",
4040
"description" : "Compose-first invoice template",
41-
"avgMillis" : 13.12,
42-
"p50Millis" : 12.88,
43-
"p95Millis" : 17.01,
44-
"maxMillis" : 19.6,
45-
"docsPerSecond" : 76.22,
46-
"avgKilobytes" : 9.72,
47-
"peakHeapMb" : 45.11
41+
"avgMillis" : 6.58,
42+
"p50Millis" : 6.41,
43+
"p95Millis" : 9.82,
44+
"maxMillis" : 10.97,
45+
"docsPerSecond" : 152.08,
46+
"avgKilobytes" : 3.03,
47+
"peakHeapMb" : 19.42
4848
}, {
4949
"scenario" : "long-token",
5050
"description" : "Long unbreakable tokens (URLs/IDs) forcing character-level wrap",
51-
"avgMillis" : 3.38,
52-
"p50Millis" : 3.15,
53-
"p95Millis" : 4.72,
54-
"maxMillis" : 5.51,
55-
"docsPerSecond" : 295.43,
56-
"avgKilobytes" : 3.97,
57-
"peakHeapMb" : 52.0
51+
"avgMillis" : 5.6,
52+
"p50Millis" : 5.28,
53+
"p95Millis" : 7.6,
54+
"maxMillis" : 8.37,
55+
"docsPerSecond" : 178.56,
56+
"avgKilobytes" : 4.08,
57+
"peakHeapMb" : 56.0
5858
}, {
5959
"scenario" : "proposal-template",
6060
"description" : "Long multi-page proposal template",
61-
"avgMillis" : 9.63,
62-
"p50Millis" : 9.24,
63-
"p95Millis" : 12.44,
64-
"maxMillis" : 13.24,
65-
"docsPerSecond" : 103.84,
66-
"avgKilobytes" : 7.68,
67-
"peakHeapMb" : 51.99
61+
"avgMillis" : 6.53,
62+
"p50Millis" : 6.21,
63+
"p95Millis" : 7.92,
64+
"maxMillis" : 10.34,
65+
"docsPerSecond" : 153.19,
66+
"avgKilobytes" : 5.27,
67+
"peakHeapMb" : 31.25
68+
}, {
69+
"scenario" : "vector-rich",
70+
"description" : "v1.8 vector surface: bar + pie charts, SVG icons, gradient path",
71+
"avgMillis" : 5.46,
72+
"p50Millis" : 5.15,
73+
"p95Millis" : 7.89,
74+
"maxMillis" : 9.21,
75+
"docsPerSecond" : 183.23,
76+
"avgKilobytes" : 7.27,
77+
"peakHeapMb" : 33.43
78+
} ],
79+
"stages" : [ {
80+
"scenario" : "cv-template",
81+
"composeMillis" : 0.45,
82+
"layoutMillis" : 1.02,
83+
"renderMillis" : 1.27,
84+
"totalMillis" : 2.8
85+
}, {
86+
"scenario" : "invoice-template",
87+
"composeMillis" : 0.27,
88+
"layoutMillis" : 1.02,
89+
"renderMillis" : 1.55,
90+
"totalMillis" : 2.9
91+
}, {
92+
"scenario" : "proposal-template",
93+
"composeMillis" : 0.34,
94+
"layoutMillis" : 2.5,
95+
"renderMillis" : 2.47,
96+
"totalMillis" : 5.3
6897
} ],
6998
"throughput" : [ {
7099
"scenario" : "invoice-template",
71100
"threads" : 1,
72101
"totalDocs" : 12,
73-
"docsPerSecond" : 121.21,
74-
"avgMillisPerDoc" : 8.25
102+
"docsPerSecond" : 283.9,
103+
"avgMillisPerDoc" : 3.52
104+
}, {
105+
"scenario" : "invoice-template",
106+
"threads" : 16,
107+
"totalDocs" : 192,
108+
"docsPerSecond" : 1693.47,
109+
"avgMillisPerDoc" : 0.59
75110
}, {
76111
"scenario" : "invoice-template",
77112
"threads" : 2,
78113
"totalDocs" : 24,
79-
"docsPerSecond" : 223.67,
80-
"avgMillisPerDoc" : 4.47
114+
"docsPerSecond" : 630.89,
115+
"avgMillisPerDoc" : 1.59
81116
}, {
82117
"scenario" : "invoice-template",
83118
"threads" : 4,
84119
"totalDocs" : 48,
85-
"docsPerSecond" : 335.65,
86-
"avgMillisPerDoc" : 2.98
120+
"docsPerSecond" : 1148.54,
121+
"avgMillisPerDoc" : 0.87
87122
}, {
88123
"scenario" : "invoice-template",
89124
"threads" : 8,
90125
"totalDocs" : 96,
91-
"docsPerSecond" : 414.73,
92-
"avgMillisPerDoc" : 2.41
126+
"docsPerSecond" : 1596.51,
127+
"avgMillisPerDoc" : 0.63
93128
} ],
94-
"totalBytes" : 3062560,
129+
"totalBytes" : 2361736,
95130
"aggregation" : "median",
96131
"sourceCount" : 5,
97-
"sourceRuns" : [ "C:\\Dev\\Java\\GraphCompose\\target\\benchmarks\\current-speed\\run-20260609-171909.json", "C:\\Dev\\Java\\GraphCompose\\target\\benchmarks\\current-speed\\run-20260609-171916.json", "C:\\Dev\\Java\\GraphCompose\\target\\benchmarks\\current-speed\\run-20260609-171924.json", "C:\\Dev\\Java\\GraphCompose\\target\\benchmarks\\current-speed\\run-20260609-171931.json", "C:\\Dev\\Java\\GraphCompose\\target\\benchmarks\\current-speed\\run-20260609-171938.json" ]
132+
"sourceRuns" : [ "target\\benchmarks\\current-speed\\run-20260726-225913.json", "target\\benchmarks\\current-speed\\run-20260726-225923.json", "target\\benchmarks\\current-speed\\run-20260726-225932.json", "target\\benchmarks\\current-speed\\run-20260726-225942.json", "target\\benchmarks\\current-speed\\run-20260726-225950.json" ]
98133
}

‎examples/src/main/java/com/demcha/examples/flagships/EngineDeckV2Example.java‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -66,9 +66,11 @@
6666
* <p>Benchmark wording is intentionally conservative. The bundled JSON is a
6767
* dated single-machine reference run without hardware or commit provenance;
6868
* allocation values are average thread-allocated MiB per render, not peak
69-
* heap, and scaling iteration metadata is absent from the historical file.
70-
* A final 2.0 release asset should refresh both data and provenance on the
71-
* release commit.</p>
69+
* heap, and scaling iteration metadata covers only the small-invoice tier.
70+
* Read the ratios rather than the timings: both libraries are measured in the
71+
* same run, so their proportions survive a change of machine while the
72+
* milliseconds do not. Refresh the data file and these figures together — see
73+
* {@code docs/operations/benchmarks.md}.</p>
7274
*
7375
* @author Artem Demchyshyn
7476
* @since 2.0.0
@@ -488,7 +490,7 @@ private static void architecturePage(SectionBuilder page) {
488490
PALE_VIOLET, VIOLET);
489491
proofCard(row, "FIXED BACKEND SPI", "FixedLayoutBackendProvider and FontMetricsProvider are discovered by ServiceLoader.",
490492
PALE_BLUE, BLUE);
491-
proofCard(row, "SEMANTIC EXPORT", "DOCX is a working POI exporter; PPTX is a slide-safe manifest skeleton in 2.0.",
493+
proofCard(row, "SEMANTIC EXPORT", "DOCX is a working POI exporter; PPTX carries a slide-safe skeleton beside its fixed backend.",
492494
PALE_MINT, MINT);
493495
});
494496
page.addParagraph(p -> pageNote(p, "The renderer boundary is explicit; Android or another backend is an extension opportunity, not a shipped 2.0 claim."));
@@ -638,7 +640,7 @@ private static DocumentNode moduleMatrix() {
638640
{"core", "authoring + model"},
639641
{"render-pdf", "fixed PDF"},
640642
{"render-docx", "semantic DOCX"},
641-
{"render-pptx", "manifest only"},
643+
{"render-pptx", "fixed PPTX"},
642644
{"templates", "opt-in presets"},
643645
{"testing", "snapshot tools"},
644646
{"graph-compose", "PDF wrapper"},
@@ -702,9 +704,9 @@ private static void referencePage(SectionBuilder page, EngineDeckData.BenchRun b
702704
});
703705

704706
page.addParagraph(p -> p
705-
.text("Scope: one developer machine; CPU, OS, JDK and commit were not stored in this historical file. "
706-
+ "The JSON stores 50/100 only for its small-invoice tier and omits scaling iteration metadata. "
707-
+ "Refresh both data and provenance on the final 2.0 commit before release claims.")
707+
.text("Scope: one developer machine, single run. The file records neither CPU, OS and JDK nor the "
708+
+ "commit measured, and its iteration counts cover only the small-invoice tier. Absolute "
709+
+ "milliseconds therefore travel badly between machines - read the ratios, not the timings.")
708710
.textStyle(body(7.9, MUTED))
709711
.lineSpacing(1.25)
710712
.margin(DocumentInsets.top(1)));
@@ -744,9 +746,9 @@ private static void scalingPage(SectionBuilder page, EngineDeckData.BenchRun ben
744746
});
745747
page.addParagraph(p -> p
746748
.rich(r -> r
747-
.plain("What 2.0 can claim today: ")
749+
.plain("What the 2.x line can claim today: ")
748750
.bold("modular packaging, renderer isolation and deterministic layout snapshots")
749-
.plain(". Refresh performance evidence separately on the release commit."))
751+
.plain(". The figures above are one machine's reference run, refreshed with the data file."))
750752
.textStyle(body(8.4, BODY))
751753
.lineSpacing(1.3)
752754
.margin(DocumentInsets.zero()));

‎examples/src/main/resources/benchmarks/comparative.json‎

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,54 @@
11
{
2-
"timestamp" : "2026-06-15 19:41:29",
2+
"timestamp" : "2026-07-26 23:00:56",
33
"warmupIterations" : 50,
44
"measurementIterations" : 100,
55
"libraries" : [ {
66
"library" : "GraphCompose Canonical",
7-
"avgTimeMs" : 2.34,
8-
"avgHeapMb" : 0.16
7+
"avgTimeMs" : 2.84,
8+
"avgHeapMb" : 0.17
99
}, {
1010
"library" : "iText 9",
11-
"avgTimeMs" : 2.48,
11+
"avgTimeMs" : 2.75,
1212
"avgHeapMb" : 0.16
1313
}, {
1414
"library" : "JasperReports",
15-
"avgTimeMs" : 4.6,
16-
"avgHeapMb" : 0.15
15+
"avgTimeMs" : 5.7,
16+
"avgHeapMb" : 0.16
1717
}, {
1818
"library" : "GraphCompose (40 rows)",
19-
"avgTimeMs" : 4.52,
20-
"avgHeapMb" : 0.84
19+
"avgTimeMs" : 5.7,
20+
"avgHeapMb" : 0.99
2121
}, {
2222
"library" : "iText 9 (40 rows)",
23-
"avgTimeMs" : 12.87,
24-
"avgHeapMb" : 2.96
23+
"avgTimeMs" : 14.54,
24+
"avgHeapMb" : 3.05
2525
}, {
2626
"library" : "JasperReports (40 rows)",
27-
"avgTimeMs" : 9.13,
28-
"avgHeapMb" : 2.52
27+
"avgTimeMs" : 9.89,
28+
"avgHeapMb" : 2.53
2929
}, {
3030
"library" : "GraphCompose (200 rows)",
31-
"avgTimeMs" : 11.75,
32-
"avgHeapMb" : 3.22
31+
"avgTimeMs" : 15.26,
32+
"avgHeapMb" : 3.93
3333
}, {
3434
"library" : "iText 9 (200 rows)",
35-
"avgTimeMs" : 50.34,
36-
"avgHeapMb" : 13.37
35+
"avgTimeMs" : 57.57,
36+
"avgHeapMb" : 13.81
3737
}, {
3838
"library" : "JasperReports (200 rows)",
39-
"avgTimeMs" : 16.45,
40-
"avgHeapMb" : 10.7
39+
"avgTimeMs" : 21.13,
40+
"avgHeapMb" : 10.81
4141
}, {
4242
"library" : "GraphCompose (1000 rows)",
43-
"avgTimeMs" : 45.27,
44-
"avgHeapMb" : 15.73
43+
"avgTimeMs" : 64.08,
44+
"avgHeapMb" : 19.16
4545
}, {
4646
"library" : "iText 9 (1000 rows)",
47-
"avgTimeMs" : 243.66,
48-
"avgHeapMb" : 65.95
47+
"avgTimeMs" : 291.03,
48+
"avgHeapMb" : 68.16
4949
}, {
5050
"library" : "JasperReports (1000 rows)",
51-
"avgTimeMs" : 48.47,
52-
"avgHeapMb" : 51.95
51+
"avgTimeMs" : 64.77,
52+
"avgHeapMb" : 52.53
5353
} ]
5454
}

0 commit comments

Comments
 (0)