fix(benchmarks): rebuild every branch-versioned module before an A/B run (#465)

* fix(benchmarks): rebuild every branch-versioned module before an A/B run

The A/B scripts installed only the engine, but the benchmark module resolves
render-pdf and templates — plus the templates tests-classifier jar, which is
never published — at the branch's own version. Both sides therefore took those
two jars from ~/.m2, and whenever the compared branches share a version, which
is the normal case for two branches cut from develop, they took the *same*
ones: any change in the PDF backend or the templates module measured a delta of
exactly zero. A stale backend jar built against the other branch's core could
also throw NoSuchMethodError mid-run instead of reporting a number.

Both modules are now installed per branch, guarded by the presence of their
pom so a pre-split layout, where the root build produces them, still works.
Mirrors the install sequence the CI performance job already uses.

* docs(benchmarks): say what an A/B run actually rebuilds, and stop compiling render-pdf's tests

The A/B section still told readers each branch is rebuilt with
install -pl :graph-compose-core "so the benchmark measures that branch's
engine" - the sentence the install fix invalidates, and the only place stating
what gets rebuilt.

render-pdf now installs with -Dmaven.test.skip=true rather than -DskipTests.
Only its main jar is needed, but -DskipTests still compiles its whole test
tree, so a branch whose render-pdf tests do not compile would abort the entire
A/B at the install step with the output sent to /dev/null. templates keeps
-DskipTests, because the benchmark resolves its tests-classifier jar and
maven.test.skip would not produce one.

Author: DemchaAV

docs/operations/benchmarks.md

@@ -230,10 +230,12 @@ The wrappers above benchmark whatever is currently checked out. To answer "is
 branch B faster or slower than branch A?" fairly on a noisy laptop, use the
 dedicated A/B scripts. They **interleave** the two branches (A,B,A,B,…) so
 thermal drift averages out, **repeat** each branch and compare **medians**, and
-**cool down** between runs. Each branch is rebuilt (`install -pl :graph-compose-core`) before its
-runs so the benchmark measures that branch's engine, and untracked benchmark
-probes are moved aside around the branch switch so they cannot break the other
-branch's compile.
+**cool down** between runs. Each branch is rebuilt before its runs so the benchmark measures that branch:
+the engine, plus `render-pdf` and `templates`, because the benchmarks module
+resolves all three at the branch's own version. Installing only the engine would
+leave both sides sharing whichever backend and template jars happened to be in
+`~/.m2`. Untracked benchmark probes are moved aside around the branch switch so
+they cannot break the other branch's compile.
 
 - **Windows (PowerShell)** — `scripts/ab-bench.ps1`, full suite (latency,
   throughput, scalability, stress, comparative):

scripts/ab-bench.sh

@@ -157,6 +157,27 @@ build_engine_and_classpath() {
   local engine_pl="."
   [ -f core/pom.xml ] && engine_pl=":graph-compose-core"
   "$MVNW" -B -ntp -DskipTests install -pl "$engine_pl" >/dev/null
+  # The benchmark also resolves render-pdf and templates (plus the templates
+  # tests-classifier jar, which is never published) at the branch's own version,
+  # so those have to be rebuilt per branch as well. Install only the engine and
+  # both sides resolve the SAME jars from ~/.m2 whenever the two branches share a
+  # version — the normal case for two branches off develop — so every change in
+  # the PDF backend or the templates module reports a delta of exactly zero.
+  # Guarded by existence: a pre-split layout builds them from the root instead.
+  # render-pdf contributes only its main jar, so skip building its tests too:
+  # -DskipTests still compiles them, and a branch whose render-pdf tests do not
+  # compile would abort the whole A/B at the install step with its output sent to
+  # /dev/null. templates must keep -DskipTests, because the benchmark resolves its
+  # tests-classifier jar and maven.test.skip would not produce one.
+  local module
+  for module in render-pdf templates; do
+    [ -f "$module/pom.xml" ] || continue
+    if [ "$module" = "render-pdf" ]; then
+      "$MVNW" -B -ntp -f "$module/pom.xml" -Dmaven.test.skip=true install >/dev/null
+    else
+      "$MVNW" -B -ntp -f "$module/pom.xml" -DskipTests install >/dev/null
+    fi
+  done
   "$MVNW" -B -ntp -f benchmarks/pom.xml test-compile dependency:build-classpath \
           -DincludeScope=test -Dmdep.outputFile=target/benchmark.classpath >/dev/null
   CP="benchmarks/target/test-classes${SEP}benchmarks/target/classes${SEP}$(cat benchmarks/target/benchmark.classpath)"

scripts/run-benchmarks.ps1

@@ -328,6 +328,22 @@ try {
         & $mavenWrapper "-B" "-ntp" "-DskipTests" "install" "-pl" $enginePl
     }
 
+    # The benchmark also resolves render-pdf and templates (plus the templates
+    # tests-classifier jar, which is never published) at the branch's own
+    # version. Installing only the engine leaves those coming from ~/.m2, so an
+    # A/B between two branches that share a version compares the SAME backend and
+    # template jars on both sides and reports a delta of exactly zero for any
+    # change in them. Guarded by existence: a pre-split layout builds them from
+    # the root instead.
+    foreach ($module in @('render-pdf', 'templates')) {
+        $modulePom = Join-Path $repoRoot "$module/pom.xml"
+        if (Test-Path $modulePom) {
+            Invoke-LoggedCommand -Name "01-install-$module" -Command {
+                & $mavenWrapper "-B" "-ntp" "-f" $modulePom "-DskipTests" "install"
+            }
+        }
+    }
+
     Invoke-LoggedCommand -Name "01-build-classpath" -Command {
         & $mavenWrapper "-B" "-ntp" "-f" $benchmarksPom "test-compile" "dependency:build-classpath" "-DincludeScope=test" "-Dmdep.outputFile=target/benchmark.classpath"
     }