Skip to content

Commit 498bb7f

Browse files
authored
Update architecture docs for unified test provider (#3313)
* docs: shrink documentation to new technical structure * docs: replace old provider lookup, diagram and table with updated information * docs: add breaking changes paragraph for JUnit for 3.6. * docs: move changes to provider model and new stack trace handling features to architecture.md * docs: improve distinction between old and new provider model title * docs: improve distinction between old and new provider model --------- Signed-off-by: Sebastian Tiemann <setie@mailbox.org>
1 parent 1ed736a commit 498bb7f

2 files changed

Lines changed: 152 additions & 291 deletions

File tree

maven-surefire-plugin/src/site/markdown/architecture.md

Lines changed: 152 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,6 @@ under the License.
1919

2020
# Apache Maven Surefire — Architecture Overview
2121

22-
> Architecture reference for the `master` branch (version 3.5.x).
23-
> For the upcoming 3.6.0 changes, see [PR #3179 — Unified JUnit Platform Provider](pr-3179-unified-provider.md).
2422

2523
## What is Surefire?
2624

@@ -32,7 +30,9 @@ Apache Maven Surefire is the test execution framework for Maven. It ships three
3230
| **maven-failsafe-plugin** | Runs integration tests during `integration-test` / `verify` phases |
3331
| **maven-surefire-report-plugin** | Generates HTML test reports from XML results |
3432

35-
Surefire supports JUnit 3, JUnit 4, JUnit 5 (Jupiter), TestNG, and plain POJO tests — each via a dedicated **provider** module. Tests execute in a **forked JVM** that communicates results back to Maven through a binary event stream protocol.
33+
Surefire supports JUnit 3, JUnit 4, JUnit 5 (Jupiter), TestNG, and plain POJO tests. <br>
34+
Until 3.5.x, each type was executed via a dedicated provider module. From 3.6.0 on, there is only one unified provider. <br>
35+
Tests execute in a **forked JVM** that communicates results back to Maven through a binary event stream protocol.
3636

3737
---
3838

@@ -71,11 +71,7 @@ graph TD
7171
end
7272
7373
subgraph "Providers — Loaded in Forked JVM"
74-
JP["surefire-junit-platform<br/><i>JUnit 5 / Jupiter</i>"]
75-
J4["surefire-junit4"]
76-
J47["surefire-junit47<br/><i>Parallel + categories</i>"]
77-
J3["surefire-junit3<br/><i>JUnit 3 + POJO</i>"]
78-
TNG["surefire-testng"]
74+
JP["surefire-junit-platform<br/><i>unified provider for all JUnit 3+ versions</i>"]
7975
end
8076
8177
subgraph "Shading"
@@ -180,6 +176,37 @@ The command line for the forked JVM is built by one of three `ForkConfiguration`
180176

181177
## Provider Model
182178

179+
### old Provider Model 3.5.x
180+
181+
```mermaid
182+
graph LR
183+
subgraph "Before — 3.5.x (5 providers)"
184+
M1["AbstractSurefireMojo"] --> PD1["ProviderDetector<br/>(priority-based)"]
185+
PD1 --> JP1["surefire-junit-platform"]
186+
PD1 --> TNG1["surefire-testng"]
187+
PD1 --> J471["surefire-junit47"]
188+
PD1 --> J41["surefire-junit4"]
189+
PD1 --> J31["surefire-junit3"]
190+
end
191+
```
192+
193+
194+
### new Provider Model since 3.6.0
195+
196+
```mermaid
197+
graph LR
198+
subgraph "After — 3.6.0 (1 provider)"
199+
M2["AbstractSurefireMojo"] --> PD2["Simplified detection"]
200+
PD2 --> JP2["surefire-junit-platform"]
201+
JP2 --> VE["Vintage Engine<br/>(JUnit 3/4)"]
202+
JP2 --> JE["Jupiter Engine<br/>(JUnit 5)"]
203+
JP2 --> TE["TestNG Engine<br/>(TestNG)"]
204+
end
205+
```
206+
207+
The five `ProviderInfo` implementations (`JUnit3ProviderInfo`, `JUnit4ProviderInfo`, `JUnitCoreProviderInfo`, `TestNgProviderInfo`, `JUnitPlatformProviderInfo`) are collapsed into a unified detection path that always selects `surefire-junit-platform`. Framework-specific configuration (TestNG groups, JUnit 4 categories, parallel execution) is now **mapped to JUnit Platform launcher configuration** rather than being handled by framework-specific providers.
208+
209+
183210
### SurefireProvider SPI
184211

185212
Every test framework adapter implements `SurefireProvider` (in `surefire-api`):
@@ -199,6 +226,8 @@ public interface SurefireProvider {
199226

200227
### Provider implementations
201228

229+
### 3.5.x: old implementation
230+
202231
| Provider | Module | Test framework | Key classes |
203232
|----------|--------|---------------|-------------|
204233
| **JUnit 3 + POJO** | `surefire-junit3` | JUnit 3.x, plain POJOs | `JUnit3Provider`, `PojoTestSetExecutor` |
@@ -207,29 +236,69 @@ public interface SurefireProvider {
207236
| **TestNG** | `surefire-testng` | TestNG 4.7+ | `TestNGProvider`, `TestNGExecutor` |
208237
| **JUnit Platform** | `surefire-junit-platform` | JUnit 5, any JUnit Platform engine | `JUnitPlatformProvider`, `LauncherAdapter` |
209238

210-
### Auto-detection
211239

212-
When no provider is manually configured, Surefire scans the test classpath and selects the **first applicable** provider:
240+
| Framework | Before (3.5.x) | After (3.6.0) |
241+
|-----------|----------------|---------------|
242+
| **JUnit 3** | Supported natively | Requires JUnit 4.12+ dependency (runs via Vintage Engine) |
243+
| **JUnit 4** | 4.0+ | **4.12+** (runs via Vintage Engine) |
244+
| **JUnit 5** | Any | Any (unchanged) |
245+
| **TestNG** | 4.7+ | **6.14.3+** (runs via TestNG JUnit Platform Engine) |
246+
| **POJO tests** | Supported | **Removed** |
213247

214-
```mermaid
215-
flowchart TD
216-
Start["Scan test classpath"] --> SPI{"SPI configured?<br/>(META-INF/services)"}
217-
SPI -->|Yes| UseSPI["Use SPI provider(s)"]
218-
SPI -->|No| JP{"JUnit Platform<br/>on classpath?"}
219-
JP -->|Yes| UseJP["Use surefire-junit-platform"]
220-
JP -->|No| TNG{"TestNG<br/>on classpath?"}
221-
TNG -->|Yes| UseTNG["Use surefire-testng"]
222-
TNG -->|No| J47{"JUnit ≥4.7 AND<br/>(parallel OR groups)?"}
223-
J47 -->|Yes| UseJ47["Use surefire-junit47"]
224-
J47 -->|No| J4{"JUnit 4.x<br/>on classpath?"}
225-
J4 -->|Yes| UseJ4["Use surefire-junit4"]
226-
J4 -->|No| J3["Use surefire-junit3<br/>(always applicable — fallback)"]
227-
```
248+
### Breaking Changes since 3.6.0
228249

229-
The priority order is defined in `AbstractSurefireMojo.createProviders()`. Surefire resolves the provider's dependencies at runtime via `SurefireDependencyResolver` and adds them to the forked JVM's classpath — the provider JAR is never a compile-time dependency of the plugin.
250+
| Change | Impact | Mitigation |
251+
|--------|--------|------------|
252+
| **JUnit 3 standalone** no longer supported | Projects using only JUnit 3 must add JUnit 4.12+ dependency | Add `junit:junit:4.12` — test code unchanged |
253+
| **JUnit 4 < 4.12** no longer supported | Upgrade to JUnit 4.12+ | Mechanical version bump |
254+
| **TestNG < 6.14.3** no longer supported | Upgrade to TestNG 6.14.3+ | Mechanical version bump |
255+
| **POJO tests** removed | Tests without framework annotations won't be found | Add `@Test` annotations |
256+
| **Category expression syntax** changed | Complex boolean group expressions may behave differently under JUnit Platform tag expressions | Review and test group filter configurations |
257+
| **Provider selection** changed | Manually configured legacy providers still work (via SPI) but auto-detection always chooses JUnit Platform | Pin surefire 3.5.x or add legacy provider as dependency |
230258

231259
---
232260

261+
#### JUnit 3 tests still work
262+
263+
JUnit 3 test code does not need to change. You only need to ensure your project depends on JUnit 4.12+ (which includes JUnit 3 API compatibility). The Vintage Engine executes JUnit 3 and JUnit 4 tests transparently.
264+
265+
#### POJO tests removed
266+
267+
The `LegacyPojoStackTraceWriter` and POJO test detection (`PojoTestSetExecutor`) are removed. Tests must use a recognized framework annotation (`@Test` from JUnit or TestNG).
268+
269+
### Group / category filtering
270+
271+
The custom JavaCC-based category expression parser (`surefire-grouper`) is replaced by JUnit Platform's native **tag expression** syntax. For most users, `<groups>` and `<excludedGroups>` configuration works unchanged, but the underlying evaluation engine is different. Complex boolean expressions may need review.
272+
273+
### Backward compatibility options
274+
275+
If upgrading causes issues, users have two fallback paths:
276+
277+
1. **Pin Surefire 3.5.x** — stay on the previous version:
278+
```xml
279+
<plugin>
280+
<groupId>org.apache.maven.plugins</groupId>
281+
<artifactId>maven-surefire-plugin</artifactId>
282+
<version>3.5.4</version>
283+
</plugin>
284+
```
285+
286+
2. **Use a legacy provider as a plugin dependency** (transitional):
287+
```xml
288+
<plugin>
289+
<groupId>org.apache.maven.plugins</groupId>
290+
<artifactId>maven-surefire-plugin</artifactId>
291+
<version>3.6.0</version>
292+
<dependencies>
293+
<dependency>
294+
<groupId>org.apache.maven.surefire</groupId>
295+
<artifactId>surefire-junit3</artifactId>
296+
<version>3.5.4</version>
297+
</dependency>
298+
</dependencies>
299+
</plugin>
300+
```
301+
233302
## Communication Protocol
234303

235304
The forked JVM communicates with Maven through a **binary event stream**. Events flow one-way: fork → Maven. Commands flow the other way: Maven → fork.
@@ -411,3 +480,61 @@ mvn compile -f surefire-grouper/pom.xml
411480
```
412481

413482
**Requirements**: Maven 3.6.3+, JDK 8+ (source level 8, `animal-sniffer` enforces Java 8 API).
483+
484+
---
485+
486+
## Stack trace filtering
487+
488+
A new `StackTraceProvider` optimizes memory usage by truncating stack traces to 15 frames and filtering JDK packages by default. This is configurable:
489+
490+
```xml
491+
<configuration>
492+
<stackTraceFilterPrefixes>
493+
<prefix>org.springframework.</prefix>
494+
<prefix>org.junit.</prefix>
495+
</stackTraceFilterPrefixes>
496+
</configuration>
497+
```
498+
499+
When not specified or empty, the default filters (`java.`, `javax.`, `sun.`, `jdk.`) apply.
500+
501+
---
502+
503+
## Stack Trace Memory Optimization
504+
505+
### Problem
506+
507+
To associate console output with test classes, Surefire captures stack traces for every output line. With full stack traces (25–30 frames typical), this consumed 600–1,800 bytes per line.
508+
509+
### Solution
510+
511+
The new `StackTraceProvider` class (in `surefire-api`) introduces:
512+
513+
- **Frame limit**: Maximum 15 frames per stack trace (sufficient to capture the test class after surefire framework frames)
514+
- **Package filtering**: JDK packages (`java.`, `javax.`, `sun.`, `jdk.`) filtered by default
515+
- **Configurable prefixes**: Users can specify custom filter prefixes that **replace** (not add to) the defaults
516+
517+
### Memory impact
518+
519+
| Metric | Before | After |
520+
|--------|--------|-------|
521+
| Frames per trace | 25–30 | ≤15 |
522+
| Bytes per output line | 600–1,800 | 300–600 |
523+
| Estimated savings || ~50% |
524+
525+
### Configuration
526+
527+
```xml
528+
<!-- Custom prefixes (replaces defaults) -->
529+
<configuration>
530+
<stackTraceFilterPrefixes>
531+
<prefix>org.springframework.</prefix>
532+
<prefix>org.junit.</prefix>
533+
</stackTraceFilterPrefixes>
534+
</configuration>
535+
```
536+
537+
```bash
538+
# Command line
539+
mvn test -Dsurefire.stackTraceFilterPrefixes=org.springframework.,org.junit.
540+
```

0 commit comments

Comments
 (0)