Speed up gtfs-rt integration tests: cache bundle builds + parallel forks

[aaronbrethorst]

Summary

The onebusaway-gtfsrt-integration-tests module was the dominant cost in CI — ~13:51 of an ~18 min job, and the source of ~67k log lines. Two causes: AbstractGtfsRealtimeIntegrationTest's @Before rebuilt the full GTFS transit-graph bundle (the expensive part) before every test method, and the suite ran serially.

• Build each dataset's bundle once per JVM, cached by integration-test path (BUILT_BUNDLES). Multiple classes share a dataset (nyct_wrong_way, st_duplicated_trips, …), and all of NyctRandomIntegrationTest's active methods share one — so this removes the bulk of redundant builds. Only the immutable on-disk bundle is shared; a fresh Spring context + GtfsRealtimeSource is still created per method, so realtime state that accumulates in context singletons (route cancellations, dynamic/added blocks) stays isolated.
• Run test classes in parallel JVMs via surefire forkCount=2 + reuseForks=true (overridable with -Dintegration.forkCount).
• Make each bundle build's temp dir process-unique (pid + counter) so parallel forks can't collide on a same-millisecond timestamp.

Local results (this module): test time ~13:51 → ~4:20, bundle builds 12 → 5, no OOM at forkCount=2.

Test plan

• Full module green via mvn test -pl onebusaway-gtfsrt-integration-tests -am: 15 classes, 0 failures / 0 errors, same skip pattern as the prior CI run
• NyctRandomIntegrationTest builds the bundle once instead of 5×, all 5 active methods still pass (370s → 254s)
• No OutOfMemoryError with forkCount=2
• Re-verified green after code-review fixes (refactor + BundleBuilder uniqueness + pom)
• CI to confirm on the real runner: memory headroom at forkCount=2 on the standard 4-core/16 GB runner, and end-to-end job wall-clock

Review notes (non-blocking)

• Runner memory is the main unknown — forkCount=2 × full transit bundle was only validated on a dev machine. If CI OOMs, set -Dintegration.forkCount=1 (the property is wired for exactly this).
• The parallel-fork temp-dir fix is structurally sound (process-unique paths) rather than empirically race-reproduced.
• Reuse note: this in-run cache complements BundleBuilder's existing manual cross-run reuse (bundle.keep / bundle.index.json); they were left as separate mechanisms rather than unified, to keep the change scoped.
• Not addressed here: the ~67k-line log spew. It's mostly INFO from bundle builds (now deduplicated) and is marginal for time; a proper fix requires untangling the runtime log4j2 reconfiguration triggered by the workflow's -Dlog4j.configuration= flag, which is out of scope.

Summary by CodeRabbit

• Tests
  • Enhanced integration test parallelization with optimized configuration.
  • Improved test bundle caching to reduce redundant processing across test runs.
  • Reduced risk of directory collision issues during parallel test execution.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7a87ec65-b5ed-4854-adf8-ef784da3eff1

📥 Commits

Reviewing files that changed from the base of the PR and between 0780dbd and 1dde779.

📒 Files selected for processing (3)

• onebusaway-gtfsrt-integration-tests/pom.xml
• onebusaway-gtfsrt-integration-tests/src/test/java/org/onebusaway/transit_data_federation/impl/realtime/gtfs_realtime/AbstractGtfsRealtimeIntegrationTest.java
• onebusaway-gtfsrt-integration-tests/src/test/java/org/onebusaway/transit_data_federation/impl/realtime/gtfs_realtime/BundleBuilder.java

---

📝 Walkthrough

Walkthrough

Integration test parallelism is enabled by three coordinated changes: BundleBuilder replaces timestamp-based temp directory naming with a PID-plus-sequence-counter uniqueId, AbstractGtfsRealtimeIntegrationTest adds a static BUILT_BUNDLES cache so bundle builds are reused across test classes within the same JVM, and pom.xml configures maven-surefire-plugin with a configurable integration.forkCount property and reuseForks=true.

Changes

Parallel Integration Test Optimization

Layer / File(s)	Summary
Bundle uniqueness and JVM-wide caching ...gtfs_realtime/BundleBuilder.java, ...gtfs_realtime/AbstractGtfsRealtimeIntegrationTest.java	BundleBuilder introduces a static AtomicInteger BUNDLE_SEQ and derives temp directory names from ProcessHandle.current().pid() + sequence counter instead of a millisecond timestamp. AbstractGtfsRealtimeIntegrationTest adds a static BUILT_BUNDLES map and rewrites setup() to look up an existing BundleContext by integration test path, building and caching one only on the first call per path; getBundleBuilder() is removed.
Maven Surefire fork configuration onebusaway-gtfsrt-integration-tests/pom.xml	Adds integration.forkCount property (default 2) and a maven-surefire-plugin build configuration setting forkCount=${integration.forkCount} and reuseForks=true, allowing the per-JVM bundle cache to persist across test classes within each fork.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically summarizes the main changes: caching bundle builds and enabling parallel test execution with forked JVMs, which directly addresses the optimization objectives.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch slow-ci

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}