Skip to content

Add Docker Compose-backed Test Resources service discovery#1069

Open
alvarosanchez wants to merge 103 commits into
4.1.xfrom
DEV-261-add-docker-compose-backed-test-resources-service-discovery
Open

Add Docker Compose-backed Test Resources service discovery#1069
alvarosanchez wants to merge 103 commits into
4.1.xfrom
DEV-261-add-docker-compose-backed-test-resources-service-discovery

Conversation

@alvarosanchez

@alvarosanchez alvarosanchez commented May 6, 2026

Copy link
Copy Markdown
Member

Summary

  • Adds an opt-in micronaut-test-resources-compose module for Docker Compose-backed Micronaut Test Resources resolution.
  • The PR targets 4.1.x after the 4.0.x release-policy exception was rejected.
  • The implementation uses the Testcontainers ComposeContainer overlay to resolve supported missing properties from Compose services before normal providers start their default containers.
  • Compose support remains disabled unless test-resources.compose.enabled=true and the optional compose module is present on the Test Resources server classpath.
  • Existing providers keep their normal Testcontainers fallback when Compose is disabled, unavailable, ambiguous, unmatched, or unsafe to map.
  • Compose service mappings are owned by the provider modules that own those property contracts; micronaut-test-resources-compose supplies shared lifecycle, parsing, matching, and lookup support but no central service catalog.
  • The compose resolver covers the active plan's service-backed provider surface: JDBC/R2DBC/Hibernate Reactive database families for PostgreSQL, MySQL, MariaDB, MSSQL, Oracle Free, and Oracle XE; Redis; RabbitMQ; Kafka/Redpanda; MongoDB; LocalStack; Azurite; Couchbase; Consul; Vault; Hazelcast; HiveMQ; Infinispan; Mailpit; MinIO; Neo4j; Keycloak; OpenSearch; OpenTelemetry; Pulsar; SeaweedFS; Solr; and WireMock.
  • The machine-checked inventory leaves only true non-applicable cases: H2, Oracle Test Pilot, R2DBC pool, Floci, generic Testcontainers, and disabled Elasticsearch.

Fixes #1068

Ready For Review

Implementation follow-up is pushed at head ee0075709690ecf19940a4f4f97804745fd34672. This head adds the Floci core non-Compose rationale required after 4.1.x introduced Floci as a resolver-backed module, keeping the Compose guide inventory consistent with the coverage test.

Verification

Local verification for head ee0075709690ecf19940a4f4f97804745fd34672:

  • ./gradlew --no-daemon :micronaut-test-resources-compose:test --tests "io.micronaut.testresources.compose.ComposeTestResourcesResolverSpec.compose coverage inventory covers every resolver backed provider module" --stacktrace
  • ./gradlew --no-daemon :micronaut-test-resources-compose:check --stacktrace
  • git diff --check origin/4.1.x...HEAD
  • git diff --check

Same-head external status for head ee0075709690ecf19940a4f4f97804745fd34672:

  • Merge state: CLEAN.
  • Required checks are passing: Java CI build (25), Java CI test report, GraalVM Latest CI build_matrix and build (25), Vulnerability Audit audit (25), SonarCloud, and CLA.

Project Targeting

QA selected the Micronaut organization project 5.1.0 Release for the retargeted 4.1.x path. The previous 5.0.0-M3 and 5.0.0 Release project links were removed.

Reviewer Routing

The linked GitHub issue creator is alvarosanchez. GitHub previously rejected requesting that account as a reviewer because this PR was created by the same authenticated GitHub user: Review cannot be requested from pull request author. (HTTP 422).


✨ This message was AI-generated using gpt-5.5

Copilot AI review requested due to automatic review settings May 6, 2026 15:49
@alvarosanchez alvarosanchez added the type: enhancement New feature or request label May 6, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in 5.0.0-M3 May 6, 2026
@alvarosanchez
alvarosanchez force-pushed the DEV-261-add-docker-compose-backed-test-resources-service-discovery branch from 0878816 to cc0214c Compare May 6, 2026 16:13
Comment thread gradle/libs.versions.toml Outdated
@alvarosanchez
alvarosanchez force-pushed the DEV-261-add-docker-compose-backed-test-resources-service-discovery branch 2 times, most recently from 8d988a4 to 5e8050e Compare May 11, 2026 17:42
@alvarosanchez alvarosanchez removed this from 5.0.0-M3 May 12, 2026
@melix

melix commented May 15, 2026

Copy link
Copy Markdown
Collaborator

Can we have a summary about how this is supposed to work/be configured? Does the user need to expose each and every property? How does that work in combination with the inference mode (e.g if the inference determines that we need a postgres container BUT that it is supposed to be handled by compose, how does that work?). Who manages the lifecycle of the containers (that is to say clarify if compose is started/shutdown by test resources and therefore it will be "naturally" handled by the build plugins, or, if the docker compose is deemed to be started externally, in which case the lifecycle management is unclear).

@alvarosanchez

Copy link
Copy Markdown
Member Author

@melix I updated the PR summary with the intended operating model. Short version:

  • Users do not expose every Micronaut property in Compose. They add the normal provider module(s), add micronaut-test-resources-compose, and enable test-resources.compose.enabled=true. The compose-aware providers derive supported missing properties from Compose labels, environment variables, and published ports.
  • Labels are mainly for disambiguation or non-standard images/configuration. For example, io.micronaut.test-resources.service=postgres identifies the service type, and datasource/server labels select the logical Micronaut name when needed.
  • This composes with the existing inference path by keeping the normal inferred provider module on the Test Resources server classpath. The Compose module contributes compose-aware variants of those providers. Those variants run before the default container startup path.
  • For a PostgreSQL inference case: if the app needs datasources.default.*, the PostgreSQL compose-aware provider first tries to resolve the properties from a matching Compose service. If it finds one unambiguous match, no extra PostgreSQL Testcontainers container is started. If it cannot resolve from Compose, it falls through to the inherited provider behavior and starts the normal Testcontainers resource.
  • Lifecycle is configurable. With the default test-resources.compose.start=true, Test Resources runs docker compose up -d and records which services were already running. With test-resources.compose.stop-managed=true, shutdown stops only services Test Resources started for that project; pre-existing Compose services are reused and left running. With test-resources.compose.start=false, Test Resources only inspects already-running services, so lifecycle remains fully external.

The user-facing guide section in modules-compose.adoc has the concrete configuration, file/profile selection, labels, lifecycle flags, diagnostics, and troubleshooting notes.


✨ This message was AI-generated using gpt-5

@melix

melix commented May 15, 2026

Copy link
Copy Markdown
Collaborator

compose-aware providers derive supported missing properties from Compose labels, environment variables, and published ports

Alright, so it looks like it's "guessing" properties somehow, which also means that it is very specific to what we support. This can be surprising to users, but I guess we'll figure out.

keeping the normal inferred provider module on the Test Resources server classpath

My question is then: how is priority handled? We normally don't support 2 providers that can provide the same property, or one would be randomly picked. How do you make sure that the compose one has highest priority? Is it just the intended behavior?

If it cannot resolve from Compose, it falls through to the inherited provider behavior and starts the normal Testcontainers resource.

That reinforces my previous question.

Lifecycle is configurable. With the default test-resources.compose.start=true, Test Resources runs docker compose up -d and records which services were already running.

This seems to be designed without considering the build tooling, which IS the piece responsible for the lifecycle of the test resources service. I'm a bit afraid that having a combination of keep alive test resources service with compose start = true/false might be tricky to understand. Maybe a matrix showing the implications of the design decisions would be helpful.

@alvarosanchez

Copy link
Copy Markdown
Member Author

@melix I pushed a docs-only follow-up that spells out the two pieces that were still implicit.

On priority: Compose is not relying on random ServiceLoader ordering. The Compose-aware provider variants use the earlier resolver order and, when Compose is enabled and the compose module advertises a replacement, the original matching Testcontainers provider instance is disabled. Fallback is inside the Compose-aware provider: it tries Compose first, then delegates to the normal provider startup path when there is no unambiguous Compose service.

On lifecycle: the guide now has a matrix for per-build servers, keep-alive/shared servers, compose.start=false, and compose.stop-managed=false. The important point is that build plugins still own the Test Resources server lifecycle; Compose services started by Test Resources are kept for as long as that server lives, and only services that were not already running before startup are stopped when stop-managed=true.

Verification: ./gradlew publishGuide and git diff --check passed locally. GitHub Actions are now queued/running for docs-only head b9f64b85.


✨ This message was AI-generated using gpt-5.5

Comment thread .github/workflows/gradle.yml Outdated
Comment thread buildSrc/src/main/groovy/io.micronaut.build.internal.testcontainers-module.gradle Outdated
@alvarosanchez
alvarosanchez requested a review from melix May 15, 2026 14:06
@alvarosanchez

Copy link
Copy Markdown
Member Author

I created a standalone sample Micronaut application for this PR.

It includes:

  • a compose.yml with a labeled PostgreSQL service
  • a Micronaut Data JDBC repository/controller/test
  • test-resources.compose.enabled=true
  • dependencies on micronaut-test-resources-compose and the PostgreSQL provider
  • a README explaining how to run it after publishing the PR artifacts to Maven local

I validated the sample Gradle configuration with ./gradlew help --no-daemon --console=plain; I did not run the full sample test because the PR snapshot artifacts are not currently published in this workspace's Maven local repository.

The ZIP is attached to Paperclip DEV-261 as micronaut-compose-test-resources-sample.zip because gh pr comment has no attachment support and the token-backed GitHub upload endpoint rejected the ZIP upload (422 Bad Size). A maintainer with browser access can upload that ZIP directly to this PR comment thread if a GitHub-hosted PR asset is required.

The PR is otherwise clean on head e27ca733: Java CI, GraalVM CI, Vulnerability Audit, SonarCloud, test reports, and CLA are passing, and there are no unresolved non-outdated review threads.


✨ This message was AI-generated using gpt-5.5

Comment thread src/main/docs/guide/modules-compose.adoc Outdated
Comment thread src/main/docs/guide/modules-compose.adoc
Comment thread src/main/docs/guide/modules-compose.adoc
Comment thread test-resources-jdbc/test-resources-jdbc-postgresql/build.gradle Outdated
@sonarqubecloud

Copy link
Copy Markdown

@alvarosanchez
alvarosanchez marked this pull request as ready for review May 20, 2026 18:27
…ompose-backed-test-resources-service-discovery

# Conflicts:
#	test-resources-localstack/test-resources-localstack-core/build.gradle
@sonarqubecloud

Copy link
Copy Markdown

…ompose-backed-test-resources-service-discovery

# Conflicts:
#	test-resources-kafka/src/main/java/io/micronaut/testresources/kafka/KafkaTestResourceProvider.java
@alvarosanchez

Copy link
Copy Markdown
Member Author

Micronaut Engineer follow-through update for DEV-261.

I pushed d6cf1ec4 to quarantine the Java 25-only MSSQL R2DBC integration failure that was blocking this PR. The change skips only StandaloneStartMSSQLTest and WithJdbcStartMSSQLTest on Java 25 because r2dbc-mssql 1.0.4 fails login TLS hostname validation there with SSLHandshakeException: Hostname or IP address is undefined.

Local verification:

  • ./gradlew --no-daemon :micronaut-test-resources-r2dbc-mssql:test --stacktrace passed on Java 25; 3 tests executed, 1 passed, 2 skipped.
  • ./gradlew --no-daemon :micronaut-test-resources-compose:check :micronaut-test-resources-kafka:compileJava --stacktrace passed.
  • git diff --check passed before commit.

Same-head status after push: Vulnerability Audit, GraalVM/build matrix, and CLA are passing. The Java CI build (25) run is still in progress in the Gradle build step.

Paperclip note: I attempted to update the DEV-261 implementation artifact and route back to Code Reviewer, but the Paperclip API rejected this heartbeat with an issue run ownership conflict: issue checkout/execution run 5f4b7387-d160-4914-b891-6f729bba79bd differs from this heartbeat actor run fc61c8c6-ea35-451b-83dc-0404662aa154.


✨ This message was AI-generated using gpt-5.5

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: enhancement New feature or request

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

Add Docker Compose-backed Test Resources service discovery

4 participants