Skip to content

fix(infra): add JVM shutdown hooks to stop Testcontainers after test execution#189

Merged
Puneethkumarck merged 1 commit into
mainfrom
fix/testcontainers-cleanup-shutdown-hooks
Mar 17, 2026
Merged

fix(infra): add JVM shutdown hooks to stop Testcontainers after test execution#189
Puneethkumarck merged 1 commit into
mainfrom
fix/testcontainers-cleanup-shutdown-hooks

Conversation

@Puneethkumarck

@Puneethkumarck Puneethkumarck commented Mar 16, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add Runtime.getRuntime().addShutdownHook() to all 10 AbstractIntegrationTest classes
  • Ensures Testcontainers (PostgreSQL, Kafka, Redis, Mailpit) are stopped when the JVM exits
  • Fixes orphaned container accumulation (~30-50 containers / 5-15 GB leaked memory per test run)

Test plan

  • Verified 21 orphaned containers cleaned up manually
  • Shutdown hooks added to all 10 services: S1, S2, S3, S4, S5, S6, S7, S10, S11, S13
  • Run full make test-integration and verify docker ps shows 0 Testcontainers after completion

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Added graceful shutdown handling for test containers across all modules, ensuring external services (databases, message queues, etc.) are properly cleaned up when tests complete. Improves test reliability and resource management.

…execution

Testcontainers started in static initializers were never stopped, causing
orphaned PostgreSQL, Kafka, Redis, and Mailpit containers to accumulate
after each test run (~30-50 containers, 5-15 GB leaked memory).

Add Runtime.getRuntime().addShutdownHook() to all 10 AbstractIntegrationTest
classes to ensure containers are stopped when the JVM exits.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Mar 16, 2026

Copy link
Copy Markdown

Walkthrough

Integration test modules across 10 microservices now include JVM shutdown hooks in their static initializers to gracefully stop Testcontainers (Redis, Kafka, PostgreSQL, Mailpit) on JVM termination, ensuring deterministic resource cleanup without altering startup sequences or public APIs.

Changes

Cohort / File(s) Summary
Multi-Container Shutdown
api-gateway-iam/.../AbstractIntegrationTest.java, merchant-iam/.../AbstractIntegrationTest.java
Shutdown hooks registered to stop Redis, Kafka, and PostgreSQL containers in controlled sequence on JVM exit. Merchant-IAM additionally stops Mailpit.
Kafka & PostgreSQL Shutdown
blockchain-custody/.../AbstractIntegrationTest.java, compliance-travel-rule/.../AbstractIntegrationTest.java, fiat-off-ramp/.../AbstractIntegrationTest.java, fiat-on-ramp/.../AbstractIntegrationTest.java, fx-liquidity-engine/.../AbstractIntegrationTest.java, ledger-accounting/.../AbstractIntegrationTest.java, merchant-onboarding/.../AbstractIntegrationTest.java, payment-orchestrator/.../AbstractIntegrationTest.java
JVM shutdown hooks added to gracefully terminate Kafka and PostgreSQL Testcontainers on application shutdown.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed Title accurately describes the core change: adding JVM shutdown hooks to gracefully stop Testcontainers across all integration test classes.
Description check ✅ Passed Description covers the problem (orphaned containers/memory leaks), the solution (shutdown hooks), and test verification steps, though integration test execution remains pending.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix/testcontainers-cleanup-shutdown-hooks
📝 Coding Plan
  • Generate coding plan for human review comments

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

CodeRabbit can approve the review once all CodeRabbit's comments are resolved.

Enable the reviews.request_changes_workflow setting to automatically approve the review once all CodeRabbit's comments are resolved.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In
`@blockchain-custody/blockchain-custody/src/integration-test/java/com/stablecoin/payments/custody/AbstractIntegrationTest.java`:
- Around line 30-37: The static initializer currently starts POSTGRES then KAFKA
and registers a shutdown hook that calls KAFKA.stop() and POSTGRES.stop(), but
if KAFKA.stop() throws the POSTGRES.stop() call is skipped and startup
exceptions can leave containers running; change the startup sequence to start
each service with try/catch so if KAFKA.start() fails you stop POSTGRES
immediately, and only register the Runtime.getRuntime().addShutdownHook(...)
after both starts succeed; in the shutdown hook wrap each stop call in its own
try/catch (call KAFKA.stop() and POSTGRES.stop() independently) so one failure
does not prevent the other from running.

In
`@merchant-iam/merchant-iam/src/integration-test/java/com/stablecoin/payments/merchant/iam/AbstractIntegrationTest.java`:
- Around line 53-58: The shutdown hook currently calls REDIS.stop(),
MAILPIT.stop(), KAFKA.stop(), POSTGRES.stop() sequentially which can abort
remaining stops if one throws; add a private helper method stopQuietly(Startable
container) that wraps container.stop() in a try/catch and ignores exceptions,
then replace each direct call in the Runtime.getRuntime().addShutdownHook lambda
with stopQuietly(REDIS), stopQuietly(MAILPIT), stopQuietly(KAFKA),
stopQuietly(POSTGRES) so each container is best-effort stopped even if others
fail.

In
`@payment-orchestrator/payment-orchestrator/src/integration-test/java/com/stablecoin/payments/orchestrator/AbstractIntegrationTest.java`:
- Around line 31-37: The static initializer should be made failure-safe:
register the shutdown hook first (or ensure it tolerates partially-initialized
state) and start containers with per-step error handling so any
previously-started container is stopped if a later start fails; in the shutdown
hook call KAFKA.stop() and POSTGRES.stop() inside separate try/catch blocks (or
use try/finally) so an exception from one does not prevent the other from
stopping, and when starting in the static block ensure that on any exception you
stop already-started resources (KAFKA or POSTGRES) before rethrowing so no
containers are orphaned.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 82484a0e-d53e-4caf-ad3d-4dd32d8b8e49

📥 Commits

Reviewing files that changed from the base of the PR and between 9c259c1 and e2705d4.

📒 Files selected for processing (10)
  • api-gateway-iam/api-gateway-iam/src/integration-test/java/com/stablecoin/payments/gateway/iam/AbstractIntegrationTest.java
  • blockchain-custody/blockchain-custody/src/integration-test/java/com/stablecoin/payments/custody/AbstractIntegrationTest.java
  • compliance-travel-rule/compliance-travel-rule/src/integration-test/java/com/stablecoin/payments/compliance/AbstractIntegrationTest.java
  • fiat-off-ramp/fiat-off-ramp/src/integration-test/java/com/stablecoin/payments/offramp/AbstractIntegrationTest.java
  • fiat-on-ramp/fiat-on-ramp/src/integration-test/java/com/stablecoin/payments/onramp/AbstractIntegrationTest.java
  • fx-liquidity-engine/fx-liquidity-engine/src/integration-test/java/com/stablecoin/payments/fx/AbstractIntegrationTest.java
  • ledger-accounting/ledger-accounting/src/integration-test/java/com/stablecoin/payments/ledger/AbstractIntegrationTest.java
  • merchant-iam/merchant-iam/src/integration-test/java/com/stablecoin/payments/merchant/iam/AbstractIntegrationTest.java
  • merchant-onboarding/merchant-onboarding/src/integration-test/java/com/stablecoin/payments/merchant/onboarding/AbstractIntegrationTest.java
  • payment-orchestrator/payment-orchestrator/src/integration-test/java/com/stablecoin/payments/orchestrator/AbstractIntegrationTest.java

Comment on lines 30 to 37
static {
POSTGRES.start();
KAFKA.start();
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
KAFKA.stop();
POSTGRES.stop();
}));
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Harden shutdown hook to avoid partial cleanup and startup leaks.

If KAFKA.stop() throws at Line 34, POSTGRES.stop() at Line 35 will not run. Also, a startup exception before hook registration can still leave orphaned containers.

Proposed hardening patch
 static {
-    POSTGRES.start();
-    KAFKA.start();
-    Runtime.getRuntime().addShutdownHook(new Thread(() -> {
-        KAFKA.stop();
-        POSTGRES.stop();
-    }));
+    try {
+        POSTGRES.start();
+        KAFKA.start();
+    } catch (RuntimeException ex) {
+        safeStop(KAFKA);
+        safeStop(POSTGRES);
+        throw ex;
+    }
+    Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+        safeStop(KAFKA);
+        safeStop(POSTGRES);
+    }, "testcontainers-shutdown"));
 }
+
+private static void safeStop(org.testcontainers.containers.GenericContainer<?> container) {
+    try {
+        if (container != null && container.isRunning()) {
+            container.stop();
+        }
+    } catch (Exception ignored) {
+        // best-effort cleanup for test infra
+    }
+}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
static {
POSTGRES.start();
KAFKA.start();
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
KAFKA.stop();
POSTGRES.stop();
}));
}
static {
try {
POSTGRES.start();
KAFKA.start();
} catch (RuntimeException ex) {
safeStop(KAFKA);
safeStop(POSTGRES);
throw ex;
}
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
safeStop(KAFKA);
safeStop(POSTGRES);
}, "testcontainers-shutdown"));
}
private static void safeStop(org.testcontainers.containers.GenericContainer<?> container) {
try {
if (container != null && container.isRunning()) {
container.stop();
}
} catch (Exception ignored) {
// best-effort cleanup for test infra
}
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@blockchain-custody/blockchain-custody/src/integration-test/java/com/stablecoin/payments/custody/AbstractIntegrationTest.java`
around lines 30 - 37, The static initializer currently starts POSTGRES then
KAFKA and registers a shutdown hook that calls KAFKA.stop() and POSTGRES.stop(),
but if KAFKA.stop() throws the POSTGRES.stop() call is skipped and startup
exceptions can leave containers running; change the startup sequence to start
each service with try/catch so if KAFKA.start() fails you stop POSTGRES
immediately, and only register the Runtime.getRuntime().addShutdownHook(...)
after both starts succeed; in the shutdown hook wrap each stop call in its own
try/catch (call KAFKA.stop() and POSTGRES.stop() independently) so one failure
does not prevent the other from running.

Comment on lines +53 to +58
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
REDIS.stop();
MAILPIT.stop();
KAFKA.stop();
POSTGRES.stop();
}));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify shutdown hooks with unguarded sequential stop() calls.
# Expected after fix: no matches.
fd 'AbstractIntegrationTest.java' \
  | xargs rg -nU 'addShutdownHook\(new Thread\(\(\) -> \{\n(?:\s+[A-Z_]+\.stop\(\);\n)+\s+\}\)\);'

Repository: Puneethkumarck/stablebridge-platform

Length of output: 6951


🏁 Script executed:

cat -n merchant-iam/merchant-iam/src/integration-test/java/com/stablecoin/payments/merchant/iam/AbstractIntegrationTest.java | sed -n '50,65p'

Repository: Puneethkumarck/stablebridge-platform

Length of output: 553


Wrap container shutdowns to prevent cleanup chain-breaks.

Sequential unguarded stop() calls on lines 54–57 will skip remaining containers if any throws. Introduce a helper to isolate failures and ensure best-effort cleanup:

private static void stopQuietly(Startable container) {
    try {
        container.stop();
    } catch (Exception ignored) {
        // best-effort during JVM shutdown
    }
}

Then replace each CONTAINER.stop(); with stopQuietly(CONTAINER);.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@merchant-iam/merchant-iam/src/integration-test/java/com/stablecoin/payments/merchant/iam/AbstractIntegrationTest.java`
around lines 53 - 58, The shutdown hook currently calls REDIS.stop(),
MAILPIT.stop(), KAFKA.stop(), POSTGRES.stop() sequentially which can abort
remaining stops if one throws; add a private helper method stopQuietly(Startable
container) that wraps container.stop() in a try/catch and ignores exceptions,
then replace each direct call in the Runtime.getRuntime().addShutdownHook lambda
with stopQuietly(REDIS), stopQuietly(MAILPIT), stopQuietly(KAFKA),
stopQuietly(POSTGRES) so each container is best-effort stopped even if others
fail.

Comment on lines 31 to +37
static {
POSTGRES.start();
KAFKA.start();
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
KAFKA.stop();
POSTGRES.stop();
}));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Make container cleanup failure-safe in startup and shutdown paths.

If KAFKA.stop() fails at Line 35, POSTGRES.stop() at Line 36 is skipped. Also, startup failures before hook registration can still orphan already-started containers.

Proposed hardening patch
 static {
-    POSTGRES.start();
-    KAFKA.start();
-    Runtime.getRuntime().addShutdownHook(new Thread(() -> {
-        KAFKA.stop();
-        POSTGRES.stop();
-    }));
+    try {
+        POSTGRES.start();
+        KAFKA.start();
+    } catch (RuntimeException ex) {
+        safeStop(KAFKA);
+        safeStop(POSTGRES);
+        throw ex;
+    }
+    Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+        safeStop(KAFKA);
+        safeStop(POSTGRES);
+    }, "testcontainers-shutdown"));
 }
+
+private static void safeStop(org.testcontainers.containers.GenericContainer<?> container) {
+    try {
+        if (container != null && container.isRunning()) {
+            container.stop();
+        }
+    } catch (Exception ignored) {
+        // best-effort cleanup for test infra
+    }
+}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@payment-orchestrator/payment-orchestrator/src/integration-test/java/com/stablecoin/payments/orchestrator/AbstractIntegrationTest.java`
around lines 31 - 37, The static initializer should be made failure-safe:
register the shutdown hook first (or ensure it tolerates partially-initialized
state) and start containers with per-step error handling so any
previously-started container is stopped if a later start fails; in the shutdown
hook call KAFKA.stop() and POSTGRES.stop() inside separate try/catch blocks (or
use try/finally) so an exception from one does not prevent the other from
stopping, and when starting in the static block ensure that on any exception you
stop already-started resources (KAFKA or POSTGRES) before rethrowing so no
containers are orphaned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant