Skip to content

Fix Jetty shutdown hang and BrokerPool read-only race#6572

Merged
line-o merged 1 commit into
eXist-db:developfrom
duncdrum:dp-jetty-shutdown-fix
Jul 13, 2026
Merged

Fix Jetty shutdown hang and BrokerPool read-only race#6572
line-o merged 1 commit into
eXist-db:developfrom
duncdrum:dp-jetty-shutdown-fix

Conversation

@duncdrum

Copy link
Copy Markdown
Contributor

Summary

  • Sets a Jetty stop timeout so shutdown does not block indefinitely on active connections
  • Moves ShutdownListenerImpl.shutdown() to a daemon thread to avoid deadlocking the Jetty shutdown-hook thread that also holds the BrokerPool write lock

This bugfix was discovered during XSD 1.1 validation integration work but is completely independent of it — JettyStart.java is the only file touched.

What Changed

exist-core/src/main/java/org/exist/jetty/JettyStart.java

  • Add serverStopRequested AtomicBoolean guard so concurrent shutdown paths (direct shutdown(), ShutdownListenerImpl, and BrokerPoolAndJettyShutdownHook JVM hook) do not race
  • Dispatch ShutdownListenerImpl.shutdown() on a daemon thread to avoid the BrokerPool write-lock / Jetty-shutdown-hook deadlock
  • Set a Jetty stop timeout to bound the shutdown wait

Test Plan

  • Start eXist-db and trigger system:shutdown() from XQuery — server stops cleanly without hang
  • Send SIGTERM — JVM hook fires, Jetty stops within the stop timeout
  • Trigger shutdown from XML-RPC API — no deadlock with BrokerPool write lock

🤖 Generated with Claude Code

Set a Jetty stop timeout so shutdown does not block indefinitely on
active connections; move ShutdownListenerImpl.shutdown() to a daemon
thread so it does not deadlock the Jetty shutdown-hook thread that
also holds the BrokerPool write lock.
@dizzzz

dizzzz commented Jul 13, 2026

Copy link
Copy Markdown
Member

Thnx for isolating this in a separate PR! It is complex and significant enough!

@dizzzz dizzzz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copilot feedback

The PR is solid, but a few optional enhancements could strengthen it:

1. Consider documenting the timeout constants: The 30-second Jetty stop timeout and 5-second catch-up margin are sensible but somewhat arbitrary. Add a comment explaining the rationale (e.g., "30s allows graceful drain of in-flight requests; 5s is a catch-up margin for lifecycle callbacks").

2. Daemon thread naming could be more specific: The thread name "JettyShutdown" is clear, but consider including context like "JettyShutdown-async" or a sequence number to aid debugging in scenarios with multiple shutdown events.

3. Consider adding a log statement in stopServerAndAwait(): When serverStopRequested.compareAndSet() returns false, it would be helpful to log that shutdown is already in progress (at debug level) to aid troubleshooting.

@dizzzz dizzzz requested review from a team and line-o July 13, 2026 16:45
@line-o line-o merged commit 1d43dda into eXist-db:develop Jul 13, 2026
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants