Fix Jetty shutdown hang and BrokerPool read-only race#6572
Conversation
Set a Jetty stop timeout so shutdown does not block indefinitely on active connections; move ShutdownListenerImpl.shutdown() to a daemon thread so it does not deadlock the Jetty shutdown-hook thread that also holds the BrokerPool write lock.
|
Thnx for isolating this in a separate PR! It is complex and significant enough! |
dizzzz
left a comment
There was a problem hiding this comment.
copilot feedback
The PR is solid, but a few optional enhancements could strengthen it:
1. Consider documenting the timeout constants: The 30-second Jetty stop timeout and 5-second catch-up margin are sensible but somewhat arbitrary. Add a comment explaining the rationale (e.g., "30s allows graceful drain of in-flight requests; 5s is a catch-up margin for lifecycle callbacks").
2. Daemon thread naming could be more specific: The thread name "JettyShutdown" is clear, but consider including context like "JettyShutdown-async" or a sequence number to aid debugging in scenarios with multiple shutdown events.
3. Consider adding a log statement in stopServerAndAwait(): When serverStopRequested.compareAndSet() returns false, it would be helpful to log that shutdown is already in progress (at debug level) to aid troubleshooting.
Summary
ShutdownListenerImpl.shutdown()to a daemon thread to avoid deadlocking the Jetty shutdown-hook thread that also holds the BrokerPool write lockThis bugfix was discovered during XSD 1.1 validation integration work but is completely independent of it — JettyStart.java is the only file touched.
What Changed
exist-core/src/main/java/org/exist/jetty/JettyStart.javaserverStopRequestedAtomicBooleanguard so concurrent shutdown paths (directshutdown(),ShutdownListenerImpl, andBrokerPoolAndJettyShutdownHookJVM hook) do not raceShutdownListenerImpl.shutdown()on a daemon thread to avoid the BrokerPool write-lock / Jetty-shutdown-hook deadlockTest Plan
system:shutdown()from XQuery — server stops cleanly without hang🤖 Generated with Claude Code