Fix shutdown hang on session processor thread lock#108
Merged
lstein merged 2 commits intolstein/feature/nicer-shutdownfrom Mar 2, 2026
Merged
Fix shutdown hang on session processor thread lock#108lstein merged 2 commits intolstein/feature/nicer-shutdownfrom
lstein merged 2 commits intolstein/feature/nicer-shutdownfrom
Conversation
…aemon Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Fix shutdown hanging issue on thread lock
Fix shutdown hang on session processor thread lock
Mar 2, 2026
lstein
added a commit
that referenced
this pull request
Mar 2, 2026
* Initial plan * Fix shutdown hang: wake session processor thread on stop() and mark daemon Co-authored-by: lstein <111189+lstein@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
lstein
added a commit
that referenced
this pull request
Mar 2, 2026
* Initial plan * Handle KeyboardInterrupt in run_app to allow single Ctrl+C shutdown Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Force os._exit(0) on KeyboardInterrupt to avoid hanging on background threads Co-authored-by: lstein <111189+lstein@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Fix graceful shutdown to wait for download/install worker threads (#102) * Initial plan * Replace os._exit(0) with ApiDependencies.shutdown() on KeyboardInterrupt Instead of immediately force-exiting the process on CTRL+C, call ApiDependencies.shutdown() to gracefully stop the download and install manager services, allowing active work to complete or cancel cleanly before the process exits. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Make stop() idempotent in download and model install services When CTRL+C is pressed, uvicorn's graceful shutdown triggers the FastAPI lifespan which calls ApiDependencies.shutdown(), then a KeyboardInterrupt propagates from run_until_complete() hitting the except block which tries to call ApiDependencies.shutdown() a second time. Change both stop() methods to return silently (instead of raising) when the service is not running. This handles: - Double-shutdown: lifespan already stopped the services - Early interrupt: services were never fully started Co-authored-by: lstein <111189+lstein@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Fix shutdown hang on session processor thread lock (#108) * Initial plan * Fix shutdown hang: wake session processor thread on stop() and mark daemon Co-authored-by: lstein <111189+lstein@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
lstein
added a commit
that referenced
this pull request
Mar 6, 2026
…i#8936) * Fix: Kill the server with one keyboard interrupt (#94) * Initial plan * Handle KeyboardInterrupt in run_app to allow single Ctrl+C shutdown Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Force os._exit(0) on KeyboardInterrupt to avoid hanging on background threads Co-authored-by: lstein <111189+lstein@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Fix graceful shutdown to wait for download/install worker threads (#102) * Initial plan * Replace os._exit(0) with ApiDependencies.shutdown() on KeyboardInterrupt Instead of immediately force-exiting the process on CTRL+C, call ApiDependencies.shutdown() to gracefully stop the download and install manager services, allowing active work to complete or cancel cleanly before the process exits. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Make stop() idempotent in download and model install services When CTRL+C is pressed, uvicorn's graceful shutdown triggers the FastAPI lifespan which calls ApiDependencies.shutdown(), then a KeyboardInterrupt propagates from run_until_complete() hitting the except block which tries to call ApiDependencies.shutdown() a second time. Change both stop() methods to return silently (instead of raising) when the service is not running. This handles: - Double-shutdown: lifespan already stopped the services - Early interrupt: services were never fully started Co-authored-by: lstein <111189+lstein@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Fix shutdown hang on session processor thread lock (#108) * Initial plan * Fix shutdown hang: wake session processor thread on stop() and mark daemon Co-authored-by: lstein <111189+lstein@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Fix: shut down asyncio executor on KeyboardInterrupt to prevent post-generation hang (#112) Fix: cancel pending asyncio tasks before loop.close() to suppress destroyed-task warnings Fix: suppress stack trace when dispatching events after event loop is closed on shutdown Fix: cancel in-progress generation on stop() to prevent core dump during mid-flight Ctrl+C Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
After a generation completes, the server hangs on
^Cbecause Python'sthreading._shutdown()blocks waiting for the non-daemonsession_processorthread to exit — which is sleeping inpoll_now_event.wait(polling_interval)and won't notice_stop_eventuntil the 1-second timeout elapses. If the processor was paused via the API,resume_event.wait()(no timeout) would block indefinitely.Changes to
session_processor_default.py:daemon=Trueon the session processor thread — Python's interpreter shutdown will no longer block on this thread even if it's stuck (e.g., mid-generation)stop()now sets_poll_now_eventand_resume_event— immediately wakes the thread from any sleeping/blocked state so it checksstop_eventand exits without waiting for the polling interval to elapseRelated Issues / Discussions
QA Instructions
^Cto shut down the server^CAlso verify with a paused processor: pause via the API, then shut down — process should still exit immediately.
Merge Plan
Checklist
What's Newcopy (if doing a release after this PR)Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.