hotfix(hooks): prevent worker crashes from poisoning the process pool#32
Merged
Conversation
_generate_chunk in the synthetic data generator could throw unhandled exceptions (connection failure, faker import error, bad preset) that killed the spawned worker process. Once a worker dies, ProcessPoolExecutor refuses all further work with "A child process terminated abruptly, the process pool is not usable anymore." - Wrapped _generate_chunk body in try/except so it returns 0 on error instead of crashing the worker - Added _reset_pool() to worker_pool.py to recreate a broken pool - Multi-CPU path now catches BrokenProcessPool and retries once with a fresh pool packaging: improve RPM and DEB package builds - Use pyproject.toml as single source of truth for dependencies - Add package→import name mapping (PyGObject→gi, pycairo→cairo, etc.) - Create sql-schema-studio-install-deps helper script for Python dependencies - Launcher now checks for missing packages and guides user - Remove requirements.txt generation, read directly from pyproject.toml - Make icon and desktop file installation more robust with fallbacks - Fix ModuleNotFoundError by properly setting PYTHONPATH docs: update README with improved installation instructions - Simplify installation steps with unified approach - Add sql-schema-studio-install-deps to post-install workflow - Remove scattered pip install commands - Add pipx as recommended installation method - Update troubleshooting section for new workflow Closes #<issue-number> (if applicable)
- _reset_pool() in worker_pool.py referenced _worker_pool but the global singleton is named _pool, causing three mypy name-defined errors
…y error _reset_pool accessed _pool._pool.shutdown() directly but mypy can't narrow the inner ProcessPoolExecutor | None to non-None. Replaced with a call to the public shutdown() method which already handles the check.
Peter-L-SVK
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
_generate_chunk in the synthetic data generator could throw unhandled
exceptions (connection failure, faker import error, bad preset) that
killed the spawned worker process. Once a worker dies, ProcessPoolExecutor
refuses all further work with 'A child process terminated abruptly,
the process pool is not usable anymore.'
instead of crashing the worker
a fresh pool
packaging: improve RPM and DEB package builds
docs: update README with improved installation instructions