Skip to content

Commit e8aa448

Browse files
committed
hotfix(hooks): prevent worker crashes from poisoning the process pool
_generate_chunk in the synthetic data generator could throw unhandled exceptions (connection failure, faker import error, bad preset) that killed the spawned worker process. Once a worker dies, ProcessPoolExecutor refuses all further work with "A child process terminated abruptly, the process pool is not usable anymore." - Wrapped _generate_chunk body in try/except so it returns 0 on error instead of crashing the worker - Added _reset_pool() to worker_pool.py to recreate a broken pool - Multi-CPU path now catches BrokenProcessPool and retries once with a fresh pool packaging: improve RPM and DEB package builds - Use pyproject.toml as single source of truth for dependencies - Add package→import name mapping (PyGObject→gi, pycairo→cairo, etc.) - Create sql-schema-studio-install-deps helper script for Python dependencies - Launcher now checks for missing packages and guides user - Remove requirements.txt generation, read directly from pyproject.toml - Make icon and desktop file installation more robust with fallbacks - Fix ModuleNotFoundError by properly setting PYTHONPATH docs: update README with improved installation instructions - Simplify installation steps with unified approach - Add sql-schema-studio-install-deps to post-install workflow - Remove scattered pip install commands - Add pipx as recommended installation method - Update troubleshooting section for new workflow Closes #<issue-number> (if applicable)
1 parent bc4a2b1 commit e8aa448

5 files changed

Lines changed: 518 additions & 149 deletions

File tree

README.md

Lines changed: 86 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -107,26 +107,27 @@ More detailed plans in [ROADMAP](https://github.com/Peter-L-SVK/sql-schema-studi
107107
- VTE/terminal
108108
- Developed on Fedora 43 Cinnamon and tested on Fedora 43 KDE Plasma 6
109109

110-
111110
## Installation from Packages
112111

113112
### Fedora / RHEL (RPM)
114113

115114
```bash
116115
# Install build dependencies
117-
sudo dnf install rpm-build rpmdevtools rpmlint python3-devel vte291-gtk4
118-
119-
pip install polars
116+
sudo dnf install rpm-build rpmdevtools rpmlint python3-devel \
117+
gtk4 gtksourceview5 vte291-gtk4
120118

121-
# Clone and build
119+
# Clone the repository
122120
git clone https://github.com/Peter-L-SVK/sql-schema-studio.git
123121
cd sql-schema-studio
124122

125123
# Build RPM package
126124
./scripts/packaging/build_rpm.sh
127125

128-
# Install
129-
sudo dnf install ~/rpmbuild/RPMS/noarch/sql-schema-studio-*.rpm
126+
# Install the RPM
127+
sudo dnf install sql-schema-studio-*.rpm
128+
129+
# After installation, install Python dependencies
130+
sql-schema-studio-install-deps
130131
```
131132

132133
### Debian / Ubuntu / Mint (DEB) / WSL2
@@ -135,27 +136,35 @@ sudo dnf install ~/rpmbuild/RPMS/noarch/sql-schema-studio-*.rpm
135136
# Install build dependencies
136137
sudo apt install -y python3-psycopg2 python3-gi python3-gi-cairo \
137138
python3-sqlparse python3-keyring python3-numpy \
138-
python3-scikit-learn python3-matplotlib python3-cairo python3-paramiko \
139-
gir1.2-gtk-4.0 gir1.2-gtksource-5 libvte-2.91-gtk4-dev \
139+
python3-matplotlib python3-cairo python3-paramiko \
140+
gir1.2-gtk-4.0 gir1.2-gtksource-5 gir1.2-vte-2.91 \
140141
python3-pipx
141142

142-
pip install polars --break-system-packages
143-
144-
# Install dependencies with pipx
145-
pipx install faker kbcstorage scikit-learn --include-deps
146-
147-
# Clone and build
143+
# Clone the repository
148144
git clone https://github.com/Peter-L-SVK/sql-schema-studio.git
149145
cd sql-schema-studio
150146

151147
# Build DEB package
152148
./scripts/packaging/build_deb.sh
153149

154-
# Install
150+
# Install the DEB package
155151
sudo dpkg -i sql-schema-studio_*.deb
156152
sudo apt --fix-broken install
153+
154+
# After installation, install Python dependencies
155+
sql-schema-studio-install-deps
156+
```
157+
158+
### Post-Installation
159+
160+
After installing the RPM or DEB package, run the dependency installer to install required Python packages:
161+
162+
```bash
163+
sql-schema-studio-install-deps
157164
```
158165

166+
This will guide you through installing the required Python dependencies (polars, scikit-learn, faker, kbcstorage, etc.) using your preferred method (pipx, pip --user, or system-wide).
167+
159168
### Run
160169

161170
```bash
@@ -170,35 +179,37 @@ sql-schema-studio
170179
sudo apt update
171180
sudo apt install -y python3-psycopg2 python3-gi python3-gi-cairo \
172181
python3-sqlparse python3-keyring python3-numpy \
173-
python3-scikit-learn python3-matplotlib python3-cairo python3-paramiko \
174-
gir1.2-gtk-4.0 gir1.2-gtksource-5 libvte-2.91-gtk4-dev \
182+
python3-matplotlib python3-cairo python3-paramiko \
183+
gir1.2-gtk-4.0 gir1.2-gtksource-5 gir1.2-vte-2.91 \
175184
pipx
176185

177-
pip install polars --break-system-packages
178-
179-
# Install pipx if not already installed (it's in the apt command above)
180-
# pipx ensures isolated Python package installations
181-
182-
# Install dependencies with pipx
183-
pipx install faker kbcstorage scikit-learn --include-deps
184-
185-
# Clone and install from source
186+
# Clone the repository
186187
git clone https://github.com/Peter-L-SVK/sql-schema-studio.git
187188
cd sql-schema-studio
188189

190+
# Install Python dependencies from pyproject.toml
191+
pipx install -e . --include-deps
192+
193+
# Or use pip (user-only)
194+
pip install --user -e .
195+
189196
# Run the application
190197
python3 -m src.main
191198
```
192199

193200
### Fedora / CentOS / RedHat
194201

195202
```bash
196-
sudo dnf install python3-gobject gtk4 gtksourceview5 libadwaita cairo python3-cairo \
197-
python3-paramiko python3-faker python3-kbcstorage vte291-gtk4
203+
sudo dnf install python3-gobject gtk4 gtksourceview5 libadwaita \
204+
cairo python3-cairo python3-paramiko vte291-gtk4 python3-pip
198205

199206
git clone https://github.com/Peter-L-SVK/sql-schema-studio.git
200207
cd sql-schema-studio
208+
209+
# Install Python dependencies from pyproject.toml
201210
pip install --user -e .
211+
212+
# Run the application
202213
python3 -m src.main
203214
```
204215

@@ -216,24 +227,19 @@ sudo apt update && sudo apt upgrade -y
216227
sudo apt install -y python3-psycopg2 python3-gi python3-gi-cairo \
217228
python3-sqlparse python3-keyring python3-numpy \
218229
python3-matplotlib python3-cairo python3-paramiko \
219-
gir1.2-gtk-4.0 gir1.2-gtksource-5 libvte-2.91-gtk4-dev \
230+
gir1.2-gtk-4.0 gir1.2-gtksource-5 gir1.2-vte-2.91 \
220231
pipx postgresql postgresql-client
221232

222-
pip install polars --break-system-packages
223-
224-
# Install pipx if not already installed
225-
# pipx ensures isolated Python package installations
233+
# Clone the repository
234+
git clone https://github.com/Peter-L-SVK/sql-schema-studio.git
235+
cd sql-schema-studio
226236

227-
# Install dependencies with pipx
228-
pipx install faker kbcstorage scikit-learn --include-deps
237+
# Install Python dependencies from pyproject.toml
238+
pipx install -e . --include-deps
229239

230240
# (Optional) Start local PostgreSQL
231241
sudo service postgresql start
232242

233-
# Clone and install from source
234-
git clone https://github.com/Peter-L-SVK/sql-schema-studio.git
235-
cd sql-schema-studio
236-
237243
# Run the application
238244
python3 -m src.main
239245

@@ -270,56 +276,62 @@ python3 -m mypy src/
270276
## Dependency Breakdown
271277

272278
### Required System Packages (installed via apt/dnf)
273-
These are essential for the application to run:
279+
These are essential for the application to run and are available in system repositories:
274280
- **PostgreSQL client**: `python3-psycopg2` - Database connectivity
275-
- **GTK4 GUI**: `python3-gi`, `python3-gi-cairo`, `gir1.2-gtk-4.0`, `gir1.2-gtksource-5` - GUI framework
281+
- **GTK4 GUI**: `python3-gi`, `python3-gi-cairo`, `gir1.2-gtk-4.0`, `gir1.2-gtksource-5`, `gir1.2-vte-2.91` - GUI framework
276282
- **SQL parsing**: `python3-sqlparse` - SQL parsing and formatting
277-
- **Data processing**: `polars`, `python3-numpy`, `python3-scikit-learn` - Analytics engine
283+
- **Data processing**: `python3-numpy` - Core data structures
278284
- **Visualization**: `python3-matplotlib`, `python3-cairo` - Charts and graphs
279285
- **SSH tunneling**: `python3-paramiko` - Secure remote connections
280-
- **Terminal**: `libvte-2.91-gtk4-dev` - Embedded terminal widget
281286
- **Keyring**: `python3-keyring` - Secure password storage
282287

283-
### Optional Packages (install via pipx or apt)
284-
These add extra features but aren't required:
285-
- **Synthetic data**: `faker` - Generate test data (pipx install faker)
286-
- **Keboola integration**: `kbcstorage` - Cloud data platform (pipx install kbcstorage)
288+
### Python Packages (installed via pip/pipx)
289+
These are installed by `sql-schema-studio-install-deps` or during development installation:
290+
- **Data processing**: `polars` - High-performance analytics engine
291+
- **Machine Learning**: `scikit-learn` - ML predictions for vacuum advisor
292+
- **Synthetic data**: `faker` - Generate test data
293+
- **Cloud integration**: `kbcstorage` - Keboola cloud platform
294+
- **Database driver**: `psycopg[binary]` - Modern PostgreSQL driver
295+
- **Data generation**: Additional packages as needed
287296

288297
## Installation Troubleshooting
289298

290299
### Common Issues
291300

292-
**1. "ModuleNotFoundError: No module named 'sklearn'"**
301+
**1. "Missing Python packages detected"**
293302
```bash
294-
# Fix: Install scikit-learn
295-
sudo apt install python3-scikit-learn # Debian/Ubuntu
296-
sudo dnf install python3-scikit-learn # Fedora
303+
# Run the dependency installer
304+
sql-schema-studio-install-deps
297305
```
298306

299-
**2. "ModuleNotFoundError: No module named 'faker'"**
307+
**2. "ModuleNotFoundError: No module named 'sklearn'"**
300308
```bash
301-
# Fix: Install faker via pipx
302-
pipx install faker
303-
# Or via apt (if available)
304-
sudo apt install python3-faker
309+
# Fix: Install scikit-learn via the dependency installer
310+
sql-schema-studio-install-deps
305311
```
306312

307-
**3. "ModuleNotFoundError: No module named 'kbcstorage'"**
313+
**3. "ModuleNotFoundError: No module named 'faker'"**
308314
```bash
309-
# Fix: Install kbcstorage via pipx
310-
pipx install kbcstorage
315+
# Fix: Install via the dependency installer
316+
sql-schema-studio-install-deps
311317
```
312318

313-
**4. GTK4 or GtkSourceView not found**
319+
**4. "ModuleNotFoundError: No module named 'polars'"**
320+
```bash
321+
# Fix: Install via the dependency installer
322+
sql-schema-studio-install-deps
323+
```
324+
325+
**5. GTK4 or GtkSourceView not found**
314326
```bash
315327
# Debian/Ubuntu
316-
sudo apt install gir1.2-gtk-4.0 gir1.2-gtksource-5 libvte-2.91-gtk4-dev
328+
sudo apt install gir1.2-gtk-4.0 gir1.2-gtksource-5 gir1.2-vte-2.91
317329

318330
# Fedora
319331
sudo dnf install gtk4 gtksourceview5 vte291-gtk4
320332
```
321333

322-
**5. WSL2 GUI not working**
334+
**6. WSL2 GUI not working**
323335
```bash
324336
# Ensure WSLg is enabled (Windows 11)
325337
wsl --update
@@ -329,6 +341,17 @@ wsl --update
329341
export DISPLAY=$(ip route list default | awk '{print $3}'):0
330342
```
331343

344+
**7. pipx not found**
345+
```bash
346+
# Debian/Ubuntu
347+
sudo apt install pipx
348+
pipx ensurepath
349+
350+
# Fedora
351+
sudo dnf install pipx
352+
pipx ensurepath
353+
```
354+
332355
## Architecture
333356

334357
```
@@ -366,5 +389,3 @@ Also discussion on repo is allowed.
366389
## License
367390

368391
GNU General Public License v3 or later. See [LICENSE](LICENSE).
369-
370-
---

0 commit comments

Comments
 (0)