Skip to content

Commit 9a13130

Browse files
committed
Phase 13.9.GB Extension: Parallel parity + capability matrix
Parallel parity (Option B — minimal parameter propagation): - agg_columns, agg_median propagated to _worker_v5 and _worker_v5_shared - fit_intercept passed to _assemble_results_v5 in all 3 paths - agg post-processing in workers (coord_to_bin, kernel weights) - forkserver/spawn context for Numba thread conflicts (256-core fix) - RuntimeError if all parallel units fail (was silent empty DataFrame) Capability matrix: 7 new features (133 total, 43 Verified, 0 Broken) SW.agg_columns, SW.wls_weights, SW.fit_intercept_false, SW.lean_output, SW.parallel_agg, SW.parallel_fit_intercept, SW.parallel_safety SW.parallel proof list updated to match actual test names Tests: 4 new parallel (12 total), 466 passed, 3 pre-existing failures Key invariance: parallel agg ≡ serial at rtol=1e-12
1 parent 917c361 commit 9a13130

5 files changed

Lines changed: 823 additions & 102 deletions

File tree

UTILS/dfextensions/groupby_regression/docs/CAPABILITY_MATRIX.md

Lines changed: 87 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,32 @@
11
# Capability Matrix — groupby_regression
22

3-
**Generated:** 2026-02-16 08:00 UTC
4-
**Phase:** 13.9.GB — GroupByRegressionEvaluator + Unified Metadata
3+
**Generated:** 2026-03-13 09:07 UTC
4+
**Phase:** Phase 13.11.GB
55
**Generator:** `scripts/generate_capability_matrix.py`
66

77
## Summary
88

99
| Status | Count | % |
1010
|--------|------:|--:|
11-
| ✅ Verified | 34 | 30.4% |
12-
| ☑️ Smoke-only | 77 | 68.8% |
11+
| ✅ Verified | 43 | 32.3% |
12+
| ☑️ Smoke-only | 89 | 66.9% |
1313
| 🧨 Broken | 0 | 0.0% |
1414
| ⚠️ Partial | 0 | 0.0% |
15-
| 📋 Planned | 1 | 0.9% |
16-
| **Total** | **112** | |
15+
| 📋 Planned | 1 | 0.8% |
16+
| **Total** | **133** | |
1717

1818
## Test Layer Distribution (excluding verbose duplicates)
1919

2020
| Layer | Count |
2121
|-------|------:|
22-
| invariance | 88 |
23-
| integration | 12 |
24-
| performance | 6 |
25-
| smoke | 211 |
26-
| validation | 24 |
27-
| **Total unique** | **341** |
22+
| invariance | 98 |
23+
| integration | 16 |
24+
| performance | 5 |
25+
| smoke | 265 |
26+
| validation | 28 |
27+
| **Total unique** | **412** |
28+
29+
⚠️ **UNCLASSIFIED tests: 27** — defaulted to smoke (fail-closed rule §3.5)
2830

2931
## groupby_regression
3032

@@ -49,16 +51,19 @@
4951

5052
| Status | Feature | Tests | Inv/Int | Bench | Tag |
5153
|--------|---------|------:|--------:|-------|-----|
52-
|| **EVAL.boundary** — Boundary handling (clamp/nan/extrapolate) | 7 | 7 | | |
53-
| ☑️ | **EVAL.construction** — Evaluator construction from dfGB | 7 | 0 | | |
54-
|| **EVAL.export_roundtrip** — JSON export/import roundtrip | 6 | 6 | | |
55-
|| **EVAL.ivar_weighting** — Inverse-variance weighted interpolation | 7 | 7 | | |
56-
| ☑️ | **EVAL.metadata_construction** — Evaluator construction from metadata | 7 | 0 | | |
57-
|| **EVAL.multi_target** — Multi-target evaluation | 3 | 3 | | |
58-
|| **EVAL.multilinear** — Multilinear interpolation | 8 | 8 | | |
59-
|| **EVAL.nearest** — Nearest-neighbor evaluation | 7 | 6 | | |
60-
|| **EVAL.sparse_grid** — Sparse grid handling (invalid bins) | 6 | 6 | | |
61-
|| **EVAL.sw_integration** — SW output integration | 2 | 2 | | |
54+
|| **EVAL.boundary** — Boundary handling invariance | 5 | 5 | | |
55+
|| **EVAL.construction** — Evaluator construction from dfGB | 16 | 2 | | |
56+
|| **EVAL.export_roundtrip** — Export/import roundtrip invariance | 6 | 6 | | |
57+
|| **EVAL.ivar_weighting** — Inverse-variance weighting invariance | 7 | 7 | | |
58+
| ☑️ | **EVAL.metadata_construction** — Evaluator construction from metadata dict | 7 | 0 | | |
59+
|| **EVAL.multi_target** — Multi-target evaluation invariance | 3 | 3 | | |
60+
|| **EVAL.multilinear** — Multilinear interpolation invariance | 16 | 16 | | |
61+
|| **EVAL.nearest** — Nearest-neighbor evaluation invariance | 4 | 4 | | |
62+
|| **EVAL.sparse_grid** — Sparse grid handling invariance | 4 | 4 | | |
63+
|| **EVAL.sw_integration** — Sliding window integration | 2 | 2 | | |
64+
|| **NLFIT.evaluator_option_a** — Evaluator Option A: interpolate params → evaluate model | 8 | 1 | | SCIPY |
65+
|| **NLFIT.evaluator_option_b** — Evaluator Option B: evaluate at corners → interpolate function values | 7 | 4 | | SCIPY |
66+
| ☑️ | **NLFIT.evaluator_params** — Evaluator parameter extraction | 1 | 0 | | SCIPY |
6267

6368
## groupby_regression_kernels
6469

@@ -77,6 +82,27 @@
7782
| ☑️ | **KERNEL.status_codes** — Status code encoding/decoding | 4 | 0 | | |
7883
| ☑️ | **KERNEL.streaming_memory** — Streaming RSS stability | 2 | 0 || NUMBA |
7984

85+
## groupby_regression_models
86+
87+
| Status | Feature | Tests | Inv/Int | Bench | Tag |
88+
|--------|---------|------:|--------:|-------|-----|
89+
|| **NLFIT.gaussian_plus_line** — Gaussian + linear background model | 2 | 2 | | SCIPY |
90+
| ☑️ | **NLFIT.model_registry** — Named model registry | 5 | 0 | | |
91+
| ☑️ | **NLFIT.p0_estimation** — Automatic initial parameter estimation | 2 | 0 | | SCIPY |
92+
93+
## groupby_regression_nonlinear
94+
95+
| Status | Feature | Tests | Inv/Int | Bench | Tag |
96+
|--------|---------|------:|--------:|-------|-----|
97+
| ☑️ | **NLFIT.callable_dispatch** — Custom callable fit dispatch | 5 | 0 | | |
98+
|| **NLFIT.cross_engine_polynomial** — Polynomial cross-engine invariance | 1 | 1 | | SCIPY |
99+
|| **NLFIT.gaussian_peak_recovery** — Gaussian peak parameter recovery | 1 | 1 | | SCIPY |
100+
|| **NLFIT.multi_target_independence** — Multi-target independence invariance | 1 | 1 | | |
101+
| ☑️ | **NLFIT.named_model_dispatch** — Named model fit dispatch via scipy.optimize.curve_fit | 3 | 0 | | SCIPY |
102+
|| **NLFIT.polynomial_exact** — Polynomial exact coefficient recovery | 1 | 1 | | SCIPY |
103+
| ☑️ | **NLFIT.sw_reuse** — Sliding window infrastructure reuse | 2 | 0 | | |
104+
|| **NLFIT.weights_sigma** — Weights→sigma mapping invariance | 1 | 1 | | SCIPY |
105+
80106
## groupby_regression_optimized
81107

82108
| Status | Feature | Tests | Inv/Int | Bench | Tag |
@@ -139,20 +165,26 @@
139165

140166
| Status | Feature | Tests | Inv/Int | Bench | Tag |
141167
|--------|---------|------:|--------:|-------|-----|
168+
| ☑️ | **SW.agg_columns** — Sliding window agg_columns (COG) | 8 | 0 | | |
142169
| ☑️ | **SW.aggregation** — Sliding window aggregation | 1 | 0 | | |
143170
|| **SW.backend_auto** — Backend auto-dispatch | 2 | 1 | | |
144171
| ☑️ | **SW.basic** — Sliding window basic 3D | 1 | 0 | | |
145172
| ☑️ | **SW.bin_helpers** — Internal bin helpers | 2 | 0 | | |
146173
| ☑️ | **SW.boundary** — Boundary truncation | 3 | 0 | | |
147174
| ☑️ | **SW.empty_window** — Empty window handling | 1 | 0 | | |
175+
| ☑️ | **SW.fit_intercept_false** — fit_intercept=False column handling | 2 | 0 | | |
148176
|| **SW.invariance** — Sliding window invariance tests | 8 | 8 | | |
177+
| ☑️ | **SW.lean_output** — Lean default output (no fit_column stats) | 2 | 0 | | |
149178
|| **SW.linear_fit** — Sliding window linear fit | 1 | 1 | | |
150179
| ☑️ | **SW.metadata** — Sliding window metadata | 1 | 0 | | |
151180
| ☑️ | **SW.min_stat** — Minimum entries enforcement | 1 | 0 | | |
152181
|| **SW.multi_predictor** — Sliding window multi-predictor | 2 | 1 | | |
153182
| ☑️ | **SW.multi_target** — Sliding window multi-target | 1 | 0 | | |
154183
| ☑️ | **SW.omitted_dims** — Omitted window dims default to 0 | 1 | 0 | | |
155-
|| **SW.parallel** — Parallel sliding window (split-column) | 8 | 3 || NUMBA |
184+
|| **SW.parallel** — Parallel sliding window (split-column) | 8 | 1 || NUMBA |
185+
| ☑️ | **SW.parallel_agg** — Parallel sliding window with agg_columns | 2 | 0 | | |
186+
| ☑️ | **SW.parallel_fit_intercept** — Parallel fit_intercept=False | 1 | 0 | | |
187+
| ☑️ | **SW.parallel_safety** — Parallel total-failure safety | 1 | 0 | | |
156188
| ☑️ | **SW.return_metadata** — Sliding window return_metadata | 1 | 0 | | |
157189
| ☑️ | **SW.selection** — Sliding window selection mask | 1 | 0 | | |
158190
| ☑️ | **SW.smoke_gate** — Realistic smoke normalised residuals | 1 | 0 | | |
@@ -161,6 +193,7 @@
161193
|| **SW.v5_dominance** — V5 algorithm dominance across all backends | 2 | 2 || NUMBA |
162194
| ☑️ | **SW.validation** — Sliding window input validation | 5 | 0 | | |
163195
| 📋 | **SW.weighted** — Sliding window weighted fits (WLS) | 0 | 0 | | PLANNED |
196+
| ☑️ | **SW.wls_weights** — WLS weights in sliding window | 5 | 0 | | |
164197

165198
## synthetic_tpc_distortion
166199

@@ -190,8 +223,38 @@
190223
| V5.performance | bench_v5.py::timing [MONITOR] | 📊 MONITOR |
191224
| XVAL.robust_v4_parity | bench_comparison.py::compute_agreement() [MONITOR] | 📊 MONITOR |
192225

226+
## ⚠️ UNCLASSIFIED Tests (fail-closed → smoke)
227+
228+
- `test_agg_columns.py::test_agg_columns_basic`
229+
- `test_agg_columns.py::test_agg_columns_matches_manual`
230+
- `test_agg_columns.py::test_agg_columns_median_optional`
231+
- `test_agg_columns.py::test_agg_columns_none_backward_compat`
232+
- `test_agg_columns.py::test_agg_columns_restores_fit_stats`
233+
- `test_agg_columns.py::test_agg_columns_restores_fit_stats`
234+
- `test_agg_columns.py::test_agg_columns_v5_matches_zerocopy`
235+
- `test_agg_columns.py::test_agg_columns_with_kernel_weights`
236+
- `test_agg_columns.py::test_default_no_fit_stats`
237+
- `test_agg_columns.py::test_default_no_fit_stats`
238+
- `test_parallel_sliding_window.py::TestParallelAggColumns::test_parallel_agg_matches_serial`
239+
- `test_parallel_sliding_window.py::TestParallelAggColumns::test_parallel_with_agg_columns`
240+
- `test_parallel_sliding_window.py::TestParallelErrorHandling::test_missing_split_column_raises`
241+
- `test_parallel_sliding_window.py::TestParallelErrorHandling::test_on_error_nan_continues`
242+
- `test_parallel_sliding_window.py::TestParallelFitIntercept::test_parallel_fit_intercept_false`
243+
- `test_parallel_sliding_window.py::TestParallelMissingUnits::test_missing_sectors`
244+
- `test_parallel_sliding_window.py::TestParallelPerformance::test_parallel_faster`
245+
- `test_parallel_sliding_window.py::TestParallelSafety::test_parallel_total_failure_raises`
246+
- `test_parallel_sliding_window.py::TestParallelSchema::test_output_has_split_columns`
247+
- `test_parallel_sliding_window.py::TestParallelSingleWorker::test_single_vs_multi_worker`
248+
- `test_wls_weights.py::test_fit_intercept_false_no_intercept_columns`
249+
- `test_wls_weights.py::test_fit_intercept_false_slope_correct`
250+
- `test_wls_weights.py::test_wls_all_paths_match`
251+
- `test_wls_weights.py::test_wls_changes_coefficients`
252+
- `test_wls_weights.py::test_wls_positive_intercept_ms`
253+
- `test_wls_weights.py::test_wls_recovers_known_slope`
254+
- `test_wls_weights.py::test_wls_uniform_weights_equals_ols`
255+
193256
---
194257

195-
*Two-tier verification per Phase 13.9.GB v02 proposal.*
258+
*Two-tier verification per Phase 13.11.GB v02 proposal.*
196259
*✅ = invariance/integration test exists. ☑️ = smoke tests only — does not catch numerical regressions.*
197260
*Verbose SW duplicates (1 files) deduplicated per §3.6.*

0 commit comments

Comments
 (0)