Skip to content

Commit 6b9fa8a

Browse files
author
miranov25
committed
docs: TECHNICAL_SUMMARY v1.5.2 — dematerialize + join caching (Phase 13.21 close)
§4.3: add dematerialize(drop=, keep=) — replaces drop_materialized. §2 Join Contract: add cache row (content-based, survives materialize). §12.1: 1521 tests, 125 invariance. Footer updated.
1 parent 6dc69aa commit 6b9fa8a

1 file changed

Lines changed: 27 additions & 12 deletions

File tree

UTILS/dfextensions/AliasDataFrame/docs/TECHNICAL_SUMMARY.md

Lines changed: 27 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,7 @@ adf.register_subframe(name, adf, index_columns)
162162
| Duplicate child keys | Parent rows expanded (many-to-one replication) |
163163
| Ordering | Parent row order preserved |
164164
| Pandas equivalent | `pd.merge(main, sub, on=keys, how='left', sort=False)` |
165+
| **Cache** | **Join indices cached across `materialize_aliases` calls; invalidated only on `register_subframe` or index-column change (Phase 13.21.ADF). O(1) content-based validation via first/last/dtype signature.** |
165166

166167
### Function Registration (Phase 13.9/13.10)
167168

@@ -460,6 +461,18 @@ class AliasDataFrame:
460461

461462
def describe_aliases(self, verbosity=0x07, pattern=None):
462463
"""Print alias definitions, dependencies, materialization status."""
464+
465+
def dematerialize(self, drop=None, keep=None) -> list:
466+
"""Drop materialized alias columns to reclaim memory.
467+
468+
Raw columns always protected. Aliases/subframes preserved —
469+
re-materialization via materialize_aliases() recovers values.
470+
Three modes: drop=[...], keep=[...], no args (drop all).
471+
Mutually exclusive drop/keep (ValueError if both).
472+
Composes with join index caching: re-materialization reuses
473+
cached join indices since index columns are unchanged.
474+
Phase 13.21.ADF. Replaces drop_materialized() (removed).
475+
"""
463476
```
464477

465478
### 4.4 Schema Management
@@ -1344,17 +1357,18 @@ Vision: automatic conversion of C++ SOA (Structure of Arrays) table definitions
13441357

13451358
## 12. Test Coverage
13461359

1347-
### 12.1 Current Status (2026-04-02)
1360+
### 12.1 Current Status (2026-04-19)
13481361

13491362
**Command:** `pytest tests/` (with ROOT enabled, Numba available, PyArrow available)
13501363

13511364
| Metric | Value |
13521365
|--------|-------|
1353-
| Collected | ~1455 |
1354-
| Passed | 1441 |
1355-
| Failed | 6 |
1356-
| Skipped | 7 |
1357-
| Runtime | ~27s (12 workers) |
1366+
| Collected | ~1537 |
1367+
| Passed | 1521 |
1368+
| Failed | 7 |
1369+
| Skipped | 8 |
1370+
| Runtime | ~26s (12 workers) |
1371+
| Invariance tests | 125 |
13581372

13591373
**Environment assumptions:** ROOT 6.x installed, `numba` available, `pyarrow` available, `dfdraw` available. Without ROOT, ~50 tests are skipped (ROOT-dependent integration tests).
13601374

@@ -1570,14 +1584,15 @@ from AliasDataFrameRDF import (setup_rdf_with_friends,
15701584
| v1.3 | 2026-02-19 | **Source-verified corrections** from 4-reviewer consolidated review (CLAUDE3, GPT, GPT3, source verification against 11,557-line AliasDataFrame.py). P0-1: `export_tree()` signature corrected. P0-2: `get_axis_unit()` removed. P1-1: `__getitem__` auto-materialization claim removed. P1-2: `__contains__` corrected. P1-3: schema key `axisLabel``title`. P1-4: `save_schema()` simplified. P2-1: `register_subframe(pre_index)` added. P2-2: `read_tree``@staticmethod`. |
15711585
| v1.4 | 2026-03-27 | **Phase 13.9/13.10 additions:** NEW §4.15 (register_function, register_polynomial_from_subframe, register_evaluator), NEW §6.5 (Registered Function System with PolynomialSpec architecture, evaluator schema contract). Updated: §1.1 feature list, §1.2 architecture diagram, §1.3 file structure (+PolynomialSpec.py), §1.4 metrics (1425 tests, 42× polynomial speedup), §2 contract snapshot, §4.3 add_alias fill_value, §12 test coverage, §13 bugs (draw_subframe_resolution ✅ fixed), §15.3 schema with registered_functions. |
15721586
| v1.5 | 2026-04-06 | **Phase 13.11/13.11.B + bug fixes.** Updated §1.2–1.4 (12,100 lines, 1441 tests, 41-feature matrix). NEW §12.6 Capability Matrix infrastructure (run_tests.sh, taxonomy-based generator, 41 features, 12 modules). Updated §12.2 (5 new test addition entries). Updated §13.1: fill_value P0 ✅, draw_lazy compound ✅, polynomial persistence ✅. Updated §13.2: CAPABILITY_MATRIX ✅, run_tests.sh ✅, PHASE_BEGIN ✅ resolved; serialization duplication added as tech debt. Updated §14 planned work. |
1587+
| v1.5.2 | 2026-04-19 | **Phase 13.21.ADF additions.** NEW in §4.3: `dematerialize(drop=, keep=)` — memory reclamation with raw-column protection; replaces removed `drop_materialized()`. Updated §2 Join Contract: join index cache row (content-based validation, survives `materialize_aliases`, invalidated on `register_subframe`). Updated §12.1: 1521 tests, 125 invariance. |
15731588

15741589
---
15751590

15761591
**END OF TECHNICAL SUMMARY**
15771592

1578-
**Document Version:** 1.5
1579-
**Phase:** 13.11.B (Capability Matrix taxonomy)
1580-
**Total Source Lines:** ~12,100 (AliasDataFrame.py) + 337 (PolynomialSpec.py) + ~800 (scripts/infrastructure)
1581-
**Total Test Files:** 48+
1582-
**Test Results:** 1441 passed, 6 failed, 7 skipped (2026-04-02)
1583-
**Capability Matrix:** 41 features, 9 verified, 64 invariance tests
1593+
**Document Version:** 1.5.2
1594+
**Phase:** 13.21.ADF (Join index caching + dematerialize)
1595+
**Total Source Lines:** ~12,862 (AliasDataFrame.py) + 337 (PolynomialSpec.py) + ~800 (scripts/infrastructure)
1596+
**Total Test Files:** 55+
1597+
**Test Results:** 1521 passed, 7 failed, 8 skipped (2026-04-19)
1598+
**Capability Matrix:** 44 features, 26 verified, 125 invariance tests

0 commit comments

Comments
 (0)