Skip to content

Commit 9fe1765

Browse files
author
miranov25
committed
docs: Update tests_README.md with comprehensive test suite documentation
- Document all 44 test files (42 Python + 2 C++) - Organize by 7 categories: Core, RDF, C++, dfdraw, Lazy, Performance, Infrastructure - Add test invariants, dependencies, CI/CD guidance - Document known xfail tests - Add dependency levels table (GPT suggestion) - Add dfdraw-specific test commands (GPT suggestion)
1 parent 2d29916 commit 9fe1765

1 file changed

Lines changed: 247 additions & 44 deletions

File tree

‎UTILS/dfextensions/AliasDataFrame/tests/tests_README.md‎

Lines changed: 247 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -1,67 +1,270 @@
1-
# AliasDataFrameTree Tests
1+
# AliasDataFrame Test Suite
2+
3+
## Overview
4+
5+
**Total Test Files**: 44 (42 Python + 2 C++)
6+
**Total Tests**: ~1190+
7+
**Last Updated**: 2025-12-11 (Phase 6.8e/c)
28

39
## Running Tests
410

511
```bash
6-
# Run all tests
7-
./run_tests.sh
12+
# Run all Python tests
13+
pytest tests/ -v
14+
15+
# Run with parallel workers
16+
pytest tests/ -n 12
17+
18+
# Run specific test file
19+
pytest tests/test_lazy_loading.py -v
820

9-
# Run with verbose output
10-
./run_tests.sh -v
21+
# Run specific test class/method
22+
pytest tests/test_lazy_loading.py::TestLazyTreeReader::test_load_single_branch -v
1123

12-
# Run specific test
13-
./run_tests.sh composite_index
14-
./run_tests.sh AliasDataFrameTree
24+
# Run C++ tests
25+
cd tests && ./run_tests.sh
26+
27+
# Run only dfdraw-related tests (useful when dfdraw is optional)
28+
pytest tests -k "draw" -v
29+
30+
# Run tests excluding dfdraw (when dfdraw not installed)
31+
pytest tests --ignore=tests/test_draw_invariance.py --ignore=tests/test_draw_chain_integration.py --ignore=tests/test_draw_lazy_integration.py -v
1532
```
1633

17-
## Test Files
34+
---
35+
36+
## Test Files by Category
37+
38+
### Core Functionality (Phase 1-4)
39+
40+
| File | Tests | Phase | Description |
41+
|------|-------|-------|-------------|
42+
| `test_alias_dataframe.py` | ~80 | 1-2 | Core AliasDataFrame functionality |
43+
| `test_alias_subframe.py` | 22 | 3A | Subframe LEFT JOIN, missing keys |
44+
| `test_data_schema.py` | ~15 | 4 | Schema structure validation |
45+
| `test_alias_data_frame_schema.py` | ~50 | 4a | Unified schema as source of truth |
46+
| `test_alias_data_frame_schema_v2.py` | ~30 | 4a | Schema v2 features |
47+
| `test_schema_serialization.py` | ~55 | 4b | JSON-safe schema serialization |
48+
| `test_schema_export_v2.py` | ~20 | 4b | Export with unified schema |
49+
| `test_schema_definition_vs_record.py` | ~10 | 4 | Schema definition validation |
50+
| `test_compression_pytest.py` | 25 | 2 | Compression monitoring |
51+
52+
### RDataFrame Integration (Phase 5)
53+
54+
| File | Tests | Phase | Description |
55+
|------|-------|-------|-------------|
56+
| `test_AliasDataFrameRDF.py` | ~150 | 5 | RDataFrame code generation |
57+
| `test_rdf_integration.py` | ~30 | 5 | RDF integration with ROOT |
58+
| `test_rdf_integration_final.py` | ~20 | 5 | Final RDF integration tests |
59+
| `test_rdf_real_data.py` | ~15 | 5 | RDF tests with real physics data |
60+
| `test_composite_keys.py` | ~40 | 5 | Multi-key composite index |
61+
| `test_backend_cpp_rvec.py` | ~25 | 5 | RVec indexing operations |
62+
| `test_dependency_tree.py` | ~15 | 5 | Alias dependency resolution |
63+
64+
### C++ Tests (Phase 3)
65+
66+
| File | Tests | Phase | Description |
67+
|------|-------|-------|-------------|
68+
| `test_composite_index.C` | 10 | 3 | N-key composite index (ROOT C++) |
69+
| `test_AliasDataFrameTree.C` | 8 | 3A | TTree::Draw compatibility (ROOT C++) |
70+
71+
### dfdraw Integration (Phase 6)
72+
73+
| File | Tests | Phase | Description |
74+
|------|-------|-------|-------------|
75+
| `test_draw_invariance.py` | 17 | 6.8e | Draw with known mathematical invariants |
76+
| `test_draw_chain_integration.py` | 26 | 6.8c | Draw across all Phase 7 combinations |
77+
| `test_draw_lazy_integration.py` | 22 | 6/7.3 | Draw with lazy loading |
78+
| `test_ttree_draw_subframe.py` | 10 | 3A | TTree::Draw with friend trees |
79+
80+
### Lazy Loading (Phase 7)
81+
82+
| File | Tests | Phase | Description |
83+
|------|-------|-------|-------------|
84+
| `test_lazy_loading.py` | 35 | 7.1 | LazyTreeReader foundation |
85+
| `test_branch_detection.py` | 51 | 7.2 | Branch auto-detection from expressions |
86+
| `test_chain_loading.py` | 51 | 7.4 | LazyChainReader multi-file |
87+
| `test_lazy_subframes.py` | 52 | 7.5 | Lazy subframe loading |
88+
89+
### Performance Optimization (Phase 8-9)
90+
91+
| File | Tests | Phase | Description |
92+
|------|-------|-------|-------------|
93+
| `test_numba_acceleration.py` | ~30 | 8 | Numba JIT-compiled scatter |
94+
| `test_arrow_compute.py` | ~25 | 9 | PyArrow compute integration |
95+
| `test_arrow_expression.py` | ~20 | 9 | Arrow expression evaluation |
96+
| `test_arrow_scatter.py` | ~25 | 9 | Arrow scatter operations |
97+
98+
### Infrastructure & Utilities
99+
100+
| File | Tests | Phase | Description |
101+
|------|-------|-------|-------------|
102+
| `test_proxy_pattern.py` | 35 | - | DataFrame-like access (`adf['x']`) |
103+
| `test_batch_materialization.py` | ~20 | 2 | Batch alias materialization |
104+
| `test_cycle_detection.py` | ~15 | - | Circular alias detection |
105+
| `test_self_referential_cycles.py` | ~10 | - | Self-referential alias handling |
106+
| `test_fill_handling.py` | ~20 | 3A | NaN fill for missing keys |
107+
| `test_join_caching.py` | ~25 | 8 | Join index caching |
108+
| `test_join_index_caching.py` | ~15 | 8 | Index lookup optimization |
109+
| `test_materialize_subframe_index.py` | ~15 | 5 | Subframe index materialization |
110+
| `test_subframe_alias_api.py` | ~20 | 3 | Subframe alias API |
111+
| `test_profiling.py` | ~10 | 3 | cProfile integration |
112+
| `test_clean_temporary.py` | ~10 | - | Temporary file cleanup |
113+
114+
### Configuration Files
18115

19116
| File | Description |
20117
|------|-------------|
21-
| `test_composite_index.C` | N-key composite index tests (Phase 3) |
22-
| `test_AliasDataFrameTree.C` | Core functionality tests |
23-
24-
## Composite Index Tests (10 tests)
25-
26-
| Test | Description | Type |
27-
|------|-------------|------|
28-
| 1. 3-Key Semantic | Join values match formula | Correctness |
29-
| 2. 4-Key Sparse | Real orbit values work | Infrastructure |
30-
| 3. Merge Safety | hadd doesn't break index | I/O |
31-
| 4. Float Detection | Non-integer rejected | Error handling |
32-
| 5. Uniqueness | All combinations correct | Correctness |
33-
| 6. Multiple Subframes | Different indices work | Critical |
34-
| 7. Index Assertion | GetTreeIndex() exists | Structural |
35-
| 8. Empty Subframe | 0-entry subframe | Edge case |
36-
| 9. No Matches | Non-overlapping keys | Edge case |
37-
| 10. Duplicate Keys | Consistent behavior | Edge case |
38-
39-
## Expected Output
118+
| `conftest.py` | Pytest fixtures and configuration |
119+
| `__init__.py` | Package initialization |
120+
| `run_tests.sh` | Shell script for C++ tests |
40121

122+
---
123+
124+
## Test Categories Summary
125+
126+
| Category | Files | Tests | Phases |
127+
|----------|-------|-------|--------|
128+
| Core Functionality | 9 | ~285 | 1-4 |
129+
| RDataFrame Integration | 6 | ~295 | 5 |
130+
| C++ Tests | 2 | 18 | 3 |
131+
| dfdraw Integration | 4 | 75 | 6, 7.3 |
132+
| Lazy Loading | 4 | 189 | 7 |
133+
| Performance Optimization | 4 | ~100 | 8-9 |
134+
| Infrastructure | 11 | ~195 | Various |
135+
| **Total** | **40** | **~1157** | |
136+
137+
---
138+
139+
## Key Test Invariants
140+
141+
Tests use deterministic relationships for exact verification:
142+
143+
```python
144+
# In generate_synthetic_data.py
145+
y_derived = 2 * x # Exact, no noise
146+
gain[sec] = 1.0 + 0.01 * sec # SectorCalib subframe
147+
gain[run,sec] = 1.0 + 0.01*sec + 0.001*(run-1000) # Calibration chain
41148
```
42-
====================================================
43-
Composite Index Tests (with semantic verification)
44-
====================================================
45-
...
46-
====================================================
47-
Results: 10/10 tests passed
48-
Assertions: 41 passed, 0 failed
49-
====================================================
50-
ALL TESTS PASSED!
51-
```
149+
150+
---
151+
152+
## C++ Test Details
153+
154+
### Composite Index Tests (`test_composite_index.C`) - 10 tests
155+
156+
| # | Test | Type | Description |
157+
|---|------|------|-------------|
158+
| 1 | 3-Key Semantic | Correctness | Join values match formula |
159+
| 2 | 4-Key Sparse | Infrastructure | Real orbit values work |
160+
| 3 | Merge Safety | I/O | hadd doesn't break index |
161+
| 4 | Float Detection | Error handling | Non-integer rejected |
162+
| 5 | Uniqueness | Correctness | All combinations correct |
163+
| 6 | Multiple Subframes | Critical | Different indices work |
164+
| 7 | Index Assertion | Structural | GetTreeIndex() exists |
165+
| 8 | Empty Subframe | Edge case | 0-entry subframe |
166+
| 9 | No Matches | Edge case | Non-overlapping keys |
167+
| 10 | Duplicate Keys | Edge case | Consistent behavior |
168+
169+
### AliasDataFrameTree Tests (`test_AliasDataFrameTree.C`) - 8 tests
170+
171+
Validates TTree::Draw compatibility from C++ side with:
172+
- Friend tree attachment
173+
- Dot notation access
174+
- Expression evaluation
175+
- Cut application
176+
177+
---
178+
179+
## Known Issues
180+
181+
### xfail Tests (Expected Failures)
182+
183+
| Test | File | Reason |
184+
|------|------|--------|
185+
| `test_draw_with_subframe` | `test_draw_chain_integration.py` | BUG-2025-12-11: draw() passes subframe names to ensure_branches(). Fix in Phase 6.8a |
186+
187+
---
52188

53189
## Writing New Tests
54190

55-
Use the assertion macros:
191+
### Python Tests
192+
193+
```python
194+
import pytest
195+
from AliasDataFrame import AliasDataFrame
196+
197+
class TestMyFeature:
198+
def test_basic_functionality(self, tmp_path):
199+
"""Test description."""
200+
# Setup
201+
df = pd.DataFrame({'x': [1, 2, 3]})
202+
adf = AliasDataFrame(df)
203+
204+
# Action
205+
adf.add_alias('y', 'x * 2')
206+
207+
# Assert
208+
assert 'y' in adf.aliases
209+
np.testing.assert_array_equal(adf['y'].values, [2, 4, 6])
210+
```
211+
212+
### C++ Tests
56213

57214
```cpp
215+
// Use assertion macros
58216
ASSERT_TRUE(condition, "message");
59217
ASSERT_EQ(actual, expected, "message");
60218
ASSERT_NEAR(actual, expected, tolerance, "message");
219+
220+
// Tests should:
221+
// 1. Create test ROOT file with known data
222+
// 2. Load via LoadADFTree()
223+
// 3. Verify results with assertions
224+
// 4. Clean up temporary files
225+
```
226+
227+
---
228+
229+
## Test Dependencies
230+
231+
### Dependency Levels
232+
233+
| Dependency | Type | Needed For | Notes |
234+
|------------|------|------------|-------|
235+
| `pytest` | required | All Python tests | Core test runner |
236+
| `pytest-xdist` | optional | Parallel execution | `-n auto` flag |
237+
| `numpy`, `pandas` | required | All tests | Core data structures |
238+
| `uproot` | required | ROOT file tests | Python ROOT I/O |
239+
| `ROOT` | optional | C++ tests, RDF tests | Heavy; tests skip if missing |
240+
| `dfdraw` | optional | Draw integration tests | Often missing in CI; tests skip |
241+
| `numba` | optional | Numba acceleration tests | Tests skip if missing |
242+
| `pyarrow` | optional | Arrow compute tests | Tests skip if missing |
243+
244+
### Installation
245+
246+
```bash
247+
# Install required test dependencies
248+
pip install pytest pytest-xdist numpy pandas uproot
249+
250+
# Optional dependencies (tests skip gracefully if missing)
251+
pip install numba pyarrow
252+
253+
# dfdraw: install from local path or package
254+
# ROOT: install via conda or system package manager
61255
```
62256

63-
Tests should:
64-
1. Create test ROOT file with known data
65-
2. Load via `LoadADFTree()`
66-
3. Verify results with assertions
67-
4. Clean up temporary files
257+
---
258+
259+
## CI/CD Integration
260+
261+
```bash
262+
# Full test suite (recommended for CI)
263+
pytest tests/ -n auto --tb=short
264+
265+
# Quick smoke test
266+
pytest tests/test_alias_dataframe.py tests/test_lazy_loading.py -v
267+
268+
# With coverage
269+
pytest tests/ --cov=AliasDataFrame --cov-report=html
270+
```

0 commit comments

Comments
 (0)