Commit 1fa5145
committed
Phase 13.19.ADF.FIX1 + Phase 13.20.ADF: vector draw diagnostic + export_tree performance
Two phases combined in one commit (both non-invasive, both verified).
=== Phase 13.19.ADF.FIX1 — vector draw() kwarg diagnostic (no source changes) ===
Diagnostic test suite for the vector draw() kwarg propagation bug.
Architect reproducer: aDF.draw('[y1..y6]:staveITS', group_by='mP3',
group_by_bins=6) produced 421 legend entries instead of 6.
K1 tests (boundary diagnostic, monkey-patch DFDraw methods):
K1_1 signature binding : PASS
K1_2 single-expr forwarding : PASS
K1_3 draw_batch forwarding : PASS
K1_4 draw_figures forwarding : SKIP (spec format)
K1_5 vector-expr forwarding : PASS
Conclusion: ADF forwards kwargs correctly. Bug was dfdraw-internal.
K2 tests (end-to-end output verification, post-dfdraw FIX1 fe007b7):
K2_1 scalar baseline : PASS
K2_2 vector+group_by legend bounded: PASS (gate)
K2_3 production reproducer mirror : PASS (architect repro)
K2_4 title not duplicated : PASS
Verified dfdraw FIX1 works end-to-end through ADF pipeline.
No ADF source changes needed for this bug.
=== Phase 13.20.ADF — export_tree metadata batching (AliasDataFrame.py) ===
Performance fix: export_tree with N subframes opened ROOT file N+1 times
for metadata writing. Each TFile.Open('UPDATE') on a 500MB+ file costs
~5-8s. With 15 subframes in production TPC calibration: ~80-130s overhead.
Fix: separate data-write (uproot, unchanged) from metadata-write (ROOT).
Batch all metadata into single TFile.Open/Close via new methods:
_write_all_data_to_uproot — recursive data write, no ROOT
_collect_metadata_targets — builds flat (adf, treename) list
_write_all_metadata_to_root — single TFile.Open for all trees
_write_metadata_to_tree — writes to already-open TFile
Also fixes latent nested-subframe crash: old code passed uproot file
object to ROOT.TFile.Open in recursive path (line 4800-4801).
E1 roundtrip tests (4/4 PASS): 0, 3, 15 subframes + timing.
E2 fix-specific tests (3/4 PASS, 1 xfail):
E2_1 single TFile.Open per export : PASS (core gate)
E2_2 nested subframe roundtrip : xfail (pre-existing read_tree
limitation — does not recursively load nested subframes;
export correctly writes them after this fix)
E2_3 backward compatibility : PASS
E2_4 standalone metadata write : PASS
Profile evidence: profile_gr11_tf0.prof, 159s/11% of 1452s total.
Expected savings: 80-130s on production files.
Cross-team review: Consolidated Performance Review item AliceO2Group#12.
Test count: 1510 passed, 7 failed, 1 error, 8 skipped.
Pre-existing failures unchanged (6F+1E from Phase 13.18 baseline).
Only new failure: E2_2 (xfail, pre-existing read_tree limitation).1 parent 1f3b45b commit 1fa5145
3 files changed
Lines changed: 657 additions & 15 deletions
File tree
- UTILS/dfextensions/AliasDataFrame
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4807 | 4807 | | |
4808 | 4808 | | |
4809 | 4809 | | |
4810 | | - | |
| 4810 | + | |
| 4811 | + | |
| 4812 | + | |
| 4813 | + | |
4811 | 4814 | | |
4812 | 4815 | | |
4813 | 4816 | | |
4814 | | - | |
4815 | | - | |
4816 | | - | |
| 4817 | + | |
| 4818 | + | |
| 4819 | + | |
| 4820 | + | |
| 4821 | + | |
4817 | 4822 | | |
4818 | | - | |
4819 | | - | |
4820 | | - | |
| 4823 | + | |
| 4824 | + | |
4821 | 4825 | | |
4822 | | - | |
| 4826 | + | |
| 4827 | + | |
4823 | 4828 | | |
4824 | 4829 | | |
4825 | 4830 | | |
4826 | 4831 | | |
4827 | | - | |
4828 | 4832 | | |
| 4833 | + | |
4829 | 4834 | | |
4830 | | - | |
| 4835 | + | |
| 4836 | + | |
| 4837 | + | |
| 4838 | + | |
| 4839 | + | |
| 4840 | + | |
| 4841 | + | |
| 4842 | + | |
| 4843 | + | |
| 4844 | + | |
| 4845 | + | |
| 4846 | + | |
| 4847 | + | |
| 4848 | + | |
| 4849 | + | |
| 4850 | + | |
| 4851 | + | |
| 4852 | + | |
| 4853 | + | |
| 4854 | + | |
| 4855 | + | |
| 4856 | + | |
| 4857 | + | |
| 4858 | + | |
| 4859 | + | |
| 4860 | + | |
| 4861 | + | |
| 4862 | + | |
| 4863 | + | |
| 4864 | + | |
4831 | 4865 | | |
4832 | 4866 | | |
4833 | 4867 | | |
4834 | | - | |
4835 | | - | |
| 4868 | + | |
| 4869 | + | |
| 4870 | + | |
| 4871 | + | |
| 4872 | + | |
| 4873 | + | |
| 4874 | + | |
| 4875 | + | |
| 4876 | + | |
| 4877 | + | |
| 4878 | + | |
| 4879 | + | |
| 4880 | + | |
| 4881 | + | |
| 4882 | + | |
| 4883 | + | |
| 4884 | + | |
| 4885 | + | |
| 4886 | + | |
4836 | 4887 | | |
4837 | 4888 | | |
4838 | 4889 | | |
4839 | | - | |
4840 | | - | |
| 4890 | + | |
| 4891 | + | |
| 4892 | + | |
| 4893 | + | |
| 4894 | + | |
| 4895 | + | |
| 4896 | + | |
| 4897 | + | |
| 4898 | + | |
4841 | 4899 | | |
4842 | 4900 | | |
4843 | 4901 | | |
| |||
4877 | 4935 | | |
4878 | 4936 | | |
4879 | 4937 | | |
4880 | | - | |
4881 | 4938 | | |
4882 | 4939 | | |
4883 | 4940 | | |
| |||
0 commit comments