Interim sweep #64 data (123 cells): at budget 8000, mean across 3 test sets —
| depth |
recall |
precision |
F1 |
| L1 |
0.9231 |
0.2544 |
0.3562 |
| L2 (current default) |
0.9267 |
0.2128 |
0.2974 |
| L3 |
0.9282 |
0.1925 |
0.2720 |
| L4 |
0.9290 |
0.1913 |
0.2698 |
L1 gives up ~0.4pt recall for +6pt precision / +6pt F1 vs the current default depth 2. Same ordering holds at 16k/32k.
To do (no new runs needed — checkpoints from run #64 suffice):
- paired bootstrap CI on per-instance recall/F1 deltas L1 vs L2 (
benchmarks/stats.py), per test set;
- if L1's F1 win is significant and the recall loss CI excludes practically-relevant regressions, change
MODE.ego_depth_extended default 2 -> 1;
- otherwise document why 2 stays.
Note: depends on the aggregate tables being depth-correct — the falsy-zero get("depth") or -1 mislabeling of L0 rows was fixed in 2cf87d2.
Interim sweep #64 data (123 cells): at budget 8000, mean across 3 test sets —
L1 gives up ~0.4pt recall for +6pt precision / +6pt F1 vs the current default depth 2. Same ordering holds at 16k/32k.
To do (no new runs needed — checkpoints from run #64 suffice):
benchmarks/stats.py), per test set;MODE.ego_depth_extendeddefault 2 -> 1;Note: depends on the aggregate tables being depth-correct — the falsy-zero
get("depth") or -1mislabeling of L0 rows was fixed in 2cf87d2.