fix(io-boundaries): preserve leaf-caller rollups in CLI filter pass (WI-rubir)

cmd_io_boundaries reconstructs BoundaryMapEntry whenever
`primitive_filter or exclude_tests` is true. Since exclude_tests=True
is the default (WI-sifif), every normal CLI invocation hit this path
and silently dropped the WI-darad rollups (leaf_callers /
entry_points_per_leaf) by omitting them from the dataclass
construction. The bakeoff cohort-001/iter-010 artifact for alertmanager
/ kafka / prometheus showed chain_count > 0 with leaf_callers=[] for
every boundary.

Extract compute_leaf_rollups() as a public io_boundary helper; have the
CLI filter path lazily build the reverse graph and recompute rollups
for the surviving chain subset. Three regression tests cover the
default exclude_tests=True path, the entry_points_per_leaf
serialization, and the --primitive filter path.

Signed-off-by: jgstern-agent <josh-agent@iterabloom.com>

Author: jgstern-agent

.ci/affected-tests.txt

@@ -1,22 +1,72 @@
 # Test selection manifest
-# Generated by smart-test at 2026-04-25T17:31:03-04:00
+# Generated by smart-test at 2026-04-25T18:29:01-04:00
 # Mode: targeted
-# Baseline: 6e8145cf3768c4673ab6c98fdcdb334b05fe3415
-# Changed files: 6
+# Baseline: 26d1236f23e7200fa351a1c875f9356c4bb3241e
+# Changed files: 7
 # Changed source files: 2
-# Selected tests: 10
+# Selected tests: 60
 #
 # === CHANGED_SOURCE_FILES ===
-packages/hypergumbo-tracker/src/hypergumbo_tracker/cli.py
-packages/hypergumbo-tracker/src/hypergumbo_tracker/validation.py
+packages/hypergumbo-core/src/hypergumbo_core/cli.py
+packages/hypergumbo-core/src/hypergumbo_core/io_boundary.py
 # === SELECTED_TESTS ===
-packages/hypergumbo-tracker/tests/test_cli.py
-packages/hypergumbo-tracker/tests/test_configure.py
-packages/hypergumbo-tracker/tests/test_fork_workflow.py
-packages/hypergumbo-tracker/tests/test_migration.py
-packages/hypergumbo-tracker/tests/test_serve.py
-packages/hypergumbo-tracker/tests/test_setup.py
-packages/hypergumbo-tracker/tests/test_store.py
-packages/hypergumbo-tracker/tests/test_sync.py
-packages/hypergumbo-tracker/tests/test_tui.py
-packages/hypergumbo-tracker/tests/test_validation.py
+packages/hypergumbo-core/tests/test_backend_cli_flag.py
+packages/hypergumbo-core/tests/test_build_grammars.py
+packages/hypergumbo-core/tests/test_cli_basic.py
+packages/hypergumbo-core/tests/test_cli_cache.py
+packages/hypergumbo-core/tests/test_cli_commands.py
+packages/hypergumbo-core/tests/test_cli_config.py
+packages/hypergumbo-core/tests/test_cli_dead_code.py
+packages/hypergumbo-core/tests/test_cli_explain.py
+packages/hypergumbo-core/tests/test_cli_io_boundaries.py
+packages/hypergumbo-core/tests/test_cli_relativize_paths.py
+packages/hypergumbo-core/tests/test_cli_routes.py
+packages/hypergumbo-core/tests/test_cli_run_behavior_map.py
+packages/hypergumbo-core/tests/test_cli_search.py
+packages/hypergumbo-core/tests/test_cli_symbols.py
+packages/hypergumbo-core/tests/test_cli_test_coverage.py
+packages/hypergumbo-core/tests/test_cli_verify_claims.py
+packages/hypergumbo-core/tests/test_file_excludes.py
+packages/hypergumbo-core/tests/test_frameworks_flag.py
+packages/hypergumbo-core/tests/test_gitleaks.py
+packages/hypergumbo-core/tests/test_handler_slices.py
+packages/hypergumbo-core/tests/test_install_extras.py
+packages/hypergumbo-core/tests/test_io_boundary.py
+packages/hypergumbo-core/tests/test_locale.py
+packages/hypergumbo-core/tests/test_max_tier.py
+packages/hypergumbo-core/tests/test_no_first_party_priority.py
+packages/hypergumbo-core/tests/test_profile.py
+packages/hypergumbo-core/tests/test_pyffi_linker.py
+packages/hypergumbo-core/tests/test_rubyffi_linker.py
+packages/hypergumbo-core/tests/test_run_behavior_map.py
+packages/hypergumbo-core/tests/test_rust_analyzer_install.py
+packages/hypergumbo-core/tests/test_schema_compliance.py
+packages/hypergumbo-core/tests/test_sketch.py
+packages/hypergumbo-core/tests/test_sketch_sanity.py
+packages/hypergumbo-core/tests/test_slice_tier_filter.py
+packages/hypergumbo-core/tests/test_stable_shape_ids.py
+packages/hypergumbo-core/tests/test_supply_chain.py
+packages/hypergumbo-core/tests/test_taint.py
+packages/hypergumbo-core/tests/test_verify_claims.py
+packages/hypergumbo-lang-common/tests/BRANCHES_test_dart.py
+packages/hypergumbo-lang-common/tests/BRANCHES_test_elixir.py
+packages/hypergumbo-lang-common/tests/test_haskell.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_cpp.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_c.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_csharp.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_go.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_java.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_js_ts.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_kotlin.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_php.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_python_ast_analysis.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_ruby.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_rust.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_scala.py
+packages/hypergumbo-lang-mainstream/tests/BRANCHES_test_swift.py
+packages/hypergumbo-lang-mainstream/tests/test_c.py
+packages/hypergumbo-lang-mainstream/tests/test_html_analysis.py
+packages/hypergumbo-lang-mainstream/tests/test_polyglot_call_site_coverage.py
+packages/hypergumbo-lang-mainstream/tests/test_py_deps.py
+packages/hypergumbo-lang-mainstream/tests/test_python_ast_analysis.py
+packages/hypergumbo-lang-mainstream/tests/test_rust.py

CHANGELOG.md

@@ -44,6 +44,7 @@ This changelog tracks the **tool version** (package releases). The **schema vers
 
 ### Fixed
 
+- **`io-boundaries` CLI dropped WI-darad leaf-caller roll-ups in the filter pass** (WI-rubir): `cmd_io_boundaries` reconstructs `BoundaryMapEntry` whenever `primitive_filter or exclude_tests` is true, and since `exclude_tests=True` is the default (WI-sifif) every normal CLI invocation hit this path. The reconstruction set `chains` / `entry_points` / `primitives_used` but omitted `leaf_callers` and `entry_points_per_leaf` (they default to empty), so the bakeoff io-boundaries.txt artifact for cohort-001/iter-010 (alertmanager, kafka, prometheus) showed `chain_count > 0` with `leaf_callers=[]` for every boundary — the WI-darad fix had landed in `compute_boundary_map` but was silently stripped before serialization. The leaf-rollup loop is now a public module helper `compute_leaf_rollups(chains, reverse_graph, entrypoint_ids)` exported from `io_boundary.py`; the CLI filter path lazily builds the reverse graph (`_build_reverse_graph`) and recomputes the rollups for the surviving chain subset. Tagged `awaits_bakeoff_validation`; the next DEEP cohort that re-runs alertmanager/kafka/prometheus should observe non-empty `leaf_callers` per net_send/fs_read/fs_write boundary, validating WI-darad's original metric movement claim.
 - **TOML `[project.scripts]` `defines_target` dst is a well-formed Python id; path moved to `meta.target_path`** (INV-nodij): the previous emitter at `toml_config.py:_extract_pyproject_scripts` set `dst="hypergumbo_core/cli.py"` (bare path, no language prefix or colons). `_parse_dangling_id`'s `len(parts)<5` fallback (`ir.py:701-706`) then stuffed the whole path into the `language` slot of the synthesized boundary node, producing 2 orphan `external_symbol` nodes per pyproject script entry on hypergumbo self-analysis with `language='hypergumbo_core/cli.py'` and `language='hypergumbo_tracker/cli.py'`. Same shape as the WI-diruj Markdown link bug, different producer. The dst is now `f"python:{module_dotted}:0-0:{target_func}:unresolved"` — a properly-formed 5-part Python id that synthesizes a clean boundary `Symbol` (`language="python"`, `kind="external_symbol"`, with `canonical_name="python:hypergumbo_core.cli:main:unresolved"`); the WI-zujip manifest awareness then auto-promotes hypergumbo_core entries to tier-2. The build-target linker (`linkers/build_target.py`) reads `target_path` from `edge.meta` first, falling back to `edge.dst` for the 8 other `defines_target` producers (xml_config, json_config, manifest_targets) which keep their existing path-as-dst contract. Self-analysis: 2 → 0 path-shaped-language orphan nodes. New regression test `test_pyproject_scripts_dst_is_well_formed_id` asserts the 5-part id shape on every pyproject script `defines_target` edge.
 - **`parse_python_dependencies` walks monorepo `packages/<pkg>/pyproject.toml`** (WI-zujip, follow-up to WI-nunuj): the shipped parser only inspected `repo_root/pyproject.toml`, which silently missed monorepo layouts where the root file is shared-tool-configuration only and actual `[project].dependencies` live in per-package files. Self-analysis dogfood on hypergumbo (which IS such a monorepo) confirmed the symptom: `parse_python_dependencies(Path("."))` returned a manifest with `entries=[]`, the wire-up at `py.py:3307` set `dependency_manifest=None`, and all 1,443 python `external_symbol` boundary nodes stayed tier-3 (including `tree_sitter`, `protobuf`, `rich`, `pyyaml`, `pygments`, `sentence_transformers` — every one of which IS declared in some `packages/*/pyproject.toml`). Replaced with `_find_pyproject_files(repo_root)` that walks the tree (skipping `DEFAULT_EXCLUDES` directories and dot-prefixed dirs so a `.venv/site-packages/<some-pkg>/pyproject.toml` cannot smuggle a fake dep), and a `parse_python_dependencies` that unions dist names across every discovered file. Single-pyproject layouts collapse to the previous behaviour. After the fix, hypergumbo self-analysis reports 90 declared imports and `tree_sitter` / `rich` / `pygments` / `yaml` / `sentence_transformers` / `pytest` / `jsonschema` / `requests` all classify tier-2. Direct parallel of WI-davan E1, which fixed the same monorepo gap on the file-discovery side via `_detect_source_roots`. Tagged `awaits_bakeoff_validation`.
 - **Orchestrator synthesises real `kind="file"` Symbols for every `make_file_id`-shape edge endpoint; ranking suppression keeps them out of top-N** (WI-ramuv, Plan B follow-up to WI-fozoh / PR #3355): a new `synthesize_file_symbols_for_dangling_edges` helper runs once at the analyzer-result chokepoint in `analyze.all_analyzers.run_all_analyzers` (after edge dedup, before `ir.create_boundary_nodes`). Any edge whose `src` or `dst` matches `{lang}:{path}:1-1:file:file` shape and has no producer-side Symbol gets one synthesised on the spot — `kind="file"`, `name=path`, `language=lang`, `origin="orchestrator_file_symbol_synthesis"`. The 20+ analyzers that previously emitted `make_file_id`-style import-edge srcs without a matching Symbol (Python, C, C++, Perl, Bash, Proto, Thrift, plus ~13 in lang-common / lang-extended1) all benefit from one orchestrator change, with no per-analyzer churn. Dart-style colon-bearing path slots (`dart:dart:io:1-1:file:file` → path `dart:io`) are preserved by stripping the fixed suffix before the language split. Co-required deliverable to keep ranking stable: a new `apply_file_kind_weights` dampener in `ranking.py` (wired into `rank_symbols` after `apply_generated_code_weights`) zeros centrality for every `kind="file"` Symbol whose language is NOT in the (currently empty) `_FILE_KIND_RANKING_ALLOWED_LANGUAGES` allow-list. Inverse of WI-gafog's CSS-variable precedent: default is "exclude from ranking", a future language whose file Symbols ARE meaningful as ranking entries (e.g. HTML pages on a static-site analyser) opts back in by joining the set. The Symbols stay in the symbol list (slice traversal, containment, per-file metrics still see them) — only their ranking weight is zeroed. Schema 0.2.3 → 0.2.4 (additive): `"file"` added to `Symbol.kind` enum. Self-analysis dogfood: canonical `python:<external>:0-0:file:file` boundary Symbol drops 1 → 0; imports edge count rises 2,136 → 9,252 with full per-file src attribution; top-30 sketch ranking contains zero `kind="file"` Symbols and is preserved within ±3 positions (TreeSitterAnalyzer, find_files, Symbol, Span, register_analyzer, …); 739 first-party file Symbols enter the graph. Tagged `awaits_bakeoff_validation`; the cohort that confirms it should also revisit WI-fozoh's "37% external-symbol drop" tag, since the denominator changes when these 739 Symbols stop counting as boundary externals.

packages/hypergumbo-core/src/hypergumbo_core/cli.py

@@ -3205,7 +3205,25 @@ class _Edge:
     # CLI users do the same via --include-tests.
     exclude_tests = getattr(args, "exclude_tests", True)
 
-    from .io_boundary import BoundaryMapEntry
+    from .io_boundary import (
+        BoundaryMapEntry,
+        _build_reverse_graph,
+        compute_leaf_rollups,
+    )
+
+    # WI-rubir: when the filter path rebuilds BoundaryMapEntry it must
+    # also recompute the WI-darad leaf-caller roll-ups for the surviving
+    # chain subset; otherwise leaf_callers / entry_points_per_leaf are
+    # silently dropped (they default to empty on the dataclass), and
+    # because exclude_tests=True is the default, every normal CLI
+    # invocation hits this path.
+    reverse_graph_for_filter: Optional[Dict[str, set[str]]] = None
+
+    def _ensure_reverse_graph() -> Dict[str, set[str]]:
+        nonlocal reverse_graph_for_filter
+        if reverse_graph_for_filter is None:
+            reverse_graph_for_filter = _build_reverse_graph(edges)
+        return reverse_graph_for_filter
 
     filtered_entries: Dict[str, BoundaryMapEntry] = {}
     for btype, entry in bmap.entries.items():
@@ -3231,11 +3249,18 @@ def _is_test_chain(chain: Any) -> bool:
             continue
 
         if primitive_filter or exclude_tests:
+            leaf_callers, entry_points_per_leaf = compute_leaf_rollups(
+                chains,
+                _ensure_reverse_graph(),
+                entrypoint_ids or None,
+            )
             filtered_entries[btype] = BoundaryMapEntry(
                 boundary=entry.boundary,
                 chains=chains,
                 entry_points=sorted({ep for c in chains for ep in c.entry_points}),
                 primitives_used=sorted({c.primitive for c in chains}),
+                leaf_callers=leaf_callers,
+                entry_points_per_leaf=entry_points_per_leaf,
             )
         else:
             filtered_entries[btype] = entry

packages/hypergumbo-core/src/hypergumbo_core/io_boundary.py

@@ -931,44 +931,67 @@ def compute_boundary_map(
             by_boundary["external_potential"] = ext_chains
 
     # Build boundary map entries, including WI-darad leaf-caller roll-ups.
-    # A "leaf caller" of an io_edge_src is an immediate caller of that src
-    # in the reverse graph; when src has no callers, src itself is its own
-    # leaf (the primitive is invoked directly from that function).
     bmap = BoundaryMap(total_io_edges=tagged_count)
     leaf_ep_cache: dict[str, set[str]] = {}
     for boundary, chains in by_boundary.items():
-        entry_points_set: set[str] = set()
-        primitives_set: set[str] = set()
-        leaf_set: set[str] = set()
-        per_leaf: dict[str, set[str]] = {}
-        for chain in chains:
-            primitives_set.add(chain.primitive)
-            for ep in chain.entry_points:
-                entry_points_set.add(ep)
-            callers = reverse_graph.get(chain.io_edge_src, set())
-            leaves = callers if callers else {chain.io_edge_src}
-            for leaf in leaves:
-                leaf_set.add(leaf)
-                if entrypoint_ids:
-                    if leaf not in leaf_ep_cache:
-                        leaf_ep_cache[leaf] = _reachable_entry_points(
-                            leaf, reverse_graph, entrypoint_ids
-                        )
-                    per_leaf.setdefault(leaf, set()).update(leaf_ep_cache[leaf])
+        leaf_callers, entry_points_per_leaf = compute_leaf_rollups(
+            chains, reverse_graph, entrypoint_ids, leaf_ep_cache,
+        )
         bmap.entries[boundary] = BoundaryMapEntry(
             boundary=boundary,
             chains=chains,
-            entry_points=sorted(entry_points_set),
-            primitives_used=sorted(primitives_set),
-            leaf_callers=sorted(leaf_set),
-            entry_points_per_leaf={
-                leaf: sorted(eps) for leaf, eps in per_leaf.items()
-            },
+            entry_points=sorted({ep for c in chains for ep in c.entry_points}),
+            primitives_used=sorted({c.primitive for c in chains}),
+            leaf_callers=leaf_callers,
+            entry_points_per_leaf=entry_points_per_leaf,
         )
 
     return bmap
 
 
+def compute_leaf_rollups(
+    chains: list[IoChain],
+    reverse_graph: dict[str, set[str]],
+    entrypoint_ids: Optional[set[str]] = None,
+    leaf_ep_cache: Optional[dict[str, set[str]]] = None,
+) -> tuple[list[str], dict[str, list[str]]]:
+    """Compute the WI-darad leaf-caller roll-ups for a chain set.
+
+    A "leaf caller" of an io_edge_src is an immediate caller of that src
+    in the reverse graph; when src has no callers, src itself is its own
+    leaf (the primitive is invoked directly from that function).
+
+    Exposed at module level so the CLI's filter pass (cmd_io_boundaries)
+    can recompute rollups for a subset of chains after dropping test-file
+    chains or applying --primitive — otherwise the BoundaryMapEntry it
+    rebuilds loses the rollups, and the bakeoff io-boundaries.txt shows
+    chain_count>0 with leaf_callers=[] (WI-rubir regression).
+
+    ``leaf_ep_cache`` lets callers reuse entry-point reachability sets
+    across multiple boundary types in the same map; pass ``None`` to use
+    a per-call scratch cache.
+    """
+    if leaf_ep_cache is None:
+        leaf_ep_cache = {}
+    leaf_set: set[str] = set()
+    per_leaf: dict[str, set[str]] = {}
+    for chain in chains:
+        callers = reverse_graph.get(chain.io_edge_src, set())
+        leaves = callers if callers else {chain.io_edge_src}
+        for leaf in leaves:
+            leaf_set.add(leaf)
+            if entrypoint_ids:
+                if leaf not in leaf_ep_cache:
+                    leaf_ep_cache[leaf] = _reachable_entry_points(
+                        leaf, reverse_graph, entrypoint_ids
+                    )
+                per_leaf.setdefault(leaf, set()).update(leaf_ep_cache[leaf])
+    return (
+        sorted(leaf_set),
+        {leaf: sorted(eps) for leaf, eps in per_leaf.items()},
+    )
+
+
 # ---------------------------------------------------------------------------
 # Boundary-tagging pass (ADR-0016 Phase 1b)
 # ---------------------------------------------------------------------------