Commit ea5d3c2
committed
feat(crossrepo): bounded cross-repo base-lib symbol linking (depth-1, capped, default-off)
Add selective cross-repo base-library symbol linking so an unresolved
callee (NOT in the local repo index) can pull the called base-lib function
DEFINITION in as the deepest dependency chunk, tagged dep_source='crosslib:<repo>'.
(1) scripts/crossrepo/build_global_symbol_index.py — stream-extracts selected
base-lib subtrees from the corpus tarball in ONE pass and indexes PUBLIC
function+type DEFINITIONS via libclang into a resumable SQLite store. A1
(boost/abseil/folly/openssl/boringssl/protobuf/eigen/fmt/glib) full public
API; A2 (std/libc) public-symbol-only with reserved-name + trivial-body
gating. Fail-loud: missing tarball / libclang load / bad subtree / 0-extract
RAISE; per-file 3rd-party header parse errors are counted+skipped.
(2) index_project.collect_transitive_deps — when --global-symbol-index is set,
an unresolved callee is looked up in the global store and recorded as a
depth-1 cross-lib pull under HARD BOUNDS (CrossLinkBudget: cap 12 deps/doc,
6144 tok/doc); pulled defs are LEAVES (their base-lib-internal callees are
never followed). build_training_documents emits them as the deepest dep
chunks (6-tuple PartInfo carrying dep_source), resolves root->base-lib
call_edges to the pulled chunk, and flows through the SAME dep_levels/topo/
materialize/pack path. DEFAULT off -> behavior byte-identical when absent.
(3) --global-symbol-index threaded through streaming_conveyor.py and
streaming_reindex.py (CLI -> fail-loud check -> process_one_repo ->
index_project), all trailing-default params (running conveyor unaffected).
Tests: tests/test_crossrepo_global_symbols.py (5/5) — real SQLite store
round-trip, public-symbol gating, budget bounds, off=unchanged, on=tagged
chunk + resolved root->crosslib call_edge. Gitignore the 116 MB build store.1 parent 7743f86 commit ea5d3c2
6 files changed
Lines changed: 1290 additions & 23 deletions
File tree
- scripts
- crossrepo
- tests
- tools/clang_indexer
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
81 | 81 | | |
82 | 82 | | |
83 | 83 | | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
0 commit comments