Skip to content

Commit ea5d3c2

Browse files
committed
feat(crossrepo): bounded cross-repo base-lib symbol linking (depth-1, capped, default-off)
Add selective cross-repo base-library symbol linking so an unresolved callee (NOT in the local repo index) can pull the called base-lib function DEFINITION in as the deepest dependency chunk, tagged dep_source='crosslib:<repo>'. (1) scripts/crossrepo/build_global_symbol_index.py — stream-extracts selected base-lib subtrees from the corpus tarball in ONE pass and indexes PUBLIC function+type DEFINITIONS via libclang into a resumable SQLite store. A1 (boost/abseil/folly/openssl/boringssl/protobuf/eigen/fmt/glib) full public API; A2 (std/libc) public-symbol-only with reserved-name + trivial-body gating. Fail-loud: missing tarball / libclang load / bad subtree / 0-extract RAISE; per-file 3rd-party header parse errors are counted+skipped. (2) index_project.collect_transitive_deps — when --global-symbol-index is set, an unresolved callee is looked up in the global store and recorded as a depth-1 cross-lib pull under HARD BOUNDS (CrossLinkBudget: cap 12 deps/doc, 6144 tok/doc); pulled defs are LEAVES (their base-lib-internal callees are never followed). build_training_documents emits them as the deepest dep chunks (6-tuple PartInfo carrying dep_source), resolves root->base-lib call_edges to the pulled chunk, and flows through the SAME dep_levels/topo/ materialize/pack path. DEFAULT off -> behavior byte-identical when absent. (3) --global-symbol-index threaded through streaming_conveyor.py and streaming_reindex.py (CLI -> fail-loud check -> process_one_repo -> index_project), all trailing-default params (running conveyor unaffected). Tests: tests/test_crossrepo_global_symbols.py (5/5) — real SQLite store round-trip, public-symbol gating, budget bounds, off=unchanged, on=tagged chunk + resolved root->crosslib call_edge. Gitignore the 116 MB build store.
1 parent 7743f86 commit ea5d3c2

6 files changed

Lines changed: 1290 additions & 23 deletions

File tree

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,3 +81,7 @@ outputs/packed/1024/build_summary.json
8181
outputs/packed/1024/build.log
8282
outputs/verification_report/_summary.json
8383
outputs/verification_report/_samples/_manifest.json
84+
outputs/crossrepo/global_symbols.sqlite
85+
outputs/crossrepo/global_symbols.sqlite-shm
86+
outputs/crossrepo/global_symbols.sqlite-wal
87+
outputs/reindexed_commits/*

0 commit comments

Comments
 (0)