Improve semantic evidence extraction and benchmark

Author: oceanusXXD

README.md

@@ -49,11 +49,11 @@ Use it when a Python project needs coding assistants to follow the current modul
 
 | Method | Gold evidence recall |
 |---|---:|
-| Path-only baseline | 0.048 |
-| AST symbols baseline | 0.357 |
+| Path-only baseline | 0.044 |
+| AST symbols baseline | 0.356 |
 | code2skill semantic scanner | 1.000 |
 
-The gold set covers route decorators, service calls, type references, data-flow edges, dynamic imports, raised exceptions, main guards, and internal dependency edges. Reproduce it with:
+The gold set covers route decorators, service calls, type references, data-flow edges, dynamic imports, re-exported symbol dependencies, raised exceptions, main guards, and internal dependency edges. Reproduce it with:
 
 ```bash
 python benchmarks/evaluate_structural_evidence.py
@@ -179,8 +179,8 @@ The default artifact directory is `.code2skill/`.
 | Path | Purpose |
 |---|---|
 | `adoption-guide.md` | Repository-specific adoption checklist and next workflow |
-| `project-summary.md` | Human-readable repository summary |
-| `skill-blueprint.json` | Structural repository blueprint |
+| `project-summary.md` | Human-readable repository summary with evidence coverage and import graph signals |
+| `skill-blueprint.json` | Structural repository blueprint with evidence counts and dependency graph stats |
 | `skill-plan.json` | LLM-planned Skill inventory |
 | `references/*.md` | Architecture, style, workflow, and API references |
 | `skills/index.md` | Generated Skill index |

README.zh-CN.md

@@ -49,11 +49,11 @@
 
 | 方法 | Gold evidence recall |
 |---|---:|
-| Path-only baseline | 0.048 |
-| AST symbols baseline | 0.357 |
+| Path-only baseline | 0.044 |
+| AST symbols baseline | 0.356 |
 | code2skill semantic scanner | 1.000 |
 
-Gold set 覆盖 route decorators、service calls、type references、data-flow edges、dynamic imports、raised exceptions、main guards 和 internal dependency edges。复现命令：
+Gold set 覆盖 route decorators、service calls、type references、data-flow edges、dynamic imports、re-exported symbol dependencies、raised exceptions、main guards 和 internal dependency edges。复现命令：
 
 ```bash
 python benchmarks/evaluate_structural_evidence.py
@@ -179,8 +179,8 @@ code2skill scan .
 | 路径 | 用途 |
 |---|---|
 | `adoption-guide.md` | 仓库级采用 checklist 和下一步工作流 |
-| `project-summary.md` | 面向人的仓库概要 |
-| `skill-blueprint.json` | 结构化仓库蓝图 |
+| `project-summary.md` | 面向人的仓库概要，包含 evidence coverage 和 import graph 信号 |
+| `skill-blueprint.json` | 结构化仓库蓝图，包含证据计数和依赖图统计 |
 | `skill-plan.json` | 模型规划出的 Skill 清单 |
 | `references/*.md` | 架构、风格、工作流和 API 参考 |
 | `skills/index.md` | 生成的 Skill 索引 |

benchmarks/evaluate_structural_evidence.py

@@ -83,7 +83,7 @@ def run_benchmark() -> dict[str, object]:
         "baselines": [
             "path-only: file path and suffix role inference",
             "ast-symbols: standard AST imports, classes, functions, and methods",
-            "code2skill-semantic: AST symbols plus routes, calls, type references, data-flow edges, dynamic imports, raised exceptions, and internal dependency resolution",
+            "code2skill-semantic: AST symbols plus routes, calls, type references, data-flow edges, dynamic imports, raised exceptions, re-exported symbols, and internal dependency resolution",
         ],
         "scope": (
             "This benchmark measures repository evidence extraction before any LLM call. "
@@ -236,7 +236,7 @@ def render_svg(report: dict[str, object]) -> str:
             f'<text x="{left + bar_width + 10:.1f}" y="{y + 23}" font-family="Arial, sans-serif" font-size="13" fill="#111827">{value:.3f} ({result["gold_hits"]}/{result["gold_total"]})</text>'
         )
     lines.append(
-        '<text x="36" y="314" font-family="Arial, sans-serif" font-size="12" fill="#6b7280">code2skill captures routes, calls, type references, data-flow, dynamic imports, exceptions, and internal dependency edges.</text>'
+        '<text x="36" y="314" font-family="Arial, sans-serif" font-size="12" fill="#6b7280">code2skill captures routes, calls, type references, data-flow, dynamic imports, re-exported symbols, exceptions, and dependency edges.</text>'
     )
     lines.append("</svg>")
     return "\n".join(lines)
@@ -246,7 +246,8 @@ def gold_facts() -> list[str]:
     return [
         "role:src/shop/main.py:entrypoint",
         "role:src/shop/api/users.py:route",
-        "import:src/shop/api/users.py:shop.core.ops",
+        "import:src/shop/__init__.py:shop.core.ops",
+        "import:src/shop/api/users.py:shop",
         "class:src/shop/core/ops.py:UserService",
         "class:src/shop/domain/accounts.py:AccountCreate",
         "class:src/shop/domain/accounts.py:AccountRecord",
@@ -259,6 +260,8 @@ def gold_facts() -> list[str]:
         "type:src/shop/core/ops.py:AccountRecord",
         "model:src/shop/domain/accounts.py:AccountCreate",
         "model:src/shop/domain/accounts.py:AccountRecord",
+        "dependency:src/shop/__init__.py->src/shop/core/ops.py",
+        "dependency:src/shop/api/users.py->src/shop/__init__.py",
         "dependency:src/shop/api/users.py->src/shop/core/ops.py",
         "dependency:src/shop/api/users.py->src/shop/domain/accounts.py",
         "dependency:src/shop/core/ops.py->src/shop/domain/accounts.py",
@@ -291,6 +294,9 @@ def gold_facts() -> list[str]:
 
 def write_fixture_repo(repo_path: Path) -> None:
     files = {
+        "src/shop/__init__.py": """
+            from shop.core.ops import UserService
+        """,
         "src/shop/main.py": """
             from fastapi import FastAPI
             from shop.api.users import router
@@ -300,7 +306,7 @@ def write_fixture_repo(repo_path: Path) -> None:
         """,
         "src/shop/api/users.py": """
             from fastapi import APIRouter
-            from shop.core.ops import UserService
+            from shop import UserService
             from shop.domain.accounts import AccountCreate
 
             router = APIRouter()

benchmarks/results/structural-evidence-benchmark.json

@@ -1,14 +1,14 @@
 {
   "name": "structural-evidence-extraction",
-  "generated_at": "2026-06-06T15:49:31.523360+00:00",
-  "gold_total": 42,
+  "generated_at": "2026-06-06T16:13:09.683771+00:00",
+  "gold_total": 45,
   "results": [
     {
       "method": "path-only",
-      "facts_found": 10,
+      "facts_found": 11,
       "gold_hits": 2,
-      "gold_total": 42,
-      "recall": 0.047619047619047616,
+      "gold_total": 45,
+      "recall": 0.044444444444444446,
       "hit_facts": [
         "role:src/shop/api/users.py:route",
         "role:src/shop/main.py:entrypoint"
@@ -31,6 +31,8 @@
         "data_flow:src/tool/runner.py:run:state<-load_state",
         "dependency:src/app/bootstrap.py->src/app/runtime/loader.py",
         "dependency:src/app/runtime/loader.py->src/app/plugins/audit.py",
+        "dependency:src/shop/__init__.py->src/shop/core/ops.py",
+        "dependency:src/shop/api/users.py->src/shop/__init__.py",
         "dependency:src/shop/api/users.py->src/shop/core/ops.py",
         "dependency:src/shop/api/users.py->src/shop/domain/accounts.py",
         "dependency:src/shop/core/ops.py->src/shop/domain/accounts.py",
@@ -43,7 +45,8 @@
         "function:src/tool/runner.py:run",
         "function:src/tool/state_store.py:load_state",
         "import:src/app/runtime/loader.py:importlib",
-        "import:src/shop/api/users.py:shop.core.ops",
+        "import:src/shop/__init__.py:shop.core.ops",
+        "import:src/shop/api/users.py:shop",
         "main_guard:src/tool/runner.py",
         "method:src/app/plugins/audit.py:AuditPlugin.record",
         "method:src/shop/core/ops.py:UserService.create",
@@ -58,10 +61,10 @@
     },
     {
       "method": "ast-symbols",
-      "facts_found": 25,
-      "gold_hits": 15,
-      "gold_total": 42,
-      "recall": 0.35714285714285715,
+      "facts_found": 26,
+      "gold_hits": 16,
+      "gold_total": 45,
+      "recall": 0.35555555555555557,
       "hit_facts": [
         "class:src/app/plugins/audit.py:AuditPlugin",
         "class:src/shop/core/ops.py:UserService",
@@ -74,7 +77,8 @@
         "function:src/tool/runner.py:run",
         "function:src/tool/state_store.py:load_state",
         "import:src/app/runtime/loader.py:importlib",
-        "import:src/shop/api/users.py:shop.core.ops",
+        "import:src/shop/__init__.py:shop.core.ops",
+        "import:src/shop/api/users.py:shop",
         "method:src/app/plugins/audit.py:AuditPlugin.record",
         "method:src/shop/core/ops.py:UserService.create",
         "method:src/tool/actions.py:ReleaseService.publish"
@@ -92,6 +96,8 @@
         "data_flow:src/tool/runner.py:run:state<-load_state",
         "dependency:src/app/bootstrap.py->src/app/runtime/loader.py",
         "dependency:src/app/runtime/loader.py->src/app/plugins/audit.py",
+        "dependency:src/shop/__init__.py->src/shop/core/ops.py",
+        "dependency:src/shop/api/users.py->src/shop/__init__.py",
         "dependency:src/shop/api/users.py->src/shop/core/ops.py",
         "dependency:src/shop/api/users.py->src/shop/domain/accounts.py",
         "dependency:src/shop/core/ops.py->src/shop/domain/accounts.py",
@@ -111,9 +117,9 @@
     },
     {
       "method": "code2skill-semantic",
-      "facts_found": 85,
-      "gold_hits": 42,
-      "gold_total": 42,
+      "facts_found": 91,
+      "gold_hits": 45,
+      "gold_total": 45,
       "recall": 1.0,
       "hit_facts": [
         "call:src/app/bootstrap.py:load_plugin",
@@ -133,6 +139,8 @@
         "data_flow:src/tool/runner.py:run:state<-load_state",
         "dependency:src/app/bootstrap.py->src/app/runtime/loader.py",
         "dependency:src/app/runtime/loader.py->src/app/plugins/audit.py",
+        "dependency:src/shop/__init__.py->src/shop/core/ops.py",
+        "dependency:src/shop/api/users.py->src/shop/__init__.py",
         "dependency:src/shop/api/users.py->src/shop/core/ops.py",
         "dependency:src/shop/api/users.py->src/shop/domain/accounts.py",
         "dependency:src/shop/core/ops.py->src/shop/domain/accounts.py",
@@ -145,7 +153,8 @@
         "function:src/tool/runner.py:run",
         "function:src/tool/state_store.py:load_state",
         "import:src/app/runtime/loader.py:importlib",
-        "import:src/shop/api/users.py:shop.core.ops",
+        "import:src/shop/__init__.py:shop.core.ops",
+        "import:src/shop/api/users.py:shop",
         "main_guard:src/tool/runner.py",
         "method:src/app/plugins/audit.py:AuditPlugin.record",
         "method:src/shop/core/ops.py:UserService.create",
@@ -165,7 +174,7 @@
   "baselines": [
     "path-only: file path and suffix role inference",
     "ast-symbols: standard AST imports, classes, functions, and methods",
-    "code2skill-semantic: AST symbols plus routes, calls, type references, data-flow edges, dynamic imports, raised exceptions, and internal dependency resolution"
+    "code2skill-semantic: AST symbols plus routes, calls, type references, data-flow edges, dynamic imports, raised exceptions, re-exported symbols, and internal dependency resolution"
   ],
   "scope": "This benchmark measures repository evidence extraction before any LLM call. It does not claim end-to-end SWE-bench issue resolution."
 }

docs/algorithm-notes.md

@@ -23,9 +23,16 @@ evidence for planning and Skill generation.
 - Import graph construction uses detailed `ImportInfo`, including `from ...
   import ...` names and dynamic imports, so package-level imports resolve to
   concrete internal files when possible.
+- Symbol-aware dependency resolution maps extracted call targets, instantiated
+  classes, type references, decorators, and raised exceptions back to internal
+  files when they match an imported alias, a package re-export, or a unique
+  repository symbol.
 - File priority combines path heuristics with content evidence. Route, service,
   model, main-guard, call-target, type-reference, and data-flow signals can
   raise selection priority.
+- Evidence coverage is summarized in the blueprint and project summary so users
+  can see how many source files, symbols, routes, calls, types, flows, dynamic
+  imports, exceptions, and dependency edges were captured.
 - Planner prompts receive dependency, call, type, and flow evidence for core
   modules. Generation prompts use the same skeleton lines when large files are
   summarized instead of inlined.
@@ -36,7 +43,8 @@ The extractor is deliberately conservative. It records shallow data-flow edges
 from assignments, loops, and context managers, but it does not attempt full
 interprocedural static analysis, control-flow reconstruction, type inference, or
 runtime import evaluation. Missing or ambiguous evidence should still be marked
-as uncertain by generated Skills.
+as uncertain by generated Skills. Plain symbol references are linked only when
+the symbol is unique in the repository or tied to an import alias/re-export.
 
 ## References
 

docs/assets/structural-evidence-benchmark.svg

@@ -13,6 +13,8 @@ to the context-selection and evidence-extraction layer that `code2skill` owns.
   https://arxiv.org/abs/2310.06770
 - RepoBench evaluates repository-level context for code completion:
   https://arxiv.org/abs/2306.03091
+- CrossCodeEval evaluates cross-file code completion across multiple languages:
+  https://arxiv.org/abs/2310.11248
 - CodeSearchNet evaluates semantic code search and retrieval quality:
   https://arxiv.org/abs/1909.09436
 
@@ -33,9 +35,10 @@ Outputs:
 - `benchmarks/results/structural-evidence-benchmark.json`
 - `docs/assets/structural-evidence-benchmark.svg`
 
-The fixture repository contains Python route, service, schema, dynamic plugin,
-runtime loader, main-guard, state, and exception-handling examples. The gold set
-contains 42 structural facts that are useful for writing grounded Skills:
+The fixture repository contains Python route, service, schema, package
+re-export, dynamic plugin, runtime loader, main-guard, state, and
+exception-handling examples. The gold set contains 45 structural facts that are
+useful for writing grounded Skills:
 
 - file roles
 - imports and internal dependency edges
@@ -44,6 +47,7 @@ contains 42 structural facts that are useful for writing grounded Skills:
 - service calls and call chains
 - type references
 - model/schema signals
+- re-exported symbol dependencies
 - dynamic imports
 - data-flow edges
 - raised exceptions
@@ -65,9 +69,9 @@ dependency edges.
 
 | Method | Gold hits | Gold total | Recall |
 |---|---:|---:|---:|
-| path-only | 2 | 42 | 0.048 |
-| ast-symbols | 15 | 42 | 0.357 |
-| code2skill-semantic | 42 | 42 | 1.000 |
+| path-only | 2 | 45 | 0.044 |
+| ast-symbols | 16 | 45 | 0.356 |
+| code2skill-semantic | 45 | 45 | 1.000 |
 
 ![Structural evidence benchmark](assets/structural-evidence-benchmark.svg)
 

docs/benchmarks.md

@@ -32,6 +32,10 @@ Expected files include:
 - `.code2skill/report.json`
 - `.code2skill/state/analysis-state.json`
 
+Open `.code2skill/project-summary.md` first. Its evidence coverage section shows
+whether the scan captured routes, calls, type references, data-flow edges,
+dynamic imports, exceptions, and internal dependency edges before any LLM call.
+
 ## 3. Preview Cost And Impact
 
 ```bash

docs/getting-started.md

@@ -34,8 +34,8 @@ This bundle separates generated Skill files from intermediate artifacts used for
 ### Review And Diagnostic Artifacts
 
 - `adoption-guide.md`: repository-specific adoption checklist and recommended next workflow.
-- `project-summary.md`: human-readable repository overview.
-- `skill-blueprint.json`: structural analysis output from Phase 1.
+- `project-summary.md`: human-readable repository overview, including evidence coverage and import graph signals.
+- `skill-blueprint.json`: structural analysis output from Phase 1, including extracted evidence counts and dependency graph stats.
 - `skill-plan.json`: planned Skill inventory.
 - `references/*.md`: supporting architecture, style, workflow, and API references.
 - `report.json`: execution metrics, mode decisions, cost estimates, affected files, affected Skills, and artifact lists.