feat(celery Wave 7 #7): MCP graph tools — query_graph_entities / expand_graph_subgraph / get_entity_detail#1759
Conversation
…nd_graph_subgraph / get_entity_detail Wave 7 §K.12.6 / §K.12.7 task #7 adds three MCP tools that let agents access the lineage graph through semantic vector search, n-hop subgraph expansion, and single-entity detail lookup. Each tool is a thin httpx wrapper around a new internal-only REST endpoint, mirroring the existing ``aperag/mcp/tools/search_graph.py`` pattern (architect ratify msg=da89237c locked Option A: 3 endpoint + 3 tool, both REST-mediated). What lands * Per-collection factory ``build_graph_search_service_for(collection)`` in ``aperag/indexing/graph_search_service.py``, mirroring ``_build_lineage_graph_store_for`` so MCP tool view handlers (#7) and the retrieval pipeline (#8) share one wiring path. * Three internal-only endpoints under ``/api/v2/collections/{id}/graphs/``: ``GET .../entities/search?q=&top_k=`` → ``GraphSearchService.search_entities``; ``POST .../subgraph/expand`` (body: entity_names + hops) → ``GraphSearchService.get_subgraph``; ``GET .../entities/{name}`` → ``LineageGraphStore.get_entity``. All read-only, all per-collection auth-scoped, all backward-compat to the user-facing curation/merge surface (Wave 7 §K.12.8 0-new- endpoint guard preserved — these endpoints are MCP-backend only, not surfaced to the frontend). * Three new ``GraphService`` methods (``search_entities`` / ``expand_subgraph`` / ``get_entity_detail``) projecting ``EntityWithLineage`` / ``RelationWithLineage`` into the public ``GraphSearchEntity`` / ``GraphRelationView`` view models, reusing ``GraphSearchService._render_description`` so MCP callers see the same compacted-or-fallback text as the retrieval pipeline. * Three MCP tools in ``aperag/mcp/tools/graph_tools.py`` registered via ``@mcp_server.tool`` and wired at the bottom of ``aperag/mcp/server.py`` (decorator-on-import pattern shared with the existing 8 D10 tools). Tests * ``tests/unit_test/mcp/test_graph_tools.py`` — 12 contract tests: registration / signature locks, httpx call shape (verb + URL + params + body + auth header), URL-encoding for unicode entity names, success / 404 / 5xx / missing-API-key error paths. * ``tests/unit_test/service/test_graph_search_service_layer.py`` — 9 service-layer tests including the EntityWithLineage projection rules (compacted preference, parts fallback, cross-lineage chunk dedup) and a byte-parity check between the MCP entity view and ``GraphSearchService.compose_context``. * ``tests/unit_test/mcp/test_capability_negotiation.py`` teardown reload list extended so the registry-snapshot test restores graph_tools annotations alongside the existing 4 search tools. 12-invariant cross-check (§K.12 invariants per architect msg=fcf580a6) * #4 vector store via ``VectorStoreConnectorAdaptor`` ✅ — factory resolves the connector via ``_build_collection_qdrant_connector``; no direct Qdrant import in the new tool / route surface. * #5 payload ``indexer="graph_entity"`` filter ✅ — inherited from ``GraphSearchService.search_entities``. * #11 candidate detection write-only / merge read-only ✅ — none of the three tools mutate state. * #12 grep-zero LightRAG ✅ — tool names + URL paths + view models are LightRAG-clean; existing borderline docstring references stay in task #10's sweep batch. 4-pattern pre-check matrix * Pattern 1 v1: legacy ``from aperag.domains.knowledge_graph.graphindex`` imports — unchanged in this PR (task #8 owns the cutover; task #10 the deletion). * Pattern 1 v2: surfaced an architect own-up — ``MCPServerEntry`` (D9 remote-server registry) is NOT the in-process tool registry the original spec referenced; the actual registration target is ``FastMCP`` via ``@mcp_server.tool``. Architect ratify msg=da89237c locked Option A. * Pattern 2: state binding — new endpoints reach the lineage tables exclusively through ``LineageGraphStore`` / ``GraphSearchService``; no direct backend table reference. * Pattern 3: registered Pydantic schemas appear in OpenAPI regen — frontend regen is a no-op for cuiwenbo task #9 (these endpoints are not consumed by the existing UI). simple-stable 4 guardrail * #1 不无限扩范围 — new endpoints exist solely to back the three spec-locked MCP tools; no new user-facing UI. * #2 尽快上线 — task #7 is unblocked of task #6 / task #8; this PR is mergeable independent of the curation-side work still in progress. * #3 简单稳定 — same httpx-based pattern as the existing 8 D10 tools; one factory function shared with task #8. * #4 私有化部署免维护 — ``API_BASE_URL`` env keeps the MCP server colocated-or-detached without code changes.
earayu
left a comment
There was a problem hiding this comment.
🟢 LGTM ✅ (huangheng pass-1, per spec §K.12.11) — GitHub 不允许同账号 approve;verdict = ready to merge
12-invariant + 4-pattern + simple-stable 4-guardrail 全 paste(hard-gate format 第八个达标 PR),3 MCP tools + 3 internal-only REST endpoints + per-collection factory + 21 unit tests + 1 capability negotiation extension。Option (A) httpx-based pattern 100% mirror 现有 8 个 MCP tool,read-only D-3 严守,naming grep-zero LightRAG (added lines 仅 invariant-meta refs)。
12-invariant cross-check(task #7 scope)
| # | Invariant | This PR | 验证依据 |
|---|---|---|---|
| 1 | L1 graph data 不污染 | n/a | read-only 服务 |
| 2 | L1 → L2 单向派生 | n/a | read-only |
| 3 | Compactor 顺序 | n/a | task #3 范畴 |
| 4 | Vector store via VectorStoreConnectorAdaptor |
✅ | build_graph_search_service_for factory 复用 _build_collection_qdrant_connector,GraphSearchService 内部已 abide |
| 5 | payload indexer="graph_entity" filter |
✅ | inherited from GraphSearchService.search_entities(PR #1756 已 verified) |
| 6 | uuid5 deterministic id | n/a | read-only |
| 7 | snapshot-diff | n/a | task #3 范畴 |
| 8 | alias_map persist | n/a | task #6 范畴 |
| 9 | upsert 透明 alias redirect | n/a | task #6 范畴(read-side alias resolution 见 minor observation 1) |
| 10 | DB 列长度 cap | ✅ | top_k 1-100 cap (route line 307-308),hops max 3 (per pydantic schema) |
| 11 | 候选检测仅写不自动合并 (D-3) | ✅ | 3 tools / 3 endpoints 全 read-only;零 mutation |
| 12 | 命名 grep-zero LightRAG | added lines gh pr diff 1759 | grep -E '^\+' | grep -i lightrag → 2 hits,都是 invariant-meta references (# Per §K.12 invariant #12 (grep-zero LightRAG) + tool names are LightRAG-grep-zero clean),不是 naming,task #10 sweep 一并清 |
Option (A) pattern conformity 验证
| 验证项 | 现有 8 tool | 本 PR | 状态 |
|---|---|---|---|
| httpx-based + REST mediated | ✅ all | ✅ all 3 tools | ✅ conform |
API_BASE_URL env env-aware |
✅ | ✅ line 47 import + 104/160/213 用 | ✅ conform |
@mcp_server.tool(annotations=_register_tool_annotation(...)) decorator |
✅ search_graph.py pattern | ✅ line 53/116/172 严格 mirror | ✅ conform |
_parse_response defensive 解析(success/404/non-2xx) |
✅ search_graph 内部 | ✅ line 224-244 同 convention | ✅ conform |
| Tool docstring "Use this when / Do not use this when" 段 | ✅ existing | ✅ all 3 tools | ✅ conform |
完整 mirror 现有 pattern,无 architecture asymmetry。👍
3 endpoint scope 验证
| Method | Path | Backed by | 范围 |
|---|---|---|---|
| GET | /api/v2/collections/{id}/graphs/entities/search?q=&top_k= |
graph_service.search_entities → GraphSearchService.search_entities |
internal-only / read-only |
| POST | /api/v2/collections/{id}/graphs/subgraph/expand body |
graph_service.expand_subgraph → GraphSearchService.get_subgraph → expand_neighbors_n_hops |
internal-only / read-only |
| GET | /api/v2/collections/{id}/graphs/entities/{name} |
graph_service.get_entity_detail → store.get_entity (direct,非 GraphSearchService — 见 observation 1) |
internal-only / read-only |
3 endpoint 全 per-collection auth-scoped (route line 299/326/356),零 user-facing UI 消费 (frontend backward-compat 不破坏 — cuiwenbo task #9 OpenAPI regen 是 no-op)。
4-pattern pre-check matrix
PR body Pattern 1 v2 (architect own-up) 显式标注 mini-pattern 5/9 第三例 — D9 MCPServerEntry URL registry 与 in-process tool registry 错位的 architect-side spec drift。Pattern 2 dependency 接口签名全对齐已 merged commits。Pattern 3 per-collection binding(factory 模式)与 retrieval pipeline 现有一致。
simple-stable 4-guardrail
PR description 4 项全显式:smallest surface for spec-locked 3 tools / ships independent / pattern reuse / API_BASE_URL env-aware deployment。
测试覆盖
21 new + 1 capability negotiation = 22 tests:
test_mcp/test_graph_tools.py(245 LOC): 3 tools happy path + 404 / non-2xx error path / params shapetest_service/test_graph_search_service_layer.py(268 LOC): service.py 3 method 完整覆盖test_mcp/test_capability_negotiation.py(+3): 3 new tools 加进 capability list
完备。
2 个 minor observation(非阻塞,sediment 给 future task)
Observation 1: get_entity_detail bypasses GraphSearchService + alias decorator
service.py:get_entity_detail (line 253-277) 用 _build_lineage_graph_store(...) 直接,没走 GraphSearchService,没走 LineageGraphStoreWithAliasRedirect decorator。
当前 OK:
- decorator (PR #1758) 仅 intercept
upsert_*(write 路径),不 redirect reads - 即使 task #8 wire 装饰器到
_build_lineage_graph_store,read pathget_entity也不 redirect - 所以
get_entity_detail("alias_name")返 None / 404
潜在 UX 问题:
- 如 user UI 显示 alias name 给 agent (e.g., 用户合并 "OpenAI" → "OpenAI Inc.";UI 仍可能显示历史 "OpenAI" alias)
- agent 通过 MCP
get_entity_detail("OpenAI")→ 404 - 用户体验:agent 找不到刚见过的实体
两条路(任选 future follow-up):
- (A)
get_entity_detailresolve alias → canonical 后再 lookup(AliasMapRepository.resolve_canonical(name)接store.get_entity(canonical)) - (B) UI 永远 surface canonical name 给 agent(前端责任 — alias 是用户合并意图,agent context 用 canonical)
建议:sediment 为 Wave 8 candidate W8-3 (read-side alias resolution at MCP / REST surface)。本 PR 可 merge,spec §K.12.7 没显式 mandate read-side resolution。
Observation 2: 2 added LightRAG meta-references
+# Per §K.12 invariant #12 (grep-zero LightRAG) the names are LightRAG-clean
+#12 the tool names are LightRAG-grep-zero clean.
都是 self-tracking comments 标注 "we comply with grep-zero rule"。strict 读取 invariant #12 仍算 hits;建议 task #10 sweep 一并清(或保留作 invariant compliance annotations,看 implementer)。
修完会 LGTM 的清单
实际上已经可以 merge ✅。observations 都是非阻塞 sediment 候选。
@chenyexuan work clean — Option (A) httpx pattern 严格 conform + tool docstring agent-guidance prose 完整("Use this when / Do not use this when")+ 22 tests 覆盖 happy path / error path / params shape 全。👍
task #8 wiring 再提醒(@chenyexuan task #8 PR 必交付)
worker_factory._build_lineage_graph_storewrap withLineageGraphStoreWithAliasRedirect(per PR #1758)- REST
/graphs/nodes/mergeroute swap toLineageEntityMerger.merge_entities(per PR #1758) - retrieval pipeline
_graph_searchcutover 走GraphSearchService(per PR #1756)
3 处 wiring 在 task #8 PR 一并交付,不做的话 PR #1758 inseparability gate + PR #1756 vector recall 都形同虚设。
@符炫炜 LGTM,可 merge。
@不穷 推进 task #7 → done after merge;critical path 余 #8 (含 3 处 wiring) + #9 + #10 = 3 PR 收尾 Wave 7。
@bryce task #10 close-out cleanup list 增量:
- 累积:task #1 (8 处 Wave 6 era 注释) / task #4 (line 249 fallback) / task #5 (6 处 docstring) / task #3 (relation payload naming + search_relations 升级 to vector recall — Wave 8 W8-1) / task #6 (无累积,全 align)
- 本轮 task #7: 2 处 invariant-meta LightRAG references in
mcp/tools/graph_tools.py+service.py,sweep 时改成 neutral 表述 e.g. "grep-zero invariant compliance"
…+ post-#1762 lint sweep) Task #8 PR #1762 merged 2026-04-28 (commit 08d9d3b). Updates to this scaffold + a small repo-wide lint sweep that pre-commit caught. This PR's substantive change (single test file): * ``_TASK8_WIRING_LANDED`` flipped True; module-level wiring skip removed. The file now gates only on ``RUN_W7_E2E_NARRATIVE=1`` (Layer 2 env gate) so local-dev pytest stays fast while CI Wave 7 lane can run bodies. * Per-step skip messages rewritten as concrete API pointers — each step names the exact merged surface its body must call (REST path, service method, repository class), so the body-fill follow-up can grep straight to the right spot. Bodies still pending implementation — they need a running stack (Postgres + Qdrant + Redis + ES + LLM provider keys). PR remains draft until bodies are filled + Layer 2 has run green at least once. Bundled lint cleanup (ruff format only, zero behavior change): * aperag/mcp/server.py — import block re-sort (post #1759 squash ordering drift). * aperag/domains/knowledge_graph/service.py * aperag/graph_curation/lineage_merge.py * tests/unit_test/mcp/test_graph_tools.py * tests/unit_test/service/test_graph_search_service_layer.py The four ``aperag/`` + ``tests/unit_test/`` files above all came in via the PR #1762 squash merge moments ago and trip ``ruff format --check``. Bundling here so this PR can land cleanly through pre-commit; the alternative is a separate "format-only" PR for code that just merged, which is more PR overhead than value.
…dies pending task #8 wiring (#1760) * test(w7-task#11): scaffold Wave 7 e2e narrative — 9-step skeleton, bodies pending task #8 wiring Per architect ratify msg=a0ba75da + huangheng CR rationale msg=0b48af2b, task #11 is the integration safety net for the three pieces of wiring that task #8 introduces: 1. ``LineageGraphStoreWithAliasRedirect`` decorator wrap in ``worker_factory._build_lineage_graph_store``. 2. REST ``POST /collections/{cid}/graphs/nodes/merge`` cutover from legacy ``GraphIndexService.merge_entities`` to the new ``GraphCurationService.merge_entities``. 3. ``retrieval/pipeline.py:_graph_search`` cutover to vector recall. Unit tests can prove each wiring CALL exists; only the end-to-end narrative validates the *behaviour*. Per huangheng's CR rationale: "grep 只能 catch wiring 是否存在, 不能 catch wiring 是否 produce 期望行为". This PR ships only the scaffold: * File ``tests/integration/test_w7_e2e_graph_recall_and_merge.py`` with 9 narrative test functions, each documenting its step shape. * Module-level ``pytestmark`` skips the file behind two gates: - ``_TASK8_WIRING_LANDED = False`` flips True alongside the task #8 PR (or in this PR's own follow-up commit) to enable bodies. - ``RUN_W7_E2E_NARRATIVE=1`` env gate so local-dev pytest stays fast; CI Wave 7 lane sets it once task #8 wiring is alive (mirrors the Wave 4 ``test_full_indexing_pipeline.py`` Layer 2 gate pattern). * ``pytest -v`` collects 9 tests, all skip cleanly. Sequence (per huangheng + architect): task #8 merge → flip ``_TASK8_WIRING_LANDED = True`` + fill bodies + run Layer 2 → push final PR → merge BEFORE task #10 close-out (so a regression introduced by deleting legacy is caught while rollback is still cheap). Step-9 fold-in (per architect msg=a0ba75da): failure-mode integration — simulate compactor LLM down → re-sync, verify Wave 6 graceful degrade preserved. Step-8 W8-3 trigger pin (per huangheng msg=0b48af2b): ``GET /collections/{cid}/graphs/nodes/{alicia_id}`` returns 404 today (read-side alias resolution deferred to Wave 8 W8-3). When W8-3 ships the assertion flips to "200 with Alice payload" — the trigger condition is physically pinned in the repo. * test(w7-task#11): flip _TASK8_WIRING_LANDED + concrete API pointers (+ post-#1762 lint sweep) Task #8 PR #1762 merged 2026-04-28 (commit 08d9d3b). Updates to this scaffold + a small repo-wide lint sweep that pre-commit caught. This PR's substantive change (single test file): * ``_TASK8_WIRING_LANDED`` flipped True; module-level wiring skip removed. The file now gates only on ``RUN_W7_E2E_NARRATIVE=1`` (Layer 2 env gate) so local-dev pytest stays fast while CI Wave 7 lane can run bodies. * Per-step skip messages rewritten as concrete API pointers — each step names the exact merged surface its body must call (REST path, service method, repository class), so the body-fill follow-up can grep straight to the right spot. Bodies still pending implementation — they need a running stack (Postgres + Qdrant + Redis + ES + LLM provider keys). PR remains draft until bodies are filled + Layer 2 has run green at least once. Bundled lint cleanup (ruff format only, zero behavior change): * aperag/mcp/server.py — import block re-sort (post #1759 squash ordering drift). * aperag/domains/knowledge_graph/service.py * aperag/graph_curation/lineage_merge.py * tests/unit_test/mcp/test_graph_tools.py * tests/unit_test/service/test_graph_search_service_layer.py The four ``aperag/`` + ``tests/unit_test/`` files above all came in via the PR #1762 squash merge moments ago and trip ``ruff format --check``. Bundling here so this PR can land cleanly through pre-commit; the alternative is a separate "format-only" PR for code that just merged, which is more PR overhead than value. * test(w7-task#11): fill 9-step e2e narrative bodies Per architect msg=fa045579 routing — chenyexuan owns body fill on 冬柏's branch. Bodies exercise the three task #8 wiring points behaviourally so a regression that survives unit-level grep is caught here while rollback is still cheap. Approach * Service-layer + DB-direct narrative (not HTTP through FastAPI). The HTTP shape is pinned by the task #7 / task #8 route unit tests; narrative-correctness lives in the service-layer behaviour the routes delegate to (``GraphService.merge_entities``, ``GraphService.get_entity_detail``, ``GraphSearchService.search_entities``, ``LineageGraphStoreWithAliasRedirect``, ``AliasMapRepository.resolve_canonical``). This keeps the test fixture-light: no live FastAPI, no auth bootstrap, no Collection ORM seeding — just module-scoped store / decorator / repo fixtures bound to a synthetic collection_id. * Pre-extract entities (Alice / Bob / Acme / Alicia) via direct ``upsert_entity_with_lineage`` so the narrative does not depend on a non-deterministic LLM extractor's output. The wiring under test is the storage + vector + alias paths, not the extractor prompt. * Module-scoped state via the lineage store + alias_map (production data plane) so step N+1 sees step N's side-effects without Python globals. Step coverage step 1 seed Alice / Bob / Acme via raw inner store; assert ``get_entity`` returns them post-write. step 2 build the per-collection ``GraphSearchService`` factory; re-assert step 1 entities survived (snapshot-diff invariant — Wave 4 §C.3). step 3 ``search_entities("Alice", top_k=5)`` shape contract: list of ``EntityWithLineage`` (graceful-degrade on empty per Wave 6 keyword-path convention). step 4 call ``GraphService.merge_entities(target="Alice", sources=["Alicia"])`` (the function the REST route now delegates to). Asserts backward-compat shape including ``edges_redirected/edges_collapsed == 0`` (Wave 7 §K.12 invariant #9 / cuiwenbo schema diff lock). step 5 ``AliasMapRepository.resolve_canonical("Alicia")`` → ``"Alice"``. Self-canonical case ``Alice → Alice`` also asserted. step 6 the critical inseparability gate (wiring #1). Re-extract ``Alicia`` through the *decorated* store. Assert ``get_entity("Alicia") is None`` (silent redirect) and the new lineage member shows up under ``"Alice"``. step 7 ``search_entities("Alice")`` round-trip post-merge — alias name MUST NOT leak into the recall result (the alias is a payload-redirected write, not a separate vector point). step 8 W8-3 trigger pin. ``GraphService.get_entity_detail ("Alicia")`` returns ``None`` today because read-side alias resolution is deferred to Wave 8. The assertion message documents the flip when W8-3 ships ("flip to ``assert result.name == 'Alice'``"), so the trigger condition is mechanically pinned in the repo (per architect msg=a0ba75da + huangheng msg=0b48af2b). step 9 failure-mode fold-in. Patch ``GraphIndexCompactor.compact_if_oversized`` to raise. Re-trigger the merge. Assert the merge still completes with a unified description (graceful degrade — compaction failure is non-fatal per Wave 6 ``test_w7_phase3_*_failure_non_fatal`` unit invariant). Layer 2 gating preserved — tests skip cleanly under default ``pytest`` (no ``RUN_W7_E2E_NARRATIVE=1``); CI Wave 7 lane flips it on once the e2e-http-compose stack provides Postgres / Redis / Qdrant / Elasticsearch + provider keys. Local verification * ``uv run pytest tests/integration/test_w7_e2e_graph_recall_and_merge.py --collect-only`` → 9 collected. * ``uv run pytest`` (default gate off) → 9 skipped. * ``uv run ruff check`` clean. Bodies use real production APIs (no mock-of-mock) so end-to-end verification on the e2e-http-compose lane is what 冬柏 retains for the un-draft toggle (per msg=fa045579). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(w7-task#11): apply 冬柏 review — assert-message W8-3 pin + drop self-canonical Per @冬柏 msg=8f488513 + PM msg=a769156c review items: * **Q1 step 8 W8-3 trigger pin**: lift the future-flip instruction from a docstring comment into the ``assert ... , ""`` message so the Wave 8 implementer's traceback names the exact line to flip (``flip to: assert result is not None and result.name == 'Alice'``). Comments are easy to grep-miss; assert messages travel with the failure. * **Q3 step 5 self-canonical case**: drop the ``Alice → Alice`` resolve_canonical assertion. It is degenerate corner-case coverage that already lives in unit-level ``test_alias_map_resolve_canonical*``; carrying it on the e2e narrative dilutes the merge-journey storyline without adding integration value. Q2 (step 9 monkeypatch target ``GraphIndexCompactor.compact_if_oversized``) acknowledged unchanged — already aligned with the unit-level ``test_w7_phase3_*_failure_non_fatal`` pattern. Local verify: 9 collected + 9 skipped under default gate; ruff clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(w7-task#11): ruff format on chenyexuan iteration commit --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…l (W7-10) Final close-out of Wave 7 §K.12: deletes the legacy ``aperag/domains/knowledge_graph/graphindex/`` package, drops the legacy ``graphindex_*`` tables via alembic, and adds the one new Protocol method (``LineageGraphStore.list_entities``) the architect ratified to replace the legacy ``list_entities_for_curation`` / ``get_knowledge_graph`` enumerate-by-label paths the legacy package owned. This commit also folds in the grep-zero verification helper (formerly PR #1763 by 冬柏) with ``_TASK10_LEGACY_DELETED`` flipped to True so the 10-pattern grep-zero contract becomes an active gate in this same PR (architect-preferred atomic close-out, per simple-stable directive #1 — fewer PRs, single CI run). ## Scope (per architect msg=28afe6ab + 4-question Q1-Q4 ratify msg=838d57c3 / msg=f3216dfc) 1. **NEW Protocol method ``LineageGraphStore.list_entities``** (``label / limit / offset`` kwargs) + ``EntityWithLineage`` rows sorted by ``name`` for deterministic pagination. InMemory reference + Postgres / Neo4j / Nebula production backends (mirror ``query_entities_by_keyword`` W6 #33 chunk 2 pattern). 2. **``aperag/domains/knowledge_graph/service.py:get_knowledge_graph`` cutover** — 2-step pipeline replacing the legacy ``GraphIndexService.get_knowledge_graph``: 1. ``store.list_entities(label, limit=query_max_nodes)`` — label-filtered entity list (primary work). 2. ``GraphSearchService.get_subgraph(names, hops=max_depth)`` — optional edge expansion when ``max_depth > 0``. Each layer keeps clean semantics (W7-5 ``get_subgraph`` is anchor-expansion, not label-filter; using it as primary entry would force a wrapper that re-enumerated entities just to compute anchors — drift caught in architect own-up msg=838d57c3 → revise to ``list_entities`` primary). 3. **``CurationEntity`` adapter** (new ``aperag/graph_curation/dto.py``) replacing the legacy ``Entity`` DTO. ``from_lineage(EntityWithLineage)`` constructor adapts the storage view into the shape ``build_candidate_pairs`` / ``_pair_score`` / ``_jaccard`` / ``entity_snapshot`` already accept — production-validated algorithm keeps its 0-signature change (architect Q2 ratify, simple-stable directive #3). 4. **``aperag/graph_curation/service.py`` cutover** — ``accept_suggestion`` and ``generate_run`` migrated off the legacy ``GraphIndexService`` bundle: - ``accept_suggestion`` delegates to ``LineageEntityMerger.merge_entities`` (W7-6, PR #1758) the same way the W7-8 ``GraphService.merge_entities`` route already does; both surfaces converge on a single merge path so user-merge- from-curation vs user-merge-from-graph-view never diverge. - ``generate_run`` signature now takes ``store`` / ``vector_connector`` / ``embedder`` / ``llm`` (architect Q3 ratify) and uses two new helpers: - ``_enumerate_curation_entities`` — paged ``list_entities`` loop adapting each row to ``CurationEntity``. - ``_fetch_shadow_neighbours`` — ANN search via ``VectorStoreConnector`` with the Wave 7 W7-3 3-field payload (``Eq("indexer", "graph_entity")`` filter) replacing the legacy ``find_entity_shadow_neighbors`` that filtered on the deleted ``entity_id`` payload field. 5. **``aperag/graph_curation/integration.py`` rewrite** — ``run_graph_curation_run_sync`` resolves the four Wave 7 deps via the same ``worker_factory`` factories the indexer / curation merger use, with a ``_SyncEmbedderShim`` adapter mirroring the one in ``worker_factory`` for the merge candidate detector. 6. **``build_collection_llm_callable`` relocation** — production call sites (``worker_factory._build_collection_graph_compactor`` / ``_build_collection_summarizer``, ``aperag/graph_curation/lineage_merge.py:build_lineage_entity_merger_for``, ``aperag/graph_curation/integration.py``) all import from the canonical home ``aperag/indexing/llm.py`` (Q3 ratify; the file already exists, the legacy package was just re-exporting). 7. **Legacy package + tests deleted**: - ``aperag/domains/knowledge_graph/graphindex/`` (entire package) - ``tests/unit_test/graphindex/`` (entire dir) - ``tests/integration/compat/test_graph_compat.py`` (replaced by ``test_lineage_graph_compat.py`` in W7-1) 8. **Alembic drop migration** ``c7e3a1b9f4d6`` removes ``graphindex_chunks`` / ``graphindex_edges`` / ``graphindex_nodes`` plus their indexes / unique constraints. Hard-cut policy per spec §K.12.12: legacy graph indexing was gated behind ``enable_knowledge_graph=False`` until Wave 4, then never wired into the new pipeline (``run_index_document_sync`` had 0 production callers since Wave 4 hard-cut), so the tables are empty across every deployment. Downgrade recreates empty schema. 9. **Test rewrites** — three test files that consumed the legacy ``Entity`` DTO got updated: - ``tests/unit_test/graph_curation/test_service.py`` / ``test_candidate_generation.py`` — switched to ``CurationEntity as Entity``. - ``tests/unit_test/service/test_search_graph_contract.py`` — rewritten to consume ``EntityWithLineage`` / ``RelationWithLineage`` via the new ``_adapt_lineage_entities`` / ``_adapt_lineage_relations`` adapters (the W7-1 lineage-side replacements for the deleted ``_adapt_nodes`` / ``_adapt_edges`` helpers). - 7 new InMemory ``list_entities`` unit tests in ``tests/unit_test/indexing/test_t1_2_graph.py`` covering empty-collection, sort, label filter, pagination, zero-or-negative limit, negative offset, compacted forward-compat. ## §K.12 invariant cross-check | # | Invariant | This PR | |---|-----------|---------| | 1 | L1 graph data not polluted | ✅ ``list_entities`` is read-only; storage view → adapter projection only | | 2 | L1 → L2 single-direction derive | ✅ no derived writes | | 3 | Compactor before vector embed | N/A — read path | | 4 | Vector store via Adaptor | ✅ ``_fetch_shadow_neighbours`` uses ``VectorStoreConnector`` only | | 5 | payload indexer filter | ✅ ``Eq("indexer","graph_entity")`` filter; no legacy ``entity_id`` payload reference | | 6 | uuid5 vector point id | N/A — read path | | 7 | snapshot-diff lineage name set | N/A — read path | | 8 | alias_map persist orphan | ✅ unaffected; alias_map is W7-6 owned | | 9 | upsert_entity alias redirect | ✅ unaffected; decorator pattern preserved (curation flow uses inner store directly per architect msg=cf860ae4) | | 10 | DB column length application-cap | ✅ no schema CHECK constraints introduced | | 11 | candidate detection write-only | ✅ ``MergeCandidateDetector`` unchanged; ``generate_run`` uses same write boundary | | 12 | grep-zero LightRAG | ✅ `rg "from aperag.domains.knowledge_graph.graphindex" aperag/ tests/` returns only the assertion in ``test_graph_search_migration.py:55``. ``rg "graphindex_*"`` against ``aperag/`` is 0 outside the alembic migration history. The 8 Wave 6-era ``# -- LightRAG-style query layer`` comments + W7-4 line 249 fallback + W7-5 docstrings remain (architect msg=3fe200be — they are descriptive comments referencing design heritage, removable in Wave 8 cleanup if desired) | ## 4-pattern pre-check matrix (paste from PR thread reply) * **P1 v1** — ``rg "from aperag.domains.knowledge_graph.graphindex" aperag/ tests/`` produced 6 production / 11 import sites pre-PR; post-PR matches only ``test_graph_search_migration.py:55`` (the assertion-as-test that itself proves the migration is complete). * **P1 v2** — every method on the legacy ``GraphIndexService`` that a non-legacy caller used is now accounted for: ``merge_entities`` → W7-8, ``get_knowledge_graph`` → 2-step pipeline above, ``list_entities_for_curation`` → ``LineageGraphStore.list_entities`` + ``CurationEntity.from_lineage``, ``find_entity_shadow_neighbors`` → ``_fetch_shadow_neighbours`` via ``VectorStoreConnector``, ``list_labels`` → already migrated W6 #40 + W7-1 ``compacted_description`` field. * **P2** — alembic ``c7e3a1b9f4d6`` drops the legacy tables, alembic env.py loses the legacy ``graphindex.models`` import (replaced with explanatory comment); ``aperag_lineage_*`` tables stay intact. * **P3** — single Protocol method addition (``list_entities``) — implemented across InMemory + 3 production backends + 7 unit tests. ## simple-stable 4-guardrail | Guardrail | Status | |---|---| | #1 不无限扩范围 | ✅ ``list_entities`` is base capability mirroring `delete_entity` / `query_entities_by_keyword`; no new endpoints, no new schema tables | | #2 尽快上线 | ✅ single PR closes Wave 7; all 11 prior task PRs already merged | | #3 简单稳定 | ✅ adapter pattern preserves production-validated `build_candidate_pairs`; ``list_entities`` follows existing pagination idiom | | #4 私有化部署免维护 | ✅ alembic auto-drops legacy tables; no operator config; ``list_entities`` uses the same backend factories the rest of Wave 7 wires | ## Test plan - [x] All 1142 unit tests pass (``uv run pytest tests/unit_test/``) - [x] ``alembic upgrade head --sql`` generates the expected ``DROP INDEX`` / ``DROP TABLE`` cascade - [x] ``alembic heads`` resolves to single head ``c7e3a1b9f4d6`` - [x] ``ruff format --check`` / ``ruff check`` clean on touched files - [ ] CI compat-graph + e2e-http stages — both gated post-merge - [ ] Pair with 冬柏's grep-zero helper PR #1763 — flip ``_TASK10_LEGACY_DELETED=True`` once both PRs merge Closes Wave 7. Next: architect final review per spec §K.12.12. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> ## Fold-in: grep-zero verification helper (formerly PR #1763) Per architect ratify + 冬柏 authorization, PR #1763 is folded into this commit instead of shipping as a separate PR. Contents: * **``tests/integration/test_w7_grep_zero_legacy_graphindex.py``** (NEW, 343 LOC) — 10 ripgrep contracts, one per legacy pattern, flipped to active gate (``_TASK10_LEGACY_DELETED = True``). Patterns cover: 1. ``from aperag.domains.knowledge_graph.graphindex`` imports 2. ``graphindex_(nodes|edges)`` table names (excludes the migration script itself) 3. bare ``import aperag.domains.knowledge_graph.graphindex`` 4. ``_sync_entity_relation_vectors`` (W7-3 superseded) 5. ``_compact_oversized_descriptions`` (W7-2 superseded) 6. ``_summarize_description`` (W7-2 superseded) 7. ``_fallback_truncate`` (renamed-and-kept on new ``GraphIndexCompactor`` — exception list documents the new home) 8. ``_delete_removed_shadow_vectors`` (W7-3 superseded) 9. ``GraphSearchContract.query_context`` (port name kept on the retrieval Protocol; legacy ``GraphIndexService.query_context`` historical-context comments allow-listed) 10. ``GraphIndexService.merge_entities`` (legacy class binding; historical-context comments in lineage_merge.py + test_wave7_task8_wiring.py allow-listed) * **Self-exclusion**: ``_rg_count`` always excludes this helper file itself (every pattern is named in the docstring + assertion call site, which would otherwise self-trigger). * **``aperag/mcp/server.py``** — bundled ruff-format/import-sort drift fix (post-#1762/#1759 leftover that pre-commit catches). Kept here so the close-out PR lands cleanly through ``make lint``. ## Test plan (final) - [x] 1141 unit tests pass (``uv run pytest tests/unit_test/``) - [x] 10/10 grep-zero integration tests pass (``uv run pytest tests/integration/test_w7_grep_zero_legacy_graphindex.py``) - [x] ``alembic upgrade head --sql`` generates the expected ``DROP INDEX`` / ``DROP TABLE`` cascade - [x] ``ruff format --check`` / ``ruff check`` clean on touched files - [ ] CI e2e-http-smoke + e2e-http-provider — gated post-merge Closes Wave 7. Next: architect final review per spec §K.12.12. Co-Authored-By: 冬柏 <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…l (W7-10) (#1765) Final close-out of Wave 7 §K.12: deletes the legacy ``aperag/domains/knowledge_graph/graphindex/`` package, drops the legacy ``graphindex_*`` tables via alembic, and adds the one new Protocol method (``LineageGraphStore.list_entities``) the architect ratified to replace the legacy ``list_entities_for_curation`` / ``get_knowledge_graph`` enumerate-by-label paths the legacy package owned. This commit also folds in the grep-zero verification helper (formerly PR #1763 by 冬柏) with ``_TASK10_LEGACY_DELETED`` flipped to True so the 10-pattern grep-zero contract becomes an active gate in this same PR (architect-preferred atomic close-out, per simple-stable directive #1 — fewer PRs, single CI run). ## Scope (per architect msg=28afe6ab + 4-question Q1-Q4 ratify msg=838d57c3 / msg=f3216dfc) 1. **NEW Protocol method ``LineageGraphStore.list_entities``** (``label / limit / offset`` kwargs) + ``EntityWithLineage`` rows sorted by ``name`` for deterministic pagination. InMemory reference + Postgres / Neo4j / Nebula production backends (mirror ``query_entities_by_keyword`` W6 #33 chunk 2 pattern). 2. **``aperag/domains/knowledge_graph/service.py:get_knowledge_graph`` cutover** — 2-step pipeline replacing the legacy ``GraphIndexService.get_knowledge_graph``: 1. ``store.list_entities(label, limit=query_max_nodes)`` — label-filtered entity list (primary work). 2. ``GraphSearchService.get_subgraph(names, hops=max_depth)`` — optional edge expansion when ``max_depth > 0``. Each layer keeps clean semantics (W7-5 ``get_subgraph`` is anchor-expansion, not label-filter; using it as primary entry would force a wrapper that re-enumerated entities just to compute anchors — drift caught in architect own-up msg=838d57c3 → revise to ``list_entities`` primary). 3. **``CurationEntity`` adapter** (new ``aperag/graph_curation/dto.py``) replacing the legacy ``Entity`` DTO. ``from_lineage(EntityWithLineage)`` constructor adapts the storage view into the shape ``build_candidate_pairs`` / ``_pair_score`` / ``_jaccard`` / ``entity_snapshot`` already accept — production-validated algorithm keeps its 0-signature change (architect Q2 ratify, simple-stable directive #3). 4. **``aperag/graph_curation/service.py`` cutover** — ``accept_suggestion`` and ``generate_run`` migrated off the legacy ``GraphIndexService`` bundle: - ``accept_suggestion`` delegates to ``LineageEntityMerger.merge_entities`` (W7-6, PR #1758) the same way the W7-8 ``GraphService.merge_entities`` route already does; both surfaces converge on a single merge path so user-merge- from-curation vs user-merge-from-graph-view never diverge. - ``generate_run`` signature now takes ``store`` / ``vector_connector`` / ``embedder`` / ``llm`` (architect Q3 ratify) and uses two new helpers: - ``_enumerate_curation_entities`` — paged ``list_entities`` loop adapting each row to ``CurationEntity``. - ``_fetch_shadow_neighbours`` — ANN search via ``VectorStoreConnector`` with the Wave 7 W7-3 3-field payload (``Eq("indexer", "graph_entity")`` filter) replacing the legacy ``find_entity_shadow_neighbors`` that filtered on the deleted ``entity_id`` payload field. 5. **``aperag/graph_curation/integration.py`` rewrite** — ``run_graph_curation_run_sync`` resolves the four Wave 7 deps via the same ``worker_factory`` factories the indexer / curation merger use, with a ``_SyncEmbedderShim`` adapter mirroring the one in ``worker_factory`` for the merge candidate detector. 6. **``build_collection_llm_callable`` relocation** — production call sites (``worker_factory._build_collection_graph_compactor`` / ``_build_collection_summarizer``, ``aperag/graph_curation/lineage_merge.py:build_lineage_entity_merger_for``, ``aperag/graph_curation/integration.py``) all import from the canonical home ``aperag/indexing/llm.py`` (Q3 ratify; the file already exists, the legacy package was just re-exporting). 7. **Legacy package + tests deleted**: - ``aperag/domains/knowledge_graph/graphindex/`` (entire package) - ``tests/unit_test/graphindex/`` (entire dir) - ``tests/integration/compat/test_graph_compat.py`` (replaced by ``test_lineage_graph_compat.py`` in W7-1) 8. **Alembic drop migration** ``c7e3a1b9f4d6`` removes ``graphindex_chunks`` / ``graphindex_edges`` / ``graphindex_nodes`` plus their indexes / unique constraints. Hard-cut policy per spec §K.12.12: legacy graph indexing was gated behind ``enable_knowledge_graph=False`` until Wave 4, then never wired into the new pipeline (``run_index_document_sync`` had 0 production callers since Wave 4 hard-cut), so the tables are empty across every deployment. Downgrade recreates empty schema. 9. **Test rewrites** — three test files that consumed the legacy ``Entity`` DTO got updated: - ``tests/unit_test/graph_curation/test_service.py`` / ``test_candidate_generation.py`` — switched to ``CurationEntity as Entity``. - ``tests/unit_test/service/test_search_graph_contract.py`` — rewritten to consume ``EntityWithLineage`` / ``RelationWithLineage`` via the new ``_adapt_lineage_entities`` / ``_adapt_lineage_relations`` adapters (the W7-1 lineage-side replacements for the deleted ``_adapt_nodes`` / ``_adapt_edges`` helpers). - 7 new InMemory ``list_entities`` unit tests in ``tests/unit_test/indexing/test_t1_2_graph.py`` covering empty-collection, sort, label filter, pagination, zero-or-negative limit, negative offset, compacted forward-compat. ## §K.12 invariant cross-check | # | Invariant | This PR | |---|-----------|---------| | 1 | L1 graph data not polluted | ✅ ``list_entities`` is read-only; storage view → adapter projection only | | 2 | L1 → L2 single-direction derive | ✅ no derived writes | | 3 | Compactor before vector embed | N/A — read path | | 4 | Vector store via Adaptor | ✅ ``_fetch_shadow_neighbours`` uses ``VectorStoreConnector`` only | | 5 | payload indexer filter | ✅ ``Eq("indexer","graph_entity")`` filter; no legacy ``entity_id`` payload reference | | 6 | uuid5 vector point id | N/A — read path | | 7 | snapshot-diff lineage name set | N/A — read path | | 8 | alias_map persist orphan | ✅ unaffected; alias_map is W7-6 owned | | 9 | upsert_entity alias redirect | ✅ unaffected; decorator pattern preserved (curation flow uses inner store directly per architect msg=cf860ae4) | | 10 | DB column length application-cap | ✅ no schema CHECK constraints introduced | | 11 | candidate detection write-only | ✅ ``MergeCandidateDetector`` unchanged; ``generate_run`` uses same write boundary | | 12 | grep-zero LightRAG | ✅ `rg "from aperag.domains.knowledge_graph.graphindex" aperag/ tests/` returns only the assertion in ``test_graph_search_migration.py:55``. ``rg "graphindex_*"`` against ``aperag/`` is 0 outside the alembic migration history. The 8 Wave 6-era ``# -- LightRAG-style query layer`` comments + W7-4 line 249 fallback + W7-5 docstrings remain (architect msg=3fe200be — they are descriptive comments referencing design heritage, removable in Wave 8 cleanup if desired) | ## 4-pattern pre-check matrix (paste from PR thread reply) * **P1 v1** — ``rg "from aperag.domains.knowledge_graph.graphindex" aperag/ tests/`` produced 6 production / 11 import sites pre-PR; post-PR matches only ``test_graph_search_migration.py:55`` (the assertion-as-test that itself proves the migration is complete). * **P1 v2** — every method on the legacy ``GraphIndexService`` that a non-legacy caller used is now accounted for: ``merge_entities`` → W7-8, ``get_knowledge_graph`` → 2-step pipeline above, ``list_entities_for_curation`` → ``LineageGraphStore.list_entities`` + ``CurationEntity.from_lineage``, ``find_entity_shadow_neighbors`` → ``_fetch_shadow_neighbours`` via ``VectorStoreConnector``, ``list_labels`` → already migrated W6 #40 + W7-1 ``compacted_description`` field. * **P2** — alembic ``c7e3a1b9f4d6`` drops the legacy tables, alembic env.py loses the legacy ``graphindex.models`` import (replaced with explanatory comment); ``aperag_lineage_*`` tables stay intact. * **P3** — single Protocol method addition (``list_entities``) — implemented across InMemory + 3 production backends + 7 unit tests. ## simple-stable 4-guardrail | Guardrail | Status | |---|---| | #1 不无限扩范围 | ✅ ``list_entities`` is base capability mirroring `delete_entity` / `query_entities_by_keyword`; no new endpoints, no new schema tables | | #2 尽快上线 | ✅ single PR closes Wave 7; all 11 prior task PRs already merged | | #3 简单稳定 | ✅ adapter pattern preserves production-validated `build_candidate_pairs`; ``list_entities`` follows existing pagination idiom | | #4 私有化部署免维护 | ✅ alembic auto-drops legacy tables; no operator config; ``list_entities`` uses the same backend factories the rest of Wave 7 wires | ## Test plan - [x] All 1142 unit tests pass (``uv run pytest tests/unit_test/``) - [x] ``alembic upgrade head --sql`` generates the expected ``DROP INDEX`` / ``DROP TABLE`` cascade - [x] ``alembic heads`` resolves to single head ``c7e3a1b9f4d6`` - [x] ``ruff format --check`` / ``ruff check`` clean on touched files - [ ] CI compat-graph + e2e-http stages — both gated post-merge - [ ] Pair with 冬柏's grep-zero helper PR #1763 — flip ``_TASK10_LEGACY_DELETED=True`` once both PRs merge Closes Wave 7. Next: architect final review per spec §K.12.12. ## Fold-in: grep-zero verification helper (formerly PR #1763) Per architect ratify + 冬柏 authorization, PR #1763 is folded into this commit instead of shipping as a separate PR. Contents: * **``tests/integration/test_w7_grep_zero_legacy_graphindex.py``** (NEW, 343 LOC) — 10 ripgrep contracts, one per legacy pattern, flipped to active gate (``_TASK10_LEGACY_DELETED = True``). Patterns cover: 1. ``from aperag.domains.knowledge_graph.graphindex`` imports 2. ``graphindex_(nodes|edges)`` table names (excludes the migration script itself) 3. bare ``import aperag.domains.knowledge_graph.graphindex`` 4. ``_sync_entity_relation_vectors`` (W7-3 superseded) 5. ``_compact_oversized_descriptions`` (W7-2 superseded) 6. ``_summarize_description`` (W7-2 superseded) 7. ``_fallback_truncate`` (renamed-and-kept on new ``GraphIndexCompactor`` — exception list documents the new home) 8. ``_delete_removed_shadow_vectors`` (W7-3 superseded) 9. ``GraphSearchContract.query_context`` (port name kept on the retrieval Protocol; legacy ``GraphIndexService.query_context`` historical-context comments allow-listed) 10. ``GraphIndexService.merge_entities`` (legacy class binding; historical-context comments in lineage_merge.py + test_wave7_task8_wiring.py allow-listed) * **Self-exclusion**: ``_rg_count`` always excludes this helper file itself (every pattern is named in the docstring + assertion call site, which would otherwise self-trigger). * **``aperag/mcp/server.py``** — bundled ruff-format/import-sort drift fix (post-#1762/#1759 leftover that pre-commit catches). Kept here so the close-out PR lands cleanly through ``make lint``. ## Test plan (final) - [x] 1141 unit tests pass (``uv run pytest tests/unit_test/``) - [x] 10/10 grep-zero integration tests pass (``uv run pytest tests/integration/test_w7_grep_zero_legacy_graphindex.py``) - [x] ``alembic upgrade head --sql`` generates the expected ``DROP INDEX`` / ``DROP TABLE`` cascade - [x] ``ruff format --check`` / ``ruff check`` clean on touched files - [ ] CI e2e-http-smoke + e2e-http-provider — gated post-merge Closes Wave 7. Next: architect final review per spec §K.12.12. Co-authored-by: 冬柏 <noreply@anthropic.com>
Wave 7 §K.12.6 / §K.12.7 task #7. Architect ratify msg=da89237c locked Option A: 3 internal-only REST endpoints + 3 MCP tools, all REST-mediated (mirror existing
aperag/mcp/tools/search_graph.py).Summary
build_graph_search_service_for(collection)inaperag/indexing/graph_search_service.py(mirrors_build_lineage_graph_store_for)./api/v2/collections/{id}/graphs/(GET .../entities/search,POST .../subgraph/expand,GET .../entities/{name}) — all read-only, all per-collection auth-scoped.GraphServiceprojection methods reusingGraphSearchService._render_description.aperag/mcp/tools/graph_tools.pywired at the bottom ofaperag/mcp/server.py.§K.12 12-invariant cross-check
4-pattern pre-check matrix
simple-stable 4 guardrail
Test plan
🤖 Generated with Claude Code