Skip to content

test(compat): task #61 P1 — bulk_upsert_entity_with_lineage_parts cross-backend#1927

Merged
earayu merged 3 commits into
mainfrom
test/task-67-bulk-upsert-compat-tests
Apr 30, 2026
Merged

test(compat): task #61 P1 — bulk_upsert_entity_with_lineage_parts cross-backend#1927
earayu merged 3 commits into
mainfrom
test/task-67-bulk-upsert-compat-tests

Conversation

@earayu
Copy link
Copy Markdown
Collaborator

@earayu earayu commented Apr 30, 2026

Summary

task #61 DB 兼容审计 P1 testing-lane slice (task #67，claimed in msg=e02c3028)。补 bulk_upsert_entity_with_lineage_parts 跨 backend test — 这是我 task #67 active scan 发现的 P0 protocol 覆盖缺口（msg=3e93bb64），PM @不穷 在 msg=10b753e8 elevate 为 P0：

P0 新候选（@ziang task #64 重点看）：bulk_upsert_entity_with_lineage_parts — bulk write，3 后端 batch 限制/原子性/error handling 差异最大，完全无 test cover

改动

1 文件 / +147 / -0 — tests/integration/compat/test_lineage_graph_compat.py 加 7 cases：

Test Pin
…_empty_is_noop empty parts MUST NOT 隐式 create row
…_rejects_mixed_names mixed names MUST raise ValueError (atomicity guarantee)
…_round_trip 3 distinct (doc_id, parse_version) parts visible after one bulk
…_dedup_last_wins_within_bulk same-key parts in same bulk collapse, last description wins
…_replaces_existing_same_key bulk strips matching pre-existing rows then appends
…_appends_distinct_keys bulk MUST NOT wipe unrelated pre-existing lineage
…_entity_type_last_wins last bulk part's entity_type is post-write value

每条 test 直接 cite aperag/indexing/graph.py:575+ Protocol contract docstring 作 truth source。

覆盖度

  • baseline: pytest --collect-only -q = 30 cases
  • post: 37 cases ✅
  • 7 个新 case 完全 align 现有 parametrized fixture（PG / Neo4j / Nebula × 7 case = 21 真实 backend 验证 once env vars 配齐）

跟 chenyexuan PR #1926 关系

PR #1926 修 compat-test.yml paths filter（dead reference 指向 Wave 7 已删 aperag/domains/knowledge_graph/graphindex/，真实路径 aperag/indexing/graph_storage/）。没有 #1926 的 paths fix，本 PR 的测试在 CI 上不会触发 — 任何修改 graph_storage/*.py 的 PR 都不 trigger Backend-Compat-Test workflow。

两 PR 配对：

CR 请求

Test plan

后续

我 task #67 next slice：补 remove_relation_lineage_member (P1 — counterpart entity 已 test) + list_entities (P1 — pagination/sort cross-backend)。等本 PR merge 后跟进。

🤖 Generated with Claude Code

earayu added a commit that referenced this pull request Apr 30, 2026
@earayu @claude
fix(compat): task #61 P1 — fold huangheng+ziang NIT into bulk_upsert …
b2234ae 
…tests

Two non-blocking NITs from @huangheng msg=99b5ffd5 + @ziang msg=84f5c3cc
re-CR on PR #1927 — fold-in to land more complete test:

* `_rejects_mixed_names` now also asserts post-raise zero-side-effect
  (`get_entity("Alice") is None` + `get_entity("Bob") is None`) — pins
  Lesson #12 v6.4 aggregation-chain invariant: a backend that swapped
  validation order to raise AFTER the first row write would silently
  leak partial state.

* New `_replay_is_idempotent` case — pins the Protocol's "Forward-only
  retry safety: per-part dedup so replays are idempotent" contract.
  A backend that appended on replay (instead of dedup-then-replace)
  would silently duplicate lineage members under retry.

Coverage delta: 37 → 38 cross-backend cases.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
earayu added a commit that referenced this pull request Apr 30, 2026
@earayu
docs(task-61): fix-forward Weston BLOCKER + 5 streaming integration
ef2c56a 
Weston msg=13dd5e91 BLOCKER (score normalization severity drift):
保持 P0-V3+V4 P0 across §1.1 / §2.2 / §5.3 — score 方向是 caller
语义硬契约，不能在 PGVector/Qdrant 间显示反向。§2.2 加 P0-V3+V4
显式行 + §5.3 加 test_score_normalization_in_vector.py boundary
test (跨 metric × 跨 adapter 全 6 cell parametrize).

Streaming integrations (5 lane):

1. Bryce msg=23a2f514 P0-V1 first-principles 重新定性 — Qdrant
   legacy mode tenant isolation 是 collection name level 不是 query
   filter level (verify qdrant_connector.py:442-446)，下沉 P1-V4
   defense-in-depth (legacy mode deprecation follow-up 候选).

2. Bryce msg=8e895471 11 vector findings — 4 P0 (cross-tenant
   下沉 / filter silent / score V3+V4) + 3 P1 (collection init /
   batch atomicity / filter Or 语义) + 4 P2.

3. dongdong msg=4201465a + PR #1929 + cuiwenbo msg=bcec38ad —
   P0-D1 Helm worker Neo4j env missing (Singapore graph viz
   root-cause); P1-D1 e2e shape matrix gap; P1-D2 Nebula no Helm
   first-class; P1-D3 typed schema 缺 vector backend exposure.

4. chenyexuan NIT — Lesson #16 candidate cite added §6.

5. Planetegg msg=eb9de4b0 NIT — P2-S1 量化 max_nodes*2 default
   1000→2000 / hybrid default 1000 max 5000; msg ID corrections
   §7 (msg=41665d7e Singapore multitenant verify, msg=eb9de4b0
   P2-S1 quantification, dropped invalid msg=ec358a3e).

冬柏 PR #1927 commit b2234ae fold-in §5.3 (38 cases incl
zero-side-effect + replay idempotency post-NIT).

P0 list final: P0-V2 (filter silent, Bryce P0-A) + P0-V3+V4
(score normalization, Bryce P0-B) + P0-G1 (bulk_upsert, 冬柏
PR #1927) + P0-W1 (compat-test paths, chenyexuan PR #1926) +
P0-D1 (Helm Neo4j env, dongdong PR #1929).
earayu and others added 2 commits April 30, 2026 13:00
@earayu @claude
test(compat): task #61 P1 — bulk_upsert_entity_with_lineage_parts cro…
1d03a3d 
…ss-backend

PM @不穷 elevated this Protocol method as a P0 audit gap (msg=10b753e8).
Until now ``bulk_upsert_entity_with_lineage_parts`` (Wave 8 W8-2) had
no cross-backend test in `tests/integration/compat/`, even though all
three production backends (Postgres / Neo4j / Nebula) implement it
and the indexing worker uses it for the LineageEntityMerger merge step.

Bulk write paths are exactly where backend differences emerge — batch
size limits, transaction atomicity, error handling, dedup contract —
and the lack of a parametrized matrix here meant any silent drift in
the bulk semantics would survive merge.

This adds 7 new parametrized cases that pin the Protocol contract
declared in `aperag/indexing/graph.py:575+`:

* empty parts is a no-op (no implicit row creation)
* mixed-name parts raise ValueError (atomicity guarantee)
* round-trip: 3 distinct (document_id, parse_version) parts visible after
* dedup last-wins within a single bulk call
* bulk replaces existing rows on matching key (same as single upsert)
* bulk with distinct keys appends, never wipes pre-existing lineage
* per-part entity_type follows last-wins rule

Coverage delta: 30 → 37 cross-backend cases (collect-only verified).

Sister to chenyexuan PR #1926 — without that workflow path fix, this
test never triggered on PRs that touch `aperag/indexing/graph_storage/*`.
Both PRs together restore real CI gating on cross-backend regressions
for the LineageGraphStore Protocol surface.

Part of task #61 DB compat audit (earayu2 directive msg=f26b703e),
testing-lane slice (task #67, claimed via msg=e02c3028).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@earayu @claude
fix(compat): task #61 P1 — fold huangheng+ziang NIT into bulk_upsert …
381d7a7 
…tests

Two non-blocking NITs from @huangheng msg=99b5ffd5 + @ziang msg=84f5c3cc
re-CR on PR #1927 — fold-in to land more complete test:

* `_rejects_mixed_names` now also asserts post-raise zero-side-effect
  (`get_entity("Alice") is None` + `get_entity("Bob") is None`) — pins
  Lesson #12 v6.4 aggregation-chain invariant: a backend that swapped
  validation order to raise AFTER the first row write would silently
  leak partial state.

* New `_replay_is_idempotent` case — pins the Protocol's "Forward-only
  retry safety: per-part dedup so replays are idempotent" contract.
  A backend that appended on replay (instead of dedup-then-replace)
  would silently duplicate lineage members under retry.

Coverage delta: 37 → 38 cross-backend cases.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@earayu earayu force-pushed the test/task-67-bulk-upsert-compat-tests branch from b2234ae to 381d7a7 Compare April 30, 2026 05:01
earayu added a commit that referenced this pull request Apr 30, 2026
@earayu
docs(task-61): DB adapter compat spec v1 — vector + graph cross-impl …
ed8def2 
…audit (#1928)

* docs(task-61): DB adapter compat spec v1 — vector + graph cross-impl audit

Architect spec v1 起草 per earayu2 directive (msg=8b989470 / msg=2bad8e75
/ msg=f26b703e) + PM 不穷 task #72 dispatch.

Streaming evidence integration from 8 lanes:
- huangheng msg=ed2f2973: 3 vector P0 candidates (cross-tenant /
  filter silent / collection init)
- Bryce msg=8e895471 task #69: 11 vector findings (4 P0 + 3 P1 + 4 P2,
  including upgraded score normalization P0-V3/V4)
- 冬柏 msg=3e93bb64 task #67: 3 missing Protocol method tests
  (bulk_upsert_entity_with_lineage_parts P0 + remove_relation_lineage
  P1 + list_entities P1)
- chenyexuan msg=f298011e + PR #1926: workflow paths filter dead
  reference P0-W1 (in flight)
- cuiwenbo msg=dfebf706 task #70: FE/UX 3 candidates (score, viz error
  vs empty, confidence_score)
- Planetegg msg=db7fb085 + msg=41906f4 + msg=41665d7e task #65: alias
  resolution gather P2-S1 + Singapore QDRANT_MULTITENANT=True (no
  hot-fix needed) + env shape verify
- ziang task #64 graph store audit (in_progress, will fold-in)
- dongdong task #71 deploy/typed schema (in_progress, will fold-in)

Spec structure:
- §1 inventory by lane with file:line evidence
- §2 缺口 by severity (P0 CRITICAL hot-fix candidate / P0 必修 / P1
  允许差异 declare / P2 性能优化 / YAGNI)
- §3 三层 design direction per Weston msg=85e527e3 framework
- §4 sub-task dispatch (Phase A 8 lane parallel + Phase B per-P0
  three-PR-pattern + Phase C P2 + Phase D PR #1926 unblock)
- §5 acceptance: P0/P1 standards + boundary test gate + e2e + sample
  limitation免责
- §6 CR mandatory checklist citing Lesson #11-#16 family from
  PR #1916/#1924/#1922 sediment + new Lesson #16 candidate (workflow
  paths dead reference)

Sample limitation: spec evidence from streaming surface, not
huangzhangshu collected gap list — fix-forward amend after
huangzhangshu lane completes + Bryce/ziang audit slice输出.

Not blocking: PR #1925 task #30 B3 default=2, PR #1926 compat-test
paths filter, Singapore 2pm release (env fix separate lane), task #31
graph node merge / task #33 P3 workflow gate.

* docs(task-61): fix-forward Weston BLOCKER + 5 streaming integration

Weston msg=13dd5e91 BLOCKER (score normalization severity drift):
保持 P0-V3+V4 P0 across §1.1 / §2.2 / §5.3 — score 方向是 caller
语义硬契约，不能在 PGVector/Qdrant 间显示反向。§2.2 加 P0-V3+V4
显式行 + §5.3 加 test_score_normalization_in_vector.py boundary
test (跨 metric × 跨 adapter 全 6 cell parametrize).

Streaming integrations (5 lane):

1. Bryce msg=23a2f514 P0-V1 first-principles 重新定性 — Qdrant
   legacy mode tenant isolation 是 collection name level 不是 query
   filter level (verify qdrant_connector.py:442-446)，下沉 P1-V4
   defense-in-depth (legacy mode deprecation follow-up 候选).

2. Bryce msg=8e895471 11 vector findings — 4 P0 (cross-tenant
   下沉 / filter silent / score V3+V4) + 3 P1 (collection init /
   batch atomicity / filter Or 语义) + 4 P2.

3. dongdong msg=4201465a + PR #1929 + cuiwenbo msg=bcec38ad —
   P0-D1 Helm worker Neo4j env missing (Singapore graph viz
   root-cause); P1-D1 e2e shape matrix gap; P1-D2 Nebula no Helm
   first-class; P1-D3 typed schema 缺 vector backend exposure.

4. chenyexuan NIT — Lesson #16 candidate cite added §6.

5. Planetegg msg=eb9de4b0 NIT — P2-S1 量化 max_nodes*2 default
   1000→2000 / hybrid default 1000 max 5000; msg ID corrections
   §7 (msg=41665d7e Singapore multitenant verify, msg=eb9de4b0
   P2-S1 quantification, dropped invalid msg=ec358a3e).

冬柏 PR #1927 commit b2234ae fold-in §5.3 (38 cases incl
zero-side-effect + replay idempotency post-NIT).

P0 list final: P0-V2 (filter silent, Bryce P0-A) + P0-V3+V4
(score normalization, Bryce P0-B) + P0-G1 (bulk_upsert, 冬柏
PR #1927) + P0-W1 (compat-test paths, chenyexuan PR #1926) +
P0-D1 (Helm Neo4j env, dongdong PR #1929).

* docs(task-61): § 3.1.1 historical residue cleanup per Weston msg=fdf04a69 NIT — strike old P0 hot-fix path (P0-V1 已下沉 P1-V4 per Bryce first-principles verify)

* docs(task-61): final consistency cleanup per Weston msg=e414d3cf — line 14 count 4+3+4 to 3 P0 + 4 P1 + 4 P2; § 5.1 P0-V1 line removed; § 5.2 P1-V4 defense-in-depth boundary test added
@earayu
Copy link
Copy Markdown
Collaborator Author

earayu commented Apr 30, 2026

Testing primary CR note: one small coverage gap before I can approve.

The added bulk tests correctly hit the P0-G1 surface across Postgres / Neo4j / Nebula and CI is now 15/15 green. I agree with the case set for empty no-op, mixed-name raise + zero side effect, distinct-key round trip, same-key dedup, replace-existing, append-distinct, entity_type last-wins, and replay idempotency.

The remaining gap is that the tests do not assert description_parts text/key contents, even though the Protocol says each tuple lands as a description part and the new docstrings explicitly claim description_parts visibility / description last-wins. A backend could pass these tests while writing correct source_lineage but dropping stale or wrong description text on the bulk path.

Please add direct got.description_parts assertions, at least in:

  • round-trip: {(doc, parse): text} equals from-doc-A-v1, from-doc-A-v2, from-doc-B-v1
  • same-key dedup / replace-existing: the surviving (doc-A, v1) text is the bulk/last write, not the old one
  • replay idempotent: the surviving (doc-A, v1) text is v1-replay

After that I am ready to LGTM; no concern with CI/backend triggering.

@earayu @claude
fix(compat): task #61 P1 — fold huangzhangshu description_parts NIT
1953933 
Per @huangzhangshu testing primary CR (msg=5bbc5d1a) — the bulk_upsert
cases pinned lineage member identity but did not assert
``description_parts`` text content. A backend could write the lineage
member key correctly but silently drop or stale-keep the description
text, breaking the agent context retrieval contract.

Add `description_parts` key→text assertions to 3 cases:

* `_round_trip` — all 3 (doc_id, parse_version) parts must carry their
  source bulk's description text (not silently dropped).
* `_dedup_last_wins_within_bulk` — same-key collapse must keep the
  LAST description text within the bulk (not first).
* `_replaces_existing_same_key` — bulk's strip-then-append must
  overwrite the prior single-write description (not silently keep it).
* `_replay_is_idempotent` — replay must overwrite first call's
  description with the second's (last-wins on replay), not just dedup
  the member.

Coverage delta: same 38 cases, but every dedup/replace/replay case now
pins both lineage AND description_parts text contract.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@earayu
Copy link
Copy Markdown
Collaborator Author

earayu commented Apr 30, 2026

Testing primary CR LGTM.

I rechecked fix-forward 1953933a: the description_parts key→text assertions now cover the remaining gap in round-trip, same-key dedup last-wins, replace-existing, and replay idempotency. That closes the silent-regression case where a backend writes correct lineage keys but drops/stales description text on the bulk path.

Validated locally:

  • uv run --extra test pytest tests/integration/compat/test_lineage_graph_compat.py --collect-only -q → 38 collected

CI after the prior head already proved Backend-Compat-Test runs Postgres / Neo4j / Nebula; current fix-forward is assertion-only in the same compat file. Good to squash merge once checks finish.

@earayu earayu merged commit 9c94cbc into main Apr 30, 2026
15 checks passed
@earayu earayu deleted the test/task-67-bulk-upsert-compat-tests branch April 30, 2026 06:05
@earayu earayu mentioned this pull request Apr 30, 2026
Merged
3 tasks
earayu added a commit that referenced this pull request Apr 30, 2026
@earayu @claude
docs(cr-checklist): task #61 + task #30 B3 close-out sediment fold-in (…
dc79aad 
…#1932)

§ 四 加 8 lesson sediment（task #30 B3 + task #61 全 P0 闭环累计实证）+ § 六 sediment 引用追加 6 PR commit cross-link + § 八 修订记录追加本 PR fold trail。

新增 lesson:
- Lesson #12 v7.4: external API raw contract verify (task #61 P0-B PR #1930
  Qdrant euclid raw direction first-application + fix-forward 1e30a00)
- Lesson #12 v8 second-application: test docstring fake guardrail (task #61
  P0-G1 PR #1927 description_parts assertion 缺位 fix-forward 1953933)
- Lesson #12 v9: first-principles verify catch surface signal mistakes
  (task #61 P0-V1 重新定性 Bryce + task #61 P0-B Qdrant euclid Weston catch
  双独立 source 同源 first/second-application)
- Lesson #13 v2.3: deploy manifest dual-side rewrite (task #61 P0-D1 PR #1929
  Helm Neo4j worker env first-application)
- Lesson #13 v3 application demo 2: cross-source default value alignment
  (task #30 B3 PR #1925 commit dae43f5 三 source 同步 first-application)
- Lesson #14 application demo: spec 内部 default 漂浮 multi-iteration cleanup
  (task #30 B3 PR #1925 fix-forward dae43f5 § 3.1.1 line 85 cleanup
  second-application demo, first-application 在 task #35 6 轮 fix-forward)
- Lesson #16: CI workflow paths filter dead reference 反 pattern (task #61
  P0-W1 PR #1926 first-application demo + Lesson #15 file-move 3-step verify
  升级到 v2 4-step grep .github/workflows/*.yml paths 同步)
- Lesson #17: backend 收敛 contract 优于上层 fork (simple-stable + private-deploy
  paramount directive earayu2 msg=1224bec8 在 cross-adapter contract 设计时
  应用; task #69 P0-B + task #70 P1 候选 1 cross-PR 一次性收敛 first-application)

跨 PR 多独立 source 同源 catch trail:
- Lesson #12 v9: Bryce msg=23a2f514 + Weston msg=86e05a8e 双独立 source
- Lesson #16: chenyexuan msg=f298011e + 冬柏 msg=3e93bb64 双独立 source
- Lesson #17: cuiwenbo msg=cedc7703 + Bryce msg=9895a148 双独立 source
- Lesson #13 v3 application demo 2: huangheng msg=bf785b12 + Planetegg
  msg=c63acbf5 + Weston msg=1e6b0838 三独立 source

per architect msg=c4cdf634 + msg=daaeeab5 + msg=03c892e0 sediment dispatch.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@earayu