Skip to content

Commit 7d1adfa

Browse files
earayuclaude
andauthored
feat(collection): task #61 P1-D3 — vector backend identity + capability matrix (#1949)
* feat(collection): task #61 P1-D3 — vector backend identity + capability matrix Project the deployment-wide ``settings.vector_db_type`` onto every collection detail read so the FE can render a "what does this vector backend actually support" panel without per-collection migration or runtime probe. Backend (output-only projection): - ``aperag/schema/common.py``: ``VectorBackendCapabilities`` + ``VectorBackendInfo`` + ``_STATIC_VECTOR_BACKEND_CAPABILITIES`` dict + ``project_vector_backend_info()`` helper. - ``aperag/domains/knowledge_base/schemas.py:Collection``: add ``vector_backend: Optional[VectorBackendInfo]``. **Intentionally NOT on ``CollectionConfig``** so the OpenAPI ``CollectionCreate`` / ``CollectionUpdate`` input shapes do not let callers mistake a deployment-wide setting for a per-collection editable knob (per dongdong msg=c2593fdd + PM msg=caf7e4df + architect msg=0044261f read-only projection lock). - ``aperag/domains/knowledge_base/service/collection_service.py``: populate ``vector_backend`` in ``build_collection_response`` from ``settings.vector_db_type``; ``None`` for unknown backends so the FE can render a placeholder without a hard failure. Cross-PR consistency with task #83 / PR #1948 (Bryce, vector adapter behavior fixes): - Bryce's connector-layer ``BACKEND_CAPABILITIES`` ClassVar declares 2 truth flags (``supports_atomic_batch_upsert`` + ``supports_legacy_mode``); this PR's schema-layer Pydantic model mirrors those values plus a 3rd schema-layer-only flag ``supports_filter_or_with_empty_parts`` which is uniformly False across adapters after task #83 P1-V3 (translator-level defense-in-depth rejects empty Or parts). - The 3rd flag stays in the schema so the FE can declare the uniform reject explicitly per spec § 2.3 P1-D3 「显示『允许差异但显式』」 — Lesson #17 backend 收敛 contract simple-stable family pattern (cite PR #1930 SearchHit normalize, PR #1935 GraphMergeSuggestionItem projection layer). Mechanical gate (per Lesson #18 lesson-sediment + mechanical-gate 双 layer codification — first established by chenyexuan PR #1933 / PR #1941, then PR #1940 ``model_validate`` boundary): 13-case unit suite in ``tests/unit_test/contracts/test_vector_backend_capability_matrix.py`` pins each capability flag, normalizes inputs, and round-trips Pydantic ``model_dump`` so future drift between schema, projection helper, and FE-consumed shape fails fast at unit-test time. FE (read-only display): - ``web/src/features/collection/types.ts``: typed mirrors ``VectorBackendInfo`` / ``VectorBackendCapabilities`` / ``VectorBackendType``. - ``web/src/app/workspace/collections/[collectionId]/settings/collection-vector-backend-card.tsx``: new component that surfaces backend identity + capability matrix in the collection settings page (above the edit form). dongdong picks up rendering polish (responsive + dark mode + final copy) on the same PR per the joint A4-style split (cuiwenbo contract layer + dongdong rendering polish + CR pair). - ``web/src/i18n/{en-US,zh-CN}/page_collections.json``: copy strings. - ``web/src/api-v2/schema.d.ts`` regenerated via ``yarn api:v2:types``. Local verification: - ``uv run --extra test pytest tests/unit_test/contracts/test_vector_backend_capability_matrix.py tests/unit_test/contracts/test_collection_v2_openapi_contract.py -q`` → 23 passed - ``make openapi-check`` → ok - ``yarn type-check --pretty false`` → 0 new errors on this PR's files (pre-existing graph-lab cosmograph + agent-runtime errors unchanged) - ``yarn lint --quiet`` → 0 warnings/errors - ``yarn i18n:check`` → ok - ``git diff --check`` → ok Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(collection): task #87 P1-D3 — convert vector_backend to computed_field Per dongdong msg=fa88e97b BLOCKER + huangzhangshu msg=5b7cba0f / msg=ee6e7af2 + Weston msg=057f642c re-final framing verify gate + PM msg=03c821b0 fix-forward direction lock: the previous regular-field ``Optional[VectorBackendInfo]`` implementation leaked the deployment projection onto every input shape that referenced ``Collection``, including ``Collection-Input`` itself, ``Agent-Input.collections``, and ``CreateTurnRequest.collections``. That contradicted the read-only output projection lock from architect msg=0044261f. Move ``Collection.vector_backend`` to a Pydantic v2 ``@computed_field`` property so OpenAPI input/output schemas auto-split: - ``Collection-Output`` now lists ``vector_backend`` with ``readonly: true`` (verified in regenerated ``web/src/api-v2/schema.d.ts``). - ``Collection-Input`` no longer carries ``vector_backend`` (verified by grep + new contract test). - ``CollectionCreate`` / ``CollectionUpdate`` / ``Agent-Input.collections`` / ``CreateTurnRequest.collections`` all inherit the cleaned ``Collection-Input``, so the deployment-wide setting can no longer be passed as a per-collection override on agent / chat-turn requests. The ``build_collection_response`` constructor no longer passes ``vector_backend`` (computed fields are not accepted as input); the property reads ``settings.vector_db_type`` lazily on each serialization. Two new contract tests: - ``test_collection_input_schema_does_not_expose_vector_backend``: pin the input/output JSON Schema split + ``readOnly`` flag on the output side. Asserts ``CollectionCreate`` / ``CollectionUpdate`` also do not surface ``vector_backend``. - ``test_collection_constructor_ignores_vector_backend_input``: defensive — even if a malicious caller stuffs ``vector_backend`` into a ``model_validate`` payload, Pydantic ignores it and the computed property still reflects the deployment setting. Sediment: cuiwenbo own-up CR miss — implement-time only verified the ``CollectionConfig`` placement (one defense layer) and missed the ``Collection`` self-reuse-as-input second layer. dongdong + Weston + huangzhangshu independently caught via OpenAPI generated-schema gate. mini-pattern 19 layer 5 candidate: "Pydantic schema placement verify must grep ``references Collection`` to catch input/output reuse risk, not only direct form-input shape" (continuing the trust-framing-miss family from PR #1935 / #1938 / #1940). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(test): consolidate vector_backend_capability_matrix imports for ruff Combine the two from aperag.schema.common import ... statements into a single block so ruff's import organization rule is satisfied. No code-behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(test): apply ruff format to vector_backend test + common.py Run `uv run ruff format` on ApeRAG/aperag/schema/common.py and ApeRAG/tests/unit_test/contracts/test_vector_backend_capability_matrix.py so `make lint` (`ruff format --check`) passes. Pure formatting; no behavior change. Other unrelated files reverted to keep this PR scope clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent d433a5e commit 7d1adfa

12 files changed

Lines changed: 13162 additions & 12577 deletions

File tree

aperag/domains/knowledge_base/schemas.py

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,14 +37,16 @@
3737
from datetime import datetime
3838
from typing import Any, Literal, Optional
3939

40-
from pydantic import AnyUrl, BaseModel, Field, conint
40+
from pydantic import AnyUrl, BaseModel, Field, computed_field, conint
4141

4242
from aperag.schema.common import (
4343
Chunk,
4444
CollectionConfig,
4545
PageResult,
4646
PaginatedResponse,
47+
VectorBackendInfo,
4748
VisionChunk,
49+
project_vector_backend_info,
4850
)
4951

5052
__all__ = [
@@ -104,6 +106,34 @@ class Collection(BaseModel):
104106
is_published: Optional[bool] = Field(False, description="Whether the collection is published to marketplace")
105107
published_at: Optional[datetime] = Field(None, description="Publication time, null when not published")
106108

109+
# task #61 P1-D3 (PR for #87): read-only projection of the deployment
110+
# vector backend identity + capability matrix. Modelled as a Pydantic
111+
# v2 ``@computed_field`` so it is **only** present on the OpenAPI
112+
# output schema (``Collection-Output``) and is **not** part of any
113+
# input shape that reuses :class:`Collection` (e.g.
114+
# ``CollectionCreate``/``CollectionUpdate``, plus the request-side
115+
# composites ``Agent-Input.collections`` /
116+
# ``CreateTurnRequest.collections`` that inherit ``Collection-Input``).
117+
# Per dongdong msg=fa88e97b BLOCKER: marking ``vector_backend`` as a
118+
# plain ``Optional[VectorBackendInfo]`` field would have leaked the
119+
# deployment-wide setting onto every input shape that references
120+
# ``Collection`` and let callers think they could submit a
121+
# per-collection override on agent / chat requests — exactly the
122+
# error this projection lock is meant to prevent (per architect
123+
# msg=0044261f read-only output projection lock + PM msg=caf7e4df).
124+
@computed_field # type: ignore[prop-decorator]
125+
@property
126+
def vector_backend(self) -> Optional[VectorBackendInfo]:
127+
# Lazy import keeps the module import graph compatible with the
128+
# G1 boundary rules: ``aperag.config`` is allowed from a domain
129+
# schema, but importing it at module-load time on every domain
130+
# schema introduces a settings-side-effect at import which a few
131+
# tests intentionally avoid. Doing the import inside the property
132+
# also matches the existing ``utils.py`` lazy-resolver style.
133+
from aperag.config import settings
134+
135+
return project_vector_backend_info(settings.vector_db_type)
136+
107137

108138
class Document(BaseModel):
109139
id: Optional[str] = None

aperag/domains/knowledge_base/service/collection_service.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,15 @@ async def validate_collection_models(self, user: str, config) -> None:
175175
)
176176

177177
async def build_collection_response(self, instance: CollectionRow) -> Collection:
178-
"""Build Collection response object for API return."""
178+
"""Build Collection response object for API return.
179+
180+
``Collection.vector_backend`` is a Pydantic v2 ``@computed_field``
181+
(per task #61 P1-D3 / PR for #87 — read-only projection of the
182+
deployment vector backend identity + capability matrix). It is
183+
populated automatically on serialization, so it is intentionally
184+
**not** passed as a constructor argument — passing it would also
185+
not work because computed fields are not accepted as input.
186+
"""
179187
return Collection(
180188
id=instance.id,
181189
title=instance.title,

aperag/schema/common.py

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -303,6 +303,131 @@ class Chunk(BaseModel):
303303
metadata: Optional[dict[str, Any]] = None
304304

305305

306+
# task #61 P1-D3 (PR for #87): expose the deployment-wide vector backend
307+
# identity + capability matrix on collection metadata reads.
308+
#
309+
# Unlike ``CollectionConfig.graph_backend_type`` which is a per-collection
310+
# choice (`postgres` / `neo4j` / `nebula`), the active vector backend is
311+
# deployment-wide via ``settings.vector_db_type`` (``aperag/config.py``).
312+
# Per architect msg=bc92ca70 we project that single deployment value onto
313+
# every collection read, so the FE can render a static "vector backend
314+
# identity + capability matrix" panel without per-collection migration or
315+
# per-collection runtime probe.
316+
#
317+
# Capability flags are **static declarations** — they mirror the spec
318+
# decisions surfaced by task #83 (P1-V1~V4, vector adapter behavior fixes).
319+
# The matrix is intentionally additive: new flags can be added without
320+
# breaking older clients that bind to a subset of fields.
321+
class VectorBackendCapabilities(BaseModel):
322+
"""Static capability matrix for the deployment's vector backend.
323+
324+
Per task-61 spec § 2.3 P1-D3 (`task-61-db-adapter-compat-spec-v1.md`)
325+
the capability flags are documented declarations of behavior that the
326+
FE may surface in a "what does this backend support" panel. Flag
327+
values are derived from the spec decisions logged on task #83 (vector
328+
adapter behavior); they are not measured at runtime. If task #83
329+
re-frames the declared values the static matrix below should track.
330+
"""
331+
332+
supports_atomic_batch_upsert: bool = Field(
333+
...,
334+
description=("Whether the backend guarantees atomic visibility of a batch upsert. Per task #83 P1-V2."),
335+
)
336+
supports_filter_or_with_empty_parts: bool = Field(
337+
...,
338+
description=(
339+
"Whether the backend accepts a top-level OR filter that contains empty/no-op parts. Per task #83 P1-V3."
340+
),
341+
)
342+
supports_legacy_mode: bool = Field(
343+
...,
344+
description=(
345+
"Whether the backend exposes a legacy compatibility mode that the "
346+
"platform may still create/read. Per task #83 P1-V4."
347+
),
348+
)
349+
350+
351+
class VectorBackendInfo(BaseModel):
352+
"""Deployment vector backend identity + capability projection.
353+
354+
Read-only on collection metadata. Projected from
355+
``settings.vector_db_type``; **not** persisted per collection. If a
356+
future task introduces a per-collection vector backend override, this
357+
schema can stay; the projection helper would then read from the
358+
collection row first and fall back to ``settings.vector_db_type``.
359+
"""
360+
361+
type: Literal["pgvector", "qdrant"] = Field(
362+
...,
363+
description=("Active deployment vector backend identity, projected from ``settings.vector_db_type``."),
364+
)
365+
capabilities: VectorBackendCapabilities = Field(
366+
...,
367+
description="Static capability declaration for the deployment vector backend.",
368+
)
369+
370+
371+
# Per task #83 P1-V* spec declaration. Capability values do **not**
372+
# change at runtime — the dict key is the ``settings.vector_db_type``
373+
# lookup. Cross-PR consistency: the 2 connector-level truth flags
374+
# (``supports_atomic_batch_upsert`` + ``supports_legacy_mode``) mirror
375+
# the ``aperag/vectorstore/base.py::BACKEND_CAPABILITIES`` ClassVar
376+
# attached to each connector by task #83 (PR #1948). The 3rd flag
377+
# (``supports_filter_or_with_empty_parts``) is schema-layer-only —
378+
# task #83 P1-V3 makes both adapters reject empty Or parts uniformly,
379+
# so the flag stays False for every backend; it remains in the schema
380+
# so the FE can render the declared behavior explicitly per spec
381+
# § 2.3 P1-D3 「显示『允许差异但显式』」(per architect msg=fe7e48cb
382+
# cross-PR ground-truth-source = connector ClassVar).
383+
_STATIC_VECTOR_BACKEND_CAPABILITIES: dict[str, VectorBackendCapabilities] = {
384+
"pgvector": VectorBackendCapabilities(
385+
# PGVector inherits PG transactional semantics for batch upsert
386+
# — mirror of ``PGVectorConnector.BACKEND_CAPABILITIES`` per
387+
# task #83 PR #1948.
388+
supports_atomic_batch_upsert=True,
389+
# Post task #83 P1-V3 (PR #1948): translator-level
390+
# defense-in-depth rejects empty Or parts on top of the
391+
# ``Or.__post_init__`` DSL-level reject, uniformly across
392+
# adapters.
393+
supports_filter_or_with_empty_parts=False,
394+
# PGVector has no separate legacy schema mode — mirror of
395+
# ``PGVectorConnector.BACKEND_CAPABILITIES`` per task #83.
396+
supports_legacy_mode=False,
397+
),
398+
"qdrant": VectorBackendCapabilities(
399+
# Qdrant batch upsert is best-effort; per task #83 P1-V2.
400+
# Mirror of ``QdrantConnector.BACKEND_CAPABILITIES``.
401+
supports_atomic_batch_upsert=False,
402+
# Per task #83 P1-V3 (PR #1948): Qdrant translator also
403+
# rejects empty Or parts. Uniform with PGVector.
404+
supports_filter_or_with_empty_parts=False,
405+
# Qdrant exposes a legacy collection mode; per task #83 P1-V4.
406+
# Mirror of ``QdrantConnector.BACKEND_CAPABILITIES``.
407+
supports_legacy_mode=True,
408+
),
409+
}
410+
411+
412+
def project_vector_backend_info(vector_db_type: str) -> Optional[VectorBackendInfo]:
413+
"""Build a static :class:`VectorBackendInfo` for the active backend.
414+
415+
The argument is the raw ``settings.vector_db_type`` string. Unknown
416+
backends return ``None`` rather than raising — the projection field is
417+
declared ``Optional[VectorBackendInfo]`` so the FE can render a
418+
"unknown backend" placeholder without a hard failure on misconfigured
419+
deployments.
420+
"""
421+
422+
backend = (vector_db_type or "").strip().lower()
423+
capabilities = _STATIC_VECTOR_BACKEND_CAPABILITIES.get(backend)
424+
if capabilities is None:
425+
return None
426+
# mypy: ``backend`` is checked against the dict keys, which match the
427+
# ``Literal['pgvector', 'qdrant']`` shape. Pydantic will re-validate.
428+
return VectorBackendInfo(type=backend, capabilities=capabilities) # type: ignore[arg-type]
429+
430+
306431
__all__ = [
307432
"ModelSpec",
308433
"KnowledgeGraphConfig",
@@ -312,4 +437,7 @@ class Chunk(BaseModel):
312437
"PaginatedResponse",
313438
"VisionChunk",
314439
"Chunk",
440+
"VectorBackendCapabilities",
441+
"VectorBackendInfo",
442+
"project_vector_backend_info",
315443
]

0 commit comments

Comments
 (0)