Skip to content

Commit dbdea41

Browse files
committed
feat: rewrite CONTRIBUTING.md with full developer onboarding guide
1 parent 935129d commit dbdea41

80 files changed

Lines changed: 6185 additions & 744 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CHANGELOG.md

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,79 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
66

77
## [Unreleased]
88

9+
### Added (Phase I7 — Metadata filtering)
10+
- **`metadata_filter` on `POST /search`** — optional JSON predicate that restricts
11+
results to records whose stored metadata satisfies all specified key-value conditions.
12+
Supports exact equality for strings/booleans/null and range operators (`gt`, `gte`,
13+
`lt`, `lte`, `eq`) for numeric fields. Example:
14+
`{"author": "Alice", "year": {"gte": 2020}}`. Both standalone and cluster paths
15+
are covered. When a filter is present the server over-fetches `k×10` candidates
16+
(capped at 5000) before post-filtering to ensure `k` results are returned.
17+
- **Python SDK**`SyncRemoteClient.search()` and `AsyncRemoteClient.search()` both
18+
accept `metadata_filter: Optional[Dict[str, Any]] = None`. `ClusterClient` and
19+
`AsyncClusterClient` inherit via `**kwargs`.
20+
21+
### Added (Phase I6 — Community layer: global sensemaking + entity extraction)
22+
- **`POST /v1/community/detect`** — Label Propagation on the existing GraphNode
23+
adjacency list (pure Rust, zero LLM). Assigns every node a `community_id`,
24+
computes an f32 centroid vector per community (average of member FxpVectors),
25+
and emits a BLAKE3 receipt over the sorted `(node_id, community_id)` map —
26+
a tamper-evident proof of community structure at that point in time.
27+
Community store cached in-process; accessible by subsequent search calls.
28+
- **`POST /v1/community/search`** — Cosine-similarity search over community
29+
centroids. Returns top-k communities ranked best-first with `member_count`
30+
and a `sample_node_ids` list. Answers "what are the themes across all
31+
documents?" — the global-sensemaking query that vector RAG cannot handle.
32+
- **`POST /v1/ingest/extract-entities`** — Sends text to the configured LLM
33+
(reuses `VALORI_EMBED_PROVIDER` credentials — no new env vars). Parses
34+
`(entity, type, description)` tuples and `(source, target, description,
35+
strength)` relationships. Embeds entity descriptions and inserts them as
36+
`Concept` graph nodes with `Relation` edges — bridges a document graph into
37+
a true entity knowledge graph.
38+
- All three endpoints exist in both **standalone** (`server.rs`) and **cluster**
39+
(`cluster_server.rs`) paths, following the mandatory dual-path rule.
40+
- `valori-kernel`: added `incoming_edges()` on `KernelState` so Label
41+
Propagation can traverse both directions of the adjacency list.
42+
- Python SDK: `community_detect()`, `community_search()`, `extract_entities()`
43+
on both `SyncRemoteClient` and `AsyncRemoteClient`.
44+
45+
### Added (Phase I5 — Tree-RAG: hierarchical retrieval with provable receipts)
46+
- **`POST /v1/tree/build`** — parse a structured/markdown document into a
47+
navigable table-of-contents tree (sections, parent/child, line ranges).
48+
Deterministic, zero-LLM, zero-embedding. Returns `{node_count, structure_map, tree}`.
49+
- **`POST /v1/tree/query`** — navigate the tree to the *right section* and answer
50+
with a breadcrumb + line-range citation and a BLAKE3-chained **retrieval receipt**.
51+
Distinguishes vocabulary-overlapping sections (e.g. "sick days" → *Sick Leave*,
52+
not *Annual Leave*) where plain vector search fails. Supports `prev_hash` to
53+
chain receipts.
54+
- **`POST /v1/tree/verify`** — replay a receipt against the tree; `valid: false`
55+
proves the stored content was altered after retrieval (tamper detection).
56+
- All three are stateless handlers — identical in standalone and cluster mode.
57+
- Python SDK: `tree_build` / `tree_query` / `tree_verify` on both
58+
`SyncRemoteClient` and `AsyncRemoteClient`.
59+
60+
### Added (Phase I5 gap-fill — server-side tree cache + hybrid retrieval)
61+
- **Server-side tree cache**`Engine` (standalone) and `DataPlaneState` (cluster) now
62+
hold a `HashMap<String, TreeIndex>` keyed by `BLAKE3(text)`. `/v1/tree/build` stores the
63+
parsed tree and returns `cache_key` in the response. Subsequent `/v1/tree/query` and
64+
`/v1/tree/hybrid` calls accept `cache_key` instead of re-transmitting the full tree.
65+
- **`POST /v1/tree/hybrid`** — single-call hybrid retrieval fusing tree-RAG section scores
66+
(term-frequency, normalized to [0,1]) with vector-search similarity scores (if
67+
`VALORI_EMBED_PROVIDER` is set). Configurable `tree_weight` (default 0.6). Returns merged,
68+
re-ranked hits with per-hit `source` tag (`"tree"` or `"vector"`), BLAKE3 receipt for the
69+
tree path, and a human-readable `reasoning` string. Available on both standalone and cluster.
70+
- **`/v1/tree/build` and `/v1/tree/query`** are now stateful (take engine state for cache
71+
read/write); `/v1/tree/verify` remains stateless (no cache dependency).
72+
- Python SDK: `tree_hybrid()` added to both `SyncRemoteClient` and `AsyncRemoteClient`.
73+
74+
### Added (Phase I4.1 — replicated metadata sidecar)
75+
- **`KernelEvent::SetMeta { key, value }`** — new kernel event storing a
76+
replicated `meta` map on `KernelState`. Cluster ingest now writes the chunk/
77+
document metadata sidecar via `raft.client_write(SetMeta)` so **all** peers
78+
share it (previously node-local on the ingesting node only).
79+
- **`/v1/memory/meta/set` + `/v1/memory/meta/get`** added to the cluster router,
80+
reading/writing through the kernel (`sm.with_state`) instead of a node-local map.
81+
982
### Added (Phase I1/I2/I3 — Built-in ingest pipeline)
1083
- **`POST /v1/ingest/document`** — server-side document chunking with five strategies:
1184
`auto` (sniffs text), `tree` (section headers), `conversation` (Q&A boundaries),

0 commit comments

Comments
 (0)