Expand QIHSE trinary search benchmark datasets

Author: john

plans/qihse_persistence_layer.md

@@ -14,8 +14,8 @@ Current landed state:
 - PR-3 read-only mmap candidate work now covers `vectors.qvec`, `metadata.qmeta`, validated `idmap.qid`, and validated direct mapping of `index.qidx` for clean snapshots.
 - PR-4 physical compaction is complete for the current row model: `compact()` rewrites the in-memory table, vector blob, and metadata arena with live rows only, then publishes the compact snapshot and regenerated `idmap.qid`/`vectors.qtri` through the existing atomic flush path.
 - PR-4 compaction fixtures now verify compact-after-mutation row/index/idmap/trinary behavior, high-ID idmap consistency, stale/corrupt derived sidecar rebuild, stale `.tmp` file ignore-on-open behavior, and WAL mutation compaction clearing WAL without resurrecting pruned rows.
-- PR-5 search-path benchmark scaffolding is present: `make bench-trinary-search-path` compares full float32 DB search against DB-backed `vectors.qtri` candidate selection plus exact float32 rerank and reports recall/order/latency. Pure trinary storage remains out of scope.
-- Latest pushed checkpoint before this slice: `b8a37a3` on `codex/qihse-file-persistence`.
+- PR-5 search-path benchmark scaffolding is present: `make bench-trinary-search-path` compares full float32 DB search against DB-backed `vectors.qtri` candidate selection plus exact float32 rerank across aligned, banded, and weighted deterministic datasets, reporting recall/order/latency for each. Pure trinary storage remains out of scope.
+- Latest pushed checkpoint before this slice: `b87150a` on `codex/qihse-file-persistence`.
 
 Resume commands:
 
@@ -46,14 +46,14 @@ Current continuation:
 
 - PR-3: validate the newly landed `index.qidx` mmap path under more corruption and compatibility cases, then decide whether UMA should wrap mapped rows directly or keep the current vector DB-owned mapping path.
 - PR-4: add deeper manifest-publication crash fixtures if needed. Public mutation APIs, mutation WAL replay/truncation, physical compaction, stale temp-file handling coverage, and WAL-plus-compaction interaction coverage are present.
-- PR-5: use the search-path benchmark to drive optional vector DB trinary acceleration. Current benchmark proves recall/order on the synthetic fixture and exposes latency variance; it is not yet a production search-path optimization.
+- PR-5: use the search-path benchmark to drive optional vector DB trinary acceleration. Current benchmark proves recall/order on three deterministic fixtures and exposes latency variance; it is not yet a production search-path optimization.
 - PR-6: add persisted anchor hints and optimizer statistics only as rebuildable, explicit-format sidecars.
 
 Recommended 3-agent split:
 
 - Agent 1 owns any remaining manifest-publication crash fixtures.
 - Agent 2 owns PR-5 optional search-path acceleration wiring.
-- Agent 3 owns broader recall/performance datasets for trinary candidate generation.
+- Agent 3 owns additional realistic recall/performance datasets for trinary candidate generation.
 
 ## 1. Background
 

plans/qihse_persistence_pr0_pr1_enhancement.md

@@ -30,8 +30,8 @@ Current implementation state:
 - PR-3 candidate work has started: read-only mmap mode maps `vectors.qvec`, `metadata.qmeta`, validated `idmap.qid`, and validated direct `index.qidx` rows for clean snapshots.
 - PR-4 physical compaction is present: delete/update/upsert API symbols write committed WAL records, replay committed mutation batches newer than the snapshot, writable open truncates torn/uncommitted mutation tails, and `compact()` rewrites live rows only before publishing the regenerated snapshot/sidecars.
 - PR-4 compaction fixture coverage now enforces row/index/live/idmap counts after physical pruning, high unsigned IDs, valid `vectors.qtri` after compact, stale/corrupt derived sidecar rebuild, stale `.tmp` file ignore-on-open behavior, and WAL mutation compaction clearing WAL without resurrecting pruned rows.
-- PR-5 search-path benchmark scaffolding is present: standalone tryte top-k exists with `make bench-trinary-codec`, DB-backed candidate generation plus exact float32 rerank exists with `make bench-trinary-db-candidate`, and `make bench-trinary-search-path` compares full float32 DB search against trinary candidates plus rerank with recall/order/latency reporting.
-- Latest pushed checkpoint before this slice: `b8a37a3` on `codex/qihse-file-persistence`.
+- PR-5 search-path benchmark scaffolding is present: standalone tryte top-k exists with `make bench-trinary-codec`, DB-backed candidate generation plus exact float32 rerank exists with `make bench-trinary-db-candidate`, and `make bench-trinary-search-path` compares full float32 DB search against trinary candidates plus rerank across aligned, banded, and weighted deterministic datasets with recall/order/latency reporting.
+- Latest pushed checkpoint before this slice: `b87150a` on `codex/qihse-file-persistence`.
 - `qihse/qihse_vector_db.c` was restored after a disk-full truncation and now contains the native persistence implementation.
 - `qihse_vector_db_create(..., db_path)` opens a file-backed native database.
 - `qihse_vector_db_open()` supports ephemeral, file-copy, read-only, and read-only mmap modes.
@@ -553,7 +553,7 @@ After the current checkpoint:
 
 - PR-3: harden mmap compatibility and corruption tests now that read-only `vectors.qvec`, `metadata.qmeta`, `idmap.qid`, and `index.qidx` mapping are present for clean snapshots.
 - PR-4: public delete/update/upsert API behavior, mutation WAL replay/truncation, physical tombstone compaction, stale temp-file ignore behavior, and WAL-plus-compaction interactions are implemented and covered by persistence tests. Deeper manifest-publication crash fixtures can still be added.
-- PR-5: DB-backed candidate generation, exact rerank, and search-path benchmark scaffolding are present; production search-path acceleration, broader recall measurement, and optional pure trinary storage remain.
+- PR-5: DB-backed candidate generation, exact rerank, and three-dataset search-path benchmark scaffolding are present; production search-path acceleration, realistic recall measurement, and optional pure trinary storage remain.
 - PR-6: optional persisted anchor hints and optimizer statistics as rebuildable sidecars.
 
 ## PR-4: Mutation and Compaction Plan
@@ -719,7 +719,7 @@ Agent 2: PR-5 search-path trinary acceleration.
 
 Agent 3: PR-5 recall/performance dataset expansion.
 
-- Add broader recall/performance datasets beyond the synthetic fixture.
+- Add realistic recall/performance datasets beyond the deterministic aligned/banded/weighted fixtures.
 - Keep reporting speed honestly; the current benchmark proves recall/order but not consistent speedup.
 - Keep `vectors.qvec` authoritative until pure trinary storage has recovery, migration, and recall tests.
 - Update plan state after production search-path acceleration lands.

qihse/Makefile

@@ -85,6 +85,8 @@ bench-trinary-search-path: lib
 	    benchmarks/qihse_trinary_db_candidate_bench.c \
 	    -L. -lqihse $(LDFLAGS)
 	LD_LIBRARY_PATH=. /tmp/qihse_trinary_search_path_bench
+	QIHSE_BENCH_DATASET=banded LD_LIBRARY_PATH=. /tmp/qihse_trinary_search_path_bench
+	QIHSE_BENCH_DATASET=weighted LD_LIBRARY_PATH=. /tmp/qihse_trinary_search_path_bench
 
 clean:
 	rm -f *.o libqihse.so qihse_benchmark qihse_benchmark_a00 \

qihse/benchmarks/qihse_trinary_db_candidate_bench.c

@@ -19,6 +19,12 @@
 #define QIHSE_BENCH_TOPK 8u
 #define QIHSE_BENCH_ITERS 32u
 
+typedef enum qihse_bench_dataset_e {
+    QIHSE_BENCH_DATASET_ALIGNED = 0,
+    QIHSE_BENCH_DATASET_BANDED = 1,
+    QIHSE_BENCH_DATASET_WEIGHTED = 2
+} qihse_bench_dataset_t;
+
 typedef struct qihse_bench_ranked_s {
     size_t row_index;
     uint64_t id;
@@ -43,41 +49,78 @@ static uint32_t qihse_bench_mix32(uint32_t value) {
     return value;
 }
 
-static float qihse_bench_query_value(size_t dim) {
+static qihse_bench_dataset_t qihse_bench_dataset(void) {
+    const char* name = getenv("QIHSE_BENCH_DATASET");
+
+    if (name && strcmp(name, "banded") == 0) {
+        return QIHSE_BENCH_DATASET_BANDED;
+    }
+    if (name && strcmp(name, "weighted") == 0) {
+        return QIHSE_BENCH_DATASET_WEIGHTED;
+    }
+    return QIHSE_BENCH_DATASET_ALIGNED;
+}
+
+static const char* qihse_bench_dataset_name(qihse_bench_dataset_t dataset) {
+    switch (dataset) {
+        case QIHSE_BENCH_DATASET_BANDED:
+            return "banded";
+        case QIHSE_BENCH_DATASET_WEIGHTED:
+            return "weighted";
+        case QIHSE_BENCH_DATASET_ALIGNED:
+        default:
+            return "aligned";
+    }
+}
+
+static float qihse_bench_query_value(qihse_bench_dataset_t dataset, size_t dim) {
+    if (dataset == QIHSE_BENCH_DATASET_BANDED) {
+        return ((dim / 8u) % 2u == 0u) ? 1.0f : -1.0f;
+    }
+    if (dataset == QIHSE_BENCH_DATASET_WEIGHTED) {
+        return (dim % 5u == 0u || dim % 5u == 1u) ? 1.0f : -1.0f;
+    }
     return (dim % 3u == 0u) ? 1.0f : -1.0f;
 }
 
 static void qihse_bench_fill_data(float* rows, float* query, uint64_t* ids) {
+    qihse_bench_dataset_t dataset = qihse_bench_dataset();
     size_t row;
     size_t dim;
 
     for (dim = 0u; dim < QIHSE_BENCH_DIMS; dim++) {
-        query[dim] = qihse_bench_query_value(dim);
+        query[dim] = qihse_bench_query_value(dataset, dim);
     }
 
     for (row = 0u; row < QIHSE_BENCH_ROWS; row++) {
-        ids[row] = 100000u + (uint64_t)row;
+        ids[row] = 100000u + ((uint64_t)dataset * 1000000u) + (uint64_t)row;
         for (dim = 0u; dim < QIHSE_BENCH_DIMS; dim++) {
             uint32_t mixed = qihse_bench_mix32((uint32_t)((row + 17u) * 131u +
-                                                          (dim + 5u) * 977u));
-            uint32_t bucket = mixed % 5u;
+                                                          (dim + 5u) * 977u +
+                                                          ((uint32_t)dataset * 65537u)));
+            uint32_t bucket = mixed % (dataset == QIHSE_BENCH_DATASET_ALIGNED ? 5u : 7u);
             float value = 0.0f;
 
-            if (bucket == 0u || bucket == 1u) {
+            if (bucket == 0u || bucket == 1u ||
+                (dataset == QIHSE_BENCH_DATASET_WEIGHTED && bucket == 2u)) {
                 value = query[dim];
-            } else if (bucket == 2u || bucket == 3u) {
+            } else if (bucket == 2u || bucket == 3u || bucket == 4u) {
                 value = -query[dim];
             }
+            if (dataset == QIHSE_BENCH_DATASET_WEIGHTED && value != 0.0f) {
+                value *= (dim % 7u == 0u) ? 2.0f : 0.75f;
+            }
             rows[(row * QIHSE_BENCH_DIMS) + dim] = value;
         }
     }
 
     for (dim = 0u; dim < QIHSE_BENCH_DIMS; dim++) {
-        rows[dim] = query[dim];
-        rows[QIHSE_BENCH_DIMS + dim] = dim == 7u ? 0.0f : query[dim];
-        rows[(2u * QIHSE_BENCH_DIMS) + dim] = dim == 11u ? -query[dim] : query[dim];
+        float scale = dataset == QIHSE_BENCH_DATASET_WEIGHTED && dim % 7u == 0u ? 2.0f : 1.0f;
+        rows[dim] = query[dim] * scale;
+        rows[QIHSE_BENCH_DIMS + dim] = dim == 7u ? 0.0f : query[dim] * scale;
+        rows[(2u * QIHSE_BENCH_DIMS) + dim] = dim == 11u ? -query[dim] : query[dim] * scale;
         rows[(3u * QIHSE_BENCH_DIMS) + dim] =
-            (dim == 13u || dim == 17u) ? -query[dim] : query[dim];
+            (dim == 13u || dim == 17u) ? -query[dim] : query[dim] * scale;
     }
 }
 
@@ -416,6 +459,7 @@ static int32_t qihse_bench_candidate_score_for_row(const size_t* candidate_rows,
 }
 
 int main(void) {
+    qihse_bench_dataset_t dataset = qihse_bench_dataset();
     char* db_path = NULL;
     qihse_vector_db_t full_db = NULL;
     float* rows = NULL;
@@ -588,8 +632,9 @@ int main(void) {
                                                   full_results,
                                                   full_topk_count);
 
-    printf("trinary_search_path_bench rows=%u dims=%u qtri_row_bytes=%zu "
+    printf("trinary_search_path_bench dataset=%s rows=%u dims=%u qtri_row_bytes=%zu "
            "candidates=%u topk=%u iterations=%u\n",
+           qihse_bench_dataset_name(dataset),
            (unsigned)QIHSE_BENCH_ROWS,
            (unsigned)QIHSE_BENCH_DIMS,
            row_bytes,