libsql-sqlite3: Break vector search distance ties by rowid deterministically

The DiskANN top-k buffer ordered candidates by distance using a strict
less-than comparison (distanceBufferInsertIdx), so when two vectors are
exactly equidistant from the query their relative order in the result fell
out of the search/visit order. That visit order depends on float rounding in
the distance computation, which varies with compiler/build flags -- e.g. the
same sources built as testfixture vs the sqlite3 shell resolved such ties
differently, and the Makefile.in changes in the 3.47.0 merge were enough to
flip the testfixture build's result for vector-index-v2-query-4 (query vector
[-1,1,1,1] is exactly equidistant from rows 'b' and 'c').

Add topCandidateInsertIdx(), which breaks exact-distance ties in the top
candidates buffer by rowid (ascending). Equidistant results now come back in
a stable, reproducible order independent of build flags or how the search
happened to reach them. Non-tied results are unaffected (the distance
comparison still decides first).

Update vector-index-v2-query-4 to assert the deterministic ordering ({d a b},
'b' being the smaller rowid of the tied pair) instead of the build-dependent
value it previously hard-coded.

Author: penberg

libsql-sqlite3/src/vectordiskann.c

@@ -923,6 +923,38 @@ int distanceBufferInsertIdx(const float *aDistances, int nSize, int nMaxSize, fl
   return nSize < nMaxSize ? nSize : -1;
 }
 
+/*
+** Like distanceBufferInsertIdx() above, but for the buffer of top candidates,
+** where each slot also has an associated node (and therefore a rowid).
+**
+** When two candidates are exactly the same distance from the query vector the
+** plain distance comparison leaves their relative order up to the search/visit
+** order. That order depends on float-rounding of the distance computation,
+** which in turn varies with compiler/build flags, so equidistant results would
+** come back in a different order from one build to another. Breaking such ties
+** by rowid (ascending) makes the ordering of equidistant results deterministic
+** and reproducible regardless of how the search happened to reach them.
+*/
+static int topCandidateInsertIdx(
+  const float *aDistances,
+  DiskAnnNode *const *aCandidates,
+  int nSize,
+  int nMaxSize,
+  float distance,
+  u64 nRowid
+){
+  int i;
+  for(i = 0; i < nSize; i++){
+    if( distance < aDistances[i] ){
+      return i;
+    }
+    if( distance == aDistances[i] && nRowid < aCandidates[i]->nRowid ){
+      return i;
+    }
+  }
+  return nSize < nMaxSize ? nSize : -1;
+}
+
 void bufferInsert(u8 *aBuffer, int nSize, int nMaxSize, int iInsert, int nItemSize, const u8 *pItem, u8 *pLast) {
   int itemsToMove;
 
@@ -1100,7 +1132,7 @@ static void diskAnnSearchCtxMarkVisited(DiskAnnSearchCtx *pCtx, DiskAnnNode *pNo
   pNode->pNext = pCtx->visitedList;
   pCtx->visitedList = pNode;
 
-  iInsert = distanceBufferInsertIdx(pCtx->aTopDistances, pCtx->nTopCandidates, pCtx->maxTopCandidates, distance);
+  iInsert = topCandidateInsertIdx(pCtx->aTopDistances, pCtx->aTopCandidates, pCtx->nTopCandidates, pCtx->maxTopCandidates, distance, pNode->nRowid);
   if( iInsert < 0 ){
     return;
   }

libsql-sqlite3/test/libsql_vector_index.test

@@ -562,6 +562,11 @@ do_test vector-index-v2-query-3 {
   execsql { SELECT t.id FROM vector_top_k('t_idx', vector('[1,1,-1,-1]'), 3) i INNER JOIN t ON t.rowid = i.id; } dbv2
 } {c b a}
 
+# Query vector [-1,1,1,1] is exactly equidistant (cosine) from rows 'b'
+# ([-100,-100,-100,-100]) and 'c' ([10,10,-10,-10]) -- both at distance 1.5.
+# The ANN search breaks distance ties by rowid (ascending), so 'b' (the
+# smaller rowid) is the deterministic 3rd nearest neighbour regardless of
+# build/float-rounding.
 do_test vector-index-v2-query-4 {
   execsql { SELECT t.id FROM vector_top_k('t_idx', vector('[-1,1,1,1]'), 3) i INNER JOIN t ON t.rowid = i.id; } dbv2
-} {d a c}
+} {d a b}