VectorChord recall may over-expand semantic/BM25 candidates and degrade answer relevance before reranking

[Sanderhoff-alt]

Bug Description

Summary

I observed a relevance regression when switching a Hindsight runtime from the default PostgreSQL/native search setup to VectorChord vector search plus VectorChord BM25.

The repro uses an original English LongMemEval sample from longmemeval_s_cleaned.json:

question_id: 75832dbd
question: Can you recommend some recent publications or conferences that I might find interesting?
gold answer: The user would prefer suggestions related to recent research papers, articles,
or conferences that focus on artificial intelligence in healthcare, particularly those that
involve deep learning for medical image analysis. They would not be interested in general
AI topics or those unrelated to healthcare.

With the same LLM, embedding model, reranker model, recall budget, and benchmark harness:

• The default PostgreSQL/native search runtime answered correctly.
• The VectorChord runtime answered incorrectly.
• VectorChord expanded the merged candidate pool enough that 210 candidates were pre-filtered before reranking.
• The final context contained the relevant medical-image-analysis facts, but also ranked many unrelated facts high enough to broaden the answer into generic AI, multi-agent RL, robotics, environmental sustainability, and other unrelated topics.

This looks like an integration/tuning issue in Hindsight's VectorChord recall path: VectorChord is doing more retrieval work, but the larger candidate set is not sufficiently gated, capped, or weighted before RRF and reranking.

Environment

Two Hindsight runtimes were compared with the same model configuration:

• Hindsight image: ghcr.io/vectorize-io/hindsight:latest
• Dataset/sample: original English LongMemEval longmemeval_s_cleaned.json, question_id=75832dbd
• Runtime LLM: GLM-5.1 through an Anthropic-compatible endpoint
• Embedding: OpenAI-compatible BAAI/bge-large-zh-v1.5
• Reranker: Cohere-compatible qwen3-reranker-0.6b
• Recall budget used by the benchmark: high
• Answer/Judge model: Claude Sonnet 4.6

The storage/search difference:

Runtime	Vector extension	Text search extension
Baseline	default PostgreSQL vector setup	native PostgreSQL text search
VectorChord runtime	vchord	vchord

Relevant VectorChord runtime settings:

HINDSIGHT_API_VECTOR_EXTENSION=vchord
HINDSIGHT_API_TEXT_SEARCH_EXTENSION=vchord
HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=BAAI/bge-large-zh-v1.5
HINDSIGHT_API_RERANKER_PROVIDER=cohere
HINDSIGHT_API_RERANKER_COHERE_MODEL=qwen3-reranker-0.6b

No API keys are included here.

Observed Result

Runtime	Correct	Retrieve time	Context tokens
Baseline native search	true	1644 ms	32732
VectorChord vector + VectorChord BM25	false	1677 ms	31782

Baseline recall log:

[1] Generate query embedding: 0.064s
[2] Parallel retrieval (3 fact_types): semantic=80(0.003s), bm25=101(0.003s), graph=205(0.012s), temporal_extraction=0.183s in 0.204s
[4] Reranking: 299 candidates scored in 1.295s
[RECALL HTTP] bank=longmemeval-s-u75832dbd handler_total=1.617s ... results=299

VectorChord recall log:

[1] Generate query embedding: 0.069s
[2] Parallel retrieval (3 fact_types): semantic=476(0.011s), bm25=510(0.011s), graph=211(0.011s), temporal_extraction=0.157s in 0.191s
[4] Reranking: 300 candidates scored in 1.362s (pre-filtered 210)
[RECALL HTTP] bank=longmemeval-s-u75832dbd handler_total=1.663s ... results=300

The key difference is candidate shape:

Source	Baseline	VectorChord
Semantic candidates	80	476
BM25 candidates	101	510
Graph candidates	205	211
Candidates scored by reranker	299	300
Candidates pre-filtered before reranker	0	210

Answer Difference

The baseline answer was judged correct because it stayed focused on healthcare AI and deep learning for medical image analysis:

The user would prefer recommendations for publications and conferences that focus on ...
Explainable AI (XAI) in Medical Image Analysis ...

The VectorChord answer was judged incorrect because it broadened the preference:

The user would prefer recommendations for publications and conferences that focus on
advanced topics in deep learning, specifically explainable AI (XAI) in medical image
analysis, multi-agent and multi-modal reinforcement learning, and AI applied to
environmental sustainability and robotics ...

The gold answer explicitly says the user would not be interested in general AI topics or topics unrelated to healthcare.

Context Difference

The relevant medical-image-analysis facts were present in both contexts.

In the baseline context, the top memories started with the most relevant facts:

Memory 1: User is interested in learning about explainable AI in medical image analysis...
Memory 3: User is interested in research papers on explainable AI in medical image analysis...
Memory 7: User works in the field of deep learning for medical image analysis...

In the VectorChord context, relevant facts were also present, but unrelated facts appeared very high in the final context:

Memory 1: Recommended steamy romance novels...
Memory 2: User is interested in learning about explainable AI in medical image analysis...
Memory 4: Assistant recommended hosting a Twitter chat on sustainable fashion...
Memory 7: Fresh Earth economic/food-system whitepaper...
Memory 13: User deployed deep learning models in production, including reinforcement learning...
Memory 24: User developed a plastic-waste detection model and explored RL cleanup vehicles...

That makes the failure mode look like context pollution: the right evidence is present, but additional high-ranked memories shift the final answer toward a broader user profile.

Why This Looks Like a Hindsight Integration/Tuning Issue

For semantic retrieval, the PostgreSQL dialect applies a fixed similarity gate:

AND (1 - (embedding <=> {embedding_param}::vector)) >= 0.3

Relevant file:

hindsight_api/engine/sql/postgresql.py

Despite the same embedding model and apparent same similarity gate, the VectorChord runtime returned far more semantic candidates for this query (476 vs 80). That may be expected due to different index behavior, but it means the downstream fusion/reranking path needs to tolerate a much larger semantic candidate pool.

For BM25, the native text-search path uses a boolean full-text candidate gate:

# native tsvector
bm25_score_expr = f"ts_rank_cd(search_vector, to_tsquery('english', {text_param}))"
bm25_order_by = f"{bm25_score_expr} DESC"
bm25_where_filter = f"AND search_vector @@ to_tsquery('english', {text_param})"

The VectorChord BM25 branch orders by BM25 distance, but has no equivalent gate or score threshold:

if text_search_extension == "vchord":
    bm25_score_expr = f"-(search_vector <&> to_bm25query('idx_memory_units_text_search', tokenize({text_param}, 'llmlingua2')))"
    bm25_order_by = f"{bm25_score_expr} DESC"
    bm25_where_filter = ""

To be clear, search_vector @@ ... is not a BM25 threshold. It is a boolean candidate gate. But it does prevent the native branch from returning arbitrary weak matches. The VectorChord path currently behaves more like "take the top-N BM25 results."

After retrieval, Hindsight fuses semantic, BM25, graph, and optionally temporal results through RRF:

merged_candidates = reciprocal_rank_fusion([semantic_results, bm25_results, graph_results])

Relevant files:

hindsight_api/engine/memory_engine.py
hindsight_api/engine/search/fusion.py

Then Hindsight truncates merged candidates to the global reranker candidate cap:

reranker_max_candidates = get_config().reranker_max_candidates
if len(merged_candidates) > reranker_max_candidates:
    merged_candidates.sort(key=lambda mc: mc.rrf_score, reverse=True)
    pre_filtered_count = len(merged_candidates) - reranker_max_candidates
    merged_candidates = merged_candidates[:reranker_max_candidates]

The default cap is:

DEFAULT_RERANKER_MAX_CANDIDATES = 300

With VectorChord enabled, the merged candidate set exceeded the reranker cap by 210 candidates on this sample. That means RRF, not the reranker, decided which 300 candidates were eligible for final cross-encoder scoring.

Impact

This makes VectorChord hard to evaluate safely as a storage/search backend. It may improve raw recall, but without source-level quality controls, the larger candidate pool can degrade precision-sensitive recall/RAG answers.

This is especially important because Hindsight's public documentation positions recall as a four-way hybrid search pipeline. If one backend/source returns hundreds of extra weak candidates, the fusion and reranking stages need controls to prevent that source from polluting the final context.

Possible Fixes

Any of these would likely make the VectorChord path safer:

1. Add configurable per-source candidate caps.

   • Example: HINDSIGHT_API_SEMANTIC_MAX_CANDIDATES
   • Example: HINDSIGHT_API_BM25_MAX_CANDIDATES
   • Or bank-level equivalents.

2. Add a configurable VectorChord BM25 score/distance threshold.

   • Example: HINDSIGHT_API_BM25_MIN_SCORE
   • Or a VectorChord-specific distance cutoff if that is the correct score semantics.

3. Add weighted RRF or per-source RRF weights.

   • Example: semantic 1.0, graph 1.0, temporal 1.0, BM25 0.25.
   • This would preserve exact-match benefits without letting BM25 dominate broad queries.

4. Add a candidate gate for VectorChord BM25.

   • For example, require a minimum token overlap or a minimum number/ratio of query terms before adding a BM25 result.

5. Let the reranker see a balanced sample from each retrieval arm before applying the global cap.

   • For example, cap each source first, then merge.
   • Or reserve quotas per source before global reranker truncation.

6. Expose a per-bank or per-request toggle for BM25.

   • This would let users keep HINDSIGHT_API_VECTOR_EXTENSION=vchord while disabling or reducing BM25 for broad semantic queries.

7. Improve trace/debug output to show source contributions after RRF and after reranking.

   • This would make it easier to diagnose whether BM25 or semantic expansion is helping or polluting a particular query.

Workarounds

A partial workaround is to use VectorChord for vector search but keep text search on the native backend:

HINDSIGHT_API_VECTOR_EXTENSION=vchord
HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native

This would help isolate whether the regression is primarily from VectorChord vector search, VectorChord BM25, or their combination.

Reducing the global recall budget may also reduce candidate flooding, but it affects semantic, BM25, graph, and temporal retrieval together, so it is not a clean fix.

Questions

1. Is VectorChord BM25 intended to return top-N results without a match gate or score/distance threshold?
2. Is the large semantic candidate increase under HINDSIGHT_API_VECTOR_EXTENSION=vchord expected with the fixed similarity >= 0.3 condition?
3. Is there a recommended tuning strategy for preventing VectorChord semantic/BM25 over-expansion from overwhelming RRF/reranking?
4. Would the maintainers consider adding per-source caps, thresholds, or RRF source weighting for the VectorChord backend?

Steps to Reproduce

Reproduction

1. Start two otherwise identical Hindsight runtimes.

2. Configure one with the default/native PostgreSQL search setup.

3. Configure the other with:

   HINDSIGHT_API_VECTOR_EXTENSION=vchord
   HINDSIGHT_API_TEXT_SEARCH_EXTENSION=vchord

4. Run the original LongMemEval English sample:

   omb run \
     --dataset longmemeval \
     --split s \
     --memory hindsight-http \
     --mode rag \
     --category single-session-preference \
     --query-id 75832dbd

Expected Behavior

Expected Behavior

Enabling VectorChord vector search and VectorChord BM25 should improve recall, but it should not make broad natural-language queries less precise by flooding RRF/reranking with weakly related candidates.

Expected behavior would be one of:

• VectorChord semantic/BM25 candidates are controlled by configurable per-source caps.
• VectorChord BM25 has a meaningful match/score/distance gate before RRF.
• BM25 and semantic arms can be weighted or capped independently in RRF.
• The reranker is allowed to see a representative balanced candidate set rather than the top 300 RRF candidates after one source over-expands.
• Users can tune or disable BM25 per bank/request while still using VectorChord vector search.

Actual Behavior

Actual Behavior

On the original LongMemEval sample:

• VectorChord semantic candidates increased from 80 to 476.
• VectorChord BM25 candidates increased from 101 to 510.
• Merged candidates exceeded the reranker cap by 210.
• The final answer broadened beyond the gold preference and was judged incorrect.

Version

v0.6.2

LLM Provider

Other

[benfrank241]

@nicoloboschi what do you think about this?

[nicoloboschi]

Picked this up. Shipped a fix on branch fix/1707-vchord-bm25-gate-per-source-cap (commit ce24d45) addressing the two highest-leverage items from the "Possible Fixes" list, and verified both against a real VectorChord container (tensorchord/vchord-suite, vchord + vchord_bm25 + llmlingua2 tokenizer).

Root cause confirmed

VectorChord's <&> operator returns the negative BM25 score and ranks every document, so a bare ORDER BY … LIMIT pads the result with zero-score, non-matching rows. Native tsvector doesn't have this problem because search_vector @@ to_tsquery(...) is a boolean match gate. The vchord arm had bm25_where_filter = "" — no gate — which is exactly the candidate flooding described here.

Fix 1 — BM25 match gate (fixes #2 / fix option 4)

The vchord BM25 arm now gates on -(search_vector <&> …) > HINDSIGHT_API_BM25_MIN_SCORE (default 0), the direct analogue of native's @@ gate. Empirical proof on a real vchord DB — retain 5 mixed memories, query "medical image analysis":

Before (no gate): all 10 rows returned

fact                                            | bm25_score
------------------------------------------------+-----------
User is deeply interested in explainable AI ... |    6.0685
User is deeply interested in explainable AI ... |    5.4831
User recommended several steamy romance ...     |    0.0000
A whitepaper on environmental sustainability... |    0.0000
User explored reinforcement learning robotics...|    0.0000
... (8 rows scoring exactly 0.0000)
(10 rows)

After (gate > 0): only genuine matches survive

User is deeply interested in explainable AI ... |    6.0685
User is deeply interested in explainable AI ... |    5.4831
(2 rows)

Live recall through the real pipeline went from a padded BM25 arm to bm25=4 (matches only), HTTP 200. Oracle's CONTAINS(...) > floor now shares the same configurable knob (behavior unchanged at the 0 default).

Fix 2 — per-source candidate cap (fix options 1 / 5)

HINDSIGHT_API_RECALL_MAX_CANDIDATES_PER_SOURCE caps each arm (semantic, BM25, graph, temporal) before RRF, so a single over-expanding backend can't fill the reranker's global budget on its own. Default 0 = disabled (no change for existing native deployments). Verified live: with the cap at 1, the retrieval log went from semantic=10, bm25=4 to semantic=1, bm25=1.

Scope notes

• The semantic over-expansion in the original report (476 vs 80) was a separate bug — vchord ANN using the wrong opclass — already fixed in fix(api): vchord ANN — use cosine opclass and dispatch tuning GUCs per backend #1668 (cosine opclass). With that + the existing >= 0.3 similarity gate, the semantic arm is no longer the flooding source.
• I deliberately left the per-source cap off by default: a good cap value is a recall/precision trade-off I didn't want to bake in without a LongMemEval/LOCOMO sweep (graph alone legitimately returns ~205 on native). The BM25 gate is the active fix for the filed symptom; the cap is the configurable safety valve.
• Did not add weighted RRF (Ai-fitness-coach #3) or a per-request BM25 toggle (Docker fixes #6) — out of scope for the filed symptom.

New env vars documented in developer/configuration.md. Unit tests cover the gate SQL (default + custom floor) and the cap helper; config-parse tests cover both env vars.