feat: Add cross-encoder reranker stage after retrieval (TASK-214)#1073
feat: Add cross-encoder reranker stage after retrieval (TASK-214)#1073stephanj wants to merge 1 commit into
Conversation
- Reranker interface + NoOpReranker (fallback) + OllamaReranker (default)
- Wires the reranker into SemanticSearchService after retrieval/RRF, before
the prompt. Retrieval widens to rerankerShortlistSize when ON; reranker
truncates to indexerMaxResults.
- Bounded by CompletableFuture + per-future deadline; WARN-level log on
timeout and partial-progress fallback to retrieval order.
- New settings: rerankResults, rerankerModelName ('llama3.2:1b'),
rerankerShortlistSize (30), rerankerTimeoutMs (2000); persisted via
DevoxxGenieStateService; RAG settings UI exposes all four.
- New RerankerValidator (mirrors NomicEmbedTextValidator) with a
PULL_RERANKER action wired into RAGSettingsHandler and the
validator status panel.
- SearchResult carries preRerankRank + rerankerScore; RAG log and
AgentMcpLogPanel surface both per hit.
- OllamaReranker uses a dedicated short-timeout OkHttpClient (no retries)
and a shared static executor pool with bounded queue.
- Unit tests for NoOpReranker, OllamaReranker (reordering, top-N
truncation, timeout fallback, partial-progress, score normalization),
RerankerValidator, the OFF/ON gating in SemanticSearchService, and
RAGSettingsConfigurable reranker round-trip semantics.
367c959 to
ace814b
Compare
|
Re C2 (workflow file revert): the sandbox's authenticating proxy refuses pushes that diff '.github/workflows/claude-code-review.yml' from origin/master to upstream/master because the OAuth app lacks 'workflow' scope. origin/master (the fork tip) is behind upstream/master and still has the older version of this file; pushing the reverted-to-upstream version fails with 'refusing to allow an OAuth App to create or update workflow'. To unblock the push I had to keep the file at origin/master's content (the older version), which is why the PR still shows the diff. The fork needs to be brought up to date with upstream/master before this file can be reverted from the sandbox \u2014 please do that on your end and the diff will disappear automatically, or merge with the "Squash and merge" or "Rebase and merge" option which will reconcile against upstream's tip. All other review items (C1, M1\u2013M4, m1\u2013m3, n1\u2013n2) are addressed in commit ace814b. |
|
Thanks for the explanation @stephanj — the OAuth proxy in our sandbox lacks
All other review items (C1, M1–M4, m1–m3, n1–n2) have been addressed in commit ace814b. |
Adds an optional reranker stage that runs after vector retrieval (and any RRF fusion) and reorders the shortlist before it reaches the prompt.
What changed
Rerankerinterface withNoOpReranker(fallback) andOllamaReranker(default, Ollama-hosted chat-completion scoring 0–10 → [0.0, 1.0]).SemanticSearchServiceafter retrieval; retrieval widens torerankerShortlistSize(default 30) when ON, reranker truncates toindexerMaxResultsfor the prompt.rerankerTimeoutMs(default 2000); on timeout original retrieval order is used and a WARN-level RAG log entry records the fallback.rerankResults(default OFF),rerankerModelName(defaultbge-reranker),rerankerShortlistSize,rerankerTimeoutMs— persisted viaDevoxxGenieStateService, exposed in RAG settings UI.SearchResultnow carriespreRerankRank+rerankerScore; both surface in the RAG log andAgentMcpLogPanelper hit.RerankerValidator(same pattern asNomicEmbedTextValidator) with aPULL_RERANKERaction wired intoRAGSettingsHandlerand the validator status panel.Why chat-completion scoring
Ollama lacks a dedicated rerank endpoint at time of writing, so
OllamaRerankerscores each candidate via a 0–10 relevance prompt, with bounded parallelism per call. When a native rerank endpoint lands, onlyscoreCandidateneeds to change.Tests
New unit tests cover: NoOp passthrough/truncation; OllamaReranker reordering, top-N truncation, timeout fallback, score normalization edge cases;
SemanticSearchServiceOFF toggle no-op, ON-toggle widening + routing through the reranker, reranker-exception fallback;RerankerValidatorfor enabled/disabled/missing-model/connection-failure paths. All 25 new tests pass; the rest of the suite is unaffected (same 210 pre-existing UI/controller failures onupstream/master).Closes TASK-214.