Skip to content

feat(fts): 8e — sqlrite-mcp bm25_search tool#82

Merged
joaoh82 merged 1 commit into
mainfrom
feat/fts-mcp-tool
May 3, 2026
Merged

feat(fts): 8e — sqlrite-mcp bm25_search tool#82
joaoh82 merged 1 commit into
mainfrom
feat/fts-mcp-tool

Conversation

@joaoh82

@joaoh82 joaoh82 commented May 3, 2026

Copy link
Copy Markdown
Owner

Summary

Fifth sub-phase of Phase 8. Adds the bm25_search MCP tool — symmetric with vector_search (Phase 7h), surfaces FTS prominently to LLM clients driving SQLRite over MCP. Resolves Q9.

// MCP tool call
{
  "name": "bm25_search",
  "arguments": {
    "table": "docs",
    "column": "body",
    "query": "rust embedded database",
    "k": 5
  }
}
// → { rows: [...], k_requested: 5, ... }

Wraps SELECT * FROM <t> WHERE fts_match(<col>, 'q') ORDER BY bm25_score(<col>, 'q') DESC LIMIT k so the LLM doesn't have to remember the WHERE pre-filter, the DESC direction, or SQL string quoting. Picks up the Phase 8b optimizer hook automatically.

What landed

  • New tool sqlrite-mcp/src/tools/bm25_search.rs:
    • Metadata declares the JSON schema (required: table, column, query; optional: k clamped to 1..=100, default 10).
    • Pre-flight checks (table exists, column is TEXT, FTS index attached) surface clean errors before any SQL runs — useful for an LLM trying to recover.
    • SQL string-literal escaper handles embedded apostrophes per SQL standard (doubled quote).
  • Wiring sqlrite-mcp/src/tools/mod.rs:
    • mod bm25_search;
    • tools/list lists it after vector_search and before ask.
    • tools/call dispatches by name.

Test plan

MCP test count went 16 → 19 (+3 bm25-specific):

  • bm25_search_returns_top_ranked_rows — 4-doc corpus, the doc with tf=2 wins; the doc without rust is filtered out (verifies the optimizer hook + WHERE pre-filter both work end-to-end through MCP).
  • bm25_search_without_index_errors_clearly — actionable error pointing at the missing CREATE INDEX … USING fts step.
  • bm25_search_rejects_non_text_column — clean error on INTEGER columns.
  • tools_list_returns_expected_set_in_default_mode updated to include bm25_search.
  • Internal: tools::bm25_search::tests::sql_string_literal_doubles_quotes.
  • cargo test -p sqlrite-mcp19 / 19 green
  • cargo test --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs — engine 303 + 73 across other crates green
  • cargo fmt --all -- --check — no diff
  • cargo clippy --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --all-targets — no new warnings on bm25_search code
  • cargo doc --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --no-deps — no doc warnings on bm25_search

Out of scope (last sub-phase)

Concern Lands in
Docs sweep — docs/fts.md, docs/file-format.md, docs/supported-sql.md, docs/mcp.md (bm25_search entry) 8f

🤖 Generated with Claude Code

Phase 8 sub-phase 8e per docs/phase-8-plan.md (Q9). Adds the
`bm25_search` MCP tool, mirroring `vector_search` (Phase 7h) for
keyword retrieval. Surfaces FTS prominently to LLM clients driving
SQLRite over MCP.

User-visible:

  bm25_search(table, column, query, k?)
    → top-k rows by BM25 relevance, JSON-formatted

Wraps `SELECT * FROM <t> WHERE fts_match(<col>, 'q') ORDER BY
bm25_score(<col>, 'q') DESC LIMIT k` so the LLM doesn't need to
remember the WHERE-pre-filter, the DESC direction, or string
quoting. Picks up the Phase 8b optimizer hook automatically.

New:

- sqlrite-mcp/src/tools/bm25_search.rs — metadata + handle.
  Pre-flight checks (table exists, column is TEXT, FTS index
  attached) surface clean errors before any SQL runs. SQL
  string-literal escaper handles embedded apostrophes per SQL
  standard (doubled quote).
- sqlrite-mcp/src/tools/mod.rs — registers bm25_search in
  tools/list (post vector_search, pre ask) and tools/call.

Tests (sqlrite-mcp 16 → 19 passing; +3 bm25-specific):

- protocol::bm25_search_returns_top_ranked_rows — 4-doc corpus,
  the doc with tf=2 wins; the doc without 'rust' is filtered out.
- protocol::bm25_search_without_index_errors_clearly — actionable
  error pointing at the missing CREATE INDEX step.
- protocol::bm25_search_rejects_non_text_column — clean error on
  INTEGER columns.
- tools_list_returns_expected_set_in_default_mode updated to
  include bm25_search.
- Internal: tools::bm25_search::tests::sql_string_literal_doubles_quotes.

Out of scope (last sub-phase):
- Docs sweep (fts.md, file-format.md, supported-sql.md, mcp.md, …) → 8f

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joaoh82 joaoh82 merged commit 3c138f9 into main May 3, 2026
16 checks passed
@joaoh82 joaoh82 mentioned this pull request May 3, 2026
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant