W1.1 — Bm25Ranker (pure-JS, always available)

[edelauna]

Context

A pure-JS BM25 ranker is the always-available baseline for dynamic tool loading. It works without any embedding provider configured, has no network dependency, and is good enough for the keyword-y queries that dominate MCP tool search ("send slack message," "read jira ticket").

Foundational for the dynamic loading feature; doesn't close a user-facing issue on its own.

Developer Notes

• New src/services/tools/Bm25Ranker.ts. Implements a Ranker interface: rank(query: string, items: ToolDoc[], k: number) → ToolDoc[].
• ToolDoc shape: { serverName: string, toolName: string, description: string }. The ranker tokenizes the concatenation ${serverName} ${toolName} ${description}.
• Tokenizer: lowercase, alphanumeric split on non-word chars. English-only acceptable for v1.
• Standard BM25 (k1=1.5, b=0.75). No external dependencies.
• Build the inverted index lazily on first rank() call; rebuild when the input set changes.
• Files: src/services/tools/Bm25Ranker.ts, src/services/tools/types.ts (Ranker interface, ToolDoc).
• Validation: unit tests on synthetic catalogs — query "send slack message" ranks slack.postMessage over jira.createIssue over postgres.query. Edge cases: empty query → empty result; query with no matches → empty result; identical-description ties stable-sorted by name.