feat(mcp): support multiple active index bindings (RAAE-1604)#629
Conversation
🛡️ Jit Security Scan Results✅ No security findings were detected in this PR
Security scan by Jit
|
|
CI is failing because of flaky tests that will be fixed by redis-developer/sql-redis#39 |
There was a problem hiding this comment.
I really like the design!
Binding runtime dataclass is nice and like how resolve binding works as the single routing entry point shared across all tools keeps the behavior consistent and how it's backward compatible for single binding servers works
One blocking issue before merge:
the teardown loop in _teardown_runtime does not protect subsequent bindings if _close_resources raises on an earlier one.
- A failed disconnect on binding 1 will propagate out of the loop immediately, leaving bindings 2 through N with open Redis connections that are never closed.
- In a partial startup failure this is exactly the path that gets exercised, so the leak is not hypothetical.
- Wrapping each iteration in a try/except that logs and continues is a small fix and should be done before this lands.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 3e441b2. Configure here.
nkanu17
left a comment
There was a problem hiding this comment.
Thank you for the changes! Looks good! 🚀
Treat the MCP `indexes` config as a real multi-binding map instead of enforcing exactly one logical binding. This is the foundation for multi-index MCP support (RAAE-1603); single-index configs and callers behave identically. Config: - Drop the "exactly one configured index binding" validator; require at least one binding with non-blank ids. - Add per-binding `description` and `read_only` fields. - Move schema inspection / runtime-mapping / search validation methods onto MCPIndexBindingConfig and remove the single-binding convenience accessors from MCPConfig. Runtime/server: - Introduce an immutable BindingRuntime bundling a binding's config, index, effective schema, vectorizer, native-hybrid support, and effective read-only flag. - Replace single-resource server state with a dict of BindingRuntime; startup inspects/validates/initializes every binding independently (one client per binding) with all-or-nothing teardown. - Resolve native-hybrid support eagerly per binding. - Size the concurrency semaphore from the max binding limit and pass a per-binding request timeout into run_guarded. - Add resolve_binding(index_id): defaults to the sole binding, raises invalid_request for omitted-on-multi and unknown ids. - Compute effective_read_only as global read-only OR per-binding read_only. Tools: - Re-thread search/upsert onto a resolved BindingRuntime instead of single-binding server accessors. Tests follow TDD: multi-binding config/startup coverage, resolve_binding semantics, semaphore sizing, per-binding teardown, and backward-compatible single-index behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three issues found in review: - upsert_records() never consulted rt.effective_read_only, so a binding configured with read_only: true could still accept writes when the global read_only flag was off. Add an explicit FORBIDDEN guard at the top of upsert_records() before any other processing. - _teardown_runtime() did not protect subsequent bindings if _close_resources() raised on an earlier one. A network error on binding 1 would propagate out of the loop, leaving bindings 2-N with open Redis connections. Wrap each iteration in try/except and log the failure so teardown is best-effort across all bindings. - _tools_registered was not reset in _teardown_runtime(), so a re-startup after a partial-startup failure would skip tool re-registration. Reset it alongside the other state clears. Also adds a clarifying comment on the semaphore size semantics. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add regression tests for the behaviors introduced in 3a0d1fe, which shipped without coverage: - upsert-records rejects a write to a read-only binding with FORBIDDEN before any backend load is attempted. - _teardown_runtime continues closing the remaining bindings when one binding's close raises, and fully clears terminal state (including _tools_registered) so a later startup re-registers tools. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both convenience methods called resolve_binding(None), which raises invalid_request on any multi-index server — a latent footgun for tool authors that only surfaced at runtime under a multi-binding config. They had no non-test callers. Remove them and route all access through resolve_binding. Integration tests that used them now use small mcp_binding_index / mcp_binding_vectorizer test helpers in tests/conftest.py, which delegate to resolve_binding and preserve the same "vectorizer is not configured" / "has not been started" errors. Drop the now-dead mirror methods from the unit test fakes. Also align test_server_shutdown_disconnects_index_when_vectorizer_close_fails with the best-effort teardown introduced in 3a0d1fe: a failing vectorizer close is logged and swallowed (index still disconnected), so shutdown no longer re-raises it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…1604) Stop resetting _tools_registered in _teardown_runtime. MCP tools are registered once on the FastMCP instance and their closures resolve the live binding at call time, so they survive teardown and stay valid across a stop/start. The lifecycle allows startup() again from STOPPED, and _register_tools() runs inside _initialize_runtime_resources, so resetting the flag caused a restart to re-register the same search-records/upsert-records names on the instance (duplicate/broken restart). The earlier "reset so a failed retry re-registers" concern does not occur: the flag is only set True after registration succeeds, and registrations persist on the instance regardless of binding teardown, so a retry that skips registration still has working tools. Update the teardown test to assert registration survives. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fulltext/hybrid MCP configs that omit `stopwords` inherit RedisVL's TextQuery default of "english", which lazily loads the nltk stopwords corpus (and attempts a network `nltk.download` at query time). That made auth/routing tests transitively depend on the nltk corpus being present on the runner — an unrelated, flaky failure mode (e.g. test_http_transport_enforces_jwt_auth failing on a runner missing corpora/stopwords/english). None of these tests assert stopword behavior; fulltext is only a vehicle for exercising auth, routing, startup, and upsert. Set `stopwords: None` explicitly on every such config so they no longer touch nltk. Tests that genuinely cover search behavior already set `stopwords: None`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-1604) Read read_scope/write_scope directly off the typed MCPAuthConfig instead of a silent getattr(..., None). The old double-getattr would fail open — yielding None and skipping the scope check — if the field were ever renamed; direct access fails loud. Mirrors the same fix applied to list_indexes.py and keeps the three tool wrappers consistent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… (RAAE-1604) Move binding teardown into startup()'s exception handler so it covers every post-begin failure, not just failures inside _initialize_runtime_resources. Previously, if initialization succeeded (clients connected, tools registered) and a later step such as _mark_running() raised, the handler marked the server stopped without tearing down, leaking Redis clients and vectorizers; a later shutdown() saw STOPPED and skipped cleanup. _initialize_runtime_resources no longer needs its own teardown (a binding that fails mid-build still closes its bare client inside _initialize_binding), giving a single fail-closed teardown path. Add a regression test for the init-succeeds-then-post-step-fails case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
118b01d to
bd2a28a
Compare
f80e98f
into
feature/raae-1603-mcp-multi-index
> Stacked on #629 (RAAE-1604). Review/merge that first; this PR targets the 1604 branch so the diff is scoped to the discovery tool. ## Motivation Once a single MCP server can expose multiple logical indexes ([RAAE-1604](https://redislabs.atlassian.net/browse/RAAE-1604)), clients need a lightweight way to discover what's available so they can pick the right index instead of guessing. This PR ([RAAE-1605](https://redislabs.atlassian.net/browse/RAAE-1605)) adds an always-registered, read-only `list-indexes` tool for exactly that, and it grounds discovery in the schema the server already inspected at startup rather than asking users to re-declare field metadata in config. ## Implementation The new tool (`redisvl/mcp/tools/list_indexes.py`) returns one entry per configured binding, in configured order: the logical `id`, an optional `description`, an `upsert_available` flag, the shared filterable `fields`, and — only when explicitly configured — a `limits` object. `upsert_available` is simply `not effective_read_only`, so it already reflects both the global `--read-only` flag and the per-index `read_only` policy resolved at startup. The `fields` list is built from the binding's effective (inspected + overridden) schema that already lives on its `BindingRuntime`, so the output stays consistent with what the index actually contains; the vector field and the configured default embed-source text field are omitted because they are implementation inputs rather than fields a client would filter on. The Redis index name (`redis_name`) is deliberately never exposed. Limits are included only when the operator set them explicitly — detected via the runtime model's `model_fields_set` — so defaults don't masquerade as deliberate overrides; per the contract this covers `max_limit` and `max_upsert_records`. The tool is registered unconditionally during the server's tool registration (alongside `search-records` and the conditionally-registered `upsert-records`) and is gated by the same read scope as search when auth is enabled, since it is read-only. - Output is deterministic and ordered by configured binding. - No new configuration surface or settings are required. ## Verification - `mypy` clean; `black`/`isort` formatted. - New unit coverage: field omission (vector + embed-source), description/limits inclusion rules, `redis_name` secrecy, read-only reflection, single- and multi-binding output, and tool registration. - New integration test starts a real two-binding server (one vector, one fulltext) and asserts the discovered fields come from the inspected schema and follow the omission rules. - Full MCP suite green: 178 unit + 45 integration (2 skipped on Redis-version gates) against Redis 8. 🤖 Generated with [Claude Code](https://claude.com/claude-code) [RAAE-1604]: https://redislabs.atlassian.net/browse/RAAE-1604?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [RAAE-1605]: https://redislabs.atlassian.net/browse/RAAE-1605?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Read-only discovery metadata with no new config; behavior is additive and gated like existing search tools when auth is enabled. > > **Overview** > Adds a read-only **`list-indexes`** MCP tool so clients can discover logical indexes on multi-binding servers before calling **`search-records`** or **`upsert-records`**. > > The tool is **always registered** during `_register_tools` (alongside search; upsert remains conditional). Each binding is returned in config order with **`id`**, optional **`description`**, **`upsert_available`** (`not effective_read_only`), filterable **`fields`** from the startup-inspected schema (vector and default embed-source text omitted), and **`limits`** only when **`max_limit`** / **`max_upsert_records`** were explicitly set. **`redis_name`** is never exposed; empty bindings raise **`RuntimeError`** like other pre-startup paths. When auth is on, the tool uses the same **read scope** as search. > > Unit and integration tests cover payload rules, registration, and a two-binding startup scenario. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 45ce686. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Motivation
The RedisVL MCP server currently binds to exactly one Redis index per process. That single-binding assumption is enforced by a config validator and baked throughout the codebase — single-resource server state, single-binding convenience accessors on
MCPConfig, and the search/upsert tools. Before the server can expose multiple logical indexes from a single endpoint (RAAE-1603), that assumption has to be removed and replaced with a real multi-binding model.This PR (RAAE-1604) does exactly that, and nothing more: it reshapes the configuration and runtime model so the server can start, inspect, validate, and serve one or many bindings, while keeping existing single-index configs and callers behaving identically. It is the foundation the rest of the epic (discovery via
list-indexes, index routing onsearch-records/upsert-records, docs) builds on, so it intentionally does not yet add any new request parameters or tools.Implementation
The core of the change is a new immutable
BindingRuntime(inredisvl/mcp/runtime.py) that bundles everything a tool call needs for one logical index: the binding config, the connectedAsyncSearchIndex, its effective (inspected + overridden) schema, an optional vectorizer, the resolved native-hybrid-search capability, and the effective read-only flag. The server now holds adict[str, BindingRuntime]keyed by logical id instead of a single set of_index/_vectorizerfields. Startup iterates every configured binding and inspects, validates, and initializes each one independently — each binding owns its own Redis client — with all-or-nothing teardown so a single bad binding fails startup cleanly without leaking connections.On the config side, the "exactly one configured index binding" validator is gone (we now simply require at least one binding with non-blank ids), and the schema-inspection, runtime-mapping, and search-validation methods move from
MCPConfigontoMCPIndexBindingConfigwhere they naturally belong per binding. The single-binding convenience accessors onMCPConfigare removed. Each binding gains optionaldescriptionandread_onlyfields, and a binding's effective write availability is computed as global--read-onlyOR the per-indexread_only. Tool resolution goes through a newserver.resolve_binding(index_id)helper that defaults to the sole binding when one is configured (preserving backward compatibility) and returns aninvalid_requesterror when an index is omitted with multiple bindings configured or when an unknown id is given. The search and upsert tools were re-threaded to operate on a resolvedBindingRuntimerather than reaching into single-binding server accessors.Additional notes:
BindingRuntime, replacing the previous lazy single-index cache.max_concurrencyacross bindings; the request timeout is sourced per-binding and passed explicitly intorun_guarded.get_index()/get_vectorizer()are retained as thin convenience wrappers overresolve_binding(None).description/read_onlydefaults,resolve_bindingrouting semantics, semaphore sizing, per-binding teardown, and three integration tests (multi-binding startup, global read-only override, and a single invalid binding failing startup), alongside the updated single-index tests that confirm backward compatibility.Verification
mypyclean across all source files;black/isortformatted.🤖 Generated with Claude Code
Note
Medium Risk
MCP startup/teardown and binding resolution affect how indexes are served; the required
run_guarded(..., timeout_seconds=)change breaks custom MCP extensions that call it directly.Overview
MCP moves from a single enforced index binding to a
dictofBindingRuntimeentries: startup inspects and initializes each configured index independently (own client, vectorizer, hybrid probe, effective read-only), withresolve_binding(index_id)defaulting when only one index is configured and rejecting ambiguous or unknown ids. Config drops the “exactly one binding” rule andMCPConfigconvenience accessors; per-bindingdescription,read_only, and schema/search helpers live onMCPIndexBindingConfig. Search/upsert tools read from the resolved runtime;run_guardednow requirestimeout_seconds=per binding (breaking for direct callers).Also in this release:
SearchIndex.drop_keysusesUNLINKinstead ofDEL; semantic routerdelete()removes the standalone route-config key;sql-redis>=0.7.1with docs forhybrid_vector_search/ FT.HYBRID; auto-release publishes to PyPI viapypa/gh-action-pypi-publishwith OIDC; version 0.22.0.Reviewed by Cursor Bugbot for commit bd2a28a. Bugbot is set up for automated code reviews on this repo. Configure here.
RedisVLMCPServer.run_guarded(operation_name, awaitable)now requires a keyword-onlytimeout_secondsargument (sourced from each binding'srequest_timeout_seconds). Any code that subclassesRedisVLMCPServeror callsrun_guardeddirectly (custom tools/plugins) must update its call sites to passtimeout_seconds=, otherwise it raisesTypeError: run_guarded() missing 1 required keyword-only argument: 'timeout_seconds'at call time. This repository has no CHANGELOG file, so this note serves as the migration callout (per review feedback on #629).