Skip to content

feat(mcp): support multiple active index bindings (RAAE-1604)#629

Merged
vishal-bala merged 8 commits into
feature/raae-1603-mcp-multi-indexfrom
feature/raae-1604-config-runtime-model
Jun 30, 2026
Merged

feat(mcp): support multiple active index bindings (RAAE-1604)#629
vishal-bala merged 8 commits into
feature/raae-1603-mcp-multi-indexfrom
feature/raae-1604-config-runtime-model

Conversation

@vishal-bala

@vishal-bala vishal-bala commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Motivation

The RedisVL MCP server currently binds to exactly one Redis index per process. That single-binding assumption is enforced by a config validator and baked throughout the codebase — single-resource server state, single-binding convenience accessors on MCPConfig, and the search/upsert tools. Before the server can expose multiple logical indexes from a single endpoint (RAAE-1603), that assumption has to be removed and replaced with a real multi-binding model.

This PR (RAAE-1604) does exactly that, and nothing more: it reshapes the configuration and runtime model so the server can start, inspect, validate, and serve one or many bindings, while keeping existing single-index configs and callers behaving identically. It is the foundation the rest of the epic (discovery via list-indexes, index routing on search-records/upsert-records, docs) builds on, so it intentionally does not yet add any new request parameters or tools.

Implementation

The core of the change is a new immutable BindingRuntime (in redisvl/mcp/runtime.py) that bundles everything a tool call needs for one logical index: the binding config, the connected AsyncSearchIndex, its effective (inspected + overridden) schema, an optional vectorizer, the resolved native-hybrid-search capability, and the effective read-only flag. The server now holds a dict[str, BindingRuntime] keyed by logical id instead of a single set of _index/_vectorizer fields. Startup iterates every configured binding and inspects, validates, and initializes each one independently — each binding owns its own Redis client — with all-or-nothing teardown so a single bad binding fails startup cleanly without leaking connections.

On the config side, the "exactly one configured index binding" validator is gone (we now simply require at least one binding with non-blank ids), and the schema-inspection, runtime-mapping, and search-validation methods move from MCPConfig onto MCPIndexBindingConfig where they naturally belong per binding. The single-binding convenience accessors on MCPConfig are removed. Each binding gains optional description and read_only fields, and a binding's effective write availability is computed as global --read-only OR the per-index read_only. Tool resolution goes through a new server.resolve_binding(index_id) helper that defaults to the sole binding when one is configured (preserving backward compatibility) and returns an invalid_request error when an index is omitted with multiple bindings configured or when an unknown id is given. The search and upsert tools were re-threaded to operate on a resolved BindingRuntime rather than reaching into single-binding server accessors.

Additional notes:

  • Native-hybrid-search support is now probed eagerly per binding at startup and stored on the BindingRuntime, replacing the previous lazy single-index cache.
  • The concurrency semaphore is a single process-wide ceiling sized from the maximum max_concurrency across bindings; the request timeout is sourced per-binding and passed explicitly into run_guarded.
  • get_index() / get_vectorizer() are retained as thin convenience wrappers over resolve_binding(None).
  • Implemented test-first: new coverage for multi-binding config loading, description/read_only defaults, resolve_binding routing semantics, semaphore sizing, per-binding teardown, and three integration tests (multi-binding startup, global read-only override, and a single invalid binding failing startup), alongside the updated single-index tests that confirm backward compatibility.

Verification

  • mypy clean across all source files; black/isort formatted.
  • 182 MCP unit tests pass.
  • 44 MCP integration tests pass (2 skipped on Redis-version gates) against Redis 8.

🤖 Generated with Claude Code


Note

Medium Risk
MCP startup/teardown and binding resolution affect how indexes are served; the required run_guarded(..., timeout_seconds=) change breaks custom MCP extensions that call it directly.

Overview
MCP moves from a single enforced index binding to a dict of BindingRuntime entries: startup inspects and initializes each configured index independently (own client, vectorizer, hybrid probe, effective read-only), with resolve_binding(index_id) defaulting when only one index is configured and rejecting ambiguous or unknown ids. Config drops the “exactly one binding” rule and MCPConfig convenience accessors; per-binding description, read_only, and schema/search helpers live on MCPIndexBindingConfig. Search/upsert tools read from the resolved runtime; run_guarded now requires timeout_seconds= per binding (breaking for direct callers).

Also in this release: SearchIndex.drop_keys uses UNLINK instead of DEL; semantic router delete() removes the standalone route-config key; sql-redis>=0.7.1 with docs for hybrid_vector_search / FT.HYBRID; auto-release publishes to PyPI via pypa/gh-action-pypi-publish with OIDC; version 0.22.0.

Reviewed by Cursor Bugbot for commit bd2a28a. Bugbot is set up for automated code reviews on this repo. Configure here.


⚠️ Breaking change for downstream authors

RedisVLMCPServer.run_guarded(operation_name, awaitable) now requires a keyword-only timeout_seconds argument (sourced from each binding's request_timeout_seconds). Any code that subclasses RedisVLMCPServer or calls run_guarded directly (custom tools/plugins) must update its call sites to pass timeout_seconds=, otherwise it raises TypeError: run_guarded() missing 1 required keyword-only argument: 'timeout_seconds' at call time. This repository has no CHANGELOG file, so this note serves as the migration callout (per review feedback on #629).

@jit-ci

jit-ci Bot commented Jun 16, 2026

Copy link
Copy Markdown

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR


Security scan by Jit

Comment thread redisvl/mcp/tools/upsert.py
@vishal-bala

Copy link
Copy Markdown
Collaborator Author

CI is failing because of flaky tests that will be fixed by redis-developer/sql-redis#39

@vishal-bala vishal-bala requested a review from nkanu17 June 25, 2026 09:58
Comment thread redisvl/mcp/server.py
Comment thread redisvl/mcp/server.py
Comment thread redisvl/mcp/server.py Outdated
Comment thread redisvl/mcp/server.py

@nkanu17 nkanu17 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the design!

Binding runtime dataclass is nice and like how resolve binding works as the single routing entry point shared across all tools keeps the behavior consistent and how it's backward compatible for single binding servers works

One blocking issue before merge:

the teardown loop in _teardown_runtime does not protect subsequent bindings if _close_resources raises on an earlier one.

  • A failed disconnect on binding 1 will propagate out of the loop immediately, leaving bindings 2 through N with open Redis connections that are never closed.
  • In a partial startup failure this is exactly the path that gets exercised, so the leak is not hypothetical.
  • Wrapping each iteration in a try/except that logs and continues is a small fix and should be done before this lands.

Comment thread redisvl/mcp/server.py
Comment thread redisvl/mcp/server.py Outdated
@vishal-bala vishal-bala requested a review from nkanu17 June 30, 2026 11:15

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3e441b2. Configure here.

Comment thread redisvl/mcp/server.py

@nkanu17 nkanu17 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the changes! Looks good! 🚀

vishal-bala and others added 8 commits June 30, 2026 15:39
Treat the MCP `indexes` config as a real multi-binding map instead of
enforcing exactly one logical binding. This is the foundation for
multi-index MCP support (RAAE-1603); single-index configs and callers
behave identically.

Config:
- Drop the "exactly one configured index binding" validator; require at
  least one binding with non-blank ids.
- Add per-binding `description` and `read_only` fields.
- Move schema inspection / runtime-mapping / search validation methods
  onto MCPIndexBindingConfig and remove the single-binding convenience
  accessors from MCPConfig.

Runtime/server:
- Introduce an immutable BindingRuntime bundling a binding's config,
  index, effective schema, vectorizer, native-hybrid support, and
  effective read-only flag.
- Replace single-resource server state with a dict of BindingRuntime;
  startup inspects/validates/initializes every binding independently
  (one client per binding) with all-or-nothing teardown.
- Resolve native-hybrid support eagerly per binding.
- Size the concurrency semaphore from the max binding limit and pass a
  per-binding request timeout into run_guarded.
- Add resolve_binding(index_id): defaults to the sole binding, raises
  invalid_request for omitted-on-multi and unknown ids.
- Compute effective_read_only as global read-only OR per-binding read_only.

Tools:
- Re-thread search/upsert onto a resolved BindingRuntime instead of
  single-binding server accessors.

Tests follow TDD: multi-binding config/startup coverage, resolve_binding
semantics, semaphore sizing, per-binding teardown, and backward-compatible
single-index behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three issues found in review:

- upsert_records() never consulted rt.effective_read_only, so a binding
  configured with read_only: true could still accept writes when the
  global read_only flag was off. Add an explicit FORBIDDEN guard at the
  top of upsert_records() before any other processing.

- _teardown_runtime() did not protect subsequent bindings if
  _close_resources() raised on an earlier one. A network error on
  binding 1 would propagate out of the loop, leaving bindings 2-N with
  open Redis connections. Wrap each iteration in try/except and log the
  failure so teardown is best-effort across all bindings.

- _tools_registered was not reset in _teardown_runtime(), so a
  re-startup after a partial-startup failure would skip tool
  re-registration. Reset it alongside the other state clears.

Also adds a clarifying comment on the semaphore size semantics.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add regression tests for the behaviors introduced in 3a0d1fe, which shipped
without coverage:

- upsert-records rejects a write to a read-only binding with FORBIDDEN before
  any backend load is attempted.
- _teardown_runtime continues closing the remaining bindings when one binding's
  close raises, and fully clears terminal state (including _tools_registered)
  so a later startup re-registers tools.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both convenience methods called resolve_binding(None), which raises
invalid_request on any multi-index server — a latent footgun for tool authors
that only surfaced at runtime under a multi-binding config. They had no
non-test callers. Remove them and route all access through resolve_binding.

Integration tests that used them now use small mcp_binding_index /
mcp_binding_vectorizer test helpers in tests/conftest.py, which delegate to
resolve_binding and preserve the same "vectorizer is not configured" /
"has not been started" errors. Drop the now-dead mirror methods from the unit
test fakes.

Also align test_server_shutdown_disconnects_index_when_vectorizer_close_fails
with the best-effort teardown introduced in 3a0d1fe: a failing vectorizer
close is logged and swallowed (index still disconnected), so shutdown no
longer re-raises it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…1604)

Stop resetting _tools_registered in _teardown_runtime. MCP tools are registered
once on the FastMCP instance and their closures resolve the live binding at
call time, so they survive teardown and stay valid across a stop/start. The
lifecycle allows startup() again from STOPPED, and _register_tools() runs inside
_initialize_runtime_resources, so resetting the flag caused a restart to
re-register the same search-records/upsert-records names on the instance
(duplicate/broken restart).

The earlier "reset so a failed retry re-registers" concern does not occur: the
flag is only set True after registration succeeds, and registrations persist on
the instance regardless of binding teardown, so a retry that skips registration
still has working tools. Update the teardown test to assert registration
survives.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fulltext/hybrid MCP configs that omit `stopwords` inherit RedisVL's
TextQuery default of "english", which lazily loads the nltk stopwords corpus
(and attempts a network `nltk.download` at query time). That made auth/routing
tests transitively depend on the nltk corpus being present on the runner — an
unrelated, flaky failure mode (e.g. test_http_transport_enforces_jwt_auth
failing on a runner missing corpora/stopwords/english).

None of these tests assert stopword behavior; fulltext is only a vehicle for
exercising auth, routing, startup, and upsert. Set `stopwords: None` explicitly
on every such config so they no longer touch nltk. Tests that genuinely cover
search behavior already set `stopwords: None`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-1604)

Read read_scope/write_scope directly off the typed MCPAuthConfig instead of a
silent getattr(..., None). The old double-getattr would fail open — yielding
None and skipping the scope check — if the field were ever renamed; direct
access fails loud. Mirrors the same fix applied to list_indexes.py and keeps
the three tool wrappers consistent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… (RAAE-1604)

Move binding teardown into startup()'s exception handler so it covers every
post-begin failure, not just failures inside _initialize_runtime_resources.
Previously, if initialization succeeded (clients connected, tools registered)
and a later step such as _mark_running() raised, the handler marked the server
stopped without tearing down, leaking Redis clients and vectorizers; a later
shutdown() saw STOPPED and skipped cleanup.

_initialize_runtime_resources no longer needs its own teardown (a binding that
fails mid-build still closes its bare client inside _initialize_binding), giving
a single fail-closed teardown path. Add a regression test for the
init-succeeds-then-post-step-fails case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vishal-bala vishal-bala force-pushed the feature/raae-1604-config-runtime-model branch from 118b01d to bd2a28a Compare June 30, 2026 13:41
@vishal-bala vishal-bala changed the base branch from feature/raae-1603-mcp-multi-index to main June 30, 2026 20:11
@vishal-bala vishal-bala changed the base branch from main to feature/raae-1603-mcp-multi-index June 30, 2026 20:11
@vishal-bala vishal-bala merged commit f80e98f into feature/raae-1603-mcp-multi-index Jun 30, 2026
98 of 101 checks passed
@vishal-bala vishal-bala deleted the feature/raae-1604-config-runtime-model branch June 30, 2026 20:12
vishal-bala added a commit that referenced this pull request Jul 1, 2026
> Stacked on #629 (RAAE-1604). Review/merge that first; this PR targets
the 1604 branch so the diff is scoped to the discovery tool.

## Motivation

Once a single MCP server can expose multiple logical indexes
([RAAE-1604](https://redislabs.atlassian.net/browse/RAAE-1604)), clients
need a lightweight way to discover what's available so they can pick the
right index instead of guessing. This PR
([RAAE-1605](https://redislabs.atlassian.net/browse/RAAE-1605)) adds an
always-registered, read-only `list-indexes` tool for exactly that, and
it grounds discovery in the schema the server already inspected at
startup rather than asking users to re-declare field metadata in config.

## Implementation

The new tool (`redisvl/mcp/tools/list_indexes.py`) returns one entry per
configured binding, in configured order: the logical `id`, an optional
`description`, an `upsert_available` flag, the shared filterable
`fields`, and — only when explicitly configured — a `limits` object.
`upsert_available` is simply `not effective_read_only`, so it already
reflects both the global `--read-only` flag and the per-index
`read_only` policy resolved at startup. The `fields` list is built from
the binding's effective (inspected + overridden) schema that already
lives on its `BindingRuntime`, so the output stays consistent with what
the index actually contains; the vector field and the configured default
embed-source text field are omitted because they are implementation
inputs rather than fields a client would filter on. The Redis index name
(`redis_name`) is deliberately never exposed. Limits are included only
when the operator set them explicitly — detected via the runtime model's
`model_fields_set` — so defaults don't masquerade as deliberate
overrides; per the contract this covers `max_limit` and
`max_upsert_records`.

The tool is registered unconditionally during the server's tool
registration (alongside `search-records` and the
conditionally-registered `upsert-records`) and is gated by the same read
scope as search when auth is enabled, since it is read-only.

- Output is deterministic and ordered by configured binding.
- No new configuration surface or settings are required.

## Verification

- `mypy` clean; `black`/`isort` formatted.
- New unit coverage: field omission (vector + embed-source),
description/limits inclusion rules, `redis_name` secrecy, read-only
reflection, single- and multi-binding output, and tool registration.
- New integration test starts a real two-binding server (one vector, one
fulltext) and asserts the discovered fields come from the inspected
schema and follow the omission rules.
- Full MCP suite green: 178 unit + 45 integration (2 skipped on
Redis-version gates) against Redis 8.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

[RAAE-1604]:
https://redislabs.atlassian.net/browse/RAAE-1604?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[RAAE-1605]:
https://redislabs.atlassian.net/browse/RAAE-1605?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Read-only discovery metadata with no new config; behavior is additive
and gated like existing search tools when auth is enabled.
> 
> **Overview**
> Adds a read-only **`list-indexes`** MCP tool so clients can discover
logical indexes on multi-binding servers before calling
**`search-records`** or **`upsert-records`**.
> 
> The tool is **always registered** during `_register_tools` (alongside
search; upsert remains conditional). Each binding is returned in config
order with **`id`**, optional **`description`**, **`upsert_available`**
(`not effective_read_only`), filterable **`fields`** from the
startup-inspected schema (vector and default embed-source text omitted),
and **`limits`** only when **`max_limit`** / **`max_upsert_records`**
were explicitly set. **`redis_name`** is never exposed; empty bindings
raise **`RuntimeError`** like other pre-startup paths. When auth is on,
the tool uses the same **read scope** as search.
> 
> Unit and integration tests cover payload rules, registration, and a
two-binding startup scenario.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
45ce686. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants