Per-source content description from private config

[jonathanMLDev]

Problem

Every instruction block and every tool description describes mechanism (how to query, routing flow, presets, gates) and never content (what the index holds). With separate MCP entries for public vs private backends — or a merged multi-source server — the model's only differentiator is often the MCP server's display name (e.g. pinecone-search vs pinecone-search-private). Namespace routing therefore depends on vibes rather than explicit corpus signals.

Representative tool descriptions are mechanism-only (guided_query, namespace_router, list_namespaces, query, list_sources).

Why committed corpus text is not the fix: src/constants.ts ships in @will-cppa/pinecone-read-only-mcp to every npx consumer, fork, and mirror. A corpus-describing sentence for the private backend would become public metadata about what the Alliance indexes privately — the same disclosure class docs/CONFIGURATION.md flags as High risk. Security is enforced by deployment profiles (public-only vs merged config), not per-query MCP authorization.

Current gap: No per-source description field exists:

// src/core/server/source-config.ts — no description/label
export interface SourceDefinition {
  name: string;
  apiKey: string;
  indexName: string;
  sparseIndexName?: string;
  rerankModel?: string;
}

list_sources returns bare names only:

// src/core/server/response-schemas.ts
sources: z.array(z.string()),

The multi-source example (examples/multi-source/pinecone-sources.json.example) has no slot for a description either.

Acceptance Criteria

Config schema

• Add optional description (or label) field to SourceDefinition and the JSON config file schema (parseSourcesConfigFile / JsonSourceFile)
• Inline PINECONE_SOURCES format unchanged (no description slot — descriptions are config-file only)
• Update examples/multi-source/pinecone-sources.json.example with a commented or placeholder description field (generic placeholder text only — no real corpus names)
• Document the field in docs/CONFIGURATION.md under multi-source / deployment-profile guidance

Runtime surfacing

• Extend list_sources response to return { name, description? }[] (breaking change to response shape — document in CHANGELOG / MIGRATION.md)
• list_sources tool description updated to mention optional per-source descriptions
• Descriptions flow from PINECONE_CONFIG_FILE at runtime only — never from committed TypeScript constants

Optional enhancements (same issue or follow-up)

• Assemble a short instructions preamble at startup from configured source descriptions when present
• Bridge the existing library-only instructions?: string hook to a CLI/env path (e.g. top-level instructions field in the private config file)

Security / non-disclosure

• No private namespace names or corpus content committed to the repo
• Example and test fixtures use generic placeholders only
• Merged configs with real descriptions stay on staff machines per deployment-profile guidance

Implementation Notes

• Files likely touched: src/core/server/source-config.ts, src/core/server/response-schemas.ts, src/core/server/tools/list-sources-tool.ts, src/core/server/server-context.ts, examples/multi-source/pinecone-sources.json.example, docs/CONFIGURATION.md
• ctx.listSources() may need to return rich objects instead of string[]; update all call sites
• Add tests: config parsing with/without description, list_sources response shape, ensure committed constants still contain no corpus text
• For two separate MCP entries (public-only vs merged), operators put the corpus sentence in each instance's private client config — not in TypeScript
• PR Add multi-source Pinecone MCP support #187 resolved the two-server duplication driver; this issue addresses the remaining routing-signal gap

References

• src/core/server/source-config.ts — SourceDefinition, parseSourcesConfigFile
• src/core/server/tools/list-sources-tool.ts — list_sources registration
• docs/CONFIGURATION.md § Deployment profiles — public-only vs merged config separation
• docs/SECURITY.md — key/namespace redaction requirements