Core vs Alliance MCP instructions and suggest-flow gate (cppalliance#125)

jonathanMLDev · zho · web-flow · commit b59469037d36 · 2026-06-03T14:35:24.000-04:00
* Core vs Alliance MCP instructions and suggest-flow gate

* addressed ai review

* Fix instruction numbering in constants.ts

---------

Co-authored-by: zho &lt;jornathanm910923@gmail.com&gt;
diff --git a/.env.example b/.env.example
@@ -27,8 +27,8 @@ PINECONE_READ_ONLY_MCP_LOG_FORMAT=text
 # Optional: Namespace + suggestion-flow cache TTL in seconds (default: 1800 = 30min)
 # PINECONE_CACHE_TTL_SECONDS=1800
 
-# Optional: Disable the suggest_query_params flow gate (default: false).
-# WARNING: turning this on bypasses the safety guard. A WARN log fires at startup.
+# Optional: Suggest-flow gate (PINECONE_DISABLE_SUGGEST_FLOW).
+# Core resolveConfig default: true (gate off). Alliance CLI default: false (gate on).
 # PINECONE_DISABLE_SUGGEST_FLOW=false
 
 # Optional: Request timeout for Pinecone calls, in milliseconds (default: 15000)
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -11,6 +11,7 @@ Tagged releases are published to npm from GitHub Actions when a **GitHub Release
 ### Changed
 
 - **Breaking (core):** `resolveConfig` requires a Pinecone index name and no longer applies Alliance index/rerank defaults. Removed exported `DEFAULT_INDEX_NAME` and `DEFAULT_RERANK_MODEL` from the package root. Rerank is opt-in when `PINECONE_RERANK_MODEL` / `rerankModel` is unset.
+- **Breaking (core):** `setupCoreServer` MCP `instructions` use `CORE_SERVER_INSTRUCTIONS` (no `guided_query` / `suggest_query_params`). `resolveConfig` defaults `disableSuggestFlow` to `true` so `query` / `count` / `query_documents` work without Alliance tools. Alliance CLI / `resolveAllianceConfig` unchanged: gate on by default, `ALLIANCE_SERVER_INSTRUCTIONS`.
 - **Alliance CLI / `resolveAllianceConfig`:** When index or rerank env/CLI values are omitted, defaults remain `rag-hybrid` and `bge-reranker-v2-m3` (API-key-only MCP configs unchanged). See [examples/alliance/.env.example](examples/alliance/.env.example).
 ### Added
 
diff --git a/README.md b/README.md
@@ -134,7 +134,7 @@ Treat **`setupCoreServer()` / `setupAllianceServer()` as one logical server per
 - **Generic bridge only:** `import { setupCoreServer, teardownServer, ... } from '@will-cppa/pinecone-read-only-mcp'`
 - **Full Alliance surface (CLI parity):** `import { setupAllianceServer, resolveAllianceConfig } from '@will-cppa/pinecone-read-only-mcp/alliance'`
 
-For the **generic bridge only**, see [examples/quickstart/mcp-demo.ts](examples/quickstart/mcp-demo.ts) (`setupCoreServer` + `resolveConfig` with required `indexName`). For the **full Alliance surface**, use `resolveAllianceConfig({ apiKey, ... })` (Alliance index/rerank defaults when omitted, same as the CLI) → `setPineconeClient(new PineconeClient(...))` → `await setupAllianceServer(config)` → connect one MCP transport. See [examples/alliance/library-embedding-demo.ts](examples/alliance/library-embedding-demo.ts) and [docs/TOOLS.md](docs/TOOLS.md#suggest-flow-gate).
+For the **generic bridge only**, see [examples/quickstart/mcp-demo.ts](examples/quickstart/mcp-demo.ts) (`setupCoreServer` + `resolveConfig` with required `indexName`; suggest-flow gate **off** by default). For the **full Alliance surface**, use `resolveAllianceConfig({ apiKey, ... })` (Alliance index/rerank defaults when omitted, suggest-flow gate **on** by default, same as the CLI) → `setPineconeClient(new PineconeClient(...))` → `await setupAllianceServer(config)` → connect one MCP transport. See [examples/alliance/library-embedding-demo.ts](examples/alliance/library-embedding-demo.ts) and [docs/TOOLS.md](docs/TOOLS.md#suggest-flow-gate).
 
 ### Custom URL generators
 
diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md
@@ -23,7 +23,7 @@ Configuration is built from **CLI flags** (when using the binary), **environment
 | `logFormat` | `logFormat` / `PINECONE_READ_ONLY_MCP_LOG_FORMAT` | `text` or `json` |
 | `cacheTtlMs` | `cacheTtlSeconds` / `PINECONE_CACHE_TTL_SECONDS` | `1800` seconds → ms |
 | `requestTimeoutMs` | `requestTimeoutMs` / `PINECONE_REQUEST_TIMEOUT_MS` | `15000` |
-| `disableSuggestFlow` | `disableSuggestFlow` / `PINECONE_DISABLE_SUGGEST_FLOW` | `false` (bool parsing: true/1/yes/on) |
+| `disableSuggestFlow` | `disableSuggestFlow` / `PINECONE_DISABLE_SUGGEST_FLOW` | **Core `resolveConfig`:** `true` (gate off). **Alliance `resolveAllianceConfig` / CLI:** `false` (gate on). Bool parsing: true/1/yes/on |
 | `checkIndexes` | `checkIndexes` / `PINECONE_CHECK_INDEXES` | `false` |
 
 **Throws** if `apiKey` or `indexName` is missing after trim.
@@ -32,10 +32,10 @@ For the full Alliance tool surface (including `suggest_query_params`, `guided_qu
 
 ### Core vs Alliance resolvers
 
-| Resolver | When to use | Index when unset | Rerank when unset |
-| -------- | ------------- | ---------------- | ----------------- |
-| `resolveConfig` | Package root, `setupCoreServer`, quickstart | **Throws** | Omitted (no rerank) |
-| `resolveAllianceConfig` | Published CLI, `setupAllianceServer` | `rag-hybrid` | `bge-reranker-v2-m3` |
+| Resolver | When to use | Index when unset | Rerank when unset | Suggest gate when unset |
+| -------- | ------------- | ---------------- | ----------------- | ----------------------- |
+| `resolveConfig` | Package root, `setupCoreServer`, quickstart | **Throws** | Omitted (no rerank) | Off (`disableSuggestFlow: true`) |
+| `resolveAllianceConfig` | Published CLI, `setupAllianceServer` | `rag-hybrid` | `bge-reranker-v2-m3` | On (`disableSuggestFlow: false`) |
 
 C++ Alliance deployers can copy [examples/alliance/.env.example](../examples/alliance/.env.example). Constants: `ALLIANCE_DEFAULT_INDEX_NAME` / `ALLIANCE_DEFAULT_RERANK_MODEL` from `@will-cppa/pinecone-read-only-mcp/alliance`.
 
diff --git a/docs/FAQ.md b/docs/FAQ.md
@@ -2,7 +2,7 @@
 
 ## Why does `query` return `FLOW_GATE`?
 
-Call **`suggest_query_params`** for that namespace first (within the cache TTL). Or use **`guided_query`**, which performs suggestion internally.
+On **Alliance** setups (`resolveAllianceConfig` / CLI), call **`suggest_query_params`** for that namespace first (within the cache TTL), or use **`guided_query`**, which performs suggestion internally. On **core** `setupCoreServer` / `resolveConfig`, the gate is **off** by default; if you enabled it (`PINECONE_DISABLE_SUGGEST_FLOW=false`), you need `suggest_query_params` or must switch to Alliance tools.
 
 ## Does `keyword_search` need `suggest_query_params`?
 
@@ -14,7 +14,7 @@ They are unified into **`query`** with `preset`: `fast`, `detailed`, or `full`.
 
 ## How do I disable the suggest gate for testing?
 
-Set **`PINECONE_DISABLE_SUGGEST_FLOW=true`** or pass **`--disable-suggest-flow`**. Prefer fixing the client flow in production.
+**Core:** gate is already off via `resolveConfig` default. **Alliance:** set **`PINECONE_DISABLE_SUGGEST_FLOW=true`** or pass **`--disable-suggest-flow`**. Prefer fixing the client flow in production when using Alliance.
 
 ## Where are benchmarks?
 
diff --git a/docs/TOOLS.md b/docs/TOOLS.md
@@ -1,14 +1,23 @@
 # MCP tools reference
 
-The server registers **nine** tools. Unless noted, failures return MCP `isError: true` with JSON matching `ToolError` (see [MIGRATION.md](./MIGRATION.md) and [README error table](../README.md#error-responses)).
+Unless noted, failures return MCP `isError: true` with JSON matching `ToolError` (see [MIGRATION.md](./MIGRATION.md) and [README error table](../README.md#error-responses)).
+
+## Core vs Alliance tool surface
+
+| Setup | Tools | MCP instructions |
+| ----- | ----- | ------------------ |
+| `setupCoreServer` (package root) | **7:** `list_namespaces`, `namespace_router`, `count`, `query`, `keyword_search`, `query_documents`, `generate_urls` | `CORE_SERVER_INSTRUCTIONS` — no `guided_query` or `suggest_query_params` |
+| `setupAllianceServer` / published CLI | **9:** core tools plus `suggest_query_params`, `guided_query` | `ALLIANCE_SERVER_INSTRUCTIONS` — includes guided/suggest quickstart |
 
 ## Suggest-flow gate
 
-Tools **`query`**, **`count`**, and **`query_documents`** require a prior successful **`suggest_query_params`** call for the **same namespace string** within the cache TTL (see `PINECONE_CACHE_TTL_SECONDS`). The gate is in-process memory (`requireSuggested`).
+When **`disableSuggestFlow`** is **`false`** (Alliance default via `resolveAllianceConfig` / CLI), tools **`query`**, **`count`**, and **`query_documents`** require a prior successful **`suggest_query_params`** call for the **same namespace string** within the cache TTL (see `PINECONE_CACHE_TTL_SECONDS`). The gate is in-process memory (`requireSuggested`).
+
+When **`disableSuggestFlow`** is **`true`** (core default via `resolveConfig`), the gate is bypassed — suitable for `setupCoreServer` embedders that do not register `suggest_query_params`.
 
 **Namespace consistency:** use the **exact same** `namespace` value (including trimming — avoid leading/trailing spaces in one call and not the other) for `suggest_query_params` and downstream gated tools. Mismatches yield `FLOW_GATE` with a suggestion to call `suggest_query_params` first.
 
-**Bypass:** set `PINECONE_DISABLE_SUGGEST_FLOW=true` or CLI `--disable-suggest-flow` (not recommended for production).
+**Core:** gate off by default; set `PINECONE_DISABLE_SUGGEST_FLOW=false` or `disableSuggestFlow: false` to enable the gate. **Alliance:** gate on by default; set `PINECONE_DISABLE_SUGGEST_FLOW=true` or CLI `--disable-suggest-flow` to bypass (not recommended for production).
 
 ---
 
diff --git a/examples/quickstart/.env.example b/examples/quickstart/.env.example
@@ -6,8 +6,8 @@ PINECONE_INDEX_NAME=my-mcp-demo
 # Optional; default is {PINECONE_INDEX_NAME}-sparse
 # PINECONE_SPARSE_INDEX_NAME=my-mcp-demo-sparse
 
-# Required for core-only demos (no suggest_query_params tool on setupCoreServer)
-PINECONE_DISABLE_SUGGEST_FLOW=true
+# Optional: set PINECONE_DISABLE_SUGGEST_FLOW=false to enable the suggest-flow gate on core
+# (requires suggest_query_params — use setupAllianceServer / Alliance CLI instead)
 
 # Optional rerank (omit if your Pinecone project does not support the model)
 # PINECONE_RERANK_MODEL=your-rerank-model-id
diff --git a/examples/quickstart/README.md b/examples/quickstart/README.md
@@ -38,7 +38,6 @@ Edit `.env`:
 
 - `PINECONE_API_KEY` — your API key
 - `PINECONE_INDEX_NAME` — dense index name (e.g. `my-mcp-demo`)
-- `PINECONE_DISABLE_SUGGEST_FLOW=true` — required for `setupCoreServer` (no `suggest_query_params` tool)
 
 Optional: `PINECONE_SPARSE_INDEX_NAME` if the sparse index is not `{dense}-sparse`.
 
@@ -68,12 +67,11 @@ npx tsx examples/quickstart/mcp-demo.ts
 
 ## 5. Optional: CLI (stdio MCP)
 
-After seeding, you can run the published binary with the same env (still disable the suggest-flow gate for ad-hoc `query` without Alliance tools):
+After seeding, you can run the published **Alliance** CLI with the same index (suggest-flow gate on by default; use `suggest_query_params` or `guided_query`):
 
 ```bash
 export PINECONE_API_KEY=...
 export PINECONE_INDEX_NAME=my-mcp-demo
-export PINECONE_DISABLE_SUGGEST_FLOW=true
 npx @will-cppa/pinecone-read-only-mcp
 ```
 
@@ -83,7 +81,7 @@ For the **full** Alliance tool surface (`suggest_query_params`, `guided_query`,
 
 | Symptom | Check |
 | ------- | ----- |
-| `FLOW_GATE` on `query` / `count` | Set `PINECONE_DISABLE_SUGGEST_FLOW=true` or use Alliance server + `suggest_query_params` |
+| `FLOW_GATE` on `query` / `count` | Core: gate is off by default via `resolveConfig`. If you set `PINECONE_DISABLE_SUGGEST_FLOW=false`, call `suggest_query_params` first or use Alliance server + `guided_query` |
 | Empty `list_namespaces` | Run `seed-data.ts`; confirm namespace `quickstart` and index names in `.env` |
 | Upsert / search errors | Indexes must support integrated embedding and `chunk_text` as the text field |
 | Hybrid partial results | Both dense and sparse indexes must contain the same record IDs in the same namespace |
diff --git a/examples/quickstart/mcp-demo.ts b/examples/quickstart/mcp-demo.ts
@@ -2,7 +2,7 @@
  * Generic quickstart: embed setupCoreServer and call core MCP tools in-process.
  *
  * Flow: list_namespaces → count → query (preset fast, no rerank).
- * Requires seeded data (seed-data.ts) and PINECONE_DISABLE_SUGGEST_FLOW=true.
+ * Requires seeded data (seed-data.ts). Core resolveConfig disables the suggest-flow gate by default.
  */
 
 import { Client } from '@modelcontextprotocol/sdk/client/index.js';
@@ -46,7 +46,6 @@ async function main(): Promise<void> {
   const config = resolveConfig({
     apiKey,
     indexName,
-    disableSuggestFlow: true,
   });
 
   setPineconeClient(
diff --git a/src/alliance/config.test.ts b/src/alliance/config.test.ts
@@ -13,6 +13,11 @@ describe('resolveAllianceConfig', () => {
     expect(cfg.rerankModel).toBe(ALLIANCE_DEFAULT_RERANK_MODEL);
   });
 
+  it('defaults disableSuggestFlow to false (suggest gate on for Alliance)', () => {
+    const cfg = resolveAllianceConfig({ apiKey: 'sk-test' }, { PINECONE_API_KEY: 'sk-test' });
+    expect(cfg.disableSuggestFlow).toBe(false);
+  });
+
   it('applies Alliance rerank default when env and overrides omit rerankModel but index is set', () => {
     const cfg = resolveAllianceConfig(
       { apiKey: 'sk-test', indexName: 'my-index' },
diff --git a/src/alliance/config.ts b/src/alliance/config.ts
@@ -3,6 +3,7 @@
  */
 
 import {
+  asBool,
   resolveConfig,
   trimOptional,
   type ConfigOverrides,
@@ -34,5 +35,8 @@ export function resolveAllianceConfig(
     trimOptional(overrides.rerankModel) ??
     trimOptional(env['PINECONE_RERANK_MODEL']) ??
     ALLIANCE_DEFAULT_RERANK_MODEL;
-  return resolveConfig({ ...overrides, indexName, rerankModel }, env);
+  const cfg = resolveConfig({ ...overrides, indexName, rerankModel }, env);
+  const disableSuggestFlow =
+    overrides.disableSuggestFlow ?? asBool(env['PINECONE_DISABLE_SUGGEST_FLOW'], false);
+  return { ...cfg, disableSuggestFlow };
 }
diff --git a/src/alliance/setup.ts b/src/alliance/setup.ts
@@ -1,4 +1,5 @@
 import type { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
+import { ALLIANCE_SERVER_INSTRUCTIONS } from '../constants.js';
 import type { ServerConfig } from '../core/config.js';
 import { resolveAllianceConfig } from './config.js';
 import { setupCoreServer } from '../core/setup.js';
@@ -13,7 +14,9 @@ import { registerSuggestQueryParamsTool } from './tools/suggest-query-params-too
  * When `config` is omitted, resolves env via {@link resolveAllianceConfig} (Alliance index/rerank defaults when unset).
  */
 export async function setupAllianceServer(config?: ServerConfig): Promise<McpServer> {
-  const server = await setupCoreServer(config ?? resolveAllianceConfig({}));
+  const server = await setupCoreServer(config ?? resolveAllianceConfig({}), {
+    instructions: ALLIANCE_SERVER_INSTRUCTIONS,
+  });
   registerBuiltinUrlGenerators();
   registerSuggestQueryParamsTool(server);
   registerGuidedQueryTool(server);
diff --git a/src/constants.test.ts b/src/constants.test.ts
@@ -0,0 +1,22 @@
+import { describe, expect, it } from 'vitest';
+import {
+  ALLIANCE_SERVER_INSTRUCTIONS,
+  CORE_SERVER_INSTRUCTIONS,
+  SERVER_INSTRUCTIONS,
+} from './constants.js';
+
+describe('server instructions', () => {
+  it('CORE_SERVER_INSTRUCTIONS does not reference Alliance-only tools', () => {
+    expect(CORE_SERVER_INSTRUCTIONS).not.toMatch(/guided_query/);
+    expect(CORE_SERVER_INSTRUCTIONS).not.toMatch(/suggest_query_params/);
+  });
+
+  it('ALLIANCE_SERVER_INSTRUCTIONS includes guided_query and suggest_query_params', () => {
+    expect(ALLIANCE_SERVER_INSTRUCTIONS).toMatch(/guided_query/);
+    expect(ALLIANCE_SERVER_INSTRUCTIONS).toMatch(/suggest_query_params/);
+  });
+
+  it('SERVER_INSTRUCTIONS aliases Alliance instructions', () => {
+    expect(SERVER_INSTRUCTIONS).toBe(ALLIANCE_SERVER_INSTRUCTIONS);
+  });
+});
diff --git a/src/constants.ts b/src/constants.ts
@@ -32,9 +32,7 @@ export const QUERY_DOCUMENTS_MAX_CHUNKS = 500;
 export const SERVER_NAME = 'Pinecone Read-Only MCP';
 export { SERVER_VERSION } from './server-version.js';
 
-export const SERVER_INSTRUCTIONS = `Quickstart for AI clients: for most user questions, call \`guided_query\` with the user's question — it does namespace routing, suggestion, and execution in one shot and returns a decision_trace you can show the user. For the **core** entrypoint (\`setupCoreServer\` / package-root \`resolveConfig\`), \`PINECONE_INDEX_NAME\` (or \`--index-name\`) is required in addition to \`PINECONE_API_KEY\`. The **Alliance** CLI and \`resolveAllianceConfig\` default the index to \`rag-hybrid\` (and rerank to \`bge-reranker-v2-m3\`) when those env vars are omitted. Misconfiguration surfaces at startup, not as tool errors. For manual flows, call \`list_namespaces\` -> \`suggest_query_params\` -> \`query\` (use preset fast/detailed/full per suggestion) or \`count\` (the suggest step is a mandatory gate). Use \`query_documents\` for full-document reading, \`keyword_search\` for exact-keyword retrieval against the sparse index, and \`generate_urls\` when records need URLs synthesized from metadata.
-
-A semantic search server that provides hybrid search capabilities over Pinecone vector indexes with automatic namespace discovery.
+const SERVER_FEATURES_AND_NOTES = `A semantic search server that provides hybrid search capabilities over Pinecone vector indexes with automatic namespace discovery.
 
 Features:
 - Hybrid Search: Combines dense and sparse embeddings for superior recall
@@ -47,12 +45,36 @@ Features:
 - Document reassembly: Use query_documents to get whole documents (chunks grouped and merged by document_number/doc_id/url) for content analysis or summarization. query_documents reranks when a rerank model is configured.
 - Keyword search: Use keyword_search to query the sparse index for lexical/keyword-only retrieval without reranking.
 
+Notes:
+- Result rows include both \`document_id\` (canonical) and \`paper_number\` (deprecated alias kept for one minor cycle; will be removed in the next major release). Prefer \`document_id\` in new code.
+- The server emits structured logs to stderr (text by default, set PINECONE_READ_ONLY_MCP_LOG_FORMAT=json for log aggregation).`;
+
+/** MCP instructions for {@link setupCoreServer} (seven core tools only). */
+export const CORE_SERVER_INSTRUCTIONS = `Quickstart for AI clients: call \`list_namespaces\` to discover namespaces, optionally \`namespace_router\` to rank candidates from user intent, then \`query\` (preset fast/detailed/full), \`count\`, \`query_documents\`, \`keyword_search\`, or \`generate_urls\` as needed. For \`setupCoreServer\` / package-root \`resolveConfig\`, \`PINECONE_INDEX_NAME\` (or \`--index-name\`) is required in addition to \`PINECONE_API_KEY\`. Misconfiguration surfaces at startup, not as tool errors.
+
+${SERVER_FEATURES_AND_NOTES}
+
 Usage:
 1. Use list_namespaces (cached) to discover available namespaces in the index. The response includes \`expires_at_iso\` so you know when to refresh.
 2. Optionally use namespace_router to choose candidate namespace(s) from user intent.
-3. Call suggest_query_params before query/count/query_documents tools (mandatory flow gate) to get suggested_fields and recommended_tool.
-4. Use count for count questions, \`query\` with the appropriate preset for chunk-level retrieval, or query_documents for full-document content.
+3. Use count for count questions, \`query\` with the appropriate preset for chunk-level retrieval, query_documents for full-document content, keyword_search for lexical retrieval, or generate_urls when records need synthesized URLs.`;
 
-Notes:
-- Result rows include both \`document_id\` (canonical) and \`paper_number\` (deprecated alias kept for one minor cycle; will be removed in the next major release). Prefer \`document_id\` in new code.
-- The server emits structured logs to stderr (text by default, set PINECONE_READ_ONLY_MCP_LOG_FORMAT=json for log aggregation).`;
+/** Alliance-only supplement appended to core instructions for {@link setupAllianceServer}. */
+export const ALLIANCE_INSTRUCTIONS_APPENDIX = `
+
+Alliance quickstart: for most user questions, call \`guided_query\` with the user's question — it does namespace routing, suggestion, and execution in one shot and returns a decision_trace you can show the user. The Alliance CLI and \`resolveAllianceConfig\` default the index to \`rag-hybrid\` (and rerank to \`bge-reranker-v2-m3\`) when those env vars are omitted.
+
+For manual flows with the full tool surface, call \`list_namespaces\` -> \`suggest_query_params\` -> \`query\` (use preset fast/detailed/full per suggestion) or \`count\` (the suggest step is a mandatory gate unless \`PINECONE_DISABLE_SUGGEST_FLOW=true\`).
+
+Alliance usage (after list_namespaces):
+4. Call suggest_query_params before query/count/query_documents tools (mandatory flow gate) to get suggested_fields and recommended_tool.
+5. Use count for count questions, \`query\` with the appropriate preset for chunk-level retrieval, or query_documents for full-document content.`;
+
+/** MCP instructions for {@link setupAllianceServer} (core tools plus Alliance tools). */
+export const ALLIANCE_SERVER_INSTRUCTIONS =
+  CORE_SERVER_INSTRUCTIONS + ALLIANCE_INSTRUCTIONS_APPENDIX;
+
+/**
+ * @deprecated Use {@link ALLIANCE_SERVER_INSTRUCTIONS} or {@link CORE_SERVER_INSTRUCTIONS}.
+ */
+export const SERVER_INSTRUCTIONS = ALLIANCE_SERVER_INSTRUCTIONS;
diff --git a/src/core/config.test.ts b/src/core/config.test.ts
diff --git a/src/core/config.ts b/src/core/config.ts
diff --git a/src/core/server.test.ts b/src/core/server.test.ts
diff --git a/src/core/server/tools/count-tool.test.ts b/src/core/server/tools/count-tool.test.ts
diff --git a/src/core/setup.ts b/src/core/setup.ts
