Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
# Required: Your Pinecone API key
PINECONE_API_KEY=your-pinecone-api-key-here

# Optional: Pinecone index name (default: rag-hybrid)
PINECONE_INDEX_NAME=rag-hybrid
# Optional: Dense (hybrid) Pinecone index name (default: rag-hybrid when unset)
# PINECONE_INDEX_NAME=rag-hybrid

# Optional: Reranking model (default: bge-reranker-v2-m3)
# Optional: Reranking model (default: bge-reranker-v2-m3 when unset)
PINECONE_RERANK_MODEL=bge-reranker-v2-m3

# Optional: Default number of results to return (default: 10)
Expand All @@ -21,7 +21,7 @@ PINECONE_READ_ONLY_MCP_LOG_LEVEL=INFO
PINECONE_READ_ONLY_MCP_LOG_FORMAT=text

# Optional: Sparse index name (default: ${PINECONE_INDEX_NAME}-sparse)
# PINECONE_SPARSE_INDEX_NAME=rag-hybrid-sparse
# PINECONE_SPARSE_INDEX_NAME=my-index-sparse

# Optional: Namespace + suggestion-flow cache TTL in seconds (default: 1800 = 30min)
# PINECONE_CACHE_TTL_SECONDS=1800
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ Tagged releases are published to npm from GitHub Actions when a **GitHub Release

## [Unreleased]

### Changed

- Package root export is the generic **core** layer (`setupCoreServer`); full CLI parity uses `@will-cppa/pinecone-read-only-mcp/alliance` (`setupAllianceServer`, built-in URL generators). `resolveConfig` uses env when set, else defaults: index **`rag-hybrid`**, rerank **`bge-reranker-v2-m3`** (constants `DEFAULT_INDEX_NAME` / `DEFAULT_RERANK_MODEL` in `src/core/config.ts`).
- When reranking was requested but `PineconeClient` has no rerank model (manual library use): `query` / `query_documents` include `rerank_skipped_reason: no_model`; `guided_query` sets `decision_trace.rerank_status: skipped_no_model`.

### Added

- `UrlGeneratorFn` type alias (same as `UrlGenerator`) and `RegisterBuiltinUrlGeneratorsOptions` with `reinstallBuiltins` on `registerBuiltinUrlGenerators()` to restore default `mailing` / `slack-Cpplang` generators after overrides; README “Custom URL generators” section and tests for custom registration and built-in override.
Expand Down
54 changes: 36 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,16 +84,24 @@ npm install
npm run build
```

## Architecture

The codebase is split into two layers:

- **`src/core/`** — generic MCP–Pinecone bridge (`PineconeClient`, `resolveConfig`, core MCP tools). Import from `@will-cppa/pinecone-read-only-mcp` (package root).
- **`src/alliance/`** — C++ Alliance app tools (`suggest_query_params`, `guided_query`, Boost/Slack URL builtins). Import from `@will-cppa/pinecone-read-only-mcp/alliance` and call `setupAllianceServer(config)` for the full tool surface (what the CLI uses).

## Configuration

You need a **Pinecone API key** and (by default) a **dense** index plus matching **sparse** index; see [docs/CONFIGURATION.md](docs/CONFIGURATION.md) for every environment variable and CLI flag.
You need a **Pinecone API key**. **Index** uses `PINECONE_INDEX_NAME` when set, else defaults to `rag-hybrid`; sparse index defaults to `{index}-sparse`. **Reranking** uses `PINECONE_RERANK_MODEL` when set, else `bge-reranker-v2-m3`. MCP configs with only `PINECONE_API_KEY` keep prior Alliance defaults. See [docs/CONFIGURATION.md](docs/CONFIGURATION.md) for every variable and CLI flag.

Quick reference:

| Variable | Required | Default |
| ----------------------------------- | ----------------------- | --------------------------------- |
| `PINECONE_API_KEY` | Yes (for live Pinecone) | — |
| `PINECONE_INDEX_NAME` | No | `rag-hybrid` |
| `PINECONE_INDEX_NAME` | No | `rag-hybrid` when unset |
| `PINECONE_RERANK_MODEL` | No | `bge-reranker-v2-m3` when unset |
| `PINECONE_SPARSE_INDEX_NAME` | No | `{index}-sparse` |
| `PINECONE_READ_ONLY_MCP_LOG_LEVEL` | No | `INFO` (`DEBUG`–`ERROR`) |
| `PINECONE_READ_ONLY_MCP_LOG_FORMAT` | No | `text` (`json` for log pipelines) |
Expand All @@ -102,31 +110,37 @@ Run `pinecone-read-only-mcp --help` for CLI equivalents (`--cache-ttl-seconds`,

### Deployment model

The server uses **process-global** memory for the suggest-flow gate (`suggest_query_params` context), namespaces cache, URL generator registry, and active configuration. **Stdio MCP (one client per Node process)** matches this model. If you embed `setupServer` behind a multi-tenant HTTP transport, isolate those structures per session yourself or treat the suggest-flow guard as best-effort only.
The server uses **process-global** memory for the suggest-flow gate (`suggest_query_params` context), namespaces cache, URL generator registry, and active configuration. **Stdio MCP (one client per Node process)** matches this model. If you embed `setupAllianceServer` behind a multi-tenant HTTP transport, isolate those structures per session yourself or treat the suggest-flow guard as best-effort only.

### Library embedding (`setupServer`)
### Library embedding

Treat **`setupServer()` as one logical server per Node process**: it mutates shared module singletons (suggest-flow map, namespaces cache, URL registry, config context, shared `PineconeClient` slot). A **second** `setupServer()` in the same process **throws** unless you call **`teardownServer()`** first.
Treat **`setupCoreServer()` / `setupAllianceServer()` as one logical server per Node process**: they mutate shared module singletons (suggest-flow map, namespaces cache, URL registry, config context, shared `PineconeClient` slot). A **second** setup call in the same process **throws** unless you call **`teardownServer()`** first.

Recommended pattern: `resolveConfig` → `setPineconeClient(new PineconeClient(...))` → `await setupServer(config)` → connect one MCP transport. For tests or re-initialization in the same process, call `teardownServer()` then `setupServer(config)` again. For isolated production tenants, prefer **one server per Node process** (or separate OS processes) rather than sharing one embedder across tenants.
- **Generic bridge only:** `import { setupCoreServer, teardownServer, ... } from '@will-cppa/pinecone-read-only-mcp'`
- **Full Alliance surface (CLI parity):** `import { setupAllianceServer } from '@will-cppa/pinecone-read-only-mcp/alliance'`

Import `setupServer` and `teardownServer` from `@will-cppa/pinecone-read-only-mcp`. See [examples/library-embedding-demo.ts](examples/library-embedding-demo.ts) and [docs/TOOLS.md](docs/TOOLS.md#suggest-flow-gate).
Recommended pattern: `resolveConfig({ apiKey, indexName, ... })` → `setPineconeClient(new PineconeClient(...))` → `await setupAllianceServer(config)` → connect one MCP transport. See [examples/library-embedding-demo.ts](examples/library-embedding-demo.ts) and [docs/TOOLS.md](docs/TOOLS.md#suggest-flow-gate).

### Custom URL generators

Namespaces other than `mailing` and `slack-Cpplang` (or different URL rules for any namespace) can use programmatic registration — no fork required.

Import `registerUrlGenerator` and types `UrlGeneratorFn` / `UrlGenerationResult` from `@will-cppa/pinecone-read-only-mcp`. Register **additional** namespaces before tools that emit URLs run (typically right after `setupServer` resolves config). To **replace** the built-in `mailing` or `slack-Cpplang` generators, call `registerUrlGenerator` **after** `setupServer`, because `setupServer` installs the defaults first.
Import `registerUrlGenerator` and types `UrlGeneratorFn` / `UrlGenerationResult` from `@will-cppa/pinecone-read-only-mcp`. Register **additional** namespaces before tools that emit URLs run. Built-in `mailing` / `slack-Cpplang` generators are installed by `setupAllianceServer` (not by `setupCoreServer`).

```ts
import {
PineconeClient,
registerUrlGenerator,
setupServer,
resolveConfig,
setPineconeClient,
type UrlGenerationResult,
type UrlGeneratorFn,
} from '@will-cppa/pinecone-read-only-mcp';
import { setupAllianceServer } from '@will-cppa/pinecone-read-only-mcp/alliance';

const server = await setupServer(config);
const config = resolveConfig({ apiKey: '...', indexName: 'your-index' });
setPineconeClient(new PineconeClient({ apiKey: config.apiKey, indexName: config.indexName }));
const server = await setupAllianceServer(config);
Comment thread
coderabbitai[bot] marked this conversation as resolved.

const myDocs: UrlGeneratorFn = (metadata): UrlGenerationResult => {
const id = typeof metadata.doc_id === 'string' ? metadata.doc_id : null;
Expand All @@ -146,7 +160,7 @@ A fuller embedding sample lives in [examples/custom-url-generator.ts](examples/c
| ------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
| [examples/suggest-flow-demo.ts](examples/suggest-flow-demo.ts) | Manual **suggest_query_params → query** flow with namespace consistency notes. |
| [examples/guided-query-demo.ts](examples/guided-query-demo.ts) | **guided_query** orchestration and how to read `decision_trace`. |
| [examples/library-embedding-demo.ts](examples/library-embedding-demo.ts) | Programmatic **setupServer** wiring without the CLI binary. |
| [examples/library-embedding-demo.ts](examples/library-embedding-demo.ts) | Programmatic **setupAllianceServer** wiring without the CLI binary. |
| [examples/custom-url-generator.ts](examples/custom-url-generator.ts) | Custom **URL generator** registration for `generate_urls` / row enrichment. |

Run with `npx tsx examples/<file>.ts` from a checkout (requires valid Pinecone env for live paths).
Expand All @@ -162,7 +176,9 @@ Add to your `claude_desktop_config.json`:
"command": "npx",
"args": ["-y", "@will-cppa/pinecone-read-only-mcp"],
"env": {
"PINECONE_API_KEY": "your-api-key-here"
"PINECONE_API_KEY": "your-api-key-here",
"PINECONE_INDEX_NAME": "your-index-name"
"PINECONE_RERANK_MODEL": "your-rerank-model"
}
}
}
Expand Down Expand Up @@ -230,12 +246,14 @@ node node_modules/@will-cppa/pinecone-read-only-mcp/dist/index.js --api-key YOUR

```
--api-key TEXT Pinecone API key (or set PINECONE_API_KEY env var)
--index-name TEXT Pinecone index name [default: rag-hybrid]
--rerank-model TEXT Reranking model [default: bge-reranker-v2-m3]
--index-name TEXT Dense index (required, or PINECONE_INDEX_NAME)
--rerank-model TEXT Reranker model (defalut: bge-reranker-v2-m3)
--log-level TEXT Logging level [default: INFO]
--help, -h Show help message
```

Run `pinecone-read-only-mcp --help` for the full option list.

## Deployment

### Production Readiness Defaults
Expand Down Expand Up @@ -264,7 +282,7 @@ docker build -t pinecone-read-only-mcp:latest .
# run (stdio MCP server)
docker run --rm -i \
-e PINECONE_API_KEY=YOUR_API_KEY \
-e PINECONE_INDEX_NAME=rag-hybrid \
-e PINECONE_INDEX_NAME=your-index-name \
pinecone-read-only-mcp:latest
```

Expand Down Expand Up @@ -436,7 +454,7 @@ Returns the **unique document count** matching a metadata filter and semantic qu

### `keyword_search`

Performs **keyword (lexical/sparse-only)** search over the dedicated sparse index (default: `rag-hybrid-sparse`, i.e. `{PINECONE_INDEX_NAME}-sparse`). Use for exact or keyword-style queries. Does not use the dense index or semantic reranking. Call `list_namespaces` first to discover namespaces; `suggest_query_params` is optional.
Performs **keyword (lexical/sparse-only)** search over the sparse index (`{PINECONE_INDEX_NAME}-sparse` by default). Use for exact or keyword-style queries. Does not use the dense index or semantic reranking. Call `list_namespaces` first to discover namespaces; `suggest_query_params` is optional.

**Parameters:**

Expand All @@ -457,7 +475,7 @@ Performs **keyword (lexical/sparse-only)** search over the dedicated sparse inde
"status": "success",
"query": "contracts C++",
"namespace": "wg21-papers",
"index": "rag-hybrid-sparse",
"index": "your-index-sparse",
"result_count": 5,
"results": [
{
Expand Down Expand Up @@ -624,7 +642,7 @@ The script prints a table of p50, p95, and p99 latencies in milliseconds and wri
PINECONE_API_KEY=your-key npm run test:search
```

If the sparse index (`rag-hybrid-sparse` by default) does not exist or has no data, the keyword search step is skipped with a warning.
If the sparse index (`{PINECONE_INDEX_NAME}-sparse`) does not exist or has no data, the keyword search step is skipped with a warning.

2. **Via MCP client:**
Start the server and call the `keyword_search` tool with `query_text`, `namespace` (from `list_namespaces`), and optional `top_k` or `metadata_filter`. Response shape is the same as the `query` tool (e.g. `results` with ids, metadata, scores; `reranked` is always `false`).
Expand Down
18 changes: 12 additions & 6 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Configuration is built from **CLI flags** (when using the binary), **environment

**CLI / `ConfigOverrides` > environment variables > built-in defaults.**

`resolveConfig` in `src/config.ts` applies this order for each field.
`resolveConfig` in `src/core/config.ts` applies this order for each field.

---

Expand All @@ -15,9 +15,9 @@ Configuration is built from **CLI flags** (when using the binary), **environment
| Field | Source | Default / notes |
| ----- | ------ | --------------- |
| `apiKey` | `apiKey` / `PINECONE_API_KEY` | **Required** (non-empty after trim) |
| `indexName` | `indexName` / `PINECONE_INDEX_NAME` | `rag-hybrid` |
| `indexName` | `indexName` / `PINECONE_INDEX_NAME` | `rag-hybrid` when env and overrides omit it |
| `sparseIndexName` | `sparseIndexName` / `PINECONE_SPARSE_INDEX_NAME` | `{indexName}-sparse` |
| `rerankModel` | `rerankModel` / `PINECONE_RERANK_MODEL` | `bge-reranker-v2-m3` |
| `rerankModel` | `rerankModel` / `PINECONE_RERANK_MODEL` | `bge-reranker-v2-m3` when env and overrides omit it |
| `defaultTopK` | `defaultTopK` / `PINECONE_TOP_K` | `10` (positive int) |
| `logLevel` | `logLevel` / `PINECONE_READ_ONLY_MCP_LOG_LEVEL` | `INFO` (`DEBUG`–`ERROR`) |
| `logFormat` | `logFormat` / `PINECONE_READ_ONLY_MCP_LOG_FORMAT` | `text` or `json` |
Expand All @@ -28,6 +28,12 @@ Configuration is built from **CLI flags** (when using the binary), **environment

**Throws** if `apiKey` is missing after trim.

For the full Alliance tool surface (including `suggest_query_params`, `guided_query`, and built-in URL generators), import from `@will-cppa/pinecone-read-only-mcp/alliance` and call `setupAllianceServer(config)`.

### Rerank model

`resolveConfig` uses `PINECONE_INDEX_NAME` / `PINECONE_RERANK_MODEL` when set; otherwise **`rag-hybrid`** and **`bge-reranker-v2-m3`**. MCP configs that only set `PINECONE_API_KEY` keep the same defaults as before.

---

## CLI flags (`parseCli` / `src/cli.ts`)
Expand All @@ -52,9 +58,9 @@ Configuration is built from **CLI flags** (when using the binary), **environment

## Library embedding

1. Build `ServerConfig` with `resolveConfig({ apiKey: '...', ... })` or pass explicit overrides.
2. Construct `PineconeClient` and `setPineconeClient(client)` before `setupServer(config)` (mirrors `src/index.ts`).
3. `await setupServer(config)` then connect an MCP transport.
1. Build `ServerConfig` with `resolveConfig({ apiKey: '...', indexName: '...', ... })` or pass explicit overrides.
2. Construct `PineconeClient` and `setPineconeClient(client)` before `setupAllianceServer(config)` (mirrors `src/index.ts`).
3. `await setupAllianceServer(config)` (or `setupCoreServer` for generic tools only) then connect an MCP transport.

See [README deployment model](../README.md#deployment-model) and [examples/library-embedding-demo.ts](../examples/library-embedding-demo.ts).

Expand Down
16 changes: 9 additions & 7 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
# Examples

| File | Description |
|------|-------------|
| [custom-url-generator.ts](./custom-url-generator.ts) | Embed the MCP server as a library, register a **custom URL generator** for a namespace, and wire `PineconeClient` + `setupServer()`. |
| [suggest-flow-demo.ts](./suggest-flow-demo.ts) | Document the **suggest_query_params → query** gate sequence and trimmed namespace usage. |
| [guided-query-demo.ts](./guided-query-demo.ts) | Document **guided_query** and the **`decision_trace`** payload. |
| [library-embedding-demo.ts](./library-embedding-demo.ts) | Minimal **library embedding** (`resolveConfig`, `setPineconeClient`, `setupServer`). |
| File | Description |
| -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| [custom-url-generator.ts](./custom-url-generator.ts) | Embed the MCP server, register a **custom URL generator**, and call `setupAllianceServer()` (full tool surface). |
| [suggest-flow-demo.ts](./suggest-flow-demo.ts) | Document the **suggest_query_params → query** gate sequence and trimmed namespace usage. |
| [guided-query-demo.ts](./guided-query-demo.ts) | Document **guided_query** and the **`decision_trace`** payload. |
| [library-embedding-demo.ts](./library-embedding-demo.ts) | Minimal **library embedding** (`resolveConfig`, `setPineconeClient`, `setupAllianceServer`). |

Run with `npx tsx examples/<file>.ts` after `npm install` (live Pinecone calls need `PINECONE_API_KEY` and related env).
**Required env for live runs:** `PINECONE_API_KEY`. Optional: `PINECONE_INDEX_NAME` (default `rag-hybrid`), `PINECONE_RERANK_MODEL` (default `bge-reranker-v2-m3`).

Run with `npm run build` then `npx tsx examples/<file>.ts` from the repo root (examples resolve the package via `dist/core` and `dist/alliance`).
Loading
Loading