docs(registries): document official v0.1 protocol + standardized default set

- New docs/registries.md: default registry table, official v0.1 protocol,
  per-entry transport classification, opt-in API keys, offline reference set.
- registry-add.md: switch examples from the now-opt-in pulse to the zero-config
  official registry.
- specs/071-official-registry-protocol: spec / plan / tasks (speckit format).

Related #566
Related #567

Co-Authored-By: Paperclip <noreply@paperclip.ing>

Author: Dumbris

docs/features/registry-add.md

@@ -35,7 +35,7 @@ Before adding, find the registry id and server id:
 ```bash
 mcpproxy registry list                      # list configured registries + ids
 mcpproxy registry search <query> -r <id>    # search one registry
-mcpproxy registry search sqlite -r pulse --tag database --limit 5
+mcpproxy registry search sqlite -r official --limit 5
 ```
 
 `registry search` flags: `--registry/-r <id>`, `--tag/-t <tag>`,
@@ -58,7 +58,7 @@ Flags:
 Example:
 
 ```bash
-mcpproxy registry add pulse github-mcp --env GITHUB_TOKEN=ghp_xxx
+mcpproxy registry add official io.github.example/github-mcp --env GITHUB_TOKEN=ghp_xxx
 # ✅ Added 'github-mcp' (quarantined — approve with: mcpproxy upstream approve github-mcp)
 ```
 
@@ -134,7 +134,7 @@ POST /api/v1/registries/{id}/refresh
 Response:
 
 ```json
-{ "registry_id": "pulse", "cleared": 3 }
+{ "registry_id": "official", "cleared": 3 }
 ```
 
 `cleared` is the number of cached entries dropped.
@@ -150,7 +150,7 @@ Use the built-in `upstream_servers` tool with `operation: "add_from_registry"`:
 
 | Parameter | Required | Description |
 |-----------|----------|-------------|
-| `registry` | yes | Registry id (e.g. `pulse`). Discover via the `list_registries` / `search_servers` tools |
+| `registry` | yes | Registry id (e.g. `official`). Discover via the `list_registries` / `search_servers` tools |
 | `id` | yes | Server id within the registry |
 | `name` | no | Name override |
 | `env_json` | no | JSON object of env / required-input values, e.g. `{"GITHUB_TOKEN":"…"}` |

docs/registries.md

@@ -0,0 +1,58 @@
+# MCP Server Registries
+
+mcpproxy discovers MCP servers through a built-in set of registries. Discovery is
+available via the `search_servers` / `list_registries` MCP tools, the
+`mcpproxy registry search|list|add` CLI, and the REST `/api/v1/registries` routes.
+
+## Default registries
+
+| ID | Name | Protocol | Key required | Notes |
+|----|------|----------|--------------|-------|
+| `official` | Official MCP Registry | `modelcontextprotocol/registry` | no | Primary, zero-config aggregator (`registry.modelcontextprotocol.io/v0.1/servers`). |
+| `reference` | Reference Servers | `builtin/reference` | no | Curated `@modelcontextprotocol` servers, **shipped in-binary** so the basics work offline. |
+| `docker-mcp-catalog` | Docker MCP Catalog | `custom/docker` | no | Signed-container MCP server inventory. |
+| `pulse` | Pulse MCP | `custom/pulse` | **yes** | Opt-in; set `MCPPROXY_REGISTRY_PULSE_API_KEY`. |
+| `smithery` | Smithery | `modelcontextprotocol/registry` | **yes** | Opt-in; set `MCPPROXY_REGISTRY_SMITHERY_API_KEY`. |
+
+Key-requiring registries are **skipped** (not failed) when no key is configured, so
+a default search always succeeds. The API-key env var is
+`MCPPROXY_REGISTRY_<ID>_API_KEY` (ID upper-cased, non-alphanumerics → `_`).
+
+User-configured registries in `mcp_config.json` (`registries: [...]`) are **merged**
+with these defaults (keyed by ID); a custom entry never drops the shipped set.
+
+## Official v0.1 protocol
+
+The official registry returns a cursor-paginated list of wrapped entries:
+
+```json
+{ "servers": [ { "server": { /* server.json */ }, "_meta": { "io.modelcontextprotocol.registry/official": { "status": "active", "isLatest": true } } } ],
+  "metadata": { "nextCursor": "..." } }
+```
+
+mcpproxy:
+
+- descends into each `.server`, and **skips** entries whose `_meta` status is
+  `deleted`/`deprecated` or that are not `isLatest`;
+- follows `metadata.nextCursor` (bounded) and passes through `version=latest` and an
+  optional `search` query.
+
+## Transport classification (local vs remote)
+
+Classification is **per transport entry** — never "has remotes ⇒ remote" (the fix
+for GH #567 / #483):
+
+| server.json | Result |
+|---|---|
+| `packages[]` present | **stdio**: launch command derived from `runtimeHint` + `runtimeArguments` + `identifier`(`@version`) + `packageArguments`; `environmentVariables[]` become required inputs. No URL. |
+| `remotes[]` only | **remote/http**: `type` + `url` become the connection endpoint; `headers[]` become required inputs. |
+| both (hybrid) | the **package** is preferred (stdio); the remote endpoint is kept as a fallback connection URL. |
+
+Because every add surface (MCP, REST, CLI) funnels through the same keystone, a
+packages-only server is added as stdio and a remotes-only server as http
+identically across all surfaces.
+
+## Adding a discovered server
+
+See [registry-add.md](features/registry-add.md). New servers are quarantined by
+default until you approve them.

specs/071-official-registry-protocol/plan.md

@@ -0,0 +1,37 @@
+# Implementation Plan — 071 Official MCP registry v0.1 protocol
+
+Authoritative design: [MCP-856 plan, BOARD-DECIDED section](/MCP/issues/MCP-856#document-plan).
+
+## Architecture
+
+All discovery surfaces funnel through `registries.SearchServers` / `registries.FindServerByID` → protocol parser → `[]ServerEntry`, and all add surfaces funnel through `server.AddServerFromRegistryRef` → `buildToServerConfig`. `buildToServerConfig` already derives transport from the entry: `InstallCmd != ""` ⇒ `stdio`, else `URL`/`ConnectURL` ⇒ `http`. Therefore the #567 fix lives entirely in the parser deciding which field to set — every surface inherits the correct classification for free.
+
+## Components
+
+- `internal/registries/official.go` (new) — official v0.1 protocol: `fetchOfficialServers` (cursor follow-loop + versioned User-Agent), `parseOfficialPage` (envelope + status filter + legacy-flat fallback), `officialServerToEntry` (per-entry classification), `buildPackageCommand` (runtime-aware command synthesis), map helpers tolerant of camelCase/snake_case.
+- `internal/registries/reference.go` (new) — `builtin/reference` source served in-binary (`referenceServers()`); 7 curated stdio servers.
+- `internal/registries/search.go` — `fetchServers` short-circuits official (pagination) and reference (in-binary); query threaded through for server-side `search`; dispatch trimmed; bespoke parsers (`parseMCPRun`, `parseMCPStore`, `parseFleur`, `parseAzureMCPDemoWithoutGuesser`, `parseRemoteMCPServers`, `buildFleurInstallCmd`) and `constructServerURL` synthesis removed.
+- `internal/config/config.go` — `DefaultRegistries()` rewired: official + reference + docker (kept) + pulse/smithery (opt-in `RequiresKey`).
+
+## Classification rule (the #567 root fix)
+
+| Input | InstallCmd | URL | ConnectURL | Transport |
+|---|---|---|---|---|
+| packages[] only | set | empty | — | stdio |
+| remotes[] only | empty | set | — | http |
+| hybrid | set (package) | empty | set (remote) | stdio |
+
+## Pagination
+
+`fetchOfficialServers` loops up to `officialMaxPages` (20 × `officialPageLimit` 100), following `metadata.nextCursor`, stopping on empty/repeated cursor.
+
+## Test strategy (TDD)
+
+- `official_test.go` — golden fixture (`testdata/official_v0.1_servers.json`): wrapped parse, status/isLatest filter, classification matrix, pagination (httptest two-page), reference offline.
+- `server/consistency_official_test.go` — end-to-end #567 guard: packages→stdio / remotes→http identical across REST + CLI keystone.
+- Updated `registry_data_test.go` / `integration_test.go` to the new default set.
+
+## Risk
+
+- Smithery (opt-in) uses the official protocol parser as a best-effort default; gated behind `RequiresKey`, so it is skipped without a key. Exact Smithery-shape parsing is a future refinement.
+- Legacy flat `modelcontextprotocol/registry` responses still parse via the `parseOpenAPIRegistry` fallback inside `parseOfficialPage`.

specs/071-official-registry-protocol/spec.md

@@ -0,0 +1,46 @@
+# Feature Spec — Official MCP registry v0.1 protocol + standardized default registries
+
+**Spec:** 071-official-registry-protocol
+**Issue:** [MCP-865](/MCP/issues/MCP-865) (keystone backend) under goal [MCP-856](/MCP/issues/MCP-856)
+**Status:** Implemented (design board-decided 2026-06-02 in [MCP-856 plan](/MCP/issues/MCP-856#document-plan); Glass-Cockpit Gate 2 satisfied at parent level)
+**Resolves by design:** GH #566 (Pulse 410), GH #567 (Fleur all-remote misclassification)
+
+## Why
+
+mcpproxy's registry discovery shipped a stale, bespoke set of registries with hand-written per-registry parsers and URL synthesis. Two production bugs trace directly to that design:
+
+- **#566** — Pulse's `v0beta` endpoint is 410-dead.
+- **#567** — a server with a non-empty `remotes[]` was classified as *remote* even when it also had local `packages[]`, so the add path synthesized a bogus `docker://`/`npx://` HTTP transport (the #483 failure class).
+
+The official Model Context Protocol registry (`registry.modelcontextprotocol.io/v0.1/servers`) is now the canonical aggregator. Adopting its protocol replaces most bespoke parsers and fixes the classification bug at the root.
+
+## User scenarios
+
+1. **Zero-config discovery.** A fresh install can `retrieve_tools`/`registry search` against the official registry with no API key.
+2. **Correct transport per server.** A server published as an npm/pypi/oci package is added as a **stdio** server (install command, no URL); a server published only as a remote endpoint is added as an **http** server. A hybrid server prefers its local package.
+3. **Offline basics.** The `@modelcontextprotocol` reference servers (filesystem, fetch, memory, time, git, sequentialthinking, everything) are discoverable even with no network, because they currently 404 on the official registry (modelcontextprotocol/servers#3047).
+4. **Opt-in proprietary registries.** Pulse and Smithery remain available but require an API key (`RequiresKey`), so they never break a default search.
+
+## Functional requirements
+
+- **FR-1 Official v0.1 parser.** Parse the wrapped `{ "server": <server.json>, "_meta": {...} }` envelope. Read `_meta["io.modelcontextprotocol.registry/official"].status`/`.isLatest`; **skip `deleted`/`deprecated`/non-latest** by default. Tolerate camelCase wire fields (`registryType`, `runtimeHint`, `nextCursor`, `isLatest`) and snake_case fallbacks.
+- **FR-2 Cursor pagination.** Follow `metadata.nextCursor` (opaque) with `cursor`+`limit`, bounded by a page cap (`officialMaxPages`). Pass through `version=latest` and an optional `search` query.
+- **FR-3 Per-entry classification (the #567 fix).** Classify per transport entry, never "remotes present ⇒ remote":
+  - `server.packages[]` → **LOCAL/stdio**: command = `runtimeHint` + `runtimeArguments[]` + `identifier`(`@version` for npm) + `packageArguments[]`; set `InstallCmd`, leave `URL` empty; `environmentVariables[]` → `RequiredInputs`.
+  - `server.remotes[]` → **REMOTE**: `type`+`url` → `URL`; `headers[]` → `RequiredInputs`.
+  - **Hybrid** → prefer the package for stdio, keep the remote as `ConnectURL`.
+- **FR-4 Default-registry rewire.** Ship: Official (primary, no key), built-in Reference, Docker MCP catalog (kept). Demote Pulse + Smithery to opt-in (`RequiresKey`). Drop Azure-demo (`mcp/v0`), mcp.run, mcpstore, Fleur, remote-mcp-servers. Remove their bespoke parsers and all `constructServerURL` URL synthesis.
+- **FR-5 Curated reference source.** Built-in `builtin/reference` registry served in-binary (no network), all stdio.
+- **FR-6 Cross-surface consistency.** A packages-only server reads `stdio` and a remotes-only server reads `remote` identically across MCP, REST, and CLI add surfaces (mirrors spec 070).
+
+## Out of scope (delegated)
+
+- User-added registries + provenance/trust model + `registry add-source` CLI → [MCP-866](/MCP/issues/MCP-866) (blocked by this).
+
+## Acceptance
+
+- Official registry shipped + searchable zero-config; `registry search/list/add` work against it.
+- packages[]→stdio, remotes[]→remote across all surfaces (regression-tested).
+- Dropped/demoted registries handled per the board-decided §4 set.
+- Reference set discoverable offline.
+- #566 + #567 resolved by design.

specs/071-official-registry-protocol/tasks.md

@@ -0,0 +1,16 @@
+# Tasks — 071 Official MCP registry v0.1 protocol
+
+Dependency-ordered; TDD (failing test first) per task.
+
+- [x] **T01** Golden fixture `testdata/official_v0.1_servers.json` (packages-only, remotes-only, hybrid, deprecated, non-latest, deleted).
+- [x] **T02** `official_test.go` — failing tests: `parseOfficialPage` envelope + status/isLatest filter; classification matrix; `fetchOfficialServers` cursor pagination; `referenceServers` offline.
+- [x] **T03** `official.go` — `parseOfficialPage` / `parseOfficialItems` / `officialServerToEntry` / `buildPackageCommand` + camelCase/snake_case-tolerant map helpers.
+- [x] **T04** `official.go` — `fetchOfficialServers` cursor follow-loop, `buildOfficialURL` (version=latest, limit, search, cursor), versioned User-Agent (#566).
+- [x] **T05** `reference.go` — built-in curated reference set (7 stdio servers), `referenceServers()`.
+- [x] **T06** `search.go` — `fetchServers` short-circuits official + reference; thread `query`; route dispatch to `parseOfficialPage`; remove URL-synthesis loop.
+- [x] **T07** `search.go` — remove dropped parsers (`parseMCPRun`, `parseMCPStore`, `parseFleur`, `buildFleurInstallCmd`, `parseRemoteMCPServers`, `parseAzureMCPDemoWithoutGuesser`, `constructServerURL`) + dead constants + their tests; keep `parseOpenAPIRegistry` as legacy-flat fallback.
+- [x] **T08** `config.go` — `DefaultRegistries()` rewire (official + reference + docker; pulse/smithery opt-in).
+- [x] **T09** Update `registry_data_test.go` + `integration_test.go` to the new default set.
+- [x] **T10** `server/consistency_official_test.go` — cross-surface #567 regression (REST + CLI).
+- [x] **T11** Docs: `docs/registries.md` (new) + README registry list; CLAUDE.md unchanged (40k gate).
+- [x] **T12** Local green: `run-linter.sh`, `go test ./internal/... -race`, `test-api-e2e.sh`. Open PR (never merge); dual-AI review.