Merge pull request #4 from vngcloud/feat/dynamic-spec-registry

feat: dynamic OpenAPI spec registry (replace bundled specs)

Author: tvt286

.github/workflows/release.yml

@@ -70,6 +70,19 @@ jobs:
             exit 1
           fi
 
+      - name: Bake docs portal URL
+        working-directory: src/greenode-mcp-server
+        env:
+          DOCS_PORTAL_URL: ${{ vars.DOCS_PORTAL_URL || 'https://docs.api.vngcloud.vn' }}
+        run: |
+          cat > greennode/greenode_mcp_server/_build_info.py <<EOF
+          """Build-time configuration baked by CI."""
+          from __future__ import annotations
+
+          DEFAULT_DOCS_PORTAL_URL = "${DOCS_PORTAL_URL}"
+          EOF
+          echo "Baked DEFAULT_DOCS_PORTAL_URL=${DOCS_PORTAL_URL}"
+
       - name: Build package
         working-directory: src/greenode-mcp-server
         run: uv build

.github/workflows/run-tests.yml

@@ -40,3 +40,7 @@ jobs:
         run: |
           uv pip install ruff
           uv run ruff check greennode/
+
+      - name: MCP protocol smoke test
+        working-directory: src/greenode-mcp-server
+        run: python3 scripts/mcp_protocol_smoke.py

CLAUDE.md

@@ -4,14 +4,14 @@
 
 GreenNode MCP Servers provide AI assistants (Claude, Cursor, Gemini, etc.) with tools to manage GreenNode services via the Model Context Protocol.
 
-- **Single server** — `greenode-mcp-server` covers all products via bundled OpenAPI specs
+- **Single server** — `greenode-mcp-server` covers all VNG Cloud products via a spec registry (`docs.api.vngcloud.vn`, fetched at startup, cached at `~/.greenode/mcp-specs/`)
 - **Namespace package** — `greennode.greenode_mcp_server`
 
 ### Available servers
 
 | Server | Package | Tools |
 |--------|---------|-------|
-| GreenNode MCP Server | `greennode.greenode_mcp_server` | search_api, call_api, 6 K8s tools |
+| GreenNode MCP Server | `greennode.greenode_mcp_server` | `search_api`, `call_api`, 6 K8s tools (`list_k8s_resources`, `get_pod_logs`, `get_k8s_events`, `list_api_versions`, `manage_k8s_resource`, `apply_yaml`) |
 
 ## Repository structure
 
@@ -20,9 +20,11 @@ greenode-mcp/
 ├── src/
 │   └── greenode-mcp-server/
 │       ├── pyproject.toml
-│       ├── specs/                    # Bundled OpenAPI specs (*.json)
 │       ├── greennode/
 │       │   └── greenode_mcp_server/
+│       │       ├── registry/        # Spec registry (provider + cache + loader)
+│       │       ├── _build_info.py   # Build-time baked DEFAULT_DOCS_PORTAL_URL
+│       │       └── ...
 │       └── tests/
 ├── scripts/
 ├── docs/
@@ -43,8 +45,12 @@ greenode-mcp/
 ## VNG Cloud API quirks
 
 - **IAM API uses camelCase**: `grantType`, `accessToken`, `expiresIn` (not snake_case OAuth2 standard)
-- **VKS API pagination is 0-based**: page 0 = first page
+- **Pagination is 1-based across VNG Cloud APIs**: page 1 = first page (standard convention for all products)
 - **API returns 202** for most successful operations (not 200)
+- **List response wrapper keys vary**: `items`, `listData`, `data`, `results`, `records` — the formatter in `api_caller.py` recognises all of them
+- **`{projectId}` / `{project_id}` path placeholders** are auto-substituted by `call_api` from `config.default_project_id` (set by `grn configure` or `GRN_DEFAULT_PROJECT_ID` env var)
+- **K8s `api_version`** is optional for common kinds (Pod, Deployment, PVC, ...) via `COMMON_API_VERSIONS` in `k8s_handler.py`; custom resources still need it explicit
+- **VKS kubeconfig endpoint** returns `{kubeConfig: "<yaml>", status: "ACTIVE"|"CREATING"|...}` — not raw YAML. `k8s_client_cache.py` extracts the `kubeConfig` field and checks `status`
 
 ## Adding a new tool (to existing server)
 
@@ -57,7 +63,7 @@ greenode-mcp/
 
 ## Adding a new MCP server (new product)
 
-The greenode-mcp-server now covers all products via bundled OpenAPI specs, so adding a new product typically means adding a new spec file to `src/greenode-mcp-server/specs/` rather than creating a separate server.
+The greenode-mcp-server covers all products via the spec registry (`docs.api.vngcloud.vn`). Adding a new product = VNG Cloud team publishes the product's OpenAPI page on the docs portal. The server picks it up on next restart — no code change, no release needed.
 
 If a truly separate server is needed:
 
@@ -74,8 +80,10 @@ See `src/greenode-mcp-server/` as reference.
 
 - **Input validation**: All resource IDs validated via `validators.validate_id()` before URL construction — prevents path traversal
 - **Write guard**: Mutating operations must check `self.allow_write` flag
-- **Sensitive data guard**: K8s Secret reads must check `self.allow_sensitive_data_access`
-- **Credential env vars supported**: `GRN_ACCESS_KEY_ID`/`GRN_SECRET_ACCESS_KEY` override credentials file (highest priority)
+- **Sensitive data guard**: Only K8s Secret reads check `self.allow_sensitive_data_access`. Pod logs and K8s events are NOT guarded — they're routine debug reads
+- **Credential env vars supported**: `GRN_ACCESS_KEY_ID`/`GRN_SECRET_ACCESS_KEY` override credentials file; `GRN_DEFAULT_PROJECT_ID` overrides config file `project_id` (highest priority)
+- **Response size cap**: `call_api` rejects responses > 800 KB with an actionable error — prevents context blow-up
+- **Row cap**: list responses truncate at 100 rows by default (with a footer telling the caller to paginate)
 - **Tokens in memory only**: Never written to disk or logged
 - **Credentials not logged**: Error messages and debug logs never include tokens or secrets
 - **Timeout**: All HTTP requests have 30s timeout
@@ -92,9 +100,10 @@ uv sync --all-extras
 uv run python -m pytest tests/ -v
 ```
 
-- 65 tests for GreenNode MCP Server
+- ~175 tests for GreenNode MCP Server
 - Uses `respx` for mocking async HTTP calls
 - Uses `pytest-asyncio` for async test support
+- MCP protocol smoke test: `python3 scripts/mcp_protocol_smoke.py` (runs the server via stdio, walks initialize → tools/list → tools/call)
 
 ## Git workflow
 
@@ -131,10 +140,17 @@ Code without docs is not done.
 
 | File | Purpose |
 |------|---------|
-| `greennode/greenode_mcp_server/server.py` | FastMCP entry point, tool registration, CLI flags |
-| `greennode/greenode_mcp_server/api_index.py` | Spec loader, in-memory index, keyword search |
+| `greennode/greenode_mcp_server/server.py` | FastMCP entry point, tool registration, CLI flags (`--refresh-specs`, `--offline`, ...) |
+| `greennode/greenode_mcp_server/api_index.py` | In-memory endpoint index, keyword search; delegates loading to `registry/` |
 | `greennode/greenode_mcp_server/api_caller.py` | call_api tool — write guard, auth injection, response formatting |
-| `greennode/greenode_mcp_server/config.py` | Config loading, REGIONS dict |
+| `greennode/greenode_mcp_server/_build_info.py` | Build-time baked `DEFAULT_DOCS_PORTAL_URL` (CI overwrites at release) |
+| `greennode/greenode_mcp_server/registry/provider.py` | `SpecProvider` Protocol, `ProductRef`, error types |
+| `greennode/greenode_mcp_server/registry/factory.py` | Selects active provider (swap here to migrate sources) |
+| `greennode/greenode_mcp_server/registry/redocly_portal.py` | Default provider — scrapes docs portal inline OpenAPI JSON |
+| `greennode/greenode_mcp_server/registry/local_dir.py` | Dev/test provider — reads specs from `GRN_MCP_SPEC_DIR` |
+| `greennode/greenode_mcp_server/registry/cache.py` | On-disk spec cache at `~/.greenode/mcp-specs/` with TTL + ETag |
+| `greennode/greenode_mcp_server/registry/loader.py` | Orchestrator — ties provider + cache + CLI flags together |
+| `greennode/greenode_mcp_server/config.py` | `GreenodeConfig` loader — reads `~/.greenode/credentials` + `~/.greenode/config` (incl. `project_id`), env-var overrides, REGIONS dict |
 | `greennode/greenode_mcp_server/auth.py` | TokenManager — async OAuth2 with auto-refresh |
 | `greennode/greenode_mcp_server/client.py` | GreenodeClient — used by K8s handler for kubeconfig fetch |
 | `greennode/greenode_mcp_server/k8s_handler.py` | 6 K8s tools |
@@ -148,7 +164,9 @@ Both projects share:
 - Same config files (`~/.greenode/credentials`, `~/.greenode/config`)
 - Same REGIONS dict (HCM-3, HAN endpoints)
 - Same IAM auth flow (camelCase fields)
-- Same env var names (`GRN_ACCESS_KEY_ID`, `GRN_SECRET_ACCESS_KEY`, `GRN_PROFILE`, `GRN_DEFAULT_REGION`, etc.)
+- Same env var names (`GRN_ACCESS_KEY_ID`, `GRN_SECRET_ACCESS_KEY`, `GRN_PROFILE`, `GRN_DEFAULT_REGION`, `GRN_DEFAULT_PROJECT_ID`)
+- Same profile section convention (`[default]` vs `[profile <name>]` in the config file, AWS-style)
+- `grn configure` auto-detects `project_id` and writes it to `~/.greenode/config` — MCP reads that value without a separate API call
 
 Key differences:
 - greenode-mcp is **async**, greenode-cli is **sync**

README.md

@@ -26,18 +26,23 @@ The [Model Context Protocol](https://modelcontextprotocol.io/) lets AI assistant
 export GRN_ACCESS_KEY_ID=your-client-id
 export GRN_SECRET_ACCESS_KEY=your-client-secret
 export GRN_DEFAULT_REGION=HCM-3
+export GRN_DEFAULT_PROJECT_ID=pro-xxxxxxxx   # required for many APIs that embed project in the URL
 ```
 
-**Option B: Credentials file (via GreenNode CLI)**
+**Option B: GreenNode CLI (recommended)**
 
 ```bash
-pip install grncli
 grn configure
 ```
 
-This creates `~/.greenode/credentials` which all MCP servers read automatically. Environment variables take priority over the credentials file.
+The wizard auto-detects your `project_id` and writes:
 
-> **Note:** All MCP servers require credentials configured via one of these methods.
+- `~/.greenode/credentials` — `client_id`, `client_secret`
+- `~/.greenode/config` — `region`, `project_id`, `output`
+
+See [greenode-cli](https://github.com/vngcloud/greennode-cli).
+
+Environment variables take priority over the files. All MCP servers in this repo read these same paths and env vars.
 
 ## Quick Start
 
@@ -68,9 +73,8 @@ greenode-mcp/
 │   └── greenode-mcp-server/            # GreenNode MCP Server
 │       ├── README.md                    # Server-specific docs, tools, security
 │       ├── pyproject.toml               # Package config + dependencies
-│       ├── specs/                       # Bundled OpenAPI specs (*.json)
 │       ├── greennode/
-│       │   └── greenode_mcp_server/     # Source code
+│       │   └── greenode_mcp_server/     # Source code (incl. registry/ module)
 │       └── tests/                       # Test suite
 ├── scripts/                             # Release scripts
 ├── docs/                                # Development guide
@@ -96,7 +100,7 @@ See [GreenNode MCP Server](src/greenode-mcp-server/) as reference.
 All GreenNode MCP servers share these security principles:
 
 - **Read-only by default** — Write operations require explicit `--allow-write` flag
-- **Sensitive data protection** — Kubernetes Secrets require `--allow-sensitive-data-access`
+- **Sensitive data protection** — Reading Kubernetes Secrets requires `--allow-sensitive-data-access`; other K8s reads (pods, deployments, logs, events) are allowed by default
 - **Credential security** — `~/.greenode/credentials` stored with `0600` permissions
 - **Input validation** — All resource IDs validated to prevent path traversal
 - **Token handling** — In memory only, never written to disk or logged