Skip to content

Commit 4dc4577

Browse files
freeznetCopilot
andauthored
feat(charts): add Helm chart for StreamNative MCP Server (#67)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 549d4a9 commit 4dc4577

23 files changed

Lines changed: 1700 additions & 84 deletions

.github/workflows/e2e.yaml

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
name: E2E Tests
2+
3+
on:
4+
push:
5+
paths:
6+
- "charts/**"
7+
- "cmd/snmcp-e2e/**"
8+
- "scripts/e2e-test.sh"
9+
- ".github/workflows/e2e.yaml"
10+
pull_request:
11+
paths:
12+
- "charts/**"
13+
- "cmd/snmcp-e2e/**"
14+
- "scripts/e2e-test.sh"
15+
- ".github/workflows/e2e.yaml"
16+
workflow_dispatch:
17+
18+
permissions:
19+
contents: read
20+
21+
jobs:
22+
e2e:
23+
runs-on: ubuntu-latest
24+
steps:
25+
- name: Check out code
26+
uses: actions/checkout@v4
27+
28+
- name: Set up Go
29+
uses: actions/setup-go@v5
30+
with:
31+
go-version-file: "go.mod"
32+
33+
- name: Set up Helm
34+
uses: azure/setup-helm@v4
35+
36+
- name: Set up Kind
37+
uses: helm/kind-action@v1
38+
with:
39+
cluster_name: kind
40+
41+
- name: Download dependencies
42+
run: go mod download
43+
44+
- name: Run E2E tests
45+
run: ./scripts/e2e-test.sh all
46+
47+
- name: Cleanup
48+
if: always()
49+
run: ./scripts/e2e-test.sh cleanup

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,4 @@ vendor
1111
.cursor/
1212
agents/
1313
.serena/
14+
.envrc

AGENTS.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,3 +135,5 @@ This file follows the AGENTS.md spec described in the Codex system message (scop
135135
---
136136

137137
Happy hacking! 🚀
138+
139+
@CLAUDE.md as reference.

CLAUDE.md

Lines changed: 108 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,15 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
66

77
```bash
88
make build # Build server binary to bin/snmcp
9+
make docker-build # Build local Docker image (both streamnative/mcp-server and streamnative/snmcp tags)
10+
make docker-build-push # Build and push multi-platform image (linux/amd64,linux/arm64)
11+
make docker-build-multiplatform # Build multi-platform image locally
12+
make docker-buildx-setup # Setup Docker buildx for multi-platform builds
13+
make license-check # Check license headers
14+
make license-fix # Fix license headers
915
go test -race ./... # Run all tests with race detection
1016
go test -race ./pkg/mcp/builders/... # Run specific package tests
1117
go test -v -run TestName ./pkg/... # Run a single test
12-
make license-check # Check license headers
13-
make license-fix # Fix license headers
14-
make docker-build # Build local Docker image
15-
make docker-build-push # Build and push multi-platform image
1618
```
1719

1820
## Architecture Overview
@@ -24,9 +26,11 @@ The StreamNative MCP Server implements the Model Context Protocol using the `mar
2426
```
2527
Client Request → MCP Server (pkg/mcp/server.go)
2628
29+
SSE/stdio transport layer (pkg/cmd/mcp/)
30+
2731
Tool Handler (from builders)
2832
29-
Session Context (pkg/mcp/ctx.go)
33+
Context Functions (pkg/mcp/ctx.go)
3034
3135
Service Client (Kafka/Pulsar/SNCloud)
3236
```
@@ -38,29 +42,44 @@ Client Request → MCP Server (pkg/mcp/server.go)
3842
- Sessions provide lazy-initialized clients for each service
3943
- Context functions (`pkg/mcp/ctx.go`) inject/retrieve sessions from request context
4044

41-
2. **Tool Builders** (`pkg/mcp/builders/`)
45+
2. **Tool Builders Framework** (`pkg/mcp/builders/`)
4246
- `ToolBuilder` interface: `GetName()`, `GetRequiredFeatures()`, `BuildTools()`, `Validate()`
4347
- `BaseToolBuilder` provides common feature validation logic
44-
- Each builder creates `[]server.ServerTool` with tool definitions and handlers
45-
- Builders in `builders/kafka/` and `builders/pulsar/` implement service-specific tools
48+
- `ToolRegistry` manages all tool builders with concurrent-safe registration
49+
- `ToolBuildConfig` specifies build parameters (ReadOnly, Features, Options)
50+
- `ToolMetadata` describes tool information (Name, Version, Description, Category, Tags)
51+
52+
3. **Tool Builders Organization**
53+
- `builders/kafka/` - Kafka-specific tool builders (connect, consume, groups, partitions, produce, schema_registry, topics)
54+
- `builders/pulsar/` - Pulsar-specific tool builders (brokers, brokers_stats, cluster, functions, functions_worker, namespace, namespace_policy, nsisolationpolicy, packages, resourcequotas, schema, sinks, sources, subscription, tenant, topic, topic_policy)
55+
- `builders/streamnative/` - StreamNative Cloud tool builders
4656

47-
3. **Tool Registration** (`pkg/mcp/*_tools.go`)
57+
4. **Tool Registration** (`pkg/mcp/*_tools.go`)
4858
- Each `*_tools.go` file creates a builder, builds tools, and adds them to the server
4959
- Tools are conditionally registered based on `--features` flag
5060
- Feature constants defined in `pkg/mcp/features.go`
5161

52-
4. **PFTools - Functions as Tools** (`pkg/mcp/pftools/`)
62+
5. **PFTools - Functions as Tools** (`pkg/mcp/pftools/`)
5363
- `PulsarFunctionManager` dynamically converts Pulsar Functions to MCP tools
5464
- Polls for function changes and auto-registers/unregisters tools
5565
- Circuit breaker pattern (`circuit_breaker.go`) for fault tolerance
5666
- Schema conversion (`schema.go`) for input/output handling
5767

68+
6. **Session Management** (`pkg/mcp/session/`)
69+
- `pulsar_session_manager.go` - LRU session cache with TTL cleanup for multi-session mode
70+
71+
7. **Transport Layer** (`pkg/cmd/mcp/`)
72+
- `sse.go` - SSE transport with health endpoints (`/healthz`, `/readyz`) and auth middleware
73+
- `server.go` - Stdio transport and common server initialization
74+
5875
### Key Design Patterns
5976

6077
- **Builder Pattern**: Tool builders create tools based on features and read-only mode
78+
- **Registry Pattern**: ToolRegistry provides centralized management of all builders
6179
- **Context Injection**: Sessions passed via `context.Context` using typed keys
6280
- **Feature Flags**: Tools enabled/disabled via string feature identifiers
6381
- **Circuit Breaker**: PFTools uses failure thresholds to prevent cascading failures
82+
- **Multi-Session Pattern**: Per-user Pulsar sessions with LRU caching for SSE mode
6483

6584
## Adding New Tools
6685

@@ -106,7 +125,7 @@ Client Request → MCP Server (pkg/mcp/server.go)
106125

107126
4. **Get Session in Handler**:
108127
```go
109-
session := mcpCtx.GetKafkaSession(ctx) // or GetPulsarSession
128+
session := mcp.GetKafkaSession(ctx) // or GetPulsarSession
110129
if session == nil {
111130
return mcp.NewToolResultError("session not found"), nil
112131
}
@@ -115,10 +134,13 @@ Client Request → MCP Server (pkg/mcp/server.go)
115134

116135
## Session Context Access
117136

118-
Handlers receive sessions via context (see `pkg/mcp/internal/context/ctx.go`):
119-
- `mcpCtx.GetKafkaSession(ctx)``*kafka.Session`
120-
- `mcpCtx.GetPulsarSession(ctx)``*pulsar.Session`
121-
- `mcpCtx.GetSNCloudSession(ctx)``*config.Session`
137+
Handlers receive sessions via context (see `pkg/mcp/ctx.go`):
138+
- `mcp.GetKafkaSession(ctx)``*kafka.Session`
139+
- `mcp.GetPulsarSession(ctx)``*pulsar.Session`
140+
- `mcp.GetSNCloudSession(ctx)``*config.Session`
141+
- `mcp.GetSNCloudOrganization(ctx)` → organization string
142+
- `mcp.GetSNCloudInstance(ctx)` → instance string
143+
- `mcp.GetSNCloudCluster(ctx)` → cluster string
122144

123145
From sessions:
124146
- `session.GetAdminClient()` / `session.GetAdminV3Client()` for Pulsar admin
@@ -144,13 +166,82 @@ When `--multi-session-pulsar` is enabled (SSE server with external Pulsar only):
144166
- **Session management**: See `pkg/mcp/session/pulsar_session_manager.go`
145167

146168
Key files:
147-
- `pkg/cmd/mcp/sse.go` - Auth middleware wraps SSEHandler()/MessageHandler()
169+
- `pkg/cmd/mcp/sse.go` - Auth middleware wraps SSEHandler()/MessageHandler(), health endpoints
148170
- `pkg/mcp/session/pulsar_session_manager.go` - LRU session cache with TTL cleanup
149171
- `pkg/cmd/mcp/server.go` - Skips global PulsarSession when multi-session enabled
150172

173+
### Health Endpoints
174+
175+
SSE server exposes health check endpoints:
176+
- `GET /mcp/healthz` - Liveness probe (always returns "ok")
177+
- `GET /mcp/readyz` - Readiness probe (always returns "ready")
178+
179+
## Feature Flags
180+
181+
Available feature flags (defined in `pkg/mcp/features.go`):
182+
183+
| Feature | Description |
184+
|---------|-------------|
185+
| `all` | Enable all features |
186+
| `all-kafka` | All Kafka features |
187+
| `all-pulsar` | All Pulsar features |
188+
| `kafka-client` | Kafka produce/consume |
189+
| `kafka-admin` | Kafka admin operations (all admin tools) |
190+
| `kafka-admin-schema-registry` | Schema Registry |
191+
| `kafka-admin-kafka-connect` | Kafka Connect |
192+
| `kafka-admin-topics` | Manage Kafka topics |
193+
| `kafka-admin-partitions` | Manage Kafka partitions |
194+
| `kafka-admin-groups` | Manage Kafka consumer groups |
195+
| `pulsar-admin` | Pulsar admin operations (all admin tools) |
196+
| `pulsar-client` | Pulsar produce/consume |
197+
| `pulsar-admin-brokers` | Manage Pulsar brokers |
198+
| `pulsar-admin-brokers-status` | Pulsar broker status |
199+
| `pulsar-admin-broker-stats` | Access Pulsar broker statistics |
200+
| `pulsar-admin-clusters` | Manage Pulsar clusters |
201+
| `pulsar-admin-functions` | Manage Pulsar Functions |
202+
| `pulsar-admin-functions-worker` | Manage Pulsar Function workers |
203+
| `pulsar-admin-namespaces` | Manage Pulsar namespaces |
204+
| `pulsar-admin-namespace-policy` | Configure namespace policies |
205+
| `pulsar-admin-ns-isolation-policy` | Manage namespace isolation policies |
206+
| `pulsar-admin-packages` | Manage Pulsar packages |
207+
| `pulsar-admin-resource-quotas` | Configure resource quotas |
208+
| `pulsar-admin-schemas` | Manage Pulsar schemas |
209+
| `pulsar-admin-subscriptions` | Manage Pulsar subscriptions |
210+
| `pulsar-admin-tenants` | Manage Pulsar tenants |
211+
| `pulsar-admin-topics` | Manage Pulsar topics |
212+
| `pulsar-admin-sinks` | Manage Pulsar IO sinks |
213+
| `pulsar-admin-sources` | Manage Pulsar Sources |
214+
| `pulsar-admin-topic-policy` | Configure topic policies |
215+
| `streamnative-cloud` | StreamNative Cloud context management |
216+
| `functions-as-tools` | Dynamic Pulsar Functions as MCP tools |
217+
218+
## Helm Chart
219+
220+
The project includes a Helm chart for Kubernetes deployment at `charts/snmcp/`:
221+
222+
```bash
223+
# Basic installation
224+
helm install snmcp ./charts/snmcp \
225+
--set pulsar.webServiceURL=http://pulsar.example.com:8080
226+
227+
# With TLS
228+
helm install snmcp ./charts/snmcp \
229+
--set pulsar.webServiceURL=https://pulsar:8443 \
230+
--set pulsar.tls.enabled=true \
231+
--set pulsar.tls.secretName=pulsar-tls
232+
```
233+
234+
The chart runs MCP Server in Multi-Session Pulsar mode with authentication via `Authorization: Bearer <token>` header.
235+
236+
## SDK Packages
237+
238+
The project includes generated SDK packages:
239+
- `sdk/sdk-apiserver/` - StreamNative Cloud API server client
240+
- `sdk/sdk-kafkaconnect/` - Kafka Connect client
241+
151242
## Error Handling
152243

153244
- Wrap errors: `fmt.Errorf("failed to X: %w", err)`
154245
- Return tool errors: `mcp.NewToolResultError("message")`
155246
- Check session nil before operations
156-
- For PFTools, use circuit breaker to handle repeated failures
247+
- For PFTools, use circuit breaker to handle repeated failures

charts/snmcp/Chart.yaml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
apiVersion: v2
2+
name: snmcp
3+
description: A Helm chart for StreamNative MCP Server with Multi-Session Pulsar support
4+
type: application
5+
version: 0.1.0
6+
appVersion: "0.0.1"
7+
home: https://github.com/streamnative/streamnative-mcp-server
8+
sources:
9+
- https://github.com/streamnative/streamnative-mcp-server
10+
maintainers:
11+
- name: StreamNative
12+
url: https://streamnative.io
13+
keywords:
14+
- mcp
15+
- streamnative
16+
- pulsar
17+
- ai
18+
- llm

0 commit comments

Comments
 (0)