Add vMCP MCPServerEntry dynamic mode reconciler (#4710)

* Add vMCP MCPServerEntry dynamic mode reconciler

Enable vMCP dynamic mode to watch MCPServerEntry resources at runtime,
automatically adding/removing them as backends without restart.

- Add CABundleData []byte to Backend and BackendTarget for dynamic mode
  CA bundle support (fetched from K8s ConfigMaps, not volume-mounted)
- Extend newBackendTransport to accept CA cert bytes alongside file path,
  with data taking precedence over path
- Set Backend.Type = BackendTypeEntry in mcpServerEntryToBackend()
- Add fetchCABundleData() to read CA PEM from ConfigMap via CABundleRef
- Extend fetchBackendResource() to try MCPServerEntry as third type
- Add MCPServerEntry watch with groupRef filtering in SetupWithManager()
- Add MCPServerEntry to ExternalAuthConfig change handler
- Add ConfigMap watch for CA bundle changes affecting entry backends

Closes #4659

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Address review feedback on dynamic mode reconciler

- F1: Make CA bundle fetch failure fatal — return nil to exclude backend
  when explicitly configured caBundleRef can't be loaded (matches auth
  config failure pattern, prevents silent TLS trust degradation)
- F2: Add field index for ConfigMap→MCPServerEntry lookup via
  SetupIndexes() — replaces full List+filter with indexed cache query
- F3: Restore source context in CA parse error messages (file path or
  "inline data") for operator debuggability
- F4: Add table-driven tests for fetchCABundleData covering all 5 code
  paths (nil ref, not found, key missing, default key, explicit key)
- F5: Extract MapAuthConfigToEntries() as exported method with 4 test
  cases covering group/auth config matching and filtering
- F6: Update architecture docs (09-operator, 10-virtual-mcp) to document
  MCPServerEntry discovery, ConfigMap watching, and field-indexed lookup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add integration tests for MCPServerEntry reconciler and CA bundle

Integration tests (Ginkgo+envtest) for BackendReconciler MCPServerEntry
lifecycle: creation with matching groupRef adds backend to registry,
mismatched groupRef excludes backend, deletion removes backend, and
registry version increments on events. Unlike MCPServer/MCPRemoteProxy,
MCPServerEntry uses Spec.RemoteURL directly so backends actually appear
in the registry during envtest.

Workload conversion tests verify BackendTypeEntry is set, CA bundle
data is fetched from ConfigMap, missing ConfigMap causes fatal failure
(returns nil), and empty Key defaults to "ca.crt".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: ChrisJBurns

docs/arch/09-operator-architecture.md

@@ -57,6 +57,7 @@ MCPServer is the fundamental building block. All other CRDs either **organize**,
                     │  │  │  │      CORE         │  │  │  │
                     │  │  │  │    MCPServer      │  │  │  │
                     │  │  │  │  MCPRemoteProxy   │  │  │  │
+                    │  │  │  │  MCPServerEntry   │  │  │  │
                     │  │  │  └───────────────────┘  │  │  │
                     │  │  └─────────────────────────┘  │  │
                     │  └───────────────────────────────┘  │
@@ -71,7 +72,7 @@ MCPServer is the fundamental building block. All other CRDs either **organize**,
 
 | Layer | CRDs | Purpose |
 |-------|------|---------|
-| **Core** | MCPServer, MCPRemoteProxy | Run or proxy MCP servers |
+| **Core** | MCPServer, MCPRemoteProxy, MCPServerEntry | Run, proxy, or declare MCP servers |
 | **Organization** | MCPGroup | Group related servers together |
 | **Aggregation** | VirtualMCPServer, VirtualMCPCompositeToolDefinition | Combine multiple servers into one endpoint |
 | **Discovery** | MCPRegistry | Help clients find available servers |
@@ -90,6 +91,7 @@ MCPServer is the fundamental building block. All other CRDs either **organize**,
 
 | CRD | Purpose |
 |-----|---------|
+| **MCPServerEntry** | Zero-infrastructure declaration of a remote MCP endpoint |
 | **MCPGroup** | Logical grouping of workloads (status tracking only) |
 | **ToolConfig** | Tool filtering and renaming configuration |
 | **MCPExternalAuthConfig** | Token exchange / header injection configuration |
@@ -108,6 +110,10 @@ graph TB
         Registry[MCPRegistry<br/>Deployment: API server]
     end
 
+    subgraph "Zero-Infrastructure"
+        Entry[MCPServerEntry<br/>No resources]
+    end
+
     subgraph "Logical Grouping"
         Group[MCPGroup<br/>No resources]
     end
@@ -134,6 +140,10 @@ graph TB
     Proxy -.->|externalAuthConfigRef| ExtAuth
     Proxy -.->|toolConfigRef| ToolCfg
     Proxy -.->|oidcConfigRef| OIDCCfg
+
+    Entry -->|groupRef| Group
+    Entry -.->|externalAuthConfigRef| ExtAuth
+    Entry -.->|caBundleRef| ConfigMap[ConfigMap<br/>CA bundle]
 ```
 
 ### MCPServer
@@ -257,6 +267,24 @@ Defines a proxy for remote MCP servers with authentication, authorization, audit
 
 **Controller**: `cmd/thv-operator/controllers/mcpremoteproxy_controller.go`
 
+### MCPServerEntry
+
+Declares a remote MCP endpoint as a zero-infrastructure catalog entry. Unlike MCPServer and MCPRemoteProxy, MCPServerEntry never creates a Deployment, Service, or Pod. vMCP connects directly to the declared remote URL.
+
+**Key fields:**
+- `remoteURL` - URL of the remote MCP server (required)
+- `groupRef` - MCPGroup membership for discovery by VirtualMCPServer
+- `externalAuthConfigRef` - Token exchange for remote service authentication
+- `caBundleRef` - Reference to a ConfigMap containing CA certificate data for TLS verification
+
+The MCPServerEntry controller is validation-only: it validates that referenced resources (groupRef, externalAuthConfigRef, caBundleRef ConfigMap) exist and updates status conditions accordingly. It never probes the remote URL or creates infrastructure.
+
+MCPServerEntry backends are discovered by vMCP in both static mode (listed at startup) and dynamic mode (watched by the BackendReconciler). In dynamic mode, ConfigMap changes trigger re-reconciliation of affected MCPServerEntry backends via a field-indexed watch on `spec.caBundleRef.configMapRef.name`.
+
+**Implementation**: `cmd/thv-operator/api/v1alpha1/mcpserverentry_types.go`
+
+**Controller**: `cmd/thv-operator/controllers/mcpserverentry_controller.go`
+
 ### MCPGroup
 
 Logically groups MCPServer resources together for organizational purposes.

docs/arch/10-virtual-mcp-architecture.md

@@ -41,6 +41,7 @@ graph TB
         B1[MCPServer]
         B2[MCPServer]
         B3[MCPRemoteProxy]
+        B4[MCPServerEntry]
     end
 
     Client[MCP Client] --> Server
@@ -54,9 +55,11 @@ graph TB
     BackendClient --> B1
     BackendClient --> B2
     BackendClient --> B3
+    BackendClient --> B4
     Health --> B1
     Health --> B2
     Health --> B3
+    Health --> B4
 
     style Server fill:#90caf9
     style Aggregator fill:#81c784
@@ -85,17 +88,20 @@ graph LR
     Group -->|contains| S1[MCPServer]
     Group -->|contains| S2[MCPServer]
     Group -->|contains| R1[MCPRemoteProxy]
+    Group -->|contains| E1[MCPServerEntry]
 
     style vMCP fill:#90caf9
     style Group fill:#ba68c8
 ```
 
 **Discovery process:**
 1. VirtualMCPServer references an MCPGroup by name
-2. All MCPServers and MCPRemoteProxies in that group are discovered
+2. All MCPServers, MCPRemoteProxies, and MCPServerEntries in that group are discovered
 3. For each backend, URL, transport type, and auth config are extracted
 4. vMCP queries each backend for available tools, resources, and prompts
 
+MCPServerEntry backends connect directly to remote MCP servers without deploying a proxy pod. They are zero-infrastructure catalog entries that declare a remote endpoint URL, optional external auth, and an optional CA bundle for TLS verification. CA bundle data is fetched from Kubernetes ConfigMaps at discovery time. In dynamic mode, the BackendReconciler watches ConfigMap changes and uses a field index on `spec.caBundleRef.configMapRef.name` to efficiently re-reconcile only the MCPServerEntry backends affected by a given ConfigMap update.
+
 **Implementation**: `pkg/vmcp/aggregator/`
 
 ## Aggregation Pipeline

pkg/vmcp/client/client.go

@@ -93,8 +93,12 @@ func NewHTTPBackendClient(registry vmcpauth.OutgoingAuthRegistry) (vmcp.BackendC
 //
 // If caBundlePath is non-empty, a custom TLS configuration is applied that trusts both
 // the system root CAs and the certificate(s) in the specified file. This is used for
-// entry-type backends with self-signed or internal CA certificates.
-func newBackendTransport(caBundlePath string) (*http.Transport, error) {
+// entry-type backends with self-signed or internal CA certificates (static mode).
+//
+// If caBundleData is non-empty, the raw PEM bytes are used directly instead of reading
+// from a file. This is used in dynamic mode where CA bundles are fetched from K8s
+// ConfigMaps at discovery time. caBundleData takes precedence over caBundlePath.
+func newBackendTransport(caBundlePath string, caBundleData []byte) (*http.Transport, error) {
 	var t *http.Transport
 	if dt, ok := http.DefaultTransport.(*http.Transport); ok {
 		t = dt.Clone()
@@ -116,20 +120,32 @@ func newBackendTransport(caBundlePath string) (*http.Transport, error) {
 		}
 	}
 
-	if caBundlePath != "" {
-		caCert, err := os.ReadFile(caBundlePath) //nolint:gosec // CA bundle path is validated by config validator (no path traversal)
+	// Resolve CA certificate PEM data: caBundleData takes precedence over caBundlePath
+	var caPEM []byte
+	switch {
+	case len(caBundleData) > 0:
+		caPEM = caBundleData
+	case caBundlePath != "":
+		var err error
+		caPEM, err = os.ReadFile(caBundlePath) //nolint:gosec // CA bundle path is validated by config validator (no path traversal)
 		if err != nil {
 			return nil, fmt.Errorf("failed to read CA bundle from %s: %w", caBundlePath, err)
 		}
+	}
 
+	if len(caPEM) > 0 {
 		caCertPool, err := x509.SystemCertPool()
 		if err != nil {
 			// Fall back to empty pool if system certs can't be loaded
 			caCertPool = x509.NewCertPool()
 		}
 
-		if !caCertPool.AppendCertsFromPEM(caCert) {
-			return nil, fmt.Errorf("failed to parse CA certificate from %s", caBundlePath)
+		if !caCertPool.AppendCertsFromPEM(caPEM) {
+			source := "inline data"
+			if len(caBundleData) == 0 && caBundlePath != "" {
+				source = caBundlePath
+			}
+			return nil, fmt.Errorf("failed to parse CA certificate from %s", source)
 		}
 
 		if t.TLSClientConfig == nil {
@@ -238,7 +254,7 @@ func (h *httpBackendClient) defaultClientFactory(ctx context.Context, target *vm
 	//
 	// Clone DefaultTransport per call so each client gets an isolated connection pool,
 	// preventing stale keep-alive connections from one backend affecting others.
-	httpTransport, err := newBackendTransport(target.CABundlePath)
+	httpTransport, err := newBackendTransport(target.CABundlePath, target.CABundleData)
 	if err != nil {
 		return nil, fmt.Errorf("failed to create transport for backend %s: %w", target.WorkloadID, err)
 	}

pkg/vmcp/client/client_test.go

@@ -108,9 +108,9 @@ func TestQueryHelpers_PartialCapabilities(t *testing.T) {
 func TestNewBackendTransport_IsolatesFromDefault(t *testing.T) {
 	t.Parallel()
 
-	t1, err1 := newBackendTransport("")
+	t1, err1 := newBackendTransport("", nil)
 	require.NoError(t, err1)
-	t2, err2 := newBackendTransport("")
+	t2, err2 := newBackendTransport("", nil)
 	require.NoError(t, err2)
 
 	// Each call must return a distinct transport — not the shared DefaultTransport.
@@ -147,6 +147,7 @@ func TestNewBackendTransport_CustomCA(t *testing.T) {
 	tests := []struct {
 		name          string
 		setupFile     func(t *testing.T) string
+		caBundleData  []byte
 		expectError   bool
 		errorContains string
 		checkResult   func(t *testing.T, tr *http.Transport)
@@ -207,14 +208,53 @@ func TestNewBackendTransport_CustomCA(t *testing.T) {
 			expectError:   true,
 			errorContains: "failed to parse CA certificate",
 		},
+		{
+			name: "valid CA data bytes applies custom TLS config",
+			setupFile: func(t *testing.T) string {
+				t.Helper()
+				return "" // no file path
+			},
+			caBundleData: generateTestCACert(t),
+			expectError:  false,
+			checkResult: func(t *testing.T, tr *http.Transport) {
+				t.Helper()
+				require.NotNil(t, tr.TLSClientConfig)
+				assert.Equal(t, uint16(tls.VersionTLS12), tr.TLSClientConfig.MinVersion)
+				assert.NotNil(t, tr.TLSClientConfig.RootCAs)
+			},
+		},
+		{
+			name: "invalid CA data bytes returns error",
+			setupFile: func(t *testing.T) string {
+				t.Helper()
+				return "" // no file path
+			},
+			caBundleData:  []byte("not-a-cert"),
+			expectError:   true,
+			errorContains: "failed to parse CA certificate from inline data",
+		},
+		{
+			name: "CA data takes precedence over file path",
+			setupFile: func(t *testing.T) string {
+				t.Helper()
+				return "/nonexistent/path.crt" // file doesn't exist but shouldn't be read
+			},
+			caBundleData: generateTestCACert(t),
+			expectError:  false,
+			checkResult: func(t *testing.T, tr *http.Transport) {
+				t.Helper()
+				require.NotNil(t, tr.TLSClientConfig)
+				assert.NotNil(t, tr.TLSClientConfig.RootCAs)
+			},
+		},
 	}
 
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
 			caPath := tt.setupFile(t)
-			tr, err := newBackendTransport(caPath)
+			tr, err := newBackendTransport(caPath, tt.caBundleData)
 
 			if tt.expectError {
 				require.Error(t, err)