Skip to content

Commit ccaa02c

Browse files
jerm-droclaude
andcommitted
Move new tests to in-process MultiSession level
New tests that don't involve auth use real in-process MCP backends via httptest.NewServer() instead of K8s. This constructs the full decorator stack (factory, conflict resolution, filter, composite tools, optimizer) without containers or a Kind cluster. Follows the existing pattern in connector_integration_test.go. K8s E2E tests remain as regression gates for auth-dependent flows and CRD-driven configuration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 1e3dacd commit ccaa02c

File tree

1 file changed

+44
-34
lines changed

1 file changed

+44
-34
lines changed

rfcs/THV-0058-inline-aggregator-filter-decorator.md

Lines changed: 44 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -298,33 +298,35 @@ The decorator-based architecture makes it straightforward to add new session-lev
298298

299299
## Testing Strategy
300300

301-
All validation is at the K8s/Ginkgo E2E level (`test/e2e/thv-operator/virtualmcp/`). Tests deploy real MCPServers + VirtualMCPServers to a Kind cluster and exercise the full stack via MCP clients.
301+
New tests that don't involve auth are written as in-process integration tests at the `MultiSession` level, using real MCP backends hosted via `httptest.NewServer()`. This constructs the full vMCP stack (factory → conflict resolution → decorators → tool calls) without a K8s dependency. The existing pattern lives in `pkg/vmcp/session/connector_integration_test.go``startInProcessMCPServer()` creates real MCP servers with registered tools, and the session factory connects to them with full capability discovery.
302+
303+
K8s/Ginkgo E2E tests (`test/e2e/thv-operator/virtualmcp/`) remain the regression gates for auth-dependent flows and CRD-driven configuration.
302304

303305
### Feature combination matrix
304306

305-
The features being modified interact with each other. The matrix below maps every relevant combination to an E2E test — either an existing one that provides coverage or a new one that must be written.
307+
The features being modified interact with each other. The matrix below maps every relevant combination to a test — either an existing one that provides coverage or a new one that must be written.
306308

307309
**Features**: **F** = Filter (per-workload), **EA** = ExcludeAll (per-workload or global), **R** = Renames/Overrides, **CR** = Conflict Resolution, **CT** = Composite Tools, **O** = Optimizer, **A** = Authn (non-anonymous)
308310

309-
| # | F | EA | R | CR | CT | O | A | E2E test | Status |
310-
|---|---|---|---|---|---|---|---|----------|--------|
311-
| 1 | | G | | prefix | | | anon | `virtualmcp_excludeall_global_test.go` | Existing |
312-
| 2 | Y | Y | | prefix | | | anon | `virtualmcp_aggregation_filtering_test.go` | Existing |
313-
| 3 | Y | Y | | prefix | Y | | anon | `virtualmcp_composite_hidden_tools_test.go` | Existing |
314-
| 4 | | | | prefix/priority/manual | | | anon | `virtualmcp_conflict_resolution_test.go` | Existing |
315-
| 5 | | | Y | prefix | | | anon | `virtualmcp_aggregation_overrides_test.go` | Existing |
316-
| 6 | Y | | Y | prefix | | | anon | `virtualmcp_toolconfig_test.go` (MCPToolConfig CRD) | Existing |
317-
| 7 | | | Y | prefix | Y | Y | anon | `virtualmcp_optimizer_composite_test.go` | Existing |
318-
| 8 | Y | Y | | prefix | Y | | anon | **`virtualmcp_composite_coercion_test.go`** | **New** |
319-
| 9 | Y | | Y | prefix | | | anon | **`virtualmcp_overrides_filter_test.go`** | **New** |
320-
| 10 | Y | Y | Y | prefix | Y | Y | anon | **`virtualmcp_full_stack_test.go`** | **New** |
321-
| 11 | | | | prefix | | | token | `virtualmcp_external_auth_test.go` | Existing |
311+
| # | F | EA | R | CR | CT | O | A | Test | Level | Status |
312+
|---|---|---|---|---|---|---|---|------|-------|--------|
313+
| 1 | | G | | prefix | | | anon | `virtualmcp_excludeall_global_test.go` | K8s E2E | Existing |
314+
| 2 | Y | Y | | prefix | | | anon | `virtualmcp_aggregation_filtering_test.go` | K8s E2E | Existing |
315+
| 3 | Y | Y | | prefix | Y | | anon | `virtualmcp_composite_hidden_tools_test.go` | K8s E2E | Existing |
316+
| 4 | | | | prefix/priority/manual | | | anon | `virtualmcp_conflict_resolution_test.go` | K8s E2E | Existing |
317+
| 5 | | | Y | prefix | | | anon | `virtualmcp_aggregation_overrides_test.go` | K8s E2E | Existing |
318+
| 6 | Y | | Y | prefix | | | anon | `virtualmcp_toolconfig_test.go` (MCPToolConfig CRD) | K8s E2E | Existing |
319+
| 7 | | | Y | prefix | Y | Y | anon | `virtualmcp_optimizer_composite_test.go` | K8s E2E | Existing |
320+
| 8 | Y | Y | | prefix | Y | | | **composite_coercion_test.go** | **In-process** | **New** |
321+
| 9 | Y | | Y | prefix | | | | **overrides_filter_test.go** | **In-process** | **New** |
322+
| 10 | Y | Y | Y | prefix | Y | Y | | **full_stack_test.go** | **In-process** | **New** |
323+
| 11 | | | | prefix | | | token | `virtualmcp_external_auth_test.go` | K8s E2E | Existing |
322324

323325
**Legend**: G = global ExcludeAllTools, Y = feature active in test, blank = not exercised.
324326

325-
### Existing tests (regression gates)
327+
### Existing K8s E2E tests (regression gates)
326328

327-
All existing tests in the matrix must pass without modification after the refactor. Any failure is a regression.
329+
All existing tests must pass without modification after the refactor. Any failure is a regression.
328330

329331
- **Row 1**: Global `ExcludeAllTools: true` hides all backend tools; MCP server still responds.
330332
- **Row 2**: Per-workload `Filter` + `ExcludeAll` applied to different backends; only filtered tools exposed.
@@ -335,58 +337,66 @@ All existing tests in the matrix must pass without modification after the refact
335337
- **Row 7**: Optimizer wraps composite tools + overridden backend tools through `find_tool`/`call_tool`.
336338
- **Row 11**: Token-based incoming auth with backend routing.
337339

338-
### New E2E tests
340+
### New in-process integration tests
341+
342+
These tests construct the full decorator stack against real in-process MCP backends (via `httptest.NewServer()` + `mcp-go`). No K8s, no containers — the session factory connects to backends over HTTP, discovers capabilities, applies conflict resolution + overrides, and wires the decorator chain. Tests call through the decorated `MultiSession` interface.
343+
344+
**Location**: `pkg/vmcp/session/` (alongside existing `connector_integration_test.go`)
339345

340346
#### Row 8: Composite tool type coercion with hidden tools (regression test for #4287)
341347

342-
**File**: `test/e2e/thv-operator/virtualmcp/virtualmcp_composite_coercion_test.go`
348+
**File**: `pkg/vmcp/session/composite_coercion_integration_test.go`
343349

344350
The specific bug this RFC fixes. When a composite tool workflow step calls a hidden backend tool with a numeric parameter, `getToolInputSchema()` fails to find the schema, skips coercion, and the backend rejects `"42"` (string) instead of `float64(42)`.
345351

346352
**Setup**:
347-
- Backend with a tool accepting an integer parameter (requires adding an `add` tool to yardstick: `{"a": integer, "b": integer}` → returns sum)
348-
- `ExcludeAll: true` for the backend
349-
- Composite tool whose workflow step calls the hidden tool via template expansion (`"{{ .params.a }}"`)
353+
- In-process MCP backend with a tool accepting integer parameters (e.g., `add` tool: `{"a": integer, "b": integer}` → returns sum)
354+
- Session factory with `ExcludeAll: true` for the backend
355+
- Composite tools decorator applied with a workflow step that calls the hidden tool via template expansion (`"{{ .params.a }}"`)
356+
- Filter decorator applied above composite tools
350357

351358
**Assertions**:
352-
- `tools/list` returns only the composite tool
353-
- Calling the composite tool succeeds — backend receives coerced numeric args
359+
- `sess.Tools()` at top of stack returns only the composite tool
360+
- `sess.CallTool()` with the composite tool succeeds — backend receives coerced numeric args
354361
- Response contains correct output (the sum), proving type coercion worked
355362

356363
**Must fail before the refactor, pass after.**
357364

358365
#### Row 9: Overrides + filter on the same backend
359366

360-
**File**: `test/e2e/thv-operator/virtualmcp/virtualmcp_overrides_filter_test.go`
367+
**File**: `pkg/vmcp/session/overrides_filter_integration_test.go`
361368

362369
No existing test exercises overrides and filter together. The `shouldAdvertiseTool` comment bug (says "before overrides" but receives post-override name) was never caught because this combination is untested.
363370

364371
**Setup**:
365-
- Backend with tool `echo`, overridden to `custom_echo`
372+
- In-process MCP backend with tool `echo`
373+
- Override config: `echo``custom_echo`
366374
- Filter configured with `["echo"]` (pre-override name)
375+
- Session constructed with full decorator chain
367376

368377
**Assertions**:
369378
- Document whether the filter matches pre- or post-override names
370-
- Verify the overridden tool is callable by its new name
379+
- Verify the overridden tool is callable by its new name via `sess.CallTool()`
371380
- Lock in the actual behavior so the refactor preserves it
372381

373382
#### Row 10: Full stack — all features combined
374383

375-
**File**: `test/e2e/thv-operator/virtualmcp/virtualmcp_full_stack_test.go`
384+
**File**: `pkg/vmcp/session/full_stack_integration_test.go`
376385

377386
Exercises the entire decorator stack with all features active simultaneously.
378387

379388
**Setup**:
380-
- Two backends with conflicting tool names (prefix conflict resolution)
389+
- Two in-process MCP backends with conflicting tool names (prefix conflict resolution)
381390
- Backend A: tool overridden (`echo``custom_echo`), plus filter hiding some tools
382391
- Backend B: `ExcludeAll: true`
383392
- Composite tool calling tools from both backends (including hidden + overridden)
384393
- Optimizer enabled
394+
- Full decorator chain: base session → composite tools → filter → optimizer
385395

386396
**Assertions**:
387-
- `tools/list` via optimizer returns only `find_tool`/`call_tool`
388-
- `find_tool` returns the composite tool + filtered backend A tools (not excluded backend B tools, not hidden backend A tools)
389-
- `call_tool` with composite tool succeeds — reaches both backends with correct original names
397+
- `sess.Tools()` via optimizer returns only `find_tool`/`call_tool`
398+
- `sess.CallTool("find_tool", ...)` returns the composite tool + filtered backend A tools (not excluded backend B tools)
399+
- `sess.CallTool("call_tool", {tool_name: compositeToolName, ...})` succeeds — reaches both backends with correct original names
390400
- Overridden tool callable by its new name through the optimizer
391401
- Backend B tools reachable via composite tool despite `ExcludeAll`
392402

@@ -408,12 +418,12 @@ Lightweight unit tests for the new filter decorator in isolation:
408418
### Verification
409419

410420
```bash
411-
# Unit tests
421+
# Unit + in-process integration tests
412422
go vet ./pkg/vmcp/... ./cmd/vmcp/...
413423
go test ./pkg/vmcp/... ./cmd/vmcp/...
414424
task lint-fix
415425

416-
# E2E tests (Kind cluster)
426+
# K8s E2E tests (Kind cluster — regression gates)
417427
task test-e2e
418428
```
419429

0 commit comments

Comments
 (0)