Skip to content

Commit d9277e0

Browse files
halter73Copilot
andcommitted
tests: align conformance fixtures with @modelcontextprotocol/conformance main
The conformance suite's main branch (post 0.2.0-alpha.2) made two breaking changes that our pinned wire-name / spec-version contracts had to follow: 1. `DRAFT-2026-v1` wire literal flipped to `2026-07-28` (commit 6f83abf in the conformance repo). 2. `incomplete-result-*` server scenarios renamed to `input-required-result-*` and extended from 8 to 14 scenarios (PR #262 in the conformance repo, follow-on to our own MRTR rename in PR #1458 where `IncompleteResult` became `InputRequiredResult`). Track A (the published 0.2.0-alpha.2 pin) is unaffected and continues to skip these tests via the wire-version-match gate introduced in `f3698c71`. Track B (a private install of the conformance `main` branch with the four merged sessionless/MRTR/error-code/tasks PRs) now exercises 12 of the 14 MRTR scenarios end-to-end against this SDK's `ConformanceServer`. Changes in this commit: - `IncompleteResultTools.cs` and `IncompleteResultPrompts.cs` — rename 6 tool wire names plus 1 prompt wire name from `test_incomplete_result_*` (and `test_tool_with_elicitation`) to `test_input_required_result_*` so the conformance scenarios can find them. The C# class names are intentionally left as `Incomplete*` to keep the diff minimal; the comment block above `RunMrtrConformanceTest` documents the asymmetry. - `ServerConformanceTests.cs` — replace the 8-row MRTR theory with the 14-row theory matching the new conformance scenario set; flip the two `--spec-version DRAFT-2026-v1` references to `2026-07-28`; mark two scenarios skipped that require server-side patterns the `ConformanceServer` tools don't yet implement (HMAC-signed requestState for `input-required-result-tampered-state`; per-request capability gating for `input-required-result-capability-check`). Those scenarios are still feature-flagged behind the wire-version gate so they only attempt to run when the installed conformance package speaks `2026-07-28`. - `CachingConformanceTests.cs` — flip the `--spec-version` reference and rewrite the doc remarks to explain the wire-version-match gate. Validation: `dotnet test` over the targeted slice (5 conformance test classes) under serial run reports 17 pass / 1 fail / 2 skip against the private `compat/conformance-draft` build. The single failure (`Sep2243.http-custom-headers`) is a pre-existing C# SDK client bug exposed by the conformance-pin bump: the client sends no `Mcp-Param-*` headers when the server's tool advertises `x-mcp-header` annotations. The bug exists on `origin/main` (`git diff origin/main..HEAD -- src/ModelContextProtocol.Core/Client` on these files is empty) and is unrelated to the SEP-2575 / SEP-2567 draft work in this PR. Tracking as a follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent f3698c7 commit d9277e0

4 files changed

Lines changed: 53 additions & 34 deletions

File tree

tests/ModelContextProtocol.AspNetCore.Tests/CachingConformanceTests.cs

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -105,12 +105,14 @@ public async ValueTask DisposeAsync()
105105
/// (tools/list, prompts/list, resources/list, resources/templates/list, resources/read).
106106
/// </summary>
107107
/// <remarks>
108-
/// The scenario is draft-only (introduced in DRAFT-2026-v1) and uses the stateless lifecycle.
109-
/// It is gated on the installed conformance package version (>= 0.2.0) and is skipped when
110-
/// running against the currently-pinned package, so it activates automatically once a
111-
/// conformance package containing the caching scenario is installed (including a local private
112-
/// build installed via <c>npm install --no-save &lt;path-to-conformance&gt;</c>). The stateless
113-
/// server is started only after the gates pass, so a skipped run binds no port.
108+
/// The scenario is draft-only (introduced in spec wire version 2026-07-28) and uses the
109+
/// stateless lifecycle. It is gated on the installed conformance package version (>= 0.2.0)
110+
/// AND on the installed package emitting the draft wire string this SDK speaks (so it stays
111+
/// skipped under conformance 0.2.0-alpha.2 which still ships the placeholder
112+
/// <c>DRAFT-2026-v1</c>). It activates automatically once a conformance package emitting
113+
/// <c>2026-07-28</c> is installed (e.g. via
114+
/// <c>npm install --no-save &lt;path-to-conformance&gt;</c>). The stateless server is
115+
/// started only after the gates pass, so a skipped run binds no port.
114116
/// </remarks>
115117
public class CachingConformanceTests(ITestOutputHelper output)
116118
{
@@ -128,7 +130,7 @@ public async Task RunCachingConformanceTest()
128130
// explicitly (and suppress the MCP_CONFORMANCE_PROTOCOL_VERSION override to avoid a
129131
// conflicting duplicate --spec-version flag).
130132
var result = await NodeHelpers.RunServerConformanceAsync(
131-
$"server --url {server.ServerUrl} --scenario caching --spec-version DRAFT-2026-v1",
133+
$"server --url {server.ServerUrl} --scenario caching --spec-version 2026-07-28",
132134
line => { try { output.WriteLine(line); } catch { } },
133135
appendProtocolVersionFromEnv: false,
134136
cancellationToken: TestContext.Current.CancellationToken);

tests/ModelContextProtocol.AspNetCore.Tests/ServerConformanceTests.cs

Lines changed: 33 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -143,15 +143,15 @@ public async Task RunConformanceTest_HttpHeaderValidation()
143143
!NodeHelpers.HasSep2243Scenarios(),
144144
"SEP-2243 conformance scenarios not available (requires conformance package >= 0.2.0).");
145145

146-
// SEP-2243 is a draft (DRAFT-2026-v1) scenario that uses the stateless lifecycle, so it
146+
// SEP-2243 is a draft (2026-07-28) scenario that uses the stateless lifecycle, so it
147147
// requires a stateless server (a stateful server rejects the un-initialized list/call
148148
// requests with JSON-RPC -32000). Use a dedicated port range so it never collides with
149149
// the stateful class fixture (300x) or the caching stateless server (301x).
150150
await using var server = await StatelessConformanceServer.StartAsync(
151151
TestContext.Current.CancellationToken, basePort: 3021);
152152

153153
var result = await RunStatelessConformanceTestAsync(
154-
$"server --url {server.ServerUrl} --scenario http-header-validation --spec-version DRAFT-2026-v1");
154+
$"server --url {server.ServerUrl} --scenario http-header-validation --spec-version 2026-07-28");
155155

156156
Assert.True(result.Success,
157157
$"Conformance test failed.\n\nStdout:\n{result.Output}\n\nStderr:\n{result.Error}");
@@ -169,32 +169,49 @@ public async Task RunConformanceTest_HttpCustomHeaderServerValidation()
169169
TestContext.Current.CancellationToken, basePort: 3024);
170170

171171
var result = await RunStatelessConformanceTestAsync(
172-
$"server --url {server.ServerUrl} --scenario http-custom-header-server-validation --spec-version DRAFT-2026-v1");
172+
$"server --url {server.ServerUrl} --scenario http-custom-header-server-validation --spec-version 2026-07-28");
173173

174174
Assert.True(result.Success,
175175
$"Conformance test failed.\n\nStdout:\n{result.Output}\n\nStderr:\n{result.Error}");
176176
}
177177

178-
// SEP-2322 (Multi Round-Trip Requests / IncompleteResult) conformance scenarios.
178+
// SEP-2322 (Multi Round-Trip Requests / InputRequiredResult) conformance scenarios.
179179
// The csharp-sdk ConformanceServer surfaces the matching tools/prompts via
180-
// ConformanceServer.Tools.IncompleteResultTools and ConformanceServer.Prompts.IncompleteResultPrompts.
180+
// ConformanceServer.Tools.IncompleteResultTools and ConformanceServer.Prompts.IncompleteResultPrompts
181+
// (the class names predate the conformance-suite rename from "incomplete-result-*" to
182+
// "input-required-result-*"; the wire-level tool names now match the new convention).
181183
// Each scenario uses the conformance harness's RawMcpSession, which negotiates 2026-07-28
182184
// so the csharp-sdk emits InputRequiredResult on the wire. These tests skip until the
183-
// upstream conformance package ships with SEP-2322 scenarios
184-
// (https://github.com/modelcontextprotocol/conformance/pull/188).
185+
// installed conformance package ships SEP-2322 scenarios <em>and</em> emits this SDK's
186+
// draft wire string (see <see cref="NodeHelpers.HasMrtrScenarios"/>).
187+
//
188+
// Two scenarios (input-required-result-tampered-state and input-required-result-capability-check)
189+
// require advanced server-side logic not yet built into the ConformanceServer:
190+
// - tampered-state: HMAC integrity protection on requestState. Server-implementer concern
191+
// outside the SDK wire surface; would need a sample tool implementing the pattern.
192+
// - capability-check: per-request reading of clientCapabilities to gate which inputRequests
193+
// are returned. SDK exposes capabilities via JsonRpcMessageContext but no current tool
194+
// conditionally emits inputRequests based on them.
195+
// These rows are skipped until matching tool implementations are added.
185196
[Theory]
186-
[InlineData("incomplete-result-basic-elicitation")]
187-
[InlineData("incomplete-result-basic-sampling")]
188-
[InlineData("incomplete-result-basic-list-roots")]
189-
[InlineData("incomplete-result-request-state")]
190-
[InlineData("incomplete-result-multiple-input-requests")]
191-
[InlineData("incomplete-result-multi-round")]
192-
[InlineData("incomplete-result-missing-input-response")]
193-
[InlineData("incomplete-result-non-tool-request")]
197+
[InlineData("input-required-result-basic-elicitation")]
198+
[InlineData("input-required-result-basic-sampling")]
199+
[InlineData("input-required-result-basic-list-roots")]
200+
[InlineData("input-required-result-request-state")]
201+
[InlineData("input-required-result-multiple-input-requests")]
202+
[InlineData("input-required-result-multi-round")]
203+
[InlineData("input-required-result-missing-input-response")]
204+
[InlineData("input-required-result-non-tool-request")]
205+
[InlineData("input-required-result-result-type")]
206+
[InlineData("input-required-result-unsupported-methods")]
207+
[InlineData("input-required-result-tampered-state", Skip = "Requires HMAC-protected requestState pattern in ConformanceServer tools (not yet implemented).")]
208+
[InlineData("input-required-result-capability-check", Skip = "Requires per-request capability-aware inputRequest gating in ConformanceServer tools (not yet implemented).")]
209+
[InlineData("input-required-result-ignore-extra-params")]
210+
[InlineData("input-required-result-validate-input")]
194211
public async Task RunMrtrConformanceTest(string scenario)
195212
{
196213
Assert.SkipWhen(!NodeHelpers.IsNodeInstalled(), "Node.js is not installed. Skipping conformance tests.");
197-
Assert.SkipWhen(!NodeHelpers.HasMrtrScenarios(), "SEP-2322 MRTR conformance scenarios not yet available in the published @modelcontextprotocol/conformance package.");
214+
Assert.SkipWhen(!NodeHelpers.HasMrtrScenarios(), "SEP-2322 MRTR conformance scenarios not yet available in the published @modelcontextprotocol/conformance package (or installed version uses a stale draft wire string).");
198215

199216
var result = await RunConformanceTestsAsync(
200217
$"server --url {fixture.ServerUrl} --scenario {scenario}");

tests/ModelContextProtocol.ConformanceServer/Prompts/IncompleteResultPrompts.cs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,8 @@ namespace ConformanceServer.Prompts;
1616
[McpServerPromptType]
1717
public sealed class IncompleteResultPrompts
1818
{
19-
[McpServerPrompt(Name = "test_incomplete_result_prompt")]
20-
[Description("SEP-2322 D1: prompts/get returns IncompleteResult until user_context is supplied.")]
19+
[McpServerPrompt(Name = "test_input_required_result_prompt")]
20+
[Description("SEP-2322 D1: prompts/get returns InputRequiredResult until user_context is supplied.")]
2121
public static GetPromptResult IncompleteResultPrompt(RequestContext<GetPromptRequestParams> context)
2222
{
2323
if (context.Params!.InputResponses is { } responses &&

tests/ModelContextProtocol.ConformanceServer/Tools/IncompleteResultTools.cs

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@ namespace ConformanceServer.Tools;
2020
public sealed class IncompleteResultTools
2121
{
2222
// ──── A1: Basic Elicitation ─────────────────────────────────────────────
23-
[McpServerTool(Name = "test_tool_with_elicitation")]
24-
[Description("SEP-2322 A1: returns IncompleteResult with elicitation/create keyed 'user_name'.")]
23+
[McpServerTool(Name = "test_input_required_result_elicitation")]
24+
[Description("SEP-2322 A1: returns InputRequiredResult with elicitation/create keyed 'user_name'.")]
2525
public static CallToolResult ToolWithElicitation(RequestContext<CallToolRequestParams> context)
2626
{
2727
if (context.Params!.InputResponses is { } responses &&
@@ -51,8 +51,8 @@ public static CallToolResult ToolWithElicitation(RequestContext<CallToolRequestP
5151
}
5252

5353
// ──── A2: Basic Sampling ────────────────────────────────────────────────
54-
[McpServerTool(Name = "test_incomplete_result_sampling")]
55-
[Description("SEP-2322 A2: returns IncompleteResult with sampling/createMessage keyed 'capital_question'.")]
54+
[McpServerTool(Name = "test_input_required_result_sampling")]
55+
[Description("SEP-2322 A2: returns InputRequiredResult with sampling/createMessage keyed 'capital_question'.")]
5656
public static CallToolResult ToolWithSampling(RequestContext<CallToolRequestParams> context)
5757
{
5858
if (context.Params!.InputResponses is { } responses &&
@@ -81,8 +81,8 @@ public static CallToolResult ToolWithSampling(RequestContext<CallToolRequestPara
8181
}
8282

8383
// ──── A3: Basic ListRoots ───────────────────────────────────────────────
84-
[McpServerTool(Name = "test_incomplete_result_list_roots")]
85-
[Description("SEP-2322 A3: returns IncompleteResult with roots/list keyed 'client_roots'.")]
84+
[McpServerTool(Name = "test_input_required_result_list_roots")]
85+
[Description("SEP-2322 A3: returns InputRequiredResult with roots/list keyed 'client_roots'.")]
8686
public static CallToolResult ToolWithListRoots(RequestContext<CallToolRequestParams> context)
8787
{
8888
if (context.Params!.InputResponses is { } responses &&
@@ -102,7 +102,7 @@ public static CallToolResult ToolWithListRoots(RequestContext<CallToolRequestPar
102102
// ──── B1: requestState round-trip ───────────────────────────────────────
103103
private const string RequestStateToken = "mrtr-conformance-state-v1";
104104

105-
[McpServerTool(Name = "test_incomplete_result_request_state")]
105+
[McpServerTool(Name = "test_input_required_result_request_state")]
106106
[Description("SEP-2322 B1: round-trips a requestState string; R2 echoes 'state-ok' on success.")]
107107
public static CallToolResult ToolWithRequestState(RequestContext<CallToolRequestParams> context)
108108
{
@@ -135,7 +135,7 @@ public static CallToolResult ToolWithRequestState(RequestContext<CallToolRequest
135135
}
136136

137137
// ──── B2: Multiple input requests in one round ──────────────────────────
138-
[McpServerTool(Name = "test_incomplete_result_multiple_inputs")]
138+
[McpServerTool(Name = "test_input_required_result_multiple_inputs")]
139139
[Description("SEP-2322 B2: returns 3 simultaneous inputRequests (elicit + sampling + roots) plus requestState.")]
140140
public static CallToolResult ToolWithMultipleInputs(RequestContext<CallToolRequestParams> context)
141141
{
@@ -177,7 +177,7 @@ public static CallToolResult ToolWithMultipleInputs(RequestContext<CallToolReque
177177
}
178178

179179
// ──── B3: Multi-round (R1 -> incomplete, R2 -> incomplete (new state), R3 -> complete) ─────
180-
[McpServerTool(Name = "test_incomplete_result_multi_round")]
180+
[McpServerTool(Name = "test_input_required_result_multi_round")]
181181
[Description("SEP-2322 B3: three-round flow whose requestState changes between rounds.")]
182182
public static CallToolResult ToolWithMultiRound(RequestContext<CallToolRequestParams> context)
183183
{

0 commit comments

Comments
 (0)