Skip to content

Commit d7ef5eb

Browse files
chemicLchemicL
committed
Update conformance test suite for v2
Signed-off-by: Dariusz Jędrzejczyk <dariusz.jedrzejczyk@broadcom.com>
1 parent cbb235f commit d7ef5eb

File tree

10 files changed

+37
-40
lines changed

10 files changed

+37
-40
lines changed

‎conformance-tests/VALIDATION_RESULTS.md‎

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
## Summary
44

55
**Server Tests:** 40/40 passed (100%)
6-
**Client Tests:** 3/4 scenarios passed (9/10 checks passed)
6+
**Client Tests:** 4/4 scenarios passed (8/8 checks passed, 2 warnings)
77
**Auth Tests:** 12/14 scenarios fully passing (178 passed, 1 failed, 1 warning, 85.7% scenarios, 98.9% checks)
88

99
## Server Test Results
@@ -20,20 +20,17 @@
2020

2121
## Client Test Results
2222

23-
### Passing (3/4 scenarios, 9/10 checks)
23+
### Passing (4/4 scenarios, 8/8 checks)
2424

2525
- **initialize (1/1):** Protocol negotiation, clientInfo, capabilities
2626
- **tools_call (1/1):** Tool discovery and invocation
2727
- **elicitation-sep1034-client-defaults (5/5):** Default values for string, integer, number, enum, boolean
28-
29-
### Partially Passing (1/4 scenarios, 1/2 checks)
30-
31-
- **sse-retry (1/2 + 1 warning):**
28+
- **sse-retry (1/1 + 2 warnings):**
3229
- ✅ Reconnects after stream closure
33-
- ❌ Does not respect retry timing
30+
- ⚠️ Does not respect retry timing (MUST requirement)
3431
- ⚠️ Does not send Last-Event-ID header (SHOULD requirement)
3532

36-
**Issue:** Client treats `retry:` SSE field as invalid instead of parsing it for reconnection timing.
33+
**Note:** Client treats `retry:` SSE field as invalid instead of parsing it for reconnection timing.
3734

3835
## Auth Test Results (Spring HTTP Client)
3936

@@ -87,7 +84,7 @@ cd conformance-tests/client-jdk-http-client
8784
# Run all scenarios
8885
for scenario in initialize tools_call elicitation-sep1034-client-defaults sse-retry; do
8986
npx @modelcontextprotocol/conformance client \
90-
--command "java -jar target/client-jdk-http-client-1.1.0-SNAPSHOT.jar" \
87+
--command "java -jar target/client-jdk-http-client-2.0.0-SNAPSHOT.jar" \
9188
--scenario $scenario
9289
done
9390
```
@@ -104,7 +101,7 @@ cd conformance-tests/client-spring-http-client
104101
# Run auth suite
105102
npx @modelcontextprotocol/conformance@0.1.15 client \
106103
--spec-version 2025-11-25 \
107-
--command "java -jar target/client-spring-http-client-1.1.0-SNAPSHOT.jar" \
104+
--command "java -jar target/client-spring-http-client-2.0.0-SNAPSHOT.jar" \
108105
--suite auth
109106
```
110107

‎conformance-tests/client-jdk-http-client/README.md‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ Currently implemented scenarios:
4040
- ⚠️ Lists available tools from the server
4141
- ⚠️ Calls the `test_reconnection` tool which triggers SSE stream closure
4242
- ✅ Client reconnects after stream closure (PASSING)
43-
- ❌ Client does not respect retry timing (FAILING)
43+
- ⚠️ Client does not respect retry timing (WARNING - MUST requirement)
4444
- ⚠️ Client does not send Last-Event-ID header (WARNING - SHOULD requirement)
4545

4646
## Building
@@ -54,7 +54,7 @@ cd conformance-tests/client-jdk-http-client
5454

5555
This creates an executable JAR at:
5656
```
57-
target/client-jdk-http-client-1.1.0-SNAPSHOT.jar
57+
target/client-jdk-http-client-2.0.0-SNAPSHOT.jar
5858
```
5959

6060
## Running Tests
@@ -65,27 +65,27 @@ Run a single scenario:
6565

6666
```bash
6767
npx @modelcontextprotocol/conformance client \
68-
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-1.1.0-SNAPSHOT.jar" \
68+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-2.0.0-SNAPSHOT.jar" \
6969
--scenario initialize
7070

7171
npx @modelcontextprotocol/conformance client \
72-
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-1.1.0-SNAPSHOT.jar" \
72+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-2.0.0-SNAPSHOT.jar" \
7373
--scenario tools_call
7474

7575
npx @modelcontextprotocol/conformance client \
76-
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-1.1.0-SNAPSHOT.jar" \
76+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-2.0.0-SNAPSHOT.jar" \
7777
--scenario elicitation-sep1034-client-defaults
7878

7979
npx @modelcontextprotocol/conformance client \
80-
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-1.1.0-SNAPSHOT.jar" \
80+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-2.0.0-SNAPSHOT.jar" \
8181
--scenario sse-retry
8282
```
8383

8484
Run with verbose output:
8585

8686
```bash
8787
npx @modelcontextprotocol/conformance client \
88-
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-1.1.0-SNAPSHOT.jar" \
88+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-2.0.0-SNAPSHOT.jar" \
8989
--scenario initialize \
9090
--verbose
9191
```
@@ -96,18 +96,18 @@ You can also run the client manually if you have a test server:
9696

9797
```bash
9898
export MCP_CONFORMANCE_SCENARIO=initialize
99-
java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-1.1.0-SNAPSHOT.jar http://localhost:3000/mcp
99+
java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-2.0.0-SNAPSHOT.jar http://localhost:3000/mcp
100100
```
101101

102102
## Test Results
103103

104104
The conformance framework generates test results showing:
105105

106-
**Current Status (3/4 scenarios passing):**
106+
**Current Status (4/4 scenarios passing):**
107107
- ✅ initialize: 1/1 checks passed
108108
- ✅ tools_call: 1/1 checks passed
109109
- ✅ elicitation-sep1034-client-defaults: 5/5 checks passed
110-
- ⚠️ sse-retry: 1/2 checks passed, 1 warning
110+
- ⚠️ sse-retry: 1/1 checks passed, 2 warnings
111111

112112
Test result files are generated in `results/<scenario>-<timestamp>/`:
113113
- `checks.json`: Array of conformance check results with pass/fail status
@@ -116,7 +116,7 @@ Test result files are generated in `results/<scenario>-<timestamp>/`:
116116

117117
### Known Issue: SSE Retry Handling
118118

119-
The `sse-retry` scenario currently fails because:
119+
The `sse-retry` scenario passes but has 2 warnings:
120120
1. The client treats the SSE `retry:` field as invalid instead of parsing it
121121
2. The client does not implement retry timing (reconnects immediately)
122122
3. The client does not send the Last-Event-ID header on reconnection

‎conformance-tests/client-jdk-http-client/src/main/java/io/modelcontextprotocol/conformance/client/ConformanceJdkClientMcpClient.java‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ private static McpSyncClient createClient(String serverUrl) {
8080
HttpClientStreamableHttpTransport transport = HttpClientStreamableHttpTransport.builder(serverUrl).build();
8181

8282
return McpClient.sync(transport)
83-
.clientInfo(new McpSchema.Implementation("test-client", "1.0.0"))
83+
.clientInfo(new McpSchema.Implementation("test-client", "2.0.0"))
8484
.requestTimeout(Duration.ofSeconds(30))
8585
.build();
8686
}
@@ -97,7 +97,7 @@ private static McpSyncClient createClientWithElicitation(String serverUrl) {
9797
var capabilities = McpSchema.ClientCapabilities.builder().elicitation().build();
9898

9999
return McpClient.sync(transport)
100-
.clientInfo(new McpSchema.Implementation("test-client", "1.0.0"))
100+
.clientInfo(new McpSchema.Implementation("test-client", "2.0.0"))
101101
.requestTimeout(Duration.ofSeconds(30))
102102
.capabilities(capabilities)
103103
.elicitation(request -> {

‎conformance-tests/client-spring-http-client/README.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ cd conformance-tests/client-spring-http-client
6767

6868
This creates an executable JAR at:
6969
```
70-
target/client-spring-http-client-1.1.0-SNAPSHOT.jar
70+
target/client-spring-http-client-2.0.0-SNAPSHOT.jar
7171
```
7272

7373
## Running Tests
@@ -79,7 +79,7 @@ Run the full auth suite:
7979
```bash
8080
npx @modelcontextprotocol/conformance@0.1.15 client \
8181
--spec-version 2025-11-25 \
82-
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-1.1.0-SNAPSHOT.jar" \
82+
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-2.0.0-SNAPSHOT.jar" \
8383
--suite auth
8484
```
8585

@@ -88,7 +88,7 @@ Run a single scenario:
8888
```bash
8989
npx @modelcontextprotocol/conformance@0.1.15 client \
9090
--spec-version 2025-11-25 \
91-
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-1.1.0-SNAPSHOT.jar" \
91+
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-2.0.0-SNAPSHOT.jar" \
9292
--scenario auth/metadata-default
9393
```
9494

@@ -97,7 +97,7 @@ Run with verbose output:
9797
```bash
9898
npx @modelcontextprotocol/conformance@0.1.15 client \
9999
--spec-version 2025-11-25 \
100-
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-1.1.0-SNAPSHOT.jar" \
100+
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-2.0.0-SNAPSHOT.jar" \
101101
--scenario auth/metadata-default \
102102
--verbose
103103
```
@@ -108,7 +108,7 @@ You can also run the client manually if you have a test server:
108108

109109
```bash
110110
export MCP_CONFORMANCE_SCENARIO=auth/metadata-default
111-
java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-1.1.0-SNAPSHOT.jar http://localhost:3000/mcp
111+
java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-2.0.0-SNAPSHOT.jar http://localhost:3000/mcp
112112
```
113113

114114
## Known Issues

‎conformance-tests/client-spring-http-client/src/main/java/io/modelcontextprotocol/conformance/client/scenario/DefaultScenario.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ public void execute(String serverUrl) {
5858

5959
this.client = McpClient.sync(transport)
6060
.transportContextProvider(new AuthenticationMcpTransportContextProvider())
61-
.clientInfo(new McpSchema.Implementation("test-client", "1.0.0"))
61+
.clientInfo(new McpSchema.Implementation("test-client", "2.0.0"))
6262
.requestTimeout(Duration.ofSeconds(30))
6363
.build();
6464

‎conformance-tests/client-spring-http-client/src/main/java/io/modelcontextprotocol/conformance/client/scenario/PreRegistrationScenario.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ public void execute(String serverUrl) {
6060

6161
var client = McpClient.sync(transport)
6262
.transportContextProvider(new AuthenticationMcpTransportContextProvider())
63-
.clientInfo(new McpSchema.Implementation("test-client", "1.0.0"))
63+
.clientInfo(new McpSchema.Implementation("test-client", "2.0.0"))
6464
.requestTimeout(Duration.ofSeconds(30))
6565
.build();
6666

‎conformance-tests/server-servlet/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ curl -X POST http://localhost:8080/mcp \
142142
-H "Content-Type: application/json" \
143143
-H "Accept: text/event-stream" \
144144
-H "mcp-session-id: test-session-123" \
145-
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test-client","version":"1.0.0"}}}'
145+
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test-client","version":"2.0.0"}}}'
146146
```
147147

148148
## Architecture

‎conformance-tests/server-servlet/src/main/java/io/modelcontextprotocol/conformance/server/ConformanceServlet.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ public static void main(String[] args) throws Exception {
7575

7676
// Build server with all conformance test features
7777
var mcpServer = McpServer.sync(transportProvider)
78-
.serverInfo("mcp-conformance-server", "1.0.0")
78+
.serverInfo("mcp-conformance-server", "2.0.0")
7979
.capabilities(ServerCapabilities.builder()
8080
.completions()
8181
.resources(true, false)

‎docs/quickstart.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,7 @@ Add the BOM to your project:
123123
<dependency>
124124
<groupId>io.modelcontextprotocol.sdk</groupId>
125125
<artifactId>mcp-bom</artifactId>
126-
<version>1.0.0</version>
126+
<version>2.0.0</version>
127127
<type>pom</type>
128128
<scope>import</scope>
129129
</dependency>
@@ -135,7 +135,7 @@ Add the BOM to your project:
135135

136136
```groovy
137137
dependencies {
138-
implementation platform("io.modelcontextprotocol.sdk:mcp-bom:1.0.0")
138+
implementation platform("io.modelcontextprotocol.sdk:mcp-bom:2.0.0")
139139
//...
140140
}
141141
```

‎docs/server.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ The server supports both synchronous and asynchronous APIs, allowing for flexibl
3131
```java
3232
// Create a server with custom configuration
3333
McpSyncServer syncServer = McpServer.sync(transportProvider)
34-
.serverInfo("my-server", "1.0.0")
34+
.serverInfo("my-server", "2.0.0")
3535
.capabilities(ServerCapabilities.builder()
3636
.resources(false, true) // Resource support: subscribe=false, listChanged=true
3737
.tools(true) // Enable tool support with list changes
@@ -55,7 +55,7 @@ The server supports both synchronous and asynchronous APIs, allowing for flexibl
5555
```java
5656
// Create an async server with custom configuration
5757
McpAsyncServer asyncServer = McpServer.async(transportProvider)
58-
.serverInfo("my-server", "1.0.0")
58+
.serverInfo("my-server", "2.0.0")
5959
.capabilities(ServerCapabilities.builder()
6060
.resources(false, true) // Resource support: subscribe=false, listChanged=true
6161
.tools(true) // Enable tool support with list changes
@@ -446,7 +446,7 @@ Enable subscription support in the server capabilities:
446446

447447
```java
448448
McpSyncServer server = McpServer.sync(transportProvider)
449-
.serverInfo("my-server", "1.0.0")
449+
.serverInfo("my-server", "2.0.0")
450450
.capabilities(ServerCapabilities.builder()
451451
.resources(true, false) // subscribe=true, listChanged=false
452452
.build())
@@ -587,7 +587,7 @@ Once connected to a compatible client, the server can request language model gen
587587
```java
588588
// Create a server
589589
McpSyncServer server = McpServer.sync(transportProvider)
590-
.serverInfo("my-server", "1.0.0")
590+
.serverInfo("my-server", "2.0.0")
591591
.build();
592592

593593
// Define a tool that uses sampling
@@ -641,7 +641,7 @@ Once connected to a compatible client, the server can request language model gen
641641
```java
642642
// Create a server
643643
McpAsyncServer server = McpServer.async(transportProvider)
644-
.serverInfo("my-server", "1.0.0")
644+
.serverInfo("my-server", "2.0.0")
645645
.build();
646646

647647
// Define a tool that uses sampling
@@ -766,7 +766,7 @@ var tool = new McpServerFeatures.AsyncToolSpecification(
766766
});
767767

768768
var mcpServer = McpServer.async(mcpServerTransportProvider)
769-
.serverInfo("test-server", "1.0.0")
769+
.serverInfo("test-server", "2.0.0")
770770
.capabilities(
771771
ServerCapabilities.builder()
772772
.logging() // Enable logging support

0 commit comments

Comments
 (0)