Skip to content

VALIDATION_RESULTS.md

Latest commit

 

History

History
82 lines (59 loc) · 2.86 KB

VALIDATION_RESULTS.md

File metadata and controls

82 lines (59 loc) · 2.86 KB

MCP Java SDK Conformance Test Validation Results

Summary

Server Tests: 37/40 passed (92.5%)
Client Tests: 3/4 scenarios passed (9/10 checks passed)

Server Test Results

Passing (37/40)

  • Lifecycle & Utilities (4/4): initialize, ping, logging-set-level, completion-complete
  • Tools (11/11): All scenarios including progress notifications ✨
  • Elicitation (10/10): SEP-1034 defaults (5 checks), SEP-1330 enums (5 checks)
  • Resources (4/6): list, read-text, read-binary, templates-read
  • Prompts (4/4): list, simple, with-args, embedded-resource, with-image
  • SSE Transport (2/2): Multiple streams
  • Security (1/2): Localhost validation passes

Failing (3/40)

  1. resources-subscribe - Not implemented in SDK
  2. resources-unsubscribe - Not implemented in SDK
  3. dns-rebinding-protection - Missing Host/Origin validation (1/2 checks)

Client Test Results

Passing (3/4 scenarios, 9/10 checks)

  • initialize (1/1): Protocol negotiation, clientInfo, capabilities
  • tools_call (1/1): Tool discovery and invocation
  • elicitation-sep1034-client-defaults (5/5): Default values for string, integer, number, enum, boolean

Partially Passing (1/4 scenarios, 1/2 checks)

  • sse-retry (1/2 + 1 warning):
    • ✅ Reconnects after stream closure
    • ❌ Does not respect retry timing
    • ⚠️ Does not send Last-Event-ID header (SHOULD requirement)

Issue: Client treats retry: SSE field as invalid instead of parsing it for reconnection timing.

Known Limitations

  1. Resource Subscriptions: SDK doesn't implement resources/subscribe and resources/unsubscribe handlers
  2. Client SSE Retry: Client doesn't parse or respect the retry: field, reconnects immediately, and doesn't send Last-Event-ID header
  3. DNS Rebinding Protection: Missing Host/Origin header validation in server transport

Running Tests

Server

# Start server
cd conformance-tests/server-servlet
../../mvnw compile exec:java -Dexec.mainClass="io.modelcontextprotocol.conformance.server.ConformanceServlet"

# Run tests (in another terminal)
npx @modelcontextprotocol/conformance server --url http://localhost:8080/mcp --suite active

Client

# Build
cd conformance-tests/client-jdk-http-client
../../mvnw clean package -DskipTests

# Run all scenarios
for scenario in initialize tools_call elicitation-sep1034-client-defaults sse-retry; do
  npx @modelcontextprotocol/conformance client \
    --command "java -jar target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
    --scenario $scenario
done

Recommendations

High Priority

  1. Fix client SSE retry field handling in HttpClientStreamableHttpTransport
  2. Implement resource subscription handlers in McpStatelessAsyncServer

Medium Priority

  1. Add Host/Origin validation in HttpServletStreamableServerTransportProvider for DNS rebinding protection