-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Expand file tree
/
Copy pathexpected-failures.yml
More file actions
52 lines (49 loc) · 2.46 KB
/
Copy pathexpected-failures.yml
File metadata and controls
52 lines (49 loc) · 2.46 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# Conformance scenarios not yet passing against the Python SDK on main.
# CI exits 0 if only these fail, exits 1 on unexpected failures or stale entries.
#
# Baseline established against @modelcontextprotocol/conformance pinned in
# .github/workflows/conformance.yml (CONFORMANCE_VERSION = 0.2.0-alpha.3).
# New conformance releases are adopted by deliberately bumping that pin and
# reconciling this file in the same change.
#
# Entries are grouped by SEP. As each SEP lands in the SDK the corresponding
# scenarios start passing and MUST be removed from this list (the runner fails
# on stale entries), so the baseline burns down per milestone.
client:
# --- Draft-spec scenarios (in `--suite draft`, also part of `--suite all`) ---
# SEP-2575 (request metadata / _meta envelope): client does not populate the
# _meta envelope or the MCP-Protocol-Version header semantics yet.
- request-metadata
# SEP-2322 (multi-round-trip requests): client does not echo requestState /
# handle IncompleteResult yet.
- sep-2322-client-request-state
# SEP-2243 (HTTP standardization): no fixture handler / client header support yet.
- http-custom-headers
- http-invalid-tool-headers
# SEP-2106 (JSON Schema $ref handling): client still dereferences network $refs.
- json-schema-ref-no-deref
# SEP-2468 (authorization response iss parameter): not implemented in the client.
- auth/iss-supported
- auth/iss-not-advertised
- auth/iss-supported-missing
- auth/iss-wrong-issuer
- auth/iss-unexpected
- auth/iss-normalized
- auth/metadata-issuer-mismatch
# SEP-2352 (authorization server migration): client does not re-register when
# PRM authorization_servers changes.
- auth/authorization-server-migration
# SEP-837 (application_type during DCR): the check only fires on draft-version
# runs; this draft scenario is the one place the client still hits it.
- auth/offline-access-not-supported
# --- Pre-existing scenarios that fail on checks added after conformance 0.1.15 ---
# SEP-2350 (scope step-up): WARNING-only; the expected-failures evaluator
# counts WARNINGs as failures.
- auth/scope-step-up
# SEP-990 (enterprise-managed authorization extension): no fixture handler /
# client support for the token-exchange + JWT bearer flow.
- auth/enterprise-managed-authorization
# The `active` suite (30 scenarios / 42 assertions) is fully green against
# mcp-everything-server. Draft-suite entries are added when the workflow
# gains a `--suite draft` step.
server: []