Skip to content

Commit 0e23854

Browse files
jeffhandleyCopilotCopilot
authored
Add README code sample validation step to release-notes skill (#1371)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 8c46411 commit 0e23854

2 files changed

Lines changed: 161 additions & 6 deletions

File tree

.github/skills/release-notes/SKILL.md

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,19 @@ When **editing an existing release**, also extract any breaking changes already
6161

6262
Use the results (confirmed breaking changes with impact ordering and detail bullets) in the remaining steps.
6363

64-
### Step 4: Review Sections
64+
### Step 4: Validate README Code Samples
65+
66+
Verify that all C# code samples in the package README files compile against the current SDK at the target commit. Follow [references/readme-snippets.md](references/readme-snippets.md) for the full procedure.
67+
68+
1. Extract `csharp`-fenced code blocks from `README.md`, `src/ModelContextProtocol.Core/README.md`, and `src/ModelContextProtocol.AspNetCore/README.md`
69+
2. Create a temporary test project at `tests/ReadmeSnippetValidation/` that references the SDK projects
70+
3. Wrap each code block in a compilable method, applying fixups for incomplete snippets (replace `...` with `null!`, add common usings)
71+
4. Build the project with `dotnet build`
72+
5. Report results — classify any failures as API mismatches (README bugs) or structural fragments
73+
6. If API mismatches are found, propose fixes and get user confirmation before editing READMEs
74+
7. Delete the temporary `tests/ReadmeSnippetValidation/` directory
75+
76+
### Step 5: Review Sections
6577

6678
Present each section for user review:
6779
1. **Breaking Changes** — sorted most → least impactful (from Step 3 results)
@@ -70,7 +82,7 @@ Present each section for user review:
7082
4. **Test Improvements** — chronological
7183
5. **Repository Infrastructure Updates** — chronological
7284

73-
### Step 5: Acknowledgements
85+
### Step 6: Acknowledgements
7486

7587
Identify contributors beyond PR authors:
7688
1. **New contributors** — first contribution to the repository in this release
@@ -85,7 +97,7 @@ Exclude anyone already listed as a PR author. Format:
8597
```
8698
Reviewers go on a single bullet, sorted by number of PRs reviewed (most first), without citing the count.
8799

88-
### Step 6: Preamble
100+
### Step 7: Preamble
89101

90102
Every release **must** have a preamble — a short paragraph summarizing the release theme that appears before the first `##` heading. The preamble is not optional. The preamble may mention the presence of breaking changes as part of the theme summary, but the versioning documentation link belongs under the Breaking Changes heading (see template), not in the preamble.
91103

@@ -94,15 +106,15 @@ Every release **must** have a preamble — a short paragraph summarizing the rel
94106

95107
Present the options and let the user choose one, edit one, or enter their own text or markdown.
96108

97-
### Step 7: Final Assembly
109+
### Step 8: Final Assembly
98110

99111
1. Combine the confirmed preamble with all sections from previous steps.
100112
2. **Notable callouts** — only if something is extraordinarily noteworthy.
101113
3. Present the **complete release notes** for user approval.
102114

103115
Follow [references/formatting.md](references/formatting.md) when composing and updating the release body.
104116

105-
### Step 8: Create or Update Draft Release
117+
### Step 9: Create or Update Draft Release
106118

107119
Display release metadata for user review:
108120
- **Title / Tag**: e.g. `v0.9.0-preview.1`
@@ -117,7 +129,7 @@ After confirmation:
117129

118130
When the user requests revisions after the initial update, always rewrite the complete body as a file — never perform in-place string replacements. See [references/formatting.md](references/formatting.md).
119131

120-
### Step 9: Bump Version
132+
### Step 10: Bump Version
121133

122134
After the draft release is created or updated, inform the user that the draft release is now associated with a known target commit (state the short SHA and commit message) and recommend proceeding with a version bump. Ask if they want to create a pull request to bump the version for the next release. If yes, invoke the **bump-version** skill and let it handle determining or requesting the target version number.
123135

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
# README Code Sample Validation
2+
3+
This reference describes how to validate that C# code samples in README files compile against the current SDK.
4+
5+
## Which READMEs to Validate
6+
7+
Validate code samples from the **package README** files — these are shipped with NuGet packages and are the primary documentation users see:
8+
9+
| README | Package |
10+
|--------|---------|
11+
| `README.md` (root) | ModelContextProtocol |
12+
| `src/ModelContextProtocol.Core/README.md` | ModelContextProtocol.Core |
13+
| `src/ModelContextProtocol.AspNetCore/README.md` | ModelContextProtocol.AspNetCore |
14+
15+
Sample README files (`samples/*/README.md`) are excluded — the samples themselves are buildable projects and are validated by CI.
16+
17+
## What to Extract
18+
19+
Extract only fenced code blocks tagged as `csharp` (` ```csharp `). Skip blocks tagged as plain ` ``` ` (shell commands, install instructions) or any other language.
20+
21+
### Handling Incomplete Snippets
22+
23+
README samples are often **incomplete** — they use `...` for placeholder values, omit `using` directives, or show only a method body. The validation wrapper must account for this:
24+
25+
- **Placeholder expressions** like `IChatClient chatClient = ...;` — replace `...` on the right-hand side of assignments with `null!`
26+
- **Missing usings** — the wrapper file supplies all common namespaces (see template below)
27+
- **Top-level statements** — wrap in an `async Task` method so `await` works
28+
- **Suppressed warnings** — disable CS1998 (async without await), CS8321 (unused local function), and similar non-substantive warnings
29+
30+
### Snippets That Cannot Be Validated
31+
32+
Some code blocks are illustrative fragments that cannot compile even with wrappers (e.g., partial class definitions shown in isolation, pseudo-code). If a snippet fails to compile after applying the standard fixups, examine the error:
33+
34+
- **API mismatch** (missing member, wrong type, wrong signature) → this is a **real bug** in the README that must be reported and fixed
35+
- **Structural issue** (missing context, incomplete fragment) → exclude this specific snippet from validation with a comment explaining why
36+
37+
## Test Project Approach
38+
39+
Create a **temporary** test project that references the SDK projects, wraps each README's code samples in compilable methods, and builds.
40+
41+
### Project File Template
42+
43+
```xml
44+
<Project Sdk="Microsoft.NET.Sdk">
45+
<PropertyGroup>
46+
<TargetFramework>net10.0</TargetFramework>
47+
<OutputType>Library</OutputType>
48+
<Nullable>enable</Nullable>
49+
<ImplicitUsings>enable</ImplicitUsings>
50+
<LangVersion>preview</LangVersion>
51+
<NoWarn>CS1998;CS8321;CS0168;CS0219;CS1591;CS8602</NoWarn>
52+
</PropertyGroup>
53+
<ItemGroup>
54+
<ProjectReference Include="..\..\src\ModelContextProtocol\ModelContextProtocol.csproj" />
55+
<ProjectReference Include="..\..\src\ModelContextProtocol.AspNetCore\ModelContextProtocol.AspNetCore.csproj" />
56+
</ItemGroup>
57+
</Project>
58+
```
59+
60+
Place the project at `tests/ReadmeSnippetValidation/ReadmeSnippetValidation.csproj`.
61+
62+
### Source File Template
63+
64+
Create one `.cs` file per README. Each file wraps the README's code blocks in static methods inside a class. Use this pattern:
65+
66+
```csharp
67+
#pragma warning disable CS1998
68+
#pragma warning disable CS8321
69+
70+
using Microsoft.AspNetCore.Builder;
71+
using Microsoft.Extensions.AI;
72+
using Microsoft.Extensions.DependencyInjection;
73+
using Microsoft.Extensions.Hosting;
74+
using Microsoft.Extensions.Logging;
75+
using ModelContextProtocol;
76+
using ModelContextProtocol.Client;
77+
using ModelContextProtocol.Protocol;
78+
using ModelContextProtocol.Server;
79+
using System.ComponentModel;
80+
using System.Text.Json;
81+
82+
namespace ReadmeSnippetValidation;
83+
84+
public static class RootReadmeSamples
85+
{
86+
// Snippet 1: Client example
87+
public static async Task ClientExample()
88+
{
89+
// ... pasted snippet code ...
90+
}
91+
92+
// Snippet 2: Server example
93+
public static async Task ServerExample()
94+
{
95+
// ... pasted snippet code ...
96+
}
97+
98+
// Snippet with attributed types (tools, prompts) go as nested or sibling types
99+
[McpServerToolType]
100+
public static class EchoTool
101+
{
102+
[McpServerTool, Description("Echoes the message back to the client.")]
103+
public static string Echo(string message) => $"hello {message}";
104+
}
105+
}
106+
```
107+
108+
### Build Commands
109+
110+
```sh
111+
# Restore and build the validation project
112+
dotnet restore tests/ReadmeSnippetValidation/ReadmeSnippetValidation.csproj
113+
dotnet build tests/ReadmeSnippetValidation/ReadmeSnippetValidation.csproj
114+
115+
### Cleanup
116+
117+
After validation, **always delete** the `tests/ReadmeSnippetValidation/` directory. It must not be committed.
118+
119+
## Reporting Results
120+
121+
### All Snippets Compile
122+
123+
Report success and move to the next step:
124+
> ✅ All README code samples compile successfully against the current SDK.
125+
126+
### Compilation Failures
127+
128+
For each failure:
129+
1. Identify the README file and which code block failed
130+
2. Show the compiler error
131+
3. Classify the error:
132+
- **API mismatch**: The README uses an API that doesn't exist or has a different signature. This indicates the README is outdated and needs updating.
133+
- **Structural**: The snippet is a fragment that can't be wrapped. Note it for exclusion.
134+
4. For API mismatches, investigate the correct current API and propose a fix
135+
5. Present all findings to the user for confirmation before making any README edits
136+
137+
### Fixing Issues
138+
139+
If the user approves fixes:
140+
1. Edit the README files directly with minimal, surgical changes
141+
2. Re-run the validation build to confirm fixes compile
142+
3. Ensure the overall solution still builds: `dotnet build`
143+
4. Include the README fixes in the same release or as a prerequisite PR — the user decides

0 commit comments

Comments
 (0)