Skip to content

Commit baa702b

Browse files
richlanderCopilot
andauthored
Update release-notes skill guidance and tool naming (dotnet#10345)
* Update release-notes skill guidance and tool naming Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clarify behavior on populated release-note branches Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add existing-branch update playbook for release notes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Factor existing-branch handling into a shared skill Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Replace failing HAL link in release-graph skill Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Recompile release-notes agentic workflow Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Collapse repeated product-boundary guidance Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent b523573 commit baa702b

23 files changed

Lines changed: 406 additions & 192 deletions

File tree

.github/aw/actions-lock.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,11 @@
2929
"repo": "github/gh-aw-actions/setup",
3030
"version": "v0.65.6",
3131
"sha": "31130b20a8fd3ef263acbe2091267c0aace07e09"
32+
},
33+
"github/gh-aw-actions/setup@v0.67.0": {
34+
"repo": "github/gh-aw-actions/setup",
35+
"version": "v0.67.0",
36+
"sha": "cde65c546c2b0f6d3f3a9492a04e6687887c4fe8"
3237
}
3338
}
3439
}

.github/skills/editorial-scoring/SKILL.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ This is how foundational but initially niche work can still earn a good score.
6161
- Build or infrastructure churn
6262
- Refactoring with no user-visible effect
6363
- VMR sync noise, dependency updates, or repo automation
64+
- Product-boundary mismatch — repo-adjacent IDE/editor/design-time tooling work when the notes are for a different product surface
6465
- Highly specialized implementation details that read like internal jargon
6566
- Features that only matter after several stacked conditions are true (for example: uses single-file publish **and** cares deeply about startup **and** is willing to do extra training/tuning)
6667
- PRs with very thin descriptions where the only concrete signal is an internal runtime/tooling term like cDAC

.github/skills/generate-changes/SKILL.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
name: generate-changes
33
description: >
44
Generate `changes.json` for a .NET release milestone by selecting the correct
5-
VMR base/head refs and running `release-notes-gen generate changes`. Handles
5+
VMR base/head refs and running `release-notes generate changes`. Handles
66
preview-only multi-branch targeting (`main` vs release branches vs tags) and
77
emits the authoritative manifest of what shipped. DO NOT USE FOR: API
88
verification/diffs (use api-diff), feature scoring (use generate-features),
@@ -17,7 +17,7 @@ This is the **VMR-aware data acquisition stage** of the release notes pipeline:
1717

1818
1. Determine which milestone(s) are active
1919
2. Resolve the correct `--base` and `--head` refs
20-
3. Run `release-notes-gen generate changes`
20+
3. Run `release-notes generate changes`
2121
4. Write `changes.json` into the correct `release-notes/` folder
2222

2323
## When to use
@@ -90,7 +90,7 @@ For each active milestone:
9090
### 4. Generate the file
9191

9292
```bash
93-
release-notes-gen generate changes <vmr-clone-path> \
93+
release-notes generate changes <vmr-clone-path> \
9494
--base <previous-release-tag> \
9595
--head <current-release-ref> \
9696
--version "<release-version>" \
@@ -103,15 +103,15 @@ Examples:
103103

104104
```bash
105105
# Preview milestone
106-
release-notes-gen generate changes ~/git/dotnet \
106+
release-notes generate changes ~/git/dotnet \
107107
--base v11.0.0-preview.3.26210.100 \
108108
--head origin/release/11.0.1xx-preview4 \
109109
--version "11.0.0-preview.4" \
110110
--labels \
111111
--output release-notes/11.0/preview/preview4/changes.json
112112

113113
# GA/patch milestone
114-
release-notes-gen generate changes ~/git/dotnet \
114+
release-notes generate changes ~/git/dotnet \
115115
--base v10.0.7 \
116116
--head v10.0.8 \
117117
--version "10.0.8" \

.github/skills/generate-features/SKILL.md

Lines changed: 50 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,8 @@ The only additions are **optional enrichment fields** on change entries, such as
3737
- `score_reason` (`string`) — short explanation of the score
3838
- `score_breakdown` (`object`) — optional per-dimension scoring details
3939
- `breaking_changes` (`bool`) — mark changes that users may need to react to, even when they are not headline items
40+
- `reverted_by` (`array<string>`) — optional PR URLs or refs that later backed out this change
41+
- `reverts` (`array<string>`) — optional change IDs or PR URLs that this entry reverts or partially reverts
4042

4143
This schema is intentionally loose and can grow as the workflow learns what it needs.
4244

@@ -65,7 +67,36 @@ Apply the **80/20 rule** from that document: prefer features that make sense to
6567

6668
Do **not** invent features from memory, roadmaps, or PR titles found elsewhere. Everything in `features.json` must trace back to `changes.json`.
6769

68-
### 2. Score what shipped
70+
### 2. Resolve revert state before scoring
71+
72+
Do a quick mechanical revert pass before assigning any positive scores. This step
73+
exists because the original PR may appear in the current `changes.json` while the
74+
revert lands in a later preview and therefore does **not** appear in the same file.
75+
76+
Start with the obvious cases:
77+
78+
```bash
79+
jq -r '.changes[] | select(.title | test("(?i)^(partial(ly)?\\s+)?revert\\b|\\bback out\\b")) | [.repo, .title, .url] | @tsv' changes.json
80+
```
81+
82+
Then, for each candidate you might promote into the draft, search the source repo
83+
for later merged PRs that explicitly revert or back out that PR number or URL:
84+
85+
```bash
86+
gh search prs --repo dotnet/<repo> --state merged \
87+
"\"This reverts https://github.com/dotnet/<repo>/pull/<number>\" OR \"revert <number>\" OR \"back out <number>\"" \
88+
--json number,title,mergedAt,url
89+
```
90+
91+
When a revert is found:
92+
93+
- annotate the original entry with `reverted_by`
94+
- annotate the revert entry with `reverts`, when both are present in the file
95+
- set the original score to `0` unless you can verify the shipped build still contains the feature
96+
97+
Do **not** promote an item to section-worthy status until this pass is complete.
98+
99+
### 3. Score what shipped
69100

70101
Look for signals such as:
71102

@@ -85,14 +116,30 @@ Down-rank or exclude:
85116
- infra and dependency churn
86117
- test-only changes
87118
- internal refactors with no user-facing impact
119+
- items that violate the product-boundary rule in `editorial-rules.md`
88120
- reverts and partial work that did not survive into the build
89121
- items that mostly require insider knowledge to understand why they matter
90122

91-
### 3. Use API evidence to refine the score
123+
### 4. Use API evidence to refine the score
92124

93125
If a change depends on public APIs, use `api-diff` / `dotnet-inspect` to confirm the API exists in the actual build. Missing or reverted APIs should be scored down or excluded.
94126

95-
### 4. Write `features.json`
127+
### 5. Merge incrementally when `features.json` already exists
128+
129+
If the milestone branch already has a `features.json`, do **not** throw that
130+
work away just because `changes.json` was regenerated. For the full rerun
131+
workflow, follow [`update-existing-branch`](../update-existing-branch/SKILL.md).
132+
133+
Use the existing file as the editorial baseline:
134+
135+
- preserve prior scores and annotations for unchanged entries
136+
- score only the newly added or materially changed IDs
137+
- revisit older scores only when new evidence or review feedback justifies it
138+
- keep useful human edits instead of flattening them back to a machine pass
139+
140+
This should feel like a **merge** operation, not a full restart.
141+
142+
### 6. Write `features.json`
96143

97144
The output typically lives next to `changes.json`:
98145

.github/skills/release-notes/SKILL.md

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
name: release-notes
3-
description: Generate and maintain .NET release notes from `features.json`. Uses `generate-changes` for authoritative shipped-change data, `generate-features` for scoring/triage, `editorial-scoring` for the shared rubric, `api-diff`/`dotnet-inspect` for API verification, and a multi-model `review-release-notes` pass for final editorial QA.
4-
compatibility: Requires GitHub MCP server or gh CLI for cross-repo queries. Pairs with the generate-changes, generate-features, editorial-scoring, api-diff, and review-release-notes skills. Claude Opus 4.6 is the default workflow model; the preferred final reviewer pair is Claude Opus 4.6 + GPT-5.4 for broader editorial feedback.
3+
description: Generate and maintain .NET release notes from `features.json`. Uses `generate-changes` for authoritative shipped-change data, `generate-features` for scoring/triage, `update-existing-branch` for incremental reruns on populated branches, `editorial-scoring` for the shared rubric, `api-diff`/`dotnet-inspect` for API verification, and a multi-model `review-release-notes` pass for final editorial QA.
4+
compatibility: Requires GitHub MCP server or gh CLI for cross-repo queries. Pairs with the generate-changes, generate-features, update-existing-branch, editorial-scoring, api-diff, and review-release-notes skills. Claude Opus 4.6 is the default workflow model; the preferred final reviewer pair is Claude Opus 4.6 + GPT-5.4 for broader editorial feedback.
55
---
66

77
# .NET Release Notes
@@ -13,16 +13,26 @@ This skill is the **editorial writing stage** of the pipeline. It turns a scored
1313
## How it works
1414

1515
1. `generate-changes` diffs `source-manifest.json` between VMR refs to produce `changes.json`
16-
2. `generate-features` reads `changes.json` and emits `features.json` with optional scores using the shared `editorial-scoring` rubric
17-
3. `api-diff` / `dotnet-inspect` verifies public APIs and catches missed reverts
18-
4. `release-notes` writes curated markdown using the higher-value entries from `features.json`
19-
5. `review-release-notes` runs a final multi-model editorial QA pass against the scoring rubric and examples
20-
6. Output is one PR per release milestone in dotnet/core, maintained incrementally
16+
2. `generate-features` reads `changes.json`, resolves revert/backout relationships, and emits `features.json` with optional scores using the shared `editorial-scoring` rubric
17+
3. `update-existing-branch` handles incremental reruns when a milestone branch already exists, merging deltas instead of restarting from scratch
18+
4. `api-diff` / `dotnet-inspect` verifies public APIs and confirms suspect features still exist in the shipped build
19+
5. `release-notes` writes curated markdown using the higher-value entries from `features.json`
20+
6. `review-release-notes` runs a final multi-model editorial QA pass against the scoring rubric and examples
21+
7. Output is one PR per release milestone in dotnet/core, maintained incrementally
22+
23+
## Existing-branch reruns
24+
25+
When the milestone branch already exists and contains drafted markdown, invoke
26+
[`update-existing-branch`](../update-existing-branch/SKILL.md). That shared
27+
skill is the canonical playbook for refreshing `changes.json`, merging the delta
28+
into `features.json`, integrating new material into existing sections, and
29+
handling review comments without clobbering human edits.
2130

2231
## Reference documents
2332

2433
- [quality-bar.md](references/quality-bar.md) — what good release notes look like
2534
- [vmr-structure.md](references/vmr-structure.md) — VMR branches, tags, source-manifest.json
35+
- [../update-existing-branch/SKILL.md](../update-existing-branch/SKILL.md) — how to refresh a populated milestone branch incrementally
2636
- [changes-schema.md](references/changes-schema.md) — the shared `changes.json` / `features.json` schema
2737
- [../editorial-scoring/SKILL.md](../editorial-scoring/SKILL.md) — the reusable scoring rubric and cut guidance
2838
- [feature-scoring.md](references/feature-scoring.md) — how to score and cut features

.github/skills/release-notes/references/api-verification.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,10 @@ When writing release notes for Preview N, the locally installed SDK is often Pre
3030

3131
### Step 1: Generate build metadata
3232

33-
`release-notes-gen generate build-metadata` reads the VMR's version info and queries the nightly NuGet feed to find the correct package versions:
33+
`release-notes generate build-metadata` reads the VMR's version info and queries the nightly NuGet feed to find the correct package versions:
3434

3535
```bash
36-
release-notes-gen generate build-metadata ~/git/dotnet \
36+
release-notes generate build-metadata ~/git/dotnet \
3737
--base v11.0.0-preview.2.26159.112 \
3838
--head origin/release/11.0.1xx-preview3 \
3939
--output build-metadata.json
@@ -122,7 +122,7 @@ If `dotnet-inspect` can't find a type:
122122

123123
1. **Check your package version first** — are you querying a Preview N-1 package while writing Preview N notes? Find the correct version (see above).
124124
2. **Named differently** — search with a broader pattern (`find "*Zstd*"` instead of the exact name).
125-
3. **Reverted**check `changes.json` for a revert PR. If the API was added and then reverted in the same preview, it didn't ship. Don't document it.
125+
3. **Reverted or backed out**first scan `changes.json` for revert titles, then search GitHub for a later merged PR that references reverting the original PR number or URL. The revert may live in the next preview and therefore be absent from the current `changes.json`. If the API or behavior is gone in the final build, don't document the original feature.
126126
4. **Internal** — the type may not be public. Don't document internal types in release notes.
127127
5. **Read the PR tests** — the PR's test files are ground truth. Tests compile and run against the actual API surface. Derive code samples from test assertions rather than guessing type names.
128128

.github/skills/release-notes/references/changes-schema.md

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Shared Schema for `changes.json` and `features.json`
22

3-
Reference for the `changes.json` file produced by `release-notes-gen generate changes`, and for the derived `features.json` file used by downstream editorial skills. One file per release milestone.
3+
Reference for the `changes.json` file produced by `release-notes generate changes`, and for the derived `features.json` file used by downstream editorial skills. One file per release milestone.
44

55
## Overview
66

@@ -83,6 +83,8 @@ writing stage to keep long-running features consistent across the release.
8383
- `score_reason` (`string`) — optional short explanation for the score
8484
- `score_breakdown` (`object`) — optional structured scoring details
8585
- `breaking_changes` (`bool`) — optional flag for changes users may need to react to, even if they are not headline features
86+
- `reverted_by` (`array<string>`) — optional list of PR URLs or refs that later reverted or backed out this change
87+
- `reverts` (`array<string>`) — optional list of change IDs or PR URLs this entry reverts or partially reverts
8688

8789
The `product` field is derived from the repo-level [component mapping](component-mapping.md). Infra repos like `arcade` and `symreader` have no `product` field. The `repo` field always matches the VMR manifest path.
8890

@@ -100,7 +102,7 @@ The `commit` field is the VMR codeflow commit in `dotnet/dotnet` that synced thi
100102

101103
## Conventions
102104

103-
- **No nulls** — required fields are always present. Optional fields (`product`, `labels`, `local_repo_commit`, `score`, `score_reason`, `score_breakdown`, `breaking_changes`) may be absent. Use `""` for missing strings, `0` for missing integers.
105+
- **No nulls** — required fields are always present. Optional fields (`product`, `labels`, `local_repo_commit`, `score`, `score_reason`, `score_breakdown`, `breaking_changes`, `reverted_by`, `reverts`) may be absent. Use `""` for missing strings, `0` for missing integers.
104106
- **Naming**`snake_case_lower` for JSON fields, `kebab-case-lower` for file names and repo slugs.
105107
- **Public URLs only** — every URL must resolve publicly.
106108
- **Commit URLs use `.diff` form** — for machine consumption.
@@ -119,6 +121,15 @@ That means a change can be:
119121

120122
In practice, a `score` around `0-4` with `breaking_changes: true` usually means **one line in a "Breaking changes" section**, not a full feature writeup.
121123

124+
## Revert annotations
125+
126+
`changes.json` is the raw shipped-change manifest. During editorial triage,
127+
`features.json` may add `reverted_by` and `reverts` when the agent finds that a
128+
candidate feature was later backed out or partially reverted.
129+
130+
These annotations exist so downstream skills can suppress the original headline
131+
without losing the provenance that explains why the item was cut.
132+
122133
## Example
123134

124135
```json

.github/skills/release-notes/references/component-mapping.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ The `runtime` manifest entry covers both Libraries and Runtime. When writing mar
3636
- **Razor → ASP.NET Core**`dotnet/razor` PRs go in `aspnetcore.md`
3737
- **Templating → SDK**`dotnet/templating` PRs go in `sdk.md`
3838
- **Roslyn** — covers both C# and Visual Basic. Check PR labels/titles to determine language. Produce `csharp.md` (and `visualbasic.md` if VB-specific features exist).
39+
- **Apply the product-boundary rule** — Razor editor code actions, language-server behavior, and other IDE-only experiences are usually tooling stories, not ASP.NET Core product notes. See `editorial-rules.md`.
3940

4041
### Infrastructure components (skip for release notes)
4142

.github/skills/release-notes/references/editorial-rules.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@ Tone, attribution, and content guidelines for .NET release notes.
6464
- **Shared audience filter** — apply the 80/20 rule from `editorial-scoring`; don't redefine a competing threshold here. Keep narrower items only when the broader audience can still see why they matter.
6565
- **The two-sentence test** — if you can only write two sentences about a feature, it's probably an engineering fix, not a feature. Cut it. A community contribution or breaking change can lift a borderline entry, but "fixed an internal bug that happened to be visible" is not a feature.
6666
- **Headlines should convey value** — a heading like "GC regions on macOS" doesn't tell the reader whether this is good or bad. Prefer headings that hint at the benefit: "GC regions enabled on macOS" or "Server GC memory model now available on macOS."
67+
- **Product-boundary rule** — exclude higher-level IDE, editor, or design-time tooling features from product notes unless the release is specifically about that tooling surface. For example, a Razor editor code action should not be presented as an ASP.NET Core feature just because it is adjacent to the web stack.
6768
- **TODO for borderline entries** — when a feature might deserve inclusion but you lack data to justify it (benchmark numbers, real-world impact, user demand), keep the entry but add an HTML `<!-- TODO -->` comment asking for the missing information. This is better than silently including a vague claim or silently cutting something that might matter. The TODO should state what's needed and link to the PR where the data might live.
6869
- **Breaking changes are separate from hype** — a breaking change can be important even when it is not exciting. Keep the score honest; use `breaking_changes: true` to preserve a short callout instead of inflating the item into a headline feature.
6970
- **Clusters can be stronger than the parts** — several related low-score items can justify one section when together they tell a clear story. Keep the individual scores honest, then merge them into one writeup instead of emitting several weak mini-features. Good examples include a group of "Unsafe evolution" changes or multiple runtime entries prefixed with `[browser]`.
@@ -90,7 +91,7 @@ Thank you [@username](https://github.com/username) for this contribution!
9091

9192
### Community contributors section
9293

93-
At the bottom of each component's notes, list ALL external contributors — not just those with documented features. Use the `community-contribution` label to identify them.
94+
At the bottom of each component's notes, list ALL external contributors — not just those with documented features. Any community contributor with **one or more merged PRs** in the milestone should get a mention exactly once in the **Community contributors** section, even if none of their work was promoted into a feature writeup. Use the `community-contribution` label as a strong signal, but do not rely on it exclusively if the merged PR history shows a clear external contribution.
9495

9596
**Vet the list** — the `community-contribution` label is sometimes wrong. Exclude usernames containing `-msft`, `-microsoft`, or other Microsoft suffixes. When in doubt about whether someone is a Microsoft employee, leave them out of the community list.
9697

.github/skills/release-notes/references/feature-scoring.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ matter during .NET release-note triage.
2525
- Test-only or infra-only work
2626
- Mechanical refactors with no user-facing impact
2727
- Churn in dependencies or tooling internals
28+
- **Product-boundary mismatch** — apply the product-boundary rule from `editorial-rules.md`; score these to cut-level unless the release is explicitly about that tooling surface
2829
- Reverts, partial implementations, or APIs that do not appear in the actual build
2930
- Changes too small or obscure to justify external attention
3031
- **Stacked audience gates** — if the reader has to care about A, then B, then be willing to do C, the addressable audience shrinks at each step and the score should usually drop hard

0 commit comments

Comments
 (0)