[Spec] TypeSpec to SDK Release Workflow - Updated Design#16176
[Spec] TypeSpec to SDK Release Workflow - Updated Design#16176samvaity wants to merge 5 commits into
Conversation
github-actions
Bot
added
azsdk-cli
There was a problem hiding this comment.
Pull request overview
This PR rewrites typespec-to-sdk-release-workflow.spec.md to reflect the current end-to-end TypeSpec→SDK release workflow design, including a 6-stage model, current tool/agent responsibilities, and a consolidated set of gaps/exceptions/questions.
Changes:
- Replaced the previous long-form “design proposal / prompts / resumable scenarios” content with a concise 6-stage workflow description (tools + owners + gaps per stage).
- Documented the current orchestration approach (“skill chaining” via
NextSteps) and outlined specific orchestration gaps and improvement areas. - Updated exceptions/limitations and open questions to reflect current governance (architect board review, breaking change review, namespace approval gating).
samvaity
force-pushed
the
samvaity/typespec-to-sdk-workflow-spec-update
branch
2 times, most recently
from
46bcd0e to
66bf8dc
Compare
haolingdong-msft
left a comment
haolingdong-msft
left a comment
There was a problem hiding this comment.
Thanks @samvaity for the refinement on the document! I like the new structure. Overall looks good to me! Just leave some comments for information sharing and clarification purpose.
Rewrites the workflow spec to document the current 6-stage TypeSpec to SDK release workflow, covering tooling, gaps, and architecture. Key changes: - Document all 6 stages with tool tables and identified gaps - Add architecture section describing current prompt chaining pattern - Add orchestration improvement gaps (state detection, NextSteps coverage) - Document auto SDK PR repair (custom code drift fix via Copilot agent) - Update exceptions with automated namespace approval and arch board review - Remove outdated release planner references - Consolidate .NET team integration points Addresses: #15363 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
samvaity
force-pushed
the
samvaity/typespec-to-sdk-workflow-spec-update
branch
from
66bf8dc to
7f52195
Compare
- Entry points: spec PR merge triggers auto release plan creation (two-stage pipeline) - Namespace approval step added after TypeSpec readiness (new packages only) - Pipeline generation: show two-stage pipeline (1: release plan, 2: SDK gen per lang) with clear PASS/FAIL branches - Step 6: clarify local gen only (pipeline auto-links PRs to release plan) - Stage 3 text: detail azsdk_run_generate_sdk / SpecWorkflowTool trigger - Open Questions: add Q6 about auto-release after SDK PR merge Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
samvaity
force-pushed
the
samvaity/typespec-to-sdk-workflow-spec-update
branch
from
58a7a62 to
9312a5e
Compare
samvaity
force-pushed
the
samvaity/typespec-to-sdk-workflow-spec-update
branch
from
d1a5fdc to
c9a61ef
Compare
- Expand Stage 5 release steps with manual vs automated detail - Restructure spec per review discussion (participants, labels, gaps) - Add quick-start sections, simplify labels, move flowchart to appendix - Add caveats for silent failures and undefined resume mechanism - Linearize step numbering (1-7) and promote Q6 to Known Gap #13 - Add high-level sequence diagram, stage overview, ARM vs data plane note Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
samvaity
force-pushed
the
samvaity/typespec-to-sdk-workflow-spec-update
branch
from
ec31d59 to
f65db95
Compare
| 3. **Wait for approvals** — Namespace approval (if new package), ARM review (if ARM spec), and API review must pass before merge | ||
| 4. **Spec PR merges → SDK generation is automatic** — Release plan is created, SDKs are generated, and SDK PRs are opened in each language repo automatically ⚠️ *Caveat: generation failures currently fail silently — see [Known Gap #4](#known-gaps-and-next-steps)* | ||
| 5. **Address SDK PR feedback** — If API review has suggestions or CI fails, resolve them (auto-repair handles custom code drift) | ||
| 6. **Release** — Once SDK PRs merge, trigger release (or auto-release if configured). Packages publish, release plan completes. |
There was a problem hiding this comment.
Additional steps, get APIView approval. While we want to revisit this, this will stay for enough time that it should be mentionned.
Today, all data-plane and C# ARM requires explicit APIView approval
There was a problem hiding this comment.
Added APIView approval as explicit step in service team quick-start and human gates callout.
| 3. **Wait for approvals** — Namespace approval (if new package), ARM review (if ARM spec), and API review must pass before merge | ||
| 4. **Spec PR merges → SDK generation is automatic** — Release plan is created, SDKs are generated, and SDK PRs are opened in each language repo automatically ⚠️ *Caveat: generation failures currently fail silently — see [Known Gap #4](#known-gaps-and-next-steps)* | ||
| 5. **Address SDK PR feedback** — If API review has suggestions or CI fails, resolve them (auto-repair handles custom code drift) | ||
| 6. **Release** — Once SDK PRs merge, trigger release (or auto-release if configured). Packages publish, release plan completes. |
There was a problem hiding this comment.
Today, the release pipeline has an approval gate as well. It seems we won't be able to waive it for security reasons
There was a problem hiding this comment.
Noted release pipeline approval gate throughout. Can't remove for security; ARM approval = Shanghai team.
|
|
||
| - **<a id="service-kpi"></a>Service KPI**: Key Performance Indicator in Service Tree for cloud life cycle and these KPIs are marked as completed when a release plan is completed according to service life cycle. | ||
| ### For Service Teams |
There was a problem hiding this comment.
Reading this, let's list the waiting points in the process where we think a human should be there:
- Namespace approval
- Spec review approval. Could be reduced to suppression review, and auto-approve with good linters
- SDK PR approval. Could be reduced to nothing once we get good at changelog, versionning, etc. For data-plane, this includes possibly an archboard or email validation, and APIView approval as well
- Release pipeline approval. Difficult to remove
There was a problem hiding this comment.
Added human gates callout box listing all approval points with notes on which can be reduced over time.
| 3. **Generation failures** — When SDK generation fails, diagnose via pipeline logs (structured error reporting is a [known gap](#known-gaps)) | ||
| 4. **Auto-repair** — `auto-sdk-build-fix` label triggers Copilot cloud agent to fix custom code drift on SDK PRs | ||
| 5. **Release coordination** — `azsdk_release_sdk` checks readiness; release pipelines publish to package registries | ||
| 6. **Track progress** — [Release plan dashboard](https://aka.ms/azsdk/releaseplan-dashboard) shows where each service is in the process |
There was a problem hiding this comment.
Feels relevant for service team as well. It depends how quick you want the quickstart section to be :)
There was a problem hiding this comment.
Moved pipeline monitoring into service team quick-start (step 5).
| **Related Tools**: | ||
| **⚠️ Gaps**: | ||
| 1. Validation steps run independently. No designed chain for: failure ordering, how breaking change results gate SDK generation, per-language generation failure reporting back to spec PR. No unified PR comment summarizing all results with next steps. | ||
| 2. **No service readiness check** — no automated verification that service onboarding prerequisites are met (service registered, team configured, emitters ready) before spec PR can merge. This is currently a manual tribal-knowledge step. |
There was a problem hiding this comment.
Correction: removed the service readiness gap (Gap #2) as it was vague and mostly covered by existing CI validation. The real gap here is endpoint liveness verification (now Gap #11 in the Known Gaps table) — verifying that API endpoints are actually deployed and working before we proceed to SDK generation. @prkannap suggested this during our review discussion. Marked as aspirational since it's a hard problem.
| **🎯 Next steps**: | ||
| - Design unified PR comment that aggregates all validation results with clear next actions per failure type. | ||
| - Investigate adding a service readiness pre-merge check (future feature). | ||
| - Investigate endpoint liveness/testing verification as a pre-merge gate for public specs. |
There was a problem hiding this comment.
Note, this is a really hard problem, I wouldn't work on that just yet. But ok to mention it as inspirational
There was a problem hiding this comment.
Marked as aspirational long-term goal in Stage 1 next steps. Not a near-term priority.
|
|
||
| **Capabilities**: | ||
| **⚠️ Gap**: Steps 7a–7c are manual today. No agent auto-triggers changelog date update or release pipeline after SDK PR merge. User must manually initiate release. Transition from Stage 4 → 5 is not agent-driven. |
There was a problem hiding this comment.
Fixed — Stage 5 gap now describes two phases (readiness + release) instead of referencing individual step numbers.
| - Retrieve APIView comments for SDK packages | ||
| - Use APIView feedback resolver mcp tool | ||
| - Trigger SDK regeneration after TypeSpec changes | ||
| **🎯 Next step**: Investigate auto-release flow (SDK PR merge → changelog date → release trigger) for packages configured for it. |
There was a problem hiding this comment.
We discussed with @raych1 , that there is two steps here:
- SDK PR readiness. Create ChangeLog, fix breaking changes, etc.
- SDK release. Auto-trigger a release pipeline when the SDK PR is merged
We'll start investigating the second one now, as this is a low hanging fruit that speeds up the process already
There was a problem hiding this comment.
Split Stage 5 into two phases: (1) SDK PR readiness (changelog, breaking changes) and (2) SDK release (auto-trigger on merge). Noted starting with phase 2 as low-hanging fruit per discussion with @raych1.
|
|
||
| **Prompt:** | ||
| **Description**: First GA releases require architect board review (human decision-making outside automated workflow). Preview releases do not require review but can request one. |
There was a problem hiding this comment.
Technically first preview requires namespace approval. This could be mentionned here.
There was a problem hiding this comment.
Added: first preview requires namespace approval in the release type table (separate from GA first release).
| 3. **Wait for approvals** — Namespace approval (if new package), ARM review (if ARM spec), and API review must pass before merge | ||
| 4. **Spec PR merges → SDK generation is automatic** — Release plan is created, SDKs are generated, and SDK PRs are opened in each language repo automatically ⚠️ *Caveat: generation failures currently fail silently — see [Known Gap #4](#known-gaps-and-next-steps)* | ||
| 5. **Address SDK PR feedback** — If API review has suggestions or CI fails, resolve them (auto-repair handles custom code drift) | ||
| 6. **Release** — Once SDK PRs merge, trigger release (or auto-release if configured). Packages publish, release plan completes. |
There was a problem hiding this comment.
you may refer this page about the responsibilities for SDK team and service team for mgmt. plane release: https://eng.ms/docs/products/azure-developer-experience/plan/mgmt-sdk-release-process
There was a problem hiding this comment.
Added mgmt plane release link in quick-start and Related Process Documentation table. Thanks!
samvaity
force-pushed
the
samvaity/typespec-to-sdk-workflow-spec-update
branch
3 times, most recently
from
3191bb8 to
2a3365f
Compare
samvaity
force-pushed
the
samvaity/typespec-to-sdk-workflow-spec-update
branch
2 times, most recently
from
0b7b873 to
1d5bad5
Compare
| 2. **Open a spec PR** — Push to `azure-rest-api-specs` and open a PR. CI validates automatically. | ||
| 3. 🧑💻 **Wait for approvals** — The following require human sign-off before spec PR can merge: | ||
| - **Namespace approval** (first preview only) — Architect approves new package namespaces | ||
| - **Spec review** — ARM review (ARM specs) + API review via APIView (all data-plane + C# ARM). Could eventually be reduced to suppression review + auto-approve with better linters. |
There was a problem hiding this comment.
APIView will be replaced with APIReviewHub generated review PRs on Github
| - **Namespace approval** (first preview only) — Architect approves new package namespaces | ||
| - **Spec review** — ARM review (ARM specs) + API review via APIView (all data-plane + C# ARM). Could eventually be reduced to suppression review + auto-approve with better linters. | ||
| 4. **Spec PR merges → SDK generation is automatic** — Release plan is created, SDKs are generated, and SDK PRs are opened in each language repo automatically ⚠️ *Caveat: generation failures currently fail silently — see [Known Gap #4](#known-gaps)* | ||
| 5. 🧑💻 **SDK PR review & approval** — SDK CI runs automatically. If API review has suggestions or CI fails, resolve them (auto-repair handles custom code drift). For management plane, Shanghai team reviews SDK PRs with release plans. APIView approval required for generated SDK API surface — review requests use a [GitHub Form template](https://github.com/Azure/azure-sdk/blob/main/.github/workflows/src/arch-board-review/ARCH-BOARD-REVIEW-PROCESS.md) (replaces email). Could eventually be reduced once changelog/versioning automation matures. |
There was a problem hiding this comment.
API approved is required. This will be via APIView (legacy) or APIReviewHub (GH-based)
|
|
||
| - **<a id="api-spec-pr"></a>API Spec Pull Request**: A pull request in the `azure-rest-api-specs` repository that contains TypeSpec changes defining or updating the service's API. | ||
| 1. **ARM review** (ARM specs only) — Review resource model correctness on spec PRs; apply `ARMSignedOff` label | ||
| 2. **API review** — Review API design via APIView tokens on spec PRs and SDK PRs; apply `api-approved` / `<lang>-api-approved` labels |
There was a problem hiding this comment.
There is no need for these labels, unless they are informational only. API approval is tracked on APIView/APIReviewHub. The labels cannot be authoritative.
There was a problem hiding this comment.
So what actually gates merge?
| - Feedback resolution: Completed | ||
| - **Service team**: Has service requirements and access to `azure-rest-api-specs` repo | ||
| - **ARM review team**: Spec PR has ARM changes + CI passes + `ARM-Review-Required` label present | ||
| - **API review architects**: APIView tokens generated (spec stage) or SDK PR open (SDK stage) |
There was a problem hiding this comment.
With ARH, the review PR must be opened. However that should slot into this. Unlike APIView, ARH PRs will not be created automatically as they are in APIView.
| │ │ │── Compile │ │ │ | ||
| │ │ │── LintDiff │ │ │ | ||
| │ │ │── Breaking chg │ │ │ | ||
| │ │ │── APIView gen │ │ │ |
There was a problem hiding this comment.
Somewhere in here we need to include APIReviewHub. It could be that opening the SDK PR triggers opening of the API review PR
There was a problem hiding this comment.
Is it right to say currently it is NOT auto-created (unlike APIView). So should we specify how is it triggred, Is it SDK PR creation? Manual action?
| - Iterate quickly on TypeSpec changes | ||
| - Generate SDKs locally for testing | ||
| - Transition to full release workflow when ready | ||
| **⚠️ Gap**: SDK breaking change detection integration in progress (being combined into validation check). Auto-repair only handles custom-code drift — not all CI failure types. API review routing relies on manual label application in some cases. API Review Hub is not yet in production — transition from APIView to API Review Hub is in progress. |
There was a problem hiding this comment.
Why does API review rely in any way on labels? Is this true for data plane? Mgmt plane? Both?
| - [ ] All sub-skills integrate seamlessly without context loss between stages | ||
| - [ ] Local and pipeline SDK generation paths both work | ||
| - [ ] Breaking change findings from CI are surfaced clearly to the user | ||
| - [ ] APIView feedback can be resolved within the workflow |
There was a problem hiding this comment.
| - [ ] APIView feedback can be resolved within the workflow | |
| - [ ] API review feedback can be resolved within the workflow |
This way you cover APIView and ARH
| │ • API review via APIView (all specs — reviewer │ | ||
| │ approves API design based on generated tokens) │ |
There was a problem hiding this comment.
With ARH we are unlikely to support TypeSpec since the actual spec PR already is a GH PR. We don't "release" TypeSpecs, so release gating is irrelevant. Similiar to the namespace approvals, the review should occur on the TypeSpec PR and approval is reflected in merging the PR.
| ┌───────────────────────────────────────────────────┐ | ||
| │ • Build → Test → Lint → Package validation │ | ||
| │ • SDK breaking change detection │ | ||
| │ • APIView generated for SDK public API surface │ |
There was a problem hiding this comment.
| │ • APIView generated for SDK public API surface │ | |
| │ • API review generated for SDK public API surface │ |
Covers both APIView and ARH
| │ 7c. Trigger release pipeline │ | ||
| │ • Manual approval gate (user triggers) │ | ||
| │ • OR auto-release if package is configured │ |
There was a problem hiding this comment.
Somewhere in here we should reflect the automated release gates. These currently come from APIView but soon will accept either APIView or ARH and then eventually will accept only ARH
- Add API Review Hub as APIView replacement (synthetic review PRs with API.md diffs, ADO Package Work Items, release-gating) - Split API review participants: ARM vs data-plane vs SDK architects - Reorder participants table by workflow progression - Elaborate labels with automation status (fully/partial/planned) - Add gaps #13-15 from tracking issue #15705 - Align with 3-workstream architecture (GitHub Forms bridge, API Review Hub, spec PR namespace vision) - Add changelog details: mgmt auto-generated, data-plane unreliable - Remove redundant stages input/output/tools table Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
samvaity
force-pushed
the
samvaity/typespec-to-sdk-workflow-spec-update
branch
from
4bc02a7 to
51e9c82
Compare
- Clarify ARH does NOT operate at spec level — TypeSpec spec PR on GitHub IS the API surface review, no separate tool needed - ARH only applies to SDK-level review (Stage 4) - Mark APIView emitter as retiring with ARH at spec level - API approval signaled on SDK PR via label/comment (mechanism TBD) - Add gaps #16-18: ARH review PR creation not automated, feedback resolution agent needs ARH compatibility, release gate transition - Update flowchart: spec-level and SDK-level review clarified - 'APIView feedback' → 'API review feedback' (covers both) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| - [Exceptions and Limitations](#exceptions-and-limitations) | ||
| - [Open Questions](#open-questions) | ||
| - [Appendix: Detailed Flowchart](#appendix-detailed-flowchart) |
There was a problem hiding this comment.
I think we also need a section in here that ties the "no spec change => release" workflow into this. They should have a similar flow and at least have the same requirements/release gates for partners.
The data shows that this is right around the 50% case for both Python and .NET. I'd imagine it is similar for the other languages.
| | Preview (first) | Namespace approval required | Fastest path; namespace approval needed for new packages | | ||
| | Preview (update) | No architect board review (can be requested) | Fastest path | | ||
| | GA (first release) | Architect board review required | Namespace approval needed for new packages | | ||
| | GA (update) | Standard review | Breaking changes require separate approval | |
There was a problem hiding this comment.
What is "standard review" in this context?
| | 9 | Release is a two-phase manual process | 5 | Phase 1 (SDK PR readiness: changelog, breaking changes) and Phase 2 (release trigger) both require manual action | Phase 2 first: auto-trigger release pipeline on SDK PR merge. Phase 1 later: automate changelog + readiness | @raych1 / TBD | | ||
| | 10 | ARM vs data plane process divergence undocumented | 2, 4 | ARM and data plane follow same high-level flow but diverge at review gates; no single doc captures differences | Linked process docs for each plane with clear divergence points | Sameeksha + Praveen | | ||
| | 11 | No endpoint liveness verification before spec PR merge | 2 | For public specs, no check that the API endpoint is tested and live before merge. SDK may be generated for an undeployed API. | Pre-merge gate or documented prerequisite verifying endpoint is tested and live | TBD | | ||
| | 12 | No auto-release after SDK PR merge | 5 | Auto SDK PR creation is actively being built (release plan → SDK generation), but auto-release after merge is undefined. This is the last missing piece for full end-to-end automation (spec merge → release plan → SDK generation → SDK PR → merge → publish). | SDK PR merge triggers changelog date update + release pipeline automatically for configured packages | TBD | |
There was a problem hiding this comment.
This also needs definition for what happens if the API is not approved. We're making the assumption here that it is automatic, and that will not always be the case across all languages.
| 3. Determine SDK generation method preference | ||
| 4. Generate SDKs and create PRs | ||
| 5. Link SDK PRs to release plan | ||
| **Description**: First GA releases require architect board review (human decision-making outside automated workflow). First preview releases require namespace approval for new packages. Subsequent preview releases do not require review but can request one. See updated [API review process](https://github.com/Azure/azure-sdk/blob/main/.github/workflows/src/arch-board-review/ARCH-BOARD-REVIEW-PROCESS.md). |
There was a problem hiding this comment.
This is true for more than just first GA release. All GA releases should expect architect review, even if some languages choose to opt out for some scenarios - the workflow must account for it.
| **Status**: Three workstreams converging on this: | ||
| 1. **GitHub Forms + Actions (PR #10037, shipped)** — Service teams submit review requests via GitHub Form in `azure-sdk` repo (replacing email). `arch-board-review.yml` template is an explicit **bridge** until API Review Hub ships. `namespace-review.yml` stays long-term. `approval-close.yml` validates authorized approvers and auto-closes issues. | ||
| 2. **API Review Hub (PR #15789, in progress)** — Replaces APIView with synthetic GitHub PRs in language repos. When this ships, `arch-board-review.yml` can be retired. Namespace approvals remain out of scope for API Review Hub. | ||
| 3. **Spec PR-based namespace approval (longer-term vision)** — Namespace approval moves to the spec PR itself. When spec PR merges, namespace is considered approved. Would eventually retire `namespace-review.yml` too and simplify E2E: one approval gate (spec merge) unlocks everything. |
There was a problem hiding this comment.
We're over-indexing on a spec creation/change triggering. Data shows that this is only the 50% case. We have just as many releases that take place without a spec change.
While the majority of our packages are for REST services, there are a non-trivial amount that have no spec whatsoever - such as the messaging services, companion packages for language ecosystems and such. It's true these are minorities, but the process must account for them as they also need package+namespace approval.
| I've been experimenting with TypeSpec and now I'm ready to publish and release. | ||
| ``` | ||
| - [ ] **Q3**: What is the .NET team's tooling stack and where are the integration points with azsdk-cli? | ||
| - Context: .NET team has developed independent tooling. Need alignment on shared infrastructure. |
There was a problem hiding this comment.
The important call-out here is "complimentary" tooling, and I'd very much appreciate if we are mindful of how we represent this.
We've been very focused on ensuring alignment with the inner loop work and azsk-cli tooling. Nothing should be competing or conflicting. Those areas with cross-language potential have been surfaced for discussion and integration.
| ``` | ||
| ┌─────────────────────────────────┐ |
There was a problem hiding this comment.
I'd recommend telling copilot to move to a mermaid diagram here. It'll be easier to get the formatting consistent.
Summary
Rewrites the TypeSpec-to-SDK release workflow spec to document the current end-to-end design, tooling landscape, and identified gaps. Restructured based on team review discussions to serve as a single authoritative internal reference.
What changed