Add agentic workflows to remain aligned with Agent Framework releases#7614
Draft
jeffhandley wants to merge 16 commits into
Draft
Add agentic workflows to remain aligned with Agent Framework releases#7614jeffhandley wants to merge 16 commits into
jeffhandley wants to merge 16 commits into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…scans Move ownership of upstream-scan tracking-PR maintenance out of the skill and onto the maintaining workflow: the skill no longer documents a refresh procedure and instead continues past the tracking PR during its preflight. Switch the tracking-state block delimiters from HTML comments to fenced yaml comments (# otel-genai-tracking:begin/end) so GitHub Actions safe-output processing, which strips HTML comments from PR bodies, preserves them. List every open upstream PR in the in-flight applicability table, and clarify that the defer-constant rule applies only to brand-new attributes with no emission site, not to changes to already-emitted conventions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a plain (non-agentic) orchestration workflow whose discovery job resolves the open-telemetry/semantic-conventions-genai ref to scan and emits a single self-contained integration target. The discovery script resolves the requested upstream ref (or the default-branch HEAD) to a commit SHA with retries and hard-fails rather than emitting an empty target. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add the agentic worker and wire the orchestrator's produce job to invoke it as a reusable workflow (workflow_call) for a single discovery target, so it runs in the orchestrator's run context and inherits the orchestrator's actor -- satisfying the worker's activation role check -- and grant it the contents, pull-requests, issues, and discussions writes it requests. A dry_run dispatch input gates the produce job so a manual run can compute and print the target without invoking the worker. A host setup step resolves the target, discovers and classifies the maintained draft PR (ours / blocked / none), reads the recorded scan state, and computes the recommended lifecycle action, writing target.json for the agent to consume. The agent applies the update-otel-genai-conventions skill to produce the integration and maintains the tracking PR through native gh-aw safe outputs (create, push, update, comment, mark-ready, no-op). A post-step validates that every full-body PR write carries the tracking identity before it can publish. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Let a person hand-start an Otel GenAI integration PR that the automation then takes over and maintains. When a PR on the evergreen branch is labeled automation + area-ai but carries no tracking marker yet, classify it as adopt: the worker writes the tracking block into its body on this run and maintains it thereafter, exactly as if it had opened the PR itself. The automation + area-ai labels are the opt-in -- any other PR occupying the branch stays human-owned and blocks the automation. Recompiled meai-otel-genai-worker.lock.yml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extend the worker to address reviewer feedback on the maintained draft PR on the next scheduled run, without triggering on review events. The agent collects the PR's submitted reviews and inline review comments itself through GitHub integrity filtering: tools.github sets min-integrity: approved, and features.integrity-reactions lets a maintainer promote an external contributor's comment to approved with an endorsement reaction. Only write-access and endorsed feedback reaches the agent, so it applies no host-side trust handling of its own. The host setup records a feedback-processed-through watermark and this run's run_started_at in target.json, and computes a body-free wake gate that counts review activity newer than the watermark so a caught-up PR still wakes the agent when new review activity exists. If any of the three activity queries fails, the gate opens automatically, keeping a transient API error from masking real, unprocessed feedback. The agent acts on feedback created after the watermark, folds in-scope feedback into the differential commit(s), rejects out-of-scope requests, resolves contradictions by recency, and advances the watermark to run_started_at. Fetch the maintained PR body with a three-attempt retry and gate the watermark read and wake gate on a successful read, so a transient blip never drops the tracking marker and replays feedback from an empty watermark. If a persistent failure still leaves the PR classified ours with an empty recorded scan ref and watermark, the agent re-reads the body itself once to recover them before scoping feedback or computing the differential, and if that read also fails treats the watermark as this run's start rather than reprocessing every earlier comment. Strip trailing inline # comments when reading the tracking block's feedback-processed-through, upstream-scan-ref, and upstream-release values. Require every iterative update to the maintained PR -- the incremental, feedback-only, and release mark-ready paths -- to fully regenerate the PR description via update-pull-request so it always reflects the current integrated state, including a feedback-only run that only advances the watermark. The behind non-draft advisory path is the exception: it only comments, so a PR a human has marked Ready for review keeps its body. Scope tracking-block extraction to the machine-managed block so human prose in the PR body cannot shadow the recorded state: read every field through tracking_block(), which emits only the last `# meai-otel-genai-worker:state:begin` .. `:state:end` range (the skill writes the live block at the very bottom, so the body ends with it). Otherwise a "> upstream-scan-ref:" line, a future "> feedback-processed-through:", or an older block a maintainer pastes above the live one would win tracking_value's first match and could force a false caught-up no-op or silently suppress real reviewer feedback. Apply the same within-block scoping in the publish guardrail so the guard and the extractor stay symmetric. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…7606) Add the `virtual` modifier to all test methods (marked with [Fact], [Theory], etc.) in Microsoft.Extensions.VectorData.ConformanceTests that were missing it, so derived types can customize behavior. Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Add a repository skill that keeps the aiagent-webapi project template's Microsoft.Agents.AI package versions aligned with the newest coherent Agent Framework release on the dotnet-public feed. Includes SKILL.md plus references for version detection, change classification, build/CI validation, snapshot + execution testing, and pull-request formatting. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add the orchestrator workflow that resolves the newest coherent Agent Framework release on the dotnet-public feed and dispatches the worker per release. Release detection is a file-based C# app (agent-framework-discover.cs) using the NuGet client SDK: it reads the feed, orders versions with NuGetVersion, and emits the whole Microsoft.Agents.AI* family's per-package at_release versions as JSON. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add the Agent Framework Template Worker: a per-release agentic workflow that maintains a single draft PR bumping the Microsoft.Agents.AI packages in eng/packages/ProjectTemplates.props (in lockstep, each at its own tier) and the aligned Major/Minor/Patch of the Microsoft.Agents.AI.ProjectTemplates package. A host setup script (agent-framework-worker-setup.sh) resolves the target, computes the bump, and CI-validates it by restoring, building, and packing the template package through the repo's Arcade build and running the snapshot + execution tests. The worker opens a fresh PR or appends an incremental update to the maintained draft. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… draft gates Add an "adopt" classification: an automation+area-ai-templates draft PR on the maintained branch with no tracking marker yet is taken over and given the full tracking block, becoming an ours PR. A PR that fails a takeover gate (missing labels, or a labeled non-draft a human is finalizing) is classified "blocked" and the worker stands down with a noop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add an author-agnostic wake gate: the setup script detects review comments and reviews created after the feedback-processed-through watermark, so a caught-up PR with new approved feedback re-produces instead of no-oping. Add the feedback update path (Step 3c) that applies in-scope requests, advances the watermark, and comments a summary. Reviewer trust is enforced by the framework's min-integrity gate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… current Augment the worker to review what changed in Agent Framework between the previously integrated version and the target release, not just bump versions. The setup script gathers the microsoft/agent-framework dotnet-* release notes for the range into af-changes.md; the agent (Step 3e) evaluates the template and other Microsoft.Extensions.AI* consumption against those changes, updates it to the currently prescribed patterns within the allowed files, and regenerates snapshots as needed. Adds the evaluate-changes skill reference and widens allowed-files to the Microsoft.Extensions.AI* libraries and the template test tree. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1500663&view=codecoverage-tab |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR is stacked behind #7611 and should merge after it. It reuses the shared
agentic-workflow infrastructure introduced there -- the Copilot PAT pool
(
copilot-pat-poolenvironment), the gh-aw scaffolding under.github/, and theshared components -- so the intended diff here is only the new Agent Framework
automation. Until #7611 lands on
main, this PR also shows its content.Adds an orchestrator + worker pair of agentic workflows, plus a supporting skill, that
keep the
aiagent-webapiproject template aligned with the latest Microsoft AgentFramework release and continuously validate the update. The design mirrors the
OpenTelemetry GenAI maintenance workflows from #7611, applied to a different change
surface.
How it works
.github/scripts/agent-framework-discover.csis a file-based C# app (NuGet client SDK) that reads the dnceng
dotnet-publicfeedand resolves the newest coherent release across the whole
Microsoft.Agents.AI*family, emitting each package's version at that release. The signal is driven purely
off the feed.
agent-framework-orchestrator.yml) runs on a daily schedule,discovers the target release, and dispatches the worker for it as a reusable workflow
(
workflow_call) so the worker runs in the orchestrator's context.agent-framework-worker.md) maintains a single draft PR that:Microsoft.Agents.AI*package pinned ineng/packages/ProjectTemplates.propsin lockstep -- each at its own stabilizationtier (stable /
-preview/-alpha) -- and aligns theMicrosoft.Agents.AI.ProjectTemplatespackage version (Major/Minor/Patch to therelease, prerelease label unchanged);
template package) and the template's snapshot + execution tests, and never publishes
an unvalidated bump;
template and any other
Microsoft.Extensions.AI*consumption to current patternswhen needed.
PR lifecycle
The worker owns one evergreen draft PR (
update-agent-framework-template, labeledautomation+area-ai-templates) and handles:the framework's
min-integrity: approvedscreening;The automation always leaves the PR a draft -- a human reviews and merges.
Authority
The repository skill
update-agent-framework-template(.github/skills/) is theauthority for how to detect the release, map it onto the template's packages, validate
the bump, evaluate cross-release changes, and format the PR. The workflows own
lifecycle and idempotency.
Files
.github/skills/update-agent-framework-template/-- skill + references.github/workflows/agent-framework-orchestrator.yml.github/workflows/agent-framework-worker.md(+ compiled.lock.yml).github/scripts/agent-framework-discover.cs.github/scripts/agent-framework-worker-setup.shMicrosoft Reviewers: Open in CodeFlow