ci: reflect trigger reason in pipeline run name and warn on ADO YAML changes#62
Open
saurabhrb wants to merge 3 commits into
Open
ci: reflect trigger reason in pipeline run name and warn on ADO YAML changes#62saurabhrb wants to merge 3 commits into
saurabhrb wants to merge 3 commits into
Conversation
…changes - Drop the `label` parameter (default 'plugin-main' was leaking into auto-triggered run names) and use Build.Reason + System.PullRequest.PullRequestNumber so PR runs show e.g. `PullRequest61` and CI runs show `IndividualCI`. Manual queues will show `Manual` (run owner can still describe via the existing mode/testFile params). - Add a PR warning workflow modeled on PowerPlatform-DataverseClient-Python that fires when `.azdo/DataversePluginEvals_PR.yml` changes. Surfaces a workflow summary and a resolvable PR review comment so the reviewer remembers to re-enable / approve the pipeline YAML in ADO (definition 32010) after merge.
Reverts the no-op newline appended to dv-overview/SKILL.md in commit c0b490c (which was only there to force the skill-evals CI to fire on the eval pipeline PR).
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Warning
ADO PR pipeline YAML change detected
This PR modifies .azdo/DataversePluginEvals_PR.yml. After merge, Azure DevOps may disable or require approval for the PR validation pipeline.
Action required (post-merge): Re-enable / approve the updated YAML for:
- DVSkillsPlugin-Evals-PR (definitionId=32010)
- https://dev.azure.com/dynamicscrm/OneCRM/_build?definitionId=32010
Please resolve this comment after completing the post-merge steps.
Workflow now checks PR review threads for an unresolved comment carrying the hidden marker <!-- ado-pipeline-yaml-pr-warning:DataversePluginEvals_PR --> and skips posting if one already exists. If the prior comment was resolved (or none exists), a new one is posted. Prevents the warning from re-appearing on every commit push during an in-progress PR.
arorashivam96
approved these changes
May 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two CI-side improvements to the eval pipeline plumbing — no skill or plugin changes.
Pipeline run name (
.azdo/DataversePluginEvals_PR.yml)PR run
buildId=20097004still rendered as... • plugin-main • ...because the YAML used alabelparameter (default: 'plugin-main') in the run name format. Parameter defaults are baked at queue time and auto-triggered (pr/ci) runs can't override them — only "Run pipeline" manual queues can — so every auto run leaks the default label.Fix: drop the
labelparameter and use Azure Pipelines runtime variables instead:Resulting run names:
20260516.1 • dry-run • PullRequest61 • dv_data.biceval.jsonmain→20260516.1 • dry-run • IndividualCI • dv_data.biceval.json20260516.1 • dry-run • Manual • dv_data.biceval.jsonThe
modeandtestFileparameters stay — those are user-facing knobs for manual runs.ADO YAML change warning workflow (
.github/workflows/ado-pipeline-yaml-pr-warning.yml)Modeled on the equivalent workflow in
microsoft/PowerPlatform-DataverseClient-Python. Triggers on PRs that touch.azdo/DataversePluginEvals_PR.ymland:::warning::line in the workflow log32010) post-mergeCloses the manual recall step that bit us when the pipeline default branch was migrated to
main.Not in scope
python .github/evals/static_checks.py) passes