Skip to content

Commit 6f0e1ed

Browse files
CopilotT-Gro
andauthored
Fix Regression PR Shepherd: restrict push scope and strengthen PR selection (#19558)
* Initial plan * Fix Regression PR Shepherd: add allowed-files restriction and strengthen instructions Add frontmatter guardrails: - allowed-files: restricts push-to-pull-request-branch to tests/**, vsintegration/tests/**, *.fsproj - protected-files: fallback-to-issue for graceful failure Strengthen agentic instructions: - Explicit prohibition on modifying .github/ files (managed by aw-auto-update) - Pre-push verification step: git diff --name-only check before every push - Rebase step includes scope verification to catch unrelated files Agent-Logs-Url: https://github.com/dotnet/fsharp/sessions/b356f0ee-75da-4a9b-9517-8be6278d62ed Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> * Address review: remove *.fsproj from allowed-files, add strict PR eligibility checks - Remove *.fsproj from allowed-files (already covered by tests/** patterns) - Add strict eligibility filters: title must start with "Add regression test:", PR must not be a draft, PR must originate from dotnet/fsharp (no forks) - Add isDraft and headRepository to gh pr list JSON fields - Add guideline: never touch draft PRs or PRs from external forks - Recompile lock file Agent-Logs-Url: https://github.com/dotnet/fsharp/sessions/2049e905-31bd-4a7d-b2be-5e81a1a2b2ee Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com>
1 parent d85a74c commit 6f0e1ed

2 files changed

Lines changed: 38 additions & 22 deletions

File tree

.github/workflows/regression-pr-shepherd.lock.yml

Lines changed: 19 additions & 19 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

.github/workflows/regression-pr-shepherd.md

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,10 @@ safe-outputs:
2828
title-prefix: "Add regression test: "
2929
labels: [AI-Issue-Regression-PR]
3030
max: 10
31+
allowed-files:
32+
- "tests/**"
33+
- "vsintegration/tests/**"
34+
protected-files: fallback-to-issue
3135
remove-labels:
3236
allowed: ["AI-thinks-issue-fixed"]
3337
max: 5
@@ -58,9 +62,17 @@ You shepherd open regression test PRs to completion. These PRs add tests for iss
5862

5963
List all open PRs with the `AI-Issue-Regression-PR` label:
6064
```bash
61-
gh pr list --label "AI-Issue-Regression-PR" --state open --json number,title,headRefName,updatedAt
65+
gh pr list --label "AI-Issue-Regression-PR" --state open --json number,title,headRefName,updatedAt,isDraft,headRepository
6266
```
6367

68+
**Strict eligibility check — apply ALL of these filters before processing a PR:**
69+
1. The PR **must** have the `AI-Issue-Regression-PR` label (enforced by the `gh pr list` filter above).
70+
2. The PR title **must** start with `Add regression test:`. If it does not, skip it — it is not a regression test PR.
71+
3. The PR **must not** be a draft (`isDraft: false`). Draft PRs are work-in-progress and must not be touched.
72+
4. The PR **must** originate from `dotnet/fsharp` — the `headRepository` owner must be `dotnet` and name must be `fsharp`. Never touch PRs from external forks.
73+
74+
**If a PR fails any of these checks, skip it silently — do not comment, do not attempt any changes.**
75+
6476
**Process at most 3 PRs per run.** Prioritize PRs that have unaddressed review feedback (Category A) or CI failures (Category B) over healthy PRs (Category C). If more than 3 PRs need work, the next scheduled run will pick up the rest.
6577

6678
For each PR, determine which category it falls into (check in this order):
@@ -86,8 +98,9 @@ Check the PR's check runs and mergeable state.
8698
1. Check out the PR branch: `gh pr checkout {number}`
8799
2. Rebase onto main: `git fetch origin main && git rebase origin/main`
88100
3. Resolve conflicts — for `.fsproj` conflicts, keep both entries in alphabetical order. For test file conflicts, keep both tests.
89-
4. Build the test project to verify: `dotnet build tests/{TestProject}/{TestProject}.fsproj -c Release`
90-
5. Push the rebased branch to update the PR
101+
4. **Verify scope**: Run `git diff --name-only origin/main` and confirm only files under `tests/` or `vsintegration/tests/` are changed. If unrelated files appear (e.g., `.github/` files from main), reset them: `git checkout origin/main -- .github/`
102+
5. Build the test project to verify: `dotnet build tests/{TestProject}/{TestProject}.fsproj -c Release`
103+
6. Push the rebased branch to update the PR
91104

92105
If any checks failed:
93106

@@ -143,6 +156,9 @@ Track which PRs you've processed and the last review comment timestamp you addre
143156
## Guidelines
144157
145158
- **Never modify files outside `tests/` or `vsintegration/tests/`** — if a fix requires changing `src/`, that's beyond scope. Comment explaining this and tag maintainers.
159+
- **Never modify `.github/` files** — workflow definitions, agent configs, skills, and lock files are managed by the dedicated `aw-auto-update` workflow. Do not touch `.github/workflows/`, `.github/agents/`, `.github/aw/`, or `.github/skills/`.
160+
- **Never touch draft PRs or PRs from external forks** — these are out of scope even if they have the right label. Skip them silently.
161+
- **Verify before pushing**: After making changes or rebasing, run `git diff --name-only` and confirm every changed file is under `tests/` or `vsintegration/tests/`. If unrelated files appear in the diff (e.g., from a rebase picking up main's changes), use `git checkout origin/main -- <file>` to restore them before committing.
146162
- **Only comment on PRs with the `AI-Issue-Regression-PR` label or their linked issues** — never comment on any other issue or PR in the repository. If you need to tag maintainers about a bug that still exists, comment on the linked issue only.
147163
- Begin every comment with: `🤖 *This is an automated response from Regression PR Shepherd.*`
148164
- When fixing review feedback, keep changes minimal — address exactly what was requested.

0 commit comments

Comments
 (0)