Skip to content

Commit 647e363

Browse files
authored
Add sample code branch push step to PR verification agent (#137)
* Add sample code branch push step to PR verification agent After verification passes, the agent now pushes the verification sample to a dedicated branch (verification/pr-<N>) for future reference. Changes: - Added Step 5.5: Push Verification Sample to Branch - Updated behavioral rules: clarify no source code changes, add branch isolation rule - Updated success criteria to include branch pushing - Updated Step 7: samples are now pushed, not deleted * Update PR verification agent to push sample code and e2e tests to branch - Add Step 5: Create E2E tests in test/e2e-azuremanaged/ for regression coverage - Add Step 6: Push verification sample + e2e tests to verification/pr-<N> branch - Remove duplicate Step 5.5 (old push logic) and consolidate into Step 6 - Update verification report format to include E2E test details and branch link - Update hard constraints to allow creating verification artifacts and e2e tests - Renumber steps for clean sequential flow (Steps 5-9) * Remove e2e test creation from PR verification agent The agent now only pushes verification samples to branch, without creating e2e tests in test/e2e-azuremanaged/.
1 parent c2e439c commit 647e363

1 file changed

Lines changed: 76 additions & 8 deletions

File tree

.github/agents/pr-verification.agent.md

Lines changed: 76 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
---
1+
---
22
name: pr-verification
33
description: >-
44
Autonomous PR verification agent that finds PRs labeled pending-verification,
@@ -304,7 +304,62 @@ If the verification **fails**, investigate:
304304
- Is the sample correct?
305305
- Retry up to 2 times before reporting failure.
306306

307-
## Step 5: Post Verification to Linked Issue
307+
## Step 5: Push Verification Sample to Branch
308+
309+
After verification passes, push the sample to a dedicated branch so it is
310+
preserved and can be reviewed.
311+
312+
### Branch Creation
313+
314+
Create a branch from the **PR's branch** (not from `main`) named:
315+
```
316+
verification/pr-<pr-number>
317+
```
318+
319+
For example, for PR #123:
320+
```bash
321+
git checkout -b verification/pr-123
322+
```
323+
324+
### Files to Commit
325+
326+
Commit the following file to the branch:
327+
328+
1. **Verification sample** — the standalone script created in Step 3.
329+
Place it at: `examples/verification/pr-<pr-number>-<short-description>.ts`
330+
(e.g., `examples/verification/pr-123-whenall-failfast.ts`)
331+
332+
### Commit and Push
333+
334+
```bash
335+
# Stage the verification sample
336+
git add examples/verification/
337+
338+
# Commit with a descriptive message
339+
git commit -m "chore: add verification sample for PR #<pr-number>
340+
341+
Verification sample: examples/verification/pr-<pr-number>-<description>.ts
342+
343+
Generated by pr-verification-agent"
344+
345+
# Push the branch
346+
git push origin verification/pr-<pr-number>
347+
```
348+
349+
### Branch Naming Rules
350+
351+
- Always use the prefix `verification/pr-`
352+
- Include only the PR number, not the issue number
353+
- Branch names must be lowercase with hyphens
354+
- If the branch already exists on the remote, skip pushing (idempotency)
355+
356+
Check if the branch already exists before pushing:
357+
```bash
358+
git ls-remote --heads origin verification/pr-<pr-number>
359+
```
360+
If it exists, skip the push and note it in the verification report.
361+
362+
## Step 6: Post Verification to Linked Issue
308363

309364
Post a comment on the **linked GitHub issue** (not the PR) with the verification report.
310365

@@ -334,6 +389,10 @@ Post a comment on the **linked GitHub issue** (not the PR) with the verification
334389

335390
</details>
336391

392+
### Sample Code Branch
393+
394+
- **Branch:** `verification/pr-<pr-number>` ([view branch](https://github.com/microsoft/durabletask-js/tree/verification/pr-<pr-number>))
395+
337396
### Results
338397

339398
| Check | Expected | Actual | Status |
@@ -353,14 +412,14 @@ Post a comment on the **linked GitHub issue** (not the PR) with the verification
353412

354413
### Conclusion
355414

356-
<PASS: "All verification checks passed. The fix works as described in the PR.">
415+
<PASS: "All verification checks passed. The fix works as described in the PR. Verification sample pushed to `verification/pr-<pr-number>` branch.">
357416
<FAIL: "Verification failed. See details above. The fix may need additional work.">
358417
```
359418

360419
**Important:** The comment must start with `<!-- pr-verification-agent -->` (HTML comment)
361420
so the idempotency check in Step 1 can detect it.
362421

363-
## Step 6: Update PR Labels
422+
## Step 7: Update PR Labels
364423

365424
After posting the verification comment:
366425

@@ -372,18 +431,26 @@ If verification **failed**, do NOT update labels. Instead:
372431
failed and needs manual review.
373432
2. Leave the `pending-verification` label in place.
374433

375-
## Step 7: Clean Up
434+
## Step 8: Clean Up
376435

377-
- Delete the temporary verification sample file (it's captured in the issue comment).
436+
- Do NOT delete the verification sample — it has been pushed to the
437+
`verification/pr-<number>` branch.
378438
- Do NOT stop the DTS emulator (other tests or agents may be using it).
439+
- Switch back to `main` before processing the next PR:
440+
```bash
441+
git checkout main
442+
```
379443

380444
## Behavioral Rules
381445

382446
### Hard Constraints
383447

384448
- **Idempotent:** Never post duplicate verification comments. Always check first.
385-
- **No code changes:** This agent does NOT modify any source files in the repository.
386-
It only creates temporary sample files, runs them, and posts results.
449+
- **Verification artifacts only:** This agent creates verification samples in
450+
`examples/verification/`. It does NOT modify any existing SDK source files
451+
in the repository.
452+
- **Push to verification branches only:** All artifacts are pushed to
453+
`verification/pr-<number>` branches, never directly to `main` or the PR branch.
387454
- **No PR merges:** This agent does NOT merge or approve PRs. It only verifies.
388455
- **Never modify generated files** (`*_pb.js`, `*_pb.d.ts`, `*_grpc_pb.js`).
389456
- **Never modify CI/CD files** (`.github/workflows/`, `eng/`, `azure-pipelines.yml`).
@@ -419,5 +486,6 @@ A successful run means:
419486
- All `pending-verification` PRs were processed (or correctly skipped)
420487
- Verification samples accurately test the PR's fix scenario
421488
- Evidence is posted to the correct GitHub issue
489+
- Verification samples are pushed to `verification/pr-<N>` branches
422490
- Labels are updated correctly
423491
- Zero duplicate work

0 commit comments

Comments
 (0)