Skip to content

Commit a8e3e62

Browse files
geetikakayrnetser
authored andcommitted
Add support for comment /test-plan for an early on-demand test impact analysis (RedHatQE#5101)
##### What this PR does / why we need it: The verified label is the team's signal that the PR has been code-reviewed and tested by author. At that point the workflow fires CodeRabbit to produce a test execution plan telling the reviewer/author which tests to actually run to validate the change before merging. So the sequence today is: 1. Author opens PR 2. Reviewer reviews the code 3. Author applies /verified - "code looks good and tested" 4. Workflow triggers CodeRabbit to generate the test execution plan 5. Author/reviewer runs the listed tests 6. PR merges 7. The problem identified here is step 4 comes after author does their side of testing. The test impact analysis would be more useful during review (between steps 2 and 3) so the reviewer can factor before signing off. everyone knows risks involved. That's exactly what comment /test-plan solves — anyone can apply it at any point during review to get the plan early. This can be done by anyone reviewing or author without marking PR verified. ##### Which issue(s) this PR fixes: Early access to a report showing the real test impact of their PR. ##### Special notes for reviewer: ##### jira-ticket: None <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added instructions: comment "/test-plan" on a PR to request an inline test execution plan review. * **Chores** * Updated CI workflow to support triggering test-plan requests from either a PR label or an on-demand PR comment. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Geetika Kapoor <gkapoor@redhat.com>
1 parent 6d15e4e commit a8e3e62

2 files changed

Lines changed: 25 additions & 4 deletions

File tree

.github/workflows/request-coderabbit-test-instructions.yml

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
# description: Add comment for CodeRabbit to add tests instructions. A change-request comment will be added to the PR.
2+
# Triggers on the 'verified' label (post-approval) or '/test-plan' comment (on-demand during review).
23

34
name: Request CodeRabbit tests instructions
45

56
on:
67
pull_request_target:
78
types: [labeled]
9+
issue_comment:
10+
types: [created]
811

912
permissions:
1013
pull-requests: write
@@ -13,8 +16,16 @@ permissions:
1316
jobs:
1417
comment-on-commit:
1518
if: |
16-
github.event.label.name == 'verified' &&
17-
!endsWith(github.event.pull_request.user.login, '[bot]')
19+
(
20+
github.event_name == 'pull_request_target' &&
21+
github.event.label.name == 'verified' &&
22+
!endsWith(github.event.pull_request.user.login, '[bot]')
23+
) || (
24+
github.event_name == 'issue_comment' &&
25+
github.event.issue.pull_request &&
26+
startsWith(github.event.comment.body, '/test-plan') &&
27+
!endsWith(github.event.comment.user.login, '[bot]')
28+
)
1829
runs-on: ubuntu-latest
1930

2031
steps:
@@ -23,14 +34,14 @@ jobs:
2334
uses: tspascoal/get-user-teams-membership@v4
2435
with:
2536
team: cnvqe-bot
26-
username: ${{ github.event.pull_request.user.login }}
37+
username: ${{ (github.event.pull_request || github.event.issue).user.login }}
2738
GITHUB_TOKEN: ${{ secrets.BOT3_TOKEN }}
2839

2940
- name: Create or update comment
3041
if: steps.check-user-team.outputs.is-member != 'true'
3142
uses: peter-evans/create-or-update-comment@v5
3243
with:
33-
issue-number: ${{ github.event.pull_request.number }}
44+
issue-number: ${{ github.event.pull_request.number || github.event.issue.number }}
3445
token: ${{ secrets.BOT3_TOKEN }}
3546
body: |
3647
@coderabbitai

docs/DEVELOPER_GUIDE.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,16 @@ It is essential to have a good commit message if you want your change to be revi
112112
- `jira-ticket: https://issues.redhat.com/browse/<jira_id>`
113113
- The card will be automatically closed once PR is merged
114114

115+
### Request a test execution plan
116+
117+
Comment on your GitHub PR:
118+
119+
```bash
120+
/test-plan
121+
```
122+
123+
This triggers CodeRabbit to analyze the PR's changed files and post an inline review comment with a test execution plan (smoke/gating impact, affected tests to run).
124+
115125
### Run the tests via a Jenkins job
116126

117127
#### Build and push a container with your changes

0 commit comments

Comments
 (0)