Skip to content

Commit c97d4fa

Browse files
Oseltamivircquil11dependabot[bot]ppalangaAnkur-singh
authored
For #304: GH comment hook (#306)
* Initial commit, for #304 * Allow testing on own PR * condense workflow * Rename Workflow * Use environments * Changed environment location * Stricter activation * Test replies * Test replies * Use token for comment perm * Forgot validation * feat: performance changelog triggered runs (as opposed to nightly) (#267) [skip-sweep] * add logic for event driven runs new single workflow that runs on merge to main, new perg-changelog.yaml to track performance changes, new logic to parse changelog, removed cron job in full sweep schedulers * testing pt 1 * raise error if yaml diff in perf changelog is not valid * remove unused imports in process_changelog.py * config data key fix * raise error if test-config subprocess fails to run * backfill changelog * backfill changelog pt 2 * backfill changelog pt 3 * backfill changelog pt 4 * backfill changelog pt 5 * backfill changelog pt 6 * add always() condition to upload changelog metadata * backfill changelog pt 7 (test) * backfill changelog pt 8 (revert test) * backfill changelog pt 9 * backfill changelog pt 11 * change if condition for jobs in run sweep workflow * debugging run sweep workflow * debugging run sweep workflow pt 2 * debugging run sweep workflow pt 3 (revert) * debugging run sweep workflow pt 4 * debugging run sweep workflow pt 5 * debugging run sweep workflow pt 6 * debugging run sweep workflow pt 7 * add always() condition to upload changelog metadata (add back, this got removed) * add bmk prefix to results * backfill changelog official * for concurrency group, use more unique sha * chore(deps): bump the github-actions group across 1 directory with 3 updates (#331) Bumps the github-actions group with 3 updates in the / directory: [actions/checkout](https://github.com/actions/checkout), [actions/upload-artifact](https://github.com/actions/upload-artifact) and [actions/download-artifact](https://github.com/actions/download-artifact). Updates `actions/checkout` from 6.0.0 to 6.0.1 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v6...8e8c483) Updates `actions/upload-artifact` from 5.0.0 to 6.0.0 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@330a01c...b7c566a) Updates `actions/download-artifact` from 6.0.0 to 7.0.0 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@018cc2c...37930b1) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 6.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions - dependency-name: actions/upload-artifact dependency-version: 6.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions - dependency-name: actions/download-artifact dependency-version: 7.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: add final newline to original perf-changelog.yaml so that there wont be erroneous negative diff [skip-sweep] (#333) * Update MI355x Deepseek-R1 FP4 SGLang Image to v0.5.6.post1 (#330) * Update amd-master.yaml * Update perf-changelog.yaml * Update dsr1_fp4_mi355x_docker.sh * Update dsr1_fp4_mi355x_docker.sh --------- Co-authored-by: Cameron Quilici <cjquilici@gmail.com> * TOCTOU * Test new env * Ready for merge * Add benchmark script for GPTOSS FP4 B200 TRT-LLM (#256) * Add benchmark script for GPTOSS FP4 B200 TRT-LLM * make changes to perf changelog --------- Co-authored-by: Cameron Quilici <cjquilici@gmail.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Cameron Quilici <cjquilici@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: ppalanga <ppalanga@amd.com> Co-authored-by: Ankur Singh <ankusingh@nvidia.com>
1 parent 156fef3 commit c97d4fa

2 files changed

Lines changed: 174 additions & 3 deletions

File tree

‎.github/workflows/e2e-tests.yml‎

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
name: End-to-End Tests
2-
run-name: e2e Test - ${{ inputs.test-name || github.event.inputs.generate-cli-command }}
2+
run-name: e2e Test - ${{ inputs.test-name || inputs.generate-cli-command || github.event.inputs.generate-cli-command }}
33

44
on:
55
workflow_dispatch:
@@ -12,21 +12,46 @@ on:
1212
description: "Name for this test run"
1313
required: false
1414
type: string
15+
ref:
16+
description: "Ref (branch/sha) to checkout for generating configs"
17+
required: false
18+
type: string
19+
workflow_call:
20+
inputs:
21+
generate-cli-command:
22+
description: "Command passed to generate matrix script"
23+
required: true
24+
type: string
25+
test-name:
26+
description: "Name for this test run"
27+
required: false
28+
type: string
29+
ref:
30+
description: "Ref (branch/sha) to checkout for generating configs"
31+
required: false
32+
type: string
1533

1634
jobs:
1735
get-jobs:
1836
runs-on: ubuntu-latest
1937
outputs:
2038
search-space-config: ${{ steps.get-jobs.outputs.search-space-config }}
2139
steps:
22-
- name: Checkout code
40+
- name: Checkout code (ref)
41+
if: ${{ inputs.ref && inputs.ref != '' }}
42+
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
43+
with:
44+
ref: ${{ inputs.ref }}
45+
46+
- name: Checkout code (default)
47+
if: ${{ !inputs.ref || inputs.ref == '' }}
2348
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
2449

2550
- id: get-jobs
2651
run: |
2752
pip install pydantic
2853
CONFIG_JSON=$(python3 ${GITHUB_WORKSPACE}/utils/matrix_logic/generate_sweep_configs.py \
29-
${{ inputs.generate-cli-command }} \
54+
${{ inputs.generate-cli-command || github.event.inputs.generate-cli-command }} \
3055
--runner-config .github/configs/runners.yaml \
3156
--config-files .github/configs/nvidia-master.yaml .github/configs/amd-master.yaml)
3257
echo "search-space-config=$CONFIG_JSON" >> $GITHUB_OUTPUT

‎.github/workflows/pr-comment-sweep.yml‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
name: Slash Command Sweep
2+
run-name: "Validate PR #${{ github.event.issue.number }}"
3+
4+
on:
5+
issue_comment:
6+
types: [created]
7+
8+
concurrency:
9+
group: "PR#${{ github.event.issue.number || github.ref_name }}"
10+
cancel-in-progress: true
11+
12+
permissions:
13+
contents: read
14+
issues: write
15+
pull-requests: write
16+
17+
jobs:
18+
get-jobs:
19+
# Only run for PR comments that start with /sweep
20+
if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/sweep') }}
21+
runs-on: ubuntu-latest
22+
outputs:
23+
pr-number: ${{ steps.parse.outputs.pr-number }}
24+
generator-args: ${{ steps.parse.outputs.generator-args }}
25+
author-can-bypass: ${{ steps.auth.outputs.can-bypass }}
26+
# Immutable ref (commit SHA) to prevent TOCTOU on refs/pull/<n>/head
27+
ref: ${{ steps.ref_comment.outputs.ref }}
28+
steps:
29+
- name: Parse PR comment (/sweep ...)
30+
id: parse
31+
if: ${{ github.event_name == 'issue_comment' && github.event.issue.pull_request && startsWith(github.event.comment.body, '/sweep') }}
32+
shell: bash
33+
env:
34+
BODY: ${{ github.event.comment.body }}
35+
PR_NUMBER: ${{ github.event.issue.number }}
36+
run: |
37+
set -euo pipefail
38+
# Require /sweep at the start of the line
39+
cmd_line=$(printf "%s" "$BODY" | awk '/^\/sweep/{print; exit}')
40+
if [[ -z "$cmd_line" ]]; then
41+
echo "No /sweep command found at comment start" >&2
42+
exit 1
43+
fi
44+
if [[ "$cmd_line" == "/sweep" ]]; then
45+
cmd_args=""
46+
else
47+
cmd_args=${cmd_line#/sweep}
48+
fi
49+
cmd_args=$(echo "$cmd_args" | xargs || true)
50+
51+
echo "generator-args=$cmd_args" >> "$GITHUB_OUTPUT"
52+
echo "pr-number=$PR_NUMBER" >> "$GITHUB_OUTPUT"
53+
54+
- name: Check author permissions
55+
id: auth
56+
if: ${{ github.event_name == 'issue_comment' && github.event.issue.pull_request }}
57+
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
58+
with:
59+
script: |
60+
const owner = context.repo.owner;
61+
const repo = context.repo.repo;
62+
const username = context.payload.comment?.user?.login;
63+
let permission = 'none';
64+
try {
65+
const res = await github.rest.repos.getCollaboratorPermissionLevel({ owner, repo, username });
66+
permission = res.data?.permission || 'none';
67+
} catch (e) {
68+
permission = 'none';
69+
}
70+
const canBypass = ['admin','maintain','write'].includes(permission);
71+
core.info(`Author ${username} permission: ${permission}; bypass=${canBypass}`);
72+
core.setOutput('can-bypass', canBypass ? 'true' : 'false');
73+
74+
# ---- PR SHA pinning ----
75+
- name: Resolve immutable PR ref (pin to head SHA)
76+
id: ref_comment
77+
if: ${{ github.event_name == 'issue_comment' && github.event.issue.pull_request && startsWith(github.event.comment.body, '/sweep') }}
78+
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
79+
with:
80+
script: |
81+
const owner = context.repo.owner;
82+
const repo = context.repo.repo;
83+
const pr = context.issue.number;
84+
const res = await github.rest.pulls.get({ owner, repo, pull_number: pr });
85+
const sha = res.data.head.sha;
86+
core.info(`Resolved PR #${pr} head SHA: ${sha}`);
87+
core.setOutput('ref', sha);
88+
89+
- name: Reply with run link
90+
if: ${{ github.event_name == 'issue_comment' && startsWith(github.event.comment.body, '/sweep') && github.repository_owner == 'InferenceMAX' }}
91+
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
92+
continue-on-error: true
93+
env:
94+
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
95+
AUTHOR: ${{ github.event.comment.user.login }}
96+
GEN_CMD: ${{ steps.parse.outputs.generator-args }}
97+
CAN_BYPASS: ${{ steps.auth.outputs.can-bypass }}
98+
PINNED_REF: ${{ steps.ref_comment.outputs.ref }}
99+
with:
100+
github-token: ${{ github.token }}
101+
script: |
102+
const owner = context.repo.owner;
103+
const repo = context.repo.repo;
104+
const issue_number = context.issue.number;
105+
const runUrl = process.env.RUN_URL;
106+
const author = process.env.AUTHOR;
107+
const genCmd = process.env.GEN_CMD || '';
108+
const canBypass = (process.env.CAN_BYPASS || '').toLowerCase() === 'true';
109+
const pinned = process.env.PINNED_REF || '';
110+
const shortSha = pinned ? pinned.slice(0, 7) : '';
111+
const approvalMsg = canBypass ? 'Approval: not required (trusted collaborator).' : "Approval: required in environment 'Outside Collaborator E2E Test'.";
112+
const body = `@${author} Kicking off a sweep.\n\nRun: ${runUrl}\nCommand: \`${genCmd}\`\nPinned ref: \`${shortSha}\`\n${approvalMsg}`;
113+
await github.rest.issues.createComment({ owner, repo, issue_number, body });
114+
115+
approval:
116+
needs: get-jobs
117+
if: ${{ github.event_name == 'issue_comment' && needs.get-jobs.outputs.pr-number != '' && needs.get-jobs.outputs.generator-args != '' && needs.get-jobs.outputs.author-can-bypass != 'true' }}
118+
runs-on: ubuntu-latest
119+
name: approval
120+
environment: Outside Collaborator E2E Test
121+
steps:
122+
- run: echo "approved"
123+
124+
validate-trusted:
125+
needs: [get-jobs]
126+
if: ${{ github.event_name == 'issue_comment' && needs.get-jobs.outputs.pr-number != '' && needs.get-jobs.outputs.generator-args != '' && needs.get-jobs.outputs.author-can-bypass == 'true' }}
127+
uses: ./.github/workflows/e2e-tests.yml
128+
name: validate (trusted author)
129+
secrets: inherit
130+
with:
131+
generate-cli-command: ${{ needs.get-jobs.outputs.generator-args }}
132+
test-name: PR #${{ needs.get-jobs.outputs.pr-number }} sweep
133+
# Use pinned SHA to prevent TOCTOU on refs/pull/<n>/head
134+
ref: ${{ needs.get-jobs.outputs.ref }}
135+
136+
validate:
137+
needs: [get-jobs, approval]
138+
if: ${{ github.event_name == 'issue_comment' && needs.get-jobs.outputs.pr-number != '' && needs.get-jobs.outputs.generator-args != '' && needs.get-jobs.outputs.author-can-bypass != 'true' && needs.approval.result == 'success' }}
139+
uses: ./.github/workflows/e2e-tests.yml
140+
name: validate
141+
secrets: inherit
142+
with:
143+
generate-cli-command: ${{ needs.get-jobs.outputs.generator-args }}
144+
test-name: PR #${{ needs.get-jobs.outputs.pr-number }} sweep
145+
# Use pinned SHA to prevent TOCTOU on refs/pull/<n>/head
146+
ref: ${{ needs.get-jobs.outputs.ref }}

0 commit comments

Comments
 (0)