Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 6 additions & 10 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,20 +204,16 @@ must still contain complete artifacts for the merge run's expected matrix.

The comment is the reuse authorization, so adding it does not trigger or cancel
a PR sweep. Once the comment is present, later commits pushed to a PR with a
full-sweep label do not start another benchmark sweep. GitHub still runs the
CPU-only `check-changelog` job on the new commit before inspecting the reuse
authorization. That job validates the complete YAML/schema, append-only entry
ordering, duplicate YAML keys, byte-for-byte preservation of historical
content, PR links, the generated sweep config, and sweep-label exclusivity.
Link-only correction PRs stop after this CPU check because they have no
benchmark matrix to generate. Only appended entries can continue to the reuse
gate and sweep setup. Removing and re-adding a sweep label explicitly starts a
new sweep.
full-sweep label do not start another benchmark sweep. The synchronize run
checks that the changelog diff can generate the setup matrix before inspecting
the authorization and continuing to setup. This catches malformed conflict
resolutions before reuse can skip setup. Removing and re-adding a sweep label
explicitly starts a new sweep.

`utils/merge_with_reuse.sh <pr-number>` is the supported merge path for reuse.
It merges `main`, preserves the current main changelog bytes, canonicalizes an
appended `XXX` link to the PR URL, pushes a fresh synchronization commit, and
waits for `check-changelog` on that exact SHA before merging.
waits for the PR checks before merging.

On the push-to-main run, `run-sweep.yml` resolves the merged PR from the merge
commit, verifies the source run is an eligible `pull_request` `run-sweep.yml`
Expand Down
49 changes: 20 additions & 29 deletions .github/workflows/run-sweep.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,14 +37,18 @@ jobs:
check-changelog:
runs-on: ubuntu-latest
if: >-
github.event_name != 'pull_request' ||
!github.event.pull_request.draft
outputs:
has-additions: ${{ steps.validate.outputs.has-additions }}
metadata-only: ${{ steps.validate.outputs.metadata-only }}
github.event_name == 'pull_request' &&
!github.event.pull_request.draft &&
(
(github.event.action != 'labeled' && github.event.action != 'unlabeled') ||
github.event.label.name == 'sweep-enabled' ||
github.event.label.name == 'full-sweep-enabled' ||
github.event.label.name == 'non-canary-full-sweep-enabled' ||
github.event.label.name == 'full-sweep-fail-fast' ||
github.event.label.name == 'full-sweep-fail-fast-no-canary'
)
steps:
- name: Reject conflicting sweep labels
if: github.event_name == 'pull_request'
env:
SWEEP_LABELS: ${{ toJson(github.event.pull_request.labels.*.name) }}
run: |
Expand All @@ -71,27 +75,13 @@ jobs:
with:
fetch-depth: 0

- name: Validate perf-changelog.yaml
id: validate
- name: Validate perf-changelog matrix
run: |
pip install pydantic pyyaml

PR_ARGS=()
if [ "${{ github.event_name }}" = "pull_request" ]; then
BASE_REF="origin/${{ github.base_ref }}"
HEAD_REF="${{ github.event.pull_request.head.sha }}"
PR_ARGS=(--pr-number "${{ github.event.pull_request.number }}")
else
BASE_REF="${{ github.event.before }}"
HEAD_REF="${{ github.event.after }}"
fi

python3 utils/validate_perf_changelog.py \
--changelog-file perf-changelog.yaml \
--base-ref "$BASE_REF" \
--head-ref "$HEAD_REF" \
--github-output "$GITHUB_OUTPUT" \
"${PR_ARGS[@]}"
--base-ref "origin/${{ github.base_ref }}" \
--head-ref "${{ github.event.pull_request.head.sha }}"

reuse-sweep-gate:
needs: check-changelog
Expand All @@ -103,7 +93,6 @@ jobs:
if: >-
always() &&
needs.check-changelog.result == 'success' &&
needs.check-changelog.outputs.has-additions == 'true' &&
github.event_name == 'pull_request' &&
github.event.action == 'synchronize' &&
!github.event.pull_request.draft &&
Expand Down Expand Up @@ -138,8 +127,10 @@ jobs:
runs-on: ubuntu-latest
if: >-
always() &&
needs.check-changelog.result == 'success' &&
needs.check-changelog.outputs.has-additions == 'true' &&
(
needs.check-changelog.result == 'success' ||
needs.check-changelog.result == 'skipped'
) &&
Comment thread
Oseltamivir marked this conversation as resolved.
(
needs.reuse-sweep-gate.result == 'skipped' ||
(
Expand Down Expand Up @@ -626,9 +617,9 @@ jobs:

reuse-ingest-artifacts:
needs: setup
# `setup` runs via always() and depends on `check-changelog` plus
# `reuse-sweep-gate`, which is skipped on every push-to-main. Without
# always() here, that skipped gate poisons this job's implicit
# `setup` runs via always() and depends on `check-changelog` and
# `reuse-sweep-gate`, which are skipped on every push-to-main. Without
# always() here, those skipped jobs poison this job's implicit
# success() check and it is skipped even when setup succeeded with
# reuse-enabled=true — silently breaking the merge-time ingest. Guard
# explicitly on setup success instead (same pattern as
Expand Down
33 changes: 0 additions & 33 deletions .github/workflows/test-changelog-gate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,36 +63,3 @@ jobs:
utils/test_validate_reusable_sweep_artifacts.py \
utils/changelog_gate_tests/test_run_sweep_gating.py \
-v

- name: Test historical push validation
run: |
expect_rejected() {
local label="$1"
shift
local output
if output="$("$@" 2>&1)"; then
echo "::error::${label} was unexpectedly accepted"
return 1
fi
echo "${label} was rejected as expected"
printf '%s\n' "$output"
}

python utils/validate_perf_changelog.py \
--changelog-file perf-changelog.yaml \
--base-ref d99c824b1c4f0b1b007631191657e458ef2a332c^ \
--head-ref d99c824b1c4f0b1b007631191657e458ef2a332c

expect_rejected "PR #1767 malformed merge state" \
python utils/validate_perf_changelog.py \
--changelog-file perf-changelog.yaml \
--base-ref 7b9843d3a6e1fe7a2d92d327e25aae57ed3506c5^ \
--head-ref 7b9843d3a6e1fe7a2d92d327e25aae57ed3506c5 \
--pr-number 1767

expect_rejected "PR #1798 invalid merge state" \
python utils/validate_perf_changelog.py \
--changelog-file perf-changelog.yaml \
--base-ref 8aaff9fbb645ae9df9ba7593fa63273f839b62fb^1 \
--head-ref 8aaff9fbb645ae9df9ba7593fa63273f839b62fb \
--pr-number 1798
9 changes: 5 additions & 4 deletions KLAUD_DEBUG.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,11 @@ python3 -c "import yaml; yaml.safe_load(open('perf-changelog.yaml'))"

Do **not** try a 3-way merge of `perf-changelog.yaml` — whitespace edits will silently re-trigger the deletion check.

After committing and pushing the resolution, wait for `check-changelog` on the
new head SHA. `/reuse-sweep-run` skips setup and GPU jobs only after this
CPU-only validation succeeds. `utils/merge_with_reuse.sh <PR>` performs this
wait automatically.
After committing and pushing the resolution, the synchronize run checks the
changelog with the same matrix processor used by setup, then checks the reuse
authorization. This catches deleted history or malformed appended entries
before reuse can skip setup. `utils/merge_with_reuse.sh <PR>` performs the push
and waits for the PR checks automatically.

---

Expand Down
Loading