Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .claude/claudebox/backport.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,10 @@ You will receive a prompt like:

Variables you need to extract from the prompt:
- `PR_NUMBER`: the PR number (e.g., `21829`)
- `TARGET_BRANCH`: the release branch (e.g., `v4-next`)
- `STAGING_BRANCH`: always `backport-to-${TARGET_BRANCH}-staging`
- `TARGET_BRANCH`: the target branch (e.g., `v4-next`, or `next` for a `port-to-next` port)
- `STAGING_BRANCH`: `backport-to-${TARGET_BRANCH}-staging` for a backport. For a
`port-to-next` port it is `port-to-next-staging`; the prompt states it explicitly
(`... (staging branch port-to-next-staging) ...`) — use the branch named there.

## Constraints

Expand Down
20 changes: 18 additions & 2 deletions .claude/skills/merge-train-infra/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,29 @@ The merge-train system is fully automated via GitHub Actions in `.github/workflo

1. **PR Creation** (`merge-train-create-pr.yml`): Triggered on push to `merge-train/*` branches. Creates a PR targeting `next` (or `v5-next` for `-v5` trains such as `merge-train/spartan-v5` and `merge-train/fairies-v5`) with the `ci-no-squash` label (plus `private-port-next` for any train that targets `v5-next`, and `ci-full-no-test-cache` for `merge-train/spartan`, `merge-train/spartan-v5`, and `merge-train/ci`). Skips merge commits and commits already in the base branch.

2. **Body Updates** (`merge-train-update-pr-body.yml`): Triggered on push to `merge-train/**` and `backport-to-*-staging` branches. Updates the PR body with meaningful commits (those containing PR references like `(#1234)`). The body wraps the commit list in `BEGIN_COMMIT_OVERRIDE` / `END_COMMIT_OVERRIDE` markers. Backport staging PRs also call `update-pr-body.sh` inline from `scripts/backport_to_staging.sh` to handle the first-push case (where the PR doesn't exist yet when the workflow fires).
2. **Body Updates** (`merge-train-update-pr-body.yml`): Triggered on push to `merge-train/**`, `backport-to-*-staging`, and `port-to-next-staging` branches. Updates the PR body with meaningful commits (those containing PR references like `(#1234)`). The body wraps the commit list in `BEGIN_COMMIT_OVERRIDE` / `END_COMMIT_OVERRIDE` markers. Backport/port staging PRs also call `update-pr-body.sh` inline from `scripts/backport_to_staging.sh` to handle the first-push case (where the PR doesn't exist yet when the workflow fires).

3. **Next Integration** (`merge-train-next-to-branches.yml`): Triggered on push to `next` and `v5-next`. A push to `next` merges `next` into each `next`-based train; a push to `v5-next` merges `v5-next` into the `-v5` trains (`merge-train/spartan-v5`, `merge-train/fairies-v5`). Both go through `scripts/merge-train/merge-next.sh`, which takes an optional second argument for the source branch (defaults to `next`). Uses `continue-on-error: true` so a conflict in one branch does not block others. Skips branches whose PR already has auto-merge enabled.

4. **Auto-Merge** (`merge-train-auto-merge.yml`): Runs hourly via cron (`0 * * * *`). Calls `scripts/merge-train/auto-merge.sh` for both merge-train (4-hour inactivity) and backport-train (8-hour inactivity) branches. Uses separate GitHub tokens: `AZTEC_BOT_GITHUB_TOKEN` for API calls and `MERGE_TRAIN_GITHUB_TOKEN` for approvals. Will not auto-merge if the last merge-queue CI run failed or was cancelled.
4. **Auto-Merge** (`merge-train-auto-merge.yml`): Runs hourly via cron (`0 * * * *`). Calls `scripts/merge-train/auto-merge.sh` for merge-train (4-hour inactivity), backport-train (`BRANCH_PATTERN=backport-to-`, 8-hour), and port-to-next (`BRANCH_PATTERN=port-to-next`, 8-hour) branches. Uses separate GitHub tokens: `AZTEC_BOT_GITHUB_TOKEN` for API calls and `MERGE_TRAIN_GITHUB_TOKEN` for approvals. Will not auto-merge if the last merge-queue CI run failed or was cancelled.

5. **Recreation & Wakeup** (`merge-train-recreate.yml`): Triggered when a PR is closed (merged). If the merged PR's head branch starts with `merge-train/`, recreates the branch from the base branch (usually `next`). Then runs `scripts/merge-train/wakeup-prs.sh` to add the `ci-wakeup-pr-after-merge` label to all open PRs targeting the branch that have passed CI and have automerge enabled. This triggers a CI re-run (typically a no-op via tree-hash cache) so those PRs can proceed through the merge queue. The label is immediately removed by a step in `ci3.yml` so it can be re-applied on subsequent merges.

6. **Failure Notification** (`merge-queue-dequeue-notify.yml`): Triggered when a PR is dequeued from the merge queue. If the PR's head branch starts with `merge-train/` and the PR was NOT merged, sends a Slack notification via `ci3/merge_train_failure_slack_notify`. That script also kicks off a ClaudeBox session to investigate/fix the dequeued PR (`ci3/slack_notify_with_claudebox_kickoff`), passing `--repo "$GITHUB_REPOSITORY"` so the session runs in the mode matching the repo the train lives on. When the train is on a private mirror (`…-private`), `claudebox.yml` selects private mode; otherwise it stays public. Without that repo hint a private-train fix session lands in public mode and cannot read the PR or open the fix.

## Label-Driven Ports (`backport.yml`)

`backport.yml` (triggered on `pull_request_target` labeled/closed) cherry-picks a merged PR onto an accumulating staging branch, then opens/updates one staging PR into a target branch. It handles two label families, both driven by `scripts/backport_to_staging.sh`:

- **`backport-to-<branch>`** (e.g. `backport-to-v5-next`): target is `<branch>` (derived from the label), staging branch `backport-to-<branch>-staging`. Direction `next` → release line.
- **`port-to-next`** (fixed, generic): target is `next`, staging branch `port-to-next-staging`. Direction: forward-port an already-merged PR straight into `next`. The workflow passes `STAGING_BRANCH` / `STAGING_PR_TITLE` / `STAGING_PR_LABELS` env overrides into the script; the staging PR carries `ci-no-squash` (required because `next` enforces squashed PRs). `port-to-next` takes precedence if both label families are present.

On cherry-pick conflict the workflow comments on the PR, posts to `#backports`, and dispatches ClaudeBox (`.claude/claudebox/backport.md`) with the staging branch to resolve manually. Staging PRs are auto-merged by the 8-hour jobs in `merge-train-auto-merge.yml`.

## Scheduled Forward-Port (`port-v5-next-to-next.yml`)

A daily bulk sweep (distinct from the per-PR `port-to-next` label) that keeps `next` fed with everything on the `v5-next` release line. `port-v5-next-to-next.yml` runs at 06:30 UTC (and on `workflow_dispatch`) and calls `scripts/port_to_next.sh <source>` (default `v5-next`). The `port-<source>-to-next` branch is long-lived: each run checks it out and merges both `next` and the source into it, then opens/updates one `ci-no-squash` PR into `next`. Accumulating (rather than rebuilding) means any conflict resolution pushed to the branch is preserved across runs. Once the PR is merged (the branch becomes an ancestor of `next`) the next run rebuilds the branch fresh from `next` with a `--force-with-lease` push; while accumulating it fast-forwards. If a run produces no delta over `next` it closes the stale PR. A merge conflict does **not** abandon the run: the conflicted merge is committed with markers (so the PR is still opened/updated as a resolution target), the script emits `conflicts` / `pr_url` step outputs, and the workflow posts the PR link and conflicted files to `#backports`. Resolve by checking out the port branch, fixing the markers, and pushing. This PR is intentionally left for human review — it is not added to the auto-merge patterns.

## CI Integration Details

### CI Mode Selection (`.github/ci3_labels_to_env.sh`)
Expand Down Expand Up @@ -94,6 +107,8 @@ When a CI run fails on an EC2 instance, it calls `merge_train_failure_slack_noti
| `.github/workflows/merge-train-update-pr-body.yml` | Updates PR body with commit list (merge-train and backport branches) |
| `.github/workflows/merge-queue-dequeue-notify.yml` | Slack notification on merge-queue dequeue |
| `.github/workflows/squashed-pr-check.yml` | Squash enforcement (skipped for `ci-no-squash`) |
| `.github/workflows/backport.yml` | Cherry-picks merged PRs to staging branches for `backport-to-*` and `port-to-next` labels |
| `.github/workflows/port-v5-next-to-next.yml` | Daily forward-port sweep of `v5-next` into `next` |

### Scripts

Expand All @@ -105,6 +120,7 @@ When a CI run fails on an EC2 instance, it calls `merge_train_failure_slack_noti
| `scripts/merge-train/squash-pr.sh` | Squashes PR commits (used by `ci-squash-and-merge` label) |
| `scripts/merge-train/wakeup-prs.sh` | Adds `ci-wakeup-pr-after-merge` label to qualifying PRs after branch recreation |
| `scripts/backport_to_staging.sh` | Cherry-picks a merged PR to a backport staging branch; creates/updates the backport PR |
| `scripts/port_to_next.sh` | Daily forward-port: accumulates `next` + source onto long-lived `port-<source>-to-next`, opens/updates the PR |

### CI Configuration

Expand Down
59 changes: 42 additions & 17 deletions .github/workflows/backport.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Automatic backport action
name: Automatic backport/port action

on:
pull_request_target:
Expand All @@ -15,7 +15,7 @@ jobs:
uses: agilepathway/label-checker@825944377ab3bce1269b38c99b718767e2ca6bbc
with:
prefix_mode: true
any_of: backport-to-
any_of: backport-to-,port-to-next
repo_token: ${{ secrets.AZTEC_BOT_GITHUB_TOKEN }}
allow_failure: true
- name: Print status
Expand All @@ -24,7 +24,7 @@ jobs:

backport:
needs: [label_checker]
name: Backport PR
name: Backport/port PR
if: github.event.pull_request.merged == true && needs.label_checker.outputs.state == 'success'
runs-on: ubuntu-latest
steps:
Expand All @@ -34,30 +34,52 @@ jobs:
fetch-depth: 0
token: ${{ secrets.AZTEC_BOT_GITHUB_TOKEN }}

- name: Extract target branch from labels
- name: Resolve port target from labels
id: extract-branch
env:
LABELS_JSON: ${{ toJson(github.event.pull_request.labels.*.name) }}
run: |
LABELS="$LABELS_JSON"
echo "All labels: $LABELS"

# Extract the branch name from backport-to-* label
TARGET_BRANCH=$(echo "$LABELS" | jq -r '.[] | select(startswith("backport-to-")) | sub("backport-to-"; "")')

if [ -z "$TARGET_BRANCH" ]; then
echo "No backport-to-* label found"
exit 1
# port-to-next is a fixed-target port straight into next. Because next
# enforces squashed PRs, the staging PR carries ci-no-squash. It takes
# precedence over backport-to-* if both are somehow present.
if echo "$LABELS" | jq -e 'index("port-to-next")' >/dev/null; then
TARGET_BRANCH="next"
STAGING_BRANCH="port-to-next-staging"
STAGING_PR_TITLE="chore: Accumulated ports to next"
STAGING_PR_LABELS="ci-no-squash"
VERB="ported"
else
# Extract the branch name from a backport-to-* label
TARGET_BRANCH=$(echo "$LABELS" | jq -r '.[] | select(startswith("backport-to-")) | sub("backport-to-"; "")')
if [ -z "$TARGET_BRANCH" ]; then
echo "No backport-to-* or port-to-next label found"
exit 1
fi
STAGING_BRANCH="backport-to-${TARGET_BRANCH}-staging"
STAGING_PR_TITLE="chore: Accumulated backports to $TARGET_BRANCH"
STAGING_PR_LABELS=""
VERB="backported"
fi

echo "target_branch=$TARGET_BRANCH" >> $GITHUB_OUTPUT
echo "staging_branch=$STAGING_BRANCH" >> $GITHUB_OUTPUT
echo "staging_pr_title=$STAGING_PR_TITLE" >> $GITHUB_OUTPUT
echo "staging_pr_labels=$STAGING_PR_LABELS" >> $GITHUB_OUTPUT
echo "verb=$VERB" >> $GITHUB_OUTPUT
echo "Target branch: $TARGET_BRANCH"
echo "Staging branch: $STAGING_BRANCH"

- name: Run backport script
id: backport
continue-on-error: true
env:
GH_TOKEN: ${{ secrets.AZTEC_BOT_GITHUB_TOKEN }}
STAGING_BRANCH: ${{ steps.extract-branch.outputs.staging_branch }}
STAGING_PR_TITLE: ${{ steps.extract-branch.outputs.staging_pr_title }}
STAGING_PR_LABELS: ${{ steps.extract-branch.outputs.staging_pr_labels }}
run: |
# Kludge. We should write this in Python or not checkout history.
# Bash has an ugly footgun with changing history while running the script.
Expand All @@ -72,13 +94,14 @@ jobs:
GH_TOKEN: ${{ secrets.AZTEC_BOT_GITHUB_TOKEN }}
run: |
TARGET_BRANCH="${{ steps.extract-branch.outputs.target_branch }}"
STAGING_BRANCH="backport-to-${TARGET_BRANCH}-staging"
STAGING_BRANCH="${{ steps.extract-branch.outputs.staging_branch }}"
VERB="${{ steps.extract-branch.outputs.verb }}"
STAGING_PR=$(gh pr list --base "$TARGET_BRANCH" --head "$STAGING_BRANCH" --json number,url --jq '.[0]')
STAGING_PR_NUMBER=$(echo "$STAGING_PR" | jq -r '.number')
STAGING_PR_URL=$(echo "$STAGING_PR" | jq -r '.url')

gh pr comment "${{ github.event.pull_request.number }}" --body \
"✅ Successfully backported to [$STAGING_BRANCH #$STAGING_PR_NUMBER]($STAGING_PR_URL)."
"✅ Successfully $VERB to [$STAGING_BRANCH #$STAGING_PR_NUMBER]($STAGING_PR_URL)."

- name: Comment on original PR (failure)
if: steps.backport.outcome == 'failure'
Expand All @@ -91,7 +114,7 @@ jobs:
gh pr comment "${{ github.event.pull_request.number }}" --body \
"❌ Failed to cherry-pick to \`$TARGET_BRANCH\` due to conflicts. (🤖) [View backport run]($WORKFLOW_URL)."

- name: Notify Slack and dispatch ClaudeBox on backport failure
- name: Notify Slack and dispatch ClaudeBox on failure
if: steps.backport.outcome == 'failure'
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
Expand All @@ -101,11 +124,13 @@ jobs:
URL: ${{ github.event.pull_request.html_url }}
AUTHOR: ${{ github.event.pull_request.user.login }}
BRANCH: ${{ steps.extract-branch.outputs.target_branch }}
STAGING_BRANCH: ${{ steps.extract-branch.outputs.staging_branch }}
VERB: ${{ steps.extract-branch.outputs.verb }}
run: |

# Post single message to #backports, derive permalink from response
TEXT=$(printf '⚠️ Backport failed: <%s|#%s %s> → `%s` (author: %s) (🤖)' \
"$URL" "$PR" "$TITLE" "$BRANCH" "$AUTHOR")
TEXT=$(printf '⚠️ %s failed: <%s|#%s %s> → `%s` (author: %s) (🤖)' \
"${VERB^}" "$URL" "$PR" "$TITLE" "$BRANCH" "$AUTHOR")
RESP=$(curl -sS -X POST https://slack.com/api/chat.postMessage \
-H "Authorization: Bearer $SLACK_BOT_TOKEN" \
-H "Content-type: application/json" \
Expand All @@ -120,8 +145,8 @@ jobs:
fi

gh workflow run claudebox.yml \
-f prompt="Backport PR #$PR ($TITLE) to $BRANCH. The automatic cherry-pick failed due to conflicts. Follow .claude/claudebox/backport.md to resolve conflicts and create a PR." \
-f prompt="${VERB^} PR #$PR ($TITLE) to $BRANCH (staging branch $STAGING_BRANCH). The automatic cherry-pick failed due to conflicts. Follow .claude/claudebox/backport.md to resolve conflicts and create a PR." \
-f link="${LINK:-$URL}" \
-f target_ref="origin/backport-to-${BRANCH}-staging" \
-f target_ref="origin/$STAGING_BRANCH" \
-f slack_channel="$CHANNEL_ID" \
-f slack_thread_ts="$TS"
9 changes: 9 additions & 0 deletions .github/workflows/merge-train-auto-merge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,12 @@ jobs:
MERGE_STRATEGY=merge \
INACTIVITY_HOURS=8 \
./scripts/merge-train/auto-merge.sh

- name: Run auto-merge script (port-to-next)
run: |
GH_TOKEN=${{ secrets.AZTEC_BOT_GITHUB_TOKEN }} \
MERGE_TRAIN_GITHUB_TOKEN=${{ secrets.MERGE_TRAIN_GITHUB_TOKEN }} \
BRANCH_PATTERN=port-to-next \
MERGE_STRATEGY=merge \
INACTIVITY_HOURS=8 \
./scripts/merge-train/auto-merge.sh
1 change: 1 addition & 0 deletions .github/workflows/merge-train-update-pr-body.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ on:
branches:
- 'merge-train/**'
- 'backport-to-*-staging'
- 'port-to-next-staging'

jobs:
update-pr-body:
Expand Down
71 changes: 71 additions & 0 deletions .github/workflows/port-v5-next-to-next.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: Port v5-next to next

# Daily forward-port: accumulate everything new from `v5-next` onto the
# long-lived `port-v5-next-to-next` branch (merging `next` and `v5-next` into
# it) and open/update one large PR against `next`. See scripts/port_to_next.sh.

on:
schedule:
# Daily at 06:30 UTC.
- cron: "30 6 * * *"
workflow_dispatch:
inputs:
source_branch:
description: "Source branch to port into next"
required: false
default: v5-next
type: string

jobs:
port:
name: Forward-port v5-next into next
if: ${{ github.event_name != 'schedule' || github.repository == 'AztecProtocol/aztec-packages' }}
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
with:
fetch-depth: 0
token: ${{ secrets.AZTEC_BOT_GITHUB_TOKEN }}

- name: Configure Git
run: |
git config --global user.name AztecBot
git config --global user.email tech@aztecprotocol.com

- name: Run port script
id: port
continue-on-error: true
env:
GH_TOKEN: ${{ secrets.AZTEC_BOT_GITHUB_TOKEN }}
run: ./scripts/port_to_next.sh "${{ github.event.inputs.source_branch || 'v5-next' }}"

- name: Notify Slack of conflicts
if: steps.port.outputs.conflicts != ''
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
SOURCE_BRANCH: ${{ github.event.inputs.source_branch || 'v5-next' }}
PR_URL: ${{ steps.port.outputs.pr_url }}
CONFLICTS: ${{ steps.port.outputs.conflicts }}
run: |
WORKFLOW_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
TEXT=$(printf '⚠️ Daily port of `%s` → `next` hit merge conflicts. A PR was opened/updated for manual resolution: %s\n*Conflicted files:*\n```\n%s```\n<%s|View run>' \
"$SOURCE_BRANCH" "${PR_URL:-(none)}" "$CONFLICTS" "$WORKFLOW_URL")
curl -sS -X POST https://slack.com/api/chat.postMessage \
-H "Authorization: Bearer $SLACK_BOT_TOKEN" \
-H "Content-type: application/json" \
-d "$(jq -n --arg c "#backports" --arg t "$TEXT" '{channel:$c, text:$t}')"

- name: Notify Slack on unexpected failure
if: steps.port.outcome == 'failure'
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
SOURCE_BRANCH: ${{ github.event.inputs.source_branch || 'v5-next' }}
run: |
WORKFLOW_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
TEXT=$(printf '❌ Daily port of `%s` → `next` failed unexpectedly (not a merge conflict). (🤖) <%s|View run>' \
"$SOURCE_BRANCH" "$WORKFLOW_URL")
curl -sS -X POST https://slack.com/api/chat.postMessage \
-H "Authorization: Bearer $SLACK_BOT_TOKEN" \
-H "Content-type: application/json" \
-d "$(jq -n --arg c "#backports" --arg t "$TEXT" '{channel:$c, text:$t}')"
Loading
Loading