|
| 1 | +# Automated PR review gate setup guide |
| 2 | + |
| 3 | +Wire your repo so that when a pull request's CI finishes, ABCA automatically triages it and — once it's green and up to date — kicks off a structured [`coding/pr-review-v1`](./USER_GUIDE.md) review, posting the findings back on the PR. The goal is to keep up with AI-authored PR volume: a review is waiting by the time a human looks, and review compute is never spent on a PR whose tests are red. |
| 4 | + |
| 5 | +> This gate is **advisory and comments-only** — it never posts a check-run, commit status, or formal approve/request-changes review. It cannot block a merge or interfere with your branch-protection rules or [Mergify](../../.mergify.yml) queue. It only reads CI state, posts one edit-in-place comment, and (on green) fires the review webhook. |
| 6 | +
|
| 7 | +## What you get |
| 8 | + |
| 9 | +When `build` (or `integ`) completes on an open PR, the gate evaluates the PR head and does exactly one of: |
| 10 | + |
| 11 | +| PR state | What the gate does | |
| 12 | +|---|---| |
| 13 | +| **CI failing** | Edits a single `❌ CI is failing` comment listing the failing check names. **No review is triggered** — no wasted compute. Re-checks on every later CI run. | |
| 14 | +| **CI still pending** | Polls briefly, then exits quietly. The next CI completion re-evaluates. | |
| 15 | +| **Merge conflict** (`dirty`) | Edits a `⚠️ Merge conflict` comment asking the author to resolve and push. | |
| 16 | +| **Behind base, no conflict** | Calls GitHub's [update-branch API](https://docs.github.com/en/rest/pulls/pulls#update-a-pull-request-branch) to merge the base in so CI re-runs, then comments `🔄 Updated branch`. (Fork PRs get a "please update your branch" comment instead — see [Fork PRs](#fork-prs-vs-same-repo-branches).) | |
| 17 | +| **Green + up to date** | HMAC-signs and POSTs to the ABCA Task API webhook to start a `coding/pr-review-v1` review, then comments `🤖 ABCA review requested`. Findings post shortly after. | |
| 18 | + |
| 19 | +All status lives in **one** comment per PR (marked with a hidden `<!-- abca-review-gate -->`), edited in place — the gate never spams. Review triggering is idempotent per commit SHA, so re-runs on the same commit don't re-review; a new commit does. |
| 20 | + |
| 21 | +## How it works |
| 22 | + |
| 23 | +``` |
| 24 | +build / integ completes → workflow_run (trusted base-repo context) |
| 25 | + ↓ |
| 26 | + review-gate.yml resolves PR head SHA |
| 27 | + ↓ |
| 28 | + aggregate check-runs + commit statuses for that SHA |
| 29 | + ┌───────────┴────────────┐ |
| 30 | + failing/pending all green |
| 31 | + ↓ ↓ |
| 32 | + comment & stop check mergeable_state |
| 33 | + ┌───────┬──────────┬─────────┐ |
| 34 | + dirty behind clean/blocked |
| 35 | + ↓ ↓ ↓ |
| 36 | + comment update-branch HMAC POST |
| 37 | + (PAT, re-runs /v1/webhooks/tasks |
| 38 | + CI) {workflow_ref: |
| 39 | + coding/pr-review-v1, |
| 40 | + repo, pr_number} |
| 41 | + ↓ |
| 42 | + ABCA read-only review agent |
| 43 | + posts structured findings on PR |
| 44 | +``` |
| 45 | + |
| 46 | +Design notes: |
| 47 | + |
| 48 | +- **Runs in the trusted base-repo context.** The workflow triggers on `workflow_run` (not `pull_request`), so `secrets`/`vars`/the PAT are available even for fork PRs. No PR code is ever checked out or executed — the gate is pure `gh api` + `curl`. |
| 49 | +- **Reviews on green + not-behind + not-dirty**, *not* strictly `mergeable_state == clean`. Under branch protection a green, conflict-free PR reports `blocked` (awaiting approval), never `clean` — and the whole point is to review *before* a human approves. |
| 50 | +- **Auto-update needs a PAT.** A branch push made with the default `GITHUB_TOKEN` does not re-trigger `build` (GitHub's recursion prevention). The update-branch call uses `AUTOMATION_GITHUB_TOKEN` so CI re-fires and the gate re-pulses. |
| 51 | +- **The review agent itself is read-only.** `coding/pr-review-v1` posts findings via the GitHub Reviews API as `COMMENT` (never approve/request-changes) — see the [User guide](./USER_GUIDE.md). |
| 52 | + |
| 53 | +## Prerequisites |
| 54 | + |
| 55 | +- ABCA stack deployed (`mise //cdk:deploy`) — note the `ApiUrl` stack output (it already includes the `/v1/` stage). |
| 56 | +- The `bgagent` CLI installed and authenticated (`bgagent configure`, `bgagent login`). |
| 57 | +- The target repo is **onboarded** to ABCA with a Blueprint (`bgagent repo …`) — `coding/pr-review-v1` requires an onboarded repo. Confirm with `bgagent repo list`. |
| 58 | +- Admin access to the GitHub repo's **Settings → Secrets and variables → Actions** (to add repo vars/secrets). |
| 59 | +- An `AUTOMATION_GITHUB_TOKEN` repo secret already exists (a PAT with `contents` + `pull-requests` write). It's shared with the `upgrade-main` / `auto-approve` workflows. |
| 60 | + |
| 61 | +## Step-by-step setup |
| 62 | + |
| 63 | +### Step 1 — Register an ABCA webhook |
| 64 | + |
| 65 | +The gate authenticates to the Task API with a per-webhook HMAC secret. Mint one: |
| 66 | + |
| 67 | +```bash |
| 68 | +bgagent webhook create --name review-gate |
| 69 | +``` |
| 70 | + |
| 71 | +Output (the secret is shown **once** — copy it now): |
| 72 | + |
| 73 | +``` |
| 74 | +Webhook: 01J… # ← this is ABCA_WEBHOOK_ID |
| 75 | +Name: review-gate |
| 76 | +Created: 2026-07-14T… |
| 77 | +
|
| 78 | +Secret (store securely — shown only once): |
| 79 | +a1b2c3… # ← this is ABCA_WEBHOOK_SECRET |
| 80 | +``` |
| 81 | + |
| 82 | +The webhook's owning Cognito user must be allowed to submit `coding/pr-review-v1`. The secret is stored server-side at `bgagent/webhook/<webhook_id>` in Secrets Manager; the value you paste into GitHub below must match it exactly. |
| 83 | + |
| 84 | +### Step 2 — Set the repo variables and secret |
| 85 | + |
| 86 | +Using the [`gh` CLI](https://cli.github.com/) against your repo (or the GitHub UI, Settings → Secrets and variables → Actions): |
| 87 | + |
| 88 | +```bash |
| 89 | +REPO=<owner>/<repo> |
| 90 | + |
| 91 | +# Variables (non-secret) — ApiUrl output, NO trailing slash (a trailing slash |
| 92 | +# produces //webhooks/tasks and the call 404s): |
| 93 | +gh variable set ABCA_TASK_API_URL --repo "$REPO" --body "https://<api-id>.execute-api.<region>.amazonaws.com/v1" |
| 94 | +gh variable set ABCA_WEBHOOK_ID --repo "$REPO" --body "01J…" |
| 95 | + |
| 96 | +# Secret — the value printed by `bgagent webhook create`: |
| 97 | +gh secret set ABCA_WEBHOOK_SECRET --repo "$REPO" --body "a1b2c3…" |
| 98 | +``` |
| 99 | + |
| 100 | +`ABCA_TASK_API_URL` is the `ApiUrl` stack output verbatim (it already ends in `/v1`); the workflow appends `/webhooks/tasks`. |
| 101 | + |
| 102 | +### Step 3 — Confirm the workflow is on the default branch |
| 103 | + |
| 104 | +`workflow_run` workflows only run from the copy of the file on the repo's **default branch**. Merge `.github/workflows/review-gate.yml` to the default branch (it ships with the repo). It is inert on any other branch. |
| 105 | + |
| 106 | +Until it's merged, you can exercise it manually: **Actions → review-gate → Run workflow**, pick the branch, and pass a `pr_number`. |
| 107 | + |
| 108 | +### Step 4 — Smoke test |
| 109 | + |
| 110 | +Verify the webhook end to end without waiting for a PR, using the same signing scheme the gate uses: |
| 111 | + |
| 112 | +```bash |
| 113 | +bgagent webhook test --repo <owner>/<repo> --secret "<ABCA_WEBHOOK_SECRET>" |
| 114 | +``` |
| 115 | + |
| 116 | +A `2xx` means the webhook + secret are wired correctly. Then open a small test PR and watch the `review-gate` workflow run in the Actions tab: a red PR should get the `❌ CI is failing` comment; a green one should get `🤖 ABCA review requested` followed by the agent's review. |
| 117 | + |
| 118 | +## Fork PRs vs same-repo branches |
| 119 | + |
| 120 | +The gate handles both, but auto-update differs: |
| 121 | + |
| 122 | +- **Same-repo branch PRs** (the common case, e.g. `bgagent/…` branches the agent opens on your fork): a `behind` branch is auto-updated via the PAT, CI re-runs, and the gate re-pulses to green. |
| 123 | +- **Cross-fork PRs**: GitHub's update-branch API requires "Allow edits by maintainers" **and** PAT write access to the fork, which usually isn't available. When the head repo differs from the base repo and the branch is behind, the gate posts a "please update your branch" comment instead of attempting the API call. |
| 124 | + |
| 125 | +## Excluding some PRs from auto-review (optional) |
| 126 | + |
| 127 | +By default the gate evaluates **every** open PR whose CI completes, including autonomous `bgagent/…` PRs. If you'd rather not auto-review certain PRs (e.g. to save compute on throwaway ones), filter in the resolve step of `review-gate.yml` — for example, skip when the head branch matches a prefix or the author is a bot. This is a workflow edit and is CODEOWNERS-gated to admins upstream. |
| 128 | + |
| 129 | +## Troubleshooting |
| 130 | + |
| 131 | +### The `review-gate` workflow doesn't run at all |
| 132 | + |
| 133 | +- It only fires from the **default-branch** copy of the file. Confirm `.github/workflows/review-gate.yml` is on the default branch, not just a feature branch. |
| 134 | +- It triggers on `build`/`integ` completion. A PR that hasn't had `build` run yet won't have pulsed the gate — push a commit or use **Run workflow** (`workflow_dispatch`). |
| 135 | + |
| 136 | +### Gate logs `ABCA webhook not configured` |
| 137 | + |
| 138 | +One of `ABCA_TASK_API_URL` / `ABCA_WEBHOOK_ID` (repo **variables**) or `ABCA_WEBHOOK_SECRET` (repo **secret**) is unset. Note vars and secrets are separate GitHub stores — check both. Re-run Step 2. |
| 139 | + |
| 140 | +### Task API returns 401 / 403 |
| 141 | + |
| 142 | +The signature didn't verify. Almost always the `ABCA_WEBHOOK_SECRET` in GitHub doesn't match the value stored at `bgagent/webhook/<id>` in Secrets Manager — re-run `bgagent webhook create` and update the secret, or confirm you copied the full value. (The secret is only shown at creation; if you lost it, create a new webhook.) |
| 143 | + |
| 144 | +### Task API returns 422 `repo not onboarded` |
| 145 | + |
| 146 | +`coding/pr-review-v1` requires the repo to be onboarded with a Blueprint. Run `bgagent repo list` and onboard it if missing. |
| 147 | + |
| 148 | +### A green PR isn't triggering a review |
| 149 | + |
| 150 | +- Check the run log for the resolved `mergeable_state`. `dirty`/`behind` are handled separately (conflict/update comments). Only genuinely green + not-behind + not-dirty triggers a review. |
| 151 | +- Review triggering is per-SHA idempotent. If the gate already commented `🤖 ABCA review requested` for the current commit, it won't fire again until a new commit lands. |
| 152 | + |
| 153 | +### The branch was auto-updated but approvals disappeared |
| 154 | + |
| 155 | +Expected. Mergify's "dismiss stale approvals on new commits" treats the update-branch merge commit as a push, so prior approvals are dismissed and re-approval is required after CI re-runs — the intended invariant, not a regression. |
0 commit comments