Skip to content

Commit 5d0d627

Browse files
authored
Require two humans for bot authored PRs (#7232)
Signed-off-by: Nicholas Gates <nick@nickgates.com>
1 parent b573853 commit 5d0d627

File tree

3 files changed

+122
-13
lines changed

3 files changed

+122
-13
lines changed

.github/workflows/approvals.yml

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
name: PR Approval Check
2+
3+
on:
4+
pull_request_review:
5+
types: [submitted, dismissed]
6+
pull_request:
7+
types: [opened, synchronize, reopened]
8+
9+
jobs:
10+
check-approvals:
11+
runs-on: ubuntu-latest
12+
steps:
13+
- name: Check required approvals
14+
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd
15+
with:
16+
script: |
17+
const pr = context.payload.pull_request;
18+
const reviews = await github.rest.pulls.listReviews({
19+
owner: context.repo.owner,
20+
repo: context.repo.repo,
21+
pull_number: pr.number,
22+
});
23+
24+
// Count unique human approvals (latest review per user)
25+
const latestByUser = {};
26+
for (const review of reviews.data) {
27+
if (review.user.type === 'Bot') continue;
28+
latestByUser[review.user.login] = review.state;
29+
}
30+
const approvalCount = Object.values(latestByUser)
31+
.filter(state => state === 'APPROVED').length;
32+
33+
// Determine if PR author is a bot/GitHub Actions
34+
const authorType = pr.user.type; // 'Bot' vs 'User'
35+
const authorLogin = pr.user.login; // e.g. 'github-actions[bot]'
36+
const isBot = authorType === 'Bot' || authorLogin.endsWith('[bot]');
37+
38+
const required = isBot ? 2 : 1;
39+
40+
console.log(`PR author: ${authorLogin} (${authorType}), isBot: ${isBot}`);
41+
console.log(`Approvals: ${approvalCount} / ${required} required`);
42+
43+
if (approvalCount < required) {
44+
core.setFailed(
45+
`This PR needs ${required} human approval(s) but has ${approvalCount}. ` +
46+
`(Author is ${isBot ? 'a bot' : 'human'})`
47+
);
48+
}

CONTRIBUTING.md

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -4,19 +4,15 @@ Welcome, and thank you for your interest in contributing to Vortex! We are delig
44

55
We ask that you read the guidelines below in order to make the process as streamlined as possible.
66

7-
## AI Assistance Notice
7+
## AI Assistance
88

99
> [!IMPORTANT]
10-
> The Vortex project allows AI-assisted contributions, which must be properly
11-
> disclosed in the pull request.
10+
> The Vortex project permits and embraces AI-assisted contributions. Contributors
11+
> should disclose usage of conversational or agentic AI tools in the PR description.
1212
>
13-
> If you are using any kind of AI assistance when contributing to Vortex, please
14-
> disclose this in your pull request, along with the extent to which it was used
15-
> (e.g. writing docs or code generation).
16-
>
17-
> Contributors are required to be able to understand the AI-assisted output and
18-
> reason about it. Should a PR indicate no visible human accountability and
19-
> involvement we reserve the right to close it.
13+
> For the full AI policy — including disclosure requirements, review standards for
14+
> AI-generated PRs, and rules for autonomous agents — see the
15+
> [contributing guide](https://vortex.dev/project/contributing.html#ai-assistance).
2016
2117
## Code Contributions
2218

@@ -35,7 +31,7 @@ The contribution process is outlined below:
3531

3632
3. Open a PR to indicate that the change is ready for review.
3733
- Ensure that you sign your work via DCO (see below).
38-
- Disclose LLM usage as described in [AI Assistance Notice](#ai-assistance-notice).
34+
- Disclose LLM usage as described in [AI Assistance](#ai-assistance).
3935
- CI requires approval from external committers.
4036

4137
## Governance

docs/project/contributing.md

Lines changed: 67 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,73 @@ git commit -s
3434

3535
## AI Assistance
3636

37-
AI-assisted contributions are permitted but must be disclosed in the pull request, along with the
38-
extent of use. Contributors must be able to understand and reason about AI-generated output.
37+
The Vortex project permits and embraces AI-assisted contributions. Today, most PRs involve some
38+
degree of AI assistance — whether through IDE autocomplete, conversational AI, or autonomous agents.
39+
This section describes our expectations around disclosure, review, and accountability.
40+
41+
### Repository Setup
42+
43+
The repository has [Claude Code](https://docs.anthropic.com/en/docs/claude-code) configured as a
44+
GitHub Action. Users with write access to the repository can mention `@claude` in PR comments or
45+
issue comments to trigger AI-powered code reviews, request changes, or generate PRs. Configuration
46+
for Claude's behavior lives in `CLAUDE.md` at the repository root.
47+
48+
### Disclosure
49+
50+
Contributors should disclose AI usage in the PR description when conversational or agentic AI tools
51+
(e.g., Claude, ChatGPT, Claude Code) were used to produce code, documentation, or tests. Standard
52+
IDE autocomplete (e.g., Copilot tab-completion) does not require disclosure.
53+
54+
AI-translated content should note that it was translated with AI assistance.
55+
56+
### Human vs. Agent PRs
57+
58+
We distinguish between two kinds of AI-assisted PRs:
59+
60+
- **Human PRs** — A human writes the PR, possibly with significant AI assistance. The human author
61+
is accountable for the code quality and correctness, just as they would be for any handwritten
62+
code. Standard review rules apply (one approving reviewer).
63+
64+
- **Agent PRs** — An autonomous AI agent (e.g., triggered via `@claude` or a scheduled GitHub
65+
Action) opens the PR with minimal human steering. Agent PRs require **two human reviewers**
66+
before merge.
67+
68+
The distinction is straightforward: if a human opened the PR, it's a human PR. If an automated
69+
agent opened it, it's an agent PR.
70+
71+
### Review Standards
72+
73+
AI-assisted code should receive extra scrutiny during review. AI tools can produce code that is
74+
superficially correct but subtly wrong — off-by-one errors, incorrect edge case handling, or tests
75+
that pass without actually exercising the intended behavior. Reviewers should pay particular
76+
attention to:
77+
78+
- Correctness of logic, not just whether it compiles and passes CI.
79+
- Tests that genuinely validate behavior rather than merely achieving coverage.
80+
- Unnecessary complexity or over-abstraction.
81+
82+
The project may use AI-powered review tools on PRs. Reviewers are free to use AI to assist their
83+
reviews without disclosure.
84+
85+
### AI Agents
86+
87+
AI agents (bots, scheduled actions, etc.) are permitted to:
88+
89+
- Open pull requests, with a human assigned as the responsible party.
90+
- Post review comments on pull requests.
91+
92+
AI agents must **not**:
93+
94+
- Merge or approve pull requests.
95+
- Push code to protected branches.
96+
97+
A human is accountable for all actions taken by an agent operating under their authority.
98+
99+
### AI-Generated Tests
100+
101+
AI-generated tests are welcome, but contributors must verify that tests actually exercise the
102+
intended behavior — not just pass. A green test suite produced by AI can give a false sense of
103+
coverage if the assertions are trivial or the setup doesn't reflect real conditions.
39104

40105
## Coding Style
41106

0 commit comments

Comments
 (0)