Skip to content

Commit 500bc41

Browse files
jorbenclaude
jorben
and
claude
authored
ci: add LLM PR review workflow with manual trigger (#39)
* ci: 👷 add LLM PR review workflow with manual trigger Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): 🐛 use x-api-key header for Anthropic API auth Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): 🐛 use vars instead of secrets for API base URL and model ID Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): 🐛 extract text content by type instead of fixed index Some models return thinking blocks before text content, causing the review text extraction to fail when using content[0]. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): 🐛 use shallow clone and defensive jq extraction Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): 🐛 improve diff handling for LLM review - Truncate on line boundary to avoid splitting multi-byte UTF-8 chars - Wrap diff in code fence for clearer LLM parsing - Add hunk header guidance in system prompt for accurate line references Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent d54cc64 commit 500bc41

1 file changed

Lines changed: 139 additions & 0 deletions

File tree

‎.github/workflows/llm-pr-review.yml‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
name: LLM PR Review
2+
3+
on:
4+
pull_request:
5+
types: [opened, synchronize, reopened]
6+
branches:
7+
- master
8+
workflow_dispatch:
9+
inputs:
10+
pr_number:
11+
description: "Pull request number to review"
12+
required: true
13+
type: number
14+
15+
concurrency:
16+
group: llm-review-${{ github.event.pull_request.number || github.event.inputs.pr_number }}
17+
cancel-in-progress: true
18+
19+
permissions:
20+
contents: read
21+
pull-requests: write
22+
23+
jobs:
24+
llm-review:
25+
runs-on: ubuntu-latest
26+
27+
steps:
28+
- name: Checkout code
29+
uses: actions/checkout@v4
30+
with:
31+
fetch-depth: 1
32+
33+
# This step uses the Anthropic Messages API.
34+
# Secrets required: APIKey; Variables required: APIBase, ModelId
35+
- name: Generate LLM review
36+
env:
37+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
38+
API_BASE: ${{ vars.ANTHROPIC_BASE_URL }}
39+
API_KEY: ${{ secrets.ANTHROPIC_AUTH_TOKEN }}
40+
MODEL_ID: ${{ vars.ANTHROPIC_MODEL }}
41+
PR_NUMBER: ${{ github.event.pull_request.number || github.event.inputs.pr_number }}
42+
REPO: ${{ github.repository }}
43+
run: |
44+
set -euo pipefail
45+
IFS=$'\n\t'
46+
47+
pr_json=$(curl -sf \
48+
-H "Authorization: Bearer $GITHUB_TOKEN" \
49+
-H "Accept: application/vnd.github+json" \
50+
"https://api.github.com/repos/$REPO/pulls/$PR_NUMBER") || {
51+
echo "::error::Failed to fetch PR metadata"
52+
exit 1
53+
}
54+
55+
pr_title=$(printf "%s" "$pr_json" | jq -r '.title // ""')
56+
pr_body=$(printf "%s" "$pr_json" | jq -r '.body // ""')
57+
58+
pr_diff=$(curl -sf \
59+
-H "Authorization: Bearer $GITHUB_TOKEN" \
60+
-H "Accept: application/vnd.github.v3.diff" \
61+
"https://api.github.com/repos/$REPO/pulls/$PR_NUMBER") || {
62+
echo "::error::Failed to fetch PR diff"
63+
exit 1
64+
}
65+
66+
max_diff_chars=131072
67+
pr_diff_trimmed=$(printf "%s" "$pr_diff" | head -c $max_diff_chars | sed '$d')
68+
69+
system_prompt=$(cat <<'EOF'
70+
You are a senior software engineer performing a pull request code review.
71+
Focus on: Correctness, Security, Performance, Maintainability, Readability, Testing.
72+
Output must be in English.
73+
74+
The diff is in unified diff format. Each hunk starts with a header like:
75+
@@ -old_start,old_count +new_start,new_count @@
76+
Use the +new_start line numbers from these hunk headers to calculate the
77+
actual file line numbers when referencing code in your review.
78+
79+
Use this exact format and omit empty sections:
80+
# Summary
81+
<1-2 sentences overall assessment>
82+
83+
# Critical
84+
- [file:line] Title: explanation...
85+
86+
# Important
87+
- [file:line] Title: explanation...
88+
89+
# Suggestion
90+
- [file:line] Title: explanation...
91+
92+
# Praise
93+
- [file:line] Title: explanation...
94+
EOF
95+
)
96+
97+
request_body=$(jq -n \
98+
--arg model "$MODEL_ID" \
99+
--arg system "$system_prompt" \
100+
--arg title "$pr_title" \
101+
--arg body "$pr_body" \
102+
--arg diff "$pr_diff_trimmed" \
103+
'{
104+
model: $model,
105+
max_tokens: 8192,
106+
system: $system,
107+
messages: [
108+
{ role: "user", content: ("Review the following pull request.\n\n## Title\n" + $title + "\n\n## Description\n" + $body + "\n\n## Diff\n```diff\n" + $diff + "\n```") }
109+
]
110+
}')
111+
112+
llm_response=$(curl -sf \
113+
-H "x-api-key: $API_KEY" \
114+
-H "Content-Type: application/json" \
115+
-H "anthropic-version: 2023-06-01" \
116+
"$API_BASE/v1/messages" \
117+
-d "$request_body") || {
118+
echo "::error::LLM API request failed"
119+
exit 1
120+
}
121+
122+
review_text=$(printf "%s" "$llm_response" | jq -r '[.content[] | select(.type == "text")] | first? | .text // ""')
123+
124+
if [ -z "$review_text" ]; then
125+
echo "::error::LLM response missing content. Raw response:"
126+
printf "%s" "$llm_response" | jq .
127+
exit 1
128+
fi
129+
130+
review_payload=$(jq -n --arg body "$review_text" '{body: $body, event: "COMMENT"}')
131+
132+
curl -sf \
133+
-H "Authorization: Bearer $GITHUB_TOKEN" \
134+
-H "Accept: application/vnd.github+json" \
135+
"https://api.github.com/repos/$REPO/pulls/$PR_NUMBER/reviews" \
136+
-d "$review_payload" >/dev/null || {
137+
echo "::error::Failed to post PR review"
138+
exit 1
139+
}

0 commit comments

Comments
 (0)