Skip to content

Commit 3200acf

Browse files
detainclaude
andcommitted
refactor(github-code-review): commit change snapshot so agent fixes are a clean diff
Review the changes on a committed baseline instead of an uncommitted working tree, so the reviewer's in-place fixes can be captured as their own diff. - processJob(): after applyPRDiff, record _base and commit the change as a snapshot (commitPrSnapshot). HEAD=base+change, _base=base — so `git diff _base HEAD` is the change under review and a later `git diff` is only the agent's fixes. - Capture fixes from git (captureAgentFixes/getAgentFileDiff) instead of a second opencode "improve" pass. Remove runOpencodeImprove, parseImproveOutput, extractFixedContent, getFileDiff and OPENCODE_IMPROVE_CMD. - resetCheckoutToCleanState wipes the snapshot commit and the _base ref so cached checkouts can't drift. - review.txt: the agent now fixes issues in place (scoped to the changed lines, no git writes) and reports in the parseable format. Worded generically so it serves commit reviews as well as PRs. - tests: add CommitBaselineTest; drop tests for the removed functions. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent dcfa122 commit 3200acf

10 files changed

Lines changed: 1660 additions & 1029 deletions

scripts/github-code-review.php

Lines changed: 888 additions & 642 deletions
Large diffs are not rendered by default.

scripts/prompts/review.txt

Lines changed: 82 additions & 102 deletions
Original file line numberDiff line numberDiff line change
@@ -1,102 +1,82 @@
1-
# Code Review Philosophy
2-
3-
## TL;DR
4-
Systematic code review across 4 layers with severity classification. Only report findings with ≥80% confidence. Include file:line references for all issues.
5-
6-
## When to Use This Skill
7-
- Before reporting implementation completion
8-
- When explicitly asked to review code
9-
- When using the `/review` command
10-
- As an independent audit after code changes
11-
12-
## The 4 Review Layers
13-
14-
### Layer 1: Correctness
15-
- Logic errors and edge cases
16-
- Error handling completeness
17-
- Type safety and null checks
18-
- Algorithm correctness
19-
- Off-by-one errors
20-
21-
### Layer 2: Security
22-
- No hardcoded secrets or API keys
23-
- Input validation and sanitization
24-
- Injection vulnerability prevention (SQL, XSS, command)
25-
- Authentication and authorization checks
26-
- Sensitive data not logged
27-
- OWASP Top 10 awareness
28-
29-
### Layer 3: Performance
30-
- No N+1 query patterns
31-
- Appropriate caching strategies
32-
- No unnecessary re-renders (React/frontend)
33-
- Lazy loading where appropriate
34-
- Memory leak prevention
35-
- Algorithmic complexity concerns
36-
37-
### Layer 4: Style & Maintainability
38-
- Adherence to project conventions (check AGENTS.md)
39-
- Code duplication (DRY violations)
40-
- Complexity management (cyclomatic complexity)
41-
- Documentation completeness
42-
- Test coverage gaps
43-
44-
## Severity Classification
45-
46-
| Severity | Icon | Criteria | Action Required |
47-
|----------|------|----------|-----------------|
48-
| Critical | 🔴 | Security vulnerabilities, crashes, data loss, corruption | Must fix before merge |
49-
| Major | 🟠 | Bugs, performance issues, missing error handling | Should fix |
50-
| Minor | 🟡 | Code smells, maintainability issues, test gaps | Nice to fix |
51-
| Nitpick | 🟢 | Style preferences, naming suggestions, documentation | Optional |
52-
53-
## Confidence Threshold
54-
**Only report findings with ≥80% confidence.**
55-
56-
If uncertain about an issue:
57-
- State the uncertainty explicitly: "Potential issue (70% confidence): ..."
58-
- Suggest investigation rather than assert a problem
59-
- Prefer false negatives over false positives (reduce noise)
60-
61-
## Review Process
62-
63-
1. **Initial Scan** - Identify all files in scope, understand the change
64-
2. **Deep Analysis** - Apply all 4 layers systematically to each file
65-
3. **Context Evaluation** - Consider surrounding code, project patterns, existing conventions
66-
4. **Philosophy Check** - Verify against code-philosophy (5 Laws) if applicable
67-
5. **Synthesize Findings** - Group by severity, deduplicate, prioritize
68-
69-
## Output Format
70-
71-
Structure your review as:
72-
73-
1. **Files Reviewed** - List all files analyzed
74-
2. **Overall Assessment** - APPROVE | REQUEST_CHANGES | NEEDS_DISCUSSION
75-
3. **Summary** - 2-3 sentence overview
76-
4. **Critical Issues** (🔴) - With file:line references
77-
5. **Major Issues** (🟠) - With file:line references
78-
6. **Minor Issues** (🟡) - With file:line references
79-
7. **Positive Observations** (🟢) - What's done well (always include at least one)
80-
8. **Philosophy Compliance** - Checklist results if applicable
81-
82-
## What NOT to Do
83-
84-
- Do NOT report low-confidence findings as definite issues
85-
- Do NOT provide vague feedback without file:line references
86-
- Do NOT skip any of the 4 layers
87-
- Do NOT forget to note positive observations
88-
- Do NOT modify any files during review
89-
- Do NOT approve without completing the full review process
90-
91-
## Adherence Checklist
92-
93-
Before completing a review, verify:
94-
- [ ] All 4 layers analyzed (Correctness, Security, Performance, Style)
95-
- [ ] Severity assigned to each finding
96-
- [ ] Confidence ≥80% for all reported issues (or uncertainty stated)
97-
- [ ] File names and line numbers included for all findings
98-
- [ ] Positive observations noted
99-
- [ ] Output follows the standard format
100-
101-
102-
Start this now and continu till its finished.
1+
You are an automated Code Review agent. Your job is to REVIEW a set of code
2+
changes AND fix the problems you find by editing the affected files in place.
3+
The changes may come from a pull request, a pushed commit, or a range of commits
4+
— the review works the same way regardless of the source.
5+
6+
## What changed
7+
8+
The changes under review have been committed as a snapshot on top of the code as
9+
it was before them. To see EXACTLY what changed, run:
10+
11+
git diff _base HEAD
12+
13+
`_base` is the baseline (the code before the change); `HEAD` is the baseline plus
14+
the changes under review. That diff — and only that diff — is your review scope.
15+
Read the surrounding code for context, but do not go looking for unrelated
16+
problems elsewhere in the repository.
17+
18+
## How to fix
19+
20+
For each real issue in the changed lines, edit the file in place to fix it:
21+
22+
- Make MINIMAL, SURGICAL changes. Touch only what is needed to fix the specific
23+
issue, scoped to the lines the change added or modified.
24+
- Do NOT refactor, reformat, rename, or "improve" code that is unrelated to the
25+
issue. No drive-by cleanups.
26+
- Preserve `declare(strict_types=1);` at the top of every PHP file.
27+
- Follow the existing style: PSR-2 plus PHP 7.4 conventions. Match the file's
28+
existing indentation and formatting.
29+
- Leave your edits UNCOMMITTED in the working tree. Do NOT run `git commit`,
30+
`git add`, `git reset`, `git stash`, `git checkout`, or any other command that
31+
stages, commits, or discards changes. The worker captures your edits with a
32+
plain `git diff` afterwards.
33+
- If you cannot fix an issue safely with a small in-place change (for example it
34+
would require an API change, a new dependency, or broad restructuring), leave
35+
the code as-is and only FLAG it.
36+
37+
## Confidence
38+
39+
Only report issues you are at least 80% confident are real. Prefer silence over
40+
noise: a false positive wastes the author's time.
41+
42+
## Output format — follow EXACTLY
43+
44+
After you finish editing, print a report. Each issue MUST be a single heading line
45+
in this exact shape (the worker parses these lines):
46+
47+
#### <emoji> <title> — <path>:<line>
48+
49+
Where:
50+
51+
- `<emoji>` is exactly one of these severity markers:
52+
🔴 critical — security holes, crashes, data loss/corruption
53+
🟠 major — bugs, missing error handling, real correctness problems
54+
🟡 minor — code smells, maintainability, small correctness risks
55+
🟢 nitpick — style, naming, documentation
56+
- `<title>` is a short summary of the issue (a few words).
57+
- `<path>` is the repository-relative file path (no spaces, no colons).
58+
- `<line>` is the line number in the changed (HEAD) version of the file.
59+
- Use a spaced em dash ` — ` between the title and `<path>:<line>`.
60+
61+
Immediately below each heading, write ONE short paragraph. It MUST state whether
62+
you FIXED the issue in place or only FLAGGED it, and briefly why. Start that
63+
paragraph with the word `FIXED` or `FLAGGED`.
64+
65+
Separate issues with a blank line. Example:
66+
67+
#### 🔴 SQL injection in user lookup — include/Api/Users.php:207
68+
FIXED: the username was concatenated straight into the query; switched it to a
69+
parameterized `$db->real_escape()` call.
70+
71+
#### 🟠 Unhandled null from getService() — include/Api/Users.php:88
72+
FLAGGED: getService() can return null here, but guarding it correctly needs a
73+
public signature change, so it is out of scope for an in-place fix.
74+
75+
After all issues, end the report with a single summary line, on its own line,
76+
in exactly this form (N = issues you fixed, M = total issues you reported):
77+
78+
Fixed N of M issues.
79+
80+
If you find no issues at all, print exactly:
81+
82+
Fixed 0 of 0 issues.

tests/phpunit/unit/CodeReviewWorker/BaseBranchFallbackTest.php

Lines changed: 6 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ public function testBaseBranchFallbackFlow(): void
3333
$checkoutCalls = [];
3434
$running = true;
3535

36-
$mockCheckoutBranch = function(string $repo, string $branch, string $path) use (&$checkoutCalls, &$running): string {
36+
$mockCheckoutBranch = function(string $repo, string $branch, string $path) use (&$checkoutCalls): string {
3737
$checkoutCalls[] = ['repo' => $repo, 'branch' => $branch, 'path' => $path];
3838
if ($branch === 'main') {
3939
return 'gh repo clone failed: main branch not found';
@@ -44,10 +44,7 @@ public function testBaseBranchFallbackFlow(): void
4444
return 'unknown branch';
4545
};
4646

47-
$mockGetPRBaseBranch = function(string $repo, int $prNumber) use (&$running): ?string {
48-
if (!$running) {
49-
return null;
50-
}
47+
$mockGetPRBaseBranch = function(string $repo, int $prNumber): string {
5148
return 'master';
5249
};
5350

@@ -86,7 +83,7 @@ public function testBaseBranchFallbackFailsWhenActualBranchCheckoutAlsoFails():
8683
return 'gh repo clone failed: branch not found';
8784
};
8885

89-
$mockGetPRBaseBranch = function(string $repo, int $prNumber): ?string {
86+
$mockGetPRBaseBranch = function(string $repo, int $prNumber): string {
9087
return 'master';
9188
};
9289

@@ -124,7 +121,7 @@ public function testGetPRBaseBranchIsCalledOnlyWhenFirstCheckoutFails(): void
124121
return 'true';
125122
};
126123

127-
$mockGetPRBaseBranch = function(string $repo, int $prNumber) use (&$getPRBaseBranchCalls): ?string {
124+
$mockGetPRBaseBranch = function(string $repo, int $prNumber) use (&$getPRBaseBranchCalls): string {
128125
$getPRBaseBranchCalls[] = ['repo' => $repo, 'prNumber' => $prNumber];
129126
return 'master';
130127
};
@@ -164,7 +161,7 @@ public function testGetPRBaseBranchNotCalledWhenShutdownBeforeFallback(): void
164161
return 'shutdown';
165162
};
166163

167-
$mockGetPRBaseBranch = function(string $repo, int $prNumber) use (&$getPRBaseBranchCalls): ?string {
164+
$mockGetPRBaseBranch = function(string $repo, int $prNumber) use (&$getPRBaseBranchCalls): string {
168165
$getPRBaseBranchCalls[] = ['repo' => $repo, 'prNumber' => $prNumber];
169166
return 'master';
170167
};
@@ -244,7 +241,7 @@ public function testActualBaseBranchSameAsBaseBranchDoesNotRetry(): void
244241
return 'gh repo clone failed: main branch not found';
245242
};
246243

247-
$mockGetPRBaseBranch = function(string $repo, int $prNumber): ?string {
244+
$mockGetPRBaseBranch = function(string $repo, int $prNumber): string {
248245
return 'main';
249246
};
250247

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
<?php
2+
declare(strict_types=1);
3+
4+
namespace Webhooks\Tests\Unit\CodeReviewWorker;
5+
6+
use PHPUnit\Framework\TestCase;
7+
8+
/**
9+
* Tests for the commit-baseline redesign in scripts/github-code-review.php:
10+
* commitPrSnapshot(), captureAgentFixes(), getAgentFileDiff(), and
11+
* resetCheckoutToCleanState(). Each test drives a real throwaway git repo so
12+
* the git plumbing is exercised end to end.
13+
*/
14+
class CommitBaselineTest extends TestCase
15+
{
16+
private string $dir = '';
17+
18+
protected function setUp(): void
19+
{
20+
require_once __DIR__ . '/../../../../scripts/github-code-review.php';
21+
$GLOBALS['verbose'] = 0;
22+
$GLOBALS['logFile'] = null;
23+
24+
$this->dir = sys_get_temp_dir() . '/crw_baseline_' . bin2hex(random_bytes(6));
25+
mkdir($this->dir, 0777, true);
26+
}
27+
28+
protected function tearDown(): void
29+
{
30+
if ($this->dir !== '' && is_dir($this->dir)) {
31+
exec('rm -rf ' . escapeshellarg($this->dir));
32+
}
33+
}
34+
35+
/** Run a git command in the test repo and return trimmed stdout+stderr. */
36+
private function git(string $args): string
37+
{
38+
return trim((string)shell_exec('cd ' . escapeshellarg($this->dir) . ' && git ' . $args . ' 2>&1'));
39+
}
40+
41+
/**
42+
* Create an initial commit on a `main` branch with the given files and
43+
* return the base commit SHA.
44+
*
45+
* @param array<string, string> $files path => contents
46+
*/
47+
private function initBaseRepo(array $files): string
48+
{
49+
$this->git('init -q');
50+
foreach ($files as $path => $contents) {
51+
file_put_contents($this->dir . '/' . $path, $contents);
52+
}
53+
$this->git('add -A');
54+
$this->git('-c user.email=setup@test -c user.name=Setup commit -q -m base');
55+
$this->git('branch -M main');
56+
return $this->git('rev-parse HEAD');
57+
}
58+
59+
public function testCommitPrSnapshotRecordsBaseAndCommitsPr(): void
60+
{
61+
$baseSha = $this->initBaseRepo(['a.php' => "<?php\necho 1;\n"]);
62+
63+
// Apply a "PR": modify a tracked file and add a new one.
64+
file_put_contents($this->dir . '/a.php', "<?php\necho 2;\n");
65+
file_put_contents($this->dir . '/b.php', "<?php\necho 'new';\n");
66+
67+
$this->assertTrue(commitPrSnapshot($this->dir, 42));
68+
69+
// _base pins the base commit; HEAD has advanced to the snapshot.
70+
$this->assertSame($baseSha, $this->git('rev-parse _base'), '_base must pin the base commit');
71+
$this->assertNotSame($baseSha, $this->git('rev-parse HEAD'), 'HEAD must advance to the snapshot');
72+
$this->assertSame('PR #42 snapshot', $this->git('log -1 --pretty=%s'));
73+
74+
// Working tree is clean and the PR shows up as _base..HEAD.
75+
$this->assertSame('', $this->git('status --porcelain'), 'tree must be clean after snapshot');
76+
$changed = $this->git('diff --name-only _base HEAD');
77+
$this->assertStringContainsString('a.php', $changed);
78+
$this->assertStringContainsString('b.php', $changed);
79+
}
80+
81+
public function testCaptureAgentFixesReturnsOnlyAgentEdits(): void
82+
{
83+
$this->initBaseRepo(['a.php' => "<?php\necho 1;\n"]);
84+
file_put_contents($this->dir . '/a.php', "<?php\necho 2;\n");
85+
file_put_contents($this->dir . '/b.php', "<?php\necho 'new';\n");
86+
$this->assertTrue(commitPrSnapshot($this->dir, 7));
87+
88+
// Agent edits one file in place, leaving it uncommitted.
89+
file_put_contents($this->dir . '/a.php', "<?php\necho 2; // fixed by agent\n");
90+
91+
$fixes = captureAgentFixes($this->dir);
92+
$this->assertSame(['a.php'], $fixes['files'], 'only the agent-edited file is listed');
93+
$this->assertStringContainsString('fixed by agent', $fixes['diff']);
94+
95+
// Per-file diff has hunks only for the edited file.
96+
$this->assertStringContainsString('fixed by agent', getAgentFileDiff($this->dir, 'a.php'));
97+
$this->assertSame('', trim(getAgentFileDiff($this->dir, 'b.php')));
98+
}
99+
100+
public function testCaptureAgentFixesEmptyWhenTreeClean(): void
101+
{
102+
$this->initBaseRepo(['a.php' => "<?php\necho 1;\n"]);
103+
file_put_contents($this->dir . '/a.php', "<?php\necho 2;\n");
104+
$this->assertTrue(commitPrSnapshot($this->dir, 1));
105+
106+
$fixes = captureAgentFixes($this->dir);
107+
$this->assertSame([], $fixes['files']);
108+
$this->assertSame('', $fixes['diff']);
109+
}
110+
111+
public function testResetCheckoutToCleanStateWipesEditsSnapshotAndBaseRef(): void
112+
{
113+
$baseSha = $this->initBaseRepo(['a.php' => "v1\n"]);
114+
115+
// Simulate a previous job's leftovers: a committed PR snapshot (which a
116+
// fresh checkoutBranch() would already have re-pointed the branch away
117+
// from — emulated here by resetting main back to the base commit), a
118+
// stale _base ref, an in-place tracked edit, and an untracked file.
119+
file_put_contents($this->dir . '/a.php', "v2\n");
120+
file_put_contents($this->dir . '/b.php', "added by PR\n");
121+
$this->assertTrue(commitPrSnapshot($this->dir, 99));
122+
$this->git('reset --hard ' . $baseSha); // emulate checkoutBranch sync to fresh base
123+
file_put_contents($this->dir . '/a.php', "agent-edit\n"); // leftover tracked edit
124+
file_put_contents($this->dir . '/c.php', "agent-untracked\n"); // leftover untracked file
125+
126+
$this->assertTrue(resetCheckoutToCleanState($this->dir, 'main'));
127+
128+
// Tree is pristine at the base commit.
129+
$this->assertSame($baseSha, $this->git('rev-parse HEAD'));
130+
$this->assertSame('', $this->git('status --porcelain'), 'working tree must be clean');
131+
$this->assertSame("v1\n", file_get_contents($this->dir . '/a.php'), 'tracked edit reverted');
132+
$this->assertFileDoesNotExist($this->dir . '/c.php', 'untracked file removed');
133+
$this->assertFileDoesNotExist($this->dir . '/b.php', 'snapshot-only file removed');
134+
135+
// The scratch _base ref is gone so a cached checkout cannot drift on it.
136+
$this->assertSame('', $this->git('branch --list _base'), '_base ref must be deleted');
137+
}
138+
}

0 commit comments

Comments
 (0)