fix(signals): rewrite centaur_review gate to check GitHub comments#444
fix(signals): rewrite centaur_review gate to check GitHub comments#444dimakis wants to merge 3 commits into
Conversation
The old implementation called a nonexistent Centaur API endpoint (/api/reviews?pr=...) that always 404'd, making the centaur_review signal watch permanently stuck. Now checks GitHub issue comments directly via gh CLI for "## Centaur Review" comments. Parses severity: - LGTM = pass - Critical/warning findings = fail (triggers retry cycle) - Info/style only = pass Also fixes type mismatch: gate config passes repo+pr but the old function expected pr_url. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
dimakis
left a comment
There was a problem hiding this comment.
Centaur Review
Found 4 issue(s) (3 warning).
server/signal-processor.ts
The rewrite from HTTP polling to gh api comment checking is a good direction, but has a null-handling bug on empty results, a regression risk for existing pr_url-format gate configs, and lacks tests for the new logic.
- 🟡 bugs (L271): When no comments match the
select, jq evaluates[] | last→null. stdout becomes"null", which passes the!stdout.trim()check.JSON.parse("null")returnsnull, andnull.bodythrows a TypeError. The function still returns the correct result (caught bycatch→resolved: false), but it relies on an exception for normal control flow. Guard against this withif (!stdout.trim() || stdout.trim() === 'null').[fixable] - 🟡 regressions (L263): Existing gate configs using the
pr_urlformat (e.g.{ type: 'centaur_review', pr_url: 'https://...' }) will fail during polling —config.repoandconfig.prwill beundefined, producing a badgh apipath. The REST resolve endpoint inapp.ts:1048and the test at signal-processor.test.ts:251 still supportpr_urlmatching, creating an inconsistency. Either migrate the resolve endpoint and tests away frompr_url, or handle both formats incheckCentaurReview.[fixable] - 🔵 unsafe_assumptions (L279):
body.includes('LGTM')is checked before the critical/warning regexes. If a review contains both 'LGTM' and findings (e.g. "LGTM overall but 1 warning about..."), it will short-circuit to pass. Consider checking for critical/warning first, or making the LGTM check more precise (e.g. checking for"LGTM"on its own line or as a standalone verdict).[fixable]
server/__tests__/signal-processor.test.ts
The rewrite from HTTP polling to gh api comment checking is a good direction, but has a null-handling bug on empty results, a regression risk for existing pr_url-format gate configs, and lacks tests for the new logic.
- 🟡 missing_tests: No unit tests cover the new
checkCentaurReviewimplementation. Key untested paths: LGTM → pass, critical/warning findings → fail, info-only findings → pass, no matching comment → not resolved, andgh apifailure → not resolved. The existing tests only coverSignalProcessorlifecycle andTaskStore.findActiveSignalTasks, not the gate checker functions. Given the project's TDD requirement, tests should accompany this change.[fixable]
Reviewed at 2579772
| 'api', | ||
| `repos/${config.repo}/issues/${config.pr}/comments`, | ||
| '--jq', | ||
| '[.[] | select(.body | startswith("## Centaur Review")) | {body: .body, created_at: .created_at}] | last', |
There was a problem hiding this comment.
🟡 bugs: When no comments match the select, jq evaluates [] | last → null. stdout becomes "null", which passes the !stdout.trim() check. JSON.parse("null") returns null, and null.body throws a TypeError. The function still returns the correct result (caught by catch → resolved: false), but it relies on an exception for normal control flow. Guard against this with if (!stdout.trim() || stdout.trim() === 'null'). [fixable]
| } | ||
|
|
||
| async function checkCentaurReview(config: { pr_url: string }): Promise<GateResult> { | ||
| async function checkCentaurReview( |
There was a problem hiding this comment.
🟡 regressions: Existing gate configs using the pr_url format (e.g. { type: 'centaur_review', pr_url: 'https://...' }) will fail during polling — config.repo and config.pr will be undefined, producing a bad gh api path. The REST resolve endpoint in app.ts:1048 and the test at signal-processor.test.ts:251 still support pr_url matching, creating an inconsistency. Either migrate the resolve endpoint and tests away from pr_url, or handle both formats in checkCentaurReview. [fixable]
| const body = comment.body; | ||
|
|
||
| // LGTM with no issues = pass | ||
| if (body.includes('LGTM')) { |
There was a problem hiding this comment.
🔵 unsafe_assumptions: body.includes('LGTM') is checked before the critical/warning regexes. If a review contains both 'LGTM' and findings (e.g. "LGTM overall but 1 warning about..."), it will short-circuit to pass. Consider checking for critical/warning first, or making the LGTM check more precise (e.g. checking for "LGTM" on its own line or as a standalone verdict). [fixable]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
dimakis
left a comment
There was a problem hiding this comment.
Centaur Review
Found 5 issue(s) (3 warning).
server/signal-processor.ts
The rewrite from HTTP-fetch to gh api is a good direction, but has a null-dereference bug on empty results (accidentally caught), a backward-compatibility gap with pr_url-based gate configs, and no test coverage for the new logic.
- 🟡 bugs (L274): When no Centaur Review comment exists on the PR, the jq filter
[...] | lastoutputs the stringnull(not empty)."null".trim()is truthy so the empty-check passes,JSON.parse("null")returns JSnull, andnull.bodythrows a TypeError. The catch block returns the correct result (resolved: false), so behavior is accidentally correct — but relying on an exception for a normal control flow path is fragile. Add an explicit null check afterJSON.parse, e.g.if (!comment) return { resolved: false, status: 'fail' };.[fixable] - 🟡 regressions (L263): The function signature changed from
{ pr_url: string }to{ repo: string; pr: number | string }, but the signal resolve endpoint inapp.ts:1048still supports matchingcentaur_reviewtasks bypr_url, and the test atsignal-processor.test.ts:251creates gate configs with onlypr_url. If any existing tasks were created withpr_url-only configs, polling viacheckGatewill cast them to{ repo, pr }(bothundefined), causinggh api repos/undefined/issues/undefined/commentsto fail silently every poll cycle. Either migrate the resolve endpoint and tests to droppr_urlsupport, or handle both config shapes incheckCentaurReview.[fixable] - 🔵 unsafe_assumptions (L280):
body.includes('LGTM')is a loose substring match. If a review contains 'LGTM' anywhere in a finding description or quoted text (e.g., "the user expects LGTM but..."), this would incorrectly classify a review-with-findings as a pass. Consider anchoring to the summary section — e.g., matching only the summary line pattern like/^.*LGTM/mafter the header, or checking a structured section.[fixable] - 🔵 style (L285): The regexes
/\d+\s+critical/and/\d+\s+warning/are tightly coupled to the Centaur review output format (e.g.,"2 critical, 3 warning"). If the format changes (e.g., to"critical: 2"), these silently stop matching and all reviews fall through to the info-only pass path. Consider extracting these patterns to constants or adding a comment noting the expected format contract.[fixable]
server/__tests__/signal-processor.test.ts
The rewrite from HTTP-fetch to gh api is a good direction, but has a null-dereference bug on empty results (accidentally caught), a backward-compatibility gap with pr_url-based gate configs, and no test coverage for the new logic.
- 🟡 missing_tests: No unit tests cover the new
checkCentaurReviewimplementation. The existing tests exerciseSignalProcessorlifecycle and task-store matching but never invokecheckGate/checkCentaurReviewdirectly. The three new code paths (LGTM → pass, critical/warning → fail, info-only → pass) and the null/empty edge case all lack coverage. These would needexecFileAsyncto be mocked (same pattern as would be needed forcheckGhCi/checkGhReview).[fixable]
Reviewed at 81a2345
| '--jq', | ||
| '[.[] | select(.body | startswith("## Centaur Review")) | {body: .body, created_at: .created_at}] | last', | ||
| ]); | ||
| if (!stdout.trim()) return { resolved: false, status: 'fail' }; |
There was a problem hiding this comment.
🟡 bugs: When no Centaur Review comment exists on the PR, the jq filter [...] | last outputs the string null (not empty). "null".trim() is truthy so the empty-check passes, JSON.parse("null") returns JS null, and null.body throws a TypeError. The catch block returns the correct result (resolved: false), so behavior is accidentally correct — but relying on an exception for a normal control flow path is fragile. Add an explicit null check after JSON.parse, e.g. if (!comment) return { resolved: false, status: 'fail' };. [fixable]
| } | ||
|
|
||
| async function checkCentaurReview(config: { pr_url: string }): Promise<GateResult> { | ||
| async function checkCentaurReview(config: { |
There was a problem hiding this comment.
🟡 regressions: The function signature changed from { pr_url: string } to { repo: string; pr: number | string }, but the signal resolve endpoint in app.ts:1048 still supports matching centaur_review tasks by pr_url, and the test at signal-processor.test.ts:251 creates gate configs with only pr_url. If any existing tasks were created with pr_url-only configs, polling via checkGate will cast them to { repo, pr } (both undefined), causing gh api repos/undefined/issues/undefined/comments to fail silently every poll cycle. Either migrate the resolve endpoint and tests to drop pr_url support, or handle both config shapes in checkCentaurReview. [fixable]
| const body = comment.body; | ||
|
|
||
| // LGTM with no issues = pass | ||
| if (body.includes('LGTM')) { |
There was a problem hiding this comment.
🔵 unsafe_assumptions: body.includes('LGTM') is a loose substring match. If a review contains 'LGTM' anywhere in a finding description or quoted text (e.g., "the user expects LGTM but..."), this would incorrectly classify a review-with-findings as a pass. Consider anchoring to the summary section — e.g., matching only the summary line pattern like /^.*LGTM/m after the header, or checking a structured section. [fixable]
| return { resolved: true, status: 'fail', artifacts: { review: data.review } }; | ||
|
|
||
| // Has findings — resolved (review exists) but status depends on severity | ||
| const hasCritical = /\d+\s+critical/.test(body); |
There was a problem hiding this comment.
🔵 style: The regexes /\d+\s+critical/ and /\d+\s+warning/ are tightly coupled to the Centaur review output format (e.g., "2 critical, 3 warning"). If the format changes (e.g., to "critical: 2"), these silently stop matching and all reviews fall through to the info-only pass path. Consider extracting these patterns to constants or adding a comment noting the expected format contract. [fixable]
- Fix null handling: guard against jq returning "null" string for empty results - Add pr_url backward compatibility: parse legacy pr_url configs into repo+pr - Check severity before LGTM: prevents false pass when review contains both - Add format contract comment for severity regexes - Add 10 unit tests for checkCentaurReview covering all code paths Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary
checkCentaurReviewgate polled a nonexistent Centaur API endpoint (/api/reviews?pr=...), making thecentaur_reviewsignal watch permanently stuckgh apifor## Centaur Reviewcommentsrepo+prbut old function expectedpr_urlTest plan
centaur_reviewgate resolves🤖 Generated with Claude Code