Fix: Review checks pass in comment but fail in GitHub checks

[sylvansys]

Branch: 71-docs/can-you-please-figure-out-why-the-conste
Plan file: /home/ben/.claude/plans/modular-prancing-tiger.md

---

Fix: Review checks pass in comment but fail in GitHub checks

Root Cause Analysis

Three separate bugs combine to produce the user-visible symptoms:

Bug A: extract-result.sh crashes under set -e -o pipefail

GitHub Actions runs bash with --noprofile --norc -e -o pipefail. The extraction strategies use commands that return non-zero when they find no match:

• Strategy 2: grep -oP returns exit 1 when no match → pipefail propagates → set -e kills the script
• Strategy 3: perl can return non-zero similarly
• Strategy 4: Python regex is fundamentally broken (the [^{}]* before \[ consumes the [ character; the non-greedy \[.*?\] stops at "files": [] inside the first check object rather than the array's closing ])

When the script crashes mid-extraction, partial state may remain, or the function returns with defaults that don't accurately reflect the review result.

Bug B: Webhook step skipped when review action fails

The workflow template uses if: matrix.agent == 'code-quality' on webhook steps. GitHub Actions implicitly prepends success() to if conditions that don't include a status function, making it effectively if: success() && matrix.agent == 'code-quality'. When Bug A causes the review action to exit 1, the webhook step is skipped, so the comment is never updated with the latest results (showing stale data from a previous run).

Bug C: Missing pull-requests: read permission in installed workflows

The template has the correct pull-requests: read, but the installed workflows on external repos have actions: read instead. Without this permission, gh pr view fails silently (errors suppressed with 2>/dev/null), changed.txt is empty, and all checks are skipped with "No changed files found". This is why PR 67 shows all checks as skipped.

---

Changes

1. Rewrite extract_review_result in extract-result.sh

File: .github/actions/shared/extract-result.sh

Changes to extract_review_result function (lines 38-87):

• Strategy 1 (json code blocks): Replace the sed command that concatenates ALL json blocks with logic that extracts each block separately, validates each with jq -e '.checks', and takes the LAST valid one (the actual result comes after any examples Claude might output)
• Strategy 2 (grep): Add || true to prevent grep exit code 1 from crashing under set -e
• Strategy 3 (perl): Add || true guard
• Strategy 4 (python): Replace the broken regex with a brace-depth-counting JSON extractor that uses json.loads for validation, add || true guard
• All validation: Change jq . to jq -e '.checks' to verify the JSON actually has the expected structure (not just any valid JSON)
• Logging: Add strategy result logging for debugging

2. Fix webhook step conditions in template

File: .github/templates/constellos-review.yml

Changes to all three webhook steps (lines 57, 104, 152):

Change each if: condition from:

if: matrix.agent == 'requirements'

to:

if: always() && matrix.agent == 'requirements'

This ensures the webhook fires even when the review action fails, so the comment is always updated with current results.

Add default value guards in each webhook run: block to handle missing outputs when the review step crashes before setting them.

3. Add permission comment in template

File: .github/templates/constellos-review.yml

Add a comment on the pull-requests: read permission line to flag it as critical for future provisioning:

pull-requests: read  # REQUIRED: agents need this to fetch changed files via gh pr view

4. Re-provision installed workflows (operational — not a code change)

After merging, the installed workflows on constellos/claude-code-plugins and constellos/constellos need to be updated to replace actions: read with pull-requests: read.

---

Files to Modify

1. .github/actions/shared/extract-result.sh — rewrite extract_review_result
2. .github/templates/constellos-review.yml — webhook conditions + output guards + permission comment

Verification

1. Extraction fix: Construct test cases by creating mock execution JSON files and running the extraction function. Test with: single json block, multiple json blocks, no json blocks (raw JSON), empty file, malformed JSON.
2. Webhook fix: After deploying, trigger a review on a test PR where Claude's output has edge-case formatting. Verify the webhook fires and comment is updated even when the review action reports failures.
3. Permission fix: After re-provisioning, verify the installed workflow has pull-requests: read and that gh pr view succeeds in CI logs.

[sylvansys]

📝 Plan updated (v1)

Timestamp: 2026-01-30T03:26:48.840Z
Plan file: /home/ben/.claude/plans/modular-prancing-tiger.md
Branch: 71-docs/can-you-please-figure-out-why-the-conste

---

Plan synced automatically via PostToolUse hook

[sylvansys]

Session Closed - No PR Created\n\nSession ID: \nReason: Session was cleared\nClosed at: 2026-01-30T05:22:05.357Z\n\nThis issue was automatically closed because the session ended without creating a pull request.\n\nReopen this issue to continue work.\n\n---\nClosed automatically via SessionEnd hook