feat: Add Constellos review workflow (#55)

sylvansys · claude · web-flow · commit acbd1e0204f8 · 2026-01-22T23:11:51.000-08:00
* feat: Add Constellos review workflow Enable AI-powered code reviews on this repo using Claude. Closes #54 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: Use conditional steps instead of dynamic uses path GitHub Actions doesn't allow template expressions in composite action uses fields. Use if conditions to route to the correct reviewer. Also use local action path (`./ `) in workflow for testing. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: Update workflow to call reviewers directly and post comments - Call reviewer actions directly with if conditions (no router) - Add review-comment step after each review to post PR comments - This fixes output handling issues with composite action routing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: Add workflow permissions for issues and pull-requests The GITHUB_TOKEN needs explicit permissions to: - Read issues (for requirements review context) - Write pull-requests (for posting review comments) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * debug: Add logging to issue detection Add debug output to understand why issue extraction is failing. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: Improve agent prompt clarity for JSON output - Use concrete examples instead of "passed|failed|skipped" placeholders - Add clear instruction to replace example values with actual findings - Remove debug logging from requirements-reviewer The placeholder format was causing Claude to output the literal text instead of choosing one status value. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * debug: Add extraction logging * fix: Extract JSON from all assistant messages, not just last Claude Code may output text in multiple turns. The JSON result could be in any message, not necessarily the last one. Join all text content and search for the JSON block in the combined output. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * debug: Show issue fetch errors instead of suppressing * Session work\n\nAuto-commit at session end to preserve work in progress.\n\nSession-ID: dd9fc88a-7441-49e8-a3ff-88965138fb6f\nSession-Timestamp: 2026-01-23T05:09:58.182Z\nBranch: 54-fix/take-a-look-at-prs-49-53-and-see-how-we-\n\n🤖 Generated with [Claude Code](https://claude.com/claude-code)\n\nCo-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix: Add multi-strategy JSON extraction for reviewer output The requirements reviewer was failing to extract JSON when Claude didn't wrap the output in ```json blocks. This adds fallback strategies: 1. Try ```json block extraction (existing behavior) 2. Fallback: grep for raw JSON object with "checks" key 3. Fallback: perl range extraction for {"checks":[ pattern Fixes requirements reviewer showing "0 passed • 0 failed • 0 skipped" Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: Add debug logging, Python extraction, and improve issue linking - Add debug logging to requirements reviewer to diagnose ALL_TEXT extraction - Add Strategy 4: Python-based multiline JSON extraction for robustness - Add Fallback 3: Detect any #123 reference in PR body (not just closing keywords) - Update tagline to describe Constellos agents and their data sources Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: Use single-line Python to avoid YAML parsing errors The multiline Python code had colons that YAML interpreted as key-value separators, causing action.yml to fail parsing. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: Add imperative instructions to requirements prompt Claude was stopping after 1 turn with just a greeting. Added explicit START IMMEDIATELY instruction and detailed step-by-step process to ensure Claude actually reads files and performs the review. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
diff --git a/.claude/settings.json b/.claude/settings.json
@@ -1,8 +1,4 @@
 {
-  "enabledPlugins": {
-    "github-orchestration@constellos": true,
-    "project-context@constellos": true
-  },
   "hooks": {
     "SessionStart": [
       {
@@ -15,5 +11,9 @@
         ]
       }
     ]
+  },
+  "enabledPlugins": {
+    "github-orchestration@constellos": true,
+    "project-context@constellos": true
   }
 }
diff --git a/.constellos/agents/code-quality.md b/.constellos/agents/code-quality.md
@@ -46,61 +46,58 @@ You must evaluate each of these checks independently:
 
 ## Output Format
 
-**CRITICAL**: Output ONLY the JSON block below. Each check MUST have a status.
+**CRITICAL**: After your review, output a single JSON block with your findings.
 
+For each check, set `status` to exactly one of: `"passed"`, `"failed"`, or `"skipped"`.
+- Use `"passed"` if no issues found
+- Use `"failed"` if any issues found
+- Use `"skipped"` if check is not applicable
+
+Example output:
 ```json
 {
   "checks": [
     {
       "name": "DRY",
-      "status": "passed|failed|skipped",
-      "result": "Brief 1-line result",
-      "reasoning": "Why this status was given",
-      "files": [
-        {
-          "path": "path/to/file.ts",
-          "line": 42,
-          "note": "Specific issue or observation"
-        }
-      ]
+      "status": "passed",
+      "result": "No code duplication found",
+      "reasoning": "Each component has unique logic with no repeated patterns",
+      "files": []
     },
     {
       "name": "YAGNI",
-      "status": "passed|failed|skipped",
-      "result": "Brief 1-line result",
-      "reasoning": "Why this status was given",
+      "status": "passed",
+      "result": "No over-engineering detected",
+      "reasoning": "Implementation matches requirements without unnecessary complexity",
       "files": []
     },
     {
       "name": "Modularity",
-      "status": "passed|failed|skipped",
-      "result": "Brief 1-line result",
-      "reasoning": "Why this status was given",
+      "status": "passed",
+      "result": "Good separation of concerns",
+      "reasoning": "Components have single responsibilities and clear interfaces",
       "files": []
     },
     {
       "name": "Maintainability",
-      "status": "passed|failed|skipped",
-      "result": "Brief 1-line result",
-      "reasoning": "Why this status was given",
+      "status": "passed",
+      "result": "Code is readable and well-structured",
+      "reasoning": "Clear naming, appropriate comments, logical organization",
       "files": []
     },
     {
       "name": "Error Handling",
-      "status": "passed|failed|skipped",
-      "result": "Brief 1-line result",
-      "reasoning": "Why this status was given",
+      "status": "skipped",
+      "result": "No error handling code in changes",
+      "reasoning": "Changes are configuration files only",
       "files": []
     }
   ],
-  "message": "Optional overall assessment (leave empty if not needed)"
+  "message": ""
 }
 ```
 
-## Status Guidelines
-- **passed**: No issues found for this check
-- **failed**: One or more issues found - be strict, fail if there are ANY violations
-- **skipped**: Check not applicable (e.g., no error handling code to review)
+**Important**: Replace the example values above with your actual findings. The status must be one of: `passed`, `failed`, or `skipped`.
 
 ## Severity in Files
 When adding files, prioritize high-severity issues:
diff --git a/.constellos/agents/requirements.md b/.constellos/agents/requirements.md
@@ -2,6 +2,8 @@
 
 You are a Requirements reviewer ensuring PR changes align with issue specifications.
 
+**IMPORTANT**: Start IMMEDIATELY by reading the context files. Do NOT just introduce yourself - perform the actual review.
+
 ## Your Role
 Verify that code changes implement what was requested, nothing more, nothing less.
 
@@ -29,56 +31,54 @@ You must evaluate each of these checks independently:
 - Issue context: `.claude/review-context/issue.json`
 
 ## Review Process
-1. Read the issue requirements carefully
-2. Review each changed file
-3. Evaluate EACH check independently
-4. Mark check as "failed" if ANY issues found
-5. Mark check as "passed" only if NO issues found
-6. Mark check as "skipped" if not applicable (e.g., no issue context)
+1. FIRST: Read `.claude/review-context/issue.json` to understand requirements
+2. THEN: Read `.claude/review-context/changed.txt` to see what files changed
+3. Review each changed file against the issue requirements
+4. Evaluate EACH check independently
+5. Mark check as "failed" if ANY issues found
+6. Mark check as "passed" only if NO issues found
+7. Mark check as "skipped" if not applicable (e.g., no issue context)
+8. Output the JSON result
 
 ## Output Format
 
-**CRITICAL**: Output ONLY the JSON block below. Each check MUST have a status.
+**CRITICAL**: After your review, output a single JSON block with your findings.
+
+For each check, set `status` to exactly one of: `"passed"`, `"failed"`, or `"skipped"`.
+- Use `"passed"` if check criteria are fully satisfied
+- Use `"failed"` if any issues are found
+- Use `"skipped"` if check is not applicable (e.g., no issue context)
 
 ```json
 {
   "checks": [
     {
       "name": "Completeness",
-      "status": "passed|failed|skipped",
-      "result": "Brief 1-line result",
-      "reasoning": "Why this status was given",
-      "files": [
-        {
-          "path": "path/to/file.ts",
-          "line": 42,
-          "note": "Missing requirement X implementation"
-        }
-      ]
+      "status": "passed",
+      "result": "All acceptance criteria implemented",
+      "reasoning": "Each requirement from the issue is addressed in the code changes",
+      "files": []
     },
     {
       "name": "Scope",
-      "status": "passed|failed|skipped",
-      "result": "Brief 1-line result",
-      "reasoning": "Why this status was given",
+      "status": "passed",
+      "result": "Changes within scope",
+      "reasoning": "All changes relate directly to the issue requirements",
       "files": []
     },
     {
       "name": "Traceability",
-      "status": "passed|failed|skipped",
-      "result": "Brief 1-line result",
-      "reasoning": "Why this status was given",
+      "status": "skipped",
+      "result": "No tests required for config change",
+      "reasoning": "This is a workflow configuration change, not a code change requiring tests",
       "files": []
     }
   ],
-  "message": "Optional overall assessment (leave empty if not needed)"
+  "message": ""
 }
 ```
 
-## Status Guidelines
-- **passed**: Check criteria fully satisfied
-- **failed**: One or more issues found - be strict, fail if there are ANY violations
-- **skipped**: Check not applicable (e.g., no issue context available)
+**Important**: Replace the example values above with your actual findings. The status must be one of: `passed`, `failed`, or `skipped`.
 
 ## Guidelines
 - Be strict about scope - extra features should be flagged
diff --git a/.github/actions/code-quality-reviewer/action.yml b/.github/actions/code-quality-reviewer/action.yml
@@ -158,8 +158,21 @@ runs:
 
         EXEC="${{ steps.review.outputs.execution_file }}"
         if [ -f "$EXEC" ]; then
-          TEXT=$(jq -r '[.[] | select(.type=="assistant") | .message.content[]? | select(.type=="text") | .text] | last // ""' "$EXEC" 2>/dev/null || echo "")
-          JSON=$(echo "$TEXT" | sed -n '/```json/,/```/{/```/d;p;}' | head -50)
+          # Get ALL text from assistant messages, joined together
+          ALL_TEXT=$(jq -r '[.[] | select(.type=="assistant") | .message.content[]? | select(.type=="text") | .text] | join("\n")' "$EXEC" 2>/dev/null || echo "")
+          # Strategy 1: Extract from ```json block
+          JSON=$(echo "$ALL_TEXT" | sed -n '/```json/,/```/{/```/d;p;}' | head -100)
+
+          # Strategy 2: If no json block, try to find raw JSON object with "checks" key
+          if ! echo "$JSON" | jq . >/dev/null 2>&1; then
+            JSON=$(echo "$ALL_TEXT" | grep -oP '\{"checks":\s*\[.*\]\s*(,"message":[^}]*)?\}' | head -1)
+          fi
+
+          # Strategy 3: Extract any JSON object starting with {"checks"
+          if ! echo "$JSON" | jq . >/dev/null 2>&1; then
+            JSON=$(echo "$ALL_TEXT" | perl -ne 'print if /\{"checks":\[/.../"message":/' | tr '\n' ' ')
+          fi
+
           if echo "$JSON" | jq . >/dev/null 2>&1; then
             # Count checks by status
             CHECKS_PASSED=$(echo "$JSON" | jq '[.checks[]? | select(.status == "passed")] | length')
diff --git a/.github/actions/requirements-reviewer/action.yml b/.github/actions/requirements-reviewer/action.yml
@@ -83,15 +83,26 @@ runs:
           [ "$ISSUE" = "null" ] && ISSUE=""
         fi
 
+        # Fallback 3: Find any issue reference (#123) in PR body
+        if [ -z "$ISSUE" ]; then
+          # PR_BODY was fetched in Fallback 1, reuse it
+          ISSUE=$(echo "$PR_BODY" | grep -oP '#\K[0-9]+' | head -1 || echo "")
+        fi
+
         echo "issue=$ISSUE" >> $GITHUB_OUTPUT
 
         # Get changed files (gracefully handle permission errors)
         gh pr view ${{ inputs.pr_number }} --json files --jq '.files[].path' > .claude/review-context/changed.txt 2>/dev/null || touch .claude/review-context/changed.txt
 
         # Get issue context if available
         if [ -n "$ISSUE" ]; then
-          gh issue view $ISSUE --json title,body > .claude/review-context/issue.json 2>/dev/null || echo "{}" > .claude/review-context/issue.json
+          echo "Fetching issue $ISSUE..."
+          if ! gh issue view "$ISSUE" --json title,body > .claude/review-context/issue.json; then
+            echo "Warning: Failed to fetch issue $ISSUE"
+            echo "{}" > .claude/review-context/issue.json
+          fi
         else
+          echo "No issue number detected, skipping issue context"
           echo "{}" > .claude/review-context/issue.json
         fi
 
@@ -187,8 +198,34 @@ runs:
 
         EXEC="${{ steps.review.outputs.execution_file }}"
         if [ -f "$EXEC" ]; then
-          TEXT=$(jq -r '[.[] | select(.type=="assistant") | .message.content[]? | select(.type=="text") | .text] | last // ""' "$EXEC" 2>/dev/null || echo "")
-          JSON=$(echo "$TEXT" | sed -n '/```json/,/```/{/```/d;p;}' | head -50)
+          # Debug: Show what we're working with
+          echo "DEBUG: Checking execution file..."
+          echo "DEBUG: File exists, size: $(wc -c < "$EXEC") bytes"
+          echo "DEBUG: Message types in file:"
+          jq -r '.[].type' "$EXEC" 2>/dev/null | sort | uniq -c || echo "DEBUG: jq failed"
+
+          # Get ALL text from assistant messages, joined together
+          ALL_TEXT=$(jq -r '[.[] | select(.type=="assistant") | .message.content[]? | select(.type=="text") | .text] | join("\n")' "$EXEC" 2>/dev/null || echo "")
+          echo "DEBUG: ALL_TEXT length: ${#ALL_TEXT}"
+          echo "DEBUG: ALL_TEXT preview: ${ALL_TEXT:0:500}"
+          # Strategy 1: Extract from ```json block
+          JSON=$(echo "$ALL_TEXT" | sed -n '/```json/,/```/{/```/d;p;}' | head -100)
+
+          # Strategy 2: If no json block, try to find raw JSON object with "checks" key
+          if ! echo "$JSON" | jq . >/dev/null 2>&1; then
+            JSON=$(echo "$ALL_TEXT" | grep -oP '\{"checks":\s*\[.*\]\s*(,"message":[^}]*)?\}' | head -1)
+          fi
+
+          # Strategy 3: Extract any JSON object starting with {"checks"
+          if ! echo "$JSON" | jq . >/dev/null 2>&1; then
+            JSON=$(echo "$ALL_TEXT" | perl -ne 'print if /\{"checks":\[/.../"message":/' | tr '\n' ' ')
+          fi
+
+          # Strategy 4: Use Python for robust multiline JSON extraction
+          if ! echo "$JSON" | jq . >/dev/null 2>&1; then
+            JSON=$(echo "$ALL_TEXT" | python3 -c "import re,sys; t=sys.stdin.read(); m=re.search(r'\{[^{}]*\"checks\"[^{}]*\[.*?\][^{}]*\}',t,re.DOTALL); print(m.group() if m else '')" 2>/dev/null)
+          fi
+
           if echo "$JSON" | jq . >/dev/null 2>&1; then
             # Count checks by status
             CHECKS_PASSED=$(echo "$JSON" | jq '[.checks[]? | select(.status == "passed")] | length')
diff --git a/.github/actions/review-comment/action.yml b/.github/actions/review-comment/action.yml
@@ -194,7 +194,7 @@ runs:
           # Create new comment
           {
             echo "$MARKER"
-            echo "Updates on your PR checks from AI reviewers."
+            echo "PR review updates from your Constellos agents. Based on linked issue(s), context files, and committed changes."
             echo ""
             echo "$AGENT_BLOCK"
           } > /tmp/comment.md
diff --git a/.github/workflows/constellos-review.yml b/.github/workflows/constellos-review.yml
@@ -0,0 +1,92 @@
+# Constellos Review Workflow
+# Runs AI-powered code reviews on PRs using Claude
+
+name: Constellos Review
+on:
+  pull_request:
+    types: [opened, synchronize, reopened]
+
+permissions:
+  contents: read
+  issues: read
+  pull-requests: write
+
+jobs:
+  # Read config and determine which agents to run
+  config:
+    runs-on: ubuntu-latest
+    outputs:
+      agents: ${{ steps.read.outputs.agents }}
+    steps:
+      - uses: actions/checkout@v4
+      - id: read
+        run: |
+          if [ -f ".constellos/config.json" ]; then
+            AGENTS=$(jq -c '[.agents | to_entries[] | select(.value.enabled) | .key]' .constellos/config.json)
+          else
+            # Default: run both agents
+            AGENTS='["requirements","code-quality"]'
+          fi
+          echo "agents=$AGENTS" >> $GITHUB_OUTPUT
+
+  # Run each enabled agent in parallel
+  review:
+    needs: config
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        agent: ${{ fromJson(needs.config.outputs.agents) }}
+    steps:
+      - uses: actions/checkout@v4
+
+      # Requirements reviewer
+      - name: Run requirements review
+        id: requirements
+        if: matrix.agent == 'requirements'
+        uses: ./.github/actions/requirements-reviewer
+        with:
+          claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
+          pr_number: ${{ github.event.number }}
+          branch: ${{ github.head_ref }}
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Post requirements comment
+        if: matrix.agent == 'requirements'
+        uses: ./.github/actions/review-comment
+        with:
+          review_name: Requirements
+          passed: ${{ steps.requirements.outputs.passed }}
+          result_json: ${{ steps.requirements.outputs.result }}
+          checks_passed: ${{ steps.requirements.outputs.checks_passed }}
+          checks_failed: ${{ steps.requirements.outputs.checks_failed }}
+          checks_skipped: ${{ steps.requirements.outputs.checks_skipped }}
+          pr_number: ${{ github.event.number }}
+          sha: ${{ github.event.pull_request.head.sha }}
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          prompt_file: .constellos/agents/requirements.md
+
+      # Code quality reviewer
+      - name: Run code-quality review
+        id: code-quality
+        if: matrix.agent == 'code-quality'
+        uses: ./.github/actions/code-quality-reviewer
+        with:
+          claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
+          pr_number: ${{ github.event.number }}
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Post code-quality comment
+        if: matrix.agent == 'code-quality'
+        uses: ./.github/actions/review-comment
+        with:
+          review_name: Code Quality
+          passed: ${{ steps.code-quality.outputs.passed }}
+          result_json: ${{ steps.code-quality.outputs.result }}
+          checks_passed: ${{ steps.code-quality.outputs.checks_passed }}
+          checks_failed: ${{ steps.code-quality.outputs.checks_failed }}
+          checks_skipped: ${{ steps.code-quality.outputs.checks_skipped }}
+          pr_number: ${{ github.event.number }}
+          sha: ${{ github.event.pull_request.head.sha }}
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          prompt_file: .constellos/agents/code-quality.md
diff --git a/action.yml b/action.yml

Original file line number	Diff line number	Diff line change
`@@ -1,8 +1,4 @@`
`1`	`1`	`{`
`2`		`- "enabledPlugins": {`
`3`		`- "github-orchestration@constellos": true,`
`4`		`- "project-context@constellos": true`
`5`		`- },`
`6`	`2`	`"hooks": {`
`7`	`3`	`"SessionStart": [`
`8`	`4`	`{`
`@@ -15,5 +11,9 @@`
`15`	`11`	`]`
`16`	`12`	`}`
`17`	`13`	`]`
	`14`	`+ },`
	`15`	`+ "enabledPlugins": {`
	`16`	`+ "github-orchestration@constellos": true,`
	`17`	`+ "project-context@constellos": true`
`18`	`18`	`}`
`19`	`19`	`}`
Original file line number	Diff line number	Diff line change
`@@ -194,7 +194,7 @@ runs:`
`194`	`194`	`# Create new comment`
`195`	`195`	`{`
`196`	`196`	`echo "$MARKER"`
`197`		`- echo "Updates on your PR checks from AI reviewers."`
	`197`	`+ echo "PR review updates from your Constellos agents. Based on linked issue(s), context files, and committed changes."`
`198`	`198`	`echo ""`
`199`	`199`	`echo "$AGENT_BLOCK"`
`200`	`200`	`} > /tmp/comment.md`