Skip to content

Commit e2ffcfc

Browse files
MarkusNeusingerMarkusNeusinger
authored
Merge pull request #4 from MarkusNeusinger/claude/github-issue-poc-plan-01QbVSDkhbSCB2w6qKJe59m5
feat: add complete automation infrastructure for plot generation
2 parents 6a83a7c + 53abd76 commit e2ffcfc

File tree

11 files changed

+1937
-5
lines changed

11 files changed

+1937
-5
lines changed

‎.github/workflows/claude.yml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,16 +19,16 @@ jobs:
1919
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
2020
runs-on: ubuntu-latest
2121
permissions:
22-
contents: read
23-
pull-requests: read
24-
issues: read
22+
contents: write # Allow commits
23+
pull-requests: write # Allow PR creation
24+
issues: write # Allow issue comments
2525
id-token: write
26-
actions: read # Required for Claude to read CI results on PRs
26+
actions: read # Required for Claude to read CI results on PRs
2727
steps:
2828
- name: Checkout repository
2929
uses: actions/checkout@v4
3030
with:
31-
fetch-depth: 1
31+
fetch-depth: 0 # Full history for better context
3232

3333
- name: Run Claude Code
3434
id: claude

‎.github/workflows/quality-check.yml‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
name: Quality Check
2+
3+
on:
4+
workflow_run:
5+
workflows: ["Test and Generate Previews"]
6+
types: [completed]
7+
8+
jobs:
9+
quality-check:
10+
name: AI Quality Evaluation
11+
runs-on: ubuntu-latest
12+
if: ${{ github.event.workflow_run.conclusion == 'success' }}
13+
permissions:
14+
contents: read
15+
pull-requests: write
16+
issues: write
17+
id-token: write
18+
19+
steps:
20+
- name: Checkout code
21+
uses: actions/checkout@v4
22+
with:
23+
fetch-depth: 0
24+
25+
- name: Download preview metadata
26+
uses: actions/download-artifact@v4
27+
with:
28+
name: preview-metadata
29+
run-id: ${{ github.event.workflow_run.id }}
30+
github-token: ${{ secrets.GITHUB_TOKEN }}
31+
32+
- name: Load metadata
33+
id: metadata
34+
run: |
35+
if [ ! -f preview_metadata.json ]; then
36+
echo "No preview metadata found, skipping quality check"
37+
exit 0
38+
fi
39+
40+
PR_NUMBER=$(jq -r '.pr_number' preview_metadata.json)
41+
BUCKET=$(jq -r '.bucket' preview_metadata.json)
42+
BASE_PATH=$(jq -r '.base_path' preview_metadata.json)
43+
CHANGED_FILES=$(jq -r '.changed_files[]' preview_metadata.json | tr '\n' ' ')
44+
45+
echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT
46+
echo "bucket=$BUCKET" >> $GITHUB_OUTPUT
47+
echo "base_path=$BASE_PATH" >> $GITHUB_OUTPUT
48+
echo "changed_files=$CHANGED_FILES" >> $GITHUB_OUTPUT
49+
50+
- name: Setup Google Cloud authentication
51+
uses: google-github-actions/auth@v2
52+
with:
53+
credentials_json: ${{ secrets.GCS_CREDENTIALS }}
54+
55+
- name: Download preview images
56+
run: |
57+
mkdir -p preview_images
58+
gsutil -m cp -r "gs://${{ steps.metadata.outputs.bucket }}/${{ steps.metadata.outputs.base_path }}/*" preview_images/ || echo "No images to download"
59+
60+
- name: Trigger quality check with @claude
61+
uses: actions/github-script@v7
62+
with:
63+
script: |
64+
const prNumber = ${{ steps.metadata.outputs.pr_number }};
65+
const bucket = '${{ steps.metadata.outputs.bucket }}';
66+
const basePath = '${{ steps.metadata.outputs.base_path }}';
67+
68+
await github.rest.issues.createComment({
69+
owner: context.repo.owner,
70+
repo: context.repo.repo,
71+
issue_number: prNumber,
72+
body: `@claude
73+
74+
## 🎯 Task: Quality Evaluation
75+
76+
Preview images have been generated and uploaded to GCS. Please evaluate the implementations.
77+
78+
### Instructions
79+
80+
1. **Preview images location**
81+
- Already downloaded to \`preview_images/\` directory in this repository
82+
- GCS Bucket: \`${bucket}\`
83+
- GCS Path: \`${basePath}/\`
84+
85+
2. **For each preview image:**
86+
- Parse filename to extract: spec_id, library, variant (format: \`{spec_id}_{library}_{variant}.png\`)
87+
- Read corresponding spec file: \`specs/{spec_id}.md\`
88+
- View the preview image carefully using your vision capabilities
89+
- Evaluate against quality criteria listed in the spec
90+
91+
3. **Check each quality criterion:**
92+
- Does it meet ALL quality criteria from the spec?
93+
- Are visual elements clear and readable?
94+
- Are colors appropriate and accessible?
95+
- Is the layout well-structured?
96+
- Score: 0-100 (≥85 to pass)
97+
98+
4. **Generate quality report with:**
99+
- Overall verdict (PASS if all implementations score ≥85, FAIL otherwise)
100+
- Score for each implementation (matplotlib and seaborn separately)
101+
- Specific feedback for each quality criterion (met/failed)
102+
- Strengths and improvements needed
103+
- Concrete, actionable suggestions if FAIL
104+
105+
5. **Update this PR:**
106+
- Post quality report as comment on this PR
107+
- Use \`gh pr edit\` to add appropriate label:
108+
- "quality-approved" if PASS
109+
- "quality-check-failed" if FAIL
110+
111+
### Scoring Guidelines
112+
- 90-100: Excellent - All criteria met, production ready
113+
- 85-89: Good - Minor issues, acceptable
114+
- 75-84: Needs improvement - regeneration recommended
115+
- <75: Rejected - Major issues, regeneration required
116+
117+
Be objective and constructive in your feedback. Focus on measurable criteria.
118+
119+
Preview images are ready - please proceed with the evaluation!`
120+
});

‎.github/workflows/spec-to-code.yml‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
name: Generate Code from Spec
2+
3+
on:
4+
issues:
5+
types: [labeled]
6+
7+
jobs:
8+
trigger-claude-code:
9+
# Only run when "approved" label is added
10+
if: github.event.label.name == 'approved'
11+
runs-on: ubuntu-latest
12+
permissions:
13+
issues: write
14+
15+
steps:
16+
- name: Extract spec ID from issue
17+
id: extract_spec
18+
env:
19+
ISSUE_BODY: ${{ github.event.issue.body }}
20+
ISSUE_TITLE: ${{ github.event.issue.title }}
21+
run: |
22+
# Try to extract spec ID from title (format: "scatter-basic-001: Title" or just "scatter-basic-001")
23+
# Case-insensitive, allows 3-4 digits, converts to lowercase
24+
SPEC_ID=$(echo "$ISSUE_TITLE" | grep -oiP '^[a-z]+-[a-z]+-\d{3,4}' | tr '[:upper:]' '[:lower:]' || echo "")
25+
26+
if [ -z "$SPEC_ID" ]; then
27+
# Try to find spec ID in body (look for markdown heading like "# scatter-basic-001:")
28+
SPEC_ID=$(echo "$ISSUE_BODY" | grep -oiP '^#\s*\K[a-z]+-[a-z]+-\d{3,4}' | tr '[:upper:]' '[:lower:]' || echo "")
29+
fi
30+
31+
if [ -z "$SPEC_ID" ]; then
32+
echo "❌ Could not extract spec ID from issue"
33+
echo "Expected format: {type}-{variant}-{001-9999}"
34+
exit 1
35+
fi
36+
37+
# Validate spec file exists
38+
if [ ! -f "specs/${SPEC_ID}.md" ]; then
39+
echo "⚠️ Warning: Spec file specs/${SPEC_ID}.md does not exist"
40+
echo "Please ensure the spec file is created before code generation"
41+
fi
42+
43+
echo "spec_id=$SPEC_ID" >> $GITHUB_OUTPUT
44+
echo "✅ Extracted spec ID: $SPEC_ID"
45+
46+
- name: Trigger Claude Code with @claude comment
47+
uses: actions/github-script@v7
48+
with:
49+
script: |
50+
const specId = '${{ steps.extract_spec.outputs.spec_id }}';
51+
52+
await github.rest.issues.createComment({
53+
owner: context.repo.owner,
54+
repo: context.repo.repo,
55+
issue_number: context.issue.number,
56+
body: `@claude
57+
58+
## 🎯 Task: Generate Plot Implementations
59+
60+
This issue has been approved! Please generate implementations for **${specId}**.
61+
62+
### Instructions
63+
64+
1. **Read the specification**
65+
- File: \`specs/${specId}.md\`
66+
- If file doesn't exist, create it from the issue body
67+
68+
2. **Read generation rules**
69+
- \`rules/generation/v1.0.0-draft/code-generation-rules.md\`
70+
- \`rules/generation/v1.0.0-draft/quality-criteria.md\`
71+
- \`rules/generation/v1.0.0-draft/self-review-checklist.md\`
72+
73+
3. **Generate matplotlib implementation**
74+
- Target: \`plots/matplotlib/scatter/${specId}/default.py\`
75+
- Follow generation rules exactly
76+
- Implement all spec requirements
77+
- Use type hints, docstrings, validation
78+
- Add standalone execution block (\`if __name__ == '__main__':\`)
79+
- Self-review: Max 3 attempts to pass quality criteria
80+
81+
4. **Generate seaborn implementation**
82+
- Target: \`plots/seaborn/scatterplot/${specId}/default.py\`
83+
- Same requirements as matplotlib
84+
85+
5. **Create Pull Request**
86+
- Branch: \`auto/${specId}\`
87+
- Title: \`feat: implement ${specId}\`
88+
- Include summary of generation results
89+
- Link to this issue
90+
91+
### Requirements
92+
- Each implementation must pass self-review before moving to next
93+
- Use deterministic sample data (fixed seeds or hardcoded data)
94+
- Code must work standalone (no external files)
95+
- Follow Python 3.10+ syntax
96+
- Commit with clear messages
97+
98+
### Expected Outcome
99+
- PR created with matplotlib and seaborn implementations
100+
- Both pass self-review
101+
- Tests will run automatically on PR
102+
103+
Please proceed with the generation!`
104+
});

0 commit comments

Comments
 (0)