Skip to content

Commit 3ffa4b2

Browse files
MarkusNeusingerMarkusNeusinger
authored
fix(workflows): improve robustness of impl-generate and impl-merge (#1308)
## Summary Fixes several workflow issues discovered during batch processing of spec-ready issues. ### Fix #1: Branch-Not-Found Errors **Problem:** `fatal: 'origin/implementation/{spec}/{library}' is not a commit` errors when workflow tries to checkout a non-existent remote branch. **Solution:** Check if remote branch exists before checkout, fall back to creating fresh branch from main. ### Fix #2: Duplicate Labels **Problem:** Issues accumulate both `impl:X:pending` and `impl:X:failed` labels when generation fails. **Solution:** Failure handler removes both `generate:X` and `impl:X:pending` when marking as failed. ### Fix #3: Auto-Close with Failures **Problem:** Issues with 8 done + 1 failed stay OPEN because auto-close only triggers on 9 done. **Solution:** Close when `done + failed = 9`, with appropriate summary (shows which libraries failed). ### Fix #4: Generation Failure Tracking **Problem:** When `impl-generate` fails (no plot.png), no PR is created → no review → no repair → library stays `pending` forever. **Solution:** Track generation failures and mark as `impl:X:failed` after 3 consecutive failures. Posts comment explaining the library may not support this plot type. ## Files Changed - `.github/workflows/impl-generate.yml` - Fixes #1, #2, #4 - `.github/workflows/impl-merge.yml` - Fix #3 ## Testing - YAML syntax validated - Logic reviewed against observed failure patterns
1 parent 02e7353 commit 3ffa4b2

File tree

2 files changed

+98
-19
lines changed

2 files changed

+98
-19
lines changed

‎.github/workflows/impl-generate.yml‎

Lines changed: 63 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -342,7 +342,15 @@ jobs:
342342
343343
# Sync with remote (Claude already pushed the code)
344344
git fetch origin
345-
git checkout -B "$BRANCH" "origin/$BRANCH"
345+
346+
# Check if remote branch exists before checkout (fixes branch-not-found error)
347+
if git ls-remote --exit-code --heads origin "$BRANCH" >/dev/null 2>&1; then
348+
git checkout -B "$BRANCH" "origin/$BRANCH"
349+
else
350+
# Branch doesn't exist on remote - create fresh from main
351+
echo "::warning::Remote branch $BRANCH not found, creating fresh from main"
352+
git checkout -B "$BRANCH" origin/main
353+
fi
346354
347355
# Restore plot files after git operations
348356
cp "/tmp/plot.png" "plots/${SPEC_ID}/implementations/plot.png" 2>/dev/null || true
@@ -613,3 +621,57 @@ jobs:
613621
else
614622
echo "success=false" >> $GITHUB_OUTPUT
615623
fi
624+
625+
# ========================================================================
626+
# Failure handling: Track failures and clean up labels
627+
# ========================================================================
628+
- name: Handle generation failure
629+
if: failure() && steps.inputs.outputs.issue_number != ''
630+
env:
631+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
632+
SPEC_ID: ${{ steps.inputs.outputs.specification_id }}
633+
LIBRARY: ${{ steps.inputs.outputs.library }}
634+
ISSUE: ${{ steps.inputs.outputs.issue_number }}
635+
run: |
636+
echo "::notice::Handling generation failure for $LIBRARY/$SPEC_ID"
637+
638+
# Count previous failed generation runs for this spec/library
639+
# Filter by both library AND spec ID to avoid counting unrelated failures
640+
FAILURE_COUNT=$(gh run list \
641+
--workflow=impl-generate.yml \
642+
--limit 20 \
643+
--json conclusion,displayTitle,createdAt \
644+
-q "[.[] | select(.conclusion == \"failure\") | select(.displayTitle | (contains(\"$LIBRARY\") and contains(\"$SPEC_ID\")))] | length" \
645+
2>/dev/null || echo "0")
646+
647+
echo "::notice::Previous failures: $FAILURE_COUNT"
648+
649+
# After 3 failures, mark as failed (current run is the 3rd)
650+
if [ "$FAILURE_COUNT" -ge 2 ]; then
651+
echo "::warning::Marking $LIBRARY as failed after multiple generation failures"
652+
653+
# Create failed label if needed
654+
gh label create "impl:${LIBRARY}:failed" --color "d73a4a" \
655+
--description "${LIBRARY} implementation failed" 2>/dev/null || true
656+
657+
# Remove stale labels and add failed (Fix #2: clean up duplicates)
658+
gh issue edit "$ISSUE" \
659+
--remove-label "generate:${LIBRARY},impl:${LIBRARY}:pending" \
660+
--add-label "impl:${LIBRARY}:failed" 2>/dev/null || true
661+
662+
# Post failure comment
663+
gh issue comment "$ISSUE" --body "## :x: ${LIBRARY} Failed
664+
665+
The **${LIBRARY}** implementation for \`${SPEC_ID}\` failed after multiple attempts.
666+
667+
This may indicate that the library cannot implement this plot type natively.
668+
669+
---
670+
:robot: *[impl-generate](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})*"
671+
672+
else
673+
# Not enough failures yet - just clean up stale generate label
674+
# Keep pending label for retry
675+
gh issue edit "$ISSUE" --remove-label "generate:${LIBRARY}" 2>/dev/null || true
676+
echo "::notice::Cleaned up generate label, keeping pending for retry"
677+
fi

‎.github/workflows/impl-merge.yml‎

Lines changed: 35 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -245,39 +245,56 @@ jobs:
245245
# Get current labels on the issue
246246
LABELS=$(gh issue view "$ISSUE" --json labels -q '.labels[].name' 2>/dev/null || echo "")
247247
248-
# Count done implementations
248+
# Count done and failed implementations
249249
DONE_COUNT=0
250+
FAILED_COUNT=0
251+
DONE_LIBS=""
252+
FAILED_LIBS=""
253+
250254
for lib in $LIBRARIES; do
251255
if echo "$LABELS" | grep -q "^impl:${lib}:done$"; then
252256
DONE_COUNT=$((DONE_COUNT + 1))
257+
DONE_LIBS="$DONE_LIBS $lib"
258+
elif echo "$LABELS" | grep -q "^impl:${lib}:failed$"; then
259+
FAILED_COUNT=$((FAILED_COUNT + 1))
260+
FAILED_LIBS="$FAILED_LIBS $lib"
253261
fi
254262
done
255263
256-
echo "::notice::Libraries done: $DONE_COUNT/9"
264+
TOTAL=$((DONE_COUNT + FAILED_COUNT))
265+
echo "::notice::Libraries: $DONE_COUNT done, $FAILED_COUNT failed, $TOTAL/9 total"
266+
267+
# Close issue if all 9 libraries are done OR done+failed=9
268+
if [ "$TOTAL" -eq 9 ]; then
269+
# Build status table
270+
TABLE="| Library | Status |\n|---------|--------|"
271+
for lib in $LIBRARIES; do
272+
if echo "$DONE_LIBS" | grep -w -q "$lib"; then
273+
TABLE="$TABLE\n| $lib | :white_check_mark: |"
274+
elif echo "$FAILED_LIBS" | grep -w -q "$lib"; then
275+
TABLE="$TABLE\n| $lib | :x: (not supported) |"
276+
fi
277+
done
278+
279+
if [ "$FAILED_COUNT" -eq 0 ]; then
280+
TITLE=":tada: All Implementations Complete!"
281+
SUMMARY="All 9 library implementations for \`${SPEC_ID}\` have been successfully merged."
282+
else
283+
TITLE=":white_check_mark: Implementations Complete"
284+
SUMMARY="${DONE_COUNT}/9 implementations merged, ${FAILED_COUNT} libraries could not implement this plot type."
285+
fi
257286
258-
# Close issue if all 9 libraries are done
259-
if [ "$DONE_COUNT" -eq 9 ]; then
260-
gh issue comment "$ISSUE" --body "## :tada: All Implementations Complete!
287+
gh issue comment "$ISSUE" --body "## ${TITLE}
261288
262-
All 9 library implementations for \`${SPEC_ID}\` have been successfully merged.
289+
${SUMMARY}
263290
264-
| Library | Status |
265-
|---------|--------|
266-
| matplotlib | :white_check_mark: |
267-
| seaborn | :white_check_mark: |
268-
| plotly | :white_check_mark: |
269-
| bokeh | :white_check_mark: |
270-
| altair | :white_check_mark: |
271-
| plotnine | :white_check_mark: |
272-
| pygal | :white_check_mark: |
273-
| highcharts | :white_check_mark: |
274-
| letsplot | :white_check_mark: |
291+
$(echo -e "$TABLE")
275292
276293
---
277294
:robot: *[impl-merge](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})*"
278295
279296
gh issue close "$ISSUE"
280-
echo "::notice::Closed issue #$ISSUE - all implementations complete"
297+
echo "::notice::Closed issue #$ISSUE - all implementations complete ($DONE_COUNT done, $FAILED_COUNT failed)"
281298
fi
282299
283300
- name: Trigger database sync

0 commit comments

Comments
 (0)