Skip to content

fix(workflows): improve robustness of impl-generate and impl-merge #1308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MarkusNeusinger merged 2 commits into from
Dec 17, 2025
+98 −19
Merged

fix(workflows): improve robustness of impl-generate and impl-merge #1308

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
67c5f0e
fix(workflows): improve robustness of impl-generate and impl-merge
MarkusNeusinger Dec 17, 2025
132042c
fix: address Copilot review feedback
MarkusNeusinger Dec 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 63 additions & 1 deletion .github/workflows/impl-generate.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,15 @@ jobs:

# Sync with remote (Claude already pushed the code)
git fetch origin
git checkout -B "$BRANCH" "origin/$BRANCH"

# Check if remote branch exists before checkout (fixes branch-not-found error)
if git ls-remote --exit-code --heads origin "$BRANCH" >/dev/null 2>&1; then
git checkout -B "$BRANCH" "origin/$BRANCH"
else
# Branch doesn't exist on remote - create fresh from main
echo "::warning::Remote branch $BRANCH not found, creating fresh from main"
git checkout -B "$BRANCH" origin/main
fi

# Restore plot files after git operations
cp "/tmp/plot.png" "plots/${SPEC_ID}/implementations/plot.png" 2>/dev/null || true
Expand Down Expand Up @@ -613,3 +621,57 @@ jobs:
else
echo "success=false" >> $GITHUB_OUTPUT
fi

# ========================================================================
# Failure handling: Track failures and clean up labels
# ========================================================================
- name: Handle generation failure
if: failure() && steps.inputs.outputs.issue_number != ''
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SPEC_ID: ${{ steps.inputs.outputs.specification_id }}
LIBRARY: ${{ steps.inputs.outputs.library }}
ISSUE: ${{ steps.inputs.outputs.issue_number }}
run: |
echo "::notice::Handling generation failure for $LIBRARY/$SPEC_ID"

# Count previous failed generation runs for this spec/library
# Filter by both library AND spec ID to avoid counting unrelated failures
FAILURE_COUNT=$(gh run list \
--workflow=impl-generate.yml \
--limit 20 \
--json conclusion,displayTitle,createdAt \
-q "[.[] | select(.conclusion == \"failure\") | select(.displayTitle | (contains(\"$LIBRARY\") and contains(\"$SPEC_ID\")))] | length" \
2>/dev/null || echo "0")

echo "::notice::Previous failures: $FAILURE_COUNT"

# After 3 failures, mark as failed (current run is the 3rd)
if [ "$FAILURE_COUNT" -ge 2 ]; then
echo "::warning::Marking $LIBRARY as failed after multiple generation failures"

# Create failed label if needed
gh label create "impl:${LIBRARY}:failed" --color "d73a4a" \
--description "${LIBRARY} implementation failed" 2>/dev/null || true

# Remove stale labels and add failed (Fix #2: clean up duplicates)
gh issue edit "$ISSUE" \
--remove-label "generate:${LIBRARY},impl:${LIBRARY}:pending" \
--add-label "impl:${LIBRARY}:failed" 2>/dev/null || true

# Post failure comment
gh issue comment "$ISSUE" --body "## :x: ${LIBRARY} Failed

The **${LIBRARY}** implementation for \`${SPEC_ID}\` failed after multiple attempts.

This may indicate that the library cannot implement this plot type natively.

---
:robot: *[impl-generate](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})*"

else
# Not enough failures yet - just clean up stale generate label
# Keep pending label for retry
gh issue edit "$ISSUE" --remove-label "generate:${LIBRARY}" 2>/dev/null || true
echo "::notice::Cleaned up generate label, keeping pending for retry"
fi
53 changes: 35 additions & 18 deletions .github/workflows/impl-merge.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,39 +245,56 @@ jobs:
# Get current labels on the issue
LABELS=$(gh issue view "$ISSUE" --json labels -q '.labels[].name' 2>/dev/null || echo "")

# Count done implementations
# Count done and failed implementations
DONE_COUNT=0
FAILED_COUNT=0
DONE_LIBS=""
FAILED_LIBS=""

for lib in $LIBRARIES; do
if echo "$LABELS" | grep -q "^impl:${lib}:done$"; then
DONE_COUNT=$((DONE_COUNT + 1))
DONE_LIBS="$DONE_LIBS $lib"
elif echo "$LABELS" | grep -q "^impl:${lib}:failed$"; then
FAILED_COUNT=$((FAILED_COUNT + 1))
FAILED_LIBS="$FAILED_LIBS $lib"
Comment on lines +257 to +260
Copy link

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DONE_LIBS and FAILED_LIBS variables accumulate library names with leading spaces (e.g., " matplotlib seaborn"). This works with the current grep usage but makes the code fragile. Consider either trimming the leading space or using an array-based approach for cleaner string handling.

Suggested change
DONE_LIBS="$DONE_LIBS $lib"
elif echo "$LABELS" | grep -q "^impl:${lib}:failed$"; then
FAILED_COUNT=$((FAILED_COUNT + 1))
FAILED_LIBS="$FAILED_LIBS $lib"
if [ -z "$DONE_LIBS" ]; then
DONE_LIBS="$lib"
else
DONE_LIBS="$DONE_LIBS $lib"
fi
elif echo "$LABELS" | grep -q "^impl:${lib}:failed$"; then
FAILED_COUNT=$((FAILED_COUNT + 1))
if [ -z "$FAILED_LIBS" ]; then
FAILED_LIBS="$lib"
else
FAILED_LIBS="$FAILED_LIBS $lib"
fi

Copilot uses AI. Check for mistakes.
fi
done

echo "::notice::Libraries done: $DONE_COUNT/9"
TOTAL=$((DONE_COUNT + FAILED_COUNT))
echo "::notice::Libraries: $DONE_COUNT done, $FAILED_COUNT failed, $TOTAL/9 total"

# Close issue if all 9 libraries are done OR done+failed=9
if [ "$TOTAL" -eq 9 ]; then
# Build status table
TABLE="| Library | Status |\n|---------|--------|"
for lib in $LIBRARIES; do
if echo "$DONE_LIBS" | grep -w -q "$lib"; then
TABLE="$TABLE\n| $lib | :white_check_mark: |"
elif echo "$FAILED_LIBS" | grep -w -q "$lib"; then
TABLE="$TABLE\n| $lib | :x: (not supported) |"
fi
done

if [ "$FAILED_COUNT" -eq 0 ]; then
TITLE=":tada: All Implementations Complete!"
SUMMARY="All 9 library implementations for \`${SPEC_ID}\` have been successfully merged."
else
TITLE=":white_check_mark: Implementations Complete"
SUMMARY="${DONE_COUNT}/9 implementations merged, ${FAILED_COUNT} libraries could not implement this plot type."
fi

# Close issue if all 9 libraries are done
if [ "$DONE_COUNT" -eq 9 ]; then
gh issue comment "$ISSUE" --body "## :tada: All Implementations Complete!
gh issue comment "$ISSUE" --body "## ${TITLE}

All 9 library implementations for \`${SPEC_ID}\` have been successfully merged.
${SUMMARY}

| Library | Status |
|---------|--------|
| matplotlib | :white_check_mark: |
| seaborn | :white_check_mark: |
| plotly | :white_check_mark: |
| bokeh | :white_check_mark: |
| altair | :white_check_mark: |
| plotnine | :white_check_mark: |
| pygal | :white_check_mark: |
| highcharts | :white_check_mark: |
| letsplot | :white_check_mark: |
$(echo -e "$TABLE")
Copy link

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The echo command with -e flag is used to render the markdown table, but the output is directly embedded in the gh issue comment body. Shell interpolation can cause issues with special characters in the TABLE variable. Consider using proper quoting or a different approach to build the comment body to avoid potential formatting issues.

Copilot uses AI. Check for mistakes.

---
:robot: *[impl-merge](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})*"

gh issue close "$ISSUE"
echo "::notice::Closed issue #$ISSUE - all implementations complete"
echo "::notice::Closed issue #$ISSUE - all implementations complete ($DONE_COUNT done, $FAILED_COUNT failed)"
fi

- name: Trigger database sync
Expand Down
Loading