Skip to content

Commit f1a1847

Browse files
rdimitrovclaude
andcommitted
Flag silent skill runs in PR body + dispatch summary
Companion to the tracking comments: when the skill concludes successfully but produces zero commits, the PR body previously read as if content was added. Reviewers had no way to distinguish "skill ran silently" from "skill produced stuff". Adds: - A new step that rev-counts commits between pre_skill SHA and HEAD. Output exposed as steps.skill_commits.outputs.count. - A [!NOTE] block in the PR body when SKILL_COMMIT_COUNT=0 and no NO_CHANGES.md was written. Lists the three likely causes (docs already ahead of pin; no doc-relevant release; skill's verification matched existing prose). - A row in the workflow_dispatch summary comment showing the commit count. Surfaced by the e2e test on PR #777: the skill ran to success but produced nothing because main was already ahead of the pinned scratch branch. PR body looked like a normal augmented PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 5e1b33d commit f1a1847

1 file changed

Lines changed: 47 additions & 0 deletions

File tree

.github/workflows/upstream-release-docs.yml

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -690,6 +690,26 @@ jobs:
690690
if they exist -- they're signal files handed off to the
691691
next workflow step, not part of the docs.
692692
693+
# Count the commits the skill itself added between pre_skill
694+
# and now. Zero commits means skill_gen and skill_review both
695+
# concluded there was nothing to change -- e.g. because main
696+
# already has the content for this release (a re-run or test
697+
# after the same content was merged via a different PR), or
698+
# because the release genuinely had no doc-relevant impact
699+
# but the skill didn't produce NO_CHANGES.md. Either way the
700+
# resulting PR is silent and reviewers can't tell that from
701+
# a successful run with content -- we emit a visible note in
702+
# the PR body when this is the case.
703+
- name: Count skill commits
704+
id: skill_commits
705+
if: always() && steps.skill_gen.conclusion == 'success'
706+
env:
707+
BASELINE_SHA: ${{ steps.pre_skill.outputs.sha }}
708+
run: |
709+
COUNT=$(git rev-list "$BASELINE_SHA..HEAD" --count)
710+
echo "count=$COUNT" >> "$GITHUB_OUTPUT"
711+
echo "Skill produced $COUNT commit(s) between pre_skill and now."
712+
693713
# Auto-apply the same formatters the project's pre-commit hook
694714
# runs, scoped to the files the skill touched. The skill's
695715
# sandbox doesn't include npm run prettier/eslint, so without
@@ -857,6 +877,7 @@ jobs:
857877
COMPARE_OK: ${{ steps.reviewers.outputs.compare_ok }}
858878
MENTION_BLOCK: ${{ steps.reviewers.outputs.mention_block }}
859879
ASSIGN_LIST: ${{ steps.reviewers.outputs.list }}
880+
SKILL_COMMIT_COUNT: ${{ steps.skill_commits.outputs.count }}
860881
run: |
861882
START='<!-- upstream-release-docs:start -->'
862883
END='<!-- upstream-release-docs:end -->'
@@ -878,6 +899,30 @@ jobs:
878899
echo "$NOTE_BLOCK"
879900
echo ""
880901
fi
902+
# When the skill ran to success but produced zero commits,
903+
# we have no NOTE_BLOCK (no NO_CHANGES.md), no content for
904+
# reviewers to look at, and a PR body that otherwise reads
905+
# as if content was added. Surface the silence explicitly.
906+
if [ "$SKILL_COMMIT_COUNT" = "0" ] && [ -z "$NOTE_BLOCK" ]; then
907+
echo "> [!NOTE]"
908+
echo "> The \`upstream-release-docs\` skill ran to success but"
909+
echo "> produced no content commits on this PR. Likely causes:"
910+
echo ">"
911+
echo "> - The docs already cover this release (e.g. this PR"
912+
echo "> was dispatched after an earlier PR for the same"
913+
echo "> tag had merged, or \`main\` is already ahead of the"
914+
echo "> pinned base)."
915+
echo "> - The release genuinely had no doc-relevant changes"
916+
echo "> but the skill did not write \`NO_CHANGES.md\` (which"
917+
echo "> would have triggered the standard 'no changes'"
918+
echo "> note above)."
919+
echo "> - The skill's source verification concluded the"
920+
echo "> existing prose already matches upstream behavior."
921+
echo ">"
922+
echo "> Only the version bump and any refreshed reference"
923+
echo "> assets are included in this PR."
924+
echo ""
925+
fi
881926
if [ -n "$AUTOGEN_NOTE" ]; then
882927
echo "$AUTOGEN_NOTE"
883928
echo ""
@@ -945,6 +990,7 @@ jobs:
945990
GEN_CONCLUSION: ${{ steps.skill_gen.conclusion }}
946991
REVIEW_CONCLUSION: ${{ steps.skill_review.conclusion }}
947992
AUTOFIX_CONCLUSION: ${{ steps.autofix.conclusion }}
993+
SKILL_COMMIT_COUNT: ${{ steps.skill_commits.outputs.count }}
948994
run: |
949995
gh pr comment "$PR_NUMBER" --body "## Upstream-release-docs run summary
950996
@@ -955,6 +1001,7 @@ jobs:
9551001
| Generation (\`skill_gen\`) | \`${GEN_CONCLUSION:-(not run)}\` |
9561002
| Editorial review (\`skill_review\`) | \`${REVIEW_CONCLUSION:-(not run)}\` |
9571003
| Autofix (prettier/eslint) | \`${AUTOFIX_CONCLUSION:-(not run)}\` |
1004+
| Skill commits produced | \`${SKILL_COMMIT_COUNT:-?}\` |
9581005
9591006
Full report and Claude's step-by-step log: $RUN_URL" || true
9601007

0 commit comments

Comments
 (0)