Skip to content

Add pr-review agent skill#865

Closed
drbenvincent wants to merge 2 commits into
mainfrom
add-pr-review-skill
Closed

Add pr-review agent skill#865
drbenvincent wants to merge 2 commits into
mainfrom
add-pr-review-skill

Conversation

@drbenvincent
Copy link
Copy Markdown
Collaborator

Summary

Adds .github/skills/pr-review/ — a new agent skill capturing the review patterns distilled from the #826 review iteration into a reusable form.

Structure:

  • SKILL.md (orchestrator, ~75 lines, well under the spec's 500-line / 5000-token recommendation)
  • reference/workflow.md — the 6-step review workflow with concrete gh commands and a claim-verification table
  • reference/what-to-look-for.md — severity-sorted patterns (4 must-fix, 6 should-fix, 7 nits, 3 process), each with a canonical example from Add outcome falsification, random placebo folds, and model clone support #826 and a "how to spot" diagnostic
  • reference/code-patterns.md — CausalPy source-code conventions (BaseExperiment, PyMCModel, _clone(), __repr__, custom exceptions, type hints, helper-promotion checklist)
  • reference/docs-patterns.md — notebook + docs conventions (single-H1, hide-input/hide-output, sampler warnings, glossary linking, MyST roles, citations, file placement)
  • reference/posting-comments.md — comment templates (multi-item summary, status update, inline suggestion, correction), tone-calibration with 3-variant offering, attribution rules
  • reference/how-to-extend.md — when to add patterns (recur 2+ times), when to prune (formalised into hooks), required structure for new entries, six-monthly review cadence

Design choices worth flagging

  • Doesn't duplicate pr-to-green or pr-workflows — explicitly delegates fix-PR work to the former and create-PR work to the latter. New skill is read-and-diagnose only.
  • Hard rule: never post review comments without explicit human approval. Repeated in SKILL.md and posting-comments.md. Same for never auto-approving via gh pr review --approve.
  • Patterns grounded in real Add outcome falsification, random placebo folds, and model clone support #826 review items rather than speculative checklists — keeps the skill from drifting into wish-list territory. how-to-extend.md documents the 4-element template (ID, title, canonical example, how-to-spot) and the rule that anything formalised into a hook must be removed from the skill at the same time.
  • Followed local convention reference/ (singular) matching the existing github-issues/ and pr-workflows/ skills, even though the spec at https://agentskills.io/specification suggests references/.
  • Frontmatter is minimal — just name and description — matching local convention.

Test plan

  • prek run --all-files passes (skill files are markdown, picked up by trailing-whitespace / EOF / codespell hooks).
  • Manual smoke: load the skill on a follow-up review (e.g. an in-flight CausalPy PR) and verify the workflow + look-fors fire correctly.
  • Verify no broken intra-skill links (every reference between SKILL.md and the six reference/*.md files resolves).

Out of scope

Related

Made with Cursor

New .github/skills/pr-review/ skill capturing the review patterns
distilled from the PR #826 review iteration. Provides an orchestrator
SKILL.md plus six focused reference files: workflow, severity-sorted
look-fors, CausalPy code conventions, docs/notebook conventions,
comment-drafting templates, and an extension/pruning protocol so the
skill stays sharp as patterns get formalised into hooks.

Designed to complement (not duplicate) the existing pr-to-green and
pr-workflows skills; explicitly defers mechanical checks to hooks/CI.

Made-with: Cursor
@drbenvincent drbenvincent added the agents Agent related issues specifically for use by developers label Apr 24, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.59%. Comparing base (deb8774) to head (cf4e26d).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #865      +/-   ##
==========================================
- Coverage   94.60%   94.59%   -0.02%     
==========================================
  Files          80       80              
  Lines       12764    12764              
  Branches      770      770              
==========================================
- Hits        12076    12074       -2     
- Misses        485      486       +1     
- Partials      203      204       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@read-the-docs-community
Copy link
Copy Markdown

read-the-docs-community Bot commented Apr 24, 2026

Documentation build overview

📚 causalpy | 🛠️ Build #32412200 | 📁 Comparing cf4e26d against latest (deb8774)

  🔍 Preview build  

1 file changed
± 404.html

The validate-notebooks pre-commit hook (#866) deterministically
enforces exactly one top-level H1 per docs notebook, so the pr-review
skill no longer needs to flag it. Remove the dedicated section from
docs-patterns.md and trim the now-stale examples from SKILL.md and
the docs-patterns intro. Keep the how-to-extend.md bullet as the
worked example of the "formalise into a hook → prune from the skill"
maintenance rule.

Made-with: Cursor
@drbenvincent
Copy link
Copy Markdown
Collaborator Author

Closing this draft in favor of #920.

I folded the strongest pieces from this PR into the new maintainer-pr-review skill in #920, including the concrete review workflow, claim-verification guidance, comment-posting safeguards, CausalPy code/docs patterns, recurring review checklist, and maintenance guidance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent related issues specifically for use by developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant