ENH: Add fix-nightly-warnings skill and CDash query scripts#5923
ENH: Add fix-nightly-warnings skill and CDash query scripts#5923blowekamp wants to merge 12 commits intoInsightSoftwareConsortium:mainfrom
Conversation
|
I have had some success with this skill. Adding the scripts is a way to reduce the number of requests needed for the AI, and improve efficiency. I have am running low on available requests with the initial development of this skill. Someone else is welcome to further refine this workflow. |
thewtex
left a comment
There was a problem hiding this comment.
Very cool!!
We may want to set up a GitHub Agentic Workflow to run this on a regular basis.
|
@blowekamp I have some improvements ready for this PR (typo fixes, ThirdParty filtering, MSVC warning support, a To let me push directly to your PR branch, could you enable "Allow edits from maintainers"? You can do this by:
(This option is in the right sidebar under the PR description when you're editing the PR, or at the bottom of the PR creation/edit page.) Once enabled, I'll be able to ====== https://github.com/InsightSoftwareConsortium/ITK/tree/skill_fix_nightly_warnings The 10 commits should probably be squished, but they seem reasonable for draft consideration. |
|
That option is already enabled. |
Address PR review comment (opens -> open), fix --limit flag typo (-limit -> --limit), correct jq pipeline to operate on .entries for get_build_warnings.py JSON output, and add explicit script directory path for agentic automation clarity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add --exclude-thirdparty flag to get_build_warnings.py to filter out warnings from Modules/ThirdParty/ paths. Update jq pipeline example in SKILL.md to filter ThirdParty in the JSON output. Warnings from vendored third-party code should not be fixed in ITK. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Provide a specific build command for local verification of warning fixes. Reference CMake/CTestCustom.cmake.in as the authoritative list of already-suppressed warning patterns to avoid duplicate work. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instruct agents to deduplicate warnings by (sourceFile, flag) across multiple CDash builds before analyzing. Make user-review step conditional on interactive vs. agentic execution mode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The display limit was computed into displayed_nodes but never used; JSON output iterated the untruncated nodes list instead. Replace displayed_nodes with in-place truncation of nodes so --limit is applied consistently to both human-readable and JSON output modes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add MSVC_FLAG_RE pattern to extract C4805, C4267, etc. warning codes from MSVC compiler output. Falls back to MSVC pattern when no GCC/Clang -W flag is found, improving grouping accuracy for Windows CDash builds. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a quick-reference table mapping common compiler warning flags to their typical mechanical fixes. Reduces agent research time for routine warnings like -Wshadow, -Wunused-parameter, and MSVC C4805. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add repeatable --exclude PATTERN flag to get_build_warnings.py that filters entries whose sourceFile contains the given pattern. Consolidates --exclude-thirdparty into the same filtering logic. Useful for excluding specific directories beyond ThirdParty. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a script that chains list -> fetch -> deduplicate into one invocation, producing an actionable summary of warnings grouped by flag. Automatically excludes ThirdParty paths and deduplicates across builds. Reduces tool calls needed for agentic automation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a flag that filters builds to only those with more warnings than the minimum across recent builds. This identifies regressions from a clean baseline rather than showing all builds with any warnings, which is more useful for automated triage. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
e0bcf58 to
7997b88
Compare
.github/skills/fix-nightly-warnings/scripts/list_nightly_warnings.py
Outdated
Show resolved
Hide resolved
- Fix typo: "opens a PR" -> "open a PR" in SKILL.md (thewtex review) - Rename nodes -> displayed_nodes after limit is applied in list_nightly_warnings.py to distinguish the display-limited subset from the full filtered list (hjmjohnson review) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
| Filename | Overview |
|---|---|
| .github/skills/fix-nightly-warnings/SKILL.md | Agent skill definition for the nightly-warning triage workflow; minor: Quality Checks section uses bare [] instead of - [ ], so checkboxes won't render correctly on GitHub. |
| .github/skills/fix-nightly-warnings/scripts/get_build_warnings.py | Paginated CDash GraphQL fetch for per-build warnings; pagination, exclusion filters, and output modes are all correct. |
| .github/skills/fix-nightly-warnings/scripts/list_nightly_warnings.py | Lists CDash builds with warnings/errors; logic for nightly-start window, sorting, and --new-warnings-only baseline heuristic is correct. |
| .github/skills/fix-nightly-warnings/scripts/triage_nightly.py | Orchestrates list + fetch into a deduplicated triage report; has a P1 JSON key inconsistency ("warnings" vs "warnings_by_flag" in the empty-builds branch) and silently swallows per-build fetch errors with no diagnostic output. |
Sequence Diagram
sequenceDiagram
participant Agent
participant triage_nightly.py
participant get_build_warnings.py
participant CDash GraphQL
Agent->>triage_nightly.py: run (--type, --since, --json)
triage_nightly.py->>CDash GraphQL: project builds query (QUERY_TEMPLATE)
CDash GraphQL-->>triage_nightly.py: edges [{node: {id, name, warnings, errors}}]
loop for each build
triage_nightly.py->>get_build_warnings.py: fetch_entries(build_id, WARNING, limit)
get_build_warnings.py->>CDash GraphQL: build warnings query (paginated)
CDash GraphQL-->>get_build_warnings.py: warning entries
get_build_warnings.py-->>triage_nightly.py: (build_meta, entries)
triage_nightly.py->>triage_nightly.py: deduplicate by (sourceFile, flag)
end
triage_nightly.py-->>Agent: JSON {builds_inspected, builds, warnings_by_flag}
Reviews (1): Last reviewed commit: "STYLE: Address review comments on fix-ni..." | Re-trigger Greptile
| if not builds_with_warnings: | ||
| if args.json_output: | ||
| print(json.dumps({"builds_inspected": 0, "warnings": []}, indent=2)) | ||
| else: | ||
| print("No builds with warnings found.", file=sys.stderr) | ||
| return |
There was a problem hiding this comment.
Inconsistent JSON key in empty-builds case
When no builds are found, the JSON output uses the key "warnings", but the normal execution path uses "warnings_by_flag". Any downstream consumer (e.g., an agent or jq pipeline) that reads .warnings_by_flag will get null for the empty case instead of [], silently producing incorrect behaviour.
| if not builds_with_warnings: | |
| if args.json_output: | |
| print(json.dumps({"builds_inspected": 0, "warnings": []}, indent=2)) | |
| else: | |
| print("No builds with warnings found.", file=sys.stderr) | |
| return | |
| if not builds_with_warnings: | |
| if args.json_output: | |
| print(json.dumps({"builds_inspected": 0, "warnings_by_flag": []}, indent=2)) | |
| else: | |
| print("No builds with warnings found.", file=sys.stderr) | |
| return |
| try: | ||
| _meta, entries = fetch_entries( | ||
| str(build_id), "WARNING", args.limit_warnings | ||
| ) | ||
| except SystemExit: | ||
| continue |
There was a problem hiding this comment.
Silent failure when all warning-fetches fail
fetch_entries calls sys.exit(2) on any network error. Catching SystemExit swallows those errors, so if every per-build fetch fails the script exits with code 0 and produces an empty result — the agent will interpret this as "no warnings" and skip the triage step entirely. At minimum, a warning should be printed for each skipped build so the caller can tell whether the result is complete.
| try: | |
| _meta, entries = fetch_entries( | |
| str(build_id), "WARNING", args.limit_warnings | |
| ) | |
| except SystemExit: | |
| continue | |
| try: | |
| _meta, entries = fetch_entries( | |
| str(build_id), "WARNING", args.limit_warnings | |
| ) | |
| except SystemExit: | |
| print(f"Warning: failed to fetch warnings for build {build_id} ({build_name}), skipping.", file=sys.stderr) | |
| continue |
| [] All targeted warnings are fixed | ||
| [] No new warnings or errors introduced | ||
| [] Changes are limited to the files affected by the warnings | ||
| [] Commit message clearly describes the fix and references the CDash issue if applicable |
There was a problem hiding this comment.
Checklist items won't render as checkboxes
GitHub Flavored Markdown requires the - [ ] syntax for interactive checkboxes. The bare [] used here will render as plain text, not tick-boxes.
| [] All targeted warnings are fixed | |
| [] No new warnings or errors introduced | |
| [] Changes are limited to the files affected by the warnings | |
| [] Commit message clearly describes the fix and references the CDash issue if applicable | |
| - [ ] All targeted warnings are fixed | |
| - [ ] No new warnings or errors introduced | |
| - [ ] Changes are limited to the files affected by the warnings | |
| - [ ] Commit message clearly describes the fix and references the CDash issue if applicable |
Summary
Adds a GitHub Copilot agent skill for the workflow of fixing ITK nightly build warnings reported on CDash, along with two helper Python scripts for querying the CDash GraphQL API.
Changes
Notes
No production source changes; purely development tooling.