feat(roadmap): gen-roadmap --check-github ground-truth validation#800
Merged
Conversation
The 2026-07-02 audit found roadmap.yaml chronically drifts from reality: tasks stayed `todo`/`in_review` while their PRs merged, an epic claimed `in_review` with no PR anywhere, and spec: links can point at nonexistent dirs producing false progress badges. `--check` only validates ROADMAP.md freshness against roadmap.yaml — nothing validated roadmap.yaml itself. Add a `--check-github` mode that cross-checks roadmap.yaml against ground truth: - PR status: MERGED but not done → ERROR; CLOSED-unmerged but in_progress/ in_review → ERROR; OPEN but done → ERROR; OPEN but todo → WARN; dangling ref → ERROR. Handles "#786" and full /pull/ URLs and lists; caches per PR. - spec: links must resolve to a real specs/ dir (ERROR); a spec shared by two distinct epics → WARN (badge double-count). - status sanity: in_review with no pr: → WARN (any item); in_progress with no pr: and no children → WARN (leaf only, umbrella epics delegate PRs); done epic with a non-done child → WARN. - exit 0 (no errors) / 1 (any error, or --strict with warnings) / 2 (gh missing or unauthenticated — offline spec/status checks still run). `--check` is untouched. Wire a non-blocking (continue-on-error) advisory step into .github/workflows/roadmap.yml. Docs updated in the generator template and roadmap.yaml header; ROADMAP.md regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deploying mcpproxy-docs with
|
| Latest commit: |
b68c590
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://75776abe.mcpproxy-docs.pages.dev |
| Branch Preview URL: | https://feat-roadmap-check-github.mcpproxy-docs.pages.dev |
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
📦 Build ArtifactsWorkflow Run: View Run Available Artifacts
How to DownloadOption 1: GitHub Web UI (easiest)
Option 2: GitHub CLI gh run download 28597689564 --repo smart-mcp-proxy/mcpproxy-go
|
Dumbris
added a commit
that referenced
this pull request
Jul 2, 2026
…ne, connect-trust/upgrade-nudge progress (#803) check-github now passes with 0 errors: scanner-simplification epic complete (#786/#792/#793/#794 incl. deep-scan trust fixes + docs sweep); connect-trust US1 preview (#802) + backup visibility (#799) done; upgrade-nudge status/log slice (#798) split out as done with the banner+config remainder tracked separately; telemetry machine_id client (#796) and hygiene check-github (#800) done. Remaining warnings are the known windows-tray no-PR-evidence items. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The 2026-07-02 planning audit found
roadmap.yamlchronically lies about reality: tasks stayedtodo/in_reviewwhile their PRs merged (#786/#792/#794/#796), an epic claimedin_reviewwith no PR anywhere, andspec:links can point at wrong/nonexistent specs producing false progress badges. Todaypython3 scripts/gen-roadmap.py --checkonly validates that ROADMAP.md is fresh vs roadmap.yaml — nothing validates roadmap.yaml vs ground truth.What this adds
A new
--check-githubmode (reads roadmap.yaml only, never regenerates ROADMAP.md):PR status cross-check (
gh pr view <n> --json state,mergedAt, cached per PR; handles#786, full/pull/URLs, and lists):done→ ERRORin_progress/in_review→ ERRORdone→ ERRORtodo→ WARNSpec-link validation:
spec:not resolving to a realspecs/<NNN>dir → ERRORStatus sanity:
in_reviewwith nopr:→ WARN (any item — the "epic in_review with no PR anywhere" case the audit hit)in_progresswith nopr:and no children → WARN (leaf only; umbrella epics legitimately delegate PRs to child tasks)doneepic with a non-done child → WARNExit codes:
0no errors ·1any error (or--strictwith warnings) ·2ghmissing/unauthenticated (offline spec+status checks still run, so it never false-fails).--checkbehaviour is untouched.CI wiring
.github/workflows/roadmap.ymlalready runs--check. Added a non-blocking (continue-on-error: true) advisory--check-githubstep in the same job (GH_TOKEN: ${{ github.token }},pull-requests: read) so drift is surfaced in the log without failing the build (roadmap.yaml is hand-maintained).Real run against current roadmap
It flags genuine live drift: #796 merged on 2026-07-02 but roadmap still says
in_review. Per the task, this drift is reported, not fixed here (beyond the doc/regeneration changes). No pytest pattern exists inscripts/(only shell.test.sh), so this real-run output is the acceptance evidence.Docs
Generator "How to regenerate" template + roadmap.yaml header updated to mention the new mode; ROADMAP.md regenerated.
🤖 Generated with Claude Code