Skip to content

chore: rework repo automation and add weekly Slack summary#493

Merged
bdeitte merged 22 commits into
mainfrom
chore/workflow-rework
Jul 17, 2026
Merged

chore: rework repo automation and add weekly Slack summary#493
bdeitte merged 22 commits into
mainfrom
chore/workflow-rework

Conversation

@bdeitte

@bdeitte bdeitte commented Jul 17, 2026

Copy link
Copy Markdown
Member

Summary

This is a wide swath of changes that is removing some of the workflows and automation that I haven't seen people using much or which we've mostly disabled. I am 100% ok with keeping any of this in- just let me know Ian or others and it's not a big deal to add anything back in.

This three pieces of GitHub automation and adds a weekly Slack summary of merged PRs. This also expects we turn off the release GitHub Slack messages in favor of the weekly summary. There's a number of other manual steps that needs to be taken before this is merged which I'll handle, mentioned below.

Removed

  • preview-screenshot-gallery — the gh-aw agent (posit-vip-triage[bot]) that
    posted Playwright screenshots into PRs. Low value; removed entirely.
  • The gh-aw agentic-workflow system — issue-triage + implement-plan and all
    their supporting pieces (lockfiles, docs/agentic-workflows.md, the triage-bot
    README, .github/aw action-lock, the gh-aw authoring agent + VS Code MCP +
    Copilot-setup gh-aw step, the ci.yml lockfile-guard job, and the gh-aw-only
    dependabot.yml ignores). Both agents rarely fired (triage was opt-in behind a
    label; implement-plan only ran for bot-authored plan PRs). No-op comments to
    issue [aw] No-Op Runs #158 were silenced en route, then subsumed by this removal.
  • The showboat demo-on-PR convention — the AGENTS.md section, the demo-*
    justfile recipes, the demo.md gitignore entry, and the validation_docs/
    archive (68 demos; their content still lives in historical PR bodies).

Added

  • .github/workflows/weekly-summary.yml + .claude/agents/weekly-summary.md: a
    Monday 16:00 UTC (and workflow_dispatch) job that gathers the week's merged
    posit-dev/vip PRs (bounded UTC-timestamp window, bot-filtered), has Claude
    pick 5–10 highlights via Bedrock, builds a Slack Block Kit payload (highlights
    linked to PRs by a deterministic number→url map; Claude-derived text is
    Slack-mrkdwn-escaped), and posts to SLACK_WEBHOOK_VIP_WEEKLY_SUMMARY.
    pull_request runs are a dry run (build + log the payload, never post).
    Modeled on Package Manager's weekly-summary, simplified to a single repo (no
    multi-repo gather, CVP/TDP enrichment, or NEWS diff).

Provisioning required before the weekly summary runs live

  • Add a SLACK_WEBHOOK_VIP_WEEKLY_SUMMARY repo secret (a new Slack incoming
    webhook for the target channel).
  • Allow posit-dev/vip to assume the claude-code-gha AWS role (account
    528395739535) via OIDC for Bedrock — PPM uses it under rstudio; vip is
    posit-dev, so the role's trust policy likely needs repo:posit-dev/vip:*.
  • Heads-up: until that AWS trust is granted, the Weekly Summary check on this
    PR (the pull_request dry run) fails at the "Configure AWS credentials" step.
    It is not a required check and does not block merge.
  • Verify the Bedrock model ID us.anthropic.claude-opus-4-8 resolves on the
    first live workflow_dispatch run (kept at parity with PPM; the us.
    cross-region inference profile is the standard invocation form).

Manual follow-ups (out of diff)

  • Run /github unsubscribe posit-dev/vip releases in Slack to stop the
    "New release published by github-actions[bot]" cards (a GitHub↔Slack app
    subscription, not repo config).
  • Remove the "Agentic Workflow Lockfile Guard" required status check from main
    branch protection/rulesets (the job is deleted; otherwise PRs wait on a check
    that never runs).
  • Deprovision the POSIT_VIP_TRIAGE GitHub App + its POSIT_VIP_TRIAGE_CLIENT_ID
    / POSIT_VIP_TRIAGE_PEM secrets (now unused by any workflow).
  • Delete the orphaned remote branch assets/preview-screenshot-gallery.
  • Triage labels (needs-bot-triage, triaged-by-bot, plan-pending-review,
    needs-human-triage, skip-triage) are now inert; issue [aw] No-Op Runs #158 ("[aw] No-Op
    Runs") can be closed.

Testing

  • selftests: 965 passed, 3 skipped.
  • ruff check + format: clean (pinned 0.15.0).
  • Weekly-summary PR-gather and Slack-payload jq (including a mrkdwn-injection
    case) verified locally against real posit-dev/vip data. The live
    Claude/Bedrock run and Slack post are validated via workflow_dispatch once
    the secret and AWS trust are in place.

bdeitte added 13 commits July 16, 2026 16:48
The preview-screenshot-gallery gh-aw agent (running as the
posit-vip-triage GitHub App) captured Playwright screenshots of PR
preview pages and posted them inline in a PR comment. The screenshots
add little value to PRs, so remove the functionality entirely.

- delete the workflow source and compiled lockfile
- delete the archived showboat demo for the removed feature
- drop the dangling reference in docs/agentic-workflows.md
- generalize the ci.yml lockfile-guard comment example

The shared POSIT_VIP_TRIAGE app secrets stay in use by the issue-triage
and implement-plan workflows, so nothing else changes.
Both the issue-triage and implement-plan gh-aw agents trigger on most
issue/PR activity and gate out ('No action taken') the vast majority of
the time. Each gate-exit emits a no-op, and gh-aw's default posts every
no-op as a comment on the shared '[aw] No-Op Runs' tracking issue (#158),
which has become pure noise.

Set 'noop: report-as-issue: false' on both workflows (as the issue body
itself documents) so gate-exit runs stop commenting there. Lockfiles
recompiled with the pinned gh-aw v0.81.6.
The issue-triage and implement-plan agents rarely did anything: triage
was opt-in behind a maintainer label, and implement-plan only fired for
bot-authored plan PRs (recent plan PRs have been human-authored). With
the screenshot gallery already gone, no gh-aw workflows remain, so remove
the whole system and its supporting tooling.

Delete:
- issue-triage and implement-plan workflow sources + compiled lockfiles
- docs/agentic-workflows.md and .github/triage-bot/README.md
- .github/aw/actions-lock.json (gh-aw compiler action-SHA lock)
- .github/agents/agentic-workflows.agent.md (gh-aw authoring agent)
- .vscode/mcp.json (local gh-aw MCP server config)

Edit:
- ci.yml: drop the agentic-lockfile-guard job (no lockfiles to check)
- dependabot.yml: drop the gh-aw-only ignore block so Dependabot manages
  the shared actions normally again
- copilot-setup-steps.yml: drop the gh-aw extension install step
- AGENTS.md: remove the two workflow entries from the CI workflows list

The POSIT_VIP_TRIAGE GitHub App and its secrets are now unused by any
workflow. Kept thoughts/shared/plans/ (design records referenced
elsewhere) and add-to-team-project.yml (unrelated project-board sync).
Building and pasting showboat demos into PR bodies added overhead without
enough payoff, so drop the whole convention:

- AGENTS.md: remove the 'Showboat demos' section (creation, timing-safe
  patterns, and the demo-save/PR-body workflow)
- justfile: remove the demo-init, demo-verify, and demo-save recipes
- .gitignore: drop the demo.md working-file entry
- delete the validation_docs/ archive (68 demos; their content still
  lives in the historical PR bodies)
- CHRONICLE_TEST_PLAN.md: drop the dangling link to a deleted demo

No CI change needed: no workflow referenced showboat or validation_docs
(the stale 'CI will run showboat verify' note went away with the removed
AGENTS.md section).
Design for a Claude-picked weekly summary of merged posit-dev/vip PRs
posted to Slack, modeled on PPM's weekly-summary but simplified to a
single repo (no multi-repo gather, CVP/TDP, or NEWS diff).
A bare-date merged:>=WEEK_START is unbounded upward and starts at 00:00,
so PRs merged between 00:00 and the Monday 16:00 UTC run time landed in
two consecutive weekly summaries. Use a run-aligned timestamp range
merged:WEEK_START_TS..WEEK_END_TS in both the plan and the spec; keep
date-only values for the Slack display. Verified the range query and bot
filter against posit-dev/vip.
Copilot AI review requested due to automatic review settings July 17, 2026 13:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR streamlines repository automation by removing several legacy/noisy GitHub automations (gh-aw agentic workflows and the PR preview screenshot agent) and the Showboat demo-on-PR convention, while adding a new GitHub Actions workflow that generates a weekly Slack summary of merged PRs using Claude (Bedrock) for highlight selection.

Changes:

  • Remove gh-aw agentic workflow system (triage + implement-plan) and related repo scaffolding/docs.
  • Remove the Showboat demo convention and delete the accumulated validation_docs/ demo archive (leaving only a .gitkeep).
  • Add a scheduled/manual “Weekly Summary” workflow that gathers merged PRs, asks Claude for 5–10 highlights, and posts a Slack Block Kit message (PR runs are dry-run only).

Reviewed changes

Copilot reviewed 84 out of 89 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
validation_docs/demo-workbench-trailing-slash.md Remove archived Showboat demo doc.
validation_docs/demo-workbench-session-fail-fast.md Remove archived Showboat demo doc.
validation_docs/demo-workbench-session-cleanup.md Remove archived Showboat demo doc.
validation_docs/demo-workbench-quit-button-versioned.md Remove archived Showboat demo doc.
validation_docs/demo-workbench-git-ops.md Remove archived Showboat demo doc.
validation_docs/demo-workbench-console-fill.md Remove archived Showboat demo doc.
validation_docs/demo-workbench-console-and-session-robustness.md Remove archived Showboat demo doc.
validation_docs/demo-vip-version-flag.md Remove archived Showboat demo doc.
validation_docs/demo-vip-fix-suspend-resume-reload.md Remove archived Showboat demo doc.
validation_docs/demo-vip-fix-headless-auth-login-skip.md Remove archived Showboat demo doc.
validation_docs/demo-vip-fix-config-repr-secrets.md Remove archived Showboat demo doc.
validation_docs/demo-version-gating.md Remove archived Showboat demo doc.
validation_docs/demo-update-requests-dependency.md Remove archived Showboat demo doc.
validation_docs/demo-update-banner-text.md Remove archived Showboat demo doc.
validation_docs/demo-totp-seed.md Remove archived Showboat demo doc.
validation_docs/demo-tls-cert-verify-diagnostic.md Remove archived Showboat demo doc.
validation_docs/demo-status-json-output.md Remove archived Showboat demo doc.
validation_docs/demo-remove-shiny-k8s-modes.md Remove archived Showboat demo doc.
validation_docs/demo-readback-routing.md Remove archived Showboat demo doc.
validation_docs/demo-publish-wheels-pypi.md Remove archived Showboat demo doc.
validation_docs/demo-preview-screenshot-agentic-workflow.md Remove archived Showboat demo doc.
validation_docs/demo-option-b-vip-tests.md Remove archived Showboat demo doc.
validation_docs/demo-move-tickets-to-team-projects.md Remove archived Showboat demo doc.
validation_docs/demo-mock-idp-e2e.md Remove archived Showboat demo doc.
validation_docs/demo-migrate-workbench-smoke-tests.md Remove archived Showboat demo doc.
validation_docs/demo-load-tests.md Remove archived Showboat demo doc.
validation_docs/demo-landing-page.md Remove archived Showboat demo doc.
validation_docs/demo-issue-467-session-cleanup.md Remove archived Showboat demo doc.
validation_docs/demo-issue-430-interactive-auth-headless-only.md Remove archived Showboat demo doc.
validation_docs/demo-issue-178-test-timeout.md Remove archived Showboat demo doc.
validation_docs/demo-issue-178-followup.md Remove archived Showboat demo doc.
validation_docs/demo-issue-170-headless-auth-url.md Remove archived Showboat demo doc.
validation_docs/demo-inline-full-skip-reasons.md Remove archived Showboat demo doc.
validation_docs/demo-ide-extension-validation.md Remove archived Showboat demo doc.
validation_docs/demo-git-clone-cleanup.md Remove archived Showboat demo doc.
validation_docs/demo-git-anonymous-auth.md Remove archived Showboat demo doc.
validation_docs/demo-gh-pages-coexistence.md Remove archived Showboat demo doc.
validation_docs/demo-force-vip-release.md Remove archived Showboat demo doc.
validation_docs/demo-fix-workbench-login-display-name.md Remove archived Showboat demo doc.
validation_docs/demo-fix-terminal-run-timeout.md Remove archived Showboat demo doc.
validation_docs/demo-fix-ssl-redirect-alb.md Remove archived Showboat demo doc.
validation_docs/demo-fix-rstudio-console-prompt-wait.md Remove archived Showboat demo doc.
validation_docs/demo-fix-package-build.md Remove archived Showboat demo doc.
validation_docs/demo-fix-locust-install-instructions.md Remove archived Showboat demo doc.
validation_docs/demo-fix-issue-183.md Remove archived Showboat demo doc.
validation_docs/demo-fix-issue-182.md Remove archived Showboat demo doc.
validation_docs/demo-fix-issue-173-headless-auth-skip.md Remove archived Showboat demo doc.
validation_docs/demo-fix-example-report-pm-license.md Remove archived Showboat demo doc.
validation_docs/demo-fix-connect-user-list-skip.md Remove archived Showboat demo doc.
validation_docs/demo-fix-280-extensions-search-input.md Remove archived Showboat demo doc.
validation_docs/demo-fix-268-insecure-tls-skip.md Remove archived Showboat demo doc.
validation_docs/demo-deterministic-lockfile.md Remove archived Showboat demo doc.
validation_docs/demo-debug-pypi-publishing.md Remove archived Showboat demo doc.
validation_docs/demo-data-source-stray-collection.md Remove archived Showboat demo doc.
validation_docs/demo-custom-extension-example.md Remove archived Showboat demo doc.
validation_docs/demo-connect-system-checks.md Remove archived Showboat demo doc.
validation_docs/demo-connect-super-post-init.md Remove archived Showboat demo doc.
validation_docs/demo-connect-content-cleanup.md Remove archived Showboat demo doc.
validation_docs/demo-connect-chronicle-status.md Remove archived Showboat demo doc.
validation_docs/demo-chronicle-tests.md Remove archived Showboat demo doc.
validation_docs/demo-chromium-revision-check.md Remove archived Showboat demo doc.
validation_docs/demo-bot-implement-issue-344.md Remove archived Showboat demo doc.
validation_docs/demo-bot-implement-issue-308.md Remove archived Showboat demo doc.
validation_docs/demo-bot-implement-issue-307.md Remove archived Showboat demo doc.
validation_docs/demo-bot-implement-issue-302.md Remove archived Showboat demo doc.
validation_docs/demo-bot-implement-issue-301.md Remove archived Showboat demo doc.
validation_docs/demo-bot-fix-issue-283.md Remove archived Showboat demo doc.
validation_docs/demo-binary-package-tests.md Remove archived Showboat demo doc.
validation_docs/.gitkeep Retain directory placeholder after demo archive removal.
justfile Remove Showboat demo helper recipes.
docs/agentic-workflows.md Remove documentation for gh-aw agentic workflows (feature removed).
CHRONICLE_TEST_PLAN.md Remove reference to a deleted demo artifact.
AGENTS.md Remove gh-aw workflow references and Showboat demo convention docs.
.vscode/mcp.json Remove VS Code MCP config for gh-aw (feature removed).
.gitignore Stop ignoring demo.md (Showboat workflow removed).
.github/workflows/weekly-summary.yml Add scheduled/manual Slack weekly summary workflow (PR runs are dry-run).
.github/workflows/preview-screenshot-gallery.md Remove preview screenshot gh-aw workflow.
.github/workflows/issue-triage.md Remove gh-aw issue triage workflow.
.github/workflows/implement-plan.md Remove gh-aw plan implementation workflow.
.github/workflows/copilot-setup-steps.yml Remove gh-aw CLI installation from Copilot setup.
.github/workflows/ci.yml Remove the gh-aw lockfile guard CI job.
.github/triage-bot/README.md Remove gh-aw triage bot documentation.
.github/dependabot.yml Remove gh-aw-specific Dependabot ignore rules.
.github/aw/actions-lock.json Remove gh-aw action lockfile.
.github/agents/agentic-workflows.agent.md Remove agent dispatcher for gh-aw workflow authoring.
.claude/agents/weekly-summary.md Add Claude agent instructions for selecting weekly PR highlights.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/weekly-summary.yml
@bdeitte

bdeitte commented Jul 17, 2026

Copy link
Copy Markdown
Member Author

🧪 vip Weekly Summary

2026-07-10 to 2026-07-17 | 15 merged PRs | v0.53.0 to v0.54.6

🚀 Features & tests

  • The CLI now supports a --version flag and a version subcommand to report the installed VIP version (PR #472)
  • New Package Manager web UI smoke tests validate the interface across multiple package ecosystems (PR #460)
  • A new workflow automatically routes team-labeled issues to the right product project board (PR #435)
  • Added a QA-focused presentation explaining the VIP test architecture (PR #468)

🔧 Fixes & chores

  • Orphaned Workbench session cleanup now escalates to a full browser UI sweep, a large overhaul of the cleanup path (PR #474)
  • Workbench tests gained a default git clone repo, more accurate skips, and reliable R version readback (PR #490)
  • Workbench terminal_run now raises ExecError immediately on failure instead of hanging (PR #445)
  • The Quarto report can now render from any directory and gained badge styling (PR #487)
  • Workbench browser tests are more resilient to single-page-app timing so cleanup reliably finds sessions

Rendered from the weekly-summary dry-run payload in run 29607435969. This was a pull_request dry run — the payload was built and logged but not posted to Slack. The release range reflects this run's rolling 7-day window (anchored at run time), so it starts at v0.53.0 rather than v0.52.4.

Raw Slack payload
{
  "text": "vip Weekly Summary",
  "blocks": [
    {
      "type": "header",
      "text": {
        "type": "plain_text",
        "text": "🧪 vip Weekly Summary",
        "emoji": true
      }
    },
    {
      "type": "context",
      "elements": [
        {
          "type": "mrkdwn",
          "text": "*2026-07-10* to *2026-07-17* | <https://github.com/posit-dev/vip/pulls?q=is%3Apr+is%3Aclosed|15 merged PRs> | <https://github.com/posit-dev/vip/releases|v0.53.0 to v0.54.6>"
        }
      ]
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "*🚀 Features &amp; tests*\n• The CLI now supports a --version flag and a version subcommand to report the installed VIP version (<https://github.com/posit-dev/vip/pull/472|PR #472>)\n• New Package Manager web UI smoke tests validate the interface across multiple package ecosystems (<https://github.com/posit-dev/vip/pull/460|PR #460>)\n• A new workflow automatically routes team-labeled issues to the right product project board (<https://github.com/posit-dev/vip/pull/435|PR #435>)\n• Added a QA-focused presentation explaining the VIP test architecture (<https://github.com/posit-dev/vip/pull/468|PR #468>)"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "*🔧 Fixes &amp; chores*\n• Orphaned Workbench session cleanup now escalates to a full browser UI sweep, a large overhaul of the cleanup path (<https://github.com/posit-dev/vip/pull/474|PR #474>)\n• Workbench tests gained a default git clone repo, more accurate skips, and reliable R version readback (<https://github.com/posit-dev/vip/pull/490|PR #490>)\n• Workbench terminal_run now raises ExecError immediately on failure instead of hanging (<https://github.com/posit-dev/vip/pull/445|PR #445>)\n• The Quarto report can now render from any directory and gained badge styling (<https://github.com/posit-dev/vip/pull/487|PR #487>)\n• Workbench browser tests are more resilient to single-page-app timing so cleanup reliably finds sessions"
      }
    }
  ]
}

bdeitte added 3 commits July 17, 2026 14:22
- Split highlights into 'Features & tests' and 'Fixes & chores' sections
  via a new per-highlight category field (schema-enforced).
- Weight features/tests and larger PRs more heavily; feed the agent each
  PR's additions/deletions/changedFiles as a size signal.
- Link the merged-PR count to the closed-PRs list and add a linked
  release range (oldest→newest tag) to the context line.
- Label each PR reference as 'PR #NNN' instead of '#NNN'.
PR #494's showboat demo (demo-478-jupyterlab-launch.md) landed on main
after this branch removed the validation_docs archive; drop it to keep
the branch's removal of the showboat convention complete.
@bdeitte
bdeitte merged commit d912850 into main Jul 17, 2026
26 checks passed
@bdeitte
bdeitte deleted the chore/workflow-rework branch July 17, 2026 19:59
ian-flores added a commit that referenced this pull request Jul 17, 2026
PR #500 was branched with a validation_docs/ demo following the showboat
workflow that #493 removed (recipes, .gitignore entry, guidance, and all
prior demos). Drop demo-480-workbench-ordering.md to align; the verification
evidence lives in the PR body instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants