Skip to content

chore: add AI QA review and release regression skills#40342

Open
jessicaschelly wants to merge 1 commit intodevelopfrom
chore/ai-qa-skills
Open

chore: add AI QA review and release regression skills#40342
jessicaschelly wants to merge 1 commit intodevelopfrom
chore/ai-qa-skills

Conversation

@jessicaschelly
Copy link
Copy Markdown
Member

@jessicaschelly jessicaschelly commented Apr 30, 2026

Proposed changes (including videos or screenshots)

This PR adds two QA-focused AI skills under skills/ and exposes that shared skill directory to
the supported assistant configs.

The new skills are:

  • qa-pr: a QA-style review workflow for branches and pull requests, focused on finding user-
    visible regressions and validating concrete bug hypotheses
  • release-qa-regression: a release-oriented regression workflow for comparing a target release/
    branch against a known-good baseline and identifying the introducing PR or commit

It also adds the skills directory mapping for:

  • .agents/
  • .claude/
  • .cursor/
  • .github/

This keeps the QA and release regression review flows reusable across the different assistant
integrations in the repository.

Issue(s)

Steps to test or reproduce

  1. Open either skills/qa-pr/SKILL.md or skills/release-qa-regression/SKILL.md and confirm the
    workflows are present.
  2. Verify that .agents/skills, .claude/skills, .cursor/skills, and .github/skills all
    point to ../skills.
  3. Run an assistant flow that loads repository skills and confirm the new skills are
    discoverable.

Further comments

This is a workflow/configuration change only. It does not change product runtime behavior.

Summary by CodeRabbit

  • Documentation

    • Added standardized QA workflow documentation for PR reviews.
    • Added standardized QA workflow documentation for release regression testing.
  • Chores

    • Added configuration references to consolidate skills directory across development environments.

@dionisio-bot
Copy link
Copy Markdown
Contributor

dionisio-bot Bot commented Apr 30, 2026

Looks like this PR is not ready to merge, because of the following issues:

  • This PR is missing the 'stat: QA assured' label
  • This PR is missing the required milestone or project

Please fix the issues and try again

If you have any trouble, please check the PR guidelines

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Apr 30, 2026

⚠️ No Changeset found

Latest commit: 4dbd974

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 30, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 56652125-cb68-42de-8ec5-5b195dae1242

📥 Commits

Reviewing files that changed from the base of the PR and between 305a242 and 4dbd974.

📒 Files selected for processing (6)
  • .agents/skills
  • .claude/skills
  • .cursor/skills
  • .github/skills
  • skills/qa-pr/SKILL.md
  • skills/release-qa-regression/SKILL.md
📜 Recent review details
🧰 Additional context used
🧠 Learnings (5)
📓 Common learnings
Learnt from: d-gubert
Repo: RocketChat/Rocket.Chat PR: 37547
File: packages/i18n/src/locales/en.i18n.json:634-634
Timestamp: 2025-11-19T12:32:29.696Z
Learning: Repo: RocketChat/Rocket.Chat
Context: i18n workflow
Learning: In this repository, new translation keys should be added to packages/i18n/src/locales/en.i18n.json only; other locale files are populated via the external translation pipeline and/or fall back to English. Do not request adding the same key to all locale files in future reviews.
Learnt from: ggazzo
Repo: RocketChat/Rocket.Chat PR: 35995
File: apps/meteor/app/api/server/v1/rooms.ts:1107-1112
Timestamp: 2026-02-23T17:53:18.785Z
Learning: In Rocket.Chat PR reviews, maintain strict scope boundaries—when a PR is focused on a specific endpoint (e.g., rooms.favorite), avoid reviewing or suggesting changes to other endpoints that were incidentally refactored (e.g., rooms.invite) unless explicitly requested by maintainers.
📚 Learning: 2026-02-24T19:05:56.710Z
Learnt from: ahmed-n-abdeltwab
Repo: RocketChat/Rocket.Chat PR: 0
File: :0-0
Timestamp: 2026-02-24T19:05:56.710Z
Learning: In Rocket.Chat PRs, keep feature PRs free of unrelated lockfile-only dependency bumps; prefer reverting lockfile drift or isolating such bumps into a separate "chore" commit/PR, and always use yarn install --immutable with the Yarn version pinned in package.json via Corepack.

Applied to files:

  • skills/release-qa-regression/SKILL.md
  • skills/qa-pr/SKILL.md
📚 Learning: 2026-02-24T19:05:56.710Z
Learnt from: ahmed-n-abdeltwab
Repo: RocketChat/Rocket.Chat PR: 0
File: :0-0
Timestamp: 2026-02-24T19:05:56.710Z
Learning: Rocket.Chat repo context: When a workspace manifest on develop already pins a dependency version (e.g., packages/web-ui-registration → "rocket.chat/ui-contexts": "27.0.1"), a lockfile change in a feature PR that upgrades only that dependency’s resolution is considered a manifest-driven sync and can be kept, preferably as a small "chore: sync yarn.lock with manifests" commit.

Applied to files:

  • skills/release-qa-regression/SKILL.md
📚 Learning: 2026-04-27T18:32:24.362Z
Learnt from: d-gubert
Repo: RocketChat/Rocket.Chat PR: 40321
File: .github/workflows/ci.yml:137-145
Timestamp: 2026-04-27T18:32:24.362Z
Learning: In `.github/workflows/ci.yml` (RocketChat/Rocket.Chat), the `diff` step under `release-versions` uses `if gh pr diff ... | grep -q ...; then ... fi`. When `GH_PR_NUM` is empty (non-pull_request triggers), the `gh` command fails but the bash `if` construct handles it gracefully — the failing pipeline makes the condition false and the then-block is skipped, so the step and job exit cleanly. This pattern is intentional and does not need a guard for non-PR event types.

Applied to files:

  • skills/release-qa-regression/SKILL.md
  • skills/qa-pr/SKILL.md
📚 Learning: 2026-02-23T17:53:18.785Z
Learnt from: ggazzo
Repo: RocketChat/Rocket.Chat PR: 35995
File: apps/meteor/app/api/server/v1/rooms.ts:1107-1112
Timestamp: 2026-02-23T17:53:18.785Z
Learning: In Rocket.Chat PR reviews, maintain strict scope boundaries—when a PR is focused on a specific endpoint (e.g., rooms.favorite), avoid reviewing or suggesting changes to other endpoints that were incidentally refactored (e.g., rooms.invite) unless explicitly requested by maintainers.

Applied to files:

  • skills/qa-pr/SKILL.md
🔇 Additional comments (6)
.cursor/skills (1)

1-1: Shared skills pointer is correctly configured.

This maps Cursor skills resolution to the centralized skills/ directory as intended.

.claude/skills (1)

1-1: Pointer target is correct for Claude integration.

../skills is the expected path to the shared skills directory.

.github/skills (1)

1-1: GitHub assistant skills mapping looks good.

The file correctly points to the shared skills source.

.agents/skills (1)

1-1: Agents skills pointer is correctly aligned with the shared directory.

No concerns with this mapping.

skills/release-qa-regression/SKILL.md (1)

1-219: Strong release-regression workflow definition.

The guide is clear, testable, and well-structured, especially around range locking, confirmation criteria, and root-cause reporting.

skills/qa-pr/SKILL.md (1)

1-222: QA PR skill content is solid and actionable.

The workflow balances speed and rigor well, with clear expectations for proof, repro quality, and final output format.


Walkthrough

This change introduces pointer files in multiple agent/tool configuration directories (.agents, .claude, .cursor, .github) that reference a centralized ../skills directory, and adds two new skill documentation files defining QA workflows for PR reviews and release regression testing.

Changes

Cohort / File(s) Summary
Skill Directory Pointers
.agents/skills, .claude/skills, .cursor/skills, .github/skills
Four simple one-line pointer files that reference the centralized ../skills directory for skill configuration discovery.
Skill Documentation
skills/qa-pr/SKILL.md, skills/release-qa-regression/SKILL.md
Two new markdown documentation files defining QA workflows: one for PR/branch reviews and one for release regression testing, including methodology, templates, and standards.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested labels

type: chore

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding QA-focused AI skills (qa-pr and release-qa-regression) and exposing the skills directory to assistant configs.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 7/8 reviews remaining, refill in 7 minutes and 30 seconds.

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.85%. Comparing base (305a242) to head (4dbd974).
⚠️ Report is 1 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop   #40342      +/-   ##
===========================================
- Coverage    69.87%   69.85%   -0.03%     
===========================================
  Files         3297     3297              
  Lines       119267   119267              
  Branches     21550    21523      -27     
===========================================
- Hits         83339    83310      -29     
- Misses       32627    32658      +31     
+ Partials      3301     3299       -2     
Flag Coverage Δ
e2e 59.73% <ø> (-0.06%) ⬇️
e2e-api 46.27% <ø> (+0.05%) ⬆️
unit 70.61% <ø> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jessicaschelly jessicaschelly changed the title chore: add ai qa skills chore: add AI QA review and release regression skills Apr 30, 2026
@jessicaschelly jessicaschelly marked this pull request as ready for review April 30, 2026 16:47
@jessicaschelly jessicaschelly requested a review from a team as a code owner April 30, 2026 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant