feat: audit confidence score

[muratkeremozcan]

Add confidence-gate knowledge fragment

Introduces a new core-tier fragment defining a 1-10 confidence scoring
discipline with a stop-and-ask rule below threshold, applied to
selectors, endpoints, risk classification, fixtures, schemas, and data
factories. Protects against the dominant LLM test-generation failure
mode: plausible-looking artifacts built on fabricated evidence.

Lifted from an external Playwright AI scaffold's agent-reliability
pattern, distilled to a tool-neutral knowledge fragment so the Murat
persona can apply it without per-workflow step wiring.

Changes:

• New: src/agents/bmad-tea/resources/knowledge/confidence-gate.md
• Updated: tea-index.csv (registered, tier=core)
• Updated: test-quality.md (cross-referenced under Related fragments)

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 4633f343-4068-4b86-849c-904bfe4dc2b5

📥 Commits

Reviewing files that changed from the base of the PR and between fcee9e3 and 838adb4.

⛔ Files ignored due to path filters (1)

• src/agents/bmad-tea/resources/tea-index.csv is excluded by !**/*.csv

📒 Files selected for processing (2)

• src/agents/bmad-tea/resources/knowledge/confidence-gate.md
• src/agents/bmad-tea/resources/knowledge/test-quality.md

---

📝 Walkthrough

Walkthrough

This PR introduces a "Confidence Gate" guidance document for the bmad-tea agent knowledge base that establishes rules for generating test artifacts with explicit confidence scores and evidence-based rationale. The gate is then linked from the existing test-quality guidance as a related fragment.

Changes

Confidence Gate Guidance

Layer / File(s)

Summary

Confidence Gate documentation and cross-reference src/agents/bmad-tea/resources/knowledge/confidence-gate.md, src/agents/bmad-tea/resources/knowledge/test-quality.md

New confidence-gate.md defines required output format (Confidence, Rationale with concrete repo/contract evidence, Unknowns list), confidence thresholds for proceeding vs. stopping, application rules for different artifact types (selectors, endpoints, schemas, fixtures, factories), anti-patterns (vanity scores, ignoring unknowns, generic questions), and positive patterns. The guidance is cross-referenced in test-quality.md Integration Points section.

---

🎯 2 (Simple) | ⏱️ ~5 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'feat: audit confidence score' is clearly related to the main change—introducing a confidence-gate knowledge fragment with 1-10 confidence scoring discipline.
Description check	✅ Passed	The description is well-related to the changeset, explaining the confidence-gate fragment's purpose, scope, and origin, plus listing the specific files changed.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/bmad-tea-audit

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

• Generate code and open pull requests
• Plan features and break down work
• Investigate incidents and troubleshoot customer tickets together
• Automate recurring tasks and respond to alerts with triggers
• Summarize progress and report instantly

Built for teams:

• Shared memory across your entire org—no repeating context
• Per-thread sandboxes to safely plan and execute work
• Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}