Commit e34abbb
Feat/watchdog (#64)
* feat: add Watchdog LLM-as-judge feature
Adds Watchdog, an LLM-based evaluation layer for AI agent tool calls.
Watchdog provides semantic security analysis that complements Rego policies.
Key features:
- Backend abstraction trait for swappable LLM providers
- OpenRouter backend implementation (HTTP API)
- Directory-based configuration (.cupcake/watchdog/)
- Custom prompts via system.txt and user.txt files
- Dry-run mode for testing without API calls
- Fail-open/fail-closed error handling
Configuration:
- Enable in rulebook.yml: `watchdog: true`
- Settings in .cupcake/watchdog/config.json
- Precedence: project -> global -> defaults
Policy integration:
- Results available at input.signals.watchdog
- Fields: allow, confidence, reasoning, concerns, suggestions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* watchdog context, image in docs
* add cucpake flow diagram, better language
* refine readme
* refine readme
* refine readme
* refine readme
* refine readme
* refine readme
* chore: ci fixes
* code and design quality fixes
* png to avif
* sad revert to png
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent c5d9151 commit e34abbb
41 files changed
Lines changed: 3344 additions & 38 deletions
File tree
- cupcake-cli
- src
- cupcake-core
- src
- engine
- watchdog
- tests
- fixtures/watchdog/config-setup
- global_only/global
- mixed_sources
- global
- project
- project_only/project
- project_overrides_global
- global
- project
- rules_context
- project
- docs
- docs
- assets
- reference
- watchdog
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
| 105 | + | |
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments