| Config | Prompts | Judge | Use case |
|---|---|---|---|
| default.yaml | All | Enabled | Standard evaluation |
| quick_test.yaml | 5 | Disabled | Fast smoke test |
| large_scale.yaml | All | Enabled | 1000+ prompts (higher concurrency, longer timeouts) |
| foundry.yaml | — | — | Foundry cloud eval settings (graders, thresholds) |
Edit default.yaml to set your endpoint URLs, baseline model, and pricing. Environment variables (${VAR}) are resolved from .env.
judge_prompts/— Local LLM-as-judge prompt templates (absolute and pairwise)grader_prompts/— Foundry cloud grader prompt templates (quality_absolute and quality_pairwise)