Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ AZURE_MODEL_ROUTER_DEPLOYMENT=model-router
# Azure OpenAI endpoint (for baseline model)
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
AZURE_OPENAI_KEY=your-azure-openai-api-key
# Must match a key in the pricing section of your config file (e.g. gpt-4o, gpt-5).
# If you use a custom deployment name, add a matching entry to configs/default.yaml
# under pricing: using your exact deployment name. Otherwise baseline costs show as $0.00.
Comment thread
leestott marked this conversation as resolved.
Outdated
AZURE_BASELINE_DEPLOYMENT=gpt-5

# Judge model (defaults to same endpoint as baseline; override to use a different model)
Expand Down
2 changes: 2 additions & 0 deletions QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,8 @@ copy .env.example .env # Windows
```
Open `.env` and set the four endpoint URLs and API keys (router, baseline, judge, optional Foundry project). The file is in `.gitignore`, so your secrets won't be committed.

> **Note:** The value you set for `AZURE_BASELINE_DEPLOYMENT` must match a key in the `pricing` section of the config file you run (e.g. `configs/default.yaml`). If you use a custom deployment name such as `gpt-4o-baseline`, add a matching entry under `pricing:` in that same config with the same input/output rates; otherwise baseline costs will show as $0.00 and the cost comparison will be inaccurate.

### 2. Pick or edit a config
- `configs/quick_test.yaml` — small, fast (~10 prompts) — good first run.
- `configs/default.yaml` — full benchmark (100 prompts).
Expand Down
Loading