Skip to content

Commit 45a7a91

Browse files
christsoclaude
andcommitted
feat(ci): add GH Models default target with copilot-cli/copilot-sdk
Update .agentv/targets.yaml: - Change default target to GitHub Models (openai provider, models.github.ai) - Add copilot-cli and copilot-sdk targets using GH_MODELS_MODEL - Keep existing pi, codex, gemini, openai, openrouter targets Update evals workflow: - Restore Copilot CLI install step - Write .env with GH_MODELS_TOKEN/GH_MODELS_MODEL (targets.yaml references these) - Remove inline targets.yaml generation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent bb57287 commit 45a7a91

2 files changed

Lines changed: 25 additions & 19 deletions

File tree

.agentv/targets.yaml

Lines changed: 16 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,22 @@
33

44
targets:
55
- name: default
6-
provider: pi-coding-agent
7-
subprovider: openrouter
8-
model: z-ai/glm-4.7
9-
api_key: ${{ OPENROUTER_API_KEY }}
10-
system_prompt: "Answer directly based on the information provided."
11-
grader_target: gemini-flash
6+
provider: openai
7+
base_url: https://models.github.ai/inference
8+
api_key: ${{ GH_MODELS_TOKEN }}
9+
model: ${{ GH_MODELS_MODEL }}
10+
11+
- name: copilot-cli
12+
provider: copilot-cli
13+
model: ${{ GH_MODELS_MODEL }}
14+
grader_target: default
15+
log_format: json
16+
17+
- name: copilot-sdk
18+
provider: copilot-sdk
19+
model: ${{ GH_MODELS_MODEL }}
20+
grader_target: default
21+
log_format: json
1222

1323
- name: pi-cli
1424
provider: pi-cli

.github/workflows/evals.yml

Lines changed: 9 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -23,22 +23,15 @@ jobs:
2323
- name: Build
2424
run: bun run build
2525

26-
- name: Configure targets
27-
run: |
28-
mkdir -p .agentv
29-
cat > .agentv/targets.yaml <<'TARGETS'
30-
targets:
31-
- name: default
32-
provider: openai
33-
base_url: https://models.github.ai/inference
34-
api_key: ${{ GH_MODELS_TOKEN }}
35-
model: ${{ GH_MODELS_MODEL }}
36-
TARGETS
26+
- name: Install GitHub Copilot CLI
27+
run: curl -fsSL https://gh.io/copilot-install | bash
3728

38-
cat > .env <<ENV
29+
- name: Configure credentials
30+
run: |
31+
cat > .env <<EOF
3932
GH_MODELS_TOKEN=${{ secrets.GH_MODELS_TOKEN || secrets.GITHUB_TOKEN }}
4033
GH_MODELS_MODEL=${{ vars.GH_MODELS_MODEL || 'openai/gpt-5-mini' }}
41-
ENV
34+
EOF
4235
4336
- name: Resolve filter and threshold
4437
id: filter
@@ -48,6 +41,9 @@ jobs:
4841
4942
- name: Run AgentV evals
5043
id: run-evals
44+
env:
45+
COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_PAT }}
46+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
5147
run: |
5248
mkdir -p .agentv/ci-results
5349

0 commit comments

Comments
 (0)