Commit b81e456
feat(studio): add Run Eval flow with suite filter, test-id filter, and target override (#947)
- Add eval-runner.ts: Hono API endpoints for discovery, launch, and status polling
- GET /api/eval/discover: discovers eval files in project
- GET /api/eval/targets: lists available target names
- POST /api/eval/run: spawns CLI eval process with validated args
- GET /api/eval/status/:id: polls running eval status
- POST /api/eval/preview: generates CLI command preview
- All endpoints also available project-scoped under /api/projects/:projectId/eval/*
- Add RunEvalModal component: two-step wizard modal
- Step 1: suite filter (text input with discovered file suggestions), test-id
pills (repeatable with glob support), target override (searchable dropdown)
- Step 2: advanced options (threshold, workers, dry-run) collapsed by default
- Live CLI preview before launch
- Run status view with stdout/stderr streaming after launch
- Add entry points on every relevant page:
- Home page (both single-project and multi-project): 'Run Eval' button
- Run detail page: 'Re-run with Filters' (prefilled with current target)
- Eval detail page: 'Run this Test' (prefilled with test ID and target)
- All project-scoped variants included
Closes #945
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 312cee3 commit b81e456
11 files changed
Lines changed: 1249 additions & 25 deletions
File tree
- apps
- cli/src/commands/results
- studio/src
- components
- lib
- routes
- evals
- projects
- $projectId_
- evals
- runs
- runs
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
| 49 | + | |
49 | 50 | | |
50 | 51 | | |
51 | 52 | | |
| |||
892 | 893 | | |
893 | 894 | | |
894 | 895 | | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
895 | 908 | | |
896 | 909 | | |
897 | 910 | | |
| |||
0 commit comments