Commit 92d20ff
Drop the ALE-Bench benchmark
It was the suite's only Family B task and the only one that bypassed the
compiler: a hand-written task directory with no build.yaml, so it got none of
the build-time validation the other five benchmarks get. Its optimizer was also
unmetered -- no gateway container, raw upstream credentials in main, no model
allow-list -- which is a fair-play hole in a benchmark whose whole point is
controlled measurement. Its sidecar factory had additionally gone stale against
SidecarEvaluationPolicy and would have died at startup.
The nested-backend refactor now supports `evaluation_backend: command`, so a
replacement Family B task can be built through the compiler and get the gateway
for free. examples/harbor-circle-packing carries a worked version of exactly
that shape, verified end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent 3aaecdd commit 92d20ff
17 files changed
Lines changed: 0 additions & 1007 deletions
File tree
- harness-engineering-bench
- ale-bench
- environment
- agent-baseline
- agent-seed
- main
- sidecar
- harness
- solution
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
39 | | - | |
This file was deleted.
Lines changed: 0 additions & 19 deletions
This file was deleted.
Lines changed: 0 additions & 8 deletions
This file was deleted.
Lines changed: 0 additions & 8 deletions
This file was deleted.
Lines changed: 0 additions & 60 deletions
This file was deleted.
Lines changed: 0 additions & 12 deletions
This file was deleted.
Lines changed: 0 additions & 42 deletions
This file was deleted.
Lines changed: 0 additions & 153 deletions
This file was deleted.
Lines changed: 0 additions & 27 deletions
This file was deleted.
0 commit comments