Skip to content

Commit a4e956a

Browse files
test: cover swe-bench-pro in the benchmark-config invariants
The config-invariant suite enumerates benchmarks explicitly, so swe-bench-pro was silently unchecked. Now that its task_source resolves, it can join the other five. All three parametrized cases pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 3d58ea7 commit a4e956a

1 file changed

Lines changed: 8 additions & 1 deletion

File tree

vero/tests/test_v05_benchmark_configs.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,14 @@
2626

2727
BENCHMARK_ROOT = Path(__file__).resolve().parents[2] / "harness-engineering-bench"
2828

29-
BENCHMARKS = ["gaia", "officeqa", "swe-atlas-qna", "tau3", "browsecomp-plus"]
29+
BENCHMARKS = [
30+
"gaia",
31+
"officeqa",
32+
"swe-atlas-qna",
33+
"tau3",
34+
"browsecomp-plus",
35+
"swe-bench-pro",
36+
]
3037

3138
# Names that would let a task reach the upstream provider directly, bypassing
3239
# the gateway's allow-list and budget.

0 commit comments

Comments
 (0)