Skip to content

Commit ac3aee3

Browse files
test: cover swe-bench-pro in the benchmark-config invariants
The config-invariant suite enumerates benchmarks explicitly, so swe-bench-pro was silently unchecked. Now that its task_source resolves, it can join the other five. All three parametrized cases pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 91421cc commit ac3aee3

1 file changed

Lines changed: 8 additions & 1 deletion

File tree

vero/tests/test_v05_benchmark_configs.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,14 @@
2020

2121
BENCHMARK_ROOT = Path(__file__).resolve().parents[2] / "harness-engineering-bench"
2222

23-
BENCHMARKS = ["gaia", "officeqa", "swe-atlas-qna", "tau3", "browsecomp-plus"]
23+
BENCHMARKS = [
24+
"gaia",
25+
"officeqa",
26+
"swe-atlas-qna",
27+
"tau3",
28+
"browsecomp-plus",
29+
"swe-bench-pro",
30+
]
2431

2532
# Names that would let a task reach the upstream provider directly, bypassing
2633
# the gateway's allow-list and budget.

0 commit comments

Comments
 (0)