fix: point swe-atlas-qna at a provisioned model and off the DinD SPOF - #58

Closed

shehabyasser-scale wants to merge 15 commits into

feat/swe-bench-pro-baseline-scaffoldfrom

fix/swe-atlas-qna-provisioning

Commits on Jul 24, 2026

Add harness-engineering-bench (benchmarks at top level)
varunursekar
committed
gaia: baseline_floor off (feedback #1)
varunursekar
committed
gaia: re-enable harness isolation (harness_user: harness)
varunursekar
committed
Bump harbor to 0.20.0 across benchmark configs and regenerate locks

varunursekar
and
claude
committed
Benchmarks: adopt .evals context naming and the evals CLI

varunursekar
and
claude
committed
Benchmarks: run inner eval sandboxes in a dedicated Modal app
varunursekar
committed
swe-atlas-qna: submit-mode selection, matching gaia
varunursekar
committed
Benchmarks: enable W&B reporting on officeqa, swe-atlas-qna, and tau3
varunursekar
committed
swe-atlas-qna: route the in-container rubric judge to the upstream
varunursekar
committed
officeqa: merge duplicate extra_harbor_args keys
varunursekar
committed
officeqa: submit-mode selection, matching gaia
varunursekar
committed
Benchmarks: bound orphaned infrastructure from killed runs
varunursekar
committed
feat: scaffold SWE-bench-Pro baseline for the vero optimizer

shehabyasser-scale
and
claude
committed

Commits on Jul 25, 2026