swe-atlas-qna: keep Modal sandboxes alive under the images' ENTRYPOINT

Root cause of every swe-atlas trial dying at setup: the task images set
ENTRYPOINT ["/bin/bash"], Modal prepends it to the sandbox command, so
harbor's default keepalive ["sh","-c","sleep infinity"] ran as
'bash sh -c ...' - bash interprets the sh binary as a script and exits
126, killing the sandbox before the first exec. Pass bash's own args
via the JSON-decoded keepalive env kwarg instead. Validated end-to-end
on Modal (3 dev trials, 379-602s wall). CONFIGURATION.md records the
empirical probe timings behind the provisional case timeouts.

Author: varunursekar

harness-engineering-bench/CONFIGURATION.md

@@ -54,7 +54,7 @@ benchmark can be checked against the others at a glance.
 | verifier_timeout_seconds | 7200 | 14400 | 28800 | 28800 |
 | harness_user | harness | harness | null ‡ | null ‡ |
 | task_services_use_upstream | false | false | true (rubric judge) | true (user-sim + grader) |
-| task-specific extras | — | `--no-force-build` (prebuilt corpus image) | — | `TAU2_*` model pins |
+| task-specific extras | — | `--no-force-build` (prebuilt corpus image) | `keepalive` --ek (ENTRYPOINT images) | `TAU2_*` model pins |
 
 ## Conventions
 
@@ -67,9 +67,10 @@ benchmark can be checked against the others at a glance.
   candidate and the baseline test evaluations.
 - **Case budgets** are 4× the partition size, i.e. four full passes.
 
-† Provisional. These enforced budgets were set without empirical data;
-revisit them against the `wall_seconds` / `mean_case_wall_seconds` metrics
-once runs have produced distributions.
+† Sized from stock-agent probes (codex on the target model, 3 development
+tasks each, full declared timeouts): tau3 trials took 202-211s (900s budget
+≈ 4x headroom) and swe-atlas trials 379-602s (1800s ≈ 3x headroom over the
+slowest). Revisit against `wall_seconds` distributions from real runs.
 
 ‡ Exception to the harness-isolation default: these tasks run LLM services
 (rubric judge, user-simulator/grader) inside their task containers, which

harness-engineering-bench/swe-atlas-qna/baseline/build.yaml

@@ -47,8 +47,12 @@ rescore_attempts: 1
 
 model: openai/gpt-5.4-mini-2026-03-17
 environment_name: ${inner_env:-modal}
-# inner eval sandboxes share a dedicated Modal app instead of the __harbor__ default
-extra_harbor_args: ["--ek", "app_name=harness-engineering-bench", "--ek", "sandbox_idle_timeout_secs=3600"]
+# inner eval sandboxes share a dedicated Modal app instead of the __harbor__ default.
+# keepalive: these images set ENTRYPOINT ["/bin/bash"], which Modal prepends to the
+# sandbox command; harbor's default keepalive ["sh","-c","sleep infinity"] then dies
+# instantly (bash interprets the sh binary as a script) and every trial fails with
+# "Sandbox already shut down". Passing bash's own args keeps the sandbox alive.
+extra_harbor_args: ["--ek", "app_name=harness-engineering-bench", "--ek", "sandbox_idle_timeout_secs=3600", "--ek", 'keepalive=["-c","sleep infinity"]']
 harbor_python_version: "3.12"
 n_attempts: 1
 max_retries: 1