feat(swebench-pro): initial support for ScaleAI/SWE-bench_Pro

Adds an agent execution pipeline for SWE-bench Pro mirroring the
existing Verified pipeline. Key differences from Verified: Docker images
come from jefzda/sweap-images (tag from the dataset's dockerhub_tag
field), evaluation runs per-instance run_script.sh/parser.py scripts
from the SWE-bench_Pro-os submodule rather than the swebench library,
and grading checks fail_to_pass ∪ pass_to_pass against output.json.

New files:
- src/tasks/swe_bench_pro.py — dataset adapter (ScaleAI/SWE-bench_Pro)
- src/agents/swe_bench_pro_environment.py — Modal environment with Pro eval
- run_swebench_pro_agent.py — agent runner
- slurm/swebench_pro_run.sh — SLURM launcher
- configs/tasks/swe_bench_pro.yaml
- configs/runs/qwen36_27b_swebench_pro_{test,full}.yaml
- tests/test_swebench_pro_evaluate_modal.py
- SWE-bench_Pro-os submodule (scaleapi/SWE-bench_Pro-os)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: andre15silva

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "SWE-bench_Pro-os"]
+	path = SWE-bench_Pro-os
+	url = https://github.com/scaleapi/SWE-bench_Pro-os

SWE-bench_Pro-os

@@ -0,0 +1,41 @@
+# Full run: Qwen3.6-27B on all 731 SWE-bench Pro instances, 8xA100 80GB
+#
+# Submit as 4-shard array job:
+#   sbatch --array=0-3 -t 02:00:00 slurm/swebench_pro_run.sh \
+#     --run-config configs/runs/qwen36_27b_swebench_pro_full.yaml
+
+# --- Model / vLLM ---
+model_id: Qwen/Qwen3.6-27B
+served_model_name: qwen36-27b
+tensor_parallel_size: 8
+max_model_len: 262144
+dtype: bfloat16
+gpu_memory_utilization: 0.9
+vllm_port: 18000
+vllm_startup_timeout_s: 900
+vllm_log_path: logs/vllm_qwen36_27b_swebench_pro_full.log
+vllm_extra_args:
+  - "--reasoning-parser"
+  - "qwen3"
+  - "--enable-auto-tool-choice"
+  - "--tool-call-parser"
+  - "qwen3_coder"
+
+# --- Generation ---
+temperature: 1.0
+top_p: 0.95
+
+# --- Agent ---
+step_limit: 100
+command_timeout: 180
+
+# --- Execution ---
+n_runs: 1
+n_instances: -1          # all instances in this shard
+n_workers: 8
+modal_app_name: program-probes-swebench-pro
+modal_timeout: 3600
+output_dir: generations/swebench_pro/qwen36_27b_full
+
+# --- SWE-bench Pro ---
+swe_bench_pro_scripts_dir: SWE-bench_Pro-os

configs/runs/qwen36_27b_swebench_pro_full.yaml

@@ -0,0 +1,41 @@
+# Test run: Qwen3.6-27B on 16 SWE-bench Pro instances, 8xA100 80GB
+#
+# Submit with:
+#   sbatch slurm/swebench_pro_run.sh --run-config configs/runs/qwen36_27b_swebench_pro_test.yaml
+
+# --- Model / vLLM ---
+model_id: Qwen/Qwen3.6-27B
+served_model_name: qwen36-27b
+tensor_parallel_size: 8
+max_model_len: 262144
+dtype: bfloat16
+gpu_memory_utilization: 0.9
+vllm_port: 18000
+vllm_startup_timeout_s: 900
+vllm_log_path: logs/vllm_qwen36_27b_swebench_pro.log
+vllm_extra_args:
+  - "--reasoning-parser"
+  - "qwen3"
+  - "--enable-auto-tool-choice"
+  - "--tool-call-parser"
+  - "qwen3_coder"
+
+# --- Generation ---
+temperature: 1.0
+top_p: 0.95
+# max_new_tokens: not set — no per-call cap, lets thinking models use the full remaining context
+
+# --- Agent ---
+step_limit: 100
+command_timeout: 180     # seconds per bash command in Modal sandbox
+
+# --- Execution ---
+n_runs: 1                # one attempt per instance
+n_instances: 16          # stop after 16 unique instances
+n_workers: 16            # 16 parallel agent threads; vLLM batches their requests
+modal_app_name: program-probes-swebench-pro
+modal_timeout: 1800      # seconds for the full sandbox lifecycle
+output_dir: generations/swebench_pro/qwen36_27b_test
+
+# --- SWE-bench Pro ---
+swe_bench_pro_scripts_dir: SWE-bench_Pro-os   # local clone of github.com/scaleapi/SWE-bench_Pro-os

configs/runs/qwen36_27b_swebench_pro_test.yaml

@@ -0,0 +1,3 @@
+dataset: ScaleAI/SWE-bench_Pro
+adapter: swe_bench_pro
+execution_timeout: 300