Skip to content

Commit 214f1aa

Browse files
committed
swe-atlas-qna: route the in-container rubric judge to the upstream
The task's verifier runs inside the Modal task container and templates EVAL_BASE_URL from OPENAI_API_BASE, unreachable-from-Modal gateway URLs aside. Enable task_services_use_upstream so the judge gets the real upstream on OPENAI_*, and point the baseline agent at the metered gateway via VERO_AGENT_INFERENCE_* (falling back to OPENAI_* locally).
1 parent c89ef84 commit 214f1aa

2 files changed

Lines changed: 18 additions & 1 deletion

File tree

harness-engineering-bench/swe-atlas-qna/baseline/build.yaml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,3 +86,12 @@ inference_gateway:
8686
max_concurrency: 64
8787
instruct_multifidelity: true
8888
instruct_exhaust_budget: true
89+
# The rubric judge runs inside the Modal task container and cannot reach the
90+
# compose-internal gateway; it gets the real upstream on OPENAI_* while the
91+
# candidate agent keeps the metered gateway on VERO_AGENT_INFERENCE_*.
92+
task_services_use_upstream: true
93+
# ACCEPTED EXPOSURE (non-adversarial optimizer): with isolation off, candidate
94+
# harness code shares the sidecar uid, so it could read the upstream key and
95+
# the held-out session state. Runs using this config get a post-run leakage
96+
# audit of every candidate version. Proper fix: per-role egress isolation.
97+
harness_user: null

harness-engineering-bench/swe-atlas-qna/baseline/target/src/atlas_agent/agent.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
from __future__ import annotations
44

55
import json
6+
import os
67
from typing import Any, override
78

89
from harbor.agents.base import BaseAgent
@@ -77,7 +78,14 @@ def __init__(self, *args: Any, **kwargs: Any) -> None:
7778
if self.model_name is None:
7879
raise ValueError("SWE-Atlas agent requires a Harbor model")
7980
self._api_model = self.model_name.removeprefix("openai/")
80-
self._client = AsyncOpenAI()
81+
# The metered per-evaluation gateway arrives on dedicated variables
82+
# (OPENAI_* carries the upstream for the task's rubric judge instead).
83+
self._client = AsyncOpenAI(
84+
api_key=os.environ.get("VERO_AGENT_INFERENCE_API_KEY")
85+
or os.environ.get("OPENAI_API_KEY"),
86+
base_url=os.environ.get("VERO_AGENT_INFERENCE_BASE_URL")
87+
or os.environ.get("OPENAI_BASE_URL"),
88+
)
8189

8290
@override
8391
async def setup(self, environment: BaseEnvironment) -> None:

0 commit comments

Comments
 (0)