Skip to content

Commit 31d8b79

Browse files
varunursekarclaude
andcommitted
Let uv-installed optimizer harnesses install as the unprivileged agent user
`uv tool install` symlinks a harness's entry point into /usr/local/bin, which the optimizer user cannot write, so the install step dies with "Failed to install executable ... Permission denied (os error 13)" and the trial ends in NonZeroAgentExitCodeError before any evaluation runs. Pointing UV_TOOL_BIN_DIR at the user's own bin directory fixes it. This blocks both literature-standard harnesses -- mini-swe-agent and swe-agent are each installed this way -- while claude-code and opencode are npm/nvm-based and never hit it, so the failure only appears when the harness dimension of the grid changes, and reads like "this harness does not work with vero" rather than a container-permissions problem. Found by vero/examples/harness-conformance in 91 seconds; mini-swe-agent then scored 1.0 on the same check in 4m57s, with 25 producer requests all 200 on chat/completions and no denials. Applied to all five benchmarks rather than the one about to be run, since the first mini-swe-agent cell of the grid would otherwise hit it whichever benchmark came first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent b40bbd2 commit 31d8b79

5 files changed

Lines changed: 30 additions & 0 deletions

File tree

harness-engineering-bench/browsecomp-plus/baseline/build.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,12 @@ agent_env:
135135
BASH_DEFAULT_TIMEOUT_MS: "14400000" # same as max: an un-timed eval must still block
136136
ENABLE_BACKGROUND_TASKS: "0"
137137
FORCE_AUTO_BACKGROUND_TASKS: "0"
138+
# Harnesses installed with `uv tool install` (mini-swe-agent, swe-agent)
139+
# default to symlinking their entry point into /usr/local/bin, which the
140+
# unprivileged optimizer user cannot write: "Failed to install executable
141+
# ... Permission denied". npm/nvm-based harnesses (claude-code, opencode)
142+
# are unaffected, so this only bites when the harness changes.
143+
UV_TOOL_BIN_DIR: "/home/agent/.local/bin"
138144

139145
task_environment:
140146
BROWSECOMP_JUDGE_MODEL: gpt-4.1

harness-engineering-bench/gaia/baseline/build.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,12 @@ agent_env:
111111
BASH_DEFAULT_TIMEOUT_MS: "3600000" # same as max: an un-timed eval must still block
112112
ENABLE_BACKGROUND_TASKS: "0"
113113
FORCE_AUTO_BACKGROUND_TASKS: "0"
114+
# Harnesses installed with `uv tool install` (mini-swe-agent, swe-agent)
115+
# default to symlinking their entry point into /usr/local/bin, which the
116+
# unprivileged optimizer user cannot write: "Failed to install executable
117+
# ... Permission denied". npm/nvm-based harnesses (claude-code, opencode)
118+
# are unaffected, so this only bites when the harness changes.
119+
UV_TOOL_BIN_DIR: "/home/agent/.local/bin"
114120

115121

116122
wandb:

harness-engineering-bench/officeqa/baseline/build.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,12 @@ agent_env:
117117
BASH_DEFAULT_TIMEOUT_MS: "10800000" # same as max: an un-timed eval must still block
118118
ENABLE_BACKGROUND_TASKS: "0" # gates auto-backgrounding only (see above)
119119
FORCE_AUTO_BACKGROUND_TASKS: "0"
120+
# Harnesses installed with `uv tool install` (mini-swe-agent, swe-agent)
121+
# default to symlinking their entry point into /usr/local/bin, which the
122+
# unprivileged optimizer user cannot write: "Failed to install executable
123+
# ... Permission denied". npm/nvm-based harnesses (claude-code, opencode)
124+
# are unaffected, so this only bites when the harness changes.
125+
UV_TOOL_BIN_DIR: "/home/agent/.local/bin"
120126

121127
wandb:
122128
project: harness-engineering-bench

harness-engineering-bench/swe-atlas-qna/baseline/build.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,4 +142,10 @@ agent_env:
142142
BASH_DEFAULT_TIMEOUT_MS: "39600000" # same as max: an un-timed eval must still block
143143
ENABLE_BACKGROUND_TASKS: "0"
144144
FORCE_AUTO_BACKGROUND_TASKS: "0"
145+
# Harnesses installed with `uv tool install` (mini-swe-agent, swe-agent)
146+
# default to symlinking their entry point into /usr/local/bin, which the
147+
# unprivileged optimizer user cannot write: "Failed to install executable
148+
# ... Permission denied". npm/nvm-based harnesses (claude-code, opencode)
149+
# are unaffected, so this only bites when the harness changes.
150+
UV_TOOL_BIN_DIR: "/home/agent/.local/bin"
145151

harness-engineering-bench/tau3/baseline/build.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,12 @@ agent_env:
139139
BASH_DEFAULT_TIMEOUT_MS: "32400000" # same as max: an un-timed eval must still block
140140
ENABLE_BACKGROUND_TASKS: "0"
141141
FORCE_AUTO_BACKGROUND_TASKS: "0"
142+
# Harnesses installed with `uv tool install` (mini-swe-agent, swe-agent)
143+
# default to symlinking their entry point into /usr/local/bin, which the
144+
# unprivileged optimizer user cannot write: "Failed to install executable
145+
# ... Permission denied". npm/nvm-based harnesses (claude-code, opencode)
146+
# are unaffected, so this only bites when the harness changes.
147+
UV_TOOL_BIN_DIR: "/home/agent/.local/bin"
142148

143149
task_environment:
144150
TAU2_USER_MODEL: openai/gpt-5.4-mini-2026-03-17

0 commit comments

Comments
 (0)