Skip to content

Commit 87d2691

Browse files
varunursekarclaude
andcommitted
Bump harbor to 0.20.0 across benchmark configs and regenerate locks
Move each benchmark to harbor 0.20.0 in lockstep so the eval resolves: build.yaml harbor_requirement (the evaluator's --with harbor[modal]) and target/pyproject.toml (the candidate's own harbor dep) must match, else `uv run --project <target> --with harbor[modal]==0.20.0` is unsatisfiable. uv.lock regenerated with `uv lock` (not hand-edited). Covers gaia, officeqa, swe-atlas-qna, tau3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent c86351e commit 87d2691

12 files changed

Lines changed: 24 additions & 24 deletions

File tree

harness-engineering-bench/gaia/baseline/build.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ agent_repo: target
66
task_source: gaia/gaia@sha256:bbc356f476e0b70ba77da11a9be7d6345918d1e4a2daade0d6dfb82ee6f7b761
77
task_manifest: ../partitions/manifest.json
88
agent_import_path: gaia_agent.agent:GaiaAgent
9-
harbor_requirement: harbor[modal]==0.18.0
9+
harbor_requirement: harbor[modal]==0.20.0
1010

1111
partition_files:
1212
development: ../partitions/development.json

harness-engineering-bench/gaia/baseline/target/pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ version = "0.1.0"
44
description = "Editable Harbor-native GAIA baseline for VeRO"
55
requires-python = ">=3.12"
66
dependencies = [
7-
"harbor==0.18.0",
7+
"harbor==0.20.0",
88
"openai==2.46.0",
99
]
1010

harness-engineering-bench/gaia/baseline/target/uv.lock

Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

harness-engineering-bench/officeqa/baseline/build.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ agent_repo: target
1212
task_source: ../tasks
1313
task_manifest: ../partitions/manifest.json
1414
agent_import_path: officeqa_agent.agent:OfficeQaAgent
15-
harbor_requirement: harbor[modal]==0.18.0
15+
harbor_requirement: harbor[modal]==0.20.0
1616
extra_harbor_args: ["--no-force-build"]
1717

1818
partition_files:

harness-engineering-bench/officeqa/baseline/target/pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ version = "0.1.0"
44
description = "Editable Harbor-native GAIA baseline for VeRO"
55
requires-python = ">=3.12"
66
dependencies = [
7-
"harbor==0.18.0",
7+
"harbor==0.20.0",
88
"openai==2.46.0",
99
]
1010

harness-engineering-bench/officeqa/baseline/target/uv.lock

Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

harness-engineering-bench/swe-atlas-qna/baseline/build.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ agent_repo: target
77
task_source: scale-ai/swe-atlas-qna@sha256:0e26bc0313ae2fc6f912b67b928e648c7f20d17d91f765f702a93042ce5be0e4
88
task_manifest: ../partitions/manifest.json
99
agent_import_path: atlas_agent.agent:AtlasAgent
10-
harbor_requirement: harbor[modal]==0.18.0
10+
harbor_requirement: harbor[modal]==0.20.0
1111

1212
partition_files:
1313
development: ../partitions/development.json

harness-engineering-bench/swe-atlas-qna/baseline/target/pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ version = "0.1.0"
44
description = "Editable Harbor-native SWE-Atlas-QnA baseline for VeRO"
55
requires-python = ">=3.12"
66
dependencies = [
7-
"harbor==0.18.0",
7+
"harbor==0.20.0",
88
"openai==2.46.0",
99
]
1010

harness-engineering-bench/swe-atlas-qna/baseline/target/uv.lock

Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

harness-engineering-bench/tau3/baseline/build.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ agent_repo: target
66
task_source: sierra-research/tau3-bench@sha256:a57304f682894ac061090769af771a3617664f3ff6e5417d4eadf8e30433e4d9
77
task_manifest: ../partitions/manifest.json
88
agent_import_path: tau3_agent.agent:Tau3Agent
9-
harbor_requirement: harbor[modal]==0.18.0
9+
harbor_requirement: harbor[modal]==0.20.0
1010

1111
partition_files:
1212
development: ../partitions/development.json

0 commit comments

Comments
 (0)