Skip to content

Commit 2fb47c8

Browse files
seungrokjclaude
andcommitted
[AMD] [AGENTX] dsv4-fp4-mi355x-vllm-agentic: force lm-eval eval framework
Export EVAL_FRAMEWORK=lm-eval so run_eval never falls back to the agentic swebench default (EVAL_FRAMEWORK takes precedence over scenario_default). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 31c0d95 commit 2fb47c8

1 file changed

Lines changed: 7 additions & 0 deletions

File tree

benchmarks/single_node/agentic/dsv4_fp4_mi355x_vllm.sh

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,13 @@ set -x
2929

3030
source "$(dirname "$0")/../../benchmark_lib.sh"
3131

32+
# Force the eval framework to lm-eval for this recipe. run_eval derives its
33+
# default as swebench for agentic scenarios (scenario_default=swebench when
34+
# IS_AGENTIC/SCENARIO_TYPE=agentic-coding), but EVAL_FRAMEWORK takes precedence
35+
# over that default (benchmark_lib.sh: framework=${EVAL_FRAMEWORK:-...}), so
36+
# setting it here makes the effective framework always lm-eval, never swebench.
37+
export EVAL_FRAMEWORK="lm-eval"
38+
3239
check_env_vars MODEL TP CONC KV_OFFLOADING TOTAL_CPU_DRAM_GB RESULT_DIR DURATION EP_SIZE DP_ATTENTION
3340

3441
if [[ -n "${SLURM_JOB_ID:-}" ]]; then

0 commit comments

Comments
 (0)