[AMD][AgentX] kimik3: force lm-eval and add an fp8 kvnone c8 eval arm

run_eval defaults agentic scenarios to swebench (benchmark_lib.sh:1464), but
EVAL_FRAMEWORK wins over that default at :1471. Export it as lm-eval the way
minimaxm3_fp4_mi355x.sh does, left overridable so EVAL_FRAMEWORK=swebench can
still reach the SWE-bench path this recipe already carries.

The eval arm runs the one config proven on this stack: fp8 KV, no offload, c8
-- the control from run 30442578333 (696 tok/s/GPU), repeated clean in
30453589555. The offload arm is deliberately excluded; it is 3/3 dead under
fp8 and is being isolated separately by the fp8b64 key.

Dispatch with --evals-only: mark_eval_entries flips single-node agentic rows
to run-eval=True (generate_sweep_configs.py:360), yielding one eval-only cell.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: hyukjlee

benchmarks/single_node/agentic/kimik3_fp4_mi355x.sh

@@ -53,6 +53,14 @@ set -x
 
 source "$(dirname "$0")/../../benchmark_lib.sh"
 
+# Force the eval framework to lm-eval, matching minimaxm3_fp4_mi355x.sh.
+# run_eval derives its default as swebench for agentic scenarios
+# (scenario_default=swebench when IS_AGENTIC/SCENARIO_TYPE=agentic-coding), but
+# EVAL_FRAMEWORK takes precedence over that default
+# (benchmark_lib.sh:1471 framework=${EVAL_FRAMEWORK:-...}). Left overridable so
+# the SWE-bench path below can still be selected with EVAL_FRAMEWORK=swebench.
+export EVAL_FRAMEWORK="${EVAL_FRAMEWORK:-lm-eval}"
+
 check_env_vars MODEL TP CONC KV_OFFLOADING TOTAL_CPU_DRAM_GB RESULT_DIR DURATION EP_SIZE
 
 if [[ -n "${SLURM_JOB_ID:-}" ]]; then

configs/amd-master.yaml

@@ -775,6 +775,29 @@ kimik3-fp4-mi355x-vllm-agentic-fp8retry:
       - { tp: 8, kv-offloading: none, conc-list: [1, 16] }
       - { tp: 8, kv-offloading: dram, kv-offload-backend: { name: vllm-simple-fp8 }, conc-list: [8] }
 
+kimik3-fp4-mi355x-vllm-agentic-fp8eval:
+  # lm-eval on the one config that is actually proven on this stack: fp8 KV,
+  # no offload, c8 -- the control cell from run 30442578333 (696 tok/s/GPU),
+  # repeated clean in 30453589555. The offload arm is excluded on purpose: it
+  # has failed 3/3 under fp8 (see the fp8b64 key below).
+  #
+  # The recipe pins EVAL_FRAMEWORK=lm-eval, so this runs lm-eval rather than
+  # run_eval's swebench default for agentic scenarios. Dispatch with
+  # --evals-only; mark_eval_entries flips single-node agentic rows to
+  # run-eval=True (generate_sweep_configs.py:360).
+  image: vllm/vllm-openai-rocm:kimi-k3
+  model: moonshotai/Kimi-K3
+  model-prefix: kimik3
+  runner: cluster:mi355x-amds
+  precision: fp4
+  framework: vllm
+  multinode: false
+  scenarios:
+    agentic-coding:
+    - dram-utilization: 0.80
+      search-space:
+      - { tp: 8, kv-offloading: none, conc-list: [8] }
+
 kimik3-fp4-mi355x-vllm-agentic-fp8b64:
   # One variable off the cell that has now failed 3/3: fp8 KV + the
   # SimpleCPUOffloadConnector, with the KV page pinned to 64 tokens.