Skip to content

Commit 7ba3650

Browse files
User Nameclaude
andcommitted
run_microbench: --thinking collision guard (P2a)
Run names + the idempotent skip are keyed by LABEL only, so running --thinking off then on under one label silently skipped the second arm as already-complete. Mirror the reasoning-effort guard: require the mode encoded in the label (nothink / think), reject otherwise with a corrected example. Closes the last review item; this was deferred while the N=3 run was executing the script. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 500c423 commit 7ba3650

1 file changed

Lines changed: 18 additions & 0 deletions

File tree

tooling/scripts/run_microbench.sh

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,24 @@ if [ -n "$REASONING_EFFORT" ] && [[ "$LABEL" != *"$REASONING_EFFORT"* ]]; then
7676
exit 2
7777
fi
7878

79+
# Same guard for --thinking: run names are keyed by LABEL only, so running
80+
# --thinking off then on under ONE label silently skips the second arm as
81+
# "already complete". Require the mode encoded in the label (nothink / think).
82+
if [ -n "$THINKING" ]; then
83+
if [ "$THINKING" = "off" ]; then
84+
want="nothink"; [[ "$LABEL" == *nothink* ]] && ok=1 || ok=0
85+
else
86+
want="think"; [[ "$LABEL" == *think* && "$LABEL" != *nothink* ]] && ok=1 || ok=0
87+
fi
88+
if [ "$ok" != "1" ]; then
89+
echo "ERROR: --thinking '$THINKING' is set but label '$LABEL' does not encode it ('$want')." >&2
90+
echo " Run names are keyed by label only, so --thinking off then on under one label collide" >&2
91+
echo " (the second arm is skipped as already-complete). Put the mode in the label, e.g.:" >&2
92+
echo " $0 $MODEL $PORT 397b-${want} ${N} \"\" ${THINKING} ${MAXLEN}" >&2
93+
exit 2
94+
fi
95+
fi
96+
7997
TOOLING="$(cd "$(dirname "$0")/.." && pwd)"
8098
REPO_ROOT="$(cd "$TOOLING/.." && pwd)"
8199
cd "$REPO_ROOT"

0 commit comments

Comments
 (0)