Skip to content

Commit 7d88227

Browse files
TheTommeh
authored andcommitted
fix(kv-cache): unblock per-side ROT_OVERRIDE knobs (default attn_rot_disable=false)
Previous state: db3595a added LLAMA_ATTN_ROT_K_OVERRIDE / _V_OVERRIDE per-side opt-in knobs but kept attn_rot_disable defaulting to TRUE for legacy LLAMA_ATTN_ROT_DISABLE compatibility. The override branches included `&& !attn_rot_disable` guards, so when LLAMA_ATTN_ROT_DISABLE is unset (default true) the per-side env knobs were silently no-ops. Users could not opt into rotation without also setting LLAMA_ATTN_ROT_DISABLE=0. Fix: flip attn_rot_disable default to false. Rotation is still OFF by default because attn_rot_k/v default to false. LLAMA_ATTN_ROT_DISABLE=1 still acts as a hard lock-out that blocks the per-side overrides for users who want a single switch to guarantee no rotation. Caught while running the cross-format KLD matrix for the rotation/PPL investigation paper — V-only override appeared to silently fail. Confirmed with logs that attn_rot_v stayed 0 even with LLAMA_ATTN_ROT_V_OVERRIDE=1 until this default flip.
1 parent 9a8f69e commit 7d88227

1 file changed

Lines changed: 5 additions & 1 deletion

File tree

src/llama-kv-cache.cpp

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -489,8 +489,12 @@ llama_kv_cache::llama_kv_cache(
489489
//
490490
// LLAMA_ATTN_ROT_DISABLE retained as a no-op alias (default OFF makes it
491491
// redundant but historical scripts may set it).
492+
// Default attn_rot_disable=false now that rotation is OFF by default. The
493+
// env var is preserved as a hard lock-out (=1 forces rotation off and
494+
// blocks overrides), useful for users who want to guarantee no rotation
495+
// regardless of any LLAMA_ATTN_ROT_*_OVERRIDE settings.
492496
const char * LLAMA_ATTN_ROT_DISABLE = getenv("LLAMA_ATTN_ROT_DISABLE");
493-
const bool attn_rot_disable = LLAMA_ATTN_ROT_DISABLE ? atoi(LLAMA_ATTN_ROT_DISABLE) : true;
497+
const bool attn_rot_disable = LLAMA_ATTN_ROT_DISABLE ? (atoi(LLAMA_ATTN_ROT_DISABLE) != 0) : false;
494498

495499
// Default: rotation OFF on both sides (safe across all tested model families).
496500
// Override per side via env vars below.

0 commit comments

Comments
 (0)