mt:: + server — wipe tier state on whole-seq seq_rm (agent task lifecycle)

Setup for the agent army: long-running workers reuse a slot for many
unrelated tasks throughout the day. Without explicit per-task cleanup,
two failure modes accumulate:

1. Memory leak: warm slots from prior tasks stay occupied (the existing
   inline comment at the seq_rm site even acknowledged this — "we only
   free a warm slot when the warm copy itself becomes obsolete, e.g.
   via clear()" — but clear() only fires on whole-cache wipe, never per
   slot).

2. Worse: stale semantic fingerprints from prior tasks persist in the
   SemanticIndex. A new task's query embedding can cosine-match against
   a prior task's chunk and trigger restoration of that chunk's K/V
   from warm/cold — attention then reads garbage, model confabulates.
   This silent-failure mode would be hard to debug in production.

Fix in two pieces:

- mt::seq_rm: when called with the whole-seq sentinel (p0<0 or p1<0,
  i.e. "task done, wipe the slot"), clear warm_pos_to_slot_,
  evicted_to_warm_, warm_insertion_order_, cold_positions_,
  warm_recur_buf_, semantic_, refill warm_free_slots_, reset
  pressure_announced_. Logs the wipe count for visibility. n_seq_max=1
  today so the wipe is global; will need per-seq scoping when mt::
  grows multi-seq support.

- server-context.cpp: self-healing kv_evict_through cursor in the
  proactive eviction trigger. When the cursor exceeds n_tokens, the
  slot was truncated (new task with no shared prefix wipes the cache
  via memory_seq_rm); reset to 0. Avoids having to instrument every
  memory_seq_rm site in the server.

Smoke (Qwen3.6-35B-A3B-Q5, --kv-tier-semantic-index bge-small.gguf,
--ctx-size 8192). Two-task isolation test with distinct needles:

  TASK A (drop NEEDLE_A, force eviction):
    backup [0,409) -> fingerprint, backup [409,818) -> fingerprint
    restore_semantic: 1 hint, 409 positions restored

  IMPLICIT WIPE (task B's prompt has no shared prefix):
    mt::seq_rm: whole-seq wipe for seq 0 — freed 818 warm slots,
                dropped 0 cold entries, cleared 2 semantic fingerprints

  TASK B (drop NEEDLE_B, ask for code):
    backup [0,409) -> fingerprint (FRESH content), backup [409,818)
    restore_semantic: 1 hint, 409 positions restored
    Model answer: "MERIDIAN-4920-SILVERFROST-PEREGRINE" (NEEDLE_B)
    NEEDLE_A nowhere in answer

Without the fix, task B either 500'd ("Context size has been exceeded"
because the stale cursor blocked eviction) or — worse — recalled
NEEDLE_A from a stale fingerprint match.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Author: kmbandy

src/memory-tier/mt-tiered.cpp

@@ -936,15 +936,54 @@ bool llama_memory_tiered::seq_rm(llama_seq_id seq_id, llama_pos p0, llama_pos p1
     if (ok) {
         // Drop the removed positions from the eviction store. We keep
         // the warm copies — those are now the only source of truth for
-        // restoration. (warm_pos_to_slot_ is unchanged here; we only
-        // free a warm slot when the warm copy itself becomes obsolete,
-        // e.g. via clear() or when the same position is re-added then
-        // re-removed.)
+        // restoration.
         if (p0 >= 0 && p1 > p0) {
             for (llama_pos p = p0; p < p1; ++p) {
                 eviction_.remove(p);
             }
         }
+
+        // Whole-seq wipe (sentinel p0=-1 or p1=-1) is the "task done"
+        // signal for an agent-style worker reusing a slot for unrelated
+        // tasks. Free warm slots so VRAM/RAM is reclaimed, drop cold-tier
+        // index entries (the on-disk file persists as scratch and gets
+        // overwritten by the next eviction), and crucially wipe the
+        // semantic fingerprint index — without this, fingerprints from
+        // the prior task can cosine-match against the new task's queries
+        // and trigger restoration of stale K/V from warm/cold, which
+        // attention would then read as garbage.
+        //
+        // n_seq_max=1 today so "for this seq" === "all of it." When mt::
+        // grows multi-seq tracking, scope these wipes by seq_id.
+        if (p0 < 0 || p1 < 0) {
+            const size_t n_warm   = warm_pos_to_slot_.size();
+            const size_t n_cold   = cold_positions_.size();
+            const size_t n_finger = semantic_.size();
+
+            warm_pos_to_slot_.clear();
+            evicted_to_warm_.clear();
+            warm_insertion_order_.clear();
+            cold_positions_.clear();
+            warm_recur_buf_.clear();
+            semantic_.clear();
+            pressure_announced_ = false;
+
+            if (warm_initialized_) {
+                warm_free_slots_.clear();
+                warm_free_slots_.reserve(warm_capacity_);
+                for (uint32_t s = warm_capacity_; s-- > 0; ) {
+                    warm_free_slots_.push_back((int) s);
+                }
+            }
+
+            if (n_warm + n_cold + n_finger > 0) {
+                LLAMA_LOG_INFO("mt::seq_rm: whole-seq wipe for seq %d — freed "
+                               "%zu warm slots, dropped %zu cold entries, "
+                               "cleared %zu semantic fingerprints\n",
+                               seq_id, n_warm, n_cold, n_finger);
+            }
+        }
+
         update_tier_state();
     }
     return ok;

tools/server/server-context.cpp

@@ -1490,6 +1490,17 @@ struct server_context_impl {
             }
 
             const int evict_threshold = (int)(cap * 0.80f);
+
+            // If the slot was truncated (new task with no shared prefix
+            // wipes the inner cache via memory_seq_rm), kv_evict_through
+            // is stale — it points past the now-much-smaller live cache.
+            // Reset to start eviction from scratch on this fresh slot.
+            // Self-heals without having to instrument every memory_seq_rm
+            // site in the server.
+            if ((int)slot.kv_evict_through > n_tokens) {
+                slot.kv_evict_through = 0;
+            }
+
             // Live-in-hot count = total positions minus what we've already
             // backed up. Without subtracting, the trigger keeps firing on
             // every step once n_tokens passes the threshold but does nothing