mt:: paged refactor — Phase 1 scaffolding (BlockPool + BlockTable)

First step of the paged-attention refactor that will eventually replace
the position-keyed warm/cold tier bookkeeping with vLLM-style
block-level abstractions. These two classes are inert data structures
in this commit — no live wiring to the existing eviction or restore
paths. The defensive n_seq_max==1 assert (commit 01d8e29) still
gates multi-seq until the wiring lands.

mt::BlockPool (mt-block-pool.{h,cpp}):
- Two free-stack pools (GPU + CPU) of physical block IDs
- ID space [0, n_gpu) = GPU blocks; [n_gpu, n_gpu+n_cpu) = CPU blocks
  → is_gpu() is a single integer comparison
- Watermark gating (vLLM-default 5%) reserves a fraction of free
  blocks against admission spikes
- LIFO allocation for cache-line reuse
- Cold tier (NVMe via KvtcStore) is OUT of scope here — that path keys
  by (layer, position) and doesn't need a physical block ID

mt::BlockTable (mt-block-table.{h,cpp}):
- Per-seq dense vector mapping logical_block_idx → physical_block_id
- O(1) lookup via get_physical(seq, logical_idx)
- Convenience get_physical_for_pos(seq, pos) does pos / block_size
- swap_block() rewrites a single entry in place — used by tier
  migration (hot→warm copy: pull new CPU block, swap entry, free
  old GPU block back to pool)
- clear_seq() returns the freed block IDs in append order so the
  caller (eventually mt::seq_rm whole-seq wipe path) can release
  them back to BlockPool

Design adapted from vLLM's vllm/v1/core/block_pool.py +
vllm/v1/worker/block_table.py (Apache 2.0). Code is independent
(C++ vs Python, llama.cpp vs vLLM runtime), architecture is theirs —
attribution will move to docs/memory-tier/CREDITS.md when this lands
as a complete subsystem.

Build: linked into libllama.so cleanly. Both files reach 100% target
build with no warnings.

Phase 2 (next): wire BlockPool + BlockTable into mt::seq_rm /
backup_seq_rm_range / restore_from_warm behind a feature flag, with
the existing position-keyed paths preserved as the default.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Author: kmbandy

src/memory-tier/mt-block-pool.cpp

@@ -0,0 +1,107 @@
+#include "mt-block-pool.h"
+
+#include "llama-impl.h"  // LLAMA_LOG_*
+
+#include <cassert>
+#include <cmath>
+
+namespace mt {
+
+void BlockPool::init(uint32_t n_gpu, uint32_t n_cpu, float watermark) {
+    total_gpu_blocks_ = n_gpu;
+    total_cpu_blocks_ = n_cpu;
+
+    // Watermark is a fraction; round up so a small reserve never floors
+    // to 0 (which would defeat the purpose). vLLM uses ceil() too.
+    watermark_gpu_ = (uint32_t) std::ceil((double) n_gpu * (double) watermark);
+    watermark_cpu_ = (uint32_t) std::ceil((double) n_cpu * (double) watermark);
+
+    // Initialize free stacks. Push descending so pop() returns block 0
+    // first — predictable in tests, no behavior cost.
+    gpu_free_.clear();
+    gpu_free_.reserve(n_gpu);
+    for (uint32_t i = n_gpu; i-- > 0; ) {
+        gpu_free_.push_back(i);
+    }
+
+    cpu_free_.clear();
+    cpu_free_.reserve(n_cpu);
+    for (uint32_t i = n_cpu; i-- > 0; ) {
+        // CPU IDs live above the GPU range so is_gpu() is a single
+        // comparison.
+        cpu_free_.push_back(n_gpu + i);
+    }
+
+    LLAMA_LOG_INFO("mt::BlockPool: init n_gpu=%u n_cpu=%u watermark=%.2f "
+                   "(reserve gpu=%u cpu=%u)\n",
+                   n_gpu, n_cpu, (double) watermark,
+                   watermark_gpu_, watermark_cpu_);
+}
+
+uint32_t BlockPool::alloc_gpu() {
+    if (gpu_free_.empty()) return kInvalidBlockId;
+    const uint32_t id = gpu_free_.back();
+    gpu_free_.pop_back();
+    return id;
+}
+
+uint32_t BlockPool::alloc_cpu() {
+    if (cpu_free_.empty()) return kInvalidBlockId;
+    const uint32_t id = cpu_free_.back();
+    cpu_free_.pop_back();
+    return id;
+}
+
+void BlockPool::free_block(uint32_t block_id) {
+    if (block_id == kInvalidBlockId) {
+        LLAMA_LOG_WARN("mt::BlockPool::free_block: kInvalidBlockId — ignoring\n");
+        return;
+    }
+
+    if (is_gpu(block_id)) {
+        assert(block_id < total_gpu_blocks_);
+        gpu_free_.push_back(block_id);
+    } else {
+        assert(block_id < total_gpu_blocks_ + total_cpu_blocks_);
+        cpu_free_.push_back(block_id);
+    }
+}
+
+bool BlockPool::is_gpu(uint32_t block_id) const {
+    return block_id < total_gpu_blocks_;
+}
+
+size_t BlockPool::n_free_gpu() const {
+    return gpu_free_.size();
+}
+
+size_t BlockPool::n_free_cpu() const {
+    return cpu_free_.size();
+}
+
+bool BlockPool::has_free_gpu_blocks(uint32_t n) const {
+    const size_t avail = gpu_free_.size();
+    if (avail <= watermark_gpu_) return false;
+    return n <= (avail - watermark_gpu_);
+}
+
+bool BlockPool::has_free_cpu_blocks(uint32_t n) const {
+    const size_t avail = cpu_free_.size();
+    if (avail <= watermark_cpu_) return false;
+    return n <= (avail - watermark_cpu_);
+}
+
+void BlockPool::reset() {
+    gpu_free_.clear();
+    gpu_free_.reserve(total_gpu_blocks_);
+    for (uint32_t i = total_gpu_blocks_; i-- > 0; ) {
+        gpu_free_.push_back(i);
+    }
+    cpu_free_.clear();
+    cpu_free_.reserve(total_cpu_blocks_);
+    for (uint32_t i = total_cpu_blocks_; i-- > 0; ) {
+        cpu_free_.push_back(total_gpu_blocks_ + i);
+    }
+}
+
+}  // namespace mt

src/memory-tier/mt-block-pool.h

@@ -0,0 +1,101 @@
+#pragma once
+
+// BlockPool — physical block allocation pool for the paged tier refactor.
+//
+// Design adapted from vLLM (Apache 2.0, vllm/v1/core/block_pool.py +
+// vllm/v1/worker/block_table.py). Code is independent (C++ vs Python),
+// architecture is theirs. See docs/memory-tier/CREDITS.md when this
+// lands as a complete subsystem.
+//
+// A "physical block" is a fixed-size chunk of KV cache storage capable
+// of holding `block_size` tokens worth of K/V for one layer. Blocks are
+// allocated from one of two pools — GPU (hot tier) or CPU (warm tier) —
+// distinguished by their ID range:
+//
+//   [0,                total_gpu_blocks)                       GPU blocks
+//   [total_gpu_blocks, total_gpu_blocks + total_cpu_blocks)    CPU blocks
+//
+// Cold tier (NVMe) is handled outside this pool — KvtcStore writes by
+// (layer, position) and doesn't need a physical block ID.
+//
+// Allocation is LIFO from a free stack; cache-line reuse on freshly-
+// freed blocks helps decode locality. Watermark gating reserves a small
+// fraction of free blocks against admission spikes so the scheduler
+// never starves itself by accepting one more request than it can
+// retire.
+//
+// PHASE 1: This is a pure data structure with no live wiring. The
+// existing position-keyed warm_pos_to_slot_ / cold_positions_ paths
+// in mt-tiered.cpp continue to work. PHASE 2 wires this in behind a
+// feature flag; PHASE 3 makes it the only path.
+//
+// Single-threaded — caller must serialize. mt:: is single-seq today
+// and the scheduler runs single-threaded, so no internal locking.
+
+#include <cstddef>
+#include <cstdint>
+#include <vector>
+
+namespace mt {
+
+// Sentinel for "no block." Chosen as ~0u so it sticks out in logs and
+// any accidental use as an array index will explode loudly.
+static constexpr uint32_t kInvalidBlockId = ~0u;
+
+class BlockPool {
+public:
+    // Initialize the pool with `n_gpu` GPU blocks and `n_cpu` CPU blocks.
+    // `watermark` is the fraction (0.0..1.0) of each pool kept in reserve
+    // before has_free_*_blocks() returns true for new admissions. 0.0
+    // disables the watermark; vLLM defaults to ~0.05 (5%).
+    void init(uint32_t n_gpu, uint32_t n_cpu, float watermark);
+
+    // Allocate one physical block from the GPU pool. Returns
+    // kInvalidBlockId when the pool is exhausted (caller should evict
+    // first). Watermark is NOT enforced here — callers that care about
+    // admission control should consult has_free_gpu_blocks() up-front.
+    // alloc_*() always allocates if anything is free.
+    uint32_t alloc_gpu();
+    uint32_t alloc_cpu();
+
+    // Return a block to its pool. Idempotent on already-free blocks
+    // (logs a warning); double-free does NOT corrupt the pool. Asserts
+    // on out-of-range IDs.
+    void free_block(uint32_t block_id);
+
+    // True if `block_id` is a GPU block (vs CPU). Inferred from the ID
+    // range — no per-block bool stored.
+    bool is_gpu(uint32_t block_id) const;
+
+    // Pool size queries. n_free_*() is the actual count of free blocks
+    // (ignores watermark); has_free_*_blocks(N) tests whether N blocks
+    // can be allocated WITHOUT crossing the watermark reserve.
+    size_t n_free_gpu() const;
+    size_t n_free_cpu() const;
+    bool   has_free_gpu_blocks(uint32_t n) const;
+    bool   has_free_cpu_blocks(uint32_t n) const;
+
+    uint32_t total_gpu_blocks() const { return total_gpu_blocks_; }
+    uint32_t total_cpu_blocks() const { return total_cpu_blocks_; }
+
+    // Reset to all-free state. Used by whole-cache wipes (mt::clear()).
+    // Block IDs are NOT renumbered; existing block-table entries that
+    // pointed into the freed range become dangling and the caller must
+    // also clear those before next use.
+    void reset();
+
+private:
+    // Free stacks (LIFO). gpu_free_ holds IDs in [0, total_gpu_blocks_),
+    // cpu_free_ holds IDs in [total_gpu_blocks_, total_gpu_blocks_ +
+    // total_cpu_blocks_). Stack push/pop is the eviction order under
+    // pure LRU; smarter eviction policies layer above.
+    std::vector<uint32_t> gpu_free_;
+    std::vector<uint32_t> cpu_free_;
+
+    uint32_t total_gpu_blocks_ = 0;
+    uint32_t total_cpu_blocks_ = 0;
+    uint32_t watermark_gpu_    = 0;  // reserve count, NOT a fraction
+    uint32_t watermark_cpu_    = 0;
+};
+
+}  // namespace mt

src/memory-tier/mt-block-table.cpp

@@ -0,0 +1,64 @@
+#include "mt-block-table.h"
+
+#include "mt-block-pool.h"  // kInvalidBlockId
+#include "llama-impl.h"     // LLAMA_LOG_*
+
+#include <cassert>
+
+namespace mt {
+
+void BlockTable::init(uint32_t max_seqs, uint32_t block_size) {
+    max_seqs_   = max_seqs;
+    block_size_ = block_size;
+    table_.clear();
+    table_.resize(max_seqs);
+    LLAMA_LOG_INFO("mt::BlockTable: init max_seqs=%u block_size=%u\n",
+                   max_seqs, block_size);
+}
+
+void BlockTable::append_block(llama_seq_id seq, uint32_t physical_block_id) {
+    assert(seq >= 0 && (uint32_t) seq < max_seqs_ && "seq out of range");
+    table_[seq].push_back(physical_block_id);
+}
+
+uint32_t BlockTable::swap_block(llama_seq_id seq, uint32_t logical_idx,
+                                uint32_t new_physical_block_id) {
+    assert(seq >= 0 && (uint32_t) seq < max_seqs_ && "seq out of range");
+    auto & row = table_[seq];
+    assert(logical_idx < row.size() && "logical_idx out of range");
+    const uint32_t old = row[logical_idx];
+    row[logical_idx] = new_physical_block_id;
+    return old;
+}
+
+uint32_t BlockTable::get_physical(llama_seq_id seq, uint32_t logical_idx) const {
+    if (seq < 0 || (uint32_t) seq >= max_seqs_) return kInvalidBlockId;
+    const auto & row = table_[seq];
+    if (logical_idx >= row.size()) return kInvalidBlockId;
+    return row[logical_idx];
+}
+
+uint32_t BlockTable::get_physical_for_pos(llama_seq_id seq, llama_pos pos) const {
+    if (pos < 0 || block_size_ == 0) return kInvalidBlockId;
+    return get_physical(seq, (uint32_t) pos / block_size_);
+}
+
+uint32_t BlockTable::num_blocks(llama_seq_id seq) const {
+    if (seq < 0 || (uint32_t) seq >= max_seqs_) return 0;
+    return (uint32_t) table_[seq].size();
+}
+
+std::vector<uint32_t> BlockTable::clear_seq(llama_seq_id seq) {
+    if (seq < 0 || (uint32_t) seq >= max_seqs_) return {};
+    std::vector<uint32_t> freed = std::move(table_[seq]);
+    table_[seq].clear();  // post-move state is unspecified; explicit clear
+    return freed;
+}
+
+void BlockTable::reset() {
+    for (auto & row : table_) {
+        row.clear();
+    }
+}
+
+}  // namespace mt

src/memory-tier/mt-block-table.h

@@ -0,0 +1,98 @@
+#pragma once
+
+// BlockTable — per-sequence logical→physical block mapping for the
+// paged tier refactor.
+//
+// Design adapted from vLLM (Apache 2.0, vllm/v1/worker/block_table.py).
+// Code is independent (C++ vs Python), architecture is theirs. See
+// docs/memory-tier/CREDITS.md when this lands as a complete subsystem.
+//
+// For each sequence:
+//
+//   table_[seq] = [physical_block_0, physical_block_1, ..., physical_block_N]
+//                  └─ logical block 0 ┘└─ logical block 1 ┘
+//
+// To find which physical block holds position `p` for sequence `s`:
+//
+//   logical_block_idx = p / block_size
+//   physical_block_id = table_[s][logical_block_idx]
+//   token_in_block    = p % block_size
+//
+// Block size is fixed at construction (typical: 16 tokens). A sequence's
+// logical block list grows monotonically as the sequence is appended;
+// blocks are released only on whole-seq wipe (clear_seq) or via tier
+// migration paths that swap a logical block's physical mapping (e.g.
+// hot→warm copy that updates the table to point at the new CPU block).
+//
+// PHASE 1: Pure data structure with no live wiring. PHASE 2 wires this
+// into the eviction trigger + restore path; PHASE 3 makes it the only
+// path. Until then the existing position-keyed warm_pos_to_slot_ /
+// cold_positions_ paths remain authoritative.
+//
+// Single-threaded.
+
+#include "llama.h"  // llama_seq_id, llama_pos
+
+#include <cstdint>
+#include <vector>
+
+namespace mt {
+
+class BlockTable {
+public:
+    // Construct + size for `max_seqs` sequences with `block_size` tokens
+    // per block. Per-seq capacity grows on demand (no upper bound at
+    // construction — the BlockPool's exhaustion check is what bounds
+    // total allocation).
+    void init(uint32_t max_seqs, uint32_t block_size);
+
+    // Append a physical block to the end of `seq`'s logical sequence.
+    // The new logical_block_idx is `num_blocks(seq) - 1` after this
+    // call. Caller is responsible for pulling the physical block from
+    // BlockPool first.
+    void append_block(llama_seq_id seq, uint32_t physical_block_id);
+
+    // Replace the physical block at logical position `idx` for `seq`.
+    // Used by tier migration: when a hot block is evicted to warm, the
+    // table entry rewrites from the old GPU id to the new CPU id while
+    // logical_block_idx stays put. Returns the OLD physical_block_id so
+    // the caller can free it back to BlockPool.
+    uint32_t swap_block(llama_seq_id seq, uint32_t logical_idx,
+                        uint32_t new_physical_block_id);
+
+    // Lookup: which physical block holds logical block `idx` of `seq`?
+    // Returns kInvalidBlockId (from mt-block-pool.h) if `seq` has no
+    // mapping or `idx` is out of range. Does NOT bounds-assert — paged
+    // attention kernels read past the live range during prefill prep
+    // and must get a sentinel rather than UB.
+    uint32_t get_physical(llama_seq_id seq, uint32_t logical_idx) const;
+
+    // For position-based callers. Same as get_physical(seq, pos /
+    // block_size). Convenience wrapper.
+    uint32_t get_physical_for_pos(llama_seq_id seq, llama_pos pos) const;
+
+    // Number of physical blocks currently held by `seq`. Always equals
+    // ceil(live_token_count / block_size).
+    uint32_t num_blocks(llama_seq_id seq) const;
+
+    // Clear all blocks for `seq` and return the freed physical block
+    // IDs in the order they were appended. Caller passes these back to
+    // BlockPool::free_block(). Used on task-lifecycle wipe (whole-seq
+    // seq_rm with sentinel range).
+    std::vector<uint32_t> clear_seq(llama_seq_id seq);
+
+    // Reset everything. Used by mt::clear() (whole-cache wipe). Caller
+    // is responsible for releasing block IDs back to BlockPool first.
+    void reset();
+
+    uint32_t block_size() const { return block_size_; }
+    uint32_t max_seqs()   const { return max_seqs_;   }
+
+private:
+    // table_[seq][logical_idx] = physical_block_id
+    std::vector<std::vector<uint32_t>> table_;
+    uint32_t                           max_seqs_   = 0;
+    uint32_t                           block_size_ = 0;
+};
+
+}  // namespace mt