Refactor LRU policy to use a pool-based design for allocation-free performance (#36)

- Transitioned to a `SlotArena` for contiguous storage of nodes, enabling O(1) slot reuse and eliminating heap allocations during steady state.
- Updated core data structures to utilize `FxHashMap` for key-to-slot lookup and maintain a doubly-linked order via `SlotId` indices.
- Enhanced concurrency support with a `ConcurrentLruCache` wrapper, detailing read/write lock requirements for operations.
- Revised documentation to reflect architectural changes, including node layout, allocation strategy, and safety invariants, ensuring clarity on performance and usage.

Author: TKorr

docs/policies/lru.md

@@ -6,48 +6,54 @@
 Evict the entry that has not been accessed for the longest time.
 
 ## Core Data Structures
-Canonical O(1) LRU:
-- `HashMap<K, NodeHandle>` mapping keys to nodes
-- Intrusive doubly-linked list storing recency order (head = MRU, tail = LRU)
 
-In `cachekit`, `src/policy/lru.rs` uses:
-- `HashMap<K, NonNull<Node<K>>>` for O(1) node access
-- an intrusive list (manual `prev`/`next` pointers) to support O(1) move-to-front
-- a separate store mapping `K -> Arc<V>` (value ownership)
-- an optional concurrent wrapper (`parking_lot::RwLock`) for multi-threaded use
+`src/policy/lru.rs` uses a pool-based design:
+
+- `SlotArena<Node<K, V>>` — contiguous `Vec` with a free list for O(1) slot reuse.
+  Each `Node` stores `prev`/`next` as `Option<SlotId>` indices, plus the key and `Arc<V>`.
+- `FxHashMap<K, SlotId>` — O(1) key-to-slot lookup.
+- `head`/`tail: Option<SlotId>` — doubly-linked recency order (head = MRU, tail = LRU).
+- `ConcurrentLruCache` — thread-safe wrapper via `parking_lot::RwLock`.
+
+At steady state (cache full), every insert evicts the tail node (returning its
+slot to the free list) then inserts the new node (reusing a free slot). No heap
+allocations occur after the initial warm-up phase.
 
 ## Operations
 
 ### `get(key)`
-- Lookup key; if present: move its node to the head (MRU) and return value.
+- Lookup key in map; if present: detach node, attach at head (MRU), return `&Arc<V>`.
 
 ### `insert(key, value)`
-- If `key` exists: update stored value and move node to head.
+- If `key` exists: replace value in-place, detach + attach at head.
 - Else:
-  - If at capacity: evict tail node, remove its key from the index and store.
-  - Insert new node at head and add to index + store.
+  - If at capacity: `pop_tail()` removes the LRU node from the arena (slot returned to free list), remove its key from the map.
+  - `arena.insert(Node{...})` reuses a free slot or appends.
+  - Insert `SlotId` into the map, attach at head.
 
 ### `pop_lru()`
-- Remove tail node (LRU) and delete from index + store.
+- Remove tail node from arena + map, return `(K, Arc<V>)`.
+
+### `peek(key)` / `peek_lru()`
+- Read-only lookup; no list reordering.
 
 ## Complexity & Overhead
-- Lookup/update/evict: O(1)
-- Memory: per-entry list pointers + hash entry + key copy (intrusive list usually stores keys, not values)
+- Lookup / update / evict: O(1)
+- Memory per entry: `SlotId` prev/next (2 × 12 bytes) + key + `Arc<V>` + `Option` discriminant in arena slot
+- Steady-state allocations: **zero** (free-list recycling)
 
 ## Concurrency Notes
-Strict global LRU mutates shared metadata on every hit; typical approaches:
-- Single lock around the entire structure (simple, but can contend)
-- Sharding (key->shard) for scalability
-- Approximate LRU (Clock) when write-amplification or contention is too high
-
-`cachekit` provides a concurrent wrapper in the LRU module; be mindful that:
-- every hit does a write (reordering), so read-heavy workloads can still contend.
+Strict global LRU mutates shared metadata on every hit; `cachekit` provides:
+- `ConcurrentLruCache` — single `RwLock` around `LruCore`.
+- `get()` requires a **write** lock (moves node to head).
+- `peek()` requires only a **read** lock (no reordering).
+- Read-heavy workloads can still contend on `get()`; prefer `peek()` when recency updates are not needed.
 
 ## Safety / Invariants (Rust)
-If using intrusive pointers (like `NonNull<Node<_>>`):
-- Nodes must not move in memory while linked (allocate in a stable arena/Box and manage lifetime).
-- Every `prev/next` pointer update must preserve list invariants; always update both sides.
-- On removal/eviction, ensure you unlink before freeing and remove from the hashmap index.
+- **No `unsafe`**: all list manipulation uses `SlotId` indices into the arena.
+- **ABA prevention**: `SlotId` includes a generation counter; stale handles return `None`.
+- **Automatic cleanup**: `SlotArena` drops all live nodes when `LruCore` is dropped.
+- `Send`/`Sync` auto-derive from the safe field types; no manual `unsafe impl`.
 
 ## References
 - Wikipedia: https://en.wikipedia.org/wiki/Cache_replacement_policies