@@ -47,10 +47,6 @@ hash table rehashing during initial population:
4747cache = cachebox.LRUCache(maxsize = 10_000 , capacity = 10_000 )
4848```
4949
50- ## Thread Safety
51- All cache operations (reads, writes, eviction) are protected by internal Rust mutexes.
52- You do ** not** need to add external synchronisation.
53-
5450## TTL and Frozen Caches
5551` Frozen ` cannot prevent TTL expiration in ` TTLCache ` or ` VTTLCache ` .
5652Items will still expire naturally even when the cache is frozen.
@@ -236,3 +232,79 @@ Stick with **lazy expiry** when:
236232- The cache sees regular traffic and on-access cleanup is sufficient.
237233- You want to avoid any background thread overhead.
238234- Memory pressure from temporarily lingering stale entries is acceptable.
235+
236+ ## Cache Stampede Prevention
237+ A cache stampede occurs when many concurrent requests find the same key missing from the cache
238+ and all proceed to recompute the value simultaneously that causing redundant work, resource spikes,
239+ or even cascading failures under heavy load. The ` @cached ` decorator prevents this by default
240+ using a per-key lock: once one caller begins computing a missing value, all other callers for the
241+ same key wait for it to finish and then reuse the result.
242+
243+ Lock-based stampede prevention is enabled by default (` lock=True ` ). For sync
244+ functions this uses ` threading.Lock ` ; for async functions it uses ` asyncio.Lock ` :
245+
246+ === "Sync"
247+
248+ ```python
249+ import cachebox
250+
251+ @cachebox.cached(cachebox.LRUCache(maxsize=256))
252+ def fetch_user(user_id: int) -> dict:
253+ # Only called once per user_id, even under concurrent load
254+ return expensive_db_query(user_id)
255+ ```
256+
257+ === "Async"
258+
259+ ```python
260+ import cachebox
261+
262+ @cachebox.cached(cachebox.LRUCache(maxsize=256))
263+ async def fetch_user(user_id: int) -> dict:
264+ # Uses asyncio.Lock automatically for async functions
265+ return await expensive_db_query(user_id)
266+ ```
267+
268+ You can use your own lock type. anything that implements ` contextlib.AbstractContextManager ` for sync functions, or
269+ ` contextlib.AbstractAsyncContextManager ` for async functions:
270+
271+ ``` python
272+ import threading
273+ import cachebox
274+
275+ # Use an RLock (re-entrant lock) instead of the default Lock
276+ @cachebox.cached (cachebox.LRUCache(maxsize = 256 ), lock = threading.RLock)
277+ def fetch_user (user_id : int ) -> dict :
278+ return expensive_db_query(user_id)
279+ ```
280+
281+ !!! warning
282+ Passing a synchronous lock to an async function (or vice versa) raises
283+ a TypeError at decoration time.
284+
285+ If your workload doesn't require it you can disable the lock entirely with ` lock=False ` or ` lock=None ` .
286+ While the default lock is safe for most use cases, there are situations where keeping it enabled causes
287+ problems or is simply unnecessary. * Recursive functions* are the most common case. Because ` threading.Lock ` is
288+ non-reentrant, a cached recursive function will deadlock the moment it calls itself.
289+
290+ ``` python
291+ # ❌ Deadlocks on any recursive call
292+ @cachebox.cached (cachebox.LRUCache(maxsize = 256 ))
293+ def factorial (n : int ) -> int :
294+ return 1 if n <= 1 else n * factorial(n - 1 )
295+ ```
296+
297+ Other cases where disabling the lock is reasonable:
298+
299+ - * Cheap computations* : if recomputing a value is nearly free, the overhead of
300+ lock contention outweighs the benefit of preventing duplicate work.
301+ - * Single-threaded environments* : no concurrency means no stampedes; the lock
302+ is pure overhead.
303+ - * Already-serialised callers* : if your architecture guarantees that only one
304+ caller can request a given key at a time (e.g. a task queue), the lock adds
305+ nothing.
306+
307+ !!! note
308+ Disabling the lock does not make cache operations unsafe. all reads and
309+ writes are still protected by internal Rust mutexes. It only means that
310+ multiple threads may compute the same missing value simultaneously.
0 commit comments