Skip to content

Commit a03ffc0

Browse files
committed
Update docs
1 parent f1da7a5 commit a03ffc0

3 files changed

Lines changed: 79 additions & 10 deletions

File tree

cachebox/utils.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -430,11 +430,12 @@ def cached(
430430
* :func:`postprocess_copy_mutables` - shallow-copy only `dict`, `list` and `set` (default).
431431
* :func:`postprocess_deepcopy` - deep-copy.
432432
* :func:`postprocess_deepcopy_mutables` - deep-copy only `dict`, `list` and `set`.
433-
lock: If ``None`` or ``False``, cache stampede preventation get disabled, but process is still thread-safe.
433+
lock: If ``None`` or ``False``, cache stampede prevention get disabled, but process is still thread-safe.
434434
If ``True``, will use ``threading.Lock`` or ``asyncio.Lock`` depends on wrapped function.
435435
Also you can pass anything that implemented ``contextlib.AbstractContextManager``
436436
(or ``contextlib.AbstractAsyncContextManager`` for async functions).
437-
(default is ``True``).
437+
(default is ``True``). See [cache stampede prevention](http://awolverp.github.io/cachebox/tips/#cache-stampede-prevention)
438+
for more.
438439
439440
Tip:
440441
Pass ``cachebox__ignore=True`` at call-time to bypass the cache.

docs/docs/api/utils.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,6 @@
1010

1111
::: cachebox.utils.Frozen
1212

13-
::: cachebox.utils.CacheInfo
14-
::: cachebox.utils.EVENT_MISS
15-
::: cachebox.utils.EVENT_HIT
16-
1713
::: cachebox.utils.cached
1814
::: cachebox.utils.is_cached
1915
::: cachebox.utils.get_cached_cache

docs/docs/tips.md

Lines changed: 76 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,10 +47,6 @@ hash table rehashing during initial population:
4747
cache = cachebox.LRUCache(maxsize=10_000, capacity=10_000)
4848
```
4949

50-
## Thread Safety
51-
All cache operations (reads, writes, eviction) are protected by internal Rust mutexes.
52-
You do **not** need to add external synchronisation.
53-
5450
## TTL and Frozen Caches
5551
`Frozen` cannot prevent TTL expiration in `TTLCache` or `VTTLCache`.
5652
Items will still expire naturally even when the cache is frozen.
@@ -236,3 +232,79 @@ Stick with **lazy expiry** when:
236232
- The cache sees regular traffic and on-access cleanup is sufficient.
237233
- You want to avoid any background thread overhead.
238234
- Memory pressure from temporarily lingering stale entries is acceptable.
235+
236+
## Cache Stampede Prevention
237+
A cache stampede occurs when many concurrent requests find the same key missing from the cache
238+
and all proceed to recompute the value simultaneously that causing redundant work, resource spikes,
239+
or even cascading failures under heavy load. The `@cached` decorator prevents this by default
240+
using a per-key lock: once one caller begins computing a missing value, all other callers for the
241+
same key wait for it to finish and then reuse the result.
242+
243+
Lock-based stampede prevention is enabled by default (`lock=True`). For sync
244+
functions this uses `threading.Lock`; for async functions it uses `asyncio.Lock`:
245+
246+
=== "Sync"
247+
248+
```python
249+
import cachebox
250+
251+
@cachebox.cached(cachebox.LRUCache(maxsize=256))
252+
def fetch_user(user_id: int) -> dict:
253+
# Only called once per user_id, even under concurrent load
254+
return expensive_db_query(user_id)
255+
```
256+
257+
=== "Async"
258+
259+
```python
260+
import cachebox
261+
262+
@cachebox.cached(cachebox.LRUCache(maxsize=256))
263+
async def fetch_user(user_id: int) -> dict:
264+
# Uses asyncio.Lock automatically for async functions
265+
return await expensive_db_query(user_id)
266+
```
267+
268+
You can use your own lock type. anything that implements `contextlib.AbstractContextManager` for sync functions, or
269+
`contextlib.AbstractAsyncContextManager` for async functions:
270+
271+
```python
272+
import threading
273+
import cachebox
274+
275+
# Use an RLock (re-entrant lock) instead of the default Lock
276+
@cachebox.cached(cachebox.LRUCache(maxsize=256), lock=threading.RLock)
277+
def fetch_user(user_id: int) -> dict:
278+
return expensive_db_query(user_id)
279+
```
280+
281+
!!! warning
282+
Passing a synchronous lock to an async function (or vice versa) raises
283+
a TypeError at decoration time.
284+
285+
If your workload doesn't require it you can disable the lock entirely with `lock=False` or `lock=None`.
286+
While the default lock is safe for most use cases, there are situations where keeping it enabled causes
287+
problems or is simply unnecessary. *Recursive functions* are the most common case. Because `threading.Lock` is
288+
non-reentrant, a cached recursive function will deadlock the moment it calls itself.
289+
290+
```python
291+
# ❌ Deadlocks on any recursive call
292+
@cachebox.cached(cachebox.LRUCache(maxsize=256))
293+
def factorial(n: int) -> int:
294+
return 1 if n <= 1 else n * factorial(n - 1)
295+
```
296+
297+
Other cases where disabling the lock is reasonable:
298+
299+
- *Cheap computations*: if recomputing a value is nearly free, the overhead of
300+
lock contention outweighs the benefit of preventing duplicate work.
301+
- *Single-threaded environments*: no concurrency means no stampedes; the lock
302+
is pure overhead.
303+
- *Already-serialised callers*: if your architecture guarantees that only one
304+
caller can request a given key at a time (e.g. a task queue), the lock adds
305+
nothing.
306+
307+
!!! note
308+
Disabling the lock does not make cache operations unsafe. all reads and
309+
writes are still protected by internal Rust mutexes. It only means that
310+
multiple threads may compute the same missing value simultaneously.

0 commit comments

Comments
 (0)