Skip to content

Commit eb9503f

Browse files
committed
proper async detach, add more comments
1 parent 9424e74 commit eb9503f

7 files changed

Lines changed: 548 additions & 534 deletions

File tree

README.md

Lines changed: 8 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# multicache
22

3-
A sharded in-memory cache for Go with optional persistence.
3+
multicache is an in-memory #golang cache library.
44

5-
Implements S3-FIFO from ["FIFO queues are all you need for cache eviction"](https://dl.acm.org/doi/10.1145/3600006.3613147) (SOSP'23). S3-FIFO matches or exceeds LRU hit rates with simpler operations and better concurrency.
5+
It's been optimized over hundreds of experiments to be the highest performing cache available - both in terms of hit rates and throughput - and also features an optional multi-tier persistent cache option.
66

77
## Install
88

@@ -28,7 +28,7 @@ cache.Set(ctx, "user:123", user) // sync write
2828
cache.SetAsync(ctx, "user:456", user) // async write
2929
```
3030

31-
GetSet deduplicates concurrent loads:
31+
GetSet deduplicates concurrent loads to prevent thundering herd situations:
3232

3333
```go
3434
user, err := cache.GetSet("user:123", func() (User, error) {
@@ -54,10 +54,11 @@ Memory cache backed by durable storage. Reads check memory first; writes go to b
5454
| Google Cloud Datastore | `pkg/store/datastore` |
5555
| Auto-detect (Cloud Run) | `pkg/store/cloudrun` |
5656

57-
All backends support S2 or Zstd compression via `pkg/store/compress`.
57+
For maximum efficiency, all backends support S2 or Zstd compression via `pkg/store/compress`.
5858

5959
## Performance
6060

61+
6162
[gocachemark](https://github.com/tstromberg/gocachemark) compares cache libraries across hit rate, latency, throughput, and memory. Overall scores (Dec 2025):
6263

6364
```
@@ -77,13 +78,13 @@ Where others win:
7778
- **Memory**: freelru and otter use less memory per entry
7879
- **Some traces**: CLOCK/LRU marginally better on purely temporal workloads (IBM Docker, Thesios)
7980

80-
Run `make bench` or see gocachemark for full results.
81+
Run `make competive-bench` for full results.
8182

8283
## Algorithm
8384

84-
S3-FIFO uses three queues: small (new entries), main (promoted entries), and ghost (recently evicted keys). New items enter small; items accessed twice move to main. The ghost queue tracks evicted keys in a bloom filter to fast-track their return.
85+
multicache uses [S3-FIFO](https://s3fifo.com/), which features three queues: small (new entries), main (promoted entries), and ghost (recently evicted keys). New items enter small; items accessed twice move to main. The ghost queue tracks evicted keys in a bloom filter to fast-track their return.
8586

86-
This implementation adds:
87+
multicache has been hyper-tuned for high performance, and deviates from the original paper in a handful of ways:
8788

8889
- **Dynamic sharding** - scales to 16×GOMAXPROCS shards; at 32 threads: 21x Get throughput, 6x Set throughput vs single shard
8990
- **Tuned small queue** - 24.7% vs paper's 10%, chosen via sweep in 0.1% increments to maximize wins across 9 production traces
@@ -92,11 +93,6 @@ This implementation adds:
9293
- **Hot item demotion** - items that were once hot (freq≥4) get demoted to small queue instead of evicted; +0.24% zipf
9394
- **Death row buffer** - 8-entry buffer per shard holds recently evicted items for instant resurrection; +0.04% meta/tencentPhoto, +0.03% wikipedia, +8% set throughput
9495
- **Ghost frequency ring buffer** - fixed-size 256-entry ring replaces map allocations; -5.1% string latency, -44.5% memory
95-
- **Cached entry hash** - hash computed once at set, reused on eviction; eliminates re-hashing overhead
96-
97-
Memory overhead is ~66 bytes/item (vs 38 for CLOCK, 119 for map-based ghost tracking)
98-
99-
Details: [s3fifo.com](https://s3fifo.com/)
10096

10197
## License
10298

0 commit comments

Comments
 (0)