You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+8-12Lines changed: 8 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,8 @@
1
1
# multicache
2
2
3
-
A sharded in-memory cache for Go with optional persistence.
3
+
multicache is an in-memory #golang cache library.
4
4
5
-
Implements S3-FIFO from ["FIFO queues are all you need for cache eviction"](https://dl.acm.org/doi/10.1145/3600006.3613147) (SOSP'23). S3-FIFO matches or exceeds LRU hit rates with simpler operations and better concurrency.
5
+
It's been optimized over hundreds of experiments to be the highest performing cache available - both in terms of hit rates and throughput - and also features an optional multi-tier persistent cache option.
@@ -54,10 +54,11 @@ Memory cache backed by durable storage. Reads check memory first; writes go to b
54
54
| Google Cloud Datastore |`pkg/store/datastore`|
55
55
| Auto-detect (Cloud Run) |`pkg/store/cloudrun`|
56
56
57
-
All backends support S2 or Zstd compression via `pkg/store/compress`.
57
+
For maximum efficiency, all backends support S2 or Zstd compression via `pkg/store/compress`.
58
58
59
59
## Performance
60
60
61
+
61
62
[gocachemark](https://github.com/tstromberg/gocachemark) compares cache libraries across hit rate, latency, throughput, and memory. Overall scores (Dec 2025):
62
63
63
64
```
@@ -77,13 +78,13 @@ Where others win:
77
78
-**Memory**: freelru and otter use less memory per entry
Run `make bench` or see gocachemark for full results.
81
+
Run `make competive-bench` for full results.
81
82
82
83
## Algorithm
83
84
84
-
S3-FIFO uses three queues: small (new entries), main (promoted entries), and ghost (recently evicted keys). New items enter small; items accessed twice move to main. The ghost queue tracks evicted keys in a bloom filter to fast-track their return.
85
+
multicache uses [S3-FIFO](https://s3fifo.com/), which features three queues: small (new entries), main (promoted entries), and ghost (recently evicted keys). New items enter small; items accessed twice move to main. The ghost queue tracks evicted keys in a bloom filter to fast-track their return.
85
86
86
-
This implementation adds:
87
+
multicache has been hyper-tuned for high performance, and deviates from the original paper in a handful of ways:
87
88
88
89
-**Dynamic sharding** - scales to 16×GOMAXPROCS shards; at 32 threads: 21x Get throughput, 6x Set throughput vs single shard
89
90
-**Tuned small queue** - 24.7% vs paper's 10%, chosen via sweep in 0.1% increments to maximize wins across 9 production traces
@@ -92,11 +93,6 @@ This implementation adds:
92
93
-**Hot item demotion** - items that were once hot (freq≥4) get demoted to small queue instead of evicted; +0.24% zipf
93
94
-**Death row buffer** - 8-entry buffer per shard holds recently evicted items for instant resurrection; +0.04% meta/tencentPhoto, +0.03% wikipedia, +8% set throughput
94
95
-**Ghost frequency ring buffer** - fixed-size 256-entry ring replaces map allocations; -5.1% string latency, -44.5% memory
95
-
-**Cached entry hash** - hash computed once at set, reused on eviction; eliminates re-hashing overhead
96
-
97
-
Memory overhead is ~66 bytes/item (vs 38 for CLOCK, 119 for map-based ghost tracking)
0 commit comments