Skip to content

Commit 1fafd5b

Browse files
TKorrTKorr
committed
Update roadmap documentation to include new cache policies
- Added entries for GDS, Hyperbolic, LeCaR, LHD, LRFU, and SIEVE policies to the roadmap. - Ensured the documentation reflects the current state of cache policy development and future directions.
1 parent 2375826 commit 1fafd5b

7 files changed

Lines changed: 219 additions & 0 deletions

File tree

‎docs/policies/roadmap/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,15 @@ If a policy moves into production code, its document should be moved back to
99

1010
## Roadmap Policies
1111

12+
- [GDS](gds.md)
1213
- [GDSF](gdsf.md)
14+
- [Hyperbolic](hyperbolic.md)
15+
- [LeCaR](lecar.md)
1316
- [LFU Aging](lfu-aging.md)
17+
- [LHD](lhd.md)
1418
- [LIRS](lirs.md)
19+
- [LRFU](lrfu.md)
1520
- [OPT](opt.md)
21+
- [SIEVE](sieve.md)
1622
- [TinyLFU / W-TinyLFU](tinylfu.md)
1723
- [TTL](ttl.md)

‎docs/policies/roadmap/gds.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# GreedyDual-Size (GDS)
2+
3+
## Goal
4+
Improve byte-level cache efficiency by preferring entries with better value-per-byte
5+
when item sizes and miss costs vary.
6+
7+
## Core Idea
8+
Each resident entry carries a priority `H`; eviction removes the smallest `H`.
9+
10+
A common GDS-style score:
11+
- `H = (cost / size) + L`
12+
13+
Where:
14+
- `cost`: estimated miss penalty (latency, backend pressure, or monetary cost)
15+
- `size`: object size in bytes
16+
- `L`: global inflation term set to the `H` of the most recently evicted entry
17+
18+
The inflation term prevents old low-value entries from sticking forever.
19+
20+
## Core Data Structures (Typical)
21+
- Hash index `K -> EntryMeta`
22+
- Min-heap keyed by `H` for victim selection
23+
- Global inflation scalar `L`
24+
25+
## Complexity & Overhead
26+
- Heap updates are usually O(log n) on insert/update/evict
27+
- More metadata than LRU/FIFO due to per-entry score and object size/cost
28+
29+
## Notes For CacheKit
30+
- Useful when optimizing byte hit rate, not just request hit rate.
31+
- Integrates naturally if entry size is already tracked in storage metadata.
32+
- Keep score updates explicit and avoid per-access allocation.
33+
34+
## References
35+
- Cao, Irani (1997): “GreedyDual-Size: An algorithm for web caching”.
36+
- Wikipedia: https://en.wikipedia.org/wiki/Cache_replacement_policies

‎docs/policies/roadmap/hyperbolic.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Hyperbolic Caching
2+
3+
## Goal
4+
Combine recency and frequency in a compact, practical scoring model with good
5+
hit-rate behavior under skewed and shifting workloads.
6+
7+
## Core Idea
8+
Each resident entry gets a score that decays with age and increases with
9+
frequency. A common formulation is proportional to:
10+
- `score = freq / age`
11+
12+
Where:
13+
- `freq`: access count or weighted access signal
14+
- `age`: time or logical ticks since insertion/update
15+
16+
Eviction removes the smallest score, so stale/low-value entries fall out first.
17+
18+
## Core Data Structures (Typical)
19+
- Hash index `K -> EntryMeta`
20+
- Per-entry counters: `freq`, `last_touch` (or logical timestamp)
21+
- Victim selector:
22+
- exact ordered structure by score, or
23+
- approximate buckets/periodic rescoring for lower overhead
24+
25+
## Complexity & Overhead
26+
- Exact score ordering is often O(log n) per score-changing update
27+
- Approximate variants trade precision for lower CPU cost
28+
- Metadata is moderate: one frequency counter + age/timestamp per entry
29+
30+
## Notes For CacheKit
31+
- Good candidate when plain LRU underperforms and full ML-style policies are too heavy.
32+
- Prefer monotonic logical clocks and integer math on hot paths.
33+
- Benchmark against `LRU`, `Heap-LFU`, `S3-FIFO`, and `TinyLFU/W-TinyLFU`.
34+
35+
## References
36+
- Blankstein et al.: “Hyperbolic Caching: Flexible Caching for Web Applications”.
37+
- Wikipedia (taxonomy context): https://en.wikipedia.org/wiki/Cache_replacement_policies

‎docs/policies/roadmap/lecar.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# LeCaR (Learning Cache Replacement)
2+
3+
## Goal
4+
Adapt eviction behavior online by learning when recency-focused or
5+
frequency-focused decisions work better for the current workload.
6+
7+
## Core Idea
8+
LeCaR treats eviction as a lightweight online-learning problem:
9+
- Keep two experts (commonly LRU-like and LFU-like signals).
10+
- Maintain weights for each expert based on recent regret/reward.
11+
- On eviction pressure, pick victims according to the weighted mixture.
12+
13+
Under shifting workloads, weights move toward the better-performing strategy.
14+
15+
## Core Data Structures (Typical)
16+
- Hash index `K -> EntryMeta`
17+
- Recency structure (LRU list or clock-style metadata)
18+
- Frequency counters (or compact approximate counters)
19+
- Learning state (expert weights, step size, recent feedback)
20+
21+
## Complexity & Overhead
22+
- Extra per-operation arithmetic for weight updates
23+
- More metadata than single-policy designs
24+
- Performance depends on stable, well-tuned feedback signals
25+
26+
## Notes For CacheKit
27+
- Best viewed as a research/benchmark policy before production defaulting.
28+
- Keep learning state compact and avoid floating-point work in hot loops if possible.
29+
- Evaluate under non-stationary workloads with hotspot shifts.
30+
31+
## References
32+
- Vietri et al. (2018): “Driving cache replacement with ML-based LeCaR”.
33+
- Wikipedia (taxonomy context): https://en.wikipedia.org/wiki/Cache_replacement_policies

‎docs/policies/roadmap/lhd.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# LHD (Learning Hit Density)
2+
3+
## Goal
4+
Maximize value per byte by using a learned estimate of hit probability over
5+
remaining lifetime, especially when object sizes vary substantially.
6+
7+
## Core Idea
8+
LHD estimates each object's future hit density (roughly expected future hits per
9+
byte over a horizon) from observed reuse behavior. Eviction prefers entries with
10+
the lowest estimated hit density.
11+
12+
Compared to simple size-aware scoring, LHD adapts estimates from workload data
13+
instead of relying on fixed formulas.
14+
15+
## Core Data Structures (Typical)
16+
- Hash index `K -> EntryMeta`
17+
- Size-aware entry metadata (`size`, age/timestamp, access stats)
18+
- Lightweight learned model/state (bucketized statistics tables)
19+
- Victim selector keyed by estimated hit density
20+
21+
## Complexity & Overhead
22+
- Higher implementation complexity than GDS/GDSF
23+
- Additional memory for model tables and per-entry predictors
24+
- Runtime cost depends on estimator complexity (can be O(1) with table lookups)
25+
26+
## Notes For CacheKit
27+
- High-value candidate for byte hit rate optimization and heterogeneous object sizes.
28+
- Start with coarse bucketed estimators to keep hot paths predictable.
29+
- Benchmark against `GDS`, `GDSF`, and `Hyperbolic` on mixed-size traces.
30+
31+
## References
32+
- Beckmann et al. (2018): “LHD: Improving Cache Hit Rate by Maximizing Hit Density”.
33+
- USENIX OSDI 2018 paper and follow-up implementations.

‎docs/policies/roadmap/lrfu.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# LRFU (Least Recently/Frequently Used)
2+
3+
## Goal
4+
5+
Provide a tunable continuum between recency (LRU-like) and frequency (LFU-like)
6+
to match different workload shapes with one policy family.
7+
8+
## Core Idea
9+
10+
LRFU assigns each entry a combined recency-frequency value with exponential
11+
decay. A tunable parameter controls how quickly older accesses lose weight:
12+
13+
- lower decay -> more LFU-like behavior
14+
- higher decay -> more LRU-like behavior
15+
16+
Eviction selects the entry with the lowest combined value.
17+
18+
## Core Data Structures (Typical)
19+
20+
- Hash index `K -> EntryMeta`
21+
- Per-entry state:
22+
- combined CRF value (combined recency-frequency)
23+
- last update timestamp/tick
24+
- Victim selector ordered by current CRF value (exact or approximate)
25+
26+
## Complexity & Overhead
27+
28+
- O(log n) with exact ordered structures
29+
- Requires per-hit value updates (or lazy recomputation) to account for decay
30+
- Metadata and math cost are higher than plain LRU/Clock
31+
32+
## Notes For CacheKit
33+
34+
- Useful as a benchmark control policy for mapping recency-vs-frequency sensitivity.
35+
- Prefer lazy decay application to reduce write amplification on hot paths.
36+
- Keep numeric updates deterministic and avoid floating-point where practical.
37+
38+
## References
39+
40+
- Lee et al. (2001): “LRFU: A Spectrum of Policies that Subsumes the Least Recently Used and Least Frequently Used Policies”.
41+
- Wikipedia (taxonomy context): [https://en.wikipedia.org/wiki/Cache_replacement_policies](https://en.wikipedia.org/wiki/Cache_replacement_policies)

‎docs/policies/roadmap/sieve.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# SIEVE
2+
3+
## Goal
4+
Provide strong scan resistance with very low metadata and update overhead.
5+
6+
## Core Idea
7+
SIEVE is a simple FIFO-like policy with a one-bit filter:
8+
- New entries are inserted as "cold".
9+
- On access, an entry is marked "hot" via a single bit.
10+
- Eviction scans from the FIFO head:
11+
- hot entry: clear bit and give it one more pass
12+
- cold entry: evict immediately
13+
14+
This creates low-cost second-chance behavior without maintaining full LRU order.
15+
16+
## Core Data Structures (Typical)
17+
- Hash index `K -> EntryMeta`
18+
- FIFO queue for insertion/eviction order
19+
- One hot/cold bit per entry
20+
21+
## Complexity & Overhead
22+
- Access path is O(1): bit set only (no relinking)
23+
- Eviction is amortized O(1) with queue head advancement
24+
- Metadata is minimal: queue links/indices + one bit
25+
26+
## Notes For CacheKit
27+
- Aligns well with contiguous storage and index-based handles.
28+
- Attractive for high-throughput paths where LRU relinking is too expensive.
29+
- Benchmark against `S3-FIFO`, `Clock`, and `LRU` on scan-heavy mixes.
30+
31+
## References
32+
- SIEVE policy publication and follow-up implementation notes.
33+
- Wikipedia (taxonomy context): https://en.wikipedia.org/wiki/Cache_replacement_policies

0 commit comments

Comments
 (0)