feat(eviction): robustify all eviction strategies, add config options, and improve docs

- Refactored all eviction algorithms (LRU, LFU, ARC, Clock, CAWOLFU) for thread safety, zero-capacity handling, and deadlock prevention.
- Inlined eviction logic in Clock and CAWOLFU to avoid recursive locking and deadlocks.
- Ensured all mutating operations use proper locking and updated value/frequency on duplicate keys.
- Improved memory hygiene by resetting pooled items before reuse.
- Added buffer size and debounce interval options for expiration trigger channel in config and HyperCache.
- Updated README to clarify package paths and usage, and fixed example commands.
- Added PRD.md with design review and suggestions.
- Updated benchmark to use LRU for proactive eviction test.
- Improved code comments and documentation for maintainability.

Author: hyp3rd

PRD.md

@@ -0,0 +1,33 @@
+# PRD
+
+Overview
+Hypercache is a thread‑safe caching library with pluggable backends and eviction algorithms, already targeting Go 1.25 and generics. The design exposes a service interface that supports middleware layers for decorating cache operations.
+
+Findings & Suggestions
+Ticker Lifecycle – Background jobs create time.Tickers but never stop them, which can leak resources; explicitly call tick.Stop() when terminating the loops.
+
+WorkerPool Resize Logic – Resizing the pool sends a quit signal pool.workers times regardless of how many workers need removal; adjust to send -diff signals and consider buffering to avoid blocking.
+
+Expired‑Key Semantics – GetMultiple returns ErrKeyExpired, yet tests expect ErrKeyNotFound for expired items, indicating a behavior mismatch or test expectation issue.
+
+Per‑Call Goroutines – Get and GetWithInfo spawn a goroutine for each expired item, which may create overhead under heavy load; consider synchronous cleanup or a dedicated worker to batch expirations.
+
+Object Pool Hygiene – ItemPoolManager returns items to the pool without resetting fields, risking retention of large values; zero the struct before reusing it.
+
+Modern Go Features – The project already uses Go 1.25; further modernization could include:
+
+Generics for typed values rather than any, providing compile‑time safety.
+
+Use of maps.Clone/maps.DeleteFunc and slices.SortFunc for cleaner collection handling.
+
+errors.Is/errors.Join to simplify error comparisons.
+
+Potential Enhancements
+
+Expose a MGet/MSet on backends for efficient multi-key operations.
+
+Provide context-aware cancellation in WorkerPool and background jobs to ease shutdown.
+
+Add methods such as Contains, Peek, Increment, or TTL refresh APIs.
+
+Integrate OpenTelemetry metrics and tracing for observability.

README.md

@@ -6,29 +6,29 @@
 
 HyperCache is a **thread-safe** **high-performance** cache implementation in `Go` that supports multiple backends with an optional size limit, expiration, and eviction of items with custom algorithms alongside the defaults. It can be used as a standalone cache, distributed environents, or as a cache middleware for a service. It can implement a [service interface](./service.go) to intercept and decorate the cache methods with middleware (default or custom).
 It is optimized for performance and flexibility, allowing to specify of the expiration and eviction intervals and providing and registering new eviction algorithms, stats collectors, and middleware(s).
-It ships with a default [historigram stats collector](./stats/stats.go) and several eviction algorithms. However, you can develop and register your own if it implements the [Eviction Algorithm interface](./eviction/eviction.go).:
+It ships with a default [historigram stats collector](./pkg/stats/stats.go) and several eviction algorithms. However, you can develop and register your own if it implements the [Eviction Algorithm interface](./pkg/eviction/eviction.go).:
 
-- [Recently Used (LRU) eviction algorithm](./eviction/lru.go)
-- [The Least Frequently Used (LFU) algorithm](./eviction/lfu.go)
-- [Cache-Aware Write-Optimized LFU (CAWOLFU)](./eviction/cawolfu.go)
-- [The Adaptive Replacement Cache (ARC) algorithm](./eviction/arc.go)
-- [The clock eviction algorithm](./eviction/clock.go)
+- [Recently Used (LRU) eviction algorithm](./pkg/eviction/lru.go)
+- [The Least Frequently Used (LFU) algorithm](./pkg/eviction/lfu.go)
+- [Cache-Aware Write-Optimized LFU (CAWOLFU)](./pkg/eviction/cawolfu.go)
+- [The Adaptive Replacement Cache (ARC) algorithm](./pkg/eviction/arc.go)
+- [The clock eviction algorithm](./pkg/eviction/clock.go)
 
 ### Features
 
 - Thread-safe
 - High-performance
 - Supports multiple, custom backends. Default backends are:
-    1. [In-memory](./backend/inmemory.go)
-    2. [Redis](./backend/redis.go)
+    1. [In-memory](./pkg/backend/inmemory.go)
+    2. [Redis](./pkg/backend/redis.go)
 - Store items in the cache with a key and expiration duration
 - Retrieve items from the cache by their key
 - Delete items from the cache by their key
 - Clear the cache of all items
 - Evitc items in the background based on the cache capacity and items access leveraging several custom eviction algorithms
 - Expire items in the background based on their duration
-- [Eviction Algorithm interface](./eviction/eviction.go) to implement custom eviction algorithms.
-- Stats collection with a default [stats collector](./stats/stats.go) or a custom one that implements the StatsCollector interface.
+- [Eviction Algorithm interface](./pkg/eviction/eviction.go) to implement custom eviction algorithms.
+- Stats collection with a default [stats collector](./pkg/stats/stats.go) or a custom one that implements the StatsCollector interface.
 - [Service interface implementation](./service.go) to allow intercepting cache methods and decorate them with custom or default middleware(s).
 
 ## Installation
@@ -64,7 +64,7 @@ ok      github.com/hyp3rd/hypercache/tests/benchmark    30.031s
 To run the examples, use the following command:
 
 ```bash
-make run-example example=eviction  # or any other example
+make run-example group=eviction  # or any other example
 ```
 
 For a complete list of examples, refer to the [examples](./__examples/README.md) directory.

config.go

@@ -124,6 +124,22 @@ func WithEvictionInterval[T backend.IBackendConstrain](evictionInterval time.Dur
 	}
 }
 
+// WithExpirationTriggerBuffer sets the buffer size of the expiration trigger channel.
+// If set to <= 0, the default (capacity/2, minimum 1) is used.
+func WithExpirationTriggerBuffer[T backend.IBackendConstrain](size int) Option[T] {
+	return func(cache *HyperCache[T]) {
+		cache.expirationTriggerBufSize = size
+	}
+}
+
+// WithExpirationTriggerDebounce sets an optional debounce interval for coalescing expiration triggers.
+// Triggers arriving within this interval after the last accepted trigger may be dropped.
+func WithExpirationTriggerDebounce[T backend.IBackendConstrain](interval time.Duration) Option[T] {
+	return func(cache *HyperCache[T]) {
+		cache.expirationDebounceInterval = interval
+	}
+}
+
 // WithMaxEvictionCount is an option that sets the max eviction count field of the `HyperCache` struct.
 // The max eviction count determines the maximum number of items that can be removed during a single eviction run.
 func WithMaxEvictionCount[T backend.IBackendConstrain](maxEvictionCount uint) Option[T] {

cspell.config.yaml

@@ -43,6 +43,7 @@ words:
   - memprofile
   - msgpack
   - mvdan
+  - nestif
   - Newf
   - nolint
   - NOVENDOR

hypercache.go

@@ -39,24 +39,28 @@ import (
 // The cache also has a mutex that is used to protect the eviction algorithm from concurrent access.
 // The stop channel is used to signal the expiration and eviction loops to stop. The evictCh channel is used to signal the eviction loop to start.
 type HyperCache[T backend.IBackendConstrain] struct {
-	backend               backend.IBackend[T]               // `backend`` holds the backend that the cache uses to store the items. It must implement the IBackend interface.
-	cacheBackendChecker   introspect.CacheBackendChecker[T] // `cacheBackendChecker` holds an instance of the CacheBackendChecker interface. It helps to determine the type of the backend.
-	itemPoolManager       *cache.ItemPoolManager            // `itemPoolManager` manages the item pool for the cache.
-	stop                  chan bool                         // `stop` channel to signal the expiration and eviction loops to stop
-	workerPool            *WorkerPool                       // `workerPool` holds a pointer to the worker pool that the cache uses to run the expiration and eviction loops.
-	expirationTriggerCh   chan bool                         // `expirationTriggerCh` channel to signal the expiration trigger loop to start
-	evictCh               chan bool                         // `evictCh` channel to signal the eviction loop to start
-	evictionAlgorithmName string                            // `evictionAlgorithmName` name of the eviction algorithm to use when evicting items
-	evictionAlgorithm     eviction.IAlgorithm               // `evictionAlgorithm` eviction algorithm to use when evicting items
-	expirationInterval    time.Duration                     // `expirationInterval` interval at which the expiration loop should run
-	evictionInterval      time.Duration                     // interval at which the eviction loop should run
-	shouldEvict           atomic.Bool                       // `shouldEvict` indicates whether the cache should evict items or not
-	maxEvictionCount      uint                              // `evictionInterval` maximum number of items that can be evicted in a single eviction loop iteration
-	maxCacheSize          int64                             // maxCacheSize instructs the cache not allocate more memory than this limit, value in MB, 0 means no limit
-	memoryAllocation      atomic.Int64                      // memoryAllocation is the current memory allocation of the cache, value in bytes
-	mutex                 sync.RWMutex                      // `mutex` holds a RWMutex (Read-Write Mutex) that is used to protect the eviction algorithm from concurrent access
-	once                  sync.Once                         // `once` holds a Once struct that is used to ensure that the expiration and eviction loops are only started once
-	statsCollectorName    string                            // `statsCollectorName` holds the name of the stats collector that the cache should use when collecting cache statistics
+	backend                    backend.IBackend[T]               // `backend`` holds the backend that the cache uses to store the items. It must implement the IBackend interface.
+	cacheBackendChecker        introspect.CacheBackendChecker[T] // `cacheBackendChecker` holds an instance of the CacheBackendChecker interface. It helps to determine the type of the backend.
+	itemPoolManager            *cache.ItemPoolManager            // `itemPoolManager` manages the item pool for the cache.
+	stop                       chan bool                         // `stop` channel to signal the expiration and eviction loops to stop
+	workerPool                 *WorkerPool                       // `workerPool` holds a pointer to the worker pool that the cache uses to run the expiration and eviction loops.
+	expirationTriggerCh        chan bool                         // `expirationTriggerCh` channel to signal the expiration trigger loop to start
+	expirationTriggerBufSize   int                               // optional override for expiration trigger channel buffer size (default: capacity/2)
+	expirationSignalPending    atomic.Bool                       // coalesces multiple expiration trigger requests
+	expirationDebounceInterval time.Duration                     // optional debounce interval between accepted triggers
+	lastExpirationTrigger      atomic.Int64                      // unix nano timestamp of last accepted trigger
+	evictCh                    chan bool                         // `evictCh` channel to signal the eviction loop to start
+	evictionAlgorithmName      string                            // `evictionAlgorithmName` name of the eviction algorithm to use when evicting items
+	evictionAlgorithm          eviction.IAlgorithm               // `evictionAlgorithm` eviction algorithm to use when evicting items
+	expirationInterval         time.Duration                     // `expirationInterval` interval at which the expiration loop should run
+	evictionInterval           time.Duration                     // interval at which the eviction loop should run
+	shouldEvict                atomic.Bool                       // `shouldEvict` indicates whether the cache should evict items or not
+	maxEvictionCount           uint                              // `evictionInterval` maximum number of items that can be evicted in a single eviction loop iteration
+	maxCacheSize               int64                             // maxCacheSize instructs the cache not allocate more memory than this limit, value in MB, 0 means no limit
+	memoryAllocation           atomic.Int64                      // memoryAllocation is the current memory allocation of the cache, value in bytes
+	mutex                      sync.RWMutex                      // `mutex` holds a RWMutex (Read-Write Mutex) that is used to protect the eviction algorithm from concurrent access
+	once                       sync.Once                         // `once` holds a Once struct that is used to ensure that the expiration and eviction loops are only started once
+	statsCollectorName         string                            // `statsCollectorName` holds the name of the stats collector that the cache should use when collecting cache statistics
 	// StatsCollector to collect cache statistics
 	StatsCollector stats.ICollector
 }
@@ -100,6 +104,8 @@ func NewInMemoryWithDefaults(capacity int) (*HyperCache[backend.InMemory], error
 //   - The eviction algorithm is set to LRU.
 //   - The expiration interval is set to 30 minutes.
 //   - The stats collector is set to the HistogramStatsCollector stats collector.
+//
+//nolint:cyclop
 func New[T backend.IBackendConstrain](bm *BackendManager, config *Config[T]) (*HyperCache[T], error) {
 	// Get the backend constructor from the registry
 	constructor, exists := bm.backendRegistry[config.BackendType]
@@ -179,15 +185,26 @@ func New[T backend.IBackendConstrain](bm *BackendManager, config *Config[T]) (*H
 		return nil, sentinel.ErrInvalidCapacity
 	}
 
-	// Initialize the expiration trigger channel with the buffer size set to half the capacity
-	hyperCache.expirationTriggerCh = make(chan bool, hyperCache.backend.Capacity()/2)
+	// Initialize the expiration trigger channel with configurable buffer size (default: half capacity)
+	buf := hyperCache.backend.Capacity() / 2
+	if hyperCache.expirationTriggerBufSize > 0 {
+		buf = hyperCache.expirationTriggerBufSize
+	}
+
+	if buf < 1 {
+		buf = 1
+	}
+
+	hyperCache.expirationTriggerCh = make(chan bool, buf)
 
 	hyperCache.startBackgroundJobs()
 
 	return hyperCache, err
 }
 
 // startBackgroundJobs starts the background jobs for the hyper cache.
+//
+//nolint:cyclop
 func (hyperCache *HyperCache[T]) startBackgroundJobs() {
 	// Initialize the eviction channel with the buffer size set to half the capacity
 	ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
@@ -206,6 +223,17 @@ func (hyperCache *HyperCache[T]) startBackgroundJobs() {
 				case <-hyperCache.expirationTriggerCh:
 					// trigger expiration
 					hyperCache.expirationLoop(ctx)
+					// clear pending flag and drain any coalesced triggers quickly
+					hyperCache.expirationSignalPending.Store(false)
+
+					drained := true
+					for drained {
+						select {
+						case <-hyperCache.expirationTriggerCh:
+						default:
+							drained = false
+						}
+					}
 				case <-hyperCache.evictCh:
 					// trigger eviction
 					hyperCache.evictionLoop(ctx)
@@ -241,6 +269,35 @@ func (hyperCache *HyperCache[T]) startBackgroundJobs() {
 	})
 }
 
+// triggerExpiration coalesces and optionally debounces expiration triggers to avoid flooding the channel.
+func (hyperCache *HyperCache[T]) triggerExpiration() {
+	// Optional debounce: if configured, drop triggers that arrive within the interval.
+	if d := hyperCache.expirationDebounceInterval; d > 0 {
+		last := time.Unix(0, hyperCache.lastExpirationTrigger.Load())
+		if time.Since(last) < d {
+			// record backpressure metric
+			hyperCache.StatsCollector.Incr(constants.StatIncr, 1)
+
+			return
+		}
+	}
+
+	// Coalesce: if a signal is already pending, skip enqueueing another.
+	if hyperCache.expirationSignalPending.Swap(true) {
+		hyperCache.StatsCollector.Incr(constants.StatIncr, 1)
+
+		return
+	}
+
+	select {
+	case hyperCache.expirationTriggerCh <- true:
+		hyperCache.lastExpirationTrigger.Store(time.Now().UnixNano())
+	default:
+		// channel full; keep pending flag set and record metric
+		hyperCache.StatsCollector.Incr(constants.StatIncr, 1)
+	}
+}
+
 // expirationLoop is a function that runs in a separate goroutine and expires items in the cache based on their expiration duration.
 func (hyperCache *HyperCache[T]) expirationLoop(ctx context.Context) {
 	hyperCache.workerPool.Enqueue(func() error {
@@ -401,11 +458,8 @@ func (hyperCache *HyperCache[T]) Get(ctx context.Context, key string) (any, bool
 	if item.Expired() {
 		// Return the item to the pool and non-blocking trigger of expiration loop
 		hyperCache.itemPoolManager.Put(item)
-
-		select {
-		case hyperCache.expirationTriggerCh <- true:
-		default:
-		}
+		// Coalesced/debounced trigger
+		hyperCache.triggerExpiration()
 
 		return nil, false
 	}
@@ -428,11 +482,8 @@ func (hyperCache *HyperCache[T]) GetWithInfo(ctx context.Context, key string) (*
 	if item.Expired() {
 		// Return the item to the pool and non-blocking trigger of expiration loop
 		hyperCache.itemPoolManager.Put(item)
-
-		select {
-		case hyperCache.expirationTriggerCh <- true:
-		default:
-		}
+		// Coalesced/debounced trigger
+		hyperCache.triggerExpiration()
 
 		return nil, false
 	}
@@ -452,11 +503,8 @@ func (hyperCache *HyperCache[T]) GetOrSet(ctx context.Context, key string, value
 		if item.Expired() {
 			// Return the item to the pool and non-blocking trigger of expiration loop
 			hyperCache.itemPoolManager.Put(item)
-
-			select {
-			case hyperCache.expirationTriggerCh <- true:
-			default:
-			}
+			// Coalesced/debounced trigger
+			hyperCache.triggerExpiration()
 
 			return nil, sentinel.ErrKeyExpired
 		}
@@ -536,11 +584,8 @@ func (hyperCache *HyperCache[T]) GetMultiple(ctx context.Context, keys ...string
 			hyperCache.itemPoolManager.Put(item)
 			// Treat expired items as not found per API semantics
 			failed[key] = sentinel.ErrKeyNotFound
-			// Non-blocking trigger of the expiration loop via channel
-			select {
-			case hyperCache.expirationTriggerCh <- true:
-			default:
-			}
+			// Coalesced/debounced trigger of the expiration loop via channel
+			hyperCache.triggerExpiration()
 		} else {
 			item.Touch() // Update the last access time and access count
 			// Add the item to the result map

pkg/eviction/arc.go

@@ -47,37 +47,47 @@ func NewARCAlgorithm(capacity int) (*ARC, error) {
 // Get retrieves the item with the given key from the cache.
 // If the key is not found in the cache, it returns nil.
 func (arc *ARC) Get(key string) (any, bool) {
-	arc.mutex.RLock()
+	arc.mutex.Lock()
+	defer arc.mutex.Unlock()
 
 	// Check t1
 	item, ok := arc.t1[key]
 	if ok {
-		arc.mutex.RUnlock()
 		arc.promote(key)
 
 		return item.Value, true
 	}
 	// Check t2
 	item, ok = arc.t2[key]
 	if ok {
-		arc.mutex.RUnlock()
 		arc.demote(key)
 
 		return item.Value, true
 	}
 
-	arc.mutex.RUnlock()
-
 	return nil, false
 }
 
 // Set adds a new item to the cache with the given key.
 func (arc *ARC) Set(key string, value any) {
-	arc.mutex.RLock()
-	defer arc.mutex.RUnlock()
-	// Check if key is already in cache
-	_, ok := arc.Get(key)
-	if ok {
+	arc.mutex.Lock()
+	defer arc.mutex.Unlock()
+
+	if arc.capacity == 0 {
+		// Zero-capacity ARC is a no-op
+		return
+	}
+
+	// If key exists in t1 or t2, update value only
+	if item, ok := arc.t1[key]; ok {
+		item.Value = value
+
+		return
+	}
+
+	if item, ok := arc.t2[key]; ok {
+		item.Value = value
+
 		return
 	}
 
@@ -93,9 +103,7 @@ func (arc *ARC) Set(key string, value any) {
 	}
 
 	item := arc.itemPoolManager.Get()
-
 	item.Value = value
-
 	arc.t1[key] = item
 	arc.c++
 
@@ -135,11 +143,13 @@ func (arc *ARC) Delete(key string) {
 // Evict removes an item from the cache and returns the key of the evicted item.
 // If no item can be evicted, it returns an error.
 func (arc *ARC) Evict() (string, bool) {
+	if arc.capacity == 0 {
+		return "", false
+	}
 	// Check t1
 	for key, val := range arc.t1 {
 		delete(arc.t1, key)
 		arc.c--
-
 		arc.itemPoolManager.Put(val)
 
 		return key, true
@@ -148,7 +158,6 @@ func (arc *ARC) Evict() (string, bool) {
 	for key, val := range arc.t2 {
 		delete(arc.t2, key)
 		arc.c--
-
 		arc.itemPoolManager.Put(val)
 
 		return key, true