feat: L2 Cache with Valkey Client Side Hash Ring by larry-dalmeida · Pull Request #4033 · zalando/skipper

larry-dalmeida · 2026-05-26T10:40:59Z

Related Issue

Description

Extends the cache() filter with an optional Valkey-backed L2 cache using a client-side consistent hash ring.

When --swarm-valkey-urls is configured, responses are stored in Valkey (L2) with automatic fallback to the in-process LRU (L1) on any Valkey error.

Storage architecture

flowchart TD
    A[Incoming Request] --> B["Skipper cache() filter"]
    B --> C{L1 LRUStorage}
    C -->|l1_hit| S[Entry Served]
    C -->|miss| D{ValkeyStorage L2}
    D -->|valkey_miss — key absent| E[Origin Fetch\nContentful CDN / Pegasus]
    D -->|valkey_get_fallback — error / timeout| C2{L1 LRUStorage fallback}
    C2 -->|hit| S
    C2 -->|miss| E
    E -->|write back: Set warms Valkey + L1<br/>min TTL: cache-l1-ttl vs entry.TTL| S

Write path: successful Valkey Set warms L1 with min(--cache-l1-ttl, entry.TTL) (default 60 s). L1 is also populated on Valkey errors (fallback). Set --cache-l1-ttl=0 for write-around.

Read path: L1 checked first - hit returns immediately (increments l1_hit), no Valkey call. L1 miss → Valkey. Valkey miss (nil) → cold miss. Valkey error → valkey_get_fallback, L1 consulted as fallback.

Valkey ring topology

flowchart LR
    subgraph Pods
        P1[Pod A]
        P2[Pod B]
        P3[Pod C]
    end

    subgraph Ring["Valkey Hash Ring (client-side)"]
        direction LR
        V1[(Shard 0)]
        V2[(Shard 1)]
        V3[(Shard 2)]
    end

    P1 -- "key → consistent hash → same shard" --> V1
    P2 -- "same key → same shard" --> V1
    P3 --> V2
    P1 --> V3

All pods share the same ring, so a cold-miss coalesced by pod A lands in the same shard that pod B would read from - no thundering herd across pods.

Observability

Counter	Meaning
`l1_hit`	L1 returned a warm entry; Valkey not consulted
`valkey_miss`	Key absent in Valkey (clean miss)
`valkey_get_fallback`	Valkey error on Get; L1 consulted
`valkey_set_fallback`	Valkey error on Set; L1 written

lru_bytes gauge is now updated by a background scraper every 10 s instead of only on eviction, so it stays accurate when capacity is not exceeded.

Storage Set and Delete errors are now logged at Warn instead of being silently discarded.

szuecs · 2026-05-29T19:56:42Z

+			// Shallow copy so NewValkeyRingClient can mutate opt.Addrs without
+			// racing against the ratelimit registry's copy of the same pointer.
+			cacheValkeyOpts := *valkeyOptions
+			cacheValkeyRing, err = skpnet.NewValkeyRingClient(&cacheValkeyOpts)


Why not passing the same client?
I know that you can not know this, but in the end we can refactor code that I want to move around a bit and some I want to delete later, so we can make this nice. :)

Please merge larry-dalmeida#2 , if you agree that this makes sense

szuecs · 2026-06-01T15:14:49Z

👍

larry-dalmeida · 2026-06-02T07:47:16Z

👍

a4180p · 2026-06-02T08:34:46Z

Write path: successful Valkey Set does not warm L1 (write-around). L1 is only populated on Valkey errors.
Read path: Valkey miss → nil (clean miss, no L1 consulted). Valkey error → L1 consulted as fallback.

That actually implies that l1 cache will be empty in case if Set was successfull and only Get fails.
Also, this model means that cache will be slower - there is no fast l1 layer, positive path always reads and writes from/to network storage.
Is that desired behaviour?
I would expect a small write/read-through L1 cache to serve as a fast local layer for the common case, but I may be missing some context around the intended use case.

szuecs · 2026-06-02T08:54:58Z

Write path: successful Valkey Set does not warm L1 (write-around). L1 is only populated on Valkey errors.
Read path: Valkey miss → nil (clean miss, no L1 consulted). Valkey error → L1 consulted as fallback.

That actually implies that l1 cache will be empty in case if Set was successful and only Get fails. Also, this model means that cache will be slower - there is no fast l1 layer, positive path always reads and writes from/to network storage. Is that desired behaviour? I would expect a small write/read-through L1 cache to serve as a fast local layer for the common case, but I may be missing some context around the intended use case.

Good point, I wondered about this and missed to ask the question.
Normally (for example CPU) you would check L1 cache and if you get a cache miss, you check L2 cache and if you have a hit you will save in L1 and respond.

larry-dalmeida · 2026-06-02T12:36:51Z

@a4180p @szuecs @MustafaSaber

Intended behavior: L1 is a degraded-mode fallback only, not hot cache layer path. Happy path always reads from and writes to L2. L1 is only populated when a valkey operation fails.

CPI L1/L2 cache hierarchy

works because 1) L2 is on-chip with 5-30ns latency. L1 saves nanoseconds, not milliseconds. 2) Cache coherence is handled at hardware level (MESI protocol) - when core writes to L1, hardware broadcasts invalidation to other core’s L1 automatically. This does not transfer here:

L1 is an LRU per Skipper pod - there can be n pods, each with own private LRU. No hardware coherence protocol

Valkey round trip is ~0.5-1ms over local cluster network - latency is higher than L1 cache but problem being solved is not sub-ms latency but rather cross-pod cache sharing.

Primary goal: cross-pod consistency

Without L2, every pod maintains an independent LRU. A cold miss on pod A fetches from origin, pod B's LRU is empty and fetches independently. Under load (e.g. a popular campaign going live), this produces an N-way thundering herd - one upstream fetch per pod, not one per cluster.

Valkey solves this because the consistent-hash ring maps a given cache key to the same shard regardless of which pod is making the request. A cold-miss coalesced by pod A writes to Valkey shard S. Pod B's next request for the same key hits shard S directly - no upstream fetch.

Why warming L1 on write undermines this goal

If L1 is warmed on a successful Valkey Set (write-through):

A pod serves future requests from its local LRU, bypassing Valkey.
Valkey Delete or TTL-based expiry does not reach L1 - the pod continues serving stale content until L1's own TTL expires.
There is no invalidation channel. Adding one (e.g. Valkey pub/sub) would require every pod to subscribe, handle reconnects, and accept a bounded staleness window on subscriber lag.

The tradeoff is: faster reads on the hot path vs. stale content served after invalidation, with non-trivial invalidation infrastructure.

Why Valkey-miss does not consult L1

A Valkey miss (nil, no error) means the key is genuinely absent - no pod has fetched and stored it yet, or the TTL expired. Consulting L1 on a clean miss would serve stale content beyond the intended TTL. The filter must go to origin.

L1 is only consulted when Valkey returns an error, because in that case we have no authoritative answer. Serving a potentially-stale L1 entry is preferable to a 5xx.

Considered alternatives

Write-through with TTL-bounded staleness

Warm L1 on every successful Set, accept that L1 entries can be served up to their TTL after a Valkey Delete. For content that is never explicitly invalidated (only TTL-expired), this is semantically equivalent to the current design - and would reduce Valkey read load.

Not chosen because explicit Delete is on the roadmap (cache invalidation on content publish events). Once that lands, write-through without an invalidation channel produces observable stale responses.

Write-through + Valkey pub/sub invalidation

Warm L1, subscribe each pod to a Valkey pub/sub channel for invalidation events. This matches the CPU L1/L2 mental model most closely.

Not chosen for this PR. The complexity cost is high (subscribe lifecycle, reconnect handling, message delivery guarantees, lag-bounded staleness), and the latency benefit does not yet justify it. This is the natural next step if Valkey read latency becomes a bottleneck.

szuecs · 2026-06-02T13:45:03Z

Considered alternatives

Write-through with TTL-bounded staleness

Warm L1 on every successful Set, accept that L1 entries can be served up to their TTL after a Valkey Delete. For content that is never explicitly invalidated (only TTL-expired), this is semantically equivalent to the current design - and would reduce Valkey read load.

Not chosen because explicit Delete is on the roadmap (cache invalidation on content publish events). Once that lands, write-through without an invalidation channel produces observable stale responses.

We could have L1 entry TTL of a fixed acceptable amount of time.
We have for example token result caching in tokeninfo filters set to 60s and it offloaded 50% of total CPU time and latency dropped from 10-20ms to <1ms.
Given our experience I would be in favour of having a fixed TTL of 60s for L1 cache in case there is an L2 cache setup configured.

larry-dalmeida · 2026-06-02T14:12:22Z

Given our experience I would be in favour of having a fixed TTL of 60s for L1 cache in case there is an L2 cache setup configured.

Good point. tokeninfo data is a strong precedent.

The write-around choice was conservative: L1 and Valkey TTLs are independent, and if a Valkey entry expires or gets evicted, an L1 entry with a longer TTL would silently serve stale content with no signal. The intent was to keep Valkey authoritative for the lifetime of every entry.

That said, your proposal sidesteps the problem cleanly.
A fixed short TTL (e.g. 60 s) on L1 should result in:

staleness is bounded and operator-visible - it's a deliberate configuration choice, not a silent race
the Delete path already calls l1.Delete unconditionally, so explicit invalidations would propagate correctly
no invalidation channel needed

Trade-off is: cache filter TTL must be meaningfully longer than the L1 TTL for the L1 layer to be useful.
For short-lived entries (< 60 s) the L1 TTL would need to be proportionally smaller, so it is likely to be configurable rather than hard-coded.

For now I will proceed with setting 60s as fixed TTL as a start.

a4180p · 2026-06-02T15:37:08Z

@larry-dalmeida

The intent was to keep Valkey authoritative for the lifetime of every entry.

After every successfull read from L1 you can call EXPIRE to valkey comand to:

prolongate TTL of the record if it exists
get a signal if it not exists and cleanup the record in L1

You can also consider using GETEX instead of GET if you wan TTLs to be updated on read ops.

szuecs · 2026-06-02T17:13:52Z

@larry-dalmeida

The intent was to keep Valkey authoritative for the lifetime of every entry.

After every successfull read from L1 you can call EXPIRE to valkey comand to:

This you can't really do because l2 cache is shared by all skipper instances
In the rate limit cases we set Expiry time on create entry. I think it's the way to go for expiration by time.

larry-dalmeida · 2026-06-05T07:52:06Z

@szuecs @a4180p @MustafaSaber thanks a lot for the feedback and patience 🧡 I will review it thoroughly and get back to you with concrete proposal. Currently I am busy with a business critical reliability related topic but I will resume this on Monday 15th June.

larry-dalmeida · 2026-06-16T05:38:00Z

Thanks for your patience folks, our 🔥 are out, resuming this today.

larry-dalmeida · 2026-06-22T06:36:58Z

Update: fixing final failing tests. PR will be ready for review shortly.
@MustafaSaber @a4180p @szuecs

larry-dalmeida · 2026-06-22T15:03:23Z

@szuecs ran all tests locally - all pass locally. Let me know if you have concerns.

The len(results)==0 check was placed after the error-checking for-loop, making it read as dead code. Reorder to guard-first for clarity: validate non-empty results, then iterate to check for errors from the SET and EXPIRE commands, then return success. Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

…evalJob Replace the func() closure field on revalJob with a *cacheFilter pointer and call job.filter.doRevalidate(job.key, job.req) directly in the worker. The closure was only binding key and req, which revalJob already carries as fields, making the indirection unnecessary. Also fixes two missing spec.Close() calls in TestCreateFilter_InvalidArgs and TestCacheFilter_SharedStorage_RouteIsolation that caused goroutine leaks. Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

larry-dalmeida · 2026-06-30T04:33:04Z

@szuecs @MustafaSaber @a4180p I've updated to reflect the feedback. Please review when you have a moment.

FYI - I've tested the L1 cache on staging successfully, today will beta test on production. Will repeat the same with L2 cache included one we've rolled out skipper updates.

szuecs · 2026-06-30T18:38:32Z

@larry-dalmeida can you check the picture in the description if this is up to date?
I think the l2 cache miss and then fallback to l1 makes no sense if we have already checked l1 first.
So on l2 cache miss we should populate all caches with the value.

szuecs · 2026-06-30T19:24:37Z

+	ListenAddr string                   // Skipper's own listener address (e.g., ":9090")
+	NetOpts    skpnet.Options           // network options for revalidation requests
+	ValkeyRing *skpnet.ValkeyRingClient // optional L2 cache backend; nil = LRU only
+	L1TTL      time.Duration            // max TTL to use when warming L1 from Valkey writes


If L2 cache is disable this comment is not true. As far as I remember this is also not optional so we should have a better comment to describe the behaviour or drop all the comments.

szuecs · 2026-06-30T19:25:10Z

+// Options configures the cache filter. All fields are required unless stated otherwise.
+type Options struct {
+	MaxBytes   int64                    // in-memory storage budget for the LRU
+	ListenAddr string                   // Skipper's own listener address (e.g., ":9090")


here it would be interesting to tell in the comment why do we need this, because it's not obvious for the reader.

szuecs · 2026-06-30T19:30:39Z

+func (s *cacheSpec) Close() {
+	select {
+	case <-s.lruBytesDone:
+		// Already closed; prevent panic on double-close.


Right now we have a race condition within this function if 2 goroutines enter the function at the same time, both can enter "default" branch leading to double close panics.

I think you can fix the guard by using a sync.Once and do:

s.once.Do(func() { ..here close all things... })

Good catch! will address that.

szuecs · 2026-06-30T19:36:11Z

+	defer s.bgWg.Done()
+	for job := range s.revalJobs {
+		if job.filter != nil {
+			job.filter.doRevalidate(job.key, job.req)


We should measure here how long the job runs and expose a metric, because if we have let's say 10k filter instances that do revalJobs it likely gets slow, because we depend on the origin to fetch the data and it depends also on the data size. It's unclear to me how it will behave so measuring is likely a must.

Will add a metric

szuecs · 2026-06-30T19:44:12Z

-	key string
-	req *http.Request // pre-cloned, safe to use after the originating request ends
+	key    string
+	req    *http.Request // pre-cloned, safe to use after the originating request ends


Interesting would be in the comment if the Body is also copied or you should not use the Body at all. Maybe it's also never having a Body, because of GET and HEAD only, but what if we have QUERY, that is already specified as RFC?

I will add support for QUERY, it should be small effort.

szuecs · 2026-06-30T19:48:22Z

-	metrics   metrics.Metrics
+	rfcMode      bool
+	coldSF       singleflight.Group // cold-miss coalescing
+	revalSF      singleflight.Group // coalesces concurrent background revalidations per key


Do you think we need to de-duplicate also in the job runner at spec level?
The problem is that multiple filter instances can create the same reval job even if we guard it with a singleflight.Group here.

Not needed (cache key includes routeID -> distinct namespaces). But the registry improves it: routes with identical config now share one cacheFilter and same revalSF, automatically deduplicating revalidation.

szuecs · 2026-06-30T19:53:29Z

+
+// Close is a no-op on individual filter instances; the real Close is on cacheSpec.
+// This exists for test compatibility.
+func (f *cacheFilter) Close() {}


TODO(szuecs): check if we need this for tests or if this can be cleaned up

szuecs · 2026-06-30T20:43:17Z

+// revalidationWorker is the single background goroutine (spec-level, shared across
+// all filter instances) that processes revalidation jobs sequentially. It calls
+// the per-instance doRevalidate method to respect each route's configuration.
+func (s *cacheSpec) revalidationWorker() {


Do we have a test case that rebuilds the routing tree while having a queue of jobs that is being worked on?

no, good call. I will add that.

Added, now verifies spec-level worker continues draining jobs when CreateFilter is called again (rebuild), and registry returns same instance.

szuecs · 2026-06-30T20:44:32Z

+	}
+}
+
 func (s *cacheSpec) CreateFilter(args []interface{}) (filters.Filter, error) {


Because we moved the revaljob queue worker to the spec, we need to make sure we do not destroy filter instances and create new ones, because we would loose all state that we have in the cache filter instance.
We have the same problem in fifo and lifo filters, which was solved by creating a separate package and a registry, because we needed more than just that.
I would propose to solve the problem by caching the filter instance in the spec.

type filterCacheKey struct { // store parsed arguments here } type cacheSpec struct { ... muFilter sync.Mutex filters map[filterCacheKey]*cacheFilter }

In CreateFilter you will parse all args and build the filterCacheKey to lookup from spec.filters. If we have a cacheFilter we return it from here. If we do not have one we create a new one and store it in the spec.filters map.
We guard in CreateFilter by spec.muFilter.Lock the whole process from before filters lookup until after we set the new created cacheFilter instance into the spec.filters map.

Interestingly we will make the whole thing more efficient at scale. If you think about 10k routes with the cache filter, we might have 90% duplicated configuration and we will then share the filter instance across all of them, so the revalidation will be also de-duplicated across different routes because we have the same filter instance and use the same singleflight group.

Why this works: https://go.dev/play/p/TWm1WhBMSdo

Thanks for the examples and comments. I've added filterCacheKey registry to cacheSpec. Routes with identical config now return the same cacheFilter pointer, preserving singleflight state across route updates and deduplicating revalidation across routes.

szuecs · 2026-06-30T20:53:19Z

+		}
+		s.metrics.IncCounter("valkey_get_fallback")
+		log.WithError(err).Warn("cache: valkey Get failed, falling back to L1")
+		return s.l1.Get(ctx, key)


We checked already in line 49 if we have a l1 cache hit, therefore this will never work or we hope to get a race condition and another call populated the cache.
Is this done by intention?

yes, intentional - there's a valid race window between the two L1 reads.
Line 49 checks L1 and misses. Line 54 then issues a blocking network call to Valkey. During that round-trip, a concurrent Set call can reach line 97 and warm L1. If Valkey then returns an error (ex: network call fails), line 62 catches the now-warm L1 entry rather than returning a cold miss. Do you think this is overkill?

szuecs · 2026-06-30T21:00:13Z

+}
+
+func (s *ValkeyStorage) Delete(ctx context.Context, key string) error {
+	// ValkeyRingClient exposes no DEL; use EXPIRE key -1 (immediate deletion per Valkey docs).


It sounds like a hack, maybe better to expose a DEL command in the ValkeyRingClient.

Added proper Del() method to ValkeyRingClient. Updated ValkeyStorage.Delete() to use Del instead of EXPIRE key -1 workaround.

Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

…ute updates Routes with identical cache filter configuration now share a single cacheFilter instance, preserving singleflight state (coldSF, revalSF) across route updates. This also deduplicates revalidation across routes with identical configuration. The registry is keyed by (ttl, errorTTL, swrWindow, staleIfError, sorted keyHeaders, rfcMode). keyHeaders are sorted to normalize order-independent configurations like 'X-Foo,X-Bar' vs 'X-Bar,X-Foo'. Filter instances are stored in cacheSpec.filters and accessed under cacheSpec.muFilter lock. Lookup-or-create happens entirely within CreateFilter; no PostProcessor stage is needed since filter config is the complete identity. Fixes PR zalando#4033 comment r3501729935 (instance loss on route update). Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

Move struct declarations to their conventional position before the functions that use them. This improves code readability without changing behavior. Also add TestCacheSpec_FilterRegistry_InFlightJobsSurviveRebuild to directly verify szuecs' question on comment r3501723301: does the spec-level worker continue draining revalidation jobs when CreateFilter is called again (simulating a route rebuild)? Answer: yes. The worker is independent of CreateFilter calls; it drains revalJobs continuously. The registry ensures the same cacheFilter instance is returned on rebuild, so the job queue remains intact. Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

Replace the EXPIRE key -1 workaround in ValkeyStorage.Delete() with a proper DEL command. This eliminates the 'hack' comment that szuecs flagged in comment r3501815470. Changes: - Add Del() method to valkeyRing (net/valkey.go) - Add Del() method to ValkeyRingClient (net/valkey.go) - Add Del() to the valkeyClient interface (filters/cache/valkey_storage.go) - Update Delete() to use Del instead of Expire key -1 (filters/cache/valkey_storage.go) - Add Del() to stubValkeyClient test stub (filters/cache/valkey_storage_test.go) This makes the code more idiomatic and removes the complexity of the EXPIRE key -1 semantics workaround. Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

The upstream metrics.Metrics interface now includes MeasureBackendZone, which was added after this branch diverged. Add a no-op stub to testMetrics to satisfy the interface and fix 'make vet' failures across all test suites. Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

a4180p · 2026-07-15T20:47:38Z

+By default entries are stored in an in-process LRU (L1) local to each Skipper process.
+When `--swarm-valkey-urls` is configured, Valkey becomes the primary shared
+store (L2) accessible by all Skipper instances via a client-side consistent hash ring. Every
+read checks L1 first; an L1 hit returns without contacting Valkey.


I think that contradicts the PR description now.
Please check the description, including the diagramm if it matches implementation.

a4180p · 2026-07-15T20:50:18Z

+    The in-process LRU (L1) is shared across all `cache()` filter instances in the same process.
+    When Valkey is configured it acts as a shared store (L2). The L1 storage budget is divided evenly
+    across 256 internal shards; a single entry larger than one shard's budget is
+    dropped with a warning log.


I don't understand this part. Why the valkey is mentioned in this note?

a4180p · 2026-07-15T21:02:24Z

+}
+
+// coalesce gates concurrent cold misses for the same key behind a single upstream
+// fetch, preventing thundering herd. All waiters receive the same response.


I think it will be beneficial to mention that thundering herd pervention is limited only to one skipper node.
If you have a fleet of N instances - your backend could receive up to N simultanious requests.
(or you need to implement distributed lock on top of valkey, in that case this comments will be true)

a4180p · 2026-07-15T21:11:58Z

+	}
 	select {
-	case f.revalJobs <- revalJob{key: key, req: cloned}:
+	case f.revalJobs <- job:


what will happen if this call happens simultaniously with Close?

a4180p · 2026-07-16T17:46:51Z

+	var e Entry
+	if err := json.Unmarshal([]byte(data), &e); err != nil {
+		return nil, fmt.Errorf("cache: decode valkey entry: %w", err)
+	}
+	return &e, nil


pr description says

automatic fallback to the in-process LRU (L1) on any Valkey error

a4180p · 2026-07-16T17:57:58Z

 	cloned := orig.Clone(context.Background())
+	var bodySnapshot []byte
+	if orig.Method == "QUERY" && orig.Body != nil && orig.Body != http.NoBody {
+		bodySnapshot, _ = io.ReadAll(orig.Body)


why do we ignore error here?

a4180p · 2026-07-16T18:00:20Z

+	// QUERY carries semantics in its body; include it in the key so different
+	// queries to the same URL produce distinct cache entries.
+	if r.Method == "QUERY" && r.Body != nil && r.Body != http.NoBody {
+		body, err := io.ReadAll(r.Body)


if err != nil we don't add body to the key, but continue to process.
is that desired behaviour?

a4180p · 2026-07-16T18:02:37Z

+}
+
+func (s *ValkeyStorage) Delete(ctx context.Context, key string) error {
+	// Valkey errors are best-effort — L1 delete always runs.


the problem is that all other skipper instances will not delete the value from l1

a4180p · 2026-07-16T18:05:26Z

 }

 const (
-	redisMetricsPrefix                    = "swarm.redis."


can we move this and similar part in valkey.go to the separate PR to make this one a bit smaller?

a4180p · 2026-07-16T18:07:27Z

 	flag.StringVar(&cfg.SwarmStaticOther, "swarm-static-other", "", "set static swarm all nodes, for example 127.0.0.1:9002,127.0.0.1:9003")

+	// cache
+	flag.DurationVar(&cfg.CacheL1TTL, "cache-l1-ttl", 60*time.Second, "maximum TTL for write-through L1 warming in the cache() filter when Valkey is configured; set to 0 to disable (write-around)")


shouldn't be there a --cache-l2-ttl also?

larry-dalmeida force-pushed the feat/cache-client-side-valkey-hash-ring branch 2 times, most recently from 7047126 to 573ebdf Compare May 27, 2026 04:40

larry-dalmeida changed the title ~~Add L2 Cache with Valkey Client Side Hash Ring~~ feat: Add L2 Cache with Valkey Client Side Hash Ring May 27, 2026

larry-dalmeida changed the title ~~feat: Add L2 Cache with Valkey Client Side Hash Ring~~ feat: L2 Cache with Valkey Client Side Hash Ring May 27, 2026

larry-dalmeida force-pushed the feat/cache-client-side-valkey-hash-ring branch from 55ba444 to c73eb72 Compare May 27, 2026 09:55

larry-dalmeida marked this pull request as ready for review May 28, 2026 07:56

szuecs added the architectural all changes in the hot path, big changes in the control plane, control flow changes in filters label May 29, 2026

szuecs reviewed May 29, 2026

View reviewed changes

szuecs requested review from MustafaSaber and a4180p June 1, 2026 20:02

MustafaSaber reviewed Jun 2, 2026

View reviewed changes

Comment thread skipper.go Outdated

larry-dalmeida marked this pull request as draft June 2, 2026 14:21

larry-dalmeida force-pushed the feat/cache-client-side-valkey-hash-ring branch 2 times, most recently from 7d1af73 to c1fb3cd Compare June 18, 2026 07:19

github-actions Bot added the documentation label Jun 19, 2026

larry-dalmeida force-pushed the feat/cache-client-side-valkey-hash-ring branch from 1071146 to 8760752 Compare June 22, 2026 06:52

larry-dalmeida marked this pull request as ready for review June 22, 2026 11:55

larry-dalmeida added 2 commits June 30, 2026 05:52

larry-dalmeida force-pushed the feat/cache-client-side-valkey-hash-ring branch from 54f559c to 2b4ed9b Compare June 30, 2026 03:54

larry-dalmeida marked this pull request as ready for review June 30, 2026 04:29

szuecs reviewed Jun 30, 2026

View reviewed changes

larry-dalmeida added 8 commits July 1, 2026 14:41

refactor: simplify Options struct comments per code review

19de285

Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

fix: replace select-based Close guard with sync.Once to prevent race

4560868

Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

feat: add reval_duration histogram metric to revalidation worker

d56c990

Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

feat: add QUERY method support to cache filter

bff47dc

Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

docs: clarify cacheFilter.Close no-op intent

4e80273

Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

docs: fix forward reference in cacheFilter.Close comment

16820c8

Signed-off-by: larry-dalmeida <hello@larrydalmeida.com>

larry-dalmeida marked this pull request as draft July 3, 2026 05:56

larry-dalmeida added 2 commits July 3, 2026 08:29

larry-dalmeida marked this pull request as ready for review July 9, 2026 07:34

a4180p reviewed Jul 16, 2026

View reviewed changes

Uh oh!

Conversation

larry-dalmeida commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issue

Description

Storage architecture

Valkey ring topology

Observability

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

szuecs commented Jun 1, 2026

Uh oh!

larry-dalmeida commented Jun 2, 2026

Uh oh!

a4180p commented Jun 2, 2026

Uh oh!

szuecs commented Jun 2, 2026

Uh oh!

Uh oh!

larry-dalmeida commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CPI L1/L2 cache hierarchy

Primary goal: cross-pod consistency

Why warming L1 on write undermines this goal

Why Valkey-miss does not consult L1

Considered alternatives

Write-through with TTL-bounded staleness

Write-through + Valkey pub/sub invalidation

Uh oh!

szuecs commented Jun 2, 2026

Considered alternatives

Write-through with TTL-bounded staleness

Uh oh!

larry-dalmeida commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

a4180p commented Jun 2, 2026

Uh oh!

szuecs commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

larry-dalmeida commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

larry-dalmeida commented Jun 16, 2026

Uh oh!

larry-dalmeida commented Jun 22, 2026

Uh oh!

larry-dalmeida commented Jun 22, 2026

Uh oh!

larry-dalmeida commented Jun 30, 2026

Uh oh!

szuecs commented Jun 30, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

szuecs Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

larry-dalmeida Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

larry-dalmeida commented May 26, 2026 •

edited

Loading

larry-dalmeida commented Jun 2, 2026 •

edited

Loading

larry-dalmeida commented Jun 2, 2026 •

edited

Loading

szuecs commented Jun 2, 2026 •

edited

Loading

larry-dalmeida commented Jun 5, 2026 •

edited

Loading

szuecs Jun 30, 2026 •

edited

Loading

larry-dalmeida Jul 9, 2026 •

edited

Loading

larry-dalmeida Jul 2, 2026 •

edited

Loading