fix(cache): make in memory cache global

[MasterPtato]

Description

Please include a summary of the changes and the related issue. Please also include relevant motivation and context.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[railway-app]

🚅 Deployed to the rivet-pr-4560 environment in rivet-frontend

Service	Status	Web	Updated (UTC)
kitchen-sink	❌ Build Failed (View Logs)	Web	Apr 8, 2026 at 8:15 pm
mcp-hub	✅ Success (View Logs)	Web	Apr 5, 2026 at 7:25 pm
website	❌ Build Failed (View Logs)	Web	Apr 5, 2026 at 7:25 pm
frontend-cloud	❌ Build Failed (View Logs)	Web	Apr 5, 2026 at 7:24 pm
ladle	❌ Build Failed (View Logs)	Web	Apr 5, 2026 at 7:24 pm
frontend-inspector	❌ Build Failed (View Logs)	Web	Apr 5, 2026 at 7:24 pm

[MasterPtato]

• chore: remove non-engine drivers #4574 : 2 dependent PRs (#4581 , #4582 )
• feat: dynamic actors #4397
• fix: driver test suite & sqlite reliability #4559
• fix: remove serverless token #4572
• fix: misc token fixes #4569
• fix(pb): clean up actor stop decision handling #4566
• fix(envoy): use global instance, add signal handlers #4565
• chore: misc fixes, add pb snapshot test #4561
• fix(cache): make in memory cache global #4560 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[claude]

PR Review: fix(cache): make in memory cache global

Summary

This PR converts two instance-level fields (InMemoryDriver.cache and CacheInner.in_flight) into process-global singletons backed by std::sync::OnceLock. It also adds an eviction listener for debug logging, a short-circuit in fetch_all_convert for empty cache key sets, and improves tracing instrumentation.

The motivation is sound: sharing one moka cache across all CacheInner instances ensures cache hits propagate regardless of which instance originally populated the cache. However there are two real issues that need addressing before merge.

---

Critical: max_capacity Is Silently Ignored After First Init

File: engine/packages/cache/src/driver.rs

OnceLock::get_or_init initialises the cache exactly once. Every subsequent call to InMemoryDriver::new(max_capacity) silently ignores the max_capacity argument. Looking at the callers:

• gasoline/src/worker.rs calls new_in_memory(10000, ups)
• engine/packages/cache/tests/*.rs calls new_in_memory(1000, None)

If a test binary runs before production code bootstraps, the global cache is initialized with capacity 1,000 instead of 10,000. The max_capacity parameter is effectively meaningless after the first call, which is misleading and could mask capacity configuration bugs.

Recommendation: Either emit a tracing::warn! when a subsequent call provides a differing max_capacity value, or panic in debug builds on mismatch, or remove the parameter entirely and use a named constant or an environment variable.

---

Significant: Global State Breaks Test Isolation

Files: engine/packages/cache/tests/fetch.rs, in_flight.rs, ttl.rs

Every test calls build_cache() which calls new_in_memory(1000, None). With the old design each call produced an independent, isolated cache. With the new design all tests in the same process share one CACHE and one IN_FLIGHT map. This means:

1. Values written by one test can be read by another; tests are no longer isolated.
2. IN_FLIGHT can retain stale broadcast::Sender entries left behind by a panicking test, potentially causing subsequent tests to hang.
3. Tests that assert a value is None become order-dependent and may spuriously fail if a parallel test inserted the same key.

The tests currently use distinct base_key strings per test, which provides some protection, but it is fragile. Any future test that reuses a key name will silently pick up cached data.

Recommendation: Either keep a per-instance cache in test builds via a #[cfg(test)] path, or add a comment in the test helper making the uniqueness requirement explicit, and consider adding a test-only clear() helper.

---

Minor: Stale In-Flight Entries After purge_local

When purge_local removes a key from the moka cache, the corresponding IN_FLIGHT entry is not removed and its broadcast::Sender is not signalled. This is pre-existing, but the move to a global IN_FLIGHT makes it more visible. Stale senders now persist for the lifetime of the process rather than being cleaned up when a CacheInner is dropped.

---

Minor: Panic Message in cache()

The message "should be initialized" is terse. Consider "CACHE accessed before InMemoryDriver::new was called" to aid debugging if it ever fires.

---

Positives

• The short-circuit for empty succeeded_cache_keys is a clean, correct optimization.
• Switching from skip(keys, getter, encoder, decoder) to skip_all, fields(?base_key) in the tracing annotation is a clear improvement.
• scc::HashMap for IN_FLIGHT is the right choice per repo conventions (avoids Arc<Mutex<HashMap>>).
• Log messages follow conventions (lowercase, structured fields).

---

Convention Check

Convention	Status
tracing macros (no println!)	Compliant
Structured logging (?key, not string format)	Compliant
Lowercase log messages	Compliant
scc::HashMap over Arc<Mutex<HashMap>>	Compliant
Comments as complete sentences, no em dashes	Compliant

[MasterPtato]

Merge activity

• Apr 8, 8:52 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Apr 8, 8:52 PM UTC: @MasterPtato merged this pull request with Graphite.