Commit 640e60a
feat(core): optional distributed object cache for query results (#1378)
* feat(core): optional distributed object cache for query results
Add an opt-in read-through cache that sits beneath the per-request cache
and above the database, so content and chrome (settings, menus,
taxonomies) reads can be served from a fast key/value store instead of
hitting D1/SQLite on every request.
- New ObjectCache abstraction (interface + descriptor + virtual module
+ per-isolate backend), mirroring the storage adapter pattern. Off by
default: when unconfigured, cachedQuery is a transparent passthrough.
- Backends: in-isolate memory (emdash/object-cache/memory via
memoryCache() from emdash/astro) and Cloudflare KV
(@emdash-cms/cloudflare/cache/kv via kvCache()).
- JSON codec preserves Date instances; content entries are snapshotted
(dropping the .edit proxy, capturing the CURSOR_RAW_VALUES symbol) and
rebuilt on read.
- Epoch-based invalidation at the repository chokepoint (content, seo,
byline, taxonomy, menu) and settings; content reads fold in shared
bylines/taxonomies epochs so author/term renames invalidate correctly.
- Auth/preview/edit-mode and isolated DBs always bypass.
Existing sites are unaffected until they opt in.
* fix(core): bound object-cache backend reads with a timeout
A KV read that stalls without resolving or rejecting (cold cross-region
read, or one queued behind the Workers connection limit) could hang the
isolate: getEpoch cached the never-settling promise and every later
cached read on that namespace reused it, poisoning the isolate until it
recycled.
- Race every backend read against a timeout (default 2000ms, configurable
via the `timeout` option on kvCache/objectCache, 0 disables). A timed-out
read degrades to a cache miss; the database stays the source of truth.
- Apply the timeout in the KV backend (get/set/delete) and in the core
read path (getEpoch + cachedQuery value read), so any backend that
stalls self-heals once the bounded read settles.
- Also switch the cache debug-log gate from process.env to
import.meta.env.DEV (repo convention).
Adds regression tests: a never-settling backend resolves via load()
instead of hanging, the namespace self-heals afterward, and the KV
backend rejects a stalled get/set.
* docs: document the object cache
* feat(core): cache per-entry taxonomy reads in the object cache
Public renders that read an entry's terms still hit D1 on every request
even with the object cache on: getEmDashEntry caches the entry (and bakes
in byline/term hydration), but templates that call getEntryTerms /
getTermsForEntries / getTerm directly fell through to D1, because only the
taxonomy *definitions* (getTaxonomyDefs) and full term *lists*
(getTaxonomyTerms) were wrapped. On a warm content-cache hit, hydration
(which used to prime the request cache for getEntryTerms) doesn't run, so
those direct calls query the database — a cache-busted load that should be
served entirely from KV still pays D1 round-trips.
Wrap the per-entry/term taxonomy reads in cachedQuery:
- getEntryTerms, getTermsForEntries — namespaced under
[content:<collection>, taxonomies]; assignments bump taxonomies and
content writes bump content:<collection>, so they invalidate correctly.
getEntryTerms keeps its requestCached wrapper so hydration priming still
short-circuits within a request.
- getTerm — namespaced under taxonomies (count is TTL-bounded).
- getTermsForEntries returns a Map (not JSON-serializable): cache it as an
array of [entryId, terms] pairs and rebuild the Map on read. Large id
batches (which come from collection hydration, already served by the
content cache) bypass the object cache to stay under KV's key-size limit.
getEntriesByTerm already delegates to the cached getEmDashCollection, and
getAllTermsForEntries only runs behind a content-cache miss, so neither
needs separate wrapping.
Test: with a configured backend, getEntryTerms and getTermsForEntries
serve the second read from KV with D1 made unavailable, and the Map
round-trips correctly.
* perf(core): fetch object-cache value and epochs in one parallel round-trip
The read path did two sequential KV round-trips per cached query: read the
namespace epoch(s) to build the key, then read the value. On a cold isolate
(epochs not yet cached in-memory) a page making several cached reads paid
that doubled latency on each one.
Make the value key epoch-independent and store the namespace epochs inside
the value envelope ({ e: epochs, v: value }). A read now fetches the value
and all epochs concurrently (Promise.all) and treats it as a HIT only when
every stored epoch still matches the current one — one round-trip instead of
two. Invalidation is unchanged from the caller's view (bump the epoch; the
next read sees a mismatch and reloads), but a stale value is now overwritten
in place under its stable key rather than orphaned under a dead epoch-keyed
name — so KV no longer accumulates orphaned generations between TTL sweeps.
Note this parallelizes the epoch/value reads *within* each cached query;
ordering across a template's awaits is still the template's concern (use
Promise.all for independent reads).
Existing object-cache, content, taxonomy, and edge-cache tests pass
unchanged (behavior is identical: hit after first load, reload after
invalidation, multi-namespace busting, timeout-to-miss).
* feat(core): cache collection-info and public comments in the object cache
These were the last per-request D1 reads on a public post render. The
<Comments> component server-renders two reads on every page — even with
content/taxonomy reads already served from KV:
- getCollectionInfo (the commentsEnabled / supports / fields lookup), and
- getComments (approved comments), when comments are enabled.
Wrap both in cachedQuery:
- getCollectionInfo → `schema` namespace, busted by invalidateUrlPatternCache
(every schema-mutation path already routes through it, so editing a
collection's settings/fields invalidates it).
- getComments → `comments` namespace, busted by any CommentRepository write
(create / status change / delete), so a new or moderated comment shows
without waiting for TTL.
With this, a warm-isolate logged-out post render makes no D1 query — the
whole render is served from KV.
Tests: getCollectionInfo and getComments serve the second read with D1
unavailable, and reload after a schema change / comment write respectively.
* fix(core): prevent a stale in-flight epoch read from reverting an invalidation
An object-cache epoch read started before `invalidateObjectCache` could
resolve afterwards and unconditionally write the pre-bump backend value over
the freshly-bumped local epoch, resurrecting values the invalidation had just
orphaned (stale content served until the value TTL, default 1h). Epochs are
monotonic, so the resolved read now merges with the current cached epoch via
Math.max and never lowers it.
* fix(core): capture object-cache epochs before load on the read-error path
When the value read errored or timed out, `cachedQuery` left `currentEpochs`
empty and re-read the epochs in the deferred write — *after* `load()`. A write
that invalidated the namespace mid-load would then be picked up by that re-read
and stamp the stale value under the new epoch, so a later read served it as a
HIT. The value and epoch reads now run concurrently but are awaited separately
(getEpoch never rejects), so the pre-load epochs are always captured, even when
the value read fails.
* fix(core): don't cache a scheduled entry that isn't visible yet
A scheduled entry becomes visible on a future clock tick, not on a write, so
an object-cache snapshot taken before its go-live time kept it hidden past that
time — until the publish sweep bumped the epoch or the value TTL (default 1h)
lapsed. getEmDashEntry now marks a resolution time-sensitive when it sees a
scheduled, not-yet-due candidate and skips caching that result.
* fix(core): invalidate the content cache on field schema changes
Creating, updating, or deleting a field changes the collection's columns, but
the field handlers invalidated nothing — cached content snapshots (which embed
field values) kept serving the old shape to anonymous visitors until a content
write or the value TTL. The field create/update/delete handlers now bump the
collection's content cache. Reorder is untouched: field order isn't part of any
cached read.
* fix(core): don't collapse a multi-key object carrying the date tag
The object-cache codec rehydrated any object with a `$$emdashDate` string key
into a Date, so a user value that happened to carry that key alongside others
lost every other field. encode() only ever emits the tag as an object's sole
key, so decode now requires exactly one key before treating it as a tagged Date.
* docs(core): correct the memory backend's expiry-clock comment
The memory backend computes expiry from `Date.now()`, not `performance.now()`.
* fix(core): include reactions and sort in the getComments cache key
getCommentsWithDb branches on `reactions` and `sort` ("best" reorders and
implies reactions), but the object-cache key folded in only collection,
contentId, and threaded. Two reads differing solely in those options collided,
so the second was served the first's snapshot (e.g. a `sort: "best"` render
getting oldest-ordered, reaction-less comments). The key now includes both,
normalizing the best-implies-reactions rule so identical results still share an
entry.
* fix(core): invalidate the comments cache when a reaction is toggled
Reaction counts and `best` ordering are folded into cached getComments reads,
but handleReactionToggle never invalidated — so a toggled reaction left stale
counts (and stale "best" order) until a comment write or the value TTL. The
toggle handler now bumps the comments cache.
* docs(core): correct object-cache bypass and staleness claims
Two claims were inaccurate: `shouldBypass` checks only edit/preview/isolated
mode, not authentication, so authenticated browsing outside edit mode is served
from the cache (safe — it only stores published content); and `revalidate`
bounds only an isolate's local epoch reuse, not total cross-isolate staleness.
On KV the real window is dominated by KV's edge-cache propagation (eventually
consistent, up to ~60s) plus `revalidate`. Updated the runtime JSDoc, the
revalidate type doc, and the deployment guide to say so.
---------
Co-authored-by: Matt Kane <m@mk.gg>
Co-authored-by: Matt Kane <mkane@cloudflare.com>1 parent cf17c9f commit 640e60a
41 files changed
Lines changed: 2786 additions & 204 deletions
File tree
- .changeset
- docs
- src/content/docs
- deployment
- reference
- packages
- cloudflare
- src
- cache
- tests/cache
- core
- src
- api/handlers
- astro
- integration
- object-cache
- comments
- database/repositories
- object-cache
- schema
- settings
- taxonomies
- tests/unit
- taxonomies
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
220 | 220 | | |
221 | 221 | | |
222 | 222 | | |
| 223 | + | |
223 | 224 | | |
224 | 225 | | |
225 | 226 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
96 | 96 | | |
97 | 97 | | |
98 | 98 | | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
99 | 115 | | |
100 | 116 | | |
101 | 117 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
91 | 107 | | |
92 | 108 | | |
93 | 109 | | |
| |||
561 | 577 | | |
562 | 578 | | |
563 | 579 | | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
564 | 613 | | |
565 | 614 | | |
566 | 615 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
75 | 79 | | |
76 | 80 | | |
77 | 81 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
0 commit comments