[WIP] perf: cache FST term-offset lookups in Dictionary.postingsList by capemox · Pull Request #435 · blevesearch/zapx

capemox · 2026-07-13T14:17:27Z

Cache FST term-offset lookups so repeated queries on hot terms skip re-traversing the FST. No functional/API changes; benchmark included.

Changes

Cache FST-resolved term offset — Dictionary.postingsList() resolves each term through the FST on every call, even though the offset never changes for a given (segment, field, term). Cache it in a per-field write-once/read-many sync.Map on the segment's inverted-index cache entry; warm lookups jump straight to postingsListFromOffset. BenchmarkPostingsListLookup (fully-warm): 243ns → 75ns (-69%), allocations unchanged.
Cache FST not-found results — extend the same cache to record misses via a termNotFoundSentinel, so high-selectivity / rare-entity disjunction queries stop re-traversing the FST for absent terms.

Real-world gain scales with term repetition across queries (the first lookup per term still pays the traversal). Marked draft pending review.

Dictionary.postingsList() resolves each term through the FST on every call. For repeated queries on the same hot terms this traversal is pure overhead: the resolved posting-list offset never changes for a given (segment, field, term). Cache the offset in a per-field termOffsetCache (sync.Map, write-once / read-many) on the segment's inverted-index cache entry. The first lookup for a term traverses the FST and stores the offset; subsequent lookups load it and jump straight to postingsListFromOffset. Benchmark (BenchmarkPostingsListLookup, added here): 5000-term segment, 200 hot terms queried round-robin (fully-warm cache), Apple M4 Pro, count=10 via benchstat: postingsList / term 243ns -> 75ns -69% (p=0.000) B/op, allocs/op unchanged (24 B, 1 alloc) This is the fully-warm ceiling: the first lookup per term still pays the FST traversal, so real-world gain scales with term repetition across queries.

Extend the term-offset cache to also record misses. High-selectivity / rare-entity disjunction queries probe many (segment, field) pairs where the term does not exist; previously postingsList() returned immediately without caching, so every query re-traversed the FST for each miss. Store termNotFoundSentinel (^uint64(0), never a valid byte offset) in termOffsetCache on the first !exists result. Subsequent queries for the same absent term in the same segment return emptyPostingsList without touching the FST reader. Same FST-skip mechanism as the offset cache in the previous commit (measured there at -69% per warm lookup), applied to the miss path.

Copilot

Pull request overview

This PR adds a per-segment/per-field cache of FST term→postings-offset lookups inside Dictionary.postingsList(), aiming to speed up repeated queries for the same terms by bypassing repeated FST traversals. It also adds a benchmark and a unit test to validate/measure the caching behavior.

Changes:

Add a per-field termOffsetCache (via sync.Map) to cache postings offsets and a not-found sentinel.
Add a fast-path in Dictionary.postingsList() to use the cached offset/sentinel before calling into the vellum FST.
Add BenchmarkPostingsListLookup and TestTermOffsetCacheWarmedByPostingsList.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.

File	Description
inverted_text_cache.go	Adds `getOrCreateEntry()` and `termOffsetCache` storage on the inverted-index cache entry.
dict.go	Implements term-offset caching + not-found sentinel fast-path in `postingsList()`.
fst_cache_bench_test.go	Adds benchmark to measure warmed postings-list lookup performance.
dict_test.go	Adds unit test verifying cache warming and not-found sentinel behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+	// termOffsetCache maps term → posting-list offset within the segment
+	// file, avoiding repeated FST traversals for repeated queries on the
+	// same term.  Uses sync.Map (write-once, read-many pattern).
+	// Only populated for terms that are found in the segment.
+	termOffsetCache sync.Map // key: string, val: uint64


+		ce = d.sb.invIndexCache.getOrCreateEntry(d.fieldID)
+		if v, ok := ce.termOffsetCache.Load(string(term)); ok {
+			offset := v.(uint64)


+	// Fast path: avoid FST traversal for terms seen in a previous query.
+	// The cache stores both found offsets and the termNotFoundSentinel so
+	// that rare-entity queries (many segment×field misses) skip the FST
+	// after the first traversal.


+	if err != nil {
+		t.Fatal(err)
+	}
+	defer seg.Close()


+	// Absent terms must be stored with the sentinel so subsequent calls
+	// skip the FST traversal (critical for rare-entity queries with many
+	// segment×field misses).
+	_, _ = d.PostingsList([]byte("notinindex"), nil, nil)


+// BenchmarkPostingsListLookup measures repeated postingsList() calls for a
+// working set of terms. §19 caches the FST-resolved offset per term, so after
+// the first lookup each call skips the FST traversal.


+// TestTermOffsetCacheWarmedByPostingsList verifies the §19 FST hot-term
+// offset cache: PostingsList() stores the FST postings offset in termOffsetCache
+// so that subsequent PostingsList calls for the same term bypass the FST
+// traversal entirely.


Thejas-bhat · 2026-07-16T18:40:57Z

+	// same term.  Uses sync.Map (write-once, read-many pattern).
+	// Populated for both found terms (the resolved offset) and absent terms
+	// (termNotFoundSentinel), so repeated misses also skip the FST.
+	termOffsetCache sync.Map // key: string, val: uint64


wouldn't this cause memory issues though? in a real world scenario where the segments are big containing large FSTs, this map could explode in size when most of the entries in the FSTs are queried.
since there's no prefix sharing and this is purely present in the memory, storage will be amplified along with the extra copy of the term in FST. i don't think this field would be compatible with mmap either, since it doesn't demand load the specific term from the mmap'd slice like vellum.

also, another question, when you performed the benchmarking what was the dataset used and how many documents where there in the biggest segment?

I was using a relatively small benchmark so nothing problematic showed up. You're right, this is a problem. Anyways I'm going to pull the fuzzy stuff into a separate PR, I'll check it again.

- Fix the termOffsetCache doc comment: it caches both found offsets and the not-found sentinel for absent terms, not only found terms. - Check the Close() error in TestTermOffsetCacheWarmedByPostingsList (matching the other tests in this file) and fail fast on the absent-term PostingsList call instead of discarding its error. - Drop the in-house "§19" section reference from the test/benchmark comments in favour of self-contained descriptions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

CascadingRadium · 2026-07-17T16:23:11Z

+	// same term.  Uses sync.Map (write-once, read-many pattern).
+	// Populated for both found terms (the resolved offset) and absent terms
+	// (termNotFoundSentinel), so repeated misses also skip the FST.
+	termOffsetCache sync.Map // key: string, val: uint64


also i dont see any eviction mechanism here?

…ion (#436) Split out from #435 per review — the automaton (fuzzy/regexp) candidate-collection optimization, as its own PR. No functional/API changes. ### Change `DictionaryIterator.Next()` unconditionally deserialized each visited term's postings list (roaring `FromBuffer` + `GetCardinality`) to populate `DictEntry.Count`. Candidate-term collectors in bleve (fuzzy/regexp/wildcard) discard that count, so it was O(candidates) wasted bitmap deserializations per query. The `omitCount` field existed to suppress this but was never wired up. New `AutomatonIteratorOmitCount` returns an iterator that leaves `Count` at zero and skips the postings read; the existing `AutomatonIterator` is unchanged (both share a helper). Consumed opt-in by bleve's fuzzy/regexp field-dict constructors (blevesearch/bleve#2382). ### Benchmark `BenchmarkFuzzyIterator` (1000-term / 3000-doc segment, fuzziness-2 automaton matching 280 multi-hit candidates), Apple M4 Pro, count=10 via benchstat, with-count vs omit-count: | sec/op | B/op | allocs/op | |---|---|---| | **−35.5%** | **−57.4%** | **−47.7%** (595→311) | All p=0.000, n=10. `TestAutomatonIteratorOmitCountSemantics` (added) asserts the omit iterator yields identical terms/order/edit-distances — only the discarded count differs. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Thejas-bhat <thejas.orkombu@couchbase.com>

steveyen added 2 commits July 10, 2026 11:30

capemox changed the title ~~perf: cache FST term-offset lookups in Dictionary.postingsList~~ [WIP] perf: cache FST term-offset lookups in Dictionary.postingsList Jul 16, 2026

capemox marked this pull request as ready for review July 16, 2026 06:01

capemox requested review from CascadingRadium, Likith101, Samsonnyyeet, Thejas-bhat, Copilot, maneuvertomars and steveyen and removed request for Copilot July 16, 2026 06:06

Copilot started reviewing on behalf of capemox July 16, 2026 06:06 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

capemox mentioned this pull request Jul 16, 2026

MB-72489: standalone query hot-path optimizations blevesearch/bleve#2381

Merged

Thejas-bhat reviewed Jul 16, 2026

View reviewed changes

capemox force-pushed the perf-standalone-opts branch from 8a24637 to 6034837 Compare July 17, 2026 13:45

capemox mentioned this pull request Jul 17, 2026

MB-72489: omit postings-count read during automaton candidate collection #436

Merged

CascadingRadium reviewed Jul 17, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP] perf: cache FST term-offset lookups in Dictionary.postingsList#435

[WIP] perf: cache FST term-offset lookups in Dictionary.postingsList#435
capemox wants to merge 3 commits into
masterfrom
perf-standalone-opts

capemox commented Jul 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Thejas-bhat Jul 16, 2026

Uh oh!

capemox Jul 17, 2026

Uh oh!

CascadingRadium Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

capemox commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Thejas-bhat Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

capemox Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

CascadingRadium Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

capemox commented Jul 13, 2026 •

edited

Loading