Skip to content

feat: add LRU eviction with entry limit to SchemaStore to prevent unbounded memory growth (fixes #604)#724

Merged
tnaum-ms merged 3 commits into
microsoft:mainfrom
hanhan761:fix-604-schema-store-lru
Jun 22, 2026
Merged

feat: add LRU eviction with entry limit to SchemaStore to prevent unbounded memory growth (fixes #604)#724
tnaum-ms merged 3 commits into
microsoft:mainfrom
hanhan761:fix-604-schema-store-lru

Conversation

@hanhan761

Copy link
Copy Markdown
Contributor

Summary

SchemaStore._analyzers Map grew unbounded — no eviction strategy existed. In long-running sessions with many collections, memory usage increased without limit.

Changes

  • LRU eviction: Added _accessOrder tracking (last-access timestamps) and _evictIfNeeded() that removes the least-recently-used entry when the cache exceeds the limit
  • Default limit: 100 entries (DEFAULT_MAX_ENTRIES)
  • Access tracking: All read methods (hasSchema, getKnownFields, getSchema, getDocumentCount) and writes (addDocuments) update access timestamps
  • Eviction telemetry: evictionCount and maxEntries measurements in schemaStore.stats event for monitoring
  • Cleanup: _accessOrder cleared in clearSchema, clearCluster, clearDatabase, reset, and dispose

Design

  • LRU policy (access-order-based) is simpler and more predictable than time-based eviction for a cache that receives bursty access patterns
  • Single _evictIfNeeded() call path in addDocuments() (only when a new entry is created) — no periodic timer needed
  • Zero behavior change for sessions with fewer than 100 collections

Issue

Fixes #604

Verification

  • Single-file change (src/documentdb/SchemaStore.ts, +81/-7)
  • Existing read/write APIs unchanged (same signatures, same return values)
  • _accessOrder cleaned up in all existing lifecycle methods

Copilot AI review requested due to automatic review settings June 2, 2026 14:37
@hanhan761 hanhan761 requested a review from a team as a code owner June 2, 2026 14:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds an LRU-based soft cache limit to SchemaStore to prevent unbounded schema analyzer growth during long-running VS Code sessions, and reports eviction metrics via telemetry.

Changes:

  • Introduces LRU tracking (_accessOrder) with a default max entry limit and eviction counter.
  • Touches cache entries on reads/writes and evicts least-recently-used entries when above the limit.
  • Publishes new telemetry measurements (evictionCount, maxEntries).

Comment thread src/documentdb/SchemaStore.ts
Comment thread src/documentdb/SchemaStore.ts
Comment thread src/documentdb/SchemaStore.ts
- Replace Date.now() timestamp scan with Map insertion order for O(1) LRU
- Evict in a while loop until size <= maxEntries (not just one entry)
- Add fallback when _accessOrder is out of sync with _analyzers
@tnaum-ms tnaum-ms added the in-triage-queue We've seen your input and will triage it label Jun 4, 2026
- Lower DEFAULT_MAX_ENTRIES from 100 to 50 collections
- Flag a stats change on eviction so eviction count flushes to telemetry
  promptly (schemaStore.stats already reports evictionCount, maxEntries,
  and currentCollectionCount/maxCollectionCount for cache size)
- Add unit tests covering the cap, LRU eviction order, read-as-recent-use,
  re-adding evicted entries, and telemetry reporting
@tnaum-ms tnaum-ms removed the in-triage-queue We've seen your input and will triage it label Jun 22, 2026
@tnaum-ms

Copy link
Copy Markdown
Collaborator

Maintainer update

I pushed a few changes to this branch to get it merge-ready (commit on top of your work):

Changes

  • Lowered the cache cap from 100 to 50 collections (DEFAULT_MAX_ENTRIES = 50).
  • Made eviction telemetry reliable: an eviction now flags a stats change so the cumulative evictionCount is flushed promptly in the schemaStore.stats event. Cache size is reported via currentCollectionCount / maxCollectionCount, alongside evictionCount and maxEntries, so we can monitor both the live map size and how often eviction is triggered.
  • Added unit tests covering: the 50-entry cap, least-recently-used eviction order, read-as-recent-use (reads refresh recency), re-adding an evicted collection, and telemetry reporting.

Validation: prettier, lint, build, and the full Jest suite (2000 tests) all pass locally.

Thanks again for this, @hanhan761. The approach (insertion-order LRU with a single eviction path in addDocuments) is clean and well-scoped.

@tnaum-ms tnaum-ms added this to the 0.9.1 milestone Jun 22, 2026
@tnaum-ms tnaum-ms merged commit f4e060e into microsoft:main Jun 22, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SchemaStore: Add memory ceiling with LRU eviction for _analyzers Map

4 participants