Skip to content

Commit ec744af

Browse files
thehabesthehabes
committed
Update docs for limit control
1 parent f0d31ba commit ec744af

4 files changed

Lines changed: 269 additions & 31 deletions

File tree

‎cache/docs/ARCHITECTURE.md‎

Lines changed: 59 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@
3838
│ │ (In-Memory) │ │ │
3939
│ │ │ │ │
4040
│ │ Max: 1000 items │ │ │
41+
│ │ Max: 1GB bytes │ │ │
4142
│ │ TTL: 5 minutes │ │ │
4243
│ │ Eviction: LRU │ │ │
4344
│ │ │ │ │
@@ -274,9 +275,10 @@ Client Write Request (CREATE/UPDATE/DELETE)
274275
│ │ Statistics │ │
275276
│ │ │ │
276277
│ │ • hits: 1234 • size: 850/1000 │ │
277-
│ │ • misses: 567 • hitRate: 68.51% │ │
278-
│ │ • evictions: 89 • ttl: 300000ms │ │
279-
│ │ • sets: 1801 • invalidations: 45 │ │
278+
│ │ • misses: 567 • bytes: 22.1MB/1000MB │ │
279+
│ │ • evictions: 89 • hitRate: 68.51% │ │
280+
│ │ • sets: 1801 • ttl: 300000ms │ │
281+
│ │ • invalidations: 45 │ │
280282
│ └──────────────────────────────────────────────────┘ │
281283
└───────────────────────────────────────────────────────────┘
282284
```
@@ -321,11 +323,48 @@ Client Write Request (CREATE/UPDATE/DELETE)
321323
│ │ │ │
322324
│ Expected Hit Rate: 60-80% for read-heavy workloads │
323325
│ Speed Improvement: 60-800x for cached requests │
324-
│ Memory Usage: ~2-10MB (1000 entries @ 2-10KB each) │
326+
│ Memory Usage: ~26MB (1000 typical entries) │
325327
│ Database Load: Reduced by hit rate percentage │
326328
└──────────────────────────────────────────────────────────────┘
327329
```
328330

331+
## Limit Enforcement
332+
333+
The cache enforces both entry count and memory size limits:
334+
335+
```
336+
┌──────────────────────────────────────────────────────────────┐
337+
│ Cache Limits (Dual) │
338+
├──────────────────────────────────────────────────────────────┤
339+
│ │
340+
│ Limit Type │ Default │ Purpose │
341+
│─────────────────┼─────────────┼──────────────────────────────│
342+
│ Length (count) │ 1000 │ Ensures cache diversity │
343+
│ │ │ Prevents cache thrashing │
344+
│ │ │ PRIMARY working limit │
345+
│ │ │
346+
│ Bytes (size) │ 1GB │ Prevents memory exhaustion │
347+
│ │ │ Safety net for edge cases │
348+
│ │ │ Guards against huge objects │
349+
│ │
350+
│ Balance: With typical RERUM queries (100 items/page), │
351+
│ 1000 entries = ~26 MB (2.7% of 1GB limit) │
352+
│ │
353+
│ Typical entry sizes: │
354+
│ • ID lookup: ~183 bytes │
355+
│ • Query (10 items): ~2.7 KB │
356+
│ • Query (100 items): ~27 KB │
357+
│ • GOG (50 items): ~13.5 KB │
358+
│ │
359+
│ The length limit (1000) will be reached first in normal │
360+
│ operation. The byte limit provides protection against │
361+
│ accidentally caching very large result sets. │
362+
│ │
363+
│ Eviction: When either limit is exceeded, LRU entries │
364+
│ are removed until both limits are satisfied │
365+
└──────────────────────────────────────────────────────────────┘
366+
```
367+
329368
## Invalidation Patterns
330369

331370
```
@@ -363,17 +402,22 @@ Client Write Request (CREATE/UPDATE/DELETE)
363402
## Configuration and Tuning
364403

365404
```
366-
┌──────────────────────────────────────────────────────────┐
367-
│ Environment-Specific Settings │
368-
├──────────────────────────────────────────────────────────┤
369-
│ │
370-
│ Environment │ CACHE_MAX_SIZE │ CACHE_TTL │
371-
│────────────────┼──────────────────┼─────────────────────│
372-
│ Development │ 500 │ 300000 (5 min) │
373-
│ Staging │ 1000 │ 300000 (5 min) │
374-
│ Production │ 2000-5000 │ 600000 (10 min) │
375-
│ High Traffic │ 5000+ │ 300000 (5 min) │
376-
└──────────────────────────────────────────────────────────┘
405+
┌──────────────────────────────────────────────────────────────────────┐
406+
│ Environment-Specific Settings │
407+
├──────────────────────────────────────────────────────────────────────┤
408+
│ │
409+
│ Environment │ MAX_LENGTH │ MAX_BYTES │ TTL │
410+
│───────────────┼────────────┼───────────┼─────────────────────────────│
411+
│ Development │ 500 │ 500MB │ 300000 (5 min) │
412+
│ Staging │ 1000 │ 1GB │ 300000 (5 min) │
413+
│ Production │ 1000 │ 1GB │ 600000 (10 min) │
414+
│ High Traffic │ 2000 │ 2GB │ 300000 (5 min) │
415+
│ │
416+
│ Recommendation: Keep defaults (1000 entries, 1GB) unless: │
417+
│ • Abundant memory available → Increase MAX_BYTES for safety │
418+
│ • Low cache hit rate → Increase MAX_LENGTH for diversity │
419+
│ • Memory constrained → Decrease both limits proportionally │
420+
└──────────────────────────────────────────────────────────────────────┘
377421
```
378422

379423
---

‎cache/docs/DETAILED.md‎

Lines changed: 46 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,17 +7,60 @@ The RERUM API implements an LRU (Least Recently Used) cache with smart invalidat
77
## Cache Configuration
88

99
### Default Settings
10-
- **Max Size**: 1000 entries
10+
- **Max Length**: 1000 entries
11+
- **Max Bytes**: 1GB (1,000,000,000 bytes)
1112
- **TTL (Time-To-Live)**: 5 minutes (300,000ms)
1213
- **Eviction Policy**: LRU (Least Recently Used)
1314
- **Storage**: In-memory (per server instance)
1415

1516
### Environment Variables
1617
```bash
17-
CACHE_MAX_SIZE=1000 # Maximum number of cached entries
18-
CACHE_TTL=300000 # Time-to-live in milliseconds
18+
CACHE_MAX_LENGTH=1000 # Maximum number of cached entries
19+
CACHE_MAX_BYTES=1000000000 # Maximum memory usage in bytes
20+
CACHE_TTL=300000 # Time-to-live in milliseconds
1921
```
2022

23+
### Limit Enforcement Details
24+
25+
The cache implements **dual limits** for defense-in-depth:
26+
27+
1. **Length Limit (1000 entries)**
28+
- Primary working limit
29+
- Ensures diverse cache coverage
30+
- Prevents cache thrashing from too many unique queries
31+
- Reached first under normal operation
32+
33+
2. **Byte Limit (1GB)**
34+
- Secondary safety limit
35+
- Prevents memory exhaustion
36+
- Protects against accidentally large result sets
37+
- Guards against malicious queries
38+
39+
**Balance Analysis**: With typical RERUM queries (100 items per page at ~269 bytes per annotation):
40+
- 1000 entries = ~26 MB (2.7% of 1GB limit)
41+
- Length limit reached first in 99%+ of scenarios
42+
- Byte limit only activates for edge cases (e.g., entries > 1MB each)
43+
44+
**Eviction Behavior**:
45+
- When length limit exceeded: Remove least recently used entry
46+
- When byte limit exceeded: Remove LRU entries until under limit
47+
- Both limits checked on every cache write operation
48+
49+
**Byte Size Calculation**:
50+
```javascript
51+
// Accurately calculates total cache memory usage
52+
calculateByteSize() {
53+
let totalBytes = 0
54+
for (const [key, node] of this.cache.entries()) {
55+
totalBytes += Buffer.byteLength(key, 'utf8')
56+
totalBytes += Buffer.byteLength(JSON.stringify(node.value), 'utf8')
57+
}
58+
return totalBytes
59+
}
60+
```
61+
62+
This ensures the byte limit is properly enforced (fixed in PR #225).
63+
2164
## Cached Endpoints
2265

2366
### 1. Query Endpoint (`POST /v1/api/query`)

‎cache/docs/SHORT.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,9 +92,12 @@ Immediately clears all cached entries (useful for testing or troubleshooting).
9292
## Configuration
9393

9494
Cache behavior can be adjusted via environment variables:
95-
- `CACHE_MAX_SIZE` - Maximum entries (default: 1000)
95+
- `CACHE_MAX_LENGTH` - Maximum entries (default: 1000)
96+
- `CACHE_MAX_BYTES` - Maximum memory usage (default: 1GB)
9697
- `CACHE_TTL` - Time-to-live in milliseconds (default: 300000 = 5 minutes)
9798

99+
**Note**: Limits are well-balanced for typical usage. With standard RERUM queries (100 items per page), 1000 cached entries use only ~26 MB (~2.7% of the 1GB byte limit). The byte limit serves as a safety net for edge cases.
100+
98101
## Backwards Compatibility
99102

100103
✅ **Fully backwards compatible**

‎cache/docs/TESTS.md‎

Lines changed: 160 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,25 +2,37 @@
22

33
## Overview
44

5-
The `cache.test.js` file provides comprehensive **unit tests** for the RERUM API caching layer, verifying that all read endpoints have functioning cache middleware.
5+
The cache testing suite includes two test files that provide comprehensive coverage of the RERUM API caching layer:
6+
7+
1. **`cache.test.js`** - Middleware functionality tests (48 tests)
8+
2. **`cache-limits.test.js`** - Limit enforcement tests (12 tests)
69

710
## Test Execution
811

9-
### Run Cache Tests
12+
### Run All Cache Tests
1013
```bash
11-
npm run runtest -- cache/cache.test.js
14+
npm run runtest -- cache/__tests__/
15+
```
16+
17+
### Run Individual Test Files
18+
```bash
19+
# Middleware tests
20+
npm run runtest -- cache/__tests__/cache.test.js
21+
22+
# Limit enforcement tests
23+
npm run runtest -- cache/__tests__/cache-limits.test.js
1224
```
1325

1426
### Expected Results
1527
```
16-
✅ Test Suites: 1 passed, 1 total
17-
✅ Tests: 36 passed, 36 total
18-
⚡ Time: ~0.33s
28+
✅ Test Suites: 2 passed, 2 total
29+
✅ Tests: 60 passed, 60 total
30+
⚡ Time: ~1.2s
1931
```
2032

2133
---
2234

23-
## What cache.test.js DOES Test
35+
## cache.test.js - Middleware Functionality (48 tests)
2436

2537
### ✅ Read Endpoint Caching (30 tests)
2638

@@ -411,10 +423,145 @@ Tests verify cache statistics are accurately tracked:
411423
- ⚠️ Cache invalidation on write operations
412424
- ⚠️ Actual MongoDB interactions
413425
- ⚠️ TTL expiration (requires time-based testing)
414-
- ⚠️ Cache eviction under max size limit
415426
- ⚠️ Concurrent request handling
416427
- ⚠️ Memory pressure scenarios
417428

429+
---
430+
431+
## cache-limits.test.js - Limit Enforcement (12 tests)
432+
433+
### What This Tests
434+
435+
Comprehensive validation of cache limit enforcement to ensure memory safety and proper eviction behavior.
436+
437+
### ✅ Length Limit Tests (3 tests)
438+
439+
#### 1. Max Length Enforcement
440+
- ✅ Cache never exceeds maxLength when adding entries
441+
- ✅ Automatically evicts least recently used (LRU) entries at limit
442+
- ✅ Eviction counter accurately tracked
443+
444+
#### 2. LRU Eviction Order
445+
- ✅ Least recently used entries evicted first
446+
- ✅ Recently accessed entries preserved
447+
- ✅ Proper head/tail management in linked list
448+
449+
#### 3. LRU Order Preservation
450+
- ✅ Accessing entries moves them to head (most recent)
451+
- ✅ Unaccessed entries move toward tail (least recent)
452+
- ✅ Eviction targets correct (tail) entry
453+
454+
### ✅ Byte Size Limit Tests (3 tests)
455+
456+
#### 1. Max Bytes Enforcement
457+
- ✅ Cache never exceeds maxBytes when adding entries
458+
- ✅ Byte size calculated accurately using `calculateByteSize()`
459+
- ✅ Multiple evictions triggered if necessary
460+
461+
**Critical Fix Verified**: Previously, byte limit was NOT enforced due to `JSON.stringify(Map)` bug. Tests confirm the fix works correctly.
462+
463+
#### 2. Multiple Entry Eviction
464+
- ✅ Evicts multiple entries to stay under byte limit
465+
- ✅ Continues eviction until bytes < maxBytes
466+
- ✅ Handles large entries requiring multiple LRU removals
467+
468+
#### 3. Realistic Entry Sizes
469+
- ✅ Handles typical RERUM query results (~27KB for 100 items)
470+
- ✅ Properly calculates byte size for complex objects
471+
- ✅ Byte limit enforced with production-like data
472+
473+
### ✅ Combined Limits Tests (2 tests)
474+
475+
#### 1. Dual Limit Enforcement
476+
- ✅ Both length and byte limits enforced simultaneously
477+
- ✅ Neither limit can be exceeded
478+
- ✅ Proper interaction between both limits
479+
480+
#### 2. Limit Prioritization
481+
- ✅ Byte limit takes precedence when entries are large
482+
- ✅ Length limit takes precedence for typical entries
483+
- ✅ Defense-in-depth protection verified
484+
485+
### ✅ Edge Cases (3 tests)
486+
487+
#### 1. Updating Existing Entries
488+
- ✅ Updates don't trigger unnecessary evictions
489+
- ✅ Cache size remains constant on updates
490+
- ✅ Entry values properly replaced
491+
492+
#### 2. Large Single Entries
493+
- ✅ Single large entry can be cached if within limits
494+
- ✅ Proper handling of entries near byte limit
495+
- ✅ No infinite eviction loops
496+
497+
#### 3. Empty Cache
498+
- ✅ Statistics accurate with empty cache
499+
- ✅ Limits properly reported
500+
- ✅ No errors accessing empty cache
501+
502+
### ✅ Real-World Simulation (1 test)
503+
504+
#### Production-Like Usage Patterns
505+
- ✅ 2000 cache operations with realistic RERUM data
506+
- ✅ Proper handling of pagination (creates duplicate keys with updates)
507+
- ✅ Statistics accurately tracked across many operations
508+
- ✅ Verifies limits are well-balanced for typical usage
509+
510+
**Key Finding**: With default limits (1000 entries, 1GB), typical RERUM queries (100 items) only use ~26 MB (2.7% of byte limit). Length limit is reached first in normal operation.
511+
512+
### Test Implementation Details
513+
514+
```javascript
515+
// Helper functions for testing with custom limits
516+
function setupTestCache(maxLength, maxBytes, ttl) {
517+
cache.clear()
518+
cache.maxLength = maxLength
519+
cache.maxBytes = maxBytes
520+
cache.ttl = ttl
521+
// Reset stats
522+
return cache
523+
}
524+
525+
function restoreDefaultCache() {
526+
cache.clear()
527+
cache.maxLength = parseInt(process.env.CACHE_MAX_LENGTH ?? 1000)
528+
cache.maxBytes = parseInt(process.env.CACHE_MAX_BYTES ?? 1000000000)
529+
cache.ttl = parseInt(process.env.CACHE_TTL ?? 300000)
530+
}
531+
```
532+
533+
### Byte Size Calculation Verification
534+
535+
Tests verify the fix for the critical bug where `JSON.stringify(Map)` returned `{}`:
536+
537+
```javascript
538+
// Before (broken): JSON.stringify(this.cache) → "{}" → 2 bytes
539+
// After (fixed): Proper iteration through Map entries
540+
calculateByteSize() {
541+
let totalBytes = 0
542+
for (const [key, node] of this.cache.entries()) {
543+
totalBytes += Buffer.byteLength(key, 'utf8')
544+
totalBytes += Buffer.byteLength(JSON.stringify(node.value), 'utf8')
545+
}
546+
return totalBytes
547+
}
548+
```
549+
550+
### Limit Balance Findings
551+
552+
| Entry Type | Entries for 1000 Limit | Bytes Used | % of 1GB |
553+
|-----------|------------------------|------------|----------|
554+
| ID lookups | 1000 | 0.17 MB | 0.02% |
555+
| Query (10 items) | 1000 | 2.61 MB | 0.27% |
556+
| Query (100 items) | 1000 | 25.7 MB | 2.70% |
557+
| GOG (50 items) | 1000 | 12.9 MB | 1.35% |
558+
559+
**Conclusion**: Limits are well-balanced. Length limit (1000) will be reached first in 99%+ of scenarios. Byte limit (1GB) serves as safety net for edge cases.
560+
561+
---
562+
563+
## What Tests Do NOT Cover
564+
418565
## Extending the Tests
419566
420567
### Adding Tests for New Endpoints
@@ -516,7 +663,8 @@ Before merging cache changes:
516663
517664
---
518665
519-
**Test Suite**: cache.test.js
520-
**Tests**: 25
521-
**Status**: ✅ All Passing
522-
**Last Updated**: October 20, 2025
666+
**Test Coverage Summary**:
667+
- **cache.test.js**: 48 tests covering middleware functionality
668+
- **cache-limits.test.js**: 12 tests covering limit enforcement
669+
- **Total**: 60 tests, all passing ✅
670+
- **Last Updated**: October 21, 2025

0 commit comments

Comments
 (0)