changes from testing, and new reports

Author: claude

cache/__tests__/cache-metrics-worst-case.sh

@@ -1,14 +1,14 @@
 # RERUM Cache Metrics & Functionality Report
 
-**Generated**: Tue Nov  4 22:07:39 CST 2025  
+**Generated**: Wed Nov  5 12:44:10 CST 2025  
 **Test Duration**: Full integration and performance suite  
 **Server**: http://localhost:3001
 
 ---
 
 ## Executive Summary
 
-**Overall Test Results**: 45 passed, 0 failed, 0 skipped (45 total)
+**Overall Test Results**: 42 passed, 4 failed, 0 skipped (46 total)
 
 ### Cache Performance Summary
 
@@ -17,7 +17,7 @@
 | Cache Hits | 6 |
 | Cache Misses | 1006 |
 | Hit Rate | 0.59% |
-| Cache Size | 7 entries |
+| Cache Size | 5 entries |
 
 ---
 
@@ -47,12 +47,12 @@
 
 | Endpoint | Cold Cache (DB) | Warm Cache (Memory) | Speedup | Benefit |
 |----------|-----------------|---------------------|---------|---------|
-| `/query` | 352ms | 23ms | -329ms | ✅ High |
-| `/search` | 336ms | 19ms | -317ms | ✅ High |
-| `/searchPhrase` | 294ms | 21ms | -273ms | ✅ High |
-| `/id` | 449 | N/A | N/A | N/A |
-| `/history` | 782 | N/A | N/A | N/A |
-| `/since` | 776 | N/A | N/A | N/A |
+| `/query` | 332ms | 22ms | -310ms | ✅ High |
+| `/search` | 61ms | 20ms | -41ms | ✅ High |
+| `/searchPhrase` | 54ms | 20ms | -34ms | ✅ High |
+| `/id` | 438 | N/A | N/A | N/A |
+| `/history` | 767 | N/A | N/A | N/A |
+| `/since` | 769 | N/A | N/A | N/A |
 
 **Interpretation**:
 - **Cold Cache**: First request hits database (cache miss)
@@ -68,13 +68,13 @@
 
 | Endpoint | Empty Cache | Full Cache (1000 entries) | Overhead | Impact |
 |----------|-------------|---------------------------|----------|--------|
-| `/create` | 53ms | 52ms | -1ms | ✅ None |
-| `/update` | 503ms | 508ms | +5ms | ✅ Negligible |
-| `/patch` | 510ms | 520ms | +10ms | ✅ Low |
-| `/set` | 497ms | 514ms | +17ms | ⚠️  Moderate |
-| `/unset` | 495ms | 512ms | +17ms | ⚠️  Moderate |
-| `/delete` | 508ms | 497ms | -11ms | ✅ None |
-| `/overwrite` | 498ms | 503ms | +5ms | ✅ Negligible |
+| `/create` | 53ms | 50ms | -3ms | ✅ None |
+| `/update` | 498ms | 510ms | +12ms | ⚠️  Moderate |
+| `/patch` | 509ms | 542ms | +33ms | ⚠️  Moderate |
+| `/set` | 495ms | 504ms | +9ms | ✅ Low |
+| `/unset` | 512ms | 511ms | -1ms | ✅ None |
+| `/delete` | 493ms | 469ms | -24ms | ✅ None |
+| `/overwrite` | 513ms | 522ms | +9ms | ✅ Low |
 
 **Interpretation**:
 - **Empty Cache**: Write with no cache to invalidate
@@ -91,14 +91,14 @@
 ### Overall Performance Impact
 
 **Cache Benefits (Reads)**:
-- Average speedup per cached read: ~329ms
+- Average speedup per cached read: ~310ms
 - Typical hit rate in production: 60-80%
-- Net benefit on 1000 reads: ~230300ms saved (assuming 70% hit rate)
+- Net benefit on 1000 reads: ~217000ms saved (assuming 70% hit rate)
 
 **Cache Costs (Writes)**:
-- Average overhead per write: ~6ms
+- Average overhead per write: ~5ms
 - Overhead percentage: ~1%
-- Net cost on 1000 writes: ~6000ms
+- Net cost on 1000 writes: ~5000ms
 - Tested endpoints: create, update, patch, set, unset, delete, overwrite
 
 **Break-Even Analysis**:
@@ -110,17 +110,17 @@ For a workload with:
 
 ```
 Without Cache:
-  800 reads × 352ms = 281600ms
+  800 reads × 332ms = 265600ms
   200 writes × 53ms = 10600ms
-  Total: 292200ms
+  Total: 276200ms
 
 With Cache:
-  560 cached reads × 23ms = 12880ms
-  240 uncached reads × 352ms = 84480ms
-  200 writes × 52ms = 10400ms
-  Total: 107760ms
+  560 cached reads × 22ms = 12320ms
+  240 uncached reads × 332ms = 79680ms
+  200 writes × 50ms = 10000ms
+  Total: 102000ms
 
-Net Improvement: 184440ms faster (~64% improvement)
+Net Improvement: 174200ms faster (~64% improvement)
 ```
 
 ---
@@ -130,9 +130,9 @@ Net Improvement: 184440ms faster (~64% improvement)
 ### ✅ Deploy Cache Layer
 
 The cache layer provides:
-1. **Significant read performance improvements** (329ms average speedup)
-2. **Minimal write overhead** (6ms average, ~1% of write time)
-3. **All endpoints functioning correctly** (45 passed tests)
+1. **Significant read performance improvements** (310ms average speedup)
+2. **Minimal write overhead** (5ms average, ~1% of write time)
+3. **All endpoints functioning correctly** (42 passed tests)
 
 ### 📊 Monitoring Recommendations
 
@@ -163,7 +163,7 @@ Consider tuning based on:
 - Server: http://localhost:3001
 - Test Framework: Bash + curl
 - Metrics Collection: Millisecond-precision timing
-- Test Objects Created: 201
+- Test Objects Created: 202
 - All test objects cleaned up: ✅
 
 **Test Coverage**:
@@ -175,6 +175,6 @@ Consider tuning based on:
 
 ---
 
-**Report Generated**: Tue Nov  4 22:07:39 CST 2025  
+**Report Generated**: Wed Nov  5 12:44:11 CST 2025  
 **Format Version**: 1.0  
 **Test Suite**: cache-metrics.sh

cache/__tests__/cache-metrics.sh

@@ -1,6 +1,6 @@
 # RERUM Cache WORST-CASE Overhead Analysis
 
-**Generated**: Tue Nov  4 21:49:52 CST 2025
+**Generated**: Wed Nov  5 13:00:30 CST 2025
 **Test Type**: Worst-case cache overhead measurement (O(n) scanning, 0 invalidations)
 **Server**: http://localhost:3001
 
@@ -58,12 +58,12 @@
 
 | Endpoint | Empty Cache (0 entries) | Full Cache (1000 entries) | Difference | Analysis |
 |----------|-------------------------|---------------------------|------------|----------|
-| `/query` | 364ms | 367ms | 3ms | ✅ No overhead (O(1) verified) |
-| `/search` | 58ms | 53ms | -5ms | ✅ No overhead (O(1) verified) |
-| `/searchPhrase` | 55ms | 52ms | -3ms | ✅ No overhead (O(1) verified) |
-| `/id` | 453ms | 442ms | -11ms | ✅ Faster (DB variance, not cache) |
-| `/history` | 781ms | 780ms | -1ms | ✅ No overhead (O(1) verified) |
-| `/since` | 764ms | 775ms | 11ms | ⚠️ Slower (likely DB variance) |
+| `/query` | 362ms | 361ms | -1ms | ✅ No overhead (O(1) verified) |
+| `/search` | 62ms | 54ms | -8ms | ✅ Faster (DB variance, not cache) |
+| `/searchPhrase` | 57ms | 51ms | -6ms | ✅ Faster (DB variance, not cache) |
+| `/id` | 442ms | 422ms | -20ms | ✅ Faster (DB variance, not cache) |
+| `/history` | 754ms | 768ms | 14ms | ⚠️ Slower (likely DB variance) |
+| `/since` | 763ms | 753ms | -10ms | ✅ Faster (DB variance, not cache) |
 
 **Key Insight**: Cache uses **O(1) hash-based lookups** for reads.
 
@@ -81,13 +81,13 @@
 
 | Endpoint | Empty Cache | Full Cache (1000 entries) | Overhead | Impact |
 |----------|-------------|---------------------------|----------|--------|
-| `/create` | 54ms | 51ms | -3ms | ✅ None |
-| `/update` | 494ms | 523ms | +29ms | ⚠️  Moderate |
-| `/patch` | 506ms | 525ms | +19ms | ⚠️  Moderate |
-| `/set` | 496ms | 549ms | +53ms | ⚠️  Moderate |
-| `/unset` | 502ms | 525ms | +23ms | ⚠️  Moderate |
-| `/delete` | 493ms | 469ms | -24ms | ✅ None |
-| `/overwrite` | 501ms | 523ms | +22ms | ⚠️  Moderate |
+| `/create` | 57ms | 51ms | -6ms | ✅ None |
+| `/update` | 491ms | 525ms | +34ms | ⚠️  Moderate |
+| `/patch` | 502ms | 535ms | +33ms | ⚠️  Moderate |
+| `/set` | 497ms | 526ms | +29ms | ⚠️  Moderate |
+| `/unset` | 510ms | 528ms | +18ms | ⚠️  Moderate |
+| `/delete` | 504ms | 515ms | +11ms | ⚠️  Moderate |
+| `/overwrite` | 495ms | 525ms | +30ms | ⚠️  Moderate |
 
 **Key Insight**: Cache uses **O(n) linear scanning** for write invalidation.
 
@@ -116,22 +116,22 @@
 - **Conclusion**: Reads are always fast, even with cache misses
 
 **Write Operations (O(n)):**
-- Average O(n) scanning overhead: ~17ms per write
-- Overhead percentage: ~3% of write time
-- Total cost for 1000 writes: ~17000ms
+- Average O(n) scanning overhead: ~21ms per write
+- Overhead percentage: ~4% of write time
+- Total cost for 1000 writes: ~21000ms
 - Tested endpoints: create, update, patch, set, unset, delete, overwrite
 - **This is WORST CASE**: Real scenarios will have cache invalidations (better than pure scanning)
 
 **This worst-case test shows:**
 - O(1) read lookups mean cache size never slows down reads
-- O(n) write scanning overhead is 17ms on average
-- Even in worst case (no invalidations), overhead is typically 3% of write time
+- O(n) write scanning overhead is 21ms on average
+- Even in worst case (no invalidations), overhead is typically 4% of write time
 
 **Real-World Scenarios:**
 - Production caches will have LOWER overhead than this worst case
 - Cache invalidations occur when writes match cached queries (productive work)
 - This test forces pure scanning with zero productive invalidations (maximum waste)
-- If 17ms overhead is acceptable here, production will be better
+- If 21ms overhead is acceptable here, production will be better
 
 ---
 
@@ -141,7 +141,7 @@
 
 **What This Test Shows:**
 1. **Read overhead**: NONE - O(1) hash lookups are instant regardless of cache size
-2. **Write overhead**: 17ms average O(n) scanning cost for 1000 entries
+2. **Write overhead**: 21ms average O(n) scanning cost for 1000 entries
 3. **Worst-case verified**: Pure scanning with zero matches
 
 **If write overhead ≤ 5ms:** Cache overhead is negligible - deploy with confidence
@@ -150,9 +150,9 @@
 
 ### ✅ Is Cache Overhead Acceptable?
 
-Based on 17ms average overhead:
+Based on 21ms average overhead:
 - **Reads**: ✅ Zero overhead (O(1) regardless of size)
-- **Writes**: ✅ Acceptable
+- **Writes**: ⚠️  Review recommended
 
 ### 📊 Monitoring Recommendations
 
@@ -194,6 +194,6 @@ Tuning considerations:
 
 ---
 
-**Report Generated**: Tue Nov  4 21:49:52 CST 2025  
+**Report Generated**: Wed Nov  5 13:00:30 CST 2025  
 **Format Version**: 1.0  
 **Test Suite**: cache-metrics.sh

cache/__tests__/rerum-metrics.sh

@@ -0,0 +1,151 @@
+# RERUM Baseline Performance Analysis (No Cache)
+
+**Generated**: Wed Nov  5 12:31:45 CST 2025
+**Server**: https://devstore.rerum.io
+**Branch**: main (no cache layer)
+**Test Duration**: 4 minutes 41 seconds
+
+---
+
+## Executive Summary
+
+**Overall Test Results**: 17 passed, 0 failed, 0 skipped (17 total)
+
+This report establishes baseline performance metrics for the RERUM API without the cache layer. These metrics can be compared against CACHE_METRICS_REPORT.md to evaluate the impact of the caching implementation.
+
+---
+
+## Endpoint Functionality Status
+
+| Endpoint | Status | Description |
+|----------|--------|-------------|
+| `/query` | ✅ Functional | Query database with filters |
+| `/search` | ✅ Functional | Full-text search |
+| `/searchPhrase` | ✅ Functional | Phrase search |
+| `/id` | ✅ Functional | Retrieve object by ID |
+| `/history` | ✅ Functional | Get version history |
+| `/since` | ✅ Functional | Get version descendants |
+| `/create` | ✅ Functional | Create new objects |
+| `/update` | ✅ Functional | Update existing objects |
+| `/patch` | ✅ Functional | Patch existing objects |
+| `/set` | ✅ Functional | Add properties to objects |
+| `/unset` | ✅ Functional | Remove properties from objects |
+| `/delete` | ✅ Functional | Delete objects |
+| `/overwrite` | ✅ Functional | Overwrite objects without versioning |
+
+---
+
+## Read Performance
+
+| Endpoint | Avg (ms) | Median (ms) | Min (ms) | Max (ms) |
+|----------|----------|-------------|----------|----------|
+| `/query` | 453 | 453 | 453 | 453 |
+| `/search` | 151 | 151 | 151 | 151 |
+| `/searchPhrase` | 136 | 136 | 136 | 136 |
+| `/id` | 530 | 530 | 530 | 530 |
+| `/history` | 852 | 852 | 852 | 852 |
+| `/since` | 864 | 864 | 864 | 864 |
+
+**Interpretation**:
+- All read operations hit the database directly (no caching)
+- Times represent baseline database query performance
+- These metrics can be compared with cached read performance to calculate cache speedup
+
+---
+
+## High-Volume Query Load Test
+
+This test performs 1000 diverse read queries to measure baseline database performance under load. It directly corresponds to the `fill_cache()` operation in cache-metrics.sh, enabling direct comparison.
+
+| Metric | Value |
+|--------|-------|
+| Total Queries | 1000 |
+| Total Time | 66 seconds (66000ms) |
+| Average per Query | 66ms |
+| Successful Queries | 1000/1000 |
+| Failed Queries | 0/1000 |
+
+**Query Distribution**:
+- Rotates through 6 endpoint types: /api/query, /api/search, /api/search/phrase, /id/{id}, /history/{id}, /since/{id}
+- Each query uses unique parameters to prevent database-level caching
+
+**Comparison with Cache**:
+- Compare this total time with the cache fill operation time in CACHE_METRICS_REPORT.md
+- This shows baseline database performance for 1000 diverse queries without caching
+- Cache fill time includes both database queries (on cache misses) and cache.set() operations
+
+---
+
+## Write Performance
+
+| Endpoint | Avg (ms) | Median (ms) | Min (ms) | Max (ms) | Successful/Total |
+|----------|----------|-------------|----------|----------|------------------|
+| `/create` | 151 | 140 | 127 | 1195 | 100/100 |
+| `/update` | 587 | 566 | 547 | 1561 | 50/50 |
+| `/patch` | 568 | 567 | 547 | 618 | 50/50 |
+| `/set` | 597 | 570 | 542 | 1079 | 50/50 |
+| `/unset` | 572 | 566 | 543 | 710 | 50/50 |
+| `/delete` | 565 | 565 | 546 | 604 | 50/50 |
+| `/overwrite` | 567 | 568 | 550 | 594 | 50/50 |
+
+**Interpretation**:
+- All write operations execute without cache invalidation overhead
+- Times represent baseline write performance
+- These metrics can be compared with cached write performance to calculate cache overhead
+
+---
+
+## Summary Statistics
+
+**Total Operations**:
+- Read operations: 6 endpoints tested
+- Write operations: 400 operations across 7 endpoints
+
+**Success Rates**:
+- Create: 100/100
+- Update: 50/50
+- Patch: 50/50
+- Set: 50/50
+- Unset: 50/50
+- Delete: 50/50
+- Overwrite: 50/50
+
+**Test Execution**:
+- Total duration: 4 minutes 41 seconds
+- Test objects created: 100
+- Server: https://devstore.rerum.io
+
+---
+
+## Comparison Guide
+
+To compare with cache performance (CACHE_METRICS_REPORT.md):
+
+1. **Read Speedup**: Calculate cache benefit
+   ```
+   Speedup = Baseline Read Time - Cached Read Time
+   Speedup % = (Speedup / Baseline Read Time) × 100
+   ```
+
+2. **Write Overhead**: Calculate cache cost
+   ```
+   Overhead = Cached Write Time - Baseline Write Time
+   Overhead % = (Overhead / Baseline Write Time) × 100
+   ```
+
+3. **Net Benefit**: Evaluate overall impact based on your read/write ratio
+
+---
+
+## Notes
+
+- This test was run against the **main branch** without the cache layer
+- All timing measurements are in milliseconds
+- Clock skew was handled gracefully (operations with negative timing marked as 0ms)
+- Test objects should be manually cleaned from MongoDB using the commands provided at test start
+
+---
+
+**Report Generated**: Wed Nov  5 12:31:45 CST 2025
+**Format Version**: 1.0
+**Test Suite**: rerum-metrics.sh