Memory Management and Cache Limits

[emirhneks]

How does MZRDB handle memory usage for very large databases? Since hot reads are near-instant (0.00ms), is there a built-in mechanism to clear the cache or limit RAM usage when the dataset grows too large?

[MZRCode]

Thanks for raising this. Based on the current implementation, MZRDB does not appear to have a general built-in cache manager with eviction, RAM limiting, or a dedicated cache-clearing API for large local datasets.

For jsondb, reads do not seem to come from a persistent in-memory cache. The behavior suggests data is read and parsed on demand, so near-instant hot-read timings alone should not be interpreted as proof of an internal RAM cache. For bsondb, memory-backed behavior is more plausible, but very large datasets may also run into BSON document size constraints, which is a separate scalability consideration.

So at the moment, the practical answer seems to be:

• no visible built-in RAM cap
• no visible cache eviction mechanism
• no explicit clearCache() style API for local adapters

If you want, you can share your benchmark results and dataset shape with us as well. That would help validate the observed behavior under larger real-world workloads.

[emirhneks]

Thanks for the clarification. Based on your feedback, I’ve conducted a quick benchmark to observe the behavior of jsondb and bsondb adapters under repeated read operations.

Key Observations from my results:

• Memory Growth: jsondb shows a significant Heap increase during repeated gets (+84MB), confirming that data is likely parsed on demand without a persistent cache, leading to higher GC pressure.
• BSON Efficiency: Interestingly, bsondb remained extremely stable after the initial load (+0MB delta on repeated reads), suggesting it handles memory much more efficiently for local datasets.
• Release Mechanism: I noticed that a manual release or closure helps drop the Heap usage significantly (from ~88MB back to ~46MB in jsondb), which provides some manual control despite the lack of an automated eviction API.

I’ve attached the benchmark table for your review. It seems that while there is no built-in RAM cap, adapter choice (BSON vs JSON) is currently the most critical factor for scalability.