BSON vs. JSON Adapters for Large Datasets

[emirhneks]

I'm curious about the performance gap between BSON and JSON adapters in MZRDB.

• Have you benchmarked read/write speeds for large datasets (e.g., 100,000+ records)?
• In which specific use cases is BSON more advantageous than JSON?

[MZRCode]

From the current MZRDB implementation and our local benchmark tests, BSON appears to be more advantageous mainly in read-heavy scenarios, because the BSON adapter keeps deserialized data in memory within the adapter instance, while the JSON adapter tends to re-read and re-parse the file more often.

That said, BSON is not automatically better for every large dataset. In the current adapter design, BSON stores the whole dataset as a single BSON document, so very large bulk payloads can hit document-size limits. In our tests, a 100,000 record bulk write caused the BSON path to fail before the benchmark completed.

So the practical takeaway is:

• BSON is better for moderately large, read-heavy workloads.
• JSON is better when human readability and manual inspection matter.
• For very large local datasets, the current BSON adapter would need a chunked or multi-file strategy before it can be treated as the better default.

If you've run benchmarks on your side, feel free to share the numbers as well. That would be useful both for us and for future users evaluating which adapter fits their workload better.

[emirhneks]

I did some research and benchmarking on MZRDB's performance. Here are the key findings based on the data:

• Read Performance: BSON is the clear winner. For bulk datasets (20k+ records), BSON's cold read is ~0.05ms, while JSON takes ~31ms.
• Write Performance: Both are similar for bulk writes (~55ms). However, JSON is faster for small, incremental updates (100 records).
• Storage: JSON files are roughly 10% smaller than BSON for these datasets.

Conclusion: BSON is ideal for read-heavy applications where speed is critical. JSON is better for human-readability and smaller, frequent write operations.