Skip to content

Commit 35e5e9c

Browse files
committed
docs: refresh benchmark numbers after the bulk-load fix
Bulk load went from ~12x behind redb to 1.6x and ahead of RocksDB, and sorted bulk load is now slightly ahead of redb, so the previous "bulk load trails badly" framing no longer describes the engine. Reads and the remaining tables are re-measured from the same runs, with redb, RocksDB, and SQLite measured alongside so the ratios hold even though absolute numbers move with host load. Restate what actually trails: the fsync-bound single-key commit, where the per-transaction costs land in full.
1 parent be901f6 commit 35e5e9c

1 file changed

Lines changed: 34 additions & 36 deletions

File tree

README.md

Lines changed: 34 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,7 @@ cargo bench --bench segment
131131
cargo bench --bench compaction
132132
cargo bench --bench authenticated_node_read
133133
cargo bench --bench read_path
134+
cargo bench --bench write_path
134135
cargo bench -p pagedb-engine-comparison --bench btree
135136
cargo bench -p pagedb-engine-comparison --bench comparison
136137
```
@@ -139,6 +140,8 @@ cargo bench -p pagedb-engine-comparison --bench comparison
139140
a multi-level tree, measuring cold authenticated leaf/internal kind discovery
140141
without resolving or compiling an external database engine. `read_path`
141142
decomposes a warm lookup into transaction open/close versus tree descent.
143+
`write_path` reports per-row allocation totals and per-row cost against
144+
transaction size.
142145

143146
The comparison suite is a non-default workspace package under
144147
[`benchmarks/engine-comparison`](./benchmarks/engine-comparison). Normal
@@ -150,40 +153,37 @@ equivalent, so retaining it would compare different feature surfaces.
150153

151154
### vs. redb (B+ tree, in-process, 1 000-key tree)
152155

153-
| Workload | pagedb | redb | vs redb |
154-
| ------------------------------- | -------: | ------: | -------------- |
155-
| Point get (per-txn, AEAD) | 507 ns | 412 ns | 1.23× slower |
156-
| Batched insert (1 000 keys/txn) | **674 µs** | 1.80 ms | **2.67× faster** |
157-
| Per-txn insert (in-memory) | 25.7 µs | 13.2 µs | 1.95× slower |
158-
| Per-txn insert (file, AEAD on) | 120.3 µs | 13.2 µs | 9.1× slower |
156+
| Workload | pagedb | redb | vs redb |
157+
| ------------------------------- | ---------: | ------: | ---------------- |
158+
| Point get (per-txn, AEAD) | 692 ns | 679 ns | 1.02× slower |
159+
| Batched insert (1 000 keys/txn) | **722 µs** | 2.10 ms | **2.91× faster** |
160+
| Per-txn insert (in-memory) | 29.1 µs | 14.5 µs | 2.01× slower |
161+
| Per-txn insert (file, AEAD on) | 140.1 µs | 14.5 µs | 9.7× slower |
159162

160-
AEAD overhead on reads is **~1.02×** (507 ns vs 495 ns plaintext+MAC) —
161-
encryption really is close to free on hot reads, because the buffer pool holds
162-
decrypted plaintext and a warm hit re-authenticates nothing.
163+
AEAD overhead on reads is **~0.9–1.0×** (692 ns AEAD vs 787 ns plaintext+MAC,
164+
i.e. inside run-to-run noise) — encryption really is free on hot reads, because
165+
the buffer pool holds decrypted plaintext and a warm hit re-authenticates
166+
nothing.
163167

164168
### vs. redb / RocksDB / SQLite (full comparison suite, 100 000 rows)
165169

166-
| Workload | pagedb | redb | RocksDB | SQLite |
167-
| ------------------------------ | ---------: | ---------: | ----------: | ---------: |
168-
| Random point read | 5.3 µs | **2.3 µs** | 5.8 µs | 23.3 µs |
169-
| Range scan (10 entries) | 9.2 µs | **3.8 µs** | 4.2 µs | 72.4 µs |
170-
| Individual write (fsync-bound) | 265.4 µs | 68.1 µs | **20.6 µs** | 72.9 µs |
171-
| Batch write (1 000 keys/txn) | 17.32 ms | 5.72 ms | **3.27 ms** | 7.22 ms |
172-
| Bulk load (100 000 rows/txn) | 2.85 s | **233 ms** | 476 ms | 322 ms |
173-
| Sorted bulk load | 1.17 s | **136 ms** | 235 ms | 161 ms |
174-
175-
**Reads are competitive; writes trail, and bulk load trails badly.** pagedb is
176-
within ~1.2× of redb on a warm 1 000-key tree and ~2.3× behind on a 100 000-row
177-
working set, where it beats RocksDB on point reads. On writes it is 1.9× behind
178-
redb per in-memory transaction and ~12× behind on bulk load. Part of that gap is
179-
the deliberate price of AEAD on every page, CoW shadow paging with A/B headers,
180-
and per-realm AAD binding — none of which redb or RocksDB carry — but the
181-
bulk-load gap is larger than that price explains and is a genuine weakness, not
182-
a design tax. Write-throughput work (group-commit tuning, write-coalescing,
183-
vectored fsync) is where the remaining headroom is.
184-
185-
Scans return owned `Vec`s, so a 10-row scan costs ~91 allocations against
186-
redb's ~14 — a lazy borrowing cursor is the obvious next read-path win.
170+
| Workload | pagedb | redb | RocksDB | SQLite |
171+
| ------------------------------ | ---------: | ---------: | ---------: | ---------: |
172+
| Random point read | 1.1 µs | **934 ns** | 951 ns | 10.5 µs |
173+
| Range scan (10 entries) | 5.8 µs | **1.8 µs** | 2.3 µs | 30.8 µs |
174+
| Individual write (fsync-bound) | 183.0 µs | 23.9 µs | **4.7 µs** | 48.9 µs |
175+
| Batch write (1 000 keys/txn) | 8.34 ms | 3.62 ms | **1.91 ms** | 4.84 ms |
176+
| Bulk load (100 000 rows/txn) | 194 ms | **122 ms** | 267 ms | 168 ms |
177+
| Sorted bulk load | 104 ms | 115 ms | 138 ms | **98 ms** |
178+
179+
**Reads and bulk load are competitive; single-key writes trail.** pagedb is
180+
level with redb on a warm 1 000-key tree and within ~1.2× on a 100 000-row
181+
working set. Bulk load is 1.6× behind redb and ahead of RocksDB; sorted bulk
182+
load is slightly ahead of redb. The gap is the fsync-bound single-key commit,
183+
where every transaction pays AEAD on each dirty page, a CoW shadow-page A/B
184+
header swap, and per-realm AAD binding — none of which redb or RocksDB carry.
185+
Batching amortises it; group-commit tuning and vectored fsync are the remaining
186+
headroom.
187187

188188
### Segment writer (append + seal)
189189

@@ -230,12 +230,10 @@ Every pagedb commit pays for:
230230
3. **Per-realm AAD binding** on every page (cross-tenant misroute protection at runtime).
231231
4. **Async I/O** — every write goes through the runtime, not a blocking syscall.
232232

233-
redb skips all of this. The trade is deliberate, not a bug — but be clear about
234-
where it lands: reads come out roughly level with redb (~1.2× behind warm,
235-
~2.3× on a 100 000-row set), not ahead, and write-heavy workloads pay the
236-
safety/portability tax until further write-path tuning lands. Bulk load is the
237-
weakest path by a wide margin; if your workload is dominated by loading large
238-
batches, measure before committing.
233+
redb skips all of this. The trade is deliberate, not a bug. Reads come out level
234+
with redb and bulk load is competitive; the cost lands on the fsync-bound
235+
single-key commit, since all four items are paid per transaction. Batch your
236+
writes and it amortises.
239237

240238
### "Why is there no mmap of encrypted bytes?"
241239

0 commit comments

Comments
 (0)