You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: scripts/ralph/progress.txt
+26Lines changed: 26 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -233,3 +233,29 @@ Raw captures retained at `/tmp/us-059-metrics-full.txt` (engine /metrics, all fa
233
233
- `prd.json` can drift behind the actual branch state. If `git log` already contains `feat: [US-048] - [...]` but `passes` is still false, fix the bookkeeping before Ralph burns another cycle re-implementing the same story.
234
234
- `cargo test -p sqlite-storage` and `cargo test -p rivetkit-sqlite-native` run cleaner as isolated story-focused filters here; a concurrent full-package run produced a hanging compaction test and native RocksDB lock noise that did not reproduce in isolated checks.
Question: for the original 5 MB bench (`largeTxInsert5MB`), engine-side commit work summed to ~233 ms but total E2E was 1128 ms, leaving ~900 ms unaccounted for. Hypothesis was that per-statement / NAPI overhead dominates the gap. To test, three new bench variants commit the same 1 MiB payload shaped three ways (different statement counts).
240
+
241
+
Environment: local RocksDB engine on `:6420` with US-048 and US-059 both landed, kitchen-sink `--prod dist/server.js` on `:3001`, namespace `fix2`, fresh actor per run (new key), `RUST_LOG=info` (via `scripts/run/engine-rocksdb.sh` default).
242
+
243
+
| Variant | Rows × payload | NAPI crossings | E2E | Server | Per-op |
| Tiny | 4096 × 256 B | 4096 | 334.6 ms | 311.8 ms | 0.1 ms |
246
+
| Medium | 256 × 4 KiB | 256 | 158.2 ms | 141.2 ms | 0.6 ms |
247
+
| One row | 1 × 1 MiB | 1 | 132.6 ms | 114.0 ms | 114.0 ms |
248
+
249
+
All three commit 1 MiB total. The floor (one-row, 1 NAPI crossing) is **132.6 ms**. Adding statements scales the time linearly:
250
+
- Tiny vs one-row: +202 ms over +4095 crossings ≈ **49 µs per extra statement**.
251
+
- Medium vs one-row: +25.6 ms over +255 crossings ≈ **100 µs per extra statement**.
252
+
253
+
Interpretation: **per-statement cost (NAPI + SQLite prepare/bind/step/finalize + arg marshaling) is the primary source of the 5 MB bench's unexplained ~900 ms.** The 5 MB bench fires 1280 INSERTs. At ~50 µs/statement (warm cache, small args) that's ~64 ms; the 5 MB bench probably has higher per-statement cost because `randomblob(4096)` produces larger bound args and dirties more pages per statement, pushing per-statement cost into the 500-700 µs range. 1280 × 600 µs ≈ 770 ms, a plausible match for the observed ~900 ms gap.
254
+
255
+
Follow-up levers (NOT part of US-048 or US-055):
256
+
- **Batched INSERT** — the existing `insertBatch` action shape (one multi-VALUES INSERT) would collapse 1280 NAPI crossings to 1. Try adding a 5 MB variant that uses batched insert to confirm.
257
+
- **Prepared statement cache** — the native VFS could cache `sqlite3_stmt` for identical SQL text across execute calls to avoid re-prepare costs.
258
+
- **JS-side payload batching** — the db.execute() API could accept an array of `[sql, args]` pairs and do N calls in one NAPI round trip.
259
+
260
+
Per-variant engine-side commit phase histograms could not be cleanly attributed because the `/metrics` histogram has been accumulating across the full engine run (88 commits total in the current window, most from earlier work). For a clean per-variant Prometheus attribution, scrape `/metrics` before and after each run.
0 commit comments