memory-perf-roadmap.md

Memory & Performance Roadmap: Beating Bun and Node on bench_json_roundtrip

Status: active. Written 2026-04-24 alongside v0.5.193. Goal: beat both Node and Bun on both time and peak RSS on the reference bench_json_roundtrip.ts workload — a 50-iteration JSON.parse + JSON.stringify loop over a ~1 MB blob with a 10k-record module-level setup array.

This doc is a living plan. Each tier has ship criteria and a rough impact estimate; update numbers and strike through items as they land.

Current standings (v0.5.211, macOS ARM64, best-of-5)

bench_json_roundtrip

Runtime	Time	Peak RSS
Perry v0.5.211 (lazy, default)	90 ms	130 MB
Perry v0.5.211 (PERRY_JSON_TAPE=0)	400 ms	137 MB
Node 25.8.0	520 ms	180 MB
Bun 1.3.12	290 ms	81 MB

bench_json_readonly (.length only)

Runtime	Time	Peak RSS
Perry v0.5.211	80 ms	90 MB
Perry v0.5.211 (forced direct)	290 ms	120 MB
Node 25.8.0	450 ms	169 MB
Bun 1.3.12	200 ms	68 MB

bench_json_readonly_indexed (.length + 3 indexed reads)

Runtime	Time	Peak RSS
Perry v0.5.211	90 ms	90 MB
Perry v0.5.211 (forced direct)	300 ms	120 MB
Node 25.8.0	450 ms	169 MB
Bun 1.3.12	210 ms	68 MB

🎯 Perry now beats BOTH Node and Bun on TIME for every measured JSON benchmark. Node beaten on both axes. Bun beaten on time by 2.1-3.2×; Bun still leads on RSS by ~50% (~70 MB vs ~90 MB). Closing the RSS gap requires tier 2 and tier 3 architectural work below.

Historical reference (bench_json_roundtrip):

Version	Time	Peak RSS	Notes
v0.5.190 (pre-fix)	316 ms	318 MB	block-persist cascade active
v0.5.192	316 ms	318 MB	segregated longlived arena (infra only)
v0.5.193	384 ms	213 MB	age-restricted block-persist + no 2-cycle grace
v0.5.194	322 ms	199 MB	block size 8 MB → 1 MB (tier 1 #1)
v0.5.195	322 ms	199 MB	NEON/SSE2 string scanner (tier 1 #3 partial)
v0.5.196	373 ms	144 MB	GC initial threshold 128 MB → 64 MB
v0.5.203	789 ms	294 MB	tape-based parse foundation (opt-in PERRY_JSON_TAPE=1)
v0.5.204	69 ms (flag on)	108 MB	lazy parse + lazy stringify memcpy
v0.5.206	69 ms (flag on)	108 MB	lazy-safe indexed access, edge case coverage
v0.5.207	69 ms (pragma)	108 MB	@perry-lazy JSDoc pragma for per-file opt-in
v0.5.208	90 ms (flag on)	130 MB	per-element sparse materialization — indexed bench cliff eliminated
v0.5.209	90 ms (flag on)	130 MB	walk cursor + adaptive threshold — sequential + random cliffs eliminated
v0.5.210	90 ms (default)	130 MB	flipped lazy to default above 1024-byte threshold

Why Perry loses each axis

Time (−136 ms vs Bun)

• JSON.parse / JSON.stringify dominate this bench's CPU.
• Perry's parser is recursive-descent with a zero-copy escape-free fast path (crates/perry-runtime/src/json.rs::DirectParser). Single-byte-at-a-time scanner.
• Bun uses a simdjson-derived SIMD parser. On the benchmark's ~1 MB blob, SIMD parse is typically 2-4× a hand-written recursive descent.
• Perry's stringify is single-pass scalar too. Same ~2× window vs. a SIMD writer.
• Everything else on this bench (allocation, GC, mark, sweep) is already fast.

RSS (+130 MB vs Bun)

• Bun uses a generational GC: young nursery (~2 MB), precisely swept every few thousand allocations. Most parse output is trivially young-gen garbage and never makes it to old space.
• Perry has one flat arena with 8 MB block granularity. Even with v0.5.193's age-restricted block-persist, the recent-5-block safety window alone reserves 40 MB, and 8 MB blocks can only be reclaimed when fully dead.
• No generational split = Perry's runtime state grows as the union of ("live set"
  • "recent-window headroom" + "any block still in use by the current allocation burst"). On a 5 MB/iter workload, that union is ~20 MB live + 40 MB window + some overshoot ≈ what we see.

The levers, ranked by impact/effort

Each item lists: impact estimate (RSS, time), effort, risk, scope.

Tier 1 — days of work, meaningful wins

1. Shrink arena block size 8 MB → 1-2 MB ✅ v0.5.194

Landed: block size 8 MB → 1 MB. Measured on bench_json_roundtrip (best-of-5, macOS ARM64):

Block size	Time	Peak RSS
8 MB (baseline)	384 ms	213 MB
2 MB	325 ms	208 MB
1 MB	322 ms	199 MB
512 KB	318 ms	200 MB (diminishing returns)

RSS win was modest vs. projection (213 → 199 MB, 7% instead of the projected 213 → 130 MB). Turns out the bulk of the 213 MB wasn't the recent-5-block window but the allocation-headroom-between-GCs — which scales with the adaptive step, not block size. Time win was the surprise: 384 ms → 322 ms (−16%), because smaller blocks = more frequent arena growth = more frequent GC triggers = the adaptive step halves faster, the workload's 60-80% freed-pct per cycle actually lands reclaim in time instead of sitting on a too-high step.

All seven regression benchmarks unchanged: 07_object_create, 12_binary_trees, 02_loop_overhead, 06_math_intensive, bench_gc_pressure, 03_array_write, bench_array_grow. Gap tests 24/28 unchanged. Runtime tests 124/124.

2. Short string optimization (SSO)

• Impact: RSS 10-30 MB savings on string-heavy benches; time ~5-15% win on workloads allocating many short strings. Strings ≤6 bytes encode directly in the NaN-boxed f64 payload (6 bytes × 8 bits = 48 bits; payload is 48 bits), skipping the entire arena/malloc path — no StringHeader, no GcHeader.
• Effort: ~1 week. Affects:
  • crates/perry-runtime/src/string.rs: new SHORT_STRING_TAG (distinct from STRING_TAG which points to heap), encoders/decoders, js_string_from_bytes dispatches on length.
  • Every is_string/as_string_ptr/string-decode site — likely 30-50 call sites across runtime and codegen.
  • Codegen emits length-aware decode for property access.
• Risk: easy to miss a decode path → NPE or wrong string content. Mitigated by a single js_string_decode(v: JSValue) -> (*const u8, u32) helper used everywhere.
• Scope: touches many files but each edit is mechanical.

Ship criteria:

• test_edge_string_* tests all pass.
• All 28 gap tests in same state (24/28, no regression).
• bench_json_roundtrip RSS drops another 10+ MB.

3. SIMD JSON parser — tactical string scanner ✅ v0.5.195

Landed: NEON + SSE2 scanner for the parse_string_bytes fast path. 16-byte chunk scan for " or \ with scalar tail. New top-level helpers find_string_terminator{,_neon,_sse2,_scalar} in json.rs.

Measured on a long-string synthetic bench (100+ char strings, 5k records × 30 iters):

• Scalar: 92-102 ms
• NEON: 75-77 ms (18% faster)

Measured on bench_json_roundtrip: no meaningful change — 322 ms stayed at 316-322 ms. The workload's strings (keys 2-6 chars, values 5-10 chars) are all <16 bytes, so the SIMD body loop never executes; every string hits the scalar tail. This was not what tier 1 #3's 2-4× projection assumed: that figure comes from real-world JSON where string payloads are typically 20-80 bytes (API responses, log lines, prose) — for that shape, a chunk scanner matters.

Didn't attempt (deferred): SIMD structural scan (finding {}[],:" positions in one sweep) — the simdjson architectural pattern. That would matter on this bench regardless of string length, because every structural byte (~4 per field) goes through the recursive-descent driver. Estimated ~2× parse speedup on bench_json_roundtrip but requires a substantial rewrite of DirectParser. Tracked as a follow-up; not worth doing before tier 1 #2 (SSO) because SSO reduces allocation-path cost which currently dominates the parse inner loop.

Tier 2 — weeks of work, structural wins

4. Escape analysis via TypeScript types

• Impact: RSS 10-30 MB on allocation-heavy benches; time ~10-20% win when short-lived intermediate objects get stack-allocated. On the JSON bench, the per-iteration tags array and nested object never escape — both become stack allocations.
• Effort: 2-3 weeks. New HIR pass in crates/perry-hir or crates/perry-transform tracking value flow. Codegen emits alloca for non-escaping objects instead of arena_alloc_gc. Non-escape proof:
  • Value not stored into an escaping container
  • Value not returned
  • Value not captured by a closure that outlives the scope
• Risk: proof bugs → UAF (stack-allocated object accessed after scope). Must be conservative: when in doubt, heap-allocate. Testing: use RUSTFLAGS="-Zsanitizer=address" on the runtime suite before shipping.
• Scope: HIR pass + codegen alloca path. Well-bounded if we keep the proof conservative.

Ship criteria:

• All workspace tests pass under -Zsanitizer=address.
• bench_json_roundtrip RSS drops another 20+ MB.
• No test regressions in gap suite or runtime tests.

5. Precise root tracking via codegen

• Impact: by itself, zero. But it's the unlock for tier 3. Once roots are precise, conservative stack scan goes away, mark_block_persisting_arena_objects goes away entirely, moving GC becomes possible.
• Effort: 3-4 weeks. Emit a per-function "shadow stack" at every safepoint: a stack-allocated array of pointers to live JS values. GC walks the shadow stack instead of the raw machine stack.
• Risk: register pressure + shadow-stack overhead. Benchmark carefully. Typical cost: 2-8% on pointer-heavy workloads; effectively free on computation-heavy workloads.
• Scope: codegen.rs + every call-site emission. Large but mechanical.

Ship criteria:

• All gap tests + runtime tests pass with conservative scan disabled.
• No benchmark regresses >5%.
• mark_block_persisting_arena_objects can be deleted.

Tier 3 — month+ of work, architectural answer

6. Generational GC (requires tier 2 #5)

• Impact: RSS drops to approximately live-set-size + nursery. Estimated bench_json_roundtrip RSS 213 → ~50 MB (below Bun). Time: minor GCs are very cheap; major GCs rarer; net ~10-20% speed win.
• Effort: 3-4 weeks on top of #5. Non-moving generational collector:
  • Young nursery: fixed 1-2 MB arena
  • All new allocations go to nursery
  • Minor GC: scan precise roots + remembered set → mark nursery → survivors promote to old arena (copy or bump in place depending on moving strategy)
  • Old arena: current mechanism (mark-sweep with block reset)
  • Write barrier: every store of a young pointer into an old object adds to remembered set (codegen emits the check)
• Risk: write barrier overhead (typically 5-10%). Promotion correctness. Mitigated by gradual rollout behind a flag.
• Scope: new gc.rs pass + codegen write barrier + new arena split.

Ship criteria:

• Perry RSS on bench_json_roundtrip ≤90 MB.
• All benchmarks within 5% of baseline speed.
• Full test suite clean.

7. Full compacting GC

• Impact: fragmentation disappears, RSS = live-set. This is the theoretical floor for non-streaming workloads.
• Effort: on top of #6, another 2-4 weeks. Requires moving GC — objects change addresses during collection. Pointers everywhere must be updated (only safe with precise roots from #5).
• Risk: large — any stale pointer (e.g. in a runtime function holding a raw reference across an allocation) = use-after-move.
• Scope: full GC rewrite.

Ship criteria: TBD after #6 lands; may not be worth it.

Recommended path

1. Now: Tier 1 #1 (smaller blocks). Fastest visible RSS drop; contained risk.
2. Next: Tier 1 #2 (SSO). Compounds with #1 — fewer allocations means smaller blocks fill more slowly.
3. Then: Tier 1 #3 (SIMD JSON). Closes the time gap to Bun; no RSS impact but gets us to "beats Node, close to Bun on time".
4. Evaluate. If Perry is within ~30 MB RSS of Bun after tier 1, the generational GC is probably still worth doing but loses some urgency. If we're still >100 MB behind, tier 2+3 is mandatory to win.
5. Tier 2 #4 (escape analysis) before #5 — it's cheaper and has immediate RSS value even without the precise-root infrastructure.
6. Tier 2 #5 + Tier 3 #6 together as the architectural overhaul. Do this only once tier 1 has shipped and been stable for a release or two.

Tradeoffs to call out explicitly

Every one of these trades codegen/runtime complexity for performance. Perry's current appeal is partly that the codegen is tractable and the runtime is readable. A generational GC with write barriers and shadow stacks changes that character substantially.

Before committing to tier 2+3, the maintainer should decide:

• Is the bench_json_roundtrip-style workload actually representative of what Perry users care about?
• Would it be more valuable to focus Perry on workloads where its current architecture wins (startup time, LLVM-optimized hot paths, native UI, static typing) and accept the RSS gap on GC-heavy benches?

If the answer is "yes, we want to win GC-heavy workloads too", then tier 2+3 is the honest path. If the answer is "Perry's niche is elsewhere", tier 1 alone is plenty.

Log

Date	Version	Change	Result
2026-04-24	v0.5.192	Tier 0 (not listed above): segregated longlived arena for caches (PR #179 scope A)	RSS unchanged, infrastructure in place
2026-04-24	v0.5.193	Tier 0 (cont.): age-restricted block-persist, adaptive step tune, drop 2-cycle grace on old blocks	RSS 318 → 213 MB (−33%); time +21%
2026-04-24	v0.5.194	Tier 1 #1: block size 8 MB → 1 MB	RSS 213 → 199 MB (−7%); time 384 → 322 ms (−16%). Now beats Node on both axes.
2026-04-24	v0.5.195	Tier 1 #3 (partial): NEON/SSE2 string scanner in json.rs	bench_json_roundtrip unchanged (strings <16 bytes); long-string synthetic bench −18% time. Infrastructure for real-world JSON workloads.
2026-04-24	v0.5.196	Tier 1 follow-up: GC initial threshold 128 MB → 64 MB	RSS 199 → 144 MB (−28%); time 322 → 373 ms (+16%, still 3% faster than Node). Now beats Node on both axes by a comfortable margin (−3% time, −23% RSS).