Use region-based allocator for TrieNodes

[yihozhang]

No description provided.

[codspeed-hq]

Merging this PR will degrade performance by 15.87%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 6 improved benchmarks
❌ 6 regressed benchmarks
✅ 38 untouched benchmarks
⏩ 190 skipped benchmarks¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	WallTime	rust_rule_match_overhead[rule_run_1]	9.6 ms	11.4 ms	-15.87%
⚡	WallTime	tests[cykjson]	365.7 ms	347.9 ms	+5.1%
⚡	WallTime	tests[python_array_optimize]	2.5 s	2.4 s	+5.09%
❌	WallTime	rust_rule_tableaction_hot_path[facts50000_funcs200]	28.9 ms	30.8 ms	-6.03%
⚡	Simulation	tests[eggcc-2mm]	11 s	10.5 s	+5.31%
❌	WallTime	rust_rule_insert_loop[ops1000_funcs200]	349.2 µs	385.4 µs	-9.39%
⚡	WallTime	rust_rule_fib[rule_run_1000]	252.5 ms	217.3 ms	+16.2%
⚡	Simulation	tests[python_array_optimize]	2.5 s	2.3 s	+5.16%
❌	Simulation	rust_rule_match_overhead[rule_run_1]	14.9 ms	17.6 ms	-15.29%
⚡	Simulation	rust_rule_fib[rule_run_1000]	338.1 ms	319.2 ms	+5.89%
❌	Simulation	tests[cykjson]	497.9 ms	535.3 ms	-6.99%
❌	Simulation	rust_rule_tableaction_hot_path[facts50000_funcs200]	37.1 ms	39.5 ms	-6.18%

---

_{Comparing yihozhang-arena-trienodes (0f348d7) with main (23dd731)}

Footnotes

1. 190 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[codecov-commenter]

Codecov Report

❌ Patch coverage is 87.73585% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.15%. Comparing base (23dd731) to head (0f348d7).

Files with missing lines	Patch %	Lines
core-relations/src/free_join/execute.rs	93.36%	13 Missing ⚠️
core-relations/src/free_join/frame_update.rs	18.75%	13 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #868      +/-   ##
==========================================
- Coverage   87.18%   87.15%   -0.03%     
==========================================
  Files          87       87              
  Lines       24933    24975      +42     
==========================================
+ Hits        21737    21767      +30     
- Misses       3196     3208      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[yihozhang]

Below is what the agent found. It's worth noting this PR does not have the change from #857, so it creates many Dense singleton subsets that would otherwise be eliminated in #857. Still, it's slightly slower than #857 after incorporating its change.

Agent's finding

Investigation summary — why arena is slower

  Repro (with tests/cykjson.egg switched to the medium token, as you had locally):

  ┌──────────────────┬───────┬───────┬───────┬─────────┐
  │                  │ wall  │ user  │  sys  │   RSS   │
  ├──────────────────┼───────┼───────┼───────┼─────────┤
  │ baseline 23dd731 │ 5.7 s │ 5.8 s │ 0.0 s │ 20 MB   │
  ├──────────────────┼───────┼───────┼───────┼─────────┤
  │ arena HEAD       │ 8.4 s │ 5.8 s │ 2.5 s │ 2.28 GB │
  └──────────────────┴───────┴───────┴───────┴─────────┘
  
  CPU work (user time) is essentially unchanged. Almost all of the regression is from:
  - +2.5 s system time from page-faulting / mmap as the arena grows
  - ~100× memory bloat (20 MB → 2.28 GB) causing TLB/cache pressure
  
  Root cause: transient TrieNodes leak for the entire plan lifetime

  I instrumented every handle.alloc call site. For a single run of cykjson (medium):

  plans=528
  header                =          0
  insert_subset         = 116,272,833   <-- 99.997% of allocations
  refine_nondense       =      3,100
  get_cached_trie_node  =      2,330   (216 cache hits)
  new_handle_calls      =          0   (single-thread, parallel-drain path unused)

  Almost every TrieNode that lives in the arena is a transient wrapper around a Subset::Dense(range) produced by RefineAtomDense in
  the drain at execute.rs:1015-1016. In the Arc-based baseline these were Arc::new(TrieNode::new(...)) and their refcount dropped to
   zero the moment the next refinement / next frame overwrote binding_info.subsets[atom], so memory was reused continuously. With
  the arena, all 116 million of them stay alive until the per-plan SharedArena is dropped at end of plan.

  The cached_children work the design was trying to optimize (get_cached_trie_node) is fewer than 3,000 allocs in this benchmark —
  that's not the hot path.

  Secondary findings (perf profile)

  In the arena run, ~10% of cycles is direct arena machinery, plus the ~6 % kernel time:
  the drain at execute.rs:1015-1016. In the Arc-based baseline these were Arc::new(TrieNode::new(...)) and their refcount dropped to
   zero the moment the next refinement / next frame overwrote binding_info.subsets[atom], so memory was reused continuously. With
  the arena, all 116 million of them stay alive until the per-plan SharedArena is dropped at end of plan.

  The cached_children work the design was trying to optimize (get_cached_trie_node) is fewer than 3,000 allocs in this benchmark —
  that's not the hot path.

  Secondary findings (perf profile)

  In the arena run, ~10% of cycles is direct arena machinery, plus the ~6 % kernel time:

    5.99%  egglog_concurrency::shared_arena::Handle::alloc
    2.07%  <egglog_concurrency::shared_arena::LocalArena as Drop>::drop
    1.90%  core::ptr::drop_in_place<...::TrieNode>
    ~6%    kernel (page faults / mmap)

  For comparison, the Arc baseline spent ~6.6% in mimalloc + Arc::drop_slow doing the same allocation work plus immediate frees. The
   bumpalo arena isn't beating mimalloc on per-alloc cost here, and it pays the deferred-drop tax on top.

  Also worth flagging (latent bug, not hit by the single-thread cykjson repro): in drain_updates_parallel at execute.rs:1076,
  &arena.new_handle() is called per RefineAtomDense update. Each new_handle() takes a Mutex lock and Box::pin(LocalArena::new()). In
   multi-threaded runs this will both contend and accumulate one LocalArena per update.

  Suggested fixes (in priority order)

  1. Don't allocate a TrieNode in insert_subset for the dense fast path. This is what's blowing up. Options:
    - Change BindingInfo.subsets to a DenseIdMap<AtomId, NodeSlot<'arena>> enum with Subset(Subset) and Node(SharedRef<...>)
  variants. Lazily promote to Node only when get_cached_trie_node actually needs a stable parent (which happened only ~2,500 times
  vs 116 M).
    - Or: switch BindingInfo.subsets to store the TrieNode by value (it's a single-occupancy slot anyway). Cached-child storage in
  cached_children still uses SharedRef.
  2. In drain_updates_parallel, hoist arena.new_handle() out of the per-update loop — create one handle per recur task and reuse it.
   The current code mutex-locks once per dense refinement.
  3. If the arena lifetime is going to stay "per plan," the perf model assumed by Pooled<ChildrenMaps> in pool/mod.rs:445 is broken
  too: those maps are returned to the pool only on TrieNode drop, which now never happens mid-plan, so the pool can't recycle them
  within a plan. Fix #1 makes this moot for transients; for the genuinely-cached ones it just means the pool is effectively unused.

  The arena design is sound for the cached TrieNodes (the ones in cached_children), but it's currently being used for the wrong set
  of nodes. Once fix #1 lands, the arena's job is to hold ~thousands of long-lived shared TrieNodes per plan instead of ~hundreds of
   millions of single-use ones, and the Arc-based atomic refcounting it eliminates should actually become measurable savings.