Misc offset data structure fixes, performance#13304
Draft
mitchellh wants to merge 19 commits into
Draft
Conversation
Bitmap allocations of at most 64 chunks only searched within a single bitmap word. Contiguous free space crossing a word boundary was therefore invisible and could force an unnecessary page growth. Larger requests could also advance past the final bitmap before returning out of memory. Search adjacent words for small cross-boundary runs and guard the final word in the large-allocation path. Add boundary regressions and a randomized oracle that verifies both placement and bitmap mutation.
RefCountedSet used maxInt(Id) both as an empty bucket marker and as the final valid bucket in its largest table. A dead-ID reuse sequence at maximum capacity could therefore treat a reset item as table-resident. The floating-point capacity calculation could also produce 65,536 for a u16 page capacity. Mark empty items with the probe sequence field, whose valid range is already capped below maxInt, and calculate the 13/16 load factor exactly. Represent the largest table request as maxInt(Id), which Layout.init rounds to the required power of two. Add maximum-capacity regressions and a randomized live-value oracle for ID reuse and displacement.
Page-local grapheme and hyperlink maps hashed 32-bit cell offsets through the generic byte-oriented Wyhash path. Hashing is paid on every lookup, insert, move, and erase even though the key is already an integer. Add an offset context that widens the key before applying the integer mixer. Widening preserves entropy in the high bits used by the map fingerprint, while avoiding the generic byte hashing path. Use the specialized context for both cell-offset maps. ReleaseFast medians use seven sequential runs after two warmups and report user CPU time. Each pass visits a 4,096-entry hyperlink map. | workload | working-set load | passes | before | after | change | |---|---:|---:|---:|---:|---:| | lookup | 100% | 40,000 | 478.518 ms | 313.179 ms | -34.6% | | remove/insert churn | 50% | 5 | 491.084 ms | 383.838 ms | -21.8% |
Starting an OSC 8 hyperlink copied its ID and URI into page memory before the ref-counted set could determine that the same live entry already existed. Repeated explicit IDs therefore paid allocation, copy, and free costs for data that was immediately discarded. Add adapted lookup support to RefCountedSet and let decoded hyperlinks hash and compare directly against page-resident entries. Probe explicit IDs before allocating strings and increment the existing reference on a match. Implicit IDs remain on the original path because they are generated uniquely and an early probe would always miss. An explicit and implicit parity test pins decoded and resident hashes to the same bit pattern so the adapted lookup cannot silently miss after either implementation changes. This also lets a live link be reused when the page string allocator has no free chunks. The old path requested a transient string copy first and could force an unnecessary page-capacity increase before de-duplicating. ReleaseFast medians use seven sequential runs after two warmups and report user CPU time. The stream reasserts one live explicit link 25,000 times while overwriting the same cell. | workload | before | after | change | |---|---:|---:|---:| | repeated explicit link | 437.925 ms | 438.051 ms | +0.0% | The isolated throughput effect is neutral; this change removes transient allocation and capacity growth rather than improving the steady-state parser path.
Add focused live-lookup and all-dead churn workloads for the terminal style set. The benchmark uses a real page-backed set so its layout and offset access match the production path. The lookup mode repeatedly acquires existing styles while preserving their liveness. The churn mode releases the complete working set between passes to model clear and redraw behavior.
Keep the resident RefCountedSet item behind a pointer while probing. The previous value copy materialized the complete generic item before checking its metadata and equality, which is especially costly for terminal styles. ReleaseFast timings are seven-run median user CPU times. | Workload | Before | After | Change | | --- | ---: | ---: | ---: | | 128 styles, live lookup | 0.80s | 0.74s | -7.5% | | 4,096 styles, live lookup | 0.92s | 0.87s | -5.4% | | Alternating SGR stream | 1.21s | 1.20s | -0.8% |
Use the two-round SplitMix64 finalizer for the folded PackedStyle key. The previous std.hash.int u64 mixer performs three multiply rounds. Style hashing is paid for every real SGR transition, and the additional round did not improve lookup behavior for this key distribution. ReleaseFast timings are seven-run median user CPU times. | Workload | Before | After | Change | | --- | ---: | ---: | ---: | | 4,096 styles, live lookup | 0.87s | 0.83s | -4.6% | | Alternating SGR stream | 1.20s | 1.13s | -5.8% | | Truecolor SGR stream | 0.68s | 0.67s | -1.5% | | Clear/redraw SGR stream | 0.68s | 0.67s | -1.5% |
Compute a RefCountedSet value hash once and pass it through lookup and insertion. A miss previously hashed before probing and then repeated the same work when insertion began. The shared private lookup path also keeps zero-capacity contexts from being evaluated before their backing data exists. ReleaseFast timings are seven-run median user CPU times. | Workload | Before | After | Change | | --- | ---: | ---: | ---: | | 128 styles, live lookup | 0.75s | 0.69s | -8.0% | | 128 styles, dead churn | 0.79s | 0.72s | -8.9% | | 4,096 styles, dead churn | 0.85s | 0.77s | -9.4% | | Alternating SGR stream | 1.13s | 1.11s | -1.8% | | Clear/redraw SGR stream | 0.67s | 0.65s | -3.0% |
Reset the RefCountedSet table in one pass when every stored item is dead. The previous add path deleted trailing entries individually, repeatedly backshifting probe sequences even though no live entry needed to survive. Deletion callbacks still run exactly once before storage is cleared, so page-owned hyperlink values retain their existing ownership semantics. ReleaseFast timings are seven-run median user CPU times. | Workload | Before | After | Change | | --- | ---: | ---: | ---: | | 128 styles, dead churn | 0.72s | 0.60s | -16.7% | | 4,096 styles, dead churn | 0.77s | 0.66s | -14.3% | | Clear/redraw SGR stream | 0.65s | 0.61s | -6.2% |
Carry the current RefCountedSet bucket through the probe loop instead of recomputing hash plus probe length and narrowing it on every iteration. The bucket remains in the set ID type and wraps with the existing table mask. ReleaseFast timings are seven-run median user CPU times. | Workload | Before | After | Change | | --- | ---: | ---: | ---: | | 128 styles, live lookup | 0.71s | 0.70s | -1.4% | | 4,096 styles, live lookup | 0.89s | 0.85s | -4.5% | | Alternating SGR stream | 1.11s | 1.11s | unchanged |
Add page-backed lookup and churn workloads for grapheme storage. The benchmark exercises the cell offset map together with the bitmap allocator, including configurable allocator load and multi-chunk grapheme values. Expose the grapheme chunk dimensions so the harness can size page storage to an exact working set without duplicating production constants.
Add direct coverage for reference counting, dead-ID reuse, addWithId, full-set behavior, and rehash requests. These paths previously depended on indirect screen and page tests. Add a test-only integrity verifier that cross-checks table buckets, probe distances, PSL statistics, liveness, and lookup reachability after each scenario.
Only reap a dead RefCountedSet occupant when the held item has an equal or longer probe sequence. Replacing a dead item with a shorter PSL lowers the value stored in that bucket and makes later live entries unreachable through the Robin Hood early-exit rule. Add a deterministic reproduction and a high-load randomized oracle. The failure could otherwise silently duplicate styles or hyperlinks under new IDs and increase page capacity pressure.
Add a replacement mode that keeps all but one requested style live while repeatedly adding and releasing unique styles. This isolates dense backward-shift deletion and ID reuse without triggering the all-dead bulk reset. Generate replacement styles on demand so long runs continue to exercise misses instead of cycling through the resident working set.
Bitmap allocators reserve complete 64-chunk words, but Page recorded the smaller requested byte counts. Doubling a small capacity could therefore recreate the same layout and make an immediate allocation retry fail again. Size the grapheme map directly from the rounded allocator raw slot capacity and record the actual grapheme and string byte capacities. Page layout is now idempotent and each capacity increase advances to larger storage.
The grapheme map previously allowed its open-addressed table to fill completely. Inserts and misses therefore degraded into full-table scans as grapheme storage approached capacity. Limit usable slots to eighty percent of the raw table. Reaching the limit grows the page before probes become pathological without doubling grapheme map metadata on pages that contain no graphemes. The benchmark caps its resident set at the same production limit. Exact row compaction now derives its minimum byte request from the same allocator-to-map layout helper used by Page.layout. This prevents layout policy changes from under-sizing a compacted map. A sweep over 1 through 4,096 entries pins the relationship. ReleaseFast medians use eleven sequential runs after two warmups and report user CPU time. Rows above the 80% boundary compare the same resident entry count before and after the new policy grows the page. | pre-growth occupancy | before | after growth | change | |---:|---:|---:|---:| | 80% | 79.622 ms | 78.454 ms | -1.5% | | 90% | 155.778 ms | 26.572 ms | -82.9% | | 94% | 216.450 ms | 15.096 ms | -93.0% | | 100% | 688.143 ms | 5.754 ms | -99.2% | Real VT medians use seven sequential runs after two warmups. | workload | before | after | change | |---|---:|---:|---:| | ASCII, 219 MiB | 225.927 ms | 227.741 ms | +0.8% | | grapheme, 1 extra codepoint | 361.388 ms | 368.598 ms | +2.0% | | grapheme, 5 extra codepoints | 449.764 ms | 456.409 ms | +1.5% | | grapheme, 32 extra codepoints | 2.024 s | 2.024 s | +0.0% | A standard production page retains 512 raw map slots, a 10,768-byte map buffer, and a 409,600-byte total layout. Its usable map limit moves from 512 to 409 entries, so capacity grows before the final allocator chunks are consumed.
Finding an in-word run of n free chunks previously intersected the bitmap with every shift from one through n minus one. Large grapheme allocations therefore performed up to 63 dependent shift-and-mask steps for every word examined. Fold each intermediate result by its represented run length instead. This preserves the cross-word first-fit fallback while reducing an in-word search to logarithmic steps. ReleaseFast medians use nine sequential runs after two warmups and report user CPU time. Grapheme churn uses 256 cells at the production 80% map limit; loop counts are calibrated per row. | extra codepoints | chunks | loops | before | after | change | |---:|---:|---:|---:|---:|---:| | 1 | 1 | 65,535 | 289.506 ms | 289.121 ms | -0.1% | | 5 | 2 | 65,535 | 210.946 ms | 188.005 ms | -10.9% | | 13 | 4 | 50,000 | 234.318 ms | 192.031 ms | -18.0% | | 29 | 8 | 25,000 | 207.009 ms | 142.711 ms | -31.1% | | 125 | 32 | 3,000 | 157.674 ms | 71.398 ms | -54.7% | | 253 | 64 | 800 | 165.088 ms | 43.548 ms | -73.6% | Real VT stream medians use seven sequential runs after two warmups. | workload | before | after | change | |---|---:|---:|---:| | VT grapheme, 5 extra codepoints | 448.057 ms | 429.095 ms | -4.2% | | VT grapheme, 32 extra codepoints | 2.005 s | 1.514 s | -24.5% |
Every allocation previously restarted its free-run search at the first bitmap. Dense grapheme and string allocators repeatedly rescanned the same full leading words, making churn grow with the size of occupied storage. Track the first bitmap that may contain free chunks. Allocation advances the hint over full words and free lowers it, preserving exact first-fit behavior. A 32-bit offset index occupies existing padding, so the allocator remains 24 bytes on 64-bit targets. ReleaseFast medians use seven sequential runs after two warmups and report user CPU time. Synthetic grapheme churn uses the production 80% map limit unless noted. | entries | chunks | occupancy | loops | before | after | change | |---:|---:|---:|---:|---:|---:|---:| | 4,096 | 1 | 80% | 2,560 | 410.475 ms | 311.512 ms | -24.1% | | 16,384 | 1 | 80% | 640 | 806.146 ms | 499.491 ms | -38.0% | | 32,768 | 1 | 80% | 256 | 895.591 ms | 477.662 ms | -46.7% | | 16,384 | 2 | 80% | 640 | 1.298 s | 171.429 ms | -86.8% | | 16,384 | 1 | 25% | 4,096 | 425.012 ms | 239.661 ms | -43.6% | The same parent and child binaries were measured through the real VT stream benchmark using fixed grapheme corpora. | VT workload | before | after | change | |---|---:|---:|---:| | grapheme, 1 extra codepoint | 365.210 ms | 238.915 ms | -34.6% | | grapheme, 5 extra codepoints | 430.772 ms | 301.518 ms | -30.0% | | grapheme, 32 extra codepoints | 1.517 s | 314.785 ms | -79.2% |
The offset audit added page-backed style-set and grapheme-map workloads to measure candidate changes. They are useful for investigation but broaden the benchmark CLI beyond the final production changes. Remove the workload implementations and registrations for now. Performance results remain in the individual commit messages, so the harnesses can be restored later if needed.
Contributor
Author
|
Btw waiting to split this PR up until Zig 0.16 work gets merged (if its within the next week or so). No pressure to vancleuver there I just know he's close and don't want to give him more work. 😄 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This audits the offset data structures: hash maps, bitmap allocators, and ref-counted sets and fixes some edge cases and improves performance for specific use cases. Grapheme-heavy VT streams improve by 29% to 65%, SGR-heavy streams improve by 2% to 8%, and normal ASCII output remains neutral.
Each commit has standalone information about a single change.
Changes overview
Correctness
Fix bitmap allocation boundaries. Small runs may cross bitmap words, large searches stop at the final word, and randomized oracle coverage verifies both placement and mutation.
Preserve ref-counted set invariants. Capacity math is exact at the largest ID, dead entries use an unambiguous sentinel, and reaping a dead Robin Hood occupant can no longer make later live entries unreachable.
Make page capacity growth and compaction exact. Page capacity records rounded bitmap storage, every growth advances the layout, and compacted grapheme pages retain enough map headroom to clone boundary entries.
Reuse live explicit hyperlinks at full string capacity. A matching resident OSC 8 entry is found before allocating transient copies, avoiding an unnecessary page growth when string storage is already full.
Performance
Specialize page offset hashing. Cell offsets use an integer mixer instead of byte-oriented Wyhash.
Trim ref-counted set hot paths. Lookups retain item pointers, insertion reuses hashes, all-dead tables reset in bulk, and probe loops advance their current bucket directly. Style hashing uses the shorter SplitMix64 finalizer.
Bound grapheme map probes. Grapheme maps retain 20% headroom and grow the page before inserts or misses degrade into full-table scans. The standard page keeps its existing 512 raw slots rather than doubling map metadata.
Accelerate bitmap searches. In-word runs are found in logarithmic steps, while a scan hint skips leading words known to be full without changing first-fit behavior or allocator size.
Else
Benchmarks
Microbenchmarks:
LLM Notes
Assisted by GPT-5.6 and Fable: I guided both to do this work in parallel, then had them each review each others work, then merged each others work in parallel in new context windows, then used GPT-5.6 to create a final single branch, then finally used Fable for final judging. At that point, I hand-reviewed everything, removed a couple changes myself, redid things for style, then hand-wrote this PR message.