perf(iterator): pre-size iters slice in Txn.NewIterator#2290
Open
shaunpatterson wants to merge 1 commit into
Open
perf(iterator): pre-size iters slice in Txn.NewIterator#2290shaunpatterson wants to merge 1 commit into
shaunpatterson wants to merge 1 commit into
Conversation
The iters slice was previously grown via append-from-nil, hitting growslice on every memtable iterator appended. In dgraph's posting list rollup with 6 memtables this hits 3 growslice events (0→1→2→ 4→8) just to fit the deterministic memtable-pass before appendIterators adds the level iterators. Computing the cap up front (len(tables) plus 1 if a pending-writes iterator exists, 0 otherwise) collapses those growslices into a single make() of exactly the needed size. Level iterators added by appendIterators may still trigger a growslice on first append past the initial cap; that's preserved as-is to avoid speculatively over-allocating for callers that don't have any level iterators. Saves 1 alloc and 16 B per iterator construction on BenchmarkRollupKeyIterator: before: 817 B/op, 15 allocs/op after: 801 B/op, 14 allocs/op In a 6-memtable workload the savings scale up: original code hits 3 growslice paths to fit just the memtables; pre-sizing skips all of them. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
itersslice inTxn.NewIteratorwas previously grown via append-from-nil, hittinggrowsliceon every memtable iterator appended. In dgraph's posting list rollup with 6 memtables this hits 3growsliceevents (0→1→2→4→8) just to fit the deterministic memtable-pass beforeappendIteratorsadds the level iterators.Computing the cap up front (
len(tables)plus 1 if a pending-writes iterator exists, 0 otherwise) collapses thosegrowslices into a singlemake()of exactly the needed size.Level iterators added by
appendIteratorsmay still trigger agrowsliceon first append past the initial cap; that's preserved as-is to avoid speculatively over-allocating for callers that don't have any level iterators.Benchmark — BenchmarkRollupKeyIterator
In a 6-memtable workload the savings scale up: original code hits 3
growslicepaths to fit just the memtables; pre-sizing skips all of them.Test plan
go test ./...passes🤖 Generated with Claude Code