Commit 4457a4a
authored
perf(alp): avoid per-combination allocations when searching exponents (#8565)
## Summary
`find_best_exponents` ran a full `encode` for every `(e, f)` candidate
just to size the result, allocating the encoded, patch and chunk-offset
buffers across the whole `e`/`f` search space. It now estimates the size
in a single allocation-free pass via
`estimate_encoded_size_for_exponents`, which reproduces the existing
`estimate_encoded_size` exactly, so the chosen exponents (and therefore
the compressed output) are bit-for-bit unchanged.
This is the throughput half of #919; the two-level stratified sampling /
top-N reuse the issue describes would change which exponents are picked
and depends on per-column scheme state (#8434), so I left it out here.
## Benchmark
`encodings/alp/benches/alp_compress.rs`, compress medians (type, length,
patch-fraction, valid-fraction):
| case | before | after | speedup |
| --- | --- | --- | --- |
| f64 (1000, 0.0, 1.0) | 39.1 us | 14.0 us | 2.8x |
| f64 (1000, 0.1, 0.25) | 45.0 us | 16.5 us | 2.7x |
| f32 (1000, 0.0, 1.0) | 14.6 us | 5.8 us | 2.5x |
| f64 (10000, 0.0, 1.0) | 53.1 us | 28.6 us | 1.9x |
| f64 (10000, 0.1, 0.25) | 73.7 us | 49.4 us | 1.5x |
## Testing
`cargo nextest run -p vortex-alp -p vortex-btrblocks` passes, including
a new equivalence test (`estimate_for_exponents_matches_full_encode`)
that asserts the allocation-free estimate equals a full `encode` +
`estimate_encoded_size` for every candidate exponent across kept,
patched and all-patched inputs. `fmt` + `clippy --all-targets
--all-features` + `cargo doc` clean.
---
I'm Korean, so sorry if any wording reads a little awkward.
---------
Signed-off-by: Han Damin <miniex@daminstudio.net>1 parent 0726fb9 commit 4457a4a
1 file changed
Lines changed: 98 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
| 34 | + | |
33 | 35 | | |
34 | 36 | | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
35 | 76 | | |
36 | 77 | | |
37 | 78 | | |
| |||
88 | 129 | | |
89 | 130 | | |
90 | 131 | | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
91 | 139 | | |
92 | 140 | | |
93 | 141 | | |
| |||
120 | 168 | | |
121 | 169 | | |
122 | 170 | | |
| 171 | + | |
123 | 172 | | |
124 | 173 | | |
125 | 174 | | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
| 175 | + | |
| 176 | + | |
132 | 177 | | |
133 | 178 | | |
134 | | - | |
| 179 | + | |
135 | 180 | | |
136 | | - | |
| 181 | + | |
137 | 182 | | |
138 | 183 | | |
139 | 184 | | |
140 | 185 | | |
141 | 186 | | |
142 | 187 | | |
143 | 188 | | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
144 | 234 | | |
145 | 235 | | |
146 | 236 | | |
| |||
0 commit comments