Commit a95e814
fix pthash memory estimate (#97)
* Account for input hash vector in construction memory estimate
estimate_num_bytes_for_construction returned max(map_bytes, search_bytes),
but the input hash vector lives across both phases. The partitioned
external builder pushes the per-partition hashes (num_keys *
sizeof(hash_type)) into in_memory_partitions and only adds the build
estimate to its running 'bytes' counter, so the bytes < config.ram gate
underestimated true residency by num_keys * sizeof(hash_type) per
partition.
For hash128 (16B) plus the search-phase peak (~25B/key), peak per-
partition residency is ~41B/key while the gate believed ~25B/key, so
config.ram=2GiB actually allowed ~3.2GB resident. Adding the input
hashes outside the max makes config.ram match observed RSS.
* Drop the redundant in-search hashes term and the comment
The previous commit added the input hashes outside the max with the
correct sizeof(hash_type), so the pre-existing num_keys * sizeof(uint64_t)
term inside num_bytes_for_search was double-counting.
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent e04a192 commit a95e814
1 file changed
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
211 | 211 | | |
212 | 212 | | |
213 | 213 | | |
214 | | - | |
215 | 214 | | |
216 | 215 | | |
217 | | - | |
| 216 | + | |
| 217 | + | |
218 | 218 | | |
219 | 219 | | |
220 | 220 | | |
| |||
0 commit comments