Skip to content

Commit 0f9c9c8

Browse files
aneubeckaneubeck
authored
Merge pull request #118 from github/aneubeck/hashbrown
replace hashbrown with std::hashmap
2 parents f2357ed + 93f929f commit 0f9c9c8

4 files changed

Lines changed: 91 additions & 99 deletions

File tree

‎crates/hash-sorted-map/OPTIMIZATIONS.md‎

Lines changed: 50 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,10 @@ and an **optimized growth strategy**. It is generic over key type, value type,
88
and hash builder.
99

1010
This document analyzes the design trade-offs versus
11-
[hashbrown](https://github.com/rust-lang/hashbrown) and records the
12-
experimental results that guided the current design.
11+
[hashbrown](https://github.com/rust-lang/hashbrown) — the Swiss-table
12+
implementation that backs `std::collections::HashMap` — and records the
13+
experimental results that guided the current design. The benchmark suite
14+
drives `std::collections::HashMap` directly with various explicit `BuildHasher` configurations.
1315

1416
---
1517

@@ -197,70 +199,69 @@ Hardware used for the current local snapshot:
197199

198200
### Insert (1000 trigrams, pre-sized)
199201

200-
| Implementation | Time (µs) | vs hashbrown |
201-
|----------------------|-----------|--------------|
202-
| FoldHashMap | 13.88 | −5% |
203-
| FxHashMap | 14.60 | ~0% |
204-
| hashbrown+Identity | 14.44 | baseline |
205-
| hashbrown::HashMap | 14.55 | +1% |
206-
| std::HashMap+FNV | 15.55 | +8% |
207-
| AHashMap | 15.59 | +8% |
208-
| **HashSortedMap** | **9.40** | **−35%** |
209-
| std::HashMap | 25.26 | +75% |
202+
| Implementation | Time (µs) | vs `std::HashMap+Identity` |
203+
|-----------------------------|-----------|----------------------------|
204+
| `std::HashMap+FoldHash` | 13.88 | −4% |
205+
| `FxHashMap` | 14.60 | +1% |
206+
| `std::HashMap+Identity` | 14.44 | baseline |
207+
| `std::HashMap+FNV` | 15.55 | +8% |
208+
| `std::HashMap+AHash` | 15.59 | +8% |
209+
| **`HashSortedMap`** | **9.40** | **−35%** |
210+
| `std::HashMap` (RandomState)| 25.26 | +75% |
210211

211212
### Reinsert (1000 trigrams, all keys exist)
212213

213-
| Implementation | Time (µs) |
214-
|----------------------|-----------|
215-
| **HashSortedMap** | **6.59** |
216-
| hashbrown+Identity | 6.95 |
214+
| Implementation | Time (µs) |
215+
|-------------------------|-----------|
216+
| **`HashSortedMap`** | **6.59** |
217+
| `std::HashMap+Identity` | 6.95 |
217218

218219
### Growth (128 → 1000 trigrams, 3 resize rounds)
219220

220-
| Implementation | Time (µs) |
221-
|----------------------|-----------|
222-
| hashbrown+Identity | 26.66 |
223-
| **HashSortedMap** | **27.50** |
221+
| Implementation | Time (µs) |
222+
|-------------------------|-----------|
223+
| `std::HashMap+Identity` | 26.66 |
224+
| **`HashSortedMap`** | **27.50** |
224225

225226
### Count (4000 trigrams, mixed insert/update)
226227

227-
| Implementation | Time (µs) |
228-
|----------------------------------|-----------|
229-
| hashbrown+Identity entry() | 15.49 |
230-
| **HashSortedMap get_or_default** | **15.88** |
231-
| **HashSortedMap entry().or_default()** | **16.15** |
228+
| Implementation | Time (µs) |
229+
|-----------------------------------------|-----------|
230+
| `std::HashMap+Identity` `entry()` | 15.49 |
231+
| **`HashSortedMap get_or_default`** | **15.88** |
232+
| **`HashSortedMap entry().or_default()`**| **16.15** |
232233

233234
### Iteration (1000 trigrams)
234235

235-
| Implementation | Time (µs) |
236-
|-------------------------------|-----------|
237-
| **HashSortedMap iter()** | **3.02** |
238-
| hashbrown+Identity iter() | 3.04 |
239-
| **HashSortedMap into_iter()** | **3.03** |
240-
| hashbrown+Identity into_iter()| 3.56 |
236+
| Implementation | Time (µs) |
237+
|--------------------------------------|-----------|
238+
| **`HashSortedMap iter()`** | **3.02** |
239+
| `std::HashMap+Identity` `iter()` | 3.04 |
240+
| **`HashSortedMap into_iter()`** | **3.03** |
241+
| `std::HashMap+Identity` `into_iter()`| 3.56 |
241242

242243
### Sort (100K trigrams)
243244

244-
| Implementation | Time (ms) |
245-
|-----------------------------|-----------|
246-
| **HashSortedMap sort_by_hash** | **1.66** |
247-
| Vec::sort_unstable | 2.20 |
245+
| Implementation | Time (ms) |
246+
|--------------------------------|-----------|
247+
| **`HashSortedMap sort_by_hash`** | **1.66** |
248+
| `Vec::sort_unstable` | 2.20 |
248249

249250
### Merge (100 maps × 100K keys each → sorted output)
250251

251-
| Implementation | Time (ms) | vs HSM merge+sort |
252-
|-----------------------------------|-----------|--------------------|
253-
| hashbrown merge presized | 160.79 | +6% |
254-
| **HashSortedMap merge presized** | **117.01**| **−23%** |
255-
| **HashSortedMap merge (no sort)** | **141.57**| **−7%** |
256-
| hashbrown merge | 163.59 | +7% |
257-
| **HashSortedMap merge + sort** | **152.34**| **baseline** |
258-
| hashbrown merge + Vec sort | 193.37 | +27% |
259-
| k-way merge sorted vecs | 445 | +192% |
252+
| Implementation | Time (ms) | vs HSM merge+sort |
253+
|-----------------------------------------------|-----------|--------------------|
254+
| `std::HashMap+Identity` merge presized | 160.79 | +6% |
255+
| **`HashSortedMap` merge presized** | **117.01**| **−23%** |
256+
| **`HashSortedMap` merge (no sort)** | **141.57**| **−7%** |
257+
| `std::HashMap+Identity` merge | 163.59 | +7% |
258+
| **`HashSortedMap` merge + sort** | **152.34**| **baseline** |
259+
| `std::HashMap+Identity` merge + Vec sort | 193.37 | +27% |
260+
| k-way merge sorted vecs | 445 | +192% |
260261

261262
**Key takeaways:**
262-
- Pre-sized insert is **~35% faster** than hashbrown+Identity
263-
- Reinsert and iter paths are now close to parity with hashbrown+Identity
264-
- Growth path is currently **~3% slower** than hashbrown+Identity
265-
- sort_by_hash is **~24% faster** than Vec::sort_unstable
266-
- merge + sort is **~21% faster** than hashbrown merge + Vec sort
263+
- Pre-sized insert is **~35% faster** than `std::HashMap+Identity`
264+
- Reinsert and iter paths are now close to parity with `std::HashMap+Identity`
265+
- Growth path is currently **~3% slower** than `std::HashMap+Identity`
266+
- sort_by_hash is **~24% faster** than `Vec::sort_unstable`
267+
- merge + sort is **~21% faster** than `std::HashMap+Identity` merge + Vec sort

‎crates/hash-sorted-map/README.md‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -53,21 +53,23 @@ Hardware used for this snapshot:
5353
- CPU frequency range: 800 MHz to 2800 MHz
5454
- Memory: 7.8 GiB RAM
5555

56-
| Scenario | HashSortedMap | Comparison | Result |
57-
| :------------------------------------------- | ------------: | :------------------------------------- | :---------- |
58-
| Insert 1000 trigrams (pre-sized) | 9.40 µs | hashbrown::HashMap: 14.55 µs | ~35% faster |
59-
| Grow from capacity 128 | 27.50 µs | hashbrown+Identity: 26.66 µs | ~3% slower |
60-
| Count 4000 trigrams (`entry().or_default()`) | 16.15 µs | hashbrown+Identity `entry()`: 15.49 µs | ~4% slower |
61-
| Iterate 1000 trigrams (`iter()`) | 3.02 µs | hashbrown+Identity `iter()`: 3.04 µs | ~1% faster |
62-
| Sort 100000 trigrams by hash | 1.66 ms | `Vec::sort_unstable`: 2.20 ms | ~24% faster |
63-
| Merge 100 sorted maps + final sort | 152.34 ms | hashbrown merge + vec sort: 193.37 ms | ~21% faster |
56+
| Scenario | HashSortedMap | Comparison | Result |
57+
| :------------------------------------------- | ------------: | :---------------------------------------- | :---------- |
58+
| Insert 1000 trigrams (pre-sized) | 9.40 µs | `std::HashMap+FoldHash`: 14.55 µs | ~35% faster |
59+
| Grow from capacity 128 | 27.50 µs | `std::HashMap+Identity`: 26.66 µs | ~3% slower |
60+
| Count 4000 trigrams (`entry().or_default()`) | 16.15 µs | `std::HashMap+Identity` `entry()`: 15.49 µs | ~4% slower |
61+
| Iterate 1000 trigrams (`iter()`) | 3.02 µs | `std::HashMap+Identity` `iter()`: 3.04 µs | ~1% faster |
62+
| Sort 100000 trigrams by hash | 1.66 ms | `Vec::sort_unstable`: 2.20 ms | ~24% faster |
63+
| Merge 100 sorted maps + final sort | 152.34 ms | `std::HashMap+Identity` merge + vec sort: 193.37 ms | ~21% faster |
64+
65+
> Note: `std::collections::HashMap` is `hashbrown` under the hood, so the benchmark drives `std::collections::HashMap` directly with the same custom `BuildHasher`s that were previously passed to `hashbrown`.
6466
6567
Key takeaways:
6668

6769
- Pre-sized inserts, sorting, and merge+sort remain the strongest paths.
68-
- Iteration is now roughly on par with `hashbrown+Identity`.
70+
- Iteration is now roughly on par with `std::HashMap+Identity`.
6971
- Growth and count/update workloads are currently slightly slower than
70-
`hashbrown+Identity` in this run.
72+
`std::HashMap+Identity` in this run.
7173

7274
## Running
7375

‎crates/hash-sorted-map/benchmarks/Cargo.toml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@ criterion = "0.8"
1818
rand = "0.10"
1919
rustc-hash = "2"
2020
ahash = "0.8"
21-
hashbrown = "0.15"
2221
foldhash = "0.1"
2322
fnv = "1"
2423
itertools = "0.14"

‎crates/hash-sorted-map/benchmarks/performance.rs‎

Lines changed: 29 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,12 @@ fn bench_insert(c: &mut Criterion) {
2626
);
2727
});
2828

29-
group.bench_function("hashbrown::HashMap", |b| {
29+
group.bench_function("std::HashMap+FoldHash", |b| {
3030
b.iter_batched(
31-
|| hashbrown::HashMap::with_capacity(trigrams.len()),
31+
|| std::collections::HashMap::<u32, usize, foldhash::fast::FixedState>::with_capacity_and_hasher(
32+
trigrams.len(),
33+
foldhash::fast::FixedState::default(),
34+
),
3235
|mut map| {
3336
for (i, &key) in trigrams.iter().enumerate() {
3437
map.insert(key, i);
@@ -52,24 +55,11 @@ fn bench_insert(c: &mut Criterion) {
5255
);
5356
});
5457

55-
group.bench_function("AHashMap", |b| {
58+
group.bench_function("std::HashMap+AHash", |b| {
5659
b.iter_batched(
57-
|| ahash::AHashMap::with_capacity(trigrams.len()),
58-
|mut map| {
59-
for (i, &key) in trigrams.iter().enumerate() {
60-
map.insert(key, i);
61-
}
62-
map
63-
},
64-
BatchSize::SmallInput,
65-
);
66-
});
67-
68-
group.bench_function("FoldHashMap", |b| {
69-
b.iter_batched(
70-
|| hashbrown::HashMap::<u32, usize, foldhash::fast::FixedState>::with_capacity_and_hasher(
60+
|| std::collections::HashMap::<u32, usize, ahash::RandomState>::with_capacity_and_hasher(
7161
trigrams.len(),
72-
foldhash::fast::FixedState::default(),
62+
ahash::RandomState::default(),
7363
),
7464
|mut map| {
7565
for (i, &key) in trigrams.iter().enumerate() {
@@ -99,10 +89,10 @@ fn bench_insert(c: &mut Criterion) {
9989
);
10090
});
10191

102-
group.bench_function("hashbrown+Identity", |b| {
92+
group.bench_function("std::HashMap+Identity", |b| {
10393
b.iter_batched(
10494
|| {
105-
hashbrown::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
95+
std::collections::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
10696
trigrams.len(),
10797
Default::default(),
10898
)
@@ -142,11 +132,11 @@ fn bench_reinsert(c: &mut Criterion) {
142132
let trigrams = trigrams();
143133
let mut group = c.benchmark_group("reinsert_1000_trigrams");
144134

145-
group.bench_function("hashbrown+Identity", |b| {
135+
group.bench_function("std::HashMap+Identity", |b| {
146136
b.iter_batched(
147137
|| {
148138
let mut map =
149-
hashbrown::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
139+
std::collections::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
150140
trigrams.len(),
151141
Default::default(),
152142
);
@@ -194,10 +184,10 @@ fn bench_grow(c: &mut Criterion) {
194184
let trigrams = trigrams();
195185
let mut group = c.benchmark_group("grow_from_128_insert_1000_trigrams");
196186

197-
group.bench_function("hashbrown+Identity", |b| {
187+
group.bench_function("std::HashMap+Identity", |b| {
198188
b.iter_batched(
199189
|| {
200-
hashbrown::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
190+
std::collections::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
201191
128,
202192
Default::default(),
203193
)
@@ -237,10 +227,10 @@ fn bench_count(c: &mut Criterion) {
237227

238228
let mut group = c.benchmark_group("count_4000_trigrams_get_or_default");
239229

240-
group.bench_function("hashbrown+Identity entry()", |b| {
230+
group.bench_function("std::HashMap+Identity entry()", |b| {
241231
b.iter_batched(
242232
|| {
243-
hashbrown::HashMap::<u32, u32, IdentityBuildHasher>::with_capacity_and_hasher(
233+
std::collections::HashMap::<u32, u32, IdentityBuildHasher>::with_capacity_and_hasher(
244234
trigrams.len(),
245235
Default::default(),
246236
)
@@ -299,11 +289,11 @@ fn bench_iter(c: &mut Criterion) {
299289

300290
let mut group = c.benchmark_group("iter_1000_trigrams");
301291

302-
group.bench_function("hashbrown+Identity iter()", |b| {
292+
group.bench_function("std::HashMap+Identity iter()", |b| {
303293
b.iter_batched(
304294
|| {
305295
let mut map =
306-
hashbrown::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
296+
std::collections::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
307297
trigrams.len(),
308298
Default::default(),
309299
);
@@ -346,11 +336,11 @@ fn bench_iter(c: &mut Criterion) {
346336
);
347337
});
348338

349-
group.bench_function("hashbrown+Identity into_iter()", |b| {
339+
group.bench_function("std::HashMap+Identity into_iter()", |b| {
350340
b.iter_batched(
351341
|| {
352342
let mut map =
353-
hashbrown::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
343+
std::collections::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
354344
trigrams.len(),
355345
Default::default(),
356346
);
@@ -507,10 +497,10 @@ fn bench_merge_sort(c: &mut Criterion) {
507497
});
508498
});
509499

510-
// ── 3. hashbrown HashMap merge, then sort into Vec ──────────────
511-
group.bench_function("hashbrown merge + Vec sort", |b| {
500+
// ── 3. std::HashMap merge, then sort into Vec ──────────────────
501+
group.bench_function("std::HashMap+Identity merge + Vec sort", |b| {
512502
b.iter(|| {
513-
let mut map = hashbrown::HashMap::<u32, u32, IdentityBuildHasher>::with_hasher(
503+
let mut map = std::collections::HashMap::<u32, u32, IdentityBuildHasher>::with_hasher(
514504
IdentityBuildHasher::default(),
515505
);
516506
for container in &hash_maps {
@@ -528,10 +518,10 @@ fn bench_merge_sort(c: &mut Criterion) {
528518
});
529519
});
530520

531-
// ── 4. hashbrown HashMap merge only (no sort) ───────────────────
532-
group.bench_function("hashbrown merge", |b| {
521+
// ── 4. std::HashMap merge only (no sort) ───────────────────────
522+
group.bench_function("std::HashMap+Identity merge", |b| {
533523
b.iter(|| {
534-
let mut map = hashbrown::HashMap::<u32, u32, IdentityBuildHasher>::with_hasher(
524+
let mut map = std::collections::HashMap::<u32, u32, IdentityBuildHasher>::with_hasher(
535525
IdentityBuildHasher::default(),
536526
);
537527
for container in &hash_maps {
@@ -557,11 +547,11 @@ fn bench_merge_sort(c: &mut Criterion) {
557547
});
558548
});
559549

560-
// ── 6. hashbrown presized merge only ────────────────────────────
561-
group.bench_function("hashbrown merge presized", |b| {
550+
// ── 6. std::HashMap presized merge only ────────────────────────
551+
group.bench_function("std::HashMap+Identity merge presized", |b| {
562552
b.iter(|| {
563553
let mut map =
564-
hashbrown::HashMap::<u32, u32, IdentityBuildHasher>::with_capacity_and_hasher(
554+
std::collections::HashMap::<u32, u32, IdentityBuildHasher>::with_capacity_and_hasher(
565555
1_000_000,
566556
IdentityBuildHasher::default(),
567557
);

0 commit comments

Comments
 (0)