Skip to content

Commit c744961

Browse files
MukjepScarletMukjepScarlet
committed
Add benchmark module
1 parent 84bc144 commit c744961

10 files changed

Lines changed: 456 additions & 67 deletions

File tree

‎README.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ This repository is a multi-module Gradle project:
66

77
- `fastutil4k-extensions-only`: generated and hand-written inline Kotlin extension APIs for fastutil and JDK collections.
88
- `fastutil4k-more-collections`: additional collection implementations built with fastutil data structures.
9+
- `benchmark`: JMH benchmarks for local performance measurement (not published).
910

1011
## Modules
1112

@@ -38,6 +39,15 @@ Provides higher-level collection utilities and data structures, including:
3839
`weightedFilterSortedBy*`, `weightedMinByOrNull*`, `weightedMaxByOrNull*`
3940
- stepped float/double ranges returning fastutil lists (`ClosedRange<Double>.step`, `ClosedRange<Float>.step`)
4041

42+
### `benchmark`
43+
44+
Independent JMH module for performance testing:
45+
46+
- core benchmarks for `Pool`, `LfuCache`, `WeightedSortedList`, `Ranges`
47+
- transform benchmark:
48+
`mapToArray { }.asList()` vs `collection.map` vs `sequence.map.toList` vs `stream.map.toList`
49+
- not part of publish artifacts
50+
4151
## Requirements
4252

4353
- JDK 8+ (toolchain target is Java 8)
@@ -49,6 +59,29 @@ Provides higher-level collection utilities and data structures, including:
4959
./gradlew clean build
5060
```
5161

62+
`benchmark` is intentionally skipped in normal `build/check/test` lifecycle.
63+
64+
## Run Benchmarks
65+
66+
```bash
67+
# Compile JMH sources
68+
./gradlew :benchmark:compileJmhKotlin
69+
70+
# Smoke run (single benchmark class, short run)
71+
./gradlew :benchmark:jmh --no-configuration-cache \
72+
-Pjmh.includes=MapTransformBenchmark \
73+
-Pjmh.warmupIterations=1 \
74+
-Pjmh.iterations=1 \
75+
-Pjmh.fork=1
76+
77+
# Full benchmark suite
78+
./gradlew :benchmark:jmh --no-configuration-cache
79+
```
80+
81+
JMH JSON output path:
82+
83+
- `benchmark/build/reports/jmh/results.json`
84+
5285
## Regenerate extension sources
5386

5487
Generated sources are produced by module tasks.

‎benchmark/README.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# benchmark
2+
3+
JMH benchmark module for `fastutil4k`.
4+
5+
This module is designed for local/manual performance evaluation:
6+
7+
- It is independent from publish artifacts.
8+
- It is not part of the default `build/check/test` lifecycle.
9+
10+
## Included benchmarks (first batch)
11+
12+
- `PoolBenchmark`
13+
- `LfuCacheBenchmark`
14+
- `WeightedSortedListBenchmark`
15+
- `RangesBenchmark`
16+
- `MapTransformBenchmark`
17+
18+
`MapTransformBenchmark` compares:
19+
20+
- `mapToArray { ... }.asList()`
21+
- `collection.map { ... }`
22+
- `sequence.map { ... }.toList()`
23+
- `stream.map { ... }.toList()`
24+
25+
## Run
26+
27+
```bash
28+
# Compile benchmark source set
29+
./gradlew :benchmark:compileJmhKotlin
30+
31+
# Smoke run with short params
32+
./gradlew :benchmark:jmh --no-configuration-cache \
33+
-Pjmh.includes=MapTransformBenchmark \
34+
-Pjmh.warmupIterations=1 \
35+
-Pjmh.iterations=1 \
36+
-Pjmh.fork=1
37+
38+
# Full suite
39+
./gradlew :benchmark:jmh --no-configuration-cache
40+
```
41+
42+
## Parameters
43+
44+
Optional command-line properties:
45+
46+
- `-Pjmh.includes=<regex>`
47+
- `-Pjmh.warmupIterations=<int>`
48+
- `-Pjmh.iterations=<int>`
49+
- `-Pjmh.fork=<int>`
50+
51+
## Output
52+
53+
- Console report from JMH
54+
- JSON report: `build/reports/jmh/results.json`
55+
56+
## Note
57+
58+
Current JMH plugin setup may conflict with Gradle configuration cache.
59+
Use `--no-configuration-cache` when running `:benchmark:jmh`.

‎benchmark/build.gradle.kts‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
plugins {
2+
kotlin("jvm")
3+
id("me.champeau.jmh") version "0.7.2"
4+
}
5+
6+
dependencies {
7+
implementation(project(":fastutil4k-extensions-only"))
8+
implementation(project(":fastutil4k-more-collections"))
9+
implementation(libs.fastutil)
10+
11+
jmh(project(":fastutil4k-extensions-only"))
12+
jmh(project(":fastutil4k-more-collections"))
13+
jmh(libs.fastutil)
14+
}
15+
16+
jmh {
17+
warmupIterations = (findProperty("jmh.warmupIterations") as String?)?.toInt() ?: 2
18+
iterations = (findProperty("jmh.iterations") as String?)?.toInt() ?: 3
19+
fork = (findProperty("jmh.fork") as String?)?.toInt() ?: 1
20+
resultFormat = "JSON"
21+
resultsFile.set(layout.buildDirectory.file("reports/jmh/results.json"))
22+
if (project.hasProperty("jmh.includes")) {
23+
includes = listOf(project.property("jmh.includes").toString())
24+
}
25+
}
26+
27+
// Keep benchmark execution explicit: `:benchmark:jmh`
28+
tasks.named("build") {
29+
enabled = false
30+
}
31+
tasks.named("check") {
32+
enabled = false
33+
}
34+
tasks.named("test") {
35+
enabled = false
36+
}

‎benchmark/src/jmh/kotlin/net/ccbluex/fastutil/LfuCacheBenchmark.kt‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
package net.ccbluex.fastutil
2+
3+
import org.openjdk.jmh.annotations.Benchmark
4+
import org.openjdk.jmh.annotations.BenchmarkMode
5+
import org.openjdk.jmh.annotations.Mode
6+
import org.openjdk.jmh.annotations.OutputTimeUnit
7+
import org.openjdk.jmh.annotations.Param
8+
import org.openjdk.jmh.annotations.Scope
9+
import org.openjdk.jmh.annotations.Setup
10+
import org.openjdk.jmh.annotations.State
11+
import org.openjdk.jmh.infra.Blackhole
12+
import java.util.concurrent.TimeUnit
13+
14+
@State(Scope.Thread)
15+
@BenchmarkMode(Mode.Throughput)
16+
@OutputTimeUnit(TimeUnit.MILLISECONDS)
17+
open class LfuCacheBenchmark {
18+
@Param("1000", "100000", "1000000")
19+
var size: Int = 0
20+
21+
private lateinit var cache: LfuCache<String, Int>
22+
private var readIndex: Int = 0
23+
private var writeIndex: Int = 0
24+
25+
@Setup
26+
fun setup() {
27+
val capacity = (size / 2).coerceAtLeast(16)
28+
cache = LfuCache(capacity)
29+
repeat(capacity) { i ->
30+
cache["seed-$i"] = i
31+
}
32+
readIndex = 0
33+
writeIndex = capacity
34+
}
35+
36+
@Benchmark
37+
fun getHotPath(bh: Blackhole) {
38+
val key = "seed-${readIndex and 127}"
39+
readIndex++
40+
bh.consume(cache[key])
41+
}
42+
43+
@Benchmark
44+
fun putWithEviction(bh: Blackhole) {
45+
val key = "new-$writeIndex"
46+
cache[key] = writeIndex
47+
writeIndex++
48+
bh.consume(cache.size)
49+
}
50+
}

‎benchmark/src/jmh/kotlin/net/ccbluex/fastutil/MapTransformBenchmark.kt‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
package net.ccbluex.fastutil
2+
3+
import org.openjdk.jmh.annotations.Benchmark
4+
import org.openjdk.jmh.annotations.BenchmarkMode
5+
import org.openjdk.jmh.annotations.Mode
6+
import org.openjdk.jmh.annotations.OutputTimeUnit
7+
import org.openjdk.jmh.annotations.Param
8+
import org.openjdk.jmh.annotations.Scope
9+
import org.openjdk.jmh.annotations.Setup
10+
import org.openjdk.jmh.annotations.State
11+
import org.openjdk.jmh.infra.Blackhole
12+
import java.util.ArrayList
13+
import java.util.concurrent.TimeUnit
14+
import kotlin.streams.toList
15+
16+
@State(Scope.Thread)
17+
@BenchmarkMode(Mode.Throughput)
18+
@OutputTimeUnit(TimeUnit.MILLISECONDS)
19+
open class MapTransformBenchmark {
20+
@Param("1000", "100000", "1000000")
21+
var size: Int = 0
22+
23+
private lateinit var input: ArrayList<Int>
24+
25+
@Setup
26+
fun setup() {
27+
input = ArrayList(size)
28+
repeat(size) { input.add(it) }
29+
}
30+
31+
@Benchmark
32+
fun mapToArrayAsList(bh: Blackhole) {
33+
bh.consume(input.mapToArray { it + 1 }.asList())
34+
}
35+
36+
@Benchmark
37+
fun collectionMap(bh: Blackhole) {
38+
bh.consume(input.map { it + 1 })
39+
}
40+
41+
@Benchmark
42+
fun sequenceMapToList(bh: Blackhole) {
43+
bh.consume(input.asSequence().map { it + 1 }.toList())
44+
}
45+
46+
@Benchmark
47+
fun streamMapToList(bh: Blackhole) {
48+
bh.consume(input.stream().map { it + 1 }.toList())
49+
}
50+
}

‎benchmark/src/jmh/kotlin/net/ccbluex/fastutil/PoolBenchmark.kt‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
package net.ccbluex.fastutil
2+
3+
import org.openjdk.jmh.annotations.Benchmark
4+
import org.openjdk.jmh.annotations.BenchmarkMode
5+
import org.openjdk.jmh.annotations.Mode
6+
import org.openjdk.jmh.annotations.OutputTimeUnit
7+
import org.openjdk.jmh.annotations.Param
8+
import org.openjdk.jmh.annotations.Scope
9+
import org.openjdk.jmh.annotations.Setup
10+
import org.openjdk.jmh.annotations.State
11+
import org.openjdk.jmh.infra.Blackhole
12+
import java.util.ArrayList
13+
import java.util.concurrent.TimeUnit
14+
15+
@State(Scope.Thread)
16+
@BenchmarkMode(Mode.Throughput)
17+
@OutputTimeUnit(TimeUnit.MILLISECONDS)
18+
open class PoolBenchmark {
19+
@Param("1000", "100000", "1000000")
20+
var size: Int = 0
21+
22+
private lateinit var pool: Pool<StringBuilder>
23+
private lateinit var sink: MutableList<StringBuilder>
24+
25+
@Setup
26+
fun setup() {
27+
pool = Pool({ StringBuilder() }, StringBuilder::clear)
28+
sink = ArrayList(64)
29+
30+
// Pre-warm pool proportional to dataset scale.
31+
val warmCount = when {
32+
size <= 1_000 -> 16
33+
size <= 100_000 -> 64
34+
else -> 256
35+
}
36+
repeat(warmCount) {
37+
pool.recycle(StringBuilder())
38+
}
39+
}
40+
41+
@Benchmark
42+
fun borrowRecycle(bh: Blackhole) {
43+
val sb = pool.borrow()
44+
sb.append('x')
45+
pool.recycle(sb)
46+
bh.consume(sb.length)
47+
}
48+
49+
@Benchmark
50+
fun borrowIntoAndRecycleAll(bh: Blackhole) {
51+
sink.clear()
52+
pool.borrowInto(sink, 32)
53+
var total = 0
54+
for (sb in sink) {
55+
sb.append('x')
56+
total += sb.length
57+
}
58+
pool.recycleAll(sink)
59+
bh.consume(total)
60+
}
61+
}

‎benchmark/src/jmh/kotlin/net/ccbluex/fastutil/RangesBenchmark.kt‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
package net.ccbluex.fastutil
2+
3+
import org.openjdk.jmh.annotations.Benchmark
4+
import org.openjdk.jmh.annotations.BenchmarkMode
5+
import org.openjdk.jmh.annotations.Mode
6+
import org.openjdk.jmh.annotations.OutputTimeUnit
7+
import org.openjdk.jmh.annotations.Param
8+
import org.openjdk.jmh.annotations.Scope
9+
import org.openjdk.jmh.annotations.State
10+
import org.openjdk.jmh.infra.Blackhole
11+
import java.util.concurrent.TimeUnit
12+
13+
@State(Scope.Thread)
14+
@BenchmarkMode(Mode.Throughput)
15+
@OutputTimeUnit(TimeUnit.MILLISECONDS)
16+
open class RangesBenchmark {
17+
@Param("1000", "100000", "1000000")
18+
var size: Int = 0
19+
20+
@Benchmark
21+
fun doubleStepCreateAndIterate(bh: Blackhole) {
22+
val list = (0.0..size.toDouble()).step(1.0)
23+
var sum = 0.0
24+
for (i in 0 until list.size) {
25+
sum += list.getDouble(i)
26+
}
27+
bh.consume(sum)
28+
}
29+
30+
@Benchmark
31+
fun floatStepCreateAndIterate(bh: Blackhole) {
32+
val list = (0f..size.toFloat()).step(1f)
33+
var sum = 0f
34+
for (i in 0 until list.size) {
35+
sum += list.getFloat(i)
36+
}
37+
bh.consume(sum)
38+
}
39+
}

0 commit comments

Comments
 (0)