Skip to content

Commit b39eb0d

Browse files
TKorrTKorr
committed
Enhance GitHub Actions workflow for benchmark reporting and documentation
- Updated the release workflow to use the latest checkout action and improved the benchmark summary generation process. - Added a new configuration file for Jekyll documentation, establishing a foundation for the CacheKit documentation site. - Enhanced the benchmarks.md file to include a section for the latest benchmark run, facilitating better tracking of performance metrics.
1 parent e85e3d0 commit b39eb0d

3 files changed

Lines changed: 137 additions & 6 deletions

File tree

‎.github/workflows/release.yml‎

Lines changed: 106 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,25 +18,125 @@ jobs:
1818
release:
1919
name: GitHub Release
2020
runs-on: ubuntu-latest
21+
env:
22+
RELEASE_TAG: ${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref_name }}
2123
steps:
22-
- uses: actions/checkout@v4
24+
- uses: actions/checkout@v6
25+
with:
26+
ref: ${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref }}
2327
- uses: dtolnay/rust-toolchain@stable
2428
- uses: Swatinem/rust-cache@v2
2529
- name: Run benchmarks
2630
run: cargo bench --no-fail-fast
31+
- name: Generate benchmark summary
32+
run: |
33+
set -euo pipefail
34+
criterion_dir="target/criterion"
35+
36+
mean_ns() {
37+
jq -r '.mean.point_estimate' "$1/new/estimates.json"
38+
}
39+
40+
mems_from() {
41+
local bench_dir="$1"
42+
local elements
43+
local ns
44+
elements=$(jq -r '.throughput.Elements' "$bench_dir/new/benchmark.json")
45+
ns=$(jq -r '.mean.point_estimate' "$bench_dir/new/estimates.json")
46+
awk -v e="$elements" -v ns="$ns" 'BEGIN { printf "%.2f", (e * 1e9 / ns) / 1e6 }'
47+
}
48+
49+
lru_get=$(mean_ns "$criterion_dir/lru_get_hit_ns")
50+
lru_insert=$(mean_ns "$criterion_dir/lru_insert_full_ns")
51+
lru_k_get=$(mean_ns "$criterion_dir/lru_k_get_hit_ns")
52+
lru_k_insert=$(mean_ns "$criterion_dir/lru_k_insert_full_ns")
53+
lfu_get=$(mean_ns "$criterion_dir/lfu_get_hit_ns")
54+
lfu_insert=$(mean_ns "$criterion_dir/lfu_insert_full_ns")
55+
lfu_touch=$(mean_ns "$criterion_dir/lfu_policy_only_touch_ns")
56+
57+
lru_insert_get=$(mems_from "$criterion_dir/lru_policy/insert_get")
58+
lru_eviction=$(mems_from "$criterion_dir/lru_policy/eviction_churn")
59+
lru_pop=$(mems_from "$criterion_dir/lru_policy/pop_lru")
60+
lru_hot=$(mems_from "$criterion_dir/lru_policy/touch_hotset")
61+
62+
lru_k_insert_get=$(mems_from "$criterion_dir/lru_k_policy/insert_get")
63+
lru_k_eviction=$(mems_from "$criterion_dir/lru_k_policy/eviction_churn")
64+
lru_k_pop=$(mems_from "$criterion_dir/lru_k_policy/pop_lru_k")
65+
lru_k_hot=$(mems_from "$criterion_dir/lru_k_policy/touch_hotset")
66+
67+
lfu_insert_get=$(mems_from "$criterion_dir/lfu_policy/insert_get")
68+
lfu_eviction=$(mems_from "$criterion_dir/lfu_policy/eviction_churn")
69+
lfu_pop=$(mems_from "$criterion_dir/lfu_pop_lfu_policy")
70+
lfu_hot=$(mems_from "$criterion_dir/lfu_get_hotset_policy")
71+
72+
lru_uniform=$(mems_from "$criterion_dir/lru_workload_hit_rate/uniform")
73+
lru_hotset=$(mems_from "$criterion_dir/lru_workload_hit_rate/hotset_90_10")
74+
lru_scan=$(mems_from "$criterion_dir/lru_workload_hit_rate/scan")
75+
76+
lru_k_uniform=$(mems_from "$criterion_dir/lru_k_workload_hit_rate/uniform")
77+
lru_k_hotset=$(mems_from "$criterion_dir/lru_k_workload_hit_rate/hotset_90_10")
78+
lru_k_scan=$(mems_from "$criterion_dir/lru_k_workload_hit_rate/scan")
79+
80+
lfu_uniform=$(mems_from "$criterion_dir/lfu_workload_hit_rate/uniform")
81+
lfu_hotset=$(mems_from "$criterion_dir/lfu_workload_hit_rate/hotset_90_10")
82+
lfu_scan=$(mems_from "$criterion_dir/lfu_workload_hit_rate/scan")
83+
84+
cat > /tmp/latest-run.md <<EOF
85+
Micro-ops (ns/op):
86+
87+
| Cache | get_hit | insert_full | policy_only_touch |
88+
| --- | --- | --- | --- |
89+
| LRU | ${lru_get%.*} | ${lru_insert%.*} | n/a |
90+
| LRU-K | ${lru_k_get%.*} | ${lru_k_insert%.*} | n/a |
91+
| LFU | ${lfu_get%.*} | ${lfu_insert%.*} | ${lfu_touch%.*} |
92+
93+
Policy throughput (Melem/s = million operations per second):
94+
95+
| Cache | insert_get | eviction_churn | pop | touch_hotset |
96+
| --- | --- | --- | --- | --- |
97+
| LRU | ${lru_insert_get} | ${lru_eviction} | ${lru_pop} | ${lru_hot} |
98+
| LRU-K | ${lru_k_insert_get} | ${lru_k_eviction} | ${lru_k_pop} | ${lru_k_hot} |
99+
| LFU | ${lfu_insert_get} | ${lfu_eviction} | ${lfu_pop} | ${lfu_hot} |
100+
101+
Workload throughput (Melem/s, 200k ops):
102+
103+
| Cache | uniform | hotset_90_10 | scan |
104+
| --- | --- | --- | --- |
105+
| LRU | ${lru_uniform} | ${lru_hotset} | ${lru_scan} |
106+
| LRU-K | ${lru_k_uniform} | ${lru_k_hotset} | ${lru_k_scan} |
107+
| LFU | ${lfu_uniform} | ${lfu_hotset} | ${lfu_scan} |
108+
EOF
109+
110+
awk '
111+
/<!-- LATEST_RUN_START -->/ {
112+
print;
113+
while ((getline line < "/tmp/latest-run.md") > 0) print line;
114+
in=1;
115+
next
116+
}
117+
/<!-- LATEST_RUN_END -->/ { in=0; print; next }
118+
!in { print }
119+
' docs/benchmarks.md > /tmp/benchmarks.md
120+
121+
mv /tmp/benchmarks.md docs/benchmarks.md
27122
- name: Package benchmark results
28123
run: tar -czf criterion-results.tar.gz target/criterion
124+
- name: Build docs site with Jekyll
125+
uses: actions/jekyll-build-pages@v1
126+
with:
127+
source: ./docs
128+
destination: ./_site
29129
- name: Prepare gh-pages publish directory
30130
run: |
31-
mkdir -p target/gh-pages/benchmarks/${{ github.ref_name }}
32-
mkdir -p target/gh-pages/benchmarks/latest
33-
cp -R target/criterion/. target/gh-pages/benchmarks/${{ github.ref_name }}/
34-
cp -R target/criterion/. target/gh-pages/benchmarks/latest/
131+
mkdir -p _site/benchmarks/${RELEASE_TAG}
132+
mkdir -p _site/benchmarks/latest
133+
cp -R target/criterion/. _site/benchmarks/${RELEASE_TAG}/
134+
cp -R target/criterion/. _site/benchmarks/latest/
35135
- name: Publish benchmark report to gh-pages
36136
uses: peaceiris/actions-gh-pages@v4
37137
with:
38138
github_token: ${{ secrets.GITHUB_TOKEN }}
39-
publish_dir: target/gh-pages
139+
publish_dir: _site
40140
- name: Create release for tag push
41141
if: github.event_name == 'push'
42142
uses: softprops/action-gh-release@v2

‎docs/_config.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
title: CacheKit Docs
2+
description: High-performance cache policies and supporting data structures.
3+
theme: jekyll-theme-minimal

‎docs/benchmarks.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,34 @@ This page links to the latest benchmark reports and the release-tag snapshots.
77

88
For local runs and raw numbers, see `benches/README.md`.
99

10+
## Latest run (release)
11+
12+
<!-- LATEST_RUN_START -->
13+
Micro-ops (ns/op):
14+
15+
| Cache | get_hit | insert_full | policy_only_touch |
16+
| --- | --- | --- | --- |
17+
| LRU | TBD | TBD | n/a |
18+
| LRU-K | TBD | TBD | n/a |
19+
| LFU | TBD | TBD | TBD |
20+
21+
Policy throughput (Melem/s = million operations per second):
22+
23+
| Cache | insert_get | eviction_churn | pop | touch_hotset |
24+
| --- | --- | --- | --- | --- |
25+
| LRU | TBD | TBD | TBD | TBD |
26+
| LRU-K | TBD | TBD | TBD | TBD |
27+
| LFU | TBD | TBD | TBD | TBD |
28+
29+
Workload throughput (Melem/s, 200k ops):
30+
31+
| Cache | uniform | hotset_90_10 | scan |
32+
| --- | --- | --- | --- |
33+
| LRU | TBD | TBD | TBD |
34+
| LRU-K | TBD | TBD | TBD |
35+
| LFU | TBD | TBD | TBD |
36+
<!-- LATEST_RUN_END -->
37+
1038
## Release summary
1139

1240
| Release | Date | Environment | Micro-ops (ns/op) | Policy throughput (Melem/s) | Workload throughput (Melem/s) | Report link |

0 commit comments

Comments
 (0)