Skip to content

Commit 5efbd5f

Browse files
committed
bench, docs: expand benchmark suite and update performance metrics
- Added std::stack+malloc and GNU Obstack to the benchmark suite. - Replaced monolithic tests with Small and Large payload scenarios. - Integrated a stateful DummyStack to prevent compiler dead-loop folding. - Added CPU hardware performance counters from perf to the analysis. - Generated new clean-line throughput charts and fully updated the README.
1 parent 1e3f2a3 commit 5efbd5f

9 files changed

Lines changed: 757 additions & 223 deletions
-242 KB
Binary file not shown.
-248 KB
Binary file not shown.
196 KB
Loading
203 KB
Loading

.gitignore

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,4 +69,6 @@ fuzzers/*_fuzzer
6969

7070
benchmark
7171
benchmark_*
72-
wasi*
72+
wasi*
73+
perf.data
74+
bench/perf*

Makefile

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -300,6 +300,7 @@ clean:
300300
rm -f $(FUZZ_BINS) $(FUZZ_DEBUG_BINS)
301301
rm -rf $(MATRIX_DIR)
302302
rm -f $(BENCH_DIR)/benchmark_* # Clean all benchmark binaries
303+
rm -f $(BENCH_DIR)/perf_benchmark
303304

304305

305306
# --- Fuzzing Targets ---
@@ -328,6 +329,31 @@ replay_%: $(FUZZ_DIR)/%_fuzzer_debug
328329
@printf "\n--- Replaying crash file: $(CRASH) on $< ---\n"
329330
@./$< $(CRASH)
330331

332+
333+
# ==========================================
334+
# Performance Profiling (Local Hardware Only)
335+
# ==========================================
336+
337+
BENCH_PERF_BIN = $(BENCH_DIR)/perf_benchmark
338+
339+
.PHONY: perf_stat perf_record
340+
341+
# Build a lightweight, isolated benchmark just for profiling
342+
$(BENCH_PERF_BIN): $(BENCH_SRCS) easy_stack.h
343+
@printf "Compiling quick profiling benchmark: $@\n"
344+
@$(CXX) $(CXXFLAGS) -DBENCH_ONLY_EASYSTACK -DBENCH_QUICK $(BENCH_SRCS) -o $@
345+
346+
# Run benchmark with hardware performance counters (cache misses, branch predictions, IPC)
347+
perf_stat: $(BENCH_PERF_BIN)
348+
@printf "\n--- Running 'perf stat' on EasyStack (Quick Profile) ---\n"
349+
perf stat -e cache-misses,cache-references,branches,branch-misses,instructions,cycles ./$(BENCH_PERF_BIN)
350+
351+
# Record CPU profile for deep hotspot analysis (inspect with 'perf report')
352+
perf_record: $(BENCH_PERF_BIN)
353+
@printf "\n--- Recording CPU Profile for EasyStack (Quick Profile) ---\n"
354+
perf record -F 99 -g ./$(BENCH_PERF_BIN)
355+
@printf "\nProfile saved. Run 'perf report' to analyze hotspots.\n"
356+
331357
# Show available tests
332358
list:
333359
@printf "Available commands:\n"
@@ -338,6 +364,8 @@ list:
338364
@printf " make coverage - build & run tests to generate coverage data for CodeCov\n"
339365
@printf " make fuzz_[name] - run the 'core' fuzzer for 5 minutes (auto-detects fuzz_*.c)\n"
340366
@printf " make replay_[name] CRASH=... - replay a specific crash file with ASCII visualization\n"
367+
@printf " make perf_stat - run benchmarks under 'perf stat' to measure cache/branch misses\n"
368+
@printf " make perf_record - record CPU profile to find hotspots (run 'perf report' afterwards)\n"
341369
@printf "\nAvailable individual tests (always with debug output):\n"
342370
@for test in $(TEST_SRCS) ; do \
343371
basename=$$(basename $${test%.c} _test); \

README.md

Lines changed: 106 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -30,13 +30,16 @@
3030

3131
<br/>
3232

33-
**An absurdly fast, header-only, platform-agnostic, and safe LIFO stack allocator utilizing an inverted bi-directional buffer layout with dynamic metadata scaling.**
33+
**A zero-compromise, header-only C/C++ LIFO stack allocator that outperforms compile-time templates and system baselines in both execution speed and memory footprint simultaneously — even under full runtime safety.**
3434

3535
## TL;DR
3636

37-
**What is it?** A portable, extremely fast, single-header stack allocator that completely decouples control paths from aligned user payloads. It eliminates inline metadata overhead entirely by growing tracking offsets forward and aligned payloads backward.
37+
**What is it?** A portable, single-header LIFO stack allocator that completely decouples control paths from aligned user payloads. It eliminates inline metadata overhead entirely by growing tracking offsets forward and aligned payloads backward.
3838

39-
**Why use it?** To achieve deterministic **O(1)** allocation and deallocation speeds that outperform traditional C allocators (like `wb_alloc`) by up to **4.1x+** and compile-time optimized C++ templates (like `Trebi`) by up to **4.7x+** in trusted mode (while remaining **2.5x+ faster** than both even with full runtime defensive safety active), all while retaining up to an **8x smaller metadata footprint**.
39+
**Why use it?** Traditional memory allocators force you to trade execution speed for runtime safety, or memory footprint for performance. `easy_stack` breaks this triple trade-off, delivering:
40+
* **Highest-in-Class Performance**: Outperforms compile-time C++ templates (like `Trebi`) by up to **2.7x+** and standard heap managers (like `malloc`) by up to **10x+** in algorithmic speed.
41+
* **Zero-Cost Safety**: Even with full defensive runtime safety, bounds checking, and API sanitization active, it retains a performance advantage over *unprotected* competitor alternatives.
42+
* **Maximum Memory Efficiency**: Uses up to an **8x smaller metadata footprint** compared to traditional inline headers and wastes exactly zero bytes on alignment padding in the control zone.
4043

4144
**How to use it?** `#define EASY_STACK_IMPLEMENTATION` in one `.c` file, then just `#include "easy_stack.h"`.
4245

@@ -130,66 +133,92 @@ Traditional stack allocators suffix or prefix each payload with inline metadata
130133

131134
## Benchmarks & Performance
132135

133-
To verify execution speed, cache resilience, and scalability under load, `easy_stack` was benchmarked against popular alternatives:
134-
* **wb_alloc (Bundy):** A widely-used, minimalist C arena allocator.
135-
* **Trebi StackAllocator:** A standard C++ LIFO stack allocator (compiled with `-flto` for maximum devirtualization).
136+
To evaluate execution speed, cache resilience, and scalability under different workloads, `easy_stack` was benchmarked against popular alternatives.
136137

137138
### Test Environment
138-
* **CPU:** AMD Ryzen 7 4700U (8 Cores / 8 Threads, Zen 2 @ up to 4.1 GHz)
139-
* **Compiler:** GCC 15.2 with `-O3 -flto -DNDEBUG`
140-
* **Scenario:** 2,000,000 iterations per run (executing up to 480,000,000 allocator operations) of nested allocations with randomized sizes (16-160 bytes) scaled across three distinct stack allocation depths (15, 30, and 100). Best of 25 runs.
139+
* **CPU**: AMD Ryzen 7 4700U (8 Cores / 8 Threads, Zen 2 @ up to 4.1 GHz)
140+
* **Compiler**: GCC 16.1 with -O3 -flto -DNDEBUG
141+
* **Scenario**: 2,000,000 iterations per run (executing up to 480,000,000 allocator operations) of nested allocations with randomized sizes scaled across three distinct stack allocation depths (15, 30, and 100). Best of 25 runs.
142+
143+
### Tested Allocators:
144+
1. **EasyStack (Contract)**: Trusted mode with validations delegated to assertions (compiled out in release).
145+
2. **EasyStack (Defensive)**: Default safety mode with full runtime bounds and API sanitization active.
146+
3. **Trebi LIFO**: A highly-optimized C++ template-based LIFO stack allocator.
147+
4. **GNU Obstack**: The glibc standard stack allocator (highly optimized C system baseline).
148+
5. **wb_alloc (Bundy)**: A popular fixed-size C arena/stack allocator.
149+
6. **std::stack + malloc**: The default standard library heap-allocated baseline.
141150

142-
### 1. Throughput & Cache-Line Scaling (Speed)
143-
144-
Rather than benchmarking only a single shallow depth, the suite tests scaling across three critical architectural boundaries. This highlights how cache layout and processor prefetching impact execution speed as stack depth increases.
145-
146-
#### Phase A: The Standard 64 KB Workload (16-bit Metadata)
151+
---
147152

148-
With a 64 KB capacity, `easy_stack` configures itself to use ultra-compact **Type 1 (16-bit)** metadata offsets.
153+
### Methodology & Metrics
149154

150-
<p align="center">
151-
<img src="https://raw.githubusercontent.com/EasyMem/easy_stack/refs/heads/main/.github/assets/throughput_64kb_chart.png" width="750" alt="Throughput Scaling vs Stack Allocation Depth at 64KB" />
152-
</p>
155+
We evaluate performance using two distinct metrics to provide a transparent picture of real-world overhead:
156+
* **RAW Throughput**: The total execution time including loop control and function-call overhead. This represents the actual performance experienced by an application calling these routines.
157+
* **PURE Algorithmic Throughput**: Calculated by subtracting the baseline harness overhead (measured via a stateful dummy call wrapper). This isolates the pure overhead of the allocator's internal logic (pointer arithmetic, dynamic metadata scaling, alignment padding math, and safety checks).
153158

154-
At shallow depths (15 objects), the entire active allocator footprint (header + 30 bytes of metadata) fits completely into a **single 64-byte L1 cache line**. This yields a hardware-limit speed of **939 Million ops/sec** (~1.06 ns per allocation/free cycle) in Contract mode, outperforming the C++ template LIFO allocator by **371%**.
159+
To prevent compile-time folding, dead-code elimination (DCE), or loop hoisting by aggressive GCC 16 optimizations, the benchmark utilizes a strict **Data Dependency Chain**. Every allocation writes a dynamic payload bound to its physical runtime address, and these payloads are read back and accumulated into a volatile global checksum sink prior to deallocation.
155160

156161
---
157162

158-
#### Phase B: The Transparent 1 MB Workload (32-bit Metadata)
159-
160-
> *"Wait a minute! You're cheating! By using a tiny 64 KB buffer, you guarantee that all metadata easily fits inside L1 cache. Let's see what happens on a larger stack."*
163+
### 1. Small Payload Workloads (16 – 128 Bytes)
164+
*Designed to simulate standard stack frame allocations, temporary object creation, and shallow trees.*
161165

162-
The objection is logically sound. To address this, the stack capacity was scaled up.
166+
<p align="center">
167+
<img src="https://raw.githubusercontent.com/EasyMem/easy_stack/refs/heads/main/.github/assets/throughput_small_payloads.png" width="800" alt="Small Payloads Throughput Scaling" />
168+
</p>
163169

164-
In **Defensive** and **Contract** modes, the capacity was increased to **1 MB**, automatically triggering the transition to 4-byte metadata offsets (`meta_type = 2`).
170+
#### RAW Results (Best of 25 runs, 2,000,000 iterations/run)
171+
| Allocator | Depth 15 (M ops/s) | Depth 30 (M ops/s) | Depth 100 (M ops/s) |
172+
| :--- | :---: | :---: | :---: |
173+
| **EasyStack (Contract)** | **576.40** | **516.74** | **493.63** |
174+
| **EasyStack (Defensive)** | **415.25** | **385.75** | **402.64** |
175+
| GNU Obstack | 396.85 | 379.33 | 311.31 |
176+
| Trebi LIFO (C++) | 365.55 | 358.51 | 360.20 |
177+
| std::stack + malloc | 209.89 | 214.63 | 188.13 |
178+
| wb_alloc (Bundy) | 208.18 | 204.88 | 229.38 |
179+
180+
#### PURE Algorithmic Results (Harness Overhead Subtracted)
181+
| Allocator | Depth 15 (M ops/s) | Depth 30 (M ops/s) | Depth 100 (M ops/s) |
182+
| :--- | :---: | :---: | :---: |
183+
| **EasyStack (Contract)** | **2747.82** | **2408.14** | **4972.83** |
184+
| **EasyStack (Defensive)** | **1076.10** | **941.91** | **1566.77** |
185+
| GNU Obstack | 870.41 | 895.82 | 720.72 |
186+
| Trebi LIFO (C++) | 732.81 | 787.78 | 1050.94 |
187+
| std::stack + malloc | 294.69 | 318.55 | 286.47 |
188+
| wb_alloc (Bundy) | 291.33 | 297.53 | 394.49 |
165189

166-
```bash
167-
# Run benchmark with custom depth
168-
make bench DEPTH=30
169-
make bench DEPTH=100
170-
```
190+
---
171191

172-
This ensures that at depth 100, the metadata array alone spans multiple cache lines, removing any artificial L1 prefetching advantages. Here is how the allocator scales under the heavier 32-bit math path:
192+
### 2. Large Payload Workloads (512 – 4096 Bytes)
193+
*Designed to simulate heavy SIMD vectors, DMA buffers, and large temporary data arrays.*
173194

174195
<p align="center">
175-
<img src="https://raw.githubusercontent.com/EasyMem/easy_stack/refs/heads/main/.github/assets/throughput_1mb_chart.png" width="750" alt="Throughput Scaling vs Stack Allocation Depth at 1MB" />
196+
<img src="https://raw.githubusercontent.com/EasyMem/easy_stack/refs/heads/main/.github/assets/throughput_large_payloads.png" width="800" alt="Large Payloads Throughput Scaling" />
176197
</p>
177198

178-
#### Key Takeaways:
179-
180-
* **Zero-Overhead Metadata Scaling:**
181-
Comparing the 64 KB buffer (16-bit) vs the 1 MB buffer (32-bit), the throughput drop in Contract mode is **less than 0.5%** at depth 15 (**934 Million ops/sec** vs 939 Million) and only **2.7%** at depth 100 (**717 Million** vs 738 Million). This demonstrates that dynamically widening metadata cells has a negligible performance impact on modern superscalar architectures.
182-
183-
* **Instruction Latency Hiding (Defensive Mode Stability):**
184-
In Defensive (Safety) mode, throughput remains completely flat at a rock-solid **~520-550 Million ops/sec** across both 64 KB and 1 MB buffers, regardless of stack allocation depth.
185-
186-
The execution of defensive `if` branches and bounds checks occupies the ALU instruction pipeline. During this execution window, the CPU's prefetcher asynchronously loads upcoming metadata cache lines in the background. By the time safety checks are evaluated, the memory is already waiting in L1, resulting in a **zero-cycle memory stall**.
187-
188-
* **Absolute Dominance:**
189-
Even under full runtime safety checks and larger capacities, **EasyStack remains up to 191% faster than Trebi (C++) and 153% faster than wb_alloc (C)**.
199+
#### RAW Results (Best of 25 runs, 2,000,000 iterations/run)
200+
| Allocator | Depth 15 (M ops/s) | Depth 30 (M ops/s) | Depth 100 (M ops/s) |
201+
| :--- | :---: | :---: | :---: |
202+
| **EasyStack (Contract)** | **589.30** | **506.21** | **494.47** |
203+
| **EasyStack (Defensive)** | **431.45** | **393.71** | **369.90** |
204+
| Trebi LIFO (C++) | 364.19 | 360.50 | 354.11 |
205+
| wb_alloc (Bundy) | 70.66 | 69.06 | 67.70 |
206+
| GNU Obstack | 57.07 | 58.45 | 56.06 |
207+
| std::stack + malloc | 43.13 | 45.57 | 42.78 |
208+
209+
#### PURE Algorithmic Results (Harness Overhead Subtracted)
210+
| Allocator | Depth 15 (M ops/s) | Depth 30 (M ops/s) | Depth 100 (M ops/s) |
211+
| :--- | :---: | :---: | :---: |
212+
| **EasyStack (Contract)** | **3834.49** | **2164.57** | **4460.48** |
213+
| **EasyStack (Defensive)** | **1215.93** | **1014.13** | **1141.30** |
214+
| Trebi LIFO (C++) | 763.55 | 793.38 | 974.88 |
215+
| wb_alloc (Bundy) | 78.64 | 77.12 | 77.09 |
216+
| GNU Obstack | 62.17 | 64.12 | 62.35 |
217+
| std::stack + malloc | 45.97 | 48.94 | 46.34 |
190218

219+
---
191220

192-
### 2. Memory Efficiency (Payload vs. Overhead)
221+
### 3. Memory Efficiency (Payload vs. Overhead)
193222

194223
Traditional stack allocators prefix each block with a fixed 16-byte inline header (on 64-bit systems). On small allocations (common for temporary stacks), this inline overhead consumes up to **50%+ of your buffer**.
195224

@@ -198,15 +227,44 @@ Traditional stack allocators prefix each block with a fixed 16-byte inline heade
198227
Below is a comparison of usable payload space in a **10 KB buffer** across different allocation sizes:
199228

200229
<p align="center">
201-
<img src="https://raw.githubusercontent.com/EasyMem/easy_stack/refs/heads/main/.github/assets/memory_efficiency_chart.png" width="700" alt="Memory Efficiency" />
230+
<img src="https://raw.githubusercontent.com/EasyMem/easy_stack/refs/heads/main/.github/assets/memory_efficiency_chart.png" width="700" alt="Memory Efficiency" />
202231
</p>
203232

204-
* **For 8-byte allocations:** `easy_stack` lets you fit **2.40x more objects** into the same memory buffer (80% usable memory vs. 33%).
205-
* **For 16-byte allocations:** `easy_stack` lets you fit **1.77x more objects** into the same memory buffer (89% usable memory vs. 50%).
233+
* **For 8-byte allocations:** `easy_stack` lets you fit **2.40x more objects** into the same memory buffer (80% usable memory vs. 33%).
234+
* **For 16-byte allocations:** `easy_stack` lets you fit **1.77x more objects** into the same memory buffer (89% usable memory vs. 50%).
206235

207-
---
208236
*Note: The chart above represents the absolute best-case scenario for the traditional allocator, assuming perfect power-of-two allocation sizes with 0 alignment padding. In real-world workloads with non-power-of-two sizes, traditional inline-header allocators suffer from internal fragmentation (wasting up to 7-15 bytes of alignment padding per block), which widens the efficiency gap even further in favor of `easy_stack`.*
209237

238+
---
239+
240+
### 4. Hardware-Level Profiling (perf)
241+
242+
To verify the microarchitectural efficiency of the allocator and isolate the hardware reasons behind these execution speeds, the benchmark suite (with `BENCH_ONLY_EASYSTACK` active) was profiled using Linux `perf` hardware performance counters on a Zen-architecture CPU.
243+
244+
The raw hardware counters and derived efficiency metrics across all tested stack depths are presented below:
245+
246+
| Hardware Metric | Depth 15 | Depth 30 | Depth 100 |
247+
| :--- | :---: | :---: | :---: |
248+
| **Instructions Retired** | 77,054,323,714 | 77,054,323,942 | 77,054,324,011 |
249+
| **CPU Cycles** | 24,655,695,668 | 24,682,754,642 | 25,028,328,076 |
250+
| **Instructions Per Cycle (IPC)** | **3.13** | **3.12** | **3.08** |
251+
| **Total Branches** | 16,351,231,347 | 16,351,231,567 | 16,351,231,582 |
252+
| **Branch Mispredictions** | 20,580 | 23,757 | 22,401 |
253+
| **Branch Misprediction Rate (%)** | **0.00013%** | **0.00015%** | **0.00014%** |
254+
| **Cache References** | 4,382,622 | 2,355,244 | 2,789,922 |
255+
| **Cache Misses** | 27,865 | 53,731 | 34,509 |
256+
| **Cache Miss Rate (%)** | **0.64%** | **2.28%** | **1.24%** |
257+
258+
---
259+
260+
### Architectural Analysis of Hardware Metrics:
261+
262+
* **Instruction Pipeline Saturation (IPC)**: Operating at **`3.08 - 3.13 IPC`** (calculated as instructions / cycles) is an exceptional result. While complex system-level workloads typically average `1.2 - 1.8 IPC` due to CPU pipeline stalls waiting for RAM data, EasyStack keeps the processor executing more than 3 instructions per clock cycle with near-zero execution stalls or data dependencies.
263+
264+
* **Near-Zero Branch Predictor Penalties**: Across billions of branches executed, the processor encountered a misprediction rate of just **`0.00013% - 0.00015%`** (calculated as branch-misses / branches). By eliminating heavy lookup loops, tree traversals, and dynamic boundary checks, the critical path compiles into a highly predictable, linear instruction stream, allowing the CPU to speculation-execute allocations hundreds of cycles ahead.
265+
266+
* **L1 Data Cache Residency**: Cache misses remain extremely low, dropping to **`0.64%`** at depth 15 and hovering at just **`1.24%`** at maximum depth (calculated as cache-misses / cache-references). By segregating metadata from aligned user payloads (Inverted Layout), the active metadata array is tightly packed and remains resident in standard 64-byte L1 CPU cache lines, bypassing main memory latency entirely.
267+
210268
## Usage
211269

212270
### 1. Integration

0 commit comments

Comments
 (0)