|
| 1 | +# Memory Pressure Benchmark |
| 2 | + |
| 3 | +Measures tracer throughput overhead across decreasing heap sizes using spring-petclinic and JMeter. |
| 4 | + |
| 5 | +Based on Doug's methodology: run petclinic with `GET /owners/3` (DB-backed endpoint) at heap sizes from 256m down to 64m, comparing no-agent baseline vs candidate agent. |
| 6 | + |
| 7 | +## Quick start |
| 8 | + |
| 9 | +```bash |
| 10 | +# Full run: baseline + candidate at all heap sizes |
| 11 | +./run.sh |
| 12 | + |
| 13 | +# With a specific agent jar |
| 14 | +./run.sh --agent /path/to/dd-java-agent.jar |
| 15 | + |
| 16 | +# Quick test with fewer heap sizes and shorter durations |
| 17 | +./run.sh --heap-sizes 256,128,64 --warmup 15 --measure 30 |
| 18 | + |
| 19 | +# With JFR profiling |
| 20 | +./run.sh --jfr |
| 21 | + |
| 22 | +# Only run candidate (skip baseline if you already have baseline numbers) |
| 23 | +./run.sh --skip-baseline |
| 24 | + |
| 25 | +# Test a specific optimization with extra agent opts |
| 26 | +./run.sh --agent-opts "-Ddd.appsec.enabled=false -Ddd.profiling.enabled=false" |
| 27 | +``` |
| 28 | + |
| 29 | +## What it does |
| 30 | + |
| 31 | +For each heap size (default: 256, 192, 128, 96, 80, 64 MiB): |
| 32 | + |
| 33 | +1. Starts petclinic with `-Xms<N>m -Xmx<N>m` (no agent) |
| 34 | +2. Runs JMeter with 8 threads for warmup + measurement |
| 35 | +3. Records stabilized throughput (measurement window only) |
| 36 | +4. Repeats with `-javaagent:dd-java-agent.jar` |
| 37 | +5. If petclinic OOMs or fails to start, records "OOM" |
| 38 | + |
| 39 | +## Output |
| 40 | + |
| 41 | +``` |
| 42 | +Heap Baseline Candidate Delta |
| 43 | +------ ----------- ----------- ----------- |
| 44 | +256m 22400 req/s 19500 req/s -12.9% |
| 45 | +192m 21750 req/s 19000 req/s -12.6% |
| 46 | +128m 21500 req/s 14750 req/s -31.4% |
| 47 | +96m 16750 req/s 13000 req/s -22.4% |
| 48 | +80m 16350 req/s 11250 req/s -31.2% |
| 49 | +64m 15000 req/s 7500 req/s -50.0% |
| 50 | +``` |
| 51 | + |
| 52 | +Results are also saved as CSV in `results/<timestamp>/summary.csv` with per-run JTL files and logs. |
| 53 | + |
| 54 | +## Dependencies |
| 55 | + |
| 56 | +Auto-downloaded on first run: |
| 57 | +- **JMeter 5.6.3** → `tools/apache-jmeter-5.6.3/` |
| 58 | +- **spring-petclinic 3.3.0** → `tools/spring-petclinic/` (built with Maven) |
| 59 | +- **dd-java-agent** → auto-detected from `dd-java-agent/build/libs/` or built via Gradle |
| 60 | + |
| 61 | +Requires: Java 17+, Python 3 (for JTL parsing), curl, git. |
| 62 | + |
| 63 | +## Comparing versions |
| 64 | + |
| 65 | +To compare two tracer versions, run separately and diff the CSVs: |
| 66 | + |
| 67 | +```bash |
| 68 | +# Run with release version |
| 69 | +./run.sh --agent /path/to/dd-java-agent-1.61.0.jar --output ./results/v1.61.0 |
| 70 | + |
| 71 | +# Run with candidate |
| 72 | +./run.sh --agent /path/to/dd-java-agent-SNAPSHOT.jar --output ./results/candidate |
| 73 | + |
| 74 | +# Compare |
| 75 | +paste -d, results/v1.61.0/summary.csv results/candidate/summary.csv |
| 76 | +``` |
0 commit comments