Commit fea722f
committed
[TASK] Add performance benchmark infrastructure
Add benchmark scripts for measuring render performance:
- benchmark/run-benchmark.sh: Cold/warm/partial scenarios
- benchmark/compare-branches.sh: Branch comparison
- Makefile targets: benchmark-cold, benchmark-warm, etc.
Uses Documentation-rendertest (94 RST files) as test project.
[TASK] Add Docker benchmark targets and gitignore for test docs
- Add Makefile targets for Docker-based benchmarks (reproducible)
- Ignore downloaded benchmark test documentation
[TASK] Add TYPO3 Core Changelog benchmark support
Add extra-large documentation benchmark (3666 files) for more
comprehensive performance testing.
Changes:
- Update download-test-docs.sh for sparse checkout from TYPO3 monorepo
- Add changelog docs type to benchmark-docker.sh
- Update HTML report with changelog results (95% faster: 997s -> 47s)
- Add changelog to all comparison tables and bar chart
- Update reproduction instructions with changelog commands
[TASK] Add real CPU/memory metrics to benchmarks
- Fix bar chart height to prevent overflow with changelog bar
- Install GNU time in Docker image for accurate metrics
- Run /usr/bin/time -v inside container (not outside)
- Capture wall time, CPU time, CPU%, and peak memory
- Update JSON output with extended metrics structure
Now reports actual PHP process metrics instead of estimates.
Example: 5.18s wall, 10.48s CPU (202%), 119.8MB memory
feat: Enhance benchmark infrastructure with profiling and parallelism matrix
Changes to benchmark-docker.sh:
- Use GUIDES_PROFILING=1 for accurate memory metrics via memory_get_peak_usage()
- Add 4th parameter for parallel mode: sequential, auto, or number (e.g., 16)
- Include parallel_mode in result filenames and JSON output
- Memory now sourced from PHP profiling instead of /usr/bin/time
New benchmark-main.sh:
- Dedicated script for benchmarking main branch with official container
- Uses ghcr.io/typo3-documentation/render-guides:latest
- Memory via /usr/bin/time (PHP profiling not available in official image)
New run-full-matrix.sh:
- Runs complete benchmark matrix:
* main branch (official container)
* feature branch: sequential (--parallel-workers=-1)
* feature branch: auto (--parallel-workers=0)
* feature branch: 16 workers (--parallel-workers=16)
- Supports small, large, changelog, or 'all' doc types
Usage examples:
./benchmark/benchmark-docker.sh cold 3 large sequential
./benchmark/benchmark-docker.sh cold 3 large auto
./benchmark/benchmark-docker.sh cold 3 large 16
./benchmark/run-full-matrix.sh large1 parent fde010d commit fea722f
24 files changed
Lines changed: 1427 additions & 339 deletions
File tree
- benchmark
- results
- docs
- tests/Integration
- tests
- confval/confval-duplicate/expected/logs
- interlink/interlink-warn/expected/logs
- raw-forbidden/expected/logs
- roles
- role-composer-warning/expected/logs
- role-issue-invalid/expected/logs
- site-set-failure/expected/logs
- youtube-invalid/expected/logs
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
| 39 | + | |
| 40 | + | |
39 | 41 | | |
40 | 42 | | |
41 | 43 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
217 | 217 | | |
218 | 218 | | |
219 | 219 | | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
0 commit comments