Skip to content

Commit fea722f

Browse files
committed
[TASK] Add performance benchmark infrastructure
Add benchmark scripts for measuring render performance: - benchmark/run-benchmark.sh: Cold/warm/partial scenarios - benchmark/compare-branches.sh: Branch comparison - Makefile targets: benchmark-cold, benchmark-warm, etc. Uses Documentation-rendertest (94 RST files) as test project. [TASK] Add Docker benchmark targets and gitignore for test docs - Add Makefile targets for Docker-based benchmarks (reproducible) - Ignore downloaded benchmark test documentation [TASK] Add TYPO3 Core Changelog benchmark support Add extra-large documentation benchmark (3666 files) for more comprehensive performance testing. Changes: - Update download-test-docs.sh for sparse checkout from TYPO3 monorepo - Add changelog docs type to benchmark-docker.sh - Update HTML report with changelog results (95% faster: 997s -> 47s) - Add changelog to all comparison tables and bar chart - Update reproduction instructions with changelog commands [TASK] Add real CPU/memory metrics to benchmarks - Fix bar chart height to prevent overflow with changelog bar - Install GNU time in Docker image for accurate metrics - Run /usr/bin/time -v inside container (not outside) - Capture wall time, CPU time, CPU%, and peak memory - Update JSON output with extended metrics structure Now reports actual PHP process metrics instead of estimates. Example: 5.18s wall, 10.48s CPU (202%), 119.8MB memory feat: Enhance benchmark infrastructure with profiling and parallelism matrix Changes to benchmark-docker.sh: - Use GUIDES_PROFILING=1 for accurate memory metrics via memory_get_peak_usage() - Add 4th parameter for parallel mode: sequential, auto, or number (e.g., 16) - Include parallel_mode in result filenames and JSON output - Memory now sourced from PHP profiling instead of /usr/bin/time New benchmark-main.sh: - Dedicated script for benchmarking main branch with official container - Uses ghcr.io/typo3-documentation/render-guides:latest - Memory via /usr/bin/time (PHP profiling not available in official image) New run-full-matrix.sh: - Runs complete benchmark matrix: * main branch (official container) * feature branch: sequential (--parallel-workers=-1) * feature branch: auto (--parallel-workers=0) * feature branch: 16 workers (--parallel-workers=16) - Supports small, large, changelog, or 'all' doc types Usage examples: ./benchmark/benchmark-docker.sh cold 3 large sequential ./benchmark/benchmark-docker.sh cold 3 large auto ./benchmark/benchmark-docker.sh cold 3 large 16 ./benchmark/run-full-matrix.sh large
1 parent fde010d commit fea722f

24 files changed

Lines changed: 1427 additions & 339 deletions

File tree

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,7 @@
88
/.phpdoc/
99
/phpdoc.dist.xml
1010
*-temp/
11+
12+
# Benchmark test documentation (downloaded on demand)
13+
/benchmark/test-docs/
14+
!/benchmark/test-docs/.gitkeep

Dockerfile

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ COPY --from=ghcr.io/php/pie:bin /pie /usr/bin/pie
3636
RUN apk add --update $PHPIZE_DEPS
3737
RUN pie install arnaud-lb/inotify && docker-php-ext-install pcntl
3838

39+
# Install GNU time for benchmarking (busybox time doesn't support -v)
40+
RUN apk add --no-cache time
3941
RUN apk del $PHPIZE_DEPS && rm -rf /var/cache/apk/* /tmp/* /usr/share/php/* /usr/local/lib/php/doc/* /usr/local/lib/php/test/*
4042

4143
COPY . /opt/guides

Makefile

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -217,3 +217,58 @@ vendor: composer.json composer.lock
217217
@echo "$(ENV_INFO)"
218218
$(PHP_COMPOSER_BIN) validate --no-check-publish
219219
$(PHP_COMPOSER_BIN) install --no-interaction --no-progress --ignore-platform-reqs
220+
221+
## LIST: Benchmark targets for performance testing
222+
223+
.PHONY: benchmark-cold
224+
benchmark-cold: ## Run cold render benchmark (no cache)
225+
@echo "$(ENV_INFO)"
226+
./benchmark/run-benchmark.sh cold 3
227+
228+
.PHONY: benchmark-warm
229+
benchmark-warm: ## Run warm render benchmark (with cache, no changes)
230+
@echo "$(ENV_INFO)"
231+
./benchmark/run-benchmark.sh warm 3
232+
233+
.PHONY: benchmark-partial
234+
benchmark-partial: ## Run partial change benchmark (one file modified)
235+
@echo "$(ENV_INFO)"
236+
./benchmark/run-benchmark.sh partial 3
237+
238+
.PHONY: benchmark-all
239+
benchmark-all: benchmark-cold benchmark-warm benchmark-partial ## Run all benchmark scenarios
240+
241+
.PHONY: benchmark-compare
242+
benchmark-compare: ## Compare benchmarks between main and current branch
243+
@echo "$(ENV_INFO)"
244+
./benchmark/compare-branches.sh main
245+
246+
## LIST: Docker-based benchmark targets (recommended for reproducibility)
247+
248+
.PHONY: benchmark-download-docs
249+
benchmark-download-docs: ## Download TYPO3 CoreApi documentation for large benchmarks
250+
./benchmark/download-test-docs.sh TYPO3CMS-Reference-CoreApi
251+
252+
.PHONY: benchmark-docker-cold
253+
benchmark-docker-cold: docker-build ## Run cold benchmark in Docker (small docs)
254+
./benchmark/benchmark-docker.sh cold 3 small
255+
256+
.PHONY: benchmark-docker-warm
257+
benchmark-docker-warm: docker-build ## Run warm benchmark in Docker (small docs)
258+
./benchmark/benchmark-docker.sh warm 3 small
259+
260+
.PHONY: benchmark-docker-partial
261+
benchmark-docker-partial: docker-build ## Run partial benchmark in Docker (small docs)
262+
./benchmark/benchmark-docker.sh partial 3 small
263+
264+
.PHONY: benchmark-docker-all
265+
benchmark-docker-all: docker-build ## Run all benchmarks in Docker (small docs)
266+
./benchmark/benchmark-docker.sh all 3 small
267+
268+
.PHONY: benchmark-docker-large
269+
benchmark-docker-large: docker-build benchmark-download-docs ## Run all benchmarks with large TYPO3 docs
270+
./benchmark/benchmark-docker.sh all 3 large
271+
272+
.PHONY: benchmark-report
273+
benchmark-report: ## Generate markdown comparison report
274+
./benchmark/generate-report.sh

0 commit comments

Comments
 (0)