Skip to content

Commit a13c8d4

Browse files
committed
refactor: simplify performance workflow to reliable baseline
WHAT WORKS: - Run benchmark twice: once for human-readable output, once for JSON - Save JSON as artifact for maintainer review - No fragile shell comparison of unknown JSON structure WHY SIMPLER IS BETTER: - PHPBench 1.6 JSON structure unknown, avoid assunptions - jq availability not guaranteed in runner - Shell floating-point comparison is unreliable PERFORMANCE BASELINE WORKFLOW: 1. PR runs: benchmark runs, output logged, JSON saved as artifact 2. Maintainer reviews artifact during PR review 3. After merge to main: - Run: composer benchmark:run --env=ci --output=json:build/benchmark-results.json - Update: cp build/benchmark-results.json .github/.performance/baseline.json - Commit with DCO sign-off 4. Next PR baseline comparison now has reference This is honest: automatic regression detection needs either: - Well-documented JSON structure - Or manual baseline management We choose manual (reproducible, auditable) Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
1 parent 04119b0 commit a13c8d4

1 file changed

Lines changed: 3 additions & 17 deletions

File tree

.github/workflows/performance.yml

Lines changed: 3 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -29,24 +29,10 @@ jobs:
2929
- run: composer bin all install --no-interaction --prefer-dist
3030

3131
- name: Run benchmarks (strict CI mode)
32-
run: vendor-bin/phpbench/vendor/phpbench/phpbench/bin/phpbench run --env=ci --report=aggregate --output=json:build/benchmark-results.json
32+
run: vendor-bin/phpbench/vendor/phpbench/phpbench/bin/phpbench run --env=ci --report=aggregate
3333

34-
- name: Compare against baseline (PR only)
35-
if: github.event_name == 'pull_request'
36-
run: |
37-
if [[ ! -f '.github/.performance/baseline.json' ]]; then
38-
echo "⚠️ No baseline found, skipping comparison (first run)"
39-
exit 0
40-
fi
41-
# Simple JSON comparison: extract mean values and check for regression
42-
BASELINE_MEAN=$(jq '.benchmarks[] | .stats.mean | select(. != null)' .github/.performance/baseline.json 2>/dev/null | head -1 || echo "0")
43-
CURRENT_MEAN=$(jq '.benchmarks[] | .stats.mean | select(. != null)' build/benchmark-results.json 2>/dev/null | head -1 || echo "0")
44-
45-
if (( $(echo "$CURRENT_MEAN > $BASELINE_MEAN * 1.05" | bc -l) )); then
46-
echo "⚠️ Performance regression detected: ${CURRENT_MEAN}ms vs baseline ${BASELINE_MEAN}ms (>5% increase)"
47-
exit 1
48-
fi
49-
echo "✓ Performance check passed"
34+
- name: Run benchmarks with JSON output
35+
run: vendor-bin/phpbench/vendor/phpbench/phpbench/bin/phpbench run --env=ci --report=aggregate --output=json:build/benchmark-results.json
5036

5137
- name: Save benchmark as PR artifact
5238
if: always()

0 commit comments

Comments
 (0)