|
| 1 | +--- |
| 2 | +name: flow-code-performance |
| 3 | +description: "Use when investigating performance issues, optimizing hot paths, adding benchmarks, or establishing performance regression guards" |
| 4 | +--- |
| 5 | + |
| 6 | +# Performance Methodology |
| 7 | + |
| 8 | +## Overview |
| 9 | + |
| 10 | +Systematic performance optimization: measure first, then fix. Never optimize without a baseline. Intuition about bottlenecks is wrong more often than it is right -- profiling data is the only reliable guide. Every optimization must be measured before and after, and guarded against regression. |
| 11 | + |
| 12 | +## When to Use |
| 13 | + |
| 14 | +- Performance requirements exist in the spec (latency budgets, throughput SLAs) |
| 15 | +- Users or monitoring report slow behavior |
| 16 | +- Adding benchmarks to hot paths or critical operations |
| 17 | +- Pre-release performance check or audit |
| 18 | +- Investigating a suspected performance regression |
| 19 | +- Optimizing build times, test suite duration, or CI pipeline speed |
| 20 | + |
| 21 | +**When NOT to use:** |
| 22 | +- Premature optimization -- no evidence of a problem exists yet |
| 23 | +- No baseline measurements exist (measure first, then come back) |
| 24 | +- Micro-optimizing code that isn't on the hot path |
| 25 | +- "Making it faster" without a target metric to hit |
| 26 | + |
| 27 | +## Core Process |
| 28 | + |
| 29 | +``` |
| 30 | +1. MEASURE --> Establish baseline with profiling tools |
| 31 | +2. IDENTIFY --> Find the actual bottleneck (profiler, not intuition) |
| 32 | +3. FIX --> Apply targeted fix to the measured bottleneck only |
| 33 | +4. VERIFY --> Re-measure. If not measurably faster, revert |
| 34 | +5. GUARD --> Add regression test/benchmark to prevent regression |
| 35 | +``` |
| 36 | + |
| 37 | +### Step 1: Measure |
| 38 | + |
| 39 | +**No baseline = no optimization.** Before touching any code, capture reproducible numbers. |
| 40 | + |
| 41 | +#### Multi-Stack Profiling Tools |
| 42 | + |
| 43 | +| Language | Profiler | Benchmark Tool | |
| 44 | +|----------|----------|----------------| |
| 45 | +| Rust | `cargo flamegraph`, `perf`, `flamegraph` | `criterion`, `cargo bench` | |
| 46 | +| Python | `cProfile`, `py-spy` | `pytest-benchmark`, `timeit` | |
| 47 | +| Go | `pprof` (built-in) | `go test -bench`, `testing.B` | |
| 48 | +| General | `time`, `perf stat`, `hyperfine` | Custom harness with warm-up runs | |
| 49 | + |
| 50 | +#### What to Capture |
| 51 | + |
| 52 | +```bash |
| 53 | +# Rust: criterion benchmark with baseline |
| 54 | +cd project && cargo bench -- --save-baseline before |
| 55 | + |
| 56 | +# Python: profile a specific function |
| 57 | +python -m cProfile -s cumtime script.py > profile_before.txt |
| 58 | + |
| 59 | +# Go: CPU profile |
| 60 | +go test -bench=BenchmarkHotPath -cpuprofile=cpu_before.prof ./... |
| 61 | + |
| 62 | +# General: wall-clock timing with statistical rigor |
| 63 | +hyperfine --warmup 3 './my_command' |
| 64 | +``` |
| 65 | + |
| 66 | +Record these numbers in a file or commit message. You need them for Step 4. |
| 67 | + |
| 68 | +### Step 2: Identify |
| 69 | + |
| 70 | +**The bottleneck is rarely where you think.** Use profiling data to find it. |
| 71 | + |
| 72 | +``` |
| 73 | +Where is time spent? |
| 74 | +|-- CPU-bound |
| 75 | +| |-- Hot loop? --> Check algorithmic complexity, data structures |
| 76 | +| |-- Redundant computation? --> Cache or hoist invariants |
| 77 | +| +-- Serialized work? --> Parallelize across cores |
| 78 | +|-- Memory-bound |
| 79 | +| |-- Excessive allocation? --> Reuse buffers, pre-allocate |
| 80 | +| |-- Cache thrashing? --> Improve data locality |
| 81 | +| +-- Unbounded growth? --> Add size limits, eviction |
| 82 | +|-- I/O-bound |
| 83 | +| |-- Database? --> Check queries, indexes, N+1 patterns |
| 84 | +| |-- Network? --> Batch requests, connection pooling |
| 85 | +| |-- Disk? --> Buffer I/O, async operations |
| 86 | ++-- Concurrency |
| 87 | + |-- Lock contention? --> Fine-grained locks, lock-free structures |
| 88 | + |-- Thread starvation? --> Tune pool sizes |
| 89 | + +-- Synchronous blocking in async? --> Use non-blocking I/O |
| 90 | +``` |
| 91 | + |
| 92 | +#### Reading Profiler Output |
| 93 | + |
| 94 | +```bash |
| 95 | +# Rust: generate flamegraph |
| 96 | +cargo flamegraph --bin my_binary -- --my-args |
| 97 | +# Look for wide bars (time) and tall stacks (deep call chains) |
| 98 | + |
| 99 | +# Python: interactive profile browser |
| 100 | +python -m cProfile -o profile.prof script.py |
| 101 | +python -m pstats profile.prof |
| 102 | +# sort by cumulative time: sort cumtime |
| 103 | + |
| 104 | +# Go: interactive pprof |
| 105 | +go tool pprof cpu_before.prof |
| 106 | +# top10, list FunctionName, web (visualize) |
| 107 | +``` |
| 108 | + |
| 109 | +### Step 3: Fix |
| 110 | + |
| 111 | +**Address the measured bottleneck only.** Do not "clean up" nearby code. Do not optimize multiple things at once. |
| 112 | + |
| 113 | +Common fixes by bottleneck type: |
| 114 | + |
| 115 | +**Algorithmic:** |
| 116 | +- Replace O(n^2) with O(n log n) or O(n) where possible |
| 117 | +- Use appropriate data structures (hashmap for lookup, sorted array for range queries) |
| 118 | +- Hoist loop invariants, eliminate redundant computation |
| 119 | + |
| 120 | +**Memory:** |
| 121 | +- Pre-allocate buffers to known sizes |
| 122 | +- Reuse allocations in hot loops (object pools, arena allocators) |
| 123 | +- Stream large data instead of loading entirely into memory |
| 124 | + |
| 125 | +**I/O and Database:** |
| 126 | +- Add indexes for filtered/sorted columns |
| 127 | +- Replace N+1 queries with joins or batch loading |
| 128 | +- Paginate unbounded queries |
| 129 | +- Use connection pooling |
| 130 | + |
| 131 | +**Concurrency:** |
| 132 | +- Replace global locks with fine-grained or reader-writer locks |
| 133 | +- Use lock-free data structures for contended hot paths |
| 134 | +- Move CPU-bound work to dedicated threads/processes |
| 135 | + |
| 136 | +### Step 4: Verify |
| 137 | + |
| 138 | +**Re-measure with the same methodology as Step 1.** Compare directly. |
| 139 | + |
| 140 | +```bash |
| 141 | +# Rust: compare against saved baseline |
| 142 | +cargo bench -- --baseline before |
| 143 | + |
| 144 | +# Python: compare profiles |
| 145 | +python -m cProfile -s cumtime script.py > profile_after.txt |
| 146 | +diff profile_before.txt profile_after.txt |
| 147 | + |
| 148 | +# Go: compare benchmarks |
| 149 | +go test -bench=BenchmarkHotPath -count=5 ./... > after.txt |
| 150 | +benchstat before.txt after.txt |
| 151 | + |
| 152 | +# General: side-by-side comparison |
| 153 | +hyperfine --warmup 3 './old_command' './new_command' |
| 154 | +``` |
| 155 | + |
| 156 | +**Decision gate:** |
| 157 | +- Measurably faster (statistically significant) --> proceed to Step 5 |
| 158 | +- Not measurably faster --> **revert the change**. It added complexity without benefit |
| 159 | +- Faster but broke correctness --> **revert**. Correctness always wins |
| 160 | + |
| 161 | +### Step 5: Guard |
| 162 | + |
| 163 | +**Add a regression guard so the improvement sticks.** |
| 164 | + |
| 165 | +```bash |
| 166 | +# Rust: criterion benchmark checked in CI |
| 167 | +# Add benchmark to benches/ directory, run in CI pipeline |
| 168 | + |
| 169 | +# Python: pytest-benchmark with --benchmark-autosave |
| 170 | +pytest --benchmark-autosave tests/benchmarks/ |
| 171 | + |
| 172 | +# Go: benchmark test committed alongside unit tests |
| 173 | +# go test -bench=. runs automatically with test suite |
| 174 | + |
| 175 | +# General: performance budget in CI |
| 176 | +# Fail the build if response time > threshold |
| 177 | +``` |
| 178 | + |
| 179 | +Guard types (choose at least one): |
| 180 | +- **Benchmark test**: committed to repo, runs in CI, fails on regression |
| 181 | +- **Performance budget**: threshold in CI config (e.g., "p95 < 200ms") |
| 182 | +- **Monitoring alert**: production metric alert for latency/throughput |
| 183 | +- **Size budget**: binary/artifact size threshold |
| 184 | + |
| 185 | +For CI performance gates, see the `flow-code-cicd` skill. |
| 186 | + |
| 187 | +## Amdahl's Law Reminder |
| 188 | + |
| 189 | +``` |
| 190 | +Speedup = 1 / ((1 - P) + P/S) |
| 191 | +
|
| 192 | +P = fraction of time in the bottleneck |
| 193 | +S = speedup factor of your optimization |
| 194 | +
|
| 195 | +Example: Bottleneck is 10% of runtime. You make it 10x faster. |
| 196 | +Speedup = 1 / (0.9 + 0.1/10) = 1 / 0.91 = 1.10x (only 10% faster!) |
| 197 | +
|
| 198 | +Lesson: Only the dominant bottleneck matters. A 2x speedup on 80% of |
| 199 | +runtime beats a 100x speedup on 5% of runtime. |
| 200 | +``` |
| 201 | + |
| 202 | +## Common Rationalizations |
| 203 | + |
| 204 | +| Excuse | Reality | |
| 205 | +|--------|---------| |
| 206 | +| "I know where the bottleneck is" | Profile first. Intuition is wrong >50% of the time. The actual bottleneck surprises even experienced engineers. | |
| 207 | +| "This optimization is obvious" | Obvious optimizations often aren't. Compilers and runtimes already optimize the obvious stuff. Measure to confirm. | |
| 208 | +| "We don't have time to benchmark" | You don't have time to optimize blindly either. A 10-minute benchmark saves hours of wasted effort on the wrong bottleneck. | |
| 209 | +| "It's fast enough" | Without a baseline, you don't know what "enough" means. Define the target, measure against it. | |
| 210 | +| "Let's optimize everything" | Amdahl's Law: only the bottleneck matters. Optimizing non-bottleneck code adds complexity for zero user-visible improvement. | |
| 211 | +| "We'll add benchmarks later" | Later never comes. The regression ships silently, and you re-discover it in production under load. | |
| 212 | +| "The fix is small, no need to re-measure" | Small fixes can have surprising effects (positive or negative). The measurement takes minutes. Just do it. | |
| 213 | + |
| 214 | +## Red Flags |
| 215 | + |
| 216 | +- Optimizing without profiling data to justify the target |
| 217 | +- Multiple optimizations applied simultaneously (can't isolate which helped) |
| 218 | +- No before/after numbers in the commit message or PR description |
| 219 | +- Optimization that makes code significantly harder to read without measured justification |
| 220 | +- "Performance refactor" that changes architecture without baseline measurements |
| 221 | +- Benchmarks that don't use warm-up runs or statistical methods (single-run timings are noise) |
| 222 | +- Reverting to debug builds or removing optimizations "temporarily" for debugging |
| 223 | + |
| 224 | +## Verification |
| 225 | + |
| 226 | +After any performance-related change, confirm: |
| 227 | + |
| 228 | +- [ ] Baseline measurements captured before optimization (specific numbers, not "it felt slow") |
| 229 | +- [ ] Profiler identified the actual bottleneck (not assumed from code reading) |
| 230 | +- [ ] After-measurements show statistically significant improvement over baseline |
| 231 | +- [ ] Regression guard added (benchmark test, CI budget, or monitoring alert) |
| 232 | +- [ ] Optimization did not break correctness (full test suite passes) |
| 233 | +- [ ] Before/after numbers documented in commit message or PR description |
| 234 | +- [ ] No unrelated "while I'm here" optimizations bundled in the same change |
| 235 | + |
| 236 | +## References |
| 237 | + |
| 238 | +- Detailed checklist: `references/performance-checklist.md` |
| 239 | +- For CI performance gates, see the `flow-code-cicd` skill |
| 240 | +- For debugging performance issues, see the `flow-code-debug` skill |
0 commit comments