Skip to content

Commit e19943f

Browse files
committed
docs(watch): preserve preliminary benchmark context
1 parent 5224c38 commit e19943f

1 file changed

Lines changed: 80 additions & 0 deletions

File tree

docs/watch-benchmark.md

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Watch Mode Benchmark: Evented vs Polling
2+
3+
Benchmark comparing the old 250ms polling watch implementation (npm `hunkdiff@0.17.0`) against the evented Chokidar-based hybrid observer (`elucid/file-watch` PR #531).
4+
5+
## Setup
6+
7+
- **Repo:** `modem/modem` (large production monorepo)
8+
- **Branch state:** dirty working tree with untracked files
9+
- **OLD binary:** `/Users/justin/.npm-global/bin/hunk` (npm release 0.17.0, 250ms `setInterval` polling)
10+
- **NEW binary:** `/Users/justin/.local/bin/hunk` (PR build from `elucid/file-watch`, Chokidar + 10s safety poll)
11+
- **Command:** `hunk diff --watch`
12+
13+
## Method
14+
15+
### Git subprocess counting
16+
17+
A wrapper script was placed ahead of real `git` on PATH. It logs every invocation with a timestamp to a per-version log file, then `exec`s the real git binary:
18+
19+
```bash
20+
#!/bin/bash
21+
printf '%s %s\n' "$(date +%s.%N)" "$*" >> "${HUNK_BENCH_GIT_LOG}"
22+
exec /nix/store/.../git "$@"
23+
```
24+
25+
Both versions were launched in separate herdr panes with `HUNK_BENCH_GIT_LOG` and `PATH` set via `--env` on `herdr pane split`. After both rendered their initial diff, the log files were zeroed and CPU times recorded. The benchmark then sampled every 10 seconds for 60 seconds using `ps -p $PID -o cputime=` and `wc -l` on the log files.
26+
27+
### Startup time
28+
29+
Both versions were launched sequentially in herdr panes on the same repo. A polling loop read each pane's visible content via `herdr pane read` at ~20ms intervals, looking for hunk's menu bar (`File View`). The elapsed time from `herdr pane run` to first detection was recorded. Each version was quit (`q`) and relaunched between trials. Five trials were run per version.
30+
31+
## Results
32+
33+
### Idle overhead (60 seconds, no file changes)
34+
35+
Sampled at 10-second intervals:
36+
37+
```
38+
t=10s OLD cpu=0:02.68 git= 199 | NEW cpu=0:01.61 git= 6
39+
t=20s OLD cpu=0:02.93 git= 319 | NEW cpu=0:01.63 git= 9
40+
t=30s OLD cpu=0:03.14 git= 439 | NEW cpu=0:01.65 git= 12
41+
t=40s OLD cpu=0:03.35 git= 559 | NEW cpu=0:01.67 git= 15
42+
t=50s OLD cpu=0:03.55 git= 679 | NEW cpu=0:01.69 git= 18
43+
t=60s OLD cpu=0:03.76 git= 799 | NEW cpu=0:01.71 git= 21
44+
```
45+
46+
| Metric | OLD (polling) | NEW (evented) | Improvement |
47+
| -------------------------------- | ------------- | ------------- | ------------- |
48+
| Git invocations in 60s | 799 | 21 | **38× fewer** |
49+
| Git calls/second | ~13.3/s | ~0.35/s ||
50+
| CPU time consumed (idle portion) | ~1.50s | ~0.13s | **~12× less** |
51+
52+
The old version fires multiple git commands per 250ms poll cycle (~13/s). The new version fires a small batch every ~10 seconds (safety poll only).
53+
54+
### Startup time (5 trials)
55+
56+
| Trial | OLD | NEW |
57+
| -------- | ---------- | ---------- |
58+
| 1 | 1708ms | 1674ms |
59+
| 2 | 1669ms | 1703ms |
60+
| 3 | 1671ms | 1670ms |
61+
| 4 | 1662ms | 1756ms |
62+
| 5 | 1684ms | 1691ms |
63+
| **Mean** | **1679ms** | **1699ms** |
64+
65+
Startup time is identical within noise (~1.7s). Both versions are dominated by the initial `git diff` load cost on this repo. The PR's improvement is entirely in the idle steady-state after the diff is rendered.
66+
67+
### Refresh latency (evented PR, separate E2E test repo)
68+
69+
| Scenario | Latency |
70+
| --------------------------- | ------- |
71+
| Simple tracked-file write | ~340ms |
72+
| Atomic temp-file + rename | ~418ms |
73+
| Large 241-line diff update | ~406ms |
74+
| New untracked file creation | ~682ms |
75+
76+
All refreshes were passive — no keyboard or mouse input was sent to hunk.
77+
78+
## Conclusion
79+
80+
The evented observer eliminates virtually all idle CPU and subprocess overhead while maintaining identical startup performance and sub-second passive refresh latency. The improvement scales with the number of open `--watch` sessions: each idle tab drops from ~13 git calls/second to ~0.35.

0 commit comments

Comments
 (0)