feat(linux): add bounded working-tree watcher#1063
Conversation
Electron's recursive Linux fs.watch implementation allocates inotify watches for every file and directory, exhausting per-user limits in large working trees. Add a disabled-by-default Linux feature that substitutes directory-only watching for working-tree watches while retaining recursive coverage through serialized topology reconciliation. Prune Git-ignored untracked directories without hiding tracked or force-added paths. Exclude .git from working-tree traversal while retaining Codex's dedicated Git subsystem and adding bounded index and info/exclude refresh watches. Share a conservative feature-specific process-wide inotify budget across working-tree and refresh watches, prioritize roots, distribute remaining capacity fairly, and retry partial, resource, and metadata coverage with bounded backoff and low-noise diagnostics. Keep Git queries and topology scans asynchronous so recovery does not block Electron. Wire the feature into Nix, watchdog and acceptance inclusion, documentation, and comprehensive tests against the current upstream bundle.
|
Failing CI check noted. It's late here now but I'll address this first thing in morning. A few things worthy of discussion:
However, that does make this first contribution quite comprehensive, instead of delaying the inevitable. |
831f16f to
53a95eb
Compare
ilysenko
left a comment
There was a problem hiding this comment.
Reviewed current head; no blockers found. The opt-in feature is isolated, covered by tests/CI, and the current-DMG feature patch was locally verified against 26.715.31251.
Summary
Fixes #983.
The upstream app targets macOS, where recursive filesystem watching can use an efficient tree-level backend. In the current Linux Electron runtime, translating this directly to
fs.watch(path, { recursive: true })allocates inotify watches for files as well as directories. Large dependency and build trees can therefore exhaust the per-user inotify pool.This adds
directory-only-working-tree-watchas a disabled-by-default Linux feature. Its runtime patch, descriptor, documentation, and focused tests live underlinux-features/directory-only-working-tree-watch/; the accompanying Nix and watchdog registrations expose and continuously validate it.When enabled, the feature:
.env.localstill produce events through their watched parent.node_modulesare pruned only when Git ignores them or the user explicitly configures their name..gitfrom ordinary working-tree traversal while retaining Codex’s dedicated Git subsystem and adding bounded refresh watches around the Git index and.git/info/exclude.fs.inotify.max_user_watches; an override is capped at 65536. Roots receive priority, with remaining capacity distributed fairly in bounded reservations.On my system, this reduced the app’s watch count from approximately 190,000 to approximately 900 with the default settings.
This is intentionally an opt-in Linux feature for initial field validation. If it proves stable, it should be reconsidered for default or core Linux compatibility: bounded watcher resource use is baseline platform behavior rather than an end-user feature. Before that promotion, persistent zero-watch coverage and operating-system resource exhaustion should receive actionable in-app notices, and a native recursive watcher backend should be evaluated as a possible simplification.
Validation
node --test linux-features/directory-only-working-tree-watch/test.js— 92/92 passed under Node.js 22, 24, and 25.node --test linux-features/*/test.js— 692/692 passed.node --test scripts/patch-linux-window-ui.test.js— 374/374 passed.node --test scripts/lib/linux-features.test.js— 10/10 passed.node --test scripts/ci/upstream-dmg-acceptance.test.js— 14/14 passed.bash tests/scripts_smoke.sh— passed.git diff --check upstream/main...HEAD— passed.831f16fb:26.715.21425ff459150991612007549270d2d28c5e78cec6bd6ac200a7ada5ed6c031369b87accepted_with_warningsnixis unavailable, or full.deb,.rpm, pacman, and AppImage artifact builds. Nix/feature registration tests and the cross-format script smoke suite passed.Known limitations:
.gitignore, index, and.git/info/excludechanges are refreshed dynamically.Checklist
CODEX.DMGand removes obsolete fallback code and tests from the affected area.