Skip to content

feat(linux): add bounded working-tree watcher#1063

Merged
ilysenko merged 2 commits into
ilysenko:mainfrom
gadicc:codex/directory-only-working-tree-watch
Jul 18, 2026
Merged

feat(linux): add bounded working-tree watcher#1063
ilysenko merged 2 commits into
ilysenko:mainfrom
gadicc:codex/directory-only-working-tree-watch

Conversation

@gadicc

@gadicc gadicc commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #983.

The upstream app targets macOS, where recursive filesystem watching can use an efficient tree-level backend. In the current Linux Electron runtime, translating this directly to fs.watch(path, { recursive: true }) allocates inotify watches for files as well as directories. Large dependency and build trees can therefore exhaust the per-user inotify pool.

This adds directory-only-working-tree-watch as a disabled-by-default Linux feature. Its runtime patch, descriptor, documentation, and focused tests live under linux-features/directory-only-working-tree-watch/; the accompanying Nix and watchdog registrations expose and continuously validate it.

When enabled, the feature:

  • Installs non-recursive watches on relevant directories, not individual files, while still receiving their child file events.
  • Within covered paths, reconciles directory creation, deletion, rename, moved-in trees, filename-less events, and same-path inode replacement.
  • Prunes only Git-ignored, untracked directories. Tracked and force-added paths remain covered, and ignored files such as .env.local still produce events through their watched parent.
  • Has no default name-based exclusions. Directories such as node_modules are pruned only when Git ignores them or the user explicitly configures their name.
  • Excludes .git from ordinary working-tree traversal while retaining Codex’s dedicated Git subsystem and adding bounded refresh watches around the Git index and .git/info/exclude.
  • Shares a process-wide budget across this feature’s working-tree and Git-refresh watches. The default limit is the smaller of 8192 and one eighth of the current kernel fs.inotify.max_user_watches; an override is capped at 65536. Roots receive priority, with remaining capacity distributed fairly in bounded reservations.
  • Preserves Codex’s focus-recovery path during partial coverage, expands coverage when capacity is released, and retries transient Git, filesystem, and watch-resource failures with bounded backoff.
  • Uses asynchronous Git subprocesses and promise-based, serialized topology scans so reconciliation does not synchronously block Electron.
  • Emits low-noise launcher-log diagnostics: one warning per partial-budget episode, one message after complete recovery, and one process-wide error for operating-system watch exhaustion.

On my system, this reduced the app’s watch count from approximately 190,000 to approximately 900 with the default settings.

This is intentionally an opt-in Linux feature for initial field validation. If it proves stable, it should be reconsidered for default or core Linux compatibility: bounded watcher resource use is baseline platform behavior rather than an end-user feature. Before that promotion, persistent zero-watch coverage and operating-system resource exhaustion should receive actionable in-app notices, and a native recursive watcher backend should be evaluated as a possible simplification.

Validation

  • node --test linux-features/directory-only-working-tree-watch/test.js — 92/92 passed under Node.js 22, 24, and 25.
  • node --test linux-features/*/test.js — 692/692 passed.
  • node --test scripts/patch-linux-window-ui.test.js — 374/374 passed.
  • node --test scripts/lib/linux-features.test.js — 10/10 passed.
  • node --test scripts/ci/upstream-dmg-acceptance.test.js — 14/14 passed.
  • bash tests/scripts_smoke.sh — passed.
  • git diff --check upstream/main...HEAD — passed.
  • Current-DMG rebuild from clean commit 831f16fb:
    • app version 26.715.21425
    • DMG SHA-256 ff459150991612007549270d2d28c5e78cec6bd6ac200a7ada5ed6c031369b87
    • feature patch applied successfully
    • acceptance verdict accepted_with_warnings
    • no blockers; all eight warnings were unrelated optional core-patch drift
  • Manual runtime measurement: approximately 190,000 watches before and approximately 900 after enabling the feature.
  • Not run locally: Nix evaluation because nix is unavailable, or full .deb, .rpm, pacman, and AppImage artifact builds. Nix/feature registration tests and the cross-format script smoke suite passed.

Known limitations:

  • Once the configured budget is full, immediate event coverage is deliberately partial until capacity becomes available. Codex’s focus refresh remains active, but does not itself rebuild watch topology.
  • Changes to a global Git excludes file require an app restart; .gitignore, index, and .git/info/exclude changes are refreshed dynamically.
  • Budget diagnostics include the affected local working-tree path in the existing local launcher log.
  • Persistent zero-watch coverage and operating-system watch exhaustion remain log-only for this opt-in release; actionable UI is explicitly deferred until before any default/core rollout.

Checklist

  • This pull request is ready for review and is no longer a draft.
  • I followed CONTRIBUTING.md, kept the change focused, edited source files rather than generated output, and removed unrelated changes.
  • If this fixes upstream drift, it targets only the latest CODEX.DMG and removes obsolete fallback code and tests from the affected area.
  • I added or updated relevant tests, ran the validation listed above, and confirmed that required CI checks pass.
  • I reviewed the final diff with my coding agent using maximum reasoning effort, addressed all findings, and reran the relevant tests.

Electron's recursive Linux fs.watch implementation allocates inotify watches for every file and directory, exhausting per-user limits in large working trees.

Add a disabled-by-default Linux feature that substitutes directory-only watching for working-tree watches while retaining recursive coverage through serialized topology reconciliation. Prune Git-ignored untracked directories without hiding tracked or force-added paths. Exclude .git from working-tree traversal while retaining Codex's dedicated Git subsystem and adding bounded index and info/exclude refresh watches.

Share a conservative feature-specific process-wide inotify budget across working-tree and refresh watches, prioritize roots, distribute remaining capacity fairly, and retry partial, resource, and metadata coverage with bounded backoff and low-noise diagnostics. Keep Git queries and topology scans asynchronous so recovery does not block Electron.

Wire the feature into Nix, watchdog and acceptance inclusion, documentation, and comprehensive tests against the current upstream bundle.
@gadicc

gadicc commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

Failing CI check noted. It's late here now but I'll address this first thing in morning.

A few things worthy of discussion:

  • Maybe an easier first pass would have been to simply replace fs.watch() with @parcel/watcher. Alone it doesn't cover as much as this patch, and also introduces a native C++ dependency, which could be a pro or a con -- we'd have to evaluate separately.

  • If the inotify budget concept is excessive for a first attempt. It definitely adds welcome value but perhaps is too ambitious.

  • Likewise for all the git handling.

However, that does make this first contribution quite comprehensive, instead of delaying the inevitable.

@gadicc
gadicc force-pushed the codex/directory-only-working-tree-watch branch from 831f16f to 53a95eb Compare July 18, 2026 00:09

@ilysenko ilysenko left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed current head; no blockers found. The opt-in feature is isolated, covered by tests/CI, and the current-DMG feature patch was locally verified against 26.715.31251.

@ilysenko
ilysenko merged commit b49b4d6 into ilysenko:main Jul 18, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex Desktop on Linux consumes ~65k inotify watches and exhausts the system limit

2 participants