Skip to content

fix(grep): bound filesystem scans - #483

Merged
ScriptedAlchemy merged 3 commits into
ScriptedAlchemy:masterfrom
HashemKhalifa:fix/grep-scan-cancellation
Aug 3, 2026
Merged

fix(grep): bound filesystem scans#483
ScriptedAlchemy merged 3 commits into
ScriptedAlchemy:masterfrom
HashemKhalifa:fix/grep-scan-cancellation

Conversation

@HashemKhalifa

Copy link
Copy Markdown
Contributor

Summary

  • run tracedecay_grep filesystem traversal and reads off the async runtime
  • bound concurrent grep scans to two and time each request out after 10 seconds
  • cooperatively cancel timed-out or abandoned blocking workers
  • prune generated dependency trees and TraceDecay/Git metadata
  • skip files larger than 2 MB before reading them

Root cause

tracedecay_grep performed a synchronous WalkBuilder traversal and
std::fs::read directly inside its async MCP handler. On repositories whose
ignore files did not cover generated dependency trees, the walker could enter
large directories such as virtual environments. When a client timed out or
disconnected, the daemon-side scan continued, and concurrent requests could
repeat the same work. The CLI fallback used the same daemon and scanner, so it
could stall behind the original request.

Behavior

  • at most two grep scans run concurrently
  • the 10-second timeout includes time waiting for a scan permit
  • timeout or caller cancellation signals the blocking worker, which checks
    cancellation during traversal, before file reads, and while matching lines
  • timeout errors suggest narrowing the request with path_glob
  • the shared generated-directory classifier prunes dependency/build output
  • .git and .tracedecay are always excluded, including linked-worktree
    .git files
  • files larger than 2,000,000 bytes are skipped using metadata before
    std::fs::read

This is intentionally separate from #481. It does not change worktree identity
resolution, daemon lifecycle, project routing, or store migration.

Validation

  • TDD regression before the fix: generated .venv match reproduced
  • grep handler unit regressions: 3 passed
  • shared AST cancellation/off-thread regressions: 2 passed
  • MCP generated-directory integration regression: 1 passed
  • cargo fmt --all -- --check: passed
  • git diff --check: passed
  • conventional commit validation: passed
  • independent exact-tip audit of
    2b48fb4b94e3bed156c307275ce428351e45f6c8: PASS, no findings

The target worktree was not registered in the local TraceDecay project index,
so semantic impact lookup was unavailable without initializing another store.
Validation stayed bounded to the exact diff and focused compiled regressions.

@changeset-bot

changeset-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: bac90ab

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ScriptedAlchemy
ScriptedAlchemy marked this pull request as ready for review August 3, 2026 02:25

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security review found no malicious indicators or auth/network/secret/workflow/dependency tampering. The generated-directory whitelist regression was fixed with TDD at the current head; scoped re-review passed with no remaining Critical or Important findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bac90abe71

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +270 to +273
let may_match_descendants = wildcard_suffix
.iter()
.enumerate()
.any(|(index, segment)| index > 0 || *segment == "**");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Handle slashless globs as matching descendants

When path_glob is a basename pattern such as *.js, wildcard_suffix contains only one non-** segment, so may_match_descendants becomes false. Under the handler's documented .gitignore-style glob semantics, that pattern matches basenames at any depth; however, for dist/app.js, filter_entry rejects the generated dist directory before the override can match the file, causing an explicit filtered search to return a false negative. Treat slashless positive patterns as potentially matching descendants.

Useful? React with 👍 / 👎.

@ScriptedAlchemy
ScriptedAlchemy merged commit be36952 into ScriptedAlchemy:master Aug 3, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants