Skip to content

👌 Replace gitignore-parser with ignore-python and add per-project follow_links#64

Merged
chrisjsewell merged 4 commits into
mainfrom
copilot/replace-file-discovery-code
Apr 24, 2026
Merged

👌 Replace gitignore-parser with ignore-python and add per-project follow_links#64
chrisjsewell merged 4 commits into
mainfrom
copilot/replace-file-discovery-code

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 24, 2026

Replaces gitignore-parser with ignore-python (Rust ignore crate bindings) for file discovery.
This provides native nested .gitignore support, improved performance,
and behavioral parity with ubcode.
Adds per-project follow_links config.

Dependency swap

  • gitignore-parser>=0.1.11ignore-python>=0.3.3

Source discovery rewrite (source_discover/source_discover.py)

  • WalkBuilder replaces Path.rglob() + gitignore_parser.parse_gitignore() + fnmatch
  • Include/exclude patterns mapped to OverrideBuilder globs
  • File type filtering stays in Python (overrides would take precedence over gitignore rules)
  • Non-existent source directories now return empty list (WalkBuilder raises; old rglob silently yielded nothing)

Walker filter alignment with ubc_codelinks

The walker configuration now replicates the Rust ignore crate's
standard_filters(gitignore) followed by hidden(false),
matching ubcode
behaviour exactly. Since the Python ignore-python bindings don't expose
standard_filters(), we set all six individual filter methods:

builder.ignore(gitignore)     # .ignore file support
builder.parents(gitignore)    # parent ignore files
builder.git_ignore(gitignore) # .gitignore
builder.git_global(gitignore) # global gitignore
builder.git_exclude(gitignore)# .git/info/exclude
builder.hidden(False)         # always show hidden files

Previously, git_global and git_exclude were hardcoded to False and
.ignore/parent ignore support was not configured.

New follow_links config

  • follow_links: bool = False added to SourceDiscoverConfig, both TypedDicts, CLI discover command, and src_trace directive passthrough

Semantic change: include/exclude

With the ignore crate's override system, include whitelists files (overriding gitignore),
then exclude removes from that set.
Previously include had absolute priority over both gitignore and exclude.

[codelinks.projects.myproject.source_discover]
src_dir = "./src"
follow_links = true
gitignore = true
include = ["**/*.cpp"]
exclude = ["build/*"]

Shared portable test fixture

Added a shared JSON test fixture (tests/data/discover_fixtures.json)
with 18 test cases covering all discovery behaviours.
The same fixture is consumed by both this project and ubc_codelinks
to verify matching behaviour across the Python and Rust implementations.

Test cases cover:

  • Basic file extension discovery
  • .gitignore (root-level and nested)
  • .ignore file support
  • include and exclude patterns (individually and combined)
  • Default include derived from language extensions
  • Hidden file discovery
  • All comment types (cpp, python, rust, cs, yaml)
  • Non-existent src_dir edge case
  • Deeply nested directory trees
  • All C++ file extensions

Documentation updates

  • configuration.rst: Added follow_links option docs, expanded gitignore section to list all supported ignore sources (nested .gitignore, .ignore, .git/info/exclude, global gitignore, parent ignore files), removed "Nested .gitignore is NOT supported" limitation note, corrected include/exclude priority description
  • discover.rst: Added follow_links to advanced filtering example
  • roadmap.rst: Marked nested .gitignore support as completed

Copilot AI and others added 2 commits April 24, 2026 12:30
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.04%. Comparing base (a2a79e4) to head (8fbce16).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #64      +/-   ##
==========================================
+ Coverage   89.80%   90.04%   +0.23%     
==========================================
  Files          29       29              
  Lines        2560     2621      +61     
  Branches      300      306       +6     
==========================================
+ Hits         2299     2360      +61     
  Misses        165      165              
  Partials       96       96              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chrisjsewell chrisjsewell marked this pull request as ready for review April 24, 2026 14:07
@chrisjsewell chrisjsewell changed the title ✨ Replace gitignore-parser with ignore-python and add per-project follow_links 👌 Replace gitignore-parser with ignore-python and add per-project follow_links Apr 24, 2026
@chrisjsewell chrisjsewell merged commit 32baf80 into main Apr 24, 2026
9 checks passed
@chrisjsewell chrisjsewell deleted the copilot/replace-file-discovery-code branch April 24, 2026 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants