Skip to content

Commit cdb4a0c

Browse files
committed
fix(extensions): use pathspec for gitignore-compatible .extensionignore matching
Replace fnmatch with pathspec.GitIgnoreSpec to get proper .gitignore semantics where * does not cross directory boundaries. This addresses review feedback on #1781. Changes: - Switch from fnmatch to pathspec>=0.12.0 (GitIgnoreSpec.from_lines) - Normalize backslashes in patterns for cross-platform compatibility - Distinguish directories from files for trailing-slash patterns - Update docs to accurately describe supported pattern semantics - Add edge-case tests: .., absolute paths, empty file, backslashes, * vs ** boundary behavior, and ! negation - Move changelog entry to [Unreleased] section
1 parent bcc8f30 commit cdb4a0c

File tree

5 files changed

+226
-24
lines changed

5 files changed

+226
-24
lines changed

CHANGELOG.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,12 @@ Recent changes to the Specify CLI and templates are documented here.
77
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
88
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
99

10+
## [Unreleased]
11+
12+
### Added
13+
14+
- feat(extensions): support `.extensionignore` to exclude files/folders during `specify extension add` (#1781)
15+
1016
## [0.2.0] - 2026-03-09
1117

1218
### Changed
@@ -62,7 +68,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
6268
- Higher-priority catalogs win on merge conflicts (same extension id in multiple catalogs)
6369
- 13 new tests covering catalog stack resolution, merge conflicts, URL validation, and `install_allowed` enforcement
6470
- Updated RFC, Extension User Guide, and Extension API Reference documentation
65-
- feat(extensions): support `.extensionignore` to exclude files/folders during `specify extension add` (#1781)
6671

6772
## [0.1.13] - 2026-03-03
6873

extensions/EXTENSION-DEVELOPMENT-GUIDE.md

Lines changed: 25 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -338,11 +338,18 @@ Extension authors can create a `.extensionignore` file in the extension root to
338338

339339
### Format
340340

341-
The file uses glob-style patterns, one per line:
341+
The file uses `.gitignore`-compatible patterns (one per line), powered by the [`pathspec`](https://pypi.org/project/pathspec/) library:
342342

343343
- Blank lines are ignored
344344
- Lines starting with `#` are comments
345-
- Patterns are matched against both the file/folder name and its path relative to the extension root
345+
- `*` matches anything **except** `/` (does not cross directory boundaries)
346+
- `**` matches zero or more directories (e.g., `docs/**/*.draft.md`)
347+
- `?` matches any single character except `/`
348+
- A trailing `/` restricts a pattern to directories only
349+
- Patterns containing `/` (other than a trailing slash) are anchored to the extension root
350+
- Patterns without `/` match at any depth in the tree
351+
- `!` negates a previously excluded pattern (re-includes a file)
352+
- Backslashes in patterns are normalised to forward slashes for cross-platform compatibility
346353
- The `.extensionignore` file itself is always excluded automatically
347354

348355
### Example
@@ -367,12 +374,22 @@ CONTRIBUTING.md
367374

368375
### Pattern Matching
369376

370-
| Pattern | Matches |
371-
|---------|---------|
372-
| `*.pyc` | Any `.pyc` file in any directory |
373-
| `tests/` | The `tests` directory (and all its contents) |
374-
| `docs/*.draft.md` | Draft markdown files inside `docs/` |
375-
| `.github` | The `.github` directory at any level |
377+
| Pattern | Matches | Does NOT match |
378+
|---------|---------|----------------|
379+
| `*.pyc` | Any `.pyc` file in any directory | — |
380+
| `tests/` | The `tests` directory (and all its contents) | A file named `tests` |
381+
| `docs/*.draft.md` | `docs/api.draft.md` (directly inside `docs/`) | `docs/sub/api.draft.md` (nested) |
382+
| `.env` | The `.env` file at any level | — |
383+
| `!README.md` | Re-includes `README.md` even if matched by an earlier pattern | — |
384+
| `docs/**/*.draft.md` | `docs/api.draft.md`, `docs/sub/api.draft.md` | — |
385+
386+
### Unsupported Features
387+
388+
The following `.gitignore` features are **not applicable** in this context:
389+
390+
- **Multiple `.extensionignore` files**: Only a single file at the extension root is supported (`.gitignore` supports files in subdirectories)
391+
- **`$GIT_DIR/info/exclude` and `core.excludesFile`**: These are Git-specific and have no equivalent here
392+
- **Negation inside excluded directories**: Because file copying uses `shutil.copytree`, excluding a directory prevents recursion into it entirely. A negation pattern cannot re-include a file inside a directory that was itself excluded. For example, the combination `tests/` followed by `!tests/important.py` will **not** preserve `tests/important.py` — the `tests/` directory is skipped at the root level and its contents are never evaluated. To work around this, exclude the directory's contents individually instead of the directory itself (e.g., `tests/*.pyc` and `tests/.cache/` rather than `tests/`).
376393

377394
---
378395

pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ dependencies = [
1313
"truststore>=0.10.4",
1414
"pyyaml>=6.0",
1515
"packaging>=23.0",
16+
"pathspec>=0.12.0",
1617
]
1718

1819
[project.scripts]

src/specify_cli/extensions.py

Lines changed: 30 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@
66
without bloating the core framework.
77
"""
88

9-
import fnmatch
109
import json
1110
import hashlib
1211
import os
@@ -19,6 +18,8 @@
1918
from datetime import datetime, timezone
2019
import re
2120

21+
import pathspec
22+
2223
import yaml
2324
from packaging import version as pkg_version
2425
from packaging.specifiers import SpecifierSet, InvalidSpecifier
@@ -285,10 +286,18 @@ def __init__(self, project_root: Path):
285286
def _load_extensionignore(source_dir: Path) -> Optional[Callable[[str, List[str]], Set[str]]]:
286287
"""Load .extensionignore and return an ignore function for shutil.copytree.
287288
288-
The .extensionignore file uses glob-style patterns (one per line).
289+
The .extensionignore file uses .gitignore-compatible patterns (one per line).
289290
Lines starting with '#' are comments. Blank lines are ignored.
290291
The .extensionignore file itself is always excluded.
291292
293+
Pattern semantics mirror .gitignore:
294+
- '*' matches anything except '/'
295+
- '**' matches zero or more directories
296+
- '?' matches any single character except '/'
297+
- Trailing '/' restricts a pattern to directories only
298+
- Patterns with '/' (other than trailing) are anchored to the root
299+
- '!' negates a previously excluded pattern
300+
292301
Args:
293302
source_dir: Path to the extension source directory
294303
@@ -300,14 +309,22 @@ def _load_extensionignore(source_dir: Path) -> Optional[Callable[[str, List[str]
300309
if not ignore_file.exists():
301310
return None
302311

303-
patterns: List[str] = []
304-
for line in ignore_file.read_text().splitlines():
312+
lines: List[str] = ignore_file.read_text().splitlines()
313+
314+
# Normalise backslashes in patterns so Windows-authored files work
315+
normalised: List[str] = []
316+
for line in lines:
305317
stripped = line.strip()
306318
if stripped and not stripped.startswith("#"):
307-
patterns.append(stripped)
319+
normalised.append(stripped.replace("\\", "/"))
320+
else:
321+
# Preserve blanks/comments so pathspec line numbers stay stable
322+
normalised.append(line)
308323

309324
# Always ignore the .extensionignore file itself
310-
patterns.append(".extensionignore")
325+
normalised.append(".extensionignore")
326+
327+
spec = pathspec.GitIgnoreSpec.from_lines(normalised)
311328

312329
def _ignore(directory: str, entries: List[str]) -> Set[str]:
313330
ignored: Set[str] = set()
@@ -316,17 +333,15 @@ def _ignore(directory: str, entries: List[str]) -> Set[str]:
316333
rel_path = str(rel_dir / entry) if str(rel_dir) != "." else entry
317334
# Normalise to forward slashes for consistent matching
318335
rel_path_fwd = rel_path.replace("\\", "/")
319-
for pattern in patterns:
320-
# Strip trailing slash so "tests/" matches directory name "tests"
321-
pat = pattern.rstrip("/")
322-
# Match against the entry name itself
323-
if fnmatch.fnmatch(entry, pat):
336+
337+
entry_full = Path(directory) / entry
338+
if entry_full.is_dir():
339+
# Append '/' so directory-only patterns (e.g. tests/) match
340+
if spec.match_file(rel_path_fwd + "/"):
324341
ignored.add(entry)
325-
break
326-
# Match against the relative path from the source root
327-
if fnmatch.fnmatch(rel_path_fwd, pat):
342+
else:
343+
if spec.match_file(rel_path_fwd):
328344
ignored.add(entry)
329-
break
330345
return ignored
331346

332347
return _ignore

tests/test_extensions.py

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1774,3 +1774,167 @@ def test_extensionignore_relative_path_match(self, temp_dir, valid_manifest_data
17741774
dest = proj_dir / ".specify" / "extensions" / "test-ext"
17751775
assert (dest / "docs" / "guide.md").exists()
17761776
assert not (dest / "docs" / "internal" / "draft.md").exists()
1777+
1778+
def test_extensionignore_dotdot_pattern_is_noop(self, temp_dir, valid_manifest_data):
1779+
"""Patterns with '..' should not escape the extension root."""
1780+
ext_dir = self._make_extension(
1781+
temp_dir,
1782+
valid_manifest_data,
1783+
extra_files={"README.md": "# Hello"},
1784+
ignore_content="../sibling/\n",
1785+
)
1786+
1787+
proj_dir = temp_dir / "project"
1788+
proj_dir.mkdir()
1789+
(proj_dir / ".specify").mkdir()
1790+
1791+
manager = ExtensionManager(proj_dir)
1792+
manager.install_from_directory(ext_dir, "0.1.0", register_commands=False)
1793+
1794+
dest = proj_dir / ".specify" / "extensions" / "test-ext"
1795+
# Everything should still be copied — the '..' pattern matches nothing inside
1796+
assert (dest / "README.md").exists()
1797+
assert (dest / "extension.yml").exists()
1798+
assert (dest / "commands" / "hello.md").exists()
1799+
1800+
def test_extensionignore_absolute_path_pattern_is_noop(self, temp_dir, valid_manifest_data):
1801+
"""Absolute path patterns should not match anything."""
1802+
ext_dir = self._make_extension(
1803+
temp_dir,
1804+
valid_manifest_data,
1805+
extra_files={"README.md": "# Hello", "passwd": "sensitive"},
1806+
ignore_content="/etc/passwd\n",
1807+
)
1808+
1809+
proj_dir = temp_dir / "project"
1810+
proj_dir.mkdir()
1811+
(proj_dir / ".specify").mkdir()
1812+
1813+
manager = ExtensionManager(proj_dir)
1814+
manager.install_from_directory(ext_dir, "0.1.0", register_commands=False)
1815+
1816+
dest = proj_dir / ".specify" / "extensions" / "test-ext"
1817+
# Nothing matches — /etc/passwd is anchored to root and there's no 'etc' dir
1818+
assert (dest / "README.md").exists()
1819+
assert (dest / "passwd").exists()
1820+
1821+
def test_extensionignore_empty_file(self, temp_dir, valid_manifest_data):
1822+
"""An empty .extensionignore should exclude only itself."""
1823+
ext_dir = self._make_extension(
1824+
temp_dir,
1825+
valid_manifest_data,
1826+
extra_files={"README.md": "# Hello", "notes.txt": "notes"},
1827+
ignore_content="",
1828+
)
1829+
1830+
proj_dir = temp_dir / "project"
1831+
proj_dir.mkdir()
1832+
(proj_dir / ".specify").mkdir()
1833+
1834+
manager = ExtensionManager(proj_dir)
1835+
manager.install_from_directory(ext_dir, "0.1.0", register_commands=False)
1836+
1837+
dest = proj_dir / ".specify" / "extensions" / "test-ext"
1838+
assert (dest / "README.md").exists()
1839+
assert (dest / "notes.txt").exists()
1840+
assert (dest / "extension.yml").exists()
1841+
# .extensionignore itself is still excluded
1842+
assert not (dest / ".extensionignore").exists()
1843+
1844+
def test_extensionignore_windows_backslash_patterns(self, temp_dir, valid_manifest_data):
1845+
"""Backslash patterns (Windows-style) are normalised to forward slashes."""
1846+
ext_dir = self._make_extension(
1847+
temp_dir,
1848+
valid_manifest_data,
1849+
extra_files={
1850+
"docs/internal/draft.md": "draft",
1851+
"docs/guide.md": "# Guide",
1852+
},
1853+
ignore_content="docs\\internal\\draft.md\n",
1854+
)
1855+
1856+
proj_dir = temp_dir / "project"
1857+
proj_dir.mkdir()
1858+
(proj_dir / ".specify").mkdir()
1859+
1860+
manager = ExtensionManager(proj_dir)
1861+
manager.install_from_directory(ext_dir, "0.1.0", register_commands=False)
1862+
1863+
dest = proj_dir / ".specify" / "extensions" / "test-ext"
1864+
assert (dest / "docs" / "guide.md").exists()
1865+
assert not (dest / "docs" / "internal" / "draft.md").exists()
1866+
1867+
def test_extensionignore_star_does_not_cross_directories(self, temp_dir, valid_manifest_data):
1868+
"""'*' should NOT match across directory boundaries (gitignore semantics)."""
1869+
ext_dir = self._make_extension(
1870+
temp_dir,
1871+
valid_manifest_data,
1872+
extra_files={
1873+
"docs/api.draft.md": "draft",
1874+
"docs/sub/api.draft.md": "nested draft",
1875+
},
1876+
ignore_content="docs/*.draft.md\n",
1877+
)
1878+
1879+
proj_dir = temp_dir / "project"
1880+
proj_dir.mkdir()
1881+
(proj_dir / ".specify").mkdir()
1882+
1883+
manager = ExtensionManager(proj_dir)
1884+
manager.install_from_directory(ext_dir, "0.1.0", register_commands=False)
1885+
1886+
dest = proj_dir / ".specify" / "extensions" / "test-ext"
1887+
# docs/*.draft.md should only match directly inside docs/, NOT subdirs
1888+
assert not (dest / "docs" / "api.draft.md").exists()
1889+
assert (dest / "docs" / "sub" / "api.draft.md").exists()
1890+
1891+
def test_extensionignore_doublestar_crosses_directories(self, temp_dir, valid_manifest_data):
1892+
"""'**' should match across directory boundaries."""
1893+
ext_dir = self._make_extension(
1894+
temp_dir,
1895+
valid_manifest_data,
1896+
extra_files={
1897+
"docs/api.draft.md": "draft",
1898+
"docs/sub/api.draft.md": "nested draft",
1899+
"docs/guide.md": "guide",
1900+
},
1901+
ignore_content="docs/**/*.draft.md\n",
1902+
)
1903+
1904+
proj_dir = temp_dir / "project"
1905+
proj_dir.mkdir()
1906+
(proj_dir / ".specify").mkdir()
1907+
1908+
manager = ExtensionManager(proj_dir)
1909+
manager.install_from_directory(ext_dir, "0.1.0", register_commands=False)
1910+
1911+
dest = proj_dir / ".specify" / "extensions" / "test-ext"
1912+
assert not (dest / "docs" / "api.draft.md").exists()
1913+
assert not (dest / "docs" / "sub" / "api.draft.md").exists()
1914+
assert (dest / "docs" / "guide.md").exists()
1915+
1916+
def test_extensionignore_negation_pattern(self, temp_dir, valid_manifest_data):
1917+
"""'!' negation re-includes a previously excluded file."""
1918+
ext_dir = self._make_extension(
1919+
temp_dir,
1920+
valid_manifest_data,
1921+
extra_files={
1922+
"docs/guide.md": "# Guide",
1923+
"docs/internal.md": "internal",
1924+
"docs/api.md": "api",
1925+
},
1926+
ignore_content="docs/*.md\n!docs/api.md\n",
1927+
)
1928+
1929+
proj_dir = temp_dir / "project"
1930+
proj_dir.mkdir()
1931+
(proj_dir / ".specify").mkdir()
1932+
1933+
manager = ExtensionManager(proj_dir)
1934+
manager.install_from_directory(ext_dir, "0.1.0", register_commands=False)
1935+
1936+
dest = proj_dir / ".specify" / "extensions" / "test-ext"
1937+
# docs/*.md excludes all .md in docs, but !docs/api.md re-includes it
1938+
assert not (dest / "docs" / "guide.md").exists()
1939+
assert not (dest / "docs" / "internal.md").exists()
1940+
assert (dest / "docs" / "api.md").exists()

0 commit comments

Comments
 (0)