Skip to content

Commit 5027134

Browse files
committed
refactor: resolve tool versions dynamically from PyPI
Replace the hardcoded version lists (versions.py, update_versions.yml) with dynamic resolution via PyPI's JSON API at hook runtime. Why: - Eliminates the need for weekly cron jobs + manual PR reviews to keep version lists current - New PyPI releases are instantly available — no lag - Adding a new tool (e.g. clang-include-cleaner) requires zero changes to version infrastructure - The hardcoded approach was a shadow copy of PyPI's ground truth, creating inconsistency risks Changes: - util.py: Add _get_pypi_versions() (PyPI JSON + stable filtering + lru_cache) and _resolve_version_from_pypi() for dynamic prefix matching - Delete versions.py — no more hardcoded lists - Delete scripts/update_versions.py and update-versions.yml — obsolete - Update release-drafter.yml — versions are now always latest from PyPI - Update copilot-instructions.md to reflect new architecture - tests/test_util.py: 45 tests covering dynamic resolution, caching, network failure, prefix matching, and all four tools
1 parent 879fd5e commit 5027134

7 files changed

Lines changed: 307 additions & 690 deletions

File tree

.github/copilot-instructions.md

Lines changed: 8 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,9 @@ Pre-commit hooks wrapper that auto-installs and runs clang-format and clang-tidy
1414
- **`clang_format.py`**: Wraps clang-format with `-i` (in-place), supports `--verbose` and `--dry-run` modes
1515
- Returns `-1` for dry-run to distinguish from actual failures
1616
- **`clang_tidy.py`**: Wraps clang-tidy, forces exit code 1 if "warning:" or "error:" in output
17-
- **`util.py`**: Version resolution and pip-based tool installation
18-
- `_resolve_version()`: Supports partial matches (e.g., "20" → "20.1.7")
19-
- `DEFAULT_CLANG_FORMAT_VERSION` and `DEFAULT_CLANG_TIDY_VERSION` read from `pyproject.toml`
20-
- **`versions.py`**: Auto-generated by `scripts/update_versions.py` (runs weekly via GitHub Actions)
17+
- **`util.py`**: Dynamic version resolution via PyPI JSON API + pip-based tool installation
18+
- `_resolve_version_from_pypi()`: Supports partial matches (e.g., "20" → "20.1.8"), resolves against PyPI in real time
19+
- No hardcoded version lists — versions are always up-to-date from PyPI
2120

2221
### Version Management Pattern
2322
```python
@@ -49,8 +48,8 @@ uv run pytest -m benchmark # Performance tests only
4948

5049
### Dependency Management
5150
- **Uses `uv`** for all dev operations (not pip directly)
52-
- **Pin versions**: Default tool versions in `pyproject.toml` dependencies section
53-
- **Update versions**: Run `python scripts/update_versions.py` (auto-runs weekly on Monday 2 AM UTC)
51+
- **Tool versions**: Resolved dynamically from PyPI at hook runtime — no manual updates needed
52+
- **Version caching**: `_get_pypi_versions()` uses `lru_cache` to avoid repeated PyPI requests within a single run
5453

5554
## Project-Specific Conventions
5655

@@ -78,8 +77,8 @@ command = ["tool-name"] + other_args # Pass through unknown args
7877

7978
## Critical Files
8079

81-
- **`pyproject.toml`**: Defines entry points, dependencies, default versions
82-
- **`versions.py`**: Auto-updated; DO NOT edit manually (see comment)
80+
- **`pyproject.toml`**: Defines entry points and dependencies
81+
- **`util.py`**: Core version resolution + tool installation logic
8382
- **`.pre-commit-hooks.yaml`**: Hook metadata for pre-commit framework
8483
- **`testing/run.sh`**: Integration test script used in CI
8584

@@ -102,10 +101,7 @@ command = ["tool-name"] + other_args # Pass through unknown args
102101
2. Pass to subprocess command or handle in Python
103102
3. Add test case in `tests/test_*.py`
104103

105-
**Update default tool versions:**
106-
1. Edit `dependencies` in `pyproject.toml`
107-
2. Run tests to ensure compatibility
108-
3. Update version in README examples
104+
**No manual version updates needed:** Tool versions are resolved dynamically from PyPI at runtime. When new versions are published to PyPI, hooks automatically discover them — no code changes required.
109105

110106
**Debug hook failures:**
111107
- Add `--verbose` to clang-format args for detailed output

.github/workflows/release-drafter.yml

Lines changed: 9 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -19,26 +19,17 @@ jobs:
1919
- name: Checkout repository
2020
uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0
2121

22-
- name: Extract default tool versions
23-
id: versions
22+
- name: Generate release notes header
2423
run: |
25-
# Get versions directly from versions.py (no package install needed)
26-
CLANG_FORMAT_VERSION=$(python3 -c "import sys; sys.path.insert(0, 'cpp_linter_hooks'); from versions import CLANG_FORMAT_VERSIONS; print(CLANG_FORMAT_VERSIONS[-1])")
27-
CLANG_TIDY_VERSION=$(python3 -c "import sys; sys.path.insert(0, 'cpp_linter_hooks'); from versions import CLANG_TIDY_VERSIONS; print(CLANG_TIDY_VERSIONS[-1])")
28-
29-
# Export to GitHub Actions environment for subsequent steps
30-
echo "CLANG_FORMAT_VERSION=$CLANG_FORMAT_VERSION" >> $GITHUB_ENV
31-
echo "CLANG_TIDY_VERSION=$CLANG_TIDY_VERSION" >> $GITHUB_ENV
32-
33-
# Log for debug
34-
echo "Default clang-format version: $CLANG_FORMAT_VERSION"
35-
echo "Default clang-tidy version: $CLANG_TIDY_VERSION"
36-
37-
# Generate release notes file
38-
echo "## 💡 Default Clang Tool Version" > release_notes.md
39-
echo "clang-format: \`$CLANG_FORMAT_VERSION\` · clang-tidy: \`$CLANG_TIDY_VERSION\`" >> release_notes.md
24+
echo "## 💡 Default Clang Tool Versions" > release_notes.md
25+
echo "" >> release_notes.md
26+
echo "Versions are resolved **dynamically from PyPI** at hook runtime. " >> release_notes.md
27+
echo "The latest stable versions are always used by default — " >> release_notes.md
28+
echo "no manual updates required." >> release_notes.md
4029
echo "" >> release_notes.md
41-
echo "You can override the default versions for by adding the \`--version\` argument under \`args\` in your pre-commit config. See [Custom Clang Tool Version](https://github.com/cpp-linter/cpp-linter-hooks?tab=readme-ov-file#custom-clang-tool-version) for details." >> release_notes.md
30+
echo "You can pin a specific version by adding the \`--version\` argument " >> release_notes.md
31+
echo "under \`args\` in your pre-commit config. " >> release_notes.md
32+
echo "See [Custom Clang Tool Version](https://github.com/cpp-linter/cpp-linter-hooks?tab=readme-ov-file#custom-clang-tool-version) for details." >> release_notes.md
4233
echo "" >> release_notes.md
4334
cat release_notes.md
4435

.github/workflows/update-versions.yml

Lines changed: 0 additions & 86 deletions
This file was deleted.

cpp_linter_hooks/util.py

Lines changed: 73 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -5,113 +5,89 @@
55
import subprocess
66
from pathlib import Path
77
import logging
8-
from typing import Optional, List, Tuple
9-
10-
if sys.version_info >= (3, 11):
11-
import tomllib
12-
else:
13-
import tomli as tomllib
14-
15-
from cpp_linter_hooks.versions import (
16-
CLANG_FORMAT_VERSIONS,
17-
CLANG_TIDY_VERSIONS,
18-
CLANG_INCLUDE_CLEANER_VERSIONS,
19-
CLANG_APPLY_REPLACEMENTS_VERSIONS,
20-
)
8+
from typing import Optional, Tuple
9+
from functools import lru_cache
10+
import json
11+
import urllib.request
12+
import re
2113

2214
LOG = logging.getLogger(__name__)
2315

2416

25-
def get_version_from_dependency(tool: str) -> Optional[str]:
26-
"""Get the version of a tool from the pyproject.toml dependencies."""
27-
pyproject_path = Path(__file__).parent.parent / "pyproject.toml"
28-
if not pyproject_path.exists():
29-
return None
30-
with open(pyproject_path, "rb") as f:
31-
data = tomllib.load(f)
32-
# Check [project].dependencies
33-
dependencies = data.get("project", {}).get("dependencies", [])
34-
for dep in dependencies:
35-
if dep.startswith(f"{tool}=="):
36-
return dep.split("==")[1]
37-
return None
17+
@lru_cache(maxsize=4)
18+
def _get_pypi_versions(tool: str) -> Tuple[Optional[str], list]:
19+
"""Fetch (latest_version, [stable_versions_descending]) from PyPI JSON API.
3820
21+
Results are cached per tool name so repeated calls within the same
22+
process reuse the last HTTP response.
23+
"""
24+
try:
25+
url = f"https://pypi.org/pypi/{tool}/json"
26+
with urllib.request.urlopen(url, timeout=10) as response:
27+
data = json.loads(response.read())
28+
except Exception as exc:
29+
LOG.warning("Failed to fetch versions for %s from PyPI: %s", tool, exc)
30+
return None, []
31+
32+
all_versions = list(data["releases"].keys())
33+
34+
# Filter out pre-release versions
35+
pre_release_pattern = re.compile(
36+
r".*(alpha|beta|rc|dev|a\d+|b\d+).*", re.IGNORECASE
37+
)
38+
stable = [v for v in all_versions if not pre_release_pattern.match(v)]
3939

40-
DEFAULT_CLANG_FORMAT_VERSION = CLANG_FORMAT_VERSIONS[-1] # latest from versions.py
41-
DEFAULT_CLANG_TIDY_VERSION = CLANG_TIDY_VERSIONS[-1] # latest from versions.py
42-
DEFAULT_CLANG_INCLUDE_CLEANER_VERSION = CLANG_INCLUDE_CLEANER_VERSIONS[-1]
43-
DEFAULT_CLANG_APPLY_REPLACEMENTS_VERSION = CLANG_APPLY_REPLACEMENTS_VERSIONS[-1]
44-
40+
if not stable:
41+
LOG.warning("No stable versions found for %s on PyPI", tool)
42+
return None, []
4543

46-
def _versions_for_tool(tool: str) -> List[str]:
47-
"""Return supported Python wheel versions for a clang tool."""
48-
_tool_map = {
49-
"clang-format": CLANG_FORMAT_VERSIONS,
50-
"clang-tidy": CLANG_TIDY_VERSIONS,
51-
"clang-include-cleaner": CLANG_INCLUDE_CLEANER_VERSIONS,
52-
"clang-apply-replacements": CLANG_APPLY_REPLACEMENTS_VERSIONS,
53-
}
54-
return _tool_map[tool]
44+
# Sort ascending by version tuple
45+
stable.sort(key=lambda x: tuple(map(int, x.split("."))))
5546

47+
latest = stable[-1]
48+
# Return descending for prefix matching (newest first)
49+
return latest, list(reversed(stable))
5650

57-
def _default_version_for_tool(tool: str) -> Optional[str]:
58-
"""Return the default Python wheel version for a clang tool."""
59-
_default_map = {
60-
"clang-format": DEFAULT_CLANG_FORMAT_VERSION,
61-
"clang-tidy": DEFAULT_CLANG_TIDY_VERSION,
62-
"clang-include-cleaner": DEFAULT_CLANG_INCLUDE_CLEANER_VERSION,
63-
"clang-apply-replacements": DEFAULT_CLANG_APPLY_REPLACEMENTS_VERSION,
64-
}
65-
return _default_map[tool]
6651

52+
def _resolve_version_from_pypi(
53+
tool: str, user_input: Optional[str]
54+
) -> Tuple[Optional[str], Optional[str]]:
55+
"""Resolve a version dynamically from PyPI.
6756
68-
def _supported_versions_message(tool: str) -> str:
69-
"""Build a user-facing list of supported wheel versions for a clang tool."""
70-
versions = ", ".join(_versions_for_tool(tool))
71-
return f"Supported {tool} wheel versions: {versions}"
57+
Returns (resolved_version, error_message). The error_message is
58+
suitable for displaying directly to the end user.
59+
"""
60+
latest, versions = _get_pypi_versions(tool)
7261

62+
if not versions:
63+
return (
64+
None,
65+
f"Could not find any stable versions of {tool} on PyPI. "
66+
"Check your network connection.",
67+
)
7368

74-
def _resolve_version(versions: List[str], user_input: Optional[str]) -> Optional[str]:
75-
"""Resolve the latest matching version based on user input and available versions."""
7669
if user_input is None:
77-
return None
78-
if user_input in versions:
79-
return user_input
80-
81-
try:
82-
# filter versions that start with the user input
83-
matched_versions = [v for v in versions if v.startswith(user_input)]
84-
if not matched_versions:
85-
raise ValueError
86-
87-
# define a function to parse version strings into tuples for comparison
88-
def parse_version(v: str):
89-
"""Convert a dotted version string into an integer tuple."""
90-
return tuple(map(int, v.split(".")))
91-
92-
# return the latest version
93-
return max(matched_versions, key=parse_version)
94-
95-
except ValueError:
96-
LOG.warning("Version %s not found in available versions", user_input)
97-
return None
70+
return latest, None
9871

72+
# Exact match
73+
if user_input in versions:
74+
return user_input, None
9975

100-
def resolve_tool_version(
101-
tool: str, version: Optional[str]
102-
) -> Tuple[Optional[str], Optional[str]]:
103-
"""Resolve a requested tool version or return a user-facing error message."""
104-
if version is None:
105-
return _default_version_for_tool(tool), None
76+
# Prefix match (e.g. "20" → "20.1.8"). Versions are newest-first,
77+
# so the first matching entry is the latest for that prefix.
78+
matched = [v for v in versions if v.startswith(user_input)]
79+
if matched:
80+
return matched[0], None
10681

107-
resolved = _resolve_version(_versions_for_tool(tool), version)
108-
if resolved is None:
109-
return (
110-
None,
111-
f"Unsupported {tool} version '{version}'.\n"
112-
f"{_supported_versions_message(tool)}",
113-
)
114-
return resolved, None
82+
# No match – help the user
83+
sample = ", ".join(versions[:15])
84+
return (
85+
None,
86+
f"Unsupported {tool} version '{user_input}'.\n"
87+
f"Latest stable version: {latest}\n"
88+
f"Available versions (sample): {sample}\n"
89+
f"Run `pip index versions {tool}` to see all available versions.",
90+
)
11591

11692

11793
def _is_version_installed(tool: str, version: str) -> Optional[Path]:
@@ -143,15 +119,19 @@ def _install_tool(tool: str, version: str) -> Optional[Path]:
143119
def resolve_install_with_diagnostics(
144120
tool: str, version: Optional[str], verbose: bool = False
145121
) -> Tuple[Optional[Path], Optional[str]]:
146-
"""Resolve/install a tool, returning a user-facing error for bad versions."""
147-
user_version, error = resolve_tool_version(tool, version)
122+
"""Resolve/install a tool, returning a user-facing error for bad versions.
123+
124+
Tool versions are resolved dynamically from PyPI — no hardcoded
125+
list is maintained in-tree.
126+
"""
127+
user_version, error = _resolve_version_from_pypi(tool, version)
148128
if error is not None:
149129
return None, error
150130

151131
if verbose:
152132
if version is None:
153133
print(
154-
f"Using default {tool} Python wheel version {user_version}",
134+
f"Using latest {tool} Python wheel version {user_version}",
155135
file=sys.stderr,
156136
)
157137
elif version == user_version:

0 commit comments

Comments
 (0)