Skip to content

Commit 601178a

Browse files
a-orenclaude
andauthored
feat(python): add uv package manager support (TC-3855) (guacsec#428)
## Summary - Add `PythonUvProvider` and `PythonProviderFactory` for automatic uv package manager detection via `uv.lock` presence - Dependency resolution uses `uv pip list --format=json` + `uv pip show` for CycloneDX SBOM generation (stack + component) - Extract shared TOML parsing into `PyprojectTomlUtils`, refactor both `PythonPyprojectProvider` and `PythonUvProvider` to use it - Fix PEP 508 extras stripping bug in `getDependencyName()` (e.g. `requests[security]` → `requests`) affecting all Python providers - Document uv support, `TRUSTIFY_DA_UV_PATH`, and CLI examples in README ## Test plan - [x] `mvn test -Dtest="Python_Uv_Provider_Test" -Dskip.junit_platform=true` — 15 uv provider tests pass - [x] `mvn test -Dtest="Python_Pyproject_Provider_Test" -Dskip.junit_platform=true` — 22 existing pyproject tests pass (regression) - [x] `mvn test -Dtest="PythonControllerRealEnvTest#get_Dependency_Name*" -Dskip.junit_platform=true` — extras fix tests pass - [x] `mvn spotless:check` — formatting passes - [x] Manual CLI test: `java -jar trustify-da-java-client-cli.jar sbom /path/to/uv-project/pyproject.toml` produces correct CycloneDX SBOM ## Acceptance Criteria - [x] `PythonProviderFactory.create()` returns `PythonUvProvider` when `uv.lock` exists - [x] `PythonProviderFactory.create()` falls back to `PythonPyprojectProvider` when no `uv.lock` - [x] `PythonUvProvider` discovers uv binary via `Operations.getExecutable("uv", "--version")` - [x] Custom binary path supported via `TRUSTIFY_DA_UV_PATH` - [x] `validateLockFile()` throws `IllegalStateException` when `uv.lock` missing - [x] `provideStack()` and `provideComponent()` generate valid CycloneDX SBOMs - [x] Existing `requirements.txt` and `pyproject.toml` flows unchanged - [x] `#trustify-da-ignore` / `#exhortignore` patterns work in uv context - [x] README documents uv support 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Add uv-based Python provider selection and shared pyproject.toml utilities while preserving existing pip/pyproject behavior. New Features: - Introduce PythonUvProvider to generate CycloneDX SBOMs for uv-managed pyproject.toml projects based on uv export output. - Add PythonProviderFactory to automatically choose between uv and pip-based pyproject providers using uv.lock detection, including workspace-aware lock-file lookup. Bug Fixes: - Ensure Python dependency name parsing strips PEP 508 extras before extracting the base package name, preventing incorrect identifiers like requests[security]. Enhancements: - Refactor pyproject.toml parsing and metadata extraction into shared PyprojectTomlUtils used by both pip and uv Python providers. - Extend Python pyproject provider to reuse shared canonicalization, license lookup, and ignore-pattern handling logic. CI: - Install the uv tool in the pull request workflow to support uv-related tests. Documentation: - Document uv package manager support for Python, including automatic lock-file based selection, custom TRUSTIFY_DA_UV_PATH configuration, and CLI SBOM usage examples. Tests: - Add comprehensive unit tests for PythonUvProvider, including graph parsing, SBOM generation, and ignore patterns for uv projects. - Add tests for PythonProviderFactory lock-file discovery across workspaces and for dependency name parsing with PEP 508 extras in real Python environments. --------- Signed-off-by: Adva Oren <aoren@redhat.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent c0688e0 commit 601178a

18 files changed

Lines changed: 1619 additions & 63 deletions

File tree

.github/workflows/pr.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,8 @@ jobs:
4343
go-version: '1.20.1'
4444
- name: Install pnpm
4545
run: npm install -g pnpm
46+
- name: Install uv
47+
uses: astral-sh/setup-uv@v6
4648
- name: get Python location
4749
id: python-location
4850
run: |

README.md

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,7 +182,7 @@ public class TrustifyExample {
182182
<li><a href="https://www.java.com/">Java</a> - <a href="https://maven.apache.org/">Maven</a></li>
183183
<li><a href="https://www.javascript.com//">JavaScript</a> - <a href="https://www.npmjs.com//">Npm</a></li>
184184
<li><a href="https://go.dev//">Golang</a> - <a href="https://go.dev/blog/using-go-modules//">Go Modules</a></li>
185-
<li><a href="https://www.python.org/">Python</a> - <a href="https://pypi.org/project/pip/">pip Installer</a> (<code>requirements.txt</code>, <code>pyproject.toml</code> with PEP 621 format). <strong>Note:</strong> Poetry-style dependencies (<code>[tool.poetry.dependencies]</code>) are not supported.</li>
185+
<li><a href="https://www.python.org/">Python</a> - <a href="https://pypi.org/project/pip/">pip Installer</a> / <a href="https://docs.astral.sh/uv/">uv</a> (<code>requirements.txt</code>, <code>pyproject.toml</code> with PEP 621 format). <strong>Note:</strong> Poetry-style dependencies (<code>[tool.poetry.dependencies]</code>) are not supported.</li>
186186
<li><a href="https://gradle.org//">Gradle</a> - <a href="https://gradle.org/install//">Gradle Installation</a></li>
187187
<li><a href="https://www.rust-lang.org/">Rust</a> - <a href="https://doc.rust-lang.org/cargo/">Cargo</a></li>
188188

@@ -393,6 +393,7 @@ System.setProperty("TRUSTIFY_DA_PYTHON3_PATH", "/path/to/python3");
393393
System.setProperty("TRUSTIFY_DA_PIP3_PATH", "/path/to/pip3");
394394
System.setProperty("TRUSTIFY_DA_PYTHON_PATH", "/path/to/python");
395395
System.setProperty("TRUSTIFY_DA_PIP_PATH", "/path/to/pip");
396+
System.setProperty("TRUSTIFY_DA_UV_PATH", "/path/to/custom/uv");
396397
// Configure proxy for all requests
397398
System.setProperty("TRUSTIFY_DA_PROXY_URL", "http://proxy.example.com:8080");
398399
// Configure Maven settings and repository
@@ -504,6 +505,11 @@ following keys for setting custom paths for the said executables.
504505
<td>TRUSTIFY_DA_PIP_PATH</td>
505506
</tr>
506507
<tr>
508+
<td><a href="https://docs.astral.sh/uv/">uv Package Manager</a></td>
509+
<td><em>uv</em></td>
510+
<td>TRUSTIFY_DA_UV_PATH</td>
511+
</tr>
512+
<tr>
507513
<td><a href="https://doc.rust-lang.org/cargo/">Cargo Package Manager</a></td>
508514
<td><em>cargo</em></td>
509515
<td>TRUSTIFY_DA_CARGO_PATH</td>
@@ -600,6 +606,14 @@ TRUSTIFY_DA_GO_MVS_LOGIC_ENABLED=false
600606

601607
Python support works with both `requirements.txt` and `pyproject.toml` manifest files. The `pyproject.toml` provider scans production dependencies from PEP 621 `[project.dependencies]` and Poetry `[tool.poetry.dependencies]` sections. Optional dependencies (`[project.optional-dependencies]`) and Poetry group dependencies (`[tool.poetry.group.*.dependencies]`) are intentionally excluded to focus on runtime dependencies.
602608

609+
##### uv Support
610+
611+
When a `uv.lock` file is present alongside `pyproject.toml`, the client automatically uses the [uv](https://docs.astral.sh/uv/) package manager for dependency resolution instead of pip. Dependency data is collected via `uv export --format requirements.txt --frozen --no-hashes --no-dev`. No additional configuration is required — the provider is selected automatically based on lock file detection.
612+
613+
To use a custom uv binary, set `TRUSTIFY_DA_UV_PATH` to the path of your uv executable.
614+
615+
##### pip Support
616+
603617
By default, Python support assumes that the package is installed using the pip/pip3 binary on the system PATH, or of the customized
604618
Binaries passed to environment variables. If the package is not installed , then an error will be thrown.
605619

@@ -683,6 +697,12 @@ Options:
683697
- `--output <path>` - Write SBOM JSON to the specified file
684698
- (default) - Print SBOM JSON to stdout
685699

700+
**SBOM for uv-managed Python project**
701+
```shell
702+
# When uv.lock is present alongside pyproject.toml, the uv provider is used automatically
703+
java -jar trustify-da-java-client-cli.jar sbom /path/to/uv-project/pyproject.toml
704+
```
705+
686706
**Image Analysis**
687707
```shell
688708
java -jar trustify-da-java-client-cli.jar image <image_ref> [<image_ref>...] [--summary|--html]
Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
/*
2+
* Copyright 2023-2025 Trustify Dependency Analytics Authors
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
*
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
package io.github.guacsec.trustifyda.providers;
18+
19+
import io.github.guacsec.trustifyda.tools.Operations;
20+
import io.github.guacsec.trustifyda.utils.Environment;
21+
import io.github.guacsec.trustifyda.utils.PyprojectTomlUtils;
22+
import java.nio.file.Files;
23+
import java.nio.file.Path;
24+
import java.util.Map;
25+
import java.util.function.Function;
26+
27+
/**
28+
* Factory for creating the appropriate {@link PythonProvider} based on lock file presence. Follows
29+
* the same pattern as {@link JavaScriptProviderFactory}.
30+
*/
31+
public final class PythonProviderFactory {
32+
33+
private static final Map<String, Function<Path, PythonProvider>> PYTHON_PROVIDERS =
34+
Map.of(PythonUvProvider.LOCK_FILE, PythonUvProvider::new);
35+
36+
/**
37+
* Creates a Python provider for {@code pyproject.toml} manifests by checking for known lock files
38+
* in the manifest directory and parent directories (for workspace members). When {@code uv.lock}
39+
* is present, returns a {@link PythonUvProvider}; otherwise falls back to {@link
40+
* PythonPyprojectProvider} (pip-based).
41+
*
42+
* @param manifestPath the path to the pyproject.toml manifest
43+
* @return the matching Python provider
44+
*/
45+
public static PythonProvider create(final Path manifestPath) {
46+
var manifestDir = manifestPath.getParent();
47+
48+
// Check manifest directory first (fast path)
49+
for (var entry : PYTHON_PROVIDERS.entrySet()) {
50+
if (Files.isRegularFile(manifestDir.resolve(entry.getKey()))) {
51+
return entry.getValue().apply(manifestPath);
52+
}
53+
}
54+
55+
// Walk up parent directories to find lock file at workspace root
56+
Path lockFileDir = findLockFileDirInParents(manifestDir);
57+
if (lockFileDir != null) {
58+
for (var entry : PYTHON_PROVIDERS.entrySet()) {
59+
if (Files.isRegularFile(lockFileDir.resolve(entry.getKey()))) {
60+
return entry.getValue().apply(manifestPath);
61+
}
62+
}
63+
}
64+
65+
// Unlike JavaScript, pip fallback is valid — no lock file required
66+
return new PythonPyprojectProvider(manifestPath);
67+
}
68+
69+
/**
70+
* Walks up from the given directory to find a parent containing a known Python lock file.
71+
* Respects {@code TRUSTIFY_DA_WORKSPACE_DIR} override, stops at uv workspace root boundaries and
72+
* the git root.
73+
*
74+
* @param startDir the directory to start searching from (typically the manifest directory)
75+
* @return the directory containing the lock file, or {@code null} if not found
76+
*/
77+
static Path findLockFileDirInParents(Path startDir) {
78+
// Environment override takes precedence
79+
String workspaceDirOverride = Environment.get("TRUSTIFY_DA_WORKSPACE_DIR");
80+
if (workspaceDirOverride != null && !workspaceDirOverride.isBlank()) {
81+
Path overrideDir = Path.of(workspaceDirOverride);
82+
for (String lockFile : PYTHON_PROVIDERS.keySet()) {
83+
if (Files.isRegularFile(overrideDir.resolve(lockFile))) {
84+
return overrideDir;
85+
}
86+
}
87+
return null;
88+
}
89+
90+
// If startDir itself is a workspace root, don't walk up to avoid escaping into a parent
91+
// workspace
92+
if (PyprojectTomlUtils.isUvWorkspaceRoot(startDir)) {
93+
return null;
94+
}
95+
96+
String gitRoot = Operations.getGitRootDir(startDir.toString()).orElse(null);
97+
Path boundary = gitRoot != null ? Path.of(gitRoot) : startDir.toAbsolutePath().getRoot();
98+
99+
Path current = startDir.toAbsolutePath().normalize().getParent();
100+
while (current != null) {
101+
for (String lockFile : PYTHON_PROVIDERS.keySet()) {
102+
if (Files.isRegularFile(current.resolve(lockFile))) {
103+
return current;
104+
}
105+
}
106+
107+
// Stop at uv workspace root boundary
108+
if (PyprojectTomlUtils.isUvWorkspaceRoot(current)) {
109+
return null;
110+
}
111+
112+
if (current.equals(boundary.toAbsolutePath().normalize())) {
113+
break;
114+
}
115+
116+
current = current.getParent();
117+
}
118+
119+
return null;
120+
}
121+
}

src/main/java/io/github/guacsec/trustifyda/providers/PythonPyprojectProvider.java

Lines changed: 15 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
import io.github.guacsec.trustifyda.sbom.SbomFactory;
2626
import io.github.guacsec.trustifyda.tools.Operations;
2727
import io.github.guacsec.trustifyda.utils.Environment;
28+
import io.github.guacsec.trustifyda.utils.PyprojectTomlUtils;
2829
import io.github.guacsec.trustifyda.utils.PythonControllerBase;
2930
import java.io.IOException;
3031
import java.nio.charset.StandardCharsets;
@@ -42,10 +43,8 @@
4243
import java.util.regex.Matcher;
4344
import java.util.regex.Pattern;
4445
import java.util.stream.Collectors;
45-
import org.tomlj.Toml;
4646
import org.tomlj.TomlArray;
4747
import org.tomlj.TomlParseResult;
48-
import org.tomlj.TomlTable;
4948

5049
/**
5150
* Provider for Python projects using {@code pyproject.toml} with <a
@@ -133,7 +132,7 @@ private void addDependencyTree(
133132
PackageURL packageURL = toPurl(pkg.name, pkg.version);
134133
sbom.addDependency(source, packageURL, null);
135134

136-
String key = canonicalize(pkg.name);
135+
String key = PyprojectTomlUtils.canonicalize(pkg.name);
137136
if (!visited.add(key)) {
138137
return;
139138
}
@@ -266,7 +265,7 @@ PipReportData parsePipReport(String reportJson) throws IOException {
266265
}
267266
String name = extractDepName(reqStr);
268267
if (name != null) {
269-
directDepNames.add(canonicalize(name));
268+
directDepNames.add(PyprojectTomlUtils.canonicalize(name));
270269
}
271270
}
272271
}
@@ -285,7 +284,7 @@ PipReportData parsePipReport(String reportJson) throws IOException {
285284
if (name == null) {
286285
continue;
287286
}
288-
String key = canonicalize(name);
287+
String key = PyprojectTomlUtils.canonicalize(name);
289288
graph.put(key, new PipPackage(name, version, new ArrayList<>()));
290289
}
291290

@@ -299,7 +298,7 @@ PipReportData parsePipReport(String reportJson) throws IOException {
299298
if (name == null) {
300299
continue;
301300
}
302-
String key = canonicalize(name);
301+
String key = PyprojectTomlUtils.canonicalize(name);
303302
PipPackage pipPkg = graph.get(key);
304303
if (pipPkg == null) {
305304
continue;
@@ -317,7 +316,7 @@ PipReportData parsePipReport(String reportJson) throws IOException {
317316
if (depName == null) {
318317
continue;
319318
}
320-
String depKey = canonicalize(depName);
319+
String depKey = PyprojectTomlUtils.canonicalize(depName);
321320
if (graph.containsKey(depKey)) {
322321
pipPkg.children.add(depKey);
323322
}
@@ -338,28 +337,18 @@ static String extractDepName(String req) {
338337
return m.find() ? m.group(1) : null;
339338
}
340339

341-
static String canonicalize(String name) {
342-
return name.toLowerCase().replaceAll("[-_.]+", "-");
343-
}
344-
345340
private TomlParseResult getToml() throws IOException {
346341
if (cachedToml == null) {
347-
TomlParseResult parsed = Toml.parse(manifestPath);
348-
if (parsed.hasErrors()) {
349-
throw new IOException(
350-
"Invalid pyproject.toml format: " + parsed.errors().get(0).getMessage());
351-
}
352-
cachedToml = parsed;
342+
cachedToml = PyprojectTomlUtils.parseToml(manifestPath);
353343
}
354344
return cachedToml;
355345
}
356346

357347
@Override
358348
protected String getRootComponentName() {
359349
try {
360-
TomlParseResult toml = getToml();
361-
String name = toml.getString("project.name");
362-
if (name != null && !name.isBlank()) {
350+
String name = PyprojectTomlUtils.getProjectName(getToml());
351+
if (name != null) {
363352
return name;
364353
}
365354
} catch (IOException e) {
@@ -371,9 +360,8 @@ protected String getRootComponentName() {
371360
@Override
372361
protected String getRootComponentVersion() {
373362
try {
374-
TomlParseResult toml = getToml();
375-
String version = toml.getString("project.version");
376-
if (version != null && !version.isBlank()) {
363+
String version = PyprojectTomlUtils.getProjectVersion(getToml());
364+
if (version != null) {
377365
return version;
378366
}
379367
} catch (IOException e) {
@@ -385,16 +373,10 @@ protected String getRootComponentVersion() {
385373
@Override
386374
public String readLicenseFromManifest() {
387375
try {
388-
TomlParseResult toml = getToml();
389-
String license = toml.getString("project.license");
390-
if (license != null && !license.isBlank()) {
376+
String license = PyprojectTomlUtils.getLicense(getToml());
377+
if (license != null) {
391378
return license;
392379
}
393-
// PEP 639: license may be in project.license.text
394-
String licenseText = toml.getString("project.license.text");
395-
if (licenseText != null && !licenseText.isBlank()) {
396-
return licenseText;
397-
}
398380
} catch (IOException e) {
399381
log.fine("Failed to parse pyproject.toml for license: " + e.getMessage());
400382
}
@@ -416,28 +398,15 @@ protected Set<PackageURL> getIgnoredDependencies(String manifestContent) {
416398
}
417399

418400
private void rejectPoetryDependencies() throws IOException {
419-
TomlParseResult toml = getToml();
420-
TomlTable poetryDeps = toml.getTable("tool.poetry.dependencies");
421-
if (poetryDeps != null) {
401+
if (PyprojectTomlUtils.hasPoetryDependencies(getToml())) {
422402
throw new IllegalStateException(
423403
"Poetry dependencies in pyproject.toml are not supported."
424404
+ " Please use PEP 621 [project.dependencies] format instead.");
425405
}
426406
}
427407

428408
private void collectIgnoredDeps() throws IOException {
429-
TomlParseResult toml = getToml();
430-
List<String> rawLines = Files.readAllLines(manifestPath);
431-
collectedIgnoredDeps = new HashSet<>();
432-
433-
// [project.dependencies] - PEP 621
434-
TomlArray projectDeps = toml.getArray("project.dependencies");
435-
if (projectDeps != null) {
436-
for (int i = 0; i < projectDeps.size(); i++) {
437-
String dep = projectDeps.getString(i);
438-
checkIgnored(rawLines, dep, dep);
439-
}
440-
}
409+
collectedIgnoredDeps = PyprojectTomlUtils.collectIgnoredDeps(manifestPath, getToml());
441410
}
442411

443412
List<String> parseDependencyStrings() throws IOException {
@@ -455,15 +424,6 @@ List<String> parseDependencyStrings() throws IOException {
455424
return deps;
456425
}
457426

458-
private void checkIgnored(List<String> rawLines, String searchToken, String depIdentifier) {
459-
for (String line : rawLines) {
460-
if (line.contains(searchToken) && containsIgnorePattern(line)) {
461-
collectedIgnoredDeps.add(depIdentifier);
462-
break;
463-
}
464-
}
465-
}
466-
467427
static final class PipPackage {
468428
final String name;
469429
final String version;

0 commit comments

Comments
 (0)