Skip to content

feat(sbom): declare python dependencies via packages directive#144

Open
reyreavman wants to merge 7 commits into
mainfrom
feat/sbom/get-python-sbom
Open

feat(sbom): declare python dependencies via packages directive#144
reyreavman wants to merge 7 commits into
mainfrom
feat/sbom/get-python-sbom

Conversation

@reyreavman

@reyreavman reyreavman commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@reyreavman reyreavman force-pushed the feat/sbom/get-python-sbom branch 2 times, most recently from 0ed0bfe to a6af422 Compare July 2, 2026 10:58
@reyreavman reyreavman force-pushed the feat/sbom/get-python-sbom branch from a6af422 to 720a22f Compare July 10, 2026 08:05
@nervgh nervgh force-pushed the feat/sbom/get-python-sbom branch 2 times, most recently from d3c5844 to 34fdf77 Compare July 10, 2026 15:40
nervgh
nervgh requested changes Jul 13, 2026
View reviewed changes
Comment thread docs/_data/werf_yaml.yml Outdated
description:
en: "Directory inside the image that contains the module files (required for go-mod)"
ru: "Директория внутри образа, содержащая файлы модуля (обязательно для go-mod)"
en: "Directory inside the image that contains the manifest files (required for go-mod and python-*)"

@nervgh nervgh Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's align our doc with names of directives:

  • manifest -> spec

Comment thread test/e2e/sbom/_fixtures/inject/poetry_simple/werf.yaml Outdated
git:
- add: /
to: /app
shell:

@nervgh nervgh Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this section exist here? I mean the test should throw the error because of network constraints.

Comment thread test/e2e/sbom/_fixtures/inject/uv_simple/werf.yaml Outdated
git:
- add: /
to: /app
shell:

@nervgh nervgh Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same here

@reyreavman reyreavman force-pushed the feat/sbom/get-python-sbom branch 2 times, most recently from deea47c to 14f1a68 Compare July 13, 2026 19:46
@reyreavman reyreavman marked this pull request as draft July 13, 2026 20:03
@reyreavman reyreavman force-pushed the feat/sbom/get-python-sbom branch 2 times, most recently from 3e68803 to 4c901b1 Compare July 13, 2026 20:33
reyreavman and others added 5 commits July 14, 2026 08:16
@reyreavman
feat(sbom): add python-{uv,pip,poetry} packages directives
2761885 
Add three new package directive types for Python ecosystems (uv, pip,
poetry) alongside the existing go-mod, mirroring the go-mod pattern
across all five layers:

- pkg/config/packages_directive.go: PythonSpec type + 3 enum constants +
  per-manager spec/lock defaults + validation
- pkg/config/raw_packages_directive.go: YAML parser with canonical types
  (python-uv/pip/poetry) and short aliases (uv/pip/poetry)
- pkg/config/packages_commands.go: install commands (pip install -r,
  poetry install --no-root, uv sync --frozen) executed in Packages stage
- pkg/sbom/managedinput/managedinput.go: wire python-package-cataloger
  (declared model) with exact-match source paths
- docs/_data/werf_yaml.yml: schema documentation for all 4 types (en+ru)

Unit tests: 12 parser entries (7 happy + 4 failure via DescribeTable),
4 command generation entries, 6 managedinput entries.

E2E tests: one Ginkgo spec per manager (uv/pip/poetry) with per-fixture
Dockerfile.builder-base + werf-giterminism.yaml. Each test builds a real
image and verifies requests@2.32.3 appears in the SBOM output.

Refs Kaiten 65682576.

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
@reyreavman
refactor(config,sbom): unify package ecosystems via PackageEcosystem …
3b528a5 
…registry

Consolidate the ad-hoc GoModSpec/PythonSpec split into a single data-
driven registry. Adding a new package manager (e.g. python-pdm, rust-
cargo, js-npm) now requires ONE entry in the ecosystems map — parser,
install commands, and SBOM wiring auto-derive from that entry.

Structural changes:
- New unified FileBasedSpec{Workdir, Spec, Lock} replaces GoModSpec and
  PythonSpec (both deleted). PackagesDirective.GoMod and .Python fields
  replaced by .FileBased.
- New PackageEcosystem{Type, Aliases, DefaultSpec, DefaultLock,
  InstallCmd, CatalogerName} registry in pkg/config/packages_directive.go
  as the single source of truth for all file-based ecosystems.
- aliasToType precomputed reverse index for O(1) short-alias lookup.
- Ecosystems() getter exposed for pkg/sbom/managedinput.

Callers migrated:
- raw_packages_directive.go: single fillFileBasedSpec replaces
  fillGoModSpec + fillPythonSpec. Strict behavior: non-string spec
  always errors (previously go-mod silently ignored).
- packages_commands.go: registry lookup replaces switch.
- managedinput.go: buildResolvers() iterates the registry.

Test consolidation:
- New pkg/config/helpers_test.go: shared directivesFromYaml helper
  (previously duplicated across go-mod and python test files).
- All test files migrated from GoMod:/Python: field references to
  FileBased:.
- It() blocks in normalizePackages and FilterBOMBySourcePaths converted
  to DescribeTable per project style.

No user-facing changes: YAML schema and CLI behavior unchanged.

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
@reyreavman
fix(config,sbom): address multi-review risks
8eadef1 
Apply fixes for 7 risks surfaced by the multi-role review:

- #1 (HIGH) buildResolvers() sorts by inputType before returning —
  deterministic resolver order across process restarts. Prevents flaky
  tests and non-reproducible SBOM output ordering.
  + regression tests: 'deterministic order across invocations' and
    'orders by inputType alphabetically'.

- #2 (HIGH) Ecosystems() returns maps.Clone(ecosystems) instead of the
  package-level map — defensive copy prevents accidental global mutation
  by callers.

- #3 (MEDIUM) pip install --no-cache-dir -r <spec> — standard Docker
  practice, prevents pip cache layer bloat in build images.

- #4 (MEDIUM) e2e pip fixture now uses short alias 'type: pip' (matches
  the exact user path from Kaiten 65682576). Previously covered only by
  unit tests; now e2e validates alias → canonical → install → SBOM.

- #5 (MEDIUM) fillFileBasedSpec rejects 'lock:' for ecosystems without
  lock semantics (pip) with 'lock is not supported for type %q'. Silent
  accept would confuse users about reproducibility guarantees.
  + failure test entry.

- #8 (MEDIUM) InstallCmd uses %q instead of %s for workdir/spec — shell-
  quotes values against injection. Defensive coding: values currently
  come from controlled werf.yaml, but template rendering ({{ env }})
  widens the attack surface.

- #11 (LOW) Drop redundant 'eco := eco' loop capture (Go 1.24 uses per-
  iteration binding natively).

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
@nervgh @reyreavman
chore(dev): add implementation plan, spec, and tasks for Python package
92e384d 
ecosystems

Document the new Python package ecosystems feature for the packages
directive, including types, alias resolution, command generation, and
SBOM integration.

Signed-off-by: Alexandr Zaytsev <alexandr.zaytsev@flant.com>
@reyreavman
feat(sbom): resolve conversations
bb0b551 
Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
@reyreavman reyreavman force-pushed the feat/sbom/get-python-sbom branch from 4c901b1 to 1e5090c Compare July 14, 2026 05:17
@reyreavman reyreavman marked this pull request as ready for review July 14, 2026 10:21
@reyreavman
chore(deps): bump syft to 1.45.1
4072782 
Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
@reyreavman reyreavman force-pushed the feat/sbom/get-python-sbom branch from 1e5090c to 68334f3 Compare July 14, 2026 10:59
@reyreavman
refactor(config,sbom): drop python aliases, use python-installed-pack…
d20a6f8 
…age-cataloger

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
@reyreavman reyreavman force-pushed the feat/sbom/get-python-sbom branch from 68334f3 to d20a6f8 Compare July 14, 2026 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nervgh nervgh nervgh requested changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@reyreavman @nervgh