Skip to content

Audit pip-install steps in CI for consistent supply-chain pinning #881

@drbenvincent

Description

@drbenvincent

What's the problem?

The repo takes a strong stance on pinning GitHub Actions by SHA — every uses: line in .github/workflows/*.yml references an action by full commit hash with a # v... comment, e.g.:

uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

That protects us from a tag being moved or an action being compromised after the fact.

The same workflows also run pip install to fetch Python tooling (e.g. pyproject2conda, anything else used by CI helpers). Those pip installs:

  • pin the top-level package version at best (after ci: pin pyproject2conda==0.22.1 to work around 0.23.0 regression #878, pyproject2conda>=0.22.1,<0.23),
  • do not pin transitive dependencies (pydantic, tomlkit, click, etc. — whatever is latest at install time gets pulled in),
  • do not use --require-hashes or any lockfile, so a maintainer-account compromise on PyPI for any of these packages would be picked up immediately on the next CI run.

Combined effect: our action pinning is hash-strict, but our Python tooling pinning is loose. A future regression in a transitive dep — exactly the class of breakage that #878 worked around at the top level — could resurrect the same kind of CI outage. A more adversarial scenario (typosquat / account takeover on a transitive dep) would also slip past today's pinning.

What should we do?

This issue is for deciding the policy, not for shipping a particular implementation. Options:

  1. Accept the asymmetry. These pip installs run in dev-only CI jobs with permissions: contents: read, so the blast radius is the runner only. Document this rationale in AGENTS.md so we remember why we chose not to harden, and close the issue.
  2. Hash-pin top-level and transitive deps for each pip-install step. Generate a constraints file with hashes (e.g. via pip-compile --generate-hashes or uv pip compile), commit it, and use pip install --require-hashes -r constraints.txt. Strongest option; adds a maintenance step every time we bump a tool.
  3. Lock once, reuse everywhere. Maintain a single requirements/ci-tools.lock (or equivalent) that all CI helper steps install from. Bumps are explicit, reproducible, and visible in PR diffs.

What this issue is asking for

  • Inventory every pip install step across .github/workflows/.
  • For each one, decide whether it warrants hardening based on what it touches and what permissions the job has.
  • Pick option 1, 2, or 3 (or a per-step mix) and apply it consistently.
  • Document the chosen policy alongside the existing action-pinning convention so future contributors don't have to re-derive it.

Context

Came out of an adversarial review of #878. The PR fixed the immediate breakage by bumping the top-level pin; this issue is about whether the underlying supply-chain posture for pip-installed CI tools should match the strictness we already apply to GitHub Actions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    devopsDevOps related

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions