Skip to content

Our TestPyPI workflows require constant maintenance #8755

Description

@alexreinking

Every few dev ("nightly") releases, our 10 GiB storage quota on TestPyPI fills up. That means I have to manually go in and delete the old releases. Deleting releases (rather than yanking or deprecating them) is frowned upon, but tolerated on the Test index because, hey, it's the test index. However, only deleting releases replenishes the quota.

I can't set up anything to do rotate old releases automatically because deleting a release requires going through 2FA.

There is a process for requesting more space for a project. But our usage (nightlies) doesn't seem justifiable for a blanket increase when each release is ~1.2GB in total size. We provide 20 wheels per release to support a wide variety of platforms (~60MB ea.).

I can imagine several solutions to this problem:

  1. Shrink the size of our wheels somehow
  2. Reduce the number of wheels somehow
  3. Host our own PyPI index (imagine: https://pypi.halide-lang.org/simple/)

I'm in favor of (3) as it's more future-proof for nightlies.

Regarding the main PyPI instance, I don't see any reason not to continue to rely on that. We can argue for a size increase there if it comes to it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev_meetingTopic to be discussed at the next dev meetingpythonIssues related to Halide/Python interop

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions