Skip to content

rewrite R2 publishing in Python#1051

Merged
zsol merged 2 commits intomainfrom
zsol/jj-rxxuotryyvop
Mar 26, 2026
Merged

rewrite R2 publishing in Python#1051
zsol merged 2 commits intomainfrom
zsol/jj-rxxuotryyvop

Conversation

@zsol
Copy link
Copy Markdown
Member

@zsol zsol commented Mar 25, 2026

This rips out the mirror uploading logic from the pythonbuild rust CLI and moves it to Python, mainly because the rust aws sdk is clearly not mature enough to support our use case, as evidenced by #1047

I've also had to start using --no-dev on some existing uv run invocations because moto wouldn't cleanly install in some build jobs (like when building CPython for Windows).

Test plan

Apart from unit tests, there's now an integration test with a local, mocked S3 provider (using moto).

Dry run release on this branch: https://github.com/astral-sh/python-build-standalone/actions/runs/23550742376

@zsol zsol force-pushed the zsol/jj-rxxuotryyvop branch 2 times, most recently from 652af57 to cbad290 Compare March 25, 2026 14:18
@zsol zsol marked this pull request as ready for review March 25, 2026 15:16
Comment on lines -29 to +27
rustls = { version = "0.23" , default-features = false, features = ["aws_lc_rs"] }
rustls = { version = "0.23", default-features = false, features = ["ring"] }
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note to reviewers: this restores the features from before we introduced the aws dependency

Comment on lines -98 to -101
datetime=$(ls dist/cpython-3.10.*-x86_64-unknown-linux-gnu-install_only-*.tar.gz | awk -F- '{print $8}' | awk -F. '{print $1}')
cargo run --release -- upload-mirror-distributions \
uv run python -m pythonbuild.mirror \
--dist dist \
--datetime ${datetime} \
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note to reviewers: datetime parsing is now bundled into the python script

@zsol zsol force-pushed the zsol/jj-rxxuotryyvop branch 2 times, most recently from a94247f to 9641cad Compare March 25, 2026 15:37
@zsol zsol force-pushed the zsol/jj-rxxuotryyvop branch from 9641cad to d0e2c1b Compare March 25, 2026 15:40
@zsol zsol requested a review from jjhelmus March 25, 2026 15:40
@zsol zsol force-pushed the zsol/jj-rxxuotryyvop branch from 2136144 to d0e2c1b Compare March 25, 2026 17:28
@zsol zsol merged commit b4910f0 into main Mar 26, 2026
987 of 990 checks passed
@zsol zsol deleted the zsol/jj-rxxuotryyvop branch March 26, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants