Support PEP 783 pyemscripten_*_wasm32 wheel platform tag#3163
Merged
Conversation
messense
force-pushed
the
feat/pep-783-emscripten
branch
from
094bce9 to
b892cad
Compare
7 tasks
The Emscripten platform-tag branch in `get_platform_tag` now resolves the wheel tag through a priority cascade so maturin produces installable wheels across the full Pyodide release spectrum: 1. `pyemscripten_<year>_<patch>_wasm32` (PEP 783, Pyodide >= 0.30 / Python 3.14+), driven by `MATURIN_PYEMSCRIPTEN_PLATFORM_VERSION` / `PYEMSCRIPTEN_PLATFORM_VERSION` or `pyodide config get pyemscripten_platform_version`. 2. `pyodide_<year>_<patch>_wasm32` (Pyodide 0.28 / 0.29), driven by `MATURIN_PYODIDE_ABI_VERSION` / `PYODIDE_ABI_VERSION` or `pyodide config get pyodide_abi_version`. 3. Legacy `emscripten_<emcc-version>_wasm32` (Pyodide <= 0.27), still driven by `MATURIN_EMSCRIPTEN_VERSION` / `emcc -dumpversion`. A warning notes that wheels in this format are not installable on PEP 783-compliant runtimes. The cascade is implemented as a pure function over an `EmscriptenVersionInputs` struct so the per-Pyodide-version behaviour can be covered deterministically by unit tests, and version segments are validated against the PEP 783 `[0-9]+_[0-9]+` regex so malformed inputs fail loudly instead of producing a tag that no installer accepts. The generated GitHub Actions workflow (`generate-ci`) now also exports `PYEMSCRIPTEN_PLATFORM_VERSION` and `PYODIDE_ABI_VERSION` from `pyodide config get`, defaulting to empty so older Pyodide releases keep producing the legacy tag. For real-Pyodide coverage, `noxfile.py` learns to parse `pyodide-lock.json` (`info.platform`, `info.abi_version`, and a future `info.pyemscripten_platform_version`) into the matching env vars plus an `EXPECTED_PLATFORM_TAG` that `test-emscripten` asserts against the produced wheel filename. The CI `test-emscripten` job is now a matrix over Pyodide 0.28 and 0.29, and `tests/emscripten_runner.js` was updated to match all three Emscripten tag families when locating the built wheel.
The previous `emit_emscripten_setup` step in `generate-ci` queried
`pyodide config get pyemscripten_platform_version`, but that key does
not exist in `pyodide-build`'s `PYODIDE_CLI_CONFIGS`. `pyodide
config get` for an unknown key prints the error message
`Config variable X not found.` to **stdout** (not stderr) and exits 1,
so the existing `2>/dev/null || true` pattern silently captured the
error string into `$GITHUB_ENV`. The downstream maturin build would
then trip the `[0-9]+_[0-9]+` validator with a confusing error.
Replace the two queries with a single robust pattern:
if v=$(pyodide config get pyodide_abi_version 2>/dev/null); then
echo PYODIDE_ABI_VERSION=$v >> $GITHUB_ENV
fi
This only writes the env var when the key is actually recognised. The
PEP 783 `pyemscripten_<year>_<patch>_wasm32` tag is then selected by
the existing cascade in `src/target/platform_tag.rs` based on
`PYTHON_VERSION >= 3.14`. `PYEMSCRIPTEN_PLATFORM_VERSION` /
`MATURIN_PYEMSCRIPTEN_PLATFORM_VERSION` env-var overrides remain
supported for users who want to set the value manually.
Pyodide 314.0.0a1's `pyodide-lock.json` only exposes
`info.abi_version = 2026_0` and `info.python = 3.14.0` — there is
no `info.pyemscripten_platform_version` field. The
`_resolve_pyodide_platform_inputs` cascade already maps that
combination to `pyemscripten_2026_0_wasm32`, matching what
`pyodide_build.build_env.wheel_platform()` produces for the same
ABI.
Pyodide >= 314.0.0a1 also ships its Emscripten output as
`pyodide.asm.mjs` instead of `pyodide.asm.js`. Make the prettier
step tolerant of either filename so `nox -s setup-pyodide` does not
fail before parsing the lock file.
Finally, add Pyodide 314.0.0-alpha.1 / Python 3.14 to the
`test-emscripten` matrix so the new `pyemscripten_*_wasm32` tag
family is exercised on CI alongside the existing `pyodide_*` tag
covered by 0.28.3 / 0.29.0.
messense
force-pushed
the
feat/pep-783-emscripten
branch
from
b892cad to
bc3d67d
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds support for the PEP 783 pyemscripten_*_wasm32 wheel platform tag by introducing a prioritized Emscripten/Pyodide tag resolution cascade, and updates CI/test tooling to validate wheels across Pyodide versions.
Changes:
- Implement Emscripten platform tag resolution cascade (
pyemscripten_*→pyodide_*→ legacyemscripten_*) with input validation and unit tests. - Update Emscripten/Pyodide CI and nox setup to export/derive the right env vars and assert the produced wheel filename tag.
- Expand wheel discovery logic in the JS runner and document new env vars.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/emscripten_runner.js | Expands wheel filename matching to cover all three platform-tag families. |
| src/target/platform_tag.rs | Introduces the tag-resolution cascade, validation, and unit tests. |
| src/ci/github/render.rs | Updates generated CI setup to export PYODIDE_ABI_VERSION when available. |
| noxfile.py | Derives platform tag inputs from pyodide-lock.json and asserts wheel tag correctness. |
| guide/src/environment-variables.md | Documents new env vars for Pyodide/PEP 783 tagging and legacy behavior. |
| AGENTS.md | Updates contributor guidance related to changelog modifications. |
| .github/workflows/test.yml | Changes the Emscripten CI job into a matrix over Pyodide versions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Four small follow-ups based on Copilot's review of PyO3#3163: 1. `tests/emscripten_runner.js`: anchor the platform-tag regex to the trailing `-{platform tag}.whl` segment of the wheel filename so project names that happen to contain `pyodide_`, `emscripten_`, etc. cannot match. Wheel filenames are `{name}-{ver}-{python tag}-{abi tag}-{platform tag}.whl`, and the platform-tag field never contains `-`, so `-{family}_[^-]+_wasm32\.whl$` is unambiguous. Verified against all three Emscripten tag families plus `linux_x86_64` and a project named `my_pyodide_2025_0_wasm32_pkg`. 2. `noxfile.py`: only emit `EMSCRIPTEN_VERSION` from `_resolve_pyodide_platform_inputs` when we actually resolved one. The previous code unconditionally inserted `EMSCRIPTEN_VERSION=""`, which would clobber a previously-set value in `$GITHUB_ENV` and break `setup-emsdk` if the lock file ever omitted `info.platform`. 3. `.github/workflows/test.yml`: the matrix comment claimed three platform-tag families but only two are exercised end-to-end (`pyodide_*` via 0.29.0, `pyemscripten_*` via 314.0.0-alpha.1). Update the comment to match and note that the legacy `emscripten_<emcc-version>_wasm32` family is no longer used by any maintained Pyodide release and remains covered by the cascade unit tests in `src/target/platform_tag.rs`. 4. `AGENTS.md`: drop the duplicate `Do not modify Changelog by hand` bullet — the next bullet already says `Do not edit Changelog.md by hand; it is generated from git history by git-cliff (see cliff.toml)`. The `eprintln!`-with-emoji warning in `emscripten_platform_tag` is intentionally left as-is: it matches the existing user-facing warning style elsewhere in maturin (`🍹`, `📦`, `❌`) and using `tracing::warn!` here would be inconsistent.
The PEP 783 cascade in `emscripten_platform_tag` was over-engineered for a 25-line decision tree: - Three near-identical env-var helpers (`pyemscripten_platform_version`, `pyodide_abi_version`, `pyodide_python_version`) all duplicated the same trim/empty/fallback dance and returned `Result<Option<String>>` despite never producing an `Err`. Collapsed into one `first_non_empty_env(&[...])` helper. - `EmscriptenVersionInputs` + the injected `emcc_lookup` closure existed purely as a test seam: `emcc_version` was never populated in production code, and the wrapper function was unused outside tests. Inlined the cascade directly into `emscripten_platform_tag`. - `validate_version_segment` (regex + `Lazy` + `bail!`) was defensive validation against trusted Pyodide-supplied inputs (sysconfig / `pyodide config get`); a malformed override would fail loudly at install/upload time anyway. Removed. - `python_version_at_least(_, 3, 14)` was generic over (major, minor) but only ever called with one constant. Renamed to `is_python_3_14_or_later` and switched parsing from `u64` to `u32`. The 8 cascade unit tests + 2 validation tests (~140 lines of scaffolding) tested little beyond format-string interpolation; end-to-end coverage is provided by the new `EXPECTED_PLATFORM_TAG` assertion in `noxfile.py` running across the Pyodide 0.29 / 314.0.0a1 CI matrix. Net effect: -262 lines from `src/target/platform_tag.rs` with no behavior change. Also tidies `noxfile.py` (replaces the asm-file lookup loop with `Path(...).glob(...)`) and adds a docstring cross-reference to keep `_resolve_pyodide_platform_inputs` aligned with the Rust cascade.
8 tasks
greateggsgreg
added a commit
to greateggsgreg/pydantic
that referenced
this pull request
May 10, 2026
The previous `requires = ['maturin @ git+...@main']` pin forced every PEP 517 build (including the MSRV job's `uv sync`) to compile maturin from source, which needs rustc 1.89 — failing under the workspace MSRV of 1.88. Revert that pin to `maturin>=1.10,<2` and instead pre-install maturin `main` into the pyemscripten CI build env, with the Makefile target switching to `pyodide build --no-isolation` so PEP 517 picks up the pre-installed maturin (which carries the PEP 783 `pyemscripten_*_wasm32` tag from PyO3/maturin#3163). Also reformat tests/test_pickle.py to satisfy `ruff format --check`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
greateggsgreg
added a commit
to greateggsgreg/pydantic
that referenced
this pull request
May 10, 2026
Adds a `core-build-pyemscripten` CI job that builds `pydantic-core` for the `pyemscripten_*_wasm32` target and runs the test suite under Pyodide, mirroring the conventions of the other `core-build-*` jobs (working directory, `twine check`, `upload-artifact` layout). - `.github/workflows/ci.yml`: new pyemscripten build/test job; pre-installs `maturin` from `main` so PEP 517 builds pick up the PEP 783 `pyemscripten_*_wasm32` tag (PyO3/maturin#3163) without forcing every other PEP 517 build to compile maturin from source under the workspace MSRV. - `.github/scripts/pyemscripten-run-tests.sh`: runner script driving `pyodide` for the build + test invocation. - `pydantic-core/Makefile`: switch to `pyodide build --no-isolation` so the pre-installed maturin is used. - `pydantic-core/pyproject.toml`, `pyproject.toml`: declare pyemscripten build/test extras. - `pydantic-core/tests/conftest.py`, `tests/conftest.py`: skip tests that cannot run under pyemscripten. - `tests/test_docs.py`, `tests/test_pickle.py`: gate cases that rely on capabilities unavailable in the wasm runtime; reformat `test_pickle.py` to satisfy `ruff format`.
sinaatalay
added a commit
to sinaatalay/typst-py
that referenced
this pull request
May 12, 2026
The original commit produced a wheel tagged `emscripten_3_1_58_wasm32`. PyPI's warehouse validator (live since 2026-04-21) only accepts `pyemscripten_<year>_<patch>_wasm32` per PEP 783, so that wheel would have been rejected at upload time. Local `emfs://` verification did not catch this because it bypasses index-side tag validation. Updates: - Bump Pyodide to 0.29.4, which natively installs `pyemscripten_*` wheels (pyodide/pyodide#6180, #6203) and ships Python 3.13 as its bundled interpreter. - Bump host Python on the runner to 3.13 to satisfy pyodide-build's xbuildenv compatibility check. - Pin maturin to >= 1.13.2, which introduced the PEP 783 tag cascade (PyO3/maturin#3163), and export `MATURIN_PYEMSCRIPTEN_PLATFORM_VERSION` derived from `pyodide config get` so the cascade emits the `pyemscripten_*` tag even on Pyodide 0.29 (where it would otherwise fall back to `pyodide_*`). - Verify step now asserts the wheel filename matches `pyemscripten_<year>_<patch>_wasm32.whl` before loading, so a PyPI-incompatible tag fails CI loudly instead of slipping through.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Emscripten platform-tag branch in
get_platform_tagnow resolves the wheel tag through a priority cascade so maturin produces installable wheels across the full Pyodide release spectrum:pyemscripten_<year>_<patch>_wasm32(PEP 783, Pyodide >= 0.30 / Python 3.14+), driven byMATURIN_PYEMSCRIPTEN_PLATFORM_VERSION/PYEMSCRIPTEN_PLATFORM_VERSIONorpyodide config get pyemscripten_platform_version.pyodide_<year>_<patch>_wasm32(Pyodide 0.28 / 0.29), driven byMATURIN_PYODIDE_ABI_VERSION/PYODIDE_ABI_VERSIONorpyodide config get pyodide_abi_version.emscripten_<emcc-version>_wasm32(Pyodide <= 0.27), still driven byMATURIN_EMSCRIPTEN_VERSION/emcc -dumpversion. A warning notes that wheels in this format are not installable on PEP 783-compliant runtimes.The cascade is implemented as a pure function over an
EmscriptenVersionInputsstruct so the per-Pyodide-version behaviour can be covered deterministically by unit tests, and version segments are validated against the PEP 783[0-9]+_[0-9]+regex so malformed inputs fail loudly instead of producing a tag that no installer accepts.The generated GitHub Actions workflow (
generate-ci) now also exportsPYEMSCRIPTEN_PLATFORM_VERSIONandPYODIDE_ABI_VERSIONfrompyodide config get, defaulting to empty so older Pyodide releases keep producing the legacy tag.For real-Pyodide coverage,
noxfile.pylearns to parsepyodide-lock.json(info.platform,info.abi_version, and a futureinfo.pyemscripten_platform_version) into the matching env vars plus anEXPECTED_PLATFORM_TAGthattest-emscriptenasserts against the produced wheel filename. The CItest-emscriptenjob is now a matrix over Pyodide 0.28 and 0.29, andtests/emscripten_runner.jswas updated to match all three Emscripten tag families when locating the built wheel.