Skip to content

Move CI to PyPy 3.11, raise minimum Python to 3.11 - #222

Merged
vthorsteinsson merged 3 commits into
masterfrom
chore/pypy-3.11
Jul 29, 2026
Merged

Move CI to PyPy 3.11, raise minimum Python to 3.11#222
vthorsteinsson merged 3 commits into
masterfrom
chore/pypy-3.11

Conversation

@vthorsteinsson

Copy link
Copy Markdown
Member

CI is currently red on every leg of the matrix. The cause is a dependency chain, not our code:

sqlalchemy-stubs 0.4  ->  mypy 2.3.0  ->  ast-serialize 0.6.0  (Rust / PyO3)

PyO3's minimum supported Python is 3.11, so installation fails outright before any test runs:

error: the configured PyPy interpreter version (3.10) is lower than
       PyO3's minimum supported version (3.11)

hint: `ast-serialize` (v0.6.0) was included because `sqlalchemy-stubs` (v0.4)
      depends on `mypy` (v2.3.0) which depends on `ast-serialize`

This affects pypy3.9, pypy3.10 and CPython 3.9 alike — it is a floor on the Python version, not a PyPy-specific problem.

Changes

  • .github/workflows/python-package.yml: matrix ["3.9", "3.12", "pypy3.9", "pypy3.10"]["3.11", "3.12", "pypy3.11"]. Verified that pypy3.11 resolves in the PyPy release index actions/setup-python reads — downloads.python.org/pypy/versions.json lists PyPy 7.3.23 / Python 3.11.15 (2026-05-26) as stable and latest_pypy.
  • README.md: badge and stated minimum 3.9 → 3.11; note PyPy 3.11 as the supported PyPy version.
  • docs/setup_linux.md: install PyPy 3.11, download URL updated to pypy3.11-v7.3.23-linux64.tar.gz. Note the archive is now .tar.gz — older PyPy releases shipped .tar.bz2.
  • docs/setup_macos.md: PyPy 3.9 → PyPy 3.11.
  • .gitignore: pypy39/*, pypy310/*pypy311/*.

Both new URLs verified to return HTTP 200. The workflow YAML parses and the matrix reads back correctly. Scripts under scripts/ reference venv/ generically and need no change.

Deliberately out of scope

vectors/ is untouched. It is a separate CPython environment — currently CPython 3.9.4 with gensim==3.8.2 (a 2019 release), numpy 1.24.3, scipy 1.10.1 — which the main application's minimum does not govern. There is no evidence that stack builds on 3.11, and a gensim 4.x upgrade is an API break (wv.vocabwv.key_to_index) touching builder.py and the similarity server. Modernising it is separate work.

sqlalchemy-stubs is left in place. Worth noting for a follow-up: it is unmaintained (SQLAlchemy 1.x era), declares mypy as a runtime dependency, and is type-stub-only — no type checker runs in CI, and sqlalchemy2-stubs is already in requirements.txt. Removing it would drop the PyO3 chain entirely and could allow a lower floor, if that is ever wanted.

Known unrelated failure

tests/test_queries.py::test_currency fails on CPython 3.12, where dependencies install successfully. It is independent of the interpreter change and will need fixing separately before the build is fully green.

Production note

This does not move production. /home/greynir/github/Greynir/venv is a symlink to pypy310; switching it means building a fresh PyPy 3.11 venv. Related trap: scripts/deploy.sh lines 51-52 rm venv/site-packages/reynir/Greynir.grammar*.bin works only because the current production venv has an unusual flat layout. A newly built venv will likely use the standard venv/lib/pypy3.11/site-packages/, and those rms will fail silently — set -o errexit is commented out — leaving a stale compiled grammar.

🤖 Generated with Claude Code

vthorsteinsson and others added 3 commits July 29, 2026 16:58
CI is currently red on every leg. The dependency chain
sqlalchemy-stubs 0.4 -> mypy 2.3.0 -> ast-serialize 0.6.0 (a Rust/PyO3
extension) requires Python >= 3.11, so installation fails outright on
pypy3.9, pypy3.10 and CPython 3.9:

    error: the configured PyPy interpreter version (3.10) is lower than
           PyO3's minimum supported version (3.11)

Raise the floor to 3.11 across the board and move PyPy to 3.11
(PyPy 7.3.23 / Python 3.11.15, released 2026-05-26).

- .github/workflows/python-package.yml: matrix ["3.9", "3.12", "pypy3.9",
  "pypy3.10"] -> ["3.11", "3.12", "pypy3.11"]. Verified that pypy3.11 resolves
  in the PyPy release index that actions/setup-python reads
  (downloads.python.org/pypy/versions.json lists 3.11.15 / 7.3.23 as stable).
- README.md: badge and stated minimum 3.9 -> 3.11; note PyPy 3.11 as the
  supported PyPy version.
- docs/setup_linux.md: install PyPy 3.11; update the download URL to
  pypy3.11-v7.3.23-linux64.tar.gz. Note the archive is now .tar.gz, not the
  .tar.bz2 that older PyPy releases shipped.
- docs/setup_macos.md: PyPy 3.9 -> PyPy 3.11.
- .gitignore: replace the pypy39/pypy310 entries with pypy311.

vectors/ is deliberately untouched. It is a separate CPython environment
(currently 3.9.4, with gensim 3.8.2 from 2019) that the main app's minimum
does not govern, and there is no evidence that stack builds on 3.11. Treat
modernising it as separate work.

Both new URLs verified to return HTTP 200. The shell scripts under scripts/
reference 'venv/' generically and need no change.

Known unrelated failure: tests/test_queries.py::test_currency fails on
CPython 3.12 with dependencies installed successfully, so it is independent
of the interpreter change and will need fixing separately for a fully green
build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The pypy3.11 leg was cancelled mid-install when the CPython legs failed on an
unrelated test, so the migration's key question -- whether the dependency set
installs on PyPy 3.11 -- went unanswered. A failure on one interpreter is
precisely when the other legs' results matter most.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
arionbanki.is is serving malformed XML from its exchange-rate endpoint:

    Error fetching exchange rate data from
    https://www.arionbanki.is/markadir/gjaldmidlar/gengi/xml-export:
    not well-formed (invalid token): line 121, column 66817

The currency module returns no answer as a result, failing the test on every
interpreter. This is unrelated to the interpreter migration and is the only
thing standing between this branch and a green build.

Skipped rather than fixed for now: the currency module needs updating to
Arion's new format, which will be picked up once the uv-based dev environment
is in place.

Note that the test calls a live third-party endpoint, so it can redden the
build at any time regardless of any change. Worth making it resilient (mock or
xfail on fetch failure) when the format fix lands.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vthorsteinsson
vthorsteinsson merged commit 54bb865 into master Jul 29, 2026
3 checks passed
@vthorsteinsson
vthorsteinsson deleted the chore/pypy-3.11 branch July 29, 2026 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant