Skip to content

Unlink PyO3 versions between FASTSim-2 and FASTSim-3#265

Open
kylecarow wants to merge 25 commits into
fastsim-3from
f3/unlink-pyo3
Open

Unlink PyO3 versions between FASTSim-2 and FASTSim-3#265
kylecarow wants to merge 25 commits into
fastsim-3from
f3/unlink-pyo3

Conversation

@kylecarow

@kylecarow kylecarow commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

This PR is branched off of work I did in #264, so that should be merged/reviewed first.

Currently we lock the PyO3 version of FASTSim-3 to the version that the latest release of FASTSim-2 used. This is to support the bridge between the FASTSim-2 Python API and FASTSim-3 Python API.

This means we cannot update the PyO3 version in FASTSim-3 without making a FASTSim-2 release!. Chad did this once to be able to update PyO3 in FASTSim-3, but this problem will recur

Instead of forcing these versions to be the same, or deleting the FASTSim-2 compatibility convenience functions altogether, this PR:

  • removes the to_fastsim2_py and try_from_fastsim2 methods that allowed Python users to convert between instances of FASTSim-2 vehicles and FASTSim-3 vehicles. (only negative change this PR introduces)
  • preserves the ability to load FASTSim-2 vehicle files as FASTSim-3 vehicles!
  • makes minor updates to how Paths are parsed to match the updated PyO3 API in the newer version
  • updates auto-generated vehicles to include new struct fields (this isnt something I caused, but a probably a consequence of not regenerating these vehicles since before speed_trac_fwd_max_meters_per_second was introduced as a field)
  • increases FASTSim compatibility through Python 3.15!!!
  • bumps to 3.1.0 due to
  • enforces all crate/package/project versions are equivalent,

Additional changes:

  • release workflow now uses pixi and full matrix of python versions. It also releases fastsim-core and fastsim-proc-macros to crates.io programmatically, after doing a 'dry run' publish
  • release workflow and build_and_test workflow use a shared core workflow that compiles, lints (not completely, linting fails right now but I dont want to make such broad changes in this PR), tests, and builds wheels
    • previously we duplicated a lot of workflow instructions, this is cleaner and easier to maintain
  • Python 3.10-3.14 are tested (and released in release workflow), 3.15 is still in beta does not yet have a conda-forge package so pixi cant resolve that dependency yet.
  • applies cargo fmt and includes cargo fmt --check as part of testing script & workflow

@kylecarow kylecarow marked this pull request as ready for review June 30, 2026 00:05
@kylecarow

kylecarow commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

@jhoshiko @robinsteuteville @jakeholden

Found a BUNCH of redundancy in the testing workflow that could be cut down

Previously we:

  • had a matrix of 3 Python versions * 4 platforms, requiring 12 runners
    • scaling this up to 5 Python versions requires 20 runners
  • ran Rust tests on all platforms
  • ran Python tests on all platforms
  • on linux only: build source distribution (tar.gz)
  • built wheels with each runner targeting a single python version
  • ran Python tests on all wheels

Now we:

  • have a matrix of 4 platforms, so only 4 runners required
  • on linux only:
    • check FASTSim version consistency
    • run Rust tests
    • run additional checks (formatting etc)
    • build source distribution (tar.gz)
  • build wheels for all supported Python versions for each platform (cibuildwheel is designed to do this, so no matrix of versions needed)
  • run Python tests on all wheels

Cutting down this redundancy also cuts down on pixi environments, so no more py310/py311/etc environments are needed any more.

I've also closed the #264 as this PR includes all of its changes and more.

@kylecarow

Copy link
Copy Markdown
Collaborator Author

Here is a summary that I generated from the PR diff with Claude, with many of my own notes and edits sprinkled in:


Summary of changes — FASTSim CI/CD overhaul + v3.1.0 packaging updates

CI/CD (biggest chunk of the diff)

  • Migrated all three GitHub Actions workflows (build_and_test, deploy_docs, release) from manual Rust/Python setup to pixi-based environments.
  • Restructured wheel building: one cibuildwheel invocation per platform (not per platform and per Python version), targeting cp310–cp314.
    • Expanded Python version compatibility due to PyO3 upgrade/unlink, described below.
    • This reduces our required Actions runners from 12 (or 20, with the increased Python versions in this PR) to only 4 runners.
      build_and_test workflow is now called from release workflow directly rather than duplicating testing code
  • Removed redundant testing of Rust + Python on all platforms+Python versions, instead test Rust only once and do - Python tests only on built wheels
  • Fixed a build failure on linux that plagues other branches too: Pillow/manylinux2014 build failure fixed by moving to manylinux_2_28.
  • Added zlib as an explicit pixi dependency to resolve a linker error (-lz not found) hit during pixi's editable install of the Rust extension.
  • Added a check_versions.py script + CI step that verifies pyproject.toml and all Cargo.toml package versions stay in sync.
    • Bumped FASTSim to v3.1.0 due to PyO3 changes described below - though this probably doesnt follow semver API rules
  • release.yaml now also does cargo publish --dry-run and cargo publish for the Rust crates alongside the existing PyPI/TestPyPI twine upload.

Packaging

  • Adopted PEP 735 dependency groups (build/release/test/wheels/docs/dev) replacing the single dev extra, with matching pixi environments.

Rust core / PyO3

  • Upgraded PyO3 0.23 → 0.29
    • Widens requires-python to <3.16, increasing support from 3.10/11/12 to 3.10/11/12/13/14 and 3.15 when that is released to conda-forge.
  • Removed the fastsim-2 Python interop layer — Vehicle.to_fastsim2(), Vehicle.try_from_fastsim2(), SimDrive.to_fastsim2(), and the SimDriveF2/VehicleF2 Python classes are all gone. fastsim-py no longer depends on fastsim-2 directly. Rust code still retains full interop functionality.
    • From Python we are still able to load FASTSim-2 vehicle files in FASTSim-3 and simulate with FASTSim-3.
    • From Python only: no longer able to load FASTSim-2 vehicles/simdrives in FASTSim-3 and simulate with FASTSim-2, or convert FASTSim-3 vehicles/simdrives to FASTSim-2.
  • Removed the now-unused test_speedup.py benchmark (compared against fastsim-2, which no longer applies).
  • These API changes are why I am thinking this PR should come with a version bump

Note on cal_and_val/ vehicle YAML changes: these weren't hand-edited — they're the automatic byproduct of running build_and_test.sh, which regenerates the vehicle files and picked up the new speed_trac_fwd_max_meters_per_second field (plus a few calibration value diffs already present in those source vehicles) across the board. That field wasnt added by this PR.

@kylecarow

kylecarow commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator Author

@robinsteuteville @jhoshiko would love to get this PR reviewed this week if possible :)

@kylecarow kylecarow removed the request for review from jakeholden July 13, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant