Add Julia package manifest for automatic dependency management#176
Merged
Conversation
`load_julia_packages` was running `Pkg.activate("diffeqpy", shared=true)`
before adding the solver packages, which moved Julia off juliacall's
managed project (`$CONDA_PREFIX/julia_env`, where PythonCall is pinned
to the ABI-matched version) and into a separate shared environment. The
shared environment then resolved an unconstrained PythonCall, breaking
the juliacall <-> PythonCall ABI and segfaulting on import.
Declare the solver stack in `diffeqpy/juliapkg.json` so juliacall
installs DifferentialEquations, OrdinaryDiffEq, StochasticDiffEq,
DelayDiffEq, Sundials, DiffEqCallbacks, and ModelingToolkit into its
managed project alongside PythonCall, with proper version resolution.
Compat ranges span the doc-tested major and the current major (e.g.
DifferentialEquations "7, 8") so the docs keep working under the
recent v8 metapackage where some sub-solvers are no longer transitive.
Drop the `Pkg.activate` line from `load_julia_packages` so the lazy
install path used by the GPU sub-modules (cuda/amdgpu/metal/oneapi)
also targets juliacall's managed project.
https://claude.ai/code/session_01G6GjTpy5ZKWq78zfdCzV4y
OrdinaryDiffEqDefault now provides a default solver for DAEProblem, so declare it in juliapkg.json so it's installed alongside the rest of the documented stack. https://claude.ai/code/session_01G6GjTpy5ZKWq78zfdCzV4y
DifferentialEquations.jl v8 trimmed its module body down to just `@reexport using SciMLBase, OrdinaryDiffEq` — StochasticDiffEq, DelayDiffEq, Sundials, and DiffEqCallbacks are no longer loaded transitively. Their `__init__` hooks register the default algorithms used by `solve(prob)` dispatch, so without them the documented `de.solve(sde_prob)`, `de.solve(dde_prob)`, and `de.solve(dae_prob)` calls would have no default to pick. Import them alongside DifferentialEquations so the docs keep working under v8. SciMLBase is already re-exported through DifferentialEquations (so `de.SDEProblem` etc. resolve), and OrdinaryDiffEqDefault is pulled in transitively by OrdinaryDiffEq and now also covers DAEProblem. https://claude.ai/code/session_01G6GjTpy5ZKWq78zfdCzV4y
The Julia-side load list now matches the package set declared in diffeqpy/juliapkg.json: de.py loads DifferentialEquations, OrdinaryDiffEq, OrdinaryDiffEqDefault, StochasticDiffEq, DelayDiffEq, Sundials, DiffEqCallbacks, and ModelingToolkit; ode.py loads OrdinaryDiffEq and OrdinaryDiffEqDefault. Drops the unused PythonCall imports (juliacall already brings PythonCall into the session, and we never used the returned reference). https://claude.ai/code/session_01G6GjTpy5ZKWq78zfdCzV4y
The default DDE-solve path (`MethodOfSteps(DefaultODEAlgorithm())`)
currently raises `MethodError: Cannot convert ... Rosenbrock23Cache{...}`
on the v7-coupled OrdinaryDiffEq stack. The asymmetry is upstream in
SciML/OrdinaryDiffEq.jl — the ODE/DAE default selectors pass
`autodiff = AutoFiniteDiff()` whereas the DDE default selector at
`lib/DelayDiffEq/src/DelayDiffEq.jl` constructs `DefaultODEAlgorithm()`
without an autodiff override, so when DelayDiffEq later builds the
composite Rosenbrock23 cache against the `ODEFunctionWrapper` the
ForwardDiff tag flows in inconsistently between the history- and
main-integrator caches. Tracked in diffeqpy #172.
Switch the test to the canonical `MethodOfSteps(Tsit5())` invocation
so PR #176's CI is not blocked on the upstream regression while the
diffeqpy stack converges on v7/v8. The test still exercises the
DDEProblem construction and the DelayDiffEq solve path; only the
default-selector codepath is bypassed.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
Investigation + workaround pushed at PR #177 (sub-PR into this branch). Brief:
Ignore until reviewed by @ChrisRackauckas. |
DifferentialEquations.jl v8 trimmed its re-exports down to SciMLBase + OrdinaryDiffEq, so symbols from the sub-solver packages (MethodOfSteps, IDA, CVODE_BDF, SOSRI, EM, etc.) are no longer accessible via `de.<name>` when `de` is just a reference to the DifferentialEquations module. The previous shape — `de = loaded[0]` — passes the smoke test but breaks the moment a user (or one of our tests) reaches for a name that used to come transitively through DifferentialEquations. Concretely, `de.MethodOfSteps(de.Tsit5())` now fails with `UndefVarError: MethodOfSteps not defined in DifferentialEquations` — the Julia error helpfully prints "a global variable of this name also exists in DelayDiffEq" — and the same shape will bite any caller using `de.IDA`, `de.SOSRI`, etc. Build a fresh `_diffeqpy_de` module in Main that `using`s each sublib (`DifferentialEquations, OrdinaryDiffEq, OrdinaryDiffEqDefault, StochasticDiffEq, DelayDiffEq, Sundials, DiffEqCallbacks, ModelingToolkit`) and point `de` at that. `using` brings each sublib's exports — and the sublib module name itself — into `_diffeqpy_de`'s scope, so `de.<name>` resolves uniformly for everything `from diffeqpy import de` users used to get pre-v8. Drop the `Main.ModelingToolkit.` prefix from the jit hackery since `ModelingToolkit` is now visible as a bare name inside `_diffeqpy_de` via the `using ModelingToolkit` line. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
PR #177 CI is now green on all four matrix jobs:
PR #177 contains two commits:
Merging PR #177 into this branch should turn this PR's CI green as well. Ignore until reviewed by @ChrisRackauckas. |
Pass explicit MethodOfSteps(Tsit5()) in DDE test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a Julia package manifest (
juliapkg.json) to enable automatic installation and management of diffeqpy's core dependencies through juliacall's managed project system, eliminating the need for manual package activation.Key Changes
diffeqpy/juliapkg.json: Declares the core solver stack (DifferentialEquations, OrdinaryDiffEq, StochasticDiffEq, DelayDiffEq, Sundials, DiffEqCallbacks, ModelingToolkit) with compatible version ranges and Julia 1.10 requirementdiffeqpy/__init__.py:Pkg.activate("diffeqpy", shared=true)call, as packages are now pre-installed by juliacallMANIFEST.in: Added inclusion of*.jsonfiles to ensure the manifest is packaged with distributionsImplementation Details
The manifest leverages juliacall's built-in package management to automatically install declared packages into its managed project alongside the ABI-matched PythonCall. This approach:
https://claude.ai/code/session_01G6GjTpy5ZKWq78zfdCzV4y