Julia 1.12 unsupported

[AntonReinhard]

Unit tests fail on 1.12

Running the tests locally on the current nightly fails, see below. The nature of the error seems to imply that it's not just some temporary regression on nightly, but an actual issue.

The syntax: malformed expression is the same error I'm getting in my own project trying to call a RuntimeGeneratedFunction.

Expected behavior

The tests should succeed and RGFs should be callable.

Minimal Reproducible Example 👇

using Pkg
Pkg.test() # with julia +nightly --project

Error & Stacktrace ⚠️

     Testing Running tests...
Test Summary: | Pass  Total   Time
Aqua          |   10     10  11.6s
Error During Test at /home/antonr/repos/RuntimeGeneratedFunctions.jl/test/runtests.jl:157
  Test threw exception
  Expression: ((#= /home/antonr/repos/RuntimeGeneratedFunctions.jl/test/runtests.jl:157 =# @RuntimeGeneratedFunction(ex))(2))(3) === 5
  syntax: malformed expression
  Stacktrace:
   [1] top-level scope
     @ ~/repos/RuntimeGeneratedFunctions.jl/test/runtests.jl:157
   [2] (::RuntimeGeneratedFunction{(:x,), var"#_RGF_ModTag", var"#_RGF_ModTag", (0x0b620387, 0x0984f55d, 0x444b0809, 0x67f78240, 0x222043e0), Expr})(args::Int64)
     @ RuntimeGeneratedFunctions ~/repos/RuntimeGeneratedFunctions.jl/src/RuntimeGeneratedFunctions.jl:150
   [3] top-level scope
     @ ~/repos/RuntimeGeneratedFunctions.jl/test/runtests.jl:515
   [4] macro expansion
     @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/Test/src/Test.jl:676 [inlined]
ERROR: LoadError: There was an error during testing
in expression starting at /home/antonr/repos/RuntimeGeneratedFunctions.jl/test/runtests.jl:157
ERROR: Package RuntimeGeneratedFunctions errored during testing

Environment (please complete the following information):

julia> Pkg.status()
Project RuntimeGeneratedFunctions v0.5.13
Status `~/repos/RuntimeGeneratedFunctions.jl/Project.toml`
  [e2ba6199] ExprTools v0.1.10
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0

julia> using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)
Project RuntimeGeneratedFunctions v0.5.13
Status `~/repos/RuntimeGeneratedFunctions.jl/Manifest.toml`
  [e2ba6199] ExprTools v0.1.10
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0

julia> versioninfo()
Julia Version 1.12.0-DEV.1492
Commit ec2e1217fd8 (2024-10-25 07:25 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 24 × AMD Ryzen 9 7900X3D 12-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, znver4)
Threads: 1 default, 0 interactive, 1 GC (on 24 virtual cores)

[AntonReinhard]

1.12 is now a release candidate and the same error still exists.

[ChrisRackauckas]

@oscardssmith will bring this to the compiler team. We need to make sure we understand what the breaking changes are and find suitable workarounds, or ask for the release to be blocked.

[oscardssmith]

this appears to be a Julia internal issue @JeffBezanson will look into it

[JeffBezanson]

The problem here is that the arguments to the internal opaque_closure expression head have changed. This should use @opaque <expr> instead. I believe that will work.

[ChrisRackauckas]

Just to clarify, so you're saying that instead of making a :opaque_closure expression https://github.com/SciML/RuntimeGeneratedFunctions.jl/blame/c0a23ca4a84908b85997a373ac2d78b6de385828/src/RuntimeGeneratedFunctions.jl#L255 we should just make this _ex = @opaque combinedef(fdef) and that will work? Since we have the expression at runtime I wasn't sure if we'd have to interpolate into the macro if we switched to that.

[oscardssmith]

just making sure I understand correctly: what we're doing now is generating now is basically the result of @macroexpand (@opaque ..., which is not stable since the lowering of @opaque can change without breaking changes, and what we should be generating is the result of Meta.@dump (@opaque ...) which is stable (well as stable as @opaque which is experimental). Is that correct?

[oscardssmith]

@ChrisRackauckas we discussed this issue (and other SciML 1.12 issues) on the compiler call today, and this one is almost completely unrelated to the other issues. Pretty much all the rest of the issues are based on Julia making the rules around import slightly stricter, while this is a minor change to lowering of @opaque

[oscardssmith]

looks like #96 does fix this.

[AntonReinhard]

Thank you for the fix!