`unanalyzable` not clearly defined

[Sbozzolo]

I am trying ExplicitImports.jl with a package with a slightly complex module structure (the package is mostly implemented in modules loaded by extensions). When I run check_no_implicit_imports, I get UnanalyzableModuleException and a message telling me that I show allow my module to be unalayzable. However, what "unanalyzable" means is not really clear. Is it an issue I can solve on my end?

(The package I want to check is ClimaUtilities.jl)

[ericphanson]

often it means dynamic include statements, but I don't see any there. If you do print_explicit_imports it should show which modules aren't analyzable. Also try setting ENV["JULIA_DEBUG"] = "ExplicitImports" first.

I wasn't able to reproduce:

julia> using ExplicitImports

julia> ENV["JULIA_DEBUG"] = "ExplicitImports"
"ExplicitImports"

julia> check_no_implicit_imports(ClimaUtilities)

[Sbozzolo]

To reproduce, you also have to load the packages that trigger extension loading:

import ExplicitImports
ENV["JULIA_DEBUG"] = "ExplicitImports"

import NCDatasets
import ClimaComms
import Interpolations
import ClimaCoreTempestRemap

using ClimaUtilities
ExplicitImports.check_no_implicit_imports(ClimaUtilities)

results in

julia> ExplicitImports.check_no_implicit_imports(ClimaUtilities)
ERROR: UnanalyzableModuleException
Module `ClimaUtilitiesClimaCoreTempestRemapExt.TempestRegridderExt` was found to be unanalyzable. Include this module in the `allow_unanalyzable` keyword argument to allow it to be unanalyzable.

Stacktrace:
 [1] should_ignore!(::Nothing, mod::Module; ignore::Tuple{})
   @ ExplicitImports ~/.julia/packages/ExplicitImports/JSYBk/src/checks.jl:260
 [2] check_no_implicit_imports(mod::Module, file::String; skip::Tuple{Module, Module, Module}, ignore::Tuple{}, allow_unanalyzable::Tuple{})
   @ ExplicitImports ~/.julia/packages/ExplicitImports/JSYBk/src/checks.jl:231
 [3] check_no_implicit_imports
   @ ~/.julia/packages/ExplicitImports/JSYBk/src/checks.jl:223 [inlined]
 [4] check_no_implicit_imports(mod::Module)
   @ ExplicitImports ~/.julia/packages/ExplicitImports/JSYBk/src/checks.jl:223
 [5] top-level scope
   @ REPL[8]:1

As for printing imports:

julia> ExplicitImports.print_explicit_imports(ClimaUtilities)
  Module ClimaUtilitiesClimaCoreTempestRemapExt.TempestRegridderExt could not be accurately analyzed, likely due to dynamic include statements. You can pass
  strict=false to attempt to get (possibly inaccurate) results anyway.

  Module ClimaUtilities.OutputPathGenerator is not relying on any implicit imports.

  Module ClimaUtilities.Utils is not relying on any implicit imports.

  Module ClimaUtilitiesClimaCoreTempestRemapExt is not relying on any implicit imports.

  Module ClimaUtilitiesClimaCoreInterpolationsExt.InterpolationsRegridderExt could not be accurately analyzed, likely due to dynamic include statements. You
  can pass strict=false to attempt to get (possibly inaccurate) results anyway.

  Module ClimaUtilitiesClimaCoreExt.TimeVaryingInputs0DExt could not be accurately analyzed, likely due to dynamic include statements. You can pass
  strict=false to attempt to get (possibly inaccurate) results anyway.

  Module ClimaUtilitiesClimaCommsExt.MPIUtilsExt could not be accurately analyzed, likely due to dynamic include statements. You can pass strict=false to
  attempt to get (possibly inaccurate) results anyway.

  Module ClimaUtilities.FileReaders is not relying on any implicit imports.

  Module ClimaUtilitiesClimaCoreInterpolationsExt is not relying on any implicit imports.

  Module ClimaUtilitiesNCDatasetsExt is not relying on any implicit imports.

  Module ClimaUtilitiesClimaCoreNCDatasetsExt.DataHandlingExt could not be accurately analyzed, likely due to dynamic include statements. You can pass
  strict=false to attempt to get (possibly inaccurate) results anyway.

  Module ClimaUtilities.DataHandling is not relying on any implicit imports.

  Module ClimaUtilities.Regridders is not relying on any implicit imports.

  Module ClimaUtilities.TimeVaryingInputs is not relying on any implicit imports.

  Module ClimaUtilities.TimeManager is not relying on any implicit imports.

  Module ClimaUtilitiesClimaCoreNCDatasetsExt is not relying on any implicit imports.

  Module ClimaUtilitiesClimaCoreNCDatasetsExt.TimeVaryingInputsExt could not be accurately analyzed, likely due to dynamic include statements. You can pass
  strict=false to attempt to get (possibly inaccurate) results anyway.

  Module ClimaUtilities is not relying on any implicit imports.

  Module ClimaUtilities.DataStructures is not relying on any implicit imports.

  Module ClimaUtilities.ClimaArtifacts is not relying on any implicit imports.

  Module ClimaUtilitiesClimaCommsExt is not relying on any implicit imports.

  Module ClimaUtilitiesClimaCoreExt is not relying on any implicit imports.

  Module ClimaUtilitiesClimaCoreNCDatasetsExt.SpaceVaryingInputsExt could not be accurately analyzed, likely due to dynamic include statements. You can pass
  strict=false to attempt to get (possibly inaccurate) results anyway.

  Module ClimaUtilities.MPIUtils is not relying on any implicit imports.

  Module ClimaUtilities.SpaceVaryingInputs is not relying on any implicit imports.

  Module ClimaUtilitiesNCDatasetsExt.NCFileReaderExt could not be accurately analyzed, likely due to dynamic include statements. You can pass strict=false to
  attempt to get (possibly inaccurate) results anyway.

[Sbozzolo]

The common thread I think I see is that the modules that cannot be analyzed are modules that are included in the extensions, as in ClimaUtilitiesClimaCoreTempestRemapExt.jl

module ClimaUtilitiesClimaCoreTempestRemapExt

include("TempestRegridderExt.jl")

end

where TempestRegridderExt.jl is a module itself (but not an extension, despite what the name might suggest).

[ericphanson]

interesting, I have a test for extensions but it doesn't include submodules in the extension, so something must be going wrong there, thanks for the report

[ericphanson]

I'm unable to reproduce this, though I have a test now for the case of submodules inside extensions in #160. I ran that on older versions of ExplicitImports as well and could not find a failure so it may not be the original issue. Do you still see unexpected UnanalyzableModuleException's somewhere?

[ericphanson]

huh, I couldn't reproduce locally, but it showed up in CI: https://github.com/JuliaTesting/ExplicitImports.jl/actions/runs/20895579449/job/60033566126?pr=160 in #160. Only 1 of the runs though. I'll try to dig in.

[ericphanson]

Aha, on Julia 1.10, pathof(ext_mod.SubmoduleExt) = nothing. That seems like a Julia bug. It works on 1.11 and 1.12.

mwe.jl

using Pkg

println("Julia version: ", VERSION)

tmp = mktempdir()
pkgdir = joinpath(tmp, "Issue79Pkg")
mkpath(joinpath(pkgdir, "src"))
mkpath(joinpath(pkgdir, "ext"))

write(joinpath(pkgdir, "Project.toml"), """
name = "Issue79Pkg"
uuid = "fa155e15-ec49-4eec-b540-f513de9043b2"
authors = ["MWE"]
version = "0.1.0"

[weakdeps]
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[extensions]
LinearAlgebraExt = "LinearAlgebra"
""")

write(joinpath(pkgdir, "src", "Issue79Pkg.jl"), """
module Issue79Pkg
end # module Issue79Pkg
""")

write(joinpath(pkgdir, "ext", "LinearAlgebraExt.jl"), """
module LinearAlgebraExt
include("SubmoduleExt.jl")
end # module LinearAlgebraExt
""")

write(joinpath(pkgdir, "ext", "SubmoduleExt.jl"), """
module SubmoduleExt
const LOCAL_CONST = 1
end # module SubmoduleExt
""")

envdir = joinpath(tmp, "env")
mkpath(envdir)
Pkg.activate(envdir; shared=false)
Pkg.develop(; path=pkgdir)
Pkg.instantiate()

using LinearAlgebra
using Issue79Pkg

ext = Base.get_extension(Issue79Pkg, :LinearAlgebraExt)
println("ext = ", ext)

if ext === nothing
    error("Extension did not load")
end

println("pathof(ext) = ", pathof(ext))
println("pathof(ext.SubmoduleExt) = ", pathof(ext.SubmoduleExt))

[ericphanson]

closing in favor of JuliaLang/julia#60642 as this is a Julia 1.10 bug