Use local dynamic libraries rather than jll binaries (#235)

* update initialisation to allow specfying a lobrary on HPC systems:

We now have 2 options:

# Option 1: point directly to your library (sets libs AND skips JLL)
export JULIA_PETSC_LIBRARY=/path/to/libpetsc.so

# Option 2: skip JLL, then use set_petsclib() in your script
export JULIA_PETSC_SKIP_JLL=1
With option 2, getlib() won't work (no libs registered), but set_petsclib("/path/to/libpetsc.so") gives you full control over scalar/int types.

* configure type of library through ENV variables:

export JULIA_PETSC_LIBRARY=/path/to/libpetsc.so
export JULIA_PETSC_SCALAR=Float64        # Float32 | ComplexFloat64 | ComplexFloat32
export JULIA_PETSC_INT=Int64             # Int32

* update docs

* update getting started page

* replace environmental variables by preferences

* oops

* load preferences

Author: boriskaus

Project.toml

@@ -9,6 +9,7 @@ Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
 MPI = "da04e1cc-30fd-572f-bb4f-1f8673147195"
 MPIPreferences = "3da0fdf6-3ccc-4f1b-acd9-58baa6c99267"
+Preferences = "21216c6a-2e73-6563-6e65-726566657250"
 OffsetArrays = "6fe1bfb0-de20-5000-8ca7-80f57d26f881"
 PETSc_jll = "8fa3689e-f0b9-5420-9873-adf6ccf46f2d"
 Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
@@ -22,6 +23,7 @@ Libdl = "^1.10"
 LinearAlgebra = "^1.10"
 MPI = "0.20"
 MPIPreferences = "0.1"
+Preferences = "1"
 OffsetArrays = "1.0"
 PETSc_jll = "3.22"
 Pkg = "^1.10"

README.md

@@ -29,19 +29,16 @@ julia>]test PETSc
 
 By default, the package uses a pre-built binary of PETSc (see [PETSc_jll](https://github.com/JuliaBinaryWrappers/PETSc_jll.jl)) along with a default installation of `MPI.jl`, so you don't have to install it on your machine.
 
-If you want to use the package with custom builds of the PETSc library, this can be done by using the function `set_petsclib` which requires you to point to the correct dynamic library (which should be compatible with the MPI version used by `MPI.jl`)
+If you want to use the package with a custom PETSc build, use `set_library!` to configure it once — the path is stored persistently in `LocalPreferences.toml` and no environment variables are needed:
 
 ```julia
 using PETSc
-
-# Create custom library instance
-petsclib = set_petsclib("/path/to/custom/libpetsc.so"; 
-                       PetscScalar=Float64, PetscInt=Int64)
-# Use it like any precompiled library
-PETSc.initialize(petsclib, log_view=true)
-# ... your code ...
-PETSc.finalize(petsclib)
+PETSc.set_library!("/path/to/custom/libpetsc.so"; PetscScalar=Float64, PetscInt=Int64)
+# Restart Julia — PETSc_jll is not loaded and your library is used automatically.
 ```
+
+To revert to the bundled binaries: `PETSc.unset_library!()`. To check the current configuration: `PETSc.library_info()`.
+
 To get an overview of available precompiled libraries:
 ```julia
 julia>using PETSc

docs/src/man/FAQ.md

@@ -2,9 +2,27 @@
 
 
 ## 1. Can I use my own PETSc library?
-Yes, see the function `set_petsclib`. You do need to compile this library as a dynamic library, and `MPI.jl` must be configured to be compatible with the MPI library used to compile PETSc. If you run this on a HPC system, and you don't know exactly which options were used, you can compile one of the PETSc examples and run it with the `-log_view` command line option. At the end of the simulation, it will give you the configuration options used on that machine.  
+Yes. You do need to compile PETSc as a dynamic (shared) library, and `MPI.jl` must be configured to be compatible with the MPI used to compile PETSc. If you run this on an HPC system and don't know the configuration options, compile one of the PETSc examples and run it with `-log_view`; the output lists all configuration options used on that machine.
 
-Please note that the version of PETSc should be compatible with the version used for the wrapper.
+Please note that the version of PETSc should be compatible with the version used for the wrappers.
+
+The recommended approach is `set_library!`, which stores the path persistently in `LocalPreferences.toml` — no environment variables needed:
+
+```julia
+using PETSc
+PETSc.set_library!("/path/to/libpetsc.so"; PetscScalar=Float64, PetscInt=Int64)
+# Restart Julia — the custom library is used automatically from here on.
+```
+
+To revert to the bundled `PETSc_jll` binaries:
+```julia
+PETSc.unset_library!()
+```
+
+For a one-off session without changing the persistent preference, use `set_petsclib` directly:
+```julia
+petsclib = PETSc.set_petsclib("/path/to/libpetsc.so"; PetscScalar=Float64, PetscInt=Int64)
+```
 
 ## 2. Help, my code crashes?
 That is very possible. If you provide a *short* minimum working example (MWE), feel free to open an issue on the github repo, so we can check it out.

docs/src/man/getting_started.md

@@ -3,7 +3,7 @@
 
 - [Getting started](#getting-started)
     - [1a. Installation using pre-built libraries](#1a-installation-using-pre-built-libraries)
-    - [1b. Installation using pre-built libraries](#1b-installation-using-pre-built-libraries)
+    - [1b. Installation using a custom PETSc build](#1b-installation-using-a-custom-petsc-build)
     - [2. Solving a linear system of equations](#2-solving-a-linear-system-of-equations)
     - [3. Nonlinear example](#3-nonlinear-example)
     - [4. Next steps](#4-next-steps)
@@ -21,14 +21,28 @@ which will install a pre-built PETSc library (`PETSc_jll`) as well as `MPI.jl` o
 
     **Windows users are therefore advised to install the [Windows Subsystem for Linux](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux) (WSL) and run PETSc.jl from within WSL.** This will provide full functionality with both serial and parallel (MPI) support.
 
-### 1b. Installation using pre-built libraries 
-On many high-performance clusters, you will have to use the provided `MPI` installation for that cluster and the default download above will not be sufficient. Alternatively, you may be interested in a PETSc installation that comes with additional external packages. Ensure that this PETSc installation is compiled as a dynamic (and not a static) library, after which you need to specify the correct library with:
+### 1b. Installation using a custom PETSc build
+Sometimes, you may be interested in a PETSc installation that comes with additional external packages, or that you compiled yourself. Ensure the library is compiled as a **dynamic** (not static) library.
+
+Use `set_library!` to configure the path once — it is stored in `LocalPreferences.toml` and no environment variables are needed afterwards:
+
+```julia
+using PETSc
+PETSc.set_library!(
+    "/path/to/custom/libpetsc.so";
+    PetscScalar = Float64,
+    PetscInt    = Int64,
+)
+# Restart Julia — PETSc_jll is not loaded and your library is used automatically.
+```
+
+To revert to the bundled binaries: `PETSc.unset_library!()`.
+
+For a one-off session without changing persistent settings, use `set_petsclib` directly:
 
 ```julia
-# Create custom library instance
-petsclib = set_petsclib("/path/to/custom/libpetsc.so"; 
-                             PetscScalar=Float64, PetscInt=Int64)
-# Use it like any precompiled library
+petsclib = PETSc.set_petsclib("/path/to/custom/libpetsc.so";
+                              PetscScalar=Float64, PetscInt=Int64)
 PETSc.initialize(petsclib, log_view=true)
 # ... your code ...
 PETSc.finalize(petsclib)

docs/src/man/lowlevel_intro.md

@@ -71,28 +71,30 @@ You can link to your own custom build of PETSc instead of using the prebuilt bin
 - Debug builds for troubleshooting
 - Integration with specific external packages
 
-To use a custom PETSc installation:
+To persistently configure a custom PETSc installation (recommended):
 
 ```julia
 using PETSc
+PETSc.set_library!("/path/to/your/libpetsc.so"; PetscScalar=Float64, PetscInt=Int64)
+# Restart Julia — PETSc_jll is not loaded and your library is used automatically.
+```
 
-# Create a custom library instance pointing to your PETSc installation
-petsclib = PETSc.set_petsclib("/path/to/your/libpetsc.so"; 
-                             PetscScalar=Float64, 
-                             PetscInt=Int64)
+To revert: `PETSc.unset_library!()`. To inspect the current config: `PETSc.library_info()`.
 
-# Initialize and use as normal
-PETSc.initialize(petsclib)
+For a one-off session without changing persistent settings:
 
+```julia
+petsclib = PETSc.set_petsclib("/path/to/your/libpetsc.so";
+                              PetscScalar=Float64, PetscInt=Int64)
+PETSc.initialize(petsclib)
 # ... your code using petsclib ...
-
 PETSc.finalize(petsclib)
 ```
 
 **Important notes for custom builds:**
-- The dynamic library path should point to `libpetsc.so` (Linux), `libpetsc.dylib` (macOS), or `libpetsc.dll` (Windows)
-- The `PetscScalar` and `PetscInt` types must match how your PETSc was configured
-- Your custom PETSc must be compatible with the MPI version used by `MPI.jl`
+- The library must be compiled as a dynamic/shared library (`libpetsc.so` / `.dylib`)
+- `PetscScalar` and `PetscInt` must match how your PETSc was configured
+- Your PETSc must be linked against the same MPI that `MPI.jl` uses
 - You can check available precompiled libraries with `[PETSc.petsclibs...]`
 
 ### 2. Zero-Based Indexing

src/LibPETSc_startup.jl

@@ -12,7 +12,11 @@ function getlibs()
     return libs
 end
 =#
-const libs = @static if !haskey(ENV, "JULIA_PETSC_LIBRARY")
+using Preferences
+
+# @load_preference is evaluated at compile time; changing a preference via
+# set_library!() triggers automatic recompilation (standard Preferences.jl behaviour).
+const libs = @static if @load_preference("library_path", nothing) === nothing
     using PETSc_jll
     (
         ((PETSc_jll.libpetsc_Float64_Real_Int64,), Float64, Int64),
@@ -25,7 +29,15 @@ const libs = @static if !haskey(ENV, "JULIA_PETSC_LIBRARY")
         ((PETSc_jll.libpetsc_Float32_Complex_Int32,), Complex{Float32}, Int32),
     )
 else
-    error("JULIA_PETSC_LIBRARY not currently working")
+    let _path   = @load_preference("library_path"),
+        _scalar = @load_preference("PetscScalar", "Float64"),
+        _int    = @load_preference("PetscInt",    "Int64")
+        _st = _scalar == "Float32"        ? Float32          :
+              _scalar == "ComplexFloat64" ? Complex{Float64} :
+              _scalar == "ComplexFloat32" ? Complex{Float32} : Float64
+        _it = _int == "Int32" ? Int32 : Int64
+        (((_path,), _st, _it),)
+    end
 end
 
 const petsc_library_file =

src/PETSc.jl

@@ -2,7 +2,7 @@
 
 module PETSc
 
-using MPI, LinearAlgebra, SparseArrays, OffsetArrays
+using MPI, LinearAlgebra, SparseArrays, OffsetArrays, Preferences
 
 MPI.Initialized() || MPI.Init()
 
@@ -23,6 +23,7 @@ using .LibPETSc
 export LibPETSc
 export audit_petsc_file
 export set_petsclib
+export set_library!, unset_library!, library_info
 
 using Libdl
 

src/init.jl

@@ -199,39 +199,47 @@ inttype(
 
 Create a custom PETSc library instance from a user-specified shared library path.
 
-This function allows you to use a custom-compiled PETSc library instead of the 
-pre-built libraries provided by PETSc_jll. The custom library must be:
-- Compiled as a shared/dynamic library (not static)
-- Built with the correct scalar type (real vs complex) and integer size
-- Compatible with the same MPI installation that MPI.jl is using
+This function allows you to use a custom-compiled PETSc library instead of the
+pre-built libraries provided by `PETSc_jll`. The custom library must be compiled as a
+shared/dynamic library (not static), built with the matching scalar type and integer
+size, and linked against the same MPI installation that `MPI.jl` uses.
+
+On HPC systems, set `JULIA_PETSC_SKIP_JLL=1` before starting Julia to prevent
+`PETSc_jll` from being precompiled (its MPI stack is typically incompatible with
+cluster MPI). Then call this function in your script to load the cluster library.
 
 # Arguments
-- `library_path::String`: Path to the custom PETSc shared library (e.g., "/path/to/libpetsc.so")
-- `PetscScalar::Type`: Scalar type used by the library. Options: `Float64`, `Float32`, 
-  `Complex{Float64}`, or `Complex{Float32}`. Default: `Float64`
-- `PetscInt::Type`: Integer type used by the library. Options: `Int32` or `Int64`. 
+- `library_path::String`: Path to the PETSc shared library (e.g. `"/path/to/libpetsc.so"`)
+- `PetscScalar::Type`: Scalar type the library was built with. One of `Float64`,
+  `Float32`, `Complex{Float64}`, `Complex{Float32}`. Default: `Float64`
+- `PetscInt::Type`: Integer type the library was built with. `Int32` or `Int64`.
   Default: `Int64`
 
 # Returns
-- A `PetscLibType` instance that can be used with `initialize()`, `finalize()`, and 
-  all other PETSc.jl functions
+A `PetscLibType` instance for use with `initialize`, `finalize`, and all PETSc.jl functions.
+
+# Environment-variable alternative
+Instead of calling `set_petsclib`, you can configure everything before Julia starts:
+```
+JULIA_PETSC_LIBRARY=/path/to/libpetsc.so   # also suppresses PETSc_jll
+JULIA_PETSC_SCALAR=Float64                  # Float32 | ComplexFloat64 | ComplexFloat32
+JULIA_PETSC_INT=Int64                       # Int32
+```
+With these set, `PETSc.getlib(; PetscScalar=Float64, PetscInt=Int64)` returns the
+custom library directly.
 
 # Examples
 ```julia
-using PETSc
-
-# For a custom double-precision real PETSc library with 64-bit indices
-petsclib = PETSc.set_petsclib("/path/to/custom/libpetsc.so"; 
-                             PetscScalar=Float64, PetscInt=Int64)
-
-# For a single-precision complex library with 32-bit indices  
-petsclib = PETSc.set_petsclib("/opt/petsc/lib/libpetsc.so"; 
-                             PetscScalar=Complex{Float32}, PetscInt=Int32)
-
-# Initialize and use the custom library
+# Double-precision real, 64-bit indices (typical HPC build)
+petsclib = PETSc.set_petsclib("/path/to/libpetsc.so";
+                              PetscScalar=Float64, PetscInt=Int64)
 PETSc.initialize(petsclib)
 # ... your code ...
 PETSc.finalize(petsclib)
+
+# Single-precision complex, 32-bit indices
+petsclib = PETSc.set_petsclib("/opt/petsc/lib/libpetsc.so";
+                              PetscScalar=Complex{Float32}, PetscInt=Int32)
 ```
 
 # See Also
@@ -248,6 +256,87 @@ function set_petsclib(library_path::String; PetscScalar::Type=Float64, PetscInt:
     return petsclib
 end
 
+"""
+    library_info()
+
+Print the current PETSc library configuration: which library is in use, how it
+was configured (preference or default JLL), and the scalar/integer types.
+"""
+function library_info()
+    pref_path   = @load_preference("library_path", nothing)
+    pref_scalar = @load_preference("PetscScalar", nothing)
+    pref_int    = @load_preference("PetscInt", nothing)
+
+    if pref_path !== nothing
+        println("Source  : LocalPreferences.toml")
+        println("Path    : ", pref_path)
+        println("Scalar  : ", something(pref_scalar, "Float64"))
+        println("Int     : ", something(pref_int,    "Int64"))
+    else
+        println("Source  : PETSc_jll (default precompiled binaries)")
+    end
+
+    println("\nLoaded libraries (this session):")
+    for lib in petsclibs
+        path = lib.petsc_library isa AbstractString ? lib.petsc_library :
+               try Libdl.dlpath(Libdl.dlopen(lib.petsc_library)) catch; string(lib.petsc_library) end
+        println("  [$(lib.PetscScalar), $(lib.PetscInt)]: ", path)
+    end
+end
+
+"""
+    set_library!(path; PetscScalar=Float64, PetscInt=Int64)
+
+Persistently configure PETSc.jl to use a custom PETSc shared library.
+
+The path and type configuration are stored in `LocalPreferences.toml` (per-project,
+git-ignorable) and take effect on the next Julia session. Recompilation is triggered
+automatically — no environment variables are needed.
+
+To revert to the default `PETSc_jll` libraries, call [`unset_library!`](@ref).
+
+# Arguments
+- `path`: path to the PETSc shared library (e.g. `"/path/to/libpetsc.so"`)
+- `PetscScalar`: scalar type the library was built with (`Float64`, `Float32`,
+  `Complex{Float64}`, `Complex{Float32}`). Default: `Float64`
+- `PetscInt`: integer type the library was built with (`Int64` or `Int32`).
+  Default: `Int64`
+
+# Examples
+```julia
+PETSc.set_library!(
+    "/project/petsc/lib/libpetsc.so";
+    PetscScalar = Float64,
+    PetscInt    = Int64,
+)
+# Restart Julia — the new library is used automatically from here on.
+```
+
+# See Also
+- [`unset_library!`](@ref): remove the preference and revert to `PETSc_jll`
+- [`set_petsclib`](@ref): load a custom library for the current session only
+"""
+function set_library!(path; PetscScalar::Type=Float64, PetscInt::Type=Int64)
+    ispath(path) || error("PETSc library not found: $path")
+    @set_preferences!(
+        "library_path" => realpath(path),
+        "PetscScalar"  => string(PetscScalar),
+        "PetscInt"     => string(PetscInt),
+    )
+    @info "PETSc library configured — restart Julia to use the new library." path PetscScalar PetscInt
+end
+
+"""
+    unset_library!()
+
+Remove the persistent custom-library preference set by [`set_library!`](@ref),
+reverting to the default `PETSc_jll` binaries on the next Julia session.
+"""
+function unset_library!()
+    @delete_preferences!("library_path", "PetscScalar", "PetscInt")
+    @info "PETSc library preference removed — restart Julia to revert to PETSc_jll."
+end
+
 
 """
     check_petsc_wrappers_version(petsclib=nothing)