Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
MPI = "da04e1cc-30fd-572f-bb4f-1f8673147195"
MPIPreferences = "3da0fdf6-3ccc-4f1b-acd9-58baa6c99267"
Preferences = "21216c6a-2e73-6563-6e65-726566657250"
OffsetArrays = "6fe1bfb0-de20-5000-8ca7-80f57d26f881"
PETSc_jll = "8fa3689e-f0b9-5420-9873-adf6ccf46f2d"
Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
Expand All @@ -22,6 +23,7 @@ Libdl = "^1.10"
LinearAlgebra = "^1.10"
MPI = "0.20"
MPIPreferences = "0.1"
Preferences = "1"
OffsetArrays = "1.0"
PETSc_jll = "3.22"
Pkg = "^1.10"
Expand Down
15 changes: 6 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,19 +29,16 @@ julia>]test PETSc

By default, the package uses a pre-built binary of PETSc (see [PETSc_jll](https://github.com/JuliaBinaryWrappers/PETSc_jll.jl)) along with a default installation of `MPI.jl`, so you don't have to install it on your machine.

If you want to use the package with custom builds of the PETSc library, this can be done by using the function `set_petsclib` which requires you to point to the correct dynamic library (which should be compatible with the MPI version used by `MPI.jl`)
If you want to use the package with a custom PETSc build, use `set_library!` to configure it once — the path is stored persistently in `LocalPreferences.toml` and no environment variables are needed:

```julia
using PETSc

# Create custom library instance
petsclib = set_petsclib("/path/to/custom/libpetsc.so";
PetscScalar=Float64, PetscInt=Int64)
# Use it like any precompiled library
PETSc.initialize(petsclib, log_view=true)
# ... your code ...
PETSc.finalize(petsclib)
PETSc.set_library!("/path/to/custom/libpetsc.so"; PetscScalar=Float64, PetscInt=Int64)
# Restart Julia — PETSc_jll is not loaded and your library is used automatically.
```

To revert to the bundled binaries: `PETSc.unset_library!()`. To check the current configuration: `PETSc.library_info()`.

To get an overview of available precompiled libraries:
```julia
julia>using PETSc
Expand Down
22 changes: 20 additions & 2 deletions docs/src/man/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,27 @@


## 1. Can I use my own PETSc library?
Yes, see the function `set_petsclib`. You do need to compile this library as a dynamic library, and `MPI.jl` must be configured to be compatible with the MPI library used to compile PETSc. If you run this on a HPC system, and you don't know exactly which options were used, you can compile one of the PETSc examples and run it with the `-log_view` command line option. At the end of the simulation, it will give you the configuration options used on that machine.
Yes. You do need to compile PETSc as a dynamic (shared) library, and `MPI.jl` must be configured to be compatible with the MPI used to compile PETSc. If you run this on an HPC system and don't know the configuration options, compile one of the PETSc examples and run it with `-log_view`; the output lists all configuration options used on that machine.

Please note that the version of PETSc should be compatible with the version used for the wrapper.
Please note that the version of PETSc should be compatible with the version used for the wrappers.

The recommended approach is `set_library!`, which stores the path persistently in `LocalPreferences.toml` — no environment variables needed:

```julia
using PETSc
PETSc.set_library!("/path/to/libpetsc.so"; PetscScalar=Float64, PetscInt=Int64)
# Restart Julia — the custom library is used automatically from here on.
```

To revert to the bundled `PETSc_jll` binaries:
```julia
PETSc.unset_library!()
```

For a one-off session without changing the persistent preference, use `set_petsclib` directly:
```julia
petsclib = PETSc.set_petsclib("/path/to/libpetsc.so"; PetscScalar=Float64, PetscInt=Int64)
```

## 2. Help, my code crashes?
That is very possible. If you provide a *short* minimum working example (MWE), feel free to open an issue on the github repo, so we can check it out.
Expand Down
28 changes: 21 additions & 7 deletions docs/src/man/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

- [Getting started](#getting-started)
- [1a. Installation using pre-built libraries](#1a-installation-using-pre-built-libraries)
- [1b. Installation using pre-built libraries](#1b-installation-using-pre-built-libraries)
- [1b. Installation using a custom PETSc build](#1b-installation-using-a-custom-petsc-build)
- [2. Solving a linear system of equations](#2-solving-a-linear-system-of-equations)
- [3. Nonlinear example](#3-nonlinear-example)
- [4. Next steps](#4-next-steps)
Expand All @@ -21,14 +21,28 @@ which will install a pre-built PETSc library (`PETSc_jll`) as well as `MPI.jl` o

**Windows users are therefore advised to install the [Windows Subsystem for Linux](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux) (WSL) and run PETSc.jl from within WSL.** This will provide full functionality with both serial and parallel (MPI) support.

### 1b. Installation using pre-built libraries
On many high-performance clusters, you will have to use the provided `MPI` installation for that cluster and the default download above will not be sufficient. Alternatively, you may be interested in a PETSc installation that comes with additional external packages. Ensure that this PETSc installation is compiled as a dynamic (and not a static) library, after which you need to specify the correct library with:
### 1b. Installation using a custom PETSc build
Sometimes, you may be interested in a PETSc installation that comes with additional external packages, or that you compiled yourself. Ensure the library is compiled as a **dynamic** (not static) library.

Use `set_library!` to configure the path once — it is stored in `LocalPreferences.toml` and no environment variables are needed afterwards:

```julia
using PETSc
PETSc.set_library!(
"/path/to/custom/libpetsc.so";
PetscScalar = Float64,
PetscInt = Int64,
)
# Restart Julia — PETSc_jll is not loaded and your library is used automatically.
```

To revert to the bundled binaries: `PETSc.unset_library!()`.

For a one-off session without changing persistent settings, use `set_petsclib` directly:

```julia
# Create custom library instance
petsclib = set_petsclib("/path/to/custom/libpetsc.so";
PetscScalar=Float64, PetscInt=Int64)
# Use it like any precompiled library
petsclib = PETSc.set_petsclib("/path/to/custom/libpetsc.so";
PetscScalar=Float64, PetscInt=Int64)
PETSc.initialize(petsclib, log_view=true)
# ... your code ...
PETSc.finalize(petsclib)
Expand Down
24 changes: 13 additions & 11 deletions docs/src/man/lowlevel_intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,28 +71,30 @@ You can link to your own custom build of PETSc instead of using the prebuilt bin
- Debug builds for troubleshooting
- Integration with specific external packages

To use a custom PETSc installation:
To persistently configure a custom PETSc installation (recommended):

```julia
using PETSc
PETSc.set_library!("/path/to/your/libpetsc.so"; PetscScalar=Float64, PetscInt=Int64)
# Restart Julia — PETSc_jll is not loaded and your library is used automatically.
```

# Create a custom library instance pointing to your PETSc installation
petsclib = PETSc.set_petsclib("/path/to/your/libpetsc.so";
PetscScalar=Float64,
PetscInt=Int64)
To revert: `PETSc.unset_library!()`. To inspect the current config: `PETSc.library_info()`.

# Initialize and use as normal
PETSc.initialize(petsclib)
For a one-off session without changing persistent settings:

```julia
petsclib = PETSc.set_petsclib("/path/to/your/libpetsc.so";
PetscScalar=Float64, PetscInt=Int64)
PETSc.initialize(petsclib)
# ... your code using petsclib ...

PETSc.finalize(petsclib)
```

**Important notes for custom builds:**
- The dynamic library path should point to `libpetsc.so` (Linux), `libpetsc.dylib` (macOS), or `libpetsc.dll` (Windows)
- The `PetscScalar` and `PetscInt` types must match how your PETSc was configured
- Your custom PETSc must be compatible with the MPI version used by `MPI.jl`
- The library must be compiled as a dynamic/shared library (`libpetsc.so` / `.dylib`)
- `PetscScalar` and `PetscInt` must match how your PETSc was configured
- Your PETSc must be linked against the same MPI that `MPI.jl` uses
- You can check available precompiled libraries with `[PETSc.petsclibs...]`

### 2. Zero-Based Indexing
Expand Down
16 changes: 14 additions & 2 deletions src/LibPETSc_startup.jl
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,11 @@ function getlibs()
return libs
end
=#
const libs = @static if !haskey(ENV, "JULIA_PETSC_LIBRARY")
using Preferences

# @load_preference is evaluated at compile time; changing a preference via
# set_library!() triggers automatic recompilation (standard Preferences.jl behaviour).
const libs = @static if @load_preference("library_path", nothing) === nothing
using PETSc_jll
(
((PETSc_jll.libpetsc_Float64_Real_Int64,), Float64, Int64),
Expand All @@ -25,7 +29,15 @@ const libs = @static if !haskey(ENV, "JULIA_PETSC_LIBRARY")
((PETSc_jll.libpetsc_Float32_Complex_Int32,), Complex{Float32}, Int32),
)
else
error("JULIA_PETSC_LIBRARY not currently working")
let _path = @load_preference("library_path"),
_scalar = @load_preference("PetscScalar", "Float64"),
_int = @load_preference("PetscInt", "Int64")
_st = _scalar == "Float32" ? Float32 :
_scalar == "ComplexFloat64" ? Complex{Float64} :
_scalar == "ComplexFloat32" ? Complex{Float32} : Float64
_it = _int == "Int32" ? Int32 : Int64
(((_path,), _st, _it),)
end
end

const petsc_library_file =
Expand Down
3 changes: 2 additions & 1 deletion src/PETSc.jl
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

module PETSc

using MPI, LinearAlgebra, SparseArrays, OffsetArrays
using MPI, LinearAlgebra, SparseArrays, OffsetArrays, Preferences

MPI.Initialized() || MPI.Init()

Expand All @@ -23,6 +23,7 @@ using .LibPETSc
export LibPETSc
export audit_petsc_file
export set_petsclib
export set_library!, unset_library!, library_info

using Libdl

Expand Down
133 changes: 111 additions & 22 deletions src/init.jl
Original file line number Diff line number Diff line change
Expand Up @@ -199,39 +199,47 @@ inttype(

Create a custom PETSc library instance from a user-specified shared library path.

This function allows you to use a custom-compiled PETSc library instead of the
pre-built libraries provided by PETSc_jll. The custom library must be:
- Compiled as a shared/dynamic library (not static)
- Built with the correct scalar type (real vs complex) and integer size
- Compatible with the same MPI installation that MPI.jl is using
This function allows you to use a custom-compiled PETSc library instead of the
pre-built libraries provided by `PETSc_jll`. The custom library must be compiled as a
shared/dynamic library (not static), built with the matching scalar type and integer
size, and linked against the same MPI installation that `MPI.jl` uses.

On HPC systems, set `JULIA_PETSC_SKIP_JLL=1` before starting Julia to prevent
`PETSc_jll` from being precompiled (its MPI stack is typically incompatible with
cluster MPI). Then call this function in your script to load the cluster library.

# Arguments
- `library_path::String`: Path to the custom PETSc shared library (e.g., "/path/to/libpetsc.so")
- `PetscScalar::Type`: Scalar type used by the library. Options: `Float64`, `Float32`,
`Complex{Float64}`, or `Complex{Float32}`. Default: `Float64`
- `PetscInt::Type`: Integer type used by the library. Options: `Int32` or `Int64`.
- `library_path::String`: Path to the PETSc shared library (e.g. `"/path/to/libpetsc.so"`)
- `PetscScalar::Type`: Scalar type the library was built with. One of `Float64`,
`Float32`, `Complex{Float64}`, `Complex{Float32}`. Default: `Float64`
- `PetscInt::Type`: Integer type the library was built with. `Int32` or `Int64`.
Default: `Int64`

# Returns
- A `PetscLibType` instance that can be used with `initialize()`, `finalize()`, and
all other PETSc.jl functions
A `PetscLibType` instance for use with `initialize`, `finalize`, and all PETSc.jl functions.

# Environment-variable alternative
Instead of calling `set_petsclib`, you can configure everything before Julia starts:
```
JULIA_PETSC_LIBRARY=/path/to/libpetsc.so # also suppresses PETSc_jll
JULIA_PETSC_SCALAR=Float64 # Float32 | ComplexFloat64 | ComplexFloat32
JULIA_PETSC_INT=Int64 # Int32
```
With these set, `PETSc.getlib(; PetscScalar=Float64, PetscInt=Int64)` returns the
custom library directly.

# Examples
```julia
using PETSc

# For a custom double-precision real PETSc library with 64-bit indices
petsclib = PETSc.set_petsclib("/path/to/custom/libpetsc.so";
PetscScalar=Float64, PetscInt=Int64)

# For a single-precision complex library with 32-bit indices
petsclib = PETSc.set_petsclib("/opt/petsc/lib/libpetsc.so";
PetscScalar=Complex{Float32}, PetscInt=Int32)

# Initialize and use the custom library
# Double-precision real, 64-bit indices (typical HPC build)
petsclib = PETSc.set_petsclib("/path/to/libpetsc.so";
PetscScalar=Float64, PetscInt=Int64)
PETSc.initialize(petsclib)
# ... your code ...
PETSc.finalize(petsclib)

# Single-precision complex, 32-bit indices
petsclib = PETSc.set_petsclib("/opt/petsc/lib/libpetsc.so";
PetscScalar=Complex{Float32}, PetscInt=Int32)
```

# See Also
Expand All @@ -248,6 +256,87 @@ function set_petsclib(library_path::String; PetscScalar::Type=Float64, PetscInt:
return petsclib
end

"""
library_info()

Print the current PETSc library configuration: which library is in use, how it
was configured (preference or default JLL), and the scalar/integer types.
"""
function library_info()
pref_path = @load_preference("library_path", nothing)
pref_scalar = @load_preference("PetscScalar", nothing)
pref_int = @load_preference("PetscInt", nothing)

if pref_path !== nothing
println("Source : LocalPreferences.toml")
println("Path : ", pref_path)
println("Scalar : ", something(pref_scalar, "Float64"))
println("Int : ", something(pref_int, "Int64"))
else
println("Source : PETSc_jll (default precompiled binaries)")
end

println("\nLoaded libraries (this session):")
for lib in petsclibs
path = lib.petsc_library isa AbstractString ? lib.petsc_library :
try Libdl.dlpath(Libdl.dlopen(lib.petsc_library)) catch; string(lib.petsc_library) end
println(" [$(lib.PetscScalar), $(lib.PetscInt)]: ", path)
end
end

"""
set_library!(path; PetscScalar=Float64, PetscInt=Int64)

Persistently configure PETSc.jl to use a custom PETSc shared library.

The path and type configuration are stored in `LocalPreferences.toml` (per-project,
git-ignorable) and take effect on the next Julia session. Recompilation is triggered
automatically — no environment variables are needed.

To revert to the default `PETSc_jll` libraries, call [`unset_library!`](@ref).

# Arguments
- `path`: path to the PETSc shared library (e.g. `"/path/to/libpetsc.so"`)
- `PetscScalar`: scalar type the library was built with (`Float64`, `Float32`,
`Complex{Float64}`, `Complex{Float32}`). Default: `Float64`
- `PetscInt`: integer type the library was built with (`Int64` or `Int32`).
Default: `Int64`

# Examples
```julia
PETSc.set_library!(
"/project/petsc/lib/libpetsc.so";
PetscScalar = Float64,
PetscInt = Int64,
)
# Restart Julia — the new library is used automatically from here on.
```

# See Also
- [`unset_library!`](@ref): remove the preference and revert to `PETSc_jll`
- [`set_petsclib`](@ref): load a custom library for the current session only
"""
function set_library!(path; PetscScalar::Type=Float64, PetscInt::Type=Int64)
ispath(path) || error("PETSc library not found: $path")
@set_preferences!(
"library_path" => realpath(path),
"PetscScalar" => string(PetscScalar),
"PetscInt" => string(PetscInt),
)
@info "PETSc library configured — restart Julia to use the new library." path PetscScalar PetscInt
end

"""
unset_library!()

Remove the persistent custom-library preference set by [`set_library!`](@ref),
reverting to the default `PETSc_jll` binaries on the next Julia session.
"""
function unset_library!()
@delete_preferences!("library_path", "PetscScalar", "PetscInt")
@info "PETSc library preference removed — restart Julia to revert to PETSc_jll."
end


"""
check_petsc_wrappers_version(petsclib=nothing)
Expand Down
Loading