Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions tutorials/introduction-to-solverbenchmark/index.jmd
Original file line number Diff line number Diff line change
Expand Up @@ -221,3 +221,34 @@ p = profile_solvers(stats, costs, costnames)
Here is a useful tutorial on how to use the benchmark with specific solver:
[Run a benchmark with OptimizationProblems](https://jso.dev/OptimizationProblems.jl/dev/benchmark/)
The tutorial covers how to use the problems from `OptimizationProblems` to run a benchmark for unconstrained optimization.

## Handling `solver_specific` in benchmark runs
Comment thread
arnavk23 marked this conversation as resolved.
Outdated

`SolverBenchmark` accepts solver-specific options via the keyword argument `solver_specific` on benchmarking functions (for example, when calling the high-level benchmarking helpers such as `bmark_solvers`).

- `solver_specific` is a mapping that associates a solver identifier (typically a `Symbol` or the solver name you use in the benchmark) with a dictionary of options specific to that solver implementation.
- Those options are passed through to the solver when the run is executed and are recorded as part of the run metadata.
- As a result, runs that differ only by solver-specific options are treated as distinct entries in the benchmark results; you can group, filter or compare them in the produced `DataFrame`s.

```julia
using SolverBenchmark, NLPModelsTest, JSOSolvers
Comment thread
arnavk23 marked this conversation as resolved.
Outdated

solver_specific = Dict(
:IPOPT => Dict("tol" => 1e-8, "max_iter" => 200),
:KNITRO => Dict("maxit" => 500)
)

results = bmark_solvers(solvers, problems; solver_specific = solver_specific)
Comment thread
arnavk23 marked this conversation as resolved.
Outdated
Comment thread
arnavk23 marked this conversation as resolved.
Outdated

first(results)
```

Notes and tips:

- If you pass different option sets for the same solver, make sure the keys you use
identify the intended solver variation unambiguously (e.g., distinct Symbols).
- When creating tables or profiles, you can add or extract columns from the per-solver
DataFrames to show which solver-specific options were used for each run.
- If you prefer the solver-specific settings to appear as explicit columns, you can
preprocess the DataFrames (extract the dict entries into columns) before creating
joined tables or profile plots.