diff --git a/tutorials/introduction-to-solverbenchmark/Project.toml b/tutorials/introduction-to-solverbenchmark/Project.toml index c032d60..04e073c 100644 --- a/tutorials/introduction-to-solverbenchmark/Project.toml +++ b/tutorials/introduction-to-solverbenchmark/Project.toml @@ -1,14 +1,18 @@ [deps] DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" +NLPModelsTest = "7998695d-6960-4d3a-85c4-e1bceb8cd856" Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80" Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7" PyPlot = "d330b81b-6aea-500a-939a-2ce795aea3ee" Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" SolverBenchmark = "581a75fa-a23a-52d0-a590-d6201de2218a" +SolverCore = "ff4d7338-4cf1-434d-91df-b86cb86fb843" [compat] DataFrames = "1.3.4" +NLPModelsTest = "0.9" Plots = "1.31.7" PyPlot = "2.10.0" SolverBenchmark = "0.5.3" +SolverCore = "0.3" diff --git a/tutorials/introduction-to-solverbenchmark/index.jmd b/tutorials/introduction-to-solverbenchmark/index.jmd index df2099b..09ef6b5 100644 --- a/tutorials/introduction-to-solverbenchmark/index.jmd +++ b/tutorials/introduction-to-solverbenchmark/index.jmd @@ -221,3 +221,26 @@ p = profile_solvers(stats, costs, costnames) Here is a useful tutorial on how to use the benchmark with specific solver: [Run a benchmark with OptimizationProblems](https://jso.dev/OptimizationProblems.jl/dev/benchmark/) The tutorial covers how to use the problems from `OptimizationProblems` to run a benchmark for unconstrained optimization. + +### Handling `solver_specific` in `stats` + +If a solver's `GenericExecutionStats` contains a `solver_specific` dictionary, then when `bmark_solvers` processes the results it creates a column in the per-solver `DataFrame` for each key in that dictionary. These columns can then be analyzed and compared alongside the standard metrics such as `status` and `elapsed_time`. + +Here is an example showing how to set a solver-specific flag so that it appears as a column in the resulting stats table and can be used for tabulation: +```julia +using NLPModelsTest, DataFrames, SolverCore, SolverBenchmark + +function newton(nlp) + stats = GenericExecutionStats(nlp) + set_solver_specific!(stats, :isConvex, true) + return stats +end + +solvers = Dict(:newton => newton) +problems = [NLPModelsTest.BROWNDEN()] +stats = bmark_solvers(solvers, problems) + +# Access the solver-specific column `:isConvex` for the `:newton` solver +df_newton = stats[:newton] +df_newton.isConvex +```