Skip to content

Commit ec20f8b

Browse files
author
Circle CI
committed
Update docs for dev (build 4264)
1 parent b73017e commit ec20f8b

22 files changed

Lines changed: 283 additions & 147 deletions

File tree

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

dev/_sources/auto_examples/run_minimal_benchmark.rst

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -69,10 +69,10 @@ To run the benchmark, just execute:
6969
<span style="color: #000080; text-decoration-color: #000080; font-weight: bold"> |--gd[lr=0.01]:</span> <span style="color: #808000; text-decoration-color: #808000; font-weight: bold">done (not enough run)</span>
7070
<span style="color: #008000; text-decoration-color: #008000; font-weight: bold">Saving result in: </span>
7171
<span style="color: #008000; text-decoration-color: #008000; font-weight: bold">/home/circleci/project/examples/minimal_benchmark/outputs/benchopt_run_2026-03</span>
72-
<span style="color: #008000; text-decoration-color: #008000; font-weight: bold">-01_19h47m53.parquet</span>
73-
Processing
72+
<span style="color: #008000; text-decoration-color: #008000; font-weight: bold">-02_12h26m27.parquet</span>
73+
Processing
7474
/home/circleci/project/examples/minimal_benchmark/outputs/benchopt_run_2026-03
75-
-01_19h47m53.parquet
75+
-02_12h26m27.parquet
7676

7777

7878
</pre></div>
@@ -121,10 +121,10 @@ To get a more precise curve, you can increase ``n`` and ``r``:
121121
<span style="color: #000080; text-decoration-color: #000080; font-weight: bold"> |--gd[lr=0.01]:</span> <span style="color: #008000; text-decoration-color: #008000; font-weight: bold">done</span>
122122
<span style="color: #008000; text-decoration-color: #008000; font-weight: bold">Saving result in: </span>
123123
<span style="color: #008000; text-decoration-color: #008000; font-weight: bold">/home/circleci/project/examples/minimal_benchmark/outputs/benchopt_run_2026-03</span>
124-
<span style="color: #008000; text-decoration-color: #008000; font-weight: bold">-01_19h48m08.parquet</span>
125-
Processing
124+
<span style="color: #008000; text-decoration-color: #008000; font-weight: bold">-02_12h26m37.parquet</span>
125+
Processing
126126
/home/circleci/project/examples/minimal_benchmark/outputs/benchopt_run_2026-03
127-
-01_19h48m08.parquet
127+
-02_12h26m37.parquet
128128

129129

130130
</pre></div>
@@ -153,7 +153,7 @@ suboptimality instead of objective value.
153153

154154
.. rst-class:: sphx-glr-timing
155155

156-
**Total running time of the script:** (0 minutes 19.234 seconds)
156+
**Total running time of the script:** (0 minutes 14.948 seconds)
157157

158158

159159
.. _sphx_glr_download_auto_examples_run_minimal_benchmark.py:

dev/_sources/auto_examples/sg_execution_times.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
Computation times
88
=================
9-
**00:19.234** total execution time for 3 files **from auto_examples**:
9+
**00:14.948** total execution time for 3 files **from auto_examples**:
1010

1111
.. container::
1212

@@ -33,7 +33,7 @@ Computation times
3333
- Time
3434
- Mem (MB)
3535
* - :ref:`sphx_glr_auto_examples_run_minimal_benchmark.py` (``run_minimal_benchmark.py``)
36-
- 00:19.234
36+
- 00:14.948
3737
- 0.0
3838
* - :ref:`sphx_glr_auto_examples_plot_run_benchmark.py` (``plot_run_benchmark.py``)
3939
- 00:00.000

dev/_sources/benchmark_workflow/manage_benchmark_results.rst

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,21 +23,21 @@ The DataFrame contains the following columns:
2323
objective and solver, displayed in the HTML page.
2424
- ``file_objective|file_solver|file_dataset``: The filename of the objective,
2525
solver, and dataset used for this line.
26-
- ``p_obj_p``: the value of the objective's parameter ``p``.
27-
- ``p_solver_p``: the value of the solver's parameter ``p``.
28-
- ``p_dateset_p``: the value of the dataset's parameter ``p``.
26+
- ``p_obj_{p}``: the value of the objective's parameter ``p``.
27+
- ``p_solver_{p}``: the value of the solver's parameter ``p``.
28+
- ``p_dataset_{p}``: the value of the dataset's parameter ``p``.
2929
- ``time``: the time taken to run the solver until this point of the performance curve.
3030
- ``stop_val``: the number of iterations or the tolerance reached by the solver.
3131
- ``idx_rep``: If multiple repetitions are run for each solver with ``--n-rep``,
3232
this column contains the repetition number.
3333
- ``sampling_strategy``: The sampling strategy used to generate the performance
3434
curve of this solver. This allow to adapt the plot in the HTML depending on
3535
each solver.
36-
- ``objective_k``: The value associated to the key ``k`` in the
36+
- ``objective_{k}``: The value associated to the key ``k`` in the
3737
``Objective.evaluate_result`` dictionary.
3838

3939
The remaining columns are informations about the system used to run the
40-
benchmark, with keys ``{'env_OMP_NUM_THREADS', 'platform', 'platform-architecture', 'platform-release', 'platform-version', 'system-cpus', 'system-processor', 'system-ram (GB)', 'version-cuda', 'version-numpy', 'version-scipy', 'benchmark-git-tag'}``.
40+
benchmark, with keys ``{'env_OMP_NUM_THREADS', 'platform', 'platform-architecture', 'platform-release', 'platform-version', 'system-cpus', 'system-processor', 'system-ram (GB)', 'version-cuda', 'version-numpy', 'version-scipy', 'benchmark-git-tag', 'run_date'}``.
4141

4242

4343
.. _collect_results:
@@ -52,6 +52,30 @@ computed can be collected using the :option:`--collect` option with
5252
produce a parquet file with all the results that have been computed so far.
5353

5454

55+
.. _merge_results:
56+
57+
Merge results from multiple runs
58+
--------------------------------
59+
60+
Multiple runs of the same benchmark can be merged together using the
61+
``benchopt merge`` command. This allows to collect multiple runs, or to
62+
aggregate results from different users. By default, the merged results contain
63+
all lines from the different result files, with no additional processing.
64+
65+
The resulting file is stored in the benchmark directory with the name
66+
``merged_results.parquet``. The name can be changed using the :option:`--output` option of the ``benchopt merge`` command.
67+
68+
It is also possible to specify the ``--overwrite`` option to only keep one line
69+
per ``objective_name``, ``solver_name``, ``dataset_name``, and
70+
``idx_repetition``. In this case, only the line from the most recent run will
71+
be kept for each evaluation.
72+
73+
Note that the run date is included in the result file in the ``run_date``
74+
columm. If it is not present in the original result file, benchopt uses
75+
the file creation date as a proxy for the run date. This allows to easily
76+
identify the most recent line when using the ``--overwrite`` option.
77+
78+
5579
Clean benchmark results
5680
-----------------------
5781

dev/_sources/sg_execution_times.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
Computation times
88
=================
9-
**00:19.234** total execution time for 3 files **from all galleries**:
9+
**00:14.948** total execution time for 3 files **from all galleries**:
1010

1111
.. container::
1212

@@ -33,7 +33,7 @@ Computation times
3333
- Time
3434
- Mem (MB)
3535
* - :ref:`sphx_glr_auto_examples_run_minimal_benchmark.py` (``../examples/run_minimal_benchmark.py``)
36-
- 00:19.234
36+
- 00:14.948
3737
- 0.0
3838
* - :ref:`sphx_glr_auto_examples_plot_run_benchmark.py` (``../examples/plot_run_benchmark.py``)
3939
- 00:00.000

dev/_sources/whats_new.rst

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,13 @@ What's new
1010
Version 1.9 - In development
1111
----------------------------
1212

13+
CLI
14+
---
15+
16+
- Add ``benchopt merge`` to merge results from multiple benchopt runs.
17+
See :ref:`merge_results` for more details.
18+
By `Thomas Moreau`_ (:gh:`892`)
19+
1320
API
1421
---
1522

@@ -29,7 +36,7 @@ FIX
2936
- Fix failure in ``benchopt test`` when using ``get_seed`` in a class.
3037
By `Thomas Moreau`_ (:gh:`889`)
3138

32-
- Fix conda env creation and ``--recreate`` option, ot include minimal reqs.
39+
- Fix conda env creation and ``--recreate`` option, to include minimal reqs.
3340
By `Thomas Moreau`_ (:gh:`891`)
3441

3542
.. _changes_1_8_1:

0 commit comments

Comments
 (0)