Skip to content

Commit 7cd821e

Browse files
h-g-sCopilot
andcommitted
docs: add PyPy benchmark tables for queens and TSP
Both CBC and HiGHS (CFFI-based) and Gurobi (gurobipy) are PyPy-compatible. PyPy 3.11 (7.3.20) delivers 4-5x speedup over CPython 3.14.4 for model building. Added separate PyPy tables under each benchmark section and updated the intro to explain PyPy compatibility. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 1b8392d commit 7cd821e

1 file changed

Lines changed: 101 additions & 12 deletions

File tree

docs/bench.rst

Lines changed: 101 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@ Benchmarks
55

66
This section presents computational experiments measuring **model creation
77
time** — the time from an empty model to a fully built, solver-ready instance
8-
— across different modelling interfaces and solver backends.
8+
— across different modelling interfaces, solver backends, and Python
9+
interpreters.
910

1011
Python-MIP communicates every problem modification directly to the solver
1112
engine rather than staging a separate intermediate model. To do this
@@ -22,13 +23,17 @@ efficiently without per-call overhead:
2223
- **Gurobi** provides its own internal buffering (``update`` mode) that
2324
python-mip relies on directly.
2425

26+
Both CBC and HiGHS use CFFI and are fully **PyPy-compatible**. Gurobi's
27+
``gurobipy`` extension is also compatible with PyPy. The PyPy JIT compiler
28+
eliminates most Python overhead, yielding 4–5× faster model creation times.
29+
2530
The ``highspy`` native batch API (``addVars`` / ``addRows`` with numpy arrays)
2631
represents the theoretical lower bound for HiGHS model creation: a single bulk
2732
call with a pre-built CSR matrix, bypassing all Python object overhead.
2833
The ``highspy`` high-level API (``addVariable`` / ``addConstr`` expression
2934
objects) is included for reference.
3035

31-
Experiments were run on CPython 3.14.4 on a Linux workstation.
36+
Experiments were run on a Linux workstation.
3237
Reproducible benchmark scripts are in the ``benchmarks/`` directory.
3338

3439

@@ -41,7 +46,7 @@ Binary integer programs: place :math:`n` non-attacking queens on an
4146
:math:`2(2n-3)` at-most-one diagonal constraints. The :math:`n=1200`
4247
instance has 1,440,000 binary variables.
4348

44-
Model creation times in seconds (CPython 3.14.4):
49+
Model creation times in seconds — **CPython 3.14.4**:
4550

4651
.. list-table::
4752
:header-rows: 1
@@ -91,10 +96,50 @@ Model creation times in seconds (CPython 3.14.4):
9196
- >8s
9297
- 0.826
9398

94-
Python-MIP with any backend is **10–12× faster** than the highspy high-level
95-
API for model creation (highspy-hl times out above n=400 with an 8 s build
96-
limit), and within a factor of 6–7 of the highspy batch numpy API which
97-
requires the user to pre-build a full CSR matrix.
99+
Model creation times in seconds — **PyPy 3.11 (7.3.20)**:
100+
101+
.. list-table::
102+
:header-rows: 1
103+
:align: center
104+
:widths: 10 16 16 16
105+
106+
* - :math:`n`
107+
- python-mip / CBC
108+
- python-mip / HiGHS
109+
- python-mip / Gurobi
110+
* - 200
111+
- 0.061
112+
- **0.053**
113+
- 0.050
114+
* - 400
115+
- **0.141**
116+
- 0.153
117+
- 0.189
118+
* - 600
119+
- **0.274**
120+
- 0.290
121+
- 0.371
122+
* - 800
123+
- **0.471**
124+
- 0.463
125+
- 0.684
126+
* - 1000
127+
- **0.774**
128+
- 0.862
129+
- 1.179
130+
* - 1200
131+
- **1.100**
132+
- 1.117
133+
- 1.620
134+
135+
PyPy delivers a **4–5× speedup** over CPython for python-mip model building.
136+
The highspy batch numpy API is slower under PyPy (numpy operations are not
137+
JIT-compiled by PyPy) and is omitted from the PyPy table.
138+
139+
Python-MIP (CPython) with any backend is **10–12× faster** than the highspy
140+
high-level API (which times out above n=400 with an 8 s build limit), and
141+
within a factor of 6–7 of the highspy batch numpy API which requires the user
142+
to pre-build a full CSR matrix.
98143

99144
Run: ``python benchmarks/queens_bench.py --build-only``
100145

@@ -124,7 +169,7 @@ constraints together eliminate all subtours.
124169
For :math:`n` cities the model has :math:`2n(n-1)` variables and
125170
:math:`2n + n(n-1) + (n-1)` constraints.
126171

127-
Model creation times in seconds (CPython 3.14.4):
172+
Model creation times in seconds — **CPython 3.14.4**:
128173

129174
.. list-table::
130175
:header-rows: 1
@@ -192,14 +237,58 @@ Model creation times in seconds (CPython 3.14.4):
192237
- >8s
193238
- 0.503
194239

240+
Model creation times in seconds — **PyPy 3.11 (7.3.20)**:
241+
242+
.. list-table::
243+
:header-rows: 1
244+
:align: center
245+
:widths: 10 16 16 16
246+
247+
* - :math:`n`
248+
- python-mip / CBC
249+
- python-mip / HiGHS
250+
- python-mip / Gurobi
251+
* - 75
252+
- **0.047**
253+
- 0.030
254+
- 0.020
255+
* - 100
256+
- **0.028**
257+
- 0.031
258+
- 0.062
259+
* - 150
260+
- **0.101**
261+
- 0.116
262+
- 0.110
263+
* - 200
264+
- **0.159**
265+
- 0.193
266+
- 0.195
267+
* - 300
268+
- **0.383**
269+
- 0.433
270+
- 0.469
271+
* - 400
272+
- **0.634**
273+
- 0.814
274+
- 0.821
275+
* - 500
276+
- **1.263**
277+
- 1.224
278+
- 1.345
279+
280+
PyPy delivers a **3–4× speedup** over CPython for the TSP flow model. At
281+
small sizes (n ≤ 50) JIT warm-up may exceed CPython; the benefit is clear
282+
from n=75 onwards.
283+
195284
The TSP flow model interleaves binary and continuous variables with
196285
variable-density rows (degree rows touch :math:`n-1` variables; capacity rows
197286
touch 2; flow-conservation rows touch :math:`2(n-1)`). Python-MIP's cache
198287
handles this automatically — no manual CSR construction required.
199-
Python-MIP is roughly **3–5× faster** than the highspy high-level API (which
200-
times out above n=300 with an 8 s build limit) and within **7–8×** of the
201-
highspy batch numpy API that requires the caller to pre-build the full CSR
202-
matrix.
288+
Python-MIP (CPython) is roughly **3–5× faster** than the highspy high-level
289+
API (which times out above n=300 with an 8 s build limit) and within **7–8×**
290+
of the highspy batch numpy API that requires the caller to pre-build the full
291+
CSR matrix.
203292

204293
To verify correctness and solve a small instance:
205294
``python benchmarks/tsp_flow_bench.py --verify``

0 commit comments

Comments
 (0)