Commit b7aba5f
feat: add sos reformulations into linopy to simplify adoption of new sos features (#549)
* The SOS constraint reformulation feature has been implemented successfully. Here's a summary:
Implementation Summary
New File: linopy/sos_reformulation.py
Core reformulation functions:
- validate_bounds_for_reformulation() - Validates that variables have finite bounds
- compute_big_m_values() - Computes Big-M values from variable bounds
- reformulate_sos1() - Reformulates SOS1 constraints using binary indicators and Big-M constraints
- reformulate_sos2() - Reformulates SOS2 constraints using segment indicators and adjacency constraints
- reformulate_all_sos() - Reformulates all SOS constraints in a model
Modified: linopy/model.py
- Added import for reformulate_all_sos
- Added reformulate_sos_constraints() method to Model class
- Added reformulate_sos: bool = False parameter to solve() method
- Updated SOS constraint check to automatically reformulate when reformulate_sos=True and solver doesn't support SOS natively
New Test File: test/test_sos_reformulation.py
36 comprehensive tests covering:
- Bound validation (finite/infinite)
- Big-M computation
- SOS1 reformulation (basic, negative bounds, multi-dimensional)
- SOS2 reformulation (basic, trivial cases, adjacency)
- Integration with solve() and HiGHS
- Equivalence with native Gurobi SOS support
- Edge cases (zero bounds, multiple SOS, custom prefix)
Usage Example
m = linopy.Model()
x = m.add_variables(lower=0, upper=1, coords=[pd.Index([0, 1, 2], name='i')], name='x')
m.add_sos_constraints(x, sos_type=1, sos_dim='i')
m.add_objective(x.sum(), sense='max')
# Works with HiGHS (which doesn't support SOS natively)
m.solve(solver_name='highs', reformulate_sos=True)
* Documentation Summary
New Section: "SOS Reformulation for Unsupported Solvers"
Added a comprehensive section (~300 lines) covering:
1. Enabling Reformulation - Shows reformulate_sos=True parameter and manual reformulate_sos_constraints() method
2. Requirements - Explains finite bounds requirement for Big-M method
3. Mathematical Formulation - Clear LaTeX math for both:
- SOS1: Binary indicators y_i, upper/lower linking constraints, cardinality constraint
- SOS2: Segment indicators z_j, first/middle/last element constraints, cardinality constraint
4. Interpretation - Explains how the constraints work intuitively with examples
5. Auxiliary Variables and Constraints - Documents the naming convention (_sos_reform_ prefix)
6. Multi-dimensional Variables - Shows how broadcasting works
7. Edge Cases Table - Lists all handled edge cases (single-element, zero bounds, all-positive, etc.)
8. Performance Considerations - Trade-offs between native SOS and reformulation
9. Complete Example - Piecewise linear approximation of x² with HiGHS
10. API Reference - Added method signatures for:
- Model.add_sos_constraints()
- Model.remove_sos_constraints()
- Model.reformulate_sos_constraints()
- Variables.sos property
* Added Tests for Multi-dimensional SOS
Unit Tests
- test_sos2_multidimensional: Tests that SOS2 reformulation with multi-dimensional variables (i, j) correctly creates:
- Segment indicators z with shape (i: n-1, j: m)
- Cardinality constraint preserves the j dimension
Integration Tests
- test_multidimensional_sos2_with_highs: Solves a multi-dimensional SOS2 problem with HiGHS and verifies:
- Optimal objective value (4 total - two adjacent non-zeros per column)
- SOS2 constraint satisfied for each j: at most 2 non-zeros, and if 2, they're adjacent
Test Results
test_sos1_multidimensional PASSED
test_sos2_multidimensional PASSED
test_multidimensional_sos1_with_highs PASSED
test_multidimensional_sos2_with_highs PASSED
The implementation correctly handles multi-dimensional variables by leveraging xarray's broadcasting - the SOS constraint is applied along the sos_dim for each combination of
the other dimensions.
* Add custom big_m parameter for SOS reformulation
Allow users to specify custom Big-M values in add_sos_constraints() for
tighter LP relaxations when variable bounds are conservative.
- Add big_m parameter: scalar or tuple(upper, lower)
- Store as variable attrs (big_m_upper, big_m_lower)
- Skip bound validation when custom big_m provided
- Scalar-only design ensures NetCDF persistence works correctly
For per-element Big-M values, users should adjust variable bounds directly.
* Add custom big_m parameter for SOS reformulation
Allow users to specify custom Big-M values in add_sos_constraints() for
tighter LP relaxations when variable bounds are conservative.
- Add big_m parameter: scalar or tuple(upper, lower)
- Store as variable attrs (big_m_upper, big_m_lower) for NetCDF persistence
- Use tighter of big_m and variable bounds: min() for upper, max() for lower
- Skip bound validation when custom big_m provided (allows infinite bounds)
Scalar-only design ensures NetCDF persistence works correctly. For
per-element Big-M values, users should adjust variable bounds directly.
* Simplification summary:
┌──────────────────────┬───────────┬───────────┬───────────┐
│ File │ Before │ After │ Reduction │
├──────────────────────┼───────────┼───────────┼───────────┤
│ sos_reformulation.py │ 377 lines │ 223 lines │ 41% │
├──────────────────────┼───────────┼───────────┼───────────┤
│ sos-constraints.rst │ 647 lines │ 164 lines │ 75% │
└──────────────────────┴───────────┴───────────┴───────────┘
Code changes:
- Merged validate_bounds_for_reformulation into compute_big_m_values
- Factored out add_linking_constraints helper in SOS2
- Used np.minimum/np.maximum instead of xr.where
- Kept proper docstrings with Parameters/Returns sections
Doc changes:
- Removed: Variable Representation, LP File Export, Common Patterns, Performance Considerations
- Trimmed: Examples to one each, Mathematical formulation to equations only
- Condensed: API reference, multi-dimensional explanation
* Revert some docs changes to be more surgical
* Add math to docs
* Improve docs
* Code simplifications:
1. sos_reformulation.py (230 → 203 lines):
- compute_big_m_values now returns single DataArray (not tuple)
- Removed all lower bound handling - only supports non-negative variables
- Removed add_linking_constraints helper function
- Simplified SOS1/SOS2 to only add upper linking constraints
2. model.py:
- Simplified big_m parameter from float | tuple[float, float] | None to float | None
- Removed big_m_lower attribute handling
3. Documentation (sos-constraints.rst):
- Updated big_m type signature
- Removed asymmetric Big-M example
- Added explicit requirement that variables must have non-negative lower bounds
4. Tests (46 → 38 tests):
- Removed tests for negative bounds
- Removed tests for tuple big_m
- Added tests for negative lower bound validation error
Rationale: The mathematical formulation in the docs assumes x ∈ ℝⁿ₊ (non-negative reals). This matches 99%+ of SOS use cases (selection indicators, piecewise linear weights).
The simplified code is now consistent with the documented formulation.
* Fix mypy
* Fix mypy
* Add constants for sos attr keys
* Add release notes
* Fix SOS reformulation: undo after solve, validate big_m, vectorize
- solve() now undoes SOS reformulation after solving, preserving model state
- Validate big_m > 0 in add_sos_constraints (fail fast)
- Vectorize SOS2 middle constraints, eliminate duplicate compute_big_m_values
- Warn when reformulate_sos=True is ignored for SOS-capable solvers
- Add tests for model immutability, double solve, big_m validation, undo
* tiny refac, plus uncovered test
* refac: move reformulating function to module
* Fix SOS reformulation: rollback, skipped attrs, undo in solve, sort coords
- Remove SOS attrs for skipped variables (size<=1, M==0) so solvers
don't see them as SOS constraints
- Wrap reformulation loop in try/except for transactional rollback
- Move undo into finally block in Model.solve() for exception safety
- Sort variables by coord values before building adjacency constraints
to match native SOS weight-based ordering
* update release notes [skip ci]
---------
Co-authored-by: Fabian Hofmann <fab.hof@gmx.de>1 parent 1b08d2b commit b7aba5f
File tree
8 files changed
+1351
-62
lines changed- doc
- linopy
- test
8 files changed
+1351
-62
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
7 | 10 | | |
8 | 11 | | |
9 | 12 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
78 | | - | |
| 78 | + | |
79 | 79 | | |
80 | 80 | | |
81 | 81 | | |
| |||
85 | 85 | | |
86 | 86 | | |
87 | 87 | | |
| 88 | + | |
| 89 | + | |
88 | 90 | | |
89 | 91 | | |
90 | 92 | | |
| |||
254 | 256 | | |
255 | 257 | | |
256 | 258 | | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
257 | 336 | | |
258 | 337 | | |
259 | 338 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
52 | 57 | | |
53 | 58 | | |
54 | 59 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
| 28 | + | |
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| |||
371 | 371 | | |
372 | 372 | | |
373 | 373 | | |
374 | | - | |
375 | | - | |
| 374 | + | |
| 375 | + | |
376 | 376 | | |
377 | 377 | | |
378 | 378 | | |
| |||
740 | 740 | | |
741 | 741 | | |
742 | 742 | | |
743 | | - | |
744 | | - | |
| 743 | + | |
| 744 | + | |
745 | 745 | | |
746 | 746 | | |
747 | 747 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
42 | 45 | | |
43 | 46 | | |
44 | 47 | | |
| |||
66 | 69 | | |
67 | 70 | | |
68 | 71 | | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
69 | 76 | | |
70 | 77 | | |
71 | 78 | | |
| |||
591 | 598 | | |
592 | 599 | | |
593 | 600 | | |
| 601 | + | |
594 | 602 | | |
595 | 603 | | |
596 | 604 | | |
| |||
604 | 612 | | |
605 | 613 | | |
606 | 614 | | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
607 | 626 | | |
608 | 627 | | |
609 | 628 | | |
610 | 629 | | |
611 | 630 | | |
612 | 631 | | |
613 | | - | |
614 | | - | |
615 | | - | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
616 | 635 | | |
617 | 636 | | |
618 | 637 | | |
| |||
624 | 643 | | |
625 | 644 | | |
626 | 645 | | |
627 | | - | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
628 | 653 | | |
629 | 654 | | |
630 | 655 | | |
| |||
891 | 916 | | |
892 | 917 | | |
893 | 918 | | |
894 | | - | |
| 919 | + | |
895 | 920 | | |
896 | 921 | | |
897 | | - | |
898 | | - | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
899 | 926 | | |
900 | | - | |
| 927 | + | |
901 | 928 | | |
902 | 929 | | |
903 | 930 | | |
904 | 931 | | |
905 | 932 | | |
| 933 | + | |
| 934 | + | |
906 | 935 | | |
907 | 936 | | |
908 | 937 | | |
| |||
1187 | 1216 | | |
1188 | 1217 | | |
1189 | 1218 | | |
| 1219 | + | |
1190 | 1220 | | |
1191 | 1221 | | |
1192 | 1222 | | |
| |||
1256 | 1286 | | |
1257 | 1287 | | |
1258 | 1288 | | |
| 1289 | + | |
| 1290 | + | |
| 1291 | + | |
| 1292 | + | |
| 1293 | + | |
1259 | 1294 | | |
1260 | 1295 | | |
1261 | 1296 | | |
| |||
1353 | 1388 | | |
1354 | 1389 | | |
1355 | 1390 | | |
1356 | | - | |
1357 | | - | |
1358 | | - | |
1359 | | - | |
1360 | | - | |
| 1391 | + | |
| 1392 | + | |
| 1393 | + | |
| 1394 | + | |
| 1395 | + | |
| 1396 | + | |
| 1397 | + | |
| 1398 | + | |
| 1399 | + | |
| 1400 | + | |
| 1401 | + | |
| 1402 | + | |
| 1403 | + | |
| 1404 | + | |
| 1405 | + | |
| 1406 | + | |
| 1407 | + | |
| 1408 | + | |
| 1409 | + | |
1361 | 1410 | | |
1362 | 1411 | | |
1363 | 1412 | | |
| |||
1406 | 1455 | | |
1407 | 1456 | | |
1408 | 1457 | | |
1409 | | - | |
1410 | | - | |
1411 | | - | |
1412 | | - | |
1413 | | - | |
1414 | | - | |
1415 | | - | |
1416 | | - | |
1417 | | - | |
1418 | | - | |
| 1458 | + | |
| 1459 | + | |
| 1460 | + | |
| 1461 | + | |
| 1462 | + | |
| 1463 | + | |
| 1464 | + | |
| 1465 | + | |
| 1466 | + | |
| 1467 | + | |
| 1468 | + | |
| 1469 | + | |
| 1470 | + | |
| 1471 | + | |
1419 | 1472 | | |
1420 | | - | |
1421 | | - | |
1422 | | - | |
1423 | | - | |
| 1473 | + | |
| 1474 | + | |
| 1475 | + | |
| 1476 | + | |
1424 | 1477 | | |
1425 | | - | |
1426 | | - | |
1427 | | - | |
1428 | | - | |
1429 | | - | |
1430 | | - | |
1431 | | - | |
1432 | | - | |
1433 | | - | |
1434 | | - | |
1435 | | - | |
1436 | | - | |
1437 | | - | |
1438 | | - | |
1439 | | - | |
| 1478 | + | |
| 1479 | + | |
1440 | 1480 | | |
1441 | | - | |
| 1481 | + | |
1442 | 1482 | | |
1443 | | - | |
1444 | | - | |
| 1483 | + | |
| 1484 | + | |
| 1485 | + | |
| 1486 | + | |
| 1487 | + | |
| 1488 | + | |
| 1489 | + | |
| 1490 | + | |
| 1491 | + | |
| 1492 | + | |
| 1493 | + | |
| 1494 | + | |
| 1495 | + | |
| 1496 | + | |
| 1497 | + | |
1445 | 1498 | | |
1446 | | - | |
| 1499 | + | |
| 1500 | + | |
| 1501 | + | |
| 1502 | + | |
1447 | 1503 | | |
1448 | 1504 | | |
1449 | 1505 | | |
| |||
0 commit comments