feat: Add Google OR-Tools CP-SAT solver#126
Conversation
…warning to panic in CP-SAT bridge; Renamed clear hint method
TeXitoi
left a comment
There was a problem hiding this comment.
This "integer only model" makes cp_sat difficult to use as a LP solver. Ideally, the weights would be scaled, but that’s not really doable without a global analysis of the LP.
|
|
||
| // Translate bounds to CP-SAT domain | ||
| let domain_lower = if def.min.is_finite() { | ||
| round_i64(def.min) |
There was a problem hiding this comment.
rounding is not correct. lower must be ceiled, and upper must be floored.
| fn expr_from_constraint_expression(&self, expression: &crate::Expression) -> LinearExpr { | ||
| let mut linear = LinearExpr::default(); | ||
| for (var, coeff) in expression.linear.coefficients.iter() { | ||
| let coeff_i64 = round_i64(*coeff); |
There was a problem hiding this comment.
I’m afraid these rounding will be very error prone for the users.
The CP-SAT primer by Dominik Krupke highlights this about fractional (continuous values):
This shows that automatic scaling can not reliably cover all buliding blocks that good_lp offers when using CP-SAT. Personally, I would be in favor of leaving this to the user. |
|
Maybe we could add a warning whenever a fractional value is passed to the rounding method, informing the user that their value will be rounded. Even considering the limitations, I think it would be highly beneficial to support CP-SAT in good_lp. It is very performant, and there are many combinatorial problem classes that do not necessarily involve continuous variables (covering, routing, knapsack packing, vehicle routing, etc.). Having CP-SAT alongside the LP and MIP solvers would also allow combining LP for relaxations and switching to CP-SAT for the integer solving phase. |
|
Note that my remark is not limited to linear variables, but also fractional weights in the objective and constraint. If the user says |
lovasoa
left a comment
There was a problem hiding this comment.
thanks ! that's a good addition.
See my suggestions below.
I haven't checked the quality of the underlying crate itself, I trust you with it.
| env: | ||
| ORTOOLS_PREFIX: /opt/ortools | ||
| RUSTFLAGS: -L /opt/ortools/lib -lprotobuf | ||
| LD_LIBRARY_PATH: /opt/ortools/lib |
There was a problem hiding this comment.
can we avoid special-casing the new solver like that ? Use the standard system library path, standard build.rs in the underlying crate, etc. Ideally you would add an option to statically link the c++ deps in the underlying crate and use it here. The goal of good-lp is to be as user-friendly and zero-config as possible.
There was a problem hiding this comment.
Use the standard system library path, standard build.rs in the underlying crate, etc. Ideally you would add an option to statically link the c++ deps in the underlying crate and use it here. The goal of good-lp is to be as user-friendly and zero-config as possible
Two problems here:
- The ortools C++ library is quite heavy and it has a lot of dependencies: Abseil, Protobuf, zlib, pthreads, other solvers again, ... I have not looked into whether that is possible at all yet, but my instinct says: NO. It also takes a non-trivial time to build from source, in the order of a couple of minutes even on a 24 thread cpu. So I think most users will just install this library as a binary release.
- The underlying crate assumes the ortools library to lie at "/opt/ortools", so that is actually the default and not the standard system library path. Another location can be provided by setting ORTOOLS_PREFIX. I agree that we should simplify this. And this is actually one of the reasons why I am working on my own ortools bridge, which will not only provide an interface to CP-SAT but also to some of their other optimizers, and handle linking better via CMake.
| assert!( | ||
| def.is_integer, | ||
| "CP-SAT does not support continuous variables. \ | ||
| Variable {} (index {}) was not declared as integer or binary. \ | ||
| Please use `.integer()` or `.binary()` on the variable definition.", | ||
| if def.name.is_empty() { | ||
| format!("v{}", var.index()) | ||
| } else { | ||
| def.name.clone() | ||
| }, | ||
| var.index() | ||
| ); |
There was a problem hiding this comment.
can we avoid crashing the entire program here ? Just let the model build and return an error on solve
There was a problem hiding this comment.
This confuses me because I found analogous behavior in the clarabel solver interface:
/// The [clarabel](https://oxfordcontrol.github.io/ClarabelDocs/stable/) solver,
/// to be used with [UnsolvedProblem::using].
pub fn clarabel(to_solve: UnsolvedProblem) -> ClarabelProblem {
// [...]
for (var, def) in variables.iter_variables_with_def() {
if def.is_integer {
panic!("Clarabel doesn't support integer variables")
}
// [...]
}
p
}There was a problem hiding this comment.
yeah, clarabel should not do that either ! pr welcome !
| /// Panics on NaN. | ||
| fn round_i64(x: f64) -> i64 { | ||
| if x.is_nan() { | ||
| panic!("CP-SAT received NaN"); |
There was a problem hiding this comment.
please do not crash the user's program. Handle errors cleanly
| // Validate: lower bound must not exceed upper bound | ||
| assert!( | ||
| domain_lower <= domain_upper, | ||
| "Invalid variable bounds: lower={} > upper={} for variable '{}'", | ||
| def.min, | ||
| def.max, | ||
| name | ||
| ); |
| self.params.relative_gap_limit.is_some() || self.params.absolute_gap_limit.is_some(); | ||
|
|
||
| let status = response.status(); | ||
| let status_from_limits = || { |
There was a problem hiding this comment.
why does this need to be a function?
| Then set the following environment variables when building and running: | ||
|
|
||
| ```bash | ||
| export ORTOOLS_PREFIX=/opt/ortools | ||
| export RUSTFLAGS="-L /opt/ortools/lib -lprotobuf" | ||
| export LD_LIBRARY_PATH=/opt/ortools/lib | ||
| ``` |
There was a problem hiding this comment.
that's non standard, and painful for users
| curl -LO 'https://github.com/google/or-tools/releases/download/v9.15/or-tools_amd64_ubuntu-24.04_cpp_v9.15.6755.tar.gz' | ||
| sudo mkdir -p /opt/ortools | ||
| sudo tar -xzf or-tools_amd64_ubuntu-24.04_cpp_v9.15.6755.tar.gz -C /opt/ortools --strip-components=1 | ||
| cd /opt/ortools && sudo make test |
There was a problem hiding this comment.
Maybe we should point to upstream instructions
| singlethread-cbc = ["coin_cbc?/singlethread-cbc"] | ||
| scip = ["russcip"] | ||
| scip_bundled = ["russcip?/bundled"] | ||
| cp_sat = ["dep:cp_sat"] |
There was a problem hiding this comment.
I looked into this and as far as I understand, it's not strictly necessary. Cargo automatically creates a feature with the same name as any optional dependency, so the cp_sat feature would exist even without the explicit cp_sat = ["dep:cp_sat"] line.
I think I added this because I tried adding cp_sat = ["cp_sat"] before and that failed, so I looked for solutions. But can probably be dropped. Only benefit is the dep: prefix makes the relationship explicit rather than relying on the implicit auto-generated feature.
| //! # Solvers | ||
| //! | ||
| //! The following solvers are available: | ||
| //! | ||
| //! - [`coin_cbc`](solvers::coin_cbc) - [COIN-OR CBC](https://www.coin-or.org/Cbc/) (enabled by default) | ||
| //! - [`cp_sat`](solvers::cp_sat) - [Google OR-Tools CP-SAT](https://developers.google.com/optimization/cp/cp_solver) (requires `cp_sat` feature) | ||
| //! - [`microlp`](solvers::microlp) - A lightweight LP solver | ||
| //! - [`lpsolve`](solvers::lpsolve) - [LpSolve](http://lpsolve.sourceforge.net/) | ||
| //! - [`highs`](solvers::highs) - [HiGHS](https://highs.dev/) | ||
| //! - [`scip`](solvers::scip) - [SCIP](https://scipopt.org/) | ||
| //! - [`lp-solvers`](solvers::lp_solvers) - A wrapper around several LP solvers | ||
| //! - [`clarabel`](solvers::clarabel) - [Clarabel](https://github.com/oxfordcontrol/Clarabel.rs) | ||
| //! |
There was a problem hiding this comment.
we can inline the readme if you want. But let's not maintain two lists
|
@gabriel-gehrke : also from the test logs, I see it prints to stdout. Can we avoid that ? |
|
For non integer weights : can we just reject them with a clean error ? And document that solver accepts only integers. Let users do the rounding on their side |
I implemented a configurable verbosity flag similar to how Highs can print to stdout (so CP-SAT just does that, for example for debugging). It can also that provide the log after terminating. In one of the tests it basically tests that this flag can be set. I can remove that test, as it is more like a "this does not cause an error" thing, and not checking whether it actually prints to stdout. |
|
Thank you for the detailed feedback, I will make sure to fix these issues. |
Oh ok no problem then, you can leave that part as is. Sorry! |
…amed variable creation only optional
|
No contribution on my end here, but this would be a really cool backend to have for what I'm working on. cp-sat is a speedy solver for the problems I used it on. |
|
So I've been thinking about the whole approach of 'not crashing the user's program even if inputs are invalid'. Due to the trait signatures in this crate (notably My proposal:
About variable/constraint handles:
I would appreciate feedback on this approach |
|
@gabriel-gehrke : yes, I think that's the way it should work. A Valid(InnerModel)/Invalid enum, all methods implemented on InnerModel, the top level implementation just forwards calls to the inner struct or does nothing. |
| if !x.is_finite() { | ||
| return Err(ResolutionError::Str(format!( | ||
| "The CP-SAT solver does not accept infinite values for coefficients, constants, \ | ||
| or variable bounds. Received `{x}`, which is not finite." | ||
| ))); | ||
| } | ||
| if x.fract() != 0.0 { | ||
| return Err(ResolutionError::Str(format!( | ||
| "The CP-SAT solver only accepts integer values for coefficients, constants, \ | ||
| or variable bounds. Received `{x}`, which is not an integer. \ | ||
| Please round or scale your values to integers." | ||
| ))); | ||
| } | ||
| if !(x >= -(2f64.powi(63)) && x < 2f64.powi(63)) { | ||
| return Err(ResolutionError::Str(format!( | ||
| "The CP-SAT solver value `{x}` is out of the i64 range." | ||
| ))); | ||
| } | ||
| Ok(x as i64) |
There was a problem hiding this comment.
we could use try_from, no ?
| [CP-SAT](https://developers.google.com/optimization/cp/cp_solver) is Google's fast constraint programming solver, | ||
| part of [OR-Tools](https://developers.google.com/optimization). It is one of the fastest open-source solvers | ||
| for combinatorial (integer) optimization problems, supporting both integer and boolean variables. | ||
| CP-SAT does **not** support continuous (floating-point) variables; using a non-integer variable will cause a panic. |
|
I opened a PR on the cp_sat crate. That PR fixes the dependency on 2 of the 3 environment variables my setup currently depends on. I do not have high hopes for it to get merged, unfortunately, as the repository has multiple open pull requests without any reaction from the maintainer(s). |
|
@TeXitoi , maybe you could have a look ? |
This PR adds a bridge to the Google OR-Tools CP-SAT solver via the
cp_satRust crate (v0.4.0), accessible through thecp_satfeature flag.CP-SAT is one of the fastest open-source solvers for combinatorial (integer) optimization problems. It supports integer and boolean variables, linear objectives, linear constraints, and advanced features like solution hints (warm-starting), time limits, MIP gaps, parallel search with different search strategies, and verbose logging.
Implementation
The bridge lives in
src/solvers/cp_sat.rsand covers:cp_sat()function: ConvertsUnsolvedProblemtoCpSatProblemSolverModelimpl:add_constraint()with equality/inequality support,solve()with full status mappingWithInitialSolutionimpl: Warm-start hints viaCpModelBuilder::add_hint()WithTimeLimitimpl: Setmax_time_in_secondson internal parametersWithMipGapimpl: Setrelative_gap_limiton internal parametersset_verbose(),set_log_to_response(),set_num_search_workers(),set_random_seed(), ...)CpSatSolutionwrapsCpSolverResponsewith access toresponse_stats(),solution_log(),additional_solutions(),into_response()Limitations & Rough Edges
Unlike most other good_lp solvers, the
cp_satcrate links dynamically to a very heavy OR-Tools shared library. This is handled by the cp_sat crate but it does so a little clumsily, relying on correct setting of environment variables (Note I am already working on a better ortools to rust interface, which could replace this crate in the future). Currently, users must:ORTOOLS_PREFIX,RUSTFLAGS="-L /opt/ortools/lib -lprotobuf",LD_LIBRARY_PATH=/opt/ortools/libAs the OR-Tools prebuilt binary is platform-specific, I currently pin it to
ubuntu-24.04in CI. The CI job runs only cp_sat-specific lib tests (cargo test --lib -- cp_sat), not integration tests which use continuous variables and would panic.For now I did not put it into
all_default_solversdue to the heavy external dependency (OR-Tools shared libraries), cp_sat is opt-in only, similar tocplex-rs.Progress on Fixes
solve().dep:cp_satsrc/lib.rs