|
| 1 | +--- |
| 2 | +id: openrust-arithmetic |
| 3 | +title: Single and double precision |
| 4 | +description: OpEn with f32 and f64 number types |
| 5 | +--- |
| 6 | + |
| 7 | +:::note Info |
| 8 | +The functionality presented here was introduced in OpEn version [`0.12.0`](https://pypi.org/project/opengen/#history). |
| 9 | +The new API is fully backward-compatible with previous versions of OpEn. |
| 10 | +::: |
| 11 | + |
| 12 | +## Overview |
| 13 | + |
| 14 | +OpEn's Rust API supports both `f64` and `f32`. |
| 15 | + |
| 16 | +Most public Rust types are generic over a scalar type `T` with `T: num::Float`, and in most places the default type is `f64`. This means: |
| 17 | + |
| 18 | +- if you do nothing special, you will usually get `f64` |
| 19 | +- if you want single precision, you can explicitly use `f32` |
| 20 | +- all quantities involved in one solver instance should use the same scalar type |
| 21 | + |
| 22 | +In particular, this applies to: |
| 23 | + |
| 24 | +- cost and gradient functions |
| 25 | +- constraints |
| 26 | +- `Problem` |
| 27 | +- caches such as `PANOCCache`, `FBSCache`, and `AlmCache` |
| 28 | +- optimizers such as `PANOCOptimizer`, `FBSOptimizer`, and `AlmOptimizer` |
| 29 | +- solver status types such as `SolverStatus<T>` and `AlmOptimizerStatus<T>` |
| 30 | + |
| 31 | +## When to use `f64` and when to use `f32` |
| 32 | + |
| 33 | +### `f64` |
| 34 | + |
| 35 | +Use `f64` when you want maximum numerical robustness and accuracy. This is the safest default for: |
| 36 | + |
| 37 | +- desktop applications |
| 38 | +- difficult nonlinear problems |
| 39 | +- problems with tight tolerances |
| 40 | +- problems that are sensitive to conditioning |
| 41 | + |
| 42 | +### `f32` |
| 43 | + |
| 44 | +Use `f32` when memory footprint and throughput matter more than ultimate accuracy. This is often useful for: |
| 45 | + |
| 46 | +- embedded applications |
| 47 | +- high-rate MPC loops |
| 48 | +- applications where moderate tolerances are acceptable |
| 49 | + |
| 50 | +In general, `f32` may require: |
| 51 | + |
| 52 | +- slightly looser tolerances |
| 53 | +- more careful scaling of the problem |
| 54 | +- fewer expectations about extremely small residuals |
| 55 | + |
| 56 | +## The default: `f64` |
| 57 | + |
| 58 | +If your functions, constants, and vectors use `f64`, you can often omit the scalar type completely. |
| 59 | + |
| 60 | +```rust |
| 61 | +use optimization_engine::{constraints, panoc::PANOCCache, Problem, SolverError}; |
| 62 | +use optimization_engine::panoc::PANOCOptimizer; |
| 63 | + |
| 64 | +let tolerance = 1e-6; |
| 65 | +let lbfgs_memory = 10; |
| 66 | +let radius = 1.0; |
| 67 | + |
| 68 | +let bounds = constraints::Ball2::new(None, radius); |
| 69 | + |
| 70 | +let df = |u: &[f64], grad: &mut [f64]| -> Result<(), SolverError> { |
| 71 | + grad[0] = u[0] + u[1] + 1.0; |
| 72 | + grad[1] = u[0] + 2.0 * u[1] - 1.0; |
| 73 | + Ok(()) |
| 74 | +}; |
| 75 | + |
| 76 | +let f = |u: &[f64], cost: &mut f64| -> Result<(), SolverError> { |
| 77 | + *cost = 0.5 * (u[0] * u[0] + u[1] * u[1]); |
| 78 | + Ok(()) |
| 79 | +}; |
| 80 | + |
| 81 | +let problem = Problem::new(&bounds, df, f); |
| 82 | +let mut cache = PANOCCache::new(2, tolerance, lbfgs_memory); |
| 83 | +let mut optimizer = PANOCOptimizer::new(problem, &mut cache); |
| 84 | + |
| 85 | +let mut u = [0.0, 0.0]; |
| 86 | +let status = optimizer.solve(&mut u).unwrap(); |
| 87 | +assert!(status.has_converged()); |
| 88 | +``` |
| 89 | + |
| 90 | +Because all literals and function signatures above are `f64`, the compiler infers `T = f64`. |
| 91 | + |
| 92 | +## Using `f32` |
| 93 | + |
| 94 | +To use single precision, make the scalar type explicit throughout the problem definition. |
| 95 | + |
| 96 | +```rust |
| 97 | +use optimization_engine::{constraints, panoc::PANOCCache, Problem, SolverError}; |
| 98 | +use optimization_engine::panoc::PANOCOptimizer; |
| 99 | + |
| 100 | +let tolerance = 1e-4_f32; |
| 101 | +let lbfgs_memory = 10; |
| 102 | +let radius = 1.0_f32; |
| 103 | + |
| 104 | +let bounds = constraints::Ball2::new(None, radius); |
| 105 | + |
| 106 | +let df = |u: &[f32], grad: &mut [f32]| -> Result<(), SolverError> { |
| 107 | + grad[0] = u[0] + u[1] + 1.0_f32; |
| 108 | + grad[1] = u[0] + 2.0_f32 * u[1] - 1.0_f32; |
| 109 | + Ok(()) |
| 110 | +}; |
| 111 | + |
| 112 | +let f = |u: &[f32], cost: &mut f32| -> Result<(), SolverError> { |
| 113 | + *cost = 0.5_f32 * (u[0] * u[0] + u[1] * u[1]); |
| 114 | + Ok(()) |
| 115 | +}; |
| 116 | + |
| 117 | +let problem = Problem::new(&bounds, df, f); |
| 118 | +let mut cache = PANOCCache::<f32>::new(2, tolerance, lbfgs_memory); |
| 119 | +let mut optimizer = PANOCOptimizer::new(problem, &mut cache); |
| 120 | + |
| 121 | +let mut u = [0.0_f32, 0.0_f32]; |
| 122 | +let status = optimizer.solve(&mut u).unwrap(); |
| 123 | +assert!(status.has_converged()); |
| 124 | +``` |
| 125 | + |
| 126 | +The key idea is that the same scalar type must be used consistently in: |
| 127 | + |
| 128 | +- the initial guess `u` |
| 129 | +- the closures for the cost and gradient |
| 130 | +- the constraints |
| 131 | +- the cache |
| 132 | +- any tolerances and numerical constants |
| 133 | + |
| 134 | +## Example with FBS |
| 135 | + |
| 136 | +The same pattern applies to other solvers. |
| 137 | + |
| 138 | +```rust |
| 139 | +use optimization_engine::{constraints, Problem, SolverError}; |
| 140 | +use optimization_engine::fbs::{FBSCache, FBSOptimizer}; |
| 141 | +use std::num::NonZeroUsize; |
| 142 | + |
| 143 | +let bounds = constraints::Ball2::new(None, 0.2_f32); |
| 144 | + |
| 145 | +let df = |u: &[f32], grad: &mut [f32]| -> Result<(), SolverError> { |
| 146 | + grad[0] = u[0] + u[1] + 1.0_f32; |
| 147 | + grad[1] = u[0] + 2.0_f32 * u[1] - 1.0_f32; |
| 148 | + Ok(()) |
| 149 | +}; |
| 150 | + |
| 151 | +let f = |u: &[f32], cost: &mut f32| -> Result<(), SolverError> { |
| 152 | + *cost = u[0] * u[0] + 2.0_f32 * u[1] * u[1] + u[0] - u[1] + 3.0_f32; |
| 153 | + Ok(()) |
| 154 | +}; |
| 155 | + |
| 156 | +let problem = Problem::new(&bounds, df, f); |
| 157 | +let mut cache = FBSCache::<f32>::new(NonZeroUsize::new(2).unwrap(), 0.1_f32, 1e-6_f32); |
| 158 | +let mut optimizer = FBSOptimizer::new(problem, &mut cache); |
| 159 | + |
| 160 | +let mut u = [0.0_f32, 0.0_f32]; |
| 161 | +let status = optimizer.solve(&mut u).unwrap(); |
| 162 | +assert!(status.has_converged()); |
| 163 | +``` |
| 164 | + |
| 165 | +## Example with ALM |
| 166 | + |
| 167 | +ALM also supports both precisions. As with PANOC and FBS, the scalar type should be chosen once and then used consistently throughout the ALM problem, cache, mappings, and tolerances. |
| 168 | + |
| 169 | +For example, if you use: |
| 170 | + |
| 171 | +- `AlmCache::<f32>` |
| 172 | +- `PANOCCache::<f32>` |
| 173 | +- `Ball2::<f32>` |
| 174 | +- closures of type `|u: &[f32], ...|` |
| 175 | + |
| 176 | +then the whole ALM solve runs in single precision. |
| 177 | + |
| 178 | +If instead you use plain `f64` literals and `&[f64]` closures, the solver runs in double precision. |
| 179 | + |
| 180 | +## Type inference tips |
| 181 | + |
| 182 | +Rust usually infers the scalar type correctly, but explicit annotations are often helpful for `f32`. |
| 183 | + |
| 184 | +Good ways to make `f32` intent clear are: |
| 185 | + |
| 186 | +- suffix literals, for example `1.0_f32` and `1e-4_f32` |
| 187 | +- annotate vectors and arrays, for example `let mut u = [0.0_f32; 2];` |
| 188 | +- annotate caches explicitly, for example `PANOCCache::<f32>::new(...)` |
| 189 | +- annotate closure arguments, for example `|u: &[f32], grad: &mut [f32]|` |
| 190 | + |
| 191 | +## Important rule: do not mix `f32` and `f64` |
| 192 | + |
| 193 | +The following combinations are problematic: |
| 194 | + |
| 195 | +- `u: &[f32]` with a cost function writing to `&mut f64` |
| 196 | +- `Ball2::new(None, 1.0_f64)` together with `PANOCCache::<f32>` |
| 197 | +- `tolerance = 1e-6` in one place and `1e-6_f32` elsewhere if inference becomes ambiguous |
| 198 | + |
| 199 | +Choose one scalar type per optimization problem and use it everywhere. |
| 200 | + |
| 201 | +## Choosing tolerances |
| 202 | + |
| 203 | +When moving from `f64` to `f32`, it is often a good idea to relax tolerances. |
| 204 | + |
| 205 | +Typical starting points are: |
| 206 | + |
| 207 | +- `f64`: `1e-6`, `1e-8`, or smaller if needed |
| 208 | +- `f32`: `1e-4` or `1e-5` |
| 209 | + |
| 210 | +The right choice depends on: |
| 211 | + |
| 212 | +- scaling of the problem |
| 213 | +- conditioning |
| 214 | +- solver settings |
| 215 | +- whether the problem is solved repeatedly in real time |
0 commit comments