Skip to content

Commit 75ee805

Browse files
amontoisonamontoison
authored
Support dense Jacobians and Hessians (#537)
1 parent 01b4d9f commit 75ee805

6 files changed

Lines changed: 108 additions & 60 deletions

File tree

‎README.md‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ If Jacobian-vector products can be computed more efficiently than by evaluating
7070

7171
The following method is defined if second-order derivatives are available:
7272

73-
* `hess(model, x, y)`: evaluate *∇²L(x,y)*, the Hessian of the Lagrangian at `x` and `y`
73+
* `hess(model, x, y)`: evaluate *∇²L(x,y)*, the Hessian of the Lagrangian at `x` and `y` as a sparse matrix
7474

7575
If Hessian-vector products can be computed more efficiently than by evaluating the Hessian explicitly, the following method may be implemented:
7676

@@ -111,16 +111,18 @@ Attribute | Type | Notes
111111
`jfree` | `Vector{Int}` | indices of "free" constraints (there shouldn't be any)
112112
`jinf` | `Vector{Int}` | indices of the visibly infeasible constraints
113113
`nnzo` | `Int` | number of nonzeros in the gradient
114-
`nnzj` | `Int` | number of nonzeros in the sparse Jacobian
115-
`lin_nnzj` | `Int` | number of nonzeros in the sparse linear constraints Jacobian
116-
`nln_nnzj` | `Int` | number of nonzeros in the sparse nonlinear constraints Jacobian
117-
`nnzh` | `Int` | number of nonzeros in the lower triangular part of the sparse Hessian of the Lagrangian
114+
`nnzj` | `Int` | number of nonzeros in the Jacobian
115+
`lin_nnzj` | `Int` | number of nonzeros in the linear constraints Jacobian
116+
`nln_nnzj` | `Int` | number of nonzeros in the nonlinear constraints Jacobian
117+
`nnzh` | `Int` | number of nonzeros in the lower triangular part of the Hessian of the Lagrangian
118118
`minimize` | `Bool` | true if `optimize == minimize`
119119
`islp` | `Bool` | true if the problem is a linear program
120120
`name` | `String` | problem name
121+
`sparse_jacobian` | `Bool` | true if the Jacobian of the constraints is sparse
122+
`sparse_hessian` | `Bool` | true if the Hessian of the Lagrangian is sparse
121123
`grad_available` | `Bool` | true if the gradient of the objective is available
122-
`jac_available` | `Bool` | true if the sparse Jacobian of the constraints is available
123-
`hess_available` | `Bool` | true if the sparse Hessian of the Lagrangian is available
124+
`jac_available` | `Bool` | true if the Jacobian of the constraints is available
125+
`hess_available` | `Bool` | true if the Hessian of the Lagrangian is available
124126
`jprod_available` | `Bool` | true if the Jacobian-vector product `J * v` is available
125127
`jtprod_available` | `Bool` | true if the transpose Jacobian-vector product `J' * v` is available
126128
`hprod_available` | `Bool` | true if the Hessian-vector product of the Lagrangian `H * v` is available

‎docs/src/api.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ If not, click on the link and go to the description.
2929

3030
- `!` means inplace;
3131
- `_coord` means coordinate format;
32+
- `_dense` means dense format;
3233
- `prod` means matrix-vector product;
3334
- `_op` means operator (as in [LinearOperators.jl](https://github.com/JuliaSmoothOptimizers/LinearOperators.jl));
3435
- `_lin` and `_nln` respectively refer to linear and nonlinear constraints.
@@ -40,13 +41,14 @@ NLPModels instances.
4041
|-------------------|-------------------------------------------|
4142
| ``f(x)`` | [`obj`](@ref), [`objgrad`](@ref), [`objgrad!`](@ref), [`objcons`](@ref), [`objcons!`](@ref) |
4243
| ``\nabla f(x)`` | [`grad`](@ref), [`grad!`](@ref), [`objgrad`](@ref), [`objgrad!`](@ref) |
43-
| ``\nabla^2 f(x)`` | [`hess`](@ref), [`hess_op`](@ref), [`hess_op!`](@ref), [`hess_coord`](@ref), [`hess_coord`](@ref), [`hess_structure`](@ref), [`hess_structure!`](@ref), [`hprod`](@ref), [`hprod!`](@ref) |
44+
| ``\nabla^2 f(x)`` | [`hess`](@ref), [`hess_op`](@ref), [`hess_op!`](@ref), [`hess_coord`](@ref), [`hess_coord!`](@ref), [`hess_dense!`](@ref), [`hess_structure`](@ref), [`hess_structure!`](@ref), [`hprod`](@ref), [`hprod!`](@ref) |
4445
| ``c(x)`` | [`cons_lin`](@ref), [`cons_lin!`](@ref), [`cons_nln`](@ref), [`cons_nln!`](@ref), [`cons`](@ref), [`cons!`](@ref), [`objcons`](@ref), [`objcons!`](@ref) |
45-
| ``J(x)`` | [`jac_lin`](@ref), [`jac_nln`](@ref), [`jac`](@ref), [`jac_lin_op`](@ref), [`jac_lin_op!`](@ref), [`jac_nln_op`](@ref), [`jac_nln_op!`](@ref),[`jac_op`](@ref), [`jac_op!`](@ref), [`jac_lin_coord`](@ref), [`jac_lin_coord!`](@ref), [`jac_nln_coord`](@ref), [`jac_nln_coord!`](@ref), [`jac_coord`](@ref), [`jac_coord!`](@ref), [`jac_lin_structure`](@ref), [`jac_lin_structure!`](@ref), [`jac_nln_structure`](@ref), [`jac_nln_structure!`](@ref), [`jac_structure`](@ref), [`jprod_lin`](@ref), [`jprod_lin!`](@ref), [`jprod_nln`](@ref), [`jprod_nln!`](@ref), [`jprod`](@ref), [`jprod!`](@ref), [`jtprod_lin`](@ref), [`jtprod_lin!`](@ref), [`jtprod_nln`](@ref), [`jtprod_nln!`](@ref), [`jtprod`](@ref), [`jtprod!`](@ref) |
46-
| ``\nabla^2 L(x,y)`` | [`hess`](@ref), [`hess_op`](@ref), [`hess_coord`](@ref), [`hess_coord!`](@ref), [`hess_structure`](@ref), [`hess_structure!`](@ref), [`hprod`](@ref), [`hprod!`](@ref), [`jth_hprod`](@ref), [`jth_hprod!`](@ref), [`jth_hess`](@ref), [`jth_hess_coord`](@ref), [`jth_hess_coord!`](@ref), [`ghjvprod`](@ref), [`ghjvprod!`](@ref) |
46+
| ``J(x)`` | [`jac_lin`](@ref), [`jac_nln`](@ref), [`jac`](@ref), [`jac_lin_op`](@ref), [`jac_lin_op!`](@ref), [`jac_nln_op`](@ref), [`jac_nln_op!`](@ref),[`jac_op`](@ref), [`jac_op!`](@ref), [`jac_lin_coord`](@ref), [`jac_lin_coord!`](@ref), [`jac_nln_coord`](@ref), [`jac_nln_coord!`](@ref), [`jac_coord`](@ref), [`jac_coord!`](@ref), [`jac_dense!`](@ref), [`jac_lin_structure`](@ref), [`jac_lin_structure!`](@ref), [`jac_nln_structure`](@ref), [`jac_nln_structure!`](@ref), [`jac_structure`](@ref), [`jprod_lin`](@ref), [`jprod_lin!`](@ref), [`jprod_nln`](@ref), [`jprod_nln!`](@ref), [`jprod`](@ref), [`jprod!`](@ref), [`jtprod_lin`](@ref), [`jtprod_lin!`](@ref), [`jtprod_nln`](@ref), [`jtprod_nln!`](@ref), [`jtprod`](@ref), [`jtprod!`](@ref) |
47+
| ``\nabla^2 L(x,y)`` | [`hess`](@ref), [`hess_op`](@ref), [`hess_coord`](@ref), [`hess_coord!`](@ref), [`hess_dense!`](@ref), [`hess_structure`](@ref), [`hess_structure!`](@ref), [`hprod`](@ref), [`hprod!`](@ref), [`jth_hprod`](@ref), [`jth_hprod!`](@ref), [`jth_hess`](@ref), [`jth_hess_coord`](@ref), [`jth_hess_coord!`](@ref), [`ghjvprod`](@ref), [`ghjvprod!`](@ref) |
4748

4849
If only a subset of the functions listed above is implemented, you can indicate which ones are not available when creating the [`NLPModelMeta`](@ref), using the keyword arguments
4950
`grad_available`, `jac_available`, `hess_available`, `jprod_available`, `jtprod_available`, and `hprod_available`.
51+
You can also specify whether the Jacobian of the constraints and the Hessian of the objective or Lagrangian are sparse using the keyword arguments `sparse_jacobian` and `sparse_hessian`.
5052

5153
## [API for NLSModels](@id nls-api)
5254

‎docs/src/guidelines.md‎

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -60,28 +60,37 @@ The following functions should be defined:
6060
- Objective (unconstrained models only need to worry about these)
6161
- `obj(nlp, x)`
6262
- `grad!(nlp, x, g)`
63-
- `hess_structure!(nlp, hrows, hcols)`
64-
- `hess_coord!(nlp, x, hvals; obj_weight=1)`
63+
- `hess_structure!(nlp, hrows, hcols)` (sparse Hessian)
64+
- `hess_coord!(nlp, x, hvals; obj_weight=1)` (sparse Hessian)
65+
- `hess_dense!(nlp, x, Hx; obj_weight=1)` (dense Hessian)
6566
- `hprod!(nlp, x, v, Hv; obj_weight=1)` (actually defaults to calling the constrained case)
6667
- Constraints (constrained models need to worry about these and the ones above)
6768
- `cons_lin!(nlp, x, c)`
6869
- `cons_nln!(nlp, x, c)`
69-
- `jac_lin_structure!(nlp, jrows, jcols)`
70-
- `jac_nln_structure!(nlp, jrows, jcols)`
71-
- `jac_lin_coord!(nlp, x, jvals)`
72-
- `jac_nln_coord!(nlp, x, jvals)`
70+
- `jac_lin_structure!(nlp, jrows, jcols)` (sparse Jacobian)
71+
- `jac_nln_structure!(nlp, jrows, jcols)` (sparse Jacobian)
72+
- `jac_lin_coord!(nlp, x, jvals)` (sparse Jacobian)
73+
- `jac_nln_coord!(nlp, x, jvals)` (sparse Jacobian)
74+
- `jac_dense!(nlp, x, Jx)` (dense Jacobian)
7375
- `jprod_lin!(nlp, x, v, Jv)`
7476
- `jprod_nln!(nlp, x, v, Jv)`
7577
- `jtprod_lin!(nlp, x, v, Jtv)`
7678
- `jtprod_nln!(nlp, x, v, Jtv)`
77-
- `hess_coord!(nlp, x, y, hvals; obj_weight=1)`
79+
- `hess_coord!(nlp, x, y, hvals; obj_weight=1)` (sparse Hessian)
80+
- `hess_dense!(nlp, x, y, Hx; obj_weight=1)` (dense Hessian)
7881
- `hprod!(nlp, x, y, v, Hv; obj_weight=1)`
7982

8083
The linear constraints are specified at the initialization of the `NLPModelMeta` using the keyword arguement `lin`.
8184
The indices of linear and nonlinear constraints are respectively available in `nlp.meta.lin` and `nlp.meta.nln`.
8285
If your model uses only linear (resp. nonlinear) constraints, then it suffices to implement the `*_lin` (resp. `*_nln`) functions.
8386
Alternatively, one could implement only the functions without the suffixes `_nln!` (e.g., only `cons!`), but this might run into errors with tools differentiating linear and nonlinear constraints.
8487

88+
If the Jacobian or the Hessian of the Lagrangian is dense, there is no need to implement the corresponding `*_structure!` and `*_coord!` methods.
89+
Only the corresponding `*_dense!` methods need to be implemented.
90+
This is specified at the initialization of [`NLPModelMeta`](@ref) through the keyword arguments `sparse_jacobian` and `sparse_hessian`.
91+
In the dense case, linear and nonlinear constraints are handled together.
92+
Only an in-place API is available for dense Jacobians and Hessians (`jac_dense!` and `hess_dense!`).
93+
8594
## [Availability of the API](@id availability-api)
8695

8796
If only a subset of the functions listed above is implemented, you can indicate which ones are not available when creating the [`NLPModelMeta`](@ref), using the keyword arguments

‎docs/src/index.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -106,16 +106,18 @@ Attribute | Type | Notes
106106
`jfree` | `Vector{Int}` | indices of "free" constraints (there shouldn't be any)
107107
`jinf` | `Vector{Int}` | indices of the visibly infeasible constraints
108108
`nnzo` | `Int` | number of nonzeros in the gradient
109-
`nnzj` | `Int` | number of nonzeros in the sparse Jacobian
110-
`lin_nnzj` | `Int` | number of nonzeros in the sparse linear constraints Jacobian
111-
`nln_nnzj` | `Int` | number of nonzeros in the sparse nonlinear constraints Jacobian
112-
`nnzh` | `Int` | number of nonzeros in the lower triangular part of the sparse Hessian of the Lagrangian
109+
`nnzj` | `Int` | number of nonzeros in the Jacobian
110+
`lin_nnzj` | `Int` | number of nonzeros in the linear constraints Jacobian
111+
`nln_nnzj` | `Int` | number of nonzeros in the nonlinear constraints Jacobian
112+
`nnzh` | `Int` | number of nonzeros in the lower triangular part of the Hessian of the Lagrangian
113113
`minimize` | `Bool` | true if `optimize == minimize`
114114
`islp` | `Bool` | true if the problem is a linear program
115115
`name` | `String` | problem name
116+
`sparse_jacobian` | `Bool` | true if the Jacobian of the constraints is sparse
117+
`sparse_hessian` | `Bool` | true if the Hessian of the Lagrangian is sparse
116118
`grad_available` | `Bool` | true if the gradient of the objective is available
117-
`jac_available` | `Bool` | true if the sparse Jacobian of the constraints is available
118-
`hess_available` | `Bool` | true if the sparse Hessian of the Lagrangian is available
119+
`jac_available` | `Bool` | true if the Jacobian of the constraints is available
120+
`hess_available` | `Bool` | true if the Hessian of the Lagrangian is available
119121
`jprod_available` | `Bool` | true if the Jacobian-vector product `J * v` is available
120122
`jtprod_available` | `Bool` | true if the transpose Jacobian-vector product `J' * v` is available
121123
`hprod_available` | `Bool` | true if the Hessian-vector product of the Lagrangian `H * v` is available

0 commit comments

Comments
 (0)