Try to fix the performances of the split API

README.md

@@ -70,7 +70,7 @@ If Jacobian-vector products can be computed more efficiently than by evaluating
 
 The following method is defined if second-order derivatives are available:
 
-* `hess(model, x, y)`: evaluate *∇²L(x,y)*, the Hessian of the Lagrangian at `x` and `y`
+* `hess(model, x, y)`: evaluate *∇²L(x,y)*, the Hessian of the Lagrangian at `x` and `y` as a sparse matrix
 
 If Hessian-vector products can be computed more efficiently than by evaluating the Hessian explicitly, the following method may be implemented:
 
@@ -111,16 +111,18 @@ Attribute          | Type          | Notes
 `jfree`            | `Vector{Int}` | indices of "free" constraints (there shouldn't be any)
 `jinf`             | `Vector{Int}` | indices of the visibly infeasible constraints
 `nnzo`             | `Int`         | number of nonzeros in the gradient
-`nnzj`             | `Int`         | number of nonzeros in the sparse Jacobian
-`lin_nnzj`         | `Int`         | number of nonzeros in the sparse linear constraints Jacobian
-`nln_nnzj`         | `Int`         | number of nonzeros in the sparse nonlinear constraints Jacobian
-`nnzh`             | `Int`         | number of nonzeros in the lower triangular part of the sparse Hessian of the Lagrangian
+`nnzj`             | `Int`         | number of nonzeros in the Jacobian
+`lin_nnzj`         | `Int`         | number of nonzeros in the linear constraints Jacobian
+`nln_nnzj`         | `Int`         | number of nonzeros in the nonlinear constraints Jacobian
+`nnzh`             | `Int`         | number of nonzeros in the lower triangular part of the Hessian of the Lagrangian
 `minimize`         | `Bool`        | true if `optimize == minimize`
 `islp`             | `Bool`        | true if the problem is a linear program
 `name`             | `String`      | problem name
+`sparse_jacobian`  | `Bool`        | true if the Jacobian of the constraints is sparse
+`sparse_hessian`   | `Bool`        | true if the Hessian of the Lagrangian is sparse
 `grad_available`   | `Bool`        | true if the gradient of the objective is available
-`jac_available`    | `Bool`        | true if the sparse Jacobian of the constraints is available
-`hess_available`   | `Bool`        | true if the sparse Hessian of the Lagrangian is available
+`jac_available`    | `Bool`        | true if the Jacobian of the constraints is available
+`hess_available`   | `Bool`        | true if the Hessian of the Lagrangian is available
 `jprod_available`  | `Bool`        | true if the Jacobian-vector product `J * v` is available
 `jtprod_available` | `Bool`        | true if the transpose Jacobian-vector product `J' * v` is available
 `hprod_available`  | `Bool`        | true if the Hessian-vector product of the Lagrangian `H * v` is available

docs/src/api.md

@@ -29,6 +29,7 @@ If not, click on the link and go to the description.
 
 - `!` means inplace;
 - `_coord` means coordinate format;
+- `_dense` means dense format;
 - `prod` means matrix-vector product;
 - `_op` means operator (as in [LinearOperators.jl](https://github.com/JuliaSmoothOptimizers/LinearOperators.jl));
 - `_lin` and `_nln` respectively refer to linear and nonlinear constraints.
@@ -40,13 +41,14 @@ NLPModels instances.
 |-------------------|-------------------------------------------|
 | ``f(x)``            | [`obj`](@ref), [`objgrad`](@ref), [`objgrad!`](@ref), [`objcons`](@ref), [`objcons!`](@ref) |
 | ``\nabla f(x)``     | [`grad`](@ref), [`grad!`](@ref), [`objgrad`](@ref), [`objgrad!`](@ref) |
-| ``\nabla^2 f(x)``   | [`hess`](@ref), [`hess_op`](@ref), [`hess_op!`](@ref), [`hess_coord`](@ref), [`hess_coord`](@ref), [`hess_structure`](@ref), [`hess_structure!`](@ref), [`hprod`](@ref), [`hprod!`](@ref) |
+| ``\nabla^2 f(x)``   | [`hess`](@ref), [`hess_op`](@ref), [`hess_op!`](@ref), [`hess_coord`](@ref), [`hess_coord!`](@ref), [`hess_dense!`](@ref), [`hess_structure`](@ref), [`hess_structure!`](@ref), [`hprod`](@ref), [`hprod!`](@ref) |
 | ``c(x)``            | [`cons_lin`](@ref), [`cons_lin!`](@ref), [`cons_nln`](@ref), [`cons_nln!`](@ref), [`cons`](@ref), [`cons!`](@ref), [`objcons`](@ref), [`objcons!`](@ref) |
-| ``J(x)``            | [`jac_lin`](@ref), [`jac_nln`](@ref), [`jac`](@ref), [`jac_lin_op`](@ref), [`jac_lin_op!`](@ref), [`jac_nln_op`](@ref), [`jac_nln_op!`](@ref),[`jac_op`](@ref), [`jac_op!`](@ref), [`jac_lin_coord`](@ref), [`jac_lin_coord!`](@ref), [`jac_nln_coord`](@ref), [`jac_nln_coord!`](@ref), [`jac_coord`](@ref), [`jac_coord!`](@ref), [`jac_lin_structure`](@ref), [`jac_lin_structure!`](@ref), [`jac_nln_structure`](@ref), [`jac_nln_structure!`](@ref), [`jac_structure`](@ref), [`jprod_lin`](@ref), [`jprod_lin!`](@ref), [`jprod_nln`](@ref), [`jprod_nln!`](@ref), [`jprod`](@ref), [`jprod!`](@ref), [`jtprod_lin`](@ref), [`jtprod_lin!`](@ref), [`jtprod_nln`](@ref), [`jtprod_nln!`](@ref), [`jtprod`](@ref), [`jtprod!`](@ref) |
-| ``\nabla^2 L(x,y)`` | [`hess`](@ref), [`hess_op`](@ref), [`hess_coord`](@ref), [`hess_coord!`](@ref), [`hess_structure`](@ref), [`hess_structure!`](@ref), [`hprod`](@ref), [`hprod!`](@ref), [`jth_hprod`](@ref), [`jth_hprod!`](@ref), [`jth_hess`](@ref), [`jth_hess_coord`](@ref), [`jth_hess_coord!`](@ref), [`ghjvprod`](@ref), [`ghjvprod!`](@ref)  |
+| ``J(x)``            | [`jac_lin`](@ref), [`jac_nln`](@ref), [`jac`](@ref), [`jac_lin_op`](@ref), [`jac_lin_op!`](@ref), [`jac_nln_op`](@ref), [`jac_nln_op!`](@ref),[`jac_op`](@ref), [`jac_op!`](@ref), [`jac_lin_coord`](@ref), [`jac_lin_coord!`](@ref), [`jac_nln_coord`](@ref), [`jac_nln_coord!`](@ref), [`jac_coord`](@ref), [`jac_coord!`](@ref), [`jac_dense!`](@ref), [`jac_lin_structure`](@ref), [`jac_lin_structure!`](@ref), [`jac_nln_structure`](@ref), [`jac_nln_structure!`](@ref), [`jac_structure`](@ref), [`jprod_lin`](@ref), [`jprod_lin!`](@ref), [`jprod_nln`](@ref), [`jprod_nln!`](@ref), [`jprod`](@ref), [`jprod!`](@ref), [`jtprod_lin`](@ref), [`jtprod_lin!`](@ref), [`jtprod_nln`](@ref), [`jtprod_nln!`](@ref), [`jtprod`](@ref), [`jtprod!`](@ref) |
+| ``\nabla^2 L(x,y)`` | [`hess`](@ref), [`hess_op`](@ref), [`hess_coord`](@ref), [`hess_coord!`](@ref), [`hess_dense!`](@ref), [`hess_structure`](@ref), [`hess_structure!`](@ref), [`hprod`](@ref), [`hprod!`](@ref), [`jth_hprod`](@ref), [`jth_hprod!`](@ref), [`jth_hess`](@ref), [`jth_hess_coord`](@ref), [`jth_hess_coord!`](@ref), [`ghjvprod`](@ref), [`ghjvprod!`](@ref)  |
 
 If only a subset of the functions listed above is implemented, you can indicate which ones are not available when creating the [`NLPModelMeta`](@ref), using the keyword arguments
 `grad_available`, `jac_available`, `hess_available`, `jprod_available`, `jtprod_available`, and `hprod_available`.
+You can also specify whether the Jacobian of the constraints and the Hessian of the objective or Lagrangian are sparse using the keyword arguments `sparse_jacobian` and `sparse_hessian`.
 
 ## [API for NLSModels](@id nls-api)
 

docs/src/guidelines.md

@@ -60,27 +60,42 @@ The following functions should be defined:
 - Objective (unconstrained models only need to worry about these)
   - `obj(nlp, x)`
   - `grad!(nlp, x, g)`
-  - `hess_structure!(nlp, hrows, hcols)`
-  - `hess_coord!(nlp, x, hvals; obj_weight=1)`
+  - `hess_structure!(nlp, hrows, hcols)` (sparse Hessian)
+  - `hess_coord!(nlp, x, hvals; obj_weight=1)` (sparse Hessian)
+  - `hess_dense!(nlp, x, Hx; obj_weight=1)` (dense Hessian)
   - `hprod!(nlp, x, v, Hv; obj_weight=1)` (actually defaults to calling the constrained case)
+
 - Constraints (constrained models need to worry about these and the ones above)
-  - `cons_lin!(nlp, x, c)`
-  - `cons_nln!(nlp, x, c)`
-  - `jac_lin_structure!(nlp, jrows, jcols)`
-  - `jac_nln_structure!(nlp, jrows, jcols)`
-  - `jac_lin_coord!(nlp, x, jvals)`
-  - `jac_nln_coord!(nlp, x, jvals)`
-  - `jprod_lin!(nlp, x, v, Jv)`
-  - `jprod_nln!(nlp, x, v, Jv)`
-  - `jtprod_lin!(nlp, x, v, Jtv)`
-  - `jtprod_nln!(nlp, x, v, Jtv)`
-  - `hess_coord!(nlp, x, y, hvals; obj_weight=1)`
+  - `cons!(nlp, x, c)`
+  - `jac_structure!(nlp, jrows, jcols)` (sparse Jacobian)
+  - `jac_coord!(nlp, x, jvals)` (sparse Jacobian)
+  - `jac_dense!(nlp, x, Jx)` (dense Jacobian)
+  - `jprod!(nlp, x, v, Jv)`
+  - `jtprod!(nlp, x, v, Jtv)`
+  - `hess_coord!(nlp, x, y, hvals; obj_weight=1)` (sparse Hessian)
+  - `hess_dense!(nlp, x, y, Hx; obj_weight=1)` (dense Hessian)
   - `hprod!(nlp, x, y, v, Hv; obj_weight=1)`
 
+When the split API is enabled (`nlp.meta.split_api = true`), the treatment of linear and nonlinear constraints and their associated Jacobians is separated, and the following additional functions must be provided:
+
+- `cons_lin!(nlp, x, c)`
+- `cons_nln!(nlp, x, c)`
+- `jac_lin_structure!(nlp, jrows, jcols)`
+- `jac_nln_structure!(nlp, jrows, jcols)`
+- `jac_lin_coord!(nlp, x, jvals)`
+- `jac_nln_coord!(nlp, x, jvals)`
+- `jprod_lin!(nlp, x, v, Jv)`
+- `jprod_nln!(nlp, x, v, Jv)`
+- `jtprod_lin!(nlp, x, v, Jtv)`
+- `jtprod_nln!(nlp, x, v, Jtv)`
+
 The linear constraints are specified at the initialization of the `NLPModelMeta` using the keyword arguement `lin`.
 The indices of linear and nonlinear constraints are respectively available in `nlp.meta.lin` and `nlp.meta.nln`.
-If your model uses only linear (resp. nonlinear) constraints, then it suffices to implement the `*_lin` (resp. `*_nln`) functions.
-Alternatively, one could implement only the functions without the suffixes `_nln!` (e.g., only `cons!`), but this might run into errors with tools differentiating linear and nonlinear constraints.
+
+If the Jacobian or the Hessian of the Lagrangian is dense, there is no need to implement the corresponding `*_structure!` and `*_coord!` methods.
+Only the corresponding `*_dense!` methods need to be implemented.
+Note that only the in-place API is provided for dense Jacobians and Hessians.
+This is specified at the initialization of [`NLPModelMeta`](@ref) through the keyword arguments `sparse_jacobian` and `sparse_hessian`.
 
 ## [Availability of the API](@id availability-api)
 

docs/src/index.md

@@ -106,16 +106,18 @@ Attribute          | Type          | Notes
 `jfree`            | `Vector{Int}` | indices of "free" constraints (there shouldn't be any)
 `jinf`             | `Vector{Int}` | indices of the visibly infeasible constraints
 `nnzo`             | `Int`         | number of nonzeros in the gradient
-`nnzj`             | `Int`         | number of nonzeros in the sparse Jacobian
-`lin_nnzj`         | `Int`         | number of nonzeros in the sparse linear constraints Jacobian
-`nln_nnzj`         | `Int`         | number of nonzeros in the sparse nonlinear constraints Jacobian
-`nnzh`             | `Int`         | number of nonzeros in the lower triangular part of the sparse Hessian of the Lagrangian
+`nnzj`             | `Int`         | number of nonzeros in the Jacobian
+`lin_nnzj`         | `Int`         | number of nonzeros in the linear constraints Jacobian
+`nln_nnzj`         | `Int`         | number of nonzeros in the nonlinear constraints Jacobian
+`nnzh`             | `Int`         | number of nonzeros in the lower triangular part of the Hessian of the Lagrangian
 `minimize`         | `Bool`        | true if `optimize == minimize`
 `islp`             | `Bool`        | true if the problem is a linear program
 `name`             | `String`      | problem name
+`sparse_jacobian`  | `Bool`        | true if the Jacobian of the constraints is sparse
+`sparse_hessian`   | `Bool`        | true if the Hessian of the Lagrangian is sparse
 `grad_available`   | `Bool`        | true if the gradient of the objective is available
-`jac_available`    | `Bool`        | true if the sparse Jacobian of the constraints is available
-`hess_available`   | `Bool`        | true if the sparse Hessian of the Lagrangian is available
+`jac_available`    | `Bool`        | true if the Jacobian of the constraints is available
+`hess_available`   | `Bool`        | true if the Hessian of the Lagrangian is available
 `jprod_available`  | `Bool`        | true if the Jacobian-vector product `J * v` is available
 `jtprod_available` | `Bool`        | true if the transpose Jacobian-vector product `J' * v` is available
 `hprod_available`  | `Bool`        | true if the Hessian-vector product of the Lagrangian `H * v` is available

docs/src/reference.md

@@ -1,17 +1,17 @@
 # Reference
-​
+
 ## Contents
-​
+
 ```@contents
 Pages = ["reference.md"]
 ```
-​
+
 ## Index
-​
+
 ```@index
 Pages = ["reference.md"]
 ```
-​
+
 ```@autodocs
 Modules = [NLPModels]
-```
+```