Default use_sparse_FOC_jac to True; pin dense path in test_inner_loop

Author: marcelolafleur

CHANGELOG.md

@@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
-- Adds an optional `use_sparse_FOC_jac` `Specifications` parameter (default off, so default runs are unchanged) that accelerates the time path iteration (TPI) household solve. When True, `scipy.optimize.root` is given a sparse (banded) finite-difference Jacobian for the stacked household Euler and labor first order conditions: the sparsity pattern is auto-detected once per problem size and the solver then needs far fewer function evaluations per Jacobian build (about 20x fewer on the default S=80 cohort solve), with an automatic fallback to dense finite differences if the Jacobian is not sparse enough to benefit or if a solve fails. The result matches the dense-finite-difference solution to within the model's resource-constraint accuracy floor on every calibration tested (OG-Core standard example, OG-ETH, OG-ZAF, OG-PHL, OG-IDN), giving roughly a 1.9-2.4x TPI speedup at no accuracy cost.
+- Adds a `use_sparse_FOC_jac` `Specifications` parameter (default `True`) that accelerates the time path iteration (TPI) household solve. With it on, `scipy.optimize.root` is given a sparse (banded) finite-difference Jacobian for the stacked household Euler and labor first order conditions: the sparsity pattern is auto-detected once per problem size and the solver then needs far fewer function evaluations per Jacobian build (about 20x fewer on the default S=80 cohort solve), with an automatic fallback to dense finite differences if the Jacobian is not sparse enough to benefit or if a solve fails. The result matches the legacy dense-finite-difference solution to within the model's resource-constraint accuracy floor on every calibration tested (OG-Core standard example, OG-ETH, OG-ZAF, OG-PHL, OG-IDN), giving roughly a 1.9-2.4x TPI speedup at no accuracy cost. Set `use_sparse_FOC_jac=False` to recover bit-identical agreement with v0.16.0 and earlier.
 
 ## [0.16.0] - 2026-06-02 12:00:00
 

docs/book/content/theory/equilibrium.md

@@ -25,7 +25,7 @@ In all of the specifications of `OG-Core`, we use a two-stage fixed point algori
 
 Our approach is to choose the minimum number of macroeconomic variables in an outer loop in order to be able to solve the household's $2JS$ Euler equations in terms of only the $\bar{n}_{j,s}$ and $\bar{b}_{j,s+1}$ variables directly, holding all other variables constant. The household system of Euler equations has a provable root solution and is orders of magnitude more tractable (less nonlinear) to solve holding these outer loop variables constant.
 
-Moreover, with the outer-loop variables held fixed, each cohort's system of $2S$ Euler equations is not only less nonlinear but structurally sparse: every equation involves at most five of the $2S$ unknowns---a household's own age and its immediate neighbors. The root finder normally probes each unknown separately when building each step ($2S = 160$ evaluations of the system when $S = 80$), but with most equations depending on only a handful of unknowns, those affecting no common equation can be probed together, cutting the count to about seven at $S = 80$---a number set by how many neighbors couple, not by $S$. The parameter `use_sparse_FOC_jac` (default `False`) turns this on; the solver falls back to the standard calculation otherwise. The structure is derived in Appendix {ref}`SecAppDerivHHjac`.
+Moreover, with the outer-loop variables held fixed, each cohort's system of $2S$ Euler equations is not only less nonlinear but structurally sparse: every equation involves at most five of the $2S$ unknowns---a household's own age and its immediate neighbors. The root finder normally probes each unknown separately when building each step ($2S = 160$ evaluations of the system when $S = 80$), but with most equations depending on only a handful of unknowns, those affecting no common equation can be probed together, cutting the count to about seven at $S = 80$---a number set by how many neighbors couple, not by $S$. The parameter `use_sparse_FOC_jac` (default `True`) controls this; set it to `False` to use the legacy dense-finite-difference Jacobian on every call. The structure is derived in Appendix {ref}`SecAppDerivHHjac`.
 
 ```{figure} ./images/HH_jac_sparsity.png
 ---

ogcore/default_parameters.json

@@ -4394,15 +4394,15 @@
     },
     "use_sparse_FOC_jac": {
         "title": "Flag to use a sparse (banded) finite-difference Jacobian in the household first order condition root finder",
-        "description": "Flag to use a sparse (banded) finite-difference Jacobian in the household first order condition root finder. When True, the sparsity pattern of the stacked Euler/labor first order conditions is auto-detected once per problem size and supplied to scipy.optimize.root, which then needs far fewer function evaluations per Jacobian build than the default dense finite differences. The solver falls back to the dense finite-difference Jacobian automatically if the Jacobian is not sparse enough to benefit or if a solve fails, so the result is unchanged.",
+        "description": "Flag to use a sparse (banded) finite-difference Jacobian in the household first order condition root finder. When True (the default), the sparsity pattern of the stacked Euler/labor first order conditions is auto-detected once per problem size and supplied to scipy.optimize.root, which then needs far fewer function evaluations per Jacobian build than the default dense finite differences. The solver falls back to the dense finite-difference Jacobian automatically if the Jacobian is not sparse enough to benefit or if a solve fails. Set to False to use the legacy dense Jacobian on every call.",
         "short_description": "Use a banded finite-difference Jacobian in the household solver",
         "param_notation": "$\\texttt{use_sparse_FOC_jac}$",
         "section_1": "Model Solution Parameters",
         "notes": "",
         "type": "bool",
         "value": [
             {
-                "value": false
+                "value": true
             }
         ],
         "validators": {

tests/test_TPI.py

@@ -388,11 +388,15 @@ def test_params_to_array(tax_func_type):
 def test_inner_loop():
     # Test TPI.inner_loop function.  Provide inputs to function and
     # ensure that output returned matches what it has been before.
+    # Explicitly disable use_sparse_FOC_jac so this regression test
+    # continues to exercise the legacy dense-finite-difference path
+    # (the sparse path is covered by test_inner_loop_sparse_FOC_jac).
     input_tuple = utils.safe_read_pickle(
         os.path.join(CUR_PATH, "test_io_data", "tpi_inner_loop_inputs.pkl")
     )
     guesses, outer_loop_vars_old, initial_values, ubi, j, ind = input_tuple
     p = Specifications()
+    p.update_specifications({"use_sparse_FOC_jac": False})
     r = outer_loop_vars_old[0]
     r_p = outer_loop_vars_old[2]
     w = outer_loop_vars_old[1]