cleanup and ggmp added

Author: MarcusMNoack

AUTHORS.rst

@@ -11,3 +11,4 @@ Contributors
 ------------
 
 * Ronald Pandolfi
+* Vardaan Tekriwal

CLAUDE.md

@@ -11,8 +11,8 @@ pip install -e ".[tests]"
 # Run the full test suite
 pytest tests/
 
-# Run a single test
-pytest tests/test_fvgp.py::TestClassName::test_method_name
+# Run a single test (tests are top-level functions, not class methods)
+pytest tests/test_fvgp.py::test_single_task_init_basic
 
 # Run tests with coverage
 pytest tests --cov=./ --cov-report=xml
@@ -38,28 +38,36 @@ Both classes are composed of internal specialist objects created at `__init__` t
 | `GPdata` | [gp_data.py](fvgp/gp_data.py) | Data validation, shape tracking, Euclidean vs. non-Euclidean |
 | `GPprior` | [gp_prior.py](fvgp/gp_prior.py) | Kernel and mean function; default is anisotropic Matérn with ARD |
 | `GPlikelihood` | [gp_likelihood.py](fvgp/gp_likelihood.py) | Noise model (variances or callable) |
-| `GPMarginalLikelihood` | [gp_marginal_likelihood.py](fvgp/gp_marginal_likelihood.py) | Log marginal likelihood; owns Cholesky/LU factorizations of K+V |
+| `GPkv` | [gp_kv.py](fvgp/gp_kv.py) | Owns K+V matrix state and all factorizations; dispatches solves/logdets across linalg modes |
+| `GPMarginalLikelihood` | [gp_marginal_likelihood.py](fvgp/gp_marginal_likelihood.py) | Log marginal likelihood and its gradient; delegates factorization to `GPkv` |
 | `GPposterior` | [gp_posterior.py](fvgp/gp_posterior.py) | Posterior mean/covariance; information-theoretic quantities |
-| `GPtraining` | [gp_training.py](fvgp/gp_training.py) | Hyperparameter optimization (scipy, hgdl async, MCMC) |
+| `GPtraining` | [gp_training.py](fvgp/gp_training.py) | Hyperparameter optimization (scipy, hgdl async, MCMC, Adam) |
 
 ### Key supporting modules
 
-- **[gp_lin_alg.py](fvgp/gp_lin_alg.py)** — CPU/GPU linear algebra backend; Cholesky, LU, sparse solvers; defines `NonPositiveDefiniteError`
+- **[gp_lin_alg.py](fvgp/gp_lin_alg.py)** — CPU/GPU linear algebra primitives; Cholesky, LU, sparse solvers; defines `NonPositiveDefiniteError`
+- **[gp_kv.py](fvgp/gp_kv.py)** — `GPkv` manages all K+V state across linalg modes: `"Chol"`, `"CholInv"`, `"Inv"`, `"sparseLU"`, `"sparseCG"`, `"sparseMINRES"`, and preconditioned variants. The mode is set at init and determines which factorization is updated when data or hyperparameters change. Custom solvers can be injected as a 3-tuple of callables.
 - **[kernels.py](fvgp/kernels.py)** — 15+ built-in kernels including Matérn, squared exponential, Wendland (compactly supported)
 - **[gp_mcmc.py](fvgp/gp_mcmc.py)** — Adaptive Metropolis–Hastings sampler used for Bayesian hyperparameter inference
+- **[gp_actor.py](fvgp/gp_actor.py)** — `AsyncOptimizer` wraps `_MCMCActor` and `_AdamActor` for non-blocking background training; used by `GPtraining` for async MCMC and Adam modes
 
 ### Scaling to large datasets (`gp2Scale`)
 
 When `gp2Scale=True`, `GP` switches to a Wendland (compactly supported) kernel producing sparse covariance matrices and uses Dask for distributed computation. This path requires a Dask client to be passed in and uses sparse linear solvers instead of dense Cholesky.
 
 ### Customization API
 
-Kernels, mean functions, and noise models are all plain Python callables with standardized signatures. Users pass them as arguments to `GP`/`fvGP` constructors. Kernel gradients can be user-supplied or computed via finite differences.
+Kernels, mean functions, and noise models are all plain Python callables with standardized signatures. Users pass them as arguments to `GP`/`fvGP` constructors. The full hyperparameter vector is shared across kernel, mean, and noise callables, but each callable must only read its reserved index range. Kernel gradients can be user-supplied or computed via finite differences.
 
 ### Information-theoretic methods
 
 `GP` exposes `gp_entropy()`, `gp_mutual_information()`, `gp_kl_div()`, and predictive metrics (`rmse`, `nlpd`, `crps`, `r2`, `picp`), all computed via `GPposterior`.
 
+### Extension modules
+
+- **[ggmp.py](fvgp/ggmp.py)** — `GGMP` (Gaussian GP for Gaussian Mixture data): fits K GMM components per station, each backed by its own `GP`; intended for distributional regression. Written by Vardaan Tekriwal. Excluded from the test suite (`# pragma: no cover`).
+- **[deep_kernel_network.py](fvgp/deep_kernel_network.py)** — `Network` (PyTorch `nn.Module`): a 3-layer ReLU network used as a feature extractor for deep kernel learning. Excluded from the test suite (`# pragma: no cover`).
+
 ## Dependencies
 
 Core: `numpy`, `scipy`, `dask`, `distributed`, `hgdl`, `loguru`

docs/source/api/ggmp.md

@@ -0,0 +1,23 @@
+# GGMP
+
+`GGMP` (Gaussian GP for Gaussian Mixture data) models distributional observations
+as Gaussian mixture models, fitting one GP per mixture component.
+
+```{eval-rst}
+.. autoclass:: fvgp.ggmp.GGMP
+    :members:
+```
+
+## hyperparameters
+
+```{eval-rst}
+.. autoclass:: fvgp.ggmp.hyperparameters
+    :members:
+```
+
+## NormalLikelihood
+
+```{eval-rst}
+.. autoclass:: fvgp.ggmp.NormalLikelihood
+    :members:
+```

docs/source/api/overview.md

@@ -6,6 +6,7 @@
 
 GP.md
 fvGP.md
+ggmp.md
 kernels.md
 gpMCMC.md
 logging.md

fvgp/__init__.py

@@ -14,9 +14,9 @@
 from .gp import GP
 from .fvgp import fvGP
 from .gp_mcmc import gpMCMC, ProposalDistribution
+from . import ggmp
 
 
-
-__all__ = ['GP', 'fvGP', 'gpMCMC', 'ProposalDistribution']
+__all__ = ['GP', 'fvGP', 'gpMCMC', 'ProposalDistribution', 'ggmp']
 
 logger.disable('fvgp')

fvgp/fvgp.py

@@ -160,24 +160,37 @@ class fvGP(GP):
         A local client is used as the default.
     gp2Scale_batch_size : int, optional
         Matrix batch size for distributed computing in gp2Scale. The default is 10000.
-    gp2Scale_linalg_mode : str, optional
-        One of ``Chol``, ``sparseLU``, ``sparseCG``, ``sparseMINRES``, ``sparseSolve``, ``sparseCGpre``
-        (incomplete LU preconditioner), or ``sparseMINRESpre``. The default is None which amounts to
-        an automatic determination of the mode. For advanced customization options
-        this can also be an iterable with three callables: the first f(K), where K is the covariance matrix
-        to compute a factorization object
-        which is available in the second and third callable. The second being the linear solve f(obj, vec),
-        and the third being the logdet=f(obj). If a factorization object is not required, the first callable
-        should return the matrix itself (K).
-    calc_inv : bool, optional
-        If True, the algorithm calculates and stores the inverse of the covariance
-        matrix after each training or update of the dataset or hyperparameters,
-        which makes computing the posterior covariance faster (3-10 times).
-        For larger problems (>5000 data points), the use of inversion should be avoided due
-        to computational instability and costs. The default is
-        False. Note, the training will not use the
-        inverse for stability reasons. Storing the inverse is
-        a good option when the dataset is not too large and the posterior covariance is heavily used.
+    linalg_mode : str, optional
+        Controls the linear-algebra backend used to solve (K+V)x=b and compute log|K+V|.
+        The default is ``None``, which selects ``"Chol"`` for standard GPs and automatically
+        picks the best sparse mode for gp2Scale GPs.
+
+        **Recommended for standard (non-gp2Scale) GPs:**
+
+        * ``"Chol"`` *(default)* — Cholesky factorization; numerically stable and memory-efficient.
+        * ``"CholInv"`` — Cholesky factorization, then explicitly stores the inverse; speeds up posterior
+          covariance evaluation 3–10×. Avoid for datasets larger than ~5 000 points due to memory
+          and numerical cost. Training always uses the Cholesky factor for stability.
+        * ``"Inv"`` — computes and stores the explicit inverse directly (no Cholesky). Only suitable for
+          very small datasets where posterior covariance is computed many times.
+
+        **Specialized for gp2Scale (sparse covariance matrices):**
+
+        * ``"sparseLU"`` — sparse LU factorization; good default for sparse systems up to ~50 000 points.
+        * ``"sparseCG"`` — sparse conjugate-gradient iterative solver.
+        * ``"sparseMINRES"`` — sparse MINRES iterative solver.
+        * ``"sparseSolve"`` — direct sparse solve via scipy.
+        * ``"sparseCGpre"`` — conjugate-gradient with an incomplete-LU preconditioner.
+        * ``"sparseMINRESpre"`` — MINRES with an incomplete-LU preconditioner.
+
+        **Custom solver (any GP):**
+
+        Pass an iterable of three callables ``[f_factor, f_solve, f_logdet]``:
+
+        * ``f_factor(K)`` — receives the covariance matrix and returns a factorization object
+          (or the matrix itself if no factorization is needed).
+        * ``f_solve(obj, b)`` — solves the linear system and returns the solution vector.
+        * ``f_logdet(obj)`` — returns the log-determinant as a scalar.
     ram_economy : bool, optional
         Only of interest if the gradient and/or Hessian of the log marginal likelihood is/are used for the training.
         If True, components of the derivative of the log marginal likelihood are
@@ -306,8 +319,7 @@ def __init__(
         gp2Scale=False,
         gp2Scale_dask_client=None,
         gp2Scale_batch_size=10000,
-        gp2Scale_linalg_mode=None,
-        calc_inv=False,
+        linalg_mode=None,
         ram_economy=False,
         args=None
     ):
@@ -339,8 +351,7 @@ def __init__(
             gp2Scale=gp2Scale,
             gp2Scale_dask_client=gp2Scale_dask_client,
             gp2Scale_batch_size=gp2Scale_batch_size,
-            gp2Scale_linalg_mode=gp2Scale_linalg_mode,
-            calc_inv=calc_inv,
+            linalg_mode=linalg_mode,
             ram_economy=ram_economy,
             args=args)