Add latent novelty query strategy using OODGuard (#1678)

* add latent novelty query strategy using OODGuard

* linting, changelong addition and readme

* improve viz

* Address review comments

---------

Co-authored-by: Kaustubh Tangsali <ktangsali@oci-nrt-cs-001-vscode-01.cm.cluster>
Co-authored-by: root <root@pool0-03775.cm.cluster>

Author: 3 people

CHANGELOG.md

@@ -32,6 +32,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   protocols and `physicsnemo.experimental.uq.VariationalGPHead`, with a
   layered structure (generic AL driver / GP-UQ recipe / aero adapter)
   designed for reuse on other UQ-based regression problems.
+- Adds `LatentNoveltyQueryStrategy` to the active-learning aero recipe,
+  a third acquisition strategy that ranks unlabeled samples by their
+  average kNN cosine distance in the encoder's learned geometry latent
+  — reusing the same `OODGuard`
+  (`physicsnemo.experimental.guardrails.embedded`) that flags
+  out-of-distribution inputs at inference time. The guard is calibrated
+  on the currently labeled set each round; round 1 falls back to
+  class-balanced random because the calibration buffer is empty. New
+  public `OODGuard.score_geometry()` method exposes the raw per-sample
+  geometry-latent kNN distance as a continuous score for downstream
+  consumers (e.g. AL acquisition) without the boolean thresholding /
+  warning emission of `OODGuard.check()`.
 
 ### Changed
 

docs/img/al_aero_shiftsuv_correlations.png

@@ -110,6 +110,7 @@ The several examples inside PhysicsNeMo can be classified based on their domains
 ### Active Learning
 
 1. [Classify the famous two-moons data distribution using Active learning](./active_learning/moons/)
+2. [Active Learning for Surface-CFD Aerodynamic Surrogates](./cfd/external_aerodynamics/active_learning_aero/)
 
 ## Additional examples
 

docs/img/al_aero_shiftsuv_summary.png

@@ -0,0 +1,18 @@
+<!-- markdownlint-disable -->
+# Active Learning for Surface-CFD Aerodynamic Surrogates
+
+This example lives under the CFD examples tree, alongside the surface-CFD
+backbone it builds on:
+[**`physicsnemo/examples/cfd/external_aerodynamics/active_learning_aero`**](../../cfd/external_aerodynamics/active_learning_aero/README.md).
+
+It demonstrates end-to-end active learning on the
+[ShiftSUV](https://huggingface.co/datasets/luminary-shift/SUV) surface-CFD
+dataset using an uncertainty-aware GeoTransolver + Variational GP head, with
+three plug-and-play acquisition strategies (UQ-driven, class-balanced random,
+and latent-novelty). The AL loop itself is problem-agnostic — only the
+physics/metrology hooks are CFD-specific — so the same recipe can drive any
+uncertainty-quantified regression task.
+
+See the [full README](../../cfd/external_aerodynamics/active_learning_aero/README.md)
+for the recipe overview, configuration, results, and adapting-to-a-new-problem
+guide.

examples/README.md

@@ -15,3 +15,4 @@ src/manifests/*.json
 # Local backup snapshots (e.g. manifest_class_*.json.bak_*).
 *.bak
 *.bak_*
+slurm_logs/

examples/active_learning/active_learning_aero/README.md

@@ -222,7 +222,7 @@ runs/geotransolver/surface/al_shiftsuv_uq/joint_uq/
 
 ## Acquisition strategies
 
-Three strategies ship with this example, all implementing the
+Four strategies ship with this example, all implementing the
 `physicsnemo.active_learning.QueryStrategy` protocol. Select via
 `++acquisition=…`:
 
@@ -231,6 +231,7 @@ Three strategies ship with this example, all implementing the
 | **JointUQQueryStrategy** | `joint_uq` | The recommended UQ-driven default. Per-sample score is `max(|disagreement|, 2 · GP_std)`, where *disagreement* is `|Cd_GP − Cd_field|` — i.e. the gap between the GP-head Cd prediction and the field-integrated Cd recovered from the same forward pass. |
 | **RandomQueryStrategy** | `random` | Pure random baseline (uniform over the pool). Use as a UQ-vs-random sanity check. |
 | **ClassBalancedRandomQueryStrategy** | `class_balanced_random` | Random *within each class*, with the per-round budget split proportionally. Use as a strong baseline that controls for class imbalance in the pool. |
+| **LatentNoveltyQueryStrategy** | `latent_novelty` | Encoder-only novelty signal: each round we calibrate the embedded `OODGuard` (from `physicsnemo.experimental.guardrails.embedded`) on the currently labeled set, then rank unlabeled samples by their average kNN cosine distance in the learned geometry-latent space. Reuses the same guardrail that flags OOD inputs at inference time as the acquisition signal. The first round falls back to class-balanced random because the calibration buffer is empty. |
 
 Adding a new strategy is a matter of subclassing `QueryStrategy` from
 `physicsnemo.active_learning.protocols`, implementing
@@ -246,9 +247,10 @@ AL setup involves an oracle that synthesizes labels on demand.
 
 The plot below summarizes a experiment on the ShiftSUV
 out-of-distribution dataset (1727 total samples; 181 held out for test,
-leaving 1546 in the trainable pool). UQ and class-balanced random both
-close the gap between the pretrained DrivAerStar Fastback-only model and a
-ShiftSUV full-data ceiling that sees every trainable sample (n = 1546). Pressure
+leaving 1546 in the trainable pool). All three acquisition strategies —
+joint UQ, class-balanced random, and latent novelty — close the gap
+between the pretrained DrivAerStar Fastback-only model and a ShiftSUV
+full-data ceiling that sees every trainable sample (n = 1546). Pressure
 and wall-shear-stress (WSS) RMS errors are reported in physical units
 after un-standardization.
 
@@ -276,11 +278,11 @@ and wall-shear-stress magnitude. The violins below show how the
 **distribution of per-sample correlations** tightens across rounds:
 the median moves toward 1.0, the 5th-percentile dashed line catches
 up (worst-case samples improve faster than the best-case ones), and
-the lower tail of the violin shrinks. Both UQ and class-balanced
-random reach median ρ > 0.97 by round 16 (n=160 labels) — well before
-the labels-needed thresholds in the table below — meaning the spatial
-patterns are already correct long before the absolute RMS hits its
-asymptote.
+the lower tail of the violin shrinks. All three strategies — UQ,
+class-balanced random, and latent novelty — reach median ρ > 0.97 by
+round 16 (n=160 labels) — well before the labels-needed thresholds in
+the table below — meaning the spatial patterns are already correct
+long before the absolute RMS hits its asymptote.
 
 ![Per-sample Spearman correlations across AL rounds](../../../../docs/img/al_aero_shiftsuv_correlations.png)
 
@@ -290,22 +292,26 @@ Numbers from the rightmost panel of the summary plot —
 **labels needed to land within X% of the full-data RMS asymptote**
 (n_pool = 1546):
 
-| Within X% of ceiling | Joint-UQ labels | Class-bal random labels | Fraction of pool |
-|----------------------|-----------------|-------------------------|------------------|
-| 100% | 220 | 210 | ~14% |
-| 50% | 410 | 390 | ~26% |
-| 25% | 650 | 630 | ~41% |
-| 10% | 910 | 930 | ~60% |
-| 5% | 1040 | 1060 | ~67% |
-
-At the largest budget reached in this run — UQ at n=1120,
-BAL at n=1100 — pressure RMS is **15.17 Pa (UQ) / 15.49 Pa (BAL)**
-against a full-data ceiling of **14.85 Pa**, i.e. +2.1% / +4.3% above
-the asymptote. Read the row as: *"to drive the
-surface-field RMS to within 5% of what training on every available
-sample would give us, we need to hand-label roughly two-thirds of the
-pool"* — and the +25% row as the more frugal *"with ~40% of the labels
-we already cut the gap-to-ceiling to a quarter of what it was."*
+| Within X% of ceiling | Joint-UQ labels | Class-bal random labels | Latent-novelty labels | Fraction of pool |
+|----------------------|-----------------|-------------------------|-----------------------|------------------|
+| 100% | 230 | 210 | 210 | ~14% |
+| 50% | 430 | 410 | 410 | ~27% |
+| 25% | 670 | 670 | 680 | ~43% |
+| 10% | 970 | 990 | 980 | ~63% |
+| 5% | 1090 | 1120 | 1120 | ~72% |
+
+At the final round of each chain (UQ at n=1150; BAL and LN at n=1140),
+pressure RMS is **15.00 Pa (UQ) / 15.30 Pa (BAL) / 15.22 Pa (LN)**
+against a full-data ceiling of **14.57 Pa**, i.e. +3.0% / +5.0% / +4.5%
+above the asymptote. Read the +5% row as: *"to drive the surface-field
+RMS to within 5% of what training on every available sample would give
+us, we need to hand-label roughly two-thirds of the pool"* — and the
++25% row as the more frugal *"with ~40% of the labels we already cut
+the gap-to-ceiling to a quarter of what it was."* Joint-UQ wins by a
+small but consistent margin at every threshold; latent novelty matches
+class-balanced random closely without using class labels at
+acquisition time, making it a viable drop-in for problems where class
+metadata is unavailable.
 
 ## Adapting to a new problem
 

examples/cfd/external_aerodynamics/active_learning_aero/.gitignore

@@ -57,8 +57,9 @@ consistency_detach_transolver: false
 al_rounds: 5                   # Number of query-train-evaluate rounds
 samples_per_round: 50          # Samples selected per round
 test_samples_per_class: 100    # Held out for fixed test set (rest goes to pool)
-acquisition: "joint_uq"        # "joint_uq" or "random"
+acquisition: "joint_uq"        # "joint_uq" | "random" | "class_balanced_random" | "latent_novelty"
 random_seed: 42                # Seed for random baseline (used when acquisition=random)
+latent_novelty_knn_k: 10       # k for OODGuard kNN scoring (only used when acquisition=latent_novelty); auto-clamped to labeled-pool size
 
 # --- Fine-tuning schedule ---
 fine_tune_epochs: 20           # Epochs per AL round

examples/cfd/external_aerodynamics/active_learning_aero/README.md

@@ -68,6 +68,7 @@
     ClassBalancedRandomQueryStrategy,
     DummyLabelStrategy,
     JointUQQueryStrategy,
+    LatentNoveltyQueryStrategy,
     RandomQueryStrategy,
 )
 from aero_metrology import FieldMetrologyStrategy
@@ -304,23 +305,33 @@ def main(cfg: DictConfig) -> None:
         )
 
     # ---- Strategies ----
-    if acquisition == "joint_uq":
-        query_strategy = JointUQQueryStrategy(
-            max_samples=samples_per_round, precision=precision
-        )
-    elif acquisition == "class_balanced_random":
-        query_strategy = ClassBalancedRandomQueryStrategy(
-            max_samples=samples_per_round, seed=random_seed
-        )
-    elif acquisition == "random":
-        query_strategy = RandomQueryStrategy(
-            max_samples=samples_per_round, seed=random_seed
-        )
-    else:
-        raise ValueError(
-            f"Unknown acquisition strategy: {acquisition!r}. "
-            f"Expected one of: 'joint_uq', 'random', 'class_balanced_random'."
-        )
+    match acquisition:
+        case "joint_uq":
+            query_strategy = JointUQQueryStrategy(
+                max_samples=samples_per_round, precision=precision
+            )
+        case "class_balanced_random":
+            query_strategy = ClassBalancedRandomQueryStrategy(
+                max_samples=samples_per_round, seed=random_seed
+            )
+        case "random":
+            query_strategy = RandomQueryStrategy(
+                max_samples=samples_per_round, seed=random_seed
+            )
+        case "latent_novelty":
+            knn_k = int(getattr(cfg, "latent_novelty_knn_k", 10))
+            query_strategy = LatentNoveltyQueryStrategy(
+                max_samples=samples_per_round,
+                precision=precision,
+                knn_k=knn_k,
+                cold_start_seed=random_seed,
+            )
+        case _:
+            raise ValueError(
+                f"Unknown acquisition strategy: {acquisition!r}. "
+                f"Expected one of: 'joint_uq', 'random', 'class_balanced_random', "
+                f"'latent_novelty'."
+            )
 
     metrology = FieldMetrologyStrategy(precision=precision)
     label_strategy = DummyLabelStrategy()