Skip to content

Beef up GaussianProcessSurrogate#745

Draft
AdrianSosic wants to merge 200 commits into
mainfrom
dev/gp
Draft

Beef up GaussianProcessSurrogate#745
AdrianSosic wants to merge 200 commits into
mainfrom
dev/gp

Conversation

@AdrianSosic
Copy link
Copy Markdown
Collaborator

@AdrianSosic AdrianSosic commented Feb 10, 2026

Rough agenda:

#743 (Cleanup)

  • Clean up GP code

#746 (GP Components)

  • Extract current GP setting into preset
  • Add EDBO / SMOOTHED_EDBO preset
  • Generalize kernel machinery into component machinery using generics
  • Add configurability for mean / likelihood
  • Turn current mean / noise factories into serializable classes using new generic mechanism
  • Add low-level support for gpytorch GP components

#747 (Index Kernel)

  • Add BayBE IndexKernel class
  • Add BayBE PositiveIndexKernel class

#763 (Kernel Arithmetic)

  • improved + and * operations for Kernels

#748 (Active Dimensions)

  • Enable active dimension control for kernels
  • Absorb hardcoded IndexKernel logic into kernel factory
  • Implement deprecation mechanism for breaking change

#776 (Kernel Factory Validation)

  • Ensure kernel factories check if they support their given parameters

#752 (CHEN Preset)

  • Add CHEN preset

#757 (BoTorch Preset)

  • Add BOTORCH preset

#789 (FitCriterion)

  • Make optimization criterion configurable

Must TODO

  • Stratification for multitask likelihoods
  • Add decorator for extending presets to apply mechanism they don't cover themselves (e.g. transfer learning)
  • Decide on new BayBE defaults (i.e. do we use BOTORCH, CHEN, etc, or some heuristic on top)
  • Documentation

Optional TODO

  • Add HVARFNER preset
  • Enable serialization for gpytorch GP components
  • Add preset Setting attribute

To be added but out of scope

  • Make optimizer configurable

AdrianSosic and others added 4 commits February 10, 2026 17:18
* Cleans up the `GaussianProcessSurrogate` class
* Adds transfer learning tests asserting that the mechanism works
regardless of which tasks are represented in the training data
@AdrianSosic AdrianSosic self-assigned this Feb 10, 2026
@AdrianSosic AdrianSosic added enhancement Expand / change existing functionality new feature New functionality on hold PR progress is awaiting for something else to continue refactor and removed on hold PR progress is awaiting for something else to continue labels Feb 10, 2026
AdrianSosic and others added 26 commits May 11, 2026 16:05
DevPR, parent is #745

Makes the optimization criterion of the `GaussianProcessSurrogate` model
configurable, in the form of a new `FitCriterion` enum. Potentially,
this might be generalized to a class-based approach in the future if
more configuration options are required, but for now the simpler
solution serves all existing use cases.
Now properly handles the regular/task parameter split
DevPR, parent is #745 

Adds the `BOTORCH` preset for GPs. 

### Important information
* I think it's critical to actually assert that the preset exactly
recovers the BoTorch behavior, in the form of a test, for mainly two
reasons:
1. The construction involves quite a few things to be configured, i.e.
handling both singletask/multitask (the latter even requiring a new
custom gpytorch module), setting all sorts of priors correctly, etc.
Blindly believing that everything is correct and then just claiming
`this is the BoTorch behavior` seems like a bad idea. The test ensures
this explicitly.
2. It also as an automatic alert mechanism for all situations when
something is changed on the BoTorch side, informing us about breaking
changes that yield different behavior but are not fully documented
(which happened already several times).
* I've also invested quite some effort to test the new multitask mean
logic, i.e. that it not only recovers the BoTorch logic but that it also
fills *one* of the missing gaps that will ultimately make our transfer
learning model *truely scale-invariant* w.r.t. the different input
tasks. In particular, I made sure that the only missing piece is the
noise model stratification, which should be added in a follow-up PR and
is the analogous to the stratification over means shipped by this PR.
(This explains the changes to the streamlit script.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dev enhancement Expand / change existing functionality new feature New functionality refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants